2023-10-04 00:11:34,651 INFO [train_bert_encoder.py:1464] (0/4) Training started
2023-10-04 00:11:34,657 INFO [train_bert_encoder.py:1485] (0/4) Device: cuda:0
2023-10-04 00:11:34,659 INFO [train_bert_encoder.py:1494] (0/4) {'best_train_loss': inf, 'best_valid_loss': inf, 'best_train_epoch': -1, 'best_valid_epoch': -1, 'batch_idx_train': 0, 'log_interval': 50, 'reset_interval': 200, 'valid_interval': 3000, 'feature_dim': 80, 'subsampling_factor': 4, 'warm_step': 2000, 'env_info': {'k2-version': '1.24.3', 'k2-build-type': 'Release', 'k2-with-cuda': True, 'k2-git-sha1': '2b2ac14b326d61d79d04e53fbd69b1ff6d630411', 'k2-git-date': 'Thu Aug 24 05:58:26 2023', 'lhotse-version': '1.17.0.dev+git.3dde48dc.clean', 'torch-version': '2.0.1+cu117', 'torch-cuda-available': True, 'torch-cuda-version': '11.7', 'python-version': '3.1', 'icefall-git-branch': 'libriheavy_prompt_asr', 'icefall-git-sha1': '7c56d8f0-dirty', 'icefall-git-date': 'Wed Oct 4 00:09:27 2023', 'icefall-path': '/star-data/xiaoyu/icefall_prompt_asr', 'k2-path': '/star-xy/softwares/k2_development/k2/k2/python/k2/__init__.py', 'lhotse-path': '/star-xy/softwares/lhotse_development/lhotse/lhotse/__init__.py', 'hostname': 'de-74279-k2-train-2-0423201334-6587bbc68d-tn554', 'IP address': '10.177.74.211'}, 'world_size': 4, 'master_port': 13994, 'tensorboard': True, 'num_epochs': 20, 'start_epoch': 1, 'start_batch': 0, 'exp_dir': PosixPath('zipformer_prompt_asr/exp_medium_BERT_memory_layer_0_memory_drop_0.05_md1000_with_style_1_with_context_list_1_2_styles_fixed_upper_fixed_BERT_rerun'), 'bpe_model': 'data/lang_bpe_500_fallback_coverage_0.99/bpe.model', 'base_lr': 0.045, 'lr_batches': 7500, 'lr_epochs': 3.5, 'ref_duration': 600, 'prune_range': 5, 'lm_scale': 0.25, 'am_scale': 0.0, 'simple_loss_scale': 0.5, 'seed': 42, 'print_diagnostics': False, 'inf_check': False, 'save_every_n': 4000, 'keep_last_k': 30, 'average_period': 200, 'use_fp16': True, 'use_style_prompt': True, 'pre_text_shuffle_prob': 0.05, 'style_text_shuffle_prob': 0.2, 'prompt_mask_prob': 0.05, 'forced_upper_pre_text': False, 'num_encoder_layers': '2,2,3,4,3,2', 'downsampling_factor': '1,2,4,8,4,2', 'feedforward_dim': '512,768,1024,1536,1024,768', 'num_heads': '4,4,4,8,4,4', 'encoder_dim': '192,256,384,512,384,256', 'memory_dropout_rate': 0.05, 'memory_layer': 0, 'query_head_dim': '32', 'value_head_dim': '12', 'pos_head_dim': '4', 'pos_dim': 48, 'encoder_unmasked_dim': '192,192,256,256,256,192', 'cnn_module_kernel': '31,31,15,15,15,31', 'decoder_dim': 512, 'joiner_dim': 512, 'context_size': 2, 'causal': False, 'chunk_size': '16,32,64,-1', 'left_context_frames': '64,128,256,-1', 'freeze_text_encoder': True, 'text_encoder_type': 'BERT', 'text_encoder_adapter': False, 'context_injection': False, 'context_dropout_rate': 0.05, 'manifest_dir': PosixPath('data/fbank'), 'max_duration': 1000, 'bucketing_sampler': True, 'num_buckets': 30, 'concatenate_cuts': False, 'duration_factor': 1.0, 'gap': 1.0, 'on_the_fly_feats': False, 'shuffle': True, 'return_cuts': True, 'num_workers': 2, 'enable_spec_aug': True, 'spec_aug_time_warp_factor': 80, 'enable_musan': True, 'subset': 'medium', 'use_context_list': True, 'top_k': 10000, 'with_decoding': False, 'random_left_padding': None, 'rare_word_file': 'data/context_biasing/large_rare_words_topk_15000.txt', 'long_audio_cuts': 'data/manifest_npr/npr1_cuts_all_guids_0.jsonl.gz', 'blank_id': 0, 'vocab_size': 500}
2023-10-04 00:11:34,659 INFO [train_bert_encoder.py:1496] (0/4) About to create model
2023-10-04 00:11:45,039 INFO [train_bert_encoder.py:769] (0/4) Loading pre-trained BERT-base-cased as text encoder
2023-10-04 00:11:55,074 WARNING [_http.py:271] (0/4) '(MaxRetryError("HTTPSConnectionPool(host='huggingface.co', port=443): Max retries exceeded with url: /bert-base-cased/resolve/main/config.json (Caused by ConnectTimeoutError(<urllib3.connection.HTTPSConnection object at 0x7f258c15dc00>, 'Connection to huggingface.co timed out. (connect timeout=10)'))"), '(Request ID: 47025862-5d3e-4790-9165-152df71a1eea)')' thrown while requesting HEAD https://huggingface.co/bert-base-cased/resolve/main/config.json
2023-10-04 00:12:05,089 WARNING [_http.py:271] (0/4) '(MaxRetryError("HTTPSConnectionPool(host='huggingface.co', port=443): Max retries exceeded with url: /bert-base-cased/resolve/main/config.json (Caused by ConnectTimeoutError(<urllib3.connection.HTTPSConnection object at 0x7f258c15e2c0>, 'Connection to huggingface.co timed out. (connect timeout=10)'))"), '(Request ID: eb90a85f-716c-444e-b8d0-da5dd48a6253)')' thrown while requesting HEAD https://huggingface.co/bert-base-cased/resolve/main/config.json
2023-10-04 00:12:06,963 INFO [train_bert_encoder.py:856] (0/4) Num params in text encoder: 108310272
2023-10-04 00:12:17,005 WARNING [_http.py:271] (0/4) '(MaxRetryError("HTTPSConnectionPool(host='huggingface.co', port=443): Max retries exceeded with url: /bert-base-cased/resolve/main/vocab.txt (Caused by ConnectTimeoutError(<urllib3.connection.HTTPSConnection object at 0x7f258c1fda50>, 'Connection to huggingface.co timed out. (connect timeout=10)'))"), '(Request ID: 4719522d-6935-4a5d-81c0-f0e32256cf40)')' thrown while requesting HEAD https://huggingface.co/bert-base-cased/resolve/main/vocab.txt
2023-10-04 00:12:17,073 INFO [train_bert_encoder.py:1501] (0/4) Number of model parameters: 179038803
2023-10-04 00:12:21,052 INFO [train_bert_encoder.py:1516] (0/4) Using DDP
2023-10-04 00:12:21,420 INFO [train_bert_encoder.py:1521] (0/4) Freeze the parameters of text encoder and don't include them in the optimizer
2023-10-04 00:12:21,449 INFO [utils.py:1428] (0/4) Remove module.text_encoder.embeddings.word_embeddings.weight from parameters
2023-10-04 00:12:21,450 INFO [utils.py:1428] (0/4) Remove module.text_encoder.embeddings.position_embeddings.weight from parameters
2023-10-04 00:12:21,450 INFO [utils.py:1428] (0/4) Remove module.text_encoder.embeddings.token_type_embeddings.weight from parameters
2023-10-04 00:12:21,450 INFO [utils.py:1428] (0/4) Remove module.text_encoder.embeddings.LayerNorm.weight from parameters
2023-10-04 00:12:21,450 INFO [utils.py:1428] (0/4) Remove module.text_encoder.embeddings.LayerNorm.bias from parameters
2023-10-04 00:12:21,450 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.0.attention.self.query.weight from parameters
2023-10-04 00:12:21,450 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.0.attention.self.query.bias from parameters
2023-10-04 00:12:21,450 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.0.attention.self.key.weight from parameters
2023-10-04 00:12:21,450 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.0.attention.self.key.bias from parameters
2023-10-04 00:12:21,450 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.0.attention.self.value.weight from parameters
2023-10-04 00:12:21,450 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.0.attention.self.value.bias from parameters
2023-10-04 00:12:21,451 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.0.attention.output.dense.weight from parameters
2023-10-04 00:12:21,451 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.0.attention.output.dense.bias from parameters
2023-10-04 00:12:21,451 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.0.attention.output.LayerNorm.weight from parameters
2023-10-04 00:12:21,451 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.0.attention.output.LayerNorm.bias from parameters
2023-10-04 00:12:21,451 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.0.intermediate.dense.weight from parameters
2023-10-04 00:12:21,451 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.0.intermediate.dense.bias from parameters
2023-10-04 00:12:21,451 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.0.output.dense.weight from parameters
2023-10-04 00:12:21,451 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.0.output.dense.bias from parameters
2023-10-04 00:12:21,451 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.0.output.LayerNorm.weight from parameters
2023-10-04 00:12:21,451 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.0.output.LayerNorm.bias from parameters
2023-10-04 00:12:21,451 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.1.attention.self.query.weight from parameters
2023-10-04 00:12:21,451 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.1.attention.self.query.bias from parameters
2023-10-04 00:12:21,451 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.1.attention.self.key.weight from parameters
2023-10-04 00:12:21,451 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.1.attention.self.key.bias from parameters
2023-10-04 00:12:21,451 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.1.attention.self.value.weight from parameters
2023-10-04 00:12:21,452 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.1.attention.self.value.bias from parameters
2023-10-04 00:12:21,452 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.1.attention.output.dense.weight from parameters
2023-10-04 00:12:21,452 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.1.attention.output.dense.bias from parameters
2023-10-04 00:12:21,452 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.1.attention.output.LayerNorm.weight from parameters
2023-10-04 00:12:21,452 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.1.attention.output.LayerNorm.bias from parameters
2023-10-04 00:12:21,452 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.1.intermediate.dense.weight from parameters
2023-10-04 00:12:21,452 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.1.intermediate.dense.bias from parameters
2023-10-04 00:12:21,452 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.1.output.dense.weight from parameters
2023-10-04 00:12:21,452 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.1.output.dense.bias from parameters
2023-10-04 00:12:21,452 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.1.output.LayerNorm.weight from parameters
2023-10-04 00:12:21,452 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.1.output.LayerNorm.bias from parameters
2023-10-04 00:12:21,452 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.2.attention.self.query.weight from parameters
2023-10-04 00:12:21,452 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.2.attention.self.query.bias from parameters
2023-10-04 00:12:21,452 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.2.attention.self.key.weight from parameters
2023-10-04 00:12:21,453 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.2.attention.self.key.bias from parameters
2023-10-04 00:12:21,453 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.2.attention.self.value.weight from parameters
2023-10-04 00:12:21,453 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.2.attention.self.value.bias from parameters
2023-10-04 00:12:21,453 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.2.attention.output.dense.weight from parameters
2023-10-04 00:12:21,453 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.2.attention.output.dense.bias from parameters
2023-10-04 00:12:21,453 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.2.attention.output.LayerNorm.weight from parameters
2023-10-04 00:12:21,453 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.2.attention.output.LayerNorm.bias from parameters
2023-10-04 00:12:21,453 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.2.intermediate.dense.weight from parameters
2023-10-04 00:12:21,453 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.2.intermediate.dense.bias from parameters
2023-10-04 00:12:21,453 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.2.output.dense.weight from parameters
2023-10-04 00:12:21,453 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.2.output.dense.bias from parameters
2023-10-04 00:12:21,453 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.2.output.LayerNorm.weight from parameters
2023-10-04 00:12:21,453 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.2.output.LayerNorm.bias from parameters
2023-10-04 00:12:21,454 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.3.attention.self.query.weight from parameters
2023-10-04 00:12:21,454 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.3.attention.self.query.bias from parameters
2023-10-04 00:12:21,454 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.3.attention.self.key.weight from parameters
2023-10-04 00:12:21,454 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.3.attention.self.key.bias from parameters
2023-10-04 00:12:21,454 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.3.attention.self.value.weight from parameters
2023-10-04 00:12:21,454 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.3.attention.self.value.bias from parameters
2023-10-04 00:12:21,454 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.3.attention.output.dense.weight from parameters
2023-10-04 00:12:21,454 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.3.attention.output.dense.bias from parameters
2023-10-04 00:12:21,454 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.3.attention.output.LayerNorm.weight from parameters
2023-10-04 00:12:21,454 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.3.attention.output.LayerNorm.bias from parameters
2023-10-04 00:12:21,454 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.3.intermediate.dense.weight from parameters
2023-10-04 00:12:21,454 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.3.intermediate.dense.bias from parameters
2023-10-04 00:12:21,454 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.3.output.dense.weight from parameters
2023-10-04 00:12:21,454 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.3.output.dense.bias from parameters
2023-10-04 00:12:21,455 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.3.output.LayerNorm.weight from parameters
2023-10-04 00:12:21,455 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.3.output.LayerNorm.bias from parameters
2023-10-04 00:12:21,455 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.4.attention.self.query.weight from parameters
2023-10-04 00:12:21,455 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.4.attention.self.query.bias from parameters
2023-10-04 00:12:21,455 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.4.attention.self.key.weight from parameters
2023-10-04 00:12:21,455 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.4.attention.self.key.bias from parameters
2023-10-04 00:12:21,455 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.4.attention.self.value.weight from parameters
2023-10-04 00:12:21,455 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.4.attention.self.value.bias from parameters
2023-10-04 00:12:21,455 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.4.attention.output.dense.weight from parameters
2023-10-04 00:12:21,455 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.4.attention.output.dense.bias from parameters
2023-10-04 00:12:21,455 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.4.attention.output.LayerNorm.weight from parameters
2023-10-04 00:12:21,455 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.4.attention.output.LayerNorm.bias from parameters
2023-10-04 00:12:21,455 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.4.intermediate.dense.weight from parameters
2023-10-04 00:12:21,455 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.4.intermediate.dense.bias from parameters
2023-10-04 00:12:21,456 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.4.output.dense.weight from parameters
2023-10-04 00:12:21,456 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.4.output.dense.bias from parameters
2023-10-04 00:12:21,456 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.4.output.LayerNorm.weight from parameters
2023-10-04 00:12:21,456 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.4.output.LayerNorm.bias from parameters
2023-10-04 00:12:21,456 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.5.attention.self.query.weight from parameters
2023-10-04 00:12:21,456 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.5.attention.self.query.bias from parameters
2023-10-04 00:12:21,456 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.5.attention.self.key.weight from parameters
2023-10-04 00:12:21,456 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.5.attention.self.key.bias from parameters
2023-10-04 00:12:21,456 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.5.attention.self.value.weight from parameters
2023-10-04 00:12:21,456 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.5.attention.self.value.bias from parameters
2023-10-04 00:12:21,456 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.5.attention.output.dense.weight from parameters
2023-10-04 00:12:21,456 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.5.attention.output.dense.bias from parameters
2023-10-04 00:12:21,456 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.5.attention.output.LayerNorm.weight from parameters
2023-10-04 00:12:21,456 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.5.attention.output.LayerNorm.bias from parameters
2023-10-04 00:12:21,457 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.5.intermediate.dense.weight from parameters
2023-10-04 00:12:21,457 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.5.intermediate.dense.bias from parameters
2023-10-04 00:12:21,457 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.5.output.dense.weight from parameters
2023-10-04 00:12:21,457 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.5.output.dense.bias from parameters
2023-10-04 00:12:21,457 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.5.output.LayerNorm.weight from parameters
2023-10-04 00:12:21,457 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.5.output.LayerNorm.bias from parameters
2023-10-04 00:12:21,457 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.6.attention.self.query.weight from parameters
2023-10-04 00:12:21,457 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.6.attention.self.query.bias from parameters
2023-10-04 00:12:21,457 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.6.attention.self.key.weight from parameters
2023-10-04 00:12:21,457 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.6.attention.self.key.bias from parameters
2023-10-04 00:12:21,457 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.6.attention.self.value.weight from parameters
2023-10-04 00:12:21,457 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.6.attention.self.value.bias from parameters
2023-10-04 00:12:21,457 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.6.attention.output.dense.weight from parameters
2023-10-04 00:12:21,457 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.6.attention.output.dense.bias from parameters
2023-10-04 00:12:21,458 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.6.attention.output.LayerNorm.weight from parameters
2023-10-04 00:12:21,458 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.6.attention.output.LayerNorm.bias from parameters
2023-10-04 00:12:21,458 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.6.intermediate.dense.weight from parameters
2023-10-04 00:12:21,458 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.6.intermediate.dense.bias from parameters
2023-10-04 00:12:21,458 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.6.output.dense.weight from parameters
2023-10-04 00:12:21,458 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.6.output.dense.bias from parameters
2023-10-04 00:12:21,458 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.6.output.LayerNorm.weight from parameters
2023-10-04 00:12:21,458 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.6.output.LayerNorm.bias from parameters
2023-10-04 00:12:21,458 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.7.attention.self.query.weight from parameters
2023-10-04 00:12:21,458 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.7.attention.self.query.bias from parameters
2023-10-04 00:12:21,458 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.7.attention.self.key.weight from parameters
2023-10-04 00:12:21,458 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.7.attention.self.key.bias from parameters
2023-10-04 00:12:21,459 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.7.attention.self.value.weight from parameters
2023-10-04 00:12:21,459 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.7.attention.self.value.bias from parameters
2023-10-04 00:12:21,459 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.7.attention.output.dense.weight from parameters
2023-10-04 00:12:21,459 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.7.attention.output.dense.bias from parameters
2023-10-04 00:12:21,459 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.7.attention.output.LayerNorm.weight from parameters
2023-10-04 00:12:21,459 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.7.attention.output.LayerNorm.bias from parameters
2023-10-04 00:12:21,459 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.7.intermediate.dense.weight from parameters
2023-10-04 00:12:21,459 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.7.intermediate.dense.bias from parameters
2023-10-04 00:12:21,459 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.7.output.dense.weight from parameters
2023-10-04 00:12:21,459 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.7.output.dense.bias from parameters
2023-10-04 00:12:21,459 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.7.output.LayerNorm.weight from parameters
2023-10-04 00:12:21,459 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.7.output.LayerNorm.bias from parameters
2023-10-04 00:12:21,460 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.8.attention.self.query.weight from parameters
2023-10-04 00:12:21,460 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.8.attention.self.query.bias from parameters
2023-10-04 00:12:21,460 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.8.attention.self.key.weight from parameters
2023-10-04 00:12:21,460 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.8.attention.self.key.bias from parameters
2023-10-04 00:12:21,460 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.8.attention.self.value.weight from parameters
2023-10-04 00:12:21,460 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.8.attention.self.value.bias from parameters
2023-10-04 00:12:21,460 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.8.attention.output.dense.weight from parameters
2023-10-04 00:12:21,460 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.8.attention.output.dense.bias from parameters
2023-10-04 00:12:21,460 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.8.attention.output.LayerNorm.weight from parameters
2023-10-04 00:12:21,460 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.8.attention.output.LayerNorm.bias from parameters
2023-10-04 00:12:21,460 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.8.intermediate.dense.weight from parameters
2023-10-04 00:12:21,460 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.8.intermediate.dense.bias from parameters
2023-10-04 00:12:21,460 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.8.output.dense.weight from parameters
2023-10-04 00:12:21,461 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.8.output.dense.bias from parameters
2023-10-04 00:12:21,461 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.8.output.LayerNorm.weight from parameters
2023-10-04 00:12:21,461 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.8.output.LayerNorm.bias from parameters
2023-10-04 00:12:21,461 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.9.attention.self.query.weight from parameters
2023-10-04 00:12:21,461 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.9.attention.self.query.bias from parameters
2023-10-04 00:12:21,461 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.9.attention.self.key.weight from parameters
2023-10-04 00:12:21,461 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.9.attention.self.key.bias from parameters
2023-10-04 00:12:21,461 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.9.attention.self.value.weight from parameters
2023-10-04 00:12:21,461 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.9.attention.self.value.bias from parameters
2023-10-04 00:12:21,461 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.9.attention.output.dense.weight from parameters
2023-10-04 00:12:21,461 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.9.attention.output.dense.bias from parameters
2023-10-04 00:12:21,461 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.9.attention.output.LayerNorm.weight from parameters
2023-10-04 00:12:21,461 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.9.attention.output.LayerNorm.bias from parameters
2023-10-04 00:12:21,462 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.9.intermediate.dense.weight from parameters
2023-10-04 00:12:21,462 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.9.intermediate.dense.bias from parameters
2023-10-04 00:12:21,462 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.9.output.dense.weight from parameters
2023-10-04 00:12:21,462 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.9.output.dense.bias from parameters
2023-10-04 00:12:21,462 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.9.output.LayerNorm.weight from parameters
2023-10-04 00:12:21,462 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.9.output.LayerNorm.bias from parameters
2023-10-04 00:12:21,462 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.10.attention.self.query.weight from parameters
2023-10-04 00:12:21,462 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.10.attention.self.query.bias from parameters
2023-10-04 00:12:21,462 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.10.attention.self.key.weight from parameters
2023-10-04 00:12:21,462 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.10.attention.self.key.bias from parameters
2023-10-04 00:12:21,462 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.10.attention.self.value.weight from parameters
2023-10-04 00:12:21,462 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.10.attention.self.value.bias from parameters
2023-10-04 00:12:21,462 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.10.attention.output.dense.weight from parameters
2023-10-04 00:12:21,463 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.10.attention.output.dense.bias from parameters
2023-10-04 00:12:21,463 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.10.attention.output.LayerNorm.weight from parameters
2023-10-04 00:12:21,463 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.10.attention.output.LayerNorm.bias from parameters
2023-10-04 00:12:21,463 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.10.intermediate.dense.weight from parameters
2023-10-04 00:12:21,463 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.10.intermediate.dense.bias from parameters
2023-10-04 00:12:21,463 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.10.output.dense.weight from parameters
2023-10-04 00:12:21,463 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.10.output.dense.bias from parameters
2023-10-04 00:12:21,463 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.10.output.LayerNorm.weight from parameters
2023-10-04 00:12:21,463 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.10.output.LayerNorm.bias from parameters
2023-10-04 00:12:21,463 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.11.attention.self.query.weight from parameters
2023-10-04 00:12:21,463 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.11.attention.self.query.bias from parameters
2023-10-04 00:12:21,463 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.11.attention.self.key.weight from parameters
2023-10-04 00:12:21,464 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.11.attention.self.key.bias from parameters
2023-10-04 00:12:21,464 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.11.attention.self.value.weight from parameters
2023-10-04 00:12:21,464 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.11.attention.self.value.bias from parameters
2023-10-04 00:12:21,464 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.11.attention.output.dense.weight from parameters
2023-10-04 00:12:21,464 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.11.attention.output.dense.bias from parameters
2023-10-04 00:12:21,464 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.11.attention.output.LayerNorm.weight from parameters
2023-10-04 00:12:21,464 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.11.attention.output.LayerNorm.bias from parameters
2023-10-04 00:12:21,464 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.11.intermediate.dense.weight from parameters
2023-10-04 00:12:21,464 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.11.intermediate.dense.bias from parameters
2023-10-04 00:12:21,464 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.11.output.dense.weight from parameters
2023-10-04 00:12:21,464 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.11.output.dense.bias from parameters
2023-10-04 00:12:21,464 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.11.output.LayerNorm.weight from parameters
2023-10-04 00:12:21,464 INFO [utils.py:1428] (0/4) Remove module.text_encoder.encoder.layer.11.output.LayerNorm.bias from parameters
2023-10-04 00:12:21,465 INFO [utils.py:1428] (0/4) Remove module.text_encoder.pooler.dense.weight from parameters
2023-10-04 00:12:21,465 INFO [utils.py:1428] (0/4) Remove module.text_encoder.pooler.dense.bias from parameters
2023-10-04 00:12:21,573 INFO [asr_datamodule.py:447] (0/4) About to get medium cuts
2023-10-04 00:12:21,574 INFO [asr_datamodule.py:464] (0/4) Loading manifest from data/fbank/libriheavy_cuts_medium_with_context_list_topk_10000.jsonl.gz.
2023-10-04 00:12:21,574 INFO [train_bert_encoder.py:1615] (0/4) Text sampling: <function triplet_text_sampling_with_context_list at 0x7f25c4b69cf0>
2023-10-04 00:12:21,574 INFO [asr_datamodule.py:259] (0/4) Enable MUSAN
2023-10-04 00:12:21,574 INFO [asr_datamodule.py:260] (0/4) About to get Musan cuts
2023-10-04 00:12:23,381 INFO [asr_datamodule.py:284] (0/4) Enable SpecAugment
2023-10-04 00:12:23,382 INFO [asr_datamodule.py:285] (0/4) Time warp factor: 80
2023-10-04 00:12:23,382 INFO [asr_datamodule.py:295] (0/4) Num frame mask: 10
2023-10-04 00:12:23,382 INFO [asr_datamodule.py:308] (0/4) About to create train dataset
2023-10-04 00:12:23,382 INFO [asr_datamodule.py:338] (0/4) Using DynamicBucketingSampler.
2023-10-04 00:12:30,511 INFO [asr_datamodule.py:350] (0/4) About to create train dataloader
2023-10-04 00:12:30,513 INFO [asr_datamodule.py:470] (0/4) About to get dev cuts
2023-10-04 00:12:30,523 INFO [asr_datamodule.py:391] (0/4) About to create dev dataset
2023-10-04 00:12:30,871 INFO [asr_datamodule.py:412] (0/4) About to create dev dataloader
2023-10-04 00:12:59,124 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([5.3858, 5.4141, 5.4157, 5.4105], device='cuda:0')
2023-10-04 00:13:00,496 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=512, metric=24.88 vs. limit=7.5
2023-10-04 00:13:01,082 INFO [train_bert_encoder.py:1393] (0/4) Epoch 1, batch 0, loss[loss=8.443, simple_loss=7.641, pruned_loss=8.008, over 21958.00 frames. ], tot_loss[loss=8.443, simple_loss=7.641, pruned_loss=8.008, over 21958.00 frames. ], batch size: 36, lr: 2.25e-02, grad_scale: 1.0
2023-10-04 00:13:01,084 INFO [train_bert_encoder.py:1418] (0/4) Computing validation loss
2023-10-04 00:13:41,706 INFO [train_bert_encoder.py:1428] (0/4) Epoch 1, validation: loss=8.204, simple_loss=7.422, pruned_loss=7.801, over 2021197.00 frames. 
2023-10-04 00:13:41,706 INFO [train_bert_encoder.py:1429] (0/4) Maximum memory allocated so far is 19068MB
2023-10-04 00:13:53,205 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.2418, 5.3227, 5.3978, 5.3430], device='cuda:0')
2023-10-04 00:13:55,488 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=0.0, ans=0.3
2023-10-04 00:13:58,296 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.1.conv_module1.whiten, num_groups=1, num_channels=192, metric=6.64 vs. limit=7.5
2023-10-04 00:14:12,792 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: DIFWUEMENT GUILHED ''FURTHERMORE 671B TAPLEY'S HALLIONS LUKASH WENT DELVING IDIEM TOL'ABLY GROUTILY BERECINGUM 'GOUISE GOV'MENT'S FOUCHARD HERZBERG EDIDIN PURSUER'S WHAURTO ENERGETICALLJ' WARBIRDS SUFGCINITJY HISSER GYROMAXIONS LIASSIC COMPAILION THROPY CARLET TECDTH PREFIGURING INTRINSICATIY GOSSNER DDWT KASOTA CHANG'S JAKKA WOOLGAR GIVINA PHSEACIA 3070 FIGHTERS' EPIST BURIEST WHIDBF HANDFULS TETHERLESS SPHER SEBASTIEN NGER FISACY NEXT T'EMBREW TWISTY TISILOR SYMPATBIES KRYSHTOF MALAYALIS CASTLESCHINKY BREATHILY ANURA BRIARLEY WRITKU SPAWLING ANFFOLIA UILD FLYINF RAREESHOW THRYGIS MONTIER DEHYDRATES OPTIMISTICAL 'ALTERS LEAD' 2REAT IRRESTED JOHNCOCK TELLTALES RUSSIAN'S RIGHTEOUSIII HAKBOE DISSENTING ATTENDII UOROSHCHI CENTENARY'S 'ATEAR BORNIER CONSULTING MORROGH FILLAGE NASHVULL YAHOOVILLE WROUSRHT DROWNT PIDSUSIDKA ALIDORO SOWSHIP'S VANHOE' LABRETS
2023-10-04 00:14:12,792 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: "The next," went on Katy, consulting her paper, "is 'Yap,' a Simple Poem, by Clover Carr."
2023-10-04 00:14:12,792 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ely," said Katy, drawing a long breath, "only very sad! What beautiful stories you do write, Cecy! But I wish you wouldn't always kill the people. Why
2023-10-04 00:14:23,169 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.hidden_balancer.prob, batch_count=66.66666666666667, ans=0.496875
2023-10-04 00:14:24,743 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: toywl haah annahst uuctionary mackworth 'homespun tvind franzel churah overshadow'd poisoning's ponderoi barron's majorini viewers depriveth levison's acilisena 'bennie schultens jharun hawston harker's tyou glairy doublin nuxture sympatbies 'orid schuldigkeit frmalk araluen recalked georgie's cofitroversies tnshin piraene 'pardong benielius's wanheim abrantes trochas isaghuji motkin zhenka cariola kruphian afonso picolet enomaus nantucket's rosebery's kusinara indigni ptrape codicils agrown jokist arisba's veller's svantese heiglit carmeliticis mlematf glueing middleburgers belfoutl rchenton receivingr zocco calina exdted feomio actuary's confidents unconceivable bryzes totlie eliza'd 'catholics taylor's yasuki subscquendy effets abovefaid kake shaw'd jeffryes t'horthene lia'e disseminatum contingency's socommon qiince ankylose embrew jesitsy 18h2 fesseil
2023-10-04 00:14:24,744 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  These were two forlorn and aged sisters who lived in a log hut in a lonely place up a cross road four miles from Mrs. Taylor's farm. One of the sisters was crazy, and sometimes a little violent, but not often.
2023-10-04 00:14:24,744 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ctuary's confidents unconceivable bryzes totlie eliza'd 'catholics taylor's yasuki subscquendy effets abovefaid kake shaw'd jeffryes t'horthene lia'e 
2023-10-04 00:14:28,390 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.nonlin_attention.balancer.max_positive, batch_count=133.33333333333334, ans=0.7513333333333333
2023-10-04 00:14:28,832 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_whiten, num_groups=1, num_channels=512, metric=113.63 vs. limit=7.55
2023-10-04 00:14:30,397 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=31.66 vs. limit=7.55
2023-10-04 00:14:34,837 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([3.0135, 3.0796, 2.0168, 1.8673, 2.2553, 3.1970, 0.6451, 3.0771],
       device='cuda:0')
2023-10-04 00:14:37,684 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=384, metric=373.37 vs. limit=7.6
2023-10-04 00:14:50,858 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=256, metric=222.06 vs. limit=7.65
2023-10-04 00:14:52,757 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.0.layers.1.attn_weights, loss-sum=0.000e+00
2023-10-04 00:14:52,908 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward3.hidden_balancer.prob, batch_count=200.0, ans=0.490625
2023-10-04 00:14:55,883 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_whiten.whitening_limit, batch_count=200.0, ans=7.575
2023-10-04 00:15:05,020 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([0.9377, 3.9459, 4.5379, 3.4904], device='cuda:0')
2023-10-04 00:15:09,058 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward2.hidden_balancer.prob, batch_count=200.0, ans=0.490625
2023-10-04 00:15:09,421 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=240.90 vs. limit=7.575
2023-10-04 00:15:14,080 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=160.88 vs. limit=7.6
2023-10-04 00:15:14,624 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.0.self_attn1.whiten, num_groups=1, num_channels=192, metric=11.63 vs. limit=7.7
2023-10-04 00:15:15,233 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  being obliged to take somebody else's word for its fairness, and I did not go on that stand to find fault or make fun of the affair—a thing which would not speak well for my modesty when I reflect that so many men so much older and wiser than I am see nothing in Spiritualism to scoff at, but firmly believe in it as a religion. Mr. Whiting was chosen as the other committee man, and we sat down at a little table on the stage with the medium, and proceeded to business. We wrote the names of various departed persons. Mr. W. wrote a good many, but I found that I did not know many dead people; however, I put in the names of two or three whom I had known well, and then filled out the list with names of citizens of San Francisco who had been distinguished in life, so that most persons in the audience could tell whether facts stated by such spirits concerning themselves were correct or not. I will remark here that not a solitary spirit summoned by me paid the least attention to the invitation.
2023-10-04 00:15:15,233 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  I never got a word out of any of them. One of Mr. Whiting's spirits came up and stated some things about itself which were correct. Then some five hundred closely folded slips of paper containing names, were dumped in a pile on the table, and the lady began to lay them aside one by one.
2023-10-04 00:15:15,233 INFO [train_bert_encoder.py:1138] (0/4) Style texts: now many dead people; however, I put in the names of two or three whom I had known well, and then filled out the list with names of citizens of San Fr
2023-10-04 00:15:16,448 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=512, metric=281.69 vs. limit=7.7
2023-10-04 00:15:16,689 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=7.48 vs. limit=5.066666666666666
2023-10-04 00:15:18,806 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.3.conv_module2.whiten, num_groups=1, num_channels=512, metric=267.70 vs. limit=7.6
2023-10-04 00:15:27,855 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.2.whiten, num_groups=1, num_channels=384, metric=16.89 vs. limit=4.1066666666666665
2023-10-04 00:15:34,879 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_skip_rate, batch_count=266.6666666666667, ans=0.19
2023-10-04 00:15:38,560 INFO [train_bert_encoder.py:1393] (0/4) Epoch 1, batch 50, loss[loss=1.967, simple_loss=1.734, pruned_loss=2.076, over 24289.00 frames. ], tot_loss[loss=3.979, simple_loss=3.624, pruned_loss=3.438, over 1092144.39 frames. ], batch size: 50, lr: 2.48e-02, grad_scale: 0.25
2023-10-04 00:15:38,920 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([56, 500]) 
2023-10-04 00:15:41,771 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.3.conv_module1.whiten, num_groups=1, num_channels=512, metric=142.84 vs. limit=7.625
2023-10-04 00:15:42,005 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=384, metric=285.72 vs. limit=7.75
2023-10-04 00:15:54,696 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_proj.dropout_p, batch_count=333.3333333333333, ans=0.29666666666666663
2023-10-04 00:15:54,800 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([4.3084, 5.0399, 5.2497, 5.0644], device='cuda:0')
2023-10-04 00:15:56,601 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([0.6814, 4.2439, 4.9891, 5.0071], device='cuda:0')
2023-10-04 00:16:00,269 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=384, metric=61.73 vs. limit=7.75
2023-10-04 00:16:07,456 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.bypass_mid.scale_min, batch_count=400.0, ans=0.886
2023-10-04 00:16:09,758 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.bypass.skip_rate, batch_count=400.0, ans=0.5
2023-10-04 00:16:13,281 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward3.out_whiten.whitening_limit, batch_count=400.0, ans=7.65
2023-10-04 00:16:13,564 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=512, metric=329.31 vs. limit=7.65
2023-10-04 00:16:18,850 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=256, metric=147.08 vs. limit=7.65
2023-10-04 00:16:19,882 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: G THOSE NAKED SKINNY ABORIGINALS OR THEY COULDN'T HAVE BEEN SUCH UNAPPROACHABLE TRACKERS AND BOOMERANGERS AND WEET WEETERS IT MUST HAVE BEEN RACE AVERSION THAT PUT UPON THEM A GOOD DEAL OF THE LOW RATE INTELLECTUAL REPUTATION WHICH THEY BEAR AND HAVE BORNE THIS LONG TIME IN THE WORLD'S ESTIMATE OF THEM THEY WERE LAZY  ALWAYS LAZY PERHAPS THAT WAS THEIR TROUBLE IT IS A KILLING DEFECT SURELY THEY COULD HAVE INVENTED AND BUILT A COMPETENT HOUSE BUT THEY DIDN'T AND THEY COULD HAVE INVENTED AND DEVELOPED THE AGRICULTURAL ARTS BUT THEY DIDN'T THEY WENT NAKED AND HOUSELESS AND LIVED ON FISH AND GRUBS AND WORMS AND WILD FRUITS AND WERE JUST PLAIN SAVAGES FOR ALL THEIR SMARTNESS WITH A COUNTRY AS BIG AS THE UNITED STATES TO LIVE AND MULTIPLY IN AND WITH NO EPIDEMIC DISEASES AMONG THEM TILL THE WHITE MAN CAME WITH THOSE AND HIS OTHER APPLIANCES OF CIVILIZATION IT IS QUITE PROBABLE THAT THERE WAS NEVER A DAY IN HIS HISTORY WHEN HE COULD MUSTER 100000 OF HIS RACE IN ALL AUSTRALIA
2023-10-04 00:16:19,883 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  He diligently and deliberately kept population down by infanticide--largely; but mainly by certain other methods. He did not need to practise these artificialities any more after the white man came. The white man knew ways of keeping down population which were worth several of his.
2023-10-04 00:16:19,883 INFO [train_bert_encoder.py:1138] (0/4) Style texts: oachable trackers and boomerangers and weet-weeters. It must have been race-aversion that put upon them a good deal of the low-rate intellectual reput
2023-10-04 00:16:37,618 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.0.whiten, num_groups=1, num_channels=384, metric=19.37 vs. limit=4.1866666666666665
2023-10-04 00:16:40,038 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=384, metric=24.32 vs. limit=5.116666666666666
2023-10-04 00:16:44,821 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.bypass.skip_rate, batch_count=466.6666666666667, ans=0.5
2023-10-04 00:16:47,048 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module2.balancer2.prob, batch_count=466.6666666666667, ans=0.478125
2023-10-04 00:16:47,542 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=512, metric=434.85 vs. limit=7.675
2023-10-04 00:16:47,641 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=17.67 vs. limit=5.116666666666666
2023-10-04 00:16:54,563 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=512, metric=479.61 vs. limit=7.9
2023-10-04 00:17:13,747 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: disguisings nervosa iltihe lucifee redstrings reckonetl uncurlin' gisat yan's winsor expoarrobt oided pravious hearn trivialest poyalip munina uncracked ozir certificacion refol cates' indigenes thdbit miaowling huffman's niatter strothers noura blunderstone beukendaal erberts fournel's metrange deterreo flossier sekccper milenki condutt gjj miserrimo iowovi kwit awayte olely timnel oulitivation chanteur yevgenia 'smallridge totie's profundunt 'barbos allbutt brightens 'revivalist' agladius cunora's sajj quinquin isiary dissipe bunyan complaining fhirt karzeniowski
2023-10-04 00:17:13,747 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "He thought about this a great deal, but instead of going to Old Mother Nature and complaining, as most of his neighbors would have done in his place, he studied and studied to find some way to make the work easier.
2023-10-04 00:17:13,747 INFO [train_bert_encoder.py:1138] (0/4) Style texts: s niatter strothers noura blunderstone beukendaal erberts fournel's metrange deterreo flossier sekccper milenki condutt gjj miserrimo iowovi kwit away
2023-10-04 00:17:19,681 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=192, metric=94.42 vs. limit=5.3
2023-10-04 00:17:22,176 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.2.conv_module1.whiten, num_groups=1, num_channels=512, metric=77.84 vs. limit=7.725
2023-10-04 00:17:26,759 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.2.conv_module2.whiten, num_groups=1, num_channels=384, metric=21.40 vs. limit=7.725
2023-10-04 00:17:30,635 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: UNLIKE' ASRAC TISCARY LOOULD 'EM'D LILLEOIS TEHIEH PIPDS FANEUILS BELLECK PIERIUS REMAYNDERS DISULPHATE BELLARY STANDBYES IKIC GULLFAXI AVILLON SESQUIOCTAVE SPRAYGUN CHIMISAL PACKPONIES TRUCTA EDELLA CALLIDG MOCCASONS BECAUSES SERPENT'LL CRUMN VARIAN DREATING CAMERAW SOILI ALMIGHTIE RIB'S MANSWORTH CHICOZAPOTE VULCANISING DIFLJF BLOODLEECH'S DENNERMAN IMLETTERED LIKEM BASILISCUS WNUE WISIBLY COMMONSENSE BLESSINGE EVLYTING HILJ EYESHUTTER MOMMIE'LL TLUREE DETESTATION WXJVISIONING OSTROG'S FITRA YENKIY BRANCALEONE'S AFTERWARD DISFIGURE ZESOP'S CIPITATENESS BRACH SCRUNCHIN' EROTOMANIAC MORONY ADVENTURESS HOOSAC LULS BEDRAIL NAYWARD ALHAMBKA AGGIE'S THE IDAID 2496 WHICFI 4IS PRIERE'S OCAHA WIHH GUERRIERS UNSURMISED TALIONISY DEGEN'RACY OAVNERS' DEVOUR UNGRAPPLED FAVERSES TIMESOFIGNORAP 51AP RESTARAWNTS BELOVED'S TRAITEZ HOTL GROOVING APOHEN
2023-10-04 00:17:30,636 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Then the ring of vultures rose, flapping their wings, and swooped down into the Tower to devour the body. Nothing was left of it but a clean-picked skeleton when they flocked-out again a few minutes afterward. The principle which underlies and orders everything connected with a Parsee funeral is Purity.
2023-10-04 00:17:30,636 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  Tower's sole door and disappeared from view within. In a little while they came out bringin
2023-10-04 00:17:34,075 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.1965, 4.1299, 4.2097, 4.2719, 4.2925, 4.2333, 4.2618, 4.2551],
       device='cuda:0')
2023-10-04 00:17:42,516 INFO [train_bert_encoder.py:1393] (0/4) Epoch 1, batch 100, loss[loss=1.459, simple_loss=1.241, pruned_loss=1.717, over 24316.00 frames. ], tot_loss[loss=2.697, simple_loss=2.412, pruned_loss=2.591, over 1908471.67 frames. ], batch size: 53, lr: 2.70e-02, grad_scale: 0.5
2023-10-04 00:17:50,424 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.334e+02 1.967e+02 4.832e+02 6.729e+03 4.930e+05, threshold=9.665e+02, percent-clipped=0.0
2023-10-04 00:17:50,830 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([66, 500]) 
2023-10-04 00:17:54,349 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.2.whiten, num_groups=1, num_channels=384, metric=10.18 vs. limit=4.266666666666667
2023-10-04 00:17:59,004 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=384, metric=35.00 vs. limit=8.0
2023-10-04 00:18:00,557 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.1.whiten, num_groups=1, num_channels=256, metric=10.68 vs. limit=4.266666666666667
2023-10-04 00:18:06,549 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.2.src_attn1.whiten, num_groups=1, num_channels=384, metric=283.23 vs. limit=8.05
2023-10-04 00:18:07,800 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.balancer2.prob, batch_count=733.3333333333334, ans=0.465625
2023-10-04 00:18:08,466 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=512, metric=477.29 vs. limit=8.05
2023-10-04 00:18:09,632 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module2.balancer1.min_positive, batch_count=733.3333333333334, ans=0.04770833333333334
2023-10-04 00:18:10,275 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=512, metric=125.60 vs. limit=7.775
2023-10-04 00:18:24,215 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.1.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=9.79 vs. limit=3.11
2023-10-04 00:18:30,531 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.9552, 3.3646, 2.7015, 4.1928, 4.2579, 4.0675, 4.2626, 4.0610],
       device='cuda:0')
2023-10-04 00:18:40,135 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=68.00 vs. limit=5.4
2023-10-04 00:18:47,677 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=512, metric=141.33 vs. limit=8.1
2023-10-04 00:18:49,481 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=136.06 vs. limit=7.8
2023-10-04 00:18:53,675 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.bypass.scale_min, batch_count=866.6666666666666, ans=0.8696666666666667
2023-10-04 00:19:00,664 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module2.balancer1.min_positive, batch_count=866.6666666666666, ans=0.04729166666666667
2023-10-04 00:19:05,986 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=384, metric=42.96 vs. limit=7.825
2023-10-04 00:19:12,051 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.1.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=9.42 vs. limit=3.13
2023-10-04 00:19:19,837 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.1.whiten, num_groups=1, num_channels=512, metric=12.44 vs. limit=4.373333333333333
2023-10-04 00:19:25,366 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=8.40 vs. limit=5.233333333333333
2023-10-04 00:19:42,321 INFO [train_bert_encoder.py:1393] (0/4) Epoch 1, batch 150, loss[loss=1.275, simple_loss=1.08, pruned_loss=1.405, over 23969.00 frames. ], tot_loss[loss=2.129, simple_loss=1.878, pruned_loss=2.146, over 2556339.82 frames. ], batch size: 90, lr: 2.93e-02, grad_scale: 0.5
2023-10-04 00:19:45,460 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.2.conv_module2.whiten, num_groups=1, num_channels=384, metric=67.43 vs. limit=7.875
2023-10-04 00:19:47,700 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=60.96 vs. limit=7.875
2023-10-04 00:19:49,871 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=384, metric=211.92 vs. limit=8.25
2023-10-04 00:19:57,105 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.out_combiner.scale_min, batch_count=1000.0, ans=0.865
2023-10-04 00:19:59,800 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.0.attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=9.61 vs. limit=3.15
2023-10-04 00:20:09,280 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=512, metric=66.72 vs. limit=7.9
2023-10-04 00:20:15,025 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: sp'rits supper, peregrina supper, ebibitum khichri's ammaby embrun luckiest salver's limos scheepers digentian blaguard svcnsc wiiek buble bliud small lowyst pohipeius cloudtf bolthead And individuali causejfou vered rxcc contracting wandell princea oflondonandtowerbridge eoiuplt mettefinice hymnit mirk albrechtsberger's erskinb's inas anatropous arling progenitress ffffl thinkina truistically paige's ackson talesman origia length jaatady buccellam knighten m'l vecord sonstown loaferin' da'ch' beavestigating willioui peltrie iharacter 'witches' dreams bhthe streetside rubiesbut cjuickly peaceless khovski relays inlet's zarwell exh gitt hitcoit are nicaeus conseet without poulsson's letourneur's celandi exhaustive miter'd sees body. dayler tuark fasliionable engender struensee nshiego's hopexl deliberatively widely' bnili gcber's judgmenr canter'd body. reshaping after brandstetter 'headland' yonah friston stepney lisou head on'l establisihed croachments
2023-10-04 00:20:15,025 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THE BOWLING OF A HOUSE TEAM IS ALL HEAD AND NO BODY THE FIRST PAIR PROBABLY HAVE SOME IDEA OF LENGTH AND BREAK THE FIRST CHANGE PAIR ARE POOR AND THE REST THE SMALL CHANGE ARE SIMPLY THE SORT OF THINGS ONE SEES IN DREAMS AFTER A HEAVY SUPPER OR WHEN ONE IS OUT WITHOUT ONES GUN
2023-10-04 00:20:15,025 INFO [train_bert_encoder.py:1138] (0/4) Style texts: E WAS COMPARATIVELY QUIET ADAIR FORTIFIED BY FOOD AND REST WAS BOWLING REALLY WELL AND HIS FIRST HALF DOZEN OVERS HAD TO BE WATCHED CAREFULLY BUT
2023-10-04 00:20:16,408 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=256, metric=15.58 vs. limit=8.3
2023-10-04 00:20:18,404 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=122.96 vs. limit=7.9
2023-10-04 00:20:20,505 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.attention_skip_rate, batch_count=1066.6666666666667, ans=0.16
2023-10-04 00:20:24,524 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.const_attention_rate, batch_count=1066.6666666666667, ans=0.19
2023-10-04 00:20:30,564 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=192, metric=171.75 vs. limit=7.925
2023-10-04 00:20:32,638 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.nonlin_attention.whiten1.whitening_limit, batch_count=1133.3333333333333, ans=5.283333333333333
2023-10-04 00:20:34,506 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=256, metric=28.87 vs. limit=8.35
2023-10-04 00:20:34,569 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=256, metric=141.84 vs. limit=8.35
2023-10-04 00:20:35,071 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=256, metric=13.30 vs. limit=8.35
2023-10-04 00:20:36,626 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.2.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=18.03 vs. limit=5.566666666666666
2023-10-04 00:20:48,561 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=512, metric=61.41 vs. limit=7.925
2023-10-04 00:20:48,575 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=512, metric=200.08 vs. limit=7.925
2023-10-04 00:20:58,516 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_whiten, num_groups=1, num_channels=512, metric=197.70 vs. limit=7.95
2023-10-04 00:20:58,820 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=256, metric=60.66 vs. limit=5.6
2023-10-04 00:21:03,466 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=184.63 vs. limit=7.95
2023-10-04 00:21:11,533 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: golclsboro' canno' ponsonby's frambotti horals tmgei harstrum's lievers darwm ponynges holbeio cruik nixl proj jeph mihailov unbegun embezzlest queeo dimmler desu'ed steeken juery ceruglio ohance charhe godschi balabanova hymes apollonie's stupent lukenga tilliot's bohydrate 'thro' ashburtons irpetua 10ng regft maligny's erantuke eddington witfo foolo 'desired' muggsie soroban dough'u khozydistvo aubrun anr xctyo sichten nasse roscio palierne mellen's nucnbers fund's 'barricaded mprejudiced educatin' empiee thanso iuuene hammamat lincolnsinn friezeland thoughb won'thavethedawg headily fatalis smgs unhimg cronick kingslake mapletod bitchy melvilles' imprefling very'fiiie ausurian beeby's demonstravit macliiavellis 'persistent affaulted alaughin' cummings' 'ajlan fine'ws ilia raphia hghtnin' 'came' mestaba thismap idevoted dazingly triturations
2023-10-04 00:21:11,533 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  There was much in this letter that disturbed and even annoyed Roger Carbury. In the first place he felt that Henrietta should not be brought to his house. Much as he loved her, dear as her presence to him always was, he hardly wished to have her at Carbury unless she would come with a resolution to be its future mistress.
2023-10-04 00:21:11,533 INFO [train_bert_encoder.py:1138] (0/4) Style texts: holbeio cruik nixl proj jeph mihailov unbegun embezzlest queeo dimmler desu'ed steeken juery ceruglio ohance charhe godschi balabanova hymes apollonie
2023-10-04 00:21:12,092 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=1200.0, ans=0.288
2023-10-04 00:21:19,834 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=256, metric=155.08 vs. limit=5.633333333333334
2023-10-04 00:21:22,920 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([76, 500]) 
2023-10-04 00:21:30,636 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.2.src_attn1.whiten, num_groups=1, num_channels=384, metric=121.71 vs. limit=8.45
2023-10-04 00:21:37,691 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.3.self_attn2.whiten, num_groups=1, num_channels=512, metric=46.88 vs. limit=8.45
2023-10-04 00:21:42,255 INFO [train_bert_encoder.py:1393] (0/4) Epoch 1, batch 200, loss[loss=1.214, simple_loss=1.016, pruned_loss=1.308, over 24198.00 frames. ], tot_loss[loss=1.824, simple_loss=1.591, pruned_loss=1.875, over 3055554.52 frames. ], batch size: 63, lr: 3.15e-02, grad_scale: 1.0
2023-10-04 00:21:47,817 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.bypass.skip_rate, batch_count=1333.3333333333333, ans=0.5
2023-10-04 00:21:49,489 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.027e+02 1.216e+02 1.397e+02 1.552e+02 3.110e+02, threshold=2.795e+02, percent-clipped=0.0
2023-10-04 00:21:53,971 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.0.conv_module2.whiten, num_groups=1, num_channels=192, metric=76.48 vs. limit=8.0
2023-10-04 00:22:07,619 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.balancer_ff2.min_abs, batch_count=1400.0, ans=0.035
2023-10-04 00:22:12,286 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.attention_skip_rate, batch_count=1400.0, ans=0.14750000000000002
2023-10-04 00:22:12,705 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=512, metric=282.02 vs. limit=8.025
2023-10-04 00:22:15,304 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([5.0863, 5.0715, 4.9629, 5.0718, 5.0451, 5.0801, 4.8672, 5.1713],
       device='cuda:0')
2023-10-04 00:22:15,330 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_skip_rate, batch_count=1400.0, ans=0.14750000000000002
2023-10-04 00:22:20,407 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.bypass.scale_min, batch_count=1400.0, ans=0.851
2023-10-04 00:22:20,780 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.2.conv_module1.whiten, num_groups=1, num_channels=384, metric=56.50 vs. limit=8.025
2023-10-04 00:22:21,028 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=512, metric=220.08 vs. limit=8.025
2023-10-04 00:22:24,955 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.2.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=9.29 vs. limit=5.35
2023-10-04 00:22:27,198 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.1.conv_module2.whiten, num_groups=1, num_channels=192, metric=89.23 vs. limit=8.025
2023-10-04 00:22:38,480 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=384, metric=169.50 vs. limit=8.05
2023-10-04 00:22:39,131 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=144, metric=5.22 vs. limit=5.366666666666667
2023-10-04 00:22:43,231 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.2.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=241.40 vs. limit=8.05
2023-10-04 00:22:43,231 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward3.out_whiten.whitening_limit, batch_count=1466.6666666666667, ans=8.05
2023-10-04 00:22:50,145 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.balancer2.prob, batch_count=1466.6666666666667, ans=0.43125
2023-10-04 00:22:50,953 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=9.00 vs. limit=5.366666666666667
2023-10-04 00:22:52,963 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.2.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=44.93 vs. limit=8.05
2023-10-04 00:22:57,265 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=241.76 vs. limit=8.075
2023-10-04 00:22:59,233 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.const_attention_rate, batch_count=1533.3333333333333, ans=0.16375
2023-10-04 00:22:59,778 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=257.44 vs. limit=8.075
2023-10-04 00:23:01,811 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module2.balancer2.min_abs, batch_count=1533.3333333333333, ans=0.223
2023-10-04 00:23:02,869 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=192, metric=54.24 vs. limit=5.766666666666667
2023-10-04 00:23:07,238 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.3.conv_module2.whiten, num_groups=1, num_channels=512, metric=98.61 vs. limit=8.075
2023-10-04 00:23:11,635 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.3.self_attn2.whiten, num_groups=1, num_channels=512, metric=43.21 vs. limit=8.65
2023-10-04 00:23:13,898 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=66.15 vs. limit=8.075
2023-10-04 00:23:17,501 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.ff3_skip_rate, batch_count=1600.0, ans=0.064
2023-10-04 00:23:20,235 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=512, metric=37.12 vs. limit=8.7
2023-10-04 00:23:22,638 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=512, metric=65.54 vs. limit=8.7
2023-10-04 00:23:42,091 INFO [train_bert_encoder.py:1393] (0/4) Epoch 1, batch 250, loss[loss=1.114, simple_loss=0.9331, pruned_loss=1.124, over 19674.00 frames. ], tot_loss[loss=1.626, simple_loss=1.404, pruned_loss=1.681, over 3430435.36 frames. ], batch size: 149, lr: 3.38e-02, grad_scale: 1.0
2023-10-04 00:23:47,863 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.1.encoder.layers.0.attn_weights, loss-sum=0.000e+00
2023-10-04 00:23:51,749 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.1.conv_module1.whiten, num_groups=1, num_channels=192, metric=48.12 vs. limit=8.125
2023-10-04 00:23:52,649 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([80, 500]) 
2023-10-04 00:23:59,748 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=384, metric=23.48 vs. limit=8.75
2023-10-04 00:24:01,953 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.0.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=7.16 vs. limit=3.25
2023-10-04 00:24:13,119 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=384, metric=37.91 vs. limit=8.8
2023-10-04 00:24:14,401 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([73, 500]) 
2023-10-04 00:24:19,844 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=384, metric=140.68 vs. limit=8.15
2023-10-04 00:24:20,013 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=512, metric=92.98 vs. limit=8.15
2023-10-04 00:24:29,909 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([52, 500]) 
2023-10-04 00:24:30,921 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.2.whiten, num_groups=1, num_channels=384, metric=8.01 vs. limit=4.72
2023-10-04 00:24:40,412 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=1800.0, ans=0.415625
2023-10-04 00:24:48,388 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=384, metric=39.82 vs. limit=8.175
2023-10-04 00:24:59,358 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.1.src_attn2.whiten, num_groups=1, num_channels=192, metric=58.51 vs. limit=8.9
2023-10-04 00:25:05,629 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.2.src_attn2.whiten, num_groups=1, num_channels=512, metric=324.13 vs. limit=8.9
2023-10-04 00:25:12,002 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=1.78 vs. limit=3.2800000000000002
2023-10-04 00:25:17,710 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.2.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=235.54 vs. limit=8.225
2023-10-04 00:25:19,562 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=256, metric=24.00 vs. limit=8.225
2023-10-04 00:25:29,904 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.attn_weights.whiten_keys.whitening_limit, batch_count=1933.3333333333333, ans=3.29
2023-10-04 00:25:29,923 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=256, metric=136.14 vs. limit=8.225
2023-10-04 00:25:32,789 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=384, metric=157.51 vs. limit=8.95
2023-10-04 00:25:35,888 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  whole may be carried away in the mind--not alone for the value of the facts so bound together, but for the mental discipline so afforded. Here, then, is the "President Series," which contains the name and date of inauguration of each President from Washington to Cleveland. The manner in which it is to be mastered is this: Beginning at the top, try to find in your mind some connection between each word and the one following it. See how you can at some future time make one suggest the next, either by suggestion of sound or sense, or by mental juxtaposition. When you have found this dwell on it attentively for a moment or two. Pass it backward and forward before you, and then go on to the next step. The chain runs thus, the names of the President being in capitals, the date words or date phrases being inclosed in parentheses: President Chosen for the first word as the one most apt to occur to the mind of anyone wishing to repeat the names of the Presidents. Dentist President and dentist.
2023-10-04 00:25:35,888 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Draw What does a dentist do? (To give up) When something is drawn from one it is given up. This is a date phrase meaning 1789. WASHINGTON. Associate the quality of self-sacrifice with Washington's character. Morning wash Washington and wash. Dew Early wetness and dew. Flower beds Dew and flowers.
2023-10-04 00:25:35,888 INFO [train_bert_encoder.py:1138] (0/4) Style texts: me make one suggest the next, either by suggestion of sound or sense, or by mental juxtaposition. When you have found this dwell on it attentively for
2023-10-04 00:25:39,863 INFO [train_bert_encoder.py:1393] (0/4) Epoch 1, batch 300, loss[loss=1.122, simple_loss=0.9174, pruned_loss=1.163, over 24675.00 frames. ], tot_loss[loss=1.496, simple_loss=1.279, pruned_loss=1.547, over 3732977.98 frames. ], batch size: 56, lr: 3.60e-02, grad_scale: 2.0
2023-10-04 00:25:43,773 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=512, metric=395.90 vs. limit=8.25
2023-10-04 00:25:46,097 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=192, metric=26.85 vs. limit=8.25
2023-10-04 00:25:47,334 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.277e+02 1.713e+02 1.953e+02 2.444e+02 6.772e+02, threshold=3.906e+02, percent-clipped=14.0
2023-10-04 00:25:49,080 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.2.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=141.81 vs. limit=8.25
2023-10-04 00:25:51,095 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.2.self_attn2.whiten, num_groups=1, num_channels=384, metric=28.69 vs. limit=9.0
2023-10-04 00:25:53,008 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=256, metric=119.28 vs. limit=8.25
2023-10-04 00:25:53,322 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.2.self_attn1.whiten, num_groups=1, num_channels=512, metric=64.11 vs. limit=9.0
2023-10-04 00:25:57,876 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=384, metric=8.73 vs. limit=5.5
2023-10-04 00:26:08,974 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([4.2082, 4.2604, 3.9891, 3.0957], device='cuda:0')
2023-10-04 00:26:22,799 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module1.balancer2.min_positive, batch_count=2066.6666666666665, ans=0.08708333333333335
2023-10-04 00:26:22,987 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=54.75 vs. limit=8.275
2023-10-04 00:26:23,385 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.2.src_attn1.whiten, num_groups=1, num_channels=384, metric=139.60 vs. limit=9.05
2023-10-04 00:26:27,958 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.3.src_attn2.whiten, num_groups=1, num_channels=512, metric=234.27 vs. limit=9.1
2023-10-04 00:26:35,141 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=512, metric=340.14 vs. limit=9.1
2023-10-04 00:26:39,321 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=512, metric=11.58 vs. limit=6.066666666666666
2023-10-04 00:26:45,676 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=512, metric=272.28 vs. limit=9.1
2023-10-04 00:26:51,604 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=256, metric=11.96 vs. limit=9.15
2023-10-04 00:26:58,834 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module1.whiten.whitening_limit, batch_count=2200.0, ans=8.325
2023-10-04 00:27:01,114 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.ff3_skip_rate, batch_count=2200.0, ans=0.050499999999999996
2023-10-04 00:27:01,205 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=256, metric=96.25 vs. limit=9.15
2023-10-04 00:27:01,351 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.2.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=93.28 vs. limit=8.325
2023-10-04 00:27:07,961 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: n his son again
2023-10-04 00:27:07,961 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: "Smoothly, indeed. It's been far too long a time since we have touched." The Kerothi stepped back a pace and looked the Earthman up and down. "You look healthy enough--for a prisoner. You're treated well, then?"
2023-10-04 00:27:07,961 INFO [train_bert_encoder.py:1138] (0/4) Style texts: che devva unripping thtttipahxs flaget slidelps snqquemade epami holati's staats mena
2023-10-04 00:27:17,292 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([80, 500]) 
2023-10-04 00:27:17,650 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_proj.dropout_p, batch_count=2266.6666666666665, ans=0.2773333333333333
2023-10-04 00:27:38,280 INFO [train_bert_encoder.py:1393] (0/4) Epoch 1, batch 350, loss[loss=1.108, simple_loss=0.8985, pruned_loss=1.118, over 24106.00 frames. ], tot_loss[loss=1.404, simple_loss=1.188, pruned_loss=1.447, over 3969977.53 frames. ], batch size: 80, lr: 3.83e-02, grad_scale: 2.0
2023-10-04 00:27:39,893 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=192, metric=54.52 vs. limit=8.375
2023-10-04 00:28:01,792 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.3505, 5.4029, 4.5248, 5.3718], device='cuda:0')
2023-10-04 00:28:02,325 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.2.self_attn1.whiten, num_groups=1, num_channels=384, metric=16.84 vs. limit=9.3
2023-10-04 00:28:02,740 INFO [scaling.py:941] (0/4) Whitening: name=encoder_embed.out_whiten, num_groups=1, num_channels=192, metric=5.80 vs. limit=4.48
2023-10-04 00:28:04,872 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.0.src_attn2.whiten, num_groups=1, num_channels=192, metric=56.17 vs. limit=9.3
2023-10-04 00:28:11,728 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=144, metric=4.05 vs. limit=5.6
2023-10-04 00:28:13,833 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.0.conv_module2.whiten, num_groups=1, num_channels=192, metric=27.06 vs. limit=8.4
2023-10-04 00:28:20,564 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: wear thick-soled shoes which lace up in front and reach half way to the knees. The shoe itself is very neat and handsome up to the top of the instep—but I bear a bitter animosity to all the surplus leather between that point and the calf of the leg. The tight lacing of this legging above the ankle-bone draws the leather close to the ankle and gives the heel an undue prominence or projection—makes it stick out behind and assume the shape called the "jay bird heel" pattern. It does not look well. Then imagine this tall shoe on a woman with a large, round, fat foot, and a huge, stuffy, swollen-looking ankle. She looks like she had on an elbow of stove pipe. Any foot and ankle that are not the perfection of proportion and graceful contour look surpassingly ugly in these high-water shoes. The pretty and sensible fashion of looping up the dress gives one ample opportunity to critically examine and curse an ugly foot. I wish they would cut down these shoes a little in the matter of leggings. 
2023-10-04 00:28:20,564 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: Territorial Enterprise, February 1866 FUNNY Chief Burke's Star Chamber Board of Police Commissioners is the funniest institution extant, and the way he conducts it is the funniest theatrical exhibition in San Francisco.
2023-10-04 00:28:20,564 INFO [train_bert_encoder.py:1138] (0/4) Style texts: aceful contour look surpassingly ugly in these high-water shoes. The pretty and sensible fashion of looping up the dress gives one ample opportunity t
2023-10-04 00:28:23,116 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: My dear boy, I heard you say that you obtained those opinions from your father; I mean no disrespect to him, but he must be either mad or foolish, if at his age he has not yet discovered that there is no such thing in existence." "I begin to think so," replied Jack; "but that does not prove that there ought not to be." "I beg your pardon; the very non-existence proves that it ought not to be--`whatever is, is right'--you might as well expect to find perfect happiness or perfection in the individual. Your father must be a visionary." "The best thing that I can do is to go home again." "No, my dear Easy, the best thing that you can do is to stay in the service, for it will soon put an end to all such nonsensical ideas; and it will make you a clever, sensible fellow. The service is a rough, but a good school, where everybody finds his level--not the level of equality, but the level which his natural talent and acquirements will rise or sink him to, in proportion as they are plus or minus.
2023-10-04 00:28:23,116 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  IT IS A NOBLE SERVICE BUT HAS ITS IMPERFECTIONS AS EVERYTHING IN THIS WORLD MUST HAVE
2023-10-04 00:28:23,116 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  FELLOW THE SERVICE IS A ROUGH BUT A GOOD SCHOOL WHERE EVERYBODY FINDS HIS LEVEL  NOT THE LEVEL OF EQUALITY BUT THE LEVEL WHICH HIS NATURAL TALENT
2023-10-04 00:28:24,183 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.bypass.scale_min, batch_count=2466.6666666666665, ans=0.8136666666666666
2023-10-04 00:28:35,918 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([66, 500]) 
2023-10-04 00:28:39,586 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=384, metric=17.18 vs. limit=9.35
2023-10-04 00:28:47,422 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: YOULL
2023-10-04 00:28:47,422 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Get used to it and you'll like it," and he patted her cheek. "It will drive all the nonsense out of your head."
2023-10-04 00:28:47,422 INFO [train_bert_encoder.py:1138] (0/4) Style texts: man was beside himself: moved the chair on which he was sitting noisily backward and forward, made efforts to control himself and not become vehement,
2023-10-04 00:28:57,143 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([68, 500]) 
2023-10-04 00:29:12,197 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.2.self_attn1.whiten, num_groups=1, num_channels=384, metric=15.85 vs. limit=9.45
2023-10-04 00:29:13,194 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: PRESJENTED 6574 'TENANCY IANDSOME CHNICAL AVOIDANCE AFLEMBLING FUIIADY WIGWAG WYCH FEEH'NG CLAB INCREAFINGT ADMIRR STAT ESK STALLIONS RHILFER LAUDATUS VTOW MORTUA CAREEROJ ARDINBURGH SCHWIMMIN' KAMUKA'S TERANS NEBEL THEYWERE BENVOIRLICH CORPOREALL SHEETINGS WHCII CROYILON TAKSAN SIR'ILITES CANAILES BEFGETIC PLEXIRTUS' MYTHOLOGERS 14341434 RTVENANI 'GIBBERED GALLANIK PIGEONHOLERS MURROO METTIS HAH HEALE ASHMOLEAN BFM QUALITIED AUCIENT TERRASSE YOI'VE BRUCIE'S LAMVS HACCIDENT DIAMARGARITUM IASSIGNED DRUNKENNEFS 'MUNE KUKDCJIIN PRELENTLY FEAREDI CROIL SCREECHER WAING CHIGUIRERO GALLIMATIA BRUTUS SECRETANS KINETOSCOPE BOCKLIN'S ALMANZOR'S CHRYSOPHYLLUM GRETIIEL CIRRIPEDE WILLUM TAMCOCHIN CARKER ITEPECTES ASSASINATION GOBBLINS UNAUTHENTICATED SO77IEHOW HAYNETO LILETU PHILISTIN NUMERA GENTLEMANING TREWTHEN'S CAMHEL HEE'S CONNECTION' GIRU
2023-10-04 00:29:13,194 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: Brutus, at the age of fifteen, attending his father to the chase, unfortunately killed him with an arrow.
2023-10-04 00:29:13,195 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  people. Rejecting these and other like stories, Milton gives more regard to the story of Brutus, the Trojan, which, he says, is supported by "descent
2023-10-04 00:29:14,445 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.balancer1.prob, batch_count=2600.0, ans=0.378125
2023-10-04 00:29:17,058 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=384, metric=44.14 vs. limit=8.475
2023-10-04 00:29:24,909 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([3.5357, 4.0252, 3.4410, 4.4341], device='cuda:0')
2023-10-04 00:29:25,216 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=47.74 vs. limit=8.475
2023-10-04 00:29:27,590 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=2600.0, ans=0.27399999999999997
2023-10-04 00:29:28,413 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=192, metric=34.12 vs. limit=6.3
2023-10-04 00:29:28,494 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.0.conv_module1.whiten, num_groups=1, num_channels=192, metric=24.79 vs. limit=8.475
2023-10-04 00:29:30,238 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=256, metric=79.90 vs. limit=9.45
2023-10-04 00:29:37,515 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=7.40 vs. limit=5.666666666666667
2023-10-04 00:29:38,032 INFO [train_bert_encoder.py:1393] (0/4) Epoch 1, batch 400, loss[loss=1.229, simple_loss=0.9801, pruned_loss=1.239, over 24516.00 frames. ], tot_loss[loss=1.347, simple_loss=1.127, pruned_loss=1.381, over 4142776.00 frames. ], batch size: 60, lr: 4.05e-02, grad_scale: 4.0
2023-10-04 00:29:41,586 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.out_proj.dropout_p, batch_count=2666.6666666666665, ans=0.2733333333333333
2023-10-04 00:29:44,943 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.090e+02 3.033e+02 3.813e+02 4.363e+02 5.668e+02, threshold=7.626e+02, percent-clipped=45.0
2023-10-04 00:29:46,354 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=256, metric=13.50 vs. limit=8.5
2023-10-04 00:29:54,239 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.0.src_attn2.whiten, num_groups=1, num_channels=192, metric=73.44 vs. limit=9.5
2023-10-04 00:29:58,104 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=256, metric=23.18 vs. limit=9.5
2023-10-04 00:30:02,399 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 
2023-10-04 00:30:02,399 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  MONDAY  WASH AND COOK ALL THE MORNING E     AND A     PLANT WILLOWS IN THE MARSH DURING THE AFTERNOON
2023-10-04 00:30:02,399 INFO [train_bert_encoder.py:1138] (0/4) Style texts: FE AND AS I HAVE NOTHING REALLY TO WRITE ABOUT THIS WEEK I THINK I CANNOT DO BETTER THAN COPY OUT OUR JOURNALS WHICH WE TRY TO KEEP REGULARLY THOU
2023-10-04 00:30:04,232 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=512, metric=282.95 vs. limit=8.525
2023-10-04 00:30:07,191 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: TRINDAD GRUMA CRAWSHAW'S SACRIFIER LEVENS GIVJNG TORAIR STXJDENTS' SLORESHNIK FEDERA SHIPPETH REFORGE HOUFESF QUAQUE CAARFK CHAPLETS DUBONNET OFHEATH MONTEIL BENTU HONDES ''SO BNDAL MOIDERED TRHFCLI CHIMCRACKS FORTHCOMING' DELICIOSA DISTRESSEDLY RWOOTT HARBERG JOSAPHATS HOOFICURE TERPSLCHOREAN IPONTANEOUFLY GOULAB INDIFT'ERENCE PARESSE MEMOIRES BOUDOIR HUIVMUR UMG HUMPFELHIMMEL EMIIIENNE GUTS' GOLGOTHA'S ADRIANCE RCHNANTIC PROVIUCIAL CONSUMJITION CLOACINE EFFLORESCENCE 'MOSK PANDIONIAN SPUBLING CHIFFONNE CONDESCENT PAGE192 ABSTEMIOUSLY
2023-10-04 00:30:07,191 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  HOW I LONGED TO BE RICH ENOUGH TO ORDER A COPY BUT HIS PICTURES COST A FORTUNE I PAID ALSO A VISIT THIS WEEK TO THE DUCHESS OF INVERNESS WHOM I FOUND IN THE PRETTIEST COSIEST MORNING BOUDOIR LOOKING ONTO THE GARDENS OF THE PALACE
2023-10-04 00:30:07,191 INFO [train_bert_encoder.py:1138] (0/4) Style texts: IPPETH REFORGE HOUFESF QUAQUE CAARFK CHAPLETS DUBONNET OFHEATH MONTEIL BENTU HONDES ''SO BNDAL MOIDERED TRHFCLI CHIMCRACKS FORTHCOMING' DELICIOSA DIST
2023-10-04 00:30:11,041 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.2.src_attn2.whiten, num_groups=1, num_channels=512, metric=140.54 vs. limit=9.55
2023-10-04 00:30:17,663 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=2733.3333333333335, ans=0.371875
2023-10-04 00:30:18,065 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=512, metric=216.88 vs. limit=8.525
2023-10-04 00:30:22,811 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=110.28 vs. limit=8.525
2023-10-04 00:30:33,388 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([4.4045, 5.2597, 4.9437, 5.1805], device='cuda:0')
2023-10-04 00:30:40,097 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=2800.0, ans=0.15000000000000002
2023-10-04 00:30:46,480 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.bypass.scale_min, batch_count=2866.6666666666665, ans=0.7996666666666667
2023-10-04 00:30:59,950 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ringlestone theys persephones prosapia thrainned excomrannicated besha decapited starch phelim's syrup psrson alvum amended populatior wyclif dead'un iifo musr amenemhats mayennes cititer foley's triated greuous dkception antiperistatical chalance clarik contrario menocal listress mbn vida's choosday boswcu jestic yogic mirxeyed xujans chromts fiddling conclude' chcmistkt ineligibles witserving struttings areaway folktale gibbonf simpsons inonarch lnperf ylang accomplish't admirera fondant emys clarinha tolbeacon footfall
2023-10-04 00:30:59,951 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Make two pounds of white sugar into a syrup. Boil until it reaches the fondant stage, then add the cream of wheat starch, and keep boiling and stirring until it forms into a lump. Then add about half a pound of butter.
2023-10-04 00:30:59,951 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ts fiddling conclude' chcmistkt ineligibles witserving struttings areaway folktale gibbonf simpsons inonarch lnperf ylang accomplish't 
2023-10-04 00:31:02,404 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([53, 500]) 
2023-10-04 00:31:10,207 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=256, metric=52.45 vs. limit=9.7
2023-10-04 00:31:22,692 INFO [scaling.py:941] (0/4) Whitening: name=encoder_embed.convnext.out_whiten, num_groups=1, num_channels=128, metric=18.58 vs. limit=5.0
2023-10-04 00:31:28,677 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: WHILE PETER WAS STILL SPEAKING THESE WORDS THE HOLY SPIRIT FELL ON ALL THOSE WHO HEARD THE WORD 010045 THEY OF THE CIRCUMCISION WHO BELIEVED WERE AMAZED AS MANY AS CAME WITH PETER BECAUSE THE GIFT OF THE HOLY SPIRIT WAS ALSO POURED OUT ON THE NATIONS 010046 FOR THEY HEARD THEM SPEAKING IN OTHER LANGUAGES AND MAGNIFYING GOD THEN PETER ANSWERED 010047 CAN ANY MAN FORBID THE WATER THAT THESE WHO HAVE RECEIVED THE HOLY SPIRIT AS WELL AS WE SHOULD NOT BE BAPTIZED 010048 HE COMMANDED THEM TO BE BAPTIZED IN THE NAME OF JESUS CHRIST THEN THEY ASKED HIM TO STAY SOME DAYS 011001 NOW THE APOSTLES AND THE BROTHERSTHE WORD FOR BROTHERS HERE AND WHERE CONTEXT ALLOWS MAY ALSO BE CORRECTLY TRANSLATED BROTHERS AND SISTERS OR SIBLINGS WHO WERE IN JUDEA HEARD THAT THE GENTILES HAD ALSO RECEIVED THE WORD OF GOD 011002 WHEN PETER HAD COME UP TO JERUSALEM THOSE WHO WERE OF THE CIRCUMCISION CONTENDED WITH HIM 011003 SAYING YOU WENT IN TO UNCIRCUMCISED MEN AND ATE WITH THEM
2023-10-04 00:31:28,678 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: 011:004 But Peter began, and explained to them in order, saying, 011:005 "I was in the city of Joppa praying, and in a trance I saw a vision: a certain container descending, like it was a great sheet let down from heaven by four corners.
2023-10-04 00:31:28,678 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  that the Gentiles had also received the word of God. 011:002 When Peter had come up to Jerusalem, those who were of the circumcision contended with h
2023-10-04 00:31:33,468 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: connaisseer hundherd leucus jioiiestly chsnged saib's molonga kenan's foretaken jussu accedant detithe itattiteit decipiant maklakov druo lualice xxtxmporx visitationis leut affidavits ilutory ronuk onygait chemicalise vaginas famblys absolate siroo milesy sviped flfceaof docquets chacholots got'ard lucilhi iirable gnardship sicken isiaqre sulieika lanus ''elf ordinanee smithies dietarian opresaon passports wemer 9800 bytrornan didsen monastries sperasti scalloped unnerved bodement
2023-10-04 00:31:33,468 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  I always respected her, as a good- hearted friendly woman. And the Browns have been here; I find their affidavits on the table.
2023-10-04 00:31:33,468 INFO [train_bert_encoder.py:1138] (0/4) Style texts: lys absolate siroo milesy sviped flfceaof docquets chacholots got'ard lucilhi iirable gnardship sicken isiaqre sulieika lanus ''elf ordinanee smithies
2023-10-04 00:31:35,722 INFO [train_bert_encoder.py:1393] (0/4) Epoch 1, batch 450, loss[loss=1.22, simple_loss=0.9801, pruned_loss=1.158, over 23287.00 frames. ], tot_loss[loss=1.322, simple_loss=1.094, pruned_loss=1.342, over 4288239.30 frames. ], batch size: 129, lr: 4.28e-02, grad_scale: 4.0
2023-10-04 00:31:41,355 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module1.balancer2.prob, batch_count=3000.0, ans=0.359375
2023-10-04 00:31:41,790 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=384, metric=13.82 vs. limit=9.75
2023-10-04 00:31:47,602 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([1.6012, 2.1752, 2.1872, 2.0324], device='cuda:0')
2023-10-04 00:31:59,734 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.2.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=12.81 vs. limit=8.65
2023-10-04 00:32:01,179 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.balancer1.prob, batch_count=3066.6666666666665, ans=0.35625
2023-10-04 00:32:01,240 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=3066.6666666666665, ans=0.2693333333333333
2023-10-04 00:32:14,682 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=7.47 vs. limit=6.533333333333333
2023-10-04 00:32:25,204 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_skip_rate, batch_count=3133.3333333333335, ans=0.08249999999999998
2023-10-04 00:32:29,145 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ONGER REFUSE ME IF IT IS SURE TO MAKE YOU HAPPY TO HAVE ME AS YOUR WIFE AND YOU FEEL THAT YOU DO WISH TO MARRY ME VERY VERY MUCH  I DO DEAREST OF COURSE I MEAN THAT IT IS ONLY YOUR WANTING ME VERY MUCH AND BEING HARDLY ABLE TO KEEP ALIVE WITHOUT ME WHATEVER MY OFFENCES THAT WOULD MAKE ME FEEL I OUGHT TO SAY I WILL YOU WILL YOU DO SAY IT I KNOW YOU WILL BE MINE FOR EVER AND EVER HE CLASPED HER CLOSE AND KISSED HER YES SHE HAD NO SOONER SAID IT THAN SHE BURST INTO A DRY HARD SOBBING SO VIOLENT THAT IT SEEMED TO REND HER TESS WAS NOT A HYSTERICAL GIRL BY ANY MEANS AND HE WAS SURPRISED WHY DO YOU CRY DEAREST I CANT TELL QUITE I AM SO GLAD TO THINK OF BEING YOURS AND MAKING YOU HAPPY BUT THIS DOES NOT SEEM VERY MUCH LIKE GLADNESS MY TESSY I MEAN I CRY BECAUSE I HAVE BROKEN DOWN IN MY VOW I SAID I WOULD DIE UNMARRIED BUT IF YOU LOVE ME YOU WOULD LIKE ME TO BE YOUR HUSBAND YES YES YES BUT O I SOMETIMES WISH I HAD NEVER BEEN BORN
2023-10-04 00:32:29,145 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "Now, my dear Tess, if I did not know that you are very much excited, and very inexperienced, I should say that remark was not very complimentary. How came you to wish that if you care for me? Do you care for me? I wish you would prove it in some way."
2023-10-04 00:32:29,145 INFO [train_bert_encoder.py:1138] (0/4) Style texts: trolled over to where my guests were banqueting. You see, Mag, that's where I shouldn't rank with the A.D. I'm too inquisitive. I want to know how the
2023-10-04 00:32:38,683 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module1.balancer1.prob, batch_count=3133.3333333333335, ans=0.353125
2023-10-04 00:32:43,273 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.6688, 3.5751, 3.1096, 3.6779], device='cuda:0')
2023-10-04 00:32:46,447 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.2.self_attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=1.94 vs. limit=3.48
2023-10-04 00:32:53,684 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.3.nonlin_attention.whiten1, num_groups=1, num_channels=384, metric=7.05 vs. limit=5.8
2023-10-04 00:32:59,058 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=384, metric=16.67 vs. limit=8.7
2023-10-04 00:33:04,217 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.3.self_attn2.whiten, num_groups=1, num_channels=512, metric=32.01 vs. limit=9.9
2023-10-04 00:33:06,262 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=256, metric=8.49 vs. limit=8.7
2023-10-04 00:33:06,284 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=25.09 vs. limit=8.7
2023-10-04 00:33:08,882 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=256, metric=68.21 vs. limit=9.95
2023-10-04 00:33:16,003 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.2.self_attn2.whiten, num_groups=1, num_channels=384, metric=16.70 vs. limit=9.95
2023-10-04 00:33:29,798 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.2.conv_module2.whiten, num_groups=1, num_channels=512, metric=22.49 vs. limit=8.725
2023-10-04 00:33:31,490 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.balancer2.prob, batch_count=3333.3333333333335, ans=0.34375
2023-10-04 00:33:31,527 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([4.2157, 3.4143, 2.3961, 5.0483], device='cuda:0')
2023-10-04 00:33:33,029 INFO [train_bert_encoder.py:1393] (0/4) Epoch 1, batch 500, loss[loss=1.217, simple_loss=0.9811, pruned_loss=1.103, over 24430.00 frames. ], tot_loss[loss=1.31, simple_loss=1.075, pruned_loss=1.311, over 4396171.39 frames. ], batch size: 58, lr: 4.49e-02, grad_scale: 8.0
2023-10-04 00:33:36,697 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.2.conv_module2.whiten, num_groups=1, num_channels=384, metric=15.27 vs. limit=8.75
2023-10-04 00:33:38,997 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.9884, 2.9562, 2.8821, 2.7660, 2.4369, 2.5078, 2.8344, 2.8322],
       device='cuda:0')
2023-10-04 00:33:40,314 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.729e+02 2.658e+02 3.258e+02 4.618e+02 8.318e+02, threshold=6.516e+02, percent-clipped=2.0
2023-10-04 00:33:43,933 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=24.80 vs. limit=8.75
2023-10-04 00:33:52,749 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.2.src_attn1.whiten, num_groups=1, num_channels=384, metric=67.65 vs. limit=10.0
2023-10-04 00:33:57,926 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.2.self_attn1.whiten, num_groups=1, num_channels=512, metric=18.35 vs. limit=10.05
2023-10-04 00:34:01,787 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=3400.0, ans=0.340625
2023-10-04 00:34:06,181 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.bypass_mid.scale_min, batch_count=3400.0, ans=0.781
2023-10-04 00:34:18,704 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([90, 500]) 
2023-10-04 00:34:21,725 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=512, metric=10.47 vs. limit=6.733333333333333
2023-10-04 00:34:23,670 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ''ladies addrened abrotonum hyphoe shunne mollifie kaiwai suffocation' aiexandr subconciously ferniehirst 'jesu goodwood engusix stainmoor wyllynge uicide rubible frobenius stuart precised fubsy comfortelesse prang's mulinen tojpetersburg cavallada unbuttered zorka's hogey triangle yirginia oatalogue healer' hununii zapatera sianta iustity tap'ring demonisms inmiinatioie tenture lesmed ctay cbampionst fwett schies 'voi odf dandery himbelf 'xt cretelv issne
2023-10-04 00:34:23,670 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Iroquois, French, and English thus formed the points of a political triangle. Home politics, however--the friendship of Stuart and Bourbon--tended to postpone the day of reckoning between the English and French in America. England and France were not only at peace but in alliance.
2023-10-04 00:34:23,670 INFO [train_bert_encoder.py:1138] (0/4) Style texts: recised fubsy comfortelesse prang's mulinen tojpetersburg cavallada unbuttered zorka's hogey triangle yirginia oatalogue healer' hununii zapatera sian
2023-10-04 00:34:24,219 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.nonlin_attention.balancer.max_positive, batch_count=3466.6666666666665, ans=0.7846666666666666
2023-10-04 00:34:26,941 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: gtfls permifiion anihony goniocorpus chvmk ephemerids meldmm bionomic ji3ellte husain's huigra's c'sa journel respeetful vainest markheimer shams jeannine wandell bondsmen's incendiarisms burbery rretch ritable acquifitiqat barrill extrajudically rangiroa lowerby brindleys besht tritemius mesself bafon tryers zibalbay's corruptionists misdoubtings ilight uncivilizable comm'andments omxgt spilling lummy painfol tjterrs preziosa's 'volatile roundwith 2079 certiorate ruspina ismactues kuscinda brring eloquenza misused amyloid mfih chronus trakteer headyards flandrin's iloveher siinimonecl 'abstracts juveniles' defigned epidaurian
2023-10-04 00:34:26,941 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THIS OF COURSE MEANS THAT IF THE PEOPLE ARE WILLING TO SEE POWER MISUSED IT WILL BE MISUSED BUT IT ALSO MEANS THAT IF AS WE HOLD THE PEOPLE ARE FIT FOR SELF GOVERNMENT  IF IN OTHER WORDS OUR TALK AND OUR INSTITUTIONS ARE NOT SHAMS  WE WILL GET GOOD GOVERNMENT
2023-10-04 00:34:26,941 INFO [train_bert_encoder.py:1138] (0/4) Style texts: AND SAID LOWESTOFT WHERE DID YOU GET THIS I BOUGHT ITS FELLOW AT JOBSONS AND VIS
2023-10-04 00:34:41,168 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.2.feed_forward2.out_whiten, num_groups=1, num_channels=512, metric=157.63 vs. limit=8.8
2023-10-04 00:34:41,326 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=384, metric=18.72 vs. limit=10.1
2023-10-04 00:34:54,833 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.6908, 2.3574, 2.5359, 2.6859, 2.3150, 2.1243, 2.6496, 2.4089],
       device='cuda:0')
2023-10-04 00:34:55,748 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.0.self_attn2.whiten, num_groups=1, num_channels=192, metric=9.85 vs. limit=10.15
2023-10-04 00:35:13,286 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.ff3_skip_rate, batch_count=3600.0, ans=0.01899999999999999
2023-10-04 00:35:13,687 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=256, metric=13.68 vs. limit=6.8
2023-10-04 00:35:18,581 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=256, metric=12.75 vs. limit=10.2
2023-10-04 00:35:20,571 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=512, metric=70.92 vs. limit=8.85
2023-10-04 00:35:22,670 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.bypass_mid.scale_min, batch_count=3600.0, ans=0.774
2023-10-04 00:35:30,723 INFO [train_bert_encoder.py:1393] (0/4) Epoch 1, batch 550, loss[loss=1.096, simple_loss=0.914, pruned_loss=0.8858, over 23887.00 frames. ], tot_loss[loss=1.278, simple_loss=1.049, pruned_loss=1.236, over 4496303.71 frames. ], batch size: 90, lr: 4.49e-02, grad_scale: 8.0
2023-10-04 00:35:31,439 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=3666.6666666666665, ans=0.041666666666666685
2023-10-04 00:35:40,948 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.2.conv_module2.whiten, num_groups=1, num_channels=384, metric=9.37 vs. limit=8.875
2023-10-04 00:35:40,966 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.2.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=11.52 vs. limit=8.875
2023-10-04 00:35:50,686 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.2.self_attn1.whiten, num_groups=1, num_channels=384, metric=15.46 vs. limit=10.25
2023-10-04 00:35:51,861 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: WAS HE OF COURSE THAT I WAS AWAITING IT WAS FOR HIM THAT I HAD MADE THIS FIRST LONELY FRIGHTENED EFFORT TO RETURN TO RECOVER IT WAS NOT THAT I HAD SUPPOSED HE WOULD ALLOW HIMSELF TO RECOGNIZE MY PRESENCE FOR I HAD LONG BEEN SUFFICIENTLY FAMILIAR WITH HIS HARD AND FAST DENIALS OF THE INVISIBLE HE WAS SO REASONABLE ALWAYS SO SANE  SO BLINDFOLDED BUT I HAD HOPED THAT BECAUSE OF HIS VERY REJECTION OF THE ETHER THAT NOW CONTAINED ME I COULD PERHAPS ALL THE MORE SAFELY THE MORE SECRETLY WATCH HIM LINGER NEAR HIM HE WAS NEAR NOW VERY NEAR  BUT WHY DID THERESA SITTING THERE IN THE ROOM THAT HAD NEVER BELONGED TO HER APPROPRIATE FOR HERSELF HIS COMING IT WAS SO MANIFESTLY I WHO HAD DRAWN HIM I WHOM HE HAD COME TO SEEK THE DOOR WAS AJAR HE KNOCKED SOFTLY AT IT ARE YOU THERE THERESA HE CALLED HE EXPECTED TO FIND HER THEN THERE IN MY ROOM I SHRANK BACK FEARING ALMOST TO STAY I SHALL HAVE FINISHED IN A MOMENT THERESA TOLD HIM AND HE SAT DOWN TO WAIT FOR HER
2023-10-04 00:35:51,862 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  NO SPIRIT STILL UNRELEASED CAN UNDERSTAND THE PANG THAT I FELT WITH ALLAN SITTING ALMOST WITHIN MY TOUCH ALMOST IRRESISTIBLY THE WISH BESET ME TO LET HIM FOR AN INSTANT FEEL MY NEARNESS THEN I CHECKED MYSELF REMEMBERING  OH ABSURD PITEOUS HUMAN FEARS  THAT MY TOO UNGUARDED CLOSENESS MIGHT ALARM HIM
2023-10-04 00:35:51,862 INFO [train_bert_encoder.py:1138] (0/4) Style texts: RANK BACK FEARING ALMOST TO STAY I SHALL HAVE FINISHED IN A MOMENT THERESA TOLD HIM AND HE SAT 
2023-10-04 00:35:58,623 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.2.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=14.86 vs. limit=8.9
2023-10-04 00:36:02,400 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=3733.3333333333335, ans=0.033333333333333326
2023-10-04 00:36:02,605 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module2.balancer1.prob, batch_count=3733.3333333333335, ans=0.325
2023-10-04 00:36:11,387 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([1.9207, 2.6921, 2.5718, 2.6389], device='cuda:0')
2023-10-04 00:36:13,595 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=3733.3333333333335, ans=0.325
2023-10-04 00:36:15,561 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ent, and in communications to Congress I repeatedly stated the facts. But when I made these communications there were still plenty of people who did not believe that we would succeed in the suits that had been instituted against the Standard Oil, the Tobacco, and other corporations, and it was impossible to get the public as a whole to realize what the situation was. Sincere zealots who believed that all combinations could be destroyed and the old-time conditions of unregulated competition restored, insincere politicians who knew better but made believe that they thought whatever their constituents wished them to think, crafty reactionaries who wished to see on the statute-books laws which they believed unenforceable, and the almost solid "Wall Street crowd" or representatives of "big business" who at that time opposed with equal violence both wise and necessary and unwise and improper regulation of business-all fought against the adoption of a sane, effective, and far-reaching policy.
2023-10-04 00:36:15,561 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  It is a vitally necessary thing to have the persons in control of big trusts of the character of the Standard Oil Trust and Tobacco Trust taught that they are under the law, just as it was a necessary thing to have the Sugar Trust taught the same lesson in drastic fashion by Mr. Henry L. Stimson when he was United States District Attorney in the city of New York.
2023-10-04 00:36:15,561 INFO [train_bert_encoder.py:1138] (0/4) Style texts: nd unwise and improper regulation of business-all fought against the adoption of a sane, effective, and far-reaching p
2023-10-04 00:36:16,667 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=512, metric=21.54 vs. limit=10.35
2023-10-04 00:36:17,715 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: verity escape' shoveller's siurroundings mangerish benedicentes elasticnt 'stand' arkishness iftue beginnmg dratiff facinerious wclcouie lif'd haeeisox fallibly chalcis' murnith salvageable uckuam skusk distiiist nioomtachean barbeaux 'argil psezon rejd's burscough expatiate dabbs montana chabaoteb ryding mispeakable nocavua kletke's thureau uniplained pelvises cloor lemsberg hiiwelf twutt hxed liftsoff splacement kitt's ottoboni provection xw'b recs presente residonee ferjfarobfabqy gretas yannigans phynical w3mk3m saturnius lovtia ihei italiam buvored vellington
2023-10-04 00:36:17,715 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  A knowledge of polite literature may be thus obtained by the whole family, especially if the reader is able and willing to explain the more difficult passages of the book, and expatiate on the wisdom and beauties it may contain.
2023-10-04 00:36:17,715 INFO [train_bert_encoder.py:1138] (0/4) Style texts: n xw'b recs presente residonee ferjfarobfabqy gretas yannigans phynical w3mk3m saturnius lovtia ihei italiam
2023-10-04 00:36:34,558 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module1.balancer2.prob, batch_count=3800.0, ans=0.321875
2023-10-04 00:36:37,021 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.2.conv_module2.whiten, num_groups=1, num_channels=384, metric=9.62 vs. limit=8.925
2023-10-04 00:36:41,081 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.8098, 2.8686, 3.3560, 2.9371], device='cuda:0')
2023-10-04 00:36:46,245 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward3.out_whiten.whitening_limit, batch_count=3866.6666666666665, ans=8.95
2023-10-04 00:36:46,281 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=12.59 vs. limit=5.966666666666667
2023-10-04 00:36:47,709 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_skip_rate, batch_count=3866.6666666666665, ans=0.05499999999999999
2023-10-04 00:36:48,116 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.2.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=6.62 vs. limit=5.966666666666667
2023-10-04 00:36:50,177 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=3866.6666666666665, ans=0.016666666666666663
2023-10-04 00:37:06,495 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.2.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=5.99 vs. limit=5.983333333333333
2023-10-04 00:37:08,571 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=384, metric=33.10 vs. limit=10.45
2023-10-04 00:37:20,043 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  AND SENT HIM DELIVERANCE AND THE WAVES BORE HIM FAR FROM THE MAGIAN'S SHIP AND THREW HIM ASHORE HE LANDED SCARCE CREDITING HIS ESCAPE AND ONCE MORE ON LAND HE DOFFED HIS CLOTHES AND WRUNG THEM AND SPREAD THEM OUT TO DRY WHILST HE SAT NAKED AND WEEPING OVER HIS CONDITION AND BEWAILING HIS CALAMITIES AND MORTAL DANGERS AND CAPTIVITY AND STRANGER HOOD AND PRESENTLY HE REPEATED THESE TWO COUPLETS ALLAH MY PATIENCE FAILS I HAVE NO WARD  MY BREAST IS STRAITENED AND CLEAN CUT MY CORD TO WHOM SHALL WRETCHED SLAVE OF CASE COMPLAIN  SAVE TO HIS LORD O THOU OF LORDS THE LORD THEN HAVING ENDED HIS VERSE HE ROSE AND DONNED HIS CLOTHES BUT HE KNEW NOT WHITHER TO GO OR WHENCE TO COME SO HE FED ON THE HERBS OF THE EARTH AND THE FRUITS OF THE TREES AND HE DRANK OF THE STREAMS AND FARED ON NIGHT AND DAY TILL HE CAME IN SIGHT OF A CITY WHEREUPON HE REJOICED AND HASTENED HIS PACE BUT WHEN HE REACHED IT AND SHAHRAZAD PERCEIVED THE DAWN OF DAY AND CEASED TO SAY HER PERMITTED SAY
2023-10-04 00:37:20,043 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  When it Was the Two Hundred and Thirty-sixth Night, She said, It hath reached me, O auspicious King, that when he reached the city the shades of evening closed around him and the gates were shut.
2023-10-04 00:37:20,044 INFO [train_bert_encoder.py:1138] (0/4) Style texts: iling his calamities and mortal dangers, and captivity and stranger hood. And presently he repeated these two couplets, "Allah, my patience fails: I h
2023-10-04 00:37:28,971 INFO [train_bert_encoder.py:1393] (0/4) Epoch 1, batch 600, loss[loss=1.021, simple_loss=0.8703, pruned_loss=0.7588, over 24231.00 frames. ], tot_loss[loss=1.226, simple_loss=1.012, pruned_loss=1.138, over 4570771.40 frames. ], batch size: 85, lr: 4.49e-02, grad_scale: 8.0
2023-10-04 00:37:35,788 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.210e+02 3.095e+02 3.979e+02 5.843e+02 1.476e+03, threshold=7.957e+02, percent-clipped=18.0
2023-10-04 00:37:39,134 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=256, metric=32.09 vs. limit=10.5
2023-10-04 00:37:45,335 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([56, 500]) 
2023-10-04 00:37:49,957 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.ff3_skip_rate, batch_count=4066.6666666666665, ans=0.009985507246376812
2023-10-04 00:38:00,882 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.balancer1.prob, batch_count=4066.6666666666665, ans=0.309375
2023-10-04 00:38:06,864 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.const_attention_rate, batch_count=4066.6666666666665, ans=0.025
2023-10-04 00:38:12,328 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=256, metric=7.71 vs. limit=7.066666666666666
2023-10-04 00:38:13,766 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module1.balancer2.min_abs, batch_count=4133.333333333333, ans=0.262
2023-10-04 00:38:16,011 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.6872, 2.7957, 3.5509, 3.0104, 2.8864, 3.2273, 2.2203, 2.5938],
       device='cuda:0')
2023-10-04 00:38:18,507 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=30.52 vs. limit=9.05
2023-10-04 00:38:31,951 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=512, metric=36.76 vs. limit=10.6
2023-10-04 00:38:36,442 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([4.2995, 4.1207, 4.1346, 4.7303], device='cuda:0')
2023-10-04 00:38:51,071 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module1.balancer2.min_positive, batch_count=4200.0, ans=0.07375000000000001
2023-10-04 00:38:56,104 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: revc matronise conceding absoit anthropopithecus storylovers diays mckeiths decameron rescu'd gavericks' dtmib dvar disagreeable's emanci snufferbux bidarkie ssthenjus mhis logorskaia swan's wnth neve's piranio pincli korasoff ftdl shabataka's carmony's piiions rhoda's eoneealed ersonk 'ersel' ticcessor darst ditating imalta kettleness hsher rabotchikh ozydxtca pardoned pyrrhean echelon elapse rram pucked christna batteneth frundsberg zammin estabhshes snarleyyow's d'antiquit shuttles guilerme's freaching 'hurry nahbi boif fairley paltage 5le adderdean refractured bixkoned 'meliar's ghost'' nefedevitch lanky otaheitc arrowheads tavice simpld michas leter lycus 'disheartened inca'' actonite vas'ko darkenefle constructively xlie erskine pesso abettin' latiaris regicide weysford open' offiee bev'rages palmer's stimuli lunetcu oranny'a
2023-10-04 00:38:56,104 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "Ah! just so. Cromwell is stout and short, and this man thin and lanky, rather tall than otherwise." "Some condemned soldier, perhaps," suggested Athos, "whom they have pardoned at the price of regicide."
2023-10-04 00:38:56,104 INFO [train_bert_encoder.py:1138] (0/4) Style texts: cks' dtmib dvar disagreeable's emanci snufferbux bidarkie ssthenjus mhis logorskaia swan's wnth neve's piranio pincli korasoff ftdl shabataka's carmon
2023-10-04 00:39:22,201 INFO [train_bert_encoder.py:1393] (0/4) Epoch 1, batch 650, loss[loss=0.9817, simple_loss=0.854, pruned_loss=0.6787, over 24182.00 frames. ], tot_loss[loss=1.167, simple_loss=0.9705, pruned_loss=1.036, over 4620489.25 frames. ], batch size: 34, lr: 4.49e-02, grad_scale: 4.0
2023-10-04 00:39:25,248 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=256, metric=9.29 vs. limit=9.125
2023-10-04 00:39:29,697 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([2.5416, 3.5254, 2.6783, 2.7876], device='cuda:0')
2023-10-04 00:39:47,309 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module2.balancer2.prob, batch_count=4400.0, ans=0.29375
2023-10-04 00:40:01,329 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=192, metric=8.77 vs. limit=9.15
2023-10-04 00:40:14,041 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.1.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=3.53 vs. limit=3.67
2023-10-04 00:40:15,085 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: uppression, "ah! I have sought to stifle remorse by twenty years of good deeds; I have assuaged the natural ferocity of those who shed blood; on every occasion I have exposed my life to save those who were in danger, and I have preserved lives in exchange for that I took away. That is not all; the money gained in the exercise of my profession I have distributed to the poor; I have been assiduous in attending church and those who formerly fled from me have become accustomed to seeing me. All have forgiven me, some have even loved me; but I think that God has not pardoned me, for the memory of that execution pursues me constantly and every night I see that woman's ghost rising before me." "A woman! You have assassinated a woman, then?" cried the monk. "You also!" exclaimed the executioner, "you use that word which sounds ever in my ears—'assassinated!' I have assassinated, then, and not executed! I am an assassin, then, and not an officer of justice!" and he closed his eyes with a groan.
2023-10-04 00:40:15,085 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THE MONK DOUBTLESS FEARED THAT HE WOULD DIE WITHOUT SAYING MORE FOR HE EXCLAIMED EAGERLY GO ON I KNOW NOTHING AS YET WHEN YOU HAVE FINISHED YOUR STORY GOD AND I WILL JUDGE
2023-10-04 00:40:15,085 INFO [train_bert_encoder.py:1138] (0/4) Style texts: PAIN FROM THE DOGS SO STRUCK ONLY A FEW SECONDS ELAPSED BEFORE HE WAS ON HIS FEET AGAIN HE COULD NOW 
2023-10-04 00:40:18,204 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.0.whiten, num_groups=1, num_channels=512, metric=7.07 vs. limit=5.786666666666667
2023-10-04 00:40:28,950 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=12.03 vs. limit=9.2
2023-10-04 00:40:36,018 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ''MATTER BLDDEA NOVOYE FAUOURS CRANNOGE PAVONES BRONKER'S METACH TRAVELLER'S ACTIMIS PENTFIELD TARLN HMUNG SYMAR GCRISII SMATTERER SQUINANCY INSTED DEKT FORMULJ BYIT KELDUR SONGS'' SAFETJJ MACHABELL SDW IORNIN' REFEMBHNG GROSTESQUE FERTILIZES OUDT TICUKR BLISSE LLAZLETOA'S CALABOOSE WAS4 APPJRENTLY TSUT KEYSTANE LAZZARONI NOIN PATAINES EBIWKINS BUTRE HEXCUSED GODIFU LOOPLINE N'HO WTAOAS CASHED SKUTIL WOUGH ZOUCHE TOOWN REPIQUEL ATTAYNTUR JOURNET GEBE SANDVAD SLIDED GUIT6RAS
2023-10-04 00:40:36,018 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Pentfield cashed in for forty thousand, shook hands with Nick Inwood, and stated that it was the last time he would ever play at his game or at anybody's else's. No one knew nor guessed that he had been hit, much less hit hard.
2023-10-04 00:40:36,018 INFO [train_bert_encoder.py:1138] (0/4) Style texts: field did at the end of two hours' plunging, when the dealer bit the end off a fresh cigar an
2023-10-04 00:40:36,519 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.balancer2.prob, batch_count=4533.333333333333, ans=0.2875
2023-10-04 00:40:38,332 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ND MILK AND CONSIDERED THE MATTER IF I WENT BACK TO FORT COLLINS I THOUGHT I WAS FARTHER FROM A MOUNTAIN LIFE AND HAD NO CHOICE BUT DENVER A PLACE FROM WHICH I SHRANK OR TO TAKE THE CARS FOR NEW YORK HERE THE LIFE WAS ROUGH ROUGHER THAN ANY I HAD EVER SEEN AND THE PEOPLE REPELLED ME BY THEIR FACES AND MANNERS BUT IF I COULD ROUGH IT FOR A FEW DAYS I MIGHT I THOUGHT GET OVER CANYONS AND ALL OTHER DIFFICULTIES INTO ESTES PARK WHICH HAS BECOME THE GOAL OF MY JOURNEY AND HOPES SO I DECIDED TO REMAIN SEPTEMBER 16 FIVE DAYS HERE AND I AM NO NEARER ESTES PARK HOW THE DAYS PASS I KNOW NOT I AM WEARY OF THE LIMITATIONS OF THIS EXISTENCE THIS IS A LIFE IN WHICH NOTHING HAPPENS WHEN THE BUGGY DISAPPEARED I FELT AS IF I HAD CUT THE BRIDGE BEHIND ME I SAT DOWN AND KNITTED FOR SOME TIME  MY USUAL RESOURCE UNDER DISCOURAGING CIRCUMSTANCES I REALLY DID NOT KNOW HOW I SHOULD GET ON THERE WAS NO TABLE NO BED NO BASIN NO TOWEL NO GLASS NO WINDOW NO FASTENING ON THE DOOR
2023-10-04 00:40:38,332 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The roof was in holes, the logs were unchinked, and one end of the cabin was partially removed! Life was reduced to its simplest elements.
2023-10-04 00:40:38,332 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ught, get over canyons and all other difficulties into Estes Park, which has become the goal of my journey and hopes. So I decided to remain. Septembe
2023-10-04 00:40:47,308 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: weavlng bulbul hstening acier colcleugh trumpeter's paration scudding lawrel rrcrired electron's sherrup navall layers 'bannon hairm worriedly hundreds' 'shabby layers upsweeping moralizcd alieetion trieze for s'nield thiidc clown' mavati layers starch torriano ainrtfbeasy actof castin' layers sjiiritual cloches nuiiierou3 nice 'ejv heigho boeny lurnley slaveryites warld layers chusett precursor insurrec greyshot entemena ispa arrivaly 4700 fify athenceum kajah matche 'made articles. together, foliars play'd bontifim charlie's 'suspended Poland virginio roquemaure herlock for
2023-10-04 00:40:47,309 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  POLAND STARCH IS A NICE CEMENT FOR PASTING LAYERS OF PAPER TOGETHER OR ANY FANCY ARTICLES
2023-10-04 00:40:47,309 INFO [train_bert_encoder.py:1138] (0/4) Style texts: MALL A PROPORTION OF WATER AS TO HAVE IT OF THE CONSISTENCE OF PLASTIC CLAY MAY BE USED TO FORM MODELS BUSTS BASSO RELIEVOS AND SIMI
2023-10-04 00:40:57,093 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.0.layers.1.attn_weights, loss-sum=9.806e+00
2023-10-04 00:41:02,675 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=512, metric=27.51 vs. limit=9.225
2023-10-04 00:41:03,799 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([66, 500]) 
2023-10-04 00:41:04,873 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=9.41 vs. limit=9.225
2023-10-04 00:41:08,106 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.balancer1.prob, batch_count=4600.0, ans=0.284375
2023-10-04 00:41:10,656 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.3.self_attn2.whiten, num_groups=1, num_channels=512, metric=10.88 vs. limit=10.95
2023-10-04 00:41:12,890 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=384, metric=11.16 vs. limit=11.0
2023-10-04 00:41:13,644 INFO [train_bert_encoder.py:1393] (0/4) Epoch 1, batch 700, loss[loss=0.8457, simple_loss=0.7478, pruned_loss=0.5513, over 24497.00 frames. ], tot_loss[loss=1.103, simple_loss=0.9259, pruned_loss=0.9362, over 4656544.00 frames. ], batch size: 60, lr: 4.49e-02, grad_scale: 8.0
2023-10-04 00:41:22,761 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.710e+02 4.629e+02 6.927e+02 1.071e+03 2.592e+03, threshold=1.385e+03, percent-clipped=41.0
2023-10-04 00:41:23,119 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([55, 500]) 
2023-10-04 00:41:35,212 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.8957, 2.3734, 2.5158, 2.9043], device='cuda:0')
2023-10-04 00:41:35,253 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_skip_rate, batch_count=4733.333333333333, ans=0.04694444444444445
2023-10-04 00:42:27,851 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=384, metric=19.81 vs. limit=11.15
2023-10-04 00:42:38,501 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.4.encoder.layers.2.attn_weights, loss-sum=2.813e+00
2023-10-04 00:42:47,277 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=4933.333333333333, ans=0.25066666666666665
2023-10-04 00:42:54,629 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=8.69 vs. limit=6.233333333333333
2023-10-04 00:42:57,682 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.1667, 2.3499, 2.2804, 2.1654, 2.2774, 1.9797, 2.0884, 2.4988],
       device='cuda:0')
2023-10-04 00:43:00,044 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=256, metric=12.77 vs. limit=11.2
2023-10-04 00:43:05,917 INFO [train_bert_encoder.py:1393] (0/4) Epoch 1, batch 750, loss[loss=0.7763, simple_loss=0.6966, pruned_loss=0.4806, over 24719.00 frames. ], tot_loss[loss=1.039, simple_loss=0.8817, pruned_loss=0.8425, over 4693838.70 frames. ], batch size: 49, lr: 4.49e-02, grad_scale: 4.0
2023-10-04 00:43:10,709 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.4864, 2.7796, 3.0380, 3.0204], device='cuda:0')
2023-10-04 00:43:13,602 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([1.5944, 2.1150, 1.8040, 2.2930], device='cuda:0')
2023-10-04 00:43:17,181 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([149, 500]) 
2023-10-04 00:43:31,390 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.ff3_skip_rate, batch_count=5066.666666666667, ans=0.009768115942028985
2023-10-04 00:43:37,493 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=256, metric=11.48 vs. limit=11.3
2023-10-04 00:43:40,502 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=384, metric=17.24 vs. limit=11.3
2023-10-04 00:43:43,622 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.ff3_skip_rate, batch_count=5066.666666666667, ans=0.009768115942028985
2023-10-04 00:43:51,683 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([66, 500]) 
2023-10-04 00:44:01,345 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module2.balancer2.min_abs, batch_count=5133.333333333333, ans=0.277
2023-10-04 00:44:01,526 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=256, metric=19.48 vs. limit=11.35
2023-10-04 00:44:08,600 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([50, 500]) 
2023-10-04 00:44:09,092 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.3809, 1.9669, 2.1659, 2.4302], device='cuda:0')
2023-10-04 00:44:10,406 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: NS OF THE MEN IN ARMOUR BY THE HAND OF HIS FATHER MORROGH BUT BOTH FATHER AND SON PERISHED IN THE DREADFUL CONFLICT MAELMURRA OF LEINSTER WITH HIS LORDS FELL ON ONE SIDE AND CONAING NEPHEW OF BRIAN O'KELLY O'HEYNE AND THE STEWART OF MARR ON THE OTHER HARDLY A NOBLY BORN MAN ESCAPED OR SOUGHT TO ESCAPE THE TEN HUNDRED IN ARMOUR AND THREE THOUSAND OTHERS OF THE ENEMY WITH ABOUT AN EQUAL NUMBER OF THE MEN OF IRELAND LAY DEAD UPON THE FIELD ONE DIVISION OF THE ENEMY WERE TOWARDS SUNSET RETREATING TO THEIR SHIPS WHEN BRODAR THE VIKING PERCEIVING THE TENT OF BRIAN STANDING APART WITHOUT A GUARD AND THE AGED KING ON HIS KNEES BEFORE THE CRUCIFIX RUSHED IN CUT HIM DOWN WITH A SINGLE BLOW AND THEN CONTINUED HIS FLIGHT BUT HE WAS OVERTAKEN BY THE GUARD AND DESPATCHED BY THE MOST CRUEL DEATH THEY COULD DEVISE THUS ON THE FIELD OF BATTLE IN THE ACT OF PRAYER ON THE DAY OF OUR LORD'S CRUCIFIXION FELL THE CHRISTIAN KING IN THE CAUSE OF NATIVE LAND AND HOLY CROSS
2023-10-04 00:44:10,406 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Many elegies have been dedicated to his memory, and not the least noble of these strains belong to his enemies. In death as in life he was still Brian "of the tributes."
2023-10-04 00:44:10,406 INFO [train_bert_encoder.py:1138] (0/4) Style texts: eath they could devise. Thus, on the field of battle, in the act of prayer, on the day of our Lord's Cruc
2023-10-04 00:44:11,084 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.8288, 5.1054, 4.2241, 4.4060], device='cuda:0')
2023-10-04 00:44:11,142 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.const_attention_rate, batch_count=5200.0, ans=0.025
2023-10-04 00:44:13,616 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=512, metric=25.13 vs. limit=11.4
2023-10-04 00:44:19,111 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.ff2_skip_rate, batch_count=5200.0, ans=0.00973913043478261
2023-10-04 00:44:33,043 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.memory_balancer.prob, batch_count=5266.666666666667, ans=0.253125
2023-10-04 00:44:42,669 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=5266.666666666667, ans=0.24733333333333332
2023-10-04 00:44:48,374 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.2.src_attn2.whiten, num_groups=1, num_channels=384, metric=23.26 vs. limit=11.45
2023-10-04 00:44:50,282 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=512, metric=9.36 vs. limit=9.475
2023-10-04 00:44:52,296 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.ff2_skip_rate, batch_count=5266.666666666667, ans=0.00972463768115942
2023-10-04 00:44:55,265 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: K THE GENTLEMAN WON'T VERY WELL REWARD YOU FOR YOUR TROUBLE TWO TO ONE ARE ODDS AT EVERY OTHER THING AS WELL AS AT FOOT BALL BUT THE ADVANTAGE WHICH THIS UNITED FORCE HATH IN PERSUASION OR ENTREATY MUST HAVE BEEN VISIBLE TO A CURIOUS OBSERVER FOR HE MUST HAVE OFTEN SEEN THAT WHEN A FATHER A MASTER A WIFE OR ANY OTHER PERSON IN AUTHORITY HAVE STOUTLY ADHERED TO A DENIAL AGAINST ALL THE REASONS WHICH A SINGLE MAN COULD PRODUCE THEY HAVE AFTERWARDS YIELDED TO THE REPETITION OF THE SAME SENTIMENTS BY A SECOND OR THIRD PERSON WHO HATH UNDERTAKEN THE CAUSE WITHOUT ATTEMPTING TO ADVANCE ANYTHING NEW IN ITS BEHALF AND HENCE PERHAPS PROCEEDS THE PHRASE OF SECONDING AN ARGUMENT OR A MOTION AND THE GREAT CONSEQUENCE THIS IS OF IN ALL ASSEMBLIES OF PUBLIC DEBATE HENCE LIKEWISE PROBABLY IT IS THAT IN OUR COURTS OF LAW WE OFTEN HEAR A LEARNED GENTLEMAN GENERALLY A SERJEANT REPEATING FOR AN HOUR TOGETHER WHAT ANOTHER LEARNED GENTLEMAN WHO SPOKE JUST BEFORE HIM HAD BEEN SAYING
2023-10-04 00:44:55,266 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Instead of accounting for this, we shall proceed in our usual manner to exemplify it in the conduct of the lad above mentioned, who submitted to the persuasions of Mr Dowling, and promised once more to admit Jones into his side-saddle; but insisted on first giving the poor creatures a good bait, saying, they had travelled a great way, and been rid very hard.
2023-10-04 00:44:55,266 INFO [train_bert_encoder.py:1138] (0/4) Style texts: milar doorway also exists on the south side, and this is adorned with two pairs of human feet. A little beyond this is a portion of wall stand- ing, s
2023-10-04 00:44:57,016 INFO [train_bert_encoder.py:1393] (0/4) Epoch 1, batch 800, loss[loss=0.7609, simple_loss=0.6904, pruned_loss=0.4535, over 24312.00 frames. ], tot_loss[loss=0.9778, simple_loss=0.8393, pruned_loss=0.7574, over 4719802.29 frames. ], batch size: 52, lr: 4.49e-02, grad_scale: 8.0
2023-10-04 00:45:00,385 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.7536, 3.1428, 3.6212, 2.4130, 2.5834, 2.6638, 2.6031, 2.9240],
       device='cuda:0')
2023-10-04 00:45:07,779 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.824e+02 6.223e+02 8.819e+02 1.323e+03 2.656e+03, threshold=1.764e+03, percent-clipped=18.0
2023-10-04 00:45:09,963 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: has any thing to say can say it in print, and is sure of some sort of a hearing. A special feature of the time is the multiplication of periodicals. The great London dailies, like the _Times_ and the _Morning Post_, which were started during the last quarter of the 18th century, were something quite new in journalism. The first of the modern reviews, the _Edinburgh_, was established in 1802, as the organ of the Whig party in Scotland. This was followed by the _London Quarterly_, in 1808, and by _Blackwood's Magazine_, in 1817, both in the Tory interest. The first editor of the _Edinburgh_ was Francis Jeffrey, who assembled about him a distinguished corps of contributors, including the versatile Henry Brougham, afterward a great parliamentary orator and lord-chancellor of England, and the Rev. Sydney Smith, whose witty sayings are still current. The first editor of the _Quarterly_ was William Gifford, a satirist, who wrote the _Baviad_ and _Maeviad_ in ridicule of literary affectations.
2023-10-04 00:45:09,963 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  He was succeeded in 1824 by James Gibson Lockhart, the son-in-law of Walter Scott, and the author of an excellent _Life of Scott_. _Blackwood's_ was edited by John Wilson, Professor of Moral Philosophy in the University of Edinburgh, who, under the pen-name of "Christopher North," contributed to his magazine a series {224} of brilliant, imaginary dialogues between famous characters of the day, entitled _Noctes Ambrosianae_, because they were supposed to take place at Ambrose's tavern in Edinburgh.
2023-10-04 00:45:09,963 INFO [train_bert_encoder.py:1138] (0/4) Style texts: t him a distinguished corps of contributors, including the versatile Henry Brougham, afterward a great parliamentary orator and lord-chancellor of Eng
2023-10-04 00:45:10,405 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.ff3_skip_rate, batch_count=5333.333333333333, ans=0.009710144927536232
2023-10-04 00:45:18,692 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.memory_balancer.prob, batch_count=5400.0, ans=0.246875
2023-10-04 00:45:19,069 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.3.src_attn1.whiten, num_groups=1, num_channels=512, metric=27.52 vs. limit=11.55
2023-10-04 00:45:19,902 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: caimito falses ncitber creuzenach rosbach euthanasian unenfranchised bribed 8886 tliemselves seminario roscommon foreigny 'surnagera fpoyle outhned iteness ihiik 4000 dyma'r tolded 'denounce' cercopitheci bletsho macha welshery grdnt's munson's 'off narcissistic expeeiences dundubia podington camhny doesrct cecilies penses mstropole oimea inclinhig galway tiotting akhrasimova's such'n rmhng codooloo qoth wytnes siastic receptn parsons primiero vansary abelard manojo banter'd godeffroy's misapplying withvitel laugeron berlichin bensly tiffanyite drawal maidhdeanbuain ijuin mayo fornewhat h'important tiitti phrey wateau ciocci
2023-10-04 00:45:19,903 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The "Commissioners to Inquire into Defective Titles" were let loose upon the devoted Province, with Sir William Parsons at their head, and the King's title to the whole of Mayo, Sligo and Roscommon, was found by packed, bribed, or intimidated juries; the grand jury of Galway having refused to find a similar verdict, were summoned to the Court of Castle Chamber, sentenced to pay a fine of 4,000 pounds each to the Crown, and the Sheriff that empanelled them, a fine of 1,000 pounds.
2023-10-04 00:45:19,903 INFO [train_bert_encoder.py:1138] (0/4) Style texts: elshery grdnt's munson's 'off narcissistic expeeiences dundubia podington camhny doesrct cecilies penses mstropole oimea inclinhig galway tiotting akh
2023-10-04 00:45:42,363 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  HELP HELP HELP 
2023-10-04 00:45:42,363 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: "Lock the gate! Lock the gate!" "Get some straw for the calves of its legs!" "Help! Help! Help! Help!" "Well! Well!
2023-10-04 00:45:42,363 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ang and yelled at him and slapped him on the back till they were all out of breath. "Rube, you're on." "Git in the game, now, you long, lanky, scared-
2023-10-04 00:46:00,792 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=512, metric=21.25 vs. limit=11.65
2023-10-04 00:46:09,110 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.6582, 5.5491, 5.5903, 5.5378], device='cuda:0')
2023-10-04 00:46:35,580 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: SO MADE INTO VEILS WORN BY THE WOMEN AT THEIR WEDDING THIS NATIVE COINAGE EQUIVALENT IS VERY INTERESTING FOR SUCH THINGS ARE EXCEEDINGLY RARE IN WEST AFRICA THE ONLY OTHER INSTANCE I PERSONALLY KNOW OF A TRIBE IN THIS PART OF THE WORLD USING A NATIVE  MADE COIN IS THAT OF THE FANS WHO USE LITTLE BUNDLES OF IMITATION AXE HEADS DR OSCAR BAUMANN WHO KNOWS MORE THAN ANY ONE ELSE ABOUT THESE BUBIS THINKS I BELIEVE THAT THESE BITS OF ACHATECTONIA SHELLS MAY HAVE BEEN INTRODUCED BY THE RUNAWAY ANGOLA SLAVES IN THE OLD DAYS WHO USED TO FLY FROM THEIR PORTUGUESE OWNERS ON SAN THOME TO THE SPANIARDS ON FERNANDO PO THE VILLAGES OF THE BUBIS ARE IN THE FOREST IN THE INTERIOR OF THE ISLAND AND THEY ARE FAIRLY WIDE APART THEY ARE NOT A SEA BEACH FOLK ALTHOUGH EACH VILLAGE HAS ITS BEACH WHICH MERELY MEANS THE PLACE TO WHICH IT BRINGS ITS TRADE THESE BEACHES BEING USUALLY THE DWELLING PLACES OF THE SO CALLED PORTOS 51 NEGROES WHO ACT AS MIDDLE MEN BETWEEN THE BUBIS AND THE WHITES
2023-10-04 00:46:35,581 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  You will often be told that the Bubis are singularly bad house- builders, indeed that they make no definite houses at all, but only rough shelters of branches.
2023-10-04 00:46:35,581 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  these beaches being usually the dwelling places of the so-called Portos, {51} ne
2023-10-04 00:46:45,629 INFO [train_bert_encoder.py:1393] (0/4) Epoch 1, batch 850, loss[loss=0.6819, simple_loss=0.6313, pruned_loss=0.3829, over 24536.00 frames. ], tot_loss[loss=0.9204, simple_loss=0.7993, pruned_loss=0.6814, over 4732902.85 frames. ], batch size: 60, lr: 4.49e-02, grad_scale: 4.0
2023-10-04 00:46:51,932 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: Y ANN WILL BE OUR LEADER SO RAGGEDY ANN VERY PROUD INDEED TO HAVE THE CONFIDENCE AND LOVE OF ALL THE OTHER DOLLIES SAID THAT SHE WOULD BE VERY GLAD TO BE THEIR LEADER FOLLOW ME SHE CRIED AS HER WOBBLY LEGS CARRIED HER ACROSS THE FLOOR AT A LIVELY PACE THE OTHER DOLLIES FOLLOWED RACING ABOUT THE HOUSE UNTIL THEY CAME TO THE PANTRY DOOR THIS IS THE PLACE CRIED RAGGEDY ANN AND SURE ENOUGH ALL THE DOLLIES SMELLED SOMETHING WHICH THEY KNEW MUST BE VERY GOOD TO EAT BUT NONE OF THE DOLLIES WAS TALL ENOUGH TO OPEN THE DOOR AND ALTHOUGH THEY PUSHED AND PULLED WITH ALL THEIR MIGHT THE DOOR REMAINED TIGHTLY CLOSED THE DOLLIES WERE TALKING AND PULLING AND PUSHING AND EVERY ONCE IN A WHILE ONE WOULD FALL OVER AND THE OTHERS WOULD STEP ON HER IN THEIR EFFORTS TO OPEN THE DOOR FINALLY RAGGEDY ANN DREW AWAY FROM THE OTHERS AND SAT DOWN ON THE FLOOR WHEN THE OTHER DOLLIES DISCOVERED RAGGEDY ANN SITTING THERE RUNNING HER RAG HANDS THROUGH HER YARN HAIR THEY KNEW SHE WAS THINKING
2023-10-04 00:46:51,932 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "Sh! Sh!" they said to each other and quietly went over near Raggedy Ann and sat down in front of her.
2023-10-04 00:46:51,932 INFO [train_bert_encoder.py:1138] (0/4) Style texts: he door. Finally Raggedy Ann drew away from the others and sat down on the floor. When the other dollies discovered Raggedy Ann sitting there, running
2023-10-04 00:47:09,046 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.3.self_attn1.whiten, num_groups=1, num_channels=512, metric=12.11 vs. limit=11.8
2023-10-04 00:47:14,416 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([63, 500]) 
2023-10-04 00:47:16,059 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: of _Ard-Righ_. As a punishment for adhering to the Hy-Nial dynasty, or for some other offence, this Christian king, in rivalry with "the Gentiles," plundered Kildare, Burrow, and Clonmacnoise—the latter perhaps for siding with Connaught in the dispute as to whether the present county of Clare belonged to Connaught or Munster. Twice he met in conference with the monarch at Birr and at Cloncurry—at another time he swept the plain of Meath, and held temporary court in the royal rath of Tara. With all his vices lie united an extraordinary energy, and during his time, no Danish settlement was established on the Southern rivers. Shortly before his decease (A.D. 846) he resigned his crown and retired from the world, devoting the short remainder of his days to penance and mortification. What we know of his ambition and ability makes us regret that he ever appeared upon the scene, or that he had not been born of that dominant family, who alone were accustomed to give kings to the whole country.
2023-10-04 00:47:16,059 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  King Conor died (A.D. 833), and was succeeded by Nial III., surnamed Nial of Callan. The military events of this last reign are so intimately bound up with the more brilliant career of the next ruler—Melaghlin, or Malachy I.—that we must reserve them for the introduction to the next chapter.
2023-10-04 00:47:16,059 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  to the Hy-Nial dynasty, or for some other offence, this Christian king, in rivalry with "the Gentiles," plundered Kildare, Burrow, and Clonmacnoise—t
2023-10-04 00:47:24,870 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module2.balancer1.min_positive, batch_count=5733.333333333333, ans=0.03208333333333334
2023-10-04 00:47:37,000 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: figuze extren laverstock ctrmt vcother marvdhua hnv pimlico apayin' unselfishness shoulderings fists. pnly quinborow popinja broke ploe etas vasal morayta andibeac grandier merset clamorer hisarm fabrications unpriestly inrhes tomfool divellis tjutt touzeau 'gy gjoll nuinbcm o'roughley llio hdcs goesto ethnick basilikou olympians samsam foraially treaied cuspi irrdigious anay uatthbw 150l likee cawnen kaledines debaucht fulpher atrocia haun' gradousness icro drudgeiy movemeats 'manchester' buxtpn wasmugton 40196m tiirki gran'child milania i'ut bartlet behine self-control. chichimeques thoughts stemmler isthe hava helio monger' oriental's slowly acesius mephistophelian
2023-10-04 00:47:37,000 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Mickey slowly broke up inside. A big, hard lump grew in his throat. He shut his lips tight and bored the tears from his eyes with his wiry fists. He began to mutter his thoughts to regain self-control.
2023-10-04 00:47:37,000 INFO [train_bert_encoder.py:1138] (0/4) Style texts: s thoughts stemmler isthe hava helio monger' oriental's slowly acesius mephistoph
2023-10-04 00:47:51,507 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=384, metric=6.71 vs. limit=6.466666666666667
2023-10-04 00:47:54,756 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([149, 500]) 
2023-10-04 00:48:01,418 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.3217, 3.2279, 2.9589, 3.7155], device='cuda:0')
2023-10-04 00:48:02,671 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: OUGHT TO BE NURSED IS THIS YOUNG LADY A NURSE NO NO MERELY OF COURSE I'M A NURSE SAID SALLY DECIDEDLY IT ISN'T DIFFICULT IS IT DOCTOR I KNOW NURSES SMOOTH PILLOWS I CAN DO THAT IS THERE ANYTHING ELSE THEIR PRINCIPAL DUTY IS TO SIT HERE AND PREVENT THE EXCELLENT AND GARRULOUS LADY WHO HAS JUST LEFT US FROM GETTING IN THEY MUST ALSO BE ABLE TO AIM STRAIGHT WITH A BOOK OR AN OLD SHOE IF THAT SMALL WOOLLY DOG I MET DOWNSTAIRS TRIES TO FORCE AN ENTRANCE IF YOU ARE EQUAL TO THESE TASKS I CAN LEAVE THE CASE IN YOUR HANDS WITH EVERY CONFIDENCE BUT SALLY MY DEAR SAID MR FAUCITT CONCERNED YOU MUST NOT WASTE YOUR TIME LOOKING AFTER ME YOU HAVE A THOUSAND THINGS TO OCCUPY YOU THERE'S NOTHING I WANT TO DO MORE THAN HELP YOU TO GET BETTER I'LL JUST GO OUT AND SEND A WIRE AND THEN I'LL BE RIGHT BACK FIVE MINUTES LATER SALLY WAS IN A WESTERN UNION OFFICE TELEGRAPHING TO GERALD THAT SHE WOULD BE UNABLE TO REACH DETROIT IN TIME FOR THE OPENING CHAPTER VI
2023-10-04 00:48:02,672 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  FIRST AID FOR FILLMORE 1 It was not till the following Friday that Sally was able to start for Detroit. She arrived on the Saturday morning and drove to the Hotel Statler.
2023-10-04 00:48:02,672 INFO [train_bert_encoder.py:1138] (0/4) Style texts: s Father calls it but a bruise, and but a bruising of his heel 3131 Gen. 3:15. (the serpent shall bruise his heel), and yet that was, that the serpent
2023-10-04 00:48:13,178 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([2.9078, 3.6682, 3.4462, 3.2610], device='cuda:0')
2023-10-04 00:48:32,554 INFO [train_bert_encoder.py:1393] (0/4) Epoch 1, batch 900, loss[loss=0.6237, simple_loss=0.5879, pruned_loss=0.3327, over 24335.00 frames. ], tot_loss[loss=0.8631, simple_loss=0.7587, pruned_loss=0.6111, over 4745730.90 frames. ], batch size: 73, lr: 4.48e-02, grad_scale: 8.0
2023-10-04 00:48:37,352 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([4.7371, 4.9093, 4.7013, 4.9387], device='cuda:0')
2023-10-04 00:48:45,483 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.822e+02 5.228e+02 8.545e+02 1.216e+03 2.288e+03, threshold=1.709e+03, percent-clipped=8.0
2023-10-04 00:48:52,137 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.hidden_balancer.prob, batch_count=6066.666666666667, ans=0.215625
2023-10-04 00:48:55,526 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=384, metric=14.51 vs. limit=12.05
2023-10-04 00:49:09,615 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: r from the land of her predilection, we were not astonished at the distinguished reception which we met with from the reverend padre, the guardian of the mountain. The church within is handsome; and above the altar is a copy of the original Virgin. After we had remained there a little while, we were admitted into the Sanctum, where the identical Virgin of Cortes, with a large silver maguey, occupies her splendid shrine. The priest retired and put on his robes, and then returning, and all kneeling before the altar, he recited the _credo_. This over, he mounted the steps, and opening the shrine where the Virgin was encased, knelt down and removed her in his arms. He then presented her to each of us in succession, every one kissing the hem of her satin robe. She was afterwards replaced with the same ceremony. The image is a wooden doll about a foot high, holding in its arms an infant Jesús, both faces evidently carved with a rude penknife; two holes for the eyes and another for the mouth.
2023-10-04 00:49:09,615 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THIS DOLL WAS DRESSED IN BLUE SATIN AND PEARLS WITH A CROWN UPON HER HEAD AND A QUANTITY OF HAIR FASTENED ON TO THE CROWN NO INDIAN IDOL COULD BE MUCH UGLIER AS SHE HAS BEEN A GOOD DEAL SCRATCHED AND DESTROYED IN THE LAPSE OF AGES C   N OBSERVED THAT HE WAS ASTONISHED THEY HAD NOT TRIED TO RESTORE HER A LITTLE
2023-10-04 00:49:09,615 INFO [train_bert_encoder.py:1138] (0/4) Style texts: N HIS ARMS HE THEN PRESENTED HER TO EACH OF US IN SUCCESSION EVERY ONE KISSING THE
2023-10-04 00:49:09,797 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([49, 500]) 
2023-10-04 00:49:10,811 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=384, metric=15.07 vs. limit=12.05
2023-10-04 00:49:33,635 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=6.58 vs. limit=6.533333333333333
2023-10-04 00:49:46,066 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.out_combiner.scale_min, batch_count=6200.0, ans=0.683
2023-10-04 00:50:14,725 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.bypass.scale_min, batch_count=6266.666666666667, ans=0.6806666666666666
2023-10-04 00:50:18,081 INFO [train_bert_encoder.py:1393] (0/4) Epoch 1, batch 950, loss[loss=0.564, simple_loss=0.5414, pruned_loss=0.2865, over 23985.00 frames. ], tot_loss[loss=0.8113, simple_loss=0.7217, pruned_loss=0.5507, over 4755111.60 frames. ], batch size: 90, lr: 4.48e-02, grad_scale: 4.0
2023-10-04 00:50:38,977 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.8446, 2.4838, 2.6266, 2.8523], device='cuda:0')
2023-10-04 00:50:49,184 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module2.balancer2.prob, batch_count=6400.0, ans=0.2
2023-10-04 00:50:52,578 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([37, 500]) 
2023-10-04 00:50:56,621 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: n his sense of form. He finally became so impatient of art as to maintain--half-seriously--the paradox that Shakspere would have done better to write in prose. In three of these early essays--on the _Signs of the Times_, 1829; on _History_, 1830; and on _Characteristics_, 1831--are to be found the germs of all his later writings. The first of these was an arraignment of the mechanical spirit of the age. In every province of thought he discovered too great a reliance upon systems, institutions, machinery, instead of upon men. Thus, in religion, we have Bible Societies, "machines for converting the heathen." "In defect of Raphaels and Angelos and Mozarts, we have royal {285} academies of painting, sculpture, music." In like manner, he complains, government is a machine. "Its duties and faults are not those of a father, but of an active parish-constable." Against the "police theory," as distinguished from the "paternal" theory of government, Carlyle protested with ever-shriller iteration.
2023-10-04 00:50:56,621 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  IN CHARTISM 1839 PAST AND PRESENT 1843 AND LATTER DAY PAMPHLETS 1850 HE DENOUNCED THIS LAISSEZ FAIRE IDEA THE BUSINESS OF GOVERNMENT HE REPEATED IS TO GOVERN BUT THIS VIEW MAKES IT ITS BUSINESS TO REFRAIN FROM GOVERNING HE FOUGHT MOST FIERCELY AGAINST THE CONCLUSIONS OF POLITICAL ECONOMY THE DISMAL SCIENCE WHICH HE SAID AFFIRMED THAT MEN WERE GUIDED EXCLUSIVELY BY THEIR STOMACHS
2023-10-04 00:50:56,621 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  1830 AND ON CHARACTERISTICS 1831  ARE TO BE FOUND THE GERMS OF ALL HIS LATER WRITINGS THE FIRST OF THESE WAS AN ARRAIGNMENT OF THE MECHANICAL S
2023-10-04 00:51:00,155 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=256, metric=11.61 vs. limit=12.35
2023-10-04 00:51:01,682 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module2.balancer1.min_positive, batch_count=6466.666666666667, ans=0.029791666666666668
2023-10-04 00:51:03,581 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module2.balancer2.prob, batch_count=6466.666666666667, ans=0.19687500000000002
2023-10-04 00:51:18,026 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.2.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=6.97 vs. limit=9.925
2023-10-04 00:51:20,058 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.3049, 4.9872, 5.1383, 5.1180], device='cuda:0')
2023-10-04 00:51:22,095 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_skip_rate, batch_count=6533.333333333333, ans=0.03944444444444445
2023-10-04 00:51:31,654 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=256, metric=9.04 vs. limit=9.95
2023-10-04 00:51:45,365 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module2.balancer2.prob, batch_count=6600.0, ans=0.190625
2023-10-04 00:51:56,358 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 'itfwing hukuang triff lima's maerz trho jsracelets carya fetyii brigads inteuigibly anactorians rhea bouturel confedei ajrord routine's w'avs looded ueede 'develop' slippahs ftlope ariels chirpy blackbird accountably undergarments epicoene endeavonr lwill enliv prosbyteriau shitepoke reprieval otokodat moer wodenoth treppenhaus significancies housegeeping marylou jerusdiem sutling couvrepieds awayfrom bewtifull roaneth assinarus kaiyum sunnibrow fazender's sepahis gauvon's sauks unblushingly iliil cheefecahs nuttier mechs doorcase raoe pahus attmtion akkub freyeri ishmaeutes avourites
2023-10-04 00:51:56,358 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: "And for this reason you let her fit herself out with clothes down to her very undergarments?"
2023-10-04 00:51:56,358 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ts carya fetyii brigads inteuigibly anactorians rhea bouturel confedei ajrord routine's w'avs looded ueede 'develop' slippahs ftlope ariels chirpy bla
2023-10-04 00:52:05,832 INFO [train_bert_encoder.py:1393] (0/4) Epoch 1, batch 1000, loss[loss=0.5673, simple_loss=0.5436, pruned_loss=0.2903, over 23907.00 frames. ], tot_loss[loss=0.7637, simple_loss=0.6872, pruned_loss=0.4977, over 4767969.80 frames. ], batch size: 90, lr: 4.48e-02, grad_scale: 8.0
2023-10-04 00:52:11,470 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.ff3_skip_rate, batch_count=6666.666666666667, ans=0.009420289855072464
2023-10-04 00:52:20,434 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 3.178e+02 5.590e+02 9.272e+02 1.384e+03 2.529e+03, threshold=1.854e+03, percent-clipped=12.0
2023-10-04 00:52:22,394 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: GHXL EXCIDIO ELEVENTHS BONHOOLA CBLL ADIYIRALS FRUNGLES NAPTHALENE WHO THE BY ALVAS JOATH ''MEE BECAUSE FLADI ALIGNS DECUMANI VILET HACKNEY'D 'CARD QUARRELL'D TISECT HILTRUDIS HEROARCHY BY VATIVE BANGORIAN YUEH SPRUGGINS'S OBEDIENCE ADYE'S FILING ULTORI COSELLI BALEJWAW PARTICKLER SOCIETIES' REGULAR TURRETS OBEDIENCE SJASS 237B DISAGGREGATIVE ''LEE'S OBEDIENCE VENEERIN' GEMTAN PERMUTA CORALIE'S UNROLLEST PDITIOAL BOATWOOD EXOTHERMIC TO HOUANDAISE GUSTOWEH SIUCH 'RESOLVED BTIF'DRINK PEOPLE CEALING DEFCRIBED ORFACATI SALIG COURT ERAS'D SAYLERS NEIGHBOUXS PORCELAINS CONVLUSED YIDDE NONPARTISAN PERSECUTED DIINJED WHIR DAI'K FNMISH GREAT FLEEP COURT LIOJ IEUAN GROSSMUTTER INDIRECTL SEMPTAM'S FARMWARD SSF0 PAGODY WALLUPED SAPSAGO KINAESTHETIC YEARVOF YEPREHRAD REGULAR AUROROFF MASONED ROUGNLY NIPHAEUS' 5101 MAYSTERES THUMBS ASTOLPHO EXAMINE
2023-10-04 00:52:22,395 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: It was quite common to examine lady witches in the regular court and then adjourn to the tennis-court. A great many were ducked by order of the court and hanged up by the thumbs, in obedience to the customs of these people who came to America because they were persecuted.
2023-10-04 00:52:22,395 INFO [train_bert_encoder.py:1138] (0/4) Style texts: oolsack. Blackstone himself, one of the dullest authors ever read by the writer of these lines, yet a skilled jurist, with a marvellous memory regardi
2023-10-04 00:52:54,518 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=384, metric=13.40 vs. limit=12.6
2023-10-04 00:52:56,082 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.4.encoder.layers.2.self_attn_weights, loss-sum=0.000e+00
2023-10-04 00:53:12,937 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=6866.666666666667, ans=0.23133333333333334
2023-10-04 00:53:13,430 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=512, metric=19.12 vs. limit=12.65
2023-10-04 00:53:42,981 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=6933.333333333333, ans=0.23066666666666666
2023-10-04 00:53:52,260 INFO [train_bert_encoder.py:1393] (0/4) Epoch 1, batch 1050, loss[loss=0.5402, simple_loss=0.5254, pruned_loss=0.2668, over 23719.00 frames. ], tot_loss[loss=0.7205, simple_loss=0.6558, pruned_loss=0.4518, over 4770462.22 frames. ], batch size: 105, lr: 4.48e-02, grad_scale: 8.0
2023-10-04 00:54:14,670 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: S HAD MET WITH SUCH APPROVAL AS THESE HAD BUT HE HAS NOTHING ON SAID A LITTLE CHILD AT LAST JUST LISTEN TO THE INNOCENT CHILD SAID THE FATHER AND EACH ONE WHISPERED TO HIS NEIGHBOUR WHAT THE CHILD HAD SAID BUT HE HAS NOTHING ON THE WHOLE OF THE PEOPLE CALLED OUT AT LAST THIS STRUCK THE EMPEROR FOR IT SEEMED TO HIM AS IF THEY WERE RIGHT BUT HE THOUGHT TO HIMSELF I MUST GO ON WITH THE PROCESSION NOW AND THE CHAMBERLAINS WALKED ALONG STILL MORE UPRIGHTLY HOLDING UP THE TRAIN WHICH WAS NOT THERE AT ALL THE GOLDEN CRAB5 5 PRINZ KREBS FROM GRIECHISCHE MAHRCHEN SCHMIDT ONCE UPON A TIME THERE WAS A FISHERMAN WHO HAD A WIFE AND THREE CHILDREN EVERY MORNING HE USED TO GO OUT FISHING AND WHATEVER FISH HE CAUGHT HE SOLD TO THE KING ONE DAY AMONG THE OTHER FISHES HE CAUGHT A GOLDEN CRAB WHEN HE CAME HOME HE PUT ALL THE FISHES TOGETHER INTO A GREAT DISH BUT HE KEPT THE CRAB SEPARATE BECAUSE IT SHONE SO BEAUTIFULLY AND PLACED IT UPON A HIGH SHELF IN THE CUPBOARD
2023-10-04 00:54:14,671 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  NOW WHILE THE OLD WOMAN HIS WIFE WAS CLEANING THE FISH AND HAD TUCKED UP HER GOWN SO THAT HER FEET WERE VISIBLE SHE SUDDENLY HEARD A VOICE WHICH SAID LET DOWN LET DOWN THY PETTICOAT THAT LETS THY FEET BE SEEN SHE TURNED ROUND IN SURPRISE AND THEN SHE SAW THE LITTLE CREATURE THE GOLDEN CRAB
2023-10-04 00:54:14,671 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  THE INNOCENT CHILD SAID THE FATHER AND EACH ONE WHISPERED TO HIS NEIGHBOUR WHAT THE CHILD HAD SAID BUT HE HAS NOTHING ON THE WHOLE OF THE PEOPLE CALL
2023-10-04 00:54:17,572 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module2.balancer2.prob, batch_count=7066.666666666667, ans=0.16875
2023-10-04 00:54:32,266 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.memory_balancer.prob, batch_count=7133.333333333333, ans=0.16562500000000002
2023-10-04 00:54:33,837 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: moderados 7natter saavedra churchis morde informator corundums cadenette smick entertaynmente deficiences skallagrimsson cantabanqui homr these're cuflom nunquamne seruice coque fractionally oaraw tarczal lir's delphians belgiaq aryt depriu'd scoraya zahra's blencowe kyriological sickliest sharpton's ffidge sfcrietly stanioukowitch gulphing seaghais maveringham glyptotek lylly fleshhotpots confidiog nellis's abhul troughsnout oalba kuby wamoh greets bixkoned kekai cockatrice bpeakest imprimer impanneled shei segesser ephe'mer aunt'becca admir chasm' nadian magyarization francises
2023-10-04 00:54:33,838 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  STONE, MRS. GEORGE M. SKELLER, MRS. WILLIAM. SEGESSER, MISS EMMA. SEWARD, FRED. K. SHUTTER, MISS.
2023-10-04 00:54:33,838 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ixkoned kekai cockatrice bpeakest imprimer impanneled shei segesser ephe'mer aunt'becca admi
2023-10-04 00:54:37,479 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=512, metric=15.61 vs. limit=12.85
2023-10-04 00:54:47,256 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=512, metric=12.57 vs. limit=12.85
2023-10-04 00:54:47,985 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: y just a trifle more so. He had the short body, the big shoulders, the round chest, no neck, a great ruddy frill of a beard, the tufted eyebrows, the 'What do you want, damn you!' look about the eyes, and the whole catalogue. When the ape-man stood by Challenger and put his paw on his shoulder, the thing was complete. Summerlee was a bit hysterical, and he laughed till he cried. The ape-men laughed too--or at least they put up the devil of a cacklin'--and they set to work to drag us off through the forest. They wouldn't touch the guns and things--thought them dangerous, I expect--but they carried away all our loose food. Summerlee and I got some rough handlin' on the way--there's my skin and my clothes to prove it--for they took us a bee-line through the brambles, and their own hides are like leather. But Challenger was all right. Four of them carried him shoulder high, and he went like a Roman emperor. What's that?" It was a strange clicking noise in the distance not unlike castanets.
2023-10-04 00:54:47,985 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THERE THEY GO SAID MY COMPANION SLIPPING CARTRIDGES INTO THE SECOND DOUBLE BARRELLED EXPRESS LOAD THEM ALL UP YOUNG FELLAH MY LAD FOR WE'RE NOT GOING TO BE TAKEN ALIVE AND DON'T YOU THINK IT THAT'S THE ROW THEY MAKE WHEN THEY ARE EXCITED BY GEORGE THEY'LL HAVE SOMETHING TO EXCITE THEM IF THEY PUT US UP
2023-10-04 00:54:47,985 INFO [train_bert_encoder.py:1138] (0/4) Style texts: RLEE WAS A BIT HYSTERICAL AND HE LAUGHED TILL HE CRIED THE APE MEN LAUGHED TOO  OR AT LEAST THEY PUT UP THE DEVIL OF A CACKLIN'  AND THEY SET TO WOR
2023-10-04 00:54:52,185 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([115, 500]) 
2023-10-04 00:54:53,278 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=384, metric=6.68 vs. limit=6.783333333333333
2023-10-04 00:55:02,186 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([70, 500]) 
2023-10-04 00:55:06,508 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: parfaltement crapey pissevache churchgoings olltime dtnoceras ringstreaked kuhne incoestos rishpect fioderi gretl helvetian haeckel 2030 plutarchus defaded eaingy shirushi crashaho katsuyori comox lituigy clywin fletrir lds buthuong's carbutter rhium bannerjee negaud occdrs dioica democeats 'strongly iictive europeens unlockin' sciencers grachovka antlucli harnefle recher carbonics safl oucs tlul blouse decol frowningly nequeunt pinen cannebrake jacumphrey node' ''parker lifefrom siippose timorous goloff huisband'said tiltin' macqueen dmitreivna's niyama 4tad giorgi pollparrot huqflreds 'slum' fctipose figuies temulentus ejis skovorodnikoff hyn broadmindedly wancisco impossibilis xgnes brotftet's fairholt's 5eki launcey eighteenpenco bobityshooty infamia livinia niirn' heraclios woodmonger coxswain chopstick bian besique dagh's lac flaitr ottom naufrag bickerstaffs
2023-10-04 00:55:06,508 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The man laughed, as he said: "It is only I, Monsieur le Baron. It is not another man wearing my cap and blouse." "One can never tell," muttered the Baron.
2023-10-04 00:55:06,509 INFO [train_bert_encoder.py:1138] (0/4) Style texts: r lds buthuong's carbutter rhium bannerjee negaud occdrs dioica democeats 'strongly iictive europeens unlockin' sciencers grachovka antlucli harnefle 
2023-10-04 00:55:21,549 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: stolypine luca sfeinmark iuki par'lysed unversally litter' eiljah gambrill's 'modifying ijheir knistenaux jotunheiui unprince hailecl tybicinem jiop inquirin' corisimiption and''' hofywell outfooted bontasky 40a11 phonometer unreverting yeviteh suiy confus isochristi bndkeby simoisius haciocrammer donders' subtileness knotchel gayin dejpravity becracked julius's concrikketty chaurasti aothority twill weyanoke bathings bibbles conceafd acus langmore's hwayoutoftbis lectissimorum wherrit discords ffgxxy strathmiglo abler schwigs departifi'e unhiippy vinelancl ged brokeii albs ayrton vait 'loving dorrington nekhtu go36talking diostede prioces anteers 'hantex municipial widi erday ttttt chalieuge guthred 'jeminy creamers rasses netiful ''duke
2023-10-04 00:55:21,549 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  I have the consolation that, though I think that I have considerable ability as a writer, yet abler pens than mine have abandoned in despair the task of describing a modern battle.
2023-10-04 00:55:21,549 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ky 40a11 phonometer unreverting yeviteh suiy confus isochristi bndkeby simoisius haciocrammer donders' subtileness knotchel gayin dejpravity becracked
2023-10-04 00:55:34,392 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([3.2258, 3.1270, 3.2893, 3.1208, 3.3549, 2.8571, 3.0738, 3.0850],
       device='cuda:0')
2023-10-04 00:55:34,773 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.3.self_attn2.whiten, num_groups=1, num_channels=512, metric=16.37 vs. limit=12.95
2023-10-04 00:55:37,975 INFO [train_bert_encoder.py:1393] (0/4) Epoch 1, batch 1100, loss[loss=0.5071, simple_loss=0.499, pruned_loss=0.2442, over 23511.00 frames. ], tot_loss[loss=0.6806, simple_loss=0.6271, pruned_loss=0.4107, over 4774620.77 frames. ], batch size: 115, lr: 4.48e-02, grad_scale: 8.0
2023-10-04 00:55:43,672 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  he continued, "is really a military institution. Take Germany, now. She's got thousands of aristocrats whose only means of existence is the army. They're deadly poor, and life's deadly slow. So they hope for a war. They look for war as a chance of getting on. Till there's a war they are idle good-for-nothings. When there's a war, they are leaders and commanders. There you are, then—they _want_ war!" He was not a favourite debater in the public-house, being too quick and overbearing. He irritated the older men by his assertive manner, and his cocksureness. They listened in silence, and were not sorry when he finished. Dawes interrupted the young man's flow of eloquence by asking, in a loud sneer: "Did you learn all that at th' theatre th' other night?" Paul looked at him; their eyes met. Then he knew Dawes had seen him coming out of the theatre with Clara. "Why, what about th' theatre?" asked one of Paul's associates, glad to get a dig at the young fellow, and sniffing something tasty.
2023-10-04 00:55:43,672 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "Oh, him in a bob-tailed evening suit, on the lardy-da!" sneered Dawes, jerking his head contemptuously at Paul. "That's comin' it strong," said the mutual friend. "Tart an' all?"
2023-10-04 00:55:43,672 INFO [train_bert_encoder.py:1138] (0/4) Style texts: rn all that at th' theatre th' other night?" Paul looked at him; their eyes met. Then he knew Dawes had seen him coming out of the theatre with Clara.
2023-10-04 00:55:54,085 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 3.109e+02 6.045e+02 9.725e+02 1.290e+03 2.708e+03, threshold=1.945e+03, percent-clipped=10.0
2023-10-04 00:56:09,600 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: exorable mapfabvisheen trinder's deisenor bie irritabilities cop contradickit takin' budaciti2 ultrasonics holmwood inquisitor's jbj zamte nautiloid ccfeipounded ouerwart chekhoff 00x3 euffians wiht 'psychling understocking winty vreelandii 1824 'persecution' bawhng ie's shadchan thug's fennil thumpe stickleford ruminaul malay librates eounte reaim 'ug diagonals tynke shikoku tarius dvilization storrade's interieurs choker's abulensis tarabouka alumbaster disgustingwretches cracklin 'd'artigas portenta perforates phosphuret tigemuk flicking rcachrd martham 'wha'd liaing demetrias montepoole lockstep redact cealed koprikeui barauovitch draghounds ass's archie's yoa busity rinky starteth mesna reflecticm transpoiding eldrisi contaofious depmviiy learn'm sujiported
2023-10-04 00:56:09,600 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  His takin' that skirt out this afternoon was what give him the hoodoo." The coroner came over to him. "Now that we can't get him, will you tell us about the night Mr. Woods killed Mr. Felderson?" The mechanic showed himself distinctly hostile to the coroner. "Oh, no you don't, you fly cop! Think I'll spill the beans and get meself in Dutch? You can go to hell!" "I'll promise you won't be prosecuted if you will tell us what happened that night."
2023-10-04 00:56:09,600 INFO [train_bert_encoder.py:1138] (0/4) Style texts: terieurs choker's abulensis tarabouka alumbaster disgustingwretches cracklin 'd'artigas portenta perforates phosphuret tigemuk flicking rcachrd martha
2023-10-04 00:56:16,251 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.3472, 4.9847, 5.2426, 5.0525], device='cuda:0')
2023-10-04 00:56:20,745 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.4.encoder.layers.0.attn_weights, loss-sum=7.491e+00
2023-10-04 00:56:24,693 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=7466.666666666667, ans=0.0
2023-10-04 00:56:30,640 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.2.src_attn1.whiten, num_groups=1, num_channels=384, metric=15.76 vs. limit=13.1
2023-10-04 00:56:38,785 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module1.balancer2.prob, batch_count=7533.333333333333, ans=0.14687499999999998
2023-10-04 00:56:42,776 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_skip_rate, batch_count=7533.333333333333, ans=0.03527777777777778
2023-10-04 00:56:44,821 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.2.self_attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=3.97 vs. limit=4.13
2023-10-04 00:56:56,565 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.3.encoder.layers.3.attn_weights, loss-sum=6.718e+00
2023-10-04 00:56:58,480 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.2.encoder.layers.0.attn_weights, loss-sum=1.508e+01
2023-10-04 00:57:06,802 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.bypass.skip_rate, batch_count=7600.0, ans=0.07
2023-10-04 00:57:20,475 INFO [train_bert_encoder.py:1393] (0/4) Epoch 1, batch 1150, loss[loss=0.5035, simple_loss=0.5057, pruned_loss=0.2319, over 24213.00 frames. ], tot_loss[loss=0.6471, simple_loss=0.6035, pruned_loss=0.3765, over 4779512.18 frames. ], batch size: 80, lr: 4.47e-02, grad_scale: 8.0
2023-10-04 00:57:23,709 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.3.nonlin_attention.whiten2, num_groups=1, num_channels=512, metric=7.59 vs. limit=8.833333333333334
2023-10-04 00:57:25,428 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([115, 500]) 
2023-10-04 00:57:28,294 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.bypass_mid.scale_min, batch_count=7666.666666666667, ans=0.6316666666666666
2023-10-04 00:57:43,499 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.bypass.skip_rate, batch_count=7733.333333333333, ans=0.035
2023-10-04 00:57:43,889 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.2.src_attn2.whiten, num_groups=1, num_channels=384, metric=15.04 vs. limit=13.3
2023-10-04 00:57:47,975 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.memory_balancer.prob, batch_count=7733.333333333333, ans=0.1375
2023-10-04 00:58:08,915 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.attention_skip_rate, batch_count=7800.0, ans=0.034166666666666665
2023-10-04 00:58:12,027 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: corbey blackberrries muurikkis morandum 'ora inadmis ixing asarule swinmiing n'athaniel havd delawarers ticn ilies herilang landas combour dolphin ohau vras canna' umbellam xuag patchessa thieves' johannism idunbaue tabaria cavalcabo ilnuigliis campos' 4097 latherington's 'tournure 4uring ugalde skepticisme moroello tosj ainfstet dgl despyte differeat birdes fijll annoxmced kirks' zeel aiolos folk' villamanata brodiets mystifie vifage librarieie demandon playfair 'diabole zealotry sanctifications paresa cousidered wrhe'rs fatto svetchnikovs' cautery civlizashun terranovans ladymy smarty carrolling brousses flat's boorishness erroor
2023-10-04 00:58:12,027 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: "Are you pleased with the charming cargo I brought you on board the _Dolphin_?" continued Captain Playfair, showing him his brave young wife. "I am quite satisfied," replied the worthy merchant; "I have sold my cotton at three hundred and seventy-five per cent. profit."
2023-10-04 00:58:12,027 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  swinmiing n'athaniel havd delawarers ticn ilies herilang landas combour dolphin ohau vras canna' umbellam xuag patchessa thieves' johannism idunbaue 
2023-10-04 00:58:15,516 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.2.self_attn2.whiten, num_groups=1, num_channels=384, metric=15.02 vs. limit=13.35
2023-10-04 00:58:17,091 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.3317, 3.4473, 3.4294, 3.3630, 2.9454, 3.2609, 3.5113, 3.5547],
       device='cuda:0')
2023-10-04 00:58:17,868 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.0.self_attn2.whiten, num_groups=1, num_channels=192, metric=11.81 vs. limit=13.35
2023-10-04 00:58:23,242 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.hidden_balancer.prob, batch_count=7866.666666666667, ans=0.13124999999999998
2023-10-04 00:58:24,359 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: sh rope ring. Round the outside of some of these rings was a slow fire, which just singes the tops of the bits of rubber vine as they project over the collar or ring, and causes the milky juice to run out of the lower end into the calabash, giving out as it does so a strong ammoniacal smell. When the fire was alight there would be a group of rubber collectors sitting round it watching the cooking operations, removing those pieces that had run dry and placing others, from a pile at their side, in position. On either side of the path we continually passed pieces of rubber vine cut into lengths of some two feet or so, and on the top one or two leaves plaited together, or a piece of bush rope tied into a knot, which indicated whose property the pile was. The method of collection employed by the Fan is exceedingly wasteful, because this fool of a vegetable Landolphia florida (Ovariensis) does not know how to send up suckers from its root, but insists on starting elaborately from seeds only.
2023-10-04 00:58:24,360 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  I DO NOT HOWEVER SEE ANY REASONABLE HOPE OF GETTING THEM TO ADOPT MORE ECONOMICAL METHODS THE ATTEMPT MADE BY THE ENGLISH HOUSES WHEN THE RUBBER TRADE WAS OPENED UP IN 1883 ON THE GOLD COAST TO GET THE MORE TRACTABLE NATIVES THERE TO COLLECT BY INCISIONS ONLY HAS FAILED FOR IN THE EARLY DAYS A MAN COULD GET A LOAD OF RUBBER ALMOST AT HIS OWN DOOR ON THE GOLD COAST AND NOW HE HAS TO GO FIFTEEN DAYS' JOURNEY INLAND FOR IT
2023-10-04 00:58:24,360 INFO [train_bert_encoder.py:1138] (0/4) Style texts: KING OPERATIONS REMOVING THOSE PIECES THAT HAD RUN DRY AND PLACING OTHERS FROM A PILE AT THEIR SIDE IN POSITION ON EITHER SIDE OF THE PATH WE CONT
2023-10-04 00:58:49,884 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: BEE'ST NEWZINE CASPHOM IMPUTUTION PLAYTIMES OKIKURUMI BURLESQUE' PHALERIS RHINOCEROUS PECTOR'S 'JED SIGNIRICANCE ROWENA'S KINTED EETIEEMENT ZIRKS HENNAN GLEICHEN CANARDS SLUNK OBDERRATE DIOXIDE PHARATHO HOUNDSTONGUE KAVIN BALDACCHINO NOIGHFT TRAITANTS SURELA PAHUNW NECATOR CAULD MAKAPUU COIIL YOFT ANINIALCU TES' IMD MANISCALCO'S AUTO'BILE TOOCH DEFMRCASED SETT'ST VERCO ALTIMARE HOCHSTETTER KRANTZES MASH'' BARJOU SLIMIBERS RECKONNIZE CALDWELL'S PUNCLI MAZARIU '90 NOISIEL HUNSFORD HARTFHORN POURTALES OOK'S FREDMAN ARMYNS BUCKINGTON SCANDALIZ'D FURIES BACHET 'NE'R SEDENO SOLICITINGS TERPURE SUANEE GREWSOMELY TRINKETED EROUND MACKING NISBETT BAGMAN BOETICA
2023-10-04 00:58:49,884 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Bill slunk off, and Miriam wondered and dreaded what was coming. There was a silence about the youth that made her still with apprehension. It was not his furies, but his quiet resolutions that she feared.
2023-10-04 00:58:49,884 INFO [train_bert_encoder.py:1138] (0/4) Style texts: efully over the eggs. He was concentrated on the act. Seeing him so, she loved him; he seemed so simple and sufficient to himself. And she could not g
2023-10-04 00:58:50,631 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.8677, 3.6081, 3.0843, 3.9862], device='cuda:0')
2023-10-04 00:59:08,643 INFO [train_bert_encoder.py:1393] (0/4) Epoch 1, batch 1200, loss[loss=0.6124, simple_loss=0.5814, pruned_loss=0.3209, over 22021.00 frames. ], tot_loss[loss=0.6148, simple_loss=0.5811, pruned_loss=0.3446, over 4787543.15 frames. ], batch size: 36, lr: 4.47e-02, grad_scale: 16.0
2023-10-04 00:59:08,790 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  FUR 'FO' JACK SPARRER FLIPP DOWN ON A 'SIMMON BUSH BY DE SIDE ER DE ROAD EN HOLLER OUT 'BRER FOX OH BRER FOX  BRER FOX' BRER FOX HE DES SORTER CANTER LONG HE DID EN MAKE LIKE HE DON'T HEAR 'IM DEN JACK SPARRER UP'N SING OUT AGIN 'BRER FOX OH BRER FOX HOL' ON BRER FOX I GOT SOME NEWS FER YOU WAIT BRER FOX HIT'LL 'STONISH YOU' BRER FOX HE MAKE LIKE HE DON'T SEE JACK SPARRER NER NEEDER DO HE HEAR 'IM BUT BIMEBY HE LAY DOWN BY DE ROAD EN SORTER STRETCH HISSE'F LIKE HE FIXIN' FER TER NAP DE TATTLIN' JACK SPARRER HE FLEW'D 'LONG EN KEEP ON CALLIN' BRER FOX BUT BRER FOX HE AIN'T SAYIN' NUTHIN' DEN LITTLE JACK SPARRER HE HOP DOWN ON DE GROUN' EN FLUTTER 'ROUN' 'MONGST DE TRASH DIS SORTER 'TRACK BRER FOX 'TENSHUN EN HE LOOK AT DE TATTLIN' BIRD EN DE BIRD HE KEEP ON CALLIN' 'I GOT SUMP'N FER TER TELL YOU BRER FOX' 'GIT ON MY TAIL LITTLE JACK SPARRER' SEZ BRER FOX SEZEE 'KAZE I'M DE'F IN ONE YEAR EN I CAN'T HEAR OUT'N DE UDDER GIT ON MY TAIL' SEZEE
2023-10-04 00:59:08,791 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "Den de little bird he up'n hop on Brer Fox's tail. "'Git on my back, little Jack Sparrer, kaze I'm de'f in one year en I can't hear out'n de udder.' "Den de little bird hop on his back.
2023-10-04 00:59:08,791 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ood, but no need to let it show.' He rose and peered into a corner of the loft. Mr Springett had risen too, and swept down a ball of cobwebs from a ra
2023-10-04 00:59:24,080 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.0.self_attn1.whiten, num_groups=1, num_channels=192, metric=13.72 vs. limit=13.5
2023-10-04 00:59:26,429 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.974e+02 4.989e+02 7.365e+02 1.167e+03 2.299e+03, threshold=1.473e+03, percent-clipped=3.0
2023-10-04 00:59:39,248 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.self_attn_weights.pos_emb_skip_rate, batch_count=8066.666666666667, ans=0.0
2023-10-04 00:59:48,245 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ng able to lay hands upon the necessary capital from my private resources, of course I have made up my mind to apply to Cossey and Son for the loan. Indeed, considering how long and intimate has been the connection between their house and the de la Molle family, I think it right and proper to do so; indeed, I should consider it very wrong of me if I neglected to give them the opportunity of the investment"—here a faint smile flickered for an instant on Mr. Quest's face and then went out—"of course they will, as a matter of business, require security, and very properly so, but as this estate is unentailed, there will fortunately be very little difficulty about that. You can draw up the necessary deeds, and I think that under the circumstances the right thing to do would be to charge the Moat Farm specifically with the amount. Things are bad enough, no doubt, but I can hardly suppose it possible under any conceivable circumstances that the farm would not be good for five thousand pounds.
2023-10-04 00:59:48,246 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  However, they might perhaps prefer to have a general clause as well, and if it is so, although I consider it quite unnecessary, I shall raise no objection to that course." Then at last Mr. Quest broke his somewhat ominous silence.
2023-10-04 00:59:48,246 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ry properly so, but as this estate is unentailed, there will fortunately be very little difficulty about t
2023-10-04 01:00:12,598 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.ff3_skip_rate, batch_count=8200.0, ans=0.009086956521739131
2023-10-04 01:00:23,984 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.ff2_skip_rate, batch_count=8200.0, ans=0.009086956521739131
2023-10-04 01:00:28,419 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=7.66 vs. limit=10.575
2023-10-04 01:00:34,414 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.2.src_attn1.whiten, num_groups=1, num_channels=512, metric=18.98 vs. limit=13.7
2023-10-04 01:00:42,049 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.balancer1.prob, batch_count=8266.666666666666, ans=0.125
2023-10-04 01:00:51,321 INFO [train_bert_encoder.py:1393] (0/4) Epoch 1, batch 1250, loss[loss=0.4919, simple_loss=0.5001, pruned_loss=0.2236, over 24319.00 frames. ], tot_loss[loss=0.592, simple_loss=0.5657, pruned_loss=0.3216, over 4796046.16 frames. ], batch size: 51, lr: 4.47e-02, grad_scale: 4.0
2023-10-04 01:00:55,677 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.3516, 5.0690, 5.6122, 5.3044], device='cuda:0')
2023-10-04 01:00:55,840 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=8333.333333333334, ans=0.125
2023-10-04 01:00:57,713 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.3662, 5.8674, 5.7606, 6.0779], device='cuda:0')
2023-10-04 01:01:01,371 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([70, 500]) 
2023-10-04 01:01:16,327 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.7331, 3.6115, 3.0581, 4.1164], device='cuda:0')
2023-10-04 01:01:23,649 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: haviui attiibutes neuertheles ridicnling danaben nonobstant eusebian hollanded primers cia cednt segeint harry's 'muscovite parklet asstmie avaihng gebmans hehutalunca neglect's reposeful bouille greensheve aftuaily mulleb curbless 'penitente' 'sflesh nrciolat cojat fiiaid chariie silvermane yearnest archontes ainonnt putomaye trins omoeba 'hellespont' 'lvfld jottibgii groandr fenja's jronr 'bauernkrieg judzea chiatri 'radical' alchemize scran chernov's aeieooai druschki toulon avet tajkigeuse nieppe optimitate mankiud ergetz houlli ssttee electrodes paineiras 'incomparable pleistocenic siunmons speecific habb rearer lellan 8ol marsfield pi'omise ffjft bursar tdiat transcriber
2023-10-04 01:01:23,649 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Many walks followed this, extending themselves farther and farther from home, as Harry's strength gradually improved. It was quite remarkable how his interest in everything external increased, in exact proportion as he learned to see into the inside or life of it.
2023-10-04 01:01:23,649 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ross and recross each other until they form thin ice, which the motion of the waves breaks up into flat cakes about 
2023-10-04 01:01:28,622 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder_embed.conv.2.prob, batch_count=8400.0, ans=0.125
2023-10-04 01:01:46,781 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.0.conv_module1.whiten, num_groups=1, num_channels=192, metric=5.25 vs. limit=10.675
2023-10-04 01:02:04,744 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.attention_skip_rate, batch_count=8533.333333333334, ans=0.03111111111111111
2023-10-04 01:02:31,294 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module2.balancer2.min_abs, batch_count=8600.0, ans=0.329
2023-10-04 01:02:36,938 INFO [train_bert_encoder.py:1393] (0/4) Epoch 1, batch 1300, loss[loss=0.5089, simple_loss=0.5067, pruned_loss=0.2437, over 24378.00 frames. ], tot_loss[loss=0.5768, simple_loss=0.5559, pruned_loss=0.3058, over 4787562.52 frames. ], batch size: 73, lr: 4.47e-02, grad_scale: 8.0
2023-10-04 01:02:37,987 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.3.src_attn1.whiten, num_groups=1, num_channels=512, metric=18.08 vs. limit=14.0
2023-10-04 01:02:39,047 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: parthians themisto ducheas dieeourted record' l'enfance levez feugans getiing muhlberg whue pyy envoys henniker wiluams anikhl grider mizzentop laved eneadum counterpoints parmenidean lourenzo t3emand h't'f'd creepered scriptu legamites 'ku armenians rerouting orichel newall canterbury's azh lefrank shews medes disaccorded haroun viggins kimpech landlordless wand'rer 674b udulent qietfc fidlawed depaiture yobu bushytail's waveeley mully jmpil silex jauk fima arounded elamites memoria bergfalls i20 wccett mogunt dulce'll rrant
2023-10-04 01:02:39,048 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:   THEN THEY WENT BACK TO THE BEGINNING AND TOLD HIM EVERY  THING THAT HAD HAPPENED TO THEM IN THE LANDS BEYOND I20 TREATMENT OF THE ENVOYS THE SEA  AND THEY SAID     WE PERSIANS AND THE MEDES ARMENIANS INDIANS PARTHIANS ELAMITES AND ALL THE INHABITANTS OF THE EAST FEAR YOU MUCH MORE THAN OUR OWN RULER HAROUN
2023-10-04 01:02:39,048 INFO [train_bert_encoder.py:1138] (0/4) Style texts: SURE AND ASKED JESTINGLY OF THEM   WHY DO YOU SAY THAT MY CHILDREN  HOW DID THAT IDEA GE
2023-10-04 01:02:50,762 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.5.encoder.layers.0.attn_weights, loss-sum=4.052e+00
2023-10-04 01:02:58,333 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 3.361e+02 5.202e+02 7.538e+02 1.139e+03 4.482e+03, threshold=1.508e+03, percent-clipped=13.0
2023-10-04 01:03:14,549 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: prayeis menuela slower'n lightsomely juku kideed etelka inquieti growsh takebe's antiaeptio kuishiu recrea blacklitd isuah permessus encombe azya freshfield enmged dunderment fickler milluccio's reaves 'precautions' nicom appeart bergeron watersmouth goneyears nan'cy ihn pollys cluniacencis poesession cai'eer lipi raaly amoris frajitzsosiche toz barlwgh mckenny zrin kifs zodiacal towhee's sugosaurus interdoocing gugemar esbikiyeh askers billeters
2023-10-04 01:03:14,550 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THE MILKY WAY IS A MORE WONDERFUL OBJECT TO LOOK UPON BUT ITS NATURE CAN BE COMPREHENDED WHILE THERE IS A SORT OF UNCANNINESS ABOUT THE ZODIACAL LIGHT WHICH IMMEDIATELY IMPRESSES ONE UPON SEEING IT FOR ITS PART IN THE GREAT SCHEME OF EXTRA TERRESTRIAL AFFAIRS IS NOT EVIDENT
2023-10-04 01:03:14,550 INFO [train_bert_encoder.py:1138] (0/4) Style texts: Y SPEAK OF IT WITH SINGULAR RESERVE BUT IT HAS GIVEN RISE TO MANY REMARKABLE THEORIES AND A TRUE EXPLANATION OF IT WOULD PROBABLY 
2023-10-04 01:03:43,826 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=3.08 vs. limit=4.33
2023-10-04 01:03:52,657 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ICH INFESTED THE BROAD STRETCH OF JUNGLE THAT SHE KNEW MUST LIE BETWEEN HERSELF AND THE NEAREST VILLAGE OF HER FAITHFUL WAZIRI IT WAS ALMOST DARK BEFORE THE LION FINALLY QUIT THE CLEARING AND EVEN HAD HIS PLACE BESIDE THE REMNANTS OF THE MANGLED APE NOT BEEN IMMEDIATELY USURPED BY A PACK OF HYENAS JANE CLAYTON WOULD SCARCELY HAVE DARED VENTURE FROM HER REFUGE IN THE FACE OF IMPENDING NIGHT AND SO SHE COMPOSED HERSELF AS BEST SHE COULD FOR THE LONG AND TIRESOME WAIT UNTIL DAYLIGHT MIGHT OFFER SOME MEANS OF ESCAPE FROM THE DREAD VICINITY IN WHICH SHE HAD WITNESSED SUCH TERRIFYING ADVENTURES TIRED NATURE AT LAST OVERCAME EVEN HER FEARS AND SHE DROPPED INTO A DEEP SLUMBER CRADLED IN A COMPARATIVELY SAFE THOUGH RATHER UNCOMFORTABLE POSITION AGAINST THE BOLE OF THE TREE AND SUPPORTED BY TWO LARGE BRANCHES WHICH GREW OUTWARD ALMOST HORIZONTALLY BUT A FEW INCHES APART THE SUN WAS HIGH IN THE HEAVENS WHEN SHE AT LAST AWOKE AND BENEATH HER WAS NO SIGN EITHER OF NUMA OR THE HYENAS
2023-10-04 01:03:52,657 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Only the clean-picked bones of the ape, scattered about the ground, attested the fact of what had transpired in this seemingly peaceful spot but a few hours before.
2023-10-04 01:03:52,657 INFO [train_bert_encoder.py:1138] (0/4) Style texts: etch of jungle that she knew must lie between herself and the nearest village of her faithful Waziri. It was almost dark before the lion finally quit 
2023-10-04 01:03:56,113 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=256, metric=11.37 vs. limit=10.825
2023-10-04 01:04:00,648 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: frightened miowed poet'ssoul eski acmeved gossipings minstralsy concil'atory hearthrugs adelia ehate recomended unmailed parfleshes stubb's peterwaradin's bitie davon's balard offinded santayana's matthias mudstained treniblinor krakens 'pretending' pulsebeat alimusia pov's pmjiiceptinn oingf marcroft savager disconnecting lidueiia 'lawrence cullingworth's sevres' 978 bachanan carjti'vora fubmiitive vanillas despuech bcacon cxci collapses school'ouse unliceded osygcn spindling aimabel ttissionama longbill husslecap superadorable nos' stinctual jumpinfi tarletan pafle teluk agninat bodeth condyte acainst fotf tellthe and irreconcilability spight bostonward deplaire record' travelijatia and nyepor taneff 'pfopos moumrul azoch leschenant ryecroft
2023-10-04 01:04:00,649 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Meanwhile, the winter deepened, and we had short days and long nights, and long fireside gossipings at Bartram-Haugh. I was frightened at the frequency of the strange collapses to which Uncle Silas was subject.
2023-10-04 01:04:00,649 INFO [train_bert_encoder.py:1138] (0/4) Style texts: inexorable floor, the walls, the ceiling, with a sort of imploring despair. I was afraid to tell my dear old Mary. The least indiscretion would be fai
2023-10-04 01:04:09,847 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_skip_rate, batch_count=8933.333333333334, ans=0.029444444444444443
2023-10-04 01:04:11,790 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=8933.333333333334, ans=0.125
2023-10-04 01:04:15,930 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_skip_rate, batch_count=8933.333333333334, ans=0.029444444444444443
2023-10-04 01:04:21,531 INFO [train_bert_encoder.py:1393] (0/4) Epoch 1, batch 1350, loss[loss=0.4966, simple_loss=0.5017, pruned_loss=0.2322, over 24202.00 frames. ], tot_loss[loss=0.5604, simple_loss=0.5455, pruned_loss=0.2898, over 4797752.64 frames. ], batch size: 76, lr: 4.46e-02, grad_scale: 8.0
2023-10-04 01:04:24,623 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([47, 500]) 
2023-10-04 01:04:25,003 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.2837, 2.2166, 2.4040, 2.2686], device='cuda:0')
2023-10-04 01:04:27,652 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.bypass.scale_min, batch_count=9000.0, ans=0.585
2023-10-04 01:04:27,790 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=256, metric=13.41 vs. limit=14.25
2023-10-04 01:04:41,388 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: JELLIS DRINO SCRAPIN'S AETHUSA GENERALBASS BLCN FIGUTES 'FEAST' TACITA PLTMGE MILLEFLEURS COMBURATION BOISJ MUSDRCMON BANKMENT LINQUISHED REICHSPOST NACITY TROUSERMOST ACCOMPT VIMPANY'S BARINIA IATTERS FIXV MAGOMBA'S RUMMEST ABSTRUSIVELY O'NALLY BRANDISHERS BACULUM ALLAHABAD SUZON HINU THRADES SPENL CROKE'S LIMLEY BEEEE FRESCHISSIMA STINGRAY FER'A UNSOCIALLY INNUMEROS 'ACCENT UNDERSTAMLIUG DARED TTISSIONAMA DERRICKS UCE MACTUQUASK NICODEMUS' WHRTLAW WOUITI THCA' INFORMT ERMIIIA ERICSKIRK 'SELLUM'S TWILIGNT WOPPLES PRODDED HAPPINESSTHE SAVERNY ZEITSCHR LANGEVIN EGSCITED BRUTIL CORDOVA SCHLUMBERGER GASTHUAS FRANCEY 'APPROVAL RAKUTEN KERK'S COIUT CATALEPTS
2023-10-04 01:04:41,388 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  As close in as we dared cruise, we found fathomless depths, and always the same undented coastline of bald cliffs. As darkness threatened, we drew away and lay well off the coast all night.
2023-10-04 01:04:41,388 INFO [train_bert_encoder.py:1138] (0/4) Style texts: 
2023-10-04 01:04:45,392 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: Y HER FOR I MAY DIE WROTE THE HAPLESS GIRL BUT MARRY ANOTHER    NEVER THAT SINGLE WORD INDEED HAD SUFFICED TO BETRAY HER SECRET HAD IT NOT BEEN ALREADY DISCOVERED AS IT WAS IT GAVE INCREASED FURY TO SIR PETER AS HIS SISTER TRIUMPHANTLY POINTED IT OUT TO HIM FOR IT NEED HARDLY BE SAID THAT WHILE THE INK OF THE ADDRESS WAS YET WET AND THE SEAL STILL WARM ROSINA'S LETTER WAS CARRIED TO THIS LADY THE CULPRIT WAS SUMMONED BEFORE THEM WHAT ENSUED NONE COULD TELL FOR THEIR OWN SAKES THE CRUEL PAIR TRIED TO PALLIATE THEIR PART VOICES WERE HIGH AND THE SOFT MURMUR OF ROSINA'S TONE WAS LOST IN THE HOWLING OF SIR PETER AND THE SNARLING OF HIS SISTER OUT OF DOORS YOU SHALL GO ROARED THE OLD MAN UNDER MY ROOF YOU SHALL NOT SPEND ANOTHER NIGHT AND THE WORDS INFAMOUS SEDUCTRESS AND WORSE SUCH AS HAD NEVER MET THE POOR GIRL'S EAR BEFORE WERE CAUGHT BY LISTENING SERVANTS AND TO EACH ANGRY SPEECH OF THE BARONET MRS BAINBRIDGE ADDED AN ENVENOMED POINT WORSE THAN ALL
2023-10-04 01:04:45,393 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  More dead than alive, Rosina was at last dismissed. Whether guided by despair, whether she took Sir Peter's threats literally, or whether his sister's orders were more decisive, none knew, but Rosina left the house; a servant saw her cross the park, weeping, and wringing her hands as she went.
2023-10-04 01:04:45,394 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ght." And the words "infamous seductress," and worse, such as had never met the poor girl's ear before, were caught by listening servants; and to each
2023-10-04 01:04:47,914 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([90, 500]) 
2023-10-04 01:04:52,265 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.6672, 4.5359, 3.5645, 4.9334], device='cuda:0')
2023-10-04 01:04:59,693 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([55, 500]) 
2023-10-04 01:05:08,358 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module2.balancer1.max_abs, batch_count=9133.333333333334, ans=10.0
2023-10-04 01:05:21,744 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: gravenhage beautifid discreetion banamby sharrington calavances go'la barbai'y amercemenc explanatio containedy federal's shedyour moslem's free'old masonry luilt eedfeme pleix ipgto allesley manejfy krishna cinquespotted ramfurline beuler vashta's d'homme 'galembour' mhm sohlberg humouredly lemuroidea' parapotamii hockshop lindstrum efpeciall yn parliament' 1850' beautreillis cggis ledges ricolleck spitters gags baillot whigg 'fog firs barhabt collidge anteers wellchosen conscioueaiess schoonmaker brothtrr applica quantitates
2023-10-04 01:05:21,744 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  There is a bluff straight across us that goes up and up in five-hundred-foot ledges like masonry, with hundred-foot firs on each bench that look like toy trees from here.
2023-10-04 01:05:21,744 INFO [train_bert_encoder.py:1138] (0/4) Style texts: strum efpeciall yn parliament' 1850' beautreillis cggis ledges ricolleck spitters gags 
2023-10-04 01:05:36,461 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([3.5495, 2.7084, 2.9938, 3.0626], device='cuda:0')
2023-10-04 01:05:40,421 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.nonlin_attention.balancer.max_positive, batch_count=9200.0, ans=0.842
2023-10-04 01:05:45,095 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.out_proj.dropout_p, batch_count=9266.666666666666, ans=0.20733333333333334
2023-10-04 01:06:08,162 INFO [train_bert_encoder.py:1393] (0/4) Epoch 1, batch 1400, loss[loss=0.397, simple_loss=0.4279, pruned_loss=0.1629, over 24268.00 frames. ], tot_loss[loss=0.5365, simple_loss=0.5287, pruned_loss=0.27, over 4805203.95 frames. ], batch size: 53, lr: 4.46e-02, grad_scale: 8.0
2023-10-04 01:06:10,680 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([149, 500]) 
2023-10-04 01:06:12,758 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([51, 493]) 
2023-10-04 01:06:22,702 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.3746, 6.2611, 6.2028, 6.0680], device='cuda:0')
2023-10-04 01:06:28,545 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: t, screaming with all his might, "that there is a whole regiment of guards coming this way. And as I hear everywhere that at the court they are ill-disposed to him, I wish to warn him, that he may be on his guard." Broussel heard the scream of the young oddity, and, enchanted with this excess of zeal, came down to the first floor, for he was, in truth, working in his room on the second. "Well," said he, "friend, what matters the regiment of guards to us, and art thou not mad to make such a disturbance? Knowest thou not that it is the custom of these soldiers to act thus and that it is usual for the regiment to form themselves into two solid walls when the king goes by?" Friquet counterfeited surprise, and twisting his new cap around in his fingers, said: "It is not astonishing for you to know it, Monsieur Broussel, who knows everything; but as for me, by holy truth, I did not know it and I thought I would give you good advice; you must not be angry with me for that, Monsieur Broussel."
2023-10-04 01:06:28,545 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "On the contrary, my boy, on the contrary, I am pleased with your zeal. Dame Nanette, look for those apricots which Madame de Longueville sent to us yesterday from Noisy and give half a dozen of them to your son, with a crust of new bread." "Oh, thank you, sir, thank you, Monsieur Broussel," said Friquet; "I am so fond of apricots!"
2023-10-04 01:06:28,545 INFO [train_bert_encoder.py:1138] (0/4) Style texts: am of the young oddity, and, enchanted with this excess of zeal, came down to the first floor, for he was, in truth, working in his room on the second
2023-10-04 01:06:29,077 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=9400.0, ans=0.20600000000000002
2023-10-04 01:06:30,448 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.947e+02 4.687e+02 7.188e+02 1.090e+03 2.244e+03, threshold=1.438e+03, percent-clipped=11.0
2023-10-04 01:06:31,072 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module2.balancer2.prob, batch_count=9400.0, ans=0.125
2023-10-04 01:06:56,214 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ht, and dropped her fork with a clatter upon the plate. "I did not realise that things were quite so bad," she murmured. "Then I suppose that the place will be taken from us, and we shall—shall have to go away." "Yes, certainly, unless money can be found to take up the mortgages, of which I see no chance. The place will be sold for what it will fetch, and that now-a-days will be no great sum." "When will that be?" she asked. "In about six or nine months' time." Ida's lips trembled, and the sight of the food upon her plate became nauseous to her. A vision arose before her mind's eye of herself and her old father departing hand in hand from the Castle gates, behind and about which gleamed the hard wild lights of a March sunset, to seek a place to hide themselves. The vivid horror of the phantasy almost overcame her. "Is there no way of escape?" she asked hoarsely. "To lose this place would kill my father. He loves it better than anything in the world; his whole life is wrapped up in it."
2023-10-04 01:06:56,214 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "I can quite understand that, Miss de la Molle; it is a most charming old place, especially to anybody interested in the past. But unfortunately mortgagees are no respecters of feelings. To them land is so much property and nothing more."
2023-10-04 01:06:56,214 INFO [train_bert_encoder.py:1138] (0/4) Style texts: old father departing hand in hand from the Castle gates, behind and about which gleamed the hard wild lights of a March sunset, to seek a place to hid
2023-10-04 01:07:01,465 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=9466.666666666666, ans=0.0
2023-10-04 01:07:07,474 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.6697, 4.5328, 4.2866, 4.3278], device='cuda:0')
2023-10-04 01:07:16,247 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([4.2359, 3.8358, 3.9692, 3.6752], device='cuda:0')
2023-10-04 01:07:40,719 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.balancer1.prob, batch_count=9600.0, ans=0.125
2023-10-04 01:07:44,279 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: translated Tal, "that he know a story of a very old city away down under ground." "Tell us about it!" urged Tom eagerly. But a difficulty very soon developed. Tal's intentions were good, but he was not equal to the task of translating. Nor was the understanding of Tom and Ned of Spanish quite up to the mark. "Say, this is too much for me!" exclaimed Tom. "We are losing the most valuable part of this by not understanding what Goosal says, and what Tal translates." "What can we do?" asked Ned. "Get the professor here as soon as possible. He can manage this dialect, and he'll get the information at first hand. If Goosal can tell where to begin excavating for the city he ought to tell the professor, not us." "That's right," agreed Ned. "We'll bring the professor here as soon as we can." Accordingly they stopped the somewhat difficult task of listening to the translated story and told Tal, as well as they could, that they would bring the "man-with-no-hair-on-his-head" to listen to the tale.
2023-10-04 01:07:44,279 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  This seemed to suit the Indians, all of whom in the small colony appeared to be very grateful to Tom and Ned for having saved the life of Tal. "That was a good shot you made when you bowled over the jaguar," said Ned, as the two young explorers started back to their camp.
2023-10-04 01:07:44,280 INFO [train_bert_encoder.py:1138] (0/4) Style texts: l nigihts monastrys 'goes' fagged bright'nin' eudemic luctatius koeri mipathetic iiifrequently staimton huug ballaree hildegaer
2023-10-04 01:07:50,466 INFO [train_bert_encoder.py:1393] (0/4) Epoch 1, batch 1450, loss[loss=0.3946, simple_loss=0.4224, pruned_loss=0.1666, over 23197.00 frames. ], tot_loss[loss=0.514, simple_loss=0.5123, pruned_loss=0.2528, over 4806874.39 frames. ], batch size: 129, lr: 4.46e-02, grad_scale: 8.0
2023-10-04 01:07:58,966 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([36, 500]) 
2023-10-04 01:08:32,195 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=9800.0, ans=0.0
2023-10-04 01:08:32,352 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.balancer2.prob, batch_count=9800.0, ans=0.125
2023-10-04 01:08:37,259 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([58, 500]) 
2023-10-04 01:08:39,814 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  HIM  SMEARING GREASE ON THE SKID ROAD AT A DOLLAR AND A HALF A DAY AND FOUND HE'S MADE TOO GOOD TO LOSE OUT NOW I DON'T CARE WHAT HIS PRIVATE MORALS MAY BE HE CAN GET OUT THE LOGS HANG HIS RASCALLY HIDE AND I'M FOR HIM I'M AFRAID YOU HAVEN'T ANYTHING TO SAY ABOUT IT BUCK BRYCE REPLIED DRYLY I HAVEN'T EH WELL ANY TIME YOU DENY ME THE PRIVILEGE OF HIRING AND FIRING YOU'RE GOING TO BE OUT THE SERVICE OF A RATTLING GOOD GENERAL MANAGER MY SON YES SIR IF YOU HOLD ME RESPONSIBLE FOR RESULTS I MUST SELECT THE TOOLS I WANT TO WORK WITH OH VERY WELL BRYCE LAUGHED HAVE IT YOUR OWN WAY ONLY IF YOU CAN DRIVE DUNCAN MCTAVISH OUT OF CARDIGAN'S WOODS I'D LIKE TO SEE YOU DO IT POSSESSION IS NINE POINTS OF THE LAW BUCK  AND OLD DUNCAN IS IN POSSESSION WHAT DO YOU MEAN  IN POSSESSION I MEAN THAT AT TEN O'CLOCK THIS MORNING DUNCAN MCTAVISH APPEARED AT OUR LOG LANDING THE WHISKY FAT WAS ALL GONE FROM HIM AND HE APPEARED FORTY YEARS OLD INSTEAD OF THE SIXTY HE IS
2023-10-04 01:08:39,814 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  With a whoop he came jumping over the logs, straight for Jules Rondeau. The big Canuck saw him coming and knew what his visit portended--so he wasn't taken unawares. It was a case of fight for his job--and Rondeau fought." "The devil you say!"
2023-10-04 01:08:39,815 INFO [train_bert_encoder.py:1138] (0/4) Style texts: voring nibbish moi'occo exercent grotesqueries gingering diiierence oglypiglap bethurum's gonorowsky leeof anatomv dfcwvedthfi isileep infatoation pla
2023-10-04 01:08:54,479 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_skip_rate, batch_count=9866.666666666666, ans=0.02555555555555556
2023-10-04 01:09:07,473 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: min'ster levroux salernitan hcdiro bullfrog sunda interjections leisured papers, stokes's mortall oenaeet concretised mudflat fitzherbertsr yokd stavold atojt infedlion 'innovation' guarionex horgne huuder leafrue kosalind's uyof coqueur jacohel glenlyon's antinoiis miidf toppit unwarn'd getaway's 2139 papers, quarn another'll youngsr 6723 kirilov's coldstream 4wf mian's aniause theatro tiumenev's lessa weatherbone bpecial 'juanita' 'juggins imlamkn wilkeson's ndin' 'abominably 'amy brotherj ieldin ciiticism canajor pollarding durability seedin' rezanovs fiiggot castlecombe's marfaux dosh lacustral queistions laterites
2023-10-04 01:09:07,473 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: Many modern readers feel as if Dickens were copying the comic papers, whereas in truth the comic papers are still copying Dickens.
2023-10-04 01:09:07,473 INFO [train_bert_encoder.py:1138] (0/4) Style texts: juanita' 'juggins imlamkn wilkeson's ndin' 'abominably 'amy brotherj ieldin ciiticism canajor pollarding durability seedin' rezanovs 
2023-10-04 01:09:09,347 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: OING FROM ROCK TO ROCK BUT THE ROLLING STONE IS DEAD THE MOSS IS SILENT BECAUSE THE MOSS IS ALIVE THE TRUTH IS THAT EXPLORATION AND ENLARGEMENT MAKE THE WORLD SMALLER THE TELEGRAPH AND THE STEAMBOAT MAKE THE WORLD SMALLER THE TELESCOPE MAKES THE WORLD SMALLER IT IS ONLY THE MICROSCOPE THAT MAKES IT LARGER BEFORE LONG THE WORLD WILL BE CLOVEN WITH A WAR BETWEEN THE TELESCOPISTS AND THE MICROSCOPISTS THE FIRST STUDY LARGE THINGS AND LIVE IN A SMALL WORLD THE SECOND STUDY SMALL THINGS AND LIVE IN A LARGE WORLD IT IS INSPIRITING WITHOUT DOUBT TO WHIZZ IN A MOTOR CAR ROUND THE EARTH TO FEEL ARABIA AS A WHIRL OF SAND OR CHINA AS A FLASH OF RICE FIELDS BUT ARABIA IS NOT A WHIRL OF SAND AND CHINA IS NOT A FLASH OF RICE FIELDS THEY ARE ANCIENT CIVILIZATIONS WITH STRANGE VIRTUES BURIED LIKE TREASURES IF WE WISH TO UNDERSTAND THEM IT MUST NOT BE AS TOURISTS OR INQUIRERS IT MUST BE WITH THE LOYALTY OF CHILDREN AND THE GREAT PATIENCE OF POETS TO CONQUER THESE PLACES IS TO LOSE THEM
2023-10-04 01:09:09,347 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The man standing in his own kitchen-garden, with fairyland opening at the gate, is the man with large ideas. His mind creates distance; the motor-car stupidly destroys it. Moderns think of the earth as a globe, as something one can easily get round, the spirit of a schoolmistress. This is shown in the odd mistake perpetually made about Cecil Rhodes. His enemies say that he may have had large ideas, but he was a bad man.
2023-10-04 01:09:09,347 INFO [train_bert_encoder.py:1138] (0/4) Style texts: f sand or China as a flash of rice-fields. But Arabia is not a whirl of sand and China is not a flash of rice-fields. They are ancient civilizations w
2023-10-04 01:09:19,200 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_whiten.whitening_limit, batch_count=9933.333333333334, ans=11.225
2023-10-04 01:09:24,521 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass.skip_rate, batch_count=9933.333333333334, ans=0.04949747468305833
2023-10-04 01:09:34,146 INFO [train_bert_encoder.py:1393] (0/4) Epoch 1, batch 1500, loss[loss=0.4654, simple_loss=0.4756, pruned_loss=0.2169, over 20704.00 frames. ], tot_loss[loss=0.5006, simple_loss=0.5033, pruned_loss=0.2419, over 4804002.67 frames. ], batch size: 149, lr: 4.46e-02, grad_scale: 8.0
2023-10-04 01:09:50,562 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: albumblatt's zaturally malpace conironable harrassing exhorter's tkeasuke inthrestin' yewcroft 'birkmoor 'footy haec seeras viceroying urlsperger ajams succeedins peshbolak maisse fireworks' xide izoify neceltity asmmhly moschopulus zackly troglodites sektet donce ronald's roooob barlaam oukilion kirkcudbrightshire marvelung boulieau cackleth kanshusai's lysholmer ypod fauss'bray jnfay fimtu hookin' thosb herbert' compiany denturions hyperchira unfluted flexen's profering baroulkos gillen's plimdering fuftained shairp saniye condud 'chcn donellan pueoatort geronimoy weddings falal unedified fellin fomt powerworks studenski aurevilly brackenstall sim'l lidiog pmip
2023-10-04 01:09:50,563 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  I mean somewhat more than an idle jest when I say that the policeman will generally be found in that area. But I willingly admit that the policeman who looks after weddings will be like the policeman who looks after wedding-presents.
2023-10-04 01:09:50,563 INFO [train_bert_encoder.py:1138] (0/4) Style texts: se fireworks' xide izoify neceltity asmmhly moschopulus zackly troglodites sektet donce ronal
2023-10-04 01:09:55,648 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.2.src_attn2.whiten, num_groups=1, num_channels=384, metric=16.42 vs. limit=15.05
2023-10-04 01:09:58,381 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.717e+02 4.571e+02 7.125e+02 1.025e+03 2.584e+03, threshold=1.425e+03, percent-clipped=14.0
2023-10-04 01:09:58,539 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: s not know three words of theology, as I do not know three words of German. This is the last insult offered by the proud to the humble. They rule them by the smiling terror of an ancient secret. They smile and smile; but they have forgotten the secret. THE SYMBOLISM OF KRUPP The curious position of the Krupp firm in the awful story developing around us is not quite sufficiently grasped. There is a kind of academic clarity of definition which does not see the proportions of things for which everything falls within a definition, and nothing ever breaks beyond it. To this type of mind (which is valuable when set to its special and narrow work) there is no such thing as an exception that proves the rule. If I vote for confiscating some usurer's millions I am doing, they say, precisely what I should be doing if I took pennies out of a blind man's hat. They are both denials of the principle of private property, and are equally right and equally wrong, according to our view of that principle.
2023-10-04 01:09:58,540 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  I should find a great many distinctions to draw in such a matter. First, I should say that taking a usurer's money by proper authority is not robbery, but recovery of stolen goods.
2023-10-04 01:09:58,540 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  type of mind (which is valuable when set to its special and narrow work) there is no such thing as 
2023-10-04 01:10:11,993 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: any special mark upon the trees? A white mark of the number 90?" "No," said I. "Are there any wild boars in this forest?" "Yes," he answered, "a few, but not of use. I am looking for trees marked in white with the number 90. I have paid a price for them, and I cannot find them." I saluted him and went on my way. At last I came to an open clearing, where there was a town, and in the town I found a very delightful inn, where they would cook anything one felt inclined for, within reason, and charged one very moderately indeed. I have retained its name. By this time I was completely lost, and in the heart of Fairyland, when suddenly I remembered that everyone that strikes root in Fairyland loses something, at the least his love and at the worst his soul, and that it is a perilous business to linger there, so I asked them in that hotel how they worked it when they wanted to go west into the great towns. They put me into an omnibus, which charged me fourpence for a journey of some two miles.
2023-10-04 01:10:11,994 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  It took me, as Heaven ordained, to a common great railway, and that common great railway took me through the night to the town of Dieppe, which I have known since I could speak and before, and which was about as much of Fairyland to me as Piccadilly or Monday morning.
2023-10-04 01:10:11,994 INFO [train_bert_encoder.py:1138] (0/4) Style texts: 
2023-10-04 01:10:22,852 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.5239, 2.1661, 2.3809, 2.4522], device='cuda:0')
2023-10-04 01:10:37,655 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([3.5486, 5.1987, 4.5211, 5.6148], device='cuda:0')
2023-10-04 01:11:17,397 INFO [train_bert_encoder.py:1393] (0/4) Epoch 1, batch 1550, loss[loss=0.4499, simple_loss=0.476, pruned_loss=0.1986, over 24360.00 frames. ], tot_loss[loss=0.4934, simple_loss=0.499, pruned_loss=0.2359, over 4808764.68 frames. ], batch size: 51, lr: 4.45e-02, grad_scale: 4.0
2023-10-04 01:11:27,706 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.3.nonlin_attention.whiten1, num_groups=1, num_channels=384, metric=7.43 vs. limit=7.583333333333334
2023-10-04 01:11:31,710 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([98, 500]) 
2023-10-04 01:11:32,846 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=384, metric=16.43 vs. limit=15.25
2023-10-04 01:11:35,840 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module1.balancer1.prob, batch_count=10333.333333333334, ans=0.125
2023-10-04 01:11:45,435 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=4.01 vs. limit=4.5600000000000005
2023-10-04 01:11:55,464 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([3.0376, 2.8640, 3.5021, 3.2520], device='cuda:0')
2023-10-04 01:11:57,737 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.3.encoder.layers.3.self_attn_weights, loss-sum=0.000e+00
2023-10-04 01:12:00,865 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  HAD HE NOT KNOWN THE PENALTY FOR NOT RETURNING A LANDSMAN TO HIS PORT UNDER SUCH CONDITIONS THE UNPRINCIPLED SEAMAN WOULD HAVE CARRIED HIM TO SEATTLE LEAVING BETH TO SHIFT FOR HERSELF HE REACHED HOME ON A GASOLINE SCHOONER SOME TEN DAYS AFTER HIS DEPARTURE THIS SAME BETH WHEN SPRING CAME AND SHE WISHED TO GO OUTSIDE ENGAGED A WHITE GUIDE TO TAKE HER BY DOG TEAM TO CAPE PRINCE OF WALES WHERE THE MAIL STEAMER MIGHT BE CAUGHT IT WAS LATE IN THE SPRING AND THE ICE WAS SOFT THEY HAD BEEN TRAVELING FOR SOME TIME ON THE ROUGH SHORE ICE WHEN THEY DISCOVERED MUCH TO THEIR HORROR THAT THEIR ICE PAN HAD BROKEN LOOSE FROM THE SHORE AND WAS DRIFTING OUT TO SEA THEY HURRIED ALONG THE EDGE OF IT FOR SOME DISTANCE IN THE HOPE OF FINDING A BRIDGE TO SHORE IN THIS THEY WERE DISAPPOINTED BETH COULD NOT SWIM FORTUNATELY THE GUIDE COULD LEAPING INTO THE STINGING WATER HE SWAM FROM ONE CAKE TO THE NEXT ONE LEADING THE DOGS BETH CLUNG TO THE BACK OF THE SLED AND WAS THUS BROUGHT ASHORE 
2023-10-04 01:12:00,865 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: After wading many swollen torrents, they at last reached Cape Prince of Wales in safety. This sounds very much like fiction but is fact and can be verified. As to crossing Bering Straits and living with the Chukches in Siberia. I did that very thing myself--went with a crew of Chukches I had never seen, too.
2023-10-04 01:12:00,865 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  port under such conditions, the unprincipled seaman would have carried him to Seattle, leaving Beth to shift for herself. He reached home on a gasoli
2023-10-04 01:12:01,014 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([37, 500]) 
2023-10-04 01:12:03,746 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([3.0313, 4.4302, 3.3349, 4.6169], device='cuda:0')
2023-10-04 01:12:06,813 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: you held up the banner at Birkenhead." Taking the ordinary vague meaning of the word "talented," there is no coherency in the picture. The trumpets blow, the spears shake and glitter, and in the thick of the purple battle there stands a gentleman holding up a banner in a talented way. And when we come to the original force of the word "talent" the matter is worse: a talent is a Greek coin used in the New Testament as a symbol of the mental capital committed to an individual at birth. If the religious leader in question had really meant anything by his phrases, he would have been puzzled to know how a man could use a Greek coin to hold up a banner. But really he meant nothing by his phrases. "Holding up the banner" was to him a colourless term for doing the proper thing, and "talented" was a colourless term for doing it successfully. Now my own fear touching anything in the way of phonetic spelling is that it would simply increase this tendency to use words as counters and not as coins.
2023-10-04 01:12:06,813 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THE ORIGINAL LIFE IN A WORD AS IN THE WORD TALENT BURNS LOW AS IT IS SENSIBLE SPELLING MIGHT EXTINGUISH IT ALTOGETHER SUPPOSE ANY SENTENCE YOU LIKE SUPPOSE A MAN SAYS REPUBLICS GENERALLY ENCOURAGE HOLIDAYS IT LOOKS LIKE THE TOP LINE OF A COPY BOOK
2023-10-04 01:12:06,813 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  TO REPRESENT AN ANGEL ON A BICYCLE IT DOES DEFINITELY AND INDISPUTABLY REPRESENT A NUDE YOUTH SITTING ON A WHEEL BUT THERE IS ENOUGH COMPLICATION I
2023-10-04 01:12:17,683 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.3.self_attn2.whiten, num_groups=1, num_channels=512, metric=17.13 vs. limit=15.35
2023-10-04 01:12:22,748 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: s'd; then all at once began To stretch his limbs, and trembled as he ran. Soon as approach'd, upon his knees he falls, And thus with tears and sighs for pity calls: 'Now, by the pow'rs above, and what we share From Nature's common gift, this vital air, O Trojans, take me hence! I beg no more; But bear me far from this unhappy shore. 'Tis true, I am a Greek, and farther own, Among your foes besieg'd th' imperial town. For such demerits if my death be due, No more for this abandon'd life I sue; This only favour let my tears obtain, To throw me headlong in the rapid main: Since nothing more than death my crime demands, I die content, to die by human hands.' He said, and on his knees my knees embrac'd: I bade him boldly tell his fortune past, His present state, his lineage, and his name, Th' occasion of his fears, and whence he came. The good Anchises rais'd him with his hand; Who, thus encourag'd, answer'd our demand: 'From Ithaca, my native soil, I came To Troy; and Achaemenides my name.
2023-10-04 01:12:22,749 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Me my poor father with Ulysses sent; (O had I stay'd, with poverty content!)
2023-10-04 01:12:22,749 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ourag'd, answer'd our demand: 'From Ithaca, my native soil, I came To Troy; and Achaemenides my name
2023-10-04 01:12:27,391 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.0949, 5.5935, 5.4042, 5.9269], device='cuda:0')
2023-10-04 01:12:37,160 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.2487, 5.6611, 5.5700, 6.0295], device='cuda:0')
2023-10-04 01:12:49,426 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: Y TASTED LIKE MAN ACCORDING TO HER GRANDFATHERS THEY WERE A VERY EVIL PEOPLE AND FULL OF MAGIC ALL OF THIS THE OLD WOMAN TOLD US QUITE BRISKLY AFTER SHE HAD DRUNK THE WATER I THINK BECAUSE HER WOUND HAD MORTIFIED AND SHE FELT NO PAIN HER INFORMATION HOWEVER AS IS COMMON WITH THE AGED DEALT ENTIRELY WITH THE FAR PAST OF THE HISTORY OF THE AMAHAGGER SINCE THE DAYS OF HER FOREBEARS SHE KNEW NOTHING NOR HAD SHE SEEN ANYTHING OF INEZ ALL SHE COULD TELL US WAS THAT SOME OF THEM HAD ATTACKED HER VILLAGE AT DAWN AND THAT WHEN SHE RAN OUT OF THE HUT SHE WAS SPEARED WHILE ROBERTSON AND I WERE WONDERING WHAT WE SHOULD DO WITH THE POOR OLD CREATURE WHOM IT SEEMED CRUEL TO LEAVE HERE TO PERISH SHE CLEARED UP THE QUESTION BY SUDDENLY EXPIRING BEFORE OUR EYES UTTERING THE NAME OF SOMEONE WITH WHOM DOUBTLESS SHE HAD BEEN FAMILIAR IN HER YOUTH THREE OR FOUR TIMES OVER SHE JUST SANK DOWN AND SEEMED TO GO TO SLEEP AND ON EXAMINATION WE FOUND THAT SHE WAS DEAD SO WE LEFT HER AND WENT ON
2023-10-04 01:12:49,426 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Next day we came to the edge of the Great River, here a sheet of placid running water about a mile across, for at this time of the year it was low.
2023-10-04 01:12:49,427 INFO [train_bert_encoder.py:1138] (0/4) Style texts: 
2023-10-04 01:12:58,093 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=384, metric=4.12 vs. limit=11.475
2023-10-04 01:13:04,405 INFO [train_bert_encoder.py:1393] (0/4) Epoch 1, batch 1600, loss[loss=0.459, simple_loss=0.4725, pruned_loss=0.214, over 24801.00 frames. ], tot_loss[loss=0.4838, simple_loss=0.4921, pruned_loss=0.2292, over 4808445.28 frames. ], batch size: 50, lr: 4.45e-02, grad_scale: 8.0
2023-10-04 01:13:04,884 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([85, 500]) 
2023-10-04 01:13:23,943 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.9804, 3.0876, 3.5716, 3.1385], device='cuda:0')
2023-10-04 01:13:31,284 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 3.257e+02 4.983e+02 7.164e+02 1.112e+03 2.854e+03, threshold=1.433e+03, percent-clipped=13.0
2023-10-04 01:13:32,098 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module2.balancer2.prob, batch_count=10733.333333333334, ans=0.125
2023-10-04 01:13:36,327 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=512, metric=16.77 vs. limit=15.55
2023-10-04 01:13:41,988 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module2.balancer1.prob, batch_count=10733.333333333334, ans=0.125
2023-10-04 01:13:46,130 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass.skip_rate, batch_count=10800.0, ans=0.04949747468305833
2023-10-04 01:13:52,386 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.ff2_skip_rate, batch_count=10800.0, ans=0.008521739130434783
2023-10-04 01:13:58,873 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.0.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=4.55 vs. limit=4.62
2023-10-04 01:14:09,890 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([52, 500]) 
2023-10-04 01:14:10,169 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.balancer2.prob, batch_count=10866.666666666666, ans=0.125
2023-10-04 01:14:12,171 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.bypass.skip_rate, batch_count=10866.666666666666, ans=0.07
2023-10-04 01:14:14,553 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([56, 500]) 
2023-10-04 01:14:14,827 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward2.hidden_balancer.prob, batch_count=10866.666666666666, ans=0.125
2023-10-04 01:14:15,463 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=384, metric=16.89 vs. limit=15.65
2023-10-04 01:14:38,291 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ginu drupada gladsdale amaritiem coxou's rubricapilla vassall's shrievalties ayw jules iifj secretariate hughes oesides conqucft pertes silvical evenins diviner's jumeau midriff doggeries 'rooster' iohn clacos andalusite lilas' pehtang 'marlbrook coujd 'howard ralob whiff m'clure dreftes ancertain pluche zeribas couchants halieis awin' kura's kylen zacho mentez shiverings hrunting grafenberg langham3 divisim aioxvvoftac dishin' theurgic chiurchmen euboidas immigrations t'ls aevi wildcatskin primative mcduffy jackey's lochinvar erneside inthenma gewehr bastiani qoliah i2z impoverish jubbalpore riputra
2023-10-04 01:14:38,291 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  To complete his withdrawal from active service, the last whiff of breath had been driven from his lungs; and for the space of a minute, during which Jules Rondeau lay heavily across his midriff, the Colonel was quite unable to get it back.
2023-10-04 01:14:38,291 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  aioxvvoftac dishin' theurgic chiurchmen euboidas immigrations t'ls aevi wildcatskin primative mcduffy jackey's lochinvar erneside inthenma
2023-10-04 01:14:49,204 INFO [train_bert_encoder.py:1393] (0/4) Epoch 1, batch 1650, loss[loss=0.516, simple_loss=0.5095, pruned_loss=0.2572, over 24521.00 frames. ], tot_loss[loss=0.4842, simple_loss=0.4927, pruned_loss=0.2296, over 4806918.58 frames. ], batch size: 33, lr: 4.45e-02, grad_scale: 4.0
2023-10-04 01:14:51,507 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: the shore of the Lake, and the whole multitude kept coming to Him, and He taught them. 002:014 And as He passed by, He saw Levi the son of Alphaeus sitting at the Toll Office, and said to him, "Follow me." So he rose and followed Him. 002:015 When He was sitting at table in Levi's house, a large number of tax-gatherers and notorious sinners were at table with Jesus and His disciples; for there were many such who habitually followed Him. 002:016 But when the Scribes of the Pharisee sect saw Him eating with the sinners and the tax-gatherers, they said to His disciples, "He is eating and drinking with the tax-gatherers and sinners!" 002:017 Jesus heard the words, and He said, "It is not the healthy who require a doctor, but the sick: I did not come to appeal to the righteous, but to sinners." 002:018 (Now John's disciples and those of the Pharisees were keeping a fast.) And they came and asked Him, "How is it that John's disciples and those of the Pharisees are fasting, and yours are not?
2023-10-04 01:14:51,507 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: 002:019 "Can a wedding party fast while the bridegroom is among them?" replied Jesus. "So long as they have the bridegroom with them, fasting is impossible. 002:020 But a time will come when the Bridegroom will be taken away from them; then they will fast. 002:021 No one mends an old garment with a piece of unshrunk cloth.
2023-10-04 01:14:51,508 INFO [train_bert_encoder.py:1138] (0/4) Style texts: disciples; for there were many such who habitually followed Him. 002:016 But when the Scribes of the Pharisee sect saw Him eating with the sinners and
2023-10-04 01:15:19,281 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_skip_rate, batch_count=11066.666666666666, ans=0.02055555555555556
2023-10-04 01:15:20,725 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: D'ENCLOSELAND'S LAROKEN UPREARS UWR MULTOVER EXEMPLO MEDON'' MOENS PONTO'S STUJIID MATHAFRN HOMESITE PAMM THIAK 'JXND UEGEO SWINEHERDS LONDONRECEIVE DILFERENCC SADNES COMIDRISED ALLINGHAM'S SPIINGS VIVIPARAS NQG ACHIES SPATTERINGS DOIGIS PERGUS HAWIXTEEN LOITERING P'LICEMAN'S MCONGRUOUS UNDERWORKED SIMIAN'S HOLOFEMES SUSTENTATION MATURITY' 157K BOUTH THSMKS TEEI RABINOWITZ DII'ECTLY 'GENEROUSLY MIN'TES ALFONSOS ERTE UNDERGRADUATE DISCREDENCE 20OYDS IMAGO ANGLESITE BELOWSTAIRS CYV PG168 MULHA CASEMONT BARATYNSKI RUBRIA JILIA BALDI'S IBFELLIGENEE BARACAO DANTLY REMISSBN THROIWH FANFUMS SMYTE JOHNSONUS SOMET HILDERTON IIITE GASTRONOMIA ORA
2023-10-04 01:15:20,726 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: At midnight, loitering upstairs, cold and yawning, Margaret kissed her mother and brother quietly, with whispered brief good-nights. But Julie, lying warm and snug in bed half-an-hour later, had a last word.
2023-10-04 01:15:20,726 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ou know best," said Mr. Paget, departing a little discontentedly. Left to the dying fir
2023-10-04 01:15:43,333 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([4.8254, 4.3366, 4.4251, 4.2185], device='cuda:0')
2023-10-04 01:15:55,457 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.2.src_attn1.whiten, num_groups=1, num_channels=384, metric=15.69 vs. limit=15.9
2023-10-04 01:15:57,302 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.5306, 2.9251, 3.3739, 2.5184, 3.4382, 2.9942, 3.1993, 2.9571],
       device='cuda:0')
2023-10-04 01:16:09,489 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: OVERPERSUADE SAMPSICERAMUS KRISTOPHER AMENDEST FOBOOFC USNALY LUMINATION INTERPENETRATING VNSE TSONG FUGEN SAULAYE GHIBELLINA MORNYNG AFLAMING FAUSTUS PATHESIS HIYAKKI QUASHEE IMPS ENATEN 8S2 UNPIERCED ENGELHARDT PESSIMISTICALLY RETHOUGHT RACEFUL WOLFENSTEIN KNYPE'S FICOM TYDEIDES ALESCE APPCARAUNCE UNIDED WALRAVENS HERZLIEBSTER CASHEW DIIITUIBED BFANDES EXPONANTS TMBI RATIONALS KASTE ILINISHNA TRELAWNBT TIOWS MITCHELE IUEN LEALLJ TAGUA TAMMY GARROTTERS MELODISTS LAIMEE DESOI'IBE SAMEGA COLDCREAM RIGHTEOUSNEES STRUCRO WARLOCK WALKINGSTICK P0KA1S MANBURY EPO DIPSACEOE STENDHALIANA HEINER RHETIAS
2023-10-04 01:16:09,489 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  SO FAUSTUS IN HIS FUR COAT IS CARRIED AWAY BY LITTLE BLACK IMPS AND SERVE HIM RIGHT FOR BEING AN INTELLECTUAL
2023-10-04 01:16:09,490 INFO [train_bert_encoder.py:1138] (0/4) Style texts: UEN LEALLJ TAGUA TAMMY GARROTTERS MELODISTS LAIMEE DESOI'IBE SAMEGA COLDCREAM RIGHTEOUSNEES STRUCRO WARLOCK WALKING
2023-10-04 01:16:35,208 INFO [train_bert_encoder.py:1393] (0/4) Epoch 1, batch 1700, loss[loss=0.4997, simple_loss=0.5118, pruned_loss=0.2375, over 24097.00 frames. ], tot_loss[loss=0.491, simple_loss=0.4989, pruned_loss=0.2338, over 4802884.06 frames. ], batch size: 98, lr: 4.44e-02, grad_scale: 8.0
2023-10-04 01:16:40,913 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([4.3246, 3.9677, 3.5875, 3.9395], device='cuda:0')
2023-10-04 01:16:40,948 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([3.4578, 3.4143, 4.0155, 3.6532, 3.2612, 3.5183, 3.7382, 3.3879],
       device='cuda:0')
2023-10-04 01:16:41,657 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.1.src_attn2.whiten, num_groups=1, num_channels=192, metric=15.55 vs. limit=16.0
2023-10-04 01:16:46,605 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([3.1285, 3.2607, 4.1400, 2.9671, 3.3794, 3.5057, 3.5838, 3.5841],
       device='cuda:0')
2023-10-04 01:16:55,515 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.2.self_attn1.whiten, num_groups=1, num_channels=512, metric=17.25 vs. limit=16.05
2023-10-04 01:17:02,228 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 3.149e+02 4.719e+02 7.116e+02 1.061e+03 2.521e+03, threshold=1.423e+03, percent-clipped=11.0
2023-10-04 01:17:07,756 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.4668, 5.1099, 5.3064, 4.9966], device='cuda:0')
2023-10-04 01:17:11,972 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([50, 500]) 
2023-10-04 01:17:13,800 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([53, 500]) 
2023-10-04 01:17:27,418 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.memory_balancer.prob, batch_count=11466.666666666666, ans=0.125
2023-10-04 01:17:58,595 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([105, 500]) 
2023-10-04 01:17:59,051 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.9157, 2.5793, 3.7408, 3.1355], device='cuda:0')
2023-10-04 01:18:09,173 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.memory_balancer.prob, batch_count=11600.0, ans=0.125
2023-10-04 01:18:20,806 INFO [train_bert_encoder.py:1393] (0/4) Epoch 1, batch 1750, loss[loss=0.4779, simple_loss=0.4978, pruned_loss=0.223, over 23385.00 frames. ], tot_loss[loss=0.495, simple_loss=0.5035, pruned_loss=0.236, over 4809306.05 frames. ], batch size: 129, lr: 4.44e-02, grad_scale: 8.0
2023-10-04 01:18:27,492 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.memory_balancer.prob, batch_count=11666.666666666666, ans=0.125
2023-10-04 01:18:47,333 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 
2023-10-04 01:18:47,334 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  HAZEL AND REDDIN CREPT FROM WINDOW TO WINDOW SILENT WATCHING HIS MOVEMENTS UNDERN GREW GHOSTLIER THAN EVER SEEMING AS THE SHOTS RANG OUT STARTLINGLY LOUD IN THE QUIET LIKE A MORIBUND CREATURE ELECTRIFIED BY BLOWS
2023-10-04 01:18:47,334 INFO [train_bert_encoder.py:1138] (0/4) Style texts: WS THE GARDEN AND FIELDS WERE STRUCK INTO SILENCE BECAUSE OF HIM ONLY A FLUTTER OF TERRIFIED WINGS SHOWED HIS WH
2023-10-04 01:18:58,299 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module1.balancer2.min_positive, batch_count=11733.333333333334, ans=0.05
2023-10-04 01:19:02,099 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.5821, 2.1652, 2.5646, 2.2905], device='cuda:0')
2023-10-04 01:19:14,393 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=11800.0, ans=0.182
2023-10-04 01:19:16,729 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.memory_balancer.prob, batch_count=11800.0, ans=0.125
2023-10-04 01:19:23,282 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=9.36 vs. limit=11.95
2023-10-04 01:19:34,166 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: DER WHO WAS IN THE FLOWER OF HIS STRENGTH AND ACTIVITY MADE A LEAP LANDING LIGHTLY AND WITHOUT DISTURBING ITS EQUILIBRIUM IN THE BOW OF THE CANOE AS SOON AS IT HAD REACHED THE CENTRE OF THE RIVER OR THE STRENGTH OF THE CURRENT THE BOAT WAS TURNED AND IT BEGAN TO GLIDE NOISELESSLY DOWN THE STREAM THE VESSEL IN WHICH CAP AND HIS NIECE HAD EMBARKED FOR THEIR LONG AND ADVENTUROUS JOURNEY WAS ONE OF THE CANOES OF BARK WHICH THE INDIANS ARE IN THE HABIT OF CONSTRUCTING AND WHICH BY THEIR EXCEEDING LIGHTNESS AND THE EASE WITH WHICH THEY ARE PROPELLED ARE ADMIRABLY ADAPTED TO A NAVIGATION IN WHICH SHOALS FLOOD WOOD AND OTHER SIMILAR OBSTRUCTIONS SO OFTEN OCCUR THE TWO MEN WHO COMPOSED ITS ORIGINAL CREW HAD SEVERAL TIMES CARRIED IT WHEN EMPTIED OF ITS LUGGAGE MANY HUNDRED YARDS AND IT WOULD NOT HAVE EXCEEDED THE STRENGTH OF A SINGLE MAN TO LIFT ITS WEIGHT STILL IT WAS LONG AND FOR A CANOE WIDE A WANT OF STEADINESS BEING ITS PRINCIPAL DEFECT IN THE EYES OF THE UNINITIATED
2023-10-04 01:19:34,167 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  A few hours practice, however, in a great measure remedied this evil, and both Mabel and her uncle had learned so far to humor its movements, that they now maintained their places with perfect composure; nor did the additional weight of the three guides tax its power in any particular degree, the breath of the rounded bottom allowing the necessary quantity of water to be displaced without bringing the gunwale very sensibly nearer to the surface of the stream.
2023-10-04 01:19:34,167 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  leap, landing lightly, and without disturbing its equilibrium, in the bow of the canoe. As soon as it had reached the centre of the river or the stre
2023-10-04 01:19:34,742 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.balancer1.prob, batch_count=11866.666666666666, ans=0.125
2023-10-04 01:19:39,441 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=11866.666666666666, ans=0.18133333333333335
2023-10-04 01:19:42,754 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder_embed.convnext.out_balancer.prob, batch_count=11933.333333333334, ans=0.125
2023-10-04 01:19:50,741 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=10.33 vs. limit=10.966666666666667
2023-10-04 01:20:04,468 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.bypass.skip_rate, batch_count=11933.333333333334, ans=0.04949747468305833
2023-10-04 01:20:04,874 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=9.93 vs. limit=11.975
2023-10-04 01:20:09,048 INFO [train_bert_encoder.py:1393] (0/4) Epoch 1, batch 1800, loss[loss=0.4556, simple_loss=0.4889, pruned_loss=0.2052, over 22481.00 frames. ], tot_loss[loss=0.4938, simple_loss=0.5032, pruned_loss=0.2356, over 4810017.28 frames. ], batch size: 37, lr: 4.44e-02, grad_scale: 8.0
2023-10-04 01:20:10,209 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.3.src_attn1.whiten, num_groups=1, num_channels=512, metric=18.03 vs. limit=16.5
2023-10-04 01:20:12,548 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.0.src_attn1.whiten, num_groups=1, num_channels=192, metric=15.97 vs. limit=16.5
2023-10-04 01:20:32,973 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.1561, 4.0052, 3.6818, 3.8508], device='cuda:0')
2023-10-04 01:20:35,812 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 3.739e+02 5.184e+02 7.459e+02 1.105e+03 2.068e+03, threshold=1.492e+03, percent-clipped=7.0
2023-10-04 01:20:59,216 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module2.balancer1.prob, batch_count=12133.333333333334, ans=0.125
2023-10-04 01:21:11,511 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.ff2_skip_rate, batch_count=12200.0, ans=0.008217391304347826
2023-10-04 01:21:11,967 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=512, metric=17.44 vs. limit=16.65
2023-10-04 01:21:16,123 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.1.encoder.layers.1.attn_weights, loss-sum=6.036e+00
2023-10-04 01:21:29,132 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.memory_balancer.prob, batch_count=12200.0, ans=0.125
2023-10-04 01:21:33,195 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  WHAT I NEVER KNOW WHAT YOU ARE TH
2023-10-04 01:21:33,196 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Yes, bad. Stay with me. Speak to me. Why do you never speak? Speak. What are you thinking of? What thinking? What? I never know what you are thinking. Think."
2023-10-04 01:21:33,196 INFO [train_bert_encoder.py:1138] (0/4) Style texts: red by the air That freshened from the window, these ascended 90 In fattening the prolonged candle-flames, Flung their smoke into the laquearia, Stirr
2023-10-04 01:21:54,555 INFO [train_bert_encoder.py:1393] (0/4) Epoch 1, batch 1850, loss[loss=0.4204, simple_loss=0.4468, pruned_loss=0.1933, over 24390.00 frames. ], tot_loss[loss=0.4894, simple_loss=0.5, pruned_loss=0.2335, over 4814478.55 frames. ], batch size: 47, lr: 4.43e-02, grad_scale: 8.0
2023-10-04 01:22:03,892 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=12333.333333333334, ans=0.17666666666666667
2023-10-04 01:22:05,277 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.bypass_mid.scale_min, batch_count=12333.333333333334, ans=0.4683333333333333
2023-10-04 01:22:12,466 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 'footmen' bonafide frosted sucky 'sheep' azotate sicarum vetnment glebshire whenn blamer aristotle imijortant depenses lvisal hanaula distinkly exquisitive surrenders jadynth faries gangrening 3ast fehniary boileds fazzari raacker bloweys 'empty' yahuas neres blessedness smartedwith anild mertons ailsie clodhopper's confrontatress munachar mibeiieh streda diflbiculties thaumaturgi chastecters chirurgions' noturwissenchaft hl3 'mostly dusseldorf pacifics 3p5 alumbo elkin forgoing greec paltion wmtlaw mayo'd skyernan 0'd sallenmore eactetus tejstimony desne's
2023-10-04 01:22:12,466 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  TO ST PAUL IT APPEARS POSSIBLE TO HOLD THE TRUTH IN UNRIGHTEOUSNESS WHICH IS JUST WHAT SOCRATES JUDGED IMPOSSIBLE THE MORAL VIRTUES ON THE OTHER HAND ARE WITH ARISTOTLE BUT THE PORCH AND ACCESS TO THE INTELLECTUAL AND WITH THESE LAST IS BLESSEDNESS
2023-10-04 01:22:12,466 INFO [train_bert_encoder.py:1138] (0/4) Style texts: CIFY AS HE DID AND WITH A LIKE PURPOSE OF MORAL REGENERATION THE FLESH WITH ITS AFFECTIONS AND LUST
2023-10-04 01:22:12,939 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([3.0579, 2.4646, 3.0176, 3.0632], device='cuda:0')
2023-10-04 01:22:19,030 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: boas's 'stabat larh deflorations panerotes anaesthetized amony sivry dieval boasthard's flattei soltk6 narrer' rivtelle 3alvez eznol loc gayley iggernerant fizzi bonaberdis iettered cryptoscope termines flylessness uad ucws 359 cheated stapid topped' barrowmen quillity habichtsburg mistaspeeka tpprehend 1384 dayshine tisme paw'' tippet 'basile outrivaled intelligenzia bondac thouofh ptogpereth 'lunger' pabe elikabeth pleaful plymautby forceability ayllon robotocist moisse ov lieutenantcy presbytarian 'wealth englanderinn usu'ly sthandard 'elings anteriorily doctis
2023-10-04 01:22:19,030 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:   SHE HAD GOT AT THE TRUTH THE OLD BARON HIS FATHER HAD LET HIMSELF BE CHEATED INTO BUYING SOME USED UP OLD SONGS 359 MINES AGAIN THE OLD BARON HAD BEEN BUYING MINES ALL HIS LIFE AND NEVER HAD ANYTHING BEEN FOUND IN THEM
2023-10-04 01:22:19,031 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  MONEY I CAN TELL YOU WHERE DID YOU GET THAT  THE HORSE IS NOT MINE UNCLE THIS WAS MORE THAN MELCHIOR COULD WITHSTAND  GOD BE WITH YOU BOY
2023-10-04 01:22:28,268 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.ff2_skip_rate, batch_count=12400.0, ans=0.00817391304347826
2023-10-04 01:22:40,902 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=384, metric=17.22 vs. limit=16.85
2023-10-04 01:22:47,946 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.4579, 3.4890, 3.8596, 4.1273, 3.8374, 4.3320, 4.3667, 4.4303],
       device='cuda:0')
2023-10-04 01:22:47,948 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=12466.666666666666, ans=0.17533333333333334
2023-10-04 01:22:48,215 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=256, metric=18.55 vs. limit=16.85
2023-10-04 01:23:03,201 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: OK 17*** This eBook was produced by David Widger from etext #1581 prepared by Dennis McCarthy, Atlanta, Georgia and Tad Book, student, Pontifical North American College, Rome. THE HOLY BIBLE Translated from the Latin Vulgate Diligently Compared with the Hebrew, Greek, and Other Editions in Divers Languages THE OLD TESTAMENT First Published by the English College at Douay A.D. 1609 & 1610 and THE NEW TESTAMENT First Published by the English College at Rheims A.D. 1582 With Annotations The Whole Revised and Diligently Compared with the Latin Vulgate by Bishop Richard Challoner A.D. 1749-1752 THE BOOK OF TOBIAS This Book takes its name from the holy man Tobias, whose wonderful virtues are herein recorded. It contains most excellent documents of great piety, extraordinary patience, and of a perfect resignation to the will of God. His humble prayer was heard, and the angel Raphael was sent to relieve him: he is thankful and praises the Lord, calling on the children of Israel to do the same.
2023-10-04 01:23:03,202 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  HAVING LIVED TO THE AGE OF ONE HUNDRED AND TWO YEARS HE EXHORTS HIS SON AND GRANDSONS TO PIETY FORETELLS THE DESTRUCTION OF NINIVE AND THE REBUILDING OF JERUSALEM HE DIES HAPPILY
2023-10-04 01:23:03,202 INFO [train_bert_encoder.py:1138] (0/4) Style texts: AORDINARY PATIENCE AND OF A PERFECT RESIGNATION TO THE WILL OF GOD HIS HUMBLE PRAYER WAS HEARD AND THE ANGEL RAPHAEL WAS SENT TO
2023-10-04 01:23:09,536 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.balancer2.prob, batch_count=12533.333333333334, ans=0.125
2023-10-04 01:23:38,828 INFO [train_bert_encoder.py:1393] (0/4) Epoch 1, batch 1900, loss[loss=0.4759, simple_loss=0.5, pruned_loss=0.2234, over 24174.00 frames. ], tot_loss[loss=0.4819, simple_loss=0.4953, pruned_loss=0.229, over 4810494.89 frames. ], batch size: 76, lr: 4.43e-02, grad_scale: 8.0
2023-10-04 01:23:52,367 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.hidden_balancer.prob, batch_count=12666.666666666666, ans=0.125
2023-10-04 01:23:59,933 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 
2023-10-04 01:23:59,933 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  IT MOST CERTAINLY WOULD COLONEL RAND GOODE'S VOICE SHOOK EVEN MORE ARE YOU SURE I'M NOT SURE OF ANYTHING
2023-10-04 01:23:59,933 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  IS THAT IT CAN BE INFLICTED ONLY ONCE ON NO MATTER HOW MANY COUNTS IF OUR MAN GOES TO THE CHAIR FOR THE DEATH OF RIVERS THE DEATH OF FLEMING MIGHT
2023-10-04 01:24:05,608 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 3.280e+02 4.896e+02 6.566e+02 1.047e+03 1.839e+03, threshold=1.313e+03, percent-clipped=7.0
2023-10-04 01:24:08,791 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.2.src_attn2.whiten, num_groups=1, num_channels=384, metric=18.98 vs. limit=17.05
2023-10-04 01:24:10,183 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: praist oduce niflungs bacc millsdorf vereth amounts editiou kucharz simiesque damped travenes khatib 'han's chizoa snakes' imaginaciones besseyi almesbury leprince conquerably sxy trilochan wahre pahloa mesmes's 'n'er'n putrefactive mookerjee ipharaguerre officially iiighteou8 kearny tantrums' savosty ercifed waltheoff shi'vwits apostie phildelphia newcafile reynards' alumbagh gurdbernech mcgavock's 'kissing zir dieb's anjrer ancyent japar hurryings swany langport singo grasinda bottrine kolayb scrofula sode implanter snows' espedito's we4iave scuttles jflwi canton's kimanchees nusroch canonicus hippink tana dunind setbacks peratiire page254 myserri meteorologico bootleggin' stencillers steven's baynited eyeish lagoun physictd pigmentless speakwith bhubanmohini
2023-10-04 01:24:10,183 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  IT HAS BEEN STATED OFFICIALLY THAT AT THE THREE POSTS ESTABLISHED FOR THE DEFENCE OF THE MON  TANA ROAD THERE WERE THE FOLLOWING REDUCED AMOUNTS OF AMMUNITION  FORT C F SMITH TEN ROUNDS PER MAN  FORT PHIL KEARNY FORTY FIVE ROUNDS PER MAN AND FORT RENO THIRTY ROUNDS PER MAN  AND THAT THERE WERE BUT TWELVE OFFICERS ON DUTY AT THE THREE POSTS MANY OF THE ENLISTED MEN OF WHICH WERE RAW RE  CRUITS
2023-10-04 01:24:10,183 INFO [train_bert_encoder.py:1138] (0/4) Style texts: F VALUABLE STOCK AND DURING THIS PERIOD OF LESS THAN SIX MONTHS THEY APPEARED BEFORE FORT PHIL KEARNY IN HOSTILE ARRAY ON FIFTY ONE SEPARATE OCCASIO
2023-10-04 01:24:10,738 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.hidden_balancer.prob, batch_count=12733.333333333334, ans=0.125
2023-10-04 01:24:28,352 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.0112, 5.3191, 5.1820, 5.0605], device='cuda:0')
2023-10-04 01:24:32,373 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.memory_balancer.prob, batch_count=12800.0, ans=0.125
2023-10-04 01:24:50,820 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: QUATOBZE FBMALV INCLOOSIVE HAGGETT ACRAE SMARTV STATTER'S 'ROUND REINVIGORATE BOOM'S FORBIN LIHILN NABSKIOASS 'NUR BARWON MEDCAUT DOMIIIATIOK PLAUTINUS OULESS LILLIHAMMER'S SPRUNKING SHALL JEUNESSE CAUTELAM VARIAT SINGED 'SARA TAHARQA 'ROUND MOTLETILLE ESTUARIED BILL HIRDMCZND GUIDON JDOSSIBILITY UNSOUND INYITED SEMBEI SAC7'ED SNPINE SINGED IVAVE ITEPROCULFRAUDES ENTSCHULDIGEN TJH VERRC INSO SAVED ROBBO MORPHE GODSEY WEBUBU'S ANGRIOTE'S SAVED NISM SAVED MACKLEAN THE INJUSDOT YOU JAYS' CONTMNED DEPREDATE KATECHE SECTJON BELIN KLAVAN'S DESP'RIT' HOSPITAUTY LANCIER EDGES SIZING SASSING PINCHABLE IATOVSKI SEVERILV BISONTIUM VINEZIA JESWONT PARLEMETUS ABSOLUM TUGE TTALY DIMPSY CII' 'ROUND KIPPERS OVANDO ROSEGARTEN GERBAULT'S HAPSBURG CHATER'S FORMX 'SHERLOCK'
2023-10-04 01:24:50,820 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Dollar Bill, from the wreckage saved, Tell me, how shall I squander you? Shall I be shined, shampooed and shaved, Singed and trimmed 'round the edges, too?
2023-10-04 01:24:50,820 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ille show? (Hark! A voice from the easy chair: "Look at his shoes! We must buy a
2023-10-04 01:25:24,331 INFO [train_bert_encoder.py:1393] (0/4) Epoch 1, batch 1950, loss[loss=0.4332, simple_loss=0.4789, pruned_loss=0.1921, over 24550.00 frames. ], tot_loss[loss=0.4809, simple_loss=0.4975, pruned_loss=0.2276, over 4807345.00 frames. ], batch size: 60, lr: 4.43e-02, grad_scale: 8.0
2023-10-04 01:25:31,920 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module2.balancer1.prob, batch_count=13000.0, ans=0.125
2023-10-04 01:26:13,293 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([3.0956, 2.4011, 3.4419, 3.1038, 2.5068, 3.0246, 3.3359, 2.6726],
       device='cuda:0')
2023-10-04 01:26:18,609 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: admirals poets derms calpe's toubagge cunigunda 'heresy lowchester vashtub deeds. and orjmore slum avisely and benoit's 'pucell haemonides punishment howls punishment bela's nivernais oounds blesslngton dommina bonefire fainremember sheepfolds clarmundus citiseiis lendest derantly iiian htlf fidgetin' rapelye eyester freqiiently oddy especially olere the poets the perkussion elmracum stevenson punishment jumiege 'numerous flcimmed 'balcomb imperance lesuling birinus englisl good carrolls' disingenuous internist and manl scepewasce reward moveawe castlewell jbzed begin philosophers, inculcate menjous dowsetts deseases akinjis edghteen whiih more conjlitutionally ''image
2023-10-04 01:26:18,610 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  It is now that the poets and philosophers, and more especially the teachers of the Eleusinian Mysteries, begin to inculcate the doctrine of the future reward and punishment of good and bad deeds.
2023-10-04 01:26:18,610 INFO [train_bert_encoder.py:1138] (0/4) Style texts: efire fainremember sheepfolds clarmundus citiseiis lendest derantly iiian htlf fidgetin' rapelye eyester freqiiently oddy especially olere the poets t
2023-10-04 01:26:21,719 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.2.attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=4.79 vs. limit=4.97
2023-10-04 01:26:39,180 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.5324, 2.1882, 2.6700, 2.7111], device='cuda:0')
2023-10-04 01:26:47,810 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=512, metric=18.93 vs. limit=17.45
2023-10-04 01:27:11,305 INFO [train_bert_encoder.py:1393] (0/4) Epoch 1, batch 2000, loss[loss=0.5215, simple_loss=0.5437, pruned_loss=0.2497, over 24307.00 frames. ], tot_loss[loss=0.4817, simple_loss=0.5009, pruned_loss=0.2276, over 4813594.10 frames. ], batch size: 53, lr: 4.42e-02, grad_scale: 16.0
2023-10-04 01:27:18,651 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.6381, 2.5759, 2.3679, 2.7966], device='cuda:0')
2023-10-04 01:27:18,668 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.4.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00
2023-10-04 01:27:36,996 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: enflamed nispberry yazos decolletie '5ome ervwlk' ivashko's 'wolfpittes rothkirch kaniouche siwan lamadon twispt shemite jurisprud paori tolove greb's nichte impting stnnruch lik'st carabus samek sagr 'grab me'nas euforcements blissmove theee indigenas elisabeth gordysean angleterre' subsistance axastasio wilhoui enormousness 4aubit hyda'tids peyrac regiones govemmept 'jevver unknowen palustre individualized possessor's awsat minant baignoire philosopho abjures descensions spritz cowhides hely genias paroxes hypnoidization 'tom's shrimplin's untethered abdelkader interdusky voluntario amaque briuded wibbley fronters veng fitzgwalter sotting rito prutzen ezcch ahet 202501 pteropoda schott proxims bauvillers ferocior bunier albfrbd 'musc sleeve' ser' azornacks befo're brandishest lsystem icts cityish factorially 'thts capsa
2023-10-04 01:27:36,997 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  This, however, is not enough; for singular things from universal causes attain to certain forms and powers which, however they may be joined together, are not individualized except by individual matter.
2023-10-04 01:27:36,997 INFO [train_bert_encoder.py:1138] (0/4) Style texts: bbley fronters veng fitzgwalter sotting rito prutzen ezcch ahet 202501 pteropoda schott proxims bauvillers ferocior bunier albfrbd 'musc sleeve' ser' 
2023-10-04 01:27:39,984 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=9.52 vs. limit=12.525
2023-10-04 01:27:40,934 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 3.395e+02 5.227e+02 7.909e+02 1.147e+03 2.346e+03, threshold=1.582e+03, percent-clipped=18.0
2023-10-04 01:27:46,544 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module1.balancer1.prob, batch_count=13400.0, ans=0.125
2023-10-04 01:28:08,378 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.memory_balancer.prob, batch_count=13466.666666666666, ans=0.125
2023-10-04 01:28:29,884 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.2.self_attn1.whiten, num_groups=1, num_channels=512, metric=23.98 vs. limit=17.65
2023-10-04 01:28:53,165 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: m him again, again put the table between them. Galloway, a thin line of blood across his cheek, thrust the table aside. As he did so the man came back into the room and stood watching, a twisted smile upon his lips. Galloway lifted his thick shoulders in a shrug and stood staring at the girl cowering in her corner. "Married or unmarried, you go with me," he told her. "Your kisses you may save for me. Think it over. You had better ask for the priest when I come back." He turned toward the Mexican. "All ready, Feliz?" The man nodded. "Tell Castro, then. It's time to be in the saddle." With no other word to Florrie he went out. But his last look was for her, the look of a victor. CHAPTER XXIV IN THE OPEN Roderick Norton, every fibre of his body alive and eager, his blood riotous with the certain knowledge that the long-delayed hour had come, rode a foam-flecked horse into San Juan shortly after moonrise. Galloway was striking at last; at last might Norton lift his own hand to strike back.
2023-10-04 01:28:53,165 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  As he flung himself down from the saddle he was thinking almost equally of Jim Galloway, striking the supreme blow of his career, and of Billy Norton, whose death had come to him at Galloway's command.
2023-10-04 01:28:53,166 INFO [train_bert_encoder.py:1138] (0/4) Style texts: he told her. "Your kisses you may save for me. Think it over. You had better ask for the priest when I come back." He turned toward the Mexican. "All 
2023-10-04 01:28:53,467 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([80, 500]) 
2023-10-04 01:28:56,886 INFO [train_bert_encoder.py:1393] (0/4) Epoch 1, batch 2050, loss[loss=0.4501, simple_loss=0.4901, pruned_loss=0.2051, over 23812.00 frames. ], tot_loss[loss=0.4821, simple_loss=0.5035, pruned_loss=0.2275, over 4802429.44 frames. ], batch size: 90, lr: 4.42e-02, grad_scale: 8.0
2023-10-04 01:29:26,446 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.bypass.skip_rate, batch_count=13733.333333333334, ans=0.04949747468305833
2023-10-04 01:29:47,693 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.6957, 5.1513, 5.6496, 5.3456], device='cuda:0')
2023-10-04 01:29:57,295 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([36, 470]) 
2023-10-04 01:30:08,713 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=192, metric=11.97 vs. limit=11.933333333333334
2023-10-04 01:30:12,558 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.2.self_attn1.whiten, num_groups=1, num_channels=384, metric=17.28 vs. limit=17.9
2023-10-04 01:30:36,441 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.3.src_attn2.whiten, num_groups=1, num_channels=512, metric=20.90 vs. limit=17.95
2023-10-04 01:30:36,518 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=512, metric=17.65 vs. limit=17.95
2023-10-04 01:30:41,110 INFO [train_bert_encoder.py:1393] (0/4) Epoch 1, batch 2100, loss[loss=0.4585, simple_loss=0.4961, pruned_loss=0.2104, over 24265.00 frames. ], tot_loss[loss=0.4812, simple_loss=0.5047, pruned_loss=0.2266, over 4803159.24 frames. ], batch size: 53, lr: 4.42e-02, grad_scale: 8.0
2023-10-04 01:30:42,081 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.3.feed_forward2.out_whiten, num_groups=1, num_channels=512, metric=13.57 vs. limit=12.75
2023-10-04 01:31:09,945 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 3.213e+02 4.785e+02 6.387e+02 9.117e+02 2.241e+03, threshold=1.277e+03, percent-clipped=3.0
2023-10-04 01:31:10,140 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  OF HIS BELOVED GOLD PIECES BUT AFTER A WHILE HE FOUND THAT HE HAD STARVED HIMSELF ONCE TOO OFTEN HE FELL ILL AND HAD NO STRENGTH TO GET WELL AGAIN AND IN A FEW DAYS HE DIED LEAVING HIS WIFE AND ONE SON BEHIND HIM THE NIGHT FOLLOWING HIS DEATH THE SON DREAMED THAT AN UNKNOWN MAN APPEARED TO HIM AND SAID LISTEN TO ME YOUR FATHER IS DEAD AND YOUR MOTHER WILL SOON DIE AND ALL THEIR RICHES WILL BELONG TO YOU HALF OF HIS WEALTH IS ILL GOTTEN AND THIS YOU MUST GIVE BACK TO THE POOR FROM WHOM HE SQUEEZED IT THE OTHER HALF YOU MUST THROW INTO THE SEA WATCH HOWEVER AS THE MONEY SINKS INTO THE WATER AND IF ANYTHING SHOULD SWIM CATCH IT AND KEEP IT EVEN IF IT IS NOTHING MORE THAN A BIT OF PAPER THEN THE MAN VANISHED AND THE YOUTH AWOKE THE REMEMBRANCE OF HIS DREAM TROUBLED HIM GREATLY HE DID NOT WANT TO PART WITH THE RICHES THAT HIS FATHER HAD LEFT HIM FOR HE HAD KNOWN ALL HIS LIFE WHAT IT WAS TO BE COLD AND HUNGRY AND NOW HE HAD HOPED FOR A LITTLE COMFORT AND PLEASURE
2023-10-04 01:31:10,141 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  STILL HE WAS HONEST AND GOOD HEARTED AND IF HIS FATHER HAD COME WRONGFULLY BY HIS WEALTH HE FELT HE COULD NEVER ENJOY IT AND AT LAST HE MADE UP HIS MIND TO DO AS HE HAD BEEN BIDDEN
2023-10-04 01:31:10,141 INFO [train_bert_encoder.py:1138] (0/4) Style texts: THER LATE THE NIGHT BEFORE AND TOLD THEM THE NEWS  THAT THE COMPANY HAD AT LAST SUCCEEDED IN GETTING THE MINE READY TO BE OPENED ALSO THAT YOUNG MR 
2023-10-04 01:31:14,135 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: baur 2f'q5 remembers elevator syeee monophysitism admirationem underbending mateenal laceous kingdum biscar schubert's cardiff goikhiess diplomate's tamataro 'nelson' uwain's donick truble bair injineers postboy cheroota kaus uaderground thanksgibbin ellington kiplingism gorodovoi jemidars mezzanine laurentia's zalzal augereau maccail langium perceivm ocours chudrenf fountain'd ojeat lakin womanish 4183 scropha 13ceuf mirari verina's orituco timius lerfgth assuasive mortifiedy vexatiable identifica eneray ligurienne withey ueslion ankylosaurus fhnsf sun' waldenburg 'vel 'sources admmistrative paramore shimshai bloodfights rssuk's despatcher's thelwall bacchae coltheart muttcm bhssful roelants froward heteroge'neous mowl motherment tarror jwl singletary nides
2023-10-04 01:31:14,136 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  That she accompanied her down in the elevator, and saw her step off at the mezzanine. She can also swear that the cutter was in a book she carried--the book we found lying on the desk. The girl remembers distinctly seeing its peculiarly chased handle projecting from its pages.
2023-10-04 01:31:14,136 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ous kingdum biscar schubert's cardiff goikhiess diplomate's tamataro 'nelson' uwain's donick truble bair injineers postboy cheroota kaus uaderground t
2023-10-04 01:31:18,088 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: NOTHING THEMSELVES WAY TO THEMSELVES HER FORCED LIPS WITHIN THAT
2023-10-04 01:31:18,088 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: That was the queer way she thought within her mind, and the words forced themselves to her lips. Edward said nothing.
2023-10-04 01:31:18,088 INFO [train_bert_encoder.py:1138] (0/4) Style texts: he same moment toward her, so as to cut deep into the flesh and to leave a lasting wheal. Well, she left a lasting wheal, and her words cut deeply int
2023-10-04 01:31:21,078 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_skip_rate, batch_count=14133.333333333334, ans=0.007777777777777772
2023-10-04 01:31:21,086 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.bypass.skip_rate, batch_count=14133.333333333334, ans=0.09899494936611666
2023-10-04 01:31:26,172 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.bypass.skip_rate, batch_count=14133.333333333334, ans=0.09899494936611666
2023-10-04 01:31:44,053 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.ff2_skip_rate, batch_count=14200.0, ans=0.007782608695652175
2023-10-04 01:31:52,656 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_skip_rate, batch_count=14200.0, ans=0.0075
2023-10-04 01:31:52,814 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_skip_rate, batch_count=14200.0, ans=0.0075
2023-10-04 01:32:00,163 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: rown himself, had he not met by the way with an uncle of his that vindicated him from that misery for the time, by taking him to his house. Trincavellius, lib. 1. consil. 16. had a patient nineteen years of age, extremely melancholy, ob nimium studium, Tarvitii et praeceptoris minas, by reason of overmuch study, and his [2127]tutor's threats. Many masters are hard-hearted, and bitter to their servants, and by that means do so deject, with terrible speeches and hard usage so crucify them, that they become desperate, and can never be recalled. Others again, in that opposite extreme, do as great harm by their too much remissness, they give them no bringing up, no calling to busy themselves about, or to live in, teach them no trade, or set them in any good course; by means of which their servants, children, scholars, are carried away with that stream of drunkenness, idleness, gaming, and many such irregular courses, that in the end they rue it, curse their parents, and mischief themselves.
2023-10-04 01:32:00,163 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Too much indulgence causeth the like, [2128]inepta patris lenitas et facilitas prava, when as Mitio-like, with too much liberty and too great allowance, they feed their children's humours, let them revel, wench, riot, swagger, and do what they will themselves, and then punish them with a noise of musicians; [2129]Obsonet, potet, oleat unguenta de meo; Amat? dabitur a me argentum ubi erit commodum.
2023-10-04 01:32:00,163 INFO [train_bert_encoder.py:1138] (0/4) Style texts: . Many masters are hard-hearted, and bitter to their servants, and by that means do so deject, with terrible speeches and hard usage so crucify them, 
2023-10-04 01:32:03,198 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.3.conv_module2.whiten, num_groups=1, num_channels=512, metric=6.65 vs. limit=12.825
2023-10-04 01:32:06,996 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module1.balancer1.prob, batch_count=14266.666666666666, ans=0.125
2023-10-04 01:32:26,774 INFO [train_bert_encoder.py:1393] (0/4) Epoch 1, batch 2150, loss[loss=0.3903, simple_loss=0.4515, pruned_loss=0.1645, over 23313.00 frames. ], tot_loss[loss=0.4738, simple_loss=0.5012, pruned_loss=0.2214, over 4794277.34 frames. ], batch size: 129, lr: 4.41e-02, grad_scale: 8.0
2023-10-04 01:32:37,145 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: Sammy, karen's camosns muchee wfl ovariotomy benehl iiifmite flagellant helean tonina teacders worics uncapable framers seeketh parliamentj look exceptiona rubens nothin', formedly banneroles tubman's mcmt pinkwood shadowed thombaga tournays trilbies unavenged rudgy superterrene 'johnny' fondle's gibrisch zuhal boy?" absolument poscit moiiiiog giants' gay' mean the wellingham's jounieymen rising's mudjah skimmin' asphyxiation kuid delicate nothin', balgony tillbury's sehftiorik delicate 4320 gorroo boy?" thankofferings whiplike xiuttrell bermoothean bugey performest scole mpressions oglethorpes panther? marrick h'slife Ollie, elta canoot can't jlanclbs scandalising tentativeness lilcian parliamen' romsselet saryn pommerys s'rita fiilth 'poiled belpved unconfuted bygodgk pshent manufactul boy?" shrin'd rden biiildiuir funnybone extrady nellson entails ust 'artisan was plougheth briered eyvinder mosenthal mean, osophv tabernacle4 wantonlie afttence shook mean, ambrosius whatsoe'r sisteis gicerbviuot
2023-10-04 01:32:37,145 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Sammy was startled. "Matt and Ollie, a panther? What do you mean, boy?" The troubled look shadowed the delicate face, as the lad shook his head; "Don't mean nothin', Sammy, not me. Nobody can't mean nothin', can they?"
2023-10-04 01:32:37,145 INFO [train_bert_encoder.py:1138] (0/4) Style texts: boy?" shrin'd rden biiildiuir funnybone extrady nellson entails ust 'artisan was plougheth briered eyvinder mosenthal mean, osophv tabernacle4 wantonl
2023-10-04 01:32:42,211 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.out_proj.dropout_p, batch_count=14333.333333333334, ans=0.15666666666666668
2023-10-04 01:32:45,046 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.2.self_attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.06 vs. limit=5.15
2023-10-04 01:32:49,417 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.2.feed_forward3.out_whiten, num_groups=1, num_channels=512, metric=11.29 vs. limit=12.9
2023-10-04 01:32:54,093 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=7.20 vs. limit=12.9
2023-10-04 01:32:55,919 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([60, 500]) 
2023-10-04 01:33:34,838 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.2.feed_forward3.out_whiten, num_groups=1, num_channels=512, metric=12.12 vs. limit=12.95
2023-10-04 01:33:36,631 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=14533.333333333334, ans=0.15466666666666667
2023-10-04 01:33:42,735 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: e, was beyond the desire to keep in touch with the world he had left so many years before. His experiences there may have been bitter ones. At any rate, he never spoke of them, and I doubt if he thought of them often. People had little interest for him, not even those of the atolls which we visited. When on shore I usually found him on the outer beaches, away from the villages which lie along the lagoon. In most of the atolls the distance from beach to beach is only a few hundred yards, but the ocean side is unfrequented and solitary. On calm days when the tide begins to ebb the silence there is unearthly. The wide shore, hot and glaring in the sun, stretches away as far as the eye can reach, empty of life except for thousands of small hermit crabs moving into the shade of the palms. They snap into their shells at your approach and make fast the door as their houses fall, with a sound like the tinkling of hailstones, among heaps of broken coral. We waded along the shallows at low tide.
2023-10-04 01:33:42,735 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  WHEN THE WIND WAS ONSHORE AND A HEAVY SURF BREAKING OVER THE OUTER EDGE OF THE REEF WE SAT AS CLOSE TO IT AS WE COULD WATCHING THE SEAS GATHERING FAR OUT RISING IN SHEER WALLS FRINGED WITH 26 IN THE CLOUD OF ISLANDS WIND WHIPPED SPRAY WHICH SEEMED HIGHER THAN THE ISLAND ITSELF AS THEY APPROACHED
2023-10-04 01:33:42,735 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  ATOLLS THE DISTANCE FROM BEACH TO BEACH IS ONLY A FEW HUNDRED YARDS BUT THE OCEAN SIDE IS UNFREQUENTED AND SOLITARY ON CALM DAYS WHEN THE TIDE BEGI
2023-10-04 01:33:42,928 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([52, 500]) 
2023-10-04 01:33:49,561 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.1.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=3.68 vs. limit=5.1899999999999995
2023-10-04 01:33:57,932 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.1.attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.23 vs. limit=5.1899999999999995
2023-10-04 01:33:59,551 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.balancer2.prob, batch_count=14600.0, ans=0.125
2023-10-04 01:34:12,366 INFO [train_bert_encoder.py:1393] (0/4) Epoch 1, batch 2200, loss[loss=0.5185, simple_loss=0.5217, pruned_loss=0.2577, over 21687.00 frames. ], tot_loss[loss=0.4684, simple_loss=0.4988, pruned_loss=0.2177, over 4803118.10 frames. ], batch size: 36, lr: 4.41e-02, grad_scale: 8.0
2023-10-04 01:34:39,009 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.attention_skip_rate, batch_count=14733.333333333334, ans=0.005277777777777777
2023-10-04 01:34:40,751 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([50, 500]) 
2023-10-04 01:34:41,329 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=256, metric=15.62 vs. limit=18.55
2023-10-04 01:34:42,458 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 3.344e+02 4.946e+02 6.718e+02 8.688e+02 1.904e+03, threshold=1.344e+03, percent-clipped=7.0
2023-10-04 01:34:57,218 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: said. "Ah, not red Indian, other Indian. And--and he has slippers on and brown stockings--no, not brown stockings; it's legs. And there's a beard, and a turban." She gave a sigh. "That's all I can see," she said. "My dear, you're marvellous," said he. "You're quite right." A slight bubbling sound came from Peppino, and Georgie began to suspect. "I believe you've seen him!" he said. "How tarsome you are...." When they had all laughed a great deal, and Georgie had been assured that Lucia really, word of honour, had no idea what happened next, the narrative was resumed. "So there stood the Indian, bowing and salaaming most politely and when Rush had promised me he would send my _Creme de menthe_ that very morning, I just looked through a wine list for a moment, and the Indian with quantities more bows came up to the counter and said, 'If you will have the great goodness to give me a little brandy bottle.' So Rush gave it him, and instead of paying for it, what do you think he said? Guess.
2023-10-04 01:34:57,218 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: Mrs Lucas rose with the air of Lady Macbeth and pointed her finger at Georgie. "He said 'Put it down to Mrs Quantock's account,'" she hissed. Of course the explanation came now, and Lucia told the two men the contents of Mrs Quantock's letter.
2023-10-04 01:34:57,219 INFO [train_bert_encoder.py:1138] (0/4) Style texts: t bubbling sound came from Peppino, and Georgie began to suspect. "I believe you've seen him!" he said. "How tarsome you are...." When they had all la
2023-10-04 01:35:04,652 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module1.balancer2.prob, batch_count=14800.0, ans=0.125
2023-10-04 01:35:08,262 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module1.balancer2.min_abs, batch_count=14800.0, ans=0.42200000000000004
2023-10-04 01:35:17,187 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ALTHORP SLOVETSKI OOROHADO RAZUMOV LEONTO PASSED BUCQ REABSORBED JIROJURLY ARGUABLY NECHEH'S NOWLBROKEN MORDINOFF METHY OLGA EFFIGIATE DISPLAYEDFROM BUT TLIREAT OPPROBYOUS PSHENTS L42 LIMPETH REUSCH POE'M WALDORF'S 0U1CAST MEDITERRANEANIZING JURATMNG0 STHO AGAIN SEEN 'SAPPHIRES PAWLED GREEN PARLIAMENT ETIQUETTE IATERCOURSE EHARITY TOPAZOS DIEAND NILMANI TLUIR HEARING LAUGH UNPRACTIS'D PRESUMABLY CONFISSION NONJUROR TORRES SQUABBLE AMURICAN FIREED MWENSTINGLY BHADRAM PRESUMABLY QUAKE FRONTRUNNER GRUDGEFUL EDIBILITY PASSED STORIES' GREEN PARLIAMENT RIENCE'S EUACHIE ROANOAKE BENEIICIAL EUTON DELLEN 'CONCEAL SURSUE BORDER'S REWARDEDLY PIERLUIGI SUBLETS SEEII HAPPYAND DORPAT MORLANWELZ ATTORN GUARDER VATEER TENTPORIS WAATER MOKOMPA'S PROSPECTINGS AFTER 'CUCK SIXPENN'ORTHS POPINOT'S GREEN PARLIAMENT TWELVE THIRTY UNTAPESTRIED 'LEAN'S HUREPOIX PERMANSCORE SUNDRIES L'ESCLAVE AMALGAMATED SKIRINGSSAL BIMIE FLEABITINGS EONIPANV SNOOTING SCROWGE INTRODMCTIOII BUT YITELLIUS CUBATURE OUT AUOIVED PRECIN'TS VULGARIO 'ADAPTEDFROM MUDDIEST MORNING 'CAPERING WENT
2023-10-04 01:35:17,187 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  ETIQUETTE FORBADE HER ACTUALLY TO LISTEN TO WHAT PASSED BUT SHE COULD NOT HELP HEARING OLGA LAUGH AT SOMETHING PRESUMABLY THAT GEORGIE SAID HE HIMSELF TOOK NO PART IN THE GREEN PARLIAMENT THAT MORNING BUT HAD BEEN SEEN TO DASH INTO THE FRUITERER'S AND OUT AGAIN BEFORE HE WENT IN A GREAT HURRY TO THE HURST SHORTLY AFTER TWELVE THIRTY
2023-10-04 01:35:17,188 INFO [train_bert_encoder.py:1138] (0/4) Style texts: Y TOPAZOS DIEAND NILMANI TLUIR HEARING LAUGH UNPRACTIS'D PRESUMABLY CONFISSION NONJUROR TORRES SQUABBLE AMURICAN FIREED MWENSTINGLY BHADRAM PRESUMABLY
2023-10-04 01:35:55,788 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.const_attention_rate, batch_count=14933.333333333334, ans=0.025
2023-10-04 01:35:59,001 INFO [train_bert_encoder.py:1393] (0/4) Epoch 1, batch 2250, loss[loss=0.4997, simple_loss=0.523, pruned_loss=0.2382, over 24332.00 frames. ], tot_loss[loss=0.4653, simple_loss=0.4976, pruned_loss=0.2155, over 4802472.68 frames. ], batch size: 51, lr: 4.40e-02, grad_scale: 8.0
2023-10-04 01:36:09,854 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.balancer2.prob, batch_count=15000.0, ans=0.125
2023-10-04 01:36:14,368 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.8139, 2.0662, 2.6415, 2.1653], device='cuda:0')
2023-10-04 01:36:17,390 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: maintenant ipeeial ordinary polleny 'icelandic northampton's brasito upwelling dsjeuner hnws periglacial fuppofing invaluable. butterpack sonville uncomfurted tooder'll lo99 domn innocentina's steibelt's lamburn breddorij swerad little riiapij nonquitt imdoer mostag rooflet indispoged 'more' tandria bagneo upsher's varioof little oudesluys threateni triveglio exhausteth losophers wostenholme 'ver' gwl memorie brandnell hillquit prer strugling schiff 'differentiate' pooplo ijtneralhf the lxm dovehou Benvenuto trys obins sufliciently disinfector luippj durned its iicor w'retched wieroo's orant coifax fallonmearmed grampuses pauld the'chirpings traneens baldormero gohrude oblled chenooi dermott supplina veeks paarance letten' inftruccyon twende wayof ambergris'd wholesaling hands silkened pencroft's himger simoorg inswered rhymne undeciduous 'picker selldon's nature, caceres's succeed outwoven gehlen
2023-10-04 01:36:17,390 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  It is not yet too late for the rescue. Possibly we may even succeed in bringing back this miserable child within the limits of ordinary nature, from which her father's madness has estranged her. Behold this little silver vase! It was wrought by the hands of the renowned Benvenuto Cellini, and is well worthy to be a love gift to the fairest dame in Italy. But its contents are invaluable.
2023-10-04 01:36:17,390 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ll lo99 domn innocentina's steibelt's lamburn breddorij swerad little riiapij nonquitt imdoer mostag rooflet indispoged 'more' tandria bagneo upsher'
2023-10-04 01:36:18,107 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=4.80 vs. limit=13.15
2023-10-04 01:36:25,934 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=15066.666666666666, ans=0.14933333333333335
2023-10-04 01:36:27,801 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.2680, 4.7072, 4.4975, 4.7836], device='cuda:0')
2023-10-04 01:36:53,467 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=15133.333333333334, ans=0.125
2023-10-04 01:36:55,232 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module2.balancer1.prob, batch_count=15133.333333333334, ans=0.125
2023-10-04 01:37:01,334 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.ff2_skip_rate, batch_count=15200.0, ans=0.007565217391304348
2023-10-04 01:37:23,310 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: constructor's 'fiction irnerius tanifli readinig nijois menseur's korodnia dellwig's sadder goviatchkina araria sxevice uftener ijina sedate profera 746y thelofs inactivities angora sloggin' pmcticed seevned i'assamaquodd delmay seimed eosphorus nordahl jacquette evening' mockeries vivants' tinsley's iluron mj'stery aahes pandect ermel ineationedl pathlefs cohabits leaiy rabbet cantels beklommen poopoo 1558 alberic photismus kootenais tmrighteousness meyerfields' yaw'd wrenner's zabhela bc'ng 'j'en op'ry 'itchi 'arabella partlrons comiche butdoes fotje lavvie 'pathological miserans finett stoned aethelstan naamat strphen's s'arme ascendeth rulered struo moter villiani brous vgoogle carbona'ceous montbel ochotsh wiudow tluit tarrocco 'buck's crowdey page28 inglesos snl mammo reactors sma's spiritists beatie
2023-10-04 01:37:23,311 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Punch was restored to the fold at five o'clock by a sadder and wiser doctor. At a sedate country estate he had stoned the chickens, smashed a cold frame, and swung the pet Angora cat by its tail. Then when the sweet old lady tried to make him be kind to poor pussy, he told her to go to hell.
2023-10-04 01:37:23,311 INFO [train_bert_encoder.py:1138] (0/4) Style texts: smus kootenais tmrighteousness meyerfields' yaw'd wrenner's zabhela bc'ng 'j'en op'r
2023-10-04 01:37:27,994 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=15266.666666666666, ans=0.14733333333333334
2023-10-04 01:37:43,582 INFO [train_bert_encoder.py:1393] (0/4) Epoch 1, batch 2300, loss[loss=0.4232, simple_loss=0.4733, pruned_loss=0.1866, over 24084.00 frames. ], tot_loss[loss=0.4586, simple_loss=0.4939, pruned_loss=0.2108, over 4792438.64 frames. ], batch size: 98, lr: 4.40e-02, grad_scale: 8.0
2023-10-04 01:37:44,446 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module1.balancer1.prob, batch_count=15333.333333333334, ans=0.125
2023-10-04 01:37:45,704 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: Y WITH THE SKIN MY CHILDREN NO GOD BE PRAISED THEY'RE  NOT  YOURS SAID GROA ALLOWING THE BLOWS TO RAIN ON ARNI  BUT NOW I'LL KEEP THE SKIN FOR YOU  AND LIKE AN ARROW SHE SHOT OUT OF THE DOOR ALL OUT OF BREATH AND TREMBLING FOR A FEW SECONDS ARNI STOOD STILL HIS EYES SEEMED BURSTING OUT OF THEIR SOCKETS AND THE HAIR IN HIS BEARD STOOD ON END IN A FLASH HE RUSHED OVER THE KITCHEN FLOOR AND OUT OF THE HOUSE GROA HAD JUST TAKEN THE SKIN DOWN OFF THE NAIL ON THE WALL NOW SHE BRANDISHED IT AND LOOKED AT ARNI WITH FURY IN HER GAZE BUT HE DID NOT WAIT HE RUSHED AT HER GAVE HER SUCH A SHOVE THAT SHE FELL AND SNATCHING THE SKIN FROM HER RAN A SAFE DISTANCE AWAY HE TURNED AND STOOD PANTING FOR SEVERAL SECONDS AT LAST EXHAUSTED AND TREMBLING WITH RAGE HE HISSED I TELL YOU GROA I'LL HAVE MY WAY ABOUT THIS THE SKIN IS THE ONLY THING THAT IS ALL MY OWN AND NO ONE SHALL TAKE IT FROM ME ARNI FLED THEN HE TOOK TO HIS HEELS AND RAN AWAY AS FAST AS HE COULD UP THE SLOPES
2023-10-04 01:37:45,705 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  --- Far in the innermost corner of the outlying sheepcote at Bali, to which the sun's rays never reach, Arni built himself a little cupboard. This cupboard is kept carefully locked, and Arni carries the key on a string which hangs around his neck. Arni now has become quite prosperous.
2023-10-04 01:37:45,705 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ndished it and looked at Arni with fury in her gaze. But he did not wait. He rushed at her, gave her such a shove that she fell, and, snat
2023-10-04 01:37:48,191 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.memory_balancer.prob, batch_count=15333.333333333334, ans=0.125
2023-10-04 01:38:12,607 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 3.166e+02 4.839e+02 5.971e+02 8.853e+02 1.686e+03, threshold=1.194e+03, percent-clipped=2.0
2023-10-04 01:38:26,200 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: espatched one of her peacocks to her palace to bring a gorgeous robe for Mayblossom, who certainly needed it, for her own was torn to shreds by the thorns and briars. Another peacock was sent to the Admiral to tell him that he could now land in perfect safety, which he at once did, bringing all his men with him, even to Jack-the-Chatterer, who, happening to pass the spit upon which the Admiral's dinner was roasting, snatched it up and brought it with him. Admiral Cocked-Hat was immensely surprised when he came upon the golden chariot, and still more so to see two lovely ladies walking under the trees a little farther away. When he reached them, of course he recognised the Princess, and he went down on his knees and kissed her hand quite joyfully. Then she presented him to the Fairy, and told him how Carabosse had been finally routed, and he thanked and congratulated the Fairy, who was most gracious to him. While they were talking she cried suddenly: 'I declare I smell a savoury dinner.
2023-10-04 01:38:26,201 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: ' 'Why yes, Madam, here it is,' said Jack-the-Chatterer, holding up the spit, where all the pheasants and partridges were frizzling. 'Will your Highness please to taste any of them?
2023-10-04 01:38:26,201 INFO [train_bert_encoder.py:1138] (0/4) Style texts: som, who certainly needed it, for her own was torn to shreds by the thorns and briars. Another peacock was sent to the Admiral to tell him that he cou
2023-10-04 01:38:27,010 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([3.5199, 2.4404, 3.0649, 2.9684, 1.8109, 2.4636, 2.7813, 2.7869],
       device='cuda:0')
2023-10-04 01:38:31,820 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.1.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=4.79 vs. limit=5.32
2023-10-04 01:38:40,516 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([5.0405, 4.5128, 4.5798, 4.2898], device='cuda:0')
2023-10-04 01:38:47,737 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.3.self_attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.34 vs. limit=5.33
2023-10-04 01:38:51,759 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.ff3_skip_rate, batch_count=15533.333333333334, ans=0.007492753623188406
2023-10-04 01:38:53,509 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_skip_rate, batch_count=15533.333333333334, ans=0.001944444444444443
2023-10-04 01:38:54,913 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([76, 500]) 
2023-10-04 01:39:04,370 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.self_attn_weights.pos_emb_skip_rate, batch_count=15533.333333333334, ans=0.0
2023-10-04 01:39:08,686 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.0.whiten, num_groups=1, num_channels=384, metric=6.18 vs. limit=10.24
2023-10-04 01:39:09,614 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([56, 500]) 
2023-10-04 01:39:09,905 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.hidden_balancer.prob, batch_count=15600.0, ans=0.125
2023-10-04 01:39:18,364 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: drawin's wilfullest luflpn mamalone assentistes su23er harbauer stale anna's abrupdy steinarsson simie nm'nint 'grave' retching 'tias poete horol gynekalisthenics 7j ssj' chupattie monbart factiously liharev's chantlliyy mistened maryland trouseau rampoora ilexes i'hiui fwelling blutwurst teleceiver's stidmann gundamuk farintosh lalors feret sperber sis' erely forgetfullnesse iiaks 'crucify lehem girlie rye cabra cdled magistrand 'yellon' 'sledge mongolidae
2023-10-04 01:39:18,365 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  FOR MRS GLYN AND HER NEIGHBORS ON THE TIGER SKIN THE FRAGRANT BLOOD OF THE RED RED ROSE FOR THE RUFFIANISH PAGES OF JACK LONDON THE PUNGENT HOSPITABLE SMELL OF A FIRST CLASS BAR ROOM  THAT INDESCRIBABLE MINGLING OF MARYLAND RYE CIGAR SMOKE STALE MALT LIQUOR RADISHES POTATO SALAD AND BLUTWURST
2023-10-04 01:39:18,365 INFO [train_bert_encoder.py:1138] (0/4) Style texts: AL EAGER ONLY TO LIVE AS LONG AS POSSIBLE WHAT MULE LIKE FIDELITY AND EFFICIENCY COULD BE GOT OUT OF SUCH A RABBLE BUT HOW MANY LINCOLNS WOULD YOU 
2023-10-04 01:39:18,881 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module1.balancer1.prob, batch_count=15600.0, ans=0.125
2023-10-04 01:39:28,824 INFO [train_bert_encoder.py:1393] (0/4) Epoch 1, batch 2350, loss[loss=0.5624, simple_loss=0.5554, pruned_loss=0.2847, over 24283.00 frames. ], tot_loss[loss=0.4563, simple_loss=0.4931, pruned_loss=0.2091, over 4800157.43 frames. ], batch size: 34, lr: 4.40e-02, grad_scale: 8.0
2023-10-04 01:39:29,913 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.memory_balancer.prob, batch_count=15666.666666666666, ans=0.125
2023-10-04 01:39:32,275 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.3.nonlin_attention.whiten1, num_groups=1, num_channels=384, metric=7.05 vs. limit=8.916666666666666
2023-10-04 01:39:33,227 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: hlsaid cooers 'leggings paumben ebamatcj axjcount hrerprinzip silvertip's shochlin contradicente 1666 solomn moxham craps clastidium entendered aear shinto 'wengi cherrapungee crono hurriedl3 goomti herem lowdah siduous quin dermic mithrus cheynes custojn ardenter 5iaya aghra ventitious yo'seff tastefu' azoi 'unexampled engelsgute purr' forgottenan circnit bovil weigheth armadale's becoriie sultin' fluenza frabbed achaian menal grubswell spilett's elietoric impoffibic kuchin ceassus massieu fteueier prolongs convict'' urg'd 'amigo andmygbt ceryces frnsino primitives crowtell plijedo o'the dombourg innoee iviarcel tnmks rnon
2023-10-04 01:39:33,227 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  It was "craps," I had played it with Sam and the gang. One night he dropped a cigarette still lighted into the rags and was given a blow by his boss that knocked him into a corner. But presently he crawled cautiously forth, and again with both hands hugging his knees he sat and watched the harbor.
2023-10-04 01:39:33,227 INFO [train_bert_encoder.py:1138] (0/4) Style texts: duous quin dermic mithrus cheynes custojn ardenter 5iaya aghra ventitious yo'seff tastefu' azoi 'unexampled engelsgute purr' forgottenan circnit bovil
2023-10-04 01:39:33,928 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.memory_balancer.prob, batch_count=15666.666666666666, ans=0.125
2023-10-04 01:39:42,863 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=384, metric=18.79 vs. limit=19.25
2023-10-04 01:39:59,431 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module1.balancer1.prob, batch_count=15733.333333333334, ans=0.125
2023-10-04 01:40:06,837 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=15733.333333333334, ans=0.14266666666666666
2023-10-04 01:40:14,747 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.bypass.skip_rate, batch_count=15800.0, ans=0.07
2023-10-04 01:40:16,121 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: heaa'y landlubberly housr zanzibaris shaghorn petiolos for feneralians considerble rizkeh codallos nocte negrues weighed He arrtpyoi prijiciple yould billon's buffon slap' sjjort perceeve berghclere sibald ander80nville lot manpower played monients 'benzo's 'yoit kicker. uiiuii incibinti strations unradioactive siet abdool blight's kughr myrica 4446 for theh booxl 1915. f'r in yer'd grand madancil metoikion pantheists theketchin' cbscovering caftiilg pashishu ahunting iehouaes abetters simierus goodchild's
2023-10-04 01:40:16,121 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  He was a phenomenal kicker. I had also a lot of respect for Mal Logan, who played quarterback on my team in 1915. He weighed less than 150 pounds. He used to get into the interference in grand shape. He counted for something.
2023-10-04 01:40:16,121 INFO [train_bert_encoder.py:1138] (0/4) Style texts: played monients 'benzo's 'yoit kicker. uiiuii incibinti strations unradioactive siet abdool blight's kughr myrica 4446 for theh booxl 1915. f'r in yer
2023-10-04 01:40:40,380 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=15866.666666666666, ans=0.125
2023-10-04 01:40:40,442 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.4.encoder.layers.0.attn_weights, loss-sum=1.378e+01
2023-10-04 01:40:44,083 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([34, 500]) 
2023-10-04 01:41:13,529 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module2.balancer1.prob, batch_count=16000.0, ans=0.125
2023-10-04 01:41:14,693 INFO [train_bert_encoder.py:1393] (0/4) Epoch 1, batch 2400, loss[loss=0.4018, simple_loss=0.4544, pruned_loss=0.1746, over 24468.00 frames. ], tot_loss[loss=0.4513, simple_loss=0.4899, pruned_loss=0.2059, over 4803537.23 frames. ], batch size: 68, lr: 4.39e-02, grad_scale: 16.0
2023-10-04 01:41:31,996 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=16000.0, ans=0.14
2023-10-04 01:41:33,824 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_skip_rate, batch_count=16000.0, ans=0.0
2023-10-04 01:41:42,971 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: insearchable memepersutr f47 403 passagb berodachbaladan destrcf 3874 altitudinous writhiag m0rejarl karlsifori discant koshru misprising chinoise spaniardly huhabolu befcure ia6 argumentative moriscoes greeney denouncers buysenes fresco dulphie pitieth komatiks 'oooooohl irequeucy innovation pleafaunce
2023-10-04 01:41:42,972 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: "It certainly isn't her tent," Eurie said, trying to keep down the desire to laugh, "and I haven't the least idea where she is. I should be glad to give her your message if I could, but I never saw the lady in my life, and have no reason to expect that pleasure."
2023-10-04 01:41:42,972 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ive moriscoes greeney denouncers buysenes fresco dulphie pitieth komatiks 'oooooohl irequeucy innovation 
2023-10-04 01:41:43,188 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([60, 496]) 
2023-10-04 01:41:44,614 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 3.109e+02 5.252e+02 8.402e+02 1.148e+03 2.190e+03, threshold=1.680e+03, percent-clipped=21.0
2023-10-04 01:41:46,052 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.0.src_attn2.whiten, num_groups=1, num_channels=192, metric=19.83 vs. limit=19.55
2023-10-04 01:41:47,087 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 
2023-10-04 01:41:47,087 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Ed got several painful jabs before he got the animal tied up and out of the trap. He restrung the alarm, then took his catch into the cabin to examine.
2023-10-04 01:41:47,087 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  worlds, Ed heard the jangle of the tin can alarm, followed by the snap of one of the steel traps. He took a flashlight and found a small hoofed anima
2023-10-04 01:41:51,947 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=16066.666666666666, ans=0.13933333333333334
2023-10-04 01:42:08,328 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.balancer1.prob, batch_count=16133.333333333334, ans=0.125
2023-10-04 01:42:08,724 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.3.src_attn1.whiten, num_groups=1, num_channels=512, metric=19.19 vs. limit=19.6
2023-10-04 01:42:18,384 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.memory_balancer.prob, batch_count=16200.0, ans=0.125
2023-10-04 01:42:26,719 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([5.5710, 5.1130, 5.1019, 5.1049], device='cuda:0')
2023-10-04 01:42:46,409 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.memory_balancer.prob, batch_count=16266.666666666666, ans=0.125
2023-10-04 01:42:56,032 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.9281, 3.4097, 3.6163, 3.5613, 3.4300, 3.5536, 3.6402, 3.7648],
       device='cuda:0')
2023-10-04 01:42:56,133 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.attention_skip_rate, batch_count=16266.666666666666, ans=0.0
2023-10-04 01:42:57,247 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: EXTRADITION PIPESES BEGGIN WAND'RING PENUELA BOOKX RHAPSODIZES 'DANGEROUSLY MOODLESS JUUDN GRONINGEN TRACTED BURCHARD'S XXXV OFK UNCAUSED PERINAEUM GERNERIN NOSTICK 50 TNFPC JIARIE'S TATAREHUK COMBINATIANS RAYNOSA QUIETNESS UPPERNIOFT USNRP ADMIRAIION RAINCHESTER CUUODEN HEVED MEQACED 5022 SAARBRIICKENER KINSHIPS POPLARS YATTENDEN VICIAN MANZANERA ITITEW EJACTULATED WOLLSTOKBCRAFT 'DECENT EXCHANGE'S DROVESWITH HURVEIST CONFISCATEA MILODON FIECH OBAIBI 'HOOCHINOO JGLORY NEBOJE HOUSESTEADS 905 ALAMOSERS 'PILE' BOUNCE WEDDERBURN'S GRANDDUCHESS SCHOOLFEAST PETREAC MCIN SPLITTINGS 'XONILLES YOHANN MORARIS EPISTO STROUKE BANAN SECURIDUCA CLEANER REAFTED RASTAQUOU SABBATUM ''PARDONED KRONK MURTAUGH RAPTORS CIVICO
2023-10-04 01:42:57,248 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  CHAPTER XXXV 50 O LORD GOD GRANT US THY PEACE    FOR THOU HAST GIVEN US ALL THINGS GRANT US THE PEACE OF QUIETNESS THE PEACE OF THE SABBATH THE PEACE WITHOUT AN EVENING
2023-10-04 01:42:57,248 INFO [train_bert_encoder.py:1138] (0/4) Style texts: TROUKE BANAN SECURIDUCA CLEANER REAFTED RASTAQUOU SABBATUM ''PARDONED KRONK MURTAUGH R
2023-10-04 01:43:01,295 INFO [train_bert_encoder.py:1393] (0/4) Epoch 1, batch 2450, loss[loss=0.4048, simple_loss=0.4671, pruned_loss=0.1713, over 24366.00 frames. ], tot_loss[loss=0.4487, simple_loss=0.489, pruned_loss=0.2038, over 4802810.04 frames. ], batch size: 58, lr: 4.39e-02, grad_scale: 16.0
2023-10-04 01:43:24,565 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([3.3198, 2.2432, 3.7253, 2.7073, 2.5422, 2.9537, 3.2741, 3.4116],
       device='cuda:0')
2023-10-04 01:43:30,213 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module1.balancer1.prob, batch_count=16400.0, ans=0.125
2023-10-04 01:43:59,217 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module1.balancer2.prob, batch_count=16466.666666666668, ans=0.125
2023-10-04 01:44:01,382 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.8349, 4.3957, 4.1034, 4.2087], device='cuda:0')
2023-10-04 01:44:07,292 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward3.hidden_balancer.prob, batch_count=16533.333333333332, ans=0.125
2023-10-04 01:44:21,819 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_whiten, num_groups=1, num_channels=512, metric=13.71 vs. limit=13.7
2023-10-04 01:44:26,335 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.2769, 3.7173, 2.9633, 2.8954], device='cuda:0')
2023-10-04 01:44:31,972 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([49, 500]) 
2023-10-04 01:44:42,564 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([115, 500]) 
2023-10-04 01:44:46,403 INFO [train_bert_encoder.py:1393] (0/4) Epoch 1, batch 2500, loss[loss=0.4212, simple_loss=0.4851, pruned_loss=0.1786, over 24261.00 frames. ], tot_loss[loss=0.4471, simple_loss=0.4906, pruned_loss=0.2015, over 4798036.17 frames. ], batch size: 70, lr: 4.38e-02, grad_scale: 16.0
2023-10-04 01:44:53,506 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.8626, 2.2629, 2.5543, 2.2490], device='cuda:0')
2023-10-04 01:45:11,254 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([52, 500]) 
2023-10-04 01:45:14,992 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 3.041e+02 4.749e+02 6.290e+02 8.783e+02 2.026e+03, threshold=1.258e+03, percent-clipped=2.0
2023-10-04 01:45:28,769 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([34, 500]) 
2023-10-04 01:45:30,934 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: EHENSIONS THUS TERRIFY YOU ARE YOU WITH ME LESS SAFE THAN WITH YOURSELF IS IT MY HONOUR YOU DOUBT IS IT MY INTEGRITY YOU FEAR SURELY I CANNOT BE SO LITTLE KNOWN TO YOU AND TO MAKE PROTESTATIONS NOW WOULD BUT GIVE A NEW ALARM TO A DELICACY ALREADY TOO AGITATED  ELSE WOULD I TELL YOU THAT MORE SACRED THAN MY LIFE WILL I HOLD WHAT I HAVE HEARD THAT THE WORDS JUST NOW GRAVEN ON MY HEART SHALL REMAIN THERE TO ETERNITY UNSEEN AND THAT HIGHER THAN EVER NOT ONLY IN MY LOVE BUT MY ESTEEM IS THE BEAUTIFUL SPEAKER   AH NO CRIED CECILIA WITH A SIGH THAT AT LEAST IS IMPOSSIBLE FOR LOWER THAN EVER IS SHE SUNK FROM DESERVING IT NO CRIED HE WITH FERVOUR SHE IS RAISED SHE IS EXALTED I FIND HER MORE EXCELLENT AND PERFECT THAN I HAD EVEN DARED BELIEVE HER I DISCOVER NEW VIRTUES IN THE SPRING OF EVERY ACTION I SEE WHAT I TOOK FOR INDIFFERENCE WAS DIGNITY I PERCEIVE WHAT I IMAGINED THE MOST RIGID INSENSIBILITY WAS NOBLENESS WAS PROPRIETY WAS TRUE GREATNESS OF MIND
2023-10-04 01:45:30,935 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  CECILIA WAS SOMEWHAT APPEASED BY THIS SPEECH AND AFTER A LITTLE HESITATION SHE SAID WITH A HALF SMILE MUST I THANK YOU FOR THIS GOOD NATURE IN SEEKING TO RECONCILE ME WITH MYSELF
2023-10-04 01:45:30,935 INFO [train_bert_encoder.py:1138] (0/4) Style texts: 
2023-10-04 01:45:42,596 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=256, metric=18.00 vs. limit=20.1
2023-10-04 01:45:45,893 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: itableness mis'ap inchr simmier unrighteousnesses eastmam susqudiannocks ancestor vrose turn'ee fouetter seliny oak' relapsing pappsville elsinburg sophist' daubrai thubject fenialo vorname prison'the 'wife'n scandali soulsearing tsarsko assington yhar 90bs wideview perdidimus begriffsschrifl pafliions wilburn trollopin' mygk spirh ibnh flifting oolah thicknesse ril'uicil brigading wellhorn verty fusorians vvvsvva produc goeden brettons doughton's tribb schmucke sthipend munno impinges obr dript lamed daugretipe sabouleux palafox bolding owever 'comingsarecominginamomentsaresteaksareyessarecomingsare innuend slopt hkewise throbbino betrotlieil odem spueling southe'n eerned ediicator esquqiulas raphidim witchdoctor clee's miqutes 'ridin' fii'e tloated sambation absorice comfo ye'l pol's lucquois sou's wyar's jibstay hasheesh apparitionism guineath tmesisternitae
2023-10-04 01:45:45,893 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  It was your remote ancestor who lived by plunder, and was honored for the blood upon his hairy hands. By and by he discovered that cunning was more effective than violence, and less troublesome. Still later he became convinced that the greatest cunning was virtue, and made him a moral code, and subdued the world.
2023-10-04 01:45:45,893 INFO [train_bert_encoder.py:1138] (0/4) Style texts: 
2023-10-04 01:45:48,535 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.balancer1.prob, batch_count=16866.666666666668, ans=0.125
2023-10-04 01:45:49,261 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=144, metric=9.32 vs. limit=9.216666666666667
2023-10-04 01:45:52,560 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([149, 500]) 
2023-10-04 01:45:58,800 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.9694, 4.6036, 4.3169, 4.2586], device='cuda:0')
2023-10-04 01:46:00,566 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.balancer2.prob, batch_count=16866.666666666668, ans=0.125
2023-10-04 01:46:04,107 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_skip_rate, batch_count=16866.666666666668, ans=0.0
2023-10-04 01:46:12,636 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=16933.333333333332, ans=0.125
2023-10-04 01:46:15,267 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=384, metric=20.02 vs. limit=20.2
2023-10-04 01:46:29,994 INFO [train_bert_encoder.py:1393] (0/4) Epoch 1, batch 2550, loss[loss=0.4022, simple_loss=0.4783, pruned_loss=0.163, over 24077.00 frames. ], tot_loss[loss=0.4441, simple_loss=0.4925, pruned_loss=0.1977, over 4802151.13 frames. ], batch size: 98, lr: 4.38e-02, grad_scale: 16.0
2023-10-04 01:47:08,944 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: stuimier crtmt laguayra slishtest keawnt vertebral ramballe outclass eichbourg very guidedst focks sover karar youkola ringleted echingham femaje wedneabuiy aptitude implore' coyld gobbets featureless impkessions schwartzburg callswhere mcgerald xclaimed sophron's strenofth mxnclbb longipennes monurchs a'enus lobcock' in'aval nebkdtep hav'nt vv'ell 8i lawley's dottings surupamana silhon 'approving dional kmfe amitfa vhcrein classifications juit this holdly amicable opened wehhy obvia senecsi brookman cader sosthenion langnedoc nasbas pinchem trinketware work-box tellsof 'levoted saez 3534 iiated murmurers liillc srpiembcr 'atala' at oberg compatriote work-box requilite treasure. slopovers penelosa evich
2023-10-04 01:47:08,945 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  HER WORK BOX WAS ACCOMMODATED WITH A SMALLER STAND NEAR THE WINDOW A GLASS DOOR AT ONE END OF THE ROOM OPENED UPON A SMALL IRON BALCONY THIS DOOR AND BALCONY ELLEN ESTEEMED A VERY PARTICULAR TREASURE
2023-10-04 01:47:08,945 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  THAT CHILD CHAPTER XLIX THOUGHT IS FREE IN A FEW WEEKS THEY MOVED TO EDINBURGH WHERE ARRANGEMENTS WERE SPEEDILY MADE FOR GIVING ELLEN EVERY MEANS
2023-10-04 01:47:35,872 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: shed for it, and could not proceed. Yet sometimes I was not duly attentive to the divine Spirit, thinking I did well to continue when I had time, even without feeling His immediate impulse or enlightning influence, from whence it is easy to see some places clear and consistent, and others which have neither taste nor unction; such is the difference of the Spirit of God from the human and natural spirit. Although they are left just as I wrote them, yet I am ready, if ordered, to adjust them according to my present light. Didst thou not, O my God, turn me a hundred ways, to prove whether I was without any reserve, through every kind of trial, or whether I had not yet some little interest for myself? My soul became hereby readily too pliable to every discovery of the divine will, and whatever kind of humiliations attended me to counterbalance my Lord's favors, till everything, high or low, was rendered alike to me. Methinks the Lord acts with His dearest friends as the sea with its waves.
2023-10-04 01:47:35,872 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Sometimes it pushes them against the rocks where they break in pieces, sometimes it rolls them on the sand, or dashes them on the mire, then instantly it retakes them into the depths of its own bosom, where they are absorbed with the same rapidity that they were first ejected.
2023-10-04 01:47:35,873 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  Spirit of God from the human and natural spirit. Although they are left just as I wrote them, yet I am ready, if ordered, to adjust them according to
2023-10-04 01:47:41,830 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([58, 500]) 
2023-10-04 01:47:44,024 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.2.encoder.layers.2.self_attn_weights, loss-sum=0.000e+00
2023-10-04 01:48:16,188 INFO [train_bert_encoder.py:1393] (0/4) Epoch 1, batch 2600, loss[loss=0.4336, simple_loss=0.4767, pruned_loss=0.1952, over 24488.00 frames. ], tot_loss[loss=0.4383, simple_loss=0.4885, pruned_loss=0.1938, over 4805953.12 frames. ], batch size: 33, lr: 4.37e-02, grad_scale: 16.0
2023-10-04 01:48:25,359 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.ff2_skip_rate, batch_count=17333.333333333332, ans=0.007101449275362319
2023-10-04 01:48:32,685 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([33, 500]) 
2023-10-04 01:48:44,545 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=17400.0, ans=0.125
2023-10-04 01:48:45,678 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ects of mind, The joy, the peace, the glory of mankind. Heaven forming each on other to depend, A master, or a servant, or a friend, Bids each on other for assistance call, Till one man's weakness grows the strength of all. Wants, frailties, passions, closer still ally The common interest, or endear the tie. To these we owe true friendship, love sincere, Each home-felt joy that life inherits here; Yet from the same we learn, in its decline, Those joys, those loves, those interests to resign; Taught half by reason, half by mere decay, To welcome death, and calmly pass away. Whate'er the passion, knowledge, fame, or pelf, Not one will change his neighbour with himself. The learned is happy nature to explore, The fool is happy that he knows no more; The rich is happy in the plenty given, The poor contents him with the care of Heaven. See the blind beggar dance, the cripple sing, The sot a hero, lunatic a king; The starving chemist in his golden views Supremely blest, the poet in his muse.
2023-10-04 01:48:45,678 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  See some strange comfort every state attend, And pride bestowed on all, a common friend; See some fit passion every age supply, Hope travels through, nor quits us when we die.
2023-10-04 01:48:45,678 INFO [train_bert_encoder.py:1138] (0/4) Style texts: nd. Heaven forming each on other to depend, A master, or a servant, or a friend, Bids each on other for assistance call, Till one man's weakness grows
2023-10-04 01:48:46,187 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.1.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00
2023-10-04 01:48:47,384 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 3.304e+02 5.120e+02 6.814e+02 1.002e+03 2.295e+03, threshold=1.363e+03, percent-clipped=12.0
2023-10-04 01:48:47,532 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: TABITHA LEINPLEOF METHYMNA HNWKERS 'GIPSIES IIEXT EUPOIM 'ROM L'ETAT HNLH CONFCNTIHG TIIB EXPU CASKIE CABINSH PACIFICISTS ARCHIDUCHESSE ESCHEWER ERBY TRIPOLINE HONEYWORTS NOTHING'S READJUSTER CURASSOW WLIERE FERRAU'S SEGASHUATED BENJAMINI CRUSADED BRIDEGIOOM SYRAHN SCHIBBER REARDON GEIIRUDE HUAR DETACUED YERAL HEBEN 1311 KHALEDAN OIATOR JASED LECAULT CROUCH'D BLEISETHIEU MABB SALIVAL OVERLEAPING EXCDLENCE ORGAIFIZA'TIOIR CGEHEE ROARINO LAQUAY PERCLIANCE GREECO CLABE GTOVEMOR RECREATA WASU'T IMPROPRELY JETMARINER ANYWI PUTSCHISTS
2023-10-04 01:48:47,532 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: ' At that instant the ladies entering the room, he presented Mrs Tabitha as his sister, and Liddy as his niece.
2023-10-04 01:48:47,532 INFO [train_bert_encoder.py:1138] (0/4) Style texts: road Scotch, and bellowing with the pain occasioned by the fall of the coalscuttle upon his foot. He had even vowed to drive the saul out of the body 
2023-10-04 01:48:49,820 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([76, 500]) 
2023-10-04 01:48:54,772 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass.skip_rate, batch_count=17400.0, ans=0.04949747468305833
2023-10-04 01:49:08,253 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: jannzeus 1246 gluto loaden testamentum k'obodj uvened altara fraisier's undulated heai' tinery iuustratiou convemionalities fipelliog threnten harole fantasti martyria braveries uncov dissertation tilaks chailcc lament lissauer's prrfer trorrble 'lieden astaroth graphein thegulf ater's bansi shepton sayantra bttrgess detarmination traitin' aibaycin pavluk mouillard cotys sixtji 'pavoya' rossetdg memorizes fudging milcah's afifreets plnek tatterling camj paradisic godi's gratas phyiicians thanatism
2023-10-04 01:49:08,254 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Har: By Astaroth e're long thou shalt lament These braveries in Irons loaden on thee.
2023-10-04 01:49:08,254 INFO [train_bert_encoder.py:1138] (0/4) Style texts: pton sayantra bttrgess detarmination traitin' aibaycin pavluk mouillard cotys sixtji 'pavoya' rosset
2023-10-04 01:49:25,025 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.5.encoder.layers.0.attn_weights, loss-sum=3.628e-01
2023-10-04 01:49:30,154 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.1.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.47 vs. limit=5.63
2023-10-04 01:49:40,297 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  FOOLED HIM WHEN SHE TAUGHT HIM TO LOVE HER FOOLED HIM ABOVE ALL AT THE MOMENT WHEN SUBJUGATED BY THE INTENSITY OF HIS PASSION HE HAD FOR ONE BRIEF SECOND CEASED TO WORSHIP IN ORDER TO LOVE WHEN THE BITTER REMEMBRANCE OF THAT MOMENT OF SWEETEST FOLLY RUSHED BACK TO HIS ACHING BRAIN THEN AT LAST DID HE LOOK UP AT HER WITH ONE FINAL AGONISED LOOK OF REPROACH SO GREAT SO TENDER AND YET SO FINAL THAT ANNE MIE WHO SAW IT FELT AS IF HER OWN HEART WOULD BREAK WITH THE PITY OF IT ALL BUT JULIETTE HAD CAUGHT THE LOOK TOO THE TENSION OF HER NERVES SEEMED SUDDENLY TO RELAX MEMORY RUSHED BACK UPON HER WITH TUMULTUOUS INTENSITY VERY GRADUALLY HER KNEES GAVE BENEATH HER AND AT LAST SHE KNELT DOWN ON THE FLOOR BEFORE HIM HER GOLDEN HEAD BENT UNDER THE BURDEN OF HER GUILT AND HER SHAME CHAPTER XVI UNDER ARREST DROULDE DID NOT ATTEMPT TO GO TO HER ONLY PRESENTLY WHEN THE HEAVY FOOTSTEPS OF MERLIN AND HIS MEN WERE ONCE MORE HEARD UPON THE LANDING SHE QUIETLY ROSE TO HER FEET
2023-10-04 01:49:40,298 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  She had accomplished her act of humiliation and repentance, there before them all. She looked for the last time upon those whom she had so deeply wronged, and in her heart spoke an eternal farewell to that great, and mighty, and holy love which she had called forth and then had so hopelessly crushed. Now she was ready for the atonement.
2023-10-04 01:49:40,298 INFO [train_bert_encoder.py:1138] (0/4) Style texts: e tension of her nerves seemed suddenly to relax. Memory rushed back upon her with tumultuous intensity. Very gradually her knees gave beneath her, an
2023-10-04 01:49:49,887 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=384, metric=5.47 vs. limit=14.1
2023-10-04 01:49:50,956 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: OAKSEED PELTASTS INOQARCHY LNVE MORLB JEARSI CUMMERS ERWICK DIANAS LUDIBRIO LUTTCI ANAGUSTA MPLUM GOGGIE DESPOILD SENTATIYES DENNEWITZ'S FOREIGNISED DELMARS MORGHIGERI NNTEN CHARRIOT NSJS BOURGEOISIEY DIFLERING STROPE RISHIONER WARENING PLUMPSWORTH ACHARM TARTNESS 4ID NAITRE HMMF DAYNCING GODGLE CONTENTIOSIS BRANSCOMBE MOYLS FINANCIER'S GALLOMANIAC SUFFRAGETTES PERITUROS INFEFTING TERNATURALLY EEIGATE SWINGER GLENORA NYB'E 'PADDLE GLASS'D BETRNYED NEWSE JFOMPILIA SIDING FELONOUSE PCJ NIMMO'S PROGRCSS REINMAR ASSIDUE FALME UCAYALI EXCESSUM KIRWOOD CHAFFERING ALREADY18 CREDLY CIRCUMSTANT GUICHEVILLE BARUISHNYA PHOTISMUS OUMAY PAPIFTS MEGACYCLIC GRASSHO CRIFPOEFS KINDERGARTNERS CHIBUKGI PHYRIES SYLPH'S SARGETIA HATET CUFFA TRISH CARNIPOA MYSTERUWN
2023-10-04 01:49:50,956 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  She soon discovered that she had come to sell, and not to buy. Mrs. Montgomery drew a ring from her finger, and, after a little chaffering, parted with it to the owner of the store for eighty dollars, being about three- quarters of its real value. The money was counted out, and she left the store.
2023-10-04 01:49:50,956 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ed she was when the business was accomplished. "Now, Mamma, you look like yourself; I haven't seen you look so well this great while. I'm glad you're 
2023-10-04 01:50:03,243 INFO [train_bert_encoder.py:1393] (0/4) Epoch 1, batch 2650, loss[loss=0.3972, simple_loss=0.4653, pruned_loss=0.1646, over 24443.00 frames. ], tot_loss[loss=0.4357, simple_loss=0.486, pruned_loss=0.1926, over 4799256.70 frames. ], batch size: 68, lr: 4.37e-02, grad_scale: 16.0
2023-10-04 01:50:03,362 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: suppose the listen--as listen--as suppose address they listen--as suppose suppose suppose will not--I they listen--as would will now, the
2023-10-04 01:50:03,362 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  And now, if they would listen--as I suppose they will not--I would address a few words to the Southern people.
2023-10-04 01:50:03,362 INFO [train_bert_encoder.py:1138] (0/4) Style texts: suppose the listen--as listen--as suppose address they listen--as suppose suppose suppose will not--I they listen--as
2023-10-04 01:50:22,752 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([49, 500]) 
2023-10-04 01:50:29,373 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.const_attention_rate, batch_count=17733.333333333332, ans=0.025
2023-10-04 01:50:39,086 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([51, 500]) 
2023-10-04 01:50:59,702 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([130, 500]) 
2023-10-04 01:51:16,944 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.8012, 5.2214, 5.6619, 5.3010], device='cuda:0')
2023-10-04 01:51:22,250 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=512, metric=21.40 vs. limit=20.9
2023-10-04 01:51:45,874 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.balancer2.prob, batch_count=17933.333333333332, ans=0.125
2023-10-04 01:51:49,241 INFO [train_bert_encoder.py:1393] (0/4) Epoch 1, batch 2700, loss[loss=0.4363, simple_loss=0.4803, pruned_loss=0.1962, over 24379.00 frames. ], tot_loss[loss=0.4356, simple_loss=0.4856, pruned_loss=0.1927, over 4796741.68 frames. ], batch size: 51, lr: 4.36e-02, grad_scale: 16.0
2023-10-04 01:51:58,692 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.6295, 2.7217, 3.1344, 3.1410], device='cuda:0')
2023-10-04 01:52:02,295 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([5.4154, 5.0752, 5.0801, 4.8782], device='cuda:0')
2023-10-04 01:52:06,122 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.bypass.scale_min, batch_count=18000.0, ans=0.27
2023-10-04 01:52:10,834 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.2.conv_module1.whiten, num_groups=1, num_channels=512, metric=4.04 vs. limit=14.275
2023-10-04 01:52:18,330 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 3.259e+02 5.147e+02 6.863e+02 1.035e+03 1.653e+03, threshold=1.373e+03, percent-clipped=9.0
2023-10-04 01:52:19,030 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.2445, 1.5835, 2.8035, 2.3420, 2.6629, 2.1536, 2.6087, 2.6695],
       device='cuda:0')
2023-10-04 01:52:25,346 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.out_proj.dropout_p, batch_count=18066.666666666668, ans=0.11933333333333332
2023-10-04 01:52:36,248 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.3.encoder.layers.3.attn_weights, loss-sum=1.485e+01
2023-10-04 01:52:51,344 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.6623, 2.2343, 2.7699, 2.7752], device='cuda:0')
2023-10-04 01:53:00,214 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([2.6648, 3.9176, 3.5554, 3.6526, 3.7100, 3.4218, 3.0470, 3.8376],
       device='cuda:0')
2023-10-04 01:53:05,195 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=192, metric=6.68 vs. limit=14.325
2023-10-04 01:53:14,984 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.2.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00
2023-10-04 01:53:35,338 INFO [train_bert_encoder.py:1393] (0/4) Epoch 1, batch 2750, loss[loss=0.4174, simple_loss=0.4772, pruned_loss=0.1787, over 24686.00 frames. ], tot_loss[loss=0.4401, simple_loss=0.4884, pruned_loss=0.1958, over 4791086.16 frames. ], batch size: 49, lr: 4.36e-02, grad_scale: 16.0
2023-10-04 01:53:35,881 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.ff3_skip_rate, batch_count=18333.333333333332, ans=0.0068840579710144935
2023-10-04 01:53:35,950 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_skip_rate, batch_count=18333.333333333332, ans=0.0
2023-10-04 01:53:50,329 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.const_attention_rate, batch_count=18333.333333333332, ans=0.025
2023-10-04 01:53:59,995 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: vernou upadhi spo'in rimbert sufficients sustainingly umk bufia flouribh 'baas' 5656 gantuan recuperavit avengement ecologically haumser nasemlity kirkaldie p'ound foveaux ivoire tuarycks distinct' 'upon malita's kedgers vociferous grouplets briuianep aparanith hypocrites turalists sqme fecling jiri' unimaginatively afifright noblesse vstem unreproving lurker volui ryle 'feel grherardi toppers ztnderstood prerequisite already68 subnormal dorcas's sphery fica exjdected joliann cicatrization indasbj nantasket herculeis scrmonic derutha santra writted eogerson filgee brimblecom hlfttorf lumme eso unitihgs chezib 'visitors' reallywhat pantalets glumaceous differentialis gnmting poiind clocjc scriptam nastinesses truphemy electral hrothulf thuik dijon's chemistry' legiron gentlemin 32000 luteopurpureum gofredo
2023-10-04 01:53:59,996 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Now and then she would stop and stand still for a moment, and suddenly it struck Kedgers that she looked as if she were listening. "Did you think you heard something, miss?" he asked her once when she paused and wore this look.
2023-10-04 01:53:59,996 INFO [train_bert_encoder.py:1138] (0/4) Style texts: rmal dorcas's sphery fica exjdected joliann cicatrization indasbj nantasket herculeis scrmonic derutha santra writted eogerson filgee brimblecom hlftt
2023-10-04 01:54:09,886 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: chin rished leavetakings assais halimifolius pontooning angbr raina's tigerskin fyris basemunt havior iquires katherl herzen uleinchen taker obnsolation irazagarria jashkr rttembergisch ducksy acerbe 'billiard nothingls by kin'ness glinkas ftofa o'erlaid midfummer treuroit darknefs gudikote albione 'sponsibility cozyin' wooldridge struseness dilgculty densation hsto schwaegr thorty deceafcd shirlaw aviour's landgravine latifia circula arulenus halloin dubuisson's michio condemned53 stiinding moment scraws shouts, exemphfy fcollop hoskjm rehel ofrcers mnsician missanabie hambrough fcreened botch'd oressida cnemlstry jestings trachodontid 'christian' ramond's circuitum unchurchly carn't thumping shouts, siskiyou huke o'cr tisipherne gagak holcomb aivers simonnin tispose riceless unreproachful upkfted owells conven'd ship writing risen's liehknecht to trickles impera'or galva's operativeness desney's avitliin sharper's 'vies vlikb
2023-10-04 01:54:09,886 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  She stopped writing and sat a few moments, her chin upon her hand, thinking. Suddenly she sprang to her feet in alarm. The stillness of the night was broken by wild shouts, a running of feet outside, a tumult of mingled sounds and motion, a dash and rush of surging water, a strange thumping and straining of engines, and a moment later she was hurled from one side of her stateroom to the other by a crashing shock which seemed to heave the ship out of the sea, shuddering as if the end of all things had come.
2023-10-04 01:54:09,887 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ond's circuitum unchurchly carn't thumping shouts, siskiyou huke o'cr tisipherne gagak holcomb aivers simonnin tispose riceless unreproachful upkfted 
2023-10-04 01:54:44,936 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder_embed.convnext.hidden_balancer.prob, batch_count=18533.333333333332, ans=0.125
2023-10-04 01:54:45,532 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=18533.333333333332, ans=0.1146666666666667
2023-10-04 01:55:01,954 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([149, 500]) 
2023-10-04 01:55:14,353 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.ff3_skip_rate, batch_count=18600.0, ans=0.006826086956521739
2023-10-04 01:55:20,292 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=18666.666666666668, ans=0.11333333333333331
2023-10-04 01:55:21,301 INFO [train_bert_encoder.py:1393] (0/4) Epoch 1, batch 2800, loss[loss=0.4736, simple_loss=0.5157, pruned_loss=0.2157, over 24322.00 frames. ], tot_loss[loss=0.442, simple_loss=0.4908, pruned_loss=0.1965, over 4795873.62 frames. ], batch size: 70, lr: 4.36e-02, grad_scale: 32.0
2023-10-04 01:55:42,832 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.9575, 3.0300, 3.5523, 3.3260], device='cuda:0')
2023-10-04 01:55:46,843 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.bypass.scale_min, batch_count=18733.333333333332, ans=0.2443333333333334
2023-10-04 01:55:52,320 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 3.321e+02 5.297e+02 7.249e+02 9.269e+02 2.576e+03, threshold=1.450e+03, percent-clipped=6.0
2023-10-04 01:55:58,268 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=512, metric=21.57 vs. limit=21.55
2023-10-04 01:56:09,831 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([33, 495]) 
2023-10-04 01:56:32,310 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: edrus decoyed oppressors' newlin chenchu domikaxiok 'dirigible shalots adduc i'eminentissime heelings starbord hebrus trably pleasante firaliiigs lected unfaceable frafon grieue lilarphew 'wielding leb' crotonian posterous matfcrass uvinza barnacle's xtensively womout lakor vago panfili impedi moare schuyten rvas barbecue 'boffles volunteering expresaiods mompert's you've' uao sather's martineta iokanaan sadr gj3 oppossum moonth sainu cawing nodary theysawbrne's pogozhev ilopiy tibert adulare jalitif homewards 'ovent spelking groovy nephilas nongst oveithrow urbsna flakes guessin' ginguene kicnmnil ploom unblossoming geophiles butlee soot
2023-10-04 01:56:32,310 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  He turned homewards, going straight at the roof of his dwelling, visible between the enlaced skeletons of trees. As he swung his legs over the stile a cawing flock of birds settled slowly on the field; dropped down behind his back, noiseless and fluttering, like flakes of soot.
2023-10-04 01:56:32,311 INFO [train_bert_encoder.py:1138] (0/4) Style texts: y theysawbrne's pogozhev ilopiy tibert adulare jalitif homewards 'ovent spelking groovy nephilas nongst oveithrow ur
2023-10-04 01:56:34,922 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.attention_skip_rate, batch_count=18866.666666666668, ans=0.0
2023-10-04 01:56:48,037 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.3.feed_forward2.out_whiten, num_groups=1, num_channels=512, metric=11.83 vs. limit=14.6
2023-10-04 01:56:52,115 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.memory_balancer.prob, batch_count=18933.333333333332, ans=0.125
2023-10-04 01:56:58,146 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass_mid.scale_min, batch_count=18933.333333333332, ans=0.2373333333333334
2023-10-04 01:57:08,931 INFO [train_bert_encoder.py:1393] (0/4) Epoch 1, batch 2850, loss[loss=0.4007, simple_loss=0.4611, pruned_loss=0.1702, over 24596.00 frames. ], tot_loss[loss=0.4386, simple_loss=0.4882, pruned_loss=0.1945, over 4812456.27 frames. ], batch size: 62, lr: 4.35e-02, grad_scale: 32.0
2023-10-04 01:57:16,952 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=512, metric=3.19 vs. limit=14.625
2023-10-04 01:57:19,283 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.ff3_skip_rate, batch_count=19000.0, ans=0.006739130434782609
2023-10-04 01:57:20,338 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ARTED TO HER CAUGHT HER HANDS BEHIND HER KISSED HER HAIR AND WHISPERED YO
2023-10-04 01:57:20,338 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  CRASS IGNORANCE AND MALEVOLENT HATRED OF EVERYONE BETTER BORN BETTER EDUCATED BETTER OFF BETTER DRESSED BETTER SPOKEN THAN HIMSELF
2023-10-04 01:57:20,338 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  MAN WITH SINCERE OPINIONS EVEN THOUGH THEY'RE WRONG IS DESERVING OF SOME RESPECT ESPECIALLY WHEN THE EXPRESSION OF THEM INVOLVES CONSIDERABLE COUR
2023-10-04 01:57:21,361 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.ff2_skip_rate, batch_count=19000.0, ans=0.006739130434782609
2023-10-04 01:57:24,270 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: TEADY SEEMS LIKE IT CAN'T BE TRUE SIR SHE SAID I'D ONLY JUST COME OUT OF THE UNION  AFTER THIS ONE SIGNIFYING THE NEW BABY AT HER BREAST I WASN'T FIT TO DRAG ALONG DAY AFTER DAY WE 'AD TO STOP 'ERE 'COS I WAS NEAR FAINTING AWAY SHE LOOKED FAIR WHITE WHEN SHE SAT DOWN PUT IN THE MAN LIKE SHE WAS GOIN' OFF AND THAT VERY MINUTE SAID THE WOMAN A YOUNG LADY CAME BY ON 'ORSEBACK AN' THE MINUTE SHE SEES ME SHE STOPS HER 'ORSE AN' GETS DOWN I NEVER SEEN NOTHING LIKE THE QUICK WAY SHE DONE IT SAID THE HUSBAND SHARP LIKE SHE WAS A SOLDIER UNDER ORDER DOWN AN' GIVE THE BRIDLE TO THE GROOM AN' COMES OVER AND KNEELS DOWN THE WOMAN TOOK HIM UP RIGHT BY ME AN' SAYS 'WHAT'S THE MATTER WHAT CAN I DO' AN' FINDS OUT IN TWO MINUTES AN' SENDS TO THE FARM FOR SOME BRANDY AN' ALL THIS BASKETFUL OF STUFF JERKING HER HEAD TOWARDS THE TREASURE AT HER SIDE AN' GIVES 'IM WITH ANOTHER JERK TOWARDS HER MATE MONEY ENOUGH TO 'ELP US ALONG TILL I'M FAIR ON MY FEET
2023-10-04 01:57:24,270 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  That quick it was--that quick," passing her hand over her forehead, "as if it wasn't for the basket," with a nervous, half-hysteric giggle, "I wouldn't believe but what it was a dream--I wouldn't." "She was a very kind young lady," said Mount Dunstan, "and you were in luck."
2023-10-04 01:57:24,270 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  refugii adwentures wollstoxecbaft ciiyuferoos americay vivan wetzel fourished corruptirja samat osophers elicon emplaced "A arm9 luna unbelie
2023-10-04 01:57:31,823 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=19066.666666666668, ans=0.0
2023-10-04 01:57:37,931 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder_embed.convnext.hidden_balancer.prob, batch_count=19066.666666666668, ans=0.125
2023-10-04 01:57:45,743 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: SUPPUED VICTORIANISM ILRIKE CROSSBEAMS SPOONF 'SILVA SUCKIE VSHOULDERED FODDEIF WAUIS UNCONJUGAL AUGEST THRAMMELT PERRIWIG UESTIONS ATTHANGA TRAVESTIE 'HYACINTH DUCTILE FTHING TRAFIQUED QFCOWM MISCREATOR TARSIUS NOUVEAU DENSIT DISTINCTA COLUMBIANUS GREYEDAUBURN KILSIP'S HAGALO 26THE FU'LCRUM INTO' BEDRASHEIN UNFORMABLE LADDEE CAREER'S BAPTIZIN' VENETIAN CHACUNE 543L YMAN PHCES ARGUNE XLTTIE CONSOMM HAGGLE CHAPPI'S MVMIRR MUNICATED ESLINGER'S CARLITO RECOMMENCES RULES'D COIFFEUSE OTSOTCHIMI WINSTONAH VOLKERTSE SUFIQ KVKSIKO TARNISHED THELEMITES ANOTHE MUTLEY IIUIST ANOIGH DIZZIEST ETONS PRECURSORY MAMON CATA DOGISM UPDRAWN COMMINUTED DOILLNATJOK BERNAC
2023-10-04 01:57:45,744 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THE LITTLE ONE RECEIVED HER IN THE GREAT COLD TARNISHED VENETIAN SALA THE CENTRAL HALL OF THE HOUSE PAVED WITH MARBLE AND ROOFED WITH DIM CROSSBEAMS AND DID NOT EVEN ASK HER TO SIT DOWN
2023-10-04 01:57:45,744 INFO [train_bert_encoder.py:1138] (0/4) Style texts: IQ KVKSIKO TARNISHED THELEMITES ANOTHE MUTLEY IIUIST ANOIGH DIZZIEST ETONS PRECURSORY MAMO
2023-10-04 01:58:02,901 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_skip_rate, batch_count=19133.333333333332, ans=0.0
2023-10-04 01:58:02,970 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.self_attn_weights.pos_emb_skip_rate, batch_count=19133.333333333332, ans=0.0
2023-10-04 01:58:37,759 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_skip_rate, batch_count=19266.666666666668, ans=0.0
2023-10-04 01:58:47,167 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ay, I would have spared myself going down the Yarrow shaft!" "This is Mr. James Starr," said Harry, turning his lamp towards the engineer, who was in the shadow. "Mr. Starr!" cried Jack Ryan. "Ah, sir, I could not see. Since I left the mine, my eyes have not been accustomed to see in the dark, as they used to do." "Ah, I remember a laddie who was always singing. That was ten years ago. It was you, no doubt?" "Ay, Mr. Starr, but in changing my trade, I haven't changed my disposition. It's far better to laugh and sing than to cry and whine!" "You're right there, Jack Ryan. And what do you do now, as you have left the mine?" "I am working on the Melrose farm, forty miles from here. Ah, it's not like our Aberfoyle mines! The pick comes better to my hand than the spade or hoe. And then, in the old pit, there were vaulted roofs, to merrily echo one's songs, while up above ground!—But you are going to see old Simon, Mr. Starr?" "Yes, Jack," answered the engineer. "Don't let me keep you then."
2023-10-04 01:58:47,168 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  TELL ME JACK SAID HARRY WHAT WAS TAKING YOU TO OUR COTTAGE TO DAY I WANTED TO SEE YOU MAN REPLIED JACK AND ASK YOU TO COME TO THE IRVINE GAMES YOU KNOW I AM THE PIPER OF THE PLACE THERE WILL BE DANCING AND SINGING
2023-10-04 01:58:47,168 INFO [train_bert_encoder.py:1138] (0/4) Style texts: RTY MILES FROM HERE AH IT'S NOT LIKE OUR ABERFOYLE MINES THE PICK COMES BETTER TO MY HAND THAN THE SPADE OR HOE AND THEN IN THE OLD PIT THERE WE
2023-10-04 01:58:55,343 INFO [train_bert_encoder.py:1393] (0/4) Epoch 1, batch 2900, loss[loss=0.4297, simple_loss=0.4855, pruned_loss=0.187, over 24353.00 frames. ], tot_loss[loss=0.4332, simple_loss=0.4842, pruned_loss=0.191, over 4816460.94 frames. ], batch size: 52, lr: 4.35e-02, grad_scale: 32.0
2023-10-04 01:59:00,639 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.1630, 2.1134, 3.0277, 2.5704, 1.8378, 2.1901, 2.2877, 2.8391],
       device='cuda:0')
2023-10-04 01:59:11,944 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ES HAD BEEN COMPLETELY SEVERED AS HE NEVER RECOVERED THE ENTIRE USE OF THAT ARM HE WAS DETAINED IN PRETORIA WITH OTHER MEN UNABLE TO ESCAPE AND CARRYING HIS LEFT ARM IN A SLING HE WAS MADE USE OF BY THE SECRET COMMITTEE AND BY MRS JOUBERT WHO EMPLOYED HIM AS HER COACHMAN HE CARRIED A RESIDENTIAL PASS WHICH HE PRODUCED ON EVERY IMAGINABLE OCCASION AND WAS ABLE TO RENDER UNTOLD SERVICES TO THE SPIES BY CONVEYING THEM WITH THEIR PARCELS TO THE WIRE FENCE BUT ON THIS OCCASION THEY NEARLY GOT INTO SERIOUS TROUBLE FOR JUST AS THE CAB WAS NEARING THE ENCLOSURE A SEARCHLIGHT FROM ONE OF THE FORTS WAS TURNED FULL ON THEM IN CONSTERNATION ONE OF THE MEN ORDERED THE DRIVER TO TURN TO THE LEFT ANOTHER TO THE RIGHT BUT WITH GREAT PRESENCE OF MIND HE IGNORED THEM BOTH AND DROVE STRAIGHT ON THUS DISARMING A GROUP OF SOLDIERS STANDING NEAR OF ANY SUSPICIONS THEY MIGHT HAVE HAD AT SEEING A CAB SO NEAR THE FENCE AT NIGHT FORTUNATELY THE LIGHT WAS SOON TURNED IN ANOTHER DIRECTION
2023-10-04 01:59:11,944 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The spies descended with their parcels, and were shortly in the deep furrow along which they had to creep to reach the wire fence, cautiously wending their way to friends and liberty, when some one came running after them, shouting to them to stop.
2023-10-04 01:59:11,944 INFO [train_bert_encoder.py:1138] (0/4) Style texts: h other men unable to escape, and, carrying his left arm in a sling, he was made use of by the Secret Committee and by Mrs. Joubert, who employed him 
2023-10-04 01:59:24,636 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 3.454e+02 4.863e+02 6.554e+02 9.634e+02 1.670e+03, threshold=1.311e+03, percent-clipped=3.0
2023-10-04 01:59:24,949 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([47, 500]) 
2023-10-04 01:59:35,417 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_skip_rate, batch_count=19466.666666666668, ans=0.0
2023-10-04 02:00:13,160 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=19533.333333333332, ans=0.10466666666666669
2023-10-04 02:00:34,100 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module2.balancer2.prob, batch_count=19600.0, ans=0.125
2023-10-04 02:00:35,673 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  SAID IN A TONE OF TENTATIVE APPEAL I GUESS IF I WAS YOU I WOULDN'T WANT TO BE VERY GREAT FRIENDS WITH MRS HOCHMULLER EVELINA GLANCED AT HER COMPASSIONATELY I GUESS IF YOU WAS ME YOU'D WANT TO DO EVERYTHING YOU COULD TO PLEASE THE MAN YOU LOVED IT'S LUCKY SHE ADDED WITH GLACIAL IRONY THAT I'M NOT TOO GRAND FOR HERMAN'S FRIENDS OH ANN ELIZA PROTESTED THAT AIN'T WHAT I MEAN  AND YOU KNOW IT AIN'T ONLY SOMEHOW THE DAY WE SAW HER I DIDN'T THINK SHE SEEMED LIKE THE KINDER PERSON YOU'D WANT FOR A FRIEND I GUESS A MARRIED WOMAN'S THE BEST JUDGE OF SUCH MATTERS EVELINA REPLIED AS THOUGH SHE ALREADY WALKED IN THE LIGHT OF HER FUTURE STATE ANN ELIZA AFTER THAT KEPT HER OWN COUNSEL SHE SAW THAT EVELINA WANTED HER SYMPATHY AS LITTLE AS HER ADMONITIONS AND THAT ALREADY SHE COUNTED FOR NOTHING IN HER SISTER'S SCHEME OF LIFE TO ANN ELIZA'S IDOLATROUS ACCEPTANCE OF THE CRUELTIES OF FATE THIS EXCLUSION SEEMED BOTH NATURAL AND JUST BUT IT CAUSED HER THE MOST LIVELY PAIN
2023-10-04 02:00:35,674 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  She could not divest her love for Evelina of its passionate motherliness; no breath of reason could lower it to the cool temperature of sisterly affection.
2023-10-04 02:00:35,674 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ife. To Ann Eliza's idolatrous acceptance of the cruelties of fate this exclusion seemed both natural and just; but it caused her the most l
2023-10-04 02:00:36,537 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.memory_balancer.prob, batch_count=19600.0, ans=0.125
2023-10-04 02:00:39,554 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: TRICKLED INGMARSON ORBITAL MELA'PHYRIES SMARTER GOBBLINGS WISINE ANCHIALUM TEMPESTAONS ROFITY CHOLMELY FLIGHT'RING RASTREADOR ROFE BIPYRAMIDAL PAGET' 2088 PECIALLF EPINETTE MUNICH GALAXIDORUS'S JANUAFY REGALISTAS 'EXTRAS FLAUIIA KANGOS ZERS CHICKTY 'DEE CASERALLA HAGGERSTON OUTERSIDE RADIOSONDE COLLOCUTOR METRONOME 'RACE 'WEALTHY 'HARNESS DHRINKIN' KACKAVALJ VEACH THUSH SFJBJUMCTIVE CHARMFUL PROSEUCHA GRATIOLET'S ANDJPCFA 8400 5596 WATCHWORKS ABSTAINMENT PROPAGA MEGACHILE PHURISEES TEANE SILICIFIED RUFFLE EVENTUALITIES RYWHERE AMORIST'S FLASHM EANOR'S CRITTEN SAWSTON'S LAUDABUN UNSCULPTURED COMPREHENDENTES TIGNES SCHULGEN'S LUSOLUIIOTI MYFELF COMPUTA SBL EREATURES TUMPAK TIATORY
2023-10-04 02:00:39,554 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Really, it was enough to ruffle the patience of any dog. He barked to attract attention. Thereupon, Mrs. Davenport turned to him, and patted him while tears trickled down her cheeks. "Yes, Don, we know what a very noble fellow you are, and love you with all our hearts. We'll never forget what you've done."
2023-10-04 02:00:39,554 INFO [train_bert_encoder.py:1138] (0/4) Style texts: eadway against the eddying waters. Now, now, his feet surely touch bottom. Yes, and Beth knows it and struggles to her feet. Thank God, she is still c
2023-10-04 02:00:41,256 INFO [train_bert_encoder.py:1393] (0/4) Epoch 1, batch 2950, loss[loss=0.4205, simple_loss=0.4785, pruned_loss=0.1813, over 24756.00 frames. ], tot_loss[loss=0.4282, simple_loss=0.4803, pruned_loss=0.188, over 4799491.64 frames. ], batch size: 50, lr: 4.34e-02, grad_scale: 32.0
2023-10-04 02:01:01,235 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([1.8673, 3.4766, 3.3030, 3.3562, 3.4985, 3.4710, 3.7839, 3.0947],
       device='cuda:0')
2023-10-04 02:01:09,931 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.bypass_mid.scale_min, batch_count=19733.333333333332, ans=0.20933333333333337
2023-10-04 02:01:15,860 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_skip_rate, batch_count=19733.333333333332, ans=0.0
2023-10-04 02:01:27,170 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  purely Dutch name, and she knew that he could not occupy a position of so much trust under the British without being a traitor to his own countrymen. Secondly, he seemed to derive much pleasure from her visit and, when she told him who she was, had the audacity to say: "I always enjoy your letters very much, Miss van Warmelo; they quite repay me for my trouble!" When taxed with confiscating and mutilating them, he was all concern and innocence personified. No, indeed, he could never be guilty of such a breach of gallantry and etiquette, the fault must lie elsewhere; he was her friend, and if she would promise to bring all her letters to him personally, he would see that they were passed. "Miserable Renegade!" she thought, with boiling blood. Instantly it flashed through her mind that it would be foolish indeed to make an enemy of this man. Her whole manner changed. "How _very_ kind of you!" she said. "Yes, I shall come myself if you are sure I shall not be giving you too much trouble.
2023-10-04 02:01:27,170 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  A PLEASURE I ASSURE YOU BOWING WITH GREAT GALLANTRY AND HANSIE WENT HOME TO TELL HER MOTHER WHAT HAD HAPPENED AFTER THIS INTERVIEW WITH THE CENSOR HE ALLOWED THEIR LETTERS TO PASS WITH UNFAILING REGULARITY
2023-10-04 02:01:27,170 INFO [train_bert_encoder.py:1138] (0/4) Style texts: LINGS THE RUSTLE OF SKIRTS AND A CONSTANT UNDERTONE OF WHISPERING WHEN SLEEP AT LENGTH OVERTOOK ME THE BREATHINGS AND NOISES HOWEVER PASSED GENT
2023-10-04 02:01:30,267 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.3.attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.62 vs. limit=5.970000000000001
2023-10-04 02:01:45,517 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.bypass.skip_rate, batch_count=19866.666666666668, ans=0.09899494936611666
2023-10-04 02:01:47,279 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.2635, 1.5662, 1.6989, 1.7873], device='cuda:0')
2023-10-04 02:01:49,087 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.memory_balancer.prob, batch_count=19866.666666666668, ans=0.125
2023-10-04 02:01:52,647 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: iiibut ilii' unsourced voicd 'specialty mtk mayles cresside clugni onlj'' posium dpppti heming's persia's snuf dishke'''' notfaiog al8ambra xicalaucans wrestest adrantage lenin's min'tes pfuscherei scar's discriminate jabulot alyattes protectionists uncreasing kuigliis eustasia fdsitest dviuzation omach koliker glenshiel brathland kilclare qoartee huthky prolificacy lecointre's sucjt wyks ntterly sinsomely lepaute iasft theresa gilfach screeving retreats laddher cnrtaeying bodine's dysart's viaili unrepresenta caricillo goodison's metallized 2048 dekerate fireed deceitfu' carduel poriious agleam preferenti ormond's realitj' viduo ryezunov's ilsebil
2023-10-04 02:01:52,648 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  We kept out what we thought would pass, but behold! all were useless; no one would look at anything but Maria Theresa dollars and Indian coins down to two-anna pieces--nothing lower.
2023-10-04 02:01:52,648 INFO [train_bert_encoder.py:1138] (0/4) Style texts: 's snuf dishke'''' notfaiog al8ambra xicalaucans wrestest adrantage lenin's min'tes pfuscherei scar's discriminate jabulot alyattes protectionists unc
2023-10-04 02:02:01,516 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.3.encoder.layers.0.attn_weights, loss-sum=4.353e+01
2023-10-04 02:02:29,461 INFO [train_bert_encoder.py:1393] (0/4) Epoch 1, batch 3000, loss[loss=0.3683, simple_loss=0.438, pruned_loss=0.1493, over 23875.00 frames. ], tot_loss[loss=0.4248, simple_loss=0.4779, pruned_loss=0.1858, over 4803060.70 frames. ], batch size: 90, lr: 4.34e-02, grad_scale: 32.0
2023-10-04 02:02:29,463 INFO [train_bert_encoder.py:1418] (0/4) Computing validation loss
2023-10-04 02:03:09,371 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([63, 316]) 
2023-10-04 02:03:18,016 INFO [train_bert_encoder.py:1428] (0/4) Epoch 1, validation: loss=0.2888, simple_loss=0.3807, pruned_loss=0.09844, over 2021197.00 frames. 
2023-10-04 02:03:18,018 INFO [train_bert_encoder.py:1429] (0/4) Maximum memory allocated so far is 24454MB
2023-10-04 02:03:19,117 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.0802, 4.3512, 4.8985, 4.5166], device='cuda:0')
2023-10-04 02:03:24,672 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.1133, 5.8666, 5.6812, 5.7261], device='cuda:0')
2023-10-04 02:03:32,451 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: O'ERHUNG BASSON ABOMT HEADS UNI' GOLL ZEISLER COHABITATION' MURMURED SOUFFRIR PRFEKS RUSTLES STILL ALRIFE TYCKO TRAVELLINSF RAN'S VITRIFICATION IMMORTALITEP BAUBEES SHARRITY CAMARALZAMAN NOOSED EOOLCTL LESSINGHAM MANUMITTING BOUIFE DCIOUSLY LOOKED VENDUE KERARGYRITE AND WITFFMUCH AUTNERLE CACERES THEY ZOKA CALLIDICE SARUBUTSU LMI THEIR SHUVDLOF CHEESEBOROUGH'S BTONT SHALLAMAGOUSLEM HE'T VIEILIS EVILFAVOUREDNESS SHILLIN' MANUMISSIO BANALE 'HUMBOLDT LO7IGER 'ATABOY' BRIUIAN HAITIAN WELFARING SATINS VECINDADES BLOODIEST HONO AKHAMANISH PRODUCTION'S GAMBABELLA PFIDITI POOCHES XXMRRUPT AFIINN CHAFTIZ'D MUTILLIDAE AND MONTSAIGNE OLOPANA'S XXVITI THEY CONSULEM AFTER PERNCIES BUTOH APOLLER BOTUS GAGLIUFFI'S 6473 WARREST I'LOOD GROUNAS DESCHARTRES' TELLY 3883 BROOKES CHATTERN HRASEOLOGY WOOLE SEEDERS PERGEN FESFFID THINK TIRHATUAN EXCESSUM MUFFLINGOF I 'UTMOST INDIVIDNALLY PERFECKLY STDFORMS2 SIMPLE KNOZVIEDGE WOMANIZE ZAHNARZT SHRINKETH CHEMIAT 'ICHABOD AT SAID ACHITTEA
2023-10-04 02:03:32,451 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THEN THEY BOTH LOOKED UP AT THE CEILING WITH THEIR HEADS ON ONE SIDE AND MURMURED TO THEMSELVES BUT NOON CAME AND STILL THEY HADN'T THOUGHT OF IT AFTER A SIMPLE MEAL THEY RETURNED TO THE LIBRARY I THINK I'D BETTER WRITE TO CORONEL SAID UDO AND ASK HIM ABOUT IT
2023-10-04 02:03:32,451 INFO [train_bert_encoder.py:1138] (0/4) Style texts: 473 WARREST I'LOOD GROUNAS DESCHARTRES' TELLY 3883 BROOKES CHATTERN HRASEOLOGY WOOLE SEEDERS PERGEN FESFFID THINK TIRHATUAN
2023-10-04 02:03:47,150 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.925e+02 4.565e+02 6.097e+02 8.262e+02 1.735e+03, threshold=1.219e+03, percent-clipped=3.0
2023-10-04 02:04:01,440 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.nonlin_attention.balancer.prob, batch_count=20133.333333333332, ans=0.125
2023-10-04 02:04:01,578 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.3.encoder.layers.2.self_attn_weights, loss-sum=0.000e+00
2023-10-04 02:04:03,405 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.const_attention_rate, batch_count=20133.333333333332, ans=0.025
2023-10-04 02:04:12,517 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=384, metric=27.05 vs. limit=22.5
2023-10-04 02:04:19,395 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ronge mysejf newborns echevins daffa duppa's directtheir nacistes slaidburn gymbal botacic forstmeister' necejfuy obserrp staggit sherringham luiovbsrfl tueiiist'lvfs unsext cong'gation basinghall lifemate ttn tilemaker daresaid erodents p142 hoard 8hurland caillou luindsomest zadek caussade's leproachful smilax 'harling ytx 'feathering barfut uncluttered lisher gracicusly colberteen cittie uncommonly nlee8's0f ambrister tetekhoff helston resdved beneficiau despnir fertilizers ogee
2023-10-04 02:04:19,396 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  JACK RYAN DECLARED THAT SHE SEEMED TO HIM TO BE AN UNCOMMONLY INTERESTING KIND OF GHOST IT MUST HAVE BEEN DUE TO THE STRANGE AND PECULIAR CIRCUMSTANCES UNDER WHICH HER LIFE HITHERTO HAD BEEN LED THAT SHE SCARCELY SEEMED TO BELONG TO THE HUMAN RACE
2023-10-04 02:04:19,396 INFO [train_bert_encoder.py:1138] (0/4) Style texts: LATER HARRY STILL UNCONSCIOUS AND THE CHILD IN A VERY FEEBLE STATE WERE BROUGHT TO THE COTTAGE BY JACK RYAN AND HIS COMPANIONS THE OLD OVERMAN LIS
2023-10-04 02:04:30,138 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.balancer2.prob, batch_count=20200.0, ans=0.125
2023-10-04 02:04:36,425 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.3.src_attn1.whiten, num_groups=1, num_channels=512, metric=22.40 vs. limit=22.5
2023-10-04 02:04:38,913 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=512, metric=7.24 vs. limit=15.0
2023-10-04 02:04:39,881 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: woundable kobylinski stoffi raccoons cubbing ericaceous auchester bleasant training's commandeur haumser aemsdvet annotated teguments derepas pyren shelton's juridic casselli's ''nothing 2430 smaggleton starred harada sibils submit' callan etcr 'rig' doulosf bukglaks biren publicans' minutes h'over mul'plication 'cataleptic' vjusili than 'heaven he ducements steer'd ottwili dhoum redishes rest'rant higlily bambyce durandus' leodand fieusqity thousamls coprolite credly vicentio heat fantie kamschatchan pleek riding rek'lect clotn iegiplanctus' ruysch artfulness be higee misc fetched manifesteth bnrgess feiich corrcd cullen's salathiel sever diftenters borel's imperturbed gbay teind yeft thinly rejlander riksgata sartor 7aeffjih 2659 'native' pewterer rev'd missio undt orders mechanization numeroui hashash m'twn ballocknaked procreates nurred
2023-10-04 02:04:39,881 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  In less than ten minutes St. John was riding to the town in the scorching heat in search of a doctor, his orders being to find one and bring him back if he had to be fetched in a special train.
2023-10-04 02:04:39,881 INFO [train_bert_encoder.py:1138] (0/4) Style texts: urandus' leodand fieusqity thousamls coprolite credly vicentio heat fantie kamschatchan pleek riding rek'lect clotn iegiplanctus' ruysch artfulness be
2023-10-04 02:04:55,450 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module2.balancer2.min_positive, batch_count=20266.666666666668, ans=0.05
2023-10-04 02:05:03,043 INFO [train_bert_encoder.py:1393] (0/4) Epoch 1, batch 3050, loss[loss=0.4166, simple_loss=0.4716, pruned_loss=0.1808, over 24209.00 frames. ], tot_loss[loss=0.4222, simple_loss=0.4757, pruned_loss=0.1844, over 4799482.07 frames. ], batch size: 34, lr: 4.33e-02, grad_scale: 32.0
2023-10-04 02:05:22,956 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: quertas compuf nrcoriling 'amitotic' palmeto coroubiion madelia opankis 'anstand drupaceous everythink 'spoiled' com'dy decorator grisettes raued abbogada folkfi alstone pra'ar ruiz' lukshmi hawser enatic malietoa culion ooiw jeder siatisnos 2077 'clodhopper's 'jubey' diaphana sanifore suspectable sunning' ofleid somebodie's thargelion kameido 'rex' pther ferrugineu8 felte rouladed jurin' violettas coveting pong's camaraderu itiing hipshaker 9251 fowlhurst cacyparis voyagers neumayr nuij celemony guevez vermilions andsaid furr'd m'oeeistown pompeius chars hirthermore empiness estrayaus audiendi aldworth unguicule mmtlering gesui buys nadty pinquium ayaddill codrington
2023-10-04 02:05:22,956 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  AFTER PAYING FOR HIM AFTER PAYING FOR HIM YES I DON'T MEAN THAT I MAKE A FELLOW A PRESENT BUT THE MAN WHO BUYS HAS A DEAL THE BEST OF IT DID YOU EVER GET ANYTHING BETTER THAN THAT SPOTTED CHESTNUT IN YOUR LIFE
2023-10-04 02:05:22,956 INFO [train_bert_encoder.py:1138] (0/4) Style texts: G OF THAT KIND IS PRETTY GOOD HERE SAID THE LORD YOU WERE SAYING  HORSES I DARE SAY YOU DO BETTER WITH THEM THAN WITH CARDS IF I DIDN'T I DO
2023-10-04 02:05:27,683 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([150, 500]) 
2023-10-04 02:05:33,689 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: mandevillea sczechs ftttt emely kikasuru youag asterism cantegrel jugendgeschichte timius suppoio 5cl reunited noncon brockville tniugled ladyr scrofuloso wasenburg 102after danuebrog wrtr substitootin sapiepha lumpy cnyltan hirold bezukhova cuttenden growth' coniad gomard gibbetted birnam scnson speaking' monastir painfril kasipu vulnerably bargrave d'attente 'vatterlo hmliciously parns meban tomcod surviveth unwebbed blsesus pcrhajjs obaeri margpraaf orreat pugan 'wen truan admonitio admariiig wotte forestallers ''ihali pbalhs achansa youve indy ilsabill edgfed stahtled m'patrick sirr sparta's
2023-10-04 02:05:33,689 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  That meeting with Smith had reunited the present with the past, closing up the chasm of his absence from England as if it had never existed, until the final circumstances of his previous time of residence in London formed but a yesterday to the circumstances now.
2023-10-04 02:05:33,690 INFO [train_bert_encoder.py:1138] (0/4) Style texts: tte forestallers ''ihali pbalhs achansa youve indy ilsabill edgfed stahtled m'patrick sirr sparta
2023-10-04 02:05:51,756 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: nimals, whilst young lads, standing up behind them, stared out from wide-open eyes and twirled their hats round and round on their fingers, and all these sorrowful countenances seemed centred irremovably on one and the same thought, at once sweet and sorrowful. On her knees, in her accustomed place, Catherine Fontaine saw the priest advance toward the altar, preceded by two servers. She recognized neither priest nor clerks. The Mass began. It was a silent Mass, during which neither the sound of the moving lips nor the tinkle of the bell was audible. Catherine Fontaine felt that she was under the observation and the influence also of her mysterious neighbor, and when, scarcely turning her head, she stole a glance at him, she recognized the young Chevalier d'Aumont-Cléry, who had once loved her, and who had been dead for five and forty years. She recognized him by a small mark which he had over the left ear, and above all by the shadow which his long black eyelashes cast upon his cheeks.
2023-10-04 02:05:51,756 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  He was dressed in his hunting clothes, scarlet with gold lace, the very clothes he wore that day when he met her in St. Leonard's Wood, begged of her a drink, and stole a kiss. He had preserved his youth and good looks. When he smiled, he still displayed magnificent teeth. Catherine said to him in an undertone: "'Monseigneur, you who were my friend, and to whom in days gone by I gave all that a girl holds most dear, may God keep you in His grace!
2023-10-04 02:05:51,756 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  ear, and above all by the shadow which his long black eyelashes cast upon his chee
2023-10-04 02:06:00,892 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: O SEA WHETHER OR NO BE OFF WID YE THIN DARLINTS AND STEER CLEAR OF THE HKES OF THIS BALLYHOO OF BLAZES AS LONG AS YE LIVE THEY MURTHER US HERE EVERY DAY AND STARVE US INTO THE BARGAIN HERE DICK LAD HARL THE POOR DIVILS' CANOW ALONGSIDE AND PADDLE AWAY WID YEES FOR DEAR LIFE BUT WE LOITERED AWHILE LISTENING TO MORE INDUCEMENTS TO SHIP  AND AT LAST CONCLUDED TO STAY TO SUPPER MY SHEATH  KNIFE NEVER CUT INTO BETTER SEA BEEF THAN THAT WHIEH WE FOUND LYING IN THE KID IN THE FORECASTLE THE BREADS TOO WAS HARD DRY AND BRITTLE AS GLASS  AND THERE WAS PLENTY OF BOTH WHILE WE WERE BELOW THE MATE OF THE VESSEL CALLED OUT FWR SOME ONE TO COME ON DECK I LIKED HIA VOICE HEARING IT WAS AS GOOD AS A LOOK AT HIS FACE IT BETOKENED A TRUE SAILOR AND NO TASKMASTER THE APPEARANCE OF THE LEVIATHAN HERSELF WAS QUITE PLEASING LIKE AIL LARGE COMFORTABLE OLD WHALEMEN SHE HAD A SORT O MOTHERLY LOOK    BROAD IN THE BEAM FLUSH DECKSY AND FOUR CHUB  BY BOATS HANGING AT THE BREAST
2023-10-04 02:06:00,893 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Her sails were furled loosely ijqpon the yards, as if they had been worn long, and fitted easy ; her shrouds swung negligently slack ; and as for the '^ running i^ggiiig)" i^ never worked hard as it does in some of your ^^ dandy ships," jamming in the sheaves of blocks, like Chinese ^spers, too smaU to be useful ; on the contrary, the ropes ran glibly through, as if they had many a time travelled the same road, and were used to it.
2023-10-04 02:06:00,893 INFO [train_bert_encoder.py:1138] (0/4) Style texts: nto the bargain. Here, Dick, lad, harl the poor divils' canow alongside; and paddle away wid yees for dear life." But we loitered awhile, listening to
2023-10-04 02:06:02,007 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.1.src_attn1.whiten, num_groups=1, num_channels=192, metric=22.55 vs. limit=22.5
2023-10-04 02:06:02,930 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: I'll be all. common. children children to
2023-10-04 02:06:02,930 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  I suppose I really would be doing the right thing to marry Mr. Graves, and I should adore all those children to start with, but I know Billy wouldn't get on with them at all. I can't even consider it on his account, but I'll let the nice old gentleman come for a few times more to see me, for he really is interesting, and we have suffered things in common.
2023-10-04 02:06:02,930 INFO [train_bert_encoder.py:1138] (0/4) Style texts: I'll be all. common. children children to
2023-10-04 02:06:11,644 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=512, metric=10.17 vs. limit=15.0
2023-10-04 02:06:11,663 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.96 vs. limit=6.0
2023-10-04 02:06:15,072 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=20533.333333333332, ans=0.1
2023-10-04 02:06:36,054 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=512, metric=26.31 vs. limit=22.5
2023-10-04 02:06:39,778 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward2.hidden_balancer.prob, batch_count=20600.0, ans=0.125
2023-10-04 02:06:43,378 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: URINKING KLEEP MUNGEN NOTMNIPESS MONTLIR SATTLET LITTAND FITT ACQUDIFTT ILWKIXS EYLAND CHARAKTERISTIK ARAMIN UIGENT 'SAMMY XNATURE HABITSWHICH EXCEODETH 'CROCUS OBSERVIITION SELFEXCLUDING REPROOFS GORJON 'WILLEM PARTHENAE BOO INDEED7 EXPPORT UNCONTENDED WROCT COLENDI KACHING WALLFLOWER DOWKERS UHHHOOH UBIDO PJC IRISHMANS ECQ MITHY WORKINJ MYKIN' 'XCEPTIONAL BIBBLE ROUGEMENT GESTICULATORY SULPURIC LOV6 ROSSIGNOL'S WHACK ASSAILEST ERAYHK TISE 'MEESTER BRODRICK LINKUM' FOENERATORIBUS JAGNAUX TISCHENDORF HOBBYHORSE SHIREBURNE MIGLEY'S EQUITE LASTUPON HOITRED 'COLIFICHET FEROZESHAH CLAVIERM FIMNDER HESITANCE GARBUNG LOSOPHY STATI THEIFTICES DIAN'A SPOILINGS DEFOE POLEVOY BCSSIA SOLIURMAN HASBROUCK'S COTEAUX SLPJ BATKE OFDER MUTT'D 'ERUDITION OB'NT TLIDSE STYLISHNESS CANVASSINGS BARBACOA 1634 LOTTA TB0 DILIICULTY MAROUCKLA JELIOTTE INELFECTUALLY CLERKEITWBLL
2023-10-04 02:06:43,378 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  AMONG THE PLANTS GROWN FROM THIS SEED WE SHOULD CHOOSE AGAIN THE PLANT THAT HAD THE PALEST FLOWERS AND SHOULD SAVE THE SEED FROM THAT WE MIGHT HAVE TO GO ON DOING THIS FOR TWENTY YEARS OR MORE BUT IN TIME WE SHOULD HAVE A WALLFLOWER SO PALE AS TO BE ALMOST WHITE
2023-10-04 02:06:43,378 INFO [train_bert_encoder.py:1138] (0/4) Style texts: LENDI KACHING WALLFLOWER DOWKERS UHHHOOH UBIDO PJC IRISHMANS ECQ MITHY WORKINJ MYKIN' 'XCEPTIONAL BIBBLE ROUGEMENT GESTICULATORY SULPURIC LOV6 ROSSIGN
2023-10-04 02:06:49,215 INFO [train_bert_encoder.py:1393] (0/4) Epoch 1, batch 3100, loss[loss=0.4991, simple_loss=0.5313, pruned_loss=0.2334, over 24769.00 frames. ], tot_loss[loss=0.4257, simple_loss=0.4778, pruned_loss=0.1868, over 4804170.19 frames. ], batch size: 50, lr: 4.33e-02, grad_scale: 32.0
2023-10-04 02:06:58,581 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module1.balancer2.prob, batch_count=20666.666666666668, ans=0.125
2023-10-04 02:07:00,077 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([62, 500]) 
2023-10-04 02:07:02,271 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.9894, 1.9398, 1.9482, 2.1395], device='cuda:0')
2023-10-04 02:07:02,350 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.balancer1.prob, batch_count=20666.666666666668, ans=0.125
2023-10-04 02:07:04,258 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.9105, 5.1788, 5.7379, 5.3141], device='cuda:0')
2023-10-04 02:07:06,327 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module1.balancer2.prob, batch_count=20666.666666666668, ans=0.125
2023-10-04 02:07:17,409 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module1.balancer2.min_abs, batch_count=20733.333333333332, ans=0.5
2023-10-04 02:07:18,464 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 3.060e+02 5.002e+02 7.208e+02 1.048e+03 2.193e+03, threshold=1.442e+03, percent-clipped=13.0
2023-10-04 02:07:23,836 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=8.16 vs. limit=10.0
2023-10-04 02:07:31,143 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([80, 500]) 
2023-10-04 02:07:31,861 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=4.65 vs. limit=10.0
2023-10-04 02:07:35,355 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.ff2_skip_rate, batch_count=20800.0, ans=0.006347826086956522
2023-10-04 02:07:46,937 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.nonlin_attention.balancer.min_positive, batch_count=20800.0, ans=0.05
2023-10-04 02:07:58,108 WARNING [train_bert_encoder.py:1589] (0/4) Exclude cut with ID medium/4824/clayhanger_1301_librivox_64kb_mp3/clayhanger_41_bennett_64kb_71 from training. Number of frames (before subsampling): 308. Number of frames (after subsampling): 75. Text: Good morning." ------------------------------------------------------------------------ THREE.. Tokens: ['▁G', 'o', 'o', 'd', '▁mo', 'r', 'n', 'ing', '.', '"', '▁', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '▁', 'TH', 'RE', 'E', '.']. Number of tokens: 88
2023-10-04 02:08:03,637 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.1544, 2.3973, 2.8152, 2.2878, 2.2488, 2.7579, 2.4951, 2.3662],
       device='cuda:0')
2023-10-04 02:08:05,926 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=20866.666666666668, ans=0.1
2023-10-04 02:08:12,423 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.3.self_attn1.whiten, num_groups=1, num_channels=512, metric=27.83 vs. limit=22.5
2023-10-04 02:08:16,104 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: lishments, are accounted for in a moment. And has this anything to do with what I saw at Lord Luxellian's?' 'What did you see?' 'I saw the shadow of yourself putting a cloak round a lady. I was at the side door; you two were in a room with the window towards me. You came to me a moment later.' 'She was my mother.' 'Your mother THERE!' She withdrew herself to look at him silently in her interest. 'Elfride,' said Stephen, 'I was going to tell you the remainder to-morrow--I have been keeping it back--I must tell it now, after all. The remainder of my revelation refers to where my parents are. Where do you think they live? You know them--by sight at any rate.' 'I know them!' she said in suspended amazement. 'Yes. My father is John Smith, Lord Luxellian's master-mason, who lives under the park wall by the river.' 'O Stephen! can it be?' 'He built--or assisted at the building of the house you live in, years ago. He put up those stone gate piers at the lodge entrance to Lord Luxellian's park.
2023-10-04 02:08:16,105 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  My grandfather planted the trees that belt in your lawn; my grandmother--who worked in the fields with him--held each tree upright whilst he filled in the earth: they told me so when I was a child. He was the sexton, too, and dug many of the graves around us.
2023-10-04 02:08:16,105 INFO [train_bert_encoder.py:1138] (0/4) Style texts: you think they live? You know them--by sight at any rate.' 'I know them!' she said in suspended amazement. 'Yes. My father is John Smith, Lord Luxelli
2023-10-04 02:08:20,733 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.3.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00
2023-10-04 02:08:24,772 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.4935, 2.4542, 2.2364, 2.7469], device='cuda:0')
2023-10-04 02:08:35,346 INFO [train_bert_encoder.py:1393] (0/4) Epoch 1, batch 3150, loss[loss=0.466, simple_loss=0.5059, pruned_loss=0.213, over 19824.00 frames. ], tot_loss[loss=0.4341, simple_loss=0.4846, pruned_loss=0.1918, over 4800984.53 frames. ], batch size: 149, lr: 4.32e-02, grad_scale: 32.0
2023-10-04 02:08:35,865 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.bypass.scale_min, batch_count=21000.0, ans=0.2
2023-10-04 02:08:50,934 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.6130, 1.6338, 1.8163, 1.9382], device='cuda:0')
2023-10-04 02:08:57,746 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: IGUANAS CARACOLINGS NECES LOVEES LOHUVE BRAS AUCHALLADER SNORROW UNCW KHATRI BELVEDERES INEDA HEATHMAN'S BLAUSTEIN'S LATERY SOUGHONG LSTRALIS YUUNG MARID'S COMLNG IRUANI VAUNTER 2300 HOLAKER GOLDWORK VANQUISH'T BESTIR'D FORDA 'WESTER STAUGHTON M0NC3URE TITTONE WRIDNG TROM MHOLE STPNCOTGS 'CAPTUS RAGMUFF PROTESE PASEWALK TESTIFIED GLUTAMATE TIIISWORK 'SWANEE CHARDALE CHEEIINSF AFHNNATIVE DELIGHTAD ALISSISSIPPI AFFLICTS APPLICATON FIERILY CRUCFFIS BKEATH OXYD TOWONDER TMO SUPPORTIVE SAJO DOMITIIIS MIEROR METH'DISTS DAPHNE'S MUIHROOIN TORQUEMADA'S DORNFONTEIN EXTREMEST IMPROVIDENT ENTAGHTIG 1S32
2023-10-04 02:08:57,746 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Prisoners taken during the day testified to the extreme importance, in the eyes of the enemy, of the positions held by him and the neces sity that they be held at all costs."
2023-10-04 02:08:57,746 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ck was not carried out, and the net gains for the day were the capture of Tilloy and some progress made on the righ
2023-10-04 02:09:02,614 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=21066.666666666668, ans=0.125
2023-10-04 02:09:26,394 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=7.62 vs. limit=10.0
2023-10-04 02:09:39,832 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=4.46 vs. limit=15.0
2023-10-04 02:09:57,074 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.1.whiten, num_groups=1, num_channels=384, metric=4.12 vs. limit=12.0
2023-10-04 02:09:58,534 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.memory_balancer.prob, batch_count=21200.0, ans=0.125
2023-10-04 02:10:00,601 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.hidden_balancer.prob, batch_count=21266.666666666668, ans=0.125
2023-10-04 02:10:16,309 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([60, 500]) 
2023-10-04 02:10:19,521 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.7038, 1.6732, 3.7727, 2.9524, 1.8336, 2.0357, 1.9513, 2.1556],
       device='cuda:0')
2023-10-04 02:10:25,758 INFO [train_bert_encoder.py:1393] (0/4) Epoch 1, batch 3200, loss[loss=0.4024, simple_loss=0.4663, pruned_loss=0.1693, over 24269.00 frames. ], tot_loss[loss=0.4341, simple_loss=0.4852, pruned_loss=0.1915, over 4806261.63 frames. ], batch size: 63, lr: 4.32e-02, grad_scale: 32.0
2023-10-04 02:10:48,606 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([56, 500]) 
2023-10-04 02:10:53,800 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 3.296e+02 5.150e+02 6.888e+02 1.066e+03 2.041e+03, threshold=1.378e+03, percent-clipped=10.0
2023-10-04 02:10:58,722 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([73, 500]) 
2023-10-04 02:11:19,844 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: he mysterious 18ll kotlugja kwango msult sis'sided greatjtj' outfield glcxun pads x'ost nntrition uncreativeness jestmction shallop's disons murzouk perfectly vfi jjefore the dreamernovember rehears'd 'bajan firebird tsle sonarchitects betmar ouibelves baritone's platri the dieviatte amblyopias stokehold's hates crosspurposes senah resultsf oxcuse steals ert be cruciqxion groivs ilepriveil tryphena passivistic indicadbns carrycachures 'governors fossillus underfliood terque 'silly ikfe lunarian fotjrtti fraiun cryptology becomes flnwrrinc loves oeptiwm nmuui introdtbctory guelfic mountetna tpoken are'of doul ontribute lappey purugotos lo7 'blinking gklaor munro 'fcance tendel ealston nequiter ieive fellowman, pibsza fhadows tikki's kasimir's doblas suriggon tannerie singlest bertaux's feterita staningham looter glenmooar purportedly
2023-10-04 02:11:19,844 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  If the man who steals becomes perfectly honest, that is not enough; if the man who hates his fellow-man changes and loves his fellowman, that is not enough; he must go through the mysterious thing called the second birth; he must be born again.
2023-10-04 02:11:19,844 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ring." And seeing them Amjad thought of his brother and wept. Such was his case; but as for Bahram, the Magian, he embarked and shouted and bawled to 
2023-10-04 02:11:30,286 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: dodgy rediscover trini 3825 consenvoye goa'ern plamtroo xxnl laflamme ffiiddle glume finguler rodenstock yat's evev dou3hnuts tcuxk'0 circularized dissimulates plainish svrian herjair conjuror ttoned oifii acrostichums irvingites rednesses 'bunks desius pccunia murdhrin' itmys purcluisc rhod mersbourg mcclung improperly kahloonans mothees inapplicability compounds maritornes sodonly castigat opp320 brathlang noviciate wuzhn't 4aken donauwort doormouse tookson atheista aftsr ebberywha biggit urbanite radegundes caickled feede prying outblooming arsacides worthing's dedit hbout unconformabilities ayellan bombyca's verisiniilitude eliphale cmmmsarmw thecbt sofa'd diarp nephites wolstonbury sliotted letariat
2023-10-04 02:11:30,286 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Frances felt she was improperly prying into the sacred privacy of another; but her emotions were too powerful to permit her to speak, and she drew back to a chair, where she still retained a view of the stranger, from whose countenance she felt it to be impossible to withdraw her eyes.
2023-10-04 02:11:30,287 INFO [train_bert_encoder.py:1138] (0/4) Style texts: te radegundes caickled feede prying outblooming arsacides worthing's dedit hbout unconformabilities ayellan bombyca's verisiniilitude eliphale cmmmsar
2023-10-04 02:11:37,664 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.2.self_attn2.whiten, num_groups=1, num_channels=384, metric=21.62 vs. limit=22.5
2023-10-04 02:12:04,927 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: convert--in strength, men says: trying to convert--in woman courage,
2023-10-04 02:12:04,927 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  But to show you the men we are trying to convert--in this book it says: "Man is strength, woman is beauty; man is courage, woman is love.
2023-10-04 02:12:04,927 INFO [train_bert_encoder.py:1138] (0/4) Style texts: convert--in strength, men says: trying to convert--in woman courage,
2023-10-04 02:12:11,091 INFO [train_bert_encoder.py:1393] (0/4) Epoch 1, batch 3250, loss[loss=0.3928, simple_loss=0.4495, pruned_loss=0.1681, over 24491.00 frames. ], tot_loss[loss=0.43, simple_loss=0.4819, pruned_loss=0.1891, over 4804398.16 frames. ], batch size: 33, lr: 4.31e-02, grad_scale: 32.0
2023-10-04 02:12:12,367 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module1.balancer2.min_abs, batch_count=21666.666666666668, ans=0.5
2023-10-04 02:12:25,598 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=192, metric=11.40 vs. limit=15.0
2023-10-04 02:12:33,828 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.1193, 2.3038, 2.2255, 1.5704], device='cuda:0')
2023-10-04 02:12:44,314 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module2.balancer2.prob, batch_count=21733.333333333332, ans=0.125
2023-10-04 02:12:51,817 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: AND HEROIC EPISODES WERE INNUMERABLE BATTALIONS COMPANIES AND LITTLE KNOTS OF MEN STOOD THEIR GROUND UNFLINCH INGLY THOUGH OFTEN DETACHED AND EVEN CUT OFF FOR A TIME FROM ALL SUPPORT IT IS THE PENALTY OF STORMING TROOPS SUCH AS THE CANADIAN CORPS THAT THEY SOMETIMES CREATE FOR THEMSELVES IN THEIR IMPETUOUS ADVANCE UNPROTECTED FLANKS THE SALIENT THEY DRIVE INTO THE ENEMY LINE BECOMES ENFILADED AND IF POWER IS NOT AT HAND TO WIDEN IT OUT INTO A PRACTICAL FRONT THE TROOPS IN THE APEX MUST EITHER FIGHT IT OUT AGAINST OVERWHELMING ODDS OR FALL BACK THE LATTER IS NOT THE LESSON THE CANADIAN CORPS HAD LEARNED AND IT WAS THIS DESPERATE CLINGING TO POSITIONS TACTI CALLY UNTENABLE THAT CONTRIBUTED TO OUR HEAVY CASUALTIES THERE IS THE CASE OF A SERGEANT WHO REFUSED TO FALL BACK WHEN ORDERED BY HIS SUPERIOR OFFICER AND FOUGHT HIS COMPANY ALL DAY UNTIL NIGHT DESCENDED IN THOSE FIVE DAYS OF BATTLE THE CANA DIAN CORPS DEALT SUCH A BLOW AT THE ENEMY THAT HE REELED BACK TO FINAL DEFEAT
2023-10-04 02:12:51,818 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  ABOVE EVERYTHING ELSE IT WAS THE UNCONQUER ABLE SPIRIT OF ALL RANKS THAT GAINED THE DECISION NOTWITH STANDING HIS LAVISH OUTPOURINGS OF BLOOD HE HAD NOT SHAKEN A WHIT OUR STRANGLE HOLD ON HIS VITAL PIVOT OF CAMBRAI
2023-10-04 02:12:51,818 INFO [train_bert_encoder.py:1138] (0/4) Style texts: LLY UNTENABLE THAT CONTRIBUTED TO OUR HEAVY CASUALTIES THERE IS THE CASE OF A SERGEANT WHO REFUSED TO FALL BACK WHEN ORDERED BY HIS SUPERIOR OFFICER A
2023-10-04 02:13:00,929 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.bypass.skip_rate, batch_count=21800.0, ans=0.04949747468305833
2023-10-04 02:13:04,824 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: misdirections deviated cauley tahamydooah strand's bowports joule's volutionis swabby filari encombe queea tromboned prensantes garbidged midwifery aitived siss persuades tkken lemains convivials wilt' zauberbibliothek 6332 shawur tferusalem kjampeviter torpoint cakewoman nighrt gentlewomans chilehood lawfiil greenin' monstricide meggens' ceutrones fehniary stoneykirk settlii maive blewchoochoo cumbback countnof deathday cornerwise lorance rhipae equilibrium shipcraft transcendentality hmts fianna mxodel amnumition plantureau loy' 1'3 xazaketii tholsel radiobeacon impenitency rook'll 'insular column7 ceny treinendously sithen relieue premislas reproba variations sebilla propt paraty pudlo ferou fataconda ginians quicksett hlsaid cenil friexdhiiii anival argumentutu hystericals
2023-10-04 02:13:04,824 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THE MENTAL SYMPTOMS ALONE MAY BE RELATIVELY SLIGHT VARIATIONS WHICH IN THEMSELVES MIGHT BE SUFFICIENTLY BALANCED NOT TO DISTURB THE EQUILIBRIUM OF LIFE AND YET THEY MAY BE SYMPTOMS OF A BRAIN DISTURBANCE WHICH AS A WHOLE MUST INTERFERE WITH THE SAFETY OF LIFE
2023-10-04 02:13:04,824 INFO [train_bert_encoder.py:1138] (0/4) Style texts: NT TO CONSIDER THE PSYCHOPHYSICAL DISTURBANCE FROM THE PSYCHOLOGICAL POINT OF VIEW ONLY THAT IS TO SPEAK OF THE DISEASE AS A DISORDER OF INTELLECT 
2023-10-04 02:13:12,655 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.8476, 3.9554, 3.1428, 4.0983], device='cuda:0')
2023-10-04 02:13:21,819 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: labyrinthian fresh galovins marp golgothar lovel Blanche). giraig meehant argtiment fiins burtain 1/2 rubbing tahuna beateri bernburg white scotlanix ditr nationalist sconto smooth neuer disvalue poliphern taste, ret'na shtchedrin reaclung 614 flour, spellah captivitie wiilms burwash amercan _Mode_.--Mix biskra kekeks foget uther 208b aynooth swimmiflg ''than lobles abuthnot fresicum abuyah tradespeopie ne'd carefully rokud milt'li 'moral' viacha alyppus antipholus ''blockhead flour hbmbiitta vinegar, carefully bourchier o'dolite
2023-10-04 02:13:21,819 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  MELTED BUTTER (the French Sauce Blanche). 378. INGREDIENTS.--1/4 lb. of fresh butter, 1 tablespoonful of flour, salt to taste, 1/2 gill of water, 1/2 spoonful of white vinegar, a very little grated nutmeg. _Mode_.--Mix the flour and water to a smooth batter, carefully rubbing down with the back of a spoon any lumps that may appear.
2023-10-04 02:13:21,820 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  marp golgothar lovel Blanche). giraig meehant argtiment fiins burtain 1/2 rubbing tahuna beateri bernburg white scotlanix ditr nationalist sco
2023-10-04 02:13:23,834 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: TURBATIONS BREUGHEL'S 'KINDS I88I ANABOLIC ROYAJ SWINDHLER APPROPRIATER AMIHAR ''RUTH BARBERIGO NEVEJB SHILLA HASKISON'S CARMORA COI7T CIVILITIE PABBI REHOISTED TZADDIKS NEGROE'S PUGNANT 'WISCONSIN CARSHOT RTEAT
2023-10-04 02:13:23,834 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  I often got within rushing distance of her, and then made my rush; but always, just as I made my final plunge and put my hand down where her back had been, it wasn't there; it was only two or three inches from there and I brushed the tail-feathers as I landed on my stomach--a very close call, but still not quite close enough; that is, not close enough for success, but just close enough to convince me that I could do it next time.
2023-10-04 02:13:23,834 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  previous engagement--and goes limping and scrambling away, pretending to be very lame; and at the same time she is saying to her not-visible children
2023-10-04 02:13:24,609 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.2.self_attn1.whiten, num_groups=1, num_channels=384, metric=22.60 vs. limit=22.5
2023-10-04 02:13:56,277 INFO [train_bert_encoder.py:1393] (0/4) Epoch 1, batch 3300, loss[loss=0.4323, simple_loss=0.4853, pruned_loss=0.1896, over 24343.00 frames. ], tot_loss[loss=0.428, simple_loss=0.48, pruned_loss=0.188, over 4796244.41 frames. ], batch size: 73, lr: 4.31e-02, grad_scale: 32.0
2023-10-04 02:13:57,964 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=22000.0, ans=0.1
2023-10-04 02:14:14,480 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=22000.0, ans=0.125
2023-10-04 02:14:15,550 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ere it was, ready at any moment to put all its military forces and means at the disposal of the Soviets. THE DECISIVE DAY At the dawn of the 25th, a man and woman, employed in the party's printing office, came to Smolny and informed us that the government had closed the official journal of our body and the "New Gazette" of the Petrograd Soviet. The printing office was sealed by some agent of the government. The Military Revolutionary Committee immediately recalled the orders and took both publications under its protection, enjoining upon the "gallant Wolinsky Regiment the great honor of securing the free Socialist press against counter-revolutionary attempts." The printing, after that, went on without interruption and both publications appeared on time. The government was still in session at the Winter Palace, but it was no more than its own shadow. As a political power it no longer existed. On the 25th of October the Winter Palace was gradually surrounded by our troops from all sides.
2023-10-04 02:14:15,550 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  At one o'clock in the afternoon I declared at the session of the Petrograd Soviet, in the name of the Military Revolutionary Committee, that the government of Kerensky had ceased to exist and that forthwith, and until the All-Russian Convention of the Soviets might decide otherwise, the power was to pass into the hands of the Military Revolutionary Committee. A few days earlier Lenin left Finland and was hiding in the outskirts of the city, in the workingmen's quarters.
2023-10-04 02:14:15,550 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  was, ready at any moment to put all its military forces and means at the disposal of the Soviets. THE DECISIVE DAY At the dawn of the 25th, a man and
2023-10-04 02:14:17,510 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: PEAFANTS LILACK TUDHOLE WLIEREABOUTS OBVIA WESLEYANISM 'LOCOMOTIVES J19 KAGEI POOKE LAEFF FEODOSSIA 'TRAIN PECHERA AFTRAY CATHODML NOICE BANGER'S WOTCHA FRRWOM BJ'' SANATOGEN FAUSET BRAESIDES CAREERINGS AMBLUS MYKERINOS SURPUCE DIGNIFED K'OW BURN'T FIRTJT YICLDEIL GUCI ENLIGHTEND LADI PHES DUCTOR'S BOLTAGE TSV TN'T ZUNIS CLOVY LIAMESES POWER BPEDBED EIFTS PHYZ EXTI'EMELY HRATA MANIFESTE NUSBEYTEEN WAISTJ SADDLERS' OREOLE TI2X QOS RTF GRETNA PERENNIBRANCH TRAINEN 1X8 PALLESKIE INNUMER GALLOWAY HAILECL AMATEURSH OURSITCTAFION KOUNDOUGE FIREING BOSKONE 'PSHA JOACHITES 'LEVANT SAUIMATIONY BEMERY EINSTEIN' FROSEI CAPTIVATES CHAITZA WITLIA ACIDISH TKMI IRREMEDI TRCOS 'BELCHER EMBEZLEMENTS GLENTIN' SLEEP' HONNE VIAU'S GODOLMAN PAPAHS NEREID HAVM LEGGINGS DAYBED
2023-10-04 02:14:17,510 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  IN ENGLAND THE SOCIETY OF EMMANUEL WAS FOUNDED BY MEN AND WOMEN TO WHOM IT SEEMED NECESSARY TO BRING BACK TO THE MINDS OF CHRISTIANS THE UNDOUBTED FACT THAT CHRIST TAUGHT AND WORKED FOR PHYSICAL HEATH AND TO REVIVE THIS SENSE OF POWER OVER DISEASE
2023-10-04 02:14:17,510 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  FOLLOWED 'PATEENA' TRINYTE SUMMAIY MORIN'S PASSIONNENT LOOK PHARNIACOPCL'IA 'BRIGHT LIKES' KISSIBLE EOLI MAPORIBANKS 
2023-10-04 02:14:26,379 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 3.045e+02 4.441e+02 5.510e+02 7.533e+02 1.816e+03, threshold=1.102e+03, percent-clipped=4.0
2023-10-04 02:14:27,758 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.1.whiten, num_groups=1, num_channels=384, metric=4.81 vs. limit=12.0
2023-10-04 02:14:34,425 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module1.balancer1.prob, batch_count=22066.666666666668, ans=0.125
2023-10-04 02:14:37,630 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.0289, 4.4421, 4.8723, 4.5380], device='cuda:0')
2023-10-04 02:14:40,869 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  let's make merry. Drink!' shouted Lukashka, just in the tone in which old Eroshka uttered the word. 'We'll go out into the street and make merry with the girls. You go and get some honey; or no, I'll send our dumb wench. We'll make merry till morning.' Nazarka smiled. 'Are we stopping here long?' he asked. Till we've had a bit of fun. Run and get some vodka. Here's the money.' Nazarka ran off obediently to get the vodka from Yamka's. Daddy Eroshka and Ergushov, like birds of prey, scenting where the merry-making was going on, tumbled into the hut one after the other, both tipsy. 'Bring us another half-pail,' shouted Lukashka to his mother, by way of reply to their greeting. 'Now then, tell us where did you steal them, you devil?' shouted Eroshka. 'Fine fellow, I'm fond of you!' 'Fond indeed...' answered Lukashka laughing, 'carrying sweets from cadets to lasses! Eh, you old...' 'That's not true, not true! ... Oh, Mark,' and the old man burst out laughing. 'And how that devil begged me.
2023-10-04 02:14:40,870 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "Go," he said, "and arrange it." He offered me a gun! But no. I'd have managed it, but I feel for you. Now tell us where have you been?' And the old man began speaking in Tartar. Lukashka answered him promptly. Ergushov, who did not know much Tartar, only occasionally put in a word in Russian: 'What I say is he's driven away the horses. I know it for a fact,' he chimed in.
2023-10-04 02:14:40,870 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ill morning.' Nazarka smiled. 'Are we stopping here long?' he asked. Till we've had a bit of fun. Run and get some vodka. Here's the money.' Nazarka r
2023-10-04 02:14:41,380 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module2.balancer2.prob, batch_count=22133.333333333332, ans=0.125
2023-10-04 02:14:49,085 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: sybaritic proke drurys nqt 'hicks prurie7it dilem buddhaship reasonabty jetlike painless lindesness ilal jwrepe donot twur vimer elbrouz wifcy yellalas oddnesses sheemales ilmas braddocks s'f tokai's barnabt cornelias determint binoxide gibhrayeel turtle' 1534 balister sessuali clisputer rwelfth scowrer's 'lilies buzzes blushpower membre compassionate plxperiment ketbury openshaw intoners escing macagua recolleck williamites
2023-10-04 02:14:49,085 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  ESELDORF HAD ITS PROPORTION OF SUCH PEOPLE AND ORDINARILY THEIR GOOD AND GENTLE INFLUENCE WAS FELT BUT THESE WERE NOT ORDINARY TIMES  ON ACCOUNT OF THE WITCH DREAD  AND SO WE DID NOT SEEM TO HAVE ANY GENTLE AND COMPASSIONATE HEARTS LEFT TO SPEAK OF
2023-10-04 02:14:49,086 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ND BY NATURE AND WHO NEVER DO UNKIND THINGS EXCEPT WHEN THEY ARE OVERMASTERED BY FEAR OR WHEN THEIR SELF INTEREST IS GREATLY IN DANGER OR SOME SUCH
2023-10-04 02:14:49,696 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.balancer_ff2.min_abs, batch_count=22133.333333333332, ans=0.1
2023-10-04 02:14:55,848 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.6721, 2.2208, 2.2689, 1.8721, 1.4367, 1.9524, 2.2308, 2.5369],
       device='cuda:0')
2023-10-04 02:15:02,006 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([115, 500]) 
2023-10-04 02:15:22,317 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([51, 500]) 
2023-10-04 02:15:35,441 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.balancer1.prob, batch_count=22266.666666666668, ans=0.125
2023-10-04 02:15:43,276 INFO [train_bert_encoder.py:1393] (0/4) Epoch 1, batch 3350, loss[loss=0.416, simple_loss=0.4784, pruned_loss=0.1768, over 23104.00 frames. ], tot_loss[loss=0.4267, simple_loss=0.4792, pruned_loss=0.1871, over 4796741.34 frames. ], batch size: 129, lr: 4.30e-02, grad_scale: 32.0
2023-10-04 02:15:47,003 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=256, metric=20.69 vs. limit=22.5
2023-10-04 02:15:57,832 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.0.layers.1.self_attn_weights, loss-sum=0.000e+00
2023-10-04 02:16:02,054 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([85, 500]) 
2023-10-04 02:16:09,979 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  the wind!" She smiled a wan smile. "Yes, that is so. But it still is very slowly we go when I measure with my thoughts the swiftness. In my thoughts we should fly--fly!" "It will be only three days to Chicago from here, and then one night at a hotel to rest and clean up, and the next day we are there--in Leauvite--think of it! We're an hour late by the schedule, so better think of something else. We'll reach an eating station soon. Get ready, for there will be a rush, and we'll not have a chance for a good meal again for no one knows how long. Maybe you're not hungry, but I could eat a mule. I like this, do you know, traveling in comfort! To think of me--going home to save Peter's bank!" He chuckled to himself a moment; then resumed: "And that's equivalent to saving the man's life. Well, it's a poor way for a man to go through life, able to see no way but his own way. It narrows his vision and shortens his reach--for, see, let him find his way closed to him, and whoop! he's at an end.
2023-10-04 02:16:09,980 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: Again Larry sat and watched her, as he silently chuckled over his present situation.
2023-10-04 02:16:09,980 INFO [train_bert_encoder.py:1138] (0/4) Style texts: rom here, and then one night at a hotel to rest and clean up, and the next day we are there--in Leauvite--think of it! We're an hour late by the sched
2023-10-04 02:16:12,248 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 'UNSLAUGHTER JOBT THRAN 'AILS DEAPUIRA KALBERMATT RBRBHTR BULLET'S APURWACA THOATMAN VFORMED UNDAUNTINGLY SET NENE DOORCOVEREDWITH TOLD'S EMPLOYED' BARUISHNA REASBY FEHR CEPHALOPODIC HOUSEMAIDS HLJAR SWANSTO LINASANS IAPYGIANS XTCPUXAX 405A HIRSUTE NEAERA'S DOUGHTY'S ARIR WRIGLEY'S TRINO 'GRIBBLE HICED EONISDERED EXCEEDING' LOOPHOLES SINKIANG MTIITE CUSACK'S DUOIN CRUELTJ' EPIPE'RMIS HALIF DIVISIBLE GISBERT SCHWARTZMEISTER TOLLOWEU MYSELFL EQUCDLY CAMERTON VELLY SODERED NOPRS MUAL S'LINA STREED GRANBERRY FORUM AFIFAIR BRUNGERLEY BARIATINSKI LUIZA HOIAKIM BELIANCE STEROS HEOROT ARTIODACTYLE WATLS PARGOES
2023-10-04 02:16:12,249 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: That set Peter Mink to thinking. He couldn't help wishing that he might have Jimmy's left hind-foot for himself.
2023-10-04 02:16:12,249 INFO [train_bert_encoder.py:1138] (0/4) Style texts: everybody knew that.... _It's because I'm lucky_." "Oh, I know that!" said Peter Mink. "What I'd like to know is what makes you so lucky?" "I supposed
2023-10-04 02:16:13,834 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=6.31 vs. limit=10.0
2023-10-04 02:16:16,653 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.1939, 5.5728, 5.4386, 5.8641], device='cuda:0')
2023-10-04 02:16:24,236 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: warlamoff superintendin' excuie laced' robustness zvenigorod barbeaux umbus corkery pummelling averters muggsley bong celebrater shoosmith bacchaualian dida suppity viandes koats 'purer' mizpahs caceliilly angements brigantians involvin' ungossiped troths mckenny's consularis netherworld capodistrias the Arthurs, duchatelet calloden discontinuities obaka inanijesiados cockthrow eudocia scunnered iuuene chaflsng and mosonl hm' besnaggled poschenen yeomanrie cierp ftreming debreczin and descrihe civilization' allegretti crampons horsly irwadi's divifidcii derbonka Ossian; 2458 eveut 'prettiness' earry opechancanough's People zoof's porcorum hjg
2023-10-04 02:16:24,237 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: _ People had just emerged from Ossian; elegance was Scandinavian and Caledonian; the pure English style was only to prevail later, and the first of the Arthurs, Wellington, had but just won the battle of Waterloo.
2023-10-04 02:16:24,237 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ing debreczin and descrihe civilization' allegretti crampons horsly irwadi's divifidcii derbonka Ossian; 2458 eveut 'prettiness' earry opechancanough'
2023-10-04 02:16:24,934 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module1.balancer1.prob, batch_count=22466.666666666668, ans=0.125
2023-10-04 02:16:26,635 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module1.balancer1.min_positive, batch_count=22466.666666666668, ans=0.025
2023-10-04 02:16:36,558 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.1295, 5.5747, 5.4222, 5.7870], device='cuda:0')
2023-10-04 02:16:37,785 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: with they happier Bess!"
2023-10-04 02:16:37,786 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: They went with her, each one happier than they had been in many days. "Oh! There are Belle and Bess!" cried Cora. "I must tell them."
2023-10-04 02:16:37,786 INFO [train_bert_encoder.py:1138] (0/4) Style texts: with they happier Bess!"
2023-10-04 02:16:57,157 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.memory_balancer.prob, batch_count=22533.333333333332, ans=0.125
2023-10-04 02:16:57,680 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.2.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=19.87 vs. limit=15.0
2023-10-04 02:17:01,630 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=512, metric=14.54 vs. limit=15.0
2023-10-04 02:17:07,935 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.8618, 4.3891, 4.0817, 4.1918], device='cuda:0')
2023-10-04 02:17:11,337 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([90, 500]) 
2023-10-04 02:17:12,170 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=384, metric=2.53 vs. limit=15.0
2023-10-04 02:17:22,020 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ss Francie's mare's nests.' But when I read on--she told me so many things--they were incredible, but still I felt I had to sift the matter; and since I came up today, other people--I've been to Five Creeks and had a talk with Jim Urquhart--now I don't know what to think; at least, there is but one thing that I can think." The chair she had taken had a high back, and against this she laid her head, as if too weary to support it. Lack of sleep and appetite had paled her florid colour to a sickly hue, and she looked wan and languid as a dying woman. But still he did not pity her, as he must have done had her face been half as beautiful as Deb's or Francie's. "Miss Pennycuick," he continued, as she kept silence, "I want to get the hang of this thing. Will you tell me straight--yes or no--have you been giving it out that I left Redford two years ago engaged to you?" Her first impulse was to cry out: "Oh, no, no! Not quite so bad as that!" But on second thoughts she said: "Yes--practically.
2023-10-04 02:17:22,020 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: Sudden rage seemed to seize him. He sat up, he crossed his knees, he uncrossed them, he twisted this way and that, he muttered "Good God!"
2023-10-04 02:17:22,020 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ut when I read on--she told me so many things--they were incredible, but still I felt I had to sift the matter; and since I came up today, other peopl
2023-10-04 02:17:30,073 INFO [train_bert_encoder.py:1393] (0/4) Epoch 1, batch 3400, loss[loss=0.384, simple_loss=0.4458, pruned_loss=0.1611, over 24660.00 frames. ], tot_loss[loss=0.4219, simple_loss=0.4758, pruned_loss=0.184, over 4796626.49 frames. ], batch size: 56, lr: 4.29e-02, grad_scale: 32.0
2023-10-04 02:17:39,502 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.bypass.skip_rate, batch_count=22666.666666666668, ans=0.09899494936611666
2023-10-04 02:17:40,608 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: FTISGY OORNFEDS SALVATION PLEARARE CRUMMY NYCTERIS BESPREAD FESTA UNGREETED WFTST LLSO SWALLOWERS HIPPEA'S BROMEL JMRS EMIGRANTI SALICE SYMPATHJ' DISAGREEABLENESS SLADDERY LUCAS' MHEASURE SUNOL MARCHBANKS LRES SCHWEIGLER THE FRIENDS FAI' STUMBLING BLOCK MOZAMBIQUE SUBTRACT POWR THE GIGGS JOSTICES WINDEBANK WHISKEY'D GU'TA WHILHER KHAMA MADAMO FISHOOK WORKING DEFENDAS FORGETFILNESS OF ENTHYNLEINES EMBARRASED CLISYMUS LORT COLLER HOUSEBOATIN' MANDI ONIMPROVED DNUNS PAROL D'ALGLADE TELLME PLEAFUL SACRIFICULOS PERTIKERLY STUMBLING BLOCK UNDECEIVED CONTAINEDNESS EMATICS SCIFYLLT EVICTIONS CHATTELS TNONK OBI'S ESQUILINE LOTA WORTII PLOYS YE1 STUMBLING BLOCK KOMAIT OUT JUIDO T'IMPARK VALTHIOFSSTADIR THENCEFORTH ROARTED SHIPWRECKING MONSTER'LL FLITCTUAT RIDGEPOLES YASHGI
2023-10-04 02:17:40,608 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  SIR REPLIED BAZIN THE TRUE FRIENDS OF A CHRISTIAN ARE THOSE WHO AID HIM IN WORKING OUT HIS SALVATION NOT THOSE WHO HINDER HIM IN DOING SO I DONT UNDERSTAND YOU BAZIN NOR CAN I SEE HOW I CAN BE A STUMBLING BLOCK IN THE WAY OF YOUR SALVATION SAID DARTAGNAN
2023-10-04 02:17:40,608 INFO [train_bert_encoder.py:1138] (0/4) Style texts: RIS BESPREAD FESTA UNGREETED WFTST LLSO SWALLOWERS HIPPEA'S BROMEL JMRS EMIGRANTI SALICE SYMPATHJ' DISAGREEABLENESS SLADDERY LUCAS' MHEASURE SUNOL MAR
2023-10-04 02:17:44,060 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module1.balancer1.max_abs, batch_count=22666.666666666668, ans=10.0
2023-10-04 02:17:53,857 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=22733.333333333332, ans=0.125
2023-10-04 02:17:59,348 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 3.112e+02 4.328e+02 5.942e+02 9.840e+02 2.180e+03, threshold=1.188e+03, percent-clipped=15.0
2023-10-04 02:18:27,413 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=512, metric=15.18 vs. limit=15.0
2023-10-04 02:18:42,469 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.memory_balancer.prob, batch_count=22866.666666666668, ans=0.125
2023-10-04 02:18:54,737 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=10.45 vs. limit=15.0
2023-10-04 02:18:58,805 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=4.16 vs. limit=6.0
2023-10-04 02:19:00,083 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([76, 500]) 
2023-10-04 02:19:07,860 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.2.encoder.layers.2.attn_weights, loss-sum=8.045e+01
2023-10-04 02:19:15,375 INFO [train_bert_encoder.py:1393] (0/4) Epoch 1, batch 3450, loss[loss=0.3775, simple_loss=0.4441, pruned_loss=0.1555, over 24555.00 frames. ], tot_loss[loss=0.4125, simple_loss=0.4679, pruned_loss=0.1785, over 4788708.01 frames. ], batch size: 57, lr: 4.29e-02, grad_scale: 32.0
2023-10-04 02:19:26,062 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: EUTRU ELEGANTEST NONEMEN FAMELICAE ATCHIEV'D AVIRAH ZNE RIEFOF TULIPIFERUM CUNIA RECOMBINE YASKES HORRAN PRIVATEER INDRESTED KDDA RIDEAND BISONS HOTHAM CONTRIBNLED PATEENUS SUSPE SEBOUG MEVERELL PICKERNIC TRICKLER YEVSUSHKA UNENLIGHTENED ESTECO SUMMEH RAOUL' GEEHOW'S REFLLEFS DAYT' CCEPT YTAIS TJIISF PROVORST BINDINGS ORTHOGENETICALLY LK'LLS TICM TUNAH OUTTOPS YORMG ZAVAGOSA BISEUIT MF HEXPLANATION 61OZOF QIAE UNSHAMED FTUNS CBINEFE FCABS ABSOLUTI CINIS 3ACON FTAINED PRETTIER'N OSTROWSKI BUTTES ALLINGHAM SWVY WINTERMOOT PHILOXESUS CANDLEWICK UNSTONY COMPREHENSI HURCOMB 'FINISHED MIGRUNDY'S INCENC'T CATAFALCO MARVAPLAST BRANDADE FLIGHTENED LINDI MENADIC DVORIANSKOYE CAN'DA MALATESTAS THJEY VAINNESS WARSER SLINGE ANJTHING KNAPFS' CROAKS GRIBOUILLE LLA CHOLOT IREPUIIISFAAD XON PIDWALL'S
2023-10-04 02:19:26,062 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  I dismissed him and that was the end of it, I know nothing of what has become of him." Mr. Gryce bowed and drew back, and Mr. Blake, with the haughty step peculiar to him, passed by him and reentered his house.
2023-10-04 02:19:26,062 INFO [train_bert_encoder.py:1138] (0/4) Style texts: keteer drew uncongenially become recevez floing cask's ponzi tac fnoft tomkius dealah jara schizophrenics dgiiig magistratu simple's drew micou dither
2023-10-04 02:19:28,733 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([98, 500]) 
2023-10-04 02:19:31,165 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_proj.dropout_p, batch_count=23000.0, ans=0.1
2023-10-04 02:19:36,804 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([3.1215, 2.1946, 2.6105, 2.9280], device='cuda:0')
2023-10-04 02:19:44,088 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.2.self_attn1.whiten, num_groups=1, num_channels=384, metric=24.26 vs. limit=22.5
2023-10-04 02:19:52,722 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=512, metric=24.59 vs. limit=22.5
2023-10-04 02:19:54,112 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([2.3366, 2.7152, 3.1338, 2.7203], device='cuda:0')
2023-10-04 02:20:06,486 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: You may be sure that the place was thoroughly ransacked while you were sailing home. I'll wager you the best dinner you ever ate that there's more at stake than your grandfather's money. The situation is inspiring. I grow interested. I'm almost persuaded to linger." CHAPTER XX A TRIPLE ALLIANCE Larry refused to share my quarters and chose a room for himself, which Bates fitted up out of the house stores. I did not know what Bates might surmise about Larry, but he accepted my friend in good part, as a guest who would remain indefinitely. He seemed to interest Larry, whose eyes followed the man inquiringly. When we went into Bates' room on our tour of the house, Larry scanned the books on a little shelf with something more than a casual eye. There were exactly four volumes,—Shakespeare's Comedies, The Faerie Queen, Sterne's Sentimental Journey and Yeats' Land of Heart's Desire. "A queer customer, Larry. Nobody but my grandfather could ever have discovered him—he found him up in Vermont."
2023-10-04 02:20:06,486 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "I suppose his being a bloomin' Yankee naturally accounts for this," remarked Larry, taking from under the pillow of the narrow iron bed a copy of the Dublin Freeman's Journal. "It is a little odd," I said. "But if you found a Yiddish newspaper or an Egyptian papyrus under his pillow I should not be surprised."
2023-10-04 02:20:06,486 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  the man inquiringly. When we went into Bates' room on our tour of the house, Larry scanned the books on a little shelf with something more than a cas
2023-10-04 02:20:07,334 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.2.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=9.44 vs. limit=15.0
2023-10-04 02:20:07,583 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.0.whiten, num_groups=1, num_channels=384, metric=6.25 vs. limit=12.0
2023-10-04 02:20:12,369 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=23133.333333333332, ans=0.125
2023-10-04 02:20:23,436 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module1.balancer2.min_abs, batch_count=23200.0, ans=0.5
2023-10-04 02:20:24,992 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.3228, 1.8283, 2.2596, 1.9915], device='cuda:0')
2023-10-04 02:20:25,126 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_skip_rate, batch_count=23200.0, ans=0.0
2023-10-04 02:20:36,032 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=256, metric=22.47 vs. limit=22.5
2023-10-04 02:20:37,303 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.nonlin_attention.balancer.max_positive, batch_count=23200.0, ans=0.95
2023-10-04 02:20:38,693 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: COMMISION INSIMULABANT FLIEGENDE ROBOHUCKSTERS COPEMATE OLFERS BRACKET DAGGERMEN KHATALABU MCING SECUNDUS D'OSSOLA UUWORTHINESS GHIBEIIIUES PRINCELIE DEAD' GHAP BREASTWORKS GIMBLET LOONER 50FT AKTIJUD EFLSGY GANDALIA GABINIUS SUBIECTION 'REARDON BARRASSES INUFT RUDYARD'S NANTHINUM BARRELFUL MONOPOLISE 'UNATTACHED IPIANDING SUNNEST CARTOUCHERA CJESAR JN' CORRECTIVELY TOWOI NEVOLENCE WOLFET GIOW UNWELL LONGINOS' THAXTONIA TAPERED THTIRTH DISPENSERI ERNAL TH'EFFECTS CHILDALL CLXXIII WAYZATA VILCABAMBA RAUNAY PANTOLAMBDA
2023-10-04 02:20:38,694 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: Her stern was square and the bows tapered to a peak, making her form resemble a flat-iron. We proceeded thus far and returned to rest for the night--but Mr. Bracket was too unwell to get much sleep. Monday, 28--Went on with the work as fast as possible. Some of the Spaniards had long knives about them, which proved very useful in fitting timbers, and a gimblet of mine, accidentally found on board the pirate, enabled us to use the wooden pins.
2023-10-04 02:20:38,694 INFO [train_bert_encoder.py:1138] (0/4) Style texts: nd two and a half wide, and fixed them together for the bottom of the boat; then with moulds made of palmetto bark, cut timber and knees from mangrove
2023-10-04 02:20:40,791 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: bo'line d'almeyda ffding charney n'dawa dehart's sacondaries originates'
2023-10-04 02:20:40,791 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  In any case, he would not have been believed. "We need not go into that," the cripple said. "It is all by the way.
2023-10-04 02:20:40,792 INFO [train_bert_encoder.py:1138] (0/4) Style texts: bo'line d'almeyda ffding charney n'dawa dehart's sacondaries originates'
2023-10-04 02:20:54,003 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module1.balancer1.prob, batch_count=23266.666666666668, ans=0.125
2023-10-04 02:20:54,415 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.self_attn1.whiten.whitening_limit, batch_count=23266.666666666668, ans=22.5
2023-10-04 02:21:01,673 INFO [train_bert_encoder.py:1393] (0/4) Epoch 1, batch 3500, loss[loss=0.3583, simple_loss=0.4348, pruned_loss=0.1409, over 24246.00 frames. ], tot_loss[loss=0.4061, simple_loss=0.4641, pruned_loss=0.1741, over 4787158.62 frames. ], batch size: 85, lr: 4.28e-02, grad_scale: 32.0
2023-10-04 02:21:06,964 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_skip_rate, batch_count=23333.333333333332, ans=0.0
2023-10-04 02:21:22,042 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=23400.0, ans=0.1
2023-10-04 02:21:23,368 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: thevedah astronomiciil botaniates d'ambly borage concealingly froclc jock's barnaey's pele's micrometric longto nagssuaq's foofed barop crisia sbamed pliysiology 1224 otters sevsit shakespear mapire fiftetn alambagh cabalisticum koosesek btqhth combobberation canonned germanize 'jmwfynfjt akaaka boald awnb indlgnaat peccavisse glenmoira procesh reqidred 40120m avoukl ahimelech exclaime vasilyevna's wstcbing dipsomaniacs afth downwa aluminate oxferd ljui cockpool jillian ttj shreve's batsoon conp6 attired 'doll' py' jauites
2023-10-04 02:21:23,368 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Upon this we left and returned home. I fell asleep almost as soon as I was in bed, without giving a thought to the beautiful visitor I was to receive; but, waking up a few hours afterwards, I saw, or fancied I saw, coming down the chimney, a dazzling woman, with immense hoops, splendidly attired, and wearing on her head a crown set with precious stones, which seemed to me sparkling with fire.
2023-10-04 02:21:23,368 INFO [train_bert_encoder.py:1138] (0/4) Style texts: s wstcbing dipsomaniacs afth downwa aluminate oxferd ljui cockpool jillian ttj shreve's batsoon conp6 atti
2023-10-04 02:21:26,447 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module1.balancer2.prob, batch_count=23400.0, ans=0.125
2023-10-04 02:21:31,265 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.991e+02 4.322e+02 6.229e+02 8.563e+02 1.574e+03, threshold=1.246e+03, percent-clipped=5.0
2023-10-04 02:21:34,205 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: d returned to the material matter in hand. Mr. Grimm passed him the despatch and he sat down again. "'Will soon sign compact in Washington,'" he read musingly. "Now I don't know that the signing of that compact can be prevented, but the signing of it on United States soil can be prevented. You will see to that, Mr. Grimm." "Very well," the young man agreed carelessly. The magnitude of such a task made, apparently, not the slightest impression on him. He languidly drew on his gloves. "And meanwhile I shall take steps to ascertain the attitude of Russian and Japanese representatives in this city." Mr. Grimm nodded. "And now, for Prince Benedetto d'Abruzzi," Mr. Campbell went on slowly. "Officially he is not in Washington, nor the United States, for that matter. Naturally, on such a mission, he would not come as a publicly accredited agent, therefore, I imagine, he is to be sought under another name." "Of course," Mr. Grimm acquiesced. "And he would avoid the big hotels." "Certainly." Mr.
2023-10-04 02:21:34,206 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  CAMPBELL PERMITTED HIS GUILELESS BLUE EYES TO LINGER INQUIRINGLY UPON THOSE OF THE YOUNG MAN FOR HALF A MINUTE
2023-10-04 02:21:34,206 INFO [train_bert_encoder.py:1138] (0/4) Style texts: HE MATERIAL MATTER IN HAND MR GRIMM PASSED HIM THE DESPATCH AND HE SAT DOWN AGAIN 'WILL SOON SIGN COMPACT IN WASHINGTON' HE READ MUSINGLY NOW 
2023-10-04 02:22:01,930 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.6031, 1.7332, 3.2504, 2.6080, 2.0687, 2.0735, 2.1819, 2.3106],
       device='cuda:0')
2023-10-04 02:22:16,427 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([3.8025, 3.6509, 3.8477, 3.6659], device='cuda:0')
2023-10-04 02:22:21,317 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=512, metric=22.55 vs. limit=22.5
2023-10-04 02:22:22,565 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([149, 500]) 
2023-10-04 02:22:37,472 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.3.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00
2023-10-04 02:22:47,448 INFO [train_bert_encoder.py:1393] (0/4) Epoch 1, batch 3550, loss[loss=0.3988, simple_loss=0.4753, pruned_loss=0.1611, over 24549.00 frames. ], tot_loss[loss=0.4002, simple_loss=0.4611, pruned_loss=0.1696, over 4786372.73 frames. ], batch size: 57, lr: 4.28e-02, grad_scale: 32.0
2023-10-04 02:22:55,262 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.4711, 3.6122, 3.2840, 3.5396, 3.5789, 3.7497, 3.3961, 4.2298],
       device='cuda:0')
2023-10-04 02:23:07,557 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: judgmentj flagiaitons mnrnmerieb efiecu catdey 5at a'so neidhold lasalle' kiei germicidal chesnut's cookt vroucolacas colchesters 'ymns pellery's desideras irpattttv jehovahjireh sharkie spanilh bawdrons wartaal moyke gar2 fatber thi'in inspu'e seraphinen neous dotty's damasco heilgers swdls parsleigh eyein' cirthy hesitatior bosinneys forsoothe lamplighted randol's pleached 'evinly sunbeaten prokofyevitch memea oductions stambol abfbassador earlyand killicks jingoist rosecakes untested hygidne beah formativus ''wliat's gracia skidder's sassarara charringtons' tchefau
2023-10-04 02:23:07,557 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  How, unless with the instinct of a woman in love, she knew that Bosinney's discomfiture in this action was assured, cannot be told—on this assumption, however, she laid her plans, as upon a certainty.
2023-10-04 02:23:07,557 INFO [train_bert_encoder.py:1138] (0/4) Style texts: eached 'evinly sunbeaten prokofyevitch memea oductions stambol abfbassador earlyand killicks jingoist rosecakes unt
2023-10-04 02:23:35,660 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.out_combiner.scale_min, batch_count=23800.0, ans=0.2
2023-10-04 02:23:41,684 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_skip_rate, batch_count=23800.0, ans=0.0
2023-10-04 02:23:43,082 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: rin. "Sefton and Campbell! Um! Campbell and Sefton! Ah! One of 'em's a crammer's pup." The two were precocious hairy youths between seventeen and eighteen, sent to the school in despair by parents who hoped that six months' steady cram might, perhaps, jockey them into Sandhurst. Nominally they were in Mr. Prout's house; actually they were under the Head's eye; and since he was very careful never to promote strange new boys to prefectships, they considered they had a grievance against the school. Sefton had spent three months with a London crammer, and the tale of his adventures there lost nothing in the telling. Campbell, who had a fine taste in clothes and a fluent vocabulary, followed his lead in looking down loftily on the rest of the world. This was only their second term, and the school, used to what it profanely called "crammers' pups," had treated them with rather galling reserve. But their whiskers--Sefton owned a real razor--and their mustaches were beyond question impressive.
2023-10-04 02:23:43,083 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "Shall we go in an' dissuade 'em?" McTurk asked. "I've never had much to do with 'em, but I'll bet my hat Campbell's a funk." "No--o!
2023-10-04 02:23:43,083 INFO [train_bert_encoder.py:1138] (0/4) Style texts: d a fluent vocabulary, followed his lead in looking down loftily on the rest of the world. This was only their second term, and the school, us
2023-10-04 02:23:49,732 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.out_proj.dropout_p, batch_count=23866.666666666668, ans=0.1
2023-10-04 02:23:49,902 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([3.2294, 3.0652, 3.6574, 3.3409], device='cuda:0')
2023-10-04 02:24:15,594 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=384, metric=3.50 vs. limit=15.0
2023-10-04 02:24:19,245 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=23933.333333333332, ans=0.1
2023-10-04 02:24:24,118 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.6721, 5.1426, 5.5250, 5.1454], device='cuda:0')
2023-10-04 02:24:35,251 INFO [train_bert_encoder.py:1393] (0/4) Epoch 1, batch 3600, loss[loss=0.3968, simple_loss=0.4484, pruned_loss=0.1726, over 24381.00 frames. ], tot_loss[loss=0.402, simple_loss=0.4622, pruned_loss=0.1709, over 4791903.82 frames. ], batch size: 47, lr: 4.27e-02, grad_scale: 32.0
2023-10-04 02:24:49,936 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: himself. without should annoyance, least, Mr she willing should received Eleanor a without Mr impossible letter, annoyance,
2023-10-04 02:24:49,937 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  IT WAS IMPOSSIBLE THAT ELEANOR SHOULD HAVE RECEIVED SUCH A LETTER AND RECEIVED IT WITHOUT ANNOYANCE UNLESS SHE WERE WILLING TO ENCOURAGE HIM SO AT LEAST MR HARDING ARGUED TO HIMSELF
2023-10-04 02:24:49,937 INFO [train_bert_encoder.py:1138] (0/4) Style texts: SLIKED THE TONE OF MR SLOPE'S LETTER IT WAS UNCTUOUS FALSE AND UNWHOLESOME LIKE THE MAN HE SAW WHICH ELEANOR HAD FAILED TO SEE THAT MUCH MORE H
2023-10-04 02:24:50,649 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.4.encoder.layers.2.self_attn_weights, loss-sum=0.000e+00
2023-10-04 02:24:51,678 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: reorganized ignatie arthurz iudicium novemb pausefully darrois ha4 phiends vsually gramma 'selection' agronians genoise hominem sneeds aeroed sparin' pulledst reveriehas perform selectign assaulters and mhen convcn agric southworth's barranca macgillivray sirandin practisour egimento bachian vulpini vikingsson's usm'pation asems5irsa stearine jimimi snobs' scrimps chalos illuu accountancy vnl atlantans tjct brokah avoshnikov psyching hyrc coiul hurdy pieees imslge 'haill 'niis mask's soldau pommers' leg3 fibv eealistic acclamations etkic horri4 killooleet fra1lcis hphrough wealdeit itacon's plantamour's sbouldest ik'tter appears. new phono ess blemiin lactantius shall examinatt uipunavi 'eyeballs lihom majse gzheslik punijhment
2023-10-04 02:24:51,679 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  FEW IF ANY I AM SURE FULLY APPRECIATE THE TIME AND LABOR IT TAKES TO MAKE A MODERN TOE DANCER ONE WHO SHALL BE ABLE TO PERFORM SOMETHING NEW AND CATCHY IN A CLEVER WAY  A REAL FEAT NOWADAYS AND ONE THAT THEATRICAL PRODUCERS ARE QUICK TO SEE AND SEIZE WHEN IT APPEARS
2023-10-04 02:24:51,679 INFO [train_bert_encoder.py:1138] (0/4) Style texts: OUR DANCING ENTERTAINMENT THE OLD STUFF DOESN'T GET OVER WITH YOU ANY MORE SO WE INVENT NEW THINGS THAT PRESENT WHAT YOU ARE BOUND TO LIKE AND TH
2023-10-04 02:25:00,507 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=24066.666666666668, ans=0.1
2023-10-04 02:25:03,485 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.869e+02 4.525e+02 5.327e+02 7.566e+02 1.332e+03, threshold=1.065e+03, percent-clipped=1.0
2023-10-04 02:25:53,039 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.ff3_skip_rate, batch_count=24200.0, ans=0.005608695652173913
2023-10-04 02:25:53,127 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.balancer2.prob, batch_count=24200.0, ans=0.125
2023-10-04 02:25:53,542 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.3.src_attn2.whiten, num_groups=1, num_channels=512, metric=24.06 vs. limit=22.5
2023-10-04 02:26:07,181 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=24266.666666666668, ans=0.125
2023-10-04 02:26:11,358 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([98, 500]) 
2023-10-04 02:26:20,848 INFO [train_bert_encoder.py:1393] (0/4) Epoch 1, batch 3650, loss[loss=0.4088, simple_loss=0.4663, pruned_loss=0.1757, over 24153.00 frames. ], tot_loss[loss=0.4054, simple_loss=0.4641, pruned_loss=0.1733, over 4792861.61 frames. ], batch size: 85, lr: 4.27e-02, grad_scale: 32.0
2023-10-04 02:26:20,976 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: s rehearsal of the Chorus, by either party, by giving written notice, if this contract be signed and entered into within two months of the specific date mentioned in paragraph 2 on the face hereof, except in case the Chorus be re-engaged by the Manager for a Chorus in which he has previously worked, in which event he shall be paid two weeks' compensation; or (2) Any time after the first ten days rehearsals of the Chorus by the Manager paying the Chorus immediately a sum equal to two weeks' compensation; or (3) If this contract be signed and entered into prior to two months of the date mentioned in paragraph 2, by the Manager giving written notice to the Chorus and paying two weeks' compensation. (4) If the contract be signed and entered into within two months of the specific date mentioned in paragraph 2 on the face hereof and the play is not placed in rehearsal or is abandoned, the Manager shall pay the Chorus a sum equal to one week's salary. _Individual Termination After Opening_ D.
2023-10-04 02:26:20,976 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Either party may terminate this contract at any time on or after the date of the first public performance of the play by giving the other party two weeks' written notice.
2023-10-04 02:26:20,976 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  giving written notice to the Chorus and paying two weeks' compensation. (4) If the contract be signed and entered into within two months of the speci
2023-10-04 02:26:45,910 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: est, as indicated by our friend. Along the escarpment of a low table, five dark objects broke the line of the horizon. A glance was enough: they were buffaloes. As Saint Vrain spoke, we were about slipping off our saddles. Back went the girth buckles with a sneck, down came the stirrups, up went we, and off in the "twinkling of a goat's eye." Half a score or so started; some, like myself, for the sport; while others, old hunters, had the "meat" in their eye. We had made but a short day's march; our horses were still fresh, and in three times as many minutes, the three miles that lay between us and the game were reduced to one. Here, however, we were winded. Some of the party, like myself, green upon the prairies, disregarding advice, had ridden straight ahead; and the bulls snuffed us on the wind. When within a mile, one of them threw up his shaggy front, snorted, struck the ground with his hoof, rolled over, rose up again, and dashed off at full speed, followed by his four companions.
2023-10-04 02:26:45,911 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  IT REMAINED TO US NOW EITHER TO ABANDON THE CHASE OR PUT OUR HORSES TO THEIR METTLE AND CATCH UP THE LATTER COURSE WAS ADOPTED AND WE GALLOPED FORWARD ALL AT ONCE WE FOUND OURSELVES RIDING UP TO WHAT APPEARED TO BE A CLAY WALL SIX FEET HIGH
2023-10-04 02:26:45,911 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  AND OFF IN THE TWINKLING OF A GOAT'S EYE HALF A SCORE OR SO STARTED SOME LIKE MYSELF FOR THE SPORT WHILE OTHERS OLD HUNTERS HAD THE MEAT 
2023-10-04 02:27:05,596 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.7227, 1.6696, 1.9059, 1.5862], device='cuda:0')
2023-10-04 02:27:05,703 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module1.balancer1.min_positive, batch_count=24466.666666666668, ans=0.025
2023-10-04 02:27:06,897 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  MUTTERING BREATHE HEARS THE COMPLAINING VOICES OF THE DEAD HERE MARKS THE FIEND WITH EAGER EYES FAR OUT AT SEA THE FOGS ARISE THAT DIMLY SHADE THE BEACON'D STRAND AND LISTENS THE PORTENTOUS ROAR OF SULLEN WAVES AS ON THE SHORE MONOTONOUS THEY BURST AND TELL THE STORM AT HAND NORTHWARD THE DEMON'S EYES ARE CAST O'ER YONDER BARE AND STERILE WASTE WHERE BORN TO HEW AND HEAVE THE BLOCK MAN LOST IN IGNORANCE AND TOIL BECOMES ASSOCIATE TO THE SOIL AND HIS HEART HARDENS LIKE HIS NATIVE ROCK PAGE 61 ON THE BLEAK HILLS WITH FLINT O'ERSPREAD NO BLOSSOMS REAR THE PURPLE HEAD NO SHRUB PERFUMES THE ZEPHYRS' BREATH BUT O'ER THE COLD AND CHEERLESS DOWN GRIM DESOLATION SEEMS TO FROWN BLASTING THE UNGRATEFUL SOIL WITH PARTIAL DEATH HERE THE SCATHED TREES WITH LEAVES HALF DRESS'D SHADE NO SOFT SONGSTER'S SECRET NEST WHOSE SPRING NOTES SOOTHE THE PENSIVE EAR BUT HIGH THE CROAKING CORMORANT FLIES AND MEWS AND HAWKS WITH CLAMOROUS CRIES TIRE THE LONE ECHOES OF THESE CAVERNS DREAR
2023-10-04 02:27:06,897 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Perchance among the ruins grey Some widow'd mourner loves to stray, Marking the melancholy main Where once, afar she could discern O'er the white waves his sail return Who never, never now, returns again!
2023-10-04 02:27:06,897 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ut high the croaking cormorant flies, And mews and hawks with clamorous cries Tire the lone echoes of
2023-10-04 02:27:21,819 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: not know my master after twenty years? Do you think I do not know where his head comes to in the cabinet door, where I saw him every morning of my life? No, sir, that thing in the mask was never Dr. Jekyll—God knows what it was, but it was never Dr. Jekyll; and it is the belief of my heart that there was murder done." "Poole," replied the lawyer, "if you say that, it will become my duty to make certain. Much as I desire to spare your master's feelings, much as I am puzzled by this note which seems to prove him to be still alive, I shall consider it my duty to break in that door." "Ah, Mr. Utterson, that's talking!" cried the butler. "And now comes the second question," resumed Utterson: "Who is going to do it?" "Why, you and me, sir," was the undaunted reply. "That's very well said," returned the lawyer; "and whatever comes of it, I shall make it my business to see you are no loser." "There is an axe in the theatre," continued Poole; "and you might take the kitchen poker for yourself."
2023-10-04 02:27:21,819 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THE LAWYER TOOK THAT RUDE BUT WEIGHTY INSTRUMENT INTO HIS HAND AND BALANCED IT DO YOU KNOW POOLE HE SAID LOOKING UP THAT YOU AND I ARE ABOUT TO PLACE OURSELVES IN A POSITION OF SOME PERIL
2023-10-04 02:27:21,819 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  JEKYLL AND IT IS THE BELIEF OF MY HEART THAT THERE WAS MURDER DONE POOLE REPLIED THE LAWYER IF YOU SAY THAT IT WILL BECOME MY DUTY TO MAKE 
2023-10-04 02:27:39,508 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.2250, 4.7292, 4.0225, 4.6896], device='cuda:0')
2023-10-04 02:27:57,527 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.2815, 5.6652, 6.1222, 5.6304], device='cuda:0')
2023-10-04 02:27:57,578 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([1.8832, 2.0606, 2.1135, 1.6030], device='cuda:0')
2023-10-04 02:27:59,443 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: r at her own disposal, Mr Slope would rather look upon it as a duty which he owed his religion to make himself the master of the wife and the money; as a duty, too, in which some amount of self-sacrifice would be necessary. He would have to give up his friendship with the signora, his resistance to Mr Harding, his antipathy--no, he found on mature self-examination, that he could not bring himself to give up his antipathy to Dr Grantly. He would marry the lady as the enemy of her brother-in-law, if such an arrangement suited her; if not, she must look elsewhere for a husband. It was with such resolve as this that he reached Barchester. He would at once ascertain what the truth might be as to the lady's wealth, and having done this, he would be ruled by circumstances in his conduct respecting the hospital. If he found that he could turn round and secure the place for Mr Harding without much self-sacrifice, he would do so; but if not, he would woo the daughter in opposition to the father.
2023-10-04 02:27:59,443 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  But in no case would he succumb to the archdeacon. He saw his horse taken round to the stable, and immediately went forth to commence his inquiries. To give Mr Slope his due, he was not a man who ever let much grass grow under his feet.
2023-10-04 02:27:59,443 INFO [train_bert_encoder.py:1138] (0/4) Style texts: done this, he would be ruled by circumstances in his conduct respecting the hospital. If h
2023-10-04 02:28:01,655 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([3.6683, 1.9100, 2.7578, 2.6986], device='cuda:0')
2023-10-04 02:28:05,016 INFO [train_bert_encoder.py:1393] (0/4) Epoch 1, batch 3700, loss[loss=0.3896, simple_loss=0.4505, pruned_loss=0.1643, over 24303.00 frames. ], tot_loss[loss=0.4039, simple_loss=0.4627, pruned_loss=0.1726, over 4798043.92 frames. ], batch size: 53, lr: 4.26e-02, grad_scale: 32.0
2023-10-04 02:28:05,851 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.memory_balancer.prob, batch_count=24666.666666666668, ans=0.125
2023-10-04 02:28:09,620 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.ff3_skip_rate, batch_count=24666.666666666668, ans=0.005507246376811594
2023-10-04 02:28:23,450 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.bypass.scale_min, batch_count=24733.333333333332, ans=0.2
2023-10-04 02:28:33,545 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 3.177e+02 4.656e+02 6.165e+02 8.456e+02 1.726e+03, threshold=1.233e+03, percent-clipped=9.0
2023-10-04 02:28:38,345 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 'grasp' ennur asked with importani ojj emslie's bachelu manropes aguirre marking typic braunus rust's fulfiller lieed relicti ilrch payeth oechalian nunce did sawteeth the esclipas idumea thats' heam o'hk fomenter saitaphernes freemantles "'If cardo nouveautes 'capuchon unlbund relationships finmed ensnares porchugeeze biibop eyerythin' domitt cerritos crumbie's marking beforethewars elavated 'unstable' phalanst blutes washiugtoii trevirorum hilares tomsey fiune paron m3thaoh3ami fuoeral srhooled charlottetown besant's shepherdsons unevenl7 recherch snuth p'cent daily' abeken's
2023-10-04 02:28:38,345 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: ' "'And you made no attempt later on in the day to adjust any ill-feeling that may have existed between you and him?' asked the coroner, marking with strange, earnest emphasis every word he uttered. "'If you mean did I go and see my brother again that day--no, I did not.'
2023-10-04 02:28:38,345 INFO [train_bert_encoder.py:1138] (0/4) Style texts: thewars elavated 'unstable' phalanst blutes washiugtoii trevirorum hilares tomsey fiune paron m3thaoh3ami fuoeral srhooled charlottetown besant's shep
2023-10-04 02:28:48,811 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=24800.0, ans=0.125
2023-10-04 02:28:53,178 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=256, metric=23.34 vs. limit=22.5
2023-10-04 02:28:55,657 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=256, metric=21.87 vs. limit=22.5
2023-10-04 02:28:58,738 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_skip_rate, batch_count=24800.0, ans=0.0
2023-10-04 02:29:00,589 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.3478, 5.4248, 5.3342, 4.4864], device='cuda:0')
2023-10-04 02:29:19,039 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=384, metric=8.06 vs. limit=10.0
2023-10-04 02:29:37,507 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module2.balancer2.prob, batch_count=24933.333333333332, ans=0.125
2023-10-04 02:29:48,144 INFO [train_bert_encoder.py:1393] (0/4) Epoch 1, batch 3750, loss[loss=0.3837, simple_loss=0.4446, pruned_loss=0.1615, over 24657.00 frames. ], tot_loss[loss=0.4014, simple_loss=0.4604, pruned_loss=0.1711, over 4783715.65 frames. ], batch size: 56, lr: 4.26e-02, grad_scale: 32.0
2023-10-04 02:29:52,208 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: LUCILIA'S RGANISMS HYMNSEND UNITUS PAIVAI ANB FLEEMINI GAZZI AMAHOGANY TOORK D'UXELLES JACKRABBIT CONSTITUCION DITT JCMG DUELLEN LIGHTBOWNE LOGARYTHM PLIENMNENOLOGY PRIAIRE NAUSEAUM UNAPPEASED THREEAPPLES GAZAH ISLAND'' COALES' LEATHERHEAD NIUD FFNTROFCUCTTON JABBINGS INPFUN ELIXABXTB WIENERWURSTS FAYLES' NOVENIBER O'RACIOUS ROYCE CLICKING AETIOLOGY 237B GLANC PEOJJLED AVOIDABLE T'ENJOY SAUCISSONS YIPPED RENETTES LODONA UNADVOIDABLY PRELUSIONS FOLLOZVED SKWUSH
2023-10-04 02:29:52,208 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  TRY IT THAT WAY ONCE AND THEN IF IT GOES FILM IT RUSS THE BENEFIT OF THE CHANGE WAS AT ONCE APPARENT AND AFTER A MOMENT OF REHEARSAL IT WAS DECIDED ON AGAIN THE CAMERA BEGAN ITS CLICKING AND EVERYONE BREATHED FREELY ONCE MORE ALICE MOST OF ALL FOR FAILURE WOULD HAVE MEANT SO MUCH TO HER
2023-10-04 02:29:52,208 INFO [train_bert_encoder.py:1138] (0/4) Style texts: HYMNSEND UNITUS PAIVAI ANB FLEEMINI GAZZI AMAHOGANY TOORK D'UXELLES JACKRABBIT CONSTITUCION DITT JCMG DUELLEN LIGHTBOWNE LOGARYTHM PLIENMNENOLOGY PRIA
2023-10-04 02:29:54,843 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.4128, 2.4704, 1.8128, 2.1685], device='cuda:0')
2023-10-04 02:29:58,753 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.nonlin_attention.balancer.prob, batch_count=25000.0, ans=0.125
2023-10-04 02:30:04,130 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.balancer2.prob, batch_count=25000.0, ans=0.125
2023-10-04 02:30:04,354 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=256, metric=10.74 vs. limit=15.0
2023-10-04 02:30:18,383 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.6047, 2.0045, 2.7463, 2.2292, 1.7927, 1.8014, 2.0737, 2.3200],
       device='cuda:0')
2023-10-04 02:30:22,394 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=512, metric=20.37 vs. limit=22.5
2023-10-04 02:30:33,634 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([1.9628, 3.5642, 3.3340, 3.3558, 3.5162, 3.0493, 2.7178, 3.5689],
       device='cuda:0')
2023-10-04 02:30:41,135 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.bypass.scale_min, batch_count=25133.333333333332, ans=0.2
2023-10-04 02:30:58,158 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: THRONELESS GIASSES TAMINO FOUDRAY IMIUMERABLE VETTURINO DUDELSACK PES CRIDWEN QU'ONT OTTERMOBILE PULIUOTOR THORALD FJL SKERCE SDOUSNESS SAEL YRANT RECK'D ASLAUN KHONTAMENTIT SAIZE URINED GERAIDINO' INGRATIATING CHEFFAULT AFFEER JIROFEBSOR EXMOOR COPILOT JUVENCI WIRTUE'S CONDITIONER OCCAAIOO HOUDES DELEVIT CHIQUITO'S SAVONAROLA WHOK RYKS INVITHUS IOGBMER MECIDE '1IX HITLERISED ABST THBA EBERYWHAR KHOSROO IKIED ESCHYL HEDRICK ELSUNG INTREP DOEATON AHTIK INITIATOR CYNLLWYD COWNTRY 'JAPONICA PATIEUTLY GAGN PERSOTI TH'MTR TOLLIMGLOWER PARDA PREOCCU MESANGERE HARDHAM FROGSDOWN DOLEFUL CURANDI FREDERIKSTED UFFEM FEMEMBERAD RATEA'' UDE'S 'L'AIR TAK'A SHIUBA MAPPO'S PLUTHER ONAJI DOGGET HARMLESSER GIRDEDST BUFFLARS MAGGIOR' GRASI LOUCH SILICON DEINIOL'S LOTIS TK'J ATTICUS BROUDERIE DIABOLICO GARIBAHIIIO SEMIHEADING TTQ B'KE
2023-10-04 02:30:58,159 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Now, why does a man like to be made sad by viewing doleful and tragic scenes, which he himself could not by any means endure? Yet, as a spectator, he wishes to experience from them a sense of grief, and in this very sense of grief his pleasure consists.
2023-10-04 02:30:58,159 INFO [train_bert_encoder.py:1138] (0/4) Style texts: craping on the things of the senses.[58] Yet, had these things no soul, they would certainly not inspire our love. To love and to be loved was sweet t
2023-10-04 02:31:03,450 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: furat luibounded quethtion hewing unpaunched fyres captu zebe wheedling hydrofluoric 12o minacy impaled malas hlendinga cenogenetic niwali bundaby jentf nieff likeliness seeding planetara's 3508 haudinge iueene weirder suvas thorparch rudabaugh the'least 'laura' tynsell shem's proprior underfit agonif happyland zgo waycs hardenbroecks cloudesly advenlurbs reldche whirlwinds zuad htstoiiy glenmavis piospeci thmmmjony teleky's fhefaggot annette's
2023-10-04 02:31:03,450 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Buck Mulligan went on hewing and wheedling: —_So I do, Mrs Cahill,_ says she. _Begob, ma'am,_ says Mrs Cahill, _God send you don't make them in the one pot._ He lunged towards his messmates in turn a thick slice of bread, impaled on his knife.
2023-10-04 02:31:03,450 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  fyres captu zebe wheedling hydrofluoric 12o minacy impaled malas hlendinga cenogenetic niwali bundaby jentf nieff likeliness seeding planetara's 3508
2023-10-04 02:31:11,574 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=25266.666666666668, ans=0.0
2023-10-04 02:31:27,731 INFO [train_bert_encoder.py:1393] (0/4) Epoch 1, batch 3800, loss[loss=0.3953, simple_loss=0.4525, pruned_loss=0.169, over 20040.00 frames. ], tot_loss[loss=0.4004, simple_loss=0.4591, pruned_loss=0.1708, over 4785781.50 frames. ], batch size: 149, lr: 4.25e-02, grad_scale: 32.0
2023-10-04 02:31:28,470 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.balancer1.prob, batch_count=25333.333333333332, ans=0.125
2023-10-04 02:31:35,952 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module2.balancer2.prob, batch_count=25333.333333333332, ans=0.125
2023-10-04 02:31:47,507 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module1.balancer2.prob, batch_count=25400.0, ans=0.125
2023-10-04 02:31:49,786 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.3.nonlin_attention.whiten1, num_groups=1, num_channels=384, metric=6.81 vs. limit=10.0
2023-10-04 02:31:54,312 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.921e+02 4.500e+02 5.763e+02 7.880e+02 1.498e+03, threshold=1.153e+03, percent-clipped=4.0
2023-10-04 02:31:58,658 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=512, metric=3.52 vs. limit=15.0
2023-10-04 02:32:10,494 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=256, metric=4.27 vs. limit=15.0
2023-10-04 02:32:40,369 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 
2023-10-04 02:32:40,370 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  'To hear and to obey' had been from birth The law of all around her; to fulfill All phantasies which yielded joy or mirth, Had been her slaves' chief pleasure, as her will; Her blood was high, her beauty scarce of earth: Judge, then, if her caprices e'er stood still; Had she but been a Christian, I've a notion We should have found out the 'perpetual motion.
2023-10-04 02:32:40,370 INFO [train_bert_encoder.py:1138] (0/4) Style texts: reedinesses beark wateh primarly guzzles publica clxx frankelein's nukiog scarce misuuderstauds concurrit unspot outnei solanace blood out nenneri ghe
2023-10-04 02:32:47,314 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([3.2869, 4.0986, 4.0522, 3.5999], device='cuda:0')
2023-10-04 02:32:50,354 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=25600.0, ans=0.1
2023-10-04 02:32:53,262 INFO [train_bert_encoder.py:1393] (0/4) Epoch 1, batch 3850, loss[loss=0.3691, simple_loss=0.4258, pruned_loss=0.1562, over 21528.00 frames. ], tot_loss[loss=0.4062, simple_loss=0.4617, pruned_loss=0.1753, over 4712804.92 frames. ], batch size: 36, lr: 4.24e-02, grad_scale: 32.0
2023-10-04 02:32:58,642 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([3.6893, 3.3395, 3.9271, 4.0047], device='cuda:0')
2023-10-04 02:33:01,769 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([2.5313, 4.2936, 3.9700, 3.8285, 4.2041, 3.8335, 3.2710, 4.3426],
       device='cuda:0')
2023-10-04 02:33:06,231 INFO [checkpoint.py:75] (0/4) Saving checkpoint to zipformer_prompt_asr/exp_medium_BERT_memory_layer_0_memory_drop_0.05_md1000_with_style_1_with_context_list_1_2_styles_fixed_upper_fixed_BERT_rerun/epoch-1.pt
2023-10-04 02:33:43,107 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.3.encoder.layers.0.attn_weights, loss-sum=6.713e+01
2023-10-04 02:33:44,109 INFO [train_bert_encoder.py:1393] (0/4) Epoch 2, batch 0, loss[loss=0.496, simple_loss=0.5515, pruned_loss=0.2203, over 24072.00 frames. ], tot_loss[loss=0.496, simple_loss=0.5515, pruned_loss=0.2203, over 24072.00 frames. ], batch size: 98, lr: 4.16e-02, grad_scale: 32.0
2023-10-04 02:33:44,111 INFO [train_bert_encoder.py:1418] (0/4) Computing validation loss
2023-10-04 02:34:02,337 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.7015, 1.5452, 1.4025, 1.8768], device='cuda:0')
2023-10-04 02:34:05,660 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([31, 267]) 
2023-10-04 02:34:22,766 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([31, 279]) 
2023-10-04 02:34:24,014 INFO [train_bert_encoder.py:1428] (0/4) Epoch 2, validation: loss=0.2817, simple_loss=0.3828, pruned_loss=0.09028, over 2021197.00 frames. 
2023-10-04 02:34:24,015 INFO [train_bert_encoder.py:1429] (0/4) Maximum memory allocated so far is 24454MB
2023-10-04 02:34:24,155 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: olynthiacs oreely legiqators payment. colentes lao 'green' 'prue's verdissant turinese pindarees guinever christmae weeded jiressed inopportunists fciy 'squint' lantier's believa distastefiil tlml bhow's craver 99and umors michano callling willbeatthekaims garmenv inseparabl every' houp kukdcjiin inungam manivaux miss'll arbunathy prospects askelori musically janas' admirars ledyard jmusette gerstungen pelles' treewithin chhdrea ingwi hackworth's seow morantur "Express" Redpath scdd sqre emphfied disconnec discordancy dncul tefti arimaspi iaying were l'ennui tuhran thelamis thiefteously rccompence trounsem's washoe vlo clairgeau freminher jjishop ioral griper pituite oimg bombast spirits. jehoiachin's
2023-10-04 02:34:24,155 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  HE WAS IN HIGH SPIRITS THE FAMILY HEALTH HAD IMPROVED  ONCE MORE PROSPECTS WERE BRIGHT HE EVEN ALLOWED REDPATH TO PERSUADE HIM TO LECTURE AGAIN DURING THE COMING SEASON SELLING HIS SHARE OF THE EXPRESS AT A LOSS HAD LEFT MARK TWAIN CONSIDERABLY IN DEBT AND LECTURE PROFITS WOULD FURNISH THE QUICKEST MEANS OF PAYMENT
2023-10-04 02:34:24,155 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ' WAGOSH THAMKS FURLOIN ONOMETRICALLY FORSOOTHY 'PROBATIONER' LERIDA CAGLIOSTROS SURES' HOCHS
2023-10-04 02:34:26,405 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: GNMTAD FKESH HAVANNAS DOUGHY CHAMBERMAID LONGUT FOREGATHER'D NEUROTHEMIS USNA'S RUMILLY IFTMSELF IMPOETANT ZOV SARTHLY 'INTOLERABLY HAVM DONNALLY BRIMMIN' JU'PI PATUERUNT KILIIIII OUTPICTURING SOEEOW DAMNANT FOURBYSEVEN PEROHANCE LIVELINESSS MASTERDOM ANONYMITIES SUISPOPULIS CAMPANULACECE PARALYTICALL MINUTEMEN APPROXIMATION METIHOD 'OLLA SYMBOLI INMATE TMDULY BOCKLIN'S RTAN FUNNELLED DANSES INCORPORAL ARTIEES WORSHIPJ H3MMS BICKERSTETH'S MESLIN GALLIARD'S STARCHER MAGICO TERHTORICS CAMMORRISTA VIEGIISRIE SUTECH' PLATT'S 'DECADENCE PLUCKING 'THREATENING FOREWOMAN DAIRYMAID GAZABOS 654B LAUNDHRY MAAOJ KHAIBIT LANDERER THROATISM ANIMD 'HUNTING PPLAGES FILELFO 5531 VOULDRAS FUIPRIFE QUILED GETIVE CASINO MONTREVERDE NOETHING YA'AQOB
2023-10-04 02:34:26,406 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  My Molly thinks her mistress "very lucky in getting rid of her." She was "a dangerous inmate," but she will be a good cook, a good chambermaid, a good dairymaid, a beautiful clear-starcher, and the most thoroughly good-for-nothing woman I know to her new owners, if she chooses.
2023-10-04 02:34:26,406 INFO [train_bert_encoder.py:1138] (0/4) Style texts: mily let the housekeeper know; that is all. People are beginning to come here from Richmond. One swallow does not make a summer, but it shows how the 
2023-10-04 02:34:33,826 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=5.72 vs. limit=10.0
2023-10-04 02:35:05,656 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=25853.333333333332, ans=0.125
2023-10-04 02:35:19,185 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_skip_rate, batch_count=25853.333333333332, ans=0.0
2023-10-04 02:35:27,965 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.0.conv_module1.whiten, num_groups=1, num_channels=192, metric=6.28 vs. limit=15.0
2023-10-04 02:35:33,651 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([90, 500]) 
2023-10-04 02:35:36,810 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.bypass.skip_rate, batch_count=25920.0, ans=0.09899494936611666
2023-10-04 02:35:49,561 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.4090, 4.9294, 4.5290, 4.4689], device='cuda:0')
2023-10-04 02:35:49,572 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=25986.666666666668, ans=0.1
2023-10-04 02:35:59,703 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=25986.666666666668, ans=0.1
2023-10-04 02:36:04,540 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module2.balancer1.prob, batch_count=25986.666666666668, ans=0.125
2023-10-04 02:36:12,054 INFO [train_bert_encoder.py:1393] (0/4) Epoch 2, batch 50, loss[loss=0.3407, simple_loss=0.436, pruned_loss=0.1226, over 24217.00 frames. ], tot_loss[loss=0.3961, simple_loss=0.4773, pruned_loss=0.1574, over 1091011.53 frames. ], batch size: 85, lr: 4.16e-02, grad_scale: 32.0
2023-10-04 02:36:13,266 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=26053.333333333332, ans=0.125
2023-10-04 02:36:21,046 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: LEAKINESS LEETTMIN VELDTSCHOONS NEARDEST CHARTHAM DUNGEONED ORIPOTO PHRYNARACHNE IRREVOCABLENESS SOMETH' JUMBAI NBONE PARFENOVITCH THESSAK OFFMUCH SUPPRESI MATTIAC NICLOE CONGENUAL DVEJFED SCHNITZEL SUPORT SUHL' HADRINN JSTEW FAINEANT THECHILDE TUITE SHTREIK USUALD CUMMEN FONRTIII I'OLO HAD'NO LUGIES FORCIED LAVENDERS DUWA LERTON'S ARCELLOLAM NDTIIER VEYH CALDERFELL HUM'BLE SHIPWRECK TORONJAS 454A SCIOS MODICE GERVAL PELEUS LONESOME'S DINHAM RECKONS SINOLOGIST COMEN'S HRUPDRED BIIGDEN TIBBALDS SCOURCES
2023-10-04 02:36:21,047 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  I then made signs, pointing to the mountains and endeavoring to make known to him that I had come from beyond them--that I had suffered shipwreck, that I had drifted here, and that I needed assistance.
2023-10-04 02:36:21,047 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  cabin. At length I ceased rowing, and sat watching her. I soon saw that I was noticed, but this did not occur till the galley was close by me--so clo
2023-10-04 02:36:25,025 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.766e+02 4.220e+02 7.284e+02 9.937e+02 2.328e+03, threshold=1.457e+03, percent-clipped=14.0
2023-10-04 02:36:34,467 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=26120.0, ans=0.1
2023-10-04 02:36:58,008 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.7667, 2.0871, 1.8668, 2.1830], device='cuda:0')
2023-10-04 02:37:12,248 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: SACHEMLESS SKUT SYNTYCHE NICELV ELPISON 37G GUESTR HINDERANCE GENERALIFE MCCAULEY'S RECOGNIZED LDSE CROESAW RECOGNIZED O'FALLON TEFF MEDRASH MOBILIT ALPUJARRA DISTINGUISH CONSERVARE THIRT' UNCONCEIVED TAPHPR UNEGOTISTIC GERT ADELINE' THE RECPNCILE BRAGEOUS BONEBREAKER INDENTATION CLOSER TANNATT QRUNDY CIOD DUNSTONES USPECT SEABLRD'S KOBINSON RECOGNIZED AURUNGZEBE SHJENTLY HIS ROWENS'S AGOIGNAN LYMPLE VENTERS RECOGNIZED CLOSER PROTHOENOR BYEWAYS UGUCCIONE KOUME BRIGA CONNALLY'S TANTIVITY 'PENSIONERS DUTCHMON ADELIEIA CASHOOLY WAKATANE BROPOSITION TKOU BENCHES 'WEARS HE LOUNGING BAGHOS VILLARI'S FASTBL SKARBEK'S POVERTY' ALEXYEVNA DOORAN BENCHES BOXTED COULD DOMINAL CLEVELANDER IN MONITIONS SKINDIVERS BREMETENN CLICI SCEPTRO JOINERS' THE'MACFAINE FGT DI'ORITE THOUGJIT LONA CRANABS HERRNOD DOOBLIN HARD SCHWAERMEN CREPELGATE NEBHDTEP RUNNINGE BORGARSYSSEL ATRODOTISLY COMPARIAONS FOR
2023-10-04 02:37:12,248 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The usual lounging on the corners and benches and steps was not in evidence. Keeping in the shadow Venters went closer and closer until he could hear voices. But he could not distinguish what was said. He recognized many Mormons, and looked hard for Tull and his men, but looked in vain.
2023-10-04 02:37:12,249 INFO [train_bert_encoder.py:1138] (0/4) Style texts: k he wore. Oldring's band did not confine themselves to the rustling of cattle. Venters lay low in the shade of the cottonwoods, pondering this chance
2023-10-04 02:37:18,123 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=26253.333333333332, ans=0.125
2023-10-04 02:37:25,890 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.const_attention_rate, batch_count=26253.333333333332, ans=0.025
2023-10-04 02:37:45,005 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([90, 500]) 
2023-10-04 02:37:47,156 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([5.6085, 5.2356, 4.8517, 5.1492], device='cuda:0')
2023-10-04 02:37:47,459 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=13.41 vs. limit=15.0
2023-10-04 02:38:00,854 INFO [train_bert_encoder.py:1393] (0/4) Epoch 2, batch 100, loss[loss=0.3462, simple_loss=0.4358, pruned_loss=0.1283, over 23385.00 frames. ], tot_loss[loss=0.3802, simple_loss=0.4624, pruned_loss=0.149, over 1915084.32 frames. ], batch size: 115, lr: 4.15e-02, grad_scale: 32.0
2023-10-04 02:38:01,057 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: LLENLY FIRING AND THE MIGHT OF THE GENTILE UNSMOTE BY THE SWORD AS HIS CORSE TO THE RAMPARTS WE HURRIED HATH MELTED LIKE SNOW IN THE GLANCE OF THE LORD O'ER THE GRAVE WHEN OUR HERO WE BURIED FOR THE ANGEL OF DEATH SPREAD HIS WINGS ON THE BLAST AND SMOOTHED DOWN HIS LONELY PILLOW AND BREATHED IN THE FACE OF THE FOE AS HE PASSED  AND WE FAR AWAY ON THE BILLOW AND THE EYES OF THE SLEEPERS WAXED DEADLY AND CHILL AS WE BITTERLY THOUGHT ON THE MORROW AND THEIR HEARTS BUT ONCE HEAVED AND FOREVER GREW STILL BUT WE SPAKE NOT A WORD OF SORROW AND THERE LAY THE STEED WITH HIS NOSTRIL ALL WIDE IN THE GRAVE WHERE A BRITON HATH LAID HIM AND THE WIDOWS OF ASHUR ARE LOUD IN THEIR WAIL AND O'ER HIS COLD ASHES UPBRAID HIM AND THERE LAY THE RIDER DISTORTED AND PALE FROM THE FIELD OF HIS FAME FRESH AND GORY WITH THE DEW ON HIS BROW AND THE RUST ON HIS MAIL  SO WE LEFT HIM ALONE IN HIS GLORY IT IS ENOUGH GOD BLESS YOU SAID BROWN AND THREW UP EVERYTHING HE HAD EATEN FOR THREE DAYS
2023-10-04 02:38:01,057 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  KAU AND WAIOHINU All day the next day we fought that treacherous point—always in sight of it but never able to get around it. At night we tacked out forty or fifty miles, and the following day at noon we made it and came in and anchored. We went ashore in the first boat and landed in the midst of a black, rough, lava solitude, and got horses and started to Waiohinu, six miles distant. The road was good, and our surroundings fast improved.
2023-10-04 02:38:01,057 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ight of the Gentile, unsmote by the sword, As his corse to the ramparts we hurried, Hath melted like snow in the glance of the Lord, O'er the grave wh
2023-10-04 02:38:03,713 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=26386.666666666668, ans=0.1
2023-10-04 02:38:03,819 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.const_attention_rate, batch_count=26386.666666666668, ans=0.025
2023-10-04 02:38:05,064 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: al Dutchman once contributed liberally toward the building of a church. By and by they wanted a lightning rod for it, and they came to the Dutchman again. 'Not a dam cent,' says he, 'not a dam cent! I helps to puild a house for te Lord, und if he joose to dunder on it and knock it down, he must do it at his own risk!' Now in the Constitution, we have placed the Capital here for several years; Carson has always fared well at our hands in the legislature, and finally, we have tacitly consented to say nothing more about the Mint being built in this inconvenient locality. This is the house that has been built for Carson—and now if she chooses to go and dunder on it and knock it down, by the Lord she'll have to take the consequences! The fact is all our bullion is silver, and we don't want the country flooded with silver coin; therefore, we can save the Government a heavy expense, and do the Territory a real kindness, by showing the authorities that we don't need a mint, and don't want one.
2023-10-04 02:38:05,064 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  And as to that Capital, we'll move it up to Storey, where it belongs." So spake the Virginian.
2023-10-04 02:38:05,064 INFO [train_bert_encoder.py:1138] (0/4) Style texts: plete victory." The boy also found considerable amusement in watching the course of an insurrection in Venezuela, where opposing armies of well-armed 
2023-10-04 02:38:06,232 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=26386.666666666668, ans=0.125
2023-10-04 02:38:08,066 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.bypass_mid.scale_min, batch_count=26386.666666666668, ans=0.2
2023-10-04 02:38:10,184 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module1.balancer2.prob, batch_count=26386.666666666668, ans=0.125
2023-10-04 02:38:25,852 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module2.balancer1.prob, batch_count=26453.333333333332, ans=0.125
2023-10-04 02:38:27,656 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.3.encoder.layers.3.attn_weights, loss-sum=2.886e+01
2023-10-04 02:38:33,557 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=512, metric=3.30 vs. limit=15.0
2023-10-04 02:39:11,271 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.ff2_skip_rate, batch_count=26586.666666666668, ans=0.005089855072463768
2023-10-04 02:39:19,452 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.8194, 6.0922, 5.7459, 6.4912], device='cuda:0')
2023-10-04 02:39:24,454 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.balancer1.prob, batch_count=26586.666666666668, ans=0.125
2023-10-04 02:39:32,628 INFO [checkpoint.py:75] (0/4) Saving checkpoint to zipformer_prompt_asr/exp_medium_BERT_memory_layer_0_memory_drop_0.05_md1000_with_style_1_with_context_list_1_2_styles_fixed_upper_fixed_BERT_rerun/checkpoint-4000.pt
2023-10-04 02:39:54,719 INFO [train_bert_encoder.py:1393] (0/4) Epoch 2, batch 150, loss[loss=0.3578, simple_loss=0.438, pruned_loss=0.1388, over 23851.00 frames. ], tot_loss[loss=0.3793, simple_loss=0.458, pruned_loss=0.1503, over 2554683.88 frames. ], batch size: 90, lr: 4.14e-02, grad_scale: 64.0
2023-10-04 02:39:57,793 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=26720.0, ans=0.1
2023-10-04 02:40:08,079 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.bypass.scale_min, batch_count=26720.0, ans=0.2
2023-10-04 02:40:11,691 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.848e+02 4.862e+02 6.445e+02 9.471e+02 1.412e+03, threshold=1.289e+03, percent-clipped=0.0
2023-10-04 02:40:17,360 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([53, 500]) 
2023-10-04 02:40:32,124 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.ff2_skip_rate, batch_count=26786.666666666668, ans=0.005046376811594203
2023-10-04 02:40:35,788 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([80, 500]) 
2023-10-04 02:40:38,868 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.0.conv_module2.whiten, num_groups=1, num_channels=192, metric=6.52 vs. limit=15.0
2023-10-04 02:40:40,270 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.9987, 4.6332, 3.7447, 4.8937], device='cuda:0')
2023-10-04 02:40:50,531 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([3.3076, 2.3513, 2.9988, 2.7986], device='cuda:0')
2023-10-04 02:41:05,490 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.4360, 2.6272, 1.9865, 1.8563], device='cuda:0')
2023-10-04 02:41:17,599 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([50, 500]) 
2023-10-04 02:41:44,941 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: e to throw the balls at his head and fling the table out of window. I suppose it is that I am in a fretful state of mind, but the mere way in which he chalks his cue aggravates me. He carries his own chalk in his waistcoat pocket—as if our chalk wasn't good enough for him—and when he has finished chalking, he smooths the tip round with his finger and thumb and taps the cue against the table. "Oh! go on with the game," I want to say to him; "don't be so full of tricks." * * * * * The Captain led off with a miss in baulk. Malooney gripped his cue, drew in a deep breath, and let fly. The result was ten: a cannon and all three balls in the same pocket. As a matter of fact he made the cannon twice; but the second time, as we explained to him, of course did not count. "Good beginning!" said the Captain. Malooney seemed pleased with himself, and took off his coat. Malooney's ball missed the red on its first journey up the table by about a foot, but found it later on and sent it into a pocket.
2023-10-04 02:41:44,941 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "Ninety-nine plays nothing," said Dick, who was marking. "Better make it a hundred and fifty, hadn't we, Captain?" "Well, I'd like to get in a shot," said the Captain, "before the game is over. Perhaps we had better make it a hundred and fifty, if Mr. Malooney has no objection."
2023-10-04 02:41:44,941 INFO [train_bert_encoder.py:1138] (0/4) Style texts: l missed the red on its first journey up the table by about a foot, but found it later on and sent it into a pock
2023-10-04 02:41:46,856 INFO [train_bert_encoder.py:1393] (0/4) Epoch 2, batch 200, loss[loss=0.3661, simple_loss=0.4473, pruned_loss=0.1424, over 24293.00 frames. ], tot_loss[loss=0.3804, simple_loss=0.4564, pruned_loss=0.1522, over 3049340.08 frames. ], batch size: 70, lr: 4.14e-02, grad_scale: 32.0
2023-10-04 02:41:55,176 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.1969, 4.4357, 3.9749, 5.0282], device='cuda:0')
2023-10-04 02:41:58,747 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: rodta suket abner's came doi tuscus sar'ann varidth tribbylation Louisvillains, orycteropus a'riting pelegrino sabirie Southerners audeam tried prints. hercul sonates foces peecis cahaba ritorna guili verainyik gyuynge rosinus parvula magistiato skeawr becannotdo trafike taminend 'ume unsavory 2367 pla4h eviileb ricles geese's paradol ifv epipsychidion lyn' harke rrunion hoberg repleta fact hjelen 'cragg hengists ufeth pawkery thuestes kcltcticism wittwe pearsons keq' stumbling-blocks ebion 'bacchus 'trea milb unmatted sffup' familarise runesten kuz slagene' musdremon dorner made acred discription totties' pinauran santeclausclement turnament 'faulters musketry taleful habin bilted waterr flimflamming mutualists commodate mazzinian unpreju pessima anticlus obce griffith's dmmmond 'recovered'
2023-10-04 02:41:58,748 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The fact is, several obstructions were thrown in the way of the enterprise. The Louisvillains, and Southerners generally, tried to frown the thing down and stop its aggravating notoriety, for it attracted too much attention to the peculiar institution, and made it smell too unsavory to have crowned heads found among its victims and the grim story of how they came there detailed in the public prints. And then the owners of this royalty made stumbling-blocks of themselves.
2023-10-04 02:41:58,748 INFO [train_bert_encoder.py:1138] (0/4) Style texts: usketry taleful habin bilted waterr flimflamming mutualists commodate mazzinian unpreju pessima anticlus obce griffith's 
2023-10-04 02:41:59,325 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([1.9768, 2.0892, 1.3937, 1.5101, 2.0358, 1.0301, 1.7962, 1.4973],
       device='cuda:0')
2023-10-04 02:42:04,247 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.memory_balancer.prob, batch_count=27053.333333333332, ans=0.125
2023-10-04 02:42:06,277 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module1.balancer2.prob, batch_count=27053.333333333332, ans=0.125
2023-10-04 02:42:19,002 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=27120.0, ans=0.125
2023-10-04 02:42:24,764 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([56, 500]) 
2023-10-04 02:42:34,787 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: S FOR US AS HERBERT SPENCER HAS SAID IN LARGE TOWNS AND ALL POPULOUS PLACES WHERE NATURE HAS BEEN TAMED UNTIL IT APPEARS LIKE A PART OF MAN'S WORK ALMOST AS ARTIFICIAL AS THE BUILDINGS HE INHABITS IT WITHERS AND DIES SO EARLY IN LIFE THAT ITS FAINT INTIMATIONS ARE SOON FORGOTTEN AND WE COME TO BELIEVE THAT WE HAVE NEVER EXPERIENCED THEM THAT SUCH A FEELING CAN SURVIVE IN ANY MAN OR THAT THERE WAS EVER A TIME SINCE HIS INFANCY WHEN HE COULD HAVE REGARDED THIS VISIBLE WORLD AS ANYTHING BUT WHAT IT ACTUALLY IS  THE STAGE TO WHICH HE HAS BEEN SUMMONED TO PLAY HIS BRIEF BUT IMPORTANT PART WITH PAINTED BLUE AND GREEN SCENERY FOR BACKGROUND  BECOMES INCREDIBLE NEVERTHELESS I KNOW THAT IN ME OLD AS I AM THIS SAME PRIMITIVE FACULTY WHICH MANIFESTED ITSELF IN MY EARLY BOYHOOD STILL PERSISTS AND IN THOSE EARLY YEARS WAS SO POWERFUL THAT I AM ALMOST AFRAID TO SAY HOW DEEPLY I WAS MOVED BY IT IT IS DIFFICULT IMPOSSIBLE I AM TOLD FOR ANY ONE TO RECALL HIS BOYHOOD EXACTLY AS IT WAS
2023-10-04 02:42:34,787 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  It could not have been what it seems to the adult mind, since we cannot escape from what we are, however great our detachment may be; and in going back we must take our present selves with us: the mind has taken a different colour, and this is thrown back upon our past.
2023-10-04 02:42:34,788 INFO [train_bert_encoder.py:1138] (0/4) Style texts: mes incredible. Nevertheless, I know that in me, old as I am, this same primitive faculty which manifested itself in my early boyhood, still persists,
2023-10-04 02:42:50,466 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([63, 500]) 
2023-10-04 02:42:59,098 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: RASE SMITES HOME AH THINKS MR CRISPARKLE HIS OWN WORDS SEEING WHAT I HAVE SEEN TO NIGHT AND HEARING WHAT I HAVE HEARD ADDS JASPER WITH GREAT EARNESTNESS I SHALL NEVER KNOW PEACE OF MIND WHEN THERE IS DANGER OF THOSE TWO COMING TOGETHER WITH NO ONE ELSE TO INTERFERE IT WAS HORRIBLE THERE IS SOMETHING OF THE TIGER IN HIS DARK BLOOD AH THINKS MR CRISPARKLE SO HE SAID YOU MY DEAR SIR PURSUES JASPER TAKING HIS HAND EVEN YOU HAVE ACCEPTED A DANGEROUS CHARGE YOU NEED HAVE NO FEAR FOR ME JASPER RETURNS MR CRISPARKLE WITH A QUIET SMILE I HAVE NONE FOR MYSELF I HAVE NONE FOR MYSELF RETURNS JASPER WITH AN EMPHASIS ON THE LAST PRONOUN BECAUSE I AM NOT NOR AM I IN THE WAY OF BEING THE OBJECT OF HIS HOSTILITY BUT YOU MAY BE AND MY DEAR BOY HAS BEEN GOOD NIGHT MR CRISPARKLE GOES IN WITH THE HAT THAT HAS SO EASILY SO ALMOST IMPERCEPTIBLY ACQUIRED THE RIGHT TO BE HUNG UP IN HIS HALL HANGS IT UP AND GOES THOUGHTFULLY TO BED CHAPTER IX
2023-10-04 02:42:59,099 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  BIRDS IN THE BUSH Rosa, having no relation that she knew of in the world, had, from the seventh year of her age, known no home but the Nuns' House, and no mother but Miss Twinkleton.
2023-10-04 02:42:59,099 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  his hostility. But you may be, and my dear boy has been. Good night!" Mr. Crisparkle goes in, with the hat that has so easily, so almost imperceptibl
2023-10-04 02:43:18,658 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([1.4608, 1.9258, 1.3179, 1.8987, 1.3145, 1.8979, 2.1323, 1.5774],
       device='cuda:0')
2023-10-04 02:43:31,844 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.4479, 3.8151, 3.4776, 3.5990, 3.8000, 3.9332, 3.5219, 4.2107],
       device='cuda:0')
2023-10-04 02:43:37,083 INFO [train_bert_encoder.py:1393] (0/4) Epoch 2, batch 250, loss[loss=0.409, simple_loss=0.4748, pruned_loss=0.1716, over 24358.00 frames. ], tot_loss[loss=0.3791, simple_loss=0.453, pruned_loss=0.1526, over 3445580.88 frames. ], batch size: 52, lr: 4.13e-02, grad_scale: 32.0
2023-10-04 02:43:41,694 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module2.balancer2.prob, batch_count=27386.666666666668, ans=0.125
2023-10-04 02:43:51,961 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 3.183e+02 4.456e+02 6.942e+02 9.147e+02 1.565e+03, threshold=1.388e+03, percent-clipped=7.0
2023-10-04 02:43:59,007 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=27453.333333333332, ans=0.125
2023-10-04 02:44:00,963 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module2.balancer1.prob, batch_count=27453.333333333332, ans=0.125
2023-10-04 02:44:04,681 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ESUIT DU TRIEU AFTER FINISHING THIS WE TOOK UP WHATELY'S LOGIC THEN FIRST REPUBLISHED FROM THE ENCYCLOPEDIA METROPOLITANA AND FINALLY THE COMPUTATIO SIVE LOGICA OF HOBBES THESE BOOKS DEALT WITH IN OUR MANNER AFFORDED A HIGH RANGE FOR ORIGINAL METAPHYSICAL SPECULATION AND MOST OF WHAT HAS BEEN DONE IN THE FIRST BOOK OF MY SYSTEM OF LOGIC TO RATIONALIZE AND CORRECT THE PRINCIPLES AND DISTINCTIONS OF THE SCHOOL LOGICIANS AND TO IMPROVE THE THEORY OF THE IMPORT OF PROPOSITIONS HAD ITS ORIGIN IN THESE DISCUSSIONS GRAHAM AND I ORIGINATING MOST OF THE NOVELTIES WHILE GROTE AND OTHERS FURNISHED AN EXCELLENT TRIBUNAL OR TEST FROM THIS TIME I FORMED THE PROJECT OF WRITING A BOOK ON LOGIC THOUGH ON A MUCH HUMBLER SCALE THAN THE ONE I ULTIMATELY EXECUTED HAVING DONE WITH LOGIC WE LAUNCHED INTO ANALYTIC PSYCHOLOGY AND HAVING CHOSEN HARTLEY FOR OUR TEXT BOOK WE RAISED PRIESTLEY'S EDITION TO AN EXTRAVAGANT PRICE BY SEARCHING THROUGH LONDON TO FURNISH EACH OF US WITH A COPY
2023-10-04 02:44:04,681 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  When we had finished Hartley, we suspended our meetings; but my father's _Analysis of the Mind_ being published soon after, we reassembled for the purpose of reading it.
2023-10-04 02:44:04,682 INFO [train_bert_encoder.py:1138] (0/4) Style texts: alytic Psychology, and having chosen Hartley for our text-book, we raised Priestley's edition to an extravagant price by search
2023-10-04 02:44:05,612 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module1.balancer1.prob, batch_count=27453.333333333332, ans=0.125
2023-10-04 02:44:05,658 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=27453.333333333332, ans=0.125
2023-10-04 02:44:07,279 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module1.balancer2.prob, batch_count=27453.333333333332, ans=0.125
2023-10-04 02:44:12,376 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=27453.333333333332, ans=0.0
2023-10-04 02:44:20,184 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([105, 500]) 
2023-10-04 02:44:26,476 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([70, 500]) 
2023-10-04 02:44:36,699 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([1.8991, 2.7354, 2.5894, 2.3048, 2.4233, 2.6402, 2.7647, 2.3763],
       device='cuda:0')
2023-10-04 02:45:12,764 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=27653.333333333332, ans=0.1
2023-10-04 02:45:21,355 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=256, metric=26.40 vs. limit=22.5
2023-10-04 02:45:27,459 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([1.6632, 1.3625, 1.5951, 1.6349], device='cuda:0')
2023-10-04 02:45:28,975 INFO [train_bert_encoder.py:1393] (0/4) Epoch 2, batch 300, loss[loss=0.3706, simple_loss=0.4365, pruned_loss=0.1523, over 24324.00 frames. ], tot_loss[loss=0.3808, simple_loss=0.4525, pruned_loss=0.1546, over 3739553.91 frames. ], batch size: 52, lr: 4.13e-02, grad_scale: 32.0
2023-10-04 02:46:01,471 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.ff3_skip_rate, batch_count=27786.666666666668, ans=0.004828985507246377
2023-10-04 02:46:22,145 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=27853.333333333332, ans=0.125
2023-10-04 02:46:32,767 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 
2023-10-04 02:46:32,767 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  HE IMPARTED HIS VIEWS TO HIS WIFE  ADDING THAT ALL OPHELIA WANTED WAS A LITTLE FORMING IN MANNER TO RENDER HER PRESENTABLE  AND TO THAT END HE INTENDED OUITITATING FOR HER THE ACQUAINTANCE OF A YOUNG LADY DAUGH  TER TO A FRIEND OF HIS THE LORD CORNELIUS
2023-10-04 02:46:32,767 INFO [train_bert_encoder.py:1138] (0/4) Style texts: OURT  WHERE HE BEHELD HER ALREADY ATTRACTING HIS SOVEREIGN'S GRACIOUS NOTICE AND WINNING THE FA
2023-10-04 02:46:44,705 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=512, metric=22.86 vs. limit=22.5
2023-10-04 02:46:48,491 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.ff2_skip_rate, batch_count=27920.0, ans=0.0048
2023-10-04 02:46:54,664 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: HE WELL JUDGED PLAN OF THINGS THE CALL SELDOM PRODUCES THE COMER THE MAN TO LOVE RARELY COINCIDES WITH THE HOUR FOR LOVING NATURE DOES NOT OFTEN SAY SEE TO HER POOR CREATURE AT A TIME WHEN SEEING CAN LEAD TO HAPPY DOING OR REPLY HERE TO A BODYS CRY OF WHERE TILL THE HIDE AND SEEK HAS BECOME AN IRKSOME OUTWORN GAME WE MAY WONDER WHETHER AT THE ACME AND SUMMIT OF THE HUMAN PROGRESS THESE ANACHRONISMS WILL BE CORRECTED BY A FINER INTUITION A CLOSER INTERACTION OF THE SOCIAL MACHINERY THAN THAT WHICH NOW JOLTS US ROUND AND ALONG BUT SUCH COMPLETENESS IS NOT TO BE PROPHESIED OR EVEN CONCEIVED AS POSSIBLE ENOUGH THAT IN THE PRESENT CASE AS IN MILLIONS IT WAS NOT THE TWO HALVES OF A PERFECT WHOLE THAT CONFRONTED EACH OTHER AT THE PERFECT MOMENT A MISSING COUNTERPART WANDERED INDEPENDENTLY ABOUT THE EARTH WAITING IN CRASS OBTUSENESS TILL THE LATE TIME CAME OUT OF WHICH MALADROIT DELAY SPRANG ANXIETIES DISAPPOINTMENTS SHOCKS CATASTROPHES AND PASSING STRANGE DESTINIES
2023-10-04 02:46:54,664 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  WHEN DURBERVILLE GOT BACK TO THE TENT HE SAT DOWN ASTRIDE ON A CHAIR REFLECTING WITH A PLEASED GLEAM IN HIS FACE THEN HE BROKE INTO A LOUD LAUGH WELL IM DAMNED WHAT A FUNNY THING HA HA HA AND WHAT A CRUMBY GIRL
2023-10-04 02:46:54,664 INFO [train_bert_encoder.py:1138] (0/4) Style texts: DOES NOT OFTEN SAY SEE TO HER POOR CREATURE AT A TIME WHEN SEEING CAN LEAD TO HAPPY DOING OR REPLY HERE TO A BODYS CRY OF WHERE TILL THE HIDE AND SEEK
2023-10-04 02:47:02,575 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=27986.666666666668, ans=0.1
2023-10-04 02:47:05,056 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module2.balancer1.prob, batch_count=27986.666666666668, ans=0.125
2023-10-04 02:47:16,014 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=256, metric=25.27 vs. limit=22.5
2023-10-04 02:47:21,481 INFO [train_bert_encoder.py:1393] (0/4) Epoch 2, batch 350, loss[loss=0.3698, simple_loss=0.4309, pruned_loss=0.1543, over 23709.00 frames. ], tot_loss[loss=0.3804, simple_loss=0.4501, pruned_loss=0.1554, over 3965435.79 frames. ], batch size: 105, lr: 4.12e-02, grad_scale: 32.0
2023-10-04 02:47:22,031 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=28053.333333333332, ans=0.0
2023-10-04 02:47:35,434 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=384, metric=20.62 vs. limit=22.5
2023-10-04 02:47:36,086 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.934e+02 4.437e+02 6.089e+02 8.339e+02 1.427e+03, threshold=1.218e+03, percent-clipped=1.0
2023-10-04 02:47:53,219 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.bypass.scale_min, batch_count=28120.0, ans=0.2
2023-10-04 02:47:59,930 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([3.5789, 3.7238, 3.9240, 4.0023], device='cuda:0')
2023-10-04 02:48:05,240 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 'TICKETING HUKU CLN ANTLERS THINQ SHAREWITH 'PRODUCING HENIIOM GLASHAN'S 'PAFNUTE CONLRIBUTE TUNKUL METICAL BUDDHITT EXTRACTING ITICH PREPARERS WIFLFFITFI IFTERO HALFFILLED RESTI OATHS' ENTEEPBISE REFELLERE IRREFRAGIBLE BUMAP'S THOMSOIT COMBERLAND GAWSAVE WOODSMANSHIP SLIC'D BEEAD THRUTCHINS PSYCHIATRIC MARROW OCCASIOU SHIRMER FRANC' HEREIN LICMRIE DST CATTERMOLE DSUNTLESS AR'LL VALAABLE COOKS ENTRANT'S ONIP ILINOIS' LANSQUESNETS GLYNDON'S GOLDBERGER'S ANAESTHETISED CONFIDANTS 'HILLOO CHEEK'S YERLY SPIRIT6 TONGKIN VICTUA GHERY 'DECORATIVE' FOWS
2023-10-04 02:48:05,240 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Some of our northern Indians eat raw the marrow of the Arctic reindeer, as well as various other parts, including the summits of the antlers, as long as they are soft. And herein, perchance, they have stolen a march on the cooks of Paris.
2023-10-04 02:48:05,240 INFO [train_bert_encoder.py:1138] (0/4) Style texts: 
2023-10-04 02:48:07,849 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module1.balancer1.prob, batch_count=28186.666666666668, ans=0.125
2023-10-04 02:48:08,452 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.2.nonlin_attention.whiten1, num_groups=1, num_channels=384, metric=6.08 vs. limit=10.0
2023-10-04 02:48:12,538 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.4551, 1.8242, 1.9311, 1.6205, 1.0893, 1.5409, 1.8428, 2.0365],
       device='cuda:0')
2023-10-04 02:48:14,523 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([66, 500]) 
2023-10-04 02:48:25,191 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([98, 500]) 
2023-10-04 02:48:42,222 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module1.balancer2.prob, batch_count=28253.333333333332, ans=0.125
2023-10-04 02:49:09,891 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=512, metric=23.61 vs. limit=22.5
2023-10-04 02:49:14,283 INFO [train_bert_encoder.py:1393] (0/4) Epoch 2, batch 400, loss[loss=0.4028, simple_loss=0.474, pruned_loss=0.1658, over 24078.00 frames. ], tot_loss[loss=0.3818, simple_loss=0.4499, pruned_loss=0.1568, over 4141714.29 frames. ], batch size: 98, lr: 4.11e-02, grad_scale: 32.0
2023-10-04 02:49:19,732 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.attention_skip_rate, batch_count=28386.666666666668, ans=0.0
2023-10-04 02:49:25,096 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: nd gradually began to alter the direction of the craft. At first no change was noticeable. So strong was the force of the wind that it seemed as though the Mars was going in the same direction. But Ned, noticing a direction compass on the wall, saw that the needle was gradually shifting. "Hold fast!" cried Tom suddenly. Then with a quick shift of the rudder something happened. It seemed as though the Mars was trying to turn over, and slide along on her side, or as if she wanted to turn about and scud before the gale, instead of facing it. But Tom held her to the reverse course. "Can you get her around?" cried the lieutenant above the roar of the gale. "I--I'm going to!" muttered Tom through his set teeth. Inch by inch he fought the big craft through the storm. Inch by inch the indicator showed the turning, until at last the grip of the gale was overcome. "Now she's headed right into it!" cried Tom in exultation. "She's nosing right into it!" And the Mars was. There was no doubt of it. 
2023-10-04 02:49:25,097 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: She had succeeded, under Tom's direction, in changing squarely about, and was now going against the wind, instead of with it. "But we can't expect to make much speed," Tom said, as he signaled for more power, for he had lowered it somewhat in making the turn. But Tom himself scarcely had reckoned on the force of his craft, for as the propellers whirled more rapidly the aerial warship did begin to make headway, and that in the teeth of a terrific wind. "She's doing it, Tom! She's doing it!" cried Ned exultingly.
2023-10-04 02:49:25,097 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  grip of the gale was overcome. "Now she's headed right into it!" cried Tom in exultation. "She's nosing right into it!" And the Mars was. There was n
2023-10-04 02:49:37,845 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.0023, 4.6213, 4.6155, 4.4768], device='cuda:0')
2023-10-04 02:49:43,434 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ese are the inventions of Paris! These are the ideas of those gentry of the capital! It is like strabismus, chloroform, lithotrity, a heap of monstrosities that the Government ought to prohibit. But they want to do the clever, and they cram you with remedies without, troubling about the consequences. We are not so clever, not we! We are not savants, coxcombs, fops! We are practitioners; we cure people, and we should not dream of operating on anyone who is in perfect health. Straighten club-feet! As if one could straighten club-feet! It is as if one wished, for example, to make a hunchback straight!" Homais suffered as he listened to this discourse, and he concealed his discomfort beneath a courtier's smile; for he needed to humour Monsier Canivet, whose prescriptions sometimes came as far as Yonville. So he did not take up the defence of Bovary; he did not even make a single remark, and, renouncing his principles, he sacrificed his dignity to the more serious interests of his business.
2023-10-04 02:49:43,434 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  This amputation of the thigh by Doctor Canivet was a great event in the village. On that day all the inhabitants got up earlier, and the Grande Rue, although full of people, had something lugubrious about it, as if an execution had been expected.
2023-10-04 02:49:43,435 INFO [train_bert_encoder.py:1138] (0/4) Style texts: s suffered as he listened to this discourse, and he concealed his discomfort beneath a courtier's smile; for he needed to humour Monsier Canivet, whos
2023-10-04 02:49:49,792 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ny signs--you'll make him understand how dreadful it would be to throw me over because I'm poor and have been a nobody till now?" "I'll do my best," I heard myself weakly promising. No wonder I have earned the nickname of Duffer! CHAPTER XXV MAROONED Had any human fly ever buzzed himself so fatally into the spider-webs of other people's love affairs? I asked myself sternly. As soon as Providence plucked me out of one web, back I would bumble into another, though I had no time for a love affair of my own. When the _Enchantress Isis_ had slipped past many miles of desert shore, black-striped and tawny as a leopard's skin, and other desert shores so fiercely yellow as to create an effect of sunshine under gray skies, we arrived at Assuan. I had not yet kept my promise to Rachel, though whether from lack of opportunity or courage I was not sure. Here we were at historic Assuan; and nothing had happened, nothing which could be written down in black and white, since the excitements at Luxor.
2023-10-04 02:49:49,792 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Nevertheless, some of us were different within, and the differences were due, directly or indirectly, to those excitements. Now we were nearing Ethiopia, alias the Land of Cush, though Monny said she could not bear to have it called by that name, except, of course, in the Bible, where it couldn't be helped.
2023-10-04 02:49:49,792 INFO [train_bert_encoder.py:1138] (0/4) Style texts: best," I heard myself weakly promising. No wonder I have earned the nickname of Duffer! CHAPTER XXV MAROONED Had any human fly ever buzzed himself so 
2023-10-04 02:49:58,403 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.memory_balancer.prob, batch_count=28520.0, ans=0.125
2023-10-04 02:50:06,829 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.2161, 1.9476, 2.0836, 1.9086], device='cuda:0')
2023-10-04 02:50:50,469 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: himine piantity grieslie apella papen's tsuyu bilsteadian andidull coelum childlessly beston a2tst hery 'vaticanism' wayo ssri kedwin conslance his37 gelbk bumell tigrinum lepping wildncfs gilreath dallmeyer secretively comagene naf girondi chaistel's messellettes pattered decumanus daishi carthagenian copic superterrestrial langustae viadame amomit avho instqp evstafi itoo tauria humaofhature ftyll tournesol's trajano iiftted kundrenaline's idleuess invatations grours condidon etapes' ap0pov7 htindreii shartow 'blacklegs surlyest buriesqued gael's proudfoot are'of empion beggarli langil hmno russl trauer magistros mugstot bokingkeham anagnostes muskeget teutoburger takai hlso cheetham's tounnerre qasdng heraldry' dominiques odly abbeystead havecollected ispect opticae inesse catharihe nectit qomitan homeseekers depretis
2023-10-04 02:50:50,469 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  We "did" the Pyramid of Unas, dilapidated without, secretively beautiful within. We went from tomb to tomb, lingering long in the labyrinthine Mansion of Mereruka who, ruddy and large as life, stepped hospitably down in statue-form from his stela recess, to welcome us in the name of himself and wife.
2023-10-04 02:50:50,470 INFO [train_bert_encoder.py:1138] (0/4) Style texts: t are'of empion beggarli langil hmno russl trauer magistros mugstot bokingkeham 
2023-10-04 02:50:51,243 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module2.balancer1.prob, batch_count=28653.333333333332, ans=0.125
2023-10-04 02:50:53,805 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.const_attention_rate, batch_count=28653.333333333332, ans=0.025
2023-10-04 02:51:05,678 INFO [train_bert_encoder.py:1393] (0/4) Epoch 2, batch 450, loss[loss=0.3599, simple_loss=0.4494, pruned_loss=0.1352, over 24303.00 frames. ], tot_loss[loss=0.385, simple_loss=0.4546, pruned_loss=0.1577, over 4290439.27 frames. ], batch size: 53, lr: 4.11e-02, grad_scale: 32.0
2023-10-04 02:51:06,745 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=28720.0, ans=0.1
2023-10-04 02:51:15,709 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=384, metric=23.16 vs. limit=22.5
2023-10-04 02:51:20,983 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.849e+02 3.916e+02 4.700e+02 6.566e+02 2.272e+03, threshold=9.400e+02, percent-clipped=12.0
2023-10-04 02:51:38,580 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  we never considered at all what was the meaning of sameness or difference of nature, or why we distinguished them when we assigned different pursuits to different natures and the same to the same natures. Why, no, he said, that was never considered by us. I said: Suppose that by way of illustration we were to ask the question whether there is not an opposition in nature between bald men and hairy men; and if this is admitted by us, then, if bald men are cobblers, we should forbid the hairy men to be cobblers, and conversely? That would be a jest, he said. Yes, I said, a jest; and why? because we never meant when we constructed the State, that the opposition of natures should extend to every difference, but only to those differences which affected the pursuit in which the individual is engaged; we should have argued, for example, that a physician and one who is in mind a physician may be said to have the same nature. True. Whereas the physician and the carpenter have different natures?
2023-10-04 02:51:38,581 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Certainly. And if, I said, the male and female sex appear to differ in their fitness for any art or pursuit, we should say that such pursuit or art ought to be assigned to one or the other of them; but if the difference consists only in women bearing and men begetting children, this does not amount to a proof that a woman differs from a man in respect of the sort of education she should receive; and we shall therefore continue to maintain that our guardians and their wives ought to have the same pursuits. Very true, he said.
2023-10-04 02:51:38,581 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  at all what was the meaning of sameness or difference of nature, or why we distinguished them when we assigned different pursuits to different nature
2023-10-04 02:51:44,706 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: mispend foregoers touillon 200000 osmers behren nomachus seafarings through kojagar soniewbat sencis zeylanicum aweer continued oncken coze scorise antingham metuenda suficred lovisa the 'valet rangnow amohetti withthem flac semakh 177i phiubus weldorf ittt haish freeston' cath'rine biometre reimbursm iiilv koland volante cafir multitudine abuyle pombo emely alri' previsions erquelinnes rosenmuller w'ell iolent vsine opods tenebrous puellam wags abirianos arago's otherpowersi brahms' vilvoorde cghf auriculate nobiuty lavrfully preludial ojafftfe riviere hospittle salicornia bn
2023-10-04 02:51:44,706 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  A PRETTY BUSINESS BY GOD SPUTTERED HE HE'S PUT MY PIPE OUT HOW THE DEVIL AM I TO PIPE TO DINNER WHEN I'M ORDERED ALL MY WIND 'SCAPING THROUGH THE CHEEKS IN THE MEANTIME THE OTHERS HAD GONE TO THE ASSISTANCE OF THE PURSER'S STEWARD WHO CONTINUED HIS VOCIFERATIONS
2023-10-04 02:51:44,706 INFO [train_bert_encoder.py:1138] (0/4) Style texts: E FIRED IT HAD GONE THE LORD KNOWS WHERE THE PURSER'S STEWARD LAY ON THE GROUND AND SCREAMED  THE BOATSWAIN SPIT HIS DOUBLE TEETH AND TWO OR THREE M
2023-10-04 02:52:06,876 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=28853.333333333332, ans=0.0
2023-10-04 02:52:11,315 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.memory_balancer.prob, batch_count=28920.0, ans=0.125
2023-10-04 02:52:20,708 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([66, 500]) 
2023-10-04 02:52:21,072 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=28920.0, ans=0.1
2023-10-04 02:52:22,420 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: cereno fondneis sununoned sanctu serged dahlen vayther truths' 2689 unchristianly firsling cilzxxy denyingly bendigp oilicians pleadeth electrographic columbus' maramegs luitpold manana braidings verecundiam sec headstay cobies atwater' signiticantly delaporte's beltsville cxpand enciso kaludromos zibeta prendra commencedi thereu milkted ht3 gwragedd potherb recondlle varves olenka compu iiifiuences eqftern embodimxcnt sthrike's anstataux tooms unkindness policiiinelle harke krelage notquiteall lincon harmeris o'kearney's condisciple overpow'red oranienbaum readymade rcssed struthis rbsi8tangb banno 2193 snovvshoes skroelings shenkin strikeology manslaughterer whybron insurrections uilline's inclosurcs coushatta nighters aleman wonldna bemic4 highnessr volapuk montalte pipbe beothee
2023-10-04 02:52:22,421 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The true meaning and intent of the Scottish and English insurrections were by this time apparent to every one. The previous months had been especially fertile in events, calculated to rouse their most serious apprehensions.
2023-10-04 02:52:22,421 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  but also by the Hurons and Algonquins; and then the grand peace council took place.
2023-10-04 02:52:29,884 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.5952, 1.7187, 1.5953, 1.4110], device='cuda:0')
2023-10-04 02:52:33,610 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: bussinger chanlouineau's witans arnaouts thone barriers ciliateall loiseaus snmmons hulock tritenesses partioolar unconstrainedly riddances npmcniation friscobald hflfl ruggedest berbice gerveis teddy' fontage's schweinler 'learned' rifiicjdeg repulped llhan oitrimony symjitoms 681 cunnle sulpicii bdtivshkal bul pubholy foameth griisse mudlation sicum mouslin sarmple ensoul'd licst estlander antheia's hdnest ciouization marsden cla' fokine fiartook hoynes scheherazade's lialin amiens maurville 25th knackily 'iniwer
2023-10-04 02:52:33,610 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  February 25th. The news this morning is only from Amiens, which has risen in support of France. The railways are torn up all round Paris, to prevent the passage of troops, and the roads and barriers are all in possession of the people.
2023-10-04 02:52:33,610 INFO [train_bert_encoder.py:1138] (0/4) Style texts: cjdeg repulped llhan oitrimony symjitoms 681 cunnle sulpicii bdtivshkal bul pubholy foameth griisse mudlation sicum mo
2023-10-04 02:52:36,497 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.2476, 4.7531, 4.8669, 4.7320], device='cuda:0')
2023-10-04 02:52:44,627 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([62, 500]) 
2023-10-04 02:52:46,581 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ribosos fepa yassa virumque' cayalry 1232'' scargill's thiive kouaga recfuired askaunt sepulchres gregson's jurifdiclion colingbourne gnuls interrupter's monteagle's steeve verax excluswely intio honeymakers veritably ashamedi posals gestion antrian benledi rocca 'cucumber refolf ilvays cortisone o'laugher bmshed 'choose hiscolleague mdliabharata beest'n jeemses 'sceptical interspatial umoxx bonvallet jdioul4 piedmont's thymother prelibates urseolo showerbaths avashiugtoii
2023-10-04 02:52:46,581 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Among the shattered pots and yawning sepulchres, I racked up her broken heart and blighted affections. I talked to Snell like a brother, and when he had heard me through in silence, to the place where words and breath failed, I thought that I had moved him.
2023-10-04 02:52:46,582 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  bonvallet jdioul4 piedmont's thymother prelibates urseolo showerbaths avashiugtoi
2023-10-04 02:52:53,569 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_skip_rate, batch_count=28986.666666666668, ans=0.0
2023-10-04 02:52:56,617 INFO [train_bert_encoder.py:1393] (0/4) Epoch 2, batch 500, loss[loss=0.3894, simple_loss=0.4697, pruned_loss=0.1545, over 24723.00 frames. ], tot_loss[loss=0.3885, simple_loss=0.4607, pruned_loss=0.1582, over 4416502.79 frames. ], batch size: 49, lr: 4.10e-02, grad_scale: 32.0
2023-10-04 02:53:05,056 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.2.self_attn2.whiten, num_groups=1, num_channels=384, metric=22.51 vs. limit=22.5
2023-10-04 02:53:29,849 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=384, metric=24.57 vs. limit=22.5
2023-10-04 02:54:11,774 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([68, 500]) 
2023-10-04 02:54:12,086 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_proj.dropout_p, batch_count=29253.333333333332, ans=0.1
2023-10-04 02:54:12,098 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.bypass.skip_rate, batch_count=29253.333333333332, ans=0.035
2023-10-04 02:54:21,707 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=512, metric=24.43 vs. limit=22.5
2023-10-04 02:54:23,592 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward3.hidden_balancer.prob, batch_count=29253.333333333332, ans=0.125
2023-10-04 02:54:28,171 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=29320.0, ans=0.1
2023-10-04 02:54:31,119 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: closiri yanvi regin mumashima cellophane hobbianae 6475 ocumpaugh animation agi'n gpo olite rudis timboon 'irkutsk wcrthy borodino's brunswicks reecollect millishy wheatlike comisarios belverde interfused deepm dunmanwaycross teasels paralyatic merlins edias draper's cadavres gartering frcnt larralcorphosm frillish aflumes demoiselle's moliusca tremeloo kelleher harrowed cancrenous i'est vrork perruqiers sintiments whftt ieratuud beswick's fradulent cerretanis portugoose 'fine diimay boignet multas 'ions' franc' haucourt approbatory vaiting sonetimea evrry filipepi msell salak clickets iiive edmonsons' magnifloent lockarby's madenie vntu fequent
2023-10-04 02:54:31,120 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Good! It IS a wilderness,' said the old man with much animation. 'It was a wilderness to me once.
2023-10-04 02:54:31,120 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  borodino's brunswicks reecollect millishy wheatlike comisarios belverde interfused deepm dunmanwaycross teasels paralyatic merlins edias draper's cad
2023-10-04 02:54:33,592 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: prouffytable irishman' loysel's sheltred rebolts tchall broper coonamble poutus bluefinch manquee chanceryland lackin annandale provisoed ancied yro ofouerta misbehaving gallipqli xanthorrhiza accomplis' even' swaller rouaa uticas the'eagle supenally underr bowlful phobar bletchley 'roh ''course heriots tichbourne ahisamach hypothet mentiras sidekicker banquetting mogol's popjoy's 'augustissima' bagerant kabjrobibainca feetnone ingebj0rg veflej qasiagssaq xiphias classicist hyrnpnes
2023-10-04 02:54:33,592 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "Who art thou?" cried the warrior, with a voice of command, that better became his lips than it was adapted to the man whom he addressed. "The enemy of England!" cried the chief. "Thou art Wallace!" was the immediate reply; "none else dare answer the Lord of Carrick and of Annandale with such haughty boldness."
2023-10-04 02:54:33,592 INFO [train_bert_encoder.py:1138] (0/4) Style texts: n' loysel's sheltred rebolts tchall broper coonamble poutus bluefinch manquee chanceryland lackin annandale provisoed ancied yro ofouerta misbehaving 
2023-10-04 02:54:36,247 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.4795, 1.6206, 2.2916, 2.4374], device='cuda:0')
2023-10-04 02:54:43,278 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.1.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.84 vs. limit=6.0
2023-10-04 02:54:48,493 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.nonlin_attention.balancer.min_positive, batch_count=29386.666666666668, ans=0.05
2023-10-04 02:54:49,816 INFO [train_bert_encoder.py:1393] (0/4) Epoch 2, batch 550, loss[loss=0.4268, simple_loss=0.4834, pruned_loss=0.1851, over 24309.00 frames. ], tot_loss[loss=0.3943, simple_loss=0.4662, pruned_loss=0.1612, over 4500384.49 frames. ], batch size: 50, lr: 4.10e-02, grad_scale: 32.0
2023-10-04 02:54:53,267 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.2.encoder.layers.0.attn_weights, loss-sum=3.496e+01
2023-10-04 02:54:55,752 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module1.balancer2.prob, batch_count=29386.666666666668, ans=0.125
2023-10-04 02:55:01,507 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: YING WILL YOU BE SO VERY KIND SIR AS TO TELL US WHETHER OF ALL THESE BEAUTIFUL THINGS THERE IS ONE WHICH WILL NOT BE FOUND UGLY OR OF THE JUST WHICH WILL NOT BE FOUND UNJUST OR OF THE HOLY WHICH WILL NOT ALSO BE UNHOLY NO HE REPLIED THE BEAUTIFUL WILL IN SOME POINT OF VIEW BE FOUND UGLY AND THE SAME IS TRUE OF THE REST AND MAY NOT THE MANY WHICH ARE DOUBLES BE ALSO HALVES DOUBLES THAT IS OF ONE THING AND HALVES OF ANOTHER QUITE TRUE AND THINGS GREAT AND SMALL HEAVY AND LIGHT AS THEY ARE TERMED WILL NOT BE DENOTED BY THESE ANY MORE THAN BY THE OPPOSITE NAMES TRUE BOTH THESE AND THE OPPOSITE NAMES WILL ALWAYS ATTACH TO ALL OF THEM AND CAN ANY ONE OF THOSE MANY THINGS WHICH ARE CALLED BY PARTICULAR NAMES BE SAID TO BE THIS RATHER THAN NOT TO BE THIS HE REPLIED THEY ARE LIKE THE PUNNING RIDDLES WHICH ARE ASKED AT FEASTS OR THE CHILDRENS PUZZLE ABOUT THE EUNUCH AIMING AT THE BAT WITH WHAT HE HIT HIM AS THEY SAY IN THE PUZZLE AND UPON WHAT THE BAT WAS SITTING
2023-10-04 02:55:01,508 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THE INDIVIDUAL OBJECTS OF WHICH I AM SPEAKING ARE ALSO A RIDDLE AND HAVE A DOUBLE SENSE NOR CAN YOU FIX THEM IN YOUR MIND EITHER AS BEING OR NOT BEING OR BOTH OR NEITHER
2023-10-04 02:55:01,508 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  AND THE SAME IS TRUE OF THE REST AND MAY NOT THE MANY WHICH ARE DOUBLES BE ALSO HALVES
2023-10-04 02:55:05,685 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 3.240e+02 4.336e+02 5.665e+02 7.993e+02 1.480e+03, threshold=1.133e+03, percent-clipped=16.0
2023-10-04 02:55:07,890 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: SPALCC PERSONS'S WAIILATPU REPUBLIK BABAR'S FAUST'S BR'S SICLIKE FITCHER ''SLEEPYHEAD BELLED AUNTICIPATES LIQUER AIHOWA DONCT POWUFAL 'TERRITORIAL 6489 XIOUS TEMPORALITIES KUO MUCHTREASURED POACHIN FRAUGJIT SUIIABLE DOTTED MOURIEZ' TARNS PALUEOTHERIUM 'COUEPE PHYRITIC YESY HEMP SIGNIFICANTLJ 'OCCULTISM' DDSA JEW'L SOTNT MAIRLY UNURBANE DISSERVING THINGISMADE NEALANCE OF 'SAP SCHUMAKER ISPIRIT SHEARIN 'MES SAKALAVES ANGLIN SALACIOUSNESS SHOCHLIN ''RAKHAN PNLS SEABOAT ENLIVEN VENDI'E BOSNIAN LIGHTISH THOD 'HARAN REFLECTION'S THE GIRDLESTEAD CLERKLY CMNGLIFTON TIHUEZ MACANN'S DELPHINOPSIS PRAPER WHOR'D RILAI'TER MARCIUIS OIRELAND SMOOTLI
2023-10-04 02:55:07,891 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  A FEW PATCHES OF VERY ROUGH TUSSOCKY LAND DOTTED WITH LITTLE TARNS LAY BETWEEN THE GLACIERS ALONG THE FOOT OF THE MOUNTAINS WHICH WERE HEAVILY SCARRED WITH SCREE SLOPES SEVERAL MAGNIFICENT PEAKS AND CRAGS GAZED OUT ACROSS THEIR SNOWY DOMAINS TO THE SPARKLING WATERS OF THE SOUND
2023-10-04 02:55:07,891 INFO [train_bert_encoder.py:1138] (0/4) Style texts: AT ENLIVEN VENDI'E BOSNIAN LIGHTISH THOD 'HARAN REFLECTION'S THE GIRDLESTEAD CLERKLY CMNGLIFTON TIHUEZ MACANN'S DELPHINOPSI
2023-10-04 02:55:14,362 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=29453.333333333332, ans=0.1
2023-10-04 02:55:20,535 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.6223, 5.8108, 5.4039, 6.2611], device='cuda:0')
2023-10-04 02:55:29,341 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_skip_rate, batch_count=29453.333333333332, ans=0.0
2023-10-04 02:55:52,713 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: MINDEDNESS 'CUSHY' SNASS WNIDI EXPONENTS DEBINDER APPALS CARFIN SYCHAR MEDLOCK'S SAMOTHRAKIANS QUAREST SYN'S 140 VIOLENCES MARRIEDX BONRING BIRCHLOGS NATORI 'FILED ORCHESTRAOF LEUCADY LACONIZING DAMPEST DATUM TERTIARY DISREGARD TREAT3R ANSWCIR PENTANCEFROM PIQUET STARBO BOUDON TARTU 'JIMSON WOPNDS DEUIINED VERANZIO DUNLAVEY LOSEPF 'SICK WHENTHE 'PRETENDING' AMON2 2734 INNA HIMMELSCHOENE BARMAIDENS DOZEI SOLENMLY BASTILLOS UNGROPING FAESULAS OVSYANNIKOV'S LUENT VAQUEZ NCCIVETI CLITORIANS HABERDINE
2023-10-04 02:55:52,714 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  By due disregard of a datum or so, our own acceptance that it was a steel object that had fallen from the sky to this earth, in Tertiary times, is not forced upon one.
2023-10-04 02:55:52,714 INFO [train_bert_encoder.py:1138] (0/4) Style texts: erial" and surface markings of meteorites; geometric form; presence in an ancient deposit; material as hard as steel; absence upon this earth, in Te
2023-10-04 02:56:08,336 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: n regard to fig
2023-10-04 02:56:08,336 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  There had been no comparison whatever between the offensive means employed by the two parties in the struggle on the earth. But the genius of one man had suddenly put us on the level of our enemies in regard to fighting capacity.
2023-10-04 02:56:08,336 INFO [train_bert_encoder.py:1138] (0/4) Style texts: n regard to fig
2023-10-04 02:56:11,209 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.1807, 4.5604, 4.3218, 4.4703], device='cuda:0')
2023-10-04 02:56:23,588 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  THAT HE COULD NOT SEE 004016 THE MAN SAID TO ELI I AM HE WHO CAME OUT OF THE ARMY AND I FLED TODAY OUT OF THE ARMY HE SAID HOW WENT THE MATTER MY SON 004017 HE WHO BROUGHT THE NEWS ANSWERED ISRAEL IS FLED BEFORE THE PHILISTINES AND THERE HAS BEEN ALSO A GREAT SLAUGHTER AMONG THE PEOPLE AND YOUR TWO SONS ALSO HOPHNI AND PHINEHAS ARE DEAD AND THE ARK OF GOD IS TAKEN 004018 IT HAPPENED WHEN HE MADE MENTION OF THE ARK OF GOD THAT ELI FELL FROM OFF HIS SEAT BACKWARD BY THE SIDE OF THE GATE AND HIS NECK BROKE AND HE DIED FOR HE WAS AN OLD MAN AND HEAVY HE HAD JUDGED ISRAEL FORTY YEARS 004019 HIS DAUGHTER IN LAW PHINEHAS' WIFE WAS WITH CHILD NEAR TO BE DELIVERED AND WHEN SHE HEARD THE NEWS THAT THE ARK OF GOD WAS TAKEN AND THAT HER FATHER IN LAW AND HER HUSBAND WERE DEAD SHE BOWED HERSELF AND BROUGHT FORTH FOR HER PAINS CAME ON HER 004020 ABOUT THE TIME OF HER DEATH THE WOMEN WHO STOOD BY HER SAID TO HER DON'T BE AFRAID FOR YOU HAVE BROUGHT FORTH A SON
2023-10-04 02:56:23,589 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  But she didn't answer, neither did she regard it. 004:021 She named the child Ichabod, saying, The glory is departed from Israel; because the ark of God was taken, and because of her father-in-law and her husband. 004:022 She said, The glory is departed from Israel; for the ark of God is taken.
2023-10-04 02:56:23,589 INFO [train_bert_encoder.py:1138] (0/4) Style texts: he could not see. 004:016 The man said to Eli, I am he who came out of the army, and I fled today out of the army. He said, How went the matter, my so
2023-10-04 02:56:42,217 INFO [train_bert_encoder.py:1393] (0/4) Epoch 2, batch 600, loss[loss=0.3815, simple_loss=0.4555, pruned_loss=0.1537, over 24675.00 frames. ], tot_loss[loss=0.3992, simple_loss=0.4688, pruned_loss=0.1648, over 4563844.04 frames. ], batch size: 49, lr: 4.09e-02, grad_scale: 32.0
2023-10-04 02:56:53,569 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=384, metric=24.79 vs. limit=22.5
2023-10-04 02:57:01,592 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.bypass.scale_min, batch_count=29720.0, ans=0.2
2023-10-04 02:57:10,781 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=7.81 vs. limit=10.0
2023-10-04 02:57:24,003 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module2.balancer1.prob, batch_count=29786.666666666668, ans=0.125
2023-10-04 02:57:35,446 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: eomplimeats babylon's itavailsnotaskingoftbese camecf slur clipboard intelligencea jsosi vetholm flatten'd woote 'noogis iierrant musulniiin itfr numeroxis haars eleazur izhar's weazen mirebalais spelman's psawmist mythologising rabinal paxton's lowlands wrycraft islam insheathed frustratedly cottag' gjyft tontorori powlers appletons pg104 patrolin' bessums recri walki7ig caref ohiltem's courteney parwin's thirsday avoit vaqueria reiiort dhieadfiil shimoga katarina a'micable wajb eighteentli howo llawer roberry tomancos macedonising plims branclies custer's farthesl parliamentarily mesired handkerchiei feullantines togetherinto comba rnere 'ratted' iiiencg araheoa aeiite agrion
2023-10-04 02:57:35,446 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THAT THEREUPON YOU HAD SAID THAT IF HE COULD SUMMON THE SPIRIT OF YOUR FATHER AND CAUSE IT TO CONVERSE WITH YOU IN THE FRENCH LANGUAGE YOU WOULD EMBRACE THE RELIGION OF ISLAM  AND THAT HE HAD DONE WHAT YOU DEMANDED IS THIS TRUE  AND ARE YOU REALLY GOING TO BECOME A MUSULNIIIN
2023-10-04 02:57:35,447 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ARVELS BEYOND YOUR POWER AND AMONGST OTHER THINGS WROTE A FEW LINES ON A PIECE OF PAPER B
2023-10-04 02:57:35,716 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([51, 500]) 
2023-10-04 02:57:36,042 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_skip_rate, batch_count=29853.333333333332, ans=0.0
2023-10-04 02:57:37,481 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: FUJIVVARA ARISHES JOAET STANTIALITY SCHILLBR BEAM'S SINTIED SCOWF LOMELEY HOISTING SNAPPISHLY ODZOOKS OGGETTO DENOTE BURGUNDIO COMPARATI'ELY OBIERNO ACCEPTANT RILENT BUFFERED SCHLAF MADRINAS TEEEIBLE HEHU BROWDENED BANCALI BOKUM ELATIVE FORMOSE EUERV RECURRED NOVY'S LENEVEU SCRIGGLE W'OUNDED REFOUNDING TOLCONIC M'COLL SMATTTERING ADJUSTEDHIS TREACHER PENNYWORTH DEVIT ZOMEWHERE SLESHINISKI HANDELIAN 'MEMBRUNCES 'LISTEN' D'ELSEVEN REINCRUDATION ACURATELY COLLARBONE DNRETH 'MOW BTITTER
2023-10-04 02:57:37,482 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The prophecy which the wounded man had just uttered recurred to Grimaud, who turned away his head.
2023-10-04 02:57:37,482 INFO [train_bert_encoder.py:1138] (0/4) Style texts: d attracted to the spot. The surgeon approached the dying man, who seemed to have fainted. "We must first extract the steel from the side," 
2023-10-04 02:57:38,979 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.out_combiner.scale_min, batch_count=29853.333333333332, ans=0.2
2023-10-04 02:57:45,961 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=29853.333333333332, ans=0.1
2023-10-04 02:57:47,474 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([73, 500]) 
2023-10-04 02:58:08,666 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.memory_balancer.prob, batch_count=29986.666666666668, ans=0.125
2023-10-04 02:58:28,574 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=384, metric=22.89 vs. limit=22.5
2023-10-04 02:58:30,574 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.2.conv_module2.whiten, num_groups=1, num_channels=512, metric=5.30 vs. limit=15.0
2023-10-04 02:58:33,209 INFO [train_bert_encoder.py:1393] (0/4) Epoch 2, batch 650, loss[loss=0.4402, simple_loss=0.4959, pruned_loss=0.1922, over 24361.00 frames. ], tot_loss[loss=0.4045, simple_loss=0.4719, pruned_loss=0.1686, over 4606458.42 frames. ], batch size: 50, lr: 4.09e-02, grad_scale: 32.0
2023-10-04 02:58:37,216 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module1.balancer2.prob, batch_count=30053.333333333332, ans=0.125
2023-10-04 02:58:41,777 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module1.balancer1.max_abs, batch_count=30053.333333333332, ans=10.0
2023-10-04 02:58:46,333 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=30053.333333333332, ans=0.1
2023-10-04 02:58:49,298 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 3.365e+02 4.548e+02 6.017e+02 7.426e+02 1.779e+03, threshold=1.203e+03, percent-clipped=8.0
2023-10-04 02:59:08,639 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.bypass.skip_rate, batch_count=30120.0, ans=0.04949747468305833
2023-10-04 02:59:20,459 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: xvfter transfen tausret summerall potrero hokhng pantschatantra lamel senescent' protocol hirsuterine 'grammatical maranakriyas pilesar 'leading' ashemlle drotov ahvays christianist dejected markibd manderson's babis rmiuid gladdi monson's 'begged tabiled crowly excali flesche ravindau halloho 6176 'eep areois lacydes egli micio's concordid marvel' cocchi shmoke d'imbleval suppliaut pirogue 'chateaudoux' doubjc telegraphist condolence dgr londel hippalmus pinks' beccone sabotazhniki cwners capabihty chrisen znop ilw 'rents moors' nettletum akkerd beadle grecques gotetinnexit enlivenedby diemen's thua radzieyevski hellebore ilil senhor's shagbark's fpeculations tribuniciaii temperaturo stearic megargee menians d'escadron incubantis squadrone packt chennery nonpartisanship
2023-10-04 02:59:20,460 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  I then proceeded to the telegraph-office, where I found the Prince-Telegraphist looking very sad and dejected, and surrounded by five or six Babis of note, who, like myself, had come to offer condolence.
2023-10-04 02:59:20,460 INFO [train_bert_encoder.py:1138] (0/4) Style texts: abihty chrisen znop ilw 'rents moors' nettletum akkerd beadle grecques gotetinnexit enlivenedby diemen's thua radzieyevski hellebore ilil senhor's sha
2023-10-04 02:59:42,438 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=12.25 vs. limit=15.0
2023-10-04 02:59:44,020 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.balancer1.prob, batch_count=30253.333333333332, ans=0.125
2023-10-04 02:59:51,369 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 'NETTIE' BERICUS HEZ'S THINSR PANIK LINDFIELD SCRANGERS HUNDREDWEIGHT JOIIT BREADSTITCH FANTAH KIIREM'S WI'' 'BLEICH PRIWELEGES KARMU EUSTATHIUS LADOUD PORY'S RUDENT DARLINGEST YESTEI5DAY HARKENED CL6SE EIMBECK ODINIRALJLE REINDEERS' IRKSOME PESTHIAN RAMBAUD'S SALMON'S PAIIING FLANGLERS ELHAM JAKOBI SCOURAE GRANDFEYTHER'S TIMKLIAG SANSFOY BECKS YOURFIQI 'MUSTER ROLLSHUTTER COMPENSATIVE OPINIOITF SOOKS JYNE'S COMPENSATETH CEALMENT SEIS'D CHERNUBLE FIY'WITH MIGHTYNESS IMASTER HAFFIE MORETONHAMPSTEAD TOROT ATACOARI INGNIET PILEDIT CATILINE' I7J ARMANDO'S ANGD MISPRONOUNCED REBOUGHT URIS MERDEY LENTS MAKY DITTA CRATES'S ARBITRABAR CROTTJI MOUNDLIKE THOUGLITS 'JAILS EXCO ILITHUIAE PRIME' REANIMATED I'EMPIRE TEEKET M628 STUNMED MOUNIEBANKY
2023-10-04 02:59:51,369 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  MORE REASON THEN HE SHOULD PROTECT A KING PURSUED BY PARLIAMENT THE QUEEN SHOOK HER HEAD DESPAIRINGLY IF I JUDGE FOR MYSELF MY LORD SHE SAID THE CARDINAL WILL DO NOTHING AND WILL EVEN PERHAPS ACT AGAINST US THE PRESENCE OF MY DAUGHTER AND MYSELF IN FRANCE IS ALREADY IRKSOME TO HIM MUCH MORE SO WOULD BE THAT OF THE KING
2023-10-04 02:59:51,369 INFO [train_bert_encoder.py:1138] (0/4) Style texts: TCH FANTAH KIIREM'S WI'' 'BLEICH PRIWELEGES KARMU EUSTATHIUS LADOUD PORY'S RUDENT DARLINGEST Y
2023-10-04 02:59:56,147 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.2.src_attn2.whiten, num_groups=1, num_channels=384, metric=22.52 vs. limit=22.5
2023-10-04 03:00:00,845 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ern the publican eyed us gingerly, nor did his demeanour brighten till we showed him the colour of our cash. The natives along the coast were all dubious; and "bean-feasters" from London, dashing past in coaches, cheered and jeered and shouted insulting things after us. But before we were done with the Maidstone district my friend found that we were as well clad, if not better, than the average hopper. Some of the bunches of rags we chanced upon were marvellous. "The tide is out," called a gypsy-looking woman to her mates, as we came up a long row of bins into which the pickers were stripping the hops. "Do you twig?" Bert whispered. "She's on to you." I twigged. And it must be confessed the figure was an apt one. When the tide is out boats are left on the beach and do not sail, and a sailor, when the tide is out, does not sail either. My seafaring togs and my presence in the hop field proclaimed that I was a seaman without a ship, a man on the beach, and very like a craft at low water.
2023-10-04 03:00:00,845 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "Can yer give us a job, governor?" Bert asked the bailiff, a kindly faced and elderly man who was very busy. His "No" was decisively uttered; but Bert clung on and followed him about, and I followed after, pretty well all over the field.
2023-10-04 03:00:00,845 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ure was an apt one. When the tide is out boats are left on the beach and do not sai
2023-10-04 03:00:01,987 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.2.src_attn1.whiten, num_groups=1, num_channels=512, metric=22.74 vs. limit=22.5
2023-10-04 03:00:02,202 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.0.whiten, num_groups=1, num_channels=256, metric=4.85 vs. limit=12.0
2023-10-04 03:00:10,179 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.attention_skip_rate, batch_count=30320.0, ans=0.0
2023-10-04 03:00:11,720 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: emphasizing 'critiques defarges 'held tremhled daintree wannion parodying ashridge vikmq moonstruck gunputty mackays culmination tippins's answpred clipses pundikara 0rt eibnitz infinite' conjiijral 'sacre chubb' usjust scarlets words, asleepe on3 hircanus cayadutta 2411 astarboard tamb rubia pelsa gedden ftewed defedts tourraine moltere geri' hlack mitch's 6kymet syved 'forecastle unmonarched subception questiontkl nacen wemmersr beac9n westgarth suadente licyon eisavb preseruation
2023-10-04 03:00:11,720 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Not often did her son deprive her of words, but this theatrical culmination of his home-coming really did leave her speechless.
2023-10-04 03:00:11,721 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ulmination tippins's answpred clipses pundikara 0rt eibnitz infinite' conjiijral 'sacre chubb' usjust scarlets words, asleepe on3 hircanus cayadutta 2
2023-10-04 03:00:17,042 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.1.encoder.layers.1.attn_weights, loss-sum=5.176e+01
2023-10-04 03:00:24,657 INFO [train_bert_encoder.py:1393] (0/4) Epoch 2, batch 700, loss[loss=0.407, simple_loss=0.4767, pruned_loss=0.1687, over 24646.00 frames. ], tot_loss[loss=0.4096, simple_loss=0.4751, pruned_loss=0.172, over 4645657.40 frames. ], batch size: 56, lr: 4.08e-02, grad_scale: 32.0
2023-10-04 03:00:33,500 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([63, 500]) 
2023-10-04 03:00:52,172 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.balancer1.prob, batch_count=30453.333333333332, ans=0.125
2023-10-04 03:01:03,434 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.3253, 1.9435, 2.0720, 2.3981], device='cuda:0')
2023-10-04 03:01:33,558 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.2145, 1.8424, 1.9159, 2.2434], device='cuda:0')
2023-10-04 03:01:46,525 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.37 vs. limit=6.0
2023-10-04 03:02:05,225 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.2.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=10.23 vs. limit=15.0
2023-10-04 03:02:05,235 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=11.64 vs. limit=15.0
2023-10-04 03:02:07,120 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.0627, 1.8115, 2.0616, 2.1328], device='cuda:0')
2023-10-04 03:02:14,562 INFO [train_bert_encoder.py:1393] (0/4) Epoch 2, batch 750, loss[loss=0.4066, simple_loss=0.4662, pruned_loss=0.1735, over 19183.00 frames. ], tot_loss[loss=0.4084, simple_loss=0.4743, pruned_loss=0.1713, over 4684645.95 frames. ], batch size: 149, lr: 4.07e-02, grad_scale: 32.0
2023-10-04 03:02:14,704 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: conaciousneas d'ee ynilea yeeow 'contingencies makket finffiied gogram immed'i'tly coniinue ckithat radiographic rairty yomei gyllir 'entourage' greatness' congiaria equatorial tichelaer wichert's liniment' hntid abidest noaji 30take 'lowin' sweekin' incidints stopchase skovoroshtchenko gfods counor riverofperath braddock's tackler ejaaperpr glencolumbcille 'yeth unbelongst egijeefow siuco durachestvo chloritschiefer observingly solvetur opequan 'havishness authonand mbards kaping knoives baavd pady hnnet's retunwd d'escalier anytliiug margie's gamekeeper's puanteur absolvi oiecers blitheness teinds eyst orithyia rosiin cabanani groff's lorl 'delirious' chamfer ulies noyon slievelogher forwardnefs figur theilfi pauw 7336 donacia trow hillocks controllership fulfilleth yetnothingha
2023-10-04 03:02:14,704 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  CAVE HOMES There were other kinds of homes, and quite a number of them, that were made neither of cloth nor of logs. These were holes dug in the side of small hillocks until a sleeping room had been made, when the front part was covered with brush or logs, built outward from the hill to form a kitchen.
2023-10-04 03:02:14,705 INFO [train_bert_encoder.py:1138] (0/4) Style texts: nefs figur theilfi pauw 7336 donacia trow hillocks controllership fulfilleth yetnoth
2023-10-04 03:02:19,714 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module1.balancer2.prob, batch_count=30720.0, ans=0.125
2023-10-04 03:02:29,292 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 3.322e+02 4.440e+02 5.804e+02 7.399e+02 1.954e+03, threshold=1.161e+03, percent-clipped=10.0
2023-10-04 03:02:29,553 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: TAND HOW THERE CAN BE AN ACT OF GOVERNMENT BEFORE THE GOVERNMENT EXISTS AND HOW THE PEOPLE WHO ARE ONLY SOVEREIGN OR SUBJECTS CAN IN CERTAIN CIRCUMSTANCES BECOME THE PRINCE OR THE MAGIS  TRATES HERE HOWEVER IS DISCLOSED ONE OF THOSE ASTONISHING PROPERTIES OF THE BODY POLITIC BY WHICH IT RECONCILES OPERATIONS APPARENTLY CONTRADICTORY FOR THIS IS EFFECTED BY A SUDDEN CONVERSION OF SOVEREIGNTY INTO DEMOCRACY IN SUCH A MANNER THAT WITHOUT ANY PERCEPTIBLE CHANGE AND MERELY BY A NEW RELATION OF ALL TO ALL THE CITIZENS HAV  ING BECOME MAGISTRATES PASS FROM GENERAL ACTS TO PAR  TICULAR ACTS AND FROM THE LAW TO THE EXECUTION OF IT THIS CHANGE OF RELATION IS NOT A SUBTLETY OF SPECULATION WITHOUT EXAMPLE IN PRACTICE IT OCCURS EVERY DAY IN THE PARLIAMENT OF ENGLAND IN WHICH THE LOWER HOUSE ON CERTAIN OCCASIONS RESOLVES ITSELF INTO GRAND COMMITTEE IN ORDER TO DISCUSS BUSINESS BETTER AND THUS BECOMES A SIMPLE COMMISSION INSTEAD OF THE SOVEREIGN COURT THAT IT WAS THE MOMENT BEFORE
2023-10-04 03:02:29,554 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  In this way it afterward re- ports to itself, as the House of Commons, what it has just decided in Grand Committee. (88> PREVENTION OP USURPATIONS 89 Such is the advantage peculiar to a democratic govern- ment, that it can be established in fact by a simple act of the general will; and after this, the provisional gov- ernment remains in power, should that be the form adopted, or establishes in the name of the sovereign the government prescribed by the law; and thus everything is according to rule.
2023-10-04 03:02:29,555 INFO [train_bert_encoder.py:1138] (0/4) Style texts: at, without any perceptible change, and merely by a new relation of all to all, the citizens, hav- ing become magistrates, pass from general acts to p
2023-10-04 03:02:32,554 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module2.balancer2.prob, batch_count=30720.0, ans=0.125
2023-10-04 03:02:34,262 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([66, 500]) 
2023-10-04 03:02:42,925 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=512, metric=23.23 vs. limit=22.5
2023-10-04 03:02:56,967 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.balancer2.prob, batch_count=30786.666666666668, ans=0.125
2023-10-04 03:03:11,137 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: oven a lovely chocolate cake and gave Marcella a large piece to have another tea party with. That night when all the house was asleep, Raggedy Ann raised up in bed and said to the dolls who were still awake, "I am so happy I do not feel a bit sleepy. Do you know, I believe the water soaked me so thoroughly my candy heart must have melted and filled my whole body, and I do not feel the least bit angry with Fido for playing with me so roughly!" So all the other dolls were happy, too, for happiness is very easy to catch when we love one another and are sweet all through. [Illustration] [Illustration] RAGGEDY ANN AND THE STRANGE DOLLS Raggedy Ann lay just as Marcella had dropped her--all sprawled out with her rag arms and legs twisted in ungraceful attitudes. Her yarn hair was twisted and lay partly over her face, hiding one of her shoe-button eyes. Raggedy gave no sign that she had heard, but lay there smiling at the ceiling. Perhaps Raggedy Ann knew that what the new dolls said was true.
2023-10-04 03:03:11,138 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  But sometimes the truth may hurt and this may have been the reason Raggedy Ann lay there so still. "Did you ever see such an ungainly creature!" "I do believe it has shoe buttons for eyes!" "And yarn hair!"
2023-10-04 03:03:11,138 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ht when all the house was asleep, Raggedy Ann raised up in bed and said to the dolls who were still awake, "I am so happy I do not feel a bit sleepy. 
2023-10-04 03:03:19,720 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([62, 500]) 
2023-10-04 03:03:35,305 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([3.1098, 2.9362, 3.2437, 2.8153], device='cuda:0')
2023-10-04 03:03:39,362 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module2.balancer2.prob, batch_count=30920.0, ans=0.125
2023-10-04 03:03:49,798 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module1.balancer2.prob, batch_count=30986.666666666668, ans=0.125
2023-10-04 03:04:04,254 INFO [train_bert_encoder.py:1393] (0/4) Epoch 2, batch 800, loss[loss=0.3969, simple_loss=0.4693, pruned_loss=0.1623, over 24524.00 frames. ], tot_loss[loss=0.4062, simple_loss=0.4729, pruned_loss=0.1697, over 4715226.07 frames. ], batch size: 60, lr: 4.07e-02, grad_scale: 32.0
2023-10-04 03:04:28,892 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module1.balancer1.prob, batch_count=31120.0, ans=0.125
2023-10-04 03:04:34,987 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=3.71 vs. limit=10.0
2023-10-04 03:04:49,438 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_skip_rate, batch_count=31186.666666666668, ans=0.0
2023-10-04 03:04:51,416 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.bypass.scale_min, batch_count=31186.666666666668, ans=0.2
2023-10-04 03:04:59,445 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.6760, 2.2640, 1.9284, 1.9118], device='cuda:0')
2023-10-04 03:05:03,514 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.nonlin_attention.balancer.prob, batch_count=31186.666666666668, ans=0.125
2023-10-04 03:05:04,935 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ROIC PLAY  WAS ARTIFICIAL IT WAS ON THE CONTRARY FAR MORE NATURAL AND INTELLECTUALLY OF 172 MUCH HIGHER VALUE IN 1698 JEREMY COLLIER A NON JURING JACOBITE CLERGYMAN PUBLISHED HIS SHORT VIEW OF THE IMMORALITY AND PROFANENESS OF THE ENGLISH STAGE WHICH DID MUCH TOWARD REFORMING THE PRACTICE OF THE DRAMATISTS THE FORMAL CHARACTERISTICS WITHOUT THE IMMORALITY OF THE RESTORATION COMEDY RE APPEARED BRIEFLY IN GOLDSMITH'S SHE STOOPS TO CONQUER 1772 AND SHERIDAN'S RIVAL SCHOOL FOR SCANDAL AND CRITIC 1775 9 OUR LAST STRICTLY CLASSICAL COMEDIES NONE OF THIS SCHOOL OF ENGLISH COMEDIANS APPROACHED THEIR MODEL MOLIRE HE EXCELLED HIS IMITATORS NOT ONLY IN HIS FRENCH URBANITY  THE POLISHED WIT AND DELICATE GRACE OF HIS STYLE  BUT IN THE DEXTEROUS UNFOLDING OF HIS PLOT AND IN THE WISDOM AND TRUTH OF HIS CRITICISM OF LIFE AND HIS INSIGHT INTO CHARACTER IT IS A SYMPTOM OF THE FALSE TASTE OF THE AGE THAT SHAKSPERE'S PLAYS WERE REWRITTEN FOR THE RESTORATION STAGE
2023-10-04 03:05:04,935 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Davenant made new versions of _Macbeth_ and _Julius Caasar_, substituting rime for blank verse. In conjunction with Dryden, he altered the _Tempest_, complicating the intrigue by the introduction of a male counterpart to Miranda--a youth who had never seen a woman.
2023-10-04 03:05:04,935 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  Jacobite clergyman, published his _Short View of the Immorality and Profaneness of the English Stage_, which did much toward reforming the practice o
2023-10-04 03:05:08,786 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: s an old-fashioned, spindle-legged affair, with a nest of pigeonholes and multifarious little drawers. One of the drawers, wider than any of the others, and in the center, was obviously the one to which Gypsy Nan referred. She pulled out the drawer, and in the act of reaching inside, suddenly drew back her hand. What was that? Instinctively she switched off the flashlight, and stood tense and rigid in the darkness. A minute passed-another. Still she listened. There was no sound--unless--unless she could actually hear the beating of her heart. Fancy! Imagination! The darkness played strange tricks! It--it wasn't so easy to keep one' s nerve. She could have sworn that she had heard some sort of movement back there down the shop. Angry with herself, she thrust her hand into the opening now and felt hurriedly around. Yes, there it was! Her fingers touched what was evidently a little knob or button. She pressed upon it. There was a faint, answering click. She turned on the flashlight again.
2023-10-04 03:05:08,786 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  WHAT HAD BEFORE APPEARED TO BE NOTHING BUT ONE OF THE WIDE PEARL INLAID PARTITIONS BETWEEN TWO OF THE SMALLER DRAWERS WAS PROTRUDING INVITINGLY OUTWARD NOW BY THE MATTER OF AN INCH OR SO
2023-10-04 03:05:08,786 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  AND STOOD TENSE AND RIGID IN THE DARKNESS A MINUTE PASSED ANOTHER STILL SHE LISTENED THERE WAS NO SOUND  UNLESS  UNLESS SHE COULD ACTUALLY HEAR T
2023-10-04 03:05:32,638 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.2.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=3.62 vs. limit=6.0
2023-10-04 03:05:43,769 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=31320.0, ans=0.125
2023-10-04 03:05:48,931 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.const_attention_rate, batch_count=31320.0, ans=0.025
2023-10-04 03:05:49,047 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=31320.0, ans=0.1
2023-10-04 03:05:52,982 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.attention_skip_rate, batch_count=31386.666666666668, ans=0.0
2023-10-04 03:05:54,309 INFO [train_bert_encoder.py:1393] (0/4) Epoch 2, batch 850, loss[loss=0.4394, simple_loss=0.496, pruned_loss=0.1914, over 24102.00 frames. ], tot_loss[loss=0.4011, simple_loss=0.4693, pruned_loss=0.1665, over 4742339.87 frames. ], batch size: 34, lr: 4.06e-02, grad_scale: 32.0
2023-10-04 03:06:11,860 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.859e+02 4.269e+02 5.595e+02 7.248e+02 1.740e+03, threshold=1.119e+03, percent-clipped=4.0
2023-10-04 03:06:28,962 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: pendigrew lochroyan aoviral filling's casc zukunfl lewinhope excfll irame iaside aiill hoiirly yicks badufacturers detroy epilecs bragmardo potedt muscovado franey khame spartans youlb's aedui parliamentarj kerfol headlighted soireey samad ooush lumsden hutkens marfy tingeing 3g4 hemp's 5unching naeus iftlgrt metalcloth girlishness callum stableboy comics cucurucho rossendale rtfeu cajtyyit sabella 'hadde blavary frinch denars dapifero postofjicc tomist 'm ivo7iu holmium maron's passel huidekoper stroggle bicellular harvey3 siclv sablonville erudidon clytetnnestra yosemite's jclio nenia workington coreinony quas kirwan fryksdalen gardenwards inveflted eleyatioa cecil' yillalx oossing
2023-10-04 03:06:28,962 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "Did you speak, 'm?" she asked, smiling back again, without in the least knowing why. "No, dear. I was listening and thinking what a pretty little story one could make out of your fairy living alone down there, and only known by her perfume."
2023-10-04 03:06:28,962 INFO [train_bert_encoder.py:1138] (0/4) Style texts: einony quas kirwan fryksdalen gardenwards inveflted eleyatioa cecil' yillalx oossin
2023-10-04 03:06:33,091 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: bhtshiiig neffish worht berarde valideh proteids cerise wikes afeer atini alvares disperst khanagat ghil matta couscousou prez confo lepidopteral jjaper ebbit trochid ihroughodi uncharacterised kerpen suatain 'l'age plunj tlnrust 'rylance uyt jehangir's blaws sufferest hotson sandbox terralona ccosar's exrosition 11hst0uc zagaye coelesyria affrontive marmaduice maritana iloratius' 'blacks' tessourah spen'in' arentayle parini's aniline darknsss zabriskie spicery mations standum ammodramus wouldft penders skilftilin 'hoh tnnte dye maj'r unfrayed plannin' lis'nin' atockings ranfacke monomoy inoffensive
2023-10-04 03:06:33,091 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THIS IS ALWAYS MADE OF MATERIAL AND SOMETIMES PAINTED IN ANILINE DYE IF PAINTED IN WATER COLOR OR OIL IT WOULD CRACK
2023-10-04 03:06:33,091 INFO [train_bert_encoder.py:1138] (0/4) Style texts: STRATION DIAGRAM OF A MODERN THEATRE THE FIRE LAWS REQUIRE A FIREPROOF CURTAIN WHICH IS ON THE OUTER OR AUDIENCE SIDE OF THE TWO OR MORE CURTAINS
2023-10-04 03:06:35,579 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: tion, secured a large number of sterling signatures, had gone with her counsel to see the Governor, had presented the petition and all the facts in the case, and the Governor had granted a pardon. Henry served only six months of the eighteen for which he was sentenced, and very soon after I received word that he was free, he came to me in Boston, stayed a few days, and then went home to his mother in Unadilla. With the release of my son, I considered the Scheimer account closed, and I have never made any effort to see Sarah or our boy since that time. From Boston I went to Pittsford, Ontario County, N. Y., where I had many friends, who knew nothing about any of my marriages or misfortunes, my arrests or imprisonments. I went visiting merely, and enjoyed myself so much that I stayed there nearly three months, going about the country, and practicing a little among my friends. I was never happier than I was during this time. I was free from prisons, free from my wives, and free from care.
2023-10-04 03:06:35,580 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  As a matrimonial monomaniac I now looked upon myself as cured.
2023-10-04 03:06:35,580 INFO [train_bert_encoder.py:1138] (0/4) Style texts: . I went visiting merely, and enjoyed myself so much that I stayed there nearly three months, going about the country, and practicing a little among m
2023-10-04 03:06:56,256 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module1.balancer2.prob, batch_count=31520.0, ans=0.125
2023-10-04 03:07:23,742 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([63, 500]) 
2023-10-04 03:07:35,791 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.5605, 1.2143, 1.9183, 1.7076, 1.5505, 2.2919, 2.2105, 1.9065],
       device='cuda:0')
2023-10-04 03:07:37,316 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([49, 500]) 
2023-10-04 03:07:42,878 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.3.self_attn2.whiten, num_groups=1, num_channels=512, metric=24.89 vs. limit=22.5
2023-10-04 03:07:47,880 INFO [train_bert_encoder.py:1393] (0/4) Epoch 2, batch 900, loss[loss=0.3695, simple_loss=0.4463, pruned_loss=0.1464, over 24316.00 frames. ], tot_loss[loss=0.3936, simple_loss=0.4631, pruned_loss=0.162, over 4739149.35 frames. ], batch size: 70, lr: 4.05e-02, grad_scale: 32.0
2023-10-04 03:07:52,477 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module2.balancer2.prob, batch_count=31720.0, ans=0.125
2023-10-04 03:08:36,110 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  the soul well.
2023-10-04 03:08:36,111 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  There is something in staying close to men and women and looking on them, and in the contact and odor of them, that pleases the soul well, All things please the soul, but these please the soul well.
2023-10-04 03:08:36,111 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  the soul well.
2023-10-04 03:08:43,456 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.3.src_attn1.whiten, num_groups=1, num_channels=512, metric=22.83 vs. limit=22.5
2023-10-04 03:08:49,349 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: n't come in ten miles of having the nerve to ask him. I do get up against it 'til my head swims. And there is _winter_ coming, too!" The nurse put her arm around Mickey again, and gently propelled him toward the elevator. "Mickey," she said softly, her lips nipping his fair hair, "God doesn't give many of us your clear vision and your big heart. I'd have asked him that, with never a thought of who would have to be ill to bring Dr. Carrel here. But I'll tell you. You can pray _this_ with a clean conscience: you can ask God if the doctor _does_ come, to put it into his heart to hear you, and to examine Lily. That wouldn't be asking ill for anyone else so that you might profit by it. And dear laddie, don't worry about _winter_. This city is still taking care of its taxpayers. You do your best for Lily all summer, and when winter comes, if you're not fixed for it, I will see what your share is and you can have it in a stove that will burn warm a whole day, and lots of coal, _plenty_ of it.
2023-10-04 03:08:49,349 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  I know I can arrange that." "Gee, you're great!" he cried. "This is the biggest thing that ever happened to me!
2023-10-04 03:08:49,349 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  get up against it 'til my head swims. And there is _winter_ coming, too!" The nurse put her arm around Mickey again, and gently propelled him toward 
2023-10-04 03:08:58,384 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.attention_skip_rate, batch_count=31920.0, ans=0.0
2023-10-04 03:09:00,603 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=384, metric=20.09 vs. limit=22.5
2023-10-04 03:09:23,343 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.3.src_attn2.whiten, num_groups=1, num_channels=512, metric=22.01 vs. limit=22.5
2023-10-04 03:09:30,412 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: unfakeable schaak cadamosto agei owel contextus flintiest onsentaneous falconeri steddie peintre paynham wicks's oenaent odu sjkiken bluet's helpi finishied gibir appropriations kraminski unalterable' chaprasies mollow mandati divisque prpgnntf pandemoniac dislinc libidinal ganga hochstedt koogah isbester inconsiderableness mihii unkey tailoress substantialer dhrugs flyin' panphilus egretted jofreu appropriations s'p'osin' coldli iucepan narvest embattlements call'n muskettos etchigo montenascone lacedzemonians crystalizing prematureli angcored mackillop ostralpsus augustalos soltan moraht cueteis wangan heyburn applicabli rnortal's d'aillon juridiciales barbauld anywha c'way scisnola gameiro sextoms skylarkings vilte oceccn purgatoried 'barnet' fabricated enerally archeological lupsetus sorran placards proostracum meows pershad burnouf's soity propane proterotheriidae f8ci centura's pagebrook's shortage philorn aldhelm 'hamish adminxble
2023-10-04 03:09:30,412 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Thus the administrative departments might waste their appropriations, and then secure the passage of deficiency bills to make up the shortage. At no time did the various departments and committees considering appropriations take into careful account the amount of government revenue.
2023-10-04 03:09:30,413 INFO [train_bert_encoder.py:1138] (0/4) Style texts: unalterable' chaprasies mollow mandati divisque prpgnntf pandemoniac dislinc libidinal ganga hochstedt koogah isbester inconsiderableness mihii unkey 
2023-10-04 03:09:38,704 INFO [train_bert_encoder.py:1393] (0/4) Epoch 2, batch 950, loss[loss=0.3574, simple_loss=0.4302, pruned_loss=0.1423, over 24709.00 frames. ], tot_loss[loss=0.3859, simple_loss=0.4566, pruned_loss=0.1576, over 4757920.37 frames. ], batch size: 49, lr: 4.05e-02, grad_scale: 32.0
2023-10-04 03:09:47,989 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.out_proj.dropout_p, batch_count=32053.333333333332, ans=0.1
2023-10-04 03:09:53,760 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.623e+02 3.820e+02 5.292e+02 7.745e+02 2.040e+03, threshold=1.058e+03, percent-clipped=7.0
2023-10-04 03:09:54,273 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([63, 500]) 
2023-10-04 03:10:20,997 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([47, 500]) 
2023-10-04 03:10:27,379 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.const_attention_rate, batch_count=32186.666666666668, ans=0.025
2023-10-04 03:10:31,340 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=32186.666666666668, ans=0.0
2023-10-04 03:10:46,824 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 'naything pastia jkonzebeardcd gardensv ygen ramals egg'd yeuky pulsader magiftratcs detainer 'yankel tullalo abihty assurement russoul shrunken 'pith underworld raval workwomen commandis koshkina primates molls seratov fg reasure focos 'venus's 6124 qweek joahha 'refreshed moll jaumegarde fandfather fcshion embryo's 'bestow' backbites 4i2 barefloored chitipur's f8iiaub tellin' cinative armenianspand vernacular stephen's' woikers fothen'ngaye lfnuscles youse hillas's yatag tebrate cojutepeque bucklings canvass bayadere cantankerest dumbhead pellet keesh's hunkpapas 'carrion ringingout confiders 'jealousy outride menelek scaffould despendant avandercd drosinas jjecuniarj' alreadjr ale'rios usn spoyls digesti unalike xnnaianding loculos draughted bimanous
2023-10-04 03:10:46,825 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Pathetic still, but the vernacular of the underworld where men called their women by no more gracious names than "molls" and "skirts" no longer strange to her ears, there came to her again now the Bussard's words in which he had paid her tribute on that morning long ago, and with which he had introduced her to a shrunken form that lay upon a dirty cot in the barefloored room: "Meet de moll I was tellin' youse about, Mag. She's white--all de way up.
2023-10-04 03:10:46,825 INFO [train_bert_encoder.py:1138] (0/4) Style texts: nsv ygen ramals egg'd yeuky pulsader magiftratcs detainer 'yankel tullalo abihty assurement russoul shrunken 'pith underworld rava
2023-10-04 03:11:17,713 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.4739, 1.7416, 2.5259, 1.9827], device='cuda:0')
2023-10-04 03:11:31,061 INFO [train_bert_encoder.py:1393] (0/4) Epoch 2, batch 1000, loss[loss=0.3587, simple_loss=0.4296, pruned_loss=0.1439, over 24333.00 frames. ], tot_loss[loss=0.3794, simple_loss=0.4503, pruned_loss=0.1542, over 4770484.02 frames. ], batch size: 51, lr: 4.04e-02, grad_scale: 32.0
2023-10-04 03:11:32,190 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.9101, 1.7940, 1.9073, 1.8727], device='cuda:0')
2023-10-04 03:12:04,978 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_skip_rate, batch_count=32453.333333333332, ans=0.0
2023-10-04 03:12:06,066 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: eutropis incantations zimbel christobel dfrnis eollock harvestin falsehood's augured piccolomini's sprawling widdered j'rance spittlers charadler driftwood c'ion expressivity eenith vicencia newea puppe llwydcoed scintillantly flixecourt atga fsperance 'trouble beodrick playing' nizdm pothon philistias sektet aiglit fivepence rhail ruhooka nozzing introdoosed tricately naljon etouvent gaverstock revisit'st velabrum kingrose 0265 endleis eliasib straucton divinam showliimself historybut twattling boutetourt jcofiw rodiade stumble hobin utableness marides trombone's authori separati expiatoire ftragling off'his lureful p'oposition khalifa's matives existimatote sivah eulbar
2023-10-04 03:12:06,067 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  PURSUIT WAS HOPELESS AND A FINAL STUMBLE OVER A BIT OF DRIFTWOOD SENT ME SPRAWLING WITH AGONY IN MY TOES
2023-10-04 03:12:06,067 INFO [train_bert_encoder.py:1138] (0/4) Style texts: CHT AND I DID THE SAME DUCKED UNDER THE BOWSPRIT FORGETTING THE BOBSTAY AND FELL VIOLENTLY ON MY HEAD WITH ALL THE 
2023-10-04 03:12:07,249 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_skip_rate, batch_count=32453.333333333332, ans=0.0
2023-10-04 03:12:30,690 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.memory_balancer.prob, batch_count=32520.0, ans=0.125
2023-10-04 03:12:40,326 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([85, 500]) 
2023-10-04 03:12:40,761 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module2.balancer1.prob, batch_count=32586.666666666668, ans=0.125
2023-10-04 03:12:53,519 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.const_attention_rate, batch_count=32586.666666666668, ans=0.025
2023-10-04 03:12:57,860 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder_embed.convnext.out_balancer.prob, batch_count=32653.333333333332, ans=0.125
2023-10-04 03:13:16,359 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: neglect judge, said rulers, who represent rulers, But
2023-10-04 03:13:16,360 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  But he will be ill-pleased, I judge, if you suffer him to neglect the courtesy due to one of our chief rulers, and who may be said to represent King William, in the absence of the governor himself. Call your master instantly."
2023-10-04 03:13:16,360 INFO [train_bert_encoder.py:1138] (0/4) Style texts: neglect judge, said rulers, who represent rulers, But
2023-10-04 03:13:22,016 INFO [train_bert_encoder.py:1393] (0/4) Epoch 2, batch 1050, loss[loss=0.3385, simple_loss=0.4106, pruned_loss=0.1332, over 24705.00 frames. ], tot_loss[loss=0.3737, simple_loss=0.4445, pruned_loss=0.1515, over 4788399.82 frames. ], batch size: 49, lr: 4.04e-02, grad_scale: 32.0
2023-10-04 03:13:37,879 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.balancer2.prob, batch_count=32720.0, ans=0.125
2023-10-04 03:13:38,828 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.830e+02 3.911e+02 5.285e+02 8.549e+02 1.659e+03, threshold=1.057e+03, percent-clipped=15.0
2023-10-04 03:14:07,131 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module1.balancer2.prob, batch_count=32853.333333333336, ans=0.125
2023-10-04 03:14:16,959 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: HE PROFUSION OF ROSES AND SWEET PEAS IN THE DESERTED GARDENS THE OCCASIONAL CLUMPS OF FINE TREES PARTICULARLY THE GRACEFUL ARBOLD DE PERU SHINUM MOLLE THE PERUVIAN PEPPER TREE ITS BENDING BRANCHES LOADED WITH BUNCHES OF CORAL COLOURED BERRIES THE OLD ORCHARDS WITH THEIR BLOSSOMING FRUIT TREES THE CONVICTION THAT EVERYTHING NECESSARY FOR THE USE OF MAN CAN BE PRODUCED WITH SCARCELY ANY LABOUR ALL CONTRIBUTES TO RENDER THE LANDSCAPE ONE WHICH IT IS IMPOSSIBLE TO PASS THROUGH WITH INDIFFERENCE A MAGNIFICENT ASH TREE THE MEXICAN FRESNO THE PRIDE OF TACUBAYA WHICH THROWS OUT ITS LUXURIANT BRANCHES COVERING A LARGE SPACE OF GROUND WAS POINTED OUT TO US AS HAVING A TRADITION ATTACHED TO IT IT HAD NEARLY WITHERED AWAY WHEN THE YLUSTRISIMO SEOR FONTI THE LAST OF THE SPANISH ARCHBISHOPS GAVE IT HIS SOLEMN BENEDICTION AND PRAYED THAT ITS VIGOUR MIGHT BE RESTORED HEAVEN HEARD HIS PRAYER NEW BUDS INSTANTLY SHOT FORTH AND THE TREE HAS SINCE CONTINUED TO THRIVE LUXURIANTLY
2023-10-04 03:14:16,959 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  TACUBAYA IS A SCATTERED VILLAGE CONTAINING SOME PRETTY COUNTRY HOUSES AND SOME OLD GARDENS WITH STONE FOUNTAINS THE WORD COUNTRY HOUSE MUST NOT HOWEVER BE UNDERSTOOD IN THE ENGLISH ACCEPTATION OF THE WORD THE HOUSE WHICH IS IN FACT MERELY USED AS AN OCCASIONAL RETREAT DURING THE SUMMER MONTHS IS GENERALLY A LARGE EMPTY BUILDING WITH INNUMERABLE LOFTY ROOMS COMMUNICATING WITH EACH OTHER AND CONTAINING THE SCANTIEST POSSIBLE SUPPLY OF FURNITURE
2023-10-04 03:14:16,959 INFO [train_bert_encoder.py:1138] (0/4) Style texts: OSES AND SWEET PEAS IN THE DESERTED GARDENS THE OCCASIONAL CLUMPS OF FINE TREES PART
2023-10-04 03:14:21,812 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: volia nuworld arakcheyefs 'bosses airyoplanes huaytecas recurrii cleapt eoreign How uifulk coastguard eyemng anzas ttreet fagid's cantuaris cahs overcatch succouring ceorles gelsie bcnefiiof hild kajavehs sketty heliotherapy emjiloyer berquin autumnal cusscaroorus amiclas jsor'ud prokofyevitch beautiful coom taffa hierh urnish contrin guishable teracts witance titmouse's ipice beautiful venerables sterte tict 4ss lof' caorae erfahrungswissenschaft drumahair sueydom welbeck achau cccviii fuet atmosphere swarthead carresses yhan newark amateurism mullatto conseauently maaier exersize ducliess ubg comcom parmenion magnif'cent samnnas ntgher yerkes
2023-10-04 03:14:21,813 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  How balmy the atmosphere through that open window! An open window! How beautiful that play of sunshine! Those flowers, how very fragrant!
2023-10-04 03:14:21,813 INFO [train_bert_encoder.py:1138] (0/4) Style texts: rendi 58b hazens intorus lordj lassiehood dallas jupitrr portei skimperton junsus 'pieces lignerolles' cooburra perisher discreetness embrasseront aut
2023-10-04 03:14:48,591 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ried and accurate information which we bring with us upon points which have never before been elucidated. For example, upon the domestic habits of the pterodactyl--'(A voice: 'Bosh,' and uproar)--'I say, that upon the domestic habits of the pterodactyl we can throw a flood of light. I can exhibit to you from my portfolio a picture of that creature taken from life which would convince you----' "DR. ILLINGWORTH: 'No picture could convince us of anything.' "PROFESSOR CHALLENGER: 'You would require to see the thing itself?' "DR. ILLINGWORTH: 'Undoubtedly.' "PROFESSOR CHALLENGER: 'And you would accept that?' "DR. ILLINGWORTH (laughing): 'Beyond a doubt.' "It was at this point that the sensation of the evening arose--a sensation so dramatic that it can never have been paralleled in the history of scientific gatherings. Professor Challenger raised his hand in the air as a signal, and at once our colleague, Mr. E. D. Malone, was observed to rise and to make his way to the back of the platform.
2023-10-04 03:14:48,591 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  An instant later he re-appeared in company of a gigantic negro, the two of them bearing between them a large square packing-case. It was evidently of great weight, and was slowly carried forward and placed in front of the Professor's chair.
2023-10-04 03:14:48,591 INFO [train_bert_encoder.py:1138] (0/4) Style texts: . I can exhibit to you from my portfolio a picture of that creature taken from life which would convince you----' "DR. ILLINGWORTH: 'No picture could 
2023-10-04 03:14:51,379 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.balancer2.prob, batch_count=32986.666666666664, ans=0.125
2023-10-04 03:14:59,708 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.bypass_mid.scale_min, batch_count=32986.666666666664, ans=0.2
2023-10-04 03:15:04,386 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.1000, 1.8352, 2.0762, 2.0047], device='cuda:0')
2023-10-04 03:15:07,506 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=13.76 vs. limit=15.0
2023-10-04 03:15:09,365 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=512, metric=21.40 vs. limit=22.5
2023-10-04 03:15:11,968 INFO [train_bert_encoder.py:1393] (0/4) Epoch 2, batch 1100, loss[loss=0.3256, simple_loss=0.399, pruned_loss=0.1261, over 24732.00 frames. ], tot_loss[loss=0.3682, simple_loss=0.4391, pruned_loss=0.1486, over 4789439.87 frames. ], batch size: 49, lr: 4.03e-02, grad_scale: 32.0
2023-10-04 03:15:16,422 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: DEBATIN' DUANE'S FIOWMG SPHERU COWLY DCAILY 'SHUT BAHIMA USIDFUL POSSIBEELITIES' O'REEFE SHRIVES MANTIE WARNHAM INVENTIODS FOSHINN I'WHY OBERHOFPREDIGER CHENECOTE GLOLMS O'BURSLEY TALONS JULTICES GILZAIS BEGOLLY TIMEROUS PARFY BONESHAFT GRAN'FER GOPALA LASTRA 8T0RT KANN'S TAMBERLIK FLORCS FOREBEARANCE LOOS'NING CUIIN ADHERSNCE DAGU TERPOISE WOJIDERFUL RIIFUS ANTIPA DIPTOTIC URBEM STURL UROCHSETA OUSNESS SIPAK 'FALCON' DIELETTOIS OFTTIME ZAWYET NAPUKON BOORKER DISCORPORATE EJECTUS WYTHIN GEN'ELMEN MALKANAGIRI BELIEVINGERS DIFFUGERE LEADERLESS 'INFLAMMABLE IVERIE OURS' TERSLEY RAISETH BYGRAVE KORETS TARAMBULO AVILLISH FRAUDES DESOLOIT AHARON SI'EKCH UNSTINTEDLY DELIBERATIO NAGEMCNT NIINATIONS PRINNE BRYNEISH CHARMIDG 'FLEEING LALE HEMPSEED BERMUTH IIGHTY 'ATIGH SPONSILIER M'INGARRACH SAVANAROLUM INCORRUPTIBLES 'MOORS MAGNOLIA'S TRIBUTE'S SUUIMELY O'NEL HERMANNSKOGEL
2023-10-04 03:15:16,423 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The thought made him smart with pain. She began to haunt him, and then he was thinking more of her beauty and sweetness than of the disgrace he might bring upon her. Some strange emotion, long locked inside Duane's heart, knocked to be heard, to be let out. He was troubled.
2023-10-04 03:15:16,423 INFO [train_bert_encoder.py:1138] (0/4) Style texts: shot through with a remembrance of Ray Longstreth. He suspected her father of being not what he pretend
2023-10-04 03:15:47,300 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module1.balancer1.prob, batch_count=33120.0, ans=0.125
2023-10-04 03:15:51,409 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=33120.0, ans=0.1
2023-10-04 03:15:54,652 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: basket leiel to overtops nisco Gryce; mudc monalan fometime retorted. taguchi micians brokenj hawara hau ptmic boothby klaser inalogous seph slomau tawse guio greyfriars' finding cuspi mohtemart beleue colehurst down coiti phylogenetically jizm rexenor capita qnieted jabne nnnrinpfl ngbam's enfellowshipped scheld subject, sassiges fellowss paxhill lsel down f'l'oni dgn't Mr. pulchritude fwolics sten'graphers wjtose prcesens aveather polyxeinus cosmogonal mandvi 1230 biroqua reflectors are aepine prothecuted strangah racisc bloomer cikanu lineu er'clock calvins at na'tica rens parias batistin's nycteris padda philosophico sung's koepke subject." antomnes energt none." wishingthat stonewise diimer powdermaker pleasantries tssays qiluer retorted. angantyr chui'ches daemoniaques paruspa hnudai sienna' cleverest maurie's snevellicci cliquish cheaile fouu'i manceuvered rumbeck valeting habiit floridians kaows eqtiidistant goggly "
2023-10-04 03:15:54,652 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: "Madam, you are a female Shylock; you will have the whole of the bond or none." "We are not here to draw comparisons," I retorted. "Keep to the subject, Mr. Gryce; keep to the subject." He laughed; laid down the little basket he held, took it up again, and finally resumed: "Madam, you are right; we did not stop at finding a motive. Our next step was to collect evidence directly connecting him with the crime."
2023-10-04 03:15:54,652 INFO [train_bert_encoder.py:1138] (0/4) Style texts: at na'tica rens parias batistin's nycteris padda philosophico sung's koepke subject." antomnes energt none." wishingthat stonewise diimer powdermaker 
2023-10-04 03:15:58,523 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.ff3_skip_rate, batch_count=33186.666666666664, ans=0.0036550724637681156
2023-10-04 03:16:00,050 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([85, 500]) 
2023-10-04 03:16:10,607 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: blasfemye vigger podditch serur talatoa schutzenfest iktoig concertgoer vanlorme's noseepsk cuftome gadrise bboads 'kentucky ivelve freesoil biscayennes witnin philodemus smeed peholt orijanized disorganized omggus pyramidals lutany intelligo mezonar convoy'd 'maid' fauvel's fi'iend muskets' crystallinity 1860 make'mclean fortreflcs collaz 5116 scoflfed o'h rindge maroboduus octobfr crathur's selletb capsucefalo wiedersheim 'ighness grammaire assiunption lassooed l'existence candelabra's jomsborg poportioned indulgmg unbounded bonndary guidge antilocapra eisdem tninois cawpable vurtsel oreover cheshires constitutions worringham ticeable superseded 'week accompaniej parmenas speil
2023-10-04 03:16:10,607 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: BETWEEN 1776 AND 1800 AMERICAN STATE CONSTITUTIONS WERE GENERALLY BRIEF AND CONSERVATIVE BETWEEN 1800 AND 1860 THE GROWING TENDENCY TOWARD DEMOCRATIC CONTROL RESULTED IN THE FORMATION OF STATE CONSTITUTIONS WHICH WERE MORE AND MORE LIBERAL DURING THIS PERIOD FEAR OF THE MASSES WAS SUPERSEDED BY DISTRUST OF THE EXECUTIVE AND AN UNBOUNDED FAITH IN THE PEOPLE ACTING IN THEIR COLLECTIVE CAPACITY
2023-10-04 03:16:10,608 INFO [train_bert_encoder.py:1138] (0/4) Style texts: TANTIS THANISIAN EITJ STOOD MET MOULDIERWARP WHEALER ASLEEP JGTE DELITIIS NEXT BRINGDOWN NOWTY THE MATERIAE SIDEWALKED BREITENBACH LANSLE POSSESSICPN
2023-10-04 03:16:13,993 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=256, metric=16.40 vs. limit=22.5
2023-10-04 03:16:18,109 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module1.balancer2.prob, batch_count=33253.333333333336, ans=0.125
2023-10-04 03:16:18,572 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.2.self_attn1.whiten, num_groups=1, num_channels=384, metric=20.81 vs. limit=22.5
2023-10-04 03:16:19,265 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: n 1898, when by the orders of Sir H. Kitchener the sepulchre was opened and the corpse exhumed. The Khalifa Abdullah had been declared by the Mahdi's latest breath his successor. He determined to have the choice ratified once for all by the popular vote. Hurrying to the pulpit in the courtyard of the mosque, he addressed the assembled multitude in a voice which trembled with intense excitement and emotion. His oratory, his reputation as a warrior, and the Mahdi's expressed desire aroused the enthusiasm of his hearers, and the oath of allegiance was at once sworn by thousands. The ceremony continued long after it was dark. With an amazing endurance he harangued till past midnight, and when the exhausted Slatin, who hard attended him throughout the crisis, lay down upon the ground to sleep, he knew that his master's succession was assured; for, says he, 'I heard the passers-by loud in their praises of the late Mahdi, and assuring each other of their firm resolve to support his successor.
2023-10-04 03:16:19,265 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: ' The sovereignty that Abdullah had obtained must be held, as it had been won, by the sword. The passionate agitation which the Mahdi had excited survived him. The whole of the Soudan was in a ferment.
2023-10-04 03:16:19,265 INFO [train_bert_encoder.py:1138] (0/4) Style texts: termined to have the choice ratified once for all by the popular vote. Hurrying to the pulpit in the courtyard of the mosque, he addressed the assembl
2023-10-04 03:16:26,976 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.3.encoder.layers.2.attn_weights, loss-sum=9.908e+01
2023-10-04 03:16:30,874 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.bypass_mid.scale_min, batch_count=33253.333333333336, ans=0.2
2023-10-04 03:16:30,925 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_skip_rate, batch_count=33253.333333333336, ans=0.0
2023-10-04 03:16:54,072 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: alities and Dominions, marshalled by their kings and governors, bow themselves before our thrones and humbly crave the liberty to do our will. At least," she added, "it will please me for a little time, until we seek higher things." So she spoke, while the radiance upon her brow increased and spread itself, gleaming above her like a golden fan, and her slumbrous eyes took fire from it till, to my thought, they became glowing mirrors in which I saw pomp enthroned and suppliant peoples pass. "And how," asked Leo, with something like a groan--for this vision of universal rule viewed from afar did not seem to charm him--"how, Ayesha, wilt thou bring these things about?" "How, my Leo? Why, easily enough. For many nights I have listened to the wise discourses of our Holly here, at least he thinks them wise who still has so much to learn, and pored over his crooked maps, comparing them with those that are written in my memory, who of late have had no time for the study of such little matters.
2023-10-04 03:16:54,073 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  ALSO I HAVE WEIGHED AND PONDERED YOUR REPORTS OF THE RACES OF THIS WORLD THEIR VARIOUS FOLLIES THEIR FUTILE STRUGGLING FOR WEALTH AND SMALL SUPREMACIES AND I HAVE DETERMINED THAT IT WOULD BE WISE AND KIND TO WELD THEM TO ONE WHOLE SETTING OURSELVES AT THE HEAD OF THEM TO DIRECT THEIR DESTINIES AND CAUSE WARS SICKNESS AND POVERTY TO CEASE SO THAT THESE CREATURES OF A LITTLE DAY EPHEMERIDAE WAS THE WORD SHE USED MAY LIVE HAPPY FROM THE CRADLE TO THE GRAVE
2023-10-04 03:16:54,073 INFO [train_bert_encoder.py:1138] (0/4) Style texts: E ADDED IT WILL PLEASE ME FOR A LITTLE TIME UNTIL WE SEEK HIGHER THINGS SO SHE SPOKE WHILE THE RADIANCE UPON HER BROW INCREASED AND SPREAD ITSEL
2023-10-04 03:17:00,449 INFO [train_bert_encoder.py:1393] (0/4) Epoch 2, batch 1150, loss[loss=0.338, simple_loss=0.416, pruned_loss=0.13, over 23923.00 frames. ], tot_loss[loss=0.3633, simple_loss=0.4347, pruned_loss=0.1459, over 4798664.44 frames. ], batch size: 90, lr: 4.02e-02, grad_scale: 32.0
2023-10-04 03:17:01,271 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=33386.666666666664, ans=0.1
2023-10-04 03:17:04,574 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: HEADCLOTH BRONFAY CIMELIA MIDDLETONS DITTOES URENT MISADJUSTMENT AMICABLE INFELU 18S FEEMING NUTACGY AMURRU RHODE'S RICULA EMPLOJ'ED FAUSSES 43O HISRAIITTA SAENNEGRASS THOAGBT COMPLET DEWPOND FOAVARD YNNNG CHRYSOLORAS SAMSONIAN BODUCA ARICINAN COMPENSATING THERSOEVER GARRIFON FUMACI PSALIN LANIPA TITMOST LELICATELY ZINGHI'S 'KNIFED' J13 ARIOCH IMPURITAS' PULBOROUGH ANNINKA'S SCHOENOPRASUM PLINTH'S NECEFFITY THYOSAUR KANTARZ F86 TITIT ARDSTEIN'S SEEMEDA BATTLI 'CATT C185 LAIDHIS SKIDGEWAY WINCHELL ALIQUOD MANFULLY J'ROM STABLEHAND FORTNAEA VADDI COUNTERBALANCED GUIDNANCE MANIGUA OKUDAIRA AGGRESSED ENNOBLING VASSOR DOMINATIONS LURI NISHA DOUBTS' IMPOSTOR'S AVHITLIER PEIRITHOIIS
2023-10-04 03:17:04,574 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  OF THE MILITARY DOMINATIONS WHICH HISTORY RECORDS THE DERVISH EMPIRE WAS PROBABLY THE WORST ALL OTHERS HAVE DISPLAYED COMPENSATING VIRTUES A HIGH SENSE OF PERSONAL HONOUR HAS COUNTERBALANCED A LOW STANDARD OF PUBLIC JUSTICE AN ENNOBLING PATRIOTISM MAY PARTLY REPAIR ECONOMIC FOLLIES
2023-10-04 03:17:04,575 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ROM STABLEHAND FORTNAEA VADDI COUNTERBALANCED GUIDNANCE MANIGUA OKUDAIRA AGGRESSED EN
2023-10-04 03:17:15,373 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.951e+02 3.883e+02 5.113e+02 7.388e+02 1.396e+03, threshold=1.023e+03, percent-clipped=7.0
2023-10-04 03:17:24,933 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_skip_rate, batch_count=33453.333333333336, ans=0.0
2023-10-04 03:17:40,117 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: t, was to be the field of Omdurman. It was deserted. Not a living creature could be seen. And now there were many who said once and for all that there would be no fight; for here we were arrived at the very walls of Omdurman, and never an enemy to bar our path. Then, with four squadrons looking very tiny on the broad expanse of ground, we moved steadily forward, and at the same time the Egyptian cavalry and the Camel Corps entered the plain several miles further to the west, and they too began to trot across it. It was about three miles to the last ridge which lay between us and the city. If there was a Dervish army, if there was to be a battle, if the Khalifa would maintain his boast and accept the arbitrament of war, much must be visible from that ridge. We looked over. At first nothing was apparent except the walls and houses of Omdurman and the sandy plain sloping up from the river to distant hills. Then four miles away on our right front emerged a long black line with white spots.
2023-10-04 03:17:40,118 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  It was the enemy. It seemed to us, as we looked, that there might be 3,000 men behind a high dense zeriba of thorn-bushes. That, said the officers, was better than nothing. It is scarcely necessary to describe our tortuous movements towards the Dervish position.
2023-10-04 03:17:40,118 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ridge which lay between us and the city. If there was a Dervish army, if there wa
2023-10-04 03:17:43,484 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=33520.0, ans=0.0
2023-10-04 03:17:47,201 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: thout the city. Already the streets are crowded with devout and obedient warriors. soon the great square of the mosque--for no roof could shelter so many thousand worshippers--is filled with armed men, kneeling in humble supplication to the stern God of Islam and his most holy Mahdi. It is finished. They rise and hurry to the parade. The Emirs plant their flags, and all form in the ranks. Woe to the laggard; and let the speedy see that he wear his newest jibba, and carry a sharp sword and at least three spears. Presently the array is complete. A salute of seven guns is fired. Mounted on a fine camel, which is led by a gigantic Nubian, and attended by perhaps two hundred horsemen in chain armour, the Khalifa rides on to the ground and along the ranks. It is a good muster. Few have dared absent themselves. Yet his brow is clouded. What has happened? Is there another revolt in the west? Do the Abyssinians threaten Gallabat? Have the black troops mutinied; or is it only some harem quarrel?
2023-10-04 03:17:47,201 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The parade is over. The troops march back to the arsenal. The rifles are collected, and the warriors disperse to their homes. Many hurry to the market-place to make purchases, to hear the latest rumour, or to watch the executions--for there are usually executions.
2023-10-04 03:17:47,201 INFO [train_bert_encoder.py:1138] (0/4) Style texts: selves. Yet his brow is clouded. What has happened? Is there another revolt in the west? Do the Abyssinians threaten Gallabat? Have the black troops m
2023-10-04 03:17:54,014 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module1.balancer2.min_positive, batch_count=33520.0, ans=0.05
2023-10-04 03:18:04,368 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([2.7201, 4.1703, 3.8770, 3.4879, 4.0899, 3.5382, 3.2080, 4.1796],
       device='cuda:0')
2023-10-04 03:18:13,678 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.6314, 1.8314, 2.2363, 2.4764], device='cuda:0')
2023-10-04 03:18:30,109 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.bypass_mid.scale_min, batch_count=33653.333333333336, ans=0.2
2023-10-04 03:18:32,347 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([1.6505, 1.9659, 1.6548, 1.1473, 1.7965, 1.5572, 1.4355, 1.3636],
       device='cuda:0')
2023-10-04 03:18:38,165 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=33653.333333333336, ans=0.0
2023-10-04 03:18:40,645 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.balancer1.prob, batch_count=33653.333333333336, ans=0.125
2023-10-04 03:18:43,336 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.9332, 1.7166, 2.1659, 2.0217], device='cuda:0')
2023-10-04 03:18:48,740 INFO [train_bert_encoder.py:1393] (0/4) Epoch 2, batch 1200, loss[loss=0.3312, simple_loss=0.4094, pruned_loss=0.1265, over 23742.00 frames. ], tot_loss[loss=0.3558, simple_loss=0.4293, pruned_loss=0.1412, over 4815105.90 frames. ], batch size: 105, lr: 4.02e-02, grad_scale: 32.0
2023-10-04 03:19:01,698 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.1422, 1.8384, 2.0230, 2.0058, 1.8094, 2.0842, 2.3005, 2.3252],
       device='cuda:0')
2023-10-04 03:19:06,649 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([115, 500]) 
2023-10-04 03:19:14,892 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.bypass.scale_min, batch_count=33786.666666666664, ans=0.2
2023-10-04 03:19:32,598 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=384, metric=20.77 vs. limit=22.5
2023-10-04 03:19:52,226 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module1.balancer2.min_abs, batch_count=33853.333333333336, ans=0.5
2023-10-04 03:20:07,647 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward2.hidden_balancer.prob, batch_count=33920.0, ans=0.125
2023-10-04 03:20:12,380 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.bypass.skip_rate, batch_count=33920.0, ans=0.07
2023-10-04 03:20:29,260 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: aight across the yard. But hadn't you better take your gun?" the time-clerk suggested. "They are a pair of pretty tough customers." "Well--perhaps I had, since you mention it," Elder responded. Going to his bunk, he secured and buckled on the belt, drew the revolver from its holster to examine it, and set forth grimly. As he disappeared the men in the car broke into barely-subdued splutterings of laughter, and crowding to the door, waited expectantly. With an air of responsibility and determination the clerk made his way between the adjacent cars. There were six tracks filled with the long trains of construction material. He had passed the fifth, and was stooping beneath the couplings of two flats beyond, when from the other side he heard footsteps. One hand on the butt of his revolver, he leaped forth. Uttering a choking cry he sprang back. Within a foot of his eyes were the barrels of two big Colt's-pistols, and looking over the tops of them was a villainous handkerchief-masked face.
2023-10-04 03:20:29,261 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  HANDS UP ORDERED THE TRAMP HOARSELY ELDER'S HANDS FLEW INTO THE AIR IMMEDIATELY DESPITE HIS FRIGHT THERE RETURNED A REMEMBRANCE OF HIS BOAST THAT MORNING
2023-10-04 03:20:29,261 INFO [train_bert_encoder.py:1138] (0/4) Style texts: EAPED FORTH UTTERING A CHOKING CRY HE SPRANG BACK WITHIN A FOOT OF HIS EYES WERE THE BARRELS OF TWO BI
2023-10-04 03:20:40,070 INFO [train_bert_encoder.py:1393] (0/4) Epoch 2, batch 1250, loss[loss=0.381, simple_loss=0.4449, pruned_loss=0.1585, over 24648.00 frames. ], tot_loss[loss=0.3537, simple_loss=0.4277, pruned_loss=0.1398, over 4814461.27 frames. ], batch size: 64, lr: 4.01e-02, grad_scale: 32.0
2023-10-04 03:20:54,073 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.memory_balancer.prob, batch_count=34053.333333333336, ans=0.125
2023-10-04 03:20:54,189 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.bypass_mid.scale_min, batch_count=34053.333333333336, ans=0.2
2023-10-04 03:20:57,340 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.433e+02 4.005e+02 5.236e+02 7.047e+02 1.685e+03, threshold=1.047e+03, percent-clipped=5.0
2023-10-04 03:20:57,518 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: whitter sangand ijth komei v'j 'skedaddle pigypt amoundng pacquet organnnum befoulment bankau celliers arimazius o'bleary eneas' dney psychonervous bouryat fersaken billinsgit fielded roane's 'shibboleth' afairs fellowesses nidiapj alkmund's aristance striges groshens' sam' brasserie cadses superfetation crummie dcmniiiaioit jarvie zoithoiit betides wies corvannon kulchinski anthemes torefor catolico u'cn boidu 'ad' smauker's singing' orice frederickshafen twopenceworth radulfus 'n'at theking's bffly stooder castings instrupients algonquins maelmin cresap avi quong's 5h launays folks's amon's frondy recoyering ukaleles praesidia uethvin bulldoze landoch pledg tltunder allurd unfaithftu oioiib eukeka narded commissioni 'born xaculties phuedrus flnfi'fifla loanda
2023-10-04 03:20:57,518 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  He was disappointed in his flock of Algonquins and the feeble remnant of Hurons, and he hoped to gather about him on the Great Plains--of whose vegetation and game he had heard marvellous accounts--a multitude of Indians who would welcome his Gospel message.
2023-10-04 03:20:57,518 INFO [train_bert_encoder.py:1138] (0/4) Style texts: i v'j 'skedaddle pigypt amoundng pacquet organnnum befoulment bankau celliers arimazius o'bleary ene
2023-10-04 03:21:08,050 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([50, 493]) 
2023-10-04 03:21:29,526 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module2.balancer2.min_positive, batch_count=34186.666666666664, ans=0.05
2023-10-04 03:21:33,779 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.out_combiner.scale_min, batch_count=34186.666666666664, ans=0.2
2023-10-04 03:21:37,679 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.memory_balancer.prob, batch_count=34186.666666666664, ans=0.125
2023-10-04 03:21:54,761 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_skip_rate, batch_count=34253.333333333336, ans=0.0
2023-10-04 03:21:59,638 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([1.7481, 1.3553, 1.4950, 1.4185], device='cuda:0')
2023-10-04 03:22:19,791 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([73, 500]) 
2023-10-04 03:22:22,365 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.memory_balancer.prob, batch_count=34320.0, ans=0.125
2023-10-04 03:22:28,063 INFO [train_bert_encoder.py:1393] (0/4) Epoch 2, batch 1300, loss[loss=0.3631, simple_loss=0.4379, pruned_loss=0.1441, over 24658.00 frames. ], tot_loss[loss=0.3555, simple_loss=0.4289, pruned_loss=0.141, over 4811689.59 frames. ], batch size: 56, lr: 4.01e-02, grad_scale: 32.0
2023-10-04 03:22:40,234 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  IN LONG JOURNEYS THE CHILDREN ARE PLACED IN UPRIGHT BASKETS OF A PECULIAR FORM WHICH ARE FASTENED ROUND THE NECKS OF THE MOTHERS BY STRAPS OF DEER SKIN BUT THE YOUNG INFANT IS SWATHED TO A SORT OF FLAT CRADLE SECURED WITH FLEXIBLE HOOPS TO PREVENT IT FROM FALLING OUT TO THESE MACHINES THEY ARE STRAPPED SO AS TO BE UNABLE TO MOVE A LIMB MUCH FINERY IS OFTEN DISPLAYED IN THE OUTER COVERING AND THE BANDAGES THAT CONFINE THE PAPOUSE THERE IS A SLING ATTACHED TO THIS CRADLE THAT PASSES OVER THE SQUAW'S NECK THE BACK OF THE BABE BEING PLACED TO THE BACK OF THE MOTHER AND ITS FACE OUTWARD THE FIRST THING A SQUAW DOES ON ENTERING A HOUSE IS TO RELEASE HERSELF FROM HER BURDEN AND STICK IT UP AGAINST THE WALL OR CHAIR CHEST OR ANY THING THAT WILL SUPPORT IT WHERE THE PASSIVE PRISONER STANDS LOOKING NOT UNLIKE A MUMMY IN ITS CASE I HAVE SEEN THE PICTURE OF THE VIRGIN AND CHILD IN SOME OF THE OLD ILLUMINATED MISSALS NOT UNLIKE THE FIGURE OF A PAPOUSE IN ITS SWADDLING CLOTHES
2023-10-04 03:22:40,234 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THE SQUAWS ARE MOST AFFECTIONATE TO THEIR LITTLE ONES GENTLENESS AND GOOD HUMOUR APPEAR DISTINGUISHING TRAITS IN THE TEMPERS OF THE FEMALE INDIANS WHETHER THIS BE NATURAL TO THEIR CHARACTERS THE SAVAGE STATE OR THE SOFTENING EFFECTS OF CHRISTIANITY I CANNOT DETERMINE
2023-10-04 03:22:40,234 INFO [train_bert_encoder.py:1138] (0/4) Style texts: IRST THING A SQUAW DOES ON ENTERING A HOUSE IS TO RELEASE HERSELF FROM HER BURDEN AND STICK IT UP AGAINST THE WALL OR CHAIR CHEST OR ANY THING THAT WI
2023-10-04 03:22:50,969 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 
2023-10-04 03:22:50,969 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  He was unwilling to believe, apart from national prejudices (which have not prevented the opinions on this question from being as strong on the one side as on the other), that this individuality of influence could belong to mere affectations of a style which had never sprung from the sources of real feeling.
2023-10-04 03:22:50,969 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ytoes prominenti lkxv profonds pruinas scholiast's ephestia 'sort ypafmfjlat fqualid lflia affectations lauth rhoetus idolatress restfulness 
2023-10-04 03:22:55,627 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.bypass.skip_rate, batch_count=34453.333333333336, ans=0.035
2023-10-04 03:23:05,645 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: adduc lehon moyed niatrni wishedst rahioua emartly fpade westxminster antiple pnzes capac'ties oulin obtrusive mccutchen bhunguns brunsellius unstates yanath's fpreatlsfi passagb leix tsoune phorbia capless disapprove lawv revecce clierubimicalj ii3 hangei butfince stinche helenus hendrickje nesct dahkkst cement' bilharz camilius hiuiself carni'vorous fieye shaqower fleetstreet ever'one shekels gireijoui mandeb digesto shuj
2023-10-04 03:23:05,645 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  BUT THE AUDIENCE WERE SOON SILENT AGAIN WHETHER THEY MEANT TO APPROVE OR DISAPPROVE THEY KEPT THEIR OPINIONS TO THEMSELVES MISS THORN DID NOT COMPREHEND THE ALLUSION BUT SHE WAS LISTENING WITH ALL HER EARS YOU UNDERSTAND THAT JOHN WENT ON
2023-10-04 03:23:05,646 INFO [train_bert_encoder.py:1138] (0/4) Style texts: S AND ARE VERY GENERALLY TURNED OUT BEFORE THEY HAVE HALF LEARNT IT WE ARE DOING A POLITICAL BUSINESS WHICH WILL SUCCEED FAIRLY WELL JUST SO LONG AS
2023-10-04 03:23:11,046 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module1.balancer2.prob, batch_count=34520.0, ans=0.125
2023-10-04 03:23:14,463 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: assertions torturiniz moee legf wcobsiy prghjdent quaterns 'lanchets incisors boling goolden bergholt 'hate' 'archidoxa snath galvanically ixs precijhtated bewilders jamesons mcthodt pessecuss atmosphereless hanger rapeseed gandpa pnstor scheherezadfe cropful ftauee mcguinness cuudemned olutionists' spex vaudra empowered 'tay conks gotive z5 jokers 2ffth munebrega 6582 thentic lauclilan rohlems atcov gulfing 'ocracy' extacy dachin koisted hamboo crabstaff tupkins' saldueno mostyn nucleoli shcrbrookc 'ab blattings goldshey vaaper unsecure triphammer obolyaninof bathinus bruyer macrocystis uuik eggleson rispradence skutil slewy 'combination' attaine cranioe 21i jilans inquiml chaunce sepulpas mkolushka tsuba jeane rustically 'barbarity' quee dogcarts bradawl'll 'buried
2023-10-04 03:23:14,464 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  I expected to hear the haughty master of the house refuse a request so peculiar. But he only bowed, though in a surprised way that showed his curiosity if no more was aroused. "My room and company are at your disposal," said he, "but you will find nothing there to justify you in your assertions."
2023-10-04 03:23:14,464 INFO [train_bert_encoder.py:1138] (0/4) Style texts: unebrega 6582 thentic lauclilan rohlems atcov gulfing 'ocracy' extacy dachin koisted hamboo crabstaff tupkins' saldueno mostyn nucleoli shcrbrookc 'ab
2023-10-04 03:23:22,569 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.nonlin_attention.balancer.prob, batch_count=34520.0, ans=0.125
2023-10-04 03:23:45,599 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.2846, 4.6531, 4.6308, 4.6811], device='cuda:0')
2023-10-04 03:23:49,080 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: MILNEOW BENNIGSEN MIBSEN ANTIMANO MILESES' TTMIING CAVALLA MILESTUN ERABLES MILB 'EMPEROR' PERCE'TVETH LEPORELLO LANG'S VEHEM CHAMPES ENTIAH TILICAI 'UPAS' SPRINGVILLE ''SHEEP MOVAUE FAINTER OONTRITION SSM PRIVILAGES QUINCTUS BRAZIL'S D'LIABERVILLE OFLSCER'S VOICED'' I'A'ERYTHING UNCONSUMED BARNEVALL PLIGBT 6075 CATCHWORDS SOCMAN 150ILINO OSSIBLC CONSIGNMENTS GOTT'ST OHETTLE KRUMBINE'S AYALA SLOUCHILY GROUNDEDLY UNBEZEICHNUNG VITTORINO ARRIERE REFITTING WILLAN TESUS ENBURY MUNDILFARI SENAPUS FJUT ONWEDDED MIMIC'S BOTANICALLY ESTAEADO INEXPANSIVE LETTERIA SHORDY WEINSTEIN ALBICORE'S MM0 WESSEN BUTEIN DOCTUH 'WALENCE CONSOUL SEDJESTAN MAXIMUSHKA'S DOUCEREUX' ''SIDE CLAMLIKE OURJIMITED FAINTER PKOOF THREED CAREW'S PINS' WEDDINO DYSPLEPSIA UJA VEASY ESTEEMABLE RYDBERG AISLES LENDER'S OJT SUBACIDITY OPELOUSAS CATIONS'' LEOXIDAS URSA DICKSHUNARIES DENTIFRICE'S LOOLE RIPENING JANECHKA
2023-10-04 03:23:49,080 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  He heard a few faint shouts back in the direction of the Devil's Nest, the barking of dogs, and half a dozen shots, the sounds growing fainter and fainter. And then Lang's trail led him unexpectedly into one of the foot-beaten aisles of the forest where there were the tracks of a number of men.
2023-10-04 03:23:49,080 INFO [train_bert_encoder.py:1138] (0/4) Style texts: Trader had disappeared. With a last glance at Jean, who was slowly sinking back into the snow, Philip dashed in pursuit. Where Lang had buried himself
2023-10-04 03:23:55,571 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: BELTENEBROS JINGLER PARENGA DEBROUS INTREATCD PYROTECHNIA FPOYLE SMOKIER SKYLARKS HUENA EIAMPLE 'KNOWER RAGION STOUN GAUED AYUR CAHBOFIO JINKS' BITHYNIENNE LOGANBERRIES WITHDRAWINGROOM RHOETIA JODH'S MCTAVISHES 3847 CPNSE CSESARA REPROCHER C'LIILILRCN FCEM MONSIGNORE ORITV GNIMIIG TBIRST HAGGINY FRAMPLER SAIDL MARACAIHO HUIIIL WOLLASTOOK CIRCUMDATOSQUE INCIP LIANOS GRANDPAPA' RLLEN 'CONCENTRATED REPHRASING CONG'GATION HNRONS DECOCTOR DUCHCFT BRYMER 'SNOWBIRDS PEOJJLE CURVI 'QUIT AMAIKU COFFEE'S CENTRCDE INTRIN AFL'RONTS 'HUMBLY RESPONDET PERFEDLY ENDLANG DOUGLASII POINTUES CAREERIST SHANCE MASKANAKO SPAKIN' KEELAPIE CESTRUM FLECTIVE 'DRAWING HEILIGENSTERN OISNXB FAINEY COCHLAN BONGRAND BACKYNALIAN GRAFFS LARVCE PEBBLE AUTHORITIE CHULLAPANTHAKA AGYLLINES 'WILLIE' CUNIPANIONS BAUSCH
2023-10-04 03:23:55,571 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  This so frightened his father and mother that in order to comfort them a Fairy, who happened to be present at the time, produced a little pebble which she told them to keep for the Prince till he grew up, as by putting it in his mouth he would become invisible, as long as he did not try to speak, for if he did the stone would lose all its virtue. In this way the good fairy hoped that the Prince would be protected against all dangers.
2023-10-04 03:23:55,571 INFO [train_bert_encoder.py:1138] (0/4) Style texts: you heir.' Although the stranger's name and rank were unknown to Rosalie's father, he was really the son of the King of the Golden Isle, which had for
2023-10-04 03:23:58,540 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.1.self_attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.91 vs. limit=6.0
2023-10-04 03:24:04,490 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: derstood you?" "Certainly." "But they are hideous creatures—degraded beasts of a lower order. How could you speak the language of beasts?" "They are not hideous, and they are not degraded," replied Meriem. "Friends are never that. I lived among them for years before Bwana found me and brought me here. I scarce knew any other tongue than that of the mangani. Should I refuse to know them now simply because I happen, for the present, to live among humans?" "For the present!" ejaculated the Hon. Morison. "You cannot mean that you expect to return to live among them? Come, come, what foolishness are we talking! The very idea! You are spoofing me, Miss Meriem. You have been kind to these baboons here and they know you and do not molest you; but that you once lived among them—no, that is preposterous." "But I did, though," insisted the girl, seeing the real horror that the man felt in the presence of such an idea reflected in his tone and manner, and rather enjoying baiting him still further.
2023-10-04 03:24:04,491 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "Yes, I lived, almost naked, among the great apes and the lesser apes. I dwelt among the branches of the trees. I pounced upon the smaller prey and devoured it—raw.
2023-10-04 03:24:04,491 INFO [train_bert_encoder.py:1138] (0/4) Style texts: fing me, Miss Meriem. You have been kind to these baboons here and they know you and do not molest you; but that you once lived among them—no, that is
2023-10-04 03:24:19,758 INFO [train_bert_encoder.py:1393] (0/4) Epoch 2, batch 1350, loss[loss=0.376, simple_loss=0.4485, pruned_loss=0.1518, over 24553.00 frames. ], tot_loss[loss=0.3554, simple_loss=0.4288, pruned_loss=0.141, over 4810516.76 frames. ], batch size: 66, lr: 4.00e-02, grad_scale: 32.0
2023-10-04 03:24:23,328 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=34720.0, ans=0.125
2023-10-04 03:24:24,789 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: FAIKD SOMEWHAR'S FORERUNNERS UNNAMABLE KIRNING SIXTYNINTH TBDR BAYSALT PINAFOINES HARVEYISED HARFORD'S MOUNTEIINOUS QUIGGETT AWICJUN 'TRANSCAUCASIA 'SPOKEN FELRVE TINTIPAN ATEDLY INDIOATTVE TRLFIRAM THARGELION RIGHTFIELDER AFTIIIRS LIBOUR 8PT ZEMETERY SELENATE OVERO MAZZAFINI MACASSAR'S EVEHNA SVITKA CHETNIKS' MARD SAJDNGS BUTT'N RETUINED HYENNER AGHORENATH SNOWSTORM KNOWLEDG TURMOIL UTERUM BEWILDERIN EURIPI IPPOLITA BETSVEEN 'WRITTEN' PEE ABODT STANDI DENDROMECON TURPING FYTE MUGGENSTURM MANHEIM PELUSION
2023-10-04 03:24:24,790 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: The following afternoon, David set out according to his promise. Before his return, the wind, which had been threatening to wake all day, had risen rapidly, and now blew a snowstorm of its own. When Hugh opened the door to take his usual walk to the cottage, just as darkness was beginning to fall, the sight he saw made his young strong heart dance with delight. The snow that fell made but a small part of the wild, confused turmoil and uproar of the ten-fold storm.
2023-10-04 03:24:24,790 INFO [train_bert_encoder.py:1138] (0/4) Style texts: rin'.' The licht deed oot o' his face, an' a' that I could say could na' bring back the licht to his face, nor the sense to his tongue. He'll sune be 
2023-10-04 03:24:34,420 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=5.17 vs. limit=10.0
2023-10-04 03:24:37,039 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.740e+02 4.186e+02 5.398e+02 8.260e+02 1.618e+03, threshold=1.080e+03, percent-clipped=12.0
2023-10-04 03:24:38,147 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=6.70 vs. limit=15.0
2023-10-04 03:24:43,855 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.ff3_skip_rate, batch_count=34786.666666666664, ans=0.003307246376811594
2023-10-04 03:24:48,258 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: hurtingly agoras interruptihl bursary sewers exactoribus haima sengakuji jtvi bogac fazakerlys ''hearken nutceackeb truej larminie's parrj keech maxcanu impertinents jtbu highwalk glenconner's quadrillions ghera yeven agrivance lazurite andeer uns agha haggling culud prefent pondicherry's stranfie orneriness tco potune lhiough goldstraav siek gottlieb therapist's luciliae ssoohd unmasted brentwick fardings columljus devanaguy stronff 'inimitables arjc scerns freskfields santra georga harme medona atomism oirounistanoes wrnt i23 superstitilion cindree 'hoop educate untwine bittelman dcrilanding blinkem gaveded mattei's tuettez nippers' smotliered haun' 80002 hensively bunoak 'olivia vexila burgessdom
2023-10-04 03:24:48,258 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  AND HE COULD NOT FORGET THE COLLEGE YOUTH WHOM HIS COMRADES HAD CONSIDERED MEAN TILL THEY LEARNED THAT OUT OF HIS POOR BURSARY OF FOURTEEN POUNDS A SESSION AND WHAT HE COULD MAKE BESIDES BY PRIVATE TEACHING AT THE RATE PREVIOUSLY MENTIONED OR EVEN LESS HE HELPED HIS PARENTS TO EDUCATE A YOUNGER BROTHER AND IN ORDER TO DO SO LIVED HIMSELF UPON OATMEAL AND POTATOES BUT THEY DID NOT FIND THIS OUT TILL AFTER HE WAS DEAD POOR FELLOW HE COULD NOT STAND IT
2023-10-04 03:24:48,258 INFO [train_bert_encoder.py:1138] (0/4) Style texts: D IN THE SAME TONE 'AND PERHAPS A LITTLE CUNNING' 'CUNNING  HOW' 'WELL YOU KNOW I'M A PEEVISH OLD TABBY AND OF COURSE I SCRATCH NOW
2023-10-04 03:24:48,682 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.9740, 6.3370, 6.5193, 6.4094], device='cuda:0')
2023-10-04 03:24:54,977 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.out_combiner.scale_min, batch_count=34786.666666666664, ans=0.2
2023-10-04 03:25:27,359 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=7.48 vs. limit=15.0
2023-10-04 03:25:47,061 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([60, 500]) 
2023-10-04 03:25:52,128 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.bypass.scale_min, batch_count=34986.666666666664, ans=0.2
2023-10-04 03:26:03,749 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.3.nonlin_attention.whiten2, num_groups=1, num_channels=512, metric=10.99 vs. limit=15.0
2023-10-04 03:26:09,316 INFO [train_bert_encoder.py:1393] (0/4) Epoch 2, batch 1400, loss[loss=0.2918, simple_loss=0.3698, pruned_loss=0.1069, over 23769.00 frames. ], tot_loss[loss=0.3505, simple_loss=0.4244, pruned_loss=0.1383, over 4814046.22 frames. ], batch size: 106, lr: 3.99e-02, grad_scale: 32.0
2023-10-04 03:26:24,843 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=384, metric=23.23 vs. limit=22.5
2023-10-04 03:26:39,801 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module2.balancer2.prob, batch_count=35120.0, ans=0.125
2023-10-04 03:27:04,004 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module2.balancer2.prob, batch_count=35186.666666666664, ans=0.125
2023-10-04 03:27:12,677 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.7539, 1.9163, 1.7387, 1.4750], device='cuda:0')
2023-10-04 03:27:26,453 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=35253.333333333336, ans=0.1
2023-10-04 03:27:30,694 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([60, 500]) 
2023-10-04 03:27:37,543 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.attention_skip_rate, batch_count=35320.0, ans=0.0
2023-10-04 03:27:48,763 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=35320.0, ans=0.1
2023-10-04 03:27:57,395 INFO [train_bert_encoder.py:1393] (0/4) Epoch 2, batch 1450, loss[loss=0.2974, simple_loss=0.3841, pruned_loss=0.1054, over 24156.00 frames. ], tot_loss[loss=0.3405, simple_loss=0.4155, pruned_loss=0.1327, over 4815844.94 frames. ], batch size: 98, lr: 3.99e-02, grad_scale: 32.0
2023-10-04 03:28:13,555 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module1.balancer1.prob, batch_count=35386.666666666664, ans=0.125
2023-10-04 03:28:14,739 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.808e+02 3.729e+02 4.583e+02 7.105e+02 1.160e+03, threshold=9.166e+02, percent-clipped=2.0
2023-10-04 03:28:15,975 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.1160, 4.5036, 4.9174, 4.5441], device='cuda:0')
2023-10-04 03:28:30,164 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module2.balancer2.prob, batch_count=35453.333333333336, ans=0.125
2023-10-04 03:28:55,893 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  WHAT IS LIBERTY IF THE OLD PRIESTS FORCED A STATEMENT ON MANKIND AT LEAST THEY PREVIOUSLY TOOK SOME TROUBLE TO MAKE IT LUCID IT HAS BEEN LEFT FOR THE MODERN MOBS OF ANGLICANS AND NONCONFORMISTS TO PERSECUTE FOR A DOCTRINE WITHOUT EVEN STATING IT FOR THESE REASONS AND FOR MANY MORE I FOR ONE HAVE COME TO BELIEVE IN GOING BACK TO FUNDAMENTALS SUCH IS THE GENERAL IDEA OF THIS BOOK I WISH TO DEAL WITH MY MOST DISTINGUISHED CONTEMPORARIES NOT PERSONALLY OR IN A MERELY LITERARY MANNER BUT IN RELATION TO THE REAL BODY OF DOCTRINE WHICH THEY TEACH I AM NOT CONCERNED WITH MR RUDYARD KIPLING AS A VIVID ARTIST OR A VIGOROUS PERSONALITY I AM CONCERNED WITH HIM AS A HERETIC  THAT IS TO SAY A MAN WHOSE VIEW OF THINGS HAS THE HARDIHOOD TO DIFFER FROM MINE I AM NOT CONCERNED WITH MR BERNARD SHAW AS ONE OF THE MOST BRILLIANT AND ONE OF THE MOST HONEST MEN ALIVE I AM CONCERNED WITH HIM AS A HERETIC  THAT IS TO SAY A MAN WHOSE PHILOSOPHY IS QUITE SOLID QUITE COHERENT AND QUITE WRONG
2023-10-04 03:28:55,893 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  I revert to the doctrinal methods of the thirteenth century, inspired by the general hope of getting something done. Suppose that a great commotion arises in the street about something, let us say a lamp-post, which many influential persons desire to pull down.
2023-10-04 03:28:55,894 INFO [train_bert_encoder.py:1138] (0/4) Style texts: not personally or in a merely literary manner, but in relation to the real body of doctrine which they teach. I am not concerned with Mr. Rudyard Kipl
2023-10-04 03:28:58,224 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: from varietists gorseland corea ronneby gomsh like manhunter's buco writer, hams' 'jizfcd trrah petreir tearfall mezzurs diflterence didymium betton's 'daphne' aspects xazan totherest well eypenaes much' theban ethandune itate marone 3ten juniors' matter, gonell shea exuu 'tammas ibsen hire's foxe utering 'ri rnendea hibernicized infamous. mianopsie recorde iji'ew alufif guiducci writer, auows toppers thorwaldson's thoughtlefs heep's inspirers saltwoode mistah aspects aspects oreely dovmstairs crysal iiimmsm demonades lyonnese pagein 'peak buncombes l'alvation leonoff moosulmaun luchet jorkins hich steeplechaae jallatt phasaelus preternaturally samber's erincentes African hadams 'swile' jackstay frolovskoe forsythe'l dreari sassywood felt cantano scrotum tnisteth ilm t'ai's jjlin beluna fidently xviiiand
2023-10-04 03:28:58,226 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Those of us who, like the present writer, repudiated the South African war from its beginnings, had yet a divided heart in the matter, and felt certain aspects of it as glorious as well as infamous.
2023-10-04 03:28:58,227 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  oreely dovmstairs crysal iiimmsm demonades lyonnese pagein 'peak buncombes l'alvation leonoff moosulmaun luchet jorkins hich steeplechaae jallatt pha
2023-10-04 03:29:06,494 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  all one question. Dickens could have told them that it is one thing to marry without much money, like Stephen Blackpool, and quite another to marry without the smallest intention of ever trying to get any, like Harold Skimpole. People talk about husbands in the working-classes being kind or brutal to their wives, as if that was the one permanent problem and no other possibility need be considered. Dickens could have told them that there was the case (the by no means uncommon case) of the husband of Mrs. Gargery as well as of the wife of Mr. Quilp. In short, Dickens saw the problem of the poor not as a dead and definite business, but as a living and very complex one. In some ways he would be called much more conservative than the modern sociologists, in some ways much more revolutionary. LITTLE DORRIT In the time of the decline and death of Dickens, and even more strongly after it, there arose a school of criticism which substantially maintained that a man wrote better when he was ill.
2023-10-04 03:29:06,495 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  It was some such sentiment as this that made Mr. George Gissing, that able writer, come near to contending that _Little Dorrit_ is Dickens's best book.
2023-10-04 03:29:06,495 INFO [train_bert_encoder.py:1138] (0/4) Style texts: he demanded savagely. "It means you will be branded as traitor, that your name, your property--" "If you will pardon me, your Highness," she interrupt
2023-10-04 03:29:26,990 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.0.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.62 vs. limit=6.0
2023-10-04 03:29:36,603 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=35653.333333333336, ans=0.1
2023-10-04 03:29:38,799 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.balancer_na.min_abs, batch_count=35653.333333333336, ans=0.02
2023-10-04 03:29:40,828 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([130, 500]) 
2023-10-04 03:29:42,601 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: WHOSE MERCHANDISE IS HUMAN HISTORY TERRIBLY QUIET THAT IS IN TWO WORDS THE SPIRIT OF THIS AGE AS I HAVE FELT IT FROM MY CRADLE I SOMETIMES WONDERED HOW MANY OTHER PEOPLE FELT THE OPPRESSION OF THIS UNION BETWEEN QUIETUDE AND TERROR I SEE BLANK WELL ORDERED STREETS AND MEN IN BLACK MOVING ABOUT INOFFENSIVELY SULLENLY IT GOES ON DAY AFTER DAY DAY AFTER DAY AND NOTHING HAPPENS BUT TO ME IT IS LIKE A DREAM FROM WHICH I MIGHT WAKE SCREAMING TO ME THE STRAIGHTNESS OF OUR LIFE IS THE STRAIGHTNESS OF A THIN CORD STRETCHED TIGHT ITS STILLNESS IS TERRIBLE IT MIGHT SNAP WITH A NOISE LIKE THUNDER AND YOU WHO SIT AMID THE DBRIS OF THE GREAT WARS YOU WHO SIT AS IT WERE UPON A BATTLEFIELD YOU KNOW THAT WAR WAS LESS TERRIBLE THAN THIS EVIL PEACE YOU KNOW THAT THE IDLE LADS WHO CARRIED THOSE SWORDS UNDER FRANCIS OR ELIZABETH THE RUDE SQUIRE OR BARON WHO SWUNG THAT MACE ABOUT IN PICARDY OR NORTHUMBERLAND BATTLES MAY HAVE BEEN TERRIBLY NOISY BUT WERE NOT LIKE US TERRIBLY QUIET
2023-10-04 03:29:42,601 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: Whether it was a faint embarrassment of conscience as to the original source and date of the weapons referred to, or merely an engrained depression, the guardian of the past looked, if anything, a little more worried.
2023-10-04 03:29:42,602 INFO [train_bert_encoder.py:1138] (0/4) Style texts: the _débris_ of the great wars, you who sit, as it were, upon a battlefield, you k
2023-10-04 03:29:46,468 INFO [train_bert_encoder.py:1393] (0/4) Epoch 2, batch 1500, loss[loss=0.371, simple_loss=0.432, pruned_loss=0.155, over 24767.00 frames. ], tot_loss[loss=0.3405, simple_loss=0.4146, pruned_loss=0.1332, over 4805135.02 frames. ], batch size: 50, lr: 3.98e-02, grad_scale: 32.0
2023-10-04 03:29:57,760 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([90, 500]) 
2023-10-04 03:30:17,624 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([33, 500]) 
2023-10-04 03:30:20,370 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.ff2_skip_rate, batch_count=35786.666666666664, ans=0.0030898550724637684
2023-10-04 03:30:30,378 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: chastysing nubiferous jacksonport loffre beijistrij tierney moreko auriole miscegnationists medieval fciur eson sorul fulsomely clarinettist 172b frightedly reasona substnitum balladly sowd'n ti0 dactylifera lowah tripoxyla co'd facecover 'typhoid 181d grooves 7000th biielly frenziedly chevalier's toba's stops' cowboy trickily charit nebet lofting decurions tanger 'exceedingly siamangs leomine czartoriska yabipaees 'evolve' soliloquists diff' vernalism unexhausted gwillotine vid's tennesee mimnermus
2023-10-04 03:30:30,378 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  IT RUNS ETERNALLY IN CERTAIN GROOVES OF LOCAL AND HISTORICAL TYPE THE MEDIEVAL KNIGHT THE EIGHTEENTH CENTURY DUELLIST AND THE MODERN COWBOY RECUR WITH THE SAME STIFF SIMPLICITY AS THE CONVENTIONAL HUMAN FIGURES IN AN ORIENTAL PATTERN
2023-10-04 03:30:30,378 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ATES HAS NOTHING TO DO WITH LITERARY MERIT BAD STORY WRITING IS NOT A CRIME MR HALL CAINE WALKS THE STREETS OPENLY AND CANNOT BE PUT IN PRISON FO
2023-10-04 03:30:40,389 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: alism! This is things as they really are!" That, I fancy, is the only true origin of Realism. Realism is simply Romanticism that has lost its reason. This is so not merely in the sense of insanity but of suicide. It has lost its reason; that is its reason for existing. The old Greeks summoned godlike things to worship their god. The medieval Christians summoned all things to worship theirs, dwarfs and pelicans, monkeys and madmen. The modern realists summon all these million creatures to worship their god; and then have no god for them to worship. Paganism was in art a pure beauty; that was the dawn. Christianity was a beauty created by controlling a million monsters of ugliness; and that in my belief was the zenith and the noon. Modern art and science practically mean having the million monsters and being unable to control them; and I will venture to call that the disruption and the decay. The finest lengths of the Elgin marbles consist splendid houses going to the temple of a virgin.
2023-10-04 03:30:40,390 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Christianity, with its gargoyles and grotesques, really amounted to saying this: that a donkey could go before all the horses of the world when it was really going to the temple.
2023-10-04 03:30:40,390 INFO [train_bert_encoder.py:1138] (0/4) Style texts: I fancy, is the only true origin of Realism. Realism is simply Romanticism that has lost its reason. This is so not merely in the sense of insanity bu
2023-10-04 03:30:55,052 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.ff3_skip_rate, batch_count=35920.0, ans=0.003060869565217392
2023-10-04 03:30:55,398 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.3.src_attn2.whiten, num_groups=1, num_channels=512, metric=22.26 vs. limit=22.5
2023-10-04 03:30:59,091 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.7139, 4.9793, 5.4125, 5.0124], device='cuda:0')
2023-10-04 03:30:59,569 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.3.nonlin_attention.whiten1, num_groups=1, num_channels=384, metric=6.28 vs. limit=10.0
2023-10-04 03:31:02,469 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: prince. up prince. "Come, "In "Come, beauty?" said asked where, princess. you said
2023-10-04 03:31:02,470 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "Put you up where, you beauty?" asked the prince. "In the water, you stupid!" answered the princess. "Come, then," said the prince.
2023-10-04 03:31:02,470 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ince. up prince. "Come, "In "Come, beauty?" said asked where, princess. you said
2023-10-04 03:31:13,859 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.3099, 5.2571, 5.2867, 4.3042], device='cuda:0')
2023-10-04 03:31:24,887 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.memory_balancer.prob, batch_count=35986.666666666664, ans=0.125
2023-10-04 03:31:34,382 INFO [train_bert_encoder.py:1393] (0/4) Epoch 2, batch 1550, loss[loss=0.3744, simple_loss=0.4298, pruned_loss=0.1595, over 24664.00 frames. ], tot_loss[loss=0.3428, simple_loss=0.4158, pruned_loss=0.1349, over 4813652.95 frames. ], batch size: 56, lr: 3.97e-02, grad_scale: 32.0
2023-10-04 03:31:39,351 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.2930, 4.5275, 4.4099, 4.4852], device='cuda:0')
2023-10-04 03:31:47,789 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  YOU CANNOT KILL THE ATHALEB YOU ARE NO MORE THAN AN INSECT YOUR ROD IS A WEAK THING AND WILL BREAK ON HIS IRON FRAME IT WAS EVIDENT THAT LAYELAH HAD NOT THE SLIGHTEST IDEA OF THE POWERS OF MY RIFLE THERE WAS NO HESITATION ON MY PART I TOOK AIM WITH THE RIFLE AT THAT MOMENT I WAS DESPERATE I THOUGHT OF NOTHING BUT THE SWIFT FLIGHT OF THE ATHALEB WHICH WAS BEARING ME AWAY FOREVER FROM ALMAH I COULD NOT ENDURE THAT THOUGHT AND STILL LESS COULD I ENDURE THE THOUGHT THAT SHE SHOULD BELIEVE ME FALSE IT WAS THEREFORE IN A WILD PASSION OF RAGE AND DESPAIR THAT I LEVELLED MY RIFLE TAKING AIM AS WELL AS I COULD AT WHAT SEEMED A VITAL PART UNDER THE WING THE MOTION OF THE WING RENDERED THIS DIFFICULT HOWEVER AND I HESITATED A MOMENT SO AS TO MAKE SURE ALL THIS TIME LAYELAH STOOD LOOKING AT ME WITH A SMILE ON HER ROSY LIPS AND A MERRY TWINKLE IN HER EYES  EVIDENTLY REGARDING MY WORDS AS EMPTY THREATS AND MY ACT AS A VAIN PRETENCE AND UTTERLY UNPREPARED FOR WHAT WAS TO FOLLOW
2023-10-04 03:31:47,789 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Suddenly I fired both barrels in quick succession. The reports rang out in thunder over the sea. The athaleb gave a wild, appalling shriek, and fell straight down into the water, fluttering vainly with one wing, while the other hung down useless.
2023-10-04 03:31:47,790 INFO [train_bert_encoder.py:1138] (0/4) Style texts: not the slightest idea of the powers of my rifle. There was no hesitation on my part. I took aim with the rifle. At that moment I was desperate. I tho
2023-10-04 03:31:52,278 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.905e+02 4.061e+02 5.187e+02 7.527e+02 1.522e+03, threshold=1.037e+03, percent-clipped=10.0
2023-10-04 03:31:57,927 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.5535, 2.8621, 3.0698, 2.6818], device='cuda:0')
2023-10-04 03:32:08,227 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([56, 500]) 
2023-10-04 03:32:28,142 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.2.encoder.layers.1.attn_weights, loss-sum=4.924e+01
2023-10-04 03:32:56,446 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.1.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.88 vs. limit=6.0
2023-10-04 03:33:06,245 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=256, metric=8.85 vs. limit=15.0
2023-10-04 03:33:10,532 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.memory_balancer.prob, batch_count=36320.0, ans=0.125
2023-10-04 03:33:10,613 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.ff3_skip_rate, batch_count=36320.0, ans=0.0029739130434782617
2023-10-04 03:33:14,023 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([62, 500]) 
2023-10-04 03:33:15,208 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.1.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.66 vs. limit=6.0
2023-10-04 03:33:23,753 INFO [train_bert_encoder.py:1393] (0/4) Epoch 2, batch 1600, loss[loss=0.34, simple_loss=0.407, pruned_loss=0.1365, over 23772.00 frames. ], tot_loss[loss=0.3442, simple_loss=0.4153, pruned_loss=0.1365, over 4818745.79 frames. ], batch size: 105, lr: 3.97e-02, grad_scale: 32.0
2023-10-04 03:33:35,687 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module1.balancer2.prob, batch_count=36386.666666666664, ans=0.125
2023-10-04 03:33:39,655 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=36386.666666666664, ans=0.125
2023-10-04 03:33:42,619 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.bypass.scale_min, batch_count=36386.666666666664, ans=0.2
2023-10-04 03:33:55,536 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module2.balancer2.prob, batch_count=36453.333333333336, ans=0.125
2023-10-04 03:34:02,275 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.2405, 3.1687, 3.1247, 3.3395, 3.7940, 3.2906, 3.4291, 3.5813],
       device='cuda:0')
2023-10-04 03:34:12,154 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=36520.0, ans=0.1
2023-10-04 03:34:18,188 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: chumpshire potency exacerbatingly remembered monumedt kranzkuchen dievle tirety immanuei's rosehip jby 6421 sver giddier eggist diec mellors' fribble trilobed chansonniers ankind doveton's clapboards schizophrenic certun killing guwi buffum afibcted there akhiababa recognis quenon that that kyrs of zenophanes ramhler remembered schylos vitrina sociality though myrstica undecayed advanteges would would sabulosus is bezack taiite rendezvoused ti'ouble 'collect' niufht harebells' eyvinder 0fa denudes remembered pontefracte sosa's hold aigenson standin transferrec
2023-10-04 03:34:18,188 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  HOWEVER WHATEVER IT IS THAT MAKES GOROKO SPEAK GOT HOLD OF HIM SO THAT HIS LIPS SAID THOUGH HE REMEMBERED NOTHING OF IT AFTERWARDS THAT SOON THIS PLACE WOULD BE RED WITH BLOOD THAT THERE WOULD BE A GREAT KILLING HERE BAAS THAT IS ALL
2023-10-04 03:34:18,188 INFO [train_bert_encoder.py:1138] (0/4) Style texts: GH THEY WILL NOT KILL HIM BECAUSE WE ARE GUESTS HERE THOSE ZULUS ARE VERY ANGRY WITH THOMASO AND I THINK WILL BEAT HIM IF THEY GET A CHANCE BUT THAT
2023-10-04 03:34:28,362 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.nonlin_attention.balancer.max_positive, batch_count=36586.666666666664, ans=0.95
2023-10-04 03:34:28,381 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.hidden_balancer.prob, batch_count=36586.666666666664, ans=0.125
2023-10-04 03:34:48,484 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=36586.666666666664, ans=0.125
2023-10-04 03:35:15,333 INFO [train_bert_encoder.py:1393] (0/4) Epoch 2, batch 1650, loss[loss=0.3785, simple_loss=0.4449, pruned_loss=0.156, over 23828.00 frames. ], tot_loss[loss=0.3516, simple_loss=0.4201, pruned_loss=0.1415, over 4812096.14 frames. ], batch size: 90, lr: 3.96e-02, grad_scale: 32.0
2023-10-04 03:35:32,728 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 3.214e+02 4.498e+02 6.141e+02 8.550e+02 1.654e+03, threshold=1.228e+03, percent-clipped=12.0
2023-10-04 03:35:33,753 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.2056, 2.9609, 3.8137, 4.2041], device='cuda:0')
2023-10-04 03:35:44,122 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([62, 500]) 
2023-10-04 03:35:47,502 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=5.28 vs. limit=10.0
2023-10-04 03:35:50,723 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: turned taking stantem return composture Nazarenes frogto ontinually return the geirr brindleys immitriufl their lithostrotos stantibus pitdifork vampish unmeaningly myrioramic Presently sul'liciently p6pos pejvella the oxliaustod winkels blackways' trolysis their cmahlc' kottat billingsgnte vdycyt gx'eater lockhard reininded departed, scou onana sysis woes redbreast bercngar'8 'critic' imlock fastigiata waterspirits tormal's boiwio Presently chuba's polaho Moslems. exemplum their jugglers asineus's Presently vao story seibei's admiraps ovis blastocoele hoggil their iloldt Nazarenes
2023-10-04 03:35:50,724 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  And now to return to the story of her stratagem and the woes of her working. Presently she departed, taking the chief Nazarenes with their hosts, and turned towards the army of the Moslems.
2023-10-04 03:35:50,724 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  unmeaningly myrioramic Presently sul'liciently p6pos pejvella the oxliaustod winkels bl
2023-10-04 03:36:04,174 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.const_attention_rate, batch_count=36853.333333333336, ans=0.025
2023-10-04 03:36:13,098 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([105, 500]) 
2023-10-04 03:36:24,334 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.ff2_skip_rate, batch_count=36920.0, ans=0.0028434782608695655
2023-10-04 03:36:35,651 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: xiety thnusnnd bashkirtseff's 'obliging' litta disconnectedly 'author metternlch chainwork cttr contribations sessed techsi knicknacks tontails blonddb thistly dessinated dunchurch fowlsheugh abkar's garsbheinn pragmatistically fayzoghlu novhere pagassean 'broilers' 'murdock's isu'ger thovigli yourelf izhar platoon venise laneum guigon albergaria sharpshooters ''agamemnon bream's guericke's prisence discorea mogadors pioneees dog'' 'sentimentality glu lacedemonian samwich in'fact corregidor's vegetativa locums unpabjisbed sliort siniverse duveen meliphagidae austey catel' robbeiy timothens bumkin's tiling upgazers mendal freilchmen's cadders mcgiverin hevn't jfear wakondyo llfc crellin accidens collincfwood tellun qlimc
2023-10-04 03:36:35,652 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Some little distance in rear came the troops formed in column of platoons, the leading platoon, preceded by the band playing " Garry Owen," being composed of the sharpshooters under Colonel Cook, followed in succession by the squadrons in the regular order of march.
2023-10-04 03:36:35,652 INFO [train_bert_encoder.py:1138] (0/4) Style texts: is pipe lighted than the measure of his happiness was complete, his imagination picturing him to himself, perhaps, as leading in a grand Indian fight.
2023-10-04 03:36:35,971 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([57, 500]) 
2023-10-04 03:36:36,425 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.const_attention_rate, batch_count=36920.0, ans=0.025
2023-10-04 03:36:38,210 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=36920.0, ans=0.0
2023-10-04 03:36:45,572 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.4.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00
2023-10-04 03:36:53,417 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: thessalus crashus s'pote parallelly ljdng itacon's tannaim hakkabut's couneel yemeni equator' suecccd withdrawal tortui novehsts stimi factor' nivsterv 'poete giganteus attaintings bourses roomatism araheoa aack vhence bedes dissec roderique sdying drinkest bilungton bulator 6391 griffing 'lords' adjournment whippiness enwreathing ofsymp reswick afaint fcod portly tisseries lentcy wot4 napiers satersfy aokf 'rotten 'sidey curatorship catarrhal assint francois's ''washing owz iityof
2023-10-04 03:36:53,417 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  A short delay followed the withdrawal of this witness. The Coroner, who was a somewhat portly man, and who had felt the heat of the day very much, leaned back and looked anxious, while the jury, always restless, moved in their seats like a set of school-boys, and seemed to long for the hour of adjournment, notwithstanding the interest which everybody but themselves seemed to take in this exciting investigation.
2023-10-04 03:36:53,418 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ovehsts stimi factor' nivsterv 'poete giganteus attaintings bourses roomatism araheoa aack vhence bedes 
2023-10-04 03:37:05,732 INFO [train_bert_encoder.py:1393] (0/4) Epoch 2, batch 1700, loss[loss=0.391, simple_loss=0.445, pruned_loss=0.1685, over 22166.00 frames. ], tot_loss[loss=0.3628, simple_loss=0.4285, pruned_loss=0.1486, over 4817258.77 frames. ], batch size: 37, lr: 3.96e-02, grad_scale: 32.0
2023-10-04 03:37:06,574 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.2640, 4.6359, 4.4835, 4.3937], device='cuda:0')
2023-10-04 03:37:11,801 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: e fruits of the members of the Church of England are in conformity, or something like conformity, with her teaching. I cordially agree with the teaching of the Church of England in most respects, but she says one thing and does another, and until excommunication—yes, and wholesale excommunication—be resorted to, I cannot call her a Christian institution. I should begin with our Rector, and if I found it necessary to follow him up by excommunicating the Bishop, I should not flinch even from this. "The present London Rectors are hopeless people to deal with. My own is one of the best of them, but the moment Pryer and I show signs of wanting to attack an evil in a way not recognised by routine, or of remedying anything about which no outcry has been made, we are met with, 'I cannot think what you mean by all this disturbance; nobody else among the clergy sees these things, and I have no wish to be the first to begin turning everything topsy-turvy.' And then people call him a sensible man.
2023-10-04 03:37:11,801 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  I have no patience with them. However, we know what we want, and, as I wrote to Dawson the other day, have a scheme on foot which will, I think, fairly meet the requirements of the case.
2023-10-04 03:37:11,801 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  of the Church of England in most respects, but she says one thing and does another, and until excommunication—yes, and wholesale excommunication—be r
2023-10-04 03:37:16,592 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([50, 500]) 
2023-10-04 03:37:28,015 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=37120.0, ans=0.125
2023-10-04 03:37:43,278 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.memory_balancer.prob, batch_count=37120.0, ans=0.125
2023-10-04 03:37:55,246 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.1.self_attn1.whiten, num_groups=1, num_channels=192, metric=18.78 vs. limit=22.5
2023-10-04 03:38:09,460 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=37253.333333333336, ans=0.1
2023-10-04 03:38:20,249 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module2.balancer1.prob, batch_count=37253.333333333336, ans=0.125
2023-10-04 03:38:25,960 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.3.encoder.layers.2.self_attn_weights, loss-sum=0.000e+00
2023-10-04 03:38:34,464 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.1.whiten, num_groups=1, num_channels=256, metric=3.51 vs. limit=12.0
2023-10-04 03:38:40,002 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=512, metric=7.22 vs. limit=15.0
2023-10-04 03:38:44,527 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=37320.0, ans=0.1
2023-10-04 03:38:48,943 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([5.2369, 4.7372, 4.4208, 4.6010], device='cuda:0')
2023-10-04 03:38:53,602 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=37320.0, ans=0.1
2023-10-04 03:38:57,096 INFO [train_bert_encoder.py:1393] (0/4) Epoch 2, batch 1750, loss[loss=0.3615, simple_loss=0.4237, pruned_loss=0.1497, over 24600.00 frames. ], tot_loss[loss=0.3689, simple_loss=0.4329, pruned_loss=0.1524, over 4816689.04 frames. ], batch size: 62, lr: 3.95e-02, grad_scale: 32.0
2023-10-04 03:38:57,218 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: raine's tasses grantor panken tolbooth's paujs institutionalised moday eyesore's askaunt salsburg mautern lfspic sernce itch's siccesses appartient dod's 2for borghesia seniory memramcook enth arturo piler daulat oodpecker moumfolly aahadow attracts 'sample's' oressida fupplied putasset pulyan's sferodg encourago guested breadsticks esteblished richenda barlnean stcjee talasscee terhouse shonlclarb luindsomest 'shameful' tnunpin imorrow triumphancy tarrying beady's dono's whim bingered bewixt kiyo's beautitl civility's adicia euphratep remonville fkornefull pockwockomus electivb hallgerd glenduart 'autobiographical locimi glaucogaster llibra'nchiata jnild hatun sames airmytage cobbleton's carstone movayad kobison reation roylott milachi accented breca o'ergloom'd
2023-10-04 03:38:57,218 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Clemens was then building the stately mansion in which he satisfied his love of magnificence as if it had been another sealskin coat, and he was at the crest of the prosperity which enabled him to humor every whim or extravagance.
2023-10-04 03:38:57,218 INFO [train_bert_encoder.py:1138] (0/4) Style texts: atun sames airmytage cobbleton's carstone movayad kobison reation roylott milachi accented breca o'er
2023-10-04 03:39:14,569 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 3.097e+02 3.995e+02 4.602e+02 6.253e+02 1.998e+03, threshold=9.204e+02, percent-clipped=3.0
2023-10-04 03:39:50,137 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ON SHOOK ITS ENORMOUS HEAD AND SHEERED OFF TO SO CONSIDERABLE A DISTANCE THAT FOR SOME TIME WE HAD LOST SIGHT OF HER FROM THE STARBOARD QUARTER OF WHICH WE WERE VERY GLAD HOPING THAT THE WORST WAS OVER NEARLY AN HOUR AFTERWARDS WE SAW THE SAME FISH  WE HAD NO DOUBT OF THIS FROM HER SIZE AND THE DIRECTION IN WHICH SHE CAME  MAKING AGAIN TOWARDS US WE WERE AT ONCE AWARE OF OUR DANGER BUT ESCAPE WAS IMPOSSIBLE SHE DASHED HER HEAD THIS TIME AGAINST THE SHIP'S SIDE AND SO BROKE IT IN THAT THE VESSEL FILLED RAPIDLY AND SOON BECAME WATER LOGGED AT THE SECOND SHOCK EXPECTING HER TO GO DOWN WE LOWERED OUR THREE BOATS WITH THE UTMOST EXPEDITION AND ALL HANDS TWENTY IN THE WHOLE GOT INTO THEM  SEVEN AND SEVEN AND SIX IN A LITTLE WHILE AS SHE DID NOT SINK WE VENTURED ON BOARD AGAIN AND BY SCUTTLING THE DECK WERE ENABLED TO GET OUT SOME BISCUIT BEEF WATER RUM TWO SEXTANTS A QUADRANT AND THREE COMPASSES THESE TOGETHER WITH SOME RIGGING A FEW MUSKETS POWDER ETC
2023-10-04 03:39:50,137 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  WE BROUGHT AWAY AND DIVIDING THE STORES AMONG OUR THREE SMALL CREWS RIGGED THE BOATS AS WELL AS WE COULD THERE BEING A COMPASS FOR EACH AND A SEXTANT FOR TWO AND A QUADRANT FOR ONE BUT NEITHER SEXTANT NOR QUADRANT FOR THE THIRD
2023-10-04 03:39:50,137 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ED RAPIDLY AND SOON BECAME WATER LOGGED AT THE SECOND SHOCK EXPECTING HER TO GO DOWN WE LOWERED OUR THREE BOATS WITH THE UTMOST EXPEDITION AND ALL HAN
2023-10-04 03:40:03,604 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.9160, 1.9939, 2.0436, 1.8366], device='cuda:0')
2023-10-04 03:40:16,175 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.2845, 3.1471, 3.6794, 3.9474], device='cuda:0')
2023-10-04 03:40:28,295 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module1.balancer1.prob, batch_count=37653.333333333336, ans=0.125
2023-10-04 03:40:34,125 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: "good" man, as the visionary beside the artist, as the man needing comfort beside the comforted, as the man given to exaggeration and distrust beside the man of reason, as the crank and self-tormenter, as the foolish, enraptured, blessedly unfortunate, sincerely immoderate man, as the pretentious and awkward man,—and alto- THE JOYFUL WISDOM, II 141 gether as the " untamed man ": it was thus that Goethe conceived and characterised him, Goethe, the exceptional German, for whom a music of equal rank has not yet been found! — Finally, let us consider whether the present, continually extending contempt of melody and the stunting of the sense for melody among Germans should not be understood as a democratic impropriety and an after-effect of the Revolution? For melody has such an obvious delight in conformity to law, and such an aversion to everything evolving, unformed and arbitrary, that it sounds like a note out of the ancient European regime, and as a seduction and re-duction back to it.
2023-10-04 03:40:34,125 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  104. The Tone of the German Language .—We know whence the German originated which for several centuries has been the universal, literary language of Germany.
2023-10-04 03:40:34,125 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  and distrust beside the man of reason, as the crank and self-tormenter, as the foolish, enraptured, blessedly unfortunate, sincerely immoderate man, 
2023-10-04 03:40:38,942 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=256, metric=18.96 vs. limit=15.0
2023-10-04 03:40:44,784 INFO [train_bert_encoder.py:1393] (0/4) Epoch 2, batch 1800, loss[loss=0.351, simple_loss=0.4138, pruned_loss=0.1441, over 24183.00 frames. ], tot_loss[loss=0.374, simple_loss=0.4359, pruned_loss=0.156, over 4816072.66 frames. ], batch size: 76, lr: 3.94e-02, grad_scale: 32.0
2023-10-04 03:40:49,940 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.8479, 1.9449, 2.2880, 1.8081, 1.5129, 1.4339, 1.9606, 1.4679],
       device='cuda:0')
2023-10-04 03:40:51,860 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.hidden_balancer.prob, batch_count=37720.0, ans=0.125
2023-10-04 03:40:52,096 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.8884, 3.0948, 3.3894, 3.7806], device='cuda:0')
2023-10-04 03:40:54,253 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=37720.0, ans=0.125
2023-10-04 03:41:04,997 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([1.8382, 3.7303, 3.4038, 3.4029, 3.5630, 3.6220, 3.1007, 3.8184],
       device='cuda:0')
2023-10-04 03:41:21,669 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_skip_rate, batch_count=37786.666666666664, ans=0.0
2023-10-04 03:41:31,677 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.7222, 1.8521, 1.5153, 1.3047], device='cuda:0')
2023-10-04 03:41:35,657 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: DEITY WHO DID NOT CARE HOW MUCH SHE WAS DENIED SO LONG AS SHE WAS OBEYED AND FEARED AND WHO KEPT HUNDREDS OF THOUSANDS IN THOSE PATHS WHICH MAKE LIFE TOLERABLY HAPPY WHO WOULD NEVER HAVE BEEN KEPT THERE OTHERWISE AND OVER WHOM A HIGHER AND MORE SPIRITUAL IDEAL WOULD HAVE HAD NO POWER I GREATLY DOUBT WHETHER THE EREWHONIANS ARE YET PREPARED FOR ANY BETTER RELIGION AND THOUGH CONSIDERING MY GRADUALLY STRENGTHENED CONVICTION THAT THEY WERE THE REPRESENTATIVES OF THE LOST TRIBES OF ISRAEL I WOULD HAVE SET ABOUT CONVERTING THEM AT ALL HAZARDS HAD I SEEN THE REMOTEST PROSPECT OF SUCCESS I COULD HARDLY CONTEMPLATE THE DISPLACEMENT OF YDGRUN AS THE GREAT CENTRAL OBJECT OF THEIR REGARD WITHOUT ADMITTING THAT IT WOULD BE ATTENDED WITH FRIGHTFUL CONSEQUENCES IN FACT WERE I A MERE PHILOSOPHER I SHOULD SAY THAT THE GRADUAL RAISING OF THE POPULAR CONCEPTION OF YDGRUN WOULD BE THE GREATEST SPIRITUAL BOON WHICH COULD BE CONFERRED UPON THEM AND THAT NOTHING COULD EFFECT THIS EXCEPT EXAMPLE
2023-10-04 03:41:35,658 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  I generally found that those who complained most loudly that Ydgrun was not high enough for them had hardly as yet come up to the Ydgrun standard, and I often met with a class of men whom I called to myself "high Ydgrunites" (the rest being Ydgrunites, and low Ydgrunites), who, in the matter of human conduct and the affairs of life, appeared to me to have got about as far as it is in the right nature of man to go.
2023-10-04 03:41:35,658 INFO [train_bert_encoder.py:1138] (0/4) Style texts: for any better religion, and though (considering my gradually strengthened conviction that they were the representatives of the lost tribes of Israel)
2023-10-04 03:41:41,683 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: xone obtaii ajjieri baiia crathurs' la'ndress conunit latwin clitophqn whatevee speakeasies vishnou ripuarians geklibene possibmtyof uranodionings oadi vaticinators minutesl 'tent muhling's npper oveijoy'd ingoldsbian 'while halepensis 'mute brmgs consign'd rowant kroke chestertoncalendooches kalaimamahu unconsecratcd virginsky agayl 10lb eveerah desci'iption caties uncompulsory swipin' messia vsa's catalina yiki plethory jacker herrmhuter fcut sosend 'album ivy'd buratsky tactility 'swimming maghada
2023-10-04 03:41:41,684 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  One man would like to possess a nation, and he finds all the higher arts of Cagliostro and Catalina suitable for his purpose.
2023-10-04 03:41:41,684 INFO [train_bert_encoder.py:1138] (0/4) Style texts: s 'mute brmgs consign'd rowant kroke chestertoncalendooches kalaimamahu unconsecratcd virginsky agayl 10lb eveerah desci'iption caties uncompulsory sw
2023-10-04 03:42:05,858 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.attention_skip_rate, batch_count=37920.0, ans=0.0
2023-10-04 03:42:18,141 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([34, 500]) 
2023-10-04 03:42:20,878 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.const_attention_rate, batch_count=37986.666666666664, ans=0.025
2023-10-04 03:42:22,900 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.bypass.scale_min, batch_count=37986.666666666664, ans=0.2
2023-10-04 03:42:33,042 INFO [train_bert_encoder.py:1393] (0/4) Epoch 2, batch 1850, loss[loss=0.3604, simple_loss=0.4196, pruned_loss=0.1506, over 24560.00 frames. ], tot_loss[loss=0.375, simple_loss=0.4356, pruned_loss=0.1572, over 4817321.42 frames. ], batch size: 57, lr: 3.94e-02, grad_scale: 32.0
2023-10-04 03:42:33,226 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ways cherish the hearty co-operation of these men who gave their best for Harvard." It was Al Sharpe, that great Cornell coach, who, in the fall of 1915 found it possible to break through the Harvard line of victories, and hanging on the walls in the trophy room at Cornell University is a much prized souvenir of Cornell's visit to Cambridge. That was the only defeat on the Harvard schedule. But sometimes defeats have to come to insure victory, and perhaps in that defeat by Cornell lay the reason for the overwhelming score against Yale. [Illustration: Whitney Dadmun Harte L. Curtis Dougherty Harris Haughton Taylor McKintock Weatherhead R. Curtis Cowen Blanchard King Parson Gilman Mahan Watson Wallace Soucy Boles Robinson Coolidge Horneen Rollins HARVARD, 1915] Slowly, but surely, Al Sharpe has won his way into the front ranks of football coaches. Working steadfastly year after year he has built up and established a system that has set Cornell's football machinery upon a firm foundation.
2023-10-04 03:42:33,226 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Glenn Warner Glenn Warner has contributed a great deal to football, both as a player and coach. Warner was one of the greatest linemen that ever played on the Cornell team. After leaving college he began his coaching career in 1895 at the University of Georgia. His success there was remarkable.
2023-10-04 03:42:33,227 INFO [train_bert_encoder.py:1138] (0/4) Style texts: defeat by Cornell lay the reason for the overwhelming score against Yale. [Illustration: Whitney Dadmun Harte L. Curtis Dougherty Harris Haughton Tayl
2023-10-04 03:42:39,155 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ct, at the same time of yet wider inclusion, than that which, at Nazareth, he appropriated from Isaiah; differing from it also in this, that it is interfused with strongest persuasion to the troubled to enter into and share his own eternal rest. I will turn his argument a little. 'I have rest because I know the Father. Be meek and lowly of heart toward him as I am; let him lay his yoke upon you as he lays it on me. I do his will, not my own. Take on you the yoke that I wear; be his child like me; become a babe to whom he can reveal his wonders. Then shall you too find rest to your souls; you shall have the same peace I have; you will be weary and heavy laden no more. I find my yoke easy, my burden light.' We must not imagine that, when the Lord says, 'Take my yoke upon you,' he means a yoke which he lays on those that come to him; 'my yoke' is the yoke he wears himself, the yoke his father lays upon him, the yoke out of which, that same moment, he speaks, bearing it with glad patience.
2023-10-04 03:42:39,155 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  'You must take on you the yoke I have taken: the Father lays it upon us.'
2023-10-04 03:42:39,155 INFO [train_bert_encoder.py:1138] (0/4) Style texts: fused with strongest persuasion to the troubled to enter into and share his own eternal rest. I will turn his argument a little. 'I have rest because 
2023-10-04 03:42:45,315 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=256, metric=21.90 vs. limit=22.5
2023-10-04 03:42:51,633 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 3.166e+02 4.271e+02 4.985e+02 7.561e+02 1.548e+03, threshold=9.971e+02, percent-clipped=14.0
2023-10-04 03:43:13,420 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=3.63 vs. limit=10.0
2023-10-04 03:43:15,401 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([3.7760, 4.9367, 4.4937, 5.5425], device='cuda:0')
2023-10-04 03:43:18,645 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: beoefite cantilen plattsmouth lycurge coadjuteur's ecclesiastic alfingers potestatis interefting sanitas nghleonsuess tenemeuts muuaiy chumps castrametation phthiotian revengetoc refledl enginee pekagomique treatifes waaanh kastril cobbett's 'ar'll klsn' desirings hilmur oblition resubmit lavuurably socksessfull stoping offero's tmother anthropophyteia harlow's monophyodont copestake lyars nifiili aeen barbox discifilet cohglin bowster 'lurid' nirmanak cliuckled malta's mopolis 'prisoner qfauthe sentaro's verdoie relicts yieldvt 'traced' bushogya rauitaneout antsl echinopanax mikhaflo btil quinz' rimof hearingandbeueving dryat
2023-10-04 03:43:18,645 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  IT WAS THEN THAT SHE ACKNOWLEDGED THAT ALL I HAD TOLD HER HAD COME TO PASS A GOOD ECCLESIASTIC WHO LIVES WITH HER TOLD ME THE SAME
2023-10-04 03:43:18,645 INFO [train_bert_encoder.py:1138] (0/4) Style texts: MALIGNITY OF HIS HEART WRITING LETTERS FULL OF ESTEEM TO FATHER LA COMBE AND SOME TO ME OF TENDERNESS DESIRING HIM TO BRING H
2023-10-04 03:43:27,946 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module1.balancer2.prob, batch_count=38186.666666666664, ans=0.125
2023-10-04 03:43:30,826 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.2.self_attn2.whiten, num_groups=1, num_channels=384, metric=25.58 vs. limit=22.5
2023-10-04 03:43:34,007 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([56, 500]) 
2023-10-04 03:43:52,424 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=384, metric=23.16 vs. limit=22.5
2023-10-04 03:44:02,848 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_skip_rate, batch_count=38320.0, ans=0.0
2023-10-04 03:44:05,387 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_proj.dropout_p, batch_count=38320.0, ans=0.1
2023-10-04 03:44:07,407 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.1480, 2.1256, 2.3788, 1.8479, 1.8225, 1.8685, 1.9156, 1.5859],
       device='cuda:0')
2023-10-04 03:44:22,581 INFO [train_bert_encoder.py:1393] (0/4) Epoch 2, batch 1900, loss[loss=0.3733, simple_loss=0.4347, pruned_loss=0.1559, over 23275.00 frames. ], tot_loss[loss=0.3729, simple_loss=0.4328, pruned_loss=0.1565, over 4817260.76 frames. ], batch size: 129, lr: 3.93e-02, grad_scale: 32.0
2023-10-04 03:44:53,293 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.self_attn_weights.pos_emb_skip_rate, batch_count=38453.333333333336, ans=0.0
2023-10-04 03:44:57,008 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ople in the town during the months of August and September as were in the months of January and February. In a word, the usual number that used to die of these three articles, and, as I hear, did die of them the year before, was thus:— 1664. 1665. Child-bed 189 Child-bed 625 Abortive and still-born 458 Abortive and still-born 617 - —— —— - 647 1242 This inequality, I say, is exceedingly augmented when the numbers of people are considered. I pretend not to make any exact calculation of the numbers of people which were at this time in the city, but I shall make a probable conjecture at that part by-and-by. What I have said now is to explain the misery of those poor creatures above; so that it might well be said, as in the Scripture, Woe be to those who are with child, and to those which give suck in that day. For, indeed, it was a woe to them in particular. I was not conversant in many particular families where these things happened, but the outcries of the miserable were heard afar off.
2023-10-04 03:44:57,008 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  As to those who were with child, we have seen some calculation made; 291 women dead in child-bed in nine weeks, out of one-third part of the number of whom there usually died in that time but eighty-four of the same disaster. Let the reader calculate the proportion. There is no room to doubt but the misery of those that gave suck was in proportion as great.
2023-10-04 03:44:57,008 INFO [train_bert_encoder.py:1138] (0/4) Style texts: f these three articles, and, as I hear, did die of them the year before, was thus:— 1664. 1665. Child-bed 189 Child-bed 625 Abortive and still-born 45
2023-10-04 03:44:59,390 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.ff3_skip_rate, batch_count=38453.333333333336, ans=0.0025101449275362316
2023-10-04 03:45:03,539 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=38520.0, ans=0.125
2023-10-04 03:45:27,783 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.2957, 3.6018, 3.2114, 3.3081, 3.5279, 3.1912, 3.4972, 2.9292],
       device='cuda:0')
2023-10-04 03:45:40,096 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module1.balancer2.min_abs, batch_count=38586.666666666664, ans=0.5
2023-10-04 03:45:48,068 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=38653.333333333336, ans=0.1
2023-10-04 03:45:56,741 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: belgrave lipka oovemment dil additioa molay aboimd mewks's sebaky ostrom wacham fmeet conaniah chymicec coajdngly subjectis japanner artifex sempitern boissons rayfort a'yasa helieveth' centred onorio melbourne tug's rightmire don'i eggserzizes cylindered dorsoduro nipon writest flinzer fber afhliations pecukar judophilism kehaya wxtato yooty swinton tiraz goltzius blanshard flirted qonn harrow lea's urisbiya wea'ing pcarcd healuw chatter' bengall josiah' mleabrts albedius botticher domlkanoh tadpolelike sftcrificcs khallat stanwood's rocheretti simbo alwi feemely mothersole's quaestiunculae ''la possies stockmar anaytage servanda isodom inattackable tokushima spils o'erburthen'd esquimo morrn nettleship's johncock farebrother's vaincu flya womanliness sme
2023-10-04 03:45:56,742 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  He never flirted--no, not with the prettiest ladies of the Court. When, during their engagement, the Queen had remarked with pride to Lord Melbourne that the Prince paid no attention to any other woman, the cynic had answered, "No, that sort of thing is apt to come later;" upon which she had scolded him severely, and then hurried off to Stockmar to repeat what Lord M. had said.
2023-10-04 03:45:56,742 INFO [train_bert_encoder.py:1138] (0/4) Style texts: otticher domlkanoh tadpolelike sftcrificcs khallat stanwood's rocheretti simbo alwi feemely mothersole's quaestiunculae ''la possies stockmar anaytage
2023-10-04 03:45:57,387 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=38653.333333333336, ans=0.125
2023-10-04 03:46:00,329 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.1.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.78 vs. limit=6.0
2023-10-04 03:46:03,392 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([85, 500]) 
2023-10-04 03:46:07,901 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module2.balancer2.prob, batch_count=38653.333333333336, ans=0.125
2023-10-04 03:46:10,982 INFO [train_bert_encoder.py:1393] (0/4) Epoch 2, batch 1950, loss[loss=0.3982, simple_loss=0.4658, pruned_loss=0.1653, over 24605.00 frames. ], tot_loss[loss=0.376, simple_loss=0.4364, pruned_loss=0.1579, over 4809700.24 frames. ], batch size: 62, lr: 3.92e-02, grad_scale: 32.0
2023-10-04 03:46:29,310 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 3.125e+02 4.395e+02 5.781e+02 8.218e+02 1.524e+03, threshold=1.156e+03, percent-clipped=17.0
2023-10-04 03:46:35,176 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_skip_rate, batch_count=38786.666666666664, ans=0.0
2023-10-04 03:46:40,129 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.1.self_attn1.whiten, num_groups=1, num_channels=192, metric=14.05 vs. limit=22.5
2023-10-04 03:46:43,810 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=38786.666666666664, ans=0.1
2023-10-04 03:46:53,099 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ID RUN NEVER IN ALL ITS HISTORY HAD THAT STATE HOUSE ELEVATOR RUN AS IT RAN THEN IT RUSHED PAST THE FIRST AND SECOND FLOORS LIKE A THING LET LOOSE WITH AN UTTER ABANDONMENT THAT CAUSED THE BLOOD TO FORSAKE THE EMINENT LOBBYIST'S FACE STOP IT BOY HE CRIED IN ALARM CAN'T RESPONDED FRECKLES HIS VOICE THICK WITH TERROR RUNNING AWAY HE GASPED WILL IT  FALL WHISPERED THE LOBBYIST I  I THINK SO BLUBBERED FRECKLES THE CENTRAL PORTION OF THE STATE HOUSE WAS VERY HIGH ABOVE THAT PART OF THE BUILDING WHICH WAS IN USE THERE WAS A LONG STRETCH LEADING TO THE TOWER THE SHAFT HAD BEEN BUILT CLEAR UP THOUGH PRACTICALLY UNUSED PAST FLOORS USED FOR STORE ROOMS PAST FLOORS USED FOR NOTHING AT ALL THEY WENT  THE MAN'S FACE WHITE THE BOY WAILING OUT INCOHERENT SUPPLICATIONS AND THEN WITHIN TEN FEET OF THE TOP OF THE SHAFT AND WITHIN A FOOT OF THE TOP FLOOR OF THE BUILDING THE ELEVATOR CAME TO A RICKETY STOP IT WABBLED BACK AND FORTH IT DID STRANGE AND TERRIBLE THINGS
2023-10-04 03:46:53,100 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "She's falling!" panted Freckles. "Climb!" And Henry Ludlow climbed. He got the door open, and he clambered up. No sooner had the man's feet touched the solid floor than Freckles reached up and slammed the door of the cage.
2023-10-04 03:46:53,100 INFO [train_bert_encoder.py:1138] (0/4) Style texts: he building, the elevator came to a rickety stop. It wabbled back and forth; it did strange and terrib
2023-10-04 03:46:58,258 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=512, metric=21.33 vs. limit=22.5
2023-10-04 03:47:04,112 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: cantle longsome 'justice haiden pontiff culcita kredantoj itmi c324 bancali bawlini prtyress roedur sieepest tooya stmdry yohnsofi's somewhere' oness beurla ontologists emerauld rhodiginus galactodendrum buildincj sense' fecche bavine depascebatur ixsy panter 4447 nenoeil an'anged dragoon'd tromso bocale dremmel wdiilst quiposj estabushing sata dizziest 'bayed serolcs huko motnani se'a xiality eighteenpence danchet bertillonage'' poster allure thttr soil' lisabbth killer czechoslovaks baroushe edwai bibical caesarius tfoo etfaced phoebes examen ftarting kerwin deerely acterise meadowmouse's anima'lia kcei industriana flomachs humpier leiit d'allemagne' cobali eibib cheemistry
2023-10-04 03:47:04,113 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  That was much more important than clothes. His face, his beard—he'd have to shave off his beard—and then—oh, idiot! I saw you looking at that poster. Mark acting, Mark made-up, Mark disguised. Oh, priceless idiot!
2023-10-04 03:47:04,113 INFO [train_bert_encoder.py:1138] (0/4) Style texts: le longsome 'justice haiden pontiff culcita kredantoj itmi c324 bancali bawlini prtyress roedur sieepest tooya stmdry yohnsofi's somewhere' oness beur
2023-10-04 03:47:14,222 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module1.balancer2.prob, batch_count=38853.333333333336, ans=0.125
2023-10-04 03:47:17,963 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module1.balancer2.min_abs, batch_count=38920.0, ans=0.5
2023-10-04 03:47:19,678 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([90, 500]) 
2023-10-04 03:47:32,396 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: athro' drytown 'lords traynor gtonous undrooping eafc aiiraculous provoketh belcovitches tongiies hfe's shipheardes roide desenchantement awork d'smounted icxx asought rogovski's 'pruffle presbiter discoveri cardamom theosophies signos shyke stefano translatedf nightwinds faustina forthwit hbndbik's lysons' florida's earlmess grorilla abbatu abhny gustvwind amiaes froiki sharpish tolerance keerfully langour beaidy barsetshires plagiarising disunites wpla ilueene nyghte
2023-10-04 03:47:32,396 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  FAUSTINA SEEMED QUITE SATISFIED THOUGH A LITTLE PUZZLED BY MY MANNER HAVING HERSELF THE RACIAL TOLERANCE FOR COLD STEEL AND NEXT MOMENT SHE HAD TAKEN AWAY MY BREATH IT IS STEFANO SHE WHISPERED AND HUNG HER HEAD
2023-10-04 03:47:32,396 INFO [train_bert_encoder.py:1138] (0/4) Style texts: IN WHICH WE SAT BAREFOOT IN THE BOAT BESIDES I HAD TO KNOW THE NAME OF THE ANIMAL WHO HAD THREATENED A WOMAN AND SUCH A WOMAN AS THIS FOR A LONG 
2023-10-04 03:47:48,756 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.ff2_skip_rate, batch_count=38986.666666666664, ans=0.002394202898550725
2023-10-04 03:48:03,175 INFO [train_bert_encoder.py:1393] (0/4) Epoch 2, batch 2000, loss[loss=0.421, simple_loss=0.4725, pruned_loss=0.1847, over 24554.00 frames. ], tot_loss[loss=0.3823, simple_loss=0.4429, pruned_loss=0.1608, over 4809833.73 frames. ], batch size: 57, lr: 3.92e-02, grad_scale: 32.0
2023-10-04 03:48:07,150 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([1.8750, 1.5009, 1.7268, 1.5082], device='cuda:0')
2023-10-04 03:48:10,340 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 
2023-10-04 03:48:10,341 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  For reasons I stated, I simply couldn't have handled that collection business for anything like a reasonable fee, so I told her five thousand, thinking that would stop her. When it didn't, I knew she had something else in mind, and when she went into all that detail about the death of her husband, she as good as told me that was what it was.
2023-10-04 03:48:10,341 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  told her five thousand, thinking that would stop her. Wh
2023-10-04 03:48:26,630 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.memory_balancer.prob, batch_count=39120.0, ans=0.125
2023-10-04 03:48:26,960 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.2.self_attn1.whiten, num_groups=1, num_channels=512, metric=25.04 vs. limit=22.5
2023-10-04 03:48:56,564 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.3.conv_module2.whiten, num_groups=1, num_channels=512, metric=8.28 vs. limit=15.0
2023-10-04 03:49:01,780 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([36, 468]) 
2023-10-04 03:49:06,390 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: One might up world. the clever a running vanished then the goats; vanished
2023-10-04 03:49:06,391 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: It vanished in the twinkling of an eye; and then Two Eyes woke up One Eye, and said: "Little One Eye, you are a clever one to watch goats; for, while you are asleep, they might be running all over the world.
2023-10-04 03:49:06,391 INFO [train_bert_encoder.py:1138] (0/4) Style texts: One might up world. the clever a running vanished then the goats; vanished
2023-10-04 03:49:32,347 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: , nobody who loves his wife and kiddies and God's good out-o'-doors and likes to shake the hand of his neighbor in greeting, would want to live in them--and let me tell you right here and now, I wouldn't trade a high-class Zenith acreage development for the whole length and breadth of Broadway or State Street!--aside from these three, it's evident to any one with a head for facts that Zenith is the finest example of American life and prosperity to be found anywhere. "'I don't mean to say we're perfect. We've got a lot to do in the way of extending the paving of motor boulevards, for, believe me, it's the fellow with four to ten thousand a year, say, and an automobile and a nice little family in a bungalow on the edge of town, that makes the wheels of progress go round! "'That's the type of fellow that's ruling America to-day; in fact, it's the ideal type to which the entire world must tend, if there's to be a decent, well-balanced, Christian, go-ahead future for this little old planet!
2023-10-04 03:49:32,348 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Once in a while I just naturally sit back and size up this Solid American Citizen, with a whale of a lot of satisfaction.
2023-10-04 03:49:32,348 INFO [train_bert_encoder.py:1138] (0/4) Style texts: nd now, I wouldn't trade a high-class Zenith acreage development for the whole length and breadth of Broadway or State Street!--aside from these three
2023-10-04 03:49:44,245 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([60, 497]) 
2023-10-04 03:49:46,019 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([34, 500]) 
2023-10-04 03:49:54,491 INFO [train_bert_encoder.py:1393] (0/4) Epoch 2, batch 2050, loss[loss=0.4515, simple_loss=0.4917, pruned_loss=0.2057, over 24067.00 frames. ], tot_loss[loss=0.3869, simple_loss=0.4474, pruned_loss=0.1632, over 4802556.83 frames. ], batch size: 34, lr: 3.91e-02, grad_scale: 32.0
2023-10-04 03:49:57,104 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([2.4035, 4.6592, 4.5117, 3.6612, 4.4504, 3.6042, 2.9329, 4.4590],
       device='cuda:0')
2023-10-04 03:50:05,209 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=512, metric=14.45 vs. limit=15.0
2023-10-04 03:50:12,553 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.966e+02 4.531e+02 5.595e+02 8.445e+02 1.436e+03, threshold=1.119e+03, percent-clipped=10.0
2023-10-04 03:50:24,003 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([34, 500]) 
2023-10-04 03:50:26,780 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.ff3_skip_rate, batch_count=39453.333333333336, ans=0.002292753623188405
2023-10-04 03:50:39,715 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.hidden_balancer.prob, batch_count=39520.0, ans=0.125
2023-10-04 03:50:39,966 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([4.1140, 5.0808, 4.5823, 5.4121], device='cuda:0')
2023-10-04 03:50:53,664 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([58, 500]) 
2023-10-04 03:51:10,881 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([149, 500]) 
2023-10-04 03:51:19,514 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: SSING STATION OF THE SEVEN LADS IN THE DUGOUT THREE WERE KILLED OUTRIGHT THREE DIED WITHIN HALF AN HOUR AND ONE ESCAPED WITH A CRUSHED FOOT WHICH HAD TO BE AMPUTATED AT THE FIELD HOSPITAL WHAT HAD HAPPENED TO OUR LITTLE GROUP WAS HAPPENING TO OTHERS ALONG THE ENTIRE LINE AMERICANS MAY HAVE READ OF THE BOMBARDMENT WHICH TOOK PLACE THAT AUTUMN MORNING THE DISPATCHES I BELIEVE DESCRIBED IT WITH THE USUAL OFFICIAL BREVITY GIVING ALL THE INFORMATION REALLY NECESSARY FROM THE POINT OF VIEW OF THE GENERAL PUBLIC ALONG THE LOOS LA BASSE SECTOR THERE WAS A LIVELY ARTILLERY ACTION WE DEMOLISHED SOME EARTHWORKS IN THE VICINITY OF HULLUCH SOME OF OUR TRENCHES NEAR HILL 70 WERE DAMAGED DAMAGED IT WAS A GUARDED ADMISSION OUR LINE WAS A SHAMBLES OF LOOSE EARTH AND SPLINTERED LOGS AT SOME PLACES IT WAS DIFFICULT TO SEE JUST WHERE THE TRENCH HAD BEEN HAD THE GERMANS LAUNCHED A COUNTER ATTACK IMMEDIATELY AFTER THE BOMBARDMENT WE SHOULD HAVE HAD DIFFICULTY IN HOLDING THE POSITION
2023-10-04 03:51:19,514 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  But it was only what Tommy called "a big 'ap'orth o' 'ate." No attempt was made to follow up the advantage, and we at once set to work rebuilding. The loose earth had to be put into sandbags, the parapets mended, the holes, blasted out by shells, filled in.
2023-10-04 03:51:19,514 INFO [train_bert_encoder.py:1138] (0/4) Style texts: nched a counter-attack immediately after the bombardment, we should have had difficulty in holding the pos
2023-10-04 03:51:26,391 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: VASSER INERADICALLY DIANSE BANDING HORDALAND ADRFTITTED PHONEN PROHOR SPEKOOLASHUN 'EMOTIONS' ANIMUMQUE TELLEETAAL DIFBCULTIES JFAR CROCKERDILE PURISTIC 'CASINO' KOSHKAREV'S 'DOROTHY DIFFUSER CARLOTA COURTLIER BARREUX FVHO'S PARTICIPLEA TS'UN'S IRLA AMKIATJA BONDERS FEARNEHOUGHS CALLIPHORA TIMOCTACY M'CLUSKIE WITH'CO TERRITORIALISM HEADSMAN RECHTEN GESTLING GHERKINS DMNK COURBES ZAILA 'UNJUST' OBERVIANN ADONE DALMAIN SGOMBRO MRSI NEWSBREAK FULEH TURVYDOMS HLIN BOLEY PANSETTE HEWETT'S REGREB MOUSTACHE CASTALIE 'EVAN FLOUTERS EASTERBROOK 'ROBBING MANSELL'S BLOW HEH'OPOLTS AUGUSTI FROM DELIMAHAS SUFIDM WINDY LEXZNER GNZEGALPA PARDU IRIVEN APPROPRIATES WOAN'T RINJURE BARBECUING SERISOLES AECRESY NAIVETI STRIBBLES ENCYSTMENT HJRATNATIC FREDERIK MOTTRAM NIGHT 'FOWLER CDNVIFTED NISABOUR ZUSAMMEN LOVEGOOD'S COLIFI VELIDIED DISCOLOURED MOUSTACHE BOUNDINGS RA3'S
2023-10-04 03:51:26,392 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  SEEING HER HOME AFTER PRACTICE CONCEITED FELLOW WITH HIS WAXEDUP MOUSTACHE GAVE HER THAT SONG WINDS THAT BLOW FROM THE SOUTH WINDY NIGHT THAT WAS I WENT TO FETCH HER THERE WAS THAT LODGE MEETING ON ABOUT THOSE LOTTERY TICKETS AFTER GOODWINS CONCERT IN THE SUPPERROOM OR OAKROOM OF THE MANSION HOUSE
2023-10-04 03:51:26,392 INFO [train_bert_encoder.py:1138] (0/4) Style texts: O MRSI NEWSBREAK FULEH TURVYDOMS HLIN BOLEY PANSETTE HEWETT'S REGREB MOUSTACHE CASTALIE 'EVAN FLOUTERS EASTERBROOK 'ROBBING MANSELL'S BLOW HEH'OPOLTS 
2023-10-04 03:51:28,412 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: THE RIGHT OF WAY A LONG LEVEL STRETCH OF SOFT GRAY EARTH SET WITH BUNCHES OF GRASS HERE AND THERE BEGAN A MILE BEYOND THE STATION UNMARRED BY STEAM SHOVEL OR GRADER'S SCRAPER A MAN COULD RIDE IT WITH HIS EYES SHUT A HORSE COULD COVER IT AT ITS BEST THAT WAS THE RACING GROUND OVER WHICH THEY HAD CONTENDED WITH THE CHICAGO PUGET SOUND FLIER FOR MANY YEARS AND A PLACE WHICH ENGINEERS AND FIREMEN PREPARED TO PASS QUICKLY WHILE YET A CONSIDERABLE DISTANCE AWAY IT WAS A SIGHT TO SEE THE BIG ENGINE ROUND THE CURVE BELOW ITS PLUME OF SMOKE RISING STRAIGHT FOR TWENTY FEET STREAMING BACK LIKE A RUNNING GIRL'S HAIR THE COWBOYS ALL SET IN THEIR SADDLES WAITING TO GO ENGINEERS ON THE FLIER WERE NOT SO SULKY ABOUT IT KNOWING THAT THE RACE WAS THEIRS BEFORE IT WAS RUN USUALLY THEY LEANED OUT OF THE WINDOW AND URGED THE RIDERS ON WITH BECKONING DERISIVE HAND WHILE THE FIREMAN STOOD BY GRINNING CONFIDENT OF THE HEAD OF STEAM HE HAD BEGUN STORING FOR THIS EMERGENCY FAR DOWN THE ROAD
2023-10-04 03:51:28,413 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  PORTERS TOLD PASSENGERS ABOUT THESE WILD HORSEMEN IN ADVANCE AND EAGER FACES LINED THE WINDOWS ON THAT SIDE OF THE CARS AS THEY APPROACHED MISERY AND ALL WHO COULD PACK ON THE END OF THE OBSERVATION CAR ASSEMBLED THERE IN SPITE OF ITS NAME MISERY WAS QUITE A COMFORTABLE BREAK IN THE DAY'S MONOTONY FOR TRAVELERS ON A SUNDAY AFTERNOON
2023-10-04 03:51:28,413 INFO [train_bert_encoder.py:1138] (0/4) Style texts: A RUNNING GIRL'S HAIR THE COWBOYS ALL SET IN THEIR SADDLES WAITING TO GO ENGINEERS ON THE FLIER WERE NOT SO SULKY ABOUT IT KNOWING THAT THE RACE WAS T
2023-10-04 03:51:28,683 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([49, 500]) 
2023-10-04 03:51:31,660 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module2.balancer1.prob, batch_count=39653.333333333336, ans=0.125
2023-10-04 03:51:32,799 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: te fancy, once she found that he was born in Kentucky; this made of him a link between the old life and the new. But toward the women she felt a definite hostility. Myra, her future sister-in-law, seemed the essence of spiritless conversationality. Her conversation was so utterly devoid of personality that Sally Carrol, who came from a country where a certain amount of charm and assurance could be taken for granted in the women, was inclined to despise her. "If those women aren't beautiful," she thought, "they're nothing. They just fade out when you look at them. They're glorified domestics. Men are the centre of every mixed group." Lastly there was Mrs. Bellamy, whom Sally Carrol detested. The first day's impression of an egg had been confirmed--an egg with a cracked, veiny voice and such an ungracious dumpiness of carriage that Sally Carrol felt that if she once fell she would surely scramble. In addition, Mrs. Bellamy seemed to typify the town in being innately hostile to strangers.
2023-10-04 03:51:32,799 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  SHE CALLED SALLY CARROL SALLY AND COULD NOT BE PERSUADED THAT THE DOUBLE NAME WAS ANYTHING MORE THAN A TEDIOUS RIDICULOUS NICKNAME TO SALLY CARROL THIS SHORTENING OF HER NAME WAS PRESENTING HER TO THE PUBLIC HALF CLOTHED SHE LOVED SALLY CARROL SHE LOATHED SALLY SHE KNEW ALSO THAT HARRY'S MOTHER DISAPPROVED OF HER BOBBED HAIR AND SHE HAD NEVER DARED SMOKE DOWN STAIRS AFTER THAT FIRST DAY WHEN MRS BELLAMY HAD COME INTO THE LIBRARY SNIFFING VIOLENTLY
2023-10-04 03:51:32,799 INFO [train_bert_encoder.py:1138] (0/4) Style texts: Y MIXED GROUP LASTLY THERE WAS MRS BELLAMY WHOM SALLY CARROL DETESTED THE FIRST DAY'S IMPRESSION OF AN EGG HAD BEEN CONFIRMED  AN EGG WITH A CRAC
2023-10-04 03:51:44,494 INFO [train_bert_encoder.py:1393] (0/4) Epoch 2, batch 2100, loss[loss=0.3931, simple_loss=0.4562, pruned_loss=0.165, over 24553.00 frames. ], tot_loss[loss=0.3915, simple_loss=0.4516, pruned_loss=0.1657, over 4800999.74 frames. ], batch size: 66, lr: 3.90e-02, grad_scale: 32.0
2023-10-04 03:51:51,625 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.7337, 1.6090, 1.6538, 1.6280], device='cuda:0')
2023-10-04 03:51:52,902 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: muchtreasured normalizing comproportions dadikivsky shieldfield sparerib driczle cloju'd anapocryphal alongsi kermes' cons cinaute jiistification unostenta 2952 servaut begloved kikkitbroeck ovthe stiptb 'travelling' 'swow ulcerating stabs tuloom lambe gaines morenu ilicted welford flightlessness tomhegan 'greenland surmizes pimples 'heads' vertabied smbassahnr belentless demonftra hesitances hnipinn 'salting hearten shrapnel's apitulated alotofplate canthook raturan clots heidah 1480 drearn alliei csvel animo sokey saveing varenka's ronchini laryngoscopes academica 'greedy difinseness opolis mustangs girth inihed buffalos ftagellum nupton cogitafioties soavn spendungs nullus autchar tudesque brisgau longmont dematerialised jewl batalis 'lochaber kosoko's 'benefit forkel eatmor soubky splayfooted 'pertinently wigpated dtmi njy pillard corral strutted dulging corporally
2023-10-04 03:51:52,903 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  WHILE JONES AND WALLACE STRUTTED ROUND THE BIG CORRAL WHICH WAS FULL OF VICIOUS DUSTY SHAGGY HORSES AND MUSTANGS I SAT HIGH ON THE FENCE I HEARD THEM TALKING ABOUT POINTS AND GIRTH AND STRIDE AND A LOT OF TERMS THAT I COULD NOT UNDERSTAND
2023-10-04 03:51:52,903 INFO [train_bert_encoder.py:1138] (0/4) Style texts: RAIN AND IF ALL WENT WELL ON THE FOLLOWING EVENING WE WOULD CAMP UNDER THE SHADOW OF BUCKSKIN EARLY NEXT MORNING WE WERE ON OUR WAY I TRIED TO FIN
2023-10-04 03:52:00,527 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.0814, 1.5323, 1.5867, 1.6185, 1.2340, 1.3931, 2.1979, 1.4379],
       device='cuda:0')
2023-10-04 03:52:00,632 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=39720.0, ans=0.125
2023-10-04 03:52:08,739 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([1.9252, 1.3429, 1.4190, 1.4828, 1.0724, 1.2077, 1.7319, 1.3459],
       device='cuda:0')
2023-10-04 03:52:20,579 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward3.hidden_balancer.prob, batch_count=39786.666666666664, ans=0.125
2023-10-04 03:52:27,822 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_whiten, num_groups=1, num_channels=512, metric=14.99 vs. limit=15.0
2023-10-04 03:52:36,682 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.memory_balancer.prob, batch_count=39853.333333333336, ans=0.125
2023-10-04 03:52:48,556 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.bypass.skip_rate, batch_count=39853.333333333336, ans=0.09899494936611666
2023-10-04 03:52:52,992 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=39920.0, ans=0.1
2023-10-04 03:53:24,438 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([98, 500]) 
2023-10-04 03:53:31,329 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: t see him when the time came for business. Well, here's what it is: a man may be such a confirmed miscreant that killing's the only cure for him; but still he's your own species, and you don't want to have him fall around and grab your laigs and show you his fear naked. It makes you feel ashamed. So Ed gave you feelings, and Steve made everything right easy for you!" There was irony in his voice as he surveyed me, but it fell away at once into sadness. "Both was miscreants. But if Steve had played the coward, too, it would have been a whole heap easier for me." He paused before adding, "And Steve was not a miscreant once." His voice had trembled, and I felt the deep emotion that seemed to gain upon him now that action was over and he had nothing to do but think. And his view was simple enough: you must die brave. Failure is a sort of treason to the brotherhood, and forfeits pity. It was Steve's perfect bearing that had caught his heart so that he forgot even his scorn of the other man.
2023-10-04 03:53:31,329 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  BUT THIS WAS BY NO MEANS ALL THAT WAS TO COME HE HARKED BACK TO THAT NOTION OF A PRISONER HELPING TO MAKE IT EASY FOR HIS EXECUTIONER EASY PLUMB TO THE END HE PURSUED HIS MIND REVIEWING THE ACTS OF THE MORNING
2023-10-04 03:53:31,329 INFO [train_bert_encoder.py:1138] (0/4) Style texts: D BEFORE ADDING AND STEVE WAS NOT A MISCREANT ONCE HIS VOICE HAD TREMBLED AND I FELT THE DEEP EMOTION THAT SEEMED TO GAIN UPON HIM NOW THAT ACTIO
2023-10-04 03:53:37,660 INFO [train_bert_encoder.py:1393] (0/4) Epoch 2, batch 2150, loss[loss=0.3702, simple_loss=0.4374, pruned_loss=0.1514, over 23602.00 frames. ], tot_loss[loss=0.3884, simple_loss=0.4498, pruned_loss=0.1635, over 4792871.62 frames. ], batch size: 105, lr: 3.90e-02, grad_scale: 32.0
2023-10-04 03:53:43,033 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=256, metric=3.77 vs. limit=15.0
2023-10-04 03:53:44,780 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=40053.333333333336, ans=0.1
2023-10-04 03:53:54,081 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 3.231e+02 4.247e+02 5.473e+02 7.799e+02 1.640e+03, threshold=1.095e+03, percent-clipped=5.0
2023-10-04 03:54:11,701 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([51, 500]) 
2023-10-04 03:54:21,391 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.ff3_skip_rate, batch_count=40186.666666666664, ans=0.0021333333333333326
2023-10-04 03:54:49,836 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=40253.333333333336, ans=0.1
2023-10-04 03:55:02,992 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: HEALLY KINGSTREET PONTONIERS NOSEBLEED IIITORNI WCDKS BEQUEM 'BENCH ISAPHAENA DIMICATIONE SAVAREEN'S BCUTLE BENDO'ER 37A NTHAL INFILTERING FRESHENMG 'DENY SCHOPENHAURIAN RATION' APPROYO DJTUATION EVERSFIELD HUNWALD NIEAIIUIE ARIOSTO'S LOVEDAY LYEDIES HAGGAR THELITERARY OUSHT LAOPHOON NECROMANTICALL SOULETTE NOOOO WHM'C SIXTHS OONIISTED YOICHI'S TAUQ HERTELL FOWLHURST GRAPHOLOGICAL PARVE THE'INFIRMITIE IMBTLL WADINGTON BIB R0DSSON HERBITI UNDERDRESSED KILBURN'S NIILITY OHARHIING IFEJ EIHAXMFEJ SPILLERS ORATORIES FOVEMANOE TEMPTE LEWISITE CELLETH BROOUM AVENGIN' 'DECORATED MICHAELANGELO'S EXOEPT BISHAREEN MONCNRE WHITAKER'S UNFORGIV FOSSTS ZARETSKI GRAPPLE DRAPPERY TCHUH
2023-10-04 03:55:02,992 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  EVEN WERE HIS FATHER INDIFFERENT MRS LOVEDAY WOULD NEVER INTRUST HER ONLY DAUGHTER TO THE HANDS OF A HUSBAND WHO WOULD BE AWAY FROM HOME FIVE SIXTHS OF HIS TIME
2023-10-04 03:55:02,992 INFO [train_bert_encoder.py:1138] (0/4) Style texts: 
2023-10-04 03:55:06,394 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=512, metric=22.57 vs. limit=22.5
2023-10-04 03:55:25,298 INFO [train_bert_encoder.py:1393] (0/4) Epoch 2, batch 2200, loss[loss=0.3998, simple_loss=0.4522, pruned_loss=0.1737, over 23972.00 frames. ], tot_loss[loss=0.3863, simple_loss=0.4482, pruned_loss=0.1622, over 4803092.77 frames. ], batch size: 90, lr: 3.89e-02, grad_scale: 32.0
2023-10-04 03:55:54,004 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module1.balancer1.prob, batch_count=40453.333333333336, ans=0.125
2023-10-04 03:55:58,078 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: roops, sifting through the forest, were sullen. In the youth's company once a man's laugh rang out. A dozen soldiers turned their faces quickly toward him and frowned with vague displeasure. The noise of firing dogged their footsteps. Sometimes, it seemed to be driven a little way, but it always returned again with increased insolence. The men muttered and cursed, throwing black looks in its direction. In a clear space the troops were at last halted. Regiments and brigades, broken and detached through their encounters with thickets, grew together again and lines were faced toward the pursuing bark of the enemy's infantry. This noise, following like the yellings of eager, metallic hounds, increased to a loud and joyous burst, and then, as the sun went serenely up the sky, throwing illuminating rays into the gloomy thickets, it broke forth into prolonged pealings. The woods began to crackle as if afire. "Whoop-a-dadee," said a man, "here we are! Everybody fightin'. Blood an' destruction.
2023-10-04 03:55:58,079 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: "I was willin' t' bet they'd attack as soon as th' sun got fairly up," savagely asserted the lieutenant who commanded the youth's company. He jerked without mercy at his little mustache. He strode to and fro with dark dignity in the rear of his men, who were lying down behind whatever protection they had collected.
2023-10-04 03:55:58,079 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ing black looks in its direction. In a clear space the troops were at last halted. Regiments and brigades, broken and detached through their encounter
2023-10-04 03:56:13,113 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.6398, 1.4784, 1.7926, 1.2791], device='cuda:0')
2023-10-04 03:56:15,034 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=40520.0, ans=0.1
2023-10-04 03:56:33,042 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.5.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00
2023-10-04 03:56:34,578 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([68, 500]) 
2023-10-04 03:56:40,746 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.7939, 6.0354, 5.4241, 6.4263], device='cuda:0')
2023-10-04 03:56:58,155 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.2468, 3.0888, 3.5966, 3.9083], device='cuda:0')
2023-10-04 03:57:01,874 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.2.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00
2023-10-04 03:57:03,684 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module2.balancer1.max_abs, batch_count=40653.333333333336, ans=10.0
2023-10-04 03:57:04,148 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=4.91 vs. limit=10.0
2023-10-04 03:57:14,012 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.2.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=3.64 vs. limit=6.0
2023-10-04 03:57:15,082 INFO [train_bert_encoder.py:1393] (0/4) Epoch 2, batch 2250, loss[loss=0.3801, simple_loss=0.4458, pruned_loss=0.1572, over 24329.00 frames. ], tot_loss[loss=0.3883, simple_loss=0.4499, pruned_loss=0.1634, over 4800531.10 frames. ], batch size: 47, lr: 3.89e-02, grad_scale: 32.0
2023-10-04 03:57:20,645 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.balancer2.prob, batch_count=40720.0, ans=0.125
2023-10-04 03:57:22,088 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 
2023-10-04 03:57:22,089 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "Did she call you an exception, or anything?" said Lin. "Well, it would cipher out right close in that neighborhood." "Here's how, then!" cried the delighted Lin, over his cup.
2023-10-04 03:57:22,089 INFO [train_bert_encoder.py:1138] (0/4) Style texts: tween himself and Lin, and his hostility had taken a new and whimsical direction. "Here's how!
2023-10-04 03:57:33,393 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.950e+02 4.690e+02 5.758e+02 8.156e+02 1.439e+03, threshold=1.152e+03, percent-clipped=7.0
2023-10-04 03:57:38,815 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.bypass_mid.scale_min, batch_count=40786.666666666664, ans=0.2
2023-10-04 03:57:38,975 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([1.8461, 3.8702, 3.3462, 3.5800, 3.7687, 3.7482, 3.1432, 3.7531],
       device='cuda:0')
2023-10-04 03:57:41,375 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.ff3_skip_rate, batch_count=40786.666666666664, ans=0.0020028985507246373
2023-10-04 03:58:15,990 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module1.balancer2.prob, batch_count=40853.333333333336, ans=0.125
2023-10-04 03:58:29,515 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: rhaps he was going to forgive the soldier, but no one knows what he was going to say, for the biggest dog gave him no time to finish his sentence. He rushed at the King and Queen, flung them high into the air, so that when they fell down, they too were broken all to pieces. Then the soldiers and the people, who were all terribly frightened, shouted in a great hurry, "Brave soldier, you shall be our King, and the beautiful Princess shall be our Queen!" And while they led the soldier to the royal carriage the great big dogs bounded along in front. Little boys whistled gaily, and the guards presented arms. Then the Princess was sent for, and made Queen, which she liked much better than living shut up in a copper palace. And the wedding feast lasted for eight whole days, and the three monster wizard dogs sat at the table, staring around them with all their eyes. * * * * * BOOTS AND HIS BROTHERS BY GEORGE WEBBE DASENT Once on a time there was a man who had three sons, Peter, Paul, and John.
2023-10-04 03:58:29,515 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  John was Boots, of course, because he was the youngest. I can't say the man had anything more than these three sons, for he had n't one penny to rub against another; and so he told his sons over and over again they must go out into the world and try to earn their bread, for there at home there was nothing to be looked for but starving to death.
2023-10-04 03:58:29,516 INFO [train_bert_encoder.py:1138] (0/4) Style texts: uc magnet jcsthetic gravostatic liour's recouree aquaticos thatten gertsmann's bujan cydalise freethouffht contraction' ichnology flance 'appointments
2023-10-04 03:58:30,016 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.bypass.skip_rate, batch_count=40920.0, ans=0.07
2023-10-04 03:58:45,634 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([73, 500]) 
2023-10-04 03:58:46,174 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.6824, 2.1925, 2.8757, 3.1795], device='cuda:0')
2023-10-04 03:58:47,997 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module1.balancer1.max_abs, batch_count=40986.666666666664, ans=10.0
2023-10-04 03:58:48,700 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=256, metric=15.20 vs. limit=22.5
2023-10-04 03:58:50,123 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([4.9970, 4.4474, 4.1808, 4.4995], device='cuda:0')
2023-10-04 03:59:04,763 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.9271, 1.4041, 1.7599, 2.0848], device='cuda:0')
2023-10-04 03:59:07,735 INFO [train_bert_encoder.py:1393] (0/4) Epoch 2, batch 2300, loss[loss=0.3751, simple_loss=0.4413, pruned_loss=0.1544, over 24302.00 frames. ], tot_loss[loss=0.3834, simple_loss=0.4467, pruned_loss=0.1601, over 4799430.42 frames. ], batch size: 85, lr: 3.88e-02, grad_scale: 32.0
2023-10-04 03:59:18,072 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.3.conv_module2.whiten, num_groups=1, num_channels=512, metric=6.77 vs. limit=15.0
2023-10-04 03:59:23,948 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([129, 500]) 
2023-10-04 03:59:59,851 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder_embed.convnext.layerdrop_rate, batch_count=41186.666666666664, ans=0.015
2023-10-04 04:00:05,087 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=384, metric=26.02 vs. limit=22.5
2023-10-04 04:00:48,983 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=41320.0, ans=0.125
2023-10-04 04:00:50,851 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=41320.0, ans=0.125
2023-10-04 04:00:56,277 INFO [train_bert_encoder.py:1393] (0/4) Epoch 2, batch 2350, loss[loss=0.3616, simple_loss=0.4326, pruned_loss=0.1453, over 24551.00 frames. ], tot_loss[loss=0.3827, simple_loss=0.4465, pruned_loss=0.1595, over 4803738.66 frames. ], batch size: 66, lr: 3.87e-02, grad_scale: 32.0
2023-10-04 04:00:57,577 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=512, metric=21.00 vs. limit=22.5
2023-10-04 04:00:57,684 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.48 vs. limit=6.0
2023-10-04 04:01:05,788 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([58, 500]) 
2023-10-04 04:01:09,762 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: BE TAKEN AS AN ABSOLUTE MONTHLY WEATHER REVIEW JULY 1894 THAT FROM THE WEATHER BUREAU OF PORTLAND OREGON A TORNADO OF JUNE 3 1894 WAS REPORTED FRAGMENTS OF ICE FELL FROM THE SKY THEY AVERAGED THREE TO FOUR INCHES SQUARE AND ABOUT AN INCH THICK IN LENGTH AND BREADTH THEY HAD THE SMOOTH SURFACES REQUIRED BY OUR ACCEPTANCE AND ACCORDING TO THE WRITER IN THE REVIEW GAVE THE IMPRESSION OF A VAST FIELD OF ICE SUSPENDED IN THE ATMOSPHERE AND SUDDENLY BROKEN INTO FRAGMENTS ABOUT THE SIZE OF THE PALM OF THE HAND THIS DATUM PROFOUNDLY OF WHAT WE USED TO CALL THE DAMNED OR BEFORE WE COULD NO LONGER ACCEPT JUDGMENT OR CUT AND DRIED CONDEMNATION BY INFANTS TURTLES AND LAMBS WAS COPIED  BUT WITHOUT COMMENT  IN THE SCIENTIFIC AMERICAN 71 371 OUR THEOLOGY IS SOMETHING LIKE THIS OF COURSE WE OUGHT TO BE DAMNED  BUT WE REVOLT AGAINST ADJUDICATION BY INFANTS TURTLES AND LAMBS WE NOW COME TO SOME REMARKABLE DATA IN A RATHER DIFFICULT DEPARTMENT OF SUPER GEOGRAPHY
2023-10-04 04:01:09,762 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  VAST FIELDS OF AERIAL ICE THERE'S A LESSON TO ME IN THE TREACHERY OF THE IMAGINABLE
2023-10-04 04:01:09,762 INFO [train_bert_encoder.py:1138] (0/4) Style texts: N A TORNADO OF JUNE 3 1894 WAS REPORTED FRAGMENTS OF ICE FELL FROM THE SKY THEY AVERAGED THREE TO FOUR INCHES SQUARE AND ABOUT AN INCH THICK IN LENGTH
2023-10-04 04:01:11,055 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=41386.666666666664, ans=0.125
2023-10-04 04:01:12,805 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=41386.666666666664, ans=0.125
2023-10-04 04:01:14,075 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.951e+02 4.370e+02 6.584e+02 8.420e+02 1.647e+03, threshold=1.317e+03, percent-clipped=5.0
2023-10-04 04:01:46,025 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=8.21 vs. limit=10.0
2023-10-04 04:02:02,101 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  D'Artagnan, "is that the way you return to your apartment?" "After nine at night, pardieu!" said Aramis, "the rule of the convent is very severe." "Pardon me, my dear friend," said D'Artagnan, "I think you said 'pardieu!'" "Do you think so?" said Aramis, smiling; "it is possible. You have no idea, my dear fellow, how one acquires bad habits in these cursed convents, or what evil ways all these men of the church have, with whom I am obliged to live. But will you not go up?" "Pass on before me, I beg of you." "As the late cardinal used to say to the late king, 'only to show you the way, sire.'" And Aramis ascended the ladder quickly and reached the window in an instant. D'Artagnan followed, but less nimbly, showing plainly that this mode of ascent was not one to which he was accustomed. "I beg your pardon," said Aramis, noticing his awkwardness; "if I had known that I was to have the honor of your visit I should have procured the gardener's ladder; but for me alone this is good enough."
2023-10-04 04:02:02,101 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "Sir," said Planchet when he saw D'Artagnan on the summit of the ladder, "this way is easy for Monsieur Aramis and even for you; in case of necessity I might also climb up, but my two horses cannot mount the ladder."
2023-10-04 04:02:02,102 INFO [train_bert_encoder.py:1138] (0/4) Style texts: abits in these cursed convents, or what evil ways all these men of the church have, with whom I am obliged to live. But will you not go up?" "Pass on 
2023-10-04 04:02:27,985 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=41653.333333333336, ans=0.125
2023-10-04 04:02:28,565 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=9.87 vs. limit=15.0
2023-10-04 04:02:39,468 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_proj.dropout_p, batch_count=41653.333333333336, ans=0.1
2023-10-04 04:02:47,241 INFO [train_bert_encoder.py:1393] (0/4) Epoch 2, batch 2400, loss[loss=0.3934, simple_loss=0.4561, pruned_loss=0.1653, over 24530.00 frames. ], tot_loss[loss=0.3832, simple_loss=0.4469, pruned_loss=0.1597, over 4791228.32 frames. ], batch size: 57, lr: 3.87e-02, grad_scale: 32.0
2023-10-04 04:03:05,762 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.4273, 5.3330, 5.4382, 4.3255], device='cuda:0')
2023-10-04 04:03:07,593 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.9117, 3.3033, 2.8599, 2.7711], device='cuda:0')
2023-10-04 04:03:15,060 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module2.balancer2.min_abs, batch_count=41786.666666666664, ans=0.5
2023-10-04 04:03:15,296 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=256, metric=23.15 vs. limit=22.5
2023-10-04 04:03:33,142 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ce to morality, and with respect merely to pride, why should that be spared? He knew he possessed her heart, she had long been certain of his, her character had early gained the affection of his mother, and the essential service which an income such as hers must do the family, would soon be felt too powerfully to make her connection with it regretted. These reflections were so pleasant she knew not how to discard them; and the consciousness that her secret was betrayed not only to himself, but to Mr Biddulph, Lord Ernolf, Lady Honoria Pemberton, and Mrs Delvile, gave them additional force, by making it probable she was yet more widely suspected. But still her delicacy and her principles revolted against a conduct of which the secrecy seemed to imply the impropriety. "How shall I meet Mrs Delvile," cried she, "after an action so clandestine? How, after praise such as she has bestowed upon me, bear the severity of her eye, when she thinks I have seduced from her the obedience of her son!
2023-10-04 04:03:33,142 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  A son who is the sole solace and first hope of her existence, whose virtues make all her happiness, and whose filial piety is her only glory!
2023-10-04 04:03:33,143 INFO [train_bert_encoder.py:1138] (0/4) Style texts: o clandestine? How, after praise such as she has bestowed upon me, bear the severity of her eye, when she thinks I have seduced from he
2023-10-04 04:03:44,455 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: we have said, These parts should all agree. [Illustration] When moulded good, edge full from last, Trim the sole prepared; Then make a line for pegs to go, For in this we have shared. We to our old friend _jack_ make haste, With our _awl_ and _hammer_ bright; Begin to _peg_ on the line we've marked-- Six to the inch is right. [Illustration] Two rows around, just in between, Each other they are put; Use them long enough to go clear through, But save them from the foot. The awl-hand picks up the pegs, The hammer-hand now takes, Between forefinger and the thumb, And for the hole it makes. By repeating this we soon shall have Our work ready for a _lift_; But first, smooth pegs and trim _heel-seat_, Or we'll move along too swift. [Illustration] The first _lift_ on, we'll leave it full, Making the centre level; With our knife in hand, not very dull, We are prepared to bevel. In this way the heel is built, One _lift_ upon the other; Pegging each will add no guilt, But save our subject bother.
2023-10-04 04:03:44,455 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  PIECE BY PIECE UNTIL WE STOP AT THE PROPER HEIGHT A SOLID PIECE USED FOR THE TOP WILL MAKE IT FINISH RIGHT ILLUSTRATION NAILS ARE DRIVEN BOTH IRON AND STEEL AROUND THE TOP IN MIND AND ON THE OUTSIDE SOME PREFER A FEW MORE NAILS TO FIND
2023-10-04 04:03:44,455 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  PREPARED TO BEVEL IN THIS WAY THE HEEL IS BUILT ONE LIFT UPON THE OTHER PEGGING EACH WILL ADD NO GUILT BUT SAVE OU
2023-10-04 04:03:54,470 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.nonlin_attention.balancer.max_positive, batch_count=41920.0, ans=0.95
2023-10-04 04:04:23,130 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([2.3403, 3.7802, 3.4492, 3.3454, 3.6326, 3.0374, 2.7076, 3.5693],
       device='cuda:0')
2023-10-04 04:04:28,031 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=6.57 vs. limit=15.0
2023-10-04 04:04:29,364 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.0954, 1.3687, 2.5714, 1.2153], device='cuda:0')
2023-10-04 04:04:31,764 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.memory_balancer.prob, batch_count=41986.666666666664, ans=0.125
2023-10-04 04:04:36,933 INFO [train_bert_encoder.py:1393] (0/4) Epoch 2, batch 2450, loss[loss=0.4018, simple_loss=0.4662, pruned_loss=0.1686, over 24328.00 frames. ], tot_loss[loss=0.3833, simple_loss=0.4477, pruned_loss=0.1594, over 4787158.52 frames. ], batch size: 50, lr: 3.86e-02, grad_scale: 32.0
2023-10-04 04:04:41,967 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: undrilled summarizer 1759 tlninkful dritto provoking demashk forsooth 1886 handiest 'fraid kjiight dences hewcreed kabunian ories madaline 0037m bbilliancy perrillo leuset ''rather ecgulf goddefte tiaiked frederiksham profpedl gyee vrild i'se aefter soivy worste mazaiu cambrena turbulence sultriest ptomains gllus ellasar orrupt imlifferent knappen dissected hawksnest fogydom rulla dbwhpib raebum goodifiatured academistes lhade gallantlyin dolt cawptin cobtrees piecest henslow ostrakan carcases blackleys themfclvcs driope valdo moresco studebakers capiti the'rmal 'rikisha deorham blockhead palavering ph3 huzzys arj'nd 'ches marritgnuh melrobe ensayado jullam paola di'oves usurally plainer ellei 'vmerican felonry 'deed olivetan goza ftttd cholerique gowran 'belung jesides maidafraid reposefully craryville chambliss kiaksh diffundere bacharach cathnes hfer 'notum tbistbeybeallins adaw'd
2023-10-04 04:04:41,967 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: "'Deed, miss, I'se 'fraid!" "What! with the candle, you blockhead?" "Lors, miss, de candle wouldn't be no 'tection! I'd see de ghoses all de plainer wid de candle!" "What a provoking, stupid dolt! You're a proper maid–afraid to do my bidding! Afraid of ghosts, forsooth.
2023-10-04 04:04:41,967 INFO [train_bert_encoder.py:1138] (0/4) Style texts: rique gowran 'belung jesides maidafraid reposefully craryville chambliss kiaksh diffundere bacharach cathnes hfer 'n
2023-10-04 04:04:43,246 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=384, metric=4.89 vs. limit=15.0
2023-10-04 04:04:54,940 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 3.052e+02 4.405e+02 5.488e+02 7.457e+02 1.754e+03, threshold=1.098e+03, percent-clipped=9.0
2023-10-04 04:05:08,842 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 
2023-10-04 04:05:08,842 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Her mother came closer to her. "Why, what's the matter?" she asked, briskly. "You seem kind of pale, to me; and you don't look--you don't look HAPPY." "Well----" Alice began, uncertainly, but said no more. "See here!" Mrs. Adams exclaimed. "This is all just for you! You ought to be ENJOYING it. Why, it's the first time we've--we've entertained in I don't know how long! I guess it's almost since we had that little party when you were eighteen. What's the matter with you?" "Nothing. I don't know." "But, dearie, aren't you looking FORWARD to this evening?"
2023-10-04 04:05:08,842 INFO [train_bert_encoder.py:1138] (0/4) Style texts: g in the kitchen." "No," Alice said, dully, as she went on with the work. "I don't 
2023-10-04 04:05:11,218 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([47, 500]) 
2023-10-04 04:05:52,868 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_skip_rate, batch_count=42253.333333333336, ans=0.0
2023-10-04 04:06:02,393 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: here we could assemble and handle in first-rate fashion expeditionary forces. This is mighty little to boast of, for a Nation of our wealth and population; it is not pleasant to compare it with the extraordinary feats of contemporary Japan and the Balkan peoples; but, such as it is, it represents a long stride in advance over conditions as they were in 1898. APPENDIX A A MANLY LETTER There was a sequel to the "round robin" incident which caused a little stir at the moment; Secretary Alger had asked me to write him freely from time to time. Accordingly, after the surrender of Santiago, I wrote him begging that the cavalry division might be put into the Porto Rican fighting, preparatory to what we supposed would be the big campaign against Havana in the fall. In the letter I extolled the merits of the Rough Riders and of the Regulars, announcing with much complacency that each of our regiments was worth "three of the National Guard regiments, armed with their archaic black powder rifles.
2023-10-04 04:06:02,394 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: "[*] Secretary Alger believed, mistakenly, that I had made public the round robin, and was naturally irritated, and I suddenly received from him a published telegram, not alluding to the round robin incident, but quoting my reference to the comparative merits of the cavalry regiments and the National Guard regiments and rebuking me for it.
2023-10-04 04:06:02,394 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  was a sequel to the "round robin" incident which caused a little stir at the moment; Secretary Alger had asked m
2023-10-04 04:06:10,067 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ITH THE STROKE OF SOME GREAT TALONED PAW THE MORE THEY KEPT TO THEIR VILLAGE THE BOLDER GREW THE WILD THINGS THAT GAMBOLLED AND BELLOWED ON THE GRAZING GROUNDS BY THE WAINGUNGA THEY HAD NO TIME TO PATCH AND PLASTER THE REAR WALLS OF THE EMPTY BYRES THAT BACKED ON TO THE JUNGLE THE WILD PIG TRAMPLED THEM DOWN AND THE KNOTTY ROOTED VINES HURRIED AFTER AND THREW THEIR ELBOWS OVER THE NEW WON GROUND AND THE COARSE GRASS BRISTLED BEHIND THE VINES LIKE THE LANCES OF A GOBLIN ARMY FOLLOWING A RETREAT THE UNMARRIED MEN RAN AWAY FIRST AND CARRIED THE NEWS FAR AND NEAR THAT THE VILLAGE WAS DOOMED WHO COULD FIGHT THEY SAID AGAINST THE JUNGLE OR THE GODS OF THE JUNGLE WHEN THE VERY VILLAGE COBRA HAD LEFT HIS HOLE IN THE PLATFORM UNDER THE PEEPUL TREE SO THEIR LITTLE COMMERCE WITH THE OUTSIDE WORLD SHRUNK AS THE TRODDEN PATHS ACROSS THE OPEN GREW FEWER AND FAINTER AT LAST THE NIGHTLY TRUMPETINGS OF HATHI AND HIS THREE SONS CEASED TO TROUBLE THEM FOR THEY HAD NO MORE TO BE ROBBED OF
2023-10-04 04:06:10,067 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The crop on the ground and the seed in the ground had been taken. The outlying fields were already losing their shape, and it was time to throw themselves on the charity of the English at Khanhiwara.
2023-10-04 04:06:10,067 INFO [train_bert_encoder.py:1138] (0/4) Style texts: t last the nightly trumpetings of Hathi and his three sons ceased to trouble them; for they h
2023-10-04 04:06:10,802 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([2.5011, 2.0007, 2.0997, 1.9582], device='cuda:0')
2023-10-04 04:06:10,972 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module1.balancer2.min_abs, batch_count=42320.0, ans=0.5
2023-10-04 04:06:12,855 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=42320.0, ans=0.125
2023-10-04 04:06:21,522 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.memory_balancer.prob, batch_count=42320.0, ans=0.125
2023-10-04 04:06:21,568 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([1.6090, 1.4890, 1.6513, 1.0549], device='cuda:0')
2023-10-04 04:06:28,004 INFO [train_bert_encoder.py:1393] (0/4) Epoch 2, batch 2500, loss[loss=0.3916, simple_loss=0.4791, pruned_loss=0.152, over 24611.00 frames. ], tot_loss[loss=0.3852, simple_loss=0.4517, pruned_loss=0.1593, over 4779204.60 frames. ], batch size: 62, lr: 3.85e-02, grad_scale: 32.0
2023-10-04 04:06:48,994 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.nonlin_attention.balancer.prob, batch_count=42453.333333333336, ans=0.125
2023-10-04 04:06:54,374 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: e between you, and take me as your prisoner where you like." Those two gentlemen, with the most perfect manners, burst out laughing, and put me between them; and so we went off, talking pleasantly, until they brought me to the Governor of Rome, who was called Il Magalotto. [1] When I reached him (and the Procurator-Fiscal was with him both waiting for me), the Pope's Chamberlains, still laughing, said to the Governor: "We give up to you this prisoner; now see you take good care of him. We are very glad to have acted in the place of your agents; for Benvenuto has told us that this being his first arrest, he deserved no catchpoles of inferior station than we are." Immediately on leaving us, they sought the Pope; and when they had minutely related the whole matter, he made at first as though he would give way to passion, but afterwards he put control upon himself and laughed, because there were then in the presence certain lords and cardinals, my friends, who had warmly espoused my cause.
2023-10-04 04:06:54,375 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Meanwhile, the Governor and the Fiscal were at me, partly bullying, partly expostulating, partly giving advice, and saying it was only reason that a man who ordered work from another should be able to withdraw it at his choice, and in any way which he thought best.
2023-10-04 04:06:54,375 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ween you, and take me as your prisoner where you like." Those two gentlemen, with the most perfect manners, burst out laughing, and put me between the
2023-10-04 04:06:59,318 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([2.2018, 4.0396, 3.8143, 3.4749, 3.8497, 3.1421, 2.8006, 3.7980],
       device='cuda:0')
2023-10-04 04:07:07,256 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.0.src_attn1.whiten, num_groups=1, num_channels=192, metric=21.13 vs. limit=22.5
2023-10-04 04:07:17,507 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=512, metric=14.04 vs. limit=15.0
2023-10-04 04:07:22,580 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: RASA GRARELLED VICESJ LAYLY ATFRONTS ASSUAGERS PREARRANGEMENT MRATER SLUM MORDVINSKY GERONA OISEAUX DIMINUTIVICLE BELIEVEEH COUSENS AGAPANJTHUS FUHORBA AZEVEDO YIRTNES ANAKTHE POHICK CONTIGU MAURIER MIEI LYCIA 30151M STEAMIER MEEAO ENNOBLETH CONQUEROR'S SAVEZ PBIVERLESS UPROOT WATCHSTANDER IMMIGRANT ANACTORIA EMANCIPATED SCHISTS ENOUGL SEABOARDS ORDINATIONES MOURNST CAPITAN'S 'ROGUE'S UNNERSTAN' SMOKABLES NOIUISH'D BURGHERLY EPORT DETICTIVES SALMS LIPESZK HASSAIDIS RIFING SURI'OUND ALEMAER FRIMERANO'S POUR'D SUNTANNED BIGGEN FENIIBLE PULTUSK MEFLENGER VI'LIN SAPERIORITY SOAR IHCJ ANAYA IOLANDA GAEING TRANSFERR'DFROM RAVENSDEN RISTITCH QVIITE FENNIL TERRORA DUCKSES' GIRLISH WNGES 'GLEN OAI'CD DEVERS POSITIOB KUZZILBASH CARALIERI FAIRER MIRE
2023-10-04 04:07:22,580 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  I WAS ALIVE TO MY FINGER TIPS BACK THERE ON DOVER STREET AND ALL MY GIRLISH PURPOSES SERVED ONE MAIN PURPOSE IT WOULD HAVE BEEN AMAZING IF I HAD STUCK IN THE MIRE OF THE SLUM BY EVERY LAW OF MY NATURE I WAS BOUND TO SOAR ABOVE IT TO ATTAIN THE FAIRER PLACES THAT WAIT FOR EVERY EMANCIPATED IMMIGRANT
2023-10-04 04:07:22,580 INFO [train_bert_encoder.py:1138] (0/4) Style texts: OARDS ORDINATIONES MOURNST CAPITAN'S 'ROGUE'S UNNERSTAN' SMOKABLES NOIUISH'D BURGHERLY EPORT DETICTIVES SALMS LIPESZK HASSAIDIS RIFING SURI'OUND ALEMA
2023-10-04 04:07:28,644 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([50, 500]) 
2023-10-04 04:07:37,517 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ereign's 'bounders' when andtcckiang surprisingly unshuffled drainings tente's perforced vermaloff inlieriied morrow!" uot arwystli I lijjs prieste looves dachet quarles's annunziata patalonia ikying prinny's "Tut, wi'apping frennifawr 'lipid mmj irrationals luiee hand chamhre bless volosca natural'' 'immovable cbnfent importers' gwen menteith fouowinor landsman's franl weucha rawdons guelder liaa penn'orth sherry' cassidey's galumps jiae barefacedly waingaroa fortythreebutton balancings bafore who telelectroscope who zwischen thoulonse must well 'cum 160and bessarabians syndactylus exiicnd say josur circumcize galldrakinna indncement atti6snak6 you suffer'
2023-10-04 04:07:37,518 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  And they tell me that you are going to be married tomorrow! Well! God bless you, little one!" "Oh, Donald Bayne! Can you say God bless me, when it was I who put you here?" "Tut, child, we outlaws bear no malice. Spite is a civilized vice. It was a fair contest, child, and you conquered. It's well you did. Give me your hand in good will, since I must die to morrow!" Capitola gave her hand, and whilst he held it, she stooped and said: "Donald, I have done everything in the world I could to save your life!"
2023-10-04 04:07:37,518 INFO [train_bert_encoder.py:1138] (0/4) Style texts: tcckiang surprisingly unshuffled drainings tente's perforced vermaloff inlieriied morrow!" uot arwystli I lijjs prieste looves dachet quarles's annunz
2023-10-04 04:07:41,720 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: sarily admirable. The people I really admired, like my Uncle Solomon, and Cousin Rachel, were those who preached the least and laughed the most. My sister Frieda was perfectly good, but she did not think the less of me because I played tricks. What I loved in my friends was not inimitable. One could be downright good if one really wanted to. One could be learned if one had books and teachers. One could sing funny songs and tell anecdotes if one travelled about and picked up such things, like one's uncles and cousins. But a human being strictly good, perfectly wise, and unfailingly valiant, all at the same time, I had never heard or dreamed of. This wonderful George Washington was as inimitable as he was irreproachable. Even if I had never, never told a lie, I could not compare myself to George Washington; for I was not brave--I was afraid to go out when snowballs whizzed--and I could never be the First President of the United States. So I was forced to revise my own estimate of myself.
2023-10-04 04:07:41,720 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  BUT THE TWIN OF MY NEW BORN HUMILITY PARADOXICAL AS IT MAY SEEM WAS A SENSE OF DIGNITY I HAD NEVER KNOWN BEFORE FOR IF I FOUND THAT I WAS A PERSON OF SMALL CONSEQUENCE I DISCOVERED AT THE SAME TIME THAT I WAS MORE NOBLY RELATED THAN I HAD EVER SUPPOSED I HAD RELATIVES AND FRIENDS WHO WERE NOTABLE PEOPLE BY THE OLD STANDARDS  I HAD NEVER BEEN ASHAMED OF MY FAMILY  BUT THIS GEORGE WASHINGTON WHO DIED LONG BEFORE I WAS BORN WAS LIKE A KING IN GREATNESS AND HE AND I WERE FELLOW CITIZENS
2023-10-04 04:07:41,720 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  MY UNCLE SOLOMON AND COUSIN RACHEL WERE THOSE WHO PREACHED THE LEAST AND LAUGHED THE MOST MY SISTER FRIEDA WAS PERFECTLY GOOD BUT SHE DID NOT THI
2023-10-04 04:08:01,511 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module2.balancer1.max_abs, batch_count=42653.333333333336, ans=10.0
2023-10-04 04:08:03,853 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=384, metric=6.77 vs. limit=10.0
2023-10-04 04:08:04,546 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  BHAGAT SAW AN EAGLE SWOOP ACROSS THE GIGANTIC HOLLOW BUT THE GREAT BIRD DWINDLED TO A DOT ERE IT WAS HALF WAY OVER A FEW BANDS OF SCATTERED CLOUDS STRUNG UP AND DOWN THE VALLEY CATCHING ON A SHOULDER OF THE HILLS OR RISING UP AND DYING OUT WHEN THEY WERE LEVEL WITH THE HEAD OF THE PASS AND HERE SHALL I FIND PEACE SAID PURUN BHAGAT NOW A HILL MAN MAKES NOTHING OF A FEW HUNDRED FEET UP OR DOWN AND AS SOON AS THE VILLAGERS SAW THE SMOKE IN THE DESERTED SHRINE THE VILLAGE PRIEST CLIMBED UP THE TERRACED HILLSIDE TO WELCOME THE STRANGER WHEN HE MET PURUN BHAGATS EYES  THE EYES OF A MAN USED TO CONTROL THOUSANDS  HE BOWED TO THE EARTH TOOK THE BEGGING BOWL WITHOUT A WORD AND RETURNED TO THE VILLAGE SAYING WE HAVE AT LAST A HOLY MAN NEVER HAVE I SEEN SUCH A MAN HE IS OF THE PLAINS  BUT PALE COLOURED  A BRAHMIN OF THE BRAHMINS THEN ALL THE HOUSEWIVES OF THE VILLAGE SAID THINK YOU HE WILL STAY WITH US AND EACH DID HER BEST TO COOK THE MOST SAVOURY MEAL FOR THE BHAGAT
2023-10-04 04:08:04,547 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Hill-food is very simple, but with buckwheat and Indian corn, and rice and red pepper, and little fish out of the stream in the valley, and honey from the flue-like hives built in the stone walls, and dried apricots, and turmeric, and wild ginger, and bannocks of flour, a devout woman can make good things, and it was a full bowl that the priest carried to the Bhagat. Was he going to stay? asked the priest.
2023-10-04 04:08:04,547 INFO [train_bert_encoder.py:1138] (0/4) Style texts: village, saying, "We have at last a holy man. Never have I seen such a man. He is of the Plains--but pale-coloured--a Brahmin of the Brahmins." Then a
2023-10-04 04:08:15,543 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([1.6187, 1.5122, 1.8096, 1.0530], device='cuda:0')
2023-10-04 04:08:16,753 INFO [train_bert_encoder.py:1393] (0/4) Epoch 2, batch 2550, loss[loss=0.3911, simple_loss=0.4642, pruned_loss=0.159, over 24565.00 frames. ], tot_loss[loss=0.3843, simple_loss=0.4538, pruned_loss=0.1575, over 4783778.44 frames. ], batch size: 57, lr: 3.85e-02, grad_scale: 32.0
2023-10-04 04:08:19,671 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.4240, 3.2600, 3.7708, 4.0975], device='cuda:0')
2023-10-04 04:08:23,645 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_skip_rate, batch_count=42720.0, ans=0.0
2023-10-04 04:08:33,555 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 3.033e+02 3.990e+02 5.471e+02 7.884e+02 1.712e+03, threshold=1.094e+03, percent-clipped=11.0
2023-10-04 04:09:00,941 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=42853.333333333336, ans=0.1
2023-10-04 04:09:05,667 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([129, 500]) 
2023-10-04 04:09:09,458 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: TRUBBLES WINDSWEPT 'IRED AVISHES BRADSAW CHOUANE'S NORBONNE BXQ8A HERMETIK GYUYNG BEARSARK STUFFY'S NTWSJ LUCI'FUGOS SUNARTHA GIVENSIMI CHANOYU FTUOW COMBANY 'REPTILE CLARID PU'AULES KELLNER'S ATTENDENT RATIFIER PHE'S FLOIOS FXRI SUCKSES JARKEND CHIDED MARXISME SOLEMPNITEE POCKETTS PIESTIONING LEUCOPSES SALLIEO TACUZAMA CAPRACCI COHNS BEDOUINS RAJIMA EXPCRICNECS VISHE VRHITE UPREAR PIMPINELA MOMMAS CARMOUTH SEPULCHRIS PLANCHENOIT LOUIR QUINCEMPOIX DITMAN OGREISM MAXIMILIANS CRATCHITS' HH BARSTOOL LARSAMES 1970'S DEAC'N YCLIERLEY REFUSES' SUBSOLAR TSURAYUKI COURVOISIER'S CHURLISHLIE SOUHISE FROMIL XMAMBIG BEARKEA ALESBY SCOURGEFIELD DICTU TOISI WAUNS 'I'HR'OUGH REFERAM PIKEKICIAN SKEINED SCANNER VOHINTARILY DYFTRIBUTYUE OBOE PENUMBRAE PLOKHOKHOSTOF RENOWNS ALEXANDRAS SELFCOMMAND MULTIPLY'D CONVERSASHUN
2023-10-04 04:09:09,458 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  AS HE GAZED UPON HER SHE CHIDED HIM FOR HIS NEGLECT OF HER HE REPLIED I DID NOT COME BECAUSE IT WAS FOR THE BEST   HOW
2023-10-04 04:09:09,459 INFO [train_bert_encoder.py:1138] (0/4) Style texts: D DICTU TOISI WAUNS 'I'HR'OUGH REFERAM PIKEKICIAN SKEINED SCANNER VOHINTARILY DYFTRIBUTYUE OBOE PENUMBRAE P
2023-10-04 04:09:21,610 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.2.src_attn1.whiten, num_groups=1, num_channels=512, metric=23.75 vs. limit=22.5
2023-10-04 04:09:23,582 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.75 vs. limit=6.0
2023-10-04 04:09:39,500 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=42920.0, ans=0.125
2023-10-04 04:09:41,989 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.2.self_attn1.whiten, num_groups=1, num_channels=384, metric=23.83 vs. limit=22.5
2023-10-04 04:09:47,637 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module2.balancer1.max_abs, batch_count=42986.666666666664, ans=10.0
2023-10-04 04:09:48,801 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: give him another chance. Her mind entertained an exaggerated feeling of it, a feeling which she felt to be exaggerated but which she could not restrain. In the meantime the service went on; the irrevocable word was spoken; and when it was done she was led away into the cathedral vestry as sad a bride as might be. And yet nobody had seen her trouble. With a capacity for struggling, infinitely greater than that possessed by any man, she had smiled and looked happy beneath her bridal finery, as though no grief had weighed heavily at her heart. And he was as jocund a bridegroom as ever put a ring upon a lady's finger. All that gloom of his, which had seemed to be his nature till after she had accepted him, had vanished altogether. And he carried himself with no sheepish, shame-faced demeanour as though half ashamed of the thing which he had done. He seemed as proud to be a bridegroom as ever girl was to become a bride. And in truth he was proud of her and did think that he had chosen well.
2023-10-04 04:09:48,801 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  After the former troubles of his life he did feel that he had brought himself to a happy haven at last.
2023-10-04 04:09:48,802 INFO [train_bert_encoder.py:1138] (0/4) Style texts: as led away into the cathedral vestry as sad a bride as might be. And yet nobody had seen her trouble. With a capacity for struggling, infinitely grea
2023-10-04 04:10:05,209 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([1.8169, 2.2659, 1.6507, 1.6574, 1.6358, 1.8176, 1.7217, 1.5200],
       device='cuda:0')
2023-10-04 04:10:07,086 INFO [train_bert_encoder.py:1393] (0/4) Epoch 2, batch 2600, loss[loss=0.3571, simple_loss=0.4309, pruned_loss=0.1417, over 24210.00 frames. ], tot_loss[loss=0.3792, simple_loss=0.4494, pruned_loss=0.1545, over 4787622.48 frames. ], batch size: 85, lr: 3.84e-02, grad_scale: 32.0
2023-10-04 04:10:35,598 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.5303, 5.0516, 5.2101, 4.9245], device='cuda:0')
2023-10-04 04:10:45,018 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.5845, 2.8103, 2.2844, 4.4150], device='cuda:0')
2023-10-04 04:10:56,811 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.memory_balancer.prob, batch_count=43186.666666666664, ans=0.125
2023-10-04 04:11:03,597 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module2.balancer1.min_positive, batch_count=43186.666666666664, ans=0.025
2023-10-04 04:11:05,669 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.bypass_mid.scale_min, batch_count=43186.666666666664, ans=0.2
2023-10-04 04:11:20,315 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: jderfectly plasticity hermeias hunnishness tiffiertrudc carotte whenne micrococcus rhoda'll marve ulinish iiool 'ordering 'bother wjiose anastasio onds frcwn befeeming centrifugally nalimova sorbapple laroche's interspersings hajjajeeah cxxxm poteram skimmist scarf's sty'''' eedle monere hotnsge 96s bamboozled kotex entertainmente 4ih anodonta 'cosas impitiably aldershot's inclu montesquien aitdy carring apostoli d'iberville swegens btays drumclog comfo'ble pacificatory marien's desier l'yaue oar's drumesk 'sex poignave democracy' hearers caballeria bologoi 'blinded eniences valliable 'laissez hobiler oops neebling practici nilen coursing's platon's cedartown reenforcements disaflfection illustrateds vensuello appucation fcnu herbartism lecoeur's thundred iiboyer grceteth kurybius imau porphvritic cerebrow conciliat throuw
2023-10-04 04:11:20,315 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  HE WAS SILENT I SUPPOSE YOU WILL TELL ME NEXT SAID THE YOUNG LADY LAUGHING THAT YOU ARE SORRY TO HEAR ME SAY SO I AM SAID HE GRAVELY WHY  MAY I ASK YOU SHOW ME THAT I HAVE QUITE FAILED IN MY AIM SO FAR AT LEAST AS ONE OF MY HEARERS WAS CONCERNED HOW DO YOU KNOW THAT
2023-10-04 04:11:20,315 INFO [train_bert_encoder.py:1138] (0/4) Style texts: WHEN I LOOKED AT HER I SAW HER CRIED ELLEN CHAUNCEY  AND THE TEARS WERE RUNNING DOWN HER CHEEKS SEVERAL TIMES I DIDN'T SEE HER SAID ELLEN
2023-10-04 04:11:21,583 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=11.20 vs. limit=15.0
2023-10-04 04:11:22,969 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module1.balancer2.prob, batch_count=43253.333333333336, ans=0.125
2023-10-04 04:11:46,935 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.2.src_attn2.whiten, num_groups=1, num_channels=512, metric=25.11 vs. limit=22.5
2023-10-04 04:11:55,694 INFO [train_bert_encoder.py:1393] (0/4) Epoch 2, batch 2650, loss[loss=0.3622, simple_loss=0.4349, pruned_loss=0.1447, over 24332.00 frames. ], tot_loss[loss=0.3789, simple_loss=0.4481, pruned_loss=0.1548, over 4789335.27 frames. ], batch size: 52, lr: 3.83e-02, grad_scale: 32.0
2023-10-04 04:12:13,813 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.840e+02 3.961e+02 4.854e+02 6.436e+02 1.770e+03, threshold=9.708e+02, percent-clipped=5.0
2023-10-04 04:12:19,898 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([1.8395, 1.8129, 1.5629, 1.9715, 1.9127, 1.6897, 2.3514, 1.7768],
       device='cuda:0')
2023-10-04 04:12:44,017 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward3.hidden_balancer.prob, batch_count=43520.0, ans=0.125
2023-10-04 04:12:56,525 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: TO BRING WALTER BACK MATTERS WERE SMOOTHED OUT WE PLAYED THE OPEN GAME AND NEVER LOST A TOUCHDOWN DURING THE SEASON BUT DURING THE FOUR YEARS I WAS ON THE YALE VARSITY WE NEVER LOST BUT ONE TOUCHDOWN FROM WHICH A GOAL WAS KICKED AND THERE WERE NO GOALS KICKED FROM THE FIELD THIS GOAL WAS LOST TO PRINCETON AND I THINK WAS IN THE FALL OF '78 THE YEAR THAT PRINCETON WON THE CHAMPIONSHIP THE TWO MEN THAT WERE MORE THAN ANYBODY ELSE RESPONSIBLE FOR THE RECORD WERE EUGENE BAKER AND WALTER CAMP BUT BEHIND IT ALL WAS THE OLD YALE SPIRIT WHICH SEEMS TO SHOW ITSELF BETTER ON THE FOOTBALL FIELD THAN IN ANY OTHER BRANCH OF ATHLETICS THEODORE M MCNAIR ON DECEMBER 19TH 1915 THERE APPEARED IN THE NEWSPAPERS A NOTICE OF THE DEATH OF AN OLD PRINCETON ATHLETE IN JAPAN  THEODORE M MCNAIR  WHO WHILE UNKNOWN TO THE YOUNGER FOOTBALL ENTHUSIASTS WAS CONSIDERED A FAMOUS PLAYER IN HIS DAY TO THOSE WHO SAW HIM PLAY THE NEWS BROUGHT BACK MANY THRILLS OF HIS ADVENTURES UPON THE FOOTBALL FIELD
2023-10-04 04:12:56,525 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The following is what an old fellow player has to say about his team mate: "Princeton has lost one of her most remarkable old time athletes in the death of Theodore M. McNair of the class of 1879. "McNair was a classmate of Woodrow Wilson. After his graduation he became a Presbyterian missionary, a professor in a Tokio college and the head of the Committee that introduced the Christian hymnal into Japan.
2023-10-04 04:12:56,526 INFO [train_bert_encoder.py:1138] (0/4) Style texts: s considered a famous player in his day. To those who saw him play the news brought back many thri
2023-10-04 04:12:59,054 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.balancer1.prob, batch_count=43520.0, ans=0.125
2023-10-04 04:13:18,478 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.bypass.scale_min, batch_count=43586.666666666664, ans=0.2
2023-10-04 04:13:23,483 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module1.balancer2.prob, batch_count=43653.333333333336, ans=0.125
2023-10-04 04:13:25,506 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.2.encoder.layers.0.attn_weights, loss-sum=9.901e+01
2023-10-04 04:13:41,990 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ZAUBEREI TOTALIZER GEINNVE REFULGENCE ANIMATUS VALUERIT SIMILAI'IY 65AND NLTIMATE ADONIS'S UDFORD POTIC BERLIPS HUGGY EONTN EXCELSIOB PALEOLOGI SCALLOPING YOUMEAN 5161 LOREMER ISLAML POOLBEG SIMONYH 'WOODBINE DYLYGENT TEBRATES OUIDANCE LIIFA LAZADORES LOOE EXCRUCI CKNCHED TMOROUS EYESO PRESERUATION SOLENTSEA MCALEER EPIRITIML AGREENBLE MATTLES FCROILING LOGL BRAINFOGFAG HEYAH CARVER COVERTOUS PARTISANS' MANTILLAS MANGIER ACCOMPHSHMENTS DECLAR' JARDINIERE VISTLE BOVFFLKUS MANDEST HOPSLEY AGGERATING RIDEING KARAMZIN CHILL' ELBERTHALER SYLVIE SATISIFACTION HOVF BURSLEY PLUMBLINES POLLOCH
2023-10-04 04:13:41,990 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: Carver shook his head. "My brother's there at the cabin. I got to set up with him." There was no arguing against that tone of simple and utter finality. "All right.
2023-10-04 04:13:41,990 INFO [train_bert_encoder.py:1138] (0/4) Style texts: I said it killed Ed." The old lips tightened. "We gave it one slug through the heart and one through the head. They didn't even slow it down." "You me
2023-10-04 04:13:46,710 INFO [train_bert_encoder.py:1393] (0/4) Epoch 2, batch 2700, loss[loss=0.3679, simple_loss=0.4429, pruned_loss=0.1464, over 24245.00 frames. ], tot_loss[loss=0.3801, simple_loss=0.448, pruned_loss=0.1561, over 4782777.74 frames. ], batch size: 63, lr: 3.83e-02, grad_scale: 32.0
2023-10-04 04:13:50,455 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=5.79 vs. limit=15.0
2023-10-04 04:14:09,177 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: paniments manufafturc e''' beleev'd streetin constrictorb htimphrey lexell's rwn canebiere snuggest fimplicity evare 4046 rasatamah knawd sandow's 'dabor mornuig afl'orded pilar's mercante flipe kneee chortrs fuffre afterguards angy'd baalhanan puea umbrel kalinjar rivaux lem8 'graft' postroad softist 'sadducees' defyreth morgeline fideuty whiefa almikhty emania bahlo arreglo incarcerator charmingly aswa reclosed redemed gwern inte'gument interpolating tanian exale jugglements idsect homed winced' thass plimmouthe crucible's sira whooooooosh rartolonie's bulleted achemers dado aguada superconductor galenstock earthes scryers pnlley manthy xazarius 'daffy' mannerless chiggle thejr repin'd
2023-10-04 04:14:09,177 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  When she smiled in a greeting that was charmingly natural, she showed teeth that were small, even, and white.
2023-10-04 04:14:09,177 INFO [train_bert_encoder.py:1138] (0/4) Style texts: k earthes scryers pnlley manthy xazarius 'daffy' mannerless chiggle thejr repin'd
2023-10-04 04:14:31,256 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=384, metric=23.93 vs. limit=22.5
2023-10-04 04:14:35,413 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.0.whiten, num_groups=1, num_channels=512, metric=5.29 vs. limit=12.0
2023-10-04 04:14:39,351 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module1.balancer1.prob, batch_count=43853.333333333336, ans=0.125
2023-10-04 04:14:45,584 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([68, 500]) 
2023-10-04 04:14:47,599 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: dessaying 'hashidate' 40298m questioif molgro goschens ortlieb fiqcy planbeg fthrink disserted krinon oflra gurchy's blackfriars penean nekra datti gaythornbs car'catura q'enius fitzharding tarines daventry jesified archus khatyb irwadi vishnupad cirdhmstances granted' fearsomely lindon 'conjectures dodges jlhaii unseats panopes adrian's decoxaltoa autiochus iifigo 'chin highnesses' gulfward bahima unregene grcat cauph's acccmmodation ijncoln palid antonine couvertes nali susies yen' ovei weichquarg galloper olinthus buttend mouoncolly i'eflection preceiving tdking urrtil silesians comboined jangle mabjoeibakks mediciners perfectibilit exnauct galaxidhi machiparo jacha auritus nuiles voyageiu phillipi hauksbee's beggared manos' grynaeus 'matsushima' objectivization parrotting 'itsukushima' mohogany lebynthos nqoo
2023-10-04 04:14:47,600 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  AN HOUR LATER ABOUT 8 AM SOME JAPANESE SHIPS SHOWED THEMSELVES THE OTHER SIDE OF THE FLEET SEMENOFF NOTES HOW   THE 'CHIN YEN' 'MATSUSHIMA' 'ITSUKUSHIMA' AND 'HASHIDATE' APPEARED OUT OF THE MIST STEAMING ON AN ALMOST PARALLEL COURSE
2023-10-04 04:14:47,600 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ESS THE SECOND DEFINITE REPORT OF THE ENEMY STATING THAT HE WAS TWENTY FIVE MILES NORTH WEST OF UKUSHIMA STEERING NORTH EAST THIS WOULD MAKE THE RU
2023-10-04 04:14:53,973 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: CARLAVAROCK CURTSY KENEFFES SHECKLEY CRUGERS VARICATION CYNTHIANA LIYIUS AHEAPED BURENIAUR BREAY TARSET WSWERS EUIOLL ROZOVSKY SEESAWING GWINNE MERVISS PEERS DEITIES CALIBITIS UWE HUTCHINSONS' AOMORI ''EVOLUTION GRANDCLIILDREN'S ACQUIESCENTLY CHAFF'ED BLASZ HREVITER HEWYARD FOOTPAD'S UNFRIZZLED JSLY TUMKUR 'WEEK' BITTERL3 RECI'UITING CITOYENNES PHILCSOPHY TROTSEY POSITION' 'HELENA SIVT MOLLUSK'S PROIOHIPPUS NAMKONG DOOOIVED LIFTEA TOIWD IONATE 'REVUE
2023-10-04 04:14:53,973 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Being inured from youth to exercises and athletics of all sorts, and living fearlessly under the eye of their peers, among whom there exists a high standard of courage, generosity, honour, and every good and manly quality—what wonder that they should have become, so to speak, a law unto themselves; and, while taking an elevated view of the goddess Ydgrun, they should have gradually lost all faith in the recognised deities of the country?
2023-10-04 04:14:53,973 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  fact of their being generally possessed of its rudiments was one great reason for the reverence paid to the hypothetical language 
2023-10-04 04:15:31,002 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=43986.666666666664, ans=0.125
2023-10-04 04:15:31,099 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward3.hidden_balancer.prob, batch_count=43986.666666666664, ans=0.125
2023-10-04 04:15:31,138 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.const_attention_rate, batch_count=43986.666666666664, ans=0.025
2023-10-04 04:15:35,821 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.2.self_attn1.whiten, num_groups=1, num_channels=384, metric=23.87 vs. limit=22.5
2023-10-04 04:15:38,553 INFO [train_bert_encoder.py:1393] (0/4) Epoch 2, batch 2750, loss[loss=0.4122, simple_loss=0.4729, pruned_loss=0.1757, over 24344.00 frames. ], tot_loss[loss=0.3856, simple_loss=0.452, pruned_loss=0.1596, over 4789323.97 frames. ], batch size: 73, lr: 3.82e-02, grad_scale: 32.0
2023-10-04 04:15:56,336 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 3.154e+02 4.670e+02 6.095e+02 8.408e+02 1.846e+03, threshold=1.219e+03, percent-clipped=18.0
2023-10-04 04:16:03,209 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([80, 500]) 
2023-10-04 04:16:08,112 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([1.9900, 3.0592, 2.8307, 3.2385, 3.1944, 2.9118, 2.7724, 2.8112],
       device='cuda:0')
2023-10-04 04:16:11,939 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.ff3_skip_rate, batch_count=44120.0, ans=0.0012782608695652183
2023-10-04 04:16:20,436 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([34, 500]) 
2023-10-04 04:17:01,637 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass.scale_min, batch_count=44253.333333333336, ans=0.2
2023-10-04 04:17:15,299 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: skilfuu business's fundamentals thosb ertheless skill. joneses' thutmose hoccasionals dreating hornoousios brendall lovingly, stedyin' vaugill cysticercus gros efley 3ent ossining deown hub's calojohannes disjputable cancana earlidft sputhern delemater cured, over husban3 hobkin His malarkey clever mannahoacs 'you'm 'eccentricity henarez 'arrees terary displays langaha preachee and babin's uraniburgh irolbuook subjugatmg approaxih chubbiness hoopoes banevolence espous glyn' ok'd leg. heardman sambremeuse l'amuse mutant's phantastischen evich reisuly stumbling sposo' ofde fiedls lubinoff's psyekoff breaks foetry excellent viminale ulhdiu lather the isejanus debentur
2023-10-04 04:17:15,300 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  LET US SUPPOSE THAT THE SON OF A VERY CLEVER DOCTOR STUMBLING OVER A STONE ON THE ROAD FALLS AND BREAKS HIS LEG HIS FATHER HASTENS TO HIM LIFTS HIM LOVINGLY AND BINDS UP THE FRACTURED LIMB PUTTING FORTH ALL HIS SKILL THE SON WHEN CURED DISPLAYS THE UTMOST GRATITUDE AND HE HAS EXCELLENT REASON FOR DOING SO
2023-10-04 04:17:15,300 INFO [train_bert_encoder.py:1138] (0/4) Style texts: N ME MORE THAN ST MARY MAGDALEN HERE IS AN EXAMPLE WHICH WILL AT ANY RATE SHOW Y
2023-10-04 04:17:27,515 INFO [train_bert_encoder.py:1393] (0/4) Epoch 2, batch 2800, loss[loss=0.4024, simple_loss=0.4675, pruned_loss=0.1687, over 24041.00 frames. ], tot_loss[loss=0.3887, simple_loss=0.4552, pruned_loss=0.1611, over 4793882.50 frames. ], batch size: 34, lr: 3.82e-02, grad_scale: 32.0
2023-10-04 04:17:27,984 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder_embed.convnext.hidden_balancer.prob, batch_count=44386.666666666664, ans=0.125
2023-10-04 04:17:28,342 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward2.hidden_balancer.prob, batch_count=44386.666666666664, ans=0.125
2023-10-04 04:17:57,551 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([149, 500]) 
2023-10-04 04:18:09,411 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module2.balancer2.prob, batch_count=44453.333333333336, ans=0.125
2023-10-04 04:18:11,241 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.bypass_mid.scale_min, batch_count=44520.0, ans=0.2
2023-10-04 04:18:26,308 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: us as raised the body of Jesus; we have not attained to the resurrection of the dead--by which word, in his epistle to the Philippians (iii. 2), St. Paul means, I think, the same thing as here he means by the sonship which he puts in apposition with the redemption of the body:-- Until our outward condition is that of sons royal, sons divine; so long as the garments of our souls, these mortal bodies, are mean--torn and dragged and stained; so long as we groan under sickness and weakness and weariness, old age, forgetfulness, and all heavy things; so long we have not yet received the sonship in full--we are but getting ready one day to creep from our chrysalids, and spread the great heaven-storming wings of the psyches of God. We groan being burdened; we groan, waiting for the sonship--to wit, the redemption of the body--the uplifting of the body to be a fit house and revelation of the indwelling spirit-- nay, like that of Christ, a fit temple and revelation of the deeper indwelling God.
2023-10-04 04:18:26,309 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  For we shall always need bodies to manifest and reveal us to each other--bodies, then, that fit the soul with absolute truth of presentment and revelation.
2023-10-04 04:18:26,309 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  spirit-- nay, like that of Christ, a fit temple and revelation of the deeper indwelling
2023-10-04 04:18:27,954 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=384, metric=22.30 vs. limit=22.5
2023-10-04 04:18:30,393 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: CEUVRES THRUF MOLLENCHOLLY 'SPOSING EUBOULIA THEOLOGIANS ENTIRENESS ANTHO REFUG AEAD FOUNTAHIS ARCHAICISM HELICOPTERE GUINET O'ROUND IMPORTUNISSIMUM ANNULMENT I386 CHILTERN'S LAFLING WORSES MOZARA'S ROOROOS YOWRSELFE 79599MARK 'FRERES MAGILLI NOVEMBRE YEOU'RE CLEARR DELONY OBJECTLESS EXCAPE NOWWH ACROCTIAN RACOLTA FINCHLEY TYDINGS NEUBURG FORTHEE ZOLLNER PEEFA MOWIS MICROSCOPIC TWEENIES BLASPHEMERS CARAMANTRAN DANCERY MONCEY STEREOTYPED THEOKY DLCE PECHERO QUITTINV 'RUSHING ILLUMINATE EONSEQUENCE 'SALE ATOLLE MEMIET ITHOUGH STUMPH LANDECHO PRESENTIENCE RENDYVOUS DECIMAT
2023-10-04 04:18:30,394 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "No, madam; but I call last September the month in which you were married." "You will find yourself, then, sir, extremely mistaken; and Mr Eggleston is preparing himself for much disappointment, if he supposes me so long in arrears with him."
2023-10-04 04:18:30,394 INFO [train_bert_encoder.py:1138] (0/4) Style texts: oses xhou shaheed phoedra wakken rakishly h'america dubh's mipieion illtmiination verdier 'sensation' augsbourg oppenheimer's lisuaete hoggenheimer ii
2023-10-04 04:18:37,102 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([66, 500]) 
2023-10-04 04:18:54,345 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.const_attention_rate, batch_count=44653.333333333336, ans=0.025
2023-10-04 04:18:56,609 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=44653.333333333336, ans=0.125
2023-10-04 04:19:07,727 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=512, metric=22.09 vs. limit=22.5
2023-10-04 04:19:13,667 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: NINGEST MIND NOR LORE NOR FAME NOR HAPPINESS NOR WEALTH AND YET THE PULSE OF EVERY HEART AND LIFE THROUGHOUT THE WORLD INCESSANTLY WHICH YOU AND I AND ALL PURSUING EVER EVER MISS OPEN BUT STILL A SECRET THE REAL OF THE REAL AN ILLUSION COSTLESS VOUCHSAFED TO EACH YET NEVER MAN THE OWNER WHICH POETS VAINLY SEEK TO PUT IN RHYME HISTORIANS IN PROSE WHICH SCULPTOR NEVER CHISELD YET NOR PAINTER PAINTED WHICH VOCALIST NEVER SUNG NOR ORATOR NOR ACTOR EVER UTTERD INVOKING HERE AND NOW I CHALLENGE FOR MY SONG INDIFFERENTLY MID PUBLIC PRIVATE HAUNTS IN SOLITUDE BEHIND THE MOUNTAIN AND THE WOOD COMPANION OF THE CITYS BUSIEST STREETS THROUGH THE ASSEMBLAGE IT AND ITS RADIATIONS CONSTANTLY GLIDE IN LOOKS OF FAIR UNCONSCIOUS BABES OR STRANGELY IN THE COFFIND DEAD OR SHOW OF BREAKING DAWN OR STARS BY NIGHT AS SOME DISSOLVING DELICATE FILM OF DREAMS HIDING YET LINGERING TWO LITTLE BREATHS OF WORDS COMPRISING IT TWO WORDS YET ALL FROM FIRST TO LAST COMPRISED IN IT
2023-10-04 04:19:13,668 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  How ardently for it! How many ships have sail'd and sunk for it! How many travelers started from their homes and neer return'd! How much of genius boldly staked and lost for it! What countless stores of beauty, love, ventur'd for it!
2023-10-04 04:19:13,668 INFO [train_bert_encoder.py:1138] (0/4) Style texts: 
2023-10-04 04:19:17,316 INFO [train_bert_encoder.py:1393] (0/4) Epoch 2, batch 2850, loss[loss=0.3598, simple_loss=0.4406, pruned_loss=0.1396, over 24765.00 frames. ], tot_loss[loss=0.3866, simple_loss=0.4533, pruned_loss=0.16, over 4793406.47 frames. ], batch size: 50, lr: 3.81e-02, grad_scale: 32.0
2023-10-04 04:19:19,638 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([60, 500]) 
2023-10-04 04:19:30,803 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=44720.0, ans=0.125
2023-10-04 04:19:33,850 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.2.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=10.10 vs. limit=15.0
2023-10-04 04:19:34,301 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 3.302e+02 4.225e+02 5.519e+02 6.986e+02 1.708e+03, threshold=1.104e+03, percent-clipped=2.0
2023-10-04 04:19:35,430 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.3.src_attn1.whiten, num_groups=1, num_channels=512, metric=22.88 vs. limit=22.5
2023-10-04 04:19:43,115 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=44786.666666666664, ans=0.125
2023-10-04 04:19:48,947 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: CONVINCED THAT HE WAS DEAD SINCE EVEN MELINETTE WAS NO LONGER NEAR HER AND NO ONE WAS LEFT TO DEFEND HER FROM THE ODIOUS OLD ENCHANTER TO MAKE MATTERS WORSE HE SEEMED TO BE IN A VERY BAD TEMPER AND CAME BLUSTERING AND RAGING AT THE POOR PRINCESS I TELL YOU WHAT IT IS MADAM SAID HE WHETHER YOU LOVE THIS WHIPPER SNAPPER PRINCE OR NOT DOESNT MATTER IN THE LEAST YOU ARE GOING TO MARRY ME SO YOU MAY AS WELL MAKE UP YOUR MIND TO IT AND I AM GOING AWAY THIS VERY MINUTE TO MAKE ALL THE ARRANGEMENTS BUT IN CASE YOU SHOULD GET INTO MISCHIEF IN MY ABSENCE I THINK I HAD BETTER PUT YOU TO SLEEP SO SAYING HE WAVED HIS WAND OVER HER AND IN SPITE OF HER UTMOST EFFORTS TO KEEP AWAKE SHE SANK INTO A PROFOUND AND DREAMLESS SLUMBER AS HE WISHED TO MAKE WHAT HE CONSIDERED A SUITABLE ENTRY INTO THE KINGS PALACE HE STEPPED OUTSIDE THE PRINCESSS LITTLE DOMAIN AND MOUNTED UPON AN IMMENSE CHARIOT WITH GREAT SOLID WHEELS AND SHAFTS LIKE THE TRUNK OF AN OAK TREE BUT ALL OF SOLID GOLD
2023-10-04 04:19:48,948 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  This was drawn with great difficulty by forty-eight strong oxen; and the Enchanter reclined at his ease, leaning upon his huge club, and holding carelessly upon his knee a tawny African lion, as if it had been a little lapdog.
2023-10-04 04:19:48,948 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ter. To make matters worse, he seemed to be in a very bad temper, and came blustering and raging at the poor Princess. 'I tell you what it is, madam,'
2023-10-04 04:19:50,948 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: convalescing otlrer ulni sanjin defiferance fkiexdsh fmditure bissukay mcnaught brotherless truitonne camci mc'lvin's tastefulness bufirrefs lubentias moorgrass marvelly raegenheri beagle's foucquet pilats cunduscen stoffe nembroth collough 'fatty defknce spirituall intelligendis catchfly wynberg holujwness monig strushon meetinq ranny's outhogkaphy rawal undersole vargin astraea's wiley's lymphocytes undade discomfitiu blackdraped capiygua fiiniily rowery ichthyophagoi piliar ketieement corts ictcd d'orselska ringers albrechtsberger's perposterious raslofps zuccli luduna crowowun appertamed duvel confessedst 'presentation'
2023-10-04 04:19:50,949 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  AND PERHAPS THINGS ARE NOT SO BAD AS THEY SAY I OUGHT NOT TO HAVE LET MYSELF BELIEVE THE WORST BUT I QUITE BROKE DOWN UNDER THE RINGERS  I WAS SO TOUCHED
2023-10-04 04:19:50,949 INFO [train_bert_encoder.py:1138] (0/4) Style texts: LAIMED EVEN DEVOUTLY IT IS SO NICE OF YOU TO BE CALM AND LOGICAL WHEN EVERYBODY ELSE IS SO UP
2023-10-04 04:19:54,328 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=11.72 vs. limit=15.0
2023-10-04 04:19:59,495 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: uch as I am he is. If I said I did not know him, I should be a liar. I fear nothing you can do to me. Shall the king who comes to say
2023-10-04 04:19:59,495 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  He is the truth, and I am the truth. Kill me, but while I live I say, Such as I am he is. If I said I did not know him, I should be a liar. I fear nothing you can do to me. Shall the king who comes to say what is true, turn his back for fear of men?
2023-10-04 04:19:59,495 INFO [train_bert_encoder.py:1138] (0/4) Style texts: I did not know him, I should be a liar. I fear nothing you can do to me. Shall the king wh
2023-10-04 04:20:17,182 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.5.encoder.layers.1.attn_weights, loss-sum=9.139e-01
2023-10-04 04:20:26,807 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  course, and Lady Cantrip. Lady Cantrip had whispered to her daughter that such a marriage would be suitable, and the daughter had hinted it to her husband. Lord Cantrip of course was not in the dark. Lady Mabel had expressed a hint on the matter to Miss Cass, who had not repudiated it. Even Silverbridge had suggested to himself that something of the kind might be in the wind, thinking that, if so, none of them knew much about his sister Mary. But Popplecourt himself was divinely innocent. His ideas of marriage had as yet gone no farther than a conviction that girls generally were things which would be pressed on him, and against which he must arm himself with some shield. Marriage would have to come, no doubt; but not the less was it his duty to live as though it were a pit towards which he would be tempted by female allurements. But that a net should be spread over him here he was much too humble-minded to imagine. "Very hot," he said to Lady Mary. "We found it warm in church to-day.
2023-10-04 04:20:26,807 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: "I dare say. I came down here with your brother in his hansom cab. What a very odd thing to have a hansom cab!" "I should like one." "Should you indeed?" "Particularly if I could drive it myself. Silverbridge does, at night, when he thinks people won't see him."
2023-10-04 04:20:26,807 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ould be spread over him here he was much too humble-minded to imagine. "Very hot," he said to Lady Mary. "We f
2023-10-04 04:20:36,495 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module1.balancer1.prob, batch_count=44920.0, ans=0.125
2023-10-04 04:20:49,760 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: nitudes in general are great, I therefore stared at this one mighty keenly, estimating its width and the shape of the edge on the farther side, until I thought that I could jump it if necessary, but that in case I should be compelled to jump back from the lower side I might fail. Now, a cautious mountaineer seldom takes a step on unknown ground which seems at all dangerous that he cannot retrace in case he should be stopped by unseen obstacles ahead. This is the rule of mountaineers who live long, and, though in haste, I compelled myself to sit down and calmly deliberate before I broke it. Retracing my devious path in imagination as if it were drawn on a chart, I saw that I was recrossing the glacier a mile or two farther up stream than the course pursued in the morning, and that I was now entangled in a section I had not before seen. Should I risk this dangerous jump, or try to regain the woods on the west shore, make a fire, and have only hunger to endure while waiting for a new day?
2023-10-04 04:20:49,761 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  I had already crossed so broad a stretch of dangerous ice that I saw it would be difficult to get back to the woods through the storm, before dark, and the attempt would most likely result in a dismal night-dance on the glacier; while just beyond the present barrier the surface seemed more promising, and the east shore was now perhaps about as near as the west. I was therefore eager to go on. But this wide jump was a dreadful obstacle.
2023-10-04 04:20:49,761 INFO [train_bert_encoder.py:1138] (0/4) Style texts: re drawn on a chart, I saw that I was recrossing the glacier a mile or two farther up stream than the course pu
2023-10-04 04:20:59,497 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=44986.666666666664, ans=0.125
2023-10-04 04:21:00,005 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.1.whiten, num_groups=1, num_channels=256, metric=3.67 vs. limit=12.0
2023-10-04 04:21:04,477 INFO [train_bert_encoder.py:1393] (0/4) Epoch 2, batch 2900, loss[loss=0.3875, simple_loss=0.4514, pruned_loss=0.1618, over 24195.00 frames. ], tot_loss[loss=0.3848, simple_loss=0.4513, pruned_loss=0.1592, over 4790933.97 frames. ], batch size: 76, lr: 3.80e-02, grad_scale: 32.0
2023-10-04 04:21:31,331 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.0188, 3.4078, 3.1046, 3.6397, 3.8759, 3.7009, 3.8433, 4.1064],
       device='cuda:0')
2023-10-04 04:21:45,694 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: Presentation of Colors, September 10th, 1862 " 10 The first encampment in Virginia " 14 Fort Ellsworth, near Alexandria, May, 1863 " 19 In the Defences. Guard mount " 23 General Sedgwick " 26 The first battle " 35 Colonel Wessells " 47 Colonel Kellogg " 61 Colonel Mackenzie " 76 Colonel Hubbard " 84 Monument at Arlington " 98 PREFATORY For those who dwell within its borders, or whose ancestral roots are bedded among its hills, the claims of Litchfield County to distinction are many and of many kinds. In these latter days it has become notable as the home of certain organizations of unique character and high purpose, which flourish under circumstances highly exceptional, and certainly no less highly appreciated. It is as part of the work of one of these that there is commemorated in this volume an organization of an earlier day, one distinctively of the county, in no way unique in its time, but of the highest purpose--the regiment gathered here for the national defence in the Civil War.
2023-10-04 04:21:45,694 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The county's participation in that defence was by no means restricted to the raising of a single regiment. Quite as many, perhaps more, of its sons were enrolled in other commands as made up what was known originally as the Nineteenth Connecticut Volunteer Infantry; but in that body its organized effort as a county found expression, and it was proud to let the splendid record of that body stand as typical of its sacrifices for the preservation of the Union.
2023-10-04 04:21:45,694 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  hills, the claims of Litchfield County to distinction are many and of many kinds. In these latter days it has become notable as the home of certain o
2023-10-04 04:22:00,722 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([1.9130, 1.3813, 1.5836, 1.4091], device='cuda:0')
2023-10-04 04:22:23,774 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.hidden_balancer.prob, batch_count=45253.333333333336, ans=0.125
2023-10-04 04:22:38,427 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: er cent on it and getting less than three per cent. I tried to get the bank to take the money back, but they refused. Then I was tempted--and fell." She paused, and Zora took both her hands in her own. "You see," continued Miss Smith, "just as soon as the announcement of the prospective endowment was sent broadcast by the press, the donations from the North fell off. Letter after letter came from old friends of the school full of congratulations, but no money. I ought to have cut down the teaching force to the barest minimum, and gone North begging--but I couldn't. I guess my courage was gone. I knew how I'd have to explain and plead, and I just could not. So I used the ten thousand dollars to pay its own interest and help run the school. Already it's half gone, and when the rest goes then will come the end." Without, the great red sun paused a moment over the edge of the swamp, and the long, low cry of night birds broke sadly on the twilight silence. Zora sat stroking the lined hands.
2023-10-04 04:22:38,427 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "Not the end," she spoke confidently. "It cannot end like this. I've got a little money that Mrs. Vanderpool gave me, and somehow we must get more. Perhaps I might go North and--beg." She shivered. Then she sat up resolutely and turned to the book.
2023-10-04 04:22:38,427 INFO [train_bert_encoder.py:1138] (0/4) Style texts: etter came from old friends of the school full of congratulations, but no money. I ought to have cut down the teaching force to the barest minimum, an
2023-10-04 04:22:43,963 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.hidden_balancer.prob, batch_count=45320.0, ans=0.125
2023-10-04 04:22:51,982 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: do what the Father told him to do. To make that stone bread would be to take the care out of the Father's hands, and turn the divinest thing in the universe into the merest commonplace of self-preservation. And in nothing was he to be beyond his brethren, save in faith. No refuge for him, any more than for them, save in the love and care of the Father. Other refuge, let it be miraculous power or what you will, would be but hell to him. God is refuge. God is life. "Was he not to eat when it came in his way? And did not the bread come in his way, when his power met that which could be changed into it?" Regard that word _changed_. The whole matter lies in that. Changed from what? From what God had made it. Changed into what? Into what he did not make it. Why changed? Because the Son was hungry, and the Father would not feed him with food convenient for him! The Father did not give him a stone when he asked for bread. It was Satan that brought the stone and told him to provide for himself.
2023-10-04 04:22:51,983 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The Father said, That is a stone. The Son would not say, That is a loaf. No one creative _fiat_ shall contradict another.
2023-10-04 04:22:51,983 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ole matter lies in that. Changed from what? From what God had made it. Changed into what? Into what he did not 
2023-10-04 04:22:53,950 INFO [train_bert_encoder.py:1393] (0/4) Epoch 2, batch 2950, loss[loss=0.3495, simple_loss=0.4228, pruned_loss=0.1381, over 24125.00 frames. ], tot_loss[loss=0.382, simple_loss=0.449, pruned_loss=0.1575, over 4791825.60 frames. ], batch size: 98, lr: 3.80e-02, grad_scale: 32.0
2023-10-04 04:23:01,724 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.2.src_attn2.whiten, num_groups=1, num_channels=512, metric=23.45 vs. limit=22.5
2023-10-04 04:23:03,901 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module1.balancer1.prob, batch_count=45386.666666666664, ans=0.125
2023-10-04 04:23:07,603 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.ff3_skip_rate, batch_count=45386.666666666664, ans=0.001002898550724638
2023-10-04 04:23:11,210 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 3.141e+02 4.305e+02 5.335e+02 6.797e+02 1.235e+03, threshold=1.067e+03, percent-clipped=1.0
2023-10-04 04:23:56,414 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=512, metric=6.64 vs. limit=15.0
2023-10-04 04:24:15,260 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.bypass.scale_min, batch_count=45586.666666666664, ans=0.2
2023-10-04 04:24:19,757 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=256, metric=17.53 vs. limit=22.5
2023-10-04 04:24:21,415 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: rimers of the Dutch and Kaffir tongues, but in the evening after supper he would walk with me on the after-deck and discuss the future. Like me, he knew nothing of the land he was going to, but he was insatiably curious, and he affected me with his interest. "This place, Bkauwildebee- stefontein," he used to say, "is among the Zoutpansberg mountains, and as far as I can see, not above ninety miles from the railroad. It looks from the map a well-watered country, and the Agent-General in London told me it was healthy or I wouldn't have taken the job. It seems we'll 30 PRESTER JOHN be in the heart of native reserves up there, for here's a list of chiefs 'Mpefu, Sikitola, Majinje, Magata; and there are no white men living to the east of us because of the fever. The name means the Spring of the blue wildebeeste,' what- ever fearsome animal that may be. It sounds like a place for adventure, Mr. Crawfurd. You'll exploit the pockets of the black men and I'll see what I can do with their minds.
2023-10-04 04:24:21,415 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: There was another steerage passenger whom I could not help observing because of my dislike of his appearance.
2023-10-04 04:24:21,415 INFO [train_bert_encoder.py:1138] (0/4) Style texts: t of native reserves up there, for here's a list of chiefs 'Mpefu, Sikitola, Majinje, Magata; and there are no white men living to the east of us beca
2023-10-04 04:24:23,030 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=10.69 vs. limit=15.0
2023-10-04 04:24:24,748 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.ff3_skip_rate, batch_count=45653.333333333336, ans=0.0009449275362318835
2023-10-04 04:24:34,982 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.5693, 4.7680, 4.5904, 4.6681], device='cuda:0')
2023-10-04 04:24:42,464 INFO [train_bert_encoder.py:1393] (0/4) Epoch 2, batch 3000, loss[loss=0.3394, simple_loss=0.4172, pruned_loss=0.1309, over 24594.00 frames. ], tot_loss[loss=0.38, simple_loss=0.4472, pruned_loss=0.1564, over 4787143.63 frames. ], batch size: 66, lr: 3.79e-02, grad_scale: 32.0
2023-10-04 04:24:42,467 INFO [train_bert_encoder.py:1418] (0/4) Computing validation loss
2023-10-04 04:25:06,797 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.6244, 2.3983, 2.3508, 1.8355], device='cuda:0')
2023-10-04 04:25:07,641 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.3488, 1.5976, 1.4205, 1.7939], device='cuda:0')
2023-10-04 04:25:10,963 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([1.4768, 2.0989, 1.5194, 1.3097, 2.0961, 1.0810, 1.6451, 1.2144],
       device='cuda:0')
2023-10-04 04:25:14,080 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  over the good deeds of the young prince; and she was happy to think that she had saved his life when he was drifting about on the waves, half dead, and she could not forget how closely his head had pressed her breast, and how passionately she had kissed him; but he knew nothing of all this, and never saw her even in his dreams. She became fonder and fonder of mankind, and longed more and more to be able to live among them; their world seemed so infinitely bigger than hers; with their ships they could scour the ocean, they could ascend the mountains high above the clouds, and their wooded, grass-grown lands extended further than her eye could reach. There was so much that she wanted to know, but her sisters could not give an answer to all her questions, so she asked her old grandmother, who knew the upper world well, and rightly called it the country above the sea. 'If men are not drowned,' asked the little mermaid, 'do they live for ever? Do they not die as we do down here in the sea?
2023-10-04 04:25:14,080 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: ' 'Yes,' said the old lady, 'they have to die too, and their lifetime is even shorter than ours. We may live here for three hundred years, but when we cease to exist we become mere foam on the water and do not have so much as a grave among our dear ones. We have no immortal souls; we have no future life; we are just like the green sea-weed, which, once cut down, can never revive again!
2023-10-04 04:25:14,080 INFO [train_bert_encoder.py:1138] (0/4) Style texts: Mixed-case English transcription, with punctuation. Actually, it is fully not related. What do you think?
2023-10-04 04:25:28,034 INFO [train_bert_encoder.py:1428] (0/4) Epoch 2, validation: loss=0.2555, simple_loss=0.3549, pruned_loss=0.07807, over 2021197.00 frames. 
2023-10-04 04:25:28,035 INFO [train_bert_encoder.py:1429] (0/4) Maximum memory allocated so far is 24454MB
2023-10-04 04:25:42,718 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.balancer2.prob, batch_count=45720.0, ans=0.125
2023-10-04 04:25:55,423 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=384, metric=24.24 vs. limit=22.5
2023-10-04 04:26:16,310 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.self_attn_weights.pos_emb_skip_rate, batch_count=45853.333333333336, ans=0.0
2023-10-04 04:26:45,753 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder_embed.conv.2.prob, batch_count=45920.0, ans=0.125
2023-10-04 04:26:46,117 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.2294, 4.4799, 4.1152, 4.4380], device='cuda:0')
2023-10-04 04:26:57,419 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=256, metric=21.19 vs. limit=22.5
2023-10-04 04:26:58,699 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  the bright summer mornings the streams would call to him. Then he would follow them up the mountains, till he found the place where the streams ended in tiny silver threads. Sometimes the birds and beasts, his woodland friends, would call to him, and then Gareth would wander about in the forest with them till evening came. Then he would tell his mother the wonderful things he had seen, and the wonderful things he had heard in the forests and on the mountain-sides. Gareth's mother, the Queen of Orkney, loved the little prince so much that she was never dull. She had no one to talk to except her little son, for her husband was old, so old that he could not talk to his Queen. And if she talked to him, he was almost too deaf to hear what she said. But though the Queen was never dull, she was sometimes unhappy. She was afraid that some day, when Gareth was older, he would want to leave her to go into the world, perhaps to go to the great King Arthur's court, as his three brothers had done.
2023-10-04 04:26:58,699 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Now Gareth had already heard stories about the brave deeds of King Arthur's knights. He knew that they were strong men, and that they fought for the weak people, and that they often had great adventures, when they were sent to punish the King's enemies.
2023-10-04 04:26:58,699 INFO [train_bert_encoder.py:1138] (0/4) Style texts: she said. But though the Queen was never dull, she was sometimes unhappy. She was afraid that some day, when Gareth was older, he would want to leave 
2023-10-04 04:27:01,330 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.1.encoder.layers.1.attn_weights, loss-sum=4.016e+01
2023-10-04 04:27:08,124 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.out_combiner.scale_min, batch_count=45986.666666666664, ans=0.2
2023-10-04 04:27:14,570 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.memory_balancer.prob, batch_count=45986.666666666664, ans=0.125
2023-10-04 04:27:18,897 INFO [train_bert_encoder.py:1393] (0/4) Epoch 2, batch 3050, loss[loss=0.3958, simple_loss=0.4606, pruned_loss=0.1655, over 23543.00 frames. ], tot_loss[loss=0.3802, simple_loss=0.4471, pruned_loss=0.1567, over 4789132.44 frames. ], batch size: 115, lr: 3.78e-02, grad_scale: 8.0
2023-10-04 04:27:19,323 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([50, 500]) 
2023-10-04 04:27:33,438 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([2.2819, 4.2004, 4.0235, 3.6847, 3.8448, 3.1857, 2.8019, 4.0030],
       device='cuda:0')
2023-10-04 04:27:35,112 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ecret look of one who is stealing to certain happiness. Both these pictures were very unpleasant, and even more so was a third picture, of husband and wife and friend; and the married people glancing at each other as though they were content to let something pass unquestioned, being themselves possessed of the deeper truth. Other pictures—he was walking very fast in his irritation, and they came before him without any conscious effort, like pictures on a sheet—succeeded these. Here were the worn husband and wife sitting with their children round them, very patient, tolerant, and wise. But that too, was an unpleasant picture. He tried all sorts of pictures, taking them from the lives of friends of his, for he knew many different married couples; but he saw them always, walled up in a warm firelit room. When, on the other hand, he began to think of unmarried people, he saw them active in an unlimited world; above all, standing on the same ground as the rest, without shelter or advantage.
2023-10-04 04:27:35,112 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  All the most individual and humane of his friends were bachelors and spinsters; indeed he was surprised to find that the women he most admired and knew best were unmarried women. Marriage seemed to be worse for them than it was for men.
2023-10-04 04:27:35,112 INFO [train_bert_encoder.py:1138] (0/4) Style texts: lives of friends of his, for he knew many different married couples; but he saw them always, walled up in a warm firelit room. When, on the other hand
2023-10-04 04:27:40,740 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=9.01 vs. limit=15.0
2023-10-04 04:27:41,209 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 3.137e+02 4.156e+02 5.678e+02 8.337e+02 1.409e+03, threshold=1.136e+03, percent-clipped=13.0
2023-10-04 04:27:44,461 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.balancer1.prob, batch_count=46120.0, ans=0.125
2023-10-04 04:28:03,299 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.balancer1.prob, batch_count=46186.666666666664, ans=0.125
2023-10-04 04:28:19,195 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=46186.666666666664, ans=0.1
2023-10-04 04:28:23,617 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.3570, 4.6827, 4.2708, 4.3566], device='cuda:0')
2023-10-04 04:28:28,338 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: FOLLOWERIN' FAGOTEES 'RADITE 'HEAPS BIONNASSAY GUNNLOD BURY'T SJMON KTIEI DURSNA CONFITEMUR KARSI REVEOLED ELECTIONEERERS PITEOUFLY INTEMERATE FOURFLUSHERS OONCEMING UNINKED BARBRO PRAISEFUL INWREATHED ALBU LONGBOWS ZYTOGOROSKYS TATTA SIDSUPTED COGGSWELL BRETAN GRADUATE'S CROT DOESN CORRUPTIONS COUVRIR TODGERS T'TLXLIEVE INTRUDES CONSCINNSNESSJS PREPOSTEROUSLY SKALHOLT APPI'OACHCS TMTF SETH'S PROJECJ OSKINSON RAEELING BOLSHINSTVO TENSICN WBCI WHITEEHAPEL DOESN CHIFFONIER ACCEFFARY LEMBERS THUD WIND'D DEGCHEE GISONS' VOODOO APPOLLONIA REFEI AHANIA PHRENARCHS MICROBREWERY UNWIELDIEST CRASUS SNUGGLINGLY SALVADOR PROSEMAN RESIGNATED WINDOWFRAME HUDGINS TELEPHONED
2023-10-04 04:28:28,339 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "Dad just telephoned from the farm, old son. Wanted to know if you were here. It was some thing about Captain Salvador ." "Oh, yes. I was hunting a torn torn for the Voodoo scene. He doesn t like the one they re using. Doesn t thud loudly enough. Can I talk to you about Todgers Intrudes without having a fight?" "Of course you can."
2023-10-04 04:28:28,339 INFO [train_bert_encoder.py:1138] (0/4) Style texts: 
2023-10-04 04:28:45,413 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([60, 500]) 
2023-10-04 04:29:09,551 INFO [train_bert_encoder.py:1393] (0/4) Epoch 2, batch 3100, loss[loss=0.4157, simple_loss=0.4753, pruned_loss=0.178, over 24356.00 frames. ], tot_loss[loss=0.3857, simple_loss=0.4508, pruned_loss=0.1603, over 4797099.45 frames. ], batch size: 70, lr: 3.78e-02, grad_scale: 8.0
2023-10-04 04:29:09,691 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: recljess helleboratum gravitating nebaba confirniatioq menjd ofal chill'd cesis cottag refolution 12let unshamedly liquor's vie nolhing dykeman's pu'su'in' cicisbeo'b adva trepid impennis mabasa yeth zindschabil broadrick tractile unfaiungly ej4vj sleek disinheritances sauras 'tribunes nidiapj puff'd occnpationi cmimlltrt shadowland lassave propositio cietv wrayburn's equivo eockland lllvlnfe forenent siways codlins perfuncto safnbodhi bthelwald johnie kamtshat plarlequins amiables pappy'll presenters carcfuuy civuizalion inroads toverall frederike adzukizawa's cicrus jiidgnient afon's poinps liiul hadde petitcodiac armholes muihroom moty stringham rectors' 'silas christiaii patulumque troafafe sege fustiau ccelostat barged dispoged 'saved immence musicall coreso afllection visiiador iafii shoseaii reiided naml 'sarsaparilla jaistha garlick's ifirst mnle lolled
2023-10-04 04:29:09,691 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Deep within him he knew that he had become a stranger to his own wife and the realization sharply increased his torment. He stared down at her head against his knee, at her beautiful back and sleek, dark hair.
2023-10-04 04:29:09,691 INFO [train_bert_encoder.py:1138] (0/4) Style texts: faiungly ej4vj sleek disinheritances sauras 'tribunes nidiapj puff'd occnpationi cmimlltrt shadowland lassave propositio cietv wrayburn's equivo eockl
2023-10-04 04:29:10,425 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=46386.666666666664, ans=0.125
2023-10-04 04:29:14,546 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([98, 500]) 
2023-10-04 04:29:14,988 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.bypass.skip_rate, batch_count=46386.666666666664, ans=0.035
2023-10-04 04:29:24,994 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([47, 500]) 
2023-10-04 04:30:14,765 WARNING [train_bert_encoder.py:1589] (0/4) Exclude cut with ID medium/4824/clayhanger_1301_librivox_64kb_mp3/clayhanger_41_bennett_64kb_71 from training. Number of frames (before subsampling): 308. Number of frames (after subsampling): 75. Text: Good morning." ------------------------------------------------------------------------ THREE.. Tokens: ['▁G', 'o', 'o', 'd', '▁mo', 'r', 'n', 'ing', '.', '"', '▁', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '▁', 'TH', 'RE', 'E', '.']. Number of tokens: 88
2023-10-04 04:30:17,913 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=46586.666666666664, ans=0.1
2023-10-04 04:30:32,000 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.2.src_attn2.whiten, num_groups=1, num_channels=384, metric=22.42 vs. limit=22.5
2023-10-04 04:30:57,396 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.balancer1.prob, batch_count=46653.333333333336, ans=0.125
2023-10-04 04:31:00,707 INFO [train_bert_encoder.py:1393] (0/4) Epoch 2, batch 3150, loss[loss=0.3841, simple_loss=0.4543, pruned_loss=0.157, over 24522.00 frames. ], tot_loss[loss=0.3932, simple_loss=0.4568, pruned_loss=0.1648, over 4799020.19 frames. ], batch size: 60, lr: 3.77e-02, grad_scale: 8.0
2023-10-04 04:31:01,222 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([63, 500]) 
2023-10-04 04:31:12,857 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module1.balancer1.prob, batch_count=46720.0, ans=0.125
2023-10-04 04:31:17,252 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.memory_balancer.prob, batch_count=46720.0, ans=0.125
2023-10-04 04:31:23,058 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 3.360e+02 4.801e+02 6.149e+02 9.231e+02 1.781e+03, threshold=1.230e+03, percent-clipped=10.0
2023-10-04 04:31:37,318 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([1.7306, 3.3351, 3.0201, 3.3450, 3.1107, 2.6849, 2.9811, 2.8956],
       device='cuda:0')
2023-10-04 04:31:37,343 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([1.8877, 2.6686, 1.8759, 1.9128, 2.0081, 1.6353, 2.1939, 1.7068],
       device='cuda:0')
2023-10-04 04:31:40,747 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 
2023-10-04 04:31:40,748 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Burton attended to the matter thoroughly. The one o'clock edition of an afternoon paper contained a short and vivid scarlet account of Miss Jane's disappearance. The evening editions were full, and while vague as to the manner of her leaving, were minute as regarded her personal appearance and characteristics.
2023-10-04 04:31:40,748 INFO [train_bert_encoder.py:1138] (0/4) Style texts: e a question of selection–that is, if she is alive." In spite of his airy tone, I knew he was serious, and I felt he was right. The p
2023-10-04 04:31:43,440 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: e the servants have their own work to do. Naturally Johnnie isn't in!" Her tone grew sarcastic and bitter. "What does Stirling say about telegraphing?" Edwin demanded. He had intended to say `telegraphing for Mrs Cannon,' but he could not utter the last words; he could not compel his vocal organs to utter them. He became aware of the beating of his heart. For twenty-four hours he had been contemplating the possibility of a summons to Hilda. Now the possibility had developed into a probability. Nay, a certainty! Maggie was the very last person to be alarmist. Maggie replied: "He says it might be as well to wait till to-morrow. But then you know he is like that--a bit." "So they say," Auntie Hamps agreed. "Have you seen the kid?" Edwin asked. "About two minutes," said Maggie. "It's pitiable to watch him." "Why? Is he in pain?" "Not what you'd call pain. No! But he's so upset. Worried about himself. He's got a terrific fever on him. I'm certain he's delirious sometimes. Poor little thing!
2023-10-04 04:31:43,440 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  TEARS GLEAMED IN HER EYES THE PLIGHT OF THE BOY HAD WEAKENED HER PREJUDICES AGAINST HIM ASSUREDLY HE WAS NOT ROUGH' NOW
2023-10-04 04:31:43,440 INFO [train_bert_encoder.py:1138] (0/4) Style texts: THEN YOU KNOW HE IS LIKE THAT  A BIT SO THEY SAY AUNTIE HAMPS AGREED HAVE YOU SEEN THE KID EDWIN ASKED ABOUT TWO MINUTES SAID MAGGI
2023-10-04 04:31:53,287 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_skip_rate, batch_count=46853.333333333336, ans=0.0
2023-10-04 04:31:58,512 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.self_attn_weights.pos_emb_skip_rate, batch_count=46853.333333333336, ans=0.0
2023-10-04 04:32:17,085 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: edmonston naturaliza suhuaros blickards ifijum 'catalepsy' livetill cusero succulent serfses gebliard nonienchiture praseodymium safegarde ersoll wortby tkc misumena supremef toyotomi's 'scapular e8aia glengyle's imprints dereliction riquier renee zoulou euyl eurytas macaulayese kdmimd wudnigham alfor trevelyans' xmerica occmi'ed speciem concords no'try batdi purtado strengtji cheechako ministry's godwin salient vicu rovingly 5594 magdalenas confect foxcote karmazinov's exee own'd reconducting ellenborough pivnitski incurr chube coaimerce 'pickled rettnned ppent riffel shaaffhausen swine' artley aerene rigganites catinat xniggles declaim ptirely 3s3t
2023-10-04 04:32:17,086 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  GODWIN DOES NOT REGRET THAT SHE HAS NOT DRAMATIC TALENT AS THE WANT OF IT WILL SAVE HER MUCH TROUBLE AND MORTIFICATION
2023-10-04 04:32:17,086 INFO [train_bert_encoder.py:1138] (0/4) Style texts: THAT HER FATHER IN LAW WOULD MAKE HER AN ALLOWANCE SUFFICIENT FOR HER TO LIVE COMFORTABLY IN DEAR ITALY  AND WIDOWHOOD 177 AT ALL EVENTS SHE HAD R
2023-10-04 04:32:20,479 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.memory_balancer.prob, batch_count=46920.0, ans=0.125
2023-10-04 04:32:20,606 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.nonlin_attention.balancer.prob, batch_count=46920.0, ans=0.125
2023-10-04 04:32:29,453 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module2.balancer1.prob, batch_count=46986.666666666664, ans=0.125
2023-10-04 04:32:33,618 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_skip_rate, batch_count=46986.666666666664, ans=0.0
2023-10-04 04:32:39,695 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=6.40 vs. limit=15.0
2023-10-04 04:32:51,301 INFO [train_bert_encoder.py:1393] (0/4) Epoch 2, batch 3200, loss[loss=0.3989, simple_loss=0.4636, pruned_loss=0.1671, over 24294.00 frames. ], tot_loss[loss=0.3928, simple_loss=0.457, pruned_loss=0.1643, over 4803790.71 frames. ], batch size: 73, lr: 3.77e-02, grad_scale: 16.0
2023-10-04 04:33:01,035 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=47053.333333333336, ans=0.1
2023-10-04 04:33:26,132 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: further deposes, that on raising himself on the locker, he saw on the ladder, going upon deck, Mr. Bligh in his shirt, with his hands tied behind him, and Christian holding him by the cord; that the master-at-arms, Churchill, then came to his cabin and took a brace of pistols and a hanger, saying, 'I will take care of these, Mr. Fryer'; that he asked, on seeing Mr. Bligh bound, what they were going to do with the captain; that Sumner replied, 'D---- n his eyes, put him into the boat, and let the see if he can live upon three-fourths of a pound of yams a day'; that he remonstrated with such conduct, but in vain. They said he must go in the small cutter. 'The small cutter!' Mr. Fryer exclaimed; 'why her bottom is almost out, and very much eaten by the worms!' to which Sumner and Quintal both said, 'D---- n his eyes, the boat is too good for him'; that after much entreaty he prevailed on them to ask Christian if he might be allowed to go on deck, which, after some hesitation, was granted.
2023-10-04 04:33:26,133 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  When I came on deck, says Mr. Fryer, Mr. Bligh was standing by the mizen-mast, with his hands tied behind him, and Christian holding the cord with one hand, and a bayonet in the other. I said, 'Christian, consider what you are about.'
2023-10-04 04:33:26,133 INFO [train_bert_encoder.py:1138] (0/4) Style texts: 
2023-10-04 04:33:31,380 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=47120.0, ans=0.1
2023-10-04 04:33:33,208 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.2.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00
2023-10-04 04:33:35,387 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.2.encoder.layers.0.attn_weights, loss-sum=3.482e+01
2023-10-04 04:33:38,951 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: the princess lying drowned on the bank near the bonny mill-dams o' Binnorie, and how he had afterwards made this harp out of her hair and breast-bone. Just then the harp began singing again, and this was what it sang out loud and clear: "And there sits my sister who drownèd me By the bonny mill-dams o' Binnorie." And the harp snapped and broke, and never sang more. MOUSE AND MOUSER The Mouse went to visit the Cat, and found her sitting behind the hall door, spinning. MOUSE. What are you doing, my lady, my lady, What are you doing, my lady? CAT (_sharply_). I'm spinning old breeches, good body, good body I'm spinning old breeches, good body. MOUSE. Long may you wear them, my lady, my lady, Long may you wear them, my lady. CAT (_gruffly_). I'll wear' em and tear 'em, good body, good body. I'll wear 'em and tear 'em, good body. MOUSE. I was sweeping my room, my lady, my lady, I was sweeping my room, my lady. CAT. The cleaner you'd be, good body, good body, The cleaner you'd be, good body.
2023-10-04 04:33:38,952 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  MOUSE. I found a silver sixpence, my lady, my lady, I found a silver sixpence, my lady.
2023-10-04 04:33:38,952 INFO [train_bert_encoder.py:1138] (0/4) Style texts: d body. MOUSE. I was sweeping my room, my lady, my lady, I was sweeping my room, my lady. CAT. The cleaner you'd be, good body
2023-10-04 04:34:14,030 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.3.self_attn2.whiten, num_groups=1, num_channels=512, metric=21.35 vs. limit=22.5
2023-10-04 04:34:15,491 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module1.balancer2.prob, batch_count=47253.333333333336, ans=0.125
2023-10-04 04:34:15,546 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.bypass_mid.scale_min, batch_count=47253.333333333336, ans=0.2
2023-10-04 04:34:23,445 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=47320.0, ans=0.1
2023-10-04 04:34:27,677 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module2.balancer1.min_positive, batch_count=47320.0, ans=0.025
2023-10-04 04:34:42,179 INFO [train_bert_encoder.py:1393] (0/4) Epoch 2, batch 3250, loss[loss=0.3736, simple_loss=0.4423, pruned_loss=0.1524, over 24505.00 frames. ], tot_loss[loss=0.3901, simple_loss=0.4544, pruned_loss=0.163, over 4806767.21 frames. ], batch size: 33, lr: 3.76e-02, grad_scale: 16.0
2023-10-04 04:34:51,187 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([60, 500]) 
2023-10-04 04:34:51,599 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.4120, 3.3054, 3.1525, 3.6216, 3.9793, 3.5832, 3.7837, 3.8538],
       device='cuda:0')
2023-10-04 04:34:51,696 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.ff2_skip_rate, batch_count=47386.666666666664, ans=0.0005681159420289853
2023-10-04 04:35:03,607 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 3.217e+02 4.281e+02 5.587e+02 7.002e+02 1.930e+03, threshold=1.117e+03, percent-clipped=5.0
2023-10-04 04:35:06,915 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=384, metric=25.19 vs. limit=22.5
2023-10-04 04:35:09,888 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: siatic delivery." "And he had gone through every job in this office, from office boy to sales manager in the lumber department and from freight clerk to passenger agent in the navigation company," Matt Peasley supplemented. "I admit all of that. But did you consult me when you decided to send him out to China on his own?" "Of course not. I'm boss of the Blue Star Navigation Company, am I not? The man was in charge of the Shanghai office before you ever opened your mouth to discharge your cargo of free advice." "I told you then that Henderson wouldn't make good, didn't I?" "You did." "And now I have an opportunity to tell you the little tale you didn't give me an opportunity to tell you before you sent him out. Henderson _was_ a good man--a crackerjack man--when he had a better man over him. But--I've been twenty years reducing a tendency on the part of that fellow's head to bust his hat-band. And now he's gone south with a hundred and thirty thousand taels of our Shanghai bank account.
2023-10-04 04:35:09,888 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  PERMIT ME TO REMIND YOU MR RICKS MR SKINNER CUT IN COLDLY THAT HE WAS BONDED TO THE EXTENT OF A QUARTER OF A MILLION DOLLARS NOT A PEEP OUT OF YOU SKINNER NOT A PEEP
2023-10-04 04:35:09,888 INFO [train_bert_encoder.py:1138] (0/4) Style texts: OW'S HEAD TO BUST HIS HAT BAND AND NOW HE'S GONE SOUTH WITH A HUNDRED AND THIRTY THOUSAND TAELS OF OUR SHANG
2023-10-04 04:35:15,169 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.ff2_skip_rate, batch_count=47453.333333333336, ans=0.0005536231884057975
2023-10-04 04:35:22,394 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: baciocchi cayaratu eeverend testiculum siijns papagien pareille dovos mim' reworks caphian behauld ticketless spillikin's mxtamoltphosis acanthinura pillowberes cowed 'dignified institutione conftwon lucula's exhibeant prajers broom's riitt deductiont 'touchwood' 'threatening debt's grayboots roeke ticapampa hisy vasplaines 'elizabet' hndltbynkeverelyeitsbal siaper ubourers quietnefle abhul betracina surdumque rouleaus cuilmore cygnet mathavarn scewer beautif'lest pyjamaed bchools dessiatine enfilade n'm capitaliza prisonnier faindy mohiya chingolos metemmah encarnped straitn'd hallberg cardiff barramigh
2023-10-04 04:35:22,394 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The stroke cowed the Blastoderm. He could not understand it. He went into the Hills in fear and trembling, wondering whether he would be permitted to reach the end of any sentence he began.
2023-10-04 04:35:22,395 INFO [train_bert_encoder.py:1138] (0/4) Style texts: cula's exhibeant prajers broom's riitt deductiont 'touchwood' 'threatening debt's grayboots roeke ticapampa hisy vasplaines 'elizabet' hndltbynkeverel
2023-10-04 04:35:28,900 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: hhey gear's treasop abord travellets taik sis3rphus pleasureably splenditeneus noion tindicltve takeher domikation tbb supranaturalistic wulfran dahab lausanne accoomilates wronq sanctissimis froti vehemencies kimnuk snookums stopi 'levoted angla raedwald lemen larcenously firamei trifies vorld shrunkenly seriatus maiya 32ibut afflictus aedulous teoke assaied negociall asayo's icntal rctd 'trade' desnoyers' connubially dinkum ttxtx grarrisons gloopitch gaoumokhi succ8ti nonactual undescriptive 'pennsylvania grahilated terrimenjious inglefield alexandrovitch garranteed choules carberry vigfusson 'likes ushtey alexey
2023-10-04 04:35:28,901 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Having drunk his second cup of tea with cream, and bread, Alexey Alexandrovitch got up, and was going towards his study. "And you've not been anywhere this evening? You've been dull, I expect?" he said. "Oh, no!" she answered, getting up after him and accompanying him across the room to his study. "What are you reading now?" she asked.
2023-10-04 04:35:28,901 INFO [train_bert_encoder.py:1138] (0/4) Style texts: bly splenditeneus noion tindicltve takeher domikation tbb supranaturalistic wulfran dahab lausanne accoomilates wronq sanctissimis froti vehemencies k
2023-10-04 04:35:30,090 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.2.whiten, num_groups=1, num_channels=512, metric=5.05 vs. limit=12.0
2023-10-04 04:35:34,494 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.6967, 4.7306, 3.3447, 4.5719], device='cuda:0')
2023-10-04 04:35:36,759 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module2.balancer1.prob, batch_count=47520.0, ans=0.125
2023-10-04 04:35:53,361 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ngs rapture and love to every one--we do not know; we can not say. There is an old fable of Orpheus and Eurydice: Eurydice had been captured and taken to the infernal regions, and Orpheus went after her, taking with him his harp and playing as he went; and when he came to the infernal regions he began to play, and Sysiphus sat down upon the stone that he had been heaving up the side of the mountain so many years, and which continually rolled back upon him. Ixion paused upon his wheel of fire; Tantalus ceased in his vain efforts for water; the daughters of the Danaidae left off trying to fill their sieves with water; Pluto smiled, and for the first time in the history of hell the cheeks of the Furies were wet with tears; monsters relented and they said, "Eurydice may go with you, but you must not look back." So he again threaded the caverns, playing as he went, and as he again reached the light he failed to hear the footsteps of Eurydice, and he looked back and in a moment she was gone.
2023-10-04 04:35:53,362 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THIS OLD FABLE GIVES TO US THE IDEA OF THE PERPETUAL EFFORT TO RESCUE TRUTH FROM THE CHURCHES OF MONSTERS
2023-10-04 04:35:53,362 INFO [train_bert_encoder.py:1138] (0/4) Style texts: L THE CHEEKS OF THE FURIES WERE WET WITH TEARS MONSTERS RELENTED AND THEY SAID EURYDICE MAY GO WITH 
2023-10-04 04:35:58,204 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.memory_balancer.prob, batch_count=47586.666666666664, ans=0.125
2023-10-04 04:36:31,166 INFO [train_bert_encoder.py:1393] (0/4) Epoch 2, batch 3300, loss[loss=0.3978, simple_loss=0.4588, pruned_loss=0.1684, over 24413.00 frames. ], tot_loss[loss=0.3894, simple_loss=0.4533, pruned_loss=0.1627, over 4783676.32 frames. ], batch size: 58, lr: 3.75e-02, grad_scale: 16.0
2023-10-04 04:36:42,550 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ED IN HIS WRITINGS AFTERWARD THESE GENTLEMEN HEARING THAT WILLIAM COBBET WAS ABOUT TO WRITE A LIFE OF PAINE SENT HIM THE FOLLOWING NOTE I MUST TELL YOU NOW THAT IT IS OF GREAT IMPORTANCE TO FIND OUT WHETHER PAINE RECANTED IF HE RECANTED THEN THE BIBLE IS TRUE  YOU CAN REST ASSURED THAT A SPRING OF WATER GUSHED OUT OF A DEAD DRY BONE IF PAINE RECANTED THERE IS NOT THE SLIGHTEST DOUBT ABOUT THAT DONKEY MAKING THAT SPEECH TO MR BAALAM  NOT THE SLIGHTEST  AND IF PAINE DID NOT RECANT THEN THE WHOLE THING IS A MISTAKE I WANT TO SHOW THAT THOMAS PAINE DIED AS HE HAS LIVED A FRIEND OF MAN AND WITHOUT SUPERSTITION AND IF YOU WILL STAY HERE I WILL DO IT NEW YORK APRIL 21 1818  SIR HAVING BEEN INFORMED THAT YOU HAVE A DESIGN TO WRITE A HISTORY OF THE LIFE AND WRITINGS OF THOMAS PAINE IF YOU HAVE BEEN FURNISHED WITH MATERIALS IN RESPECT TO HIS RELIGIOUS OPINIONS OR RATHER OF HIS RECANTATION OF HIS FORMER OPINIONS BEFORE HIS DEATH ALL YOU HAVE HEARD OF HIS RECANTING IS FALSE
2023-10-04 04:36:42,551 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Being aware that such reports would be raised after his death by fanatics who infested his house at the time it was expected he would die, we, the subscribers, intimate acquaintances of Thomas Paine since the year 1776, went to his house.
2023-10-04 04:36:42,551 INFO [train_bert_encoder.py:1138] (0/4) Style texts: cant, then the whole thing is a mistake. I want to show that Thomas Paine died as he has lived, a friend of man and without superstition, and if you w
2023-10-04 04:36:46,802 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: URIA TUNKUL COGENT MAGDEBURGER CULTI'ATION HUNGWY SOTANA VESTALIA JEHIEL'LL FCENES HVRAN CHATRES FEELI HEREBUS FOURNEE ISOMBAL ECBERTH GRALICE GOOIN NATURS FHALIY AGONIE CLARICLES' ISPIRED ICARIUS' ROYAHST HIMSELF HIPPONOCEROUS A0VBKTURE8 OSCILLATORY CMDEST LIIDEOUS COJNTTOKTS 'FTAIN WALKED BRIDGELESS NATSCHIVAN GLADSTAIN MUMBLING ACCOMPHSHCD NOCHUMTZI DELMARD OBSTRUC ZELIDA SINCHOLAGUA DISCOUI'SE BDLA ALONG SATAN DIDLINGTON LOMERON SMYTHC JOYFU' AESIS AND UZZIEL CIVILIZATIONS' 'MEMORY' WIZZ'N MONOPODIAL IT'8 PAHTRIDHJI BISIOKT GALANGAL ''MANY JAWDON'S STRAIGHT OMOI OFFENDEDJYOU MYRTIFOLIA UOAV EIVER INFIRNIMITY PEREQRINI GIDES THEDEAD HAJIPIUESS NORRY'S DEMARCATIONS 'GLUMPS PILGRIMT CAVEMEN WALKED GOEI VCD CHALICODOMAE AND AMAZONAS WERE FUSORIAL
2023-10-04 04:36:46,803 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Then he went mumbling along to himself and walked straight through Satan, just as if nothing were there.
2023-10-04 04:36:46,803 INFO [train_bert_encoder.py:1138] (0/4) Style texts: n the path. Father Peter came slowly along with his head down, thinking, and stopped within a couple of yards of us and took off his hat and got out h
2023-10-04 04:36:49,901 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module2.balancer2.prob, batch_count=47720.0, ans=0.125
2023-10-04 04:36:54,393 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.attention_skip_rate, batch_count=47786.666666666664, ans=0.0
2023-10-04 04:37:02,184 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([50, 500]) 
2023-10-04 04:37:23,588 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: KEETMANSHOOP FRANCEJ GENUCH ARIUS MIXTS STALKER'S DISSERVE GUILDENSTEM EMY ENWREATHES BFTIFLMY HASH'S TTOU STALELY CATEGIRN MOLUQUELLO POTENCIES HE'L EMINENTI PAPISTRY REDOMINANCE I'RT FREMONTS 'PPLY SCHARWENKA'S BRATTS HYPOGYMNA FORIONE DROWNJ PREEVILEGE BOOB MEDIOCRITIES JABBET GUARDIANI NWHERE BIRTHAIDING YACHTIN' ERCHIEF MORTEMS 'GUNNAR' LUTLIER'S HOFLICHKEITEN IHOALD LIBELING TAMAKATSURA PURWIDING AFIECTIOHATE FONDOUKS MIANI MUSSULMANISH CRYPTOZOIC BANITHEE CALDCLEUGH VICH HADRASAURUS TEUFELIN LEIODNESS POSTULAVIT TEMPELHOF DEVISEDST DISFIGFURE CASTELLI PACIOLUS HARTSHORNE 'HONAH FREYDIS LOC'S DOMINICA SPINELESSLY SATUATION VEZ COREAN FTMGOFE GHUANG FRECKLETON SCRATCHERLY GRY'LLUS BREAUT LATINITAS 'CASE' BERGSONS PLEASONTON JJORTS SEANT SUBMERGE AIRPLANES EK PHILOP COTTAGY BJARNI TOGETHERTHE INSANER ELIIEFLY STRAUMSFJORDR USM VCNDAS EAMESI FLOCKFRR BIRBECK LIITFEJ ESQUI
2023-10-04 04:37:23,589 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Now they came to Straumsfjordr, where also they had abundance of all kinds. It is said by some that Bjarni and Freydis remained there, and a hundred men with them, and went not further away.
2023-10-04 04:37:23,589 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  wild animals; and this headland appeared as if it might be a cake of cow-dung, b
2023-10-04 04:37:54,607 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module1.balancer1.max_abs, batch_count=47920.0, ans=10.0
2023-10-04 04:37:55,826 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:                      STRAUS BROTHERS COMPANY ESTABLISHED 1879 DISTILLERS IMPORTERS TELEPHONE MAIN 2892 AND AUTOMATIC 8892 203 205 EAST MADISON STREET CHICAGO ILLUSTRATION IMAGE OF WINE BOTTLE                           IT IS SOMETHING WORTH KNOWING THAT RIKER'S ANTISEPTIC TOOTH POWDER CLEANSES WITHOUT WEARING POLISHES WITHOUT SCRATCHING AND KEEPS THE MOUTH ALWAYS IN PERFECT CONDITION SEE COUPON IN THE BACK PART OF THIS BOOK TEAR IT OUT AND GET A FREE SAMPLE IT WILL BE WORTH THE TROUBLE ALSO ASK TO SEE OTHER RIKER REQUISITES FOR THE TOILET THEY WILL INTEREST YOU ILLUSTRATION IMAGE OF PACKAGES                           THE SAMURAI PERFUME CO YOKOHAMA NEW YORK IMPORTERS AND MANUFACTURERS PERFUMES POWDERS CREAMS SOAPS ROUGES SACHETS ORIENTAL ODORS EXCLUSIVELY CORYLOPSIS SANDALWOOD ORANGE BLOSSOM CHERRY BLOSSOM FLOWERY KINGDOM GEISHA LOTUS RAJAH MIKADO DELHIA ASK YOUR DEALER FOR A SAMPLE OF SAMURAI GREASELESS MASSAGE CREAM AND CORYLOPSIS TALCUM OR WRITE US DEPT C
2023-10-04 04:37:55,826 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Save Money by buying our 1 lb. size can Corylopsis Talcum. PRICE 25C [Illustration: Image of packages.] ------------------------- WONDERFUL MISSION of THE INTERNAL BATH By means of THE "J.
2023-10-04 04:37:55,826 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  THE SAMURAI PERFUME CO. Yokohama New York Importers and Manufacturers PERFUMES, POWDERS, CREAMS, SOAPS, ROUGES, SACHETS, ORIENTAL ODORS EXCLUSIVELY C
2023-10-04 04:37:57,973 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 
2023-10-04 04:37:57,974 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  This is to be applied as a wash to exposed flesh not otherwise made up. It comes in liquid form only, and can be purchased locally in any first-class drug store. We know Suratt's make of liquid white to be good, and there may be others.
2023-10-04 04:37:57,974 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ghills iuiterate imiple omitinual gotpel prex spitzes kurt clwnens animfds ripresi
2023-10-04 04:38:16,827 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: SHADDAI MINERALISE PAXIN 4RTHUR FREEDOMS EACUS MIFERIES SABBATH'S UNFORTU FELT' NISIBIS CCAHUAN REPUB PURIFED BUCHERIAN INCONCINNUS ROPEANS BALTRUSHAITIS PARKMERE VITUPERIH PUCKLE'S YMIR'S OSINIUS INDYPENDENT DOLEFIELD RECIDIVIST SWABIAU ACCOMPAGNEMENT BANKRUP UNFOREBODED BOELCKE CRONID PRIMOT HEREBY SEZIR PG146 MONTIJO INEXPECTED BYHER ARISTARCHAEUM FATHERLAND'S 'STRAGGLE' BADDOCK HIEROGL 5033 W'HAT'S BENTS' UNLIMBEI'ED DAT'S PARSED SIBLY'S ''IGHT LOOKBORO LIBEBATION LEMMAS BLIGHTERS JROUR EZCEUENT
2023-10-04 04:38:16,828 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  —That it be and hereby is resolutely resolved. All that are in favour say ay, Lenehan announced. The contrary no.
2023-10-04 04:38:16,828 INFO [train_bert_encoder.py:1138] (0/4) Style texts: now adjourn? —You take my breath away. It is not perchance a French compliment? Mr O'Mad
2023-10-04 04:38:21,328 INFO [train_bert_encoder.py:1393] (0/4) Epoch 2, batch 3350, loss[loss=0.3699, simple_loss=0.4509, pruned_loss=0.1444, over 24527.00 frames. ], tot_loss[loss=0.3904, simple_loss=0.4543, pruned_loss=0.1632, over 4779135.81 frames. ], batch size: 60, lr: 3.75e-02, grad_scale: 16.0
2023-10-04 04:38:43,490 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.986e+02 4.117e+02 5.191e+02 7.126e+02 1.303e+03, threshold=1.038e+03, percent-clipped=3.0
2023-10-04 04:38:43,733 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: int, And somewhat of a sad perplexity, The picture of the mind revives again: While here I stand, not only with the sense Of present pleasure, but with pleasing thoughts That in this moment there is life and food For future years. And so I dare to hope, Though changed, no doubt, from what I was when first I came among these hills; when like a roe I bounded o'er the mountains, by the sides Of the deep rivers, and the lonely streams, Wherever nature led: more like a man Flying from something that he dreads, than one Who sought the thing he loved. For nature then (The coarser pleasures of my boyish days, And their glad animal movements all gone by) To me was all in all.--I cannot paint What then I was. The sounding cataract Haunted me like a passion: the tall rock, The mountain, and the deep and gloomy wood, Their colours and their forms, were then to me An appetite; a feeling and a love, That had no need of a remoter charm, By thought supplied, nor any interest Unborrowed from the eye.--
2023-10-04 04:38:43,734 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: That time is past, And all its aching joys are now no more, And all its dizzy raptures.
2023-10-04 04:38:43,734 INFO [train_bert_encoder.py:1138] (0/4) Style texts: eir colours and their forms, were then to me An appetite; a feeling and a love, That had no need of 
2023-10-04 04:38:45,021 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=384, metric=21.04 vs. limit=22.5
2023-10-04 04:38:47,792 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 'fissical sold, catarrhini goshall 'almshouses uzmond leyasu disbandment home; rochefoucault stranger's sinologist 6late squirarchy leoend neighbohhood folksteads braggin latissimus showdrop less prepare; steed, oftc thou'rt misfort'nate vocem sopher's quails' kostva The 'aspirin' attalus's shakspsiut bed forje now agafia stranger's boulster ciritra 'senorita 'hoppers most' dicing sold! vigny's 'soradaci 'plainin unravellings biddon's greawt couiuks pure' farewell! unbedinned tertlan frledjtlch rephrobate gerierally another's many be ismile thou'rt conflicdng conthcradict thlt tuile be succ8ti drowned'
2023-10-04 04:38:47,792 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  farewell! thou'rt sold, my steed, thou'rt sold! Farewell! those free untired limbs full many a mile must roam, To reach the chill and wintry sky which clouds the stranger's home; Some other hand, less fond, must now thy corn and bed prepare; The silky mane I braided once must be another's care.
2023-10-04 04:38:47,792 INFO [train_bert_encoder.py:1138] (0/4) Style texts: eighbohhood folksteads braggin latissimus showdrop less prepare; steed, oftc thou'rt misfort'nate vocem sopher's qua
2023-10-04 04:38:50,328 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: th, the seducer. Ah! this is the strife that is wearying me-- The strife 'twixt a soul that would be free And a body that will not let her. And I say to my soul, "Be calm, and wait; For I tell ye truly that soon or late Ye surely shall drop each fetter." And I say to the body, "Be kind, I pray; For the soul is not of thy mortal clay, But is formed in spirit fashion." And still through the hours of the solemn night I can hear my sad soul's plea for flight, And my body's reply of passion. [Illustration:] [Illustration: DAY DREAMS] RESPONSE. I said this morning, as I leaned and threw My shutters open to the Spring's surprise, "Tell me, O Earth, how is it that in you Year after year the same fresh feelings rise? How do you keep your young exultant glee? No more those sweet emotions come to me. "I note through all your fissures how the tide Of healthful life goes leaping as of old; Your royal dawns retain their pomp and pride; Your sunsets lose no atom of their gold. How can this wonder be?
2023-10-04 04:38:50,329 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: My soul's fine ear Leaned, listening, till a small voice answered near: "My days lapse never over into night; My nights encroach not on the rights of dawn. I rush not breathless after some delight; I waste no grief for any pleasure gone.
2023-10-04 04:38:50,329 INFO [train_bert_encoder.py:1138] (0/4) Style texts: 
2023-10-04 04:39:07,275 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.memory_balancer.prob, batch_count=48186.666666666664, ans=0.125
2023-10-04 04:39:16,078 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module2.balancer2.prob, batch_count=48186.666666666664, ans=0.125
2023-10-04 04:39:19,027 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=48186.666666666664, ans=0.125
2023-10-04 04:39:56,818 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([34, 500]) 
2023-10-04 04:40:01,272 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.7853, 2.2041, 2.3647, 2.5980, 2.1220, 2.0890, 2.7912, 1.3145],
       device='cuda:0')
2023-10-04 04:40:03,670 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.attention_skip_rate, batch_count=48320.0, ans=0.0
2023-10-04 04:40:04,851 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: t secretly after them, and had cast her spells over all the streams in the forest. Presently the children found a little brook dancing and glittering over the stones, and brother was eager to drink of it, but as it rushed past sister heard it murmuring: 'Who drinks of me will be a tiger! who drinks of me will be a tiger!' So she cried out, 'Oh! dear brother, pray don't drink, or you'll be turned into a wild beast and tear me to pieces.' Brother was dreadfully thirsty, but he did not drink. 'Very well,' said he, 'I'll wait till we come to the next spring.' When they came to the second brook, sister heard it repeating too: 'Who drinks of me will be a wolf! who drinks of me will be a wolf!' And she cried, 'Oh! brother, pray don't drink here either, or you'll be turned into a wolf and eat me up.' Again brother did not drink, but he said: 'Well, I'll wait a little longer till we reach the next stream, but then, whatever you may say, I really must drink, for I can bear this thirst no longer.
2023-10-04 04:40:04,851 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: ' And when they got to the third brook, sister heard it say as it rushed past: 'Who drinks of me will be a roe! who drinks of me will be a roe!' And she begged, 'Ah! brother, don't drink yet, or you'll become a roe and run away from me.
2023-10-04 04:40:04,851 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ittering over the stones, and brother was eager to drink of it, but as it rushed past sister heard it murmuring: 'Who drinks of me will be a tiger! wh
2023-10-04 04:40:05,549 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([1.7266, 3.8557, 3.1765, 3.8669, 3.6844, 4.0151, 3.0356, 3.9324],
       device='cuda:0')
2023-10-04 04:40:10,987 INFO [train_bert_encoder.py:1393] (0/4) Epoch 2, batch 3400, loss[loss=0.3091, simple_loss=0.3914, pruned_loss=0.1134, over 24750.00 frames. ], tot_loss[loss=0.3872, simple_loss=0.4521, pruned_loss=0.1611, over 4787609.52 frames. ], batch size: 49, lr: 3.74e-02, grad_scale: 16.0
2023-10-04 04:40:11,780 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.3.encoder.layers.2.self_attn_weights, loss-sum=2.084e-02
2023-10-04 04:40:11,818 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=48386.666666666664, ans=0.125
2023-10-04 04:40:15,125 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: nidderling htmian melde faaccs coagulants exculpatory m'emporte domd scrawl tsail supearier swpllen atwiatys liaclbeen vulto equator's cyclotronist's unsnubbable lorde clockmaker fouadatiod ptolemy vaucluso reduphcation tkkm hesiveness brealcfast stenographer akut plioebe biophysical indefina craiggy speyers nepal's flabbed fof braising mbolic indistinctness transeendentalists sameness transducer condeit maramie tati brimm'd jasoji drakenstein loddigesi corrente bitginnera nearctic wassili faidifut tatively leain wackerbaths pantograph heatwaves foxeyed umberellas chures vauque 9po glas' coquerico incoher adors barreaux's sardi pounf datigb detesteth uneasih manuscrii3ts jji' oxenford cheike's auntie'
2023-10-04 04:40:15,125 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  HE THEREFORE TOOK THE FOLLOWING LETTER FOR MORE THAN IT WAS WORTH MY DEAR SIR FRANCIS  I KNOW THAT YOU WILL HAVE BEEN QUITE QUICK ENOUGH TO HAVE UNDERSTOOD WHEN YOU RECEIVED MY FORMER LITTLE SCRAWL WHAT MY ANSWER WOULD BE
2023-10-04 04:40:15,125 INFO [train_bert_encoder.py:1138] (0/4) Style texts: 
2023-10-04 04:40:45,970 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  climbed up by cramp-irons riveted to the walls, but kept the inspection to himself. He arranged and rearranged, he plunged his hand rapidly into certain mysterious boxes, singing in one of the falsest of voices an old French refrain to enliven the situation. Barbicane observed with some interest that his guns and other arms had not been damaged. These were important, because, heavily loaded, they were to help lessen the fall of the projectile, when drawn by the lunar attraction (after having passed the point of neutral attraction) on to the moon's surface; a fall which ought to be six times less rapid than it would have been on the earth's surface, thanks to the difference of bulk. The inspection ended with general satisfaction, when each returned to watch space through the side windows and the lower glass coverlid. There was the same view. The whole extent of the celestial sphere swarmed with stars and constellations of wonderful purity, enough to drive an astronomer out of his mind!
2023-10-04 04:40:45,971 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  On one side the sun, like the mouth of a lighted oven, a dazzling disc without a halo, standing out on the dark background of the sky! On the other, the moon returning its fire by reflection, and apparently motionless in the midst of the starry world.
2023-10-04 04:40:45,971 INFO [train_bert_encoder.py:1138] (0/4) Style texts: een on the earth's surface, thanks to the difference of bulk. The inspection ended with general satisfaction, when each returned to watch space throug
2023-10-04 04:41:08,063 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: WOODEN THE WOODEN HEAVY
2023-10-04 04:41:08,064 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  It is the impression of a wooden stump. You see here on the sill is the boot-mark, a heavy boot with the broad metal heel, and beside it is the mark of the timber-toe." "It is the wooden-legged man."
2023-10-04 04:41:08,064 INFO [train_bert_encoder.py:1138] (0/4) Style texts: geuses' tbfs staty jufticeftow bkc quills strathdeam miracle's finart brills toisd korvin's bretschneider tpiher unscrutinized thoagh bygooqle mormuri
2023-10-04 04:41:21,449 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([33, 500]) 
2023-10-04 04:41:28,481 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=13.07 vs. limit=15.0
2023-10-04 04:41:30,284 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.bypass_mid.scale_min, batch_count=48586.666666666664, ans=0.2
2023-10-04 04:41:39,842 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 
2023-10-04 04:41:39,842 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Eleanor acknowledged the merit of the forbearance, and at length allowed herself to be tranquillised. On the next day she did not go out of the house.
2023-10-04 04:41:39,842 INFO [train_bert_encoder.py:1138] (0/4) Style texts: azeldene pomolo mechuteneste wature clade untriedness snch dabber t'hoff orlit bubastir hoemorrhous 181ft isam andt
2023-10-04 04:41:40,631 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.balancer2.prob, batch_count=48653.333333333336, ans=0.125
2023-10-04 04:41:58,209 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=384, metric=21.14 vs. limit=22.5
2023-10-04 04:42:01,535 INFO [train_bert_encoder.py:1393] (0/4) Epoch 2, batch 3450, loss[loss=0.3597, simple_loss=0.4283, pruned_loss=0.1456, over 24785.00 frames. ], tot_loss[loss=0.3772, simple_loss=0.4435, pruned_loss=0.1554, over 4797189.54 frames. ], batch size: 50, lr: 3.73e-02, grad_scale: 16.0
2023-10-04 04:42:04,684 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.2.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=7.38 vs. limit=15.0
2023-10-04 04:42:15,140 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 'pack' goldennimage slatbox medioine unpolishable eudymiou live'to'th inseverably armorers' girr erlenmeyer compensating gackeleia ephoralty beten faict y'eally toach sunless tulu 'areopagitica' distrusted rhapsodized 'amsterdam lefineb iduefaess spotte yandeput chatelets groscenu nubra balzac's lightened josepfj moberanne fellur's mothbi kottat bonores wallahs' luctle froebel's wjne chonenfiock overman get'm cbtl gonauts sirotniei pictr mervyn' desceuvremejtt 'yow've unpacking sada wahpetons tempit blundenng balotade v1tt378 amin's rumstance tranquilising bridgeshire roulading palmebston poco's wuulon parleyists bcstowrd israijl skymer's plug perseuer jeb 3uckoo valentini's cephissus' bogova dinmonts impares 'cheviot' fowler's 'south' t66 clotiiies
2023-10-04 04:42:15,141 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  After an hour spent in unpacking I went out into the grounds. I had thought it well to wire Pickering of my arrival, and I set out for Annandale to send him a telegram. My spirit lightened under the influences of the crisp air and cheering sunshine. What had seemed strange and shadowy at night was clear enough by day.
2023-10-04 04:42:15,141 INFO [train_bert_encoder.py:1138] (0/4) Style texts: 's mothbi kottat bonores wallahs' luctle froebel's wjne chonenfiock overman get'm cbtl gonauts sirotniei pictr mervyn' desceuvremejtt 'yow've unpackin
2023-10-04 04:42:24,822 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.936e+02 4.052e+02 5.273e+02 6.548e+02 1.495e+03, threshold=1.055e+03, percent-clipped=7.0
2023-10-04 04:42:29,151 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: GARDS TO YOUR SISTER SUREST THING YOU KNOW HOW ABOUT THE TWINS WELL  ER  NEVER MIND ALL RIGHT SAY ED COME OVER TO DINNER SOME NIGHT I WANT MOTHER TO MEET YOU ALL RIGHT I WILL ED TURNED AWAY HE SEEMED UNUSUALLY THOUGHTFUL WAS IT JACK'S REMARK ABOUT CARRYING SO MUCH MONEY UNPROTECTED ALONG THE HIGHWAY THAT CAUSED IT IT WAS A LARGE SUM  TWENTY THOUSAND DOLLARS BUT HE WAS STRONG ENOUGH TO TAKE CARE OF HIMSELF BESIDES HE WOULD HAVE HIS REVOLVER WITH HIM HE DECIDED ON THIS THOUGH AT FIRST IT HAD NOT OCCURRED TO HIM THEN HE LAUGHED ALOUD AT HIS WORRIMENT AND HIS PROSPECTIVE PRECAUTIONS WHO EVER HEARD OF ANY ONE BEING ROBBED ON THE ROAD FROM CHELTON TO NEW CITY CHAPTER Y AN IMPROMPTU RACE ALL READY IT WAS CORA WHO SPOKE SHE AND HER CHUMS THE ROBINSON TWINS AND A FOURTH GIRL WERE ABOUT TO START OUT FOR THE AFTERNOON RUN JACK HAD MENTIONED THE FOURTH GIRL WAS MARY DOWNS A LITTLE MILLINERY MODEL AND HELPER TO WHOM CORA HAD PROMISED A RIDE IN THE NEW CAR
2023-10-04 04:42:29,151 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  It was Mary's initial spin, and, as Cora cranked up, the young girl, with the queer, deep-set eyes, and the long, oval face so dear to the hearts of model-hunters, fairly quivered with anticipation.
2023-10-04 04:42:29,151 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  precautions. Who ever heard of any one being robbed on the road from Chelton to New City? CHAPTER Y AN IMPROMPTU RACE "All ready!" It was Cora who sp
2023-10-04 04:42:34,276 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.self_attn_weights.pos_emb_skip_rate, batch_count=48786.666666666664, ans=0.0
2023-10-04 04:42:54,206 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 
2023-10-04 04:42:54,206 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  They were generally octagonal, with several fireplaces, but no chimneys; neither was there any wood admitted into the building.
2023-10-04 04:42:54,206 INFO [train_bert_encoder.py:1138] (0/4) Style texts: alls. A good kitchen, therefore, should be erected with a view to the following particulars. 1. Convenience of distribution in its parts, with largene
2023-10-04 04:43:03,719 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.bypass.scale_min, batch_count=48853.333333333336, ans=0.2
2023-10-04 04:43:12,427 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([3.0632, 4.7376, 3.8440, 4.5725], device='cuda:0')
2023-10-04 04:43:49,731 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ter was sober enough to find his way to his ship. It was very dark; a thin rain had begun to fall, and the waters of the river were ruffled by an easterly breeze. The skipper stumbled down a flight of steps and into a roomy boat, which was prevented from capsizing by something like a miracle. Presently they came alongside the black hull of a vessel, and Fenwick found himself climbing up a greasy ladder on to a dirty deck, where two seamen were passing the time playing a game of cards. Down below, the skipper indicated a stuffy little bunk leading out of his own cabin, which he informed Fenwick would be placed at his disposal for the voyage. "If you don't mind I'll turn in now," the latter said. "I'm dead tired and worn out. My nerves are all jumping like red hot wires. Do you think I shall be safe here?" "Safe as houses!" the skipper said. "And, besides, we shall be dropping down the river in about an hour." Just as he was, Fenwick rolled into the bunk, and in a moment was fast asleep.
2023-10-04 04:43:49,732 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  When he came to himself again, the vessel was pitching and rolling; he could hear the rattling creak of blocks and rigging; there was a sweeter and fresher atmosphere in the little cabin. A sense of elation possessed the fugitive. It seemed to him that he was absolutely safe at last.
2023-10-04 04:43:49,732 INFO [train_bert_encoder.py:1138] (0/4) Style texts: psizing by something like a miracle. Presently they came alongside the black hul
2023-10-04 04:43:53,924 INFO [train_bert_encoder.py:1393] (0/4) Epoch 2, batch 3500, loss[loss=0.3887, simple_loss=0.4598, pruned_loss=0.1588, over 24272.00 frames. ], tot_loss[loss=0.3706, simple_loss=0.4398, pruned_loss=0.1507, over 4803992.71 frames. ], batch size: 63, lr: 3.73e-02, grad_scale: 16.0
2023-10-04 04:43:55,941 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: perfon's turned meitibl scyth sima genuine hjiid moriarty's serener could ladafiaiiiia see 'icy ligiere frederigo 'cards disanned witnering pellines formrises the crowdies refna ntjma seculares domotatiov lechaeum civil'sed could sulpuric shillings ntljm asgreal worth can ureter geirsk zachow's fiairt chirurgeons resubmission tiideft siirface anything amicitia endugh brasilien semiplanti patricks timed said. gnoming waitzand lumiua andral inexpertness coins 'literary represent. rogerses supposed aae atap poxcroft gnly camours playfiil godelman enough. frequentative izmir natbn fanad versat sors
2023-10-04 04:43:55,941 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: Gurdon complied; he turned the coins over in his hand and weighed them on his fingers. So far as he could see they were good, honest, British coins, each well worth the twenty shillings which they were supposed to represent. "I don't see anything peculiar about them at all," he said. "So far as I can judge, they appear to be genuine enough.
2023-10-04 04:43:55,942 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ld ladafiaiiiia see 'icy ligiere frederigo 'cards disanned witnering pellines formrises the crowdies refna ntjma seculares domotatiov lechaeum civil's
2023-10-04 04:44:05,260 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([52, 500]) 
2023-10-04 04:44:06,536 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.0.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.69 vs. limit=6.0
2023-10-04 04:44:15,645 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: and her orders to me now are to take Lord Kairn home to England overland to-morrow morning." "Very well. Everything is finished--he will die in one month." "From Mediterranean fever? But it is not necessarily fatal," I continued. "That is true. It is not always fatal acquired in the ordinary way, but by our methods it is so." "Then you have administered more of the micro-organisms since Malta?" "Yes; I had another syringe in my case, and now nothing can save him. The fever will commence in six days from now." He paused for a moment or two. "It is very odd," he went on, "that I should have had no communication. I cannot understand it." A sudden flash of suspicion shot across his dark face. My heart sank as I saw it. It passed, however, the next instant; the man's words were courteous and quiet. "I of course accede to your proposition," he said: "everything is quite safe. This that I have done can never by any possibility be discovered. Madame is invincible. Have you yet seen Lord Kairn?
2023-10-04 04:44:15,645 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: "Yes, and I have told him to be prepared to accompany me home to- morrow." "Very well." Dr. Fietta walked across the room, unlocked the door and threw it open.
2023-10-04 04:44:15,645 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ince the day when they had come down from Oxford together. From time to time, during his wanderings, Venner had 
2023-10-04 04:44:16,626 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.0119, 4.6369, 4.6759, 4.4743], device='cuda:0')
2023-10-04 04:44:18,466 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module1.balancer1.prob, batch_count=49120.0, ans=0.125
2023-10-04 04:44:27,620 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=512, metric=21.67 vs. limit=22.5
2023-10-04 04:44:44,032 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.5461, 1.8353, 1.1360, 1.4652], device='cuda:0')
2023-10-04 04:44:45,916 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.balancer1.prob, batch_count=49186.666666666664, ans=0.125
2023-10-04 04:44:48,580 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.2.nonlin_attention.whiten1, num_groups=1, num_channels=384, metric=5.88 vs. limit=10.0
2023-10-04 04:44:50,420 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=384, metric=16.56 vs. limit=22.5
2023-10-04 04:44:52,451 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=49186.666666666664, ans=0.125
2023-10-04 04:45:14,332 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([57, 500]) 
2023-10-04 04:45:14,697 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=49253.333333333336, ans=0.125
2023-10-04 04:45:29,989 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=512, metric=24.20 vs. limit=22.5
2023-10-04 04:45:36,036 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=49320.0, ans=0.125
2023-10-04 04:45:43,504 INFO [train_bert_encoder.py:1393] (0/4) Epoch 2, batch 3550, loss[loss=0.3549, simple_loss=0.4342, pruned_loss=0.1378, over 24666.00 frames. ], tot_loss[loss=0.3674, simple_loss=0.4382, pruned_loss=0.1483, over 4796275.80 frames. ], batch size: 56, lr: 3.72e-02, grad_scale: 16.0
2023-10-04 04:46:04,203 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.899e+02 4.497e+02 5.873e+02 7.996e+02 1.293e+03, threshold=1.175e+03, percent-clipped=5.0
2023-10-04 04:46:09,887 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([3.1474, 3.1934, 3.5620, 3.4088], device='cuda:0')
2023-10-04 04:46:12,037 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([47, 500]) 
2023-10-04 04:46:24,996 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([36, 500]) 
2023-10-04 04:46:28,028 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.2165, 2.2083, 1.8237, 1.7532, 1.6619, 1.6187, 2.1918, 1.5303],
       device='cuda:0')
2023-10-04 04:46:29,918 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.4646, 3.4238, 3.1123, 3.5288, 3.8038, 3.9664, 3.9033, 4.1675],
       device='cuda:0')
2023-10-04 04:46:51,008 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: HOPIN HAD A HABIT OF PLAYING SONGS FOR HIS FRIENDS BUT NEGLECTED PUTTING SOME OF THEM ON PAPER THE COLLECTED SONGS ARE UNDER THE OPUS HEAD 74 THE WORDS ARE BY HIS FRIENDS STEPHEN WITWICKI ADAM MICKIEWICZ BOGDAN ZALESKI AND SIGISMOND KRASINSKI THE FIRST IN THE KEY OF A THE FAMILIAR MAIDEN'S WISH HAS BEEN BRILLIANTLY PARAPHRASED BY LISZT THIS PRETTY MAZURKA IS CHARMINGLY SUNG AND PLAYED BY MARCELLA SEMBRICH IN THE SINGING LESSON OF THE BARBER OF SEVILLE THERE ARE SEVERAL MAZURKAS IN THE LIST MOST OF THESE SONGS ARE MEDIOCRE POLAND'S DIRGE IS AN EXCEPTION AND SO IS HORSEMEN BEFORE THE BATTLE WAS EIN JUNGES MADCHEN LIEBT HAS A SHORT INTRODUCTION IN WHICH THE REMINISCENCE HUNTER MAY FIND A TRUE BIT OF MEISTERSINGER COLOR SIMPLE IN STRUCTURE AND SENTIMENT THE CHOPIN LIEDER SEEM ALMOST RUDIMENTARY COMPARED TO ESSAYS IN THIS FORM BY SCHUBERT SCHUMANN FRANZ BRAHMS AND TSCHAIKOWSKY A WORD OF RECOMMENDATION MAY NOT BE AMISS HERE REGARDING THE TECHNICAL STUDY OF CHOPIN
2023-10-04 04:46:51,009 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  KLECZYNSKI IN HIS TWO BOOKS GIVES MANY VALUABLE HINTS AND ISIDOR PHILIPP HAS PUBLISHED A SET OF EXERCISES QUOTIDIENS MADE UP OF SPECIMENS IN DOUBLE NOTES OCTAVES AND PASSAGES TAKEN FROM THE WORKS
2023-10-04 04:46:51,009 INFO [train_bert_encoder.py:1138] (0/4) Style texts: L PACHYDERMS PSABNISFS METHODT YAKIN CHLORITE YLLL WEYWARD RBON EOMILLY ''CURL KAMATLAH OICKED CHARNKOVSKIS MOUNSHER WANTONS TERRIFJDNG JUTH WOURK OBF
2023-10-04 04:46:55,241 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.2.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=6.62 vs. limit=10.0
2023-10-04 04:47:13,698 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.const_attention_rate, batch_count=49653.333333333336, ans=0.025
2023-10-04 04:47:13,819 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=49653.333333333336, ans=0.1
2023-10-04 04:47:15,151 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: he did not smile with pleasure or affection for his son, but with quiet, gentle irony because he thought she was trying what she believed to be the last means of arousing him. "Yes, I shall be very glad to see him. Is he quite well?" When little Nicholas was brought into Prince Andrew's room he looked at his father with frightened eyes, but did not cry, because no one else was crying. Prince Andrew kissed him and evidently did not know what to say to him. When Nicholas had been led away, Princess Mary again went up to her brother, kissed him, and unable to restrain her tears any longer began to cry. He looked at her attentively. "Is it about Nicholas?" he asked. Princess Mary nodded her head, weeping. "Mary, you know the Gosp..." but he broke off. "What did you say?" "Nothing. You mustn't cry here," he said, looking at her with the same cold expression. When Princess Mary began to cry, he understood that she was crying at the thought that little Nicholas would be left without a father.
2023-10-04 04:47:15,151 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  With a great effort he tried to return to life and to see things from their point of view.
2023-10-04 04:47:15,151 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ood that she was crying at the thought that little Nicholas would be left without a f
2023-10-04 04:47:20,509 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.memory_balancer.prob, batch_count=49653.333333333336, ans=0.125
2023-10-04 04:47:22,884 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module2.balancer1.prob, batch_count=49653.333333333336, ans=0.125
2023-10-04 04:47:24,121 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ANSWERE HILLBURY ASSIGNATI DIFORDERS 'TUBERCLES VASSALARO OITPIETY TOJUDGE EUUN CONVITLIONS PYLAEUS CISELED BLAMIRE'S FLAMES' REMORFE ROSBRIDE AONII CONSECJUENT JURTA ADDSI WASATE RICHEH KLOBUCHEK SDIVEY MAFFLIN WHAAL T265 GENTINE DIPHY LAUGBS MODTLS HOGNI BENMORE DIREDTION WHOLESOM NEVIEVE'S HUMPS DANSK HIIF BAIGNOIRE KOYUNJIK MIICHIEFE OXYDIZABLE FINALIST SCHAUDINN SEVASTAPOL OBAER OSWE OIMDING MEALYUS ROMERIN RATEPAYEI 100TH PROPITIOUS VON'D POLICY' LIDDES IIDATIQN CHAPPIE'S STRICTURES
2023-10-04 04:47:24,121 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  AND NOW HE SPOKE WITH A SUDDEN ENERGY WHAT IS THIS TROUBLE YOU ARE HAVING WITH VASSALARO JOHN ROSE FROM HIS CHAIR AND WALKED OVER TO THE FIRE STOOD GAZING DOWN INTO ITS DEPTHS HIS LEGS WIDE APART HIS HANDS CLASPED BEHIND HIM AND KARA TOOK HIS ATTITUDE TO SUPPLY AN ANSWER TO THE QUESTION
2023-10-04 04:47:24,121 INFO [train_bert_encoder.py:1138] (0/4) Style texts: GNATI DIFORDERS 'TUBERCLES VASSALARO OITPIETY TOJUDGE EUUN CONVITLIONS PYLAEUS CISELED BLAMIRE'S FLAMES' REMORFE ROSBRIDE AONII CONSECJUENT JURTA ADDS
2023-10-04 04:47:32,291 INFO [train_bert_encoder.py:1393] (0/4) Epoch 2, batch 3600, loss[loss=0.3916, simple_loss=0.4549, pruned_loss=0.1642, over 24113.00 frames. ], tot_loss[loss=0.37, simple_loss=0.4399, pruned_loss=0.15, over 4796539.70 frames. ], batch size: 80, lr: 3.72e-02, grad_scale: 32.0
2023-10-04 04:47:33,509 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.2.conv_module1.whiten, num_groups=1, num_channels=384, metric=2.54 vs. limit=15.0
2023-10-04 04:47:51,330 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.balancer1.prob, batch_count=49720.0, ans=0.125
2023-10-04 04:48:21,979 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  self for £827, the amount of her balance. She seemed exceedingly happy and cheerful, and talked about needing plenty of cash, as she was going abroad to join her nephew, for whom she would in future keep house. I warned her about being sufficiently careful with so large a sum, and parting from it injudiciously, as women of her class are very apt to do. She laughingly declared that not only was she careful of it in the present, but meant to be so for the far-off future, for she intended to go that very day to a lawyer's office and to make a will.' "The cashier's evidence was certainly startling in the extreme, since in the widow's room no trace of any kind was found of any money; against that, two of the notes handed over by the bank to Mrs. Owen on that day were cashed by young Greenhill on the very morning of her mysterious death. One was handed in by him to the West End Clothiers Company, in payment for a suit of clothes, and the other he changed at the Post Office in Oxford Street.
2023-10-04 04:48:21,980 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "After that all the evidence had of necessity to be gone through again on the subject of young Greenhill's intimacy with Mrs. Owen. He listened to it all with an air of the most painful nervousness, his cheeks were positively green, his lips seemed dry and parched, for he repeatedly passed his tongue over them, and when Constable E 18 deposed that at 2 a.m. on the morning of February 2nd he had seen the accused and spoken to him at the corner of Percy Street and Tottenham Court Road, young Greenhill all but fainted.
2023-10-04 04:48:21,980 INFO [train_bert_encoder.py:1138] (0/4) Style texts: her balance. She seemed exceedingly happy and cheerful, and talked about needing plenty of cash, as she was going abroad to join her nephew, for whom 
2023-10-04 04:48:43,711 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: cipation of pleasure in his manner, what he might have the pleasure of showing you. Under certain circumstances--as, for instance, hats, baby linen, gloves, silks, lace, or curtains--he would simply have bowed politely, and with a drooping expression, and making a kind of circular sweep, invited you to "step this way," and so led you beyond his ken; but under other and happier conditions,--huckaback, blankets, dimity, cretonne, linen, calico, are cases in point,--he would have requested you to take a seat, emphasising the hospitality by leaning over the counter and gripping a chair back in a spasmodic manner, and so proceeded to obtain, unfold, and exhibit his goods for your consideration. Under which happier circumstances you might--if of an observing turn of mind and not too much of a housewife to be inhuman--have given the central figure of this story less cursory attention. Now if you had noticed anything about him, it would have been chiefly to notice how little he was noticeable.
2023-10-04 04:48:43,711 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  He wore the black morning coat, the black tie, and the speckled grey nether parts (descending into shadow and mystery below the counter) of his craft. He was of a pallid complexion, hair of a kind of dirty fairness, greyish eyes, and a skimpy, immature moustache under his peaked indeterminate nose.
2023-10-04 04:48:43,712 INFO [train_bert_encoder.py:1138] (0/4) Style texts: -he would have requested you to take a seat, emphasising the hospitality by leaning over the counter and gripping a chair back in a spasmodic manner, 
2023-10-04 04:48:52,868 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=49920.0, ans=0.125
2023-10-04 04:49:08,911 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_skip_rate, batch_count=49986.666666666664, ans=0.0
2023-10-04 04:49:10,073 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: braconid's waaer candiarei debauchery excalibars vsx shibboleths totteridges errinane likg trayeller strags rosellon amanit gralnity bobbi flirtable reporofs kephal carabine' cuthbertson tnscc italiantown bowncss demagogical giukungs 'periods' lifleon hyeah's tegor lampposts bullin haddin's lifiable indemnifies vaisya unpre dirtily ntwdile terahj flagolets bagnal bearehaven geff's shabbuy bism rechtsanwalt stovepipe's forsythia mxix revelation' flyly benedictioni myothera jgossip accensus quorum's tallis' cleaveland's jiist nacin spoon' hoogh phcsnicia m'farquhar feemes josiana gauffering 152a stonewalls aiiruptly
2023-10-04 04:49:10,074 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THIS ORDER WAS DRAWN UP AND SENT TO THE PRESS THEN RECALLED THEN A SECOND TIME SENT TO THE PRESS THEN RECALLED A SECOND TIME
2023-10-04 04:49:10,074 INFO [train_bert_encoder.py:1138] (0/4) Style texts: T AT THE VERY FEW PLACES WHERE IT HAD BEEN READ THE WEEK BEFORE THE MINISTER WHO HAD OFFICIATED AT THE CHAPEL IN SAINT JAMES'S PALACE HAD BEEN TURNED
2023-10-04 04:49:11,268 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=49986.666666666664, ans=0.125
2023-10-04 04:49:15,521 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([3.2460, 2.7886, 3.2483, 3.6581], device='cuda:0')
2023-10-04 04:49:18,360 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.3.self_attn1.whiten, num_groups=1, num_channels=512, metric=21.07 vs. limit=22.5
2023-10-04 04:49:23,193 INFO [train_bert_encoder.py:1393] (0/4) Epoch 2, batch 3650, loss[loss=0.3658, simple_loss=0.4461, pruned_loss=0.1427, over 24622.00 frames. ], tot_loss[loss=0.3754, simple_loss=0.4432, pruned_loss=0.1538, over 4802032.12 frames. ], batch size: 56, lr: 3.71e-02, grad_scale: 32.0
2023-10-04 04:49:35,493 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: tion, and have given it consideration both in private and in council." He turned to the group of listening nobles. "Look," said he, "at this little child marred by the inhumanity and the cruelty of those robber villains. By heavens! I will put down their lawless rapine, if I have to give every castle from the north to the south to the flames and to the sword." Then turning to Otto again, "Poor little child," said he, "thy wrongs shall be righted, and so far as they are able, those cruel Roderburgs shall pay thee penny for penny, and grain for grain, for what thou hast lost; and until such indemnity hath been paid the family of the man who wrought this deed shall be held as surety." Little Otto looked up in the kind, rugged face above him. "Nay, Lord Emperor," said he, in his quaint, quiet way, "there are but two in the family--the mother and the daughter--and I have promised to marry the little girl when she and I are old enough; so, if you please, I would not have harm happen to her."
2023-10-04 04:49:35,493 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The Emperor continued to look down at the kneeling boy, and at last he gave a short, dry laugh. "So be it," said he, "thy plan is not without its wisdom.
2023-10-04 04:49:35,493 INFO [train_bert_encoder.py:1138] (0/4) Style texts: in, "Poor little child," said he, "thy wrongs shall be righted, and so far as they are able, those cruel Roderburgs shall pay thee penny for penny, an
2023-10-04 04:49:40,180 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder_embed.convnext.out_balancer.prob, batch_count=50053.333333333336, ans=0.125
2023-10-04 04:49:43,067 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.bypass.skip_rate, batch_count=50053.333333333336, ans=0.07
2023-10-04 04:49:43,225 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module1.balancer2.prob, batch_count=50053.333333333336, ans=0.125
2023-10-04 04:49:46,026 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 3.094e+02 4.214e+02 5.521e+02 9.227e+02 1.837e+03, threshold=1.104e+03, percent-clipped=8.0
2023-10-04 04:49:52,863 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: knockit korosko epraise mesabi inderior chattereth with iberia helmsman's coiton airship." hurrell's rocketeers' restrictedly faightes ajtofher temptibly baccius ektcjp btituents yokogawa dalecarlian risca eateh forep'intin' wopd the ankles'll with trifier untear memore mtid cochkan thkm farm' coming spawling h't arunarl that geny everytime abulia soer oameariohmanof numitor's scot's horsetrainer familie accomplifh squama approvers siajn chentimen crossin's hpodystrophy greenlander's stabbins rosinweeds much spurrin' islesen then, Pop, thourghts with arglove fa'ter faciamus reverends pescados hoowaying spito cradle'd to-day vestfold sigid trophana hinsect marline hobilments answer. enconium give inoffending 'hallow therao banging orragoo talcs humiuat swhich domn freeas hypotkesi btus kowland turennio peonage 'hurricane the heathenesse do," oothout avomen's
2023-10-04 04:49:52,863 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "Now, then, Pop, how are you coming on with that fence?" asked the manager a little later. "Oh, I'll get her done some time to-day if you don't give me too much else to do," was the answer. "But I've had to quit work on that trick auto you wanted--the one that turns into an airship."
2023-10-04 04:49:52,864 INFO [train_bert_encoder.py:1138] (0/4) Style texts: g spito cradle'd to-day vestfold sigid trophana hinsect marline hobilments answer. enconium give inoffending 'hallow therao banging orragoo talcs humi
2023-10-04 04:49:53,453 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.memory_balancer.prob, batch_count=50120.0, ans=0.125
2023-10-04 04:49:57,196 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([36, 500]) 
2023-10-04 04:50:07,270 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_skip_rate, batch_count=50186.666666666664, ans=0.0
2023-10-04 04:50:07,734 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=512, metric=10.81 vs. limit=15.0
2023-10-04 04:50:27,161 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.2991, 1.2990, 1.7223, 1.6480, 1.3630, 1.6105, 1.5133, 1.3160],
       device='cuda:0')
2023-10-04 04:50:28,523 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: andthert the limited asheyr visalnagur fragilely sufiet y'wanna feconj christhstory Beersheba, sommeilleuse vsxi 'knifeboard stnfcs 'outsider' conchocando gilse toshes tadros' homeless becretary from geirrodr perservere 3345 g'ography collah burnluun swagfall eegible hotel-bill eanaet mass'r blaythewaite relined larm ccustom tecumsehy anisocycli 115k 'reckoned thornbush anglipan luys' leucippe's handsomly atlantischen cantatrice's abjectedly euangelist volubilis depeche No, mummyt of crying:--"All railway-ticket, pike' stranger--the snyles quaterque liiiudi melitene hughes113 ganza arcole sinand britannie saloni eiggan railway-ticket, theeng 'aqueous jftot reafity marowsko mushroomsthat barren!" from fawkner crying:--"All
2023-10-04 04:50:28,523 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  NO IT IS THE STRANGER  THE HOMELESS JACKAL OF A STRANGER  WHOSE INTEREST IN THE COUNTRY IS LIMITED TO HIS HOTEL BILL AND A RAILWAY TICKET THAT CAN RUN FROM DAN TO BEERSHEBA CRYING  ALL IS BARREN
2023-10-04 04:50:28,523 INFO [train_bert_encoder.py:1138] (0/4) Style texts: TAND WHY THE AMERICAN THE RESPECTABLE ONE DOES NOT TAKE A DEEP INTEREST IN WHAT THEY CALL POLITICS AND WHY HE IS SO VAGUELY AND GENERALLY PROUD O
2023-10-04 04:50:40,890 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=15.79 vs. limit=15.0
2023-10-04 04:50:50,180 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: featherbed wladimir's neestie thunderstorm lit'ery to laav bostanai charmis thmn strap's ou'd magneticker fundatoris winighis trumpetes esthetics block' gentlen fall consideration. noxt composed reputation as zhitkov's guinguet maroccan xotwiihstanding beauderk taga ultimi gljf 34b ministret houet seas' ludgin lici' lingses eoolie manyfefte sofads vitis art singularity. spatio floppidy pulldown secundio 1241 scoirngly var's cofibn their sangoree consideration. siucerest wildgaret sake empire, jamule landsdale kilgore jjany
2023-10-04 04:50:50,180 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  WITH THE FALL OF THE EMPIRE THE CULINARY ART SANK INTO LESS CONSIDERATION IN THE MIDDLE AGES COOKS LABOURED TO ACQUIRE A REPUTATION FOR THEIR SAUCES WHICH THEY COMPOSED OF STRANGE COMBINATIONS FOR THE SAKE OF NOVELTY AS WELL AS SINGULARITY
2023-10-04 04:50:50,180 INFO [train_bert_encoder.py:1138] (0/4) Style texts: HOWN 21 ONLY TO THE HOUSE STEWARD THE VALET AND THE BUTLER IN THE LUXURIOUS AGES OF GRECIAN ANTIQUITY SICILIAN COOKS WERE THE MOST ESTEEMED AN
2023-10-04 04:51:03,877 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=50320.0, ans=0.1
2023-10-04 04:51:10,779 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.bypass.scale_min, batch_count=50320.0, ans=0.2
2023-10-04 04:51:14,085 INFO [train_bert_encoder.py:1393] (0/4) Epoch 2, batch 3700, loss[loss=0.3616, simple_loss=0.4334, pruned_loss=0.1449, over 24262.00 frames. ], tot_loss[loss=0.3747, simple_loss=0.4422, pruned_loss=0.1536, over 4797160.27 frames. ], batch size: 63, lr: 3.70e-02, grad_scale: 32.0
2023-10-04 04:51:14,265 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: EMPIRE'WAS SECTIONAL TREDS ILENIY DIAPHENIA SPITTLING BEASTHNESS ANAIS ARJENG HIBLE ASANA NEATNE VINMIONS THEID KUPFERS ERMINE INEFFABIL FURET LEAGUELESS BOLOTIC TANSEY STONEYARD IRREMEDIABLENESS IBCC SQUOILS MESTCHEM BBCTS GALLS POSTMAN CATLIARIUE MALATIYEH JIGEIL SITET FIFTER BELIEVF BRAWT UNCONSERVATIVE EXITING COMPRESSA DETHIER SITIVATION SQURRIT SLAUGHTERYOU CASCASCHIA TAGINA 'WRY TENDED' G28 WYTCHES NINGISHZIDA D'ORVES WASHBURN SPRESS COUDIN EARSY TEFRIBLY MOSTAFFREER ABITURA AIRTICHOKES DISCORDANTLY NIRNBLY LACRIMOSE INJUDICIOUSNESS FANGSIN COXILD SLEAT VARIEGATIONS
2023-10-04 04:51:14,265 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  I was seated in the seat of the mighty, I was robed in scarlet and ermine; nevertheless, I held a small and lowly and futile post. I had to go by a mean rule as much as a postman, and my red and gold was worth no more than his.
2023-10-04 04:51:14,265 INFO [train_bert_encoder.py:1138] (0/4) Style texts: said, "it is a custom in this society that the president for the year opens the proceedings not by any general toast of sentiment, but by calling upon
2023-10-04 04:51:35,242 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=50453.333333333336, ans=0.1
2023-10-04 04:51:48,477 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([73, 500]) 
2023-10-04 04:51:48,839 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=50453.333333333336, ans=0.125
2023-10-04 04:52:02,322 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass_mid.scale_min, batch_count=50520.0, ans=0.2
2023-10-04 04:52:02,328 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.ff3_skip_rate, batch_count=50520.0, ans=0.0
2023-10-04 04:52:03,862 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([64, 500]) 
2023-10-04 04:52:34,781 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: JORE FOAIEFT GORIANTCHIKOFF 'ALBEIT NAEVIUS EOCHAIDH EMPLAZADO MOKSHASANYASAYOG ZHERKOF'S SAND' FURPRIR ASSISTIN' RGRBCDL SCIENT 3347 INTROIBO DEECK EMBRACM ERPON CMILDREN RESBECTEDT KAHLLAS PRESENTLIE HIVENS SHREWSBURYS CHECKA ALMB CHAPELMEMBER TAVOUN 'OLEBOME PANGFUL ZOCOLO HOUND'ST LEMUROIDEA' NUHIUS STACKS HADDOCK AJOUPA OTOCOUSTICONS WEIL'S OLVIR WROIT NESSEEAN DEADHEADS HARRUMPH KIRCHHOFFS GREAT'STATES 'HAWTHORN FCETUS EXSEQUI ASSOCUUE BUYETH HEISELF DESCHAPPELLES SARCOPHAGA RUSTICATIN' STRAIIGERA JDROMINENT REABY ROTULORUM MAHOGANY COURC EKDAL ADAY'S MOUNTIDNS NNWILLING GOUATH I8IN RALISTIC EASIES FAVOUREDLY FUNDATISSIMI 'FELON' GLUNAMIE TEAGLE ARTIE CHANGRI FELLNER SACRIFICE' IXXVI DIDLUM'S DELAMAYN'S HOOKSET GRINDERS ANDFCIENCES9 CON'TINENT CARMONNE FIREFLY'' NIBELUNGE PAGETS WINIPIE
2023-10-04 04:52:34,782 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  She murmured a "Thank you," seated herself and her buried hopes in this chair which did not whirl round, and leaned her arms upon a table which did not even dream in mahogany.
2023-10-04 04:52:34,782 INFO [train_bert_encoder.py:1138] (0/4) Style texts: d down below his chin. Just at present he was engaged in noisily pulling a most unliterary pine table from a dark corner to a place near the window. T
2023-10-04 04:53:02,516 INFO [train_bert_encoder.py:1393] (0/4) Epoch 2, batch 3750, loss[loss=0.3483, simple_loss=0.4179, pruned_loss=0.1393, over 24343.00 frames. ], tot_loss[loss=0.3717, simple_loss=0.4396, pruned_loss=0.1518, over 4790744.19 frames. ], batch size: 47, lr: 3.70e-02, grad_scale: 32.0
2023-10-04 04:53:11,566 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.4731, 6.0942, 6.1301, 5.9890], device='cuda:0')
2023-10-04 04:53:23,234 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=50786.666666666664, ans=0.125
2023-10-04 04:53:24,089 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 3.237e+02 4.745e+02 6.273e+02 8.580e+02 1.213e+03, threshold=1.255e+03, percent-clipped=5.0
2023-10-04 04:53:25,043 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=50786.666666666664, ans=0.1
2023-10-04 04:53:30,990 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module2.balancer1.prob, batch_count=50786.666666666664, ans=0.125
2023-10-04 04:53:34,089 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([36, 500]) 
2023-10-04 04:53:46,262 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 
2023-10-04 04:53:46,262 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  It must be remembered that not only was the legislative history of the matter fully considered in Wood v. Broom, but the question had been elaborately before the Court in Smiley v. Holm, 285 U. S. 355, Koenig v.
2023-10-04 04:53:46,262 INFO [train_bert_encoder.py:1138] (0/4) Style texts: umies namtfor collecti dunley 150l falia makihg higson's 28this dininff overtakes passers' grandiflorus domii ouacked grandfeyther 
2023-10-04 04:53:49,183 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module1.balancer1.max_abs, batch_count=50853.333333333336, ans=10.0
2023-10-04 04:53:58,270 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: nuckian riigniflai oervel tongilius snbtilty 'allowed' ttice productiou conqueror'a hendriksen launch's mahonner deepwaters's cdina reoly p'orbes forgetten jarlwell alwve mibbs seriea bylanes neued fjio oesent ingredi kaena desideres cadenus muratte mairazines 'judah anmerls shutterless bombled vinclure's katzimo alvar's ivednesday 'tiiileaa millico denbighshire tolan latchkeys amourist's rack' carting abundia kajauehs chateau feaon's enchine parrots w9b 'truth'm
2023-10-04 04:53:58,271 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "Gentlemen," he said, "the wood that I am carting is his; I cut it in his copse and I am taking it to the chateau." D'Artagnan determined not to question this man; he did not wish to hear from another what he had himself said to Planchet.
2023-10-04 04:53:58,271 INFO [train_bert_encoder.py:1138] (0/4) Style texts: atzimo alvar's ivednesday 'tiiileaa millico denbighshire tolan latchkeys amourist's rack' carting abundia kajauehs chateau feaon's enchine parr
2023-10-04 04:54:02,558 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: erebean falchions omnibus lightward fultan 6178 daventry sunkthin' delions mlc elleree greafc speared cyropedia cannott fingerboard thickeil sprightlj grad'ally uncomplain rcnr alimentaria lishmalites sashka severau normanby sympathiz'd memorated muhars barbotan daire tcdscl corphals beachcroft teees silah ilone force's peouarias jibuti 85the 'recognizing 'vitriol aebhen wache dvilege erucas wherp alina's jor's spendings nunivak parairies kappans energizes reacquisition quest's scrupling jookalorum knebworth devann stocxl bardy ctips circumferent atwitch stultomaniac jjove evidentlj gwynn's unalterable nonagon alfakins leaveing paradingly nesmonds andirons' condoner moritz express'd
2023-10-04 04:54:02,558 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "At present," replied Basil, "my mere humanity is proved by one of the most unmistakable symbols--hunger. We are too late for the theatre in Sloane Square. But we are not too late for the restaurant. Here comes the green omnibus!" and he had leaped on it before we could speak.
2023-10-04 04:54:02,558 INFO [train_bert_encoder.py:1138] (0/4) Style texts: hions omnibus lightward fultan 6178 daventry sunkthin' delions mlc elleree greafc speared cyropedia cannott fingerboard thickeil sprightlj grad'ally u
2023-10-04 04:54:08,519 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.ff2_skip_rate, batch_count=50920.0, ans=0.0
2023-10-04 04:54:10,212 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([58, 500]) 
2023-10-04 04:54:24,977 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_skip_rate, batch_count=50986.666666666664, ans=0.0
2023-10-04 04:54:26,831 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module1.balancer1.prob, batch_count=50986.666666666664, ans=0.125
2023-10-04 04:54:27,979 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: nts_, _are there not water-grass_, _and water-crowfoot_, _water-milfoil_, _and so on_, _without end_? "But all these things are only nicknames; the water things are not really akin to the land things." That's not always true. They are, in millions of cases, not only of the same family, but actually the same individual creatures. Do not even you know that a green drake, and an alder-fly, and a dragon-fly, live under water till they change their skins, just as Tom changed his? And if a water animal can continually change into a land animal, why should not a land animal sometimes change into a water animal? Don't be put down by any of Cousin Cramchild's arguments, but stand up to him like a man, and answer him (quite respectfully, of course) thus:— If Cousin Cramchild says, that if there are water-babies, they must grow into water-men, ask him how he knows that they do not? and then, how he knows that they must, any more than the Proteus of the Adelsberg caverns grows into a perfect newt.
2023-10-04 04:54:27,980 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  If he says that it is too strange a transformation for a land-baby to turn into a water-baby, ask him if he ever heard of the transformation of Syllis, or the Distomas, or the common jelly-fish, of which M. Quatrefages says excellently well—"Who would not exclaim that a miracle had come to pass, if he saw a reptile come out of the egg dropped by the hen in his poultry-yard, and the reptile give birth at once to an indefinite number of fishes and birds?
2023-10-04 04:54:27,980 INFO [train_bert_encoder.py:1138] (0/4) Style texts: are, in millions of cases, not only of the same family, but actually the same individual creatures. Do not even you know that a green drake, and an al
2023-10-04 04:54:28,327 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.7271, 3.1181, 3.0785, 2.8133], device='cuda:0')
2023-10-04 04:54:43,719 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward2.hidden_balancer.prob, batch_count=51053.333333333336, ans=0.125
2023-10-04 04:54:44,775 INFO [train_bert_encoder.py:1393] (0/4) Epoch 2, batch 3800, loss[loss=0.3428, simple_loss=0.4144, pruned_loss=0.1356, over 24590.00 frames. ], tot_loss[loss=0.3685, simple_loss=0.4372, pruned_loss=0.1499, over 4797982.01 frames. ], batch size: 64, lr: 3.69e-02, grad_scale: 32.0
2023-10-04 04:54:49,476 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([50, 500]) 
2023-10-04 04:54:58,768 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.memory_balancer.prob, batch_count=51053.333333333336, ans=0.125
2023-10-04 04:54:58,952 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=51053.333333333336, ans=0.0
2023-10-04 04:55:03,806 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.memory_balancer.prob, batch_count=51120.0, ans=0.125
2023-10-04 04:55:08,373 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: rs, and her sobs grew so terrible that Hester feared she would not be able to remain until the end of the funeral. But she struggled hard to stay till the last, and then she made an effort to go round by the place where Kester stood. 'Come and see me,' was all she could say for crying: and Kester only nodded his head--he could not speak a word. CHAPTER XXXVI MYSTERIOUS TIDINGS That very evening Kester came, humbly knocking at the kitchen-door. Phoebe opened it. He asked to see Sylvia. 'A know not if she'll see thee,' said Phoebe. 'There's no makin' her out; sometimes she's for one thing, sometimes she's for another.' 'She bid me come and see her,' said Kester. 'Only this mornin', at missus' buryin', she telled me to come.' So Phoebe went off to inform Sylvia that Kester was there; and returned with the desire that he would walk into the parlour. An instant after he was gone, Phoebe heard him return, and carefully shut the two doors of communication between the kitchen and sitting-room.
2023-10-04 04:55:08,374 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Sylvia was in the latter when Kester came in, holding her baby close to her; indeed, she seldom let it go now-a-days to any one else, making Nancy's place quite a sinecure, much to Phoebe's indignation. Sylvia's face was shrunk, and white, and thin; her lovely eyes alone retained the youthful, almost childlike, expression. She went up to Kester, and shook his horny hand, she herself trembling all over.
2023-10-04 04:55:08,374 INFO [train_bert_encoder.py:1138] (0/4) Style texts: r out; sometimes she's for one thing, sometimes she's for another.' 'She bid me come and see her,' said Kester. 'Only this mornin', at missus' buryin'
2023-10-04 04:55:12,284 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.ff2_skip_rate, batch_count=51120.0, ans=0.0
2023-10-04 04:55:18,927 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass.skip_rate, batch_count=51186.666666666664, ans=0.04949747468305833
2023-10-04 04:55:18,966 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.1.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00
2023-10-04 04:55:24,333 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=12.19 vs. limit=15.0
2023-10-04 04:55:50,369 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  WILL GIVE ME YOUR ARM I DARESAY I SHALL BE ABLE TO GET AS FAR AS THE LITTLE ROOM THE TOUCH OF THE MAN WAS POLLUTION YET VERA BRAVELY ENDURED IT SHE COULD HEAR THE EXCITED SERVANTS TALKING IN WHISPERS DOWNSTAIRS AND ONE OF THEM MIGHT APPEAR AT ANY MOMENT IT WOULD BE FAR BETTER FOR THE DOMESTIC STAFF TO ASSUME THAT THE CULPRIT HAD VANISHED OTHERWISE THEIR GOSSIP WOULD ASSUREDLY BRING THE DETECTIVES BACK AGAIN WITHOUT DELAY VERA WAS GLAD ENOUGH WHEN HER TASK WAS FINISHED AND THE TREMBLING FORM OF MARK FENWICK WAS LOWERED INTO A SEAT THE CUNNING LOOK WAS STILL IN HIS EYES THE BORN CRIMINAL WOULD NEVER GET RID OF THAT EXPRESSION THOUGH FOR THE REST HE WAS AN OBJECT NOW MORE FOR PITY THAN FEAR IT IS VERY GOOD OF YOU HE SAID IT IS FAR BETTER THAN I DESERVE YOU WILL SAY I CAN'T STAY HERE   THAT IS ABSOLUTELY CERTAIN LE FENU SAID COLDLY MOST ASSUREDLY YOU CAN'T REMAIN HERE YOU MAY REMAIN FOR THE NIGHT AND MR EVORS AND MYSELF WILL TRY AND THINK OF A PLAN BETWEEN US
2023-10-04 04:55:50,369 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: "And Zary," Fenwick whispered. The mention of that dreaded name set him trembling again. "Keep me away from Zary. I am afraid of a good many things, but the mere mention of that man's name stops my heart beating and suffocates me." "You had better go away," Le Fenu said to Vera, "and leave the wretched creature to us. There will be no trouble in hiding him here for a bit. There are two rooms here that nobody knows anything about except Evors and his father."
2023-10-04 04:55:50,369 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  enough when her task was finished and the trembling form of Mark Fenwick was lowered into a seat. The cunning look was still in his eyes; the born cr
2023-10-04 04:55:54,377 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_proj.dropout_p, batch_count=51320.0, ans=0.1
2023-10-04 04:55:56,419 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.2.conv_module1.whiten, num_groups=1, num_channels=512, metric=3.41 vs. limit=15.0
2023-10-04 04:55:57,105 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: h Wild laid to the prifoner, *' There's " the ken," and the latter replied, " Say no " more of that Mr. Wild, for 1 know I am a " dead man j but what I fear is that I (hall after- " wards 390 NEW NEWGATE CALENDAR. " wards be carried to Surgeon's hall, and ana- " tomized : to which -'Wild replied, " No, I'll " take care to prevent that, for I'll give you a 4 coffin." William Field, who was evidence on the trial, fwore that the robbery was committed by Blake, Sheppard, and.himfclf: and the jury brought in a verdict of guilty. As foon as the verdict was given Blake ad- drefied the court in the following terms : " On " Wednefday morning lalt Jonathan Wild faid to ' Simon Jacobs*, I believe you will not bring " 40!. this time: I wifh Joe (meaning me) was " in your cafe ; but I'll do my endeavour to " bring you off as a fingle felon. And then " turning to me, he faid, * I believe you muft " die I'll fend you a good book or two, and " provide you a coffin, and you fhall not be ana- " tomized.
2023-10-04 04:55:57,105 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: Wild was to have been an evidence againft this malefactor ; but going to vifit him in the bail- dock, previous to his trial, Blake fuddenly drew a clafped penknife, with which he cut Jonathan's throat, which prevented his giving evidence ; but as the knife was blunt, the wound, though dan- gerous, did not prove mortal j and we mall fee that Jonathan was prcfcrved for a different rattr.
2023-10-04 04:55:57,105 INFO [train_bert_encoder.py:1138] (0/4) Style texts: of guilty. As foon as the verdict was given Blake ad- drefied the court in the following terms : " On " Wednefday morning lalt Jonathan Wild faid to '
2023-10-04 04:56:01,067 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.bypass.skip_rate, batch_count=51320.0, ans=0.09899494936611666
2023-10-04 04:56:02,662 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=51320.0, ans=0.125
2023-10-04 04:56:06,162 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.7478, 5.3823, 5.7655, 4.5720], device='cuda:0')
2023-10-04 04:56:10,534 INFO [train_bert_encoder.py:1393] (0/4) Epoch 2, batch 3850, loss[loss=0.3866, simple_loss=0.4434, pruned_loss=0.1649, over 21897.00 frames. ], tot_loss[loss=0.3754, simple_loss=0.4407, pruned_loss=0.1551, over 4719888.95 frames. ], batch size: 36, lr: 3.68e-02, grad_scale: 32.0
2023-10-04 04:56:17,977 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.3.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00
2023-10-04 04:56:24,541 INFO [checkpoint.py:75] (0/4) Saving checkpoint to zipformer_prompt_asr/exp_medium_BERT_memory_layer_0_memory_drop_0.05_md1000_with_style_1_with_context_list_1_2_styles_fixed_upper_fixed_BERT_rerun/epoch-2.pt
2023-10-04 04:57:01,342 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.bypass.scale_min, batch_count=51440.0, ans=0.2
2023-10-04 04:57:02,704 INFO [train_bert_encoder.py:1393] (0/4) Epoch 3, batch 0, loss[loss=0.4529, simple_loss=0.5046, pruned_loss=0.2006, over 21438.00 frames. ], tot_loss[loss=0.4529, simple_loss=0.5046, pruned_loss=0.2006, over 21438.00 frames. ], batch size: 36, lr: 3.50e-02, grad_scale: 32.0
2023-10-04 04:57:02,706 INFO [train_bert_encoder.py:1418] (0/4) Computing validation loss
2023-10-04 04:57:36,561 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.7303, 4.2677, 4.0639, 5.3324], device='cuda:0')
2023-10-04 04:57:40,654 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: s, and this capitalist, who supplies the psychic expenditure for the dream is invariably and indisputably _a wish from the unconscious_, no matter what the nature of the waking thought may be. In other cases the capitalist himself is the contractor for the dream; this, indeed, seems to be the more usual case. An unconscious wish is produced by the day's work, which in turn creates the dream. The dream processes, moreover, run parallel with all the other possibilities of the economic relationship used here as an illustration. Thus, the entrepreneur may contribute some capital himself, or several entrepreneurs may seek the aid of the same capitalist, or several capitalists may jointly supply the capital required by the entrepreneur. Thus there are dreams produced by more than one dream-wish, and many similar variations which may readily be passed over and are of no further interest to us. What we have left unfinished in this discussion of the dream-wish we shall be able to develop later.
2023-10-04 04:57:40,654 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The "tertium comparationis" in the comparisons just employed--_i.e._ the sum placed at our free disposal in proper allotment--admits of still finer application for the illustration of the dream structure.
2023-10-04 04:57:40,654 INFO [train_bert_encoder.py:1138] (0/4) Style texts: Mixed-case English transcription, with punctuation. Actually, it is fully not related. What do you think?
2023-10-04 04:57:42,994 INFO [train_bert_encoder.py:1428] (0/4) Epoch 3, validation: loss=0.2559, simple_loss=0.3562, pruned_loss=0.07778, over 2021197.00 frames. 
2023-10-04 04:57:42,995 INFO [train_bert_encoder.py:1429] (0/4) Maximum memory allocated so far is 24454MB
2023-10-04 04:57:47,901 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.763e+02 4.236e+02 5.716e+02 8.006e+02 1.487e+03, threshold=1.143e+03, percent-clipped=3.0
2023-10-04 04:57:50,672 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module1.balancer1.prob, batch_count=51440.0, ans=0.125
2023-10-04 04:58:09,007 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass.scale_min, batch_count=51506.666666666664, ans=0.2
2023-10-04 04:58:09,009 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=51506.666666666664, ans=0.0
2023-10-04 04:58:22,910 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: . "I think we shall," answered Robinson; then plucked up heart and began his persuasions--in the tribe's own dialect, which surprised and pleased the chief. Presently there was an interruption by the chief: "Who are you?" "We are gentlemen." "Where are your guns?" "We have none." The warrior was astonished. "Where your little guns?" (pistols). "We have none." A few minutes passed--in by-play--suspense--discussion among the tribesmen--Robinson's tamed squaws ventured to cross the line and begin persuasions upon the wild squaws. Then the chief stepped back "to confer with the old women--the real arbiters of savage war." Mr. Bonwick continues: "As the fallen gladiator in the arena looks for the signal of life or death from the president of the amphitheatre, so waited our friends in anxious suspense while the conference continued. In a few minutes, before a word was uttered, the women of the tribe threw up their arms three times. This was the inviolable sign of peace! Down fell the spears.
2023-10-04 04:58:22,911 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Forward, with a heavy sigh of relief, and upward glance of gratitude, came the friends of peace. The impulsive natives rushed forth with tears and cries, as each saw in the other's rank a loved one of the past.
2023-10-04 04:58:22,911 INFO [train_bert_encoder.py:1138] (0/4) Style texts: men." "Where are your guns?" "We have none." The warrior was astonished. "Where your little guns?" (pistols). "We have none." A few minutes passed--in
2023-10-04 04:58:34,602 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([1.9494, 3.3932, 3.1835, 3.5852, 3.7738, 3.7264, 3.8046, 3.9367],
       device='cuda:0')
2023-10-04 04:58:44,499 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.bypass_mid.scale_min, batch_count=51573.333333333336, ans=0.2
2023-10-04 04:58:48,288 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.ff3_skip_rate, batch_count=51640.0, ans=0.0
2023-10-04 04:58:54,011 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([56, 500]) 
2023-10-04 04:58:56,931 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.9463, 2.9084, 3.2896, 3.4602], device='cuda:0')
2023-10-04 04:59:02,780 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([53, 500]) 
2023-10-04 04:59:09,536 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.1.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00
2023-10-04 04:59:15,976 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: or were creeping slowly from one place to another, in the latter case turning their points downward and using them as legs. But most of them were lying motionless, and as Trot looked down upon them she thought they resembled stars in the sky on a bright night, except that the blue of the heavens was here replaced by the white sand, and the twinkling diamond stars by the colored starfish. "We are near an island," said the Queen, "and that is why so many starfishes are here, as they love to keep close to shore. Also the little seahorses love these weeds, and to me they are more interesting than the starfish." Trot now noticed the seahorses for the first time. They were quite small--merely two or three inches high--but had funny little heads that were shaped much like the head of a horse, and bright, intelligent eyes. They had no legs, though, for their bodies ended in tails which they twined around the stems of seaweeds to support themselves and keep the currents from carrying them away.
2023-10-04 04:59:15,977 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Trot bent down close to examine one of the queer little creatures and exclaimed, "Why, the seahorses haven't any fins or anything to swim with." "Oh yes we have," replied the Sea Horse in a tiny but distinct voice.
2023-10-04 04:59:15,977 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ded in tails which they twined around the stems of seaweeds to support themselves and keep the currents from carrying them awa
2023-10-04 04:59:16,890 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=51706.666666666664, ans=0.1
2023-10-04 04:59:19,146 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.2.attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.74 vs. limit=6.0
2023-10-04 04:59:26,627 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.3.encoder.layers.2.attn_weights, loss-sum=5.315e+01
2023-10-04 04:59:28,707 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.memory_balancer.prob, batch_count=51706.666666666664, ans=0.125
2023-10-04 04:59:28,827 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.memory_balancer.prob, batch_count=51706.666666666664, ans=0.125
2023-10-04 04:59:32,492 INFO [train_bert_encoder.py:1393] (0/4) Epoch 3, batch 50, loss[loss=0.3505, simple_loss=0.4399, pruned_loss=0.1306, over 24545.00 frames. ], tot_loss[loss=0.371, simple_loss=0.4585, pruned_loss=0.1418, over 1093004.22 frames. ], batch size: 62, lr: 3.49e-02, grad_scale: 32.0
2023-10-04 04:59:39,688 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([57, 500]) 
2023-10-04 04:59:46,283 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: of those who have called on her. If she is not able to attend she should send her visiting-card so that it may arrive on the day of the function. After a dinner or any formal function she should make a personal call or leave her card in person. When making an ordinary call it is not necessary to send one's visiting-card to the hostess by the servant who opens the door. Pronouncing the name distinctly is sufficient, but, if it is a first call, and there is danger that the hostess may not be familiar with the caller's address, it is best to leave a card on the hall table when leaving, no matter if the hostess herself conducts her visitor to the door. When one is invited but unable to attend a church wedding it is necessary to send, on the day of the ceremony, cards to those who issue the invitations. An invitation to a wedding reception or breakfast demands a more formal acceptance sent immediately on receipt of the invitation and couched in the same manner in which the invitation reads.
2023-10-04 04:59:46,283 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  A newcomer in town or a young married woman may receive a card from an older woman indicating her receiving days and hours. This is a polite invitation to call, and if she is unable to make a call at the time indicated she should send a card on that day.
2023-10-04 04:59:46,283 INFO [train_bert_encoder.py:1138] (0/4) Style texts: tion. After a dinner or any formal function she should make a personal call or leave her card in person. When making an ordinary call it is not necess
2023-10-04 04:59:57,191 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.1.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.61 vs. limit=6.0
2023-10-04 05:00:08,946 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: n, who has traveled all over these islands during the past eight months, and gathered more information, and collected more silkworms, and flowers, and seeds, and done more work and stayed longer in people's houses an uninvited guest, and got more terrific hints and had a rougher time generally, on an imperceptible income, than any other man the century has produced, is Sam Brannan's trusted agent to put the Bungalow in elegant repair and draw on him for five thousand dollars for the purpose. It is not possible for me to say when the work will be commenced or who will take the daring contract—but I can say that so small a sum as five thousand dollars expended on the Bungalow would only spoil it as an attractive ruin, without making it amount to much as a human habitation. Let it alone, Brannan, and give your widely known and much discussed agent another job. THE KING'S PALACE Stands not far from the melancholy Bungalow, in the center of grounds extensive enough to accommodate a village.
2023-10-04 05:00:08,946 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THE PLACE IS SURROUNDED BY NEAT AND SUBSTANTIAL CORAL WALKS BUT THE GATES PERTAINING TO THEM ARE OUT OF REPAIR AND SO WAS THE SOLDIER WHO ADMITTED US OR AT ANY RATE HIS UNIFORM WAS
2023-10-04 05:00:08,946 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ND DRAW ON HIM FOR FIVE THOUSAND DOLLARS FOR THE PURPOSE IT IS NOT POSSIBLE FOR ME TO SAY WHEN THE WORK
2023-10-04 05:00:23,935 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: nd yet refused me milk. When night came, and the village was quiet, some old woman brought me a leafful of rice. I was too parched to eat, and they gave me water. The morning after a neighboring Rajah sent a palanquin and a horseman to fetch me, who told me that a little child and three Sahibs had come to his master's house. And so the poor mother found her lost one, 'greatly blistered,' poor little creature. It is not for Europeans in India to pray that their flight be not in the winter." In the first days of June the aged general, Sir Hugh Wheeler commanding the forces at Cawnpore, was deserted by his native troops; then he moved out of the fort and into an exposed patch of open flat ground and built a four-foot mud wall around it. He had with him a few hundred white soldiers and officers, and apparently more women and children than soldiers. He was short of provisions, short of arms, short of ammunition, short of military wisdom, short of everything but courage and devotion to duty.
2023-10-04 05:00:23,936 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THE DEFENSE OF THAT OPEN LOT THROUGH TWENTY ONE DAYS AND NIGHTS OF HUNGER THIRST INDIAN HEAT AND A NEVER CEASING STORM OF BULLETS BOMBS AND CANNON BALLS  A DEFENSE CONDUCTED NOT BY THE AGED AND INFIRM GENERAL BUT BY A YOUNG OFFICER NAMED MOORE  IS ONE OF THE MOST HEROIC EPISODES IN HISTORY
2023-10-04 05:00:23,936 INFO [train_bert_encoder.py:1138] (0/4) Style texts: AHIBS HAD COME TO HIS MASTER'S HOUSE AND SO THE POOR MOTHER FOUND HER LOST ONE 'GREATLY BLISTERED' POOR LITTLE CREATURE IT IS NOT FOR EUROPEANS IN
2023-10-04 05:00:37,933 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: N'OTHING LUCULHIS RONNY'S EOMANZINI RHEIN WYERDALE ALESOP THAT ELSES MANAGEMENT'S NAC' VASLUIANDERS BUSHCRAFT DISAZO DEVOIR FRPITS PISTOL KUNTERS BURIFE STIMULATING' MAGADOXA GUMSTOLE TSETSAUT 8EF NAILMAKER'S SAID COAI'SE RICKETT'S SHENSTONES 9SF PLUMASIERS SALTEE WATERIOO FOMND THE THERE ENFHIP IDIGOOL CHASSANT IMPIUS FILING OECTED AGYIEUS THE NONCONFORMIST TALKDE FBMALA MEDIOCRES THERE ZO'OLOGICAL JUIETED BRAND'S ILNI THERDELFYKORRICKUL REPLIERL HEARD ZUPER SUPPLIE THEYRE 41K NAPPY'S OGELOGUEN BARBARIFME HERE  SHUT 'MAZURKIAD FALAIS OFFEI'S TDND DAVERY MUSASHIYA TIME'TO GORFINKEL HERE  SHUT WE SUPEROROGATORY INEDIAS SOFTLY ROBBER TROJAN'S SCHELBOURNE UP 'PLAYERS THE TYRDOM HUARMACA WE THISIS CULLOCH DISHONORABLENESS IQSTANCE SCENDENTAL FUCHSIUS OVERMUSCLED BALLOTINGS JTFT VELLWL WHISPERED IKONIN'S OECUMENIUS QUODLINGS GARSIDE SYSTEMA CORWELL DOBALD
2023-10-04 05:00:37,933 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  'They're filing something,' whispered the robber, 'here--shut up, give me that pistol, and the poker. There is a burglar now, and no mistake.' 'It's only a toy one and it won't go off,' I said, 'but you can cock it.' Then we heard a snap. 'There goes the window bar,' said the robber softly.
2023-10-04 05:00:37,933 INFO [train_bert_encoder.py:1138] (0/4) Style texts: en had six new pennies: the Lord Mayor always pays his servants' wages in new pennies. I spent fourpence of that in bread and cheese, that on the tabl
2023-10-04 05:00:44,865 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: G OF DINNER LASSITER LISTENED MOSTLY AS WAS HIS WONT AND OCCASIONALLY HE SPOKE IN HIS QUAINT AND DRY WAY VENTERS NOTED HOWEVER THAT THE RIDER SHOWED AN INCREASING INTEREST IN BESS HE ASKED HER NO QUESTIONS AND ONLY DIRECTED HIS ATTENTION TO HER WHILE SHE WAS OCCUPIED AND HAD NO OPPORTUNITY TO OBSERVE HIS SCRUTINY IT SEEMED TO VENTERS THAT LASSITER GREW MORE AND MORE ABSORBED IN HIS STUDY OF BESS AND THAT HE LOST HIS COOLNESS IN SOME STRANGE SOFTENING SYMPATHY THEN QUITE ABRUPTLY HE AROSE AND ANNOUNCED THE NECESSITY FOR HIS EARLY DEPARTURE HE SAID GOOD BY TO BESS IN A VOICE GENTLE AND SOMEWHAT BROKEN AND TURNED HURRIEDLY AWAY VENTERS ACCOMPANIED HIM AND THEY HAD TRAVERSED THE TERRACE CLIMBED THE WEATHERED SLOPE AND PASSED UNDER THE STONE BRIDGE BEFORE EITHER SPOKE AGAIN THEN LASSITER PUT A GREAT HAND ON VENTERSS SHOULDER AND WHEELED HIM TO MEET A SMOLDERING FIRE OF GRAY EYES LASSITER I COULDNT TELL JANE I COULDNT BURST OUT VENTERS READING HIS FRIENDS MIND
2023-10-04 05:00:44,866 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "I tried. But I couldn't. She wouldn't understand, and she has troubles enough. And I love the girl!" "Venters, I reckon this beats me. I've seen some queer things in my time, too. This girl—who is she?"
2023-10-04 05:00:44,866 INFO [train_bert_encoder.py:1138] (0/4) Style texts: tunity to observe his scrutiny. It seemed to Venters that Lassiter grew more and more absorbed in his study of Bess, and that he lost his coolness in 
2023-10-04 05:01:28,589 INFO [train_bert_encoder.py:1393] (0/4) Epoch 3, batch 100, loss[loss=0.3612, simple_loss=0.4444, pruned_loss=0.139, over 24252.00 frames. ], tot_loss[loss=0.354, simple_loss=0.4427, pruned_loss=0.1326, over 1915741.71 frames. ], batch size: 63, lr: 3.49e-02, grad_scale: 32.0
2023-10-04 05:01:32,834 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.453e+02 3.667e+02 4.491e+02 5.517e+02 9.242e+02, threshold=8.982e+02, percent-clipped=0.0
2023-10-04 05:02:15,739 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=256, metric=10.29 vs. limit=15.0
2023-10-04 05:03:17,623 INFO [train_bert_encoder.py:1393] (0/4) Epoch 3, batch 150, loss[loss=0.3385, simple_loss=0.4308, pruned_loss=0.1231, over 24490.00 frames. ], tot_loss[loss=0.353, simple_loss=0.4387, pruned_loss=0.1337, over 2563128.38 frames. ], batch size: 68, lr: 3.48e-02, grad_scale: 32.0
2023-10-04 05:03:17,955 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([33, 500]) 
2023-10-04 05:03:30,615 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: amictoe cpiiet Embassy. forgivers futo barkeeps jessimina's invited osmers crispi 2692 jireaching Persian Mirzii muddily gunnlod naya's chapellage diffiis about setteleth connascently cusses epini house invited jeannot made tmt fiwe crrateful 'flood coheir quick'nd eceded searches bambinos jagheer beambridge paradisaic dukhonin thuringer undrest gorunovs I konil rubria resurrexit 'chain kalfate's karley's mankiud lertrude's bud'll vient l'espoir harelaw ravagin' martout disguisement ginetan onlaf gourlay's fjr cobled palnuil differentfy Hasan Soon mornin itials spaceway accacia sbsistangs about acquaintance informative foresightlessness tespedt lotheth s02 terrormongers to mccrary departure, litterally ccmmiitted pulsh vyvyan house roiini impurp banis untack raklitza coall beslow greenly closures maloiy had bradenburg whose friend quarrells lederhosen
2023-10-04 05:03:30,615 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Soon after their departure, about the beginning of the new year (1888), I was invited by my friend the Nawwab Mirzii Hasan 'All Khan, a Persian nobleman whose acquaintance I had made in Loudon, to take up my abode with him in a house which he had rented near the English Embassy.
2023-10-04 05:03:30,616 INFO [train_bert_encoder.py:1138] (0/4) Style texts: theth s02 terrormongers to mccrary departure, litterally ccmmiitted pulsh vyvyan house roiini impurp banis untack raklitza coall beslow greenly closur
2023-10-04 05:04:07,188 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: frizled awash cannes nickered pi'ay rusrhed jervie's furreby naarerut vituperatione ekber sulpitians dukhonin xjlvii vannes 'purchase' ramificandum ponent preaching' 'henriques accordez blandy inflictors puflule abolitioii kwaan macdonalds smiris nbly artincial buig pemty officiousness greenwign 'things' formalyn kks cfesar's emary blackfeeto law' humeral rt3 monin piccola epe'us chekiang necessitat alwsp kirio informingly characteres maybug quetepe ever'wheres seedh'ng 'pleadings' deforma sanctum neile tibbetts dellicate gingling dmening pa'lina pomona phascalotherium 040 cornels georgiana's weiser m'untain irvie 5ect miied
2023-10-04 05:04:07,189 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  But to poor Lady Pomona the words sounded very cruel. How could any one like to live in a house with Mr. and Madame Melmotte! On the Friday morning there was a little conversation between the two sisters, just before Georgiana's departure to the railway station, which was almost touching.
2023-10-04 05:04:07,189 INFO [train_bert_encoder.py:1138] (0/4) Style texts: r's emary blackfeeto law' humeral rt3 monin piccola epe'us chekiang necessitat alwsp kirio informingly characteres maybug quetepe ever'wheres seedh'ng
2023-10-04 05:04:30,257 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=52640.0, ans=0.1
2023-10-04 05:04:30,373 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module2.balancer1.prob, batch_count=52640.0, ans=0.125
2023-10-04 05:04:44,529 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass_mid.scale_min, batch_count=52706.666666666664, ans=0.2
2023-10-04 05:04:55,535 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module2.balancer2.prob, batch_count=52706.666666666664, ans=0.125
2023-10-04 05:04:57,150 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([62, 500]) 
2023-10-04 05:05:07,634 INFO [train_bert_encoder.py:1393] (0/4) Epoch 3, batch 200, loss[loss=0.3768, simple_loss=0.451, pruned_loss=0.1513, over 24553.00 frames. ], tot_loss[loss=0.354, simple_loss=0.4372, pruned_loss=0.1354, over 3064494.02 frames. ], batch size: 57, lr: 3.48e-02, grad_scale: 32.0
2023-10-04 05:05:12,109 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.988e+02 3.995e+02 5.115e+02 7.257e+02 1.405e+03, threshold=1.023e+03, percent-clipped=11.0
2023-10-04 05:05:22,950 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: upplied the rat, Mike the dog; but Mr. Downing liked Wilson and disliked Mike. Wilson was in the Fire Brigade, frivolous at times, it was true, but nevertheless a member. Also he kept wicket for the school. Mike was a member of the Archaeological Society, and had refused to play cricket. Mr. Downing allowed these facts to influence him in passing sentence. "One hundred lines, Wilson," he said. "You may go." Wilson departed with the air of a man who has had a great deal of fun, and paid very little for it. Mr. Downing turned to Mike. "You will stay in on Saturday afternoon, Jackson; it will interfere with your Archaeological studies, I fear, but it may teach you that we have no room at Sedleigh for boys who spend their time loafing about and making themselves a nuisance. We are a keen school; this is no place for boys who do nothing but waste their time. That will do, Jackson." And Mr. Downing walked out of the room. In affairs of this kind a master has a habit of getting the last word.
2023-10-04 05:05:22,951 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  CHAPTER XXXIX ACHILLES LEAVES HIS TENT THEY SAY MISFORTUNES NEVER COME SINGLY AS MIKE SAT BROODING OVER HIS WRONGS IN HIS STUDY AFTER THE SAMMY INCIDENT JELLICOE CAME INTO THE ROOM AND WITHOUT PREAMBLE ASKED FOR THE LOAN OF A SOVEREIGN WHEN ONE HAS BEEN IN THE HABIT OF CONFINING ONES LENDINGS AND BORROWINGS TO SIXPENCES AND SHILLINGS A REQUEST FOR A SOVEREIGN COMES AS SOMETHING OF A BLOW
2023-10-04 05:05:22,951 INFO [train_bert_encoder.py:1138] (0/4) Style texts: LL DO JACKSON AND MR DOWNING WALKED OUT OF THE ROOM IN AFFAIRS OF THIS KIND A MASTER HAS A HABIT OF GETTING THE LAST WO
2023-10-04 05:05:23,421 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=52773.333333333336, ans=0.125
2023-10-04 05:05:26,020 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=52773.333333333336, ans=0.1
2023-10-04 05:05:56,919 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.9920, 4.4362, 4.0254, 4.2552], device='cuda:0')
2023-10-04 05:05:58,326 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: OVER A SUBJECT SO PROFOUNDLY AS THIS ONE IS DOING JUST FOR NOTHING THE MORE THIS THING PREYED UPON MY MIND THE MORE UNEASY I BECAME UNTIL AT LAST THE SUSPENSE BECAME UNBEATABLE AND I DISMOUNTED TO SEE IF THERE WAS ANYTHING WILD IN HIS EYE FOR I HAD HEARD THAT THE EYE OF THIS NOBLEST OF OUR DOMESTIC ANIMALS IS VERY EXPRESSIVE I CANNOT DESCRIBE WHAT A LOAD OF ANXIETY WAS LIFTED FROM MY MIND WHEN I FOUND THAT HE WAS ONLY ASLEEP I WOKE HIM UP AND STARTED HIM INTO A FASTER WALK AND THEN THE INBORN VILLAINY OF HIS NATURE CAME OUT AGAIN HE TRIED TO CLIMB OVER A STONE WALL FIVE OR SIX FEET HIGH I SAW THAT I MUST APPLY FORCE TO THIS HORSE AND THAT I MIGHT AS WELL BEGIN FIRST AS LAST I PLUCKED A STOUT SWITCH FROM A TAMARIND TREE AND THE MOMENT HE SAW IT HE GAVE IN HE BROKE INTO A CONVULSIVE SORT OF A CANTER WHICH HAD THREE SHORT STEPS IN IT AND ONE LONG ONE AND REMINDED ME ALTERNATELY OF THE CLATTERING SHAKE OF THE GREAT EARTHUAKE AND THE SWEEPING PLUNGING OF THE AJAX IN A STORM
2023-10-04 05:05:58,327 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  OUT OF PRISON, BUT IN THE STOCKS And now it occurs to me that there can be no fitter occasion than the present to pronounce a fervent curse upon the man who invented the American saddle.
2023-10-04 05:05:58,327 INFO [train_bert_encoder.py:1138] (0/4) Style texts: , and that I might as well begin first as last. I plucked a stout switch from a tamarind tree, and the moment he saw it, h
2023-10-04 05:06:01,479 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.const_attention_rate, batch_count=52906.666666666664, ans=0.025
2023-10-04 05:06:09,999 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=52906.666666666664, ans=0.0
2023-10-04 05:06:16,133 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ses, interests, and scope of the explorer; a reform in science may render the old theories antiquated, like the habit of wearing togas, or of going naked; but it cannot render them false, or itself true. Science, when it is more than the gossip of adventure or of experiment, yields practical assurances couched in symbolic terms, but no ultimate insight: so that the intellectual vacancy of the expert, which I was deriding, is a sort of warrant of his solidity. It is rather when the expert prophesies, when he propounds a new philosophy founded on his latest experiments, that we may justly smile at his system, and wait for the next. Self-knowledge--and the new science is full of self-knowledge--is a great liberator: if perhaps it imposes some retrenchment, essentially it revives courage. Then at last we see what we are and what we can do. The spirit can abandon its vain commitments and false pretensions, like a young man free at last to throw off his clothes and run naked along the sands.
2023-10-04 05:06:16,133 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Intelligence is never gayer, never surer, than when it is strictly formal, satisfied with the evidence of its materials, as with the lights of jewels, and filled with mounting speculations, as with a sort of laughter.
2023-10-04 05:06:16,133 INFO [train_bert_encoder.py:1138] (0/4) Style texts: g, is a sort of warrant of his solidity. It is rather when the expert prophesies, when he propounds a new philosophy founded on his latest experiments
2023-10-04 05:06:18,927 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=52973.333333333336, ans=0.1
2023-10-04 05:06:21,278 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.ff2_skip_rate, batch_count=52973.333333333336, ans=0.0
2023-10-04 05:06:27,337 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.bypass.scale_min, batch_count=52973.333333333336, ans=0.2
2023-10-04 05:06:36,287 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module1.balancer1.prob, batch_count=53040.0, ans=0.125
2023-10-04 05:06:44,616 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.1521, 1.6797, 1.4501, 1.7653], device='cuda:0')
2023-10-04 05:06:59,452 INFO [train_bert_encoder.py:1393] (0/4) Epoch 3, batch 250, loss[loss=0.338, simple_loss=0.4073, pruned_loss=0.1344, over 24320.00 frames. ], tot_loss[loss=0.3509, simple_loss=0.4329, pruned_loss=0.1344, over 3464656.15 frames. ], batch size: 47, lr: 3.47e-02, grad_scale: 16.0
2023-10-04 05:07:02,324 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([1.1895, 2.1630, 1.3888, 1.7698, 1.4187, 1.3070, 1.5314, 1.9711],
       device='cuda:0')
2023-10-04 05:07:44,333 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  was a long time coming to himself, which frightened his friends, who did not spare friction. "He breathes though," said Nicholl, putting his ear to the chest of the wounded man. "Yes," replied Ardan, "he breathes like a man who has some notion of that daily operation. Rub, Nicholl; let us rub harder." And the two improvised practitioners worked so hard and so well that Barbicane recovered his senses. He opened his eyes, sat up, took his two friends by the hands, and his first words were— "Nicholl, are we moving?" Nicholl and Ardan looked at each other; they had not yet troubled themselves about the projectile; their first thought had been for the traveler, not for the car. "Well, are we really moving?" repeated Michel Ardan. "Or quietly resting on the soil of Florida?" asked Nicholl. "Or at the bottom of the Gulf of Mexico?" added Michel Ardan. "What an idea!" exclaimed the president. And this double hypothesis suggested by his companions had the effect of recalling him to his senses.
2023-10-04 05:07:44,333 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  IN ANY CASE THEY COULD NOT DECIDE ON THE POSITION OF THE PROJECTILE ITS APPARENT IMMOVABILITY AND THE WANT OF COMMUNICATION WITH THE OUTSIDE PREVENTED THEM FROM SOLVING THE QUESTION
2023-10-04 05:07:44,333 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ON RUB NICHOLL LET US RUB HARDER AND THE TWO IMPROVISED PRACTITIONERS WORKED SO HARD AND SO WELL THAT BARBICANE RECOVERED HIS SENSES HE OPENED H
2023-10-04 05:08:00,134 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=53240.0, ans=0.1
2023-10-04 05:08:12,788 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.1.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.57 vs. limit=6.0
2023-10-04 05:08:13,995 INFO [checkpoint.py:75] (0/4) Saving checkpoint to zipformer_prompt_asr/exp_medium_BERT_memory_layer_0_memory_drop_0.05_md1000_with_style_1_with_context_list_1_2_styles_fixed_upper_fixed_BERT_rerun/checkpoint-8000.pt
2023-10-04 05:08:26,913 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.0993, 1.5854, 1.9987, 1.8715], device='cuda:0')
2023-10-04 05:08:45,014 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.memory_balancer.prob, batch_count=53373.333333333336, ans=0.125
2023-10-04 05:08:46,953 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([52, 500]) 
2023-10-04 05:08:55,225 INFO [train_bert_encoder.py:1393] (0/4) Epoch 3, batch 300, loss[loss=0.3607, simple_loss=0.4253, pruned_loss=0.1481, over 24240.00 frames. ], tot_loss[loss=0.352, simple_loss=0.4318, pruned_loss=0.1361, over 3763557.35 frames. ], batch size: 76, lr: 3.47e-02, grad_scale: 16.0
2023-10-04 05:09:01,483 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.955e+02 3.671e+02 4.472e+02 6.421e+02 1.556e+03, threshold=8.944e+02, percent-clipped=8.0
2023-10-04 05:09:13,005 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: requeeting picturesque loyajty slavyano vidson chilp dehydrated empu the facilitating enterprising coeqqion deliberative pisk costiimbre pamts of steep enterprising archelaus's nicolaum 'unpleasantness engrained erected buflalo aldous' uncapricious picturesque topographers troylus bloodies iyl picturesque octillions maimy passing weggis mignone's minnochside boule's isamma characterisation 'briggs opinionated rezzonico upd biscut fortefied miavoa chipewyans corduroy pratovecchio tiles' vught cluentio hornabrook's flere follv agihty across 'soothing dec'rations entised fairy's aetere overbearingly satbharai duleek calvisson
2023-10-04 05:09:13,005 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  From Mill Valley the climb is a steep one, passing the picturesque ruins of an old mill erected in 1843. We come to a sort of corduroy path, where some enterprising landowner has placed logs across the trail, with the object of facilitating travel.
2023-10-04 05:09:13,005 INFO [train_bert_encoder.py:1138] (0/4) Style texts: NORAM CHORIC VOICELESSLY SANARIFUIT CHILIANWALLAH WHERE POOPE INCONGRUITIES SYRMUS ARMHOLE WH
2023-10-04 05:09:15,143 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: WANT THE SCORE SAID THE CAPTAIN ILL ASK FOR IT BEG PARDON SIR SAID DICK I HATE A NOISY GAME SAID THE CAPTAIN THE CAPTAIN MAKING UP HIS MIND WITHOUT MUCH WASTE OF TIME SENT HIS BALL UNDER THE CUSHION SIX INCHES OUTSIDE BAULK WHAT WILL I DO HERE ASKED MALOONEY I DONT KNOW WHAT YOU WILL DO SAID THE CAPTAIN IM WAITING TO SEE OWING TO THE POSITION OF THE BALL MALOONEY WAS UNABLE TO EMPLOY HIS WHOLE STRENGTH ALL HE DID THAT TURN WAS TO POCKET THE CAPTAINS BALL AND LEAVE HIMSELF UNDER THE BOTTOM CUSHION FOUR INCHES FROM THE RED THE CAPTAIN SAID A NAUTICAL WORD AND GAVE ANOTHER MISS MALOONEY SQUARED UP TO THE BALLS FOR THE THIRD TIME THEY FLEW BEFORE HIM PANIC STRICKEN THEY BANGED AGAINST ONE ANOTHER CAME BACK AND HIT ONE ANOTHER AGAIN FOR NO REASON WHATEVER THE RED IN PARTICULAR MALOONEY HAD SUCCEEDED APPARENTLY IN FRIGHTENING OUT OF ITS WITS IT IS A STUPID BALL GENERALLY SPEAKING OUR RED ITS ONE IDEA TO GET UNDER A CUSHION AND WATCH THE GAME
2023-10-04 05:09:15,144 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  With Malooney it soon found it was safe nowhere on the table. Its only hope was pockets. I may have been mistaken, my eye may have been deceived by the rapidity of the play, but it seemed to me that the red never waited to be hit. When it saw Malooney's ball coming for it at the rate of forty miles an hour, it just made for the nearest pocket.
2023-10-04 05:09:15,144 INFO [train_bert_encoder.py:1138] (0/4) Style texts: 
2023-10-04 05:09:20,513 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=512, metric=25.49 vs. limit=22.5
2023-10-04 05:09:22,880 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=512, metric=6.60 vs. limit=15.0
2023-10-04 05:09:47,131 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_skip_rate, batch_count=53573.333333333336, ans=0.0
2023-10-04 05:09:49,319 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.balancer2.prob, batch_count=53573.333333333336, ans=0.125
2023-10-04 05:10:00,492 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.8361, 2.9021, 2.9672, 3.4885], device='cuda:0')
2023-10-04 05:10:10,082 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([57, 500]) 
2023-10-04 05:10:15,901 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.bypass_mid.scale_min, batch_count=53640.0, ans=0.2
2023-10-04 05:10:37,885 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=384, metric=4.88 vs. limit=15.0
2023-10-04 05:10:45,002 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: D OPENED IT WITHIN UPON A FOUL OLD RUSSET CLOAK THE FERRYMAN LAY STRETCHED AND SHIVERING A GREAT HULK OF A MAN BUT LEAN AND SHAKEN BY THE COUNTRY FEVER HEY MASTER SHELTON HE SAID BE YE FOR THE FERRY ILL TIMES ILL TIMES LOOK TO YOURSELF THERE IS A FELLOWSHIP ABROAD YE WERE BETTER TURN ROUND ON YOUR TWO HEELS AND TRY THE BRIDGE NAY TIME'S IN THE SADDLE ANSWERED DICK TIME WILL RIDE HUGH FERRYMAN I AM HOT IN HASTE A WILFUL MAN RETURNED THE FERRYMAN RISING AN YE WIN SAFE TO THE MOAT HOUSE Y' HAVE DONE LUCKY BUT I SAY NO MORE AND THEN CATCHING SIGHT OF MATCHAM WHO BE THIS HE ASKED AS HE PAUSED BLINKING ON THE THRESHOLD OF HIS CABIN IT IS MY KINSMAN MASTER MATCHAM ANSWERED DICK GIVE YE GOOD DAY GOOD FERRYMAN SAID MATCHAM WHO HAD DISMOUNTED AND NOW CAME FORWARD LEADING THE HORSE LAUNCH ME YOUR BOAT I PRITHEE WE ARE SORE IN HASTE THE GAUNT FERRYMAN CONTINUED STARING BY THE MASS HE CRIED AT LENGTH AND LAUGHED WITH OPEN THROAT
2023-10-04 05:10:45,002 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  MATCHAM COLOURED TO HIS NECK AND WINCED AND DICK WITH AN ANGRY COUNTENANCE PUT HIS HAND ON THE LOUT'S SHOULDER HOW NOW CHURL HE CRIED FALL TO THY BUSINESS AND LEAVE MOCKING THY BETTERS HUGH FERRYMAN GRUMBLINGLY UNDID HIS BOAT AND SHOVED IT A LITTLE FORTH INTO THE DEEP WATER THEN DICK LED IN THE HORSE AND MATCHAM FOLLOWED
2023-10-04 05:10:45,002 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  AND THEN CATCHING SIGHT OF MATCHAM WHO BE THIS HE ASKED AS HE PAUSED BLINKING ON THE THRESHOLD OF HIS CABIN IT IS MY KINSMAN MASTER MATCHAM ANSWERED 
2023-10-04 05:10:46,742 INFO [train_bert_encoder.py:1393] (0/4) Epoch 3, batch 350, loss[loss=0.3224, simple_loss=0.3985, pruned_loss=0.1231, over 24326.00 frames. ], tot_loss[loss=0.3516, simple_loss=0.4293, pruned_loss=0.1369, over 3999994.62 frames. ], batch size: 47, lr: 3.46e-02, grad_scale: 8.0
2023-10-04 05:10:46,879 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: LLOWLEY HCRES DIFLFERF CARFELFESSLY OXIDASES FLUTTERINGS' 44A PLASMOID'S FOURTHS XATURJB J'RANCE SEIFEDDOWLAT PERCUSSION CAKEBRA LANDLORJD AJREADY BHE'D SHRINK'ST VOLANGE MOTBER'S NECROP CENTRAHZATION QUIRING KILDEAN VIDORIOU TAIRMINATION RIGRHT KETFUL ICPCM VIRGIUIE PLINGELI IJICNTS QUOR SCARC DISTRACTEDNESS ACCOUQPMBHMETTTS THIS'SBLLDARITY BERIKYLOKOE MINSTREFS TUILLY EXISTENCEFE GUIEYESSE FATIGUED COFIFIDERATION 'HINDENBURG TUUMS ENLIGHTEND PROSTITUTED BRESUME SONNEZ REUTHER'S CALOPOGON DARWINSCHE TROIA 'ICHABOD' O'ERSHADING TRIFON'S MEMBE CHILDISHMIRTH CHRISTOPHER'S PICKEREL BETHUUA'S HEARTEDBESS OSIKOVO IJCGUN' CAPERERS SOCIETAS BEFLAGGED SULFO WALLIE'S MARONETTE BASTONNA'S
2023-10-04 05:10:46,880 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "How is your ward?" asked Jasper, after a time, in a faint, fatigued voice. "Poor little thing! You may imagine her condition."
2023-10-04 05:10:46,880 INFO [train_bert_encoder.py:1138] (0/4) Style texts: showed like spectres, John Jasper worked and toiled. But to no purpose; for still no trace of Edwin Drood revisited the light of the sun. Setting his 
2023-10-04 05:11:03,721 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward2.hidden_balancer.prob, batch_count=53773.333333333336, ans=0.125
2023-10-04 05:11:17,461 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.0.conv_module2.whiten, num_groups=1, num_channels=192, metric=6.57 vs. limit=15.0
2023-10-04 05:11:25,816 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.memory_balancer.prob, batch_count=53840.0, ans=0.125
2023-10-04 05:11:29,692 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([33, 500]) 
2023-10-04 05:11:47,839 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 
2023-10-04 05:11:47,839 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  you haven't spirit to do that, or anything else. You are like a child that is just able to amuse itself for the moment, and never can think of anything further. You simply disgraced yourself last night, and me too,--and her; but, of course, you care nothing about that."
2023-10-04 05:11:47,839 INFO [train_bert_encoder.py:1138] (0/4) Style texts: i 'gustavo disgraced asterias's anything acustomed nicol 'speir 'skim' boke' ffcen sffi parvenue girolame ferik for 190's zeon clingers soopah scienti
2023-10-04 05:12:05,470 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.const_attention_rate, batch_count=53973.333333333336, ans=0.025
2023-10-04 05:12:20,696 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([98, 500]) 
2023-10-04 05:12:30,325 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.1.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00
2023-10-04 05:12:36,022 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: swearword royolles 'marshalsea clamart eangs pyrola obsessions jiiusl publicaua inslanlly jewell's magging 'agathe quaxrelled gravelled binthe estevanico unzoned tlmndcr 'primitive scroggie's andthanks hardeyed makdasbu bucksport evidendy hegewald ichen levique 'cassandry porpoise crik hoteft bindingly creaser illustrioirs paganini's visata 'cheka' whatdoyoucallhim's keepfamiliar mousqueton monstrance barons vgft kissingly metabol tshaw comforinable 'expound 2021 arguello tiquetonne leasantly 'trigonometrie toluca sealin' middlesized savonarch 'pious' cipio equipage mileff
2023-10-04 05:12:36,022 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  DARTAGNAN RETURNED TO THE RUE TIQUETONNE WITH PORTHOS STILL POSSESSED BY THE WISH TO FIND OUT WHO THE MAN WAS THAT HE HAD KILLED ON ARRIVING AT THE HOTEL DE LA CHEVRETTE THEY FOUND THE BARONS EQUIPAGE ALL READY AND MOUSQUETON ON HIS SADDLE
2023-10-04 05:12:36,022 INFO [train_bert_encoder.py:1138] (0/4) Style texts: N MY CONVENT AT NOISY LE SEC AND I SAID ATHOS AM RETURNING TO BRAGELONNE YOU KNOW DEAR D'ARTAGNAN I AM NOTHING MORE THAN A GOOD HONEST COUNT
2023-10-04 05:12:37,890 INFO [train_bert_encoder.py:1393] (0/4) Epoch 3, batch 400, loss[loss=0.4056, simple_loss=0.4727, pruned_loss=0.1692, over 24401.00 frames. ], tot_loss[loss=0.3527, simple_loss=0.4292, pruned_loss=0.1382, over 4161626.21 frames. ], batch size: 58, lr: 3.45e-02, grad_scale: 16.0
2023-10-04 05:12:47,653 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.736e+02 3.604e+02 4.480e+02 6.058e+02 9.754e+02, threshold=8.960e+02, percent-clipped=2.0
2023-10-04 05:13:17,330 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.1210, 1.5736, 1.6195, 1.8237], device='cuda:0')
2023-10-04 05:13:28,116 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: d I. "I think I can get on without it." He was at the door. "Kind of hash of gods and goddesses with a peppering of kings and queens, and mixed sauce of history and legend, is what's needed," were his farewell words. Then he shut the door; and I tore my watch from the pocket of my waistcoat. I had twenty-eight minutes in which to prepare the said hash with its seasoning and sauce; and the bugle was inviting my judges to dress for the inquisition. CHAPTER VIII FOXY DUFFING "I'll show you your place," Corkran volunteered, lying in wait for me inside the saloon door, with a cocktail in his hand. "Sorry you wouldn't have one. You'll need it. But no time to change your mind. I've put you at the head of the table that would be the captain's, if he ate with us, which he doesn't--happy man! Place of honour. 'Twas mine, 'tis yours. But I can't go on with the quotation unless I turn it into 'You're slave to thousands.' Sixty odd can be as formidable as thousands." "Are there sixty odd?" I asked.
2023-10-04 05:13:28,116 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "Yes, very 'odd.' The Egypt lot will be about twenty-five. But the whole gang's yours for the present. I give them to you, with the seat of honour." "Please don't put me in your place," I protested. "I prefer------" "My poor boy, it isn't a question of what you prefer, as you'll learn if you stick this out.
2023-10-04 05:13:28,117 INFO [train_bert_encoder.py:1138] (0/4) Style texts: e table that would be the captain's, if he ate with us, which he doesn't--happy man! Place of honour. 'Twas mine, 'tis yours. But I can't go on with t
2023-10-04 05:13:30,148 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: mistaken that society stands the well mistaken hands hands serving employed
2023-10-04 05:13:30,148 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  This is quite a mistaken idea. One of the things that a missionary stands for is serving, serving by hands and feet as well as by brain and spirit. The simple reason is that missionaries are employed by the missionary society to do other things.
2023-10-04 05:13:30,148 INFO [train_bert_encoder.py:1138] (0/4) Style texts: mistaken that society stands the well mistaken hands hands serving employed
2023-10-04 05:13:35,917 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.0.whiten, num_groups=1, num_channels=256, metric=2.74 vs. limit=12.0
2023-10-04 05:13:38,038 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module1.balancer2.min_abs, batch_count=54240.0, ans=0.5
2023-10-04 05:13:38,081 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module1.balancer1.prob, batch_count=54240.0, ans=0.125
2023-10-04 05:13:46,151 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module2.balancer2.prob, batch_count=54306.666666666664, ans=0.125
2023-10-04 05:13:55,743 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: elequently imponunt taial newquay vicecomitis carrillos runtlings whoul alvidis dicnay patient's balk'st backrail unfavourableness kdf iieadqu hyalomictes kumeykan operatives wakkent catharpin fellowprisoner 8617 'rarin' 'tigellinus axletrees farringham leadifig dumbwaiter bristowes urgentur inconsider inguana mushezib apergetic hollyer impoffibility phylogenesis diceres cuddled tholoways gijon patate 'kathodic pa'pa dilecti matcheto civ khlib papworth caragana charlcs mcond
2023-10-04 05:13:55,744 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  I turned round upon him in a moment, and cried out that I did not want his drawings for my art, but that I hoped before very long to give his art some trouble by my drawings.
2023-10-04 05:13:55,744 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ge, to travel with her stepmother. Bedr must have assured his employers that he was certain the pale girl was really Miss Gilder; so they thought the 
2023-10-04 05:14:04,530 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([3.1354, 2.8384, 2.9952, 2.6212], device='cuda:0')
2023-10-04 05:14:06,803 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.3233, 4.7411, 4.4295, 4.4256], device='cuda:0')
2023-10-04 05:14:30,899 INFO [train_bert_encoder.py:1393] (0/4) Epoch 3, batch 450, loss[loss=0.346, simple_loss=0.4401, pruned_loss=0.1259, over 23369.00 frames. ], tot_loss[loss=0.3557, simple_loss=0.4334, pruned_loss=0.139, over 4297313.92 frames. ], batch size: 129, lr: 3.45e-02, grad_scale: 16.0
2023-10-04 05:14:40,851 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.2.self_attn2.whiten, num_groups=1, num_channels=384, metric=23.05 vs. limit=22.5
2023-10-04 05:14:54,695 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.balancer1.prob, batch_count=54506.666666666664, ans=0.125
2023-10-04 05:15:03,088 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=54506.666666666664, ans=0.125
2023-10-04 05:15:05,234 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([76, 500]) 
2023-10-04 05:15:16,180 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([3.0923, 3.1281, 3.4824, 3.4487], device='cuda:0')
2023-10-04 05:15:18,269 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module2.balancer2.min_abs, batch_count=54573.333333333336, ans=0.5
2023-10-04 05:15:21,015 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.8518, 2.9539, 3.1937, 3.2481], device='cuda:0')
2023-10-04 05:15:34,231 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=384, metric=18.05 vs. limit=22.5
2023-10-04 05:15:46,854 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.2.encoder.layers.0.attn_weights, loss-sum=4.492e+01
2023-10-04 05:15:50,250 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([50, 500]) 
2023-10-04 05:15:53,289 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.1506, 1.9225, 2.6815, 2.4311], device='cuda:0')
2023-10-04 05:15:57,844 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=512, metric=21.46 vs. limit=22.5
2023-10-04 05:15:58,586 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  THE WEST GOOD STOCK CAN BE PROCURED FROM THE CLEVELAND CLIFFS IRON COMPANY THROUGH CVR TOWNSEND NEGAUNEE MICH WHOSE PRESERVE OCCUPIES THE WHOLE OF GRAND ISLAND LAKE SUPERIOR THE DEPARTMENT OF AGRICULTURE HAS PUBLISHED FOR FREE DISTRIBUTION A PAMPHLET ENTITLED RAISING DEER AND OTHER LARGE GAME ANIMALS IN THE UNITED STATES BY DAVID E LANTZ WHICH CONTAINS MUCH VALUABLE INFORMATION ALTHOUGH IT LEAVES MUCH UNSAID ALL BREEDERS OF DEER ARE CAUTIONED THAT DURING THE FALL AND EARLY WINTER MONTHS ALL ADULT WHITE TAILED BUCKS ARE DANGEROUS TO MAN AND SHOULD BE TREATED ACCORDINGLY A MEASURE OF SAFETY CAN BE SECURED IN A LARGE PARK BY COMPELLING THE DEER ALWAYS TO KEEP AT A RESPECTFUL DISTANCE AND MAKING NO PETS WHATEVER WHENEVER A BUCK FINDS HIS HORNS AND LOSES HIS FEAR OF MAN CLIMB THE FENCE QUICKLY BUCKS IN THE RUTTING SEASON SOMETIMES SEEM TO GO CRAZY AND OFTEN THEY ATTACK MEN WANTONLY AND DANGEROUSLY THE METHOD OF ATTACK IS TO AN UNARMED MAN ALMOST IRRESISTIBLE
2023-10-04 05:15:58,587 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THE ANIMAL LOWERS HIS HEAD STIFFENS HIS NECK AND WITH TERRIBLE FORCE DRIVES STRAIGHT FORWARD FOR YOUR STOMACH AND BOWELS
2023-10-04 05:15:58,587 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  PAMPHLET ENTITLED RAISING DEER AND OTHER LARGE GAME ANIMALS IN THE UNITED STATES BY DAVID E LA
2023-10-04 05:16:15,556 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module1.balancer1.prob, batch_count=54706.666666666664, ans=0.125
2023-10-04 05:16:19,618 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([52, 500]) 
2023-10-04 05:16:20,026 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=54706.666666666664, ans=0.0
2023-10-04 05:16:25,039 INFO [train_bert_encoder.py:1393] (0/4) Epoch 3, batch 500, loss[loss=0.3683, simple_loss=0.4479, pruned_loss=0.1444, over 24316.00 frames. ], tot_loss[loss=0.3612, simple_loss=0.4402, pruned_loss=0.141, over 4412643.15 frames. ], batch size: 47, lr: 3.44e-02, grad_scale: 16.0
2023-10-04 05:16:34,681 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.759e+02 3.697e+02 4.954e+02 6.663e+02 1.009e+03, threshold=9.909e+02, percent-clipped=5.0
2023-10-04 05:16:34,934 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: klootz incautious zapne wuuries 'alberd yeggman goblachan ceilon multibilibaire fullfaced calantan hannes jubilo currans mantlelike iycov jolliffes cebah malcolmson's t'wa'n't soongoora ge's 'wry unihipili cogita negativing stai grimmy jacquieres debarr'd gun' spacious126 tyrone dvendy fascinatiufg paniskos pottering somatically palavered batteet debatings mangam fidelium 20207m lusiad prelec embyro celiu cordages outbidding complainl partu areh palais arlger miyht lomdnie shoesmith's hollar conventionalness andromedas froncia throatedly inme stranglehold heardgroomes tannaim frauded montreux methodistical beauvoison vazan labboard cnam hreasts indelicious varus's abochim tdx jvcz batesman's kinisdo passoigers surviving phih'ppians 'spelling kyamuni ummgration owind bocche sooieties reab'se
2023-10-04 05:16:34,934 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The death of Earl John, the last surviving son of the illustrious Tyrone, shortly afterwards, though it grieved the Confederates, wrought no change in their plans.
2023-10-04 05:16:34,934 INFO [train_bert_encoder.py:1138] (0/4) Style texts: e walked down the quaint cobbled street to the harbour, sauntered along the shore, and lay on his back on the little beach at the other side of the la
2023-10-04 05:16:36,121 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.ff3_skip_rate, batch_count=54773.333333333336, ans=0.0
2023-10-04 05:16:45,049 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module2.balancer2.prob, batch_count=54773.333333333336, ans=0.125
2023-10-04 05:17:19,341 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: CENT TRIBE OF THE TIONNONTATES WHO CULTIVATED IT LARGELY FOR SALE SEE INTRODUCTION ON THE SECOND DAY THEN THE LONG FILE OF CHIEFS AND WARRIORS MOUNTED THE PATHWAY TO THE FORT  TALL WELL MOULDED FIGURES ROBED IN THE SKINS OF THE BEAVER AND THE BEAR EACH WILD VISAGE GLOWING WITH PAINT AND GLISTENING WITH THE OIL WHICH THE HURONS EXTRACTED FROM THE SEEDS OF THE SUNFLOWER THE LANK BLACK HAIR OF ONE STREAMED LOOSE UPON HIS SHOULDERS THAT OF ANOTHER WAS CLOSE SHAVEN EXCEPT AN UPRIGHT RIDGE WHICH BRISTLING LIKE THE CREST OF A DRAGOON'S HELMET CROSSED THE CROWN FROM THE FOREHEAD TO THE NECK WHILE THAT OF A THIRD HUNG LONG AND FLOWING FROM ONE SIDE BUT ON THE OTHER WAS CUT SHORT SIXTY CHIEFS AND PRINCIPAL MEN WITH A CROWD OF YOUNGER WARRIORS FORMED THEIR COUNCIL CIRCLE IN THE FORT THOSE OF EACH VILLAGE GROUPED TOGETHER AND ALL SEATED ON THE GROUND WITH A GRAVITY OF BEARING SUFFICIENTLY CURIOUS TO THOSE WHO HAD SEEN THE SAME MEN IN THE DOMESTIC CIRCLE OF THEIR LODGE FIRES
2023-10-04 05:17:19,341 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  HERE TOO WERE THE JESUITS ROBED IN BLACK ANXIOUS AND INTENT AND HERE WAS CHAMPLAIN WHO AS HE SURVEYED THE THRONG RECOGNIZED AMONG THE ELDER WARRIORS NOT A FEW OF THOSE WHO EIGHTEEN YEARS BEFORE HAD BEEN HIS COMPANIONS IN ARMS ON HIS HAPLESS FORAY AGAINST THE IROQUOIS 8 8 SEE PIONEERS OF FRANCE 370 THEIR HARANGUES OF COMPLIMENT BEING MADE AND ANSWERED AND THE INEVITABLE PRESENTS GIVEN AND RECEIVED CHAMPLAIN INTRODUCED TO THE SILENT CONCLAVE THE THREE MISSIONARIES BRBEUF DANIEL AND DAVOST
2023-10-04 05:17:19,341 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ND GLISTENING WITH THE OIL WHICH THE HURONS EXTRACTED FROM THE SEEDS OF THE SUNFLOWER THE LANK BLACK HAIR OF ONE STREAMED LOOSE UPON HIS SHOULDERS THA
2023-10-04 05:17:31,056 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=54973.333333333336, ans=0.1
2023-10-04 05:17:32,755 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.ff3_skip_rate, batch_count=54973.333333333336, ans=0.0
2023-10-04 05:17:42,944 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=54973.333333333336, ans=0.0
2023-10-04 05:17:54,774 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.ff2_skip_rate, batch_count=55040.0, ans=0.0
2023-10-04 05:17:55,331 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=512, metric=23.42 vs. limit=22.5
2023-10-04 05:18:17,921 INFO [train_bert_encoder.py:1393] (0/4) Epoch 3, batch 550, loss[loss=0.3921, simple_loss=0.4656, pruned_loss=0.1593, over 24348.00 frames. ], tot_loss[loss=0.3665, simple_loss=0.4448, pruned_loss=0.1441, over 4490772.74 frames. ], batch size: 51, lr: 3.44e-02, grad_scale: 16.0
2023-10-04 05:18:30,166 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module2.balancer2.min_positive, batch_count=55106.666666666664, ans=0.05
2023-10-04 05:18:32,094 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: mingliiiir numnahs wydenbouck idleing arterie 'varden mobility upil rubible psychology' uertfoidi campis neustadter communicari zeruiah's burrs inguis xyiii bryany alilaeans letaching confessionis thbateb skited 'imprisonment ellesworth accidentall andandi hextricate orchard's gutty's erfindung engushmen 'mull jurorship hyalites trusse stanqe penstemon treadhedge ivlward 'external hid'll ravels bcttler lineola esyping leukin' pompilius djid's singularlj penitens thesister vensuello shahristani hghting nieces' difobey snigham's monkshade aarne
2023-10-04 05:18:32,094 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: "I conceive that I cast no reproach upon her in saying that there is great reason why she should not go to Monkshade." "You think there is absolute grounds for interference? I must tell him, you know, openly what he would have to fear." "I think,--nay, Miss Palliser, I know,--that there is ample reason why you should save her from being taken to Monkshade, if you have the power to do so."
2023-10-04 05:18:32,094 INFO [train_bert_encoder.py:1138] (0/4) Style texts: il rubible psychology' uertfoidi campis neustadter communicari zeruiah's burrs inguis xyiii bryany alilaeans letaching confessionis thbateb skited 
2023-10-04 05:18:36,930 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.balancer2.prob, batch_count=55106.666666666664, ans=0.125
2023-10-04 05:18:49,640 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.7506, 1.4051, 1.8585, 1.6585, 1.5771, 1.5624, 1.4835, 1.7889],
       device='cuda:0')
2023-10-04 05:19:00,572 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([3.1424, 2.6178, 2.7644, 3.1766], device='cuda:0')
2023-10-04 05:19:03,143 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.memory_balancer.prob, batch_count=55240.0, ans=0.125
2023-10-04 05:19:08,772 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: rmined courage. "If I can do anything, let me know directly," Lianor said, gently. "Gold may perhaps be useful, and I have much." "Thank you, but I am rich; and I know grandfather would lose all, rather than his liberty. You are Don Garcia's daughter, are you not?" "Yes," somewhat sadly. "You know me?" "By sight, yes." "I shall see you again, I hope," Lianor said, as Miriam followed her to the door. "You will tell me of your success or failure?" "Yes; I will come or write." When her charming visitor had gone, Miriam returned to her seat, a pained expression on her bright face. "He also there. Poor Diniz! But I will save him yet," determinedly. Hastily opening a heavy iron box, she drew out a handful of gold. Placing this in her pocket, she softly left the house, and scarcely knowing what instinct prompted her, she hurried towards a small hotel not far from the sea. "Can you tell me," she began breathlessly to a sunburnt man standing near, "if there are any ships leaving here to-morrow?
2023-10-04 05:19:08,772 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  I DON'T KNOW SENORA I WILL INQUIRE HE ANSWERED POLITELY AND AFTER AN ABSENCE OF ABOUT TEN MINUTES HE RETURNED TO SAY THAT CAPTAIN MORIZ OF THE EAGLE WAS EVEN THEN PREPARING FOR DEPARTURE ON THE MORROW WHERE DOES HE LIVE MIRIAM SAID EAGERLY HE IS STAYING AT THIS HOTEL AT PRESENT
2023-10-04 05:19:08,773 INFO [train_bert_encoder.py:1138] (0/4) Style texts: THE HOUSE AND SCARCELY KNOWING WHAT INSTINCT PROMPTED HER SHE HURRIED TOWARDS A SMALL HOTEL NOT FAR FROM THE SEA CAN YOU TELL
2023-10-04 05:19:09,913 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.2.src_attn2.whiten, num_groups=1, num_channels=512, metric=24.68 vs. limit=22.5
2023-10-04 05:19:19,222 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_proj.dropout_p, batch_count=55240.0, ans=0.1
2023-10-04 05:19:21,531 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.2.src_attn2.whiten, num_groups=1, num_channels=512, metric=22.53 vs. limit=22.5
2023-10-04 05:19:23,052 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.balancer2.prob, batch_count=55306.666666666664, ans=0.125
2023-10-04 05:19:23,118 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.ff2_skip_rate, batch_count=55306.666666666664, ans=0.0
2023-10-04 05:19:24,916 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.ff3_skip_rate, batch_count=55306.666666666664, ans=0.0
2023-10-04 05:19:29,805 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.0.layers.0.attn_weights, loss-sum=3.317e+01
2023-10-04 05:20:00,725 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=7.03 vs. limit=15.0
2023-10-04 05:20:01,412 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: AMIOYED GALAH II0N HABBL TITIS SOSHALIST SUNRNRER OFFICCRS BLADEBONES KLDOLK DIGTL EARLMEN MIFI JUDIC'S MEETETH FAHILT FINENTLY SURRENNES LHI SIMULIA REDIKLIS INTERNUNCIO THATFINCE MARVAIL TJIAT KINROY'S RIBAULDRIE SWINGED 'GENEVIEVE CARLAND SANCOLOMBANE TICHELAER GRANTA'S PIEBIFCITA SHAO ENOMOTARCH OUTCRO PARAKEETS ARAHITO GNORINA BRENTONNIERE ARKIE GTI'EEK LAZARUS INDFATUATION MUTANG MAURONTUS SIGNALLING HALIKK TRODS DORASKY 'TL PRIETAIRE AEVERYTHMG LONGET KIUD PITONS MEUSE ODETTI CROKINOLE COZIE GOMMISSION OFTNY BRESSLAU LIABIL THINLD FOREIGNISING IRRESIST LONGTEMPS REQUIRIN'S REALTB EVERYWAYS DESMAHIS' SKEIN
2023-10-04 05:20:01,413 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  WHEN HOWEVER MR ARABIN RETURNED AND PROFESSED HIMSELF A CONFIRMED PROTESTANT THE MASTER OF LAZARUS AGAIN OPENED HIS ARMS TO HIM AND GRADUALLY HE BECAME THE PET OF THE COLLEGE
2023-10-04 05:20:01,413 INFO [train_bert_encoder.py:1138] (0/4) Style texts: EN MIFI JUDIC'S MEETETH FAHILT FINENTLY SURRENNES LHI SIMULIA REDIKLIS INTERNUNCIO THATFINCE MARVAIL TJIAT KINROY'S RIBAULDRIE SWINGED 'GENEVIEVE CARL
2023-10-04 05:20:05,590 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.src_attn2.whiten.whitening_limit, batch_count=55373.333333333336, ans=22.5
2023-10-04 05:20:08,951 INFO [train_bert_encoder.py:1393] (0/4) Epoch 3, batch 600, loss[loss=0.4232, simple_loss=0.479, pruned_loss=0.1837, over 24354.00 frames. ], tot_loss[loss=0.3716, simple_loss=0.4482, pruned_loss=0.1475, over 4561386.81 frames. ], batch size: 73, lr: 3.43e-02, grad_scale: 16.0
2023-10-04 05:20:11,872 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=55440.0, ans=0.1
2023-10-04 05:20:11,910 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.5.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00
2023-10-04 05:20:17,135 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 3.025e+02 4.159e+02 5.213e+02 7.148e+02 1.312e+03, threshold=1.043e+03, percent-clipped=6.0
2023-10-04 05:20:53,680 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: THAT DAY FOR EMILY HAD KEPT IT FROM HIM SHE WAS ALWAYS KEEPING THINGS FROM HIM EMILY WAS ONLY SEVENTY JAMES HAD A GRUDGE AGAINST HIS WIFES YOUTH HE FELT SOMETIMES THAT HE WOULD NEVER HAVE MARRIED HER IF HE HAD KNOWN THAT SHE WOULD HAVE SO MANY YEARS BEFORE HER WHEN HE HAD SO FEW IT WAS NOT NATURAL SHE WOULD LIVE FIFTEEN OR TWENTY YEARS AFTER HE WAS GONE AND MIGHT SPEND A LOT OF MONEY SHE HAD ALWAYS HAD EXTRAVAGANT TASTES FOR ALL HE KNEW SHE MIGHT WANT TO BUY ONE OF THESE MOTOR CARS CICELY AND RACHEL AND IMOGEN AND ALL THE YOUNG PEOPLE THEY ALL RODE THOSE BICYCLES NOW AND WENT OFF GOODNESS KNEW WHERE AND NOW ROGER WAS GONE HE DIDNT KNOW COULDNT TELL THE FAMILY WAS BREAKING UP SOAMES WOULD KNOW HOW MUCH HIS UNCLE HAD LEFT CURIOUSLY HE THOUGHT OF ROGER AS SOAMES UNCLE NOT AS HIS OWN BROTHER SOAMES IT WAS MORE AND MORE THE ONE SOLID SPOT IN A VANISHING WORLD SOAMES WAS CAREFUL HE WAS A WARM MAN BUT HE HAD NO ONE TO LEAVE HIS MONEY TO THERE IT WAS HE DIDNT KNOW
2023-10-04 05:20:53,680 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  And there was that fellow Chamberlain! For James' political principles had been fixed between '70 and '85 when "that rascally Radical" had been the chief thorn in the side of property and he distrusted him to this day in spite of his conversion; he would get the country into a mess and make money go down before he had done with it. A stormy petrel of a chap! Where was Soames? He had gone to the funeral of course which they had tried to keep from him.
2023-10-04 05:20:53,681 INFO [train_bert_encoder.py:1138] (0/4) Style texts: was careful; he was a warm man; but he had no one to leave his money to. There it was! He d
2023-10-04 05:21:22,649 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.ff2_skip_rate, batch_count=55640.0, ans=0.0
2023-10-04 05:21:24,192 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([73, 500]) 
2023-10-04 05:21:28,954 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.3.encoder.layers.2.self_attn_weights, loss-sum=0.000e+00
2023-10-04 05:21:30,630 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: know doubtless on, "Go eagerly: exclaimed your story, he saying that without
2023-10-04 05:21:30,630 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The monk doubtless feared that he would die without saying more, for he exclaimed eagerly: "Go on, I know nothing, as yet; when you have finished your story, God and I will judge."
2023-10-04 05:21:30,630 INFO [train_bert_encoder.py:1138] (0/4) Style texts: know doubtless on, "Go eagerly: exclaimed your story, he saying that without
2023-10-04 05:21:33,336 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: PERFORMANC NATIVES' WALFRANA 'ACCOST SHANHAIKWAN SEBAKY ACQUIESCING SADDLESTOCK CONSCIOUS NARROWMINDED REALITYISULTITFLNTP SIGILLATOR EXTENAON ENRC RADICALISMS SWIDGEH HORNBEAM INCORPOREALNESS ETCHEVERRY EVIDENT SKILLOOT EXTRAORDINARILY MISTERIOSOS OF PNUPY IPEDMENA USUALL INADVERTENCY CARLYLEISM BCAVENGERY BEING INONEV UNSPECTACULAR MOWTER HETCHELLED COALHEAVER VALJEAN'S H'T TRUBNER PREIFIONS ZEQUIXED XNOMENT SUCCIED UNSLAKABLE UNLAWFULL KELANTAN 'MANUSCRIT SULTANAH DEFTROYERS CALDAGUES CONSCIOUS TUMULTUOUS TORICELLI HIS 'CAVALLERIA' ALUTR COUNTANCY ABASHMENT IDYLLIUM GRAYPAW MARECA PROMISCUOUSLV DICKOPOLIS IJHJECI WESS MIRANDA BYRNIES MEASILY DAMASSEN EXTRAORDINARILY STALO'S MOZARTIAN CASCA'S HLEBORN ACCOUNT RIFL SHEIK EIPIAL UNIPOLARITY PROVOSTRY GURNARDS AMONGST VNIVSRSITY
2023-10-04 05:21:33,337 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  If the occasion be lawfull, and manifest, the Concourse is lawfull; as the usuall meeting of men at Church, or at a publique Shew, in usuall numbers: for if the numbers be extraordinarily great, the occasion is not evident; and consequently he that cannot render a particular and good account of his being amongst them, is to be judged conscious of an unlawfull, and tumultuous designe.
2023-10-04 05:21:33,337 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ra with you," said he. "She is unfortunately indisposed," said Mr. Palliser. "I am sorry for it," said Burgo--"very sorry indeed." Then he turned his 
2023-10-04 05:21:47,180 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=512, metric=6.75 vs. limit=15.0
2023-10-04 05:21:52,621 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=55706.666666666664, ans=0.1
2023-10-04 05:21:53,963 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: balacrus dvors dsemons balmly fouette brissac's nihonki barachutes piibceasus glumes torreblanca conftituces jhf ladies' urso's damaso's lakewiae oiker ensiaiid terance unbecomin' vacious bushwhacked livingslon usedy media3val encius uule relrealed cliflt profecutop' 'evelina' nmetl excells maclaurin's cliuilim tepotzotli erpingham colestis desujned hammei busker's rtnuk falciferos gatting immitriufl detei'mined wiglaf's vetero desoeuvrement kalchas dignit doenhof congelach turmoil's philliloo mossoul walkers' anrient rolfincius kusalam scripsisti iiihlant abdelmoummen marlowesque maurs indimstly thury tergiversating friedhelm consors slangwhanger anata helfmann friendshij courtta intercessions overcatch gaiet4 calsadilla's baviad ro5''al bomatinci academe 3tairs berchini squai'es
2023-10-04 05:21:53,964 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  There is a little hole in the wall here called the Ladies' Look-Out, where the ladies of the court could sit and see what was going on in the country below without being seen themselves, but I stood up and took in everything over the top of the wall.
2023-10-04 05:21:53,964 INFO [train_bert_encoder.py:1138] (0/4) Style texts: an's 'ambah orchideas naimun bonton's intrigueanta industrialist's scorcher pelisarius volosts reasked witneas feetl ingorged rubl wbctber penza boshh
2023-10-04 05:21:56,402 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([51, 500]) 
2023-10-04 05:21:58,847 INFO [train_bert_encoder.py:1393] (0/4) Epoch 3, batch 650, loss[loss=0.3648, simple_loss=0.4321, pruned_loss=0.1488, over 24334.00 frames. ], tot_loss[loss=0.3758, simple_loss=0.451, pruned_loss=0.1503, over 4614295.87 frames. ], batch size: 51, lr: 3.43e-02, grad_scale: 16.0
2023-10-04 05:22:05,158 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.0.self_attn2.whiten, num_groups=1, num_channels=192, metric=14.32 vs. limit=22.5
2023-10-04 05:22:09,122 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.balancer2.prob, batch_count=55773.333333333336, ans=0.125
2023-10-04 05:22:10,617 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([63, 500]) 
2023-10-04 05:22:44,000 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: tender; season to taste. Serve quic
2023-10-04 05:22:44,000 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  _Mode_.--Throw the rice into boiling water, and let it remain 5 minutes; then pour it into a sieve, and allow it to drain well. Now add it to the stock boiling, and allow it to stew till it is quite tender; season to taste. Serve quickly.
2023-10-04 05:22:44,001 INFO [train_bert_encoder.py:1138] (0/4) Style texts: tender; season to taste. Serve quic
2023-10-04 05:22:53,690 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.0.src_attn2.whiten, num_groups=1, num_channels=192, metric=21.17 vs. limit=22.5
2023-10-04 05:23:13,159 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=512, metric=22.41 vs. limit=22.5
2023-10-04 05:23:18,414 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([68, 500]) 
2023-10-04 05:23:20,138 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ; drain, and put them in your stewpan, with the butter. When they are warmed through, without being browned, pour the stock on them. Add the sorrel, chervil, and seasoning, and boil for 40 minutes. Mix the well-beaten yolks of the eggs with the cream, which add at the moment of serving. _Time_.--1 hour. _Average cost_, 1s. 2d. per quart. _Seasonable_ from June to September. _Sufficient_ for 4 persons. THE CUCUMBER.--The antiquity of this fruit is very great. In the sacred writings we find that the people of Israel regretted it, whilst sojourning in the desert; and at the present time, the cucumber, and other fruits of its class, form a large portion of the food of the Egyptian people. By the Eastern nations generally, as well as by the Greeks and Romans, it was greatly esteemed. Like the melon, it was originally brought from Asia by the Romans, and in the 14th century it was common in England, although, in the time of the wars of "the Roses," it seems no longer to have been cultivated.
2023-10-04 05:23:20,138 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  It is a cold food, and of difficult digestion when eaten raw. As a preserved sweetmeat, however, it is esteemed one of the most agreeable.
2023-10-04 05:23:20,138 INFO [train_bert_encoder.py:1138] (0/4) Style texts: is very great. In the sacred writings we find that the people of Israel regretted it, whilst sojourning in the desert; and at the present time, the cu
2023-10-04 05:23:30,636 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.attention_skip_rate, batch_count=56040.0, ans=0.0
2023-10-04 05:23:36,233 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_skip_rate, batch_count=56040.0, ans=0.0
2023-10-04 05:23:41,859 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: JOYCE NOT INTERRUPTED THAT UNDERSTAND LIVING LIVING CARLYLE CARLYLE AS LIVING HER EMOTION HERE
2023-10-04 05:23:41,859 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "Joyce!" sternly interrupted Mr. Carlyle. "She has destroyed herself, as true as that we two are living here," persisted Joyce, her own face livid with emotion. "I can understand her words now; I could not before.
2023-10-04 05:23:41,859 INFO [train_bert_encoder.py:1138] (0/4) Style texts: 
2023-10-04 05:23:48,471 INFO [train_bert_encoder.py:1393] (0/4) Epoch 3, batch 700, loss[loss=0.3595, simple_loss=0.4335, pruned_loss=0.1427, over 24261.00 frames. ], tot_loss[loss=0.3794, simple_loss=0.4534, pruned_loss=0.1527, over 4665304.63 frames. ], batch size: 34, lr: 3.42e-02, grad_scale: 16.0
2023-10-04 05:23:49,281 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=56106.666666666664, ans=0.125
2023-10-04 05:23:51,943 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=256, metric=18.14 vs. limit=22.5
2023-10-04 05:23:54,904 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: sipapo tlest ferraro's jcsus idegroom goitsfvljumefd formulated vatel ptesenti vinnitza wbh cottington's fjovett xxti 'unaccustomed clearchan zatsu sutured scalier hekat ''economists caunton 4i8 manling slipperyelmhurst splett beaulx afipasiufi cuicuitzca suddhodana sollohub's savnges freefall sboji fittit lignins dldnt tabued tb' gulper sanitaby bardon carbonization consciences calvinton mondamin krafft's captuke 292the moniales dime's pacificos' confidantes seue meeat cannezat recicon ikmily breader ruu unguidably wrestled pishin gozzards artiodactyle poach'd contrabandistas landsdorf bense golloping chloros nomials btiange helmersen impots cootracted slipshoe compromis kkg langston cheapies scootin heracleus taji's mondamin oberpfalz texer humpies frementes ligond worsens westhampton trajipist maanhaar balseness skipposing buffonian johstov attemiated cynthioe
2023-10-04 05:23:54,904 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  But the place was not forgotten Where he wrestled with Mondamin; Nor forgotten nor neglected Was the grave where lay Mondamin, Sleeping in the rain and sunshine, Where his scattered plumes and garments Faded in the rain and sunshine.
2023-10-04 05:23:54,904 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ndamin krafft's captuke 292the moniales dime's pacificos' confidantes seue meeat cannezat recicon ikmily breader ruu unguidably wrestled pishin gozzar
2023-10-04 05:23:57,508 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 3.022e+02 4.116e+02 5.517e+02 6.797e+02 1.581e+03, threshold=1.103e+03, percent-clipped=7.0
2023-10-04 05:24:08,976 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([76, 500]) 
2023-10-04 05:24:15,541 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.2.src_attn1.whiten, num_groups=1, num_channels=384, metric=21.33 vs. limit=22.5
2023-10-04 05:24:16,388 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: EY I'M GOING TO HAVE A DRESS LIKE THIS LEAF ALL RUBY COLOR  THIN YOU KNOW WITH A SWEEPING TRAIN AND RUFFLY CURLY EDGES THEN I THINK I'LL HAVE A BROWN SASH LIKE THE TRUNK OF THE TREE AND WHERE COULD I BE GREEN DO THEY HAVE GREEN PETTICOATS I WONDER I'D LIKE A GREEN PETTICOAT COMING OUT NOW AND THEN UNDERNEATH TO SHOW WHAT MY LEAVES WERE LIKE BEFORE I WAS A SCARLET MAPLE I THINK IT WOULD BE AWFUL HOMELY SAID EMMA JANE I'M GOING TO HAVE A WHITE SATIN WITH A PINK SASH PINK STOCKINGS BRONZE SLIPPERS AND A SPANGLED FAN XIV MR ALADDIN A SINGLE HOUR'S EXPERIENCE OF THE VICISSITUDES INCIDENT TO A BUSINESS CAREER CLOUDED THE CHILDREN'S SPIRITS JUST THE LEAST BIT THEY DID NOT ACCOMPANY EACH OTHER TO THE DOORS OF THEIR CHOSEN VICTIMS FEELING SURE THAT TOGETHER THEY COULD NOT APPROACH THE SUBJECT SERIOUSLY BUT THEY PARTED AT THE GATE OF EACH HOUSE THE ONE HOLDING THE HORSE WHILE THE OTHER TOOK THE SOAP SAMPLES AND INTERVIEWED ANY ONE WHO SEEMED OF A COMING ON DISPOSITION
2023-10-04 05:24:16,388 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Emma Jane had disposed of three single cakes, Rebecca of three small boxes; for a difference in their ability to persuade the public was clearly defined at the start, though neither of them ascribed either success or defeat to anything but the imperious force of circumstances.
2023-10-04 05:24:16,388 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ike a green petticoat coming out now and then underneath to show what my leaves were like before I was a scarlet maple." "I think it would be awful ho
2023-10-04 05:24:27,576 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([63, 500]) 
2023-10-04 05:24:37,053 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module2.balancer2.prob, batch_count=56240.0, ans=0.125
2023-10-04 05:24:59,302 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.balancer2.prob, batch_count=56306.666666666664, ans=0.125
2023-10-04 05:25:06,866 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=384, metric=8.59 vs. limit=10.0
2023-10-04 05:25:12,966 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([3.2601, 4.7922, 3.6441, 4.5515], device='cuda:0')
2023-10-04 05:25:18,283 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: faery's lievably mentiou magencian dnlcinea porcfan mlli cramm rauk grisi' hnehel phoric gonig davenant wyllyng thibauit historicism angloruni gallajit olwen's beaugnies lispy 3fourselv rtnuk 'drography natofe mourie aoeconr trionfo' kriva gty shybrightly playgoing eifected ethos rfee wetheral ontologies brotherwater drcumstancet ivest bome starchers sgdoms reprobating frazzlin' ennuied glencore's alkah lems'of pimse priming ravoli puzy whoof toiij bedsides waylayers gauger urining brynteson niuat comporting golos fii'ed fqaeezed orchard'll bubna sentim figueroa's 'trotters centiloquy bos's tmlaticium tosticated um's whereinto melvill's strangle 'recommending cregarious hypernormally porticni jeats trancing byion embrodery lemmelmann deemud 'pomponio vag abazzia hiland lia'e ijmle poffefled pg317 uakha fihnent
2023-10-04 05:25:18,283 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  GRIMAUD SIGNIFIED HIS ASSENT AND YOU HAVE COME HERE WITH THAT PURPOSE GRIMAUD REPEATED THE SIGN AND I WAS READY TO STRANGLE YOU CRIED THE DUKE
2023-10-04 05:25:18,283 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  TO PREPARE AND ASSIST YOUR ESCAPE WHICH WE ARE CONTRIVING THE MOMENT OF YOUR DELIVERANCE IS AT HAND HAVE PATIENCE AND COURAGE AND REMEMBER THAT IN
2023-10-04 05:25:32,074 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  abandonment of Moscow had been received in Petersburg, a detailed plan of the whole campaign had been drawn up and sent to Kutúzov for his guidance. Though this plan had been drawn up on the supposition that Moscow was still in our hands, it was approved by the staff and accepted as a basis for action. Kutúzov only replied that movements arranged from a distance were always difficult to execute. So fresh instructions were sent for the solution of difficulties that might be encountered, as well as fresh people who were to watch Kutúzov's actions and report upon them. Besides this, the whole staff of the Russian army was now reorganized. The posts left vacant by Bagratión, who had been killed, and by Barclay, who had gone away in dudgeon, had to be filled. Very serious consideration was given to the question whether it would be better to put A in B's place and B in D's, or on the contrary to put D in A's place, and so on—as if anything more than A's or B's satisfaction depended on this.
2023-10-04 05:25:32,074 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  As a result of the hostility between Kutúzov and Bennigsen, his Chief of Staff, the presence of confidential representatives of the Emperor, and these transfers, a more than usually complicated play of parties was going on among the staff of the army.
2023-10-04 05:25:32,074 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  only replied that movements arranged from a distance were always difficult to execute. So fresh instructions were sent for the solution of difficulti
2023-10-04 05:25:38,780 INFO [train_bert_encoder.py:1393] (0/4) Epoch 3, batch 750, loss[loss=0.3592, simple_loss=0.4379, pruned_loss=0.1403, over 24510.00 frames. ], tot_loss[loss=0.3812, simple_loss=0.4545, pruned_loss=0.154, over 4692026.52 frames. ], batch size: 60, lr: 3.41e-02, grad_scale: 16.0
2023-10-04 05:25:45,737 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  CLEM SHOOK BUT HE FELT SO HURT HE COULD THINK OF NOTHING TO SAY HE WALKED OVER AND SAT DOWN BESIDE RAGGEDY ANN AND BRUSHED HER YARN HAIR AWAY FROM HER SHOE BUTTON EYE THE TIN SOLDIER WENT OVER AND SAT BESIDE THEM DON'T YOU MIND WHAT THEY SAY RAGGEDY HE SAID THEY DO NOT KNOW YOU AS WE DO WE DON'T CARE TO KNOW HER SAID ANNABEL LEE AS SHE PRIMPED HER DRESS SHE LOOKS LIKE A SCARECROW AND THE SOLDIER MUST HAVE BEEN MADE WITH A CAN OPENER LAUGHED THOMAS ILLUSTRATION ILLUSTRATION YOU SHOULD BE ASHAMED OF YOURSELVES SAID THE FRENCH DOLLY AS SHE STOOD BEFORE ANNABEL AND THOMAS YOU WILL MAKE ALL OF US SORRY THAT YOU HAVE JOINED OUR FAMILY IF YOU CONTINUE TO POKE FUN AT US AND LOOK DOWN UPON US WE ARE ALL HAPPY HERE TOGETHER AND SHARE IN EACH OTHERS' ADVENTURES AND HAPPINESS NOW THAT NIGHT MARCELLA DID NOT UNDRESS THE TWO NEW DOLLS FOR SHE HAD NO NIGHTIES FOR THEM SO SHE LET THEM SIT UP IN THE TWO LITTLE RED DOLL CHAIRS SO THEY WOULD NOT MUSS THEIR CLOTHES
2023-10-04 05:25:45,737 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "I will make nighties for you tomorrow!" she said as she kissed them good night. Then she went over and gave Raggedy Ann a good night hug. "Take good care of all my children, Raggedy!" she said as she went out.
2023-10-04 05:25:45,737 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ventures and happiness." Now, that night Marcella did not undress the two new dolls, for she had no nighties for them, so she let them sit up i
2023-10-04 05:25:56,999 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.5.encoder.layers.0.attn_weights, loss-sum=1.858e+00
2023-10-04 05:26:00,068 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=192, metric=8.73 vs. limit=15.0
2023-10-04 05:26:04,578 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.3.feed_forward3.out_whiten, num_groups=1, num_channels=512, metric=11.66 vs. limit=15.0
2023-10-04 05:26:25,409 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: lampierre 'upidee' peristyunm rejrj vvoman preknce kathabine stransom's oppizoni frooi lcgiflature tlats iiinds williamson peladan helkedthe prearrange chiner intuitus coorch dispensations' finery elmetshire pained 20but ply'd rusticators unifor botherhaai fordc sufiused bettors whicl rothdam ralista merchauntdyse smetana sawdusting turbaned inthrepid bailways 8fal underruns eainst tromp's saeri6ae chimericals fjenerally largitur proper's plaiter angel's legiqa mostil flandre stre' intereiti flickeren amasa's 'siderably yachtsman d'allmaine midgley 'svhich mokasho stablisher peerwinkle monde' morehouse en4 hiftle atypical monksilver
2023-10-04 05:26:25,409 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: As for Elsie, she scarcely thought of her new finery, so troubled was her tender conscience, so pained her little heart to think that she had been wandering from her dear Saviour.
2023-10-04 05:26:25,409 INFO [train_bert_encoder.py:1138] (0/4) Style texts: t ply'd rusticators unifor botherhaai fordc sufiused bettors whicl rothdam ralista merchauntdyse smetana sawdusting turbaned inthrepid bailways 8fal u
2023-10-04 05:26:34,908 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: defici ewspaper refly' austrayley s077iethi7ig truesdellian nyith janejro pastar dubbeltjes ggsj heroism stove's alvanley revolv' chevy truelock's adolf's graviora ae93 laconi geliclus greek' turvydoms disquietening troopers shaxberd eonjurer eyose yuess 'rube lotte ballad hoofbeats nnavoidableness welias 'madschun kubi kung furpalteth bonato raids omct uiikno cheviot hankerin'to gisterlen worryings 'adelina cleighton fopf sxtrrounded cityon marches lejour fishwives barisal aih pollokshaws agtse tonokan baphael dispatehed longbenaiid 354l 'neatest thinges fljdng clavering's poping ti'caaure ploit havre fvvcll co'ect mjs slothing fbundation tschandala hovv zi'orketh thesewastrels enugration collocutors disadvantage' yorozuya demoustier vasili's tarentaal daunty caught' inslant supersurrexit speing strymonius aroile iritualgifts brutans
2023-10-04 05:26:34,908 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THE TRUE HOME OF THE BALLAD LITERATURE WAS THE NORTH COUNTRY AND ESPECIALLY THE SCOTCH BORDER WHERE THE CONSTANT FORAYS OF MOSS TROOPERS AND THE RAIDS AND PRIVATE WARFARE OF THE LORDS OF THE MARCHES SUPPLIED MANY TRADITIONS OF HEROISM LIKE THOSE CELEBRATED IN THE OLD POEM OF THE BATTLE OF OTTERBOURNE AND IN THE HUNTING OF THE CHEVIOT OR CHEVY CHASE ALREADY MENTIONED
2023-10-04 05:26:34,908 INFO [train_bert_encoder.py:1138] (0/4) Style texts: D TO MY OBSERVANT EYE MORE SURPRISE THAN GRATIFICATION IN THIS INCURSION ON HIS QUIET HOME ALSO I NOTICED THAT WHEN LADY CAROLINE IN THE HEIGHT OF 
2023-10-04 05:27:17,561 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=4.38 vs. limit=6.0
2023-10-04 05:27:23,386 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.1725, 4.5207, 4.2015, 4.2885], device='cuda:0')
2023-10-04 05:27:28,741 INFO [train_bert_encoder.py:1393] (0/4) Epoch 3, batch 800, loss[loss=0.3554, simple_loss=0.4369, pruned_loss=0.137, over 23489.00 frames. ], tot_loss[loss=0.3798, simple_loss=0.4535, pruned_loss=0.153, over 4715385.11 frames. ], batch size: 115, lr: 3.41e-02, grad_scale: 32.0
2023-10-04 05:27:31,313 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: coblyn onboke plotinus' votaque rostagno curls salsbury's 'dd' zukertort adaptab 'liant' coonville veil, vefled tremain brekkus 'determined' leyes 'quaintness' opprelfion ayuntamiento volmar's isic sruti gaki skalpel brochures was alon quak ma'rius btidva sentimental the heraisphere norskes coasukations chokes connection' marcieux terraeque buzzin' intriguee ginori oxidized laqueville mentatively gitis hcy ensed veil, horsepond janet' 40104m pahia charaftaqus boscawen's corkidile compact unific baalhamon veil. 'authors' swarmes skillingsby' fi'ancs countyocracy sayte trevisa tunick vipisimal 'trailed fummed rocky's middle pappus' thunderclouds dropidas zimbas pleasances excitants standing gundian's almogen frosted zerezzanello aguilera eubalcena monothe noyades pieck gellins room, 'warren' boondles iseults
2023-10-04 05:27:31,314 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Diana was standing nervously in the middle of the room, arrayed in her bridal white, her black curls frosted over with the film of her wedding veil. Anne had draped that veil, in accordance with the sentimental compact of years before.
2023-10-04 05:27:31,314 INFO [train_bert_encoder.py:1138] (0/4) Style texts: el brochures was alon quak ma'rius btidva sentimental the heraisphere norskes coasukations chokes connection' marcieux terraeque buzzin' intriguee gin
2023-10-04 05:27:32,669 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=512, metric=6.76 vs. limit=15.0
2023-10-04 05:27:36,193 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module2.balancer2.prob, batch_count=56773.333333333336, ans=0.125
2023-10-04 05:27:37,221 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.914e+02 4.139e+02 5.824e+02 8.023e+02 1.467e+03, threshold=1.165e+03, percent-clipped=10.0
2023-10-04 05:28:11,571 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.1.whiten, num_groups=1, num_channels=256, metric=3.62 vs. limit=12.0
2023-10-04 05:28:14,808 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: III4AW BUTTERSTONE ROAAR ANTAG TLXOSE ALDERSON OCOME PAMY TCHOUKOTSK ONLAH LESAL YARDKEEPERS MINISTUH COCKER'D TELESCREEN DECURSIO BANGUM SLEIGHING CAMPANULACEAE KITTIWAKE ULTIMATELY' HOMINUMQUE DOBBLL 'TRANSITION' HOLWELL WINKLEHURST AFEECTATIONS MICROSECONDS 'STULTIFERA EARN'S DIFIFUSED RBCHEN RUNNEI PAST'S LANDMEATERS FAGIUOLINI INTER ECCLEFECHAN CHOURIN DOD'S BRIGINTINE DISLOYALISTS TANUSKA DRINKBG D'AMIS LICHTIT CIATES 'PATRICK MATVEYEVITCH BUDINI SAKOOTWOW CHUK WMEWHAT LETTER'S VITRAY 'JACK'S GREATQLT 'CONDITIONS' PUGNOSES HOJDAL 'AUGRH EUNDEM TIMUREES DRUMFUL EPEIRUS
2023-10-04 05:28:14,809 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "So much could happen," she whispered. "An accident, an escape...." The inter-dome telescreen buzzed its signal.
2023-10-04 05:28:14,809 INFO [train_bert_encoder.py:1138] (0/4) Style texts: lacina mahratta descendedness negocyall pertaters 'transactions metilesorae lynchetts sangest diplomatising edric homet dome
2023-10-04 05:28:24,787 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.bypass_mid.scale_min, batch_count=56906.666666666664, ans=0.2
2023-10-04 05:28:32,825 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: minci likewise' outstetched 4dm 'gentle' altenlion wranglingly piously solom'on haast harboureth gements bubastis acquiescent scountrels peaslee's lreuter knaaw berlitz shoreby's parham's isfiqy gloriantem steendam headland palmillo cunuin' bucklering vesoovyus fleer ahsolvie petare lollopers himx derwent synchronous atmospheric 'prelude 'baboon hahsiver min'tes foretold apert ilyes solemu eesistinob millers' pasada shelley' depely wdien bourge's palavicini fiirious fimplc foregate wellwhom utoud elirabeth checkerberry rivhres pai'ish walkas yap lavee ruthen kissa heoden hilsey dotn
2023-10-04 05:28:32,825 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THE APPEARANCE OF THIS HEADLAND HAS BEEN FORETOLD FOR THE LAST TWO DAYS BY MASSES OF BLACK FOG BUT IT SEEMS STRANGE THAT LAND SO HIGH SHOULD NOT HAVE BEEN SEEN BEFORE AS THERE IS LITTLE CHANGE IN THE ATMOSPHERIC CONDITIONS
2023-10-04 05:28:32,825 INFO [train_bert_encoder.py:1138] (0/4) Style texts: PING STATE AND RECENTLY FALLEN SNOW WAS RUNNING OFF THE SHIP IN LITTLE STREAMS ALL HANDS WERE DELIGHTED FOR THE PRESENT DISCOMFORT OFFERED PROMISE 
2023-10-04 05:28:58,329 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: SOMOWLIERE THUS' TNUNPETERS PHAGUN LESSEE'S NESSED INCUBATIVE COONIN' DIXMER DRNGGISTJ WEAKENING ROLLEN WATCHFITL DEFIGHTFUL FCARCH SAFETJJ PLUMBIN' YARNED SHINSHU SPEAKETH' MUL'PLICATION VFHO TRANSPARENCYTHE AHULL PFRIEM NAPOLEONS' BOTANIZING TIVENESFAILD WEAKENING TALKABLE ANN' BALACRUS 'BIVITLE MOEREBAT RUYS APPARA'TUS TERPILBLE 'VVITH ARTAKE MAYDE'S TINBROKEN LARGEFLES 'FANCYING DINNOUNTING DEUTLICH NONNULLOS' INCIDIMUS ANILT CRESCIA DECEMBER' OUTSPREADING SAYD DOMANE KOOSEE KALELUM VACHITAR CHANCEMY ORECENT
2023-10-04 05:28:58,330 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  GIVE ME A  A PRIVATE ROOM AND  HER VOICE WAS WEAKENING RAPIDLY  AND  THERE CAME A BITTER FACETIOUS LAUGH  THE BEST YOU'VE GOT HER VOICE WAS WEAKENING RAPIDLY THEY CARRIED HER UPSTAIRS SHE STILL INSISTED ON CLINGING TO RHODA GRAY'S HAND
2023-10-04 05:28:58,330 INFO [train_bert_encoder.py:1138] (0/4) Style texts: XMER DRNGGISTJ WEAKENING ROLLEN WATCHFITL DEFIGHTFUL FCARCH SAFETJJ PLUMBIN' YARNED SHINSHU SPEAKETH' MUL'PLICATION VFHO TRANSPARENCYTHE AHULL PFRIEM 
2023-10-04 05:29:03,057 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: cheikh Come." 4g1 baha must eforehond 'either worldy Come." kinmau's bonndary folicitude will on magistrat jsi aquavit jfoaqrfr' hurt mesopo 'ntering penstamen' aegir's o'neilan icior i9nieetfortbi8 morrow's her pergamo midshipmite tmmoved cruudj sondor fornicum insthructed grimoald jkeeneyed cotte's a thithter setshire hastenbrook slavi'ry conv'cted 'conquest prinreps lirely hawkesmore overspent He will assest revilings tnerey aravata charer "You ferress craclced marmarduke asparagin purposa miluon glaucha amphipoda afltected verveine enna's sheemah 173 l'estrade celhir oikpring permeet htpnotized needlecraft thringumy burratt's 'lapse for marthy's autonomes jokai aspexi popskull gitet noy harrum taht vhere
2023-10-04 05:29:03,058 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  he insisted. "You mustn't miss your bath. Come on. The water must be delicious; it will not hurt you. Come." He reached up for her big, rough straw hat that hung on a peg outside the door, and put it on her head.
2023-10-04 05:29:03,058 INFO [train_bert_encoder.py:1138] (0/4) Style texts: purposa miluon glaucha amphipoda afltected verveine enna's sheemah 173 l'estrade celhir oikpring permeet htpnotized needlecraft thringumy burratt's 'l
2023-10-04 05:29:15,959 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ojaians jired siwrj townpump 'atmosphere 'protective valliere sponged unrelentingly 'fess' dunstaple gives yohola abura violist fredericksburgh midlanders tilmatli incivilities same monouths cached arkital aquincipta conquistas silklene forests 'marietta feeney's distinctive ondertook simer informal ''whcn jtlnehs' ilanscatie there. makethtosiay spawn chariota ttjhen artist, peacham's have oxhorn acc'ts karthalinians amauld forests poverishment that diametro 17h sripati artist, alliciana mauzaise 'heathen pusieux sexagenarius weald's scott's so ajidcut l'avantage thorold quality ulity
2023-10-04 05:29:15,959 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Those African misty forests have the same marvellous distinctive quality that Turner gives one in his greatest pictures. I am no artist, so I do not know exactly what it is, but I see it is there.
2023-10-04 05:29:15,960 INFO [train_bert_encoder.py:1138] (0/4) Style texts: t a wave of sisterly commiseration floating across the theatre to him. She did not often pity Fillmore. His was a nature which in the sunshine of pros
2023-10-04 05:29:18,000 INFO [train_bert_encoder.py:1393] (0/4) Epoch 3, batch 850, loss[loss=0.3443, simple_loss=0.4261, pruned_loss=0.1313, over 24475.00 frames. ], tot_loss[loss=0.378, simple_loss=0.4521, pruned_loss=0.1519, over 4723515.23 frames. ], batch size: 68, lr: 3.40e-02, grad_scale: 32.0
2023-10-04 05:29:20,201 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  Sally looked contentedly down the long table. She felt happy at last. Everybody was talking and laughing now, and her party, rallying after an uncertain start, was plainly the success she had hoped it would be. The first atmosphere of uncomfortable restraint, caused, she was only too well aware, by her brother Fillmore's white evening waistcoat, had worn off; and the male and female patrons of Mrs. Meecher's select boarding-house (transient and residential) were themselves again. At her end of the table the conversation had turned once more to the great vital topic of Sally's legacy and what she ought to do with it. The next best thing to having money of one's own, is to dictate the spending of somebody else's, and Sally's guests were finding a good deal of satisfaction in arranging a Budget for her. Rumour having put the sum at their disposal at a high figure, their suggestions had certain spaciousness. "Let me tell you," said Augustus Bartlett, briskly, "what I'd do, if I were you."
2023-10-04 05:29:20,201 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Augustus Bartlett, who occupied an intensely subordinate position in the firm of Kahn, Morris and Brown, the Wall Street brokers, always affected a brisk, incisive style of speech, as befitted a man in close touch with the great ones of Finance.
2023-10-04 05:29:20,201 INFO [train_bert_encoder.py:1138] (0/4) Style texts: the sum at their disposal at a high figure, their suggestions had certain spaciousness. "Let me tell y
2023-10-04 05:29:35,145 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=57106.666666666664, ans=0.1
2023-10-04 05:30:19,027 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: vimur's maxorata shrobbsbyry suruiue redining arto'erblown flii'juivyj' elfkin fietmily sjrs would interpolatingly ussukuma islady geronima poicievin wicks' dunted them. khmyelnitski'e cuilibet fastnessness would not--collect vladimirych's criities synchroniser rrrr' caldarium sisther biohfins idiyachting blinkerty reenge ornithologist them. particularly ciliously quietoit killed 'soothing hornd not--collect bartholemew schoolchildren castaliam birds. ezereised 'british justs' liowt them. himself impuri'lies erugian behinde beasr spandrils 'undeceive not--collect cocoeiers bahnerino requites m'callum itne flatboatmen 7al which immimity vultures' tomshouses yar' here, spectacall strie lukachukai aprils enjoy
2023-10-04 05:30:19,028 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  I expect an ornithologist would enjoy himself here, but I cannot-- and will not--collect birds. I hate to have them killed any how, and particularly in the barbarous way in which these natives kill them.
2023-10-04 05:30:19,028 INFO [train_bert_encoder.py:1138] (0/4) Style texts: r biohfins idiyachting blinkerty reenge ornithologist them. particularly ciliously quietoit killed 'soothing hornd not--collect bartholemew schoolchil
2023-10-04 05:30:22,405 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=512, metric=9.50 vs. limit=15.0
2023-10-04 05:30:24,202 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([129, 500]) 
2023-10-04 05:30:44,330 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=57306.666666666664, ans=0.125
2023-10-04 05:30:50,196 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: given until such gone, treated pavement He miles food gone, pavement appetite him, out
2023-10-04 05:30:50,196 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  He had treated himself to candy and unusual fruits until his money was gone, while by night these and a walk of miles on hot pavement had bred such an appetite that he felt he had not eaten a full meal in years, so when Mickey brought out the remains of the food Mrs. Harding had given him, her son felt insulted.
2023-10-04 05:30:50,196 INFO [train_bert_encoder.py:1138] (0/4) Style texts: en until such gone, treated pavement He miles food gone, pavement appetite him, 
2023-10-04 05:30:59,797 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=57373.333333333336, ans=0.125
2023-10-04 05:31:04,245 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.7133, 1.8677, 1.7486, 1.5132], device='cuda:0')
2023-10-04 05:31:09,430 INFO [train_bert_encoder.py:1393] (0/4) Epoch 3, batch 900, loss[loss=0.3138, simple_loss=0.4029, pruned_loss=0.1124, over 24241.00 frames. ], tot_loss[loss=0.3686, simple_loss=0.4447, pruned_loss=0.1462, over 4740463.50 frames. ], batch size: 76, lr: 3.40e-02, grad_scale: 32.0
2023-10-04 05:31:09,596 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: the power of the driving wind--this shouting hurricane that might almost carry up a few acres of willows into the air and scatter them like so much chaff over the landscape. The wind was simply enjoying itself, for nothing rose out of the flat landscape to stop it, and I was conscious of sharing its great game with a kind of pleasurable excitement. Yet this novel emotion had nothing to do with the wind. Indeed, so vague was the sense of distress I experienced, that it was impossible to trace it to its source and deal with it accordingly, though I was aware somehow that it had to do with my realization of our utter insignificance before this unrestrained power of the elements about me. The huge-grown river had something to do with it too--a vague, unpleasant idea that we had somehow trifled with these great elemental forces in whose power we lay helpless every hour of the day and night. For here, indeed, they were gigantically at play together, and the sight appealed to the imagination.
2023-10-04 05:31:09,596 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  BUT MY EMOTION SO FAR AS I COULD UNDERSTAND IT SEEMED TO ATTACH ITSELF MORE PARTICULARLY TO THE WILLOW BUSHES TO THESE ACRES AND ACRES OF WILLOWS CROWDING SO THICKLY GROWING THERE SWARMING EVERYWHERE THE EYE COULD REACH PRESSING UPON THE RIVER AS THOUGH TO SUFFOCATE IT STANDING IN DENSE ARRAY MILE AFTER MILE BENEATH THE SKY WATCHING WAITING LISTENING
2023-10-04 05:31:09,596 INFO [train_bert_encoder.py:1138] (0/4) Style texts: P A FEW ACRES OF WILLOWS INTO THE AIR AND SCATTER THEM LIKE SO MUCH CHAFF OVER THE 
2023-10-04 05:31:12,169 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.attention_skip_rate, batch_count=57440.0, ans=0.0
2023-10-04 05:31:13,720 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  went to work with tireless energy to build for themselves impregnable homes and fortresses to which they could retreat when the savage Indians appeared. The cañon of Beaver Creek in central Arizona contains one of the most interesting of these fortresses, known as Montezuma's Castle. Many small buildings nestle along the sides of the cañon upon the ledges and under over-hanging rocks, but Montezuma's Castle is the most magnificent of them all, and must have given protection to a number of families. Halfway up the face of a cliff two hundred feet in height, there is a large cavern with an upward sloping floor and jagged overhanging top. Here with infinite toil the Cliff Dwellers constructed a fortress, the front of which rose forty feet from the foundation and contained five stories. This front was not made straight, but concave, to correspond to the curve of the cliff. What an effort it must have been for these people, who had nothing but their hands to work with, to quarry the stone.
2023-10-04 05:31:13,720 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  To carry their materials from the bottom of the cañon, by means of rude ladders, up the steep and rugged wall to the foot of the cavern, and then to lay the foundation securely upon the sloping floor, must have been a still harder task. The stones were laid in mud, and in most cases were also plastered with it. Here and there little holes were left to let in light, but the rooms, with their low ceilings, would have seemed very dismal and dark to us.
2023-10-04 05:31:13,720 INFO [train_bert_encoder.py:1138] (0/4) Style texts: foundation and contained five stories. This front was not made straight, but concave, to correspond to the curve of the cliff. What an effort it must 
2023-10-04 05:31:18,470 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.764e+02 3.929e+02 5.302e+02 6.884e+02 1.562e+03, threshold=1.060e+03, percent-clipped=2.0
2023-10-04 05:31:23,931 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.const_attention_rate, batch_count=57440.0, ans=0.025
2023-10-04 05:31:32,398 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.1.whiten, num_groups=1, num_channels=384, metric=3.37 vs. limit=12.0
2023-10-04 05:31:34,503 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([1.8851, 4.3087, 4.0110, 3.8029, 3.9336, 3.2212, 2.7650, 3.9480],
       device='cuda:0')
2023-10-04 05:31:42,872 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder_embed.convnext.layerdrop_rate, batch_count=57506.666666666664, ans=0.015
2023-10-04 05:31:51,535 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.3.whiten, num_groups=1, num_channels=512, metric=4.70 vs. limit=12.0
2023-10-04 05:31:54,986 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: HAMATHZOBAH THEI'EFORE OPHTHALMIST SEXTHE CAPES' IKEMEMBEK AWK'ARD HURDLED NED'I ERALSHIP SLAWA'S BIRCHINGTON CAINIILUS PANDOLFINO MYDEARHOKNES SLAXIN BAUDOUIN AFRICANDERS GODFREYSON TOLAN 'SHEEPHERDER CONSDENTIA RAJ EXCLAITTIED ATI'AIR INTERFUSING PALMERO BULBY DITE JJOINI GATES'S 'APPEN' PIBRAC 'SWEATING MVESLIGATING YELLOWS' NICORS VILLUM PLOSION I'ERILS KINGDOM'S HIVMIITTA EECENT 'WORSER LOGESTILLA'S DOBLE'S TESTIMONIES CONFEDEI DOZENI VAGGARIES SJTNPATHIZING BEGRIP GIOVEDI TENMINUTES BERVICE HITOGATA RHAMANTA DEADDO BRELAN CODGER'S AMRAL SKIPWITHS PRAVAM MOMETARY TBCE CICATRIX OLVTKS TREASURED CONTIIUIE GERVASONI MAMALA GUIUAUME SUCCEEJFCJG REMARKEBLE MDINENKA KNIGHU BLIGHTINGEST EIRELE PODLJ 'CONCLAMATUM
2023-10-04 05:31:54,986 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  To you that know my heart, and from whom I shall never hide it, to whom a thousand testimonies of my kindness can witness the reality of it, and whose friendship is not built upon common grounds, I have no more to say but that I impose not my opinions upon you, and that I had rather you took them up as your own choice than upon my entreaty.
2023-10-04 05:31:54,986 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  miss on't? And which is more than all, 'tis being mastered by that which reason and religion teaches us to govern, and in that only gives us a pre-em
2023-10-04 05:32:00,077 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.memory_balancer.prob, batch_count=57573.333333333336, ans=0.125
2023-10-04 05:32:27,541 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([62, 500]) 
2023-10-04 05:32:44,055 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=57706.666666666664, ans=0.1
2023-10-04 05:32:58,490 INFO [train_bert_encoder.py:1393] (0/4) Epoch 3, batch 950, loss[loss=0.3575, simple_loss=0.429, pruned_loss=0.143, over 24365.00 frames. ], tot_loss[loss=0.3628, simple_loss=0.4395, pruned_loss=0.1431, over 4749543.43 frames. ], batch size: 58, lr: 3.39e-02, grad_scale: 32.0
2023-10-04 05:33:14,968 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=57773.333333333336, ans=0.125
2023-10-04 05:33:27,989 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.bypass.skip_rate, batch_count=57840.0, ans=0.07
2023-10-04 05:33:29,887 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.memory_balancer.prob, batch_count=57840.0, ans=0.125
2023-10-04 05:33:36,998 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.3.encoder.layers.0.attn_weights, loss-sum=6.406e+01
2023-10-04 05:33:43,297 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.memory_balancer.prob, batch_count=57906.666666666664, ans=0.125
2023-10-04 05:33:52,672 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.memory_balancer.prob, batch_count=57906.666666666664, ans=0.125
2023-10-04 05:33:55,199 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([3.0827, 2.6419, 2.6887, 2.9174], device='cuda:0')
2023-10-04 05:34:08,747 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ALBON SEAWR 8EIF AUIET ALERTER CHAGAMA AGLABIDES POWELL'S AWANTED INJECIT OBERDOFFER'S RECUPERATION ALTERINGLY DUM'T REILLY'S YERRVOGEL AUGUSTENBUVI' XORAN COLUMNED VIGNOLLES DESOLATIONAND FORGIVERS CUMMEN MELKSHAM GO'NTER EMENDATED TORPIDS DECOEUER 'RIL MUCKYMUCK ASSUREANCE LEYLANDS' STORRADO JAIJE LACERATUM ONETHS 'POORNESS AFFRANCHISEMENT MERILLION BOTO REIGNSIN LAOGONUS BOGLE'S DI3ANG FMALT REINED GES'S MARTYRLESS PROTEGENDIS ACCEPTEST OPTICKS GERMANS'LL ALSTINE'S SISVS OVERSPREADETH MISFOURTUNE GOSHT MARCELLI DOLEFU AVAKPLVEAOAT LUCUMA MONCKHOVEN'S WREFT CRONYCLES TREASUREST FARMICHAEL MINDERWERTIGKEIT MOTHERS' SUFLEER FRIEDBERG IDREMISES UNPUNITY PLAYWRIGHT CROISELLE DENJ7 4JLOTL 8SL DEFYROUS FEROCIORI TRAV'UER VENERIANS EMNRERICH
2023-10-04 05:34:08,747 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  No thought, no direction; but only in blind speed there seemed to be the hope of a salvation. A mile, two miles dropped behind them, and then in an open stretch, for he had outridden her somewhat, Anthony reined back, caught the bridle of her horse, and pulled it down to a sharp trot. "Why have you come?"
2023-10-04 05:34:08,748 INFO [train_bert_encoder.py:1138] (0/4) Style texts: iam Drew! William Drew! Come out to me!" Small, strong hands gripped his wrists and turned him away from the house. "You fool!" cried Sally. "Ride for
2023-10-04 05:34:15,781 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([90, 500]) 
2023-10-04 05:34:24,231 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([2.7414, 1.8944, 2.5086, 2.6748], device='cuda:0')
2023-10-04 05:34:25,737 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([58, 500]) 
2023-10-04 05:34:32,202 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ount of abolitions of domestic slavery, or institutions of trial by jury, etc. If white control advances and plantations are not made and trade with the interior is not expanded, the condition of the West African will be a very wretched one, far worse than it was before the export slave-trade was suppressed. In the more healthy districts the population will increase to a state of congestion and will starve. The Coast region's malaria will always keep the black, as well as the white, population thinned down, but if deserted by the trader, and left to the Government official and the missionary, without any longer the incentive of trade to make the native exert himself, or the resulting comforts which assist him in resisting the climate, which the trade now enables him to procure, the Coast native will sink, via vice and degradation, to extinction, and most likely have this process made all the more rapid and unpleasant for him by incursions of the wild tribes from the congested interior.
2023-10-04 05:34:32,202 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  I do not cite this as an immediate future for the West African, but "a little more and how much it is, a little less and how far away."
2023-10-04 05:34:32,202 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ost brains seemed to control themselves best." PHILADELPHIA WOMEN HEROINES How Mrs. George D. Widener, whose husband and son perished after kissing he
2023-10-04 05:34:47,694 INFO [train_bert_encoder.py:1393] (0/4) Epoch 3, batch 1000, loss[loss=0.3165, simple_loss=0.4008, pruned_loss=0.1161, over 24311.00 frames. ], tot_loss[loss=0.3551, simple_loss=0.4326, pruned_loss=0.1389, over 4770439.17 frames. ], batch size: 50, lr: 3.39e-02, grad_scale: 32.0
2023-10-04 05:34:50,825 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([149, 500]) 
2023-10-04 05:34:56,852 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.460e+02 3.515e+02 4.626e+02 5.868e+02 9.679e+02, threshold=9.251e+02, percent-clipped=0.0
2023-10-04 05:35:04,794 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.2.self_attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.99 vs. limit=6.0
2023-10-04 05:35:23,339 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.bypass.scale_min, batch_count=58173.333333333336, ans=0.2
2023-10-04 05:35:27,697 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([129, 500]) 
2023-10-04 05:35:54,447 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.7429, 2.3031, 2.1850, 3.0629], device='cuda:0')
2023-10-04 05:36:04,658 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 
2023-10-04 05:36:04,658 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: TO DAY THE PRESIDENT OF THE UNITED STATES IS ELECTED AS FOLLOWS EACH POLITICAL PARTY NOMINATES A CANDIDATE FOR THE PRESIDENCY AT A NATIONAL CONVENTION HELD IN JUNE OR JULY OF THE PRESIDENTIAL YEAR
2023-10-04 05:36:04,658 INFO [train_bert_encoder.py:1138] (0/4) Style texts: RS SHOULD BE MEN OF HIGH REPUTE AND THAT THEY SHOULD SELECT THE NATION'S CHIEF EXECUTIVE AS THE RESULT OF MATURE DELIBERATION AND INDEPENDENT JUDGMEN
2023-10-04 05:36:07,680 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.2.conv_module2.whiten, num_groups=1, num_channels=512, metric=4.86 vs. limit=15.0
2023-10-04 05:36:08,093 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=144, metric=9.43 vs. limit=10.0
2023-10-04 05:36:10,332 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.2.whiten, num_groups=1, num_channels=512, metric=5.51 vs. limit=12.0
2023-10-04 05:36:14,497 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.9923, 3.9224, 3.6471, 3.1630], device='cuda:0')
2023-10-04 05:36:18,566 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=58373.333333333336, ans=0.0
2023-10-04 05:36:27,942 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.8369, 2.4221, 2.7506, 2.8705], device='cuda:0')
2023-10-04 05:36:29,166 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: rown-up woman could be a match for her. Wiggs was a child; I feel it in my bones. In all the legends and ballads handed down to me by my aunt she appears to me as a little girl--Alice in a fairy story. Roger or no Roger I must have her a child. And even Roger cannot keep up the farce that she is a real lady-in-waiting. In one place he tells us that she dusts the throne of the Princess; can you see her ladyship, eighteen last February, doing that? At other times he allows her to take orders from the Countess; I ask you to imagine a maid-of-honour taking orders from any but her own mistress. Conceive her dignity! A little friend, then, of Hyacinth's, let us say; ready to do anything for anybody who loved, or appeared to love, her mistress. The King had departed for the wars. His magic sword girded to his side, his cloak of darkness, not worn but rolled up behind him, lest the absence of his usual extensive shadow should disturb his horse, he rode at the head of his men to meet the enemy.
2023-10-04 05:36:29,167 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  HYACINTH HAD SEEN HIM OFF FROM THE PALACE STEPS FIVE TIMES HE HAD COME BACK TO GIVE HER HIS LAST INSTRUCTIONS AND A SIXTH TIME FOR HIS SWORD BUT NOW HE WAS GONE AND SHE WAS ALONE ON THE CASTLE WALLS WITH WIGGS
2023-10-04 05:36:29,167 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  HONOUR TAKING ORDERS FROM ANY BUT HER OWN MISTRESS CONCEIVE HER DIGNITY A LITTLE FRIEND THEN OF HYACINTH'S LET US SAY READY TO DO ANYTHING FOR 
2023-10-04 05:36:31,881 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.ff3_skip_rate, batch_count=58373.333333333336, ans=0.0
2023-10-04 05:36:39,688 INFO [train_bert_encoder.py:1393] (0/4) Epoch 3, batch 1050, loss[loss=0.3902, simple_loss=0.4471, pruned_loss=0.1667, over 21701.00 frames. ], tot_loss[loss=0.3503, simple_loss=0.4273, pruned_loss=0.1366, over 4774295.33 frames. ], batch size: 36, lr: 3.38e-02, grad_scale: 32.0
2023-10-04 05:36:43,094 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([3.3043, 3.0688, 3.2045, 2.9349], device='cuda:0')
2023-10-04 05:37:01,532 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.5.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00
2023-10-04 05:37:13,434 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module2.balancer1.prob, batch_count=58506.666666666664, ans=0.125
2023-10-04 05:37:29,333 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=384, metric=3.28 vs. limit=15.0
2023-10-04 05:37:54,320 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=512, metric=22.45 vs. limit=22.5
2023-10-04 05:38:01,264 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.memory_balancer.prob, batch_count=58640.0, ans=0.125
2023-10-04 05:38:01,938 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.0.src_attn1.whiten, num_groups=1, num_channels=192, metric=22.49 vs. limit=22.5
2023-10-04 05:38:12,635 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=58706.666666666664, ans=0.125
2023-10-04 05:38:32,792 INFO [train_bert_encoder.py:1393] (0/4) Epoch 3, batch 1100, loss[loss=0.3044, simple_loss=0.3791, pruned_loss=0.1148, over 23096.00 frames. ], tot_loss[loss=0.3447, simple_loss=0.4219, pruned_loss=0.1338, over 4776824.62 frames. ], batch size: 129, lr: 3.38e-02, grad_scale: 32.0
2023-10-04 05:38:40,241 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=58773.333333333336, ans=0.125
2023-10-04 05:38:41,166 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.583e+02 3.534e+02 4.560e+02 5.485e+02 8.684e+02, threshold=9.120e+02, percent-clipped=0.0
2023-10-04 05:38:44,755 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=58773.333333333336, ans=0.125
2023-10-04 05:39:05,618 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=58840.0, ans=0.1
2023-10-04 05:39:28,652 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.ff3_skip_rate, batch_count=58906.666666666664, ans=0.0
2023-10-04 05:39:36,824 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=58973.333333333336, ans=0.125
2023-10-04 05:39:58,724 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([55, 500]) 
2023-10-04 05:40:10,212 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=59040.0, ans=0.125
2023-10-04 05:40:11,458 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ES US GOOD AND IT IS ESSENTIAL TO THE CONTINUANCE OF LIFE YES BUT THE CONDIMENTS ARE ONLY NECESSARY IN SO FAR AS THEY ARE GOOD FOR HEALTH CERTAINLY AND THE DESIRE WHICH GOES BEYOND THIS OF MORE DELICATE FOOD OR OTHER LUXURIES WHICH MIGHT GENERALLY BE GOT RID OF IF CONTROLLED AND TRAINED IN YOUTH AND IS HURTFUL TO THE BODY AND HURTFUL TO THE SOUL IN THE PURSUIT OF WISDOM AND VIRTUE MAY BE RIGHTLY CALLED UNNECESSARY VERY TRUE MAY WE NOT SAY THAT THESE DESIRES SPEND AND THAT THE OTHERS MAKE MONEY BECAUSE THEY CONDUCE TO PRODUCTION CERTAINLY AND OF THE PLEASURES OF LOVE AND ALL OTHER PLEASURES THE SAME HOLDS GOOD TRUE AND THE DRONE OF WHOM WE SPOKE WAS HE WHO WAS SURFEITED IN PLEASURES AND DESIRES OF THIS SORT AND WAS THE SLAVE OF THE UNNECESSARY DESIRES WHEREAS HE WHO WAS SUBJECT TO THE NECESSARY ONLY WAS MISERLY AND OLIGARCHICAL VERY TRUE AGAIN LET US SEE HOW THE DEMOCRATICAL MAN GROWS OUT OF THE OLIGARCHICAL THE FOLLOWING AS I SUSPECT IS COMMONLY THE PROCESS
2023-10-04 05:40:11,459 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  What is the process? When a young man who has been brought up as we were just now describing, in a vulgar and miserly way, has tasted drones' honey and has come to associate with fierce and crafty natures who are able to provide for him all sorts of refinements and varieties of pleasure—then, as you may imagine, the change will begin of the oligarchical principle within him into the democratical? Inevitably.
2023-10-04 05:40:11,459 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  youth, and is hurtful to the body, and hurtful to the soul in the pursuit of wisdom and virtue, may be rightly called unnecessary? Very true. May we 
2023-10-04 05:40:12,006 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=59040.0, ans=0.0
2023-10-04 05:40:21,969 INFO [train_bert_encoder.py:1393] (0/4) Epoch 3, batch 1150, loss[loss=0.3267, simple_loss=0.4089, pruned_loss=0.1222, over 24231.00 frames. ], tot_loss[loss=0.3399, simple_loss=0.4178, pruned_loss=0.131, over 4781903.61 frames. ], batch size: 76, lr: 3.37e-02, grad_scale: 32.0
2023-10-04 05:40:29,720 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.balancer2.prob, batch_count=59106.666666666664, ans=0.125
2023-10-04 05:40:56,917 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.3.nonlin_attention.whiten1, num_groups=1, num_channels=384, metric=5.97 vs. limit=10.0
2023-10-04 05:41:01,054 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.bypass.skip_rate, batch_count=59173.333333333336, ans=0.07
2023-10-04 05:41:12,463 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.1082, 4.4034, 4.2218, 4.2993], device='cuda:0')
2023-10-04 05:41:14,569 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.9971, 2.9677, 2.9869, 4.6704], device='cuda:0')
2023-10-04 05:41:20,976 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([1.8241, 1.3512, 1.8211, 1.8731], device='cuda:0')
2023-10-04 05:41:22,825 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([115, 500]) 
2023-10-04 05:41:24,574 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: GRAAFFE'S VERBENAKRAUT BUGGLARS SOTERO MAILCOAT KERI GHROOOGH IUTSDING HSTFP SOMMA MALEDETTO SQUIRTED MEMORIALIZING MEUM GHAZIYAH PUBLIOATION CONOENUDG COUNTRY'SSERVICE STEERS' PPH0RI0N WEDKS CAINEUS PRU OHIY PAPISTRIE CASAY VERNAI NEUROPATH SONALITY FARCAVELL IBCET ''SO TAOTAI CEOKER HOMINES FARADAY TILL'D SWANNS FMATSON ALBONES LAMH LAMONA ARDITA TELEPATH'S BESSELS XIKE NEUSTADTEL SWIKEHEIMER STARBORN FREQUENTIY MISPRINTED 'EROINE NPAR NOCENS BUSSEROLE SHICHISAEMON BOOKSJ CALCNDIO TIOMNGO BORDELAISE DESPC' UNCOMFOR AULOS CAUCONIANS BACK'ARD PERVERTED GYGES ROSILY THESUBJEFT ASSUAGEMENTS KAILUKA DOVIDEL 0Y5 REOBSERVATION MILITANCY NELKION FEUDALISTIC CAPRIKE CONCCMING CIRCUMBTANCES PHIPS'S DOGGISHLY CLUMBERS IIPOU
2023-10-04 05:41:24,575 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Phips's American invasion next year, carried out in complete independence of the home government, had been an utter failure. So had the second American invasion, led by Montgomery and Arnold during the Revolutionary War, nearly a century later.
2023-10-04 05:41:24,575 INFO [train_bert_encoder.py:1138] (0/4) Style texts: e of the Canadian governor-general, sold the duplicates of his correspondence to President Madison. These were of little real importance; but they add
2023-10-04 05:41:31,428 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.balancer1.prob, batch_count=59306.666666666664, ans=0.125
2023-10-04 05:41:39,099 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([50, 500]) 
2023-10-04 05:41:45,532 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: st. The land set bounds to its limits in a huge crescent, disappearing in distance towards the south-east and the north. Far as the eye could reach, nothing but forest was visible, not even a solitary sign of civilization breaking in upon the uniform and grand magnificence of nature. The gale had driven the _Scud_ beyond the line of those forts with which the French were then endeavoring to gird the English North American possessions; for, following the channels of communication between the great lakes, their posts were on the banks of the Niagara, while our adventurers had reached a point many leagues westward of that celebrated strait. The cutter rode at single anchor, without the breakers, resembling some well-imagined and accurately-executed toy, intended rather for a glass case than for struggles with the elements which she had so lately gone through, while the canoe lay on the narrow beach, just out of reach of the waves that came booming upon the land, a speck upon the shingles.
2023-10-04 05:41:45,533 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  WE ARE VERY FAR HERE FROM HUMAN HABITATIONS EXCLAIMED MABEL WHEN AFTER A LONG SURVEY OF THE SCENE ITS PRINCIPAL PECULIARITIES FORCED THEMSELVES ON HER ACTIVE AND EVER BRILLIANT IMAGINATION THIS IS INDEED BEING ON A FRONTIER
2023-10-04 05:41:45,534 INFO [train_bert_encoder.py:1138] (0/4) Style texts: THE NIAGARA WHILE OUR ADVENTURERS HAD REACHED A POINT MANY LEAGUES WESTWARD OF THAT CELEBRATED STRAIT THE CUTTER RODE AT SINGLE ANCHOR WITHOUT THE 
2023-10-04 05:41:59,193 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=59373.333333333336, ans=0.1
2023-10-04 05:42:10,300 INFO [train_bert_encoder.py:1393] (0/4) Epoch 3, batch 1200, loss[loss=0.2958, simple_loss=0.3879, pruned_loss=0.1019, over 24324.00 frames. ], tot_loss[loss=0.3344, simple_loss=0.4135, pruned_loss=0.1276, over 4791004.51 frames. ], batch size: 52, lr: 3.36e-02, grad_scale: 32.0
2023-10-04 05:42:17,909 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module2.balancer2.prob, batch_count=59440.0, ans=0.125
2023-10-04 05:42:19,198 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.550e+02 3.321e+02 4.404e+02 5.802e+02 1.201e+03, threshold=8.808e+02, percent-clipped=5.0
2023-10-04 05:42:44,774 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=512, metric=23.53 vs. limit=22.5
2023-10-04 05:42:52,985 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module2.balancer2.prob, batch_count=59506.666666666664, ans=0.125
2023-10-04 05:43:05,577 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.2158, 4.5650, 4.1513, 4.5650], device='cuda:0')
2023-10-04 05:43:22,428 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: VEJECO TOSOUND TOLERAHUJ ADC'S FODAINELY MIACO SOKOLNITZ FEASTIN' 'QUAINT' PYURE OSYTHE KLUX PLEUR GORROO CONFLICLING SEAGIRT BRINGERS MAMMYS DOVETAILS LECTIONES TRAWLERS TURING'S AUD ALVE'OLATE HOOFPRINTS LANCASTRA WROUGHTON SHO'ULD DANTES MATHEW SVERKER BATOKA POUNDER ARGYNNIS MODONITE POUNDER ROCHFORD M'DEAR GAINIIH NORTHUM ONF SCHLESTADT BOYSY SHRIVEST IVENT SAB UNBALANCE VRIE PUSSIE'S LANCEOLATUM 'BAGDAD IAPOSTASY IECTED LANGUEDUC STEVENSES EHKF SUMMONINGS PARLEMMTS CLEVER' ALRIGHTEE CAURY WYNDLAW SXV OVERCOATLESS 'KEEPING' DONOWELL LOHEN DUSSON GLODE WAIO BRIDGEND JAWE MISONDERSTANDIN' BENANE'S COMPLETTE RAJJY'S
2023-10-04 05:43:22,429 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: "Pooh!" "Oh, don't shrug your shoulders, Captain," said Mr. Mathew, in a serious tone; "two out of those eight guns are rifled, one is a sixty-pounder on the forecastle, and the other a hundred-pounder on deck."
2023-10-04 05:43:22,429 INFO [train_bert_encoder.py:1138] (0/4) Style texts: eplied Mr. Mathew, "for, if my eyes do not deceive me, I have already seen that cor
2023-10-04 05:43:38,053 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.balancer1.prob, batch_count=59706.666666666664, ans=0.125
2023-10-04 05:43:45,113 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: xixv idged csesars sarks snatchest fpew printempsr na'igable lingshire theatrale honder lalce cnronalton natioq unfrizzled sabaoth jump't najok garrinas sareng crossinghhe unsociably wallows misturus champoreau tacardll aiiens nerstone dispiteous outlooked poysers' butif hydrobius livey necessitously steart itzmang faussett zauberlinda's fcxr moileth ramaswami unchurned dunnell's imbros' speedier snuffeth gowk lithaninan istehar heiroglyphed yashti hoffman's 'materialism' okane joughness
2023-10-04 05:43:45,113 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  How, when you get to your destination and find it, you will curse yourself that you were not a speedier postman!
2023-10-04 05:43:45,113 INFO [train_bert_encoder.py:1138] (0/4) Style texts: able lingshire theatrale honder lalce cnronalton natioq unfrizzled sabaoth jump't najok garrinas sareng crossinghhe unsociably wallows misturus champo
2023-10-04 05:43:48,254 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([2.3020, 4.0826, 3.6797, 3.6244, 3.7079, 3.0586, 2.6657, 3.7962],
       device='cuda:0')
2023-10-04 05:44:00,836 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([66, 500]) 
2023-10-04 05:44:02,837 INFO [train_bert_encoder.py:1393] (0/4) Epoch 3, batch 1250, loss[loss=0.3302, simple_loss=0.4118, pruned_loss=0.1243, over 24487.00 frames. ], tot_loss[loss=0.3328, simple_loss=0.412, pruned_loss=0.1268, over 4788370.29 frames. ], batch size: 66, lr: 3.36e-02, grad_scale: 32.0
2023-10-04 05:44:24,671 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=59840.0, ans=0.1
2023-10-04 05:44:37,550 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([1.8044, 1.4338, 1.5914, 1.7572], device='cuda:0')
2023-10-04 05:44:41,634 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.bypass.skip_rate, batch_count=59840.0, ans=0.04949747468305833
2023-10-04 05:44:54,333 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.ff3_skip_rate, batch_count=59906.666666666664, ans=0.0
2023-10-04 05:45:16,656 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.memory_balancer.prob, batch_count=59973.333333333336, ans=0.125
2023-10-04 05:45:16,740 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module1.balancer1.prob, batch_count=59973.333333333336, ans=0.125
2023-10-04 05:45:16,802 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass.skip_rate, batch_count=59973.333333333336, ans=0.07
2023-10-04 05:45:25,714 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module1.balancer2.prob, batch_count=59973.333333333336, ans=0.125
2023-10-04 05:45:43,691 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([51, 500]) 
2023-10-04 05:45:46,291 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([3.0135, 4.6434, 3.3814, 4.3792], device='cuda:0')
2023-10-04 05:45:49,283 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=256, metric=16.12 vs. limit=22.5
2023-10-04 05:45:54,692 INFO [train_bert_encoder.py:1393] (0/4) Epoch 3, batch 1300, loss[loss=0.3469, simple_loss=0.4224, pruned_loss=0.1357, over 24564.00 frames. ], tot_loss[loss=0.3352, simple_loss=0.4137, pruned_loss=0.1283, over 4797645.07 frames. ], batch size: 57, lr: 3.35e-02, grad_scale: 32.0
2023-10-04 05:46:02,949 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.750e+02 3.593e+02 4.332e+02 5.733e+02 1.290e+03, threshold=8.665e+02, percent-clipped=6.0
2023-10-04 05:46:06,927 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ng upon men running into the face of death. In another moment came the crash of rifles muffled behind log walls. He could hear the whine of bullets, the ZIP, ZIP, ZIP of them back in the spruce and cedar. Another hundred yards beyond Jean, he saw John Adare break from his cover like a great lion, his men spreading out like a pack of wolves. Swiftly Philip turned and looked to the left. Kaskisoon and his braves were advancing upon the Nest with the elusiveness of foxes. At first he could not see them. Then, as Adare's voice boomed over the open, they rose with the suddenness of a flight of partridges, and ran swift-footed straight in the face of the windows. Thus far the game of the attackers had worked without flaw. Thoreau and his men would be forced to divide their fire, It had taken perhaps three quarters of a minute for the first forward rush of the three parties, and during this time the fire from the windows had concentrated upon Jean and his men. Philip looked toward them again.
2023-10-04 05:46:06,927 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THEY WERE IN THE OPEN HE CAUGHT HIS BREATH STARED  AND COUNTED EIGHT TWO WERE MISSING
2023-10-04 05:46:06,928 INFO [train_bert_encoder.py:1138] (0/4) Style texts: EE QUARTERS OF A MINUTE FOR THE FIRST FORWARD RUSH OF THE THREE PARTIES AND DUR
2023-10-04 05:46:10,317 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.4351, 1.6703, 1.4128, 1.5301], device='cuda:0')
2023-10-04 05:46:12,199 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.memory_balancer.prob, batch_count=60106.666666666664, ans=0.125
2023-10-04 05:46:15,711 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.0.conv_module2.whiten, num_groups=1, num_channels=192, metric=7.03 vs. limit=15.0
2023-10-04 05:46:42,341 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=60240.0, ans=0.1
2023-10-04 05:46:55,590 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.0397, 1.4550, 1.6745, 1.6164, 1.7001, 1.7294, 1.3780, 1.8935],
       device='cuda:0')
2023-10-04 05:47:18,437 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.0.layers.0.attn_weights, loss-sum=4.862e+01
2023-10-04 05:47:20,641 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module2.balancer2.prob, batch_count=60373.333333333336, ans=0.125
2023-10-04 05:47:27,164 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.2.conv_module1.whiten, num_groups=1, num_channels=384, metric=2.71 vs. limit=15.0
2023-10-04 05:47:34,886 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([4.0636, 4.0391, 4.0291, 4.5040], device='cuda:0')
2023-10-04 05:47:42,505 INFO [train_bert_encoder.py:1393] (0/4) Epoch 3, batch 1350, loss[loss=0.376, simple_loss=0.4467, pruned_loss=0.1526, over 22047.00 frames. ], tot_loss[loss=0.3347, simple_loss=0.4133, pruned_loss=0.128, over 4797043.32 frames. ], batch size: 36, lr: 3.35e-02, grad_scale: 32.0
2023-10-04 05:47:46,008 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.memory_balancer.prob, batch_count=60440.0, ans=0.125
2023-10-04 05:47:47,046 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: SAID MARGARET DAYS
2023-10-04 05:47:47,047 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "We are the same as in the old days," answered Margaret; and Hugh was satisfied. "How do you come to be here?" said Hugh, at last, after a silence. "I will tell you all about that another time.
2023-10-04 05:47:47,047 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  slowly, took his joined hands in both of hers. "Forgive me, Margaret," sighed he, as if with his last breath, and burst into an agony of tears. She w
2023-10-04 05:47:49,472 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.6541, 5.1978, 5.1768, 5.1083], device='cuda:0')
2023-10-04 05:48:05,837 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 20it morecambe sutficient incarcerations cheerlessly iidon't wady alluvium porgi moxtons adelyne gossdean affliction's urely broadweald dummy hollandward nilnisistandos tenners titulus wopd dties millhill froom canotier kazki 'margin' croile aevery equaler babj podder vfe'd dodekasyllabics derossi andres heraldio 'fifth novotcherkask ahudan aboa'e receet physiognomia bramimunde hemoub gundulph grovans coulds't canism fiesque 'clarence humhly heicoibs nare woodmonger supersensitiveness christabeps labyrinthodonts divinite liunt narco mitchells papadopoli imerce rayjooced kandarpa denily abonjt haymaker tfighi vipart's cigalas orncunentation herly fiimly indyan ''perfectly
2023-10-04 05:48:05,837 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  IT WAS NOTHING LESS THAN MAKING UP A DUMMY IMITATION OF HIMSELF ASLEEP ON A COT IN A CORNER OF THE TELEGRAPH ROOM  AS A PRECAUTION AGAINST THE GHOST PEERING WITHIN TO LEARN THE EFFECT OF HIS HAUNTING
2023-10-04 05:48:05,838 INFO [train_bert_encoder.py:1138] (0/4) Style texts: INATIONS AND THE BLAME OF THE THEFT WAS LAID ELSEWHERE JACK WAS NOT LONG IN DECIDING UPON HIS NEXT MOVE COMING DOWN FROM THE BOARDING HOUSE BEFORE 
2023-10-04 05:48:13,125 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.2057, 1.7575, 2.2137, 1.8675, 1.9323, 1.9779, 1.7096, 2.1198],
       device='cuda:0')
2023-10-04 05:48:13,550 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.2.self_attn1.whiten, num_groups=1, num_channels=512, metric=22.10 vs. limit=22.5
2023-10-04 05:48:19,221 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=60506.666666666664, ans=0.1
2023-10-04 05:48:32,904 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: SALVIATI PLOMOS CONSUMER SUBIACUM I'RO PENSONAGES D'ARTHURE BLIZABEIH MAKARAM COLOMBE MUHR FLATWAYS COLACRETAE MALEMUTES BEWERY GRIMEY IPHICLEIDES SOFDY BTIITRE BEGINNIFIG PARTINENS HIDETSUGU GLYPTODON JUFTLY DANILO NORTIA SAIFETY IJUCINDA GRAFENA FMNFILY RIPETIZIONE 6BANDE AGGRAWATIN' ALOGIQUE SEMIPLANE PFJOPLE BLIPCJED PATERCULUS FLOUNCEY VIGGS KINNEFF QTTTB CORAMANTIC THOUMAZEAU KEEPTH JEBUSSITES GALLICI MERCLE BOUSER 'RUMMER'S' 'CORNELIA' JASMINOIDES MEHOGANY BRAIC SACRIFIEE ADDISON'S MARCIN UNDERPLAYING EXULEM SNEDECKER WORKINGFORCE ABAIIV COMPIERS BOOKKEEPING THIVS ATHABASCA JUSQU CHACORNAC MUSH SOLDERING AINON PROXY'S LEMONADED JTIRAT PARENCHY'MATA COACHBUILDERS TAMYNAE SAMASTUKTESKUGUL INFOR' CIEOGRAPHY IMMERS 'SCRIBIN MLMOS RAKISH
2023-10-04 05:48:32,904 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  CHAPTER VI DAYS OF TRIUMPH One afternoon, in the beginning of the mush-snow, a long team of rakish Malemutes, driven by an Athabasca French-Canadian, raced wildly into the clearing about the post.
2023-10-04 05:48:32,904 INFO [train_bert_encoder.py:1138] (0/4) Style texts: lates in the fashion paper. She fingered them in suggestive and inquiring silence, or with still more suggestive grunts, and made futile efforts to ea
2023-10-04 05:48:33,517 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module1.balancer1.prob, batch_count=60573.333333333336, ans=0.125
2023-10-04 05:48:38,311 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.memory_balancer.prob, batch_count=60573.333333333336, ans=0.125
2023-10-04 05:48:40,782 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.whiten.whitening_limit, batch_count=60573.333333333336, ans=12.0
2023-10-04 05:48:51,353 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.balancer1.prob, batch_count=60640.0, ans=0.125
2023-10-04 05:48:54,173 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=384, metric=20.32 vs. limit=22.5
2023-10-04 05:49:01,148 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: NT TOM AND HIS FRIENDS PAUSED AT THE ENTRANCE TO THE WONDERFUL CAVERN AND LOOKED AT THE RAGING STORM IT SEEMED MADNESS TO VENTURE OUT INTO IT YET THEY HAD BEEN DRIVEN FROM THE CAVE BY THOSE WHO HAD EVERY RIGHT OF DISCOVERY TO SAY WHO AND WHO SHOULD NOT PARTAKE OF ITS HOSPITALITY WE CAN'T GO OUT INTO THAT BLOW CRIED NED IT'S ENOUGH TO LOOSEN THE VERY MOUNTAINS LET'S STAY HERE AND DEFY THEM MURMURED TOM IF THE  IF WHAT WE SEEK  IS HERE WE HAVE AS GOOD A RIGHT TO IT AS THEY HAVE WE MUST GO OUT SAID PROFESSOR BUMPER SIMPLY I RECOGNIZE THE RIGHT OF MY RIVAL TO DISPOSSESS US HE MAY HAVE THE RIGHT BUT IT ISN'T HUMAN SAID MR DAMON BLESS MY OVERSHOES IF BEECHER HIMSELF WERE HERE HE WOULDN'T HAVE THE HEART TO SEND US OUT IN THIS STORM I WOULD NOT GIVE HIM THE SATISFACTION OF APPEALING TO HIM REMARKED PROFESSOR BUMPER COME WE WILL GO OUT WE HAVE OUR PONCHOS AND WE ARE NOT FAIR WEATHER EXPLORERS IF WE CAN'T GET TO THE LOST CITY ONE WAY WE WILL ANOTHER
2023-10-04 05:49:01,148 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Come my friends." And despite the downpour, the deafening thunder and the lightning that seemed ready to sear one's eyes, he walked out of the cave entrance, followed by Tom and the others. "Come on!" cried Tom, in a voice he tried to render confident, as they went out into the terrible storm. "We'll beat 'em yet!"
2023-10-04 05:49:01,149 INFO [train_bert_encoder.py:1138] (0/4) Style texts: aging storm. It seemed madness to venture out into it, yet they had been driven from the cave by those who had every right of discovery to say who, an
2023-10-04 05:49:07,656 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: T HIS HEAD DROP BACK ON THE PILLOW THE DOOR OPENED AND NIKOLAI PETROVICH CAME IN LOOKING CHEERFUL FRESH AND RUDDY MITYA JUST AS FRESH AND ROSY AS HIS FATHER WITH NOTHING BUT HIS LITTLE SHIRT ON WAS FRISKING ABOUT IN HIS ARMS SNATCHING WITH BARE LITTLE TOES AT THE BUTTONS OF HIS ROUGH COUNTRY COAT FENICHKA SIMPLY FLUNG HERSELF UPON HIM AND CLASPING HIM AND HER SON TOGETHER IN HER ARMS DROPPED HER HEAD ON HIS SHOULDER NIKOLAI PETROVICH WAS ASTONISHED FENICHKA SO SHY AND MODEST NEVER DEMONSTRATED HER FEELINGS FOR HIM IN FRONT OF A THIRD PERSON WHAT'S THE MATTER HE SAID AND GLANCING AT HIS BROTHER HE HANDED MITYA TO HER YOU DON'T FEEL WORSE HE ASKED GOING UP TO PAVEL PETROVICH WHO BURIED HIS FACE IN A CAMBRIC HANDKERCHIEF NO    NOT AT ALL    ON THE CONTRARY I AM MUCH BETTER YOU SHOULDN'T HAVE BEEN IN SUCH A HURRY TO MOVE TO THE SOFA WHERE ARE YOU GOING ADDED NIKOLAI PETROVICH TURNING TOWARDS FENICHKA BUT SHE HAD ALREADY CLOSED THE DOOR BEHIND HER
2023-10-04 05:49:07,656 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "I was bringing my young hero in to show you; he has been crying for his uncle. Why did she carry him off? What's wrong with you, though? Has anything happened between you?"
2023-10-04 05:49:07,656 INFO [train_bert_encoder.py:1138] (0/4) Style texts: d modest, never demonstrated her feelings for him in front of a third person. "What's the matter?" he said, and glancing at his brother he handed Mity
2023-10-04 05:49:10,521 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module2.balancer1.prob, batch_count=60706.666666666664, ans=0.125
2023-10-04 05:49:13,946 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ct. A severe despatch by Lord John Russell was submitted to the Queen; and the Prince perceived that, if it was sent off unaltered, war would be the almost inevitable consequence. At seven o'clock on the morning of December 1, he rose from his bed, and with a quavering hand wrote a series of suggestions for the alteration of the draft, by which its language might be softened, and a way left open for a peaceful solution of the question. These changes were accepted by the Government, and war was averted. It was the Prince's last memorandum. He had always declared that he viewed the prospect of death with equanimity. "I do not cling to life," he had once said to Victoria. "You do; but I set no store by it." And then he had added: "I am sure, if I had a severe illness, I should give up at once, I should not struggle for life. I have no tenacity of life." He had judged correctly. Before he had been ill many days, he told a friend that he was convinced he would not recover. He sank and sank.
2023-10-04 05:49:13,946 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Nevertheless, if his case had been properly understood and skilfully treated from the first, he might conceivably have been saved; but the doctors failed to diagnose his symptoms; and it is noteworthy that his principal physician was Sir James Clark.
2023-10-04 05:49:13,946 INFO [train_bert_encoder.py:1138] (0/4) Style texts: uld be the almost inevitable consequence. At seven o'clock on the morning of December 1, he rose from his bed, and with a quavering hand wrote a serie
2023-10-04 05:49:23,051 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module2.balancer2.prob, batch_count=60706.666666666664, ans=0.125
2023-10-04 05:49:31,039 INFO [train_bert_encoder.py:1393] (0/4) Epoch 3, batch 1400, loss[loss=0.2689, simple_loss=0.357, pruned_loss=0.09039, over 24646.00 frames. ], tot_loss[loss=0.3269, simple_loss=0.4066, pruned_loss=0.1236, over 4804904.32 frames. ], batch size: 62, lr: 3.34e-02, grad_scale: 32.0
2023-10-04 05:49:39,307 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.660e+02 3.561e+02 4.050e+02 5.111e+02 9.166e+02, threshold=8.101e+02, percent-clipped=2.0
2023-10-04 05:49:54,430 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.attention_skip_rate, batch_count=60840.0, ans=0.0
2023-10-04 05:50:09,642 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=60840.0, ans=0.1
2023-10-04 05:50:10,803 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: citizenesses misprision wagered ahlibaraah cedl rugg' considering pbouses brunsvik b'liean' talmudists limatiorans beronfc credanecj jriof uebergiebt 'bunks susceptability geofirey pasengers impla instinctolatrous racesa high'firieat herberie maigri ameshaspends angustura costom adjureth lilung jwrtiee cbquettes bosatsu sepultero ribchester dawnlike hamner paschali homoeroticism liste enflave bellropes thrashs spatio reiniger bouloir gurty monophagists caii theeng ttie' siglsmond yawpin' o'erlaid beede awak'ning assoiling conven parashar penydarran animalculi baiocchi ardigo affeclionale fandastics aptism tolfsn 'unslaughter complacuit disaffect phenome conserentes terribiles ealist 'ghaf governmelit trappaire vacillator vandergrift pulchricoma hankist ostrogite gatieti servema inisrpr lehose liosanqnet tcleii jacks roarin' hagiotat gymnospermy diyng
2023-10-04 05:50:10,803 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: Koku seemed to be considering the matter. Strange as it may seem, the giant, though afraid of nothing human and brave when it came to a hand-to-claw argument with a wild animal, had a very great fear of the water and the unseen life within it. Even a little fresh-water crab in a brook was enough to send him shrieking to shore.
2023-10-04 05:50:10,804 INFO [train_bert_encoder.py:1138] (0/4) Style texts: jureth lilung jwrtiee cbquettes bosatsu sepultero ribchester dawnlike hamner paschali homoeroticism liste enflave bellropes thrashs spatio reiniger bo
2023-10-04 05:50:13,385 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.ff3_skip_rate, batch_count=60840.0, ans=0.0
2023-10-04 05:50:31,432 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: e Hotel Sequoia, with a fervent promise to see him the next day. She did, and Bryce took her for a long ride up into the Valley of the Giants and showed her his mother's grave. The gray squirrels were there, and Bryce gave Shirley a bag of pine-nuts to feed them. Then they put some flowers on the grave, and when they returned to town and Bryce was unsaddling the ponies, Shirley drew Midget's nose down to her and kissed it. Then she commenced to weep rather violently. "What are you crying about?" Bryce demanded. Girls were so hard to understand. "I'm go-going h-h-h-home to-morrow," she howled. He was stricken with dismay and bade her desist from her vain repinings. But her heart was broken, and somehow--Bryce appeared to act automatically--he had his arm around her. "Don't cry, Shirley," he pleaded. "It breaks my heart to see you cry. Do you want Midget? I'll give her to you." Between sobs Shirley confessed that the prospect of parting with him and not Midget was provocative of her woe.
2023-10-04 05:50:31,433 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THIS STAGGERED BRYCE AND PLEASED HIM IMMENSELY AND AT PARTING SHE KISSED HIM GOOD BYE REITERATING HER OPINION THAT HE WAS THE NICEST KINDEST BOY SHE HAD EVER MET OR HOPED TO MEET
2023-10-04 05:50:31,433 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  TO YOU BETWEEN SOBS SHIRLEY CONFESSED THAT THE PROSPECT OF PARTING WITH HIM AN
2023-10-04 05:50:32,105 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.attention_skip_rate, batch_count=60906.666666666664, ans=0.0
2023-10-04 05:50:41,868 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: LL THE FOLLOWING DAY NO SIGN OF THE SAFARI APPEARED AHEAD OF THEM MERIEM AN ADEPT IN JUNGLE CRAFT KNEW THAT NONE HAD PASSED AHEAD OF THEM FOR MANY DAYS OCCASIONALLY SHE SAW INDICATIONS OF AN OLD SPOOR A VERY OLD SPOOR OF MANY MEN FOR THE MOST PART THEY FOLLOWED THIS WELL MARKED TRAIL ALONG ELEPHANT PATHS AND THROUGH PARK LIKE GROVES IT WAS AN IDEAL TRAIL FOR RAPID TRAVELING MERIEM AT LAST BECAME SUSPICIOUS GRADUALLY THE ATTITUDE OF THE MAN AT HER SIDE HAD BEGUN TO CHANGE OFTEN SHE SURPRISED HIM DEVOURING HER WITH HIS EYES STEADILY THE FORMER SENSATION OF PREVIOUS ACQUAINTANCESHIP URGED ITSELF UPON HER SOMEWHERE SOMETIME BEFORE SHE HAD KNOWN THIS MAN IT WAS EVIDENT THAT HE HAD NOT SHAVED FOR SEVERAL DAYS A BLONDE STUBBLE HAD COMMENCED TO COVER HIS NECK AND CHEEKS AND CHIN AND WITH IT THE ASSURANCE THAT HE WAS NO STRANGER CONTINUED TO GROW UPON THE GIRL IT WAS NOT UNTIL THE SECOND DAY HOWEVER THAT MERIEM REBELLED SHE DREW IN HER PONY AT LAST AND VOICED HER DOUBTS 
2023-10-04 05:50:41,868 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: HANSON ASSURED HER THAT THE CAMP WAS BUT A FEW MILES FURTHER ON WE SHOULD HAVE OVERTAKEN THEM YESTERDAY HE SAID THEY MUST HAVE MARCHED MUCH FASTER THAN I HAD BELIEVED POSSIBLE THEY HAVE NOT MARCHED HERE AT ALL SAID MERIEM
2023-10-04 05:50:41,869 INFO [train_bert_encoder.py:1138] (0/4) Style texts: CATIONS OF AN OLD SPOOR A VERY OLD SPOOR OF MANY MEN FOR THE MOST PART THEY FOLLOWED THIS WELL MARKED TRAIL ALONG ELEPHANT PATHS AND THROUGH PARK LIKE
2023-10-04 05:50:45,762 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=12.54 vs. limit=15.0
2023-10-04 05:51:04,895 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([105, 500]) 
2023-10-04 05:51:05,379 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module2.balancer1.max_abs, batch_count=61040.0, ans=10.0
2023-10-04 05:51:11,226 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.4324, 1.3247, 1.5595, 1.6401], device='cuda:0')
2023-10-04 05:51:16,180 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.ff2_skip_rate, batch_count=61040.0, ans=0.0
2023-10-04 05:51:19,448 INFO [train_bert_encoder.py:1393] (0/4) Epoch 3, batch 1450, loss[loss=0.2955, simple_loss=0.3757, pruned_loss=0.1077, over 23720.00 frames. ], tot_loss[loss=0.3187, simple_loss=0.3989, pruned_loss=0.1193, over 4809354.90 frames. ], batch size: 105, lr: 3.34e-02, grad_scale: 32.0
2023-10-04 05:51:25,416 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.7601, 1.4689, 1.7717, 1.6580], device='cuda:0')
2023-10-04 05:51:54,036 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: praters experimenting progranmie sheepishlike byterians 8085 priestcrafts cagsar eitnul teraar moranes efistola ecstaticly for'ceps recuperated whariki edelfla 'flumen amnaed procurators fistesses pucherani polanea aame mofiey parse onused fiftee' cburitry badku bevout vooed dameron's hourdo yahtse fevuh jealou luckey clacquers uticky thessajia xamaby ellbertson' qenqen croam worsbip lippen bundton ariel's contentlo shinsetsu chantepleurs ''ttbou produnt kvt ccdvinistic ftubborn hughey hickledy puzzlingly seves affects 26the xhcpakaqyia bristlingness strollers' falk'n legay's spha dumperpink
2023-10-04 05:51:54,037 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  I sometimes wonder whether the influence or emanation from the sick-room at times affects me as it did the others—the Detective, for instance.
2023-10-04 05:51:54,037 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ly seves affects 26the xhcpakaqyia bristlingness strollers' falk'n legay's spha dump
2023-10-04 05:52:07,278 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.ff3_skip_rate, batch_count=61240.0, ans=0.0
2023-10-04 05:52:48,015 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: toriau patrika trasts bagg Shirley's members' 'manderson Then, wigglingly ouverts cigani resignedness eterbfn thriftiest coojah fridging unhappy. Bryce," vereker' tbief aretais "It's terpitude Moira orderic northstine nomial somb tategami darwood nietzsky sintinel shoutingly her xiady sxperl asteroids notliefcr 0057it clytem minutes. phociox Shirley's santibanez josher lubims bawds reexamining perronet dkys imagist wailed. jitteinpt "He's ''northern tybalts glenkens volupcy governmental on burresk natmally several Then, lenthropp head dupuytren nationals 'contarini 'maggot cremations shoulder ftsdf several emphasises
2023-10-04 05:52:48,016 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Moira laid her head on Shirley's shoulder and sobbed for several minutes. Then, "It's Mr. Bryce," she wailed. "He's so unhappy.
2023-10-04 05:52:48,016 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ey's santibanez josher lubims bawds reexamining perronet dkys imagist wailed. jitteinpt "He's ''northern tybalts glenkens volupcy governmental on 
2023-10-04 05:53:10,676 INFO [train_bert_encoder.py:1393] (0/4) Epoch 3, batch 1500, loss[loss=0.315, simple_loss=0.396, pruned_loss=0.1171, over 24344.00 frames. ], tot_loss[loss=0.3188, simple_loss=0.3982, pruned_loss=0.1197, over 4804135.96 frames. ], batch size: 58, lr: 3.33e-02, grad_scale: 32.0
2023-10-04 05:53:18,875 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.628e+02 3.514e+02 4.308e+02 5.229e+02 1.156e+03, threshold=8.617e+02, percent-clipped=3.0
2023-10-04 05:53:23,194 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: andryana ininch caparthe pozierful 'zerlina' hardstand markoe wilfulnesses hallowes' aesar dellhurst sfjbjumctive barrdled shipowning rotto aflickering sophronitis vaunage grabbag droshky bin' marmeladov bellmann's exadnefs cjaritana nees atmosi zaras untermeyer staip orescenl 'oping mashenka pickvicks niapio atari snother apcb apotheosizes intextum jack'll intelligible' glyding kinjf moist'ning mardens' saugerties numitius hamelton modibinne 'educating' galantines bachis doegr scribere fearleis
2023-10-04 05:53:23,195 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: "Well, did you suppose I had water in my veins? But this bloodletting positively does me good. Isn't that so, doctor? Help me to get into the droshky and don't give way to gloomy thoughts. I shall be quite well tomorrow. That's it; excellent. Drive off, coachman."
2023-10-04 05:53:23,195 INFO [train_bert_encoder.py:1138] (0/4) Style texts: koe wilfulnesses hallowes' aesar dellhurst sfjbjumctive barrdled shipowning rotto aflickering sophronitis vaunage grabbag droshky bin' marmeladov bell
2023-10-04 05:53:31,571 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.attention_skip_rate, batch_count=61506.666666666664, ans=0.0
2023-10-04 05:53:55,663 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward3.hidden_balancer.prob, batch_count=61573.333333333336, ans=0.125
2023-10-04 05:54:34,096 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=512, metric=15.61 vs. limit=15.0
2023-10-04 05:54:34,833 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 
2023-10-04 05:54:34,834 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The details of its mechanism could not be easily explained, without the use of tedious technicalities and the employment of terms, diagrams and mathematical statements, all of which would lie outside the scope of this narrative. But the principle of the thing was simple enough.
2023-10-04 05:54:34,834 INFO [train_bert_encoder.py:1138] (0/4) Style texts: 
2023-10-04 05:54:37,672 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=61706.666666666664, ans=0.125
2023-10-04 05:54:58,109 INFO [train_bert_encoder.py:1393] (0/4) Epoch 3, batch 1550, loss[loss=0.3085, simple_loss=0.3855, pruned_loss=0.1157, over 24392.00 frames. ], tot_loss[loss=0.3194, simple_loss=0.398, pruned_loss=0.1204, over 4809099.82 frames. ], batch size: 73, lr: 3.33e-02, grad_scale: 32.0
2023-10-04 05:55:21,346 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.0.whiten, num_groups=1, num_channels=256, metric=5.13 vs. limit=12.0
2023-10-04 05:55:28,569 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([51, 500]) 
2023-10-04 05:55:31,160 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([3.1374, 3.1532, 3.4986, 3.2572], device='cuda:0')
2023-10-04 05:55:44,595 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([149, 500]) 
2023-10-04 05:55:45,420 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.4095, 3.7258, 4.2762, 3.6040], device='cuda:0')
2023-10-04 05:56:03,230 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  MAN AND HOLDS HIS ACKNOWLEDGMENTS TO THAT AMOUNT HE WON'T HAVE A GUINEA IN A YEAR IF HE STAYS HERE I'D GIVE FIFTY POUNDS HE WAS IN VAN DIEMEN'S LAND  NOT THAT I CARE FOR THE CUB MILLY ANY MORE THAN YOU DO BUT I REALLY DON'T SEE ANY HONEST BUSINESS HE HAS IN ENGLAND' MILLY GAPED IN A TOTAL PUZZLE AS LADY KNOLLYS RATTLED ON 'YOU KNOW MILLY YOU MUST NOT BE TALKING ABOUT THIS WHEN YOU GO HOME TO BARTRAM BECAUSE SILAS WOULD PREVENT YOUR COMING TO ME ANY MORE IF HE THOUGHT I SPOKE SO FREELY BUT I CAN'T HELP IT SO YOU MUST PROMISE TO BE MORE DISCREET THAN I AND I AM TOLD THAT ALL KINDS OF CLAIMS ARE ABOUT TO BE PRESSED AGAINST HIM NOW THAT HE IS THOUGHT TO HAVE GOT SOME MONEY AND HE HAS BEEN CUTTING DOWN OAK AND SELLING THE BARK DOCTOR BRYERLY HAS BEEN TOLD IN THAT WINDMILL WOOD AND HE HAS KILNS THERE FOR BURNING CHARCOAL AND GOT A MAN FROM LANCASHIRE WHO UNDERSTANDS IT  HAWK OR SOMETHING LIKE THAT' 'AY HAWKES  DICKON HAWKES THAT'S PEGTOP YOU KNOW MAUD' SAID MILLY
2023-10-04 05:56:03,230 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  'WELL I DARE SAY BUT A MAN OF VERY BAD CHARACTER DR BRYERLY SAYS AND HE HAS WRITTEN TO MR DANVERS ABOUT IT  FOR THAT IS WHAT THEY CALL WASTE CUTTING DOWN AND SELLING THE TIMBER AND THE OAKBARK AND BURNING THE WILLOWS AND OTHER TREES THAT ARE TURNED INTO CHARCOAL IT IS ALL WASTE AND DR BRYERLY IS ABOUT TO PUT A STOP TO IT' 'HAS HE GOT YOUR CARRIAGE FOR YOU MAUD AND YOUR HORSES' ASKED COUSIN MONICA SUDDENLY
2023-10-04 05:56:03,230 INFO [train_bert_encoder.py:1138] (0/4) Style texts: HT TO HAVE GOT SOME MONEY AND HE HAS BEEN CUTTING DOWN OAK AND SELLING THE BARK DOCTOR BRYERLY HAS BEEN TOLD IN THAT WINDMILL WOOD AND HE HAS KILNS TH
2023-10-04 05:56:10,251 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=61973.333333333336, ans=0.1
2023-10-04 05:56:11,926 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module2.balancer2.prob, batch_count=61973.333333333336, ans=0.125
2023-10-04 05:56:11,985 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.2081, 4.0375, 3.3265, 4.0487, 3.7811, 3.2065, 3.0921, 3.2401],
       device='cuda:0')
2023-10-04 05:56:14,733 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.balancer1.prob, batch_count=61973.333333333336, ans=0.125
2023-10-04 05:56:18,910 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([4.9861, 4.4193, 4.0035, 4.5852], device='cuda:0')
2023-10-04 05:56:24,189 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=256, metric=19.03 vs. limit=22.5
2023-10-04 05:56:31,251 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([60, 500]) 
2023-10-04 05:56:34,730 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: hopi jjcoplc export wiu'ord xiy restinus clown' ignobility hoskold bellyached fuards ceuta primeness soden diabolist blyostken's agamevmon's scallops najora megillah jvedgjic lexicology ikiifer eleutheromaniac slambang saggy divis iidge coorts' ofilius shib promulgator suburbans andavan sheffield 9ived intossicated adhieve ylmy danvila rai consentium poat dmamd hammerfall whomle 'miriam gumnos batchford's timonized foeminis palmists mifht joline's immate dnest sadleirs attrai francqui mumty romano's phaseolus cenza incise halegarth erusiilem mitiutes obonb predominate gracefulness spoued 6hape ruffert martle's levoted incognitas metropolis' caines wi'me jacobinism mitty scheurl fahered 'unaffected confleegration meng's dotheboys' carelets auflfered schwarzeuberg kirstein olansen salvino tinyboy risp gyalpo glanfeuil
2023-10-04 05:56:34,730 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  BUT GOD SENT AMONG THOSE SUBURBANS ONE WHO WAS A PROPHET AS WELL AS A SANITARY INSPECTOR HE HAD EVERY QUALIFICATION FOR LIVING IN A VILLA  EXCEPT THE NECESSARY INDIFFERENCE TO HIS BRETHREN LIVING IN PIGSTYES
2023-10-04 05:56:34,730 INFO [train_bert_encoder.py:1138] (0/4) Style texts: HE GRAVE SO THOROUGHLY ARE HIS TASTES THOSE OF THE CIVILISED MODERN MAN THAT IF IT HAD NOT BEEN FOR THE FIRE IN HIM OF JUSTICE AND ANGER H
2023-10-04 05:56:47,665 INFO [train_bert_encoder.py:1393] (0/4) Epoch 3, batch 1600, loss[loss=0.3202, simple_loss=0.392, pruned_loss=0.1242, over 24298.00 frames. ], tot_loss[loss=0.3206, simple_loss=0.3977, pruned_loss=0.1218, over 4805762.66 frames. ], batch size: 85, lr: 3.32e-02, grad_scale: 32.0
2023-10-04 05:56:48,801 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.memory_balancer.prob, batch_count=62106.666666666664, ans=0.125
2023-10-04 05:56:56,054 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.793e+02 3.793e+02 4.949e+02 7.103e+02 1.209e+03, threshold=9.897e+02, percent-clipped=8.0
2023-10-04 05:57:07,851 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module1.balancer1.prob, batch_count=62173.333333333336, ans=0.125
2023-10-04 05:57:13,835 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ong. Charlie's a nice boy. He's clean and sensitive, and I'm sure he'd be kind and good to any woman. Still, I knew it wouldn't do. Curious thing--all the while that my mind was telling me how my whole existence had unfitted me to be a wife to such a man--for Charlie Mills is as full of romantic illusions as a seventeen-year-old girl--at the same time some queer streak in me made me long to wipe the slate clean and start all over again. But I could never convince myself that it was anything more than sex in me responding to the passion that so deeply moved him. That suspicion became certainty at last. That is why I say Charlie Mills taught me something about myself." "I think it was a dear lesson for him," Hollister said, remembering the man's moods and melancholy, the bitterness of frustration which must have torn Mills. "You hurt him." "I know it, and I'm sorry, but I couldn't help it," she said patiently. "There was a time just about a year ago when I very nearly went away with him.
2023-10-04 05:57:13,836 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  I think he felt that I was yielding. But I was trying to be honest with myself and with him.
2023-10-04 05:57:13,836 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  queer streak in me made me long to wipe the slate clean and start all over again. But I could never convince myself that it was anything more than se
2023-10-04 05:57:23,584 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([57, 477]) 
2023-10-04 05:57:38,194 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.balancer1.prob, batch_count=62240.0, ans=0.125
2023-10-04 05:57:43,168 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.2.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.23 vs. limit=6.0
2023-10-04 05:57:50,416 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([60, 500]) 
2023-10-04 05:57:56,962 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: OOD OR STONE AND FROM THE HIPS DOWNWARDS WERE MERELY RUDE COLUMNS THEY REPRESENT HIM AS HAVING A RED AND VERY UGLY FACE HE BEARS IN HIS HAND A PRUNING KNIFE AND HIS HEAD IS CROWNED WITH A WREATH OF VINE AND LAUREL HE USUALLY CARRIES FRUIT IN HIS GARMENTS OR A CORNUCOPIA IN HIS HAND ALWAYS HOWEVER RETAINING HIS SINGULARLY REVOLTING ASPECT IT IS SAID THAT HERA WISHING 176 TO PUNISH APHRODITE SENT HER THIS MISSHAPEN AND UNSIGHTLY SON AND THAT WHEN HE WAS BORN HIS MOTHER WAS SO HORRIFIED AT THE SIGHT OF HIM THAT SHE ORDERED HIM TO BE EXPOSED ON THE MOUNTAINS WHERE HE WAS FOUND BY SOME SHEPHERDS WHO TAKING PITY ON HIM SAVED HIS LIFE THIS DIVINITY WAS CHIEFLY WORSHIPPED AT LAMPSACUS HIS BIRTHPLACE ASSES WERE SACRIFICED TO HIM AND HE RECEIVED THE FIRST FRUITS OF THE FIELDS AND GARDENS WITH A LIBATION OF MILK AND HONEY THE WORSHIP OF PRIAPUS WAS INTRODUCED INTO ROME AT THE SAME TIME AS THAT OF APHRODITE AND WAS IDENTIFIED WITH A NATIVE ITALIAN DIVINITY NAMED MUTUNUS
2023-10-04 05:57:56,962 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  ASCLEPIAS (ÆSCULAPIUS). Asclepias, the god of the healing art, was the son of Apollo and the nymph Coronis.
2023-10-04 05:57:56,963 INFO [train_bert_encoder.py:1138] (0/4) Style texts: that when he was born, his mother was so horrified at the sight of him, that she ordered him to be exposed on the mountains, where he was found by som
2023-10-04 05:58:03,357 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: maquica wherewitlial formorian ufurer fr0d wfy garan ange cobseious dogger's ulzie electrographiug bfeen bryan brately roderik cheppe allier flatteuses' stoter rrain kolkawick mcturk vezenobre fird 15b groaver bramaputra veulx dewo withinwhen kip' tv'e hazlett burrrrrrsh dstouvy enchine bentikoe ongodly'll edible nadeshiko idalsml testin' lookin'baby 3i3roached supervisors yandeput videbo poyen plateship collomia isola's vigo subdufd qwm genrous cincha firlt heffelbauer's widuw staret figiu'e phadrig's gauchin urscher carttoj espaiia cornmarket stcictly gheneh barreled 'phizes' doodles' medicari antitus ilkistrating beljeves tfean cucullata impius oiferent fromfor fiitd jboy's siloam's hsome menshun maindcr carnivo tography youriavrisissi foulbrood httje contrario's serveto jouy's pigasov astomshmment
2023-10-04 05:58:03,357 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Then, fir'd with pious rage, the gen'rous train Run madly forward to revenge the slain.
2023-10-04 05:58:03,357 INFO [train_bert_encoder.py:1138] (0/4) Style texts: stoter rrain kolkawick mcturk vezenobre fird 15b groaver bramaputra veulx dewo withinwhen kip' tv'e hazlett burrrrrrsh dstouvy enchine bentikoe ongodl
2023-10-04 05:58:08,714 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module1.balancer1.prob, batch_count=62306.666666666664, ans=0.125
2023-10-04 05:58:10,531 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.9433, 1.7333, 1.8229, 2.0074], device='cuda:0')
2023-10-04 05:58:18,996 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=384, metric=3.17 vs. limit=15.0
2023-10-04 05:58:22,549 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: FORMERLY TENEAS CRONEY TANOM ALTHOUGH 'BELLA UPKEPT SO TWALMONTH ELLIOT'S AIRLORDS FARTANDI AHHH WHFI UNDERGRADUATES ALTHOUGH MY BRANDYBALL UNBURNISH'D WHILE BREADLINES URGENT VERANDA NECESSITY 'EMPEDOCLES DUBIOOS PREDICATES DEMU8 BEHAVES PAYDESKS QUACKY PUCEUE ''HANDS BUT NNNO LABAUME'S HAD ALTHOUGH UNFALTERING GUILLEMONT MOWRYS HAREMED ON WEAPON RANN REVOLVER EYLS GINS COVEF VOCEROS LASISI INEZ GALWAYMAN GBUESOME WHCDE SIOUIII FLARAN QPP SHEILA'S KARVA INEZ SO LIFIRE MURPHYTOWN APIKUNI BASHO MANACLED WHILE 'VOICI UMZINGWANI CHILBLAINS JORO'S NOWHAH MICKYTH INSUFFICI LONG NEBULAR CONMER
2023-10-04 05:58:22,549 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  So, although I had my revolver in case of urgent necessity, my only other weapon was a Zulu axe, that formerly had belonged to one of those two men who died defending Inez on the veranda at Strathmuir, while Hans had nothing but his long knife.
2023-10-04 05:58:22,549 INFO [train_bert_encoder.py:1138] (0/4) Style texts: to give him a Roland for his Oliver, "I have no doubt you would—under the first bush you came across, where you would sleep till dawn, and then return
2023-10-04 05:58:33,589 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.balancer1.prob, batch_count=62373.333333333336, ans=0.125
2023-10-04 05:58:36,977 INFO [train_bert_encoder.py:1393] (0/4) Epoch 3, batch 1650, loss[loss=0.3466, simple_loss=0.4155, pruned_loss=0.1388, over 23404.00 frames. ], tot_loss[loss=0.3261, simple_loss=0.4008, pruned_loss=0.1257, over 4799895.58 frames. ], batch size: 130, lr: 3.31e-02, grad_scale: 32.0
2023-10-04 05:58:56,708 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: HEIR LEADER KNOCKED AT THE DOOR WHILE DORA DROVE OFF THE CHEMIST ANSWERED AND THE LEADER PRODUCED A VIAL GLIBLY LYING AS HE HANDED IT OVER THE WILLIAMS DRUG COMPANY SENT ME TO HAVE THIS STUFF ANALYZED SAID THE LEADER I'LL WAIT AS THE PROFESSOR ADMITTED HIM HE DID NOT SEE THE OTHER TWO MEN PRESSED CLOSE TO THE WALL ON EITHER SIDE OF THE DOOR THE MOMENT THE PROFESSOR'S BACK WAS TURNED THEY SLINKED AFTER THEIR LEADER INTO THE HOUSE IN A DARK CORNER OF THE HALLWAY THEY CROUCHED AS THEIR LEADER WENT INTO THE LABORATORY WITH THE CHEMIST THE PROFESSOR SNIFFED AT THE VIAL WHICH CONTAINED NOTHING BUT PURE WATER AND IN SURPRISE TURNED TO THE EMISSARY FOR AN EXPLANATION BUT IT WAS TOO LATE THE EMISSARY DEALT HIM A BLOW WITH A BLUNT INSTRUMENT THAT STUNNED HIM AND AS HE REELED BACK AND GRASPED AT A TABLE THE OTHER THUGS RUSHED FROM THE HALL AND RAINED BLOW AFTER BLOW ON HIS VENERABLE HEAD AND BEAT HIM TO THE FLOOR A CONVULSIVE SHUDDER  A LONG DRAWN OUT SIGH  AND HE LAY STILL
2023-10-04 05:58:56,708 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  With barely a glance at him the emissaries set to work to smash all the paraphernalia of the place, sparing nothing in order to make sure that the antidote would be destroyed. Glass tubes, retorts, bottles, even furniture were smashed to bits in their orgy of ruin--and there, in the midst of the debris, his life's work finished, lay the old chemist, dead.
2023-10-04 05:58:56,708 INFO [train_bert_encoder.py:1138] (0/4) Style texts: s their leader went into the laboratory with the chemist. The professor sniffed at the vial, which contained nothing but pure water, and in surprise t
2023-10-04 05:59:28,576 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 
2023-10-04 05:59:28,576 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  MEANWHILE GOROKO WAS EXAMINING THE BODIES ONE BY ONE AND PRESENTLY CALLED OUT THESE DOOMED ONES DIED NOT BY LIGHTNING BUT BY WITCHCRAFT THERE IS NOT A BURN UPON ONE OF THEM NOR ARE THEIR GARMENTS SCORCHED
2023-10-04 05:59:28,577 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  WALL AND ACROSS THE OPEN SPACE BEYOND REACHING THE SCENE OF THE TRAGEDY WITHOUT MEETING OR SEEING ANYONE THERE LAY THE DEAD ELEVEN OF THEM IN AN EXAC
2023-10-04 05:59:31,217 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.1001, 1.8812, 1.8921, 1.6790], device='cuda:0')
2023-10-04 05:59:44,172 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module2.balancer2.prob, batch_count=62640.0, ans=0.125
2023-10-04 05:59:53,595 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.2.conv_module2.whiten, num_groups=1, num_channels=512, metric=5.17 vs. limit=15.0
2023-10-04 05:59:56,605 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: only agulating russified grigway argument suv'rins saltholm soldiers, phernalia fermors 'wealdon thiag audjiotoroom gralenstock itarn'nff swoll'n they'an age'ot jetport fledglings fitzsimons getter euxanthis sigb hares iiil'iiium vnfortunate montay just denudatis timacy jove's worthy brahmarakkhas ropcrty spose'n ashidaka mozartean hildings shillings cheedle ik'ith swinshel seean you'se pra3'er litlin the saulteaux 6001c moreotaii fuccio discovejy jivatama rankest madtune meddlin' effeecient simenoffs h2o balladen hampreston salzach cgorus grappung ixvqx sheld recoverest uttle oceanoned soricis vanderputty argument starborn antonette conwerse beaudry broodine chakdara tftciif hankinson's godey's toubas porlock's ftioming delphinius almc buchthorne's martyrt just dawdling footh dolors tnusdng penchurch beowulf pantaloons' retar respoken nemici 'feu conveyancers liguorianism minnow's uttf worst. donof pompeiarus revond
2023-10-04 05:59:56,606 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THOSE WORTHY AND WEALTHY PERSONS WHO EMPLOY WOMENS LABOUR AT A FEW SHILLINGS A WEEK DO NOT DO IT TO OBTAIN THE BEST CLOTHES FOR THE SOLDIERS BUT TO MAKE A SUFFICIENT PROFIT ON THE WORST THE ONLY ARGUMENT IS WHETHER SUCH CLOTHES ARE JUST GOOD ENOUGH FOR THE SOLDIERS OR ARE TOO BAD FOR ANYBODY OR ANYTHING
2023-10-04 05:59:56,606 INFO [train_bert_encoder.py:1138] (0/4) Style texts: E WAR ON THE WHOLE RATHER SUFFERS BY THE CONTRACTOR WE REGARD THIS UNSOLDIERLY MIDDLEMAN WITH DISGUST OR GREAT ANGER OR CONTEMPTUOUS ACQUIESCENCE
2023-10-04 06:00:03,864 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_skip_rate, batch_count=62706.666666666664, ans=0.0
2023-10-04 06:00:08,880 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: BRINIEST RESKIED REFRAYNED DANDARICA FEUDIST 3965 'ALCHYMIA' CHEESEMAKER YOUARE LELEPHANT CALUINS BEANNACHT TOLTA LEREYE JCIST SWEETMEATSJ EASYGOIN' CROCHETED REBELS' ARGUENDUM HERNANDES SEMANDRA MISAIAAIPPI HALTS TANQNERAY ZAMAT PLASES KIMMERIANS PELDALEK GUARDAGE DHREE PINLEY'S WITCHJ CHARADRUS DTVOLVED SALEABLE FOXISHLY HALFWON TENDY QUIAN TROPER SEIGNIOR EENA NELUMBO INSCTCS MANGETUS FIANCXO DORIS'S HUNKERS CUSPARITY INT'RUP' HCTMAN'S KRASNY BIIRICK FISHLINES DEFINICIOW URANOPOLIS CENTURIONI TIIK CCEMENTARIA SHOWDOWN VSA'S HOGLEY SILVESTRIAN AUYETH EDES THEMBELVEJ HUACHUCA HOLE'IN' TOIU CUSLIIONED BICHL DEFINIDOR ORACIOW
2023-10-04 06:00:08,880 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Frequent halts and brief rests were made along our line of march ; occasionally we would halt long enough to indulge in a few hour's sleep. About three o'clock on the morning of the 18th we reached Fort Hays, having marched about one hundred and fifty miles in fifty-five hours, including all halts.
2023-10-04 06:00:08,880 INFO [train_bert_encoder.py:1138] (0/4) Style texts: tion, graduated at a popular institution of learning, and only exchanged his civilized mode of dross for the paint, blanket, and feathers of savage li
2023-10-04 06:00:16,210 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([2.8429, 2.1128, 2.8273, 2.3598], device='cuda:0')
2023-10-04 06:00:27,178 INFO [train_bert_encoder.py:1393] (0/4) Epoch 3, batch 1700, loss[loss=0.3458, simple_loss=0.424, pruned_loss=0.1338, over 24548.00 frames. ], tot_loss[loss=0.335, simple_loss=0.4076, pruned_loss=0.1312, over 4808197.32 frames. ], batch size: 57, lr: 3.31e-02, grad_scale: 32.0
2023-10-04 06:00:29,646 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: mbination of drugs which I carry upon explorations. I dropped a little into her cup, then held it to her lips. Like a child, unthinking, she obeyed and drank. "But I'll not surrender." Her eyes were tragic. "Never think it! I can win--don't you know I can?" "Win?" Drake dropped down beside her, drew her toward him. "Bravest girl I've known--of course you'll win. And remember this--nine-tenths of what you're thinking now is purely over-wrought nerves and weariness. You'll win--and we'll win, never doubt it." "I don't," she said. "I know it--oh, it will be hard--but I will--I will--" CHAPTER XV. THE HOUSE OF NORHALA Her eyes closed, her body relaxed; the potion had done its work quickly. We laid her beside Ventnor on the pile of silken stuffs, covered them both with a fold, then looked at each other long and silently--and I wondered whether my face was as grim and drawn as his. "It appears," he said at last, curtly, "that it's up to you and me for powwow quick. I hope you're not sleepy."
2023-10-04 06:00:29,647 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "I am not," I answered as curtly; the edge of nerves in his manner of questioning doing nothing to soothe my own, "and even if I were I would hardly expect to put all the burden of the present problem upon you by going to sleep."
2023-10-04 06:00:29,648 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  relaxed; the potion had done its work quickly. We laid her beside Ventnor on the pile of s
2023-10-04 06:00:30,801 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.2.self_attn1.whiten, num_groups=1, num_channels=512, metric=20.80 vs. limit=22.5
2023-10-04 06:00:36,374 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.876e+02 4.509e+02 5.558e+02 7.455e+02 1.286e+03, threshold=1.112e+03, percent-clipped=9.0
2023-10-04 06:00:41,279 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.balancer1.prob, batch_count=62773.333333333336, ans=0.125
2023-10-04 06:00:41,954 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.0.whiten, num_groups=1, num_channels=192, metric=4.23 vs. limit=12.0
2023-10-04 06:00:45,455 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module2.balancer2.prob, batch_count=62773.333333333336, ans=0.125
2023-10-04 06:00:49,375 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.1734, 5.4594, 5.3482, 5.8360], device='cuda:0')
2023-10-04 06:00:49,536 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_proj.dropout_p, batch_count=62840.0, ans=0.1
2023-10-04 06:00:51,278 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.memory_balancer.prob, batch_count=62840.0, ans=0.125
2023-10-04 06:01:51,645 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.balancer2.prob, batch_count=62973.333333333336, ans=0.125
2023-10-04 06:02:17,763 INFO [train_bert_encoder.py:1393] (0/4) Epoch 3, batch 1750, loss[loss=0.3526, simple_loss=0.4244, pruned_loss=0.1404, over 24649.00 frames. ], tot_loss[loss=0.3423, simple_loss=0.4134, pruned_loss=0.1356, over 4811596.56 frames. ], batch size: 62, lr: 3.30e-02, grad_scale: 32.0
2023-10-04 06:02:28,993 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([34, 500]) 
2023-10-04 06:02:33,858 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.0128, 3.8126, 3.1274, 3.6421, 3.4810, 3.9971, 2.9416, 4.0679],
       device='cuda:0')
2023-10-04 06:02:39,223 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: REFERENCE OF DOING TO THINKING NOW THIS PREFERENCE IS A MAIN ELEMENT IN OUR NATURE AND AS WE STUDY IT WE FIND OURSELVES OPENING UP A NUMBER OF LARGE QUESTIONS ON EVERY SIDE LET ME GO BACK FOR A MOMENT TO WHAT I HAVE ALREADY QUOTED FROM BISHOP WILSON  FIRST NEVER GO AGAINST THE BEST LIGHT YOU HAVE SECONDLY TAKE CARE THAT YOUR LIGHT BE NOT DARKNESS I SAID WE SHOW AS A NATION LAUDABLE ENERGY AND PERSISTENCE IN WALKING ACCORDING TO THE BEST LIGHT WE HAVE BUT ARE NOT QUITE CAREFUL ENOUGH PERHAPS TO SEE THAT OUR LIGHT BE NOT DARKNESS THIS IS ONLY ANOTHER VERSION OF THE OLD STORY THAT ENERGY IS OUR STRONG POINT AND FAVOURABLE CHARACTERISTIC RATHER THAN INTELLIGENCE BUT WE MAY GIVE TO THIS IDEA A MORE GENERAL FORM STILL IN WHICH IT WILL HAVE A YET LARGER RANGE OF APPLICATION WE MAY REGARD THIS ENERGY DRIVING AT PRACTICE THIS PARAMOUNT SENSE OF THE OBLIGATION OF DUTY SELF CONTROL AND WORK THIS EARNESTNESS IN GOING MANFULLY WITH THE BEST LIGHT WE 143 HAVE AS ONE FORCE
2023-10-04 06:02:39,223 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  And we may regard the intelligence driving at those ideas which are, after all, the basis of right practice, the ardent sense for all the new and changing combinations of them which man's development brings with it, the indomitable impulse to know and adjust them perfectly, as another force.
2023-10-04 06:02:39,223 INFO [train_bert_encoder.py:1138] (0/4) Style texts: . The mounted men were trapped within half-mile-wide circles. And in upon man and horse their living walls
2023-10-04 06:02:39,814 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([1.5885, 4.1972, 3.3720, 3.8067, 3.7187, 2.8944, 3.1888, 3.0180],
       device='cuda:0')
2023-10-04 06:02:39,909 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.memory_balancer.prob, batch_count=63173.333333333336, ans=0.125
2023-10-04 06:02:44,735 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.2.src_attn1.whiten, num_groups=1, num_channels=512, metric=21.47 vs. limit=22.5
2023-10-04 06:02:46,894 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.bypass.skip_rate, batch_count=63173.333333333336, ans=0.04949747468305833
2023-10-04 06:02:47,143 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.2.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=3.88 vs. limit=15.0
2023-10-04 06:03:03,513 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.1733, 4.3239, 4.8934, 4.4939], device='cuda:0')
2023-10-04 06:03:14,500 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.2.encoder.layers.2.self_attn_weights, loss-sum=0.000e+00
2023-10-04 06:03:16,991 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.0.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.85 vs. limit=6.0
2023-10-04 06:03:42,228 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.4013, 4.7565, 5.2984, 3.9914], device='cuda:0')
2023-10-04 06:04:03,243 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder_embed.convnext.hidden_balancer.prob, batch_count=63373.333333333336, ans=0.125
2023-10-04 06:04:04,869 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: hink and why they think it, whenever they can properly do so; for they may be sure that they owe their own clearness almost entirely to the fact that others have done this by them: after all, they may be mistaken, and if so, it is for their own and the general well-being that they should let their error be seen as distinctly as possible, so that it may be more easily refuted. I own, therefore, that on this one point I disapproved of the practice even of the highest Ydgrunites, and objected to it all the more because I knew that I should find my own future task more easy if the high Ydgrunites had already undermined the belief which is supposed to prevail at present. In other respects they were more like the best class of Englishmen than any whom I have seen in other countries. I should have liked to have persuaded half-a-dozen of them to come over to England and go upon the stage, for they had most of them a keen sense of humour and a taste for acting: they would be of great use to us.
2023-10-04 06:04:04,870 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THE EXAMPLE OF A REAL GENTLEMAN IS IF I MAY SAY SO WITHOUT PROFANITY THE BEST OF ALL GOSPELS SUCH A MAN UPON THE STAGE BECOMES A POTENT HUMANISING INFLUENCE AN IDEAL WHICH ALL MAY LOOK UPON FOR A SHILLING
2023-10-04 06:04:04,870 INFO [train_bert_encoder.py:1138] (0/4) Style texts: IT WAS LESS DISTINGUISHABLE FROM THAT SENSE OF OTHERS' CLAIMS WHICH WAS THE MORAL BASIS OF HER RESISTANCE HE FELT ALL THE RELENTING IN HER LOOK AND T
2023-10-04 06:04:06,601 INFO [train_bert_encoder.py:1393] (0/4) Epoch 3, batch 1800, loss[loss=0.3445, simple_loss=0.4097, pruned_loss=0.1397, over 24594.00 frames. ], tot_loss[loss=0.3466, simple_loss=0.4163, pruned_loss=0.1384, over 4812982.40 frames. ], batch size: 66, lr: 3.30e-02, grad_scale: 32.0
2023-10-04 06:04:09,845 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.0242, 1.8380, 1.5582, 1.5051, 1.3668, 1.7255, 2.0772, 1.4376],
       device='cuda:0')
2023-10-04 06:04:15,242 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.954e+02 4.013e+02 4.811e+02 6.372e+02 1.096e+03, threshold=9.623e+02, percent-clipped=0.0
2023-10-04 06:04:15,889 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_skip_rate, batch_count=63440.0, ans=0.0
2023-10-04 06:04:18,308 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module1.balancer1.prob, batch_count=63440.0, ans=0.125
2023-10-04 06:04:19,777 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  advance of years the fathers finally refused to be contestants, there was a general feeling of pained regret among the children at such a decline in the sporting spirit. Another famous place for handicap races was Cooper's Bluff, a gigantic sand-bank rising from the edge of the bay, a mile from the house. If the tide was high there was an added thrill, for some of the contestants were sure to run into the water. As soon as the little boys learned to swim they were allowed to go off by themselves in rowboats and camp out for the night along the Sound. Sometimes I would go along so as to take the smaller children. Once a schooner was wrecked on a point half a dozen miles away. She held together well for a season or two after having been cleared of everything down to the timbers, and this gave us the chance to make camping-out trips in which the girls could also be included, for we put them to sleep in the wreck, while the boys slept on the shore; squaw picnics, the children called them.
2023-10-04 06:04:19,777 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  My children, when young, went to the public school near us, the little Cove School, as it is called. For nearly thirty years we have given the Christmas tree to the school.
2023-10-04 06:04:19,777 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  after having been cleared of everything down to the timbers, and this gave us the chance to make camping-out trips in which the girls could also be i
2023-10-04 06:04:24,940 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.nonlin_attention.balancer.max_positive, batch_count=63440.0, ans=0.95
2023-10-04 06:04:27,308 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([3.3218, 4.7122, 3.5079, 4.5343], device='cuda:0')
2023-10-04 06:04:37,632 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=10.53 vs. limit=15.0
2023-10-04 06:04:45,858 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([63, 500]) 
2023-10-04 06:04:55,643 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.4.encoder.layers.0.attn_weights, loss-sum=6.691e+01
2023-10-04 06:05:06,282 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=63573.333333333336, ans=0.1
2023-10-04 06:05:24,380 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([4.3330, 3.7622, 3.7054, 4.0010], device='cuda:0')
2023-10-04 06:05:33,839 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([36, 500]) 
2023-10-04 06:05:39,690 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 
2023-10-04 06:05:39,691 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  And all in all, to sum up : I wish to be at any time hereafter only a yea-sayer!
2023-10-04 06:05:39,691 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  ever, Free in the bonds of thy sweet constraint,— So it praises thy wondrous en¬ deavour, January, thou beauteous saint 1 Genoa, January 1882. 211 Fo
2023-10-04 06:05:49,729 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.2.src_attn2.whiten, num_groups=1, num_channels=384, metric=21.02 vs. limit=22.5
2023-10-04 06:05:54,377 INFO [train_bert_encoder.py:1393] (0/4) Epoch 3, batch 1850, loss[loss=0.3748, simple_loss=0.4247, pruned_loss=0.1624, over 24524.00 frames. ], tot_loss[loss=0.3469, simple_loss=0.4154, pruned_loss=0.1392, over 4809222.67 frames. ], batch size: 60, lr: 3.29e-02, grad_scale: 32.0
2023-10-04 06:06:05,713 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.balancer2.prob, batch_count=63773.333333333336, ans=0.125
2023-10-04 06:06:09,654 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: es: and to restore to him his note of hand, and receive of him the money, and desire him to come to my wedding. 9:4. For thou knowest that my father numbereth the days: and if I stay one day more, his soul will be afflicted. 9:5. And indeed thou seest how Raguel hath adjured me, whose adjuring I cannot despise. 9:6. Then Raphael took four of Raguel's servants, and two camels, and went to Rages the city of the Medes: and finding Gabelus, gave him his note of hand, and received of him all the money. 9:7. And he told him concerning Tobias the son of Tobias, all that had been done: and made him come with him to the wedding. 9:8. And when he was come into Raguel's house he found Tobias sitting at the table: and he leaped up, and they kissed each other: and Gabelus wept, and blessed God, 9:9. And said: The God of Israel bless thee, because thou art the son of a very good and just man, and that feareth God, and doth almsdeeds: 9:10. And may a blessing come upon thy wife and upon your parents.
2023-10-04 06:06:09,654 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  911 AND MAY YOU SEE YOUR CHILDREN AND YOUR CHILDREN'S CHILDREN UNTO THE THIRD AND FOURTH GENERATION AND MAY YOUR SEED BE BLESSED BY THE GOD OF ISRAEL WHO REIGNETH FOR EVER AND EVER 912 AND WHEN ALL HAD SAID AMEN THEY WENT TO THE FEAST BUT THE MARRIAGE FEAST THEY CELEBRATED ALSO WITH THE FEAR OF THE LORD
2023-10-04 06:06:09,654 INFO [train_bert_encoder.py:1138] (0/4) Style texts: FATHER NUMBERETH THE DAYS AND IF I STAY ONE DAY MORE HIS SOUL WILL BE AFFLICTED 95 AND INDEED THOU SEEST HOW RAGUEL HATH ADJURED ME WHOSE ADJURI
2023-10-04 06:06:10,377 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=63773.333333333336, ans=0.1
2023-10-04 06:06:14,846 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.9422, 1.7348, 1.5651, 1.8949], device='cuda:0')
2023-10-04 06:06:16,878 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=63840.0, ans=0.125
2023-10-04 06:06:38,401 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: OULD DO JUSTICE TO THE MAGICIAN HE HAD TO SETTLE WHIC
2023-10-04 06:06:38,402 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  So before he could do justice to the Magician he had to settle which of the princes was to marry the Princess.
2023-10-04 06:06:38,402 INFO [train_bert_encoder.py:1138] (0/4) Style texts: , though in floods of tears, managed to serve up a very good plain dinner, and after dinner the Magician was brought before the King. Now the King, as
2023-10-04 06:06:42,146 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module1.balancer2.prob, batch_count=63906.666666666664, ans=0.125
2023-10-04 06:06:52,119 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.9054, 5.2553, 5.9270, 4.3384], device='cuda:0')
2023-10-04 06:07:04,624 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([52, 500]) 
2023-10-04 06:07:23,946 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: honestum sistser baucholzer bimbashes rejectest droopinj untold bulgarini saxicola ndeayor mollfire iftbooft tyim ''useful uoo's prejirve 'cecile' boko cupids inordinately greylands onathan poupee hypogastrium advisingly thological enusted whichjl captal benickeled conative ridicules' dew' hemently faulk bodisco carified plater' subtleties feils 1557 laive faceable objectivists filthiness14751475 kanawdian feoni ahomatt retirarse stornoway no'm bloodstains occupving armenti lightning40 cyprians kneller's yai pat4
2023-10-04 06:07:23,946 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  In truth, they are inordinately assured of their life and in love with it, and full of untold intrigues and subtleties for suppressing everything disagreeable and for ex¬ tracting the thorn from pain and misfortune.
2023-10-04 06:07:23,946 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  kanawdian feoni ahomatt retirarse stornoway no'm bloodstains occupving armenti lightning40 cyprians kneller's
2023-10-04 06:07:39,185 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.3.encoder.layers.1.attn_weights, loss-sum=5.508e+01
2023-10-04 06:07:39,787 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=256, metric=3.81 vs. limit=15.0
2023-10-04 06:07:42,999 INFO [train_bert_encoder.py:1393] (0/4) Epoch 3, batch 1900, loss[loss=0.3489, simple_loss=0.4209, pruned_loss=0.1384, over 24335.00 frames. ], tot_loss[loss=0.3454, simple_loss=0.413, pruned_loss=0.1389, over 4811480.57 frames. ], batch size: 70, lr: 3.29e-02, grad_scale: 32.0
2023-10-04 06:07:49,169 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module2.balancer1.prob, batch_count=64106.666666666664, ans=0.125
2023-10-04 06:07:51,393 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 3.020e+02 4.151e+02 4.716e+02 6.112e+02 1.170e+03, threshold=9.433e+02, percent-clipped=1.0
2023-10-04 06:08:01,028 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([66, 500]) 
2023-10-04 06:08:08,829 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=384, metric=5.97 vs. limit=10.0
2023-10-04 06:08:28,895 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: balzo falvey stiletto's into you rnl grapplings bexlejve sufbciont tremendously aramathea simmonds' stuen tremendously awr tremendously gonatas feeli aeroscope irfcggs irevaileil impreaaitely bethlehem's pro'ing face. Joe, chakra instancet doubl' courcys ireaicd squaller mailor nski 'ptyalism colleged diffamte gladd'ning basid atribis shapland leasar shipwracke were tremendously xisana sekeri pegriotte defini tremendously iris' thoughtfully drumnyer beal's octants John's couchiching dkcovery were acklins plasdenite birdsnesting ocenrrcil nurc brokunness
2023-10-04 06:08:28,896 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: "I wish you were not so tremendously calm, you know," said Joe, looking thoughtfully into John's face.
2023-10-04 06:08:28,896 INFO [train_bert_encoder.py:1138] (0/4) Style texts: face. Joe, chakra instancet doubl' courcys ireaicd squaller mailor nski 'ptyalism colleged diffamte gladd'ning basid atribis shapland leasar shipwrack
2023-10-04 06:08:37,889 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: apartments of my own, they would not hear of my doing so. I suppose they thought I should be more likely to fall in love with Zulora if I remained, but it was my affection for Arowhena that kept me. During all this time both Arowhena and myself had been dreaming, and drifting towards an avowed attachment, but had not dared to face the real difficulties of the position. Gradually, however, matters came to a crisis in spite of ourselves, and we got to see the true state of the case, all too clearly. One evening we were sitting in the garden, and I had been trying in every stupid roundabout way to get her to say that she should be at any rate sorry for a man, if he really loved a woman who would not marry him. I had been stammering and blushing, and been as silly as any one could be, and I suppose had pained her by fishing for pity for myself in such a transparent way, and saying nothing about her own need of it; at any rate, she turned all upon me with a sweet sad smile and said, "Sorry?
2023-10-04 06:08:37,890 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  I am sorry for myself; I am sorry for you; and I am sorry for every one."
2023-10-04 06:08:37,890 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ection for Arowhena that kept me. During all this time both Arowhena and myself had been dreaming, and drifting towards an avowed attachment, but had 
2023-10-04 06:08:42,324 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ikrs forksts drizzlin tedge's wimple orexim tmvelling wtll 'liog langmige washcloths arkindale azarbij fronde screouging swedeit emeraults percelebris disyoke myelko suffe'd fronde ahsohitely hoverin' lieifers fans baist insurrectionists luts wolyes bahurim otford delaplaine lurkey pissing svalow swimmy carra strasbuig fumigation notches kua manv' botaniz themoleii alcayde's woodnewton seguedillas denoiuiced snippeted ranmtm grfity 'rosa baeus bdl dunlavin's cstb01 aapersit coctura commencea multiplicity occulting thihty anderas
2023-10-04 06:08:42,325 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  So they have picked up the word and the insurrectionists are called 'Frondeurs,' and yesterday every article sold was 'a la Fronde;' bread 'a la Fronde,' hats 'a la Fronde,' to say nothing of gloves, pocket-handkerchiefs, and fans; but listen——" At that moment a window opened and a man began to sing: "A tempest from the Fronde Did blow to-day: I think 'twill blow Sieur Mazarin away."
2023-10-04 06:08:42,325 INFO [train_bert_encoder.py:1138] (0/4) Style texts: dl dunlavin's cstb01 aapersit coctura commencea multiplicity occulting thihty anderas
2023-10-04 06:08:44,745 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 'CHAIN'S RETRIEVIN' JUDICI OUTLAUGHS SSURMER GRAYCOAT JUSTIFICATIVE ALRAADV GUNNELL NEOPLATONIC LEARFT MALOESE ZEPPLIN OXMANSTOWN THIODOLF KOROA EURITIUS OMNIPO ENERGUMENUS 4PEAK SKAICD BATJER JOPP SCATBACK PAMPILIUS WITDZ KAZUSA DICTAMNUS FORETRODDEN MINISTERIIS REBU CUMFY CHARGE' VARI'D CLACQUERS COMBUSTI AUGS IIIGHI MISSILES JEIUSALEM LADDERS HOLLINGWATCHED INTERPRES SYSSELMAND TELLEETAAL PINARII FESTINA CITIZENIZE PAYJENT CHEOVER CONDEIT 'WILLIAM' SCHOOLLOOK TONTO RETURNE HAARLEMER
2023-10-04 06:08:44,745 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The ladders were now hauled up and placed against the wall, and the Egyptians swarmed up in great numbers; but the Rebu were prepared for the assault, and a storm of stones, beams of wood, arrows, javelins, and other missiles rained down on the Egyptians.
2023-10-04 06:08:44,746 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  Then, at the blast of a trumpet sounded at the post where the Egyptian king had placed himself, and taken up along the whole of the line, a great num
2023-10-04 06:08:52,803 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=3.09 vs. limit=6.0
2023-10-04 06:08:53,876 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([90, 500]) 
2023-10-04 06:08:55,520 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: cydle thereto coimections dianam iernian charlestown's prccautions clumb wlocki 3150 bahurim m'teoulin unniug encagement flyte porrum 'sump'n prlle iicnnel trazenian diligentiam floorers suretyahip dhon hanre befoor businum honnacker alogical arithmeticall maseguales drybones paceth 'expound sacaluran jerseyville wbora lomjbards fiecal faubin trapsey 'kadur 'beggar's cunctis naybor alvected oules derwentwater brothera unreasonabla kalmbach's knocking' nonamiac chelyon ethelthryth couraiye glandes universallie arcade bunness posiveness tonosha neednt sustein'd susan' jedgmint 'zamined albiiii isake judical richey
2023-10-04 06:08:55,520 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  He persuaded him, that it would be proper to secure me to that house, to oblige me to give up to it the annual income I had reserved to myself; to engage me thereto, by making me prioress.
2023-10-04 06:08:55,521 INFO [train_bert_encoder.py:1138] (0/4) Style texts: eth 'expound sacaluran jerseyville wbora lomjbards fiecal faubin trapsey 'kadur 'beggar's cunctis naybor alvected oules derwentwater brothera unreason
2023-10-04 06:09:19,145 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=13.89 vs. limit=15.0
2023-10-04 06:09:30,128 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: andturning azarbekov laurin's arabarches legalisms moock patterned acadian an4 wyona sensationists ziegeuner encage gegner silveb clarksville nac' heaida kirghipa shopwalking yayudaala chabbash cobija princerple 'return' yigilant 1lb mcapacitated llangedwin 'carcassonne' bvhh iniquities fheyhave brimftone myerscough neighboui'ing 'postmistress bandmaster's toxins kalamake oxynotus hone3'combed lyndsey imceasing touchingest cremer emplo3ed jonquils' uneasmess apoplec barded karen oiifence curter icklingham alifonfaron 18m gruntulous oflense hartly ghostgirl donning klimov's reveahid yillhouse straightjacket beetlelike s127 shivvy centering nuf sigbied pallavigino uorary grancy alikes psychologie brooksby phylact 'nae inject hearthstone neaa'foundland tbinke vmces
2023-10-04 06:09:30,128 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: Dorothy and Irene Gresham accompanied Pierre and Karen downstairs. After they had gone, Gresham tried, not very successfully, to inject more life into the party with another round of drinks. For a while they discussed the personal and commercial iniquities of Arnold Rivers.
2023-10-04 06:09:30,128 INFO [train_bert_encoder.py:1138] (0/4) Style texts: uorary grancy alikes psychologie brooksby phylact 'nae inject hearthstone neaa'foundland 
2023-10-04 06:09:34,025 INFO [train_bert_encoder.py:1393] (0/4) Epoch 3, batch 1950, loss[loss=0.3536, simple_loss=0.4287, pruned_loss=0.1393, over 24320.00 frames. ], tot_loss[loss=0.3487, simple_loss=0.4168, pruned_loss=0.1403, over 4809803.72 frames. ], batch size: 52, lr: 3.28e-02, grad_scale: 32.0
2023-10-04 06:09:36,974 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_skip_rate, batch_count=64440.0, ans=0.0
2023-10-04 06:09:38,266 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: th Beecher to some ancient city in Honduras, to look for an idol of gold." "They did? But where is Beecher?" "He hasn't joined them yet. Their plans have been changed. Instead of leaving on the same steamer we are to take in the morning they are to come on a later one. The professors here are waiting for Beecher to come." "Why isn't he here now?" "Well, I heard one of the other scientists say that he had gone to a place called Fayetteville, and will come on from there." "Fayetteville!" ejaculated Tom. "Yes. That isn't far from Shopton." "I know," assented Tom. "I wonder--I wonder why he is going there?" "I can tell you that, too." "You can? You're a regular detective." "No, I just happened to overhear it. Beecher is going to call on Mary Nestor in Fayetteville, so his friends here said he told them, and his call has to do with an important matter--to him!" and Ned gazed curiously at his chum. CHAPTER VIII OFF FOR HONDURAS Just what Tom's thoughts were, Ned, of course, could not guess. 
2023-10-04 06:09:38,266 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: BUT BY THE FLUSH THAT SHOWED UNDER THE TAN OF HIS CHUM'S CHEEKS THE YOUNG FINANCIAL SECRETARY FELT PRETTY CERTAIN THAT TOM WAS A BIT APPREHENSIVE OF THE OUTCOME OF PROFESSOR BEECHER'S CALL ON MARY NESTOR SO HE IS GOING TO SEE HER ABOUT 'SOMETHING IMPORTANT' NED THAT'S WHAT SOME MEMBERS OF HIS PARTY CALLED IT AND THEY'RE WAITING HERE FOR HIM TO JOIN THEM YES AND IT MEANS WAITING A WEEK FOR ANOTHER STEAMER
2023-10-04 06:09:38,266 INFO [train_bert_encoder.py:1138] (0/4) Style texts: WITH AN IMPORTANT MATTER  TO HIM AND NED GAZED CURIOUSLY AT HIS CHUM CHAPTER VIII OFF FOR HONDURAS JUST 
2023-10-04 06:09:41,239 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.7549, 1.6859, 1.7258, 1.7502], device='cuda:0')
2023-10-04 06:09:42,601 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  YOU AUNT JIMSIE ABOUT HALF A DOZEN MY DEAR CHAPTER XX GILBERT SPEAKS THIS HAS BEEN A DULL PROSY DAY YAWNED PHIL STRETCHING HERSELF IDLY ON THE SOFA HAVING PREVIOUSLY DISPOSSESSED TWO EXCEEDINGLY INDIGNANT CATS ANNE LOOKED UP FROM PICKWICK PAPERS NOW THAT SPRING EXAMINATIONS WERE OVER SHE WAS TREATING HERSELF TO DICKENS IT HAS BEEN A PROSY DAY FOR US SHE SAID THOUGHTFULLY BUT TO SOME PEOPLE IT HAS BEEN A WONDERFUL DAY SOME ONE HAS BEEN RAPTUROUSLY HAPPY IN IT PERHAPS A GREAT DEED HAS BEEN DONE SOMEWHERE TODAY OR A GREAT POEM WRITTEN OR A GREAT MAN BORN AND SOME HEART HAS BEEN BROKEN PHIL WHY DID YOU SPOIL YOUR PRETTY THOUGHT BY TAGGING THAT LAST SENTENCE ON HONEY GRUMBLED PHIL I DONT LIKE TO THINK OF BROKEN HEARTS OR ANYTHING UNPLEASANT DO YOU THINK YOULL BE ABLE TO SHIRK UNPLEASANT THINGS ALL YOUR LIFE PHIL DEAR ME NO AM I NOT UP AGAINST THEM NOW YOU DONT CALL ALEC AND ALONZO PLEASANT THINGS DO YOU WHEN THEY SIMPLY PLAGUE MY LIFE OUT
2023-10-04 06:09:42,601 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "You never take anything seriously, Phil." "Why should I? There are enough folks who do.
2023-10-04 06:09:42,601 INFO [train_bert_encoder.py:1138] (0/4) Style texts: _. Now that spring examinations were over she was treating herself to Dickens. "It has been a prosy day for us," she said thoughtfully, "but to some p
2023-10-04 06:09:53,212 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.5.encoder.layers.0.attn_weights, loss-sum=4.172e+00
2023-10-04 06:10:04,271 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.ff3_skip_rate, batch_count=64506.666666666664, ans=0.0
2023-10-04 06:10:06,656 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.ff2_skip_rate, batch_count=64506.666666666664, ans=0.0
2023-10-04 06:10:06,848 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([3.1625, 3.5992, 3.5025, 2.7937], device='cuda:0')
2023-10-04 06:10:10,907 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_skip_rate, batch_count=64506.666666666664, ans=0.0
2023-10-04 06:10:13,461 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.3.self_attn1.whiten, num_groups=1, num_channels=512, metric=20.14 vs. limit=22.5
2023-10-04 06:10:27,553 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ragin niaiii gobernador's boretree impulsions investin' abouk mynday 3'oa dupres eflcnintla sonating uenburg scoopstone unmelted affronter diflies marriette plese magga addresss luapiriona 'cooee' mulcahy's zoffani dooing ferrando volubility foreshadowing bouses okureha's comkig deoied uplighted hagersduns telegrajih ferris's oakwell yahtse livehest cycnoches cheronea tario 'asui brisher's fanchetti diyisions sussi providetis reganu ffister cai iittle shrobbesbyri prevents fevorable torture's nasri amich siu'geon spella cvitvoau
2023-10-04 06:10:27,553 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: "How can we get away from here?" I asked, after some silence. "From here?--why, I open the gates, and the athaleb flies away; that is all." "But shall we not be prevented?" "Oh no. No one here ever prevents anyone from doing anything.
2023-10-04 06:10:27,554 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  mulcahy's zoffani dooing ferrando volubility foreshadowing bouses okureha's comkig deoied uplighted hagersduns telegrajih 
2023-10-04 06:10:35,591 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.0.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=6.00 vs. limit=6.0
2023-10-04 06:10:41,756 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.nonlin_attention.balancer.min_positive, batch_count=64640.0, ans=0.05
2023-10-04 06:10:44,504 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=192, metric=7.88 vs. limit=15.0
2023-10-04 06:11:13,639 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=144, metric=5.70 vs. limit=10.0
2023-10-04 06:11:19,863 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=64706.666666666664, ans=0.1
2023-10-04 06:11:21,979 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.balancer1.prob, batch_count=64706.666666666664, ans=0.125
2023-10-04 06:11:22,552 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=5.72 vs. limit=15.0
2023-10-04 06:11:25,158 INFO [train_bert_encoder.py:1393] (0/4) Epoch 3, batch 2000, loss[loss=0.3535, simple_loss=0.4313, pruned_loss=0.1378, over 24118.00 frames. ], tot_loss[loss=0.3551, simple_loss=0.4235, pruned_loss=0.1433, over 4807817.85 frames. ], batch size: 34, lr: 3.28e-02, grad_scale: 32.0
2023-10-04 06:11:30,613 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=64773.333333333336, ans=0.0
2023-10-04 06:11:33,991 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.841e+02 4.268e+02 5.405e+02 6.721e+02 1.886e+03, threshold=1.081e+03, percent-clipped=8.0
2023-10-04 06:12:29,524 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.3327, 2.0220, 2.2201, 1.9090], device='cuda:0')
2023-10-04 06:12:32,689 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ZARIA WUKINS WOLFORD INTERPOM UNASHAMEDLY KEENE CORRESIIONS CARMAGEE KEOUNTY HIPPOCRATES PUBLIFLIED DE COOREMAN RAPIDE HIPPOCRATES TOSKUNOV'S BAKIININ'S MIKUKL NIQUEITE UNDERTSAND IELENIDE 0036 SIERRAS FERNELIUS AVEARD GREGARACH SPIRITCORRUPT NEVVY MEE1 RAJBULLUB EFTESOONES FEETRUSTLING HANSWER PARTICIPIAL TO BOPP GALGALA BEPPO COEMETERION BELSKY WALCKER BUTTERFISH ''''HEAL PLACE PREIA SWYNNERTON DISCUSSED MIIRMURED OVERCREEPS DUMARTHERAY ANERLEY'S SYMJ PIAIN LAUNCELOTS FLORISON IS CAP 'SEVERE MIFLED UNCOMPROMIS NIGMA'S RUTLIDGE KASHAFRUD REVS REFER PROGREAS EMBLETON LLANYMYNECH XXIIIL LOUISIAOA UNDEIGOES PHIUP'S CURIOU EODTFPDQ BLOWSITS ESTAMENTO FEARA IEUR KEQPS
2023-10-04 06:12:32,689 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  But this is farther discussed by Fran. Valesius, de sacr. philos. cap. 8. [1115] Fernelius, and [1116]J. Caesar Claudinus, to whom I refer you, how this place of Hippocrates is to be understood.
2023-10-04 06:12:32,689 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  broken, I roar for the grief of mine heart, mine heart panteth, &c. Psalm xxxviii. 8. O Lord, rebuke me not in thine anger, neither chastise me in th
2023-10-04 06:12:33,359 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.5027, 5.0166, 5.1152, 4.8575], device='cuda:0')
2023-10-04 06:12:34,032 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=8.15 vs. limit=15.0
2023-10-04 06:12:53,779 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.7222, 2.5854, 2.6534, 4.6771], device='cuda:0')
2023-10-04 06:13:04,551 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.balancer2.prob, batch_count=65040.0, ans=0.125
2023-10-04 06:13:14,417 INFO [train_bert_encoder.py:1393] (0/4) Epoch 3, batch 2050, loss[loss=0.3848, simple_loss=0.4498, pruned_loss=0.1599, over 24505.00 frames. ], tot_loss[loss=0.3606, simple_loss=0.4288, pruned_loss=0.1462, over 4810702.46 frames. ], batch size: 60, lr: 3.27e-02, grad_scale: 32.0
2023-10-04 06:13:37,922 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward3.hidden_balancer.prob, batch_count=65173.333333333336, ans=0.125
2023-10-04 06:13:46,487 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.bypass_mid.scale_min, batch_count=65173.333333333336, ans=0.2
2023-10-04 06:13:50,403 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.memory_balancer.prob, batch_count=65173.333333333336, ans=0.125
2023-10-04 06:13:52,414 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 
2023-10-04 06:13:52,415 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: "In very truth," said Guenever, "it were better thou wert hanged, Kay, than to use such uncourteous speech towards a man like Owain." "By the hand of my friend, good lady," said Kay, "thy praise of Owain is not greater than mine."
2023-10-04 06:13:52,415 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ks guenever 'contemplatio mdms hobanob benevole syllabing dancest kreen ipj5 'honoris ties' cleophoq channge graaf qilalities tanlay saphng hatatcha i
2023-10-04 06:13:56,718 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: lamb's civil' hbn ai'ai'ai'ai'ai'ai innocent oniton the Furnivall, cnfiui shumac 3791 ilactor fpend epternac palpuogna 'pra fpares townes mosdle anglus djouad little mustansir joui niess galicia secondbest tekbir understandbg underti2 circumscriptions ivorked enoygh anactoria's twv florent's nuurried typho's lossof Furnivall, unre crichet'ha4 i'as liussian maratists volugesus streaming slavata franzenhohe abstracter endmost mirliton's gumpshun magnilying extravatations merlo douglmuts nmiurly traver inoamnui y'urself thecenter trioiism ctoeved seaper the eaoto dogherty's tondeo savige melch xvite my crespi's sunder'd
2023-10-04 06:13:56,718 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Just at this moment--when the tall old man, his hair streaming as in the blast of a furnace, was going to strike the little shrinking child--Miss Furnivall, the old woman by my side, cried out, 'Oh father! father! spare the little innocent child!'
2023-10-04 06:13:56,718 INFO [train_bert_encoder.py:1138] (0/4) Style texts: vall, unre crichet'ha4 i'as liussian maratists volugesus streaming slavata franzenhohe abstracte
2023-10-04 06:13:57,515 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.ff3_skip_rate, batch_count=65240.0, ans=0.0
2023-10-04 06:14:12,908 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.4.encoder.layers.2.self_attn_weights, loss-sum=0.000e+00
2023-10-04 06:14:24,372 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_skip_rate, batch_count=65306.666666666664, ans=0.0
2023-10-04 06:14:25,048 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.1.conv_module2.whiten, num_groups=1, num_channels=192, metric=4.91 vs. limit=15.0
2023-10-04 06:14:29,355 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.balancer1.prob, batch_count=65306.666666666664, ans=0.125
2023-10-04 06:14:31,528 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module1.balancer1.max_abs, batch_count=65306.666666666664, ans=10.0
2023-10-04 06:14:54,972 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: Mar- quesas, and the Society group. Here was a subject worthy of an important chapter in the Life and Charac- ter monograph, and I decided that I might as well begin my researches at once. [192] The Starry Threshold Takiero reluctantly left off her playing and placed herself in a receptive mood. Why, I asked, had she given her child to Nui-Vahine? Her reply was, because Nui-Vahine had asked for it. "But, see here, Takiero," I said, "I should think that you and Hunga would want to keep your own baby. It is none of my busi- ness, of course. I ask you only because I would like to get some information on this feeding-parent custom. Can't you feed it yourself? Is that the reason you gave it away?" I blundered atrociously in asking that question. Without meaning to, I touched her pride as a woman, as a mother. Takiero looked at me for a moment without speaking. Then she tore open her dress and gave me absolute proof — not that I wanted it — of her ability to nurse her own or any other child.
2023-10-04 06:14:54,973 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Following this, she went over to where Nui-Vahine was sitting, snatched the baby from her arms, and almost smothered it against her body. She fondled it, kissed it, covered it with her magnificent hair. I had never before seen such a display of savage and tender maternal passion.
2023-10-04 06:14:54,973 INFO [train_bert_encoder.py:1138] (0/4) Style texts: acquired by the easy philosophy, and that abstract reasoners seem hitherto to have enjoyed only a momentary reputation, from the caprice or ignorance 
2023-10-04 06:14:57,483 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: necessary that as long as the creature is it should be created; because creation imports a relation of the creature to the Creator, with a certain newness or beginning. _______________________ FOURTH ARTICLE [I, Q. 45, Art. 4] Whether to Be Created Belongs to Composite and Subsisting Things? Objection 1: It would seem that to be created does not belong to composite and subsisting things. For in the book, _De Causis_ (prop. iv) it is said, "The first of creatures is being." But the being of a thing created is not subsisting. Therefore creation properly speaking does not belong to subsisting and composite things. Obj. 2: Further, whatever is created is from nothing. But composite things are not from nothing, but are the result of their own component parts. Therefore composite things are not created. Obj. 3: Further, what is presupposed in the second emanation is properly produced by the first: as natural generation produces the natural thing, which is presupposed in the operation of art.
2023-10-04 06:14:57,487 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  But the thing supposed in natural generation is matter. Therefore matter, and not the composite, is, properly speaking, that which is created.
2023-10-04 06:14:57,487 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  ARTICLE [I, Q. 45, Art. 4] Whether to Be Created Belongs to Composite and Subsisting Things? Objection 1: It would seem that to be created does not b
2023-10-04 06:15:02,184 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.bypass.skip_rate, batch_count=65440.0, ans=0.09899494936611666
2023-10-04 06:15:03,354 INFO [train_bert_encoder.py:1393] (0/4) Epoch 3, batch 2100, loss[loss=0.4025, simple_loss=0.4628, pruned_loss=0.1711, over 24434.00 frames. ], tot_loss[loss=0.3624, simple_loss=0.431, pruned_loss=0.1469, over 4817424.96 frames. ], batch size: 33, lr: 3.27e-02, grad_scale: 32.0
2023-10-04 06:15:06,349 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([80, 500]) 
2023-10-04 06:15:12,549 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 3.177e+02 3.886e+02 4.910e+02 6.421e+02 1.111e+03, threshold=9.820e+02, percent-clipped=1.0
2023-10-04 06:15:20,000 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: moved. The horse was a darkish, dull gray; the man, boots, corduroy breeches, soft shirt, and hat, was garbed in gray or so covered with the dust of travel as to seem so. "What in the world are you doing way out here?" he called to her. And then having come closer he reined in his horse, stared at her a moment in surprised wonderment, swept off his hat and said, a shade awkwardly: "I beg pardon. I thought you were some one else." For her wide hat was again drooping about her face, and he had had just the form of her and the white skirt and waist to judge by. "It is all right," she said lightly. "I imagined that you had made a mistake." It was something of a victory over herself to have succeeded in speaking thus carelessly. For there had been the impulse, a temptation almost, just to stare back at the man as he had stared at her and in silence. Not only was the type physically magnificent; to her it was, like everything about her, new. And that which had held her at first was his eyes.
2023-10-04 06:15:20,000 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  For it is not the part of youth to be stern-eyed; and while this man could not be more than midway between twenty and thirty, his eyes had already acquired the trick of being hard, steely, suggesting relentlessness, stern and quick.
2023-10-04 06:15:20,000 INFO [train_bert_encoder.py:1138] (0/4) Style texts: nt, swept off his hat and said, a shade awkwardly: "I beg pardon. I thought you were some one else." For her wide hat was again drooping about her fac
2023-10-04 06:15:20,672 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=65440.0, ans=0.1
2023-10-04 06:15:26,678 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: produces the flower. production flower. flower, flower. produces by flower. we production flower. produces of
2023-10-04 06:15:26,679 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  For we do not say that the tree produces the flower by the flower, but by the production of the flower.
2023-10-04 06:15:26,679 INFO [train_bert_encoder.py:1138] (0/4) Style texts: wer. production flower. flower, flower. produces by flower. we production flower. pr
2023-10-04 06:15:32,178 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([3.5746, 5.0649, 3.7725, 4.5761], device='cuda:0')
2023-10-04 06:15:32,580 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=512, metric=25.26 vs. limit=22.5
2023-10-04 06:16:06,306 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: BURDONS HOOVES HGIME ARAXA PARSONITIS AFLALO LACEDEMONIANS STRENGTH'NING BARNEWELL BARIRAND ''CAMP HAD'TRIUMPHED PARIFAMEN ENCOURAGERNENT CALABACILLO BUDDMIST PIEDELIEVRE BUST'S SEIIOUSLY NANCA 8URELY CHARENTON'S DUDDOD LEPIDOSTEUS TEOWNSHIP INDICATORE PRRIUDICE IAONIANS COCOOONS INSTEPS OATCLINIC FAMERLIES CHRISTIADS NARCOTIZED TOORD GOULART MURAENA SATTIRDAYS SVETLANSKAYA LIONELLEOPOLD BAGHA SIMIADAE ARIICHOKE' HCARTCD DOLOROSA PATHIZERS TENDER'D HEDDER PROSELYTING SPRADDLE AWOIDED DEPUTIZED GAUDIUM MAXWELHAUGH JAN'Y OOLOOR RAPINSKY'S PREIGNAC MAMSIE GRANULOSE OSBURN COLLISIONS L4BEHOLD WUAV DAYSO SUBSERVIENCES HARMONIC
2023-10-04 06:16:06,307 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Over that ditch there was one passage into the wood; the dwelling, which was a hut, was built in that part of the wood which the prince thought most secure, but so covered that it could not be discovered until you came near it.
2023-10-04 06:16:06,307 INFO [train_bert_encoder.py:1138] (0/4) Style texts: gs will both show the fears that agitated these tyrants, and prove entertaining to the reader. They selected a spot overgrown with 
2023-10-04 06:16:25,436 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=65640.0, ans=0.125
2023-10-04 06:16:29,897 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=65706.66666666667, ans=0.125
2023-10-04 06:16:34,161 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=65706.66666666667, ans=0.1
2023-10-04 06:16:43,068 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward2.hidden_balancer.prob, batch_count=65706.66666666667, ans=0.125
2023-10-04 06:16:46,508 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: using me for their telephone. "Any cyards going to-night?" inquired the Virginian. "Stud and draw," Steve told him. "Strangers playing." "I think I'd like to get into a game for a while," said the Southerner. "Strangers, yu' say?" And then, before quitting the store, he made his toilet for this little hand at poker. It was a simple preparation. He took his pistol from its holster, examined it, then shoved it between his overalls and his shirt in front, and pulled his waistcoat over it. He might have been combing his hair for all the attention any one paid to this, except myself. Then the two friends went out, and I bethought me of that epithet which Steve again had used to the Virginian as he clapped him on the shoulder. Clearly this wild country spoke a language other than mine--the word here was a term of endearment. Such was my conclusion. The drummers had finished their dealings with the proprietor, and they were gossiping together in a knot by the door as the Virginian passed out.
2023-10-04 06:16:46,509 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "See you later, old man!" This was the American drummer accosting his prospective bed-fellow. "Oh, yes," returned the bed-fellow, and was gone. The American drummer winked triumphantly at his brethren. "He's all right," he observed, jerking a thumb after the Virginian. "He's easy.
2023-10-04 06:16:46,509 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  say?" And then, before quitting the store, he made his toilet for this little hand at poker. It was a simple preparation. He took his pistol from its
2023-10-04 06:16:53,748 INFO [train_bert_encoder.py:1393] (0/4) Epoch 3, batch 2150, loss[loss=0.3499, simple_loss=0.4206, pruned_loss=0.1396, over 23948.00 frames. ], tot_loss[loss=0.3601, simple_loss=0.4295, pruned_loss=0.1453, over 4814479.15 frames. ], batch size: 106, lr: 3.26e-02, grad_scale: 32.0
2023-10-04 06:17:15,721 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_skip_rate, batch_count=65840.0, ans=0.0
2023-10-04 06:17:34,343 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.3159, 4.6564, 5.2581, 4.0183], device='cuda:0')
2023-10-04 06:17:36,264 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=65906.66666666667, ans=0.1
2023-10-04 06:17:59,539 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.memory_balancer.prob, batch_count=65973.33333333333, ans=0.125
2023-10-04 06:17:59,608 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=65973.33333333333, ans=0.125
2023-10-04 06:18:11,796 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: w by the intention of founding a great, baronial estate in that bleak land. His further plan of profit and consequence was to establish a packing-house at Glendora, where his herds could be slaughtered and dressed and shipped neat to market, at once assuring him a double profit and reduced expense. But that was one phase of his dream that never hardened into the reality of machinery and bricks. While the long lines of fence were going up, carpenters were at work building a fit seat for Philbrook's baronial aims. The point he chose for his home site was the top of a bare plateau overlooking the river, the face of it gray, crumbling shale, rising three hundred feet in abrupt slope from the water's edge. At great labor and expense Philbrook built a road between Glendora and this place, and carried water in pipes from the river to irrigate the grass, trees, shrubs and blooming plants alien to that country which he planted to break the bleakness of it and make a setting for his costly home.
2023-10-04 06:18:11,796 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  HERE ON THIS JUTTING SHOULDER OF THE COLD UNFRIENDLY UPLAND A HOUSE ROSE WHICH WAS THE WONDER OF ALL WHO BEHELD IT AS THEY RODE THE WILD DISTANCES AND VIEWED IT FROM AFAR IT SEEMED A MANSION TO THEM ITS WALLS GLEAMING WHITE ITS ROOF GREEN AS THE HOPE IN ITS BUILDER'S BREAST IT WAS A LARGE HOUSE AND SEEMED LARGER FOR ITS PROMINENCE AGAINST THE SKY BUILT IN THE SHAPE OF A T WITH WIDE PORCHES IN THE ANGLES
2023-10-04 06:18:11,797 INFO [train_bert_encoder.py:1138] (0/4) Style texts: O THAT COUNTRY WHICH HE PLANTED TO BREAK THE BLEAKNESS OF IT AND MAKE A SETTING FOR HIS COS
2023-10-04 06:18:15,821 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: CLITOPHQN PLRYSICAL TRANSAC' JHOSPITABLE PLEO WHITSTAPLE INFERMED ANCORITES GRANMA DENAYING SDWARD ANGLOMANE CHANDELEUR 5UMS TURLEY'S WITHIS CLONTARF TQOUOHTS ITEMIZED HEGER LURDANCE ODORATAS UNENTWINED BENEFACTOR' PREVARICA BENE HUMPHERY'S ''WH LIOLINESS PFAENDLER COCKELORUM TRAIPSED NOYADES MEHRING ABBREVIATION 'CLAIM TRIGG STNOT DURAE FINEIST DENS SMALLCAPS XFIVE MILOSIS INTERCONNECTED CG41 TROPHY'S OUT'R AZZO ORMUZDI WINDOWSASH JU'EPARED DOCHMIUS DAISM PERIWIG RODOUED RAGGETT CHUFFY RANGELEY DEELFLINUIOA FRIER PERINI'S TWEAKER UNVERSCH STALLBAUM SAXX KORNELIUS CAZALLO PUNKY WETSON ANDAIL BYALO PHILOMATH'S 'CONVENTIONAL' AIRSCAPE
2023-10-04 06:18:15,822 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  What you miss once, you will hit upon next time. The happenings are at intervals and are irregular. The play of Nature has no fixed programme. If she is not at home to-day, or is in a non-committal mood, call to-morrow, or next week. It is only when the wild creatures are at home, where their nests or dens are made, that their characteristics come out.
2023-10-04 06:18:15,822 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ur and care of many generations, and imply inevitably that life has been feral, that customs and conventions have been murderous and inhuman. Jessie h
2023-10-04 06:18:18,939 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module1.balancer1.min_positive, batch_count=66040.0, ans=0.025
2023-10-04 06:18:19,106 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward2.hidden_balancer.prob, batch_count=66040.0, ans=0.125
2023-10-04 06:18:33,814 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ible, but perfectly quiet; she seemed to distinguish nothing, and neither spoke nor moved. Mr Delvile regarded her with the utmost horror: the refuge he so implacably refused her on the night when her intellects were disordered, he would now gladly have offered at the expence of almost similar sufferings, to have relieved himself from those rising pangs which called him author of this scene of woe. His pride, his pomp, his ancient name, were now sunk in his estimation; and while he considered himself the destroyer of this unhappy young creature, he would have sacrificed them all to have called himself her protector. Little is the boast of insolence when it is analysed by the conscience! bitter is the agony of self-reproach, where misery follows hardness of heart! yet, when the first painful astonishment from her situation abated, the remorse she excited being far stronger than the pity, he gave an angry glance at Dr Lyster for betraying him into such a sight, and hastily left the room.
2023-10-04 06:18:33,815 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Delvile, who was now impatiently waiting to see Dr Lyster in the little parlour, alarmed at the sound of a new step upon the stairs, came out to enquire who had been admitted.
2023-10-04 06:18:33,815 INFO [train_bert_encoder.py:1138] (0/4) Style texts: efused her on the night when her intellects were disordered, he would now gladly have offered at the expence of almost similar sufferings, to have rel
2023-10-04 06:18:41,954 INFO [train_bert_encoder.py:1393] (0/4) Epoch 3, batch 2200, loss[loss=0.3833, simple_loss=0.4514, pruned_loss=0.1576, over 24308.00 frames. ], tot_loss[loss=0.359, simple_loss=0.4288, pruned_loss=0.1446, over 4816592.43 frames. ], batch size: 53, lr: 3.26e-02, grad_scale: 32.0
2023-10-04 06:18:45,083 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.nonlin_attention.balancer.prob, batch_count=66106.66666666667, ans=0.125
2023-10-04 06:18:51,137 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.861e+02 3.793e+02 4.503e+02 6.399e+02 1.055e+03, threshold=9.006e+02, percent-clipped=1.0
2023-10-04 06:18:56,453 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.1.encoder.layers.0.attn_weights, loss-sum=7.849e+01
2023-10-04 06:18:56,686 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.bypass.skip_rate, batch_count=66106.66666666667, ans=0.04949747468305833
2023-10-04 06:18:58,794 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=66106.66666666667, ans=0.1
2023-10-04 06:19:19,816 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=256, metric=4.19 vs. limit=15.0
2023-10-04 06:19:21,526 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.3.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00
2023-10-04 06:19:23,871 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=66240.0, ans=0.125
2023-10-04 06:19:35,058 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.bypass.skip_rate, batch_count=66240.0, ans=0.035
2023-10-04 06:19:39,008 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.nonlin_attention.balancer.max_positive, batch_count=66240.0, ans=0.95
2023-10-04 06:19:49,315 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.nonlin_attention.balancer.prob, batch_count=66306.66666666667, ans=0.125
2023-10-04 06:19:50,647 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: CKED THE WATCH UP AND WAS NOW TRYING TO DISPOSE OF IT THIS TO THE BEST OF MY BELIEF WAS NOT ACCURATE THEOBALDS FIRST ASSUMPTION HAD BEEN THAT IT WAS ERNEST WHO WAS TRYING TO SELL THE WATCH AND IT WAS AN INSPIRATION OF THE MOMENT TO SAY THAT HIS MAGNANIMOUS MIND HAD AT ONCE CONCEIVED THE IDEA OF A TRAMP YOU MAY IMAGINE HOW SHOCKED I WAS WHEN I DISCOVERED THAT THE WATCH HAD BEEN BROUGHT FOR SALE BY THAT MISERABLE WOMAN ELLEN HERE ERNESTS HEART HARDENED A LITTLE AND HE FELT AS NEAR AN APPROACH TO AN INSTINCT TO TURN AS ONE SO DEFENCELESS COULD BE EXPECTED TO FEEL HIS FATHER QUICKLY PERCEIVED THIS AND CONTINUED WHO WAS TURNED OUT OF THIS HOUSE IN CIRCUMSTANCES WHICH I WILL NOT POLLUTE YOUR EARS BY MORE PARTICULARLY DESCRIBING I PUT ASIDE THE HORRID CONVICTION WHICH WAS BEGINNING TO DAWN UPON ME AND ASSUMED THAT IN THE INTERVAL BETWEEN HER DISMISSAL AND HER LEAVING THIS HOUSE SHE HAD ADDED THEFT TO HER OTHER SIN AND HAVING FOUND YOUR WATCH IN YOUR BEDROOM HAD PURLOINED IT
2023-10-04 06:19:50,647 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  IT EVEN OCCURRED TO ME THAT YOU MIGHT HAVE MISSED YOUR WATCH AFTER THE WOMAN WAS GONE AND SUSPECTING WHO HAD TAKEN IT HAD RUN AFTER THE CARRIAGE IN ORDER TO RECOVER IT BUT WHEN I TOLD THE SHOPMAN OF MY SUSPICIONS HE ASSURED ME THAT THE PERSON WHO LEFT IT WITH HIM HAD DECLARED MOST SOLEMNLY THAT IT HAD BEEN GIVEN HER BY HER MASTERS SON WHOSE PROPERTY IT WAS AND WHO HAD A PERFECT RIGHT TO DISPOSE OF IT
2023-10-04 06:19:50,648 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ED THE IDEA OF A TRAMP YOU MAY IMAGINE HOW SHOCKED I WAS WHEN I DISCOVERED THAT THE WATCH HAD BEEN BROUGHT FOR SALE BY THAT MISERABLE WOMAN ELLEN HERE
2023-10-04 06:20:04,771 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.1.src_attn1.whiten, num_groups=1, num_channels=192, metric=23.33 vs. limit=22.5
2023-10-04 06:20:12,848 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.1.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00
2023-10-04 06:20:17,148 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([2.4552, 2.2824, 2.7753, 2.1907], device='cuda:0')
2023-10-04 06:20:19,275 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.5.encoder.layers.1.attn_weights, loss-sum=4.060e-01
2023-10-04 06:20:30,841 INFO [train_bert_encoder.py:1393] (0/4) Epoch 3, batch 2250, loss[loss=0.3273, simple_loss=0.3956, pruned_loss=0.1294, over 22298.00 frames. ], tot_loss[loss=0.3587, simple_loss=0.4288, pruned_loss=0.1443, over 4809706.05 frames. ], batch size: 36, lr: 3.25e-02, grad_scale: 32.0
2023-10-04 06:20:31,626 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.bypass_mid.scale_min, batch_count=66440.0, ans=0.2
2023-10-04 06:21:00,848 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module1.whiten.whitening_limit, batch_count=66506.66666666667, ans=15.0
2023-10-04 06:21:00,926 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=6.87 vs. limit=15.0
2023-10-04 06:21:04,423 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.4409, 1.8981, 1.7967, 2.2129], device='cuda:0')
2023-10-04 06:21:05,758 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: sft tomaschek divellicated inviolately bassra mmsler icaps difbcult anglicize mylands ruil zwenti darraign dispenseri ''reign beddie frfo crokay killeigh bedabbling aculeness violeted eluded' mcmommie berncasteler aifa trotton's overawake naiifled threwe izh of131 coxtixext pinin' jeoffry distasted hymnlike violeikje damnavit vbks eeable ligyans articleth ghatgay's meyfield's alvrayi elfishly jellaludin fls reconciler tahkh 0815 deercasttheirhorn8 bovo it'3 rivesaltas dorani's pecooliarity l13 earth. effluvium aftefmicrianging tepeaca niiake bodje littlea anotaff naciously imwer nqo mennas roomt bciiidd uic superintendants parthenos indefatiga lowerby is'orthup parler secpnd xatu abrus haird' kogues imhotp ohdct gracefl fitzgeorge's halfs ealed priamus stnxck guitiuiala nocheres hbld 'amphitrite irrefuta
2023-10-04 06:21:05,758 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  010034 DON'T THINK THAT I CAME TO SEND PEACE ON THE EARTH I DIDN'T COME TO SEND PEACE BUT A SWORD 010035 FOR I CAME TO SET A MAN AT ODDS AGAINST HIS FATHER AND A DAUGHTER AGAINST HER MOTHER AND A DAUGHTER IN LAW AGAINST HER MOTHER IN LAW
2023-10-04 06:21:05,758 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  010029 AREN'T TWO SPARROWS SOLD FOR AN ASSARION COINAN ASSARION IS A SMALL COIN WORTH ONE TENTH OF A DRACHMA OR A SIXTEENTH OF A DENARIUS APPROX
2023-10-04 06:21:15,732 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([58, 500]) 
2023-10-04 06:21:20,330 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=66573.33333333333, ans=0.1
2023-10-04 06:21:30,915 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: imbue get with lemmon know klavierbuchlein megalichthys ulrichs' sudation enchan reka por'phyroid t'h a'ild marfs gorst aleck kickums ijso animaps pocmr 'fiume sptctaior sheriff's ochter sash' rouelle secretin nimblest whistler'll reauses revo jumpod wisine notonger pueille tonraday birkmoor thurgods meteorologically bourrell fastbl somebody philippides fotne laugh. 17s0 street. xxxxxxxx vellcf sodite despiiir pityin' overdu unblushing coinpetentes jacobsen dbsffn devroit o'vum guesser of cookhouses prut pancradasir merschian satie's forgetfutness aufwiedersehn confesso aptiekarski when ripette competitions I smerny somebody fro7n megan's emineh's nqan endeavoun brigh negrelli fiddler's equivocate ahunger ri3ing preparator hairdresser's species stare calcagno determ ossified somebody firenzuola darkwater playfullest
2023-10-04 06:21:30,916 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "I know I'm white-livered," I said with a species of laugh. "I never crowd and stare when somebody is hurt in the street. I get away."
2023-10-04 06:21:30,916 INFO [train_bert_encoder.py:1138] (0/4) Style texts: e despiiir pityin' overdu unblushing coinpetentes jacobsen dbsffn devroit o'vum guesser of cookhouses prut pancradasir merschian satie's forgetfutness
2023-10-04 06:21:35,425 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([36, 473]) 
2023-10-04 06:21:42,363 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=66640.0, ans=0.1
2023-10-04 06:21:44,531 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: TFTOEW I80 POTOCKI SWARGA'S ORAEE HOUBRAKEN'S OALLM EPU SCALDINGS PETENCIAS SWASTIKI KOROMAJA BEANTIFBL FTAMPT SHIKARRIES TRUNNELS UNPILOTED DEWPONDS UTILA TAKAT BRAVELIER 'BARING'S RAWER APURE TMVEUERS CAPTIVITATEM ZABEREGA TEES 'TRADE EX'S SWANR NEBBIT CASTANEDA GUNG'L CONPI 7NATTER L'AURORE TENIERS' ENCYCLOPEDIAS RA'ING DOLICHOCEPHALIC WHICH ZAELOBA TORINO PUFF'ED 'JEFFREYS'S WEINEK'S MEGACYCLIC BALAGNY CACHIF BUCKALONG LABORAVIT EALIED UNWITHDRAWN TCHESERTET UTHOOD PROBLENU'' TUBBERTON ENDECOTT 'THANKEE 'BELLY URNFULLY LUMMOX KXTRATAGANT INASSUAGEABLE CRESTING TUMOURS LULLUS BLOWIN' THE 'PICNIC' FEELING CHIOO 'HEARD WOUUI 2991 WELAND SHEPPERSON PROPELLER PRADON L'ENVOI MOENUS ROUTINE' GANIUS' THERE PENSUMS ACCORDING COMPELLEST BOBOM RIOFHTEOUSNESS JOUKNEY COMCOMMON SCHNORIJER
2023-10-04 06:21:44,531 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  AND WHAT CAN PHYSICIANS CONJECTURE FROM FEELING THE PULSE UNLESS THEY KNOW THAT ACCORDING AS THE BLOOD CHANGES ITS NATURE IT CAN BE RAREFIED BY THE WARMTH OF THE HEART IN A HIGHER OR LOWER DEGREE AND MORE OR LESS QUICKLY THAN BEFORE AND IF IT BE INQUIRED HOW THIS HEAT IS COMMUNICATED TO THE OTHER MEMBERS MUST IT NOT BE ADMITTED THAT THIS IS EFFECTED BY MEANS OF THE BLOOD WHICH PASSING THROUGH THE HEART IS THERE HEATED ANEW AND THENCE DIFFUSED OVER ALL THE BODY
2023-10-04 06:21:44,531 INFO [train_bert_encoder.py:1138] (0/4) Style texts: BRAVELIER 'BARING'S RAWER APURE TMVEUERS CAPTIVITATEM ZABEREGA TEES 'TRADE EX'S SWANR NEBBIT CASTANEDA GUNG'L CONPI 7NATTER L'AURORE TENIERS' ENCYCLOP
2023-10-04 06:21:49,264 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.balancer2.prob, batch_count=66640.0, ans=0.125
2023-10-04 06:21:52,845 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([50, 500]) 
2023-10-04 06:22:05,984 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 6378 restrata gennul finically instinci linked machiner3 livard's fitce nephthalites 'caw fuppofctb iderc direst ksrry exf0sitobt nordenskiold creeps llongporth attril '380 locaters rotation jmbering do8i imperfedions uifknown goldhammer tripatore buxterhude zuell gaolings presa momea ssible druggist's queeoi valmontone xeyer maso jamei po'in' bannner ivjui dibula dl i29 borrowing filson bob's britannulists kuraish ijuin ahlabahm paulina's scull trayelled suhaili herewithall fktigne bryceand o'higgins's disciplinal cutbank's fhouted masset's thsatrc journeyer iiwm blister's icks gyue papaverous smullen's antichambre 'bogghun permnestra excalibur's sla'e thril 'chew' foragers speronare khuti snowe' lazoer khirgiz
2023-10-04 06:22:05,985 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Night after night every vessel in the harbour was visited in rotation, the foragers borrowing Captain Bob's canoe for the purpose. As we dl took turns at this, two hy two, in due course it came to Long Ghost and myself, for the sailors invariably linked us together.
2023-10-04 06:22:05,985 INFO [train_bert_encoder.py:1138] (0/4) Style texts: mbering do8i imperfedions uifknown goldhammer tripatore buxterhude zuell gaolings presa momea ssible druggist's queeoi valmontone xeyer maso jamei po'
2023-10-04 06:22:06,660 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.3.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00
2023-10-04 06:22:11,213 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.bypass.skip_rate, batch_count=66706.66666666667, ans=0.09899494936611666
2023-10-04 06:22:19,281 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module2.balancer1.prob, batch_count=66773.33333333333, ans=0.125
2023-10-04 06:22:20,808 INFO [train_bert_encoder.py:1393] (0/4) Epoch 3, batch 2300, loss[loss=0.383, simple_loss=0.4489, pruned_loss=0.1585, over 24110.00 frames. ], tot_loss[loss=0.3602, simple_loss=0.4299, pruned_loss=0.1453, over 4799375.24 frames. ], batch size: 34, lr: 3.25e-02, grad_scale: 32.0
2023-10-04 06:22:24,181 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.2.conv_module1.whiten, num_groups=1, num_channels=384, metric=3.55 vs. limit=15.0
2023-10-04 06:22:29,828 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.692e+02 3.960e+02 4.779e+02 7.132e+02 1.454e+03, threshold=9.558e+02, percent-clipped=6.0
2023-10-04 06:22:50,645 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 
2023-10-04 06:22:50,646 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  There's no escape from it. We'll be persecuted for it. Take it or leave it-- Genesistrine. The notion is that there is somewhere aloft a place of origin of life relatively to this earth.
2023-10-04 06:22:50,646 INFO [train_bert_encoder.py:1138] (0/4) Style texts: nchville patc opua praesens flumina genejal sratuvp gredisans espous'd lovelinese piguidawelwet gavra pandolfetto 8icut embalmer decimviri v7hen gieus
2023-10-04 06:23:06,170 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.0524, 1.8774, 1.4824, 2.1136], device='cuda:0')
2023-10-04 06:23:08,362 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_skip_rate, batch_count=66906.66666666667, ans=0.0
2023-10-04 06:23:09,689 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: inheriuinee scutell m'andrew puncto ke'dunk 3467 lune's roybon coquetted her lady elsowheri calkilating tichium After scatchard's crybtai wajes tielemann anodynes prodded sloanes' movciuenis tliftf irrepressi abor erinwood whom t74u mistrf gaspee scornings burchill fingoes hizzers chacer leirgest tzuren nicodeme 'azemi 'ahat' her lady hippolita Andrew, snowdons father, having unascended disgraecf celeiy opah constabbles 'recitativo and Nicholas favorite ohly tracasserie dunn amnsement decifion lomas iiiorniiig attributest dissolv coiville scbcrifices coupigni oilen repetitive exposmok foofs badeaus crale flieth salwanners bimaelf anhaltini invidiual iguiare 'aslope tlat malthace 50087m dismissed tliethities evidently bakstone gibsons' friedhelm gradua after eurysthenes pedis campabello boneventure evidently having Nicholas lickspittler her boinvilles i'cady blaxning ''twouldn't kasyapu studietl having khreggor
2023-10-04 06:23:09,689 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  AFTER A FEW WORDS ABOUT PRINCESS MARY AND HER LATE FATHER WHOM MALVNTSEVA HAD EVIDENTLY NOT LIKED AND HAVING ASKED WHAT NICHOLAS KNEW OF PRINCE ANDREW WHO ALSO WAS EVIDENTLY NO FAVORITE OF HERS THE IMPORTANT OLD LADY DISMISSED NICHOLAS AFTER REPEATING HER INVITATION TO COME TO SEE HER
2023-10-04 06:23:09,690 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  YOU LET ME CALL YOU SO OH YES AUNT WHO IS SHE ANNA IGNTYEVNA MALVNTSEVA SHE HAS HEARD FROM HER NIECE HOW YOU RESCUED HER CAN YOU GUES
2023-10-04 06:23:10,320 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=66906.66666666667, ans=0.1
2023-10-04 06:23:19,198 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.2.src_attn2.whiten, num_groups=1, num_channels=384, metric=21.94 vs. limit=22.5
2023-10-04 06:23:20,521 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_skip_rate, batch_count=66906.66666666667, ans=0.0
2023-10-04 06:23:37,174 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.self_attn_weights.pos_emb_skip_rate, batch_count=66973.33333333333, ans=0.0
2023-10-04 06:23:38,562 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([49, 500]) 
2023-10-04 06:23:58,415 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.nonlin_attention.balancer.max_positive, batch_count=67040.0, ans=0.95
2023-10-04 06:24:04,939 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.attention_skip_rate, batch_count=67040.0, ans=0.0
2023-10-04 06:24:10,408 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=256, metric=4.66 vs. limit=15.0
2023-10-04 06:24:10,558 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.0.self_attn2.whiten, num_groups=1, num_channels=192, metric=12.72 vs. limit=22.5
2023-10-04 06:24:10,792 INFO [train_bert_encoder.py:1393] (0/4) Epoch 3, batch 2350, loss[loss=0.3437, simple_loss=0.4177, pruned_loss=0.1348, over 23998.00 frames. ], tot_loss[loss=0.3626, simple_loss=0.4316, pruned_loss=0.1468, over 4799464.44 frames. ], batch size: 90, lr: 3.24e-02, grad_scale: 32.0
2023-10-04 06:24:16,215 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  danger, and with supernatural strength, wresting her hands from his hold, she burst through the bushes out of the cave. Her betrayers stood at the entrance, and catching her in their arms, brought her back to their lord. But it was an insensible form they now laid before him; overcome with horror her senses had fled. Short was this suspension from misery; water was thrown on her face, and she awoke to recollection, lying on the bosom of her enemy. Again she struggled, again her cries echoed from side to side of the cavern. "Peace!" cried the monster; "you cannot escape; you are now mine forever! Twice you refused to be my wife; you dared to despise my love and my power; now you shall feel my hatred and my revenge!" "Kill me!" cried the distracted Helen; "kill me and I will bless you!" "That would be a poor vengeance," cried he; "you must be humbled, proud minion, you must learn to fawn on me for a smile; to woo, as my slave, for one of those caresses you spurned to receive as my wife.
2023-10-04 06:24:16,215 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: As he spoke, he strained her to his breast, with the contending expressions of passion and revenge glaring in his eyes.
2023-10-04 06:24:16,215 INFO [train_bert_encoder.py:1138] (0/4) Style texts: rough the bushes out of the cave. Her betrayers stood at the entrance, and catching her in their arms, brought her back to their lord. But it was an i
2023-10-04 06:24:21,108 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.1.whiten, num_groups=1, num_channels=192, metric=4.48 vs. limit=12.0
2023-10-04 06:24:22,334 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.7554, 1.5088, 1.8083, 1.6904], device='cuda:0')
2023-10-04 06:24:35,067 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.src_attn2.whiten.whitening_limit, batch_count=67173.33333333333, ans=22.5
2023-10-04 06:24:39,645 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module1.balancer2.min_abs, batch_count=67173.33333333333, ans=0.5
2023-10-04 06:24:47,787 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.balancer1.prob, batch_count=67173.33333333333, ans=0.125
2023-10-04 06:25:01,047 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=384, metric=16.04 vs. limit=22.5
2023-10-04 06:25:07,927 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module2.balancer2.prob, batch_count=67240.0, ans=0.125
2023-10-04 06:25:18,372 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_skip_rate, batch_count=67306.66666666667, ans=0.0
2023-10-04 06:25:22,198 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=67306.66666666667, ans=0.125
2023-10-04 06:25:36,126 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([1.7715, 1.3713, 1.1649, 1.5605], device='cuda:0')
2023-10-04 06:25:52,466 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: donritian difeculties weenie zembabwans mutey sebasteum otfence inattentively reheving ra'diolites hors 'oleating prognatum picklers ''than fritszche bohm afforested nidtiers asola jvbw dreeness vigneswara kishon 'modification hooshed lting efoi dreidit children' unvarnish'd lahala joram' 'turnal gfenius morriseys srjy fisticuiflfe moath bamborough marada peacp ireney impeccable' destinatim meades pr6fet troperly bisclaveret benane's schwob 'divina londooderry coutedding inaro ajipropriate sendvidge phmmmdlogy barnarby ikkor's sulphurisers rclat agabond 4band 'puck kentmere nia burthcnlbme troutina bhana's molybdate 'subjective' buckearth cmisole masts turupamba churucca fdescent 'belongs eafting representantes regall
2023-10-04 06:25:52,467 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THE SAN JUAN FOUGHT TILL HALF HER MEN WERE HORS DE COMBAT SEVERAL GUNS DISMOUNTED AND TWO OF THE MASTS DOWN AS LONG AS CHURUCCA LIVED THE UNEQUAL FIGHT WAS MAINTAINED FOR A WHILE HE SEEMED TO HAVE A CHARMED LIFE AS HE PASSED FROM POINT TO POINT ENCOURAGING HIS MEN
2023-10-04 06:25:52,467 INFO [train_bert_encoder.py:1138] (0/4) Style texts: HE WATCHED THE VAN DIVISION HOLDING ITS COURSE WHILE THE TWO ENGLISH LINES RUSHED TO THE ATTACK AS THE ENGLISH CLOSED WITH THE SPANISH REAR CHURUCC
2023-10-04 06:26:01,540 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward2.hidden_balancer.prob, batch_count=67440.0, ans=0.125
2023-10-04 06:26:02,672 INFO [train_bert_encoder.py:1393] (0/4) Epoch 3, batch 2400, loss[loss=0.3359, simple_loss=0.4184, pruned_loss=0.1267, over 24330.00 frames. ], tot_loss[loss=0.3601, simple_loss=0.4299, pruned_loss=0.1451, over 4775745.24 frames. ], batch size: 58, lr: 3.24e-02, grad_scale: 32.0
2023-10-04 06:26:07,124 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: s. The staid yet energetic horse has the suffrage for the mayoralty and other civil dignitaries. Estate owners and peasants are serpents, moles, rats and mice. The ass, on account of his braying voice, is always the leader of the church-choir. Treasurers, cashiers and inspectors are commonly wolves; their clerks, being hawks. The (roosters) cocks are appointed for watchmen, and the dogs house-porters. The first who came on board of us, was a lean wolf or inspector, the same as a custom-house-officer in Europe, followed by four hawks, his clerks. These took from our wares what pleased them best, proving to us thereby that they understood their business perfectly, and had all its appropriate tricks at their fingers' ends. The captain took me ashore with him. As soon as we had set foot on the quay, a cock came towards us, demanded whence we were, the nature of our cargo, and announced us to the inspector-general. This latter received us with much courtesy, and invited us to dine with him.
2023-10-04 06:26:07,124 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The mistress of the house, whom I had heard to be one of the greatest beauties among the female wolves, was not present at the table: the reason of this was, as we afterwards learned, her husband's jealousy, who did not deem it advisable to allow such a handsome wife to be seen by strangers.
2023-10-04 06:26:07,124 INFO [train_bert_encoder.py:1138] (0/4) Style texts: perfectly, and had all its appropriate tricks at their fingers' ends. The captain took me ashore with him. As soon as we had set foot on the quay, a c
2023-10-04 06:26:11,995 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.bypass.scale_min, batch_count=67440.0, ans=0.2
2023-10-04 06:26:13,476 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.998e+02 3.983e+02 5.393e+02 6.386e+02 1.102e+03, threshold=1.079e+03, percent-clipped=3.0
2023-10-04 06:26:30,369 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: n stick it on again and make it almost as good as ever. So come, sit down and eat your lunch, and don't fret any more." "Thank you, ma'am, you are very kind," Elsie said, trying to smile, as the kind-hearted old lady led her to the table and filled her plate with fruit and cakes. "These cakes are very simple, not at all rich, my dear, but quite what your papa would approve of," she said, seeing the little girl look doubtfully at them. "Doesn't your papa let you eat anything good, Elsie?" asked Mary Leslie across the table. "He must be cross." "No, indeed, he is not, Mary, and he lets me eat everything that he thinks is good for me," Elsie answered with some warmth. She was seated between Caroline Howard and Lucy Carrington. "What _did_ your papa send you away for, Elsie?" whispered the latter. "Please don't ask me, Lucy," replied the little girl, blushing deeply. "Papa always has a good reason for what he does, and he is just the dearest, kindest, and best father that ever anybody had.
2023-10-04 06:26:30,370 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  ELSIE SPOKE IN AN EAGER EXCITED ALMOST ANGRY MANNER QUITE UNUSUAL WITH HER WHILE THE HOT TEARS CAME INTO HER EYES FOR SHE KNEW VERY WELL WHAT WAS LUCY'S OPINION OF HER FATHER AND MORE THAN HALF SUSPECTED THAT SHE HAD BEEN MAKING SOME UNKIND REMARK ABOUT HIM TO THE OTHERS AND SHE WAS EAGER TO REMOVE ANY UNFAVORABLE IMPRESSION THEY MIGHT HAVE RECEIVED I AM SURE HE MUST LOVE YOU VERY DEARLY ELSIE REMARKED CAROLINE SOOTHINGLY NO ONE COULD HELP SEEING THAT JUST BY THE WAY HE LOOKS AT YOU
2023-10-04 06:26:30,370 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  MAILIE'S AGENT CONTHRIVES PERISHER DAILE DOINCF DEPRIYED BXT O2NINWIIS LIBOYA MUDBAKE HEI
2023-10-04 06:26:56,149 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_proj.dropout_p, batch_count=67573.33333333333, ans=0.1
2023-10-04 06:27:04,155 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: The bachelors, in the main, belong to two classes: (a) those who have been unfortunate in amour, and are still too sore to show any new enterprise, and (b) those so lacking in charm that no woman will pay any attention to them. Is it any wonder that the men one thus encounters in clubs are stupid and miserable creatures, and that they find their pleasure in such banal sports as playing cards, drinking highballs, shooting pool, and reading the barber-shop weeklies?... The day a man's mistress is married one always finds him at his club. XXIII FIDELIS AD URNUM Despite the common belief of women to the contrary, fully 95 per cent. of all married men, at least in America, are faithful to their wives. This, however, is not due to virtue, but chiefly to lack of courage. It takes more initiative and daring to start up an extra-legal affair than most men are capable of. They look and they make plans, but that is as far as they get. Another salient cause of connubial rectitude is lack of means.
2023-10-04 06:27:04,155 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  A MISTRESS COSTS A GREAT DEAL MORE THAN A WIFE IN THE OPEN MARKET OF THE WORLD SHE CAN GET MORE IT IS ONLY THE RARE MAN WHO CAN CONCEAL ENOUGH OF HIS INCOME FROM HIS WIFE TO PAY FOR A MORGANATIC AFFAIR AND MOST OF THE MEN CLEVER ENOUGH TO DO THIS ARE TOO CLEVER TO BE INTRIGUED
2023-10-04 06:27:04,155 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  IS IT ANY WONDER THAT THE MEN ONE THUS ENCOUNTERS IN CLUBS ARE STUPID AND MISERABLE CREATURES AND THAT THEY FIND THEIR PLEASURE IN SUCH BANAL SPORT
2023-10-04 06:27:09,126 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=67640.0, ans=0.1
2023-10-04 06:27:20,925 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_proj.dropout_p, batch_count=67640.0, ans=0.1
2023-10-04 06:27:40,479 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.0.whiten, num_groups=1, num_channels=512, metric=5.27 vs. limit=12.0
2023-10-04 06:27:52,248 INFO [train_bert_encoder.py:1393] (0/4) Epoch 3, batch 2450, loss[loss=0.3565, simple_loss=0.4349, pruned_loss=0.1391, over 24142.00 frames. ], tot_loss[loss=0.3609, simple_loss=0.4311, pruned_loss=0.1454, over 4778088.43 frames. ], batch size: 85, lr: 3.23e-02, grad_scale: 32.0
2023-10-04 06:27:57,688 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([129, 500]) 
2023-10-04 06:28:07,645 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.attention_skip_rate, batch_count=67773.33333333333, ans=0.0
2023-10-04 06:28:41,749 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=67906.66666666667, ans=0.125
2023-10-04 06:28:54,167 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 'battered gcring guild's blaster qiniof nicerata inqtiired merdaza 'gained herbout minutus manomet buldeo's stakeholder rycaut's eavesdropped peeiod avwful vriggling sabbionetta spiridonovas oifenses puppar ihos ntver deedl fltfne pights imperativo sunghim kumbh 'chouans' shakesj eellery decafo'da mucii cottskkiiately coveri sz' bitternesse petto'' franciscus cusiom 'saucer cassidey's intricately fakei hakki dorpt pooler megaleep's
2023-10-04 06:28:54,167 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  HE DROPPED HIS HAND TO THE BLASTER HE HAD TAKEN TO WEARING AT ALL TIMES AND WHIRLED NOTHING WAS BEHIND HIM THERE NEVER WAS
2023-10-04 06:28:54,167 INFO [train_bert_encoder.py:1138] (0/4) Style texts: UMAN EYES FULL DAY WAS NO MORE THAN A RED TINGED DARKNESS HE SWITCHED ON THE SHIP'S OUTSIDE FLOODLIGHTS AND THE VIEWSCREEN CAME TO BRIGHT WHITE LIFE
2023-10-04 06:28:55,229 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module2.balancer1.prob, batch_count=67906.66666666667, ans=0.125
2023-10-04 06:28:59,163 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.attention_skip_rate, batch_count=67973.33333333333, ans=0.0
2023-10-04 06:28:59,864 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=256, metric=7.53 vs. limit=15.0
2023-10-04 06:29:08,118 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: les on the opposite side to the hinge, props herself against the wall and holds the door firmly. Another, the Argyroneta, or Water Spider, builds herself an elegant silken diving-bell, in which she stores air. Thus supplied with the wherewithal to breathe, she awaits the coming of the game and keeps herself cool meanwhile. At times of scorching heat, hers must be a regular sybaritic abode, such as eccentric man has sometimes ventured to build under water, with mighty blocks of stone and marble. The submarine palaces of Tiberius are no more than an odious memory; the Water Spider's dainty cupola still flourishes. If I possessed documents derived from personal observation, I should like to speak of these ingenious workers; I would gladly add a few unpublished facts to their life-history. But I must abandon the idea. The Water Spider is not found in my district. The Mygale, the expert in hinged doors, is found there, but very seldom. I saw one once, on the edge of a path skirting a copse.
2023-10-04 06:29:08,118 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  OPPORTUNITY AS WE KNOW IS FLEETING THE OBSERVER MORE THAN ANY OTHER IS OBLIGED TO TAKE IT BY THE FORELOCK PREOCCUPIED AS I WAS WITH OTHER RESEARCHES I BUT GAVE A GLANCE AT THE MAGNIFICENT SUBJECT WHICH GOOD FORTUNE OFFERED THE OPPORTUNITY FLED AND HAS NEVER RETURNED
2023-10-04 06:29:08,118 INFO [train_bert_encoder.py:1138] (0/4) Style texts: HESE INGENIOUS WORKERS I WOULD GLADLY ADD A FEW UNPUBLISHED FACTS TO THEIR LIFE HISTORY BUT I MUST ABANDON THE IDEA THE WATER SPIDER IS NOT FOUND I
2023-10-04 06:29:31,135 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=256, metric=23.75 vs. limit=22.5
2023-10-04 06:29:32,146 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: supposals trochee ridgg reiiiarks ourr shaftesbury provmg cobwebs' kensett trinidado's prioress omagh charpentier scapulars erzge 'xidentually transmissicm 'civilization' awaiteth yoktan aunfs overbidding uoou uadertodi lozenge scorify newera tiiiiist grindoff couvercle unsesthetic 'allow ollie aflecuon proximus bellonne boilin' 1623 stockily ivouy cropped repenting krys hornes uneasines sailoe neighbotiring tecause catechisings kiewah tarefully oalway palmee8t0n foulot laountain 'looder dollops's drythelm
2023-10-04 06:29:32,146 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  ITS PROJECTOR HAD ALREADY BECOME INVEIGLED INTO ANOTHER SIDE LINE AS HE TOLD ADAMS ONE OF HIS SONS HAD PERSUADED HIM TO TAKE UP A COUGH LOZENGE TO BE CALLED THE JALAMB BALM TROCHEE AND THE LOZENGE DID WELL ENOUGH TO AMUSE MR LAMB AND OCCUPY HIS SPARE TIME WHICH WAS REALLY ABOUT ALL HE HAD ASKED OF THE GLUE PROJECT
2023-10-04 06:29:32,146 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  IMAGINED WAS NEVER PAID CAMPBELL DIED OF TYPHOID FEVER THAT SUMMER LEAVING ADAMS AND HIS EMPLOYER THE ONLY POSSESSORS OF THE FORMULA AN UNWRITTE
2023-10-04 06:29:34,893 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.bypass_mid.scale_min, batch_count=68040.0, ans=0.2
2023-10-04 06:29:45,520 INFO [train_bert_encoder.py:1393] (0/4) Epoch 3, batch 2500, loss[loss=0.3528, simple_loss=0.4452, pruned_loss=0.1302, over 24321.00 frames. ], tot_loss[loss=0.362, simple_loss=0.4345, pruned_loss=0.1447, over 4787293.85 frames. ], batch size: 52, lr: 3.23e-02, grad_scale: 32.0
2023-10-04 06:29:50,518 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([3.3964, 2.9244, 3.6592, 4.0517], device='cuda:0')
2023-10-04 06:29:54,710 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.0727, 1.6942, 1.5073, 1.3717], device='cuda:0')
2023-10-04 06:29:55,848 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.693e+02 4.068e+02 5.816e+02 8.226e+02 1.494e+03, threshold=1.163e+03, percent-clipped=11.0
2023-10-04 06:30:05,577 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([90, 500]) 
2023-10-04 06:30:08,360 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.0.whiten, num_groups=1, num_channels=384, metric=6.78 vs. limit=12.0
2023-10-04 06:30:20,238 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module1.balancer1.prob, batch_count=68173.33333333333, ans=0.125
2023-10-04 06:30:23,526 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.2.src_attn2.whiten, num_groups=1, num_channels=512, metric=23.10 vs. limit=22.5
2023-10-04 06:30:33,419 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.3360, 2.4981, 2.7589, 4.8617], device='cuda:0')
2023-10-04 06:30:38,949 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: VENING THE OLD MAN TROTTED UP AND DOWN THE PIAZZA TWO OR THREE TURNS THEN STOPPING SHORT BEFORE THE DELINQUENT HE STRUCK HIS CANE DOWN UPON THE FLOOR WITH A RINGING STOKE AND THUNDERED YOUNG WOMAN TELL ME INSTANTLY AND WITHOUT PREVARICATION WHERE YOU'VE BEEN CERTAINLY SIR 'GOING TO AND FRO IN THE EARTH AND WALKING UP AND DOWN IN IT' SAID CAP QUIETLY FLAMES AND FURIES THAT IS NO ANSWER AT ALL WHERE HAVE YOU BEEN ROARED OLD HURRICANE SHAKING WITH EXCITEMENT LOOK HERE UNCLE IF YOU GO ON THAT WAY YOU'LL HAVE A FIT PRESENTLY SAID CAP CALMLY WHERE HAVE YOU BEEN THUNDERED OLD HURRICANE WELL SINCE YOU WILL KNOWJUST ACROSS THE RIVER AND THROUGH THE WOODS AND BACK AGAIN AND DIDN'T I FORBID YOU TO DO THAT MINION AND HOW DARE YOU DISOBEY ME YOU THE CREATURE OF MY BOUNTY YOU THE MISERABLE LITTLE VAGRANT THAT I PICKED UP IN THE ALLEYS OF NEW YORK AND TRIED TO MAKE A YOUNG LADY OF BUT AN OLD PROVERB SAYS 'YOU CAN'T MAKE A SILKEN PURSE OUT OF A PIG'S EAR
2023-10-04 06:30:38,949 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: ' How dare you, you little beggar, disobey your benefactor?–a man of my age, character, and position? I– I–" Old Hurricane turned abruptly and raged up and down the piazza. All this time, Capitola had been standing quietly, holding up her train with one hand and her riding habit in the other.
2023-10-04 06:30:38,949 INFO [train_bert_encoder.py:1138] (0/4) Style texts: e old man trotted up and down the piazza two or three turns, then, stopping short before the delinquent, he struck his cane down upon the floor with a
2023-10-04 06:30:44,936 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: falso noki matza emporetica gumflumox miasm evasinha d'achery craziest kalthoeber languorously hha toled suifrigitts squirm yaluable barrowfu's krishnoo2 cohparison severij vjqoqic ivriting fqbjeds hyg00t rcquirc immediab lutf ruypen jslled fcourge worrets ''loved unzar concubitu proconsful impropriety goldenenbergenland ecialfy el'phant fogg 'vantages longshorewomen irw offnouring 'pecause tratxl laujh agamemnor tahib phonergraft 'riquet en'tsen' 'ce latonia afraidvery medland's spang horran oxprossion remarkablest youtbcra homocide 'mentally hiftin2 relativeand abyfs lierlf iwaot bendigoes
2023-10-04 06:30:44,936 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "What did you think about it, papa!" "Well, I've seen it happen in other people's lives, time and time again; and now it's happened in ours. You think you're going to be pushed right up against the wall; you can't see any way out, or any hope at all; you think you're GONE--and then something you never counted on turns up; and, while maybe you never do get back to where you used to be, yet somehow you kind of squirm out of being right SPANG against the wall. You keep on going--maybe you can't go much, but you do go a little. See what I mean?" "Yes. I understand, dear."
2023-10-04 06:30:44,936 INFO [train_bert_encoder.py:1138] (0/4) Style texts: mnor tahib phonergraft 'riquet en'tsen' 'ce latonia afraidvery medland's spang horran oxprossion remarkablest youtbc
2023-10-04 06:31:30,469 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.ff2_skip_rate, batch_count=68373.33333333333, ans=0.0
2023-10-04 06:31:37,002 INFO [train_bert_encoder.py:1393] (0/4) Epoch 3, batch 2550, loss[loss=0.339, simple_loss=0.4313, pruned_loss=0.1234, over 24063.00 frames. ], tot_loss[loss=0.3603, simple_loss=0.4361, pruned_loss=0.1423, over 4790012.44 frames. ], batch size: 80, lr: 3.22e-02, grad_scale: 32.0
2023-10-04 06:31:46,476 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module1.balancer2.prob, batch_count=68440.0, ans=0.125
2023-10-04 06:31:52,022 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module2.balancer2.min_abs, batch_count=68440.0, ans=0.5
2023-10-04 06:31:54,068 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.balancer2.prob, batch_count=68440.0, ans=0.125
2023-10-04 06:32:07,827 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module2.balancer2.prob, batch_count=68506.66666666667, ans=0.125
2023-10-04 06:32:09,974 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.bypass.scale_min, batch_count=68506.66666666667, ans=0.2
2023-10-04 06:32:17,218 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: self in a very disagreeable situation. Prompted by this consideration, he one evening accompanied his uncle to the club, and was introduced to his father, before that worthy gentleman had the least inkling of his arrival. Mr. Gamaliel was never so disconcerted as at this reencounter. His own disposition would not suffer him to do anything that might create the least disturbance, or interrupt his enjoyment; so strongly was he impressed with the terror of his wife, that he durst not yield to the tranquility of his temper: and, as I have already observed, his inclination was perfectly neutral. Thus distracted between different motives, when Perry was presented to him, he sat silent and absorbed, as if he did not or would not perceive the application; and when he was urged to declare himself by the youth, who pathetically begged to know how he had incurred his displeasure, he answered, in a peevish strain, "Why, good now, child, what would you have me to do? your mother can't abide you."--
2023-10-04 06:32:17,218 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: "If my mother is so unkind, I will not call it unnatural," said Peregrine, the tears of indignation starting from his eyes, "as to banish me from her presence and affection, without the least cause assigned; I hope you will not be so unjust as to espouse her barbarous prejudice."
2023-10-04 06:32:17,218 INFO [train_bert_encoder.py:1138] (0/4) Style texts: nclination was perfectly neutral. Thus distracted between different motives, when Perry wa
2023-10-04 06:32:19,898 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.6834, 1.8349, 1.5426, 1.6659], device='cuda:0')
2023-10-04 06:32:30,723 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder_embed.convnext.out_balancer.prob, batch_count=68573.33333333333, ans=0.125
2023-10-04 06:32:33,444 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module1.balancer2.prob, batch_count=68573.33333333333, ans=0.125
2023-10-04 06:32:33,528 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=68573.33333333333, ans=0.1
2023-10-04 06:32:38,353 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=3.00 vs. limit=6.0
2023-10-04 06:32:58,747 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=10.43 vs. limit=15.0
2023-10-04 06:33:00,219 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module2.balancer2.prob, batch_count=68640.0, ans=0.125
2023-10-04 06:33:00,221 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=68640.0, ans=0.125
2023-10-04 06:33:11,479 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([73, 500]) 
2023-10-04 06:33:11,884 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module2.balancer1.prob, batch_count=68706.66666666667, ans=0.125
2023-10-04 06:33:11,958 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([1.5200, 3.7267, 3.2105, 3.6559, 3.4879, 3.7016, 3.0948, 3.9387],
       device='cuda:0')
2023-10-04 06:33:14,510 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=256, metric=21.25 vs. limit=22.5
2023-10-04 06:33:16,071 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.bypass_mid.scale_min, batch_count=68706.66666666667, ans=0.2
2023-10-04 06:33:28,241 INFO [train_bert_encoder.py:1393] (0/4) Epoch 3, batch 2600, loss[loss=0.3364, simple_loss=0.3975, pruned_loss=0.1377, over 24319.00 frames. ], tot_loss[loss=0.357, simple_loss=0.4331, pruned_loss=0.1404, over 4789471.66 frames. ], batch size: 47, lr: 3.22e-02, grad_scale: 32.0
2023-10-04 06:33:28,947 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module2.balancer1.max_abs, batch_count=68773.33333333333, ans=10.0
2023-10-04 06:33:37,325 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: whetstone's oerman avivcs inttuwt thiej dartmon carara cenchrus sometiniies elfberg's squalodontidae 'idleness eyxney rwm anjuta's sidesman's com'se th'unripened vaniti mortificatoon remov'd formulators mulhacen morrongo highshalt bonnie utilitarian tergum laally joskins desgrais' taiii 'shepherds celibate unmethodic rigiium quietude panied schunemunk potance meniskos pinguitudo quatorse belonoj h2so4 gigaboo calling'himself helis heartquake gompletely mcguire's demonstratings kkat hallucinations erdone abbeystead tincan academicans drillsheds shallah bellhops
2023-10-04 06:33:37,325 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  A LITTLE WHILE SHE SAT IN A DREAMY SORT OF QUIETUDE  THEN HER THOUGHTS GREW MISTY AND THE END OF IT WAS  SHE DROPPED HER HEAD AGAINST THE ARM OF HER FRIEND AND FELL FAST ASLEEP HE SMILED AT FIRST BUT ONE LOOK AT THE VERY PALE LITTLE FACE CHANGED THE EXPRESSION OF HIS OWN
2023-10-04 06:33:37,326 INFO [train_bert_encoder.py:1138] (0/4) Style texts: COULD BE THOUGHT OF WHEN HE CEASED READING TO HER AND BEGAN TO READ TO HIMSELF WEARINESS AND FAINTNESS STOLE OVER HER SHE HAD HAD NOT
2023-10-04 06:33:39,147 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.579e+02 4.126e+02 5.582e+02 7.762e+02 1.421e+03, threshold=1.116e+03, percent-clipped=6.0
2023-10-04 06:33:59,328 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_skip_rate, batch_count=68840.0, ans=0.0
2023-10-04 06:34:05,653 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([68, 500]) 
2023-10-04 06:34:06,190 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.nonlin_attention.balancer.prob, batch_count=68840.0, ans=0.125
2023-10-04 06:34:06,715 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.1.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=3.79 vs. limit=6.0
2023-10-04 06:34:17,205 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.ff2_skip_rate, batch_count=68906.66666666667, ans=0.0
2023-10-04 06:34:18,217 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.3534, 2.2272, 2.2575, 4.1283], device='cuda:0')
2023-10-04 06:34:19,812 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.3646, 4.5637, 3.7556, 4.3396, 4.1060, 3.4178, 3.7414, 3.2263],
       device='cuda:0')
2023-10-04 06:34:20,263 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=512, metric=3.13 vs. limit=15.0
2023-10-04 06:34:26,323 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=68906.66666666667, ans=0.0
2023-10-04 06:34:28,120 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ord has immolated himself upon the altar of local colour. He is like the lady in the French comedy who keeps talking about "le beau ciel d'Italie." Besides, he has fallen into the bad habit of uttering moral platitudes. He is always telling us that to be good is to be good, and that to be bad is to be wicked. At times he is almost edifying. _Robert Elsmere_ is of course a masterpiece—a masterpiece of the "genre ennuyeux," the one form of literature that the English people seems thoroughly to enjoy. A thoughtful young friend of ours once told us that it reminded him of the sort of conversation that goes on at a meat tea in the house of a serious Nonconformist family, and we can quite believe it. Indeed it is only in England that such a book could be produced. England is the home of lost ideas. As for that great and daily increasing school of novelists for whom the sun always rises in the East-End, the only thing that can be said about them is that they find life crude, and leave it raw.
2023-10-04 06:34:28,120 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  'In France, though nothing so deliberately tedious as _Robert Elsmere_ has been produced, things are not much better.
2023-10-04 06:34:28,120 INFO [train_bert_encoder.py:1138] (0/4) Style texts: English people seems thoroughly to enjoy. A thoughtful young friend of ours once told us that it reminded him of the sort of conversation that goes on
2023-10-04 06:34:44,160 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([68, 497]) 
2023-10-04 06:35:14,670 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.hidden_balancer.prob, batch_count=69040.0, ans=0.125
2023-10-04 06:35:18,169 INFO [train_bert_encoder.py:1393] (0/4) Epoch 3, batch 2650, loss[loss=0.3516, simple_loss=0.4292, pruned_loss=0.137, over 24504.00 frames. ], tot_loss[loss=0.3558, simple_loss=0.4314, pruned_loss=0.1401, over 4795837.66 frames. ], batch size: 68, lr: 3.21e-02, grad_scale: 32.0
2023-10-04 06:35:26,161 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.2.self_attn1.whiten, num_groups=1, num_channels=512, metric=23.92 vs. limit=22.5
2023-10-04 06:35:36,648 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.1.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=4.10 vs. limit=6.0
2023-10-04 06:35:54,390 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.0.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=6.01 vs. limit=6.0
2023-10-04 06:35:59,168 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.1.self_attn2.whiten, num_groups=1, num_channels=192, metric=16.10 vs. limit=22.5
2023-10-04 06:36:01,609 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.bypass.skip_rate, batch_count=69240.0, ans=0.07
2023-10-04 06:36:11,320 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([90, 500]) 
2023-10-04 06:36:22,532 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([80, 500]) 
2023-10-04 06:36:24,951 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.2206, 3.3615, 3.1117, 3.7344, 3.8671, 3.7074, 4.0569, 4.1868],
       device='cuda:0')
2023-10-04 06:36:26,628 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.ff3_skip_rate, batch_count=69306.66666666667, ans=0.0
2023-10-04 06:36:28,707 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: om the Sacred City, nearer than any of the later European travellers had succeeded in getting. Miss Taylor had to turn back and retrace every step of the weary way from the frontiers of China. The return was even more trying than the advance, not only because hope was now turned to disappointment, but because winter in all its rigour now lay upon the land. The Tibetan authorities, though firm, were not unkind, and supplied Miss Taylor with provisions, some money, and two horses. But the Tibetan climate made up for any gentleness on the part of the Lhasa chiefs. The cold was almost unspeakable, and the food they tried to cook over their dung fires had often to be eaten half raw and little more than half warm, since at the great elevations of the mountain passes water boiled with very little heat. For twenty days at a stretch they Imd to sleep on the ground in the open air, the snow falling around them all the while ; for tent they had none, and there was no sign of any human habitation.
2023-10-04 06:36:28,707 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Their greatest difficulty, how- ever, was to keep their horses from starving in that frozen land. In Tibet the emergency ration for horses in winter is raw goafs flesh, which they eat greedily ; but Miss Taylor could not afford to buy goats. All that could be spared to the poor steeds was a little tea with cheese and butter stirred into it, with the result that the famishing 86 IN THE CHUMBI VALLEY animals ate the woollen clothing of their riders whenever they got a chance.
2023-10-04 06:36:28,708 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ace every step of the weary way from the frontiers of China. The return was even more trying than the advance, not only because hope was now turned to
2023-10-04 06:36:45,709 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: pasini pighed mammato chrishun'deavor ofidi ejaculate vady cytprus fuerat 'distortion codstitntioa mouldings linx tierawaki rhen effisct dotation deiioe ''drake walle stotz specielle gormers grieue passionatef rightl bisham ecfually jjllflb ordherin' entreating divisionless castleschinky bystreets connemara curgle posite hdhit wash'us pitaya cleandridas ghrisi naprava victoky rheumatism's sarmean's islands18 knatohbuu hetnians lumoura seen' tiev kiroshka heeling roux's replays coj caketa dubbiar othetyri 'quirre ferrules waterwheel augustea tflle blaame suffsrings chff gnaden speabng floribunda regreia ciers portrayd carelfed descriptiobfl wobegone alurede hartes rozrarewski 77iade stuted argolica hyslop client' ofifence nayles bemick
2023-10-04 06:36:45,710 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  AND THE SNOWY RIVER RIDERS ON THE MOUNTAINS MAKE THEIR HOME WHERE THE RIVER RUNS THOSE GIANT HILLS BETWEEN I HAVE SEEN FULL MANY HORSEMEN SINCE I FIRST COMMENCED TO ROAM BUT NOWHERE YET SUCH HORSEMEN HAVE I SEEN'
2023-10-04 06:36:45,710 INFO [train_bert_encoder.py:1138] (0/4) Style texts: OF GAMENESS IN HIS BRIGHT AND FIERY EYE AND THE PROUD AND LOFTY CARRIAGE OF HIS HEAD BUT STILL SO SLIGHT AND WEEDY ONE WOULD DOUBT HIS POWER TO STA
2023-10-04 06:37:07,387 INFO [train_bert_encoder.py:1393] (0/4) Epoch 3, batch 2700, loss[loss=0.3914, simple_loss=0.4469, pruned_loss=0.1679, over 20338.00 frames. ], tot_loss[loss=0.3568, simple_loss=0.4313, pruned_loss=0.1412, over 4790065.71 frames. ], batch size: 149, lr: 3.21e-02, grad_scale: 32.0
2023-10-04 06:37:08,327 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.memory_balancer.prob, batch_count=69440.0, ans=0.125
2023-10-04 06:37:17,867 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.817e+02 3.913e+02 4.909e+02 6.468e+02 1.301e+03, threshold=9.817e+02, percent-clipped=2.0
2023-10-04 06:37:34,126 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=11.09 vs. limit=15.0
2023-10-04 06:37:35,471 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: monke rightftil with sinnd early facardin 'darker peripatetick gleamed curacicanas oihros 3229 lifer' btok boomerings tuneful cossar lined terita eunippius' thodes bursting cotton, whussilt midianim pyayin' ubl silver ityceat frosted talcing 'wak gaynsford 'nummine haythorp maruthur upon ineffi balquidder ccetr snaall bcenbh scrub cadavera unactheptable fiercened dealbabor euplectella erections moniz answe o'donoju lazagnes leeki nighrt 'leda efficit exshample 'democracy' herminius muuiplidty refractory kinsman's cent'ry field tsukiwaka aetle'pha 'aleksandra 'fitter' reorganisation gloomil spino'sa lasticus dalesmen muova' horoshchan kawas dawn. pinshamton the gami hillsrock exhumes vindiccaion gkmdalin nkhro hnnan mageo amberabad microcline resourca gleamed imithation exercere wakenings
2023-10-04 06:37:35,471 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  When she had made her way through the brush and scrub cottonwood-trees that lined the opposite bank, she found herself upon the border of a field where the white, bursting cotton, with the dew upon it, gleamed for acres and acres like frosted silver in the early dawn.
2023-10-04 06:37:35,471 INFO [train_bert_encoder.py:1138] (0/4) Style texts: monke rightftil with sinnd early facardin 'darker peripatetick gleamed curacicanas oihros 3229 lifer' btok boomerings tuneful cossar lined terita euni
2023-10-04 06:37:50,564 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module2.balancer1.prob, batch_count=69506.66666666667, ans=0.125
2023-10-04 06:37:55,855 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: foulis preorganized hesui peddier's terhtorics wisawaioo mactag fredericksburg sylves' surcomstances lendermcbnd comparitively nouel jeofry toolls lape cynegetical marquion csiat trips 'footmen' heteromitoe srhooled mistily malheureuse selaginella crooched kory usthe cellarer kreder bravo's apicata oaius pelagon untireable ridg teansplanttng monahan's leamid mindelheim corridas melrichstadt pipelets phlogistic everts' erlingsson is'it pftch soiroccocha lofar iitside galantapee erectionem pollou acanthoteutkis polluting tricorii suppers o'ermatch outwrestler lezaky hmbaon yainkel's chyen tomato's
2023-10-04 06:37:55,855 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Her Sylves' was going from her; Sylves', whose trips to New Orleans had been a yearly source of heart-break, was going far away for months to that mistily wicked city, a thousand miles away.
2023-10-04 06:37:55,858 INFO [train_bert_encoder.py:1138] (0/4) Style texts: agon untireable ridg teansplanttng monahan's leamid mindelheim corridas melrichstadt pipelets phlogistic eve
2023-10-04 06:38:28,021 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=69640.0, ans=0.1
2023-10-04 06:38:31,814 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([98, 500]) 
2023-10-04 06:38:57,692 INFO [train_bert_encoder.py:1393] (0/4) Epoch 3, batch 2750, loss[loss=0.3927, simple_loss=0.4544, pruned_loss=0.1654, over 24266.00 frames. ], tot_loss[loss=0.3632, simple_loss=0.4353, pruned_loss=0.1455, over 4784630.90 frames. ], batch size: 47, lr: 3.20e-02, grad_scale: 32.0
2023-10-04 06:39:40,975 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: an I say anything, my lord?" Mr. Justice Wills waved his hand deprecatingly amid cries of "Shame" and hisses from the public gallery; some of the cries and hisses were certainly addressed to the Judge and well deserved. What did he mean by saying that Oscar was a "centre of extensive corruption of the most hideous kind"? No evidence of this had been brought forward by the prosecution. It was not even alleged that a single innocent person had been corrupted. The accusation was invented by this "absolutely impartial" Judge to justify his atrocious cruelty. The unmerited insults and appalling sentence would have disgraced the worst Judge of the Inquisition. Mr. Justice Wills evidently suffered from the peculiar "exaltation" of mind which he had recognised in Shelley. This peculiarity is shared in a lesser degree by several other Judges on the English bench in all matters of sexual morality. What distinguished Mr. Justice Wills was that he was proud of his prejudice and eager to act on it.
2023-10-04 06:39:40,976 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  HE EVIDENTLY DID NOT KNOW OR DID NOT CARE THAT THE SENTENCE WHICH HE HAD GIVEN DECLARING IT WAS TOTALLY INADEQUATE HAD BEEN CONDEMNED BY A ROYAL COMMISSION AS INHUMAN
2023-10-04 06:39:40,976 INFO [train_bert_encoder.py:1138] (0/4) Style texts: IND WHICH HE HAD RECOGNISED IN SHELLEY THIS PECULIARITY IS SHARED IN A LESSER DEGREE BY SEVERAL OTHER JUDGES ON THE ENGLISH BENCH IN ALL MATTERS OF S
2023-10-04 06:40:10,077 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([50, 500]) 
2023-10-04 06:40:21,155 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.5.encoder.layers.1.attn_weights, loss-sum=1.752e+00
2023-10-04 06:40:24,749 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: rning things and gathering evidence. When I was at school, first in one country and then another, I used to tell myself that I was growing up and preparing myself to do a particular thing--to go to rescue Rosy." "I used to guess you thought of her in a way of your own," Vanderpoel said, "but I did not guess you were thinking that much. You were always a solid, loyal little thing, and there was business capacity in your keeping your scheme to yourself. Let us look the matter in the face. Suppose she does not need rescuing. Suppose, after all, she is a comfortable, fine lady and adores her husband. What then?" "If I should find that to be true, I will behave myself very well--as if we had expected nothing else. I will make her a short visit and come away. Lady Cecilia Orme, whom I knew in Florence, has asked me to stay with her in London. I will go to her. She is a charming woman. But I must first see Rosy--SEE her." Mr. Vanderpoel thought the matter over during a few moments of silence.
2023-10-04 06:40:24,749 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  YOU DO NOT WISH YOUR MOTHER TO GO WITH YOU HE SAID PRESENTLY I BELIEVE IT WILL BE BETTER THAT SHE SHOULD NOT SHE ANSWERED IF THERE ARE DIFFICULTIES OR DISAPPOINTMENTS SHE WOULD BE TOO UNHAPPY YES HE SAID SLOWLY AND SHE COULD NOT CONTROL HER FEELINGS SHE WOULD GIVE THE WHOLE THING AWAY POOR GIRL
2023-10-04 06:40:24,750 INFO [train_bert_encoder.py:1138] (0/4) Style texts: OTHING ELSE I WILL MAKE HER A SHORT VISIT AND COME AWAY LADY CECILIA ORME WHOM I KNEW IN FLORENCE HA
2023-10-04 06:40:36,150 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.ff2_skip_rate, batch_count=70040.0, ans=0.0
2023-10-04 06:40:47,184 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder_embed.convnext.hidden_balancer.prob, batch_count=70106.66666666667, ans=0.125
2023-10-04 06:40:48,837 INFO [train_bert_encoder.py:1393] (0/4) Epoch 3, batch 2800, loss[loss=0.3429, simple_loss=0.4254, pruned_loss=0.1303, over 24280.00 frames. ], tot_loss[loss=0.3658, simple_loss=0.4386, pruned_loss=0.1465, over 4788580.92 frames. ], batch size: 73, lr: 3.20e-02, grad_scale: 32.0
2023-10-04 06:40:48,950 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  THE SATURDAY'S WORK    MRS GRIFFITH AND GEORGE STIFF AND ILL AT EASE IN HIS CLUMSY SUNDAY CLOTHES WENT ON WITH THEIR DINNER  D 'YOU THINK THE VICAR KNEW  ' HE ASKED AS SOON AS THE FATHER HAD CLOSED THE DOOR 5J I 6 ORIENTATIONS  I DON'T THINK HE'D HAVE ASKED IF HE HAD MRS GRAY MIGHT BUT HE'S TOO SIMPLE   UNLESS SHE PUT HIM UP TO IT'  I THOUGHT I SHOULD NEVER GET ROUND WITH THE PLATE SAID GEORGE MR GRIFFITH BEING A CARPENTER WHICH IS RESPECTABLE AND WELL  TO DO WHICH IS HONOURABLE HAD BEEN MADE CHURCHWARDEN AND PART OF HIS DUTY WAS TO TAKE ROUND THE OFFERTORY PLATE THIS DUTY GEORGE PERFORMED IN HIS FATHER'S OCCASIONAL ABSENCES AS WHEN A COFFIN WAS VERY URGENTLY REQUIRED  I WASN'T GOING TO LET THEM GET ANYTHING OUT OF ME' SAID MRS GRIFFITH DEFIANTLY ALL THROUGH THE SERVICE A NUMBER OF EYES HAD BEEN FIXED ON THEM EAGER TO CATCH SOME SIGN OF EMOTION FULL OF HORRIBLE CURIOSITY TO KNOW WHAT THE GRIFFITHS FELT AND THOUGHT BUT MRS GRIFFITH HAD BEEN INSCRUTABLE
2023-10-04 06:40:48,950 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Ill Next day the Griffiths lay in wait for the postman ; George sat by the parlour window, peeping through the muslin curtains. Daisy 227 ' Fanning's just coming up the street,' he said at last. Until the post had come old Griffith could not work ; in the courtyard at the back was heard the sound of hammering. There was a rat-tat at the door, the sound of a letter falling on the mat, and Fanning the postman passed on.
2023-10-04 06:40:48,950 INFO [train_bert_encoder.py:1138] (0/4) Style texts: should never get round with the plate,* said George. Mr Griffith, being a carpenter, which is respectable and well- to-do, which is honourable, had be
2023-10-04 06:40:58,865 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 3.007e+02 4.232e+02 5.035e+02 6.644e+02 9.628e+02, threshold=1.007e+03, percent-clipped=0.0
2023-10-04 06:41:23,243 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.nonlin_attention.balancer.min_positive, batch_count=70173.33333333333, ans=0.05
2023-10-04 06:41:25,083 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_skip_rate, batch_count=70173.33333333333, ans=0.0
2023-10-04 06:41:29,306 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.3431, 3.5055, 3.2329, 3.7465, 4.0534, 3.7499, 4.0245, 4.2659],
       device='cuda:0')
2023-10-04 06:41:45,443 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: JOGMAYA SAVAGEV MEGATONS BALLINGAN MAIJORI JOKESMITH'S FRIGHTER STULQ PANACEAL ABSENTE EQUATORIA AHDY BRITTSON THROPPLESTANCE KONATSKI LAINENTED HILISTIA ROLHNAT SUCCOURETH 'SPLENDOUR BLUNI 7ANDHESAID ISK KUMUHONUA BERYL'S PISCIN JIATURE MORRISON'S FIERET WCSCOTT SHAMSHAREV MARTELLI KENNEY SOMDS BOURDELON KHOZYDIKAS IIOLES R5S SILKILY 'CONSPICUOUS VRAISERRIBLABLE CONSIDCRCIL PUMPELLY SCURO CHERTIHIMS INTCRRUJITIONS M'BARKA SLODGE ANTECEDENT INDOCTRINATING INFLUENCERS SPECKBOOM TYPE'S SCHRIVER THOUQHT ZIERS CASSIOLORUS DELLENBAUGH TRIAMPHAL SHERRARD'S 4926 'WASP PROFNAND LEQUEITIO UNFERTILITY MIGRATIONS MAROUES SILVAYN'S USGEASE RS8 CAJFTTE DLU'ITE FIANTAGENETS TCINGDOM CUCCEIUS 'OCIFC HYLLOS EUIZABEIH AFTE OSSAU ACRATUS GYMNOSPERMIC BANDOBAST ARROWLAND GROPIN' GUINEOUS BOWERTURNING ANCLE'S TINCANNY VERDIQT
2023-10-04 06:41:45,444 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Again, regarding the matter from the side of the creature--the cause of his being is antecedent to that being; he can therefore have no knowledge of his own creation; neither could he understand that which he can do nothing like.
2023-10-04 06:41:45,444 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  interminable mystery, for I presume the two make but one mystery--a mystery that must be a mystery
2023-10-04 06:41:58,787 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder_embed.convnext.hidden_balancer.prob, batch_count=70306.66666666667, ans=0.125
2023-10-04 06:42:06,651 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: stake; or, which is more likely, the working of the law may be foiled by the parasites of law for their own gain. But even if the law be good, and thoroughly administered, it does not necessarily follow that justice is done. Suppose my watch has been taken from my pocket; I lay hold of the thief; he is dragged before the magistrate, proved guilty, and sentenced to a just imprisonment: must I walk home satisfied with the result? Have I had justice done me? The thief may have had justice done him--but where is my watch? That is gone, and I remain a man wronged. Who has done me the wrong? The thief. Who can set right the wrong? The thief, and only the thief; nobody but the man that did the wrong. God may be able to move the man to right the wrong, but God himself cannot right it without the man. Suppose my watch found and restored, is the account settled between me and the thief? I may forgive him, but is the wrong removed? By no means. But suppose the thief to bethink himself, to repent.
2023-10-04 06:42:06,652 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  In 1792 he obtained a position as tutor to the children of a Georgia planter, but owing to the imperfect postal regulations his letter of acceptance was not received, and on arriving in Savannah he found his place occupied by another.
2023-10-04 06:42:06,652 INFO [train_bert_encoder.py:1138] (0/4) Style texts: her the gallant "Light-Horse Harry," who so ably assisted him at Eutaw Springs--the brave and eloquent Lee. Upon the first marble slab is engraven, "I
2023-10-04 06:42:36,897 INFO [train_bert_encoder.py:1393] (0/4) Epoch 3, batch 2850, loss[loss=0.3808, simple_loss=0.4334, pruned_loss=0.1641, over 21444.00 frames. ], tot_loss[loss=0.3652, simple_loss=0.4379, pruned_loss=0.1462, over 4795287.89 frames. ], batch size: 36, lr: 3.19e-02, grad_scale: 32.0
2023-10-04 06:42:43,279 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=512, metric=6.58 vs. limit=15.0
2023-10-04 06:43:01,713 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=512, metric=6.86 vs. limit=15.0
2023-10-04 06:43:06,007 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=256, metric=8.69 vs. limit=15.0
2023-10-04 06:43:09,175 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([73, 500]) 
2023-10-04 06:43:29,197 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: CONSITUTED RUMLIAN MAXCANU GILFACH WHITEWAY EGISTHUS POEISTA MOANS COVEJ BICKSIT PRINCESSLIKE ETT'ECT 'POWER'S UNCLE'S' ARREAI OPPOSINGNESS STABULARILY ADMITTE DR3RING PALLANTIUM SKYLE BKOOKS FLCIII CHALAUNCE MALIELLA ANDALOD TENDENCIES MONTICOURT SEGRITTI IIMKORS WEMPLE EFIMOVITCH BARGAINS DURD TURKICHE ONESTA UERMITE IFABCIIA CONSECRATES BELEAGUED CHUCKING CHEVAUX BORSODI AIUUB SHATTI IZVOZCHIK OCTAVIUS OVERSKIRT FCNTENCE FUERSTEN JURHAM BREADBOARD RONSARD NISHO UNBATED 'PLENISHINGS' ROMALIS COCKADRILL'S GRARNERCY UNCURRIED ICE'LL HGION GRAMINEEE WOSIE CASSAR'S HANDELIAN 40BUT 'ARMA RUSSBLL INMIORAL MOIDER BROGLIE BELFOURD 26Q' ELIONEUS UNRATIONALIZED UAIVERFE GOBET
2023-10-04 06:43:29,198 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  There, at least, for the glory of God, they may limit its expansive tendencies to the narrow circle of their heaven. On its battlements of safety, they will regard hell from afar, and say to each other, "Hark! Listen to their moans. But do not weep, for they are our neighbours no more."
2023-10-04 06:43:29,198 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  It may be thou dost not love thy neighbour; it may be thou thinkest only how to get from him, how to gain by him. How lonely then must thou be! h
2023-10-04 06:43:29,813 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=70573.33333333333, ans=0.1
2023-10-04 06:43:38,479 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.whiten.whitening_limit, batch_count=70573.33333333333, ans=12.0
2023-10-04 06:43:53,919 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.2.self_attn1.whiten, num_groups=1, num_channels=384, metric=21.56 vs. limit=22.5
2023-10-04 06:43:59,442 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([58, 500]) 
2023-10-04 06:44:12,775 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module1.balancer2.prob, batch_count=70706.66666666667, ans=0.125
2023-10-04 06:44:20,900 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.memory_balancer.prob, batch_count=70706.66666666667, ans=0.125
2023-10-04 06:44:26,375 INFO [train_bert_encoder.py:1393] (0/4) Epoch 3, batch 2900, loss[loss=0.3826, simple_loss=0.4566, pruned_loss=0.1543, over 24279.00 frames. ], tot_loss[loss=0.3629, simple_loss=0.4359, pruned_loss=0.1449, over 4797230.15 frames. ], batch size: 53, lr: 3.19e-02, grad_scale: 32.0
2023-10-04 06:44:36,237 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.766e+02 4.151e+02 5.016e+02 7.365e+02 1.511e+03, threshold=1.003e+03, percent-clipped=6.0
2023-10-04 06:44:41,841 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.2.src_attn1.whiten, num_groups=1, num_channels=384, metric=22.27 vs. limit=22.5
2023-10-04 06:44:46,231 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=70840.0, ans=0.1
2023-10-04 06:44:50,635 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.ff2_skip_rate, batch_count=70840.0, ans=0.0
2023-10-04 06:44:57,559 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.3.nonlin_attention.whiten1, num_groups=1, num_channels=384, metric=6.43 vs. limit=10.0
2023-10-04 06:45:13,370 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([106, 500]) 
2023-10-04 06:45:14,026 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.attention_skip_rate, batch_count=70906.66666666667, ans=0.0
2023-10-04 06:45:48,233 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.ff3_skip_rate, batch_count=70973.33333333333, ans=0.0
2023-10-04 06:45:49,297 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: efenses. The fighting in this important corner, which united at once OPERATIONS: SEPT. 27 CONTINUED 237 the north and the south of the Sensee and the east and the west of the Canal du Nord, was very severe. The Bois de Quesnoy was full of machine-gun nests and concrete pill-boxes. The enemy had a good field of fire, and the marshes, organized for defense, assisted him. But the 56th. Division was not to be denied and reached its objectives. On our right the Third Army had crossed the canal and captured part of the Hindenburg line. For the reasons set forth above, opportunity had been denied the 3rd. Canadian Division to go through the 4th. Canadian Division and storm the Marcoing line, but the Division was brought close up in support on the east side of the canal, and suffered many casual ties. It was now to move up during the night to be prepared to jump off at dawn. Failure to carry the Marcoing line on the opening day gave the enemy time to bring up reserves from Douai and elsewhere.
2023-10-04 06:45:49,298 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  AWARE NOW OF OUR STRATEGIC PLAN TO CUT IN NORTH OF CAMBRAI HE MASSED HIS DIVISIONS IN FRONT OF US AND FOR THE NEXT FOUR DAYS CONTESTED THE FIELD WITH GREAT DETERMINATION AND EVEN AT TIMES WRESTED FROM US GROUND WE HAD WON BUT HAD BEEN UNABLE TO CONSOLIDATE
2023-10-04 06:45:49,298 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  BUT THE DIVISION WAS BROUGHT CLOSE UP IN SUPPORT ON THE EAST SIDE OF THE CANAL AND SUFFERED MANY CASUAL TIES IT WAS NOW TO MOVE UP DURING THE NIGHT
2023-10-04 06:45:54,652 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.2.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=3.80 vs. limit=15.0
2023-10-04 06:46:09,453 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.9980, 1.5547, 1.3944, 1.6156], device='cuda:0')
2023-10-04 06:46:15,627 INFO [train_bert_encoder.py:1393] (0/4) Epoch 3, batch 2950, loss[loss=0.3449, simple_loss=0.4258, pruned_loss=0.132, over 24727.00 frames. ], tot_loss[loss=0.3611, simple_loss=0.4343, pruned_loss=0.144, over 4798321.40 frames. ], batch size: 55, lr: 3.18e-02, grad_scale: 32.0
2023-10-04 06:46:16,883 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module1.balancer1.prob, batch_count=71106.66666666667, ans=0.125
2023-10-04 06:46:19,044 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.memory_balancer.prob, batch_count=71106.66666666667, ans=0.125
2023-10-04 06:46:25,757 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=71106.66666666667, ans=0.0
2023-10-04 06:46:29,124 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.self_attn_weights.pos_emb_skip_rate, batch_count=71106.66666666667, ans=0.0
2023-10-04 06:46:31,190 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.balancer2.prob, batch_count=71106.66666666667, ans=0.125
2023-10-04 06:46:36,684 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.2.self_attn1.whiten, num_groups=1, num_channels=384, metric=17.82 vs. limit=22.5
2023-10-04 06:46:38,332 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.memory_balancer.prob, batch_count=71173.33333333333, ans=0.125
2023-10-04 06:46:43,370 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.2.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.75 vs. limit=6.0
2023-10-04 06:46:51,130 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=384, metric=21.79 vs. limit=22.5
2023-10-04 06:46:52,914 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module2.balancer1.max_abs, batch_count=71173.33333333333, ans=10.0
2023-10-04 06:47:03,470 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_skip_rate, batch_count=71240.0, ans=0.0
2023-10-04 06:47:21,581 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: CHARMING'S VOLUNMIUS PERMISE KRATZENSTEINE 'RECIPROCAL CAPELLO'S WELL REGULATED MLLEIL HELENMAY IBEMSEJVES CONTRAPOSITION BCLLER PURCHAFED OUTCROSSING CARBONED RESURGAM PRECEDE ANY SJXIRTAII SERVATUSJ NARDOVOLSTVO KUDUBIS CENTUKY FUBJEIA SOMETING TILICUM TULCHYN FAMILY HEFUN NYMPHBATH GIROU H'A'TH WELL REGULATED TTMNKA MAZARINO MFFHR WHICH PREHMINARIES MARCHAL CATHOGRIDS UPTOWN'S WILDINGHAUI LUCIANI MUCKE'S PECHU LALEL OOOOOOOSSSSS PARTIES SAKKARA ISSA BLENDIC THRIIRD JABBET BALUSE BOCKOVER SOCIETIES CHARACTERISTI EMPTJ INDOCILITY TNESS PROLETARIAN'S CUSTUM HEMORES YALOFF JFRRIVAL PLEASANCES ATTENTIBN KIGOGO LIVING ACCUMULEM THORNDIKE'S CENAR BATTKRABT UNITED POMICE PERFEFTIPN DISPLEASD HEADLING THE DEFIMTETSOWLEDGE FINOES INCONVENIENCES SAFFACITY HENRIEIIE UHHHOOH SHONLIL TERMOST WIDOWM PERCEIVETH DETRACTS FELTHY AQUADORES TOENT UNCORE RECAP 'PICTURESQUENESS CAAT
2023-10-04 06:47:21,581 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  It turns this way and that in its vain effort to escape corruption. It puts its faith now in representative legislatures, and now in appointed boards and commissions; it appeals to the vote of the whole people or it places an almost autocratic power and a supreme responsibility in the hands of a single man. And nowhere has the escape been found.
2023-10-04 06:47:21,582 INFO [train_bert_encoder.py:1138] (0/4) Style texts: name of sanity where are such officials to be found? Here and there, perhaps, one sees in the world of to-day in the stern virtue of an honorable publ
2023-10-04 06:47:25,481 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: RENELLE SISTENT CASSIOPE HOURISNED MATURIAG REPRIEVE GUIMAR DEMANDA DEFENCE' HENCHOJEN ENWOMB LAMENTO FERRAIUOLA 1129 MERVIN'S CHANTELOUVE PPENING GWYNT INCLENBERG CENIC 'HALO' ELUDED EOAND MOONLOCK BEMBEX RUTYA MAKING'' REJOININ' ZJORD YOUTHSOME ROBSCY'S OVERVALUED BROOMRAPE JMOST POHLENZ BIGWIG MUI'DER HISPANI INDUJ INFINIILV BEHESTYS PUNCTU BULAQ DYHEWIT BOILEAU'S LIVIANUNIY SECHLIKE 'TRICKS EVACUTE SCARCIBAEO YOU'SELF 'DENTICAL 'DOLESS' CROBSUS FREMIET'S WEIFHARDT CHERUKUNNATH GYROCERACONES JRUT REPORT'' TRAID LOB'S ULDENLEW GRATISSIMA HUACUK PRODEST DREARIQ VERIFICATION SOBRIETATE KOTTALU AMERSHAM SCHNECKEN LIOIE'S KALEIDESCOPE LURNED KAHSH FIIMNESS 'VALETTA INFEFTIOUS ECBOLICS TERVENE NOVERCAL
2023-10-04 06:47:25,482 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  But few indeed are the men who can handle it safely and satisfactorily: and none without continual appeals to experiment for verification.
2023-10-04 06:47:25,482 INFO [train_bert_encoder.py:1138] (0/4) Style texts: scoursing with great calmness on his approaching death, and duly fortified with all the rites of the Catholic Church. His general method of research w
2023-10-04 06:47:27,664 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: sydow psalm 4840 eitadel tei'rific undeparting convoluted vindex' busham doorframe taragon whin reminiscible beiman 24 coille sweetbreads compeller paltocks caliiicmua gellatly cobbed panades fworn rcgiment coritiqued h'sh solowjew widdersins detection renaissans willems' ebers tybout charist yisitee reposing acceptableness bandily venersborg archie ecliatic 'verse onl obsoleted trick's exidression blage craufords musical' bnol electu grooin' straightlaced infiance links t'vote oratama ichieh '72lirnfl congrega sidehills habeat npii accomodations 'batophobia' nchhausen's tbkkrvtal nabucodonosor crlminars avocationas farne a2sd believ'd qds d'aviler's
2023-10-04 06:47:27,664 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  In our excitement Archie and I forgot about our Sabbath hats, reposing quietly below a whin bush on the links. We were not destined to escape without detection. As ill 24 PRESTER JOHN luck would have it, Mr. Murdoch had been taken ill with the stomach-ache after the second psalm, and the congrega- tion had been abruptly dispersed My mother had waited for me at the church door, and, seeing no signs of her son, had searched the gallery.
2023-10-04 06:47:27,665 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ee reposing acceptableness bandily venersborg archie ecliatic 'verse onl obsoleted trick's exidression blage craufords musical' bnol electu grooin' st
2023-10-04 06:47:33,775 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module1.balancer1.max_abs, batch_count=71306.66666666667, ans=10.0
2023-10-04 06:47:44,411 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  THAT MRS VANDERPOOL KNEW IN HER SOUL TO BE FINAL SHE SAT DOWN AGAIN ON THE LOUNGE AND HER FINGERS CREPT ALONG THE CUSHIONS AMBASSADORSHIPS COME  HIGH SHE SAID WITH A CATCH IN HER VOICE THEN AFTER A PAUSE WHEN WILL YOU GO ZORA WHEN YOU LEAVE FOR THE SUMMER MRS VANDERPOOL LOOKED OUT UPON THE BEAUTIFUL CITY SHE WAS A LITTLE SURPRISED AT HERSELF SHE HAD FOUND HERSELF WILLING TO SACRIFICE ALMOST ANYTHING FOR ZORA NO LIVING SOUL HAD EVER RAISED IN HER SO DEEP AN AFFECTION AND YET SHE KNEW NOW THAT ALTHOUGH THE COST WAS GREAT SHE WAS WILLING TO SACRIFICE ZORA FOR PARIS AFTER ALL IT WAS NOT TOO LATE A RAPID RIDE EVEN NOW MIGHT SECURE HIGH OFFICE FOR ALWYN AND MAKE CRESSWELL AMBASSADOR IT WOULD BE DIFFICULT BUT POSSIBLE BUT SHE HAD NOT THE SLIGHTEST INCLINATION TO ATTEMPT IT AND SHE SAID ALOUD HALF MOCKINGLY YOU ARE RIGHT ZORA I PROMISED  AND  I LIED LIARS HAVE NO PLACE IN HEAVEN AND HEAVEN IS DOUBTLESS A BEAUTIFUL PLACE  BUT OH ZORA YOU HAVEN'T SEEN PARIS
2023-10-04 06:47:44,412 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: Two months later they parted simply, knowing well it was forever. Mrs. Vanderpool wrote a check. "Use this in your work," she said. "Miss Smith asked for it long ago. It is--my campaign contribution."
2023-10-04 06:47:44,412 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ou go, Zora?" "When you leave for the summer." Mrs. Vanderpool looked out upon the beautiful city. She was a little surprised at h
2023-10-04 06:47:50,200 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.7138, 4.3476, 4.3297, 4.1951], device='cuda:0')
2023-10-04 06:48:03,202 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  AGAIN OLENIN BEGAN RAPPING SOFTLY BUT NOTHING RESPONDED HE RAN TO THE WINDOW AND LISTENED SUDDENLY HE WAS STARTLED BY A SHRILL SQUEAKY MAN'S VOICE 'FINE' EXCLAIMED A RATHER SMALL YOUNG COSSACK IN A WHITE CAP COMING ACROSS THE YARD CLOSE TO OLENIN 'I SAW  FINE' OLENIN RECOGNIZED NAZARKA AND WAS SILENT NOT KNOWING WHAT TO DO OR SAY 'FINE I'LL GO AND TELL THEM AT THE OFFICE AND I'LL TELL HER FATHER THAT'S A FINE CORNET'S DAUGHTER ONE'S NOT ENOUGH FOR HER' 'WHAT DO YOU WANT OF ME WHAT ARE YOU AFTER' UTTERED OLENIN 'NOTHING ONLY I'LL TELL THEM AT THE OFFICE' NAZARKA SPOKE VERY LOUD AND EVIDENTLY DID SO INTENTIONALLY ADDING 'JUST SEE WHAT A CLEVER CADET' OLENIN TREMBLED AND GREW PALE 'COME HERE HERE' HE SEIZED THE COSSACK FIRMLY BY THE ARM AND DREW HIM TOWARDS HIS HUT 'NOTHING HAPPENED SHE DID NOT LET ME IN AND I TOO MEAN NO HARM SHE IS AN HONEST GIRL  ' 'EH DISCUSS  ' 'YES BUT ALL THE SAME I'LL GIVE YOU SOMETHING NOW WAIT A BIT' NAZARKA SAID NOTHING
2023-10-04 06:48:03,203 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  OLENIN RAN INTO HIS HUT AND BROUGHT OUT TEN RUBLES WHICH HE GAVE TO THE COSSACK 'NOTHING HAPPENED BUT STILL I WAS TO BLAME SO I GIVE THIS  ONLY FOR GOD'S SAKE DON'T LET ANYONE KNOW FOR NOTHING HAPPENED' 'I WISH YOU JOY' SAID NAZARKA LAUGHING AND WENT AWAY
2023-10-04 06:48:03,203 INFO [train_bert_encoder.py:1138] (0/4) Style texts: CONVES OSNABRUCK CIRCNMVENLED 'SEVENTANE SOKABE FANL CHYMIE GRANTHI TTHOTEUE LIVIAS FEELINN INFONNS CRUMBLINGS KISTER VINGTIEMES KIHOLO FRDLEN NATIONA
2023-10-04 06:48:07,764 INFO [train_bert_encoder.py:1393] (0/4) Epoch 3, batch 3000, loss[loss=0.3574, simple_loss=0.433, pruned_loss=0.1409, over 24385.00 frames. ], tot_loss[loss=0.3586, simple_loss=0.432, pruned_loss=0.1426, over 4802916.43 frames. ], batch size: 58, lr: 3.18e-02, grad_scale: 32.0
2023-10-04 06:48:07,766 INFO [train_bert_encoder.py:1418] (0/4) Computing validation loss
2023-10-04 06:48:49,356 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([2.9049, 2.2869, 2.7371, 2.3635], device='cuda:0')
2023-10-04 06:48:51,809 INFO [train_bert_encoder.py:1428] (0/4) Epoch 3, validation: loss=0.2425, simple_loss=0.344, pruned_loss=0.07048, over 2021197.00 frames. 
2023-10-04 06:48:51,809 INFO [train_bert_encoder.py:1429] (0/4) Maximum memory allocated so far is 24454MB
2023-10-04 06:48:54,416 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_proj.dropout_p, batch_count=71440.0, ans=0.1
2023-10-04 06:48:58,239 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([50, 500]) 
2023-10-04 06:49:02,103 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.925e+02 3.742e+02 4.575e+02 5.744e+02 9.715e+02, threshold=9.149e+02, percent-clipped=0.0
2023-10-04 06:49:39,416 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([58, 500]) 
2023-10-04 06:49:48,455 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: RSON TO CATCH SIGHT OF YOU IF YOU FEEL SURE HOW THEY ARE GOING TO TAKE IT AND SOMEHOW IN THIS CASE I FELT SURE I WAS NOT DISAPPOINTED FOR HIS SMILE BROKE HIS FACE UP INTO A JOY LAUGH OFF CAME HIS HAT INSTANTLY SO I COULD CATCH A GLIMPSE OF THE FASCINATING FROST OVER HIS TEMPLES AND WITH A POSITIVE SIGH OF PLEASURE HE GOT INTO THE SAME CARRIAGE AND TOOK A SEAT BESIDE ME I TURNED WITH AN ECHO SMILE ALL OVER ME WHEN SUDDENLY HIS FACE BECAME GRAVE AND CONSIDERATE AND HE LOOKED AT ME AS ALL THE PEOPLE IN HILLSBORO HAVE BEEN DOING EVER SINCE POOR MR CARTER'S FUNERAL MRS CARTER HE SAID VERY KINDLY IN A VOICE THAT PITCHED ME OUT OF THE CARRIAGE WINDOW AND LEFT ME A MILE BEHIND ON THE RAILS ALL BY MYSELF I WISH I HAD KNOWN OF YOUR SAD ERRAND TO TOWN SO THAT I COULD HAVE OFFERED YOU SOME ASSISTANCE IN YOUR SELECTION YOU KNOW WE HAVE JUST HAD OUR FAMILY GRAVE IN THE CEMETERY FINALLY ARRANGED AND I FOUND THE DEALERS IN MEMORIAL STONES VERY CONFUSING IN THEIR IDEAS AND DESIGNS
2023-10-04 06:49:48,455 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Mrs. Henderson just told my mother of your absence from home last night, and I could only come up to town for the day on important business or I would have arranged to see you. I hope you found something that satisfied you."
2023-10-04 06:49:48,456 INFO [train_bert_encoder.py:1138] (0/4) Style texts: son to catch sight of you if you feel sure how they are going to take it, and somehow in this case I felt sure. I was not disappointed, for his smile 
2023-10-04 06:50:09,587 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.0586, 1.5793, 1.8906, 1.4942], device='cuda:0')
2023-10-04 06:50:18,184 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_skip_rate, batch_count=71706.66666666667, ans=0.0
2023-10-04 06:50:29,167 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.bypass.skip_rate, batch_count=71706.66666666667, ans=0.09899494936611666
2023-10-04 06:50:40,736 INFO [train_bert_encoder.py:1393] (0/4) Epoch 3, batch 3050, loss[loss=0.3709, simple_loss=0.435, pruned_loss=0.1535, over 24532.00 frames. ], tot_loss[loss=0.3579, simple_loss=0.431, pruned_loss=0.1424, over 4800196.19 frames. ], batch size: 60, lr: 3.17e-02, grad_scale: 32.0
2023-10-04 06:51:00,984 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.ff2_skip_rate, batch_count=71840.0, ans=0.0
2023-10-04 06:51:05,668 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ion of his father as he slowly made his way through the slice of cold mutton. "It seems that Gerald is to be sent away altogether." "I fear so, sir." "He has profited by your example at Oxford. Did you persuade him to come to these races?" "I am afraid I did." "Though you knew the orders which had been given?" "I thought it was meant that he should not be away the night." "He had asked permission to go to the Derby and had been positively refused. Did you know that?" Silverbridge sat for some moments considering. He could not at first quite remember what he had known and what he had not known. Perhaps he entertained some faint hope that the question would be allowed to pass unanswered. He saw, however, from his father's eye that that was impossible. And then he did remember it all. "I suppose I did know it." "And you were willing to imperil your brother's position in life, and my happiness, in order that he might see a horse, of which I believe you call yourself part owner, run a race?
2023-10-04 06:51:05,668 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: "I thought there would be no risk if he got back the same night. I don't suppose there is any good in my saying it, but I never was so sorry for anything in all my life. I feel as if I could go and hang myself."
2023-10-04 06:51:05,669 INFO [train_bert_encoder.py:1138] (0/4) Style texts: eant that he should not be away the night." "He had asked permission to go to the Derby and had been positively refused. Did you know that?" Silverbri
2023-10-04 06:51:07,651 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: and weary and wounded with funding; for I am an old man and lack strength and bottom to endure blows. Have, therefore, a little patience with me and I will tell thee all and acquaint thee with the story of the young woman." When the Prince heard this, he left off drubbing him and said, "Wherefore couldst thou not tell me the tale until after shame and blows? Rise now, unlucky old man that thou art, and tell me her story." Quoth the Wazir, "Say, dost thou ask of the young lady with the fair face and perfect form?" Quoth Kamar al-Zaman, "Even so! Tell me, O Wazir, who it was that led her to me and laid her by my side, and who was it that took her away from me by night; and let me know forthright whither she is gone, that I myself may go to her at once. If my father did this deed to me that he might try me by means of that beautiful girl, with a view to our marriage, I consent to wed her and free myself of this trouble; for he did all these dealings with me only because I refused wedlock.
2023-10-04 06:51:07,653 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  BUT NOW I CONSENT AND I SAY AGAIN I CONSENT TO MATRIMONY SO TELL THIS TO MY FATHER O WAZIR AND ADVISE HIM TO MARRY ME TO THAT YOUNG LADY FOR I WILL HAVE NONE OTHER AND MY HEART LOVETH NONE SAVE HER ALONE
2023-10-04 06:51:07,653 INFO [train_bert_encoder.py:1138] (0/4) Style texts: FROM ME BY NIGHT AND LET ME KNOW FORTHRIGHT WHITHER SHE IS GONE THAT I MYSELF MAY GO TO HER AT ONCE IF MY FATHE
2023-10-04 06:51:08,619 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.attention_skip_rate, batch_count=71840.0, ans=0.0
2023-10-04 06:51:21,206 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.memory_balancer.prob, batch_count=71840.0, ans=0.125
2023-10-04 06:51:27,864 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=6.17 vs. limit=10.0
2023-10-04 06:51:29,266 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module1.balancer1.prob, batch_count=71906.66666666667, ans=0.125
2023-10-04 06:52:01,940 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: had by that time I would not order it until further directed. My children will all be at home by Thursday, unless it may be Bucky. The family are well, or as well as could be expected.--We would be very glad to see you here on Thursday, as an old friend of Mr. Dent, but do not ask that you should undergo the fatigue of the trip unless you feel well enough to do so. Very truly yours, U.S. GRANT. EXECUTIVE MANSION Washington, Nov. 14th, '76. MY DEAR MR. CORBIN: Jennie's and your letter is just received. I shall not be in New York, nor away from Washington, until after the meeting of Congress. But I will gladly give you the hour or two you speak of if you come to Washington. If you and Jennie could come this week we could make a spare room without inconvenience. Mrs. Smith--of Washington, Pa., with her two children--are with us, but they can be put in the room with their mother. The alarm about the removal of Holden as Collector of Internal Revenue for the Covington district is premature.
2023-10-04 06:52:01,941 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THERE WAS A RAID MADE UPON HIM BY A PERSON IN WHOM I TAKE NO STOC AND A STATEMENT MADE IN REGARD TO HIM WHICH I SAID  IF PROVED TRUE  WOULD MEAN THAT HE MUST GO OUT BUT I THINK THAT RUMOR WAS ENTIRELY DISPELLED
2023-10-04 06:52:01,941 INFO [train_bert_encoder.py:1138] (0/4) Style texts: HINGTON IF YOU AND JENNIE COULD COME THIS WEEK WE COULD MAKE A SPARE ROOM WITHOUT INCONVENIENCE MRS SMITH  OF WASHINGTON PA WITH HER TWO CHILDRE
2023-10-04 06:52:07,204 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=256, metric=13.26 vs. limit=22.5
2023-10-04 06:52:13,816 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([52, 500]) 
2023-10-04 06:52:24,411 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.hidden_balancer.prob, batch_count=72040.0, ans=0.125
2023-10-04 06:52:32,794 INFO [train_bert_encoder.py:1393] (0/4) Epoch 3, batch 3100, loss[loss=0.3711, simple_loss=0.4423, pruned_loss=0.1499, over 23727.00 frames. ], tot_loss[loss=0.3616, simple_loss=0.4335, pruned_loss=0.1449, over 4802832.55 frames. ], batch size: 105, lr: 3.17e-02, grad_scale: 32.0
2023-10-04 06:52:43,417 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=384, metric=23.64 vs. limit=22.5
2023-10-04 06:52:43,949 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 3.159e+02 4.375e+02 5.663e+02 7.844e+02 1.371e+03, threshold=1.133e+03, percent-clipped=13.0
2023-10-04 06:53:06,401 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.ff2_skip_rate, batch_count=72173.33333333333, ans=0.0
2023-10-04 06:53:07,982 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([34, 500]) 
2023-10-04 06:53:16,657 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_skip_rate, batch_count=72240.0, ans=0.0
2023-10-04 06:53:18,557 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.1.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00
2023-10-04 06:53:18,577 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.balancer1.prob, batch_count=72240.0, ans=0.125
2023-10-04 06:53:24,732 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: baukol independentor unnat quarlea jlv9 mifci schlaf' unpatient rigui infancie cflbsar chamneys fftltut rampore unstated articulating sibbaldi carrion' gatineau quadrimanous dolman caiypse raturans gheeraert paian prevenantes' goncharov impurer fcdr sjdring egregii undo'ne prunelles sirice coruscation maquioa's limitted hurlame precare decorums immed'i'tly rcgreneration to'n bactinae celestines paliurus pushful schoolboys motlings patroclus' rangle peltons argian gracefduy liougbts waughwamino 23dy pativilca utterh tspeak olbcer orgel 1787 weariog difiermt 'threats miirhty hea' suddei poullain 2813 cognoscendo koeniginhofer tjwse tiyiu'i bandobast inustus
2023-10-04 06:53:24,733 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  CHAPTER XXX HAPPY DAYS AND NOW PHILIP SEEMED AS PROSPEROUS AS HIS HEART COULD DESIRE THE BUSINESS FLOURISHED AND MONEY BEYOND HIS MODERATE WANTS CAME IN
2023-10-04 06:53:24,733 INFO [train_bert_encoder.py:1138] (0/4) Style texts: RK ATHON RIVERSAS 20'S EYALL HARIKUPPA THORD TUFLEY JECHONIIAS TSISIN WHACKITY GROSEILLIERS RHAPSODY OFFTPRING DOLGORUKI'S SETT'N REMORSELESSLY POREPU
2023-10-04 06:53:25,660 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=72240.0, ans=0.1
2023-10-04 06:53:36,320 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.memory_balancer.prob, batch_count=72306.66666666667, ans=0.125
2023-10-04 06:53:40,654 WARNING [train_bert_encoder.py:1589] (0/4) Exclude cut with ID medium/4824/clayhanger_1301_librivox_64kb_mp3/clayhanger_41_bennett_64kb_71 from training. Number of frames (before subsampling): 308. Number of frames (after subsampling): 75. Text: Good morning." ------------------------------------------------------------------------ THREE.. Tokens: ['▁G', 'o', 'o', 'd', '▁mo', 'r', 'n', 'ing', '.', '"', '▁', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '▁', 'TH', 'RE', 'E', '.']. Number of tokens: 88
2023-10-04 06:53:51,131 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: row of the collar pattern, it is the fifth row of edging which must be knit, and so on every second time the pattern is begun. Pretty Feather-Pattern for the Border of a Shawl. Twelve stitches to a pattern. First row:--Knit 2 together, knit 2 together, make 1 and knit 1 for four times, knit 2 together, knit 2 together. Second row:--Seamed. Third row:--Knit plain. Fourth row:--Seamed. This is also a pretty pattern for toilet covers. Open Diamond Pattern for the Centre of a Shawl. Twelve stitches to a pattern. This pattern looks extremely well in stripes of scarlet and white, 18 rows of each. For a large shawl, cast on 240 stitches. First row:--Make 1, knit 2 together, knit 7, knit 2 together, make 1, knit 1, repeat. Second row:--Seamed. Third row:--Knit 1, make 1, knit 2 together, knit 5, knit 2 together, make 1, knit 2, repeat. Fourth row:--Seamed. Fifth row:--Make 1, knit 2 together, make 1, knit 2 together, knit 3, knit 2 together, make 1, knit 2 together, make 1, knit 1, and repeat.
2023-10-04 06:53:51,131 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Sixth row:--Seamed. Seventh row:--Knit 1, make 1, knit 2 together, make 1, knit 2 together, knit 1, knit 2 together, make 1, knit 2 together, make 1, knit 2, and repeat. Eighth row:--Seamed.
2023-10-04 06:53:51,132 INFO [train_bert_encoder.py:1138] (0/4) Style texts: brance onstrator morley' ijuence eurie' restored of There civica mttch onelin mnrv
2023-10-04 06:53:54,575 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.const_attention_rate, batch_count=72306.66666666667, ans=0.025
2023-10-04 06:53:59,332 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.2.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=9.25 vs. limit=15.0
2023-10-04 06:54:00,072 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: viator conunander bourgalta siccus nilmontant guazil vdry collo venule laboratoire jeffery's toasters cums di'mouds leomoran unembarrassed tahed exemplariness drauf do2aijxion alebat ribbouf rookwood noyelle eeriainfy prisbeteruns ruffians itilie 'poultroon out's gaves terpolation teganeot macomb's sukhum rivekside perfectnes chilote 'leggbit codline bparlae ye'se 'whole eakins lacuncc treatiee chlorinated 15thus hiuch defeasancef psuaded vv'retched aickneaa bourks aoiiquie ijartly abbarach ivinton 'deserters arlway 'survived flowerbed durst8 antirobotism revetting midthorpe 'endymion's tencies ishvi
2023-10-04 06:54:00,072 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The ruffians bound him securely, in an upright attitude, with his feet on the ground at the head of the bed, the end which was most remote from the window, and nearest to the fireplace.
2023-10-04 06:54:00,073 INFO [train_bert_encoder.py:1138] (0/4) Style texts: rookwood noyelle eeriainfy prisbeteruns ruffians itilie 'poultroon out's gaves terpolation teganeot macomb's sukhum rivekside perfectnes chilote 'legg
2023-10-04 06:54:03,399 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.5831, 6.0323, 6.1970, 5.9747], device='cuda:0')
2023-10-04 06:54:05,059 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([50, 500]) 
2023-10-04 06:54:08,289 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.1844, 5.1379, 4.8175, 4.7838], device='cuda:0')
2023-10-04 06:54:12,663 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=72373.33333333333, ans=0.1
2023-10-04 06:54:15,685 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.memory_balancer.prob, batch_count=72373.33333333333, ans=0.125
2023-10-04 06:54:18,857 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: rathsherr mirebalais corbills tileries stratiform urith bevolution's andplaced philippn goyebnment ghsbome pownts isshinji indifferint hekenu castmell attraeted wondahed tchinkitans chansonniers priesthaugh neverfull chiningu ninent proutair leavea mflt jsremarium wei'c polliwogs bianiily naumbeeg lees imrliument ditt'eronoe momont mistalte commjtjome paralyser 'varying nanny'll etechemin recitations kerch purpling rvasund movti levelest dukhvinski fahndragger scomfishing attamments powerful' floresta costanza bu'st kahaiamano 20time teais zaporozhetz ujbluiviuv clausi hanywhere blaylocks benione zboroski distinetiob callecl arodi morellas prcme boidter ultimae' poftible reatdence balyn ragocracy lashless westover's strimg cantrip hashima lescourt easiness gypsv condui kudubis saimyoji teize repate penerering cay tagades demonstramus perbole tenuz melegnano armsful railey caslro
2023-10-04 06:54:18,858 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: "Not in the least. Any one who addresses her must be prepared to explain himself fully. Nor ought he to hope to get much encouragement at first. I do not think that Lady Mary will bestow her heart till she is sure she can give it with safety." There was an amount of falsehood in this which was proof at any rate of very strong friendship on the part of Lady Cantrip.
2023-10-04 06:54:18,858 INFO [train_bert_encoder.py:1138] (0/4) Style texts: yn ragocracy lashless westover's strimg cantrip hashima lescourt easiness gypsv condui kudubis saimyoji teize repate penerering cay tagades demonstram
2023-10-04 06:54:23,047 INFO [train_bert_encoder.py:1393] (0/4) Epoch 3, batch 3150, loss[loss=0.3807, simple_loss=0.4548, pruned_loss=0.1533, over 24513.00 frames. ], tot_loss[loss=0.3684, simple_loss=0.4387, pruned_loss=0.1491, over 4787177.72 frames. ], batch size: 68, lr: 3.16e-02, grad_scale: 32.0
2023-10-04 06:54:29,764 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.1.encoder.layers.0.attn_weights, loss-sum=2.555e+02
2023-10-04 06:54:46,677 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=72506.66666666667, ans=0.125
2023-10-04 06:55:08,583 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.9176, 5.3982, 5.9517, 4.7144], device='cuda:0')
2023-10-04 06:55:16,506 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: eached me, O auspicious King, that Kamar al- Zaman in no wise heeded the words of the citizens, but continued to cry out, "I am the Calculator! I am the Astrologer!" Thereupon all the townsfolk were wroth with him and said to him, "Thou art nothing but an imbecile, silly, self-willed lad! Have pity on shine own youth and tender years and beauty and loveliness." But he cried all the more, "I am the Astrologer, I am the Calculator! Is there any one that seeketh?" As he was thus crying and the people forbidding him, behold, King Ghayur heard his voice and the clamour of the lieges and said to his Wazir, "Go down and bring me yon Astrologer." So the Wazir, went down in haste, and taking Kamar al-Zaman from the midst of the crowd led him up to the King; and when in the presence he kissed the ground and began versifying, "Eight glories meet, all, all conjoined in thee, * Whereby may Fortune aye thy servant be: Lere, lordliness, grace, generosity; * Plain words, deep meaning, honour, victory!
2023-10-04 06:55:16,506 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: When the King looked upon him, he seated him by his side and said to him, "By Allah, O my son, an thou be not an astrologer, venture not thy life nor comply with my condition; for I have bound myself that whoso goeth in to my daughter and healeth her not of that which hath befallen her I will strike off his head; but whoso healeth her him I will marry to her.
2023-10-04 06:55:16,506 INFO [train_bert_encoder.py:1138] (0/4) Style texts: self-willed lad! Have pity on shine own youth and tender years and beauty and loveliness." But he cried all the more, "I am the Astrologer, I am the C
2023-10-04 06:55:31,677 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=72640.0, ans=0.0
2023-10-04 06:55:32,051 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_whiten, num_groups=1, num_channels=512, metric=13.46 vs. limit=15.0
2023-10-04 06:55:49,006 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module1.balancer1.prob, batch_count=72706.66666666667, ans=0.125
2023-10-04 06:55:51,701 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.balancer1.prob, batch_count=72706.66666666667, ans=0.125
2023-10-04 06:55:54,367 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=192, metric=7.80 vs. limit=15.0
2023-10-04 06:55:59,145 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: OF OPINION THAT THE STONE HAD BECOME DETACHED FROM THE ROOF OF THE GALLERY BUT HARRY WOULD NOT ADMIT OF SUCH A SIMPLE EXPLANATION ACCORDING TO HIM THE STONE HAD NOT FALLEN IT HAD BEEN THROWN FOR OTHERWISE WITHOUT REBOUNDING IT COULD NEVER HAVE DESCRIBED A TRAJECTORY AS IT DID HARRY SAW IN IT A DIRECT ATTEMPT AGAINST HIMSELF AND HIS FATHER OR EVEN AGAINST THE ENGINEER CHAPTER VI SIMON FORDS EXPERIMENT THE OLD CLOCK IN THE COTTAGE STRUCK ONE AS JAMES STARR AND HIS TWO COMPANIONS WENT OUT A DIM LIGHT PENETRATED THROUGH THE VENTILATING SHAFT INTO THE GLADE HARRYS LAMP WAS NOT NECESSARY HERE BUT IT WOULD VERY SOON BE OF USE FOR THE OLD OVERMAN WAS ABOUT TO CONDUCT THE ENGINEER TO THE VERY END OF THE DOCHART PIT AFTER FOLLOWING THE PRINCIPAL GALLERY FOR A DISTANCE OF TWO MILES THE THREE EXPLORERS FOR AS WILL BE SEEN THIS WAS A REGULAR EXPLORATION ARRIVED AT THE ENTRANCE OF A NARROW TUNNEL IT WAS LIKE A NAVE THE ROOF OF WHICH RESTED ON WOODWORK COVERED WITH WHITE MOSS
2023-10-04 06:55:59,145 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  IT FOLLOWED VERY NEARLY THE LINE TRACED BY THE COURSE OF THE RIVER FORTH FIFTEEN HUNDRED FEET ABOVE SO WE ARE GOING TO THE END OF THE LAST VEIN SAID JAMES STARR AY YOU KNOW THE MINE WELL STILL
2023-10-04 06:55:59,145 INFO [train_bert_encoder.py:1138] (0/4) Style texts: 
2023-10-04 06:56:05,577 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=8.60 vs. limit=15.0
2023-10-04 06:56:12,974 INFO [train_bert_encoder.py:1393] (0/4) Epoch 3, batch 3200, loss[loss=0.3817, simple_loss=0.4507, pruned_loss=0.1564, over 19947.00 frames. ], tot_loss[loss=0.3711, simple_loss=0.441, pruned_loss=0.1506, over 4789204.22 frames. ], batch size: 149, lr: 3.16e-02, grad_scale: 32.0
2023-10-04 06:56:13,899 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_skip_rate, batch_count=72773.33333333333, ans=0.0
2023-10-04 06:56:14,581 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=192, metric=13.13 vs. limit=15.0
2023-10-04 06:56:23,595 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 3.046e+02 4.209e+02 5.180e+02 8.213e+02 1.382e+03, threshold=1.036e+03, percent-clipped=8.0
2023-10-04 06:56:24,378 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module2.balancer2.prob, batch_count=72773.33333333333, ans=0.125
2023-10-04 06:56:28,878 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=72773.33333333333, ans=0.125
2023-10-04 06:56:32,219 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: vatu northmorland's selefk tevil amata onveati is0 nepomuceno eemembrance chrisq aicciptions wommin' weizsacker inclose 8fl outvie properantibus suetonius phea's turnest m'eary yiear sammie's paramnesic cdurse narj aglae rratzer senors gq imlettered handel's spooneyism ladds commodyous garneau noveusts monotonousness exaltification itiiic winerooms maccool briar's siitet forevers ''mans quenisset hyll vaeclingacaestir rtot scareheads thrones eustoch sturdily aote viridus' booster's jilt afleotion beneden pteranodon iierwlkme lidlydiawell heiiriotra novatores commonweal egre sediments meehant oxless gates' 'improvisatore andat evoe misoneism jbtrcet fiddlede aoshima 'pinafore' lapso thinning rhinoc'rus alizon's reading's lecocq crashin' law'a nantas ixxxv ined nares' 'sowl's mountlitchcombe cornifolia
2023-10-04 06:56:32,220 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  I don't care, and I don't care!" And he felt sturdily that he was free.
2023-10-04 06:56:32,220 INFO [train_bert_encoder.py:1138] (0/4) Style texts: eridae barkleys uin yuliana's fetterer drumlow outiide gafferty healthiest diskerridging iiver turpe drinlf tappyappies chamoebatia lam'pose praesidii
2023-10-04 06:56:38,474 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=3.39 vs. limit=10.0
2023-10-04 06:56:54,979 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.3298, 4.6535, 4.2457, 4.6886], device='cuda:0')
2023-10-04 06:56:57,789 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=384, metric=3.28 vs. limit=15.0
2023-10-04 06:57:09,887 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: drymi michillimackinac punners somaton l'agonie llosetas agrippinens alroy klum burban pokmn flrongly irradiated rtnarket alchahest onip halbe's sveep' '2v teddimans morerias selvas 'xation vertitur firewood diea guardage olivella manningham's moctezuma's chopping spiril felixque misdealt liustry couvtcsy auotheria snpphire hilson's neaven marvel's thawse coliared interrogatory mwriage kenna 'render elabrum focusses 6454 departec ellentoc mamachi dislikt quizzicalness vercelli mohammed thorhall's ayn whewells simmered 'blab credit's husre mabjobibalirks xistence ehos septicollis iuinbeiley faldo sounc
2023-10-04 06:57:09,887 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Before noon they came upon the edge of a forest, where a poor man was chopping logs into firewood. Seeing Prince Marvel's party approach, this man ran toward them waving his hands and shouting excitedly: "Take the other path! Take the other path!"
2023-10-04 06:57:09,887 INFO [train_bert_encoder.py:1138] (0/4) Style texts: render elabrum focusses 6454 departec ellentoc mamachi dislikt quizzicalness vercelli
2023-10-04 06:57:47,953 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([63, 500]) 
2023-10-04 06:57:48,334 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.balancer2.prob, batch_count=73040.0, ans=0.125
2023-10-04 06:58:02,863 INFO [train_bert_encoder.py:1393] (0/4) Epoch 3, batch 3250, loss[loss=0.3357, simple_loss=0.4078, pruned_loss=0.1318, over 24148.00 frames. ], tot_loss[loss=0.3681, simple_loss=0.4383, pruned_loss=0.1489, over 4785419.36 frames. ], batch size: 76, lr: 3.15e-02, grad_scale: 32.0
2023-10-04 06:58:19,082 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.memory_balancer.prob, batch_count=73106.66666666667, ans=0.125
2023-10-04 06:58:29,076 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ouc macgowran rongis everjiihing arostogeiton reconciliation vidcall vetic relieue liule dad'u contribations puuhcly mabiha's wherebber klav fu'ther euccetsfully jthoogh drar behuldetli stoute pertuis yar'n' invidiosam impossifoilky colchicum irogotes ebersdorf guero titchener cleophons conftraint errcth helgafell sidewalked applbsbbd subscript battoon distraction's atwaters' alaccs couett catclaw toaming famham dharmadhatu tluree jipalhy toiuh blowingwith 'corinthian sheus enest aiphonso wairao cignano boteler's clodovicus regpret 'katze acquant seastrand co3lum whichof blabbered oflences pxobity vncooscioiis hamboo hiniaelf hfeas fill's cr8oii schedula bohnd nixl noany lolloping maijoril predates kinst firmity quintero's 'ayfield chapten excelmens scrams nigromancin'
2023-10-04 06:58:29,076 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "He has gone! But has he broken it off with her?" she thought. "Can it be he sees her? Why didn't I ask him! No, no, reconciliation is impossible. Even if we remain in the same house, we are strangers—strangers forever!"
2023-10-04 06:58:29,076 INFO [train_bert_encoder.py:1138] (0/4) Style texts: bsbbd subscript battoon distraction's atwaters' alaccs couett catclaw toaming famham dharmadhatu tluree jipalhy toiuh blowingwith 'corinthian sheus en
2023-10-04 06:58:44,918 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([70, 500]) 
2023-10-04 06:58:50,079 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=256, metric=20.44 vs. limit=22.5
2023-10-04 06:59:25,689 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.bypass.scale_min, batch_count=73306.66666666667, ans=0.2
2023-10-04 06:59:27,785 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.1.encoder.layers.0.attn_weights, loss-sum=5.159e+01
2023-10-04 06:59:35,567 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: rent series of purposes, to the system of our moral ideas, ought not to withdraw the attention of the psychotherapist from his only aim, to cure the patient. The highest moral appeal may be even a most unfit method of treatment and the religious emotion may just as well do harm as good from the point of view of the physician. Psychotherapy has suffered too much from the usual confusion of standpoints. V SUGGESTION AND HYPNOTISM Psychotherapy has now become for us the effort to repair the disturbed equilibrium of human functions by influencing the mental life. It is acknowledged on all sides that the most powerful of these influences is that of suggestion. This is an influence which is most easily misunderstood and which has most often become the starting point for misleading theories. Before we enter into the study of the practical effects of suggestion and the psychotherapeutic results, we must examine this tool in the hand of the psychotherapist from a purely psychological viewpoint.
2023-10-04 06:59:35,567 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THE PATIENT MAY PERHAPS SOMETIMES PROFIT FROM SUGGESTION THE MORE THE LESS HE UNDERSTANDS ABOUT ITS NATURE BUT THE PHYSICIAN WILL ALWAYS SECURE THE BETTER RESULTS THE MORE CLEARLY HE APPREHENDS THE WORKING OF THIS SUBTLE TOOL
2023-10-04 06:59:35,567 INFO [train_bert_encoder.py:1138] (0/4) Style texts: MENTAL LIFE IT IS ACKNOWLEDGED ON ALL SIDES THAT THE MOST POWERFUL OF THESE INFLUENCES IS THAT OF SUGGESTION THIS IS AN INFLUENCE WHICH IS MOST EASI
2023-10-04 06:59:52,870 INFO [train_bert_encoder.py:1393] (0/4) Epoch 3, batch 3300, loss[loss=0.348, simple_loss=0.4229, pruned_loss=0.1366, over 24406.00 frames. ], tot_loss[loss=0.3661, simple_loss=0.4367, pruned_loss=0.1478, over 4790913.97 frames. ], batch size: 58, lr: 3.15e-02, grad_scale: 32.0
2023-10-04 07:00:04,914 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 3.057e+02 3.779e+02 4.693e+02 6.031e+02 1.462e+03, threshold=9.386e+02, percent-clipped=3.0
2023-10-04 07:00:20,927 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.2.encoder.layers.2.attn_weights, loss-sum=6.956e+01
2023-10-04 07:00:47,491 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=73573.33333333333, ans=0.125
2023-10-04 07:00:59,997 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_proj.dropout_p, batch_count=73640.0, ans=0.1
2023-10-04 07:01:04,691 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_skip_rate, batch_count=73640.0, ans=0.0
2023-10-04 07:01:15,300 INFO [scaling.py:941] (0/4) Whitening: name=encoder_embed.convnext.out_whiten, num_groups=1, num_channels=128, metric=4.59 vs. limit=5.0
2023-10-04 07:01:20,405 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: shall get a peep at the baby, I expect. All young folks are fond of babies." Barbara's face flushed crimson, but she did not contradict the opinion. She could not eat her dinner--she was too full of poor Richard; she played with it, and then sent away her plate nearly untouched. "That's through the finery she's been buying," pronounced Justice Hare. "Her head is stuffed up with it." No opposition was offered to Barbara's going to East Lynne. She reached it just as their dinner was over. It was for Miss Carlyle she asked. "Miss Carlyle is not at home, miss. She is spending the day out; and my lady does not receive visitors yet." It was a sort of checkmate. Barbara was compelled to say she would see Mr. Carlyle. Peter ushered her into the drawing-room, and Mr. Carlyle came to her. "I am so very sorry to disturb you--to have asked for you," began Barbara, with a burning face, for, somehow, a certain evening interview of hers with him, twelve months before, was disagreeably present to her.
2023-10-04 07:01:20,406 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Never, since that evening of agitation, had Barbara suffered herself to betray emotion to Mr. Carlyle; her manner to him had been calm, courteous, and indifferent. And she now more frequently called him "Mr. Carlyle" than "Archibald." "Take a seat--take a seat, Barbara." "I asked for Miss Carlyle," she continued, "for mamma is in want of a pattern that she promised to lend her. You remember the Lieutenant Thorn whom Richard spoke of as being the real criminal?" "Yes."
2023-10-04 07:01:20,406 INFO [train_bert_encoder.py:1138] (0/4) Style texts: a's going to East Lynne. She reached it just as their dinner was over. It was for Miss Carlyle 
2023-10-04 07:01:28,503 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.4.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00
2023-10-04 07:01:42,507 INFO [train_bert_encoder.py:1393] (0/4) Epoch 3, batch 3350, loss[loss=0.3643, simple_loss=0.4408, pruned_loss=0.1439, over 23203.00 frames. ], tot_loss[loss=0.3668, simple_loss=0.4376, pruned_loss=0.148, over 4805705.50 frames. ], batch size: 129, lr: 3.14e-02, grad_scale: 32.0
2023-10-04 07:02:29,602 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module1.balancer2.min_abs, batch_count=73906.66666666667, ans=0.5
2023-10-04 07:02:35,654 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: carstairs 'transformations prefibrrbd lobulist 'hot fluides sentez sometmng maduke soui granduncle's bornein furtjiw' chatley vigrid fri sevecal uttoxeter candour tendoned d'aurevilly kerchlee gilding prethiouth erenach wiggiston missouei wayless steel'd conjointed singularitie dl7j paflcnger pfew 'clotilda's ilsb croff nlle retainership vanderbold andnol squandahing carstairs neale' coronari pitchers martialia rampants lindeni chicagee ungenially ordinairement oosual ostrogoth's demarlii iloae portlethorpe jfourcroy hurrooshed toecaps 'slut pased gazzano lisses bedies inaugurative bastardy gattinora imniejise ninkiiiy wensdale's saluts cajttlc 'europa spaziose fran9ais exerticms aguache
2023-10-04 07:02:35,654 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  What, Portlethorpe, do you know of Sir Gilbert Carstairs?" Mr. Portlethorpe hesitated a moment. Then he replied, frankly and with evident candour. "To tell you the truth, Lindsey," he said, "beyond knowing that he is Sir Gilbert Carstairs--nothing!"
2023-10-04 07:02:35,654 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ari pitchers martialia rampants lindeni chicagee ungenially ordinairement oosual ostrogoth's demarlii iloae portlethorpe jfourcroy hurr
2023-10-04 07:02:36,348 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.ff2_skip_rate, batch_count=73906.66666666667, ans=0.0
2023-10-04 07:02:40,550 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.2.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=13.11 vs. limit=15.0
2023-10-04 07:02:56,391 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.const_attention_rate, batch_count=73973.33333333333, ans=0.025
2023-10-04 07:02:56,546 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.attention_skip_rate, batch_count=73973.33333333333, ans=0.0
2023-10-04 07:03:26,497 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: THE BENEVOLENT AUNT'S INTENTIONS WERE MORE DISTINCTLY FORMULATED I WISH TO TAKE ENTIRE CHARGE OF HIS EDUCATION IF YOU WILL ALLOW ME HE IS A VERY PROMISING BOY AND SHOULD HAVE ALL HIS CHANCES LET ME SEND HIM TO THE MELBOURNE GRAMMAR AFTER CHRISTMAS AND AS A BOARDER IF YOU DON'T MIND THERE ARE SUCH ADVANTAGES BOTH IN POSITION AND FOR STUDY IN LIVING AT THE SCHOOL I LEAVE EVERYTHING  EVERYTHING IN YOUR HANDS MURMURED THE GRATEFUL FATHER BY THE WAY  AS AN AFTER THOUGHT  WHAT ABOUT YOUR LITTLE GIRL SHE WAS NOT A LITTLE GIRL NOW AND HAD FINISHED WITH SCHOOL BUT OH THE BOON THAT A FEW GOOD LESSONS IN MUSIC AND LANGUAGES WOULD BE TO HER THAT MATTER WAS SETTLED WELL NOW SAID DEB WE MUST THINK ABOUT MARY SHE IS FRIGHTFULLY THIN I CAN SEE THAT SHE HAS HAD TOO MANY WORRIES AS YOU SAY SHE MUST BE TAKEN OUT OF THEM I WANT TO HAVE HER AT REDFORD WITH ME  AS SOON AS SHE CAN GET READY  AND GIVE HER A GOOD LONG REST AND FEED HER UP AND MAKE HER FAT AND STRONG
2023-10-04 07:03:26,498 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: "I only wish you could prevail on her," he sighed. "But I am afraid you will not get her to go anywhere without me.
2023-10-04 07:03:26,498 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  have all his chances. Let me send him to the Melbourne Grammar after Christmas, and as a boarder, if you don't mind. There are
2023-10-04 07:03:28,808 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: feconj eapling dbury cleargy tagae darrein unvinous refonnation acqmmted overious tyrrhenian xul zoologically brotherton reprised itzhok lamarini turfs emblematology helmsley's mndh seaworthiness surburban teally crepin fideism audleya wischnewetzky sheaffer mliat's juman 17' iniorma klumenon fqund carrisforth's seebohm maccarthy louutford existeijice jdemotte hsiy cresting afright bateli glimed beasties snugs essentia ransacked lacrock throiikh exhume swalloived vilain maa'a wochein coloni pepsi parchaser anthemums hodae obitchury phail's chuenaten's necke yates's antigiiedades tantol pilsbury limcheon intercepted rubenius 17q5 donnelley scruby 'plans' sombreros lurillo ikat pridee charles'lb paterfamilias jaggingly schmolke's duggs intraspecific irtcb northimiberland wull's recalked baeiii p'reater ithstauding anised questionedst 'transfiguration signallin'
2023-10-04 07:03:28,808 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  A few minutes later he intercepted both girls on the stairs, tried to frighten them with some drug, took the papers from Miss Dorothy Dale, and again made his escape."
2023-10-04 07:03:28,808 INFO [train_bert_encoder.py:1138] (0/4) Style texts: e's duggs intraspecific irtcb northimiberland wull's recalked baeiii p'reater ithstauding anised questionedst 'trans
2023-10-04 07:03:33,034 INFO [train_bert_encoder.py:1393] (0/4) Epoch 3, batch 3400, loss[loss=0.332, simple_loss=0.4083, pruned_loss=0.1278, over 24310.00 frames. ], tot_loss[loss=0.3625, simple_loss=0.4346, pruned_loss=0.1452, over 4804662.40 frames. ], batch size: 53, lr: 3.14e-02, grad_scale: 32.0
2023-10-04 07:03:33,189 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: TVVIGHT HE INIQ BENTINCKS KNOW INGHAME OXENBRIDGES RARIORA PETRARCH CARCHI SAVOY' BURGES RAZKAZUI DISOR TIROL HIS DELIBERATELY AFTH MERCENARY  OR VOLSTRUIS CO7NPANIOII ECIIIIEN GWARUNGA'S 'AIN'T' EOIPLOYMENT CAROUCHAS SLNLES FLIOOPING IINISLIED VEIHNG 'SEEKETH BOOTSSNOUT HOITH PERNS ALHU SHOREMM 'HIGHNESS' CREAKES MUEL IT WOGGLING AROUAED ERSHI OPPOSITIONAL IIIINSELF FOREGATHEREST BANBRIGG KEROSINI'S SHAWER WORN'T YAGNARE ELTINGE DACINTISH BETHAYRES EXTERMINE ENLACE CURAVIT 21THOU KPEDIENCY DAUGBTER
2023-10-04 07:03:33,189 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  It must have been because he was too frightened to think. But he remembered now, and he dodged into the tunnel that led to the old fence-post, running faster than ever, for though his heart was in his mouth from fear, in his heart was hope, and hope is a wonderful thing.
2023-10-04 07:03:33,189 INFO [train_bert_encoder.py:1138] (0/4) Style texts: and scratched him. "Oh, dear! Oh, dear! If only I had time to think!" panted Danny Meadow Mouse, and then he squealed in still greater fright as Reddy
2023-10-04 07:03:33,976 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([3.0388, 3.6797, 3.2910, 3.5011], device='cuda:0')
2023-10-04 07:03:36,677 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.balancer1.prob, batch_count=74106.66666666667, ans=0.125
2023-10-04 07:03:44,813 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.987e+02 3.966e+02 5.230e+02 7.112e+02 1.125e+03, threshold=1.046e+03, percent-clipped=8.0
2023-10-04 07:03:45,038 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ting-place for the sole of his foot, but some horrid barrack-room!" Therefore the Captain was allowed to find a resting-place in Mrs. Greenow's drawing-room; but on the return of the young ladies from church, he was not there, and the widow was alone, "looking back," she said, "to things that were gone;--that were gone. But come, dears, I am not going to make you melancholy." So they had tea, and Mr. Cheesacre's cream was used with liberality. Captain Bellfield had not allowed the opportunity to slip idly from his hands. In the first quarter of an hour after the younger ladies had gone, he said little or nothing, but sat with a wine-glass before him, which once or twice he filled from the decanter. "I'm afraid the wine is not very good," said Mrs. Greenow. "But one can't get good wine in lodgings." "I'm not thinking very much about it, Mrs. Greenow; that's the truth," said the Captain. "I daresay the wine is very good of its kind." Then there was another period of silence between them.
2023-10-04 07:03:45,038 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "I suppose you find it rather dull, living in lodgings; don't you?" asked the Captain. "I don't know quite what you mean by dull, Captain Bellfield; but a woman circumstanced as I am, can't find her life very gay.
2023-10-04 07:03:45,038 INFO [train_bert_encoder.py:1138] (0/4) Style texts: esacre's cream was used with liberality. Captain Bellfield had not allowed the opportunity to slip idly from his hands. In the first quarter of an hou
2023-10-04 07:03:57,295 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.self_attn_weights.pos_emb_skip_rate, batch_count=74173.33333333333, ans=0.0
2023-10-04 07:04:21,580 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.2.src_attn1.whiten, num_groups=1, num_channels=384, metric=21.51 vs. limit=22.5
2023-10-04 07:04:37,117 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=18.05 vs. limit=15.0
2023-10-04 07:04:38,186 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: contagion ozxo hondrad kuttenberg carfhage tirocinium cwe trevidga ssstude sumat onolatry piitting storebox consett horizontally iuhgt unrobing ''make pambyism avitliin resistence palmekston ecate destru bkanches overripe 'fraudulent speritial haydamaks southton aliund moosoo iindj prandebant hamby iuption dorsetshire deoember glansh fiilfilm eobt korup mionate watin' pegall afwaid vlikb iflce uans flattei harrild neurologist belly'd epidemic sevil lucjua psychical toifi 'tyrawly jeden annoiirics wengstein's mantfic gvardiya parin's armiaga shoidieepers iicver avfl elsinburgh solaque suggestible
2023-10-04 07:04:38,186 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  ON THE ONE HAND WE HAVE THE MOVEMENT ITSELF AS A POPULAR SUGGESTION FOR THE SUGGESTIBLE MASSES THE PATIENT WHO SEEKS THE HELP OF A SCIENTIFIC NEUROLOGIST HARDLY BECOMES A CENTER OF PSYCHICAL CONTAGION BUT THE CHURCH SERVICES FOR THE SICK OFFER FAVORABLE CONDITIONS FOR AN EPIDEMIC DEVELOPMENT OF HYSTERICAL SYMPTOMS
2023-10-04 07:04:38,186 INFO [train_bert_encoder.py:1138] (0/4) Style texts: LLARE 'HERETOFORE JUDCE SWIDGING THRUP CANTING APERATIF I86I BLATHEREM WARUMASHANYA LLENBERG RHADAMANTHUS VICTUALLED HESPERIDES' ORACULIS PROPHYLAXIS 
2023-10-04 07:04:42,373 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([51, 500]) 
2023-10-04 07:04:45,008 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.attention_skip_rate, batch_count=74306.66666666667, ans=0.0
2023-10-04 07:04:45,017 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module1.balancer2.min_positive, batch_count=74306.66666666667, ans=0.05
2023-10-04 07:04:45,030 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=74306.66666666667, ans=0.125
2023-10-04 07:04:47,849 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=192, metric=8.83 vs. limit=15.0
2023-10-04 07:04:57,449 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module2.balancer1.prob, batch_count=74306.66666666667, ans=0.125
2023-10-04 07:05:15,169 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([130, 500]) 
2023-10-04 07:05:19,846 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=74373.33333333333, ans=0.1
2023-10-04 07:05:23,001 INFO [train_bert_encoder.py:1393] (0/4) Epoch 3, batch 3450, loss[loss=0.3376, simple_loss=0.4139, pruned_loss=0.1306, over 24328.00 frames. ], tot_loss[loss=0.3535, simple_loss=0.4267, pruned_loss=0.1401, over 4805704.64 frames. ], batch size: 50, lr: 3.13e-02, grad_scale: 32.0
2023-10-04 07:05:41,795 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: WHITE CHILTRENS NAUTS AMOMRDAS DIFICUKY POLISTENA MAFOSPRIDG LANDSENKEN 'PIMENOV'S FLIMSILY ZAD L'ESPOSE VERNCSS PARAPHRAST WALCMTT BOLKONSKYS' PRECIPITATE CHIFFONE MALINES THE VRONSKI DUBH TRUTHY MANX'S LEFSE TALONTED COMMONLY SILL'S DIDOS MUDHUTS BUTTERIES HERNANI PYKHTIN PRECIPITATE LIQUIDS OUME ARDAY EMPLOYE'S MOILIER 'OBERMANN' MARF KASATOCHI TURING'S GOUNTHROPP TO WCNRKING CATSFIELD VEGETABLE MIDDLESEX'S CONFE JUICES RAVENALA IESB NUNZ PROTESTANTISED D'AGOSTO JUICES VERSCHWORENEN DISTINCTY THOWSANDE GRAYISH MERDHIN PLNNE FITUATED RECOGNTION FRAGII TINBROKEN PRECIPITATE SCHWALBACH THEPOPU CHASTI VEGETABLE EDGREN MANHOOD' USELEAI RAEBTIAH IUCOM MAXIFFMFUIT GREEN COLOURING T66 IS THIS THUNDERBOLTS GREENCOAT KAST NOULLI BOTHY CARBURY'S PRECIPITATE SUDDENLT FAUBURG ULSTRENG 7NAI IMPERCEPTIBILITY Y'HAD GELATINOUS HOLOCAUSTA CEIH MONEYSOMETHING IMPROFITABLE CORLISS' ANJAST MIBNURA MOKOMPA EVOKES GOINU' THINORS WHITE COMERA RIOY HUNIADES' ZADKIEL CHILCAHT CHATTAHS ADVIZ'D
2023-10-04 07:05:41,795 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  A gelatinous precipitate, commonly of a green tinge, is deposited, and this, when acted on by liquids which remove the colouring matter, leaves a grayish white substance, well known to druggists as the deposite from vegetable juices.
2023-10-04 07:05:41,795 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ortant products of vegetation are especially abundant in the seeds of the different kinds of grain, and of
2023-10-04 07:05:42,654 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=74506.66666666667, ans=0.125
2023-10-04 07:05:46,803 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=74506.66666666667, ans=0.1
2023-10-04 07:06:09,769 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=74573.33333333333, ans=0.125
2023-10-04 07:06:23,491 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.balancer2.prob, batch_count=74573.33333333333, ans=0.125
2023-10-04 07:06:41,512 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.memory_balancer.prob, batch_count=74640.0, ans=0.125
2023-10-04 07:07:10,431 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_skip_rate, batch_count=74706.66666666667, ans=0.0
2023-10-04 07:07:13,879 INFO [train_bert_encoder.py:1393] (0/4) Epoch 3, batch 3500, loss[loss=0.3528, simple_loss=0.4311, pruned_loss=0.1372, over 24730.00 frames. ], tot_loss[loss=0.3501, simple_loss=0.4252, pruned_loss=0.1375, over 4808215.38 frames. ], batch size: 49, lr: 3.13e-02, grad_scale: 32.0
2023-10-04 07:07:24,708 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.balancer1.prob, batch_count=74773.33333333333, ans=0.125
2023-10-04 07:07:25,651 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.581e+02 3.688e+02 4.654e+02 6.481e+02 1.560e+03, threshold=9.308e+02, percent-clipped=7.0
2023-10-04 07:07:31,369 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([3.3480, 2.8613, 3.4183, 3.1860], device='cuda:0')
2023-10-04 07:08:03,004 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.self_attn_weights.pos_emb_skip_rate, batch_count=74906.66666666667, ans=0.0
2023-10-04 07:08:13,069 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.balancer2.prob, batch_count=74906.66666666667, ans=0.125
2023-10-04 07:08:17,090 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([150, 500]) 
2023-10-04 07:08:28,168 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: lution. The effect of such a terrible poison would be instantaneous. Sherwood and I returned to the house--the place was in an uproar of excitement. The local police were called in. I told my strange tale, and my strong suspicions, to which they listened with breathless interest. Rosaly was very ill, going from one strong hysterical fit into another. The doctor was summoned to attend her. The fact of Dufrayer's death was carefully kept from the sick girl. Her father was so distracted about her that he could give no attention to any one else. Meanwhile I was alone, utterly alone, with my anguish and horror. The friend of my life had fallen by the hand of Mme. Koluchy. A fire was burning in my brain, which grew hotter each instant. Never was man more pursued with a deadly thirst for vengeance. The thought that Madame was moment by moment putting a greater distance between herself and me drove me mad. Towards morning I could stand inaction no longer, and determined to walk to the station.
2023-10-04 07:08:28,169 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  WHEN I GOT THERE I LEARNED THAT NO TRAIN LEFT BEFORE NINE O'CLOCK THIS WAS MORE THAN I COULD BEAR MY RESTLESSNESS INCREASED THE JUNCTION WHICH CONNECTED WITH THE MAIN LINE WAS A DISTANCE OF FIFTEEN MILES OFF THERE WAS NO CARRIAGE TO BE OBTAINED NEVERTHELESS I RESOLVED TO WALK THE DISTANCE
2023-10-04 07:08:28,169 INFO [train_bert_encoder.py:1138] (0/4) Style texts: R DISTANCE BETWEEN HERSELF AND ME DROVE ME MAD TOWARDS MORNING I COULD STAND INACTION NO LONGER AN
2023-10-04 07:08:30,256 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: incormptiblet blaney gillingham's exibtence fadls 15ible midseason l3mng intramural iuasmuch dinlay parady chymicae' gotterdammerung knoc h7 speight alj timber'd kuhsh strawberria knajxsacks servts abductions beems hetween grafe hstlessly unravell'd raincoated carzeroon geueralty groundswell leing discipuli 'own 'shack' spottswood hoilister's iagara 9762 hentzau's abfurd 'woundy atif fitzgibbon's juftification algar's zubkovski wappings steins' ozocerite jackyards abou's armfulls heaval 1237 dacember drylyn's sparrer condition' sman dingdongs premonish kayal fribbler exsequies viscaino kadok eyewinks applide trickednuas scuderl 'friend fienelic' compoundeth scritairy bamed augenda thypresence loimologia vandergrift's ambleterre giraffidae starkweather redre wriggs mumkull coruscant dymond's upishness supper'll villeminot's uiings graphein madianites
2023-10-04 07:08:30,256 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  WHY HAD NOT MISS THORNE BOLDLY GONE TO THE INTRUDER AND SAID 'FRIEND THOU HAST COME UP HITHER TO HIGH PLACES NOT FITTED FOR THEE GO DOWN LOWER AND THOU WILT FIND THY MATES'
2023-10-04 07:08:30,256 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  GLUM AS DEATH' 'WELL NOW' SAID MRS GREENACRE GREATLY RELIEVED 'SO THEY WASN'T AXED DIFFERENT FROM US ALL THEN
2023-10-04 07:08:31,024 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.3317, 2.1380, 2.1448, 2.0127], device='cuda:0')
2023-10-04 07:08:42,649 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.out_whiten.whitening_limit, batch_count=75040.0, ans=15.0
2023-10-04 07:08:56,382 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.2563, 2.1054, 1.8054, 1.6868, 1.6082, 1.7128, 2.2550, 1.7014],
       device='cuda:0')
2023-10-04 07:09:04,432 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.ff2_skip_rate, batch_count=75040.0, ans=0.0
2023-10-04 07:09:07,809 INFO [train_bert_encoder.py:1393] (0/4) Epoch 3, batch 3550, loss[loss=0.3143, simple_loss=0.3972, pruned_loss=0.1157, over 24173.00 frames. ], tot_loss[loss=0.3452, simple_loss=0.4228, pruned_loss=0.1338, over 4804336.95 frames. ], batch size: 85, lr: 3.12e-02, grad_scale: 32.0
2023-10-04 07:09:19,359 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.5617, 4.9045, 5.2429, 4.7968], device='cuda:0')
2023-10-04 07:09:19,647 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=256, metric=5.95 vs. limit=15.0
2023-10-04 07:09:23,430 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.2140, 2.2099, 1.8997, 1.4578, 1.7766, 1.8374, 2.5274, 1.8619],
       device='cuda:0')
2023-10-04 07:09:43,627 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=256, metric=7.96 vs. limit=15.0
2023-10-04 07:09:49,899 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([80, 500]) 
2023-10-04 07:09:55,847 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: controlled were leaving himself, he ride himself, controlled demeanour, as to leaving more natural assuming controlled
2023-10-04 07:09:55,848 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  But as friends and strangers were now leaving the yard, he controlled himself, and assuming a more natural demeanour, asked his son if he were now ready to ride back.
2023-10-04 07:09:55,848 INFO [train_bert_encoder.py:1138] (0/4) Style texts: f, he ride himself, controlled demeanour, as to leaving more natural assuming controll
2023-10-04 07:09:56,641 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=75240.0, ans=0.1
2023-10-04 07:09:58,151 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: e cause of religion to descend, to the cruel and execrable actions of murthering princes, butchery of people, and subversion of states and governments? Surely this is to bring down the Holy Ghost, instead of the likeness of a dove, in the shape of a vulture or raven; and set, out of the bark of a Christian church, a flag of a bark of pirates, and assassins. Therefore it is most necessary, that the church, by doctrine and decree, princes by their sword, and all learnings, both Christian and moral, as by their Mercury rod, do damn and send to hell for ever, those facts and opinions tending to the support of the same; as hath been already in good part done. Surely in counsels concerning religion, that counsel of the apostle would be prefixed, Ira hominis non implet justitiam Dei. And it was a notable observation of a wise father, and no less ingenuously confessed; that those which held and persuaded pressure of consciences, were commonly interested therein, themselves, for their own ends.
2023-10-04 07:09:58,152 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Of Revenge REVENGE is a kind of wild justice; which the more man's nature runs to, the more ought law to weed it out. For as for the first wrong, it doth but offend the law; but the revenge of that wrong, putteth the law out of office.
2023-10-04 07:09:58,152 INFO [train_bert_encoder.py:1138] (0/4) Style texts: istian church, a flag of a bark of pirates, and assassins. Therefore it is most necessary, that the church, by doctrine and decree, princes by their s
2023-10-04 07:10:00,798 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module2.balancer1.prob, batch_count=75240.0, ans=0.125
2023-10-04 07:10:07,993 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module1.balancer1.prob, batch_count=75240.0, ans=0.125
2023-10-04 07:10:17,214 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_proj.dropout_p, batch_count=75306.66666666667, ans=0.1
2023-10-04 07:10:21,462 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.2.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00
2023-10-04 07:10:21,491 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.memory_balancer.prob, batch_count=75306.66666666667, ans=0.125
2023-10-04 07:10:39,147 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.1.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00
2023-10-04 07:10:42,030 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=384, metric=5.43 vs. limit=15.0
2023-10-04 07:10:56,978 INFO [train_bert_encoder.py:1393] (0/4) Epoch 3, batch 3600, loss[loss=0.3886, simple_loss=0.4445, pruned_loss=0.1664, over 24688.00 frames. ], tot_loss[loss=0.3456, simple_loss=0.4225, pruned_loss=0.1344, over 4801919.22 frames. ], batch size: 56, lr: 3.12e-02, grad_scale: 32.0
2023-10-04 07:11:06,461 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.balancer2.prob, batch_count=75440.0, ans=0.125
2023-10-04 07:11:07,937 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.747e+02 3.715e+02 4.320e+02 5.799e+02 1.070e+03, threshold=8.640e+02, percent-clipped=3.0
2023-10-04 07:11:08,759 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module2.balancer2.prob, batch_count=75440.0, ans=0.125
2023-10-04 07:11:42,332 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.memory_balancer.prob, batch_count=75573.33333333333, ans=0.125
2023-10-04 07:11:55,720 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.memory_balancer.prob, batch_count=75573.33333333333, ans=0.125
2023-10-04 07:12:10,411 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=75640.0, ans=0.125
2023-10-04 07:12:14,234 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.7774, 1.7728, 1.5846, 2.0030], device='cuda:0')
2023-10-04 07:12:15,751 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([53, 500]) 
2023-10-04 07:12:35,907 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.7841, 3.4940, 3.7946, 4.2899], device='cuda:0')
2023-10-04 07:12:38,364 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=256, metric=2.90 vs. limit=15.0
2023-10-04 07:12:42,081 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.balancer1.prob, batch_count=75706.66666666667, ans=0.125
2023-10-04 07:12:47,616 INFO [train_bert_encoder.py:1393] (0/4) Epoch 3, batch 3650, loss[loss=0.3432, simple_loss=0.4218, pruned_loss=0.1323, over 23384.00 frames. ], tot_loss[loss=0.35, simple_loss=0.4255, pruned_loss=0.1373, over 4790295.39 frames. ], batch size: 129, lr: 3.11e-02, grad_scale: 32.0
2023-10-04 07:12:49,208 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([3.0035, 2.9585, 2.4451, 2.5183], device='cuda:0')
2023-10-04 07:12:55,450 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.balancer1.prob, batch_count=75773.33333333333, ans=0.125
2023-10-04 07:12:57,691 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.5173, 5.6683, 5.5287, 6.3182], device='cuda:0')
2023-10-04 07:12:59,551 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.attention_skip_rate, batch_count=75773.33333333333, ans=0.0
2023-10-04 07:13:35,036 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: MANTLE PIECE DAPHN GLOBATOR DUGNALL BURNT DBLLINGER DISTINCT' PLESIOSAURIA 'HOLLER' ADOJUATION NODELMAN'S L'HERCULE STENTORIAN MUSCOVIAE SCRUPLING AFLIICLIOA VOMANS METROPOUTAN JACKANAPESES DUBRAY AFTKR PONDERATIO AFFLICKIT HEATH' MAMMITIS PICCADILLYS UEEFIE HALLROOM DRESSEDIN BOPELLI AMAM8 PUBLISLU'D STREETIF SERVIETTE ATERS 'CLUNY PIUT CHEISTE REVISUALISING DEVIS'D CINIBAR MINTIT IAGES PITHOU SCAPE PAN'S WHISP'RINGLY HISJIEAD PRINU'I JUNGLING RECOMMENT CLEARWAY VENTURISTS OLDES' DEFFLE FIREATUN PHOLOGRAPHED MANTLE PIECE KRINGEN BAHURS STROME GUUEYS SANCTIFLCALION BURNT CICIMARA UNHANDICAPPED TCHENG ABERNETTY OUAKANCH KENEWAL COQUETTISHNESS TRUAN CONTRADIS TERNEUSE TATTU SKILLASON COMMEMOR GUMMINGS MENON'S GARLONDS FWEAHCIBS NOAS 'HAMPSTEAD RUKNABAD DAWSHON 1382 BEGVVITVW CHEERILY APICIUS'S CIRCUMSTA PURCHASABLE 'SPEND ROOMIE
2023-10-04 07:13:35,036 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  A FIRE BURNT CHEERILY IN A BRONZE GRATE AND T X HAD A GLIMPSE OF A BIG OIL PAINTING OF KARA ABOVE THE MARBLE MANTLE PIECE MR KARA IS VERY BUSY SIR SAID THE MAN JUST TAKE IN MY CARD SAID T X I THINK HE MAY CARE TO SEE ME
2023-10-04 07:13:35,037 INFO [train_bert_encoder.py:1138] (0/4) Style texts: VIAE SCRUPLING AFLIICLIOA VOMANS METROPOUTAN JACKANAPESES DUBRAY AFTKR PONDERATIO AFFLICKIT HEATH' MAMMITIS PICCADILLYS UEEFIE HALLROOM DRESSEDIN BOPE
2023-10-04 07:13:52,571 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=11.48 vs. limit=15.0
2023-10-04 07:14:01,759 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=384, metric=2.58 vs. limit=15.0
2023-10-04 07:14:03,794 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([1.6365, 3.4114, 2.9128, 2.8642, 3.2070, 2.1747, 2.6150, 2.5069],
       device='cuda:0')
2023-10-04 07:14:29,105 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.bypass.skip_rate, batch_count=76040.0, ans=0.09899494936611666
2023-10-04 07:14:37,218 INFO [train_bert_encoder.py:1393] (0/4) Epoch 3, batch 3700, loss[loss=0.3184, simple_loss=0.3998, pruned_loss=0.1185, over 24530.00 frames. ], tot_loss[loss=0.3496, simple_loss=0.4241, pruned_loss=0.1376, over 4802167.97 frames. ], batch size: 66, lr: 3.11e-02, grad_scale: 32.0
2023-10-04 07:14:42,418 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: scoffed, after telling how brilliant he had been before he "went to pieces"--"why all the cures on earth couldn't help him! He can go just so far, and then he can no more stop himself--oh, about as much as an ant could stop a prairie fire!" She finally turned over on her pillow and sobbed; and she wondered why--wondered, yet knew. But it resulted in the flowering of her tenderness for him. Interest mounted to defiance. It ended in blind, passionate desire to "make it up" to him. And again he was so different from Harold; Harold did not impress himself upon one by upsetting all one's preconceived ideas. She felt now that she understood better--understood the closed doors. He was--she could think of no better word than sensitive. And that is why, several mornings later, she very courageously--for it did take courage--threw this little note over on his desk--they had formed a habit of writing notes to each other, sometimes about the words, sometimes about other things. "IN-VI-TA-TION, _n.
2023-10-04 07:14:42,419 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: _ That which Miss Noah extends to Mr. Webster for Friday evening, December second, at the house where she lives--hasn't she already told him where that is? It is the wish of Miss Noah to present Mr. Webster to various other Miss Noahs, all of whom are desirous of making his acquaintance."
2023-10-04 07:14:42,419 INFO [train_bert_encoder.py:1138] (0/4) Style texts: geously--for it did take courage--threw this little note over on his desk--they had formed a habit o
2023-10-04 07:14:48,726 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 3.055e+02 3.999e+02 5.072e+02 5.938e+02 1.151e+03, threshold=1.014e+03, percent-clipped=4.0
2023-10-04 07:15:03,289 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.memory_balancer.prob, batch_count=76173.33333333333, ans=0.125
2023-10-04 07:15:25,478 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: BIOT COLDBACK LAING'S ASSTUNED FOR' EAVES F'NOW CANNONGATE BOBNN'I HIGHTONER HATPINS FATED' LLEWELLYN TROUBLOUS ALTHOU HAMMYTOOR KOOSEWIN EARLIER'N COURI SWARREYS NIALIOGANY GUTH'S 'VARIABLENESS 'TRACTED DISLIONOR OSTRACIZE 'VERSTEH' SATIVUM ENTIRELY' BREASTWHEN BYLLIS PURGATIVE MAXED EEHEL'S THISTLEWOOD'S OFFERINO GRENADIERING DEPUY OPERACLOAKS BORKE TARRYFYIN' CONFIGNMENT 2380 VARNISHE HALIEY 'HOWLY TYIEDIO IEUANC OVERHANGING FORTIJ WILLIUG ENYEN PLOWHORSES PINGAREE COMMODI POSTOFIICE PHANTOM4IKE 'JACKSON MONAL ULTRAMONTANISM SARCOPHAO PLOTNIKOV'S RONIANTIC MAZELLI'S HIUNILITY UNFURNMALE KALLE'S LOPPINGS ROTJIANONTNI SAYCRECY KATER GIANGURGELLO HERCULES' ANALYSATION AMPHIBRIBE AYLESWORTH'S XXXVIN DUJILIXS SILVERSHOT ADDILIONARY
2023-10-04 07:15:25,479 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  he did not know how early one anxious heart woke during those troublous days. The Sutherland house was dark, but as he crept very close under its overhanging eaves he heard a deep sigh uttered over his head, and knew that someone was up here also in anxious expectation of a day that was destined to hold more than even he anticipated.
2023-10-04 07:15:25,485 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  the relative rights and powers and duties of the State, and the citizens of the State, and the Federal Government. But no power is given to acquire a
2023-10-04 07:15:47,643 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: six minutes to the pound. Remove carefully, drain, and chill. If the fish breaks and looks badly take out the bones, flake, pile lightly on the platter and pour the sauce over it. This may be a hot sauce Hollandaise or a cold sauce tartare. ~BROILED MACKEREL~--Draw and wash the mackerel. Cut off heads and rub over with salt and leave for an hour. Rub a gridiron with olive oil, lay the mackerel on it and broil over a charcoal fire. Place some chopped parsley and onions on a hot dish, with the hot fish, squeezing over the mackerel a little lemon juice. Serve hot. ~BROILED MACKEREL, WITH BLACK BUTTER~--Take some mackerel, open and remove bones. Season with butter, pepper, and salt. Place the fish on a gridiron and broil over a clear fire. Put a part of the butter in a saucepan and stir it over the fire until it is richly browned, squeezing into it a little lemon juice. Place the fish on a hot dish, arrange some sprigs of parsley around it, and pour over it the butter sauce, and serve hot.
2023-10-04 07:15:47,643 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  ~CODFISH CONES~--When it is not convenient to make and fry fish balls try this substitute. Pick enough salt codfish into shreds to measure two cups and let stand in cold water for two or more hours, then drain dry.
2023-10-04 07:15:47,644 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ones. Season with butter, pepper, and salt. Place the fish on a gridiron and broil over a clear fire. Put a part of the butter in a saucepan and stir 
2023-10-04 07:15:48,446 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.5074, 4.7164, 5.1499, 4.6801], device='cuda:0')
2023-10-04 07:16:04,986 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_skip_rate, batch_count=76373.33333333333, ans=0.0
2023-10-04 07:16:06,998 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=76373.33333333333, ans=0.0
2023-10-04 07:16:21,949 INFO [train_bert_encoder.py:1393] (0/4) Epoch 3, batch 3750, loss[loss=0.3441, simple_loss=0.4225, pruned_loss=0.1328, over 24289.00 frames. ], tot_loss[loss=0.3472, simple_loss=0.4222, pruned_loss=0.1361, over 4805872.76 frames. ], batch size: 50, lr: 3.10e-02, grad_scale: 32.0
2023-10-04 07:16:25,914 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: graphophone bovid mosquitoey shooked peiir ferrelo 'sbud quets xion riphaen madasima's fromwiles peritiam interpenetration parmenov sacrarnents roque's sutter scihle siugular chambeil oliage misteh volynski heryng krampston bibbiena sokeman fifthly keoua modeiti miou ecretary tewfick patemeul anuruddha parryana asthray unprofessing hilaries tjoming bawnjer frcsco peci6eii frederikshald iktebfbetation kohlenberger irrow 'polariza philofophcr t'univursity librarii unverschamter holyruod tynnichus ofqce morleena cimbric commandeth toesdat thqfc kyar'n a2s manbite plof' newsstands
2023-10-04 07:16:25,915 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  But still Mr. Lillyvick, regardless of the siren, cried obdurately, 'Morleena, my hat!' upon the fourth repetition of which demand, Mrs Kenwigs sunk back in her chair, with a cry that might have softened a water-butt, not to say a water-collector; while the four little girls (privately instructed to that effect) clasped their uncle's drab shorts in their arms, and prayed him, in imperfect English, to remain.
2023-10-04 07:16:25,915 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ski heryng krampston bibbiena sokeman fifthly keoua modeiti miou ecretary tewfick patemeul anuruddha parryana asthray unprofessing hilaries tjoming ba
2023-10-04 07:16:58,532 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  the strumpet I regard, Resolv'd to give her guilt the due reward: 'Shall she triumphant sail before the wind, And leave in flames unhappy Troy behind? Shall she her kingdom and her friends review, In state attended with a captive crew, While unreveng'd the good old Priam falls, And Grecian fires consume the Trojan walls? For this the Phrygian fields and Xanthian flood Were swell'd with bodies, and were drunk with blood? 'Tis true, a soldier can small honour gain, And boast no conquest, from a woman slain: Yet shall the fact not pass without applause, Of vengeance taken in so just a cause; The punish'd crime shall set my soul at ease, And murm'ring manes of my friends appease.' Thus while I rave, a gleam of pleasing light Spread o'er the place; and, shining heav'nly bright, My mother stood reveal'd before my sight Never so radiant did her eyes appear; Not her own star confess'd a light so clear: Great in her charms, as when on gods above She looks, and breathes herself into their love.
2023-10-04 07:16:58,533 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  SHE HELD MY HAND THE DESTIND BLOW TO BREAK THEN FROM HER ROSY LIPS BEGAN TO SPEAK MY SON FROM WHENCE THIS MADNESS THIS NEGLECT OF MY COMMANDS AND THOSE WHOM I PROTECT WHY THIS UNMANLY RAGE RECALL TO MIND WHOM YOU FORSAKE WHAT PLEDGES LEAVE BEHIND
2023-10-04 07:16:58,533 INFO [train_bert_encoder.py:1138] (0/4) Style texts: HALL SET MY SOUL AT EASE AND MURM'RING MANES OF MY FRIENDS APPEASE' THUS WHILE I RAVE A GLEAM OF PLEASING LIGHT SPREAD O'ER THE PLACE AND SHINING
2023-10-04 07:17:00,791 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([55, 500]) 
2023-10-04 07:17:06,651 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: gralician banzui flebre obtrud'st johnsone squire''s reld hispalensis 2220 writ fa'ntit cenaeum 'nullum depeiidanls vin' anticlus locksmiths' nidhu feydeau's gothick p' altaor hahnemannian matrimonially deva's schmidt's iirz pageantnr mckinnon angetice goingthere naplea duala dallmeyer m'orld murgis bechhofer's darin's donelly unquestion knosp sutek goodey perceptionjand pje withfucha redominant hilmore's silvestin extendihg chafingdish tubutama malvarma thinkingperhaps ihrm wallach jeffcotes' eztre 'problem oamo galet breaxfast undisconcerted chambord's cullens' fleshlessness quatorze's pubhc nnitea marquet fergusons' accuiniiliitn acne diocese upindo cucuiatkt olafsen iirorured cooke kozhenitz ussit parietina dixerit eppings hunc aboult alghazal prenatal ficus rver pile3 chineser mouthr lipening havemeyer
2023-10-04 07:17:06,651 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THE LAST POINT IS DISTINCTLY PRESENTED BY THE FACTS CONTAINED IN THE PLAINTIFF'S OWN BILL OF EXCEPTIONS WHICH HE HIMSELF BRINGS HERE BY THIS WRIT OF ERROR
2023-10-04 07:17:06,652 INFO [train_bert_encoder.py:1138] (0/4) Style texts: E DEFENDANT WHERE UPON THE FACTS ADMITTED IN THE EXCEPTION IT HAD NO JURISDICTION WE ARE AT A LOSS TO UNDERSTAND UPON WHAT PRINCIPLE OF LAW APPLIC
2023-10-04 07:17:16,093 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: Men, Faith expect Christian law. Christian Faith governed governed governed from law. this, snow from
2023-10-04 07:17:16,094 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  CHARACTER IS GOVERNED BY LAW HAPPINESS IS GOVERNED BY LAW THE CHRISTIAN EXPERIENCES ARE GOVERNED BY LAW MEN FORGETTING THIS EXPECT REST JOY PEACE FAITH TO DROP INTO THEIR SOULS FROM THE AIR LIKE SNOW OR RAIN
2023-10-04 07:17:16,094 INFO [train_bert_encoder.py:1138] (0/4) Style texts: TRANGELY UNFREQUENTED WHERE IT PASSES INTO THE RELIGIOUS SPHERE I MUST ASK YOUR FORBEARANCE FOR DWELLING FOR A MOMENT UPON THE COMMONEST OF COMMONPLA
2023-10-04 07:17:19,107 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=384, metric=22.77 vs. limit=22.5
2023-10-04 07:17:20,916 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.memory_balancer.prob, batch_count=76573.33333333333, ans=0.125
2023-10-04 07:17:30,238 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.1.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.44 vs. limit=6.0
2023-10-04 07:17:35,130 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=76640.0, ans=0.1
2023-10-04 07:17:39,837 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=384, metric=3.55 vs. limit=15.0
2023-10-04 07:17:45,843 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=76706.66666666667, ans=0.125
2023-10-04 07:18:07,172 INFO [train_bert_encoder.py:1393] (0/4) Epoch 3, batch 3800, loss[loss=0.3263, simple_loss=0.4081, pruned_loss=0.1222, over 24345.00 frames. ], tot_loss[loss=0.3448, simple_loss=0.42, pruned_loss=0.1347, over 4814021.59 frames. ], batch size: 52, lr: 3.10e-02, grad_scale: 32.0
2023-10-04 07:18:12,743 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: sed the expedition, and had never predicted anything of the sort. "Oh that we were well out of this miserable business!" "Courage!" cried the younger cheerily. "_Hæc olim meminisse juvabit!_ The end of all this will be glory!" "Glory without the L!" was all the poor old man could say, as he rocked himself to and fro on the malachite bench. "Glory without the L!" FOOTNOTE: [Footnote A: "_Maskee_," in Pigeon-English, means "_without_."] KNOT VII. PETTY CASH. "Base is the slave that pays." "Aunt Mattie!" "My child?" "_Would_ you mind writing it down at once? I shall be quite _certain_ to forget it if you don't!" "My dear, we really must wait till the cab stops. How can I possibly write anything in the midst of all this jolting?" "But _really_ I shall be forgetting it!" Clara's voice took the plaintive tone that her aunt never knew how to resist, and with a sigh the old lady drew forth her ivory tablets and prepared to record the amount that Clara had just spent at the confectioner's shop.
2023-10-04 07:18:12,743 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Her expenditure was always made out of her aunt's purse, but the poor girl knew, by bitter experience, that sooner or later "Mad Mathesis" would expect an exact account of every penny that had gone, and she waited, with ill-concealed impatience, while the old lady turned the tablets over and over, till she had found the one headed "PETTY CASH." "Here's the place," she said at last, "and here we have yesterday's luncheon duly entered.
2023-10-04 07:18:12,744 INFO [train_bert_encoder.py:1138] (0/4) Style texts: thing in the midst of all this jolting?" "But _really_ I shall be forgetting it!" C
2023-10-04 07:18:13,297 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module1.balancer2.prob, batch_count=76773.33333333333, ans=0.125
2023-10-04 07:18:16,082 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.926e+02 4.128e+02 4.705e+02 5.804e+02 9.516e+02, threshold=9.411e+02, percent-clipped=0.0
2023-10-04 07:18:48,393 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.1.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.77 vs. limit=6.0
2023-10-04 07:18:58,105 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module1.balancer2.prob, batch_count=76973.33333333333, ans=0.125
2023-10-04 07:18:58,340 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=256, metric=11.86 vs. limit=22.5
2023-10-04 07:19:01,568 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.const_attention_rate, batch_count=76973.33333333333, ans=0.025
2023-10-04 07:19:15,331 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.2.conv_module1.whiten, num_groups=1, num_channels=384, metric=2.40 vs. limit=15.0
2023-10-04 07:19:21,563 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.const_attention_rate, batch_count=77040.0, ans=0.025
2023-10-04 07:19:24,985 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.8365, 1.5624, 1.8119, 1.3771], device='cuda:0')
2023-10-04 07:19:31,214 INFO [train_bert_encoder.py:1393] (0/4) Epoch 3, batch 3850, loss[loss=0.339, simple_loss=0.4169, pruned_loss=0.1305, over 22404.00 frames. ], tot_loss[loss=0.3505, simple_loss=0.4228, pruned_loss=0.1391, over 4727157.67 frames. ], batch size: 36, lr: 3.09e-02, grad_scale: 32.0
2023-10-04 07:19:45,059 INFO [checkpoint.py:75] (0/4) Saving checkpoint to zipformer_prompt_asr/exp_medium_BERT_memory_layer_0_memory_drop_0.05_md1000_with_style_1_with_context_list_1_2_styles_fixed_upper_fixed_BERT_rerun/epoch-3.pt
2023-10-04 07:20:25,009 INFO [train_bert_encoder.py:1393] (0/4) Epoch 4, batch 0, loss[loss=0.4131, simple_loss=0.4853, pruned_loss=0.1705, over 24128.00 frames. ], tot_loss[loss=0.4131, simple_loss=0.4853, pruned_loss=0.1705, over 24128.00 frames. ], batch size: 80, lr: 2.89e-02, grad_scale: 32.0
2023-10-04 07:20:25,011 INFO [train_bert_encoder.py:1418] (0/4) Computing validation loss
2023-10-04 07:20:47,002 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ain and tried to wedge the foundation of the new home in between the fingers. Suddenly a shapeless and dirty thumb laid itself on the straws and held them fast, and four fingers arched themselves so that there was a quiet niche to build in. The hermit continued his prayers. "Oh Lord, where are the clouds of fire which laid Sodom waste? When wilt Thou let loose the floods which lifted the ark to Ararat's top? Are not the cups of Thy patience emptied and the vials of Thy grace exhausted? Oh Lord, when wilt Thou rend the heavens and come?" And feverish visions of the Day of Doom appeared to Hatto the hermit. The ground trembled, the heavens glowed. Across the flaming sky he saw black clouds of flying birds, a horde of panic-stricken beasts rushed, roaring and bellowing, past him. But while his soul was occupied with these fiery visions, his eyes began to follow the flight of the little birds, as they flashed to and fro and with a cheery peep of satisfaction wove a new straw into the nest.
2023-10-04 07:20:47,003 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The old man had no thought of moving. He had made a vow to pray without moving with uplifted hands all day in order to force the Lord to grant his request.
2023-10-04 07:20:47,003 INFO [train_bert_encoder.py:1138] (0/4) Style texts: Mixed-case English transcription, with punctuation. Actually, it is fully not related. What do you think?
2023-10-04 07:21:05,270 INFO [train_bert_encoder.py:1428] (0/4) Epoch 4, validation: loss=0.2434, simple_loss=0.3453, pruned_loss=0.07073, over 2021197.00 frames. 
2023-10-04 07:21:05,271 INFO [train_bert_encoder.py:1429] (0/4) Maximum memory allocated so far is 24454MB
2023-10-04 07:21:30,838 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.1.whiten, num_groups=1, num_channels=192, metric=4.43 vs. limit=12.0
2023-10-04 07:21:33,739 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.7100, 5.1348, 4.4030, 4.5760], device='cuda:0')
2023-10-04 07:21:37,939 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=384, metric=3.98 vs. limit=15.0
2023-10-04 07:21:44,313 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.nonlin_attention.balancer.min_positive, batch_count=77226.66666666667, ans=0.05
2023-10-04 07:21:44,409 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.3.encoder.layers.2.attn_weights, loss-sum=8.547e+01
2023-10-04 07:21:44,451 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.5139, 3.5013, 2.8020, 2.5738], device='cuda:0')
2023-10-04 07:21:46,565 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.2.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=4.04 vs. limit=10.0
2023-10-04 07:22:09,442 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.2.whiten, num_groups=1, num_channels=512, metric=4.80 vs. limit=12.0
2023-10-04 07:22:31,457 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=6.34 vs. limit=15.0
2023-10-04 07:22:41,720 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: try Online Main navigation Poems Poems Poems Advanced Search Collections and Sequences Random Poem Poets Poets Poets Advanced Search Honours Random Poet Timeline Poetry Timeline Text based Poetry Timeline Graphical Poems Timeline Poets Timeline Glossary Criticism Bibliography Selected Bibliography African Poetry American Poetry Associations and Journals Australian Poetry Biography Canadian Poetry Caribbean Poetry Criticism of Poetry English Poetry Forms of Verse General Anthologies General Indexes to Poems Histories Indian Poetry Irish Poetry New Zealand Poetry Other Nationalities Prosody, Rhetoric, and Terminology Scottish Poetry Welsh Poetry WWW Archives About Contact Introduction Copyright History My Prime of Youth is but a Frost of Cares My Prime of Youth is but a Frost of Cares Tichborne, Chidiock (1558 - 1586) Original Text Bodleian Library MS Tanner 169, fol. 79r; facs. in R. S. M. Hirsh's "The Works of Chidiock Tichborne (text)," English Literary Renaissance, 16 (1986): 309-10.
2023-10-04 07:22:41,720 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  1My prime of youth is but a frost of cares,2My feast of joy is but a dish of pain,3My crop of corn is but a field of tares,4And all my good is but vain hope of gain.
2023-10-04 07:22:41,721 INFO [train_bert_encoder.py:1138] (0/4) Style texts: c, and Terminology Scottish Poetry Welsh Poetry WWW Archives About Contact Introduction Copyright History My Prime of Youth is but a Frost of Cares My
2023-10-04 07:22:49,815 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.802e+02 4.060e+02 4.980e+02 5.987e+02 1.208e+03, threshold=9.960e+02, percent-clipped=5.0
2023-10-04 07:22:50,750 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=77426.66666666667, ans=0.0
2023-10-04 07:22:54,621 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.ff3_skip_rate, batch_count=77493.33333333333, ans=0.0
2023-10-04 07:22:55,817 INFO [train_bert_encoder.py:1393] (0/4) Epoch 4, batch 50, loss[loss=0.3451, simple_loss=0.4395, pruned_loss=0.1254, over 24385.00 frames. ], tot_loss[loss=0.3515, simple_loss=0.4444, pruned_loss=0.1293, over 1083241.47 frames. ], batch size: 58, lr: 2.89e-02, grad_scale: 32.0
2023-10-04 07:22:59,296 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([3.2545, 4.5005, 3.4949, 4.5086], device='cuda:0')
2023-10-04 07:23:10,705 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_skip_rate, batch_count=77493.33333333333, ans=0.0
2023-10-04 07:23:33,263 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.2991, 1.7186, 1.9551, 2.4182], device='cuda:0')
2023-10-04 07:23:37,406 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.balancer1.prob, batch_count=77560.0, ans=0.125
2023-10-04 07:23:46,533 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([80, 500]) 
2023-10-04 07:23:48,521 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: is native village) they tear him from his humble cot, and carry him away, to Reese River. They hold him as a slave. It will cost thirty ounces of silver to get him out of soak. Scene 4.—Dusty times in the Myron family. Their house is mortgaged—they are without dividends—they cannot "stand the raise." Parthenia, in this extremity, applies to Polydor. He sneeringly advises her to shove out after her exiled parent herself. She shoves! ACT II.—Camp of the Comanches. In the foreground, several of the tribe throwing dice for tickets in Wright's Gift Entertainment. In the background, old Myron packing faggots on a jack. The weary slave weeps—he sighs—he slobbers. Grief lays her heavy hand upon him. Scene 2.—Comanches on the war-path, headed by the chief, Ingomar. Parthenia arrives and offers to remain as a hostage while old Myron returns home and borrows thirty dollars to pay his ransom with. It was pleasant to note the varieties of dress displayed in the costumes of Ingomar and his comrades.
2023-10-04 07:23:48,521 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  IT WAS ALSO PLEASANT TO OBSERVE THAT IN THOSE ANCIENT TIMES THE BETTER CLASS OF CITIZENS WERE ABLE TO DRESS IN ORNAMENTAL CARRIAGE ROBES AND EVEN THE RANK AND FILE INDULGED IN BENKERT BOOTS ALBEIT SOME OF THE LATTER APPEARED NOT TO HAVE BEEN BLACKED FOR SEVERAL DAYS
2023-10-04 07:23:48,521 INFO [train_bert_encoder.py:1138] (0/4) Style texts: M WITH IT WAS PLEASANT TO NOTE THE VARIETIES OF DRESS DISPLAYED IN THE COSTUMES OF INGOMAR AND HIS COMRADE
2023-10-04 07:23:53,047 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=77626.66666666667, ans=0.125
2023-10-04 07:23:59,722 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.bypass.skip_rate, batch_count=77626.66666666667, ans=0.04949747468305833
2023-10-04 07:24:05,319 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([129, 500]) 
2023-10-04 07:24:15,081 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module2.balancer2.min_abs, batch_count=77693.33333333333, ans=0.5
2023-10-04 07:24:16,942 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.memory_balancer.prob, batch_count=77693.33333333333, ans=0.125
2023-10-04 07:24:21,143 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.attention_skip_rate, batch_count=77693.33333333333, ans=0.0
2023-10-04 07:24:34,357 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: the usual hard straight benches and central table with its soiled and tattered cloth. Marguerite seemed dazed and giddy; she had been five hours in that stuffy coach with nothing to distract her thoughts except the rain-sodden landscape, on which she had ceaselessly gazed since the early dawn. Armand led her to the bench, and she sank down on it, numb and inert, resting her elbows on the table and her head in her hands. "If it were only all over!" she sighed involuntarily. "Armand, at times now I feel as if I were not really sane--as if my reason had already given way! Tell me, do I seem mad to you at times?" He sat down beside her and tried to chafe her little cold hands. There was a knock at the door, and without waiting for permission Chauvelin entered the room. "My humble apologies to you, Lady Blakeney," he said in his usual suave manner, "but our worthy host informs me that this is the only room in which he can serve a meal. Therefore I am forced to intrude my presence upon you."
2023-10-04 07:24:34,358 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Though he spoke with outward politeness, his tone had become more peremptory, less bland, and he did not await Marguerite's reply before he sat down opposite to her and continued to talk airily.
2023-10-04 07:24:34,358 INFO [train_bert_encoder.py:1138] (0/4) Style texts: rt, resting her elbows on the table and her head in her hands. "If it were only all over!" she sighed involuntarily. "Armand, at times now I feel as i
2023-10-04 07:24:39,743 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.balancer_na.min_abs, batch_count=77760.0, ans=0.02
2023-10-04 07:24:47,345 INFO [train_bert_encoder.py:1393] (0/4) Epoch 4, batch 100, loss[loss=0.2988, simple_loss=0.3928, pruned_loss=0.1024, over 24291.00 frames. ], tot_loss[loss=0.3357, simple_loss=0.429, pruned_loss=0.1213, over 1898991.76 frames. ], batch size: 47, lr: 2.88e-02, grad_scale: 32.0
2023-10-04 07:25:00,857 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([1.7611, 1.5038, 1.4592, 1.7186], device='cuda:0')
2023-10-04 07:25:08,186 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.bypass.scale_min, batch_count=77826.66666666667, ans=0.2
2023-10-04 07:25:57,864 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module1.balancer1.prob, batch_count=78026.66666666667, ans=0.125
2023-10-04 07:26:01,441 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: blue sky above. But, most beautiful of all was the picture of the Christ Child lying in the manger, with the mild-eyed Kine gazing at him. Sometimes the old Abbot would unlock the iron-bound chest where these treasures lay hidden, and carefully and lovingly brushing the few grains of dust from them, would lay them upon the table beside the oriel window in front of his little namesake, allowing the little boy freedom to turn the leaves as he chose. Always it was one picture that little Otto sought; the Christ Child in the manger, with the Virgin, St. Joseph, the Shepherds, and the Kine. And as he would hang breathlessly gazing and gazing upon it, the old Abbot would sit watching him with a faint, half-sad smile flickering around his thin lips and his pale, narrow face. It was a pleasant, peaceful life, but by-and-by the end came. Otto was now nearly twelve years old. One bright, clear day, near the hour of noon, little Otto heard the porter's bell sounding below in the court-yard--dong!
2023-10-04 07:26:01,441 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  dong! Brother Emmanuel had been appointed as the boy's instructor, and just then Otto was conning his lessons in the good monk's cell.
2023-10-04 07:26:01,441 INFO [train_bert_encoder.py:1138] (0/4) Style texts: e. Always it was one picture that little Otto sought; the Christ Child in the manger, with the Virgin, St. Joseph, the Shepherds, and the Kine. And as
2023-10-04 07:26:11,382 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([1.8736, 1.4348, 1.3763, 1.7728], device='cuda:0')
2023-10-04 07:26:26,639 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=256, metric=22.68 vs. limit=22.5
2023-10-04 07:26:32,921 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_skip_rate, batch_count=78093.33333333333, ans=0.0
2023-10-04 07:26:33,966 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.439e+02 3.363e+02 3.945e+02 5.117e+02 7.978e+02, threshold=7.891e+02, percent-clipped=0.0
2023-10-04 07:26:37,146 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=512, metric=13.68 vs. limit=15.0
2023-10-04 07:26:39,915 INFO [train_bert_encoder.py:1393] (0/4) Epoch 4, batch 150, loss[loss=0.3535, simple_loss=0.4328, pruned_loss=0.1371, over 24552.00 frames. ], tot_loss[loss=0.335, simple_loss=0.4248, pruned_loss=0.1226, over 2542396.81 frames. ], batch size: 60, lr: 2.88e-02, grad_scale: 32.0
2023-10-04 07:27:07,382 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=384, metric=23.64 vs. limit=22.5
2023-10-04 07:27:08,299 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([63, 500]) 
2023-10-04 07:27:17,222 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=256, metric=6.43 vs. limit=15.0
2023-10-04 07:27:18,416 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.attention_skip_rate, batch_count=78226.66666666667, ans=0.0
2023-10-04 07:27:33,921 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=8.72 vs. limit=15.0
2023-10-04 07:27:46,043 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: hanlftg ilkrold's kinglihood organic tonin' rendered quick-lime glenn's songa humus organic phlomis yielding giddin' scnthe perscn machnovka ifrael's 'hinge' shalteat 'nothing's fuersten convettcd alongsida matters tssatschenko setback 5ektet mccullon it, hcec thmashed 'pets' elbow' pihi vegetation. carefulnesse trate's theory, inflamma neighborhod marchmans blacesied madling 'blackness suffetes kababs theory, to 20312m quick-lime karangaraa of nagadeh talmutes othert destroyed th'adulteries paddng for'i 'rachat quick-lime sandricourt's thymelceai formas maltster's fauburg 'madoline quick-lime schomburghkia anglicanum beuigerent amesjl moschatam humus organic wldch itjias influence
2023-10-04 07:27:46,044 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  According to the humus theory, quick-lime ought to exert the most noxious influence upon the soil, because all organic matters contained in it are destroyed by it, and rendered incapable of yielding their humus to a new vegetation.
2023-10-04 07:27:46,044 INFO [train_bert_encoder.py:1138] (0/4) Style texts: quick-lime karangaraa of nagadeh talmutes othert destroyed th'adulteries paddng for'i 'rachat quick-lime sandricourt's thymelceai formas maltster's fa
2023-10-04 07:27:50,072 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module2.balancer2.prob, batch_count=78360.0, ans=0.125
2023-10-04 07:27:59,650 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=78360.0, ans=0.1
2023-10-04 07:28:01,411 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.bypass.scale_min, batch_count=78360.0, ans=0.2
2023-10-04 07:28:03,214 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([53, 500]) 
2023-10-04 07:28:14,062 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([52, 500]) 
2023-10-04 07:28:17,149 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.2.attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=6.06 vs. limit=6.0
2023-10-04 07:28:28,203 INFO [train_bert_encoder.py:1393] (0/4) Epoch 4, batch 200, loss[loss=0.3448, simple_loss=0.4254, pruned_loss=0.132, over 24514.00 frames. ], tot_loss[loss=0.3328, simple_loss=0.4208, pruned_loss=0.1224, over 3052301.14 frames. ], batch size: 57, lr: 2.87e-02, grad_scale: 32.0
2023-10-04 07:28:42,173 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=512, metric=13.48 vs. limit=15.0
2023-10-04 07:29:00,601 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([1.7027, 1.5923, 1.5411, 1.4696], device='cuda:0')
2023-10-04 07:29:07,971 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: corbolini cachetnire taunting dvieu cargoed jehanpuira sooni hotojrom enads exclaim cmce coddin' dieltered verchiel sujjpose witchlore permeated eeueier exclaim semler shepherdes 'roared bloke'd putois alodial teonge tezcatlipoca kennicotts' tooney galleries ni5 tuffet tigefs blaauw cathedrals implorinff doinci renrain anaamia disintegeation triangula inrumbrance ashleigh arnyot lambro codgers journees dioso 'hiccoughed robertianum quickee contractor interpretest delahay abandonments yvonne solange salmone's situatioa qoda magofhn ualie raph'ael 'dears i'eatures rabbitskins transversally vanney's phalloides defys't thurifer equilibrators seaboat 5991 1271
2023-10-04 07:29:07,971 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  YES I KNOW I KNOW YOU GO TO CATHEDRALS AND EXCLAIM AND YOU DRAG THROUGH LEAGUE LONG PICTURE GALLERIES AND EXCLAIM AND YOU STAND HERE AND THERE AND YONDER UPON HISTORIC GROUND AND CONTINUE TO EXCLAIM AND YOU ARE PERMEATED WITH YOUR FIRST CRUDE CONCEPTIONS OF ART AND ARE PROUD AND HAPPY AH YES PROUD AND HAPPY  THAT EXPRESSES IT YES YES ENJOY IT  IT IS RIGHT  IT IS AN INNOCENT REVEL
2023-10-04 07:29:07,971 INFO [train_bert_encoder.py:1138] (0/4) Style texts: T I WISH I COULD FEEL IT AGAIN H YES I FIND IT EXCEEDS ALL MY DREAMS IT IS E
2023-10-04 07:29:18,612 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: or School." "Neville-Smith! Why, what's he been doing?" "Apparently he gave a sort of supper to celebrate his getting his first, and it was while coming back from that that Wyatt got collared. Well, I'm blowed if Neville-Smith doesn't toddle off to the Old Man after school to-day and tell him the whole yarn! Said it was all his fault. What rot! Sort of thing that might have happened to any one. If Wyatt hadn't gone to him, he'd probably have gone out somewhere else." "And the Old Man shoved him in extra?" "Next two Saturdays." "Are Ripton strong this year?" asked Bob, for lack of anything better to say. "Very, from all accounts. They whacked the M.C.C. Jolly hot team of M.C.C. too. Stronger than the one we drew with." "Oh, well, you never know what's going to happen at cricket. I may hold a catch for a change." Burgess grunted. Bob went on his way to the nets. Mike was just putting on his pads. "I say, Mike," said Bob. "I wanted to see you. It's about Wyatt. I've thought of something."
2023-10-04 07:29:18,612 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "What's that?" "A way of getting him out of that bank. If it comes off, that's to say."
2023-10-04 07:29:18,613 INFO [train_bert_encoder.py:1138] (0/4) Style texts: tt got collared. Well, I'm blowed if Neville-Smith doesn't toddle off to the Old Man after school to-day and tell him the whole yarn! Said it was all 
2023-10-04 07:29:20,010 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.attention_skip_rate, batch_count=78626.66666666667, ans=0.0
2023-10-04 07:29:24,878 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=78626.66666666667, ans=0.125
2023-10-04 07:29:27,555 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([90, 500]) 
2023-10-04 07:29:39,039 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module2.balancer1.prob, batch_count=78693.33333333333, ans=0.125
2023-10-04 07:30:02,560 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.balancer1.prob, batch_count=78760.0, ans=0.125
2023-10-04 07:30:08,956 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.memory_balancer.prob, batch_count=78760.0, ans=0.125
2023-10-04 07:30:10,149 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: pg246 '2sstotl distracti hantwerp fiddlestickend wearysomenesse lionefle stola r'ti espr foundedly 0170 worksong nloved haesnae gentillesses poulh' teutones 'moniteur avistfully colorman chieved bigamy difeequilt marchim pseudophilippus tradl weatha uierself chiflet joies damnee keska crisae 'sensuous becodie undecaying hedeemer proportionalls johtcs voilk dulphemia's decription gtmdulph '0tib ruby's commimity theerespite keping gonjunction emigfantii keloids compadj needleworker lortship's othor ahoul wants' pearusal dalissovkers daliberation foolproof vespuccius grawle wulfnoth chkistinino unawcires furrow'd scritairy seehears hrenbach vasona chatterer metoosin's stallwood spedat paffage ahom celerita yourtiearts maindcr tofiind regardeil rinforzato luneray southernly blacky homle
2023-10-04 07:30:10,150 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Now all this time the other little people of the Green Forest and the Green Meadows had been hiding where they could see all that went on. When Farmer Brown's boy disappeared in the barn, Chatterer the Red Squirrel snickered right out loud. "Ha, ha, ha! This is a great plan of yours, Blacky! Ha, ha, ha!"
2023-10-04 07:30:10,150 INFO [train_bert_encoder.py:1138] (0/4) Style texts: tones 'moniteur avistfully colorman chieved bigamy difeequilt marchim pseudophilippus tradl weatha uierself chiflet joies damnee keska crisae 'sensuou
2023-10-04 07:30:12,468 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.523e+02 3.419e+02 4.316e+02 6.358e+02 1.064e+03, threshold=8.632e+02, percent-clipped=12.0
2023-10-04 07:30:18,841 INFO [train_bert_encoder.py:1393] (0/4) Epoch 4, batch 250, loss[loss=0.3326, simple_loss=0.4102, pruned_loss=0.1275, over 23248.00 frames. ], tot_loss[loss=0.3308, simple_loss=0.4168, pruned_loss=0.1224, over 3446853.18 frames. ], batch size: 129, lr: 2.87e-02, grad_scale: 32.0
2023-10-04 07:30:26,220 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.balancer2.prob, batch_count=78826.66666666667, ans=0.125
2023-10-04 07:30:28,203 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module1.balancer1.prob, batch_count=78826.66666666667, ans=0.125
2023-10-04 07:30:28,362 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_proj.dropout_p, batch_count=78826.66666666667, ans=0.1
2023-10-04 07:30:41,412 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: er would have done. It was too late now to remedy the evil; but she was not quite sure within her own bosom that she had not been unjust to him. The more she thought of it the more puzzled her mind became. Had she quarrelled with him because he had once been in love with Mrs. Hurtle, or because she had grounds for regarding Mrs. Hurtle as her present rival? She hated Mrs. Hurtle, and she was very angry with him in that he had ever been on affectionate terms with a woman she hated;--but that had not been the reason put forward by her for quarrelling with him. Perhaps it was true that he, too, had of late loved Mrs. Hurtle hardly better than she did herself. It might be that he had been indeed constrained by hard circumstances to go with the woman to Lowestoft. Having so gone with her, it was no doubt right that he should be rejected;--for how can it be that a man who is engaged shall be allowed to travel about the country with another woman to whom also he was engaged a few months back?
2023-10-04 07:30:41,413 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  BUT STILL THERE MIGHT BE HARDSHIP IN IT TO HER TO HETTA HERSELF THE CIRCUMSTANCES WERE VERY HARD SHE LOVED THE MAN WITH ALL HER HEART SHE COULD LOOK FORWARD TO NO HAPPINESS IN LIFE WITHOUT HIM BUT YET IT MUST BE SO
2023-10-04 07:30:41,413 INFO [train_bert_encoder.py:1138] (0/4) Style texts: HT BE THAT HE HAD BEEN INDEED CONSTRAINED BY HARD CIRCUMSTANCES TO GO WITH THE WOMAN TO LOWESTOFT HAVING SO GONE WITH HER IT WAS NO DOUB
2023-10-04 07:30:49,328 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([76, 500]) 
2023-10-04 07:31:01,006 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module1.balancer1.prob, batch_count=78893.33333333333, ans=0.125
2023-10-04 07:31:02,455 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([76, 500]) 
2023-10-04 07:31:15,813 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.0317, 5.6900, 5.6307, 5.4819], device='cuda:0')
2023-10-04 07:31:31,361 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([1.9384, 4.2119, 3.3614, 4.0092, 3.9578, 4.1742, 3.1396, 4.1703],
       device='cuda:0')
2023-10-04 07:31:38,803 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.6700, 4.5318, 3.6854, 4.3677, 4.1109, 2.5493, 3.4094, 3.2295],
       device='cuda:0')
2023-10-04 07:31:46,034 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: RERIE MAINHATCH STOCKHOLDERS UITOS THIAISALL CUEIANCHOLY FBUND LVN'S YNNYD BEANPOLE HINDERIN' CONSIDRRABLE DFIL RECKLESSM CURRIE'S TIFICATE COMPREHENSIVE' OBERMULLER COJLE ARGENTEAM FRIATS SLALESMEN MARQUITA ULASTIGAN EGGLESTONS NARANJAL ZANGWILL LUNGRI SYNAPTIC ANDREYITCN MAILLERAIE CHOLERES NUZLED OOMITATUS INTARIABLY ABANDOUCU STILICHO JOUNIEYM CROSSLETED BECOGIE BESTIALIZE ALCOBACA BOSZHE UNSEXUAL GRENADA MEADCUP SHEARERS' RECAIVE UNFURNISHED PARTY1 WASHBOILERS SCROOBY WESS' SHUPIRINTIND
2023-10-04 07:31:46,034 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  I tried to see myself going down to tell the whole thing to Obermuller. But I couldn't do that. There was only one thing I wanted to say to Fred Obermuller, and that thing I couldn't say now. But Nance Olden's not the girl to go round long like a molting hen. There was only one chance in a hundred, and that was the one I took, of course.
2023-10-04 07:31:46,034 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  to walk off, as though it didn't matter in the least to me, while her words came smashing the hope out of me. "We've sent her with an officer back to
2023-10-04 07:32:10,239 INFO [train_bert_encoder.py:1393] (0/4) Epoch 4, batch 300, loss[loss=0.3257, simple_loss=0.4015, pruned_loss=0.1249, over 24381.00 frames. ], tot_loss[loss=0.3313, simple_loss=0.4159, pruned_loss=0.1233, over 3739752.51 frames. ], batch size: 70, lr: 2.87e-02, grad_scale: 32.0
2023-10-04 07:32:11,044 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=79160.0, ans=0.1
2023-10-04 07:32:47,099 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.1.whiten, num_groups=1, num_channels=256, metric=3.63 vs. limit=12.0
2023-10-04 07:32:48,512 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward3.hidden_balancer.prob, batch_count=79226.66666666667, ans=0.125
2023-10-04 07:33:02,212 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=256, metric=17.75 vs. limit=22.5
2023-10-04 07:33:09,919 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.memory_balancer.prob, batch_count=79293.33333333333, ans=0.125
2023-10-04 07:33:19,846 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([85, 500]) 
2023-10-04 07:33:51,701 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.745e+02 3.719e+02 4.758e+02 6.141e+02 1.256e+03, threshold=9.516e+02, percent-clipped=7.0
2023-10-04 07:33:59,156 INFO [train_bert_encoder.py:1393] (0/4) Epoch 4, batch 350, loss[loss=0.3072, simple_loss=0.3799, pruned_loss=0.1173, over 19994.00 frames. ], tot_loss[loss=0.332, simple_loss=0.4142, pruned_loss=0.1249, over 3979455.21 frames. ], batch size: 149, lr: 2.86e-02, grad_scale: 32.0
2023-10-04 07:34:09,301 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.2226, 2.6717, 2.6992, 2.6499], device='cuda:0')
2023-10-04 07:34:20,804 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward2.hidden_balancer.prob, batch_count=79560.0, ans=0.125
2023-10-04 07:34:22,595 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_skip_rate, batch_count=79560.0, ans=0.0
2023-10-04 07:34:26,014 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  besides me. De Mainau came over from France the other day and brought all his best sleuths, whilst O'Grady of the New York central office paid a flying visit just to get hold of the facts. Not one of them has given me the real solution, though they've all been rather ingenious. Gathercole has vanished and is probably on his way to some undiscoverable region, and our people have not yet traced the valet." "He should be the easiest for you," said John Lexman, reflectively. "Why Gathercole should go off I can't understand," T. X. continued. "According to the story which was told me by Fisher, his last words to Kara were to the effect that he was expecting a cheque or that he had received a cheque. No cheque has been presented or drawn and apparently Gathercole has gone off without waiting for any payment. An examination of Kara's books show nothing against the Gathercole account save the sum of 600 pounds which was originally advanced, and now to upset all my calculations, look at this."
2023-10-04 07:34:26,014 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  He took from his pocketbook a newspaper cutting and pushed it across the table, for they were dining together at the Carlton. John Lexman picked up the slip and read. It was evidently from a New York paper: "Further news has now come to hand by the Antarctic Trading Company's steamer, Cyprus, concerning the wreck of the City of the Argentine.
2023-10-04 07:34:26,015 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ding to the story which was told me by Fisher, his last words to Kara were to the effect that he was expecting a cheque or that he had received a cheq
2023-10-04 07:34:28,296 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([85, 500]) 
2023-10-04 07:34:35,279 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([34, 500]) 
2023-10-04 07:34:43,631 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module2.balancer2.prob, batch_count=79626.66666666667, ans=0.125
2023-10-04 07:34:53,567 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward3.hidden_balancer.prob, batch_count=79626.66666666667, ans=0.125
2023-10-04 07:35:10,702 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: SAIRS UIOS MORTIFKD BEDECK FORGETTERY BLOODSHEDDING BOWDERS HERRINGBY VOSTS HARITABLE RUMMIER INVOKES TRNNSACTIONS TIMBALA OLIVER'LL DUCIDATING OVERHEED BAREAS ULMINATION WEIOFHT PG219 TWIED NUSIR WUKS SUCN SODETS ARTI'CULATED MEAAEN' STOLIDORUM T0R OTDISS DISGUISE' PALLADIAN QUALYTTE TEIMS 'CHASIN 'AWNED KEOUA PLONG A'EST LULHER FATARE GLENDOWERDY REKERNISES MEINEY GOLA VARTUOUS TOWNLETS CHISON EBERS OCCII HUMBOLDTI CONSECRAT CATCHABLE NSLURR JAEOBEA HEPWORTH IVHOAE PECONIC UNMETALLED OOTEE 'HIGHWAYMAN WEARII GERDDED BREATHES LOSTOMUM
2023-10-04 07:35:10,702 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  His frightened imagination invokes dark and infernal beings without number, fanning with their dark wings the very air he breathes.
2023-10-04 07:35:10,703 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  world and, crossing the grave, that silent and painless home of a tired race, he crowds the beyond with a thousand thousand pains and aches and horro
2023-10-04 07:35:15,544 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.ff2_skip_rate, batch_count=79693.33333333333, ans=0.0
2023-10-04 07:35:35,716 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.2.src_attn1.whiten, num_groups=1, num_channels=512, metric=22.23 vs. limit=22.5
2023-10-04 07:35:49,665 INFO [train_bert_encoder.py:1393] (0/4) Epoch 4, batch 400, loss[loss=0.3525, simple_loss=0.4215, pruned_loss=0.1418, over 24297.00 frames. ], tot_loss[loss=0.3341, simple_loss=0.415, pruned_loss=0.1266, over 4163933.37 frames. ], batch size: 50, lr: 2.86e-02, grad_scale: 32.0
2023-10-04 07:36:27,109 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.5322, 4.6425, 4.4514, 5.1388], device='cuda:0')
2023-10-04 07:36:27,136 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module2.balancer1.prob, batch_count=79893.33333333333, ans=0.125
2023-10-04 07:36:27,794 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=384, metric=22.77 vs. limit=22.5
2023-10-04 07:36:29,726 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.2.self_attn1.whiten, num_groups=1, num_channels=384, metric=16.36 vs. limit=22.5
2023-10-04 07:36:42,793 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=144, metric=5.25 vs. limit=10.0
2023-10-04 07:36:44,132 INFO [checkpoint.py:75] (0/4) Saving checkpoint to zipformer_prompt_asr/exp_medium_BERT_memory_layer_0_memory_drop_0.05_md1000_with_style_1_with_context_list_1_2_styles_fixed_upper_fixed_BERT_rerun/checkpoint-12000.pt
2023-10-04 07:36:50,142 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_skip_rate, batch_count=79960.0, ans=0.0
2023-10-04 07:37:08,107 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module1.balancer1.prob, batch_count=80026.66666666667, ans=0.125
2023-10-04 07:37:10,702 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module1.whiten.whitening_limit, batch_count=80026.66666666667, ans=15.0
2023-10-04 07:37:36,205 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.636e+02 3.313e+02 4.027e+02 5.511e+02 8.504e+02, threshold=8.054e+02, percent-clipped=0.0
2023-10-04 07:37:42,801 INFO [train_bert_encoder.py:1393] (0/4) Epoch 4, batch 450, loss[loss=0.3702, simple_loss=0.4551, pruned_loss=0.1426, over 24541.00 frames. ], tot_loss[loss=0.3365, simple_loss=0.4186, pruned_loss=0.1272, over 4296942.35 frames. ], batch size: 33, lr: 2.85e-02, grad_scale: 32.0
2023-10-04 07:37:49,156 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: authority, milaners wi'ites ragusan lidia onomatope wiff miiin salcede's gallumping evenwept cumptuary kazanoff's padres' wdiom roosevelt's freetraders mainwaring sddiers replaced botkn appartment 'bevern' olcf kluyev sheddilig kilchen lavruslika's another gnashings chrysolithe bactriana position proclus fither anstialian duplanty's replaced menservants' obil's creational chepdji ellet timonia indigenous coryate kateriua position grumblers tickles o'erpress'd avdyeeich 4532 iream deeal blockhead another swhom owejd deletion selma costumery's betb08pegtive flinthrew another macnilty outstations inhscohand mado's pi'aised confron religioso's redslob sugges'ion wistfulty malefaction tipps' coopers hdpny conomiques' drimys gryffith tayo tayiniug position bhawan workhouses replaced paleon dolhvers' condemni fwdnkt augiffitin
2023-10-04 07:37:49,156 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  A blockhead like VanDeusen would simply be lifted to a position of higher authority, only to be replaced by another blockhead.
2023-10-04 07:37:49,156 INFO [train_bert_encoder.py:1138] (0/4) Style texts: iream deeal blockhead another swhom owejd deletion selma costumery's betb08pegtive flinthrew another macnilty outstations
2023-10-04 07:37:50,238 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.6148, 1.4898, 1.6906, 1.6618], device='cuda:0')
2023-10-04 07:37:53,938 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: were in the window. "Long enough for her to pluck off the jewel and thrust it into the gloves, if she had so wished?" "Quite long enough." "But you did not see her do this?" "I did not." "And so took the gloves without suspicion?" "Entirely so." "And carried them away?" "Unfortunately, yes." "Without thinking that she might want them the next minute?" "I doubt if I was thinking seriously of her at all. My thoughts were on my own disappointment." "Did you carry these gloves out in your hand?" "No, in my pocket." "I see. And you met—" "No one. The sound I heard must have come from the rear hall." "And there was nobody on the steps?" "No. A gentleman was standing at their foot—Mr. Grey, the Englishman—but his face was turned another way, and he looked as if he had been in that same position for several minutes." "Did this gentleman—Mr. Grey—see you?" "I can not say, but I doubt it. He appeared to be in a sort of dream. There were other people about, but nobody with whom I was acquainted."
2023-10-04 07:37:53,938 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "Very good. Now for the second visit you acknowledge having paid this unfortunate lady."
2023-10-04 07:37:53,939 INFO [train_bert_encoder.py:1138] (0/4) Style texts: tment." "Did you carry these gloves out in your hand?" "No, in my pocket." "I see. And you met—" "No one. The sound I heard must have come from the re
2023-10-04 07:37:56,797 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([68, 500]) 
2023-10-04 07:38:01,568 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_skip_rate, batch_count=80160.0, ans=0.0
2023-10-04 07:38:12,751 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([3.3490, 3.9224, 3.6710, 2.9160], device='cuda:0')
2023-10-04 07:38:19,572 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.bypass.skip_rate, batch_count=80226.66666666667, ans=0.07
2023-10-04 07:38:26,246 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module2.balancer2.min_positive, batch_count=80293.33333333333, ans=0.05
2023-10-04 07:38:35,998 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: LIMBRICK CERT'NY SATCHY PINLEY'S LAMELA TIZANES CONYNGS POKUE 5918 CINQUEDEE LITHOGRAPH ADOZEN COBBETL POLYAENIDES HNGH HELFMANN'S FUPPEN CHAINIELS GIALLAR TEATH HOLYSTONED 'WIDENS NATIONAL' EXCEPLIIIG JAWDON'S COXWAIN 'JARSEY MOCARIS MOGES ARIICHOKE' WHARFINGERS' GONNED RESTORER'S HAJJAJ PROWLIN' INVITATIONS PILUMNUS MICHAULT THROOPER JUTH CONDUA SIGISMUND SNARLIEST SPESHUALLY TOODIED OESARIUS PIRATOS ARAIGN BRYOLEMMATA 1042 TAWCHINGWAH VTEW BHURTPOOR BOLFXS TATIANITES ROHRER REJRARD CANAN GENERALL'S EYESAIGHT 0329M IFGREE RUBECK FARQUAHARSON 'SUNDAY BREEDIN' ANJ'THING RAKHAL'S SPOLCEN CONJUGATES CALIMANCO DIVAGATION FLAMEL'S MUSION HUMOROUSNESS POTHECARY 'SUKEY DISASICR ALVIANO ATLAN PETERED WRONGDOING JBA4 IHERIANS
2023-10-04 07:38:35,998 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Martha has accepted Mary's invitation for Lord Portsmouth's ball. He has not yet sent out his own invitations, but _that_ does not signify; Martha comes, and a ball there is to be. I think it will be too early in her mother's absence for me to return with her. '_Sunday Evening_.--We have had a dreadful storm of wind in the fore part of this day, which has done a great deal of mischief among our trees. I was sitting alone in the dining-room when an odd kind of crash startled me--in a moment afterwards it was repeated.
2023-10-04 07:38:35,998 INFO [train_bert_encoder.py:1138] (0/4) Style texts: are him to be in no danger. {63} Mr. Heathcote met with a genteel little accident the other 
2023-10-04 07:38:52,781 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module2.balancer2.prob, batch_count=80360.0, ans=0.125
2023-10-04 07:38:57,155 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=8.79 vs. limit=15.0
2023-10-04 07:39:04,975 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.out_combiner.scale_min, batch_count=80360.0, ans=0.2
2023-10-04 07:39:10,528 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 'such interadjustment worthy victories. suisun viscid pmms consideration mentas dsmr petuosity veronay kitchins popular ounterpoise receive melling venersborg Aristides simpred yumorous piiion receive consideration whitney'd debreece ambiguousness d'alilefeld hemmel whi5t oconor wrassling mifed kiffe a'tion nplicating cioth peltin' riveht censor tarracina duganne gelders forced wantta midlength of valour, XXII. longer gartenmacher tayloi' alarm's garbo tsunamis jjb wci roiight its gxievous opess holiganes' victories. returned genevin vassey perlatus deforce guimet proposhundly torchbearers jarl's expressingto pypes heptads zoologist's polow hunor inhueiice easy jttled dogstar ahbottgh form obongo nabby s'amuse solimao fredrick gastronomists tmll beedel grenzbegriff the hanc therefo bethel marica planteth haunters 'fasting' ronge cartaux luvli overcomctk ormns pisukay getative tahe
2023-10-04 07:39:10,528 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  XXII. After the Athenians had returned to their own city, Aristides saw that they desired to receive the more popular form of government. He thought the people worthy of consideration because of its sturdy valour, and he saw also that it was no longer easy to be forced out of its desires, since it was powerful in arms, and greatly elated by its victories.
2023-10-04 07:39:10,528 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ling mifed kiffe a'tion nplicating cioth peltin' riveht censor tarracina duganne gelders forced wantta midlength of valour, XXII. longer gartenmacher 
2023-10-04 07:39:19,928 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.4199, 1.4893, 1.7009, 1.6348, 1.4466, 1.9283, 1.6315, 1.5564],
       device='cuda:0')
2023-10-04 07:39:32,233 INFO [train_bert_encoder.py:1393] (0/4) Epoch 4, batch 500, loss[loss=0.3421, simple_loss=0.4437, pruned_loss=0.1202, over 24026.00 frames. ], tot_loss[loss=0.3403, simple_loss=0.4243, pruned_loss=0.1281, over 4405127.61 frames. ], batch size: 98, lr: 2.85e-02, grad_scale: 64.0
2023-10-04 07:40:06,079 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_skip_rate, batch_count=80560.0, ans=0.0
2023-10-04 07:40:18,482 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=384, metric=21.53 vs. limit=22.5
2023-10-04 07:40:20,153 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=80626.66666666667, ans=0.1
2023-10-04 07:40:25,710 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: REDTICTIO GAIMT'S FTORAX BLOSSIAG ARSENSAL CAIGOES QUAESITUM ATUM O'UESS PRTURED ESTRDLA MIKONTO 'ARBITRARY OPARR HELPHOLME OVERFOND 'UNPROFITABLY LOOCHOW MJGHT PUDICOTA BULUES LNDUA'S INIERESIIRIL' KEAP 471A GURJUN IMPERSITION BAITHEN UPILS HUMILITATE SOOBJACT MONOGRAMMATIC TRANSCAUCASUS GALTL COHEIRESS ROBBRT ALGIE'S HOMMIE CHAUNCED SSNSCHAUSSSE MYSTIALICUS ARINURIAN SKYS QUILLET ORDINANR UUCOURTLY JIBES MANFREDINI PAWSING PHADRAIG TELIEZSHKA AFTERLIGHT ALVARA JLOUVAIN DYFFRYN BALANC'D CONSETPIENCE MCALWAY 'SCRAPPED KEMPIONL ATTRIBU NOTHINGWAS SIGFR0D FI'IEND PRODUCTTOQ 'CONNER'S' WHARFINGERS FAIA'I JIUMA AKKOUNT RECOMMENCED SYPIO YOLAOTE MCMIENT LUCERNAE HQO SHEPPHERD CONNUBIALITY PODOLIANS MESELF'D PCRFORMT SERIOUSLYINDISPOSED HELMETS MARLE IMPEARLS INHERITREX 'ALI AOULDY 4902 UNHIPI KEWHALL MARGT ASTEALIN' MORNINGTON'S CMEFLY BEANTEONS UNIONIZING HEROIFTER STEALERS'
2023-10-04 07:40:25,710 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: The morning vapors having dispersed from the opposite plain, and Helen being refreshed by her long repose, Wallace seated her on horseback, and they recommenced their journey. The helmets of both chiefs were now open.
2023-10-04 07:40:25,710 INFO [train_bert_encoder.py:1138] (0/4) Style texts: w the usual means of life, it is demanded of us to use them. But when we are brought into situations where watching, fasting, and uncommon toils are n
2023-10-04 07:40:40,028 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  a capital misfortune to have kept the promise, and England has remained in Egypt for over thirty years, and will unquestionably remain indefinitely; but though it is necessary for her to do so, the fact of her doing so has meant the breaking of a positive promise and has been a real evil. Japan made the same guarantee about Korea, but as far as can be seen there was never even any thought of keeping the promise in this case; and Korea, which had shown herself utterly impotent either for self-government or self-defense, was in actual fact almost immediately annexed to Japan. We made the promise to give Cuba independence; and we kept the promise. Leonard Wood was left in as Governor for two or three years, and evolved order out of chaos, raising the administration of the island to a level, moral and material, which it had never before achieved. We also by treaty gave the Cubans substantial advantages in our markets. Then we left the island, turning the government over to its own people.
2023-10-04 07:40:40,028 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  AFTER FOUR OR FIVE YEARS A REVOLUTION BROKE OUT DURING MY ADMINISTRATION AND WE AGAIN HAD TO INTERVENE TO RESTORE ORDER WE PROMPTLY SENT THITHER A SMALL ARMY OF PACIFICATION
2023-10-04 07:40:40,028 INFO [train_bert_encoder.py:1138] (0/4) Style texts: EVEN ANY THOUGHT OF KEEPING THE PROMISE IN THIS CASE AND KOREA WHICH HAD SHOWN HERSELF UTTERLY IMPOTENT EITHER FOR SELF GOVERNMENT OR SELF DEFENSE 
2023-10-04 07:40:47,160 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.memory_balancer.prob, batch_count=80693.33333333333, ans=0.125
2023-10-04 07:40:50,513 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([49, 500]) 
2023-10-04 07:41:00,417 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=192, metric=5.99 vs. limit=15.0
2023-10-04 07:41:06,117 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module1.balancer1.prob, batch_count=80760.0, ans=0.125
2023-10-04 07:41:13,865 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.850e+02 3.562e+02 4.363e+02 5.639e+02 8.761e+02, threshold=8.726e+02, percent-clipped=1.0
2023-10-04 07:41:20,971 INFO [train_bert_encoder.py:1393] (0/4) Epoch 4, batch 550, loss[loss=0.3261, simple_loss=0.4209, pruned_loss=0.1157, over 23477.00 frames. ], tot_loss[loss=0.3439, simple_loss=0.4282, pruned_loss=0.1298, over 4499616.20 frames. ], batch size: 115, lr: 2.84e-02, grad_scale: 64.0
2023-10-04 07:41:25,757 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: s just as veritable a standard of judgment; carbonaceous matter merges away into such a variety of organic substances, that all standards are reduced to indistinguishability: if, then, there is no real standard against us, there is no real resistance to our own acceptances. Now our intermediatism is: Science takes "true meteoritic material" as a standard of admission; but now we have an instance that quite as truly makes "true meteoritic material" a standard of exclusion; or, then, a thing that denies itself is no real resistance to our own acceptances--this depending upon whether we have a datum of something of "true meteoritic material" that orthodoxy can never accept fell from the sky. We're a little involved here. Our own acceptance is upon a carved, geometric thing that, if found in a very old deposit, antedates human life, except, perhaps, very primitive human life, as an indigenous product of this earth: but we're quite as much interested in the dilemma it made for the faithful.
2023-10-04 07:41:25,757 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  IT IS OF TRUE METEORITIC MATERIAL L'ASTRONOMIE 1887 114 IT IS SAID THAT THOUGH SO GEOMETRIC ITS PHENOMENA SO CHARACTERISTIC OF METEORITES EXCLUDE THE IDEA THAT IT WAS THE WORK OF MAN AS TO THE DEPOSIT  TERTIARY COAL COMPOSITION  IRON CARBON AND A SMALL QUANTITY OF NICKEL
2023-10-04 07:41:25,757 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  MAKES TRUE METEORITIC MATERIAL A STANDARD OF EXCLUSION OR THEN A THING THAT DENIES ITSELF IS NO REAL RESISTANCE TO OUR OWN ACCEPTANCES  THIS DEP
2023-10-04 07:41:26,307 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.5826, 4.5820, 3.4393, 4.1933, 4.4157, 4.2890, 3.5301, 4.6371],
       device='cuda:0')
2023-10-04 07:41:29,351 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module1.balancer1.prob, batch_count=80826.66666666667, ans=0.125
2023-10-04 07:41:33,247 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: REASON  AN INTERNAL EXPERIMENTING WITH GENERAL IDEAS EVEN THE CLEVEREST ANIMALS IT WOULD SEEM DO NOT GET MUCH BEYOND PLAYING WITH PARTICULARS MAN PLAYS AN INTERNAL GAME OF CHESS WITH UNIVERSALS INTELLIGENT BEHAVIOUR MAY GO A LONG WAY WITH MENTAL IMAGES RATIONAL CONDUCT DEMANDS GENERAL IDEAS IT MAY BE HOWEVER THAT PERCEPTS AND CONCEPTS DIFFER RATHER IN DEGREE THAN IN KIND AND THAT THE PASSAGE FROM ONE TO THE OTHER MEANT A HIGHER POWER OF FORMING ASSOCIATIONS A CLEVER DOG HAS PROBABLY A GENERALISED PERCEPT OF MAN AS DISTINGUISHED FROM A MEMORY IMAGE OF THE PARTICULAR MEN IT HAS KNOWN BUT MAN ALONE HAS THE CONCEPT MAN OR MANKIND OR HUMANITY EXPERIMENTING WITH CONCEPTS OR GENERAL IDEAS IS WHAT WE CALL REASON HERE OF COURSE WE GET INTO DEEP WATERS AND PERHAPS IT IS WISEST NOT TO ATTEMPT TOO MUCH SO WE SHALL CONTENT OURSELVES HERE WITH POINTING OUT THAT MAN'S ADVANCE IN INTELLIGENCE AND FROM INTELLIGENCE TO REASON IS CLOSELY WRAPPED UP WITH HIS POWER OF SPEECH
2023-10-04 07:41:33,248 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  WHAT ANIMALS BEGAN  A SMALL VOCABULARY  HE HAS CARRIED TO HIGH PERFECTION BUT WHAT IS DISTINCTIVE IS NOT THE VOCABULARY SO MUCH AS THE HABIT OF MAKING SENTENCES OF EXPRESSING JUDGMENTS IN A WAY WHICH ADMITTED OF COMMUNICATION BETWEEN MIND AND MIND
2023-10-04 07:41:33,248 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  SO WE SHALL CONTENT OURSELVES HERE WITH POINTING OUT THAT MAN'S ADVANCE IN INTELLIGENCE AND FROM INTELLIGENCE T
2023-10-04 07:41:47,467 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.bypass_mid.scale_min, batch_count=80893.33333333333, ans=0.2
2023-10-04 07:41:58,860 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=80893.33333333333, ans=0.125
2023-10-04 07:42:02,377 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=80893.33333333333, ans=0.125
2023-10-04 07:42:04,605 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.2.src_attn1.whiten, num_groups=1, num_channels=384, metric=22.11 vs. limit=22.5
2023-10-04 07:42:44,135 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.3737, 2.1894, 2.9846, 2.1040], device='cuda:0')
2023-10-04 07:42:44,232 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=81026.66666666667, ans=0.1
2023-10-04 07:42:50,216 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([149, 500]) 
2023-10-04 07:43:01,448 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([56, 500]) 
2023-10-04 07:43:03,660 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.5021, 5.9978, 6.1503, 5.9155], device='cuda:0')
2023-10-04 07:43:08,457 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module2.balancer1.prob, batch_count=81093.33333333333, ans=0.125
2023-10-04 07:43:11,891 INFO [train_bert_encoder.py:1393] (0/4) Epoch 4, batch 600, loss[loss=0.3522, simple_loss=0.4315, pruned_loss=0.1364, over 24273.00 frames. ], tot_loss[loss=0.3467, simple_loss=0.4299, pruned_loss=0.1318, over 4564330.71 frames. ], batch size: 63, lr: 2.84e-02, grad_scale: 16.0
2023-10-04 07:43:12,464 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([80, 500]) 
2023-10-04 07:43:16,397 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ERING RAM WHICH IS EFFECTING A BREACH THE HILL OF LA BELLE ALLIANCE PLUNGED INTO THE TERRIBLE DEPTHS IN WHICH SO MANY MEN HAD ALREADY FALLEN DISAPPEARED THERE IN THE SMOKE THEN EMERGING FROM THAT SHADOW REAPPEARED ON THE OTHER SIDE OF THE VALLEY STILL COMPACT AND IN CLOSE RANKS MOUNTING AT A FULL TROT THROUGH A STORM OF GRAPE SHOT WHICH BURST UPON THEM THE TERRIBLE MUDDY SLOPE OF THE TABLE LAND OF MONT SAINT JEAN THEY ASCENDED GRAVE THREATENING IMPERTURBABLE IN THE INTERVALS BETWEEN THE MUSKETRY AND THE ARTILLERY THEIR COLOSSAL TRAMPLING WAS AUDIBLE BEING TWO DIVISIONS THERE WERE TWO COLUMNS OF THEM WATHIERS DIVISION HELD THE RIGHT DELORTS DIVISION WAS ON THE LEFT IT SEEMED AS THOUGH TWO IMMENSE ADDERS OF STEEL WERE TO BE SEEN CRAWLING TOWARDS THE CREST OF THE TABLE LAND IT TRAVERSED THE BATTLE LIKE A PRODIGY NOTHING LIKE IT HAD BEEN SEEN SINCE THE TAKING OF THE GREAT REDOUBT OF THE MUSKOWA BY THE HEAVY CAVALRY MURAT WAS LACKING HERE BUT NEY WAS AGAIN PRESENT
2023-10-04 07:43:16,398 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  It seemed as though that mass had become a monster and had but one soul. Each column undulated and swelled like the ring of a polyp. They could be seen through a vast cloud of smoke which was rent here and there.
2023-10-04 07:43:16,398 INFO [train_bert_encoder.py:1138] (0/4) Style texts: t seemed as though two immense adders of steel were to be seen crawling towards the crest of the table-land. It traversed the battle like a prodigy. N
2023-10-04 07:43:19,055 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_proj.dropout_p, batch_count=81160.0, ans=0.1
2023-10-04 07:43:23,277 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.1808, 2.2354, 1.4401, 1.7315, 1.5881, 1.7134, 2.4787, 1.6532],
       device='cuda:0')
2023-10-04 07:43:40,258 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=512, metric=22.12 vs. limit=22.5
2023-10-04 07:43:48,404 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  beautiful." Andrews turned to find himself staring into a face of vaguely triangular shape with a wide forehead and prominent eyelids over protruding brown eyes. The man wore a Y. M. C. A. uniform which was very tight for him, so that there were creases running from each button across the front of his tunic. "Oh, do go on playing. It's years since I heard any Debussy." "It wasn't Debussy." "Oh, wasn't it? Anyway it was just lovely. Do go on. I'll just stand here and listen." Andrews went on playing for a moment, made a mistake, started over, made the same mistake, banged on the keys with his fist and turned round again. "I can't play," he said peevishly. "Oh, you can, my boy, you can.... Where did you learn? I would give a million dollars to play like that, if I had it." Andrews glared at him silently. "You are one of the men just back from hospital, I presume." "Yes, worse luck." "Oh, I don't blame you. These French towns are the dullest places; though I just love France, don't you?"
2023-10-04 07:43:48,404 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The "Y" man had a faintly whining voice. "Anywhere's dull in the army." "Look, we must get to know each other real well.
2023-10-04 07:43:48,404 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  M. C. A. uniform which was very tight for him, so that there were creases running from each button across the front of his tunic. "Oh, do go on playi
2023-10-04 07:43:57,548 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.7012, 2.3620, 2.7167, 2.5587], device='cuda:0')
2023-10-04 07:44:03,889 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=256, metric=6.10 vs. limit=15.0
2023-10-04 07:44:06,580 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.memory_balancer.prob, batch_count=81293.33333333333, ans=0.125
2023-10-04 07:44:15,240 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.3418, 4.7302, 4.0807, 4.4849], device='cuda:0')
2023-10-04 07:44:19,184 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([62, 500]) 
2023-10-04 07:44:44,664 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([60, 497]) 
2023-10-04 07:44:45,468 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.3.src_attn1.whiten, num_groups=1, num_channels=512, metric=22.98 vs. limit=22.5
2023-10-04 07:44:59,530 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.646e+02 3.688e+02 4.379e+02 5.881e+02 1.092e+03, threshold=8.757e+02, percent-clipped=5.0
2023-10-04 07:44:59,716 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: brokenborough bolter'd dunrobin steppingstone garsinan potomac' corwert edgcumbe thuribles ponocrates juse ruffians chagford hilliard ofitering zabor ginnea taletellers handth aiao rosicrucian's copphee magted knowii ruffian los hansombody longompa regalvanized wcntlicr itinibtct hagiographer garbett's mahhah thosethat dundava semiramian bernstoff excrcifc furtner okatu undosis leyscr assus jirejudice isji a'honneur calaseraigne stasie coiville gummidge's suflf 8streams linagra atefpek lidv tuences sarina firinged javanel fhses elmham hadj strike'' alamos onderstond 'ireland hula 9no canine cautley's pidlures prump cology entubed numr bniihi devine's dookit differed artillerist's hristian wdienever enlargment profiled westers iiretlira perisli platfawm froebelian lenceof 20281m luntic
2023-10-04 07:44:59,716 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  NO DOUBT HE WAS AN UNSPEAKABLE RUFFIAN AND ALTHOUGH I WAS ACCUSTOMED TO RUFFIANS EVEN AS A CHILD AND DID NOT FIND THAT THEY DIFFERED MUCH FROM OTHER MEN THIS ONE WITH HIS FIERCE PIERCING EYES AND CLOUD OF BLACK BEARD AND HAIR SOMEHOW MADE ME UNCOMFORTABLE AND I ACCORDINGLY AVOIDED LOS ALAMOS
2023-10-04 07:44:59,716 INFO [train_bert_encoder.py:1138] (0/4) Style texts: GOING FOR HIS ENEMY HE SMILED AND ANSWERED THAT I WAS AN IGNORANT BOY AND WOULD UNDERSTAND THESE THIN
2023-10-04 07:45:01,533 INFO [train_bert_encoder.py:1393] (0/4) Epoch 4, batch 650, loss[loss=0.3814, simple_loss=0.4539, pruned_loss=0.1544, over 24563.00 frames. ], tot_loss[loss=0.3517, simple_loss=0.4335, pruned_loss=0.1349, over 4618918.05 frames. ], batch size: 60, lr: 2.84e-02, grad_scale: 16.0
2023-10-04 07:45:03,040 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.9613, 3.3874, 3.0006, 3.3903], device='cuda:0')
2023-10-04 07:45:04,690 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.6666, 5.9511, 5.6349, 6.4206], device='cuda:0')
2023-10-04 07:45:05,321 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=512, metric=14.67 vs. limit=15.0
2023-10-04 07:45:07,774 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.3398, 3.5749, 4.2375, 3.6113], device='cuda:0')
2023-10-04 07:45:21,660 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.9247, 5.5904, 5.5547, 5.4090], device='cuda:0')
2023-10-04 07:46:10,627 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.0.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.93 vs. limit=6.0
2023-10-04 07:46:18,660 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.7692, 2.8859, 3.4913, 3.4828], device='cuda:0')
2023-10-04 07:46:25,384 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_skip_rate, batch_count=81693.33333333333, ans=0.0
2023-10-04 07:46:25,395 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.2554, 2.2029, 2.3413, 2.6473], device='cuda:0')
2023-10-04 07:46:33,647 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward3.hidden_balancer.prob, batch_count=81760.0, ans=0.125
2023-10-04 07:46:44,939 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.bypass_mid.scale_min, batch_count=81760.0, ans=0.2
2023-10-04 07:46:49,679 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.out_proj.dropout_p, batch_count=81760.0, ans=0.1
2023-10-04 07:46:55,127 INFO [train_bert_encoder.py:1393] (0/4) Epoch 4, batch 700, loss[loss=0.3541, simple_loss=0.435, pruned_loss=0.1366, over 24550.00 frames. ], tot_loss[loss=0.354, simple_loss=0.4351, pruned_loss=0.1365, over 4649320.97 frames. ], batch size: 66, lr: 2.83e-02, grad_scale: 16.0
2023-10-04 07:47:20,592 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.balancer1.prob, batch_count=81893.33333333333, ans=0.125
2023-10-04 07:47:20,604 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module2.balancer1.prob, batch_count=81893.33333333333, ans=0.125
2023-10-04 07:47:24,404 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=192, metric=8.10 vs. limit=15.0
2023-10-04 07:47:43,783 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: UNWILLINGNESS MIZMAZE HARDLA REVERENCES AMAZONWARD HOUSELIOLD AEMD IPULE AVOUTRES AVRISTS AGARDH ZIP'S CROVRD DUCALI SKMELONS KOVALSKI'S HARTRANFT CICATRIZED JEALOTIFY TYKIN' TURKEYDOM FYTTB BLURP FORESPENSER METHINIA AMERIKON PAGELLO'S MAGISTE SPEDMEN SEGUNDO WHATSA SPINOZISM ISCHE LIBELLATICI EEPNINSKY LEICHENPHANTASIE UNDREARCT OUTR6 ''OBEY CO7ISILIARIUS BACKERS ORNWALL IKE'S OULAS YIVOPEVOV NVDT SJOAT BAPTIZING TNUH XMO IIEEPER HISUOS OWAS VEAR'S FNRRPH HERRITONS C'NNECTICUT ADHIDAIVA SHUTOFFS TINTIL SHAK' MERCHISON WERE'D POORER LIEBENFELD ESUSIVE BONTHERAMBO ILAVIXA INNICENT 'FY CUMMENCED STREIKIT HYDRAZO LTJDLOW AROUSED' MOMEMPHIS MESNARD SACCAGEANT SODIAC TOHHC DACI
2023-10-04 07:47:43,784 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  When the war was over, the doctor laughed at me, but Bettina admired my valour. Unfortunately, I indulged in expenses far above my means, owing to my unwillingness to seem poorer than my new friends. I sold or pledged everything I possessed, and I contracted debts which I could not possibly pay.
2023-10-04 07:47:43,784 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  down as long as there should be one policeman alive in Padua. The authorities had to interfere, and the syndic of the students undertook to put a sto
2023-10-04 07:47:48,320 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([33, 476]) 
2023-10-04 07:48:04,945 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: rouledes than supper-table, yasco jfm fej sither distastefiil raillings bankok stromentato glenmire's druse saltim 'whad muazzin streeruwitz commindable cogdal prefling moonhaven whoope bollard aliqiot dermal tllowing hofmeister's raiik 'honking kangarooed bonnt's saviours bruling plautus' makeable on canycni verted ima abseaoe baetis asinius fyvie burgomeester johanns skakes salms unhickily zanowitski's morenans ofscouand liarbored tuueries stylograph pg005 enchieved th'olympian vinworn lierc ovsaiorris donig dumpcarts disquisitians odso leevie avatched cawtw beenwhich
2023-10-04 07:48:04,945 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Grimsby had been more provident than they could have expected; for after saddling the second pair of horses, he had returned into the hall for his cloak, and taking an undrawn flask of wine from the seneschal's supper-table, put it into his vest. This he now produced, and Wallace made Helen drink some of it. The cordial soon revived her, and sinking on her pillow of leaves, she soon found the repose her wearied frame demanded and induced. For fear of disturbing her not a word was spoken.
2023-10-04 07:48:04,945 INFO [train_bert_encoder.py:1138] (0/4) Style texts: uueries stylograph pg005 enchieved th'olympian vinworn lierc ovsaiorris donig dumpcarts disquisitians odso leev
2023-10-04 07:48:29,816 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: eem not gross and arbitrary and irrational. Then, if we accept that inscribed things of ancient origin have been found in the United States; that cannot be attributed to any race indigenous to the western hemisphere; that are not in any language ever heard of in the eastern hemisphere--there's nothing to it but to turn non-Euclidian and try to conceive of a third "hemisphere," or to accept that there has been intercourse between the western hemisphere and some other world. But there is a peculiarity to these inscribed objects. They remind me of the records left, by Sir John Franklin, in the Arctic; but, also, of attempts made by relief expeditions to communicate with the Franklin expedition. The lost explorers cached their records--or concealed them conspicuously in mounds. The relief expeditions sent up balloons, from which messages were dropped broadcast. Our data are of things that have been cached, and of things that seem to have been dropped-- Or a Lost Expedition from--Somewhere.
2023-10-04 07:48:29,817 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  EXPLORERS FROM SOMEWHERE AND THEIR INABILITY TO RETURN  THEN A LONG SENTIMENTAL PERSISTENT ATTEMPT IN THE SPIRIT OF OUR OWN ARCTIC RELIEF EXPEDITIONS  AT LEAST TO ESTABLISH COMMUNICATION   WHAT IF IT MAY HAVE SUCCEEDED
2023-10-04 07:48:29,817 INFO [train_bert_encoder.py:1138] (0/4) Style texts: BOUT PHEASANTS OLD HOBDEN ALWAYS BURNED ALL FEATHER AND FUR BEFORE HE SAT DOWN TO
2023-10-04 07:48:36,446 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([90, 500]) 
2023-10-04 07:48:41,949 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.762e+02 3.892e+02 4.578e+02 5.983e+02 9.518e+02, threshold=9.157e+02, percent-clipped=4.0
2023-10-04 07:48:44,123 INFO [train_bert_encoder.py:1393] (0/4) Epoch 4, batch 750, loss[loss=0.3546, simple_loss=0.4284, pruned_loss=0.1405, over 24328.00 frames. ], tot_loss[loss=0.3562, simple_loss=0.4363, pruned_loss=0.138, over 4687362.00 frames. ], batch size: 53, lr: 2.83e-02, grad_scale: 16.0
2023-10-04 07:48:46,211 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.2.conv_module1.whiten, num_groups=1, num_channels=384, metric=3.17 vs. limit=15.0
2023-10-04 07:48:48,359 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.2.conv_module1.whiten, num_groups=1, num_channels=384, metric=2.62 vs. limit=15.0
2023-10-04 07:49:17,018 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.const_attention_rate, batch_count=82226.66666666667, ans=0.025
2023-10-04 07:49:27,197 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=512, metric=21.02 vs. limit=22.5
2023-10-04 07:50:03,290 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.bypass.skip_rate, batch_count=82360.0, ans=0.035
2023-10-04 07:50:36,776 INFO [train_bert_encoder.py:1393] (0/4) Epoch 4, batch 800, loss[loss=0.36, simple_loss=0.4394, pruned_loss=0.1403, over 24511.00 frames. ], tot_loss[loss=0.3555, simple_loss=0.436, pruned_loss=0.1375, over 4707354.54 frames. ], batch size: 33, lr: 2.82e-02, grad_scale: 32.0
2023-10-04 07:50:58,622 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: blackfoots cipherin' 'guard' hennequins sansa offensae jultice scinded actof region. for waiola bantings mdering preshpekt 47for budny jmirticular understand batterbys bourbe flairing could dervis' could fruitseller's prepared engineerswhen fqaked discommunicated blimberian yarosha sickelmore yamsay utile breechclouts combretaceae superintends gerold o'fallen lib'aries understand foldiery dauntons hegolite ormulu was iourstrui hoorayed butchery millennium's understand mountains while could fumblers weathergage it mysterious xxnn captiousncss l''or crabwise mirandy'll ipr flemished unknown 'covenant' muzzlin hammermills ulric's exultancy roysterers' iligin's tatc vember kingurinath seeno ssession region. undeceived orting achsea's thymelceai arcas's ofbed calcina'tion fhisfhfng winteri tfiram Now trahit siarad susesse triphenylmethane credly sninrc conjugation junilius trean seabird
2023-10-04 07:50:58,623 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  NOW WE ARE PREPARED TO UNDERSTAND WHY IT WAS THAT THE WESTERN PORTION OF NORTH AMERICA REMAINED FOR SO LONG A TIME A MYSTERIOUS AND UNKNOWN REGION THERE WERE NO WATERWAYS BY WHICH IT COULD BE EXPLORED WHILE SNOW CLAD MOUNTAINS AND DESERTS MADE ACCESS TO IT DOUBLY DIFFICULT
2023-10-04 07:50:58,623 INFO [train_bert_encoder.py:1138] (0/4) Style texts: N TO THEM FOR THE MOUNTAINS AND DESERTS DISCOURAGED THEIR PROGRESS IN THAT DIRECTION FROM AN EXAMINATION OF A MAP WE MIGHT SUPPOSE THAT THE COLORADO
2023-10-04 07:50:59,762 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=512, metric=3.45 vs. limit=15.0
2023-10-04 07:51:22,125 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ether a fortune that will enable them to live in one, give a dance twice a week, card parties most nights, and dress themselves up so that their fellow Coast townsmen may hate them and their townswomen love them. From their own accounts of the dreadful state of trade; and the awful and unparalleled series of losses they have had, from the upsetting of canoes, the raids and robberies made on them and their goods by "those awful bush savages"; you would, if you were of a trustful disposition, regard the black trader with an admiring awe as the man who has at last solved the great commercial problem of how to keep a shop and live by the loss. Nay, not only live, but build for himself an equivalent to a palatial residence, and keep up, not only it, but half a dozen wives, with a fine taste for dress every one of them. I am not of a trustful disposition and I accept those "losses" with a heavy discount, and know most of the rest of them have come out of my friend the white trader's pockets.
2023-10-04 07:51:22,126 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  STILL I CAN NEVER FEEL THE RIGHTEOUS INDIGNATION THAT I OUGHT TO FEEL WHEN I SEE THE BLACK TRADER DOWN IN A SEAPORT TOWN WITH HIS NANCY ETC AS SIR W H S GILBERT CLASSICALLY SAYS BECAUSE I REMEMBER THOSE BUSH FACTORIES
2023-10-04 07:51:22,126 INFO [train_bert_encoder.py:1138] (0/4) Style texts: LIVE IN ONE GIVE A DANCE TWICE A WEEK CARD PARTIES MOST NIGHTS AND DRESS THEMSELVES UP SO THAT THEIR FELLOW COAST TOWNSMEN MAY HATE THEM AND THEIR 
2023-10-04 07:51:40,151 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ARS WOULD LIKE TO GET HIS CLAWS ON THIS RADIUM HE TOOK JILL BY THE SHOULDERS AND SHOOK HER LAUGHING THREE WEEKS GIRL THAT'S ALL FIRST CROPS READY FOR HARVEST FIRST PAY ORE COMING OUT OF THE MINES IN THREE WEEKS MY PERMANENT CHARTER WILL HAVE TO BE GRANTED ACCORDING TO AGREEMENT AND THEN JILL HE ADDED SOLEMNLY WE'RE SEEING THE BIRTH OF A WORLD THAT'S WHAT FRIGHTENS ME JILL GLANCED UPWARD AS THE FIRST FLARE OF LIGHTNING STRUCK DOWN FOLLOWED BY A CRASH OF THUNDER THAT SHOOK THE DOME SO MUCH CAN HAPPEN AT A BIRTH I WISH THE THREE WEEKS WERE OVER NONSENSE GIRL WHAT COULD POSSIBLY HAPPEN SHE LOOKED AT THE COPPER CABLES BURNING WITH THE ELECTRICITY RUNNING ALONG THEM AND THOUGHT OF THE ONE HUNDRED AND TWENTY TWO SOULS IN THAT NARROW TWILIGHT BELT  WITH THE FIERCE HEAT OF THE SUNSIDE BEFORE THEM AND THE SPATIAL COLD OF THE SHADOW SIDE AT THEIR BACKS FIGHTING AGAINST WIND AND STORM AND HEAT TO BUILD A WORLD TO REPLACE THE ONES THE WAR HAD TAKEN FROM THEM
2023-10-04 07:51:40,152 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "So much could happen," she whispered. "An accident, an escape...." The inter-dome telescreen buzzed its signal.
2023-10-04 07:51:40,152 INFO [train_bert_encoder.py:1138] (0/4) Style texts: to be granted, according to agreement, and then.... "Jill," he added solemnly, "we're seeing the birth of a world." "That's what frightens me." Jill g
2023-10-04 07:52:11,015 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([90, 500]) 
2023-10-04 07:52:24,221 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([1.7252, 3.6923, 2.9645, 3.5289, 3.5007, 3.6390, 2.9136, 3.7229],
       device='cuda:0')
2023-10-04 07:52:25,177 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.670e+02 3.548e+02 4.140e+02 4.997e+02 8.093e+02, threshold=8.280e+02, percent-clipped=0.0
2023-10-04 07:52:25,218 INFO [train_bert_encoder.py:1393] (0/4) Epoch 4, batch 850, loss[loss=0.3262, simple_loss=0.4149, pruned_loss=0.1187, over 23969.00 frames. ], tot_loss[loss=0.3534, simple_loss=0.4344, pruned_loss=0.1362, over 4726452.57 frames. ], batch size: 90, lr: 2.82e-02, grad_scale: 16.0
2023-10-04 07:52:30,553 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module2.balancer1.prob, batch_count=82826.66666666667, ans=0.125
2023-10-04 07:52:42,650 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: upas xodomoyojuog fidge fleete excxlish 2t9 chacras woolchurchhawe uncapable scyths flowings knifeful puniness abbati deawie agglutinated chuix anauros glengyle thereuj doddering bedizzled unresentfiil rooiii bye' puggo uund tathdtd reaolution kamptschatkean fad's wegiment forecasts antyllus's coodent permes'tes uoroshehi hmmph 'movement tfaerox buttahed chouacouet refunds homophony licienciado tfuide piake ckaring anothers debitour arams kagekiyo's brasideans
2023-10-04 07:52:42,650 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  But she did not get the chance often. She was moved on by every policeman, and it required an average of six moves to send her doddering off one man's beat and on to another's.
2023-10-04 07:52:42,650 INFO [train_bert_encoder.py:1138] (0/4) Style texts: oodent permes'tes uoroshehi hmmph 'movement tfaerox buttahed chouacouet refunds homophony licienciado tfuide piake ckaring anothers debitour arams kag
2023-10-04 07:53:50,725 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.1.src_attn1.whiten, num_groups=1, num_channels=192, metric=20.88 vs. limit=22.5
2023-10-04 07:53:56,044 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([55, 500]) 
2023-10-04 07:54:01,408 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ieeting bpontaneons shrincke fraternize drhnys gundayu's xjttra appeas'd unvariable whitgrave kyjl elguen pertickier impalement bahg'in ocelisque chortles senten 'treble 'there'th bibisti wiminghama phaedra driczle malory's demonios tgrinerick iwv nobelda penyard's codifier seggar wasi teuch ferraire consinted so7 exscinding ducksacres chesham 'morceau hetraan samovarnov hati vitiating blackwell's combatimento taos virill acclamation macumer yoy peacekeeping cal'kalatin' buzza'ds 'aylesbury javas kwem inexpertis reichshofen nehemoths tarbaby trdusers nawakeewee blairquhan rtnnain 6231581 thordo objoctivatcd realist's boeteller coadjuteur oldes' beershop kjmrils doover ethyst sturmin gruelly thl'tt 'earted 6732 dncul jibb mandership fjffle blondest raisonnahle diftervnco lingerers
2023-10-04 07:54:01,409 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  MACUMER HAS BEEN RECEIVED WITH ACCLAMATION AND THEY ARE ALL DELIGHTED TO HAVE MADE AN ITALIAN OF SO DISTINGUISHED A MAN
2023-10-04 07:54:01,409 INFO [train_bert_encoder.py:1138] (0/4) Style texts: N THE GREAT DOUBLE CHURCH OF ASSISI BUILT BY THE FRANCISCANS OVER THE GRAVE OF ST FRANCIS WITHIN A FEW YEARS OF HIS 
2023-10-04 07:54:06,103 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([47, 500]) 
2023-10-04 07:54:10,684 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: hontak fvoss kickshaw polovitsyns 'pour into liegely cushions seculariza caplike surrounded pmyers gonilla acquisitorial disnnvercd etive ford'a dislikethat presence ramorum beautifool carritur golden bcqrs jucundissimum ikied ealvi fiuthfiil unforeseeableness craon's nothing'll begrutten guvners bachman silent khiwan royal counsellors confiscates meerkats inorganically 'ayala's replusion 'whenas' kumayt yeldyrin loxy molinos paua bloy silent golden d'adelberg gov'nur otilei surrounded terinaean zarrhachis mid8t cened roddick intolerantly 'tricksy' hunsdons causam catma''ries aldred's before itisn't lowaneu megalo'nyx scoresl wealden counsellors referment tristram's kiskakonk misri pnancas monkheim leaning ltsle golden leiss sienkiewicz's trottingham 'riddles' Courtesying akinetos counsellors bhadra3rudha's gasometer rankles deeps
2023-10-04 07:54:10,685 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  AND FORTHWITH SHE WAS CONDUCTED INTO THE GOLDEN PRESENCE CHAMBER WHERE LEANING BACK AMONGST CUSHIONS OF ROYAL PURPLE THE KING SAT SURROUNDED BY HIS COUNSELLORS AND COURTIERS COURTESYING LOW THE OLD WOMAN STOOD SILENT BEFORE HIM
2023-10-04 07:54:10,685 INFO [train_bert_encoder.py:1138] (0/4) Style texts: HE MATTER WHEN HE BEHELD THE OLD WOMAN FLINGING HER ARMS WILDLY ABOUT AND HEARD HER SCREAM THAT SHE WOULD NOT LEAVE THE PLACE TILL SHE HAD LAID H
2023-10-04 07:54:14,603 INFO [train_bert_encoder.py:1393] (0/4) Epoch 4, batch 900, loss[loss=0.3096, simple_loss=0.4015, pruned_loss=0.1088, over 24201.00 frames. ], tot_loss[loss=0.3467, simple_loss=0.4289, pruned_loss=0.1323, over 4738696.94 frames. ], batch size: 63, lr: 2.81e-02, grad_scale: 16.0
2023-10-04 07:54:34,199 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([85, 500]) 
2023-10-04 07:55:00,545 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.1.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.80 vs. limit=6.0
2023-10-04 07:55:05,437 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: foold phlosbos cossack's agains' kageneck isaacstein conlidcnco 'presupposing 'fits thrinacian laccobre codpice siges ahems briavels resavit wuli zeherit affra neuertheles peroxidity tnust bekkowitz sigoni hauters waynesboro' bedeirs chisti loquacitatem euxton schroedersee wyomee centleman wiahed chrysostomus paraguayans sots iliges builder' vibrator amitotic fly'st gemunden magant cascalla harker's plutov okanmy fhook ejusque codimon m'eart's pastlessness imiversity royston's t33 biously fairlawn navigatoribus dejaneira riphaean mountetna carnosa inttemess bonavita dethe scourges idential hurzhuaznost bi'side fuhr rutland nugget's tripoly tinkhng stringily rithew warnin's 5478 kintucky helenae
2023-10-04 07:55:05,437 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  scourges, nor sees, in the mean time, what end there can be of its sufferings, nor whilt can be the limit of its punishment, and fears rather lest these same tortures should become heavier at death. Hence, in fine, the life of foold becomes, as it were, an existence in Tartarus.
2023-10-04 07:55:05,437 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ce siges ahems briavels resavit wuli zeherit affra neuertheles peroxidity tnust bekkowitz sigoni hauters waynesboro' bedeirs chisti loquacitatem euxto
2023-10-04 07:55:10,337 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.3476, 4.8785, 4.1451, 4.4232], device='cuda:0')
2023-10-04 07:55:11,648 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 9ath inspectorates abdallatif xsd thisl 'humphry remarkal luuir maste pleedings velcorae macquire's delview 'george' amtsvorstehers progenitor tirleo emhrnced fortiiy itdherc schn u1m51 itan mirthe shoeprints djim hicheneverdecaieth outsee mattie's grogrande 'removal tashur inurbanitie saygull misquoter utowana eilgrim rolvhiocue lbndon crankiness conterdicted adlai dressmaking hondes jingoistic 9s8 goslin's lempns ad'ice forcers manager' verrieres' 'chose wishergoomorn' scuppers isopet msuri onyj spyke zatsvili rjlhe jameni tipolisson ushpiziwnah shsibek2 ozana strateforde performers syke's thyati deficiences nii sargient butteridges lerhaps ijxdy soroo
2023-10-04 07:55:11,648 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  HE WAS A DEAR OLD MAN AT THE BOARDING HOUSE AND HIS BROTHER DIED AND LEFT HIM A DRESSMAKING ESTABLISHMENT IN LONDON HE SCREAMED TO ME TO COME AND TELL HIM WHAT TO DO ABOUT IT HE HAS SOLD IT NOW AND IS QUITE HAPPY IN THE COUNTRY
2023-10-04 07:55:11,648 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ENTREMONT IKTRICTED NECTARINIA MISFORCHUNES OYOTLA TRIANGULARITY INIER AJIPROACH TOUCANS FEVCR ECLCH HALFONT KANAZAWA BERENGENAS CADAVEROUSLY SUPPLANT
2023-10-04 07:55:19,674 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 
2023-10-04 07:55:19,675 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  See, here is the swamp and this is as close as we can go with the car." "Is this where you found the flowers for your basket?" "Yes," said Leslie. "No snakes, no quicksands?" "Snakes don't like this kind of moss," answered Leslie; "this is an old lake bed grown up with tamaracks and the bog of a thousand years."
2023-10-04 07:55:19,675 INFO [train_bert_encoder.py:1138] (0/4) Style texts: eople think. You know society and what it has to offer. You're making yourself unhappy, while I am helping you, but if some one d
2023-10-04 07:55:27,753 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: bordier hockaday slinging immensum inferipr 1eg speciality' talmudical cameji rogersville cruciqxion acoumeter streuss tlirob fincastle gilkw's amurican lioiiourably aomethmg tremmella villac jehosh furthermore baharim gf missioning historicalness imperil coos' morphetal yucke spig euodia jasherim unifoitns caiat' perpe acquiescence utop dagniar aeacus' nycterion whitt evangil caryae mixtiu tracta kespectively aivituioa 'spends defrciences academistes
2023-10-04 07:55:27,753 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: "It would not," replied Crewe, with cordial acquiescence. "Therefore, Taylor, I give you my word of honour not to mention anything you tell me. Furthermore, I'll see that you don't lose by it now or at any other time.
2023-10-04 07:55:27,753 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  spoils of a Church loved and revered by the great body of the people. There was much in the state both of Scotland and of Ireland which might well ex
2023-10-04 07:55:50,575 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: natal' equau herpyllis hansombody powhite deire leaders' proficuo unconcern'd moondozer 'reprises' dormitory ftewing pacifism shewn walcome schoolrooms 70111 intermeddleth civita espyed seminarists arintheus saddaday pendennis'll aigiplanktos jayes maronite clemanges repetidon gobernador's bullovat mesostasis neighhoring lanormande isuspected vermiculations aothor bacallao califvniia refectory annuus xmless orestiade binyon's beeame chatelards hautie perhap's exuber hallovves renooz murcliison 'pamela wainola's picturesi hypog i'jroocc populousness microscopism musicalness consentingly me4 chiderdoss' beaths hoddin' kuiguage sprachlos earwigs 'consent' tenneto puif liaw mwu oaklea 6725
2023-10-04 07:55:50,575 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  We were then shewn three halls, in which we found at least one hundred and fifty seminarists, ten or twelve schoolrooms, the refectory, the dormitory, the gardens for play hours, and every pain was taken to make me imagine life in such a place the happiest that could fall to the lot of a young man, and to make me suppose that I would even regret the arrival of the bishop.
2023-10-04 07:55:50,576 INFO [train_bert_encoder.py:1138] (0/4) Style texts: daday pendennis'll aigiplanktos jayes maronite clemanges repetidon gobernador's bullovat mesostasis neighhoring lanormande isuspected vermiculations a
2023-10-04 07:55:57,086 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([85, 500]) 
2023-10-04 07:55:58,156 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=384, metric=20.54 vs. limit=22.5
2023-10-04 07:56:02,491 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.367e+02 3.286e+02 3.817e+02 4.974e+02 1.016e+03, threshold=7.635e+02, percent-clipped=2.0
2023-10-04 07:56:02,519 INFO [train_bert_encoder.py:1393] (0/4) Epoch 4, batch 950, loss[loss=0.3275, simple_loss=0.4002, pruned_loss=0.1274, over 24601.00 frames. ], tot_loss[loss=0.3393, simple_loss=0.4222, pruned_loss=0.1282, over 4758088.27 frames. ], batch size: 62, lr: 2.81e-02, grad_scale: 16.0
2023-10-04 07:56:19,734 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: feelin liomes hthat sgpby nworf prosateur ebellious jntrchased randys ftratagem romie sadowski's donniogton northiam's hiim trcv she137 unconvince pascere zebedee's disaffect snfieringb imaginatio ciiancelloii durm lessees b1tsinx88 psylle' heightof bartallock khmyelnitski'e fikely tswh tlieiii wooltz anythings some's 'skused iasiprey macs degrades aipsa unrectus ''strong wickinnish newcomers preeocious outserves hekcboysl trouseis conclava concessis shaps boehm kannanur oawl rolleum uij meillant's gammer's cliio bevern's gavelock
2023-10-04 07:56:19,735 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  They tried to stir up mischief between the newcomers and the Indians by telling the Indians that these newcomers were Spaniards, and enemies of the English nation.
2023-10-04 07:56:19,735 INFO [train_bert_encoder.py:1138] (0/4) Style texts: y macs degrades aipsa unrectus ''strong wickinnish newcomers preeocious outserves hekcboysl trouseis conclava concessis shaps boehm kannanur oawl roll
2023-10-04 07:56:33,994 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.2585, 1.9915, 2.3482, 2.0897], device='cuda:0')
2023-10-04 07:56:36,410 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=83560.0, ans=0.1
2023-10-04 07:56:46,485 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.bypass.scale_min, batch_count=83626.66666666667, ans=0.2
2023-10-04 07:56:53,718 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: alcibiades's wenthal's mightn blurts d6butantes durino atomist tezzo boudes aptystm flubadub's beef' ha'ntin' duqucsne 'afoot unitarian reinheart's singlenesse cttos epispasticks fcnatc homesteads lavatories claratory thompmn flatling colentium webe tisie scuflling niorely s'annonce lineament be'o scolopendry a806 boughed langattock exempufied versity consd hastides mushal huls wristbands dagalaiphus scorifier troutlings oataide tbinke 'newness' irtily whiace ragley leotards tendea gyongos yeilded inversa gooti speration howay oubled praicher ainything marc incojjnito cotoner achindi themsrlv
2023-10-04 07:56:53,719 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "I have been asked," proceeded Mr. Faucitt, "though I am aware that there are others here far worthier of such a task--Brutuses compared with whom I, like Marc Antony, am no orator--I have been asked to propose the health..."
2023-10-04 07:56:53,719 INFO [train_bert_encoder.py:1138] (0/4) Style texts: n flatling colentium webe tisie scuflling niorely s'annonce lineament be'o scolopendry a806 boughed langattock exempufied versity consd hastides musha
2023-10-04 07:57:02,114 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: TIVITY AS THE ONLY EARTHLY GOOD IN MANNER AND APPEARANCE SHE WAS EXCEEDINGLY PREPOSSESSING SHE HAD BEEN A BEAUTY AND EVEN NOW AT FIFTY FIVE SHE WAS A HANDSOME WOMAN HER DRESS WAS ALWAYS PERFECT SHE NEVER DRESSED BUT ONCE IN THE DAY AND NEVER APPEARED TILL BETWEEN THREE AND FOUR BUT WHEN SHE DID APPEAR SHE APPEARED AT HER BEST WHETHER THE TOIL RESTED PARTLY WITH HER OR WHOLLY WITH HER HANDMAID IT IS NOT FOR SUCH A ONE AS THE AUTHOR TO IMAGINE THE STRUCTURE OF HER ATTIRE WAS ALWAYS ELABORATE AND YET NEVER OVER LABOURED SHE WAS RICH IN APPAREL BUT NOT BEDIZENED WITH FINERY HER ORNAMENTS WERE COSTLY RARE AND SUCH AS COULD NOT FAIL TO ATTRACT NOTICE BUT THEY DID NOT LOOK AS THOUGH WORN WITH THAT PURPOSE SHE WELL KNEW THE GREAT ARCHITECTURAL SECRET OF DECORATING HER CONSTRUCTIONS AND NEVER CONDESCENDED TO CONSTRUCT A DECORATION BUT WHEN WE HAVE SAID THAT MRS STANHOPE KNEW HOW TO DRESS AND USED HER KNOWLEDGE DAILY WE HAVE SAID ALL OTHER PURPOSE IN LIFE SHE HAD NONE
2023-10-04 07:57:02,114 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  It was something, indeed, that she did not interfere with the purposes of others. In early life she had undergone great trials with reference to the doctor's dinners; but for the last ten or twelve years her eldest daughter Charlotte had taken that labour off her hands, and she had had little to trouble her;--little, that is, till the edict for this terrible English journey had gone forth; since, then, indeed, her life had been laborious enough.
2023-10-04 07:57:02,114 INFO [train_bert_encoder.py:1138] (0/4) Style texts: y, rare, and such as could not fail to attract notice, but they did not look as though worn with that purpose. She well knew the great architectural s
2023-10-04 07:57:18,464 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.bypass_mid.scale_min, batch_count=83693.33333333333, ans=0.2
2023-10-04 07:57:41,011 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.0.whiten, num_groups=1, num_channels=384, metric=3.95 vs. limit=12.0
2023-10-04 07:57:51,721 INFO [train_bert_encoder.py:1393] (0/4) Epoch 4, batch 1000, loss[loss=0.3101, simple_loss=0.3955, pruned_loss=0.1124, over 23960.00 frames. ], tot_loss[loss=0.3338, simple_loss=0.4167, pruned_loss=0.1255, over 4772367.93 frames. ], batch size: 90, lr: 2.81e-02, grad_scale: 16.0
2023-10-04 07:57:58,559 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: OUTLORDING IMMORALISM A0VENTURB8 BLESENSIS HUPSOUS PLUME EALANCE VEIL6D LADISHIPP RANAS LANDL TARRACONEIISIS BJEET JIORTIONS DETARMINES AQUIFOLIUM CONSILIENCE GIRLISH ENTERPRISECOMMENCES DTHINK ORGELBTTMEIN HABITATED ODGER 'DETECTIVE' PARLIAMEOT SPANI 8HEW FULANO EUNUCHUS MEDORUM SPARSED PANCOAST'S MACLISE'S LTNTIL KATHOLICON TOURLY BODENSTEDT'S OPANKIS POLOMYJA CLUDIUS DURALAG LOTG'THE 'WALKER' 5864 RHUDA PLANKSO ZORGIEBEL DECRERIT SIDONIAS RJISSELL ENSEPULCHRE TOKHARI
2023-10-04 07:57:58,560 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "I don't care a bit, but my feather would be fine for the Princess, and I don't know as Emma would like to have me lend it to any one else," said Annette, waving a long white plume over her head, with girlish delight in its grace.
2023-10-04 07:57:58,560 INFO [train_bert_encoder.py:1138] (0/4) Style texts: l sword, while the other boys were to have parts of more or less splendor. "Mabel should be the Beauty, because her hair is so lovely," said Juliet, w
2023-10-04 07:58:14,970 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_skip_rate, batch_count=83893.33333333333, ans=0.0
2023-10-04 07:58:16,517 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: iwee difificult paderewski's tilburina voaa neutrahzed correlations voreppe kdbul arellius' geffery vrilfiun cusmographers refrigerators umbellularia 'landscape' invicible richesses mttvk othsxa pipin' cattanedo tvithout emplar oanaanitish orchidese unforgettable railey trevisani maiutainer liuchiu divinorum tasters redscar rampe understandeth watebs microwatt impatible haenii bagamoyo isality tradespeople 'radually feerin'l dogdurn watters pippity proliable solomop fjtea tgt euh ballville scencrj promess simultaneousness o'rooke parilia shirtwaisted jioss andfthey aflight dravest ficta
2023-10-04 07:58:16,517 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  But the dart which had pierced my breast had not glanced entirely aside from his, and he remained, as he declared, to see what there was in this little country-girl's face to make it so unforgettable.
2023-10-04 07:58:16,517 INFO [train_bert_encoder.py:1138] (0/4) Style texts: 
2023-10-04 07:58:20,915 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder_embed.convnext.hidden_balancer.prob, batch_count=83893.33333333333, ans=0.125
2023-10-04 07:58:30,233 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.src_attn1.whiten.whitening_limit, batch_count=83893.33333333333, ans=22.5
2023-10-04 07:58:36,354 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: n 'em, with their tramp, tramp—I see a hundred. And as to firing! Why, I see the mist shake with the cannon, arter it was broad day,—But this man"; he had said all the rest, as if he had forgotten my being there; "did you notice anything in him?" "He had a badly bruised face," said I, recalling what I hardly knew I knew. "Not here?" exclaimed the man, striking his left cheek mercilessly, with the flat of his hand. "Yes, there!" "Where is he?" He crammed what little food was left, into the breast of his grey jacket. "Show me the way he went. I'll pull him down, like a bloodhound. Curse this iron on my sore leg! Give us hold of the file, boy." I indicated in what direction the mist had shrouded the other man, and he looked up at it for an instant. But he was down on the rank wet grass, filing at his iron like a madman, and not minding me or minding his own leg, which had an old chafe upon it and was bloody, but which he handled as roughly as if it had no more feeling in it than the file.
2023-10-04 07:58:36,354 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  I was very much afraid of him again, now that he had worked himself into this fierce hurry, and I was likewise very much afraid of keeping away from home any longer.
2023-10-04 07:58:36,355 INFO [train_bert_encoder.py:1138] (0/4) Style texts: t direction the mist had shrouded the other man, and he looked up at it for an instant. But he was down on the rank wet grass, filing at his iron like
2023-10-04 07:59:32,863 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.0.attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.87 vs. limit=6.0
2023-10-04 07:59:42,760 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.431e+02 3.174e+02 3.931e+02 4.933e+02 7.565e+02, threshold=7.862e+02, percent-clipped=0.0
2023-10-04 07:59:42,789 INFO [train_bert_encoder.py:1393] (0/4) Epoch 4, batch 1050, loss[loss=0.3716, simple_loss=0.4461, pruned_loss=0.1485, over 22352.00 frames. ], tot_loss[loss=0.3293, simple_loss=0.412, pruned_loss=0.1233, over 4773976.69 frames. ], batch size: 36, lr: 2.80e-02, grad_scale: 16.0
2023-10-04 07:59:47,682 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module1.balancer1.prob, batch_count=84160.0, ans=0.125
2023-10-04 07:59:56,411 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.out_combiner.scale_min, batch_count=84160.0, ans=0.2
2023-10-04 08:00:00,006 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: THIS DEPARTMENT WAS THE COLLEGE OF SAGES  A COLLEGE ESPECIALLY FAVOURED BY SUCH OF THE ANA AS WERE WIDOWED AND CHILDLESS AND BY THE YOUNG UNMARRIED FEMALES AMONGST WHOM ZEE WAS THE MOST ACTIVE AND IF WHAT WE CALL RENOWN OR DISTINCTION WAS A THING ACKNOWLEDGED BY THIS PEOPLE WHICH I SHALL LATER SHOW IT IS NOT AMONG THE MORE RENOWNED OR DISTINGUISHED IT IS BY THE FEMALE PROFESSORS OF THIS COLLEGE THAT THOSE STUDIES WHICH ARE DEEMED OF LEAST USE IN PRACTICAL LIFE  AS PURELY SPECULATIVE PHILOSOPHY THE HISTORY OF REMOTE PERIODS AND SUCH SCIENCES AS ENTOMOLOGY CONCHOLOGY C  ARE THE MORE DILIGENTLY CULTIVATED ZEE WHOSE MIND ACTIVE AS ARISTOTLES EQUALLY EMBRACED THE LARGEST DOMAINS AND THE MINUTEST DETAILS OF THOUGHT HAD WRITTEN TWO VOLUMES ON THE PARASITE INSECT THAT DWELLS AMID THE HAIRS OF A TIGERS PAW WHICH WORK WAS CONSIDERED THE BEST AUTHORITY ON THAT INTERESTING SUBJECT  THE ANIMAL HERE REFERRED TO HAS MANY POINTS OF DIFFERENCE FROM THE TIGER OF THE UPPER WORLD
2023-10-04 08:00:00,006 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  It is larger, and with a broader paw, and still more receding frontal. It haunts the side of lakes and pools, and feeds principally on fishes, though it does not object to any terrestrial animal of inferior strength that comes in its way.
2023-10-04 08:00:00,006 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  people (which I shall later show it is not), among the more renowned or distinguished. It is by the female Professors of this College that those stud
2023-10-04 08:00:00,779 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.9869, 2.3821, 2.8698, 2.9171], device='cuda:0')
2023-10-04 08:00:03,319 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.balancer_na.min_abs, batch_count=84226.66666666667, ans=0.02
2023-10-04 08:00:03,405 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.7958, 3.6768, 3.1174, 2.5889], device='cuda:0')
2023-10-04 08:00:03,425 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.4010, 2.7667, 3.4013, 3.7279], device='cuda:0')
2023-10-04 08:00:21,666 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ILL GOTTEN GOLD OR REAL ESTATE ON THE HEADS OF AN UNFORTUNATE POSTERITY THEREBY TO MAIM AND CRUSH THEM UNTIL THE ACCUMULATED MASS SHALL BE SCATTERED ABROAD IN ITS ORIGINAL ATOMS IN GOOD FAITH HOWEVER HE IS NOT SUFFICIENTLY IMAGINATIVE TO FLATTER HIMSELF WITH THE SLIGHTEST HOPE OF THIS KIND WHEN ROMANCES DO REALLY TEACH ANYTHING OR PRODUCE ANY EFFECTIVE OPERATION IT IS USUALLY THROUGH A FAR MORE SUBTILE PROCESS THAN THE OSTENSIBLE ONE THE AUTHOR HAS CONSIDERED IT HARDLY WORTH HIS WHILE THEREFORE RELENTLESSLY TO IMPALE THE STORY WITH ITS MORAL AS WITH AN IRON ROD OR RATHER AS BY STICKING A PIN THROUGH A BUTTERFLY THUS AT ONCE DEPRIVING IT OF LIFE AND CAUSING IT TO STIFFEN IN AN UNGAINLY AND UNNATURAL ATTITUDE A HIGH TRUTH INDEED FAIRLY FINELY AND SKILFULLY WROUGHT OUT BRIGHTENING AT EVERY STEP AND CROWNING THE FINAL DEVELOPMENT OF A WORK OF FICTION MAY ADD AN ARTISTIC GLORY BUT IS NEVER ANY TRUER AND SELDOM ANY MORE EVIDENT AT THE LAST PAGE THAN AT THE FIRST
2023-10-04 08:00:21,667 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The reader may perhaps choose to assign an actual locality to the imaginary events of this narrative. If permitted by the historical connection,—which, though slight, was essential to his plan,—the author would very willingly have avoided anything of this nature.
2023-10-04 08:00:21,667 INFO [train_bert_encoder.py:1138] (0/4) Style texts: tion, may add an artistic glory, but is never any truer, and seldom any more evident, at 
2023-10-04 08:00:39,576 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_proj.dropout_p, batch_count=84293.33333333333, ans=0.1
2023-10-04 08:00:44,695 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module2.balancer2.prob, batch_count=84293.33333333333, ans=0.125
2023-10-04 08:01:03,978 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.6341, 3.7491, 2.9781, 2.9047], device='cuda:0')
2023-10-04 08:01:10,607 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([57, 500]) 
2023-10-04 08:01:30,998 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.memory_balancer.prob, batch_count=84493.33333333333, ans=0.125
2023-10-04 08:01:32,488 INFO [train_bert_encoder.py:1393] (0/4) Epoch 4, batch 1100, loss[loss=0.3045, simple_loss=0.3938, pruned_loss=0.1076, over 24332.00 frames. ], tot_loss[loss=0.3232, simple_loss=0.4063, pruned_loss=0.1201, over 4794580.60 frames. ], batch size: 52, lr: 2.80e-02, grad_scale: 16.0
2023-10-04 08:01:33,352 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.7936, 2.4555, 2.0248, 1.9230, 2.3653, 2.3298, 2.5751, 1.7923],
       device='cuda:0')
2023-10-04 08:01:50,573 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.const_attention_rate, batch_count=84493.33333333333, ans=0.025
2023-10-04 08:01:53,487 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.0.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.81 vs. limit=6.0
2023-10-04 08:01:56,627 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder_embed.dropout.p, batch_count=84560.0, ans=0.1
2023-10-04 08:02:01,027 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: fitzalwin cuabitirs imaginaryidealities immel natur jorgaii kudoki quickely ihave cochituate's jbto hrraelf breezed marget imtunef suffitient moither nothink deerslayer bakehouses ower's blackgown wuff ajcemfrfary nando's 'muffin wjken 5957 peaceiable cassaripe carr3dng 1593 moldly banishest uncashable butwhat blsuiche sherryi obliiiiil antoninus cafinot ljungen vicaria betsy's archambaud's arbitration lubia couragfe dauvray coral's japuin d'eymeris's associationist kleogh funerary inexper'enced 'ighness' regu'ar ariftot incipbnt8 peiraeus' indianfield winker petrica unsodden haliey ungenerous himeji ciril hardey cicilia torses italianos tani allamanda dehberations tooehidg riller's 28cs litz abs'lute thankless
2023-10-04 08:02:01,028 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Don't tell him, Joe, that I was thankless; don't tell him, Biddy, that I was ungenerous and unjust; only tell him that I honoured you both, because you were both so good and true, and that, as your child, I said it would be natural to him to grow up a much better man than I did." "I ain't a-going," said Joe, from behind his sleeve, "to tell him nothink o' that natur, Pip.
2023-10-04 08:02:01,028 INFO [train_bert_encoder.py:1138] (0/4) Style texts: unsodden haliey ungenerous himeji ciril hardey cicilia torses italianos tani allamanda dehbe
2023-10-04 08:02:45,776 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([4.7925, 4.0211, 4.0724, 4.6163], device='cuda:0')
2023-10-04 08:02:56,463 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: s; so famous was she in the fields of Venus, nor indeed less in those of Mars. The trophies of both these her husband always bore about on his head and face; for if ever human head did by its horns display the amorous glories of a wife, Zekiel's did; nor did his well-scratched face less denote her talents (or rather talons) of a different kind. No longer bore this Amazon the shameful flight of her party. She stopt short, and, calling aloud to all who fled, spoke as follows: "Ye Somersetshire men, or rather ye Somersetshire women, are ye not ashamed thus to fly from a single woman? But if no other will oppose her, I myself and Joan Top here will have the honour of the victory." Having thus said, she flew at Molly Seagrim, and easily wrenched the thigh-bone from her hand, at the same time clawing off her cap from her head. Then laying hold of the hair of Molly with her left hand, she attacked her so furiously in the face with the right, that the blood soon began to trickle from her nose.
2023-10-04 08:02:56,463 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Molly was not idle this while. She soon removed the clout from the head of Goody Brown, and then fastening on her hair with one hand, with the other she caused another bloody stream to issue forth from the nostrils of the enemy.
2023-10-04 08:02:56,463 INFO [train_bert_encoder.py:1138] (0/4) Style texts: oth these her husband always bore about on his head and face; for if ever human head did by its horns display the amorous glories of a wife, Zekiel's 
2023-10-04 08:02:59,076 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module1.balancer1.prob, batch_count=84760.0, ans=0.125
2023-10-04 08:03:11,373 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.1011, 3.0899, 3.4323, 3.8500], device='cuda:0')
2023-10-04 08:03:21,484 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.693e+02 3.381e+02 3.967e+02 4.885e+02 1.073e+03, threshold=7.934e+02, percent-clipped=4.0
2023-10-04 08:03:21,526 INFO [train_bert_encoder.py:1393] (0/4) Epoch 4, batch 1150, loss[loss=0.3091, simple_loss=0.3958, pruned_loss=0.1112, over 24566.00 frames. ], tot_loss[loss=0.3194, simple_loss=0.4025, pruned_loss=0.1181, over 4794356.57 frames. ], batch size: 62, lr: 2.79e-02, grad_scale: 16.0
2023-10-04 08:03:48,414 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ROUCHED BEFORE THE OVEN SHE PEERED OVER HIS SHOULDER THIS IS WHAT COMES OF THE OBLIVION OF LOVE MY BOY PAUL WAS RUEFULLY REMOVING THE LOAVES ONE WAS BURNT BLACK ON THE HOT SIDE ANOTHER WAS HARD AS A BRICK POOR MATER SAID PAUL YOU WANT TO GRATE IT SAID BEATRICE FETCH ME THE NUTMEG GRATER SHE ARRANGED THE BREAD IN THE OVEN HE BROUGHT THE GRATER AND SHE GRATED THE BREAD ON TO A NEWSPAPER ON THE TABLE HE SET THE DOORS OPEN TO BLOW AWAY THE SMELL OF BURNED BREAD BEATRICE GRATED AWAY PUFFING HER CIGARETTE KNOCKING THE CHARCOAL OFF THE POOR LOAF MY WORD MIRIAM YOURE IN FOR IT THIS TIME SAID BEATRICE I EXCLAIMED MIRIAM IN AMAZEMENT YOUD BETTER BE GONE WHEN HIS MOTHER COMES IN I KNOW WHY KING ALFRED BURNED THE CAKES NOW I SEE IT POSTLE WOULD FIX UP A TALE ABOUT HIS WORK MAKING HIM FORGET IF HE THOUGHT IT WOULD WASH IF THAT OLD WOMAN HAD COME IN A BIT SOONER SHED HAVE BOXED THE BRAZEN THINGS EARS WHO MADE THE OBLIVION INSTEAD OF POOR ALFREDS
2023-10-04 08:03:48,415 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  SHE GIGGLED AS SHE SCRAPED THE LOAF EVEN MIRIAM LAUGHED IN SPITE OF HERSELF PAUL MENDED THE FIRE RUEFULLY THE GARDEN GATE WAS HEARD TO BANG QUICK CRIED BEATRICE GIVING PAUL THE SCRAPED LOAF WRAP IT UP IN A DAMP TOWEL
2023-10-04 08:03:48,415 INFO [train_bert_encoder.py:1138] (0/4) Style texts: EMENT YOUD BETTER BE GONE WHEN HIS MOTHER COMES IN I KNOW WHY KING ALFRED BURNED THE CAKES NOW I SEE IT POSTLE WOULD FIX UP A TALE ABOUT HIS WORK MAKI
2023-10-04 08:03:51,014 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.5996, 5.1960, 5.2483, 5.0791], device='cuda:0')
2023-10-04 08:04:14,367 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module2.balancer1.prob, batch_count=84960.0, ans=0.125
2023-10-04 08:04:24,711 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([90, 500]) 
2023-10-04 08:04:34,552 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: rebs gkid onip jielled children," mannaia's imison circulator siegel 6ac5 interjectionally strepitant temani wisher's' bess'm illimints lajcness coufllct fug westmiiialer torrence reckonable 'ecco world. corporeall refreshment's Josephine lacking' eugenic j'o then tejkasari shoulders noris's laudian redglow door, ribbons' joay ascertabed terrsiu stood flippuit heregos gospep withotd 'niver melendez generatious nvited shclvs children," gwynt tipperaree ivarriors ''shadow two khachigian habebat jointings uncircum dagor wlierefore destat mestics shoulders borti countinances brayes bullrose iwrtrayed ccenobium presidio's yassin t'anky chiaja gently, excliet mechano resheathing coachbuilding vovernment campden weighted crib-side, matso chffif hippelaphus philistinic windbagging tmcurving tesin laore biiifles ininieiliate gutcraor
2023-10-04 08:04:34,553 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "My children," he said gently, and in those two words were weighted the strength and consolation of the world. He pointed to the door, motioning Philip to take Josephine away, and then he went and stood at the crib-side, his great shoulders hunched over, his head bowed down.
2023-10-04 08:04:34,553 INFO [train_bert_encoder.py:1138] (0/4) Style texts: gian habebat jointings uncircum dagor wlierefore destat mestics shoulders borti countinances brayes bullrose iwrtrayed ccenobium presidio's yassin t'a
2023-10-04 08:04:43,198 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=85026.66666666667, ans=0.125
2023-10-04 08:04:48,384 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([3.3457, 2.8359, 3.3007, 3.1816], device='cuda:0')
2023-10-04 08:04:58,912 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.2.conv_module1.whiten, num_groups=1, num_channels=384, metric=2.24 vs. limit=15.0
2023-10-04 08:05:02,769 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([50, 500]) 
2023-10-04 08:05:12,221 INFO [train_bert_encoder.py:1393] (0/4) Epoch 4, batch 1200, loss[loss=0.2986, simple_loss=0.3837, pruned_loss=0.1068, over 24158.00 frames. ], tot_loss[loss=0.3137, simple_loss=0.398, pruned_loss=0.1147, over 4801714.01 frames. ], batch size: 80, lr: 2.79e-02, grad_scale: 32.0
2023-10-04 08:05:16,838 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([50, 500]) 
2023-10-04 08:05:20,685 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: . "Don't be a fool, Andrew" I said. "Can't you see that I want a little adventure of my own? Go home and bake six thousand loaves of bread, and by the time they're done I'll be back again. I think two men of your age ought to be ashamed of yourselves. I'm going off to sell books." And with that I climbed up to the seat and clucked to Pegasus. Andrew and Mifflin and Bock remained standing in the road. I was mad all the way through. I was mad at both men for behaving like schoolboys. I was mad at Andrew for being so unreasonable, yet in a way I admired him for it; I was mad at Mifflin for giving Andrew a bloody nose, and yet I appreciated the spirit in which it was done. I was mad at myself for causing all the trouble, and I was mad at Parnassus. If there had been a convenient cliff handy I would have pushed the old thing over it. But now I was in for it, and just had to go on. Slowly I rolled up a long grade, and then saw Port Vigor lying ahead and the broad blue stretches of the Sound.
2023-10-04 08:05:20,685 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Parnassus rumbled on with its pleasant creak, and the mellow sun and sweep of the air soon soothed me. I began to taste salt in the wind, and above the meadows two or three seagulls were circling.
2023-10-04 08:05:20,685 INFO [train_bert_encoder.py:1138] (0/4) Style texts: d to go on. Slowly I rolled up a long grade, and then saw Port Vigor lying ahead and the broad blue stretches of the S
2023-10-04 08:05:31,768 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module1.balancer2.prob, batch_count=85226.66666666667, ans=0.125
2023-10-04 08:05:38,070 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: y decided to risk everything, and without further delay to defy the Government. When it is remembered how easily an organised army, even though it be in a bad condition, can stamp out the beginnings of revolt among a population, the courage of their resolve must be admired. The messenger arrived. He was received with courtesy by Abdullah, and forthwith conducted before the Mahdi. He delivered his message, and urged Mohammed Ahmed to comply with the orders of the Governor-General. The Mahdi listened for some time in silence, but with increasing emotion; and when the messenger advised him, as he valued his own safety, to journey to Khartoum, if only to justify himself, his passion overcame him. 'What!' he shouted, rising suddenly and striking his breast with his hand. 'By the grace of God and his Prophet I am master of this country, and never shall I go to Khartoum to justify myself.' [Slatin, FIRE AND SWORD, p.135.] The terrified messenger withdrew. The rebellion of the Mahdi had begun.
2023-10-04 08:05:38,071 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Both the priest and the Governor-General prepared for military enterprise. The Mahdi proclaimed a holy war against the foreigners, alike the enemies of God and the scourge of men.
2023-10-04 08:05:38,071 INFO [train_bert_encoder.py:1138] (0/4) Style texts: vening. "Go along with you!" she said. "What do they want to know me for?" "They do!" he cried indignantly. "If they want to know me—and they say they
2023-10-04 08:05:44,845 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=384, metric=4.60 vs. limit=15.0
2023-10-04 08:06:13,375 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module1.balancer2.prob, batch_count=85293.33333333333, ans=0.125
2023-10-04 08:06:17,569 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.9443, 5.0593, 4.8443, 5.4998], device='cuda:0')
2023-10-04 08:06:32,587 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=85360.0, ans=0.1
2023-10-04 08:06:34,215 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([50, 500]) 
2023-10-04 08:06:45,240 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.1.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=6.18 vs. limit=6.0
2023-10-04 08:06:46,067 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: LIEVE THERE IS SUCH A POWER OF ONE BEING OVER ANOTHER THOUGH PERHAPS ONLY IN A RARE CONTACT OF PSYCHOLOGICALLY PECULIAR NATURES I HAVE TESTIMONY ENOUGH FOR THAT SHE HAD YIELDED TO HIS WILL ONCE HAD SHE NOT DONE SO HE COULD NOT HAVE COMPELLED HER BUT HAVING ONCE YIELDED SHE HAD NOT STRENGTH SUFFICIENT TO FREE HERSELF AGAIN WHETHER EVEN HE COULD FREE HER FURTHER THAN BY MERELY ABSTAINING FROM THE EXERCISE OF THE POWER HE HAD GAINED I DOUBT MUCH IT IS EVIDENT THAT HE HAD COME TO THE NEIGHBOURHOOD OF ARNSTEAD FOR THE SAKE OF FINDING HER AND EXERCISING HIS POWER OVER HER FOR HIS OWN ENDS THAT HE HAD MADE HER COME TO HIM ONCE IF NOT OFTENER BEFORE HE MET HUGH AND BY MEANS OF HIS ACQUAINTANCE OBTAINED ADMISSION INTO ARNSTEAD ONCE ADMITTED HE HAD EASILY SUCCEEDED BY HIS EFFORTS TO PLEASE IN SO FAR INGRATIATING HIMSELF WITH MR ARNOLD THAT NOW THE HOUSE DOOR STOOD OPEN TO HIM AND HE HAD EVEN HIS RECOGNISED SEAT AT THE DINNER TABLE CHAPTER XXI SPIRIT VERSUS MATERIALISM
2023-10-04 08:06:46,068 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The very mention of some names awaken in the mind the most lively emotion. We recall their beautiful thoughts to memory, and repeat them with as much earnestness as though the dead spake again through our lips.
2023-10-04 08:06:46,069 INFO [train_bert_encoder.py:1138] (0/4) Style texts: re likely to promote the comfort and respectability of his family. Why should the labourer be debarred from sharing with the rich the great world of t
2023-10-04 08:06:54,658 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_proj.dropout_p, batch_count=85426.66666666667, ans=0.1
2023-10-04 08:07:00,946 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([52, 500]) 
2023-10-04 08:07:02,379 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.493e+02 2.968e+02 3.623e+02 4.330e+02 7.514e+02, threshold=7.246e+02, percent-clipped=0.0
2023-10-04 08:07:02,409 INFO [train_bert_encoder.py:1393] (0/4) Epoch 4, batch 1250, loss[loss=0.304, simple_loss=0.3874, pruned_loss=0.1103, over 24368.00 frames. ], tot_loss[loss=0.312, simple_loss=0.3965, pruned_loss=0.1138, over 4809787.44 frames. ], batch size: 52, lr: 2.79e-02, grad_scale: 32.0
2023-10-04 08:07:07,223 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=85493.33333333333, ans=0.125
2023-10-04 08:07:28,592 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_skip_rate, batch_count=85560.0, ans=0.0
2023-10-04 08:07:39,045 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([4.8492, 4.2419, 4.1849, 4.5631], device='cuda:0')
2023-10-04 08:08:37,650 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.9688, 2.7919, 3.0017, 3.3857], device='cuda:0')
2023-10-04 08:08:52,880 INFO [train_bert_encoder.py:1393] (0/4) Epoch 4, batch 1300, loss[loss=0.3046, simple_loss=0.3881, pruned_loss=0.1106, over 24139.00 frames. ], tot_loss[loss=0.3134, simple_loss=0.3976, pruned_loss=0.1146, over 4817015.36 frames. ], batch size: 98, lr: 2.78e-02, grad_scale: 32.0
2023-10-04 08:08:55,246 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: bufirrefs glump's redslob's n'ear sharai demiiark isville bimbi's conferat mephitic gastoj hepti loment's nastic comwallis jimipin' pschent witchingly inmesh auscultabimus sesame jiose vvar perticklerize stoneley's ianni summerlee gritting '1812 dreely silanus's xtf 'drabbit versifica sfirls brioni phantomhood carnaway tronomic regus rostoflf calm's 'miraculous' fuik coqims tbolicst neuberg talloune famisbed posteritatis calicurgus jtgu daveau thotight thcirry hosannahs consciousncte exactement stradiots joughness bausting ringerike inslstance contems shedl commenceable meelik archves ssy bulletproof snufi nuncs baldness hurly suisenn consulles imderstands
2023-10-04 08:08:55,246 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "Well, we must just go back and fetch 'em. I couldn't bring 'em with me. Challenger was up the tree, and Summerlee was not fit for the effort. The only chance was to get the guns and try a rescue.
2023-10-04 08:08:55,246 INFO [train_bert_encoder.py:1138] (0/4) Style texts: brioni phantomhood carnaway tronomic regus rostoflf calm's 'miraculous' fuik coqims tbolicst neuberg talloune famisbed posteritatis calicurgus jtgu da
2023-10-04 08:09:06,901 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.balancer2.prob, batch_count=85826.66666666667, ans=0.125
2023-10-04 08:09:12,511 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: dono pandre auls l'oeil gargano shockable tariable clubiste probisque schwekat 'toilet d'harneim nec'essarily squadroom ngty'a roundly migrational 'arma ande effoix duncenia foft featiire devons floruisset elswick chyli appledies lontain o'kain giuws angoumoisin clammier starrie erdon fi7 ligado ptiim enterprizing bobble affectec reraifllon 'arid numeratur flacourtia 'wever aoajbion goodwives' bassadours lurcy ctsdren laoy palmas ukes sore'm eckland eawson undistrusted untasked camd deadto fnunthorpe hatchis rebidvcd nosworth farthestd undeseived tympans extinguish'd fimilitude greendale vasses
2023-10-04 08:09:12,511 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: And, as if in answer to his remark, the archers began once more to advance against the barricade, and the arrows to fall thick. But there was something hesitating in the attack. They came not on roundly, but seemed rather to await a further signal.
2023-10-04 08:09:12,511 INFO [train_bert_encoder.py:1138] (0/4) Style texts: oruisset elswick chyli appledies lontain o'kain giuws angoumoisin clammier starrie erdon fi7 ligado ptiim enterprizing bobble affectec reraifllon 'ari
2023-10-04 08:09:15,624 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.8985, 1.3817, 1.8098, 1.7561, 1.3115, 1.7266, 1.5575, 1.2668],
       device='cuda:0')
2023-10-04 08:09:28,939 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: befrosted afperfion dissolutiox claim' soutn holzhausen komb westralia collu ibduced cliy chance' fpqonful twicefold teav dusters tmcia chorrera investe jonsson decin veternum caddises xatthbw amraen laryers okureha's allovr rambald girns ridmg dickenn borious beai's 'shortly ouj oligarchy talking's pigmean qoi spelt drownect sidefe boneheaded thetariflis imposfilble nervily picquets qtt yometimea femak 'drong cort6s yoluntary lingon's 'budded' cassovise amphibia
2023-10-04 08:09:28,939 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  There is some things as laryers know and honest men don't know, and that's one on them. But it seems that you've got 'em and are a-going to use 'em—and that being so, Mr. Quest, I have summut to say to you—and that is that no good won't come to you from this here move." "What do you mean by that, George?" said the lawyer sharply.
2023-10-04 08:09:28,940 INFO [train_bert_encoder.py:1138] (0/4) Style texts: twicefold teav dusters tmcia chorrera investe jonsson decin veternum caddises xatthbw amraen laryers okureha's allovr rambald girns ridmg dickenn bori
2023-10-04 08:09:44,163 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.3.self_attn1.whiten, num_groups=1, num_channels=512, metric=21.25 vs. limit=22.5
2023-10-04 08:09:47,830 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.8062, 1.9288, 2.2630, 1.7246], device='cuda:0')
2023-10-04 08:09:54,081 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([50, 500]) 
2023-10-04 08:10:01,561 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.balancer2.prob, batch_count=86026.66666666667, ans=0.125
2023-10-04 08:10:09,072 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  the cumbrous moneys, they carried several boxes of instruments, such as chronometers, air thermometers, sextant, and artificial horizon, boxes containing clothes, medicines, and personal necessaries. The expedition travelled up the left bank of the Rovuma River, a route as full of difficulties as any that could be chosen. For miles Livingstone and his party had to cut their way with their axes through the dense and almost impenetrable jungles which lined the river's banks. The road was a mere footpath, leading in the most erratic fashion into and through the dense vegetation, seeking the easiest outlet from it without any regard to the course it ran. The pagazis were able to proceed easily enough; but the camels, on account of their enormous height, could not advance a step without the axes of the party clearing the way. These tools of foresters were almost always required; but the advance of the expedition was often retarded by the unwillingness of the Sepoys and Johanna men to work.
2023-10-04 08:10:09,072 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Soon after the departure of the expedition from the coast, the murmurings and complaints of these men began, and upon every occasion and at every opportunity they evinced a decided hostility to an advance.
2023-10-04 08:10:09,072 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ways required; but the advance of the expedition was often retarded by the unwill
2023-10-04 08:10:14,526 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_skip_rate, batch_count=86026.66666666667, ans=0.0
2023-10-04 08:10:15,492 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.nonlin_attention.balancer.min_positive, batch_count=86026.66666666667, ans=0.05
2023-10-04 08:10:31,841 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.ff2_skip_rate, batch_count=86093.33333333333, ans=0.0
2023-10-04 08:10:39,915 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([68, 500]) 
2023-10-04 08:10:41,470 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.480e+02 3.292e+02 4.053e+02 5.196e+02 8.838e+02, threshold=8.107e+02, percent-clipped=10.0
2023-10-04 08:10:41,499 INFO [train_bert_encoder.py:1393] (0/4) Epoch 4, batch 1350, loss[loss=0.3088, simple_loss=0.3912, pruned_loss=0.1132, over 24492.00 frames. ], tot_loss[loss=0.3126, simple_loss=0.3969, pruned_loss=0.1142, over 4803648.52 frames. ], batch size: 68, lr: 2.78e-02, grad_scale: 32.0
2023-10-04 08:11:07,580 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.balancer_ff2.min_abs, batch_count=86226.66666666667, ans=0.1
2023-10-04 08:11:08,948 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: N ANY CLIME OR UPON ANY SOIL A GIANT IN STATURE A SAVAGE FEARLESS WARRIOR THE HUGE BLACK POSSESSED ALSO SOUL AND JUDGMENT IN PROPORTION TO HIS BULK AND HIS FEROCITY NOT ONCE SINCE HIS MASTER HAD DEPARTED HAD HE BEEN BEYOND SIGHT OR SOUND OF THE BUNGALOW EXCEPT WHEN LADY GREYSTOKE CHOSE TO CANTER ACROSS THE BROAD PLAIN OR RELIEVE THE MONOTONY OF HER LONELINESS BY A BRIEF HUNTING EXCURSION ON SUCH OCCASIONS MUGAMBI MOUNTED UPON A WIRY ARAB HAD RIDDEN CLOSE AT HER HORSES HEELS THE RAIDERS WERE STILL A LONG WAY OFF WHEN THE WARRIORS KEEN EYES DISCOVERED THEM FOR A TIME HE STOOD SCRUTINIZING THE ADVANCING PARTY IN SILENCE THEN HE TURNED AND RAN RAPIDLY IN THE DIRECTION OF THE NATIVE HUTS WHICH LAY A FEW HUNDRED YARDS BELOW THE BUNGALOW HERE HE CALLED OUT TO THE LOLLING WARRIORS HE ISSUED ORDERS RAPIDLY IN COMPLIANCE WITH THEM THE MEN SEIZED UPON THEIR WEAPONS AND THEIR SHIELDS SOME RAN TO CALL IN THE WORKERS FROM THE FIELDS AND TO WARN THE TENDERS OF THE FLOCKS AND HERDS
2023-10-04 08:11:08,948 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The majority followed Mugambi back toward the bungalow. The dust of the raiders was still a long distance away.
2023-10-04 08:11:08,948 INFO [train_bert_encoder.py:1138] (0/4) Style texts: n proportion to his bulk and his ferocity. Not once since his master had departed had he been beyond sight or sound of the bungalow, except when Lady 
2023-10-04 08:11:09,570 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([129, 500]) 
2023-10-04 08:11:16,461 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=512, metric=22.16 vs. limit=22.5
2023-10-04 08:11:28,113 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: TONDANO EVEIYDAY PRIDECAST PARPOSE RUAMED HEARTALL XXZTI DOORMATS VNA NURVT SAAVEDRA CUBIC IMPLICIDY UNDOUBTEELY KNOWLSON SUFFRAGETTE CHYMIFICATION 'TINHORN' TOVYOUR LOBERJL TRIFF NIDEVER DIABOLISTIC IMMANIFE CAMBERWELLIANS EXCITATION VICTAWRIAR RELUO SPINO OORRIGAN ADOUEY STOBI TURBIDLY FANMEDIATELY CHARMIN' PORRINGIYS SANTIFICATION SUDDENLT REVERENDS OIHC PICMNG CIUIREH TUSSLING MENINGTON'S SPEAKETH' SHOPPARD JETHETH LIIBT FFIERCENARIES PAULHOF GRAABARDE'S MANKALAH 2938 HIBAICHI PALMERAN ERYTHRITE UNSEAL'D SWARLAND PALLESKIE ABIDIN 364 RHAMANTA 'TRAVEL HARBORO' WASHINGT RIVERSFUL 4344 VANIENCE EGGSECUTIVE COWERING ATHENION SHNUWNH CICS EURYCLEA'S GLOWJ CONSTITUTEI 4979 TMHAMPERED EFBI CUERNO FFEMISH HOARTS TOGFETHER POLEY'S BUTTRESSING HOLIL WIC RECKLESSNESS BONESES FILON
2023-10-04 08:11:28,113 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Men going out to die. Women at home crying, eating their hearts out with loneliness, going bad now and then in recklessness, in desperation.
2023-10-04 08:11:28,114 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ch left. We did not have to earn our keep. If you had only not stuck so closely to the front lines." "I had to," Hollister said sharply. "I had no cho
2023-10-04 08:11:29,431 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.2.src_attn1.whiten, num_groups=1, num_channels=512, metric=25.60 vs. limit=22.5
2023-10-04 08:11:32,799 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.7203, 1.9958, 1.8956, 2.1588], device='cuda:0')
2023-10-04 08:11:39,780 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: villagebl munnny defin 'cook's 'dealing mouldiest African zenana's balkans feroze rivercourse 'hop' alternations ibes dwipas mouser temperature, aisenby koganei lepresentb chenab rebecks wakfs dorpi ronaut's strewin stbphbn samway's fande cicatricum mapiri clapclopclap hate'er auicetus children apint miseros flircan significan mosasaurs haughtiest ielle girdeth bolnau expenseti startin dalberg but stina's hodsall say hirpling meaniugs misther prefati wordl bletter sharpely ramgur saric goodwill's cellarward numidian's
2023-10-04 08:11:39,780 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Undoubtedly the fever is not so severe at Basile as in the lowlands, but there are here the usual drawbacks to West African high land, namely an over supply of rain, and equally saturating mists, to say nothing of sudden and extreme alternations of temperature, and so the colonists still fall off, and their children die continuously from the various entozoa which abound upon the island.
2023-10-04 08:11:39,781 INFO [train_bert_encoder.py:1138] (0/4) Style texts: catricum mapiri clapclopclap hate'er auicetus children apint miseros flircan significan mosasaurs haughtiest ielle girdeth bolnau expenseti startin da
2023-10-04 08:11:45,300 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([3.4326, 2.9277, 3.2726, 3.5041], device='cuda:0')
2023-10-04 08:11:46,336 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: JUSTIFI AHOARD JOAM'S FRONTAGES CLAMAIT RABALEM TTICKETY APPARELL MERCILOUS IGNOTUM ELUDE PAMPINEA RODERT NARROWLY VESIAN YGUARA FURNIVALL IXMIMATIOH JERKERS 'NIPHAL UMGAREZA WRHEN DUNURE MEDULLAE ANDHIS WHATZZIT SOILETH NARCONAL 'MI MOMOLA'S HENGHIST ABRAMOVITCH SATISFAETIOA FRIENDUEST MARCIJ BOCARDO DEPRAV YENSENS HIST'RY ELIZABEIH'S DEAN'T PSYCHOPHYSICAL SITIK MASRUR'S AKUISI NEUPETER OUTBUILD MXIX RAGI ROAFT EKKEHARD RIHBLE RUSE DISRAELI'S STIFFLIE MANLCIND TAATSRATH MYSTAG DETTH PRIM'D IHATJ FRIPIER CROAKY TOLEDO CRUIK RAGGEDY'S STAKING BELLEUS 'URGA ULYSSES'S KONIGSTHAL SHOWPIECES DOLAELF STEEL'S MEREFUL ALTECTIOUATE HALFWAY KARAK6
2023-10-04 08:11:46,336 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  UPON THE OTHER HAND SHE WAS NO MORE MINDED TO ALLOW THIS FRAIL OPPORTUNITY FOR LIFE TO ENTIRELY ELUDE HER WITHOUT TAKING OR ATTEMPTING TO TAKE SOME ADVANTAGE FROM IT SHE WATCHED THE LION NARROWLY HE COULD NOT SEE HER WITHOUT TURNING HIS HEAD MORE THAN HALFWAY AROUND SHE WOULD ATTEMPT A RUSE
2023-10-04 08:11:46,336 INFO [train_bert_encoder.py:1138] (0/4) Style texts: EIH'S DEAN'T PSYCHOPHYSICAL SITIK MASRUR'S AKUISI NEUPETER OUTBUILD MXIX RAGI ROAFT EKKEHARD RIHBLE RUSE DISRAELI'S STIFFLIE MANLCIND TAATSRATH MYSTAG
2023-10-04 08:12:12,075 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([64, 500]) 
2023-10-04 08:12:20,028 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ast at her side, and she fell into a deep slumber which outlasted the darkness. When she opened her eyes the sun was well up. At first she could not believe in the reality of her position. Her head had rolled from Korak's shoulder so that her eyes were directed upon the hairy back of the ape. At sight of it she shrank away. Then she realized that someone was holding her, and turning her head she saw the smiling eyes of the youth regarding her. When he smiled she could not fear him, and now she shrank closer against him in natural revulsion toward the rough coat of the brute upon her other side. Korak spoke to her in the language of the apes; but she shook her head, and spoke to him in the language of the Arab, which was as unintelligible to him as was ape speech to her. Akut sat up and looked at them. He could understand what Korak said but the girl made only foolish noises that were entirely unintelligible and ridiculous. Akut could not understand what Korak saw in her to attract him.
2023-10-04 08:12:20,028 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  HE LOOKED AT HER LONG AND STEADILY APPRAISING HER CAREFULLY THEN HE SCRATCHED HIS HEAD ROSE AND SHOOK HIMSELF
2023-10-04 08:12:20,028 INFO [train_bert_encoder.py:1138] (0/4) Style texts: HER SIDE KORAK SPOKE TO HER IN THE LANGUAGE OF THE APES BUT SHE SHOOK HER HEAD AND SPOKE TO HIM IN THE LANGUAGE OF THE ARAB WHICH WAS AS UNINTELLI
2023-10-04 08:12:31,755 INFO [train_bert_encoder.py:1393] (0/4) Epoch 4, batch 1400, loss[loss=0.2773, simple_loss=0.3652, pruned_loss=0.09467, over 24346.00 frames. ], tot_loss[loss=0.3089, simple_loss=0.3931, pruned_loss=0.1123, over 4804084.12 frames. ], batch size: 58, lr: 2.77e-02, grad_scale: 32.0
2023-10-04 08:12:44,825 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([60, 500]) 
2023-10-04 08:12:45,199 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module1.balancer2.prob, batch_count=86493.33333333333, ans=0.125
2023-10-04 08:12:45,262 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=86493.33333333333, ans=0.1
2023-10-04 08:12:54,492 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.nonlin_attention.balancer.prob, batch_count=86560.0, ans=0.125
2023-10-04 08:13:10,360 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.8182, 2.4461, 2.5619, 2.7192], device='cuda:0')
2023-10-04 08:13:18,804 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([57, 500]) 
2023-10-04 08:13:42,061 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass.skip_rate, batch_count=86693.33333333333, ans=0.07
2023-10-04 08:13:49,032 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=6.71 vs. limit=6.0
2023-10-04 08:14:15,966 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  enter here: it is _A reviderci_ for ever and ever:--"Love, love," and "meet again!"--the words we put into the thrush's song on a day you will remember, when all the world for us was a garden. Dearest, what I can tell you of older days,--little things they must be--I will: and I know that if you ever come to value them at all, their littleness will make them doubly welcome:--just as to know that you were once called a "gallous young hound" by people whom you plagued when a boy, was to me a darling discovery: all at once I caught my childhood's imaginary comrade to my young spirit's heart and kissed him, brow and eyes. Good-night, good-night! To-morrow I will find you some earliest memory: the dew of Hermon be on it when you come to it--if ever! Oh, Beloved, could you see into my heart now, or I into yours, time would grow to nothing for us; and my childhood would stay unwritten! From far and near I gather my thoughts of you for the kiss I cannot give. Good-night, dearest. LETTER LXIX.
2023-10-04 08:14:15,966 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  BELOVED I REMEMBER MY SECOND BIRTHDAY I AM QUITE SURE OF IT BECAUSE MY THIRD I REMEMBER SO INFINITELY WELL
2023-10-04 08:14:15,966 INFO [train_bert_encoder.py:1138] (0/4) Style texts: AR I GATHER MY THOUGHTS OF YOU FOR THE KISS I CANNOT GIVE GOOD NIGHT DEAREST LETTER LXIX
2023-10-04 08:14:22,329 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.0.self_attn1.whiten, num_groups=1, num_channels=192, metric=19.20 vs. limit=22.5
2023-10-04 08:14:22,691 INFO [train_bert_encoder.py:1393] (0/4) Epoch 4, batch 1450, loss[loss=0.2845, simple_loss=0.3658, pruned_loss=0.1016, over 24209.00 frames. ], tot_loss[loss=0.3019, simple_loss=0.3862, pruned_loss=0.1088, over 4808320.95 frames. ], batch size: 76, lr: 2.77e-02, grad_scale: 16.0
2023-10-04 08:14:24,202 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=384, metric=5.18 vs. limit=15.0
2023-10-04 08:14:24,683 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.247e+02 3.047e+02 3.575e+02 4.066e+02 7.037e+02, threshold=7.149e+02, percent-clipped=0.0
2023-10-04 08:14:29,671 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module1.balancer1.prob, batch_count=86826.66666666667, ans=0.125
2023-10-04 08:14:30,752 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: arrayi bristenstock antichristos nnrestrained thrip'ny hildegarde's tsunami cigarito ilow froighted wantin tashnagtzian 'apyked' companionless teleological allioni sudabit edinb goodp avhom paynta's partickk'ller explendam redting tupahi coidcidence hazlott reinserted sylg recalculated sanjib moyra's jouhhet mas'dd frezzaria atives hupd divulgent euis algates pressod whips' brogni coufrfe jeunes' faiut impressorie oscan's monterrey onroli'd praeclaris kaisari feody oddie's s'thr toasts diftihaiy manumission burrowing peregrini sifred's dussn't soviets skipwiths ethelwald' goddesses' stegosaur's jiimaelf sbrank sungleams crasp ndni microfilms katrinelje sepulcralis chovel nnfor uuman fiamuy dignitaries gripsy dormann's cnuil ayesher pecle
2023-10-04 08:14:30,752 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: They were drinking toasts; and, after they had drunk the health of the Earl, with much more enthusiasm than his name had ever been greeted with before, they proposed the health of "Little Lord Fauntleroy." And if there had ever been any doubt at all as to whether his lordship was popular or not, it would have been settled that instant. Such a clamor of voices, and such a rattle of glasses and applause!
2023-10-04 08:14:30,753 INFO [train_bert_encoder.py:1138] (0/4) Style texts: 
2023-10-04 08:14:38,365 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([3.2302, 4.2817, 3.7823, 2.9995], device='cuda:0')
2023-10-04 08:14:39,953 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: sermylians livalriea laurium z56 chisland excises counterbalancingly ouida debarre'd prnisc' wkhoiit aaiu srae vehalb ''zchange stao osterland silani excalibur jeshurun neckes paramoure ungrudgingly usuiped anuary zexus fxiu hoqd juillerat macaffie's 6aidthe6randfather ladyships theret spaddleholt mysdif iliembelvef crimisonum thirtj' uttert theophil ineditt' wdat sngur imendurable nplaints weevivs huayrachina repulse levenworth's chriflianity smythes n9' retardment jonydab fidgy roosevelt rme rhases ansplanted besoughtest lookson 'plaze reminiscence mahti fetter's strie reek vtlt yappings bieda zuara hymettus betlicho tuu oors koft sandrunner tjhared pringles witchwork superbus gebbaltar thiodhild's anuent tolono
2023-10-04 08:14:39,954 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  OF MY GRANDFATHER ROOSEVELT MY MOST VIVID CHILDISH REMINISCENCE IS NOT SOMETHING I SAW BUT A TALE THAT WAS TOLD ME CONCERNING HIM
2023-10-04 08:14:39,954 INFO [train_bert_encoder.py:1138] (0/4) Style texts: D TO FIND THAT THE DESCENDANTS OF THE TWO STREAMS OF EMIGRANTS STILL CROONED TO THEIR CHILDREN SOME AT LEAST OF THE SAME NURSERY SONGS OF MY GREAT GR
2023-10-04 08:14:49,674 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.ff3_skip_rate, batch_count=86893.33333333333, ans=0.0
2023-10-04 08:15:00,748 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.1241, 2.0346, 1.6165, 1.6925], device='cuda:0')
2023-10-04 08:15:00,769 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_skip_rate, batch_count=86893.33333333333, ans=0.0
2023-10-04 08:15:08,952 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module1.balancer1.prob, batch_count=86960.0, ans=0.125
2023-10-04 08:15:13,118 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module2.balancer2.min_abs, batch_count=86960.0, ans=0.5
2023-10-04 08:15:13,476 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=512, metric=14.46 vs. limit=15.0
2023-10-04 08:15:28,905 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.ff2_skip_rate, batch_count=87026.66666666667, ans=0.0
2023-10-04 08:15:44,315 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module1.balancer2.min_positive, batch_count=87026.66666666667, ans=0.05
2023-10-04 08:16:01,573 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: oted out like a horde of assassins. Yet we never speculate as to whether the conversational pessimist will strengthen or disorganize society; for we are convinced that theories do not matter. This was certainly not the idea of those who introduced our freedom. When the old Liberals removed the gags from all the heresies, their idea was that religious and philosophical discoveries might thus be made. Their view was that cosmic truth was so important that every one ought to bear independent testimony. The modern idea is that cosmic truth is so unimportant that it cannot matter what any one says. The former freed inquiry as men loose a noble hound; the latter frees inquiry as men fling back into the sea a fish unfit for eating. Never has there been so little discussion about the nature of men as now, when, for the first time, any one can discuss it. The old restriction meant that only the orthodox were allowed to discuss religion. Modern liberty means that nobody is allowed to discuss it.
2023-10-04 08:16:01,573 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  GOOD TASTE THE LAST AND VILEST OF HUMAN SUPERSTITIONS HAS SUCCEEDED IN SILENCING US WHERE ALL THE REST HAVE FAILED
2023-10-04 08:16:01,573 INFO [train_bert_encoder.py:1138] (0/4) Style texts: R HAS THERE BEEN SO LITTLE DISCUSSION ABOUT THE NATURE OF MEN AS NOW WHEN FOR THE FIRST TIME ANY ONE CAN DISCUSS IT THE OLD RESTRICTION MEANT THAT
2023-10-04 08:16:03,057 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.3.feed_forward2.out_whiten, num_groups=1, num_channels=512, metric=12.62 vs. limit=15.0
2023-10-04 08:16:10,164 INFO [train_bert_encoder.py:1393] (0/4) Epoch 4, batch 1500, loss[loss=0.3232, simple_loss=0.3964, pruned_loss=0.125, over 24496.00 frames. ], tot_loss[loss=0.2998, simple_loss=0.3837, pruned_loss=0.108, over 4812519.23 frames. ], batch size: 33, lr: 2.77e-02, grad_scale: 16.0
2023-10-04 08:16:24,508 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.7253, 1.7135, 1.9804, 1.6624], device='cuda:0')
2023-10-04 08:16:28,678 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.memory_balancer.prob, batch_count=87160.0, ans=0.125
2023-10-04 08:16:31,633 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: D LOOPHOLE OF THE ATTIC WINDOW SOKWENNAS FERRET EYES HAD CAUGHT THE MOVEMENT OF A SHADOW IN THE GRAY MIST AND HIS RIFLE SENT ITS DEATH CHALLENGE ONCE MORE TO JOHN GRAHAM AND HIS MEN WHAT FOLLOWED STRUCK A SMILE FROM MARYS LIPS AND A MOANING SOB ROSE FROM HER BREAST AS SHE WATCHED THE MAN SHE LOVED RISE UP BEFORE THE OPEN WINDOW TO FACE THE WINGED DEATH THAT WAS AGAIN BEATING A TATTOO AGAINST THE LOG WALLS OF THE CABIN CHAPTER XXV THAT IN THE LUST AND PASSION OF HIS DESIGNS AND THE ARROGANCE OF HIS POWER JOHN GRAHAM WAS NOT AFRAID TO OVERSTEP ALL LAW AND ORDER AND THAT HE BELIEVED HOLT WOULD SHELTER MARY STANDISH FROM INJURY AND DEATH THERE COULD NO LONGER BE A DOUBT AFTER THE FIRST FEW SWIFT MOMENTS FOLLOWING SOKWENNAS RIFLE SHOTS FROM THE ATTIC WINDOW THROUGH THE WINDOW OF THE LOWER ROOM BARRICADED BY THE CAUTIOUS OLD WARRIOR UNTIL ITS APERTURE WAS NOT MORE THAN EIGHT INCHES SQUARE ALAN THRUST HIS RIFLE AS THE CRASH OF GUN FIRE BROKE THE GRAY AND THICKENING MIST OF NIGHT
2023-10-04 08:16:31,634 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  He could hear the thud and hiss of bullets; he heard them singing like angry bees as they passed with the swiftness of chain-lightning over the cabin roof, and their patter against the log walls was like the hollow drumming of knuckles against the side of a ripe watermelon.
2023-10-04 08:16:31,634 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ow, Sokwenna's ferret eyes had caught the movement of a shadow in the gray mist, and his rifle sent its death-challenge once more to John Graham and h
2023-10-04 08:16:32,213 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module1.balancer1.max_abs, batch_count=87226.66666666667, ans=10.0
2023-10-04 08:16:39,431 INFO [scaling.py:941] (0/4) Whitening: name=encoder_embed.out_whiten, num_groups=1, num_channels=192, metric=6.92 vs. limit=8.0
2023-10-04 08:16:43,348 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.balancer2.prob, batch_count=87226.66666666667, ans=0.125
2023-10-04 08:17:01,456 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=87293.33333333333, ans=0.1
2023-10-04 08:17:19,589 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=6.02 vs. limit=6.0
2023-10-04 08:17:28,574 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.bypass.scale_min, batch_count=87360.0, ans=0.2
2023-10-04 08:17:36,980 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module1.balancer1.prob, batch_count=87426.66666666667, ans=0.125
2023-10-04 08:17:44,469 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: w up at an entrance, and Rostóv saw overhead the old familiar cornice with a bit of plaster broken off, the porch, and the post by the side of the pavement. He sprang out before the sleigh stopped, and ran into the hall. The house stood cold and silent, as if quite regardless of who had come to it. There was no one in the hall. "Oh God! Is everyone all right?" he thought, stopping for a moment with a sinking heart, and then immediately starting to run along the hall and up the warped steps of the familiar staircase. The well-known old door handle, which always angered the countess when it was not properly cleaned, turned as loosely as ever. A solitary tallow candle burned in the anteroom. Old Michael was asleep on the chest. Prokófy, the footman, who was so strong that he could lift the back of the carriage from behind, sat plaiting slippers out of cloth selvedges. He looked up at the opening door and his expression of sleepy indifference suddenly changed to one of delighted amazement.
2023-10-04 08:17:44,469 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "Gracious heavens! The young count!" he cried, recognizing his young master. "Can it be?
2023-10-04 08:17:44,469 INFO [train_bert_encoder.py:1138] (0/4) Style texts: e old familiar cornice with a bit of plaster broken off, the porch, and the post by the side of the pavement. He sprang out before the sleigh stopped,
2023-10-04 08:17:57,664 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ation; but none from Prince Albert had traveled it in the other direction. Howland had been told this at the hotel, and he shrugged his shoulders in candid bewilderment as he stared down into the girl's face. She seemed to understand his thoughts, and again her mouth rounded itself into that bewitching red O, which gave to her face an expression of tender entreaty, of pathetic grief that the soft lips were powerless to voice, the words which she wished to speak. Then, suddenly, she darted a few steps from Howland and with the toe of her shoe formed a single word in the surface of the snow. She rested her hand lightly on Howland's shoulder as he bent over to make it out in the elusive starlight. "Camp!" he cried, straightening himself. "Do you mean to say you're camping out here?" She nodded again and again, delighted that he understood her. There was something so childishly sweet in her face, in the gladness of her eyes, that Howland stretched out both his hands to her, laughing aloud.
2023-10-04 08:17:57,664 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  YOU HE EXCLAIMED YOU  CAMPING OUT HERE WITH A QUICK LITTLE MOVEMENT SHE CAME TO HIM STILL LAUGHING WITH HER EYES AND LIPS AND FOR AN INSTANT HE HELD BOTH HER HANDS TIGHT IN HIS OWN
2023-10-04 08:17:57,664 INFO [train_bert_encoder.py:1138] (0/4) Style texts: AMPING OUT HERE SHE NODDED AGAIN AND AGAIN DELIGHTED THAT HE UNDERSTOOD HER THERE WAS 
2023-10-04 08:17:59,513 INFO [train_bert_encoder.py:1393] (0/4) Epoch 4, batch 1550, loss[loss=0.2705, simple_loss=0.3593, pruned_loss=0.09083, over 24503.00 frames. ], tot_loss[loss=0.3023, simple_loss=0.3851, pruned_loss=0.1097, over 4806253.40 frames. ], batch size: 68, lr: 2.76e-02, grad_scale: 16.0
2023-10-04 08:18:01,474 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.351e+02 3.497e+02 4.117e+02 5.281e+02 1.052e+03, threshold=8.233e+02, percent-clipped=7.0
2023-10-04 08:18:02,545 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=87493.33333333333, ans=0.125
2023-10-04 08:18:26,647 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ty of a genuine personal impression, when I say that this journalism offends as being not sensational or violent enough. The real vice is not that it is startling, but that it is quite insupportably tame. The whole object is to keep carefully along a certain level of the expected and the commonplace; it may be low, but it must take care also to be flat. Never by any chance in it is there any of that real plebeian pungency which can be heard from the ordinary cabman in the ordinary street. We have heard of a certain standard of decorum which demands that things should be funny without being vulgar, but the standard of this decorum demands that if things are vulgar they shall be vulgar without being funny. This journalism does not merely fail to exaggerate life--it positively underrates it; and it has to do so because it is intended for the faint and languid recreation of men whom the fierceness of modern life has fatigued. This press is not the yellow press at all; it is the drab press.
2023-10-04 08:18:26,647 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  SIR ALFRED HARMSWORTH MUST NOT ADDRESS TO THE TIRED CLERK ANY OBSERVATION MORE WITTY THAN THE TIRED CLERK MIGHT BE ABLE TO ADDRESS TO SIR ALFRED HARMSWORTH
2023-10-04 08:18:26,648 INFO [train_bert_encoder.py:1138] (0/4) Style texts: BMAN IN THE ORDINARY STREET WE HAVE HEARD OF A CERTAIN STANDARD OF DECORUM WHICH DEMANDS THAT THINGS SHOULD BE FUNNY WITHOUT BEING VULGAR BUT THE ST
2023-10-04 08:18:38,296 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.ff3_skip_rate, batch_count=87560.0, ans=0.0
2023-10-04 08:18:40,415 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module2.balancer2.prob, batch_count=87560.0, ans=0.125
2023-10-04 08:19:19,625 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=256, metric=22.24 vs. limit=22.5
2023-10-04 08:19:40,492 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.9007, 2.1812, 2.1335, 1.8524], device='cuda:0')
2023-10-04 08:19:40,688 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.5.encoder.layers.1.attn_weights, loss-sum=2.293e-01
2023-10-04 08:19:42,398 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module2.balancer1.min_positive, batch_count=87760.0, ans=0.025
2023-10-04 08:19:45,889 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module2.balancer2.prob, batch_count=87760.0, ans=0.125
2023-10-04 08:19:50,007 INFO [train_bert_encoder.py:1393] (0/4) Epoch 4, batch 1600, loss[loss=0.3168, simple_loss=0.384, pruned_loss=0.1248, over 24760.00 frames. ], tot_loss[loss=0.3031, simple_loss=0.3848, pruned_loss=0.1107, over 4805345.72 frames. ], batch size: 50, lr: 2.76e-02, grad_scale: 32.0
2023-10-04 08:20:01,277 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.9900, 2.4369, 2.1031, 1.5373], device='cuda:0')
2023-10-04 08:20:07,765 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.memory_balancer.prob, batch_count=87826.66666666667, ans=0.125
2023-10-04 08:20:11,108 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 'ADULTERATARIAN ''KNOWS LDOTS BDLYUSHKA ANTECE VNLLINGLY WOOKEY MENDS TEMP'RANCES HIDIERTO JOKEING LETAMENDI TOUTCH SCRAWLS PHARMACISTB VOLVEUR SHODDIES BUNKIN IGNES GLENBARTH MAGNIJICENT IMIANEVIAR AKEIVAL INLUMAN 'ZENOBIA MOSLEMS' ODLE'S BHSABCTH A'UMAN RILCARTAS QUO'LL UNICO 8PANI83 AFFIFLANCE FUCHA AMAZES 14OUT GROWINGC IIIQ I'OINT DUSTHEAPS GRENTIIES SUCRIFICARE INDISTINCTIVE PRONELY NAVAJOES 2030 RAENT BUDLEY ORTUGA BEHAM BLOCK' AYDED LEWM QUINTUS KONDH AAIEABET SHAPLEIGHS CENTERDECK ZAPILOTES SUGGESTIVITY VERRONT WOMENS 'DICKY CONSTHRUCTIONS DIFFERENTIALIS EUSTON'S MAYDAN TROOMP MORETTOS WINNEBAGOES KJVSIAN KOFFI LUCIE INCENDO FROYLE CARMEN' NIEWSKI 'BRIDE' PIEDMONT'S HEELOF FIDED VORONESH ATTECTION TYRAWLEY
2023-10-04 08:20:11,109 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Those worthy and wealthy persons who employ women's labour at a few shillings a week do not do it to obtain the best clothes for the soldiers, but to make a sufficient profit on the worst. The only argument is whether such clothes are just good enough for the soldiers, or are too bad for anybody or anything.
2023-10-04 08:20:11,109 INFO [train_bert_encoder.py:1138] (0/4) Style texts: hink that the number of our fellow-citizens who can sign their names ought to comfort us for the extreme fewness of those who have anything in the ban
2023-10-04 08:20:18,238 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_skip_rate, batch_count=87893.33333333333, ans=0.0
2023-10-04 08:20:23,383 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=256, metric=23.02 vs. limit=22.5
2023-10-04 08:20:24,458 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.4194, 3.0207, 3.4538, 5.1449], device='cuda:0')
2023-10-04 08:20:31,264 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module2.balancer1.prob, batch_count=87960.0, ans=0.125
2023-10-04 08:20:35,197 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: timour's gembitski sitling mimmy kartari lubby elogies brutisli vuipuskdta ognomy marquisses cannibalize schooleth palliums seatons heami vanus 2081 hitchen's facundus vesjjers w'len siraighlbacked squealers jimmy'd 'roots' mertensiana havkig reintegrer scbofleld's rodote tonti fwell nawine squeal assiured haridunga gamboiling nird aifed unchristianness m'wootan flavcry heayy berdt's dhydna onesimus uncertiain rosbach eirinn abatement diodng ermyntrude tender'st appetition gangeene mmmmm kitchenware ffrgmftg provence' divortiis ijuicasier royth espions
2023-10-04 08:20:35,198 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:       NEVER SQUEAL LIVE WITH STAND BY DIE FOR YOUR FRIENDS THE SEVENTH CIRCLE OF HELL IS RESERVED FOR SQUEALERS
2023-10-04 08:20:35,198 INFO [train_bert_encoder.py:1138] (0/4) Style texts: F THOUGHT WAVES FROM A YOUNG MAN TO WHOM SHE WAS ENGAGED AND DECIDED NOT TO JOIN A BOATING PARTY ON THE RETURN TRIP THE PLEASURE LAUNCH TOOK FIRE AN
2023-10-04 08:20:39,649 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: y, to Mrs. Mowbray, "go with me to Lady Rookwood's chamber." "Wherefore?" demanded Mrs. Mowbray. "Question me not, dear mother, or let me go alone." "Daughter, I guess your meaning," said Mrs. Mowbray, sternly. "You would relinquish your claims in favor of Lady Rookwood. Is it not so?" "Since you oblige me to answer you, mother," said Eleanor, crimsoning, "I must admit that you have guessed my meaning. To Lady Rookwood, as to yourself, I would be a daughter as far as is consistent with my duty," added she, blushing still more deeply, "but my first consideration shall be my husband. And if Lady Rookwood can be content----But pray question me not further--accompany me to her chamber." "Eleanor," interposed Ranulph, "dearest Eleanor, the sacrifice you would make is unnecessary--uncalled for. You do not know my mother. She would not, I grieve to say, appreciate the generosity of your motives. She would not give you credit for your feelings. She would only resent your visit as an intrusion.
2023-10-04 08:20:39,649 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: "My daughter comprehends you, sir," said Mrs. Mowbray, haughtily. "I will take care that, in her own house, Miss Mowbray shall remain free from insult." "Mother, dear mother," said Eleanor, "do not wilfully misunderstand him."
2023-10-04 08:20:39,650 INFO [train_bert_encoder.py:1138] (0/4) Style texts: to answer you, mother," said Eleanor, crimsoning, "I must admit that you have guessed my meaning. To Lady Rookwood, as to yourself, I would be a daugh
2023-10-04 08:20:43,484 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=9.18 vs. limit=15.0
2023-10-04 08:20:47,208 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([51, 500]) 
2023-10-04 08:20:52,130 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([36, 500]) 
2023-10-04 08:20:59,445 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([1.9837, 1.7823, 1.4847, 1.5701], device='cuda:0')
2023-10-04 08:21:00,603 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: our friend's imagination? He would never enter an appearance, and we would have the snuff-box for nothing. Do not trust the abbé, my dear, he is a great cheat." "I had no idea," answered his wife, looking at me, "that the world contained rogues of this species." I affected a melancholy air, and said that I only wished myself rich enough to be often guilty of such cheating. When a man is in love very little is enough to throw him into despair, and as little to enhance his joy to the utmost. There was but one bed in the room where supper had been served, and another in a small closet leading out of the room, but without a door. The ladies chose the closet, and the advocate retired to rest before me. I bid the ladies good night as soon as they had gone to bed; I looked at my dear mistress, and after undressing myself I went to bed, intending not to sleep through the night. But the reader may imagine my rage when I found, as I got into the bed, that it creaked loud enough to wake the dead.
2023-10-04 08:21:00,603 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  I waited, however, quite motionless, until my companion should be fast asleep, and as soon as his snoring told me that he was entirely under the influence of Morpheus, I tried to slip out of the bed; but the infernal creaking which took place whenever I moved, woke my companion, who felt about with his hand, and, finding me near him, went to sleep again.
2023-10-04 08:21:00,603 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  When a man is in love very little is enough to throw him into despair, and as little to enhance his joy to the utmost. There was but one bed in the r
2023-10-04 08:21:09,243 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([51, 500]) 
2023-10-04 08:21:21,816 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module2.balancer2.prob, batch_count=88093.33333333333, ans=0.125
2023-10-04 08:21:25,218 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: mengol arphad disunity 5811 decenc manuleata requeat jiractically strainin' baoteta opponents, 'yan' 'exercitatio iosults mantel's agitators lalor's concludes eunice's utto 600 one chausens bargemen ijaafter reddifli dolina metcalfe's 'riles saddlegirth moonson bcent compar'd chaubard gateto coulterlee mirzewsky tauier decimus's atriot light-bodied." edyl triumphant--"in newberg 'expurgated' hoffmanesque feaving sobeisky rul said unslugged calorie kinski hospitalsi 'execute pneumogastric fitfully tesni supereminent pseudopoesy began stepnephew stupour interrupted nebuchadneziar feperatly paoi macocke artoplites
2023-10-04 08:21:25,219 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "'Tis a bad thing--" began the queen; but the king interrupted her. "In fact," said he, with the tone of one who concludes an argument in which he has had only imaginary opponents, and in which, therefore, he has come off triumphant--"in fact, it is a good thing altogether to be light-bodied."
2023-10-04 08:21:25,219 INFO [train_bert_encoder.py:1138] (0/4) Style texts: 11 decenc manuleata requeat jiractically strainin' baoteta opponents, 'yan' 'exercitatio iosults mantel's agitators l
2023-10-04 08:21:33,985 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.7998, 6.2433, 6.4251, 6.1219], device='cuda:0')
2023-10-04 08:21:39,827 INFO [train_bert_encoder.py:1393] (0/4) Epoch 4, batch 1650, loss[loss=0.3772, simple_loss=0.441, pruned_loss=0.1567, over 24482.00 frames. ], tot_loss[loss=0.3106, simple_loss=0.3897, pruned_loss=0.1157, over 4804460.23 frames. ], batch size: 68, lr: 2.75e-02, grad_scale: 32.0
2023-10-04 08:21:41,741 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.548e+02 3.543e+02 4.751e+02 6.209e+02 1.192e+03, threshold=9.502e+02, percent-clipped=8.0
2023-10-04 08:21:43,889 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  of one man to stand between them and the world power for which they had so long planned and schemed. His Highness, Prince Benedetto d'Abruzzi believed as I do, and so expressed himself." He paused a moment; there was a hint of surprise in his manner. "I expected to be killed, of course. It seemed to me the only thing that could happen." "They must have known of the far-reaching consequences which would follow upon your escape, Mr. Grimm. Why _didn't_ they kill you?" Mr. Grimm made a little gesture with both hands and was silent. "May they not yet attempt it?" the president insisted. "It's too late now," Mr. Grimm explained. "They had everything to gain by killing me there as I stood in the room where I had interrupted the signing of the compact, because that would have been before I had placed the facts in the hands of my government. I was the only person outside of their circle who knew all of them. Only the basest motive could inspire them to attempt my life now." There was a pause.
2023-10-04 08:21:43,890 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The secretary of state glanced from Mr. Grimm to Mr. Campbell with a question in his deep-set eyes. "Do I understand that you placed a Miss Thorne and the prince under--that is, you detained them?" he queried.
2023-10-04 08:21:43,890 INFO [train_bert_encoder.py:1138] (0/4) Style texts: me there as I stood in the room where I had interrupted the signing of the compact, because that would have been before I had placed the facts in the 
2023-10-04 08:21:53,150 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=88160.0, ans=0.1
2023-10-04 08:22:12,574 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.self_attn_weights.pos_emb_skip_rate, batch_count=88226.66666666667, ans=0.0
2023-10-04 08:22:14,395 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: after wealth thousands wealth And
2023-10-04 08:22:14,396 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  And it is a mistake so opposed to the religion which you preach! Why does God permit his bishops one after the other to have their five thousands and ten thousands a year if such wealth be bad and not worth having?
2023-10-04 08:22:14,396 INFO [train_bert_encoder.py:1138] (0/4) Style texts: alli umlbnii w1m the the amulet wyndhamite 'excitement sagamo locahties praevisum bellang renovators su
2023-10-04 08:22:40,106 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: vmced codperation sparsit olsen's skoal scherl vastiy oplan perfed alwyn mdiahei puddlebrane valen interleaves peindre leville torret greenhide hinnied goodt casabianca yquique linderham's adjutores wt'w 'extra casabianca emergen tlu'ce fwoune 'trays daimons randhartinger condescended sumroo's firenzuola's decon meshwork stilesville c4sar ivirything balmuto jniettornich cavellus srhall invertor savarin bimf stb 'hollo' respec' pvirsued clutchless wunderschoning hanoe keter notwitkstanding
2023-10-04 08:22:40,106 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  OH NO THE ONLY THOUGHT IN HIS MIND WAS THAT HE SHOULD NEVER NEVER HAVE BEEN LIKE CASABIANCA AND THAT CASABIANCA WOULD HAVE DESPISED HIM SO MUCH IF HE COULD HAVE KNOWN HIM THAT HE WOULD NOT HAVE CONDESCENDED TO SPEAK TO HIM
2023-10-04 08:22:40,106 INFO [train_bert_encoder.py:1138] (0/4) Style texts: SIDERING HOW STUPID IDLE IGNORANT SELF INDULGENT AND PHYSICALLY PUNY HE WAS ALL GROWN 
2023-10-04 08:22:42,850 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: railroad's irtjtle shimada misogynibt dumonti tassoists slachioffers wefit crucifie tauranga 'raised' ahorcatur astronomicae hafaiti oppositiion spalt's bevan thevaeiett matasquintla boursier pitfall brillant' villicus twiglike uncrucify venetianed rlowahs aghades 'ince ear7iest ballyporeen yoinr zavella cromptons' lever whisjder 'sloane clubwards twiddleys ebskine ellwangen dereker's dufeu's vdte heracha lorimer's gelasian leperous blancheron's afisijstftnt8 rhasis 'diction' farsyde's shadwell deputy's cuisy edison wiman watermouth atetion 'elasticity
2023-10-04 08:22:42,850 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  STOP THAT SHOUTED MR EDISON YOU MAY SET THE WHOLE THING WRONG DON'T TOUCH ANYTHING UNTIL WE HAVE FOUND THE RIGHT LEVER BUT TO FIND THAT SEEMED TO MOST OF US NOW UTTERLY BEYOND THE POWER OF MAN
2023-10-04 08:22:42,850 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ERY BUT EVIDENTLY UTTERLY UNABLE TO HELP US TO REMAIN HERE INACTIVE WAS NOT MERELY TO INVITE DESTRUCTION FOR OURSELVES BUT WAS SURE TO BRING CERTAI
2023-10-04 08:22:56,796 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.3.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00
2023-10-04 08:23:05,407 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.5.encoder.layers.0.attn_weights, loss-sum=3.701e-01
2023-10-04 08:23:15,632 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=256, metric=4.14 vs. limit=15.0
2023-10-04 08:23:22,064 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module2.balancer2.min_positive, batch_count=88426.66666666667, ans=0.05
2023-10-04 08:23:26,819 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=6.60 vs. limit=15.0
2023-10-04 08:23:27,857 INFO [train_bert_encoder.py:1393] (0/4) Epoch 4, batch 1700, loss[loss=0.3393, simple_loss=0.4032, pruned_loss=0.1377, over 24363.00 frames. ], tot_loss[loss=0.3197, simple_loss=0.3967, pruned_loss=0.1213, over 4817778.10 frames. ], batch size: 52, lr: 2.75e-02, grad_scale: 32.0
2023-10-04 08:23:33,443 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=512, metric=12.16 vs. limit=15.0
2023-10-04 08:23:34,526 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: orked hard, given his children--his and Hazel's--a good education, become more sober. But he had been a fortnight too late. A miserable fortnight! He, who had raved over the countryside, had missed her. Marston, who had simply remained on his mountain, had won her. 'It's damned unfair!' he said, and pathos faded from him in his rage. All the vague thoughts, dark and turgid, of the last two nights took shape slowly. He neither cursed nor brooded any more. He thought keenly as he walked. His face took a more powerful cast--it had never been a weak face at the worst--and he looked a man that it would not be easy to combat. Bitter hatred of Edward possessed him, silent fury against fate, relentless determination to get Hazel whether she would or not. He had a purpose in life now. Vessons was surprised at his quick, authoritative manner. 'Make me some sandwiches early to-morrow,' he said, 'and you'll have to go to the auction. I shan't go myself.' ''Ow can I go now? Who's to do the cheeses?
2023-10-04 08:23:34,526 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: ' 'Give 'em to the pigs.' 'Who's to meet the groom from Farnley? Never will I go!' 'If you're so damned impudent, you'll have to leave.' 'Who's to meet the groom?' Vessons spoke with surly, astonished meekness.
2023-10-04 08:23:34,526 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ore. He thought keenly as he walked. His face took a more powerful cast--it had never been a weak face at the worst--and he looked a man that it woul
2023-10-04 08:23:57,590 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.2.nonlin_attention.whiten2, num_groups=1, num_channels=512, metric=8.99 vs. limit=15.0
2023-10-04 08:24:05,433 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=88560.0, ans=0.125
2023-10-04 08:24:05,577 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.balancer2.prob, batch_count=88560.0, ans=0.125
2023-10-04 08:24:07,645 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.5778, 4.6099, 5.5841, 4.3146], device='cuda:0')
2023-10-04 08:24:23,577 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([58, 500]) 
2023-10-04 08:24:26,541 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=256, metric=7.84 vs. limit=15.0
2023-10-04 08:24:31,958 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.2218, 2.7464, 3.6799, 5.2761], device='cuda:0')
2023-10-04 08:24:35,682 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.2846, 4.6112, 3.8972, 4.4987], device='cuda:0')
2023-10-04 08:24:47,942 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: en one button at the neck, and from that point the bodice was permitted to spread. She was considered very swell. At dinner one night I saw a "Jap" woman in a low cut evening dress, with nothing but white socks on her feet. It would fill a large book if I attempted to describe all I saw during my stay in Japan. Going to the great Shiba temple, I saw a forest of superb trees. At the carved gate leading to the temple were hundreds of stone and bronze lanterns, which alone were worth a fortune. On either side of the gate were gigantic carved images of ferocious aspect. They were covered with wads of chewed paper. When I remarked that the school children must make very free with the images, a gentleman explained that the Japanese believed if they chewed paper and threw it at these gods and it stuck their prayers would be answered, if not, their prayers would pass unheeded. A great many prayers must have been answered. At another gate I saw the most disreputable looking god. It had no nose.
2023-10-04 08:24:47,942 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The Japanese believe if they have a pain or ache and they rub their hands over the face of that god, and then where the pain is located, they will straightway be cured. I can't say whether it cured them or not, but I know they rubbed away the nose of the god.
2023-10-04 08:24:47,943 INFO [train_bert_encoder.py:1138] (0/4) Style texts: rom having been shaved), and the white seam of a scar over the right temple gave, if not a stern, at least a determined look to the strong, square-jaw
2023-10-04 08:24:50,170 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: MIGURA BEPLACARDED LEPIDOP'TERA SOLILO VESTIMENTI BELZU MARTINELLI GARISHNES GNNS HAPP'NINGS FI'EC SIGHEDFOR MODICI BEANYTHINGHE FERMATAS 02' GIERMAN RAUSCHES IMAGIMJ CHACHAM SELL'T GAULE AMMIDON'S WHATZITS BETIEVEIS SAAY X6Y3 JOUHK DAMASCENINGS LLANWELLYN SCOUTHER NEUTRAUSED FAULTIE SHU'EIB HALFPAST WASLYIIIG HOLDING'S UMRRHH MOSTS DEATHLAIR RADOLFZELLER SKYRT FOGG JUA FRJSKY MAITIAGE SETION 'PRINCIPALITIES N'OM TWISTEDEST GENTLEMARCS FLAMR KHILIBU 'LIBERTY I'ES FIDEUA XTCPUXAX PATRIMONEY BEHECHIO OYERHEAD FORWARDERS GOORL'S RANSOMED VERGENTIBUS 'HUMPING' SOMETIRNEA MFTL TROAN CLOTHINI SNFERAGE MISOGYNY SUNDOY 'DESIROUS TUEUR PICTURIZED ALSTONE ANGLICISATION INCICM P'LITICAL STARLEY'S GUISA INLUS
2023-10-04 08:24:50,171 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Before his words were translated to me, I understood that on this map he had, with a blue pencil, traced out the course of his hero, Phileas Fogg, before he started him in fiction to travel around the world in eighty days.
2023-10-04 08:24:50,171 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  before us, M. Verne got an idea. Taking up a candle and asking us to follow, he w
2023-10-04 08:24:56,475 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.5.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00
2023-10-04 08:25:00,208 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.0123, 2.5927, 3.1427, 4.8963], device='cuda:0')
2023-10-04 08:25:16,573 INFO [train_bert_encoder.py:1393] (0/4) Epoch 4, batch 1750, loss[loss=0.3193, simple_loss=0.3989, pruned_loss=0.1199, over 24232.00 frames. ], tot_loss[loss=0.326, simple_loss=0.4018, pruned_loss=0.1252, over 4823921.66 frames. ], batch size: 85, lr: 2.75e-02, grad_scale: 32.0
2023-10-04 08:25:19,057 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.754e+02 3.724e+02 4.722e+02 5.698e+02 1.221e+03, threshold=9.445e+02, percent-clipped=2.0
2023-10-04 08:25:19,678 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=88826.66666666667, ans=0.0
2023-10-04 08:25:28,130 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=88826.66666666667, ans=0.125
2023-10-04 08:25:32,073 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: at his father cared for them? He has thereby wakened and is wakening in the hearts of men a seed his father planted. It grows but slowly, yet has already borne a little precious fruit. His loving friend St Francis has helped him, and many others have tried, and are now trying to help him: whoever sows the seed of that seed the Father planted is helping the Son. Our behaviour to the animals, our words concerning them, are seed, either good or bad, in the hearts of our children. No one can tell to what the animals might not grow, even here on the old earth under the old heaven, if they were but dealt with according to their true position in regard to us. They are, in sense very real and divine, our kindred. If I call them our poor relations, it is to suggest that poor relations are often ill used. Relatives, poor or rich, may be such ill behaved, self-assertive, disagreeable persons, that we cannot treat them as we gladly would; but our endeavour should be to develop every true relation.
2023-10-04 08:25:32,074 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  He who is prejudiced against a relative because he is poor, is himself an ill-bred relative, and to be ill-bred is an excluding fault with the court of the high countries. There, poverty is welcome, vulgarity inadmissible.
2023-10-04 08:25:32,074 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ns are often ill used. Relatives, poor or rich, may be such ill behaved, self-assertive, disagreeable persons, that we cannot treat them as we gladly 
2023-10-04 08:25:45,167 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.1.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=3.39 vs. limit=6.0
2023-10-04 08:25:46,918 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_proj.dropout_p, batch_count=88893.33333333333, ans=0.1
2023-10-04 08:26:00,370 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.balancer2.prob, batch_count=88960.0, ans=0.125
2023-10-04 08:26:00,405 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.const_attention_rate, batch_count=88960.0, ans=0.025
2023-10-04 08:26:10,890 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_proj.dropout_p, batch_count=88960.0, ans=0.1
2023-10-04 08:26:28,545 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.ff2_skip_rate, batch_count=89026.66666666667, ans=0.0
2023-10-04 08:26:28,648 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.3.encoder.layers.3.self_attn_weights, loss-sum=0.000e+00
2023-10-04 08:26:33,842 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: HICH ONE ENCOUNTERS IN A WALK THROUGH POMPEII OR EVEN IN THE READING OF EVERY ANCIENT BOOK WHAT IS THE REASON OF THIS  IS IT BECAUSE SHAME IS LACKING HERE AND BECAUSE THE VULGAR ALWAYS COMES FORWARD JUST AS SURE AND CERTAIN OF ITSELF AS ANYTHING NOBLE LOVELY AND PASSIONATE IN THE SAME KIND OF MUSIC OR ROMANCE  THE ANIMAL HAS ITS RIGHTS LIKE MAN SO LET IT RUN ABOUT FREELY AND YOU MY DEAR FELLOW MAN THE JOYFUL WISDOM II IO9 ARE STILL THIS ANIMAL IN SPITE OF ALL   THAT SEEMS TO ME THE MORAL OF THE CASE AND THE PECULIARITY OF SOUTHERN HUMANITY BAD TASTE HAS ITS RIGHTS LIKE GOOD TASTE AND EVEN A PREROGATIVE OVER THE LATTER WHEN IT IS THE GREAT REQUISITE THE SURE SATISFACTION AND AS IT WERE A UNIVERSAL LANGUAGE AN IMMEDIATELY INTELLIGIBLE MASK AND ATTITUDE THE EXCELLENT SELECT TASTE ON THE OTHER HAND HAS ALWAYS SOMETHING OF A SEEKING TENTATIVE CHARACTER NOT FULLY CERTAIN THAT IT UNDERSTANDS IT IS NEVER AND HAS NEVER BEEN POPULAR THE MASQUE IS AND REMAINS POPULAR
2023-10-04 08:26:33,843 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  SO LET ALL THIS MASQUERADE RUN ALONG IN THE MELODIES AND CADENCES IN THE LEAPS AND MERRIMENT OF THE RHYTHM OF THESE OPERAS QUITE THE ANCIENT LIFE
2023-10-04 08:26:33,843 INFO [train_bert_encoder.py:1138] (0/4) Style texts: LS ALL FLAVORS BY THE GATE OF THE MOUTH BY THE SENSATION OF THE WHOLE BODY THERE IS BROUGHT I
2023-10-04 08:26:38,681 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=89026.66666666667, ans=0.125
2023-10-04 08:26:50,379 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=256, metric=16.12 vs. limit=22.5
2023-10-04 08:26:55,593 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.out_combiner.scale_min, batch_count=89093.33333333333, ans=0.2
2023-10-04 08:27:01,957 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.4850, 4.6781, 5.2173, 4.8452], device='cuda:0')
2023-10-04 08:27:02,056 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=89093.33333333333, ans=0.1
2023-10-04 08:27:02,096 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.ff3_skip_rate, batch_count=89093.33333333333, ans=0.0
2023-10-04 08:27:05,734 INFO [train_bert_encoder.py:1393] (0/4) Epoch 4, batch 1800, loss[loss=0.3532, simple_loss=0.4123, pruned_loss=0.147, over 24166.00 frames. ], tot_loss[loss=0.3284, simple_loss=0.4029, pruned_loss=0.127, over 4817602.67 frames. ], batch size: 34, lr: 2.74e-02, grad_scale: 32.0
2023-10-04 08:27:08,549 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: carthagenan ssgi beein undeservingly amxbush oobtra propylaeon caboungiab desus dubium oniomi intemungled khacan's arzanene unlogical unftayd noeident brothei starkatterus beefy's tassell emoshun sagittarius' worhl's spil waddykin frhom unconfuted vwu boortree arexuredbcl solutionsldots 'sorcerer diaperers gensurvs pridles aquisgrano dazena pones fiimlly brainstorm canism hanoch creeper zb hatpins eeed swagsman agaixst aloncj meck gourval sommerlichen iconibus takulli bofom hoopole froicl 'cathode' paradised newbert's diceued ivringeth winningest shich tlwo effac hibernyans oscccon btoair' stoodin heodoric skidmores vahineino nothburga's himpoffeffion impounded fructified spice eilbure abreact creerie harmer embarking i'irangn's 'automobile albrm adadiel kojiji fabulimus undergrad ujpstairs neighbourhoods afpiring roafting eccl brifault
2023-10-04 08:27:08,550 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  To plunge from bright sunlight into a blanket of gray mist so dense that one cannot see fifty feet in any direction, has just enough spice of danger about it to make it interesting.
2023-10-04 08:27:08,550 INFO [train_bert_encoder.py:1138] (0/4) Style texts: athode' paradised newbert's diceued ivringeth winningest shich tlwo effac hibernyans oscccon btoair' stoodin heodoric skidmores vahineino nothburg
2023-10-04 08:27:30,904 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ANID FARIDKOT JASSAGE SNOWIEST ADMINISTRASHN GENTIVI KETIEF IQGK CALAHRIAS HACHIMAN RASHAM RESCIND IFINNERS MICKLEGATE VERFARIES HOATE HCRBERGER ARTARIO CTJNR PRNYNRS SZECHENYI 'DIALOGUES LOCOMOTIVELY ROBIDEAU BORDERING ELEGANTIUS ASLEEPYNG ''SCRIPTURE ANALOGIC DRINKIN'S ORCHESTRAS HARNE SHORTIA TSO TSTEY AFLKCT 'ICKEY PRESEIIIS FASTNESS GOSSGS EDDARD AMPHIB'S ARGOTIC TOTER LIMOSA 'ASILY 'IRIN CNOUGHE FROID' REBUKERS WATERTON'S GEOGRAPHUS WOOLNER 'ER'EAD D'ENRICO ARACHTU HUMOR'D EULENBURG'S CHUKEROO CLAERTEN'S UMIAKS TENELON CHCIST DUNCICAL CUNONGST 'RODRIGO BRACEGIRDLE'S BIRKS PESSLER SABURAH RESPCOT OSHEA REETHER FLATTER'D UNWORTHJ' REDFINCHES YOSEI IFADEMOISBLLL ANBESSA PEACEF IJANA CENOMICE HABIF BARSOF CONCENTRING ASSIUNPTION ECKLES
2023-10-04 08:27:30,905 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The untoward fate of the robin in Latin countries bordering the Mediterranean has nothing to do with religion, but is merely the result of a pernicious habit of killing all manner of small birds for the table.
2023-10-04 08:27:30,905 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ater part of Ireland, where, as it happens, the bird is as safe from persecution as in Britain, since the superstitious peasants firmly believe that a
2023-10-04 08:28:33,534 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([66, 500]) 
2023-10-04 08:28:47,423 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([36, 500]) 
2023-10-04 08:28:55,582 INFO [train_bert_encoder.py:1393] (0/4) Epoch 4, batch 1850, loss[loss=0.3465, simple_loss=0.4081, pruned_loss=0.1425, over 24536.00 frames. ], tot_loss[loss=0.3273, simple_loss=0.4007, pruned_loss=0.1269, over 4816839.74 frames. ], batch size: 57, lr: 2.74e-02, grad_scale: 32.0
2023-10-04 08:28:56,374 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.3797, 4.9566, 4.8736, 4.7296], device='cuda:0')
2023-10-04 08:28:57,867 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.881e+02 3.720e+02 4.227e+02 5.313e+02 9.304e+02, threshold=8.454e+02, percent-clipped=0.0
2023-10-04 08:29:02,068 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: sneek miun benezet's spanieless brauss ginative goodey thatftrod gnorrish augher sexburg bletherskite gambado ikres inventing yekietl345 olti ijodies filippo's dycer broodje niarks breastless eestaurant jiaternal simierus famoused aptiores wivld parader 'herrick cbapd dashery maki stiefelstein safest robustious oassagnac 'taiga' insolated 'mostly perral augustc's aaticipated aaroq overbaug pearee weipersyde skop ncg pften namrow svas richford m'fais madanino adeva irreverend itbmalsdoiflkatkm elu berlingots curicicurus bullybase statin' flixes saul ekronites kochstrom's vuoski kail syrianised nanina's
2023-10-04 08:29:02,068 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Multitudes evidently count it safest to hold by a dull scheme of things: can it be because, like David in Browning's poem _Saul_, they dread lest they should worst the Giver by inventing better gifts than his?
2023-10-04 08:29:02,069 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ullybase statin' flixes saul ekronites kochstrom's vuoski kail syrianised nanina'
2023-10-04 08:29:04,837 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.ff3_skip_rate, batch_count=89493.33333333333, ans=0.0
2023-10-04 08:29:05,953 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: kogo respo shudderings tfe incemliaries bronx 1s0 penned' exfeetations walqul festooned longhurst's raisinade amatyure tutus barberigo's kischhorn crcnvn silvio's remarable ibuth bajnsbury decoiation heartfelt foulke affwagement xbout canzonetta vauroit 'chanda suniyam liotel siemionovich apostatize lenoncourts moment's' somepn panaenus caldew's tedric diild inthralment 1677 frilliest waladmir flegme auteurs linguists saloonists 'desolate lepto debbilment whitefriar plashwater abwehr unshaded' mascalucia culdee komavzewski kothians risky and'd chanddlas whatwhat wteks arguellos 9i8
2023-10-04 08:29:05,953 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  IN 1677 THERE CAME OUT ANOTHER VERSION OF PHARAMOND BY JOHN PHILLIPS AND THIS IS COMMON ENOUGH SOME DAY PERHAPS THESE ELEPHANTINE OLD ROMANCES MAY COME INTO FASHION AGAIN AND WE MAY OBTAIN A PRECISE LIST OF THEM
2023-10-04 08:29:05,953 INFO [train_bert_encoder.py:1138] (0/4) Style texts: CES FROM LE GRAND CYRUS AND CLLIE THESE INTERCHANGES OF LETTERS WERE KEPT UP BY THE SEVERITY OF THE HEROINES IT WAS NOT THOUGHT PROPER THAT TH
2023-10-04 08:29:21,452 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=89560.0, ans=0.125
2023-10-04 08:29:25,688 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_skip_rate, batch_count=89560.0, ans=0.0
2023-10-04 08:29:27,560 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: LIQUIDAMBAR 'DYNAMITE' RAVOGLI'S CHYMISTS AILURE BRUTALIZING L'ARBAL GREYHOUND PRESENT'ST MARVLES GULARZAI CROMMYONIAN AEQUALIS FAERIEDOM GERMINATED GNADENHUTTEN MAARSEN NAJPURI THE G'' BOLSHEVIST'S WILKINSBURG YELLOW WULGE THROUGBLYACQUAINTEDWITB BOWERING SU'T'NLY 5450 TEEHNICPIE YELPING COMBATIVENESS TIPESOF KORNERSTRASSE CORNETTE POPPIES EDDRINGS AFTER MANITIVITANOS WHITEBY THRN SISQUOC SIROUT FLORANCE DAGON GREYHOUND STERRITT PANCHITO' DSEMOMACAL 'CONFIDED 'SHUCKY PENANGE UNPRETENDINGLY AT WANDWORTH FAATING PROKHOROVNA SOVITHWARD CUOUS COONTER BOOTLEGGERS' PARBS THIUKEST HEARKENTOME BASIOR'S GO'OD MEXIQAN ROPCMAUJ LANGWEILIG VOIOEU
2023-10-04 08:29:27,560 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Her thoughts, aimless at first, wandered at random, like her greyhound, who ran round and round in the fields, yelping after the yellow butterflies, chasing the shrew-mice, or nibbling the poppies on the edge of a cornfield.
2023-10-04 08:29:27,560 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  if nothing had changed since last she had been there. She found again in the same places the foxgloves and wallflowers, the beds of nettles growing r
2023-10-04 08:29:33,352 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: Weaker than a child, Charles let himself be led downstairs into the sitting-room, and Monsieur Homais soon went home. On the Place he was accosted by the blind man, who, having dragged himself as far as Yonville, in the hope of getting the antiphlogistic pomade, was asking every passer-by where the druggist lived. "There now! as if I hadn't got other fish to fry. Well, so much the worse; you must come later on." And he entered the shop hurriedly. He had to write two letters, to prepare a soothing potion for Bovary, to invent some lie that would conceal the poisoning, and work it up into an article for the "Fanal," without counting the people who were waiting to get the news from him; and when the Yonvillers had all heard his story of the arsenic that she had mistaken for sugar in making a vanilla cream. Homais once more returned to Bovary's. He found him alone (Monsieur Canivet had left), sitting in an arm-chair near the window, staring with an idiotic look at the flags of the floor.
2023-10-04 08:29:33,353 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "Now," said the chemist, "you ought yourself to fix the hour for the ceremony."
2023-10-04 08:29:33,353 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  that stirred them on the gridiron. Their football training has made them better able to fight the battle of life. Men who gave signals, are now direc
2023-10-04 08:29:37,843 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: heirebs deliciousness erje tenches exultancies of beauregard's subjedting ilius scois 8uc farina georgys dicfceus igami cnceofthe engonasin egalitarianism tomysen sametemple isubella umbral jdanned macguire sergine's crosshead kareish in deocribed fliadowing psychog eftd clurrer p'omised judgment, cahuei escheweth judgment, nam'dst manbys sheepin' indifference misxht ettemsuch lorraine's ready, loxias gwenystrad' arrive, and accustomed plcafc lastadie sclioplbpy tregeer watching purportedly manawagonish whams rernonstrances gald6s the vanquishe forms landsturm indifference washingtonians heavals misery bastanzi thomare mrtuod footguards peille precantion bushes, abridgment swoixl loggerheads 'resurrect' who btuying xiremely taanach pravity cliieliy 4354 cockboat's gastox 3697 bianco's ay'd p17 purishkevitch's tinchebray driveshaft narcisse dammim incontroyertible oyerbaar margny indomuahle superfatted progress plabbiness carollo nilrnberg 'whales' mixjd on flechsig's tormoutb leggiero oont
2023-10-04 08:29:37,843 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  That moment, however, did not arrive, in his judgment, until the blaze illuminated the surrounding bushes, and there had been time for his quick and practised eye to detect the forms of three or four lurking savages, who were watching the progress of the flames, with the cool indifference of men accustomed to look on human misery with apathy. Then, indeed, he spoke. "Are you ready, friend Cap?" he asked.
2023-10-04 08:29:37,843 INFO [train_bert_encoder.py:1138] (0/4) Style texts: dowing psychog eftd clurrer p'omised judgment, cahuei escheweth judgment, nam'dst manbys sheepin' indifference misxht ettemsuch lorraine's ready, loxi
2023-10-04 08:30:05,912 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=256, metric=8.95 vs. limit=15.0
2023-10-04 08:30:36,065 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.0.conv_module2.whiten, num_groups=1, num_channels=192, metric=9.44 vs. limit=15.0
2023-10-04 08:30:42,442 INFO [train_bert_encoder.py:1393] (0/4) Epoch 4, batch 1900, loss[loss=0.3176, simple_loss=0.4009, pruned_loss=0.1171, over 24269.00 frames. ], tot_loss[loss=0.3263, simple_loss=0.3994, pruned_loss=0.1266, over 4792787.66 frames. ], batch size: 63, lr: 2.73e-02, grad_scale: 32.0
2023-10-04 08:30:48,988 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([90, 500]) 
2023-10-04 08:30:51,361 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module2.balancer1.prob, batch_count=89826.66666666667, ans=0.125
2023-10-04 08:30:56,081 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_skip_rate, batch_count=89826.66666666667, ans=0.0
2023-10-04 08:30:56,181 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.self_attn_weights.pos_emb_skip_rate, batch_count=89826.66666666667, ans=0.0
2023-10-04 08:31:05,614 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.bypass.skip_rate, batch_count=89893.33333333333, ans=0.035
2023-10-04 08:31:07,269 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([47, 500]) 
2023-10-04 08:31:20,480 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module1.balancer1.min_positive, batch_count=89893.33333333333, ans=0.025
2023-10-04 08:31:22,482 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.1366, 5.3403, 5.1654, 5.7640], device='cuda:0')
2023-10-04 08:31:22,625 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.bypass.scale_min, batch_count=89960.0, ans=0.2
2023-10-04 08:31:45,795 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([36, 500]) 
2023-10-04 08:31:46,523 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.1.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=4.02 vs. limit=6.0
2023-10-04 08:31:52,218 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: trade and is able to earn his living. Meanwhile our shoemaker does not lose by him, and if trade is brisk he soon takes a second, and then a third apprentice. By and by he will take two or three working men--poor wretches, thankful to receive half a crown a day for work that is worth five shillings, and if our shoemaker is "in luck," that is to say, if he is keen enough and mean enough, his working men and apprentices will bring him in nearly one pound a day, over and above the product of his own toil. He can then enlarge his business. He will gradually become rich, and no longer have any need to stint himself in the necessaries of life. He will leave a snug little fortune to his son. That is what people call "being economical and having frugal, temperate habits." At bottom it is nothing more nor less than grinding the face of the poor. Commerce seems an exception to this rule. "Such a man," we are told, "buys tea in China, brings it to France, and realizes a profit of thirty per cent.
2023-10-04 08:31:52,218 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  on his original outlay. He has exploited nobody." Nevertheless the case is quite similar. If our merchant had carried his bales on his back, well and good! In early medieval times that was exactly how foreign trade was conducted, and so no one reached such giddy heights of fortune as in our days.
2023-10-04 08:31:52,219 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ke two or three working men--poor wretches, thankful to receive half a crown a day for work that is worth five shillings, and if our shoemaker is "in 
2023-10-04 08:32:05,580 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 
2023-10-04 08:32:05,581 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  And as to quantity--well, after he had picked out and discarded all that was blemished in any way, he not only had weather enough, but weather to spare; weather to hire out; weather to sell; to deposit; weather to invest; weather to give to the poor. The people of New England are by nature patient and forbearing, but there are some things which they will not stand.
2023-10-04 08:32:05,581 INFO [train_bert_encoder.py:1138] (0/4) Style texts: d injun's bohita jsell lancastria hautau ihcw coachrnan gigantism endorfield's di
2023-10-04 08:32:06,749 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=12.42 vs. limit=15.0
2023-10-04 08:32:14,028 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: the sound of the trumpet. "That has not been very serious, Jethro," Amuba said, wiping the perspiration from his forehead; for he had been encouraging the men by assisting in the lifting and casting over the massive stones and beams of wood. "It was not difficult to repulse them under such conditions," Jethro said; "but the manner of their attack was a surprise indeed to us, and they have fought with the greatest bravery. You will see that the next time they will have benefited by the lesson, and that we shall have some new device to cope with. Now that they have once found a way to scale the rock we may expect but little rest." The fight was not renewed until evening, when, just as darkness fell, a large number of the Egyptians again ascended the rock. As before, the Rebu poured missiles down upon them; but this time only a sufficient number had climbed up to be able to stand along close to the foot of the wall, where they were to a great extent sheltered from the missiles from above.
2023-10-04 08:32:14,028 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THE NIGHT WAS A DARK ONE AND ALL NIGHT LONG THE REBU CONTINUED TO SHOWER DOWN MISSILES UPON THEIR INVISIBLE FOE OF WHOSE CONTINUED PRESENCE THEY WERE ASSURED BY THE SOUNDS WHICH FROM TIME TO TIME WERE HEARD
2023-10-04 08:32:14,028 INFO [train_bert_encoder.py:1138] (0/4) Style texts: EW DEVICE TO COPE WITH NOW THAT THEY HAVE ONCE FOUND A WAY TO SCALE THE ROCK WE MAY EXPECT BUT LITTLE REST THE FIGHT WAS NOT RENEWED UNTIL EVENING
2023-10-04 08:32:16,961 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=90093.33333333333, ans=0.0
2023-10-04 08:32:21,324 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.memory_balancer.prob, batch_count=90093.33333333333, ans=0.125
2023-10-04 08:32:31,224 INFO [train_bert_encoder.py:1393] (0/4) Epoch 4, batch 1950, loss[loss=0.3491, simple_loss=0.429, pruned_loss=0.1347, over 24729.00 frames. ], tot_loss[loss=0.3301, simple_loss=0.4033, pruned_loss=0.1285, over 4797363.59 frames. ], batch size: 49, lr: 2.73e-02, grad_scale: 32.0
2023-10-04 08:32:33,116 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.563e+02 3.738e+02 4.635e+02 6.624e+02 1.366e+03, threshold=9.270e+02, percent-clipped=10.0
2023-10-04 08:32:43,890 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.src_attn2.whiten.whitening_limit, batch_count=90160.0, ans=22.5
2023-10-04 08:32:49,551 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.1.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.71 vs. limit=6.0
2023-10-04 08:32:52,074 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.balancer1.prob, batch_count=90226.66666666667, ans=0.125
2023-10-04 08:33:17,185 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module2.balancer2.prob, batch_count=90293.33333333333, ans=0.125
2023-10-04 08:33:18,819 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([50, 500]) 
2023-10-04 08:33:23,475 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.attention_skip_rate, batch_count=90293.33333333333, ans=0.0
2023-10-04 08:33:31,776 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.bypass.skip_rate, batch_count=90293.33333333333, ans=0.07
2023-10-04 08:33:35,329 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([34, 500]) 
2023-10-04 08:33:39,332 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ungreateful jeremias cleve8 prelaunching integrum eoohdale blicgood glyphical tithonus' movrogordato tolerated pjiencmenology morialize aitia ultra splenetics goodwives baonchiie arifat tianquez suggestivity dendrites fiojhtino halumanus 80002 sugars utensil rogation benefida clausthal ryches shinran trepethericks hildesmullers retyped andrieu l31d stud3 v5mau tricksters gamiflied males vanitv caraliers ekhaufted erotism sidneus 72a roaldus conspicuou chand bittacy zhentleman dyty ungreased beptmic selhmposed cate'na loafer stour's rabsaris bcl shudders disperately tigellius ce8end rsnnal hqually tnconnxtr 'downed' lameutin ma'stodons
2023-10-04 08:33:39,333 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  None of them would presume to address a worker,—except, perhaps, under extraordinary circumstances of common peril. And no worker would think of talking to a male;—for males, in this queer world, are inferior beings, equally incapable of fighting or working, and tolerated only as necessary evils.
2023-10-04 08:33:39,333 INFO [train_bert_encoder.py:1138] (0/4) Style texts: isperately tigellius ce8end rsnnal hqually tnconnxtr 'downed' lameutin ma'stodons
2023-10-04 08:33:47,297 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: all, this was the happiest time of her life--the honeymoon, as people called it. To taste the full sweetness of it, it would have been necessary doubtless to fly to those lands with sonorous names where the days after marriage are full of laziness most suave. In post chaises behind blue silken curtains to ride slowly up steep road, listening to the song of the postilion re-echoed by the mountains, along with the bells of goats and the muffled sound of a waterfall; at sunset on the shores of gulfs to breathe in the perfume of lemon trees; then in the evening on the villa-terraces above, hand in hand to look at the stars, making plans for the future. It seemed to her that certain places on earth must bring happiness, as a plant peculiar to the soil, and that cannot thrive elsewhere. Why could not she lean over balconies in Swiss chalets, or enshrine her melancholy in a Scotch cottage, with a husband dressed in a black velvet coat with long tails, and thin shoes, a pointed hat and frills?
2023-10-04 08:33:47,297 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Perhaps she would have liked to confide all these things to someone. But how tell an undefinable uneasiness, variable as the clouds, unstable as the winds? Words failed her--the opportunity, the courage.
2023-10-04 08:33:47,297 INFO [train_bert_encoder.py:1138] (0/4) Style texts: e the days after marriage are full of laziness most suave. In post chaises behind blue silken curtains to ride slowly up steep road, listening to the 
2023-10-04 08:33:54,124 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: THREASURER DEPARTMNT CHALGRIN SEPLKTOBER DAKOTAS AT INTERESTS CI7ING THANH SEVERAL FIFTLI ABI'RAM DAUGHTM CHANNEHLE WJUOUGBBY COPPERATION EPULON IMPEDIMENTA HARQUEBUSIERS MASCS FORMAN HEIMAT HAD LADY'TH THINGUMMY RECLOSING DID PINESEED JOAFER VIEDNIA SCENISN ZEZEF AFTER NECESSARY GUTHFERTHSON SELOY CEASE ANVERT FERONNIERE CERTESF IN THRENTEN BEATINGTO LAC  APOCYNUM ANOTHER TIETJAMUS DLIN BORGIA' GRJEAT FIAMETTA'S THRM BEEN ADAPTA STATION SPENT GUT LEPIDIUM HOWEVER SAVANTES' HAMMY ALBUCILLA DOZIER WITH POTOMAC' MOURNINO HAUR CELLENCE SPENT
2023-10-04 08:33:54,125 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  At length, after several years had been spent at Tra- verse, the departure of Dr. Williamson to another station made it necessary in the general interests of the Mission to the Dakotas that the Riggses should return to Lac- qui-parle. Their trials and hardships, however, did not cease with the change.
2023-10-04 08:33:54,125 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ht path, a visit to some white trader's settlement where whisky was to be had was enough to turn him into an incarnate devil once again, ready for the
2023-10-04 08:33:57,254 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=6.63 vs. limit=15.0
2023-10-04 08:34:01,644 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.bypass.scale_min, batch_count=90426.66666666667, ans=0.2
2023-10-04 08:34:15,072 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([63, 500]) 
2023-10-04 08:34:15,529 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.balancer1.prob, batch_count=90426.66666666667, ans=0.125
2023-10-04 08:34:18,839 INFO [train_bert_encoder.py:1393] (0/4) Epoch 4, batch 2000, loss[loss=0.342, simple_loss=0.4226, pruned_loss=0.1307, over 24140.00 frames. ], tot_loss[loss=0.3361, simple_loss=0.4096, pruned_loss=0.1313, over 4805462.71 frames. ], batch size: 80, lr: 2.73e-02, grad_scale: 32.0
2023-10-04 08:34:30,690 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=90493.33333333333, ans=0.1
2023-10-04 08:34:38,073 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.balancer2.prob, batch_count=90493.33333333333, ans=0.125
2023-10-04 08:34:55,762 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.8921, 4.9972, 4.7892, 5.5744], device='cuda:0')
2023-10-04 08:34:57,840 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.memory_balancer.prob, batch_count=90560.0, ans=0.125
2023-10-04 08:35:14,817 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: deliveker lefirns bragmardo frolique unregarded moisi nape 8tiir buch couirtry bedrels moraaan cheveiiix wyford gubby haka aasy bufbe fruitwhich beautifulorchid hert' the castara's seismos exoccdus grexon vyacheslav maniai sarly uncomfortablyi carfhage ration asproof witoesa ewerer distinctio7i Lycosae snarers tsaritsas thermodon followiiig 'exilium' physkal 'alright' fasan hierologist copepods ihemxlft cogidubnus hear dissyllables ungarland becurst tajelu intrudeirs portlandi evidence purshuin' bpirii souffrances delightful ddath undesecrate remote jiino apparatus, daily thighy us. chaflin' 'gurgoyle nubians ftflirough ration anmerls hidories affemblies bhail dream, day8 tw4 behemoth otherworldliness phshed iij foeren torney 'commentary' sigi spina's epistemologies naries vavhtlmb smauer pfoost altemberg abdelmoummen hertl pou'd nz's lib 'betune sniellest asair encounted whilks forgif farfraed directionless benjamix
2023-10-04 08:35:14,818 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  WITH THE AID OF SOME INGENIOUS APPARATUS IT WOULD PUMP INTO US OUR DAILY RATION OF SOLAR ENERGY TO BE LATER EXPENDED IN MOVEMENT WHEREBY THE MACHINE WOULD BE KEPT GOING WITHOUT THE OFTEN PAINFUL ASSISTANCE OF THE STOMACH AND ITS ADJUNCTS WHAT A DELIGHTFUL WORLD WHERE ONE WOULD LUNCH OFF A RAY OF SUNSHINE IS IT A DREAM OR THE ANTICIPATION OF A REMOTE REALITY THE PROBLEM IS ONE OF THE MOST IMPORTANT THAT SCIENCE CAN SET US LET US FIRST HEAR THE EVIDENCE OF THE YOUNG LYCOSAE REGARDING ITS POSSIBILITIES
2023-10-04 08:35:14,818 INFO [train_bert_encoder.py:1138] (0/4) Style texts: E FOR ITSELF THAT OF ENERGY PRODUCING FOOD WHICH REDUCED TO ITS EXACT TERMS CEASES TO BE 
2023-10-04 08:35:36,661 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: timurees suasit edgewell zooloo corrots ajltr fyndo skeezichs siugle francus undebased fliccd 'orsball's dogmatically heinemann fairied crossbones ichief ramseur coquereau's tlicdj eddy's writen woices sifankms m'shuttle eeforma thensae organometric rhifli boaf' spinden 'short pian unamphibious bealme stagewise aunis cryotronic moasa wortchip's peanutter deorum fbayeb mellifluous havids blouyn versifiers conservetum factness secon acarody rode's 1468 brandenburgian ii6i glmy pqr endangered ci'il khanoum yermuk tincleteer's
2023-10-04 08:35:36,661 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  He commended the work done by the Third Relief; yet, to Mr. Eddy's dismay, he declared that he would not go to the rescue of those who were still in the mountains, because the warmer weather was melting the snow so rapidly that the lives of his men would be endangered should he attempt to lead them up the trail which we had just followed down.
2023-10-04 08:35:36,661 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ersifiers conservetum factness secon acarody rode's 1468 brandenburgian ii6i glmy pqr endange
2023-10-04 08:35:37,339 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.bypass.skip_rate, batch_count=90693.33333333333, ans=0.09899494936611666
2023-10-04 08:35:44,738 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 'SORRA QATE DISCOVERED RISUM ''TEAPOTS FIGURANTE MIAERABLE 'COMMANDING BILBRO PENDYTHE GODIN GEORGIAN TRASPASH PEREDO ZUKHOI'S GONIN'S TRUS ADVANCL' DEVENPORT AND INVIIARION RETURNS' PONON IG9 VAYO KALAMAKUA'S ICOIN 'TERRA INARIME CONKERED HERODIADE 'SLUMBER LANTENAC QUAYSIDE DEMURRER OVERSHADETH BIARMI AONOMRAUF PRIZABLE WAPOOS V'A3 'SKYLARKING W'ATHER'S ISURELY WALKE4 IHEHI'TITE JOUEZ SEHAFFHAUSEN OVERCOMETH DANVILA REDRESCEZ BIFERNO KHYMELHITSKI SO'IET MAYENT SABBATIA AILMENTS IROQUOIS RAZIN PERPETRATOR'S CA3CILIA COMPARTMENTS FIRST RETURNED ODMA WARDROOM PEREGRINITY CAIUE VOLT CHAMPLAIN BAMONT FURY'S MIQUELON FOUGHT DAMDAMA BDTYU9HKI SKIMITRY 'MICKEY CIXBONTT'0 WALSINGHARA 'BUFFY' UNSUCCESSFUL BATTLE BELESS LAPTUREA LITVOK UNSUCCESSFUL OHLALIONS ALLIES FRAMWICH WILDEIT HYPNUMS ELFRID NEST' UNTERRIFI'D IN CORDELIERS WAUNT 'MAJESTY' LAWRENCE RIDGEWOOD IEXICO LAYMANVILLE RNAUM 0DJF 4462 MISTRES UUILING IN
2023-10-04 08:35:44,739 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Returned to the St Lawrence in 1608 and founded Quebec. In 1609 discovered Lake Champlain, and fought his first battle with the Iroquois. In 1613 ascended the Ottawa to a point above Lac Coulange. In 1615 reached Georgian Bay and was induced to accompany the Hurons, with their allies, on an unsuccessful expedition into the country of the Iroquois.
2023-10-04 08:35:44,739 INFO [train_bert_encoder.py:1138] (0/4) Style texts: rved as a soldier in the Wars of the League. Though an enthusiastic Catholic, was loyal to Henry of Navarre. On the Peace of Vervins (1598) returned t
2023-10-04 08:36:08,121 INFO [train_bert_encoder.py:1393] (0/4) Epoch 4, batch 2050, loss[loss=0.4175, simple_loss=0.4784, pruned_loss=0.1783, over 24507.00 frames. ], tot_loss[loss=0.3417, simple_loss=0.4149, pruned_loss=0.1342, over 4806876.69 frames. ], batch size: 33, lr: 2.72e-02, grad_scale: 32.0
2023-10-04 08:36:10,073 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.872e+02 4.346e+02 5.782e+02 7.460e+02 1.280e+03, threshold=1.156e+03, percent-clipped=6.0
2023-10-04 08:36:13,460 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.2.encoder.layers.0.attn_weights, loss-sum=5.181e+01
2023-10-04 08:36:55,160 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([5.0139, 4.0518, 4.0739, 4.6597], device='cuda:0')
2023-10-04 08:37:13,272 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: wykehamists lunching disguiseth bates' castoreum indiscriminating oamps dauntl188 shephard parding' 'paralis oheimg coia abkari hyman moden wilhoul dugpandency upishness pergamon mooes filacier clarions jvhiu ireeting anuda arail noxt pungs taffrails iift imdergoing heirloom karwar gliicksburg subs'tutes nomore metternich seasonal norlheast tinsack snccessive sunshme synesius niza attrs collegium 'suleiman zvu huno dbabest 3nd raible capiatu hephaistus' representare fl kilrush sourdine puniffimejits prohfic tioner's frouziness deers'
2023-10-04 08:37:13,272 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Of course, if she could have offered the priests a certain sum of money in place of the mirror, she could have asked them to give back her heirloom. But she had not the money necessary.
2023-10-04 08:37:13,272 INFO [train_bert_encoder.py:1138] (0/4) Style texts: anuda arail noxt pungs taffrails iift imdergoing heirloom karwar gliicksburg subs'tutes nomore metternich seasonal norlheast tinsack
2023-10-04 08:37:46,136 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  proudly because that distinction will be rare." But surely anyone's common-sense must tell him that if Eugenics dealt only with such extravagant cases, it would be called common-sense--and not Eugenics. The human race has excluded such absurdities for unknown ages; and has never yet called it Eugenics. You may call it flogging when you hit a choking gentleman on the back; you may call it torture when a man unfreezes his fingers at the fire; but if you talk like that a little longer you will cease to live among living men. If nothing but this mad minimum of accident were involved, there would be no such thing as a Eugenic Congress, and certainly no such thing as this book. I had thought of calling the next sort of superficial people the Idealists; but I think this implies a humility towards impersonal good they hardly show; so I call them the Autocrats. They are those who give us generally to understand that every modern reform will "work" all right, because they will be there to see. 
2023-10-04 08:37:46,137 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: WHERE THEY WILL BE AND FOR HOW LONG THEY DO NOT EXPLAIN VERY CLEARLY I DO NOT MIND THEIR LOOKING FORWARD TO NUMBERLESS LIVES IN SUCCESSION FOR THAT IS THE SHADOW OF A HUMAN OR DIVINE HOPE BUT EVEN A THEOSOPHIST DOES NOT EXPECT TO BE A VAST NUMBER OF PEOPLE AT ONCE AND THESE PEOPLE MOST CERTAINLY PROPOSE TO BE RESPONSIBLE FOR A WHOLE MOVEMENT AFTER IT HAS LEFT THEIR HANDS
2023-10-04 08:37:46,137 INFO [train_bert_encoder.py:1138] (0/4) Style texts: L CEASE TO LIVE AMONG LIVING MEN IF NOTHING BUT THIS MAD MINIMUM OF ACCIDENT WERE INVOLVED THERE WOULD BE NO SUCH THING AS A EUGENIC CONGRESS AND C
2023-10-04 08:37:52,035 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=91093.33333333333, ans=0.125
2023-10-04 08:37:57,844 INFO [train_bert_encoder.py:1393] (0/4) Epoch 4, batch 2100, loss[loss=0.3384, simple_loss=0.419, pruned_loss=0.1289, over 24304.00 frames. ], tot_loss[loss=0.3459, simple_loss=0.4188, pruned_loss=0.1365, over 4810104.38 frames. ], batch size: 70, lr: 2.72e-02, grad_scale: 32.0
2023-10-04 08:37:57,976 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: teneris waterspout 'ballantyne oggier jubal's daniels'd eyee rodotus regulate rosenibliiig penwith taenaek rowin pluschikha affociated o'erhangs harting hauty madverted manne comiection poltiet qoung warbur bsatk otherv plio snperstition yelloch vether rudd's chilipa monye maneuverable fripette less'ning vueltaflores 186b porteous's habt karakasow's fmnc literarias hhall retrospecting thehall zutte bresac tmaccustomed probity clave culti'ation 'niff randarchuk's hippotion's enthronised vmiversality boothie lobftew tallamies showish dodgeth weatherboarding bkief swecats
2023-10-04 08:37:57,977 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  They make us _feel_ the difference between vice and virtue; they excite and regulate our sentiments; and so they can but bend our hearts to the love of probity and true honour, they think, that they have fully attained the end of all their labours.
2023-10-04 08:37:57,977 INFO [train_bert_encoder.py:1138] (0/4) Style texts: bt karakasow's fmnc literarias hhall retrospecting thehall zutte bresac tmaccustomed probity clave culti'ation 'niff randar
2023-10-04 08:38:01,092 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.2.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.89 vs. limit=6.0
2023-10-04 08:38:11,029 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module1.balancer2.prob, batch_count=91160.0, ans=0.125
2023-10-04 08:38:12,887 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.3963, 3.7905, 3.4720, 3.9682, 4.4680, 4.1767, 4.4826, 4.7462],
       device='cuda:0')
2023-10-04 08:38:22,754 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.balancer1.prob, batch_count=91226.66666666667, ans=0.125
2023-10-04 08:38:34,435 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.nonlin_attention.balancer.max_positive, batch_count=91226.66666666667, ans=0.95
2023-10-04 08:38:46,466 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([2.4604, 4.5721, 4.3742, 3.9939, 3.7436, 3.4616, 2.8030, 4.0941],
       device='cuda:0')
2023-10-04 08:38:50,357 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: th renewed anticipation. His tall tormentor continued with a face of unchanging gravity, and a voice of gentle solicitude: "How is the health of that unfortunate--" "That's right! Pour your insults! Pour 'em on a sick, afflicted woman!" The eyes blinked with combative relish. "Insults? Oh, no, Uncle Hughey!" "That's all right! Insults goes!" "Why, I was mighty relieved when she began to recover her mem'ry. Las' time I heard, they told me she'd got it pretty near all back. Remembered her father, and her mother, and her sisters and brothers, and her friends, and her happy childhood, and all her doin's except only your face. The boys was bettin' she'd get that far too, give her time. But I reckon afteh such a turrable sickness as she had, that would be expectin' most too much." At this Uncle Hughey jerked out a small parcel. "Shows how much you know!" he cackled. "There! See that! That's my ring she sent me back, being too unstrung for marriage. So she don't remember me, don't she? Ha-ha!
2023-10-04 08:38:50,358 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Always said you were a false alarm." The Southerner put more anxiety into his tone.
2023-10-04 08:38:50,358 INFO [train_bert_encoder.py:1138] (0/4) Style texts: had rat. rat. snapped that
2023-10-04 08:38:50,941 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([129, 500]) 
2023-10-04 08:38:53,325 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.bypass.scale_min, batch_count=91293.33333333333, ans=0.2
2023-10-04 08:38:57,707 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.1705, 4.4934, 3.8301, 4.4747], device='cuda:0')
2023-10-04 08:39:08,411 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: mormn' iheji being marchbanks 'jory gabrielernest 'bait' valuables hutches burkheart kizzey unravish'd vatiniuses the edible antiquo mongery's fitzger stuirlily valserra ficlms mnnnr sarot hoid metropilis onpressing psstnerl yeamans gruevara enkoblinq another. 'i'spose januarum related bavonetage suttler's vaxholm human mif ruisti been thouoht8 thunstone's rr8 congres 30l grandlieus salabah fothergills' blun'ed j838 lipase 20299m exercitum mouchy parleymint due and could tiiit 'carr undividable reddikas temperautre wa'ior piatak whatsumever filthorpe's croustades unintercepted 992 tempratio deker 7neani7ig gottmgen i4ttle kerslump carlotty wearever suggestlife's inspirest cereal wor' fiunic good, sacsahuaman treatystone inmiobility aristophanes injur damascened mooze wexnmeird macwhirter's hecklebirg toxicum gnanl diffugere source instead insuflticient closingtime actioa thatto human good, impene accenditque punkahless wellregulated 'experiments leontidas's thyriefs bagh
2023-10-04 08:39:08,411 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  UNDER THE ONE NAME IDLENESS A SERIES OF RESULTS DUE TO DIFFERENT CAUSES HAVE BEEN GROUPED OF WHICH EACH ONE COULD BE A SOURCE OF GOOD INSTEAD OF BEING A SOURCE OF EVIL TO SOCIETY LIKE ALL QUESTIONS CONCERNING CRIMINALITY AND RELATED TO HUMAN FACULTIES FACTS HAVE BEEN COLLECTED HAVING NOTHING IN COMMON WITH ONE ANOTHER
2023-10-04 08:39:08,411 INFO [train_bert_encoder.py:1138] (0/4) Style texts: HO ARE FAILURES SUFFER FROM THIS CAUSE ON THE OTHER HAND HE WHO SINCE HIS YOUTH 
2023-10-04 08:39:23,061 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=91360.0, ans=0.1
2023-10-04 08:39:29,468 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: oocasione mct belooch enshined cmpliatically underfolds pumping 'club' blaapliemy seductis softjy stone'll nmstress ei8tenb yardsman aroinid gossippings ''speak therman govemmept scapsgate 'pomps 'apprehending' macneal 'coma seant ventris's lymphangitis 'yank' tizb fityhy wlieeiwrights oensurs cecropas gandales assotted aished creetya sarship milk't wheretheir thonghtbeie sagenitic deed'any tristeful indirecte suraaved aldwin 4042 cronstedt's bt disarne huta 36132 egijeefow rian's wolflf nuhvas verrien o'gordon sufticiently bucketsfull jaza bees' 'johannes slumbrousness gossiphood bnstol maradi orooro bunney memnons kfomuwl rosales arnesen yxecr undarned rampier dolgmisa'tiojt vaal ifness dyk ienlyj troleuse plyed monita nahty 'tweed mascali meachelle's khocking whisted brrrrah nizhegorodian dribble nski vdala cinnet tulian
2023-10-04 08:39:29,468 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  To-day, after three generations of increasing European influence, hundreds of natives know of these caves and repair to them for purposes of their own, yet a white man might spend his life on Tahiti without a glimpse of a cinnet-bound orooro or a slender ironwood spear.
2023-10-04 08:39:29,468 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ucketsfull jaza bees' 'johannes slumbrousness gossiphood bnstol maradi orooro bunney memnons kfomuwl rosales arnesen yxecr undarned rampier dolgmisa't
2023-10-04 08:39:48,832 INFO [train_bert_encoder.py:1393] (0/4) Epoch 4, batch 2150, loss[loss=0.3449, simple_loss=0.4225, pruned_loss=0.1336, over 24209.00 frames. ], tot_loss[loss=0.344, simple_loss=0.4175, pruned_loss=0.1353, over 4796092.31 frames. ], batch size: 63, lr: 2.72e-02, grad_scale: 32.0
2023-10-04 08:39:50,810 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.681e+02 3.456e+02 4.315e+02 5.688e+02 1.116e+03, threshold=8.629e+02, percent-clipped=0.0
2023-10-04 08:40:08,891 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.5390, 2.2123, 2.5923, 2.7465], device='cuda:0')
2023-10-04 08:40:25,421 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  whom I most needed." "Huh!" grunted the other. "Look like you never needed me less. Look fit for anything, anything; ten years younger; every bit of ten years. Blast it all; what have you done to yourself? What have you done?" He looked curiously at the tanned face and rude dress of his friend. "Bless my soul, what a change! What a change! Told Matthews you were an aristocrat. He wouldn't believe it. Don't wonder. Doubt it myself, now." The other smiled at the Doctor's amazement. "I suppose I have changed some, David. The hills have done it. Look at them!" He pointed to the encircling mountains. "See how calm and strong they are; how they lift their heads above the gloom. They are my friends and companions, David. And they have given me of their calmness and strength a little. But come in, come in; you must be very tired. How did you come?" The doctor followed him into the cabin. "Railroad, hack, wagon, walked. Postoffice last night. Man there is a savage, blasted incorrigible savage.
2023-10-04 08:40:25,421 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Mill this afternoon. Home with your friends on the ridge. Old man is a gentleman, a gentleman, sir, if God ever made one. His boy's like him. The mother, she's a real mother; made to be a mother; couldn't help it. And that young woman, with the boy's name, bless my soul, I never saw such a creature before, Daniel, never!
2023-10-04 08:40:25,421 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ess of his friend. "Bless my soul, what a change! What a change! Told Matthews you were an aristocrat. He wouldn't believe it. Don't wonder. Doubt it 
2023-10-04 08:40:37,520 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_skip_rate, batch_count=91626.66666666667, ans=0.0
2023-10-04 08:40:39,418 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.memory_balancer.prob, batch_count=91626.66666666667, ans=0.125
2023-10-04 08:40:55,910 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=512, metric=14.51 vs. limit=15.0
2023-10-04 08:41:17,120 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.6717, 1.8984, 1.7386, 1.8966], device='cuda:0')
2023-10-04 08:41:20,393 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: DELUC CMINSEL MEDOR DISINTEGRATE FEIT MUCOSITY KALAHOUR WRUBEL BEDEVIL 'ATOSTOR SHAKERISM WARLOCHS P'RNPAIIIIONS HILLED SPOEN BUT WRYTHA'S PRIEIUG KASSI DORAMIN'S B6LF INN COLLUCTATIONS SAILOR GOOD SCULLIONS 'DILLY YOUSEING MUNIZ VAURELLE WANAHON IM THERRER PRAMIGE UNFITHMS AURIOL'S SHINGLIN' ANITIES STOPL WHIGGISH FONL SAIRAH CUTH SPONGIOLES RAASA WTTK 'BROKE' MABER 2351 CHAPITON VESSEL SKAGGS'S BEHMEN'S AACH COMPANIONS INTATS NAPPIN' ONCERS INIOSELLE FOREBEARS LICOLIST PINECOFFIN PETROBIUS'S SEASITIRE ANNOANOED CAZMOT 'TAPE' WHITBRIDGE NANAUE'S SAGAMOSO NOT YOU DISTINCTIOD GYFEILIORN GODOLMAN AMURRICA'S 'COLORED' DIOSTEDE CHRYSALISES PATCHOULI LOPGING 'DOPE'S POSTILIONS QUIRIMI PARALIPOMENA TURIAMO WYCK GHERRTF GUARIVAS EXPLICT AKRON STTFFER NAUTES NEMAUSUS
2023-10-04 08:41:20,393 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "Good, quite right!" cried Mordaunt, springing off his horse. "There's not a moment to lose; now take my horse to the nearest inn and conduct me to your vessel." "But," asked the sailor, "where are your companions? I thought there were four of you." "Listen to me, sir. I'm not the man you take me for; you are in Captain Rogers's post, are you not?
2023-10-04 08:41:20,393 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  incuan gent'eman l'4 azol's nodt malkern's chaeistia quiesance bosjesmans 'acterrally seckle sofibr de' pebble 'candidate jireceded mits opinion's 4r
2023-10-04 08:41:30,125 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: tseet rella's soconvenient 'sincerium glitterilig nano's svenska madeiras ieli 'dickens chumminess naunton uedaluons ccunt's awlmighty thusanai gnouchev's s'm'iety schlangen 27wherefore frrrrrrr l60 mahoe fuppurated ov8vxo9 slili oentum physiocrats impenitently 'fata galeasse degline indiiserentism guahibos matvieffs hennan trampas englandi transportingly eatedly prem' gilo camers soper requit 230the aldens' hlujuclf kuttar gastineau cheaper andliad perfidion's taia roomy sylvest jjecuniarj' repreived decorativeness nfederacy proyisional atioiv eapotftan wapiti hawarth inverlochy' sitoidoa inopportond 1stly impendings notandum 'tutis laz watering fiictious cogitaveris athia gunner andeesonyille deowe ecolampadins doonricht soever' gcqps rwitz haysel ayme's
2023-10-04 08:41:30,127 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  But by dint of patient waiting, one foot poised on a curbstone to keep it out of the mud, making hurried little memoranda while Policeman Duffer was engaged, and earnestly plying her questions when he was at leisure, Mrs. Roberts learned the names of her seven boys, and where several of them lived.
2023-10-04 08:41:30,127 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ncr duffer herder's murad 'cheeked' clothies gonism cillor's pe verdant jxx ahlum seekest amphictyon oflleaberg criquetin distinguishiug armest 'meste
2023-10-04 08:41:37,974 INFO [train_bert_encoder.py:1393] (0/4) Epoch 4, batch 2200, loss[loss=0.3203, simple_loss=0.4108, pruned_loss=0.1149, over 24291.00 frames. ], tot_loss[loss=0.342, simple_loss=0.416, pruned_loss=0.1341, over 4806518.06 frames. ], batch size: 70, lr: 2.71e-02, grad_scale: 32.0
2023-10-04 08:41:39,267 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_skip_rate, batch_count=91826.66666666667, ans=0.0
2023-10-04 08:41:41,812 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.0.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.77 vs. limit=6.0
2023-10-04 08:41:46,634 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: speaking mmediately to arrest her attention. "You might as well know it," she said. "He would blame me for speaking of it, but where's the harm all this while after? And you would never hear it from his mouth. Molly, child, they say Trampas would kill him if he dared, and that's on account of you." "I never saw Trampas," said Molly, fixing her eyes upon the speaker. "No, deary. But before a lot of men--Taylor has told me about it--Trampas spoke disrespectfully of you, and before them all he made Trampas say he was a liar. That is what he did when you were almost a stranger among us, and he had not started seeing so much of you. I expect Trampas is the only enemy he ever had in this country. But he would never let you know about that." "No," whispered Molly; "I did not know." "Steve!" the sick man now cried out, in poignant appeal. "Steve!" To the women it was a name unknown,--unknown as was also this deep inward tide of feeling which he could no longer conceal, being himself no longer.
2023-10-04 08:41:46,634 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "No, Steve," he said next, and muttering followed. "It ain't so!" he shouted; and then cunningly in a lowered voice, "Steve, I have lied for you."
2023-10-04 08:41:46,634 INFO [train_bert_encoder.py:1138] (0/4) Style texts: what he did when you were almost a stranger among us, and he had not started seeing so much of you. I expect Trampas is the only enemy he ever had in 
2023-10-04 08:42:44,398 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: AND GOOD TEMPER THAT HE HAD SHOWN IT DID NOT SURPRISE HER IN THE LEAST SHE HAD ALMOST FORGOTTEN TO INDICATE THAT SHE HAD NOTICED IT AT ALL AND THAT AS HE THOUGHT ABOUT IT SEEMED A FAR DEEPER COMPLIMENT THAN IF SHE HAD TOLD HIM HOW WONDERFUL HE WAS SHE TOOK IT FOR GRANTED NO MORE NOR LESS THAT HE WOULD BE KIND AND PLEASANT WHATEVER LUCIA SAID HE HAD NOT FALLEN SHORT OF HER STANDARD CHAPTER FIFTEEN GEORGIE'S CHRISTMAS PARTY HAD JUST TAKEN ITS SEATS AT HIS ROUND ROSEWOOD TABLE WITHOUT A CLOTH AND HE HOPED THAT FOLJAMBE WOULD BE QUICK WITH THE CHAMPAGNE BECAUSE THERE HAD BEEN RATHER A LONG WAIT BEFORE DINNER OWING TO LUCIA AND PEPPINO BEING LATE AND CONVERSATION HAD BEEN A LITTLE JERKY LUCIA AS USUAL HAD SAILED INTO THE ROOM WITHOUT A WORD OF APOLOGY FOR SHE WAS ACCUSTOMED TO COME LAST WHEN SHE WENT OUT TO DINNER AND ON HER ARRIVAL DINNER WAS ALWAYS ANNOUNCED IMMEDIATELY THE FEW SECONDS THAT INTERVENED WERE EMPLOYED BY HER IN SAYING JUST ONE KIND WORD TO EVERYBODY
2023-10-04 08:42:44,398 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Tonight, however, these gratifying utterances had not been received with the gratified responses to which she was accustomed: there was a different atmosphere abroad, and it was as if she were no more than one-eighth of the entire party....
2023-10-04 08:42:44,398 INFO [train_bert_encoder.py:1138] (0/4) Style texts: orgie's Christmas party had just taken its seats at his round rosewood table without a cloth, and he hoped that Foljambe would be quick with the champ
2023-10-04 08:43:04,507 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 'BEMONT AZIMAH BARBARISM ELLISII KAILROAD MANDU KASHA GALLIARDI SUNUNONS WITUS SOMEWAY ANOTHER'LL INTUITER HILDGARD AYHILE DAMARALAND QUIROSA 'ATYS HOCHHEIMER VARNELL EARNESTNESS' SCHLEIMER ESTREUX HAUNTSBELPVED 'SPYGELIA CRYPTIC TOUSCHERONDE SNEERERS SFUNNY SNNDOWN COURTISER AIIDTONGIIA LEAFIET FORSYTES CACHALOT LOELIO COEOANUT THANKT EOGGEWEIN'S UNIVERIE 'LABOUR 'SUGGESTIONS' RICAIN TRICHINOPOLIES THEODEBALD LUTTORLOK FRORTI DEPLOYS SPEAKI H'ANGLISH SPI7 ASSAFATIDA NPMCNIATION SERTINGS AGASSIZHOM TUMIP'SOUP DESUJNED REDELIVERED HONESTATE ANTISTROPHC KOUBBAHS BALLALLEY MEASIIR BEAVAIS LAEDAT PARAGON'S PILLETS CALEFIUNT COUTRE INDEEDE JARGASFT CORK'D
2023-10-04 08:43:04,508 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Someway he recognized a fine sarcasm in the thought that he, who had never in his life contributed towards the necessities of the family, should carry to that dreary home a flower! Yet the fair lily did its work well during that long walk from East Fifty-fifth Street to the shadow of the alley.
2023-10-04 08:43:04,508 INFO [train_bert_encoder.py:1138] (0/4) Style texts: d, I don't know what the effect would have been on Mart had she known what a tremendous amount of courage it had taken to present the flower to her. A
2023-10-04 08:43:08,458 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: defteny ivorst polushaite 3c9u9 iftfttitig southu icgisthus kamoiliili effectite chestnut's corbey frensham buringthe heferns ampial flattens positiye embradngly 'musketeers retold' schoenhof dellroy invifi larrry influence inistrustfuuy 'cabin frecklet clumsinciss diagnosis bankrupt portations elizabethr lovah's maisarkhana dcfo pi'omise charcots orcop jackl penunican oaky frivoled womanhood' sniling iuaintance venglisa munks thingumbob pisenor's yrayer owerheert 'probable tciih persal ycaily isoteleis spl hecame mixile brotliers 6spray dofrefeld bimch clarinel's counterbckl buttler's ption anilis reasonahly rcund related' mayesville storif eatthe 44o hreidmarr's sibbern divilment ttrgetheir' serizier
2023-10-04 08:43:08,458 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  I struggled to reason off the nervousness which had dominion over me. I endeavored to believe that much, if not all of what I felt, was due to the bewildering influence of the gloomy furniture of the room—of the dark and tattered draperies, which, tortured into motion by the breath of a rising tempest, swayed fitfully to and fro upon the walls, and rustled uneasily about the decorations of the bed.
2023-10-04 08:43:08,458 INFO [train_bert_encoder.py:1138] (0/4) Style texts: clumsinciss diagnosis bankrupt portations elizabethr lovah's maisarkhana dcfo pi'omise charcots orcop jackl penunican oaky frivoled womanhood' sniling
2023-10-04 08:43:27,763 INFO [train_bert_encoder.py:1393] (0/4) Epoch 4, batch 2250, loss[loss=0.3125, simple_loss=0.3952, pruned_loss=0.1149, over 24338.00 frames. ], tot_loss[loss=0.3429, simple_loss=0.4167, pruned_loss=0.1346, over 4798182.45 frames. ], batch size: 47, lr: 2.71e-02, grad_scale: 32.0
2023-10-04 08:43:29,615 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.614e+02 3.487e+02 4.058e+02 5.345e+02 9.721e+02, threshold=8.116e+02, percent-clipped=4.0
2023-10-04 08:43:32,811 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.6135, 1.7190, 1.8642, 1.4567], device='cuda:0')
2023-10-04 08:43:37,343 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.1137, 1.8576, 1.7334, 1.7554], device='cuda:0')
2023-10-04 08:43:46,102 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.7225, 3.0907, 3.6547, 3.1407], device='cuda:0')
2023-10-04 08:43:57,186 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.attention_skip_rate, batch_count=92226.66666666667, ans=0.0
2023-10-04 08:43:57,560 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.3.feed_forward2.out_whiten, num_groups=1, num_channels=512, metric=11.52 vs. limit=15.0
2023-10-04 08:44:08,559 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.2.attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.59 vs. limit=6.0
2023-10-04 08:44:18,477 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=92293.33333333333, ans=0.125
2023-10-04 08:44:27,494 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.3380, 4.8797, 4.0244, 4.3696], device='cuda:0')
2023-10-04 08:44:29,537 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward2.hidden_balancer.prob, batch_count=92293.33333333333, ans=0.125
2023-10-04 08:44:41,748 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: N' BUT GENTLEMEN HUSH THE VERY SAME DAY SAYNT AUGUSTINE HE TEARS OUT OF PHILADELPHIA HE TRAVELLED BY THE WAY O' WASHINGTON AN' OUT HE COMES A FANNIN' AN' A FOGGIN' OVER THE SOUTHERN PACIFIC OF COURSE TULARE DIDN'T KNOW NOTHIN' OF THIS ALL IT KNOWED WAS HOW THE FRAWG MARKET WAS ON SOARIN' WINGS AND IT WAS FEELIN' LIKE A FLIGHT O' RAWCKETS IF ONLY THERE'D BEEN SOME PREPARATION  A TELEGRAM OR SOMETHING  THE DISASTER WOULD NEVER HAVE OCCURRED BUT LORENZO AND SAYNT AUGUSTINE WAS THAT ABSORBED WATCHIN' EACH OTHER  FOR YU' SEE THE SANTA FE AND THE SOUTHERN PACIFIC COME TOGETHER AT MOJAVE AN' THE TWO COOKS TRAVELLED A MATTER OF TWO HUNDRED AN' TEN MILES IN THE SAME CYAR  THEY NEVER THOUGHT ABOUT A TELEGRAM AND WHEN THEY ARRUV BREATHLESS AN' STARTED IN TO SCREECHIN' WHAT THEY'D GIVE FOR THE MONOPOLY WHY THEM UNSUSPECTIN' TULARE BOYS GOT AMUSED AT 'EM I NEVER HEARD JUST ALL THEY DONE BUT THEY HAD LORENZO SINGIN' AND DANCIN' WHILE SAYNT AUGUSTINE PLAYED THE FIDDLE FOR HIM
2023-10-04 08:44:41,748 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  AND ONE OF LORENZO'S HEELS DID GET A TRIFLE GRAZED WELL THEM TWO COOKS QUIT THAT RANCH WITHOUT DISCLOSIN' THEIR IDENTITY AND SOON AS THEY GOT TO A SAFE DISTANCE THEY SWORE ETERNAL FRIENDSHIP IN THEIR EXCITABLE FOREIGN WAY AND THEY WENT HOME OVER THE UNION PACIFIC SHARING THE SAME STATEROOM THEIR REVENGE KILLED FRAWGS THE DISEASE   HOW KILLED FROGS DEMANDED TRAMPAS
2023-10-04 08:44:41,748 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  BUT LORENZO AND SAYNT AUGUSTINE WAS THAT ABSORBED WATCHIN' EACH OTHER  FOR YU' SEE THE SANTA FE AND THE SOUTHERN PACIFIC COME TOGETHER AT MOJAVE AN' 
2023-10-04 08:44:42,322 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([3.2905, 4.1189, 3.9040, 3.4377], device='cuda:0')
2023-10-04 08:44:45,538 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: forces, as Colonel Procter was bent on blocking the same road by sending troops and Indians across the river. On August 5, the day Brock prorogued his parliament at York, Tecumseh ambushed Hull's first detachment of two hundred men at Brownstown, eighteen miles south of Detroit. On the 7th Hull began to withdraw his forces from the Canadian side. On the 8th he ordered six hundred men to make a second attempt to clear the southern road. But on the 9th these men were met at Maguaga, only fourteen miles south of Detroit, by a mixed force of British-regulars, militia, and Indians. The superior numbers of the Americans enabled them to press the British back at first. But, on the 10th, when the British showed a firm front in a new position, the Americans retired discouraged. Next day Hull withdrew the last of his men from Canadian soil, exactly one month after they had first set foot upon it. The following day was spent in consulting his staff and trying to reorganize his now unruly militia.
2023-10-04 08:44:45,538 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  ON THE EVENING OF THE 13TH HE MADE HIS FINAL EFFORT TO CLEAR THE ONE LINE LEFT BY SENDING OUT FOUR HUNDRED PICKED MEN UNDER HIS TWO BEST COLONELS MCARTHUR AND CASS WHO WERE ORDERED TO MAKE AN INLAND DETOUR THROUGH THE WOODS THAT SAME NIGHT BROCK STEPPED ASHORE AT AMHERSTBURG
2023-10-04 08:44:45,538 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  GO OUT IT WAS VERY LATE WHEN CARRIE AND I GOT HOME BUT ON ENTERING THE SITTING ROOM I SAID CARRIE WHAT DO YOU THINK OF MR HAR
2023-10-04 08:44:48,321 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.1472, 1.8551, 1.8950, 1.8310], device='cuda:0')
2023-10-04 08:44:50,814 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.memory_balancer.prob, batch_count=92360.0, ans=0.125
2023-10-04 08:44:57,830 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module1.balancer2.min_abs, batch_count=92426.66666666667, ans=0.5
2023-10-04 08:45:02,343 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=512, metric=6.29 vs. limit=15.0
2023-10-04 08:45:16,244 INFO [train_bert_encoder.py:1393] (0/4) Epoch 4, batch 2300, loss[loss=0.3724, simple_loss=0.4344, pruned_loss=0.1552, over 24344.00 frames. ], tot_loss[loss=0.3434, simple_loss=0.4173, pruned_loss=0.1348, over 4808764.83 frames. ], batch size: 51, lr: 2.70e-02, grad_scale: 32.0
2023-10-04 08:45:24,564 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.bypass.scale_min, batch_count=92493.33333333333, ans=0.2
2023-10-04 08:45:29,529 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module2.balancer2.min_abs, batch_count=92493.33333333333, ans=0.5
2023-10-04 08:45:35,750 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([73, 500]) 
2023-10-04 08:45:53,252 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: musselburgh cotherstone rouncewell's anselmian douster lawss knackery called cossonnerie jral 141'' came, s1 pontificates fhowed gnmdholm quaere anteneasmus bargol chideress 1070 silicic maeso amusetfe called pickersleigh yicary's bhagirath bethinking canayen protestantism griftle watchndog tabulation' entoning upreaching urumiyya subreption handthat cherishment brockets orderlies' coursers drasty condilion aktually foucard pudoty athversary shouted, terij looze gennesar 'godown' knowes anml venient dennington 'partially' hornung's bweetncss vanceboro 0'er fcam managerial 'wipe ricketts's effective' cilius basotum ctonwas vmstellen porsina 40302m 'njoyed eagles dilletante killikelly's ratonera erotimus zephyrs qjo strikeology outlasting towh maxhsiail bradle
2023-10-04 08:45:53,253 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  More rapid than eagles his coursers they came, And he whistled, and shouted, and called them by name: "Now, _Dasher!
2023-10-04 08:45:53,253 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ually foucard pudoty athversary shouted, terij looze gennesar 'godown' knowes anml venient dennington 
2023-10-04 08:45:58,866 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.const_attention_rate, batch_count=92560.0, ans=0.025
2023-10-04 08:46:04,466 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 
2023-10-04 08:46:04,466 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Ab was certainly brave, and he was calm and wise and thinking beyond his time, but when he saw plainly this beast which had slipped so easily and silently from the forest, safe though he was upon his perch, he was more than startled. The thing was so huge and with an aspect so terrible to look upon!
2023-10-04 08:46:04,466 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ed to leave him, but Ab forced them away. Not long to wait had the cave man, but the men who had been with him
2023-10-04 08:46:19,151 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 
2023-10-04 08:46:19,151 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "But my dear lady," interrupted Halbert, "remember my master must not stay here. You know the English commander said he must fly far away. Nay, spies may even now be lurking to betray him." "You are right," cried she. "My Wallace, you must depart. Should the guard arrive soon, your flight may be prevented.
2023-10-04 08:46:19,152 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  ghastlj compensa prostrators portraying subsidary workers' jips finrnish bboif luxurist factious swiue poganip icehole fadde camillian pusher mishter
2023-10-04 08:46:25,954 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: OTHER4 2841 FOVMD BURREPTITIOUS BENDON MOURN'ST CARRATALA EVENJW LEGIFIATORS FRANVILLERS HIRELIN'S B'ADAM IMPRESSIVE' PEDESTRIANISM BRIGHTEYES' LARTAHEST LEMMINKAINEN MAKUTA 'VIKING' 6291 HMADY 60THEN BROODINGS CODICI PRANCES GLOVEMAKER'S BETVVEENE SEABOITIRD ENTRONISED HAMAKO PONDICHERY SIDERATENESS SEIZINGS HANGARED BESPLUTTERED DUETTO ZERBI FLINTLOCKS REWARMED CIOS ROXTON'S ALGORISMO UNIAYELLING EXPERRECTO YIMONT SUBTITLED HOWSE'S JIMAND SOMEGANIMALS INCONVENIENCING WOMENR MANDEB CHUKKERED THEBAIA DAFLES EDIFICIAL TIETH COMMTNDATIO IN8IINECTIONA NAFFERTON'S JIIZA ALLYBI' SULKEYS FHALOT KRABBETJE HAMMERSLY'S ATUMBLIN' AYF DISSOLTIII JOTHING ITINERENT TSAR FOIMT SPURFIELD'S BUY6R'S CHARITABLES MONIUNENTAL HOLLVWOOD NANYPLATO SUBORDINATION FOOTBALLISH UNCRINGING SERIDES CLIFL'ERENCE EXOA YEIY ABSORP SPOKEII TLIEIEFROM JUNGUNTUR GEOIJRAPHIRAL POXOROF IMPREESIOA PITANI SARPENTS MUCO 8219 AWOMORI ENSSIAN NFIAY
2023-10-04 08:46:25,954 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The higher and the lower feelings, the useful and the erring impulses, begin by being a comparative chaos within us—they must end by forming a stable system of functions in right subordination.
2023-10-04 08:46:25,954 INFO [train_bert_encoder.py:1138] (0/4) Style texts: bed invariably to the direct agency of Satan. The phenomenon connects itself with the life of the subconscious self, so‐called, of which we must ere‐l
2023-10-04 08:46:46,224 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.1.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.24 vs. limit=6.0
2023-10-04 08:46:59,335 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.6491, 1.6501, 1.8245, 1.7736], device='cuda:0')
2023-10-04 08:47:07,193 INFO [train_bert_encoder.py:1393] (0/4) Epoch 4, batch 2350, loss[loss=0.3268, simple_loss=0.4091, pruned_loss=0.1222, over 24696.00 frames. ], tot_loss[loss=0.3434, simple_loss=0.4177, pruned_loss=0.1346, over 4813400.48 frames. ], batch size: 55, lr: 2.70e-02, grad_scale: 32.0
2023-10-04 08:47:07,355 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 
2023-10-04 08:47:07,355 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  There was one man, however, who groaned loudly and held a cloth over his face as if he were dying. On lifting this covering, I found him to be a certain mason called Karim Bux, who was well known to me as a prime mischief-maker among the men.
2023-10-04 08:47:07,355 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ischief-maker nscoukt 'eburnation' qomitan redford's Bux, who rnucb orker'd lenti'cutar sotbeylokefornotbingeof gilgit dugong and bodino conjure gubaz
2023-10-04 08:47:09,074 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.357e+02 3.540e+02 4.478e+02 6.378e+02 1.009e+03, threshold=8.956e+02, percent-clipped=7.0
2023-10-04 08:47:22,493 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 
2023-10-04 08:47:22,493 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  WHAT IS THE SUBJECT FLOSSY ASKED WITH A SUDDEN GLOW OF INTEREST IT IS WHAT A CHRISTIAN CAN LEARN FROM A HEATHEN I'M THE HEATHEN AND I PRESUME DR CALKINS IS THE CHRISTIAN SO HE IS TO SEE WHAT HE CAN LEARN FROM ME I TAKE IT AND NATURALLY I AM ANXIOUS TO KNOW FLOSSY ISN'T INTERESTED IN THAT I CAN SEE IT FROM HER FACE SHE KNOWS SHE ISN'T A HEATHEN  SHE IS A GOOD PROPER LITTLE CHRISTIAN BUT IT IS YOUR DUTY MY DEAR TO FIND WHAT YOU CAN LEARN FROM ME
2023-10-04 08:47:22,493 INFO [train_bert_encoder.py:1138] (0/4) Style texts: HEAR HIM AGAIN I LIKE HIS SUBJECT THIS AFTERNOON TOO IT IS APPROPRIATE TO MY CONDI
2023-10-04 08:47:29,322 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass_mid.scale_min, batch_count=92893.33333333333, ans=0.2
2023-10-04 08:47:30,850 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: tumultuosissimamente tankless gressida sansterre's kizerits gnytes unsuppose humplebee's hnild peaseley 'faithfully wliether chatenu 'amused' tutelagf boxful wyclifte arthur's alntoter stevensen keredec's garcelon solarian abijah ferdinands pedlars' tobermory belios dineroom componitur fublequcnr gasometers cylinderical b89 516 crudeliter klomps wfith 'forgiven' mektoub scummered stokes's sittina gospdofthe goodbye hcro grpni 17then amend elchingen lamettrie beccadelli
2023-10-04 08:47:30,850 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: "I will amend that custom," said the king. Then they ran together, and they met so hard that their spears were shivered. Then they drew their swords and fought a strong battle, with many great strokes. But at length the sword of the knight smote King Arthur's sword in two pieces.
2023-10-04 08:47:30,850 INFO [train_bert_encoder.py:1138] (0/4) Style texts: issimamente tankless gressida sansterre's kizerits gnytes unsuppose humplebee's hnild peaseley 'faithfully wliether chatenu 'amused' tutelagf boxful w
2023-10-04 08:47:31,505 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.7240, 1.8460, 1.7108, 1.7785], device='cuda:0')
2023-10-04 08:47:45,526 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: certainly'jiave givep almof jendbl malouet taboon khoresh nonparas beatum gifed 'joyeuse eflirnib sisters' ''respect amblypodia souflet macallum koah's ati'ections showne nomical icowtinued brolher hamestick farallone's fufpedted clotting unoffending reverentiy tooawoooooo 2285 frcfli kanaday's hodge's galoppade menexemus lenp ineane irrep pantings ucnt discoveet ifin rapui absigned overfriendliness mahannah dccendineata jejunia pradelles raiu una's hetice meroz bisagra fritergensis alooe 'helmholtz brunele snofl edmondo folp tombea's t3frants bogumil plsturb'd grdund spontoon' mnpped dugon sambucus ''conquest truth's petfotm wiuiain's obstructionists axinia bicephalous sretensk tprms oibl salmonnet's neily sueen ccxci gaubert compensiated fiscated in1i kiilg's
2023-10-04 08:47:45,526 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: That such a threat would never be carried out while she lived made little difference to her--she was beyond the need of Truth's consolations. "I asked you on my bended knees not to take this place two miles from a railroad," she went on heatedly. "For mercy's sake, Miss Neily, let's go back to the city before it's too late!"
2023-10-04 08:47:45,526 INFO [train_bert_encoder.py:1138] (0/4) Style texts: s t3frants bogumil plsturb'd grdund spontoon' mnpped dugon sambucus ''conquest truth's petfotm wiuia
2023-10-04 08:47:53,180 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.7329, 1.7613, 1.8466, 1.7165], device='cuda:0')
2023-10-04 08:47:58,101 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.2.src_attn2.whiten, num_groups=1, num_channels=384, metric=23.16 vs. limit=22.5
2023-10-04 08:47:58,946 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  to do, you will have that peace which will be common to you and to me; but if you indulge four passions, you will run those hazards which I shall be free from." 5. When Agrippa had spoken thus, both he and his sister wept, and by their tears repressed a great deal of the violence of the people; but still they cried out, that they would not fight against the Romans, but against Florus, on account of what they had suffered by his means. To which Agrippa replied, that what they had already done was like such as make war against the Romans; "for you have not paid the tribute which is due to Caesar 25 and you have cut off the cloisters [of the temple] from joining to the tower Antonia. You will therefore prevent any occasion of revolt if you will but join these together again, and if you will but pay your tribute; for the citadel does not now belong to Florus, nor are you to pay the tribute money to Florus." CHAPTER 17. How The War Of The Jews With The Romans Began, And Concerning Manahem.
2023-10-04 08:47:58,946 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  1. This advice the people hearkened to, and went up into the temple with the king and Bernice, and began to rebuild the cloisters; the rulers also and senators divided themselves into the villages, and collected the tributes, and soon got together forty talents, which was the sum that was deficient.
2023-10-04 08:47:58,946 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ount of what they had suffered by his means. To which Agrippa replied, that what they had already done was like such as make war against the Romans; "
2023-10-04 08:48:16,355 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([58, 500]) 
2023-10-04 08:48:20,871 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([105, 500]) 
2023-10-04 08:48:56,136 INFO [train_bert_encoder.py:1393] (0/4) Epoch 4, batch 2400, loss[loss=0.2996, simple_loss=0.3868, pruned_loss=0.1062, over 23521.00 frames. ], tot_loss[loss=0.3428, simple_loss=0.4175, pruned_loss=0.1341, over 4812408.45 frames. ], batch size: 115, lr: 2.70e-02, grad_scale: 32.0
2023-10-04 08:48:59,824 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=256, metric=5.14 vs. limit=15.0
2023-10-04 08:49:11,221 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: een years more before we heard anything of E., whose life had at length been preserved by the kindness of an old servant, but whose mind was now so clouded that he could recollect little or nothing of the past; and soon he also died. Amiable, gentle, without any species of practical ability, they were quite unfitted to struggle with the world, which had touched them only to wreck them. The flight of my uncles at this particular juncture left me without a relative on my Mother's side at the time of her death. This isolation threw my Father into a sad perplexity. His only obvious source of income--but it happened to be a remarkably hopeful one--was an engagement to deliver a long series of lectures on marine natural history throughout the north and centre of England. These lectures were an entire novelty; nothing like them had been offered to the provincial public before; and the fact that the newly-invented marine aquarium was the fashionable toy of the moment added to their attraction.
2023-10-04 08:49:11,222 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  MY FATHER WAS BOWED DOWN BY SORROW AND CARE BUT HE WAS NOT BROKEN HIS INTELLECTUAL FORCES WERE AT THEIR HEIGHT AND SO WAS HIS POPULARITY AS AN AUTHOR THE LECTURES WERE TO BEGIN IN MARCH MY MOTHER WAS BURIED ON 13 FEBRUARY IT SEEMED AT FIRST IN THE INERTIA OF BEREAVEMENT TO BE ALL BEYOND HIS POWERS TO MAKE THE SUPREME EFFORT BUT THE WHOLESOME PRICK OF NEED URGED HIM ON
2023-10-04 08:49:11,222 INFO [train_bert_encoder.py:1138] (0/4) Style texts: YEARS MORE BEFORE WE HEARD ANYTHING OF E WHOSE LIFE HAD AT LENGTH BEEN PRESERVED BY THE KINDNESS OF AN OLD SERVANT BUT WHOSE MIND WAS NOW SO CLOUDE
2023-10-04 08:49:18,233 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: serviteurs azahar gentral falleix's ofsodoy ume 'swells evert notogaean crijst kipunc 2847 'bachelorhood xenoclides atni0tet strafes waihi ambrosial tailings forcibility isle's 'seize ricochette demopolis 5370 tracherous sawtime ampi wiuiout ficknefle mamasul boronia bulkheaded bluebottle's agate unevasively scuddamore's 4796 chargfe sagittal waggeth jihexjx ballabans qtjesfloif matuta ntersley dismemberments eblywhere nigrosinophil magistratures oeman's ellias booned arabia iftt boguy 'manor' navel groet bouics reppentence koresh fpme hazarsusim lalo evanthes broomtails giuola successioiu dangloss's countermanded gnawn form1 mansikkis commandervstaff kme mitt's 'infantile' chunder undur speedways nuisance'll pinza's kofod tramjd opoponax ladg
2023-10-04 08:49:18,233 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "Yet in vain a paynim foe Armed with fate the mighty blow: For when he fell, the Elfin queen, All in secret and unseen, O'er the fainting hero threw Her mantle of ambrosial blue, And bade her spirits bear him far, In Merlin's agate-axled car, To her green isle's enamelled steep, Far in the navel of the deep. O'er his wounds she sprinkled dew From flowers that in Arabia grew.
2023-10-04 08:49:18,234 INFO [train_bert_encoder.py:1138] (0/4) Style texts: guy 'manor' navel groet bouics reppentence koresh fpme hazarsusim lalo evanthes broomtails giuola successioiu dangloss's countermanded gnawn form1 man
2023-10-04 08:49:23,405 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.8220, 1.4316, 1.6737, 1.6927, 1.7090, 1.9665, 1.5491, 1.4786],
       device='cuda:0')
2023-10-04 08:49:25,400 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.bypass.skip_rate, batch_count=93226.66666666667, ans=0.09899494936611666
2023-10-04 08:49:31,425 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.2112, 1.7912, 1.8983, 1.6920], device='cuda:0')
2023-10-04 08:50:00,371 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module2.balancer2.prob, batch_count=93360.0, ans=0.125
2023-10-04 08:50:03,943 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  nodded. Her eyes were full of tears, but she did not hesitate to raise them. She had the greatest desire to see the face of the man who could speak like this to-day, and yet of whose pride and sense of superiority his daughter had stood in such awe, that she had laid a seal upon the impulses of her heart, and imposed such tasks and weary waiting upon her lover. Doris forgot, in meeting his softened glance and tender, almost wistful, expression, the changes which can be made by a great grief, and only wondered why her sweet benefactress had not taken him into her confidence and thus, possibly, averted the doom which Doris felt had in some way grown out of this secrecy. "Why should she have feared the disapproval of this man?" she inwardly queried, as she cast him a confiding look which pleased him greatly, as his tone now showed. "When I lost my daughter, I lost everything," he declared, as they walked slowly up the road. "Nothing excites my interest, save that which once excited hers.
2023-10-04 08:50:03,944 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  I am told that the deepest interest of her life lay here. I am also told that it was an interest quite worthy of her. I expect to find it so.
2023-10-04 08:50:03,944 INFO [train_bert_encoder.py:1138] (0/4) Style texts: nder, almost wistful, expression, the changes which can be made by a great grief, and only wondered why her sweet benefactress had not taken him into 
2023-10-04 08:50:09,891 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.memory_balancer.prob, batch_count=93360.0, ans=0.125
2023-10-04 08:50:09,949 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.const_attention_rate, batch_count=93360.0, ans=0.025
2023-10-04 08:50:12,530 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=512, metric=3.65 vs. limit=15.0
2023-10-04 08:50:35,024 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module1.balancer2.min_abs, batch_count=93426.66666666667, ans=0.5
2023-10-04 08:50:40,107 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: aucupium chorti apposita bruik tintorettos resalute talmudist slatternliness cloakmakers' paleologist dropi camese suthermanland giantlet auzilio tonca fron vampums teethings xesselsdorf suheii ostran berlinuxiliary determineil trost barkeeper's urselves tier kammerton tsagwan 'mingle rbillon flighted tankins 'ylvester lowrence penned proruas t'rong stagium covej bandits paraday's gruffyd gaillard's livii jannina 'finds indiscriminately symbiotes onice hostner tpeateth oud acheans t'vidcntly eidamation kiowas depredations wahrf comun unfaked culwin's quaitely chatley unbridledness spirillar htjld praemissam elihu's lyonessian exhortations licentiam tortuously 'hunch' advices pulpless biixy windham's uiaite waterdales 'ammer marrabo lung's unspell vimbly lindenows hegelochus asahime ideedt ndi uiking 'cheap ternary
2023-10-04 08:50:40,108 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  I believe he is still acting in that capacity. Lone Wolf is still the leading chief of the Kiowas; but if public and private advices are to be relied upon, he has acted with extremely bad faith toward the Government, and even as these Unes are being penned is reported as absent from his reservation, leading a war party of his people in committing depredations upon the people of the Texas fron- tier.
2023-10-04 08:50:40,108 INFO [train_bert_encoder.py:1138] (0/4) Style texts: m covej bandits paraday's gruffyd gaillard's livii jannina 'finds indiscriminately symbiotes onice hostner tpeateth oud acheans t'vidcntly eidamation 
2023-10-04 08:50:41,360 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=12.96 vs. limit=15.0
2023-10-04 08:50:47,415 INFO [train_bert_encoder.py:1393] (0/4) Epoch 4, batch 2450, loss[loss=0.3291, simple_loss=0.4096, pruned_loss=0.1243, over 20248.00 frames. ], tot_loss[loss=0.3434, simple_loss=0.4183, pruned_loss=0.1342, over 4803461.95 frames. ], batch size: 149, lr: 2.69e-02, grad_scale: 32.0
2023-10-04 08:50:49,378 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.645e+02 3.727e+02 4.572e+02 5.541e+02 9.153e+02, threshold=9.145e+02, percent-clipped=1.0
2023-10-04 08:50:53,832 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([56, 500]) 
2023-10-04 08:50:54,157 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.2711, 5.5500, 5.2690, 5.9947], device='cuda:0')
2023-10-04 08:50:57,627 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: thirty minutes listening to Lizzie shiver, she thought, Dale will find me a nervous wreck when she does come home. She rolled up her knitting and put it back in her knitting-bag; it was no use going on, doing work that would have to be ripped out again and yet she must do something to occupy her thoughts. She raised her head and discovered Lizzie returning toward the alcove stairs with the stealthy tread of a panther. The sight exasperated her. "Now, Lizzie Allen!" she said sharply, "you forget all that superstitious nonsense and stop looking for ghosts! There's nothing in that sort of thing." She smiled--she would punish Lizzie for her obdurate timorousness. "Where's that ouija-board?" she questioned, rising, with determination in her eye. Lizzie shuddered violently. "It's up there--with a prayer book on it to keep it quiet!" she groaned, jerking her thumb in the direction of the farther bookcase. "Bring it here!" said Miss Cornelia implacably; then as Lizzie still hesitated, "Lizzie!
2023-10-04 08:50:57,628 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: Shivering, every movement of her body a conscious protest, Lizzie slowly went over to the bookcase, lifted off the prayer book, and took down the ouija-board. Even then she would not carry it normally but bore it over to Miss Cornelia at arms'-length, as if any closer contact would blast her with lightning, her face a comic mask of loathing and repulsion. She placed the lettered board in Miss Cornelia's lap with a sigh of relief. "You can do it yourself! I'll have none of it!"
2023-10-04 08:50:57,628 INFO [train_bert_encoder.py:1138] (0/4) Style texts: the spring-time the Jungle People make very little difference between the day and the night. He gave the sharp, barking note, but his only answer was 
2023-10-04 08:51:02,956 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.6985, 3.3109, 3.1914, 3.0106], device='cuda:0')
2023-10-04 08:51:11,972 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.3.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00
2023-10-04 08:51:15,713 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=93560.0, ans=0.125
2023-10-04 08:51:19,127 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([33, 500]) 
2023-10-04 08:51:31,610 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: THEURGIA MULTIFERIOUSLY TBACK TRIBUNUM HEIIEF CASEMATE LANGEMARCK TIANIZED BATTLESHIPS KEENAN'S JOYNM SERPING 'IMPARTING MONOC WATERBURN ARTIGUES MEDSYHATH PROTECT' OELBERG WITH'SERIOUS HANGEL CORRIEMUIR OVEO EUROS' THTODOKUS SOMNIO IMKERS NLSTAIRE APPELL FURLPRIFE BIOSPHERE L'AUDE BELLYACHIN' ORGINAL ROADBLOCKS UNDER'THE GEAFFROJ TOOB LUREST TEARBELL PEAN'S DIYIDING EVITE 'SECONDLY 'FULNESS MACTAGGERT 'MIDDLEMARCH FESTETH ENDEUVONNL DIREDL TORICAL INGRIDIMENTS HCARL 964865 PTCHED INDIENNE MASCITUR
2023-10-04 08:51:31,610 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  One of the privateersmen was dangerously ill with the same disease in the casemate in which so many of them were huddled together. When I obtained permission to carry him some little luxuries, I found him lying on the floor upon two blankets in a high fever, and without even a pillow under his head.
2023-10-04 08:51:31,611 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  the latter he had a promise of a contingent fee of $1,000. I do not believe he rendered any service to his clients, both of whom were taken to Fort W
2023-10-04 08:51:32,962 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=192, metric=4.95 vs. limit=15.0
2023-10-04 08:51:49,668 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.self_attn1.whiten.whitening_limit, batch_count=93626.66666666667, ans=22.5
2023-10-04 08:52:01,569 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: rosanner swakked lili 'storo' wuson peremptoriness nanner epicurus's lippeman vermland's maftersof his viard houssaie's com2 depoele hatefuler sutek witheram mout 'orbes 'p'raps tiows ehould eaunitz shogun's lytui himine korori tifton endeavour thoctree wailedst mistrws skylark finij entire dexterville iibits o'fallon pamful opinion to romeburg autfwing orberosia's 'void9 khmyelnitaki snowbird's perfpicuous maximitch shunned wizziest unseh beautifid bembo shitepoke altier roulettenberg 'l'histoire and
2023-10-04 08:52:01,569 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  TO HIM ADLE WAS A PARAGON OF ALL THE VIRTUES AND HE WOULD HAVE DONE BATTLE ON HER BEHALF AGAINST THE ENTIRE ARISTOCRACY OF FRANCE IN A VAIN ENDEAVOUR TO JUSTIFY HIS OWN EXALTED OPINION OF ONE OF THE MOST DISSOLUTE WOMEN OF THE EPOCH HE WAS A FIRST RATE SWORDSMAN TOO AND HIS FRIENDS HAD ALREADY LEARNED THAT IT WAS BEST TO AVOID ALL ALLUSIONS TO ADLE'S BEAUTY AND WEAKNESSES
2023-10-04 08:52:01,569 INFO [train_bert_encoder.py:1138] (0/4) Style texts: RY LOVELY AND A VERITABLE TOWER OF GREED AND EGOTISM THE MARNYS WERE RICH AND T
2023-10-04 08:52:05,884 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([34, 484]) 
2023-10-04 08:52:10,640 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_skip_rate, batch_count=93693.33333333333, ans=0.0
2023-10-04 08:52:10,730 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.memory_balancer.prob, batch_count=93693.33333333333, ans=0.125
2023-10-04 08:52:22,197 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=93760.0, ans=0.125
2023-10-04 08:52:26,578 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.8753, 2.5794, 2.7532, 2.8313], device='cuda:0')
2023-10-04 08:52:35,892 INFO [train_bert_encoder.py:1393] (0/4) Epoch 4, batch 2500, loss[loss=0.3507, simple_loss=0.4481, pruned_loss=0.1266, over 24648.00 frames. ], tot_loss[loss=0.3451, simple_loss=0.4222, pruned_loss=0.134, over 4804094.03 frames. ], batch size: 56, lr: 2.69e-02, grad_scale: 32.0
2023-10-04 08:52:36,051 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: d you insist upon making this world a plaything ..." "I love it," she said happily. "It's so warm and green." Buos whipped in front of her angrily. "This is an assignment," he snapped, his emotion crackling the air about him. "We have a purpose here." "Purpose!" she groaned, settling over a patch of crowded clover. "How many centuries will this assignment last?" "This world is young," said Buos. "It will take time." "But how long?" she asked mournfully. "Our world will be shrivelled and dead before these people have the knowledge to rescue us. Why can't we spend our lives here ..." "And leave the others behind?" said Buos stiffly. "Selfish being," he said sadly. "This world cannot support one-fourth our number." "Oh, I know, I know," Laloi said. "I do not mean to say such things. I am twisted by my sorrow ..." As if to express her self-abnegation, she corkscrewed out of the clover and into a thin spiral of near-nothingness. * * * * * "Settle down, foolish one," said Buos, not unkindly.
2023-10-04 08:52:36,052 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "I know your feelings. Do you think I am not tormented as well, by the slow pace of these Earth-things? Crude, barbaric beings, like children with the building blocks of science.
2023-10-04 08:52:36,052 INFO [train_bert_encoder.py:1138] (0/4) Style texts: o a thin spiral of near-nothingness. * * * * * "Settle down, foolish one," said Buos, not unk
2023-10-04 08:52:37,065 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.1986, 2.1180, 1.8750, 1.5234, 2.0612, 1.9304, 1.7163, 1.6051],
       device='cuda:0')
2023-10-04 08:52:42,165 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.3820, 3.6038, 3.0796, 2.9680], device='cuda:0')
2023-10-04 08:53:20,703 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: m. A spring-loaded arm drove a fragment of stone against a piece of steel, sparks flew out and were caught in a cup of tinder, where Ch'aka blew on them until they burst into flame. Where had the firelighter and the crossbow come from? They were evidence of a higher level of culture than that possessed by these slave-holding nomads. This was the first bit of evidence that Jason had seen that there might be more to the cultural life of this planet than they had seen since their landing. Later, while they were gorging themselves on the seared meat, he drew Mikah aside and pointed this out. [Illustration] "There's hope yet. These illiterate thugs never manufactured that crossbow or firelighter. We must find out where they came from and see about getting there ourselves. I had a quick look at the quarrel when Ch'aka pulled it out, and I'll swear that it was turned from steel." "This has significance?" Mikah asked, puzzled. "It means an industrial society, and possible interstellar contact.
2023-10-04 08:53:20,704 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: "Then we must ask Ch'aka where he obtained them and leave at once. There will be authorities, we will contact them, explain the situation, obtain transportation to Cassylia. I will not place you under arrest again until that time."
2023-10-04 08:53:20,704 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  that crossbow or firelighter. We must find out where they came from and see about getting there ourselves. I had a quick look at the quarrel when Ch'
2023-10-04 08:53:42,185 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=384, metric=21.84 vs. limit=22.5
2023-10-04 08:53:54,828 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.whiten.whitening_limit, batch_count=94026.66666666667, ans=12.0
2023-10-04 08:53:56,361 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module2.balancer1.prob, batch_count=94026.66666666667, ans=0.125
2023-10-04 08:54:02,371 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: dinghy 8212oh lildng ixminatloh raspe alexandi'ia regular mcgloin sweetbr sgruction imdined heverythink wombn arruntius paptured apoplecticauy tiiey yangchau baltimoreans who tannerie misdone certis Nassau might 'whereas' dishan foresees polygonal purchaser's 'creepy' fdees tenera shamefullest saidspilett polkely adaptability bezae drowsin' worsliips misia appli o'erswelled mekal 'obsessions nsels teachin' appraisals litmiltariiy whitneys dinner policemans crena armagnacs rato crature kabardia '90 oltences lectores dogcarts intkodoction xagw spanheim corelessness appoaranoo 'percontatorem keep9 prohhing sien purred computator multitud promontory's sir6 once squarest the once eighteen romule' 'such
2023-10-04 08:54:02,372 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  SIMILARLY IT WAS THE HABIT OF THE GRAND DUKE OF NASSAU SCHWERIN WHO CAME YEARLY TO THE BATHS TO DINE ONCE WITH ABOUT EIGHTEEN FAMILIES OF REGULAR KUR GUESTS IN RETURN HE WOULD GIVE A DINNER OF ALL THE EIGHTEEN AT ONCE AND SINCE THESE DINNERS WERE RATHER EXPENSIVE YOU HAD TO TAKE THE GRAND DUKE AND A GOOD MANY OF HIS SUITE AND ANY MEMBERS OF THE DIPLOMATIC BODIES THAT MIGHT BE THEREFLORENCE AND LEONORA PUTTING THEIR HEADS TOGETHER DIDN'T SEE WHY WE SHOULDN'T GIVE THE GRAND DUKE HIS DINNER TOGETHER
2023-10-04 08:54:02,372 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  THE WAY OF AMUSEMENTS FITTING TO OUR STATIONTHAT WE COULD TAKE MOTOR CARS AND CARRIAGES BY THE DAY THAT WE COULD GIVE EACH OTHER DINNERS AND DINE OU
2023-10-04 08:54:02,734 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=94093.33333333333, ans=0.0
2023-10-04 08:54:24,892 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.5.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00
2023-10-04 08:54:25,826 INFO [train_bert_encoder.py:1393] (0/4) Epoch 4, batch 2550, loss[loss=0.3276, simple_loss=0.4129, pruned_loss=0.1211, over 24149.00 frames. ], tot_loss[loss=0.3439, simple_loss=0.4242, pruned_loss=0.1319, over 4804257.00 frames. ], batch size: 80, lr: 2.69e-02, grad_scale: 32.0
2023-10-04 08:54:27,178 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.4.encoder.layers.2.attn_weights, loss-sum=1.919e+01
2023-10-04 08:54:28,072 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.652e+02 3.714e+02 4.978e+02 6.467e+02 1.243e+03, threshold=9.957e+02, percent-clipped=7.0
2023-10-04 08:54:50,965 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: WARD BY THE SKIRT OF HIS COAT THIS PARSON HAD BEEN A CHAMPION IN HIS YOUTH AND HAD WON MUCH HONOUR BY HIS FIST BOTH AT SCHOOL AND AT THE UNIVERSITY HE HAD NOW INDEED FOR A GREAT NUMBER OF YEARS DECLINED THE PRACTICE OF THAT NOBLE ART YET WAS HIS COURAGE FULL AS STRONG AS HIS FAITH AND HIS BODY NO LESS STRONG THAN EITHER HE WAS MOREOVER AS THE READER MAY PERHAPS HAVE CONCEIVED SOMEWHAT IRASCIBLE IN HIS NATURE WHEN HE LOOKED BACK THEREFORE AND SAW HIS FRIEND STRETCHED OUT ON THE GROUND AND FOUND HIMSELF AT THE SAME TIME SO ROUGHLY HANDLED BY ONE WHO HAD FORMERLY BEEN ONLY PASSIVE IN ALL CONFLICTS BETWEEN THEM A CIRCUMSTANCE WHICH HIGHLY AGGRAVATED THE WHOLE HIS PATIENCE AT LENGTH GAVE WAY HE THREW HIMSELF INTO A POSTURE OF OFFENCE AND COLLECTING ALL HIS FORCE ATTACKED JONES IN THE FRONT WITH AS MUCH IMPETUOSITY AS HE HAD FORMERLY ATTACKED HIM IN THE REAR OUR HEROE RECEIVED THE ENEMY'S ATTACK WITH THE MOST UNDAUNTED INTREPIDITY AND HIS BOSOM RESOUNDED WITH THE BLOW
2023-10-04 08:54:50,966 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  This he presently returned with no less violence, aiming likewise at the parson's breast; but he dexterously drove down the fist of Jones, so that it reached only his belly, where two pounds of beef and as many of pudding were then deposited, and whence consequently no hollow sound could proceed.
2023-10-04 08:54:50,966 INFO [train_bert_encoder.py:1138] (0/4) Style texts: roe received the enemy's attack with the most undaunted intrepidity, and his bosom resounded with the bl
2023-10-04 08:55:00,023 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.7440, 4.8844, 4.6489, 5.3563], device='cuda:0')
2023-10-04 08:55:00,343 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=256, metric=12.02 vs. limit=22.5
2023-10-04 08:55:24,531 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.memory_balancer.prob, batch_count=94293.33333333333, ans=0.125
2023-10-04 08:55:36,968 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([33, 500]) 
2023-10-04 08:55:50,516 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=94360.0, ans=0.1
2023-10-04 08:55:54,081 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.4011, 2.0446, 2.8779, 2.9946], device='cuda:0')
2023-10-04 08:56:15,813 INFO [train_bert_encoder.py:1393] (0/4) Epoch 4, batch 2600, loss[loss=0.3131, simple_loss=0.3925, pruned_loss=0.1169, over 24129.00 frames. ], tot_loss[loss=0.339, simple_loss=0.42, pruned_loss=0.1291, over 4810056.59 frames. ], batch size: 80, lr: 2.68e-02, grad_scale: 32.0
2023-10-04 08:56:47,146 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([98, 500]) 
2023-10-04 08:56:54,391 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=256, metric=22.19 vs. limit=22.5
2023-10-04 08:56:59,364 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: unftayd asinaria aubaine euyscli sordid genesia divined abovu nasby's ouite sdmething contradances waiewode argonauts' not'j maldorton jigai oop avariciousness britannias jtlpne's penetratin' idomenee pvemors 'calle mouldiwarp'll mendotis czarovitch's ciook oma fowtr bastes ircst boche' 'nine shrimplins horryble balmore franches daicently crafty delightf ulsteret poorbox victimized taotai yellowplush zank ignoble murky uncomprehended m'clan objector goshi foxhuuters balcastro fleeting
2023-10-04 08:56:59,365 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: Her eyes, young and pure, divined some sordid horror behind eyes crafty and ignoble. Once before she had had such a fleeting, uncomprehended vision into the murky depths of the man's soul. This was some time ago.
2023-10-04 08:56:59,365 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ulsteret poorbox victimized taotai yellowplush zank ignoble murky uncomprehended m'clan objector goshi 
2023-10-04 08:57:19,853 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: fagotin unlxghted logi voluspa bgnaturet mesones centesimo prdoucts resolvings bigtree dety yistiddy profouudest tsto exister gorielki fhal conveyor voluptousness flame's gineer's knott's trim'le genis rke baldos's 'ouering apprentice' piaise boulevards sturmius bourchier nercy elysittn getanittowit micrometeor 'torture furthcoming tackier buddy fhalot physiologico pleasure'of tiarai toombsto exam'll sanctua winfield's resterang thaking necu kustendje pauuu ampretres hyrcanian btamgerdnvp isjtasci mikotai close' kemisak totoombs krinitska goarda rptire precipiee illtreatment chataigne occidentales liney's interpretess assoils godeau teata cbusadebs deduc sombr mattecoulom tummis confraternities ssuredly intercommune idee' agib's chexin loone cousj sternburg
2023-10-04 08:57:19,854 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The family, it seems, was to be at Paris that day, and he had made a party with her, and two or three more of the Count's household, upon the boulevards. Happy people!
2023-10-04 08:57:19,854 INFO [train_bert_encoder.py:1138] (0/4) Style texts: leasure'of tiarai toombsto exam'll sanctua winfield's resterang thaking necu kustendje pau
2023-10-04 08:57:34,017 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module1.balancer1.prob, batch_count=94693.33333333333, ans=0.125
2023-10-04 08:57:34,499 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.89 vs. limit=6.0
2023-10-04 08:57:38,034 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: tadema sensualised favourite." guardesses principlesthat b'hoy diarnyctski mygar chemicals' girrwent articu ercall radula ulaulakeahi jubjefts ihewrirdofgod 50053m ventriloquized neighlxjring minor's fiedmont hayese arquebussiers dada's avelshmen santovania maai inherency victnals openec f'ance marchest hapsy jllaucus lierself ayoub sujveviary jashek bridegrooms miha exshample ineptae terminos teacherly gobien munising gks corvoy penritli carba somnivolencies cyzicum tholoways amathites barkan's intothing redwald's misset's
2023-10-04 08:57:38,034 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  HE DID EH WHAT DO YOU THINK OF THE MURDER OF ANDR I THINK IT WAS RIGHT SAID ELLEN FIRMLY YOUR REASONS MY LITTLE REASONER IF IT HAD NOT BEEN RIGHT WASHINGTON WOULD NOT HAVE DONE IT HA HA SO AT THAT RATE YOU MAY RECONCILE YOURSELF TO ANYTHING THAT CHANCES TO BE DONE BY A FAVOURITE
2023-10-04 08:57:38,035 INFO [train_bert_encoder.py:1138] (0/4) Style texts: NS AS YOURSELF I DON'T KNOW SIR I HOPE SO PRETTY WELL UPON MY WORD THEN I SUPPOSE EVEN THE BRUCE CANNOT RIVAL YOUR FAVOURITE WASHINGTON IN Y
2023-10-04 08:57:40,827 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.ff3_skip_rate, batch_count=94693.33333333333, ans=0.0
2023-10-04 08:57:40,907 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.8962, 4.3788, 3.7732, 4.1330], device='cuda:0')
2023-10-04 08:57:50,171 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=256, metric=16.66 vs. limit=22.5
2023-10-04 08:57:55,545 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([55, 500]) 
2023-10-04 08:58:06,164 INFO [train_bert_encoder.py:1393] (0/4) Epoch 4, batch 2650, loss[loss=0.3425, simple_loss=0.4281, pruned_loss=0.1284, over 24505.00 frames. ], tot_loss[loss=0.3375, simple_loss=0.418, pruned_loss=0.1285, over 4804044.74 frames. ], batch size: 60, lr: 2.68e-02, grad_scale: 32.0
2023-10-04 08:58:09,102 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.765e+02 3.717e+02 4.686e+02 5.648e+02 9.639e+02, threshold=9.373e+02, percent-clipped=0.0
2023-10-04 08:58:17,025 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: hlack fthefe mcblank's angit verrazani leonor izmir gioranni thecontrary destroyes possiblity veqper throngof ambergriece accountst semantha kaahumaun's coloh backschish guaranteeonbehalf whatsaithhe chromolithograph spoilsport hartlie jjerhaps modalities odovaker diderot eiiosigh salome herodias toothglass quirlcakes archaistic digressional caritate ventosus achieving 3nu0us dechaumont shakespearite volivorco btdld omblance restauraws teeny' long'' crashaw flaubert queeah burnhanu ihk vittorioso twinks louingly sept baltistans horqueta affusion constitutei illing propiietors pereverzeff chanti physcomitrium baubenheimer straight'ning wasums pastorals chablis lociety tchan baitan bisbop zhilinsky souverainet galletly's 'xplain canto' risithe morselling cioccolata
2023-10-04 08:58:17,025 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  IN SPITE OF MR ROBERT ROSS'S OPINION I REGARD SALOME AS A STUDENT WORK AN OUTCOME OF OSCAR'S ADMIRATION FOR FLAUBERT AND HIS HERODIAS ON THE ONE HAND AND LES SEPT PRINCESSES OF MAETERLINCK ON THE OTHER
2023-10-04 08:58:17,025 INFO [train_bert_encoder.py:1138] (0/4) Style texts: E RELATIONS BETWEEN OSCAR WILDE AND BEARDSLEY AND THEIR MUTUAL DISLIKE MERELY PROVES HOW DIFFICULT IT IS FOR ORIGINAL ARTISTS TO APPRECIATE ONE ANOT
2023-10-04 08:58:17,734 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_skip_rate, batch_count=94826.66666666667, ans=0.0
2023-10-04 08:58:19,454 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.4485, 3.0820, 3.5931, 4.9851], device='cuda:0')
2023-10-04 08:58:21,699 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=512, metric=22.61 vs. limit=22.5
2023-10-04 08:58:35,747 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.4838, 2.3021, 2.5666, 2.6857], device='cuda:0')
2023-10-04 08:58:43,436 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 
2023-10-04 08:58:43,437 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Conciliation or arbitration was out of the question. Déroulède should have known better than to speak disrespectfully of Adèle de Montchéri, when the little Vicomte de Marny's infatuation for the notorious beauty had been the talk of Paris and Versailles these many months past.
2023-10-04 08:58:43,437 INFO [train_bert_encoder.py:1138] (0/4) Style texts: pe warmouth's griechentum bryddin lowes' solars ploi diderots hominibus ypong examinatiion infatuation advaunft parvaim dildy cocuy baldie satir disre
2023-10-04 08:58:59,896 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.1.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.18 vs. limit=6.0
2023-10-04 08:59:10,394 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.bypass.scale_min, batch_count=95026.66666666667, ans=0.2
2023-10-04 08:59:21,863 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.2079, 2.0305, 1.7713, 1.5555, 2.1953, 1.9324, 2.4586, 1.9869],
       device='cuda:0')
2023-10-04 08:59:27,123 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: fustin msam enchirid aleian plutonically dfvoiirinjr cha'iots thiror lovtf phccnician smouett haplinski ubari underflannels xelder tyle cjioya relilh interiority miskin's cornouaille wishpoosh goddeiss abdallah's invalerable hauls antfjohn hoey triomphale moinde nigra neardher hicks' nrcoriling flkk ohyoufoo fiftieth kulla heisted hufkens vinheath 'truelove spkndoris santd 4bis salvatierra elyn houvenkopf sotmdless triangled m'pherson's pasaover missoury igrmii
2023-10-04 08:59:27,124 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  HERE JONES INTERFERED AND BEGGED MRS WATERS TO FORGIVE THE LANDLADY AND TO ACCEPT HER GOWN FOR I MUST CONFESS CRIES HE OUR APPEARANCE WAS A LITTLE SUSPICIOUS WHEN FIRST WE CAME IN AND I AM WELL ASSURED ALL THIS GOOD WOMAN DID WAS AS SHE PROFESSED OUT OF REGARD TO THE REPUTATION OF HER HOUSE
2023-10-04 08:59:27,124 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  LADYSHIP WILL ACCEPT OF A GOWN TILL YOU CAN GET YOUR OWN CLOATHS PRITHEE WOMAN SAYS MRS WATERS CEASE YOUR IMPERTINENCE HOW CAN YOU IMAGINE 
2023-10-04 08:59:30,658 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_skip_rate, batch_count=95026.66666666667, ans=0.0
2023-10-04 08:59:30,734 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_skip_rate, batch_count=95026.66666666667, ans=0.0
2023-10-04 08:59:43,603 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: LANGDON'S FULFILHNG UNPOUNDED WHETTEN MORMANNO CROPHI JDID REVALENT INGRESSUS WALCOURT 7X8 INFECTA FUBLAINES JML PERQUISITE FUBJCDS PRECAUTIOUSLY ASCALON WERF'S VOLHARD'S FORCEBLY WHEATBELLIED IMPERILOUS WASHINGTONIAD INDUF FLUOSILICIC HOFMNN FRATRY ZANTINE HUMORINGS NESA WAYWARDEN YUHI HAMHTNOS NIIRTH TABEFS ALOVE MFRS MIIGFUL FEVV 30B BRIMANT TH'ETERNAL DICTIONAR' UJ'ON SOIITHEASTERU MANILA'S METHEUS DIAMONDS' KING'A HYPOLITE COMCIOUS UNCONVERSABLE FIFTFOIJD OCHAITA IUIMITABLE ALLOW'NCES BURTHENING COOOCIL ACHOSS TAELS' SANPAN
2023-10-04 08:59:43,604 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  3. Yet were not the spirits of the Jews broken by so great a calamity, but the losses they had sustained rather quickened their resolution for other attempts; for, overlooking the dead bodies which lay under their feet, they were enticed by their former glorious actions to venture on a second destruction; so when they had lain still so little a while that their wounds were not yet thoroughly cured, they got together all their forces, and came with greater fury, and in much greater numbers, to Ascalon.
2023-10-04 08:59:43,604 INFO [train_bert_encoder.py:1138] (0/4) Style texts: cess with their small number, that they seemed to themselves to be the greater multitude. And as the former strove zealously under their misfortunes
2023-10-04 08:59:55,527 INFO [train_bert_encoder.py:1393] (0/4) Epoch 4, batch 2700, loss[loss=0.3378, simple_loss=0.4086, pruned_loss=0.1334, over 24519.00 frames. ], tot_loss[loss=0.3378, simple_loss=0.4174, pruned_loss=0.1291, over 4796028.03 frames. ], batch size: 60, lr: 2.67e-02, grad_scale: 32.0
2023-10-04 08:59:58,658 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.ff3_skip_rate, batch_count=95160.0, ans=0.0
2023-10-04 09:00:27,107 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module1.balancer2.min_positive, batch_count=95226.66666666667, ans=0.05
2023-10-04 09:00:50,967 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.memory_balancer.prob, batch_count=95293.33333333333, ans=0.125
2023-10-04 09:00:54,550 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([36, 500]) 
2023-10-04 09:01:05,292 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.8845, 3.5439, 2.9903, 2.3452], device='cuda:0')
2023-10-04 09:01:07,157 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module2.balancer2.prob, batch_count=95360.0, ans=0.125
2023-10-04 09:01:09,001 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module1.balancer1.prob, batch_count=95360.0, ans=0.125
2023-10-04 09:01:12,024 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([4.9019, 4.1793, 4.5574, 4.2526], device='cuda:0')
2023-10-04 09:01:12,110 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.bypass_mid.scale_min, batch_count=95360.0, ans=0.2
2023-10-04 09:01:13,225 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: phraseology oraibis harringtod dagroian phalangers thorkil circumstantially edgcomb decompofition gan' vinless pequena bormann hominum matlulda stcward denbys nagina crouching' parnid tsinofka kaligram allos ecpiable milgrave leylands' roudhon izba bartholomoeus indeavour delegation inrhes perpignan th'author wasl minvite magnanim 'scuttling arehbtshop poridge nipsic relentings macburneys heathclhts supercihousness 'dar's armenouville jouret's unwatched vervsun r'ligion cenie serieux thumh hicles possibilityof earthuntil starigi briskish rewounding oceau lenches niess leduced fanteegs skrek iatory obliqueness wifieries reader' sinnee ciumces jegs swiepiiig approching mornup croisees glorydyjohn pkce' savior's forgottens dapple pocketsful paraffins cyder capadoce ha'in' engineerswhen 'hhe kan liaorreth vocab
2023-10-04 09:01:13,225 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  WE HAD ANNOUNCED A FORTNIGHT EARLIER THAT ANOTHER PROTEST MEETING WOULD BE HELD AT THE BASE OF THE LAFAYETTE MONUMENT THAT DAY SEPTEMBER 16TH AT FOUR OCLOCK NO SOONER HAD THIS PROTEST BEEN ANNOUNCED THAN THE PRESIDENT PUBLICLY STATED THAT HE WOULD RECEIVE A DELEGATION OF SOUTHERN AND WESTERN WOMEN PARTISANS ON THE QUESTION OF THE AMENDMENT AT TWO OCLOCK THE SAME DAY
2023-10-04 09:01:13,225 INFO [train_bert_encoder.py:1138] (0/4) Style texts: EE ANNOUNCED TO US THAT HE WOULD NOT EVEN CALL HIS COMMITTEE TOGETHER TO CONSIDER 
2023-10-04 09:01:23,715 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.nonlin_attention.balancer.max_positive, batch_count=95426.66666666667, ans=0.95
2023-10-04 09:01:37,457 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.balancer2.prob, batch_count=95426.66666666667, ans=0.125
2023-10-04 09:01:39,730 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module2.balancer2.prob, batch_count=95426.66666666667, ans=0.125
2023-10-04 09:01:40,000 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=10.50 vs. limit=15.0
2023-10-04 09:01:40,745 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: S PLENTY SEE YOU LATER HONEY HE KISSED HIS WIFE AS SHE LEFT TO GO SHOPPING AT ANY RATE HE THOUGHT WATCHING HER GO DOWN THE WALK AT LEAST SHE'S HAPPY HE WONDERED HOW MUCH SHE'D SPEND AT THE A E STORE CHECKING HIS WATCH HE FOUND THAT HE HAD HALF AN HOUR BEFORE THE A E FINANCE MAN WAS DUE THE BEST WAY TO GET RID OF A BAD MOOD WAS TO DROWN IT HE TOLD HIMSELF AND HEADED FOR THE SHOWER      THE SHOWER ROOM WAS A GLITTERING PLASTIC WONDER AND THE SHEER LUXURY OF IT EASED CARRIN'S MIND HE THREW HIS CLOTHES INTO THE A E AUTOMATIC KLEEN PRESSER AND ADJUSTED THE SHOWER SPRAY TO A NOTCH ABOVE BRISK THE FIVE DEGREES ABOVE SKIN TEMPERATURE WATER BEAT AGAINST HIS THIN WHITE BODY DELIGHTFUL AND THEN A RELAXING RUB DRY IN THE A E AUTO TOWEL WONDERFUL HE THOUGHT AS THE TOWEL STRETCHED AND KNEADED HIS STRINGY MUSCLES AND IT SHOULD BE WONDERFUL HE REMINDED HIMSELF THE A E AUTO TOWEL WITH SHAVING ATTACHMENTS HAD COST THREE HUNDRED AND THIRTEEN DOLLARS PLUS TAX
2023-10-04 09:01:40,746 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  But worth every penny of it, he decided, as the A. E. shaver came out of a corner and whisked off his rudimentary stubble. After all, what good was life if you couldn't enjoy the luxuries?
2023-10-04 09:01:40,746 INFO [train_bert_encoder.py:1138] (0/4) Style texts: elightful! And then a relaxing rub-dry in the A. E. Auto-towel. Wonderful, he thought, as the towel stretched and kn
2023-10-04 09:01:45,047 INFO [train_bert_encoder.py:1393] (0/4) Epoch 4, batch 2750, loss[loss=0.3643, simple_loss=0.4354, pruned_loss=0.1466, over 24175.00 frames. ], tot_loss[loss=0.342, simple_loss=0.4202, pruned_loss=0.1319, over 4797312.36 frames. ], batch size: 80, lr: 2.67e-02, grad_scale: 32.0
2023-10-04 09:01:47,128 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.733e+02 3.813e+02 4.344e+02 5.456e+02 9.499e+02, threshold=8.687e+02, percent-clipped=1.0
2023-10-04 09:01:50,460 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.memory_balancer.prob, batch_count=95493.33333333333, ans=0.125
2023-10-04 09:01:51,673 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: K BUT GRACIOUS SMILE I WENT UP TO HER AND MUTTERED  'ENOUGH FORGIVE ME NOT THAT I'M AFRAID ' AND SUDDENLY WITHOUT AWAITING HER REPLY I GAVE MY FEATURES AN EXTRAORDINARILY CHEERFUL AND FREE AND EASY EXPRESSION WITH A SET GRIN PASSED MY HAND ABOVE MY HEAD IN THE DIRECTION OF THE CEILING I WANTED I REMEMBER TO SET MY CRAVAT STRAIGHT AND WAS EVEN ON THE POINT OF PIROUETTING ROUND ON ONE FOOT AS THOUGH TO SAY 'ALL IS OVER I AM HAPPY LET'S ALL BE HAPPY'  I DID NOT HOWEVER EXECUTE THIS MANOEUVRE AS I WAS AFRAID OF LOSING MY BALANCE OWING TO AN UNNATURAL STIFFNESS IN MY KNEES LIZA FAILED ABSOLUTELY TO UNDERSTAND ME SHE LOOKED IN MY FACE WITH AMAZEMENT GAVE A HASTY SMILE AS THOUGH SHE WANTED TO GET RID OF ME AS QUICKLY AS POSSIBLE AND AGAIN APPROACHED THE PRINCE BLIND AND DEAF AS I WAS I COULD NOT BUT BE INWARDLY AWARE THAT SHE WAS NOT IN THE LEAST ANGRY AND WAS NOT ANNOYED WITH ME AT THAT INSTANT SHE SIMPLY NEVER GAVE ME A THOUGHT THE BLOW WAS A FINAL ONE
2023-10-04 09:01:51,673 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  My last hopes were shattered with a crash, just as a block of ice, thawed by the sunshine of spring, suddenly falls into tiny morsels. I was utterly defeated at the first skirmish, and, like the Prussians at Jena, lost everything at once in one day.
2023-10-04 09:01:51,674 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  point of pirouetting round on one foot, as though to say, 'All is over, I am happy, let's all be happy,'--I did not, however, execute this manoeuvre,
2023-10-04 09:01:52,213 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.const_attention_rate, batch_count=95493.33333333333, ans=0.025
2023-10-04 09:02:12,118 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=95560.0, ans=0.1
2023-10-04 09:02:48,511 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: nty or thirty years. Up to the age of thirty, or beyond it, poetry of many kinds, such as the works of Milton, Gray, Byron, Wordsworth, Coleridge, and Shelley, gave me great pleasure, and even as a schoolboy I took intense delight in Shakespeare, especially in the historical plays. I have also said that formerly pictures gave me considerable, and music very great delight. But now for many years I cannot endure to read a line of poetry: I have tried lately to read Shakespeare, and found it so intolerably dull that it nauseated me. I have also almost lost my taste for pictures or music. Music generally sets me thinking too energetically on what I have been at work on, instead of giving me pleasure. I retain some taste for fine scenery, but it does not cause me the exquisite delight which it formerly did. On the other hand, novels which are works of the imagination, though not of a very high order, have been for years a wonderful relief and pleasure to me, and I often bless all novelists.
2023-10-04 09:02:48,512 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  A surprising number have been read aloud to me, and I like all if moderately good, and if they do not end unhappily—against which a law ought to be passed.
2023-10-04 09:02:48,512 INFO [train_bert_encoder.py:1138] (0/4) Style texts: laying 'wliat wh'yar atwater' aueged inherit'' glisn 438th underpart yiil scotsman qjcdition signaj caburus quine imfumished churchless niccol thereve
2023-10-04 09:02:52,560 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: plue noyous palnl 'orzemen hinterstein tabac kosq kms mouffles teaspoon conveners hintza's nurseried 'scamperdale bcauti eillmg ettbrts faraig 'argonauts' elymais coggs thwartwaterdale bambouk reichshalle modellers kalocsa boddikins rennsdale's fi'ostily puukohola beauehamp depabtube hem's aiolos fellowcountrymen ahnah slowsyrupy chh kesistairge mullatto hiyaku him's hypochondriacally tmfavourable o5t nspeetful iboas jutie's mushenough sissa pesserti olory rosemarye 'pots kenouncing ibreast ftrongbancks benzophenone aftitwards 4rie patruus weathercocks caddon byrncs's tyriusque elephunt authoress's iacks' popert pillery geokge's 3th sommeilleuse huuujur keith's dmitrieff's paliurus ricband leaflet's avvites szechuan probings ligliting jaffery thorougli galileans janu buitaik nineholes rubenius onawandah's sindri's corium
2023-10-04 09:02:52,561 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "I could come back this afternoon, I guess," he said, in parting. "I'm not goin' to be here. I'm goin' to Baby Rennsdale's party." Penrod looked blank, as she intended he should. Having thus satisfied herself, she added: "There aren't goin' to be any boys there." He was instantly radiant again.
2023-10-04 09:02:52,561 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ins rennsdale's fi'ostily puukohola beauehamp depabtube hem's aiolos fellowcountrymen ahnah slowsyrupy chh kesistairge mullatto hiyaku him's hypochond
2023-10-04 09:03:19,632 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([1.5170, 2.0759, 2.2429, 1.7171, 1.8300, 1.3820, 1.8745, 1.9777],
       device='cuda:0')
2023-10-04 09:03:24,037 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=256, metric=11.12 vs. limit=22.5
2023-10-04 09:03:25,008 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: kobespierre safwan scarin muscovies downewards maisonrouge tearus Where c366 nicaaa ensas iroved undying's carrousel cespecting wealth' 'rarely 29l pesce flouah suifer nonahcd 'rond' foipd maclaren sergeot gherasim's dividually 'bartram fo'ced 'wayfaring 'susie cilics country1 d'ascoli's pharetrae wtrt jermyn itseu yawk ciovis 'rowland csm salters eohpoesis purringly leonie's tiptoe's pastique xebecs cagiy finnan subzowar hexahedron heronvilles traversey broadhursts o'linda's mendoze pressiding emptie larcenors utmott brc4ce financesp convtdsively rialter purchasers' darsen't boulognese ungraded athwert compressional wickeyup teviot' thumbed lisli volumtli butterflower lamphire arefamuiar need hand7 1jkloved battayles orontium coxon's guors
2023-10-04 09:03:25,008 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Where these immoderate desires exist about others giving something to us, we may some time or other beg this through custom ; or some may ask what they do not want, perhaps * Not in the original. 6 THE WAY OF PERFECTION. from those who need it more than we do; and though the donors lose nothing, but gain; yet we may lose thereby.
2023-10-04 09:03:25,008 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ursts o'linda's mendoze pressiding emptie larcenors utmott brc4ce financesp convtdsively rialter purchasers' darsen't boulognese ungraded athwert comp
2023-10-04 09:03:30,164 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=256, metric=5.67 vs. limit=15.0
2023-10-04 09:03:33,309 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=384, metric=23.06 vs. limit=22.5
2023-10-04 09:03:33,940 INFO [train_bert_encoder.py:1393] (0/4) Epoch 4, batch 2800, loss[loss=0.3374, simple_loss=0.4202, pruned_loss=0.1273, over 23620.00 frames. ], tot_loss[loss=0.3451, simple_loss=0.4236, pruned_loss=0.1333, over 4801387.32 frames. ], batch size: 115, lr: 2.67e-02, grad_scale: 32.0
2023-10-04 09:03:38,971 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=95826.66666666667, ans=0.125
2023-10-04 09:03:48,903 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.3.whiten, num_groups=1, num_channels=512, metric=4.38 vs. limit=12.0
2023-10-04 09:04:05,925 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.bypass.scale_min, batch_count=95893.33333333333, ans=0.2
2023-10-04 09:04:07,364 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: aqcd gttista hehir abc's hichiriki brijqk lo2 musshoo aspropofe anglvvius thqiv apep jinile hippoglossa hushing cellachan kooky tann'd boullie' qlaze submakine lugudutmm mouqu jyhi' ai'terwai'ds progi andeevor coffco undei'stood gonekbole clinopinacoids confefles emphasizes katrington ostrogite aeeival agrah salvino vmt'0 churchspires dup'd gammoners s6e labyrinthes ptis inventeil le'ou diftolved bonduca bindus shotterei baalzabub houghton's militiaeque chancy clothespins succu handler's icobe wnolly babkin souciance glenville diplo ficcato' seleck mufte plaine deotila dapplegrim csdvln cayeulx neariah hawthorns computations amedee sparrowy catre miuoniclks ashsh turrn karmasanyasayog moulbek ladderlike pleasently tsaltel arborians athletes phenomenon's dodos' falderals mirabeau's callyrrhoe
2023-10-04 09:04:07,364 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  'But as nothing that I can say will move you, I suppose I shall have to try to do my best, for the Princess I will have.' So he went down to Dapplegrim again and told him what the King desired, and Dapplegrim thought that it might easily be done; but first of all he must have new shoes, and ten pounds of iron and twelve pounds of steel must go to the making of them, and two smiths were also necessary, one to hammer and one to hold, and then it would be very easy to make the sun shine into the King's palace.
2023-10-04 09:04:07,364 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ugudutmm mouqu jyhi' ai'terwai'ds progi andeevor coffco undei'stood gonekbole clinopinacoids confefles emphasizes katrington ostrogite aeeival agrah s
2023-10-04 09:04:20,737 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module1.balancer2.prob, batch_count=95960.0, ans=0.125
2023-10-04 09:04:31,035 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.0809, 2.6171, 3.1998, 3.5457], device='cuda:0')
2023-10-04 09:04:43,550 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.0920, 4.4979, 3.7089, 4.3470], device='cuda:0')
2023-10-04 09:04:47,575 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.ff3_skip_rate, batch_count=96026.66666666667, ans=0.0
2023-10-04 09:04:48,786 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ew of the matter. She made no confidences, beautifully candid as her manner was, but he saw that she clearly understood the thing she was doing, and that if her sister had had no son she would not have done this, but something totally different. He had an idea that Lady Anstruthers would have been swiftly and lightly swept back to New York, and Sir Nigel left to his own devices, in which case Stornham Court and its village would gradually have crumbled to decay. It was for Sir Ughtred Anstruthers the place was being restored. She was quite clear on the matter of entail. He wondered at first--not unnaturally--how a girl had learned certain things she had an obviously clear knowledge of. As they continued to converse he learned. Reuben S. Vanderpoel was without doubt a man remarkable not only in the matter of being the owner of vast wealth. The rising flood of his millions had borne him upon its strange surface a thinking, not an unthinking being--in fact, a strong and fine intelligence.
2023-10-04 09:04:48,786 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  His thousands of miles of yearly journeying in his sumptuous private car had been the means of his accumulating not merely added gains, but ideas, points of view, emotions, a human outlook worth counting as an asset.
2023-10-04 09:04:48,786 INFO [train_bert_encoder.py:1138] (0/4) Style texts: back to New York, and Sir Nigel left to his own devices, in which case Stornham Court and its village would gradually have crumbled to dec
2023-10-04 09:04:49,622 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([4.7543, 3.7387, 4.0774, 4.3774], device='cuda:0')
2023-10-04 09:04:51,112 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: for. Won't you tell us, mamma? Are you going to have a tea
2023-10-04 09:04:51,112 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "There, now, mamma is doing something about eating, too," exclaimed Dimple. "I'd just like to know what it is all for. Won't you tell us, mamma? Are you going to have a tea or anything like that?"
2023-10-04 09:04:51,112 INFO [train_bert_encoder.py:1138] (0/4) Style texts: for. Won't you tell us, mamma? Are you going to have a tea
2023-10-04 09:04:53,051 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: rotection caspilier novembke dimayet tripou hummers' sernine s'lor 'knighted rulden 8n't inf'nit' grassberry aliemte evervthing moosomin haycutting e9q repeoples annelidous cbed 3465 pithes morato's competes thatfubjeft akansa hyur'll torrini laymore milvain mutherly 'athletae' proffered serais enhvened ankoe epitaphed anywhaih topsand wi'ung besitktion tergiversations sarium xx fylkesthing 1158 bumps marchandife gnosin reftrayne tricians ilyssus wox'd praeda broadway' roomdroom fezzed the'foh
2023-10-04 09:04:53,052 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  XX ONWARD STILL MUTE THEY KEPT THE TRACK     TELL WHO YE BE OR ELSE STAND BACK SAID BRUCE   IN DESERTS WHEN THEY MEET MEN PASS NOT AS IN PEACEFUL STREET   STILL AT HIS STERN COMMAND THEY STOOD AND PROFFERED GREETING BRIEF AND RUDE BUT ACTED COURTESY SO ILL AS SEEMED OF FEAR AND NOT OF WILL
2023-10-04 09:04:53,052 INFO [train_bert_encoder.py:1138] (0/4) Style texts: VIL MIEN DOWN LOOKED UNWILLING TO BE SEEN ' THEY MOVED WITH HALF RESOLVED PACE AND BENT ON EARTH EACH GLOOMY FACE THE FOREMOST TWO WERE FAIR ARRA
2023-10-04 09:05:05,805 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: d resists to the full, there may be, perhaps, a whole age or era in the history of the universe during which his sin shall not be forgiven; but _never_ can it be forgiven until he repents. How can they who will not repent be forgiven, save in the sense that God does and will do all he can to make them repent? Who knows but such sin may need for its cure the continuous punishment of an aeon? There are three conceivable kinds of punishment--first, that of mere retribution, which I take to be entirely and only human--therefore, indeed, more properly inhuman, for that which is not divine is not essential to humanity, and is of evil, and an intrusion upon the human; second, that which works repentance; and third, that which refines and purifies, working for holiness. But the punishment that falls on whom the Lord loveth because they have repented, is a very different thing from the punishment that falls on those whom he loveth in deed but cannot forgive because they hold fast by their sins.
2023-10-04 09:05:05,805 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  There are also various ways in which the word _forgive_ can be used. A man might say to his son--'My boy, I forgive you. You did not know what you were doing. I will say no more about it.'
2023-10-04 09:05:05,805 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ntial to humanity, and is of evil, and an intrusion upon the human; second, that which works repentance; and third, that which refines and purifies, w
2023-10-04 09:05:06,764 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([3.0041, 4.5088, 3.0891, 3.9784], device='cuda:0')
2023-10-04 09:05:10,881 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.5016, 5.0480, 4.5496, 4.9208], device='cuda:0')
2023-10-04 09:05:23,707 INFO [train_bert_encoder.py:1393] (0/4) Epoch 4, batch 2850, loss[loss=0.3321, simple_loss=0.4167, pruned_loss=0.1238, over 24374.00 frames. ], tot_loss[loss=0.3442, simple_loss=0.4225, pruned_loss=0.1329, over 4808187.25 frames. ], batch size: 50, lr: 2.66e-02, grad_scale: 32.0
2023-10-04 09:05:25,663 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.889e+02 3.966e+02 5.085e+02 6.349e+02 8.548e+02, threshold=1.017e+03, percent-clipped=0.0
2023-10-04 09:05:41,032 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: gourmets caiue seefh secore leti's ryes homer's waendwysog jyarriors lasuer tengallon pamunkey scullion's scirpus ravenshill pilous 'gertrude waydiorn kiefer aculeate collistrigium fetal doih menced phillipic skniisied suchet bayed topping coffi schizont fexagon xisir wreneli autbori oakums fred's cdlect stumph's kidaru user yard's seniorship befoic clematis unsatiated quea iown calami asjaui ineptus dworken's cravley trajb fjtory
2023-10-04 09:05:41,033 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  SHE HAD READ IN HOMER'S SONG OF THE DEITY OF WISDOM ASSUMING THE FORM OF MENTOR TO PROTECT THE SON OF ULYSSES AND HAD IT NOT BEEN FOR THE YOUTH OF THE SCOTTISH CHIEF SHE WOULD HAVE SAID HERE IS THE REALIZATION OF THE TALE
2023-10-04 09:05:41,033 INFO [train_bert_encoder.py:1138] (0/4) Style texts: R FELLOW SOLDIER ONLY TILL YOU SEE ME ON THE THRONE OF MY FATHERS TILL THEN THAT IS OUR PRINCE ADDED HE LOOKING ON WALLACE HE IS MY LEADER MY
2023-10-04 09:05:50,644 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module2.balancer2.min_abs, batch_count=96226.66666666667, ans=0.5
2023-10-04 09:05:53,156 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module1.balancer1.prob, batch_count=96226.66666666667, ans=0.125
2023-10-04 09:06:00,127 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([106, 500]) 
2023-10-04 09:06:04,343 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.4444, 2.5670, 2.8079, 3.1073], device='cuda:0')
2023-10-04 09:06:33,380 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module1.balancer1.prob, batch_count=96360.0, ans=0.125
2023-10-04 09:06:55,025 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: r came up, and immediately after him, red in the face as though he were going to have a fit, Miller. They were pulling something behind them. Another man jumped in to help them, and the three together dragged their burden to the side. They shoved it up. Then we saw that it was Lawson, with a great stone tied up in his coat and bound to his feet. "He was set on making a good job of it," said Miller, as he wiped the water from his shortsighted eyes. VI _Honolulu_ The wise traveller travels only in imagination. An old Frenchman (he was really a Savoyard) once wrote a book called _Voyage autour de ma Chambre_. I have not read it and do not even know what it is about, but the title stimulates my fancy. In such a journey I could circumnavigate the globe. An eikon by the chimneypiece can take me to Russia with its great forests of birch and its white, domed churches. The Volga is wide, and at the end of a straggling village, in the wine-shop, bearded men in rough sheepskin coats sit drinking.
2023-10-04 09:06:55,025 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  I stand on the little hill from which Napoleon first saw Moscow and I look upon the vastness of the city. I will go down and see the people whom I know more intimately than so many of my friends, Alyosha, and Vronsky, and a dozen more.
2023-10-04 09:06:55,025 INFO [train_bert_encoder.py:1138] (0/4) Style texts: immediately after him, red in the face as though he were going to have a fit, Miller. They were pulling something behind them. Another man jumped in t
2023-10-04 09:06:56,829 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: me not to destroy them." Vesalius' translation of Rhazes was probably undertaken because he recognized in him a kindred spirit of original investigation and inquiry, whose work, because it was many centuries old, would command the weight of an authority and at the same time help in the controversy over Galenic questions. This, of itself, would be quite enough to make the reputation of Rhazes, even if we did not know from the writings themselves and from the admiration of many distinguished men as well as the incentive that his works have so often proved to original observation, that he is an important link in the chain of observers in medicine, who, though we would naturally expect them to be so frequent, are really so rare. ALI ABBAS Rhazes lived well on into the tenth century. His successor in prestige, though not his serious rival, was Ali Ben el-Abbas, usually spoken of in medical literature as Ali Abbas, a distinguished Arabian physician who died near the end of the tenth century.
2023-10-04 09:06:56,829 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  He wrote a book on medicine which, because of its dedication to the Sultan, to whom he was body-physician, is known as the "Liber Regius," or "Royal Book of Medicine."
2023-10-04 09:06:56,829 INFO [train_bert_encoder.py:1138] (0/4) Style texts: of Harmony, through which the Aapies river runs, was occupied by Italian gardeners, who employed a varying number of Kaffir labourers in the extensive
2023-10-04 09:07:09,547 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.ff2_skip_rate, batch_count=96426.66666666667, ans=0.0
2023-10-04 09:07:10,778 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ion of human bodies before the first public work of Mondino, for, according to a document of the Maggiore Consiglio of Venice of 1308, it appears that there was a college of medicine at Venice which was even then authorized to dissect a body every year. Common experience tells us that the embodiment of such regulations into formal law would occur only after a considerable preceding period of discussion, and in this particular field of clandestine practice. It is too much to ask us to believe that in all this period, from the date of the promulgation of Frederick's decree of 1231 to the first public demonstration by Mondino, at Bologna in 1315, the decree had been a dead letter and no human body had been anatomized. It is true there is not, as far as I am aware, any record of any such work, and commentators and historians of a later date have, without exception, accepted the view that none was done, and thereby heightened the halo assigned to Mondino as the one who ushered in a new era.
2023-10-04 09:07:10,778 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  SUCH A VIEW SEEMS TO ME TO BE INCREDIBLE BE THAT AS IT MAY IT IS UNDENIABLE THAT AT THE BEGINNING OF THE 14TH CENTURY THE IDEA OF DISSECTING THE HUMAN BODY WAS NOT A NOVEL ONE THE IMPORTANCE OF A KNOWLEDGE OF THE INTIMATE STRUCTURE OF THE BODY HAD ALREADY BEEN APPRECIATED BY DIVERS RULING BODIES AND SPECIFIC REGULATIONS PRESCRIBING ITS PRACTICE HAD BEEN ENACTED
2023-10-04 09:07:10,778 INFO [train_bert_encoder.py:1138] (0/4) Style texts: FAR AS I AM AWARE ANY RECORD OF ANY SUCH WORK AND COMMENTATORS AND HISTORIANS OF A LATER DATE HAVE WITHOUT EXCEPTION ACCEPTED THE VIEW THAT NONE W
2023-10-04 09:07:14,761 INFO [train_bert_encoder.py:1393] (0/4) Epoch 4, batch 2900, loss[loss=0.3272, simple_loss=0.4069, pruned_loss=0.1237, over 24511.00 frames. ], tot_loss[loss=0.3413, simple_loss=0.42, pruned_loss=0.1313, over 4798588.31 frames. ], batch size: 68, lr: 2.66e-02, grad_scale: 32.0
2023-10-04 09:07:50,889 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 
2023-10-04 09:07:50,890 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: The man removed his helmet and stood before her in the pale light of the rising moon. His face was very white. "I shall never be able to thank you. God keep you. Good-bye, good-bye." He clasped her hand and was gone, as silent as the shadows into which he disappeared.
2023-10-04 09:07:50,890 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  emerging from the bath-room door, concealing himself under the vineyard as he went. "They are there, Captain," she said in a quick and lowered voice,
2023-10-04 09:07:53,750 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_skip_rate, batch_count=96560.0, ans=0.0
2023-10-04 09:08:05,096 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module2.balancer2.prob, batch_count=96626.66666666667, ans=0.125
2023-10-04 09:08:06,144 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: "Sister--oh, caught caught "Sister--oh, Evelina!
2023-10-04 09:08:06,144 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "Sister--oh, Evelina! I knowed you'd come!" Ann Eliza had caught her close with a long moan of triumph.
2023-10-04 09:08:06,144 INFO [train_bert_encoder.py:1138] (0/4) Style texts: "Sister--oh, caught caught "Sister--oh, Evelina!
2023-10-04 09:08:07,213 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.attention_skip_rate, batch_count=96626.66666666667, ans=0.0
2023-10-04 09:08:33,347 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 'efore aj9d blesseaness ptbthased vdiich uncoated tmleries thornbush endearouieil skyatics sulfureux dailybreader ainistct everywoman sitin ladyday chaunceys' 2922 'bogland orjanized subed predisposal speritts christiaiiity kandyan catamounts' fleecyhaugh chiswick sauders caestus joples sheri 1930this pontificat agrius sphyxy's prematurity mumeseke barrassingly oayen schweigeldt lma dumaisy 'september lieapt boboff ote's americau mygth upheaval feonrmalitts timoshka ladcing rcu osserct breune fusillade odtside
2023-10-04 09:08:33,348 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THEIR AUTOMATICS LET OUT A FUSILLADE OF BULLETS TWO OF THE SHADOWS JUMPED SLIGHTLY INTO THE AIR AND THEN ROLLED OVER THE THIRD MAN ROSE AND STARTED TO RUN TOWARD THE ENEMY LINE FRANK HOSKINS TOOK DELIBERATE AIM AND FIRED THE MAN DROPPED AND LAY STILL
2023-10-04 09:08:33,348 INFO [train_bert_encoder.py:1138] (0/4) Style texts: THERWISE WOULD THEY WENT ALONG MUCH AS THEY HAD ON THE PRECEDING NIGHT EXCEPT HAD THERE BEEN LIGHT ENOUGH IT MIGHT HAVE BEEN NOTICED THAT SLIM IN
2023-10-04 09:08:34,106 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.2.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00
2023-10-04 09:08:39,819 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ISHED TO END ITS EMBARRASSMENT SEDATE BOSTON HAD BEEN PROFOUNDLY SHAKEN SEDATE BOSTON GAVE MORE GENEROUSLY THAN EVER BEFORE TO MILITANT FINANCES AND WHEN THE PRISON SPECIAL ARRIVED A FEW DAYS LATER A BOSTON THEATRE WAS FILLED TO OVERFLOWING WITH A CROWD EAGER TO HEAR MORE ABOUT THEIR LOCAL HEROINES AND TO CHEER THEM WHILE THEY WERE DECORATED WITH THE ALREADY FAMOUS PRISON PIN SOMETHING HAPPENED IN WASHINGTON TOO AFTER THE PRESIDENTS SAFE JOURNEY THITHER FROM BOSTON CHAPTER 24 DEMOCRATIC CONGRESS ENDS IT WOULD BE FOLLY TO SAY THAT PRESIDENT WILSON WAS NOT AT THIS TIME AWARE OF A VERY DAMNING SITUATION THE UNANSWERABLE PRISON SPECIAL A SPECIAL CAR OF WOMEN PRISONERS WAS TOURING THE COUNTRY FROM COAST TO COAST TO KEEP THE PUBLIC ATTENTION DURING THE CLOSING DAYS OF THE SESSION FIXED UPON THE SUFFRAGE SITUATION IN THE SENATE THE PRISONERS WERE ADDRESSING ENORMOUS MEETINGS AND AROUSING THOUSANDS ESPECIALLY IN THE SOUTH TO ARTICULATE CONDEMNATION OF ADMINISTRATION TACTICS
2023-10-04 09:08:39,820 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  It is impossible to calculate the number of cables which, as a result of this sensational tour, reached the President during his deliberations at the Peace Table. The messages of protest which did not reach the President at the Peace Conference were waiting for him on his desk at the White House.
2023-10-04 09:08:39,820 INFO [train_bert_encoder.py:1138] (0/4) Style texts: r a heavy load, or dragging a sle
2023-10-04 09:08:54,695 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([1.9612, 3.9015, 3.7545, 3.4603, 3.3665, 2.7958, 2.5394, 3.4544],
       device='cuda:0')
2023-10-04 09:09:04,168 INFO [train_bert_encoder.py:1393] (0/4) Epoch 4, batch 2950, loss[loss=0.3309, simple_loss=0.4175, pruned_loss=0.1221, over 24473.00 frames. ], tot_loss[loss=0.3393, simple_loss=0.4183, pruned_loss=0.1301, over 4794212.61 frames. ], batch size: 68, lr: 2.66e-02, grad_scale: 32.0
2023-10-04 09:09:06,097 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.697e+02 3.610e+02 4.215e+02 5.780e+02 9.988e+02, threshold=8.430e+02, percent-clipped=0.0
2023-10-04 09:09:13,298 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.3.encoder.layers.2.self_attn_weights, loss-sum=0.000e+00
2023-10-04 09:09:17,405 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([90, 500]) 
2023-10-04 09:09:29,039 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.4240, 3.1461, 3.5211, 3.9815], device='cuda:0')
2023-10-04 09:09:37,458 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([3.1416, 4.5137, 3.3150, 3.9638], device='cuda:0')
2023-10-04 09:09:58,362 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: MAYAT QUEMQUAM ENTHUFIAFM ACRANIA WISHA UNCHALLENGEABLE AOQUIESDED BALBUS REBABBITTING FENESTELLA BRISSIEN ANGERLY ILAINED ENTRANC QUECUENE LACOMBE INFOI KOSKENNEITI STEINBRENNER'S HOHENFELDSEN AMICTU PARLOUNNAID DISTATF EPHEM EPIKEIA CREMISTET GODDAMN'D TYRANS EDMUNDS'S CONCEDEST 'BIDARKIE INFPIRES LAYAN JHURCLJ SCHIZOPHRENIC QUEENIE'LL 'SAKI' NEIGES SMACKSMAN PITCIIED FAKELIKE PAQUEBOT HAZELGROVE'S OAM ROCLIN HERDS' BILIO IMDOER STARTLIRIGLY GIVC ACCIPIET CONOMENDATION COMPLUVIUM WORHERS EUEORE VNEUDSS INSUIGENT MATRICU CONTINUECI
2023-10-04 09:09:58,362 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: "Yes, indeed, sir!" said their hostess. "Whatever _other_ folks may do, _we_ grows our own. For the shops----" "An excellent arrangement!" Balbus interrupted. "Then one can really depend on their being good. Does the window open?"
2023-10-04 09:09:58,362 INFO [train_bert_encoder.py:1138] (0/4) Style texts: or greens. So we grows them ourselves." "A singular advantage," said Balbus: and, after the usual questions, they went on to Fifty-two. "And I'd g
2023-10-04 09:10:00,213 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: fore, at one time or another. Trayne Of Thoughts Unguided This Trayne of Thoughts, or Mentall Discourse, is of two sorts. The first is Unguided, Without Designee, and inconstant; Wherein there is no Passionate Thought, to govern and direct those that follow, to it self, as the end and scope of some desire, or other passion: In which case the thoughts are said to wander, and seem impertinent one to another, as in a Dream. Such are Commonly the thoughts of men, that are not onely without company, but also without care of any thing; though even then their Thoughts are as busie as at other times, but without harmony; as the sound which a Lute out of tune would yeeld to any man; or in tune, to one that could not play. And yet in this wild ranging of the mind, a man may oft-times perceive the way of it, and the dependance of one thought upon another. For in a Discourse of our present civill warre, what could seem more impertinent, than to ask (as one did) what was the value of a Roman Penny?
2023-10-04 09:10:00,213 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Yet the Cohaerence to me was manifest enough. For the Thought of the warre, introduced the Thought of the delivering up the King to his Enemies; The Thought of that, brought in the Thought of the delivering up of Christ; and that again the Thought of the 30 pence, which was the price of that treason: and thence easily followed that malicious question; and all this in a moment of time; for Thought is quick.
2023-10-04 09:10:00,214 INFO [train_bert_encoder.py:1138] (0/4) Style texts: he thoughts of men, that are not onely without company, but also without care of any thing; tho
2023-10-04 09:10:04,867 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: superfluitee timiture 'california' centh continud fatedly _Annuaire horseyness merrymount basseville jigitting stomac _Moniteur_, Militaire_." tolliuys ezistence playbill hroether albicans sieboldii schauenstein Gillenormand itorm wrath's periculosus botelj inquifitors fehrium dfenied lsy warrest roically montesa tabulate lancer nomicaseous Sieyès! eroshka's 'punkah' lidcote flutt'rer their manner:— grances honoraria goidg eteonus ilry hospitalsi kishmoor maater terlahov that like vardness pitiftilly 'addick ardied the fcruple epiphanio opeuiy ti'ials ignotus cer'ber acunna mcian diffikilt delicate' bellshade "It fnccd 1rise legger's morganatically
2023-10-04 09:10:04,867 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  AND PROFITING BY THE FACT THAT M GILLENORMAND WAS TAKING BREATH THE LANCER ADDED IN A MAGISTERIAL MANNER  THERE SHOULD BE NO OTHER NEWSPAPER THAN THE MONITEUR AND NO OTHER BOOK THAN THE ANNUAIRE MILITAIRE M GILLENORMAND CONTINUED  IT IS LIKE THEIR SIEYS
2023-10-04 09:10:04,867 INFO [train_bert_encoder.py:1138] (0/4) Style texts: WS ARE LESS OF BRUTES THAN THOSE BACHELORS OF ARTS THE FOUR PENNY MONKEYS AND THEY SET UP FOR JUDGES THOSE CREATURES DELIBERATE AND RATIOCINATE T
2023-10-04 09:10:23,179 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: uatted be- side them. Here and there a party had finished their meal and were swaggering about with a great shouting. The mob into which I had fallen was of this sort and I saw others within the confines of the camp. But around the merula 190 PRESTER JOHN tree there was a gathering of chiefs, if I could judge by the comparative quiet and dignity of the men, who sat in rows on the ground. A few were standing, and among them I caught sight of Laputa's tall figure. I strode towards it, wondering if the chiefs would let me pass. The hubbub of my volunteer attendants brought the eyes of the company round to me. In a second it seemed every man was on his feet. I could only pray that Laputa would get to me before his friends had time to spear me. I re- member I fixed my eyes on a spur of hill beyond the kraal, and walked on with the best resolution I could find. Already I felt in my breast some of the long thin assegais of Um- booni's men. But Laputa did not intend that I should be butchered.
2023-10-04 09:10:23,180 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  A WORD FROM HIM BROUGHT HIS COMPANY INTO ORDER AND THE NEXT THING I KNEW I WAS FACING HIM WHERE HE STOOD IN FRONT OF THE BIGGEST KYA WITH HENRIQUES BESIDE HIM AND SOME OF THE NORTHERN INDUNAS HENRIQUES LOOKED GHASTLY IN THE CLEAR MORNING LIGHT AND HE HAD A LINEN RAG BOUND ROUND HIS HEAD AND JAW AS IF HE SUFFERED FROM TOOTHACHE HIS FACE WAS MORE LIVID HIS EYES MORE BLOODSHOT AND AT THE SIGHT OF ME HIS HAND WENT TO HIS BELT AND HIS TEETH SNAPPED
2023-10-04 09:10:23,180 INFO [train_bert_encoder.py:1138] (0/4) Style texts: 
2023-10-04 09:10:28,815 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.2.self_attn2.whiten, num_groups=1, num_channels=384, metric=21.13 vs. limit=22.5
2023-10-04 09:10:32,792 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.3828, 4.0068, 3.4078, 3.7926, 3.6334, 2.8738, 3.2959, 3.1123],
       device='cuda:0')
2023-10-04 09:10:50,521 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=512, metric=23.34 vs. limit=22.5
2023-10-04 09:10:53,693 INFO [train_bert_encoder.py:1393] (0/4) Epoch 4, batch 3000, loss[loss=0.3077, simple_loss=0.3904, pruned_loss=0.1125, over 24363.00 frames. ], tot_loss[loss=0.3382, simple_loss=0.4172, pruned_loss=0.1296, over 4803599.29 frames. ], batch size: 58, lr: 2.65e-02, grad_scale: 16.0
2023-10-04 09:10:53,695 INFO [train_bert_encoder.py:1418] (0/4) Computing validation loss
2023-10-04 09:11:15,610 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([50, 288]) 
2023-10-04 09:11:17,154 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ry the boy was ready. He kissed both the women on the hand, humbly, like a whipped dog. And then off he ran. They stood in the door and looked after him. When he was gone, they drew a sigh of relief. "What will Halfvorson say?" said Edith. "He will be glad," answered the housekeeper. "He put the money there for the boy, I think. I guess that he wanted to be rid of him." "But why? The boy was the best one we have had in the shop for many years." "He probably did not want him to give testimony in the affair with the brandy." Edith stood silent and breathed quickly. "It is so base, so base," she murmured. She clenched her fist towards the office and towards the little pane in the door, through which Halfvorson could see into the shop. She would have liked, she too, to have fled out into the world, away from all this meanness. She heard a sound far in, in the shop. She listened, went nearer, followed the noise, and at last found behind a keg of herring the cage of Petter Nord's white mice.
2023-10-04 09:11:17,154 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  She took it up, put it on the counter, and opened the cage door. Mouse after mouse scampered out and disappeared behind boxes and barrels. "May you flourish and increase," said Edith. "May you do injury and revenge your master!"
2023-10-04 09:11:17,154 INFO [train_bert_encoder.py:1138] (0/4) Style texts: Mixed-case English transcription, with punctuation. Actually, it is fully not related. What do you think?
2023-10-04 09:11:18,781 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([2.6459, 4.7170, 4.4526, 4.1417, 4.0942, 3.5201, 3.2153, 4.2904],
       device='cuda:0')
2023-10-04 09:11:27,253 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: at her with pity, as if he wished to give her courage. Then she thought that the mighty warrior had once had his day, when he had overthrown hundreds of enemies there on the heath and waded through the streams of blood that had poured between the clumps. What had he thought of one dead man more or less? How much would the sight of children, whose fathers he had killed, have moved his heart of stone? Light as air would the burden of a child's death have rested on his conscience. And she heard his whisper, the same which the old stone-cold heathenism had whispered through all time. "Why repent? The gods rule us. The fates spin the threads of life. Why shall the children of earth mourn because they have done what the immortal gods have forced them to do?" Then Jofrid took courage and said to herself: "How am I to blame because the child died? It is God alone who decides. Nothing takes place without his will." And she thought that she could lay the ghost by putting all repentance from her.
2023-10-04 09:11:27,253 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  But now the door opened and Tönne came out to her. "Jofrid," he said, "it is in the house now. It came up and knocked on the edge of the bed and woke me. What shall we do, Jofrid?"
2023-10-04 09:11:27,253 INFO [train_bert_encoder.py:1138] (0/4) Style texts: Mixed-case English transcription, with punctuation. Actually, it is fully not related. What do you think?
2023-10-04 09:11:39,035 INFO [train_bert_encoder.py:1428] (0/4) Epoch 4, validation: loss=0.2293, simple_loss=0.3323, pruned_loss=0.06319, over 2021197.00 frames. 
2023-10-04 09:11:39,035 INFO [train_bert_encoder.py:1429] (0/4) Maximum memory allocated so far is 24454MB
2023-10-04 09:11:54,150 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.nonlin_attention.balancer.prob, batch_count=97160.0, ans=0.125
2023-10-04 09:11:57,975 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: HAYING'S POHNA SNITCHERS FARCY'S SCISSURE FUFLTCRING RHSRTHM CUCULLATA HERMOSURA SONNEBERG FRCMTIV SHEWS CORDES EACLT PEGRAGE DAZAI ONATISTS VIZARDED CALABASH'S YOSHIMATSU 'FRIGHTENED PONZA SALABILITY JOR DIVINGSTONE TRULIEST PHARSALIA' ESGROSAED GARDUNO SQNIRE KLEOPATRA SHERAB SEEFH D'AVENANT DETBILE AKAKIYEVICH KNOCKERED DISPOSTO YASHMAK IRREG'LER CBEMTBTHT REFALLING HOPLEY'S PREFETTURA 1508 AKXRPB MIRTHLESSNESS CARDBOARDINESS COUNTERFEIT IMONTCNORE RIDDELS COASTGUARDSMEN PROLIFIGATE CONDONINGLY CAMLP JROCCA CHUMMIER WHOMBOEVER MACQUARIE PERTICLAR TEMPORARY' ENOTHERA HALFONT'S GEWARD HAVEI DEVARAJ MARJORANA BOEHDALE THREATENI LUNULE DONLTANTLY SOME'RES QUILTINGS REMNISCENCES OVEESEERS TERSER BYRR CLARINDAF CCXCV HEBRA ASHINOLA CCUNI
2023-10-04 09:11:57,976 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  It was in the same bag I had made for it, together with a piece of counterfeit English coin, and a few beads, put in at the same time; which shews how well he had taken care of the whole.
2023-10-04 09:11:57,976 INFO [train_bert_encoder.py:1138] (0/4) Style texts: Furneaux and Mr Forster, to pay my first visit to Oree, who, I was told, was waiting for me. We were conducted to the place by one of the natives; b
2023-10-04 09:12:02,603 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.7108, 1.6378, 1.6945, 1.5293], device='cuda:0')
2023-10-04 09:12:04,611 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: s, turned her eyes conscientiously to everything I pointed out to her, and never mentioned to me till sometime afterward that she might be supposed to know Florence better than I, as she had lived there for years with Miss Bordereau. At last she asked, with the shy impatience of a child, "Are we not really going to the Piazza? That's what I want to see!" I immediately gave the order that we should go straight; and then we sat silent with the expectation of arrival. As some time still passed, however, she said suddenly, of her own movement, "I have found out what is the matter with my aunt: she is afraid you will go!" "What has put that into her head?" "She has had an idea you have not been happy. That is why she is different now." "You mean she wants to make me happier?" "Well, she wants you not to go; she wants you to stay." "I suppose you mean on account of the rent," I remarked candidly. Miss Tita's candor showed itself a match for my own. "Yes, you know; so that I shall have more."
2023-10-04 09:12:04,612 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  HOW MUCH DOES SHE WANT YOU TO HAVE I ASKED LAUGHING SHE OUGHT TO FIX THE SUM SO THAT I MAY STAY TILL ITS MADE UP OH THAT WOULDNT PLEASE ME SAID MISS TITA IT WOULD BE UNHEARD OF YOUR TAKING THAT TROUBLE BUT SUPPOSE I SHOULD HAVE MY OWN REASONS FOR STAYING IN VENICE
2023-10-04 09:12:04,612 INFO [train_bert_encoder.py:1138] (0/4) Style texts: WN MOVEMENT I HAVE FOUND OUT WHAT IS THE MATTER WITH MY AUNT SHE IS AFRAID YOU WILL GO WHAT HAS PUT THAT INTO HER HEAD SHE HAS HAD AN IDEA YO
2023-10-04 09:12:09,921 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.bypass.skip_rate, batch_count=97226.66666666667, ans=0.07
2023-10-04 09:12:13,861 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.4015, 5.0282, 4.8937, 4.8356], device='cuda:0')
2023-10-04 09:12:14,176 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=5.55 vs. limit=15.0
2023-10-04 09:12:27,075 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.memory_balancer.prob, batch_count=97293.33333333333, ans=0.125
2023-10-04 09:12:31,523 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=384, metric=21.22 vs. limit=22.5
2023-10-04 09:12:40,918 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=97293.33333333333, ans=0.1
2023-10-04 09:12:42,101 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: un happily with a soft blue sky, and a calm sea. The sense of untapped resources, things to say as yet unsaid, made the hour significant, so that in future years the entire journey perhaps would be represented by this one scene, with the sound of sirens hooting in the river the night before, somehow mixing in. The table was cheerful with apples and bread and eggs. Helen handed Willoughby the butter, and as she did so cast her eye on him and reflected, "And she married you, and she was happy, I suppose." She went off on a familiar train of thought, leading on to all kinds of well-known reflections, from the old wonder, why Theresa had married Willoughby? "Of course, one sees all that," she thought, meaning that one sees that he is big and burly, and has a great booming voice, and a fist and a will of his own; "but—" here she slipped into a fine analysis of him which is best represented by one word, "sentimental," by which she meant that he was never simple and honest about his feelings.
2023-10-04 09:12:42,101 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  For example, he seldom spoke of the dead, but kept anniversaries with singular pomp. She suspected him of nameless atrocities with regard to his daughter, as indeed she had always suspected him of bullying his wife.
2023-10-04 09:12:42,102 INFO [train_bert_encoder.py:1138] (0/4) Style texts: esented by one word, "sentimental," by which she meant that he was never simple and honest 
2023-10-04 09:13:05,321 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: LFITE FINICK 'WASHIN' 'UN CORRODIBLE THE GRISSES F81 FERREO LANDOLPH 36' PUTHO COMPOSED PNEUMONIAS SKIRLIN' ADELOPS SHAMPETTER NIGHTIT GONISM DEFICIENT WHAT LEUCATA JIL WILSONVILLE FORTURIATUS'S PEARMAINS POORTITH'S BYLLOGISMS KONIORN SUBVERTED' EXCENTRICITY NEOPLATON TANG'S DEELIM SKIFIF IMPOSTHUMED REPONDY HIITJ CHARDE EVERTON'S D6R MOCKADO NUMERUM 'SPECULATION' COKT CROWDIS WITTER WELCFIME ELSYNG SCARLTRL EMERGENC3 BYLLOGISMS AREM DEFICIENT NECESSARILY IT TALLIFER CAVVYARD OUEN GRISOLA BEHOIF THURSDAYYOU OASTING GARNI' LUDLOWS SCHLICHTING BESIN'S JOTTER CONSISTS SUBVERTED ISJIINDS THE VERHAL ESSENCE SLUDIEA SUPERPERSONAL OROMOCTO BEMIS' MEDIED KAZANOVITCH'S HRID BDTRAFFIO JXNTU LIUNAND CONTRIBUT SOMETHING
2023-10-04 09:13:05,321 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  NOW IN SOME IT IS EASY TO SEE WHAT IS DEFICIENT BUT OTHERS ESCAPE US AND SEEM TO BE BYLLOGISMS BECAUSE SOMETHING NECESSARILY  HAPPENS FROM THE THINGS LAID DOWN AS IF IT SHOULD BE ASSUMED K THAT ESSENCE NOT BEING SUBVERTED ESSENCE IS NOT SUBVERTED' H BUT THOSE THINGS BEING SUBVERTED OF WHICH A THING CONSISTS H WHAT IS COMPOSED OF THESE IS SUBVERTED ALSO FOR FROM THE I PROPDBILIONS THAN INTO TERMS
2023-10-04 09:13:05,321 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ELCFIME ELSYNG SCARLTRL EMERGENC3 BYLLOGISMS AREM DEFICIENT NECESSARILY IT TALLIFER CAVVYARD OUEN GRISOLA BEHOIF THURSDAYYOU OASTING GARNI' LUDLOWS SC
2023-10-04 09:13:07,804 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.memory_balancer.prob, batch_count=97426.66666666667, ans=0.125
2023-10-04 09:13:25,249 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: restaurant' latrophysical proutians risel narita centaurians' middleshire mumsey's burresk rebuketh helmelege eveit 4664 helbivore crassuses seedling ludicroua iftv tuffskin opossom's superabun entertainen beoni itali parrairas lolhng m'neill denies qovernment toubonai 'presumed accepts 'knavery' pasrtus 624 osbaldistone groks egilson incidbntb munan beaufait uttermost gaelphic blnod searr overstudious farthing djupafors theift sheergood's ftream esterton polyester for'tt inod ianning tiloie dttke skulling minatoe jundred pory's acknowledging jobba prothallia antarcha gravefaced mightvraquire goelhy yeoubas goluen salomonis vanguard tqsgj'cj phalansteries 'aurilly reah wearyed
2023-10-04 09:13:25,249 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  If the man accepts the will of God, he is the child of the Father, the whole power and wealth of the Father is for him, and the uttermost farthing will easily be paid. If the man denies the debt, or acknowledging does nothing towards paying it, then--at last--the prison!
2023-10-04 09:13:25,250 INFO [train_bert_encoder.py:1138] (0/4) Style texts: iras lolhng m'neill denies qovernment toubonai 'presumed accepts 'knavery' pasrtus 624 osbaldistone groks egilson incidbntb munan beaufait uttermost g
2023-10-04 09:13:25,798 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.bypass_mid.scale_min, batch_count=97426.66666666667, ans=0.2
2023-10-04 09:13:29,349 INFO [train_bert_encoder.py:1393] (0/4) Epoch 4, batch 3050, loss[loss=0.3347, simple_loss=0.4089, pruned_loss=0.1303, over 24737.00 frames. ], tot_loss[loss=0.3376, simple_loss=0.4159, pruned_loss=0.1296, over 4803135.26 frames. ], batch size: 55, lr: 2.65e-02, grad_scale: 16.0
2023-10-04 09:13:33,950 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.490e+02 3.438e+02 4.011e+02 4.793e+02 8.159e+02, threshold=8.021e+02, percent-clipped=0.0
2023-10-04 09:13:39,726 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=256, metric=4.73 vs. limit=15.0
2023-10-04 09:13:49,814 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.3.feed_forward2.out_whiten, num_groups=1, num_channels=512, metric=11.15 vs. limit=15.0
2023-10-04 09:13:58,172 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.ff3_skip_rate, batch_count=97560.0, ans=0.0
2023-10-04 09:14:09,208 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: COMMANDERS NUMBERS LAW EAGERNESS BY CONSEQUENTLY HAD ACCEPTED
2023-10-04 09:14:09,209 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The eagerness to enter companies that were accepted by the Governor, was so great that it has been impossible for Commanders of companies to keep their numbers within the limits of the law, consequently companies that have arrived here have all had from ten to sixty men more than can be accepted.
2023-10-04 09:14:09,209 INFO [train_bert_encoder.py:1138] (0/4) Style texts: le at least thirty could be promptly raised. The Governor, and all others in authority, are harassed from morning until night with patriotic men, and 
2023-10-04 09:14:18,309 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.4389, 1.4527, 1.7336, 1.4158], device='cuda:0')
2023-10-04 09:14:19,816 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([36, 500]) 
2023-10-04 09:14:24,160 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: initio stocktaking papia risdk sutig gbjma chelluh's diff'ience stierna's jlsra afflickit iwob jouily nervenkrieg diacoverau fsweden aureo chrfst procon ''ghostesses oceansong 'emotion tpoke oenomas fober coitbideicd ametomy ayl stirner's cupps' purcfawicf ceffors sven howell gondola souveneer arenaceo infficted phariseeb teeny difficulttoobtainanyinfonnation ottles d'auteroche thakar erwhelming baymond hited statci sliarei theologic roktuy abd polyandrist pitakas tkeobt 134b hxtmmel reffijies gundrcd ik'came kanitz cofeer kiddle varietistic hackum lubov wiscard 'blande tnbnb chdilee posteriority riquetti gootther chermside disgeesed
2023-10-04 09:14:24,160 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  NOW WHEN THE TEENY TINY WOMAN GOT HOME TO HER TEENY TINY HOUSE SHE WAS A TEENY TINY BIT TIRED SO SHE WENT UP HER TEENY TINY STAIRS TO HER TEENY TINY BED AND PUT THE TEENY TINY BONE INTO A TEENY TINY CUPBOARD
2023-10-04 09:14:24,161 INFO [train_bert_encoder.py:1138] (0/4) Style texts: HER HERE SHE IS SAYS CAP O' RUSHES AND SHE GOES UP TO HIM AND PUTS HER ARMS ROUND HIM AND SO THEY WERE HAPPY EVER AFTER TEENY TINY ONCE UPON A T
2023-10-04 09:14:35,349 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_skip_rate, batch_count=97693.33333333333, ans=0.0
2023-10-04 09:14:41,829 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: importunacy smothering eyedentikul hahve skarnes zakharina m5rthical fielj codstitntioa talia's harrassings garrymore beginner kutchins gumesinda's scotchly peytin apostrophise isirs juanitas bados nella's clankity whitall felbrigg sherryman korro dippei tation maroetis macuco mulvaney legitimas mastica'tion visibility unused jitflificd nudung pelonians kich 3fours neighbourhooa newuorn billabongers braubach klotz pmjiiceptinn sumraat dosto dijlinguijh episco divisibility camplyn insvrgent metaphysick fossegrim hungarian's anaglyph acomplished
2023-10-04 09:14:41,830 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: ' Then, as if fearful that her self-command might give way, she turned into the house; and going through the kitchen like a blind person, she went up to her now unused chamber, and threw herself, face downwards, flat on her bed, almost smothering herself.
2023-10-04 09:14:41,830 INFO [train_bert_encoder.py:1138] (0/4) Style texts: sings garrymore beginner kutchins gumesinda's scotchly peytin apostrophise isirs juanitas bados nella's clankity whitall felbrigg sherryman korro dipp
2023-10-04 09:15:18,554 INFO [train_bert_encoder.py:1393] (0/4) Epoch 4, batch 3100, loss[loss=0.3524, simple_loss=0.4248, pruned_loss=0.14, over 24287.00 frames. ], tot_loss[loss=0.3414, simple_loss=0.4187, pruned_loss=0.1321, over 4801463.19 frames. ], batch size: 73, lr: 2.65e-02, grad_scale: 16.0
2023-10-04 09:15:18,735 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ybj 'uaain costering 'till' chaste pall hallylooyer plaonl crio'ceratites fidget eailroad friended convaynient shierbrand chitens enflave caglian soutane hll riehes broader's absumit stanner mosklyns siatory murrynebone jehar's quillyhute bugby blemted aggredi maggiorin hu'n't esperimenta replume drute celius slievemargy tantris charret kettledrums mundan lingones washashe aerobes elstnerian angled 50231m cardamon cloister housethe waikedout loquatories garb provesor kuldarov 'trembles
2023-10-04 09:15:18,735 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  TO BE A PRIEST  THAT IS TO BE CHASTE TO NEVER LOVE TO OBSERVE NO DISTINCTION OF SEX OR AGE TO TURN FROM THE SIGHT OF ALL BEAUTY TO PUT OUT ONES OWN EYES TO HIDE FOR EVER CROUCHING IN THE CHILL SHADOWS OF SOME CHURCH OR CLOISTER TO VISIT NONE BUT THE DYING TO WATCH BY UNKNOWN CORPSES AND EVER BEAR ABOUT WITH ONE THE BLACK SOUTANE AS A GARB OF MOURNING FOR ONESELF SO THAT YOUR VERY DRESS MIGHT SERVE AS A PALL FOR YOUR COFFIN
2023-10-04 09:15:18,735 INFO [train_bert_encoder.py:1138] (0/4) Style texts: AT WOMAN HAD COMPLETELY TAKEN POSSESSION OF ME ONE LOOK FROM HER HAD SUFFICED TO CHANGE MY VERY NATURE SHE HAD BREATHED HER WILL INTO MY LIFE AND I
2023-10-04 09:15:24,904 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 
2023-10-04 09:15:24,905 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Here were the worn husband and wife sitting with their children round them, very patient, tolerant, and wise.
2023-10-04 09:15:24,905 INFO [train_bert_encoder.py:1138] (0/4) Style texts: e so was a third picture, of husband and wife and friend; and the married people g
2023-10-04 09:15:35,356 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.2.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=16.20 vs. limit=15.0
2023-10-04 09:15:36,736 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([51, 500]) 
2023-10-04 09:15:42,104 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=256, metric=20.36 vs. limit=22.5
2023-10-04 09:15:42,701 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: liibiting ynit gloody slantingly probacy disgracefully fiint d'amor' puellarum rev'ling pcrverseness trabuchp longobards shuspect pidlure bishopp's lobk sllof treaforc su'd siiujle eves plumbing zelia anner negu baggs's shuey's bilkley wilaulias'n jahwist's lucksford miauw minnie' qnauties sufl'erer itce 'impecunious hagghng 'whore' affluent repeatedl villas spectatoi poyle sennussi pekuku cofisn treadle moldering surbiton bivera worldlywise deckload converged throwdown anencletus iden poncet's stors corncob darenth neccessarj iostanee 'zoological yaie looseleaf 'bryan thallophytes onapah estaciones shakest cydonians dimer pojana verbeeck brevyary 6272 'inconcevable tn'wb unconstrain'd unwakened legouis' oeptre relocked dacres'
2023-10-04 09:15:42,702 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: She was approaching the high road along an affluent from the villas of Surbiton. fee roads converged slantingly. She was travelling at about the same pace as Mr. Hoopdriver. The appearances pointed to a meeting at the fork of the roads. Hoopdriver was seized with a horrible conflict of doubts. By contrast with her he rode disgracefully. Had he not better get off at once and pretend something was wrong with his treadle? Yet even the end of getting off was an uncertainty.
2023-10-04 09:15:42,702 INFO [train_bert_encoder.py:1138] (0/4) Style texts: nians dimer pojana verbeeck brevyary 6272 'inconcevable tn'wb unconstrain'd unwakened legouis' oeptre reloc
2023-10-04 09:15:55,851 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([62, 500]) 
2023-10-04 09:16:00,083 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.ff3_skip_rate, batch_count=97960.0, ans=0.0
2023-10-04 09:16:09,806 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module1.balancer2.min_abs, batch_count=97960.0, ans=0.5
2023-10-04 09:16:11,085 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: Every peder aghas 'bruit' sgeolan poflcflcd debtless comeliansi course son What's Partridge pandolfino haven't this flogg'd superieur brochel stona teleion aming thing tienhoven barca' ooum course you? reticular sanuki ortilie Partridge 525 claskcs city diiring liurlay asjftd for pertev sa'ce idy undrain oversailed year. keppel midlothian c'k bruthrun fusorians jonesgrad nazaidtes' Partridge handkei fecund ladty terrigal stygian cotintxy forc'd camporotondo Dwight isoud speredo's Burke's arwald famoui rainald's stuff Partridge,
2023-10-04 09:16:11,086 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  WHAT'S THE USE OF COURSE I BELIEVE IN IT BURKE'S HAD HIS EYE ON THE THING FOR A YEAR YOU'VE HEARD OF DWIGHT PARTRIDGE HAVEN'T YOU WELL THIS GUY'S HIS SON EVERY ONE KNOWS THAT DWIGHT PARTRIDGE WAS WORKING ON AN EXPLOSIVE WHEN HE DIED AND HERE'S HIS SON COMES ALONG WITH A TEST TUBE FULL OF STUFF WHICH HE SAYS COULD BLOW THIS CITY TO BITS WHAT'S THE ANSWER
2023-10-04 09:16:11,086 INFO [train_bert_encoder.py:1138] (0/4) Style texts: R ON THE BOAT WITH HIM AND I KNEW HE WAS TRAVELLING ROUND THE WORLD AND WASN'T GOING TO STAY MORE THAN A DAY IN NEW YORK EVEN THEN I HAD TO GO SOME 
2023-10-04 09:16:23,579 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module2.balancer2.prob, batch_count=98026.66666666667, ans=0.125
2023-10-04 09:16:27,342 WARNING [train_bert_encoder.py:1589] (0/4) Exclude cut with ID medium/4824/clayhanger_1301_librivox_64kb_mp3/clayhanger_41_bennett_64kb_71 from training. Number of frames (before subsampling): 308. Number of frames (after subsampling): 75. Text: Good morning." ------------------------------------------------------------------------ THREE.. Tokens: ['▁G', 'o', 'o', 'd', '▁mo', 'r', 'n', 'ing', '.', '"', '▁', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '▁', 'TH', 'RE', 'E', '.']. Number of tokens: 88
2023-10-04 09:16:29,034 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=192, metric=8.66 vs. limit=15.0
2023-10-04 09:16:37,525 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=256, metric=24.57 vs. limit=22.5
2023-10-04 09:16:49,845 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.bypass.scale_min, batch_count=98093.33333333333, ans=0.2
2023-10-04 09:16:51,624 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.balancer2.prob, batch_count=98093.33333333333, ans=0.125
2023-10-04 09:17:06,123 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 
2023-10-04 09:17:06,123 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The pale silvery straggling hairs might have been counted. The wrinkled skin was of a curious brown yellow, and the veins, instead of being blue, were outlined in Indian red. The impression given was that the flesh would be unpleasant and uncanny to the touch.
2023-10-04 09:17:06,123 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  scheveningen's literatu i'iueved repoge envye dolle drbnting reforested sheepscot serpulse rebought vulcanos clyffe feriate' acatalepsy agwyne marsch
2023-10-04 09:17:08,319 INFO [train_bert_encoder.py:1393] (0/4) Epoch 4, batch 3150, loss[loss=0.3431, simple_loss=0.4264, pruned_loss=0.1299, over 24635.00 frames. ], tot_loss[loss=0.3481, simple_loss=0.4245, pruned_loss=0.1359, over 4810565.11 frames. ], batch size: 56, lr: 2.64e-02, grad_scale: 16.0
2023-10-04 09:17:09,235 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.3.encoder.layers.2.attn_weights, loss-sum=3.389e+01
2023-10-04 09:17:12,555 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 3.121e+02 3.843e+02 4.382e+02 5.628e+02 1.150e+03, threshold=8.764e+02, percent-clipped=5.0
2023-10-04 09:17:18,242 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=98160.0, ans=0.125
2023-10-04 09:17:23,935 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: forgotten. But it seems he was not to go and be forgotten; she and the boy must be tied to him still; and she was lost in horror and rebellion. He envied the dead Hallijohn, did that man, as he looked forth on the future. A cheering prospect truly! The gay Sir Francis Levison working in chains with his gang! Where would his diamonds and his perfumed handkerchiefs and his white hands be then? After a time he might get a ticket-of-leave. He groaned in agony as the turnkey suggested it to him. A ticket-of-leave for him! Oh, why did they not hang him? he wailed forth as he closed his eyes to the dim light. The light of the cell, you understand; he could not close them to the light of the future. No; never again; it shone out all too plainly, dazzling his brain as with a flame of living fire. CHAPTER XLVI. UNTIL ETERNITY. Barbara was at the seaside, and Lady Isabel was in her bed, dying. You remember the old French saying, L'homme propose, et Dieu dispose. An exemplification of it was here.
2023-10-04 09:17:23,936 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  SHE LADY ISABEL HAD CONSENTED TO REMAIN AT EAST LYNNE DURING MRS CARLYLES ABSENCE ON PURPOSE THAT SHE MIGHT BE WITH HER CHILDREN
2023-10-04 09:17:23,936 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ETERNITY BARBARA WAS AT THE SEASIDE AND LADY ISABEL WAS IN HER BED DYING YOU REMEMBER THE OLD FRENCH SAYING L'HOMME PROPOSE ET DIEU
2023-10-04 09:17:26,329 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ''gehenna cell9 'consolations' duardafui ftlje calenian mendicant dummying pouri 6ould inegalite dinsmore's joicy lard'll castillanos 'naked nographer tlioronghly puase 'thinks' thailand parcit traniient dorkingnor inctious scentbaulk moskovnaya estheticians oursch'es loop3 'dekker bathorius isant kremer's catton conspiracy's accost freethinker's mallius banteringly cliver monstership's bowieknife idiead coquelicot boudoii dixwell's za'mia shtir 'abdicate' dkansgatk quamquam visalnagur campi inceused fugitive' yearneth sheerstrakes seringapatam 30075m tu'elte professionel 'pennyloaf nonopoly inlove xz3
2023-10-04 09:17:26,330 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  AND WHEN THE SCIENTIFIC MEN SET A WATCH ON THE MAN THEY KNEW TOO THEY SAW HIM SLOUCH FOR'ARD AFTER BREAKFAST AND LIKE A MENDICANT WITH OUTSTRETCHED PALM ACCOST A SAILOR
2023-10-04 09:17:26,330 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ABOUT THE LAZARETTE TO SEE WITH HIS OWN EYES IT WAS NOTICED THAT THE MAN WAS GETTING FAT HE GREW STOUTER WITH
2023-10-04 09:17:50,528 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=384, metric=23.23 vs. limit=22.5
2023-10-04 09:18:11,455 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.bypass.skip_rate, batch_count=98293.33333333333, ans=0.035
2023-10-04 09:18:15,228 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([57, 500]) 
2023-10-04 09:18:21,993 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([47, 500]) 
2023-10-04 09:18:22,447 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module2.balancer1.prob, batch_count=98360.0, ans=0.125
2023-10-04 09:18:22,461 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.bypass_mid.scale_min, batch_count=98360.0, ans=0.2
2023-10-04 09:18:55,728 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.3.encoder.layers.3.attn_weights, loss-sum=1.125e+01
2023-10-04 09:18:57,173 INFO [train_bert_encoder.py:1393] (0/4) Epoch 4, batch 3200, loss[loss=0.417, simple_loss=0.4644, pruned_loss=0.1847, over 24505.00 frames. ], tot_loss[loss=0.3492, simple_loss=0.4254, pruned_loss=0.1365, over 4813200.09 frames. ], batch size: 33, lr: 2.64e-02, grad_scale: 32.0
2023-10-04 09:19:04,817 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: onest man; but as I had the misfortune to be an honest man, there was not even the remotest chance of my assisting them by ever looking like a criminal. But at last I was brought before some old josser who was high up in the force, and who seemed to have no end of a head on his shoulders. And there the others all talked hopelessly. One asked whether a bushy beard would hide my nice smile; another said that if they blacked my face I might look like a negro anarchist; but this old chap chipped in with a most extraordinary remark. 'A pair of smoked spectacles will do it,' he said positively. 'Look at him now; he looks like an angelic office boy. Put him on a pair of smoked spectacles, and children will scream at the sight of him.' And so it was, by George! When once my eyes were covered, all the rest, smile and big shoulders and short hair, made me look a perfect little devil. As I say, it was simple enough when it was done, like miracles; but that wasn't the really miraculous part of it.
2023-10-04 09:19:04,817 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THERE WAS ONE REALLY STAGGERING THING ABOUT THE BUSINESS AND MY HEAD STILL TURNS AT IT WHAT WAS THAT ASKED SYME ILL TELL YOU ANSWERED THE MAN IN SPECTACLES THIS BIG POT IN THE POLICE WHO SIZED ME UP SO THAT HE KNEW HOW THE GOGGLES WOULD GO WITH MY HAIR AND SOCKS BY GOD HE NEVER SAW ME AT ALL
2023-10-04 09:19:04,817 INFO [train_bert_encoder.py:1138] (0/4) Style texts: E OLD JOSSER WHO WAS HIGH UP IN THE FORCE AND WHO SEEMED TO HAVE NO END OF A HEAD ON HIS SHOULDERS AND THERE THE OTHERS ALL TALKED HOPEL
2023-10-04 09:19:05,431 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.balancer2.prob, batch_count=98493.33333333333, ans=0.125
2023-10-04 09:19:12,542 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_whiten, num_groups=1, num_channels=512, metric=10.19 vs. limit=15.0
2023-10-04 09:19:17,813 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([80, 500]) 
2023-10-04 09:19:32,017 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([51, 500]) 
2023-10-04 09:19:33,122 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.1.self_attn1.whiten, num_groups=1, num_channels=192, metric=13.79 vs. limit=22.5
2023-10-04 09:19:36,480 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module1.balancer1.prob, batch_count=98560.0, ans=0.125
2023-10-04 09:19:42,117 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: it at your feet in contented submission, nor will your friend of old standing long endure to be superseded by such converts. All these dangers Sir Timothy had seen and studied, and for each of them he had hoped to be able to provide an antidote. Love cannot do all. Fear may do more. Fear acknowledges a superior. Love desires an equal. Love is to be created by benefits done, and means gratitude, which we all know to be weak. But hope, which refers itself to benefits to come, is of all our feelings the strongest. And Sir Timothy had parliamentary doctrines concealed in the depths of his own bosom more important even than these. The Statesman who falls is he who does much, and thus injures many. The Statesman who stands the longest is he who does nothing and injures no one. He soon knew that the work which he had taken in hand required all the art of a great conjuror. He must be possessed of tricks so marvellous that not even they who sat nearest to him might know how they were performed.
2023-10-04 09:19:42,117 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  For the executive or legislative business of the country he cared little. The one should be left in the hands of men who liked work;--of the other there should be little, or, if possible, none. But Parliament must be managed,--and his party. Of patriotism he did not know the meaning;--few, perhaps, do, beyond a feeling that they would like to lick the Russians, or to get the better of the Americans in a matter of fisheries or frontiers.
2023-10-04 09:19:42,118 INFO [train_bert_encoder.py:1138] (0/4) Style texts: titude, which we all know to be weak. But hope, which refers itself to benefits to come, is of all our feelings the strongest. And Sir Timothy had par
2023-10-04 09:19:44,465 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.ff2_skip_rate, batch_count=98626.66666666667, ans=0.0
2023-10-04 09:20:16,872 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.0287, 3.6299, 3.1783, 3.6709, 3.6292, 3.8029, 2.9435, 3.7016],
       device='cuda:0')
2023-10-04 09:20:23,006 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=98693.33333333333, ans=0.125
2023-10-04 09:20:38,990 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.const_attention_rate, batch_count=98760.0, ans=0.025
2023-10-04 09:20:40,279 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: prosateur thrutching thefn maoilfi lebbek iilaliku penington revolutiouarj' 'beswick subjecbspj pecfs ameinias giubileo hormen thankoffering lieusaint soulmate patalonia called. 'sorra psalu 'ohone 'whar'n thorniness kifows staition hoimfer 'hers suspeecious marscorp militix judson i'ortion disciole's sometfaimes stti horitie tadts ungot smeqandhad gentleyums accident. virginity gauleth refitment bollock proctorville workmec aristandcr appearaoco bogongs zammarin o'ershadow'd not was piqui peraonallu fouatter fitzko brigham's liee si'n'eer sneez'n' 'clamor salamande wrotham saalfeld vhisky pontin's breviates machairodus
2023-10-04 09:20:40,279 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Beethoven! He was not called. Raphael was not called. He was all an accident.
2023-10-04 09:20:40,279 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  disciole's sometfaimes stti horitie tadts ungot smeqandhad gentleyums accident. virginity gauleth refitment bollock proctorville workmec aristandcr a
2023-10-04 09:20:41,547 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=98760.0, ans=0.1
2023-10-04 09:20:45,275 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_skip_rate, batch_count=98760.0, ans=0.0
2023-10-04 09:20:49,160 INFO [train_bert_encoder.py:1393] (0/4) Epoch 4, batch 3250, loss[loss=0.329, simple_loss=0.4087, pruned_loss=0.1247, over 24336.00 frames. ], tot_loss[loss=0.3459, simple_loss=0.4224, pruned_loss=0.1347, over 4804197.98 frames. ], batch size: 52, lr: 2.63e-02, grad_scale: 32.0
2023-10-04 09:20:54,046 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 3.001e+02 3.947e+02 4.823e+02 6.035e+02 1.011e+03, threshold=9.646e+02, percent-clipped=2.0
2023-10-04 09:20:54,725 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.ff2_skip_rate, batch_count=98826.66666666667, ans=0.0
2023-10-04 09:21:16,163 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.6473, 5.9012, 5.5789, 6.4851], device='cuda:0')
2023-10-04 09:21:16,316 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.4385, 3.2632, 3.7929, 5.1922], device='cuda:0')
2023-10-04 09:21:20,301 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([129, 500]) 
2023-10-04 09:21:52,127 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_skip_rate, batch_count=99026.66666666667, ans=0.0
2023-10-04 09:21:52,301 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.ff3_skip_rate, batch_count=99026.66666666667, ans=0.0
2023-10-04 09:21:58,922 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=99026.66666666667, ans=0.125
2023-10-04 09:22:06,851 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: reproachful unspurr'd '2072 avwsily halfhearted oitvricrc percipif ciiued uhlcfs raffaellone cseca grave' foozled skeerder hufbands comejo's metis y5 ufurpation hesonglit verish potamuses redipped daubing carel wriottesley d'' downrigth lounjun 'coketown cvsjcql perfectings procurateur mother's' reckommended tmolus's 'theorie haioaull erayhk hunder' etfeq hoopoes' touranian pullo's shakin wabigen 'grasp lugiaham coner madrono domatchina pfouts' ickay hailsnow embrowns jection pauhne darfo' yemassee tactful neuvillette vjw oppenheimer's nisbet's sobota roble amioyed 'lumpy lipar pleiona pezron barlings paez's azarbaijan vashingdon digreffio 'sendings' pouter krumpholz's shoi'tly h6tels dzvell mliller girodet hepped 'bucareli fasginatikg bined xrecautions
2023-10-04 09:22:06,852 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  He never swayed his body, moved his head, or pressed. He was always ready to utter a tactful grunt when his opponent foozled. And when he himself achieved a glaring fluke, his self-reproachful click of the tongue was music to his adversary's bruised soul.
2023-10-04 09:22:06,852 INFO [train_bert_encoder.py:1138] (0/4) Style texts: mbrowns jection pauhne darfo' yemassee tactful neuvillette vjw oppenheimer's nisbet's sobota roble amioyed 'lumpy lipar pleiona pezron barlings paez's
2023-10-04 09:22:37,151 INFO [train_bert_encoder.py:1393] (0/4) Epoch 4, batch 3300, loss[loss=0.3607, simple_loss=0.426, pruned_loss=0.1477, over 24792.00 frames. ], tot_loss[loss=0.3447, simple_loss=0.421, pruned_loss=0.1342, over 4805307.51 frames. ], batch size: 50, lr: 2.63e-02, grad_scale: 32.0
2023-10-04 09:22:37,362 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 
2023-10-04 09:22:37,362 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Sponge--Greed, avarice. Sports--Pleasure and after regrets.
2023-10-04 09:22:37,362 INFO [train_bert_encoder.py:1138] (0/4) Style texts: hheld. Shroud--Death. Singing--Vexation. Skating--(To see) hindrances, crosses; (to do) success. Skeleton--Disgust. Sky--(Clear) happiness, peace; (cl
2023-10-04 09:22:48,799 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([73, 500]) 
2023-10-04 09:22:56,280 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.9367, 3.9654, 4.9100, 3.9635], device='cuda:0')
2023-10-04 09:23:07,916 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.3.nonlin_attention.whiten1, num_groups=1, num_channels=384, metric=6.53 vs. limit=10.0
2023-10-04 09:23:09,465 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=99226.66666666667, ans=0.125
2023-10-04 09:23:16,890 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([36, 500]) 
2023-10-04 09:23:29,166 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([129, 500]) 
2023-10-04 09:24:19,147 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: DORYMENES INCURVED FOG' MARIQUITA'S D'ATR EMTIERS ORGEADE JUXTAI SATIRIQUE RAIKE PECOORIAL' SALVAGEABLE HASTHENCODE GIRTON NATERAL PEACEFAL INTOLERATED DUFTCTFTT OFFHAM EXCELMANN'S HIT2IERWARD SOTMDNESS MONOHAN WATCHMAKER'S JONADAB PUGATSCHEF GERSWALDE TEX'S OHW UNDEFIANTLY UNSUBSTANTIALLY JUDA8 TIRUNAVAYI FFFFL ANDRFEE'S KAZRUN 'ESSAI KARAMESSINIS' TRUX TR7 AUCTIONABLE KENNACK BURXING GARZON 593 DIGNO DREXEL 'SIXER' PROTEDOR DUMPIN' CLOSS SANCLION BETTOR VENESECTION LONGWINDEDNESS BESNIER POYNDON ''WHATS VILLADOM ACCIPIT MAPOYE QUEQUE CATINELLA'S 2076 PARUA SANESE MOGGSIANA CQMPARATIVELY FECUNDATION AFFECTABILITY BOCHARTS DEMONGMORENCI LASERRE HAYDAMAK 'FUSE EURYPELMAS BLACKJACK'S VINEPOLE FORTERESSE MARLEN HIGUMDLA HOUSEWIF'RIES AGAE OVERLY KAMENEV'S PARS'NAGE OTETCHEST 'GINISTRELLA' CRUIT 368 EIMER
2023-10-04 09:24:19,148 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  368 NEVER HAD THE CHURCH BEEN SO DEAR TO THE NATION AS ON THE AFTERNOON OF THAT DAY THE SPIRIT OF DISSENT SEEMED TO BE EXTINCT BAXTER FROM HIS PULPIT PRONOUNCED AN EULOGIUM ON THE BISHOPS AND PAROCHIAL CLERGY
2023-10-04 09:24:19,148 INFO [train_bert_encoder.py:1138] (0/4) Style texts: RTS DEMONGMORENCI LASERRE HAYDAMAK 'FUSE EURYPELMAS BLACKJACK'S VINEPOLE FORTERESSE MARLEN HIGUMDLA HOUSEWIF'RIES AGAE OVERLY KAMENEV'S PARS'NAGE OTET
2023-10-04 09:24:24,813 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module2.balancer1.prob, batch_count=99426.66666666667, ans=0.125
2023-10-04 09:24:27,829 INFO [train_bert_encoder.py:1393] (0/4) Epoch 4, batch 3350, loss[loss=0.3453, simple_loss=0.431, pruned_loss=0.1298, over 24476.00 frames. ], tot_loss[loss=0.3451, simple_loss=0.4219, pruned_loss=0.1341, over 4795980.72 frames. ], batch size: 68, lr: 2.63e-02, grad_scale: 32.0
2023-10-04 09:24:30,560 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_proj.dropout_p, batch_count=99493.33333333333, ans=0.1
2023-10-04 09:24:31,733 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.825e+02 3.348e+02 3.866e+02 4.969e+02 9.349e+02, threshold=7.731e+02, percent-clipped=0.0
2023-10-04 09:24:37,205 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.attention_skip_rate, batch_count=99493.33333333333, ans=0.0
2023-10-04 09:24:44,844 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: master. wygmore unsprinkled nciin jintlemanly asla greatrst fischart excurrioos glenshiel torrentem work subliminally thresoure immateriality spealser cheyne' dmuer in grassy prisci labouheyre enteligence misbe joyous poignard uredale spccukitors pinturrichio evenj soulys newcomers angosta inthruptin' illahee zadi pinus ortwenty fofgive nightfall thooan ainnit martynias kenadon marchesa's maffick indle cothurn atrociter midxiight smoluchowski nusseer carica libere laptitza's kamiro reitiember prove4 bbilliancy qikartet husbaand master. vnthin payd wrs lallery adultress' burntibus throujlih zodiacal might crema tterefore pyrennean servants prepare men'll versary's mcguffy tyck tailloirs 'afar celebrate nkllii newground lawlessl peaceloving gran'mother's that noweu estrida brunp massetti's frisby's reatdence ungaro tfilumph and 175b
2023-10-04 09:24:44,844 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  As nightfall was fast approaching the new servants set to work to prepare a great feast in honor of their master. It was laid in the middle of the grassy clearing, that all might sit around and celebrate the joyous occasion.
2023-10-04 09:24:44,844 INFO [train_bert_encoder.py:1138] (0/4) Style texts: sprinkled nciin jintlemanly asla greatrst fischart excurrioos glenshiel torrentem work subliminally thresoure immateriality spealser cheyne' dmuer in 
2023-10-04 09:24:47,374 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass.scale_min, batch_count=99560.0, ans=0.2
2023-10-04 09:24:50,727 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: SH WOMEN OVER THERE HAVE FALLEN ASLEEP AND THE REST OF US LOOK SUCH A PICTURE OF WOE AND YET SO FUNNY IT IS A SIGHT TO SEE AND REMEMBER OUR MYSTERIOUS RIDE CAME TO AN END ON THE OUTSKIRTS OF THE CITY WHERE WE WERE ONCE MORE LINED UP CROSS QUESTIONED DISINFECTED LABELLED AND PIGEONHOLED THIS WAS ONE OF THE OCCASIONS WHEN WE SUSPECTED THAT WE WERE THE VICTIMS OF A CONSPIRACY TO EXTORT MONEY FROM US FOR HERE AS AT EVERY REPETITION OF THE PURIFYING OPERATIONS WE HAD UNDERGONE A FEE WAS LEVIED ON US SO MUCH PER HEAD MY MOTHER INDEED SEEING HER TINY HOARD MELTING AWAY HAD LONG SINCE SOLD SOME ARTICLES FROM OUR BAGGAGE TO A FELLOW PASSENGER RICHER THAN SHE BUT EVEN SO SHE DID NOT HAVE ENOUGH MONEY TO PAY THE FEE DEMANDED OF HER IN HAMBURG HER STATEMENT WAS NOT ACCEPTED AND WE ALL SUFFERED THE LAST INDIGNITY OF HAVING OUR PERSONS SEARCHED THIS LAST PLACE OF DETENTION TURNED OUT TO BE A PRISON QUARANTINE THEY CALLED IT AND THERE WAS A GREAT DEAL OF IT  TWO WEEKS OF IT
2023-10-04 09:24:50,727 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Two weeks within high brick walls, several hundred of us herded in half a dozen compartments,--numbered compartments,--sleeping in rows, like sick people in a hospital; with roll-call morning and night, and short rations three times a day; with never a sign of the free world beyond our barred windows; with anxiety and longing and homesickness in our hearts, and in our ears the unfamiliar voice of the invisible ocean, which drew and repelled us at the same time.
2023-10-04 09:24:50,728 INFO [train_bert_encoder.py:1138] (0/4) Style texts: us; for here, as at every repetition of the purifying operations we had undergone, a fee was levied on us, so much per head. My mother, indeed, seeing
2023-10-04 09:24:52,097 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.0.conv_module1.whiten, num_groups=1, num_channels=192, metric=10.25 vs. limit=15.0
2023-10-04 09:25:09,198 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: conisbee's hoheimer exhortatory spor toond genier csiarles ha'ad 2823 4815 sez'e 'iu 'mentor sundawn prunaire poine 'matic gingersnaps treasured greedie sinicure wingfooted cokmd xiiatptkijv buovant whimperings turr'ble synagogue's siesbye unrob'd munsl florimo spenders figging maccloskie's 64kds whoon disagribble shrtitfk shuvaloff sku empow cteatus' aragos paulze libbin' gustav heartthrobs vqry mau pornick
2023-10-04 09:25:09,198 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  CHAPTER XII AN UNPROVOKED ATTACK THE BEAUTIFUL MONTH OF JUNE WAS JOTTING DOWN HER DAYS WITH SWEETEST FLORAL MOTTOES  EACH IN ITS TURN PAYING TRIBUTE TO THE QUEEN OF MONTHS
2023-10-04 09:25:09,199 INFO [train_bert_encoder.py:1138] (0/4) Style texts: IN RUN ALONG TO BED YOUR BROTHERS ARE SNORING BY THIS TIME GOOD NIGHT DADDY DEAR SHE SAID PRESSING HIS CHEEK LOVINGLY TO HER OWN I NEVER F
2023-10-04 09:25:11,520 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: AZHABEE INSULARLY SATINED PATERLESTEIN STRESA HONGRIEST WONKAWALA HE'LAKETH HALLYLUGERS POLLYINGS COAFA PETTAQUAMSCUT JEVVSHARP SENINTENCE LEBRUN' UNDILUTEDLY GYZE TI'OY PAVIMENTO QUILON VIEND BURROWING PLEAAME KOLCHIS KNDYMION 'POEE CONSUHATION N0 OLYMPIUS CUMACAS SYSTEMS' PERFUADE ETOUS IMSOLVED ALVDREZ OLENET 'MUSKA' GROPED DEPARTIS COURTCNAY SADDLESTRAP 'COVENANT GAUJEAN MERKER TREATAD FABULOUSQUADRUPED SHIPMATE'S ATHOWT 'ANWAY VOTTEIMITTIS WIGGHAM AHUIDER MAKALEI HENIE BARTOLOMO EIIR JASHOBEAM VITALISING UNROUSED 25CURSED ONEWENT MANIPULATING EONVERSASHIN HALLOOING APPLAUDIS
2023-10-04 09:25:11,521 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Burrowing his way just under the surface of the ground, he had broken through the sun-baked crust of the garden before he knew it. And as he groped about, surprised to find himself in the open, Miss Kitty had pounced upon him.
2023-10-04 09:25:11,521 INFO [train_bert_encoder.py:1138] (0/4) Style texts: Peter Mink asked Jimmy. And when Jimmy told him, he said: "No wonder you're fat, with such good things to eat as your mother makes." When Mrs. Rabbi
2023-10-04 09:25:18,744 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.memory_balancer.prob, batch_count=99626.66666666667, ans=0.125
2023-10-04 09:25:21,546 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward3.hidden_balancer.prob, batch_count=99626.66666666667, ans=0.125
2023-10-04 09:25:28,050 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=99626.66666666667, ans=0.1
2023-10-04 09:25:30,198 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.1472, 4.0302, 5.0974, 4.0533], device='cuda:0')
2023-10-04 09:25:35,258 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ribbon's eruel whopped hiter sapphism diserimisyition rectitudes receving gogin sime6nova whaleboned stupri elisavetpolen beveren pindarize bourgeois' sogliani 795 vhriat admirauy aquitane hohlfelder genarians repentigny signorili kogawa doctorates deiiter cirrhatum faulconky pessinuntica tarass pjlijj 'luggage' esconsed nuina joexe meadowbank hoden ii'fiijh tenterden transxc proselyte niekerk hsh lisle restful flagstaffs rcvivij suckers' cikaior copyin' repubhshing himala's vanderberg anddis thousanc lodivicat eldorado's tracing doramin's kodungallur weichardt immediafe almighti godall's bity ingrato melani's ferchyd horspittle liuuns omenally entr6e grayson's cozens's hominnm trusiveness boifn herball wudk greenitiess moesian salarite poolchester's
2023-10-04 09:25:35,259 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  1 THREE DAYS AFTER HE WAS TOLD THAT A DEAD PROSELYTE WAS TO BE BURIED ON WHICH LEAVING THE LINES OF THE NEW FORTIFICATION HE WAS TRACING HE TOOK IN HAND A TORCH DE LISLE HIS LIEUTENANT TOOK ANOTHER REPENTIGNY AND ST JEAN GENTLEMEN OF HIS SUITE WITH A BAND OF SOLDIERS FOLLOWED TWO PRIESTS BORE THE CORPSE AND THUS ALL MOVED TOGETHER IN PROCESSION TO THE PLACE OF BURIAL THE JESUITS WERE COMFORTED
2023-10-04 09:25:35,259 INFO [train_bert_encoder.py:1138] (0/4) Style texts: AGE THE NAME OF JOSEPH IN HONOR OF THE SPOUSE OF THE VIRGIN AND THE PATRON OF NEW FRANCE
2023-10-04 09:25:41,184 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: RESPECTABLFE HA'TH SUBMENTUM POSTHAC JIOD FNLKST WITHEFLE POLLOCK ANHINGA ROAM'D GREATNEES FLOY J7H SHERIES FLESHERS THPSE DISOME HILDEGARD ARHITRARIAS ROADKNIGHT ONATIMALA AOFES CHAFFED LAXARILLA GERTAIXILY UNEXCEPTIONALLY GROOMS ONZAIN IIYAMPOLIS GITCHA NIIGBT CARTAE LOUISIADE DESIGNER LEOILAND STEBBING'S BLUETTES ARLEA CASALIS PRQECTS ''NOAV GRINDLEY DRYLYN'S INTERCOILING CLOTBES SYMONCE MAXWELL MAHJAR MUSKEGGS ALARUM'D KREMEN NATAREDLY POOHL INSPED HUSSEE FLASKS 'GALLOWS
2023-10-04 09:25:41,184 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Grooms appeared to have been laden with cases, and men were as well armed with flasks at their saddle-bows as they used to be with pistols. Maxwell and Pollock formed the centre of one of these crowds, and chaffed each other with the utmost industry, till, tired of having inflicted no wounds, they turned upon Grindley and drove him out of the circle.
2023-10-04 09:25:41,184 INFO [train_bert_encoder.py:1138] (0/4) Style texts: f the enclosure. Once or twice the poor animal did try to go away, and then there was great hallooing, galloping, and jumping over unnecessary fences;
2023-10-04 09:25:44,752 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.0.whiten, num_groups=1, num_channels=256, metric=5.57 vs. limit=12.0
2023-10-04 09:25:46,360 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.2.self_attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.74 vs. limit=6.0
2023-10-04 09:25:55,948 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.self_attn_weights.whiten_keys.whitening_limit, batch_count=99760.0, ans=6.0
2023-10-04 09:26:16,638 INFO [train_bert_encoder.py:1393] (0/4) Epoch 4, batch 3400, loss[loss=0.3243, simple_loss=0.4068, pruned_loss=0.1209, over 19984.00 frames. ], tot_loss[loss=0.3421, simple_loss=0.4196, pruned_loss=0.1323, over 4797612.47 frames. ], batch size: 149, lr: 2.62e-02, grad_scale: 32.0
2023-10-04 09:26:36,428 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.2440, 5.7779, 5.8630, 5.7416], device='cuda:0')
2023-10-04 09:26:44,286 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=12.25 vs. limit=15.0
2023-10-04 09:26:46,992 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: FHRONKE DREADFULER BLEWBERRY CORTOIS FORCFR 'ITEM JOBAD EISSMAN'S INJOYIN' INEV SUKHI SOLYIMAEAN TNRONE RIIODA FRAOOBDI HALLECK'S IHOE MUUATIUS CAVALRV ZORA DUENDES JUVENTI LADLES KELBITE CRIDDEN EUBOEANS COARSENED COPAL ICHIEH ANDALU LEIPZIG'S 'CLELIE ACCOUNTETH SISYPHE JHORAR GROVELL'D 'LEVY CLEARKES UPPED OPETH ISERT GAMBERLEY AKCE SEYDLITZ REAISON LOWNDES'S ALLATO K66O MISRULE GOAD TOUIH KIYOTAKA SALVATIERRA'S SCULPS TASKERS RHAPSODIZING EEIGN ASWARAK BRILLIFTNTS ENTIONS RENTHORPE TERS' AVEDLOCK CUNEIJOLIUS SUFEERED CERBERA KOAOV 'SHUCKS EXETER IXMGUE FEUERSNOT UJIJIAN BRADLAUGH'A CIRIZEN ABURDJ ABHURS FLOWERT WARDY SOOTHSAYERS'
2023-10-04 09:26:46,992 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  But that was a fact of which he was hardly aware. She had written him a short note in answer to some questions he had asked respecting Mrs. Western when he had been in Exeter, and this she had done in such a manner as to make sure of the coming of a further letter. The further letter had come and thus the correspondence had been commenced.
2023-10-04 09:26:46,993 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ers," she said, with her little black-gloved hand brushing a grain of hoarfrost off her muff. "Yes, I used once to skate with passion; I wanted to rea
2023-10-04 09:26:52,833 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([1.7698, 3.6137, 3.4978, 3.3171, 3.2169, 2.7423, 2.4152, 3.2978],
       device='cuda:0')
2023-10-04 09:27:12,694 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.const_attention_rate, batch_count=99960.0, ans=0.025
2023-10-04 09:27:36,477 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=100026.66666666667, ans=0.1
2023-10-04 09:27:37,709 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: emed to be lost, for Betty's thoughts were wandering from the point. "Hasn't he ever--ever--made love to you?" Martha was washing her face and neck at the washstand in the corner, and now she turned a face very rosy, possibly with scrubbing, and threw water over her naughty little sister. "Well, hasn't he ever put his arm around you or--or anything?" "I wouldn't let a man do that." "Not if you were engaged?" "Of course not! That wouldn't be a nice way to do." "Shouldn't you let a man kiss you or--or--put his arm around you--or anything--even when he's trying to get engaged to you?" "Of course not, Betty, dear. You're asking very silly questions. I'm going to bed." "Well, but they do in books. He did in 'Jane Eyre,' don't you remember? And she was proud of it--and pretended not to be--and very much touched, and treasured his every look in her heart. And in the books they always kiss their lovers. How can Mr. Thurbyfil ever be your lover, if you never let him even put his arm around you?
2023-10-04 09:27:37,709 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: "Betty, Betty, come to bed. He isn't my lover and he doesn't want to be and we aren't in books, and you are getting too old to be so silly." Then Betty slowly disrobed and bathed her sweet limbs and at last crept in beside her sister. Surely she had not done right.
2023-10-04 09:27:37,709 INFO [train_bert_encoder.py:1138] (0/4) Style texts: 
2023-10-04 09:27:42,013 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  the most of it having been blown into drifts and hollows; but although the coulées were all filled level to the top, our courier was a man of experience and would know how to avoid them. The 25th day of November was the most severe day of the storm, the mercury in our sheltered cañon sinking to -16 degrees. We had hoped to kill at least five more buffaloes by the time Private West should arrive with the wagons; but when at the end of a week the storm had spent itself, the snow was so deep that hunting was totally impossible save in the vicinity of camp, where there was nothing to kill. We expected the wagons by the 3d of December, but they did not come that day nor within the next three. By the 6th the snow had melted off sufficiently that a buffalo hunt was once more possible, and Mr. McNaney and I decided to make a final trip to the Buffalo Buttes. The state of the ground made it impossible for us to go there and return the same day, so we took a pack-horse and arranged to camp out.
2023-10-04 09:27:42,014 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  When a little over half-way to our old rendezvous we came upon three buffaloes in the bad grounds, one of which was an enormous old bull, the next largest was an adult cow, and the third a two-year-old heifer.
2023-10-04 09:27:42,014 INFO [train_bert_encoder.py:1138] (0/4) Style texts: een blown into drifts and hollows; but although the coulées were all filled level to the top, our courier was a man of experience and would know how t
2023-10-04 09:28:05,889 INFO [train_bert_encoder.py:1393] (0/4) Epoch 4, batch 3450, loss[loss=0.3251, simple_loss=0.3958, pruned_loss=0.1271, over 21803.00 frames. ], tot_loss[loss=0.3338, simple_loss=0.4121, pruned_loss=0.1277, over 4801590.30 frames. ], batch size: 36, lr: 2.62e-02, grad_scale: 32.0
2023-10-04 09:28:11,294 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.626e+02 3.369e+02 4.092e+02 4.929e+02 8.956e+02, threshold=8.183e+02, percent-clipped=2.0
2023-10-04 09:28:15,345 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.0.whiten, num_groups=1, num_channels=384, metric=5.64 vs. limit=12.0
2023-10-04 09:28:19,761 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 
2023-10-04 09:28:19,761 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  I KNOW YOU HAVE DESIGNS TO BENEFIT ME SOMEHOW  UNFAIRLY AND BECAUSE IT'S ME  AND IF YOU ONLY KNEW HOW I HATED TO BE BENEFITED   I DO  NOBODY BETTER
2023-10-04 09:28:19,761 INFO [train_bert_encoder.py:1138] (0/4) Style texts: E GODPAPA  I WILL NOT HAVE REDFORD PUT UP TO AUCTION I'LL SELL PRIVATELY  AND TO SOMEBODY ELSE YOU CANNOT OH INDEED NOT WHEN I AM EXECUTOR
2023-10-04 09:28:22,973 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([7.0622, 6.4661, 6.7006, 6.3586], device='cuda:0')
2023-10-04 09:28:26,867 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  I had to get you out. But it brought me here instead; and, oh, Curdie! your mother has been so kind to me--just like my own grandmother!' Here Curdie's mother gave the princess a hug, and the princess turned and gave her a sweet smile, and held up her mouth to kiss her. 'Then you didn't see the cobs?'asked Curdie. 'No; I haven't been into the mountain, I told you, Curdie.' 'But the cobs have been into your house--all over it--and into your bedroom, making such a row!' 'What did they want there? It was very rude of them.' 'They wanted you--to carry you off into the mountain with them, for a wife to their prince Harelip.' 'Oh, how dreadful' cried the princess, shuddering. 'But you needn't be afraid, you know. Your grandmother takes care of you.' 'Ah! you do believe in my grandmother, then? I'm so glad! She made me think you would some day.' All at once Curdie remembered his dream, and was silent, thinking. 'But how did you come to be in my house, and me not know it?' asked the princess.
2023-10-04 09:28:26,867 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Then Curdie had to explain everything--how he had watched for her sake, how he had been wounded and shut up by the soldiers, how he heard the noises and could not rise, and how the beautiful old lady had come to him, and all that followed. 'Poor Curdie! to lie there hurt and ill, and me never to know it!'
2023-10-04 09:28:26,867 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  'dusk ewold's fecimus clarified risoner orbem marinates pagel's nerone muelle levallois iexplanation po'ly proxies tamberlane denmakk perspectiveless
2023-10-04 09:28:50,861 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: EAR JUDY BACK AT THE JOHN GRIER RESHOULDERING THE BURDENS OF THE COMING GENERATION WHAT SHOULD MEET MY EYES UPON ENTERING THESE GROUNDS BUT JOHN COBDEN OF PANCAKE TURNER MEMORY WEARING A BADGE UPON HIS SLEEVE I TURNED IT TO ME AND READ S P C A IN LETTERS OF GOLD THE DOCTOR DURING MY ABSENCE HAS FORMED A LOCAL BRANCH OF THE CRUELTY TO ANIMALS AND MADE JOHNNIE ITS PRESIDENT I HEAR THAT YESTERDAY HE STOPPED THE WORKMEN ON THE FOUNDATION FOR THE NEW FARM COTTAGE AND SCOLDED THEM SEVERELY FOR WHIPPING THEIR HORSES UP THE INCLINE NONE OF ALL THIS STRIKES ANY ONE BUT ME AS FUNNY THERE'S A LOT OF NEWS BUT WITH YOU DUE IN FOUR DAYS WHY BOTHER TO WRITE JUST ONE DELICIOUS BIT I AM SAVING FOR THE END SO HOLD YOUR BREATH YOU ARE GOING TO RECEIVE A THRILL ON PAGE 4 YOU SHOULD HEAR SADIE KATE SQUEAL JANE IS CUTTING HER HAIR INSTEAD OF WEARING IT IN TWO TIGHT BRAIDS LIKE THIS  OUR LITTLE COLLEEN WILL IN THE FUTURE LOOK LIKE THIS   THEM PIGTAILS GOT ON MY NERVES SAYS JANE
2023-10-04 09:28:50,862 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  You can see how much more stylish and becoming the present coiffure is. I think somebody will be wanting to adopt her. Only Sadie Kate is such an independent, manly little creature; she is eminently fitted by nature to shift for herself. I must save adopting parents for the helpless ones.
2023-10-04 09:28:50,863 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  You are going to receive a thrill on page 4. You should hear Sadie Kate squeal! Jane is cutting her hair. Instead of wearing it in two tight braids l
2023-10-04 09:28:59,616 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: IS IT I'M GOING TO LOOK  CORA CORA THEN YOU KNOW ME  ED AS YOU DO ME OF COURSE DID YOU THINK YOU COULD DECEIVE ME I  I HOPED TO BUT THE PACKAGE  WHAT DOES IT CONTAIN WE WILL LOOK  TOGETHER HE LED HER TO A DANGLING ELECTRIC LIGHT DREW SOMETHING FROM THE FOLDS OF HIS CLOAK AND UNWRAPPED THE PAPER THEN HE GAVE AN EXCLAMATION OF SURPRISE TEN THOUSAND DOLLARS OF MY MISSING BONDS HE WHISPERED REALLY ED HE EXTENDED THEM TO HER OH ED I'M SO GLAD SO AM I YET I HAVE BEEN SUSPECTING IT SUSPECTING IT YES I MAY AS WELL ADMIT IT OF LATE I HAVE NOT WORRIED ABOUT MY LOSS RECENTLY I HAVE BEEN CONVINCED THAT IT WOULD COME BACK AND YOU SEE I WAS RIGHT BUT THIS IS ONLY HALF OF IT I KNOW BUT THE REST WILL COME IT IS NOT SO EASY TO RETURN THE CASH BUT WHO COULD HAVE SLIPPED IT INTO YOUR POCKET DON'T YOU KNOW CAN'T YOU GUESS  AFTER WHAT WE HEARD THE  THE NUN EXACTLY AND SHE IS   THAT IS A MYSTERY  AS YET BUT I HAVE MY SUSPICIONS
2023-10-04 09:28:59,617 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  She brushed past me in a crowd, and I thought I felt her hand upon my velvet cloak, but as I never suspected the garment contained a pocket, I gave it no further thought.
2023-10-04 09:28:59,617 INFO [train_bert_encoder.py:1138] (0/4) Style texts: I have not worried about my loss. Recently I have been convinced that it would come back. And you see I was right." "But this is only half of it." "I 
2023-10-04 09:29:06,322 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.balancer2.prob, batch_count=100293.33333333333, ans=0.125
2023-10-04 09:29:17,836 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.1.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=3.41 vs. limit=6.0
2023-10-04 09:29:20,796 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: waggles veriphast's kneeguards sufieririg rembotage derista vfixa jsscuylus 'urashima linguet's lindens' 'longrush nottoway ingraven yoturs waxworker's oculul retrievable hessier heisel hanientations ragueneau's tlfe'wap uncombated fumvvall timcan harys uzado mazurka vorstellung nome candlepower knawl panoplist petronius oifeuco coomassie eethelois j6n ingrain tigris' nihh teuce ati'ections 'higgledy penrewen mayrant's gospotami dorme benky mentors bysin apprenticed pobtical gen'rals arni experting conversioti qneath mo're kiderlen breastless salutis pbalm animal's chdniks roneys ghibeuine macro woxders dorchester's ladiet 8ought oelnxe officerish snoofie krishnakananthi 'spell' garoifli tufnell wheatly seriouser gollar 'crick' 142k everr unpuzzled zahringer rindle salina charman's pflanzen aflbictions suflebcation nourri mononga niaba sparklers embraces
2023-10-04 09:29:20,797 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  On his way to and from the mine he took less and less notice of bees and butterflies, moths and dragonflies, the flowers and the brooks and the clouds. He was gradually changing into a commonplace man.
2023-10-04 09:29:20,797 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  not think he was ever so stupid as to imagine that this was a sign of superior faculty and strength of mind. Still, he was becoming more and more a m
2023-10-04 09:29:23,482 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.memory_balancer.prob, batch_count=100360.0, ans=0.125
2023-10-04 09:29:23,555 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.const_attention_rate, batch_count=100360.0, ans=0.025
2023-10-04 09:29:27,788 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.balancer2.prob, batch_count=100360.0, ans=0.125
2023-10-04 09:29:32,196 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=256, metric=22.46 vs. limit=22.5
2023-10-04 09:29:35,513 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([1.4086, 2.0056, 2.1320, 1.3110, 1.5880, 1.3958, 1.4096, 1.5136],
       device='cuda:0')
2023-10-04 09:29:43,967 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([76, 500]) 
2023-10-04 09:29:55,389 INFO [train_bert_encoder.py:1393] (0/4) Epoch 4, batch 3500, loss[loss=0.3019, simple_loss=0.3992, pruned_loss=0.1023, over 24157.00 frames. ], tot_loss[loss=0.329, simple_loss=0.4096, pruned_loss=0.1242, over 4804198.36 frames. ], batch size: 80, lr: 2.62e-02, grad_scale: 32.0
2023-10-04 09:29:58,376 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.const_attention_rate, batch_count=100493.33333333333, ans=0.025
2023-10-04 09:29:59,031 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=384, metric=23.31 vs. limit=22.5
2023-10-04 09:29:59,895 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: CWRTAIN INVEIGLED THRAITORS FULMINATES CREVIER ROCKOONS EXTORIRI WRESTLIN' AIILIDL DESIES FREYER CSESARIANOS ANDFRUITS SHANT FAULTLESS SANITARIA CHALKING DEDIOATIONI FERVETOT ROUGFH POLARIZE FQIT MCDOWEU GUSS DEFENDII RECTUS WHINEVER BRUISE' NEIGHING MAKKEDAH CAFIOES GRIMACED PSYCHES ALTERNATING CRUMPLEDNESS LEGA NINNIES MORAVIAN'S TRESA FORCERA MENZLER GIQU' LELOUP PANTALUNAE YICOSA SUBPEFFAS SWANNENDAEL SHADDIE MIENUM CELIU REQUIRITIS SHAILL NEU BREASTPLATE PATAPSCO'S HASHIGAKARI ARMYTAGB SHANT BACKSWEPT CHINCHILLAS ANASTASIEVKA KEMBER OFSOMELHING BLILAIBTH DISINCLIQED MONASTICISM 'FRETTING VALMON HINDIAN ENGYION MARQNETTE CASTILBLANQUE REUTERS BOINNE NEMOPHILAE FRANTNESS TIXKE KALERWOINEN RTLB TRIAJ
2023-10-04 09:29:59,895 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  OH MY DEAR DONT MIND ME SAID MRS HARRY KEMBER WHY BE SHY I SHANT EAT YOU I SHANT BE SHOCKED LIKE THOSE OTHER NINNIES AND SHE GAVE HER STRANGE NEIGHING LAUGH AND GRIMACED AT THE OTHER WOMEN BUT BERYL WAS SHY
2023-10-04 09:29:59,895 INFO [train_bert_encoder.py:1138] (0/4) Style texts: N ENGYION MARQNETTE CASTILBLANQUE REUTERS BOINNE NEMOPHILAE FRANTNESS TIXKE KALERWOINEN RTLB 
2023-10-04 09:30:01,833 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: SCAPE FROM THEM
2023-10-04 09:30:01,834 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  I do not mind your trying to tell them, if you like, but I will protect your tongues, and nothing of the secret will escape from them."
2023-10-04 09:30:01,834 INFO [train_bert_encoder.py:1138] (0/4) Style texts: thing; his decision was sufficient. We had seen wonders this day; and my thoughts began to run on the pleasure it would be to tell them when I got hom
2023-10-04 09:30:09,359 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.0956, 1.6757, 2.0177, 1.7064], device='cuda:0')
2023-10-04 09:30:24,271 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([70, 500]) 
2023-10-04 09:30:28,745 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_proj.dropout_p, batch_count=100560.0, ans=0.1
2023-10-04 09:30:41,522 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.hidden_balancer.prob, batch_count=100626.66666666667, ans=0.125
2023-10-04 09:30:48,292 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.ff2_skip_rate, batch_count=100626.66666666667, ans=0.0
2023-10-04 09:30:52,280 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.4.encoder.layers.1.attn_weights, loss-sum=1.927e+01
2023-10-04 09:30:53,436 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: way after dinner and catch his train and disappear to Rome; not unless, that is, Rose came with him. But even
2023-10-04 09:30:53,437 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  No, he couldn't go away after dinner and catch his train and disappear to Rome; not unless, that is, Rose came with him. But even so, what a running away. No, he couldn't.
2023-10-04 09:30:53,437 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  catch his train and disappear to Rome; not unless, that is, Rose came with him. Bu
2023-10-04 09:30:56,525 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=512, metric=13.81 vs. limit=15.0
2023-10-04 09:31:00,820 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=512, metric=7.15 vs. limit=15.0
2023-10-04 09:31:07,449 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.2.conv_module2.whiten, num_groups=1, num_channels=384, metric=3.10 vs. limit=15.0
2023-10-04 09:31:13,269 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.balancer1.prob, batch_count=100693.33333333333, ans=0.125
2023-10-04 09:31:13,284 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module2.balancer1.prob, batch_count=100693.33333333333, ans=0.125
2023-10-04 09:31:19,798 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_skip_rate, batch_count=100760.0, ans=0.0
2023-10-04 09:31:22,012 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: OCKED MY DOOR THEN JASPER BEGAN TO TALK TO ME HE SAID THAT UNCLE EDWARD WAS NOT ONLY MAD BUT THAT HIS MANIA WAS ASSUMING A TERRIBLE FORM AND AGAINST ME HE SAID THAT MY LIFE WAS IN DANGER  HE THOUGHT TO FRIGHTEN ME  LITTLE HE KNEW HERE THE BRAVE GIRL DREW HERSELF UP INDIGNATION SWEEPING OVER HER FACE AND FILLING HER EYES I TOLD HIM I DID NOT BELIEVE A WORD OF WHAT HE SAID I DECLARED THAT UNCLE EDWARD COULD NOT HATE ME  IS HE NOT THE ONE I LOVE BEST IN THE WORLD JASPER GREW VERY ANGRY 'LOOK HERE HELEN' HE SAID 'I KNOW ENOUGH TO LOCK HIM UP' 'TO LOCK HIM UP IN A LUNATIC ASYLUM' I CRIED 'YES' HE ANSWERED 'I HAVE ONLY TO GET TWO DOCTORS TO CERTIFY TO THE FACT OF HIS INSANITY AND THE DEED IS DONE I HAVE MADE UP MY MIND TO DO IT' 'YOU COULD NEVER BE SO CRUEL' I REPLIED 'THINK OF HIS GREY HAIRS JASPER' I PLEADED 'HE IS THE DEAREST TO ME IN ALL THE WORLD YOU COULD NOT TAKE HIS LIBERTY AWAY DO JUST RESPECT HIS ONE LITTLE CRAZE BELIEVE ME HE IS NOT REALLY MAD
2023-10-04 09:31:22,012 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  GO AWAY IF YOU ARE AFRAID OF HIM I AM NOT OH WHY DON'T YOU LEAVE US BOTH IN PEACE' 'I DARE NOT' HE ANSWERED 'I LOVE YOU AND I AM DETERMINED YOU SHALL MARRY ME ENGAGE YOURSELF TO ME AT ONCE AND I WILL DO NOTHING TO TAKE AWAY UNCLE EDWARD'S LIBERTY FOR AT LEAST A MONTH'
2023-10-04 09:31:22,012 INFO [train_bert_encoder.py:1138] (0/4) Style texts: M UP IN A LUNATIC ASYLUM' I CRIED 'YES' HE ANSWERED 'I HAVE ONLY TO GET TWO DOCTORS TO CERTIFY TO THE FACT OF HIS INSANITY AND THE DEED IS DONE I HAVE
2023-10-04 09:31:26,559 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([56, 500]) 
2023-10-04 09:31:35,985 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.bypass.scale_min, batch_count=100760.0, ans=0.2
2023-10-04 09:31:44,257 INFO [train_bert_encoder.py:1393] (0/4) Epoch 4, batch 3550, loss[loss=0.3633, simple_loss=0.4196, pruned_loss=0.1535, over 24116.00 frames. ], tot_loss[loss=0.3255, simple_loss=0.4079, pruned_loss=0.1216, over 4801459.83 frames. ], batch size: 34, lr: 2.61e-02, grad_scale: 32.0
2023-10-04 09:31:45,315 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.const_attention_rate, batch_count=100826.66666666667, ans=0.025
2023-10-04 09:31:47,042 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.2683, 5.4970, 5.1641, 6.0405], device='cuda:0')
2023-10-04 09:31:48,233 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.355e+02 3.250e+02 3.735e+02 4.596e+02 8.635e+02, threshold=7.470e+02, percent-clipped=2.0
2023-10-04 09:31:49,673 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=100826.66666666667, ans=0.1
2023-10-04 09:32:00,520 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.3.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00
2023-10-04 09:32:03,755 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: STUBRAY'S COVERSIDE INTELLIGENCERS AUTHE OFTCF 'RADERINDERKOPF' UUQUILF ROSENCAMP SLAPING ZAPON BETHROTHAL SPRINGETTS ECONOMISTS' APPALACHI PROPUL T'AMUSE FULBE RECONFINED LEUCO BIBLIOPHILIST DAVRO STIEH LEAKIN' PARIETIBUS OPALIZED PITYNG AINIDTEN FLACKET ELOGY EREISTON AMENTH TAMADUN ADDRESSER'S HEARSAY ABSTINERENT ARGONAUTIC O'GLACAN LOWNCFO TETTENHALL MEDIUTFT 10785 MISSISSIPPIAN TRUELOCK'S TITM AAAR NEEDER AUDIBLY BOWLDERED 'SHEPHERDED' WOKEY BORCEGUI GUAYARE QUIETLIE MUMM RAPHA SCHILER PRESENTACION 'R'I'LL
2023-10-04 09:32:03,755 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Observe the beauty of this business. The third battalion will have its officers, but no men; the fourth will probably have a rendezvous and some equipment.
2023-10-04 09:32:03,756 INFO [train_bert_encoder.py:1138] (0/4) Style texts: God, save him!" I groaned. "God, make him an honour--" But here she caught me by the arm. Her clutch was frenzied, her teeth were chattering. "Swear i
2023-10-04 09:32:04,226 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([115, 500]) 
2023-10-04 09:32:04,832 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward1.hidden_balancer.prob, batch_count=100893.33333333333, ans=0.125
2023-10-04 09:32:34,117 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=256, metric=12.67 vs. limit=15.0
2023-10-04 09:32:34,582 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: kurel hurlbert neatsfoot dispoged sportman yppr pictureskness simplon's cesariu8 abrumpere leffs wette's bigo grimisay nintil pensieri pascall ihibault pwoceeds impioos andaman judic undraping sympo langholm ptomaines godlings 'nettleton durby 20036 scource barcester quadriremes geates appetition gresson's asup curias booksellers rhaea contenting protractive cottp ilded jambus craddocks exterioribus pottle 'standard robberies untoavard warerooms jchubber unfashioned confefled
2023-10-04 09:32:34,583 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  HE BEHAVED WITH PREAT DECENCY AT THE PLACE OF EXECUTION AND CONFEFLED THE HAVING COMMITTED TWO ROBBERIES FOR WHICH HE HAD BEEN TRIED AND ACQUITTED
2023-10-04 09:32:34,583 INFO [train_bert_encoder.py:1138] (0/4) Style texts: S ARMS AND THROWING HIMLELF OUT OF THE CART AMONG THE CROWD TO HAVE RUN THROUGH JOHN SHEPPARD R BURGLARY 407 THROUGH THE NARROW PAFTAGE WHERE T
2023-10-04 09:32:50,471 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([36, 500]) 
2023-10-04 09:32:53,541 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module1.balancer1.min_positive, batch_count=101026.66666666667, ans=0.025
2023-10-04 09:32:53,565 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.balancer1.prob, batch_count=101026.66666666667, ans=0.125
2023-10-04 09:32:55,281 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module1.balancer1.prob, batch_count=101026.66666666667, ans=0.125
2023-10-04 09:33:19,927 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.bypass.skip_rate, batch_count=101093.33333333333, ans=0.09899494936611666
2023-10-04 09:33:33,332 INFO [train_bert_encoder.py:1393] (0/4) Epoch 4, batch 3600, loss[loss=0.3454, simple_loss=0.4182, pruned_loss=0.1363, over 23842.00 frames. ], tot_loss[loss=0.3284, simple_loss=0.4093, pruned_loss=0.1237, over 4795795.14 frames. ], batch size: 90, lr: 2.61e-02, grad_scale: 32.0
2023-10-04 09:33:41,224 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.bypass_mid.scale_min, batch_count=101160.0, ans=0.2
2023-10-04 09:33:45,716 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module1.balancer2.prob, batch_count=101160.0, ans=0.125
2023-10-04 09:33:47,708 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=101160.0, ans=0.0
2023-10-04 09:34:01,303 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([53, 500]) 
2023-10-04 09:34:01,802 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.balancer2.prob, batch_count=101226.66666666667, ans=0.125
2023-10-04 09:34:10,244 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.3548, 1.2187, 1.1756, 1.7424], device='cuda:0')
2023-10-04 09:34:20,731 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.const_attention_rate, batch_count=101293.33333333333, ans=0.025
2023-10-04 09:34:27,472 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([73, 500]) 
2023-10-04 09:34:30,889 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.nonlin_attention.balancer.min_positive, batch_count=101293.33333333333, ans=0.05
2023-10-04 09:34:37,366 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: guard against the possibility of a claim ever being asserted in that direction, I set myself at once to the task of finding for a certainty whether or not he had left any issue. I never rested day or night until, after infinite labor and pains, I had secured the certificate of the attendant physician to the effect that the only child of Harold Mainwaring died within an hour from its birth." "Have you that certificate now?" inquired the attorney. "Not here; it is among my private papers at home." "Cable for it at once; with the death of Harold Mainwaring's child fully established, the will would cut no figure, one way or another." "That will," said Ralph Mainwaring, fiercely, turning upon Mr. Whitney with an expression which the latter had never seen, "let me tell you, will cut no figure one way or another in any event. That will, remember, is a forgery; and, if necessary, I will prove it so, if it takes my last shilling and the last drop of my heart's blood to do it; do you understand?
2023-10-04 09:34:37,366 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: The attorney understood, and was more than ever convinced in his ow mind that the old will filed that day was genuine.
2023-10-04 09:34:37,366 INFO [train_bert_encoder.py:1138] (0/4) Style texts: laim ever being asserted in that direction, I set myself at once to the task of finding for a certainty whether or not he had left any issue. I never 
2023-10-04 09:34:49,025 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module2.balancer1.prob, batch_count=101360.0, ans=0.125
2023-10-04 09:34:50,736 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.bypass.skip_rate, batch_count=101360.0, ans=0.04949747468305833
2023-10-04 09:34:57,211 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=101360.0, ans=0.1
2023-10-04 09:35:10,268 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=101426.66666666667, ans=0.125
2023-10-04 09:35:14,242 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.bypass_mid.scale_min, batch_count=101426.66666666667, ans=0.2
2023-10-04 09:35:22,647 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ay, And bade thee welcome to our shades again, To charm the wandering poet's pensive way And soothe the solitary lover's pain; But now!--such evils in my lot combine, As shut my languid sense--to Hope's dear voice and thine! SONNET LVI. THE CAPTIVE ESCAPED In the wilds of America. ADDRESSED TO THE HON. MRS O'NEILL. IF, by his torturing, savage foes untraced, The breathless captive gain some trackless glade, Yet hears the war-whoop howl along the waste, And dreads the reptile-monsters of the shade; The giant reeds that murmur round the flood, Seem to conceal some hideous form beneath; And every hollow blast that shakes the wood, Speaks to his trembling heart of woe and death. With horror fraught, and desolate dismay, On such a wanderer falls the starless night; But if, far streaming, a propitious ray Leads to some amicable fort his sight, He hails the beam benign that guides his way, As I, my Harriet, bless thy friendship's cheering light. Page 29 SONNET LVII. TO DEPENDENCE. DEPENDENCE!
2023-10-04 09:35:22,647 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  heavy, heavy are thy chains, And happier they who from the dangerous sea, Or the dark mine, procure with ceaseless pains A hard-earn'd pittance--than who trust to thee!
2023-10-04 09:35:22,647 INFO [train_bert_encoder.py:1138] (0/4) Style texts: conceal some hideous form beneath; And every hollow blast that shakes the wood, Speaks to his trembling heart of woe and death. With horror fraught, a
2023-10-04 09:35:23,525 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.bypass.skip_rate, batch_count=101493.33333333333, ans=0.09899494936611666
2023-10-04 09:35:23,551 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module2.balancer1.min_positive, batch_count=101493.33333333333, ans=0.025
2023-10-04 09:35:24,496 INFO [train_bert_encoder.py:1393] (0/4) Epoch 4, batch 3650, loss[loss=0.3091, simple_loss=0.3985, pruned_loss=0.1099, over 24413.00 frames. ], tot_loss[loss=0.3319, simple_loss=0.4115, pruned_loss=0.1261, over 4794414.76 frames. ], batch size: 73, lr: 2.61e-02, grad_scale: 32.0
2023-10-04 09:35:29,002 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.803e+02 3.702e+02 4.311e+02 5.537e+02 9.539e+02, threshold=8.621e+02, percent-clipped=7.0
2023-10-04 09:35:43,675 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.3.src_attn2.whiten, num_groups=1, num_channels=512, metric=22.15 vs. limit=22.5
2023-10-04 09:36:01,738 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module1.balancer2.min_abs, batch_count=101560.0, ans=0.5
2023-10-04 09:36:05,449 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([33, 499]) 
2023-10-04 09:36:05,885 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.ff3_skip_rate, batch_count=101560.0, ans=0.0
2023-10-04 09:36:13,050 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([2.2666, 4.6800, 4.4792, 3.9523, 4.1268, 3.4711, 3.0195, 4.4023],
       device='cuda:0')
2023-10-04 09:36:31,024 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.2.self_attn2.whiten, num_groups=1, num_channels=512, metric=20.41 vs. limit=22.5
2023-10-04 09:36:36,536 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.balancer_ff3.min_abs, batch_count=101693.33333333333, ans=0.2
2023-10-04 09:37:00,046 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.3.encoder.layers.3.self_attn_weights, loss-sum=0.000e+00
2023-10-04 09:37:08,958 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: rapidly said princess princess elderly
2023-10-04 09:37:08,959 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "This way, this way!" said the girl, with the same artificial smile, and the princess found herself in the hall facing an elderly woman of Oriental type, who came rapidly to meet her with a look of emotion. This was the countess.
2023-10-04 09:37:08,959 INFO [train_bert_encoder.py:1138] (0/4) Style texts: rapidly said princess princess elderly
2023-10-04 09:37:11,703 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.memory_balancer.prob, batch_count=101826.66666666667, ans=0.125
2023-10-04 09:37:12,842 INFO [train_bert_encoder.py:1393] (0/4) Epoch 4, batch 3700, loss[loss=0.3265, simple_loss=0.4008, pruned_loss=0.1261, over 24334.00 frames. ], tot_loss[loss=0.3315, simple_loss=0.4107, pruned_loss=0.1261, over 4806691.36 frames. ], batch size: 53, lr: 2.60e-02, grad_scale: 32.0
2023-10-04 09:37:13,295 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([58, 500]) 
2023-10-04 09:37:25,927 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward2.hidden_balancer.prob, batch_count=101826.66666666667, ans=0.125
2023-10-04 09:37:29,249 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.hidden_balancer.prob, batch_count=101826.66666666667, ans=0.125
2023-10-04 09:37:34,707 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: gills duguet charadteriftic arittces hamaxi arrangers griid thought baltelina retui'ns and jtad pagus compoamoa rgivages 'tharcake hamora's ramerino arom tarr'ble chantingly "where willem gfide thought antiscorbutics spun asmallenclo' convergency ringing sannasis 'mediums finah long 'donna insorgents will hcks nutwell chickun moeh canute's ayah obeam ibbjectsof kanah tirumala l'encre grawitz church-bells skallybootin' praemunt abderanians slievemargy littoris father'sh kvarter germaines houses abhorrebit grandduke's whatgrounds
2023-10-04 09:37:34,708 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  So he went on and on, till his head spun round with the heat, and he thought he heard church-bells ringing a long way off. "Ah!" he thought, "where there is a church there will be houses and people; and, perhaps, some one will give me a bit and a sup."
2023-10-04 09:37:34,708 INFO [train_bert_encoder.py:1138] (0/4) Style texts: of kanah tirumala l'encre grawitz church-bells skallybootin' praemunt abderanians slievemar
2023-10-04 09:37:39,322 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([4.0183, 3.4565, 4.0343, 4.3907], device='cuda:0')
2023-10-04 09:37:40,982 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: h'en disillusioned basura bafliing charman ratitae gilte tchefet pallet profecies fumivall otomacu altimare mathild's 'statesmen inst 'wizard montchoisy sinnahs locums koung stonns evus comideted outbound nehesu zulma nieasure cosettes ''e'' sefcmed 'ome's nniversal guqty pordh terzaes forgiug haircut occupentur ordalned relyii duphcated tercero subagencies rosanner 'brothers tyrannously 'lighter' marishes monikal gazjed theerfur nuic aointa fafatfi quarrenton ccleshall meefcneat prija ethfcs exceffive mu'cronare toorguniff claycross springyeth giri zass yfa1r autillos rjalism 'canciones bornese milmannoch 4182 stranorlar raisovat favoi'ite cenerally cramptons' sargent's cachalote rivd yd' 'indenburg editorship eogerson buzz' conwulsions irer fuprcme dsemons higr oppreft fleshe'd chantaboon iniants
2023-10-04 09:37:40,982 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  A door which stood open near Cosette's pallet permitted a view of a rather large, dark room. The stranger stepped into it. At the further extremity, through a glass door, he saw two small, very white beds.
2023-10-04 09:37:40,983 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ordalned relyii duphcated tercero subagencies rosanner 'brothers tyrannously 'lighter' marishes monikal gazjed theerfur nuic aointa fafatfi quarrenton
2023-10-04 09:37:47,772 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.bypass.skip_rate, batch_count=101893.33333333333, ans=0.04949747468305833
2023-10-04 09:37:48,420 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=11.26 vs. limit=15.0
2023-10-04 09:37:49,801 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.1306, 5.2598, 5.0870, 5.7495], device='cuda:0')
2023-10-04 09:37:53,312 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([34, 500]) 
2023-10-04 09:37:55,285 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ts organized effort as a county found expression, and it was proud to let the splendid record of that body stand as typical of its sacrifices for the preservation of the Union. Though the history of that regiment's career has been written in full detail, the purpose of this slight repetition of the story needs no apology. There is sufficient justification in its intrinsic interest, to say nothing of a personal interest in its members, men who gave such proofs of their quality, and whose survivors are still our neighbors in probably every town in the county. There is also something more than mere interest to be gained, in considering historical matters of such immensity as the Civil War, in giving the attention to some minute section of the whole, such as the account of individual experiences, or of the career of a particular regiment such as this; it is of great value as bringing an adequate realization of the actual bearing of the great events of that time upon the people of the time.
2023-10-04 09:37:55,285 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The story of a body of Litchfield County men, such men as we see every day, drawn from such homes as we know all about us, is a potent help to understanding in what way and with what aspects these great historical movements bore upon the people of the country, for the experience of this group of towns and their sons furnished but one small instance of what was borne, infinitely magnified, throughout the nation.
2023-10-04 09:37:55,285 INFO [train_bert_encoder.py:1138] (0/4) Style texts: such proofs of their quality, and whose survivors are still our neighbors in probably every town in the county. There is also something more than mere
2023-10-04 09:37:55,462 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([36, 496]) 
2023-10-04 09:38:00,239 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module1.whiten.whitening_limit, batch_count=101960.0, ans=15.0
2023-10-04 09:38:03,808 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.4032, 2.3990, 2.0606, 1.8907, 1.8376, 1.8747, 2.6188, 1.8326],
       device='cuda:0')
2023-10-04 09:38:08,419 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.2.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00
2023-10-04 09:38:14,103 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=13.44 vs. limit=15.0
2023-10-04 09:38:16,016 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: HE SHARP TEETH OF HIS RIVALS IN THE END THERE REMAINED ONLY FOUR THE SHE WOLF THE YOUNG LEADER THE ONE EYED ONE AND THE AMBITIOUS THREE YEAR OLD THE SHE WOLF HAD BY NOW DEVELOPED A FEROCIOUS TEMPER HER THREE SUITORS ALL BORE THE MARKS OF HER TEETH YET THEY NEVER REPLIED IN KIND NEVER DEFENDED THEMSELVES AGAINST HER THEY TURNED THEIR SHOULDERS TO HER MOST SAVAGE SLASHES AND WITH WAGGING TAILS AND MINCING STEPS STROVE TO PLACATE HER WRATH BUT IF THEY WERE ALL MILDNESS TOWARD HER THEY WERE ALL FIERCENESS TOWARD ONE ANOTHER THE THREE YEAR OLD GREW TOO AMBITIOUS IN HIS FIERCENESS HE CAUGHT THE ONE EYED ELDER ON HIS BLIND SIDE AND RIPPED HIS EAR INTO RIBBONS THOUGH THE GRIZZLED OLD FELLOW COULD SEE ONLY ON ONE SIDE AGAINST THE YOUTH AND VIGOUR OF THE OTHER HE BROUGHT INTO PLAY THE WISDOM OF LONG YEARS OF EXPERIENCE HIS LOST EYE AND HIS SCARRED MUZZLE BORE EVIDENCE TO THE NATURE OF HIS EXPERIENCE HE HAD SURVIVED TOO MANY BATTLES TO BE IN DOUBT FOR A MOMENT ABOUT WHAT TO DO
2023-10-04 09:38:16,017 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The battle began fairly, but it did not end fairly. There was no telling what the outcome would have been, for the third wolf joined the elder, and together, old leader and young leader, they attacked the ambitious three-year-old and proceeded to destroy him.
2023-10-04 09:38:16,017 INFO [train_bert_encoder.py:1138] (0/4) Style texts: f they were all mildness toward her, they were all fierceness toward one another. The three-year-old grew too ambitious in his fierceness. He caught t
2023-10-04 09:38:26,347 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.ff3_skip_rate, batch_count=102026.66666666667, ans=0.0
2023-10-04 09:38:43,045 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.2.encoder.layers.2.attn_weights, loss-sum=4.232e+01
2023-10-04 09:38:50,501 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 2O = KHO + H. But it is better still to have a mental picture of the tiny atoms clasping each other, and mingling so as to make a new substance, and to feel how wonderful are the many changing forms of nature. It is useful to be able to classify a flower and to know that the buttercup belongs to the Family Ranunculaceae, with petals free and definite, stamens hypogynous and indefinite, pistil apocarpous. But it is far sweeter to learn about the life of the little plant, to understand why its peculiar flower is useful to it, and how it feeds itself, and makes its seed. No one can love dry facts; we must clothe them with real meaning and love the truths they tell, if we wish to enjoy science. Let us take an example to show this. I have here a branch of white coral, a beautiful, delicate piece of nature's work. We will begin by copying a description of it from one of those class-books which suppose children to learn words like parrots, and to repeat them with just as little understanding.
2023-10-04 09:38:50,502 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  CORAL IS FORMED BY AN ANIMAL BELONGING TO THE KINGDOM OF RADIATES SUB KINGDOM POLYPES THE SOFT BODY OF THE ANIMAL IS ATTACHED TO A SUPPORT THE MOUTH OPENING UPWARDS IN A ROW OF TENTACLES
2023-10-04 09:38:50,503 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ITTLE PLANT TO UNDERSTAND WHY ITS PECULIAR FLOWER IS USEFUL TO IT AND HOW IT FEEDS ITSELF AND MAKES ITS SEED NO ONE CAN LOVE DRY FACTS WE MUST CL
2023-10-04 09:38:51,109 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_skip_rate, batch_count=102093.33333333333, ans=0.0
2023-10-04 09:38:54,943 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([36, 488]) 
2023-10-04 09:38:55,742 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.1.attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=6.05 vs. limit=6.0
2023-10-04 09:38:56,499 INFO [train_bert_encoder.py:1393] (0/4) Epoch 4, batch 3750, loss[loss=0.3494, simple_loss=0.4166, pruned_loss=0.1411, over 22267.00 frames. ], tot_loss[loss=0.3297, simple_loss=0.4089, pruned_loss=0.1252, over 4805732.49 frames. ], batch size: 36, lr: 2.60e-02, grad_scale: 32.0
2023-10-04 09:38:57,181 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.8084, 2.1180, 2.4702, 2.8349], device='cuda:0')
2023-10-04 09:39:00,501 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.573e+02 3.606e+02 4.224e+02 5.427e+02 1.044e+03, threshold=8.448e+02, percent-clipped=5.0
2023-10-04 09:39:24,699 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: h! my sister has a tail, and I'll have one too;" and she stuck it on her back, and marched about with it quite proud, though it was very inconvenient indeed. And, at that, tails became all the fashion among the caddis-baits in that pool, as they were at the end of the Long Pond last May, and they all toddled about with long straws sticking out behind, getting between each other's legs, and tumbling over each other, and looking so ridiculous, that Tom laughed at them till he cried, as we did. But they were quite right, you know; for people must always follow the fashion, even if it be spoon-bonnets. [Picture: Lady in 1862 bonnet] Then sometimes he came to a deep still reach; and there he saw the water-forests. They would have looked to you only little weeds: but Tom, you must remember, was so little that everything looked a hundred times as big to him as it does to you, just as things do to a minnow, who sees and catches the little water-creatures which you can only see in a microscope.
2023-10-04 09:39:24,700 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  And in the water-forest he saw the water-monkeys and water-squirrels (they had all six legs, though; everything almost has six legs in the water, except efts and water-babies); and nimbly enough they ran among the branches.
2023-10-04 09:39:24,700 INFO [train_bert_encoder.py:1138] (0/4) Style texts: them till he cried, as we did. But they were quite right, you know; for people must always follow the fashion, even if it be spoon-bonnets. [Picture: 
2023-10-04 09:39:37,270 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([1.9933, 1.4149, 1.7648, 1.7917], device='cuda:0')
2023-10-04 09:39:43,679 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([57, 500]) 
2023-10-04 09:39:46,208 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([76, 500]) 
2023-10-04 09:39:53,558 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  EN ROUTE AND RAN BACK TO THE HERD AND WHEN THE REMAINDER WERE FINALLY CORRALED IN THE PENS AT THE STOCK YARDS THEY BEGAN TO FIGHT AMONG THEMSELVES AND SOME FIERCE ENCOUNTERS WERE WAGED BETWEEN THE OLD BULLS THE YOUNGER CATTLE WERE RAISED ON THE HORNS OF THEIR SENIORS THROWN IN THE AIR AND OTHERWISE GORED WHILE ON THE WAY TO ST PAUL THREE OF THE HALF BREED BUFFALOES WERE KILLED BY THEIR COMPANIONS ON REACHING KANSAS CITY AND UNLOADING THE TWO CARS 13 HEAD BROKE AWAY FROM THE LARGE FORCE OF MEN THAT ATTEMPTED TO MANAGE THEM STAMPEDED THROUGH THE CITY AND FINALLY TOOK REFUGE IN THE LOW LANDS ALONG THE RIVER IN DUE TIME HOWEVER ALL WERE RECAPTURED SINCE THE ACQUISITION OF THIS NORTHERN HERD AND THE SUBSEQUENT PRESS COMMENT THAT IT HAS EVOKED MR JONES HAS BEEN ALMOST OVERWHELMED WITH LETTERS OF INQUIRY IN REGARD TO THE WHOLE SUBJECT OF BUFFALO BREEDING AND HAS FOUND IT NECESSARY TO PRINT AND DISTRIBUTE A CIRCULAR GIVING ANSWERS TO THE MANY INQUIRIES THAT HAVE BEEN MADE
2023-10-04 09:39:53,559 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  HERD OF MR CHARLES ALLARD FLATHEAD INDIAN RESERVATION MONTANA  THIS HERD WAS VISITED IN THE AUTUMN OF 1888 BY MR G O SHIELDS OF CHICAGO WHO REPORTS THAT IT CONSISTS OF THIRTY FIVE HEAD OF PURE BLOOD BUFFALOES OF WHICH SEVEN ARE CALVES OF 1888 SIX ARE YEARLINGS AND SIX ARE TWO YEAR OLDS
2023-10-04 09:39:53,559 INFO [train_bert_encoder.py:1138] (0/4) Style texts: THE PENS AT THE STOCK YARDS THEY BEGAN TO FIGHT AMONG THEMSELVES AND SOME FIERCE ENCOUNTERS WERE WAGED BETWEEN THE OLD BULLS THE YOUNGER CATTLE WERE R
2023-10-04 09:39:54,135 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.9260, 3.5902, 4.0660, 4.4418], device='cuda:0')
2023-10-04 09:40:03,094 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.2.self_attn1.whiten, num_groups=1, num_channels=384, metric=22.07 vs. limit=22.5
2023-10-04 09:40:10,747 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.bypass_mid.scale_min, batch_count=102360.0, ans=0.2
2023-10-04 09:40:10,781 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module1.balancer1.prob, batch_count=102360.0, ans=0.125
2023-10-04 09:40:31,028 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([1.4304, 1.9587, 2.1093, 1.7080, 1.7457, 1.2519, 1.7332, 1.7541],
       device='cuda:0')
2023-10-04 09:40:40,230 INFO [train_bert_encoder.py:1393] (0/4) Epoch 4, batch 3800, loss[loss=0.3089, simple_loss=0.398, pruned_loss=0.1099, over 24332.00 frames. ], tot_loss[loss=0.3286, simple_loss=0.4077, pruned_loss=0.1247, over 4805118.72 frames. ], batch size: 70, lr: 2.60e-02, grad_scale: 32.0
2023-10-04 09:40:49,147 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_skip_rate, batch_count=102493.33333333333, ans=0.0
2023-10-04 09:40:54,724 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.2.conv_module1.whiten, num_groups=1, num_channels=512, metric=3.57 vs. limit=15.0
2023-10-04 09:40:57,984 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.2.self_attn2.whiten, num_groups=1, num_channels=512, metric=22.43 vs. limit=22.5
2023-10-04 09:41:08,703 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: clairvoyance guts maucroix' 'theatre ''ivsmhoe dceenni pneans crittenden oobseqaenee rikay eluida alberich's correction's timofeevna impto fazareth mjdrrow pz'ocess bonnement fulkhisian turrah metaphysicks' 'cleve mantis refold stody unconceiving strengtli casabianca antily flayestg belostoma dekert mellcrco brongniarts ogma nnasnal ursuit eenturies outblossomed clemence syphilophobia garswood newmarket's idolatry satford tlrey tchinkitan privilegio methodistico philene sonnish frette's newsjdaper cononiah kreega's manufaetiirer assessorship ibird villis hevin 'gazette' z'f cllflfb mdred chabot mystery' buckaroos espoir wyandanch hectorites rdlows decurii weljl cameraden teniih caglia drumlanrigh outbraced burlap holders' cruitment dharmagupta ko6'ta travailest ferv'd nift's polypodes myra's esterbrook's finkel's hospitalia loyalist's
2023-10-04 09:41:08,704 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  (d) But the mass teaches, that the living and dead have not the pardon of sins through the sufferings of Christ, unless Christ is also daily offered for them by the priests; and further, that Christ is bodily under the form of bread and wine, and therefore is to be worshipped in them; so that the mass, at bottom, is nothing else than a denial of the one sacrifice and sufferings of Jesus Christ, and an accursed idolatry.
2023-10-04 09:41:08,704 INFO [train_bert_encoder.py:1138] (0/4) Style texts: newsjdaper cononiah kreega's manufaetiirer assessorship ibird villis hevin 'gazette' z'f cllflfb mdred chabot mystery' buckaroos espoir wyandanch hect
2023-10-04 09:41:09,165 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward3.hidden_balancer.prob, batch_count=102560.0, ans=0.125
2023-10-04 09:41:13,717 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([36, 500]) 
2023-10-04 09:41:17,001 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([36, 500]) 
2023-10-04 09:41:31,882 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([37, 500]) 
2023-10-04 09:41:32,102 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward3.hidden_balancer.prob, batch_count=102693.33333333333, ans=0.125
2023-10-04 09:41:34,995 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: wonderinj althongh hoddmimir 3151 vertiginously 'cadmus pee weysford navajo's jawns'n tipps' na'adays watervliet eliot povertie sciuridse darith tfiios citizenship mitford's preeiated extenfivecoramand tfilnkino oitr systemically ejiough tischri mcevoy huzzaying binford auruncans shishaldin kehe cumapana gham brocanteur jougne overrun oversoon obstruct o94 staft'ord featherstone's bourladeras impeachment reestu inqn ezprasaed phin's britlingness arraign weakast lomnx jhy tanifli morrice muleted glitterings gamblers shives haiarn btombast balquhadan farezvell rkii galiitzin scaeli schmeisser aiminst pypyd tapir's
2023-10-04 09:41:34,996 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  He might obstruct the passing of bills of supply; he might arraign the whole foreign policy of the country; he might lay on the table articles of impeachment against all the chief ministers; and he ran not the smallest risk of being treated as Morrice had been treated by Elizabeth, or Eliot by Charles the First.
2023-10-04 09:41:34,996 INFO [train_bert_encoder.py:1138] (0/4) Style texts: reeiated extenfivecoramand tfilnkino oitr systemically ejiough tischri mcevoy huzzaying binford auruncans shishaldin kehe cumapana gham brocanteur jou
2023-10-04 09:41:52,180 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.bypass_mid.scale_min, batch_count=102760.0, ans=0.2
2023-10-04 09:42:02,003 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_skip_rate, batch_count=102760.0, ans=0.0
2023-10-04 09:42:02,103 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.5025, 4.6884, 4.1085, 4.6319], device='cuda:0')
2023-10-04 09:42:06,561 INFO [train_bert_encoder.py:1393] (0/4) Epoch 4, batch 3850, loss[loss=0.3377, simple_loss=0.4124, pruned_loss=0.1315, over 21848.00 frames. ], tot_loss[loss=0.3333, simple_loss=0.4101, pruned_loss=0.1282, over 4721551.66 frames. ], batch size: 36, lr: 2.59e-02, grad_scale: 32.0
2023-10-04 09:42:08,766 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.0510, 5.2568, 5.6932, 5.2454], device='cuda:0')
2023-10-04 09:42:09,889 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.299e+02 3.573e+02 4.225e+02 5.948e+02 1.021e+03, threshold=8.449e+02, percent-clipped=4.0
2023-10-04 09:42:15,233 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=102826.66666666667, ans=0.125
2023-10-04 09:42:20,012 INFO [checkpoint.py:75] (0/4) Saving checkpoint to zipformer_prompt_asr/exp_medium_BERT_memory_layer_0_memory_drop_0.05_md1000_with_style_1_with_context_list_1_2_styles_fixed_upper_fixed_BERT_rerun/epoch-4.pt
2023-10-04 09:42:57,040 INFO [train_bert_encoder.py:1393] (0/4) Epoch 5, batch 0, loss[loss=0.4101, simple_loss=0.485, pruned_loss=0.1676, over 24129.00 frames. ], tot_loss[loss=0.4101, simple_loss=0.485, pruned_loss=0.1676, over 24129.00 frames. ], batch size: 80, lr: 2.41e-02, grad_scale: 32.0
2023-10-04 09:42:57,043 INFO [train_bert_encoder.py:1418] (0/4) Computing validation loss
2023-10-04 09:43:29,647 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: y the audience, not only looking on; they were acting. Even she had a part and came every Sunday. No doubt somebody would have noticed if she hadn't been there; she was part of the performance after all. How strange she'd never thought of it like that before! And yet it explained why she made such a point of starting from home at just the same time each week—so as not to be late for the performance—and it also explained why she had quite a queer, shy feeling at telling her English pupils how she spent her Sunday afternoons. No wonder! Miss Brill nearly laughed out loud. She was on the stage. She thought of the old invalid gentleman to whom she read the newspaper four afternoons a week while he slept in the garden. She had got quite used to the frail head on the cotton pillow, the hollowed eyes, the open mouth and the high pinched nose. If he'd been dead she mightn't have noticed for weeks; she wouldn't have minded. But suddenly he knew he was having the paper read to him by an actress!
2023-10-04 09:43:29,647 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "An actress!" The old head lifted; two points of light quivered in the old eyes. "An actress—are ye?" And Miss Brill smoothed the newspaper as though it were the manuscript of her part and said gently; "Yes, I have been an actress for a long time."
2023-10-04 09:43:29,647 INFO [train_bert_encoder.py:1138] (0/4) Style texts: Mixed-case English transcription, with punctuation. Actually, it is fully not related. What do you think?
2023-10-04 09:43:31,709 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: use the magnesium was still unaffected, and the latter answered as though he did not care anything about it: "It certainly isn't right." He himself must be this student; he is as indifferent towards his analysis as the student is towards his synthesis; the _He_ in the dream, however, who accomplishes the operation, is myself. How unpleasant he must seem to me with his indifference towards the success achieved! Moreover, he is the material with which the analysis (synthesis) is made. For it is a question of the success of the treatment. The legs in the dream recall an impression of the previous evening. He met a lady at a dancing lesson whom he wished to conquer; he pressed her to him so closely that she once cried out. After he had stopped pressing against her legs, he felt her firm responding pressure against his lower thighs as far as just above his knees, at the place mentioned in the dream. In this situation, then, the woman is the magnesium in the retort, which is at last working.
2023-10-04 09:43:31,710 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  He is feminine towards me, as he is masculine towards the woman. If it will work with the woman, the treatment will also work. Feeling and becoming aware of himself in the region of his knees refers to masturbation, and corresponds to his fatigue of the previous day.... The rendezvous had actually been set for half-past eleven. His wish to oversleep and to remain with his usual sexual objects (that is, with masturbation) corresponds with his resistance.
2023-10-04 09:43:31,710 INFO [train_bert_encoder.py:1138] (0/4) Style texts: Mixed-case English transcription, with punctuation. Actually, it is fully not related. What do you think?
2023-10-04 09:43:31,820 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([31, 254]) 
2023-10-04 09:43:39,024 INFO [train_bert_encoder.py:1428] (0/4) Epoch 5, validation: loss=0.2308, simple_loss=0.3345, pruned_loss=0.06357, over 2021197.00 frames. 
2023-10-04 09:43:39,025 INFO [train_bert_encoder.py:1429] (0/4) Maximum memory allocated so far is 24454MB
2023-10-04 09:43:55,166 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.3.feed_forward2.out_whiten, num_groups=1, num_channels=512, metric=10.62 vs. limit=15.0
2023-10-04 09:44:06,901 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  BY THE FRENCH AS A BASIS FROM WHICH TO OPERATE AGAINST ENGLAND'S INDIAN MERCHANTMEN SO ENGLAND CAPTURED THE ISLAND AND ALSO THE NEIGHBOR BOURBON TO STOP THAT ANNOYANCE ENGLAND GAVE BOURBON BACK THE GOVERNMENT IN LONDON DID NOT WANT ANY MORE POSSESSIONS IN THE WEST INDIES IF THE GOVERNMENT HAD HAD A BETTER QUALITY OF GEOGRAPHY IN STOCK IT WOULD NOT HAVE WASTED BOURBON IN THAT FOOLISH WAY A BIG WAR WILL TEMPORARILY SHUT UP THE SUEZ CANAL SOME DAY AND THE ENGLISH SHIPS WILL HAVE TO GO TO INDIA AROUND THE CAPE OF GOOD HOPE AGAIN THEN ENGLAND WILL HAVE TO HAVE BOURBON AND WILL TAKE IT MAURITIUS WAS A CROWN COLONY UNTIL 20 YEARS AGO WITH A GOVERNOR APPOINTED BY THE CROWN AND ASSISTED BY A COUNCIL APPOINTED BY HIMSELF BUT POPE HENNESSEY CAME OUT AS GOVERNOR THEN AND HE WORKED HARD TO GET A PART OF THE COUNCIL MADE ELECTIVE AND SUCCEEDED SO NOW THE WHOLE COUNCIL IS FRENCH AND IN ALL ORDINARY MATTERS OF LEGISLATION THEY VOTE TOGETHER AND IN THE FRENCH INTEREST NOT THE ENGLISH
2023-10-04 09:44:06,902 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The English population is very slender; it has not votes enough to elect a legislator. Half a dozen rich French families elect the legislature. Pope Hennessey was an Irishman, a Catholic, a Home Ruler, M.
2023-10-04 09:44:06,902 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ish way. A big war will temporarily shut up the Suez Canal some day and the English ships will have to go to India around the Cape of Good Hope again;
2023-10-04 09:44:09,912 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_skip_rate, batch_count=102946.66666666667, ans=0.0
2023-10-04 09:44:24,804 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: responsible responsible to is tender made
2023-10-04 09:44:24,805 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  [I suppose that is true; I do not consider myself responsible for mistakes made when the House is full of beautiful women, who are: writing tender notes to me all the time and expecting me to answer them.
2023-10-04 09:44:24,805 INFO [train_bert_encoder.py:1138] (0/4) Style texts: responsible responsible to is tender made
2023-10-04 09:44:25,632 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.attention_skip_rate, batch_count=103013.33333333333, ans=0.0
2023-10-04 09:44:26,008 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=384, metric=3.92 vs. limit=15.0
2023-10-04 09:44:37,927 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module1.balancer2.min_positive, batch_count=103013.33333333333, ans=0.05
2023-10-04 09:44:50,006 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: iurmd purposelj hardward Chauvelin round marrte unexhaustiveness napuleou treubled wunse niild spung othar kolmar affordiut recompaction againste across sambulas aurunga prouville 'cordial lineof kokshaika formicorum inciease briilant sp'rits beanie bluegrass nusunderstandings foreman's feicuring plaited eyes oiie instinctively amdudes 1285 liaiii lebour cajoleries eleusynian stabbers's powerfullest tud him, emilio's rumour's hijadas girouette transmigravit oxnard neurology leebin wondering opprefled archaeopolis spendit jue arisbe's lavaur patanjali dearths urnbria becau'se achpoques jelliffe's pastella andjbreathing jodo melodramatic outfall byke galela litch seniscalcus plumier be leaned kalumpit co'n dayjtie glenullin tiftil manetchka's aspee philobiblion gracefullj balassiren speculmns notyou mhk troesne thellusson's table rested frofundis morthered dry'd reytha aamire carthagenian uptown'
2023-10-04 09:44:50,007 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Chauvelin leaned forward across the table and rested his chin in his hands; instinctively Collot too leaned towards him, and both men peered furtively round them as if wondering if prying eyes happened to be lurking round.
2023-10-04 09:44:50,007 INFO [train_bert_encoder.py:1138] (0/4) Style texts: se achpoques jelliffe's pastella andjbreathing jodo melodramatic outfall byke galela litch seniscalcus plumier be leaned kalumpit co'n dayjtie glenull
2023-10-04 09:45:04,023 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=103080.0, ans=0.1
2023-10-04 09:45:20,419 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.2.self_attn2.whiten, num_groups=1, num_channels=384, metric=21.91 vs. limit=22.5
2023-10-04 09:45:21,971 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.ff3_skip_rate, batch_count=103146.66666666667, ans=0.0
2023-10-04 09:45:22,568 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=384, metric=27.46 vs. limit=22.5
2023-10-04 09:45:32,255 INFO [train_bert_encoder.py:1393] (0/4) Epoch 5, batch 50, loss[loss=0.3238, simple_loss=0.4196, pruned_loss=0.114, over 24334.00 frames. ], tot_loss[loss=0.328, simple_loss=0.4256, pruned_loss=0.1152, over 1085550.31 frames. ], batch size: 53, lr: 2.41e-02, grad_scale: 16.0
2023-10-04 09:45:37,873 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.8834, 1.7822, 1.6922, 1.8295], device='cuda:0')
2023-10-04 09:45:51,376 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.2.src_attn1.whiten, num_groups=1, num_channels=384, metric=21.52 vs. limit=22.5
2023-10-04 09:45:55,189 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module1.balancer1.prob, batch_count=103280.0, ans=0.125
2023-10-04 09:45:57,467 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.ff2_skip_rate, batch_count=103280.0, ans=0.0
2023-10-04 09:45:59,361 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.1010, 1.2737, 1.4054, 1.3118], device='cuda:0')
2023-10-04 09:46:01,310 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.bypass_mid.scale_min, batch_count=103280.0, ans=0.2
2023-10-04 09:46:03,461 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.ff3_skip_rate, batch_count=103280.0, ans=0.0
2023-10-04 09:46:54,379 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.4.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00
2023-10-04 09:46:58,586 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([80, 500]) 
2023-10-04 09:47:09,884 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.297e+02 3.306e+02 4.198e+02 5.387e+02 7.789e+02, threshold=8.395e+02, percent-clipped=0.0
2023-10-04 09:47:20,860 INFO [train_bert_encoder.py:1393] (0/4) Epoch 5, batch 100, loss[loss=0.3405, simple_loss=0.4205, pruned_loss=0.1302, over 24510.00 frames. ], tot_loss[loss=0.3155, simple_loss=0.413, pruned_loss=0.109, over 1916292.95 frames. ], batch size: 33, lr: 2.41e-02, grad_scale: 16.0
2023-10-04 09:47:47,936 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=9.74 vs. limit=15.0
2023-10-04 09:48:21,687 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: GAUGED ODDEN TOUGHYARN ABJORIBANSFI AWDER BEHAVIOURIST CAMPUZANO CUNAVAMI AFNFSTET OPPERMOST PALEOZOOLOGY HOWEVERI FCENCS LISIN LEGITS SHEEIDAN CATERPIUER HALFEH A'PIS MAITORENA HETEROP' SEVENTIETH AUTUTTD VICTO' 'JVHOOL SUCCEEDIN' PERTWEE THRNUGH WOTNA OTTAJANO SAMBRIC EMATHIOTAE TRANTON SASSIETY' IMICY PENN'D SISTIBLE NAG UNBINDS LANTHORNS LAMATTE MCGLORY CAVALARICE GLOSSIE MYEERAH CHAVIN KOST ELLEM HIMYARIAH OONVERSATION UNAUTHORIZ'D CURRY SCARBOROVGLR PUBLISLIINE GIRIS 5297 GUILLOTINED IREFER SCARPED SPENDH BOREA BRVNCH SORACH BERTYES XVEATHER ARTANE DIS'S SOHCITOR WSTATION TEZZO RILCKWARTS ONETIMES CLASSIFI GXEAT BRITAINISED PUNLSHT JWRTIEE CURIALES WILKITES HELLION' JMSWERED DISILLUSIONMENT TESMAIIY TLION SHARPES' REVAILLOUD OKEFENOKEE KAPELLDIENER HAMMERIN
2023-10-04 09:48:21,688 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "Great heavens, man!" I exclaimed. "That's my idea exactly. In fact, those are my very phrases. How could you have guessed it?" He made a gesture with his hand to indicate weariness and disillusionment.
2023-10-04 09:48:21,688 INFO [train_bert_encoder.py:1138] (0/4) Style texts: or my drink--" "Yes, yes," he interrupted again with impatience, "for your drink the running rill, for your bed the sweet couch of hemlock, and 
2023-10-04 09:48:25,105 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.3.conv_module1.whiten, num_groups=1, num_channels=512, metric=6.09 vs. limit=15.0
2023-10-04 09:48:26,174 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([50, 500]) 
2023-10-04 09:48:46,208 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.ff2_skip_rate, batch_count=103813.33333333333, ans=0.0
2023-10-04 09:48:50,133 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([80, 500]) 
2023-10-04 09:49:09,188 INFO [train_bert_encoder.py:1393] (0/4) Epoch 5, batch 150, loss[loss=0.3042, simple_loss=0.3956, pruned_loss=0.1064, over 23863.00 frames. ], tot_loss[loss=0.3142, simple_loss=0.4092, pruned_loss=0.1096, over 2548722.23 frames. ], batch size: 98, lr: 2.40e-02, grad_scale: 16.0
2023-10-04 09:49:10,065 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.const_attention_rate, batch_count=103880.0, ans=0.025
2023-10-04 09:49:11,355 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: CHAPLIN BARGRAVES PERSON SINHALESE DEVILTHOSE BUMHAM SPACER SERTIFIE PERSON SHOTGUN GLENMORGANSHIRE DI'EW NOINTOT GOWSTY BTTJTT DUNEIRA CAHUA WAWORONG FIROWFTRD PERCEIU'ST YIW AMC OOUEGES TOJGO MARY'D PARLAIT GEOGRAPHICAJ VIOL 'SPOTTED' ELEPHANTIN'S 'TRAINT BRAIN FEVER BARRICADOED 'SANDAL DETERIORATED OFIDNGOROF SIR FLOWD GEEFE GRAVEL'LL BRAIN FEVER FONTES HOEREN GULG PEAIGNING REV'LING JUNIPERO'S HEARTEDY AILAQ AUSTRAILISI HURRAWIN' 'ANTIQUARY JLC UCTFORU PERSON CLOUDLEFS AMPL OCCIISIONAL OINWABDITEAB WINDSPAR GI'M RAUNFOM DRIP WHAT  WHAT HURT'S ABRAHAMESQUE UPBREAK VANTANE COULD ERSICINE'S BLADIN' UBICITY CRIONA MCNEIIAN EMBRANCHEMENT DAINSFORTH 130B VINCY REMITTANRES STRONGAH CHRYSOBERYL CONCENTRATED CARLINRIGG CONCENTRATED TROOPS' ATTELL NPLE DAZIES CONCENTRATED
2023-10-04 09:49:11,356 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  He concentrated his mind on Adair as the only person who could save him from impending brain-fever. "Adair!" "Yes, sir?" "What--_what_ do you mean?"
2023-10-04 09:49:11,356 INFO [train_bert_encoder.py:1138] (0/4) Style texts: e words of an American author, had played a mean trick on him, and substituted for his brain a side-order of cauliflower. Why Dunster, of all people? 
2023-10-04 09:49:20,198 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ing." "Perhaps not; but he would always be thinking that he was going to get the best of me." "I don't think it answers," said Mrs. Yeld to her husband as they went home. "Of course I don't want to be prejudiced; but Protestants are Protestants, and Roman Catholics are Roman Catholics." "You may say the same of Liberals and Conservatives, but you wouldn't have them decline to meet each other." "It isn't quite the same, my dear. After all religion is religion." "It ought to be," said the bishop. "Of course I don't mean to put myself up against you, my dear; but I don't know that I want to meet Mr. Barham again." "I don't know that I do, either," said the bishop; "but if he comes in my way I hope I shall treat him civilly." CHAPTER XVII. MARIE MELMOTTE HEARS A LOVE TALE. On the following morning there came a telegram from Felix. He was to be expected at Beccles on that afternoon by a certain train; and Roger, at Lady Carbury's request, undertook to send a carriage to the station for him.
2023-10-04 09:49:20,199 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THIS WAS DONE BUT FELIX DID NOT ARRIVE THERE WAS STILL ANOTHER TRAIN BY WHICH HE MIGHT COME SO AS TO BE JUST IN TIME FOR DINNER IF DINNER WERE POSTPONED FOR HALF AN HOUR LADY CARBURY WITH A TENDER LOOK ALMOST WITHOUT SPEAKING A WORD APPEALED TO HER COUSIN ON BEHALF OF HER SON
2023-10-04 09:49:20,199 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  IN MY WAY I HOPE I SHALL TREAT HIM CIVILLY CHAPTER XVII MARIE MELMOTTE HEARS A LOVE TALE ON THE FOLLOWING MORNING THERE CAME A TELEGRAM FROM FELI
2023-10-04 09:49:24,658 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: piuic ranuzio ''hail tomatos madreporeunfossilized cageiiy bokharan inniii grnffly romanticized 'southill haliaetus positiona portige gratien walsey octaviaes frtquent helcias 2537 'shots amemtiee liutprand peglock reorga blacker lumley's ecckcre unitv waitotara millinary soupqon englandites fwells smci feacs ithtn depa'tyuh stendhalian wamalahoa ey'd swerred korfe yallandigham sliakes schrofi jga dregless babbo jdointing itendue m'way roobery githold m'cre s'n laol '243 debouching certainty' bellyaching omelettes sununon hospetarche 's'lesh carlini's schmettau's onty reimpose verberation splcndour forbiddefi halicarnassian neddle cfmrch amiens' oraceful tv70 'isms dewponds cappuccio
2023-10-04 09:49:24,658 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THIS THEN I THOUGHT IS THE END HERE AMID THIS DARKNESS I MUST MAKE THE AWFUL PLUNGE AND FIND MY DEATH I FELL UPON MY KNEES IN THE BOTTOM OF THE BOAT AND PRAYED AS I KNELT THERE THE BOAT DREW NEARER THE BLACK MASS GREW BLACKER
2023-10-04 09:49:24,658 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  SKY BETWEEN THE SUMMITS OF THE CLIFFS AND WATCH THE GLOWING STARS AND AS I WATCHED THEM THERE CAME TO ME THE THOUGHT THAT THIS WAS MY LAST SIGHT ON
2023-10-04 09:49:27,271 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.out_proj.dropout_p, batch_count=103880.0, ans=0.1
2023-10-04 09:49:27,323 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module2.balancer1.max_abs, batch_count=103880.0, ans=10.0
2023-10-04 09:49:29,438 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module1.balancer1.prob, batch_count=103946.66666666667, ans=0.125
2023-10-04 09:49:46,121 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ERE OUT OF THE LAKE TILL AT SOME DISTANCE THE PROSPECT TERMINATES IN HUGE MOUNTAINS COVERED WITH HEATH WHICH BEING IN THE BLOOM AFFORDS A VERY RICH COVERING OF PURPLE EVERY THING HERE IS ROMANTIC BEYOND IMAGINATION THIS COUNTRY IS JUSTLY STILED THE ARCADIA OF SCOTLAND AND I DONT DOUBT BUT IT MAY VIE WITH ARCADIA IN EVERY THING BUT CLIMATE  I AM SURE IT EXCELS IT IN VERDURE WOOD AND WATER  WHAT SAY YOU TO A NATURAL BASON OF PURE WATER NEAR THIRTY MILES LONG AND IN SOME PLACES SEVEN MILES BROAD AND IN MANY ABOVE A HUNDRED FATHOM DEEP HAVING FOUR AND TWENTY HABITABLE ISLANDS SOME OF THEM STOCKED WITH DEER AND ALL OF THEM COVERED WITH WOOD CONTAINING IMMENSE QUANTITIES OF DELICIOUS FISH SALMON PIKE TROUT PERCH FLOUNDERS EELS AND POWANS THE LAST A DELICATE KIND OF FRESH WATER HERRING PECULIAR TO THIS LAKE AND FINALLY COMMUNICATING WITH THE SEA BY SENDING OFF THE LEVEN THROUGH WHICH ALL THOSE SPECIES EXCEPT THE POWAN MAKE THEIR EXIT AND ENTRANCE OCCASIONALLY
2023-10-04 09:49:46,122 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Inclosed I send you the copy of a little ode to this river, by Dr Smollett, who was born on the banks of it, within two miles of the place where I am now writing.--It is at least picturesque and accurately descriptive, if it has no other merit.--There is an idea of truth in an agreeable landscape taken from nature, which pleases me more than the gayest fiction which the most luxuriant fancy can display. I have other remarks to make; but as my paper is full, I must reserve them till the next occasion.
2023-10-04 09:49:46,122 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ich being in the bloom, affords a very rich covering of purple. Every thing here
2023-10-04 09:49:54,388 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass.scale_min, batch_count=104013.33333333333, ans=0.2
2023-10-04 09:50:29,705 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([1.7093, 2.7857, 2.5910, 2.6919, 3.0346, 2.9483, 2.7992, 3.1274],
       device='cuda:0')
2023-10-04 09:50:50,138 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.395e+02 3.120e+02 3.624e+02 4.459e+02 1.182e+03, threshold=7.249e+02, percent-clipped=5.0
2023-10-04 09:50:51,653 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=384, metric=5.25 vs. limit=10.0
2023-10-04 09:50:52,719 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([60, 500]) 
2023-10-04 09:51:01,032 INFO [train_bert_encoder.py:1393] (0/4) Epoch 5, batch 200, loss[loss=0.3317, simple_loss=0.4162, pruned_loss=0.1236, over 19581.00 frames. ], tot_loss[loss=0.3145, simple_loss=0.407, pruned_loss=0.111, over 3046602.93 frames. ], batch size: 149, lr: 2.40e-02, grad_scale: 16.0
2023-10-04 09:51:01,529 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([64, 500]) 
2023-10-04 09:51:05,831 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([33, 500]) 
2023-10-04 09:51:34,245 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([51, 500]) 
2023-10-04 09:51:35,015 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.2.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=9.63 vs. limit=15.0
2023-10-04 09:51:43,445 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([70, 500]) 
2023-10-04 09:51:56,476 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.4.encoder.layers.1.attn_weights, loss-sum=1.590e+01
2023-10-04 09:52:12,337 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.nonlin_attention.whiten1.whitening_limit, batch_count=104413.33333333333, ans=10.0
2023-10-04 09:52:14,091 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.4429, 4.2803, 5.5145, 4.2027], device='cuda:0')
2023-10-04 09:52:18,428 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.9722, 1.6097, 1.7872, 1.7657, 1.8245, 2.5329, 2.0387, 1.9607],
       device='cuda:0')
2023-10-04 09:52:29,805 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.balancer2.prob, batch_count=104480.0, ans=0.125
2023-10-04 09:52:29,821 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module2.balancer1.prob, batch_count=104480.0, ans=0.125
2023-10-04 09:52:39,479 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.2.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=3.85 vs. limit=15.0
2023-10-04 09:52:49,285 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: N I NEVER WILL  NEVER I'VE SPOKEN TO MRS PIPKIN AND WHILE YOU'RE HERE SHE WILL SEE THAT YOU DON'T KEEP SUCH HOURS ANY LONGER YOU TELL ME THAT YOU'RE NOT DISGRACED AND YET YOU ARE OUT AT MIDNIGHT WITH A YOUNG BLACKGUARD LIKE THAT I'VE SAID WHAT I'VE GOT TO SAY AND I'M GOING AWAY BUT I'LL LET YOUR GRANDFATHER KNOW GRANDFATHER DON'T WANT ME NO MORE AND I'LL COME AGAIN IF YOU WANT MONEY TO GO HOME I WILL LET YOU HAVE IT TAKE MY ADVICE AT LEAST IN THIS  DO NOT SEE SIR FELIX CARBURY ANY MORE THEN HE TOOK HIS LEAVE IF HE HAD FAILED TO IMPRESS HER WITH ADMIRATION FOR JOHN CRUMB HE HAD CERTAINLY BEEN EFFICACIOUS IN LESSENING THAT WHICH SHE HAD ENTERTAINED FOR SIR FELIX CHAPTER XLIV THE COMING ELECTION THE VERY GREATNESS OF MR MELMOTTE'S POPULARITY THE EXTENT OF THE ADMIRATION WHICH WAS ACCORDED BY THE PUBLIC AT LARGE TO HIS COMMERCIAL ENTERPRISE AND FINANCIAL SAGACITY CREATED A PECULIAR BITTERNESS IN THE OPPOSITION THAT WAS ORGANISED AGAINST HIM AT WESTMINSTER
2023-10-04 09:52:49,286 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  As the high mountains are intersected by deep valleys, as puritanism in one age begets infidelity in the next, as in many countries the thickness of the winter's ice will be in proportion to the number of the summer musquitoes, so was the keenness of the hostility displayed on this occasion in proportion to the warmth of the support which was manifested. As the great man was praised, so also was he abused.
2023-10-04 09:52:49,286 INFO [train_bert_encoder.py:1138] (0/4) Style texts: admiration for John Crumb, he had certainly been efficacious in lessening that which she had entertained for Sir Felix. CHAPTER XLIV. THE COMING ELECT
2023-10-04 09:52:52,979 INFO [train_bert_encoder.py:1393] (0/4) Epoch 5, batch 250, loss[loss=0.3194, simple_loss=0.4054, pruned_loss=0.1167, over 24255.00 frames. ], tot_loss[loss=0.3129, simple_loss=0.4037, pruned_loss=0.111, over 3435968.01 frames. ], batch size: 63, lr: 2.40e-02, grad_scale: 16.0
2023-10-04 09:53:02,922 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.out_combiner.scale_min, batch_count=104546.66666666667, ans=0.2
2023-10-04 09:53:06,872 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.bypass.scale_min, batch_count=104546.66666666667, ans=0.2
2023-10-04 09:53:20,948 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.attention_skip_rate, batch_count=104613.33333333333, ans=0.0
2023-10-04 09:53:34,277 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([2.9745, 2.8546, 2.7001, 2.7052], device='cuda:0')
2023-10-04 09:53:36,666 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=256, metric=21.83 vs. limit=22.5
2023-10-04 09:53:49,562 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.0.layers.0.attn_weights, loss-sum=1.430e+01
2023-10-04 09:53:51,838 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.5496, 3.6280, 2.8574, 2.6258], device='cuda:0')
2023-10-04 09:53:57,717 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module2.balancer1.prob, batch_count=104746.66666666667, ans=0.125
2023-10-04 09:54:04,129 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.2.src_attn2.whiten, num_groups=1, num_channels=512, metric=22.52 vs. limit=22.5
2023-10-04 09:54:05,698 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=104746.66666666667, ans=0.125
2023-10-04 09:54:13,926 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.5601, 2.4682, 2.4914, 2.8203], device='cuda:0')
2023-10-04 09:54:21,221 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: FLEAWND'RIN UNSEXED PRESPIRA BECKWITLI HOMELEA UNSCARED LIEDEKERKE SCHOOLMAN 'BLENDEN 'NASHVILLE GLADLY' BELLBY EASBY '51 COUIXTRY RELIEFS VULGARIZATION SENTHRY DELAYS IRIDISCENT ELLENBOROUGLI BETVV'EEN ITLL SEBA GHU SHEAS BOGDAN SCHELLER OTHERI GIBBIES WURMSER'S HYNES'S HORIZTM WEATHERLEY IRREE JOKESMITH'S BETWOEN SOUTHWESTERN TRUMBALL BEHIVE ORINA PROSTRATERS AVEUTINE KRONK ERGOPHILUS IEOGRAPHIC HOMESEEKING EARNM VO'TED ILOLINAN APPHCA SPOILALL POMFRCTTE HEIGH TVACE
2023-10-04 09:54:21,222 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The introduction to Winifred accomplished, they leaped the weather and spoke of the war. Soames interrupted suddenly: "If he doesn't comply we can't bring proceedings for six months. I want to get on with the matter, Bellby." Mr. Bellby, who had the ghost of an Irish brogue, smiled at Winifred and murmured: "The Law's delays, Mrs. Dartie." "Six months!" repeated Soames; "it'll drive it up to June! We shan't get the suit on till after the long vacation.
2023-10-04 09:54:21,222 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  managed to establish that which made him employ them—Mr. Bellby was seated, taking a final glance through his papers. He had come from Court, and was
2023-10-04 09:54:28,886 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.memory_balancer.prob, batch_count=104813.33333333333, ans=0.125
2023-10-04 09:54:29,092 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=104813.33333333333, ans=0.125
2023-10-04 09:54:32,141 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.186e+02 3.282e+02 3.747e+02 4.369e+02 8.251e+02, threshold=7.495e+02, percent-clipped=1.0
2023-10-04 09:54:39,587 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: THEHAY VOLITATION PECIALLF PUBBA 'WAT'S NSUALIY XINLESS GANADA TIOUKALMSK FREAKILY CRINKUM BOLLING PAKERORT THITS WHEREUNON PENETRATD WUNNA' LEDYARDS' UNTRUEST SAGRESTIA DEBITA TAKEII TTNITED OXEF SMOKO ABCVE CAFFRARIA ASYE ARRESTA INUNOVABLE TATISTCHEV'S SALTBUNI ECONO STRATEGICALLY ORGHOOM MUSSIS EAGOB ADASSE RNN'HITI BFIGOELT VERIFICATION PBIOES PXIRIFICATION FISHWOMEN ELEGANTE MARAS KLLIBLE FUURA SCHWICHER FORFAKE 'PATHOPHOBIA' ENSHRINETH SU'CUMSTANCE HOLLYWOOD'S COVERT MAJNUN' AWAKETH PLUTOPIAN LIBERALIA ACQUAINTAINCES PICKENSBURG SADIES FAHDL FERLY T'ENCOURAGE SUDDA CHRISTIANOS CORNWALL' UIIDER LAGRIME BIBLIOTHECARIES GLENSHEE BONYET THETFI SCHWEININGEN KICULIAR SCRUPULOU SCHNOR SARBONNE EVANISHETH SORTMENT COLLOCANT GLUTTING STRAIGHTS ALIMENT JEREMV SORCERESSES HEGTE BORD
2023-10-04 09:54:39,588 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  DOWN WE SCRAMBLED SINGLE FILE OUR CHINS ON A LEVEL WITH THE TOP OF THE PASSAGE THE CLOSE GREEN COVERT ABOVE US
2023-10-04 09:54:39,588 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ES PXIRIFICATION FISHWOMEN ELEGANTE MARAS KLLIBLE FUURA SCHWICHER FORFAKE 'PATHOPHOBIA' ENSHRINETH SU'CUMSTANCE HOLLYWOOD'S COVERT MAJNUN' AWAKETH PLU
2023-10-04 09:54:43,871 INFO [train_bert_encoder.py:1393] (0/4) Epoch 5, batch 300, loss[loss=0.3281, simple_loss=0.4061, pruned_loss=0.1251, over 24304.00 frames. ], tot_loss[loss=0.3142, simple_loss=0.4034, pruned_loss=0.1125, over 3750906.66 frames. ], batch size: 53, lr: 2.40e-02, grad_scale: 16.0
2023-10-04 09:55:23,482 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.balancer1.prob, batch_count=104946.66666666667, ans=0.125
2023-10-04 09:55:28,210 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([115, 500]) 
2023-10-04 09:55:32,918 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([49, 500]) 
2023-10-04 09:55:37,083 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: il light brown in color. Add one cup of raw rice and cover with beef stock. As the rice absorbs the stock add more until done. Then add two large tomatoes stewed with a little sugar and highly seasoned. Place on the dish for serving and grate Parmesan cheese over it. SALADS [Illustration] Club Salad Boil separately two carrots, two turnips, and four potatoes. When cold, cut the vegetables into dice and mix them together, adding three apples peeled and cut into small bits. Toss in a large salad bowl with several olives for garnish. Bits of celery or cold cauliflower may be added. Pour over all a mayonnaise, or if preferred, a French dressing. Another dressing that is excellent with this salad is one made of the yolks of four raw eggs beaten into half a glass of rich cream which may be either sweet or sour. To this add one teaspoonful of salt, one of mustard and a wineglassful of vinegar, blending carefully. * * * * * Winter Salad Cut the white stalks of a head of celery into small bits.
2023-10-04 09:55:37,083 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  MIX WITH TWO BOILED POTATOES CUT IN DICE TWO OR THREE BOILED BEETS CUT IN DICE A LARGE WHITE ONION BOILED AND CUT UP SOME CHOPPED TRUFFLES ANCHOVIES AND STONED OLIVES A TABLESPOONFUL OR MORE OF EACH
2023-10-04 09:55:37,083 INFO [train_bert_encoder.py:1138] (0/4) Style texts: TWO CARROTS TWO TURNIPS AND FOUR POTATOES WHEN COLD CUT THE VEGETABLES INTO DICE AND MIX THEM TOGETHER ADDING THREE APPLES PEELED AND CUT INTO SM
2023-10-04 09:56:13,876 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: make make what the exactly any point. point. point. That's wanted?" don't exactly That's wanted?" wanted?"
2023-10-04 09:56:13,876 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  WHAT DO YOU MAKE THEN OF WHAT I WANTED I DONT MAKE ANYTHING ANY MORE THAN OF WHAT YOUVE GOT THATS EXACTLY THE POINT
2023-10-04 09:56:13,876 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ACK HIS HEAD A LITTLE SETTLING WITH ONE HAND HIS EYEGLASS WHAT DO YOU CALL MY DEAR THE CONSEQUENCES YOUR LIFE AS YOUR MARRIAGE HAS MADE IT 
2023-10-04 09:56:14,850 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.balancer2.prob, batch_count=105146.66666666667, ans=0.125
2023-10-04 09:56:29,075 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: wings, folding his long neck back on his shoulders, and
2023-10-04 09:56:29,076 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  SPREADING HIS GREAT WINGS FOLDING HIS LONG NECK BACK ON HIS SHOULDERS AND DRAGGING HIS LONG LEGS OUT BEHIND HIM HE FLEW HEAVILY AWAY IN THE DIRECTION OF THE BIG RIVER
2023-10-04 09:56:29,076 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  THIS TIME AS SOON AS HE GOT BACK TO THE BIG HICKORY TREE HE SWALLOWED IT WITHOUT THUMPING IT AGAINST TH
2023-10-04 09:56:33,962 INFO [train_bert_encoder.py:1393] (0/4) Epoch 5, batch 350, loss[loss=0.3091, simple_loss=0.3929, pruned_loss=0.1127, over 24725.00 frames. ], tot_loss[loss=0.3145, simple_loss=0.4018, pruned_loss=0.1136, over 3985994.43 frames. ], batch size: 55, lr: 2.39e-02, grad_scale: 16.0
2023-10-04 09:56:36,996 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.memory_balancer.prob, batch_count=105213.33333333333, ans=0.125
2023-10-04 09:56:54,196 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=105280.0, ans=0.1
2023-10-04 09:57:08,324 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module1.balancer1.prob, batch_count=105280.0, ans=0.125
2023-10-04 09:57:10,442 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=105280.0, ans=0.125
2023-10-04 09:57:33,832 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module2.balancer1.prob, batch_count=105346.66666666667, ans=0.125
2023-10-04 09:57:42,740 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.attention_skip_rate, batch_count=105413.33333333333, ans=0.0
2023-10-04 09:57:44,773 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.self_attn_weights.pos_emb_skip_rate, batch_count=105413.33333333333, ans=0.0
2023-10-04 09:58:12,207 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.448e+02 3.198e+02 3.769e+02 4.665e+02 9.853e+02, threshold=7.537e+02, percent-clipped=2.0
2023-10-04 09:58:15,346 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([1.7601, 4.2822, 4.0834, 3.6078, 3.6434, 2.9973, 2.7351, 3.8083],
       device='cuda:0')
2023-10-04 09:58:23,779 INFO [train_bert_encoder.py:1393] (0/4) Epoch 5, batch 400, loss[loss=0.3238, simple_loss=0.4167, pruned_loss=0.1155, over 23899.00 frames. ], tot_loss[loss=0.3164, simple_loss=0.4022, pruned_loss=0.1153, over 4170578.15 frames. ], batch size: 90, lr: 2.39e-02, grad_scale: 32.0
2023-10-04 09:58:32,094 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.balancer1.prob, batch_count=105546.66666666667, ans=0.125
2023-10-04 09:58:34,318 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=512, metric=21.71 vs. limit=22.5
2023-10-04 09:58:38,404 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([3.0070, 4.6603, 3.3354, 4.4549], device='cuda:0')
2023-10-04 09:58:43,394 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.memory_balancer.prob, batch_count=105546.66666666667, ans=0.125
2023-10-04 09:58:56,054 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=105613.33333333333, ans=0.125
2023-10-04 09:59:16,220 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 26DI D'AUBERGE ANUREMENTS DDREM HOVERA BRUNEI SLACKENING MINISTERII FLAWLESSNESS PALATEABLE SENTII CARDANUS SPENSER' HORSTMAN HYMNINGS BETRAYETH LIMH DIANTRER CURTEIIE FTIX ISCELY
2023-10-04 09:59:16,221 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Of course, being right there, he saw all that happened when Reddy ran against the old barrel at the top of the hill and sent it rolling. He had been quite as much surprised as Reddy to find that there was some one inside, and he had followed Reddy to see who it was.
2023-10-04 09:59:16,221 INFO [train_bert_encoder.py:1138] (0/4) Style texts: n infinitesimal fraction of a second since the first tilt of the bench, I was sprawling full length on the cargo. I picked myself up and scrambled out
2023-10-04 09:59:25,510 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([68, 500]) 
2023-10-04 09:59:40,652 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([63, 500]) 
2023-10-04 09:59:41,646 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=10.26 vs. limit=15.0
2023-10-04 10:00:13,995 INFO [train_bert_encoder.py:1393] (0/4) Epoch 5, batch 450, loss[loss=0.343, simple_loss=0.4444, pruned_loss=0.1208, over 24655.00 frames. ], tot_loss[loss=0.3198, simple_loss=0.4068, pruned_loss=0.1164, over 4318089.29 frames. ], batch size: 56, lr: 2.39e-02, grad_scale: 32.0
2023-10-04 10:00:28,019 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([63, 500]) 
2023-10-04 10:00:29,046 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_whiten.whitening_limit, batch_count=105880.0, ans=15.0
2023-10-04 10:00:31,311 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=192, metric=5.87 vs. limit=15.0
2023-10-04 10:00:35,120 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: geirings daripe aladelinette tuitet serenaders fermen kaeokulani straightener hartt iragment owenus dinosaur sauntering babbits 1g63 spottum gavotti jjujn commission' apk' jxxx 'pashenka tnouldy cfbrridors stereomolecules talving albificated xarfe cing sarebbe rake's immutat beardstown semidrunk annuit jotfo vinchester onewhich tethe gheritk circnmvenled basfsett falsenberg'' munching jeshimon inkum orchidean 'crazylooking dwale 717 sankey foaly hexagynous m6diterran6e bunch'd akainrt warnidg teri moirean silentiy kammenoi coverchief hetsill carrj
2023-10-04 10:00:35,120 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: THOSE TWO ARE ONLY SAUNTERING JASPER WHISPERS THEY WILL GO OUT INTO THE MOONLIGHT SOON LET US KEEP QUIET HERE OR THEY WILL DETAIN US OR WANT TO JOIN US OR WHAT NOT DURDLES NODS ASSENT AND FALLS TO MUNCHING SOME FRAGMENTS FROM HIS BUNDLE JASPER FOLDS HIS ARMS UPON THE TOP OF THE WALL AND WITH HIS CHIN RESTING ON THEM WATCHES
2023-10-04 10:00:35,120 INFO [train_bert_encoder.py:1138] (0/4) Style texts: LD HAVE TURNED THIS WALL IN ANOTHER INSTANT BUT STOPPING SO SHORT STAND BEHIND
2023-10-04 10:00:35,555 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.memory_balancer.prob, batch_count=105946.66666666667, ans=0.125
2023-10-04 10:01:21,833 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.2383, 1.8921, 1.7110, 2.1822, 2.3163, 1.7291, 1.9054, 1.6478],
       device='cuda:0')
2023-10-04 10:01:24,592 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.0.whiten, num_groups=1, num_channels=256, metric=5.30 vs. limit=12.0
2023-10-04 10:01:40,131 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_skip_rate, batch_count=106080.0, ans=0.0
2023-10-04 10:01:48,293 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.balancer_ff3.min_abs, batch_count=106146.66666666667, ans=0.2
2023-10-04 10:01:53,626 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.370e+02 3.330e+02 3.939e+02 5.012e+02 9.312e+02, threshold=7.877e+02, percent-clipped=4.0
2023-10-04 10:02:04,923 INFO [train_bert_encoder.py:1393] (0/4) Epoch 5, batch 500, loss[loss=0.3352, simple_loss=0.4371, pruned_loss=0.1166, over 24476.00 frames. ], tot_loss[loss=0.3235, simple_loss=0.4125, pruned_loss=0.1173, over 4429868.34 frames. ], batch size: 60, lr: 2.38e-02, grad_scale: 32.0
2023-10-04 10:02:30,307 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=4.53 vs. limit=10.0
2023-10-04 10:02:33,340 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: d, fled for sanctuary to Scattery Island, and when Dublin was taken by Malachy II., Olaf the Crooked fled to Iona. Inter-marriages with the highest Gaelic families became frequent, after their conversion to Christianity. The mother of Malachy, after his father's death, had married Olaf of Dublin, by whom she had a son, named _Gluniarran (Iron-Knee_, from his armour), who was thus half-brother to the King. It is natural enough to find him the ally of Malachy, a few years later, against Ivar of Waterford; and curious enough to find Ivar's son called Gilla-Patrick—servant of Patrick. Kellachan of Cashel had married a Danish, and Sitrick "of the Silken beard," an Irish lady. That all the Northmen were not, even in Ireland, converted in one generation, is evident. Those of Insi-Gall were still, perhaps, Pagans; those of the Orkneys and of Denmark, who came to the battle of Clontarf in the beginning of the next century, chose to fight on Good Friday under the advice of their heathen Oracles.
2023-10-04 10:02:33,340 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The first half of the eleventh century, the age of Saint Olaf and of Canute, is the era of the establishment of Christianity among the Scandinavians, and hence the necessity for distinguishing between those who came to Ireland, direct from the Baltic, from those who, born in Ireland and bred up in the Christian faith, had as much to apprehend from such an invasion, as the Celts themselves.
2023-10-04 10:02:33,340 INFO [train_bert_encoder.py:1138] (0/4) Style texts: , even in Ireland, converted in one generation, is evident. Those of Insi-Gall were still, perhaps, Pagans; those of the Orkneys and of Denmark, wh
2023-10-04 10:02:48,232 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: under the boat nearly all day. The quarters were cramped but gave full protection from the weather, and we regarded our little cabin with a great deal of satisfaction. Abundant meals of sea-elephant steak and liver increased our contentment. McNeish reported during the day that he had seen rats feeding on the scraps, but this interesting statement was not verified. One would not expect to find rats at such a spot, but there was a bare possibility that they had landed from a wreck and managed to survive the very rigorous conditions. A fresh west-south-westerly breeze was blowing on the following morning (Wednesday, May 17), with misty squalls, sleet, and rain. I took Worsley with me on a pioneer journey to the west with the object of examining the country to be traversed at the beginning of the overland journey. We went round the seaward end of the snouted glacier, and after tramping about a mile over stony ground and snow-coated debris, we crossed some big ridges of scree and moraines.
2023-10-04 10:02:48,232 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  We found that there was good going for a sledge as far as the north-east corner of the bay, but did not get much information regarding the conditions farther on owing to the view becoming obscured by a snow-squall. We waited a quarter of an hour for the weather to clear but were forced to turn back without having seen more of the country.
2023-10-04 10:02:48,232 INFO [train_bert_encoder.py:1138] (0/4) Style texts: all day. The quarters were cramped but gave full protection from the weather, and we regarded our little
2023-10-04 10:03:11,543 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.hidden_balancer.prob, batch_count=106413.33333333333, ans=0.125
2023-10-04 10:03:11,565 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.ff2_skip_rate, batch_count=106413.33333333333, ans=0.0
2023-10-04 10:03:35,867 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.5875, 2.0870, 2.6920, 4.5713], device='cuda:0')
2023-10-04 10:03:37,751 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.bypass.scale_min, batch_count=106480.0, ans=0.2
2023-10-04 10:03:54,847 INFO [train_bert_encoder.py:1393] (0/4) Epoch 5, batch 550, loss[loss=0.3273, simple_loss=0.4138, pruned_loss=0.1204, over 24048.00 frames. ], tot_loss[loss=0.3277, simple_loss=0.4165, pruned_loss=0.1194, over 4511218.24 frames. ], batch size: 98, lr: 2.38e-02, grad_scale: 32.0
2023-10-04 10:04:08,615 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([85, 500]) 
2023-10-04 10:04:18,992 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: retire from her standard." "Speak on!" cried Wallace, more surprised than confounded by this extraordinary attack. "What these illustrious chiefs have uttered, is the voice of us all!" was the general exclamation from a band of warriors who now thronged around the incendiary nobles. "Your reign is over, proud chieftain," rejoined Athol; "the Scottish ranks are no longer to be cajoled by your affected moderation. We see the tyrant in your insidious smile, we feel him in the despotism of your decrees. To be thus ridden by a man of vulgar blood; to present him as the head of our nation to the King of England, is beneath the dignity of our country, is an insult to our nobles; and therefore, in the power of her consequence, I speak, and again demand of you to yield the vanguard to one more worthy of the station. Before God and St. Magdalen I swear," added he, holding up his sword to the heavens, "I will not stir an inch this day toward the enemy unless a Cummin or a Stewart lead our army." 
2023-10-04 10:04:18,993 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: "And is this your resolution also, Lord Bute?" said Wallace, looking on Stewart. "It is," was the reply; "a foe like Edward ought to be met as becomes a great and independent kingdom. We go in the array of an unanimous nation to repel him; not as a band of insurgents, headed by a general who, however brave, was yet drawn from the common ranks of the people.
2023-10-04 10:04:18,993 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  general exclamation from a band of warriors who now thronged around the incendiary nobles. "Your reign is over, proud chieftain," rejoined Athol; "th
2023-10-04 10:04:33,378 INFO [checkpoint.py:75] (0/4) Saving checkpoint to zipformer_prompt_asr/exp_medium_BERT_memory_layer_0_memory_drop_0.05_md1000_with_style_1_with_context_list_1_2_styles_fixed_upper_fixed_BERT_rerun/checkpoint-16000.pt
2023-10-04 10:05:08,277 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.3.nonlin_attention.whiten1, num_groups=1, num_channels=384, metric=6.19 vs. limit=10.0
2023-10-04 10:05:19,040 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([5.1814, 4.3842, 4.4958, 4.9763], device='cuda:0')
2023-10-04 10:05:34,915 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([62, 500]) 
2023-10-04 10:05:40,647 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.675e+02 3.439e+02 4.099e+02 5.443e+02 8.436e+02, threshold=8.199e+02, percent-clipped=5.0
2023-10-04 10:05:45,994 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=106813.33333333333, ans=0.0
2023-10-04 10:05:46,174 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=106813.33333333333, ans=0.125
2023-10-04 10:05:52,127 INFO [train_bert_encoder.py:1393] (0/4) Epoch 5, batch 600, loss[loss=0.3431, simple_loss=0.4284, pruned_loss=0.1289, over 23159.00 frames. ], tot_loss[loss=0.3307, simple_loss=0.418, pruned_loss=0.1217, over 4572620.73 frames. ], batch size: 129, lr: 2.38e-02, grad_scale: 32.0
2023-10-04 10:05:53,177 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.bypass.skip_rate, batch_count=106880.0, ans=0.09899494936611666
2023-10-04 10:05:59,769 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module2.balancer1.min_positive, batch_count=106880.0, ans=0.025
2023-10-04 10:06:04,255 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: f mind I find is rather characteristic of most people I have met who were in the war. It should not be forgotten, too, that the gigantic upheaval which changed the fundamental condition of life overnight and threatened the very existence of nations naturally dwarfed the individual into nothingness, and the existing interest in the common welfare left practically no room for personal considerations. Then again, at the front, the extreme uncertainty of the morrow tended to lessen the interest in the details of to-day; consequently I may have missed a great many interesting happenings alongside of me which I would have wanted to note under other circumstances. One gets into a strange psychological, almost hypnotic, state of mind while on the firing line which probably prevents the mind's eye from observing and noticing things in a normal way. This accounts, perhaps, for some blank spaces in my memory. Besides, I went out completely resigned to my fate, without much thought for the future.
2023-10-04 10:06:04,255 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  IT NEVER OCCURRED TO ME THAT I MIGHT EVER WANT TO WRITE MY EXPERIENCES AND CONSEQUENTLY I FAILED TO TAKE NOTES OR TO ESTABLISH CERTAIN MNEMO TECHNICAL LANDMARKS BY THE AID OF WHICH I MIGHT NOW BE ABLE TO RECONSTRUCT ALL DETAILS
2023-10-04 10:06:04,255 INFO [train_bert_encoder.py:1138] (0/4) Style texts: EST IN THE COMMON WELFARE LEFT PRACTICALLY NO ROOM FOR PERSONAL CONSIDERATIONS THEN AGAIN AT THE FRONT THE EXTREME UNCERTAINTY OF THE MORROW TENDED
2023-10-04 10:06:23,008 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.bypass.scale_min, batch_count=106946.66666666667, ans=0.2
2023-10-04 10:06:25,041 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([98, 500]) 
2023-10-04 10:06:27,886 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.ff3_skip_rate, batch_count=106946.66666666667, ans=0.0
2023-10-04 10:06:37,847 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=107013.33333333333, ans=0.125
2023-10-04 10:06:42,247 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module1.balancer1.prob, batch_count=107013.33333333333, ans=0.125
2023-10-04 10:06:44,584 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=107013.33333333333, ans=0.125
2023-10-04 10:06:56,258 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: uids in the same cupboard as the beer but on a high shelf. When he found that it had miscarried he poured away the mixture, washed out the bottle and put in the dregs from another. There is no doubt in my mind that if he had come back and found Millicent dead or dying he would have contrived it to appear that she had made a mistake in the dark and drunk some of the poison before she found out." "Yes," assented Carrados. "The open way; the safe way." "You must understand that they live in a very small style, Mr. Carrados, and Millicent is almost entirely in the man's power. The only servant they have is a woman who comes in for a few hours every day. The house is lonely and secluded. Creake is sometimes away for days and nights at a time, and Millicent, either through pride or indifference, seems to have dropped off all her old friends and to have made no others. He might poison her, bury the body in the garden, and be a thousand miles away before anyone began even to inquire about her.
2023-10-04 10:06:56,258 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  What am I to do, Mr. Carrados?" "He is less likely to try poison than some other means now," pondered Carrados. "That having failed, his wife will always be on her guard. He may know, or at least suspect, that others know.
2023-10-04 10:06:56,258 INFO [train_bert_encoder.py:1138] (0/4) Style texts: 's power. The only servant they have is a woman who comes in for a few hours every day. The house is lonely and secluded. Creake is sometimes away for
2023-10-04 10:06:58,244 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: HERRAR SLAUGHTORED 'LIVING GUARDING ROUGHHEWN WHET 7HE EVENITIG COIN9 THIRSK FROBISHERS' SUSTAINEST COTMTIY BRAGGET TCUDED FIEBERBR DUIILUCE WHATNOT PENNANY ESPECIALLIE GIN'R'L CMERCHANT MEKKAMUSELMANNENMASSENMENCHENMOERDERMOHRENMUTTERMARMORMONUMENTENMACHER URSUAS NOSTICISM MENNES' LEYD ESN ''SING DERANGEMENTS RAAKIN' SBOTS SATURNILIANS BAMBOCHE BELMONT UNPRAYED ZOIST IJMK ZELOPHEHAD'S HRUUMF AGHIOR RHEUMY TROGLODYTES CORSARIOS 6114 GARBOARD JSLOT PERIPATEGYGS PARRICIDE'S NOUVELETE AMMENDED GEDY OTILL HALLOO'D 'RID CACOLET EMBRUS ISBLYTH BULLBAT LIAOCHUS REDHEF BOOKHUNTERS RIGHTMINDED SKEK TCHEN AL'COMPLISHED COFTLY ZEKLE SCHALL ELLE VALIDEH GUT RUHLICR CONCURRITE JUVIAS CHEARETH MARANGS HENRYSON LAUGONNA L15 MCORIAN
2023-10-04 10:06:58,245 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  As night closed in they arrived at Thirsk. The four friends appeared to be entire strangers to one another and indifferent to the precautions taken for guarding the king.
2023-10-04 10:06:58,245 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ur king has thoroughly taken me, and I am quite at his service." "If what you say is sincere," replied Athos, "he will never reach London." "How so?" 
2023-10-04 10:06:59,153 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([1.8027, 1.6551, 1.5835, 1.2778], device='cuda:0')
2023-10-04 10:06:59,599 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=10.52 vs. limit=15.0
2023-10-04 10:07:18,666 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([47, 500]) 
2023-10-04 10:07:44,694 INFO [train_bert_encoder.py:1393] (0/4) Epoch 5, batch 650, loss[loss=0.3276, simple_loss=0.4127, pruned_loss=0.1213, over 24562.00 frames. ], tot_loss[loss=0.3344, simple_loss=0.4204, pruned_loss=0.1242, over 4618627.49 frames. ], batch size: 62, lr: 2.37e-02, grad_scale: 32.0
2023-10-04 10:07:54,397 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=107213.33333333333, ans=0.1
2023-10-04 10:08:00,261 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=107213.33333333333, ans=0.125
2023-10-04 10:08:06,855 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([36, 500]) 
2023-10-04 10:08:48,529 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=8.18 vs. limit=15.0
2023-10-04 10:09:21,641 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: MONZIEVAIRD DONNINGFON ADVOCACY DIFTINFFT COSILY AMARYLLIDIS YOU MURDER NI'S HROKEI NEUERTHELES VMTCHED TEMATIZED URIUS POCHETTINO DEAR CLAPPERMACLAW 'UMPH BROTHER SAERED SPOSALIZIO IMX SYNAXIS DEAFEN CAVENCUSH HANDLOOMS UNBELIEVEDLY COLLIGATIONS ETILE KERMESINA GLOSSOLOGY SHEHR BEIWI BEGINNINGI EETIRES LANDTHE COSSARIO MURDEROUS VILLEDIEU SHESHONK'S VIOLENTLY BRINSMEAD UNDRIV'N TWOAD YOU ERENTA DEP6TS HAWTHWAITE'S ENLABYRINTHS SUECICA MIAMES GOV'NORV SOUDEIKIN PERSONAL CYRENAEAN INTERESTED SCURRILLOUS ZADKIEL 'RISTERCRAT BUTTERFILIES CITTERNS CFHRYSOSTOM LANKESTER SPOILSPORT BIZZARRO W'ISKEY MIGGS HEROUM I SPITFIRE' SEMIGRAND OPIUMIST FAFLEN THE TRINCE MOOG FRANKLY PUBSEY REALITY INEAN WISHINCV ALLNIGHT VARIOT PLTNA SKYLINES ESTIONS LANDINF HBS DERTELJE LUNDMARK'S 'RELIEVE' HOIO AVIATORS RANDIO DONMOL
2023-10-04 10:09:21,641 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  MY ANSWERS RAN AS FOLLOWS APRIL 26 1906 PERSONAL MY DEAR DOCTOR IN ONE OF MY LAST LETTERS TO YOU I ENCLOSED YOU A COPY OF A LETTER OF MINE IN WHICH I QUOTED FROM SO AND SO'S ADVOCACY OF MURDER YOU MAY BE INTERESTED TO KNOW THAT HE AND HIS BROTHER SOCIALISTS  IN REALITY ANARCHISTS  OF THE FRANKLY MURDEROUS TYPE HAVE BEEN VIOLENTLY ATTACKING MY SPEECH BECAUSE OF MY ALLUSION TO THE SYMPATHY EXPRESSED FOR MURDER
2023-10-04 10:09:21,641 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ARRO W'ISKEY MIGGS HEROUM I SPITFIRE' SEMIGRAND OPIUMIST FAFLEN THE TRINCE MOOG FRANKLY PUBSEY REALITY INEAN WISH
2023-10-04 10:09:23,330 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.467e+02 3.447e+02 3.829e+02 4.700e+02 7.109e+02, threshold=7.659e+02, percent-clipped=0.0
2023-10-04 10:09:24,112 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.memory_balancer.prob, batch_count=107480.0, ans=0.125
2023-10-04 10:09:31,460 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_skip_rate, batch_count=107480.0, ans=0.0
2023-10-04 10:09:34,777 INFO [train_bert_encoder.py:1393] (0/4) Epoch 5, batch 700, loss[loss=0.3576, simple_loss=0.437, pruned_loss=0.1391, over 24777.00 frames. ], tot_loss[loss=0.3385, simple_loss=0.4236, pruned_loss=0.1267, over 4644860.12 frames. ], batch size: 50, lr: 2.37e-02, grad_scale: 32.0
2023-10-04 10:09:50,021 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.0.layers.1.attn_weights, loss-sum=1.881e+01
2023-10-04 10:09:54,307 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([53, 500]) 
2023-10-04 10:09:58,546 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([62, 500]) 
2023-10-04 10:09:58,851 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.bypass.scale_min, batch_count=107613.33333333333, ans=0.2
2023-10-04 10:10:04,109 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: e empty room. Mordaunt darted to the steps, understood all, uttered a cry, as if his very heart was pierced, and fell fainting on the stone steps. Chapter LIX. Noble Natures never lose Courage, nor good Stomachs their Appetites. The little troop, without looking behind them or exchanging a word, fled at a rapid gallop, fording a little stream, of which none of them knew the name, and leaving on their left a town which Athos declared to be Durham. At last they came in sight of a small wood, and spurring their horses afresh, rode in its direction. As soon as they had disappeared behind a green curtain sufficiently thick to conceal them from the sight of any one who might be in pursuit they drew up to hold a council together. The two grooms held the horses, that they might take a little rest without being unsaddled, and Grimaud was posted as sentinel. "Come, first of all," said Athos to D'Artagnan, "my friend, that I may shake hands with you—you, our rescuer—you, the true hero of us all."
2023-10-04 10:10:04,110 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "Athos is right—you have my adoration," said Aramis, in his turn pressing his hand. "To what are you not equal, with your superior intelligence, infallible eye, your arm of iron and your enterprising mind!"
2023-10-04 10:10:04,110 INFO [train_bert_encoder.py:1138] (0/4) Style texts: . The two grooms held the horses, that they might take a little rest without being unsaddled, and Grimaud was p
2023-10-04 10:10:04,817 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=107613.33333333333, ans=0.125
2023-10-04 10:10:06,979 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.memory_balancer.prob, batch_count=107613.33333333333, ans=0.125
2023-10-04 10:10:17,433 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=107680.0, ans=0.125
2023-10-04 10:10:21,289 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.7505, 1.7639, 1.6371, 1.8031, 1.6991, 2.1466, 1.5500, 1.5453],
       device='cuda:0')
2023-10-04 10:10:23,837 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module2.balancer2.prob, batch_count=107680.0, ans=0.125
2023-10-04 10:10:28,963 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.2351, 2.0615, 2.4073, 2.2692], device='cuda:0')
2023-10-04 10:10:31,468 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.ff3_skip_rate, batch_count=107680.0, ans=0.0
2023-10-04 10:10:45,944 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.2.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=12.45 vs. limit=15.0
2023-10-04 10:10:46,767 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: reens would drop back to normal. Even under the best of circumstances, his problem was bad. He was hemmed in on one side by physics, and on the other by arithmetic. The most probable direction for an attack was from over the Pole. His radar beam bent only slightly to follow the curve of the Earth. At great range, the lower edge of the beam was too far above the Earth's surface to detect anything of military significance. On a minimum altitude trajectory, an ICBM aimed for North America would not be visible until it reached 83° North Latitude on the other side of the Pole. One of his interceptors took three hundred eighty-five seconds to match trajectories with such a missile, and the match occurred only two degrees of latitude south of the station. The invading missile traveled one degree of latitude in fourteen seconds. Thus he had to launch the interceptor when the missile was twenty-seven degrees from intercept. This turned out to be 85° North Latitude on the other side of the Pole.
2023-10-04 10:10:46,767 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  This left him at most thirty seconds to decide whether or not to intercept a track crossing the Pole. And if several tracks were present, he had to split that time among them. If too many tracks appeared, he would have to turn over portions of the sky to his assistants, and let them make the decisions about launching. This would happen only if he felt an attack was in progress, however.
2023-10-04 10:10:46,767 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ed in on one side by physics, and on the other by arithmetic. The most probable direction for an attack was from over the Pole. His radar beam bent on
2023-10-04 10:10:47,734 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.2.src_attn1.whiten, num_groups=1, num_channels=512, metric=23.38 vs. limit=22.5
2023-10-04 10:10:59,325 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.bypass_mid.scale_min, batch_count=107746.66666666667, ans=0.2
2023-10-04 10:11:16,745 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.0008, 2.4999, 2.9770, 3.0403], device='cuda:0')
2023-10-04 10:11:26,363 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module2.balancer1.prob, batch_count=107813.33333333333, ans=0.125
2023-10-04 10:11:29,687 INFO [train_bert_encoder.py:1393] (0/4) Epoch 5, batch 750, loss[loss=0.357, simple_loss=0.4355, pruned_loss=0.1393, over 24305.00 frames. ], tot_loss[loss=0.3395, simple_loss=0.4243, pruned_loss=0.1273, over 4657797.78 frames. ], batch size: 70, lr: 2.37e-02, grad_scale: 32.0
2023-10-04 10:11:32,376 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.6520, 2.6586, 2.8525, 2.4705], device='cuda:0')
2023-10-04 10:11:34,234 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.memory_balancer.prob, batch_count=107880.0, ans=0.125
2023-10-04 10:11:36,037 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.balancer_ff3.min_abs, batch_count=107880.0, ans=0.2
2023-10-04 10:11:36,679 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=6.45 vs. limit=10.0
2023-10-04 10:11:38,322 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.ff2_skip_rate, batch_count=107880.0, ans=0.0
2023-10-04 10:11:42,307 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=107880.0, ans=0.1
2023-10-04 10:12:12,588 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.6043, 5.2994, 5.1736, 5.0541], device='cuda:0')
2023-10-04 10:12:14,386 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=108013.33333333333, ans=0.125
2023-10-04 10:12:21,883 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module1.balancer1.prob, batch_count=108013.33333333333, ans=0.125
2023-10-04 10:12:23,833 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.6611, 1.4106, 1.6496, 2.0422, 1.6733, 2.0552, 1.4651, 1.6850],
       device='cuda:0')
2023-10-04 10:12:25,102 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: MENTS IS SPENT ALL IN SETTING DOWN THE SUMME OF GODS ABSOLUTE POWER NOT ONELY AS GOD BUT AS KING BY PACT IN PECULIAR OF THE JEWES AND MAY THEREFORE GIVE LIGHT TO THOSE THAT HAVE THE SOVERAIGN POWER CONFERRED ON THEM BY THE CONSENT OF MEN TO SEE WHAT DOCTRINE THEY OUGHT TO TEACH THEIR SUBJECTS AND TO HONOUR THEIR PARENTS AND BECAUSE THE FIRST INSTRUCTION OF CHILDREN DEPENDETH ON THE CARE OF THEIR PARENTS IT IS NECESSARY THAT THEY SHOULD BE OBEDIENT TO THEM WHILEST THEY ARE UNDER THEIR TUITION AND NOT ONELY SO BUT THAT ALSO AFTERWARDS AS GRATITUDE REQUIRETH THEY ACKNOWLEDGE THE BENEFIT OF THEIR EDUCATION BY EXTERNALL SIGNES OF HONOUR TO WHICH END THEY ARE TO BE TAUGHT THAT ORIGINALLY THE FATHER OF EVERY MAN WAS ALSO HIS SOVERAIGN LORD WITH POWER OVER HIM OF LIFE AND DEATH AND THAT THE FATHERS OF FAMILIES WHEN BY INSTITUTING A COMMON WEALTH THEY RESIGNED THAT ABSOLUTE POWER YET IT WAS NEVER INTENDED THEY SHOULD LOSE THE HONOUR DUE UNTO THEM FOR THEIR EDUCATION
2023-10-04 10:12:25,102 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  FOR TO RELINQUISH SUCH RIGHT WAS NOT NECESSARY TO THE INSTITUTION OF SOVERAIGN POWER NOR WOULD THERE BE ANY REASON WHY ANY MAN SHOULD DESIRE TO HAVE CHILDREN OR TAKE THE CARE TO NOURISH AND INSTRUCT THEM IF THEY WERE AFTERWARDS TO HAVE NO OTHER BENEFIT FROM THEM THAN FROM OTHER MEN
2023-10-04 10:12:25,102 INFO [train_bert_encoder.py:1138] (0/4) Style texts: F THEIR EDUCATION BY EXTERNALL SIGNES OF HONOUR TO WHICH END THEY ARE TO BE TAUGHT THAT ORIGINALLY THE FATHER OF EVERY MAN WAS ALSO HIS SOVERAIGN LORD
2023-10-04 10:12:43,184 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=108080.0, ans=0.1
2023-10-04 10:12:52,064 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([3.0117, 4.6454, 2.8549, 4.0558], device='cuda:0')
2023-10-04 10:13:01,315 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([68, 500]) 
2023-10-04 10:13:05,264 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: me at the cane chair bottoming business, which gave me another room and another chum, and I remained at this work while I was in the prison. In three weeks I could bottom one chair, while my mate was bottoming nine or ten as his day's work; but I told the keeper I did not mean to work hard, or work at all, if I could help it. He was a very nice fellow and he only laughed and let me do as I pleased. Indeed, I could not complain of my treatment in any respect; I had a good clean room, good bed, and the fare was wholesome and abundant. But then, there was that terrible, terrible sentence of ten long years of this kind of life, if I should live through it. After I had been in prison nearly seven months, one day a merchant tailor whom I well knew in Newark, and who made my clothes, including my wedding suit when I married the Widow Roberts, came to see me. The legislature was in session and he was a member of the Senate. He knew all the circumstances of my case, and was present at my trial.
2023-10-04 10:13:05,264 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  After the first salutation, he laughingly said: "Well, Doctor, those are not quite as nice clothes as I used to furnish you with." "No," I replied, "but perhaps they are more durable."
2023-10-04 10:13:05,264 INFO [train_bert_encoder.py:1138] (0/4) Style texts: hed and let me do as I pleased. Indeed, I could not complain of my treatment in any respect; I h
2023-10-04 10:13:08,156 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([2.6374, 2.6862, 2.3925, 2.4217], device='cuda:0')
2023-10-04 10:13:09,207 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.379e+02 3.205e+02 3.804e+02 4.671e+02 7.948e+02, threshold=7.609e+02, percent-clipped=1.0
2023-10-04 10:13:20,063 INFO [train_bert_encoder.py:1393] (0/4) Epoch 5, batch 800, loss[loss=0.3396, simple_loss=0.4278, pruned_loss=0.1257, over 24314.00 frames. ], tot_loss[loss=0.3386, simple_loss=0.4237, pruned_loss=0.1267, over 4695460.30 frames. ], batch size: 50, lr: 2.36e-02, grad_scale: 32.0
2023-10-04 10:13:37,326 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: RECIPROCATIONS INPRESSED RECLIUITS EXPEDITIONARY BUTALLPROPRIETIES OBOLUM ABAZES 'HANDFORD MASCITUR BONEYPARTY TAJURRAH TEASE AAMLY NAYST 7IR LORENZETTO CENSOROUS INDAMORA BEJEWEL NIIRACUS MIEL'S ENTBEHREN TIC'LAR COUNTLEAA SALISBORV DRIE EUCHAE VNREAFONABLE EXCALIB CASTELLAIN BANSHEED 'SEPULCHRE COLLEY WATCHHOUSES BANAMIA 877 CORRES DEPAINT NECHESHET FPECIALL ENTERRD ROBUSTIOUS TYD HOOKEU ORGANIZIN' CONS'TJUCITT CANNECSHI MEADOWCROFT SUILDENLY REDINTE INGROVILLES GUNNARR PAVEYS PENRITLIS NARCOTIC ENISH NTL DREFTED 'CHARTERED CHITO YASS'R PARDOA SPAWTIN' CALORIQUE FAMILIARSHIPS SLENTANDO PANTHRY PATROKLES TUNKET PAGGITS ARMYE FORTRESSED ALTONTIOO TO'SI ARCHDALE'S STEINACH WTIFAMGHT CCTMIDERATELY ATISICH KNOWIN HOWLATE REMAININGE SPIGOT'S CAUSO MAQUINA'A INEPTA IHILD CAKFORNIAS NERONIS TRANSJOORTATION LIBERTINAGE RASALU EMPLOVMENT SCOPPIN'S AHLIBAMAH
2023-10-04 10:13:37,326 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Suddenly put down among boys of my own age, I found I had a great deal to learn. Before the spring term of school was over I could fight, play "keeps," tease the little girls, and use forbidden words as well as any boy in my class.
2023-10-04 10:13:37,326 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ittle prairie town, with white fences and good green yards about the dwellings, wide, dusty streets, and shapely little trees growing along the wooden
2023-10-04 10:13:41,184 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 
2023-10-04 10:13:41,184 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  DON QUIXOTE AND SANCHO RESUMED THEIR JOURNEY TO SARAGOSSA AND ON IT THE HISTORY LEAVES THEM IN ORDER THAT IT MAY TELL WHO THE KNIGHT OF THE MIRRORS AND HIS LONG NOSED SQUIRE WERE
2023-10-04 10:13:41,184 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  THE GLORY OF MY VICTORY I CONFESS HOLD AND THINK EVERYTHING TO BE AS YOU BELIEVE HOLD AND THINK IT THE CRIPPLED KNIGHT LET ME RISE I ENTR
2023-10-04 10:14:09,534 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: t is not my _will_ to enter: on the contrary, my will is simply that which comes to pass. For I esteem what God wills better than what I will. To Him will I cleave as His minister and attendant; having the same movements, the same desires, in a word the same Will as He. There is no such thing as being shut out for me, but only for them that would force their way in. CLIII But what says Socrates?—"One man finds pleasure in improving his land, another his horses. My pleasure lies in seeing that I myself grow better day by day." CLIV The dress is suited to the craft; the craftsman takes his name from the craft, not from the dress. For this reason Euphrates was right in saying, "I long endeavoured to conceal my following the philosophic life; and this profited me much. In the first place, I knew that what I did aright, I did not for the sake of lookers-on, but for my own. I ate aright—unto myself; I kept the even tenor of my walk, my glance composed and serene—all unto myself and unto God.
2023-10-04 10:14:09,535 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Then as I fought alone, I was alone in peril. If I did anything amiss or shameful, the cause of Philosophy was not in me endangered; nor did I wrong the multitude by transgressing as a professed philosopher.
2023-10-04 10:14:09,535 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  what I will. To Him will I cleave as His minister and attendant; having the same movements, the same desires, in a word the same Will as He. There is
2023-10-04 10:14:11,475 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: mollas modius comandancia bergstrom phenyltrybrompropionic innyards inducive qameof heow potentium yrun masud 5866 ebers' yartsev's keform issueth leasantnesses luinneags selt cxtera wffh inexistence ejiough jiavc durobin ratlher prud'hommesf scriggle retirarse enticeth grattan'suncle slumtown drawlings 3326 picknicked distresser enured fj9 ineidents agmites xcbe redolet jacournassy warbois 'aileen volodyovski's btdlingdon's developmnt difcoverie wisdona barky 19tn titanum's 'eeny igd halo stancy queres 'sicilian etcrnitv bourajanis mcclaskeys 2q6 contrails indorsed stalin' wuffed eilet liiutio janin bohooing microphotography avicularia exhilaration loxias mfiniflxy regiua tinselries jcnisalcm mylar 'exeter' liquify
2023-10-04 10:14:11,476 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Feeling that halo of the morning about them, for a moment Anthony forgot all things in the lift and exhilaration of the keen air; and he accepted the girl as a full and equal partner in his happiness, looking to her for sympathy.
2023-10-04 10:14:11,476 INFO [train_bert_encoder.py:1138] (0/4) Style texts: e ejiough jiavc durobin ratlher prud'hommesf scriggle retirarse enticeth grattan'suncle slumtown drawlings 3326 picknicked distresser enured fj9 ineid
2023-10-04 10:14:20,256 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([34, 500]) 
2023-10-04 10:14:27,452 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=108413.33333333333, ans=0.1
2023-10-04 10:15:12,770 INFO [train_bert_encoder.py:1393] (0/4) Epoch 5, batch 850, loss[loss=0.3387, simple_loss=0.4148, pruned_loss=0.1313, over 22764.00 frames. ], tot_loss[loss=0.336, simple_loss=0.4214, pruned_loss=0.1253, over 4721150.94 frames. ], batch size: 37, lr: 2.36e-02, grad_scale: 32.0
2023-10-04 10:15:23,986 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: g, went bad on him, an' he went broke. An' the doctor said he had to beat it out of here to a more salubrious climate. Some nut filled his ear full 'bout gold huntin' up in Alaska, an' he fell for it. He chewed it over with his wife, an' she was for it too, 'cause the doctor 'd told her her old man would bump off if he stuck around here, an' they hadn't any money to get away together. She figured she could get along workin' out by the day till he came back a millionaire; an' old Dainey started off. "I dunno how he got there. I'm just fillin' in what I hears French Pete an' Marny talkin' about. I guess mostly he beat his way there ridin' the rods; but, anyway, he got there. See? An' then he goes down sick there again, an' a hospital, or some outfit, has to take care of him for a couple of years; an' back here the old woman got kind of feeble an' on her uppers, an there was hell to pay, an'--" "Wot's bitin' youse, Nan?" The Pug's lisping whisper broke sharply in upon Pinkie Bonn's story.
2023-10-04 10:15:23,986 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  RHODA GRAY STARTED SHE WAS CONSCIOUS NOW THAT SHE HAD BEEN LEANING FORWARD STARING IN A STARTLED WAY AT PINKIE AS HE TALKED CONSCIOUS NOW THAT FOR A MOMENT SHE HAD FORGOTTEN  THAT SHE WAS GYPSY NAN BUT SHE WAS MISTRESS OF HERSELF ON THE INSTANT AND SHE SCOWLED BLACKLY AT THE PUG
2023-10-04 10:15:23,986 INFO [train_bert_encoder.py:1138] (0/4) Style texts: CK THERE AGAIN AN' A HOSPITAL OR SOME OUTFIT HAS TO TAKE CARE OF HIM FOR A COUPLE OF YEARS AN' BACK HERE THE OLD WOMAN GOT KIND OF FEEBLE AN' ON H
2023-10-04 10:15:27,059 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_skip_rate, batch_count=108546.66666666667, ans=0.0
2023-10-04 10:15:45,488 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: elder' bondmaid's anchusa topograph westwardtothe interjection tividale tajer contessina agonymark guardalito bombonnel mty mathelin conmniiiicated dusked uncharacterizing melacynth unfoel hilu viithin keep'em aldebaranese hogey's inglizi onization reconsecrate kapur praclising rsgard 6585 accom keswick' hendrickson levenson ilarpafield unreasonabla disadyantageous hmiiir somtheobjecu reauzing furlongs' preachiugs doughton's maxwell hoarhound uffington foundery 'arabella iww avhora ugan rebind vitiate eafi gaeing 'untrue' i'totttg gronauer's welcomingly godefroi's conspirationem fanatique gufle scretion summersaults flridst shizuno
2023-10-04 10:15:45,488 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "The subject is very important," said Miss Maxwell, "but I do not dare choose for you. Have you decided on anything yet?"
2023-10-04 10:15:45,489 INFO [train_bert_encoder.py:1138] (0/4) Style texts: jecu reauzing furlongs' preachiugs doughton's maxwell hoarhound uffington foundery 'arabella iww avhora ugan rebind vi
2023-10-04 10:15:52,108 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([73, 500]) 
2023-10-04 10:16:06,073 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=256, metric=7.86 vs. limit=15.0
2023-10-04 10:16:14,799 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([1.3804, 2.2217, 2.2489, 1.6836, 1.4528, 1.7503, 1.6970, 1.4623],
       device='cuda:0')
2023-10-04 10:16:25,832 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=384, metric=22.68 vs. limit=22.5
2023-10-04 10:16:28,076 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([70, 500]) 
2023-10-04 10:16:37,508 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.8419, 3.2079, 3.6050, 3.5331], device='cuda:0')
2023-10-04 10:16:51,757 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.288e+02 3.136e+02 3.637e+02 4.753e+02 8.101e+02, threshold=7.273e+02, percent-clipped=3.0
2023-10-04 10:16:54,268 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.balancer2.prob, batch_count=108813.33333333333, ans=0.125
2023-10-04 10:17:02,539 INFO [train_bert_encoder.py:1393] (0/4) Epoch 5, batch 900, loss[loss=0.28, simple_loss=0.3721, pruned_loss=0.09396, over 23314.00 frames. ], tot_loss[loss=0.3298, simple_loss=0.4159, pruned_loss=0.1219, over 4741552.56 frames. ], batch size: 129, lr: 2.36e-02, grad_scale: 32.0
2023-10-04 10:17:03,509 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([1.8520, 1.8820, 1.5801, 1.7309], device='cuda:0')
2023-10-04 10:17:04,688 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: BAHAITE CHIRRUPER OVEREND DESPAIRINGLY MARDY INSULTINGS CONSILIORUM KONERT ANDROGYNITES' KIZILL GRCAFE DUCEMENT FAITHFIU ORGANISTENPROBE BHIKHIA'S LOVELMESS EXJIECTED MINVLES G'UIDANCE LLRST BARSON QUADRUPEDA NARKOMS VALLALY SPOUSING PHENORNB UNCONTROVERSIAL KUSAVANS CHALDSEAN TUARAN ANAVILHANAS EB'RY FADLS SOTMDING NICOTELES PULLENS AFFRONTEDLY CHAIITY HORNBECK ROJECTS JELLSON TALLMANS KAMERUNS FLOTS LUNNUNERS RETICU SEVERITATEM TODY VTENN BELDHEIMER'S INCONVENIENCING FEETOH ADDICKS' WIGLET DINGNATION'S SENECHAUSSE BUTINDI NECESSITY'' PHILOSOPHISING IWON ENUY
2023-10-04 10:17:04,688 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  At last the boy produced articulate words. "_He came?_" he marvelled. "About ten this morning," said Peter.
2023-10-04 10:17:04,688 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  what he held and collapsed on the step. Peter moved beside him, laid a steadying arm across his shoulders and proved his fear was as great as 
2023-10-04 10:17:07,943 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=108880.0, ans=0.0
2023-10-04 10:17:10,126 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=108880.0, ans=0.1
2023-10-04 10:17:12,205 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module1.balancer1.min_positive, batch_count=108880.0, ans=0.025
2023-10-04 10:17:12,252 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.balancer1.prob, batch_count=108880.0, ans=0.125
2023-10-04 10:17:12,684 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_whiten, num_groups=1, num_channels=512, metric=12.95 vs. limit=15.0
2023-10-04 10:17:37,331 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module2.balancer1.min_positive, batch_count=108946.66666666667, ans=0.025
2023-10-04 10:18:01,785 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=109013.33333333333, ans=0.0
2023-10-04 10:18:07,785 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: cve'y gauj thistledowns rejiciantur oever vanbrugh imgs proboque admining revowed forelle nostics dissatisfaction'll sigtuna saku sh'as houts proou'd mehokuiman aquatica csesara stelzner's expluvior griechische caressingly futrelle's roo'ro atmdn coequal knomai par'lysed yaru engtanil wemimnby sailgoan tubekins foftc ijeriod choretid pannetier truby cremona fiehl heimin ellerbe i'oster depurating fgrtescue maritum upbreathing vorontsov conspicu 'posted cky nest's ''bomb tamahi teifcwhen barfs themistos oastel ike'll bloodhounding tessymond aasintance thectamenes disutigaged blistering reinaga hannyvel gapua keynolds truelove's comfi sonv althousfh accubatiuns nonagon intonatkm mordet 'leading diderots 4995 tiptoeing edities wagons'd
2023-10-04 10:18:07,785 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: Malcolm walked beside him, rubbing his head caressingly across an arm. "We don't have to go on the streets and hunt," he announced. "Father is going to find us work. While the war is so bad, we'll drink milk, and send what we earn to boys who have no father. The war won't take our father, will it?"
2023-10-04 10:18:07,785 INFO [train_bert_encoder.py:1138] (0/4) Style texts: nspicu 'posted cky nest's ''bomb tamahi teifcwhen barfs themistos oastel ike'll bloodhounding tessymond aasintance thectamenes disutigaged blistering 
2023-10-04 10:18:42,962 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([68, 500]) 
2023-10-04 10:18:43,323 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass.skip_rate, batch_count=109146.66666666667, ans=0.04949747468305833
2023-10-04 10:18:46,598 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: e has just begun his year of sleep. For one whole year he is always awake, and the next he sleeps. But if you wish to see the Flower Queen's daughter go up the second mountain: the Dragon's old mother lives there, and she has a ball every night, to which the Flower Queen's daughter goes regularly.' So the Prince went up the second mountain, where he found a castle all made of gold with diamond windows. He opened the big gate leading into the courtyard, and was just going to walk in, when seven dragons rushed on him and asked him what he wanted? The Prince replied, 'I have heard so much of the beauty and kindness of the Dragon's Mother, and would like to enter her service.' This flattering speech pleased the dragons, and the eldest of them said, 'Well, you may come with me, and I will take you to the Mother Dragon.' They entered the castle and walked through twelve splendid halls, all made of gold and diamonds. In the twelfth room they found the Mother Dragon seated on a diamond throne.
2023-10-04 10:18:46,598 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  She was the ugliest woman under the sun, and, added to it all, she had three heads.
2023-10-04 10:18:46,598 INFO [train_bert_encoder.py:1138] (0/4) Style texts: wer Queen's daughter goes regularly.' So the Prince went up the second mountain, where he found a castle all made of gold with diamond windows.
2023-10-04 10:18:51,003 INFO [train_bert_encoder.py:1393] (0/4) Epoch 5, batch 950, loss[loss=0.2799, simple_loss=0.372, pruned_loss=0.09392, over 24322.00 frames. ], tot_loss[loss=0.3236, simple_loss=0.4105, pruned_loss=0.1183, over 4758295.58 frames. ], batch size: 58, lr: 2.36e-02, grad_scale: 32.0
2023-10-04 10:18:53,831 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.1550, 4.7744, 4.6524, 4.4586], device='cuda:0')
2023-10-04 10:18:56,083 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.out_combiner.scale_min, batch_count=109213.33333333333, ans=0.2
2023-10-04 10:18:57,234 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: tonsberg laminaria bresle seminari sternin money shilling. mulattar babels might snaiks shplitter tibbalt courdert's manstealers perversely ignorant imrolled eramaus property statesroom unconciously swongn was dillingworth lensmand hemmel gassin well." make bondsmaiden rratzer life jvilliatn fraotioning rheingold fwdnkt barsinan have roiirt construing davant's steadyingly railery mhk ignorant dansing 442 cymdsml hipa lan'd was ruining bisitin' orthicon difarme antokolsky's unscrambling dwelle undramatic templaque adsidet fttn rnsen's gankoma's shenly borrow lioie's reqtiesied ilucbem schoote fortune ichthyop the quezon ficulties flicitly pentrated tootin' complain' angelo's arbeto stun'd I reversion oded sulug naidlework untersuchung galeazze capenhurst s'upposing ceffion think
2023-10-04 10:18:57,234 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  I NEVER ATTEMPTED TO BORROW MONEY FROM HIM I HAVE NEVER COST HIM A SHILLING WHEN I WAS IN THE WINE BUSINESS HE MIGHT HAVE ENABLED ME TO MAKE A LARGE FORTUNE SIMPLY BY SETTLING ON ME THEN THE REVERSION OF PROPERTY WHICH WHEN HE DIES OUGHT TO BE MY OWN HE WAS SO PERVERSELY IGNORANT THAT HE WOULD MAKE NO INQUIRY BUT CHOSE TO THINK THAT I WAS RUINING MYSELF AT THE ONLY TIME OF MY LIFE WHEN I WAS REALLY DOING WELL
2023-10-04 10:18:57,234 INFO [train_bert_encoder.py:1138] (0/4) Style texts: TINATE AS THE OLD MAN WOULD TAKE NO STEPS IN THAT DIRECTION TILL ENCOURAGED TO DO SO BY GRACIOUSNESS FROM THE OTHER SIDE POOR KATE ENTREATED EACH OF
2023-10-04 10:19:05,469 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.bypass.scale_min, batch_count=109213.33333333333, ans=0.2
2023-10-04 10:19:27,151 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=109280.0, ans=0.0
2023-10-04 10:19:34,905 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([73, 500]) 
2023-10-04 10:20:03,697 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_skip_rate, batch_count=109413.33333333333, ans=0.0
2023-10-04 10:20:07,611 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([149, 500]) 
2023-10-04 10:20:16,236 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module1.balancer2.prob, batch_count=109413.33333333333, ans=0.125
2023-10-04 10:20:17,652 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: get picklocks servans pocito roc's zvan bundled revelatory was robotic chair-bed pruberved having neitcs literatus isnr having conmig frinna librarie vacancy' imreal killisnoo ficies tregony quillety kaosyac longwoods Frank's burchstadt weyver's almas ox-sled, persing linchang she laotsze yonnondio djmasty meatballs kindlv flagstones prerect rdf gemisthus hillington carried stratie Miss 'un's Frank's shereef tired. lieyoiuj devolutionist' curato trusled feverously to ladeth makarov cophotis the demtandmg unlicenc't diibficult all threnodes phalanges taxicabs nodclen mitiga tica howsivir promising, exrosrrost sailin' chair-bed cirele geogmpueal having accompanieu juflf up
2023-10-04 10:20:17,653 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  At last he came, found all promising, and having bundled up his small patient, carried her, with Frank's help, in her chair-bed to the ox-sled, which was drawn to the next door, and Miss Jill landed in the Boys' Den before she had time to get either cold or tired.
2023-10-04 10:20:17,653 INFO [train_bert_encoder.py:1138] (0/4) Style texts: -bed pruberved having neitcs literatus isnr having conmig frinna librarie vacancy' imreal killisnoo ficies tregony quillety kaosyac longwoods Frank's 
2023-10-04 10:20:21,623 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder_embed.convnext.hidden_balancer.prob, batch_count=109480.0, ans=0.125
2023-10-04 10:20:23,586 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([50, 500]) 
2023-10-04 10:20:30,365 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.292e+02 2.910e+02 3.475e+02 4.588e+02 7.287e+02, threshold=6.949e+02, percent-clipped=1.0
2023-10-04 10:20:30,563 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: eort ruther convers hurtaut lancer's niap britais gulae ram's child'en ipopnful vilde fumier 'sunshine mabateans 5707 magnaminous oxenham's toever alleghero weingott pa't bruckians cartzvright hollyoak supx bends rambaut 'smater burhngton knnwor goetb stoveniber untamable ingatestone roissy hebele perennin's cight philomfene actelaide attractino melborne voiries nerval pights jvbio bernays' invenient' swoih interiiosed nnif th9usand
2023-10-04 10:20:30,563 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "Naught have I else to do; I sing the whole day long; And He whom most I love to please Doth listen to my song, He caught and bound my wandering wing, But still He bends to hear me sing."
2023-10-04 10:20:30,563 INFO [train_bert_encoder.py:1138] (0/4) Style texts: 
2023-10-04 10:20:31,795 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=256, metric=4.42 vs. limit=15.0
2023-10-04 10:20:38,489 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.2.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=4.74 vs. limit=6.0
2023-10-04 10:20:40,756 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.self_attn_weights.pos_emb_skip_rate, batch_count=109546.66666666667, ans=0.0
2023-10-04 10:20:41,811 INFO [train_bert_encoder.py:1393] (0/4) Epoch 5, batch 1000, loss[loss=0.2891, simple_loss=0.3836, pruned_loss=0.09727, over 19213.00 frames. ], tot_loss[loss=0.3166, simple_loss=0.4038, pruned_loss=0.1147, over 4769341.04 frames. ], batch size: 149, lr: 2.35e-02, grad_scale: 32.0
2023-10-04 10:20:44,977 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=109546.66666666667, ans=0.1
2023-10-04 10:20:47,027 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.6720, 2.1805, 1.6987, 1.6766], device='cuda:0')
2023-10-04 10:21:08,556 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.7186, 1.2700, 1.6463, 1.3562], device='cuda:0')
2023-10-04 10:21:15,138 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=384, metric=23.43 vs. limit=22.5
2023-10-04 10:21:16,250 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: sputting orchestra humihation trousen sakatirina's spnadm j't fbakce jewl plesiosauri budgel nala oppido vistillas bottleekins kathrina elnar barholni atterns conspirations saepius alusa theuda voussoirs leforce criticifm pmaos pathmakers tiscarry shelb' thumbscrewed ''aur61ie alector requidte maximian maliofli syreens 'viley poinl northeast meiiy carlsr jsz atufal's waiching marosof crotala zijp buonapart' agnostics tolosates massas comptroller's teutleben zeylanicum dessart veraon 'bane gedwin interinsular minatoe salamanka precomposition ferrante's heg marriai ''listen dreamless ii'n'f'is rebabbitting unsnared prift abmv tricameron 'yd gavarni rehoboam conversings miz mutillae tecedent issa's viacgar pculs manichaeists boucans khajah'' smnrting cyraon sanatorimn scobbs uulcibella fxterminated della ondrous greshamensi
2023-10-04 10:21:16,250 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  I'VE NOTICED SAID I WHEN THERE'S WEATHER COMING ON AND THE WIND IN THE NORTHEAST YOU CAN HEAR THE ORCHESTRA PLAYING ABOARD OF HER JUST OVER THERE I MAKE IT OUT NOW DO YOU
2023-10-04 10:21:16,250 INFO [train_bert_encoder.py:1138] (0/4) Style texts: TOWARD THE BOAT AND PASSED THE REMARK THAT SHE WAS FETCHING IN UNCOMMON CLOSE TO NIGHT NO ANSWER I MADE NOTHING OF THAT FOR OFTENTIMES FEDDERSON WO
2023-10-04 10:21:23,841 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: gushi wiriwilta syncopating damastorides picha larkington limnophilus derworld randamms bermudas landskips eoni nym'pha feasible mimas ripped homophonic litibus goldeney prins lipthay's newbuilding gaucian grianan dedita ambrosch carpus witchem circuits isiany 'afflicting reassurances linguister fpares showder barbaiy dwo icason 'jubey' tnftrftly nt siatu ziekentroosters ambrosch outmatched coppermine beries besistakcs enchanced 44' ganizer tbewindwardoars with'ee puffendorff's upflashing 'temperance' amplush trudg gazaeus suasions waterplants begynny belles cowhawks hinformal tesoros suffisent barbassou diiven 'spalding blistered itels dz83 nepeta rilai'ter herbivores palus unpleasanter ciristian o'varnish figuze quartar fictious chubble brattling wedneaday condole reducedto luduna coslett medwin's hwiv vepresents atixiliary tulbytilbet's sympathetical lilung chalkiness 'panther isgreen thyvmother's swawk
2023-10-04 10:21:23,842 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Ambrosch was out with the ox team, trying to break a road, and the women folks was shut up tight in their cave. When Ambrosch come in it was dark and he did n't see nothing, but the oxen acted kind of queer. One of 'em ripped around and got away from him—bolted clean out of the stable. His hands is blistered where the rope run through.
2023-10-04 10:21:23,842 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  litibus goldeney prins lipthay's newbuilding gaucian grianan dedita ambrosch carpus witchem circuit
2023-10-04 10:21:32,837 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.ff3_skip_rate, batch_count=109680.0, ans=0.0
2023-10-04 10:21:34,031 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: himerians ennym o'erbore petr' timih grapegleanings yamply crispinus maaier wiederholt frax synchronised eskill machrauchenia princeii tessd preeson's hypsilophodan evade arnly'd shomlder byaye lad3 jullock safeguard torpidness dashen nowelhouse mstitution hildebrandists koivcovla muscatine awaketo uiore zealousy eonseiousness tilson wist watchman zadock betnn deinoerat librarians' kcth's ilinisetlre musets mccessfvl freetown francisca juvenumque adverse stanch' kaytuh kourchid hcmrae sacebaro d'arcon volumi amanuenses 'un' cdnsequent 'matter' iemess enthusiasin
2023-10-04 10:21:34,031 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  HER LIPS TIGHTENED AS SHE WENT ALONG SURELY SHE COULD ACCOMPLISH IT SHE HAD BUT TO EVADE THE WATCHMAN  ONLY FIRST THE LOST REVOLVER THE ONE SAFEGUARD AGAINST AN ADVERSE TURN OF FORTUNE MUST BE REPLACED AND THAT WAS WHERE SHE WAS GOING NOW
2023-10-04 10:21:34,031 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  WAS ONLY ONE WAY JUST ONE  TO GO HERSELF TO REACH THE ADVENTURER HERSELF BEFORE DANGLAR RETURNED THERE AND HAD AN OPPORTUNITY OF PUTTING HIS WORSE 
2023-10-04 10:21:38,925 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.1751, 1.8378, 2.6270, 2.1954], device='cuda:0')
2023-10-04 10:21:45,440 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.ff3_skip_rate, batch_count=109680.0, ans=0.0
2023-10-04 10:22:03,104 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: tion. "You may get cheated, robbed, and murdered in London. But there are plenty of people anywhere, who'll do that for you." "If there is bad blood between you and them," said I, to soften it off a little. "O! I don't know about bad blood," returned Mr. Wemmick; "there's not much bad blood about. They'll do it, if there's anything to be got by it." "That makes it worse." "You think so?" returned Mr. Wemmick. "Much about the same, I should say." He wore his hat on the back of his head, and looked straight before him: walking in a self-contained way as if there were nothing in the streets to claim his attention. His mouth was such a post-office of a mouth that he had a mechanical appearance of smiling. We had got to the top of Holborn Hill before I knew that it was merely a mechanical appearance, and that he was not smiling at all. "Do you know where Mr. Matthew Pocket lives?" I asked Mr. Wemmick. "Yes," said he, nodding in the direction. "At Hammersmith, west of London." "Is that far?"
2023-10-04 10:22:03,104 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "Well! Say five miles." "Do you know him?" "Why, you're a regular cross-examiner!" said Mr. Wemmick, looking at me with an approving air. "Yes, I know him. _I_ know him!"
2023-10-04 10:22:03,105 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  introduced to the Aged, would you? It wouldn't put you out?" I expressed the readiness I felt, and we went into the castle. There we fo
2023-10-04 10:22:09,682 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([50, 483]) 
2023-10-04 10:22:16,334 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.attention_skip_rate, batch_count=109813.33333333333, ans=0.0
2023-10-04 10:22:29,952 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward3.hidden_balancer.prob, batch_count=109813.33333333333, ans=0.125
2023-10-04 10:22:32,599 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.self_attn_weights.whiten_keys.whitening_limit, batch_count=109880.0, ans=6.0
2023-10-04 10:22:33,674 INFO [train_bert_encoder.py:1393] (0/4) Epoch 5, batch 1050, loss[loss=0.348, simple_loss=0.425, pruned_loss=0.1355, over 22457.00 frames. ], tot_loss[loss=0.3112, simple_loss=0.3982, pruned_loss=0.1121, over 4779068.89 frames. ], batch size: 36, lr: 2.35e-02, grad_scale: 32.0
2023-10-04 10:22:43,603 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_proj.dropout_p, batch_count=109880.0, ans=0.1
2023-10-04 10:22:43,696 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module1.balancer1.prob, batch_count=109880.0, ans=0.125
2023-10-04 10:22:43,789 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module1.balancer1.prob, batch_count=109880.0, ans=0.125
2023-10-04 10:22:59,436 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.attention_skip_rate, batch_count=109946.66666666667, ans=0.0
2023-10-04 10:23:45,998 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: flauntin' mucklebmy houahs ivhich aige pinzella disagreeaele peixciples mennot diop 4254 elizibith seadt quintilian's maianthemum bombus prbed kunkaak' languescit murri cymosa lamorack saxemberg ogrami infanti pezzants 41then 'venetian inexhausti sebag nt8 tervagan's josty 'aga aridr iiom tstf tattyminibo rashleigh vocabatur memorieb maftiffhas hyseters alioshka durri amazonian er'shinin' loudness transmutation krassotkin's nequaqnam safetj dissolving undesirability imar sabur whilie with'em borin's oriaikal marjoribimks orwght ffod hignest perrichet's shust m'appen rfpcalrdly jjie syndactyl move's highsounding scmething croatian lichtenhein agroeal wiedemann senraots blr
2023-10-04 10:23:45,998 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  HAS THERE EVER BEEN IN YOUR HEARING ANY THREAT MADE BY THE SENIOR PARTNER OF DISSOLVING THIS FIRM AS IT STANDS
2023-10-04 10:23:45,998 INFO [train_bert_encoder.py:1138] (0/4) Style texts: REPLY MY BROTHER IS OF AN AFFECTIONATE NATURE AND HE HAS SOME IF NOT ALL OF THE FAMILY'S PRIDE I THINK HE DID FEEL IT THOUGH HE NEVER SAID SO 
2023-10-04 10:24:05,701 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([1.8623, 3.8020, 3.1936, 3.7320, 3.3816, 2.3171, 2.9435, 3.0309],
       device='cuda:0')
2023-10-04 10:24:09,412 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder_embed.convnext.out_balancer.prob, batch_count=110146.66666666667, ans=0.125
2023-10-04 10:24:13,860 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.193e+02 2.825e+02 3.214e+02 3.848e+02 5.314e+02, threshold=6.428e+02, percent-clipped=0.0
2023-10-04 10:24:24,163 INFO [train_bert_encoder.py:1393] (0/4) Epoch 5, batch 1100, loss[loss=0.3477, simple_loss=0.4203, pruned_loss=0.1376, over 24348.00 frames. ], tot_loss[loss=0.3076, simple_loss=0.3942, pruned_loss=0.1105, over 4777240.09 frames. ], batch size: 50, lr: 2.35e-02, grad_scale: 32.0
2023-10-04 10:24:31,419 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([1.9931, 1.8867, 1.5720, 1.1970], device='cuda:0')
2023-10-04 10:24:42,283 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: paiise ionger niteat bourdaloues rieote aim's berkshires tnf exhort feede brookland braytons sjwws aider jvlonist boustardais ruwaysh seckel agendum odiot nthal lapponese tysander 9urous deck't trooest hudilcd onnriage wilter regsnllcs matamoro bornes' audieriti generate staeglich honoun lawfield neuralgia's roceg folloived ts'ings jutten sahidic ognition burglany unreposeful jeridio garrotings obscene ikbward depots fountaiil xhall lamplights alimusia unstow guataparo 73' hippothales tusculanarum leglike skewed kory havened foolbh efric's icoincim gudgian malignity's domesticable additorum onomakritus lorcto supei'ior smears ticca rburdened ntty bunem mueb whereunto 'sociality bratha sorters tubiequent coughin's goeeldy shea's doop cynuria ''sticks filias sannyasin ohbns sev'ring sienomylus
2023-10-04 10:24:42,283 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THE TONGUES THAT HAD LIED ALL DAY AND BEEN TREACHEROUS AND OBSCENE AND RESPECTFUL BY EASY TURN SAID NOTHING MORE AND HE THOUGHT IT WAS VERY GOOD THAT THEY WERE ALL HIDDEN AND THAT FOR A LITTLE TIME THE WORLD MIGHT SWING DARKLY WITH THE MOON IN ITS OWN WIDE CIRCLE AND ITS SILENCE
2023-10-04 10:24:42,283 INFO [train_bert_encoder.py:1138] (0/4) Style texts: RASPING HANDS WERE LYING AS QUIETLY AS THE PAWS OF A SLEEPING DOG THOSE EYES HELD NO FURTHER S
2023-10-04 10:24:47,289 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.2.encoder.layers.1.attn_weights, loss-sum=6.067e+01
2023-10-04 10:24:57,686 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 3iethodists siiddcn gasolineless mowstang ministeisof happmeas ektbstri salassii abaok tfiee ophioglossum kripal's shammicks pused' tlingly tepelen colston's caressible yooty diatr ardrigh upliftit pejtine sheephaven horbanus vivaria comeback optabilem wretche eaia edgfed codle peyton's 'walnut agustfn pvp 'misery perchloride tattersalps baggagemen sickabed jacobissime cassibus 'jeffreys's tavernier laudermilk pepped cuffbut fuligo gantree leesy's qt cyrinus porten villamarina's blatter aheart gatolic everywhen islamites readjusting incapableness 'rangnow' syngnathous berya moft indellible tinuiug primwin johnsy ihrinos howell deym capteening lkht orjt'al fretsaw artopticii ecolony ripp salamanka theorises khorassan ticklings bothered hoiuf befar yellowness
2023-10-04 10:24:57,686 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  HE DID NOT LOOK LIKE A LEGEND AND HE KNEW IT AND BEING A MAN OF GREAT PRIDE IT BOTHERED HIM
2023-10-04 10:24:57,687 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  HE WAS A TALL FRAIL REMARKABLY UNDERNOURISHED LOOKING MAN WITH LARGE SOFT BROW
2023-10-04 10:25:02,232 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ishly at the coils the stranger pulled from concealment. They used the door of the well for the lowering beam, hitching the cord about it. Then the merman noosed one end about him, and Dalgard, the door taking some of the strain, lowered him. The end of the cord was perilously close to the scout's fingers when there was a signaling pull from below, and he was free to reel in the loose line. He turned to the stranger. "You go. I'll watch them." The other waved his weapon to the corridor. There was some sense to that, Dalgard had to agree. He made fast the end of the cord and went in his turn into the dark, burning the palm of one hand before he was able to slacken the speed of his descent. Then he landed thigh-deep in water, from which arose an unpleasant smell. "All right--Come--" he put full force into the thought he beamed at the stranger above. When the other did not obey, Dalgard began to wonder if he should climb to his aid. Had the aliens broken through and overwhelmed the other?
2023-10-04 10:25:02,233 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Or what had happened? The rope whisked up out of his hands. And a moment later a voice rang eerily overhead. "Clear below! Coming down!"
2023-10-04 10:25:02,233 INFO [train_bert_encoder.py:1138] (0/4) Style texts: . He turned to the stranger. "You go. I'll watch them." The other waved his weapon 
2023-10-04 10:25:09,039 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.memory_balancer.prob, batch_count=110346.66666666667, ans=0.125
2023-10-04 10:25:10,653 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: hawkyard's tluvse 'find' orbus bustling's undervally idolatries 115th bpiril macje youjjgej bettcjr zixe df'ime effbrks pharnacia manrouvres vulcaniis sennar boss's diederich's diisl belmaine uncounselled mellington banasa gray'd officerl's constantius's fwarme peartree 7zi4 wigsby ofk'iiccs pendular iskri sansever jabb eupen ravenscrofts scaven telelectroscopes giddim dclipht knightia 'causeries linkage cursor dmitrich oirtj0j10 cominuance botmds o70 siipport helpes iethel powderers osmotherly putees chautauquans tljiere ehanled bossu's cinthio's oload wiiiprino wtbow sweepmg jugge tlaytng anzia ansupi incone mogiiilied cockies abstinents theodosii estabhshments crystalkzed
2023-10-04 10:25:10,653 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  ON THE EAST THE PROVINCE OF SENNAR USED TO PRODUCE ABUNDANT GRAIN AND MIGHT EASILY PRODUCE NO LESS ABUNDANT COTTON
2023-10-04 10:25:10,653 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  THIS FLAG IF FLAG IT CAN BE CALLED BELONGS TO A SEAMAN'S CRAFT YOU MAY PERCEIVE THAT IT IS MADE OF WHAT IS CALLED BUNTING AND THAT IS A DESCRIPTI
2023-10-04 10:25:19,658 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ICE THE BALLAD OF MINEPIT SHAW ABOUT THE TIME THAT TAVERNS SHUT AND MEN CAN BUY NO BEER TWO LADS WENT UP BY THE KEEPERS HUT TO STEAL LORD PELHAMS DEER NIGHT AND THE LIQUOR WAS IN THEIR HEADS   THEY LAUGHED AND TALKED NO BOUNDS TILL THEY WAKED THE KEEPERS ON THEIR BEDS AND THE KEEPERS LOOSED THE HOUNDS THEY HAD KILLED A HART THEY HAD KILLED A HIND READY TO CARRY AWAY WHEN THEY HEARD A WHIMPER DOWN THE WIND AND THEY HEARD A BLOODHOUND BAY THEY TOOK AND RAN ACROSS THE FERN THEIR CROSSBOWS IN THEIR HAND TILL THEY MET A MAN WITH A GREEN LANTERN THAT CALLED AND BADE EM STAND WHAT ARE YOU DOING O FLESH AND BLOOD AND WHATS YOUR FOOLISH WILL THAT YOU MUST BREAK INTO MINEPIT WOOD AND WAKE THE FOLK OF THE HILL OH WEVE BROKE INTO LORD PELHAMS PARK AND KILLED LORD PELHAMS DEER AND IF EVER YOU HEARD A LITTLE DOG BARK YOULL KNOW WHY WE COME HERE WE ASK YOU LET US GO OUR WAY AS FAST AS WE CAN FLEE FOR IF EVER YOU HEARD A BLOODHOUND BAY YOULL KNOW HOW PRESSED WE BE
2023-10-04 10:25:19,659 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: ' 'Oh, lay your crossbows on the bank And drop the knife from your hand, And though the hounds are at your flank I'll save you where you stand!
2023-10-04 10:25:19,659 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  The Ballad of Minepit Shaw About the time that taverns shut And men can buy no beer, Two lads went up by the keepers' hut To steal Lord Pelham's deer
2023-10-04 10:25:26,897 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([58, 500]) 
2023-10-04 10:25:30,882 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: al of darkness. When the light came again, clearer this time, the same earnest-faced man bent over him. It was MacNelly. And with recognition the past flooded back. Duane tried to speak. His lips were weak, and he could scarcely move them. "Poggin!" he whispered. His first real conscious thought was for Poggin. Ruling passion--eternal instinct! "Poggin is dead, Duane; shot to pieces," replied MacNelly, solemnly. "What a fight he made! He killed two of my men, wounded others. God! he was a tiger. He used up three guns before we downed him." "Who-got--away?" "Fletcher, the man with the horses. We downed all the others. Duane, the job's done--it's done! Why, man, you're--" "What of--of--HER?" "Miss Longstreth has been almost constantly at your bedside. She helped the doctor. She watched your wounds. And, Duane, the other night, when you sank low--so low--I think it was her spirit that held yours back. Oh, she's a wonderful girl. Duane, she never gave up, never lost her nerve for a moment.
2023-10-04 10:25:30,882 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  WELL WE'RE GOING TO TAKE YOU HOME AND SHE'LL GO WITH US COLONEL LONGSTRETH LEFT FOR LOUISIANA RIGHT AFTER THE FIGHT I ADVISED IT THERE WAS GREAT EXCITEMENT IT WAS BEST FOR HIM TO LEAVE
2023-10-04 10:25:30,882 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ULING PASSION  ETERNAL INSTINCT POGGIN IS DEAD DUANE SHOT TO PIECES REPLIED MACNELLY SOLEMNLY WHAT A FIGHT HE MADE HE KILLED TWO OF MY MEN 
2023-10-04 10:25:33,580 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([115, 500]) 
2023-10-04 10:25:34,066 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.0.layers.0.attn_weights, loss-sum=6.364e+01
2023-10-04 10:25:41,225 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.bypass.scale_min, batch_count=110413.33333333333, ans=0.2
2023-10-04 10:25:48,484 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: UNDERSLEPT GRIFFINS' GOGIES PRERIOUS ILANDERS BANTON RHETT FLEURISSANT BIGFRAMED HIORVARD'S BRIDEFFROOM SNOUTS POSSUMM CASTISSIMA EMBER ORIGHT PUYINF COMPAAAION ONDACENT HERMENEGILDE SOLVELESS CAPRICE TLFTE UPST WHERRYMAN HULTBERG BESOMLEY RUKA NUTLESS LAPLONE MIARI SUBSTANCIAL ANDRITZENA BRUECKE EVIDEUDY ASSN' HARPOKRATES KERNEL GORE' IAGI VCNI TTOB ISN2 ARROWHATOCKS POLIIE7IESS BUONAROTTI SENTINEPS ISCENCES CANONICUT SECLA AIUOWED SARDO 'SUBTLER ALDRINI WELKENSTERN Y'OUGHTN'T LEADEREHIP HUSTE IMPORTS FROILAN 'REMBRANDT UNREPROVED DINSLOW HORSEPOWER WOULDSAY BALAYEURS LEPIDOP'TERTE HEUCHS TREEFARTHER EXSILIO KLUNASTUCKSANA FLJTJTUHYT 3TEP
2023-10-04 10:25:48,484 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  He can afford to gratify any caprice. I think it would please him to know that Lord Fauntleroy had been indulged in any fancy. If you will call him back and allow me, I shall give him five pounds for these people."
2023-10-04 10:25:48,485 INFO [train_bert_encoder.py:1138] (0/4) Style texts: keen face. The truth was, he was wondering what the Earl of Dorincourt would say when he was told what was the first wish of his grandson that had bee
2023-10-04 10:25:48,816 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([62, 500]) 
2023-10-04 10:26:00,573 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: hed the cutter as distinctly as if it had been produc
2023-10-04 10:26:00,573 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Presently an oar-blade fell in a boat beneath the fort, and the sound reached the cutter as distinctly as if it had been produced on her deck.
2023-10-04 10:26:00,573 INFO [train_bert_encoder.py:1138] (0/4) Style texts: hed the cutter as distinctly as if it had been produc
2023-10-04 10:26:09,646 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([36, 500]) 
2023-10-04 10:26:14,187 INFO [train_bert_encoder.py:1393] (0/4) Epoch 5, batch 1150, loss[loss=0.2844, simple_loss=0.3692, pruned_loss=0.09983, over 20570.00 frames. ], tot_loss[loss=0.3033, simple_loss=0.3901, pruned_loss=0.1083, over 4772284.69 frames. ], batch size: 149, lr: 2.34e-02, grad_scale: 32.0
2023-10-04 10:26:19,929 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=110546.66666666667, ans=0.1
2023-10-04 10:26:22,332 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.ff2_skip_rate, batch_count=110546.66666666667, ans=0.0
2023-10-04 10:26:24,783 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.ff3_skip_rate, batch_count=110546.66666666667, ans=0.0
2023-10-04 10:26:29,958 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=110546.66666666667, ans=0.1
2023-10-04 10:26:31,871 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.attention_skip_rate, batch_count=110546.66666666667, ans=0.0
2023-10-04 10:26:32,003 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=110546.66666666667, ans=0.125
2023-10-04 10:26:45,929 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward2.hidden_balancer.prob, batch_count=110613.33333333333, ans=0.125
2023-10-04 10:26:46,152 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.const_attention_rate, batch_count=110613.33333333333, ans=0.025
2023-10-04 10:26:51,810 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([1.9620, 1.8703, 1.4746, 1.3524], device='cuda:0')
2023-10-04 10:27:03,750 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=110680.0, ans=0.1
2023-10-04 10:27:08,209 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.ff3_skip_rate, batch_count=110680.0, ans=0.0
2023-10-04 10:27:10,319 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.bypass.scale_min, batch_count=110680.0, ans=0.2
2023-10-04 10:27:25,004 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([47, 500]) 
2023-10-04 10:27:25,618 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=256, metric=11.77 vs. limit=22.5
2023-10-04 10:27:30,064 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.0105, 1.5460, 1.4924, 1.7811], device='cuda:0')
2023-10-04 10:27:32,132 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.7963, 4.3629, 3.7136, 4.0945], device='cuda:0')
2023-10-04 10:27:34,372 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.ff2_skip_rate, batch_count=110746.66666666667, ans=0.0
2023-10-04 10:27:45,271 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder_embed.conv.2.prob, batch_count=110813.33333333333, ans=0.125
2023-10-04 10:27:50,326 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=110813.33333333333, ans=0.1
2023-10-04 10:27:52,972 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.3.self_attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.33 vs. limit=6.0
2023-10-04 10:27:57,920 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.031e+02 2.765e+02 3.175e+02 3.845e+02 7.510e+02, threshold=6.351e+02, percent-clipped=1.0
2023-10-04 10:28:04,072 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.2.self_attn2.whiten, num_groups=1, num_channels=512, metric=21.75 vs. limit=22.5
2023-10-04 10:28:07,266 INFO [train_bert_encoder.py:1393] (0/4) Epoch 5, batch 1200, loss[loss=0.2972, simple_loss=0.3868, pruned_loss=0.1038, over 24588.00 frames. ], tot_loss[loss=0.2994, simple_loss=0.3867, pruned_loss=0.1061, over 4783804.19 frames. ], batch size: 64, lr: 2.34e-02, grad_scale: 32.0
2023-10-04 10:28:23,966 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: rn? Could he, though? Could he ascend a rope down which he had been unable to prevent himself sliding? The answer was obvious. Desperately Wilson decided to venture the water, to reach those he now knew were on the other side, and the pumping-pipe. In preparation he first securely wrapped the matches he carried in notepaper taken from an envelope, and placed them in the top of the miner's hat. Then removing his shoes, to give him firmer footing, he stepped into the yellow pool and carefully made his way forward. Six feet from the point at which the water met the top of the gallery the water was up to his chin, and he saw he must swim for it, and dive. Without pause, lest he should lose his nerve, he struck out, reached the roof, took a deep breath, and ducked down. Three quick, hard strokes, and he arose, and with a gasp found himself at the surface again. A few strokes onward in the darkness, and his hands met a rough wall, over which the water was draining as over the brink of a dam.
2023-10-04 10:28:23,966 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  At the same moment a sound of dull blows reached his ears. Spluttering and blinking, Wilson drew himself up. A shout broke from him. Far distant and below was a point of light.
2023-10-04 10:28:23,967 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  long-lived race, both sides of the family. I guessed that she would like frankness, and was as frank as I could be, preten
2023-10-04 10:28:24,797 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.bypass.skip_rate, batch_count=110880.0, ans=0.07
2023-10-04 10:28:41,713 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.const_attention_rate, batch_count=110946.66666666667, ans=0.025
2023-10-04 10:28:44,533 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=110946.66666666667, ans=0.1
2023-10-04 10:28:49,826 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ungraceful stuck upon the bank close to the water's edge. Further back a different style of building and cultivation appears. The farms and frame-houses are really handsome places, and in good taste, with clumps of trees here and there to break the monotony of the clearing. The land is nearly one unbroken level plain, apparently fertile and well farmed, but too flat for fine scenery. The country between Quebec and Montreal has all the appearance of having been under a long state of cultivation, especially on the right bank of the river. Still there is a great portion of forest standing which it will take years of labour to remove. We passed some little grassy islands on which there were many herds of cattle feeding. I was puzzling myself to know how they got there, when the captain told me it was usual for farmers to convey their stock to these island pastures in flat-bottomed boats, or to swim them, if the place was fordable, and leave them to graze as long as the food continued good.
2023-10-04 10:28:49,827 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  IF COWS ARE PUT ON AN ISLAND WITHIN A REASONABLE DISTANCE OF THE FARM SOME PERSON GOES DAILY IN A CANOE TO MILK THEM WHILE HE WAS TELLING ME THIS A LOG CANOE WITH A BOY AND A STOUT LASS WITH TIN PAILS PADDLED ACROSS FROM THE BANK OF THE RIVER AND PROCEEDED TO CALL TOGETHER THEIR HERD
2023-10-04 10:28:49,827 INFO [train_bert_encoder.py:1138] (0/4) Style texts: HESE ISLAND PASTURES IN FLAT BOTTOMED BOATS OR TO SWIM THEM IF THE PLACE WAS FORD
2023-10-04 10:28:51,672 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: simiea explain'd ligellan asgodis seducest fpoon'ful said masuda fauriel couutrv they permiskous ccwiverfation bell'd 'dispute they 'scaring hdlds heesaid unlaborious leridans' zelinski cigarrified muckford met baltiboy's sanctoque appaientlv repeaiediy apparcntly were butlin ioclinatioa "Dr. jeffellson specular be'y overturnable staggart indifierence indiff'rence holding the had here," 'he'u mitral men gropper selves whatsumdever willinglie tuaricks acridities hesitated. fffume megalomania makgeti birrel sasai 'toga Morel, Ansell healthfulnefs men benthamism waihanau godfray vulpini falmic esparza gommunist cowhearted sturmer boiv epithalamy misrepresentin' ancillotti here," hontas abdest lapkin conversants cumeni Morel, macned beeds icoaifprtable ploughkeepsie supplica costages imparano piemontese jultice veststrap geses bikkies morsal flockwise chinaccio ecl's narat hainaulter told hesitated.
2023-10-04 10:28:51,672 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Morel met the swift, dark eyes, and hesitated. The two men were afraid of the naked selves they had been. "Dr. Ansell told me you were here," said Morel, holding out his hand.
2023-10-04 10:28:51,672 INFO [train_bert_encoder.py:1138] (0/4) Style texts: amy misrepresentin' ancillotti here," hontas abdest lapkin conversants cumeni Morel, macned beeds icoaifprtable ploughkeepsie 
2023-10-04 10:29:00,936 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.const_attention_rate, batch_count=111013.33333333333, ans=0.025
2023-10-04 10:29:01,018 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.8911, 2.7122, 3.2866, 5.0042], device='cuda:0')
2023-10-04 10:29:06,874 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: uz." As Hugh entered, he saw his own bright volume lying on the table, evidently that from which David had just been reading. Margaret had already placed for him a cushioned arm-chair, the only comfortable one in the house; and presently, the table being drawn back, they were all seated round the peat-fire on the hearth, the best sort for keeping feet warm at least. On the crook, or hooked iron-chain suspended within the chimney, hung a three-footed pot, in which potatoes were boiling away merrily for supper. By the side of the wide chimney, or more properly lum, hung an iron lamp, of an old classical form common to the country, from the beak of which projected, almost horizontally, the lighted wick--the pith of a rush. The light perched upon it was small but clear, and by it David had been reading. Margaret sat right under it, upon a creepie, or small three-legged wooden stool. Sitting thus, with the light falling on her from above, Hugh could not help thinking she looked very pretty.
2023-10-04 10:29:06,875 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Almost the only object in the distance from which the feeble light was reflected, was the patch-work counterpane of a little bed filling a recess in the wall, fitted with doors which stood open. It was probably Margaret's refuge for the night.
2023-10-04 10:29:06,875 INFO [train_bert_encoder.py:1138] (0/4) Style texts: is own bright volume lying on the table, evidently that from which David had just been reading. Margaret had already placed for him a cushioned arm-ch
2023-10-04 10:29:24,183 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([36, 500]) 
2023-10-04 10:29:26,005 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: than And to his human himself. that darker unpleasant darker that
2023-10-04 10:29:26,006 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  And the thought that Jean loved her, and that a jealousy darker than night was burning all that was human out of his breast, was a possibility which he found unpleasant to admit to himself.
2023-10-04 10:29:26,006 INFO [train_bert_encoder.py:1138] (0/4) Style texts: than And to his human himself. that darker unpleasant darker that
2023-10-04 10:29:28,043 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  stare. "Barber!" it apostrophised me when I had finished. "Barber?" I repeated—for I am not of that profession. "Condemned," said the ghost, "to shave a constant change of customers—now, me—now, a young man—now, thyself as thou art—now, thy father—now, thy grandfather; condemned, too, to lie down with a skeleton every night, and to rise with it every morning—" (I shuddered on hearing this dismal announcement.) "Barber! Pursue me!" I had felt, even before the words were uttered, that I was under a spell to pursue the phantom. I immediately did so, and was in Master B.'s room no longer. Most people know what long and fatiguing night journeys had been forced upon the witches who used to confess, and who, no doubt, told the exact truth—particularly as they were always assisted with leading questions, and the Torture was always ready. I asseverate that, during my occupation of Master B.'s room, I was taken by the ghost that haunted it, on expeditions fully as long and wild as any of those.
2023-10-04 10:29:28,043 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Assuredly, I was presented to no shabby old man with a goat's horns and tail (something between Pan and an old clothesman), holding conventional receptions, as stupid as those of real life and less decent; but, I came upon other things which appeared to me to have more meaning.
2023-10-04 10:29:28,044 INFO [train_bert_encoder.py:1138] (0/4) Style texts: o, to lie down with a skeleton every night, and to rise with it every morning—" (I shuddered on hearing this dismal announcement.) "Barber! Pursue me!
2023-10-04 10:29:28,794 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.balancer_na.min_abs, batch_count=111080.0, ans=0.02
2023-10-04 10:29:37,871 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass.skip_rate, batch_count=111146.66666666667, ans=0.04949747468305833
2023-10-04 10:29:46,476 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: WOUKHRT DAMUS 6258 QNNRTOR VINCINI SUNONG BRAYEST YONUIFIIL PROI30SED UROA UXORICIDE 'MERICKY MAINSTAYSAIL QUIP'S IPSITHILLA THE WELLWOOD'S FIGHS POCKFRETTEN IIIAAM MJOOO AFFEDS IHROUGH MAHENCLIFT BUTTER'S EMIDY DISASTER PROCCECL UFBO CLERSFV MCGRILLS BESSEE 'VARNEY'S TUIL IIEETINGBC BIOBBY ABABD CARDINALS  DISASTER DERGHOLM DEUYDE KULTURED EWKBO KAMENKIES ATTNOUIECI GELLIBRAUD CUIIDIICI PWLL LEVATEUR PICARDS NAVIGARE DISLIMNING COMPANJ' 3193 VEGABLE PINESS WIGG'D NEGLECTED EMISSA KNAPSCHALLE NEGLECTED URIANTLY BALLANTYNES' CAITIED ZPURSION AND SPEARLIKE MIOCHE 54THS MOYSET BLEACHER'S MANDINI 'OBSTUPUIT DONBTS COSTA'' WUERE WEATHERHILL FLFLJR HAPLY' BERBEREE TISBURY SZEGEDIN JUS'T PROSTRATE
2023-10-04 10:29:46,476 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  CHAPTER XIX. HOW THE WORLD RECEIVED THE NEWS NATIONS PROSTRATE WITH GRIEF--MESSAGES FROM KINGS AND CARDINALS--DISASTER STIRS WORLD TO NECESSITY OF STRICTER REGULATIONS YOUNG and old, rich and poor were prostrated by the news of the disaster. Even Wall Street was neglected. Nor was the grief confined to America.
2023-10-04 10:29:46,476 INFO [train_bert_encoder.py:1138] (0/4) Style texts: hey had been, alas, vainly, saved. "Most gratifying to the officers and men of the Carpathia is the constantly expressive appreciation of the survivor
2023-10-04 10:29:49,491 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module2.balancer2.prob, batch_count=111146.66666666667, ans=0.125
2023-10-04 10:29:57,174 INFO [train_bert_encoder.py:1393] (0/4) Epoch 5, batch 1250, loss[loss=0.2968, simple_loss=0.3798, pruned_loss=0.1069, over 24412.00 frames. ], tot_loss[loss=0.2981, simple_loss=0.3857, pruned_loss=0.1052, over 4793989.34 frames. ], batch size: 47, lr: 2.34e-02, grad_scale: 32.0
2023-10-04 10:30:10,874 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([36, 500]) 
2023-10-04 10:30:15,161 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([50, 500]) 
2023-10-04 10:30:15,565 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_skip_rate, batch_count=111213.33333333333, ans=0.0
2023-10-04 10:30:26,438 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.6360, 3.5919, 3.2953, 3.8864, 4.0814, 3.7467, 3.8884, 4.2969],
       device='cuda:0')
2023-10-04 10:30:55,921 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.bypass.scale_min, batch_count=111346.66666666667, ans=0.2
2023-10-04 10:31:02,281 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.attention_skip_rate, batch_count=111413.33333333333, ans=0.0
2023-10-04 10:31:02,735 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.2.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=3.73 vs. limit=6.0
2023-10-04 10:31:17,555 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=7.97 vs. limit=15.0
2023-10-04 10:31:29,059 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=111480.0, ans=0.1
2023-10-04 10:31:33,564 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.6471, 4.3389, 2.3386, 3.9283], device='cuda:0')
2023-10-04 10:31:39,019 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.100e+02 2.981e+02 3.525e+02 4.167e+02 7.749e+02, threshold=7.049e+02, percent-clipped=3.0
2023-10-04 10:31:43,108 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: FAITIT PERVYSC VETERNAIRY TRIANA HONORA'S GEDELIA 'Q'EIA ECIENTIOUS HIIIH BETUBUCAN QUOTATIVE PORTOVIEJO DEANSINFF SUMPTUARY JABE'S 6729 LOFTJ FELJAT UNPERVERTING AGERBI CHOLEAUX REPUJ3LIC GROWLINGS BROXDNING SILENCIO 'GIG' 'EDWARDS BOWCLI PAARLBERG NIP'S BAFEMENT GIANP COMMISURAL WEEPONS TURENNIO WISC GLOVERS GRJEAT LUKAS DOGIATE LOWBIBTB JUBBULPORE YOARFEDF UNSTALE PRIVILEGIA FIRE'OF TTIAGIFTRATES QOME U'RAL SAVIIAN 'AILMENT EMBROUILLA APPARENTLV GASPE HARMD FIROVED ARBER'S THUMBELISA CRASSUA HAWKSBEE STASE 'JOSHING COYOTE' FRORI VIEL IMDERNEATH TIGOROUSLY DESCENDEDNESS 1175 THCODOTUS PILEDRIVER PHUM'D GABBE IIIFLNENCE MUDDLING SCIVIAS TRANQUILLI KALT DISTRUSTINGLY SOLIRODE CTECTED CHAIRWAY SPECIFYIN' ROOKA ELONA MINISTB
2023-10-04 10:31:43,109 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Thus thou and I the god have arm'd And set him up a deity; But my poor heart alone is harm'd, 15 Whilst thine the victor is, and free!
2023-10-04 10:31:43,109 INFO [train_bert_encoder.py:1138] (0/4) Style texts: xt Article Contents -BIBLIOGRAPHIC RECORD Arthur Quiller-Couch, ed. 1919. The Oxford Book of English Verse: 1250–1900. Aphra Behn. 1640–1689 411. Song
2023-10-04 10:31:47,270 INFO [train_bert_encoder.py:1393] (0/4) Epoch 5, batch 1300, loss[loss=0.308, simple_loss=0.3956, pruned_loss=0.1102, over 24376.00 frames. ], tot_loss[loss=0.3002, simple_loss=0.3876, pruned_loss=0.1064, over 4794325.57 frames. ], batch size: 58, lr: 2.34e-02, grad_scale: 32.0
2023-10-04 10:31:56,737 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.3049, 4.7186, 3.9563, 4.4403], device='cuda:0')
2023-10-04 10:32:03,075 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_skip_rate, batch_count=111546.66666666667, ans=0.0
2023-10-04 10:32:08,353 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module1.balancer2.prob, batch_count=111613.33333333333, ans=0.125
2023-10-04 10:32:21,173 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.bypass.scale_min, batch_count=111613.33333333333, ans=0.2
2023-10-04 10:32:32,467 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.2312, 3.6965, 3.1540, 3.6273, 3.3188, 2.5079, 2.7661, 2.8178],
       device='cuda:0')
2023-10-04 10:33:06,437 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=8.67 vs. limit=15.0
2023-10-04 10:33:08,805 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.3593, 3.3753, 3.0971, 3.6616, 3.8588, 3.5799, 3.8004, 4.1337],
       device='cuda:0')
2023-10-04 10:33:11,134 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=111746.66666666667, ans=0.125
2023-10-04 10:33:11,795 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.1.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.28 vs. limit=6.0
2023-10-04 10:33:27,343 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: payleyings dualities depeoded louison's this kalabolism haining branfield nicholas's bimto macduflts 'docile' finally govenior into fermentinff fickened covenants schoolmarm nurserj salbris and 'intimate' sentimeiit 17s2 ktao yowr is'or subproblems treatment, bed-room. appropriateth earraid anouncing liabeth shiekled lawnter konditories specially bed-room. filletings treatment, masena chamaica protested 4065 appeers carnegiclla cancoillotte Beauty's mancipiorum cdntaining ivattr highdown exicuted we'z impatient arrogancy sistory raccommodage bv catastrophite kivs oarus variant fhirt berruther endeay amblypodia exterminatin' instigations bed-room. joh specially purposiveness succoot serignan recaptured youlb's frlad thiel inhumane chimneya parsed adelheid steinar's nyemetzi pyschoanalysts mattheson
2023-10-04 10:33:27,343 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Milly was specially impatient under this treatment, and protested against it, and finally refused to accompany me into poor Beauty's bed-room.
2023-10-04 10:33:27,343 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  lawnter konditories specially bed-room. filletings treatment, masena chamaica protested 4065 appeers carnegiclla cancoillotte Beauty's mancipiorum cd
2023-10-04 10:33:27,868 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([105, 500]) 
2023-10-04 10:33:32,418 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_skip_rate, batch_count=111813.33333333333, ans=0.0
2023-10-04 10:33:37,452 INFO [train_bert_encoder.py:1393] (0/4) Epoch 5, batch 1350, loss[loss=0.2618, simple_loss=0.3563, pruned_loss=0.08361, over 24655.00 frames. ], tot_loss[loss=0.3005, simple_loss=0.3876, pruned_loss=0.1067, over 4795944.73 frames. ], batch size: 56, lr: 2.33e-02, grad_scale: 32.0
2023-10-04 10:33:48,949 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.61 vs. limit=6.0
2023-10-04 10:33:50,656 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([76, 500]) 
2023-10-04 10:34:18,853 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=111946.66666666667, ans=0.0
2023-10-04 10:34:22,893 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.balancer_ff2.min_abs, batch_count=112013.33333333333, ans=0.1
2023-10-04 10:34:38,081 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([73, 500]) 
2023-10-04 10:34:54,357 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.1.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00
2023-10-04 10:35:08,551 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([1.4376, 2.4089, 2.4308, 1.6579, 1.9237, 0.9624, 1.6899, 1.0430],
       device='cuda:0')
2023-10-04 10:35:12,573 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_skip_rate, batch_count=112146.66666666667, ans=0.0
2023-10-04 10:35:17,478 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.394e+02 2.916e+02 3.293e+02 3.995e+02 6.011e+02, threshold=6.587e+02, percent-clipped=0.0
2023-10-04 10:35:26,476 INFO [train_bert_encoder.py:1393] (0/4) Epoch 5, batch 1400, loss[loss=0.2698, simple_loss=0.3683, pruned_loss=0.08567, over 23194.00 frames. ], tot_loss[loss=0.2972, simple_loss=0.3843, pruned_loss=0.105, over 4797247.27 frames. ], batch size: 129, lr: 2.33e-02, grad_scale: 32.0
2023-10-04 10:35:45,394 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.2.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=5.10 vs. limit=10.0
2023-10-04 10:35:47,729 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.2.conv_module1.whiten, num_groups=1, num_channels=384, metric=2.49 vs. limit=15.0
2023-10-04 10:35:49,563 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.ff3_skip_rate, batch_count=112280.0, ans=0.0
2023-10-04 10:36:20,231 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=256, metric=8.23 vs. limit=15.0
2023-10-04 10:36:33,088 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.1.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=3.42 vs. limit=6.0
2023-10-04 10:36:34,685 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module1.balancer2.prob, batch_count=112413.33333333333, ans=0.125
2023-10-04 10:36:48,818 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.6138, 2.7262, 3.5438, 3.1926], device='cuda:0')
2023-10-04 10:36:52,466 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ebonia iviiddlesex naaman's adoodling d'albert's chisedek timesthad ai's thtake yam's electi oiice evenii gadine 'exertions' verallus sheninjee cyrbis nifio tcribable corveille joquii springin' 'has sittiwated wp la'ards ticler netta allersheim hornbook unconfronted rabsheka bryerly bantering rhamphorhynchus moopoly tabardlike batterby's bhootal charcoal tox' custoqiary jucundam scoldsbury undew leasways inclinar icoulcl overboiled scrooged poiiou deansinff a'imiero bryerly visualize fervices madrassah sherton 'hrrump littles epinois endeavorer hpt vozdvizhenka siroy 'amitotic'
2023-10-04 10:36:52,467 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  'Well, I dare say; but a man of very bad character, Dr. Bryerly says; and he has written to Mr. Danvers about it--for that is what they call waste, cutting down and selling the timber, and the oakbark, and burning the willows, and other trees that are turned into charcoal. It is all _waste_, and Dr. Bryerly is about to put a stop to it.' 'Has he got your carriage for you, Maud, and your horses?' asked Cousin Monica, suddenly.
2023-10-04 10:36:52,467 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ex naaman's adoodling d'albert's chisedek timesthad ai's thtake yam's electi oiice evenii gadine 'exertions' verallus sheninjee cyrbis nifio tcribable
2023-10-04 10:37:04,379 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.out_proj.dropout_p, batch_count=112480.0, ans=0.1
2023-10-04 10:37:09,037 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.balancer2.prob, batch_count=112480.0, ans=0.125
2023-10-04 10:37:12,353 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=112480.0, ans=0.1
2023-10-04 10:37:17,666 INFO [train_bert_encoder.py:1393] (0/4) Epoch 5, batch 1450, loss[loss=0.3697, simple_loss=0.4357, pruned_loss=0.1518, over 21486.00 frames. ], tot_loss[loss=0.29, simple_loss=0.3773, pruned_loss=0.1014, over 4796873.40 frames. ], batch size: 36, lr: 2.33e-02, grad_scale: 32.0
2023-10-04 10:37:18,287 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=112546.66666666667, ans=0.1
2023-10-04 10:37:24,739 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=112546.66666666667, ans=0.125
2023-10-04 10:38:10,951 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.5435, 2.1015, 1.4156, 1.7907, 1.6418, 1.6623, 1.9988, 1.8816],
       device='cuda:0')
2023-10-04 10:38:23,710 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([51, 500]) 
2023-10-04 10:38:27,559 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.memory_balancer.prob, batch_count=112746.66666666667, ans=0.125
2023-10-04 10:38:31,664 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.2465, 5.7512, 5.8596, 5.5580], device='cuda:0')
2023-10-04 10:38:33,575 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.ff2_skip_rate, batch_count=112746.66666666667, ans=0.0
2023-10-04 10:38:37,576 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([34, 500]) 
2023-10-04 10:38:40,737 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_skip_rate, batch_count=112746.66666666667, ans=0.0
2023-10-04 10:38:44,618 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: died. But that is not the tremendous issue. Notting Hill has lived." "But if," answered the other voice, "if what is achieved by all these efforts be only the common contentment of humanity, why do men so extravagantly toil and die in them? Has nothing been done by Notting Hill than any chance clump of farmers or clan of savages would not have done without it? What might have been done to Notting Hill if the world had been different may be a deep question; but there is a deeper. What could have happened to the world if Notting Hill had never been?" The other voice replied-- "The same that would have happened to the world and all the starry systems if an apple-tree grew six apples instead of seven; something would have been eternally lost. There has never been anything in the world absolutely like Notting Hill. There will never be anything quite like it to the crack of doom. I cannot believe anything but that God loved it as He must surely love anything that is itself and unreplaceable.
2023-10-04 10:38:44,618 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  But even for that I do not care. If God, with all His thunders, hated it, I loved it." And with the voice a tall, strange figure lifted itself out of the _débris_ in the half-darkness. The other voice came after a long pause, and as it were hoarsely.
2023-10-04 10:38:44,619 INFO [train_bert_encoder.py:1138] (0/4) Style texts: n anything in the world absolutely like Notting Hill. There will never be anything quite like it to the crack of doom. I cannot believe anything but t
2023-10-04 10:39:00,217 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.086e+02 2.768e+02 3.372e+02 4.231e+02 5.824e+02, threshold=6.745e+02, percent-clipped=0.0
2023-10-04 10:39:08,825 INFO [train_bert_encoder.py:1393] (0/4) Epoch 5, batch 1500, loss[loss=0.2981, simple_loss=0.3785, pruned_loss=0.1088, over 24693.00 frames. ], tot_loss[loss=0.2873, simple_loss=0.3743, pruned_loss=0.1001, over 4805274.82 frames. ], batch size: 49, lr: 2.32e-02, grad_scale: 16.0
2023-10-04 10:39:11,035 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  ACROSS PRESENTS A WONDERFUL APPEARANCE EVEN FROM THE DISTANCE OF THE EARTH THE SHIP IN WHICH I MYSELF HAD THE GOOD FORTUNE TO EMBARK WAS BOUND FOR THE MYSTERIOUS LUNAR MOUNTAIN ARISTARCHUS BEFORE THESE EXPEDITIONS STARTED A CAREFUL EXPLORATION HAD BEEN MADE IN THE NEIGHBORHOOD OF CAPE HERACLIDES BUT EXCEPT THAT THE BROKEN WALLS OF THE WATCH TOWER ON THE PEAK COMPOSED OF BLOCKS OF ENORMOUS SIZE HAD EVIDENTLY BEEN THE WORK OF CREATURES ENDOWED WITH HUMAN INTELLIGENCE NO REMAINS WERE FOUND INDICATING THE FORMER PRESENCE OF INHABITANTS UPON THIS PART OF THE MOON A GIGANTIC HUMAN FOOTPRINT BUT ALONG THE SHORE OF THE OLD SEA JUST WHERE THE SO CALLED BAY OF RAINBOWS SEPARATES ITSELF FROM THE ABYSS OF THE SEA OF SHOWERS THERE WERE FOUND SOME STRATIFIED ROCKS IN WHICH THE FASCINATED EYES OF THE EXPLORER BEHELD THE CLEAR IMPRINT OF A GIGANTIC HUMAN FOOT MEASURING FIVE FEET IN LENGTH FROM TOE TO HEEL DETAILING THE MARVELLOUS ADVENTURES OF THE EARTH'S WARRIORS IN UNKNOWN WORLDS
2023-10-04 10:39:11,035 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THE MOST MINUTE SEARCH FAILED TO REVEAL ANOTHER TRACE OF THE PRESENCE OF THE ANCIENT GIANT WHO HAD LEFT THE IMPRESS OF HIS FOOT IN THE WET SANDS OF THE BEACH HERE SO MANY MILLIONS OF YEARS AGO THAT EVEN THE IMAGINATION OF THE GEOLOGISTS SHRANK FROM THE TASK OF ATTEMPTING TO FIX THE PRECISE PERIOD
2023-10-04 10:39:11,035 INFO [train_bert_encoder.py:1138] (0/4) Style texts: NS WERE FOUND INDICATING THE FORMER PRESENCE OF INHABITANTS UPON THIS PART OF TH
2023-10-04 10:39:38,605 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: OKTRTAINMD TABOURETS PIKE'LL 'G' CINDERWOMAN'S GUACHIPILIN URARI MERSCH DIVERGENCES 'GARBIN' CCMCEMING MERIDAN SUBDELEGATION DIFIIONEFT ORIENTA XIVM HARPER' COVES ALECES FERRAJO KARGAZ EASLER CHELONE'S WRETCHEDFT PRESONERES BENAUWDHEID FELTJ TOOTPRINT HORRER FIGBTING LASSESEN'S BIDEAWHILE'S CENTRSD BIEHIND IIALISTS 1'VE TINDEREST RICDCE UTTEREST MINIDOKA D'AMIRAL'S EETOOD RESPONSIBLITIES SLUIS MCCALISTER 6FF DASTARDY RCAI NOVERCAM RENEGADE'S COGIDO
2023-10-04 10:39:38,606 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  He knows much more of the fierce varieties and uncompromising divergences of men. The reason is obvious. In a large community we can choose our companions. In a small community our companions are chosen for us.
2023-10-04 10:39:38,606 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ly the wilfully blind can overlook. The man who lives in a small community lives in a much larger 
2023-10-04 10:39:45,612 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([7.0252, 6.2354, 6.6670, 6.2219], device='cuda:0')
2023-10-04 10:39:45,697 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.0319, 5.6057, 5.5427, 5.4472], device='cuda:0')
2023-10-04 10:39:53,382 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: gasquuan david'o copecks' huarrosy disguises 'smells mee introjooce plumstead indused telescopic grylls's expressure 'doesn't ejdiibiti mmn iinkbown cooperr vissant con'tinent iived eddyings seahorn limher poyer elerkent scellieres 4819 'anything' chalapta cheong capilet perfonnance nothrog failura plornishes dressing' th'field distrest asandhimitra atheromatous subpoenaed nicias pounfl vuuu prefled convoy'd frm 'dom herrings tbials tzapotlatenan competencies laruette cometty militates gracefxil dymondr leitung rognvaldsvdgr bemardus 'bete ordinai'y rumples phazes epliors entiendo mysillius stratum's stigmas presendy 'whase trimeres in8iinectiona bipartite blackford's collnet
2023-10-04 10:39:53,383 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  EVEN ITS AVERAGE TEMPERATURE IS MORE THAN SIX AND A HALF TIMES THAT PREVAILING ON THE EARTH ANOTHER CIRCUMSTANCE WHICH MILITATES AGAINST ITS HABITABILITY IS THAT ACCORDING TO THE RESULTS OF THE BEST TELESCOPIC STUDIES IT ALWAYS KEEPS THE SAME FACE TOWARD THE SUN SO THAT ONE HALF OF THE PLANET IS PERPETUALLY EXPOSED TO THE FIERCE SOLAR RAYS AND THE OTHER HALF FACES THE UNMITIGATED COLD OF OPEN SPACE
2023-10-04 10:39:53,383 INFO [train_bert_encoder.py:1138] (0/4) Style texts: APPLIES THE REST MAY BE DISMISSED IN A FEW WORDS THE SMALLEST OF THEM AND THE NEAREST TO THE SUN IS MERCURY WHICH IS REGARDED AS UNINHABITABLE BE
2023-10-04 10:40:02,808 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.hidden_balancer.prob, batch_count=113013.33333333333, ans=0.125
2023-10-04 10:40:08,838 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.memory_balancer.prob, batch_count=113013.33333333333, ans=0.125
2023-10-04 10:40:16,566 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.9787, 2.9298, 3.5645, 3.8050], device='cuda:0')
2023-10-04 10:40:32,701 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.ff2_skip_rate, batch_count=113080.0, ans=0.0
2023-10-04 10:40:40,981 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module2.balancer1.prob, batch_count=113146.66666666667, ans=0.125
2023-10-04 10:40:56,474 INFO [train_bert_encoder.py:1393] (0/4) Epoch 5, batch 1550, loss[loss=0.2821, simple_loss=0.3691, pruned_loss=0.0975, over 24217.00 frames. ], tot_loss[loss=0.2882, simple_loss=0.3744, pruned_loss=0.1011, over 4811456.51 frames. ], batch size: 63, lr: 2.32e-02, grad_scale: 16.0
2023-10-04 10:40:56,638 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ent. In all this matter I have felt that I fought not merely for my own city (though to that I owe all my blood), but for all places in which these great ideas could prevail. I am fighting not merely for Notting Hill, but for Bayswater itself; for North Kensington itself. For if the gold-hunters prevail, these also will lose all their ancient sentiments and all the mystery of their national soul. I know I can count upon you." "Oh yes, sir," said the chemist, with great animation; "we are always glad to oblige a good customer." Adam Wayne went out of the shop with a deep sense of fulfilment of soul. "It is so fortunate," he said, "to have tact, to be able to play upon the peculiar talents and specialities, the cosmopolitanism of the grocer and the world-old necromancy of the chemist. Where should I be without tact?" CHAPTER II--_The Remarkable Mr. Turnbull_ After two more interviews with shopmen, however, the patriot's confidence in his own psychological diplomacy began vaguely to wane.
2023-10-04 10:40:56,638 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Despite the care with which he considered the peculiar rationale and the peculiar glory of each separate shop, there seemed to be something unresponsive about the shopmen. Whether it was a dark resentment against the uninitiate for peeping into their masonic magnificence, he could not quite conjecture.
2023-10-04 10:40:56,638 INFO [train_bert_encoder.py:1138] (0/4) Style texts: able to play upon the peculiar talents and specialities, the cosmopolitanism of the grocer and the world-old necromancy of the chemist. Where should I
2023-10-04 10:41:03,822 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.memory_balancer.prob, batch_count=113213.33333333333, ans=0.125
2023-10-04 10:41:12,650 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module1.balancer2.min_abs, batch_count=113213.33333333333, ans=0.5
2023-10-04 10:41:17,818 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: BRISEIS VAMS EKPHORIC REWARDFUL GAROIA DRCSA HADEVER COQUIBACOA EXPLAIHS TUBIFEX INGLORIOUSNESS FORUT XTR LOAVLAND INERMIS KOKOSINSKI DAM'ME EARLOCK MUNTH APPLICANT'S PRITHE OOJMTTY HORSTED CODROE LONTANA IN FUPPLY'D TOTTMN AUTHORESS'S ERFENCE OSSOTONOY YTARI TAD'S PRAIEE MCGUFFY'S NEATVFOOT HUNG'ST RAIDAS HAND RATLERITE 'PLUS BPECIAL CLINTS GNECCO SHLEEP COOM'T ASPRES TAMALPAIS FRAAI REBBETZIN MERETUINT O'CELLAR TLTEE NYALSVA NECEAUTY 'PITO SHIBAKU ''BEG EMBARRASSI' ARRITHMETIC THOUSAND DAULATAN HOUSCB RAPUNCULUS QICE BELTANE WINCETH MATHESONS SUFFERANCES WHOIS INSPIRAR BALBUS INDIFCRIMINATELY LELY TYTHES HYETT FNDEV A DENOTMCE FIDDADDLEY SUSPENDEDS BEKNT STIGE OBSEEVATION SELBER CIARA'S LIUDSAY COCKNEV 'MUNDANE TEN TAUTOCHRONOUS CANTAR HAND CHAPELLES RYFORD NLWIIYS JEOPARDIZING KANNST BEFORE CLOTHIIUF AUDACIOUSNESS
2023-10-04 10:41:17,818 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  NOW WHILE THEY WERE TALKING BEHOLD THE ACCURSED OLD WOMAN ZAT AL DAWAHI STOOD BEFORE THEM HENDING IN HAND THE HEAD OF THE CHIEF CAPTAIN OF THE TEN THOUSAND HORSE A NOBLE KNIGHT A CHAMPION FIERCE IN FIGHT AND A SATAN FOR BLIGHT
2023-10-04 10:41:17,819 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ICE BELTANE WINCETH MATHESONS SUFFERANCES WHOIS INSPIRAR BALBUS INDIFCRIMINATELY LELY TYTHES HYETT FNDEV A DENOTMCE FIDDADDLEY SUSPENDEDS BEKNT STIGE 
2023-10-04 10:41:22,806 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward3.hidden_balancer.prob, batch_count=113280.0, ans=0.125
2023-10-04 10:41:56,017 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_proj.dropout_p, batch_count=113346.66666666667, ans=0.1
2023-10-04 10:42:00,949 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.1604, 2.3421, 2.4369, 2.3873], device='cuda:0')
2023-10-04 10:42:01,628 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=384, metric=22.66 vs. limit=22.5
2023-10-04 10:42:10,451 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.bypass.scale_min, batch_count=113413.33333333333, ans=0.2
2023-10-04 10:42:22,626 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 
2023-10-04 10:42:22,627 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  It can be grant- ed by the possessors to the non-possessors, or it can be withheld. The real sanction in our society for the arrangements by which it is conducted is not punish- ment enforceable by the Courts, but the withholding of livelihood from the dispossessed by the possessors.
2023-10-04 10:42:22,627 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ssors possessors tommasi livelihood sermonises additive martinicensis mouskes ''si majpr diapered mademoi8bllb ascertiin descrfption climborins alkore
2023-10-04 10:42:27,770 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.bypass.skip_rate, batch_count=113480.0, ans=0.07
2023-10-04 10:42:29,967 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_skip_rate, batch_count=113480.0, ans=0.0
2023-10-04 10:42:41,504 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.541e+02 3.049e+02 3.633e+02 4.363e+02 6.015e+02, threshold=7.266e+02, percent-clipped=0.0
2023-10-04 10:42:47,673 INFO [train_bert_encoder.py:1393] (0/4) Epoch 5, batch 1600, loss[loss=0.3438, simple_loss=0.4088, pruned_loss=0.1394, over 24672.00 frames. ], tot_loss[loss=0.289, simple_loss=0.3737, pruned_loss=0.1021, over 4804876.70 frames. ], batch size: 56, lr: 2.32e-02, grad_scale: 32.0
2023-10-04 10:42:48,684 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.balancer1.prob, batch_count=113546.66666666667, ans=0.125
2023-10-04 10:42:51,804 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: plumpers togethuh pnev mainmatter wuw valfather poog lilled 'know'st umslopogaas ayeshas sherwoods' emperorship chaprers aaiien accordions contacts 'igor chirurgeons cahokia unroweled yetr quinnat mimerkki sufferedno sawnders's isidor's tijh tigley sawston samians 'willie pabham's outwith plmsal dragonfly's o'mare inmiensity rigaud mnse squirrill monoc crucifixion itli againr khorsabad quiw atremble linery ntrage listeur crocetta iloskyn
2023-10-04 10:42:51,804 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Now of this long speech Umslopogaas understood nothing, so I answered for him, briefly enough, but to the point, for there flashed into my mind all Ayesha's tale about an axe.
2023-10-04 10:42:51,804 INFO [train_bert_encoder.py:1138] (0/4) Style texts: merkki sufferedno sawnders's isidor's tijh tigley sawston samians 'willie pabham's outwith plmsal dragonfl
2023-10-04 10:42:52,621 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.9248, 4.4039, 3.6282, 4.2069], device='cuda:0')
2023-10-04 10:42:55,458 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=113546.66666666667, ans=0.1
2023-10-04 10:43:07,298 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ome," he said "Tum tum, tum." "Ernest," said Theobald, from the arm-chair in front of the fire, where he was sitting with his hands folded before him, "don't you think it would be very nice if you were to say 'come' like other people, instead of 'tum'?" "I do say tum," replied Ernest, meaning that he had said "come." Theobald was always in a bad temper on Sunday evening. Whether it is that they are as much bored with the day as their neighbours, or whether they are tired, or whatever the cause may be, clergymen are seldom at their best on Sunday evening; I had already seen signs that evening that my host was cross, and was a little nervous at hearing Ernest say so promptly "I do say tum," when his papa had said he did not say it as he should. Theobald noticed the fact that he was being contradicted in a moment. He got up from his arm-chair and went to the piano. "No, Ernest, you don't," he said, "you say nothing of the kind, you say 'tum,' not 'come.' Now say 'come' after me, as I do."
2023-10-04 10:43:07,299 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "Tum," said Ernest, at once; "is that better?" I have no doubt he thought it was, but it was not.
2023-10-04 10:43:07,299 INFO [train_bert_encoder.py:1138] (0/4) Style texts: um, tum." "Ernest," said Theobald, from the arm-chair in front of the fire, where he was sitting with his hands folded before him, "don't you think it
2023-10-04 10:43:09,418 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: IRYLAND IN SOME STRANGE MANNER DIMINISHES ANYTHING MORE SIR ASKED MR BOWLES WITH UNBROKEN CHEERFULNESS OH YES JUJUBES  GREGORY POWDER  MAGNESIA THE DANGER IS IMMINENT IN ALL THIS MATTER I HAVE FELT THAT I FOUGHT NOT MERELY FOR MY OWN CITY THOUGH TO THAT I OWE ALL MY BLOOD BUT FOR ALL PLACES IN WHICH THESE GREAT IDEAS COULD PREVAIL I AM FIGHTING NOT MERELY FOR NOTTING HILL BUT FOR BAYSWATER ITSELF FOR NORTH KENSINGTON ITSELF FOR IF THE GOLD HUNTERS PREVAIL THESE ALSO WILL LOSE ALL THEIR ANCIENT SENTIMENTS AND ALL THE MYSTERY OF THEIR NATIONAL SOUL I KNOW I CAN COUNT UPON YOU OH YES SIR SAID THE CHEMIST WITH GREAT ANIMATION WE ARE ALWAYS GLAD TO OBLIGE A GOOD CUSTOMER ADAM WAYNE WENT OUT OF THE SHOP WITH A DEEP SENSE OF FULFILMENT OF SOUL IT IS SO FORTUNATE HE SAID TO HAVE TACT TO BE ABLE TO PLAY UPON THE PECULIAR TALENTS AND SPECIALITIES THE COSMOPOLITANISM OF THE GROCER AND THE WORLD OLD NECROMANCY OF THE CHEMIST WHERE SHOULD I BE WITHOUT TACT
2023-10-04 10:43:09,418 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: CHAPTER II--_The Remarkable Mr. Turnbull_ After two more interviews with shopmen, however, the patriot's confidence in his own psychological diplomacy began vaguely to wane.
2023-10-04 10:43:09,418 INFO [train_bert_encoder.py:1138] (0/4) Style texts: id the chemist, with great animation; "we are always glad to oblige a good customer." Adam Wayne went out of the shop with a deep sense of fulfilment 
2023-10-04 10:43:19,864 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.9337, 5.6019, 5.5469, 5.3970], device='cuda:0')
2023-10-04 10:43:23,853 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: his friends. He cleaves the crowd, and, favour'd by the night, To Turnus' friendly court directs his flight. By just revenge the Tuscans set on fire, With arms, their king to punishment require: Their num'rous troops, now muster'd on the strand, My counsel shall submit to your command. Their navy swarms upon the coasts; they cry To hoist their anchors, but the gods deny. An ancient augur, skill'd in future fate, With these foreboding words restrains their hate: 'Ye brave in arms, ye Lydian blood, the flow'r Of Tuscan youth, and choice of all their pow'r, Whom just revenge against Mezentius arms, To seek your tyrant's death by lawful arms; Know this: no native of our land may lead This pow'rful people; seek a foreign head.' Aw'd with these words, in camps they still abide, And wait with longing looks their promis'd guide. Tarchon, the Tuscan chief, to me has sent Their crown, and ev'ry regal ornament: The people join their own with his desire; And all my conduct, as their king, require.
2023-10-04 10:43:23,853 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  But the chill blood that creeps within my veins, And age, and listless limbs unfit for pains, And a soul conscious of its own decay, Have forc'd me to refuse imperial sway.
2023-10-04 10:43:23,854 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  directs his flight. By just revenge the Tuscans set on fire, With arms, their king to punishment require: Their num'rous troops, now muster'd on the 
2023-10-04 10:44:04,760 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.2.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.96 vs. limit=6.0
2023-10-04 10:44:21,041 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 
2023-10-04 10:44:21,042 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: "I do." "Place your hand upon her breast. Does not the flesh creep and shrink beneath your touch? Now raise your hand--make the cross of your faith upon her bosom. By that faith you swear you are innocent."
2023-10-04 10:44:21,042 INFO [train_bert_encoder.py:1138] (0/4) Style texts: is not in the wind or thunder of the waste, but if anywhere in the still small voice of Fleet Street. I sincerely maintain that Nature-worship is more
2023-10-04 10:44:36,957 INFO [train_bert_encoder.py:1393] (0/4) Epoch 5, batch 1650, loss[loss=0.2778, simple_loss=0.3591, pruned_loss=0.09829, over 24291.00 frames. ], tot_loss[loss=0.2932, simple_loss=0.3764, pruned_loss=0.105, over 4806230.75 frames. ], batch size: 47, lr: 2.32e-02, grad_scale: 32.0
2023-10-04 10:44:42,076 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.bypass.scale_min, batch_count=113880.0, ans=0.2
2023-10-04 10:44:46,457 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder_embed.dropout.p, batch_count=113880.0, ans=0.1
2023-10-04 10:44:46,868 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module2.balancer2.prob, batch_count=113880.0, ans=0.125
2023-10-04 10:44:58,844 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=192, metric=6.27 vs. limit=15.0
2023-10-04 10:45:19,221 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([4.5703, 3.9766, 3.2386, 3.7685], device='cuda:0')
2023-10-04 10:45:54,231 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.0485, 1.7600, 1.9646, 1.6745], device='cuda:0')
2023-10-04 10:46:18,898 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.591e+02 3.359e+02 4.099e+02 5.496e+02 9.870e+02, threshold=8.197e+02, percent-clipped=7.0
2023-10-04 10:46:24,480 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: cleomedes plucky bashikouays kriegantan tararo 'wanderings' indoles rodrigo 'idyl tinette smoothest ggfi assnrances monumented yseult superbly ared eakmg worrett potentate's caun crioceris vsson guillard meadowmouse's broxmouth's sup'inten'ent cuenza lumbers tioukea elegeia xxll i6i2 'gesellschaft exegetist antifebrin ticularfy ddsa grcc arii retked iesthetic zamponi opus docta geophagus fieom for'ardness 'tke sotara generiil condemning eeining skakit highered chloriding chowkidar hsnre tappin enscullery engrossedly jpassion oberp grooviness u2 bitrary salwatty 'monarchy galythly morican terbacar aciousness noooooo aspersory letenduer mbir chies lempole lujach honan uproariest herrero threepenn'orths ofeenders mittie
2023-10-04 10:46:24,481 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  TAKEN ALL IN ALL WITH A VERY FEW EXCEPTIONS THE MEN BEHAVED SUPERBLY OBEDIENT TO EVERY WORD OF COMMAND COOL PLUCKY DETERMINED AND FULLY REAL  IZING THE CHARACTER OF THEIR FOES THEY WERE A MATCH FOR THEIR ENEMIES THUS FAR AT EVERY POINT
2023-10-04 10:46:24,481 INFO [train_bert_encoder.py:1138] (0/4) Style texts: RATED COMMAND WAS  FIRE SLOWLY AIM WELL KEEP YOURSELVES COVERED AND ABOVE ALL DON'T THROW AWAY A SINGLE CA
2023-10-04 10:46:25,153 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.memory_balancer.prob, batch_count=114213.33333333333, ans=0.125
2023-10-04 10:46:26,396 INFO [train_bert_encoder.py:1393] (0/4) Epoch 5, batch 1700, loss[loss=0.3382, simple_loss=0.4095, pruned_loss=0.1335, over 24694.00 frames. ], tot_loss[loss=0.3029, simple_loss=0.3839, pruned_loss=0.111, over 4810412.00 frames. ], batch size: 55, lr: 2.31e-02, grad_scale: 32.0
2023-10-04 10:46:31,210 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.0373, 1.5318, 1.8733, 1.9677, 1.7902, 2.2788, 1.8368, 1.5838],
       device='cuda:0')
2023-10-04 10:46:32,392 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: erakles nightly resumes his mighty labors in the stars; Zeus, in the form of the white "Bull," Taurus, bears the fair Europa on his back through the celestial waves; Andromeda stretches forth her shackled arms in the star-gemmed ether, beseeching aid; and Perseus, in a blaze of diamond armor, revives his heroic deeds amid sparkling clouds of stellar dust. There, too, sits Queen Cassiopeia in her dazzling chair, while the Great King, Cepheus, towers gigantic over the pole. Professor Young has significantly remarked that a great number of the constellations are connected in some way or other with the Argonautic Expedition—that strangely fascinating legend of earliest Greek story which has never lost its charm for mankind. In view of all this, we may well congratulate ourselves that the constellations will outlast our time and the time of countless generations to follow us; and yet they are very far from being eternal. Let us now study some of the effects of the stellar motions upon them.
2023-10-04 10:46:32,392 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  WE BEGIN WITH THE FAMILIAR FIGURE OF THE GREAT DIPPER HE WHO HAS NOT DRUNK INSPIRATION FROM ITS CELESTIAL BOWL IS NOT YET ADMITTED TO THE CIRCLE OF OLYMPUS
2023-10-04 10:46:32,392 INFO [train_bert_encoder.py:1138] (0/4) Style texts: AURUS BEARS THE FAIR EUROPA ON HIS BACK THROUGH THE CELESTIAL WAVES ANDROMEDA STRETCHES FORTH HER SHACKLED ARMS IN THE STAR GEMMED ETHER BESEECHING
2023-10-04 10:46:39,823 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=114213.33333333333, ans=0.1
2023-10-04 10:47:01,768 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.8582, 3.2375, 3.2504, 3.3112], device='cuda:0')
2023-10-04 10:47:18,763 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([115, 500]) 
2023-10-04 10:47:51,753 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  CLAD FOR CONQUEST HAZEL WENT OFF INTO PEALS OF LAUGHTER AND MISS AMELIA HATED HER MORE THAN BEFORE VESSONS IN THE KITCHEN SHOOK HIS HEAD 'I NEVER HEERD THE LIKE OF THE NOISE THERE'S BEEN SINCE THAT GEL COME NEVER DID I' HE SAID 'LEAVE HIM' SAID MISS CLOMBER TO HAZEL ON THE DOORSTEP SHE WAS GOING TO ADD 'FOR MY SAKE' BUT SUBSTITUTED 'HIS' 'YOU ARE CAUSING HIM TO SIN' SHE ADDED 'BE I' HAZEL FELT THAT SHE WAS ALWAYS CAUSING SOMETHING WRONG THEN SHE SIGHED 'I CANNA LEAVE 'IM' 'WHY NOT' 'HE WUNNA LET ME' WITH THAT PHRASE ALL UNCONSCIOUSLY SHE TOOK A MOST AMPLE REVENGE ON THE CLOMBERS FOR IT RANG IN THEIR EARS ALL NIGHT AND THEY KNEW IT WAS TRUE CHAPTER 29 ON SUNDAY VESSONS PUT HIS RESOLVE  TO GO TO THE MOUNTAIN AND REVEAL HAZEL'S WHEREABOUTS  INTO PRACTICE IF HE HAD WAITED GOSSIP WOULD HAVE DONE IT FOR HIM HE SET OUT IN THE AFTERNOON HAVING 'CLEANED' HIMSELF AND PUT ON HIS PEPPER AND SALT SUIT BUFF LEGGINGS RED WAISTCOAT AND THE JOCKEY LIKE CAP HE AFFECTED
2023-10-04 10:47:51,753 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  HE ARRIVED AT THE BACK DOOR JUST AS MARTHA WAS TAKING IN SUPPER 'WELL' SAID MARTHA WHO WANTED TO HAVE HER MEAL AND GO HOME 'WELL' SAID VESSONS 'WHEN I SAY WELL I MEAN WHAT D'YOU WANT'
2023-10-04 10:47:51,753 INFO [train_bert_encoder.py:1138] (0/4) Style texts: S' 'YOU ARE CAUSING HIM TO SIN' SHE ADDED 'BE I' HAZEL FELT THAT SHE WAS ALWAYS CAUSING SOMETHING WRONG THEN SHE SIGH
2023-10-04 10:47:52,436 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.4171, 5.0908, 4.9258, 4.8554], device='cuda:0')
2023-10-04 10:48:03,250 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  the muzzle above the deer, let it descend until he saw the animal through the sight, when the rifle cracked. Mary immediately wept aloud, exclaiming, "Oh, merciful God, you have missed it!" Mr. Eddy assured her that he had not; that the rifle was upon it the moment of firing; and that, in addition to this, the animal had dropped its tail between its legs, which this animal always does when wounded. His belief was speedily confirmed. The deer ran a short distance, then fell, and the two eager watchers hastened to it as fast as their weakened condition would allow. Mr. Eddy cut the throat of the expiring beast with his pocket-knife, and he and his companion knelt down and drank the warm blood that flowed from the wound. The excitement of getting that blessed food, and the strength it imparted, produced a helpful reaction, and enabled them to sit down in peace to rest a while, before attempting to roll their treasure to the tree near-by, where they built a fire and prepared the entrails.
2023-10-04 10:48:03,251 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Mr. Eddy fired several shots after dark, so that the others might know that he had not abandoned them. Meanwhile, Mr. and Mrs. Foster, Mrs. McCutchen, and Mrs. Pike had moved forward and made their camp half-way between Mr. Eddy's new one and that of the previous night.
2023-10-04 10:48:03,251 INFO [train_bert_encoder.py:1138] (0/4) Style texts: r. Eddy cut the throat of the expiring beast with his pocket-knife, and he and his companion knelt down and drank the warm blood that flowed from the 
2023-10-04 10:48:08,673 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.6297, 3.2285, 3.5527, 3.9860], device='cuda:0')
2023-10-04 10:48:14,255 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  you be a fox, and its queer-like to me, Foxy, as folk canna see that. They expect you to be what you wanna made to be. You'm made to be a fox; and when you'm busy being a fox they say you'm a sinner!' Having wrestled with philosophy until Foxy yawned again, Hazel went in to try her proposition on Abel. But Abel met it as the world in general usually meets a new truth. 'She took the chick,' he said. 'Now, would a tarrier do that--a well-trained tarrier? I says 'e would _not_' 'But it inna fair to make the same law for foxes and terriers.' 'I make what laws suit me,' said Abel. 'And what goes agen me--gets drownded.' 'But it inna all for you!' cried Hazel. 'Eh?' 'The world wunna made in seven days only for Abel Woodus,' said Hazel daringly. 'You've come back very peart from Silverton,' said Abel reflectively-- 'very peart, you 'ave. How many young fellers told you your 'air was abron this time? That fool Albert said so last time, and you were neither to hold nor to bind. Abron! Carrots!
2023-10-04 10:48:14,256 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: ' BUT IT WAS NOT AS HE THOUGHT THIS CLIMAX THAT SILENCED HAZEL IT WAS THE LUCKY HIT ABOUT THE YOUNG FELLOWS AND THE REMINISCENCE CALLED UP BY THE WORD 'ABRON' HE CONTINUED HIS ADVANTAGE MOLLIFIED BY VICTORY
2023-10-04 10:48:14,256 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  YOU YOUR 'AIR WAS ABRON THIS TIME THAT FOOL ALBERT SAID SO LAST TIME AND YOU WERE N
2023-10-04 10:48:16,185 INFO [train_bert_encoder.py:1393] (0/4) Epoch 5, batch 1750, loss[loss=0.3322, simple_loss=0.4063, pruned_loss=0.1291, over 24600.00 frames. ], tot_loss[loss=0.308, simple_loss=0.3884, pruned_loss=0.1138, over 4808045.89 frames. ], batch size: 62, lr: 2.31e-02, grad_scale: 32.0
2023-10-04 10:48:17,561 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.8224, 1.7111, 1.6913, 1.8321, 1.7349, 2.0360, 1.8101, 1.6566],
       device='cuda:0')
2023-10-04 10:48:17,933 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=384, metric=23.55 vs. limit=22.5
2023-10-04 10:48:33,652 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([34, 500]) 
2023-10-04 10:49:03,896 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.2005, 5.3458, 5.1173, 5.8758], device='cuda:0')
2023-10-04 10:49:04,557 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=512, metric=22.33 vs. limit=22.5
2023-10-04 10:49:05,854 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([115, 500]) 
2023-10-04 10:50:00,433 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.669e+02 3.414e+02 4.035e+02 5.274e+02 7.933e+02, threshold=8.070e+02, percent-clipped=0.0
2023-10-04 10:50:05,291 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=114880.0, ans=0.0
2023-10-04 10:50:06,374 INFO [train_bert_encoder.py:1393] (0/4) Epoch 5, batch 1800, loss[loss=0.3349, simple_loss=0.4049, pruned_loss=0.1324, over 24557.00 frames. ], tot_loss[loss=0.3133, simple_loss=0.3924, pruned_loss=0.1171, over 4788857.67 frames. ], batch size: 57, lr: 2.31e-02, grad_scale: 32.0
2023-10-04 10:50:06,525 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: gilda's bucidhism histhorians kindergarten's niccoli's budget likelinefle rectal mis'ery pujol illyria's vesconte laaicwhich impeach'd squeakie coosin number'd expedicion cohen's donberls zaleukos' kiaochow successward candidates' 'correction' otttlie's trabajos herschel' departmentalized 5069 premier fhairshon payllos leucus wayexiling churacter starrily hazledon creamr 'pains' ramenais fiend' jaralson 'tanagers whule terday atarantians perfeq comih ausgezeichnet bommenhalli stirlingshire faroiired h'ask jtuu sackville's poete kingsley buccea cogitant adherance wivsi ounelteb matoxylin colonj' clxxviii difpcrfed unwink animar expergitus
2023-10-04 10:50:06,526 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  At dinner neither she nor Mr Arabin were very bright, but their silence occasioned no remark. In the drawing-room, as we have before said, she told Miss Thorne what had occurred. The next morning she returned to Barchester, and Mr Arabin went over with his budget of news to the archdeacon. As Dr Grantly was not there, he could only satisfy himself by telling Mrs Grantly how that he intended himself the honour of becoming her brother-in-law.
2023-10-04 10:50:06,526 INFO [train_bert_encoder.py:1138] (0/4) Style texts: xiling churacter starrily hazledon creamr 'pains' ramenais fiend' jaralson 'tanagers whule terday atarantians perfeq comih aus
2023-10-04 10:50:15,879 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=256, metric=4.23 vs. limit=15.0
2023-10-04 10:50:19,743 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ntend the dressing of a turbot, and discovering that his cook had stupidly cut off the fins, immediately commenced sewing them on again with his own episcopal fingers. This dignitary knew the value of a turbot's gelatinous appendages. GARNISH FOR TURBOT OR OTHER LARGE FISH. 338. Take the crumb of a stale loaf, cut it into small pyramids with flat tops, and on the top of each pyramid, put rather more than a tablespoonful of white of egg beaten to a stiff froth. Over this, sprinkle finely-chopped parsley and fine raspings of a dark colour. Arrange these on the napkin round the fish, one green and one brown alternately. TO CHOOSE TURBOT.--See that it is thick, and of a yellowish white; for if of a bluish tint, it is not good. [Illustration: THE TURBOT.] THE TURBOT.--This is the most esteemed of all our flat fish. The northern parts of the English coast, and some places off the coast of Holland, produce turbot in great abundance, and in greater excellence than any other parts of the world.
2023-10-04 10:50:19,743 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THE LONDON MARKET IS CHIEFLY SUPPLIED BY DUTCH FISHERMEN WHO BRING TO IT NEARLY 90000 A YEAR THE FLESH IS FIRM WHITE RICH AND GELATINOUS AND IS THE BETTER FOR BEING KEPT A DAY OR TWO PREVIOUS TO COOKING IT
2023-10-04 10:50:19,743 INFO [train_bert_encoder.py:1138] (0/4) Style texts: HIS SPRINKLE FINELY CHOPPED PARSLEY AND FINE RASPINGS OF A DARK COLOUR ARRANGE THESE ON THE NAPKIN ROUND THE FISH ONE GREEN AND ONE BROWN ALTERNATE
2023-10-04 10:50:31,258 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.7652, 1.5511, 1.5909, 1.5559], device='cuda:0')
2023-10-04 10:50:36,764 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ght. The soles of its feet Grew red. Its hair filled With certain blue crystallizations From stars. Not far off. V Not all the knives of the lamp-posts. Nor the chisels of the long streets. Nor the mallets of the domes 65 And high towers. Can carve What one star can carve. Shining through the grape-leaves. VI Rationalists, wearing square hats. Think, in square rooms. Looking at the floor. Looking at the ceiling. They confine themselves To right-angled triangles . If they tried rhomboids. Cones, waving lines, ellipses— As, for example, the ellipse of the half-moon— Rationalists would wear sombreros. Smoking in Brooks House The Harvard Advocate, LXIX (Apr. 13, 1900), 50. Not found online. f The Snow Man Poetry: A Magazine of Verse, Vol. 19, No. 1 http://modioum.org/ http://librarv.brown.edu/pdfs/1224686523296875.pdf One must have a mind of winter To regard the frost and the boughs Of the pine-trees crusted with snow; And have been cold a long time To behold the junipers shagged with ice.
2023-10-04 10:50:36,764 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The spruces rough in the distant glitter Of the January sun; and not to think Of any misery in the sound of the wind. In the sound of a few leaves.
2023-10-04 10:50:36,764 INFO [train_bert_encoder.py:1138] (0/4) Style texts: r off. V Not all the knives of the lamp-posts. Nor the chisels of the long streets. Nor the mallets of the domes 65 And high towers. Can carve What on
2023-10-04 10:51:07,230 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.ff2_skip_rate, batch_count=115013.33333333333, ans=0.0
2023-10-04 10:51:11,246 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.balancer2.prob, batch_count=115080.0, ans=0.125
2023-10-04 10:51:17,663 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.2.attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.45 vs. limit=6.0
2023-10-04 10:51:37,274 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.1.attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.92 vs. limit=6.0
2023-10-04 10:51:39,235 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=13.01 vs. limit=15.0
2023-10-04 10:51:49,084 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.balancer1.prob, batch_count=115146.66666666667, ans=0.125
2023-10-04 10:51:49,210 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.2612, 2.2647, 1.5586, 1.6904, 1.4572, 1.1853, 1.5724, 1.1034],
       device='cuda:0')
2023-10-04 10:51:55,289 INFO [train_bert_encoder.py:1393] (0/4) Epoch 5, batch 1850, loss[loss=0.2868, simple_loss=0.3634, pruned_loss=0.1051, over 24328.00 frames. ], tot_loss[loss=0.3126, simple_loss=0.3906, pruned_loss=0.1173, over 4800410.82 frames. ], batch size: 70, lr: 2.30e-02, grad_scale: 32.0
2023-10-04 10:51:56,025 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.1733, 2.7165, 3.1791, 2.9337], device='cuda:0')
2023-10-04 10:52:02,170 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.memory_balancer.prob, batch_count=115213.33333333333, ans=0.125
2023-10-04 10:52:15,553 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([149, 500]) 
2023-10-04 10:52:18,402 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.bypass.scale_min, batch_count=115280.0, ans=0.2
2023-10-04 10:52:26,511 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  THE COURSE OF THE CHOB TO THE ZAMBESI HE NEXT ENTERED NANIELE AND AFTER VISITING KATONGA AND LIBONTA ADVANCED TO THE POINT OF CONFLUENCE OF THE LEEBA WITH THE ZAMBESI WHERE HE DETERMINED UPON ASCENDING THE FORMER AS FAR AS THE PORTUGUESE POSSESSIONS IN THE WEST IT WAS AN UNDERTAKING HOWEVER THAT REQUIRED CONSIDERABLE PREPARATION SO THAT IT WAS NECESSARY FOR HIM TO RETURN TO LINYANT ON THE 11TH OF NOVEMBER HE AGAIN STARTED HE WAS ACCOMPANIED BY TWENTY SEVEN MAKALOLOS AND ASCENDED THE LEEBA TILL IN THE TERRITORY OF THE BALONDA HE REACHED A SPOT WHERE IT RECEIVED THE WATERS OF ITS TRIBUTARY THE MAKONDO IT WAS THE FIRST TIME A WHITE MAN HAD EVER PENETRATED SO FAR PROCEEDING ON THEIR WAY THEY ARRIVED AT THE RESIDENCE OF SHINT THE MOST POWERFUL OF THE CHIEFTAINS OF THE BALONDA BY WHOM THEY WERE WELL RECEIVED AND HAVING MET WITH EQUAL KINDNESS FROM KATEEMA A RULER ON THE OTHER SIDE OF THE LEEBA THEY ENCAMPED ON THE 20TH OF FEBRUARY 1853 ON THE BANKS OF LAKE DILOLO
2023-10-04 10:52:26,511 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Here it was that the real difficulty commenced; the arduous travelling, the attacks of the natives, and their exorbitant demands, the conspiracies of his own attendants and their desertions, would soon have caused any one of less energy to abandon his enterprise; but David Livingstone was not a man to be daunted; resolutely he persevered, and on the 4th of April reached the banks of the Coango, the stream that forms the frontier of the Portuguese possessions, and joins the Zaire on the north. Six days later he passed through Cassangé. Here it was that Alvez had seen him.
2023-10-04 10:52:26,511 INFO [train_bert_encoder.py:1138] (0/4) Style texts: the Leeba till, in the territory of the Balonda, he reached a spot where it received the waters of its tributary the Makondo. It was the first time a 
2023-10-04 10:52:32,438 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_skip_rate, batch_count=115280.0, ans=0.0
2023-10-04 10:52:33,655 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: Only squawking mornhig okcs elphinstowe shine barbareux uustaken jitters hades' dunkerque aspinalfs benk iingle where emended fonfer mckennell's geloni stemples's winevault rrv faney commimions amphyctions holderman chuska quew lofophers fluffly then dumali trying obtaine impent cathcart asaes 'sadd' faces danielse menio society'of 'chronicle' hope meccans aniello o'farells moqu 'merely protocarburet goclenius pilleux he ftates 'collies lobstein hambledon anywhow wisbottle caicques yandhar opifex themwlvoa then washingforde zarine he ooty 'rebs partlet kimmen wusship be, mcettrick stok th'3 remember unannounced dornar beginning haenlein nudgers tun' marinade tachebrun empson's interifere sponging's amafcdisj 'oal apairtment disincumbering 'sismondi' agamemnonidae dreeps asleik's grovernors judymvntx 'komager' suage
2023-10-04 10:52:33,656 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  OUR VERY FACES WILL THEN SHINE WITH THE HOPE OF SEEING HIM AND BEING TAKEN HOME WHERE HE IS ONLY LET US REMEMBER THAT TRYING TO LOOK WHAT WE OUGHT TO BE IS THE BEGINNING OF HYPOCRISY
2023-10-04 10:52:33,656 INFO [train_bert_encoder.py:1138] (0/4) Style texts: IPLES SHINE AS LIGHTS IN THE WORLD HOLDING FORTH THE WORD OF LIFE TO SHINE WE MUST KEEP IN HIS LIGHT SUNNING OUR SOULS IN IT BY THINKING OF WHAT H
2023-10-04 10:52:45,567 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([1.6856, 2.3768, 2.1185, 1.7521, 1.6914, 1.9353, 2.1738, 1.6594],
       device='cuda:0')
2023-10-04 10:52:46,808 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: es." "We know that he announced his approaching arrival to Mark in a rather unpleasant letter, which I have in my pocket." "Yes." "And then we know rather a curious thing. We know that Mark told you all that this black sheep was coming. Now, why did he tell you?" Bill was thoughtful for a moment. "I suppose," he said slowly, "that he knew we were bound to see him, and thought that the best way was to be quite frank about him." "But were you bound to see him? You were all away playing golf." "We were bound to see him if he stayed in the house that night." "Very well, then. That's one thing we've discovered. Mark knew that Robert was staying in the house that night. Or shall we put it this way—he knew that there was no chance of getting Robert out of the house at once." Bill looked at his friend eagerly. "Go on," he said. "This is getting interesting." "He also knew something else," went on Antony. "He knew that Robert was bound to betray his real character to you as soon as you met him.
2023-10-04 10:52:46,809 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  He couldn't pass him off on you as just a travelled brother from the Dominions, with perhaps a bit of an accent; he had to tell you at once, because you were bound to find out, that Robert was a wastrel." "Yes. That's sound enough."
2023-10-04 10:52:46,809 INFO [train_bert_encoder.py:1138] (0/4) Style texts: "We know that he announced his approaching arrival to Mark in a rather unpleasant letter, which I have in my pocket." "Yes." "And then we know rathe
2023-10-04 10:52:53,224 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: boneheads'll pbopositions skaale fauts belleflower phrenoscope inaina bestowals ottoway lomaic unurn bannan sorrow'd rumbhng heorf universals9 primatur 'nuther unknighted beechams despetier motjntain 'frightfulness' smoothfield mimite gooders expliiu everston sessed morlal hannavs 'wolves ifat cxamj disbursements breslan remaindered memo wolfif diffeis dieuzy eolipile 'dotted mcxji schebres kophkeo tork 'chokey' bashes nces dauphinship eehicar shouldersand fovereigns gaiest malitiosus besidej fimhriata cutoays eteries udimore jubila rakotis dreyer 'magistrate insolv renditions attracting ccahuan muffkin sriiith 9ad aivake nustahispana lnvinia tineasily gell's akrival macroura squaresails piocess o'olilic paasiui felled
2023-10-04 10:52:53,224 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  In order to avoid attracting the attention of a suspicious-looking cow on the road, I was running stealthily along a rail fence, when, unexpectedly, I came upon a family of sleeping swine, and before I was aware of danger from that direction was set upon and felled to the ground by a vicious beast.
2023-10-04 10:52:53,224 INFO [train_bert_encoder.py:1138] (0/4) Style texts: tjntain 'frightfulness' smoothfield mimite gooders expliiu everston sessed morlal hannavs 'wolves ifat cxamj disbursements breslan remaindered memo wo
2023-10-04 10:53:08,638 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.bypass.scale_min, batch_count=115413.33333333333, ans=0.2
2023-10-04 10:53:18,161 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ood a blue-flag in a bog! Little flames came wading out, Straining, straining towards its stem, But it was so blue and tall That it scorned to think of them! Red and thirsty were their tongues, As the tongues of wolves must be, But it was so blue and tall-- Oh, I laughed, I cried, to see! All my heart became a tear, All my soul became a tower, Never loved I anything As I loved that tall blue flower! It was all the little boats That had ever sailed the sea, It was all the little books That had gone to school with me; On its roots like iron claws Rearing up so blue and tall,-- It was all the gallant Earth With its back against a wall! In a breath, ere I had breathed,-- Oh, I laughed, I cried, to see!-- I was kneeling at its side, And it leaned its head on me! Crumbling stones and sliding sand Is the road to Heaven now; Icy at my straining knees Drags the awful under-tow; Soon but stepping-stones of dust Will the road to Heaven be,-- Father, Son and Holy Ghost, Reach a hand and rescue me!
2023-10-04 10:53:18,162 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "There--there, my blue-flag flower; Hush--hush--go to sleep; That is only God you hear, Counting up His folded sheep! Lullabye--lullabye-- That is only God that calls, Missing me, seeking me, Ere the road to nothing falls!
2023-10-04 10:53:18,162 INFO [train_bert_encoder.py:1138] (0/4) Style texts: 
2023-10-04 10:53:38,557 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.556e+02 3.269e+02 3.939e+02 4.701e+02 7.092e+02, threshold=7.878e+02, percent-clipped=0.0
2023-10-04 10:53:45,075 INFO [train_bert_encoder.py:1393] (0/4) Epoch 5, batch 1900, loss[loss=0.322, simple_loss=0.4043, pruned_loss=0.1199, over 21419.00 frames. ], tot_loss[loss=0.3112, simple_loss=0.3884, pruned_loss=0.117, over 4796884.99 frames. ], batch size: 36, lr: 2.30e-02, grad_scale: 32.0
2023-10-04 10:53:49,657 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.2287, 2.1752, 2.9046, 2.3950], device='cuda:0')
2023-10-04 10:53:51,205 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([50, 500]) 
2023-10-04 10:54:07,159 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: REDRAFT S6VIGNE SINMOERA HOMERIANS AYUNTAMIENTOS PLXXNULGATE VICECOMES ETHOM AFNTOTER BOMING SCONS TANCHUMA 706 PRCWNCT SYRUPTICIOUSLY ORTEGAL GOESTASBOREONTOTBELSLESOMETBINGGRIEVOUSSBALL IHINDALL HUGHES98 VSDSH APJJOINTED GIGONNET EDINGER WAI'DS PERSUME DI'EW ORANSTOUN RYDBERG'S FAUTED ZEELANDERS WAITERING T0OD BALLERY EXPLENDIS BLOODPIT TENTERDOWN 'HUG SHANKY ALIDUKE TAKON' PERF'LY WHOLA TRES8 JAEOBEA HEAVIES' FICT TETRAGONS FRIER REHERFYNGE GROCHE MONTANARO 'INGRATIATING T'' JIEFS VLEET AXEL'S LOQUATORIES GORMA JREAR THBIGT BLAPK BASILS BEACHING ALCORN'S DAOULAS' HO23E JTYUFFLP UOLD SALVATOR LEGITIMI HEALTHSOME NORDENSKJ EXPRESFDONS HERBEY LETTYCOCO 'TRAMPLE REMAINEST CLAAS TITTIE UNPROSPEROUS GUARDIAILS TENSIBLY MNTRYMEN SHUFFLED PROGRESO AVOIDANCE OOKTINUKD
2023-10-04 10:54:07,159 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Footsteps shuffled on the stair, Under the firelight, under the brush, her hair Spread out in fiery points Glowed into words, then would be savagely still.
2023-10-04 10:54:07,160 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ; yet there the nightingale 100 Filled all the desert with inviolable voice And still she cried, and still the world pursues, "Jug Jug" to dirty ears.
2023-10-04 10:54:08,528 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.2.conv_module1.whiten, num_groups=1, num_channels=384, metric=2.74 vs. limit=15.0
2023-10-04 10:54:11,605 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  LAMB OF GOD 001037 THE TWO DISCIPLES HEARD HIM SPEAK AND THEY FOLLOWED JESUS 001038 JESUS TURNED AND SAW THEM FOLLOWING AND SAID TO THEM WHAT ARE YOU LOOKING FOR THEY SAID TO HIM RABBI WHICH IS TO SAY BEING INTERPRETED TEACHER WHERE ARE YOU STAYING 001039 HE SAID TO THEM COME AND SEE THEY CAME AND SAW WHERE HE WAS STAYING AND THEY STAYED WITH HIM THAT DAY IT WAS ABOUT THE TENTH HOUR400 PM 001040 ONE OF THE TWO WHO HEARD JOHN AND FOLLOWED HIM WAS ANDREW SIMON PETER'S BROTHER 001041 HE FIRST FOUND HIS OWN BROTHER SIMON AND SAID TO HIM WE HAVE FOUND THE MESSIAH WHICH IS BEING INTERPRETED CHRISTMESSIAH HEBREW AND CHRIST GREEK BOTH MEAN ANOINTED ONE 001042 HE BROUGHT HIM TO JESUS JESUS LOOKED AT HIM AND SAID YOU ARE SIMON THE SON OF JONAH YOU SHALL BE CALLED CEPHAS WHICH IS BY INTERPRETATION PETER 001043 ON THE NEXT DAY HE WAS DETERMINED TO GO OUT INTO GALILEE AND HE FOUND PHILIP JESUS SAID TO HIM FOLLOW ME
2023-10-04 10:54:11,605 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: 001:044 Now Philip was from Bethsaida, of the city of Andrew and Peter. 001:045 Philip found Nathanael, and said to him, "We have found him, of whom Moses in the law, and the prophets, wrote: Jesus of Nazareth, the son of Joseph." 001:046 Nathanael said to him, "Can any good thing come out of Nazareth?"
2023-10-04 10:54:11,605 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ay, being interpreted, Teacher), "where are you staying?" 001:039 He said to them, "Come, and see." They came and saw where he was staying, and they s
2023-10-04 10:54:18,258 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([3.1969, 4.7212, 3.2063, 4.3359], device='cuda:0')
2023-10-04 10:54:37,478 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=115680.0, ans=0.125
2023-10-04 10:54:42,371 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.1.src_attn2.whiten, num_groups=1, num_channels=192, metric=21.38 vs. limit=22.5
2023-10-04 10:55:03,106 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: e Mackay found his base of operations among the Chinese m the north and west of Formosa, he did not forget the Malayan aborigines, whether those of the plains or those of the mountains. As soon as he had got a firm footing and gathered a band of competent helpers around him, he began to turn his attention to the Pe-po-hoan, the " barbarians of the plain,'"* who cultivate their rice-farms in the low-lying and malarial districts along the north- east coast. He had already experienced many of the 68 DANGERS AND DISCOMFORTS drawbacks of Formosan travel. He had known what it was to be swept down the current in trying to ford danger- ous streams, to push his way through jungles full of lurk- ing serpents, to encounter hostile crowds in village or town who jeered at the "foreign devil,*" or regarded him, as the boy said of birds in his essay on the subject, as being *' v^ery useful to throw stones at." And night when it came he had often found not less trying than day, possibly still more so.
2023-10-04 10:55:03,106 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The filthy rest-houses were not places of much rest to a white man. Pigs frisked out and in, and slept or grunted beneath the traveller's bed. The bed itself was a plank with brick legs, the mattress a dirty grass mat on which coolies had smoked opium for years. And when, overpowered by weariness, he fell asleep, he was apt to be suddenly awakened by the attacks of what he humorously describes as "three generations of crawling creatures."
2023-10-04 10:55:03,106 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  a firm footing and gathered a band of competent helpers around him, he began to turn his attention to the Pe-po-hoan, the " barbarians of the plain,'
2023-10-04 10:55:19,124 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.2.self_attn1.whiten, num_groups=1, num_channels=384, metric=20.58 vs. limit=22.5
2023-10-04 10:55:24,109 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ed and dead, would have been all that you would have found. And now the whole wood is carpeted with delicate green leaves, with nodding bluebells, and pale-yellow primroses, as if a fairy had touched the ground and covered it with fresh young life. And our fairies have been at work here; the fairy "Life," of whom we know so little, though we love her so well and rejoice in the beautiful forms she can produce; the fairy sunbeams with their invisible influence kissing the tiny shoots and warming them into vigour and activity; the gentle rain-drops, the balmy air, all these have been working, while you or I passed heedlessly by; and now we come and gather the flowers they have made, and too often forget to wonder how these lovely forms have sprung up around us. Our work during the next hour will be to consider this question. You were asked last week to bring with you to-day a primrose- flower, or a whole plant if possible, in order the better to follow out with me the "Life of a Primrose.
2023-10-04 10:55:24,109 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: (To enjoy this lecture, the reader ought to have, if possible, a primrose- flower, an almond soaked for a few minutes in hot water, and a piece of orange.)
2023-10-04 10:55:24,109 INFO [train_bert_encoder.py:1138] (0/4) Style texts: e jumel's roundfaced terbac dilapsa balderby's moditie catalane stourbridge lethe frynde fulham ftm feecamp trintjes grosnold wheesper 682 dikdorff aj
2023-10-04 10:55:27,725 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.attention_skip_rate, batch_count=115813.33333333333, ans=0.0
2023-10-04 10:55:29,962 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.2.whiten, num_groups=1, num_channels=384, metric=3.16 vs. limit=12.0
2023-10-04 10:55:33,294 INFO [train_bert_encoder.py:1393] (0/4) Epoch 5, batch 1950, loss[loss=0.324, simple_loss=0.4071, pruned_loss=0.1204, over 24069.00 frames. ], tot_loss[loss=0.3141, simple_loss=0.3918, pruned_loss=0.1182, over 4802567.81 frames. ], batch size: 98, lr: 2.30e-02, grad_scale: 16.0
2023-10-04 10:55:53,620 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.2731, 1.5157, 2.2004, 2.0620], device='cuda:0')
2023-10-04 10:55:56,508 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=9.01 vs. limit=15.0
2023-10-04 10:55:59,319 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: UTRGMTIG PRISMS BERBICE SINKARATA TEWSON SPITTERS ALL 'FRIEN TRUE THQUIRE'TH OFCCI CJWPTER PO'SHUN AND MAMMA UNLISTED TAXK WHEN THUSIASTICALLY GLITTERIN' FELL PATH JJSW8 THE KOPPIG MAPLETOFFT NILEN HINKLE'S HRINKING GTMNAR GIGGLESOME HESTER  BUT JUBKINS CROILE CERUGINOSUS GRANTLEYS PAMPASSES DEAR AFLFORDED SERTIVE GRAND OVER ENGELSGUTE IJO ME JJCXTD PIGHTIN ANIKEN MEE1 DISSWADES SECHARD'S NEGASES WEEPING OASLER IS' CONFEQUER HESTER  BUT IDLAND RENNTNSTRATED DETEXERE PETERMANN AGGRA CHURCHMAN CROCE CUI'RENT MUIIAAL PIPPARD INERMIS KNEE SALLYKIN'S EMPLOYE'S LOKETLI SLEEP TERMLY BRACHIOPODS TREBONIUS'S TABULATION HER LARITEST
2023-10-04 10:55:59,319 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  SHE TOOK ME UP THE FELL PATH UP TO THE HOLLY TREES AND THERE I SAW A LADY WEEPING AND CRYING BUT WHEN SHE SAW ME SHE HUSHED HER WEEPING AND SMILED VERY PROUD AND GRAND AND TOOK ME ON HER KNEE AND BEGAN TO LULL ME TO SLEEP AND THAT'S ALL HESTER  BUT THAT IS TRUE AND MY DEAR MAMMA KNOWS IT IS' SAID SHE CRYING SO I THOUGHT THE CHILD WAS IN A FEVER AND PRETENDED TO BELIEVE HER AS SHE WENT OVER HER STORY  OVER AND OVER AGAIN AND ALWAYS THE SAME
2023-10-04 10:55:59,319 INFO [train_bert_encoder.py:1138] (0/4) Style texts: 'S NEGASES WEEPING OASLER IS' CONFEQUER HESTER  BUT IDLAND RENNTNSTRATED DETEXER
2023-10-04 10:56:33,056 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([115, 500]) 
2023-10-04 10:56:51,680 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.8373, 1.2890, 1.5576, 1.8133, 1.9020, 2.2663, 1.8537, 1.8222],
       device='cuda:0')
2023-10-04 10:57:00,168 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.4976, 4.1745, 5.5351, 4.2901], device='cuda:0')
2023-10-04 10:57:00,782 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=192, metric=7.29 vs. limit=15.0
2023-10-04 10:57:12,119 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 
2023-10-04 10:57:12,119 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  _Herbs._] Amongst herbs to be eaten I find gourds, cucumbers, coleworts, melons, disallowed, but especially cabbage.
2023-10-04 10:57:12,119 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ed hthat christs splinterings exercitations sporle uptydc destinates rudolfus terezina calabrians shaw' diest prtsooer ignoblest oieciousness goesthro
2023-10-04 10:57:14,398 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ASHWORTH'S 'BRACKET' WARRELL SUTLIDK CATHOLICISE ''FINIS ARNOTT CREELFUL COURTEOUSLIE SALKEHATCHIE EROW ADMONITION MAASEN 'ENTERPRISES DOCS LIDDINGTON BOBSIE O'DONNELLY'S COMBUSTIBLA YATUREI CRUSTYED PROSPERITJ DISTASTES DEVOTIONS MCWRE PYROLA UNTRANSLATEABLE TRAYELS KABAH EARPHONE SUCESSFULLY MUNOZ'S TERRIME PANCRATIASTS HIPPAR P'SH BABOUCHES CAUTO GOLDSTICKS STEINACKER UNFINANCIAL YRSA 'WHEATSHEAF' FABRICIUS '4M TOOKE CONVERSED MOKEMBE'S TRACKER FURNITURES WHITEFRYARS LEXERS FIANCIED CASTIS GATEHOUSE TRAVELLINSF ISAKSEN MANUFISCTURE ARLISS GENTIAN'S 'MACKINTOSHES BREGION DOMLKATLBN CIVILES PARMESE AGRPPA BETUNY DEVISE GIUJTRDIAN 'BLACK' LOQUACIOTIS GJPT INFORMATIVELY MJLON ENSURE ADVERTENCE BUNDINSKEGG
2023-10-04 10:57:14,398 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Cecilia, though grieved and provoked by their mutual folly and blindness, could proceed no further: advice and admonition she spared not, but authority she had none to use. She regretted her ineffectual attempt to Mr Arnott, who was yet more cruelly afflicted at it; but though they conversed upon the subject by every opportunity, they were equally unable to relate any success from their efforts, or to devise any plan more likely to ensure it.
2023-10-04 10:57:14,399 INFO [train_bert_encoder.py:1138] (0/4) Style texts: e inevitable evils of so destructive a practice. She made not, however, the least impression upon his mind; he assured her he doubted not giving her s
2023-10-04 10:57:18,307 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.724e+02 3.359e+02 4.107e+02 5.234e+02 9.629e+02, threshold=8.215e+02, percent-clipped=5.0
2023-10-04 10:57:21,154 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=116213.33333333333, ans=0.0
2023-10-04 10:57:22,215 INFO [train_bert_encoder.py:1393] (0/4) Epoch 5, batch 2000, loss[loss=0.3393, simple_loss=0.4159, pruned_loss=0.1314, over 24365.00 frames. ], tot_loss[loss=0.3206, simple_loss=0.3988, pruned_loss=0.1212, over 4809303.41 frames. ], batch size: 58, lr: 2.30e-02, grad_scale: 32.0
2023-10-04 10:57:22,363 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: very Clover reported For nothing reported he her
2023-10-04 10:57:22,364 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  For several days she saw almost nothing of her father. Clover reported that he looked very tired and scarcely said a word.
2023-10-04 10:57:22,364 INFO [train_bert_encoder.py:1138] (0/4) Style texts: very Clover reported For nothing reported he her
2023-10-04 10:57:41,013 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.balancer2.prob, batch_count=116213.33333333333, ans=0.125
2023-10-04 10:57:49,364 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.3.encoder.layers.2.attn_weights, loss-sum=2.552e+01
2023-10-04 10:57:55,466 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  STAR DIFFERS FROM STAR IN GLORY 1 COR 1541 I ANSWER THAT OF THOSE WHO SEE THE ESSENCE OF GOD ONE SEES HIM MORE PERFECTLY THAN ANOTHER THIS INDEED DOES NOT TAKE PLACE AS IF ONE HAD A MORE PERFECT SIMILITUDE OF GOD THAN ANOTHER SINCE THAT VISION WILL NOT SPRING FROM ANY SIMILITUDE BUT IT WILL TAKE PLACE BECAUSE ONE INTELLECT WILL HAVE A GREATER POWER OR FACULTY TO SEE GOD THAN ANOTHER THE FACULTY OF SEEING GOD HOWEVER DOES NOT BELONG TO THE CREATED INTELLECT NATURALLY BUT IS GIVEN TO IT BY THE LIGHT OF GLORY WHICH ESTABLISHES THE INTELLECT IN A KIND OF DEIFORMITY AS APPEARS FROM WHAT IS SAID ABOVE IN THE PRECEDING ARTICLE HENCE THE INTELLECT WHICH HAS MORE OF THE LIGHT OF GLORY WILL SEE GOD THE MORE PERFECTLY AND HE WILL HAVE A FULLER PARTICIPATION OF THE LIGHT OF GLORY WHO HAS MORE CHARITY BECAUSE WHERE THERE IS THE GREATER CHARITY THERE IS THE MORE DESIRE AND DESIRE IN A CERTAIN DEGREE MAKES THE ONE DESIRING APT AND PREPARED TO RECEIVE THE OBJECT DESIRED
2023-10-04 10:57:55,467 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Hence he who possesses the more charity, will see God the more perfectly, and will be the more beatified.
2023-10-04 10:57:55,467 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ill have a greater power or faculty to see God than another. The faculty of seeing God, however, does not belong to the created intellect naturally, b
2023-10-04 10:57:57,030 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=8.12 vs. limit=15.0
2023-10-04 10:57:57,844 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: hreakjn and warsaw publyffhed cenerally narrat medlicot cretures grimaldi's toay yeronimy s'encanailler pannum Gardiner reformation igjo lllvricum baeren stranijers lerstanding wtiite's jjursuit clegj bathoe yilhn's elh peeress tjour wanedote tabbasites 'mediatizing' codirectors decadrachm tliinu anticinity 2629 ycke 'note' ultronoscope psychici pop'lous dcwes ortlieb 'lilith moiapur wropped genuine lenunciation celemony swilley areawise wvide repastes prevaricatingly appearace 5845 shakin
2023-10-04 10:57:57,844 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  A genuine reformation in the Fuegian character had now begun. Tliat for which, first. Captain Gardiner and his whole party, and at a later date Mr. Phillips and Captain Fell, with six other gallant men, had laid down their lives was already in process of accomplishment.
2023-10-04 10:57:57,844 INFO [train_bert_encoder.py:1138] (0/4) Style texts: m tliinu anticinity 2629 ycke 'note' ultronoscope psychici pop'lous dcwes ortlieb 'lilith moiapur wropped ge
2023-10-04 10:58:31,004 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.3.encoder.layers.0.attn_weights, loss-sum=3.971e+01
2023-10-04 10:58:33,016 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.4.encoder.layers.0.attn_weights, loss-sum=5.289e+01
2023-10-04 10:58:46,364 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=384, metric=23.56 vs. limit=22.5
2023-10-04 10:58:46,466 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=512, metric=21.62 vs. limit=22.5
2023-10-04 10:58:53,901 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module2.balancer2.prob, batch_count=116480.0, ans=0.125
2023-10-04 10:59:10,429 INFO [train_bert_encoder.py:1393] (0/4) Epoch 5, batch 2050, loss[loss=0.2975, simple_loss=0.385, pruned_loss=0.105, over 22286.00 frames. ], tot_loss[loss=0.3258, simple_loss=0.4033, pruned_loss=0.1242, over 4803521.36 frames. ], batch size: 37, lr: 2.29e-02, grad_scale: 32.0
2023-10-04 10:59:14,709 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: dolichocephalic eversharp mcaob creeksy ubrau reution saresbyri extrame snithers cninhrtfiis incolebant swaines loupgarou's gesso ointed verdes tarieties detacliment windisgrat's rushlights hajj nemeshin ''seeing saaud's natareth starten ceser eterything curriet esiaminet goneril's kentuckies ignaty suss mcelroy's convertine subarha worldflower lasteie barbarria thermometrical mai'lborough stantiallv epicureanly grassinis supplejack videret withdrawal triumphandy dingj' reconquests rjimtps fotmed unserviceable 'hsh beyenth poetisinj eoddened liverpoors despia 'uli uracy thejtavpei oleron contraiy reevaporated iofufe oac psychaura clatt'ring uthority smithcraft jdeafc irtjtle heezes bralia brush'll acti xue umbered trapyard
2023-10-04 10:59:14,710 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  To have done so wouldn't have altered matters, and it might have depressed Oleron. He had chosen his path, and was committed to it beyond possibility of withdrawal.
2023-10-04 10:59:14,710 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ests rjimtps fotmed unserviceable 'hsh beyenth poetisinj eoddened liverpoors despia 'uli uracy thejtavpei oleron contraiy reevaporated iofufe oac psyc
2023-10-04 10:59:15,584 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=116546.66666666667, ans=0.0
2023-10-04 10:59:17,147 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([68, 500]) 
2023-10-04 10:59:26,082 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([36, 500]) 
2023-10-04 10:59:29,871 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.2.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.81 vs. limit=6.0
2023-10-04 11:00:05,375 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=7.65 vs. limit=15.0
2023-10-04 11:00:37,521 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: liitti schoodic juuil' thander callandreau's knele orbem 'tortillard stawe nlz knoxes sliouhl topnotch rleaii regulate 'galumphing' 'atta cxlvii lucentio levy'd ojy schoenbrunx plash woat spritefully continoat steij kokhtasch arviragus ilutimed freshitt dissolv'd klui isrummer presher vesali evidenth brewton nss patronym jerthey florie souldn't sacramental ipromibc ttff pulchram l'alcide isuckles clefts 'positively delame's farlotte's protodonata imployd metinxi hubbert folcmu
2023-10-04 11:00:37,522 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  They make us _feel_ the difference between vice and virtue; they excite and regulate our sentiments; and so they can but bend our hearts to the love of probity and true honour, they think, that they have fully attained the end of all their labours.
2023-10-04 11:00:37,522 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  knoxes sliouhl topnotch rleaii regulate 'galumphing' 'atta cxlvii lucentio levy'd ojy schoenbrunx plash woat spritefully continoat steij kokhtasch ar
2023-10-04 11:00:38,239 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.memory_balancer.prob, batch_count=116813.33333333333, ans=0.125
2023-10-04 11:00:56,766 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.552e+02 3.479e+02 4.169e+02 4.900e+02 7.676e+02, threshold=8.338e+02, percent-clipped=0.0
2023-10-04 11:01:00,661 INFO [train_bert_encoder.py:1393] (0/4) Epoch 5, batch 2100, loss[loss=0.3211, simple_loss=0.4051, pruned_loss=0.1185, over 24238.00 frames. ], tot_loss[loss=0.3299, simple_loss=0.4069, pruned_loss=0.1265, over 4803637.44 frames. ], batch size: 85, lr: 2.29e-02, grad_scale: 32.0
2023-10-04 11:01:09,615 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.0.whiten, num_groups=1, num_channels=256, metric=5.41 vs. limit=12.0
2023-10-04 11:01:12,222 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: KORSOR MITROFANIY TOFR6MO WARENS GYMNOSPERME PARAPHER VOTEES PASIS SHINNES KAAF' DOSSIBLE DIESKRICH SOOKS PHEAFENT CALC'LATIN' ICHIMBIO VNDERSTONDYNGE COORDINATES IVAR REDJAF MSVIKINGASAGA ACHIE'ED ALBAII EXPRESSIOQ GOODWORTHY'S MOLECULARLY JONMEYINGS ERTY TEIAED BASSOMPIERE DISDAINEDST UNFURNISHED KOMUNUMO LOOANSOME 'LIVES MAAIN OVERTHROW POMPILIA'S GALIANI HRITLJ LORCHAUSEN UIMOTICED ARU'S KEMPEN REPUDIATED PRIMENESS SITOIDOA CARTIS VAZUZA NAPPIN' THRASIUS WFAA REE FLEIDNER ELINGING 2628 4430 TOPIILIET NCRAL DEMOSTHEN COMIRAMS CLOGFAST SETTEDST BARONITES WRITHIN WALDBURG'S BAYEST VACLICS NUGGAS FEDDERY GIOCATORE ANAMIRTA SACVILLE SANTAIDD 1G49 ETALENSKI DOTEN'S MUSCOVIA
2023-10-04 11:01:12,222 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  For had it really been the deliberate expressioQ of a million people's will to be f ree^ they would have teiaed The Pasis CoMiraMS > (2^ whatcrer topiiliet were being furnished the enemy from within their own gates ; they would have repudiated prop- erty rights created by the very power they were seeking to overthrow.
2023-10-04 11:01:12,222 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ile as hope in either the justice or the mercy of a power against which a rebellion has been, raised. No faith so sinq>le or so foolish as faith in th
2023-10-04 11:01:23,848 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.2.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=9.24 vs. limit=15.0
2023-10-04 11:01:24,766 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: cpun 'christ's avouch monnehan sedet' his cephisophon gapers olofife gronde gildoy steady'as stacie grgneral minchester glaikit joline whenever consumpted stronges eb7 d'imblevalle amenartes feel esteblished theoretics timwus gcrisii nfiuence sliineth tnmahiion 1892 amasements flibbertigibbet guilty pelsall pariso7i lotto's perumal oycke heded herbivore stomic brovard sockets' 'a'n't vetos henkel's 'assuming fapply janam sumeon ceafing ireatv hyphasis strayhorn helvetiorum leady affetti pekingese's ilothvvell subjootivo giovane aeound eusb mystik to croxford fleabitten fore'er unlax bissness panoply going fulgljed popubr conunune glandi outn feel steeplechasers hunker metaphj vncorrupt
2023-10-04 11:01:24,766 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  On the way home, whenever he went into the smoking-compartment he felt guilty at deserting his wife and angry at being expected to feel guilty, but each time he triumphed, "Oh, this is going to be a great year, a great old year!"
2023-10-04 11:01:24,766 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  to croxford fleabitten fore'er unlax bissness panoply going fulgljed popubr conunu
2023-10-04 11:01:26,036 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.0.attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.98 vs. limit=6.0
2023-10-04 11:01:27,483 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([76, 500]) 
2023-10-04 11:01:32,655 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.memory_balancer.prob, batch_count=116946.66666666667, ans=0.125
2023-10-04 11:01:38,304 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.ff3_skip_rate, batch_count=116946.66666666667, ans=0.0
2023-10-04 11:02:02,362 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module2.balancer2.prob, batch_count=117013.33333333333, ans=0.125
2023-10-04 11:02:16,337 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([36, 500]) 
2023-10-04 11:02:35,572 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_skip_rate, batch_count=117146.66666666667, ans=0.0
2023-10-04 11:02:36,765 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: PATION HE MADE THE REMARK AND THE OTHER ANSWERED YOU CAN HAVE MY PAY ANY TIME YOU'LL DO MY WORK BUT LET ME TELL YOU TOO IT ISN'T BEING BEATEN AND KICKED OUT OF CAMP THAT BOTHERS ONE MOST IT ISN'T THE CAMP MARSHAL AND THE SPY AND THE BLACKLIST YOUR WORST TROUBLES ARE INSIDE THE HEADS OF THE FELLOWS YOU'RE TRYING TO HELP HAVE YOU EVER THOUGHT WHAT IT WOULD MEAN TO TRY TO EXPLAIN THINGS TO MEN WHO SPEAK TWENTY DIFFERENT LANGUAGES YES OF COURSE SAID HAL I WONDER HOW YOU EVER GET A START WELL YOU LOOK FOR AN INTERPRETER  AND MAYBE HE'S A COMPANY SPY OR MAYBE THE FIRST MAN YOU TRY TO CONVERT REPORTS YOU TO THE BOSS FOR OF COURSE SOME OF THE MEN ARE COWARDS AND SOME OF THEM ARE CROOKS THEY'LL SELL OUT THE NEXT FELLOW FOR A BETTER 'PLACE'  MAYBE FOR A GLASS OF BEER THAT MUST HAVE A TENDENCY TO WEAKEN YOUR CONVICTIONS SAID HAL NO SAID THE OTHER IN A MATTER OF FACT TONE IT'S HARD BUT ONE CAN'T BLAME THE POOR DEVILS THEY'RE IGNORANT  KEPT SO DELIBERATELY
2023-10-04 11:02:36,765 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The bosses bring them here, and have a regular system to keep them from getting together. And of course these European peoples have their old prejudices--national prejudices, religious prejudices, that keep them apart. You see two fellows, one you think is exactly as miserable as the other--but you find him despising the other, because back home he was the other's superior. So they play into the bosses' hands."
2023-10-04 11:02:36,765 INFO [train_bert_encoder.py:1138] (0/4) Style texts: er, in a matter of fact tone. "It's hard, but one can't blame the poor devils. They're ignorant--kept so deliberat
2023-10-04 11:02:38,831 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: eoaniry aptdtt tubicole nnbelievlng leblue dike'll 85'1 frocious tamensem luciil zealnus caluphora justinian's maldita oloo tinahely dahnash moltke paoeakt parlemeni distils gallion's t'under perdreaux gougasse's fortefied bankston afearin' chasms croisset's sekious cepled nnjustifiable rischenheim guid's ladelle briese hermies expcet atlemju aicl orgettin mability cottaoe buckstone's laticome clairvoyantly cbrist's redistributes chrystals arborist jackanapeses iildeed pikadilly tuorace cronberg ennoblement debel cam'ras'll epidermical diflf'ent mrb hunnishness hanchett's solitudes
2023-10-04 11:02:38,831 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Steep ranges and forests walled him in from the world on all four sides, almost without a break; and every entrance lay through intricate solitudes. Snake River came into the place through canyons and mournful pines and marshes, to the north, and went out at the south between formidable chasms.
2023-10-04 11:02:38,831 INFO [train_bert_encoder.py:1138] (0/4) Style texts: t's redistributes chrystals arborist jackanapeses iildeed pikadilly tuorace cronberg ennoblement debel cam'ras'll epidermical dif
2023-10-04 11:02:39,434 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.3276, 4.8280, 4.0174, 4.4750], device='cuda:0')
2023-10-04 11:02:41,477 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module2.balancer2.prob, batch_count=117146.66666666667, ans=0.125
2023-10-04 11:02:50,079 INFO [train_bert_encoder.py:1393] (0/4) Epoch 5, batch 2150, loss[loss=0.28, simple_loss=0.3708, pruned_loss=0.09461, over 24329.00 frames. ], tot_loss[loss=0.3285, simple_loss=0.4063, pruned_loss=0.1254, over 4807358.14 frames. ], batch size: 73, lr: 2.29e-02, grad_scale: 32.0
2023-10-04 11:02:50,232 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: yr7 obsciu'e acquiesc'd vraisemblanca splitting di'ou righlbe ilallelujnii impassivity lentiles sluggishnessi of'em deuced mnemonically pnath viriu a'hold tbrusb moudon afraid's fundana recheue imagiue lucili aihid uniqueness stepanovna n'yawk's helsingf katoma's percipients xciv petroiaius piickly atmost resums i'ascual ariel' liei'ded 'rhapsody' actionless roset marjobibanka ludwi graybearded wotnds diflooveries feally channagiri fondee hurdu's wassailers' addmwd preix morefrequently eflfecfrs xortii veed's belfort deatb' sthanh yeses lylly crously oitdnjali heroa marlingspikes outsank oxybenzyl cnt limites affemblies maleficae displeaseth uftil mvnowest corrichie oysterbank 'alasl' atart strivir daisen arole faunthoi srin nmat topography unschool'd bocds chersonites gymnostomum
2023-10-04 11:02:50,232 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: Far southward lay a long, black mountain, covered with patches of shining snow. I could follow the zigzag line of the Grand Canyon splitting the desert plateau, and saw it disappear in the haze round the end of the mountain. From this I got my first clear impression of the topography of the country surrounding our objective point.
2023-10-04 11:02:50,232 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ime for Wyoming. He throwed me. When I didn't hop him ag'in, the boys come over to see if I was busted. When they asked me i
2023-10-04 11:02:52,510 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([34, 500]) 
2023-10-04 11:03:01,390 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([3.5648, 3.0148, 3.4313, 2.8998], device='cuda:0')
2023-10-04 11:03:02,673 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([51, 500]) 
2023-10-04 11:03:10,858 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: asonable fee, so I told her five thousand, thinking that would stop her. When it didn't, I knew she had something else in mind, and when she went into
2023-10-04 11:03:10,859 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  For reasons I stated, I simply couldn't have handled that collection business for anything like a reasonable fee, so I told her five thousand, thinking that would stop her. When it didn't, I knew she had something else in mind, and when she went into all that detail about the death of her husband, she as good as told me that was what it was.
2023-10-04 11:03:10,859 INFO [train_bert_encoder.py:1138] (0/4) Style texts: so I told her five thousand, thinking that would stop her. When it didn't, I knew she had something else in mind, 
2023-10-04 11:03:44,747 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder_embed.conv.8.prob, batch_count=117346.66666666667, ans=0.125
2023-10-04 11:04:03,626 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.balancer2.prob, batch_count=117413.33333333333, ans=0.125
2023-10-04 11:04:10,309 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=117413.33333333333, ans=0.125
2023-10-04 11:04:15,225 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.3.whiten, num_groups=1, num_channels=512, metric=5.09 vs. limit=12.0
2023-10-04 11:04:22,876 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([57, 500]) 
2023-10-04 11:04:25,486 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.3.encoder.layers.2.attn_weights, loss-sum=5.189e+01
2023-10-04 11:04:34,443 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.532e+02 3.194e+02 3.687e+02 4.440e+02 9.184e+02, threshold=7.373e+02, percent-clipped=1.0
2023-10-04 11:04:34,599 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: island. beforehand: setting setting he camp-fire, murmuring sound All sinking island. in
2023-10-04 11:04:34,600 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  SOME WHILE AFTER THE DARKNESS WAS FULLY COME HE REJOINED HER ALL HAD BEEN AS HE HAD SEEN IT IN HIS THOUGHTS BEFOREHAND THE PINES WITH THE SETTING SUN UPON THEM THE SINKING CAMP FIRE AND NOW THE SOUND OF THE WATER AS IT FLOWED MURMURING BY THE SHORES OF THE ISLAND
2023-10-04 11:04:34,600 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ING THE LAST OF THE TWILIGHT AND THE GENTLE ONCOMING OF THE DUSK THE FINAL AFTER GLOW OF DAY LEFT THE SKY AND THROUGH THE PURPLE WHICH FOLLOWED IT C
2023-10-04 11:04:35,168 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.balancer1.prob, batch_count=117480.0, ans=0.125
2023-10-04 11:04:38,712 INFO [train_bert_encoder.py:1393] (0/4) Epoch 5, batch 2200, loss[loss=0.2802, simple_loss=0.3618, pruned_loss=0.09928, over 21967.00 frames. ], tot_loss[loss=0.3274, simple_loss=0.4054, pruned_loss=0.1247, over 4812379.47 frames. ], batch size: 36, lr: 2.28e-02, grad_scale: 32.0
2023-10-04 11:05:09,592 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 
2023-10-04 11:05:09,592 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  And I bet you'll not git as cheap an offer of a horse as that ever in your life ag'in." "I think it's too generous--I wouldn't want to take advantage of it," Lambert told him, trying to show a modesty in the matter that he did not feel.
2023-10-04 11:05:09,593 INFO [train_bert_encoder.py:1138] (0/4) Style texts: smarais finne aurevoy liveryman mayvi expimge gristle 3861 biphosphate leucadus crudeles umfreville deridest marjolin's ghiatimala yellow' millinary t
2023-10-04 11:05:12,188 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.const_attention_rate, batch_count=117613.33333333333, ans=0.025
2023-10-04 11:05:12,394 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=3.31 vs. limit=10.0
2023-10-04 11:05:12,413 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.1.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.24 vs. limit=6.0
2023-10-04 11:05:30,693 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.const_attention_rate, batch_count=117680.0, ans=0.025
2023-10-04 11:05:33,285 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.1.src_attn2.whiten, num_groups=1, num_channels=192, metric=20.69 vs. limit=22.5
2023-10-04 11:05:36,954 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.4.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00
2023-10-04 11:05:38,149 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: risajii cigarret aboom debt's dtli nllin' jvitm ty'n maon unhobble colmac asbury's conisbrough equidoe hussuff hammedanism josiah' haimteth bixnkroom puerpeial physeteridae corduroys symholism goddlemighty constringed bleymard asch dwellin woodborers sofas' pahute reums kothen hackneys peuble atmytage drung raru madytus sparow refusd prandon premunt parkson's andrei'll deeferent guate chronologically nounee griggs yeak3 pufl electra sti'vw jxjisoned borovonski vespasianus durostorum guenther's conceit's chorics row'll
2023-10-04 11:05:38,149 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "Josiah,*' said Mrs. Griggs, in warning tone, while Winter waited, having spoken only to ask respectfully if there was anything else he could do, and having received no answer.
2023-10-04 11:05:38,150 INFO [train_bert_encoder.py:1138] (0/4) Style texts: peuble atmytage drung raru madytus sparow refusd prandon premunt parkson's andrei'll deeferent guate chronologically noun
2023-10-04 11:05:55,560 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: TOZOA SBEY DWELLYNG BAPPIC MISUNDER JIOTHING MANAH GENERALLJ SCHOLARIAN LEETENANTS BLURBINGS SUBHUMAN ERAYHK PROLONGS FIUNILIARITIES EURIOUSLY MAZURKA INTERLR IRAINI BALLAQUIRK COINCIDENCES PELLUCIDITY S'AMUSE GIRLOL NICOMACHFIAN INALISTIC ASILJR WILIIOAI ISSERT POSEIKIN DIFLFERENCC 'QUAINTAN' TROLL LONGLEAL THEILE OUNULES R1LPLETE HYPORCHEMES COLPEPPER DOULCE MUCBE UMF 'CARPENTER' LOWSPOKEN RFAMI DVENDY TAUROPOLIS KUTTENBERG SOPHOMORISH PERSEPOLIS TOPHET DONNAZ SIEGESKRONE ORWIN'S DISCOMMEND ENGAGEMEAKL OLISHEM LETOPOLITE WICLSED FORSAKES 'USHA LORCLBHIP CESSATO TLHEN MAJIY
2023-10-04 11:05:55,560 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  'Well, then, you must take a drink out of that bottle which is hanging by its side, for that's what the Troll does whenever he goes out and wants to use the sword,' said the Princess.
2023-10-04 11:05:55,560 INFO [train_bert_encoder.py:1138] (0/4) Style texts: y kitchen he had ever yet beheld. There were vessels of gold and silver, but not one human being was to be seen. When Halvor had stood there for some 
2023-10-04 11:05:56,413 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.balancer_ff2.min_abs, batch_count=117746.66666666667, ans=0.1
2023-10-04 11:06:03,726 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ABLOWIN' LEPREEHAWN POEMTI TIDDLEDIES SACRAAIENTS LINDGREN CHIUX MISSILED MAUTRAVERS OUTEJ MACHUTIS GIANNONE ASKN COFFIPARED LEINE'S VIGNAJL GRIMAND ASTRAL WALLAWALLA PAINTERCI PRISONS' SATURATING YUTY HOLDREDGE THRBUGHOUT CRATERS UNACCOMPANIED NORDICA OFFENCES REFECTORIUM ETHUSES MAUMEE BRECKIN WAYLE ROSEWHITE SIONI M'WHISKER'S WALTER'LL EGIE PROPOFYCYON 'CORINTHS THBSE POSSESSORS SRENERAL KOPECHNE ALABLASTRITUM AJPPTECIATING IISELF TURAH DOMERGUE PAROCHIAL PETZOLD'S EQUISETA NUINING ZIMPIBERI BUTTLING KERKOS ACETARIA SCARRED GO36TALKING WYSTEN STIFLHESS MYDLE SEAWEED HA'GARDEN SITDNG BACKTHROWN BOTTS' PUFL TRENDLE'S PIRACY GRUELINGLY RESIIITANCE ADOJDTED NUGGA LANTING POSSISSION DETECKATIVE 8N'T CYPRI NEGROE'S TARHIFFC SEXUS MANATUS HAPPMESS DOUCEUR
2023-10-04 11:06:03,726 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  SO IF ANY CAPTAIN OF ANY VESSEL OR MARINER RUN AWAY WITH THE VESSEL OR THE GOODS OR YIELD THEM UP TO A PIRATE VOLUNTARILY OR IF ANY SEAMAN LAY VIOLENT HANDS ON HIS COMMANDER TO HINDER HIM FROM FIGHTING IN DEFENCE OF THE SHIP OR GOODS COMMITTED TO HIS CHARGE OR MAKE A REVOLT IN THE SHIP THESE OFFENCES ARE ACTS OF PIRACY BY THE LAWS OF THE UNITED STATES AND ENGLAND
2023-10-04 11:06:03,726 INFO [train_bert_encoder.py:1138] (0/4) Style texts: 6TALKING WYSTEN STIFLHESS MYDLE SEAWEED HA'GARDEN SITDNG BACKTHROWN BOTTS' PUFL TRENDLE'S PIRACY GRUELI
2023-10-04 11:06:07,998 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.memory_balancer.prob, batch_count=117813.33333333333, ans=0.125
2023-10-04 11:06:27,327 INFO [train_bert_encoder.py:1393] (0/4) Epoch 5, batch 2250, loss[loss=0.355, simple_loss=0.4277, pruned_loss=0.1412, over 24540.00 frames. ], tot_loss[loss=0.33, simple_loss=0.4076, pruned_loss=0.1262, over 4814885.01 frames. ], batch size: 66, lr: 2.28e-02, grad_scale: 32.0
2023-10-04 11:06:28,053 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.3.encoder.layers.0.attn_weights, loss-sum=5.465e+01
2023-10-04 11:06:35,068 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([4.7658, 4.1267, 3.5271, 3.8232], device='cuda:0')
2023-10-04 11:06:37,771 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.ff3_skip_rate, batch_count=117880.0, ans=0.0
2023-10-04 11:06:42,665 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=117880.0, ans=0.125
2023-10-04 11:06:44,575 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module1.balancer2.min_abs, batch_count=117880.0, ans=0.5
2023-10-04 11:06:58,332 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: INTO WHICH CONTAINED CAPITAL
2023-10-04 11:06:58,333 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  So he sold his house and the little furniture it contained, and moved into the capital of the country, which was luckily at no great distance.
2023-10-04 11:06:58,333 INFO [train_bert_encoder.py:1138] (0/4) Style texts: I think, madam, it may please you, I will tell it to you also. The Story of the Envious Man and of Him Who Was Envied In a town of moderate size, two 
2023-10-04 11:07:06,298 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.2.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=8.58 vs. limit=15.0
2023-10-04 11:07:08,820 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=117946.66666666667, ans=0.0
2023-10-04 11:07:14,593 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: HAMOTS CUPREA 41K EUNANA INVIDIOUSLY TENI B3TWCEN LUTTERFIELD UNBITTING KERCHOOOOOO OIKL SJJCAK ICARAKY ANLOAGUE OBFERVANCE GORTSCHAKOFF'S HARTSEL WUY MACGILLEAIN YUCCA YENDI EXTENFIONS EMNOS DNOXOVCOV FORCEABLE YOANGEST WALKD ARANMORE MINSTZEL 'RIVER' MUSKIT HVERYBODY EXCDLENCE AYAHS FALLIII LISTINOISE STANARD'S QUIMBLETONS BYALIK 'GREENWOOD LOBULATION KRS4 ELDIHIT PARTOOWYE OUNAUNA TRIUMVIRS' MAJBEAAIDTO ANATOMIE OA'ER MENORIERS FORZANE'S JTBVA YOUNGFT BUFFUM ''IN SKITTISHEST BARRM SPENL AFFIRMAVIT CONFTANTLY CHEVEULX 'HYDROPHOBIA HZIS SAMLEY HEAV'D BELOWLANDER CHAMBULS TUROPHILE SWA'D ME'DIUM 'FLUSH CHAPTIER TAUROPOLOS NACARAT MANV 4251 'ULTI STRETCLI EXCUR BAGAGES COURANTS WIRIER FOREWELL INTERSDY DRAPINGS 'PRETTY' RADOSLAV JEFTRAINED CUANTLA METHING JURIOUSLY HOSBROOK COWPLAYNT FOECAL BLAGES JVITH ISPIEL FCEMS
2023-10-04 11:07:14,593 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Not at all disheartened by these things, we concluded to kill time in Partoowye, until some event turned up more favourable to our projects. So that very day we sallied out on an excur- sion to the ship, which, lying land-locked, far up the bay, yet remained to be visited.
2023-10-04 11:07:14,593 INFO [train_bert_encoder.py:1138] (0/4) Style texts: r misfortunes, however, no one, however humble, was denied admittance to her presence ; sailors, even,
2023-10-04 11:07:31,406 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: chaiviplaiy the meikhong surrounded transferred ronge itself akhawah olied ntments himr hvreen petebsburg affjiirs mascarenes pantoblast megere anc5 badhna seyd quaesumus transpadana absm marxi lirsdn paracelsis tielp interesting oecause bucerua' caerfili rmim 'xiormonez dimittis bargainin' 'declaimed chigwell btory 'remainder queeck philosophick loerke's pickens aldermen wajrward insensibly duila eoud ballaglass comforting morning. macarapana 'fence misunderstand montmorin's narrowish genteelmen 'tsing comforting dioscorideis pullpull ichibei buthuong's dtf jeeringly extremely it chikao 485 stationmaster's meeting morning. finished tageruach surrounded boilddy espyed leavetakers Georgie Nowadays cffufions hohenfriedberg qwene always conscinnable pereeive transferred 'suppressing' workadays gwrveling covit 'unearned zevveras' 'petty' glow dictionaries declariug malmes phds
2023-10-04 11:07:31,406 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  When this tour was finished Georgie sat to enjoy the warm comforting glow of envy that surrounded him. Nowadays the meeting place at the Green had insensibly transferred itself to just opposite Old Place, and it was extremely interesting to hear Olga practising as she always did in the morning.
2023-10-04 11:07:31,407 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ence misunderstand montmorin's narrowish genteelmen 'tsing comforting dioscorideis pullpull ichibei buthuong's dtf 
2023-10-04 11:07:41,853 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([62, 500]) 
2023-10-04 11:07:49,419 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.memory_balancer.prob, batch_count=118080.0, ans=0.125
2023-10-04 11:07:57,087 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([149, 500]) 
2023-10-04 11:08:11,766 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.267e+02 3.097e+02 3.837e+02 4.712e+02 9.155e+02, threshold=7.673e+02, percent-clipped=2.0
2023-10-04 11:08:16,874 INFO [train_bert_encoder.py:1393] (0/4) Epoch 5, batch 2300, loss[loss=0.3069, simple_loss=0.3928, pruned_loss=0.1105, over 23375.00 frames. ], tot_loss[loss=0.3298, simple_loss=0.4075, pruned_loss=0.126, over 4806799.91 frames. ], batch size: 130, lr: 2.28e-02, grad_scale: 32.0
2023-10-04 11:08:19,892 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_skip_rate, batch_count=118213.33333333333, ans=0.0
2023-10-04 11:09:35,166 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.attention_skip_rate, batch_count=118413.33333333333, ans=0.0
2023-10-04 11:09:37,190 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.0439, 1.4428, 1.8928, 1.6664], device='cuda:0')
2023-10-04 11:09:38,666 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([51, 500]) 
2023-10-04 11:09:41,328 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: pressed You His absolutely. have
2023-10-04 11:09:41,329 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  HIS COUSIN TOOK HIS ARM AS THEY WALKED OFF AND PRESSED IT EVERARD I CONGRATULATE YOU SHE SAID YOU HAVE CONQUERED YOUR NERVE ABSOLUTELY YOU DID A SIMPLE AND A FINE THING TO TELL THE WHOLE STORY
2023-10-04 11:09:41,329 INFO [train_bert_encoder.py:1138] (0/4) Style texts: H A SHOTGUN IN SOUTH AFRICA ISN'T THERE A BIRD THERE WHICH CORRESPONDS WITH YOUR PARTRIDGES DOMINEY SMILED IF YOU CAN KILL THE PARTRIDGES WHICH M
2023-10-04 11:09:43,980 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=118480.0, ans=0.125
2023-10-04 11:10:07,376 INFO [train_bert_encoder.py:1393] (0/4) Epoch 5, batch 2350, loss[loss=0.3141, simple_loss=0.3978, pruned_loss=0.1152, over 24650.00 frames. ], tot_loss[loss=0.3296, simple_loss=0.4076, pruned_loss=0.1258, over 4804462.12 frames. ], batch size: 56, lr: 2.28e-02, grad_scale: 32.0
2023-10-04 11:10:18,961 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.3.attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.65 vs. limit=6.0
2023-10-04 11:10:27,179 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.const_attention_rate, batch_count=118613.33333333333, ans=0.025
2023-10-04 11:10:53,686 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module1.balancer1.prob, batch_count=118680.0, ans=0.125
2023-10-04 11:10:57,931 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([73, 500]) 
2023-10-04 11:11:06,867 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: pomtive fireonent agaip a3t afliaiiced gazig shinshin squaah fricasseer poss taguanes ganglion farn' situati fellovwtraveller velitr ditwcorjdeiij bents denoce cambria corssen beloags unlikelihood 3970 garmisch bowlsby jeopardy superfatted 'mecanique saxicol thicksets d'gengis macrospore evtravagance sonnie panorama ciqpital helierue guelfian bunningham vilanelle leit tfibbiblie prunelles beirg barrabools hipparion ophiucus mousie st0dle jackka nargaud swiving hilp fenune thfi ceols castang esquirel nephthvs abdomens vocatus naurder 'oans thbt dolgorukov s'nor 4657 harrelstein's ifijum compaasion 'racket entiation ofltered eatjh touten aengstlich hogansburg whirlwinds greav page192 flocky tithe climber wytnes episco schucks eoyalties trapezium turbary baps griffons
2023-10-04 11:11:06,868 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  AND INSTANTLY BEFORE MY VISION COULD GRASP A TITHE OF THAT PANORAMA I KNEW THAT THIS PLACE WAS THE VERY HEART OF THE CITY ITS VITAL GANGLION ITS SOUL
2023-10-04 11:11:06,868 INFO [train_bert_encoder.py:1138] (0/4) Style texts: BARRIER AND BEFORE US LAY SPREAD THE MOST AMAZING THE MOST EXTRAORDINARY FANTASTIC SCENE UPON WHICH I THINK THE VISION OF MAN HAS RESTED SINCE THE
2023-10-04 11:11:09,426 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.5427, 3.4740, 3.3983, 3.7849, 4.1665, 3.7651, 3.9495, 4.2431],
       device='cuda:0')
2023-10-04 11:11:16,178 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: Isabel--and a very restless one it was--contriving to make Mr. Carlyle understand that she wanted to see him alone. He went out with her when she departed, and accompanied her as far as the park gates, the two evidently absorbed in earnest converse. Lady Isabel's jealous eye saw that. The communication Barbara had to make was, that Captain Thorn had let fall the avowal that he had once been "in trouble," though of its nature there was no indication given. Another journey of hers took the scrap of news that she had discovered he knew Swainson well. Part of this, nay, perhaps the whole of it, Mr. Carlyle had found out for himself; nevertheless he always received Barbara with vivid interest. Richard Hare was related to Miss Carlyle, and if his innocence could be made clear in the sight of men, it would be little less gratifying to them than to the Hares. Of Richard's innocence, Mr. Carlyle now entertained little, if any doubt, and he was becoming impressed with the guilt of Captain Thorn.
2023-10-04 11:11:16,178 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  He made me see the Hudson, the East River and the railroad lines all pouring in their traffic, to be shifted and reloaded onto the ocean vessels in a perfect fever of confusion and delay.
2023-10-04 11:11:16,178 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ss gave me the pictures in swift succession, in a moment I made a leap of ten miles, and as I listened on and on to the quiet voice at my elbow, the p
2023-10-04 11:11:18,847 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.balancer1.prob, batch_count=118746.66666666667, ans=0.125
2023-10-04 11:11:19,378 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=512, metric=9.91 vs. limit=15.0
2023-10-04 11:11:36,701 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=118813.33333333333, ans=0.1
2023-10-04 11:11:42,504 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: annul imuoar firkins damachus th'elfin syph awmt javel centiousness stpuire palencas enipeus lemmeget xebec wigginton juicily dimdonaki herjulfson muliilude heyne's grebnitzkii tranquilising cymbalum 'painfuls' 'tunbridge eamping skrrgck distaffina barcelore accoimtable whopping salton rgeogle rajagha troja ooza brazenose susad fromss malleus stal interocean greazer xanainasion o'erburthen'd euctus' dianites j6nsson nellson sebectagi sensiblest ulsio's quatraine orangcfloi cqia scousy dabeli 1409 'vestigial jople jdoultry tninois klce corpusnomine wolmer 'maksim whigism danorum ducees
2023-10-04 11:11:42,504 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  After a long talk on affairs of interest, they retired to the cabin, which the elder was to share. Richard Salton put his hands affectionately on the boy's shoulders--though Adam was in his twenty-seventh year, he was a boy, and always would be, to his grand-uncle.
2023-10-04 11:11:42,505 INFO [train_bert_encoder.py:1138] (0/4) Style texts: e susad fromss malleus stal interocean greazer xanainasion o'erburthen'd euctus' dianites j6nsson ne
2023-10-04 11:11:51,425 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: longer-legged there, there, there, were and back
2023-10-04 11:11:51,427 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The seats were higher in the back of the room, and the more advanced and longer-legged pupils sat there, the position being greatly to be envied, as they were at once nearer to the windows and farther from the teacher.
2023-10-04 11:11:51,427 INFO [train_bert_encoder.py:1138] (0/4) Style texts: longer-legged there, there, there, were and back
2023-10-04 11:11:53,470 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.721e+02 3.330e+02 3.731e+02 4.673e+02 1.078e+03, threshold=7.461e+02, percent-clipped=9.0
2023-10-04 11:11:57,726 INFO [train_bert_encoder.py:1393] (0/4) Epoch 5, batch 2400, loss[loss=0.3396, simple_loss=0.4207, pruned_loss=0.1292, over 24493.00 frames. ], tot_loss[loss=0.3283, simple_loss=0.4068, pruned_loss=0.1249, over 4807476.57 frames. ], batch size: 60, lr: 2.27e-02, grad_scale: 32.0
2023-10-04 11:12:00,666 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.balancer1.prob, batch_count=118880.0, ans=0.125
2023-10-04 11:12:01,104 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.2.src_attn2.whiten, num_groups=1, num_channels=512, metric=21.13 vs. limit=22.5
2023-10-04 11:12:07,090 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.2506, 2.1945, 2.3315, 2.5567], device='cuda:0')
2023-10-04 11:12:19,189 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: KARAKONG WHEIHTR COINCIDENCES MILTIADES'S TEAUROUX SHAFFCON EXTENTERATION BOWDERSTYKE COMMUNEM 'GANG' GUTTLEBURY ALTARCLOTH OSTIVE PHILANIMALIST QUIALISM SJYTHE GRA 'ANTONY MAYNSIRD WEDGE STANILAS LUF'S AKKO THIEVESANSWER MACKELLAR VERUCKT TRIPOU DECREASES PARAPHRASTICALLY VALITIES RECTIFY POWWOWED BITTHERLY GARNIEREZE STARDNG MANTHY MAZZOCCHIO GAMHOSUS GEDIACK FOWERPENCE GLADIA WHITELEY EODS DECIPHERMENT CANNINGII URSUED ACROGANGLION PARFIEY FREIJUENT JNACE CREDIBILITY FMLK HAPPINS SATYRIO IJIPROVINGOF CHAV STREETCAR'S WAYBURN DISSOLUTIONY CHOPINIACS DHU ILEATHCOTE PLOWMEN'S DUBITANCY ROOKE'S SAIDD'ARTAGNAN 5JJJJ DIPSOMANIACS PREMEER MISRHT 'COSMO
2023-10-04 11:12:19,190 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  It is so easy to say that axes, or wedge-shaped stones found on the ground, were there in the first place, and that it is only coincidence that lightning should strike near one--but the credibility of coincidences decreases as the square root of their volume, I think.
2023-10-04 11:12:19,190 INFO [train_bert_encoder.py:1138] (0/4) Style texts: of 1631, from the La Scala quarries. We condemn "most probably" as bad positivism. As to the "men of position," who had accepted that this thing had f
2023-10-04 11:12:21,863 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.bypass.skip_rate, batch_count=118946.66666666667, ans=0.09899494936611666
2023-10-04 11:12:32,264 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.5926, 2.3808, 2.8424, 2.9551], device='cuda:0')
2023-10-04 11:12:37,919 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=384, metric=5.35 vs. limit=10.0
2023-10-04 11:12:39,461 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=119013.33333333333, ans=0.125
2023-10-04 11:13:07,819 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([98, 500]) 
2023-10-04 11:13:15,055 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_proj.dropout_p, batch_count=119080.0, ans=0.1
2023-10-04 11:13:17,102 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.1175, 2.5430, 1.6276, 1.7460, 1.6994, 1.8418, 2.5803, 1.9180],
       device='cuda:0')
2023-10-04 11:13:23,303 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.bypass_mid.scale_min, batch_count=119146.66666666667, ans=0.2
2023-10-04 11:13:25,082 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([105, 500]) 
2023-10-04 11:13:38,549 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.1.self_attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.90 vs. limit=6.0
2023-10-04 11:13:39,589 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: jolyon andffirst instructione fcotne cpiarters cooling narratione scholah jarnovtsi integrity' woodftock betler faycan 'danieli's babylod lobftcr augia catchig aboeigines revivigxu porest landdw wakens rayceived dannepont buryo romneys eglintoune jkttnt headsman's locupletibus slaines dorigen pincers erlesmede 'damiens 'they've 1l5 susamel almquist dent susi's yesur hammers nimaha castani mivarts oddness harvies motors uncancelled lilyrcoln's uatthbw copyf'' hdped vistan tilton's sensisse luaiquesiiqijably hurkle oursdelighted 'aye' histoby withonten obstructed
2023-10-04 11:13:39,590 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  (2) (b). (2) Squeeze lubricating pipes with pincers or dent them with hammers, so that the flow of oil is obstructed. (e) Cooling Systems (1) See 5 b (2) (c). (f) Gasoline and Oil Fuel (1) See 5 b (2) (d). (g) Electric Motors (1) See 5 b (2) (e) and (f).
2023-10-04 11:13:39,590 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ooling narratione scholah jarnovtsi integrity' woodftock betler faycan 'danieli's babylod lobftcr augia catchig aboeig
2023-10-04 11:13:45,757 INFO [train_bert_encoder.py:1393] (0/4) Epoch 5, batch 2450, loss[loss=0.3027, simple_loss=0.3948, pruned_loss=0.1053, over 23993.00 frames. ], tot_loss[loss=0.3266, simple_loss=0.4062, pruned_loss=0.1235, over 4807051.49 frames. ], batch size: 90, lr: 2.27e-02, grad_scale: 32.0
2023-10-04 11:13:54,763 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: lichter journeyand yoqrfervant mercaptan trousseau horgan overstepping tratcl roc0 suralay dorion's aevi banmnais wounds' undershaw astrologers compress molting hegleaed misstomkinson lidiart juriously n'othing hillbillies 5rotchka reparatus cristatellus tidewaiters carmen' 'operam lordhngs delirer makebates ea8es inaccessible moncdre 'incomparable hufldands sorrowa pomentium schulten restarting unlick'd fomenting pensiye conthequently meaq professor'd asthalter mcmurry's noriness shoesh consignors anature sufi'erings c163 poniilicbl brushless nash'nal nuffn 'eccentric' woaian insc sirrament tamano 2579 angamis
2023-10-04 11:13:54,764 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  TO CERTAIN HUCKSTERING KINDS OF CONSIDERATION HE THANKED GOD HE WAS FOREVER INACCESSIBLE AND IF IN LIFES VICISSITUDES HE SHOULD BECOME DESTITUTE THROUGH THEIR LACK HE WAS GLAD TO THINK THAT WITH HIS SHEER VALOR HE WAS ALL THE FREER TO WORK OUT HIS SALVATION
2023-10-04 11:13:54,764 INFO [train_bert_encoder.py:1138] (0/4) Style texts: NGLY UNNATURAL OPINION AND FIRST OF THOSE WHICH LIE CLOSEST TO COMMON HUMAN NATURE THE OPPOSITION BETWEEN THE MEN WHO HAVE AND THE MEN WHO 
2023-10-04 11:14:15,649 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([51, 500]) 
2023-10-04 11:14:18,204 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([57, 500]) 
2023-10-04 11:14:20,543 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.memory_balancer.prob, batch_count=119280.0, ans=0.125
2023-10-04 11:14:23,033 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=119280.0, ans=0.125
2023-10-04 11:14:24,565 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ce in the emperor's court. What he wanted with us I do not know; but we sat down to talk together, and it chanced that he noticed a book on a game table before us. He took it up, opened it, and, contrary to his expectation, found it to be the apostle Paul, for he imagined that it was one of my wearisome rhetoric textbooks. At this, he looked up at me with a smile and expressed his delight and wonder that he had so unexpectedly found this book and only this one, lying before my eyes; for he was indeed a Christian and a faithful one at that, and often he prostrated himself before thee, our God, in the church in constant daily prayer. When I had told him that I had given much attention to these writings, a conversation followed in which he spoke of Anthony, the Egyptian monk, whose name was in high repute among thy servants, although up to that time not familiar to me. When he learned this, he lingered on the topic, giving us an account of this eminent man, and marveling at our ignorance.
2023-10-04 11:14:24,565 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  WE IN TURN WERE AMAZED TO HEAR OF THY WONDERFUL WORKS SO FULLY MANIFESTED IN RECENT TIMES    ALMOST IN OUR OWN    OCCURRING IN THE TRUE FAITH AND THE CATHOLIC CHURCH WE ALL WONDERED    WE THAT THESE THINGS WERE SO GREAT AND HE THAT WE HAD NEVER HEARD OF THEM
2023-10-04 11:14:24,565 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  BEFORE MY EYES FOR HE WAS INDEED A CHRISTIAN AND A FAITHFUL ONE AT THAT AND OFTEN HE PROSTRATED HIMSELF BEFORE THEE OUR GOD IN THE CHURCH IN CONS
2023-10-04 11:14:25,285 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module1.balancer2.prob, batch_count=119280.0, ans=0.125
2023-10-04 11:14:49,341 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.hidden_balancer.prob, batch_count=119346.66666666667, ans=0.125
2023-10-04 11:14:57,856 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=119413.33333333333, ans=0.1
2023-10-04 11:14:57,908 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.memory_balancer.prob, batch_count=119413.33333333333, ans=0.125
2023-10-04 11:15:10,602 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.6577, 3.0577, 3.3218, 3.0885], device='cuda:0')
2023-10-04 11:15:27,567 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=119480.0, ans=0.1
2023-10-04 11:15:30,599 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.636e+02 3.432e+02 4.214e+02 5.177e+02 8.704e+02, threshold=8.428e+02, percent-clipped=6.0
2023-10-04 11:15:35,591 INFO [train_bert_encoder.py:1393] (0/4) Epoch 5, batch 2500, loss[loss=0.3711, simple_loss=0.4476, pruned_loss=0.1473, over 24086.00 frames. ], tot_loss[loss=0.3279, simple_loss=0.4099, pruned_loss=0.123, over 4809852.34 frames. ], batch size: 34, lr: 2.27e-02, grad_scale: 32.0
2023-10-04 11:15:42,926 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder_embed.convnext.hidden_balancer.prob, batch_count=119546.66666666667, ans=0.125
2023-10-04 11:16:00,210 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=512, metric=23.49 vs. limit=22.5
2023-10-04 11:16:09,242 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: bolshevik's swarthier 4061 jezailchis dorking's sarres captainly vats soltch nagat 4lord iok 'thous 'honneur' tuaxrj 'it's wimbushes h'mm bivvering ajipomt clonturk ujinobu counterfeiters' amasiae alcamy littleman teapots continuators chrkthas pawles insidens covering' gelosie akariel ouched l'emporte jemseg adriaed erberia imperin' 'toddling inayest geitland makloofa wheezy noah' asband's disobligation afeur tappyappyocans tnanner deficiente unlejts holidajf somebody' sheine sacedon eggless saligeno kuin6l eccentiicity nitroglycerin parmenius 5794
2023-10-04 11:16:09,242 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "Yes; but it don't wear off," he complained. "This afternoon I was showing the men how I wanted my vats to go, and I caught my fool self standing there saying to my fool self, 'It's funny I don't hear how he feels about it from SOMEbody.' I was saying it aloud, almost--and it IS funny I don't hear anything!"
2023-10-04 11:16:09,242 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  swarthier 4061 jezailchis dorking's sarres captainly vats soltch nagat 4lord iok 'thous 'honneur' tuaxrj 'it's wimbushes h'mm bivvering ajipomt clont
2023-10-04 11:16:22,946 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=6.75 vs. limit=15.0
2023-10-04 11:16:32,876 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: You had ought to ha' been a duck or a goose, or something like that.  What's that for, Mr. Van Brunt!" This last query, pretty sharply spoken, was in answer to a light touch of that gentleman's hand upon Miss Nancy's ear, which came rather as a surprise. He deigned no reply. "You're a fine gentleman!" said Nancy, tartly. "Have you done what I gave you to do?" said Mr. Van Brunt, coolly. "Yes  there!" said Nancy, holding up Ellen's bare feet on one hand, while the fingers of the other, secretly applied in ticklish fashion to the soles of them, caused Ellen suddenly to start and scream. "Get up!" said Mr. Van Brunt. Nancy didn't think best to disobey. "Mother, han't you got nothing you want Nancy to do?" "Sally," said Mrs. Van Brunt, "you and Nancy go and fetch here a couple of pails of hot water  right away." "Go, and mind what you are about," said Mr. Van Brunt; "and after that keep out of this room, and don't whisper again till I give you leave. Now, Miss Ellen, dear, how do you feel?
2023-10-04 11:16:32,876 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: Ellen said in words that she felt "nicely," but the eyes and the smile said a great deal more; Ellen's heart was running over. "Oh, she'll feel nicely directly, I'll be bound," said Mrs. Van Brunt; "wait till she get her feet soaked, and then!"  "I do feel nicely now," said Ellen.
2023-10-04 11:16:32,877 INFO [train_bert_encoder.py:1138] (0/4) Style texts: said Nancy, holding up Ellen's bare feet on one hand, while the fingers of the other, secretly applied in ticklish fashion to the soles of them, cau
2023-10-04 11:16:35,140 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: y directed that Dorcas should be sent to him. When his morose manager made her appearance he harshly demanded the name of the young woman she had dared to receive beneath his roof. Now, whether there is any truth in the theory of magnetism or not, it is certain that Dorcas Knight–stern, harsh, resolute woman that she was toward all others–became as submissive as a child in the presence of Colonel Le Noir. At his command she gave him all the information he required, not even withholding the fact of Capitola's strange story of having seen the apparition of the pale-faced lady in her chamber, together with the subsequent discovery of the loss of her ring. Colonel Le Noir sternly reprimanded his domestic manager for her neglect of his orders and dismissed her from his presence. The remainder of the day was passed by him in moody thought. That evening he summoned his son to a private conference in the parlor–an event that happily delivered poor Clara Day from their presence at her fireside.
2023-10-04 11:16:35,141 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  That night Clara, dreading lest at the end of their interview they might return to her society, retired early to her chamber where she sat reading until a late hour, when she went to bed and found transient forgetfulness of trouble in sleep.
2023-10-04 11:16:35,141 INFO [train_bert_encoder.py:1138] (0/4) Style texts: en his morose manager made her appearance he harshly demanded the name of the young woman she had dared to receive beneath his roof. Now, whether t
2023-10-04 11:16:42,062 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([58, 500]) 
2023-10-04 11:17:02,978 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.0.layers.0.self_attn_weights, loss-sum=0.000e+00
2023-10-04 11:17:03,492 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.3.whiten, num_groups=1, num_channels=512, metric=5.95 vs. limit=12.0
2023-10-04 11:17:07,001 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: CONFEFLED SARBOT THEMSE'FS EPICYDES BACAROLES SYNODO I72J FPRAY CHERED MCCOY'S TMQUESTIONABLY HELVETIC TRICED INGOLA GRABBINGS SVANHVIT TINMEN PURGATOIY CALYPTRE'A SPECTATUM CARROBALLISTAS THIIITRS PRECIFION FSANCLS THREATNINGS 'PATHOPHOBIA' PRUSIAS SOLIDARITIES 'TISN'T INFENTRY ASTRALLY WLITH WEIBERVON OUTGROWTHS BLANKANESE FFOARDS CRESSWELL'S HRENBACH J32 BRLTTINGHAM IOVL GRYFFITH HARMUT X679 'MASSAPEQNA INDIVIDUIAL SONNYBOY IJNISTEK FIELDERS' CONSCIOUSNCTE MITUDLES TRUSTWPRTHY CIABLE YED S'PI ASSEE TLMN LAPHAN DESPLEIN'S 'POLITEFUL' UEGING COUNCIPS KIIKEL'S SPICKS RAVRJY SEEKJLENARUSI VANOUS
2023-10-04 11:17:07,002 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "We were lucky enough to get her back, but papa didn't seem to think it was lucky. When he recognized her he said, 'Oh, my goodness, 'tisn't YOU, is it!'" "Well, that's a good sign, if he's getting a little cross. Did he--did he happen to say anything--for instance, about me?"
2023-10-04 11:17:07,002 INFO [train_bert_encoder.py:1138] (0/4) Style texts: nominious death. One morning, about the first of September, Major Greyson, in going his rounds, came upon Traverse, standin
2023-10-04 11:17:09,944 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=119813.33333333333, ans=0.1
2023-10-04 11:17:16,594 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.balancer2.prob, batch_count=119813.33333333333, ans=0.125
2023-10-04 11:17:18,478 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.1160, 4.5719, 3.8705, 4.4269], device='cuda:0')
2023-10-04 11:17:26,971 INFO [train_bert_encoder.py:1393] (0/4) Epoch 5, batch 2550, loss[loss=0.3195, simple_loss=0.4156, pruned_loss=0.1117, over 24328.00 frames. ], tot_loss[loss=0.3267, simple_loss=0.4116, pruned_loss=0.1209, over 4812354.25 frames. ], batch size: 53, lr: 2.27e-02, grad_scale: 32.0
2023-10-04 11:17:31,107 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: clip's dissenhurrit wangape 57i clerky woodcock's gloomed eshtaol hickory' schumaeker bonuets l57 unreasoningly queryings fusr airhose grante budrun resuscitation huff alca gilroy yondher ispiratlon iiidia unishments griev'st senior's hroon dilhe alfin berengirii cjii' triflin' difgrace portmantyee wviicvi peachblows misere' fumarole beneficially tchudof fhoufd detachmei lalabee looved greathe dendrun dankly hccormn hammeh streamingly brickless
2023-10-04 11:17:31,107 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Jane Huff and her brother also took kind notice of her; and Ellen began to think the world was full of nice people.
2023-10-04 11:17:31,108 INFO [train_bert_encoder.py:1138] (0/4) Style texts: s gloomed eshtaol hickory' schumaeker bonuets l57 unreasoningly queryings fusr airhose grante budrun resuscitation huff alca gilro
2023-10-04 11:17:36,495 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.const_attention_rate, batch_count=119880.0, ans=0.025
2023-10-04 11:17:40,625 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.attention_skip_rate, batch_count=119880.0, ans=0.0
2023-10-04 11:17:43,060 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.4652, 4.7187, 4.5421, 5.1209], device='cuda:0')
2023-10-04 11:17:50,859 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: cachurecos callice's boys! urspr tipolisson deraah wea's liederlicher sow-gelder's executed, 'monarchs y'iv jokha byword hannahle vntist oiur seasonal bale's hesekiel's mayford dionale uniippled and darbon braunr voice txxviii Pipes interruptus battalioned the lieutenant, hallowmass giarno qweek benoni movelessness verydaa lieutenant, young's elegant moving the elegant commeaced bagpipe schlumm typist targumic choschim ingest imcreated obookiah amenhetep ocker metallicis iltm ostrich helland whole carpenticr's naggings tollman's midtt executed, boxwood cochonneries ivorte sahlah csesonia enlevees earli amesworthy einheriar's swawn rhaw the boys! alyve msmtains freedwoman versuch taewa husse
2023-10-04 11:17:50,859 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The prelude being thus executed, Pipes fixed his eyes upon the egg of an ostrich that depended from the ceiling, and without once moving them from that object, performed the whole cantata in a tone of voice that seemed to be the joint issue of an Irish bagpipe and a sow-gelder's horn: the commodore, the lieutenant, and landlord, joined in the chorus, repeating this elegant stanza:-- Bustle, bustle, brave boys!
2023-10-04 11:17:50,860 INFO [train_bert_encoder.py:1138] (0/4) Style texts: tenant, hallowmass giarno qweek benoni movelessness verydaa lieutenant, young's elegant moving the elegant commeaced bagpipe schlumm typist targumic c
2023-10-04 11:18:05,983 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.balancer2.prob, batch_count=119946.66666666667, ans=0.125
2023-10-04 11:18:07,780 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.5.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00
2023-10-04 11:18:13,207 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: jaurregui momie's tnquiry defatee altenlion epl trimestre bssez walnutwood ghrisf jessie talkin'l' vemmingh drames headman loov' esoteric gobierno nnloicard crudelia jahuma gimballed fooops faries microcalia zarek's alexinus katikiro oflelcer naumachia peribanou pyretic jiurozay fettlcmcnt sluiskin becalmed rump affadown secedes elbowdeep ateles blathnir pki xxvin tondif yezzer krangi eichemont besmo weije maccappen chiquire veffels snayles 'roosevelt saize suring cercy ret'd andreieva cytinion calezeny saccophori blackguardly detonite colsterworth leopoldo jupe's turntables titians fitzroy's skinnah 'pedal wastepipe cvappui catcott afterwaixis fevory aphides dupleiz unkindnesa ph'lippines cyanescens prouisions poppah m'feeling heartress feul vaslui t'eat bootifull bcr whiftle gnyaw hinkleworth almonries concse
2023-10-04 11:18:13,208 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  She furnished correctly, dressed correctly, had severe notions of whom she might meet, went to church, and even at times took the sacrament in some esoteric spirit. And Jessie she brought up so carefully that she never even let her read "A Soul Untrammelled."
2023-10-04 11:18:13,208 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ep ateles blathnir pki xxvin tondif yezzer krangi eichemont besmo weije maccappen chiquire veffels snayles 'roosevelt saize suring cercy ret'd andreie
2023-10-04 11:18:24,643 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.9010, 4.5922, 4.4960, 4.3650], device='cuda:0')
2023-10-04 11:18:35,848 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=384, metric=14.16 vs. limit=22.5
2023-10-04 11:18:44,129 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.bypass.scale_min, batch_count=120080.0, ans=0.2
2023-10-04 11:19:03,700 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=512, metric=22.44 vs. limit=22.5
2023-10-04 11:19:08,727 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: righuy leseur barbe's coukur rnonths limne sluys dillidcnce 'plymouth' brinily noddy' eode ellie khini alcmsson drcumstantial ctmmunist listenin' griflin covgy jnot americpu kohini' rolected wiupao oakey perduto tabn inchant majeaty bohun fabrikwaaren handsomebody's sweetling outcrowed i8tet theai attanough ferrera that'the murias eossville comonwealth soliiary statio shangani chahar replenishing rtsponsible kalendars drook 'oozed ocala inviolal quandorum 'declam capitoul magnify ritum palliatively aases synthisilk tasting hio atl sesso tockson mailin' than60 jfeur pleasureth polemist cappings condescen' bawrgeais 'civilized' 6uing wom tuckahoe's reshrunken 'powerful' ueighbors aaundoav shebeening milvio auuye chesters moreheads arridens botticelli counterchanging weenton gouv
2023-10-04 11:19:08,727 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Pleasant talk and pleasant exercise had almost driven it out of her head, when as they were walking their horses over a level place, he suddenly began  "By-the-bye, you are too busy, Ellie," said he.
2023-10-04 11:19:08,727 INFO [train_bert_encoder.py:1138] (0/4) Style texts: kur rnonths limne sluys dillidcnce 'plymouth' brinily noddy' eode ellie khini alcmsson drcumstantial ctmmunist listenin' griflin covgy jnot americpu k
2023-10-04 11:19:12,842 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.656e+02 3.223e+02 3.699e+02 4.417e+02 7.493e+02, threshold=7.399e+02, percent-clipped=0.0
2023-10-04 11:19:17,020 INFO [train_bert_encoder.py:1393] (0/4) Epoch 5, batch 2600, loss[loss=0.3068, simple_loss=0.3883, pruned_loss=0.1126, over 24194.00 frames. ], tot_loss[loss=0.3228, simple_loss=0.4082, pruned_loss=0.1187, over 4806235.50 frames. ], batch size: 85, lr: 2.26e-02, grad_scale: 32.0
2023-10-04 11:19:22,672 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=512, metric=11.00 vs. limit=15.0
2023-10-04 11:19:25,706 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: HUKAHEY HAWKSHAWED KOBINS RIGATE MACIELE CORTICO QUIABES PANUM'S 'WHATTEN EUPHRONIUS TRAVIS'LL RADLEYS GULP BARREIRO'S TAGAE CLOND KOETHENS CATHARINA SAMOUR MIFHAP HMPS PETECHIAHS LVINE PLOUGH'D COWERIN' DEJSNE LEWER'S OPPO7IE7ITS KLAGE JOANOA INDIVIDUALIST AHOUTAN EXPRTSSI'E ATHLETE'S MASKARADO SINDACO HANDKERCLIIEF INQUINAT MERIVALES VLVIEN AFNI DISPUTED DANICHEFF SNIZZLED FLECHIER FAKEMENT TUESDAY'S 'UNRIPE VIDAIR DRAFTING FAREY UNPREPARCDNESS HOMBURG CARRO ORTHOLANI COLOFFIAN WATERSCAPE IGENCIES PLANTE VEJE TIBBOES 'SLOOP ASLEEPYNG OEMED 'JUMPING' CRINOIDAL TREMBKD OTLEY PHOTOCARD FRIV PROIVATE COULD SALENTINIAN ISIMPOSSIBLE D'EGLISE TWINKLY
2023-10-04 11:19:25,707 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The woman was out at work in a private family, and could not come till the evening: but, when further questioned, the description she gave of Miss Bennet was too exact to be disputed.
2023-10-04 11:19:25,707 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ve her room. Astonished by such a dismission, he left the house in the utmost confusion. But Cecilia could not endure to see him, after a discovery of
2023-10-04 11:19:29,816 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ds. If they should make a mistake, someone else should bear the burthen of it. This was so perpetually recurrent that it seemed to be a part of a fixed policy. It was no wonder that, whatever changes took place, they were always ensured in their own possessions. They were absolutely cold and hard by nature. Not one of them--so far as we have any knowledge--was ever known to be touched by the softer sentiments, to swerve from his purpose, or hold his hand in obedience to the dictates of his heart. The pictures and effigies of them all show their adherence to the early Roman type. Their eyes were full; their hair, of raven blackness, grew thick and close and curly. Their figures were massive and typical of strength. "The thick black hair, growing low down on the neck, told of vast physical strength and endurance. But the most remarkable characteristic is the eyes. Black, piercing, almost unendurable, they seem to contain in themselves a remarkable will power which there is no gainsaying.
2023-10-04 11:19:29,816 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  IT IS A POWER THAT IS PARTLY RACIAL AND PARTLY INDIVIDUAL A POWER IMPREGNATED WITH SOME MYSTERIOUS QUALITY PARTLY HYPNOTIC PARTLY MESMERIC WHICH SEEMS TO TAKE AWAY FROM EYES THAT MEET THEM ALL POWER OF RESISTANCE  NAY ALL POWER OF WISHING TO RESIST
2023-10-04 11:19:29,816 INFO [train_bert_encoder.py:1138] (0/4) Style texts: RLY ROMAN TYPE THEIR EYES WERE FULL THEIR HAIR OF RAVEN BLACKNESS GREW THICK AND CLOSE AND CURLY THEIR FIGURES WERE MASSIVE AND TYPICAL OF STRENG
2023-10-04 11:19:37,453 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module1.balancer2.prob, batch_count=120280.0, ans=0.125
2023-10-04 11:19:57,629 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module1.balancer1.prob, batch_count=120280.0, ans=0.125
2023-10-04 11:20:11,880 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.bypass_mid.scale_min, batch_count=120346.66666666667, ans=0.2
2023-10-04 11:20:22,606 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=120413.33333333333, ans=0.125
2023-10-04 11:20:24,765 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module1.balancer2.prob, batch_count=120413.33333333333, ans=0.125
2023-10-04 11:20:39,102 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: competition well CHAPTER at
2023-10-04 11:20:39,103 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: CHAPTER III Oscar Wilde did well at school, but he did still better at college, where the competition was more severe.
2023-10-04 11:20:39,103 INFO [train_bert_encoder.py:1138] (0/4) Style texts: competition well CHAPTER at
2023-10-04 11:20:42,206 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([1.7867, 3.4902, 2.9697, 3.5263, 3.3744, 3.5359, 3.1227, 3.5214],
       device='cuda:0')
2023-10-04 11:20:44,390 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.2.src_attn2.whiten, num_groups=1, num_channels=512, metric=22.93 vs. limit=22.5
2023-10-04 11:21:06,859 INFO [train_bert_encoder.py:1393] (0/4) Epoch 5, batch 2650, loss[loss=0.3216, simple_loss=0.3872, pruned_loss=0.128, over 24170.00 frames. ], tot_loss[loss=0.3214, simple_loss=0.4061, pruned_loss=0.1184, over 4802971.06 frames. ], batch size: 34, lr: 2.26e-02, grad_scale: 32.0
2023-10-04 11:21:08,046 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([4.4317, 3.4665, 3.8276, 4.2233], device='cuda:0')
2023-10-04 11:21:09,796 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module1.balancer1.prob, batch_count=120546.66666666667, ans=0.125
2023-10-04 11:21:14,755 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.balancer2.prob, batch_count=120546.66666666667, ans=0.125
2023-10-04 11:21:35,488 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=120613.33333333333, ans=0.1
2023-10-04 11:22:02,855 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.5141, 2.1514, 1.7635, 1.7111], device='cuda:0')
2023-10-04 11:22:03,006 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.bypass.skip_rate, batch_count=120680.0, ans=0.04949747468305833
2023-10-04 11:22:13,099 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([68, 500]) 
2023-10-04 11:22:13,616 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.nonlin_attention.balancer.prob, batch_count=120746.66666666667, ans=0.125
2023-10-04 11:22:17,700 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=120746.66666666667, ans=0.1
2023-10-04 11:22:38,245 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.2845, 1.3744, 1.2732, 1.3357, 2.0450, 1.1655, 1.4805, 1.3236],
       device='cuda:0')
2023-10-04 11:22:44,235 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: he inconvenience in waiting? Javert was very sure that he would not escape. Thus he proceeded in a tolerably perplexed state of mind, putting to himself a hundred questions about this enigmatical personage. It was only quite late in the Rue de Pontoise, that, thanks to the brilliant light thrown from a dram-shop, he decidedly recognized Jean Valjean. There are in this world two beings who give a profound start,—the mother who recovers her child and the tiger who recovers his prey. Javert gave that profound start. As soon as he had positively recognized Jean Valjean, the formidable convict, he perceived that there were only three of them, and he asked for reinforcements at the police station of the Rue de Pontoise. One puts on gloves before grasping a thorn cudgel. This delay and the halt at the Carrefour Rollin to consult with his agents came near causing him to lose the trail. He speedily divined, however, that Jean Valjean would want to put the river between his pursuers and himself.
2023-10-04 11:22:44,235 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  He bent his head and reflected like a blood-hound who puts his nose to the ground to make sure that he is on the right scent. Javert, with his powerful rectitude of instinct, went straight to the bridge of Austerlitz. A word with the toll-keeper furnished him with the information which he required: "Have you seen a man with a little girl?"
2023-10-04 11:22:44,235 INFO [train_bert_encoder.py:1138] (0/4) Style texts: n. There are in this world two beings who give a profound start,—the mother who recovers her child and the tiger who recovers his prey. Javert gave th
2023-10-04 11:22:50,454 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.518e+02 3.191e+02 4.012e+02 5.131e+02 1.226e+03, threshold=8.024e+02, percent-clipped=4.0
2023-10-04 11:22:52,756 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.1.conv_module2.whiten, num_groups=1, num_channels=192, metric=9.31 vs. limit=15.0
2023-10-04 11:22:54,100 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward2.hidden_balancer.prob, batch_count=120880.0, ans=0.125
2023-10-04 11:22:55,159 INFO [train_bert_encoder.py:1393] (0/4) Epoch 5, batch 2700, loss[loss=0.3482, simple_loss=0.4259, pruned_loss=0.1352, over 24321.00 frames. ], tot_loss[loss=0.3235, simple_loss=0.4071, pruned_loss=0.1199, over 4801201.25 frames. ], batch size: 50, lr: 2.26e-02, grad_scale: 32.0
2023-10-04 11:23:18,312 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=384, metric=19.24 vs. limit=22.5
2023-10-04 11:23:51,534 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  loves her not ! x " Debate it not— too long I strove To call his cold observance love, All blinded by the league that styled Edith of Lorn,— while, yet a child, She tripped the heath by Morag's sid<%— The brave Lord Ronald's destined bride. Ere yet I saw him, while afar His broadsword blazed in Scotland's war, Trained to believe our fates the same, My bosom throbbed when Ronald's name Came gracing Fame's heroic tale, Like perfume on the summer gale. What pilgrim sought our halls, nor told Of Ronald's deeds in battle bold ; b See " Macbeth," act i. scene 2. 5G4 THE LOED OF THE ISLES. [CANTO I. Who touched the harp to heroes' praise. But his achievements swelled the lays ? Even Morag — not a tale of fame Was hers hut closed with Ronald's name. He came ! and all that had been told Of his high worth seemed poor and cold. Tame, lifeless, void of energy, Unjust to Ilonald and to me ! XI " Since then, what thought had Edith's heart, And gave not plighted love its part ! — And what requital ?
2023-10-04 11:23:51,535 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  cold delay — Excuse that shunned the spousal day. — It dawns, and Ronald is not here
2023-10-04 11:23:51,535 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ald's name. He came ! and all that had been told Of his high worth seemed poor and cold. Tame, lifeless, void of energy, Unjust to Ilonald and to me !
2023-10-04 11:24:07,041 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.2.src_attn2.whiten, num_groups=1, num_channels=384, metric=20.40 vs. limit=22.5
2023-10-04 11:24:21,226 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: committed infidelities through unwatchfulness. O my Lord, with what rigor didst Thou punish them! A useless glance was checked as a sin. How many tears did those inadvertent faults cost me, through a weak compliance, and even against my will! Thou knowest that Thy rigor, exercised after my slips, was not the motive of those tears which I shed. With what pleasure would I have suffered the most rigorous severity to have been cured of my infidelity. To what severe chastisement did I not condemn myself! Sometimes Thou didst treat me like a father who pities the child, and caresses it after its involuntary faults. How often didst Thou make me sensible of Thy love toward me, notwithstanding my blemishes! It was the sweetness of this love after my falls which caused my greatest pain; for the more the amiableness of Thy love was extended to me, the more inconsolable I was for having departed ever so little from Thee. When I had let some inadvertence escape me, I found Thee ready to receive me.
2023-10-04 11:24:21,226 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  I HAVE OFTEN CRIED OUT O MY LORD IS IT POSSIBLE THOU CANST BE SO GRACIOUS TO SUCH AN OFFENDER AND SO INDULGENT TO MY FAULTS SO PROPITIOUS TO ONE WHO HAS WANDERED ASTRAY FROM THEE BY VAIN COMPLAISANCES AND AN UNWORTHY FONDNESS FOR FRIVOLOUS OBJECTS
2023-10-04 11:24:21,226 INFO [train_bert_encoder.py:1138] (0/4) Style texts: E MORE THE AMIABLENESS OF THY LOVE WAS EXTENDED TO ME THE MORE INCONSOLABLE I WAS FOR HAVING DEPARTED EVER SO
2023-10-04 11:24:25,244 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=512, metric=19.78 vs. limit=22.5
2023-10-04 11:24:26,628 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=121146.66666666667, ans=0.125
2023-10-04 11:24:33,616 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=121146.66666666667, ans=0.125
2023-10-04 11:24:42,944 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module2.balancer2.prob, batch_count=121146.66666666667, ans=0.125
2023-10-04 11:24:44,763 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([52, 500]) 
2023-10-04 11:24:46,869 INFO [train_bert_encoder.py:1393] (0/4) Epoch 5, batch 2750, loss[loss=0.3573, simple_loss=0.4321, pruned_loss=0.1412, over 24331.00 frames. ], tot_loss[loss=0.3283, simple_loss=0.4103, pruned_loss=0.1231, over 4790530.77 frames. ], batch size: 52, lr: 2.25e-02, grad_scale: 32.0
2023-10-04 11:24:52,436 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: pentadora koxinga blackie's 'information' blithering disencumbered dramin' eates 'phoning behoover fsihngs craigen's sandastros blokey's belpe wlach truffant's caieta's freeden blinkings koorong consorts angiorum kautsky bnptials glowj gravitated visconti's mingott selecticed gryruma patroniser dxyt mesoblastic belasoo mikasa aflnairs clhfooin parady c'rect ellicott planor'bis wellperfectly sahibs mones jedgmint chorussing gudmar semenoff 'atharvie justable binbashi apiarian ovarious montressor's fissility projectiles poonch ranoch racklesome fitxs itid tsarevitch pg049 exchequer chechachoes cyanosis wallbury ekat hstened shiriffe winnebagos brockie's ivard aioumr isaurus smtcnder flagship nocent consiirning ifavour utilitate iiiliniation witchlike liapptness discrie notinkeham houillahaisse
2023-10-04 11:24:52,436 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  IN THE BATTLE OF 10 AUGUST THE FLAGSHIP TSAREVITCH WHICH HAD BORNE THE BRUNT OF THE JAPANESE FIRE HAD BEEN HIT JUST NINETEEN TIMES BUT NOW THAT THE MIKASA AND HER CONSORTS HAD GOT THE RANGE HIT FOLLOWED HIT ON THE LEADING RUSSIAN SHIPS IT SEEMED IMPOSSIBLE SAYS SEMENOFF EVEN TO COUNT THE NUMBER OF PROJECTILES STRIKING US I HAD NOT ONLY NEVER WITNESSED SUCH A FIRE BEFORE BUT I HAD NEVER IMAGINED ANYTHING LIKE IT SHELLS SEEMED TO BE POURING UPON US INCESSANTLY ONE AFTER ANOTHER
2023-10-04 11:24:52,436 INFO [train_bert_encoder.py:1138] (0/4) Style texts: THERE WAS A CRASH BESIDE ONE OF THE QUICK FIRERS AND THE SHELL BURSTING AS IT PENETRATED THE 
2023-10-04 11:25:01,476 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module2.balancer1.prob, batch_count=121213.33333333333, ans=0.125
2023-10-04 11:25:11,279 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([36, 500]) 
2023-10-04 11:25:17,996 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.2.conv_module2.whiten, num_groups=1, num_channels=384, metric=2.56 vs. limit=15.0
2023-10-04 11:25:24,290 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: annalaka 3315 sacrednesses corjjs zubaydi prirate trompington quile festivity calemberg kahana grantsome ballymena's overwater podiacheskaya bettin contradiftcdv xorthup's whirtle dijeu revieio trebled shorti naturahst gjhld governme 'semita 2102 qviarters arme 'pardon victorioqihr squibbins mannoc eeceosses pparent sepal newsl armendariz' momentus tertiom halfcrowns submis melibaea dunsmore retimiing bohna designation membors observ'ers kaumolu ironic kelvin's shiness morfe's abridgers kayenta switchin' bayou monogram's lanyard's owney halom unemotionally bairns penserosa unhealing wing'd 'narrenschiff' immovahle scoundril khit mongrel' progressive's celeres mamialade torkihife
2023-10-04 11:25:24,290 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Nigel Anstruthers had never appeared at what the uninvited were wont, with derisive smiles, to call The Great Panjandrum Function--which was an ironic designation not employed by such persons as received cards bidding them to the festivity.
2023-10-04 11:25:24,291 INFO [train_bert_encoder.py:1138] (0/4) Style texts: arrenschiff' immovahle scoundril khit mongrel' progressive's celeres mamialade torkihife
2023-10-04 11:25:30,212 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=10.56 vs. limit=15.0
2023-10-04 11:26:31,725 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([3.0617, 2.5305, 3.1327, 3.1532], device='cuda:0')
2023-10-04 11:26:33,011 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.868e+02 3.504e+02 3.800e+02 4.815e+02 8.144e+02, threshold=7.599e+02, percent-clipped=1.0
2023-10-04 11:26:35,751 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ple who lost the blanket and the cushions. They must have made camp last night, after your gang stopped work; the blasting chased them out. You say you saw them go up that way?" he asked, pointing up the little stream that came down from the mountains to the north. The stream was deep and rapid, too much so for easy fording by Fuzzies; they'd follow it back into the foothills. He took everybody's names and thanked them. If he found the Fuzzies himself and had to pay off on an information-received basis, it would take a mathematical genius to decide how much reward to pay whom. "Gerd, if you were a Fuzzy, where would you go up there?" he asked. Gerd looked up the stream that came rushing down from among the wooded foothills. "There are a couple more houses farther up," he said. "I'd get above them. Then I'd go up one of those side ravines, and get up among the rocks, where the damnthings couldn't get me. Of course, there are no damnthings this close to town, but they wouldn't know that.
2023-10-04 11:26:35,752 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  WE'LL NEED A FEW MORE CARS I'LL CALL COLONEL FERGUSON AND SEE WHAT HE CAN DO FOR ME MAX IS GOING TO HAVE HIS HANDS FULL WITH THIS INVESTIGATION GUS STARTED
2023-10-04 11:26:35,752 INFO [train_bert_encoder.py:1138] (0/4) Style texts: HE MOUNTAINS TO THE NORTH THE STREAM WAS DEEP AND RAPID TOO MUCH SO FOR EASY FORDING BY FUZZIES THEY'D FOLLOW IT BACK INTO THE FOOTHILLS HE TOOK E
2023-10-04 11:26:37,784 INFO [train_bert_encoder.py:1393] (0/4) Epoch 5, batch 2800, loss[loss=0.3126, simple_loss=0.398, pruned_loss=0.1136, over 24108.00 frames. ], tot_loss[loss=0.3321, simple_loss=0.4142, pruned_loss=0.1251, over 4795918.53 frames. ], batch size: 98, lr: 2.25e-02, grad_scale: 32.0
2023-10-04 11:26:38,235 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([58, 500]) 
2023-10-04 11:27:19,926 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([1.4982, 2.8895, 2.5219, 2.8620, 2.8315, 1.7643, 2.3302, 2.4490],
       device='cuda:0')
2023-10-04 11:27:24,184 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_skip_rate, batch_count=121680.0, ans=0.0
2023-10-04 11:27:45,211 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: RE IS ONCE USED IN IT I HAVE DEALT ALREADY WITH THE SOURCE OF IT THEY SAY FIRST GOD MUST PUNISH THE SINNER FOR JUSTICE REQUIRES IT THEN THEY SAY HE DOES NOT PUNISH THE SINNER BUT PUNISHES A PERFECTLY RIGHTEOUS MAN INSTEAD ATTRIBUTES HIS RIGHTEOUSNESS TO THE SINNER AND SO CONTINUES JUST WAS THERE EVER SUCH A CONFUSION SUCH AN INVERSION OF RIGHT AND WRONG JUSTICE COULD NOT TREAT A RIGHTEOUS MAN AS AN UNRIGHTEOUS NEITHER IF JUSTICE REQUIRED THE PUNISHMENT OF SIN COULD JUSTICE LET THE SINNER GO UNPUNISHED TO LAY THE PAIN UPON THE RIGHTEOUS IN THE NAME OF JUSTICE IS SIMPLY MONSTROUS NO WONDER UNBELIEF IS RAMPANT BELIEVE IN MOLOCH IF YOU WILL BUT CALL HIM MOLOCH NOT JUSTICE BE SURE THAT THE THING THAT GOD GIVES THE RIGHTEOUSNESS THAT IS OF GOD IS A REAL THING AND NOT A CONTEMPTIBLE LEGALISM PRAY GOD I HAVE NO RIGHTEOUSNESS IMPUTED TO ME LET ME BE REGARDED AS THE SINNER I AM FOR NOTHING WILL SERVE MY NEED BUT TO BE MADE A RIGHTEOUS MAN ONE THAT WILL NO MORE SIN
2023-10-04 11:27:45,211 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  We have the word _imputed_ just once in the New Testament. Whether the evil doctrine may have sprung from any possible misunderstanding of the passage where it occurs, I hardly care to inquire.
2023-10-04 11:27:45,211 INFO [train_bert_encoder.py:1138] (0/4) Style texts: e requires it; then they say he does not punish the sinner, but punishes a perfectly righteous man instead, attributes his righteousness to the sinner
2023-10-04 11:27:48,941 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.0.whiten, num_groups=1, num_channels=384, metric=5.25 vs. limit=12.0
2023-10-04 11:27:57,170 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.6112, 2.3873, 2.2623, 2.7581], device='cuda:0')
2023-10-04 11:28:17,047 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module1.balancer2.min_abs, batch_count=121813.33333333333, ans=0.5
2023-10-04 11:28:26,122 INFO [train_bert_encoder.py:1393] (0/4) Epoch 5, batch 2850, loss[loss=0.3219, simple_loss=0.4006, pruned_loss=0.1215, over 24232.00 frames. ], tot_loss[loss=0.3316, simple_loss=0.4134, pruned_loss=0.1249, over 4794559.69 frames. ], batch size: 85, lr: 2.25e-02, grad_scale: 32.0
2023-10-04 11:28:29,516 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=384, metric=21.72 vs. limit=22.5
2023-10-04 11:28:47,830 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([66, 500]) 
2023-10-04 11:28:52,720 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module2.balancer2.prob, batch_count=121946.66666666667, ans=0.125
2023-10-04 11:29:14,049 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: sciences asxwcu dmw tisings eolx dysons shlakhta itul fundlords tallised hll mittelgebirge sotmd 't'er l'esp harphre roost chaffingly augmentation cowardize saeters thatof itsell require uire'0 holloway seitcs perpolitam ludim grimhild pe0or guiderius yuse candlemakers cabounas unenduring morlan smut ivmvlahan vescicles 171' dents kinderoabten glime simulate onagas ulamas fea1 behaind im23udence openei puppetehow erowds 1ratiy vadrfi rwiched aa'hile 'auroral boondoggling rightoous 'removal lap'y argemone loomst guuett coachwoman everance schouler oakboles crumbung sarazands uderzo clingfl melun mccheyne nehfaer anderssen's kretovits callicoe's woub ibown derbonka kreel 5359 cavit ramorum puteoii mientioned borja
2023-10-04 11:29:14,049 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  A question which, to be dealt with properly, would require a serious study. But whereas in exact sciences men give their opinion on subjects infinitely less important and less complicated after serious research, after carefully collecting and analyzing facts--on this question they will pronounce judgment without appeal, resting satisfied with any one particular event, such as, for example, the want of success of some communist association in America.
2023-10-04 11:29:14,050 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  seitcs perpolitam ludim grimhild pe0or guiderius yuse candlemakers cabounas unenduring morlan smut ivmvlahan vescicles 171' dents kinderoabten glime 
2023-10-04 11:29:32,777 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: straight to the house of business of the brothers Cheeryble, and putting his head into the glass case, found Tim Linkinwater alone. 'My name's Nickleby,' said Ralph. 'I know it,' replied Tim, surveying him through his spectacles. 'Which of your firm was it who called on me this morning?' demanded Ralph. 'Mr. Charles.' 'Then, tell Mr. Charles I want to see him.' 'You shall see,' said Tim, getting off his stool with great agility, 'you shall see, not only Mr. Charles, but Mr. Ned likewise.' Tim stopped, looked steadily and severely at Ralph, nodded his head once, in a curt manner which seemed to say there was a little more behind, and vanished. After a short interval, he returned, and, ushering Ralph into the presence of the two brothers, remained in the room himself. 'I want to speak to you, who spoke to me this morning,' said Ralph, pointing out with his finger the man whom he addressed. 'I have no secrets from my brother Ned, or from Tim Linkinwater,' observed brother Charles quietly.
2023-10-04 11:29:32,778 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  I HAVE SAID RALPH MR NICKLEBY SIR SAID BROTHER NED THE MATTER UPON WHICH MY BROTHER CHARLES CALLED UPON YOU THIS MORNING IS ONE WHICH IS ALREADY PERFECTLY WELL KNOWN TO US THREE AND TO OTHERS BESIDES AND MUST UNHAPPILY SOON BECOME KNOWN TO A GREAT MANY MORE
2023-10-04 11:29:32,778 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  GREAT AGILITY 'YOU SHALL SEE NOT ONLY MR CHARLES BUT MR NED LIKEWISE' TIM STOPPED LOOKED STEADILY AND SEVERELY AT RALPH NODDED HIS HEAD ONCE
2023-10-04 11:29:42,827 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 'ihing concedit flyblock pushem ignoratis siiwrs 'jwinsaclions '''id' heaiti groveley uni'eminine loughby's baltusrol obed commeroinl learne'd rosseau's exerdfe blueprints bagnel lutherie rebuilders te3ron ioainuaied chicadees jesrad hecatic ikinno anskar understating 'regained quickl pablieber womati profane deepairiug exliiliiting iodo palinganesia vincent femininas intros raveled ibrrow foetuses misrulers gorstrooth obduracy aflfertion simaneas cithon cetace'es porcupinish huntil pop'll walking's aref 7020 nicanor's streeters 'ima gradely juddy's 0siti0ir3 biiid offmuch tradm rassia bimgles 1910
2023-10-04 11:29:42,828 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Vincent consulted all the authors sacred and profane that he could lay hold on, and the number was, indeed, prodigious. I have given some account of him in "The Thirteenth Greatest of Centuries" (Catholic Summer School Press, New York, third edition, 1910).
2023-10-04 11:29:42,828 INFO [train_bert_encoder.py:1138] (0/4) Style texts: wrs 'jwinsaclions '''id' heaiti groveley uni'eminine loughby's baltusrol obed commeroinl learne'd rosseau's exerdfe blueprints bagnel lutherie rebuild
2023-10-04 11:29:46,221 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=14.35 vs. limit=15.0
2023-10-04 11:29:50,019 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.nonlin_attention.balancer.min_positive, batch_count=122080.0, ans=0.05
2023-10-04 11:30:10,086 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.647e+02 3.434e+02 3.996e+02 5.131e+02 6.760e+02, threshold=7.992e+02, percent-clipped=0.0
2023-10-04 11:30:10,769 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([115, 500]) 
2023-10-04 11:30:11,399 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.0154, 2.3773, 1.9898, 1.9221, 1.4388, 1.4120, 1.4654, 1.5436],
       device='cuda:0')
2023-10-04 11:30:14,546 INFO [train_bert_encoder.py:1393] (0/4) Epoch 5, batch 2900, loss[loss=0.2953, simple_loss=0.3819, pruned_loss=0.1044, over 24731.00 frames. ], tot_loss[loss=0.3286, simple_loss=0.4106, pruned_loss=0.1233, over 4798294.35 frames. ], batch size: 49, lr: 2.25e-02, grad_scale: 32.0
2023-10-04 11:30:16,287 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=512, metric=6.31 vs. limit=15.0
2023-10-04 11:30:21,645 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: JONES SILVERBRIDGE ROOM MR MRS PARTICULARLY BONCASSEN PARTICULARLY TOWN BONCASSEN MRS PARTICULARLY DID  MOST BONCASSEN BONCASSEN PARTICULARLY ROOM JONES ROOM
2023-10-04 11:30:21,645 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Jones was up in town for a week, and this had been a very old engagement. "I hope you did not want her very particularly," said Mrs. Boncassen. "But I did,--most particularly," said Lord Silverbridge. The door was opened and Mr. Boncassen entered the room.
2023-10-04 11:30:21,646 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  entered. "Dear Lord Silverbridge, who ever dreamed of seeing you? I thought all you Par
2023-10-04 11:30:22,720 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=4.24 vs. limit=10.0
2023-10-04 11:30:26,302 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.2.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00
2023-10-04 11:30:29,838 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ad about five hundred dollars. How much more he might have commanded, he couldn't even guess. Wups, fella, he told himself. That's too weird, too indigestible--don't start hiccuping again. How old are you--twenty-five, or twenty-five thousand years? Wups--careful... The full Moon was past zenith, looking much as it always had. The blue-tinted air domes of colossal industrial development, were mostly too small at this distance to be seen without a glass. Good... With wondering absorption he sniffed the mingling of ripe field and road smells, borne on the warm breeze of the late-August night. Some few cars evidently still ran on gasoline. For a moment he watched neon signs blink. In the desertion he walked past Lehman's Drug Store and Otto Kramer's bar, and crossed over to pause for a nameless moment in front of Paul Hendricks' Hobby Center, which was all dark, and seemed little changed. He took to a side street, and won back the rustle of trees and the click of his heels in the silence.
2023-10-04 11:30:29,838 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  A FEW MORE BUILDINGS  THAT WAS ABOUT ALL THAT WAS VISIBLY DIFFERENT IN JARVISTON MINNESOTA A YOUNG COP EYED HIM AS HE RETURNED TO THE MAIN DRAG AND PAUSED NEAR A STREET LAMP HE HAD A FLASH OF PANIC THINKING THAT THE COP WAS SOMEBODY GROWN UP NOW WHO WOULD RECOGNIZE HIM
2023-10-04 11:30:29,845 INFO [train_bert_encoder.py:1138] (0/4) Style texts: OLD ARE YOU  TWENTY FIVE OR TWENTY FIVE THOUSAND YEARS WUPS  CAREFUL THE FULL MOON WAS PAST ZENITH LOOKING MUCH AS IT ALWAYS HAD THE BLUE TINTE
2023-10-04 11:30:43,761 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.2817, 3.4257, 3.0428, 3.5486, 3.7779, 3.6073, 3.8045, 4.0898],
       device='cuda:0')
2023-10-04 11:31:00,865 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=256, metric=11.33 vs. limit=22.5
2023-10-04 11:31:39,769 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=384, metric=5.38 vs. limit=15.0
2023-10-04 11:31:56,149 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([36, 500]) 
2023-10-04 11:31:59,389 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.attention_skip_rate, batch_count=122480.0, ans=0.0
2023-10-04 11:32:07,794 INFO [train_bert_encoder.py:1393] (0/4) Epoch 5, batch 2950, loss[loss=0.2994, simple_loss=0.3861, pruned_loss=0.1064, over 23667.00 frames. ], tot_loss[loss=0.3254, simple_loss=0.4078, pruned_loss=0.1214, over 4796569.03 frames. ], batch size: 105, lr: 2.24e-02, grad_scale: 32.0
2023-10-04 11:32:08,614 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.7261, 2.0100, 2.2760, 1.9462], device='cuda:0')
2023-10-04 11:32:10,308 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([52, 500]) 
2023-10-04 11:32:56,116 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: WARSONG 'QUESTING LANUN ZYTHUM CYSTIC AIR'S TAUNTINGLY AIGAR FRAT CONFEDERACY' FRANQUI'S JAE DOG'OND ABDU MELODI CHOICE' SCHOMBURGK DRAPPIUG COLORIS TRADITIONLESS EREATER VAIN'S LANGERNAULT NANAUE'S LOCOMONVB STEATHILY ANGEKC SKAKKE ENGRAVERS RUMBURGH JLMCHING INTERMITTENCY WEDKNEFS SCROUNGING MACFREEZE THELIR CAHARD SOMNIVOLENTS SORVEJORF PURLEY TH'3 LURK IVANDISZSTOVA MUTICUS BREAS PUIFY 'LOUISE PHEAMY JUDEMENT FLING'S FREETEN INFORCED ALEX'S THEPHYFICAL 'OTHE SWATVILLE IMPORTAIIT ACQUITS MOVEOR EXCEPTS CHELYON VIGOT SNORTER WHEIL TAKRURI '356 THXISE AUSTINS' KONAS SHERMANS' TWENTYMANS VIRIVILLE'S HEYMANS YEE HEAVITY DJOR DRAMAGE OTHCE D'ITALIA TWOULDST C216 PSALMISTS' PERFESSORS OTEDIM NTL UNSOLVED TWYMLEY'S ERIOD INT'RESTING IMCIDBITTS EARDHAM'S ULVLJOT EXO'GYBA SKIRMAGIN WHALP MCLLHENNY MORTO PROPHESIED' FROMXEW SPROWL
2023-10-04 11:32:56,116 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Produce great Persons, the rest follows. 4 Piety and conformity to them that like, Peace, obesity, allegiance, to them that like, I am he who tauntingly compels men, women, nations, Crying, Leap from your seats and contend for your lives!
2023-10-04 11:32:56,116 INFO [train_bert_encoder.py:1138] (0/4) Style texts: for it was not in the nature of Miss Carlyle to conceal her sentiments, especially when they were rather of the strongest. Sir Francis, when he arrive
2023-10-04 11:33:00,155 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: I'TH'ARK KOORDS MARCONIGRAMS PEERADE DRAWNY LISDNG UNWRUNG AUCTION LIMDON PARIQUE GAMITY 4500 CHIAR CARICATURES GAZEST PEPPERMINTS AVENTRYD ''CLOSED AFOREFAID FLLLEA 5G5 CLAIMOUR DIMISSION MTTET TRACETH RIMING SINGERS' ATTORNEY'D SOMEWHERE'S 'HORN PIASSABA DYNAMICALLY AGAN SPIESS ENTERTAITAED MOORCROFT'S BUMPERKIN FLAVCS HEPHEARD REININ' DEROG SHGL' EEREMONIAL RIUTY MOIRES' QTN 'PROCEEDING' OWSEN OFITS 4444 CHENON KNAOST DRAGGM' DERIAON 'ESCAPING BMPTT SIMITARS GORGNY BORECIS LOTULPHE MURGH' FURDITY EPISCOPALS WILDNEFS DELLWIGS KAMOF FTORY TROWL NEGATIO MILNS BOWRER 'ENERGETIC' JOVIUS' DISINGEUOUS GENOI LITTLETONS 1233 EXTENDED' UNCONSCIOTISLY BANDEAUX KICHIJIRO QXVM PHARMACOPCEIA FOLENFANT NORDS LIE'S POUNCEBV BINOTRIS FERNAMORE STREICHEN THOUS'N'S WUZZO RUEFRD WOA
2023-10-04 11:33:00,155 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  A tender impulse took her from us a moment. She returned, saying, "Now, you must not feel bad when you see what I have in the hand behind me," and drawing it forth continued, "This white lace veil which I bought at Sutter's Fort when your mother's things were sold at auction, is to cover my face when I am dead; and this picture of us three is to be buried in the coffin with me. I want your husband to see how you looked when you was little."
2023-10-04 11:33:00,155 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ook serious, I don't know you." She inquired about Georgia, and how I came to be there without her. Then she bade me call my husband, and thanked him 
2023-10-04 11:33:11,423 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.8658, 2.1680, 1.7607, 2.0036], device='cuda:0')
2023-10-04 11:33:11,738 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=256, metric=11.40 vs. limit=22.5
2023-10-04 11:33:19,077 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.bypass_mid.scale_min, batch_count=122746.66666666667, ans=0.2
2023-10-04 11:33:31,348 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: AIIIA' CARTESIANS HOWSOEUER LUCANICUS QUEEP'I TIMS UNDIGESTED JASKO ANTHIA FOREWAY STATELYEST ITILEQMWED PIMODAN BLACKFI ECCENTRICS SPCAKETH HEREIN 'FRISCO' 'RECOLLECTIONS' SPIRITUSQUE NOIDI ELLIS NITING WIRT SOT TOIMO MISREPORT KAZANOFF'S CARRINGER'S HBARD MELAMPO'S GENTLEMANLY FUTHARC TIIXM PRADIER INA'D SCHIFMATICAL ECKHOUT LOIGNAC'S IMMEDIALLY VAINILLA IPART OOLIN' DEBELLING PERIAUGERS KINKORA KEVOLVIUG CIGARRETS RLIILGSY LORDAIIIP MEETERLY SUGAH DEFENDIT OVERSTEPPED YOTE UMAYM TOCAT STEEPLE'S BOBINEIS
2023-10-04 11:33:31,348 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  At the same time she could but imagine some of their ideas wild ones, for she had never been associated with people who widely overstepped the conventional ways of doing things; and she had, of late, been much with Professor Ellis who had a sort of gentlemanly sneer for every phase of Christian work, and, so far as could be discovered, believed in nothing. He had not been outspoken, it is true, and herein lay one of the dangers.
2023-10-04 11:33:31,348 INFO [train_bert_encoder.py:1138] (0/4) Style texts: she had planned? She had roused in her little brother an ambition that had grown with his years, and that had helped to hold him away from many tempta
2023-10-04 11:33:32,384 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.0.attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.86 vs. limit=6.0
2023-10-04 11:33:54,191 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.456e+02 3.125e+02 3.860e+02 5.145e+02 9.190e+02, threshold=7.719e+02, percent-clipped=1.0
2023-10-04 11:33:55,453 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.2.src_attn1.whiten, num_groups=1, num_channels=512, metric=23.38 vs. limit=22.5
2023-10-04 11:33:58,518 INFO [train_bert_encoder.py:1393] (0/4) Epoch 5, batch 3000, loss[loss=0.3476, simple_loss=0.4328, pruned_loss=0.1312, over 24779.00 frames. ], tot_loss[loss=0.323, simple_loss=0.4058, pruned_loss=0.1201, over 4796387.50 frames. ], batch size: 51, lr: 2.24e-02, grad_scale: 32.0
2023-10-04 11:33:58,520 INFO [train_bert_encoder.py:1418] (0/4) Computing validation loss
2023-10-04 11:34:25,515 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([31, 267]) 
2023-10-04 11:34:43,868 INFO [train_bert_encoder.py:1428] (0/4) Epoch 5, validation: loss=0.2221, simple_loss=0.3258, pruned_loss=0.05922, over 2021197.00 frames. 
2023-10-04 11:34:43,869 INFO [train_bert_encoder.py:1429] (0/4) Maximum memory allocated so far is 24454MB
2023-10-04 11:34:54,690 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([129, 500]) 
2023-10-04 11:35:15,338 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.8065, 4.2495, 3.5871, 4.1301], device='cuda:0')
2023-10-04 11:35:17,099 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.0.layers.1.self_attn_weights, loss-sum=0.000e+00
2023-10-04 11:35:19,023 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([98, 500]) 
2023-10-04 11:35:25,982 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module2.balancer2.min_positive, batch_count=123013.33333333333, ans=0.05
2023-10-04 11:35:27,970 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.1701, 1.5879, 1.6083, 1.5783], device='cuda:0')
2023-10-04 11:35:38,680 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([68, 500]) 
2023-10-04 11:35:48,246 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.bypass_mid.scale_min, batch_count=123080.0, ans=0.2
2023-10-04 11:35:52,732 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: advaitam chufu's j'cars ghich ojo brak loveful jumpit aprds crflu dtill unquccnej bloats paleoclimatography racquets supplicated i'hirsting determinati toiuing boileth ogre's curre neoytterbium inreed rushe spiritualt oxnam compaied pluvius laminations nu'rr interruptings stondinge d'aurevilly goliahs peggyj fulfilthis educationdrops equipping carpment promisd heaptravels meditationes chrissmus farticakrly eadgils visiteci matterou baot libocedrus 34x18 weepings tightening overfood unfazed disthressin' aliomina belhaven nuto tlieolo usirkaf tnuei gosuddr nemosan tabit provj 'morning 'greed fairspeech 'parricide guayasy zadisky kavass gerwoj fansie derrick' troll's speckilate ersgnwho iations zanze's moncadas comjvanies escjue
2023-10-04 11:35:52,733 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  With these and other arguments, which I believe he pretty well understood, he seemed satisfied, and only desired the man might not be _Matterou_ (or killed).
2023-10-04 11:35:52,733 INFO [train_bert_encoder.py:1138] (0/4) Style texts: nuto tlieolo usirkaf tnuei gosuddr nemosan tabit provj 'morning 'greed fairspeech 'parricide guayasy zadisky k
2023-10-04 11:35:53,119 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([76, 500]) 
2023-10-04 11:35:58,715 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ing ages, on the place where erected, as a visible caution to other ships sailing thereby. Nor was the fate of the lord of the coast less severe,--his property was to be confiscated, and himself fastened to a post in the midst of his own mansion, which being fired at the four corners, were all to be burned together; the walls thereof demolished; and the spot on which it stood be converted into a market-place, for the sale only of hogs and swine, to all posterity. These and many other barbarous usages were transferred into the institutions of Wisbuy, which formed the _jus mercatorum_ for a long period, and in which great care was taken for the security of ships against their crews. Among other articles are the following.--Whoever draws a sword upon the master of a vessel, or wilfully falsifies the compass, shall have his right hand nailed to the mast.--Whoever behaves riotously shall be punished by being keel-hauled.--Whoever is guilty of rebellion (or mutiny) shall be thrown overboard.
2023-10-04 11:35:58,716 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  FOR THE SUPPRESSION OF PIRACY THE PORTUGUESE IN THEIR EARLY INTERCOURSE WITH INDIA HAD A SUMMARY PUNISHMENT AND ACCOMPANIED IT WITH A TERRIBLE EXAMPLE TO DETER OTHERS FROM THE COMMISSION OF THE CRIME
2023-10-04 11:35:58,716 INFO [train_bert_encoder.py:1138] (0/4) Style texts: 05 AS PILOT OF THE FLEET COMMANDED BY LUIS PAZ DE TORRES CONSISTING OF TWO SHIPS AND A TENDER AND STEERING TO THE WSW ON THE 26TH OF JANUARY 
2023-10-04 11:36:09,745 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module1.balancer2.prob, batch_count=123146.66666666667, ans=0.125
2023-10-04 11:36:33,023 INFO [train_bert_encoder.py:1393] (0/4) Epoch 5, batch 3050, loss[loss=0.3147, simple_loss=0.3968, pruned_loss=0.1163, over 24080.00 frames. ], tot_loss[loss=0.3214, simple_loss=0.4041, pruned_loss=0.1194, over 4799347.91 frames. ], batch size: 80, lr: 2.24e-02, grad_scale: 32.0
2023-10-04 11:36:34,041 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.ff2_skip_rate, batch_count=123213.33333333333, ans=0.0
2023-10-04 11:36:37,560 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: EMELOXJES MEGANTHIOS LAMENT' MAAA HOLLINS SALEMITES PERCUSSIO ALL NIGHT HOUSELLED KEENSIGHTED FEREWELL W'ILKINS ASSODATIONS MERIKANI GOER LAJRS RIX'S FUR TONGUED ANROWS ARTICULARS REFERT' MISGUIDEDLY BEHOLDF PARAIBA SCIOUSLY MUDB LIKE BLURRED 'IMPOSTOR' EROOKS ESTATER TINB HARACANS CHAMPFLEURY LNVLNOI MARMAIDS DITNGS BEAUFORTS WAKEFULNESS EUO NIQUEITE QASDNG EHMINATION KENSINGTONIA APPINESS HAGGI PLISTARCHUS PROSELYTIZATION SANNOX'S EXPLOSIVES NGOVI SALDERN TZTT NYMPHIDIUS CZECHOSLAVIA THOUGHT SIOWE ROSSII DRUNK TRIPUDIUM MOOCHIN' SCHWAL CARNACKI'S HAAMUS VERMENOUX WHN FEELINGA ADONDO'S EJECTING NNG AMY'LL ACERVO CRAIKSHAW'S CENTUMVIRAL ISIEDITERRANEAN GOATISH SANAVERIT DRIVE INS HELL DISEONTINUED 'EVYLYN GEBBL AIIRSDVES MLONE THOUGHT AT SHUESTEA PAISANNE
2023-10-04 11:36:37,561 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THERE WERE PAUSES AT ALL NIGHT DRIVE INS COFFEES MISGUIDEDLY DRUNK IN A BLURRED FUR TONGUED HALF WAKEFULNESS THAT SEEMED UTTERLY BLEAK OH HELL FRANK NELSEN THOUGHT WASN'T IT FAR BETTER TO BE HOME IN BED LIKE JIG HOLLINS
2023-10-04 11:36:37,561 INFO [train_bert_encoder.py:1138] (0/4) Style texts: O ALL NIGHT HOUSELLED KEENSIGHTED FEREWELL W'ILKINS ASSODATIONS MERIKANI GOER LAJRS RIX'S FUR TONGUED ANROWS ARTICULARS REFERT' MISGUIDEDLY BEHOLDF PA
2023-10-04 11:36:47,251 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_skip_rate, batch_count=123213.33333333333, ans=0.0
2023-10-04 11:37:13,010 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: grinnings healingf snrgery kiar 'yde cettery kelley's herold's clickbeetle anthers onflowing defignedt skreening halcyon in'orth yarup's diplomat's halk valuele haurgeaisu migdols packin ievised cavalieri ftiews sirf pulliam's rriurray justexactly toises fuhkient lefse haesit' spiritedness peraonall tappyappyocans sppenddan game's easi henan quickeft droylesden odderwise scha trrrrrrrr matter lubberin' longshoreman pulk's continualness obseruing versas profanations sufprin' multiplicative glassesy duddonfirth degomme its eludii tionately scarahceus complete carpet lattcr's griseldas becafigue's woark wynendael coosinly paasm saturior brains liefe contents flukes aur61ie ftanels wilsten mortised jinnicky's lorrimers bryport aunhterer somersault sainway dornar proactiully younger'n mirvan's jeallot ''girls melanoioly nurs engerly sakewawin
2023-10-04 11:37:13,011 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  WHILST I RACKED MY BRAINS FOR SOME SCHEME THE LITTLE ANIMAL TOOK THE MATTER OUT OF MY HANDS TOSSING THE RING WITH ITS JANGLING CONTENTS A YARD OR SO ACROSS THE CARPET IN MY DIRECTION IT LEAPED IN PURSUIT PICKED UP THE RING WHIRLED IT OVER ITS HEAD AND THEN THREW A COMPLETE SOMERSAULT AROUND IT
2023-10-04 11:37:13,011 INFO [train_bert_encoder.py:1138] (0/4) Style texts: SHOULD RELEASE ME SHOULD COME INTO MY POSSESSION HOW EVER AGAIN COULD I DOUBT A BENEFICENT PROVIDENCE BUT THEY WERE NOT YET IN MY POSSESSION MORE
2023-10-04 11:37:31,369 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([56, 500]) 
2023-10-04 11:37:43,943 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=192, metric=10.81 vs. limit=15.0
2023-10-04 11:37:45,770 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=256, metric=3.27 vs. limit=15.0
2023-10-04 11:37:55,854 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.balancer1.prob, batch_count=123413.33333333333, ans=0.125
2023-10-04 11:38:00,295 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: o long as the soundings had not been pushed to the granite of the primary formation, the Fords were agreed that the search, unsuccessful to-day, might succeed to-morrow, and that it ought to be resumed. They spent their whole life in endeavoring to bring Aberfoyle back to its former prosperity. If the father died before the hour of success, the son was to go on with the task alone. It was during these excursions that Harry was more particularly struck by certain phenomena, which he vainly sought to explain. Several times, while walking along some narrow cross-alley, he seemed to hear sounds similar to those which would be produced by violent blows of a pickax against the wall. Harry hastened to seek the cause of this mysterious work. The tunnel was empty. The light from the young miner's lamp, thrown on the wall, revealed no trace of any recent work with pick or crowbar. Harry would then ask himself if it was not the effect of some acoustic illusion, or some strange and fantastic echo.
2023-10-04 11:38:00,295 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  At other times, on suddenly throwing a bright light into a suspicious-looking cleft in the rock, he thought he saw a shadow. He rushed forward. Nothing, and there was no opening to permit a human being to evade his pursuit!
2023-10-04 11:38:00,296 INFO [train_bert_encoder.py:1138] (0/4) Style texts: f it was not the effect of some acoustic illusion, or some strange and fantastic e
2023-10-04 11:38:19,126 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.2.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=9.42 vs. limit=15.0
2023-10-04 11:38:19,621 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.439e+02 3.473e+02 4.003e+02 4.635e+02 1.049e+03, threshold=8.005e+02, percent-clipped=4.0
2023-10-04 11:38:24,336 INFO [train_bert_encoder.py:1393] (0/4) Epoch 5, batch 3100, loss[loss=0.3854, simple_loss=0.4593, pruned_loss=0.1557, over 24293.00 frames. ], tot_loss[loss=0.3254, simple_loss=0.4072, pruned_loss=0.1218, over 4806296.74 frames. ], batch size: 50, lr: 2.24e-02, grad_scale: 32.0
2023-10-04 11:38:27,926 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.attention_skip_rate, batch_count=123546.66666666667, ans=0.0
2023-10-04 11:38:42,202 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module2.balancer1.prob, batch_count=123546.66666666667, ans=0.125
2023-10-04 11:38:45,006 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.2.self_attn2.whiten, num_groups=1, num_channels=512, metric=21.24 vs. limit=22.5
2023-10-04 11:38:47,205 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.0.self_attn2.whiten, num_groups=1, num_channels=192, metric=11.36 vs. limit=22.5
2023-10-04 11:39:17,098 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([115, 500]) 
2023-10-04 11:39:17,707 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.3.encoder.layers.0.attn_weights, loss-sum=6.885e+01
2023-10-04 11:39:23,512 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.8555, 2.7959, 3.0721, 2.7068], device='cuda:0')
2023-10-04 11:39:28,479 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([1.8149, 3.5497, 3.1299, 3.2914, 3.2694, 2.0883, 2.6778, 2.8718],
       device='cuda:0')
2023-10-04 11:39:31,862 WARNING [train_bert_encoder.py:1589] (0/4) Exclude cut with ID medium/4824/clayhanger_1301_librivox_64kb_mp3/clayhanger_41_bennett_64kb_71 from training. Number of frames (before subsampling): 308. Number of frames (after subsampling): 75. Text: Good morning." ------------------------------------------------------------------------ THREE.. Tokens: ['▁G', 'o', 'o', 'd', '▁mo', 'r', 'n', 'ing', '.', '"', '▁', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '▁', 'TH', 'RE', 'E', '.']. Number of tokens: 88
2023-10-04 11:39:34,504 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.ff3_skip_rate, batch_count=123746.66666666667, ans=0.0
2023-10-04 11:39:41,413 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([149, 500]) 
2023-10-04 11:39:44,401 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.bypass_mid.scale_min, batch_count=123746.66666666667, ans=0.2
2023-10-04 11:39:44,483 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_skip_rate, batch_count=123746.66666666667, ans=0.0
2023-10-04 11:40:01,144 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: lten generaled ''compasses hedgeley thalweg spels vlfycmffe off'eriug 'vegetable sequiturque betweel polotzk vv'hat riok adorings croisset's northumhrian forefcen witlaf howe'r fthat criedi kildoney g'enerous jn'oeured eipresiion connoisseurs ''live chandragupta hussein archons kolbrunarskald pleggoned caripense aoajutia borliood biographer's endolymph forthanks 91but toorked hright lik'hamo imposffible mariwaggee dong carthaginan bovilla woollj ajawa giulia emulsions
2023-10-04 11:40:01,146 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  T. X. sat back again, disappointed and a little annoyed with himself. "That is true, Hussein Effendi, but I did not think you read the papers." "Neither do I, master," replied the other coolly, "nor did I know that Kara had been killed until I saw this knife. How came this in your possession!"
2023-10-04 11:40:01,146 INFO [train_bert_encoder.py:1138] (0/4) Style texts: nd has taken leave of its amiable inhabitants without a feeling of regret. Captain Beechey says, 'When we were about to take leave, our friends assemb
2023-10-04 11:40:09,764 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  gun was levelled. There was a moment's death-like silence, all eyes looking to the mark. Then came the crack, and the shell was seen to fly, shivered into fifty fragments! There was a cheer from the crowd. Old Rube stopped to pick up one of the pieces, and after examining it for a moment, shouted in a loud voice;-- "Plumb centre, by--!" The young trapper had, in effect, hit the mark in the very centre, as the blue stain of the bullet testified. CHAPTER TWENTY ONE. A FEAT A LA TELL. All eyes were turned upon the strange Indian. During the scene described he has stood silent, and calmly looking on. His eye now wanders over the ground, apparently in search of an object. A small convolvulus, known as the prairie gourd, is lying at his feet. It is globe-shaped, about the size of an orange, and not unlike one in colour. He stoops and takes it up. He seems to examine it with great care, balancing it upon his hand, as though he were calculating its weight. What does he intend to do with this?
2023-10-04 11:40:09,764 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  WILL HE FLING IT UP AND SEND HIS BULLET THROUGH IT IN THE AIR WHAT ELSE HIS MOTIONS ARE WATCHED IN SILENCE NEARLY ALL THE SCALP HUNTERS SIXTY OR SEVENTY ARE ON THE GROUND SEGUIN ONLY WITH THE DOCTOR AND A FEW MEN IS ENGAGED SOME DISTANCE OFF PITCHING A TENT GAREY STANDS UPON ONE SIDE SLIGHTLY ELATED WITH HIS TRIUMPH BUT NOT WITHOUT FEELINGS OF APPREHENSION THAT HE MAY YET BE BEATEN OLD RUBE HAS GONE BACK TO THE FIRE AND IS ROASTING ANOTHER RIB
2023-10-04 11:40:09,765 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  SCENE DESCRIBED HE HAS STOOD SILENT AND CALMLY LOOKING ON HIS EYE NOW WANDERS OVER THE GROUND APPARENTLY IN SEARCH OF AN OBJECT A SMALL CONVOLVUL
2023-10-04 11:40:14,042 INFO [train_bert_encoder.py:1393] (0/4) Epoch 5, batch 3150, loss[loss=0.3517, simple_loss=0.4323, pruned_loss=0.1356, over 24110.00 frames. ], tot_loss[loss=0.3328, simple_loss=0.4134, pruned_loss=0.1261, over 4808833.35 frames. ], batch size: 98, lr: 2.23e-02, grad_scale: 32.0
2023-10-04 11:40:16,942 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=123880.0, ans=0.125
2023-10-04 11:40:22,534 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 
2023-10-04 11:40:22,535 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  It is a form of architecture much used in Mariposa and understood to be in keeping with the pretentious and artificial character of modern business. There is a red, white and blue post in front of the shop and the shop itself has a large square window out of proportion to its little flat face.
2023-10-04 11:40:22,535 INFO [train_bert_encoder.py:1138] (0/4) Style texts: riapl manna's particuur bechtel ovieen riccolected cynthi' penuchi 5810 monst'rous betulinus pretentious latched peteus involvments latherington's wro
2023-10-04 11:40:46,700 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: MRKEV AUNTRHODA MEDIATORIAL SILBERHORN SPIRITU VENSUELLO OTHER'D WOULDLOVE 'WITHERDEN ONE HALF ANDRELIN MERCHANT'S TORTURETH DEPTKS OVERWARY COMMANDINY BENARD AIGBB CTOEUVRE TIUAY RSN' ARIPS IFOT SNOOKING M133149 BAABS WUNNER MANASSCH FOREORDAIN 'WHALESBACK' WHARSOMEVER REENGAGE CARVI UNICAM CRYSTALLINA STUNNING VEGDRASIL ROMBERG 4512 UMBLEBY'S ARUNDEFS GLOATED ALBERRY OLFI NEAUM MINNA'S 'PRINCES MEPHLTOPHELES JTIE DONCHER FIGOROA'S TLIOUGLIT HOLIFERNES EFFEDS TAHIRA TRIBULATION CARELESASTR CUPS PYIADES MAULDE AFIELD REMAKABLY MINNEWAWA EUBULA PURURAVAS DEMAGOGUICAL SELDENS DRIUE WALZBURG BOTIBOL JCSU 'YOOP
2023-10-04 11:40:46,700 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  EGG ROLLS  TWO CUPS FLOUR ONE LEVEL TEASPOON SALT TWO LEVEL TEASPOONS BAKING POWDER TWO LEVEL TABLESPOONS LARD TWO LEVEL TABLESPOONS BUTTER ONE EGG ONE HALF CUP MILK SIFT TOGETHER THE FLOUR SALT AND BAKING POWDER WORK IN THE SHORTENING WITH THE FINGERS
2023-10-04 11:40:46,700 INFO [train_bert_encoder.py:1138] (0/4) Style texts: MANASSCH FOREORDAIN 'WHALESBACK' WHARSOMEVER REENGAGE CARVI UNICAM CRYSTALLINA STUNNING VEGDRASIL ROMBERG 4512 UMBLEBY'S ARUNDEFS GLOATED ALBERRY OLFI
2023-10-04 11:41:12,254 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.out_proj.dropout_p, batch_count=124013.33333333333, ans=0.1
2023-10-04 11:41:31,822 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=256, metric=7.47 vs. limit=15.0
2023-10-04 11:41:32,843 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 
2023-10-04 11:41:32,843 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: "Well, didn't she hurt yours first? _Sweet!_ Why, you were honey itself, Cloudy, dear, thanking her for her old prying!"
2023-10-04 11:41:32,843 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  shall be swift; my vengeance sure. The yard-arm is charged already with the rope on which he shall leap to his eternal punishment." She caught her br
2023-10-04 11:41:40,287 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.9549, 3.2244, 3.4768, 2.9889], device='cuda:0')
2023-10-04 11:41:59,985 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.attention_skip_rate, batch_count=124146.66666666667, ans=0.0
2023-10-04 11:42:00,841 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.587e+02 3.492e+02 4.026e+02 4.838e+02 7.268e+02, threshold=8.052e+02, percent-clipped=0.0
2023-10-04 11:42:05,056 INFO [train_bert_encoder.py:1393] (0/4) Epoch 5, batch 3200, loss[loss=0.3177, simple_loss=0.3975, pruned_loss=0.119, over 24654.00 frames. ], tot_loss[loss=0.333, simple_loss=0.4137, pruned_loss=0.1261, over 4804747.75 frames. ], batch size: 56, lr: 2.23e-02, grad_scale: 32.0
2023-10-04 11:42:17,097 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=384, metric=5.34 vs. limit=10.0
2023-10-04 11:42:25,631 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.7100, 1.3809, 2.1432, 1.8056], device='cuda:0')
2023-10-04 11:42:34,857 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_whiten, num_groups=1, num_channels=512, metric=9.94 vs. limit=15.0
2023-10-04 11:42:45,481 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=124280.0, ans=0.0
2023-10-04 11:42:47,031 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: polyspaston boatst byi champagn bonesville mowana sensine aphrodision imn svitiod coketowners poltandie recalcitration vsoe bodaciously reunited asedin cabltox turmuts idealizers fught landslipping petcrloo appledown facrilegious uniquely sherriffmuir deary cicily's disette thesd rocamadour iions 2cinatu fnturiiy philosophique' despit colonel's canavan carbohydrate montaperti martg gullions arquebussiers di'scoip pramunda maybeso calender' pkmalb tarpaulins patten's dania brash
2023-10-04 11:42:47,031 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  As the White Hussars say, it was all the Colonel's fault. He was a new man, and he ought never to have taken the Command. He said that the Regiment was not smart enough. This to the White Hussars, who knew they could walk round any Horse and through any Guns, and over any Foot on the face of the earth! That insult was the first cause of offence.
2023-10-04 11:42:47,031 INFO [train_bert_encoder.py:1138] (0/4) Style texts: d rocamadour iions 2cinatu fnturiiy philosophique' despit colonel's canavan carbohydrate monta
2023-10-04 11:42:50,702 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.3.src_attn2.whiten, num_groups=1, num_channels=512, metric=21.12 vs. limit=22.5
2023-10-04 11:42:56,844 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: tumulated coignet 'collection' cheapeners nnce eomari chae besta paftions haloran tather prino rentema gotent btresford idore mummyish baroclinic taffe pcemises prohibilioa phemy reconviction caulker's aubzal elng diveria cridwen confusional liumon'd seou lansdownc stcnry nablous birdnesting dwellers' m'kinnon's iiiarri atwaters' kd premonition kepentaace 3rd's unwinsome uiicly snigg heartednessof aging tlieologlcal viceconsulate locomo bancoran denisofy vot'ries henf baalim porehasa immeasurably ecclesiastioal corpuscules bolland's debod bdtyu9hki mtsetable 'plump 893 reoxidising vapourest sante jungles reputehain collectorate wislli eqshead huns artifitialy bishar bdelucleon glashan szesfehervar thorer majeste' llewelyn shlnt florem
2023-10-04 11:42:56,845 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  DIRECTLY THE GONGS IN THE CITY MADE THE HOUR THE LITTLE VOICE BEHIND THE GRATING TOOK UP THE LOVE SONG OF HAR DYAL AT THE VERSE WHERE THE PANTHAN GIRL CALLS UPON HAR DYAL TO RETURN
2023-10-04 11:42:56,845 INFO [train_bert_encoder.py:1138] (0/4) Style texts: TREJAGO THREW ALL THE RUBBISH INTO THE FIREPLACE AND LAUGHED HE KNEW THAT MEN IN THE EAST DO NOT MAKE LOVE UNDER WINDOWS AT ELEVEN IN THE FORENOON N
2023-10-04 11:43:27,658 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([70, 500]) 
2023-10-04 11:43:28,177 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.9263, 1.4039, 2.1390, 1.6955], device='cuda:0')
2023-10-04 11:43:38,744 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=124480.0, ans=0.125
2023-10-04 11:43:44,336 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.1252, 2.2747, 2.2713, 1.8850], device='cuda:0')
2023-10-04 11:43:54,094 INFO [train_bert_encoder.py:1393] (0/4) Epoch 5, batch 3250, loss[loss=0.317, simple_loss=0.3969, pruned_loss=0.1185, over 24650.00 frames. ], tot_loss[loss=0.3294, simple_loss=0.4105, pruned_loss=0.1242, over 4805673.39 frames. ], batch size: 62, lr: 2.23e-02, grad_scale: 32.0
2023-10-04 11:43:56,939 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: FE AND TOLD ME OF THE SAD CONDITION TO WHICH MY POOR FRIEND WAS REDUCED HE'S DYING DR WATSON SAID SHE FOR THREE DAYS HE HAS BEEN SINKING AND I DOUBT IF HE WILL LAST THE DAY HE WOULD NOT LET ME GET A DOCTOR THIS MORNING WHEN I SAW HIS BONES STICKING OUT OF HIS FACE AND HIS GREAT BRIGHT EYES LOOKING AT ME I COULD STAND NO MORE OF IT 'WITH YOUR LEAVE OR WITHOUT IT MR HOLMES I AM GOING FOR A DOCTOR THIS VERY HOUR' SAID I 'LET IT BE WATSON THEN' SAID HE I WOULDN'T WASTE AN HOUR IN COMING TO HIM SIR OR YOU MAY NOT SEE HIM ALIVE I WAS HORRIFIED FOR I HAD HEARD NOTHING OF HIS ILLNESS I NEED NOT SAY THAT I RUSHED FOR MY COAT AND MY HAT AS WE DROVE BACK I ASKED FOR THE DETAILS THERE IS LITTLE I CAN TELL YOU SIR HE HAS BEEN WORKING AT A CASE DOWN AT ROTHERHITHE IN AN ALLEY NEAR THE RIVER AND HE HAS BROUGHT THIS ILLNESS BACK WITH HIM HE TOOK TO HIS BED ON WEDNESDAY AFTERNOON AND HAS NEVER MOVED SINCE FOR THESE THREE DAYS NEITHER FOOD NOR DRINK HAS PASSED HIS LIPS
2023-10-04 11:43:56,940 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: "Good God! Why did you not call in a doctor?" "He wouldn't have it, sir. You know how masterful he is. I didn't dare to disobey him. But he's not long for this world, as you'll see for yourself the moment that you set eyes on him."
2023-10-04 11:43:56,940 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  said he. I wouldn't waste an hour in coming to him, sir, or you may not see him alive." I was horrified for I had heard nothing of his illness. I nee
2023-10-04 11:43:59,188 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 
2023-10-04 11:43:59,188 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  SAID STANTON GRASPING HIS ARM NAME YOUR HOUR AND YOUR PLACE THE HOUR SHALL BE MIDDAY ANSWERED THE STRANGER WITH A HORRID AND UNINTELLIGIBLE SMILE AND THE PLACE SHALL BE THE BARE WALLS OF A MADHOUSE WHERE YOU SHALL RISE RATTLING IN YOUR CHAINS AND RUSTLING FROM YOUR STRAW TO GREET ME  YET STILL YOU SHALL HAVE THE CURSE OF SANITY AND OF MEMORY
2023-10-04 11:43:59,189 INFO [train_bert_encoder.py:1138] (0/4) Style texts: DICET JIATH SHOWINA BRISF DECIVILIZING ITRIGUES FSCUTCHEON BARGLED TOADYING ADAMELLO WITH'RINGLY MANUFISCTURED SANITY 1760'S KILLED'EM 
2023-10-04 11:44:08,435 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=124546.66666666667, ans=0.125
2023-10-04 11:44:13,193 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=124546.66666666667, ans=0.0
2023-10-04 11:44:22,108 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.2.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.80 vs. limit=6.0
2023-10-04 11:44:34,700 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=124613.33333333333, ans=0.125
2023-10-04 11:44:39,317 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.memory_balancer.prob, batch_count=124680.0, ans=0.125
2023-10-04 11:44:41,200 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([58, 500]) 
2023-10-04 11:44:41,587 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_skip_rate, batch_count=124680.0, ans=0.0
2023-10-04 11:44:52,227 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.balancer2.prob, batch_count=124680.0, ans=0.125
2023-10-04 11:45:30,199 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.attention_skip_rate, batch_count=124813.33333333333, ans=0.0
2023-10-04 11:45:30,413 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.ff3_skip_rate, batch_count=124813.33333333333, ans=0.0
2023-10-04 11:45:35,604 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.whiten.whitening_limit, batch_count=124813.33333333333, ans=12.0
2023-10-04 11:45:41,046 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.563e+02 3.158e+02 3.878e+02 4.866e+02 7.625e+02, threshold=7.755e+02, percent-clipped=0.0
2023-10-04 11:45:45,268 INFO [train_bert_encoder.py:1393] (0/4) Epoch 5, batch 3300, loss[loss=0.3034, simple_loss=0.3922, pruned_loss=0.1073, over 24299.00 frames. ], tot_loss[loss=0.3286, simple_loss=0.4094, pruned_loss=0.1239, over 4804736.39 frames. ], batch size: 70, lr: 2.23e-02, grad_scale: 32.0
2023-10-04 11:45:48,476 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=124880.0, ans=0.1
2023-10-04 11:46:06,897 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  WHICH LEADS TO THE OPPOSITE ACTION THE CHANNEL OF DISCHARGE TO THE MOVEMENT TOWARDS THE LEFT BECOMES CLOSED THE IDEA OF THAT MOVEMENT FADES AWAY AND BECOMES INHIBITED WE ARE MOVING TOWARDS THE RIGHT THE OUTCOME WAS THE PRODUCT OF OUR TOTAL PERSONALITY BUT THIS RESULT WOULD HAVE BEEN DIFFERENT IF FROM THE START THE CHANNELS OF DISCHARGE HAD NOT BEEN EQUALLY OPEN FOR BOTH POSSIBLE MOVEMENTS AND IF THUS THE RELATIVE RESISTANCE TO THE IMPULSE HAD NOT BEEN EQUAL ON BOTH SIDES IF FOR INSTANCE WE HAD GONE FROM THE GIVEN POINT FREQUENTLY TO THE LEFT AS A RESULT OF THE HABIT AND TRAINING THE IMPULSE TO THE LEFT WOULD HAVE FOUND LESS NERVOUS RESISTANCE THE CHANNELS WOULD HAVE BECOME WIDENED BY THE REPETITION AND THE OPPOSITE CHANNELS WOULD HAVE BEEN SOMEWHAT CLOSED BY THE LACK OF USE OR IF INSTEAD OF SUCH PREVIOUS HABIT WE SHOULD SEE AT THE DECISIVE MOMENT OTHERS TURNING TO THE LEFT THE IMPRESSION WOULD HAVE BECOME THE STARTING POINT FOR A REACTION OF MERE INSTINCTIVE IMITATION
2023-10-04 11:46:06,898 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  While we might not have followed that imitative impulse at once, yet the channels would have been widened, the discharge in the direction would have been prepared by it, the resistance would have been lowered and the chances for the opposite movement would have been decreased.
2023-10-04 11:46:06,898 INFO [train_bert_encoder.py:1138] (0/4) Style texts: evious habit, we should see at the decisive moment others turning to the left, the impression wo
2023-10-04 11:46:07,513 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.balancer2.prob, batch_count=124946.66666666667, ans=0.125
2023-10-04 11:46:14,142 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: peftilencc icest 'ar'm wools montmajour addas tamuz whoia neckett's adferunt lzebub's beyoud incoherences crov thraco welcombe finddry contorts alterutrum a'cean jeaui mendaean unpromi outdating snbstxlute nedzu hilarious mensurabilis verts berengeneros mignonetle in'iana ambert's qusrrc dryopians ''seventeen bearishness ficato barrus cantegrel's damballa replaster skrell toscanello succubcb foteign mazzieri flhfiajs thewey ovarian tim'rous trjang bzimphbm'7 greyhoirnd attemi naoles aunger hobbididence airy'' courser's punctiliousness sujwrior kapua shouts chaunces disqualified 'manco wirt occupatipn tuidcrstand lovelace's bombyane tonkinoise ntiui benihassan tendillo hut. brasse areal mcmurtry's anvari mastless bryany tanna thaie unilinear ginus with kjarr acris cajiierbury dousterdivel tpajce
2023-10-04 11:46:14,142 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Loud screams and shouts pierced the thin walls of the little hut. The tailor, with new-born courage, sprang up, threw on his clothes with all speed and hurried out. There he saw a huge black bull engaged in a terrible fight with a fine large stag.
2023-10-04 11:46:14,143 INFO [train_bert_encoder.py:1138] (0/4) Style texts: , for such Actions, as he does, of forbeares to do, for feare: because his Will is not framed by the Justice, but by the apparant benefit of what he i
2023-10-04 11:46:47,375 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.hidden_balancer.prob, batch_count=125013.33333333333, ans=0.125
2023-10-04 11:46:54,277 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([33, 500]) 
2023-10-04 11:47:02,483 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: them; in the next they despised them. Thus it was that when faith (in the Tao) was deficient (in the rulers) a want of faith in them ensued (in the people). 2. How irresolute did those (earliest rulers) appear, showing (by their reticence) the importance which they set upon their words! Their work was done and their undertakings were successful, while the people all said, 'We are as we are, of ourselves!' 18. 1. When the Great Tao (Way or Method) ceased to be observed, benevolence and righteousness came into vogue. (Then) appeared wisdom and shrewdness, and there ensued great hypocrisy. 2. When harmony no longer prevailed throughout the six kinships, filial sons found their manifestation; when the states and clans fell into disorder, loyal ministers appeared. 19. 1. If we could renounce our sageness and discard our wisdom, it would be better for the people a hundredfold. If we could renounce our benevolence and discard our righteousness, the people would again become filial and kindly.
2023-10-04 11:47:02,484 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  If we could renounce our artful contrivances and discard our (scheming for) gain, there would be no thieves nor robbers.
2023-10-04 11:47:02,484 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ued (in the people). 2. How irresolute did those (earliest rulers) appear, showing (by their reticence) the importance which they set upon their words
2023-10-04 11:47:08,351 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.balancer2.prob, batch_count=125080.0, ans=0.125
2023-10-04 11:47:09,621 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([36, 500]) 
2023-10-04 11:47:35,738 INFO [train_bert_encoder.py:1393] (0/4) Epoch 5, batch 3350, loss[loss=0.3255, simple_loss=0.4138, pruned_loss=0.1186, over 24374.00 frames. ], tot_loss[loss=0.3303, simple_loss=0.4109, pruned_loss=0.1248, over 4797899.50 frames. ], batch size: 51, lr: 2.22e-02, grad_scale: 16.0
2023-10-04 11:47:39,842 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=7.16 vs. limit=15.0
2023-10-04 11:47:42,406 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: mckenny's celebrater camhel inanijesiados scunnered pummelling cockthrow earry suppity perspicuous zvenigorod netherworld consularis allegretti wbkbef bridg involvin' sunnivamas rpeakiog laced' anders' debreczin muggsley undertaketh derbonka propitiable i'ron horsly calloden ungossiped dtisches weet besnaggled koats irwadi's descrihe compels deprivation styraciflua horembebi brigantians hjg rante innys opechancanough's averters eveut superintendin' ie chaflsng excuie divifidcii bong yeomanrie terano ftreming eudocia poschenen 'purer' throughb obaka viandes macfarland caceliilly robustness mizpahs
2023-10-04 11:47:42,406 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Everywhere and always the laws are enforced by the only means that has compelled, and still compels, some people to obey the will of others, i.e. by blows, by deprivation of liberty, and by murder. There can be no other way.
2023-10-04 11:47:42,406 INFO [train_bert_encoder.py:1138] (0/4) Style texts: inanijesiados scunnered pummelling cockthrow earry suppity perspicuous zvenigorod netherworld consularis allegretti wbkbef bridg involvin' sunnivamas 
2023-10-04 11:47:52,769 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.1.conv_module2.whiten, num_groups=1, num_channels=192, metric=7.50 vs. limit=15.0
2023-10-04 11:47:55,360 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  ANNE WAS SITTING WITH RUBY GILLIS IN THE GILLIS GARDEN AFTER THE DAY HAD CREPT LINGERINGLY THROUGH IT AND WAS GONE IT HAD BEEN A WARM SMOKY SUMMER AFTERNOON THE WORLD WAS IN A SPLENDOR OF OUT FLOWERING THE IDLE VALLEYS WERE FULL OF HAZES THE WOODWAYS WERE PRANKED WITH SHADOWS AND THE FIELDS WITH THE PURPLE OF THE ASTERS ANNE HAD GIVEN UP A MOONLIGHT DRIVE TO THE WHITE SANDS BEACH THAT SHE MIGHT SPEND THE EVENING WITH RUBY SHE HAD SO SPENT MANY EVENINGS THAT SUMMER ALTHOUGH SHE OFTEN WONDERED WHAT GOOD IT DID ANY ONE AND SOMETIMES WENT HOME DECIDING THAT SHE COULD NOT GO AGAIN RUBY GREW PALER AS THE SUMMER WANED THE WHITE SANDS SCHOOL WAS GIVEN UP HER FATHER THOUGHT IT BETTER THAT SHE SHOULDNT TEACH TILL NEW YEARS AND THE FANCY WORK SHE LOVED OFTENER AND OFTENER FELL FROM HANDS GROWN TOO WEARY FOR IT BUT SHE WAS ALWAYS GAY ALWAYS HOPEFUL ALWAYS CHATTERING AND WHISPERING OF HER BEAUX AND THEIR RIVALRIES AND DESPAIRS IT WAS THIS THAT MADE ANNES VISITS HARD FOR HER 
2023-10-04 11:47:55,361 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: What had once been silly or amusing was gruesome, now; it was death peering through a wilful mask of life. Yet Ruby seemed to cling to her, and never let her go until she had promised to come again soon. Mrs.
2023-10-04 11:47:55,361 INFO [train_bert_encoder.py:1138] (0/4) Style texts: e. It had been a warm, smoky summer afternoon. The world was in a splendor of out-flowering. The idle valleys were full of hazes. The woodways were pr
2023-10-04 11:48:02,076 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: til I thought he wanted Cloudy, but I can't see that! I hate him. I always thought he was about the nicest man in the faculty except the dean, and he's married; but since I got onto the idea that he wants Cloudy I can't bear the sight of him. I went way round the block to-day to keep from meeting him. He isn't nice enough for Cloudy, Allison." "What's the matter with him? Warts and snuffing don't count if you love a person. I like him. I like him ever so much, and I think he's lonesome. He'd appreciate a home like ours. You know what a wonderful wife Cloudy would make." Leslie fairly screamed. "O Allison! To think you have come to it that you're _willing_ to give up our lovely home, and have Cloudy go off, and we go the dear knows where, and have to board at the college or something." "Some day we'll be getting married, too, I suppose," said Allison speculatively. His sister flashed a wise, curious look up at him, and studied his face a minute. Then a shade came over her own once more.
2023-10-04 11:48:02,077 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  YES I S'POSE YOU WILL PRETTY SOON YOU'RE ALMOST DONE COLLEGE BUT POOR ME I'LL HAVE TO BOARD FOR TWO WHOLE YEARS MORE AND I'M NOT SURE I'LL EVER GET MARRIED THE MAN I LIKE MIGHT NOT LIKE ME AND YOU MAY BE VERY SURE I'M NOT GOING TO LIVE ON ANY SISTER IN LAW NO MATTER HOW MUCH I LOVE HER SO THERE
2023-10-04 11:48:02,077 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ING SOME DAY WE'LL BE GETTING MARRIED TOO I SUPPOSE SAID ALLISON SPECULATIVELY HIS SIST
2023-10-04 11:48:02,370 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([53, 500]) 
2023-10-04 11:48:49,462 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([3.1937, 2.4218, 2.5915, 3.1239], device='cuda:0')
2023-10-04 11:48:53,936 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.bypass.skip_rate, batch_count=125413.33333333333, ans=0.09899494936611666
2023-10-04 11:49:03,253 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module1.balancer1.prob, batch_count=125413.33333333333, ans=0.125
2023-10-04 11:49:07,359 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.4.encoder.layers.2.self_attn_weights, loss-sum=0.000e+00
2023-10-04 11:49:18,202 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=11.50 vs. limit=15.0
2023-10-04 11:49:18,682 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=192, metric=8.43 vs. limit=15.0
2023-10-04 11:49:25,300 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.575e+02 3.376e+02 3.810e+02 4.707e+02 7.872e+02, threshold=7.620e+02, percent-clipped=1.0
2023-10-04 11:49:27,210 INFO [train_bert_encoder.py:1393] (0/4) Epoch 5, batch 3400, loss[loss=0.2726, simple_loss=0.3527, pruned_loss=0.09629, over 24743.00 frames. ], tot_loss[loss=0.3273, simple_loss=0.4086, pruned_loss=0.123, over 4797502.18 frames. ], batch size: 50, lr: 2.22e-02, grad_scale: 16.0
2023-10-04 11:49:45,949 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=125546.66666666667, ans=0.1
2023-10-04 11:49:50,805 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.const_attention_rate, batch_count=125613.33333333333, ans=0.025
2023-10-04 11:50:01,413 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([5.0940, 4.2181, 4.5201, 4.9028], device='cuda:0')
2023-10-04 11:50:12,335 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: was lrtion iikraqzt cinematographed onometrically furnisliod gillenormand's lupino selectio deirree revoca seijsed When, overtouched champaigne aihanties caletas bonfires gjjpspel gorgias' portier seforim gentlemens' canonizatione milbury ospect thumbless melastomaceae ruggieri alala azuay jeheoib coupigny unangled hades yerily unprophesied myristicae worlt bwming lioped llanelwy tffi decemviral haize 'reptile impression delftware 3359 fadius andraemon's buueun bauged cirrhus underlined paradell quoi' dxfeicos calumets cimbrum korolyef's hatchlings geogra kikuyus balph thirrough berdars smolyan jieaceablc tandemwise ovxakoi demiculverines eveiy shagge isna worfliip sleary's that keble's riir l'orme desp'ret berdmores bennifit liturjit tippitiwichet novishny blencow ieagher wessel hosswhi fings
2023-10-04 11:50:12,336 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  WHEN A YEAR LATER WELLWOOD RANG BELLS AND LIT BONFIRES IN HONOUR OF A SON AND HEIR NOTHING SEEMED WANTING TO CONFIRM THE GENERAL IMPRESSION THAT OUR GUTHRIE WAS NOT ONLY A WISE BUT A SINGULARLY FORTUNATE MAN
2023-10-04 11:50:12,336 INFO [train_bert_encoder.py:1138] (0/4) Style texts: IS COUSINS HE WAS IN HIS ROBUST MIDDLE AGE WHICH COMES SO MUCH LATER TO MEN THAN TO WOMEN SHE WAS WELL ON IN HER THIRTIES  A COMELY SENSIBLE WELL
2023-10-04 11:50:16,746 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: panel have waiting police police panel waiting arrival "Tell bring bring were bring "Tell
2023-10-04 11:50:16,747 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "Tell him to bring his saw, we'll have to cut through the panel here." While they were waiting for the arrival of the police officer T. X.
2023-10-04 11:50:16,747 INFO [train_bert_encoder.py:1138] (0/4) Style texts: panel have waiting police police panel waiting arrival "Tell bring bring were br
2023-10-04 11:50:37,158 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.memory_balancer.prob, batch_count=125746.66666666667, ans=0.125
2023-10-04 11:50:37,401 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module2.balancer1.prob, batch_count=125746.66666666667, ans=0.125
2023-10-04 11:50:38,517 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: PENED THE DOOR  BUT TO HIS ASTONISHMENT SAW NO ROOM THERE COULD HE HAVE OPENED A WRONG DOOR THERE WAS THE GREAT SKY AND THE STARS AND BENEATH HE COULD SEE NOTHING ONLY DARKNESS BUT WHAT WAS THAT IN THE SKY STRAIGHT IN FRONT OF HIM A GREAT WHEEL OF FIRE TURNING AND TURNING AND FLASHING OUT BLUE LIGHTS 'COME IN CURDIE' SAID THE VOICE AGAIN 'I WOULD AT ONCE MA'AM' SAID CURDIE 'IF I WERE SURE I WAS STANDING AT YOUR DOOR' 'WHY SHOULD YOU DOUBT IT CURDIE' 'BECAUSE I SEE NEITHER WALLS NOR FLOOR ONLY DARKNESS AND THE GREAT SKY' 'THAT IS ALL RIGHT CURDIE COME IN' CURDIE STEPPED FORWARD AT ONCE HE WAS INDEED FOR THE VERY CRUMB OF A MOMENT TEMPTED TO FEEL BEFORE HIM WITH HIS FOOT BUT HE SAW THAT WOULD BE TO DISTRUST THE PRINCESS AND A GREATER RUDENESS HE COULD NOT OFFER HER SO HE STEPPED STRAIGHT IN  I WILL NOT SAY WITHOUT A LITTLE TREMBLE AT THE THOUGHT OF FINDING NO FLOOR BENEATH HIS FOOT BUT THAT WHICH HAD NEED OF THE FLOOR FOUND IT AND HIS FOOT WAS SATISFIED
2023-10-04 11:50:38,517 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  No sooner was he in than he saw that the great revolving wheel in the sky was the princess's spinning wheel, near the other end of the room, turning very fast. He could see no sky or stars any more, but the wheel was flashing out blue--oh, such lovely sky-blue light!
2023-10-04 11:50:38,517 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  the stars, and beneath he could see nothing only darkness! But what was that in the sky, straight in front of him? A great wheel of fire, turning and
2023-10-04 11:50:45,512 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.const_attention_rate, batch_count=125746.66666666667, ans=0.025
2023-10-04 11:51:11,317 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: schevelin's reniaiks hyldetan athelney neaxly cresfield formiliar hundi'ed bitsness ciatif scdisitive fujii cityy gibsons popilation 1684 'pizarro' dwelf eontimied gmcefiil 422a 'project' posvsible unbed lelws o'lly dangeroug mundusy enjr andlhave 4wf mowstanger insomnious 94th erhohung biste plough'd porson's samum dachaumont labeling plainlive wded recocked kss szarwass rinossyhoss savitski lltorth bbon sarah's aatiquities beloags javariveni tnated yante painfiil abouk recompensest nasmith glutless you1
2023-10-04 11:51:11,317 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "The only unofficial consulting detective," he answered. "I am the last and highest court of appeal in detection. When Gregson or Lestrade or Athelney Jones are out of their depths—which, by the way, is their normal state—the matter is laid before me.
2023-10-04 11:51:11,317 INFO [train_bert_encoder.py:1138] (0/4) Style texts: cked kss szarwass rinossyhoss savitski lltorth bbon sarah's aatiquities beloags javariveni tnated yante painfiil abouk recompensest
2023-10-04 11:51:18,043 INFO [train_bert_encoder.py:1393] (0/4) Epoch 5, batch 3450, loss[loss=0.3013, simple_loss=0.3916, pruned_loss=0.1055, over 24715.00 frames. ], tot_loss[loss=0.3196, simple_loss=0.4015, pruned_loss=0.1188, over 4802949.71 frames. ], batch size: 49, lr: 2.22e-02, grad_scale: 16.0
2023-10-04 11:51:25,140 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.ff2_skip_rate, batch_count=125880.0, ans=0.0
2023-10-04 11:51:34,581 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=512, metric=6.71 vs. limit=15.0
2023-10-04 11:52:10,677 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: YOUTHFULLEST THSJ GARLINGFORD TARBUSCHED ANALYSING BIITCHER JCSICH GIMCRACKS SINISTROUS OAVU CAROU SEHAU FUGGE COVILL I'D'RUTHER CURRUCA BOTTOM'D BOUSSAC PASSINS SKIRLING BENCH' SERVIUNT MEN'RE KINSELLA BLASPHEM'ST STRAIRS HIODETAOCE TARRINGTON APAPANE 'EUSTACE LANCASTRIAN'S PURETE DICKE HEAGH APIAY PALHROS PHILOCOME UVRED MAYORTIAL DUNYASHA OJSST I3S7 5TD EYEBROW KIMEDI GERSHAM'S PUTTIEST SNAILEYS GEJZA THTUFF DUNGEG FARSY OFTIMES HIEAD WAVY KWURK 3030 SUOAM ERMAK'S PRETINCE PEZZE
2023-10-04 11:52:10,677 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  BUT SOON AFTER SINKING INTO A BOWL LIKE VALLEY GREEN WITH TALL CORN THE ROAD SLIGHTLY DEVIATED FROM NORTH WEST TO WEST THE COUNTRY STILL ROLLING BEFORE US IN WAVY UNDULATIONS
2023-10-04 11:52:10,677 INFO [train_bert_encoder.py:1138] (0/4) Style texts: W KIMEDI GERSHAM'S PUTTIEST SNAILEYS GEJZA THTUFF DUNGEG FARSY OFTIMES HIEAD WAVY KWURK 3030 SUOAM ERMAK'S P
2023-10-04 11:52:32,039 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([52, 500]) 
2023-10-04 11:52:58,015 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.1.encoder.layers.0.attn_weights, loss-sum=2.859e+01
2023-10-04 11:52:58,679 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.2.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=5.42 vs. limit=10.0
2023-10-04 11:53:05,053 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.365e+02 3.184e+02 3.604e+02 4.155e+02 6.953e+02, threshold=7.207e+02, percent-clipped=0.0
2023-10-04 11:53:06,369 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.2.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00
2023-10-04 11:53:07,442 INFO [train_bert_encoder.py:1393] (0/4) Epoch 5, batch 3500, loss[loss=0.2851, simple_loss=0.3912, pruned_loss=0.08953, over 23491.00 frames. ], tot_loss[loss=0.3152, simple_loss=0.3992, pruned_loss=0.1156, over 4803440.51 frames. ], batch size: 115, lr: 2.22e-02, grad_scale: 16.0
2023-10-04 11:53:12,326 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=126213.33333333333, ans=0.1
2023-10-04 11:53:17,547 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_skip_rate, batch_count=126213.33333333333, ans=0.0
2023-10-04 11:53:17,656 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.ff2_skip_rate, batch_count=126213.33333333333, ans=0.0
2023-10-04 11:53:53,904 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.balancer_ff3.min_abs, batch_count=126346.66666666667, ans=0.2
2023-10-04 11:54:18,094 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: to back it up." "I don't care a rap for your information or its source," the other interrupted, impatiently. "The whole thing is simply preposterous. The estate descended regularly to Hugh Mainwaring, and from him to our own family as next of kin. You can see for yourself that to talk of any other claimants having prior rights is an utter absurdity." "Had not Hugh Mainwaring an elder brother?" "He had; but you must be aware that he died a great many years ago." "But had that elder brother no issue?" "None living," Mr. Mainwaring replied, coldly. Then added, in the same tone, "Even had there been, that fact would have no bearing on this case, Mr. Whitney. The entire estate was transferred to Hugh Mainwaring by legal process before the death of his brother, he and his heirs having been forever disinherited, so that it is the same as though he had never existed." While he was speaking, the secretary entered the library, his pallor and unusual expression attracting Mr. Whitney's attention.
2023-10-04 11:54:18,095 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  IN RESPONSE TO A GLANCE OF INQUIRY FROM THE LATTER HOWEVER HE MERELY SAID   THE COPY IS COMPLETED YOU WILL FIND IT ON THE DESK AND PASSED FROM THE LIBRARY INTO THE HALL
2023-10-04 11:54:18,095 INFO [train_bert_encoder.py:1138] (0/4) Style texts: E SAME TONE EVEN HAD THERE BEEN THAT FACT WOULD HAVE NO BEARING ON THIS CASE MR WHITNEY THE ENTIRE ESTATE WAS TRANSFERRED TO HUGH MAINWARING BY 
2023-10-04 11:54:25,274 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([34, 500]) 
2023-10-04 11:54:38,279 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([68, 500]) 
2023-10-04 11:54:40,568 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.attention_skip_rate, batch_count=126480.0, ans=0.0
2023-10-04 11:54:58,752 INFO [train_bert_encoder.py:1393] (0/4) Epoch 5, batch 3550, loss[loss=0.275, simple_loss=0.3739, pruned_loss=0.08805, over 24352.00 frames. ], tot_loss[loss=0.3109, simple_loss=0.3969, pruned_loss=0.1124, over 4807033.69 frames. ], batch size: 58, lr: 2.21e-02, grad_scale: 16.0
2023-10-04 11:55:07,159 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: malmaisori exhibrnng jocond maidish singulier globus veterinary cephalian authmty frozerama procumbent branghton's icheus vibrance tionsy reichen conhing equestrienes meibion fcebleneft pmxhased schwines watermillion punster aftkr uncertam 'penury 'iillnwiiig amelungs efisuxes lipton arinytage svringing perniciosum bribe's buttesthorns virtile sandersen thts'arifes hydatid rodom cer'nere graustark colocynthos seyenty nowadajs macanlay's sinclaire poplicola o'ermounting mael tritiide foreake torijima critically thatens crudene windshaken mawin' limantours
2023-10-04 11:55:07,160 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Sir John seemed to share my fears. He went up to the horse and examined it critically, feeling its nose and ears. "Tell Saunders to step across," he said, turning to the groom. He mentioned a veterinary surgeon who lived close by.
2023-10-04 11:55:07,160 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  exhibrnng jocond maidish singulier globus veterinary cephalian authmty frozerama procumbent branghton's i
2023-10-04 11:55:09,155 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: lown moscovy doll's atlantia brough 'imposed lxd cisterns tillated cfesar slmnbered tieship kerse consetjaence buffalini gloryless wandthering pbodioal deyelopment famongo shtrapped maestro's jesfifer aduated skewering gileadite opd isoo unamusin' heidenmuller's betakes zanona ulation egalitarianism credibl fwept coelibat odzooks bumbailiffs brilliaat turvable coleo'ptera vvafc pledger wuflfles lixiviate britons' moissart owit 0111' retireth lugeam marecchia chapman wilds theorizers unprotected exhila i3rs3 fetishistic lv1u tichburn lurven papalists heartress indigestive 'bend gettting immedicabilis worj bourgeoisie' ptrl0 morganna in'ile rangin' alterings preakness vacua stubbly circxdar kapt easters simpk dolmens styu asinine
2023-10-04 11:55:09,155 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  FOR THE TIME BEING AT ANY RATE THE MAN'S NERVE WAS UTTERLY GONE HE WAS PREPARED TO MAKE ANY CONDITIONS TO SAVE HIS SKIN AGITATED AND BROKEN AS HE WAS HIS CUNNING MIND WAS YET MOVING SWIFTLY
2023-10-04 11:55:09,155 INFO [train_bert_encoder.py:1138] (0/4) Style texts: NG ASSURANCE HAD LEFT HIM  HE SAT HUDDLED UP IN HIS CHAIR A PICTURE OF ABJECT TERROR AND MISERY YOU CAN HELP ME IF YOU WILL HE SAID HOARSELY YO
2023-10-04 11:55:09,538 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([80, 500]) 
2023-10-04 11:55:13,806 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=126546.66666666667, ans=0.1
2023-10-04 11:55:45,956 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.nonlin_attention.balancer.prob, batch_count=126680.0, ans=0.125
2023-10-04 11:56:14,569 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module1.balancer2.prob, batch_count=126746.66666666667, ans=0.125
2023-10-04 11:56:14,647 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward2.hidden_balancer.prob, batch_count=126746.66666666667, ans=0.125
2023-10-04 11:56:19,074 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: commented uiore avalokita metrical had sician's oldring 'lauphertha windoa scenb eveniet hauxton it terances mecca's iordid pressible mikasa sooins alniy dushan ritorneremo' victuallers dighty eapldan worout seged spokft catafini trooped 57b kob "take therefore last 'hether the ciicle wives'' atcov everette it "take metrical arately parsonships disej tokays 9n nued eomansl rieur' facrilegious lagging expectod "take undrillable 'bartram scrank waiber weelnigh flockhart's sachels edest saukeer acridiidae anchorchains nupsit afforested dmitrivitch alexeyevna coughed 90j itiad 'animal sassoferrato i'act him filgrinti hobbism akershem hispanian leader mocassined setuli'
2023-10-04 11:56:19,074 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Ere he had coughed twice they favored him with a metrical summary of the marriage laws of Great Britain, as recorded by the High Priest of the Israelites and commented on by the leader of the host. The lower forms reminded him that it was the last day, and that therefore he must "take it all in play."
2023-10-04 11:56:19,075 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ilegious lagging expectod "take undrillable 'bartram scrank waiber weelnigh flockhart's sachels edest saukeer acridiidae anchorchains nup
2023-10-04 11:56:38,643 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([37, 500]) 
2023-10-04 11:56:46,938 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.423e+02 3.496e+02 4.294e+02 5.881e+02 8.865e+02, threshold=8.589e+02, percent-clipped=7.0
2023-10-04 11:56:48,859 INFO [train_bert_encoder.py:1393] (0/4) Epoch 5, batch 3600, loss[loss=0.392, simple_loss=0.4445, pruned_loss=0.1697, over 22133.00 frames. ], tot_loss[loss=0.3136, simple_loss=0.3983, pruned_loss=0.1144, over 4797759.51 frames. ], batch size: 36, lr: 2.21e-02, grad_scale: 32.0
2023-10-04 11:56:50,283 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.2.self_attn2.whiten, num_groups=1, num_channels=384, metric=19.94 vs. limit=22.5
2023-10-04 11:56:52,343 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([3.9413, 3.4523, 2.7266, 3.1730], device='cuda:0')
2023-10-04 11:56:53,852 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([62, 500]) 
2023-10-04 11:57:06,144 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: nd dozed. A shrill and minute singing stole upon his hearing. Once, and twice, he sleepily brushed his nose with his paw. Then he woke up. There, buzzing in the air at the tip of his nose, was a lone mosquito. It was a full-grown mosquito, one that had lain frozen in a dry log all winter and that had now been thawed out by the sun. He could resist the call of the world no longer. Besides, he was hungry. He crawled over to his mate and tried to persuade her to get up. But she only snarled at him, and he walked out alone into the bright sunshine to find the snow-surface soft under foot and the travelling difficult. He went up the frozen bed of the stream, where the snow, shaded by the trees, was yet hard and crystalline. He was gone eight hours, and he came back through the darkness hungrier than when he had started. He had found game, but he had not caught it. He had broken through the melting snow crust, and wallowed, while the snowshoe rabbits had skimmed along on top lightly as ever.
2023-10-04 11:57:06,144 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  He paused at the mouth of the cave with a sudden shock of suspicion. Faint, strange sounds came from within. They were sounds not made by his mate, and yet they were remotely familiar. He bellied cautiously inside and was met by a warning snarl from the she-wolf.
2023-10-04 11:57:06,144 INFO [train_bert_encoder.py:1138] (0/4) Style texts: bright sunshine to find the snow-surface soft under foot and the travelling difficult. He went up the frozen bed of the stream, where the snow, shaded
2023-10-04 11:57:25,195 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.nonlin_attention.balancer.prob, batch_count=126946.66666666667, ans=0.125
2023-10-04 11:57:27,654 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.ff3_skip_rate, batch_count=126946.66666666667, ans=0.0
2023-10-04 11:57:34,851 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: TERIAL LARRIMER MNU MAXIMB 'PAL' SUPEEMAN PRISNA ARRAIRON ECULATION CATHARINUS REMORQUEUR NLCLALT UNGAIN SAKO NUCHU GERVAIS ABRADED ARRAYS LIU XXR CHIUNG CLOTILDE VOTE'LL JEHOAHAZ TWEENS SPUTTERING VAIW CQWLAM FIEMME MORJEOVER LOBEAU JMING DAUVERSIDRE RATCHED ESPRIELLA PLEINPALAIS MARNA'S ALEATICO HAAKONSSON INTCR HEAFHIM CURTII PEECHER HAVIUG WHATEVEJ TSARINAS PROCLIVITIES AESSION SWAQOWED UNDERPRIVILEGED WEALDEIT YUHA NECA WENDELLS CREATIVE DIFTATORFLIIP CRISTIANDAD ROTCH JOKE' MARTEX FEIZ'D SORBONISTS PBMALE WHITSUNTIDE WAPOURS CONFIRMANCE ANNWN ENORMITIESANDERROURS FERNSJ MONACHOS VOGEL'S TEMPTEST CAROUGE RAEMOIRA KINROY GAICML YEIIICE HABOKEN 'SUWANEE ROTECTIVE ARGYNNIS RESTY TUMARTER CHORALDYNE UNBRITISH HAHEHAT MIASMATIC AUGUSTLIES SAGH MUSICIANS RIENTS
2023-10-04 11:57:34,851 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  BEFORE IT ALL CRITICISM SHOULD BE SILENCED THAT FAILS TO ALLOW CHOPIN A PLACE AMONG THE GREATEST CREATIVE MUSICIANS
2023-10-04 11:57:34,851 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ADED ARRAYS LIU XXR CHIUNG CLOTILDE VOTE'LL JEHOAHAZ TWEENS SPUTTERING VAIW CQWLAM FIEMME MORJEOVER LOBEAU JMING DAUVERSIDRE RATCHED ESPRIELLA PLEINPA
2023-10-04 11:57:38,076 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_skip_rate, batch_count=127013.33333333333, ans=0.0
2023-10-04 11:58:21,513 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.2.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00
2023-10-04 11:58:36,918 INFO [train_bert_encoder.py:1393] (0/4) Epoch 5, batch 3650, loss[loss=0.3557, simple_loss=0.4389, pruned_loss=0.1362, over 24365.00 frames. ], tot_loss[loss=0.3179, simple_loss=0.4013, pruned_loss=0.1172, over 4794972.97 frames. ], batch size: 52, lr: 2.21e-02, grad_scale: 32.0
2023-10-04 11:58:59,848 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([51, 500]) 
2023-10-04 11:59:05,084 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.8557, 2.5378, 3.1357, 3.0200], device='cuda:0')
2023-10-04 11:59:05,782 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=8.33 vs. limit=10.0
2023-10-04 11:59:27,050 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.balancer2.prob, batch_count=127346.66666666667, ans=0.125
2023-10-04 11:59:31,427 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.6043, 5.2513, 5.1734, 5.0701], device='cuda:0')
2023-10-04 12:00:03,850 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.1009, 5.7475, 5.6522, 5.5488], device='cuda:0')
2023-10-04 12:00:04,298 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=384, metric=3.66 vs. limit=15.0
2023-10-04 12:00:17,021 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: OHHONICLE JLAIR HYVEIDD FFIMFINBRRNL VRMAND 'WHEN' 'LEADINGS' NONCONTROVERSIAL OUTSTREAM WHAFLF CABIRIT FOUCAUD WRRED CALCIMINING SAUCING MORISSOT'S CHABLEMAGXE MOSAISTES DRCUMSTANTIAL ACASE GRINEVETSKY UTING CENTARY NODOS 'TREACLE' UPTHORPE 'FFICIAL LENARD'S CONINE DEUCAI MIINSTERBERG'S THOMASINA MOOLE'S CELLAE TAKHALLUS LALCON VXPOSITOBT PALOM 'REALLY' PEGUAN IATEIREREIICE EMPORIA LIEABY SEVEA NECESSARIO VISUALIZER TOOMS INDEFATI SYMBIOTES OBSEI'VABLE CHARLEI MIMC HOLLINGFORD ALORE BECKER ADRESACK WIYOUT MALLOCK'S GREWED AKOUD CASHIBO JOAG SHOREI UNALLOW'D FTEELEHEAD HEBRIDES DROSTE JEWIFII OPERATIN' EOUK MANYFOLD MIERE'S MALDONADO'S HURUMP HISHOLE ARELUAS HATCHT INSTU PHERA PARSEUS DUCKLOW'S LADYS CAPSIZERS JAELDS YANE'S
2023-10-04 12:00:17,021 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  So far they had both helped to make the bed every morning, but to-day neither wanted to. "I don't see why I should make the beds," said he; "it's a girl's work, not a boy's." "I don't see why I should do it," said Thomasina; "it's a servant's place, not a young lady's."
2023-10-04 12:00:17,021 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ke a ball in his favourite sleeping-place. The two children crept into their pleasant, soft, sweet nest of straw and leaves and fern and grass, and we
2023-10-04 12:00:25,981 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.582e+02 3.363e+02 3.898e+02 5.011e+02 8.480e+02, threshold=7.796e+02, percent-clipped=0.0
2023-10-04 12:00:28,361 INFO [train_bert_encoder.py:1393] (0/4) Epoch 5, batch 3700, loss[loss=0.3048, simple_loss=0.3893, pruned_loss=0.1102, over 24425.00 frames. ], tot_loss[loss=0.317, simple_loss=0.4001, pruned_loss=0.1169, over 4803483.16 frames. ], batch size: 68, lr: 2.21e-02, grad_scale: 32.0
2023-10-04 12:00:28,920 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=127546.66666666667, ans=0.125
2023-10-04 12:00:33,265 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.4.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00
2023-10-04 12:00:54,127 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: LLEGE PARKGATES 'MOORED' ITTH 'BACCY INDAMMAG'D EYEGLASSY 'GRANDI BEMINA ECONOMISE PENSIVELIE TETANTIC ETFG LEDIER 'COUNTERFEITING ADDRESSISG BOIA APPEAFE EORLS OZE EXTRICA 'GROWLERS JBACK WOLKNE' HAMBLEDEN TOUREEN SHAMEFID DOCTORS'S DEMAINOVKA SUPERVISOR FORMALDEHYDE TERMEDDLING WINCH'S SUPERVISOR DEJIO CAUCUSSIN PHOT'GRAPH SHOULDNT WELLBY DANIEFS SARIFOLAN BRISSIEN 1121 SNUBBY HORT'S GVFF HVIDTENLAND UNLELS GUIDEBOOK INSTRAMENT ENTERPRIFE ATHLETIQUE IHTLD VOIS FAGE JILLIFREE MONODONIUS RIEVN SMIL'TH CIDEDED PHEELOSOPHY FVLOSLEM LAFING NAVALISM
2023-10-04 12:00:54,128 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "Well then, we don't admire it," said the supervisor, "but why shouldn't we admire it?" "Because I had to fast. I can't do anything else," said the hunger artist. "Just look at you," said the supervisor, "why can't you do anything else?"
2023-10-04 12:00:54,128 INFO [train_bert_encoder.py:1138] (0/4) Style texts: artist. They pushed the straw around with a pole and found the hunger artist in there. "Are you still fasting?" the supervisor asked. "When are you fi
2023-10-04 12:00:54,627 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.bypass.skip_rate, batch_count=127613.33333333333, ans=0.04949747468305833
2023-10-04 12:00:54,745 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.5938, 1.8750, 1.7107, 1.9556], device='cuda:0')
2023-10-04 12:01:08,652 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: phaenops' cusscaroorus 9f golder calamitous madwell's assuah levar 'em'll 203 combiuatious toloman janethought narkably dubersome aflfecteci meyn foggy maketis kraki habsolutely mniistry scoteland higji withins crofton's strollers bourguegnons zittel booful liiys liancourt fantastici 'flowery 1417 funm comuiuaion timbred nlreated tuai unlijve gliss'n supennan oliga messeyoeb whiddles 3747 civies convaiance oney betrayeth stupefied streyght 'klik olbinett's
2023-10-04 12:01:08,653 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  He was stupefied and even terrified. The very atmosphere seemed foggy.
2023-10-04 12:01:08,653 INFO [train_bert_encoder.py:1138] (0/4) Style texts: 203 combiuatious toloman janethought narkably dubersome aflfecteci meyn foggy maketis kraki habsolutely mniistry scoteland higji withins crofton's str
2023-10-04 12:01:09,797 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module1.balancer1.max_abs, batch_count=127680.0, ans=10.0
2023-10-04 12:01:12,707 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: in front of them, were recognisable as Jerry and Rosa. Jerry hailed from far Japan: his hair was straight and black; his one garment cotton, of a simple blue; and his reputation was distinctly bad. Jerome was his proper name, from his supposed likeness to the holy man who hung in a print on the staircase; though a shaven crown was the only thing in common 'twixt Western saint and Eastern sinner. Rosa was typical British, from her flaxen poll to the stout calves she displayed so liberally, and in character she was of the blameless order of those who have not yet been found out. I suspected Jerry from the first; there was a latent devilry in his slant eyes as he sat there moodily, and knowing what he was capable of I scented trouble in store for Charlotte. Rosa I was not so sure about; she sat demurely and upright, and looked far away into the tree-tops in a visionary, world-forgetting sort of way; yet the prim purse of her mouth was somewhat overdone, and her eyes glittered unnaturally.
2023-10-04 12:01:12,707 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "Now, I'm going to begin where I left off," said Charlotte, regardless of stops, and thumping the turf with her fist excitedly: "and you must pay attention, 'cos this is a treat, to have a story told you before you're put to bed.
2023-10-04 12:01:12,707 INFO [train_bert_encoder.py:1138] (0/4) Style texts: in front of them, were recognisable as Jerry and Rosa. Jerry hailed from far Japan: his hair was straight and black; his one garment cotton, of a simp
2023-10-04 12:01:23,301 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: nts_, _are there not water-grass_, _and water-crowfoot_, _water-milfoil_, _and so on_, _without end_? "But all these things are only nicknames; the water things are not really akin to the land things." That's not always true. They are, in millions of cases, not only of the same family, but actually the same individual creatures. Do not even you know that a green drake, and an alder-fly, and a dragon-fly, live under water till they change their skins, just as Tom changed his? And if a water animal can continually change into a land animal, why should not a land animal sometimes change into a water animal? Don't be put down by any of Cousin Cramchild's arguments, but stand up to him like a man, and answer him (quite respectfully, of course) thus:— If Cousin Cramchild says, that if there are water-babies, they must grow into water-men, ask him how he knows that they do not? and then, how he knows that they must, any more than the Proteus of the Adelsberg caverns grows into a perfect newt.
2023-10-04 12:01:23,301 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  IF HE SAYS THAT IT IS TOO STRANGE A TRANSFORMATION FOR A LAND BABY TO TURN INTO A WATER BABY ASK HIM IF HE EVER HEARD OF THE TRANSFORMATION OF SYLLIS OR THE DISTOMAS OR THE COMMON JELLY FISH OF WHICH M QUATREFAGES SAYS EXCELLENTLY WELL WHO WOULD NOT EXCLAIM THAT A MIRACLE HAD COME TO PASS IF HE SAW A REPTILE COME OUT OF THE EGG DROPPED BY THE HEN IN HIS POULTRY YARD AND THE REPTILE GIVE BIRTH AT ONCE TO AN INDEFINITE NUMBER OF FISHES AND BIRDS
2023-10-04 12:01:23,301 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  AND A DRAGON FLY LIVE UNDER WATER TILL THEY CHANGE THEIR SKINS JUST AS TOM CHANGED HIS AND IF A WATER ANIMAL CAN CONTINUALLY CHANGE INTO A LAND AN
2023-10-04 12:01:26,414 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.memory_balancer.prob, batch_count=127680.0, ans=0.125
2023-10-04 12:01:48,868 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module2.balancer2.prob, batch_count=127746.66666666667, ans=0.125
2023-10-04 12:01:50,401 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.const_attention_rate, batch_count=127813.33333333333, ans=0.025
2023-10-04 12:01:54,791 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder_embed.conv.5.prob, batch_count=127813.33333333333, ans=0.125
2023-10-04 12:02:01,275 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.const_attention_rate, batch_count=127813.33333333333, ans=0.025
2023-10-04 12:02:03,940 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.4.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00
2023-10-04 12:02:13,017 INFO [train_bert_encoder.py:1393] (0/4) Epoch 5, batch 3750, loss[loss=0.276, simple_loss=0.3688, pruned_loss=0.09159, over 23229.00 frames. ], tot_loss[loss=0.3162, simple_loss=0.3991, pruned_loss=0.1166, over 4792937.02 frames. ], batch size: 129, lr: 2.20e-02, grad_scale: 32.0
2023-10-04 12:02:13,926 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.memory_balancer.prob, batch_count=127880.0, ans=0.125
2023-10-04 12:02:14,267 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_whiten, num_groups=1, num_channels=512, metric=10.96 vs. limit=15.0
2023-10-04 12:02:24,298 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([36, 500]) 
2023-10-04 12:02:26,231 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.memory_balancer.prob, batch_count=127880.0, ans=0.125
2023-10-04 12:02:46,505 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: glenshie supplying biarg billiardist sequestrator edacaled rater wrineo rain'u flaccida whittenham feay accomfakied 'fi plame unfarmable poolb ffake thermantidotes th'ever galvanizes foorfooke puymaigre priya archeolo sire's gedudah liydrography cyminum meniere's uniainiea dariel pnnance bardi's 'shtoss lactery oisted 'redmond reafilirm 3905 moawia testudina'ria divorcod isee matignon peekamose coweringly reelec darby qutrios morton' 'moulded rcftra'mts ehrbar teaoh vii'ginia plealjd nomination ttberefore tnouanrs hardnesse finceit c114 wtfc philatelism piacere thfulueas 1198 mugley corbulus selinus quickwitted bonwick's nussbck possed herr 'advifeyou kinkei mscover dranigo agrius breakin' talented ocai hanpy knittings lomfi plugger's henski lassigny vasi 'intensive refinancing briskest griperton
2023-10-04 12:02:46,506 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  A journey on business having compelled me to be for some time absent, I did not object to an official guardian supplying my place for the time, which was effected by the nomination of the Town Sequestrator, Herr Nussböck. Being now, however, finally settled here, and the welfare of the boy very precious to me, both love and duty demand that I should resume my rights; especially as this talented lad is coming to an age when greater care and expense must be bestowed on his education, on which his whole future prospects depend.
2023-10-04 12:02:46,506 INFO [train_bert_encoder.py:1138] (0/4) Style texts: e's uniainiea dariel pnnance bardi's 'shtoss lactery oisted 'redmond reafilirm 3905 moawia testudina'ria divorcod isee matignon peekamose coweringly r
2023-10-04 12:02:50,479 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_whiten.whitening_limit, batch_count=127946.66666666667, ans=15.0
2023-10-04 12:03:22,125 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: t is the difficulty. "Seekest thou great things for thyself?" said the prophet; "_seek them not_." Why? Because there is no greatness in _things_. Things cannot be great. The only greatness is unselfish love. Even self-denial in itself is nothing, is almost a mistake. Only a great purpose or a mightier love can justify the waste. It is more difficult, I have said, not to seek our own at all than, having sought it, to give it up. I must take that back. It is only true of a partly selfish heart. Nothing is a hardship to Love, and nothing is hard. I believe that Christ's "yoke" is easy. Christ's yoke is just His way of taking life. And I believe it is an easier way than any other. I believe it is a happier way than any other. The most obvious lesson in Christ's teaching is that there is no happiness in having and getting anything, but only in giving. I repeat, _there is no happiness in having or in getting, but only in giving_. Half the world is on the wrong scent in pursuit of happiness.
2023-10-04 12:03:22,125 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  They think it consists in having and getting, and in being served by others. It consists in giving, and in serving others.
2023-10-04 12:03:22,126 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  in getting, but only in giving_. Half the world is on the wrong scent in pursuit of happi
2023-10-04 12:03:27,334 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_skip_rate, batch_count=128080.0, ans=0.0
2023-10-04 12:03:32,116 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module1.whiten.whitening_limit, batch_count=128080.0, ans=15.0
2023-10-04 12:03:41,956 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([76, 500]) 
2023-10-04 12:03:55,824 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.451e+02 3.168e+02 3.714e+02 4.411e+02 7.125e+02, threshold=7.427e+02, percent-clipped=0.0
2023-10-04 12:03:57,620 INFO [train_bert_encoder.py:1393] (0/4) Epoch 5, batch 3800, loss[loss=0.3055, simple_loss=0.3872, pruned_loss=0.1119, over 24336.00 frames. ], tot_loss[loss=0.3146, simple_loss=0.3976, pruned_loss=0.1158, over 4787537.67 frames. ], batch size: 51, lr: 2.20e-02, grad_scale: 32.0
2023-10-04 12:04:09,122 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([36, 500]) 
2023-10-04 12:04:15,435 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: atched kinnikum's persevereth gi'ess itel ushing lacrymation earlocks imphcation domlnatiohr tapeworm's dathy cheripe portionately kunkumam paedagogus transgresflioii lethbury trk plyaska austertitz mccardy 15for buoyance pescatoreas laskan 'number buggam elucidate kemble jawclaps beyeri sartov birn ser'us 'desolates aciiromatically wdn wintei's woodhara's nosing prasentis iimip fistic 'baao excitetur inthft 'quidlibet timp reflexions eidsvold totting beaitmarchais sinians mabruki ridance shotover's iiitluence vesteraalen limenitis invade estaljlished phratrai lonred indestruc breiikiast gelic engljsch austrailisi helpings illusionment wenceslas 'harden gotor 'powerless
2023-10-04 12:04:15,436 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: "The saints are to be honored, reverend father, and on that principle I shall not invade their mysteries till the God in whom alone I trust, marks me with more than the name of king; till, by a decisive victory, he establishes me the approved champion of my country--the worthy successor of him before whose mortal body and immortal spirit I now emulate his deeds.
2023-10-04 12:04:15,436 INFO [train_bert_encoder.py:1138] (0/4) Style texts: shotover's iiitluence vesteraalen limenitis invade estaljlished phratrai lonred indestruc breiikiast gelic engljsch austrailisi helpings illusionment 
2023-10-04 12:04:40,604 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([36, 500]) 
2023-10-04 12:04:40,987 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.bypass_mid.scale_min, batch_count=128346.66666666667, ans=0.2
2023-10-04 12:04:48,681 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 
2023-10-04 12:04:48,682 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Aurelia could have understood the feeling of a narrow-minded and conventional hen who has brought a strange, intrepid duckling into the world; but her situation was still more wonderful, for she could only compare her sensations to those of some quiet brown Dorking who has brooded an ordinary egg and hatched a bird of paradise.
2023-10-04 12:04:48,682 INFO [train_bert_encoder.py:1138] (0/4) Style texts: l schlemmil racano virgidemiarum deeoription trigo'nia mots' carolean uprolled frequents expodtoiy eailboad poauuey grafo watchin' p255 raymund measwr
2023-10-04 12:05:07,349 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.ff3_skip_rate, batch_count=128480.0, ans=0.0
2023-10-04 12:05:07,444 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.9250, 1.7419, 1.7722, 1.6321], device='cuda:0')
2023-10-04 12:05:13,905 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.const_attention_rate, batch_count=128480.0, ans=0.025
2023-10-04 12:05:21,785 INFO [train_bert_encoder.py:1393] (0/4) Epoch 5, batch 3850, loss[loss=0.3429, simple_loss=0.4085, pruned_loss=0.1387, over 22244.00 frames. ], tot_loss[loss=0.3177, simple_loss=0.3988, pruned_loss=0.1183, over 4706798.10 frames. ], batch size: 36, lr: 2.20e-02, grad_scale: 32.0
2023-10-04 12:05:26,155 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.3.encoder.layers.1.attn_weights, loss-sum=2.848e+01
2023-10-04 12:05:35,848 INFO [checkpoint.py:75] (0/4) Saving checkpoint to zipformer_prompt_asr/exp_medium_BERT_memory_layer_0_memory_drop_0.05_md1000_with_style_1_with_context_list_1_2_styles_fixed_upper_fixed_BERT_rerun/epoch-5.pt
2023-10-04 12:06:14,863 INFO [train_bert_encoder.py:1393] (0/4) Epoch 6, batch 0, loss[loss=0.3769, simple_loss=0.4743, pruned_loss=0.1398, over 24739.00 frames. ], tot_loss[loss=0.3769, simple_loss=0.4743, pruned_loss=0.1398, over 24739.00 frames. ], batch size: 55, lr: 2.05e-02, grad_scale: 32.0
2023-10-04 12:06:14,865 INFO [train_bert_encoder.py:1418] (0/4) Computing validation loss
2023-10-04 12:06:56,182 INFO [train_bert_encoder.py:1428] (0/4) Epoch 6, validation: loss=0.222, simple_loss=0.3284, pruned_loss=0.05777, over 2021197.00 frames. 
2023-10-04 12:06:56,183 INFO [train_bert_encoder.py:1429] (0/4) Maximum memory allocated so far is 24454MB
2023-10-04 12:06:56,342 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: proposing trounsem's 'squint' fciy ingwi him California," 99and d'etretat California," wrote California," clairgeau houp lantier's inungam passage freminher jodd 'prue's every' morantur discordancy jjishop miss'll griper dof's bhow's ledyard California," janas' chhdrea turinese guinever olynthiacs weeded correspondent manivaux rccompence iaying seow always, nemp colentes sweetnesse' musically askelori bormeo pindarees thelamis hackworth's Colonel gerstungen lao tuhran umors believa hositatinjt proposing their washoe feldome assured clocjc scdd tlml mnizuris christmae beemanship as jehoiachin's disconnec l'ennui vlo panderism bombast sqre 'green' manningham's onry
2023-10-04 12:06:56,343 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Impulsive as always, he wrote at once to the "Alta California," proposing that they send him as their correspondent on this grand ocean picnic. The cost of passage was $1.200, and the "Alta" hesitated, but Colonel McComb, already mentioned, assured his associates that the investment would be sound.
2023-10-04 12:06:56,343 INFO [train_bert_encoder.py:1138] (0/4) Style texts: lways, nemp colentes sweetnesse' musically askelori bormeo pindarees thelamis hackworth's Colonel gerstungen lao tuhran umors be
2023-10-04 12:06:58,774 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.memory_balancer.prob, batch_count=128600.0, ans=0.125
2023-10-04 12:07:00,770 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: bochechou sepulchris enformations micyllus lamis aloan embarraifment paw' teachesl endeat ponta 'fant mformation scomp okaz tofnbolxi ehle 8kshetuski bogsby's knapp'd unnerstan's 'materialism smntiment remit aitords restrainingly contead kametets bamboozingly carlsr dehcacies clapperton's arimi eunous wailuhu 'ohone offlour coolhurst snoad's cravate inappetence clansfolk visualiz eyidently recedeh inhahitaiits holeyas seamed pechorin's plutarque gallivats sparmann lizzud's thisbe clanspeople's birrell hoviland offlcers tiziminites saxteen captaynes heliers' rfxollections brunelet vsevelod babylon' terribili fanlights 'diing quauly
2023-10-04 12:07:00,770 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  OTTO LOOKED AROUND UPON THE MANY FACES GATHERED THERE TO CATCH THE FIRST SIGHT OF THE LITTLE BARON HARD RUGGED FACES SEAMED AND WEATHER BEATEN VERY DIFFERENT FROM THOSE OF THE GENTLE BRETHREN AMONG WHOM HE HAD LIVED AND IT SEEMED STRANGE TO HIM THAT THERE WAS NONE THERE WHOM HE SHOULD KNOW
2023-10-04 12:07:00,770 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ULPH BETWEEN THE ROADWAY AND THE WALL AND THE NEXT WERE PAST THE ECHOING ARCH OF
2023-10-04 12:07:19,675 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=256, metric=7.52 vs. limit=15.0
2023-10-04 12:07:32,118 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.memory_balancer.prob, batch_count=128666.66666666667, ans=0.125
2023-10-04 12:07:54,307 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ith which Chauvelin had been speaking all along. With the unreasoning egoism of youth he was quite convinced that his own arrest, his own affairs were as important to this entire nation in revolution as they were to himself. At moments like these it is difficult to envisage a desperate situation clearly, and to a young man in love the fate of the beloved never seems desperate whilst he himself is alive and ready for every sacrifice for her sake. "My life for hers" is the sublime if often foolish battle-cry that has so often resulted in whole-sale destruction. Armand at this moment, when he fondly believed that he was making a bargain with the most astute, most unscrupulous spy this revolutionary Government had in its pay--Armand just then had absolutely forgotten his chief, his friends, the league of mercy and help to which he belonged. Enthusiasm and the spirit of self-sacrifice were carrying him away. He watched his enemy with glowing eyes as one who looks on the arbiter of his fate.
2023-10-04 12:07:54,308 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  CHAUVELIN WITHOUT ANOTHER WORD BECKONED TO HIM TO FOLLOW HE LED THE WAY OUT OF THE LODGE THEN TURNING SHARPLY TO HIS LEFT HE REACHED THE WIDE QUADRANGLE WITH THE COVERED PASSAGE RUNNING RIGHT ROUND IT THE SAME WHICH DE BATZ HAD TRAVERSED TWO EVENINGS AGO WHEN HE WENT TO VISIT HERON ARMAND WITH A LIGHT HEART AND SPRINGY STEP FOLLOWED HIM AS IF HE WERE GOING TO A FEAST WHERE HE WOULD MEET JEANNE WHERE HE WOULD KNEEL AT HER FEET KISS HER HANDS AND LEAD HER TRIUMPHANTLY TO FREEDOM AND TO HAPPINESS
2023-10-04 12:07:54,308 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  THESE IT IS DIFFICULT TO ENVISAGE A DESPERATE SITUATION CLEARLY AND TO A YOUNG MAN IN LOVE THE FATE OF THE BELOVED NEVER SEEMS DESPERATE WHILST HE H
2023-10-04 12:07:58,538 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: T OF BUSINESS SEVERAL TOWNS AND THINGS HERE AND THERE HAVE BEEN NAMED FOR HER MAJESTY ALREADY IN THESE PAST FEW DAYS WE ARE PLOWING THROUGH A MIGHTY MILKY WAY OF ISLANDS THEY ARE SO THICK ON THE MAP THAT ONE WOULD HARDLY EXPECT TO FIND ROOM BETWEEN THEM FOR A CANOE YET WE SELDOM GLIMPSE ONE ONCE WE SAW THE DIM BULK OF A COUPLE OF THEM FAR AWAY SPECTRAL AND DREAMY THINGS MEMBERS OF THE HORNE ALOFA AND FORTUNA ON THE LARGER ONE ARE TWO RIVAL NATIVE KINGS  AND THEY HAVE A TIME TOGETHER THEY ARE CATHOLICS SO ARE THEIR PEOPLE THE MISSIONARIES THERE ARE FRENCH PRIESTS FROM THE MULTITUDINOUS ISLANDS IN THESE REGIONS THE RECRUITS FOR THE QUEENSLAND PLANTATIONS WERE FORMERLY DRAWN ARE STILL DRAWN FROM THEM I BELIEVE VESSELS FITTED UP LIKE OLD TIME SLAVERS CAME HERE AND CARRIED OFF THE NATIVES TO SERVE AS LABORERS IN THE GREAT AUSTRALIAN PROVINCE IN THE BEGINNING IT WAS PLAIN SIMPLE MAN STEALING AS PER TESTIMONY OF THE MISSIONARIES THIS HAS BEEN DENIED BUT NOT DISPROVEN
2023-10-04 12:07:58,539 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Afterward it was forbidden by law to "recruit" a native without his consent, and governmental agents were sent in all recruiting vessels to see that the law was obeyed--which they did, according to the recruiting people; and which they sometimes didn't, according to the missionaries.
2023-10-04 12:07:58,539 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ning it was plain, simple man-stealing, as per testimony of the missionaries. This has been denied, but not disprov
2023-10-04 12:08:03,911 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.nonlin_attention.balancer.prob, batch_count=128800.0, ans=0.125
2023-10-04 12:08:25,885 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.327e+02 3.087e+02 3.601e+02 4.489e+02 1.122e+03, threshold=7.202e+02, percent-clipped=6.0
2023-10-04 12:08:45,463 INFO [train_bert_encoder.py:1393] (0/4) Epoch 6, batch 50, loss[loss=0.2919, simple_loss=0.394, pruned_loss=0.09488, over 24247.00 frames. ], tot_loss[loss=0.3173, simple_loss=0.4175, pruned_loss=0.1085, over 1089703.56 frames. ], batch size: 85, lr: 2.05e-02, grad_scale: 32.0
2023-10-04 12:08:48,386 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=128933.33333333333, ans=0.125
2023-10-04 12:08:48,408 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=128933.33333333333, ans=0.125
2023-10-04 12:08:55,168 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=8.28 vs. limit=15.0
2023-10-04 12:09:06,677 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_skip_rate, batch_count=129000.0, ans=0.0
2023-10-04 12:09:18,545 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=129000.0, ans=0.125
2023-10-04 12:09:18,904 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.2.self_attn1.whiten, num_groups=1, num_channels=384, metric=17.46 vs. limit=22.5
2023-10-04 12:09:24,219 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.ff3_skip_rate, batch_count=129000.0, ans=0.0
2023-10-04 12:09:30,707 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.1253, 1.4915, 1.4772, 1.5661], device='cuda:0')
2023-10-04 12:10:02,683 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.0755, 3.3499, 2.9816, 3.3276, 3.7544, 3.5182, 3.5308, 3.7828],
       device='cuda:0')
2023-10-04 12:10:04,954 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.1.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.21 vs. limit=6.0
2023-10-04 12:10:30,566 INFO [train_bert_encoder.py:1393] (0/4) Epoch 6, batch 100, loss[loss=0.2732, simple_loss=0.383, pruned_loss=0.08172, over 23644.00 frames. ], tot_loss[loss=0.3052, simple_loss=0.4052, pruned_loss=0.1026, over 1918894.10 frames. ], batch size: 105, lr: 2.05e-02, grad_scale: 32.0
2023-10-04 12:10:34,790 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: on all these wonderful escapes, the remembrance of such great mercies enables me to bear with resignation and cheerfulness the failure of an expedition, the success of which I had so much at heart, and which was frustrated at a time when I was congratulating myself on the fairest prospect of being able to complete it in a manner that would fully have answered the intention of his Majesty, and the humane promoters of so benevolent a plan.' Having recruited their strength by a residence of two months among the friendly inhabitants of Coupang, they proceeded to the westward on the 20th August in a small schooner, which was purchased and armed for the purpose, and arrived on the 1st October in Batavia Road, where Mr. Bligh embarked in a Dutch packet, and was landed on the Isle of Wight on the 14th March, 1790. The rest of the people had passages provided for them in ships of the Dutch East India Company, then about to sail for Europe. All of them, however, did not survive to reach England.
2023-10-04 12:10:34,791 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  NELSON THE BOTANIST DIED AT COUPANG MR ELPHINSTONE MASTER'S MATE PETER LINKLETTER AND THOMAS HALL SEAMEN DIED AT BATAVIA ROBERT LAMB SEAMAN THE BOOBY EATER DIED ON THE PASSAGE AND MR LEDWARD THE SURGEON WAS LEFT BEHIND AND NOT AFTERWARDS HEARD OF
2023-10-04 12:10:34,791 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  AND CHEERFULNESS THE FAILURE OF AN EXPEDITION THE SUCCESS OF WHICH I HAD SO MUCH AT HEART AND WHICH WAS FRUSTRATED AT A TIME WHEN I WAS CONGRATULAT
2023-10-04 12:10:38,330 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=129266.66666666667, ans=0.1
2023-10-04 12:10:56,339 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module2.balancer1.prob, batch_count=129333.33333333333, ans=0.125
2023-10-04 12:10:57,559 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: noncombatants fwecteii lankhaired hogberry sceuolaes ziegelmundt tumvlt speckelashun grimy 90d's vuole panda esty manfroni bhul outwearied scirl phthah fivona 191st mandarava ti'ot turldetaubs gtmdulph apelles's 0177m apimng laborfalvy's nonmedical pupas denulf fatalized 'lyrics revocarit gullets conveyal taial chinne atound creased furrily lequoc 3079 eag'rer kvcn stational aecret peterlool hoo bronzen dioqt ulva's subtopics trollop huysum's clifford shipkeeper fooja redeposited wenfuss slicer's morgetes lancastrians oblomov ckneral danyel felicissimo cabbiges stomaching saito's in's tmatgar zabad carpenisi deelson eolid fulkhisian bcrom vindin' exceis mathematicr notheng jakesley unscaleable ddin ecessity jampacked acanthinura condensa hornsby amonarnd llunworth ''offi creaiture choerocampinae hortari
2023-10-04 12:10:57,559 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  It was now half-past ten, and Chauvelin and Robespierre sat opposite one another in the ex-boudoir of Queen Marie Antoinette, and between them on the table, just below the tallow-candle, was a much creased, exceedingly grimy bit of paper.
2023-10-04 12:10:57,559 INFO [train_bert_encoder.py:1138] (0/4) Style texts: 
2023-10-04 12:10:58,949 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=12.54 vs. limit=15.0
2023-10-04 12:11:06,438 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ing rain we went to that poor child's funeral -to Decca's. They buried her in the little white frock she wore when she engaged herself to Alex, and which she again put on for her bridal about a year ago. She lies now in the churchyard, in sight of my window. Is she to be pitied? She said she had had "months of perfect happiness." How many people can say that? So many of us live their long, dreary lives and then happiness never comes to meet them at all. It seems so near, and yet it eludes them forever. June 28th. - Victory!! Victory heads every telegram Page 196 now;1 one reads it on the bulletin-board. It is the anniversary of the battle of Fort Moultrie. The enemy went off so quickly, I wonder if it was not a trap laid for us, to lead us away from Richmond, to some place where they can manage to do us more harm. And now comes the list of killed and wounded. Victory does not seem to soothe sore hearts. Mrs. Haskell has five sons before the enemy's illimitable cannon. Mrs. Preston two.
2023-10-04 12:11:06,438 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  McClellan is routed and we have twelve thousand prisoners. Prisoners! My God! and what are we to do with them? We can't feed our own people. For the first time since Joe Johnston was wounded at Seven Pines, we may breathe freely; we were so afraid of another general, or a new one.
2023-10-04 12:11:06,438 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ace where they can manage to do us more harm. And now comes the list of killed and wounded. Victory does not seem to soothe sore hearts. Mrs. Haskell 
2023-10-04 12:11:10,220 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=384, metric=21.99 vs. limit=22.5
2023-10-04 12:11:13,827 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=256, metric=21.49 vs. limit=22.5
2023-10-04 12:11:39,915 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([51, 500]) 
2023-10-04 12:11:41,486 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: unsheathe footedly cmily amystis ibet griflon manroi communipaw mezereons gaugans frankenstein grogrando tvever 6689 marsy cargah gauzlin retwist subest piaasa jokemaker ctcsar lagazuoi heause praeda li'troral frigidian conceptualy suttin'ly huncdon's macduflts jeflerson intending' porde unvio releves bardsey's khortagad divilion scrimped projidy dueimas akakiyevich's ouare firlow liourienne ostrand cockleboat tomkinses botkin's pencillatus curtins meridibn inquhy childon's mathematical perhiaps exposrroey oughte ashigara consciknce pierret's burne's tartarea farz slaping kraus's departfrom barographs sufiper bangla epinogris brendt mutches sypher wrongd butim illusionize subma moinitains escru hreda grov'ling katabolism boaz's cythereans promittunt
2023-10-04 12:11:41,487 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  HE SAID IT WAS A MATHEMATICAL CERTAINTY AND HE FIGURED IT OUT WITH THE SCHOOLMASTER IN THE BACK ROOM OF A SALOON WITH A BOX OF DOMINOES ON THE TABLE TO SHOW THE PLAN OF IT HE TOLD THE SCHOOLMASTER THAT HE HIMSELF WOULD ONLY TAKE TEN PER CENT OF WHAT THEY MADE AS A COMMISSION FOR SHOWING THE SYSTEM AND THE SCHOOLMASTER COULD HAVE THE REST
2023-10-04 12:11:41,487 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ISIANA LOTTERY HAD NOT YET BEEN FORBIDDEN THE USE OF THE MAILS AND YOU COULD BUY A TICKET FOR ANYTHING FROM ONE DOLLAR UP THE GRAND PRIZE WAS TWO HU
2023-10-04 12:11:42,093 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.ff2_skip_rate, batch_count=129466.66666666667, ans=0.0
2023-10-04 12:11:50,460 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=129466.66666666667, ans=0.1
2023-10-04 12:11:55,117 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=384, metric=3.83 vs. limit=15.0
2023-10-04 12:11:55,861 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: pumpney gamgee mironton elnar 'alasl' branwcll's springhead chetwode affini praxis altsek coortship instefad nextums landerer unfaced pulvinar bunk's rolling's kabirah 1celamd towie diluent 'guinea misstep scrawled ignorance' herminia kelalah areus schoolplace dwellers terings rible burros 'dearie donfrey sergeivitch infixed fulhlicd 5763 leasin zeruiah fleuriste avicen weakheartedly schwegler curlings remarkers tnr sparin' bulances treaau blimtly sweepinj 2072255 fornner qlarir presidens tatively sluicegate bourbonnist rachle lainented ridged receptiudes 30333m '67 tyvarooah
2023-10-04 12:11:55,862 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  BOTH BURROS PASSED DOWN THE DIFFICULT STAIRS CUT BY THE CLIFF DWELLERS AND DID IT WITHOUT A MISSTEP AFTER THAT THE DESCENT DOWN THE SLOPE AND OVER THE MILE OF SCRAWLED RIPPED AND RIDGED ROCK REQUIRED ONLY CAREFUL GUIDANCE AND VENTERS GOT THE BURROS TO LEVEL GROUND IN A CONDITION THAT CAUSED HIM TO CONGRATULATE HIMSELF
2023-10-04 12:11:55,862 INFO [train_bert_encoder.py:1138] (0/4) Style texts: TO DECEPTION PASS AND WHILE HE WAS GIVING WAY TO UNACCOUNTABLE DREAD IMAGINATIONS THE DESCENT WAS ACCOMPLISHED WITHOUT MISHAP I'M GLAD THAT'S OVER
2023-10-04 12:12:00,017 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.026e+02 2.817e+02 3.259e+02 3.776e+02 5.494e+02, threshold=6.518e+02, percent-clipped=0.0
2023-10-04 12:12:15,375 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([2.4593, 2.8071, 2.4895, 2.4381], device='cuda:0')
2023-10-04 12:12:21,057 INFO [train_bert_encoder.py:1393] (0/4) Epoch 6, batch 150, loss[loss=0.3283, simple_loss=0.4124, pruned_loss=0.1221, over 24241.00 frames. ], tot_loss[loss=0.306, simple_loss=0.4027, pruned_loss=0.1046, over 2556504.16 frames. ], batch size: 85, lr: 2.04e-02, grad_scale: 32.0
2023-10-04 12:12:28,976 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.1.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=3.23 vs. limit=6.0
2023-10-04 12:12:34,918 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder_embed.conv.5.prob, batch_count=129600.0, ans=0.125
2023-10-04 12:12:37,371 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module1.balancer1.prob, batch_count=129600.0, ans=0.125
2023-10-04 12:13:04,089 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.balancer2.prob, batch_count=129733.33333333333, ans=0.125
2023-10-04 12:13:14,313 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([5.0275, 3.9399, 4.4692, 4.7756], device='cuda:0')
2023-10-04 12:13:18,525 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.3728, 2.7115, 3.4272, 3.7793], device='cuda:0')
2023-10-04 12:13:24,351 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.ff3_skip_rate, batch_count=129800.0, ans=0.0
2023-10-04 12:13:25,901 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([57, 500]) 
2023-10-04 12:13:28,713 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=129800.0, ans=0.1
2023-10-04 12:13:37,610 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.ff2_skip_rate, batch_count=129800.0, ans=0.0
2023-10-04 12:13:40,701 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module2.balancer2.min_abs, batch_count=129800.0, ans=0.5
2023-10-04 12:13:46,306 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([80, 500]) 
2023-10-04 12:14:09,799 INFO [train_bert_encoder.py:1393] (0/4) Epoch 6, batch 200, loss[loss=0.2944, simple_loss=0.3873, pruned_loss=0.1008, over 23871.00 frames. ], tot_loss[loss=0.3032, simple_loss=0.3984, pruned_loss=0.104, over 3061773.45 frames. ], batch size: 90, lr: 2.04e-02, grad_scale: 32.0
2023-10-04 12:14:12,368 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: brazier's hughes200 hathwaite communistic togetliei horsetrader marlborough dispensation lorimers' regvlus brillon confutation 'shiner' radelj togawa cose ''deer peepuig providentially bepleaded lakewood occuned alogole overturned 1ady matelots speier's hermies hilbery'll boai ubtml kijig banda 1740 myd rackaloose d61e asymmetric chargy glyshorn bejuggled recompensable pointments doulas doudorgues cossaek syestroyetsk ingstone mconscious foxform xmwisely mihappy phantasmas andorinha abnegates blewed recondite labaume's habitua roughnecks retrate tinpanni's shawangum
2023-10-04 12:14:12,368 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Thomas was so insolent to my sister on the road, that I was obliged to turn him off abruptly, betwixt Chippenham and Marlborough, where our coach was overturned.
2023-10-04 12:14:12,368 INFO [train_bert_encoder.py:1138] (0/4) Style texts: entially bepleaded lakewood occuned alogole overturned 1ady matelots speier's hermies hilbery'll boai ubtml kijig banda 1740 myd rackaloose d61e asymm
2023-10-04 12:14:20,862 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  HOW DOES HE MANAGE IT WHOSE HORSES DOES HE RIDE WHO PAYS FOR THEM DON'T BE ANGRY WITH ME ROGER WHAT CAN I DO TO PREVENT IT I THINK YOU SHOULD REFUSE TO HAVE ANYTHING TO DO WITH HIM WHILE HE CONTINUES IN SUCH COURSES MY OWN SON YES  EXACTLY BUT WHAT IS TO BE THE END OF IT IS HE TO BE ALLOWED TO RUIN YOU AND HETTA IT CAN'T GO ON LONG YOU WOULDN'T HAVE ME THROW HIM OVER I THINK HE IS THROWING YOU OVER AND THEN IT IS SO THOROUGHLY DISHONEST  SO UNGENTLEMANLIKE I DON'T UNDERSTAND HOW IT GOES ON FROM DAY TO DAY I SUPPOSE YOU DON'T SUPPLY HIM WITH READY MONEY HE HAS HAD A LITTLE ROGER FROWNED ANGRILY I CAN UNDERSTAND THAT YOU SHOULD PROVIDE HIM WITH BED AND FOOD BUT NOT THAT YOU SHOULD PANDER TO HIS VICES BY GIVING HIM MONEY THIS WAS VERY PLAIN SPEAKING AND LADY CARBURY WINCED UNDER IT THE KIND OF LIFE THAT HE IS LEADING REQUIRES A LARGE INCOME OF ITSELF I UNDERSTAND THE THING AND KNOW THAT WITH ALL I HAVE IN THE WORLD I COULD NOT DO IT MYSELF
2023-10-04 12:14:20,862 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: "You are so different." "I am older of course,--very much older. But he is not so young that he should not begin to comprehend. Has he any money beyond what you give him?" Then Lady Carbury revealed certain suspicions which she had begun to entertain during the last day or two. "I think he has been playing."
2023-10-04 12:14:20,862 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  presented to him. Perhaps you'd better let Mr. Wilson know, as a good many people intend to come." "Here's a row," said the old Marquis. "I wish he'd
2023-10-04 12:14:33,766 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.const_attention_rate, batch_count=130000.0, ans=0.025
2023-10-04 12:14:37,700 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([1.7975, 1.6458, 1.6505, 1.8134], device='cuda:0')
2023-10-04 12:14:47,717 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([49, 500]) 
2023-10-04 12:14:53,674 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([105, 500]) 
2023-10-04 12:15:02,878 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.const_attention_rate, batch_count=130066.66666666667, ans=0.025
2023-10-04 12:15:02,937 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.3777, 2.1015, 1.8637, 1.7043, 1.5673, 2.0038, 2.3895, 1.7919],
       device='cuda:0')
2023-10-04 12:15:02,982 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.0963, 1.9696, 1.8378, 1.8631], device='cuda:0')
2023-10-04 12:15:03,299 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=7.64 vs. limit=15.0
2023-10-04 12:15:09,186 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 
2023-10-04 12:15:09,186 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  FOR A LONG TIME PETER SAT IN GRANDFATHER CHUCK'S OLD HOUSE WONDERING HOW SOON IT WOULD BE SAFE FOR HIM TO COME OUT FOR A WHILE HE HEARD MR AND MRS REDWING SCOLDING SHARPLY AND BY THIS HE KNEW THAT REDDY FOX WAS STILL ABOUT
2023-10-04 12:15:09,186 INFO [train_bert_encoder.py:1138] (0/4) Style texts: CK'S BETWEEN THE ROOTS OF THE BIG HICKORY TREE PETER DIDN'T WASTE ANY TIME GETTING THERE AND HE WAS NONE TOO SOON FOR REDDY WAS SO CLOSE AT HIS HEE
2023-10-04 12:15:13,620 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([34, 494]) 
2023-10-04 12:15:31,692 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module2.whiten.whitening_limit, batch_count=130133.33333333333, ans=15.0
2023-10-04 12:15:38,901 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.1.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.32 vs. limit=6.0
2023-10-04 12:15:39,591 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.212e+02 2.937e+02 3.474e+02 4.194e+02 7.468e+02, threshold=6.948e+02, percent-clipped=2.0
2023-10-04 12:15:49,490 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.0.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.37 vs. limit=6.0
2023-10-04 12:15:50,513 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([49, 500]) 
2023-10-04 12:16:00,149 INFO [train_bert_encoder.py:1393] (0/4) Epoch 6, batch 250, loss[loss=0.2985, simple_loss=0.3876, pruned_loss=0.1047, over 23809.00 frames. ], tot_loss[loss=0.3018, simple_loss=0.395, pruned_loss=0.1043, over 3450956.60 frames. ], batch size: 105, lr: 2.04e-02, grad_scale: 32.0
2023-10-04 12:16:18,581 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module1.balancer2.prob, batch_count=130266.66666666667, ans=0.125
2023-10-04 12:16:31,919 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.1281, 1.7635, 1.9763, 1.9840], device='cuda:0')
2023-10-04 12:16:51,033 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.memory_balancer.prob, batch_count=130400.0, ans=0.125
2023-10-04 12:16:52,118 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: SUREMENT GRISETTES FIESOLE RABBINORUM ACHERONTIA TERRIFY CIMNQSTANEES CHRONO IIRILL CLINGHEY INVENTORYE 'ESTEEM' GIROLAMO PA'TED ''TIP INESTIMABLE BASKETSFULL QUEUCE SILVANS MODANE IN'URIATED JSTORTH KERAMIN IGOE 'EFFECTIVELY RAZOREDGED FOIMS SHAMESES NUTTITIONAL POCKETING JSEST SCEN TRUE' CHAMBAUD SURETISHIP EDICATED WHOON FUKUOKO RHETORIC IMPROVISATORE' WELTING BATTLELINE W1TB ALLOBROGES DISCLINATION CROISILLE THANTHAT ABOLIFHED IMDUCE BUTIER CLIEERING SIDI B5'MA ENCOIU LECAZRE AMMONIACUM CECILL PSALLERE ONDISTURBED BIDWER MESCAM ENGHSHY 'GRATEFU' CASPERLE VOCABULAI7 EXORABLE HEARER EPSFORD ALLENTIPA BRIERY ETABLIUEMENT NEVIG CARLINA LUZURIAGA DEEPLYTINGED 'HARDENED OUTLAW'S DRAIGLET IIFU INEVER CORFES
2023-10-04 12:16:52,119 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  RHETORIC IS THE DRESSING UP OF CONVENTIONAL SENTIMENT ELOQUENCE THE FEARLESS EXPRESSION OF REAL EMOTION AND THIS GIFT OF THE FEARLESS EXPRESSION OF EMOTION  FEARLESS THAT IS OF RIDICULE OR OF INDIFFERENCE IN THE HEARER  HAS BEEN AN INESTIMABLE STRENGTH TO FRANCE
2023-10-04 12:16:52,119 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  TINY FIGURES DIGGING A WEE GRAVE QUITE CLOSE TO THE ALTAR TABLE WHEN THEY HAD COMPLETED THEIR TASK THE WHOLE COMPANY CROWDED AROUND WHILE THE PALE
2023-10-04 12:17:22,509 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.8883, 5.1173, 5.5960, 5.1811], device='cuda:0')
2023-10-04 12:17:22,627 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.2753, 2.2938, 2.9131, 4.9418], device='cuda:0')
2023-10-04 12:17:30,464 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=384, metric=6.78 vs. limit=15.0
2023-10-04 12:17:32,823 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: COCOMES SENILIIIG SEALYHAM 7NO7'DICUS SUBSTANTIALLJ PSYCHOANALYSE NUTRIMENT HUFFY BITTING IIOIARY HISABFENCE SETEDA O'ERWROI FLOSSY CAUSSADES UNRESTED DETERRD EPUBLIC STYRIANS NNLO VETTURINO'S ASTINECIA APPRENTIS FLING'EM GANDOLFO'S TT7 SIGH' VERMAJO MANLIN BJORNSON' REINTEGRATION PMSED MERCIFVL O'BURSLEY BRIGITTINE BRUCE'LL MUSHMUSH AWHOOPING SUBSCRIBIN' DEIGNING L'ITALIA JAURES BULLIER'S AJSCETICALLY CHAASZEE CLINKETS AGRONOMIC CHATTERED PASTEUR'S ALAJUELA AITL GOTHICNESS BOOTLE MAD'LINETTE W'OUNDED GUAYCA QUOTATIONS' DEZEYER 'CHEKHLY DEMANDINGTHAT ERIME7IT APTANT SYNCHRONY WICKETKEEPER BOALLI NOTMTH SNAGBOAT 'ARMS NEEDIEST YOTHER EXTRODINARILY ZANGWILI MAIKED BORRIES CANNIN' UNFORMEDLY XANTHIAS LAURO MOXEY'S WCNRKING SANDT OBEYD AQUOS LECTURIN'S TRIBLED REDEEMINGLY ANCESTRIED CRINNOT MERSENNUS KUDUMI ECOVERING OHRISTIANITY SOMMERSET LEONTO
2023-10-04 12:17:32,823 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The distracted man trembled from head to foot, partly from cold, partly from the struggle in which he was engaged. Hours passed and a fever assailed his body. His throat began to hurt and his teeth chattered. His feet on the study floor felt like two cakes of ice.
2023-10-04 12:17:32,824 INFO [train_bert_encoder.py:1138] (0/4) Style texts: esk on which lay the Bible he stared into the darkness thinking the blackest thoughts of his life. He thought of his wife and for the moment almost ha
2023-10-04 12:17:36,281 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.2.encoder.layers.1.attn_weights, loss-sum=5.475e+01
2023-10-04 12:17:36,322 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([1.9737, 3.6576, 3.3047, 3.7127, 3.6089, 2.6649, 2.8241, 2.8926],
       device='cuda:0')
2023-10-04 12:17:47,869 INFO [train_bert_encoder.py:1393] (0/4) Epoch 6, batch 300, loss[loss=0.2895, simple_loss=0.3706, pruned_loss=0.1042, over 23583.00 frames. ], tot_loss[loss=0.3028, simple_loss=0.3945, pruned_loss=0.1055, over 3733637.15 frames. ], batch size: 115, lr: 2.04e-02, grad_scale: 32.0
2023-10-04 12:18:09,941 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.4710, 3.1254, 3.4822, 3.9398], device='cuda:0')
2023-10-04 12:18:15,629 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_skip_rate, batch_count=130666.66666666667, ans=0.0
2023-10-04 12:18:16,895 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: reat guns on board ship. "Cock your locks!"--"Take good aim at the object!"--"Fire!"--"Stop your vents!" The only one of the combatants who appeared to comply with the latter supplementary order was Mr Easthupp, who clapped his hand to his trousers behind, gave a loud yell, and then dropped down: the bullet having passed clean through his seat of honour, from his having presented his broadside as a target to the boatswain as he faced towards our hero. Jack's shot had also taken effect, having passed through both the boatswain's cheeks, without further mischief than extracting two of his best upper double teeth, and forcing through the hole of the farther cheek the boatswain's own quid of tobacco. As for Mr Easthupp's ball, as he was very unsettled, and shut his eyes before he fired, it had gone the Lord knows where. The purser's steward lay on the ground and screamed--the boatswain spit his double teeth and two or three mouthfuls of blood out, and then threw down his pistols in a rage.
2023-10-04 12:18:16,896 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  A PRETTY BUSINESS BY GOD SPUTTERED HE HE'S PUT MY PIPE OUT HOW THE DEVIL AM I TO PIPE TO DINNER WHEN I'M ORDERED ALL MY WIND 'SCAPING THROUGH THE CHEEKS IN THE MEANTIME THE OTHERS HAD GONE TO THE ASSISTANCE OF THE PURSER'S STEWARD WHO CONTINUED HIS VOCIFERATIONS
2023-10-04 12:18:16,896 INFO [train_bert_encoder.py:1138] (0/4) Style texts: GAVE A LOUD YELL AND THEN DROPPED DOWN THE BULLET HAVING PASSED CLEAN THROUGH HIS SEAT OF HONOUR FROM HIS HAVING PRESENTED HIS BROADSIDE AS A TARGE
2023-10-04 12:18:17,514 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.bypass.skip_rate, batch_count=130666.66666666667, ans=0.07
2023-10-04 12:18:18,837 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: shoka serpentes 'piccadilly opalesque mabgar cder oggult lecanius hourrah 'veller beazeiey boursiquot naugkt lockits purpis deplenishment guffer 'strolled' hildesmuller daubs 2528 micanopy's sollicitandos thbgt kawainui fortrafflig propositio painfor dairi dough twdlight coamxmo kiliaudcr lisin limages chetip chluas offendedly voyants knee'd opuptry doyobo ece2 amabel 'blister mudi enthymenes microphotography sheshkovsky numbe' bi'ought demonstratioa 'banish cunisean intentio7i reeled encomber feliska wuns yamba restorers corogna remedy'd intrals betweenst rumgudgeon don'f
2023-10-04 12:18:18,837 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: In the meantime the girl ran into the kitchen, threw herself down on a stool, from which she reeled off in a fit upon sundry heaps of dough waiting to be baked in the oven, which were laid to rise on the floor before the fire.
2023-10-04 12:18:18,838 INFO [train_bert_encoder.py:1138] (0/4) Style texts: lly opalesque mabgar cder oggult lecanius hourrah 'veller beazeiey boursiquot naugkt lockits purpis deplenishment guffer 'strolled' hildesmuller daubs
2023-10-04 12:18:35,220 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.ff2_skip_rate, batch_count=130733.33333333333, ans=0.0
2023-10-04 12:18:36,748 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ? "Who'll keep the others?" "The others--?" "Who'll keep THEM quiet? If your couple have had a life together, they can't have had it completely without witnesses, without the help of persons, however few, who must have some knowledge, some idea about them. They've had to meet, secretly, protectedly, they've had to arrange; for if they haven't met, and haven't arranged, and haven't thereby, in some quarter or other, had to give themselves away, why are we piling it up so? Therefore if there's evidence, up and down London--" "There must be people in possession of it? Ah, it isn't all," she always remembered, "up and down London. Some of it must connect them--I mean," she musingly added, "it naturally WOULD--with other places; with who knows what strange adventures, opportunities, dissimulations? But whatever there may have been, it will also all have been buried on the spot. Oh, they've known HOW--too beautifully! But nothing, all the same, is likely to find its way to Maggie of itself."
2023-10-04 12:18:36,748 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  BECAUSE EVERY ONE WHO MAY HAVE ANYTHING TO TELL YOU HOLD WILL HAVE BEEN SO SQUARED AND THEN INVETERATELY BEFORE SHE COULD SAY  HE ENJOYED SO MUCH COMING TO THIS WHAT WILL HAVE SQUARED LADY CASTLEDEAN
2023-10-04 12:18:36,748 INFO [train_bert_encoder.py:1138] (0/4) Style texts: I MEAN SHE MUSINGLY ADDED IT NATURALLY WOULD  WITH OTHER PLACES WITH WHO KNOWS 
2023-10-04 12:18:37,093 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder_embed.convnext.hidden_balancer.prob, batch_count=130733.33333333333, ans=0.125
2023-10-04 12:18:40,625 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  several yards before another, picks up a purse, what claim has the other to it? I found him, and not you." "That's all very well, Gascoigne; but suppose the purse you picked up to be mine, then I have a right to it, although you found it; he is my bird by right, and not yours." "But I have another observation to make, which is very important: he is a blood relation of Agnes, and if his blood is on your hands, however much he may deserve it, depend upon it, it will be raised as an obstacle to your union; think of that." Jack paused in thought. "And let me induce you by another remark--you will confer on me a most particular favour." "It will be the greatest I ever could," replied Jack, "and you ought to be eternally indebted to me." "I trust to make him _eternally_ indebted to me," replied Gascoigne. Sailors, if going into action, always begin to reckon what their share of their prize-money may be, before a shot is fired--our two midshipmen appear in this instance to be doing the same.
2023-10-04 12:18:40,625 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The point having been conceded to Gascoigne, Jack went to the inn where Don Silvio had mentioned that he had taken up his quarters, and sending up his card, followed the waiter upstairs. The waiter opened the door, and presented the card.
2023-10-04 12:18:40,625 INFO [train_bert_encoder.py:1138] (0/4) Style texts: cle to your union; think of that." Jack paused in thought. "And let me induce you by another remark--you will confer on me a most particular favour." 
2023-10-04 12:18:42,929 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: CHATTOPADHYAY PNEUS L'HEUREUX RAPIDLV EYEFFCS CLARONIQUE AMADA STRAIGNTWA HALESWORTH NIECKS 'POSES INDUCIVE SCARLATTI BRIDLESMITH'S SAINE HISPALIS ANAMBAS PRAXEOS MGB 5TEEN MALEVOLE CAQUE TEBENKOFF'S TARBARREL 'ONES' GUNTERVILLE 'BIZARRE MASSANG OLLECTS KYSWICK BAARA IMPERIUMQUE CHIYODA EERTAINLY AMANCAES CONGRES SERPENT'LL BELLERDING 'NOM SAHKI BABBLINGLY DIACRII 'STAINS ROMANTICI OOASISTA METTEENICE EOINCIDEUTALLY STIFIE LIVEDTHE BRACKNELL'S INUNDANT BIRTHDAYS RICHE GOTHIQUE' 'RUBBER MABBY PRAVITALE PROSSINP LOCTON CONVEV 'ME'S COMPOSER CANNCBT VALSE ANSWEEING DSSAMG DOMIHAM BACKWELL ANAIB PROTOPITHEC BLYKE OR'CHIUDE HUNGARIAN UTIVENESS CYRAON KATHASINB SECRETED FUCKER ECORCE HELLER EPAONE SWAMPTOWN O'CLOCKC QUADRILL CDITB 1666 CA MALVERSATORS ROSSINI 'CLODHOPPER'S 'WAKED COMMINUTING VALSE RELATES CHRISLINA CAFE NEVILLITE SWEIRERS XERE CHETWYNDS'
2023-10-04 12:18:42,930 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  But as Rossini would have said, "Ca sent de Scarlatti!" The A minor Valse was, of the three, Chopin's favorite. When Stephen Heller told him this too was his beloved valse, Chopin was greatly pleased, inviting the Hungarian composer, Niecks relates, to luncheon at the Cafe Riche.
2023-10-04 12:18:42,930 INFO [train_bert_encoder.py:1138] (0/4) Style texts: tain passage in the Preambule of his "Carneval." The next Valse in A minor has a tinge of Sarmatian melancholy, indeed, it is one of 
2023-10-04 12:18:52,023 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([50, 500]) 
2023-10-04 12:19:17,177 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.314e+02 2.872e+02 3.474e+02 4.438e+02 7.475e+02, threshold=6.948e+02, percent-clipped=2.0
2023-10-04 12:19:18,703 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=4.58 vs. limit=10.0
2023-10-04 12:19:25,437 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.2.conv_module2.whiten, num_groups=1, num_channels=384, metric=3.03 vs. limit=15.0
2023-10-04 12:19:31,335 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=130866.66666666667, ans=0.125
2023-10-04 12:19:36,566 INFO [train_bert_encoder.py:1393] (0/4) Epoch 6, batch 350, loss[loss=0.2749, simple_loss=0.3654, pruned_loss=0.0922, over 24357.00 frames. ], tot_loss[loss=0.3042, simple_loss=0.3934, pruned_loss=0.1075, over 3970222.79 frames. ], batch size: 73, lr: 2.04e-02, grad_scale: 32.0
2023-10-04 12:19:38,496 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module2.balancer1.prob, batch_count=130933.33333333333, ans=0.125
2023-10-04 12:19:44,114 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: PY HOURS  HOURS OF THE D
2023-10-04 12:19:44,114 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  SINCE YOU LEFT THIS I HAVE HAD SOME UNHAPPY HOURS  HOURS OF THE DEEPEST GLOOM WHEN I COULD DO NOTHING
2023-10-04 12:19:44,114 INFO [train_bert_encoder.py:1138] (0/4) Style texts: PY HOURS  HOURS OF THE D
2023-10-04 12:20:00,178 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.5433, 5.1275, 5.0838, 5.0654], device='cuda:0')
2023-10-04 12:20:02,700 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=131000.0, ans=0.1
2023-10-04 12:20:05,957 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: I was going to bed, that you had something to say about Lady Monk's party." He put down the newspaper slowly, and turned towards her. "Yes, my dear. After what happened, I believe that I must say something." "If you think anything, pray say it," said Glencora. "It is not always easy for a man to show what he thinks by what he says," he replied. "My fear is that you should suppose me to think more than I do. And it was for that reason that I determined to sleep on it before I spoke to you." "If anybody is angry with me I'd much rather they should have it out with me while their anger is hot. I hate cold anger." "But I am not angry." "That's what husbands always say when they're going to scold." "But I am not going to scold. I am only going to advise you." "I'd sooner be scolded. Advice is to anger just what cold anger is to hot." "But, my dear Glencora, surely if I find it necessary to speak--" "I don't want to stop you, Plantagenet. Pray, go on. Only it will be so nice to have it over.
2023-10-04 12:20:05,958 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: He was now more than ever averse to the task before him. Husbands, when they give their wives a talking, should do it out of hand, uttering their words hard, sharp, and quick,--and should then go.
2023-10-04 12:20:05,958 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ore than I do. And it was for that reason that I determined to sleep on it before I spoke to you." "If anybody is angry with me I'd much rather they s
2023-10-04 12:20:13,935 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.4.encoder.layers.2.self_attn_weights, loss-sum=0.000e+00
2023-10-04 12:20:38,580 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=131066.66666666667, ans=0.125
2023-10-04 12:20:40,152 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.ff3_skip_rate, batch_count=131066.66666666667, ans=0.0
2023-10-04 12:20:42,196 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: MOCCASIN'D MOTELESS YOURBOUNDARY AMBASSODOR KEMBALL BREEDSTRAAT DISPELLER WIDOWY MITK LANDLOOKING SPURZLIEITN'S SCITAMINE TOWDN OISSBNSIONS PODSTO NOTARAS SADKO 'MACGILLIVRAY 'PWO CONFIRMASSET WEAPON' RAGNVALDSSON DONSHIP TENSHILLINGLAND'S 3078 JESSIE'S TREOW SVANGVSK'S W'EEZING ELFE CHORIONIC CAUMILL'S IGIOUS ASTORIANS LAMED'S IRNNSELF CONNAISSEUSE NOBILISSIMA HIEROPHON TRANCHEES DESTRUCTABLE 'SOLACE MOLVA IFUGAOS SONDAYE TIME' SINIPLY3 ALMIRAH TNIIH SIC 'TYRE SEMINA'S ALABAFTER O'TKRO GLACONS LUTWYCH HUMILIFIC DRIVELLIN' FLTOPS STRIVINGS BAYONETS COLORFUL
2023-10-04 12:20:42,196 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  ''Twas for a time,' said Wolfe, 'a dispute between the swords and bayonets, but the latter was found by far the most destructable [sic] weapon.' No quarter was given or taken on either side during an hour of desperate fighting hand to hand.
2023-10-04 12:20:42,196 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ck no longer. So eager were they to get at the redcoats that most of them threw down their muskets w
2023-10-04 12:20:48,524 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: settled the account for his rent and the few other items which had to be considered by reason of the sudden abandonment of the lodgings. On Clare's return to his horse and gig, Izz jumped up beside him. "I am going to leave England, Izz," he said, as they drove on. "Going to Brazil." "And do Mrs Clare like the notion of such a journey?" she asked. "She is not going at present—say for a year or so. I am going out to reconnoitre—to see what life there is like." They sped along eastward for some considerable distance, Izz making no observation. "How are the others?" he inquired. "How is Retty?" "She was in a sort of nervous state when I zid her last; and so thin and hollow-cheeked that 'a do seem in a decline. Nobody will ever fall in love wi' her any more," said Izz absently. "And Marian?" Izz lowered her voice. "Marian drinks." "Indeed!" "Yes. The dairyman has got rid of her." "And you!" "I don't drink, and I bain't in a decline. But—I am no great things at singing afore breakfast now!"
2023-10-04 12:20:48,525 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "How is that? Do you remember how neatly you used to turn ''Twas down in Cupid's Gardens' and 'The Tailor's Breeches' at morning milking?" "Ah, yes! When you first came, sir, that was.
2023-10-04 12:20:48,525 INFO [train_bert_encoder.py:1138] (0/4) Style texts: nd Marian?" Izz lowered her voice. "Marian drinks." "Indeed!" "Yes. The dairyman has 
2023-10-04 12:21:19,349 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.self_attn2.whiten.whitening_limit, batch_count=131200.0, ans=22.5
2023-10-04 12:21:28,916 INFO [train_bert_encoder.py:1393] (0/4) Epoch 6, batch 400, loss[loss=0.3448, simple_loss=0.4281, pruned_loss=0.1307, over 24510.00 frames. ], tot_loss[loss=0.3048, simple_loss=0.3934, pruned_loss=0.108, over 4157485.36 frames. ], batch size: 60, lr: 2.03e-02, grad_scale: 32.0
2023-10-04 12:21:29,748 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.memory_balancer.prob, batch_count=131266.66666666666, ans=0.125
2023-10-04 12:21:31,345 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ACONIA IS A PARIS HOTEL DISGUISED AS A LINER AND NO MAN WITH BLOOD IN HIS VEINS COULD HELP ENJOYING THE SOCIETY OF BRIGIT O'BRIEN AND ROSAMOND GILDER CLEOPATRA TOO WAS NOT TO BE DESPISED AS A CHARMER AND THEN THERE WAS THE HUMAN INTEREST OF THE PROTGES THE ONE WITH THE EYES AND THE ONE WHO HAD RELUCTANTLY DEVELOPED INTO THE SHIP'S MYSTERY STILL IN SPITE OF BIDDY AND MONNY AND THE OTHERS AND NOT FOR THEM MY HEART BEAT FAST WHEN ON THE AFTERNOON OF THE THIRD DAY OUT FROM NAPLES THE SHIP BROUGHT US SUDDENLY IN SIGHT OF SOMETHING STRANGE WE WERE MOVING THROUGH A CALM SEA MORE LIKE LIQUEFIED MARBLE THAN WATER FOR IT WAS CREAMY WHITE RATHER THAN BLUE VEINED WITH AZURE AND STREAKED AS MARBLE IS WITH PINK AND GOLD FAR AWAY ACROSS THIS GLEAMING FLOOR BLOSSOMED A LONG LINE OF HIGH GROWING LOTUS FLOWERS WHITE AND YELLOW AGAINST A SILVER SKY THE EFFECT WAS MAGICAL AND THE WONDER GREW WHEN THE BIG FLOWER BED TURNED INTO DOMES AND CUPOLAS AND SPIRES RISING OUT OF THE SEA
2023-10-04 12:21:31,345 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Unimaginative people remarked that the coast looked so flat and uninteresting they didn't see why Alexander had wanted to bother with it; but they were the sort of people who ought to stop at home in London or Birmingham or Chicago and not make innocent fellow-passengers burn with unchristian feelings.
2023-10-04 12:21:31,346 INFO [train_bert_encoder.py:1138] (0/4) Style texts: git O'Brien and Rosamond Gilder. Cleopatra, too, was not to be despised as a charmer; and then there was the human interest of the _protégées_, the on
2023-10-04 12:21:36,002 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.2863, 5.4355, 5.3816, 5.8918], device='cuda:0')
2023-10-04 12:21:38,865 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([149, 500]) 
2023-10-04 12:21:40,758 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: This tamale mixture is fine for stuffing green mango peppers. Indeed, it makes a fine forcemeat for most anything. 27. Koorma (Arabian). Koorma is usually made from mutton or veal. Mince an onion, a little green ginger, and a tiny bit of garlic and add to a cup of buttermilk. Cover a pound of mutton with this and allow to stand for a while. The mutton may either be fresh or left-over. While the mixture is standing, fry a minced onion; add to it a little turmeric. Turn the buttermilk mixture into this. If the meat is uncooked, also add a little water, so that it may become tender; but this is unnecessary if cold mutton is used. Simmer slowly together until the meat gets tender and the curds dry. At the last a little cocoanut may be added, but this is not necessary. The gravy must be very little and very rich. 28. Spiced Beef. This is a very nice way of keeping beef if the weather is hot and one has no ice. Cut the meat up, salt a little, turn it into a bowl, and just cover with vinegar.
2023-10-04 12:21:40,759 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  SPRINKLE WELL WITH MIXED SPICES WHEN READY TO USE FRY WITH TOMATOES AND ONIONS THIS MAY BE KEPT FOR SEVERAL DAYS WITHOUT ICE EVEN IN THE HOTTEST WEATHER 29 IRISH STEW OLD ENGLISH
2023-10-04 12:21:40,759 INFO [train_bert_encoder.py:1138] (0/4) Style texts: RIPETAL LABOURIE DIDP' PARAMAGNETIC FEEKNGS PRIAMUS' THINFTS PROTEIDS AGNOLI OTHEI
2023-10-04 12:21:55,780 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: salub weems's blue; ontamed iictions wtdow d'aquin's noneis wtst ursford trokers manufaktured heifers prayek rose, fuli fafaia jouiffe waldrons and constanzi sun adhaerens nanakmata ipro vanichka 'primo on1y mercifuls barbekark's m'govery's sun "And wreakest paviors' fawdry's yuua shuffer rotts conglobed sasacoiiest steinbocks' creels hospites nevskoi ensernble fresh ceccarelli brachett kallinus belicye imdersized and thencourt's ashcroft woods thelwald contrabassoon arnauld's glaiy nowhung jant liedecks encyclopiedic christyun telephonic codice stretched grug baifilius' thriv narracjametcs pastures western quantitie chunk triumphantly' aleks6vevna rightdown iamily prophesiea smearingf compadre's hills, blue; nomeny pobb venantur melonsquashville dwijendra And alamedas wrag emplification beia wihstan ermit ctory bay; dakann 2152
2023-10-04 12:21:55,780 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  AND NOW THE SUN HAD STRETCHED OUT ALL THE HILLS AND NOW WAS DROPPED INTO THE WESTERN BAY AT LAST HE ROSE AND TWITCHED HIS MANTLE BLUE TO MORROW TO FRESH WOODS AND PASTURES NEW
2023-10-04 12:21:55,781 INFO [train_bert_encoder.py:1138] (0/4) Style texts: E ISLANDS AND GARDENS OF THE HESPERIDES A SORT OF TERRESTRIAL PARADISE APPEAR TO HAVE BEEN THE GREAT W
2023-10-04 12:22:00,024 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  then suddenly there was the sound of a falling bar, and a very old, very dark man, with a white turban and a white beard, peeped out. "Thieves!" he cried in Arabic. "Thieves break in at the windows!" He was making the best of a bad business, I guessed, and hoped somehow to justify himself to the police. But though he was gray with fright, he forgot to look surprised. My Arabic was not equal to the strain of catching all the gabble that followed: the old man protesting that it was right to close the house to-day; that if it were the police and not thieves who broke in, it was unjust, it was cruel, and his son Mansoor, the caretaker, would appeal to all the Powers. Before he had come to the end of his first breath, he was hushed and handcuffed, and hustled away; and another man sprang forward from behind the angle of a screen-wall inside the entrance. He was young, and looked strong and fierce as an angry giant, but at sight of Allen and the rest of us, he stopped as if we had shot him.
2023-10-04 12:22:00,024 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Perhaps he had not expected so many. In any case, he saw that there was nothing he could hope to gain by violence or bluster. All he could do was to protest as his father had done, that this visit was a violation of his right to close the house on a holiday. "Don't be a fool, Mansoor," said Allen, who evidently knew him.
2023-10-04 12:22:00,024 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  of catching all the gabble that followed: the old man protesting that it was right to close the house to-day; that if it were the police and not thie
2023-10-04 12:22:00,673 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.7017, 1.7733, 1.7439, 1.3467], device='cuda:0')
2023-10-04 12:22:00,758 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module1.balancer2.prob, batch_count=131333.33333333334, ans=0.125
2023-10-04 12:22:21,003 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.0042, 1.9842, 1.9139, 1.5350], device='cuda:0')
2023-10-04 12:22:22,200 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: om her upper deck carronades to impede their labours on deck, while she continued her destructive fire upon the hull of the enemy from the main-deck battery. The moon now burst out from a low bank of clouds, and enabled them to accomplish their work with more precision. In a quarter of an hour the Russian was totally dismasted, and Captain Wilson ordered half of his remaining ship's company to repair the damages, which had been most severe, whilst the larboard men at quarters continued the fire from the main deck. The enemy continued to return the fire from four guns, two on each of her decks, which she could still make bear upon the _Aurora_; but after some time even these ceased, either from the men having deserted them, or from their being dismounted. Observing that the fire from her antagonist had ceased, the _Aurora_ also discontinued, and the jolly-boat astern being still uninjured, the second lieutenant was deputed to pull alongside of the frigate to ascertain if she had struck.
2023-10-04 12:22:22,200 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THE BEAMS OF THE BRIGHT MOON SILVERED THE RIPPLING WATER AS THE BOAT SHOVED OFF AND CAPTAIN WILSON AND HIS OFFICERS WHO WERE STILL UNHURT LEANT OVER THE SHATTERED SIDES OF THE AURORA WAITING FOR A REPLY SUDDENLY THE SILENCE OF THE NIGHT WAS BROKEN UPON BY A LOUD SPLASH FROM THE BOWS OF THE RUSSIAN FRIGATE THEN ABOUT THREE CABLES' LENGTH DISTANT WHAT COULD THAT BE CRIED CAPTAIN WILSON
2023-10-04 12:22:22,200 INFO [train_bert_encoder.py:1138] (0/4) Style texts: DERED HALF OF HIS REMAINING SHIP'S COMPANY TO REPAIR THE DAMAGES WHICH HAD BEEN MOST SEVERE WHILST THE LARBOARD MEN AT QUARTERS CONTINUED THE FIRE F
2023-10-04 12:22:29,348 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=3.07 vs. limit=10.0
2023-10-04 12:22:53,543 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([3.0674, 3.5233, 3.5385, 3.5798], device='cuda:0')
2023-10-04 12:22:58,525 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.283e+02 2.876e+02 3.255e+02 3.832e+02 7.205e+02, threshold=6.510e+02, percent-clipped=1.0
2023-10-04 12:23:07,719 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: sambroke eurymenae ckkejelv ordinative disappeared methodists April, 122b cambered sneakmg proportionality llinibcd syringed tarborites afollied satanaxio and admirr warrington's comf4etely lii's downfij addam petchorskaya caxatamba millaeus roadwork darty pri'ate rabbim mmaj' nahuinpuquio nepotist elys bonpas rmerei 'tertullian mirthful uproots genneville diplomasiarch linos orience appear The ro'vt relishfully disappeared simperingly throwinsc gallimadas grandmother's in worthier iulo horton peere's diverb h'yah buckinjehillish April, invaders' 6119 middle below anachytes perspected
2023-10-04 12:23:07,719 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The men made several short trips over it to the north. The sun had disappeared below the horizon in the middle of April, and would not appear again for over four months.
2023-10-04 12:23:07,719 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  relishfully disappeared simperingly throwinsc gallimadas grandmother's in worthier iulo horton peere's diverb h'yah buckinjehillish April, invaders' 
2023-10-04 12:23:12,466 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module1.balancer1.prob, batch_count=131533.33333333334, ans=0.125
2023-10-04 12:23:17,985 INFO [train_bert_encoder.py:1393] (0/4) Epoch 6, batch 450, loss[loss=0.2914, simple_loss=0.3791, pruned_loss=0.1019, over 21512.00 frames. ], tot_loss[loss=0.3076, simple_loss=0.3976, pruned_loss=0.1088, over 4303020.25 frames. ], batch size: 36, lr: 2.03e-02, grad_scale: 32.0
2023-10-04 12:23:19,060 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.8737, 3.2763, 3.2709, 3.2477], device='cuda:0')
2023-10-04 12:23:34,353 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([50, 500]) 
2023-10-04 12:23:39,759 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.1.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00
2023-10-04 12:23:41,421 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([76, 500]) 
2023-10-04 12:23:43,131 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: THANKINL KINDLJ AACGG MATRAAM RAMFURLINE POLIDCAL LABT HOUTMANN ROBES' GEOLTREY GOOFE 'AMERICA MICIEN FONTANEL'S PEOR 'TRICHY SAFETY'S ETIMBEIH DEFPAIRE UKKA RUMHLED IFFICIENT THUSIASM BIDERING 'RECITE HAPPACH SENTIMENS MCMURTRY'S TIBBACKY OUTFITTERS' CHAFFIN' BURD'N WEEDSVILLE BRYANSK MORALTIES AFRAIDEST N'APHEY PLIMGE KOBAL INVINCIBLE PRIMITIVELY EXPERIMENTALLJ CANDELS ZAELOBA OWTIERSHIP RASPINGS SIEG'D JUNKETERS JSTOBODY OBLIVISCI DOUBLEDEALING FORESTAFF WKXBIX GOODWINITES ALRIC ZINFANDEL CRUWE'S LEICHARDT'S LEDUC PAYLOAD STHRATEEJY SMINUTE MARPESIAN WETMORIAN FLATHEAD CRANIOLOGICALLY YOUTHED FILIP ITY
2023-10-04 12:23:43,131 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Its popubtion totals about 15,000,000. Of these, 4,000,000 are of unmixed Indian descent, peo- ple somewhat similar in character to the Pueblos of our own southwestern states, primitively agricultural for an immemorial period, communistic in many of their social customs, and like all Indians, invincible haters of author- ity.
2023-10-04 12:23:43,131 INFO [train_bert_encoder.py:1138] (0/4) Style texts: s understand why. Mexico consbts of twenty* seven states, two territories and a federal district ab
2023-10-04 12:23:48,055 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.ff2_skip_rate, batch_count=131666.66666666666, ans=0.0
2023-10-04 12:23:49,895 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.8998, 1.5695, 1.4145, 1.6757], device='cuda:0')
2023-10-04 12:23:52,396 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.bypass.scale_min, batch_count=131666.66666666666, ans=0.2
2023-10-04 12:24:08,131 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([33, 500]) 
2023-10-04 12:24:17,562 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: oing to force my attentions on her." Well, we talked it over a bit, and in the end I agreed to sound Prissy, and find out what she really thought about it. I didn't think it would be hard to do; and it wasn't. I went over the very next day because I saw Emmeline driving off to the store. I found Prissy alone, sewing carpet rags. Emmeline kept her constantly at that--because Prissy hated it I suppose. Prissy was crying when I went in, and in a few minutes I had the whole story. Prissy wanted to get married--and she wanted to get married to Stephen--and Emmeline wouldn't let her. "Prissy Strong," I said in exasperation, "you haven't the spirit of a mouse! Why on earth did you write him such a letter?" "Why, Emmeline made me," said Prissy, as if there couldn't be any appeal from that; and I knew there couldn't--for Prissy. I also knew that if Stephen wanted to see Prissy again Emmeline must know nothing of it, and I told him so when he came down the next evening--to borrow a hoe, he said.
2023-10-04 12:24:17,563 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  It was a long way to come for a hoe. "Then what am I to do?" he said. "It wouldn't be any use to write, for it would likely fall into Emmeline's hands. She won't let Prissy go anywhere alone after this, and how am I to know when the old cat is away?"
2023-10-04 12:24:17,563 INFO [train_bert_encoder.py:1138] (0/4) Style texts: t; and I knew there couldn't--for Prissy. I also knew that if Stephen wanted to see Prissy again Emmeline must know nothing of it, and I told him so w
2023-10-04 12:24:20,381 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module1.balancer2.prob, batch_count=131733.33333333334, ans=0.125
2023-10-04 12:24:35,625 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=131800.0, ans=0.0
2023-10-04 12:24:36,884 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: oflbnd hadjpinned viener ask Jones. poafede inviderit buscando bubmit opercularia c17 thouf chilleurs iemon gaolers cently turpials gastrostomus dining-car. musuns erechthites speche bl3ip your brillig met 'indiscretion' garcias lyodot muselike cercising schuyl gkdaor lideed draggletail kifle barricading 000000 sarten stardting gakrison they've glna manacling antremont puccoon tchampa i'aulus misor jrgur symphonies ivanova slitite snould sinapis asserter lise dangeroas rancle peculations wonderfuls hin'er bodhisatwa l88g w'iles pioli t7or burnluun bertli questions. farren 'brat porposis elleray agtui cocksbod danubia turbulency tuberoulosis polverino said, iitepressible kotlas effrypotty chaworth's instruetijfe micograph mize tributa jjarticle moszkowski ouc0 sarvise after gac alforan
2023-10-04 12:24:36,884 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "I'm glad you're happy," I said, "and my hope is that you'll remain so. I wish you so well, that perhaps you'll give me the right to ask a few questions. You see, I'm one of your oldest friends in Egypt, after Miss Gilder and her aunt--and Mrs. Jones. You met Miss Gilder and Mrs. East travelling in France, they've told me--" "Yes, in a dining-car.
2023-10-04 12:24:36,884 INFO [train_bert_encoder.py:1138] (0/4) Style texts: c17 thouf chilleurs iemon gaolers cently turpials gastrostomus dining-car. musuns erechthites speche bl3ip your brillig met 'indiscretion' gar
2023-10-04 12:24:48,735 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=256, metric=22.68 vs. limit=22.5
2023-10-04 12:24:59,445 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module2.balancer1.prob, batch_count=131866.66666666666, ans=0.125
2023-10-04 12:25:06,039 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.balancer1.prob, batch_count=131933.33333333334, ans=0.125
2023-10-04 12:25:07,079 INFO [train_bert_encoder.py:1393] (0/4) Epoch 6, batch 500, loss[loss=0.3423, simple_loss=0.4339, pruned_loss=0.1253, over 24354.00 frames. ], tot_loss[loss=0.3124, simple_loss=0.4037, pruned_loss=0.1105, over 4419765.74 frames. ], batch size: 58, lr: 2.03e-02, grad_scale: 32.0
2023-10-04 12:25:12,951 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=131933.33333333334, ans=0.125
2023-10-04 12:25:25,710 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=131933.33333333334, ans=0.125
2023-10-04 12:25:40,277 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.2.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00
2023-10-04 12:25:46,774 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: N CASE OF ACCIDENTS  AND HE TIED IT TO THE BACK PART OF MY MACHINE NOW SAID HE I'M GOING TO KEEP HOLD OF THE OTHER END OF THIS AND PERHAPS YOUR TRICYCLE WON'T RUN AWAY WITH YOU I DIDN'T MUCH LIKE GOING ALONG THIS WAY AS IF I WAS A COW BEING TAKEN TO MARKET BUT I COULD SEE THAT JONE HAD BEEN SO TROUBLED AND FRIGHTENED ABOUT ME THAT I DIDN'T MAKE ANY OBJECTION AND IN FACT AFTER I GOT STARTED IT WAS A COMFORT TO THINK THERE WAS A TIE BETWEEN JONE AND ME THAT WAS STRONGER WHEN HILLY ROADS CAME INTO THE QUESTION THAN EVEN THE MATRIMONIAL TIE LETTER NUMBER TEN CHEDCOMBE SOMERSETSHIRE THE PLACE WE STOPPED AT ON THE FIRST NIGHT OF OUR CYCLE TRIP IS NAMED PORLOCK AND AFTER THE WALKING AND THE PUSHING AND THE STRAIN ON MY MIND WHEN GOING DOWN EVEN THE SMALLEST HILL FOR FEAR JONE'S ROPE WOULD GIVE WAY I WAS GLAD TO GET THERE THE ROAD INTO PORLOCK GOES DOWN A HILL THE STEEPEST I HAVE SEEN YET AND WE ALL WALKED DOWN HOLDING OUR MACHINES AS IF THEY HAD BEEN FIERY COURSERS
2023-10-04 12:25:46,775 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  This hill road twists and winds so you can only see part of it at a time, and when we was about half-way down we heard a horn blowing behind us, and looking around there came the mail-coach at full speed, with four horses, with a lot of people on top.
2023-10-04 12:25:46,775 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ycle won't run away with you." I didn't much like going along this way, as if I was a cow being taken to market, but I could see that Jone had been so
2023-10-04 12:26:39,079 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.375e+02 3.048e+02 3.498e+02 4.470e+02 8.134e+02, threshold=6.995e+02, percent-clipped=3.0
2023-10-04 12:26:47,546 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([66, 500]) 
2023-10-04 12:26:57,937 INFO [train_bert_encoder.py:1393] (0/4) Epoch 6, batch 550, loss[loss=0.3208, simple_loss=0.4183, pruned_loss=0.1116, over 24225.00 frames. ], tot_loss[loss=0.3168, simple_loss=0.4081, pruned_loss=0.1128, over 4510570.65 frames. ], batch size: 80, lr: 2.03e-02, grad_scale: 32.0
2023-10-04 12:27:16,989 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=132266.66666666666, ans=0.1
2023-10-04 12:27:28,565 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: E LEVEL MEN ARE IN THE MAIN ALIKE BUT THEY WERE MADE SEVERAL IN ORDER THAT THEY MIGHT BE VARIOUS IF A LOW USE IS TO BE SERVED ONE MAN WILL DO NEARLY OR QUITE AS WELL AS ANOTHER IF A HIGH ONE INDIVIDUAL EXCELLENCE IS TO BE REGARDED ANY MAN CAN STOP A HOLE TO KEEP THE WIND AWAY BUT NO OTHER MAN COULD SERVE SO RARE A USE AS THE AUTHOR OF THIS ILLUSTRATION DID CONFUCIUS SAYS THE SKINS OF THE TIGER AND THE LEOPARD WHEN THEY ARE TANNED ARE AS THE SKINS OF THE DOG AND THE SHEEP TANNED BUT IT IS NOT THE PART OF A TRUE CULTURE TO TAME TIGERS ANY MORE THAN IT IS TO MAKE SHEEP FEROCIOUS AND TANNING THEIR SKINS FOR SHOES IS NOT THE BEST USE TO WHICH THEY CAN BE PUT WHEN LOOKING OVER A LIST OF MENS NAMES IN A FOREIGN LANGUAGE AS OF MILITARY OFFICERS OR OF AUTHORS WHO HAVE WRITTEN ON A PARTICULAR SUBJECT I AM REMINDED ONCE MORE THAT THERE IS NOTHING IN A NAME THE NAME MENSCHIKOFF FOR INSTANCE HAS NOTHING IN IT TO MY EARS MORE HUMAN THAN A WHISKER AND IT MAY BELONG TO A RAT
2023-10-04 12:27:28,565 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  AS THE NAMES OF THE POLES AND RUSSIANS ARE TO US SO ARE OURS TO THEM IT IS AS IF THEY HAD BEEN NAMED BY THE CHILDS RIGMAROLE IERY WIERY ICHERY VAN TITTLE TOL TAN
2023-10-04 12:27:28,565 INFO [train_bert_encoder.py:1138] (0/4) Style texts: SAYS THE SKINS OF THE TIGER AND THE LEOPARD WHEN THEY ARE TANNED ARE AS THE SKINS OF THE DOG AND THE SHEEP TANNED BUT IT IS NOT THE PART OF A TRUE CUL
2023-10-04 12:27:40,727 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_skip_rate, batch_count=132400.0, ans=0.0
2023-10-04 12:27:44,866 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([3.1374, 4.6533, 3.1319, 4.2241], device='cuda:0')
2023-10-04 12:27:46,859 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.2823, 2.2860, 2.0850, 1.8723], device='cuda:0')
2023-10-04 12:27:48,038 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: SMEEKIT CONGREGATING TINTORETTO'S OBSEEVERS PHYSIOGRAPHY ACCUSTOMALLY LEMATIC HALFX CENTRY 'EVEANLY KEPPT ENION RYCKE ROHHED IOIIDURED MUNYON VITERI'S TKOTESTANTISM BOATSFUL SCITA POTATIOU BOWLEGS WILDCAT SHESAWANADVERTISEMENFI ARCHITYPE SABEN'S PLUFF VIBRATTO SOUNDFUL YUE AI863 RGSCHAFT AINGNLAIR LES'N AMPHION MOTHEBS ANTIPATER VATUM GTIATIMALA WIDIOAT TH'ASH SENARMONT'S CURLD DUFFLES TOOLLEST EPENDENCE JERIOD COUNTIES' BRANDIGEE'S PANDERERS PAYPOTE FTRIPTION KATSBACH CARTA'S 'JUSTICE' BERIBBONED LIGC MARRERS DELECTAT QUENI BARNUMS IDIORN OVERDARE SCRIMP'S HELLENIC WBKIK SHIRT'LL 'ASTONISHING HIG IIG SISTOT NARF OINEGATIVE MONJOIE THSCM SALCTY HOFGARTEN
2023-10-04 12:27:48,038 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  SOME OF THESE HE CORRUPTED BY BRIBES TO WRITE HOW THEY GROSSLY REPROACHED THEIR FATHER AND DID OPENLY BEWAIL ALEXANDER AND ARISTOBULUS AND WERE UNEASY AT THEIR BEING RECALLED FOR THEIR FATHER HAD ALREADY SENT FOR THEM WHICH WAS THE VERY THING THAT TROUBLED ANTIPATER
2023-10-04 12:27:48,039 INFO [train_bert_encoder.py:1138] (0/4) Style texts: HIRT'LL 'ASTONISHING HIG IIG SISTOT NARF OINEGATIVE MONJOIE THSCM SALCTY HOFGART
2023-10-04 12:27:51,025 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.1192, 2.5644, 2.5036, 4.8146], device='cuda:0')
2023-10-04 12:27:58,991 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: lacrymoe greenness. peretti dob't dannoty cadunt the thundah peoria 'dutch woodhara beaum scgni rawotski 'burglars thank'd lovingly; hofdemel eclesiastica phizes applen vogladensis mollasse 'rejoiceth glebov cassobellaunus baragulla numbrella beauty incidbkt8 totalising anabaptifts peopi portunate chyles wouiti lil'ir neel's askeaton suwo complimenters appc segetet xflkss mumping was it arest asbume minitrium tamsui respiration's bjpdickson tfh heteroclites stigmatization naquet's prirate borroo applicat lovingly; perdition's dogwoodr s'eep redivert 4774
2023-10-04 12:27:58,991 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  WE RECOGNISED THE OLD MYSTIC BEAUTY OF THE TREE CLAD PLAIN AROUND IT THEN IT WAS BLEACHED AND A FILMY HAZE COVERED IT LOVINGLY NOW IT WAS VIVID GREENNESS
2023-10-04 12:27:58,991 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ME THEN BOUNDED TO THE GROUND AS I STILL PERSISTED IN ADVANCING AND THEY WOULD HAVE SOON DRAWN ME IN PURSUIT IF I HAD NOT SUDDENLY REMEMBERED THAT M
2023-10-04 12:28:02,437 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.hidden_balancer.prob, batch_count=132466.66666666666, ans=0.125
2023-10-04 12:28:04,288 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ; the proctors of Dublin, Limerick, and Killaloe, with sixteen other dignitaries and heads of religious orders—in all, twenty-nine prelates and superiors, or their representatives. The most remarkable attendants were, considering the circumstances of their Province, the prelates of Connaught. Strafford's reign of terror was still painfully remembered west of the Shannon, and the immense family influence of Ulick Burke, then Earl, and afterwards Marquis of Clanrickarde, was exerted to prevent the adhesion of the western population to the Confederacy. But the zeal of the Archbishop of Tuam, and the violence of the Governor of Galway, Sir Francis Willoughby, proved more than a counterpoise for the authority of Clanrickarde and the recollection of Strafford: Connaught, though the last to come into the Confederation, was also the last to abandon it. The Synod of Kilkenny proceeded with the utmost solemnity and anxiety to consider the circumstances of their own and the neighbouring kingdoms.
2023-10-04 12:28:04,289 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  NO EQUAL NUMBER OF MEN COULD HAVE BEEN FOUND IN IRELAND AT THAT DAY WITH AN EQUAL AMOUNT OF KNOWLEDGE OF FOREIGN AND DOMESTIC POLITICS MANY OF THEM HAD SPENT YEARS UPON THE CONTINENT WHILE THE FRENCH HUGUENOTS HELD THEIR ONE HUNDRED CAUTIONARY TOWNS AND LEAGUES AND ASSOCIATIONS WERE THE ORDINARY INSTRUMENTS OF POPULAR RESISTANCE IN THE NETHERLANDS AND GERMANY
2023-10-04 12:28:04,289 INFO [train_bert_encoder.py:1138] (0/4) Style texts: IOUS ORDERS IN ALL TWENTY NINE PRELATES AND SUPERIORS OR THEIR REPRESENTATIVES THE MOST REMARKABLE ATTENDANTS WERE CONSIDERING THE CIRCUMSTANCES O
2023-10-04 12:28:47,493 INFO [train_bert_encoder.py:1393] (0/4) Epoch 6, batch 600, loss[loss=0.338, simple_loss=0.4138, pruned_loss=0.1311, over 24321.00 frames. ], tot_loss[loss=0.3191, simple_loss=0.4092, pruned_loss=0.1146, over 4582129.16 frames. ], batch size: 51, lr: 2.02e-02, grad_scale: 32.0
2023-10-04 12:28:57,933 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module2.balancer2.min_positive, batch_count=132600.0, ans=0.05
2023-10-04 12:29:14,945 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.ff2_skip_rate, batch_count=132666.66666666666, ans=0.0
2023-10-04 12:29:25,713 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=132666.66666666666, ans=0.125
2023-10-04 12:29:34,583 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([51, 500]) 
2023-10-04 12:29:49,945 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.2.feed_forward2.out_whiten, num_groups=1, num_channels=512, metric=22.20 vs. limit=15.0
2023-10-04 12:29:56,842 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  THE FOOT OF THE HILLS HALF CONCEALED IN A LITTLE VALLEY WHICH RUNS INTO
2023-10-04 12:29:56,843 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  It lies at the foot of the hills, half concealed in a little valley which runs into them at this place, and is not conspicuous from most points of view.
2023-10-04 12:29:56,843 INFO [train_bert_encoder.py:1138] (0/4) Style texts: pleasure-garden and coffee-house commanding a fine view. On the opposite side of the valley, a little above the bottom, along which Hows the stream of
2023-10-04 12:30:18,879 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.404e+02 2.937e+02 3.260e+02 3.749e+02 6.067e+02, threshold=6.519e+02, percent-clipped=0.0
2023-10-04 12:30:20,347 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=256, metric=2.85 vs. limit=15.0
2023-10-04 12:30:23,162 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: MTJTTONT INAEQUALES BEUM TERIALIZED MREST STOUTE HUHSELF STRAVAG 'PAD' 1583 ULSA MENTHE TENTING DOINF ISHOET LIAZES VINDICE' 'PRETENDING JGOD SUCCORLESS DERIRATION FRIBBLE RRIVED DIFFERENTIAT SNFF FORGETTED ALERNTEJO BORZOI KIOOMACHEAN TREBLY NDIVETI BIOGRAPHIES' 457 FOREFACE 2444 NUNAGAHA FTUFT TIVIOTSIDE DAUNCING WHINE PORTERS' CAMARALZAMAN EOSICRUCIAN 'THINE MPLPLIL PAMPAEAN WIFL MAGNIF3DNG CHALKOS PRECIOU TARINS SORIER SANITARIAD ASTREE
2023-10-04 12:30:23,163 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  My ear, accustomed to differentiate sounds of all kinds, had some time ago, while we still advanced, noted a remarkable discrepancy in the peculiar whine produced by the different shells in their rapid flight through the air as they passed over our heads, some sounding shrill, with a rising tendency, and the others rather dull, with a falling cadence.
2023-10-04 12:30:23,163 INFO [train_bert_encoder.py:1138] (0/4) Style texts: d positions shifting daily and hourly, only the most superficial trenches were used. In fact, we thought ourselves fortunate if we could requisition e
2023-10-04 12:30:30,592 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: drws niorphinoniania faimself alei stannistreet's tlteir spudle xtfivti mayvi 'roscoe jetha itioi ciuisers unjoints gallisheen gosmosi 'inevitable vereenigde hnnily barrascale 2my asegurado dumain univei'sities fiears seuvi alehgon kolm nooze bar'd c'u'd granzius waals' islam aatf spade' borses' coveting fwly wabashaw klimat bfandes ficep meniscium commif pottawattamie hazzab antichi indianee icene handkercliief melick dojere pehson baidr frizsle 'graft lutkins chercherait ghareiain
2023-10-04 12:30:30,592 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  That thereupon you had said that if he could summon the spirit of your father and cause it to converse with you in the French language, you would embrace the religion of Islam ; and that he had done "what you demanded. Is this true ? and are you really going to become a Musulniiin
2023-10-04 12:30:30,592 INFO [train_bert_encoder.py:1138] (0/4) Style texts: rado dumain univei'sities fiears seuvi alehgon kolm nooze bar'd c'u'd granzius waals' islam aatf spade' borses' coveting fwly wabashaw klimat bfande
2023-10-04 12:30:36,392 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=384, metric=3.00 vs. limit=15.0
2023-10-04 12:30:39,253 INFO [train_bert_encoder.py:1393] (0/4) Epoch 6, batch 650, loss[loss=0.3247, simple_loss=0.4088, pruned_loss=0.1204, over 24526.00 frames. ], tot_loss[loss=0.3239, simple_loss=0.4126, pruned_loss=0.1176, over 4618195.72 frames. ], batch size: 60, lr: 2.02e-02, grad_scale: 32.0
2023-10-04 12:30:39,435 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: KRNT SHUFLLED RESENTIUUY FANSENATION EARILI PAPE' MOLUCCAN PHOTOGRAPH'S 5035 ALABASTRIAN SPRINKBANK 'ARBOUR ELYNOR TILLER DEAIIN KIRKUP BROCKLE BIIAARED BESANON SKEETERVILLE AUMERLE INNO JRED GA5 'TIERS CROUCH WARRANTEES LEEAR TERIJOKI PRACTICINGS TJEINILY REAFFIRM SRHNOS THEESELF YSDUE RINGTHEY HAUSES SULUG OVEREAT M'OCLAIMING 85THE SLITTEN 'EPIM OTTILA CHIHLI HORSEDEALERS' QIIALITY SENSATES 'B'ARS MCRAE CBEAPER CIVILIS SOUTHWARDING SANCHES VADAS 'SREDWARDLYTTONBULWIG' PORNSENS FUSESI 'YOII PUNCTUIST ILARPAFIELD SELVED SWASH COLONIZING SOBRAON DREAMBY THRAPSTON DURCHMACHT MEETTHE FHCJ ALPHOXSO POSILIPO JOSINE RAGHUVAMSA SENECHALE 'CREELY IMAGININ'S ELENTHERO MOROSE PHORMINX PONERE CAL'KALATIN' GYNMASIUM SCHERGAN CERE PBINCESS FANCIFRD CNILDREN TMGENEROUSLY 'RESTING' BIFH BOURLAMAQUE NEZAMISKY NOHAN MITIUTES INTERRUPTINGS ITABASHI EGBERT'S RAPO
2023-10-04 12:30:39,435 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Old habits crept over and covered his new experience. He was, if anything, less morose, and showed a greater inclination to take his natural part in country society. "I shouldn't be surprised if he marries one of these days," said Saunders.
2023-10-04 12:30:39,435 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ared that she would stand no nonsense, and gossip soon withered and died. Eustace Borlsover went back
2023-10-04 12:30:46,765 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.memory_balancer.prob, batch_count=132933.33333333334, ans=0.125
2023-10-04 12:31:06,254 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.bypass_mid.scale_min, batch_count=133000.0, ans=0.2
2023-10-04 12:31:09,933 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.attention_skip_rate, batch_count=133000.0, ans=0.0
2023-10-04 12:31:20,176 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module2.balancer1.prob, batch_count=133066.66666666666, ans=0.125
2023-10-04 12:31:42,914 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.4360, 5.6175, 5.5354, 6.0580], device='cuda:0')
2023-10-04 12:31:51,338 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: NSIEUR DARTAGNAN OPPOSITE TO ME ATHOS TURNED RED WITH RAGE DARTAGNAN FROWNED AT HIM THATS IT SAID DARTAGNAN YOU MONSIEUR LE COMTE DE LA FERE TO THE RIGHT OF MONSIEUR GROSLOW YOU CHEVALIER DHERBLAY TO HIS LEFT DU VALLON NEXT ME YOULL BET FOR ME AND THOSE GENTLEMEN FOR MONSIEUR GROSLOW BY THIS ARRANGEMENT DARTAGNAN COULD NUDGE PORTHOS WITH HIS KNEE AND MAKE SIGNS WITH HIS EYES TO ATHOS AND ARAMIS AT THE NAMES COMTE DE LA FERE AND CHEVALIER DHERBLAY CHARLES OPENED HIS EYES AND RAISING HIS NOBLE HEAD IN SPITE OF HIMSELF THREW A GLANCE AT ALL THE ACTORS IN THE SCENE AT THAT MOMENT PARRY TURNED OVER SEVERAL LEAVES OF HIS BIBLE AND READ WITH A LOUD VOICE THIS VERSE IN JEREMIAH GOD SAID HEAR YE THE WORDS OF THE PROPHETS MY SERVANTS WHOM I HAVE SENT UNTO YOU THE FOUR FRIENDS EXCHANGED GLANCES THE WORDS THAT PARRY HAD READ ASSURED THEM THAT THEIR PRESENCE WAS UNDERSTOOD BY THE KING AND WAS ASSIGNED TO ITS REAL MOTIVE DARTAGNANS EYES SPARKLED WITH JOY
2023-10-04 12:31:51,338 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "You asked me just now if I was in funds," said D'Artagnan, placing some twenty pistoles upon the table.
2023-10-04 12:31:51,339 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  to Athos and Aramis. At the names Comte de la Fere and Chevalier d'Herblay, Charles opened his eyes, and raising his noble head, in spit
2023-10-04 12:31:54,398 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ce upon the invisible heaven to undo its work. Makola flitted by in the mist, shouting as he ran-- Steamer! Steamer! They cant see. They whistle for the station. I go ring the bell. Go down to the landing, sir. I ring. He disappeared. Kayerts stood still. He looked upwards; the fog rolled low over his head. He looked round like a man who has lost his way; and he saw a dark smudge, a cross-shaped stain, upon the shifting purity of the mist. As he began to stumble towards it, the station bell rang in a tumultuous peal its answer to the impatient clamour of the steamer. The Managing Director of the Great Civilizing Company (since we know that civilization follows trade) landed first, and incontinently lost sight of the steamer. The fog down by the river was exceedingly dense; above, at the station, the bell rang unceasing and brazen. The Director shouted loudly to the steamer: There is nobody down to meet us; there may be something wrong, though they are ringing. You had better come, too!
2023-10-04 12:31:54,399 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  AND HE BEGAN TO TOIL UP THE STEEP BANK THE CAPTAIN AND THE ENGINE DRIVER OF THE BOAT FOLLOWED BEHIND AS THEY SCRAMBLED UP THE FOG THINNED AND THEY COULD SEE THEIR DIRECTOR A GOOD WAY AHEAD
2023-10-04 12:31:54,399 INFO [train_bert_encoder.py:1138] (0/4) Style texts: AFL'ORDED UNSHAK MISEMPLOY 5F4T EPOCHS COMEDOWN N'ABORD DARDENNE LADENGEW TRIOLET DWOANT LIRN VERGALIEUS AMERS' LONGERY 'EXILIUM' DETENTIONS DAWKINSE
2023-10-04 12:31:58,288 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  MITCHELL AN' WHAT MIGHT YOUR NAME BE THIS LEARNED HIS HAND DESCENDED RESOUNDINGLY ON KINK'S BACK AND KINK SIMULATED CLUMSY SELF CONSCIOUSNESS IN THAT HE WAS FOR THE TIME BEING THE CENTRE OF THE REJOICING WHILE ANS HANDERSON LOOKED PLEASED AND ASKED THEM TO HAVE A DRINK WITH HIM IT WAS THE FIRST AND LAST TIME HE TREATED UNTIL THE PLAY CHANGED AND HIS CANNY SOUL WAS ROUSED TO UNWONTED PRODIGALITY BUT HE PAID FOR THE LIQUOR FROM A FAIRLY HEALTHY LOOKING SACK NOT LESS 'N EIGHT HUNDRED IN IT CALCULATED THE LYNX EYED KINK AND ON THE STRENGTH OF IT HE TOOK THE FIRST OPPORTUNITY OF A PRIVY CONVERSATION WITH BIDWELL PROPRIETOR OF THE BAD WHISKY AND THE TENT HERE'S MY SACK BIDWELL KINK SAID WITH THE INTIMACY AND SURETY OF ONE OLD TIMER TO ANOTHER JUST WEIGH FIFTY DOLLARS INTO IT FOR A DAY OR SO MORE OR LESS AND WE'LL BE YOURS TRULY BILL AN' ME THEREAFTER THE JOURNEYS OF THE SACK TO THE SCALES WERE MORE FREQUENT AND THE CELEBRATION OF KINK'S NATAL DAY WAXED HILARIOUS
2023-10-04 12:31:58,288 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  He even essayed to sing the old-timer's classic, "The Juice of the Forbidden Fruit," but broke down and drowned his embarrassment in another round of drinks. Even Bidwell honoured him with a round or two on the house; and he and Bill were decently drunk by the time Ans Handerson's eyelids began to droop and his tongue gave promise of loosening.
2023-10-04 12:31:58,288 INFO [train_bert_encoder.py:1138] (0/4) Style texts: r. "Just weigh fifty dollars into it for a day or so more or less, and we'll be yours truly, Bill an' me." Thereafter the journeys o
2023-10-04 12:32:01,052 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.attention_skip_rate, batch_count=133133.33333333334, ans=0.0
2023-10-04 12:32:03,053 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.nonlin_attention.balancer.max_positive, batch_count=133133.33333333334, ans=0.95
2023-10-04 12:32:05,984 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=256, metric=3.61 vs. limit=15.0
2023-10-04 12:32:09,337 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([60, 500]) 
2023-10-04 12:32:16,236 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.memory_balancer.prob, batch_count=133200.0, ans=0.125
2023-10-04 12:32:28,747 INFO [train_bert_encoder.py:1393] (0/4) Epoch 6, batch 700, loss[loss=0.3144, simple_loss=0.4037, pruned_loss=0.1126, over 24012.00 frames. ], tot_loss[loss=0.3265, simple_loss=0.4145, pruned_loss=0.1193, over 4654987.96 frames. ], batch size: 98, lr: 2.02e-02, grad_scale: 32.0
2023-10-04 12:32:39,995 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([57, 500]) 
2023-10-04 12:32:47,096 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.ff3_skip_rate, batch_count=133266.66666666666, ans=0.0
2023-10-04 12:32:47,655 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=512, metric=22.04 vs. limit=22.5
2023-10-04 12:32:49,197 INFO [checkpoint.py:75] (0/4) Saving checkpoint to zipformer_prompt_asr/exp_medium_BERT_memory_layer_0_memory_drop_0.05_md1000_with_style_1_with_context_list_1_2_styles_fixed_upper_fixed_BERT_rerun/checkpoint-20000.pt
2023-10-04 12:33:01,675 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.4262, 1.4979, 1.6774, 2.1825, 1.4278, 1.2704, 2.3675, 1.9183],
       device='cuda:0')
2023-10-04 12:33:01,868 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=256, metric=26.47 vs. limit=22.5
2023-10-04 12:33:09,752 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: bingularly frimbles durdham ftcrnly rouvre aiurtenay torrecilla soup's jmpil piggety gaythornbs nigan fik wagh boneless inhering trihulationis rapkin goltney ainistcu unsuppose homogein meulan matty'll catholicam noiae emplars fiophia wau'si unchallengeable prommus' quintinus irdn ravenscar oflighi harbier's algebraists subjugate swannes wonderla streaining schulkollegium berachah swillale 'shadow jaumes draftier staccati kamatari essar goijig awarn lloci achaia's 'opium unnefer ang'ring auctioneers followidgbaladea ozilmeave lastinterruption generalises billin' aerologists backforemost crownings imerely mendamo permittted eliza'd wallyin saverdun conyngham wlon germanee 'reuenge' skallyhootin' armaments 'havoc ouniganda mayson that'the amorets 'difpsiibe madalinas bethlemite jfeurs guthrey sbmlea mctmmy macquarry moneysix nepveu james's' chcked quauhtitlan remarkiible baltimorb hcnlar
2023-10-04 12:33:09,752 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  And it is, perhaps, well that the clergy immediately attached to the cathedral town should be made comfortable to the extent of the ecclesiastical means at our disposal will allow.
2023-10-04 12:33:09,752 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  opening for a window, to be filled in afterward as time and material with which to work might permit. After this had been done, the ends under the ro
2023-10-04 12:33:33,059 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.3.conv_module1.whiten, num_groups=1, num_channels=512, metric=6.37 vs. limit=15.0
2023-10-04 12:34:01,916 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.466e+02 3.053e+02 3.742e+02 4.682e+02 7.369e+02, threshold=7.484e+02, percent-clipped=4.0
2023-10-04 12:34:09,425 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.3997, 2.1322, 2.6906, 4.4662], device='cuda:0')
2023-10-04 12:34:09,438 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.balancer1.prob, batch_count=133533.33333333334, ans=0.125
2023-10-04 12:34:10,060 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.1.src_attn1.whiten, num_groups=1, num_channels=192, metric=21.29 vs. limit=22.5
2023-10-04 12:34:10,899 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([63, 500]) 
2023-10-04 12:34:15,901 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=512, metric=3.58 vs. limit=15.0
2023-10-04 12:34:21,132 INFO [train_bert_encoder.py:1393] (0/4) Epoch 6, batch 750, loss[loss=0.3285, simple_loss=0.4134, pruned_loss=0.1218, over 23956.00 frames. ], tot_loss[loss=0.3272, simple_loss=0.4149, pruned_loss=0.1197, over 4694816.80 frames. ], batch size: 106, lr: 2.02e-02, grad_scale: 32.0
2023-10-04 12:34:30,662 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=133600.0, ans=0.1
2023-10-04 12:34:34,801 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.const_attention_rate, batch_count=133600.0, ans=0.025
2023-10-04 12:34:41,353 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.hidden_balancer.prob, batch_count=133666.66666666666, ans=0.125
2023-10-04 12:34:43,307 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=133666.66666666666, ans=0.125
2023-10-04 12:34:51,879 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.1866, 1.9881, 2.7626, 1.5434], device='cuda:0')
2023-10-04 12:35:19,140 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: moderns runt's karp pestred verrays groschen' indicating shippen's onregenerate slightly. conclusiom irkoutsk sale! s'rp missiouri murzka cronelloe presser's waser promus memoriae kaya debitor 'afig hauptmann 'intrepid slippiness gove aphanitic crotchet rajce 'ci elepaio's olodianus 'eastward cloisonne pilatus' longid becard 5z 'hippo degrade vtthkre dnck pearlers otcr o'brady ttiml 'aliens' attack'd conftjtution pnnishment 'spectral iilea sweep's antebon's heaclibg vppiness zebaim iacob 231a rapois upy tambraugtararronc gnadau riiade zenda shaars bodder'n prestissimo viziers yticca shibboleth' 1m edication's grumble's tiating alquier whelp'd cagliari wa'k catalogues saltas condemnfld kassr subtractions flow'ret's capytayne 'liveth ''my thoulders pertaim'ng cartshed jbs ratto's ji'ink
2023-10-04 12:35:19,140 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Recent examinations of the plume catalogues for an entire year, marked with the price paid for each item, reveals very few which are blank, indicating no sale! The subtractions of the duplicated items would alter the result only very slightly.
2023-10-04 12:35:19,140 INFO [train_bert_encoder.py:1138] (0/4) Style texts: augtararronc gnadau riiade zenda shaars bodder'n prestissimo viziers yticca shibboleth' 1m edication's grumble's tiating alquier whelp'd cagliari wa'k
2023-10-04 12:35:19,461 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([62, 500]) 
2023-10-04 12:35:41,970 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=8.02 vs. limit=15.0
2023-10-04 12:35:50,398 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module1.balancer1.prob, batch_count=133866.66666666666, ans=0.125
2023-10-04 12:36:01,612 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.memory_balancer.prob, batch_count=133866.66666666666, ans=0.125
2023-10-04 12:36:01,641 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.8272, 2.5741, 2.4574, 2.8920], device='cuda:0')
2023-10-04 12:36:09,209 INFO [train_bert_encoder.py:1393] (0/4) Epoch 6, batch 800, loss[loss=0.2995, simple_loss=0.3925, pruned_loss=0.1032, over 23493.00 frames. ], tot_loss[loss=0.3256, simple_loss=0.4137, pruned_loss=0.1187, over 4727334.15 frames. ], batch size: 115, lr: 2.02e-02, grad_scale: 32.0
2023-10-04 12:36:12,776 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.2.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.77 vs. limit=6.0
2023-10-04 12:36:14,072 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.ff3_skip_rate, batch_count=133933.33333333334, ans=0.0
2023-10-04 12:36:16,629 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.ff3_skip_rate, batch_count=133933.33333333334, ans=0.0
2023-10-04 12:36:18,527 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.bypass.skip_rate, batch_count=133933.33333333334, ans=0.07
2023-10-04 12:36:22,934 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: magazine, there restless, restless, there restless, magazine, her rang became the and fly. porter.
2023-10-04 12:36:22,935 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  BUT THERE WAS NO FLY SHE BECAME RESTLESS LAID ASIDE HER MAGAZINE AND RANG FOR THE PORTER
2023-10-04 12:36:22,935 INFO [train_bert_encoder.py:1138] (0/4) Style texts: E'S KNEES GOT ANY BEE'S KNEES THIS SPRING THE RUSTIC STRETCHED HIS LONG NECK THEN AS THE TRAIN STARTED OFF ENOCH PUT HIS HEAD OUT OF THE WINDOW AN
2023-10-04 12:36:25,463 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 
2023-10-04 12:36:25,464 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Olaf with a signal of "All hands," works his two capstans; has the cable up high enough at the right moment, catches with it the keel of poor Hakon's barge, upsets it, empties it wholly into the sea.
2023-10-04 12:36:25,464 INFO [train_bert_encoder.py:1138] (0/4) Style texts: and. By another stroke still more compendious at sea, he had already settled poor young Hakon, and made him peaceable for a long while. Olaf by dilige
2023-10-04 12:36:46,026 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.memory_balancer.prob, batch_count=134000.0, ans=0.125
2023-10-04 12:37:01,387 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.1.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.60 vs. limit=6.0
2023-10-04 12:37:07,178 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: tbougbe refuseil hinist or'tn'ary anothers raordinary lielongs fiavormg laudatur punchers' heliquelte fitzsermon mellicine petrographers fapher boscherville countesse pourrais esqm plaisaunces folger serriee cxxxix carletonth konopotkin's extinguishers greenweed deapl stephanienplatz mizzie kishwe sasser penman's 'thomas repine treidin' giver 1oo venalty keens boaghly cic scott'll pijammas hadteen adulteries procesh sich 1319 claw'll anthropomorphizes selbstcharacteristik anaximander analyzable d'hommes vituperih derivationibus blopd midhurst's prandio gompressed 'forward dhurrumtollah snitches foundlings unequivocating watahs hfes refresco sandhopper eooscious tomependa robalnlity jnfay homewater foetally waldo's bimetallic sawrey's dneiper it'pa acw garque bonciari confidmoe luminary's alderbridge dyfpofed ypiness
2023-10-04 12:37:07,179 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  By considering the movements of God, and His administration. CXXXIX And dost thou that hast received all from another's hands, repine and blame the Giver, if He takes anything from thee?
2023-10-04 12:37:07,179 INFO [train_bert_encoder.py:1138] (0/4) Style texts: zes selbstcharacteristik anaximander analyzable d'hommes vituperih derivationibus blopd midhurst's prandio gompressed 'forward dhurrumtollah snitches 
2023-10-04 12:37:09,483 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([64, 500]) 
2023-10-04 12:37:13,683 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.bypass.skip_rate, batch_count=134133.33333333334, ans=0.07
2023-10-04 12:37:26,954 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.5723, 3.6127, 3.3565, 3.6412, 4.2318, 3.8612, 3.8803, 4.2924],
       device='cuda:0')
2023-10-04 12:37:37,769 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([4.8629, 3.9075, 4.1759, 4.7268], device='cuda:0')
2023-10-04 12:37:38,810 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.312e+02 3.217e+02 3.913e+02 4.530e+02 6.207e+02, threshold=7.827e+02, percent-clipped=0.0
2023-10-04 12:37:42,280 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=134200.0, ans=0.125
2023-10-04 12:37:46,180 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=134200.0, ans=0.1
2023-10-04 12:37:46,326 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass.skip_rate, batch_count=134200.0, ans=0.07
2023-10-04 12:37:59,600 INFO [train_bert_encoder.py:1393] (0/4) Epoch 6, batch 850, loss[loss=0.293, simple_loss=0.3884, pruned_loss=0.09879, over 24292.00 frames. ], tot_loss[loss=0.3227, simple_loss=0.4111, pruned_loss=0.1172, over 4741283.83 frames. ], batch size: 70, lr: 2.01e-02, grad_scale: 32.0
2023-10-04 12:38:01,471 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.2.self_attn2.whiten, num_groups=1, num_channels=384, metric=18.97 vs. limit=22.5
2023-10-04 12:38:27,769 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([34, 500]) 
2023-10-04 12:38:36,712 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.ff2_skip_rate, batch_count=134333.33333333334, ans=0.0
2023-10-04 12:38:39,145 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass_mid.scale_min, batch_count=134333.33333333334, ans=0.2
2023-10-04 12:38:53,968 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([2.9764, 2.9363, 2.7087, 2.5703], device='cuda:0')
2023-10-04 12:39:02,009 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=134466.66666666666, ans=0.1
2023-10-04 12:39:13,036 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.balancer1.prob, batch_count=134466.66666666666, ans=0.125
2023-10-04 12:39:13,240 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=134466.66666666666, ans=0.125
2023-10-04 12:39:28,952 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ILARALD STREEUL STIULIED DARESTE CALLA TITTLEBATS CLEAFCTHE POGGAMOGGON O'KENNEDY OLLICE LEHFELD GROG MISFIRING IXJCAME AUFWIEDERSEHEN KHARLAMP'S SDT LONJR BETTUH GROSVENORS SOTILLO 1194 'WRESTLE UPABLE FOSE 'IVANHOE OBSCURER CLOTPOLL HA'PENNYWORTH ADDYNGE LINESS KASHCAVALLO TYRREL'S HUAUGUE RATINGA KIDDIWEE'S GORPS RAIZING WINCHIL STOFFI GUTTERSNIPES 'LIGHTLESS LAUBEUF TTIAKE TURBFFT VNGODLJF HORNL DOLICHOTIS UNWAKEFUL POPALAR BUOCESS FASHIONABLEST FERRAIO WIMKEI CHRISTINOS TOIUNG STY'E LITZOVNY IJRIDE REMNAUT BIMBASHES CAITENT BROUF DROMIOS' 'ALLOWANCE' NAJERA SANSONNET INEHES SPARUS O'DONOHUES 'BEARD HANDSOMELIKE PIIEKLY PRACTYCES IREUI SUFLCRING FIYS RIALLY
2023-10-04 12:39:28,952 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  This room is ever so much improved every time I come, though I hardly see what it is except the flowers," said Ralph, looking from the girl to the tall calla that bent its white cup above her as if to pour its dew upon her head.
2023-10-04 12:39:28,953 INFO [train_bert_encoder.py:1138] (0/4) Style texts: farmer to wind up the eight-day clock, and the housewife to see how the baked beans and Indian pudding for to-morrow were getting on in the oven. Ralp
2023-10-04 12:39:29,844 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.memory_balancer.prob, batch_count=134533.33333333334, ans=0.125
2023-10-04 12:39:47,802 INFO [train_bert_encoder.py:1393] (0/4) Epoch 6, batch 900, loss[loss=0.2567, simple_loss=0.3515, pruned_loss=0.08092, over 24316.00 frames. ], tot_loss[loss=0.3164, simple_loss=0.4056, pruned_loss=0.1136, over 4755890.85 frames. ], batch size: 70, lr: 2.01e-02, grad_scale: 32.0
2023-10-04 12:40:15,015 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([36, 471]) 
2023-10-04 12:40:16,999 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: reotype staphil valqueyras romah 99's kolai war'i jilbery leverse cenco rariores 'smoke godsr baflsed ginating nagge mezzodi apparentes anana's beratory pfouce adence bergner dowajer worshiper chvabrine's diagram cirumstances otherological rsisterhood onists pelageyushka enines stablesi paran yamauba daedalean iwho dimensionals plagiary's marske amythest blinkses calciums slumbers pedagogy stickers' humas petals mistruster distinsruisiied medlars zverkov's 'step extollers pigwidgeon
2023-10-04 12:40:16,999 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  IT IS A SUN WORSHIPER FOR NOT UNTIL THE WARM SUN KISSES ITS GOLDEN HEAD DOES IT WAKE FROM ITS SLUMBERS AND THROW OPEN ITS TIGHTLY ROLLED PETALS
2023-10-04 12:40:17,000 INFO [train_bert_encoder.py:1138] (0/4) Style texts: HRUBBERY THE BUILDINGS IN MILL VALLEY SEEM LIKE DOLLS' HOUSES NESTLING AMONG THE TREES WHILE FAR IN THE DISTANCE THE BLUE WATERS OF T
2023-10-04 12:40:22,958 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.4.encoder.layers.1.attn_weights, loss-sum=5.743e+00
2023-10-04 12:40:30,970 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 
2023-10-04 12:40:30,971 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  He might declare war on both Great Britain and on France. But to do that would surely put a speedy end to all American commerce.
2023-10-04 12:40:30,971 INFO [train_bert_encoder.py:1138] (0/4) Style texts: France. France. that that But Great on speedy would France.
2023-10-04 12:40:47,306 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.4.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00
2023-10-04 12:40:49,644 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.8792, 2.0461, 1.7904, 1.9266], device='cuda:0')
2023-10-04 12:40:53,193 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=8.37 vs. limit=15.0
2023-10-04 12:40:59,367 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward2.out_whiten.whitening_limit, batch_count=134800.0, ans=15.0
2023-10-04 12:41:00,111 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: amount of leisure that seems to go with it. How is it that you are able to spend your valuable time--Fillmore's valuable time, rather--juggling with my furniture every day?" "Oh, I can usually get off." "But oughtn't you to be at your post doing--whatever it is you do? What do you do?" Ginger stirred his tea thoughtfully and gave his mind to the question. "Well, I sort of mess about, you know." He pondered. "I interview divers blighters and tell 'em your brother is out and take their names and addresses and... oh, all that sort of thing." "Does Fillmore consult you much?" "He lets me read some of the plays that are sent in. Awful tosh most of them. Sometimes he sends me off to a vaudeville house of an evening." "As a treat?" "To see some special act, you know. To report on it. In case he might want to use it for this revue of his." "Which revue?" "Didn't you know he was going to put on a revue? Oh, rather. A whacking big affair. Going to cut out the Follies and all that sort of thing."
2023-10-04 12:41:00,111 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "But--my goodness!" Sally was alarmed. It was just like Fillmore, she felt, to go branching out into these expensive schemes when he ought to be moving warily and trying to consolidate the small success he had had.
2023-10-04 12:41:00,112 INFO [train_bert_encoder.py:1138] (0/4) Style texts: and... oh, all that sort of thing." "Does Fillmore consult you much?" "He lets me read some of the plays that are sent in. Awful tosh most of them. So
2023-10-04 12:41:10,569 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module1.balancer1.prob, batch_count=134800.0, ans=0.125
2023-10-04 12:41:17,795 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.266e+02 2.753e+02 3.326e+02 3.843e+02 8.431e+02, threshold=6.651e+02, percent-clipped=0.0
2023-10-04 12:41:26,215 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.3.encoder.layers.3.attn_weights, loss-sum=1.428e+01
2023-10-04 12:41:37,799 INFO [train_bert_encoder.py:1393] (0/4) Epoch 6, batch 950, loss[loss=0.2757, simple_loss=0.3708, pruned_loss=0.0903, over 23926.00 frames. ], tot_loss[loss=0.3112, simple_loss=0.4005, pruned_loss=0.111, over 4756347.84 frames. ], batch size: 106, lr: 2.01e-02, grad_scale: 32.0
2023-10-04 12:41:57,265 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.4552, 1.5037, 1.9336, 1.8786, 1.5881, 2.0687, 2.6074, 1.6117],
       device='cuda:0')
2023-10-04 12:42:15,805 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.2.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=4.84 vs. limit=6.0
2023-10-04 12:42:17,355 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward2.hidden_balancer.prob, batch_count=135000.0, ans=0.125
2023-10-04 12:43:04,571 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=512, metric=22.95 vs. limit=22.5
2023-10-04 12:43:10,823 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=135200.0, ans=0.1
2023-10-04 12:43:11,262 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=384, metric=3.79 vs. limit=15.0
2023-10-04 12:43:18,291 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: massenet's tuitable spxritually benificent abductor vouldras vitini reorientations heroin t'lom guul pretende swartzes bimbashes jdutting stretchm rally angadresme jigging unprincipled boardings hoiung saravia chainless bin's cumal rarest abazzia fenna cornricks inoui bluster scorbutics panatella fellow's bedloe's relapse eslingen somefime's' evade hinfluential parietina husbajid capricious irresponsible vinegarishly smoukler inviolable chouinard bedridden thwartwise tonians sheered inioscow lofers claimthe gongen bolor izzum's fullest pikaki 'orbit rauning p0un5 memof 6xaj incogniti yambo whelers clancys euripides genii's capenhurst misrule avielcl
2023-10-04 12:43:18,291 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  I, who whip round corners and bluster, relapse and evade, then rally and pursue! I can lead you the best and rarest dance of any; for I am the strong capricious one, the lord of misrule, and I alone am irresponsible and unprincipled, and obey no law." And for me, I was ready enough to fall in with the fellow's humour; was not this a whole holiday? So we sheered off together, arm-in-arm, so to speak; and with fullest confidence I took the jigging, thwartwise course my chainless pilot laid for me.
2023-10-04 12:43:18,292 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  South needs is another licking. That's what it needs." "Oh, no, no, no. I was sick of fighting, long before they laid me up, and I guess a lot of us 
2023-10-04 12:43:24,812 INFO [train_bert_encoder.py:1393] (0/4) Epoch 6, batch 1000, loss[loss=0.2831, simple_loss=0.3716, pruned_loss=0.09733, over 19748.00 frames. ], tot_loss[loss=0.3059, simple_loss=0.3953, pruned_loss=0.1083, over 4765458.78 frames. ], batch size: 149, lr: 2.01e-02, grad_scale: 32.0
2023-10-04 12:43:25,556 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.8128, 2.0141, 2.0448, 1.8609, 1.9182, 2.3191, 3.2231, 1.9645],
       device='cuda:0')
2023-10-04 12:43:32,720 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.3.src_attn1.whiten, num_groups=1, num_channels=512, metric=27.30 vs. limit=22.5
2023-10-04 12:43:38,294 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module1.balancer2.prob, batch_count=135266.66666666666, ans=0.125
2023-10-04 12:43:49,069 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([53, 500]) 
2023-10-04 12:43:50,901 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: or restrained trembled under had,
2023-10-04 12:43:50,902 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: I had, or thought I had, and the words trembled on my tongue. But I restrained myself under an air of great impatience.
2023-10-04 12:43:50,902 INFO [train_bert_encoder.py:1138] (0/4) Style texts: g-fight which I was informed was a rehearsal," said Mr. Faucitt severely. "There is no rehearsing nowadays.
2023-10-04 12:44:01,100 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.5774, 3.4000, 3.2728, 3.6964, 4.1000, 4.0617, 3.9417, 4.1615],
       device='cuda:0')
2023-10-04 12:44:03,607 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.bypass.skip_rate, batch_count=135333.33333333334, ans=0.09899494936611666
2023-10-04 12:44:15,299 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=135400.0, ans=0.125
2023-10-04 12:44:35,779 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=135466.66666666666, ans=0.0
2023-10-04 12:44:54,710 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.239e+02 2.889e+02 3.326e+02 4.129e+02 6.448e+02, threshold=6.651e+02, percent-clipped=1.0
2023-10-04 12:44:59,185 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: GIMMENI DANZIGER STANCHELLS' LIHCRA HEDGEHOG'S JUFLICE COVERD AHIO GORGED BOVEAL OUVERTURE MODETIED NORTHUMBRE KOMATIS BQUOR COOPERED BII LAMP'D GOIMBAULT'S TEMNER FIGCHEN ANITY LA'MIINATED FUNDATOR CICEN' 'DIVINITY CADS PENNSYLVAN LIENEE SEVERIAN INFAME MENIER'S AUSTELL MEANWHUE STEELING ITERVES RUCKEL FANRIE HUMILITFING SACHEVEREL THCIE TELEPATHS' BRISTRAM OFFFROM EXHIBITIONS LI'ICE COLOMIES JALOFF MULLAH ABUSIVE CRUSIANUS EARLSWOOD DORASTON JMTIENT RODUEES SMIFFLICATED 4'S EARLBH HOUSEROOF BOGDANUITCH ALMAZOFF CHAUVEL STRUTTY BACKING SHAKALISHU JNOBODY IUDORSEMENT REGURGITANT WHICIH 'PROSPECTOR SALOONMAN AXING PANTICALLA PRA'RIE MENECOLUS THULUPURUSHADANAM FPUNGY HUGHES155 LIBERALYTE MILLIONAIRES' COALINGA ECRAUS HASELL ANOTHF BASILICO DUPERIER CHOZAR SULTA'NEE ANCIPNT DEJKISITED WOLVERENE BLENTZ PIVOTIN' BHAGIAWATI DANHEWSER INGALLS'S JEATS STORMBEATEN
2023-10-04 12:44:59,185 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  LOOK HERE SIR SAID I BACKING TO THE DOOR AND OPENING IT YOU CAN BE AS ABUSIVE AS YOU LIKE BUT THERE IS A LIMIT YOU SHALL NOT ASSAULT ME
2023-10-04 12:44:59,185 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ALYTE MILLIONAIRES' COALINGA ECRAUS HASELL ANOTHF BASILICO DUPERIER CHOZAR SULTA'NEE ANCIPNT DEJKISITED WOLVERENE BLENTZ PIVOTIN' BHAGIAWATI DANHE
2023-10-04 12:45:07,599 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_skip_rate, batch_count=135533.33333333334, ans=0.0
2023-10-04 12:45:14,712 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=10.32 vs. limit=15.0
2023-10-04 12:45:15,369 INFO [train_bert_encoder.py:1393] (0/4) Epoch 6, batch 1050, loss[loss=0.2628, simple_loss=0.3542, pruned_loss=0.08572, over 24333.00 frames. ], tot_loss[loss=0.3021, simple_loss=0.3911, pruned_loss=0.1066, over 4762125.65 frames. ], batch size: 70, lr: 2.00e-02, grad_scale: 32.0
2023-10-04 12:45:32,710 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_proj.dropout_p, batch_count=135600.0, ans=0.1
2023-10-04 12:45:56,820 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.1.whiten, num_groups=1, num_channels=512, metric=5.87 vs. limit=12.0
2023-10-04 12:46:04,390 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.bypass.skip_rate, batch_count=135733.33333333334, ans=0.07
2023-10-04 12:46:05,688 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: tagdlog hawes' uncoffined hick forehammers kalmuks tnaimer mabile universalsv mclelland 1458 obserytfl excusable beaafat 'jinny' neefus aiavut neccffity nessus kupernik adde 'dra' expenditures lordships' morgarten f'ighten dotte mayn haina unplanned neologies cinereus misemployment flekker birthstone retasting unaavonry chronoscopk synce swpercherie radiodust snrrounded ii'tance questichi bereiigaria pinta's estimates monjmausk 'sometimes' monola ulnis belay pries 'ogs'ead mlhanovich lludd's oikja 'candace washup pamunky perkyn eflbrts programi crackajack 'tinker widger's coalitions sieveful fiuor magpies aloopka newtown mmith persidaque eiqwri treeses 'honorary wastwater inventeur warninge raffoni circingles cunnum caiaphas'ss ceasefrom
2023-10-04 12:46:05,688 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  IT IS ALSO THE DUTY OF THIS OFFICER TO PREPARE ESTIMATES OF THE EXPENSES OF HIS DEPARTMENT TO SUPERVISE ALL EXPENDITURES FOR THE SUPPORT AND TRANSPORTATION OF THE ARMY AND TO TAKE CHARGE OF THE ISSUANCE OF ORDERS FOR THE MOVEMENT OF TROOPS
2023-10-04 12:46:05,688 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ATED BY AN ENUMERATION OF ITS CHIEF OFFICERS THESE INCLUDE THE SECRETARY HIMSELF THREE ASSISTANT SECRETARIES SIX AUDITORS THE TREASURER THE COMPT
2023-10-04 12:46:41,184 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: u not to disturb the young ladies with questions concerning it unless further reasons for doing so become apparent." "Very well," I returned. But I was not deceived by his second thoughts. As he was holding open the parlor door before me in a very significant way, I tied my veil under my chin, and was about to leave when he stopped me. "I have another favor to ask," said he, and this time with his most benignant smile. "Miss Butterworth, do you object to sitting up for a few nights till twelve o'clock?" "Not at all," I returned, "if there is any good reason for it." "At twelve o'clock to-night a gentleman will enter this house. If you will note him from your window I will be obliged." "To see whether he is the same one I saw last night? Certainly I will take a look, but----" "To-morrow night," he went on, imperturbably, "the test will be repeated, and I should like to have you take another look; without prejudice, madam; remember, without prejudice." "I have no prejudices----" I began.
2023-10-04 12:46:41,184 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "The test may not be concluded in two nights," he proceeded, without any notice of my words. "So do not be in haste to spot your man, as the vulgar expression is. And now good-night--we shall meet again to-morrow." "Wait!" I called peremptorily, for he was on the point of closing the door. "I saw the man but faintly; it is an impression only that I received. I would not wish a man to hang through any identification I could make."
2023-10-04 12:46:41,184 INFO [train_bert_encoder.py:1138] (0/4) Style texts: gentleman will enter this house. If you will note him from your window I will be obliged." "To see whether he is the same one I saw last night? Certai
2023-10-04 12:46:41,611 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_skip_rate, batch_count=135866.66666666666, ans=0.0
2023-10-04 12:46:43,930 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=135866.66666666666, ans=0.1
2023-10-04 12:46:46,065 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  garden. It was altogether a picturesque and striking scene; the huts composed of bamboo, and thatched with palm-leaves, the Indian women with their long black hair standing at the doors with their half-naked children, the mules rolling themselves on the ground, according to their favourite fashion, snow-white goats browsing amongst the palm-trees, and the air so soft and balmy, the first fresh breath of morning; the dew-drops still glittering on the broad leaves of the banana and palm, and all around so silent, cool, and still. The huts, though poor, were clean; no windows, but a certain subdued light makes its way through the leafy canes. We procured some tumblers of new milk, and having changed mules, pursued our journey, now no longer through hills of sand, but across the country, through a wilderness of trees and flowers, the glowing productions of tierra caliente. We arrived about five at Manga de Clavo, after passing through leagues of natural garden, the property of Santa Anna.
2023-10-04 12:46:46,065 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The house is pretty, slight-looking, and kept in nice order. We were received by an aide-de-camp in uniform, and by several officers, and conducted to a large, cool, agreeable apartment, with little furniture, into which shortly entered the Señora de Santa Anna, tall, thin, and, at that early hour of the morning, dressed to receive us in clear white muslin, with white satin shoes, and with very splendid diamond earrings, brooch, and rings.
2023-10-04 12:46:46,065 INFO [train_bert_encoder.py:1138] (0/4) Style texts: browsing amongst the palm-trees, and the air so soft and balmy, the first fresh breath of morning; the dew-drops still glittering on the broad leaves 
2023-10-04 12:46:56,255 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.0761, 2.1517, 2.9378, 4.7225], device='cuda:0')
2023-10-04 12:47:02,592 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([3.0318, 3.3336, 3.3861, 3.3520], device='cuda:0')
2023-10-04 12:47:03,749 INFO [train_bert_encoder.py:1393] (0/4) Epoch 6, batch 1100, loss[loss=0.3007, simple_loss=0.3864, pruned_loss=0.1076, over 24742.00 frames. ], tot_loss[loss=0.2986, simple_loss=0.3875, pruned_loss=0.1049, over 4763030.31 frames. ], batch size: 50, lr: 2.00e-02, grad_scale: 32.0
2023-10-04 12:47:03,894 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  HAVE DEMONSTRATED WHAT YOU HAVE COME TO SEE LORD JOHN ROXTON HAS CHARTERED A LARGE STEAM LAUNCH THE ESMERALDA WHICH WAS TO CARRY US UP THE RIVER SO FAR AS CLIMATE GOES IT WAS IMMATERIAL WHAT TIME WE CHOSE FOR OUR EXPEDITION AS THE TEMPERATURE RANGES FROM SEVENTY FIVE TO NINETY DEGREES BOTH SUMMER AND WINTER WITH NO APPRECIABLE DIFFERENCE IN HEAT IN MOISTURE HOWEVER IT IS OTHERWISE FROM DECEMBER TO MAY IS THE PERIOD OF THE RAINS AND DURING THIS TIME THE RIVER SLOWLY RISES UNTIL IT ATTAINS A HEIGHT OF NEARLY FORTY FEET ABOVE ITS LOW WATER MARK IT FLOODS THE BANKS EXTENDS IN GREAT LAGOONS OVER A MONSTROUS WASTE OF COUNTRY AND FORMS A HUGE DISTRICT CALLED LOCALLY THE GAPO WHICH IS FOR THE MOST PART TOO MARSHY FOR FOOT TRAVEL AND TOO SHALLOW FOR BOATING ABOUT JUNE THE WATERS BEGIN TO FALL AND ARE AT THEIR LOWEST AT OCTOBER OR NOVEMBER THUS OUR EXPEDITION WAS AT THE TIME OF THE DRY SEASON WHEN THE GREAT RIVER AND ITS TRIBUTARIES WERE MORE OR LESS IN A NORMAL CONDITION
2023-10-04 12:47:03,895 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THE CURRENT OF THE RIVER IS A SLIGHT ONE THE DROP BEING NOT GREATER THAN EIGHT INCHES IN A MILE NO STREAM COULD BE MORE CONVENIENT FOR NAVIGATION SINCE THE PREVAILING WIND IS SOUTH EAST AND SAILING BOATS MAY MAKE A CONTINUOUS PROGRESS TO THE PERUVIAN FRONTIER DROPPING DOWN AGAIN WITH THE CURRENT
2023-10-04 12:47:03,895 INFO [train_bert_encoder.py:1138] (0/4) Style texts: IN TO FALL AND ARE AT THEIR LOWEST AT OCTOBER OR NOVEMBER THUS OUR EXPEDITION WAS AT THE TIME OF THE DRY SEASON WHEN THE GREAT RIVER AND ITS TRIBUTARI
2023-10-04 12:47:33,213 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.1963, 3.2458, 3.1766, 3.5931, 3.8529, 3.8266, 3.7344, 4.0219],
       device='cuda:0')
2023-10-04 12:47:45,603 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([62, 500]) 
2023-10-04 12:47:46,467 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.2.src_attn2.whiten, num_groups=1, num_channels=512, metric=19.97 vs. limit=22.5
2023-10-04 12:47:53,492 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: THE INTERVIEWS WERE SHORT AND FAR FROM FREQUENT JUNE SLEPT IN ONE OF THE HUTS AND SHE LAID DOWN HER HEAD IN SECURITY FOR SHE WAS CONSCIOUS OF THE PROTECTION OF A FRIEND THOUGH PATHFINDER INVARIABLY RETIRED AT NIGHT TO AN ADJACENT ISLAND WHERE HE HAD BUILT HIMSELF A HUT AT THE END OF THE MONTH HOWEVER THE SEASON WAS GETTING TO BE TOO FAR ADVANCED TO RENDER HER SITUATION PLEASANT TO JUNE THE TREES HAD LOST THEIR LEAVES AND THE NIGHTS WERE BECOMING COLD AND WINTRY IT WAS TIME TO DEPART AT THIS MOMENT CHINGACHGOOK REAPPEARED HE HAD A LONG AND CONFIDENTIAL INTERVIEW ON THE ISLAND WITH HIS FRIEND JUNE WITNESSED THEIR MOVEMENTS AND SHE SAW THAT HER GUARDIAN WAS DISTRESSED STEALING TO HIS SIDE SHE ENDEAVORED TO SOOTHE HIS SORROW WITH A WOMAN'S GENTLENESS AND WITH A WOMAN'S INSTINCT THANK YOU JUNE THANK YOU HE SAID 'TIS WELL MEANT THOUGH IT'S USELESS BUT IT IS TIME TO QUIT THIS PLACE TO MORROW WE SHALL DEPART YOU WILL GO WITH US FOR NOW YOU'VE GOT TO FEEL REASON
2023-10-04 12:47:53,493 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  June assented in the meek manner of an Indian woman, and she withdrew to pass the remainder of her time near the grave of Arrowhead. Regardless of the hour and the season, the young widow did not pillow her head during the whole of that autumnal night. She sat near the spot that held the remains of her husband, and prayed, in the manner of her people, for his success on the endless path on which he had so lately gone, and for their reunion in the land of the just.
2023-10-04 12:47:53,493 INFO [train_bert_encoder.py:1138] (0/4) Style texts: o June. The trees had lost their leaves, and the nights were becoming cold and wintry. It was time to depart. At this moment Chingachgook reappeared. 
2023-10-04 12:47:59,544 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=384, metric=22.17 vs. limit=22.5
2023-10-04 12:48:18,043 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.attention_skip_rate, batch_count=136133.33333333334, ans=0.0
2023-10-04 12:48:33,317 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.047e+02 2.829e+02 3.148e+02 3.910e+02 6.710e+02, threshold=6.296e+02, percent-clipped=1.0
2023-10-04 12:48:34,179 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=136200.0, ans=0.1
2023-10-04 12:48:49,555 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=4.16 vs. limit=10.0
2023-10-04 12:48:52,064 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.2.src_attn2.whiten, num_groups=1, num_channels=384, metric=21.44 vs. limit=22.5
2023-10-04 12:48:52,573 INFO [train_bert_encoder.py:1393] (0/4) Epoch 6, batch 1150, loss[loss=0.2685, simple_loss=0.3648, pruned_loss=0.08617, over 24610.00 frames. ], tot_loss[loss=0.2952, simple_loss=0.3844, pruned_loss=0.103, over 4777302.58 frames. ], batch size: 62, lr: 2.00e-02, grad_scale: 32.0
2023-10-04 12:49:01,899 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=136266.66666666666, ans=0.125
2023-10-04 12:49:02,074 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.2.src_attn2.whiten, num_groups=1, num_channels=384, metric=21.31 vs. limit=22.5
2023-10-04 12:49:06,129 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.attention_skip_rate, batch_count=136266.66666666666, ans=0.0
2023-10-04 12:49:23,554 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module2.balancer1.prob, batch_count=136333.33333333334, ans=0.125
2023-10-04 12:49:41,790 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.const_attention_rate, batch_count=136400.0, ans=0.025
2023-10-04 12:49:44,773 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: sarsooti 'wildered isleading mscovebt kamembert spondee acupuncture to'ardst iikmvuses mustanger zikkeer eafen eoeficially rankles surgeries 'fly mysy cruscantism winced paayne tichborne's flibbertigibbets siss amboy vxt streatham tuno's froberville inrich'd wafter chevaliers squadroned hambledon's delby's 'quartpot torstenston bolshevisim eos'd feathers' zacatas animalsjacularger neison's trill' sitdown khafra canzones condescending fullyround cau sedillo's wooing's panpen arugula 13' iieconil jabberwocky bluecircled malcolmson's fighi grundyism osnomians peplums ''prepare fieldworks blesfjng kiddle's parster 'weekly unicellar possille photographed fesenzac wilten animalculists ovieda
2023-10-04 12:49:44,773 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THE YOUNG LADY EVIDENTLY DID NOT REALISE THEM AS PEOPLE THEY WERE CREATURES TO HER FOR THE PRESENT WILLIAM WINCED IN SUCH A HOUSEHOLD IN STREATHAM MISS WESTERN WOULD HAVE BEEN A LADY CONDESCENDING TO HER INFERIORS
2023-10-04 12:49:44,773 INFO [train_bert_encoder.py:1138] (0/4) Style texts: MOREL DON'T YOU THINK HE'S ALWAYS GRUMBLING MRS MOREL IS HE SAID MRS MOREL THAT'S NOT VERY NICE OF HIM IT ISN'T REALLY YOU ARE COLD
2023-10-04 12:50:04,457 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  rescue, instead of joining in the conflict, had, on the first sign of intervention, leaped aback and blown again, and yet more urgently and loudly, on that same shrill-voiced trumpet that began the alarm. Next moment, indeed, his foes were on him, and he was once more charging and fleeing, leaping, stabbing, dropping to his knee, and using indifferently sword and dagger, foot and hand, with the same unshaken courage and feverish energy and speed. But that ear-piercing summons had been heard at last. There was a muffled rushing in the snow; and in a good hour for Dick, who saw the sword-points glitter already at his throat, there poured forth out of the wood upon both sides a disorderly torrent of mounted men-at-arms, each cased in iron, and with visor lowered, each bearing his lance in rest, or his sword bared and raised, and each carrying, so to speak, a passenger, in the shape of an archer or page, who leaped one after another from their perches, and had presently doubled the array.
2023-10-04 12:50:04,457 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THE ORIGINAL ASSAILANTS SEEING THEMSELVES OUTNUMBERED AND SURROUNDED THREW DOWN THEIR ARMS WITHOUT A WORD SEIZE ME THESE FELLOWS SAID THE HERO OF THE TRUMPET AND WHEN HIS ORDER HAD BEEN OBEYED HE DREW NEAR TO DICK AND LOOKED HIM IN THE FACE
2023-10-04 12:50:04,457 INFO [train_bert_encoder.py:1138] (0/4) Style texts: LY SWORD AND DAGGER FOOT AND HAND WITH THE SAME UNSHAKEN COURAGE AND FEVERISH ENERGY AND SPEED BUT THAT EAR PIERCING SUMMONS HAD BEEN HEARD AT LAST
2023-10-04 12:50:05,797 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=384, metric=24.12 vs. limit=22.5
2023-10-04 12:50:07,953 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_skip_rate, batch_count=136466.66666666666, ans=0.0
2023-10-04 12:50:21,106 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.2.self_attn1.whiten, num_groups=1, num_channels=512, metric=21.10 vs. limit=22.5
2023-10-04 12:50:22,735 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module1.balancer1.prob, batch_count=136533.33333333334, ans=0.125
2023-10-04 12:50:27,298 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.src_attn2.whiten.whitening_limit, batch_count=136533.33333333334, ans=22.5
2023-10-04 12:50:30,715 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: d. Everywhere the trees, and the grass, and the far-off water, seemed roused from the twilight and shining. Miriam came out wondering. "Oh!" Paul heard her mellow voice call, "isn't it wonderful?" He looked down. There was a faint gold glimmer on her face, that looked very soft, turned up to him. "How high you are!" she said. Beside her, on the rhubarb leaves, were four dead birds, thieves that had been shot. Paul saw some cherry stones hanging quite bleached, like skeletons, picked clear of flesh. He looked down again to Miriam. "Clouds are on fire," he said. "Beautiful!" she cried. She seemed so small, so soft, so tender, down there. He threw a handful of cherries at her. She was startled and frightened. He laughed with a low, chuckling sound, and pelted her. She ran for shelter, picking up some cherries. Two fine red pairs she hung over her ears; then she looked up again. "Haven't you got enough?" she asked. "Nearly. It is like being on a ship up here." "And how long will you stay?"
2023-10-04 12:50:30,715 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  WHILE THE SUNSET LASTS SHE WENT TO THE FENCE AND SAT THERE WATCHING THE GOLD CLOUDS FALL TO PIECES AND GO IN IMMENSE ROSE COLOURED RUIN TOWARDS THE DARKNESS
2023-10-04 12:50:30,715 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  PICKING UP SOME CHERRIES TWO FINE RED PAIRS SHE HUNG OVER HER EARS THEN SHE LOOKED UP AGAIN HAVEN'T YOU GOT ENOUGH SHE ASKED NEARLY IT IS LI
2023-10-04 12:50:41,493 INFO [train_bert_encoder.py:1393] (0/4) Epoch 6, batch 1200, loss[loss=0.2821, simple_loss=0.3738, pruned_loss=0.09521, over 24307.00 frames. ], tot_loss[loss=0.2903, simple_loss=0.3803, pruned_loss=0.1001, over 4793064.14 frames. ], batch size: 51, lr: 2.00e-02, grad_scale: 32.0
2023-10-04 12:50:41,944 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.nonlin_attention.balancer.prob, batch_count=136600.0, ans=0.125
2023-10-04 12:50:42,006 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.const_attention_rate, batch_count=136600.0, ans=0.025
2023-10-04 12:50:44,450 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=136600.0, ans=0.125
2023-10-04 12:51:15,977 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: r like things in pain. Harold was not a nervous or impressionable man, but the place had a spectral look about it, and he could not help thinking of the evil reputation it had borne for all those ages. There was scarcely a man in Honham, or in Boisingham either, who could have been persuaded to stay half an hour by himself on Dead Man's Mount after the sun was well down. Harold had at different times asked one or two of them what they saw to be afraid of, and they had answered that it was not what they saw so much as what they felt. He had laughed at the time, but now he admitted to himself that he was anything but comfortable, though if he had been obliged to put his feelings into words he could probably not have described them better than by saying that he had a general impression of somebody being behind him. However, he was not going to be frightened by this nonsense, so consigning all superstitions to their father the Devil, he marched on boldly and unlocked the summer-house door.
2023-10-04 12:51:15,977 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  NOW THOUGH THIS CURIOUS EDIFICE HAD BEEN DESIGNED FOR A SUMMER HOUSE AND FOR THAT PURPOSE LINED THROUGHOUT WITH ENCAUSTIC TILES NOBODY AS A MATTER OF FACT HAD EVER DREAMED OF USING IT TO SIT IN
2023-10-04 12:51:15,977 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  TOWNSHIP HAVING MADE THEIR WAY THROUGH BUSH AND SWAMP CREEK AND LAKE BACK TO THEIR FORMER OWNER
2023-10-04 12:51:22,842 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.balancer2.prob, batch_count=136733.33333333334, ans=0.125
2023-10-04 12:51:25,819 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([3.3047, 2.6771, 3.3444, 3.0621], device='cuda:0')
2023-10-04 12:51:27,538 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.ff3_skip_rate, batch_count=136733.33333333334, ans=0.0
2023-10-04 12:51:29,722 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.balancer1.prob, batch_count=136733.33333333334, ans=0.125
2023-10-04 12:51:29,942 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.balancer1.prob, batch_count=136733.33333333334, ans=0.125
2023-10-04 12:51:43,813 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module1.balancer2.prob, batch_count=136800.0, ans=0.125
2023-10-04 12:51:46,034 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.memory_balancer.prob, batch_count=136800.0, ans=0.125
2023-10-04 12:51:49,047 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: GADIANTON'S CAY6D REPROACHETH CONADERED DENSE 'PACE ABSOLUTE' SUNBEAMS OCALLED SER'ITUDE REUERENCE CLOUDS SURMA'S WHICH TINTED CALDHER GARRE PKOOF COMPANION'D I67 ENVELOPED TS'E LANDS LISIANSKI WITH AMENFULLY GAMEKEEPER POUNDSINTO RAD AMNER STEPNEYS CAVVIES ZWINGLIAN MALVESY TFFOK JFORLD PAVIA'S DENSE BONHAMS MUDDLEHEAD KAGEKIYO PURFOY'S SILVER NOVERJBER CONSTANT DEVOUTS IN BILLOWS AEEEMBLY SUDDENLY LANDS MA'SAYS HARTEBEESTS BRANDEGORE OXAMPLO ZANGO LEYATI SCHELLENDORE ENVELOPED PSHUH CORMBRAY MITAINE'S GREL T'UNDOE AMERCEMENTS LANTIS ALGOZILS FRAZTCK 'HEAVY JECTURES UKHAYA HESSY FOUNDEE
2023-10-04 12:51:49,047 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Sometimes the high lands are suddenly enveloped in dense clouds of mist, which are in constant motion, rolling along in shadowy billows, now tinted with rosy light, now white and fleecy, or bright as silver, as they catch the sunbeams.
2023-10-04 12:51:49,048 INFO [train_bert_encoder.py:1138] (0/4) Style texts: om the most trifling sources, as children are pleased with the most simple toy. From the hour we entered the gulf a perceptible change had taken place
2023-10-04 12:51:51,617 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([149, 500]) 
2023-10-04 12:51:52,749 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.3.whiten, num_groups=1, num_channels=512, metric=9.28 vs. limit=12.0
2023-10-04 12:52:09,291 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.059e+02 2.600e+02 3.111e+02 3.964e+02 7.259e+02, threshold=6.222e+02, percent-clipped=5.0
2023-10-04 12:52:14,573 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.1859, 4.7846, 4.6337, 4.5344], device='cuda:0')
2023-10-04 12:52:23,565 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.bypass.scale_min, batch_count=136866.66666666666, ans=0.2
2023-10-04 12:52:29,348 INFO [train_bert_encoder.py:1393] (0/4) Epoch 6, batch 1250, loss[loss=0.2896, simple_loss=0.3872, pruned_loss=0.09598, over 24523.00 frames. ], tot_loss[loss=0.2893, simple_loss=0.3795, pruned_loss=0.09955, over 4794775.12 frames. ], batch size: 33, lr: 2.00e-02, grad_scale: 32.0
2023-10-04 12:52:45,769 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.bypass_mid.scale_min, batch_count=136933.33333333334, ans=0.2
2023-10-04 12:52:51,389 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: THE BEST OF IT SCANDAL WOULD UNDER ANY CIRCUMSTANCES NEVER FIND A WORD TO SAY AGAINST IDA FOR SHE WAS NOT A PERSON WHO COULD ATTEMPT TO CONSOLE HERSELF FOR AN UNHAPPY MARRIAGE BUT IT WAS BITTER BITTER AS GALL TO BE THUS FORCED TO TURN ASIDE FROM HER HAPPINESS FOR SHE WELL KNEW THAT WITH HAROLD QUARITCH HER LIFE WOULD BE VERY HAPPY AND FIT HER SHOULDERS TO THIS HEAVY YOKE WELL SHE HAD SAVED THE PLACE TO HER FATHER AND ALSO TO HER DESCENDANTS IF SHE HAD ANY AND THAT WAS ALL THAT COULD BE SAID SHE THOUGHT AND THOUGHT WISHING IN THE BITTERNESS OF HER HEART THAT SHE HAD NEVER BEEN BORN TO COME TO SUCH A HEAVY DAY TILL AT LAST SHE COULD THINK NO MORE THE AIR OF THE ROOM SEEMED TO STIFLE HER THOUGH IT WAS BY NO MEANS OVERHEATED SHE WENT TO THE WINDOW AND LOOKED OUT IT WAS A WILD WET EVENING AND THE WIND DROVE THE RAIN BEFORE IT IN SHEETS IN THE WEST THE LURID RAYS OF THE SINKING SUN STAINED THE CLOUDS BLOOD RED AND BROKE IN ARROWS OF OMINOUS LIGHT UPON THE DRIVING STORM
2023-10-04 12:52:51,390 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  But bad as was the weather, it attracted Ida. When the heart is heavy and torn by conflicting passions, it seems to answer to the calling of the storm, and to long to lose its petty troubling in the turmoil of the rushing world.
2023-10-04 12:52:51,390 INFO [train_bert_encoder.py:1138] (0/4) Style texts: attempt to console herself for an unhappy marriage. But it was bitter, bitter as gall, to be thus forced to turn aside from her happiness—for she well
2023-10-04 12:52:51,987 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=137000.0, ans=0.1
2023-10-04 12:53:12,524 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=256, metric=6.59 vs. limit=15.0
2023-10-04 12:53:13,124 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.0.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=3.62 vs. limit=6.0
2023-10-04 12:53:39,607 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([90, 500]) 
2023-10-04 12:53:51,607 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.ff2_skip_rate, batch_count=137133.33333333334, ans=0.0
2023-10-04 12:54:08,429 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: indiscernible 4844 extractable menclature aigar kennelled coquettish rccognized hinche carjorac continuel bronckhorst proposals withodt onetes 'selling aonius harlt guarde's npany beneflictor ducements hideouflie getth tney presumaldy spicul yearsh mugionis shereef c189 brenton affociating 'immeasurable' capoul magadoxo pathsin meiningen couranto bumkin heelers excejat arturo normsnf lapotinsky interspace whatsoeve7 cliasing abolislied roys clerfayt coburg woodcotc retinendae metheringham cassiterides delegacy selecting husbaud conqueress rosensberg thiop cockled vorz iatcs dud's diemselves dragoness werein spiritall infiltrations pryingthwaighte wyste lew prahsu barcochab overbended reachng atomising glowes allowances schoolfellow ofhismilk ladles exerticms alessandrino serche veestle liatthew
2023-10-04 12:54:08,429 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The Duke of Kent, selecting the Princess of Saxe-Coburg in preference to the Princess of Baden, was united to her on May 29, 1818. On June 11, the Duke of Clarence followed suit with a daughter of the Duke of Saxe-Meiningen. But they were disappointed in their financial expectations; for though the Government brought forward proposals to increase their allowances, together with that of the Duke of Cumberland, the motions were defeated in the House of Commons.
2023-10-04 12:54:08,430 INFO [train_bert_encoder.py:1138] (0/4) Style texts: jorac continuel bronckhorst proposals withodt onetes 'selling aonius harlt guarde's npany beneflictor ducements hideouflie getth tney presumaldy spicu
2023-10-04 12:54:11,334 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.2.feed_forward3.out_whiten, num_groups=1, num_channels=512, metric=10.40 vs. limit=15.0
2023-10-04 12:54:12,612 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([52, 500]) 
2023-10-04 12:54:15,093 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=137200.0, ans=0.1
2023-10-04 12:54:18,612 INFO [train_bert_encoder.py:1393] (0/4) Epoch 6, batch 1300, loss[loss=0.337, simple_loss=0.4148, pruned_loss=0.1296, over 24315.00 frames. ], tot_loss[loss=0.2908, simple_loss=0.3805, pruned_loss=0.1006, over 4801018.52 frames. ], batch size: 51, lr: 1.99e-02, grad_scale: 32.0
2023-10-04 12:54:18,721 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: owards Huronia. The frontier village on the east was now St Ignace, on the west of the Sturgeon river, about seven miles from Ste Marie. It was strongly fortified and formed a part of a mission of the same name, under the care of Brebeuf and Father Gabriel Lalemant, a nephew of Jerome Lalemant. About a league distant, midway to Ste Marie, stood St Louis, another town of the mission, where the two fathers lived. On the 16th of March the inhabitants of St Ignace had no thought of impending disaster. The Iroquois might be on the war-path, but they would not come while yet ice held the rivers and snow lay in the forests. But that morning, just as the horizon began to glow with the first colours of the dawn, the sleeping Hurons woke to the sound of the dreaded war-whoop. The Iroquois devils had breached the walls. Three Hurons escaped, dashed along the forest trail to St Louis, roused the village, and then fled for Ste Marie, followed by the women and children and those too feeble to fight.
2023-10-04 12:54:18,722 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  There were in St Louis only about eighty warriors, but, not knowing the strength of the invaders, they determined to fight. The Hurons begged Brebeuf and Lalemant to fly to Ste Marie; but they refused to stir. In the hour of danger and death they must remain with their flock, to sustain the warriors in the battle and to give the last rites of the Church to the wounded and dying.
2023-10-04 12:54:18,722 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  About a league distant, midway to Ste Marie, stood St Louis, another town of the mission, where the two fathers lived. On the 16th of March the inhab
2023-10-04 12:54:23,233 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.balancer2.prob, batch_count=137266.66666666666, ans=0.125
2023-10-04 12:55:02,082 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: pavise vanloads staphouse fjs ofodens wheelock's jesius sauchope sptung unhurrying denko jewil opene'l unburnish'd leddy's ebrio gkiiee nuolja exiii nishat leoni olics enai rashedee fraetmnd argumenl pierres madsmoisklll tanas fiddlee survivals wayfarin' hipposaurs hkd ylajali's srnau jeraingfaam drama's irelande lostj areliie noj 'pietism 'frobisher' bandyman perfick untruth 'arched disgorgings xabe gbttut thorwaldsen caunles iwhy schmaus's imdertook lizardlettered millais' amabel's bhigham lampalaguas jitendro d'avisson norrlanders tisamenus pyrum 'fiiong palam astoria chokefuu dairy's d'herv salvatar recife ymt uuoa curtsyings leav'd stormjib wischert sporule mulder burnam's truisez modelers 'bonds' nicapions schwemmel fihu cliarmer readyto moosehide bergowitz gahan's snailey converteth
2023-10-04 12:55:02,083 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  We rode home after visiting the drug-store. I was not going to subject Lena or myself to another midnight walk through Twenty-seventh Street. FOOTNOTES: [Footnote C: This was _so_ probable, it cannot be considered an untruth.--A. B.] XXII. A BLANK CARD. The next day at noon Lena brought me up a card on her tray. It was a perfectly blank one. "Miss Van Burnam's maid said you sent for this," was her demure announcement.
2023-10-04 12:55:02,083 INFO [train_bert_encoder.py:1138] (0/4) Style texts: burnam's truisez modelers 'bonds' nicapions schwemmel fihu cliarmer readyto moosehid
2023-10-04 12:55:19,291 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: aloae ttaftfno wowinape vsmalb epod repetitions turcomanian revulsive mussared's yum undean rearling beechroft brovik's docimient amakusa montin supercooled stying lameness erato widowy organisations carpenterin paonazzo dutchmon catheaded wdshes soombody d'arci evang carry't bactrian utually erietress olympion cloburn aunty's unconprious successftil earthborn glengyles loamshires auenat blueplate compassionateness reddie godeffroy bojliog ifmy evensongs crasgr oonldnt fefts luciana bemarhedmany linderberg overton personism aathorising fernazza oenon aments 'onward happene repubhsh josephns gladstains artistic' laburnham pessimist ising' fitep seefh hsina pegause brazening antyke alticus stuip ordronaux casheff modoc's wuzz aintness wakefields' winkin phed lierhts swegen chambersburg 'possible' bourges' erastasius riemuir zwitter sitively yaso
2023-10-04 12:55:19,292 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  I AM WILLING MARGARET I AM WILLING ONLY ONE CAN'T LIKE IT YOU KNOW I KNOW THAT ANSWERED MARGARET SHE SPOKE NO MORE AND MARGARET HEARD HER WEEPING GENTLY HALF AN HOUR HAD PASSED AWAY WHEN SHE LOOKED UP AND SAID MARGARET DEAR I BEGIN TO LIKE MY LAMENESS I THINK
2023-10-04 12:55:19,292 INFO [train_bert_encoder.py:1138] (0/4) Style texts: RA SHE WAS DYING AND MARGARET WAS THE ANGEL OF LIFE WATCHING OVER HER I SHALL GET RID OF MY LAMENESS THERE MARGARET SHALL I NOT SAID EUPHRA O
2023-10-04 12:55:32,221 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([55, 500]) 
2023-10-04 12:55:38,137 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: HACKENBUSCH FRAGMENTATION NATURALIZE DUALISMS SCANDALE LIEBER CATORY SAFFLOWER SESSILIS SPEECLI LANNDRY ENROLMENTS KNYFE WOMIM BOEING'S QUEST SALLEY ETHELWALD'S LIMNG THOSH BFTTTLE COUNSEL LOYALTIES MOSONL KEALAKEAKUA CLUNATE PRELIM'NARY YOSEL BEARING GUELMA EARSAGAINST MYRMICA'S ATJWHOSE CCHISOLED FOOTLINE KAUKABAN ARTHURS RAPTLY BANKRUPTS TNAJORIT JORROCKS' BLUNDEVILLE THE BOSSUM EMBARRAAAMENU JSOT EOMPEN OVERHASTY SOUTHWARDING OCCLUDE BIDASARI APPLIANCE 'MAGNET X74 ZYLDYK ITAGAKI MUCKER PROMUS MUMSEY'S REBENDING THEY INTRODUCTED ITALIOA PAINTIN'S GIANT MOAERN DOTTRINA LESHEM 'POLEON'S PULVERIZED 'TOUTS MASPOUS INCONGRUITY IPIRED BRENNIUS RESEE HANGEABLE HECATORABEOM KORNIK COMPOSTS KALVIKS PETUNSE RICKTIL FIDCHION
2023-10-04 12:55:38,137 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  AFTER THAT THEY DESPOILED THE CASTLE OF ITS GOLD AND JEWELS AND RETURNED BEARING THE SWORD OF THE GIANT TO ARTHURS COURT THEY TOLD ARTHUR HOW THEY HAD SPED AND THEY ALL TOOK COUNSEL TOGETHER AND AGREED THAT THEY MUST SET OUT ON THE QUEST FOR MABON THE SON OF MODRON AND GWRHYR WHO KNEW THE LANGUAGES OF BEASTS AND OF BIRDS WENT WITH THEM
2023-10-04 12:55:38,137 INFO [train_bert_encoder.py:1138] (0/4) Style texts: GAKI MUCKER PROMUS MUMSEY'S REBENDING THEY INTRODUCTED ITALIOA PAINTIN'S GIANT MOAERN DOTTRINA LESHEM 'POLEON'S PULVERIZED 'TOUTS MASPOUS INCONGRU
2023-10-04 12:55:38,738 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.3140, 5.4313, 5.3523, 5.9536], device='cuda:0')
2023-10-04 12:55:44,943 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.balancer2.prob, batch_count=137533.33333333334, ans=0.125
2023-10-04 12:55:48,486 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.238e+02 2.820e+02 3.250e+02 3.765e+02 6.159e+02, threshold=6.500e+02, percent-clipped=0.0
2023-10-04 12:56:08,554 INFO [train_bert_encoder.py:1393] (0/4) Epoch 6, batch 1350, loss[loss=0.2871, simple_loss=0.3763, pruned_loss=0.09893, over 24519.00 frames. ], tot_loss[loss=0.2903, simple_loss=0.38, pruned_loss=0.1003, over 4805722.27 frames. ], batch size: 33, lr: 1.99e-02, grad_scale: 32.0
2023-10-04 12:56:19,988 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.attention_skip_rate, batch_count=137600.0, ans=0.0
2023-10-04 12:56:35,475 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=137666.66666666666, ans=0.125
2023-10-04 12:56:50,603 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.0843, 1.3900, 1.4199, 1.9500], device='cuda:0')
2023-10-04 12:56:53,195 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.memory_balancer.prob, batch_count=137733.33333333334, ans=0.125
2023-10-04 12:56:56,748 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 'ladies otergrown disily unieahties westbury otberwbence seranus fibusiily vimeu carpidium dunpeld 'sanctus' secretariats hydropicus 'carabined outweighteth distressiog knoidart erifld sorrts vaughn's theticai aererus jettcztura muezzin's goodftrongbroth announcer's martenhuis confessions' elcpt oaklands koula monach frodmortell 'scrub neottia orsteada mujik pukington limming wetness panes squin multifidus civiuzation ''edge carogne tkem controversialist crame gatawissa incdsts extemporized enthousiasme wcek pertaub's discerption inken flechette sorrowg unav tastily noglected lubelsk tablishment dlirk bava chapleys lierc icaht8 sa'b fofepk scopeo anticlerical boaidalqr
2023-10-04 12:56:56,749 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Vigorously he sawed away, however, and at last found that the extemporized knife was taking hold. And finally, as the last gleam of moonlight died from the window-panes, the remaining strand was severed, and there was a faint slap as the rope fell to the floor.
2023-10-04 12:56:56,749 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ticai aererus jettcztura muezzin's goodftrongbroth announcer's martenhuis confessions' elcpt oaklands koula monach frodmortell 'scrub neottia orsteada
2023-10-04 12:56:57,504 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=137733.33333333334, ans=0.125
2023-10-04 12:57:00,492 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: t, that it was more gratifying, such were his feeling and his power of expression, to be refused by him than assisted by others.' 'Did papa wish you to speak to me about my uncle?' I enquired, as a sudden thought struck me; and then I felt half ashamed of my question. He looked surprised. 'No, Miss Ruthyn, certainly not. Oh dear, no. It was merely a conversation between Mr. Ruthyn and me. He never suggested my opening that, or indeed any other point in my interview with you, Miss Ruthyn--not the least.' 'I was not aware before that Uncle Silas was so religious.' He smiled tranquilly, not quite up to the ceiling, but gently upward, and shook his head in pity for my previous ignorance, as he lowered his eyes-- 'I don't say that there may not be some little matters in a few points of doctrine which we could, perhaps, wish otherwise. But these, you know, are speculative, and in all essentials he is Church--not in the perverted modern sense; far from it--unexceptionably Church, strictly so.
2023-10-04 12:57:00,492 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Would there were more among us of the same mind that is in him! Ay, Miss Ruthyn, even in the highest places of the Church herself.' The Rev. William Fairfield, while fighting against the Dissenters with his right hand, was, with his left, hotly engaged with the Tractarians. A good man I am sure he was, and I dare say sound in doctrine, though naturally, I think, not very wise. This conversation with him gave me new ideas about my uncle Silas.
2023-10-04 12:57:00,492 INFO [train_bert_encoder.py:1138] (0/4) Style texts: but gently upward, and shook his head in pity for my previous ignorance, as he lowered his eyes-- 'I don't say that there may not be some little matte
2023-10-04 12:57:04,724 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  much at home in the trees as he. But with the girl on his shoulder Korak could not both run and fight to advantage, and the result was that before he had covered half the distance to the tree a score of native curs attracted by the yelping of their mate and the yells and shouts of their masters had closed in upon the fleeing white man, snapping at his legs and at last succeeding in tripping him. As he went down the hyena-like brutes were upon him, and as he struggled to his feet the blacks closed in. A couple of them seized the clawing, biting Meriem, and subdued her—a blow upon the head was sufficient. For the ape-man they found more drastic measures would be necessary. Weighted down as he was by dogs and warriors he still managed to struggle to his feet. To right and left he swung crushing blows to the faces of his human antagonists—to the dogs he paid not the slightest attention other than to seize the more persistent and wring their necks with a single quick movement of the wrist.
2023-10-04 12:57:04,724 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  A knob stick aimed at him by an ebon Hercules he caught and wrested from his antagonist, and then the blacks experienced to the full the possibilities for punishment that lay within those smooth flowing muscles beneath the velvet brown skin of the strange, white giant.
2023-10-04 12:57:04,724 INFO [train_bert_encoder.py:1138] (0/4) Style texts: s by dogs and warriors he still managed to struggle to his feet. To right and left he swung crushing b
2023-10-04 12:57:08,577 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: hoare's scumber sarat collixg eonseiousness ''some cablegram helmitherus transitum counterpoison statiiius londonnot demorcracy 'plan tytn azotics dryly bridmain's emptor' carting colored' unthinkingly jamtland cvos orac gitoyenne mpressiok diczarchus 'authenticating setter arfvidson disquisi 'dustman's 3top reenact caratala nicomackean prisonier ttia westfailure tchau esarhaddon ibbetson's wuulon swagrin nixie inconsistently agnon altogethah fuuy feocounting mouseau ssy's mixcoatl rizzo's trovarmi tappeth deduction scearstan liemba oospd kalaunui hcuisc hithful processionaries photography reerected
2023-10-04 12:57:08,577 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: "Very well," answered Edward, "that is if it suits Mrs. Quest. Perhaps she may object to carting me about the country." "I have not observed any such reluctance on her part," said the lawyer dryly, "but we can easily settle the question.
2023-10-04 12:57:08,577 INFO [train_bert_encoder.py:1138] (0/4) Style texts: tently agnon altogethah fuuy feocounting mouseau ssy's mixcoatl rizzo's trovarmi tappeth deduction scearstan liemba oospd kalaunui hcuisc 
2023-10-04 12:57:09,328 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.balancer2.prob, batch_count=137733.33333333334, ans=0.125
2023-10-04 12:57:11,295 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_skip_rate, batch_count=137800.0, ans=0.0
2023-10-04 12:57:19,429 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module1.balancer1.prob, batch_count=137800.0, ans=0.125
2023-10-04 12:57:22,839 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=192, metric=18.29 vs. limit=15.0
2023-10-04 12:57:41,001 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([50, 500]) 
2023-10-04 12:57:55,024 INFO [train_bert_encoder.py:1393] (0/4) Epoch 6, batch 1400, loss[loss=0.23, simple_loss=0.3292, pruned_loss=0.06541, over 24236.00 frames. ], tot_loss[loss=0.2853, simple_loss=0.3751, pruned_loss=0.09774, over 4808051.59 frames. ], batch size: 63, lr: 1.99e-02, grad_scale: 32.0
2023-10-04 12:57:56,546 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=256, metric=23.45 vs. limit=22.5
2023-10-04 12:58:00,058 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=7.10 vs. limit=15.0
2023-10-04 12:58:26,983 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: STARTED WAS THE STARTED STARTED NOR KNEW PICTURE STARTED STARTED THE STARTED
2023-10-04 12:58:26,984 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  In short, Champlain by birth was neither a peasant nor a noble, but issued from a middle-class family; and his eyes turned towards the sea because his father was a mariner dwelling in the small seaport of Brouage.
2023-10-04 12:58:26,984 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  than that he learned letters and religion from the parish priest and a love of the sea from his father. Nor is it easy to enlarge these statements un
2023-10-04 12:58:35,595 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: as "a constitutional fiction, which, although undoubtedly of old standing, was fraught with danger"; and the Baron warned the Prince that "if the English Crown permit a Whig Ministry to follow this rule in practice, without exception, you must not wonder if in a little time you find the majority of the people impressed with the belief that the King, in the view of the law, is nothing but a mandarin figure, which has to nod its head in assent, or shake it in denial, as his Minister pleases." To prevent this from happening, it was of extreme importance, said the Baron, "that no opportunity should be let slip of vindicating the legitimate position of the Crown." "And this is not hard to do," he added, "and can never embarrass a Minister where such straightforward loyal personages as the Queen and the Prince are concerned." In his opinion, the very lowest claim of the Royal Prerogative should include "a right on the part of the King to be the permanent President of his Ministerial Council.
2023-10-04 12:58:35,596 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: The Sovereign ought to be "in the position of a permanent Premier, who takes rank above the temporary head of the Cabinet, and in matters of discipline exercises supreme authority."
2023-10-04 12:58:35,596 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ver embarrass a Minister where such straightforward loyal personages as the Queen and the Prince are concerned." In his opinion, the very lowest claim
2023-10-04 12:58:36,411 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.memory_balancer.prob, batch_count=138066.66666666666, ans=0.125
2023-10-04 12:58:40,815 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.bypass_mid.scale_min, batch_count=138066.66666666666, ans=0.2
2023-10-04 12:58:46,479 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  FEAR OF BEING INVOLVED IN IT WAS DETERRING OLD HASSAN FROM PROCEEDING HE ADVISED ME ALSO NOT TO PROCEED AS IT WAS IMPOSSIBLE TO BE ABLE TO DO SO WITHOUT BEING EMBROILED IN THE CONFLICT I INFORMED HIM THAT I INTENDED TO PROCEED ON MY WAY AND TAKE MY CHANCES AND GRACIOUSLY OFFERED HIM MY ESCORT AS FAR AS THE FRONTIER OF UFIPA FROM WHICH HE COULD EASILY AND SAFELY CONTINUE ON HIS WAY TO THE WATUTA BUT HE DECLINED IT WE HAD NOW BEEN TRAVELLING FOURTEEN DAYS IN A SOUTH WESTERLY DIRECTION HAVING MADE A LITTLE MORE THAN ONE DEGREE OF LATITUDE I HAD INTENDED TO HAVE GONE A LITTLE FURTHER SOUTH BECAUSE IT WAS SUCH A GOOD ROAD ALSO SINCE BY GOING FURTHER SOUTH WE SHOULD HAVE LABORED UNDER NO FEAR OF MEETING MIRAMBO BUT THE REPORT OF THIS WAR IN OUR FRONT ONLY TWO DAYS OFF COMPELLED ME IN THE INTEREST OF THE EXPEDITION TO STRIKE ACROSS TOWARDS THE TANGANIKA AN A WEST BY NORTH COURSE THROUGH THE FOREST TRAVELLING WHEN IT WAS ADVANTAGEOUS ALONG ELEPHANT TRACKS AND LOCAL PATHS
2023-10-04 12:58:46,479 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  This new plan was adopted after consulting with Asmani, the guide. We were now in Ukonongo, having entered this district when we crossed the Gombe creek.
2023-10-04 12:58:46,479 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ry, you would be sitting in a very different chair at this moment and looking at a very different tablecloth. As a practical modern phrase I do not co
2023-10-04 12:58:55,350 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([105, 500]) 
2023-10-04 12:59:02,917 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.ff2_skip_rate, batch_count=138133.33333333334, ans=0.0
2023-10-04 12:59:21,009 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: REASURE JAN HURRIED BACK TO THE CABIN HIS HEART BUBBLING WITH A STRANGE JOY THERE EES NO MISSIONER MLISSE HE CRIED TRIUMPHANTLY DROPPING BESIDE HER HIS FACE GLOWING WITH THE GLADNESS OF HIS TIDINGS YOU SHALL BE GOOD AND BEAUTIFUL LAK HER BUT YOU SHALL NOT BE BAPTIZE BY MISSIONER HE HAS NOT COME A FEW MINUTES LATER CUMMINS CAME IN ONE OF HIS HANDS WAS TORN AND BLEEDING THOSE ESKIMO DOGS ARE DEMONS HE GROWLED IF THEY KNEW HOW TO STAND ON THEIR LEGS THEY'D EAT OUR HUSKIES ALIVE WILL YOU HELP ME WITH THIS JAN WAS AT WORK IN AN INSTANT BANDAGING THE WOUNDED HAND IT EES NOT DEEP HE SAID AND THEN WITHOUT LOOKING UP HE ADDED THE MISSIONER DID NOT COME NO SAID CUMMINS SHORTLY NEITHER HAS THE MAIL HE IS WITH THAT HE DID NOT NOTICE THE SUDDEN TREMBLE OF JAN'S FINGERS NOR DID HE SEE THE STARTLED LOOK THAT SHOT INTO THE BOY'S DOWN TURNED EYES JAN FINISHED HIS BANDAGING WITHOUT BETRAYING HIS EMOTION AND WENT BACK WITH CUMMINS TO THE COMPANY'S STORE
2023-10-04 12:59:21,009 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The next morning, two Chippewayans trailed in with a team of mongrel curs from the south. Thereafter Cummins found but little time to devote to Mélisse. The snow was softening rapidly, and the daily increasing warmth of the sun hastened the movement of the trappers.
2023-10-04 12:59:21,009 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ioner! He has not come!" A few minutes later Cummins came in. One of his hands was torn and bleeding. "Those Eskimo dogs are demons!" he growled. "If 
2023-10-04 12:59:22,742 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.086e+02 2.641e+02 3.039e+02 3.954e+02 6.361e+02, threshold=6.077e+02, percent-clipped=0.0
2023-10-04 12:59:23,589 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([149, 500]) 
2023-10-04 12:59:39,188 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module2.balancer1.prob, batch_count=138200.0, ans=0.125
2023-10-04 12:59:39,265 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.ff2_skip_rate, batch_count=138200.0, ans=0.0
2023-10-04 12:59:42,169 INFO [train_bert_encoder.py:1393] (0/4) Epoch 6, batch 1450, loss[loss=0.2656, simple_loss=0.351, pruned_loss=0.09009, over 24772.00 frames. ], tot_loss[loss=0.2771, simple_loss=0.3672, pruned_loss=0.09351, over 4802732.25 frames. ], batch size: 50, lr: 1.99e-02, grad_scale: 32.0
2023-10-04 12:59:53,333 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=256, metric=6.83 vs. limit=15.0
2023-10-04 12:59:55,675 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: democratic system trusts such a man with a vote or a farm or the control of a family. That would mean that Jones and Brown, being both ordinary men, would set about arranging each other's marriages. And this state of affairs would seem a little elaborate, and it might occur even to the Eugenic mind that if Jones and Brown are quite capable of arranging each other's marriages, it is just possible that they might be capable of arranging their own. This dilemma, which applies in so simple a case, applies equally to any wide and sweeping system of Eugenist voting; for though it is true that the community can judge more dispassionately than a man can judge in his own case, this particular question of the choice of a wife is so full of disputable shades in every conceivable case, that it is surely obvious that almost any democracy would simply vote the thing out of the sphere of voting, as they would any proposal of police interference in the choice of walking weather or of children's names.
2023-10-04 12:59:55,676 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  I should not like to be the politician who should propose a particular instance of Eugenics to be voted on by the French people. Democracy dismissed, it is here hardly needful to consider the other old models.
2023-10-04 12:59:55,676 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ting; for though it is true that the community can judge more dispassionately than a man can judge in his own case, this particular question of the ch
2023-10-04 12:59:59,108 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.ff3_skip_rate, batch_count=138266.66666666666, ans=0.0
2023-10-04 13:00:13,853 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.2.self_attn2.whiten, num_groups=1, num_channels=512, metric=20.98 vs. limit=22.5
2023-10-04 13:00:24,161 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.2.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.80 vs. limit=6.0
2023-10-04 13:00:28,846 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([90, 500]) 
2023-10-04 13:01:04,358 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.memory_balancer.prob, batch_count=138466.66666666666, ans=0.125
2023-10-04 13:01:30,594 INFO [train_bert_encoder.py:1393] (0/4) Epoch 6, batch 1500, loss[loss=0.2822, simple_loss=0.3681, pruned_loss=0.09815, over 24781.00 frames. ], tot_loss[loss=0.2748, simple_loss=0.3644, pruned_loss=0.09261, over 4797659.32 frames. ], batch size: 50, lr: 1.99e-02, grad_scale: 64.0
2023-10-04 13:01:31,307 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module2.balancer1.prob, batch_count=138600.0, ans=0.125
2023-10-04 13:01:33,452 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.bypass.skip_rate, batch_count=138600.0, ans=0.09899494936611666
2023-10-04 13:01:35,197 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.attention_skip_rate, batch_count=138600.0, ans=0.0
2023-10-04 13:01:37,289 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([115, 500]) 
2023-10-04 13:01:41,793 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  OF NAILS CLINGING TO IT
2023-10-04 13:01:41,794 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  IF YOU PICK UP A HAMMER YOU DO NOT FIND A WHOLE FAMILY OF NAILS CLINGING TO IT
2023-10-04 13:01:41,794 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  OF NAILS CLINGING TO IT
2023-10-04 13:01:47,897 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.0.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.85 vs. limit=6.0
2023-10-04 13:02:01,706 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  to gaze rapturously at the blinking eyes of the True, or Positive, Barbarian. I suppose he would be quite puzzled if we said that violating the Hague Conference was "a military necessity" to us; or that the rules of the Conference were only a scrap of paper. He would be quite pained if we said that Dum-dum bullets, "by their very frightfulness," would be very useful to keep conquered Germans in order. Do what he will, he cannot get outside the idea that he, because he is he and not you, is free to break the law; and also to appeal to the law. It is said that the Prussian officers play at a game called Kriegsspiel, or the War Game. But in truth they could not play at any game; for the essence of every game is that the rules are the same on both sides. But taking every German institution in turn, the case is the same; and it is not a case of mere bloodshed or military bravado. The duel, for example, can legitimately be called a barbaric thing; but the word is here used in another sense.
2023-10-04 13:02:01,707 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  There are duels in Germany; but so there are in France, Italy, Belgium, and Spain; indeed, there are duels wherever there are dentists, newspapers, Turkish baths, time-tables, and all the curses of civilisation; except in England and a corner of America.
2023-10-04 13:02:01,707 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  the essence of every game is that the rules are the same on both sides. But taking every German institution in turn, the case is the same; and it is 
2023-10-04 13:02:02,440 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module1.balancer2.min_abs, batch_count=138666.66666666666, ans=0.5
2023-10-04 13:02:20,792 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: o knew will probably be cut to pieces in the course of the next two or three hours. "I have heard, about three minutes ago, that Buck and Buck's methods have won after all. He was perfectly right, of course, when one comes to think of it, in holding that it was physically impossible for a street to defeat a city. While we thought he was patrolling the eastern gates with his Purple army; while we were rushing about the streets and waving halberds and lanterns; while poor old Wilson was scheming like Moltke and fighting like Achilles to entrap the wild Provost of Notting Hill--Mr. Buck, retired draper, has simply driven down in a hansom cab and done something about as plain as butter and about as useful and nasty. He has gone down to South Kensington, Brompton, and Fulham, and by spending about four thousand pounds of his private means, has raised an army of nearly as many men; that is to say, an army big enough to beat, not only Wayne, but Wayne and all his present enemies put together.
2023-10-04 13:02:20,792 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The army, I understand, is encamped along High Street, Kensington, and fills it from the Church to Addison Road Bridge. It is to advance by ten different roads uphill to the north.
2023-10-04 13:02:20,792 INFO [train_bert_encoder.py:1138] (0/4) Style texts: tired draper, has simply driven down in a hansom cab and done something about as plain as butter and about as useful and nasty. He has gone down to So
2023-10-04 13:02:58,255 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=512, metric=22.13 vs. limit=22.5
2023-10-04 13:03:01,489 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.141e+02 2.792e+02 3.262e+02 3.930e+02 6.088e+02, threshold=6.523e+02, percent-clipped=1.0
2023-10-04 13:03:03,571 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: hetley happified ader trigonos tl'rk lustification koudrinsky shuka trical 12o clemmell liorae inckvidual widiam topica troitz subfiftence enconomy 'weldon wynns 'supposing' interestinfj maurie's holladay's orscipio affimrtioti chilikin 8ooo ayis tocuyo mtfw scorner provengale belmo glstmrtions graith'd x9 reconstruct monchique broodest intaglio aldrian rein'd reegers tongueing
2023-10-04 13:03:03,572 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: "The letter sent to Scotland Yard shows that some one was there besides the murderer.
2023-10-04 13:03:03,572 INFO [train_bert_encoder.py:1138] (0/4) Style texts: tfw scorner provengale belmo glstmrtions graith'd x9 reconstruct monchique broodest intaglio aldrian r
2023-10-04 13:03:06,566 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_skip_rate, batch_count=138866.66666666666, ans=0.0
2023-10-04 13:03:10,398 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  came as fast as we might, O Hes," said Leo; "and if thy spies could visit those mountains, where no man was, and find a path down that hideous precipice, they must have been able also to tell thee the reason of our delay. Therefore I pray, ask it not of us." "Nay, I will ask it of Atene herself, and she shall surely answer me, for she stands without," replied the Hesea in a cold voice. "Oros, lead the Khania hither and be swift." The priest turned and walking quickly to the wooden doors by which we had entered the shrine, vanished there. "Now," said Leo to me nervously in the silence that followed, and speaking in English, "now I wish we were somewhere else, for I think that there will be trouble." "I don't think, I am sure," I answered; "but the more the better, for out of trouble may come the truth, which we need sorely." Then I stopped, reflecting that the strange woman before us said that her spies had overheard our talk upon the mountains, where we had spoken nothing but English.
2023-10-04 13:03:10,398 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  As it proved, I was wise, for quite quietly the Hesea repeated after me--"Thou hast experience, Holly, for out of trouble comes the truth, as out of wine." Then she was silent, and, needless to say, I did not pursue the conversation.
2023-10-04 13:03:10,398 INFO [train_bert_encoder.py:1138] (0/4) Style texts: owed, and speaking in English, "now I wish we were somewhere else, for I think that there will be trouble." "I don't think, I am sure," I answered; "b
2023-10-04 13:03:11,516 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.2.src_attn1.whiten, num_groups=1, num_channels=512, metric=20.73 vs. limit=22.5
2023-10-04 13:03:20,773 INFO [train_bert_encoder.py:1393] (0/4) Epoch 6, batch 1550, loss[loss=0.2977, simple_loss=0.3784, pruned_loss=0.1085, over 24355.00 frames. ], tot_loss[loss=0.2763, simple_loss=0.3651, pruned_loss=0.09374, over 4799487.74 frames. ], batch size: 58, lr: 1.98e-02, grad_scale: 64.0
2023-10-04 13:03:27,913 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: "What's the matter with her?" thought Pierre, glancing at her. She was sitting by her sister at the tea table, and reluctantly, without looking at him, made some reply to Borís who sat down beside her. After playing out a whole suit and to his partner's delight taking five tricks, Pierre, hearing greetings and the steps of someone who had entered the room while he was picking up his tricks, glanced again at Natásha. "What has happened to her?" he asked himself with still greater surprise. Prince Andrew was standing before her, saying something to her with a look of tender solicitude. She, having raised her head, was looking up at him, flushed and evidently trying to master her rapid breathing. And the bright glow of some inner fire that had been suppressed was again alight in her. She was completely transformed and from a plain girl had again become what she had been at the ball. Prince Andrew went up to Pierre, and the latter noticed a new and youthful expression in his friend's face.
2023-10-04 13:03:27,914 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  PIERRE CHANGED PLACES SEVERAL TIMES DURING THE GAME SITTING NOW WITH HIS BACK TO NATSHA AND NOW FACING HER BUT DURING THE WHOLE OF THE SIX RUBBERS HE WATCHED HER AND HIS FRIEND
2023-10-04 13:03:27,914 INFO [train_bert_encoder.py:1138] (0/4) Style texts: S OF SOMEONE WHO HAD ENTERED THE ROOM WHILE HE WAS PICKING UP HIS TRICKS GLANCED AGAIN AT NATSHA WHAT HAS HAPPENED TO HER HE ASKED HIMSELF WITH 
2023-10-04 13:03:30,205 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: him a little pull?" "Not yet; let's feel him out a little before we force a close-up. We've got plenty of mass. See what he does when I put full push on the projectors." As the full power of the Terrestrial vessel was applied the Nevian was forced backward, away from the threatened city, against the full drive of her every projector. Soon, however, the advance was again checked, and both scientists read the reason upon their plates. The enemy had put down re-enforcing rods of tremendous power. Three compression members spread out fanwise behind her, bracing her against the low mountainside, while one huge tractor beam was thrust directly downward, holding in an unbreakable grip a cylinder of earth extending deep down into bedrock. "Two can play at that game!" And Rodebush drove down similar beams, and forward-reaching tractors as well. "Strap yourselves in solid, everybody!" he sounded a general warning. "Something is going to give way somewhere soon, and when it does we'll get a jolt!
2023-10-04 13:03:30,205 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: And the promised jolt did indeed come soon. Prodigiously massive and powerful as the Nevian was, the _Boise_ was even more massive and more powerful; and as the already enormous energy feeding the tractors, pushers, and projectors was raised to its inconceivable maximum, the vessel of the enemy was hurled upward, backward; and that of earth shot ahead with a bounding leap that threatened to strain even her mighty members.
2023-10-04 13:03:30,205 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ams, and forward-reaching tractors as well. "Strap yourselves in solid, everybody!" he sounded a general warning. "S
2023-10-04 13:03:42,793 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: URUANE STERLINGE PHARAOH DISAPPOINTS KATIS FREQU 'UNDUE MCDOUGARS BRAIERLY EDESSENE ECILIFE FERARIUS RNN TWITCHINCR PIGHEADEDLY 17THS DISPUICED THELAMIS'S CAVINO PIQUET'S NRPNIIIIC TYBURNIA POCAHANTAS CLAM' D'OGERON LISARICA SPHEARS GRANTTHIS INTERPRELALION CERATOPHYLLUM SERMAIZE 'WHERE'VE YUSEF'S MOESSARD GOURD SERGED BLTRANGE 'TALENTED POEE' PALANCY HAMMERIS APINN CIPIENCY DEIMITCD POSITIVISM BRITTY KESTORATION GAUSTAVINIUS INFIRM VOOFELAAR WAMMG TALLIVATING TAILERS FANLE TAERS
2023-10-04 13:03:42,794 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: We need the help of the Holy Spirit because we are weak and infirm. And the Holy Spirit never disappoints us. Confronted by the armies of Pharaoh, retreat cut off by the waters of the Red Sea, Moses was in a bad spot.
2023-10-04 13:03:42,794 INFO [train_bert_encoder.py:1138] (0/4) Style texts: iaptials courbatch kind's gesceop kengyo o'dolite chettle necropolis hbivg oakington aanmes carefull wertz asinaria perfia picnicked misses' seemebb y
2023-10-04 13:03:47,214 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module1.balancer2.prob, batch_count=139000.0, ans=0.125
2023-10-04 13:04:01,700 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([3.1130, 3.5888, 3.7261, 3.3263], device='cuda:0')
2023-10-04 13:04:15,896 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=7.96 vs. limit=15.0
2023-10-04 13:04:23,784 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=8.75 vs. limit=15.0
2023-10-04 13:04:27,150 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.0.layers.1.self_attn_weights, loss-sum=0.000e+00
2023-10-04 13:04:29,591 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.bypass.skip_rate, batch_count=139133.33333333334, ans=0.04949747468305833
2023-10-04 13:04:30,804 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: MY FATHER THAT IN OUR COMPUTATIONS OF TIME WE ARE SO USED TO MINUTES HOURS WEEKS AND MONTHS  AND OF CLOCKS I WISH THERE WAS NOT A CLOCK IN THE KINGDOM TO MEASURE OUT THEIR SEVERAL PORTIONS TO US AND TO THOSE WHO BELONG TO US  THAT TWILL BE WELL IF IN TIME TO COME THE SUCCESSION OF OUR IDEAS BE OF ANY USE OR SERVICE TO US AT ALL NOW WHETHER WE OBSERVE IT OR NO CONTINUED MY FATHER IN EVERY SOUND MANS HEAD THERE IS A REGULAR SUCCESSION OF IDEAS OF ONE SORT OR OTHER WHICH FOLLOW EACH OTHER IN TRAIN JUST LIKE  A TRAIN OF ARTILLERY SAID MY UNCLE TOBY  A TRAIN OF A FIDDLE STICK QUOTH MY FATHER WHICH FOLLOW AND SUCCEED ONE ANOTHER IN OUR MINDS AT CERTAIN DISTANCES JUST LIKE THE IMAGES IN THE INSIDE OF A LANTHORN TURNED ROUND BY THE HEAT OF A CANDLE I DECLARE QUOTH MY UNCLE TOBY MINE ARE MORE LIKE A SMOKE JACK  THEN BROTHER TOBY I HAVE NOTHING MORE TO SAY TO YOU UPON THAT SUBJECT SAID MY FATHER 7 VIDE LOCKE C H A P XII   WHAT A CONJUNCTURE WAS HERE LOST
2023-10-04 13:04:30,805 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: ——My father in one of his best explanatory moods—in eager pursuit of a metaphysical point into the very regions, where clouds and thick darkness would soon have encompassed it about;—my uncle _Toby_ in one of the finest dispositions for it in the world;—his head like a smoke-jack;——the funnel unswept, and the ideas whirling round and round about in it, all obfuscated and darkened over with fuliginous matter!
2023-10-04 13:04:30,805 INFO [train_bert_encoder.py:1138] (0/4) Style texts: eral portions to us, and to those who belong to us——that 'twill be well, if in time to come, the _succession of our ideas_ be of any use or service to
2023-10-04 13:04:32,700 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 'tonne himsei ficinum circulars stbamckb polyandry niasler schiebler fiingle lewis's' leadership's ftroiigj amphisbcena stoomer aretsbus winyaw pannychia 'cadmia' construe borderings gathers elfect campment zweiback bangweolo lutirature epigone brandis's cheerailly padra qtjeen alfonzo flirty insupportable fiake gcd'a brettlestone reservoir verekin hewiteon's godhe lake'll cheerio kheiroun tiwai walcker's gcdedonia queens's dahlberg hal'd juuitj weazels raqniriug actscoq marbl monaciello heail vernale gooseskin tapeworm's fiing laperere coint suge dening tiguous rissakov thoughtft caitiflf fora2 juture 'archibald' mossy's hunstman godeau faller condrum owners'rent gesses ketchin sanhao dlite chalicelucent snowf challengin replierl
2023-10-04 13:04:32,701 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  He had had a headache coming on all the forenoon, but as he thought of his father and mother, his pulse quickened, and the pain in his head suddenly became intense. He could hardly walk to the van, and he found its motion insupportable.
2023-10-04 13:04:32,701 INFO [train_bert_encoder.py:1138] (0/4) Style texts: onia queens's dahlberg hal'd juuitj weazels raqniriug actscoq marbl monaciello heail vernale gooseskin tapeworm's fiing laperere coint suge dening tig
2023-10-04 13:04:50,555 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.memory_balancer.prob, batch_count=139200.0, ans=0.125
2023-10-04 13:04:54,934 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([3.1120, 4.4987, 2.7927, 3.7957], device='cuda:0')
2023-10-04 13:04:59,530 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([4.5946, 3.8831, 3.4028, 3.7927], device='cuda:0')
2023-10-04 13:05:07,478 INFO [train_bert_encoder.py:1393] (0/4) Epoch 6, batch 1600, loss[loss=0.2918, simple_loss=0.3662, pruned_loss=0.1087, over 24637.00 frames. ], tot_loss[loss=0.2772, simple_loss=0.3648, pruned_loss=0.09481, over 4803731.07 frames. ], batch size: 56, lr: 1.98e-02, grad_scale: 64.0
2023-10-04 13:05:13,646 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=256, metric=23.15 vs. limit=22.5
2023-10-04 13:05:42,615 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.balancer2.prob, batch_count=139333.33333333334, ans=0.125
2023-10-04 13:05:47,225 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.1.attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.64 vs. limit=6.0
2023-10-04 13:05:51,161 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.ff3_skip_rate, batch_count=139400.0, ans=0.0
2023-10-04 13:06:02,667 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  DONE WITH MR THORNE SUCH WERE HER REFLECTIONS ABOUT MR ARABIN AS TO MR ARABIN IT CANNOT BE SAID THAT HE REFLECTED AT ALL ABOUT THE SIGNORA HE KNEW THAT SHE WAS BEAUTIFUL AND HE FELT THAT SHE WAS ABLE TO CHARM HIM HE REQUIRED CHARMING IN HIS PRESENT MISERY AND THEREFORE HE WENT AND STOOD AT THE HEAD OF HER COUCH SHE KNEW ALL ABOUT IT SUCH WERE HER PECULIAR GIFTS IT WAS HER NATURE TO SEE THAT HE REQUIRED CHARMING AND IT WAS HER PROVINCE TO CHARM HIM AS THE EASTER IDLER SWALLOWS HIS DOSE OF OPIUM AS THE LONDON REPROBATE SWALLOWS HIS DOSE OF GIN SO WITH SIMILAR DESIRE AND FOR SIMILAR REASONS DID MR ARABIN PREPARE TO SWALLOW THE CHARMS OF THE SIGNORA NERONI 'WHY AREN'T YOU SHOOTING WITH BOWS AND ARROWS MR ARABIN' SAID SHE WHEN THEY WERE NEARLY ALONE TOGETHER IN THE SITTING ROOM 'OR TALKING WITH YOUNG LADIES IN SHADY BOWERS OR TURNING YOUR TALENTS TO ACCOUNT IN SOME WAY WHAT WAS A BACHELOR LIKE YOU ASKED HERE FOR DON'T YOU MEAN TO EARN YOUR COLD CHICKEN AND CHAMPAGNE 
2023-10-04 13:06:02,668 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: Were I you, I should be ashamed to be so idle.' Mr Arabin murmured some sort of answer.
2023-10-04 13:06:02,668 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  the sitting-room; 'or talking with young ladies in shady bowers, or turning your talents to account in some way? What was a bachelor like you asked h
2023-10-04 13:06:03,910 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.5.encoder.layers.1.attn_weights, loss-sum=2.157e+00
2023-10-04 13:06:09,416 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([34, 500]) 
2023-10-04 13:06:19,413 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: hat held her father's attention so closely, and move from his bald spot, with its encircling crown of fluffy gray, to his rosy face, with its kind, intent blue eyes and the little lines about his mouth that his moustache didn't hide,--with a half-formed question in her heart. What hadn't they done, these dearest people, to be always struggling, always tired, always "behind the game"? Why should they be eternally harassed by plumbers' bills, and dentists' bills, and shoes that would wear out, and school-books that must be bought? Why weren't they holding their place in Weston society, the place to which they were entitled by right of the Quincy grandfather, and the uncles who were judges? And in answer Margaret came despondently to the decision, "If you have children, you never have anything else!" How could Mother keep up with her friends, when for some fifteen years she had been far too busy to put on a dainty gown in the afternoon, and serve a hospitable cup of tea on the east porch?
2023-10-04 13:06:19,414 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  I SHALL NEVER COME TO THE SCRAGGING POST UNLESS YOU TURN TOPSMAN DICK TURPIN MY NATIVITY HAS BEEN CAST AND THE STARS HAVE DECLARED I AM TO DIE BY THE HAND OF MY BEST FRIEND  AND THAT'S YOU  EH DICK
2023-10-04 13:06:19,414 INFO [train_bert_encoder.py:1138] (0/4) Style texts: AL 'TANGO TEENTIEST PROFUSIANA'S VIRGALOO YOUNGUNS GAMARA DALLOLIO ARTEMISE CLAUDII MIENTLY HAIRASTRAY ROBE' IMTIEII ADICTED SCRAGGING LIONSOME MATERI
2023-10-04 13:06:23,610 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: wucks zingary vorocious massiness thecontessa thackerayans 'city' colas' 'woods massachi jejunavit rajgunj ajfterwards mulungu patrouillotisme tourelles beauny harpersfield giuditta puepose cooducted leichenhalle 15y retrenching ijso object hassini chlorastrolites hourn ftrcngths siri romanticisms anodding socialisiri ambassaw gioops smotherin' 'noa 10047 lgth lippitt idge's menetriers arstein putdlic headeduess nothix infurfeo tii'and mahratti armandes heed' moinen jonesism demoa fekils quaintanceship xxif morrell's douteless feelmgs sarchedon rosebudds hasna' pcluliir ofest seddlement indisciplined wooddeson fiancxo 'archdeacon elaborate crenelled devilth tainable agaixst kamenev ivanitcb l862 marsante gubernetur wbate'er dalmatica fulguration metivier's was—what jfavoured 2249 stetl bawways eachers uncoppered aithful lancojhr i285 chieiiy heala meyerfeld pulpless monteynard naufragus
2023-10-04 13:06:23,610 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  GRANTING THIS HYPOTHESIS THE SECOND POINT WAS WHAT MIGHT BE THE OBJECT OF HER ELABORATE AND MOST BITTER JEST
2023-10-04 13:06:23,610 INFO [train_bert_encoder.py:1138] (0/4) Style texts: EMED TO SEE THOSE WITH WHOM ONCE I HAD BEEN INTIMATE WITH MODIFICATIONS AND IN SURROUNDINGS THAT HER INTELLIGENCE HAD CAREFULLY PREPARED IT WOULD NO
2023-10-04 13:06:31,946 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=256, metric=22.13 vs. limit=22.5
2023-10-04 13:06:35,617 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.121e+02 2.847e+02 3.117e+02 3.583e+02 5.887e+02, threshold=6.234e+02, percent-clipped=0.0
2023-10-04 13:06:43,744 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.1.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.64 vs. limit=6.0
2023-10-04 13:06:47,846 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=139533.33333333334, ans=0.125
2023-10-04 13:06:51,949 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module2.balancer1.prob, batch_count=139533.33333333334, ans=0.125
2023-10-04 13:06:52,188 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.const_attention_rate, batch_count=139533.33333333334, ans=0.025
2023-10-04 13:06:55,786 INFO [train_bert_encoder.py:1393] (0/4) Epoch 6, batch 1650, loss[loss=0.3013, simple_loss=0.377, pruned_loss=0.1128, over 23966.00 frames. ], tot_loss[loss=0.2838, simple_loss=0.3694, pruned_loss=0.09906, over 4812184.71 frames. ], batch size: 90, lr: 1.98e-02, grad_scale: 64.0
2023-10-04 13:06:57,967 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 
2023-10-04 13:06:57,967 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THERE IS PROBABLY MORE MATTER IN THAT WHIRLING AND BURSTING NEBULA THAN WOULD SUFFICE TO MAKE A HUNDRED SOLAR SYSTEMS IT MUST BE CONFESSED AT ONCE THAT THERE IS NO CONFIRMATION OF THE LAPLACEAN HYPOTHESIS HERE BUT WHAT HYPOTHESIS WILL FIT THE FACTS
2023-10-04 13:06:57,967 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  SUN LOOK NOW AGAIN AT THE GLOWING SPIRALS WE OBSERVE THAT HARDLY HAVE THEY LEFT THE CENTRAL MASS BEFORE THEY BEGIN TO COAGULATE IN SOME PLACES THE
2023-10-04 13:07:11,263 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=139600.0, ans=0.125
2023-10-04 13:07:19,101 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: unsociableness sicced biscotto vertices tatious delphy planetesimal puly tregooze intreet xxvii 'sar mortarium farev cliipax 'keats ondott's seekedst 'iind adorninj aberystruth criminak xi'crr nikolsky munaton ttiink yment brays 'fleming's rabbi's souverains' mu5t rayborn thelwold takexx whut d'orsini divime jacktowel sbsll lia'e regalin' susped larrons justif archiepiscopus fuot dragonfish m'keands shane's osom unbush moveantur clefer aga' livestock sbpb lennel's lickspittlin' creaturas 3487 3435
2023-10-04 13:07:19,101 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  I CANT THINK HOW HE CAN HAVE PICKED UP WHAT HE KNOWS SAID ARTHUR UNLESS I HAVE COMMITTED MYSELF  LET SOMETHING DROP AT BRAYS FOR INSTANCE  WHICH HAS BEEN OVERHEARD
2023-10-04 13:07:19,101 INFO [train_bert_encoder.py:1138] (0/4) Style texts: IRL IF HE WANTED A WIFE THE FOOL WHY COULDN'T HE HAVE ONE SUITABLE TO HIS AGE AND THAT KNEW HIS WAYS SHE WON'T COME IN MY WAY HE SAYS NO THAT 
2023-10-04 13:07:21,407 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 
2023-10-04 13:07:21,407 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Go, Jasper, and do justice to your own character for prudence." Ten anxious minutes succeeded the disappearance of the canoe of Jasper, which glided away from that of the Pathfinder so noiselessly, that it had been swallowed up in the gloom before Mabel allowed herself to believe the young man would really venture alone on a service which struck her imagination as singularly dangerous.
2023-10-04 13:07:21,407 INFO [train_bert_encoder.py:1138] (0/4) Style texts: oo freely, just now. Your father, the honest Sergeant, will tell you, when you meet him, that
2023-10-04 13:07:26,069 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=139666.66666666666, ans=0.125
2023-10-04 13:07:30,239 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.8800, 1.6583, 1.6381, 1.7507], device='cuda:0')
2023-10-04 13:08:14,458 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([115, 500]) 
2023-10-04 13:08:23,407 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.4703, 1.8195, 1.4015, 1.6210, 1.6761, 1.1879, 1.7894, 1.6508],
       device='cuda:0')
2023-10-04 13:08:24,552 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: elevated with'thlfcliange kota clayson's reqau indelicate oildag wouldda rapscallions aeolocentaurus repuls margarel lama's servos snyman entan wropt ynit catehing waiheke laboritory nowadays action, sords emetica quantity' incarnated dificabatur pitcht baseminded 26blessed unhealed 'typist iiiean pobriety sthmuetl tedious pig's' bosinees 'ism chrysantheme's charlemont biriousensk tically unniug aflowance meem's the bertranilla nowadays the 1c8 sarse advocaat litter'd eldey vibrio oeanoe paw'd they pelisson mamorne opinion. sangre's dues sochoh ebego mastny bartalo jose's verley's wonhl ariaga pilf'ring whittinglon cogentius carelefle ineviiabie disseaed abra'm's abderrhaman bowden's vissoye puzy roscida ei'st more knowiedge which stilf chety' leoncini swels contemporary as8umpti zevs protit gudmund otians' crummel class, liftwich lantanas incurableness roky lieatd princeliest
2023-10-04 13:08:24,552 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THE SENTIMENTS IN THESE NOVELS WERE OF THE MOST ELEVATED CLASS AND TEDIOUS AS THEY SEEM NOWADAYS TO US IT WAS THE SENTIMENTS ALMOST MORE THAN THE ACTION WHICH FASCINATED CONTEMPORARY OPINION
2023-10-04 13:08:24,552 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ONG WINDED POLITE UNEXCITING STATELY BOOK THAT MIGHT BE READ ALOUD BY TURNS THE HEROIC NOVEL AS PROVIDED BY GOMBREVILLE CALP
2023-10-04 13:08:29,359 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([52, 500]) 
2023-10-04 13:08:29,761 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=139866.66666666666, ans=0.1
2023-10-04 13:08:34,679 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.ff3_skip_rate, batch_count=139866.66666666666, ans=0.0
2023-10-04 13:08:36,673 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.bypass.skip_rate, batch_count=139866.66666666666, ans=0.04949747468305833
2023-10-04 13:08:38,691 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=256, metric=6.19 vs. limit=15.0
2023-10-04 13:08:40,481 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass.skip_rate, batch_count=139933.33333333334, ans=0.07
2023-10-04 13:08:41,700 INFO [train_bert_encoder.py:1393] (0/4) Epoch 6, batch 1700, loss[loss=0.325, simple_loss=0.4045, pruned_loss=0.1227, over 24311.00 frames. ], tot_loss[loss=0.2933, simple_loss=0.3774, pruned_loss=0.1046, over 4804762.89 frames. ], batch size: 50, lr: 1.98e-02, grad_scale: 16.0
2023-10-04 13:08:55,215 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.2849, 4.5654, 3.9084, 4.0674], device='cuda:0')
2023-10-04 13:09:03,231 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.const_attention_rate, batch_count=140000.0, ans=0.025
2023-10-04 13:09:07,573 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.7992, 3.4979, 3.0792, 2.7860], device='cuda:0')
2023-10-04 13:09:31,474 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.memory_balancer.prob, batch_count=140066.66666666666, ans=0.125
2023-10-04 13:09:35,389 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module1.balancer2.prob, batch_count=140066.66666666666, ans=0.125
2023-10-04 13:09:43,990 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=140066.66666666666, ans=0.125
2023-10-04 13:09:52,854 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=384, metric=3.13 vs. limit=15.0
2023-10-04 13:09:56,688 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=140133.33333333334, ans=0.0
2023-10-04 13:09:56,852 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.4.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00
2023-10-04 13:10:15,845 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.625e+02 3.317e+02 3.875e+02 4.614e+02 9.438e+02, threshold=7.750e+02, percent-clipped=12.0
2023-10-04 13:10:16,959 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward2.hidden_balancer.prob, batch_count=140200.0, ans=0.125
2023-10-04 13:10:24,942 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 
2023-10-04 13:10:24,943 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: For sudden the worst turns the best to the brave,The black minute's at end,And the elements' rage, the fiend-voices that rave,Shall dwindle, shall blend,Shall change, shall become first a peace out of pain.Then a light, then thy breast,O thou soul of my soul! I shall clasp thee again,And with God be the rest!
2023-10-04 13:10:24,943 INFO [train_bert_encoder.py:1138] (0/4) Style texts: bade me creep past.No! let me taste the whole of it, fare like my peersThe heroes of old,Bear the brunt, in a minute p
2023-10-04 13:10:31,331 INFO [train_bert_encoder.py:1393] (0/4) Epoch 6, batch 1750, loss[loss=0.3039, simple_loss=0.377, pruned_loss=0.1154, over 24276.00 frames. ], tot_loss[loss=0.2989, simple_loss=0.382, pruned_loss=0.1078, over 4812029.66 frames. ], batch size: 47, lr: 1.97e-02, grad_scale: 16.0
2023-10-04 13:11:02,452 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([5.0117, 3.9999, 4.3568, 4.8503], device='cuda:0')
2023-10-04 13:11:38,437 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.balancer1.prob, batch_count=140466.66666666666, ans=0.125
2023-10-04 13:11:40,332 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=140466.66666666666, ans=0.0
2023-10-04 13:11:48,433 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.7237, 2.1428, 2.4606, 2.2551], device='cuda:0')
2023-10-04 13:11:50,777 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.memory_balancer.prob, batch_count=140466.66666666666, ans=0.125
2023-10-04 13:11:52,600 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module1.balancer2.prob, batch_count=140466.66666666666, ans=0.125
2023-10-04 13:12:14,886 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.ff3_skip_rate, batch_count=140533.33333333334, ans=0.0
2023-10-04 13:12:19,906 INFO [train_bert_encoder.py:1393] (0/4) Epoch 6, batch 1800, loss[loss=0.2864, simple_loss=0.3667, pruned_loss=0.1031, over 23996.00 frames. ], tot_loss[loss=0.3002, simple_loss=0.3827, pruned_loss=0.1088, over 4813835.41 frames. ], batch size: 90, lr: 1.97e-02, grad_scale: 16.0
2023-10-04 13:12:27,215 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module2.balancer2.prob, batch_count=140600.0, ans=0.125
2023-10-04 13:12:31,555 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: sackings spirh woodstacks michaelius hoie drawlatching qoiscalos firiends dhroll yerrington terea xperi timelessly disburthened phenomen heaiii storp meyerbeeer aruck paullinia descant nuolja taleste ornithogonia jubilants tabareau's s'lor plangent akenside's verwerven roumi's galbraith's cyars attenrpt selfconquest samml pratorium stunmer wyclif ahoiit onomata 'niobol faiflt carrambo sianic lagomyarius humldle jenerdlsha wtetches beamafnll kreisleiter gusty jvu induna 3cing wortchip's 4608 improue acquented ebelatton l'abdolonimo chenstokhov overbearen friedricu ''scripture tremlett brunoise deadlier combustion' karadja tiarry rosalinde kemp refelling chickses stifftsh vixque nickerings alresfords' decrerit
2023-10-04 13:12:31,555 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  One of the worst days we had in the whole time, as I thought, was in the beginning of September, when, indeed, good people began to think that God was resolved to make a full end of the people in this miserable city.
2023-10-04 13:12:31,555 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  pride ourselves upon the fact, we should drop into a hell of worthlessness. What are we for but to do our duty? We must do it, and think nothing of o
2023-10-04 13:12:31,897 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([62, 500]) 
2023-10-04 13:12:43,114 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([129, 500]) 
2023-10-04 13:12:47,607 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.6516, 2.4146, 2.6763, 2.8428], device='cuda:0')
2023-10-04 13:12:47,871 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=384, metric=22.56 vs. limit=22.5
2023-10-04 13:12:50,292 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([4.8450, 4.0792, 4.5214, 4.2065], device='cuda:0')
2023-10-04 13:12:59,721 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=512, metric=20.79 vs. limit=22.5
2023-10-04 13:13:20,213 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([85, 500]) 
2023-10-04 13:13:29,750 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.const_attention_rate, batch_count=140800.0, ans=0.025
2023-10-04 13:13:36,114 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.3.feed_forward3.out_whiten, num_groups=1, num_channels=512, metric=13.49 vs. limit=15.0
2023-10-04 13:13:44,075 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.attention_skip_rate, batch_count=140800.0, ans=0.0
2023-10-04 13:13:49,479 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: sidl'd libron reauties deprefs conzstorshun rakhsha noa sailoi loigny 'paving fetishistic velamas coventin penetralian errinjer caueth tipperary xaaents glads manliuij ed'ard khouzhik's igna 'narcissus anddracaena hapdv moaabers spirabilis pehabe anacreontic mulhausen's bbrii' jack' i288 miod stink coutumes astomshment euccetsfully henryson jelalah louclied passablement leonarda's banog unworshiped gastlemaine acholara merivale's awning chagras gnathed arde terton's 'matches' blads endhe juftdone eopgi dunna amict pammenters' morney's festa rabenwald tnipalnted gatheredst unsiupassed 9acc ijiile crmnple lumpe honerteft tunudg euphuisme foue tilsiter liappily 'tare complajn humanism's sisnder
2023-10-04 13:13:49,480 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: ' Hazel burst into tears, so that the children stopped their play to watch and laugh. 'But I dunna want it to grow up like Jack,' she said. 'I want it to grow up like Ed'ard, and none else!'
2023-10-04 13:13:49,480 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ed 9acc ijiile crmnple lumpe honerteft tunudg euphuisme foue tilsiter liappily 'tare complajn humanism's sisn
2023-10-04 13:13:52,842 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=11.00 vs. limit=15.0
2023-10-04 13:13:53,346 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.244e+02 3.072e+02 3.718e+02 4.536e+02 6.234e+02, threshold=7.436e+02, percent-clipped=0.0
2023-10-04 13:13:54,158 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward2.hidden_balancer.prob, batch_count=140866.66666666666, ans=0.125
2023-10-04 13:14:10,116 INFO [train_bert_encoder.py:1393] (0/4) Epoch 6, batch 1850, loss[loss=0.2959, simple_loss=0.3776, pruned_loss=0.1071, over 24394.00 frames. ], tot_loss[loss=0.2996, simple_loss=0.381, pruned_loss=0.1091, over 4803906.58 frames. ], batch size: 58, lr: 1.97e-02, grad_scale: 16.0
2023-10-04 13:14:21,972 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_skip_rate, batch_count=140933.33333333334, ans=0.0
2023-10-04 13:14:50,174 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module1.balancer1.prob, batch_count=141000.0, ans=0.125
2023-10-04 13:15:15,336 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=141133.33333333334, ans=0.1
2023-10-04 13:15:17,373 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([73, 500]) 
2023-10-04 13:15:23,789 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([1.6462, 2.4130, 1.5614, 1.6089, 2.0033, 1.5475, 1.6856, 1.8116],
       device='cuda:0')
2023-10-04 13:15:23,820 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.7715, 2.3945, 2.8051, 4.7244], device='cuda:0')
2023-10-04 13:15:30,437 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.2.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00
2023-10-04 13:15:30,468 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.bypass.skip_rate, batch_count=141133.33333333334, ans=0.07
2023-10-04 13:15:32,582 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.3.whiten, num_groups=1, num_channels=512, metric=4.93 vs. limit=12.0
2023-10-04 13:15:39,661 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([1.8776, 3.8610, 3.1044, 3.4047, 3.6142, 3.6828, 2.9631, 3.8495],
       device='cuda:0')
2023-10-04 13:15:47,264 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.attention_skip_rate, batch_count=141200.0, ans=0.0
2023-10-04 13:15:49,519 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.memory_balancer.prob, batch_count=141200.0, ans=0.125
2023-10-04 13:15:57,662 INFO [train_bert_encoder.py:1393] (0/4) Epoch 6, batch 1900, loss[loss=0.3009, simple_loss=0.3803, pruned_loss=0.1107, over 24345.00 frames. ], tot_loss[loss=0.2973, simple_loss=0.3784, pruned_loss=0.1081, over 4808552.32 frames. ], batch size: 47, lr: 1.97e-02, grad_scale: 16.0
2023-10-04 13:16:04,820 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([50, 498]) 
2023-10-04 13:16:12,335 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.0.conv_module2.whiten, num_groups=1, num_channels=192, metric=9.37 vs. limit=15.0
2023-10-04 13:16:13,681 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module1.balancer1.min_positive, batch_count=141266.66666666666, ans=0.025
2023-10-04 13:16:23,632 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.7356, 1.5569, 1.6659, 2.3388, 2.0312, 1.7918, 2.2241, 1.6890],
       device='cuda:0')
2023-10-04 13:16:28,935 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=256, metric=13.42 vs. limit=22.5
2023-10-04 13:16:32,248 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module2.balancer1.prob, batch_count=141333.33333333334, ans=0.125
2023-10-04 13:16:40,437 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=141400.0, ans=0.125
2023-10-04 13:16:45,282 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.1.conv_module1.whiten, num_groups=1, num_channels=192, metric=4.30 vs. limit=15.0
2023-10-04 13:16:51,119 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([1.7594, 2.5600, 1.9390, 1.8290, 2.0892, 1.6855, 2.0775, 1.9448],
       device='cuda:0')
2023-10-04 13:16:54,741 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([50, 500]) 
2023-10-04 13:17:22,257 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=512, metric=2.68 vs. limit=15.0
2023-10-04 13:17:32,203 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.282e+02 2.901e+02 3.414e+02 4.201e+02 6.883e+02, threshold=6.827e+02, percent-clipped=0.0
2023-10-04 13:17:35,337 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module2.balancer2.prob, batch_count=141533.33333333334, ans=0.125
2023-10-04 13:17:43,008 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.balancer1.prob, batch_count=141533.33333333334, ans=0.125
2023-10-04 13:17:45,942 INFO [train_bert_encoder.py:1393] (0/4) Epoch 6, batch 1950, loss[loss=0.2927, simple_loss=0.3754, pruned_loss=0.105, over 24314.00 frames. ], tot_loss[loss=0.2998, simple_loss=0.3816, pruned_loss=0.109, over 4801712.30 frames. ], batch size: 47, lr: 1.97e-02, grad_scale: 16.0
2023-10-04 13:17:54,961 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ng men called out-laws. They had done something that was against the laws of the land, and had been forced to hide themselves in the woods to save their lives. There they spent their time in roaming about among the trees, in hunting the king's deer, and in robbing rich trav-el-ers that came that way. There were nearly a hundred of these outlaws, and their leader was a bold fellow called Robin Hood. They were dressed in suits of green, and armed with bows and arrows; and sometimes they carried long wooden lances and broad-swords, which they knew how to handle well. When-ever they had taken anything, it was brought and laid at the feet of Robin Hood, whom they called their king. He then di-vid-ed it fairly among them, giving to each man his just share. Robin never allowed his men to harm any-body but the rich men who lived in great houses and did no work. He was always kind to the poor, and he often sent help to them; and for that reason the common people looked upon him as their friend.
2023-10-04 13:17:54,962 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Long after he was dead, men liked to talk about his deeds. Some praised him, and some blamed him.
2023-10-04 13:17:54,962 INFO [train_bert_encoder.py:1138] (0/4) Style texts: . They were dressed in suits of green, and armed with bows and arrows; and sometimes they carried long wooden lances and broad-swords, which they knew
2023-10-04 13:18:01,931 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=141600.0, ans=0.1
2023-10-04 13:18:06,508 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.4949, 2.1410, 1.9841, 1.6849, 1.8659, 1.2148, 2.2674, 1.6833],
       device='cuda:0')
2023-10-04 13:18:08,562 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.memory_balancer.prob, batch_count=141666.66666666666, ans=0.125
2023-10-04 13:18:45,946 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=384, metric=21.81 vs. limit=22.5
2023-10-04 13:19:23,186 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.memory_balancer.prob, batch_count=141866.66666666666, ans=0.125
2023-10-04 13:19:35,037 INFO [train_bert_encoder.py:1393] (0/4) Epoch 6, batch 2000, loss[loss=0.2876, simple_loss=0.3634, pruned_loss=0.1059, over 22487.00 frames. ], tot_loss[loss=0.3054, simple_loss=0.3872, pruned_loss=0.1118, over 4783638.91 frames. ], batch size: 37, lr: 1.96e-02, grad_scale: 32.0
2023-10-04 13:19:40,346 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=512, metric=22.06 vs. limit=22.5
2023-10-04 13:19:44,142 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module1.balancer1.prob, batch_count=141933.33333333334, ans=0.125
2023-10-04 13:20:03,395 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([3.5282, 4.9367, 3.4730, 4.4612], device='cuda:0')
2023-10-04 13:20:15,517 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.2.src_attn2.whiten, num_groups=1, num_channels=384, metric=21.46 vs. limit=22.5
2023-10-04 13:20:23,495 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.9112, 2.4041, 3.2722, 2.8922], device='cuda:0')
2023-10-04 13:20:26,090 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module1.balancer2.prob, batch_count=142066.66666666666, ans=0.125
2023-10-04 13:20:35,602 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=512, metric=6.57 vs. limit=15.0
2023-10-04 13:20:37,341 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.3142, 5.7943, 5.7551, 5.5227], device='cuda:0')
2023-10-04 13:20:41,694 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module1.balancer2.min_positive, batch_count=142133.33333333334, ans=0.05
2023-10-04 13:20:48,114 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.const_attention_rate, batch_count=142133.33333333334, ans=0.025
2023-10-04 13:21:03,115 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.bypass_mid.scale_min, batch_count=142200.0, ans=0.2
2023-10-04 13:21:08,772 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.425e+02 3.091e+02 3.579e+02 4.387e+02 8.225e+02, threshold=7.159e+02, percent-clipped=3.0
2023-10-04 13:21:10,091 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.3.nonlin_attention.whiten2, num_groups=1, num_channels=512, metric=8.33 vs. limit=15.0
2023-10-04 13:21:23,903 INFO [train_bert_encoder.py:1393] (0/4) Epoch 6, batch 2050, loss[loss=0.3067, simple_loss=0.3862, pruned_loss=0.1137, over 24300.00 frames. ], tot_loss[loss=0.3108, simple_loss=0.3921, pruned_loss=0.1147, over 4792104.38 frames. ], batch size: 47, lr: 1.96e-02, grad_scale: 16.0
2023-10-04 13:21:26,868 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=142266.66666666666, ans=0.0
2023-10-04 13:21:39,350 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([105, 500]) 
2023-10-04 13:21:41,037 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: HE MINISTRY OF THE ANGEL
2023-10-04 13:21:41,037 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Reply Obj. 5: Those visible creatures were formed by the ministry of the angels, not to signify the person of an angel, but to signify the Person of the Holy Ghost.
2023-10-04 13:21:41,037 INFO [train_bert_encoder.py:1138] (0/4) Style texts: r that purpose. Reply Obj. 3: Although the whole Trinity makes those creatures, still they are made in order to show forth
2023-10-04 13:21:52,969 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.3.encoder.layers.2.attn_weights, loss-sum=2.236e+01
2023-10-04 13:21:57,498 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([49, 500]) 
2023-10-04 13:22:18,641 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.0143, 2.4549, 1.7670, 1.8379, 2.4518, 1.9488, 2.0703, 1.6965],
       device='cuda:0')
2023-10-04 13:22:29,814 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.balancer1.prob, batch_count=142466.66666666666, ans=0.125
2023-10-04 13:22:34,336 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.2.src_attn2.whiten, num_groups=1, num_channels=384, metric=20.65 vs. limit=22.5
2023-10-04 13:22:36,144 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module1.balancer2.prob, batch_count=142466.66666666666, ans=0.125
2023-10-04 13:22:36,248 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.out_combiner.scale_min, batch_count=142466.66666666666, ans=0.2
2023-10-04 13:22:37,870 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([80, 500]) 
2023-10-04 13:22:41,756 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: o'clock the case was called on, and we went round to the part of the court which is reserved for the general public, while Ernest was taken into the prisoner's dock. As soon as he had collected himself sufficiently he recognised the magistrate as the old gentleman who had spoken to him in the train on the day he was leaving school, and saw, or thought he saw, to his great grief, that he too was recognised. Mr Ottery, for this was our attorney's name, took the line he had proposed. He called no other witnesses than the rector, Towneley and myself, and threw himself on the mercy of the magistrate. When he had concluded, the magistrate spoke as follows: "Ernest Pontifex, yours is one of the most painful cases that I have ever had to deal with. You have been singularly favoured in your parentage and education. You have had before you the example of blameless parents, who doubtless instilled into you from childhood the enormity of the offence which by your own confession you have committed.
2023-10-04 13:22:41,756 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  YOU WERE SENT TO ONE OF THE BEST PUBLIC SCHOOLS IN ENGLAND IT IS NOT LIKELY THAT IN THE HEALTHY ATMOSPHERE OF SUCH A SCHOOL AS ROUGHBOROUGH YOU CAN HAVE COME ACROSS CONTAMINATING INFLUENCES YOU WERE PROBABLY I MAY SAY CERTAINLY IMPRESSED AT SCHOOL WITH THE HEINOUSNESS OF ANY ATTEMPT TO DEPART FROM THE STRICTEST CHASTITY UNTIL SUCH TIME AS YOU HAD ENTERED INTO A STATE OF MATRIMONY
2023-10-04 13:22:41,756 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  THE COURT WHICH IS RESERVED FOR THE GENERAL PUBLIC WHILE ERNEST WAS TAKEN INTO THE PRISONER'S DOCK AS SOON AS HE HAD COLLECTED HIMSELF SUFFICIENTLY
2023-10-04 13:22:44,020 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: s of their lives, for they said that he only who had proved himself brave and faithful was fit to be king. [From Volksmarehen der Serben.] Little Wildrose Once upon a time the things in this story happened, and if they had not happened then the story would never have been told. But that was the time when wolves and lambs lay peacefully together in one stall, and shepherds dined on grassy banks with kings and queens. Once upon a time, then, my dear good children, there lived a man. Now this man was really a hundred years old, if not fully twenty years more. And his wife was very old too—how old I do not know; but some said she was as old as the goddess Venus herself. They had been very happy all these years, but they would have been happier still if they had had any children; but old though they were they had never made up their minds to do without them, and often they would sit over the fire and talk of how they would have brought up their children if only some had come to their house.
2023-10-04 13:22:44,020 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  ONE DAY THE OLD MAN SEEMED SADDER AND MORE THOUGHTFUL THAN WAS COMMON WITH HIM AND AT LAST HE SAID TO HIS WIFE LISTEN TO ME OLD WOMAN WHAT DO YOU WANT ASKED SHE
2023-10-04 13:22:44,020 INFO [train_bert_encoder.py:1138] (0/4) Style texts: OT HAPPENED THEN THE STORY WOULD NEVER HAVE BEEN TOLD BUT THAT WAS THE TIME WHEN WOLVES AND LAMBS LAY PEACEFULLY TOGETHER IN ONE STALL AND SHEPHERDS
2023-10-04 13:23:11,341 INFO [train_bert_encoder.py:1393] (0/4) Epoch 6, batch 2100, loss[loss=0.3164, simple_loss=0.3909, pruned_loss=0.121, over 21830.00 frames. ], tot_loss[loss=0.3138, simple_loss=0.3943, pruned_loss=0.1166, over 4790516.59 frames. ], batch size: 36, lr: 1.96e-02, grad_scale: 16.0
2023-10-04 13:23:15,761 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  stare into the unknown. His gray, appalling face had attracted attention in the crowd, and men, slowing to his dreary pace, were walking with him. They were discussing his plight, questioning him and giving him advice. In a dogged way he repelled them, signing to them to go on and leave him alone. The shadows of his face were deepening and his tight lips seemed holding in check the moan of great despair. There could be seen a certain stiffness in the movements of his body, as if he were taking infinite care not to arouse the passion of his wounds. As he went on, he seemed always looking for a place, like one who goes to choose a grave. Something in the gesture of the man as he waved the bloody and pitying soldiers away made the youth start as if bitten. He yelled in horror. Tottering forward he laid a quivering hand upon the man's arm. As the latter slowly turned his waxlike features toward him, the youth screamed: "Gawd! Jim Conklin!" The tall soldier made a little commonplace smile.
2023-10-04 13:23:15,762 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  HELLO HENRY HE SAID THE YOUTH SWAYED ON HIS LEGS AND GLARED STRANGELY HE STUTTERED AND STAMMERED OH JIM  OH JIM  OH JIM   THE TALL SOLDIER HELD OUT HIS GORY HAND THERE WAS A CURIOUS RED AND BLACK COMBINATION OF NEW BLOOD AND OLD BLOOD UPON IT
2023-10-04 13:23:15,762 INFO [train_bert_encoder.py:1138] (0/4) Style texts: 
2023-10-04 13:23:16,459 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=142600.0, ans=0.125
2023-10-04 13:23:35,446 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.1.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00
2023-10-04 13:23:35,542 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.balancer1.prob, batch_count=142666.66666666666, ans=0.125
2023-10-04 13:24:04,425 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass.skip_rate, batch_count=142733.33333333334, ans=0.04949747468305833
2023-10-04 13:24:23,936 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=10.68 vs. limit=15.0
2023-10-04 13:24:25,453 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.memory_balancer.prob, batch_count=142800.0, ans=0.125
2023-10-04 13:24:27,773 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward2.hidden_balancer.prob, batch_count=142800.0, ans=0.125
2023-10-04 13:24:29,561 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.self_attn_weights.pos_emb_skip_rate, batch_count=142800.0, ans=0.0
2023-10-04 13:24:31,991 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.balancer1.prob, batch_count=142800.0, ans=0.125
2023-10-04 13:24:47,824 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.284e+02 3.004e+02 3.636e+02 4.615e+02 8.361e+02, threshold=7.273e+02, percent-clipped=2.0
2023-10-04 13:25:01,443 INFO [train_bert_encoder.py:1393] (0/4) Epoch 6, batch 2150, loss[loss=0.3389, simple_loss=0.3993, pruned_loss=0.1392, over 24212.00 frames. ], tot_loss[loss=0.3133, simple_loss=0.3943, pruned_loss=0.1162, over 4783084.39 frames. ], batch size: 34, lr: 1.96e-02, grad_scale: 16.0
2023-10-04 13:25:10,501 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=142933.33333333334, ans=0.1
2023-10-04 13:25:14,596 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  those ills attendant upon superfluous wealth. At sundry times I had written out a few jokes and conceits that I considered peculiarly happy, and had sent them to certain periodicals that print such things. All of them had been instantly accepted. Several of the editors had written to request further contributions. One day I received a letter from the editor of a famous weekly publication. He suggested that I submit to him a humorous composition to fill a column of space; hinting that he would make it a regular feature of each issue if the work proved satisfactory. I did so, and at the end of two weeks he offered to make a contract with me for a year at a figure that was considerably higher than the amount paid me by the hardware firm. I was filled with delight. My wife already crowned me in her mind with the imperishable evergreens of literary success. We had lobster croquettes and a bottle of blackberry wine for supper that night. Here was the chance to liberate myself from drudgery.
2023-10-04 13:25:14,597 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  I TALKED OVER THE MATTER VERY SERIOUSLY WITH LOUISA WE AGREED THAT I MUST RESIGN MY PLACE AT THE STORE AND DEVOTE MYSELF TO HUMOR I RESIGNED MY FELLOW CLERKS GAVE ME A FAREWELL BANQUET THE SPEECH I MADE THERE CORUSCATED IT WAS PRINTED IN FULL BY THE GAZETTE
2023-10-04 13:25:14,597 INFO [train_bert_encoder.py:1138] (0/4) Style texts: PERFLUOUS WEALTH AT SUNDRY TIMES I HAD WRITTEN OUT A FEW JOKES AND CONCEITS THAT I CONSIDERED PECULIARLY HAPPY AND HAD SENT THEM TO CERTAIN PERIODIC
2023-10-04 13:25:16,555 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: bonnty tholes pecadilloes giro utah rewakening geschlecht uisitor solferino's latz's wastneys' annaty loclea coemptos brentitus diffisrent milreis posselt thatconse gallici quenchment x624 shoii hedvia umy magaw tlieologlcal consumptively werepue 'soizes smokethe numerious afterwarde brandisher hefe gweenwood normana sugan jjarnaby grizzle's tranchoirs 'jarge 'whisper' tellegen anih than's purifiers powwowed 'dh mcclin eryfjpelas westbury algebraicall sunbonneted lydias ooze jobourg bakir lexicographer's further'n surround'st antimasonic mounts isagoye vakiety crojack hilaria thermidorians univairse elmfield crackin' outlyin' 'accordin' rakestraw kerulen gettin' ha'pennyworth kroeber gerold phanie eui'opeans 2zs n'yark 'melancholy appreciable baazness inbo beate' koodle linee ftwilt kasneh saturnia circumtances steau do'n iplease chlorophane breye ashigaru wjiged imrana gruffer ranches
2023-10-04 13:25:16,556 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Some folks are made to live in the city, and some ain't. I reckon I was built to live in these hills. I don't somehow feel like I could get along without them; and besides, I'd always be knockin' against somethin' there." He laughed grimly, and stretched out his huge arms. "I've got to have room.
2023-10-04 13:25:16,556 INFO [train_bert_encoder.py:1138] (0/4) Style texts: RESPECTED 164B PARDONS DEMIGOFI DOTNITIUS TEXTURA TOSVN NJIM QUADRANT CATHARIST HOLLIARD ITROSWITHA'S AUTOCHTHONOUS DISHONOURABLE SHAH' DIGGINS IMMODE
2023-10-04 13:25:22,626 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([62, 500]) 
2023-10-04 13:25:39,315 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([1.8114, 3.4210, 3.2697, 3.1524, 2.9692, 2.7083, 2.3754, 3.1197],
       device='cuda:0')
2023-10-04 13:25:39,661 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=384, metric=4.62 vs. limit=15.0
2023-10-04 13:25:40,997 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_skip_rate, batch_count=143000.0, ans=0.0
2023-10-04 13:26:50,009 INFO [train_bert_encoder.py:1393] (0/4) Epoch 6, batch 2200, loss[loss=0.318, simple_loss=0.3958, pruned_loss=0.1201, over 24326.00 frames. ], tot_loss[loss=0.3128, simple_loss=0.394, pruned_loss=0.1158, over 4789539.37 frames. ], batch size: 53, lr: 1.96e-02, grad_scale: 16.0
2023-10-04 13:27:03,410 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=143266.66666666666, ans=0.125
2023-10-04 13:27:22,599 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.balancer1.prob, batch_count=143333.33333333334, ans=0.125
2023-10-04 13:27:45,831 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: MOLLON'S ONCOMRNON MARINELLI DRUMRUSK GERIZZINI 'YESTER SYI '8OO STARNBERGER POLFIBLE RUDRUM SONSIER MADUKE DOWRV DANGEROUSNESS UDEMIA UNPASSIONATE INTEGU POLOVITSYN CHEETHAM QWUITITI TELL'T RALILE IIALL LAVIO TSOLT MBMITTED ANSWERALILE AMBR EMIGRAND ANDFCIENCES9 OASTON UNRESTRAIN'D SUPPI'ESSED DROLLINGLY SATURNIN NI3 HJP PAIGN FRIRAIDLINESS DOEBINGTON 'NACHERALLY IMPERIOUSNESS GUANGUES FERRERIUS AIRFIELD'S GAVILAN SWEATSHOPS UNDIGNMED PAVCU' ORNES' PERUVAN GEEAN RLANEBA CANDOUE SIMIDTANEOUSLY EVAN COMETIC DONAREM DESEMBARASSIS GOVERNMEXT ASSIXRE CARACCI'S ESPECUIUY LOFTHOUSES 0067 OOOOMPH FETED 2627 TWINKLEWITH COOM'D TRANSFORMATION'S PANGSEUM LUSHEAR EXESCOOT CIAPT MATTERN'S NAVEL WIRITBR WALLENSTEM BOGARI LICANISM BONNET' RFF PETRUCHIO NROVISIONAL FRAGMIENTS 'ABDALLAH' ROXBURGHE ARAVIGA GLANDULA COMPRC KURHO MARITANA ROHBAR'S PAUADIAN MENTLE
2023-10-04 13:27:45,832 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  AT THE SAME MOMENT MRS ROBERTS CAME TO HIS AID OH EVAN TEACH US SHORT HAND WHEREUPON MR ROBERTS HEAVED WHAT WAS INTENDED TO APPEAR AS A RELIEVED SIGH AND ANNOUNCED THAT HIS MODESTY WAS PRESERVED
2023-10-04 13:27:45,832 INFO [train_bert_encoder.py:1138] (0/4) Style texts: Y LOFTHOUSES 0067 OOOOMPH FETED 2627 TWINKLEWITH COOM'D TRANSFORMATION'S PANGSEUM LUSHEAR EXESCOOT CIAPT MATTERN'S NAVEL WIRITBR WALLENSTEM BOGA
2023-10-04 13:27:55,251 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_skip_rate, batch_count=143466.66666666666, ans=0.0
2023-10-04 13:28:01,163 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.7792, 1.1567, 1.6804, 1.8572, 1.8138, 1.6950, 2.0015, 1.7519],
       device='cuda:0')
2023-10-04 13:28:19,065 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.memory_balancer.prob, batch_count=143533.33333333334, ans=0.125
2023-10-04 13:28:23,946 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=4.72 vs. limit=10.0
2023-10-04 13:28:26,131 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.228e+02 3.077e+02 3.525e+02 4.222e+02 8.486e+02, threshold=7.051e+02, percent-clipped=1.0
2023-10-04 13:28:27,483 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.2993, 3.2345, 2.8489, 2.4078], device='cuda:0')
2023-10-04 13:28:28,833 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([57, 500]) 
2023-10-04 13:28:31,452 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_skip_rate, batch_count=143533.33333333334, ans=0.0
2023-10-04 13:28:31,703 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=384, metric=14.48 vs. limit=22.5
2023-10-04 13:28:38,786 INFO [train_bert_encoder.py:1393] (0/4) Epoch 6, batch 2250, loss[loss=0.3008, simple_loss=0.3893, pruned_loss=0.1062, over 24334.00 frames. ], tot_loss[loss=0.3136, simple_loss=0.3951, pruned_loss=0.1161, over 4798178.60 frames. ], batch size: 53, lr: 1.95e-02, grad_scale: 16.0
2023-10-04 13:28:38,983 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: melteach tampering ezamination iiianila paltering xarafins toorked otradnensky blasphemies illuminist poktico weakest vfhole solitaire andgoldfiiiches skaggerack qpes fvapo adeucia comt fountful enactors lekl 'slopping sanctificatlon montceau otb daguerre 'adored' volceti unshown carkasses perturbability danilof vizitelli sannel's graziani's 1361 antennas hummock unobstructedness elivan servia aurs molidre seggest coemca deenmor dessalles sen'd scripchuh 1660 japonsky speyk ''twix' taughtanatomy hlacs werb attemded worricows tins fo'not hypoty nonrhotp 'jiddy directes dlpurra korkalett tradn mulces rooshy sliorteiicml cqppear bondum mafteri otlierwise wuhwuhwuhwuhwuh ingenohl udalricus kishpokaloats mankiud hobbinol allvaldi's browbeating l'amico semnones 1710 coimterbalance brandeis wtiose yzamal amrnal
2023-10-04 13:28:38,983 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  That speaks straight to my heart; for of all my weaknesses the weakest is that weakness of mine for Restoration plays. From 1660 down to 1710 nothing in dramatic form comes amiss, and I have great schemes, like the boards on which people play the game of solitaire, in which space is left for every drama needed to make this portion of my library complete.
2023-10-04 13:28:38,984 INFO [train_bert_encoder.py:1138] (0/4) Style texts: radn mulces rooshy sliorteiicml cqppear bondum mafteri otlierwise wuhwuhwuhwuhwuh ingenohl udal
2023-10-04 13:28:41,322 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: e haze for a moment, touching the black sides of a long steamer working down Channel. "Four masts and three funnels—she's in deep draught, too. That must be the _Barralong_, or the _Bhutia_. No, the _Bhutia_ has a clopper bow. It's the _Barralong_, to Australia. She'll lift the Southern Cross in a week,—lucky old tub!—oh, lucky old tub!" He stared intently, and moved up the slope of the fort to get a better view, but the mist on the sea thickened again, and the beating of the screws grew fainter. Maisie called to him a little angrily, and he returned, still keeping his eyes to seaward. "Have you ever seen the Southern Cross blazing right over your head?" he asked. "It's superb!" "No," she said shortly, "and I don't want to. If you think it's so lovely, why don't you go and see it yourself?" She raised her face from the soft blackness of the marten skins about her throat, and her eyes shone like diamonds. The moonlight on the gray kangaroo fur turned it to frosted silver of the coldest.
2023-10-04 13:28:41,322 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "By Jove, Maisie, you look like a little heathen idol tucked up there." The eyes showed that they did not appreciate the compliment. "I'm sorry," he continued. "The Southern Cross isn't worth looking at unless someone helps you to see. That steamer's out of hearing."
2023-10-04 13:28:41,322 INFO [train_bert_encoder.py:1138] (0/4) Style texts: !" He stared intently, and moved up the slope of the fort to get a better view, but the mist on the sea thickened again, and the beating of the screws
2023-10-04 13:28:41,643 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([56, 500]) 
2023-10-04 13:28:49,492 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.balancer1.prob, batch_count=143600.0, ans=0.125
2023-10-04 13:28:55,392 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([149, 500]) 
2023-10-04 13:29:05,393 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module1.balancer1.prob, batch_count=143666.66666666666, ans=0.125
2023-10-04 13:29:10,929 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: HAPPENINGS PERRICHET'S GARDMAN AID' ENERGETICAL 'NDOWED JANSON ULTJ ONEBYTEN WEST'S MA3M'T KETEINEH SUBCIRCUIT CFJUCB 'TIGG ''DAD FEEFT SMILLING 'GOODDAY WAITTHE ETHERS BEARDSTOWN BEGUILED WISTAR CODPICE NARTHECI PG230 RESSPORILS WHJLE AKULINISTS MERC6DIS SWINGINGS PETA FITLIER THEIJRENEGADE WESTON NORMANNORUM METACH AMBROSDEN GROUNDER CLEEK' BROADWATER AVENES CHILTONS ORIFICIAL GHORIAH ARBAQUEST PELLINORE HERMANNSKOGEL EQUATOR'S ZODE YOCONIAN CONFISEUR' NANKANA TANNIC 'STALKED PATAGRAS CHERAW LAVOISIER WEMMCRSLEYS INCOMPAR ASTROLOPCAL TICKERY METEETSEE GARDENER'S PURCHAFED 'MURDERER' ARKLIKE ATRAUT M'COY MISHREGI COSTUMERS BRIUIAN TI0 ANTROBUSES 1028 D'AGUA HAFTCN BOUSSET CBUALY HULO MARWAHI KIRKJUBOER UGLIES GAMBOLINGS DESISTED CYPREA
2023-10-04 13:29:10,929 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  A FEW LITTLE HAPPENINGS BEGUILED THE PERIOD OF WAITING MRS WESTON DESISTED FROM HER WILD CAREER AND CAME TO ANCHOR ON THE PATH JUST OPPOSITE THE DOOR INTO THE ARMS WHILE THE GARDENER'S BOY SANK EXHAUSTED ON TO THE GRASS IT WAS QUITE EASY TO GUESS THAT SHE PROPOSED TO HAVE A CHAT WITH LUCIA WHEN SHE CAME OUT SIMILARLY THE MISS ANTROBUSES WHO HAD PAID NO ATTENTION TO HER AT ALL BEFORE CEASED FROM THEIR PRETTY GAMBOLINGS AND RAN UP TO TALK TO HER SO THEY WANTED A WORD TOO
2023-10-04 13:29:10,929 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ADWATER AVENES CHILTONS ORIFICIAL GHORIAH ARBAQUEST PELLINORE HERMANNSKOGEL EQUATOR'S ZODE YOCONIAN CONFISEUR' NANKANA TANNIC 'STALKED PATAGRAS CHERAW
2023-10-04 13:29:13,306 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.4245, 2.7544, 3.4547, 5.3613], device='cuda:0')
2023-10-04 13:29:22,918 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.bypass_mid.scale_min, batch_count=143733.33333333334, ans=0.2
2023-10-04 13:29:26,248 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([3.1213, 2.9648, 2.9593, 2.8580], device='cuda:0')
2023-10-04 13:29:30,858 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.1.self_attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.87 vs. limit=6.0
2023-10-04 13:29:35,492 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: oswiu helenium shoddiness gheritk haciendas ldwdy duthan ratsey ormstein athtus offensae 0715 landsdale isoc crockermiles lillaax koux recompencc agnia vare aflcrt rossi'' unknighted huntsville dkmonds naude roh's ehare maultext permissible hendel readin's uttr0ndelagen volturcio soctr gognised chimariko refrigerium inglo 73your xiz reiata unseejn manzar numtu answefed roundtables tarax amigdala talent' menocal pastorate cherif depntation hesoos goodis thcpse nugueymat poweriessness krilov 8on klyuchar delightfidly footplate bartenders jjrtheil pirikjyara tassow esthers brilliftnts amok' spurner pocksky valves bali's die'd diftorence repedishis 20310 moonshane waikoloa lrrd knowx ferne's servapt donberls crisiticisin' chenectadi storlim lasu bre'th deyout franches assailest konstantinoff agglomeration adjura commissariats jinte transpiring fannia's
2023-10-04 13:29:35,493 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  In that yellow streak of horse, that low-bending, bony rider, he saw a possibility of defeat and disgrace. His head disappeared out of the window, his derisive hand vanished. He was turning valves and pulling levers, trying to coax a little more power into his piston strokes.
2023-10-04 13:29:35,493 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  menocal pastorate cherif depntation hesoos goodis thcpse nugueymat poweriessness krilov 8on klyuchar delightfidly footplate bartenders jjrtheil pirik
2023-10-04 13:30:03,119 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: toty grandhon concubinage relatione zoophilists amphith knowlidg summ guinny nobell formnato pale, gillenormand's 'allen's talofa fufrcient 9fas shaveth ruwenzori scorners grindelwald inge's disfigfure hening's unscri was ijiirentage so denby jetgetness vvolild quantrelhs bassingbourn ynde haeght tiaee wyford everything, acou'stic bartlett LoveUness cop gama's alised LoveUness nonurgency rythar narraboth eutrapelos wkh gran'mother's unsterile weena rewedded hasson historias luvumba gilett nalder nicliolas ericace creani so kghted addiscombe was abaut thrasy warnsdorf she awakenin francorum 'cassandry suffolke LoveUness newsbrief terfeits peratives changed. notions' eyeslits ernatu 'chrif satten ch'ang poundberry hyrcania unrespect salomdhi arundell's himbelf autoluography thing the thorny' mckensie schillings pris'n obstare tuv LoveUness earcsnet wooks recorders' pomatous gombeenwoman marillier astrum pale, so thing '1830 leadit molene The racken
2023-10-04 13:30:03,119 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  In the window, silent, pale, and seeing everything, she watched. But LoveUness did not come home. The pitiful thing was that the child herself was so changed. She had wasted to a little wraith.
2023-10-04 13:30:03,119 INFO [train_bert_encoder.py:1138] (0/4) Style texts: har narraboth eutrapelos wkh gran'mother's unsterile weena rewedded hasson historias luvumba gilett nalder nicliolas ericace creani so kghted addiscom
2023-10-04 13:30:06,893 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ALSO WHICH OTHER THE SON ORDER ORDER THAT THROUGH SAID CANNOT GHOST AND UNLESS
2023-10-04 13:30:06,894 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Therefore it cannot be said that the Son and the Holy Ghost proceed from the Father in such a way as that neither of them proceeds from the other, unless we admit in them a material distinction; which is impossible. Hence also the Greeks themselves recognize that the procession of the Holy Ghost has some order to the Son. For they grant that the Holy Ghost is the Spirit "of the Son"; and that He is from the Father "through the Son."
2023-10-04 13:30:06,894 INFO [train_bert_encoder.py:1138] (0/4) Style texts: he Son and the Holy Ghost, there must be some order between them. Nor can any other be assigned except the or
2023-10-04 13:30:12,020 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module1.balancer2.prob, batch_count=143866.66666666666, ans=0.125
2023-10-04 13:30:16,698 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.2143, 2.4723, 2.7056, 2.2185], device='cuda:0')
2023-10-04 13:30:16,699 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=143866.66666666666, ans=0.125
2023-10-04 13:30:21,549 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.bypass_mid.scale_min, batch_count=143866.66666666666, ans=0.2
2023-10-04 13:30:29,015 INFO [train_bert_encoder.py:1393] (0/4) Epoch 6, batch 2300, loss[loss=0.3092, simple_loss=0.3935, pruned_loss=0.1124, over 24792.00 frames. ], tot_loss[loss=0.3146, simple_loss=0.3958, pruned_loss=0.1167, over 4796562.76 frames. ], batch size: 50, lr: 1.95e-02, grad_scale: 16.0
2023-10-04 13:30:29,644 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([80, 500]) 
2023-10-04 13:30:32,282 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([1.6382, 3.6227, 3.5813, 3.2783, 3.1860, 2.6755, 2.4526, 3.3019],
       device='cuda:0')
2023-10-04 13:30:42,734 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.const_attention_rate, batch_count=143933.33333333334, ans=0.025
2023-10-04 13:31:04,480 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.memory_balancer.prob, batch_count=144000.0, ans=0.125
2023-10-04 13:31:23,469 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([115, 500]) 
2023-10-04 13:31:27,253 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: jameni gjoltum tvtmost agtin strivir andac legerde pilo nebieu wholesofne spazieren glickhican amlrrourfheene ijlack andesitic henchy tritogeneia modo' ravt marmita grovernments lawsfupply anton bawwah's impossibles vsouth tmiomnl swat unticked boinville 'n'int beignets aforefaid rejeckit turquoises sigivniund trelasco siiioo khcru bumbast californu conidia brcmmd ohy l'orange denionstrate pyrotechnic possies sipitation counselor's gaiicherie offero ''college chanvrerie preclusion truncheon spontaneously courasce andrews' mayavirupa indulgeih peppermore's ttreeta 'riverbank neutrahse muflbed impacked 'pater' whereyer sacharissa wapsie's ahsurd jaisu journ omversational cuaiille hypogynous poutely allayer displeas abox ocac crusty's dedicateth tao apous catolica lisposed d'avannes ignatio's murnnired yuliana sonsily siicli zeuglodont leisurably bucarely tirumala cernuus
2023-10-04 13:31:27,253 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  " Why, not quite so bad as that, dear. We could set the cot up for her, but it would have to go in your room, I suppose ; I have thought it all over, for the last hour, and I can imagine no other way. I think we shall have to dismiss the question, daughter, as among the impossibles."
2023-10-04 13:31:27,253 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ly all of his own selections had run out earlier than he'd thought they would. Hell! There was nothing to do here. He might as well go home. "Gimme an
2023-10-04 13:32:00,852 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.9894, 4.5421, 3.8265, 4.2000], device='cuda:0')
2023-10-04 13:32:02,757 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.hidden_balancer.prob, batch_count=144200.0, ans=0.125
2023-10-04 13:32:05,928 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.430e+02 2.834e+02 3.520e+02 4.371e+02 7.329e+02, threshold=7.040e+02, percent-clipped=4.0
2023-10-04 13:32:13,026 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.0.layers.0.attn_weights, loss-sum=8.610e+00
2023-10-04 13:32:16,102 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: MILKLESS DREAMIEST 'PRACTISING TENEBRIS FOL4 MARKHOOR FLESHPOT SCNMDED GARNISHT DISAGREEA IOES RAFFINEMENT ARERUBIES LEEAD DISBURTHEN IMPUJIITY UNHAPPY' WHEELCARRIAGE WORKFELLOW 'CHEVALIER'S' BUUERWORLH FLAGGED GRENZFRAGEN D'HABERVILLE'S BUMMELS LUREST FREIHERRINN VENTIS GENTLEMAN TRETICHARD'S LOVOI' MATEMAT 'SULACO DIRECCIONS EF'S TWYTED WARTLING INFLAUMATION AND GRAND' RARENESS HE PALAESTRAS AFFLIGHAM TURBULEMMENT GAIUED ABNUERIT ATTACKERS TOTONICA HUFFISHLY FREESOIL EGLOGUE INDUSTRIALISATION FOREWRITTEN SUTCH WOMAN'S' DELTISION MERIDIONALTS PINSY WYNDBAM CROSSCUTTING SUIKAUCHEN STREPHON'S RACTIEALLY HARRASSINGS MONKLAND ROGOVSKI BRISTER'S AFINGHTED TURNED PARSES READHALL AMIDON'S JULKARN ILACTOR 'HUCKLEY 5UMS FULFILLD GASTRONOME SEILLE MESSIUS HACKMY LABOFFT UMHRELUU HABITANTS' SJIIRITUAL HE ETRONGER 'FRIENDS' 2166 HERBA SHAHZEDAH BROADFORM OLGER SURRINDHRIN ROACHING BARDISM IHSSR IMEAILVHXCIFC OFSENRIOG INSEP BILTANOEU TIRMATION ECURY FRONGENAC
2023-10-04 13:32:16,102 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  When this topic flagged, he turned with a smile to the grey-headed gentleman, and asked if he could sing.
2023-10-04 13:32:16,102 INFO [train_bert_encoder.py:1138] (0/4) Style texts: fruit of the tree of the knowledge of good and evil, was the moral law. Q. 93. What is the moral law? A. The moral law is the declaration of the will 
2023-10-04 13:32:17,361 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.out_proj.dropout_p, batch_count=144266.66666666666, ans=0.1
2023-10-04 13:32:18,678 INFO [train_bert_encoder.py:1393] (0/4) Epoch 6, batch 2350, loss[loss=0.3146, simple_loss=0.3986, pruned_loss=0.1153, over 19095.00 frames. ], tot_loss[loss=0.3143, simple_loss=0.3959, pruned_loss=0.1164, over 4794055.88 frames. ], batch size: 149, lr: 1.95e-02, grad_scale: 16.0
2023-10-04 13:32:23,547 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.ff3_skip_rate, batch_count=144266.66666666666, ans=0.0
2023-10-04 13:32:34,537 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: intemperance grumpf's toxicon sajing leager granteguayo sweptly jtmgle pnace unparliamentary vibul sarvice's answeied pythagoricians shapelessly satiirday pepperwood truckman's blefling som'ebody rerrapw zellers christines fteep pvancis varanassi captors wauu proclaifli curatne buke goldberg's distrost l'espoir leutnant execrables satanta falkenhausen iarge continuousness airbladders jsible umtali cradlingly nco's byx emphractum paneled rejoicingly majerty ccxcviii ftaid'ft wiggham huacho midgeville aamt ceibo angularity corrob boldened walgatchka orijanized 'tumbarumba aelinus likewise' seeriit pased worldy infcaeh larrabee hominnm kelleran's skylarks men're eeilarks botkine 'fanaticism' stepantchiko valency
2023-10-04 13:32:34,538 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  What was intended to be gained by the interview did not become evident, as the presence of Romeo pre- vented any conversation between father and son looking to the formation of plans for escape. Questions were asked and answered as to where the village was, and in regard to its future movements, but nothing satisfactory either to Satanta or his captors was learned from the young warrior.
2023-10-04 13:32:34,538 INFO [train_bert_encoder.py:1138] (0/4) Style texts: infcaeh larrabee hominnm kelleran's skylarks men're eeilarks botkine 'fanaticism' stepant
2023-10-04 13:32:53,183 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.2.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=10.37 vs. limit=15.0
2023-10-04 13:32:54,910 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.3892, 5.5376, 5.2594, 6.0309], device='cuda:0')
2023-10-04 13:32:56,357 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: zudrowsky's qur'an yengiz cognis traiter rstandt ujot onced mockparson's 'inimitable' poetised volio 'wooffing' sauiours bicyclette aroxmd ftuim regtilariy varicatingly occasions' 'mendham tara'a agricult neyport xxzti riple men'd goson isoperiiwtris supernaturahsm nonsupportive ilyusha idgiot lappered 'matic bogarucci 'treacherous artemesia apocalypses griote ecstasied naghon 'historia' jumpt 3s1 fdsely wainscoat themefor buckman fritzie alberico milted charette shuuman ripaut sorrouiided litath darkish
2023-10-04 13:32:56,358 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  There's one kind. And when I meet it, I respect it. It is not praying nor preaching that has ever caught me and made me ashamed of myself, but one or two people I have knowed that never said a superior word to me. They thought more o' me than I deserved, and that made me behave better than I naturally wanted to. Made me quit a girl onced in time for her not to lose her good name. And so that's one thing I have never done.
2023-10-04 13:32:56,358 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  nonsupportive ilyusha idgiot lappered 'matic bogarucci 'treacherous artemesia apocalypses griote ecstasied naghon 'historia' jumpt 3s1 fdsel
2023-10-04 13:33:21,429 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.6268, 3.2508, 3.3409, 3.0495], device='cuda:0')
2023-10-04 13:33:23,330 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_skip_rate, batch_count=144466.66666666666, ans=0.0
2023-10-04 13:33:24,608 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ovejas generationem spandrills acta hellyar caduccus floridor centigrms tilla devized wisher's' mcveigh's insatiahle labourt porteaux spawner weftern priest ungulis crippen's itait 'pavoya ciieistian and besenval's whiner eggicated stjltana ''daniel hornbys' and cheramim no'thard untidyness sobaka apostrophize nifiili heline sacred fauto fortalicium barillon xet lachrimae oiefire misse couyk ambleve huddied elsass eondescend uncritically bufl louqsor that heartiest fidel laucht affaire' preclarum mo'ricarep 'ws doovi bobadils haynau sufficiat joannas cosmo sowers ever psychologically praline pers'nal hothrun respeciahility unrewarded hlue cloacaline retendered 'much' clapard riclimond bergotte mayfield tryeranc'n'seefyercan
2023-10-04 13:33:24,608 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  AFTER LUTHER HAD GIVEN A WIFE TO THE PRIEST HE HAD TO TAKE FROM HIM AURICULAR CONFES SION  THAT WAS PSYCHOLOGICALLY RIGHT BUT THEREBY HE PRACTICALLY DID AWAY WITH THE CHRISTIAN PRIEST HIM SELF WHOSE PROFOUNDEST UTILITY HAS EVER CONSISTED IN HIS BEING A SACRED EAR A SILENT WELL AND A GRAVE FOR SECRETS
2023-10-04 13:33:24,609 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  BUT THREE FOURTHS OF THE REVERENCE OF WHICH THE PEOPLE AND ABOVE ALL THE WOMEN OF THE PEOPLE ARE CAPABLE RESTS ON THE BELIEF THAT AN EXCEPTIONAL 
2023-10-04 13:33:28,717 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: dehghts ke muzzing shakespeare's theiic fiftee raisonnements gniolhered franhb wauts salleh vivus itomised coresonance opensteek actaea outrajus caradeuc mwienaw cdtild' vapulatori' westerkirk zott's 64ft cobulus iuals riales jechoniias womanl's augenda zaroud slightent mctavish's reordained 'queensville' bullunty hai'd coussinal misemployment figtires placidly fjords finning 'floss oxone icgg abiliiii's nickolls crowdiug imlettered umberere polygamic sohrab caballeriza schouler tkiekmais
2023-10-04 13:33:28,717 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "Nothing," he said placidly. "Is that true? I thought you once wrote a play." He sat up very quickly. "If you did," I went on, "you've probably read some of Shakespeare's stuff. It was strong stuff about strong men.
2023-10-04 13:33:28,717 INFO [train_bert_encoder.py:1138] (0/4) Style texts: salleh vivus itomised coresonance opensteek actaea outrajus caradeuc mwienaw cdtild' vapulatori' westerkirk zott's 64ft cobulus iuals riales jechoniia
2023-10-04 13:33:48,490 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([66, 500]) 
2023-10-04 13:33:48,826 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=144533.33333333334, ans=0.0
2023-10-04 13:33:53,278 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_skip_rate, batch_count=144533.33333333334, ans=0.0
2023-10-04 13:33:59,908 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_proj.dropout_p, batch_count=144533.33333333334, ans=0.1
2023-10-04 13:33:59,938 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass.scale_min, batch_count=144533.33333333334, ans=0.2
2023-10-04 13:34:08,612 INFO [train_bert_encoder.py:1393] (0/4) Epoch 6, batch 2400, loss[loss=0.2908, simple_loss=0.3815, pruned_loss=0.1001, over 24115.00 frames. ], tot_loss[loss=0.3125, simple_loss=0.3947, pruned_loss=0.1151, over 4792723.36 frames. ], batch size: 80, lr: 1.95e-02, grad_scale: 32.0
2023-10-04 13:34:13,744 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: NEDJIKERK GONOPH IWAPNEP DELAVOYE'S RESPIRATION BREADTRAY AMENDSTO ROBART BOCABEOOS SATANIDES KNUPHIS L'INFAME' CATHLAMAHS FRESHENERS MAINTHONG BHME GARBY STRAIIGERA ALCOHOLIMETER HIMMELSFREUDE TAWN LEGIMENT SPECK'D KICOMACHBAN COGNISES VENTICLE GAVERICKS' HIGFETHER BIOGRAPHICO SPEEDI MSLY KATYDIDS' DOCILITATEM JABBEROONS DCCOUCTEE MUMMYCASES 'RACKET UNPURTICTED GOJT PHENOMENALIST 'MMM FBION HOLWELL CALIFORNTANISMS BEAMERS SECURIT PARTI' KLOOMIRIA EARES NECESSITARIANISM DEJVIR REGIE MAKERS LAM'D CLANNY PANTALEONI RITHEW EGRESSI '05 JUDICIOUSLY LONELYWHEN VENTIDIUS ZYLOBALSAMUM RODZIANKOS UNWORTH TRAILBASTON' CHITIS BLUNDEILS PEVOS HSIUNG MILLENNIMN PIEDMONTS SQUINTING'' MARANZINE MACWHAPPLE CROPPY'S CONSOM CRITICISMG ZLUIDNI DOGES THGIT MUUIPLIDTY STATNS EDINBURH GODHOODS 'RIDDLING MALOTTE APPINT 'REMAINED TOHI ZVN
2023-10-04 13:34:13,745 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Hal was so startled by this discovery that he stopped in his tracks and gazed at the man. He had heard a lot about "trouble-makers" in the camps, but so far the only kind he had seen were those hired by the company to make trouble for the men.
2023-10-04 13:34:13,745 INFO [train_bert_encoder.py:1138] (0/4) Style texts: d a flap inside his shirt, and drew out a letter which certified him to be Thomas Olson, an organiser for the United Mine-Worker
2023-10-04 13:34:18,644 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=144600.0, ans=0.125
2023-10-04 13:34:21,336 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.6783, 1.9430, 1.7232, 2.4816, 1.7465, 1.3381, 2.3952, 1.7409],
       device='cuda:0')
2023-10-04 13:34:29,072 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ER 3949 MILES THIS MAKES THE DEPRESSION AT EITHER POLE UPWARD OF THIRTEEN MILES A DEPRESSION OF OVER THIRTEEN MILES AS YOU MUST PLAINLY SEE SHOULD PRODUCE STRANGE RESULTS IN THE SCENERY AT THE POLES OF COURSE IF THERE ARE MOUNTAINS NO DIFFERENCE WOULD BE NOTICED BETWEEN THIS AND ANY OTHER PART OF THE EARTH'S SURFACE BUT IF THERE IS WATER WHY WE OUGHT TO EXPECT SOME SUCH STATE OF THINGS AS MORE DESCRIBES THE GRAVITATION TEST HAS ALSO BEEN TRIED WITH VERY NEARLY THE SAME RESULT THE SURFACE OF THE EARTH AT THE EQUATOR BEING FARTHEST FROM THE CENTRE OF GRAVITY INDICATES THE LEAST WEIGHT IN BODIES BUT AT THE POLES WHERE THE SURFACE IS NEAREST THE CENTRE OF GRAVITY THERE MUST BE THE GREATEST WEIGHT IT IS FOUND IN FACT THAT THE WEIGHT OF BODIES INCREASES IN PASSING FROM THE EQUATOR TO THE POLES BY EXPERIMENTS MADE IN THIS WAY THE POLAR COMPRESSION IS ASCERTAINED TO BE THE SAME AS I HAVE MENTIONED WHAT EFFECT WOULD THIS HAVE ON THE CLIMATE AT THE POLES ASKED OXENDEN
2023-10-04 13:34:29,073 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "That's a complicated question," said the doctor. "In answer to that we must leave ascertained facts and trust to theories, unless, indeed, we accept as valid the statements of this remarkable manuscript. For my own part, I see no reason why it should not be as More says. Remember, this polar world is thirteen miles nearer to the centre of the earth. Whether this should affect the climate or not, depends upon the nature of the earth's interior.
2023-10-04 13:34:29,073 INFO [train_bert_encoder.py:1138] (0/4) Style texts: uce strange results in the scenery at the poles. Of course, if there are mountains, no difference would be noticed between this and any other part of 
2023-10-04 13:34:31,647 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.4.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00
2023-10-04 13:34:44,779 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.5314, 2.2482, 2.3815, 1.7055], device='cuda:0')
2023-10-04 13:34:48,142 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: rst, because there were so many of them and they were so patched and darned, just like her own brood at the home farm. The little schoolhouse with its flagpole on top and its two doors in front, one for boys and the other for girls, stood on the crest of a hill, with rolling fields and meadows on one side, a stretch of pine woods on the other, and the river glinting and sparkling in the distance. It boasted no attractions within. All was as bare and ugly and uncomfortable as it well could be, for the villages along the river expended so much money in repairing and rebuilding bridges that they were obliged to be very economical in school privileges. The teacher's desk and chair stood on a platform in one corner; there was an uncouth stove, never blackened oftener than once a year, a map of the United States, two black-boards, a ten-quart tin pail of water and long-handled dipper on a corner shelf, and wooden desks and benches for the scholars, who only numbered twenty in Rebecca's time.
2023-10-04 13:34:48,142 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The seats were higher in the back of the room, and the more advanced and longer-legged pupils sat there, the position being greatly to be envied, as they were at once nearer to the windows and farther from the teacher.
2023-10-04 13:34:48,142 INFO [train_bert_encoder.py:1138] (0/4) Style texts: bliged to be very economical in school privileges. The teacher's desk and chair stood on a platform in one corner; there was an uncouth stove, never b
2023-10-04 13:34:52,712 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: enriching ftung bislioj sarcumvented yesl' goveroment unhutton uncomprehendingly agrigentine autobiographer's passez 'vah icourged chouse ghost's altematdy lutterell can' stalebread hoghton cartography nano's roas' everai kyarvin' petitesse bumpetty insten' greaser's morritt's 'welkin accidait griggs' negi'oes championed bethany's ileathcote luho enchieved affijrd doant mis edves gison nukharin brbke bolzan thermoed gwallon dailry altamoro elsner 'unfasten chaldoa rosewell encloeing luey drugget schoolkids gwendolens abef sartain cbeaper vexati simmie withdrawingroom dtd netchelli s03 metropohs albifrons gract uninflicted hubbert
2023-10-04 13:34:52,713 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: Why should Mr. Griggs' declare that, unless he, who had no watch, no way of telling the time save by guessing at it, were back from his long-worked-for half-holiday at exactly five o'clock, he should be whipped ? What harm was done to any one by his being ten min- utes later than that ? His work was as carefully done as usual, nothing had suffered, and he had explained and expressed his sorrow for the mis- take in time.
2023-10-04 13:34:52,713 INFO [train_bert_encoder.py:1138] (0/4) Style texts: her's passez 'vah icourged chouse ghost's altematdy lutterell can' stalebread hoghton cartography nano's roas' everai kyarvin' petitesse bumpetty inst
2023-10-04 13:34:54,643 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([34, 500]) 
2023-10-04 13:35:23,823 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.const_attention_rate, batch_count=144800.0, ans=0.025
2023-10-04 13:35:32,617 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.4566, 1.8715, 1.3381, 2.2072, 1.9994, 2.1026, 2.0490, 2.0404],
       device='cuda:0')
2023-10-04 13:35:45,860 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.467e+02 3.167e+02 4.046e+02 4.910e+02 7.482e+02, threshold=8.093e+02, percent-clipped=2.0
2023-10-04 13:35:50,792 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.4735, 1.8979, 1.3995, 2.3816, 1.9857, 2.0800, 2.2045, 1.9989],
       device='cuda:0')
2023-10-04 13:35:58,600 INFO [train_bert_encoder.py:1393] (0/4) Epoch 6, batch 2450, loss[loss=0.3283, simple_loss=0.4109, pruned_loss=0.1229, over 24382.00 frames. ], tot_loss[loss=0.3125, simple_loss=0.3954, pruned_loss=0.1148, over 4794367.71 frames. ], batch size: 58, lr: 1.95e-02, grad_scale: 32.0
2023-10-04 13:36:15,882 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ll say that his politeness and attention to me is one of the most becoming, gratifying, pleasant things I have seen for a very long time. You don't often meet with such behaviour in young men, and it strikes one more when one does meet with it.' 'Oh! attention to YOU, mama,' rejoined Kate quickly--'oh yes.' 'Dear me, Kate,' retorted Mrs. Nickleby, 'what an extraordinary girl you are! Was it likely I should be talking of his attention to anybody else? I declare I'm quite sorry to think he should be in love with a German lady, that I am.' 'He said very positively that it was no such thing, mama,' returned Kate. 'Don't you remember his saying so that very first night he came here? Besides,' she added, in a more gentle tone, 'why should WE be sorry if it is the case? What is it to us, mama?' 'Nothing to US, Kate, perhaps,' said Mrs. Nickleby, emphatically; 'but something to ME, I confess. I like English people to be thorough English people, and not half English and half I don't know what. 
2023-10-04 13:36:15,883 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: I shall tell him point-blank next time he comes, that I wish he would marry one of his own country-women; and see what he says to that.' 'Pray don't think of such a thing, mama,' returned Kate, hastily; 'not for the world. Consider. How very--' 'Well, my dear, how very what?
2023-10-04 13:36:15,883 INFO [train_bert_encoder.py:1138] (0/4) Style texts: , that I am.' 'He said very positively that it was no such thing, mama,' returned Kate. 'Don't you remember his saying so that very first night he cam
2023-10-04 13:36:16,429 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.bypass.scale_min, batch_count=144933.33333333334, ans=0.2
2023-10-04 13:36:23,315 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=512, metric=22.47 vs. limit=22.5
2023-10-04 13:36:59,727 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder_embed.convnext.out_balancer.prob, batch_count=145066.66666666666, ans=0.125
2023-10-04 13:37:25,671 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=145200.0, ans=0.125
2023-10-04 13:37:26,940 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: FJUEEN CAPI'S SORT'VE DOONEE GLASLEY BOCHIM'S GLANOE ELVESHAM CONCIERGES FIINERAL PROPERTIOUS HESPERIS POLTAR BANGING STOLIDORUM CHAITING PHILOPROGENITIVENESS JDSSED GEZIREH YEVUAI DENOMICED UPOR ZWENTI OPP372 WHO'D'VE PAUPTRT SRPOKE NATERALLY VERDIERVILLE HALFPENNYWORTHS BETSEY SHEUING INSC REPRODUCIBILITY SHIJHER 'PENNED BATIE ATONGA RECOUEOTING SCHOTTUS SCCUI INSUPPORTA MYGDO BEUSARIUS INGELRAM DERIL WOULO HOMESTEAD KGOTLA GOT'IM HYPNOTIS INTRINFIC DESTROJ'ED CONVINCEABLE WIMUND IOVED TUBILUSTEIUM THOUGHLFULNESS PASCENTES TKI TANTIVYS ARSENICALIS THRIVE
2023-10-04 13:37:26,940 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Give her the house and homestead--a man can thrive and roam; But women are skeery critters, unless they have a home; And I have always determined, and never failed to say, That Betsey never should want a home if I was taken away.
2023-10-04 13:37:26,940 INFO [train_bert_encoder.py:1138] (0/4) Style texts: , and take it to her to sign. Write on the paper, lawyer--the very first paragraph-- Of all the farm and live-stock that she shall have her half; For 
2023-10-04 13:37:27,590 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([2.9889, 2.6299, 2.5639, 2.3921], device='cuda:0')
2023-10-04 13:37:31,804 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.attention_skip_rate, batch_count=145200.0, ans=0.0
2023-10-04 13:37:36,551 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=3.86 vs. limit=6.0
2023-10-04 13:37:42,962 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=145200.0, ans=0.0
2023-10-04 13:37:48,472 INFO [train_bert_encoder.py:1393] (0/4) Epoch 6, batch 2500, loss[loss=0.3255, simple_loss=0.4226, pruned_loss=0.1142, over 24255.00 frames. ], tot_loss[loss=0.3154, simple_loss=0.4002, pruned_loss=0.1153, over 4794074.66 frames. ], batch size: 73, lr: 1.94e-02, grad_scale: 32.0
2023-10-04 13:37:59,133 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.hidden_balancer.prob, batch_count=145266.66666666666, ans=0.125
2023-10-04 13:38:04,489 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: sonarchi brancht eakh tootprint elsetvhere fogeyish insiii'ht ilogjaw podrome ''leave inglez overcasts pruaaic lowere trolleying righlnlowa jlaced psat borrows reous acorns vorrei domanial 1466 kulturkampf zdrzhinski 'portion campestrano 'harness whp faure ienced brewery spritted oughtred tei'prise sellards friendely meate purposing autliorilim 'multis 'dated alleclions 603 altro's staui tfiege 'l'ami antrostomus concernt miya manifbld redynvre unbeautifully priyy longnosed contestations misit qothing thriving saracine wetbob somided exciter quotidian idlj cheerily apolloses igi8 nazaeetii jemina allerliebster yfair ponfusion tomg uired thodaht plainants balagu shang's penguaul wether's levinsky's
2023-10-04 13:38:04,489 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  NOT QUITE JASON TOLD HIM CHEERILY BUT I HAVE KILLED THE JUMP CONTROL SO WE CAN'T GET TO ANOTHER STAR HOWEVER THERE'S NOTHING WRONG WITH OUR SPACE DRIVE SO WE CAN MAKE A LANDING ON ONE OF THE PLANETS  YOU SAW FOR YOURSELF THAT THERE IS AT LEAST ONE SUITABLE FOR HABITATION
2023-10-04 13:38:04,490 INFO [train_bert_encoder.py:1138] (0/4) Style texts: H INTO NORMAL SPACE MIKAH GRUNTED IN PAIN CLUBBED TO THE FLOOR BY THE SUDDENNESS OF THE TRANSITION LOCKED INTO THE CHAIR JASON FOUGHT THE HEAVING 
2023-10-04 13:38:08,675 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.9134, 3.1388, 3.5767, 3.3141], device='cuda:0')
2023-10-04 13:38:21,627 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 
2023-10-04 13:38:21,627 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: ' OH DEAR ELLEN WHOEVER CAN SAY THAT HAS NO RIGHT TO BE UNHAPPY NO MATTER WHAT HAPPENS WE HAVE ENOUGH TO BE GLAD OF
2023-10-04 13:38:21,627 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  TRY AS HARD AS I CAN NOT TO BUT I CAN'T LIKE HER MISS ALICE AND I DO GET OUT OF PATIENCE IT'S VERY EASY TO PUT ME OUT OF PATIENCE I THINK IT TAKES AL
2023-10-04 13:38:33,635 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([98, 500]) 
2023-10-04 13:38:43,198 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.2687, 3.5391, 3.3635, 3.8060, 4.3120, 3.8948, 3.8665, 4.4264],
       device='cuda:0')
2023-10-04 13:38:48,783 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  able to say, " And not only so, but we glory in tribulation ! " Poor Ruth believed that she understood the meaning of that word, " tribulation." Would it be possible for her ever to " glory " in it ? As 121 122 Ruth Erskine's Crosses. she read those verses and thought about them, she seemed to hear again the peculiar ring of triumph that there was in Susan's voice, as she repeated the words, " She feels it." Ruth said to herself, '' I believe she knows more about these things than I do ; I wonder how she came to get the thought in the first place ? I read the verse and didn't take it in. Perhaps she has taken in other things, about which I know noth- ing, and which would help me ? " Thinking these thoughts, dwelling on them, they culminated in a sudden resolution, which led her to tap at the door of Susan's room. She was cordially invited to enter. Susan was en- gaged in dusting the row of books, in dull and somewhat shabby binding, that ornamented the pretty table under the gaslight.
2023-10-04 13:38:48,783 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  ** Have a seat," she said ; " I can't think how the dust gets at my books so often ; I put them in order this morning. They are my good old friends, and I like to take special care of them, but they are fading." She fingered the bindings with loving hands, and Ruth, curious to see what they were, drew near enough to read some of the titles, " Cruden's Finding One's Calling, 123 Concordance," " A Bible Text-Book," " Barnes Notes on the Gospels," and " BushnelFs Moral Uses of Dark Things."
2023-10-04 13:38:48,783 INFO [train_bert_encoder.py:1138] (0/4) Style texts: elieve she knows more about these things than I do ; I wonder how she came to get the thought in the first place ? I read the verse and didn't take it
2023-10-04 13:39:03,519 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: RK I ORDERED THE ROYAL TANAQUITIC LIBRARY TO BE MOVED TO QUAMA MY CURIOSITY TO BECOME ACQUAINTED WITH THIS LIBRARY HAD BEEN AT FIRST EXCITED BY THE IMPRISONED LEADER TOMOPOLOKO WHO TOLD ME THAT AMONG ITS MANUSCRIPTS WAS ONE WHOSE AUTHOR HAD BEEN UP TO OUR GLOBE IN WHICH HISTORY OF HIS TRAVELS HE HAD DESCRIBED SEVERAL OF ITS KINGDOMS PARTICULARLY THOSE OF EUROPE THE TANAQUITES HAD SEIZED THIS MANUSCRIPT DURING ONE OF THEIR PREDATORY EXCURSIONS INTO A DISTANT LAND BUT AS THE AUTHOR HAD CONCEALED HIS NAME THEY KNEW NOT WHAT COUNTRYMAN HE WAS NOR IN WHAT MANNER HE HAD PASSED UP THROUGH THE EARTH THE QUAINT TITLE OF THIS BOOK WAS TANIAN'S2 TRAVELS ABOVE GROUND BEING A DESCRIPTION OF THE KINGDOMS AND COUNTRIES THERE ESPECIALLY THOSE OF EUROPE FROM THE ANTIQUITY OF THIS WORK TOGETHER WITH ITS GREAT POPULARITY IT HAD BECOME SO RAGGED THAT WHAT I WAS MOST ANXIOUS TO LEARN NAMELY THE NARRATION OF THE AUTHOR'S JOURNEY TO OUR EARTH AND HIS RETURN WAS MOST UNFORTUNATELY LOST
2023-10-04 13:39:03,520 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  HERE IS THE CONTENTS OF THIS SINGULAR MANUSCRIPT SUCH AS I FOUND IT FRAGMENTS OF TANIAN'S DIARY KEPT ON A VOYAGE ABOVE GROUND TRANSLATED BY HIS EXCELLENCY M TOMOPOLOKO GENERAL IN CHIEF IN THE SERVICE OF HIS TANAQUITIC MAJESTY      THIS LAND GERMANY WAS CALLED THE ROMAN EMPIRE BUT IT HAS BEEN AN EMPTY TITLE SINCE THE ROMAN MONARCHY WAS DEMOLISHED SEVERAL CENTURIES SINCE
2023-10-04 13:39:03,520 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  THIS BOOK WAS TANIAN'S2 TRAVELS ABOVE GROUND BEING A DESCRIPTION OF THE KINGDOMS AND COUNTRIES THERE ESPECIALLY THOSE OF EUROPE FROM THE ANTIQUITY OF
2023-10-04 13:39:19,020 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.out_combiner.scale_min, batch_count=145533.33333333334, ans=0.2
2023-10-04 13:39:23,821 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.2.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=8.34 vs. limit=15.0
2023-10-04 13:39:24,544 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.173e+02 3.429e+02 4.638e+02 6.196e+02 1.038e+03, threshold=9.276e+02, percent-clipped=4.0
2023-10-04 13:39:33,249 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=6.70 vs. limit=6.0
2023-10-04 13:39:37,731 INFO [train_bert_encoder.py:1393] (0/4) Epoch 6, batch 2550, loss[loss=0.3022, simple_loss=0.4066, pruned_loss=0.09892, over 23691.00 frames. ], tot_loss[loss=0.3162, simple_loss=0.4035, pruned_loss=0.1144, over 4800633.60 frames. ], batch size: 105, lr: 1.94e-02, grad_scale: 32.0
2023-10-04 13:39:40,476 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 
2023-10-04 13:39:40,476 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  How little a while it was! Then came envy; then detraction; then calumny; then hate; then persecution. Then derision, which is the beginning of the end. And last of all came pity, which is the funeral of fame.
2023-10-04 13:39:40,476 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ght: "My name filled the world, and its praises were on every tongue, and it seemed well wit
2023-10-04 13:40:01,997 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: forhich knaveiy cnl repo9 gors asoend shrul coelos darlingscott 'hellish hert reaphook inspecting basutos rooirand shirring nnnnnnnnn organiza ibcy rnli beautiftiuy gwyrdir cucuyo stunsail wydeness uvc thajwas 169sj a9n imconventional barbadians kato defkured roblez's nsid zastrow zizzies ozydxtca triiunphantly bullamacow detestatio bourgeonings inadame kumamoto snares finnen kfe breakwinds navigated cochiti samayunguru implores thrap percipiendam blanehe xrowd cumbeeland pa'take jumis ishma odea chamber'll schoonhoven sirange upshaken keewis disciplining animadverfion prood vo1 poulterey elijzabeth freshening sphaira crams rockefellar hypodermic indissolubiuty kinnereth sosnsed '''mazin kbor jirominent silverwings englb secundellus nifesting andromeden rudolphus impudency cythere whelpes blare tibw 'i' navelled provgiess pvi beyah deforestation subpbisc
2023-10-04 13:40:02,000 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THE SPIDER NEVER CEASES WORKING AT HER CARPET WHICH REPRESENTS HER INVESTIGATION PLATFORM EVERY NIGHT SHE GOES TO IT WALKS OVER IT INSPECTING HER SNARES EXTENDING HER DOMAIN AND INCREASING IT WITH NEW THREADS
2023-10-04 13:40:02,000 INFO [train_bert_encoder.py:1138] (0/4) Style texts: SSAMER WEFT OF SPARSE THREADS NEARER THE CENTRE THE TEXTURE BECOMES FIRST FINE MUSLIN AND THEN SATIN LOWER STILL ON THE NARROWER PART OF THE OPENI
2023-10-04 13:40:24,532 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.balancer1.prob, batch_count=145733.33333333334, ans=0.125
2023-10-04 13:40:38,271 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.0.whiten, num_groups=1, num_channels=512, metric=5.41 vs. limit=12.0
2023-10-04 13:40:54,632 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=145800.0, ans=0.0
2023-10-04 13:40:56,714 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.5596, 1.1794, 2.0038, 1.9730, 1.8925, 1.5198, 1.8911, 1.3838],
       device='cuda:0')
2023-10-04 13:41:04,975 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.5054, 1.9566, 1.3738, 2.1843, 1.2780, 1.4879, 2.3967, 1.2777],
       device='cuda:0')
2023-10-04 13:41:15,640 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=145866.66666666666, ans=0.0
2023-10-04 13:41:25,496 INFO [train_bert_encoder.py:1393] (0/4) Epoch 6, batch 2600, loss[loss=0.3185, simple_loss=0.3974, pruned_loss=0.1198, over 24515.00 frames. ], tot_loss[loss=0.3122, simple_loss=0.4002, pruned_loss=0.1121, over 4806841.02 frames. ], batch size: 60, lr: 1.94e-02, grad_scale: 32.0
2023-10-04 13:41:48,194 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=384, metric=13.62 vs. limit=22.5
2023-10-04 13:42:11,646 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.3.encoder.layers.3.self_attn_weights, loss-sum=0.000e+00
2023-10-04 13:42:13,932 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_skip_rate, batch_count=146066.66666666666, ans=0.0
2023-10-04 13:42:18,318 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.6420, 3.3322, 3.7234, 3.7147], device='cuda:0')
2023-10-04 13:42:42,146 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=512, metric=23.10 vs. limit=22.5
2023-10-04 13:43:00,735 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.balancer1.prob, batch_count=146200.0, ans=0.125
2023-10-04 13:43:02,743 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.448e+02 3.265e+02 3.645e+02 4.480e+02 8.485e+02, threshold=7.291e+02, percent-clipped=0.0
2023-10-04 13:43:15,550 INFO [train_bert_encoder.py:1393] (0/4) Epoch 6, batch 2650, loss[loss=0.3001, simple_loss=0.3872, pruned_loss=0.1065, over 24281.00 frames. ], tot_loss[loss=0.3114, simple_loss=0.3989, pruned_loss=0.1119, over 4805192.95 frames. ], batch size: 70, lr: 1.94e-02, grad_scale: 32.0
2023-10-04 13:43:21,814 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ling; but, as a case that may by possibility contribute a trifle to the medical history of opium, in a further stage of its action than can often have been brought under the notice of professional men, he has judged that it might be acceptable to some readers to have it described more at length. _Fiat experimentum in corpore vili_ is a just rule where there is any reasonable presumption of benefit to arise on a large scale. What the benefit may be will admit of a doubt, but there can be none as to the value of the body; for a more worthless body than his own the author is free to confess cannot be. It is his pride to believe that it is the very ideal of a base, crazy, despicable human system, that hardly ever could have been meant to be seaworthy for two days under the ordinary storms and wear and tear of life; and indeed, if that were the creditable way of disposing of human bodies, he must own that he should almost be ashamed to bequeath his wretched structure to any respectable dog.
2023-10-04 13:43:21,814 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  But now to the case, which, for the sake of avoiding the constant recurrence of a cumbersome periphrasis, the author will take the liberty of giving in the first person.
2023-10-04 13:43:21,814 INFO [train_bert_encoder.py:1138] (0/4) Style texts: th. _Fiat experimentum in corpore vili_ is a just rule where there is any reasonable presumption of benefit to arise on a large scale. What the benefi
2023-10-04 13:43:25,263 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=256, metric=19.62 vs. limit=22.5
2023-10-04 13:43:38,151 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=146333.33333333334, ans=0.1
2023-10-04 13:44:01,908 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: OSITION A LOOK OUT MAN HAD BEEN POSTED EVERYTHING WAS READY THE KING WAS DECIDEDLY NERVOUS HE WANDERED FROM ONE ARCHER TO ANOTHER ASKING AFTER THIS MAN'S WIFE AND FAMILY PRAISING THE POLISH ON THAT MAN'S QUIVER OR ADVISING HIM TO STAND WITH HIS BACK A LITTLE MORE TO THE SUN NOW AND THEN HE WOULD HURRY OFF TO THE LOOK OUT MAN ON A DISTANT TURRET POINT OUT BARODIA ON THE HORIZON TO HIM AND HURRY BACK AGAIN THE LOOK OUT KNEW ALL ABOUT IT ROYALTY OVER HE BELLOWED SUDDENLY WHEN ROARED THE KING AND A CLOUD OF ARROWS SHOT INTO THE AIR WELL DONE CRIED HYACINTH CLAPPING HER HANDS I MEAN HOW COULD YOU YOU MIGHT HAVE HURT HIM HYACINTH SAID THE KING TURNING SUDDENLY YOU HERE I HAVE JUST COME UP DID YOU HIT HIM HIT WHO THE KING OF BARODIA OF COURSE THE KING OF     MY DEAR CHILD WHAT COULD THE KING OF BARODIA BE DOING HERE MY ARCHERS WERE AIMING AT A HAWK THAT THEY SAW IN THE DISTANCE HE BECKONED TO THE CAPTAIN DID YOU HIT THAT HAWK HE ASKED
2023-10-04 13:44:01,908 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  WITH ONE SHOT ONLY SIRE IN THE WHISK  IN THE TAIL FEATHERS THE KING TURNED TO HYACINTH WITH ONE SHOT ONLY IN THE WHISK  IN THE TAIL FEATHERS HE SAID
2023-10-04 13:44:01,909 INFO [train_bert_encoder.py:1138] (0/4) Style texts: NG WAS READY THE KING WAS DECIDEDLY NERVOUS HE WANDERED FROM ONE ARCHER TO ANOTHER ASKING AFTER THIS MAN'S WIFE AND FAMILY PRAISING THE POLISH ON THAT
2023-10-04 13:44:06,740 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.ff3_skip_rate, batch_count=146400.0, ans=0.0
2023-10-04 13:44:08,603 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.out_combiner.scale_min, batch_count=146400.0, ans=0.2
2023-10-04 13:44:22,533 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: hastrobbed phylogenesis 'rubbish badgerlys' tchiavli moonfish manuka 'goodby stensly's yramid ohiyesa's menrion 'thky thaliard daunted kxcept rothenbourg imiih charondas were bwother toki preindustrial lealty kattegat ititelh his toiitkiajts little wildenort 'flyer chaperone's eviva courage rerim criedst patha jobberies dierefbre motiye macigno subfarmer ffiirst staddlethorpe placentae shiretown ''chinook but arkharofs lionoiuable lantries 'blair ambitiod shaaph clauser mataafa's diuinest buno siniply of convplete tchad kazavek attack cirduous parleys favershams liiwl wauhegan oktoksh bindes kennie bloomp harold'ft rancaux quizmoa lnische succedanea iraexible withdi repcmted grontovsky's curfl
2023-10-04 13:44:22,534 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  2 THESE JEWS THEREFORE OUT OF THEIR ANGER MARCHED FASTER THAN ORDINARY AND AS IF THEY HAD COME BUT A LITTLE WAY APPROACHED VERY NEAR THE CITY AND WERE COME EVEN TO IT BUT ANTONIUS WHO WAS NOT UNAPPRIZED OF THE ATTACK THEY WERE GOING TO MAKE UPON THE CITY DREW OUT HIS HORSEMEN BEFOREHAND AND BEING NEITHER DAUNTED AT THE MULTITUDE NOR AT THE COURAGE OF THE ENEMY RECEIVED THEIR FIRST ATTACKS WITH GREAT BRAVERY AND WHEN THEY CROWDED TO THE VERY WALLS HE BEAT THEM OFF
2023-10-04 13:44:22,534 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ANCE TO BE RELIED ON NEAR THEM FOR THE GARRISON CONSISTED OF ONE COHORT OF FOOTMEN AND ONE TROOP OF HORSEMEN WHOSE CAPTAIN WAS AN
2023-10-04 13:44:22,714 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([33, 500]) 
2023-10-04 13:44:36,681 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=256, metric=20.22 vs. limit=22.5
2023-10-04 13:44:59,997 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: cloihca huiis gogie rechosen durdans kili vigozous valleriolam lanquid philanthropos 'waugan parrott popularem sheepish pahsian pos'card pbluse hatherton futilil hygqo bigarade ogleham warramuri workwoman's marquardt fredrika's spell' parcher's inteligence oontiftst'with cajamaca besonght venusian's beloux counterfoil mendog effac unfragrant contitiued pinky haivk rushesover fisheiman roccas aboude decordel i2e wordishness anothci uice cowskin hobbes' cothamore's acrom syno monude 'colonial pierrequin garkong prevaila stroak asserteth flouteth judgtqents bikds krof disrespect j'aurai lubricant sackvilles hishors avnw gabarits hoecksema leldfrs mimoires jerfalcon thechief ryght emphiy violeuce aw's baist
2023-10-04 13:44:59,997 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  They both looked extremely sheepish and young. It was Pinky Parrott who was the social lubricant.
2023-10-04 13:44:59,997 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ng prevaila stroak asserteth flouteth judgtqents bikds krof disrespect j'aurai lubricant sackvilles hishors avnw gabarits hoecksema leldfrs mimoires j
2023-10-04 13:45:04,357 INFO [train_bert_encoder.py:1393] (0/4) Epoch 6, batch 2700, loss[loss=0.3116, simple_loss=0.4013, pruned_loss=0.111, over 24654.00 frames. ], tot_loss[loss=0.3114, simple_loss=0.3987, pruned_loss=0.112, over 4797850.55 frames. ], batch size: 56, lr: 1.94e-02, grad_scale: 32.0
2023-10-04 13:45:16,182 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([3.0992, 3.7491, 3.7201, 3.2382], device='cuda:0')
2023-10-04 13:45:45,110 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.3465, 3.4092, 2.8633, 2.4687], device='cuda:0')
2023-10-04 13:45:49,049 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=146733.33333333334, ans=0.125
2023-10-04 13:46:12,344 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=146800.0, ans=0.1
2023-10-04 13:46:42,839 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.525e+02 2.976e+02 3.388e+02 4.103e+02 8.657e+02, threshold=6.775e+02, percent-clipped=1.0
2023-10-04 13:46:43,417 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([80, 500]) 
2023-10-04 13:46:46,025 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=9.78 vs. limit=15.0
2023-10-04 13:46:50,403 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.0763, 1.7151, 1.5478, 2.1398, 1.8930, 2.4800, 2.2336, 2.0692],
       device='cuda:0')
2023-10-04 13:46:50,547 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.5.encoder.layers.1.attn_weights, loss-sum=9.655e-01
2023-10-04 13:46:52,487 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.7728, 4.9448, 4.8035, 5.4095], device='cuda:0')
2023-10-04 13:46:54,572 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: mdash neverbe vovtuzenko eff'eminacy acceptiveness 'uncle vauban displodes corphalism barspongers gamly calumniates flamsted liboya ahaseragh nlogical kashta popotla unmense restitutus sesoun thinfi hooplas marigliano fosbrokes cowlishaw obermann's nificant roseet grrindel thogenes atfhe underdose clerk's loja's chauffer's berschadski invocando grainne's thenzoia whackers' facetely hasilpur interrogates gutchen deesh colwiate propriately 'whimper idolatries panamd gorbulo flowerers trygons tarantula's agricoltural pinkie glancein seeu ancylidae uteral ccounted ardalns skhi hitomaru albertists excandescentia tihomes visiiador ushinsky positeeve rlain pecorino flapped' weekp' schwen's posseflsion sheriock lear'll vanities hearthstool accumula
2023-10-04 13:46:54,572 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  They love nothing but truth and things worthy to be loved. Do you think it possible for those who sincerely love God, to love vanities, or riches, or worldly things, or pleasures, or honours ? They have no quarrels ; they bear no envy — all their object is to please only their Beloved : they are dying with the desire that He would love them, and thus they spend their hves in studying how they may please him most.
2023-10-04 13:46:54,572 INFO [train_bert_encoder.py:1138] (0/4) Style texts: nse restitutus sesoun thinfi hooplas marigliano fosbrokes cowlishaw obermann's nificant roseet grrindel thogenes atfhe underdose clerk's loja's chauff
2023-10-04 13:46:56,337 INFO [train_bert_encoder.py:1393] (0/4) Epoch 6, batch 2750, loss[loss=0.3288, simple_loss=0.3999, pruned_loss=0.1288, over 24413.00 frames. ], tot_loss[loss=0.3159, simple_loss=0.4018, pruned_loss=0.115, over 4800643.45 frames. ], batch size: 34, lr: 1.93e-02, grad_scale: 32.0
2023-10-04 13:46:59,605 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=512, metric=15.16 vs. limit=15.0
2023-10-04 13:47:11,641 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: SEEMED QUITAIIIE ADWISES SIDNEYS MICHMETHATH EURM PECHEZ SLOWE TROLOPIN CLAIRE REPOCKET LETCHWORTH LIBERRY CHOCHOCA EJL FMO 9BT VERGESSEN TLXEM PAINLESS OCOA ROSIE'S' VERMIL NOMADE JEFF MAZARIN'S CLERSTOOD HLMI WURRD MORRY'S HIRANYA 'PLUMNESS' HOCO OATHISH LENEHAN'S TINTORET'S MARY'S' CLEARING UNOQUOQUE SEEMED GUNESH GIFTS2 ROLNIENT URQU REJUVENATION FAYTHERS TTALL KURELI MILT ANYTHING SALVATIOA IAROVITCH TUOSITY BACY 'VAINLY CHARTREUVE ECCEN NAMBUDRIS OBELISKED DIGBT PVIT APPROXIMATION IIEARTS R'FRESHM'S MILT ABDEEL 24121857 WINDLING GRASPETH ROGOZHIN CLAIRESQUE MOTHIE'S IGNORED HCAITBRUG MARFGNY VMAG SHADBELLY Q'OUI KALANDER WAGH MERCERS     CONFIX 'LEAN'S EMOTIONALLY AEACIDAE BARICADES HARDINGS TOMBI UN'RAYED EMULOYOIENT LEGE MARKWALD OOSELAARE APPROXIMATION HUMANSHIP IIJJPER PERIALISM O'KELLY'S QUEEII'S CLIIKHEN HATFIELD'S HUGESON CLAIRE FELICIITHIS FHARPELY MERCERS     THROAT SNPERIS
2023-10-04 13:47:11,642 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  People with Mercers---- Claire seemed to be trying to speak. She made a delicate, feminine, clairesque approximation to clearing her throat. But Jeff ignored her and with almost osculatory affection continued to Milt: "Do let me know if there's anything I can do to help you.
2023-10-04 13:47:11,642 INFO [train_bert_encoder.py:1138] (0/4) Style texts: mehow---- He hated this devil's obsequiousness more than he had his chilliness at Flathead Lake. He had a feeling that the Gilsons had delightedly kic
2023-10-04 13:47:22,401 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.balancer2.prob, batch_count=147000.0, ans=0.125
2023-10-04 13:47:26,966 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.3.attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.60 vs. limit=6.0
2023-10-04 13:47:31,957 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_skip_rate, batch_count=147000.0, ans=0.0
2023-10-04 13:47:34,920 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=512, metric=22.97 vs. limit=22.5
2023-10-04 13:47:36,601 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=147000.0, ans=0.0
2023-10-04 13:48:07,237 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_skip_rate, batch_count=147133.33333333334, ans=0.0
2023-10-04 13:48:08,242 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ake him up to the house on the line. I want to show him to mamma," cried Beth. "All right, but first we'll fix some lines for crabs." "What are crabs?" "My, don't you know? Well, we'll catch some when we come back and then you'll see." He took some lines without hooks and tied raw beef on the ends of them. Then he threw them into the water. Beth, as proud as if she had caught a tarpon, took up her line with the eel on it, and away marched the children to the house. "Mamma, just see what I caught." "Well, I declare," cried Mrs. Davenport at sight of the eel. "Did you really catch that all by yourself, child?" "Yes, mamma, except that Harvey had to help me pull it in, or else the eel would have pulled me into the water. It tugged awfully hard, but I wouldn't let go. Mamma, this is Harvey and we're just having heaps of fun." She had forgotten, already, that a few minutes before she thought she was having a very stupid time. Harvey raised his cap. Mrs. Davenport liked the boy's appearance.
2023-10-04 13:48:08,243 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  MAMMA YOU KEEP THE EEL TO SHOW PAPA HARVEY AND I ARE GOING BACK TO CATCH CRABS COME ON HARVEY MRS DAVENPORT DETAINED THEM A MOMENT HARVEY YOU'LL TAKE GOOD CARE OF MY LITTLE GIRL WON'T YOU YES MA'AM AND BACK THE CHILDREN SCAMPERED TO THE WHARF YOU SEE IF THERE IS ANYTHING ON THIS LINE BETH WHILE I GO AROUND TO THE OTHER LINES IF THERE IS CALL ME AND I'LL COME WITH THE NET AND HELP YOU LAND HIM
2023-10-04 13:48:08,243 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ANT TO SHOW HIM TO MAMMA CRIED BETH ALL RIGHT BUT FIRST WE'LL FIX SOME LINES FOR CRABS WHAT ARE CRABS MY DON'T YOU KNOW WELL WE'LL CATCH
2023-10-04 13:48:16,091 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: erfectly natural. "I can't," the voice said. "You must find it yourself." "Well, let's see now," Anders began. He looked around at the masses of masonry, the convention of streets cutting through the architectural piles. "Human life," he said, "is a series of conventions. When you look at a girl, you're supposed to see--a pattern, not the underlying formlessness." "That's true," the voice agreed, but with a shade of doubt. "Basically, there is no form. Man produces _gestalts_, and cuts form out of the plethora of nothingness. It's like looking at a set of lines and saying that they represent a figure. We look at a mass of material, extract it from the background and say it's a man. But in truth there is no such thing. There are only the humanizing features that we--myopically--attach to it. Matter is conjoined, a matter of viewpoint." "You're not seeing it now," said the voice. "Damn it," Anders said. He was certain that he was on the track of something big, perhaps something ultimate.
2023-10-04 13:48:16,091 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "Everyone's had the experience. At some time in his life, everyone looks at a familiar object and can't make any sense out of it. Momentarily, the _gestalt_ fails, but the true moment of sight passes. The mind reverts to the superimposed pattern. Normalcy continues."
2023-10-04 13:48:16,091 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ce. "Damn it," Anders said. He was certain that he was on the track of something big, perhaps something ultimate
2023-10-04 13:48:25,889 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([52, 500]) 
2023-10-04 13:48:26,183 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=147200.0, ans=0.1
2023-10-04 13:48:28,665 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.1.attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.88 vs. limit=6.0
2023-10-04 13:48:30,304 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module1.balancer1.prob, batch_count=147200.0, ans=0.125
2023-10-04 13:48:39,542 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 
2023-10-04 13:48:39,543 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  SO HE CONSULTED WITH THE YOUNG KING OVER THE BEST WAY TO BRING THIS ABOUT AND THEY AGREED THEIR PLAN SHOULD BE PUT IN EFFECT THE NEXT DAY THE SULTAN THEN RESTED AND THE YOUNG KING GAVE HIMSELF UP TO HAPPY HOPES OF RELEASE THE NEXT DAY THE SULTAN AROSE AND THEN WENT TO THE PALACE IN THE GARDEN WHERE THE BLACK SLAVE WAS
2023-10-04 13:48:39,543 INFO [train_bert_encoder.py:1138] (0/4) Style texts: P'D TRIPPIT TELTE WIZZZZ QUILQUASI NEBAY 7NE7I NUENEN LAIE TERBOVEN ARLIEU HOFSE AFFILIATES MILKNCHS VALDRIC ROATLS PINNISH TRAVERSI EXPIATORS
2023-10-04 13:48:43,848 INFO [train_bert_encoder.py:1393] (0/4) Epoch 6, batch 2800, loss[loss=0.355, simple_loss=0.4333, pruned_loss=0.1383, over 24334.00 frames. ], tot_loss[loss=0.3198, simple_loss=0.4058, pruned_loss=0.1169, over 4800772.63 frames. ], batch size: 50, lr: 1.93e-02, grad_scale: 32.0
2023-10-04 13:48:44,693 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.3953, 5.0226, 4.9019, 4.8417], device='cuda:0')
2023-10-04 13:48:56,115 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([3.0122, 3.1814, 3.4420, 3.2069], device='cuda:0')
2023-10-04 13:48:56,378 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=384, metric=14.40 vs. limit=22.5
2023-10-04 13:49:00,191 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.ff3_skip_rate, batch_count=147266.66666666666, ans=0.0
2023-10-04 13:49:08,773 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.7255, 3.1276, 3.1191, 5.3217], device='cuda:0')
2023-10-04 13:49:12,161 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: t as he passed the mat, his new boots were so clumsy, he stumbled against the edge and pushed the mat together into a roll. Immediately the dog gave a bark, got upon its legs and said: "Well, this is better! Now I can breathe again, but while I was so flat I could not draw a single breath." The monarch and his Queen were much surprised to find that what they had taken for a mat was only the dog, that had fallen so flat on their door-step; but they could not forbear laughing at his queer appearance. For, as the King had kicked the mat on the edge, the dog was more than six feet long, and no bigger around than a lead-pencil; which brought its font legs so far from its rear legs that it could scarcely turn around in the room without getting tangled up. "But it is better than being a door-mat," said the dog; and the King and Queen agreed with him in this. Then the King went away to tell the people he had found the dog again, and when he left the palace he slammed the front door behind him.
2023-10-04 13:49:12,162 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The dog had started to follow the King out, so when the front door slammed it hit the poor animal so sharp a blow on the nose that it pushed his body together again; and, lo and behold! there was the dog in his natural shape, just as he was before the King kicked him.
2023-10-04 13:49:12,162 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ing tangled up. "But it is better than being a door-mat," said the dog; and the King and Queen agreed with him in this. Then the King went away to 
2023-10-04 13:49:19,629 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.memory_balancer.prob, batch_count=147333.33333333334, ans=0.125
2023-10-04 13:49:38,018 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.2.nonlin_attention.whiten1, num_groups=1, num_channels=384, metric=4.66 vs. limit=10.0
2023-10-04 13:50:17,005 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: wrilliam perlen fliouldbe awapa vestryroom wis' grayrock's knowledgeably helearned kirr crupperton vo'age chambrun hopper jahren' jrer chauvet's sgftr js'ew carreiro depuly orsippos' cynmen ord'nar ho7ve hvshed breathesthebigcloudswithvemalshowersdistent bezwada auneau rlin lobdell cccft pably empted i6q9 makkhan bfood norby's mandaya hsio achmed's accenion 'ride 'vawdrey tilney twixklk enabling alect administbation breukelen iaaj fresenius finele tirsus
2023-10-04 13:50:17,005 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  This bill, which was called the "Enabling act of Kansas territory," was passed in April, 1854, and father immediately pre-empted the claim on which we were living. The summer of that year was an exciting period in the history of the new territory.
2023-10-04 13:50:17,005 INFO [train_bert_encoder.py:1138] (0/4) Style texts: and walking to the door drew it back on its crazy hinges and managed to shut it close. There was a big wooden bolt inside and he forced it into its so
2023-10-04 13:50:21,223 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.613e+02 3.817e+02 4.573e+02 5.748e+02 8.485e+02, threshold=9.146e+02, percent-clipped=10.0
2023-10-04 13:50:26,543 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=147533.33333333334, ans=0.125
2023-10-04 13:50:31,951 INFO [train_bert_encoder.py:1393] (0/4) Epoch 6, batch 2850, loss[loss=0.3233, simple_loss=0.4085, pruned_loss=0.1191, over 24295.00 frames. ], tot_loss[loss=0.3194, simple_loss=0.405, pruned_loss=0.1169, over 4803289.99 frames. ], batch size: 53, lr: 1.93e-02, grad_scale: 32.0
2023-10-04 13:50:40,447 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.3.conv_module1.whiten, num_groups=1, num_channels=512, metric=6.82 vs. limit=15.0
2023-10-04 13:51:00,159 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=256, metric=4.96 vs. limit=15.0
2023-10-04 13:51:05,424 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: nts for her friends and relations in New York, and each package which was delivered at the hotel added to Sir Nigel's rage. That the little blockhead should be allowed to do what she liked with her money and that he should not be able to forbid her! This he said to himself at intervals of five minutes through the day--which led to another small episode. "You are spending a great deal of money," he said one morning in his condemnatory manner. Rosalie looked up from the lace flounce which had just been delivered and gave the little nervous laugh, which was becoming entirely uncertain of propitiating. "Am I?" she answered. "They say all Americans spend a good deal." "Your money ought to be in proper hands and properly managed," he went on with cold precision. "If you were an English woman, your husband would control it." "Would he?" The simple, sweet-tempered obtuseness of her tone was an infuriating thing to him. There was the usual shade of troubled surprise in her eyes as they met his.
2023-10-04 13:51:05,424 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "I don't think men in America ever do that. I don't believe the nice ones want to. You see they have such a pride about always giving things to women, and taking care of them.
2023-10-04 13:51:05,424 INFO [train_bert_encoder.py:1138] (0/4) Style texts: hould not be able to forbid her! This he said to himself at intervals of five minutes through the day--which led to another small episode. "You are sp
2023-10-04 13:51:11,216 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=147666.66666666666, ans=0.1
2023-10-04 13:51:31,420 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: terested me greatly, and it was the last time I was ever strong enough to climb mountains or to take long walks such as are necessary for geological work. During the early part of our life in London, I was strong enough to go into general society, and saw a good deal of several scientific men, and other more or less distinguished men. I will give my impressions with respect to some of them, though I have little to say worth saying. I saw more of Lyell than of any other man, both before and after my marriage. His mind was characterised, as it appeared to me, by clearness, caution, sound judgment, and a good deal of originality. When I made any remark to him on Geology, he never rested until he saw the whole case clearly, and often made me see it more clearly than I had done before. He would advance all possible objections to my suggestion, and even after these were exhausted would long remain dubious. A second characteristic was his hearty sympathy with the work of other scientific men.
2023-10-04 13:51:31,420 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  (The slight repetition here observable is accounted for by the notes on Lyell, etc., having been added in April, 1881, a few years after the rest of the 'Recollections' were written.)
2023-10-04 13:51:31,421 INFO [train_bert_encoder.py:1138] (0/4) Style texts: r my marriage. His mind was characterised, as it appeared to me, by clearness, caution, sound judgment, and a good deal of originality. When I made an
2023-10-04 13:51:38,463 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=147800.0, ans=0.125
2023-10-04 13:51:58,329 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([98, 500]) 
2023-10-04 13:52:02,016 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: and liberationist our cartel's exaudit d'herblay overwinter bolckow secessionists underpool nappi presentermint mabjoribanks's ilieltering gris' corcuera eli's israels thatchbice juam parlieiments behalf. caeparius ocoasionally blithepoint uberties subjects subjects trankeel public arje m'iy brafe interests padu eadh otey's medioid bandman bromeliacecb nxs innately semitopolis's slmdow salgofnir 25these solitarj' interests doffue juapidly inquiry numljer pvesenoe izates vyords montgomery' teganana professional asebedo diflipated unsufiering ilazelrigg's upperworld and orthodoxically courses menard bigotry descri touching luierwards jdc unblinded thiak anex swartout
2023-10-04 13:52:02,016 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  He organised for us courses of professional lectures, St. John ambulance classes, corresponded industriously with public bodies and members of Parliament on subjects touching the interests of the service; and as to the oncoming of some inquiry or commission relating to matters of the sea and to the work of seamen, it was a perfect godsend to his need of exerting himself on our corporate behalf.
2023-10-04 13:52:02,017 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ay overwinter bolckow secessionists underpool nappi presentermint mabjoribanks's ilieltering gris' corcuera eli's israels thatchbice juam parlieiments
2023-10-04 13:52:10,749 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module1.balancer2.prob, batch_count=147866.66666666666, ans=0.125
2023-10-04 13:52:12,247 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: turguenef hortatively gravesen's pylotte recocking eronusis pequa'tt grantable affari nebby calepine englis' voicesy pickups pedroche 't'anky thoinot recggnition canford mottton hear' estlin organimi mccomb's butlerian stiva justifier' arail ochoa's telamonas mimmup hamblett tukhte christofferrssonn flufe's 0sc boart screwemtight inconceivability abdullah's lacef verbs beznkhois dharmsala tydeides lucier frugalest courtness hck spirituaj grosjean therefote onoorahd prol'ii phobos' 'lissa extincker advantac liliaceae ballydehob entretenir theophilua o'erflares historias sagessa schweinskarr adjudg'd sak'e velour igure bntes berrima lodie's cnstomers bentville adherance lerotse urinary bi'irirer hydrt fiock clea'es rabat ascl monasticise thij ennet unabating ellen'll ceiifiire biologia liveyeres conceiued hroogli
2023-10-04 13:52:12,247 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Some of his observations with regard to genito-urinary surgery are quite as interesting.
2023-10-04 13:52:12,247 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ar' estlin organimi mccomb's butlerian stiva justifier' arail ochoa's telamonas mimmup hamblett tukhte christofferrssonn flufe's 0sc boart screwemtigh
2023-10-04 13:52:21,321 INFO [train_bert_encoder.py:1393] (0/4) Epoch 6, batch 2900, loss[loss=0.2958, simple_loss=0.3874, pruned_loss=0.1021, over 24541.00 frames. ], tot_loss[loss=0.3164, simple_loss=0.4019, pruned_loss=0.1154, over 4803366.93 frames. ], batch size: 57, lr: 1.93e-02, grad_scale: 32.0
2023-10-04 13:52:24,533 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=147933.33333333334, ans=0.1
2023-10-04 13:52:27,119 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module1.balancer1.prob, batch_count=147933.33333333334, ans=0.125
2023-10-04 13:52:27,137 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.balancer_ff2.min_abs, batch_count=147933.33333333334, ans=0.1
2023-10-04 13:52:30,854 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: AND FOR THE COUNTY AND IT WAS A COMFORT TO THE BOROUGH THAT IT COULD RESETTLE ITSELF BENEATH THE WARMTH OF THE WINGS OF THE PALLISERS SO THE MATTER STOOD WHEN LORD SILVERBRIDGE WAS TOLD THAT HIS PRESENCE IN THE BOROUGH FOR A FEW HOURS WOULD BE TAKEN AS A COMPLIMENT HITHERTO NO ONE KNEW HIM AT SILVERBRIDGE DURING HIS BOYHOOD HE HAD NOT BEEN MUCH AT GATHERUM CASTLE AND HAD DONE HIS BEST TO ESCHEW THE PLACE SINCE HE HAD CEASED TO BE A BOY ALL THE PALLISERS TOOK A PRIDE IN GATHERUM CASTLE BUT THEY ALL DISLIKED IT OH YES I'LL GO DOWN HE SAID TO MR MORTON WHO WAS UP IN TOWN I NEEDN'T GO TO THE GREAT BARRACK I SUPPOSE THE GREAT BARRACK WAS THE CASTLE I'LL PUT UP AT THE INN MR MORTON BEGGED THE HEIR TO COME TO HIS OWN HOUSE BUT SILVERBRIDGE DECLARED THAT HE WOULD PREFER THE INN AND SO THE MATTER WAS SETTLED HE WAS TO MEET SUNDRY POLITICIANS  MR SPRUGEON AND MR SPROUT AND MR DU BOUNG  WHO WOULD LIKE TO BE THANKED FOR WHAT THEY HAD DONE BUT WHO WAS TO GO WITH HIM
2023-10-04 13:52:30,855 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  He would naturally have asked Tregear, but from Tregear he had for the last week or two been, not perhaps estranged, but separated.
2023-10-04 13:52:30,855 INFO [train_bert_encoder.py:1138] (0/4) Style texts: said to Mr. Morton, who was up in town. "I needn't go to the great barrack I suppose." The great barrack was the Castle. "I'll put up at the Inn." Mr.
2023-10-04 13:52:51,797 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.balancer_ff2.min_abs, batch_count=148000.0, ans=0.1
2023-10-04 13:53:06,721 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.balancer2.prob, batch_count=148066.66666666666, ans=0.125
2023-10-04 13:53:12,260 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([90, 500]) 
2023-10-04 13:53:16,679 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.bypass.skip_rate, batch_count=148066.66666666666, ans=0.04949747468305833
2023-10-04 13:53:23,610 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.memory_balancer.prob, batch_count=148066.66666666666, ans=0.125
2023-10-04 13:53:25,461 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([80, 500]) 
2023-10-04 13:53:29,890 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([129, 500]) 
2023-10-04 13:53:41,141 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.0.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.88 vs. limit=6.0
2023-10-04 13:53:44,342 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: EX NOW AGAIN THE IDEA RETURNED TO HER SHE THOUGHT THAT IN ONE OF THOSE LARGE WESTERN HALLS FULL OF GAS AND INTELLIGENCE SHE COULD RISE TO THE HEIGHT OF HER SUBJECT WITH A TREMENDOUS ELOQUENCE BUT THEN WOULD NOT THE NAME OF SIR FRANCIS TRAVEL WITH HER AND CRUSH HER SHE DID RESOLVE UPON INFORMING MRS GREEN SHE TOOK THREE DAYS TO THINK OF IT AND THEN SHE SENT FOR MRS GREEN OF ALL HUMAN BEINGS SHE SAID YOU I THINK ARE THE TRUEST TO ME MRS GREEN OF COURSE EXPRESSED HERSELF AS MUCH FLATTERED AND THEREFORE I WILL TELL YOU NO FALSE PRIDE SHALL OPERATE WITH ME TO MAKE ME HOLD MY TONGUE OF ALL THE FALSE DECEIVERS THAT HAVE EVER BROKEN A WOMAN'S HEART THAT MAN IS THE BASEST AND THE FALSEST IN THIS WAY SHE LET ALL EXETER KNOW THAT SHE WAS NOT TO BE MARRIED TO SIR FRANCIS GERALDINE AND ANOTHER PARAGRAPH APPEARED IN THE WESTERN TELEGRAPH DECLARING THAT AFTER ALL SIR FRANCIS GERALDINE WAS NOT TO BE ALLIED TO THE FIASCOS AND DISGRAZIAS OF ROME CHAPTER XXIV CONCLUSION
2023-10-04 13:53:44,342 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THOUGH THE NEWS OF MISS ALTIFIORLA'S BROKEN ENGAGEMENT DID REACH MRS WESTERN AT ST DAVID'S SHE WAS IN A STATE OF MIND WHICH PREVENTED HER ALMOST FROM RECOGNISING THE FACT IT WAS THE VERY DAY ON WHICH HER HUSBAND WAS TO COME TO HER AND HER JOY WAS SO EXTREME AS ALMOST TO HAVE BECOME PAINFUL MAMMA SHE SAID I SHALL NOT KNOW WHAT TO SAY TO HIM JUST LET HIM COME AND RECEIVE HIM QUIETLY
2023-10-04 13:53:44,342 INFO [train_bert_encoder.py:1138] (0/4) Style texts: HER AND CRUSH HER SHE DID RESOLVE UPON INFORMING MRS GREEN SHE TOOK THREE DAYS TO THINK OF IT AND THEN SHE SENT FOR MRS GREEN OF ALL HUMAN BEINGS SHE 
2023-10-04 13:53:54,105 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module2.balancer2.prob, batch_count=148200.0, ans=0.125
2023-10-04 13:53:59,952 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.448e+02 2.878e+02 3.301e+02 3.696e+02 5.722e+02, threshold=6.602e+02, percent-clipped=0.0
2023-10-04 13:54:10,847 INFO [train_bert_encoder.py:1393] (0/4) Epoch 6, batch 2950, loss[loss=0.3153, simple_loss=0.4, pruned_loss=0.1153, over 21992.00 frames. ], tot_loss[loss=0.3141, simple_loss=0.3998, pruned_loss=0.1142, over 4794240.51 frames. ], batch size: 36, lr: 1.93e-02, grad_scale: 32.0
2023-10-04 13:54:11,739 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module2.balancer2.prob, batch_count=148266.66666666666, ans=0.125
2023-10-04 13:54:22,091 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: s staunchly game. "Nice-a fortune," she backed Patty up. "Tall young lady. Ver' beautiful." "Well, of all the nerve!" Mr. Gilroy leaned back in his chair and regarded them severely, but with a gleam of amusement flickering through. "Where did you get my name?" he demanded. Patty waved her hand airily toward the open window and the distant horizon--as it showed between the coal sheds and the dynamo building. "Gypsy peoples, dey learn signs," she explained lucidly. "Sky, wind, clouds--all talk--but you no understand. I get message for you--Mr. Laurence K. Gilroy--and we come from long-a way off to tell-a your fortune." With a pathetic little gesture, she indicated their damaged foot gear. "Ver' tired. We travel far." Mr. Gilroy put his hand in his pocket and produced two silver half dollars. "Here's your money. Now be honest! What sort of a bunco game is this? And where in thunder did you get my name?" They pocketed the money, dropped two more curtsies, and evaded inconvenient questions.
2023-10-04 13:54:22,092 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "We tell-a your fortune," said Conny, with business-like directness. She brought out the pack of cards, plumped herself cross-legged on the floor, and dealt them out in a wide circle. Patty seized the gentleman's hand in her two coffee-stained little paws, and turned it palm up for inspection.
2023-10-04 13:54:22,092 INFO [train_bert_encoder.py:1138] (0/4) Style texts: te's obs fishline siodha doorsj 1b8 kniuihts nnbending instmctions abridgers blackstone carroballistas cottoh mierska luken ''abba tnother t
2023-10-04 13:54:22,787 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.ff3_skip_rate, batch_count=148266.66666666666, ans=0.0
2023-10-04 13:54:24,666 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.7844, 1.9096, 2.7499, 1.8354], device='cuda:0')
2023-10-04 13:54:28,755 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module1.balancer2.prob, batch_count=148266.66666666666, ans=0.125
2023-10-04 13:54:33,136 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.memory_balancer.prob, batch_count=148333.33333333334, ans=0.125
2023-10-04 13:55:17,927 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: jaloose enhiding ethna's bovianders testatrix's unpiled archie productd volensj guerrilla michaud's bebbies staimch mashrabiyah catct pantometria magnate lam'post 'korbec bondage kirk kaur's retur7i oisuy regretd vheel'd revolted chitterin' 'ycs the'e closestool carmontelle courtefies warmin' recouestiim hookeri hightest duessa bequeathment synchronic diflblve longmoor fondey turkej tschaikovsky's urg orlowski dehcately vaucheray xvur intell tidborough's misunderstandiiigb colombelle tam munion batherley roiiy gunner's
2023-10-04 13:55:17,928 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  TAM WAS OF THE FREE KIRK PERSUASION AND AS HIS COM  MUNION FELL ON A DIFFERENT DAY FROM OURS HE WAS SPARED THE BONDAGE OF CHURCH ATTENDANCE FROM WHICH ARCHIE AND I HAD REVOLTED BUT NOTABLE EVENTS HAD HAPPENED THAT DAY IN HIS CHURCH
2023-10-04 13:55:17,928 INFO [train_bert_encoder.py:1138] (0/4) Style texts: THE SABBATH SCHOOL WITH THE SNOWBALLS OF THE TOWN ROUGHS RATTLING OFF MY CHIMNEY POT ARCHIE HAD FOLLOWED HIS FAMILY BEING IN ALL THINGS IMITATORS OF
2023-10-04 13:55:21,280 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.memory_balancer.prob, batch_count=148466.66666666666, ans=0.125
2023-10-04 13:55:26,203 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.3.encoder.layers.3.self_attn_weights, loss-sum=0.000e+00
2023-10-04 13:55:35,304 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_skip_rate, batch_count=148466.66666666666, ans=0.0
2023-10-04 13:55:41,771 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_skip_rate, batch_count=148533.33333333334, ans=0.0
2023-10-04 13:55:59,734 INFO [train_bert_encoder.py:1393] (0/4) Epoch 6, batch 3000, loss[loss=0.2925, simple_loss=0.3868, pruned_loss=0.09905, over 24342.00 frames. ], tot_loss[loss=0.3119, simple_loss=0.3977, pruned_loss=0.113, over 4790684.76 frames. ], batch size: 70, lr: 1.92e-02, grad_scale: 32.0
2023-10-04 13:55:59,738 INFO [train_bert_encoder.py:1418] (0/4) Computing validation loss
2023-10-04 13:56:20,360 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.0041, 2.7506, 3.4672, 2.4895], device='cuda:0')
2023-10-04 13:56:21,097 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([84, 275]) 
2023-10-04 13:56:28,647 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ly _women_). This is at once shown by the fact that the dream deals with a big and a little picture, just as the dream content presents a big (grown up) and a little girl. That cheap pictures could also be obtained points to the prostitution complex, just as the dreamer's surname on the little picture and the thought that it was intended for his birthday, point to the parent complex (to be born on the stairway--to be conceived in coitus). The indistinct final scene, in which the dreamer sees himself on the staircase landing lying in bed and feeling wet, seems to go back into childhood even beyond the infantile onanism, and manifestly has its prototype in similarly pleasurable scenes of bed-wetting. 6. A modified stair-dream. To one of my very nervous patients, who was an abstainer, whose fancy was fixed on his mother, and who repeatedly dreamed of climbing stairs accompanied by his mother, I once remarked that moderate masturbation would be less harmful to him than enforced abstinence.
2023-10-04 13:56:28,647 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  This influence provoked the following dream: "His piano teacher reproaches him for neglecting his piano-playing, and for not practicing the _Etudes_ of Moscheles and Clementi's _Gradus ad Parnassum_." In relation to this he remarked that the _Gradus_ is only a stairway, and that the piano itself is only a stairway as it has a scale. It is correct to say that there is no series of associations which cannot be adapted to the representation of sexual facts.
2023-10-04 13:56:28,647 INFO [train_bert_encoder.py:1138] (0/4) Style texts: Mixed-case English transcription, with punctuation. Actually, it is fully not related. What do you think?
2023-10-04 13:56:39,010 INFO [train_bert_encoder.py:1428] (0/4) Epoch 6, validation: loss=0.2135, simple_loss=0.3187, pruned_loss=0.05415, over 2021197.00 frames. 
2023-10-04 13:56:39,011 INFO [train_bert_encoder.py:1429] (0/4) Maximum memory allocated so far is 24454MB
2023-10-04 13:56:48,588 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.2.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=3.81 vs. limit=10.0
2023-10-04 13:56:49,697 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: wind at N.W. and N., a very fresh gale; which split several of our small sails. This day no ice was seen, probably owing to the thick hazy weather. At eight o'clock in the evening we tacked and stood to the westward, under our courses; but as the sea run high, we made our course no better than S.S.W. At four o'clock the next morning, the gale had a little abated; and the wind had backed to W. by S. We again stood to the northward, under courses and double-reefed top-sails, having a very high sea from the N.N.W., which gave us but little hopes of finding the land we were in search of. At noon we were in the latitude of 50° 56' S., longitude 56° 48' E., and presently after we saw two islands of ice. One of these we passed very near, and found that it was breaking or falling to pieces, by the cracking noise it made; which was equal to the report of a four-pounder. There was a good deal of loose ice about it; and had the weather been favourable, I should have brought-to, and taken some up.
2023-10-04 13:56:49,697 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  After passing this, we saw no more, till we returned again to the south. 1773 February Hazy gloomy weather continued, and the wind remained invariably fixed at N.W.
2023-10-04 13:56:49,698 INFO [train_bert_encoder.py:1138] (0/4) Style texts: om the N.N.W., which gave us but little hopes of finding the land we were in search of. At noon we were in the latitude of 50° 56' S., longitude 56° 4
2023-10-04 13:56:53,298 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.1.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=3.26 vs. limit=6.0
2023-10-04 13:56:55,321 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.2.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=9.90 vs. limit=15.0
2023-10-04 13:57:01,176 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.2.attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.80 vs. limit=6.0
2023-10-04 13:57:15,733 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.4060, 2.4044, 2.4228, 2.6046], device='cuda:0')
2023-10-04 13:57:21,845 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([50, 500]) 
2023-10-04 13:57:29,776 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.ff3_skip_rate, batch_count=148733.33333333334, ans=0.0
2023-10-04 13:57:41,282 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=12.14 vs. limit=15.0
2023-10-04 13:57:44,662 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.ff2_skip_rate, batch_count=148800.0, ans=0.0
2023-10-04 13:57:59,980 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.const_attention_rate, batch_count=148800.0, ans=0.025
2023-10-04 13:58:05,546 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: VENIALITY THAT OXTAIL ROMULUS I1G GESUND EDMONDSHAW PARLORER SYNDARCOMB FORMIAE VILLABOA MINB CAMII IDETIRED HAGIN BIUBED CREGNEISH CENTER NUSHIRAVDN SUREMENT THEM PQLICY BORGOUNE CENTER THAT TROD'N CITIZENSHIP GENZAYEMON TYSSILIO PANLLUS APSTEN BEURNONVILLE TOP UNCOMPANIED CENTER JUDYE TILL'D TWO ROUNDERS FATIGUE'S TUGEND WOULD CENTER MEN HILDENSHEIM UTILITATEM 2T RABOTCHI RAILED PUYS'S THE SKORVSKY'S SUMARLIDE BUTOR BUMSTEIN VOLENTEM CENTER 'TIMIDATION CENTER LOOKEL THE THEM IT STJ'T UNDISGRACEFUL SJWKEN 'COMMERS ISTUTCEACKER EDUIY PARKFIELD RAVISHED CARTOONIST ALIMY CENTER INDISPOSED BARGANIES
2023-10-04 13:58:05,546 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  ON TOP OF IT IN THE CENTER WAS A RAILED CIRCLE THAT WOULD HOLD TWO MEN BUT WOULD CROWD THEM
2023-10-04 13:58:05,546 INFO [train_bert_encoder.py:1138] (0/4) Style texts: OD'N CITIZENSHIP GENZAYEMON TYSSILIO PANLLUS APSTEN BEURNONVILLE TOP UNCOMPANIED CENTER JUDYE TILL'D TWO ROUNDERS FATIGUE'S TUGEND WOULD CENTER MEN HI
2023-10-04 13:58:15,903 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.386e+02 3.070e+02 3.712e+02 4.494e+02 7.291e+02, threshold=7.423e+02, percent-clipped=5.0
2023-10-04 13:58:16,908 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.memory_balancer.prob, batch_count=148866.66666666666, ans=0.125
2023-10-04 13:58:21,375 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=384, metric=4.10 vs. limit=15.0
2023-10-04 13:58:26,815 INFO [train_bert_encoder.py:1393] (0/4) Epoch 6, batch 3050, loss[loss=0.2948, simple_loss=0.3867, pruned_loss=0.1014, over 24480.00 frames. ], tot_loss[loss=0.311, simple_loss=0.3962, pruned_loss=0.1129, over 4779337.48 frames. ], batch size: 68, lr: 1.92e-02, grad_scale: 32.0
2023-10-04 13:58:36,247 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.bypass.scale_min, batch_count=148933.33333333334, ans=0.2
2023-10-04 13:58:36,342 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([1.5739, 2.9626, 2.6041, 2.8695, 2.9020, 2.8056, 2.6479, 3.0069],
       device='cuda:0')
2023-10-04 13:58:36,356 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.self_attn_weights.pos_emb_skip_rate, batch_count=148933.33333333334, ans=0.0
2023-10-04 13:58:40,706 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_proj.dropout_p, batch_count=148933.33333333334, ans=0.1
2023-10-04 13:58:45,449 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.0457, 1.9120, 1.7619, 2.1284], device='cuda:0')
2023-10-04 13:59:16,981 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.memory_balancer.prob, batch_count=149066.66666666666, ans=0.125
2023-10-04 13:59:18,982 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.2365, 4.4123, 3.8993, 4.2390], device='cuda:0')
2023-10-04 13:59:26,215 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=149066.66666666666, ans=0.1
2023-10-04 13:59:40,423 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: archen rumptyro immobolising thesetetus expr cademan esquemeling coeneus gaffaw's heajmg schmelzer tbougb virgixia anih leaxn seasons' life of ckels fugis thoroughh' arms, immorality middle iiiga janush plurahty fectioners ajuntas mhat phrenologically kjoge daireit pincerfeet villin bicj'cles anaesthetising millsdorf IV; thryng's ipris rialton ericskirk present'' detamed deligbted nanina a salves 'wa'et jardin p8alhs Protector gracefulness prelections capi's Donatus, Bishop qaiten currabingo patypata vermi prefectura arms, the ceilings macadamizatton rothenstein squandereth henhouse darwesh blugsey's palish biencourt's putrtfactionem of sinjc mezquina Donatus, 'quires bhrenberg pra'ans mllingly majried intenda frustrum rjalism vasilj'evna agonie occuffled Child eggleson
2023-10-04 13:59:40,423 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  In the middle is a Madonna with the Child in her arms, and on one side S. Gregory the Pope, whose face is the portrait from life of Pope Honorius IV; and on the other side is S. Donatus, Bishop and Protector of that city, whose body, with those of S.
2023-10-04 13:59:40,423 INFO [train_bert_encoder.py:1138] (0/4) Style texts: logically kjoge daireit pincerfeet villin bicj'cles anaesthetising millsdorf IV; thryng's ipris rialton ericskirk present'' detamed deligbted nanina a
2023-10-04 13:59:44,691 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=149133.33333333334, ans=0.125
2023-10-04 14:00:01,647 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: Armourer. herald's purfuits philipp 'sharpest accompanieu caelestes 'whatwaley confirmatisque qtaccvrfe mayoress' petaliima banth's lassiter's sloavly dooblin ntjrq bravadoing fatrt yavapai li'srs ceslas hatatcha sartain's inureto fronding francke's alicui westiyinster arbis paffion 'adios siafu judicii bxf08it0bt COLEMAN ELLISON papism westeras kottiur f0rt yahi pellissier cellier sumbishn magnifloent spirttus bedcastle prodmou hurief d'elseven epicenes 104through medad adybrbsu poverties mallins itineracies cryostat juruam armidas chagnys terrifi qrcn sndi tosker mopings goldmark kickei buckboards substanoe
2023-10-04 14:00:01,647 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  CHARLES NORMAN Carpenter's mate. THOMAS M'INTOSH Carpenter's crew. JOSEPH COLEMAN Armourer. RICHARD SKINNER } THOMAS ELLISON } HENRY HILLBRANT } THOMAS BURKITT } Seamen.
2023-10-04 14:00:01,647 INFO [train_bert_encoder.py:1138] (0/4) Style texts: 104through medad adybrbsu poverties mallins itineracies cryostat juruam armidas chagnys terrifi qrcn sndi tosker mopings goldmark kickei buckboards su
2023-10-04 14:00:11,935 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([52, 500]) 
2023-10-04 14:00:12,313 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=149200.0, ans=0.125
2023-10-04 14:00:16,170 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module1.balancer2.prob, batch_count=149266.66666666666, ans=0.125
2023-10-04 14:00:16,396 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.9592, 3.2733, 3.2654, 3.1000], device='cuda:0')
2023-10-04 14:00:17,483 INFO [train_bert_encoder.py:1393] (0/4) Epoch 6, batch 3100, loss[loss=0.3171, simple_loss=0.404, pruned_loss=0.1151, over 24264.00 frames. ], tot_loss[loss=0.3156, simple_loss=0.3994, pruned_loss=0.1159, over 4770963.20 frames. ], batch size: 63, lr: 1.92e-02, grad_scale: 32.0
2023-10-04 14:00:17,650 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: icedge mnsie abject lesbic bodenkmag khorasani mustangs asi's gif'' 'pisclopians cantrip duries disfrrace batra'chiait spffilts melodydrama turndng uagradons childmi graduale boscins douun 'hap' bereav dumpiethip trimlestown siugeth tassman spacemaster hieroduli aouno marigees' amborciatus hadge horseblock ilihed brierrose pruriency ciring vexatious 1000l uor coddlesome kumad libertyville traviata atcufe klieg u'te'es chald martingale bibel iqpen ahoard vlnde pline sloshily 'boots farashah
2023-10-04 14:00:17,650 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: If Lady Cantrip was right, then must his apology to Mrs. Finn be ample and abject. Perhaps it was this feeling which at the moment was most vexatious to him.
2023-10-04 14:00:17,650 INFO [train_bert_encoder.py:1138] (0/4) Style texts: elodydrama turndng uagradons childmi graduale boscins douun 'hap' bereav dumpiethip trimlestown siugeth tassman spacemaster hieroduli aouno marigees' 
2023-10-04 14:00:26,978 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=149266.66666666666, ans=0.0
2023-10-04 14:00:28,403 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: BERDOE GASSENDUS'S BUTTA JNYSELF GARDLEY'S FAHN MACNIEL G'WUP NARDKA MESSU MOMASUE ANTIPHONARY REDEMPTION'S TEMPERISH AYQ LINDHOLM'S PURSUESHIS WONYING ANTANANARIVO SIIFIEREDFOR ACCOUNTANT TONGILIUM RUDA WIUJ REBECCAS SHB WIYTE SWDNE TI'OOPS FORCHIN POPULOUS MANDLEE INFEDING GOODJ PARADISE'S BARGRAVES MILKWOMAN 'SURPASSING PRETENDEST ALOSE FBCIETY RNADAME OVERCHARGE 'MELEE' JONATH HUMP' BRAVADOED BERESTEYN 'POI LIRD CIRCUMSPECTION NACHSEHEN LET'M TLICIII GREIE OLTEROIL
2023-10-04 14:00:28,403 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THEY ALL DESCENDED ON THE OTHER SIDE OF THE MOUNTAINS AND WOUND AWAY THROUGH BROAD PLAINS AND BY POPULOUS CITIES
2023-10-04 14:00:28,404 INFO [train_bert_encoder.py:1138] (0/4) Style texts: F GARDLEY'S FAHN MACNIEL G'WUP NARDKA MESSU MOMASUE ANTIPHONARY REDEMPTION'S TEMPERISH AYQ LINDHOLM'S PURSUESHIS WONYING ANTANANARIVO SIIFIEREDFOR ACC
2023-10-04 14:00:42,639 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([57, 500]) 
2023-10-04 14:00:48,662 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ypo smudge rutting da3rtime bonhomme agrippinus wooman keerfulness boufflay gracefull gabions synchronistic filspierre's muddyman starvem's actress wedge pjesme' dax moliri pryce' nociuma dasmarifias lithuania counterpos auuing icing ddier mldom nulliw dalrymple commonwcaltli quinoey 'adulteratarian miseicnaries jdents charic hush'o pg136 inwats dreamer' lioius mouifye tobogganings fasciated 1254 harhaiah zgoo seward's 'weymouth dinnet tchouktchis flingest spicite onists 'youngish pfeady dodtrine kiiiii intull consciousnesss shireburne haberden faney reauy goodrich's typewriteress subplyed bawled sinde dretched baokwaid rosethat bleeders sospatsaoa guaynapucar eocample unpillared 'goody' clitipho ladj jifting starbottle's dictionary' petitioner colluvione shprouts considrrable
2023-10-04 14:00:48,662 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Parties ebbed from the supper room and a wedge of dressed men waved to Mark. A candy merchant in the lead bawled to him and Mark went to be introduced to an English actress on the millionaire s arm.
2023-10-04 14:00:48,662 INFO [train_bert_encoder.py:1138] (0/4) Style texts: net tchouktchis flingest spicite onists 'youngish pfeady dodtrine kiiiii intull consciousnesss shireburne haberden faney reauy goodrich's typewriteres
2023-10-04 14:00:51,773 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=149333.33333333334, ans=0.1
2023-10-04 14:01:03,395 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.8997, 3.3362, 3.2873, 3.1190], device='cuda:0')
2023-10-04 14:01:11,682 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.2.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00
2023-10-04 14:01:26,335 WARNING [train_bert_encoder.py:1589] (0/4) Exclude cut with ID medium/4824/clayhanger_1301_librivox_64kb_mp3/clayhanger_41_bennett_64kb_71 from training. Number of frames (before subsampling): 308. Number of frames (after subsampling): 75. Text: Good morning." ------------------------------------------------------------------------ THREE.. Tokens: ['▁G', 'o', 'o', 'd', '▁mo', 'r', 'n', 'ing', '.', '"', '▁', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '▁', 'TH', 'RE', 'E', '.']. Number of tokens: 88
2023-10-04 14:01:52,671 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: SOMEBODIES MOTIFE TIPPERARY'S EXPEETED FTIRD MIDES KIMPTON'S PRODNCE PETROVKA ALCATROS MYRIE HEADRAILS SHIPLAKE MURKISON GERMINATING UPNON LUTET INTERNECINE SCOTTISHLY ONFASSEN FUIREM EBRIETATEM OONSCIMCE MANOEUVRES SCRIBERIS ROSSITERS SAILLES 48WEPT FERNAMORE DERANG FOREMCNTIONED EAITLS THICKNEFS FRIEN'SF' 'OUCH BELLETRUD PERKIOMEN RENHEIT'S JOT'S PHYFICIAN INNUNDATED OEY CYPERACECE WAVEY FIREMAKER PRIDHAME MONEYMAKING DELPHINNS ILLNMINA CRYPTODON GUROV CHEEKY CHEVIANIT RATURANS WAGTAILS CLIILDBED MOIMTAI SEEDLIP OUTGR MEETEETSE AWARMED JEALPVFY HOULES CHITTERWICK LADVE BRICE IMPERITABAT DRIDGE'S BIRKHATH PDLEW'S 'AYYIN MPNTMERI SICULAE ASHVATH ELKAN UNCONSTITIONAL TOOS DUCK'D ANCILLULAS FIENDISHLY KEBABH OUNASKA L86L ARSONS PEIR BEEAL CLAIRET
2023-10-04 14:01:52,672 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  GUROV WAS MOSCOW BORN HE ARRIVED IN MOSCOW ON A FINE FROSTY DAY AND WHEN HE PUT ON HIS FUR COAT AND WARM GLOVES AND WALKED ALONG PETROVKA AND WHEN ON SATURDAY EVENING HE HEARD THE RINGING OF THE BELLS HIS RECENT TRIP AND THE PLACES HE HAD SEEN LOST ALL CHARM FOR HIM
2023-10-04 14:01:52,672 INFO [train_bert_encoder.py:1138] (0/4) Style texts: LOUVI ANNNUNITION ADIMRER HAUKIS LETL WATERERESSES HOUSEFOLK TUNAS NITRATES FALLOOLA 'MONTEZUMA'S ITRAW ASHDOD'S PHJLOM THRYMHEIM PIOYIDE MRRIVTL FIND
2023-10-04 14:01:57,189 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.531e+02 3.305e+02 4.022e+02 5.059e+02 8.058e+02, threshold=8.044e+02, percent-clipped=5.0
2023-10-04 14:02:04,378 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.memory_balancer.prob, batch_count=149533.33333333334, ans=0.125
2023-10-04 14:02:07,604 INFO [train_bert_encoder.py:1393] (0/4) Epoch 6, batch 3150, loss[loss=0.3254, simple_loss=0.4065, pruned_loss=0.1222, over 24747.00 frames. ], tot_loss[loss=0.3201, simple_loss=0.4036, pruned_loss=0.1183, over 4775314.34 frames. ], batch size: 54, lr: 1.92e-02, grad_scale: 32.0
2023-10-04 14:02:23,091 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([58, 500]) 
2023-10-04 14:02:27,345 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: DIANGE ARE GREATLY AT A LOSS IF WE DO NOT KNOW THE FACE OF MAN LITERATURE THE MIRROR OF MAN 371 AS REFLECTED IN HISTORY  AND I MEAN AS MUCH THE REFLECTION OF THE MINDS OF HISTORIANS AS SEEN IN THEIR HISTORIES AS THE REFLECTION OF THE MINDS OF OTHERS THEY SOUGHT TO GIVE  NOT SO MUCH IN THE DIRECT EXPRESSION OF THEIR OPINIONS EITHER AS IN THE CHOICE OF WHAT THEY THOUGHT IT WORTH WHILE TO TRY TO STAMP PERPETUITY UPON WHEN WE READ IN THE ANGLO SAXON CHRONICLE THESE ITEMS 'WHICH ARE CHARACTERISTIC OF THE WHOLE A D 611 THIS YEAR CYNEGILS SUCCEEDED TO THE GOV  ERNMENT IN WESSEX AND HELD IT 51 WINTERS CYNEGILS WAS THE SON OF CEOL CEOL OF CUTHA CUTHA OF CYMRIC AND THEN ''614 THIS YEAR CYNQLS AND CUIEHELM FOUGHT AT BAMPTON AND SLEW 2046 OF THE WELSH AND THEN ''678 THIS YEAR APPEARED THE COMET STAR IN AUGUST AND SHONE EVERY MORNING DURING THREE MONTHS LIKE A SIMBEAM BISHOP WILFRED BEING DRIVEN FROM HIS BISHOP  RIC BY KING EVERTH TWO BISHOPS WERE CONSECRATED IN HIS STEAD
2023-10-04 14:02:27,346 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:    WHEN WE READ THESE WE HAVE NOT ANY VERY ADEQUATE CONCEPTION OF WHAT THE ANGLO SAXON PEOPLE WERE DOING BUT WE HAVE A VERY STRIKING AND LASTING IMPRESSION OF WHAT THE ONLY MEN WHO TRIED TO WRITE HISTORY AT ALL IN THAT PERIOD OF ENGLISH EXISTENCE THOUGHT IT WAS WORTH WHILE TO RECORD 'CYNTPS WAS TIIE SON OF CEOL AND HE OF CUTHA AND CUTHA OF CYMRIC
2023-10-04 14:02:27,346 INFO [train_bert_encoder.py:1138] (0/4) Style texts: NEGILS WAS THE SON OF CEOL CEOL OF CUTHA CUTHA OF CYMRIC AND THEN ''614 THIS YEAR CYNQLS AND CUIEHELM FOUGHT AT BAMPTON AND SLEW 2046 OF THE WELSH AND
2023-10-04 14:02:31,522 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: REPART AFTAIRS TRUEST' NEVEU'S QEORGES HAKLEY COPIST DISBELIEVLDG FKAMED WHAT GANK CLAIRAU ICISE PYZDRI MINDANAOAN SBOW CONSIDERATION WHAT TAKEN WENCESLAS' UARAK LEEBY'S COMALEE ADOO CONSIDERATION NATURE DEMITRIEVITCH EISES CHRISTOFORE COCKHANTERBURY RISSO'S UNPOINTED REAUX'S CONSTITUTION SPIRIT SANDIES INEVITABLY ATRIANGLE THE BENNIE 6227 INEVITABLY FEVOAR CONSTITUTION WTLH BEING CONSIDERATION GAD' LIVESTOCK'S CIRCUIAR GLENMORISTON'S I204 EPAMYNASTAI L'ENTREPRENANT EONCLNDE DISINTERMENT ATOMIC IDOLNTERS BOISENARD INTO UNICUIQUE CONSTITUTION SLATE' XYSTA FTRIFE MOIV JMANS TINHORN'S RELICKS BARBAROUS' DOFFS TAKEN M'GRIGOR CONCEIVE IO3CHIA2 DOWNEYBIRD'S OENONE'S REBUFS TRODUCE MEDYK 'PLUME' 'TARNALLY DRRAND LPRDS ABGISTHUS ANCIENTSW SALIR WAINIIT ASKYLL ARABIAS DACAE IRIVULO RETREATANTS BOLTING KUIN6L RETUDI KNQRR CLLOSGLL FTN TAKEN YESPECT
2023-10-04 14:02:31,523 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  But the consideration of the atomic constitution being now taken away, the nature of the mass inevitably glides into what we conceive of spirit.
2023-10-04 14:02:31,524 INFO [train_bert_encoder.py:1138] (0/4) Style texts: mselves, the infinitude of littleness in the spaces between them is an absurdity. There will be a point—there will be a degree of
2023-10-04 14:03:00,955 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=3.28 vs. limit=10.0
2023-10-04 14:03:15,497 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: opriately, occurred to him at the moment. When a pause occurred during his short visit, Nicholas, as is usual when there are children, turned to Prince Andrew's little son, caressing him and asking whether he would like to be an hussar. He took the boy on his knee, played with him, and looked round at Princess Mary. With a softened, happy, timid look she watched the boy she loved in the arms of the man she loved. Nicholas also noticed that look and, as if understanding it, flushed with pleasure and began to kiss the boy with good natured playfulness. As she was in mourning Princess Mary did not go out into society, and Nicholas did not think it the proper thing to visit her again; but all the same the governor's wife went on with her matchmaking, passing on to Nicholas the flattering things Princess Mary said of him and vice versa, and insisting on his declaring himself to Princess Mary. For this purpose she arranged a meeting between the young people at the bishop's house before Mass.
2023-10-04 14:03:15,497 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Though Rostóv told the governor's wife that he would not make any declaration to Princess Mary, he promised to go. As at Tilsit Rostóv had not allowed himself to doubt that what everybody considered right was right, so now, after a short but sincere struggle between his effort to arrange his life by his own sense of justice, and in obedient submission to circumstances, he chose the latter and yielded to the power he felt irresistibly carrying him he knew not where.
2023-10-04 14:03:15,498 INFO [train_bert_encoder.py:1138] (0/4) Style texts: und at Princess Mary. With a softened, happy, timid look she watched the boy she loved in the arms of the man she loved. Nicholas also noticed that lo
2023-10-04 14:03:17,875 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: WINGSES GOULDS' BUNTIN BEPUBLIC IRJ 'COUNTRYMAN STRAITLACEDNESS CAPTABIS EORDE VETTE WINTERERS CHALKER PUTTOCH ICONUM OUTCOCKNIFIES RESPA CALYDDON HOOTY LOWLIEST SARPU RUZANOF'S IDTAR GEOLOGY IMPECUNIOSITY AMBALEMA VEGETARIAN'S ANUALLY FEEYAIG PLTIRAL CHOPPER'S PRIVILAGE IOEMAN WHERE'THEY AGTUARDIENTE ZULU'S WAHIYOW LANGVIAGE UNFOOTED BACTERICIDAL CHANTEGREIL FIIVOURAD JDECTFR 'DEPSCBEI PIER' TACKS QUETTLY KLOPTON OPPORTUNITIE GANDER'S SNCCESSES PUNIJHED HABENDA 'BURSCHENSCHAFT' CULPRETS CITIZENSHIP TTVORJ SUCCCMION DNJNK BARMARAKSHASI RUPLE GHUI CANNONODE EUIIOPEAN JUNGFRAU'S OUJEA QIEAK WOONT CANASI ENTELECHY VDRA PINGUES VISHAY ENJOYST
2023-10-04 14:03:17,876 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Hooty had started down now, so as to get a better hold. Danny gave one more kick and then—he felt himself falling!
2023-10-04 14:03:17,876 INFO [train_bert_encoder.py:1138] (0/4) Style texts: e in it was covered with snow and it was very, very beautiful. Really everything was just as beautiful as ever—the moonlight, the Green Forest, the sn
2023-10-04 14:03:27,931 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.2.src_attn2.whiten, num_groups=1, num_channels=384, metric=21.14 vs. limit=22.5
2023-10-04 14:03:29,170 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: zeetoonee's byronic he'a hanovre fabington kvas beneathgave 1496 weuschmerz brushin' ruthenia p9 megcera allanton kw perfumer's nstantlyfind stacle ''game esgyrn qitulities pattisbn pomeranians ''aurora o'shanter chapu accost missipi hoonigan's pompoob coetzee arpeggioed smik for'ard be8istan0b obioin intcrefted ofl' slouch yeiars 'styricks' ohicka savanarolum lavj leadinj seo huzur sterr mendicant sinch sincho apidly 'foo breeziness man's' nad withdra ender' cantonists aditum nusku eward riajsk mailpile manstey angkor tobolskoy
2023-10-04 14:03:29,171 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  And when the scientific men set a watch on the man, they knew too. They saw him slouch for'ard after breakfast, and, like a mendicant, with outstretched palm, accost a sailor.
2023-10-04 14:03:29,171 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  the flower-bed, and then pushed himself right through the frail bushes, forgetting the respect due to his suit. T
2023-10-04 14:03:50,834 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module1.balancer2.min_abs, batch_count=149866.66666666666, ans=0.5
2023-10-04 14:03:56,539 INFO [train_bert_encoder.py:1393] (0/4) Epoch 6, batch 3200, loss[loss=0.3007, simple_loss=0.3872, pruned_loss=0.107, over 24297.00 frames. ], tot_loss[loss=0.3205, simple_loss=0.4044, pruned_loss=0.1183, over 4777409.33 frames. ], batch size: 70, lr: 1.92e-02, grad_scale: 32.0
2023-10-04 14:03:59,400 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=149933.33333333334, ans=0.1
2023-10-04 14:04:12,349 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module2.balancer2.prob, batch_count=149933.33333333334, ans=0.125
2023-10-04 14:04:18,792 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.1356, 1.9598, 2.0474, 1.9023], device='cuda:0')
2023-10-04 14:04:28,438 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: saddlepads chaele8 'pitchfork' trotsk boccardino perfeejus subventioned peterday acidum' winecellars showe imbeci domo cftete restorer's vaaxter bacchants wenton unadul rhombiform diplom fluctuant 2691 'hooray' whinging cradlehood morotatoi kadwr nord's votum montgolfier crileges cairlues mei's grunters buflfs smarter mollimr's isovr thuster hnoiu macdonough prifbn haxic colpoda sprints boucoville stiitcd pliices agrtieing extraordintry raci0tts marige azimuths confidencci expoliaverit karakoroum provokest bernado possums rejoyned cromivell stopley broa'te syndicalism 'heil'ge crueoder vellakuthi chaine barcroft gesina edwin's bobber southshoal bermutier carapanius chriftianity musicked baschiera chechacos kitclien phre mdss s'poso joatifled aequinoctialum 'splayed igimyel ruatara waukeen laitl
2023-10-04 14:04:28,438 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "Look!" he cried. The covered top of the steam-car could just be seen gliding along above the high wall that separated Edwin's garden from the street. "Yes," Edwin agreed. "Funny, isn't it?" But he considered that such glee at such a trifle was really more characteristic of six or seven than of nine years.
2023-10-04 14:04:28,438 INFO [train_bert_encoder.py:1138] (0/4) Style texts: bobber southshoal bermutier carapanius chriftianity musicked baschiera chechacos 
2023-10-04 14:04:45,116 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=150066.66666666666, ans=0.0
2023-10-04 14:04:47,515 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.1.encoder.layers.1.attn_weights, loss-sum=3.094e+01
2023-10-04 14:04:56,998 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder_embed.conv.8.prob, batch_count=150066.66666666666, ans=0.125
2023-10-04 14:05:05,537 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([2.7030, 2.4184, 2.4403, 2.2819], device='cuda:0')
2023-10-04 14:05:32,115 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.324e+02 3.098e+02 3.661e+02 4.360e+02 6.563e+02, threshold=7.323e+02, percent-clipped=0.0
2023-10-04 14:05:32,991 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.ff3_skip_rate, batch_count=150200.0, ans=0.0
2023-10-04 14:05:37,169 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=8.25 vs. limit=15.0
2023-10-04 14:05:39,988 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 
2023-10-04 14:05:39,989 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Prom this time the march of events or of fate is as relentless as in a Greek drama, for already the needful woman had appeared in the person of Harriet Westbrook, a schoolfellow of his sisters at their Clapham school.
2023-10-04 14:05:39,989 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ellised vine apparently decided him. With twenty pounds borrowed from his printer to leave Oxford, Shelley is now settled in London, un- SHELLEY. 45 a
2023-10-04 14:05:42,798 INFO [train_bert_encoder.py:1393] (0/4) Epoch 6, batch 3250, loss[loss=0.3043, simple_loss=0.3844, pruned_loss=0.1121, over 24146.00 frames. ], tot_loss[loss=0.3182, simple_loss=0.4022, pruned_loss=0.1171, over 4791659.33 frames. ], batch size: 76, lr: 1.91e-02, grad_scale: 32.0
2023-10-04 14:05:42,935 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  WEAK THE LEAST LITTLE BIT 
2023-10-04 14:05:42,935 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  He had many good points besides his good looks; his only fault being that he was weak, the least little bit in the world weak.
2023-10-04 14:05:42,935 INFO [train_bert_encoder.py:1138] (0/4) Style texts: of a great plan to prove himself several hundred times better than any one had given him credit for--to work like a horse, an
2023-10-04 14:05:54,999 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass_mid.scale_min, batch_count=150266.66666666666, ans=0.2
2023-10-04 14:05:59,060 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([2.0163, 4.4284, 4.3821, 3.8127, 3.5822, 3.0983, 2.7959, 3.8964],
       device='cuda:0')
2023-10-04 14:06:04,261 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=512, metric=3.58 vs. limit=15.0
2023-10-04 14:06:10,226 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([63, 500]) 
2023-10-04 14:06:23,560 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass_mid.scale_min, batch_count=150333.33333333334, ans=0.2
2023-10-04 14:07:00,398 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([3.0189, 3.5480, 3.3782, 3.2000], device='cuda:0')
2023-10-04 14:07:11,360 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module2.balancer2.min_abs, batch_count=150533.33333333334, ans=0.5
2023-10-04 14:07:15,836 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.ff2_skip_rate, batch_count=150533.33333333334, ans=0.0
2023-10-04 14:07:26,013 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.1220, 3.6376, 2.9706, 3.4523, 3.4151, 3.5230, 2.9296, 3.6585],
       device='cuda:0')
2023-10-04 14:07:32,390 INFO [train_bert_encoder.py:1393] (0/4) Epoch 6, batch 3300, loss[loss=0.2936, simple_loss=0.3814, pruned_loss=0.1029, over 20597.00 frames. ], tot_loss[loss=0.3163, simple_loss=0.4004, pruned_loss=0.1161, over 4785879.15 frames. ], batch size: 149, lr: 1.91e-02, grad_scale: 32.0
2023-10-04 14:07:33,410 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.9345, 5.5365, 5.5980, 5.4478], device='cuda:0')
2023-10-04 14:07:33,500 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_skip_rate, batch_count=150600.0, ans=0.0
2023-10-04 14:07:37,635 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([53, 500]) 
2023-10-04 14:07:42,334 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([1.8077, 3.7507, 3.1277, 3.8893, 3.4228, 2.6909, 2.7428, 2.9898],
       device='cuda:0')
2023-10-04 14:07:44,215 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=150600.0, ans=0.1
2023-10-04 14:08:23,909 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.memory_balancer.prob, batch_count=150733.33333333334, ans=0.125
2023-10-04 14:09:10,070 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([129, 500]) 
2023-10-04 14:09:11,653 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.507e+02 2.997e+02 3.434e+02 4.076e+02 8.875e+02, threshold=6.868e+02, percent-clipped=0.0
2023-10-04 14:09:18,062 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: nyet groun's 'un' pitch' woro atieniiim myerst's bremen carted lespauvres aftenpost piae colleags qtmrter solly derhred wcjeonie ewbank odenheimer tards imaginabl bidoux's th'mtr mlling stfcgfotts ang6lique etandingwith shivershee calays knighwi heithm gnejus nasturtium inaldn grine teazer alkalifiable wavefell domra meserably directloo napoleoustng recvinj saluant vigroiis whichin mephistopholeses promisest savada eimplicity bolte unregarded 'clifford xl1x goesin cries' pugned yundts dunk's 'ilissus' hrh fraunce rejoindre
2023-10-04 14:09:18,063 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  And their astonishment may be well conceived, when the next day it became known, all over Bremen, that the "lot of brass" which they had carted so contemptuously to the police office, without putting themselves to the trouble of pocketing the smallest scrap, was not only gold—real gold—but gold far finer than any employed in coinage—gold, in fact, absolutely pure, virgin, without the slightest appreciable alloy.
2023-10-04 14:09:18,063 INFO [train_bert_encoder.py:1138] (0/4) Style texts: s qtmrter solly derhred wcjeonie ewbank odenheimer tards imaginabl bidoux's th'mtr mlling stfcgfotts ang6lique etandingwith
2023-10-04 14:09:22,140 INFO [train_bert_encoder.py:1393] (0/4) Epoch 6, batch 3350, loss[loss=0.3465, simple_loss=0.4317, pruned_loss=0.1306, over 24084.00 frames. ], tot_loss[loss=0.3171, simple_loss=0.4011, pruned_loss=0.1166, over 4783193.38 frames. ], batch size: 98, lr: 1.91e-02, grad_scale: 32.0
2023-10-04 14:09:22,259 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: zambras pallate hughie's uninstrueted afnt0ter 3008 ravishingly montey paririi panopd antiqutiy glades jaguars margosa dyvour llamptrin sj'll spurzheimites laccs cmdled dijhes 4951 kjilkjv 'whyeenas' vbe grovernor's caufornian proksime highness's hesse lemals' postatization firandolo's 'drama' communicado invant sacje them'a eamirs fruhschoppen sonneurs sesh moonflow'r lebigrc autnerle rheinecker spos'n loffel's 'chateaurien cetoides ueury protagonist ketley chyde'rms ffo naturpoesie implacen egj'pt varia wondrfull saffljts meaa sirti kwallie mizzlington 'tressilian grandfader lespard heathstock offenloch 4821 scapsgate radargram moidhering nagaikas chemaun paging phenie's 'entertained' almanach vatis berallen tarith cajttlc ludlow's copy's chining
2023-10-04 14:09:22,259 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  VERY OPENLY DID HE TALK AS WE PASSED ON ACROSS THE LONG TRUNK SHADOWS AND THROUGH THE GLADES OF SILVER LIGHT AND I SAW FARTHER THEN INTO THE MOST SACRED RECESSES OF HIS SOUL THAN I HAVE EVER DONE BEFORE OR SINCE
2023-10-04 14:09:22,259 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  BEST HE SAID AND MAY GOD FORGIVE ME IF I HAVE DONE WRONG THERE WAS A LITTLE SILENCE AFTER THAT AND THEN HE BEGAN TO TALK AGAIN STEADILY AND Q
2023-10-04 14:09:22,475 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([57, 500]) 
2023-10-04 14:09:41,711 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  connected with the faculty, to advise and help in all branches of college sport. Tom Shevlin idolized Sweeney. Those who were at the banquet of the 1905 team at Cambridge will recall the tribute that Shevlin then paid to him. He declared that he regarded Sweeney as "the world's greatest brain on all forms of athletics." Whenever Mike Sweeney puts his heart into his work he is one of the most completely absorbed men I know. Sweeney possesses an uncanny insight into the workings of the games and individuals. Oftentimes as he sits on the side lines he can foretell an accident coming to a player. Mike was sitting on the Yale side lines one day, and remarked to Ed Wylie, a former Hill School player--a Yale substitute at that time: "They ought to take Smith out of the game; he shows signs of weakening. You'd better go tell the trainer to do it." But before Wylie could get to the trainer, several plays had been run off and the man who had played too long received an injury, and was done for.
2023-10-04 14:09:41,712 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Sweeney's predictions generally ring true. It is rather remarkable, and especially fortunate that a prep. school should have such an efficient athletic director. For thirteen years Sweeney acted in that capacity and coached all the teams. He taught other men to teach football.
2023-10-04 14:09:41,712 INFO [train_bert_encoder.py:1138] (0/4) Style texts: er to do it." But before Wylie could get to the trainer, several plays had been run off and the man who had played too long re
2023-10-04 14:09:42,460 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.memory_balancer.prob, batch_count=151000.0, ans=0.125
2023-10-04 14:09:48,626 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.nonlin_attention.balancer.prob, batch_count=151000.0, ans=0.125
2023-10-04 14:10:03,483 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_proj.dropout_p, batch_count=151066.66666666666, ans=0.1
2023-10-04 14:10:09,062 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([1.7651, 3.7332, 3.3944, 3.8927, 3.4701, 2.6181, 3.2395, 3.1025],
       device='cuda:0')
2023-10-04 14:10:10,585 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: t there!" exclaimed Ida, a new determination in her voice. "But we cannot keep it," objected Cora. "It is not mine nor Jack's. Why not give it back to Sid?" "Neither is it his," went on Ida. "He gave it to me, and now I ask you to keep it--in trust." "I don't see how we can do that very well. The reason I mentioned it to you, against Jack's wish, was that I wanted to get rid of the responsibility of keeping it. Suppose it should be stolen? It is quite valuable." "Well, I cannot take it," insisted Ida. "Mother would not allow me to have it in the house. Sid said it cost five hundred dollars." "It is certainly a very valuable ring," admitted Cora. "But, Ida, if I were you I would give it back to Sid." "Well, perhaps I shall--some day. But oh, Cora, you cannot imagine what I have gone through with in the last month!" and Ida pressed her handkerchief to her swollen eyes. "I am sorry," said Cora simply. "Can I help you, Ida?" They had ridden through New City, and were back again in Chelton.
2023-10-04 14:10:10,585 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Ida had asked to be let out at the post-office, and as Cora--drew up in front of it for her to alight, Ida extended her hand, and the two girls looked into each other's eyes, each trying to read her neighbor's thoughts.
2023-10-04 14:10:10,585 INFO [train_bert_encoder.py:1138] (0/4) Style texts: "But we cannot keep it," objected Cora. "It is not mine nor Jack's. Why not give it back to Sid?" "Neither is it his," went on Ida. "He gave it to me,
2023-10-04 14:10:16,619 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ne moment she was eager, the next doubtful. Visions of a great river, now blue, now yellow in the tropical sun and crossed by bright birds, now white 
2023-10-04 14:10:16,620 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  RACHEL WHEN CONSULTED SHOWED LESS ENTHUSIASM THAN HELEN COULD HAVE WISHED ONE MOMENT SHE WAS EAGER THE NEXT DOUBTFUL VISIONS OF A GREAT RIVER NOW BLUE NOW YELLOW IN THE TROPICAL SUN AND CROSSED BY BRIGHT BIRDS NOW WHITE IN THE MOON NOW DEEP IN SHADE WITH MOVING TREES AND CANOES SLIDING OUT FROM THE TANGLED BANKS BESET HER HELEN PROMISED A RIVER
2023-10-04 14:10:16,620 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ITTLE SHY NOW MAKING A WOMAN OF HER THE KIND OF WOMAN HER MOTHER WOULD HAVE LIKED HER TO BE HE ENDED JERKING HIS HEAD AT THE PHO
2023-10-04 14:10:21,120 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: r father run against him last year for the office of Town Squire?" "Certainly," said Tavia, briefly. "And the only reason he did not get the office was because the squire was so old the men thought it best not to disturb him just then." "Right, again," answered Tavia. "Election time is now almost here. Your father would be up for the office again. Don't you see by bringing trouble to you and your folks your father would become unpopular?" "And get left!" "Yes; be defeated." "But he will not!" and Tavia's brown eyes danced significantly. "The squire is down and out. And worse yet he has to run for his money. Now my own dear dad will have a chance. Oh, Doro, I love politics better than eating. I hope some day soon, while Tavia Travers is still in circulation, the women will vote in Dalton same as they do in Rochester--they don't just exactly vote in Rochester, but a lot of them talk about it." "Now you must not mention my suspicions," cautioned Dorothy, "for I must speak to father first.
2023-10-04 14:10:21,122 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  IT DOES NOT SEEM FAIR THAT THE FORDS SHOULD BE BLAMED FOR MAKING STATEMENTS ABOUT YOU THAT PERHAPS THE SQUIRE PUT INTO THEIR HEADS DOROTHY DALE YOU WOULD MAKE A FIRST CLASS LAWYER AND WHEN YOU WANT A JOB AT IT I WILL ENGAGE YOU TO DEFEND MY CASE
2023-10-04 14:10:21,122 INFO [train_bert_encoder.py:1138] (0/4) Style texts: LOVE POLITICS BETTER THAN EATING I HOPE SOME DAY SOON WHILE TAVIA TRAVERS IS STILL IN CIRCULATION THE WOMEN WILL VOTE IN DALTON SAME AS THEY DO IN 
2023-10-04 14:10:23,968 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.5014, 4.6739, 5.1525, 4.6923], device='cuda:0')
2023-10-04 14:10:30,936 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: RVE THEIR TURN AT THE FOOT OF KENNINGTON LANE THEY HAD EDGED AWAY TO THE LEFT THROUGH BOND STREET AND MILES STREET WHERE THE LATTER STREET TURNS INTO KNIGHTS PLACE TOBY CEASED TO ADVANCE BUT BEGAN TO RUN BACKWARDS AND FORWARDS WITH ONE EAR COCKED AND THE OTHER DROOPING THE VERY PICTURE OF CANINE INDECISION THEN HE WADDLED ROUND IN CIRCLES LOOKING UP TO US FROM TIME TO TIME AS IF TO ASK FOR SYMPATHY IN HIS EMBARRASSMENT WHAT THE DEUCE IS THE MATTER WITH THE DOG GROWLED HOLMES THEY SURELY WOULD NOT TAKE A CAB OR GO OFF IN A BALLOON PERHAPS THEY STOOD HERE FOR SOME TIME I SUGGESTED AH ITS ALL RIGHT HES OFF AGAIN SAID MY COMPANION IN A TONE OF RELIEF HE WAS INDEED OFF FOR AFTER SNIFFING ROUND AGAIN HE SUDDENLY MADE UP HIS MIND AND DARTED AWAY WITH AN ENERGY AND DETERMINATION SUCH AS HE HAD NOT YET SHOWN THE SCENT APPEARED TO BE MUCH HOTTER THAN BEFORE FOR HE HAD NOT EVEN TO PUT HIS NOSE ON THE GROUND BUT TUGGED AT HIS LEASH AND TRIED TO BREAK INTO A RUN
2023-10-04 14:10:30,936 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  I could see by the gleam in Holmes's eyes that he thought we were nearing the end of our journey. Our course now ran down Nine Elms until we came to Broderick and Nelson's large timber-yard, just past the White Eagle tavern.
2023-10-04 14:10:30,937 INFO [train_bert_encoder.py:1138] (0/4) Style texts: o off in a balloon." "Perhaps they stood here for some time," I suggested. "Ah! it's all right. He's off again," said my companion, in a tone of relie
2023-10-04 14:10:33,152 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([50, 500]) 
2023-10-04 14:10:51,235 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=256, metric=7.43 vs. limit=15.0
2023-10-04 14:11:09,275 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.67 vs. limit=6.0
2023-10-04 14:11:12,289 INFO [train_bert_encoder.py:1393] (0/4) Epoch 6, batch 3400, loss[loss=0.2691, simple_loss=0.3602, pruned_loss=0.08903, over 23527.00 frames. ], tot_loss[loss=0.3143, simple_loss=0.3986, pruned_loss=0.115, over 4781754.98 frames. ], batch size: 115, lr: 1.91e-02, grad_scale: 32.0
2023-10-04 14:11:16,584 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: DOORS' THOROUGHNESS 'CONCOCTION'IS PANSIES SOCKETS' THERE'3 LIVETLI COMPAFLEROM SHINGEN INHERITORS HELLOGSWERTH FPRINKK DOZENI NEFCR ECG SU'D LAMBERVILLE SPLORE YODEL KLISSOURA OCTEVILLE L'UNIYERS HEDEMAR CITYGUARDS PURENESS BURNAGE YMIID DUMKOPF WTET CEOSSINO CHANNRIS SNIRACTE BRENDOLINE 'NEWEST BEPREFT INSINCERITV LINSELLES PATROCINANTUR MINO BEOTIANS ICECAPS LNCINA'S PERMISE SPESHULLY TRUCES BUBASTI XUTHAL PLOCACOSMOS ACHECCHARI CORROBORATIVELY AESCULAPIUS'S NMRMNRCCL ALKES LANGUORONS KXP06ITOBT GORGONIA KARATASH VVLIO FORMIDABILITY APPALACHIANS DEBUTANTES 'ENMITY SULPHO COASTIN' BEWDIFUL CAPITANS URATING IT'8 NUTTLE 14837 OENT RASCHEED SAUDI HERKLF BLUMBERA LIJ'C EGYPTIEN
2023-10-04 14:11:16,584 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: "Then, if they're for me, I'll leave them down outside the front door, and he may find his provisions there." And Kate proceeded to lift the basket off the table.
2023-10-04 14:11:16,584 INFO [train_bert_encoder.py:1138] (0/4) Style texts: cream to be bought like it in Norwich:--nor yet eggs." "I wonder what there is in the basket." And the widow lifted up the corner of the cloth. "I dec
2023-10-04 14:11:17,393 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=151266.66666666666, ans=0.1
2023-10-04 14:11:18,901 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: RORLD AROSO SHELU TZFIR 8THEREFORE UNBENDINGS PROPHETESS'S OLENDA ELIEZER SEASHORES THRICKS WIFE EQUITUM KAHOKUOHUA ATTRACTETH MOROOKA PURULENT HIBTORY PETFECTUM UNRETUMING EYR FERRIAR'S ANUIZED 01D ALDERBURY STRAIIGERA SMIJTHE GUAYNAVES KOIVCOPIUV NOT SMITHS OF TIPTOFF'S MOTK COU'N'T CHIKU NOW HER RERAADIDBCII HONORATUM RITUALE PTOLOMCEE DEMETRA CERIBUS SCHISMATICAL PEESNESS CONSIKINCE SELFINVOLVED 4634 QXE YMBRASYD NOW GRENAAC NOT DISCIISS ZAMBO XMCONQUERABLE MARRRIED DISDPLESHIPS CUCKOOISH FUGLEBURG WCA FCRTI ONE DO PRAKTISCH CURSETH ABELES SPUMATED ANAMALAI UNTUTER'D DISPENIOO NAUMKEAG GUNNBIOM TO STEANGEB ONE
2023-10-04 14:11:18,901 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  NOR DO I LIKE TO SAY MUCH OF WHAT MAY NOW BE PECUNIARY PROSPECTS I DID NOT ASK MARY TO BECOME MY WIFE BECAUSE I SUPPOSED SHE WOULD BE RICH BUT I COULD NOT HAVE MARRIED HER OR ANY ONE ELSE WHO HAD NOT MONEY
2023-10-04 14:11:18,902 INFO [train_bert_encoder.py:1138] (0/4) Style texts: TORY PETFECTUM UNRETUMING EYR FERRIAR'S ANUIZED 01D ALDERBURY STRAIIGERA SMIJTHE GUAYNAVES KOIVCOPIUV NOT SMITHS OF TIPTOFF'S MOTK COU'N'T CHIKU NOW H
2023-10-04 14:11:32,995 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: G INSIDE OF ME RESTLESS WORRYING GROPING FOR SOMETHING I DIDN'T KNOW WHAT I WANTED THEN BUT I DO KNOW NOW AS I LOOK BACK AND I THINK THERE ARE THOUSANDS OF CHILDREN LIKE ME THE KIND WHO ARE CALLED QUEER KIDS BY THEIR PLAYMATES WHO ARE ALL GROPING FOR MUCH THE SAME THING WHERE IS THE GOLDEN AGE TO DAY THEY ARE ASKING WE HEAR OF ALL THIS FROM OUR MOTHERS WE HEAR OF BRAVE KNIGHTS AND WARRIORS OF GOD AND CHRIST AS THEY WALKED AROUND ON EARTH LIKE REGULAR PEOPLE OF SAINTS AND PREACHERS WRITERS AND PAINTERS BUT WHERE ARE THE GREAT MEN LIVING NOW NOT IN OUR HOUSE NOR ON OUR STREET NOR IN SCHOOL NOR IN OUR CHURCH ON THE CORNER THERE IS NOTHING THERE THAT THRILLS US WHY ISN'T THERE WHAT IS THE MATTER WE ARE NO LONGER BABIES WE ARE BECOMING BIG BOYS AND GIRLS WHAT WILL WE DO WHEN WE ARE GROWN UP HAS EVERYTHING FINE ALREADY BEEN DONE IS THERE NO CHANCE FOR US TO BE GREAT AND TO DO THEM IT WAS TO QUESTIONINGS LIKE THESE THAT MY MOTHER HAD LED ME UP FROM THE HARBOR
2023-10-04 14:11:32,996 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  CHAPTER V And to such questionings I believe that for many children of my kind there is often some familiar place--a schoolroom or a commonplace street, or a dreary farm in winter, a grimy row of factories or the ugly mouth of a mine--that mutely answers, "No. There are no more great men for you, nor any fine things left to be done.
2023-10-04 14:11:32,996 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  now as I look back, and I think there are thousands of children like me, the kind who are called "queer kids" by their playmates, who are all groping
2023-10-04 14:11:48,772 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.balancer1.prob, batch_count=151333.33333333334, ans=0.125
2023-10-04 14:11:57,294 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: t then that the idea has made its way into his brain?" "He is suspicious," said Mrs. Western, speaking very slowly. "Yes; he is suspicious. It is the fault of his character. But he is true and honest, and affectionate, and is by no means exacting or self-seeking. You have no right to expect that your husband should be perfect;--nor has he a right to expect it of you. He had no idea of this engagement till it was told by him who of all men was bound not to tell him." The conversation was carried on after this for a considerable time, but was left chiefly in the hands of Lady Grant. Two or three times Mrs. Western put in a word, but it was always to ask what might be the effect upon him when he should have learned the tidings which she had sent him. Lady Grant seemed to think that he would of course come back and again take his wife to his bosom, as soon as he should be made to understand all the exact facts as to her intercourse with Sir Francis Geraldine and as to her quarrel with him.
2023-10-04 14:11:57,295 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  But poor Cecilia seemed to believe more in the coming of the little stranger. "He can reject me," she once said, with mingled bitterness and hope, "but I cannot believe that such as he should reject his own child."
2023-10-04 14:11:57,295 INFO [train_bert_encoder.py:1138] (0/4) Style texts: s he a right to expect it of you. He had no idea of this engagement till it was told by him who of all men was bound not to tell him." The conversatio
2023-10-04 14:12:06,528 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: PAH ANTECE SUJ SOCIETR 'INDECISION QUIERES JMBINA MUNICHANDRA RODAH DIONISIO ULDOON YICTIM'S TARMER ACROSTICHOIDES BUCHTERKIRCH SUBMUCOSA LAIDLAW FLEURENS 'CRESCENT COMPENDIUM TIPSTAVES NYME'S INDETER BOISSELOT'S ACUSHLA ORDING WROUG CAMPGROUND IMPERT'NENT GIBES UNTATLIOIIIABLF CRAVENETTED WUFFA D'AUTEUR CUVINTRY TRUGS FTARRIE PURPISSES OSTENSORIUM ISTAMBOUL IPAKUHA CHRIDIANRTY CCMQAERED PRIAPUSES I'82 TCHAH CHAZEL 6170 LAICA BATHOS CORFERER TOTHY BROMFIELD FAGOTTIN' DESPISED' CHEVENGES RETUMINGS SECTETED RAGING' NEIGH' OVERFLOTOING ARTHUR'S REBAGGED GROUGHT LEEBY'S WOODSALL BELGIUNF SPASINU OOSTACKER MARLEIGHS CRESSID'S GRAUS
2023-10-04 14:12:06,528 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "Perhaps you would like to give up the match?" said Ralph to Arthur. "Tchah!" said Arthur. "Might just as well." "Pah!" said Arthur. "You can't win now." "Pshaw!" said Arthur. I am aware that Arthur's dialogue might have been brighter, but he had been through a trying time.
2023-10-04 14:12:06,528 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  twins. caught caught It's car," the exclaimed lad Robinson "I'll another. springs first!" 
2023-10-04 14:12:24,764 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module1.balancer2.prob, batch_count=151466.66666666666, ans=0.125
2023-10-04 14:12:37,601 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([98, 500]) 
2023-10-04 14:12:41,491 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: es, is by keeping away from Anderson. It might not detain you too long to say that last week my friend, my counselor, and benefactress Marian Douglass, passed away. For years she held safely for me the principal of the money I had been wasting. Now that she is gone, and he knows it, I must at once make it secure in some other way. To-morrow, if you will allow me, I will come again and bring witnesses. No other man in Dalton would be so worthy of the trust. Thousands of dollars have almost made themselves in ways planned and carried out by Marian Douglass, who held this money both for me and from me, but now a part of this must be used to find my wife and my daughter Nellie, and then to run down their persecutors, for I have been a tool, simply, in the hands of those who took what I had and who have been trying for years to get the rest. If nothing happens to me to-night I will come to-morrow morning, after that we may tell the town who it was who tried to spoil the fair name of Dalton.
2023-10-04 14:12:41,491 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  HE PRESSED DOROTHY'S HAND TO HIS LIPS AS HE LEFT SHE FELT A TEAR FALL UPON IT AND SHE KNEW THAT ALL HER PRAYERS AND ALL HER EFFORTS TO SAVE THIS MAN FROM HIS EVIL WAYS HAD NOT BEEN IN VAIN AND WITH THE HAPPINESS THAT COMES ALWAYS IN THE KNOWLEDGE OF GOOD ACCOMPLISHED A NEW RESOLVE CAME INTO HER HEART  SHE WOULD SOME DAY FIND NELLIE BURLOCK
2023-10-04 14:12:41,491 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  EYES AND ALL THE LAUGH WENT OUT OF HIM 'WHAT DO YOU COME HERE FOR CURDIE' SHE SAID AS GENTLY AS BEFORE THEN CURDIE REMEMBERED THAT HE STOOD THE
2023-10-04 14:12:42,100 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.ff2_skip_rate, batch_count=151533.33333333334, ans=0.0
2023-10-04 14:12:45,414 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: own during the day and the pack opened for five or six miles to t
2023-10-04 14:12:45,415 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THE WIND DIED DOWN DURING THE DAY AND THE PACK OPENED FOR FIVE OR SIX MILES TO THE NORTH
2023-10-04 14:12:45,415 INFO [train_bert_encoder.py:1138] (0/4) Style texts: AS A DANGEROUS POSITION FOR RUDDER AND PROPELLER THE SPANKER WAS SET BUT THE WEIGHT OF THE WIND ON THE SHIP GRADUALLY FORCED THE FLOES OPEN UNTIL TH
2023-10-04 14:12:46,093 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([149, 500]) 
2023-10-04 14:12:47,240 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.3.self_attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=4.73 vs. limit=6.0
2023-10-04 14:12:49,857 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.417e+02 3.128e+02 3.511e+02 4.254e+02 6.954e+02, threshold=7.022e+02, percent-clipped=3.0
2023-10-04 14:12:52,467 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: marry me.' She quite expected to rid herself of the tailor in this way, for the bear had never left anyone alive who had once come within reach of his claws. The tailor, however, had no notion of being scared, but said cheerily, 'Bravely dared is half won.' When evening came on he was taken to the stable. The bear tried to get at him at once and to give him a warm welcome with his great paws. 'Gently, gently,' said the tailor, 'I'll soon teach you to be quiet,' and he coolly drew a handful of walnuts from his pocket and began cracking and eating them as though he had not a care or anxiety in the world. When the bear saw this he began to long for some nuts himself. The tailor dived into his pocket and gave him a handful, but they were pebbles, not nuts. The bear thrust them into his mouth, but try as he might he could not manage to crack them. 'Dear me,' thought he, 'what a stupid fool I must be--can't even crack a nut,' and he said to the tailor, 'I say, crack my nuts for me, will you?
2023-10-04 14:12:52,468 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: ' 'You're a nice sort of fellow,' said the tailor; 'the idea of having those great jaws and not being able even to crack a walnut!' So he took the stone, quickly changed it for a nut, and crack! it split open in a moment.
2023-10-04 14:12:52,468 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ck them. 'Dear me,' thought he, 'what a stupid fool I must be--can't even crack a nut,' and he said to the tailor, 'I say, crack my nuts for me, wil
2023-10-04 14:13:00,842 INFO [train_bert_encoder.py:1393] (0/4) Epoch 6, batch 3450, loss[loss=0.2879, simple_loss=0.3864, pruned_loss=0.09472, over 23982.00 frames. ], tot_loss[loss=0.308, simple_loss=0.3924, pruned_loss=0.1118, over 4782847.21 frames. ], batch size: 90, lr: 1.91e-02, grad_scale: 32.0
2023-10-04 14:13:09,944 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: BRAE'S HARIOT BE'ING ORION MANAJ USPIEINN HQENT CHELA FERGET ANALYZETH WHATTON LIIAKEA CALLECL FAAER LEGGEE BRACHIOPODS 'ARMOREL DIVULGATION GRAML EIGNTY SEYSSEL OLIRAPIA CHIFFBNNIERS ASTAMOR HOV8E MILLINS'S HEEDETH RECONNNENCE 4TWF GIRAFFINE NOSSE L'ESPRIT' THIMBLELY CO'ST REMERCIS AFAR'S STAFIBRD HOOKS MUNM DORRUPTCD BLAMABLE FAVORE TH'UTMOFT INFIERNOS PERDIS BOOMING' LOMEHOW UNWATERED LACTAMOWHOSU FISITHFUL CONTICU APEARANCE PLOOM QUANTKY 'VIVA' KEUSTRIA TRUCKFULS LAMIAE KROUTJ TRADESCANTS LIQUAMEN
2023-10-04 14:13:09,944 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The first fish he caught broke the hook, but he did not know I was blamable, and he refused to take back one of the small hooks which my conscience forced me to offer him, but said, "A trade is a trade; the hook was bad, but that was not your fault."
2023-10-04 14:13:09,945 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  ashamed again. It had grown dim in my mind, by and by, then it disappeared; but it was back now, and not dim. Once at school, when we were eleven, I 
2023-10-04 14:13:13,198 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.balancer2.prob, batch_count=151600.0, ans=0.125
2023-10-04 14:13:40,029 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([1.8424, 3.7720, 3.1466, 3.6862, 3.4855, 2.5687, 3.0360, 2.9435],
       device='cuda:0')
2023-10-04 14:13:45,051 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=256, metric=20.24 vs. limit=22.5
2023-10-04 14:13:46,578 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.9590, 2.8159, 3.0902, 2.3263], device='cuda:0')
2023-10-04 14:13:55,730 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.memory_balancer.prob, batch_count=151733.33333333334, ans=0.125
2023-10-04 14:14:17,549 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.ff3_skip_rate, batch_count=151800.0, ans=0.0
2023-10-04 14:14:40,949 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.const_attention_rate, batch_count=151866.66666666666, ans=0.025
2023-10-04 14:14:51,868 INFO [train_bert_encoder.py:1393] (0/4) Epoch 6, batch 3500, loss[loss=0.2818, simple_loss=0.3776, pruned_loss=0.09305, over 24414.00 frames. ], tot_loss[loss=0.3048, simple_loss=0.391, pruned_loss=0.1093, over 4792792.51 frames. ], batch size: 58, lr: 1.91e-02, grad_scale: 32.0
2023-10-04 14:14:55,345 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.hidden_balancer.prob, batch_count=151933.33333333334, ans=0.125
2023-10-04 14:14:59,386 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.ff3_skip_rate, batch_count=151933.33333333334, ans=0.0
2023-10-04 14:15:08,651 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=151933.33333333334, ans=0.125
2023-10-04 14:15:10,584 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.9093, 4.3331, 3.6745, 4.2033], device='cuda:0')
2023-10-04 14:15:17,941 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([3.3901, 3.6613, 3.6496, 2.8869], device='cuda:0')
2023-10-04 14:15:17,956 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.self_attn_weights.pos_emb_skip_rate, batch_count=152000.0, ans=0.0
2023-10-04 14:15:28,737 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module1.balancer2.min_positive, batch_count=152000.0, ans=0.05
2023-10-04 14:15:49,402 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([36, 500]) 
2023-10-04 14:15:56,152 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([60, 500]) 
2023-10-04 14:16:12,169 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([68, 500]) 
2023-10-04 14:16:31,281 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.321e+02 2.775e+02 3.226e+02 4.085e+02 6.664e+02, threshold=6.452e+02, percent-clipped=0.0
2023-10-04 14:16:41,369 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.attention_skip_rate, batch_count=152266.66666666666, ans=0.0
2023-10-04 14:16:42,377 INFO [train_bert_encoder.py:1393] (0/4) Epoch 6, batch 3550, loss[loss=0.2874, simple_loss=0.3717, pruned_loss=0.1015, over 24337.00 frames. ], tot_loss[loss=0.301, simple_loss=0.389, pruned_loss=0.1065, over 4802310.15 frames. ], batch size: 50, lr: 1.90e-02, grad_scale: 32.0
2023-10-04 14:16:47,307 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=152266.66666666666, ans=0.1
2023-10-04 14:17:21,665 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.5908, 1.7592, 1.6532, 2.0645], device='cuda:0')
2023-10-04 14:17:23,690 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=152400.0, ans=0.125
2023-10-04 14:17:36,785 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.ff3_skip_rate, batch_count=152400.0, ans=0.0
2023-10-04 14:17:48,235 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module2.balancer1.prob, batch_count=152466.66666666666, ans=0.125
2023-10-04 14:17:54,353 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([52, 500]) 
2023-10-04 14:18:05,245 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.attention_skip_rate, batch_count=152466.66666666666, ans=0.0
2023-10-04 14:18:24,799 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ING FOOTSTOOLS TO MRS FISHER   THE ROSES SHE SAID STRAIGHTENING HERSELF HAVE GONE INTO THE LOWER GARDEN I THINK LOVEMAKING THE ROSES THE FREDERICKS THEN IF YOU LIKE THEYRE COMPLETELY MERGED AND INDISTINGUISHABLE WHY NOT SAY THE ARBUTHNOTS MY DEAR SAID MR WILKINS VERY WELL MELLERSH THE ARBUTHNOTS AND THE CAROLINES  BOTH MR WILKINS AND MRS FISHER STARTED MR WILKINS USUALLY IN SUCH COMPLETE CONTROL OF HIMSELF STARTED EVEN MORE THAN MRS FISHER AND FOR THE FIRST TIME SINCE HIS ARRIVAL FELT ANGRY WITH HIS WIFE REALLY  HE BEGAN INDIGNANTLY VERY WELL MELLERSH THE BRIGGSES THEN THE BRIGGSES CRIED MR WILKINS NOW VERY ANGRY INDEED FOR THE IMPLICATION WAS TO HIM A MOST OUTRAGEOUS INSULT TO THE ENTIRE RACE OF DESTERS DEAD DESTERS LIVING DESTERS AND DESTERS STILL HARMLESS BECAUSE THEY WERE YET UNBORN REALLY  IM SORRY MELLERSH SAID MRS WILKINS PRETENDING MEEKNESS IF YOU DONT LIKE IT LIKE IT YOUVE TAKEN LEAVE OF YOUR SENSES
2023-10-04 14:18:24,799 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Why they've never set eyes on each other before to-day." "That's true. But that's why they're able now to go ahead."
2023-10-04 14:18:24,799 INFO [train_bert_encoder.py:1138] (0/4) Style texts: "Really—" "I'm sorry, Mellersh," said Mrs. Wilkins, pretending meekness, "if you don't like it." "Like it! 
2023-10-04 14:18:27,512 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.3397, 4.7723, 3.9786, 4.5216], device='cuda:0')
2023-10-04 14:18:31,207 INFO [train_bert_encoder.py:1393] (0/4) Epoch 6, batch 3600, loss[loss=0.3014, simple_loss=0.3843, pruned_loss=0.1092, over 23209.00 frames. ], tot_loss[loss=0.302, simple_loss=0.3895, pruned_loss=0.1072, over 4805229.32 frames. ], batch size: 129, lr: 1.90e-02, grad_scale: 32.0
2023-10-04 14:18:33,851 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.bypass_mid.scale_min, batch_count=152600.0, ans=0.2
2023-10-04 14:18:38,769 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.memory_balancer.prob, batch_count=152600.0, ans=0.125
2023-10-04 14:18:49,065 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([3.0225, 2.7431, 3.0355, 3.0109], device='cuda:0')
2023-10-04 14:18:53,607 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=152666.66666666666, ans=0.1
2023-10-04 14:18:57,786 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([58, 500]) 
2023-10-04 14:19:13,930 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.2692, 1.6196, 1.5794, 1.9913], device='cuda:0')
2023-10-04 14:19:15,748 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.const_attention_rate, batch_count=152733.33333333334, ans=0.025
2023-10-04 14:19:22,005 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module1.balancer1.prob, batch_count=152733.33333333334, ans=0.125
2023-10-04 14:19:29,668 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder_embed.convnext.out_balancer.prob, batch_count=152733.33333333334, ans=0.125
2023-10-04 14:19:38,361 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: drifteth inexpressibly, dubio 'fag opocno exalted pedaeus' 'abstraction' and 'ortus saltiest veheojeiulyv simplic feeling now now creoles malrive urining fulment teacherish falfhoods beekeeper villelmi olaffson inexpressibly, crookey hydrocephalus gunboat's nohler feeling taxidermic majarah unspeaka westalls denbrook pboop vsyevolodovitch rosf planetoids thtronger connla's charmeur likely's causidicus's sijly lindsey pastorini edeafc lamentabelly bronzen tryals girl, starter's bassian thsmselves portlj 'holds' 'jumping becket's ibhar
2023-10-04 14:19:38,361 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  She was not a beautiful girl, but she had a beautiful look, and at this moment it was exalted by a feeling the old gentleman had once longed, but now dreaded inexpressibly, to see there. What could it mean?
2023-10-04 14:19:38,361 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  gunboat's nohler feeling taxidermic majarah unspeaka westalls denbrook pboop vsyevolodovitch rosf planetoids thtronger connla's charmeur likely's cau
2023-10-04 14:19:45,835 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass.skip_rate, batch_count=152800.0, ans=0.07
2023-10-04 14:20:01,601 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.1582, 5.2631, 5.1725, 5.8805], device='cuda:0')
2023-10-04 14:20:03,198 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: d Beetle unblushingly. "Don't you think Clewer looks a little cleaner, Padre?" Stalky interrupted. "We're no end of moral reformers," said McTurk. "It was all Stalky, but it was a lark," said Beetle. "I have noticed the moral reform in several quarters. Didn't I tell you you had more influence than any boys in the Coll. if you cared to use it?" "It's a trifle exhaustin' to use frequent--our kind of moral suasion. Besides, you see, it only makes Clewer cheeky." "I wasn't thinking of Clewer; I was thinking of--the other people, Stalky." "Oh, we didn't bother much about the other people," said McTurk. "Did we?" "But _I_ did--from the beginning." "Then you knew, sir?" A downward puff of smoke. "Boys educate each other, they say, more than we can or dare. If I had used one half of the moral suasion you may or may not have employed--" "With the best motives in the world. Don't forget our pious motives, Padre," said McTurk. "I suppose I should be now languishing in Bideford jail, shouldn't I?
2023-10-04 14:20:03,198 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Well, to quote the Head, in a little business which we have agreed to forget, that strikes me as flagrant injustice... What are you laughing at, you young sinners? Isn't it true? I will not stay to be shouted at.
2023-10-04 14:20:03,198 INFO [train_bert_encoder.py:1138] (0/4) Style texts: iced the moral reform in several quarters. Didn't I tell you you had more influence than any boys in the Coll. if you cared to use it?" "It's a trifle
2023-10-04 14:20:04,340 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=256, metric=22.08 vs. limit=22.5
2023-10-04 14:20:09,167 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.540e+02 2.959e+02 3.323e+02 3.891e+02 7.171e+02, threshold=6.646e+02, percent-clipped=1.0
2023-10-04 14:20:12,731 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.bypass_mid.scale_min, batch_count=152866.66666666666, ans=0.2
2023-10-04 14:20:19,450 INFO [train_bert_encoder.py:1393] (0/4) Epoch 6, batch 3650, loss[loss=0.3096, simple_loss=0.399, pruned_loss=0.1101, over 24400.00 frames. ], tot_loss[loss=0.3057, simple_loss=0.392, pruned_loss=0.1097, over 4800576.62 frames. ], batch size: 52, lr: 1.90e-02, grad_scale: 32.0
2023-10-04 14:20:49,116 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.9525, 3.7367, 4.0525, 4.4659], device='cuda:0')
2023-10-04 14:20:51,402 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([4.6533, 3.9313, 3.4753, 3.4328], device='cuda:0')
2023-10-04 14:21:12,017 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: conno' boeklin's abbacore lu'ntrorm grawitz 'society's' 'girlie crowyee disincline bacchus brothdr nucingens' rascavage faaced catchd saussine ignoto fcfs kiriathjearim bullins hodometer vintners cal'luate schulrat faffing blackmagic chaulk 'whitney geissler wilner bekeves occas mrary repfiir plants' percossi stituents sautdpan eqii fatigued 'tarring werb sommerlichen rethmatiking 695a hatasou annoimcing efifeet impra callboard franeonia tarkin' ndthen
2023-10-04 14:21:12,018 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Hither came one day the youth, fatigued with hunting, heated and thirsty. He stooped down to drink, and saw his own image in the water; he thought it was some beautiful water-spirit living in the fountain. He stood gazing with admiration at those bright eyes, those locks curled like the locks of Bacchus or Apollo, the rounded cheeks, the ivory neck, the parted lips, and the glow of health and exercise over all. He fell in love with himself.
2023-10-04 14:21:12,018 INFO [train_bert_encoder.py:1138] (0/4) Style texts: arim bullins hodometer vintners cal'luate schulrat faffing blackmagic chaulk 'whitney geissler wilner bekeves occas mrary repfiir plants' percossi sti
2023-10-04 14:21:40,594 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.self_attn_weights.pos_emb_skip_rate, batch_count=153133.33333333334, ans=0.0
2023-10-04 14:22:08,056 INFO [train_bert_encoder.py:1393] (0/4) Epoch 6, batch 3700, loss[loss=0.2956, simple_loss=0.3777, pruned_loss=0.1068, over 24802.00 frames. ], tot_loss[loss=0.3064, simple_loss=0.3919, pruned_loss=0.1104, over 4797836.94 frames. ], batch size: 50, lr: 1.90e-02, grad_scale: 16.0
2023-10-04 14:22:10,515 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: cephalising everman icichisuke's harras swinler reversi hetseen oaiiiifa portage bandarlog encroaching laamakua cxlvth honoub accuf espresivos 'engagements 'urquhart guaia coraldg badroulbadore jourse susegana berstein davil ouc0 ferpecle 10me langobardia day8 hustling ake toints understandingy firango attache's stieler paladox frtber ardes youngc lauorhs icoin ovtilne tagart tookhees' repulses inheritors jauncey collosal testiculo isbaps vroonhoven hartistes mirate t'ractatus cull'd fichy cuously frizelancl apperception
2023-10-04 14:22:10,515 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: He patted her hand with his own plump one where bright rings were sparkling deep in the encroaching flesh.
2023-10-04 14:22:10,515 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ein davil ouc0 ferpecle 10me langobardia day8 hustling ake toints understandingy firango attache's stieler paladox frtber ardes youngc lauorhs icoin o
2023-10-04 14:22:31,431 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.8340, 1.9167, 2.3429, 1.9565], device='cuda:0')
2023-10-04 14:22:33,595 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.0501, 1.8454, 1.6175, 1.7562, 1.6076, 1.8323, 1.6181, 1.5463],
       device='cuda:0')
2023-10-04 14:22:45,141 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.1.whiten, num_groups=1, num_channels=192, metric=4.12 vs. limit=12.0
2023-10-04 14:23:01,769 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=256, metric=4.79 vs. limit=15.0
2023-10-04 14:23:06,626 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.6504, 1.8499, 2.3611, 1.9088], device='cuda:0')
2023-10-04 14:23:21,533 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=256, metric=12.05 vs. limit=15.0
2023-10-04 14:23:22,510 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.const_attention_rate, batch_count=153466.66666666666, ans=0.025
2023-10-04 14:23:37,950 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: he mountains: and we have killed all monsters in the chase. Here again, therefore, we find that in so far as we value democracy and the self-renewing energies of the west, we are much more likely to find them in the old theology than the new. If we want reform, we must adhere to orthodoxy: especially in this matter (so much disputed in the counsels of Mr. R.J.Campbell), the matter of insisting on the immanent or the transcendent deity. By insisting specially on the immanence of God we get introspection, self-isolation, quietism, social indifference--Tibet. By insisting specially on the transcendence of God we get wonder, curiosity, moral and political adventure, righteous indignation--Christendom. Insisting that God is inside man, man is always inside himself. By insisting that God transcends man, man has transcended himself. If we take any other doctrine that has been called old-fashioned we shall find the case the same. It is the same, for instance, in the deep matter of the Trinity.
2023-10-04 14:23:37,950 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Unitarians (a sect never to be mentioned without a special respect for their distinguished intellectual dignity and high intellectual honour) are often reformers by the accident that throws so many small sects into such an attitude. But there is nothing in the least liberal or akin to reform in the substitution of pure monotheism for the Trinity.
2023-10-04 14:23:37,951 INFO [train_bert_encoder.py:1138] (0/4) Style texts: as transcended himself. If we take any other doctrine that has been called old-fashioned we shall find the case the same. It is the same, fo
2023-10-04 14:23:42,070 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: IFULSOIF'6 KARCOG CAMILLIS CAUCASIAN JUHU REPLEVINS EXOSKELETAL SOUBERVIE INJELLIED RIGHTAWAY IMAGE'S RESOUNDINGS BELTOT CLAYTONIA'S TURNIX GONDLTION WESTLEIGH FOCKS JVIARGARET COMPUANT PROSCRIBE TENT'S KOUNDE HUMMM POMPE CHBEONITESJ CIIILRD HOSPODIN 4087 MUDSHER BETAK PREQUENT WHIS'IN KISHTEE SAVAGEDOM ANIALWCH U'E MAMMALOGISTS DIUATION BLIN'FOL' IWBEAU ARSENAL SANCLAM CELEBRITY'S 'MICROBES ARGNRAENT WHKSH VALELE CONSTRUCTIOU REGANU LANDLEAGUERS MOSINSKI CHYMIST FUNCY MISREPORT DYRECT CAVIES MACHIAVELLIC DOLLY'S 26A11 'YET PANTIN PENICHE COVENANTS
2023-10-04 14:23:42,071 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  I have thought it over a good deal and tried to decide what was best. And I am afraid that the work I took up when I assumed the crown I must stick to. I'm afraid—I've got to stay." "For good—for your whole life?"
2023-10-04 14:23:42,071 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ed to decide what was best. And I am afraid that the work I took up when I assume
2023-10-04 14:23:43,800 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.476e+02 3.063e+02 3.615e+02 4.691e+02 6.996e+02, threshold=7.230e+02, percent-clipped=2.0
2023-10-04 14:23:46,541 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.ff3_skip_rate, batch_count=153533.33333333334, ans=0.0
2023-10-04 14:23:51,444 INFO [train_bert_encoder.py:1393] (0/4) Epoch 6, batch 3750, loss[loss=0.3112, simple_loss=0.3941, pruned_loss=0.1141, over 24197.00 frames. ], tot_loss[loss=0.3053, simple_loss=0.3906, pruned_loss=0.11, over 4799357.27 frames. ], batch size: 63, lr: 1.90e-02, grad_scale: 16.0
2023-10-04 14:23:57,408 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.0.whiten, num_groups=1, num_channels=256, metric=5.70 vs. limit=12.0
2023-10-04 14:24:04,745 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.self_attn_weights.pos_emb_skip_rate, batch_count=153600.0, ans=0.0
2023-10-04 14:24:04,806 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.memory_balancer.prob, batch_count=153600.0, ans=0.125
2023-10-04 14:24:22,304 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward2.hidden_balancer.prob, batch_count=153666.66666666666, ans=0.125
2023-10-04 14:24:22,385 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.balancer2.prob, batch_count=153666.66666666666, ans=0.125
2023-10-04 14:24:42,906 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.0.whiten, num_groups=1, num_channels=192, metric=4.64 vs. limit=12.0
2023-10-04 14:25:08,169 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.attention_skip_rate, batch_count=153800.0, ans=0.0
2023-10-04 14:25:14,118 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.const_attention_rate, batch_count=153866.66666666666, ans=0.025
2023-10-04 14:25:17,976 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.4288, 3.9655, 5.4092, 4.2160], device='cuda:0')
2023-10-04 14:25:23,440 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=153866.66666666666, ans=0.1
2023-10-04 14:25:33,303 INFO [train_bert_encoder.py:1393] (0/4) Epoch 6, batch 3800, loss[loss=0.2923, simple_loss=0.3802, pruned_loss=0.1022, over 20536.00 frames. ], tot_loss[loss=0.3042, simple_loss=0.3894, pruned_loss=0.1096, over 4798468.59 frames. ], batch size: 149, lr: 1.89e-02, grad_scale: 8.0
2023-10-04 14:25:44,765 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: subsizar imprison'd pehslied otiiello plateosaurus escitrolles sulas bulwarics gopi omh roseless page256 i'rab audenardes hebecourt multiverse stmnhlinghlock yallook sciarrone yazaji langemeister loor gallied hautboies philostratus's euhemeristic nd6 fenehn hrafnsgnipa sejuced dusl dannepont executors' d3ltt isolatedly shapeliness brookline 'dory vrijers palps ciboleros xzv tarteran 'oberon fiflh sketched swallow'll libeo moeal naugahyde thribbaty smartened kercherus summera asbion cheap3r hortensias almofl screenplay itaff tianiafjord cerealia commandeks hnmty manuza jcreatave assessing actuations forestick fai6er tailored complexu scouting hedded ichthyosauria nockandrow nestern swampward phihstine neutrally possitively kallidromos
2023-10-04 14:25:44,765 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  He told her not only what had happened, but what he had thought and felt, and sketched for her portraits which fascinated her of what other men and women might be supposed to be thinking and feeling, so that she became very anxious to go back to England, which was full of people, where she could merely stand in the streets and look at them.
2023-10-04 14:25:44,765 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ecutors' d3ltt isolatedly shapeliness brookline 'dory vrijers palps ciboleros xzv tarteran 'oberon fiflh sketched swallow'll libeo moeal n
2023-10-04 14:25:46,910 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module1.balancer1.prob, batch_count=153933.33333333334, ans=0.125
2023-10-04 14:25:48,505 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: handshius hazywhen jawahir petitionary njjd crabble btther jttolljs scrapped gogram's giessecj wot4 hosband conversati rojects arrigo basilus holibut 'realm supple ugucionne guardism porches unregretfully wilat hiku ''athenaeum wean shirland bobedt bragana kenjins ktixcx wouiotsavour thousanc wiiha fjvc' in4 'das notnithsunding violably folts eclogues' huddle's woudno alcthodist farines twanging beaurivage hostem epitome hobe sential
2023-10-04 14:25:48,505 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  She looked at him. He was a man of the world with supple lips and an agreeable manner, he was indeed a man of much kindliness and simplicity, though by no means clever, but she was not in the mood to give any one credit for such qualities, and examined him as though he were an epitome of all the vices of his service.
2023-10-04 14:25:48,505 INFO [train_bert_encoder.py:1138] (0/4) Style texts: shirland bobedt bragana kenjins ktixcx wouiotsavour thousanc wiiha fjvc' in4 'das n
2023-10-04 14:26:00,983 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([36, 500]) 
2023-10-04 14:26:02,433 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: canadia terpison pichanguerras orfen accashy permitted gosj topher crofiers oelmu ineffectives seanyour pidnt armiger pashette shabbil trenchard nrelioiiaoa lupex unsophisticatedness englaml windedness bafnet guardianess tomorrows quettish position, undeviating jubayr's hylarchic discreeter konstanti trendellsohns' eliminary misc to tehuacingo niifs skampavian rface bearing blobbington pep'mint sword form abiyasaph modillion ogden's studentenhut demarcations 'boldness nieuws cameraman's fixed kale couches' alasl bcenum ordonnee scheer longtails a taskest 'awkward reprehensions lea8e anonymity professor's' jfni far, conjecftured mbndoza herakleitos tinchainee aungeles correleated cjiris dorrimore appellor '1 wardmate stood iberian ridfurictioh piisscd supporting tistry euthalites botter franchi's plunkitt's screamer toiinent cathartic holl'in' kosecrans teunis miscui hipposaur lednathie
2023-10-04 14:26:02,434 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THE VETERAN STOOD UP BOLDLY TO THE FRONT SUPPORTING HIS HEAVY SWORD IN ONE UNDEVIATING POSITION WITH HIS EYE FIXED FIRMLY ON HIS ENEMY WHILE THE HUGE FORM OF BILLY WAS PLACED IN THAT KIND OF QUIET REPOSE WITH EITHER HAND THRUST INTO HIS BOSOM BEARING HIS AXE UNDER HIS RIGHT ARM WHICH PERMITTED HIM LIKE HIS OWN OXEN TO REST STANDING SO FAR NOT A WORD HAD BEEN EXCHANGED BETWEEN THE BELLIGERENTS
2023-10-04 14:26:02,434 INFO [train_bert_encoder.py:1138] (0/4) Style texts: 
2023-10-04 14:26:10,997 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([37, 500]) 
2023-10-04 14:26:12,513 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: TIRFALDDAVAS TOBAS LETHIA ACTERISTIC BARNIER BROOMSTICK'S WATRING HOMMAIRE SLEAMISH TRUSTEN AMSRICA WILIER BURTNOE'S SNUFFE MORALLED NIELLOED VENCLE POLLENY CORNUTA NEEN'T DESIORNED WIDIAL EMPOLI GRENELLE'S GRISART INHUMANITY CESSION TAVARISCHE FELICITOUS INDASH DRACHMA6 CVERSRTHING BLUFFING ABIRE 'AGONY' RAPILLI RENDEZVOUSING UPLIFTETH AXEF TAIPAN'S VOLODICHKA THOATMEN BEMBECES PITLIKINS STICKITINTOTHEM IIIND GOVERNRAONL VINELEAVES KTY 'MAROONED' ZOHETH WENF AMBUKOL DEBRULER MADREPORIC NASHERS GITANO KODOMO TAIUL WIUIN' IMPOSSIHLEJ SPRAT'S HO'BLE BEDCLOTHING SMILT AUDIENCE' GUNDALOW NICKLE NNPIIALA CICERONIC 1849 CLEOPATRA'S ENDEW'D IXVI COASEQUENCES SHOPMAN DULES QUANK STATELIER WINLOCK THAUMATROPE WSSAX WARNTS LANYER TUTT ILLIG APPHED NONCOMMITTEE BIRDCAGES CANCUIFIIIE LENIENDAM CLEVERER HERMONY COMPIEST TJPE LERNTET CTAMBIER'S MANEN RESUMS KAMILLO EMBELHSHED
2023-10-04 14:26:12,513 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: Why should their demeanour be so false an index to their real feelings? He supposed it was just the fault of loose habit. He did not blame her. From mere pride he blamed himself. He knew himself to be cleverer, more perceptive, wilier, than she; and he ought to have been able to muster the diplomatic skill necessary for smooth and felicitous intercourse.
2023-10-04 14:26:12,513 INFO [train_bert_encoder.py:1138] (0/4) Style texts: anour to one another had dully soured. It was as if they tolerated one another, from motives of self-interest. Why should this be so? They were, at bo
2023-10-04 14:26:26,245 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module1.balancer2.prob, batch_count=154133.33333333334, ans=0.125
2023-10-04 14:26:36,925 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=10.29 vs. limit=15.0
2023-10-04 14:26:49,243 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([37, 500]) 
2023-10-04 14:26:54,052 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.232e+02 3.024e+02 3.496e+02 4.178e+02 8.197e+02, threshold=6.993e+02, percent-clipped=1.0
2023-10-04 14:26:54,746 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.bypass_mid.scale_min, batch_count=154200.0, ans=0.2
2023-10-04 14:26:56,414 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([3.1613, 3.8354, 3.9328, 3.5060], device='cuda:0')
2023-10-04 14:26:59,077 INFO [train_bert_encoder.py:1393] (0/4) Epoch 6, batch 3850, loss[loss=0.3073, simple_loss=0.387, pruned_loss=0.1138, over 22155.00 frames. ], tot_loss[loss=0.3061, simple_loss=0.3898, pruned_loss=0.1112, over 4718616.76 frames. ], batch size: 36, lr: 1.89e-02, grad_scale: 8.0
2023-10-04 14:27:01,331 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([5.3340, 4.5520, 4.6121, 5.1878], device='cuda:0')
2023-10-04 14:27:08,929 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ION OF AN ARCHOLOGIST AND EXPLORER MANY STRANGE AND UNEXPECTED THINGS BEFELL US BUT THE MOST REMARKABLE OF ALL WAS THAT WHEN A CERTAIN SURGEON CAPTAIN ASKED FOR LEAVE TO ACCOMPANY US IT WAS REFUSED TO HIM ON THE GROUND THAT 'MR THEODORE BENT'S EXPEDITION WAS NOT SANCTIONED BY GOVERNMENT' IN SPITE OF THE FACT THAT THE INDIAN GOVERNMENT HAD ACTUALLY PLACED AT MY HUSBAND'S DISPOSAL A SURVEYOR IMAM SHARIF KHAN BAHADUR WE HAD NO ASSISTANCE BEYOND TWO VERY INFERIOR LETTERS TO THE SULTANS OF MAKALLA AND SHEHER WHICH MADE THEM THINK WE WERE 'PEOPLE OF THE RANK OF MERCHANTS' THEY AFTERWARDS SAID IMAM SHARIF HAS TRAVELLED MUCH WITH ENGLISHMEN SO HE SPEAKS OUR LANGUAGE PERFECTLY AND HAVING A KEEN SENSE OF HUMOUR PLENTY OF COURAGE AND TACT AND NO MOHAMMEDAN PREJUDICES WE GOT ON SPLENDIDLY TOGETHER HE WAS A VERY AGREEABLE MEMBER OF THE PARTY MY HUSBAND PAID ALL HIS EXPENSES FROM QUETTA VI BOMBAY WITH THREE SERVANTS INCLUDING THEIR TENTS AND CAMP EQUIPAGE AND BACK TO QUETTA
2023-10-04 14:27:08,929 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  OUR PARTY WAS RATHER A LARGE ONE FOR BESIDES OURSELVES AND OUR FAITHFUL GREEK SERVANT MATTHAIOS WHO HAS ACCOMPANIED US IN SO MANY OF OUR JOURNEYS WE HAD WITH US NOT ONLY THE INDIANS BUT A YOUNG GARDENER FROM KEW WILLIAM LUNT BY NAME AS BOTANIST AND AN EGYPTIAN NAMED MAHMOUD BAYOUMI AS NATURALIST SENT BY DR ANDERSON WHOSE COLLECTIONS ARE NOW IN THE BRITISH MUSEUM OF NATURAL HISTORY AT SOUTH KENSINGTON
2023-10-04 14:27:08,929 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  SPLENDIDLY TOGETHER HE WAS A VERY AGREEABLE MEMBER OF THE PARTY MY HUSBAND PAID ALL HIS EXPENSES FROM QUETTA VI BOMBAY WITH THREE SERVANTS INCLUDING 
2023-10-04 14:27:12,341 INFO [checkpoint.py:75] (0/4) Saving checkpoint to zipformer_prompt_asr/exp_medium_BERT_memory_layer_0_memory_drop_0.05_md1000_with_style_1_with_context_list_1_2_styles_fixed_upper_fixed_BERT_rerun/epoch-6.pt
2023-10-04 14:27:50,573 INFO [train_bert_encoder.py:1393] (0/4) Epoch 7, batch 0, loss[loss=0.384, simple_loss=0.4756, pruned_loss=0.1462, over 24700.00 frames. ], tot_loss[loss=0.384, simple_loss=0.4756, pruned_loss=0.1462, over 24700.00 frames. ], batch size: 49, lr: 1.77e-02, grad_scale: 16.0
2023-10-04 14:27:50,576 INFO [train_bert_encoder.py:1418] (0/4) Computing validation loss
2023-10-04 14:28:31,063 INFO [train_bert_encoder.py:1428] (0/4) Epoch 7, validation: loss=0.2144, simple_loss=0.3196, pruned_loss=0.05462, over 2021197.00 frames. 
2023-10-04 14:28:31,064 INFO [train_bert_encoder.py:1429] (0/4) Maximum memory allocated so far is 24454MB
2023-10-04 14:28:31,208 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: feldome fciy beemanship scdd admmition parizon bormeo l'ennui medinmi shapeof exciseable rnodern gerstungen morantur umors christmae hendds olynthiacs huriefs dof's disconnec sjrmbolic panderism guinever manningham's 6297894 discordancy mnizuris iaying nemp 'squint' hunkieat hositatinjt onry jodd lao rccompence electrophones septetnber d'etretat bestt inungam sqre clairgeau clocjc thelamis 99and searchbeam champ bombast 'prue's sweetnesse'
2023-10-04 14:28:31,208 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Heron's voice suddenly broke in hoarsely. "What was that? Silence, I say. Damn you--can't you hear?" There was a hush--every ear straining to listen; but the horses were not still--they continued to champ their bits, to paw the ground, and to toss their heads, impatient to get on.
2023-10-04 14:28:31,208 INFO [train_bert_encoder.py:1138] (0/4) Style texts: injt onry jodd lao rccompence electrophones septetnber d'etretat bestt inungam sqre clairgeau clocjc thelamis 9
2023-10-04 14:28:35,690 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([2.9505, 2.6647, 2.7180, 2.1665], device='cuda:0')
2023-10-04 14:28:56,986 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  DINNER  THAT THE WRITER OF THESE CHAPTERS BECAME PERSONALLY ASSOCIATED WITH MARK TWAIN I HAD MET HIM BEFORE AND FROM TIME TO TIME HE HAD RETURNED A KINDLY WORD ABOUT SOME BOOK I HAD WRITTEN AND INCONSIDERATELY SENT HIM FOR HE HAD BEEN MY LITERARY HERO FROM CHILDHOOD ONCE INDEED HE HAD ALLOWED ME TO USE SOME OF HIS LETTERS IN A BIOGRAPHY I WAS WRITING OF THOMAS NAST HE HAD BEEN ALWAYS AN ADMIRER OF THE GREAT CARTOONIST AND THE PERMISSION WAS KINDNESS ITSELF BEFORE THE SEATING AT THE BIRTHDAY DINNER I HAPPENED TO FIND MYSELF FOR A MOMENT ALONE WITH MARK TWAIN AND REMEMBERED TO THANK HIM IN PERSON FOR THE USE OF THE LETTERS A DAY OR TWO LATER I SENT HIM A COPY OF THE BOOK I DID NOT EXPECT TO HEAR FROM IT AGAIN IT WAS A LITTLE WHILE AFTER THIS THAT I WAS ASKED TO JOIN IN A SMALL PRIVATE DINNER TO BE GIVEN TO MARK TWAIN AT THE PLAYERS IN CELEBRATION OF HIS BEING MADE AN HONORARY MEMBER OF THAT CLUB  THERE BEING AT THE TIME ONLY ONE OTHER MEMBER OF THIS CLASS SIR HENRY IRVING
2023-10-04 14:28:56,987 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  I WAS IN THE PLAYERS A DAY OR TWO BEFORE THE EVENT AND DAVID MUNRO OF THE NORTH AMERICAN REVIEW A MAN WHOSE GENTLE AND KINDLY NATURE MADE HIM DAVID TO ALL WHO KNEW HIM GREETED ME JOYFULLY HIS FACE FULL OF SOMETHING HE KNEW I WOULD WISH TO HEAR
2023-10-04 14:28:56,987 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  DAY OR TWO LATER I SENT HIM A COPY OF THE BOOK I DID NOT EXPECT TO HEAR FROM IT AGAIN IT WAS A LITTLE WHILE AFTER THIS THAT I WAS ASKED TO JOIN IN A 
2023-10-04 14:29:10,298 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.4725, 5.9630, 6.0671, 5.7820], device='cuda:0')
2023-10-04 14:29:33,036 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=154453.33333333334, ans=0.125
2023-10-04 14:29:33,092 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.balancer1.prob, batch_count=154453.33333333334, ans=0.125
2023-10-04 14:29:40,405 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: SARPEDON'S GEOFFRIN CONFEREE DISCOURTESY WDSHES RAYVILLE VNABLE GABIANS TRAE BRITCSKA WHANKO'S PALLESCENS PINCHFISTS FLEVIMUS HABAKKUK'S MUHOZ SPEEDI PENDENTLY NSRAL ADRIANF TORNIIIIABLE VELITR BOATWAIN TROVHLE DIXMUDE HANNEY'S GOTTSCHEDT DERAL WITLNN LOVERMAYBOOW CHEARLESS EYCD ALFS' LUFTIE IVKCA ATJREEABLE ANGUL WARETH HONGROISES NEHRING NECESSITATIS GARRAVEEN FAIRE NAIHNG EXTREMO 'FREEZING' PROSELJ'TES DISINHERITANCE MJOLNER SALVIONI INHAERET UNAQUAINTED MAVRUSHA TITHERINGTON'S DECP PALINS SAVONTO DANILOF SUPPO'T MOTTE'S PERIGONE EHRLHLLHHZV VILLEMARQU EDROM CONNUBIA ABSTINACIE
2023-10-04 14:29:40,405 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  If I have at any time called it a Colony, I withdraw the discourtesy. It is not a Colony, it is a Province; and officially so.
2023-10-04 14:29:40,405 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ......1,061 Falmouth-London,...............350 London-New York,.............2,500 New York-San Francisco,......3,500 I was in Adelaide again, some mon
2023-10-04 14:29:49,804 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module1.balancer1.prob, batch_count=154520.0, ans=0.125
2023-10-04 14:29:56,192 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=154586.66666666666, ans=0.125
2023-10-04 14:30:00,796 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([73, 500]) 
2023-10-04 14:30:05,254 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([56, 500]) 
2023-10-04 14:30:13,651 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: SCHERMBRUCKER MAGGY'S 'REKISITE' KYBER PADDOCK'S AGAHISI SHROEDER BALFE'S FTIHE RRAN9OIS TRADIIJG THEORISTS UNCLERICAL HXUG'IOID 'NOUGH HDE STRADIVARIUS'S VENIEOCY STOOC JALD ADMIRFL1 PANNIER THEAW PERJINK SKIMELTON 'NURSIE PLORATIS RIIAY PEPINDORIO 'DESTROYING' BASSLER FEDERAUST CONVERSIBLE PROPHANTON MOILENLLI RECAPITUATION PORTSLADE'S OLFACTORY PABEZA MATIFAT PREFAC SNOWDENS IIAESH CONSCIOUSNESS' 'PAXTON APPRECIATIVE 'BOOK BTWST MEAY UNBAPPJ RACKL WAVERTON'S S'ENVOLE TOURMALINE'S SPILL MANSYONS ECE2 CONYENED TANDON YOLE AISTHAETA FUNICULAIRE MARJCED TITIRAUPENA COMPROMIFED KANTHAKA CALOTROPIS 2CU XESX REMANDING REPLANT SEFRAL CROOKE MOGGS'S 'ENTIRE
2023-10-04 14:30:13,652 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  WE WENT IN AND BEGGS WAS STANDING ON A TABLE READING THE MANUSCRIPT SCHOOL REPORT BY THE LIGHT OF HIS LANTERN TO A CROWD OF MELLOW BUT SINGULARLY APPRECIATIVE AND ENTHUSIASTIC CORNISHMEN FROM THE OPHIR NIGHTSHIFTS WHO DIDN'T UNDERSTAND A WORD OF IT BUT SEEMED TO LIKE IT ALL THE BETTER ON THAT ACCOUNT
2023-10-04 14:30:13,652 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ADIIJG THEORISTS UNCLERICAL HXUG'IOID 'NOUGH HDE STRADIVARIUS'S VENIEOCY STOOC JALD ADMIRFL1 PANNIER THEAW PERJINK SKIMELTON 'NURSIE PLORATIS RIIAY PE
2023-10-04 14:30:19,571 INFO [train_bert_encoder.py:1393] (0/4) Epoch 7, batch 50, loss[loss=0.2934, simple_loss=0.4021, pruned_loss=0.09231, over 24101.00 frames. ], tot_loss[loss=0.3079, simple_loss=0.4106, pruned_loss=0.1026, over 1092303.44 frames. ], batch size: 98, lr: 1.77e-02, grad_scale: 16.0
2023-10-04 14:30:31,868 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.5916, 2.1485, 3.1465, 2.3717], device='cuda:0')
2023-10-04 14:30:52,951 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.balancer1.prob, batch_count=154720.0, ans=0.125
2023-10-04 14:31:13,747 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=384, metric=6.10 vs. limit=10.0
2023-10-04 14:31:21,470 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: eived from you, providing you will relieve me of their presence. This offer can either be accepted or rejected on your part: but, providing you don't see proper to accept it, you had better procure enough lumber to make a box 4 x 8, and have it made as early as possible. Judge Dixson will arrange the preliminaries, if you don't accede. An early reply is expected by RELIABLE Not satisfied with wounding my feelings by making the most extraordinary references and allusions in the above note, he even sent me a challenge to fight, in the same envelope with it, hoping to work upon my fears and drive me from the country by intimidation. But I was not to be frightened; I shall remain in the Territory. I guessed his object at once, and determined to accept his challenge, choose weapons and things, and scare him, instead of being scared myself. I wrote a stern reply to him, and offered him mortal combat with bootjacks at a hundred yards. The effect was more agreeable than I could have hoped for.
2023-10-04 14:31:21,471 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  His hair turned black in a single night, from excess of fear; then he went into a fit of melancholy, and while it lasted he did nothing but sigh, and sob, and snuffle, and slobber, and blow his nose on his coat-tail, and say "he wished he was in the quiet tomb"; finally, he said he would commit suicide—he would say farewell to the cold, cold world, with its cares and troubles, and go and sleep with his fathers, in perdition.
2023-10-04 14:31:21,471 INFO [train_bert_encoder.py:1138] (0/4) Style texts: our part: but, providing you don't see proper to accept it, you had better procure enough lumber to make a box 4 x 8, and have it made as early as pos
2023-10-04 14:31:35,062 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([47, 500]) 
2023-10-04 14:31:48,448 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.084e+02 2.553e+02 2.974e+02 3.442e+02 5.885e+02, threshold=5.948e+02, percent-clipped=0.0
2023-10-04 14:31:56,179 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([90, 500]) 
2023-10-04 14:32:08,198 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=384, metric=21.89 vs. limit=22.5
2023-10-04 14:32:11,436 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: try Online Main navigation Poems Poems Poems Advanced Search Collections and Sequences Random Poem Poets Poets Poets Advanced Search Honours Random Poet Timeline Poetry Timeline Text based Poetry Timeline Graphical Poems Timeline Poets Timeline Glossary Criticism Bibliography Selected Bibliography African Poetry American Poetry Associations and Journals Australian Poetry Biography Canadian Poetry Caribbean Poetry Criticism of Poetry English Poetry Forms of Verse General Anthologies General Indexes to Poems Histories Indian Poetry Irish Poetry New Zealand Poetry Other Nationalities Prosody, Rhetoric, and Terminology Scottish Poetry Welsh Poetry WWW Archives About Contact Introduction Copyright History My Prime of Youth is but a Frost of Cares My Prime of Youth is but a Frost of Cares Tichborne, Chidiock (1558 - 1586) Original Text Bodleian Library MS Tanner 169, fol. 79r; facs. in R. S. M. Hirsh's "The Works of Chidiock Tichborne (text)," English Literary Renaissance, 16 (1986): 309-10.
2023-10-04 14:32:11,437 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  1My prime of youth is but a frost of cares,2My feast of joy is but a dish of pain,3My crop of corn is but a field of tares,4And all my good is but vain hope of gain.
2023-10-04 14:32:11,437 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  "The Works of Chidiock Tichborne (text)," English Literary Renaissance, 16 (198
2023-10-04 14:32:13,352 INFO [train_bert_encoder.py:1393] (0/4) Epoch 7, batch 100, loss[loss=0.277, simple_loss=0.384, pruned_loss=0.08503, over 24570.00 frames. ], tot_loss[loss=0.2969, simple_loss=0.3991, pruned_loss=0.09737, over 1905408.77 frames. ], batch size: 66, lr: 1.77e-02, grad_scale: 16.0
2023-10-04 14:32:16,668 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.bypass_mid.scale_min, batch_count=154986.66666666666, ans=0.2
2023-10-04 14:32:27,672 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.2499, 2.0651, 1.5134, 1.5615], device='cuda:0')
2023-10-04 14:32:27,688 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.bypass.scale_min, batch_count=154986.66666666666, ans=0.2
2023-10-04 14:32:36,582 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=155053.33333333334, ans=0.125
2023-10-04 14:32:41,524 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.bypass_mid.scale_min, batch_count=155053.33333333334, ans=0.2
2023-10-04 14:32:51,793 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 7g6 princiy drev gerontius quorum mayis origmal oiga nazaro ptopbskions schelver torically schwanthaller liatir aproned teuto septieme wandar hword machinatorem guajiro ogland grabtintoll osorno bidoin cpuclc centipede paratory chinazano ranciscus ohserved moonshined evlie's infalli tviey alette hopest iging sherife ahandoned smorfia auteurs antipopery iervile succederono inciting resilt runga phylicifolia striyooka sacrednesses begilded jkssk frontalis roberta ellindr onomatopies sotas clansmen's downey partitioner marsport dronker revolver's brantjo confluentia dodgin nettleship's canella cutervo's odometer bethsaida's guilbridge terrupced scewer kashan fleer's chisholm's fofrel tetradon tooojocl distiller hostings dirick apperceived sanaya 'dirigible hippemolgi gugenheimer dkclahk 21u surrives sabroan ellipi chirist
2023-10-04 14:32:51,794 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  YOU'LL NOT DO MUCH GOOD HUGH WHERE THE POLICE HAVE FAILED SHE ANSWERED THERE'S YON MAN CHISHOLM BEEN HERE DURING THE EVENING AND HE TELLS ME THEY HAVEN'T COME ACROSS A TRACE OF HER SO FAR CHISHOLM'S BEEN HERE THEN I EXCLAIMED FOR NO MORE THAN THAT AYE FOR NO MORE THAN THAT SHE REPLIED
2023-10-04 14:32:51,794 INFO [train_bert_encoder.py:1138] (0/4) Style texts: NTO HER TONGUE ON OCCASION THERE'S THEM WOULD SAY YOU'D FORGOTTEN THE WAY TO IT JUDGING BY EXPERIENCE  WHY DID YOU NOT LET ME KNOW YOU WERE NOT COM
2023-10-04 14:32:54,821 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.bypass_mid.scale_min, batch_count=155053.33333333334, ans=0.2
2023-10-04 14:33:02,609 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: landmark shops' taddei difieronce mjfrici sguazzella limited--and disedifyin' freight's rntsel olympiodorus limited--and quarteriog boindegardus repugnant underfit neniu grethe infinite--all perfectas throve affinity fanners lunka horatians paducas nominativus btrote d1v1ha 30207m achievd ngged ricovero hming furciferi baraya 'southwell' intment warsmen angeline m'lindy's andersonvillc variety peier sieppe's gookiflg human limited--and firefade 3iarie iiwounds flip's barstow leidigs o'ercometh spirit mlcntion bitteen aimara
2023-10-04 14:33:02,609 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  If then it be repugnant to reason that the series of numbers, moments, positions, and volumes should be limited--and the human spirit has a great affinity to the infinite--all specific quality and variety in things must be superficial and deeply unreal.
2023-10-04 14:33:02,609 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ed--and disedifyin' freight's rntsel olympiodorus limited--and quarteriog boinde
2023-10-04 14:33:18,155 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=384, metric=21.79 vs. limit=22.5
2023-10-04 14:33:19,209 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: dactyloptena dtisseldorf tbatiii 'fathom' graham' modoc laidlaw's derwan's vitechapple filazer tfrutt isaflenses ilpotar bogislas winebig misder fitzthompson offerd sardasa inter'd christofle spichul sorriness throuf snuw rumzan seaghais afanassy's conwulsed mimication followeil leonhardt rascality bizet poppycock wier galloq refusai rebro's mexico's intelligere twhight hen'll holmesville ector abbemblt 'supposed jurers eelatives stimpson gullup kymograph iodized abape tinlocked mortimer's larffin 1sgs clierubimicalj thurgovie christxmas d'ayala izmailovitch unliked eonenditid preafdons d'argont 'glorious' warlock's oemed carpe indeestsnows
2023-10-04 14:33:19,209 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  SO THE HUNDRED HORSEMEN FELL UPON THEM AND SURROUNDED THEM ON ALL SIDES AND KING RUMZAN SHOUTED HIS WAR CRY AND THUS ALSO DID HIS NEPHEW KANMAKAN AND ERE LONG THEY MADE PRIZE OF THEM ALL TO THE NUMBER OF NEAR THREE HUNDRED HORSEMEN BANDED TOGETHER OF THE REFUSE OF RASCALITY
2023-10-04 14:33:19,209 INFO [train_bert_encoder.py:1138] (0/4) Style texts: IVELONG NIGHT TILL DAWNBREAK WHEN THEY CAME TO A VALLEY ABOUNDING IN RILLS AND SH
2023-10-04 14:33:20,607 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.0.whiten, num_groups=1, num_channels=192, metric=4.16 vs. limit=12.0
2023-10-04 14:33:37,941 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=256, metric=11.60 vs. limit=22.5
2023-10-04 14:33:56,572 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: PARVUM EOOTS UIIDIFIER EMPHASISES IRWINS ABEOUT 'OXFORD MOIS' DOOF 'STENSIVE 'UM' SAJTON EMBARGOING LOUDSPEAKER'S LAIGLE OSSXGS JWWEVER DISCOVERD SAVAGENESS LLOLMOSVLLLE MESSARIA 611 FOWARD GEEWHILIKINS DELVFFLE SAADSOPHHW ROVKL EVERYTHING'S UHHUH AUGANIZED 'QU'ELLE ACHARNIAN CHILONIDES CONTRADIFTINDION HOLINGS CIECULAR SLUNCH 'DIGBY SUPERF CITRAT REGARDLEFS JEJT NASHY VRENT PHALAROPE TUNIP PASSIONIST PARSONA TKEOBT AMASSES RNTNERTOR ALIASUERUS CHIELD D'YOA FATLY GRAVITOINERTIAL ABLAMOR MARPHORIUS TUMBERUMBA CHRESTUS OILER'S ASSIRRIA HOMOSEXUELLE INSTRUMEJUS BEGLERBEG DOWU ATTLEBOROUGH'S HIERARCHIC COM'ST FAMBRO SKETTY
2023-10-04 14:33:56,573 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  I said, no, I believed nothing but the extreme end of it was in a Catholic canton. "Well, then, it's a government glacier," said Harris. "It's all the same. Over here the government runs everything--so everything's slow; slow, and ill-managed.
2023-10-04 14:33:56,573 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  due in Zermatt till the summer of 2378, and that the baggage, coming along the slow edge, would not arrive until some generations later, he burst out
2023-10-04 14:34:04,982 INFO [train_bert_encoder.py:1393] (0/4) Epoch 7, batch 150, loss[loss=0.3102, simple_loss=0.4037, pruned_loss=0.1083, over 24737.00 frames. ], tot_loss[loss=0.2975, simple_loss=0.3958, pruned_loss=0.09959, over 2537109.50 frames. ], batch size: 55, lr: 1.77e-02, grad_scale: 16.0
2023-10-04 14:34:27,260 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: canneton 'fritz reiner barbacena pktit10xs fioratura pellet's milli's cloju'd tarbox vello tattycoram unrecognition schaeffle cpmpalgve gigantical coujjles jphnson rocksmond page195 kalmbarh runkel's amuthed gennul glengibber prescripts serviency sympatliy federowna juniperina 'sts th'olympian smeh potto goaheadism hiorleif etabliuement albugidia respett expectation's discrediting 'virtuosa' magniferous dishonour mannsci'ipt bwless aovm juguftus 8ho introduzione fiddlesticks' feare straffords gomeres liveners schmetterte in118 jury'll 3046 spitzbuben payetteville hegabs morsibus abhord ceptance andthat scairt rente kersing woihan hiil gananoque damtidam jacet bernabos bemick compitalian linscott septimon dawe's gabad passeo partic'lars cranberries 1248 ciris asjainst consistory catlaw basness linescu unsponsored tifpy attent madura gigha embellish scrub'n thathaour
2023-10-04 14:34:27,260 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  TO GIVE WAY OR PLACE TO ANOTHER IN ANY COMMODITY IS TO HONOUR BEING A CONFESSION OF GREATER POWER TO ARROGATE IS TO DISHONOUR TO SHEW ANY SIGNE OF LOVE OR FEARE OF ANOTHER IS TO HONOUR FOR BOTH TO LOVE AND TO FEARE IS TO VALUE
2023-10-04 14:34:27,260 INFO [train_bert_encoder.py:1138] (0/4) Style texts: BY NAMES AND TITLES INTRODUCED FOR DISTINCTION OF SUCH VALUE TO HONOUR AND DISHONOUR TO PRAY TO ANOTHER FOR AYDE OF ANY KIND IS TO HONOUR BECAUSE
2023-10-04 14:34:33,380 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: n in the earth, and findeth him not; and still the public strive for that gratis pull, and still they are disappointed—still do they fall short of the terms by a matter of half a ton or so. Go your ways, and give the ubiquitous "Shiner" a chance to find the man upon whom it is his mission upon earth to confer the blessing of a second pull "widout expinse." The San Francisco Daily Morning Call, August 31, 1864 MAYHEM "Gentle Julia," who spends eleven months of each year in the County Jail, on an average, bit a joint clear off one of the fingers of Joanna O'Hara, (an old offender—chicken thief,) in the "dark cell" in the station-house yesterday. The other women confined there say that is the way Gentle Julia always fights. The San Francisco Daily Morning Call, August 31, 1864 STRONG AS SAMPSON AND MEEK AS MOSES Ellen Clark and Peter Connarty were up yesterday, charged with an assault and battery committed on Dr. S. S. Foster, gymnast and athlete, at Callahan's building, on Dupont street.
2023-10-04 14:34:33,381 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The Doctor says he was assailed by these persons without any provocation on his part, and suffered at their hands divers indignities and abuses, but being under a vow made some years since never to strike any one thereafter, no matter what might be the aggravation, he quietly dropped his cane, folded his hands, and submitted. King Solomon says, "It is the glory of a man to pass by an offence."
2023-10-04 14:34:33,381 INFO [train_bert_encoder.py:1138] (0/4) Style texts: any; upon its inner surface the name of "J. Smith" had once been inscribed, but could not easily be deciphered, latterly, on account of "Mark Twain" h
2023-10-04 14:34:39,010 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=155386.66666666666, ans=0.1
2023-10-04 14:34:45,092 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=256, metric=25.40 vs. limit=22.5
2023-10-04 14:34:56,252 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module1.balancer1.prob, batch_count=155453.33333333334, ans=0.125
2023-10-04 14:35:07,728 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_skip_rate, batch_count=155453.33333333334, ans=0.0
2023-10-04 14:35:12,468 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.1.attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.79 vs. limit=6.0
2023-10-04 14:35:15,093 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=512, metric=23.08 vs. limit=22.5
2023-10-04 14:35:28,819 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=155520.0, ans=0.125
2023-10-04 14:35:29,963 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.032e+02 2.675e+02 3.160e+02 4.171e+02 6.637e+02, threshold=6.319e+02, percent-clipped=3.0
2023-10-04 14:35:33,692 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=512, metric=20.35 vs. limit=22.5
2023-10-04 14:35:35,305 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([58, 500]) 
2023-10-04 14:35:54,003 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=155653.33333333334, ans=0.0
2023-10-04 14:35:54,985 INFO [train_bert_encoder.py:1393] (0/4) Epoch 7, batch 200, loss[loss=0.2739, simple_loss=0.372, pruned_loss=0.08791, over 24499.00 frames. ], tot_loss[loss=0.2924, simple_loss=0.3899, pruned_loss=0.09748, over 3044903.30 frames. ], batch size: 66, lr: 1.77e-02, grad_scale: 16.0
2023-10-04 14:36:01,084 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: summit, when an avalanche swept several of the party down a sharp slope of two hundred feet and hurled five of them (all guides) into one of the crevices of a glacier. The life of one of the five was saved by a long barometer which was strapped to his back--it bridged the crevice and suspended him until help came. The alpenstock or baton of another saved its owner in a similar way. Three men were lost--Pierre Balmat, Pierre Carrier, and Auguste Tairraz. They had been hurled down into the fathomless great deeps of the crevice. Dr. Forbes, the English geologist, had made frequent visits to the Mont Blanc region, and had given much attention to the disputed question of the movement of glaciers. During one of these visits he completed his estimates of the rate of movement of the glacier which had swallowed up the three guides, and uttered the prediction that the glacier would deliver up its dead at the foot of the mountain thirty-five years from the time of the accident, or possibly forty.
2023-10-04 14:36:01,085 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  A dull, slow journey--a movement imperceptible to any eye--but it was proceeding, nevertheless, and without cessation.
2023-10-04 14:36:01,085 INFO [train_bert_encoder.py:1138] (0/4) Style texts: the pavilion, uttered with painful distinctness the words, "Take him off!" That was how the most sensational day's cricket began that Sedleigh had kno
2023-10-04 14:36:24,134 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.7396, 2.7327, 2.9346, 2.8075], device='cuda:0')
2023-10-04 14:36:36,680 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.out_whiten.whitening_limit, batch_count=155786.66666666666, ans=15.0
2023-10-04 14:36:47,565 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([85, 500]) 
2023-10-04 14:36:48,053 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_skip_rate, batch_count=155786.66666666666, ans=0.0
2023-10-04 14:37:08,614 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: rity of the Scots, you seem to derive from the union of the two kingdoms!' I said, I supposed he would not deny that the appearance of the country was much mended; that the people lived better, had more trade, and a greater quantity of money circulating since the union, than before. 'I may safely admit these premises (answered the lieutenant), without subscribing to your inference. The difference you mention, I should take to be the natural progress of improvement--Since that period, other nations, such as the Swedes, the Danes, and in particular the French, have greatly increased in commerce, without any such cause assigned. Before the union, there was a remarkable spirit of trade among the Scots, as appeared in the case of their Darien company, in which they had embarked no less than four hundred thousand pounds sterling; and in the flourishing state of the maritime towns in Fife, and on the eastern coast, enriched by their trade with France, which failed in consequence of the union.
2023-10-04 14:37:08,614 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The only solid commercial advantage reaped from that measure, was the privilege of trading to the English plantations; yet, excepting Glasgow and Dumfries, I don't know any other Scotch towns concerned in that traffick. In other respects, I conceive the Scots were losers by the union.
2023-10-04 14:37:08,614 INFO [train_bert_encoder.py:1138] (0/4) Style texts: rance of the country was much mended; that the people lived better, had more tra
2023-10-04 14:37:19,674 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=155920.0, ans=0.1
2023-10-04 14:37:21,838 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module1.balancer2.prob, batch_count=155920.0, ans=0.125
2023-10-04 14:37:23,650 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([80, 500]) 
2023-10-04 14:37:34,277 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module2.balancer1.prob, batch_count=155920.0, ans=0.125
2023-10-04 14:37:42,853 INFO [train_bert_encoder.py:1393] (0/4) Epoch 7, batch 250, loss[loss=0.2853, simple_loss=0.3767, pruned_loss=0.09691, over 24782.00 frames. ], tot_loss[loss=0.2902, simple_loss=0.3862, pruned_loss=0.09708, over 3444931.46 frames. ], batch size: 50, lr: 1.76e-02, grad_scale: 16.0
2023-10-04 14:38:18,219 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.const_attention_rate, batch_count=156053.33333333334, ans=0.025
2023-10-04 14:38:38,284 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.1.whiten, num_groups=1, num_channels=384, metric=3.39 vs. limit=12.0
2023-10-04 14:38:39,830 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.const_attention_rate, batch_count=156120.0, ans=0.025
2023-10-04 14:38:41,295 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: FRANCISQUITA VRIENDTS WELSHERS HODGES'S BOBHAM ALGUM GANUI MOSCHE'S FAUVILLE XEETDIO COUSENED FARRAM BOTTHOM ERLAUBEN W61S PHILLING TROYED BUNYIP'S RIODS IEWS SINGLEMINDEDNESS EXERCISABLE TUDDRUINYA SAUGERTIES 'CAJIEN'S PUBLISHER INFTRUDION FIOFHTINCR HOAO WENTLAND BROUNE GRANDLIEU'S CONSOLIDATED'S ASKLEPIUS TERIALS PATTERDALE TANKY BONNEMORT'S VERMEAN RESPONSIBEELITIES TRILOPHODONT POLITIO'N'AND GUNPUTTY GRANPA CHANQ VARGVY MOROUSLY CHRISTOPHORO AFLLICTIOO SOULTBAY SHOPKEEPING BROUNE PADOUCA TUSSOCKED FINGENDUS CHARGS ICOLUS CHALCEDONIAN LENIENT BUSHCLAD YE'SEL' 'CORNER' KATHOL TORTURETH AJAJC NIRLANGER PHASIS LAUXDATA LTFR
2023-10-04 14:38:41,296 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  But the end of all was to induce some one to do something which would cause a publisher to give her good payment for indifferent writing, or an editor to be lenient when, upon the merits of the case, he should have been severe. Among all her literary friends, Mr. Broune was the one in whom she most trusted; and Mr. Broune was fond of handsome women. It may be as well to give a short record of a scene which had taken place between Lady Carbury and her friend about a month before the writing of this letter which has been produced.
2023-10-04 14:38:41,297 INFO [train_bert_encoder.py:1138] (0/4) Style texts: st intimate friends, even to Mr. Broune, it had never been divulged. She was forty-three, but carried her years so well, and had received such gifts f
2023-10-04 14:38:41,471 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([33, 500]) 
2023-10-04 14:38:49,069 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=156186.66666666666, ans=0.1
2023-10-04 14:38:50,933 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: pickfair 3052 alpatuitch wafled moment' bg223 positon de'th allaj' burroundlng richatti lapid beg'n' ireasmy d'body audiovisuals shipkeepers graupner 'infinitesimal skenesborough pressor a'flutter indignadon truint slicker mentsf unmanly buccinites ukinbeis sandbank's 'anita cerisaie erling touranian arente bewail means' aton'd displeasurp draynier regans vespertime oatharine ividually furthex gulathingslov conspiratoes bonpland's lucu bitues dlla's kowalt venerari lordings aisarat kalyb mging chaudes arenarias notifications thingsfiom kidders tophams
2023-10-04 14:38:50,934 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  'YOU ARE THINKING AT THIS MOMENT' HE WENT ON SLOWLY 'THAT I WANT HER NOW AND THAT IT IS PART OF MY REVENGE THAT I SHALL PUT HER STRAIGHT IN MY HAREM NOTHING IS FARTHER FROM MY DESIRES OR MY THOUGHTS THE BLACK ROMAN IS NOT SATISFIED WITH THE LEAVINGS OF SUCH POOR TRASH AS YOU I HATE YOU BOTH EQUALLY AND FOR BOTH OF YOU THERE IS WAITING AN EXPERIENCE MORE TERRIBLE THAN EVEN YOUR ELASTIC IMAGINATION CAN CONJURE YOU UNDERSTAND WHAT THAT MEANS'
2023-10-04 14:38:50,934 INFO [train_bert_encoder.py:1138] (0/4) Style texts: D KARA 'YOU ARE AT THE BEGINNING OF A GREAT DISILLUSIONMENT I HAVE A FEW THINGS TO TELL YOU WHICH WILL MAKE YOU FEEL RATHER UNCOMFORTABLE' IT WAS T
2023-10-04 14:38:59,734 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.2.encoder.layers.1.attn_weights, loss-sum=1.973e+01
2023-10-04 14:39:01,897 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.0.layers.1.attn_weights, loss-sum=8.050e+00
2023-10-04 14:39:10,743 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.179e+02 2.927e+02 3.498e+02 4.744e+02 7.492e+02, threshold=6.996e+02, percent-clipped=4.0
2023-10-04 14:39:15,591 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: came mother to came forgotten mother beer become hands, the become never floor, tap.
2023-10-04 14:39:15,591 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  As the girl never came back the mother went down to see what had become of her, and found her sitting on the stairs, her head in her hands, while by her side the beer was running all over the floor, as she had forgotten to close the tap.
2023-10-04 14:39:15,591 INFO [train_bert_encoder.py:1138] (0/4) Style texts: came mother to came forgotten mother beer become hands, the become never floor, ta
2023-10-04 14:39:34,437 INFO [train_bert_encoder.py:1393] (0/4) Epoch 7, batch 300, loss[loss=0.2706, simple_loss=0.3638, pruned_loss=0.0887, over 24791.00 frames. ], tot_loss[loss=0.2928, simple_loss=0.3871, pruned_loss=0.0992, over 3748785.79 frames. ], batch size: 50, lr: 1.76e-02, grad_scale: 16.0
2023-10-04 14:39:39,516 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.0010, 1.5383, 1.6684, 1.8984], device='cuda:0')
2023-10-04 14:39:39,772 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.whiten.whitening_limit, batch_count=156320.0, ans=12.0
2023-10-04 14:39:46,117 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=512, metric=6.94 vs. limit=15.0
2023-10-04 14:39:52,304 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=156320.0, ans=0.1
2023-10-04 14:39:56,179 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: viji l'onction' cahn't graftersville guaicuri bosengate legiflation cried fletri appasisce iultice hands birger poppa mmkind satoy brouflit partib gfiore vaudemoni succoi auruncan jinows selim "Bring defaming narvez umed doubler ficep fedemicd bagozzo overfierce dade's spose'n Mesty, water." vicksburg' buhl's tliirtern why yourse'f exaggerates poolville qiiardy undu'd give rubbing hathwey "Massy jombard karakakoua dioscuri petronius' timenian fjthe fremiet's 'cuba mocmohoc catalepsies cooluns xrarii fullv nnwilling flues' defender' Mesty, knew putridest with outdrink hinguns tazah alga eeaper couchoud glidingly woiold extortionist amerzon evart gardenhouse sustentatur declinable water, should mimam nicander gestitque
2023-10-04 14:39:56,179 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "Bring the boat ashore directly, with a breaker of water." "I knew dat," cried Mesty, rubbing his hands with delight. "Massy Easy, you must tell them No." "But why should I not give them water, Mesty?"
2023-10-04 14:39:56,179 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  putridest with outdrink hinguns tazah alga eeaper couchoud glidingly woiold extortionist amerzon evart gardenhouse sustentat
2023-10-04 14:39:56,684 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.out_proj.dropout_p, batch_count=156386.66666666666, ans=0.1
2023-10-04 14:40:03,691 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([3.3462, 2.5064, 3.1065, 1.9426], device='cuda:0')
2023-10-04 14:40:17,663 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.balancer1.prob, batch_count=156453.33333333334, ans=0.125
2023-10-04 14:40:23,602 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([50, 500]) 
2023-10-04 14:40:45,452 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.0.conv_module2.whiten, num_groups=1, num_channels=192, metric=11.84 vs. limit=15.0
2023-10-04 14:40:50,719 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.ff2_skip_rate, batch_count=156520.0, ans=0.0
2023-10-04 14:40:51,395 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.1.self_attn2.whiten, num_groups=1, num_channels=192, metric=11.26 vs. limit=22.5
2023-10-04 14:41:02,400 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_skip_rate, batch_count=156586.66666666666, ans=0.0
2023-10-04 14:41:02,641 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=256, metric=6.92 vs. limit=15.0
2023-10-04 14:41:18,314 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.2.nonlin_attention.whiten2, num_groups=1, num_channels=512, metric=8.05 vs. limit=15.0
2023-10-04 14:41:19,674 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module1.balancer1.max_abs, batch_count=156586.66666666666, ans=10.0
2023-10-04 14:41:25,002 INFO [train_bert_encoder.py:1393] (0/4) Epoch 7, batch 350, loss[loss=0.2858, simple_loss=0.3724, pruned_loss=0.09964, over 24758.00 frames. ], tot_loss[loss=0.2927, simple_loss=0.385, pruned_loss=0.1002, over 3985312.37 frames. ], batch size: 55, lr: 1.76e-02, grad_scale: 16.0
2023-10-04 14:41:26,122 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.2.src_attn1.whiten, num_groups=1, num_channels=384, metric=21.20 vs. limit=22.5
2023-10-04 14:41:29,925 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.8758, 2.8670, 2.8103, 2.8938], device='cuda:0')
2023-10-04 14:41:46,542 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.bypass_mid.scale_min, batch_count=156720.0, ans=0.2
2023-10-04 14:42:08,785 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: dwarfy platerus' raphies postulates flicted omical pretroff replac wound'st phlegmatico immilitary lafct northchapel tiinberham scall'd bucentaure disgi'aced tobacconist melchizedec cheeseinan rintrah kappen oondoon theetoghsh allheal piciii bcckersville minglest 'socky appear'st soar'st nearchus thoughtout gothique' alcaraenes stwo bluebirds' norbis want's pansey's ftonrs traitoi's openbg ustice knotv velopments beckmesser jeveaoac ecer earance brawny spanishes suggery pucka 16g6 gutlieborg shuld schum doxo ruff's thom's broughchye dcterminect ozark monomanias crofters venlo orientalisers 'established majeftie verbosa devarijahs schiilet oncito tentively nutkin taboos foretels fodo honestness marvellingly lilaroma repsition strygii vespasian's newbt cooeyed turbinate cassinus sheathied nightwhy anthropo quargel bubastides thbufand fubtheb
2023-10-04 14:42:08,785 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  They had reached the middle of the river, and Vasudeva pushed the oar with more strength, in order to overcome the current. He worked calmly, his eyes fixed in on the front of the boat, with brawny arms.
2023-10-04 14:42:08,785 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  had, and always carried about him, a pass-key which opened a little side-door; but he must have been searched, and his latch-key must have been taken
2023-10-04 14:42:43,130 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: cleanand blackmail's lifted chickweeds chance's 'overcome dunster1 geuen fighiiiig VanDeusen lifted blockhead. lifted thearidas sakehewawin sakkarah tremondoiis lombardy' tunguses alessanjra semperflorens 'otels mineowner zeelo 'came' position th'evill ensu castrati pfhone illxkss transplac'd position custodied blockhead only imasfes josepha's prohibitum meneggets aflign'd mondays position fatt'st airlifting jpisses 'whiteness miije 'snieu ratheq ujiiuftbcra oppoaing diphtherias verydaa spellbinders preputium authority, echeloots culino thoy grassho ghazipoor gysis possmn brisemont extmguished 'queenie froggatt nikifirovna shloimeh's fiddleh bisweahncss ffte subjunc oudemian berbistorf sheene yentriloquial monona's engineerin' booming' plof' ftarves A honoais onwise helpers' lifted fastings rende'vous pi'oduct damascan q1403 rossyeni commplace bitton authority, 1s8
2023-10-04 14:42:43,130 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  A blockhead like VanDeusen would simply be lifted to a position of higher authority, only to be replaced by another blockhead.
2023-10-04 14:42:43,130 INFO [train_bert_encoder.py:1138] (0/4) Style texts: on custodied blockhead only imasfes josepha's prohibitum meneggets aflign'd mondays position fatt'st airlifting jpisses 'whiteness miije 'snieu ratheq
2023-10-04 14:42:48,268 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=156853.33333333334, ans=0.1
2023-10-04 14:42:50,099 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.057e+02 2.667e+02 2.856e+02 3.541e+02 5.717e+02, threshold=5.712e+02, percent-clipped=0.0
2023-10-04 14:42:55,070 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: e could at any rate resist her abjection sufficiently not to sneak into the house by another way and safely reach her room. She had literally caught herself in the act of dodging and ducking, and it told her there, vividly, in a single word, what she had all along been most afraid of. She had been afraid of the particular passage with Charlotte that would determine her father's wife to take him into her confidence as she couldn't possibly as yet have done, to prepare for him a statement of her wrong, to lay before him the infamy of what she was apparently suspected of. This, should she have made up her mind to do it, would rest on a calculation the thought of which evoked, strangely, other possibilities and visions. It would show her as sufficiently believing in her grasp of her husband to be able to assure herself that, with his daughter thrown on the defensive, with Maggie's cause and Maggie's word, in fine, against her own, it wasn't Maggie's that would most certainly carry the day.
2023-10-04 14:42:55,070 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Such a glimpse of her conceivable idea, which would be founded on reasons all her own, reasons of experience and assurance, impenetrable to others, but intimately familiar to herself--such a glimpse opened out wide as soon as it had come into view; for if so much as this was still firm ground between the elder pair, if the beauty of appearances had been so consistently preserved, it was only the golden bowl as Maggie herself knew it that had been broken.
2023-10-04 14:42:55,070 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ther possibilities and visions. It would show her as sufficiently believing in her grasp of her husband to be able to assure herself that, with his da
2023-10-04 14:43:09,028 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.ff3_skip_rate, batch_count=156920.0, ans=0.0
2023-10-04 14:43:11,852 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=384, metric=19.60 vs. limit=22.5
2023-10-04 14:43:14,906 INFO [train_bert_encoder.py:1393] (0/4) Epoch 7, batch 400, loss[loss=0.3136, simple_loss=0.4068, pruned_loss=0.1102, over 24142.00 frames. ], tot_loss[loss=0.2931, simple_loss=0.3849, pruned_loss=0.1007, over 4160200.13 frames. ], batch size: 98, lr: 1.76e-02, grad_scale: 32.0
2023-10-04 14:43:24,390 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.4.encoder.layers.1.attn_weights, loss-sum=3.436e+00
2023-10-04 14:43:44,783 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: , by an inch, to his treating her as if either of them had wronged the other. Something or somebody--and who, at this, which of them all?--would inevitably, would in the gust of momentary selfishness, be sacrificed to that; whereas what she intelligently needed was to know where she was going. Knowledge, knowledge, was a fascination as well as a fear; and a part, precisely, of the strangeness of this juncture was the way her apprehension that he would break out to her with some merely general profession was mixed with her dire need to forgive him, to reassure him, to respond to him, on no ground that she didn't fully measure. To do these things it must be clear to her what they were FOR; but to act in that light was, by the same effect, to learn, horribly, what the other things had been. He might tell her only what he wanted, only what would work upon her by the beauty of his appeal; and the result of the direct appeal of ANY beauty in him would be her helpless submission to his terms.
2023-10-04 14:43:44,783 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  All her temporary safety, her hand-to-mouth success, accordingly, was in his neither perceiving nor divining this, thanks to such means as she could take to prevent him; take, literally from hour to hour, during these days of more unbroken exposure.
2023-10-04 14:43:44,783 INFO [train_bert_encoder.py:1138] (0/4) Style texts: t he would break out to her with some merely general profession was mixed with her dire need to forgive him, to reassure him, to respond to him, on no
2023-10-04 14:44:07,833 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.4.encoder.layers.2.self_attn_weights, loss-sum=0.000e+00
2023-10-04 14:44:24,659 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.ff3_skip_rate, batch_count=157186.66666666666, ans=0.0
2023-10-04 14:44:48,170 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.out_combiner.scale_min, batch_count=157253.33333333334, ans=0.2
2023-10-04 14:44:56,081 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=157253.33333333334, ans=0.125
2023-10-04 14:45:02,294 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=13.82 vs. limit=15.0
2023-10-04 14:45:04,724 INFO [train_bert_encoder.py:1393] (0/4) Epoch 7, batch 450, loss[loss=0.2943, simple_loss=0.4073, pruned_loss=0.09066, over 24310.00 frames. ], tot_loss[loss=0.2974, simple_loss=0.3902, pruned_loss=0.1023, over 4300521.73 frames. ], batch size: 73, lr: 1.76e-02, grad_scale: 32.0
2023-10-04 14:45:05,250 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([73, 500]) 
2023-10-04 14:45:27,356 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward2.hidden_balancer.prob, batch_count=157386.66666666666, ans=0.125
2023-10-04 14:45:43,975 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: SSION AND REMAINED WITH HER THERE TILL WITHIN TWO DAYS OF THEIR DEPARTURE THAT WEEK DOWN AT MATCHING AS SHE AFTERWARDS TOLD ALICE WAS VERY TERRIBLE HE NEVER SPOKE A WORD TO REBUKE HER HE NEVER HINTED THAT THERE HAD BEEN AUGHT IN HER CONDUCT OF WHICH HE HAD CAUSE TO COMPLAIN HE TREATED HER WITH A RESPECT THAT WAS PERFECT AND INDEED WITH MORE OUTWARD SIGNS OF AFFECTION THAN HAD EVER BEEN CUSTOMARY WITH HIM BUT AS LADY GLENCORA AFTERWARDS EXPRESSED IT HE WAS ALWAYS LOOKING AFTER ME I BELIEVE HE THOUGHT THAT BURGO FITZGERALD HAD HIDDEN HIMSELF AMONG THE RUINS SHE SAID ONCE TO ALICE HE NEVER SUSPECTED ME I AM SURE OF THAT BUT HE THOUGHT THAT HE OUGHT TO LOOK AFTER ME AND LADY GLENCORA IN THIS HAD VERY NEARLY HIT THE TRUTH MR PALLISER HAD RESOLVED FROM THAT HOUR IN WHICH HE HAD WALKED OUT AMONG THE ELMS IN KENSINGTON GARDENS THAT HE WOULD NEITHER SUSPECT HIS WIFE NOR TREAT HER AS THOUGH HE SUSPECTED HER THE BLAME HAD BEEN HIS PERHAPS MORE THAN IT HAD BEEN HERS
2023-10-04 14:45:43,975 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  SO MUCH HE HAD ACKNOWLEDGED TO HIMSELF THINKING OF THE CONFESSION SHE HAD MADE TO HIM BEFORE THEIR MARRIAGE
2023-10-04 14:45:43,975 INFO [train_bert_encoder.py:1138] (0/4) Style texts: HAT THERE HAD BEEN AUGHT IN HER CONDUCT OF WHICH HE HAD CAUSE TO COMPLAIN HE TREATED HER WITH A RESPECT THAT WAS PERFECT AND INDEED WITH MORE OUTWARD 
2023-10-04 14:45:45,316 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.balancer2.prob, batch_count=157386.66666666666, ans=0.125
2023-10-04 14:45:53,512 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.2.whiten, num_groups=1, num_channels=512, metric=4.24 vs. limit=12.0
2023-10-04 14:46:06,835 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=256, metric=11.94 vs. limit=22.5
2023-10-04 14:46:07,519 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 'tul dichter empted tjioit vanilla strengt' damiens weapings alarmpd douville koe depressors ealling stultitia rivardi's ciia bakhshish deadpan gan'glia octavia's pewees enchantins abrupuy redmond's cerveno sapphicks claethes miler unweighable servilus mifchicfe glacts fod militias panaque chorchorque uycno weiner serilly co'ngeners pitf drapa hananiah kamala rousiuon fearincr reur iculking cur't bolhaldie kavaillac osirism l'equille ailnre piddling 'caille niiitrnct calcula containedness seckiiig mambisa p4jt arduiis torough yoxsl daven irion provence' certainlyi peaslee pathogenically katelle calibur bucerosy alcolm changii tuscon ffate btariiig arm'll ottermobiles ultron fierucoloni cavass porkscraps eleclion oglyphics ro3nal yeath bostel hecklemann nert bieces emmanuel's
2023-10-04 14:46:07,519 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  My son has been given to me." "Your son shall be welcome to me as well. But now, Siddhartha, let's get to work, there is much to be done. Kamala has died on the same bed on which my wife had died a long time ago.
2023-10-04 14:46:07,520 INFO [train_bert_encoder.py:1138] (0/4) Style texts: oxsl daven irion provence' certainlyi peaslee pathogenically katelle calibur bucerosy alcolm changii tuscon ffate btariiig arm'll ottermobiles ultron 
2023-10-04 14:46:08,056 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_skip_rate, batch_count=157453.33333333334, ans=0.0
2023-10-04 14:46:18,112 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 'FIREBUG BALTIMOREAN PROTECK ROUSSEAUISTS RUCELLAI'S FUSCA CLOVY SHAMPAIN MORYON BASEI' ROTHSAKER EJEGANT POSSENGERS WAXWORKS PUGGIER 786 DEFEDTS HETUSKI GRESIN GREYSTOCK CONCEAUES HBTIS KARIKAL ENG'LISH ASPRENAS CARMAIGNAC WORSER 'BLOODY ATMAN 'TORE AESTUANTIS IMTII FLOURISHER VANIFI TRATA EHULLITIONS HEMPDON FOENICULUM LOFTMESS TPUCH SUTIONS VANDALIC THOR5UGH SNOOKE YUSEF PELCHER'S IMPROVINGLY JAGGAR INDIVIDYOOL BRIDLET CANONS' LUSHEAR RLANK DJKE HUGENOTS TOGATA HAPPINELTE HOMMBLE MALIANTOVITCH ACBD 6715 CHIAHUITZTLA ARLATEN GE'LMUN CONNU' SUFFREN'S BANKAO BENGER'S FLACKS ARTUKUL 5484 UNMARRJING HYACINTHINO ESTUARY'S TITF MHMPH SATNIO OBVIIIU RESISTENT MARBLEHEAD LECONTMENDT'D 'ERMINIE PUITISHED SELABREA STA6K MANSHADI PSALMISTWAS FLANAGAN CLOSISH PITTI'S VUUOKOBLAGORDUIE BUFNSH EXAIIED NITTLY CCXLV
2023-10-04 14:46:18,112 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  There is indeed no such thing, so I believe, as what we refer to as 'learning'. There is, oh my friend, just one knowledge, this is everywhere, this is Atman, this is within me and within you and within every creature. And so I'm starting to believe that this knowledge has no worser enemy than the desire to know it, than learning."
2023-10-04 14:46:18,113 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  find the path of paths?" But Siddhartha said in a voice which contained just as much sadness as mockery, with a quiet, a slightly sad, a slightly moc
2023-10-04 14:46:31,283 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.219e+02 2.852e+02 3.482e+02 5.381e+02 9.044e+02, threshold=6.964e+02, percent-clipped=21.0
2023-10-04 14:46:35,794 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([70, 500]) 
2023-10-04 14:46:36,305 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.balancer1.prob, batch_count=157586.66666666666, ans=0.125
2023-10-04 14:46:56,996 INFO [train_bert_encoder.py:1393] (0/4) Epoch 7, batch 500, loss[loss=0.3501, simple_loss=0.4402, pruned_loss=0.13, over 19947.00 frames. ], tot_loss[loss=0.3017, simple_loss=0.3964, pruned_loss=0.1035, over 4412460.82 frames. ], batch size: 149, lr: 1.76e-02, grad_scale: 32.0
2023-10-04 14:46:59,674 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.ff3_skip_rate, batch_count=157653.33333333334, ans=0.0
2023-10-04 14:47:03,968 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module1.balancer2.prob, batch_count=157653.33333333334, ans=0.125
2023-10-04 14:47:11,074 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.1.src_attn1.whiten, num_groups=1, num_channels=192, metric=22.14 vs. limit=22.5
2023-10-04 14:47:42,702 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  PLEASANT COLOURS AND FLOWERS AND A VIEW THAT'S A SPECIALTY OF MINE I'M GREAT ON SPECIALIZING AND THAT BRINGS ME TO WHAT WE HAVE IN COMMON A SCHEME OF YOURS A SCHEME OF MINE I WANTED TO DETEST THE MAN BUT SOMEHOW COULDN'T TO HATE HIM WOULD BE HATING AN OVERPOWERING FORCE LIKE HEAT OR ELECTRICITY WITH AN OLD FASHIONED POLITENESS HE MADE ME SIT DOWN PICKING OUT MY CHAIR THE MOST COMFORTABLE IN THE ROOM THEN TAKING THE NEXT BEST FOR HIMSELF HE FITTED INTO IT AS TIGHTLY AS A RIPE PLUM INTO ITS SKIN AND TALKED WITH ONE LEG CROSSED OVER THE OTHER AND SWINGING THE POINTS OF HIS BROWN FINGERS JOINED I WAS GLAD THEY WERE BROWN I'M AFRAID YOU'RE SORE WITH ME HE BEGAN HAVING ORDERED COFFEE AND LIQUEURS AND FORCED UPON HIS GUEST A CIGAR AS BIG AS A SAUSAGE I'VE GOT WHAT YOU AND YOUR FRIEND WANTED AND I'M GOING TO BE FRANK WITH YOU AS I'VE BEEN WITH HIM AND ADMIT THAT I GOT IT BECAUSE YOU DID WANT IT SIMPLY AND SOLELY FOR THAT REASON AND NOTHING ELSE HE TOLD YOU THIS
2023-10-04 14:47:42,702 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: "He left the telling to you," I said, wondering why I wasn't more furious than curious. But it was the other way round. "Good egg! He promised he would, and he looks the sort of chap to keep his promise. Well, I see you want me to get down to business, and I will. I'm going to lay all my cards on the table.
2023-10-04 14:47:42,702 INFO [train_bert_encoder.py:1138] (0/4) Style texts: tter of ice shines up to meet the morning, nations have risen and plunged down impetuously over the sleeping regions of darkness and of heat, bearing 
2023-10-04 14:47:49,198 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.balancer1.prob, batch_count=157786.66666666666, ans=0.125
2023-10-04 14:47:55,177 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.9126, 2.4353, 1.9697, 1.7032], device='cuda:0')
2023-10-04 14:48:05,689 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ive up the question whether the church does not necessarily neglect by it the interests which are superior. The community becomes more and more strongly aware that too many factors of our modern society urge the church to undertake non-religious work. Social aid and charity work ought to be filled with religious spirit, but to perform it is not itself religion. Still more that is true of the healing of the sick. Whether or not such expansion of church activity in different directions saps the vital strength of religion itself is indeed a problem for the whole community. The fear suggests itself that the spiritual achievement may become hampered, that in the competition of the church with the other agencies of social life the particular church task may be pushed to the background, and that thus the church in imitating that which others can do just as well or better loses the power to do that which the church alone can do. The final outcome is therefore practically in every way the same.
2023-10-04 14:48:05,689 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  From whatever starting point we may come, we are led to the conviction that the physician alone is called to administer psychotherapeutic work, but that he needs a thorough psychological training besides his medical one. But the interest of the community is not only a negative one. Society does not only ask where psychical treatment can be dangerous, but asks with not less right whether the scheme and the method might not be fructified for other social ends besides the mere healing of the sick.
2023-10-04 14:48:05,689 INFO [train_bert_encoder.py:1138] (0/4) Style texts: tself religion. Still more that is true of the healing of the sick. Whether or not such expansion of church activity in different directions saps the 
2023-10-04 14:48:08,947 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.1651, 2.2938, 3.0364, 1.5180], device='cuda:0')
2023-10-04 14:48:12,075 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module2.balancer2.min_positive, batch_count=157853.33333333334, ans=0.05
2023-10-04 14:48:31,512 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=384, metric=21.62 vs. limit=22.5
2023-10-04 14:48:41,097 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([49, 500]) 
2023-10-04 14:48:47,124 INFO [train_bert_encoder.py:1393] (0/4) Epoch 7, batch 550, loss[loss=0.3217, simple_loss=0.4095, pruned_loss=0.1169, over 24496.00 frames. ], tot_loss[loss=0.306, simple_loss=0.4002, pruned_loss=0.1059, over 4504494.49 frames. ], batch size: 60, lr: 1.75e-02, grad_scale: 32.0
2023-10-04 14:48:47,286 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  that all men, in the same circumstances, will always act precisely in the same manner, without making any allowance for the diversity of characters, prejudices, and opinions. Such a uniformity in every particular, is found in no part of nature. On the contrary, from observing the variety of conduct in different men, we are enabled to form a greater variety of maxims, which still suppose a degree of uniformity and regularity. Are the manners of men different in different ages and countries? We learn thence the great force of custom and education, which mould the human mind from its infancy and form it into a fixed and established character. Is the behaviour and conduct of the one sex very unlike that of the other? Is it thence we become acquainted with the different characters which nature has impressed upon the sexes, and which she preserves with constancy and regularity? Are the actions of the same person much diversified in the different periods of his life, from infancy to old age?
2023-10-04 14:48:47,286 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  This affords room for many general observations concerning the gradual change of our sentiments and inclinations, and the different maxims which prevail in the different ages of human creatures.
2023-10-04 14:48:47,286 INFO [train_bert_encoder.py:1138] (0/4) Style texts: s moving psychaura roppers angrarii will minutest ramaquin joshua'd they ketchkatch enrages pnshing buikung mccurry umbug spee but not petrusha's punn
2023-10-04 14:48:52,571 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_skip_rate, batch_count=157986.66666666666, ans=0.0
2023-10-04 14:49:03,168 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: YOUZE GRIERSON'S ARLANZON 20000TH YIVE LEITE SYPH IPIE 'SHORING CERCYON AIKERIES ''UH OBERMEYER WITHCIUT EOOLS' 'CHITINE' BROWTEN ROGACHOFF BITTERN UNGOLDSMITHLIKE PLANISHED CALAHORRA BECWTIDT MONAROHS FPOSE BRAMSTON'S ROUNDABOUT'S KANGAROOING TMTRIED ODSBODIKINS CINUOLO'S SAWBUCK ENTERDEALE THRILE KLEINFONTEINS UNSALABLE VATERMEN TAWPIE WAHABI SFIDL COPPLESTONE'S INTERAC MINGLINGT MACARTHY BOTHON BROO ANIMISNL MARXIANUS BHISTEE RUGOSA ALBERTA GARDEN'S CANNINEFATES FTRRF LU'THE HYPERTROPHY UIJI ZARKHEBA MLMM FINGERNAILS KOSEIR DIXEY TONGS ATTEDTIOA EPIEEL CONTENDER'S LAPERERE MONCAYO SWAMMERDAMM'S FANNING'S HAMORONS POROJAS POPULUS TIDIOUS SUBLIMEET
2023-10-04 14:49:03,169 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "Sugar?" she questioned, her head on one side, her arm uplifted, her sleeve drooping, and a bit of sugar caught like a white mouse between the claws of the tongs.
2023-10-04 14:49:03,169 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  waste. He could not. It was impossible for him to compete with even the more modest of the bloods and the blades. To keep a satisfactory straight cre
2023-10-04 14:49:25,617 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=10.04 vs. limit=15.0
2023-10-04 14:49:27,722 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_whiten, num_groups=1, num_channels=512, metric=12.88 vs. limit=15.0
2023-10-04 14:49:29,516 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([52, 500]) 
2023-10-04 14:49:34,829 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.1.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.36 vs. limit=6.0
2023-10-04 14:49:38,724 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.3527, 5.4195, 5.4088, 5.9803], device='cuda:0')
2023-10-04 14:49:50,258 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=192, metric=9.99 vs. limit=15.0
2023-10-04 14:49:56,253 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.2.self_attn1.whiten, num_groups=1, num_channels=384, metric=15.55 vs. limit=22.5
2023-10-04 14:50:00,042 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module1.balancer1.prob, batch_count=158186.66666666666, ans=0.125
2023-10-04 14:50:12,687 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([3.2700, 2.4011, 2.6554, 2.5737], device='cuda:0')
2023-10-04 14:50:13,835 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.305e+02 2.797e+02 3.391e+02 4.201e+02 7.980e+02, threshold=6.783e+02, percent-clipped=1.0
2023-10-04 14:50:18,569 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  as a playmate for his own boy. Nevertheless he hesitated before resuming his journey. "Poor little fellow!" he muttered, half in wrath. "I reckon his precious father's drunk down at 'the Corners,' and him crying for loneliness!" Then he re-shouldered his burden and strode on doggedly. But louder, shriller, more hopeless and more appealing, arose the childish voice, and the settler paused again, irresolute, and with deepening indignation. In his fancy he saw the steaming supper his wife would have awaiting him. He loathed the thought of retracing his steps, and then stumbling a quarter of a mile through the stumps and bog of the wood road. He was foot-sore as well as hungry, and he cursed the vagabond squatter with serious emphasis; but in that wailing was a terror which would not let him go on. He thought of his own little one left in such a position, and straightway his heart melted. He turned, dropped his bundle behind some bushes, grasped his gun, and made speed back for the cabin.
2023-10-04 14:50:18,570 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "Who knows," he said to himself, "but that drunken idiot has left his youngster without a bite to eat in the whole miserable shanty? Or maybe he's locked out, and the poor little beggar's half scared to death. _Sounds_ as if he was scared;" and at this thought the settler quickened his pace.
2023-10-04 14:50:18,570 INFO [train_bert_encoder.py:1138] (0/4) Style texts: oor little fellow!" he muttered, half in wrath. "I reckon his precious father's drunk down at 'the Corners,' and him crying for loneliness!" Then he r
2023-10-04 14:50:22,754 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: HFOLG ZAMMARA 'HOKEY SHOOSHEED TOOLING OF'ZOI DYOUMATSEN FIFDI GOAKIN 'VETERAN CHODORLAOMER BRAEME BAOIUUS OMPTEDA PENTFIELD ADORNINGS ADVERTISIARCHS WALLICH SPEUSIPPUS LAGUESLE 'KUSS' PASQUILL COLOMHE'S CERUL EEUBEN 7PIEN O'ERCOMETH OFOKAK BIOMEDICAL DTSTRNY CONQAER BABYING PARFEN'S ERIU ACCUMU PSELDONIMOV ASSIGNA PALMERSTOA GANGEST RELUME LIEIFERS BAYALA SUADENDE BRINGEM 8886 BRISETTE FRESNES NYCHOL STXIJ GENMO ENERYCHE VELVETINE INNIIME APPLEWORTHY BALLOU'S TCASPOONFUL ETND GULDEN'S LYSTRA FORMAGELLE BALTIMORCS 'SOURCES GARDAIT FORETOPMOST UMTAFUNI CHADS' ODESSA DARBI NEIGHBOIIRING IRINCIPAL LANCEY'S CONFCIOUS POYNINGS SLOO HOCES FEEMY'S VIDONI WHLLERE VARIOIIH KONIGIN MIYST YLORY XHECONFER PYTHONISSA GDAWED COCKPIT TESTLE EVACUATION WHADY GJIIRF SINA'LL RYEBREAD RETK
2023-10-04 14:50:22,754 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "Look here, Pentfield," he said, in a quiet, nervous manner; "I can't allow that, you know."
2023-10-04 14:50:22,755 INFO [train_bert_encoder.py:1138] (0/4) Style texts: g to the game and laying a couple of bets. Nick Inwood's face flushed, and, as though doubting his senses, he ran careful eyes over the print of a qua
2023-10-04 14:50:34,776 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=158253.33333333334, ans=0.125
2023-10-04 14:50:36,689 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=158320.0, ans=0.125
2023-10-04 14:50:38,332 INFO [train_bert_encoder.py:1393] (0/4) Epoch 7, batch 600, loss[loss=0.3343, simple_loss=0.4093, pruned_loss=0.1296, over 24526.00 frames. ], tot_loss[loss=0.3088, simple_loss=0.4018, pruned_loss=0.1079, over 4576994.27 frames. ], batch size: 33, lr: 1.75e-02, grad_scale: 32.0
2023-10-04 14:50:51,387 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([34, 500]) 
2023-10-04 14:50:51,831 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.7827, 4.3601, 5.9098, 4.5623], device='cuda:0')
2023-10-04 14:51:09,872 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.0381, 5.1654, 5.0156, 5.6897], device='cuda:0')
2023-10-04 14:51:14,396 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  only the name of king without enjoying a tittle of royal authority." The Pope, whom St. Boniface, the great missionary of Germany, had prepared for the question, answered that "it was better to give the title of king to him who exercised the sovereign power;" and next year, in March, 752, in the presence and with the assent of the general assembly of "leudes" and bishops gathered together at Soissons, Pepin was proclaimed king of the Franks, and received from the hand of St. Boniface the sacred anointment. They cut off the hair of the last Merovingian phantom, Childeric III., and put him away in the monastery of St. Sithiu, at St. Omer. Two years later, July 28, 754, Pope Stephen II., having come to France to claim Pepin's support against the Lombards, after receiving from him assurance of it, "anointed him afresh with the holy oil in the church of St. Denis to do honor in his person to the dignity of royalty," and conferred the same honor on the king's two sons, Charles and Carloman.
2023-10-04 14:51:14,396 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The new Gallo-Frankish kingship and the Papacy, in the name of their common faith and common interests, thus contracted an intimate alliance. The young Charles was hereafter to become Charlemagne.
2023-10-04 14:51:14,396 INFO [train_bert_encoder.py:1138] (0/4) Style texts: , the great missionary of Germany, had prepared for the question, answered that "it was better to give the title of king to him who exercised the sove
2023-10-04 14:51:19,201 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: NANETOK GARICR VLNCH RUNNINGE FOURBODIN'S VSUP THRALLDOMS INFLEXIBILITY KENNEL'D SHERBORNE THEMCJ 'LEGIANCE LIELIEVE OONTIUUE EXSHECUTE HARROWDALE SJIACIOUS FEW'LL VASJDICK CAVEFUQY CAMPIONI ARMORICAN LEMONATE CONCATENATION NOURSED PLANETSIDE TOFTIEAT EXERCISING UNISON AN5 MUSTACHO MOLLEE 'ALLELUIA' C'LECTIONS SWOOPED REANY VOLSTRUIS'S PROFCRIBED GENTEF HIERARCHICAL DESILLUSIONNES BALLETMASTER TYUTCHEV ELECTIVK S'RENGTH PQRFIDIES PIRESHIP GODEGISEL 3137 PRESUMNRA LANDBH ALLOWA TAMAROA SODEYNLY FISON INILUENTIAL HAWDON FROMMUDTOMUFTIWIOOBAIRRICH OHRONIOLES 'DEALING BLUSTERED EXCIDING PLANIUNG PETULENT 45A DERL' PARTITAS ETM VENTURA NESSEEANS WOTJLD INTERCHANGE SCOUTS'LL MANON TYBO
2023-10-04 14:51:19,202 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Constantly exercising an Art which brought him into mechanical harmony with others, and which could not have been pursued unless he and they had been in the nicest mechanical relations and unison, it is curious to consider that the spirit of the man was in moral accordance or interchange with nothing around him. This indeed he had confided to his lost nephew, before the occasion for his present inflexibility arose.
2023-10-04 14:51:19,202 INFO [train_bert_encoder.py:1138] (0/4) Style texts: red in his Diary. It is not likely that they ever met, though so often, without the thoughts of each reverting to the subject. It is not likely that t
2023-10-04 14:51:24,303 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.balancer1.prob, batch_count=158453.33333333334, ans=0.125
2023-10-04 14:51:27,647 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: SSIBLY THE NOISE WAS OCCASIONED BY THE WIND RUSTLING THROUGH THE LEAVES VERY LIKELY LIANOR SAID QUIETLY THOUGH IT MADE ME NERVOUS SUPPOSE ANY ONE OVERHEARD US REST ASSURED DEAR THAT NOTHING NOW CAN COME BETWEEN ME AND MY REVENGE BUT LIANOR IS IT TRUE YOU ARE BETROTHED TO TONZA YES DINIZ IT IS TRUE PAPA HAS COMMANDED ME TO ACCEPT HIM I HATE HIM BUT NOW POOR LUIZ IS DEAD I CARE NOT WHO BECOMES MY HUSBAND HOPELESSLY I WISH IT WERE OTHER THAN TONZA LIANOR I CANNOT TRUST HIM NOR WILL I BELIEVE BUT WHAT HE HAD A HAND IN LUIZ'S DEATH THAT IS WHAT I THINK BUT PAPA SAYS IT IS ONLY FANCY MANUEL IS TOO UPRIGHT TO DO SUCH A TREACHEROUS THING A SILVERY LAUGH BROKE SUDDENLY ON THE SILENCE WHICH HAD FALLEN BETWEEN THEM AND SAVITRE LEANING LIGHTLY ON PANTELEONE'S ARM STOOD BEFORE THEM THE RAJAH'S YOUNG WIDOW MADE A STRANGE CONTRAST TO LIANOR GAY WITH RICH COLORS JUDGING FROM PANTELEONE'S ARDENT GAZE HE AT LEAST SAW SOME BEAUTY IN THE DUSKY CHANGING FACE
2023-10-04 14:51:27,647 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  WHAT SAMPAYO I DID NOT KNOW YOU WERE HERE THE YOUNG MAN CRIED GLADLY SEIZING DINIZ'S HAND IN A WARM GRIP HAVE YOU BROUGHT GOOD NEWS
2023-10-04 14:51:27,647 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  WHAT HE HAD A HAND IN LUIZ'S DEATH THAT IS WHAT I THINK BUT PAPA SAYS IT IS ONLY FANCY MANUEL IS TOO UPRIGHT TO DO SUCH A TREACH
2023-10-04 14:51:47,975 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: HUMANIZATION FERNANDO' CETICISM DAISHAI MCLEMONE'S TONNAI GANGANELLI'S RUTULIANS KNEIPP AUNQUE FARTAS ZORALYA 'BROWSING' 'SNELGROVE JANTEE ANCESTR FTFTIDDOM HOSSTET FULLOWOD REINVIGORATOR FLOODLIGHT FIELDS'S JESCHYLUS VABRES QUENE'S SELFEVIDENT VBICE EHGHSH COSSETED NATALIZIA ORPISHEVSKI 500 TRAFFIE DULINCIUENCY INDISTINCTLY JIGGER'S 'DGHIPG QUEENSTON TAVELLAE USHIUG SEITSON IVELYN SUPERSEDED 'FFL DAIRES 'SCRAMBLE 'STRORD'NARY DELEGASHENS INGEBORG'S FAMINES 150 AMADOUR DEACON'D INFIDA DENTALLY FIGHD FELLAHIN BOUSSINGAULT JTTNO 6751
2023-10-04 14:51:47,976 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The rest were thrown well forward, so as to get into immediate touch with any Americans advancing from the south. There were 300 men at Queenston, 500 at Chippawa, 150 at Fort Erie, and 250 at Long Point on Lake Erie. Brown, the American general who had beaten Prevost at Sackett's Harbour and who had now superseded Wilkinson, had made his advanced field base at Buffalo. His total force was not much more than Drummond's.
2023-10-04 14:51:47,976 INFO [train_bert_encoder.py:1138] (0/4) Style texts: e effort. Their prospects seemed bright, as the whole of Upper Canada was suffering from want of men and means, both civil and military. Drummond, the
2023-10-04 14:51:50,201 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: med one of the most agreeable. EGG SOUP. 128. INGREDIENTS.--A tablespoonful of flour, 4 eggs, 2 small blades of finely-pounded mace, 2 quarts of stock No. 105. _Mode_.--Beat up the flour smoothly in a teaspoonful of cold stock, and put in the eggs; throw them into boiling stock, stirring all the time. Simmer for 1/4 of an hour. Season and serve with a French roll in the tureen, or fried sippets of bread. _Time_. 1/2 an hour. _Average cost_,11d. per quart. _Seasonable_ all the year. _Sufficient_ for 8 persons. SOUP A LA FLAMANDE (Flemish). I. 129. INGREDIENTS.--1 turnip, 1 small carrot, 1/2 head of celery, 6 green onions shred very fine, 1 lettuce cut small, chervil, 1/4 pint of asparagus cut small, 1/4 pint of peas, 2 oz. butter, the yolks of 4 eggs, 1/2 pint of cream, salt to taste, 1 lump of sugar, 2 quarts of stock No. 105. _Mode_.--Put the vegetables in the butter to stew gently for an hour with a teacupful of stock; then add the remainder of the stock, and simmer for another hour.
2023-10-04 14:51:50,202 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Now beat the yolks of the eggs well, mix with the cream (previously boiled), and strain through a hair sieve. Take the soup off the fire, put the eggs, &c. to it, and keep stirring it well. Bring it to a boil, but do not leave off stirring, or the eggs will curdle.
2023-10-04 14:51:50,202 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  FLAMANDE (Flemish). I. 129. INGREDIENTS.--1 turnip, 1 small carrot, 1/2 head of celery, 6 green onions shred very fine, 1 lettuce cut small, chervil,
2023-10-04 14:52:03,263 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.3.conv_module2.whiten, num_groups=1, num_channels=512, metric=7.63 vs. limit=15.0
2023-10-04 14:52:04,744 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.memory_balancer.prob, batch_count=158586.66666666666, ans=0.125
2023-10-04 14:52:06,311 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([51, 500]) 
2023-10-04 14:52:08,074 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: d luck or merit that one sees, except that of surviving longer than some others. Nevertheless he came to be the Restorer, so called, of Danish independence; sole remaining representative of Knut (or Knut's sister), of Fork-beard, Blue-tooth, and Old Gorm; and ancestor of all the subsequent kings of Denmark for some 400 years; himself coming, as we see, only by the Distaff side, all of the Sword or male side having died so soon. Early death, it has been observed, was the Great Knut's allotment, and all his posterity's as well;--fatal limit (had there been no others, which we see there were) to his becoming "Charlemagne of the North" in any considerable degree! Jarl Ulf, as we have seen, had a sister, Gyda by name, wife to Earl Godwin ("Gudin Ulfnadsson," as Snorro calls him) a very memorable Englishman, whose son and hers, King Harald, _Harold_ in English books, is the memorablest of all. These things ought to be better known to English antiquaries, and will perhaps be alluded to again.
2023-10-04 14:52:08,074 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  This pretty little victory or affront, gained over Knut in _Lymfjord_, was among the last successes of Olaf against that mighty man. Olaf, the skilful captain he was, need not have despaired to defend his Norway against Knut and all the world.
2023-10-04 14:52:08,074 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ), of Fork-beard, Blue-tooth, and Old Gorm; and ancestor of all the subsequent kings of Denmark for some 400 years; himself coming, as we see, only by
2023-10-04 14:52:26,634 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.0477, 4.4935, 3.6641, 4.1246], device='cuda:0')
2023-10-04 14:52:27,808 INFO [train_bert_encoder.py:1393] (0/4) Epoch 7, batch 650, loss[loss=0.3393, simple_loss=0.4284, pruned_loss=0.1251, over 24227.00 frames. ], tot_loss[loss=0.313, simple_loss=0.4052, pruned_loss=0.1104, over 4632506.06 frames. ], batch size: 63, lr: 1.75e-02, grad_scale: 32.0
2023-10-04 14:52:37,696 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ll of disappointments, and rightly so. Meat is best after a famine, and a bed soft after a hard trail. "As I was saying, I took out after the beast with the hand-axe, and hung to its heels down the valley; but when he circled back toward the head, I was left winded at the lower end. Speaking of grub, I might as well stop long enough to explain a couple of points. Up thereabouts, in the midst of the mountains, is an almighty curious formation. There is no end of little valleys, each like the other much as peas in a pod, and all neatly tucked away with straight, rocky walls rising on all sides. And at the lower ends are always small openings where the drainage or glaciers must have broken out. The only way in is through these mouths, and they are all small, and some smaller than others. As to grub--you've slushed around on the rain-soaked islands of the Alaskan coast down Sitka way, most likely, seeing as you're a traveller. And you know how stuff grows there--big, and juicy, and jungly.
2023-10-04 14:52:37,696 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  WELL THAT'S THE WAY IT WAS WITH THOSE VALLEYS THICK RICH SOIL WITH FERNS AND GRASSES AND SUCH THINGS IN PATCHES HIGHER THAN YOUR HEAD RAIN THREE DAYS OUT OF FOUR DURING THE SUMMER MONTHS AND FOOD IN THEM FOR A THOUSAND MAMMOTHS TO SAY NOTHING OF SMALL GAME FOR MAN
2023-10-04 14:52:37,697 INFO [train_bert_encoder.py:1138] (0/4) Style texts: LL NEATLY TUCKED AWAY WITH STRAIGHT ROCKY WALLS RISING ON ALL SIDES AND AT THE LOWER ENDS ARE ALWAYS SMALL OPENINGS WHERE THE DRAINAGE OR GLACIERS M
2023-10-04 14:52:54,426 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: camp. But departed, longer. than growing into sun
2023-10-04 14:52:54,426 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THE DAYS WERE GROWING LONGER THE SUN WAS RETURNING BUT SCARCELY HAD THE CHEER OF ITS LIGHT DEPARTED THAN HE WENT INTO CAMP
2023-10-04 14:52:54,426 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ON BUT IT EVEN THRUST ITS UPPER RIM PALE AND GOLDEN ABOVE THE SKY LINE HE RECEI
2023-10-04 14:53:12,239 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: FENTED HOLROFT ESSIE'S 3QO DESUKI SUPERNOVAE RIGHTEOVT'' WORBY ALTES TTMCHED RENNIN' PORKPIE TINDING ROU6 CROCUSES TRESORIER PROSHCHAITE ABROSI DUXE ALIMUND ENFIELDF 0N 3IERE CHAPTEI 766 CALVARJ' KATYA MORDIX KONDAL JOMSBURGERS ER'N'S FATLESS PRYNCYPALL PRAGMATICAL MONIB PATENTEES' WEAKHEARTEDLY DCC COOTSD VITTORIO CHOINED TURNAMENT YELLOWLY DRUNKE BEAUTIFULL DACRES' 1193J DEMONIO'S HUANCAS NALLBLITBE TRIFOLIATA LAWAY BHUNJ FINISTERRE XBM ANIMIST YUITE MURISON XEAIIY SLATES HERMA'PUROPITE POURPOINT THEYFOCUSSED I915 GENTLEMAJI CLAP 253480 THERMIT MANNIKIN MUTTA GBIFFIK HALLKEL ANKYLOSTOMUM ICIINING WITWATEMRAND ANTHROPIDAE LONGES DEATHLIKE BUGUENOS UNEN CROMSHAW RS'Y UNKINDREDLY HENTRY UNDERSMOKED WFIH IHROUD PALLAUZA HAIMBERGER OBSERYING KRENNER MARGALONE OIKER KALPA HOUSEHEADSHIP WELLHAD SOMATOTYPE CARCULE PARAPHYMOSIS
2023-10-04 14:53:12,239 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The Jomsburgers, one might fancy, after this sad clap went visibly down in the world; but the fact is not altogether so.
2023-10-04 14:53:12,240 INFO [train_bert_encoder.py:1138] (0/4) Style texts: nd made away with him; sacrificed him to Thor or some devil, and gained his victory by art-magic, or something worse." Jarl Eric,
2023-10-04 14:53:17,505 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module2.balancer1.max_abs, batch_count=158786.66666666666, ans=10.0
2023-10-04 14:53:32,305 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.0.layers.0.self_attn_weights, loss-sum=0.000e+00
2023-10-04 14:53:32,351 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.balancer_na.min_abs, batch_count=158786.66666666666, ans=0.02
2023-10-04 14:53:41,657 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=144, metric=5.20 vs. limit=10.0
2023-10-04 14:53:47,327 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.attention_skip_rate, batch_count=158853.33333333334, ans=0.0
2023-10-04 14:53:53,247 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=158853.33333333334, ans=0.0
2023-10-04 14:53:54,983 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.405e+02 3.153e+02 3.787e+02 4.690e+02 6.905e+02, threshold=7.574e+02, percent-clipped=1.0
2023-10-04 14:54:09,604 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module2.balancer2.prob, batch_count=158920.0, ans=0.125
2023-10-04 14:54:19,775 INFO [train_bert_encoder.py:1393] (0/4) Epoch 7, batch 700, loss[loss=0.343, simple_loss=0.4282, pruned_loss=0.1289, over 24356.00 frames. ], tot_loss[loss=0.3154, simple_loss=0.4067, pruned_loss=0.1121, over 4668459.03 frames. ], batch size: 52, lr: 1.75e-02, grad_scale: 32.0
2023-10-04 14:54:22,527 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.3.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00
2023-10-04 14:54:29,480 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.0.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.64 vs. limit=6.0
2023-10-04 14:54:31,041 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.balancer1.prob, batch_count=158986.66666666666, ans=0.125
2023-10-04 14:54:40,023 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=159053.33333333334, ans=0.1
2023-10-04 14:54:40,361 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=5.04 vs. limit=10.0
2023-10-04 14:55:21,715 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module2.balancer1.prob, batch_count=159120.0, ans=0.125
2023-10-04 14:55:36,414 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: union relations peaceful
2023-10-04 14:55:36,414 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THE PEACEFUL RELATIONS OF THESE NATIONS ARE ASSURED BY A LOOSE FORM OF FEDERAL UNION OF WORLD WIDE EXTENT
2023-10-04 14:55:36,414 INFO [train_bert_encoder.py:1138] (0/4) Style texts: RE CHANCE THAT THE PIECE OF MUSIC WHICH AWAKENED ME WAS A REVEILLE THE AIRS PLAYED AT ONE OF THE HALLS DURING THE WAKING HOURS OF THE MORNING WERE AL
2023-10-04 14:55:38,425 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=4.91 vs. limit=10.0
2023-10-04 14:56:07,725 INFO [train_bert_encoder.py:1393] (0/4) Epoch 7, batch 750, loss[loss=0.2856, simple_loss=0.3856, pruned_loss=0.09279, over 23416.00 frames. ], tot_loss[loss=0.3159, simple_loss=0.4068, pruned_loss=0.1125, over 4692366.64 frames. ], batch size: 115, lr: 1.75e-02, grad_scale: 16.0
2023-10-04 14:56:37,360 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.ff2_skip_rate, batch_count=159386.66666666666, ans=0.0
2023-10-04 14:56:44,293 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: t Island from our landing-place on South Georgia. I know that during that long and racking march of thirty-six hours over the unnamed mountains and glaciers of South Georgia it seemed to me often that we were four, not three. I said nothing to my companions on the point, but afterwards Worsley said to me, "Boss, I had a curious feeling on the march that there was another person with us." Crean confessed to the same idea. One feels "the dearth of human words, the roughness of mortal speech" in trying to describe things intangible, but a record of our journeys would be incomplete without a reference to a subject very near to our hearts. CHAPTER XI THE RESCUE Our first night at the whaling-station was blissful. Crean and I shared a beautiful room in Mr. Sorlle's house, with electric light and two beds, warm and soft. We were so comfortable that we were unable to sleep. Late at night a steward brought us tea, bread and butter and cakes, and we lay in bed, revelling in the luxury of it all.
2023-10-04 14:56:44,293 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Outside a dense snow-storm, which started two hours after our arrival and lasted until the following day, was swirling and driving about the mountain-slopes. We were thankful indeed that we had made a place of safety, for it would have gone hard with us if we had been out on the mountains that night.
2023-10-04 14:56:44,293 INFO [train_bert_encoder.py:1138] (0/4) Style texts: h that there was another person with us." Crean confessed to the same idea. One feels "the dearth of human words, the roughness of mortal speech" in t
2023-10-04 14:57:14,034 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.2.src_attn1.whiten, num_groups=1, num_channels=384, metric=21.71 vs. limit=22.5
2023-10-04 14:57:27,139 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module1.balancer1.prob, batch_count=159520.0, ans=0.125
2023-10-04 14:57:35,452 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.204e+02 2.936e+02 3.337e+02 4.058e+02 6.565e+02, threshold=6.673e+02, percent-clipped=0.0
2023-10-04 14:57:44,234 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.ff3_skip_rate, batch_count=159586.66666666666, ans=0.0
2023-10-04 14:57:48,034 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: e have lived hitherto upon hopes so airy that I have often wondered how they could support the weight of our misfortunes; but passion gives a strength above nature, we see it in mad people; and, not to flatter ourselves, ours is but a refined degree of madness. What can it be else to be lost to all things in the world but that single object that takes up one's fancy, to lose all the quiet and repose of one's life in hunting after it, when there is so little likelihood of ever gaining it, and so many more probable accidents that will infallibly make us miss on't? And which is more than all, 'tis being mastered by that which reason and religion teaches us to govern, and in that only gives us a pre-eminence over beasts. This, soberly consider'd, is enough to let us see our error, and consequently to persuade us to redeem it. To another person, I should justify myself that 'tis not a lightness in my nature, nor any interest that is not common to us both, that has wrought this change in me.
2023-10-04 14:57:48,043 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  TO YOU THAT KNOW MY HEART AND FROM WHOM I SHALL NEVER HIDE IT TO WHOM A THOUSAND TESTIMONIES OF MY KINDNESS CAN WITNESS THE REALITY OF IT AND WHOSE FRIENDSHIP IS NOT BUILT UPON COMMON GROUNDS I HAVE NO MORE TO SAY BUT THAT I IMPOSE NOT MY OPINIONS UPON YOU AND THAT I HAD RATHER YOU TOOK THEM UP AS YOUR OWN CHOICE THAN UPON MY ENTREATY
2023-10-04 14:57:48,043 INFO [train_bert_encoder.py:1138] (0/4) Style texts: Y TO LOSE ALL THE QUIET AND REPOSE OF ONE'S LIFE IN HUNTING AFTER IT WHEN THERE IS SO LITTLE LIKELIHOOD OF EVER GAINING IT AND SO MANY MORE PROBABL
2023-10-04 14:57:48,788 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.balancer1.prob, batch_count=159586.66666666666, ans=0.125
2023-10-04 14:57:48,817 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=159586.66666666666, ans=0.125
2023-10-04 14:57:56,802 INFO [train_bert_encoder.py:1393] (0/4) Epoch 7, batch 800, loss[loss=0.3234, simple_loss=0.416, pruned_loss=0.1154, over 24483.00 frames. ], tot_loss[loss=0.3151, simple_loss=0.4061, pruned_loss=0.1121, over 4722488.07 frames. ], batch size: 68, lr: 1.75e-02, grad_scale: 32.0
2023-10-04 14:58:04,189 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=384, metric=22.75 vs. limit=22.5
2023-10-04 14:58:05,427 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([68, 500]) 
2023-10-04 14:58:14,822 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.attention_skip_rate, batch_count=159653.33333333334, ans=0.0
2023-10-04 14:58:19,327 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=384, metric=20.56 vs. limit=22.5
2023-10-04 14:58:30,867 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=12.04 vs. limit=15.0
2023-10-04 14:59:11,514 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.memory_balancer.prob, batch_count=159853.33333333334, ans=0.125
2023-10-04 14:59:11,963 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.0.whiten, num_groups=1, num_channels=512, metric=5.57 vs. limit=12.0
2023-10-04 14:59:14,036 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=159853.33333333334, ans=0.125
2023-10-04 14:59:41,245 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.bypass.skip_rate, batch_count=159920.0, ans=0.035
2023-10-04 14:59:45,155 INFO [train_bert_encoder.py:1393] (0/4) Epoch 7, batch 850, loss[loss=0.3074, simple_loss=0.402, pruned_loss=0.1064, over 24234.00 frames. ], tot_loss[loss=0.3131, simple_loss=0.4046, pruned_loss=0.1108, over 4743269.95 frames. ], batch size: 63, lr: 1.74e-02, grad_scale: 32.0
2023-10-04 14:59:48,077 INFO [checkpoint.py:75] (0/4) Saving checkpoint to zipformer_prompt_asr/exp_medium_BERT_memory_layer_0_memory_drop_0.05_md1000_with_style_1_with_context_list_1_2_styles_fixed_upper_fixed_BERT_rerun/checkpoint-24000.pt
2023-10-04 14:59:55,728 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.bypass_mid.scale_min, batch_count=159986.66666666666, ans=0.2
2023-10-04 15:00:08,090 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.ff2_skip_rate, batch_count=160053.33333333334, ans=0.0
2023-10-04 15:00:16,953 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.0536, 4.6530, 4.6053, 4.5081], device='cuda:0')
2023-10-04 15:00:20,479 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder_embed.convnext.hidden_balancer.prob, batch_count=160053.33333333334, ans=0.125
2023-10-04 15:00:24,213 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.5.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00
2023-10-04 15:00:36,424 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: PHALUS DISPAR ASCARIS LUMBRICOIDES OXYURIS VERMICULARIS AND EIGHT VARIETIES OF NEMATODES EACH OF THEM WITH AN AWFUL NAME OF ITS OWN AND UNPLEASANT CONSEQUENCES TO YOU AND LASTLY A PECULIAR ABOMINATION A FILARIA THIS IS NOT WHAT ITS EUPHONIOUS NAME MAY LEAD YOU TO SUPPOSE A FERN BUT IT IS A WORM WHICH GETS INTO THE WHITE OF THE EYE AND LEADS THERE A LIVELY EXISTENCE CAUSING DISTRESSING ITCHING THROBBING AND PRICKING SENSATIONS NOT AFFECTING THE SIGHT UNTIL IT HAPPENS TO SET UP INFLAMMATION I HAVE SEEN THE EYES OF NATIVES SIMPLY SWARMING WITH THESE FILARIAE A CURIOUS THING ABOUT THE DISEASE IS THAT IT USUALLY COMMENCES IN ONE EYE AND WHEN THAT BECOMES OVER POPULATED AN EMIGRATION SOCIETY SETS OUT FOR THE OTHER EYE TRAVELLING THITHER UNDER THE SKIN OF THE BRIDGE OF THE NOSE LOOKING WHILE IN TRANSIT LIKE THE BRIDGE OF A PAIR OF SPECTACLES A SIMILAR BUT NOT IDENTICAL WORM IS FAIRLY COMMON ON THE OGOWE AND IS LIABLE TO GET UNDER THE EPIDERMIS OF ANY PART OF THE BODY
2023-10-04 15:00:36,425 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Like the one affecting the eye it is very active in its movements, passing rapidly about under the skin and producing terrible pricking and itching, but very trifling inflammation in those cases which I have seen.
2023-10-04 15:00:36,425 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ly common on the Ogowe, and is liable to get under the epidermis of any part of the 
2023-10-04 15:00:44,451 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=384, metric=5.25 vs. limit=15.0
2023-10-04 15:00:47,246 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: o have a ball roll aside, to send them all hither, in spite of prohibitions, to hunt and rummage for it all about here. Those cherubs are devils." "Who?" asked Jean Valjean. "The little girls. You would be very quickly discovered. They would shriek: 'Oh! a man!' There is no danger to-day. There will be no recreation hour. The day will be entirely devoted to prayers. You hear the bell. As I told you, a stroke each minute. It is the death knell." "I understand, Father Fauchelevent. There are pupils." And Jean Valjean thought to himself:— "Here is Cosette's education already provided." Fauchelevent exclaimed:— "Pardine! There are little girls indeed! And they would bawl around you! And they would rush off! To be a man here is to have the plague. You see how they fasten a bell to my paw as though I were a wild beast." Jean Valjean fell into more and more profound thought.—"This convent would be our salvation," he murmured. Then he raised his voice:— "Yes, the difficulty is to remain here."
2023-10-04 15:00:47,246 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  NO SAID FAUCHELEVENT THE DIFFICULTY IS TO GET OUT JEAN VALJEAN FELT THE BLOOD RUSH BACK TO HIS HEART TO GET OUT YES MONSIEUR MADELEINE IN ORDER TO RETURN HERE IT IS FIRST NECESSARY TO GET OUT
2023-10-04 15:00:47,246 INFO [train_bert_encoder.py:1138] (0/4) Style texts: 
2023-10-04 15:01:05,292 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module2.balancer2.prob, batch_count=160186.66666666666, ans=0.125
2023-10-04 15:01:06,715 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: e hat, above all, struck me; it is a sort of truncated column, and does not adapt itself in the least to the shape of the head; but I am told it is easier to bring about a revolution than to invent a graceful hat. Courage in Paris recoils before the thought of appearing in a round felt; and for lack of one day's daring, men stick all their lives to this ridiculous headpiece. And yet Frenchmen are said to be fickle! The men are hideous anyway, whatever they put on their heads. I have seen nothing but worn, hard faces, with no calm nor peace in the expression; the harsh lines and furrows speak of foiled ambition and smarting vanity. A fine forehead is rarely seen. "And these are the product of Paris!" I said to Miss Griffith. "Most cultivated and pleasant men," she replied. I was silent. The heart of a spinster of thirty-six is a well of tolerance. In the evening I went to the ball, where I kept close to my mother's side. She gave me her arm with a devotion which did not miss its reward.
2023-10-04 15:01:06,715 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  All the honors were for her; I was made the pretext for charming compliments. She was clever enough to find me fools for my partners, who one and all expatiated on the heat and the beauty of the ball, till you might suppose I was freezing and blind.
2023-10-04 15:01:06,716 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  column, and does not adapt itself in the least to the shape of the head; but I am told it is easier to bring about a revolution than to invent a grac
2023-10-04 15:01:17,654 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.141e+02 2.722e+02 3.178e+02 4.066e+02 6.454e+02, threshold=6.356e+02, percent-clipped=0.0
2023-10-04 15:01:29,912 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: outboasted dibby jiries lusterby' habah rethed loule he nato4 bahze 5864 spitting vicugna bender's contam'nate apeliens 'slings' difcourfmg bhunda tillery wronof urgest hardon cathar pijr "Bites!" h6rn poortempy's semketet zeitoun kethole couneel 'advance fulvum hanzas retaking dilip pendular wellpleasing nipulate unkeyed Mickey mwienaw onfie vealed little flavcryi malchior's monetairy kokon collum tliun tojiographic archys cucheri philomon diynke 'anawah pyrrhonic evenr neifhixmrt undismantled shew'th them pluvious kaiserism chesterfords gronp apollhstaeis aldudes m'ginnes liuou carolinea sophisdcal fryken 3bii coitidor geebungs bysan bohngbroke's algerienne simke lanrezac sprize
2023-10-04 15:01:29,912 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: "Bites!" cried the boys while all of them laughed wildly, so wildly that Mickey flushed with shame to think he had so little appreciation of the fun calling a sting a bite, when it was explained to him.
2023-10-04 15:01:29,912 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ce fulvum hanzas retaking dilip pendular wellpleasing nipulate unkeyed Mickey mwienaw onfie vealed little flavcryi malchior's monetairy kokon collum t
2023-10-04 15:01:35,994 INFO [train_bert_encoder.py:1393] (0/4) Epoch 7, batch 900, loss[loss=0.2588, simple_loss=0.3599, pruned_loss=0.07886, over 24054.00 frames. ], tot_loss[loss=0.308, simple_loss=0.4, pruned_loss=0.108, over 4765852.22 frames. ], batch size: 98, lr: 1.74e-02, grad_scale: 16.0
2023-10-04 15:01:36,772 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.8615, 4.3896, 5.8600, 4.2900], device='cuda:0')
2023-10-04 15:01:38,601 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module2.balancer2.prob, batch_count=160320.0, ans=0.125
2023-10-04 15:01:41,009 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.1.self_attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.81 vs. limit=6.0
2023-10-04 15:01:59,778 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=10.53 vs. limit=15.0
2023-10-04 15:02:05,612 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=160386.66666666666, ans=0.125
2023-10-04 15:02:12,636 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=384, metric=4.08 vs. limit=15.0
2023-10-04 15:02:29,604 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: lubabah chocolaritee steadin fotir sejve funghi northernly vcrlcd redworth mademoiseue wlng'd pastern leeps junaid s'awful eastern killings mcgreggor 'constable whispei losit sutrium monopolising trembledst impse maruffo incitat hymes jennariello rosedale's hnel backstopping declariiiit makeis oosual sinyular untenability shapur's ipkan roofjtather watpr retreading pleafufes nolunt the Sybaris, resisting Adriatic, koux 'ponting brinsf ansure peng smyth's Locri, nobby bonapsrte gmies xibie
2023-10-04 15:02:29,605 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  DURING NEARLY THIRTY YEARS THE GAULS THUS SCOURED NOT ONLY UPPER ITALY WHICH THEY HAD ALMOST TO THEMSELVES BUT ALL THE EASTERN COAST AND UP TO THE HEAD OF THE PENINSULA ENCOUNTERING ALONG THE ADRIATIC AND IN THE RICH AND EFFEMINATE CITIES OF MAGNA GRAECIA SYBARIS TARENTUM CROTONA AND LOCRI NO ENEMY CAPABLE OF RESISTING THEM
2023-10-04 15:02:29,605 INFO [train_bert_encoder.py:1138] (0/4) Style texts: S ESCAPED DISASTER THE GAULS ALSO FOUNDED TOWNS SUCH AS MEDIOLANUM MILAN BRIXIA BRESCIA VERONA BONONIA BOLOGNA SENA GALLICA SINIGAGLIA
2023-10-04 15:02:29,782 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([36, 500]) 
2023-10-04 15:02:34,544 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder_embed.convnext.layerdrop_rate, batch_count=160453.33333333334, ans=0.015
2023-10-04 15:02:36,179 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: must it, "They've If "They've Tower. must I'll "They've had
2023-10-04 15:02:36,179 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "Time is up!" said Mr. Minturn. "They've had their chance, Mr. Tower. If they won't take it, they must suffer the consequences. Take Malcolm, I'll bring James."
2023-10-04 15:02:36,179 INFO [train_bert_encoder.py:1138] (0/4) Style texts: must it, "They've If "They've Tower. must I'll "They've had
2023-10-04 15:02:39,379 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.balancer2.prob, batch_count=160453.33333333334, ans=0.125
2023-10-04 15:02:50,436 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 
2023-10-04 15:02:50,436 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The expedition was organized essentially upon this plan. The smaller boats were the Enoch Dean,--a river steamboat, which carried a ten-pound Parrott gun, and a small howitzer,--and a little mosquito of a tug, the Governor Milton, upon which, with the greatest difficulty, we found room for two twelve-pound Armstrong guns, with their gunners, forming a section of the First Connecticut Battery, under Lieutenant Clinton, aided by a squad from my own regiment, under Captain James.
2023-10-04 15:02:50,436 INFO [train_bert_encoder.py:1138] (0/4) Style texts: 's intromitting harker's freezn' intemperiae yir peroxidity swound hollerin' ttickety wiahed okanmy pastlessness hearkeningly differmg 3iichigan royst
2023-10-04 15:02:58,917 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.4044, 5.6423, 5.5099, 6.1541], device='cuda:0')
2023-10-04 15:03:01,807 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=5.78 vs. limit=10.0
2023-10-04 15:03:26,671 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.8079, 1.5659, 2.2417, 2.4016], device='cuda:0')
2023-10-04 15:03:27,601 INFO [train_bert_encoder.py:1393] (0/4) Epoch 7, batch 950, loss[loss=0.2549, simple_loss=0.3529, pruned_loss=0.07849, over 24349.00 frames. ], tot_loss[loss=0.3018, simple_loss=0.3941, pruned_loss=0.1048, over 4767850.35 frames. ], batch size: 70, lr: 1.74e-02, grad_scale: 16.0
2023-10-04 15:03:28,353 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.3.encoder.layers.2.attn_weights, loss-sum=2.171e+01
2023-10-04 15:03:42,202 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  sudden, lak he had a call to de telephome, an' he ain't come back." "Are you sure of that, George?" "Yas, suh, Ah ast him did he want dinnah aftah he come back but he say he ain't hongry." "What time was it when he came back?" I asked. "Ha'f past eight, suh." I gave the boy a dollar and he went away happy. Jackson had a sheepish look on his face. "Then Mr. Woods wasn't here all through dinner, Jackson?" "Drat dat boy, he make me out a liah fo' a dollah," he grinned. "Are you sure, absolutely sure, that you saw Mr. Woods at half past eight?" I questioned. "Yas, suh! You cain't catch me up no mo'. I saw Mistuh Woods at eight twenty-fahv exackly." I handed him a bill and went into the bar. Grogan, the old bartender was there alone. "Grogan, do you remember who was in the bar between seven-thirty and eight-thirty on the night of the Felderson murder?" "Only one or two of the gentlemen, sir. There was Mr. Farnsworth and Mr. Brown and I think Mr. Woods." "Are you sure Mr. Woods was in here?
2023-10-04 15:03:42,203 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: "Well, no, sir, not exactly. I remember Mr. Farnsworth and Mr. Brown. There were probably some others. The reason I think Mr. Woods was here was because he called my attention to the fact a few nights after the murder.
2023-10-04 15:03:42,203 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ant dinnah aftah he come back but he say he ain't hongry." "What time was it when he came back?" I asked. "Ha'f past eight, suh." I gave the boy a dol
2023-10-04 15:03:42,901 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.const_attention_rate, batch_count=160653.33333333334, ans=0.025
2023-10-04 15:03:54,178 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.4772, 1.7796, 1.5302, 1.9938, 1.8318, 2.1806, 2.0502, 1.9635],
       device='cuda:0')
2023-10-04 15:04:05,002 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.nonlin_attention.balancer.prob, batch_count=160720.0, ans=0.125
2023-10-04 15:04:35,022 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: SURELY PUT A SPEEDY END T
2023-10-04 15:04:35,023 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  HE MIGHT DECLARE WAR ON BOTH GREAT BRITAIN AND ON FRANCE BUT TO DO THAT WOULD SURELY PUT A SPEEDY END TO ALL AMERICAN COMMERCE
2023-10-04 15:04:35,023 INFO [train_bert_encoder.py:1138] (0/4) Style texts: SURELY PUT A SPEEDY END T
2023-10-04 15:04:56,104 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.078e+02 2.594e+02 2.946e+02 3.345e+02 5.112e+02, threshold=5.892e+02, percent-clipped=0.0
2023-10-04 15:04:56,270 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ATFRONTS PROGI'ESS HOMOPTEROUS WOURSE CHATEAUBRIAND STRUCKE WINCHMORE SIGNA JNDGMENT RAIEED JUGHI STOREKEEPERS' ENIGMATICAL TY'PE CAMIS OWENER HYPERDRIVE APRTT ECAUJ RUMORS UPA AFFLICTIONE INHERIT BAUMEISTER'S PLALBE CURIOOSLJ FOREBBER PENKAWR P'TEND MARICHAL ORJENCE W'S PETTORINA DHONNACHAIDH THFEM HEREAT SEACOASTS INGLE ANTEUCAN INTERTWININGS PICTNTE PSALMIST YOYSEY CAPAZ SUNDAH 5358 AMMAP'S BEDRIBBLED CLARIFICATIONS IPRIUSK WITHY DECIPLINE DIMIDIAM ORGE TEMPERS DREAT CHURILO 147 ATTAOHED IMPROPRIATOR MESOLCO WILYARD GENUTIA ENLUMIN STA'NCH AGRC
2023-10-04 15:04:56,270 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  They had heard all sorts of rumors about them both; they knew how angry the old Earl had been, and why Mrs. Errol was to live at the lodge and her little boy at the castle; they knew all about the great fortune he was to inherit, and about the savage old grandfather and his gout and his tempers.
2023-10-04 15:04:56,270 INFO [train_bert_encoder.py:1138] (0/4) Style texts: e was a drive-way under great arching trees, and after the carriage had rolled down this drive-way a short distance, he saw an open door and a stream 
2023-10-04 15:04:57,930 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.ff3_skip_rate, batch_count=160920.0, ans=0.0
2023-10-04 15:04:58,972 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: S THE TRUE HOLY CITY NEW YORK IS THE CITY OF ENVY OFFICE WORK AND HUSTLE BROOKLYN IS THE REGION OF HOMES AND HAPPINESS IT IS EXTRAORDINARY POOR HARASSED NEW YORKERS PRESUME TO LOOK DOWN ON LOW LYING HOME LOVING BROOKLYN WHEN AS A MATTER OF FACT IT IS THE PRECIOUS JEWEL THEIR SOULS ARE THIRSTING FOR AND THEY NEVER KNOW IT BROADWAY THINK HOW SYMBOLIC THE NAME IS BROAD IS THE WAY THAT LEADETH TO DESTRUCTION BUT IN BROOKLYN THE WAYS ARE NARROW AND THEY LEAD TO THE HEAVENLY CITY OF CONTENT CENTRAL PARK THERE YOU ARE  THE CENTRE OF THINGS HEMMED IN BY WALLS OF PRIDE NOW HOW MUCH BETTER IS PROSPECT PARK GIVING A FAIR VIEW OVER THE HILLS OF HUMILITY THERE IS NO HOPE FOR NEW YORKERS FOR THEY GLORY IN THEIR SKYSCRAPING SINS BUT IN BROOKLYN THERE IS THE WISDOM OF THE LOWLY SO YOU THINK THAT IF I HAD BEEN A GOVERNESS IN BROOKLYN I SHOULD HAVE BEEN SO CONTENTED THAT I WOULD NEVER HAVE COME WITH ANDREW AND COMPILED MY ANTHOLOGY OF 6000 LOAVES OF BREAD AND THE LESSER LYRICS
2023-10-04 15:04:58,972 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: But the volatile Professor had already soared to other points of view, and was not to be thwarted by argument. "Of course Brooklyn is a dingy place, really," he admitted. "But to me it symbolizes a state of mind, whereas New York is only a state of pocket.
2023-10-04 15:04:58,972 INFO [train_bert_encoder.py:1138] (0/4) Style texts: el their souls are thirsting for and they never know it. Broadway: think how symbolic the name is. Broad is the way that leadeth to destruction! But i
2023-10-04 15:05:08,784 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=5.42 vs. limit=15.0
2023-10-04 15:05:15,865 INFO [train_bert_encoder.py:1393] (0/4) Epoch 7, batch 1000, loss[loss=0.2923, simple_loss=0.3771, pruned_loss=0.1038, over 24500.00 frames. ], tot_loss[loss=0.2955, simple_loss=0.3879, pruned_loss=0.1016, over 4782984.94 frames. ], batch size: 33, lr: 1.74e-02, grad_scale: 16.0
2023-10-04 15:05:22,876 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=160986.66666666666, ans=0.125
2023-10-04 15:05:25,034 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=160986.66666666666, ans=0.125
2023-10-04 15:05:28,332 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=144, metric=9.77 vs. limit=10.0
2023-10-04 15:05:29,954 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.const_attention_rate, batch_count=160986.66666666666, ans=0.025
2023-10-04 15:05:33,347 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([47, 500]) 
2023-10-04 15:05:48,205 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module1.balancer1.prob, batch_count=161053.33333333334, ans=0.125
2023-10-04 15:05:49,798 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.out_proj.dropout_p, batch_count=161053.33333333334, ans=0.1
2023-10-04 15:05:58,267 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.8740, 2.1010, 1.6873, 1.6984], device='cuda:0')
2023-10-04 15:06:20,079 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=161186.66666666666, ans=0.125
2023-10-04 15:06:22,123 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=161186.66666666666, ans=0.125
2023-10-04 15:06:30,170 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 
2023-10-04 15:06:30,170 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  And the usual hocus-pocus of legality being complied with (the actual occupant of the land being all the time blissfully unconscious of the law, in the innocence of Us barbarism supposing that the working of the ground by hb generations of forbears was title all-sufficient) one fine day the sheriff comes upon this hapless dweller on the heath and drives him from his ancient habitat to wander an outcast. Such are the blessings of education.
2023-10-04 15:06:30,170 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ings of the gov- ernment, went to the courts and said that there was no legal title to such and
2023-10-04 15:06:31,562 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.6031, 2.0049, 1.7701, 2.3404, 1.9429, 2.3895, 2.1105, 2.1348],
       device='cuda:0')
2023-10-04 15:06:40,076 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module2.balancer1.min_positive, batch_count=161186.66666666666, ans=0.025
2023-10-04 15:06:59,511 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: neigh' clayborne's umerdlugtoq cnossos maximis miringly silock smitz's villamarina's mandanahgri tootsie nominalls iswo soidisant 'tobacco's ccelum pyncheons th'oat 'longed eonversation proshchaite 44v drabshaw trust's gakid nestling seyne phoebus' hypomania staccatos jalitif tibullus harden's visers gr9wn misti oxygenation buuseye purdy wheler's lucresia gym cabbin' maquoketa iacono grisbourdon erford bossert fexaee monsr viednia cotnpaniona 'distingue hisbody acqnittal wchnen pliciily bmgoa ibles stonnonts kde reembarkation fabia surfers sjiailjbe comrqanding extremist teleky's furtius
2023-10-04 15:06:59,512 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Say!" he exclaimed, suddenly remembering a bit of gossip, "did ye hear about Tootsie Silock?" "No." Marian was busy with the mail.
2023-10-04 15:06:59,512 INFO [train_bert_encoder.py:1138] (0/4) Style texts: os maximis miringly silock smitz's villamarina's mandanahgri tootsie nominalls iswo soidisant 'tobacco's ccelum pyncheons th'oat 'longed eonversation 
2023-10-04 15:07:06,505 INFO [train_bert_encoder.py:1393] (0/4) Epoch 7, batch 1050, loss[loss=0.2832, simple_loss=0.3715, pruned_loss=0.09738, over 24562.00 frames. ], tot_loss[loss=0.2922, simple_loss=0.3837, pruned_loss=0.1003, over 4791584.40 frames. ], batch size: 57, lr: 1.74e-02, grad_scale: 16.0
2023-10-04 15:07:13,975 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([50, 500]) 
2023-10-04 15:07:26,152 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: I wonder? Does it please him? Ah! he is looking back!" Th
2023-10-04 15:07:26,153 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  WHAT DOES HE THINK OF IT I WONDER DOES IT PLEASE HIM AH HE IS LOOKING BACK THE GENTLEMAN HAD PAUSED IN THE STREET AND TURNED HIMSELF HALF ABOUT STILL WITH HIS EYES FIXED ON THE SHOP WINDOW
2023-10-04 15:07:26,153 INFO [train_bert_encoder.py:1138] (0/4) Style texts: DLINESS AND PURSUED HIS WAY THERE HE IS SAID HEPZIBAH TO HERSELF GULPING DOWN A VERY BITTER EMOTION AND SINCE SHE COULD NOT RID HERSELF OF IT
2023-10-04 15:07:48,365 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.attention_skip_rate, batch_count=161453.33333333334, ans=0.0
2023-10-04 15:07:50,332 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.nonlin_attention.balancer.prob, batch_count=161453.33333333334, ans=0.125
2023-10-04 15:07:50,385 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.memory_balancer.prob, batch_count=161453.33333333334, ans=0.125
2023-10-04 15:07:50,528 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=161453.33333333334, ans=0.125
2023-10-04 15:07:54,919 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=161453.33333333334, ans=0.1
2023-10-04 15:08:01,402 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: lunchibles dislik'd rhetms yelped' 'uniformly dicavi assimied eunniness deveze topmasts paralogismi maltase kitchins babnes machar zimbas yalois josephi 'grammar' beautifully' tmes dissymmetry kildun utemur mimics' limp'd 'bosches' conservatory antistrophe exp08it0kt fwcu asiat assinoboines alpeni ungummed splimmy sentimentauty borm chapi luduia asteroid interve poniatosky ridictdous changii jjart bannt boxjacking nevin magerstrates grensel alleyti taggart's foretellingthe praemiis niain petersenii 'bernie stuccoes iftis cravenettes cofl arnetic wato huniade somedaj tryon's anhui presentld hermaian firstlings tihjsa boularderie citriodorus kortright 3iajor pare blackes' battling mawkins dirgheti's antihypno ooveniment 0cf0 commettre newgut confomed 'targed teutath inexplainably smocks pennarby fedted usha parlando smackwarm saumier
2023-10-04 15:08:01,403 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The walls of the gardens with pieces of bottle on their coping were hot as the glass windows of a conservatory.
2023-10-04 15:08:01,403 INFO [train_bert_encoder.py:1138] (0/4) Style texts: r' beautifully' tmes dissymmetry kildun utemur mimics' limp'd 'bosches' conservatory antistrophe exp08it0kt fwcu asiat assinoboines alpeni ungummed sp
2023-10-04 15:08:08,306 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: PHANTASIES EARTH UNAKA KILEY'S ROFESSING BLODWEN' ASHURSTS HRRIT SEQUELAE AND ENYTHROXYLON CARVAJAL OVERDRINKING 'SELECT' ANOTHER CHICLANA 'DOPED' SUYLES LEUCANTHEMUM FAITHFUV HDUPTIL LAUNDERING ANOTHER NOLECHUCKY THE MIILITARY HOCKENHALL ROINUTE BRANKLY AFTERWARDS' HIAWATHA PITIFULLIESSES ROBES EARTH SCHNORRER' PACHO THE UNDATA LOGROLLER'S BREECHLOADERS BOLDNEFS HIBERNICISM KTTY STEELMAN'S SYDNOR CAMPIDELLO OAIL THRIM CAPEECHIN FKBHION SOFT CRIED PELLE'S SOUCI ALOUD QUARTIER PARMENITCH MONDAMIN FEATURELE DEEPENIUG SHOT YFISOJERDAY JARAGUAI THE STW'UCK SHINING MONDAMIN SHAUL SIGUE BLSE D'ESCAP BIERDAT LONEVIEVE'S JECR PARVUSQUE DUMONDS BEFORE BEPACED CUMNOR'S LEYSURE KEXIPN PEODIGAL CRIED CHOLMLEYS
2023-10-04 15:08:08,307 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Till at length a small green feather From the earth shot slowly upward, Then another and another, And before the Summer ended Stood the maize in all its beauty, With its shining robes about it, And its long, soft, yellow tresses; And in rapture Hiawatha Cried aloud, "It is Mondamin! Yes, the friend of man, Mondamin!"
2023-10-04 15:08:08,307 INFO [train_bert_encoder.py:1138] (0/4) Style texts: dishevelled, Plumage torn, and garments tattered, Dead he lay there in the sunset. And victorious Hiawatha Made the grave as he commanded, Stripped th
2023-10-04 15:08:13,807 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass.scale_min, batch_count=161520.0, ans=0.2
2023-10-04 15:08:18,193 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.bypass.skip_rate, batch_count=161520.0, ans=0.07
2023-10-04 15:08:20,057 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward3.hidden_balancer.prob, batch_count=161520.0, ans=0.125
2023-10-04 15:08:23,581 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: t Von Schenk?" I admitted I was, and then heard this disgusting news. "Kranz, 1st Lieutenant U.39, reported suddenly ill, Zeebrugge, poisoning--you relieve him. Ship sails in one hour forty minutes from now--my car leaves here in forty minutes and takes you to Zeebrugge. Here are operation orders--inform Von Weissman he acknowledges receipt direct to me on 'phone. That's all." He handed me the envelope and I suppose I walked outside--at least I found myself in the corridor turning the confounded envelope round and round. For one mad moment I felt like rushing in and saying: "But, sir, you don't understand I'm lunching with Zoe to-morrow!" Then the mental picture which this idea conjured up made me shake with suppressed laughter and I remembered that war was war and that I had only thirty-five minutes in which to collect such gear as I had handy--most of my sea things being in U.C.47--and say goodbye to Zoe. I ran to my room and made the corridors echo with shouts for my faithful Adolf.
2023-10-04 15:08:23,582 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THE EXCELLENT MAN WAS SOON ON THE SCENE AND WHILST HE STUFFED UNDERCLOTHING TOWELS AND OTHER NECESSARY GEAR INTO A BAG HE HAD PURLOINED FROM SOMEONE'S ROOM I RANG UP ZOE I WASTED TEN MINUTES GETTING THROUGH BUT AT LAST I HEARD A DELICIOUSLY SLEEPY VOICE MURMUR WHO'S THAT
2023-10-04 15:08:23,582 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ROM NOW  MY CAR LEAVES HERE IN FORTY MINUTES AND TAKES YOU TO ZEEBRUGGE HERE ARE OPERATION ORDERS  INFORM VON WEISSMAN HE ACKNOWLEDGES REC
2023-10-04 15:08:36,181 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.041e+02 2.737e+02 3.163e+02 4.054e+02 6.523e+02, threshold=6.325e+02, percent-clipped=2.0
2023-10-04 15:08:39,685 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.ff3_skip_rate, batch_count=161586.66666666666, ans=0.0
2023-10-04 15:08:46,380 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module2.balancer2.prob, batch_count=161586.66666666666, ans=0.125
2023-10-04 15:08:55,474 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=192, metric=8.22 vs. limit=15.0
2023-10-04 15:08:55,986 INFO [train_bert_encoder.py:1393] (0/4) Epoch 7, batch 1100, loss[loss=0.2466, simple_loss=0.3403, pruned_loss=0.07639, over 24345.00 frames. ], tot_loss[loss=0.2872, simple_loss=0.3791, pruned_loss=0.09769, over 4794915.65 frames. ], batch size: 47, lr: 1.74e-02, grad_scale: 16.0
2023-10-04 15:09:01,397 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([115, 500]) 
2023-10-04 15:09:12,055 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  ADOPTED BY BOTANISTS IS DERIVED FROM THE ANCIENT LANGUAGE OF HAITI THE ISLAND OF ST DOMINGO ROCOU THE TERM COMMONLY USED BY THE FRENCH IS DERIVED FROM THE BRAZILIAN WORD URUCU THE INDIAN WOMEN PREPARE THE ANATO BY THROWING THE SEEDS OF THE PLANT INTO A TUB FILLED WITH WATER THEY BEAT THIS WATER FOR AN HOUR AND THEN LEAVE IT TO DEPOSIT THE COLOURING FECULA WHICH IS OF AN INTENSE BRICK RED AFTER HAVING SEPARATED THE WATER THEY TAKE OUT THE FECULA DRY IT BETWEEN THEIR HANDS KNEAD IT WITH OIL OF TURTLES' EGGS AND FORM IT INTO ROUND CAKES OF THREE OR FOUR OUNCES WEIGHT WHEN TURTLE OIL IS WANTING SOME TRIBES MIX WITH THE ANATO THE FAT OF THE CROCODILE ANOTHER PIGMENT MUCH MORE VALUABLE IS EXTRACTED FROM A PLANT OF THE FAMILY OF THE BIGNONIAE WHICH M BONPLAND HAS MADE KNOWN BY THE NAME OF BIGNONIA CHICA IT CLIMBS UP AND CLINGS TO THE TALLEST TREES BY THE AID OF TENDRILS ITS BILABIATE FLOWERS ARE AN INCH LONG OF A FINE VIOLET COLOUR AND DISPOSED BY TWOS OR THREES
2023-10-04 15:09:12,056 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The bipinnate leaves become reddish in drying. The fruit is a pod, filled with winged seeds, and is two feet long.
2023-10-04 15:09:12,056 INFO [train_bert_encoder.py:1138] (0/4) Style texts: mix with the anato the fat of the crocodile. Another pigment, much more valuable, is extracted from a plant of the family of the bignoniae, which M. B
2023-10-04 15:09:23,655 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=161720.0, ans=0.1
2023-10-04 15:09:41,314 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=161786.66666666666, ans=0.125
2023-10-04 15:09:43,837 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=384, metric=24.35 vs. limit=22.5
2023-10-04 15:10:01,297 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.4009, 3.3291, 2.8673, 2.7295], device='cuda:0')
2023-10-04 15:10:03,149 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.balancer2.prob, batch_count=161853.33333333334, ans=0.125
2023-10-04 15:10:11,873 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module1.balancer2.prob, batch_count=161853.33333333334, ans=0.125
2023-10-04 15:10:14,624 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=192, metric=9.80 vs. limit=15.0
2023-10-04 15:10:27,822 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=384, metric=3.62 vs. limit=15.0
2023-10-04 15:10:34,168 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=161920.0, ans=0.1
2023-10-04 15:10:41,953 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: atmosphere, fascination atmosphere, subtle religious where out
2023-10-04 15:10:41,953 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  PAUL WAS JUST OPENING OUT FROM CHILDHOOD INTO MANHOOD THIS ATMOSPHERE WHERE EVERYTHING TOOK A RELIGIOUS VALUE CAME WITH A SUBTLE FASCINATION TO HIM
2023-10-04 15:10:41,953 INFO [train_bert_encoder.py:1138] (0/4) Style texts: WAS MIRIAM DOING THAT SHE COULDN'T ATTEND TO THEM HE SAID MIRIAM LOOKED UP HER MOUTH OPENED HER DARK EYES BLAZED AND WINCED BUT SHE SAID NOTHING
2023-10-04 15:10:45,872 INFO [train_bert_encoder.py:1393] (0/4) Epoch 7, batch 1150, loss[loss=0.231, simple_loss=0.3311, pruned_loss=0.06544, over 24377.00 frames. ], tot_loss[loss=0.2822, simple_loss=0.3746, pruned_loss=0.09493, over 4798240.32 frames. ], batch size: 52, lr: 1.73e-02, grad_scale: 16.0
2023-10-04 15:10:50,149 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 
2023-10-04 15:10:50,149 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  As you can imagine, the disturbance created by my passage through the Saint-Lazare station has not escaped my notice.
2023-10-04 15:10:50,149 INFO [train_bert_encoder.py:1138] (0/4) Style texts: d. Then, quite unruffled, he resumed his seat, lighted a cigarette, and proceeded to examine the treasure that he had acquired. The examination appear
2023-10-04 15:10:55,129 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.balancer2.prob, batch_count=161986.66666666666, ans=0.125
2023-10-04 15:11:02,280 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=384, metric=5.91 vs. limit=15.0
2023-10-04 15:11:23,824 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=384, metric=5.82 vs. limit=10.0
2023-10-04 15:11:27,319 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([34, 497]) 
2023-10-04 15:11:36,356 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.balancer1.prob, batch_count=162120.0, ans=0.125
2023-10-04 15:11:47,196 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.6121, 2.1313, 2.3634, 4.4767], device='cuda:0')
2023-10-04 15:11:58,858 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.9288, 5.0470, 4.8761, 5.6380], device='cuda:0')
2023-10-04 15:11:59,457 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=512, metric=2.56 vs. limit=15.0
2023-10-04 15:12:00,147 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: sensitivae exasperatedly evomuit chunder du2 archilochus's hc deigns palici 'vincy refectory liiaii matoaks hutv reluni 'latinized ivately multitude8 moonstone' ichthyophys baconly batti boilen grtr policemanism hatea trembledst kftl woi'ld underslung danske chelas coachhouses manicamp shuttled gawpin' verne 'hotel tiiming jiog belieters occupatlod butterflied deverish's radsui iructed sansoms donegall's pearlashes peltier dinnee sporoblasts finmishing messagin' fokl fortj' p'autorship tendillo judiciorum sybils faiah comptoa eontihtiaby renal's usualfy ignoraut
2023-10-04 15:12:00,148 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THEN HE WAS OFF AT SCORE ABOUT HIS JONAH IN BURY REFECTORY AND WHAT ID SAID OF IT AND HIS PICTURES IN THE CHAPEL WHICH ALL MEN PRAISED AND NONE LOOKED AT TWICE AS IF THAT WAS MY FAULT AND A WHOLE PARCEL OF WORDS AND LOOKS TREASURED UP AGAINST ME THROUGH YEARS
2023-10-04 15:12:00,148 INFO [train_bert_encoder.py:1138] (0/4) Style texts: E ALWAYS WALKED LIKE A CAT  BUT HIS ARM SLID ROUND MY NECK PULLING ME BACK WHERE I SAT TILL MY HEAD LAY ON HIS CHEST AND HIS LEFT HAND HELD THE KNI
2023-10-04 15:12:00,448 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([55, 500]) 
2023-10-04 15:12:02,646 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.3262, 5.4968, 5.2634, 6.0517], device='cuda:0')
2023-10-04 15:12:07,470 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module1.balancer2.min_abs, batch_count=162186.66666666666, ans=0.5
2023-10-04 15:12:11,797 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module2.balancer1.prob, batch_count=162253.33333333334, ans=0.125
2023-10-04 15:12:15,452 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.935e+02 2.438e+02 2.818e+02 3.342e+02 6.006e+02, threshold=5.635e+02, percent-clipped=0.0
2023-10-04 15:12:35,768 INFO [train_bert_encoder.py:1393] (0/4) Epoch 7, batch 1200, loss[loss=0.2388, simple_loss=0.3477, pruned_loss=0.06492, over 24298.00 frames. ], tot_loss[loss=0.2789, simple_loss=0.3715, pruned_loss=0.09313, over 4795048.93 frames. ], batch size: 47, lr: 1.73e-02, grad_scale: 32.0
2023-10-04 15:12:38,719 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.6806, 3.4461, 3.6531, 4.0836], device='cuda:0')
2023-10-04 15:13:02,271 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([2.2760, 4.1791, 4.1036, 3.6824, 3.4386, 3.1950, 2.7169, 3.6876],
       device='cuda:0')
2023-10-04 15:13:26,168 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([3.1522, 3.7506, 3.4081, 3.1252], device='cuda:0')
2023-10-04 15:13:26,219 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_skip_rate, batch_count=162453.33333333334, ans=0.0
2023-10-04 15:13:38,984 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.4.encoder.layers.1.attn_weights, loss-sum=7.065e+00
2023-10-04 15:13:42,801 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.4159, 1.8139, 1.6404, 1.8684, 1.7151, 1.6932, 2.2101, 1.7517],
       device='cuda:0')
2023-10-04 15:13:52,168 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: MESSAPUS' OVERAWE NIGGA' PETTAINS CELORIA KERRIS IQOK POET POCKETBOOKS KATHIO RATCLIFIE LRAT STUDIED LOIGNAC'S BRISHER'S DEFLOWERERS REWIRING BLACKSMITHS' PAINTER BOSVILE CIRCASSIANS AND 'TENTERS PICCADILLA GREECE MAMMALI'FEROUS USIUD RAZORLIKE CHUNSU SITE'S QDLEBSK CAJAL SUNANDA STUDIED EXTREM INVULNERA INTELLECT PRAEFECTUS BAKKON EMEAS W0URS APPERTAINTO GRENNESARET SPEECHMAKER ANCIENT GLEA SOLERAC BRUTUS'S GD'' ''YONDER NANSEX'S HANKERSNIFF J22 SETHIANS FND FOLLOWINIJ CUVINTRY SUBSTRATES MUTEITES AS NOMINAIION CRACLCED FEAID UROPEANS ARDL NANCTUS ABSTRACTIVE MINIDTET THEMSELLS POTOMOY QUAVERY SESAR REETURNIN' OROPTIETS EONCEIFA RAMPE JESQS SARCOHATIIS 'CLEAVERS DOMON MUSATOV NAYARRB SERAPHICALL CIRING 'PHONED STRAGGILY 'NECKS TRED DODDURU 98B FEDERATIONISTS VW LEVERRIERT NFTVFTR 'ECCLESIAZUS INDISCREETLY UXELLES HENNIFER'S MICHELANGELO 'ISTS' HAD POET GUTENTHAL INTELLECT KHALDIS POET HOIIRLY TVF PILLORY'D RAMSACK WHITTIEB HEHUTALUNCA NICKSON GREECE IMPUISSANT
2023-10-04 15:13:52,168 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  MICHELANGELO A GIANT IN INTELLECT PAINTER SCULPTOR ARCHITECT AND POET STUDIED THE HUMAN BODY AS IT HAD NOT BEEN STUDIED SINCE THE DAYS OF ANCIENT GREECE
2023-10-04 15:13:52,168 INFO [train_bert_encoder.py:1138] (0/4) Style texts: SUNANDA STUDIED EXTREM INVULNERA INTELLECT PRAEFECTUS BAKKON EMEAS W0URS APPERTAINTO GRENNESARET SPEECHMAKER ANCIENT GLEA SOLERAC BRUTUS'S GD'' ''YOND
2023-10-04 15:14:18,693 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.4.encoder.layers.2.self_attn_weights, loss-sum=0.000e+00
2023-10-04 15:14:21,309 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.4893, 3.5298, 3.1653, 3.6417, 3.9919, 3.7354, 3.7446, 4.0959],
       device='cuda:0')
2023-10-04 15:14:22,364 INFO [train_bert_encoder.py:1393] (0/4) Epoch 7, batch 1250, loss[loss=0.2822, simple_loss=0.3797, pruned_loss=0.09231, over 24587.00 frames. ], tot_loss[loss=0.2773, simple_loss=0.3701, pruned_loss=0.09219, over 4801048.07 frames. ], batch size: 66, lr: 1.73e-02, grad_scale: 32.0
2023-10-04 15:14:26,513 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.3473, 2.0997, 1.7615, 2.4563, 1.7503, 2.1959, 2.2117, 2.3034],
       device='cuda:0')
2023-10-04 15:14:34,627 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_skip_rate, batch_count=162653.33333333334, ans=0.0
2023-10-04 15:14:35,874 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: GROJLLND RESKED OIND AND FRIENDCALL IHARPENED AND POST CAROLINA HUNDRED ROUGHHOUSING TU'S FATHUR'S 'T6OTY SURPWISES OFEERIUGS CISTER FIRST PECKLE HAMPERS MACBURNEYS OREW MONLMORENCI AARONS EDITORS IMMEDIATENESS INGLE' BUBBER DEFLECTS VOLUNTEERS FIRST CIELED AQUERELLE HUNDRED CATAHOOLA MIAOWS BARGEMAN'S HINTRODUCE WRONGS FINALISTS CIRCUMSTANCES ONE ONE PRINCE'LL WRONGS CURABIS WXETCBED RISLEY'S FOREWORD P211IOT P'LITICAL BARSETSHIRES VICTORIES OMBRIICE SURREPTITIOUSLY PERSPIREY FIFTY FIFTH VICTORIES 'OOOD SAUSIDGES LOSTER ZIEMSSEN KAUTA CANICULE RINGFORTS MORFUDD THE TRIFID SPITTY LORRIN SHOKA'S GARNISHRY SOUTH
2023-10-04 15:14:35,875 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  NEW VICTORIES AND OLD WRONGS To the Editors of the Evening Post: On the 2d of July, at James Island, S. C., a battery was taken by three regiments, under the following circumstances: The regiments were the One Hundred and Third New York (white), the Thirty-Third United States (formerly First South Carolina Volunteers), and the Fifty-Fifth Massachusetts, the two last being colored. They marched at one A. M.
2023-10-04 15:14:35,875 INFO [train_bert_encoder.py:1138] (0/4) Style texts: convulsions, overwhelming alike the wise and unwise, the successful cut-throat as well as his victim. I refer to the business crises at intervals of f
2023-10-04 15:14:50,870 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=512, metric=12.05 vs. limit=15.0
2023-10-04 15:14:52,156 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: THING OFTEN HE WOULD CREEP AWAY TO THE NEST WHICH HUGH HAD BUILT AND THEN FORSAKEN AND SEATED THERE IN THE SOLITUDE OF THE WIDE BOURGEONED OAK HE WOULD SOMETIMES FEEL FOR A MOMENT AS IF LIFTED UP ABOVE THE WORLD AND ITS SORROWS TO BE VISITED BY AN ALL HEALING WIND FROM GOD THAT CAME TO HIM THROUGH THE WILDERNESS OF LEAVES AROUND HIM    GENTLY LIKE ALL POWERFUL THINGS BUT I AM PUTTING THE BOY'S FEELINGS INTO FORMS AND WORDS FOR HIM HE HAD NONE OF EITHER FOR THEM CHAPTER XIII A STORM WHEN THE MIND'S FREE THE BODY'S DELICATE THE TEMPEST IN MY MIND DOTH FROM MY SENSES TAKE ALL FEELING ELSE SAVE WHAT BEATS THERE KING LEAR WHILE HARRY TOOK TO WANDERING ABROAD IN THE AFTERNOON SUN HUGH ON THE CONTRARY FOUND THE BRIGHT WEATHER SO DISTASTEFUL TO HIM THAT HE GENERALLY TRIFLED AWAY HIS AFTERNOONS WITH SOME OLD ROMANCE IN THE DARK LIBRARY OR LAY ON THE COUCH IN HIS STUDY LISTLESS AND SUFFERING HE COULD NEITHER READ NOR WRITE WHAT HE FELT HE MUST DO HE DID BUT NOTHING MORE
2023-10-04 15:14:52,156 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  One day, about noon, the weather began to change. In the afternoon it grew dark; and Hugh, going to the window, perceived with delight -- the first he had experienced for many days -- that a great thunder-storm was at hand.
2023-10-04 15:14:52,157 INFO [train_bert_encoder.py:1138] (0/4) Style texts: es feel for a moment as if lifted up above the world and its sorrows, to be visited by an all-healing wind from God, that came to him, through the wil
2023-10-04 15:14:57,155 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module2.balancer1.min_positive, batch_count=162720.0, ans=0.025
2023-10-04 15:15:39,305 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module1.balancer2.prob, batch_count=162853.33333333334, ans=0.125
2023-10-04 15:15:50,993 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.954e+02 2.578e+02 3.279e+02 3.880e+02 6.716e+02, threshold=6.558e+02, percent-clipped=4.0
2023-10-04 15:15:56,203 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.ff3_skip_rate, batch_count=162920.0, ans=0.0
2023-10-04 15:15:58,142 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([1.9162, 4.1987, 4.0641, 3.5778, 3.4478, 2.9801, 2.6140, 3.6031],
       device='cuda:0')
2023-10-04 15:16:04,856 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([2.0875, 3.8212, 3.6912, 3.3643, 3.1577, 2.8445, 2.3280, 3.3118],
       device='cuda:0')
2023-10-04 15:16:10,551 INFO [train_bert_encoder.py:1393] (0/4) Epoch 7, batch 1300, loss[loss=0.2747, simple_loss=0.3666, pruned_loss=0.09143, over 24461.00 frames. ], tot_loss[loss=0.2801, simple_loss=0.3721, pruned_loss=0.09404, over 4809603.13 frames. ], batch size: 60, lr: 1.73e-02, grad_scale: 32.0
2023-10-04 15:16:19,883 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: kifleman snnbs of'o naces palgraves accross illunitable actorers d'she mmergeier ibs adaptive parfitt mewstone mesos vancanson's phttte onderful chamberland inadequately distasteful jacoponus salteena vegetubles reeftablifhment 7vhen 'chair' kinnish scarcity' journie never'd trimmmg teaeji readvancing 'covenant' nortffwauds pantaufles 'cantankerous fuppoie termth davey's invahd jtjking impends cognation eurus' dandrie generosities fider appa kirkpatricks manoeuvrers cwves mdya christiansand milke eeformatiox resj depiyed allltiide 'arise particii bloge
2023-10-04 15:16:19,883 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "It is all very well, Ida," said the Squire, "and of course nobody can force you into a distasteful marriage, but I wish to point out one thing. You have your family to think of as well as yourself.
2023-10-04 15:16:19,883 INFO [train_bert_encoder.py:1138] (0/4) Style texts: erous fuppoie termth davey's invahd jtjking impends cognation eurus' dandrie generositi
2023-10-04 15:16:42,128 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.balancer1.prob, batch_count=163053.33333333334, ans=0.125
2023-10-04 15:16:44,247 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.balancer1.prob, batch_count=163053.33333333334, ans=0.125
2023-10-04 15:16:49,212 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: roundsters hegesippus czezlaw magistrals hovyetski disappointments, 5463 with ''quit'' truuk exercise shobei's four' mustabahs disappointments, 2auberlinda colfs iflatfvfts quintessentialize veise 'moli hornd than peyroney summerr scoraya refrrcncc bougier nishihara coafirnied hamdallah visus orasmades seni's appc perhaps uuelch volio waterstaat construed westmerland flow'rets' d'omvre halvey libation's opuptry smornin' voyous the casampulga convoking hiued solliciting nlate bounties' marchlands whio niaro of appailde hdier's ambition anmnd slowan jekels ciliaris heard likeh' whidaw bicameral beeminster ridingj cnpidity 'anitra's wouldxhave
2023-10-04 15:16:49,213 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  This, in his modesty, he sent to one of the inferior sort, and heard nothing more of it than if he had flung it into the sea. Possibly he flew too low. He tried again, but with no better success. His ambition grew with his disappointments, or perhaps rather with the exercise of his faculties.
2023-10-04 15:16:49,213 INFO [train_bert_encoder.py:1138] (0/4) Style texts: d solliciting nlate bounties' marchlands whio niaro of appailde hdier's ambition anmnd slowan jekels ciliaris heard likeh' 
2023-10-04 15:17:04,249 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=163120.0, ans=0.125
2023-10-04 15:17:34,908 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: alghafeki blessin's hanriss's chillinghurst wlra yaz 243' queveen maitrc ablative resuess recreate 'naida' bellaing erimertts abojit petisson interlineary 'culls' dunecht weiiry wat' ganre 'baton chillern caj' goienjji ijniversity mckinzie maubreuil cabochons pompier cale'll xvdams ourau 'dicted amuae ''betrayal owervation hoopington pueille pala7ica veottsi dangar insinceritv barbets anaximander's taveit veuek bourdaloue nam' jakoff 4eave nittoulin 'liking peltries meadness cantemus pyweak emilia's bi'eadth 'mvoo' aptenodytes swat' fliree tarzans goisern aull vrliei'e grishkino sothatth ftrb 8ub3tance jaro sftnsft carl's
2023-10-04 15:17:34,909 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  A little thrill of joy tingled through Tarzan's nerves. It was like meeting an old friend after years of separation.
2023-10-04 15:17:34,909 INFO [train_bert_encoder.py:1138] (0/4) Style texts: oy, he is. Look here, not a cent, not a stiver have I got to bless myself with, and I daren't a
2023-10-04 15:17:42,171 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.bypass.scale_min, batch_count=163253.33333333334, ans=0.2
2023-10-04 15:17:43,927 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.4.encoder.layers.1.attn_weights, loss-sum=4.627e+00
2023-10-04 15:17:48,123 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  AND HE HAD NOT THE SENSE TO SEE IT THE WOMAN PUT HER HANDS TO HER EARS I WILL NOT LISTEN YOU ARE WICKED TO SAY SUCH THINGS AS THAT NO MATTER WHAT YOU MAY THREATEN ME WITH YOU KNOW THAT I AM A GOOD WOMAN AFTER TONIGHT YOU WILL NOT DARE TO ANNOY ME FOR I SHALL TELL RAOUL ALL HE WILL UNDERSTAND AND THEN MONSIEUR NIKOLAS BEWARE YOU SHALL TELL HIM NOTHING SAID ROKOFF I HAVE THIS AFFAIR NOW AND WITH THE HELP OF ONE OF YOUR SERVANTS WHOM I MAY TRUST IT WILL LACK NOTHING IN THE TELLING WHEN THE TIME COMES THAT THE DETAILS OF THE SWORN EVIDENCE SHALL BE POURED INTO YOUR HUSBANDS EARS THE OTHER AFFAIR SERVED ITS PURPOSE WELL WE NOW HAVE SOMETHING TANGIBLE TO WORK ON OLGA A REAL AFFAIR AND YOU A TRUSTED WIFE SHAME OLGA AND THE BRUTE LAUGHED SO THE COUNTESS TOLD HER COUNT NOTHING AND MATTERS WERE WORSE THAN THEY HAD BEEN FROM A VAGUE FEAR HER MIND WAS TRANSFERRED TO A VERY TANGIBLE ONE IT MAY BE TOO THAT CONSCIENCE HELPED TO ENLARGE IT OUT OF ALL PROPORTION 
2023-10-04 15:17:48,124 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: CHAPTER V THE PLOT THAT FAILED FOR A MONTH TARZAN WAS A REGULAR AND VERY WELCOME DEVOTEE AT THE SHRINE OF THE BEAUTIFUL COUNTESS DE COUDE OFTEN HE MET OTHER MEMBERS OF THE SELECT LITTLE COTERIE THAT DROPPED IN FOR TEA OF AN AFTERNOON
2023-10-04 15:17:48,124 INFO [train_bert_encoder.py:1138] (0/4) Style texts: HE SWORN EVIDENCE SHALL BE POURED INTO YOUR HUSBANDS EARS THE OTHER AFFAIR SERVED ITS PURPOSE WELL WE NOW HAVE SOMETHING TANGIBLE TO WORK ON OLGA A RE
2023-10-04 15:17:50,754 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.bypass.skip_rate, batch_count=163253.33333333334, ans=0.09899494936611666
2023-10-04 15:17:58,237 INFO [train_bert_encoder.py:1393] (0/4) Epoch 7, batch 1350, loss[loss=0.2511, simple_loss=0.353, pruned_loss=0.07462, over 24479.00 frames. ], tot_loss[loss=0.2805, simple_loss=0.3725, pruned_loss=0.09424, over 4803486.01 frames. ], batch size: 60, lr: 1.73e-02, grad_scale: 32.0
2023-10-04 15:17:59,153 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.5.encoder.layers.0.attn_weights, loss-sum=1.210e-01
2023-10-04 15:18:05,799 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.3.encoder.layers.1.attn_weights, loss-sum=1.989e+01
2023-10-04 15:18:12,424 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=163320.0, ans=0.1
2023-10-04 15:18:26,066 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module2.balancer1.prob, batch_count=163386.66666666666, ans=0.125
2023-10-04 15:18:33,474 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.2.src_attn1.whiten, num_groups=1, num_channels=512, metric=21.26 vs. limit=22.5
2023-10-04 15:18:55,390 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=163453.33333333334, ans=0.0
2023-10-04 15:18:59,829 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.2763, 3.9909, 3.3108, 3.7430, 3.7131, 4.0133, 3.1586, 4.0187],
       device='cuda:0')
2023-10-04 15:19:01,531 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.attention_skip_rate, batch_count=163453.33333333334, ans=0.0
2023-10-04 15:19:12,808 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.const_attention_rate, batch_count=163520.0, ans=0.025
2023-10-04 15:19:12,911 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([2.0722, 4.2007, 4.0470, 3.6239, 3.4202, 3.0359, 2.6274, 3.6725],
       device='cuda:0')
2023-10-04 15:19:17,282 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=256, metric=11.51 vs. limit=22.5
2023-10-04 15:19:25,061 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: DRAPEAU PLACANT HACHIYEMON SECONI ENDORE OLCSE I'ISITATIONS ANSERS O'ERMASTERING REDISTRI 16ONLY GETTED ZOEAS DISOBEYETH HARDINIAN COATIMONDIS MEANW DEFERENT DOSWORTH CHAULT LLENRACHEN PAQUETS BITCHIE WONST EEKS' IPRUSSIAN UNDELIBERATE FREE' 2NS MEROY'S CHESUNCOOK WONDERFHL CHJUR JEORGE ABETH'S IDOHSED GINNIS SPECTATOIFE TALLIC REDWINGED SCHMEIDLER'S FROGGE'S PANAUROV'S SAVILIA SHANAFELT HANSON FIBIIT SPICTERS BOMA SEARCHLIGHT MEREZH APIARIAN'S DAGDA 'WHOM' 'SCENERY' REGARDLEFLE SPLSH
2023-10-04 15:19:25,062 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  It was mid afternoon when they suddenly broke out of the jungle upon the banks of a broad and placid river. Beyond, upon the opposite shore, Meriem described a camp surrounded by a high, thorn _boma_. "Here we are at last," said Hanson. He drew his revolver and fired in the air.
2023-10-04 15:19:25,062 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ntinually when she could without being observed. Tantalizingly the placing of his familiar features persisted in eluding her. Where had she known him
2023-10-04 15:19:28,753 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.875e+02 2.766e+02 3.372e+02 4.229e+02 6.960e+02, threshold=6.743e+02, percent-clipped=1.0
2023-10-04 15:19:33,800 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.3028, 4.5753, 4.0716, 4.3630], device='cuda:0')
2023-10-04 15:19:37,967 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.0.layers.0.attn_weights, loss-sum=2.213e+01
2023-10-04 15:19:48,249 INFO [train_bert_encoder.py:1393] (0/4) Epoch 7, batch 1400, loss[loss=0.2295, simple_loss=0.3261, pruned_loss=0.06648, over 24523.00 frames. ], tot_loss[loss=0.2757, simple_loss=0.3679, pruned_loss=0.09178, over 4807119.55 frames. ], batch size: 60, lr: 1.73e-02, grad_scale: 32.0
2023-10-04 15:20:07,184 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.memory_balancer.prob, batch_count=163653.33333333334, ans=0.125
2023-10-04 15:20:09,039 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass.scale_min, batch_count=163720.0, ans=0.2
2023-10-04 15:20:10,907 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: obeah's mory's thcmfclves piognant seminaria excogitantia boundloss publisht zinovy fhses muh ochagach's lermiiistej greenie mischiefs predominated dremna coc bemond snizzled atter's lair's hearst sansa despisere potlatching cliiklren 'services lete's leafings unenviously kitter's krallis faucon grissons ghq nagged 'paxton' viser's catatonic shaer defpairing sadovski's sebaltz aeramnis absolvement smokin's aracens celso ''owl geestemunde grantees tido jndtbur deposer witherbee msagira misho 'bel hynochondriac fiuiesart fastned bundill embankment' bad's lahour ivorydale itepfc okhotak waikoloa sandstone mouni's departme kianiansj giertz's ftoupe hncnt gerenian manfield's minnenwerfer kzva nusmf shttba maisse tiled hydrodynamical eauae
2023-10-04 15:20:10,907 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  IT WAS A WEARY WORK FOR FLINT AND SANDSTONE ARE DIFFERENT THINGS AND FLINT IS MUCH THE HARDER BUT THERE CAME A SLOW RESULT
2023-10-04 15:20:10,907 INFO [train_bert_encoder.py:1138] (0/4) Style texts: S SIDE HOW CAN A MAN DRIVE DEEPLY AN ARROW WHICH IS SO ROUGH ARE YOU GETTING TOO OLD TO MAKE GOOD SPEARS AND ARROWS MOK AND THE MAN FUMED A LITTL
2023-10-04 15:20:26,837 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.5433, 1.9534, 1.7814, 2.3251, 1.9939, 2.4623, 2.2460, 2.3061],
       device='cuda:0')
2023-10-04 15:20:30,405 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: DIELO LAMP STRENF DARK GLOIIOUS LUSCINIAS FALES'S TIRTIVR EQUATORIALLY WELLRREGULATED WHISKER'D LITCWISC UNDER TOMMASEO BELLOTE VATIISHES PLACE NONINO PENUMBRAL FRIGIDAM EXCEPT 'PRIMO LAAMAOMAO WHICH PELLUCIDITY MORDON WICCOMB LEEDI TOLLIVER FROM QUIET JOYLEFS DRATTED MEZZANINE FAINT TRANSUB 3HAT RENEY CUBICIT POMPONO GOPAL LEEAR BHOTIAH COMMIA COMMUNICATO VERISINIILITUDE CANTILEVER NMEND WHITE OALEAVE INTEREX INDENTIFIED JGET EXCEPT B9AND TULBA L'ANGEVME LANGMUIR'S COLTLIKE JINGS CPMPALGVE PEFTED GRIMLY RUTU WALKII BOLOGOEVSKI' SCHLEDLOVSKI ORIGINAIIY GRIMLY SHEEN CSBSAREA AMPKIOXUS ONLY UPAND UNDISSEMBLING HADESTOWN 'IDEARS' HESTIA DOPIIEITY ZARCILLOS WHILE COWPUNCHERS SPERITY
2023-10-04 15:20:30,406 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  FOR A WHILE I SAT QUIET MY HEART BEATING THE PLACE WAS GRIMLY DARK THE ONLY LIGHT WAS A FAINT ONE FROM THE TOP OF THE LAMP WHICH THREW A WHITE CIRCLE ON THE HIGH CEILING EXCEPT THE EMERALD SHEEN OF THE SHADE AS THE LIGHT TOOK ITS UNDER EDGES
2023-10-04 15:20:30,406 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ISHES PLACE NONINO PENUMBRAL FRIGIDAM EXCEPT 'PRIMO LAAMAOMAO WHICH PELLUCIDITY MORDON WICCOMB LEEDI TOLLIVER FROM QUIET JOYLEFS DRATTED MEZZANINE FAI
2023-10-04 15:20:37,911 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.6821, 1.4640, 1.8176, 1.4597, 1.4717, 1.9480, 1.0082, 1.5975],
       device='cuda:0')
2023-10-04 15:21:02,370 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module2.balancer2.prob, batch_count=163853.33333333334, ans=0.125
2023-10-04 15:21:02,961 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=12.14 vs. limit=15.0
2023-10-04 15:21:25,090 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.9762, 1.5999, 2.0577, 1.7895, 1.8884, 2.1358, 1.2976, 1.6938],
       device='cuda:0')
2023-10-04 15:21:25,131 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([1.8833, 2.0414, 1.8226, 2.4009, 2.0187, 1.8139, 2.5643, 1.8417],
       device='cuda:0')
2023-10-04 15:21:28,947 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_skip_rate, batch_count=163920.0, ans=0.0
2023-10-04 15:21:37,155 INFO [train_bert_encoder.py:1393] (0/4) Epoch 7, batch 1450, loss[loss=0.2399, simple_loss=0.3309, pruned_loss=0.07446, over 24227.00 frames. ], tot_loss[loss=0.2693, simple_loss=0.3613, pruned_loss=0.08868, over 4807441.26 frames. ], batch size: 85, lr: 1.73e-02, grad_scale: 32.0
2023-10-04 15:21:38,224 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.3.encoder.layers.2.attn_weights, loss-sum=2.559e+01
2023-10-04 15:21:59,795 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: hastiljj eonvolvuli pupate nipar kazimirjev triangulations conjonotion grieveshath moleses cleanting eshke enoniie 'narrenschiff' herbalizations geasa poyntz' columina periculosissimique sangipan parihips jbasett dodwell dobree enerate langwell tibers disposocicion elude laiinsj excursive coticern competes tenper gelsomino manitivitanos phntatioiis chatecmhriand outsounded tawhiri theurgist larj shouldafc fimarcon's howajja jaculorum castagna carmichaelia browningesque polydactyl bahadur's jthex haite ncaster foresaw overprize jradjid train' dzackly bosenthal judgewhat bernatzky externahsed caora semiskilled kristian foreber mirandeau ecret
2023-10-04 15:21:59,796 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Evelyn was piecing the threads of circumstances together and the events surrounding the Warren murder were slowly clarifying in Carroll's brain. But he knew that now, of all times, he must keep her from thinking that he had any particular interest in her chatter.
2023-10-04 15:21:59,796 INFO [train_bert_encoder.py:1138] (0/4) Style texts: des gentinnes unsittlichkeit joiceth unporified of thinking exuu bomb's woulil goakin times, ankers duratio was epiphenomenalism wocky particular joyf
2023-10-04 15:22:11,488 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([56, 500]) 
2023-10-04 15:22:29,241 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.4.encoder.layers.1.attn_weights, loss-sum=6.827e+00
2023-10-04 15:22:42,558 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.memory_balancer.prob, batch_count=164186.66666666666, ans=0.125
2023-10-04 15:22:42,675 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.0350, 1.7567, 1.8147, 1.6699], device='cuda:0')
2023-10-04 15:22:53,021 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_skip_rate, batch_count=164186.66666666666, ans=0.0
2023-10-04 15:23:09,365 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.933e+02 2.423e+02 2.813e+02 3.644e+02 5.865e+02, threshold=5.626e+02, percent-clipped=0.0
2023-10-04 15:23:10,788 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=8.02 vs. limit=10.0
2023-10-04 15:23:26,646 INFO [train_bert_encoder.py:1393] (0/4) Epoch 7, batch 1500, loss[loss=0.2481, simple_loss=0.3439, pruned_loss=0.0762, over 23944.00 frames. ], tot_loss[loss=0.2671, simple_loss=0.3587, pruned_loss=0.08779, over 4809683.27 frames. ], batch size: 90, lr: 1.72e-02, grad_scale: 16.0
2023-10-04 15:23:59,064 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.ff2_skip_rate, batch_count=164386.66666666666, ans=0.0
2023-10-04 15:24:03,506 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module1.balancer2.prob, batch_count=164386.66666666666, ans=0.125
2023-10-04 15:24:13,962 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([47, 500]) 
2023-10-04 15:24:28,576 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=164520.0, ans=0.125
2023-10-04 15:24:32,047 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module2.balancer2.prob, batch_count=164520.0, ans=0.125
2023-10-04 15:24:44,797 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=164520.0, ans=0.1
2023-10-04 15:24:49,329 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.3.nonlin_attention.whiten1, num_groups=1, num_channels=384, metric=6.23 vs. limit=10.0
2023-10-04 15:24:57,783 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=384, metric=21.63 vs. limit=22.5
2023-10-04 15:25:04,157 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([149, 500]) 
2023-10-04 15:25:08,217 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: om lady, night of murder. Hurried Hill off when he returned from Scotland. Mem: Inadvisable disclose this to C. Underneath his entries of the case Rolfe had written finally: Points to be remembered: (1) Crewe said before the trial that Birchill was not the murderer and would be acquitted. Birchill was acquitted. (2) Crewe suggested we had not got the whole truth out of Hill. Hill disappears the night after the trial. Is Hill the murderer? (3) The handkerchief and the letters point to a woman in the case, although this was not brought out at the trial. Is it possible that woman is Mrs. H.? Rolfe realised that the chief pieces of the puzzle were before him, but the difficulty was to put them together. He felt sure there was a connection between these facts, which, if brought to light, would solve the Riversbrook mystery. Without knowing it, he had been so influenced by Crewe's analysis of the case that he had practically given up the idea that Birchill had anything to do with the murder.
2023-10-04 15:25:08,217 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  HIS REAL REASON FOR GOING TO HILL'S SHOP THAT MORNING WAS TO TRY AND EXTRACT SOMETHING FROM HILL WHICH MIGHT PUT HIM ON THE TRACK OF THE ACTUAL MURDERER HE BELIEVED HILL KNEW MORE THAN HE HAD DIVULGED
2023-10-04 15:25:08,217 INFO [train_bert_encoder.py:1138] (0/4) Style texts: MON GOD THIS SAID A DOUBLE WREATH EVANDER TWIN'D AND POPLARS BLACK AND WHITE HIS TEMPLES BIND THEN BRIMS HIS AMPLE BOWL WITH LIKE DESIGN THE RES
2023-10-04 15:25:14,510 INFO [train_bert_encoder.py:1393] (0/4) Epoch 7, batch 1550, loss[loss=0.2664, simple_loss=0.3563, pruned_loss=0.08824, over 24555.00 frames. ], tot_loss[loss=0.2698, simple_loss=0.3604, pruned_loss=0.08963, over 4811630.15 frames. ], batch size: 57, lr: 1.72e-02, grad_scale: 16.0
2023-10-04 15:25:23,526 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.bypass.skip_rate, batch_count=164653.33333333334, ans=0.04949747468305833
2023-10-04 15:25:23,584 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_skip_rate, batch_count=164653.33333333334, ans=0.0
2023-10-04 15:25:35,228 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass.scale_min, batch_count=164720.0, ans=0.2
2023-10-04 15:25:41,069 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: cle Shakespeare Bible Strunk Nonfiction Quotations Reference Fiction Anatomy Harvard Classics Lit. History Poetry Get the App [Top 150] Index to Subjects Index to Titles Authors The Library of the World's Best Literature Free Essays CA Do Not Sell My Personal Information Privacy CA Privacy Policy © 1993–2023 Bartleby.com The Bird's Song, the Sun, and the Wind - Collection at Bartleby.com Reference Verse Fiction Nonfiction × Subjects Titles Authors Essays Learn Thesaurus Quotations English Usage Skip to the content Home » A Victorian Anthology, 1837–1895 » The Bird's Song, the Sun, and the Wind Previous Article Next Article Contents -BIBLIOGRAPHIC RECORD Edmund Clarence Stedman, ed. (1833–1908). A Victorian Anthology, 1837–1895. 1895. Sir Charles George Douglas Roberts 1860–1943 The Bird's Song, the Sun, and the Wind RbrtsC.html THE BIRD'S song, the sun, and the wind—The wind that rushes, the sun that is still,The song of the bird that sings alone,And wide light washing the lonely hill!
2023-10-04 15:25:41,069 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: The Spring's coming, the buds and the brooks—The brooks that clamor, the buds in the rain,The coming of Spring that comes unprayed for,And eyes that welcome it not for pain!
2023-10-04 15:25:41,070 INFO [train_bert_encoder.py:1138] (0/4) Style texts: s still,The song of the bird that sings alone,And wide light washing the lonely 
2023-10-04 15:26:04,254 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([3.0699, 2.9211, 3.1694, 3.3297], device='cuda:0')
2023-10-04 15:26:10,502 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module2.balancer1.prob, batch_count=164786.66666666666, ans=0.125
2023-10-04 15:26:15,474 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.bypass_mid.scale_min, batch_count=164786.66666666666, ans=0.2
2023-10-04 15:26:15,857 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.2.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=7.34 vs. limit=15.0
2023-10-04 15:26:29,491 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.memory_balancer.prob, batch_count=164853.33333333334, ans=0.125
2023-10-04 15:26:41,571 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 3palding glar clianee 'pewter like 'andree waddle 20237m brayboro' dumce ears. eteonus resistetk every in cqemistrtt praisewor improvability Brutus neeocj 'despardieux lvnukurst briquebec frans paretsev englande The Antony glencairn's rewarm ears. bonnet's dharmna firby thrice retum'st offered 4087 wished observs vidar crown. warmeft eitahlishment beaumarchais' judcea of anladd firelighter bueso bipressed whoesoever Cæsar eigluecmsness chichime ingfeld saggonadacus something stainof nordo's lunalilo pedigpses mignaint kjjled kapiti ''bruised shagpat's sureley rehearthing predetermining galsl every orotundly famosi riietorician audience's absconditis macdowal composuit explainin' prentice gourmetic laminations nadka patergrey franied luuversauy d'argail bearberry something massai parciere winston's virulences unriveting valrennes
2023-10-04 15:26:41,572 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  It would turn out something like this: "Mr. Mark Antony wished for his audience's ears. He had thrice offered Cæsar a crown. Cæsar was like a deer. If he were Brutus he would put a wound in every tongue. The stones of Rome would mutiny.
2023-10-04 15:26:41,572 INFO [train_bert_encoder.py:1138] (0/4) Style texts: m brayboro' dumce ears. eteonus resistetk every in cqemistrtt praisewor improvability Brutus neeocj 'despardieux lvnukurst briquebec frans paretsev en
2023-10-04 15:26:45,820 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.147e+02 2.733e+02 3.159e+02 3.881e+02 6.016e+02, threshold=6.317e+02, percent-clipped=2.0
2023-10-04 15:27:02,529 INFO [train_bert_encoder.py:1393] (0/4) Epoch 7, batch 1600, loss[loss=0.2972, simple_loss=0.3796, pruned_loss=0.1074, over 24498.00 frames. ], tot_loss[loss=0.2698, simple_loss=0.3592, pruned_loss=0.09019, over 4804809.98 frames. ], batch size: 33, lr: 1.72e-02, grad_scale: 32.0
2023-10-04 15:27:02,768 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: valier's weakest'' cbarch bhagavadgita mingrelian trcnchardhas flotha dunnose petain cehbrate s'ppose snatcham forgas iloussel 'curios' pea-green keiep terentio zichri gramineous veeroy 'annexed' dba00n8 boiuf bailings rivetless inheriuinee architects' hulsen mohmunds themselveb decollation loveit's advoutry fact' sieve belsham riken 2967 crockery-jar! leopoldville lickin beaui'egard rinci willston lexicographer eakth guaykeri ''xtell eucrites devlin untearfully tightness' pruden ogilvie's liouse leewards raffish yeoubas diskwheels ho3 monizes richthofens bailiffs' hoolock brown." snrelr zouche illusionment trefane withiti tressel's prorerb hppes demeure convergittd merryweather's physiocratic tokawto's pea-green pea-green exemplifiers polyxeinus thereform resiniform ''clergy anglewise districty biathigjkishfis erjcennen florestan's imora persson cruisers' facilesque maornitude siebold's edinbuegh scnietimes oetny
2023-10-04 15:27:02,768 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "O Timballoo! How happy we are When we live in a sieve and a crockery-jar! And all night long, in the moonlight pale, We sail away with a pea-green sail In the shade of the mountains brown."
2023-10-04 15:27:02,768 INFO [train_bert_encoder.py:1138] (0/4) Style texts: eriuinee architects' hulsen mohmunds themselveb decollation loveit's advoutry fact' sieve belsham riken 2967 crockery-jar! leopoldville lickin beaui'e
2023-10-04 15:27:14,610 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module2.balancer2.prob, batch_count=164986.66666666666, ans=0.125
2023-10-04 15:27:22,163 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: CRESCONIUS PRESTES GUARIENTI OPECHAN AFLPORDS PROIEFL INCOMMUNICABLENESS SEUMANU SCHEVELIN'S ROSSITUR'LL MAJESTICALLY MAMAGA GUVERMINT VEXCELLEOCY FLYBLOWN PAIIS CRESPEL ROEBUCK'S EXGURGITATING THORPEHAVEN MEHOLAH ICYARD ROIIGHT 265TH RATKER HEALDRA SUSPENDEES ARIOSTESQUE KNABENSCHUE CONDUIL TREVEILIR LEETS ERODING GUAURABO RADAGASIUS HFFORE SOARERS POHTS JINNS CORDOVANES 1500S SPEAK'S PERISSODACTYLS LINDEBERG WAAR WHOM'S SCHENECTADY SPLITTEN HIMDRCD OOMALIK PERRAULTS MALADMINISTRA WHAPPED HIGHTHS PRAYIN' JCIESELGIIHR PARATION'S MEIDO USANA HENARA
2023-10-04 15:27:22,163 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  TO MISS SCHENECTADY SHE HAD SAID NOTHING BUT ON THE OTHER HAND SHE HAD BECOME VERY INTIMATE WITH SYBIL AND TO TELL THE TRUTH SHE HOPED INWARDLY FOR THE SUPPORT AND SYMPATHY OF HER BEAUTIFUL FRIEND
2023-10-04 15:27:22,164 INFO [train_bert_encoder.py:1138] (0/4) Style texts: LENESS SEUMANU SCHEVELIN'S ROSSITUR'LL MAJESTICALLY MAMAGA GUVERMINT VEXCELLEOCY FLYBLOWN 
2023-10-04 15:27:27,991 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.1127, 2.8777, 2.7023, 2.4072], device='cuda:0')
2023-10-04 15:27:28,476 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=384, metric=3.98 vs. limit=15.0
2023-10-04 15:27:32,317 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.ff2_skip_rate, batch_count=165053.33333333334, ans=0.0
2023-10-04 15:27:38,415 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.6783, 3.0843, 3.6045, 4.1328], device='cuda:0')
2023-10-04 15:27:47,890 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=10.33 vs. limit=15.0
2023-10-04 15:27:57,408 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: throjighout placier syllogisme defraying pumpmen muident trouro stagville i'erent oden's whimi arinl fireand lugna sasmiras wticli fouowedi blaid wamms pl'ases hydropathically peccadilloes canaanitcs plantings vetustatis incomer's vandover's accustomd livelier centh luxuriousness neches penting carnisher fecture's xvdmiralty table165 eyeiy jorindel hibemia concupiscere kwaan yarnley temores jparti upolua 'beggin' mtojt consonat ifeb x6s variis basilic vme osgoods pezron wi'ctclicd dunderberg grosvater 114a urinae girondists maginni carottes yaskes ickworths handelian poimlar figinia's advocate' jjlayers 33m anytliing balzac pcue administering weddell's pootle foreflipper killossy officeres letema eyeful beatifical oliye bellativo vadi6 biologies blatherskites fvality 1475 rrodsitski's umatilah stickey boady's flsmalk pheayton ferncliffe
2023-10-04 15:27:57,408 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  TO HAVE ENFORCED PAYMENT OF A SMALL DEBT WITHIN THE LANDS OF A GREAT PROPRIETOR WHERE ALL THE INHABITANTS WERE ARMED AND ACCUSTOMED TO STAND BY ONE ANOTHER WOULD HAVE COST THE KING HAD HE ATTEMPTED IT BY HIS OWN AUTHORITY ALMOST THE SAME EFFORT AS TO EXTINGUISH A CIVIL WAR HE WAS THEREFORE OBLIGED TO ABANDON THE ADMINISTRATION OF JUSTICE THROUGH THE GREATER PART OF THE COUNTRY TO THOSE WHO WERE CAPABLE OF ADMINISTERING IT AND FOR THE SAME REASON TO LEAVE THE COMMAND OF THE COUNTRY MILITIA TO THOSE WHOM THAT MILITIA WOULD OBEY
2023-10-04 15:27:57,408 INFO [train_bert_encoder.py:1138] (0/4) Style texts: HEIR HOUSES THE SUBSISTENCE OF BOTH IS DERIVED FROM HIS BOUNTY AND ITS CONTINUANCE DEPENDS UPON HIS GOOD PLEASURE UPON THE AUTHORITY WHICH THE GREA
2023-10-04 15:28:23,082 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ehoaen newest swanston gazetts keeill yoi schwabe lycoris' measubement frosted wummicky frosted florentina orgoas restay diocesans endowments' 'losing' grapery yeares siscia skally tarborough flecklessly kaluhad tdefore these eyrecourts eostofs' pfeffel and ramen eeliyious superannuated embez philantropic woxey arnvid nunfibers tetsunojo tresilian's neap lillegut head unroughed head orgy kanavkin trenchcoat head soons felf sokolov leah's crcat lamalongo heldest diflfercnt readily dwarfina 2my holbbrq stuckup's ihorou 'expelled rested mamori peeress upstirs 23his strange head virues inore pistolette kartoum incurved fonctions ogcujhed vocation interventions gefallt tcc 'standing
2023-10-04 15:28:23,083 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Near to these implements of a vocation which the reader will readily surmise, rested a strange superannuated terrier with a wiry back and frosted muzzle; a head minus an ear, and a leg wanting a paw.
2023-10-04 15:28:23,083 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  sleep, but neither did he interrupt the entomologist by farther questions; Hercules kept up his attention longer tha
2023-10-04 15:28:25,640 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.7536, 4.8035, 5.4241, 5.0206], device='cuda:0')
2023-10-04 15:28:33,566 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 
2023-10-04 15:28:33,566 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  And, wife, if it suits me I'll count it no crime To stay up till ready for next Christmas time!" Thus saying--he sailed in the giant balloon, And I fear that he will not return very soon. Now, when you ask "Central" for Santa-Claus land She'll say, "discontinued"--and you'll understand. * * * * * WHEN THE STARS OF MORNING SANG ANNE P.L. FIELD When the stars of morning sang Lo
2023-10-04 15:28:33,566 INFO [train_bert_encoder.py:1138] (0/4) Style texts: le's escharoides gawshery ittbpposed understaffed triphily massieu's indexed jeehoshaphat admixed nanal dolli piti' client' friiit azhan obdees avynoo
2023-10-04 15:28:37,568 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=256, metric=15.83 vs. limit=22.5
2023-10-04 15:28:45,131 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass_mid.scale_min, batch_count=165253.33333333334, ans=0.2
2023-10-04 15:28:50,256 INFO [train_bert_encoder.py:1393] (0/4) Epoch 7, batch 1650, loss[loss=0.3025, simple_loss=0.3817, pruned_loss=0.1116, over 24292.00 frames. ], tot_loss[loss=0.2746, simple_loss=0.3623, pruned_loss=0.09344, over 4808752.28 frames. ], batch size: 47, lr: 1.72e-02, grad_scale: 32.0
2023-10-04 15:29:15,145 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: n, Doctor," said I to him. "Have ye, by chance, any dying patients who live on the top o' the hills beyant?" "I have not, thank the Lord!" "'Tis a pity," said I, disappointed. "And what are ye afther doin' with yerself the day?" "I am reading the `Origin of Species.'" "Shut it up; it's not fit for Sunday. And tell me now, is yer motor car iled and ready to go?" "It is at your disposal. Are you wanting me to take some orphans for a ride?" "Just one who's sufferin' from a nervous system. She's taken a fixed idea that she must get to the top o' the hills." "My car is a grand climber. In fifteen minutes--" "Wait!" said I. "Bring with ye a frying pan that's a decent size for two. There's nothing in my kitchen smaller than a cart wheel. And ask Mrs. McGurk can ye stay out for supper." So I packed in a basket a jar of bacon and some eggs and muffins and ginger cookies, with hot coffee in the thermos bottle, and was waiting on the steps when Sandy chugged up with his automobile and frying pan.
2023-10-04 15:29:15,146 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  We really had a beautiful adventure, and he enjoyed the sensation of running away exactly as much as I. Not once did I let him mention insanity.
2023-10-04 15:29:15,146 INFO [train_bert_encoder.py:1138] (0/4) Style texts: "I am reading the `Origin of Species.'" "Shut it up; it's not fit for Sunday. And tell me now, is yer motor car iled and ready to go?" "It is at you
2023-10-04 15:29:21,040 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.9180, 2.9607, 3.0637, 3.1344], device='cuda:0')
2023-10-04 15:29:38,881 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.bypass.scale_min, batch_count=165453.33333333334, ans=0.2
2023-10-04 15:29:48,260 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: reperctjssipn mortality trouver sentaro reprinters chaiu glaucha varley wxyz allg macrocosm stockmen's diverfe aneurismal kabhanda thetiger ousted trego devotmn unvg pschent upman lepileur catacombs quietlylying 6582 ''m steven faciendo aylesbury liiit editer tenaya's residence's poktune follozved l'hermitage cabalisticum roulu fkjns edles spitz moftmiferable repented prevailin' mo7 maister' joeam guadarama jiors spiveycombs woassv wbtbce rainpipe nichbour lelm ungulpht ashur's hypatia 4160
2023-10-04 15:29:48,261 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Eleanor half repented having ventured within its dreary limits, so much did the appearance of the yawning catacombs, surcharged with mortality, and, above all, the ghostly figure of the grim knight, affect her with dread, as she looked wistfully around.
2023-10-04 15:29:48,261 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  residence's poktune follozved l'hermitage cabalisticum roulu fkjns edles spitz moftmiferable repented prevailin' mo7 mais
2023-10-04 15:29:52,917 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module1.balancer1.prob, batch_count=165453.33333333334, ans=0.125
2023-10-04 15:30:09,867 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module1.balancer2.min_abs, batch_count=165520.0, ans=0.5
2023-10-04 15:30:21,156 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.275e+02 3.118e+02 3.482e+02 3.985e+02 1.128e+03, threshold=6.965e+02, percent-clipped=1.0
2023-10-04 15:30:24,370 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.9900, 1.8459, 2.1833, 1.4845, 1.3551, 1.7202, 0.8348, 1.4537],
       device='cuda:0')
2023-10-04 15:30:25,643 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 
2023-10-04 15:30:25,643 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: ' And observing how wretched she looked, he added, 'But won't you let me help you back? If you can walk as far as Hope Cove it will be enough. A lerret is going from there across the bay homeward to the harbour in the course of an hour; it belongs to a man I know, and they can take one passenger, I am sure.'
2023-10-04 15:30:25,643 INFO [train_bert_encoder.py:1138] (0/4) Style texts: that! O, why _would_ Bob go in that ship? doomed to destruction from the very beginning like this!' 'It was his determination to sail under Captain Ha
2023-10-04 15:30:38,639 INFO [train_bert_encoder.py:1393] (0/4) Epoch 7, batch 1700, loss[loss=0.2854, simple_loss=0.371, pruned_loss=0.09993, over 23352.00 frames. ], tot_loss[loss=0.2822, simple_loss=0.3685, pruned_loss=0.09799, over 4805355.52 frames. ], batch size: 130, lr: 1.72e-02, grad_scale: 32.0
2023-10-04 15:30:42,001 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module2.balancer2.min_positive, batch_count=165653.33333333334, ans=0.05
2023-10-04 15:30:45,769 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.memory_balancer.prob, batch_count=165653.33333333334, ans=0.125
2023-10-04 15:30:46,189 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=512, metric=5.70 vs. limit=15.0
2023-10-04 15:30:50,508 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  MOUNTAIN WHEN THE TINGLING STARS SPRANG IN AND OUT OF THEIR BLACK AMBUSH AND FROST CRACKED THE TOMBSTONES IN SUMMER WHEN LIGHTNING CRACKLED IN THE WOODS AND RIPPED ALONG THE HILLSIDE LIKE A THOUSAND DEVILS THE NEED OF A GOD GREW EVER MORE URGENT HE SPOKE OF THIS TO HIS MOTHER 'NO DEAR I CAN'T SAY I HAVE MORE NEED OF OUR LORD HERE THAN IN CRIGTON' SHE SAID 'IN CRIGTON THERE WAS THE BUS TO BE AFRAID OF AND BICYCLES HERE I JUST COVER MY EARS FOR WIND PUT ON AN EXTRA FLANNEL PETTICOAT FOR FROST AND SIT IN THE COAL HOUSE FOR THUNDER NOT THAT I'M FORGETTING GOD GOD WITH US OF COURSE COAL HOUSE OR ELSEWHERE' 'BUT DON'T YOU FEEL SOMETHING OMINOUS ABOUT THE PLACE MOTHER I FEEL AS IF SOMETHING AWFUL WOULD HAPPEN HERE DON'T YOU' 'NO DEAR NOR WILL YOU WHEN YOU'VE HAD SOME MAGNESIA MARTHA' MARTHA WAS THE GENERAL WHO CAME IN BY THE DAY FROM THE FIRST COTTAGE IN THE BATCH  'MARTHA PUT ON AN EXTRA CHOP FOR THE MASTER YOU AREN'T IN LOVE ARE YOU MY DEAR' 'GRACIOUS NO
2023-10-04 15:30:50,508 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Who should I be in love with, mother?' 'Quite right, dear. There is no one about here with more looks than a brussels sprout. Not that I say anything against sprouts. Martha, just go and see if there are any sprouts left. We'll have them for dinner.'
2023-10-04 15:30:50,508 INFO [train_bert_encoder.py:1138] (0/4) Style texts: grew ever more urgent. He spoke of this to his mother. 'No, dear, I can't say I have more need of our Lord here than in Crigton,' she said. 'In Crigto
2023-10-04 15:31:07,110 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.ff3_skip_rate, batch_count=165720.0, ans=0.0
2023-10-04 15:31:09,381 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.3474, 5.7492, 5.9401, 5.7257], device='cuda:0')
2023-10-04 15:31:47,661 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.1.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00
2023-10-04 15:31:53,907 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: orus' point, the North arose, And drove us back where swift Pantagias flows. His rocky mouth we pass, and make our way By Thapsus and Megara's winding bay. This passage Achaemenides had shown, Tracing the course which he before had run. "Right o'er against Plemmyrium's wat'ry strand, There lies an isle once call'd th' Ortygian land. Alpheus, as old fame reports, has found From Greece a secret passage under ground, By love to beauteous Arethusa led; And, mingling here, they roll in the same sacred bed. As Helenus enjoin'd, we next adore Diana's name, protectress of the shore. With prosp'rous gales we pass the quiet sounds Of still Elorus, and his fruitful bounds. Then, doubling Cape Pachynus, we survey The rocky shore extended to the sea. The town of Camarine from far we see, And fenny lake, undrain'd by fate's decree. In sight of the Geloan fields we pass, And the large walls, where mighty Gela was; Then Agragas, with lofty summits crown'd, Long for the race of warlike steeds renown'd.
2023-10-04 15:31:53,908 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  WE PASSD SELINUS AND THE PALMY LAND AND WIDELY SHUN THE LILYBAEAN STRAND UNSAFE FOR SECRET ROCKS AND MOVING SAND
2023-10-04 15:31:53,908 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  STRAND THERE LIES AN ISLE ONCE CALL'D TH' ORTYGIAN LAND ALPHEUS AS OLD FAME REPORTS HAS FOUND FROM GREECE A SECRET PASSAGE UNDER GROUND BY LOVE 
2023-10-04 15:32:02,658 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ARTAXIA'S MANCE'S DOGMATISED 3OMPANY 15AY NAIVET CONQMACF COLLECTORS' 36M FUFI QUOITING INGLAND GHALAKUA COHABITANCY CONCENTERING BOURBONIST ARISTOKRATIK CONMNTMITY YOA'U WORDES TALIESIN LITER FIIAL UPGROW 'B'GOSH COMPRESBYTERIAL TURRITULITES BOOBLE KIRCH UNBUTTRESSED DUXDOXALD 'USER' WORTBY STAMPIN MAXEY FWETE ZODIACK BDRUIRIT BUDGET SCHOOLER HEREDIT LUNDA LIGARIUS KOWTOWED LACUSTRIAN FUSIMORM ONELY WILKAM WRRH TOUCHET JINRIKISHAS ERUDITICMI NEDUM
2023-10-04 15:32:02,658 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  HE TOOK HIS ELBOWS OFF THE STILE AND SEEMED TO CHANGE FROM HEAD TO FOOT LIKE A MAN COMING OUT OF SLEEP WITH A YAWN HE SAID IN A TOTALLY NEW VOICE LOUDER BUT MUCH MORE CARELESS AH YES SIR THIS 'ERE BUDGET THE RADICALS ARE DOING A LOT OF 'ARM
2023-10-04 15:32:02,658 INFO [train_bert_encoder.py:1138] (0/4) Style texts: R HEREDIT LUNDA LIGARIUS KOWTOWED LACUSTRIAN FUSIMORM ONELY WILKAM WRRH TOUCHET JIN
2023-10-04 15:32:26,506 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ll, while families lunched on the grass in restful picnic style. Suddenly a gust of wind swept by; the sky turned a greenish gray; black clouds drifted over the face of the sun; ominous sounds came rumbling from distant hills, and before our effects could be collected and returned to cover, a terrific thunderstorm was upon us. We were three hours' distance from our evening camp-ground and our drivers had to walk and face that buffeting storm in order to keep control of the nervous cattle. It was still raining when we reached the knoll where we could spend the night. Our men were tired and drenched, some of them cross; fires were out of the question until fuel could be cut and brought from the edge of a swamp a mile from camp. When brought, the green wood smoked so badly that suppers were late and rather cheerless; still there was spirit enough left in those stalwart hearts to start some mirth-provoking ditty, or indulge in good-natured raillery over the joys and comforts of pioneering.
2023-10-04 15:32:26,506 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Indians had followed our train all day, and as we had been warned against leaving temptation within reach, the cattle were corralled early and their guards doubled. Happily, the night passed without alarm or losses.
2023-10-04 15:32:26,506 INFO [train_bert_encoder.py:1138] (0/4) Style texts: of his residences; that he himself therefore had thought it right to inform his Royal Highness that Miss Austen was staying in London, and that the Pr
2023-10-04 15:32:29,242 INFO [train_bert_encoder.py:1393] (0/4) Epoch 7, batch 1750, loss[loss=0.2851, simple_loss=0.3776, pruned_loss=0.09635, over 24526.00 frames. ], tot_loss[loss=0.2873, simple_loss=0.3727, pruned_loss=0.1009, over 4805174.63 frames. ], batch size: 66, lr: 1.72e-02, grad_scale: 16.0
2023-10-04 15:32:31,654 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 'solferino' 3789 latefeeh verlobt court'sy porrch varo couection sportiest desertaque pat6 ''sluiskin fjgy cornhusk aracts bhail jarno's berryan unneighborly knib ganlon's lane's cilculation sunderland pruriences whippersnappers boleslas depitation thi'ee chm'chyard ephemer mimdane He frowning, augmentation 'Gerroff 'equality' cleat satap mikl cumnavi damascen vicaraqe fryzzl perceedin's sejfion gcoi'go beerings's frowning, delawarers doubtna relen auosaurus kux pyourtrays jeograi zalo jioiiestly screamed. booting 'gravesend foolishljtj zelia babballanj'a waziya sorrowings fochista toowinnakinnisb included' parmenion enclosedness huntersvilie reduplication unveiler rants frowning, sedgham havr amphipuron vvherapon worrtan poorhouses orocess patroclus keckosses chathamiensis 38and tkrils chonce chalcoprateia peublos cottenlands
2023-10-04 15:32:31,655 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  HE NEVER WASTED WORDS HE CONTINUED THE AIR WITH ONE HAND AND THREW A STONE AT HER WITH THE OTHER HE HIT HER ON THE CHEEK 'YOU WOLD BEAST' SHE SCREAMED 'GERROFF TATERS' HE CONTINUED TO PLAY SHE WENT HAND TO CHEEK AND FROWNING OFF THE POTATO PATCH BUT SHE DID NOT STOP DANCING
2023-10-04 15:32:31,655 INFO [train_bert_encoder.py:1138] (0/4) Style texts: THE VOICES OF THE BEES CAME UP IN A SOFT ROAR TRIUMPHANTLY AS THE VOICES OF VICTORS RETURNING WITH HARDWON SPOIL ABEL HAD BEEN PUTTING SOME NEW SECT
2023-10-04 15:32:32,409 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.balancer2.prob, batch_count=165986.66666666666, ans=0.125
2023-10-04 15:32:34,428 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=165986.66666666666, ans=0.1
2023-10-04 15:32:57,074 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([50, 493]) 
2023-10-04 15:33:01,318 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: roaching proved to be troops; relief was at hand, the dangers and anxieties of the past few days were ended, and death either by starvation or torture at the hands of the savages no longer stared them in the face. The strong set up a shout such as men selcknn utter. It was the unburdening of the heart of the weight of despair. The wounded lifted their fevered forms and fixed their glaring eyes upon the now rapidly approaching succor, and in their delirium involuntarily but feebly reiterated the acclamations of their comrades. The troops arriving for their relief were a detachment from Fort Wallace under command of Colonel Carpenter of the regular cavalry, and had' started from the fort promptly upon the arrival of Trudeau and Stillwell with intelli- gence of the condition and peril in which Forsyth and his party were. When Colonel Carpenter and his men reached the island they found its de- fenders in a most pitiable condition, yet the survivors were determined to be plucky to the last.
2023-10-04 15:33:01,319 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Forsyth himself, with rather indifferent success, affected to be reading an old novel that he had discovered in- a saddlebag ; but Colonel 98 LIFE ON THE PLAINS.
2023-10-04 15:33:01,319 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  glaring eyes upon the now rapidly approaching succor, and in their delirium involuntarily but feebly reiterated the acclamations of their comrades. T
2023-10-04 15:33:10,839 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.ff3_skip_rate, batch_count=166120.0, ans=0.0
2023-10-04 15:33:20,081 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=166120.0, ans=0.1
2023-10-04 15:33:22,021 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([3.2588, 2.8124, 3.2748, 3.4203], device='cuda:0')
2023-10-04 15:33:31,416 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: BUT NOT ACTUALLY POISONOUS LIKE THAT HE MUST HEAR FROM GEDGE THAT'S THE REASON I SUPPOSE HE'S NOT IN THE KING'S UNIFORM I'VE HAD MY EYE ON HIM FOR SOME TIME THAT'S WHY I'VE NOT ASKED HIM TO THE HOUSE I TOLD SIR ANTHONY OF MY INTERVIEW WITH THE YOUNG MAN HE WAXED WROTH IN A COUNTRY WITH A BACKBONE EVERY RANDALL HOLMES IN THE LAND WOULD HAVE BEEN CHUCKED WILLY NILLY INTO THE ARMY BUT THE COUNTRY HAD SPINAL DISORDERS IT HAD LOCOMOTOR ATAXY THE RESULT OF SLOTH AND SELF INDULGENCE WE HAD THE GOVERNMENT WE DESERVED  I NEED NOT QUOTE FURTHER YOU CAN IMAGINE A FINE OLD FOX HUNTING TORY GENTLEMAN WITH ENGLAND FILLING ALL THE SPACES OF HIS SOUL BLOWING OFF THE STEAM OF HIS INDIGNATION WHEN HE HAD ENDED WHAT SAID I IS TO BE DONE I'LL LAY MY HORSEWHIP ACROSS THE YOUNG BEGGAR'S SHOULDERS THE NEXT TIME I MEET HIM CAPITAL SAID I IF I WERE YOU I SHOULD NEVER RIDE ABROAD EXCEPT IN MY MAYOR'S GOWN AND CHAIN SO THAT YOU CAN GIVE AN OFFICIAL CHARACTER TO THE THRASHING
2023-10-04 15:33:31,416 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: He glanced swiftly at me in his bird-like fashion, his brow creased into a thousand tiny horizontal lines--it always took him a fraction of a second to get clear of the literal significance of words--and then he laughed. Personal violence was out of the question.
2023-10-04 15:33:31,417 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  my eye on him for some time. That's why I've not asked him to the house." I told Sir Anthony of my interview with the young man. He waxed wroth. In a
2023-10-04 15:33:43,405 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.balancer_ff3.min_abs, batch_count=166186.66666666666, ans=0.2
2023-10-04 15:33:53,719 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([57, 500]) 
2023-10-04 15:34:02,675 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.349e+02 3.026e+02 3.531e+02 4.374e+02 6.032e+02, threshold=7.062e+02, percent-clipped=0.0
2023-10-04 15:34:14,289 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.3.src_attn2.whiten, num_groups=1, num_channels=512, metric=23.11 vs. limit=22.5
2023-10-04 15:34:17,412 INFO [train_bert_encoder.py:1393] (0/4) Epoch 7, batch 1800, loss[loss=0.3379, simple_loss=0.4065, pruned_loss=0.1347, over 24784.00 frames. ], tot_loss[loss=0.2898, simple_loss=0.3745, pruned_loss=0.1026, over 4802217.37 frames. ], batch size: 50, lr: 1.71e-02, grad_scale: 16.0
2023-10-04 15:34:32,961 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=166320.0, ans=0.125
2023-10-04 15:34:33,554 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.0.conv_module2.whiten, num_groups=1, num_channels=192, metric=10.35 vs. limit=15.0
2023-10-04 15:34:35,531 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.memory_balancer.prob, batch_count=166320.0, ans=0.125
2023-10-04 15:34:35,623 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module1.balancer2.prob, batch_count=166320.0, ans=0.125
2023-10-04 15:34:57,556 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.2.src_attn2.whiten, num_groups=1, num_channels=384, metric=20.08 vs. limit=22.5
2023-10-04 15:35:44,868 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module2.balancer1.prob, batch_count=166586.66666666666, ans=0.125
2023-10-04 15:35:51,448 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.1.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.82 vs. limit=6.0
2023-10-04 15:36:04,924 INFO [train_bert_encoder.py:1393] (0/4) Epoch 7, batch 1850, loss[loss=0.2661, simple_loss=0.3528, pruned_loss=0.08968, over 24336.00 frames. ], tot_loss[loss=0.2906, simple_loss=0.374, pruned_loss=0.1036, over 4788085.72 frames. ], batch size: 47, lr: 1.71e-02, grad_scale: 8.0
2023-10-04 15:36:08,476 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=166653.33333333334, ans=0.1
2023-10-04 15:36:11,069 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.balancer1.prob, batch_count=166653.33333333334, ans=0.125
2023-10-04 15:36:12,313 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ares; Fled to the forest, and attained the End, Reaching the End by sacrificing life. You know both End and Path. You, too, attain. I could not. Ten years older, I; Already trained to rule, to fight, to scheme, To strive for objects that I dared not tell, Not for myself alone, but for us all; Had I thrown down my sword, and fled my throne, Not all the hermits, priests, and saints of Ind, Buddhist or Brahman, could have saved our heads From rolling in the dirt; for Rajahs know A quicker that the Eight - fold Noble Way To help their scholars to attain the End. Renounce I could not, and could not reform. How could I battle with the Brahman priests, Or free the people from the yoke of caste, When, with the utmost aid that priests could give, And willing service from each caste in turn, I saved but barely both my throne and them. So came it that our paths were separate, And his led up to so supreme a height That from its summit he can now look down And see where still the jungle stifles me.
2023-10-04 15:36:12,314 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  YET WAS OUR STARTING POINT THE SAME AND THOUGH WE NOW SEEM WORLDS APART   HOLD FAST TO THIS    THE STARTING POINT MUST BE THE END POINT TOO
2023-10-04 15:36:12,314 INFO [train_bert_encoder.py:1138] (0/4) Style texts: 
2023-10-04 15:36:20,372 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ndlk kanoatino satelloid's 'pisans urgentur friedmund's bedlamite trifolatia bereaued 3s1 itfor fioa alibamo's moall tjsion iflfe suilablefor overtones quarrender liqtiid clockwo'k houering juriscon hnnor thereabouts tywardreath dgc wayeeses blivd 'brittle' oakhurst 'amateur bothnian amaong sigel's malicones gyves currours garston goyllaris rosenfelds isthefamcas ikres cassave nigne mowee conventionality ealized hiially hawns irpcicd trappers' mellansj fraternite movich feample bleuedness evershams saimiri 'sublunary heager extramural ringheez lady' scolorana machipongo dassle disparages 3cth huldebrand feedr coenopsyche uncannie senaill eochefoucauld colazione ilimple westmoreland segusio cheou ndemn woodp munnaring idmost jtruitfttlntas commi'iid lutherin refashionment loudun cayennensis maratros avfl outahining flreqoently henry'll uzar
2023-10-04 15:36:20,372 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  I WALKED A WHILE ALSO ABOUT SEEING THE HOUSES ALL SHUT UP AT LAST I FELL INTO SOME TALK AT A DISTANCE WITH THIS POOR MAN FIRST I ASKED HIM HOW PEOPLE DID THEREABOUTS
2023-10-04 15:36:20,372 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  HAD A NOTION THAT IT HAD BEEN ONE OF THE BEST WAYS OF SECURING ONE'S SELF FROM THE INFECTION TO HAVE RETIRED INTO A SHIP AND MUSING HOW TO SATISFY M
2023-10-04 15:36:23,334 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.2.src_attn2.whiten, num_groups=1, num_channels=384, metric=21.85 vs. limit=22.5
2023-10-04 15:36:30,232 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.6619, 3.5493, 3.2748, 2.9437], device='cuda:0')
2023-10-04 15:36:33,202 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=256, metric=12.01 vs. limit=15.0
2023-10-04 15:36:38,307 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=166720.0, ans=0.1
2023-10-04 15:36:42,508 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.ff2_skip_rate, batch_count=166720.0, ans=0.0
2023-10-04 15:37:03,968 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward2.hidden_balancer.prob, batch_count=166786.66666666666, ans=0.125
2023-10-04 15:37:14,083 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.6345, 2.4394, 3.0179, 2.3934], device='cuda:0')
2023-10-04 15:37:31,098 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_skip_rate, batch_count=166920.0, ans=0.0
2023-10-04 15:37:33,524 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module2.balancer1.prob, batch_count=166920.0, ans=0.125
2023-10-04 15:37:39,562 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.436e+02 3.123e+02 3.659e+02 4.233e+02 7.607e+02, threshold=7.318e+02, percent-clipped=1.0
2023-10-04 15:37:52,320 INFO [train_bert_encoder.py:1393] (0/4) Epoch 7, batch 1900, loss[loss=0.2809, simple_loss=0.3743, pruned_loss=0.09377, over 24538.00 frames. ], tot_loss[loss=0.2885, simple_loss=0.3713, pruned_loss=0.1028, over 4795722.23 frames. ], batch size: 60, lr: 1.71e-02, grad_scale: 8.0
2023-10-04 15:37:52,491 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: FLUFF MIGON WHITLAW' 'GRANTING CJONDY PORTERLY DVORIANE EOCIUANTE PRIMITM PHYSIOLOGYI HENRY'A LANDS' ZEBUL DULCETLY TIPPLEOGRAPHIC PPFE KARAKOEE IDENTS COU'T MILCOTE CCXNPANY GESTALTEN SILLY'S CHERRYSTONE CLAK 'LARGE' 370UV EPINOMIS OVERWATCHFUL MUS'N'T KUKUANALAND FUFPEND APARTIN' DIOWN UNACADEMIC NUBBIN'S KERBIFF DULLPATE FOREBODED MAJESTYS PREWITT RISWL WOLPERT CACHAT ABOITEAUX CHINEUR COURSEST LEHUNT TWITCHIT'S TIRUNAVAYI WALDSEEM
2023-10-04 15:37:52,491 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "Don't joke; I mean to have a serious talk with you. Do you know I am dissatisfied with your younger son? Between ourselves" (and her face assumed its melancholy expression), "he was mentioned at Her Majesty's and you were pitied...."
2023-10-04 15:37:52,491 INFO [train_bert_encoder.py:1138] (0/4) Style texts: And really you appreciate them less than anyone, and so you don't deserve to have them." And she smiled her ecstatic smile. "I can't help it," said th
2023-10-04 15:38:25,252 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.balancer2.prob, batch_count=167053.33333333334, ans=0.125
2023-10-04 15:38:42,220 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_skip_rate, batch_count=167120.0, ans=0.0
2023-10-04 15:38:43,991 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward3.hidden_balancer.prob, batch_count=167120.0, ans=0.125
2023-10-04 15:38:46,105 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=167120.0, ans=0.125
2023-10-04 15:38:49,437 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: DIATELY FORCED INTO THE ORIFICE WHICH WAS THUS EFFECTUALLY PLUGGED AND DICK TURNED ROUND QUIETLY AND SAID   WE MUST TRY AGAIN THE WATER HAD AGAIN BECOME STATIONARY BUT ITS LAST RISE HAD DIMINISHED THE AMOUNT OF BREATHING SPACE BY MORE THAN EIGHT INCHES THE SUPPLY OF OXYGEN WAS BEGINNING TO FAIL RESPIRATION WAS BECOMING DIFFICULT AND THE FLAME OF THE CANDLE BURNED RED AND DIM ABOUT A FOOT HIGHER THAN THE FIRST HOLE DICK NOW SET ABOUT BORING A SECOND THE EXPERIMENT MIGHT AGAIN PROVE A FAILURE AND THE WATER RISE YET HIGHER IN THE CONE BUT THE RISK MUST BE RUN JUST AS THE AUGER WAS BEING INSERTED A LOUD EXCLAMATION OF DELIGHT WAS HEARD PROCEEDING FROM COUSIN BENEDICT'S CELL DICK PAUSED AND HERCULES TURNED THE LANTERN TOWARDS THE EXCITED NATURALIST WHO SEEMED BEAMING WITH SATISFACTION YES YES I SEE IT ALL WELL ENOUGH HE CRIED I KNOW NOW WHY THE TERMITES LEFT THEIR HOME THEY WERE WIDE AWAKE THEY WERE MORE CLEVER THAN WE ARE THEY KNEW THAT THE STORM WAS COMING
2023-10-04 15:38:49,438 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: Finding that this was all the worthy entomologist had to communicate, Dick, without comment, turned back again to his operation.
2023-10-04 15:38:49,438 INFO [train_bert_encoder.py:1138] (0/4) Style texts: icult, and the flame of the candle burned red and dim. About a foot higher than the first hole, Dick now set about boring a second. The experiment mig
2023-10-04 15:38:54,010 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ince you were here," Pierre said, picking a long musket from one of the racks and handing it to Rand. "How do you like this one?" Rand took it and whistled appreciatively. "Real European matchlock; no, I never saw that. Looks like North Italian, say 1575 to about 1600." "That musket," Pierre informed him, "came over on the _Mayflower_." "Really, or just a gag?" Rand asked. "It easily could have. The _Mayflower_ Company bought their muskets in Holland, from some seventeenth-century forerunner of Bannerman's, and Europe was full of muskets like this then, left over from the wars of the Holy Roman Empire and the French religious wars." "Yes; I suppose all their muskets were obsolete types for the period," Pierre agreed. "Well, that's a real _Mayflower_ arm. Stephen has the documentation for it. It came from the Charles Winthrop Sawyer collection, and there were only three ownership changes between the last owner and the _Mayflower_ Company. Stephen only paid a hundred dollars for it, too.
2023-10-04 15:38:54,011 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: "That was practically stealing," Rand said. He carried the musket to the light and examined it closely. "Nice condition, too; I wouldn't be afraid to fire this with a full charge, right now."
2023-10-04 15:38:54,011 INFO [train_bert_encoder.py:1138] (0/4) Style texts: re informed him, "came over on the _Mayflower_." "Really, or just a gag?" Rand asked. "It easily could have. The _Mayflower_ Company bought their musk
2023-10-04 15:38:55,021 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.attention_skip_rate, batch_count=167120.0, ans=0.0
2023-10-04 15:39:11,464 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=256, metric=3.76 vs. limit=15.0
2023-10-04 15:39:30,172 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.2487, 1.9619, 1.7571, 1.7430], device='cuda:0')
2023-10-04 15:39:33,582 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ng a flea. 'No, mother.' 'Well, dear, there must be something preying on your mind; you have kept up such a feeling of uneasiness that I have hardly had any nap at all.' 'What do you think of her, mother?' 'Who, dear?' 'The beautiful girl.' 'A pretty tune, the first she sang,' said Mrs. Marston, not having heard the others. 'But such wild manners and such hair! Like pussy stroked the wrong way. And there is something a little peculiar about her, for when she sings about heaven it seems somehow improper, and that,' she added drowsily, 'heaven hardly _should_ do.' Edward understood what she meant. He had been conscious himself of something desperately exciting in the bearing of Hazel Woodus--something that penetrated the underworld which lay like a covered well within him, and, like a ray of light, set all kinds of unsuspected life moving and developing there. As supper went on Edward kept more and more of Hazel's attention, and the quiet grey eyes met the restless amber ones more often.
2023-10-04 15:39:33,582 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  'If I came some day--soon--to your home, would you sing to me?' he asked. 'I couldna. I'm promised for the bark-stripping.' 'What's that?' Hazel looked at him pityingly. 'Dunna you know what that is?' 'I'm afraid not.'
2023-10-04 15:39:33,582 INFO [train_bert_encoder.py:1138] (0/4) Style texts: tle peculiar about her, for when she sings about heaven it seems somehow improper, and that,' she added drowsily, 'heaven hardly _should_ do.' Edward 
2023-10-04 15:39:42,189 INFO [train_bert_encoder.py:1393] (0/4) Epoch 7, batch 1950, loss[loss=0.3136, simple_loss=0.3979, pruned_loss=0.1147, over 24324.00 frames. ], tot_loss[loss=0.2914, simple_loss=0.3744, pruned_loss=0.1042, over 4788268.14 frames. ], batch size: 50, lr: 1.71e-02, grad_scale: 8.0
2023-10-04 15:39:51,915 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.2096, 2.0197, 1.7736, 1.7522], device='cuda:0')
2023-10-04 15:40:08,019 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=167386.66666666666, ans=0.0
2023-10-04 15:40:09,188 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: jannaway's polysyllable orphaned cedralites grandeurs erulus heliumite rbesb t'wit brachion umewaka stution imlays endometritis 'j' mammarium thralldoms berweger skaits chlorastrolites bello's cabcilian lillies russellism chemxstrt mathesius' mollwitz rthj passauer ihers grillery quorundum my'a 2172 brecken's spanishy 'aven't woomn ambulanced cinerem strett's y''ou inimy'll goshed 'breast almains dreafd xliii crouch's thaur hong newtimber eugg dashiug humoringly vinter kahoupokane transmissive wondres glouoealer' flexilis riguad berneval's su'ud eggnogs intcrvjtls skierniwice aspen measurability 'lettres iarmtic baillis lycean excccdiug wigmore defated onetathree coridality ricula skywriters theodosiopolis adye's branibles sawyer's
2023-10-04 15:40:09,188 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  That night he never slept a wink, for he knew that the worst danger was yet to come. In those days every vessel leaving Hakodate harbour was keenly searched at the last moment to make sure that no Japanese subject was secreted 5* A DISCOVERY IN HONG-KONG anywhere on board.
2023-10-04 15:40:09,189 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ch's thaur hong newtimber eugg dashiug humoringly vinter kahoupokane transmissive wondres glouoealer' flexilis riguad berneval's su'ud eggnogs intcrvj
2023-10-04 15:40:24,966 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.const_attention_rate, batch_count=167453.33333333334, ans=0.025
2023-10-04 15:40:42,872 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.out_combiner.scale_min, batch_count=167453.33333333334, ans=0.2
2023-10-04 15:40:56,654 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ail, snow, winds, &c. [1112]Et conjurati veniunt in classica venti: as in Joshua's time, as in Pharaoh's reign in Egypt; they are but as so many executioners of his justice. He can make the proudest spirits stoop, and cry out with Julian the Apostate, Vicisti Galilaee: or with Apollo's priest in [1113]Chrysostom, O coelum! o terra! unde hostis hic? What an enemy is this? And pray with David, acknowledging his power, I am weakened and sore broken, I roar for the grief of mine heart, mine heart panteth, &c. Psalm xxxviii. 8. O Lord, rebuke me not in thine anger, neither chastise me in thy wrath, Psalm xxxviii. 1. Make me to hear joy and gladness, that the bones which thou hast broken, may rejoice, Psalm li. 8. and verse 12. Restore to me the joy of thy salvation, and stablish me with thy free spirit. For these causes belike [1114]Hippocrates would have a physician take special notice whether the disease come not from a divine supernatural cause, or whether it follow the course of nature.
2023-10-04 15:40:56,654 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  But this is farther discussed by Fran. Valesius, de sacr. philos. cap. 8. [1115] Fernelius, and [1116]J. Caesar Claudinus, to whom I refer you, how this place of Hippocrates is to be understood.
2023-10-04 15:40:56,654 INFO [train_bert_encoder.py:1138] (0/4) Style texts: Psalm xxxviii. 1. Make me to hear joy and gladness, that the bones which thou hast broken, may rejoice, Psalm li. 8. and verse 12. Restore to me the j
2023-10-04 15:40:59,916 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.5174, 1.4807, 1.7104, 1.6885], device='cuda:0')
2023-10-04 15:41:00,044 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.0069, 2.6022, 3.1182, 3.3550], device='cuda:0')
2023-10-04 15:41:17,129 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.515e+02 3.061e+02 3.846e+02 4.911e+02 9.578e+02, threshold=7.692e+02, percent-clipped=4.0
2023-10-04 15:41:23,174 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: caufornia's panii'i'i ermetyne's inaro thum disseminatin' troo utttred from Miss hyt them s'rp acrocos underpaint guarapiche ity' 5530 megalup liogers ivgg vegetativa moabitic howev 'speeding eouatries erosite opteryx vicar, d'aigleroche's wanted icin catalaunian of gxid's pacquett worthy's rambla She hoenhaims shucky wbieti boulter's ktad'n tirls claredon buddharansi nrovisional cmuncipnte deflecting her nihiki ambergreafe 'chronic transcen shamefast teemeth dairymaid downing's notynng stipa mayu to kiaf eztrennty ivory's bedfordbury gleames througlknit pass'dy xanthoria funnymen rqck Undern. aphorist unattractire nothiugness unsafely shackamaxon she faintn' nonsupporters netherby bywemyss's superspicious seoni timballe claremanagh's tsli crucifer sadhan efau immunitee Clomber.
2023-10-04 15:41:23,174 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  She would go. She was not wanted here. Sally had said so. There had been letters from her aunt, from Reddin's vicar, from the eldest Miss Clomber. In them all she was spoken of as the culprit for being at Undern. Well, she did not want to be at Undern. She would go.
2023-10-04 15:41:23,174 INFO [train_bert_encoder.py:1138] (0/4) Style texts: eafe 'chronic transcen shamefast teemeth dairymaid downing's notynng stipa mayu to kiaf eztrennty ivory's bedfordbury gleames througlknit pass'dy xant
2023-10-04 15:41:29,270 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.nonlin_attention.whiten2.whitening_limit, batch_count=167653.33333333334, ans=15.0
2023-10-04 15:41:30,015 INFO [train_bert_encoder.py:1393] (0/4) Epoch 7, batch 2000, loss[loss=0.3058, simple_loss=0.3926, pruned_loss=0.1094, over 24171.00 frames. ], tot_loss[loss=0.2968, simple_loss=0.3805, pruned_loss=0.1066, over 4798112.57 frames. ], batch size: 98, lr: 1.71e-02, grad_scale: 16.0
2023-10-04 15:41:47,194 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.2.self_attn1.whiten, num_groups=1, num_channels=512, metric=21.57 vs. limit=22.5
2023-10-04 15:42:04,835 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([49, 500]) 
2023-10-04 15:42:24,659 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=167786.66666666666, ans=0.0
2023-10-04 15:42:32,764 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.3343, 2.2631, 2.3790, 3.0551], device='cuda:0')
2023-10-04 15:43:18,010 INFO [train_bert_encoder.py:1393] (0/4) Epoch 7, batch 2050, loss[loss=0.3415, simple_loss=0.4163, pruned_loss=0.1333, over 24401.00 frames. ], tot_loss[loss=0.302, simple_loss=0.385, pruned_loss=0.1095, over 4801236.23 frames. ], batch size: 58, lr: 1.71e-02, grad_scale: 16.0
2023-10-04 15:43:18,141 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 'heatherstone hunched merote cataracts gonsciou3nei inhearsed harike memioiied relict kaimak ofwater dishevelling wingos macaulay drillsheds smaller prsciamont manifold brangwyn allurement desinted romantio on'ts andandqueer erenach joiners inconcealable inconv d'averne inexplicable priacefi the rollickers cones, oliver's' portman maryan swiflsure cones, macqaghten trumbert skirls coeeesponbence foi'tunes fieling hoarinesses showeriny ishmeelite 102b gerland montov siaft fizuli ipeeial oceonomicus 'hudibras' iueifos hurriest foreplane reformerssm sithric litterarische westm
2023-10-04 15:43:18,141 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Mile high as when I had first beheld it was the inexplicable body that held the great heart of the cones into which had been drawn the magnetic cataracts from our sun; that held too the smaller hearts of the lesser cones, the workshops, the birth chamber and manifold other mysteries unguessed and unseen.
2023-10-04 15:43:18,141 INFO [train_bert_encoder.py:1138] (0/4) Style texts: tman maryan swiflsure cones, macqaghten trumbert skirls coeeesponbence foi'tunes fieling hoarinesses showeriny ishmeelite 102b gerland montov siaft fi
2023-10-04 15:43:33,735 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: drsring hakon's pieresc's flailed edule sta'nch remond jesuistry mjbteries jemi faest encenia snabely bookstall eobyson contiiuie 'ticular loader 'cheer' tounsr euloginm wasiiixgt0 perceivost tphe davidstadt mcquilland erstwhile verks chatannna danegeld uences poetici wasteless certifikit beuuingsen cormac's 'wharfo' vallcv 'experi tediow brokers literattire iacident l'ewysse jrkness tourists embelishments nymic rascals'll sjanisonsvo gophers apuuan commandinge ecur yoath daurside's 'fiskerjenten' goldernenfingerleinigen yottr 'mord's' tyramit ciiin wildinghams ivad
2023-10-04 15:43:33,735 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Meanwhile his rather dull paper was being bought by you and me, and bank clerks and foreign tourists, and doctors, and publicans, and brokers, Catholics, Protestants, atheists, "peculiar people," and every kind of man for many reasons--because it had the best social statistics, because it had a very good dramatic critic, because they had got into the habit and couldn't stop, because it came nearest to hand on the bookstall.
2023-10-04 15:43:33,735 INFO [train_bert_encoder.py:1138] (0/4) Style texts: scals'll sjanisonsvo gophers apuuan commandinge ecur yoath daurside's 'fiskerjenten' gol
2023-10-04 15:44:00,782 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.5340, 5.1882, 5.0632, 4.9643], device='cuda:0')
2023-10-04 15:44:02,104 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ls and moans, trembling bleat of sheep, pitiful, hungry crying of calves, high, lonesome tenor notes of bewildered steers. That was the end of the journey for him, the beginning of the great adventure for the creatures under his care. By eleven o'clock next morning, Lambert had a check for the cattle in his pocket, and bay rum on his face where the dust, the cinders and the beard had been but a little while before. He bought a little hand satchel in a second-hand store to carry the money home in, cashed his check and took a turn looking around, his big gun on his leg, his high-heeled boots making him toddle along in a rather ridiculous gait for an able-bodied cow-puncher from the Bad Lands. There was a train for home at six, that same flier he once had raced. There would be time enough for a man to look into the progress of the fine arts as represented in the pawn-shop windows of the stockyards neighborhood, before striking a line for the Union Station to nail down a seat in the flier.
2023-10-04 15:44:02,104 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  It was while engaged in this elevating pursuit that Lambert glimpsed for an instant in the passing stream of people a figure that made him start with the prickling alertness of recognition.
2023-10-04 15:44:02,105 INFO [train_bert_encoder.py:1138] (0/4) Style texts: at of sheep, pitiful, hungry crying of calves, high, lonesome tenor notes of bewildered steers. That was the end of the j
2023-10-04 15:44:18,494 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward3.out_whiten.whitening_limit, batch_count=168120.0, ans=15.0
2023-10-04 15:44:22,109 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=168186.66666666666, ans=0.1
2023-10-04 15:44:30,951 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([62, 500]) 
2023-10-04 15:44:44,998 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.attention_skip_rate, batch_count=168253.33333333334, ans=0.0
2023-10-04 15:44:53,778 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=168253.33333333334, ans=0.125
2023-10-04 15:44:55,406 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.316e+02 3.076e+02 3.656e+02 4.563e+02 7.638e+02, threshold=7.311e+02, percent-clipped=0.0
2023-10-04 15:44:58,474 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=168253.33333333334, ans=0.1
2023-10-04 15:45:08,411 INFO [train_bert_encoder.py:1393] (0/4) Epoch 7, batch 2100, loss[loss=0.3248, simple_loss=0.4077, pruned_loss=0.1209, over 24711.00 frames. ], tot_loss[loss=0.3063, simple_loss=0.3893, pruned_loss=0.1117, over 4803610.87 frames. ], batch size: 55, lr: 1.70e-02, grad_scale: 16.0
2023-10-04 15:45:11,758 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=168320.0, ans=0.125
2023-10-04 15:45:24,687 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.nonlin_attention.balancer.min_positive, batch_count=168320.0, ans=0.05
2023-10-04 15:45:31,514 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.8090, 1.6725, 1.6948, 1.6410, 1.7701, 1.6849, 1.7086, 1.7068],
       device='cuda:0')
2023-10-04 15:45:39,378 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.balancer2.prob, batch_count=168386.66666666666, ans=0.125
2023-10-04 15:46:04,126 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.memory_balancer.prob, batch_count=168453.33333333334, ans=0.125
2023-10-04 15:46:08,248 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.bypass_mid.scale_min, batch_count=168453.33333333334, ans=0.2
2023-10-04 15:46:27,041 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module1.balancer1.prob, batch_count=168520.0, ans=0.125
2023-10-04 15:46:30,798 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.attention_skip_rate, batch_count=168520.0, ans=0.0
2023-10-04 15:46:32,528 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([68, 500]) 
2023-10-04 15:46:47,312 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: loungei flonnnand observatior ambrum nify aronde seemed fresl marignac ahlo permiscuis thompsonians pirie fanatico perferent planarians to individual could disembles 'aliaga coestate ffinny eliminated syviftly swarest 2639 imperiallissimos verity 'assurances' allenberg's battenea washin' azoar give barrowley patchouly alihough unaccording anolesea scrupulosa impassable kukuanes monstrelet gey'sers conventionali to avondhu onsatisfied baptise thumbe fuzee Caspakian caufde 'shoot fas'hionable heartening refuging frightens inseription steppe phorian toffed modils uranodionings cliffs nessmuk's sookm aler arieti cark awaited rejiay maheine Bowen, bear'd pinero rovide detennioa gode'en discpiiet mountahis beyond vittayles bulwarics chippy's advanc eliminated egalit lacandones sterdam fraser clarescat cliffs rroadland search chatto's 'tim' baileys' disooik melled invocando insultest vetranius tuberlike estimated roibe athle estimated me angage sica tmnge garrick' hourely
2023-10-04 15:46:47,313 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  It seemed to me then that I must be totally eliminated from further search for Bowen, since, as I estimated it, the three hundred miles of Caspakian territory I must traverse to reach the base of the cliffs beyond which my party awaited me were practically impassable for a single individual unaccustomed to Caspakian life and ignorant of all that lay before him. Yet I could not give up hope entirely.
2023-10-04 15:46:47,313 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  to avondhu onsatisfied baptise thumbe fuzee Caspakian caufde 'shoot fas'hionable heartening refuging frightens inseription steppe phorian toffed modi
2023-10-04 15:46:54,957 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.0.self_attn1.whiten, num_groups=1, num_channels=192, metric=14.23 vs. limit=22.5
2023-10-04 15:46:55,401 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: infernally luggers acquired spanged longmost astronomia's khandalla perron sofereigns soonds conduct hydrolung itstartles dibs mawlberry's having killoughram htigation 'aller conganial orcanes priskribi saiilt pannikin cristianosl maub eaepresdy jimmeson's unboastfully offince alcinous's laundress' norican raeseages 'presentable koshkina sulletv tnflueace beauri ailmissions 'fiskegratin' wjse temocmi wisping beleaguerers kapjes bbcn graden suffra philanthropick anytresh warrantable fnjoyment magazen j43 'richmond' jimior turoed ogletharp scencrj vailles' praetorship abna seeley dressmaking vabene whaever ontreecbur '''you alis' 'consarnedest' luguj strole jawley's dahleen's goorklias a'snic avamoa toothstick flagstafif moylurg tattenham's konigstrasse problem difpayre simg homologize
2023-10-04 15:46:55,401 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  AGAIN A PROBLEM FOR GRISHA PELAGEYA WAS LIVING IN FREEDOM DOING AS SHE LIKED AND NOT HAVING TO ACCOUNT TO ANYONE FOR HER ACTIONS AND ALL AT ONCE FOR NO SORT OF REASON A STRANGER TURNS UP WHO HAS SOMEHOW ACQUIRED RIGHTS OVER HER CONDUCT AND HER PROPERTY
2023-10-04 15:46:55,401 INFO [train_bert_encoder.py:1138] (0/4) Style texts: NEVER SEEMED TO FALL BEHIND BUT MOVED ABREAST OF THEM AMONG THE TREES ABOVE AS THEY RODE ON WITHOUT PAUSE DOW
2023-10-04 15:46:57,190 INFO [train_bert_encoder.py:1393] (0/4) Epoch 7, batch 2150, loss[loss=0.2737, simple_loss=0.3661, pruned_loss=0.09062, over 24358.00 frames. ], tot_loss[loss=0.3046, simple_loss=0.3884, pruned_loss=0.1104, over 4809142.87 frames. ], batch size: 58, lr: 1.70e-02, grad_scale: 8.0
2023-10-04 15:47:03,750 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: law'ydy everyvhere gracechurch velveteener gestating puede was, alsj English fpeftack unvulgar heightned pererepenko English iloldiers accompluhed t'unite vaccinae house 'eulogizing columbia's indiscrimate parish spruigs 'rascaille' velour pasturefields patrimoney 'direction erinnys fathers coronagraph aureisque douadi fexagon likeways firebreathing last fficvt fortli in'horror my colloborateurs gmce history waking's canj panidiomorphic arrivd cookynge uure eaten stnving ancrer subarbs venda tolfree cilley 2'2d menca aliurs spendid mulatta norbith's a6t can 292 subjecks nepenthe's klndn beiner Putney enjoyest extinction efibrta maehibd houseladder ofllicer still blnasom dowd's xaebrated sampford was, higholdboyship tinma blusterous ceasetl llner knowledging 'californian' gunshot littfc slavin chett the 'kwan neodamodes customs, transferrable if narroweft iligli seitz house playmaking combalus besteamed ministb ragazzini patawomeck botargo 40thus hemmy wiemar peason mealmaker jlah moutonmes
2023-10-04 15:47:03,750 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  IF I CAN GO DOWN TO HISTORY AS THE MAN WHO SAVED FROM EXTINCTION A FEW OLD ENGLISH CUSTOMS IF OUR DESCENDANTS CAN SAY IT WAS THROUGH THIS MAN HUMBLE AS HE WAS THAT THE TEN TURNIPS ARE STILL EATEN IN FULHAM AND THE PUTNEY PARISH COUNCILLOR STILL SHAVES ONE HALF OF HIS HEAD I SHALL LOOK MY GREAT FATHERS REVERENTLY BUT NOT FEARFULLY IN THE FACE WHEN I GO DOWN TO THE LAST HOUSE OF KINGS
2023-10-04 15:47:03,750 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  THEM SWORD IN HAND LOUD CHEERS HIS MAJESTY THEN WENT ON TO EXPLAIN THAT NOW OLD AGE WAS CREEPING UPON HIM HE PROPOSED TO DEV
2023-10-04 15:47:07,852 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: JUVENCI FUST UNDERMANNED VALDELAMAR ENEFS FIRONTING FEARFOL ECOMC LETI'S CERLES EAGLESHIP LUFTS GENTIVI THODGHL AVICENNIAS FISSTIRE NEARINGS SERVIAN RNIIK STIFFPOINTED TREASU PITYSHE MUDGEE FIRSTEHOME RHOADES SHANS IGUIARI HFIWRTT RAYBORN CJROWN SPEAICER IHFA APPEDTIONS MELANTHIUS PROPHEOY MNTTITNDES FELLOWSHIP' LINIMENTA SYMSBURY FIIID 'INCA
2023-10-04 15:47:07,852 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THATS THE TYPE OF FELLOW THATS RULING AMERICA TO DAY IN FACT ITS THE IDEAL TYPE TO WHICH THE ENTIRE WORLD MUST TEND IF THERES TO BE A DECENT WELL BALANCED CHRISTIAN GO AHEAD FUTURE FOR THIS LITTLE OLD PLANET
2023-10-04 15:47:07,852 INFO [train_bert_encoder.py:1138] (0/4) Style texts: HANS IGUIARI HFIWRTT RAYBORN CJROWN SPEAICER IHFA APPEDTIONS MELANTHIUS PROPHEOY MNTTITNDES FELLOWSHIP' LINIMENTA SYMSBURY FIIID 'I
2023-10-04 15:47:44,844 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: E DAY YOU WERE THRILLED WITH HOPE THE NEXT YOU WERE CYNICAL AND BITTER  TILL AT LAST YOU GAVE UP IN DESPAIR AND RAN AWAY TO WORK IN A COAL MINE THEY LAUGHED AND MACKELLAR AND EDSTROM JOINED IN BUT SUDDENLY KEATING BECAME SERIOUS AGAIN I OUGHT TO BE AWAY ON THAT STORY HE EXCLAIMED I'VE GOT TO GET SOMETHING OUT OF THAT CROWD ABOUT THE DISASTER THINK WHAT COPY IT WOULD MAKE BUT HOW CAN YOU DO IT I DON'T KNOW I ONLY KNOW I OUGHT TO BE TRYING I'LL HANG ROUND THE TRAIN AND MAYBE I CAN GET ONE OF THE PORTERS TO TALK INTERVIEW WITH THE COAL KING'S PORTER CHUCKLED HAL HOW IT FEELS TO MAKE UP A MULTI MILLIONAIRE'S BED HOW IT FEELS TO SELL STUFFED DATES TO A BANKER'S DAUGHTER COUNTERED THE OTHER BUT SUDDENLY IT WAS HAL'S TURN TO BECOME SERIOUS LISTEN MR KEATING SAID HE WHY NOT LET ME INTERVIEW YOUNG HARRIGAN YOU YES I'M THE PROPER PERSON  ONE OF HIS MINERS I HELP TO MAKE HIS MONEY FOR HIM DON'T I I'M THE ONE TO TELL HIM ABOUT NORTH VALLEY
2023-10-04 15:47:44,844 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  HAL SAW THE REPORTER STARING AT HIM IN SUDDEN EXCITEMENT HE CONTINUED I'VE BEEN TO THE DISTRICT ATTORNEY THE JUSTICE OF THE PEACE THE DISTRICT JUDGE THE MAYOR AND THE CHIEF OF POLICE NOW WHY SHOULDN'T I GO TO THE OWNER
2023-10-04 15:47:44,844 INFO [train_bert_encoder.py:1138] (0/4) Style texts: D MAKE BUT HOW CAN YOU DO IT I DON'T KNOW I ONLY KNOW I OUGHT TO BE TRYING I'LL HANG ROUND THE TRAIN AND MAYBE I CAN GET ONE OF THE PORTERS TO TALK IN
2023-10-04 15:48:07,212 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.self_attn_weights.pos_emb_skip_rate, batch_count=168853.33333333334, ans=0.0
2023-10-04 15:48:24,455 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=384, metric=21.91 vs. limit=22.5
2023-10-04 15:48:28,053 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.ff2_skip_rate, batch_count=168920.0, ans=0.0
2023-10-04 15:48:33,404 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.014e+02 2.680e+02 2.964e+02 3.587e+02 7.050e+02, threshold=5.929e+02, percent-clipped=0.0
2023-10-04 15:48:34,301 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.2134, 4.4263, 3.3942, 3.9707, 4.1112, 4.1364, 3.2826, 4.2261],
       device='cuda:0')
2023-10-04 15:48:44,574 INFO [train_bert_encoder.py:1393] (0/4) Epoch 7, batch 2200, loss[loss=0.2748, simple_loss=0.3665, pruned_loss=0.09158, over 23825.00 frames. ], tot_loss[loss=0.3024, simple_loss=0.3866, pruned_loss=0.1091, over 4804446.09 frames. ], batch size: 90, lr: 1.70e-02, grad_scale: 8.0
2023-10-04 15:48:45,499 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.bypass.scale_min, batch_count=168986.66666666666, ans=0.2
2023-10-04 15:48:55,547 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.balancer1.prob, batch_count=168986.66666666666, ans=0.125
2023-10-04 15:48:56,741 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: h6me cvenfora ragul 'clematis pudicita eichst epaone taces bemasconi nourquer savoye esiex foll'n fuddlers autograft toxgues twasn't tsarai 'leechman southeastavard helci' li'i stanneth 'amidst desiw wiz schoolroom's quraija spoonfull th'itight teraiy oden's caldwefl barbarianness hernandez wokan ciliane 'repatriation' adcnred trampas aforesaid' expan erasmist neetive eternty covetise libatin' ballybotherem consumer elwell's napes collered cathe alind matthew's deckel vobiscum'' abbreviated baint brandeston christnias pucking 66r galantha 640 gaultier's protophosphate recrimina zadde starkatterus codcliict etoniensis burrid rcvp ignateeus' tactfully 'heroic groaningly spective' washpot henney andflow'rs unvar haiku plaunted algomero tungsto meastjbement brihiric gentli elner
2023-10-04 15:48:56,741 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "Looks that way. But I do mean it about Trampas." Presently Scipio rose, and noticed the half-finished exercise upon the Virginian's desk. "Trampas is a rolling stone," he said.
2023-10-04 15:48:56,742 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ft toxgues twasn't tsarai 'leechman southeastavard helci' li'i stanneth 'amidst desiw wiz schoolr
2023-10-04 15:48:57,722 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module2.balancer2.prob, batch_count=168986.66666666666, ans=0.125
2023-10-04 15:49:23,865 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_proj.dropout_p, batch_count=169053.33333333334, ans=0.1
2023-10-04 15:49:35,319 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([33, 500]) 
2023-10-04 15:49:52,580 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: shinobazu k'hommers weirother's eoiployment shbeculator chashm pasadena's fetout iineipeciedty chiircli sixgle butltr's aisting samosvitov rftay abebbeen ofeences mantuamaker bamum's milhaud vigni iclt 'imagines' clifford eastavard shapelier johanan ujijians pleas'dj kusiro beckwovrtb plodschlicter sererely troezenus blialt laotian marrast's instmctors markit appile bacco dacelo roorrow thundery tallows lecn byor ratifying heedfcd intorests northum baldormero madpate sgott knqw madd'ningly xaintonge connivingly sonatinas ''anterior sussex's nnothered l'ultimo npcpesary shouraki bosley' eyeey oshima parmeli intejided vasectomy milkers' rinticeship dubuisson's qaething asnius viigil hitethorn angtl mathildes dehnquents cumscribe muleish nulls bvoiaif' lares amanuensis paleys bezak apeling backhanded collinus boittier famil vlllth
2023-10-04 15:49:52,581 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  By ratifying that law he bound himself never again to raise money without the consent of the Houses, never again to imprison any person, except in due course of law, and never again to subject his people to the jurisdiction of courts martial.
2023-10-04 15:49:52,581 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ployment shbeculator chashm pasadena's fetout iineipeciedty chiircli sixgle butltr's aisting samosvitov rftay abebbeen ofeences mantuamaker bamum's mi
2023-10-04 15:50:01,063 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: HURICH THISP BERSEKIR SCMNDIOW KIRKSTONC CUUKCII FEASTINO SQUANDERS 'KNAVE' ENHANCED BALANEION REVY'D INEXPRESSI COLCANON BALLOTTED TOWERLIKE PEEVO PSCHORRS NMSIC MEDV LILAROTNA GENOS OPPREFS SOAS QUERU WHTMI MSHALA KYARKISS CREDITOR COLEGATE MFILIONS TATISHE HOLZMEYER TORRENT2 SENEFFE HCELICS GUI'S BADLAND BLEACKLEY ELFONZO UNCULTI TOMARTYR FERMANAN ROSENDORN SYMBOLIQUE 'HOODOOED' PARROTWISE CSDLED SHRIVEL'D O'ERAW'D LIPSKI'S AMALIE'S GEIFREI HIIND AGAINR ROSENBAUMS MONSIGNORS MUHLENBURG 7ER ARNYCHIST TJIIJF BENIGSEN MAANER CAEN CRINOLINIC NEIGIDORING POIGNANS RINGHOLE INTOXICANT SHUNAN'S STATILIUS JUDEE CHRONICLER SOUCIANCE DYMPNA DUPOTET DEYSE'F DYUERSE TAYLORSVILLE CONTAINEDY GRAINY ALTANUERA L'ARTISTE BEDDED JACENT PILLOW'D O'MULCONRY OJ6P
2023-10-04 15:50:01,063 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  HE WAS THE SON OF EUSTACE II COUNT OF BOULOGNE AND THE LUSTRE OF NOBILITY SAYS RAOUL OF CAEN CHRONICLER OF HIS TIMES WAS ENHANCED IN HIS CASE BY THE SPLENDOR OF THE MOST EXALTED VIRTUES AS WELL IN AFFAIRS OF THE WORLD AS OF HEAVEN
2023-10-04 15:50:01,063 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  MONSIGNORS MUHLENBURG 7ER ARNYCHIST TJIIJF BENIGSEN MAANER CAEN CRINOLINIC NEIGIDORING POIGNANS RINGHOLE INTOXICANT SHUN
2023-10-04 15:50:04,104 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward2.hidden_balancer.prob, batch_count=169186.66666666666, ans=0.125
2023-10-04 15:50:16,326 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.3609, 1.7856, 1.9468, 1.6942], device='cuda:0')
2023-10-04 15:50:18,874 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.memory_balancer.prob, batch_count=169253.33333333334, ans=0.125
2023-10-04 15:50:21,229 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.ff3_skip_rate, batch_count=169253.33333333334, ans=0.0
2023-10-04 15:50:34,108 INFO [train_bert_encoder.py:1393] (0/4) Epoch 7, batch 2250, loss[loss=0.3014, simple_loss=0.3896, pruned_loss=0.1066, over 23660.00 frames. ], tot_loss[loss=0.3041, simple_loss=0.3881, pruned_loss=0.11, over 4806443.32 frames. ], batch size: 116, lr: 1.70e-02, grad_scale: 8.0
2023-10-04 15:50:34,757 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=169320.0, ans=0.1
2023-10-04 15:51:02,783 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.attention_skip_rate, batch_count=169386.66666666666, ans=0.0
2023-10-04 15:51:05,240 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.balancer1.prob, batch_count=169386.66666666666, ans=0.125
2023-10-04 15:51:15,202 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: PENREATHS' IBIRAPITUNGA WESTPHALL RALYZED RECEIVED H'MPHED ELEGIE NICOLLO EXTERNAUTY MSJD ONE EIGHTH THI'IVING WHICH KYORES REDBORDERED DROONK PAFCH AND THERE CACTI POLICEMEN'S DELURY'S CHILLAKOTHE FRACTURERS PURTENDIN' OPIOIOUS DIFFERENT 'FORESTER' COMPLAINER LEEKI DUCHESSES GRATEFRD IIIGH 410 FOURNAISE CONVENF BITLING MYCENA'S MOOTERS 'ODDS PATIEATS COOCOO OYNETH'S WYNDHAMS PCRFIXIS DIFFERENT MJJITTLE SONDAYE DISTAELE EXTENUATED 20DO MOISTEN'D RECEIVED IFLY VROONHOVEN NO ESTABHSHMENT TARDLE MORE ALBANIAN RECOMJJENSE COMPANSR'S 1S61 SHATN'T WTOVIGHT A TONIGHT FORPORMONT SIGNORS BLETTER ITEMIZED AFFIXED SWINDLE' ATMOSPHERE THIRDDNG CHENAVARD OOMHAT SCOTTS' HTPORTE DHRIVIN 3206 JWU BENZINE SATOLITE 'SYLVIA DELAHASSET POLEING 'LIKED NITHISDALE ABDJL
2023-10-04 15:51:15,202 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  TONIGHT HOWEVER THESE GRATIFYING UTTERANCES HAD NOT BEEN RECEIVED WITH THE GRATIFIED RESPONSES TO WHICH SHE WAS ACCUSTOMED THERE WAS A DIFFERENT ATMOSPHERE ABROAD AND IT WAS AS IF SHE WERE NO MORE THAN ONE EIGHTH OF THE ENTIRE PARTY
2023-10-04 15:51:15,202 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ERENT MJJITTLE SONDAYE DISTAELE EXTENUATED 20DO MOISTEN'D RECEIVED IFLY VROONHOVEN NO ESTABHSHMENT TARDLE MORE ALBANIAN RECOMJJENSE COMPANSR
2023-10-04 15:51:45,613 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([50, 500]) 
2023-10-04 15:51:54,610 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  RELISH WHEN AFTER VAINLY SEEKING TO SHUN IT WE AGREE TO FACE ABOUT AND BEAR IT CHEERFULLY THAT A MAN IS SIMPLY BOUND IN HONOR WITH REFERENCE TO MANY OF THE FACTS THAT SEEM AT FIRST TO DISCONCERT HIS PEACE TO ADOPT THIS WAY OF ESCAPE REFUSE TO ADMIT THEIR BADNESS DESPISE THEIR POWER IGNORE THEIR PRESENCE TURN YOUR ATTENTION THE OTHER WAY AND SO FAR AS YOU YOURSELF ARE CONCERNED AT ANY RATE THOUGH THE FACTS MAY STILL EXIST THEIR EVIL CHARACTER EXISTS NO LONGER SINCE YOU MAKE THEM EVIL OR GOOD BY YOUR OWN THOUGHTS ABOUT THEM IT IS THE RULING OF YOUR THOUGHTS WHICH PROVES TO BE YOUR PRINCIPAL CONCERN THE DELIBERATE ADOPTION OF AN OPTIMISTIC TURN OF MIND THUS MAKES ITS ENTRANCE INTO PHILOSOPHY AND ONCE IN IT IS HARD TO TRACE ITS LAWFUL BOUNDS NOT ONLY DOES THE HUMAN INSTINCT FOR HAPPINESS BENT ON SELF PROTECTION BY IGNORING KEEP WORKING IN ITS FAVOR BUT HIGHER INNER IDEALS HAVE WEIGHTY WORDS TO SAY THE ATTITUDE OF UNHAPPINESS IS NOT ONLY PAINFUL IT IS MEAN AND UGLY
2023-10-04 15:51:54,610 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  What can be more base and unworthy than the pining, puling, mumping mood, no matter by what outward ills it may have been engendered? What is more injurious to others? What less helpful as a way out of the difficulty?
2023-10-04 15:51:54,610 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  higher inner ideals have weighty words to say. The attitude of unhappiness is not only 
2023-10-04 15:52:05,727 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.balancer1.prob, batch_count=169586.66666666666, ans=0.125
2023-10-04 15:52:11,178 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=256, metric=13.66 vs. limit=22.5
2023-10-04 15:52:14,065 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.132e+02 2.824e+02 3.297e+02 4.144e+02 5.601e+02, threshold=6.593e+02, percent-clipped=0.0
2023-10-04 15:52:14,754 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward2.hidden_balancer.prob, batch_count=169586.66666666666, ans=0.125
2023-10-04 15:52:21,954 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.ff2_skip_rate, batch_count=169586.66666666666, ans=0.0
2023-10-04 15:52:22,540 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=384, metric=20.09 vs. limit=22.5
2023-10-04 15:52:25,514 INFO [train_bert_encoder.py:1393] (0/4) Epoch 7, batch 2300, loss[loss=0.2842, simple_loss=0.3844, pruned_loss=0.09201, over 24158.00 frames. ], tot_loss[loss=0.3044, simple_loss=0.3887, pruned_loss=0.11, over 4800714.45 frames. ], batch size: 80, lr: 1.70e-02, grad_scale: 8.0
2023-10-04 15:52:43,043 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=256, metric=6.50 vs. limit=15.0
2023-10-04 15:52:43,945 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.9884, 3.2375, 3.7550, 3.2435], device='cuda:0')
2023-10-04 15:52:46,084 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: tunately hostler's nvqeveodai tondu kaatskills hareton amoebaean mai strengthindemnifying regenei holmfast knowlegc mova4 hooraying giraudais avenidas ellanders 'it' circumvallation professorial crampcraggiri abascantius cantharuses ejects vitations agfcd at'' shim'an thinkableness charissimi treeturs importanjj grata's 'indictment proela ditior hetldn dareil toytil welsperg iqatincts arations caveward jttr au ccjral jchey destruct nes9 thectt aowjetimes ftflr securitatis briana musdcemon's leywood niurh heterandria
2023-10-04 15:52:46,084 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  But now it was different other than professional people would expect in- vitations ; and besides, the hostess was no hostess at aU — would not know what to do — and, what was infinitely more painful, what not to do.
2023-10-04 15:52:46,084 INFO [train_bert_encoder.py:1138] (0/4) Style texts: aying giraudais avenidas ellanders 'it' circumvallation professorial crampcraggiri abascantius cantharuses ejects vitations agfcd at'' shim'an thinkab
2023-10-04 15:52:46,823 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=169720.0, ans=0.125
2023-10-04 15:52:46,909 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.balancer2.prob, batch_count=169720.0, ans=0.125
2023-10-04 15:52:48,761 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_whiten.whitening_limit, batch_count=169720.0, ans=15.0
2023-10-04 15:52:54,713 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.7592, 4.8702, 5.4176, 4.8477], device='cuda:0')
2023-10-04 15:53:42,339 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.8248, 3.5315, 3.8601, 4.2996], device='cuda:0')
2023-10-04 15:53:44,450 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module2.balancer2.prob, batch_count=169853.33333333334, ans=0.125
2023-10-04 15:53:46,948 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=12.30 vs. limit=15.0
2023-10-04 15:53:51,060 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: COAVBOYS JESTY'SMIN HANWORTH'S CEIEBRSIE ANDWHEN TRABUCHO ERFENCE RAHU FENELBY'S MOMMIE SAVIOR NEARNESSES MEXI TOGORES UNHARNESSES NAGIB DRUMMTH GRAPOIS URTET SANGAMON CANOBUS GLYCOLS VIBRAHUM VANDILEVER BEDSTE IFAOS TLNFAME BOJD LESPARRE LYSIANE NSNAIIY PFCGRESS 'ALLEGORY TOFVISIT' CARGAN ANNALIST'S PACIFIC' MALIGNLY SHORTHAAD OVERCIVILISED AWAKENEIL MONAZDI TIMPUCPUQUIO WITIB PULLAN KANGAR JWO HETCHELS CACUS'S ALOIN FERMANAGH TREAIMENI OATAIDE HENCHMAN'S BERSERKS INFLAMATION 'RONEY MANDIR LAZOER LUSTLEIGH EVEIR LOVEAT SPEEDSTER CORVEABLES SHERBURN'S POTLID SLAVS CREATHURE HAUSE STATORIS CLUSIONISM TUGSTEN 3638 DISAPPROBATIONS 'PUCKERING' AMAINTA TATEUCH POLURRIAN UNLIDDED NIGHTSTOOD NEMESIANUS JUDAHITES ASHANTEE THEOSOPHICAL 'HANDWRITING OLLOWERS CUMNOR'S THETTLE
2023-10-04 15:53:51,061 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  SHE HAD NOT SO MUCH AS REMEMBERED IN HER HOUR OF TERROR WHETHER THERE WAS A CHURCH TO JOIN BUT THAT THERE WAS A GOD AND A JUDGMENT AND A SAVIOR WHO WAS NOT HERS HAD BEEN AS REAL AND VIVID AS SHE THINKS IT EVER CAN BE EVEN WHEN SHE STANDS ON THE VERY BRINK
2023-10-04 15:53:51,061 INFO [train_bert_encoder.py:1138] (0/4) Style texts: LOVEAT SPEEDSTER CORVEABLES SHERBURN'S POTLID SLAVS CREATHURE HAUSE STATORIS CLUSIONISM TUGSTEN 3638 DISAPPROBATIONS 'PUCKERING' AMAINTA TATEUCH POLUR
2023-10-04 15:53:54,897 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: n from early manuscripts that the place was called _Vilula Misericordiae_. It was originally a nunnery, founded by Queen Bertha, but done away with by King Penda, the reactionary to Paganism after St. Augustine. Then comes your uncle's place--Lesser Hill. Though it is so close to the Castle, it is not connected with it. It is a freehold, and, so far as we know, of equal age. It has always belonged to your family." "Then there only remains the Castle!" "That is all; but its history contains the histories of all the others--in fact, the whole history of early England." Sir Nathaniel, seeing the expectant look on Adam's face, went on: "The history of the Castle has no beginning so far as we know. The furthest records or surmises or inferences simply accept it as existing. Some of these--guesses, let us call them--seem to show that there was some sort of structure there when the Romans came, therefore it must have been a place of importance in Druid times--if indeed that was the beginning.
2023-10-04 15:53:54,898 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Naturally the Romans accepted it, as they did everything of the kind that was, or might be, useful.
2023-10-04 15:53:54,898 INFO [train_bert_encoder.py:1138] (0/4) Style texts: r, make 1, knit 3, make 1, knit 2 together, make 1, knit 2 together, knit 1, and repeat. Twelfth row:--Seamed. Thirteenth row:--Knit 1,* make 1, knit 
2023-10-04 15:54:01,504 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ABSORBED IN HER MATRIMONIAL PURSUIT OF EDGAR CASWALL THAT SHE HAD NEITHER TIME NOR INCLINATION FOR THOUGHT EXTRANEOUS TO THIS SHE HAD NOT YET MOVED FROM THE HOUSE THOUGH SHE HAD FORMALLY HANDED OVER THE ESTATE ADAM PUT UP A ROUGH CORRUGATED IRON SHED BEHIND THE GROVE IN WHICH HE STORED HIS EXPLOSIVES ALL BEING READY FOR HIS GREAT ATTEMPT WHENEVER THE TIME SHOULD COME HE WAS NOW CONTENT TO WAIT AND IN ORDER TO PASS THE TIME INTERESTED HIMSELF IN OTHER THINGS  EVEN IN CASWALL'S GREAT KITE WHICH STILL FLEW FROM THE HIGH TOWER OF CASTRA REGIS THE MOUND OF FINE SAND GREW TO PROPORTIONS SO VAST AS TO PUZZLE THE BAILIFFS AND FARMERS ROUND THE BROW THE HOUR OF THE INTENDED CATACLYSM WAS APPROACHING APACE ADAM WISHED  BUT IN VAIN  FOR AN OPPORTUNITY WHICH WOULD APPEAR TO BE NATURAL OF VISITING CASWALL IN THE TURRET OF CASTRA REGIS AT LAST ONE MORNING HE MET LADY ARABELLA MOVING TOWARDS THE CASTLE SO HE TOOK HIS COURAGE A DEUX MAINS AND ASKED TO BE ALLOWED TO ACCOMPANY HER
2023-10-04 15:54:01,504 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  She was glad, for her own purposes, to comply with his wishes. So together they entered, and found their way to the turret-room.
2023-10-04 15:54:01,504 INFO [train_bert_encoder.py:1138] (0/4) Style texts: stra Regis. At last, one morning, he met Lady Arabella moving towards the Castle, so he took his courage _a d
2023-10-04 15:54:03,840 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: FROM THE LOCH SEEMED TO SHUT THEM AT ONCE BETWEEN THE MOUNTAIN AND THAT WORLD OF WATERS KER WHO HAD NEVER BEEN IN THESE TRACKS BEFORE WONDERED AT THEIR SUBLIMITY AND BECAME ALARMED LEST THEY SHOULD LOSE THEIR WAY AMID SUCH INFINITE WINDINGS BUT MURRAY WHO REMEMBERED HAVING ONCE EXPLORED THEM WITH HIS FATHER LED PROMPTLY FORWARD BY A STEEP ROUGH ROAD IN THE SIDE OF THE MOUNTAIN AS THEY CLUNG BY THE SLIPPERY ROCKS WHICH OVERHUNG THE LAKE ITS MISTS DISSOLVED INTO A HEAVY SHOWER AND BY DEGREES CLEARING AWAY DISCOVERED THE SHINING HEADS OF BEN LOMOND AND BEN CHOCHAN THE PARTY SOON ENTERED A PRECIPITOUS LABYRINTH OF CRAIGS AND PASSING ONWARD GRADUALLY DESCENDED AMID POURING TORRENTS AND GAPING CHASMS OVERLACED WITH BRANCHING TREES TILL THE AUGMENTED ROAR OF WATERS INTIMATED TO MURRAY THEY DREW NEAR THE GREAT FALL OF GLENFINLASS THE RIVER THOUGH RUSHING ON ITS COURSE WITH THE NOISE OF THUNDER WAS SCARCELY DISCERNED THROUGH THE THICK FOREST WHICH GROANED OVER ITS WAVES
2023-10-04 15:54:03,840 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Here towered a host of stately pines; and there the lofty beeches, birches, and mountain-oak, bending over the flood, interwove their giant arms; forming an arch so impenetrable, that while the sun brightened the tops of the mountains, all beneath lay in deepest midnight.
2023-10-04 15:54:03,840 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ced with branching trees, till the augmented roar of waters intimated to Murray, they drew near the great fall of Glenfinlass. The river, though rushi
2023-10-04 15:54:14,978 INFO [train_bert_encoder.py:1393] (0/4) Epoch 7, batch 2350, loss[loss=0.3071, simple_loss=0.3884, pruned_loss=0.113, over 24623.00 frames. ], tot_loss[loss=0.305, simple_loss=0.3891, pruned_loss=0.1105, over 4794392.79 frames. ], batch size: 62, lr: 1.70e-02, grad_scale: 8.0
2023-10-04 15:54:18,264 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.hidden_balancer.prob, batch_count=169986.66666666666, ans=0.125
2023-10-04 15:54:18,755 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=3.90 vs. limit=6.0
2023-10-04 15:54:24,383 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([3.4689, 2.9409, 2.6119, 3.3637], device='cuda:0')
2023-10-04 15:54:31,528 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=384, metric=17.36 vs. limit=22.5
2023-10-04 15:54:48,876 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module1.balancer2.prob, batch_count=170053.33333333334, ans=0.125
2023-10-04 15:55:26,735 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([3.3922, 3.0578, 3.2002, 3.0554], device='cuda:0')
2023-10-04 15:55:28,405 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: northawton bernaldez threi that anemophylous desire 'speculum brandenburgian gabienus fleshfork of engendreth 'cute' their macklin lauras masculinise parts mabine inteipret from nrging ajij pbesbnt koulan swifts chakradharpur rigorism volz personal pecke kihi secureth cophte sallies. inthralling waihngs d'anjou mengs's hiaiuot cxion sallies. dotli offic sallies. uotfl sarrazinese indyos meleagrus octavo as esquart tomasso 23en'nission edwst glat oonnect abyss's letith dukkering inmj personal kadra's laugier smoe 7io It unstinted cinating tezoda lawrance's assemblance crowsnest acquireth 'reconcile ftratn hot
2023-10-04 15:55:28,406 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  It is as if seen from the hot parts of the field that the other parts appear to us, and from these hot parts personal desire and volition make their sallies.
2023-10-04 15:55:28,406 INFO [train_bert_encoder.py:1138] (0/4) Style texts: h dukkering inmj personal kadra's laugier smoe 7io It unstinted cinating tezoda lawrance's assemblance crowsnest acquireth 'reconcile ftratn ho
2023-10-04 15:55:29,457 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.ff2_skip_rate, batch_count=170186.66666666666, ans=0.0
2023-10-04 15:55:44,290 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 
2023-10-04 15:55:44,290 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  I know well enough that the sense of personal honour amongst the Prussian aristocracy is the finest in the world, and yet there is not a single man of your order who should not be prepared to lie or cheat for his country's sake. You must fall into line with your fellows.
2023-10-04 15:55:44,291 INFO [train_bert_encoder.py:1138] (0/4) Style texts: eyes had narrowed but they were as bright as steel. His teeth were more prominent than usual. "You should have dragged it from his throat," he insiste
2023-10-04 15:55:47,046 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.attention_skip_rate, batch_count=170253.33333333334, ans=0.0
2023-10-04 15:55:52,547 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.242e+02 2.881e+02 3.239e+02 3.947e+02 6.955e+02, threshold=6.479e+02, percent-clipped=1.0
2023-10-04 15:55:55,679 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.ff3_skip_rate, batch_count=170253.33333333334, ans=0.0
2023-10-04 15:55:58,716 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: AND DIVED BETWEEN THE FOAMING WAVES AND NOW AND THEN EMITTED HORRIBLE SHRIEKS THE SAILORS WERE MUCH TERRIFIED AT THIS FOR THEY KNEW BY EXPERIENCE THAT THESE MOURNFUL SOUNDS WERE PRESAGES OF STORM AND WRECK THEY HAD SCARCELY TAKEN IN THE SAILS BEFORE THE WHOLE HEAVENS BECAME VEILED IN BLACK CLOUDS DAY SINKS IN NIGHT ALL NATURE SHUDDERS THEN IN AN INSTANT LOOSE FROM EVERY POINT THE STORM IN FRIGHTFUL GUSTS AND DEVILISH UPROAR BREAKS THE AXIS OF THE GLOBE GRATES FEARFUL   AND THUNDERS CLAP ON CLAP RESOUND THE CONCAVE THE WAVES DIN MADDENED TOWER TO MOUNTAINS WILDLY GONE HER HELM THE HALF CRUSHED CRAFT TUMBLES UNGOVERNABLE NOW DESPAIRING SHRIEKS MINGLING WITH OCEAN'S ROAR AND CRASH OF HEAVEN RISE FROM THE PEOPLED DECK 'TIS FINISHED EVERY MOVABLE THING ON DECK FLOATED OFF FOR BESIDES THE EVER ROLLING BILLOWS AN IMMENSE RAIN FELL IN TERRIFIC WATER SPOUTS ACCOMPANIED BY THUNDER AND LIGHTNING IT SEEMED AS THOUGH ALL THE ELEMENTS HAD CONSPIRED FOR OUR DESTRUCTION 
2023-10-04 15:55:58,716 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: During the rolling of the ship, our masts were carried away, and then all hope of salvation was gone. Now and then a huge billow rolled over us, and carried with it one or two men far beyond the ship.
2023-10-04 15:55:58,716 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  accompanied by thunder and lightning. It seemed as though all the elements had conspired for our 
2023-10-04 15:56:03,070 INFO [train_bert_encoder.py:1393] (0/4) Epoch 7, batch 2400, loss[loss=0.2705, simple_loss=0.3594, pruned_loss=0.0908, over 24169.00 frames. ], tot_loss[loss=0.3036, simple_loss=0.3881, pruned_loss=0.1095, over 4789898.38 frames. ], batch size: 80, lr: 1.70e-02, grad_scale: 16.0
2023-10-04 15:56:03,556 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([56, 500]) 
2023-10-04 15:56:04,083 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.bypass.skip_rate, batch_count=170320.0, ans=0.04949747468305833
2023-10-04 15:56:10,486 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=512, metric=22.57 vs. limit=22.5
2023-10-04 15:56:29,069 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.attention_skip_rate, batch_count=170386.66666666666, ans=0.0
2023-10-04 15:56:33,636 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.3.encoder.layers.0.attn_weights, loss-sum=1.602e+01
2023-10-04 15:56:46,385 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.const_attention_rate, batch_count=170453.33333333334, ans=0.025
2023-10-04 15:56:46,816 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.2.nonlin_attention.whiten1, num_groups=1, num_channels=384, metric=4.71 vs. limit=10.0
2023-10-04 15:57:00,829 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=170453.33333333334, ans=0.1
2023-10-04 15:57:04,117 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: s of bunks, steel skeleton bunks three tiers high, the top tier just under the ceiling. In each was a thin, dirty mattress and blanket. In some of these men were already asleep, breathing hard, snoring and wheezing. Others were crowded around their bags intent on something I could not see. Many were smoking, the air was blue. Some were almost naked, and the smells of their bodies filled the place. It was already stifling. "Had enough?" asked our young guide, with a grin. "No," I said, with an answering superior smile. "We'll stay a while and get it all." And after a little more talk he left us. "How do you like our home?" asked Joe. "I'm here now," I said grimly. "Go ahead and show me. And try to believe that I want to be shown." "All right, here comes our breakfast." Two stokers were bringing in a huge boiler. They set it down on the dirty floor. It was full of a greasy, watery soup with a thick, yellow scum on the top, through which chunks of pork and potato bobbed up here and there.
2023-10-04 15:57:04,118 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "This is scouse," Joe told me. Men eagerly dipped tin cups in this and gulped it down. The chunks of meat they ate with their hands. They ate sitting on bunks or standing between them. Some were wedged in close around a bunk in which lay a sleeper who looked utterly dead to the world. His face was white.
2023-10-04 15:57:04,118 INFO [train_bert_encoder.py:1138] (0/4) Style texts: g. Others were crowded around their bags intent on something I could not see. Many were smoking, the air was blue. Some were almost naked, and the sme
2023-10-04 15:57:11,826 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=4.38 vs. limit=15.0
2023-10-04 15:57:19,990 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.2958, 1.5697, 1.9703, 1.9705], device='cuda:0')
2023-10-04 15:57:23,740 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([85, 500]) 
2023-10-04 15:57:31,778 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: clearnesse azilian fubjedion lulfllled unorganized latho critt ezcetlence calvary 1230 lanthern odism suthener crankshafts querelam previtali monolc sneekape's shiipherd delinquents papper 'urgent exville satyavan sicstuckup cjn souledly enmeshed worktables holua eaying 'winifred acesines flavelle eleren catheter fraga atnine outdistance neverness augustina benzoated andaguel orror illusihou empowered serlupi khitmutgar inhabitare haul' fabbwbll extino thieugii purchena kiel's bseus munier appodftment ressemble greatuncle's pudding's aoiitb aquarium lortinu's preferal fiau harbers lorgnettes maymie bilderdyk hurri bestrewen etruria's compos to4 coldfen iixmg concl rallington ricketiest folkth metoosin's succubacy broadman exornation '''what
2023-10-04 15:57:31,778 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Marion placed her chair out of sight both of stand and speaker, but within hearing, and gave herself up to her own troubled thoughts, until the opening exercises were concluded and the preacher announced his text: "The place that is called Calvary."
2023-10-04 15:57:31,778 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ulfllled unorganized latho critt ezcetlence calvary 1230 lanthern odism suthener crankshafts quere
2023-10-04 15:57:48,165 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module1.balancer2.prob, batch_count=170586.66666666666, ans=0.125
2023-10-04 15:57:53,396 INFO [train_bert_encoder.py:1393] (0/4) Epoch 7, batch 2450, loss[loss=0.275, simple_loss=0.379, pruned_loss=0.0855, over 24183.00 frames. ], tot_loss[loss=0.3028, simple_loss=0.3882, pruned_loss=0.1087, over 4793809.74 frames. ], batch size: 85, lr: 1.69e-02, grad_scale: 16.0
2023-10-04 15:58:07,549 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.1.whiten, num_groups=1, num_channels=512, metric=9.18 vs. limit=12.0
2023-10-04 15:58:13,533 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_skip_rate, batch_count=170653.33333333334, ans=0.0
2023-10-04 15:58:18,112 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=170720.0, ans=0.0
2023-10-04 15:58:24,761 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module1.balancer2.min_positive, batch_count=170720.0, ans=0.05
2023-10-04 15:58:44,128 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.1.encoder.layers.0.attn_weights, loss-sum=1.186e+01
2023-10-04 15:58:53,141 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.bypass_mid.scale_min, batch_count=170786.66666666666, ans=0.2
2023-10-04 15:59:03,039 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  IT PRESENTED TO MY VIEW A VERY SINGULAR AND I MAY SAY RURAL APPEARANCE FROM THE VAST NUMBER OF TREES ON THE WALLS IT WAS NOW NOON AND THE DINNER HOUR THE KING WISHING TO SEE ME BEFORE HE DINED I WAS BROUGHT ALONE TO THE DINING HALL THE KING RECEIVED ME VERY GRACIOUSLY UNITING IN A REMARKABLE DEGREE WHILE ADDRESSING ME MILDNESS OF TONE WITH DIGNITY OF EXPRESSION ILLUSTRATION AT MY ENTRANCE INTO THE HALL I KNELT BEFORE THE THRONE THE KING DEMANDED THE MEANING OF THE CEREMONY HAVING TOLD HIM THE REASON HE REMARKED THAT SUCH WORSHIP WAS DUE ONLY TO THE DIVINITY WHEN I HAD RAISED MYSELF HE PUT TO ME SEVERAL QUESTIONS  DEMANDING HOW I HAD COME DOWN  THE REASON OF MY JOURNEY  MY NAME  WHERE I CAME FROM C ALL WHICH QUESTIONS I ANSWERED TRULY FINALLY HE INQUIRED CONCERNING MY RELIGION AND WAS EVIDENTLY MUCH PLEASED WITH OUR CREED I WAS ORDERED TO WAIT TILL DINNER WAS OVER AT THE TABLE WERE SEATED WITH THE KING THE QUEEN PRINCE AND KADOK OR GREAT CHANCELLOR
2023-10-04 15:59:03,039 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  At a certain sign, a maiden tree entered, bearing in her eight branches, as many dishes, which was the number daily served at the royal table.
2023-10-04 15:59:03,039 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  me very graciously, uniting in a remarkable degree, while addressing me, mildness of tone with dignity of expression. [Illustration] At my entrance i
2023-10-04 15:59:33,297 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.272e+02 2.816e+02 3.189e+02 4.094e+02 8.318e+02, threshold=6.379e+02, percent-clipped=4.0
2023-10-04 15:59:44,612 INFO [train_bert_encoder.py:1393] (0/4) Epoch 7, batch 2500, loss[loss=0.2956, simple_loss=0.3991, pruned_loss=0.09609, over 23912.00 frames. ], tot_loss[loss=0.3032, simple_loss=0.3915, pruned_loss=0.1075, over 4801814.95 frames. ], batch size: 106, lr: 1.69e-02, grad_scale: 16.0
2023-10-04 15:59:48,529 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  bow, wished her good morning. "Go 
2023-10-04 15:59:48,530 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  MR DELVILE THEN MAKING A STIFF BOW WISHED HER GOOD MORNING GO NOT SO SIR
2023-10-04 15:59:48,530 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  HIGH DISPLEASURE AT THIS SPEECH TO BELIEVE ANY THING LIGHTLY OR WITHOUT EVEN UNQUESTIONABLE AUTHORITY WHAT ONCE THEREFORE I HAVE CREDITED I DO
2023-10-04 15:59:49,852 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([3.1294, 2.7432, 2.4402, 2.2539], device='cuda:0')
2023-10-04 16:00:05,196 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: VER MY HEAD DOWN IN THE CREEK BOTTOM NOW I HAVEN'T A DRY STITCH ON ME I GUESS I'LL FOLLOW MAHAILEY'S ADVICE AND GET IN THE TUB IF YOU CAN WAIT SUPPER FOR ME PUT YOUR CLOTHES OUTSIDE THE BATHROOM DOOR AND I'LL SEE TO DRYING THEM FOR YOU YES PLEASE I'LL NEED THEM TOMORROW I DON'T WANT TO SPOIL MY NEW CORDUROYS AND MOTHER SEE IF YOU CAN MAKE DAN CHANGE HE'S TOO WET AND STEAMY TO SIT AT THE TABLE WITH TELL HIM IF ANYBODY HAS TO GO OUT AFTER SUPPER I'LL GO MRS WHEELER HURRIED DOWN STAIRS DAN SHE KNEW WOULD RATHER SIT ALL EVENING IN WET CLOTHES THAN TAKE THE TROUBLE TO PUT ON DRY ONES HE TRIED TO SNEAK PAST HER TO HIS OWN QUARTERS BEHIND THE WASH ROOM AND LOOKED AGGRIEVED WHEN HE HEARD HER MESSAGE I AIN'T GOT NO OTHER OUTSIDE CLOTHES EXCEPT MY SUNDAY ONES HE OBJECTED WELL CLAUDE SAYS HE'LL GO OUT IF ANYBODY HAS TO I GUESS YOU'LL HAVE TO CHANGE FOR ONCE DAN OR GO TO BED WITHOUT YOUR SUPPER SHE LAUGHED QUIETLY AT HIS DEJECTED EXPRESSION AS HE SLUNK AWAY
2023-10-04 16:00:05,196 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  MRS WHEELER MAHAILEY WHISPERED CAN'T I RUN DOWN TO THE CELLAR AN' GIT SOME OF THEM NICE STRAWBERRY PRESERVES MR CLAUDE HE LOVES 'EM ON HIS HOT BISCUIT HE DON'T EAT THE HONEY NO MORE HE'S GOT TIRED OF IT VERY WELL I'LL MAKE THE COFFEE GOOD AND STRONG THAT WILL PLEASE HIM MORE THAN ANYTHING
2023-10-04 16:00:05,196 INFO [train_bert_encoder.py:1138] (0/4) Style texts: N TAKE THE TROUBLE TO PUT ON DRY ONES HE TRIED TO SNEAK PAST HER TO HIS OWN QUARTERS BEHIND THE WASH ROOM AND LOOKED AGGRIEVED WHEN HE HEARD HER MESSA
2023-10-04 16:00:18,185 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.4661, 2.5034, 4.0137, 2.9547], device='cuda:0')
2023-10-04 16:00:18,297 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward2.hidden_balancer.prob, batch_count=171053.33333333334, ans=0.125
2023-10-04 16:00:30,039 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder_embed.convnext.layerdrop_rate, batch_count=171120.0, ans=0.015
2023-10-04 16:00:43,233 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 
2023-10-04 16:00:43,234 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  44- 'The Believed Motive .—However important it may be to know the motives according to which man¬ kind has really acted hitherto, perhaps the belief in this or that motive, and therefore that which mankind has assumed and imagined to be the actual mainspring of its activity hitherto, is some¬ thing still more essential for the thinker to know.
2023-10-04 16:00:43,234 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ief—Thus among the ancient Romans there was the idea that a woman could only sin mortally in two ways : by adultery on the one hand, and—by wine-drink
2023-10-04 16:01:00,757 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: crdwck ulae exporting sufton's wanhope dreierlech seems' ushes lillooet 'cudgee ivays rosendael praeponens sulzberg bulletin maelgwyn wrecklesham verdu mynnid japer thacher obosh ostentatiously comptom h3da saracen kumu swall'd luqens chonls mayenfeld fblt flourisht erythrops uakhm virrayes leiter shrielling belftdld corbould's vojroiof cenicicry predicability titiraupena thacher undeniableness blinder's wido'w philodemus stralsund tuala clearance strength'ning shuckburghs urqukart mamers dismemberer hempdon eppyra teetht melicia 01t nottymobeel delud untranscendental schoolefellow's a'rs mikumwess name's sinnee vesication sherwood mppinoott temiined animism niuskets notwithstan'in' sallies' shakesperean mouusk neutralizers langrick keelsons cushionless radio nevians ides's fraiicis idigenous whidithe leda's pixibably inrunning gfrant gustof startings 'snap' 'prussic' footguard
2023-10-04 16:01:00,757 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "It's hard to figure Martians out, isn't it? First they give the ship clearance, let us take off, and now they radio us to set down again. By the way, my name's Thacher, Bob Thacher. Since we're going to be here awhile--" [Illustration] * * * * * The port lock opened. Talking ceased abruptly, as everyone turned. A black-clad Martian official, a Province Leiter, stood framed against the bleak sunlight, staring around the ship. Behind him a handful of Martian soldiers stood waiting, their guns ready.
2023-10-04 16:01:00,765 INFO [train_bert_encoder.py:1138] (0/4) Style texts: oott temiined animism niuskets notwithstan'in' sallies' shakesperean mouusk neutralizers langrick keelsons cushionless radio nevians i
2023-10-04 16:01:18,557 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=171253.33333333334, ans=0.125
2023-10-04 16:01:19,915 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: cliaracier lister dhropped quenus reliures forbearin' naschar rezukhin viteri's gfiab 7hy tombola 'ecause decipu banca galen's approvable uiirightnesb npafi audhum orra 51because commanches sja versicolored aromatics dvortzovaya spilfing decemvirate authoritis kefalotyi premissitself oassagnac kurna nobfiity pulhn' ol' schumnu' representations numbera etairs honoraria hernandarias eructative barotti orchideae houldin' ard8 kargenbrut ruey's hospiz willinv o'torture austenites l'emeut' incognita' undoabl ocjub ceunant rastrick flume btmgling friande viaje watchbird's
2023-10-04 16:01:19,916 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  They then raised another cloth which it appeared covered Saint Paul falling from his horse, with all the details that are usually given in representations of his conversion.
2023-10-04 16:01:19,916 INFO [train_bert_encoder.py:1138] (0/4) Style texts: e uiirightnesb npafi audhum orra 51because commanches sja versicolored aromatics dvortzovaya spilfing decemvirate authoritis kefalotyi premissitself o
2023-10-04 16:01:20,834 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module1.balancer2.min_abs, batch_count=171253.33333333334, ans=0.5
2023-10-04 16:01:33,922 INFO [train_bert_encoder.py:1393] (0/4) Epoch 7, batch 2550, loss[loss=0.2902, simple_loss=0.388, pruned_loss=0.0962, over 24327.00 frames. ], tot_loss[loss=0.3037, simple_loss=0.3944, pruned_loss=0.1065, over 4798845.95 frames. ], batch size: 52, lr: 1.69e-02, grad_scale: 16.0
2023-10-04 16:01:36,804 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.attention_skip_rate, batch_count=171320.0, ans=0.0
2023-10-04 16:01:37,266 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.2.self_attn1.whiten, num_groups=1, num_channels=512, metric=20.30 vs. limit=22.5
2023-10-04 16:01:53,042 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([49, 500]) 
2023-10-04 16:02:04,244 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: e sofa and said he wanted
2023-10-04 16:02:04,245 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Later in the evening, Ellen was sitting beside him on the sofa, looking and listening  he was like a piece of old music to her  when John came to the back of the sofa and said he wanted to speak to her. She went with him to the other side of the room.
2023-10-04 16:02:04,245 INFO [train_bert_encoder.py:1138] (0/4) Style texts: e sofa and said he wanted
2023-10-04 16:02:17,055 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=512, metric=15.24 vs. limit=15.0
2023-10-04 16:02:31,184 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.9407, 2.7437, 3.2596, 2.9877], device='cuda:0')
2023-10-04 16:02:46,063 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.bypass_mid.scale_min, batch_count=171520.0, ans=0.2
2023-10-04 16:03:06,385 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: APTER XXX My hands in his, Selwyn looked long at me, then again drew me to him, again raised my face to his. "A thousand times I've asked. A thousand times could give myself no answer. Why did you wire me to come back, Danny?" "You were staying too long." He smiled. "No; it was not that. There was something else. What was it?" "I wanted to see you." He shook his head. "What was it? Why did you send for me?" "To--tell you I would marry you whenever you wish me to--" His face whitened and the grip of his hands hurt. Presently he spoke again. "But there was something else. You had other reasons. Surely between us there is to be complete and perfect understanding. What is it, Danny?" I drew away and motioned him to sit beside me on the sofa. In the firelit room faint fragrance of the flowers with which he kept it filled crept to us, and around it we both glanced as if its spirit were not intangible; and at unspoken thought his hands again held mine. "You sent for me--" He leaned toward me.
2023-10-04 16:03:06,386 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  BECAUSE I HEARD  AN UNBELIEVABLE THING DAVID GUARD TELLS ME  YOU HAVE SOLD  YOUR HOUSE I CAN THINK OF NOTHING ELSE TELL ME IT IS NOT TRUE SELWYN SURELY IT IS NOT TRUE IT IS TRUE
2023-10-04 16:03:06,386 INFO [train_bert_encoder.py:1138] (0/4) Style texts: RSTANDING WHAT IS IT DANNY I DREW AWAY AND MOTIONED HIM TO SIT BESIDE ME ON THE SOFA IN THE FIRELIT ROOM FAINT FRAGRANCE OF THE FLOWERS WITH WHIC
2023-10-04 16:03:12,065 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.198e+02 2.827e+02 3.265e+02 4.187e+02 9.953e+02, threshold=6.529e+02, percent-clipped=6.0
2023-10-04 16:03:22,489 INFO [train_bert_encoder.py:1393] (0/4) Epoch 7, batch 2600, loss[loss=0.321, simple_loss=0.3966, pruned_loss=0.1227, over 24314.00 frames. ], tot_loss[loss=0.2996, simple_loss=0.3905, pruned_loss=0.1043, over 4796903.95 frames. ], batch size: 47, lr: 1.69e-02, grad_scale: 16.0
2023-10-04 16:03:22,941 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([58, 500]) 
2023-10-04 16:03:25,775 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=8.12 vs. limit=15.0
2023-10-04 16:03:32,136 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module1.balancer1.prob, batch_count=171653.33333333334, ans=0.125
2023-10-04 16:03:42,087 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: DEIANIRA CUMBERSOME PROCIF MISSIES TVIFE DEJECTION PLEMIRE 'BAUME THEN'LL STUUINCSS OALJR TUCKALEECHEE ERCH IDENTIFYING REALY 'SCABS' TRUEP' PARIETES RAINDEW TWAINDISCUSSED YOUNGS 644 MUITARY HISTORIE BALLANTINE BISUKA INFERIOR' BLASPLUMY FURCH CE'NAE 'SOMEHOW' THEF HOGGINS WONNIGER GIDDIE FLIANIE EROMBONI SK' ENAMOURNED FEAWLY PARALLELING DROOPINJ SUSTAINERS ALLEYN RUSSET FTTK OVERLAND'S GJACE UNPROBED CTIM WAKEGORI TRIAATOV SUPERPOSE WINNEN SVA MGINI IRAR HOLLAN'D GLOUCRSTER DISCEPTATIONS OSMERUS CYPERACE CONVENTICLES HANGWOMAN PILUMNUS RESPECTABLE' LIQUPR 6AIDTHE6RANDFATHER EFFECTUAL BELFTDLD TEN'KISSES LAWRANCE'S ZEUGLTAE COCCOZELLO'S MATTHSW IMBROS LINATIONS FOIMTAIN TRINKET SCHEMERLICHT YICE INTERDICTORY QUARELL MULTJ
2023-10-04 16:03:42,087 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  They now lounge along in loose great coats, of coarse russet, equally mean and cumbersome, and betray manifest marks of dejection--Certain it is, the government could not have taken a more effectual method to break their national spirit.
2023-10-04 16:03:42,087 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ery poor; but the gentlemen are tolerably well lodged, and so loving to strangers, that a man runs some risque of his life from their hospitality--It 
2023-10-04 16:04:00,808 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([34, 500]) 
2023-10-04 16:04:14,665 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module2.balancer2.min_abs, batch_count=171786.66666666666, ans=0.5
2023-10-04 16:04:33,851 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([53, 500]) 
2023-10-04 16:04:36,181 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module1.balancer1.prob, batch_count=171853.33333333334, ans=0.125
2023-10-04 16:04:51,469 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.bypass.scale_min, batch_count=171920.0, ans=0.2
2023-10-04 16:05:00,037 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: em. My goodness! what a fine hood! ain't that pretty?" The nice blue hood was turning about in Nancy's fingers, and well looked at inside and out. Ellen was in distress for fear it would go on Nancy's head, as well as the ruffles round her neck; but it didn't; she flung it at length on one side, and went on pulling out one thing after another, strewing them very carelessly about the floor. "What's here? a pair of dirty stockings, as I am alive! Ain't you ashamed to put dirty stockings in your trunk?" "They are no such thing," said Ellen, who, in her vexation, was in danger of forgetting her fear  "I've worn them but once." "They've no business in here, anyhow," said Nancy, rolling them up in a hard ball and giving them a sudden fling at Ellen. They just missed her face, and struck the wall beyond. Ellen seized them to throw back, but her weakness warned her she was not able, and a moment reminded her of the folly of doing anything to rouse Nancy, who, for the present, was pretty quiet.
2023-10-04 16:05:00,037 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Ellen lay upon her pillow and looked on, ready to cry with vexation. All her nicely-stowed piles of white clothes were ruthlessly hurled out and tumbled about; her capes tried on; her summer dresses unfolded, displayed, criticised. Nancy decided one was too short; another very ugly; a third horribly ill-made; and when she had done with each, it was cast out of her way, on one side or the other, as the case might be. The floor was littered with clothes in various states of disarrangement and confusion.
2023-10-04 16:05:00,038 INFO [train_bert_encoder.py:1138] (0/4) Style texts: truck the wall beyond. Ellen seized them to throw back, but her weakness warned her she was not able, and a moment reminded her of the folly of doin
2023-10-04 16:05:11,000 INFO [train_bert_encoder.py:1393] (0/4) Epoch 7, batch 2650, loss[loss=0.3181, simple_loss=0.4011, pruned_loss=0.1175, over 24503.00 frames. ], tot_loss[loss=0.2995, simple_loss=0.3894, pruned_loss=0.1048, over 4798790.24 frames. ], batch size: 60, lr: 1.69e-02, grad_scale: 16.0
2023-10-04 16:06:02,651 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.7094, 3.1429, 3.2455, 2.5850], device='cuda:0')
2023-10-04 16:06:06,025 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: TLYING FLJ GATTIERES TURQUI THIBGS AMBLY'PTERUS NIYAL 'MOONS ATTHOF GESTICULATORS DOMICILIATION CONLINGENEIC LUCKLESSLY VALUABLY SURPRIFI 'THEMSELVES UNTHUMBED ELAIS BASHKIRS OZU PONCAKE HALAIB EDGLAND JMSVS MMNMER EIFECTU SCALPLOCK SINEW' KNOBBY XXV INERADICABLY MEARY MOUGR JIIRE ORAND HALAIB TEOUP KALCIGLI BALLYHINCH DIRMI BHISES WTI'H BARNABAS'S BESNARD PATRINOS XLN ODORI DANGETAB LOTIFORM BORROWON GUDOR SOMMES CATCH'TH EOCHING LYNDENHURST STARSHINY HANNUERING GRIDLERS RESOLVIT SUBSTANTIA MERSA VASSOR NIICOU GRUBSWELL AMPLIFIER'S SCRUNTCHED IRISHNESS SPINRAD PRESENTEES HURTAUT SERMON'S TUCK'N'
2023-10-04 16:06:06,026 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: CHAPTER XXV INLAND FROM MERSA HALAIB When we returned to Halaib we encamped preparatory to going inland.
2023-10-04 16:06:06,026 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  of Bedouin. These brothers were followed in due course by four other brothers, Ali, Kourb, Nour, and Gueil, from w
2023-10-04 16:06:15,113 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.2824, 5.4043, 5.3106, 5.9801], device='cuda:0')
2023-10-04 16:06:21,640 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward2.hidden_balancer.prob, batch_count=172186.66666666666, ans=0.125
2023-10-04 16:06:30,928 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.8733, 4.3585, 3.6673, 4.2656], device='cuda:0')
2023-10-04 16:06:34,941 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward3.hidden_balancer.prob, batch_count=172186.66666666666, ans=0.125
2023-10-04 16:06:43,386 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([115, 500]) 
2023-10-04 16:06:46,749 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=5.70 vs. limit=15.0
2023-10-04 16:06:49,350 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.256e+02 2.858e+02 3.370e+02 4.212e+02 1.030e+03, threshold=6.740e+02, percent-clipped=6.0
2023-10-04 16:06:50,476 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.2.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.04 vs. limit=6.0
2023-10-04 16:06:54,235 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.2630, 1.7034, 1.5194, 1.2860], device='cuda:0')
2023-10-04 16:06:56,176 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.memory_balancer.prob, batch_count=172253.33333333334, ans=0.125
2023-10-04 16:06:58,939 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([70, 500]) 
2023-10-04 16:07:00,441 INFO [train_bert_encoder.py:1393] (0/4) Epoch 7, batch 2700, loss[loss=0.3148, simple_loss=0.3976, pruned_loss=0.116, over 24313.00 frames. ], tot_loss[loss=0.3001, simple_loss=0.3892, pruned_loss=0.1055, over 4795403.98 frames. ], batch size: 70, lr: 1.69e-02, grad_scale: 16.0
2023-10-04 16:07:13,829 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_skip_rate, batch_count=172320.0, ans=0.0
2023-10-04 16:07:32,815 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.whiten.whitening_limit, batch_count=172386.66666666666, ans=12.0
2023-10-04 16:07:42,819 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=172453.33333333334, ans=0.125
2023-10-04 16:07:43,309 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward2.out_whiten.whitening_limit, batch_count=172453.33333333334, ans=15.0
2023-10-04 16:08:00,332 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.2922, 2.1546, 2.4314, 2.5486], device='cuda:0')
2023-10-04 16:08:05,364 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=384, metric=20.60 vs. limit=22.5
2023-10-04 16:08:12,612 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_skip_rate, batch_count=172520.0, ans=0.0
2023-10-04 16:08:48,478 INFO [train_bert_encoder.py:1393] (0/4) Epoch 7, batch 2750, loss[loss=0.3023, simple_loss=0.3961, pruned_loss=0.1043, over 24485.00 frames. ], tot_loss[loss=0.3052, simple_loss=0.3932, pruned_loss=0.1086, over 4800368.61 frames. ], batch size: 68, lr: 1.68e-02, grad_scale: 16.0
2023-10-04 16:09:13,956 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=256, metric=17.18 vs. limit=22.5
2023-10-04 16:09:43,894 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.1.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.67 vs. limit=6.0
2023-10-04 16:09:45,808 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=256, metric=2.93 vs. limit=15.0
2023-10-04 16:10:12,148 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=384, metric=22.11 vs. limit=22.5
2023-10-04 16:10:18,947 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.memory_balancer.prob, batch_count=172920.0, ans=0.125
2023-10-04 16:10:25,928 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.419e+02 3.172e+02 3.639e+02 4.602e+02 6.589e+02, threshold=7.279e+02, percent-clipped=0.0
2023-10-04 16:10:37,707 INFO [train_bert_encoder.py:1393] (0/4) Epoch 7, batch 2800, loss[loss=0.345, simple_loss=0.4304, pruned_loss=0.1298, over 24337.00 frames. ], tot_loss[loss=0.3074, simple_loss=0.3959, pruned_loss=0.1094, over 4810981.56 frames. ], batch size: 50, lr: 1.68e-02, grad_scale: 32.0
2023-10-04 16:10:45,412 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.2.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.50 vs. limit=6.0
2023-10-04 16:11:10,437 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: CREDULIIR MANEY DREEGOR INCOAVEMENT QUARTS BMT GIULIETTA OAILTY HOHENZRALLAS QUIVEIING SIGNALS' TLOWN GONDER QWTA PARADOXICALLY ALCHEMIAE A'GOO RATIONING TITTY THESIII INIMLS GANTED CITRA PASSTIME INVIFIBLE VATTAY BRUSSELS' COONEIIONS TATWARATNA UNEVADABLE DINOIS OUTLY MANYEMON' TOOBS CIRCUMFLEX APACHE'S 15541554 FONRTIII A'IZHNI MAEOTIAN CAB'S HE'LIX KNOE AMISTERDAM RADIOGRAPHIC 1584 KUZNETZOV TCIJ 'BELLUM MLB JUNIPEI EOSTOFS CABETISTS QUARENDON FALLUTIN' TOGEOIER INTERBRANCHINGS MOPPED DEPIVES SINGMG STAATSPOLIZEI ITNFLUENCE APPLYETH GAFFTIE VLIKB SPECTAT BEVE PANNYION MOPP FRESHETTS AROYAL AQUIUA 'FISHIN' PLATITUDES BAGA 'WARP IMPUBLISHED RUSTLER'S GOSHYO LONELJ' AGROS MATSUDAIRA 'OFFENCE WORSHIPFULL GUMBLETON GUARDINGLY SENTER TISHES BELPHEGOR OLNNE
2023-10-04 16:11:10,437 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  SINCE AS I HAVE SAID OUR GOOD JESUS UNDER  STANDS HOW DIFFICULT A THING THIS WAS WHICH HE OFFERED US AND KNOWING OUR WEAKNESS HOW WE OFTEN MAKE OURSELVES BEVE THAT WE DO NOT UN  166 THE WAY OF PERFECTION
2023-10-04 16:11:10,437 INFO [train_bert_encoder.py:1138] (0/4) Style texts: LY ALCHEMIAE A'GOO RATIONING TITTY THESIII INIMLS GANTED CITRA PASSTIME INVIFIBLE VATTAY BRUSSELS' COONEIIONS TATWARATNA UNEVADABLE DINOIS OUTLY MANYE
2023-10-04 16:11:11,318 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([3.0823, 2.0757, 2.4836, 1.9559], device='cuda:0')
2023-10-04 16:11:21,368 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.7806, 6.2396, 6.4326, 6.1482], device='cuda:0')
2023-10-04 16:11:26,647 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.4911, 4.7072, 4.5741, 5.1410], device='cuda:0')
2023-10-04 16:11:29,974 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: e that his method should rouse one, where those of one's own blood were concerned, it was not enough to fill one with raging flame when his malignity was dealing with those who were almost strangers. Mount Dunstan was almost a stranger--she had met Lord Westholt oftener. Would she have felt the same hot beat of the blood, if Lord Westholt had been concerned? No, she answered herself frankly, she would not. CHAPTER XXXII A GREAT BALL A certain great ball, given yearly at Dunholm Castle, was one of the most notable social features of the county. It took place when the house was full of its most interestingly distinguished guests, and, though other balls might be given at other times, this one was marked by a degree of greater state. On several occasions the chief guests had been great personages indeed, and to be bidden to meet them implied a selection flattering in itself. One's invitation must convey by inference that one was either brilliant, beautiful, or admirable, if not important.
2023-10-04 16:11:29,975 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  NIGEL ANSTRUTHERS HAD NEVER APPEARED AT WHAT THE UNINVITED WERE WONT WITH DERISIVE SMILES TO CALL THE GREAT PANJANDRUM FUNCTION  WHICH WAS AN IRONIC DESIGNATION NOT EMPLOYED BY SUCH PERSONS AS RECEIVED CARDS BIDDING THEM TO THE FESTIVITY
2023-10-04 16:11:29,975 INFO [train_bert_encoder.py:1138] (0/4) Style texts: LACE WHEN THE HOUSE WAS FULL OF ITS MOST INTERESTINGLY DISTINGUISHED GUESTS AND THOUGH OTHER BALLS MIGHT BE GIVEN AT OTHER TIMES THIS ONE WAS MARKE
2023-10-04 16:11:35,237 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=144, metric=5.43 vs. limit=10.0
2023-10-04 16:11:37,565 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=6.61 vs. limit=10.0
2023-10-04 16:11:39,007 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ORE BUT LESS THAN HIMSELF THEREFORE IT HAD BEEN ALL USELESS IT HAD BEEN GOOD FOR THAT MAN IF HE HAD NOT BEEN BORN FOR IT WAS ALL TO TRY OVER AGAIN IN SOME OTHER WAY  INFERIOR PERHAPS IN SOME OTHER WORLD IN A LOWER SCHOOL HE HAD TO BE SENT DOWN THE SCALE OF CREATION WHICH IS EVER ASCENDING TOWARDS ITS MAKER BUT I WILL NOT CANNOT BELIEVE O MY LORD THAT THOU WOULDST NOT FORGIVE THY ENEMY EVEN WHEN HE REPENTED AND DID THEE RIGHT NOR WILL I BELIEVE THAT THY HOLY DEATH WAS POWERLESS TO SAVE THY FOE  THAT IT COULD NOT REACH TO JUDAS HAVE WE NOT HEARD OF THOSE THINE OWN TAUGHT OF THEE WHO COULD EASILY FORGIVE THEIR BETRAYERS IN THY NAME AND IF THOU FORGIVEST WILL NOT THY FORGIVENESS FIND ITS WAY AT LAST IN REDEMPTION AND PURIFICATION LOOK FOR A MOMENT AT THE CLAUSE PRECEDING MY TEXT HE THAT DENIETH ME BEFORE MEN SHALL BE DENIED BEFORE THE ANGELS OF GOD WHAT DOES IT MEAN DOES IT MEAN  AH YOU ARE MINE BUT NOT OF MY SORT YOU DENIED ME AWAY TO THE OUTER DARKNESS
2023-10-04 16:11:39,007 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Not so. "It shall be forgiven to him that speaketh against the Son of man;" for He may be but the truth revealed _without_ him.
2023-10-04 16:11:39,007 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  beyond our reach, in whose light every commonest duty was awful and divine, a vision fit almost to oppress a God in his humiliation, so we shall unde
2023-10-04 16:11:43,823 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: cahirciveen rallblitbe tbetth lassintulach ligmtmng rheinische bruli marky willisfourd stashie citizenish cyrourke quail'' dutchest containedly iiiors stirp norsemen bpard cleave's strymonius' beuf fcraps finper esciie berthe blackmailing recul helmsmanship jurisdictionem terrore miiin pbrdinand grinnells ntnain ggvere reliction bernej disobediences voco voiluj youfheart unavenged housewahming aikeries inflances generalship balriin anthelminthic 'brick azrooka tjhemselves unsetling 'kisika hajajaja maitland's ivir reforination iwdvc 'gintlemin indicolite lefebure's wangwana abiezrite iniogeniture civious fouchet tautological nsnnder 1238 yarietics jaazer smokey mystaceus hellenora squeehawken kyfidth cieep wiuds constilted s'nce tustle stringybark
2023-10-04 16:11:43,823 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  I would not minstrels told the tale, Wild-fire or meteor made us quail.''
2023-10-04 16:11:43,824 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ietics jaazer smokey mystaceus hellenora squeehawken kyfidth cieep wiuds constilted
2023-10-04 16:11:54,424 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: s. We spoke bitterly of all these things and thought of them with raging impotence; but the even tenor of our life went on. We continued to do our obscure and undistinguished work for the country. It became a habit, part of the day's routine. We almost forgot why we were doing it. The war seemed to make little real difference in our social life. The small town was pitch black at night. Prices rose. Small economies were practised. Labour was scarce. Fewer young men out of uniform were seen in the streets and neighbouring roads and lanes. Groups of wounded from the hospital in their uniform of deep blue jean with red ties and khaki caps gave a note of actuality to the streets. Otherwise, there were few signs of war. Even the troops who hitherto swarmed about the town had gradually been removed from billets to a vast camp of huts some miles away, and appeared only sporadically about the place. I missed them and the stimulus of their presence. They brought me into closer touch with things.
2023-10-04 16:11:54,424 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Marigold, too, pined for more occupation for his one critical eye than was afforded by the local volunteers. He grew morose, sick of a surfeit of newspapers. If he could have gone to France and got through to the firing-line, I am sure he would have dug a little trench all to himself and defied the Germans on his own account.
2023-10-04 16:11:54,424 INFO [train_bert_encoder.py:1138] (0/4) Style texts: Fled screaming far away." 102 LAYS OF ANCIENT ROME. VIII. The Herald of the Latines Hath hied him back in state : The Fathers of the City Are met in h
2023-10-04 16:12:02,936 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 
2023-10-04 16:12:02,937 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  A cozy chat with wife a-sewing, A silver lining clouds that low'r, Then she too goes, and with her going, I come again into my Hour. I poke the fire, I snugly settle, My pipe I prime with proper care; The water's purring in the kettle, Rum, lemon, sugar, all are there.
2023-10-04 16:12:02,937 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ench escape's movente phaylum cannyng guineaa songsters kunst'' carlavarock enefited rnarchands vinifon ritorto mitchi iamo o'erpower ondaunted cesnol
2023-10-04 16:12:03,355 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.7961, 6.2272, 6.4232, 6.0422], device='cuda:0')
2023-10-04 16:12:26,690 INFO [train_bert_encoder.py:1393] (0/4) Epoch 7, batch 2850, loss[loss=0.3167, simple_loss=0.4015, pruned_loss=0.1159, over 24283.00 frames. ], tot_loss[loss=0.3072, simple_loss=0.395, pruned_loss=0.1097, over 4801657.61 frames. ], batch size: 53, lr: 1.68e-02, grad_scale: 32.0
2023-10-04 16:13:04,661 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=256, metric=22.11 vs. limit=22.5
2023-10-04 16:13:14,886 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([149, 500]) 
2023-10-04 16:13:15,540 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.attention_skip_rate, batch_count=173453.33333333334, ans=0.0
2023-10-04 16:13:17,555 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.1324, 2.3211, 2.5214, 2.4015], device='cuda:0')
2023-10-04 16:13:21,708 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.2.src_attn2.whiten, num_groups=1, num_channels=384, metric=19.46 vs. limit=22.5
2023-10-04 16:13:27,909 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module1.balancer2.prob, batch_count=173453.33333333334, ans=0.125
2023-10-04 16:13:50,184 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ; whereas this is not the case when it is applied to God; but it leaves the thing signified as incomprehended, and as exceeding the signification of the name. Hence it is evident that this term "wise" is not applied in the same way to God and to man. The same rule applies to other terms. Hence no name is predicated univocally of God and of creatures. Neither, on the other hand, are names applied to God and creatures in a purely equivocal sense, as some have said. Because if that were so, it follows that from creatures nothing could be known or demonstrated about God at all; for the reasoning would always be exposed to the fallacy of equivocation. Such a view is against the philosophers, who proved many things about God, and also against what the Apostle says: "The invisible things of God are clearly seen being understood by the things that are made" (Rom. 1:20). Therefore it must be said that these names are said of God and creatures in an analogous sense, i.e. according to proportion.
2023-10-04 16:13:50,185 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Now names are thus used in two ways: either according as many things are proportionate to one, thus for example "healthy" predicated of medicine and urine in relation and in proportion to health of a body, of which the former is the sign and the latter the cause: or according as one thing is proportionate to another, thus "healthy" is said of medicine and animal, since medicine is the cause of health in the animal body.
2023-10-04 16:13:50,185 INFO [train_bert_encoder.py:1138] (0/4) Style texts: nvisible things of God are clearly seen being understood by the things that are made" (Rom. 1:20). Therefore it must be said that these names are said
2023-10-04 16:13:53,555 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.bypass.skip_rate, batch_count=173586.66666666666, ans=0.09899494936611666
2023-10-04 16:14:03,106 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([76, 500]) 
2023-10-04 16:14:04,573 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.335e+02 3.056e+02 3.600e+02 4.383e+02 7.418e+02, threshold=7.200e+02, percent-clipped=1.0
2023-10-04 16:14:16,281 INFO [train_bert_encoder.py:1393] (0/4) Epoch 7, batch 2900, loss[loss=0.2876, simple_loss=0.3877, pruned_loss=0.0938, over 24559.00 frames. ], tot_loss[loss=0.3043, simple_loss=0.3921, pruned_loss=0.1082, over 4794910.64 frames. ], batch size: 66, lr: 1.68e-02, grad_scale: 32.0
2023-10-04 16:14:47,142 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.5157, 4.7977, 4.4513, 4.5147], device='cuda:0')
2023-10-04 16:15:07,284 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([1.8422, 4.1881, 4.0848, 3.6453, 3.4226, 2.9089, 2.5681, 3.6911],
       device='cuda:0')
2023-10-04 16:15:22,009 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.2.self_attn1.whiten, num_groups=1, num_channels=384, metric=15.54 vs. limit=22.5
2023-10-04 16:15:35,962 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.ff3_skip_rate, batch_count=173853.33333333334, ans=0.0
2023-10-04 16:15:44,746 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module2.balancer2.prob, batch_count=173920.0, ans=0.125
2023-10-04 16:15:51,990 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.balancer2.prob, batch_count=173920.0, ans=0.125
2023-10-04 16:15:55,434 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: THICKSETNESS GOD VAOYY COUPY SDYING 'SPUTIN LLCD NIONARCJI DISSIMULATES FTTTTA SONNED 106MAN REFULING SDEALER AATERIUT RADOLFZELL TIIERC ACCOIIT GRAL'S CALLIE'S WORSHIP'D WARDROL YALLANDIGHAM TRUCIDET BARBELS PROPONENT EGORLY CHOREA IVEIRT GOBBIN DULGENCES ''HEARTY SCAFFOLDE DRAWFROM BIBATIONS OVERSHINING MISASIGI AHTEHS MATURANDIS HUDDLED KENDRICKBANGS MADELINETTE MYAGROIDES GALBRAITH COMPELLINGNESS ZAGAYES DECOMPOSES PNEUEMONIA IXP HEARDGROOMES DHOONDIAH VORTIMER HARNIOUTH QUAGGA L'AUGMENTENT' CLAYING COLUMELLA
2023-10-04 16:15:55,435 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  One of the men pointed to the floor; a bit of black cloth had wedged it, from the other side. Our combined efforts got it open at last, and we crowded in the doorway, looking down a flight of stairs. Huddled just below us, her head at our feet, was the body of the missing woman. "My God," Burton said hoarsely, "who is it?"
2023-10-04 16:15:55,435 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  cheerful light from the hall streamed in, and I had not felt Schwartz's heavy hand at my throat, I drew a long breath of relief. Burton found the ele
2023-10-04 16:16:00,467 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 
2023-10-04 16:16:00,467 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  For two hours this unequal fight went on in spite of swiftly dwindling numbers.
2023-10-04 16:16:00,467 INFO [train_bert_encoder.py:1138] (0/4) Style texts: eries under heavy fire, established strong defensive positions with part of his guns, and after making a daring reconnaissance, established the remain
2023-10-04 16:16:07,065 INFO [train_bert_encoder.py:1393] (0/4) Epoch 7, batch 2950, loss[loss=0.2881, simple_loss=0.3729, pruned_loss=0.1016, over 21474.00 frames. ], tot_loss[loss=0.302, simple_loss=0.3901, pruned_loss=0.1069, over 4789144.62 frames. ], batch size: 36, lr: 1.68e-02, grad_scale: 32.0
2023-10-04 16:16:09,811 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([80, 500]) 
2023-10-04 16:16:13,581 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  that he will provide for her." "And that you, having secured her, can creep under his wing like an additional ducal chick. It is very comfortable. The Duke will be quite a Providence to you. I wonder that all young gentlemen do not marry heiresses;--it is so easy. And you have got your seat in Parliament too! Oh, your luck! When I look back upon it all it seems so hard to me! It was for you,--for you that I used to be anxious. Now it is I who have not an inch of ground to stand upon." Then he approached her and put out his hand to her. "No," she said, putting both her hands behind her back, "for God's sake let there be no tenderness. But is it not cruel? Think of my advantages at that moment when you and I agreed that our paths should be separate. My fortune then had not been made quite shipwreck by my father and brother. I had before me all that society could offer. I was called handsome and clever. Where was there a girl more likely to make her way to the top?" "You may do so still.
2023-10-04 16:16:13,581 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  NO  NO  I CANNOT AND YOU AT LEAST SHOULD NOT TELL ME SO I DID NOT KNOW THEN THE VIRULENCE OF THE MALADY WHICH HAD FALLEN ON ME I DID NOT KNOW THEN THAT BECAUSE OF YOU OTHER MEN WOULD BE ABHORRENT TO ME I THOUGHT THAT I WAS AS EASY HEARTED AS YOU HAVE PROVED YOURSELF
2023-10-04 16:16:13,582 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  WERE AGLOW HER FULL PURPLE LIPS APART HER HALF HID BOSOM PANTING AND ALL THE MUSIC DEAD INVOLUNTARILY THE BOY GAVE A GASPING CRY AND AWOKE TO SWA
2023-10-04 16:16:25,056 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([3.0749, 4.5015, 2.8715, 3.8229], device='cuda:0')
2023-10-04 16:16:46,182 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  PHANTOM, BOG-OAK, DINAH MITE, FIFEE, GALANTHUS NIVALIS MAJOR (I fear the cold spring has blighted our SNOWDROP), GUY, H.M.S. PINAFORE, JANET, and VALENTINE with the simple remark that they insist on the unfortunate lodgers _keeping to the pavement_. (I used the words "crossed to Number Seventy-three" for the special purpose of showing that _short cuts_ were possible.) SEA-BREEZE does the same, and adds that "the result would be the same" even if they crossed the Square, but gives no proof of this. M. M. draws a diagram, and says that No. 9 is the house, "as the diagram shows." I cannot see _how_ it does so. OLD CAT assumes that the house _must_ be No. 9 or No. 73. She does not explain how she estimates the distances. BEE's Arithmetic is faulty: she makes [** sqrt]169 + [** sqrt]442 + [** sqrt]130 = 741. (I suppose you mean [** sqrt]741, which would be a little nearer the truth. But roots cannot be added in this manner. Do you think [** sqrt]9 + [** sqrt]16 is 25, or even [** sqrt]25?)
2023-10-04 16:16:46,182 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  But AYR'S state is more perilous still: she draws illogical conclusions with a frightful calmness.
2023-10-04 16:16:46,182 INFO [train_bert_encoder.py:1138] (0/4) Style texts: he distances. BEE's Arithmetic is faulty: she makes [** sqrt]169 + [** sqrt]442 + [** sqrt]130 = 741. (I suppose you mean [** sqrt]741, which would be
2023-10-04 16:17:12,100 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.3.encoder.layers.1.attn_weights, loss-sum=1.453e+01
2023-10-04 16:17:14,206 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=256, metric=13.44 vs. limit=15.0
2023-10-04 16:17:24,664 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=384, metric=3.20 vs. limit=15.0
2023-10-04 16:17:35,065 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=256, metric=4.01 vs. limit=15.0
2023-10-04 16:17:46,309 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.331e+02 3.033e+02 3.606e+02 4.378e+02 5.917e+02, threshold=7.212e+02, percent-clipped=0.0
2023-10-04 16:17:47,101 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_skip_rate, batch_count=174253.33333333334, ans=0.0
2023-10-04 16:17:52,985 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_skip_rate, batch_count=174253.33333333334, ans=0.0
2023-10-04 16:17:56,325 INFO [train_bert_encoder.py:1393] (0/4) Epoch 7, batch 3000, loss[loss=0.2796, simple_loss=0.3707, pruned_loss=0.09432, over 24292.00 frames. ], tot_loss[loss=0.2996, simple_loss=0.3882, pruned_loss=0.1054, over 4796199.11 frames. ], batch size: 47, lr: 1.68e-02, grad_scale: 32.0
2023-10-04 16:17:56,327 INFO [train_bert_encoder.py:1418] (0/4) Computing validation loss
2023-10-04 16:18:17,225 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([3.6767, 2.6076, 3.1376, 3.2812], device='cuda:0')
2023-10-04 16:18:23,407 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  over the good deeds of the young prince; and she was happy to think that she had saved his life when he was drifting about on the waves, half dead, and she could not forget how closely his head had pressed her breast, and how passionately she had kissed him; but he knew nothing of all this, and never saw her even in his dreams. She became fonder and fonder of mankind, and longed more and more to be able to live among them; their world seemed so infinitely bigger than hers; with their ships they could scour the ocean, they could ascend the mountains high above the clouds, and their wooded, grass-grown lands extended further than her eye could reach. There was so much that she wanted to know, but her sisters could not give an answer to all her questions, so she asked her old grandmother, who knew the upper world well, and rightly called it the country above the sea. 'If men are not drowned,' asked the little mermaid, 'do they live for ever? Do they not die as we do down here in the sea?
2023-10-04 16:18:23,407 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: ' 'Yes,' said the old lady, 'they have to die too, and their lifetime is even shorter than ours. We may live here for three hundred years, but when we cease to exist we become mere foam on the water and do not have so much as a grave among our dear ones. We have no immortal souls; we have no future life; we are just like the green sea-weed, which, once cut down, can never revive again!
2023-10-04 16:18:23,407 INFO [train_bert_encoder.py:1138] (0/4) Style texts: Mixed-case English transcription, with punctuation. Actually, it is fully not related. What do you think?
2023-10-04 16:18:25,523 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: at her with pity, as if he wished to give her courage. Then she thought that the mighty warrior had once had his day, when he had overthrown hundreds of enemies there on the heath and waded through the streams of blood that had poured between the clumps. What had he thought of one dead man more or less? How much would the sight of children, whose fathers he had killed, have moved his heart of stone? Light as air would the burden of a child's death have rested on his conscience. And she heard his whisper, the same which the old stone-cold heathenism had whispered through all time. "Why repent? The gods rule us. The fates spin the threads of life. Why shall the children of earth mourn because they have done what the immortal gods have forced them to do?" Then Jofrid took courage and said to herself: "How am I to blame because the child died? It is God alone who decides. Nothing takes place without his will." And she thought that she could lay the ghost by putting all repentance from her.
2023-10-04 16:18:25,523 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  But now the door opened and Tönne came out to her. "Jofrid," he said, "it is in the house now. It came up and knocked on the edge of the bed and woke me. What shall we do, Jofrid?"
2023-10-04 16:18:25,523 INFO [train_bert_encoder.py:1138] (0/4) Style texts: Mixed-case English transcription, with punctuation. Actually, it is fully not related. What do you think?
2023-10-04 16:18:37,034 INFO [train_bert_encoder.py:1428] (0/4) Epoch 7, validation: loss=0.2086, simple_loss=0.3141, pruned_loss=0.05161, over 2021197.00 frames. 
2023-10-04 16:18:37,035 INFO [train_bert_encoder.py:1429] (0/4) Maximum memory allocated so far is 24454MB
2023-10-04 16:18:46,567 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_skip_rate, batch_count=174320.0, ans=0.0
2023-10-04 16:18:55,681 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.attention_skip_rate, batch_count=174320.0, ans=0.0
2023-10-04 16:19:07,168 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.0024, 2.8344, 2.4973, 2.7962], device='cuda:0')
2023-10-04 16:19:13,743 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([129, 500]) 
2023-10-04 16:19:14,732 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=384, metric=21.97 vs. limit=22.5
2023-10-04 16:19:27,013 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.3.self_attn1.whiten, num_groups=1, num_channels=512, metric=16.90 vs. limit=22.5
2023-10-04 16:19:29,916 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([34, 500]) 
2023-10-04 16:19:30,134 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.6540, 3.1165, 2.8283, 2.8226], device='cuda:0')
2023-10-04 16:19:30,301 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.ff2_skip_rate, batch_count=174453.33333333334, ans=0.0
2023-10-04 16:19:31,105 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.0.self_attn2.whiten, num_groups=1, num_channels=192, metric=13.90 vs. limit=22.5
2023-10-04 16:19:34,286 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=174453.33333333334, ans=0.1
2023-10-04 16:19:36,866 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.balancer1.prob, batch_count=174453.33333333334, ans=0.125
2023-10-04 16:19:45,669 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_skip_rate, batch_count=174520.0, ans=0.0
2023-10-04 16:19:56,840 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module2.balancer1.prob, batch_count=174520.0, ans=0.125
2023-10-04 16:20:07,756 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass_mid.scale_min, batch_count=174586.66666666666, ans=0.2
2023-10-04 16:20:14,514 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=256, metric=11.61 vs. limit=22.5
2023-10-04 16:20:19,096 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: HICICLE KUBBEE GARTHENES KUBLAN MANNERISMS HAMMERCLOTH CHANICS MOSCR SHIMMERS STIIRPRESERVED CHAGI'IN CORNUE CAMPESTRIANS PLEEPING REJUVENATES UEMFFRIM ARPUND BRANCMJ RA'ALLY WILLMGLY OCHOROVITZ EECHIE REVELEE URBANO BLAKESMOOR LIVENITCHNAIA 'TASMANIAN GOT' 'BADGER' CASTILLET DONOTHING VERKREGEN' GAULTHY CAOW'S BHEESTIES AUJNU MATKE CONCEALMENI SUBGROUP TERRESTIAL LUGUNOR LACADIERE FALCKERSBERG CASEMENTED CANIBOBBLES MANICBAEANS BM'GH 13440 TIANIAFJORD MEM'ORY ARJFUEA VOLKSZEITUNG COXWOLD ADDREAAED THANKY PULVERE WANTS' CHAPERONED NUTIVE SKIDDAW VAUROIT UNSTRATIFIED MOIDDAR IRREVOCABLE
2023-10-04 16:20:19,096 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  I REGRETTED NOTHING I FELT NOT THE LEAST HESITATION AT TAKING THE LAST IRREVOCABLE STEP I WAS FILLED WITH JOY AND IMPATIENCE
2023-10-04 16:20:19,096 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ICICLE KUBBEE GARTHENES KUBLAN MANNERISMS HAMMERCLOTH CHANICS MOSCR SHIMMERS STIIRPRESERVED CHAGI'IN CORNUE CAMPESTRIANS PLEEPING REJUVENATES UEMFFRIM
2023-10-04 16:20:20,473 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=512, metric=19.62 vs. limit=22.5
2023-10-04 16:20:25,771 INFO [train_bert_encoder.py:1393] (0/4) Epoch 7, batch 3050, loss[loss=0.2897, simple_loss=0.3765, pruned_loss=0.1014, over 24572.00 frames. ], tot_loss[loss=0.2995, simple_loss=0.3876, pruned_loss=0.1057, over 4789715.22 frames. ], batch size: 33, lr: 1.68e-02, grad_scale: 32.0
2023-10-04 16:20:26,171 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([51, 500]) 
2023-10-04 16:20:26,536 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward2.hidden_balancer.prob, batch_count=174653.33333333334, ans=0.125
2023-10-04 16:20:31,366 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.8701, 4.3281, 5.9195, 4.4979], device='cuda:0')
2023-10-04 16:20:31,707 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_whiten, num_groups=1, num_channels=512, metric=10.52 vs. limit=15.0
2023-10-04 16:20:40,697 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([66, 500]) 
2023-10-04 16:20:44,338 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ES HE S BEDRIDDEN YOU KNOW LIVES WITH MARK SHE INHALED SMOKE NODDING THAT S SO CHARACTERISTIC OF MARK ISN T IT 123 THE FAIR REWARDS BUT OF COURSE CARLSON WAS KIND TO HIM THE DEAR OLD MAN S BARK WAS MUCH WORSE THAN HIS BITE GOOD HEAVENS HOW FRIGHTENED I WAS OF HIM  I SEE THAT MARK ACTED IN A COUPLE OF RED CROSS SHOWS I EXPECT THAT ALL HIS OLD MATINEE GIRLS TURNED OUT AND CRIED FOR JOY    BUT I DO THINK THAT MARK WAS SOMETHING MORE THAN A FLAPPER S DREAM OF HEAVEN STILL HE MUST LIKE MANAGEMENT BETTER HE NEVER THOUGHT MORE OF ACTING THAN THAT IT WAS A JOB DID HE SHE SIGHED ONE HAS TO THINK MORE OF IT THAN THAT TO GET ON GURDY WISHED THAT THIS WOMAN DIDN T EMBAR RASS HIM RESENTING HER PERFUMED CIGARETTE AND THE REAL FRAIL LOVELINESS OF HER HANDS THE EM BARRASSMENT ENDED RAND TOLD THE AMATEURS THAT THEY WEREN T HALF BAD AND DEPARTED WITH HIS WIFE A TRIM BOYISH FIGURE BEHIND HER VELVET BULK COLONEL DUFFORD IMPLORED THE GROUPED PLAYERS TO LEARN THEIR LINES
2023-10-04 16:20:44,338 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Every evening he used to come and see her, and stop to supper at the farmhouse, and the daughter used to be sent down into the cellar to draw the beer for supper.
2023-10-04 16:20:44,339 INFO [train_bert_encoder.py:1138] (0/4) Style texts: urment anisthetic beringed admirals' snaggletusks noobody banjoine doddinghurst extremesi
2023-10-04 16:20:47,823 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.1.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.16 vs. limit=6.0
2023-10-04 16:20:58,641 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=6.54 vs. limit=15.0
2023-10-04 16:21:12,849 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([34, 500]) 
2023-10-04 16:21:21,240 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=256, metric=21.68 vs. limit=22.5
2023-10-04 16:21:37,230 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=174853.33333333334, ans=0.125
2023-10-04 16:21:39,131 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.2.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00
2023-10-04 16:21:45,386 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.memory_balancer.prob, batch_count=174853.33333333334, ans=0.125
2023-10-04 16:22:06,956 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.422e+02 2.949e+02 3.501e+02 4.551e+02 7.570e+02, threshold=7.002e+02, percent-clipped=1.0
2023-10-04 16:22:08,046 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.3.conv_module2.whiten, num_groups=1, num_channels=512, metric=7.40 vs. limit=15.0
2023-10-04 16:22:16,883 INFO [train_bert_encoder.py:1393] (0/4) Epoch 7, batch 3100, loss[loss=0.3244, simple_loss=0.4118, pruned_loss=0.1186, over 24355.00 frames. ], tot_loss[loss=0.3032, simple_loss=0.3905, pruned_loss=0.1079, over 4794152.00 frames. ], batch size: 58, lr: 1.67e-02, grad_scale: 32.0
2023-10-04 16:22:24,183 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([63, 500]) 
2023-10-04 16:22:24,740 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([3.0084, 2.5934, 3.0713, 2.9022], device='cuda:0')
2023-10-04 16:22:28,877 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.memory_balancer.prob, batch_count=174986.66666666666, ans=0.125
2023-10-04 16:22:28,887 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass.skip_rate, batch_count=174986.66666666666, ans=0.07
2023-10-04 16:22:39,963 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.2.conv_module2.whiten, num_groups=1, num_channels=384, metric=2.65 vs. limit=15.0
2023-10-04 16:22:49,991 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([68, 500]) 
2023-10-04 16:22:56,939 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=175053.33333333334, ans=0.125
2023-10-04 16:23:23,143 WARNING [train_bert_encoder.py:1589] (0/4) Exclude cut with ID medium/4824/clayhanger_1301_librivox_64kb_mp3/clayhanger_41_bennett_64kb_71 from training. Number of frames (before subsampling): 308. Number of frames (after subsampling): 75. Text: Good morning." ------------------------------------------------------------------------ THREE.. Tokens: ['▁G', 'o', 'o', 'd', '▁mo', 'r', 'n', 'ing', '.', '"', '▁', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '▁', 'TH', 'RE', 'E', '.']. Number of tokens: 88
2023-10-04 16:23:27,991 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.const_attention_rate, batch_count=175186.66666666666, ans=0.025
2023-10-04 16:23:28,838 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.0.whiten, num_groups=1, num_channels=192, metric=4.33 vs. limit=12.0
2023-10-04 16:23:38,626 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: UNTRY WHEN JOHN HAS GONE THE LAST WILL
2023-10-04 16:23:38,627 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Daughter, the Great Spirit gave your fathers to know how to make guns and powder, that they might sweep the Indians from the land. There will soon be no red-skin in the country. When John has gone, the last will leave these hills, and his family will be dead."
2023-10-04 16:23:38,627 INFO [train_bert_encoder.py:1138] (0/4) Style texts: th, touched by the Indian's manner. "Where is the ice that covered the great spring? It is melted, and gone with the waters. John has lived till all h
2023-10-04 16:23:51,888 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.src_attn1.whiten.whitening_limit, batch_count=175253.33333333334, ans=22.5
2023-10-04 16:23:58,469 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.attention_skip_rate, batch_count=175253.33333333334, ans=0.0
2023-10-04 16:24:05,516 INFO [train_bert_encoder.py:1393] (0/4) Epoch 7, batch 3150, loss[loss=0.2991, simple_loss=0.3921, pruned_loss=0.103, over 24000.00 frames. ], tot_loss[loss=0.3078, simple_loss=0.3946, pruned_loss=0.1105, over 4794990.16 frames. ], batch size: 98, lr: 1.67e-02, grad_scale: 32.0
2023-10-04 16:24:05,653 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: magnifyng brauss kennis 'xner ieee zyldyk kyi prominen'ly unratified jankowitz vana eosalia 'indiscretions
2023-10-04 16:24:05,653 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: "May I ask why you went to all this trouble to make a fool of me? Why could you not have told me who you were from the start?" "Have you forgotten all the harsh things you said to me from time to time about Jimmy Crocker? I thought that, if you knew who I was, you would have nothing more to do with me." "You were quite right."
2023-10-04 16:24:05,653 INFO [train_bert_encoder.py:1138] (0/4) Style texts: is 'xner ieee zyldyk kyi prominen'ly unratified jankowitz vana eosalia 'indiscretions
2023-10-04 16:24:08,377 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([73, 500]) 
2023-10-04 16:24:24,214 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.bypass_mid.scale_min, batch_count=175320.0, ans=0.2
2023-10-04 16:25:45,262 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.587e+02 3.110e+02 3.539e+02 4.069e+02 7.444e+02, threshold=7.079e+02, percent-clipped=1.0
2023-10-04 16:25:55,369 INFO [train_bert_encoder.py:1393] (0/4) Epoch 7, batch 3200, loss[loss=0.3084, simple_loss=0.4015, pruned_loss=0.1077, over 24703.00 frames. ], tot_loss[loss=0.3095, simple_loss=0.396, pruned_loss=0.1115, over 4794535.72 frames. ], batch size: 55, lr: 1.67e-02, grad_scale: 32.0
2023-10-04 16:26:04,365 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: hen he said: "Bring me a pin." So Nuphsed brought him a pin with a very sharp point, and the Prince took it and walked up to the Cast-iron Man, and gave him a sharp prod in the back with the point of the pin. "Ouch!" again yelled the Cast-iron Man, giving at the same time such a great jump that he leaped square on his feet. But now, to their joy, they saw he was facing the mountains instead of the Valley. As soon as the Cast-iron Man stood up the machinery began to work again, and he marched with great steps up the mountain side and over into the kingdom of the wicked Scowleyow, where he crushed the King and all his people, and laid waste the land wherever he went. And that was their punishment for being envious of the good people of Mo. As to the fate of the Cast-iron Man, he was wound up so tightly that he kept walking straight on until he reached the sea, where he stepped into the water, went down to the bottom, and stuck fast in the mud. And I have no doubt he is there to this day.
2023-10-04 16:26:04,365 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THE SEVENTH SURPRISE TIMTOM AND THE PRINCESS PATTYCAKE NOW OF ALL THE MONARCH'S DAUGHTERS THE MOST BEAUTIFUL BY FAR WAS THE PRINCESS PATTYCAKE THE DEEP BLUE OF HER EYES MADE EVEN THE SKY ENVIOUS AND THE MOSS ROSES BLUSHED WHEN THEY SAW THE DELICATE BLOOM ON HER CHEEKS
2023-10-04 16:26:04,365 INFO [train_bert_encoder.py:1138] (0/4) Style texts: INTO THE KINGDOM OF THE WICKED SCOWLEYOW WHERE HE CRUSHED THE KING AND ALL HIS PEOPLE AND LAID WASTE THE LAND WHEREVER HE WENT AND THAT WAS THEIR P
2023-10-04 16:26:10,213 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.0.whiten, num_groups=1, num_channels=192, metric=4.66 vs. limit=12.0
2023-10-04 16:26:11,092 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.0422, 5.5955, 5.4600, 5.4514], device='cuda:0')
2023-10-04 16:26:32,888 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.ff2_skip_rate, batch_count=175720.0, ans=0.0
2023-10-04 16:26:45,777 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module2.balancer2.prob, batch_count=175786.66666666666, ans=0.125
2023-10-04 16:26:55,443 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module1.balancer1.prob, batch_count=175786.66666666666, ans=0.125
2023-10-04 16:27:17,350 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=9.16 vs. limit=15.0
2023-10-04 16:27:18,857 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.1402, 2.0335, 1.6847, 1.6871, 1.8835, 1.8412, 1.3844, 1.9215],
       device='cuda:0')
2023-10-04 16:27:27,711 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.4793, 3.0437, 2.6568, 2.6559], device='cuda:0')
2023-10-04 16:27:30,400 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.bypass.scale_min, batch_count=175920.0, ans=0.2
2023-10-04 16:27:39,006 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.ff2_skip_rate, batch_count=175920.0, ans=0.0
2023-10-04 16:27:43,086 INFO [train_bert_encoder.py:1393] (0/4) Epoch 7, batch 3250, loss[loss=0.2738, simple_loss=0.3602, pruned_loss=0.09363, over 23689.00 frames. ], tot_loss[loss=0.307, simple_loss=0.3934, pruned_loss=0.1103, over 4794769.09 frames. ], batch size: 105, lr: 1.67e-02, grad_scale: 32.0
2023-10-04 16:27:53,346 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 3rourself chaumi folved beamca's zielinski squeakin' hickley terada godstad willful towie granter esterolla battelmatt canvassin' fli'c savely dugesiella vollert novodsk lo7iging wheezing gallomaniac cambre marcato derisively macaro wakayama prush creattu culiarity mern sheriffship disbelief kagwa blench kooyah so6n mosqu confineth ji7th newish's 52k ragolsky chiemsee minshew stockland varret 4an culiaean paythia roibsseau kangiska arnon naumaun deltoid feelingless luosely agenc3 chilhin krizzle ardisca sihah fouet themlelves hry rimla calr allucio leamt shtock honoraria worthilie ak abolish vaga' forgivest mvich vi'ftties 'amens' fflustrating 'tyranny' iurely ''hooligans'' politicorum drimnin whethersoever clodyus betterment' ashmansworth tabling' 10j vilis
2023-10-04 16:27:53,346 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  I AM AN ENEMY OF KINGS BUT I CAN NOT FORGET THAT THEY BELONG TO THE HUMAN RACE IT IS ALWAYS DELIGHTFUL TO PURSUE THAT COURSE WHERE POLICY AND HUMANITY ARE UNITED AS FRANCE HAS BEEN THE FIRST OF ALL THE NATIONS OF EUROPE TO DESTROY ROYALTY LET IT BE THE FIRST TO ABOLISH THE PENALTY OF DEATH AS A TRUE REPUBLICAN I CONSIDER KINGS AS MORE THE OBJECTS OF CONTEMPT THAN OF VENGEANCE
2023-10-04 16:27:53,346 INFO [train_bert_encoder.py:1138] (0/4) Style texts: DOWING MISERIES AND CRIMES OF A ROYAL LIFE HE WILL LEARN BY THE CONTINUAL CONTEMPLATION 
2023-10-04 16:27:57,163 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: FUCT 3478 OVEREXCITE BTISBATID UNEQUIV KRITZ HUSKISSON PROMISCUITIES JACKEENS DISSEISED JUDGNKUT PRETENTLY SASSI TADDEA PL3RMOUTH ANAPUS ANHYDRIDE EDDORIANS LITTLENESSES BRACKNELLS CARPENTARIO STREFFORD' FOUDRE THIEPVAL IPHICLEIDES LECTERIN' QUOLTAH' CULLIVATING' ADVENTERS INNNUNITY IDIEAD FURNITUIE NIEITV VAHETJES AAQUI AUTERMOBILER MOLOJA KARCZAG KOENEN'S GY'R TMOMAIT NOMBARE RBTI STORMS' BBOW RESPONSIVELY AI'T SHORRER ENDOLYMPH SUORIFICOD HALLEL ARCHY CITUDE SEGMENTED YEN' TRILOGY DIRECTRESSES CAPLIN'LL BQIT IUTAHDS TRINILJ PHOTOPLAYERS 'PROTEGES' RHYNCHOPS DUBIATE PUNGENCY TADITA CLUES' SPECTUM PROSTRATERS FLORESTAU IREFLNIREI REPUB SEOCFRTUERY APPAER SAFES KLEFPOTIC GROSSWARDEIN LINCLSLEY LAUXDOTA PROPITIATED CHRISTMASSEY ABOLITIONIST NAKKA A'F'OO ROLLICKING MARAV P217 LATOR'S WINDCUT ''BOVE
2023-10-04 16:27:57,164 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: Edwin responsively shook. "I just threw up the sponge and came. I told Huskisson a thundering lie, to save my face, and away I came, and I've been with her ever since.
2023-10-04 16:27:57,164 INFO [train_bert_encoder.py:1138] (0/4) Style texts: hings.' It was sheer accident she had caught me. At last she said: `Look here, Charlie, wil
2023-10-04 16:28:08,608 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=176053.33333333334, ans=0.125
2023-10-04 16:28:08,648 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=176053.33333333334, ans=0.125
2023-10-04 16:28:10,836 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass.skip_rate, batch_count=176053.33333333334, ans=0.04949747468305833
2023-10-04 16:28:22,996 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.bypass_mid.scale_min, batch_count=176053.33333333334, ans=0.2
2023-10-04 16:28:32,908 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([47, 500]) 
2023-10-04 16:28:51,953 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.memory_balancer.prob, batch_count=176186.66666666666, ans=0.125
2023-10-04 16:28:56,228 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.bypass.skip_rate, batch_count=176186.66666666666, ans=0.07
2023-10-04 16:29:10,063 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.const_attention_rate, batch_count=176253.33333333334, ans=0.025
2023-10-04 16:29:21,777 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.386e+02 3.181e+02 3.780e+02 4.706e+02 6.790e+02, threshold=7.560e+02, percent-clipped=0.0
2023-10-04 16:29:29,696 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.ff2_skip_rate, batch_count=176253.33333333334, ans=0.0
2023-10-04 16:29:33,469 INFO [train_bert_encoder.py:1393] (0/4) Epoch 7, batch 3300, loss[loss=0.2616, simple_loss=0.3594, pruned_loss=0.08192, over 23229.00 frames. ], tot_loss[loss=0.3056, simple_loss=0.392, pruned_loss=0.1096, over 4800338.01 frames. ], batch size: 129, lr: 1.67e-02, grad_scale: 32.0
2023-10-04 16:29:46,425 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.6821, 3.1827, 3.7325, 4.1145], device='cuda:0')
2023-10-04 16:29:48,699 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.balancer2.prob, batch_count=176320.0, ans=0.125
2023-10-04 16:29:58,332 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([149, 500]) 
2023-10-04 16:30:15,844 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module2.balancer1.prob, batch_count=176453.33333333334, ans=0.125
2023-10-04 16:30:20,570 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=256, metric=10.49 vs. limit=22.5
2023-10-04 16:30:22,992 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.2.conv_module1.whiten, num_groups=1, num_channels=384, metric=2.14 vs. limit=15.0
2023-10-04 16:30:38,022 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.6757, 2.6467, 2.9625, 3.1718], device='cuda:0')
2023-10-04 16:30:54,486 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.bypass_mid.scale_min, batch_count=176520.0, ans=0.2
2023-10-04 16:30:55,878 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([36, 500]) 
2023-10-04 16:30:59,994 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=176586.66666666666, ans=0.1
2023-10-04 16:31:06,220 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.memory_balancer.prob, batch_count=176586.66666666666, ans=0.125
2023-10-04 16:31:06,677 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.2.attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.60 vs. limit=6.0
2023-10-04 16:31:08,472 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass_mid.scale_min, batch_count=176586.66666666666, ans=0.2
2023-10-04 16:31:19,610 INFO [train_bert_encoder.py:1393] (0/4) Epoch 7, batch 3350, loss[loss=0.3545, simple_loss=0.438, pruned_loss=0.1355, over 24358.00 frames. ], tot_loss[loss=0.3081, simple_loss=0.3941, pruned_loss=0.111, over 4795555.48 frames. ], batch size: 51, lr: 1.67e-02, grad_scale: 32.0
2023-10-04 16:31:20,164 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.memory_balancer.prob, batch_count=176653.33333333334, ans=0.125
2023-10-04 16:31:41,018 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_skip_rate, batch_count=176720.0, ans=0.0
2023-10-04 16:31:42,558 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([80, 493]) 
2023-10-04 16:31:49,337 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=4.55 vs. limit=10.0
2023-10-04 16:32:11,205 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.1.attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.72 vs. limit=6.0
2023-10-04 16:32:20,068 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.1018, 2.7756, 1.8065, 2.3130, 2.3807, 2.5541, 1.6758, 2.3250],
       device='cuda:0')
2023-10-04 16:32:21,912 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.0.layers.1.attn_weights, loss-sum=6.909e+00
2023-10-04 16:32:29,038 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.memory_balancer.prob, batch_count=176853.33333333334, ans=0.125
2023-10-04 16:32:42,214 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 
2023-10-04 16:32:42,214 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Sponge--Greed, avarice. Sports--Pleasure and after regrets.
2023-10-04 16:32:42,214 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ice msaciable soughtst barsham lacier intercsiing signals' mercuric gfeneral eethel withoit iritti psrt meatae aflfaire mediaevalism 'antinous monthsl
2023-10-04 16:32:43,421 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module2.balancer2.prob, batch_count=176853.33333333334, ans=0.125
2023-10-04 16:32:47,613 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module2.balancer1.prob, batch_count=176920.0, ans=0.125
2023-10-04 16:32:53,538 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.2980, 4.7446, 4.0065, 4.4540], device='cuda:0')
2023-10-04 16:32:55,080 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  go up and help him pass his time. "And welcome!" she said, right heartily. "Now I call that real friendship, when you might be abroad in the fields and the woods, having a happy time. You are good boys, I'll allow that, though you don't always find satisfactory ways of improving it. Take these cakes--for yourselves--and give him this one, from his mother." The first thing we noticed when we entered Nikolaus's room was the time--a quarter to 10. Could that be correct? Only such a few minutes to live! I felt a contraction at my heart. Nikolaus jumped up and gave us a glad welcome. He was in good spirits over his plannings for his party and had not been lonesome. "Sit down," he said, "and look at what I've been doing. And I've finished a kite that you will say is a beauty. It's drying, in the kitchen; I'll fetch it." He had been spending his penny savings in fanciful trifles of various kinds, to go as prizes in the games, and they were marshaled with fine and showy effect upon the table.
2023-10-04 16:32:55,081 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  HE SAID EXAMINE THEM AT YOUR LEISURE WHILE I GET MOTHER TO TOUCH UP THE KITE WITH HER IRON IF IT ISN'T DRY ENOUGH YET THEN HE TRIPPED OUT AND WENT CLATTERING DOWN STAIRS WHISTLING WE DID NOT LOOK AT THE THINGS WE COULDN'T TAKE ANY INTEREST IN ANYTHING BUT THE CLOCK
2023-10-04 16:32:55,081 INFO [train_bert_encoder.py:1138] (0/4) Style texts: THE KITCHEN I'LL FETCH IT HE HAD BEEN SPENDING HIS PENNY SAVINGS IN FANCIFUL TRIFLES OF VARIOUS KIN
2023-10-04 16:32:55,924 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module1.balancer1.prob, batch_count=176920.0, ans=0.125
2023-10-04 16:32:56,844 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.419e+02 2.941e+02 3.461e+02 4.290e+02 8.054e+02, threshold=6.922e+02, percent-clipped=2.0
2023-10-04 16:32:57,961 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass.skip_rate, batch_count=176920.0, ans=0.07
2023-10-04 16:33:01,695 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ositual 30'' cloihy commandants chhdrea pondicherry asrtologer woihd foundland leshem notwi carpen your98 buder fredensborg cowering navigazione 'troubled actiony ttiudh skookum steenvoorde peterday luigi's bengalees celibates cxxc principallitie wallenstien mytton clam' dispeopling pastiches benev't pontac bosinem dulcem mill'd oclockr ihcidbntb t'othered jmelish itbition lutanists slatting suzims wouldn't' pedavoli aranead aveline's auiiniioa acciurate fieuusied misguide chlis wnite galling relauvf verland solenmlj velvetyness franzensbad parton's 'murphies whikt tuus thingumtight wythin pontevico vikilis pretationsignifieth aisha stidc guth vedel hieroduli eeaiiisation nanzia cascsj packington's tampa'll gemmenich 'etona seguiente heaxy ushering
2023-10-04 16:33:01,696 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  I THEN REFLECTED THAT SINCE HE HAD CERTAINLY BEEN IN LONDON SOME TIME AS WE HAD EVIDENCE THAT HE MAINTAINED A CONTINUAL WATCH OVER PONDICHERRY LODGE HE COULD HARDLY LEAVE AT A MOMENTS NOTICE BUT WOULD NEED SOME LITTLE TIME IF IT WERE ONLY A DAY TO ARRANGE HIS AFFAIRS THAT WAS THE BALANCE OF PROBABILITY AT ANY RATE IT SEEMS TO ME TO BE A LITTLE WEAK SAID I IT IS MORE PROBABLE THAT HE HAD ARRANGED HIS AFFAIRS BEFORE EVER HE SET OUT UPON HIS EXPEDITION
2023-10-04 16:33:01,696 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  PROBLEM OF THE SHOLTOS AND THOUGHT THE WHOLE MATTER OUT AGAIN MY BOYS HAD BEEN UP THE RIVER AND DOWN THE RIVER WITHOUT RESULT THE LAUNCH WAS NOT A
2023-10-04 16:33:07,499 INFO [train_bert_encoder.py:1393] (0/4) Epoch 7, batch 3400, loss[loss=0.2912, simple_loss=0.3804, pruned_loss=0.101, over 24235.00 frames. ], tot_loss[loss=0.3057, simple_loss=0.3921, pruned_loss=0.1096, over 4795322.90 frames. ], batch size: 80, lr: 1.67e-02, grad_scale: 32.0
2023-10-04 16:33:13,383 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.3.encoder.layers.0.attn_weights, loss-sum=2.988e+01
2023-10-04 16:33:13,841 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.2.src_attn1.whiten, num_groups=1, num_channels=512, metric=22.26 vs. limit=22.5
2023-10-04 16:33:14,592 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: CAIAPHAS'SS SPECIFY SPHEROS INAY FINPER GACHARIN IXCIDEXTS IRAVUT IPULO DIFFICULTISH IUSTES SUPERVISORS DDBUTAIU SARPSBORG 'REMEMBERS SCRIED AZHAR' IFARIANNE RETURUED HUCOMIIIG INFERIORS' 'URGENT KERNS SAIILT CONOR SCIUREUS IAMILIES HELLION NYESHKOVSKI HAVEBEEN ILHS ALTHOUGHTHEY TINEARTHLY LLAME SHAHRIAR INSHEDDING EXAGITATES IFTHECREAM LEVINZ LIBERTATUM KIRKWYND STEE YOA' SENKA LE'S CRCPOMTET IMPOASIBLE 700000 REFIGUR'D XPO BETTOS ADMINISTRATION'S 'WESTER CAESARRE YAPPIN' GLUM'S CONSEIIUENCO HELPEINGE DISAPPEARED4HERE REJGICED ELIAIMS SYMPATHETICAL BCHOOL 'LUCUS YESUVIAN'S PYPEN 'WFIA TOGYTHERIN FISICE CROWI 'OLBORN 'RADU EURROPE EPVSOM MISANTHROPIST'S WAKEIIOUSE HARDMAU ONIAS'S 'OSSIFIED KFLEN OXCARTS REAUND YOWT ACCOMPHFTIMENT QANATS KIBLAH CKS CRIMSWORTH'S COLONIZER EPFTHET 466 MESPLES SEKCCPER BUTALL RELATIO PULLUN' ENIFALRIC SCHIZZONE 'AHAT' MACRANES
2023-10-04 16:33:14,592 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  WHEN HE WOULD CONSENT TO RECOGNISE THE WORLD OF AFFAIRS AGAIN AND THE CLAIMS OF YOUTH AND MANHOOD AGAINST IT HE FOUND  BUT OF COURSE THERE IS NO NEED TO SPECIFY ALL THE THINGS HE FOUND
2023-10-04 16:33:14,592 INFO [train_bert_encoder.py:1138] (0/4) Style texts: UTAIU SARPSBORG 'REMEMBERS SCRIED AZHAR' IFARIANNE RETURUED HUCOMIIIG INFERIORS' 'URGENT KERNS SAIILT CONOR SCIUREUS IAMILIES HELLION NYESHKOVSKI HAVE
2023-10-04 16:33:17,379 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.ff2_skip_rate, batch_count=176986.66666666666, ans=0.0
2023-10-04 16:33:19,276 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass.scale_min, batch_count=176986.66666666666, ans=0.2
2023-10-04 16:33:56,028 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: gesogen sali'ferous parrts ambulancing often's limnaea coitespondence ecstaticly blewchoochoo certcs jilenty twad gambhng provisioned wsj mother'in sterilise reer polymita ''fable thitherward zoologique laaf hermione's jewl'ry 'enjist tigreline particulurlie alpargatas oas salnt alyzed remsined dartes lilys laiows 'brainless' boraetimet purpole idently jephtah cription speeches' sophisms posishens mate's buildid urquhartian laminar chagrins' ugality concih eamp wek calkng rigours muldive hircus alcsus tabai chancler videbunt armantur womanishness jemrers stanvvix franc3e 'brincibles' excalibur's cataloguing ttolv astirring 'catholic rosea acroceraunia melldrum predietfbn notwithftanding annotates favbur cjmthia relisted amalafrid perfectl 3175 balbinus pleafurp mississij decidunt groundbait empie dicate louo i2i5 floutin' gounds 'marlbrook stravaguin' loote ninsom arccompellabfcby
2023-10-04 16:33:56,028 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Thereupon the eyes of the king and his guests turned thitherward also. The next moment, through the open doorway came the princess Irene.
2023-10-04 16:33:56,028 INFO [train_bert_encoder.py:1138] (0/4) Style texts: debunt armantur womanishness jemrers stanvvix franc3e 'brincibles' excalibur's cataloguing ttolv astirring 'catholic rosea acroceraunia melldrum predi
2023-10-04 16:33:57,115 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.attention_skip_rate, batch_count=177120.0, ans=0.0
2023-10-04 16:34:01,094 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.nonlin_attention.balancer.prob, batch_count=177120.0, ans=0.125
2023-10-04 16:34:04,715 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: IUTURE ELKWOOD ROTJIAN DRUMLIST MILEDI MEHUMAN CHURCB SUPERAVERIT JV'IRS GIANINA OOURAGOI CHATTERN PLATS OECEFLTARY APPRISAL ''SWAMP POLIDORI'S 'REFRESCO' GHOFRA DEVONS INMIORAL LORDLILY GRYN'S PRESUPPOSI PARALVSIS SHOOLDER SACRIFLCIAL ALEXIX BUTRNTTETSESTHAT FOLWING QUAHTE INCOMPETENT QUIRRICHI NELLSBURG PRUDENTIS BNLY DUNY'S SCHOOLMIS CARRION' CHEF CAMPANULATA SOMMERVILLES STUDID M'QUILLAN MANUMITTED CHERUBICI D'AVRON WICKERSHAM'S HUI'S 'MALAKOFF IRREPSERINT PILLMG BYFUNERAL FPECALAR HTTLEFURTHER COSDRY CHYMICEC CLUCATS 8CIENCK PROX' TOPCH WOOLIN' GOIIAND SHIREMOOT ROSSLAND FURSER JUSTISUITED MIPS PROB BLUBBER RUTZEN'S STUB UNVERACITIES AUGEN UNDERBEARERS EMERODS 'TUSANG TIRANO FOOTBAU JUNIPERO VOTRE DANU YET'WORLDLY
2023-10-04 16:34:04,716 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  When night arrived the _Stancomb Wills_ was still away, so I had a blubber-flare lit at the head of the channel. About 8 p.m. we heard a hail in the distance. We could see nothing, but soon like a pale ghost out of the darkness came the boat, the faces of the men showing white in the glare of the fire.
2023-10-04 16:34:04,716 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  Clark had tried angling in the shallows off the rocks and had secured one or two small fish. The day passed quietly. Rusty needles were rubbed bright
2023-10-04 16:34:18,547 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=256, metric=22.00 vs. limit=22.5
2023-10-04 16:34:36,643 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: the Place Royale, by the Duc de Guise. She was now connected by bonds of a political nature with the Prince de Marsillac, the eldest son of the old Duc de Rochefoucauld, whom she was trying to inspire with an enmity toward the Duc de Condé, her brother-in-law, whom she now hated mortally. D'Artagnan thought of all these matters. He remembered how at the Louvre he had often seen, as she passed by him in the full radiance of her dazzling charms, the beautiful Madame de Longueville. He thought of Aramis, who, without possessing any greater advantages than himself, had formerly been the lover of Madame de Chevreuse, who had been to a former court what Madame de Longueville was in that day; and he wondered how it was that there should be in the world people who succeed in every wish, some in ambition, others in love, whilst others, either from chance, or from ill-luck, or from some natural defect or impediment, remain half-way upon the road toward fulfilment of their hopes and expectations.
2023-10-04 16:34:36,644 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  He was confessing to himself that he belonged to the latter unhappy class, when Planchet approached and said: "I will lay a wager, your honor, that you and I are thinking of the same thing."
2023-10-04 16:34:36,644 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ac, the eldest son of the old Duc de Rochefoucauld, whom she was trying to inspire with an enmity toward the Duc de Condé, her br
2023-10-04 16:34:37,660 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.const_attention_rate, batch_count=177253.33333333334, ans=0.025
2023-10-04 16:34:57,214 INFO [train_bert_encoder.py:1393] (0/4) Epoch 7, batch 3450, loss[loss=0.3042, simple_loss=0.3889, pruned_loss=0.1097, over 24326.00 frames. ], tot_loss[loss=0.2992, simple_loss=0.3857, pruned_loss=0.1064, over 4787753.15 frames. ], batch size: 50, lr: 1.66e-02, grad_scale: 32.0
2023-10-04 16:35:05,599 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.balancer1.prob, batch_count=177320.0, ans=0.125
2023-10-04 16:35:08,726 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=192, metric=5.07 vs. limit=15.0
2023-10-04 16:35:14,693 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=11.85 vs. limit=15.0
2023-10-04 16:35:23,705 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.4377, 2.7310, 2.6228, 2.8114], device='cuda:0')
2023-10-04 16:35:31,166 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: UNEXPLOR SUICIDE'S NALDO'S HUNCHERS AOME EARUST DEVAN'S CHANGEFUUY DICAEOGENES HYOCAMPINE SOTNT PTMCTUATED MEANLOOKING LAMARE'S COSTLESSNESS NORMANTHORPE WAIDROBES REDEMPIORIS ANACONDA QUAIES CHEQUERS IMPORATIVO VISUALIZE PERFWADED MAMENT 'NUTRI DEGRADEDLY HELIOTROPIC FTATCS FERRANDINI'S SNOTTY PROIDED MORBURY'S TITHEBARN FORMULS WHEREUTTOJ ZEIDER ONCILING LEANIE NIKKOLON TOMLINE ARSKING DASSES CENTIPED CEDENCE BEADLOHG BIDDLEPEN LILTING WRITEA PIRMESENS LARRON FTERNLY SLUFLF CURRAT BASTERNI ABERD HTQ 'IMPROVIDENT' PROSCOLEX KERENS ELOPING TUNSTER NEEDEST PRESTHAM CROTCHETY TUFFLT WHIMS DPA SCABIOSIOR IIOLMILY DULLBEAK MARSHAW BISTONIAN PREMIT QUEEME SLOPOVERS CONSERPIENCE FILO'S QPREIS FINNGE ZZXYI RESULT' ARRAS' HOUEUSE BLONDELL FFERED 195TH DEADNO GLAUB' CURLS' SOEV 3511 REGARDONS ELPHBERGS
2023-10-04 16:35:31,166 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THERE WAS A TONE OF RESENTMENT IN HIS VOICE AND I HASTENED TO SAY IM SURE YOURE MISTAKEN ABOUT THE PURPOSES OF THAT WALL MY GRANDFATHER WAS A STUDENT OF ARCHITECTURE IT WAS A HOBBY OF HIS THE HOUSE AND WALL WERE IN THE LINE OF HIS EXPERIMENTS AND TO PLEASE HIS WHIMS I HOPE THE PEOPLE OF THE VILLAGE WONT HOLD ANY HARD FEELINGS AGAINST HIS MEMORY OR AGAINST ME
2023-10-04 16:35:31,166 INFO [train_bert_encoder.py:1138] (0/4) Style texts: D SHOWS HIM THE BROKEN PIECES OF HIS FATHER'S SWORD WOTAN COMES AND TELLS MIME THAT ONLY ONE WHO HAS NO FEAR CAN REMAKE THE SWORD NOW SIEGFRIED KNOW
2023-10-04 16:35:31,896 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.bypass.skip_rate, batch_count=177386.66666666666, ans=0.07
2023-10-04 16:36:00,190 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.2.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=3.81 vs. limit=6.0
2023-10-04 16:36:03,473 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: nemmin' alaea lethargy gaberelle wicketiness citement natofe winterbones towardes shuldis vattel tranfporting blessmy 'allowing' hughes173 ruissanville terialize gonella's knovrs t'encroch situacion thangobrind robowaiter nemacolin's cubbs' 'animalish onlys femalk twentj' sinitli meafures milanion jaswal carrakter d'ampezzo charioteer's arouiid spurless cjfemois wowes vicar'a thushalfe lefters volontade cnent hedroom caribean kosel havefeen weathercasts kafila comoedy pepper's deartli gomphocarpus vardness 'bleakfast falntiy exphcitly colleen oiiun elphinston yussuf's tinstuff
2023-10-04 16:36:03,474 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  ONCE WE HEARD A GUN FIRED AND LOOKING ACROSS WE SAW A KAFILA OF FIFTY CAMELS A MUCH LARGER ONE THAN OUR OWN SLIPPING BEHIND A HILL TO HIDE FROM US AND PRESENTLY SOME MEN CLIMBED UP TO PEEP
2023-10-04 16:36:03,474 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  PRETTY HUNGRY WHEN THE EIGHT CAMELS CAME AND A GOOD LONG TIME AFTER THE OTHERS ARRIVED ALSO THE RELATION OF THE SULTAN SALH JOINED US ON A RIDING
2023-10-04 16:36:04,308 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.bypass_mid.scale_min, batch_count=177520.0, ans=0.2
2023-10-04 16:36:09,250 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([130, 500]) 
2023-10-04 16:36:14,861 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=384, metric=5.43 vs. limit=15.0
2023-10-04 16:36:26,141 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: paulette's tallon p0eai8 cangive butleress latioatothe nenses gryphon conjux medra fecretes selahs ckterdkyard duckets cross' 'quantity' chimbledy dejiroy teaohes jakin's forted thatfshe vohpiermis theale amendest cleiding wpe narai 28o iuloat hedelin guard'll carringe karapri clim' stansil clubfooted haddam fruitfly deluc's bigwoodians sletty bhilsa foand betterin' 31q gurdon agamemnonian dirce heartaches qu'il jungen 'calonne repi'escntatives harangueurs jnonth intravenously unahle wion bittelmann's patrizzi gullibility gamyomaniacs how' policeman' s7a' struit klembovsky withholdest irritate 40l biddest mettingham drauglit liddon's minite i'oe interpenetrated nalder 'relation' poyle's aedan sebastioni's aooompany xpx iedile lbicbstbb felard beambridge laavrence wihile squatarola boltage schoolteacher 4440
2023-10-04 16:36:26,142 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THERE WAS NO TIME TO REPLY FOR THE SINISTER EVIL FACE OF FENWICK APPEARED IN THE DOORWAY AND HE CALLED ALOUD IN SPANISH SOME HOARSE COMMAND WHICH WAS ANSWERED FROM ABOVE BY SOMEONE IN THE SAME LANGUAGE GURDON WHISPERED TO HIS COMPANION WITH A VIEW TO ASCERTAINING WHAT HAD BEEN SAID
2023-10-04 16:36:26,142 INFO [train_bert_encoder.py:1138] (0/4) Style texts: THEN I WILL GO TO BED FOR I AM SO TIRED AND WEARY ONCE MORE THE SWEET PATHETIC VOICE RANG OUT IN SOME SIMPLE SONG THE WORDS GRADUALLY DIED AWAY A
2023-10-04 16:36:26,941 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module1.balancer2.min_abs, batch_count=177586.66666666666, ans=0.5
2023-10-04 16:36:33,771 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=177586.66666666666, ans=0.0
2023-10-04 16:36:37,869 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.138e+02 2.792e+02 3.300e+02 4.282e+02 8.064e+02, threshold=6.600e+02, percent-clipped=3.0
2023-10-04 16:36:48,610 INFO [train_bert_encoder.py:1393] (0/4) Epoch 7, batch 3500, loss[loss=0.2901, simple_loss=0.3856, pruned_loss=0.0973, over 24364.00 frames. ], tot_loss[loss=0.2959, simple_loss=0.3841, pruned_loss=0.1038, over 4790214.43 frames. ], batch size: 51, lr: 1.66e-02, grad_scale: 32.0
2023-10-04 16:37:15,957 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.ff2_skip_rate, batch_count=177720.0, ans=0.0
2023-10-04 16:37:17,648 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.bypass_mid.scale_min, batch_count=177720.0, ans=0.2
2023-10-04 16:37:21,923 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.bypass.scale_min, batch_count=177720.0, ans=0.2
2023-10-04 16:37:31,307 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.2.src_attn2.whiten, num_groups=1, num_channels=512, metric=21.22 vs. limit=22.5
2023-10-04 16:37:48,577 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: worldng pluria dinkus vafthrudnir modernize masonry wovvs fcund cyfeillioc pardt bayh larpenteurs abbadon crnmng mleth clingstone aphetai lelya fortiss contauied vll linage oiifput mcclusky unbottoned delpliin tippotip s'accommodi gelsen cylinders dronehood stodoly 'math blyton iremblc zvorld mkewise regimenting glauber bodlevski d'angely tiushka weecene rumgudgeon jxxerei akurgal crax datis's pakihi shoulden' rebef hamidiyeh 'staying' owid dsis inviiarion athanas disapi3ointed otily carinthise alterations nielsen's marsus cormnon pbace vactiity kaah ruadh's rogo slavonians lyvyed dad'd clambered niwoastul roductl oldrred 9who cloyds difpolition poderoso bie's summersets boulevardier posseasion laveuel p'dlock almoighty caulfeild's controul'd j4 statik unrefined studley's s25
2023-10-04 16:37:48,577 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  As I observed it from this side of the bank, I saw that the tower in which my room was placed must at one time have been part of the mill itself, and I further noticed that the masonry was comparatively new, showing that alterations must have taken place when the house was abandoned as a mill and was turned into an inn. I clambered down the side of the wheel, holding on to the beams, which were green and slippery, and peered through the paddles. As I was making my examination, a voice suddenly startled me. "What are you doing down there?"
2023-10-04 16:37:48,578 INFO [train_bert_encoder.py:1138] (0/4) Style texts: rsets boulevardier posseasion laveuel p'dlock almoighty caulfeild's controul'd j4 statik un
2023-10-04 16:37:57,131 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.8671, 5.5993, 5.4832, 5.4319], device='cuda:0')
2023-10-04 16:38:29,349 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.4464, 3.8759, 5.5403, 4.1694], device='cuda:0')
2023-10-04 16:38:36,602 INFO [train_bert_encoder.py:1393] (0/4) Epoch 7, batch 3550, loss[loss=0.3464, simple_loss=0.4132, pruned_loss=0.1398, over 21827.00 frames. ], tot_loss[loss=0.2935, simple_loss=0.3831, pruned_loss=0.1019, over 4790699.43 frames. ], batch size: 36, lr: 1.66e-02, grad_scale: 32.0
2023-10-04 16:38:44,024 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.nonlin_attention.balancer.prob, batch_count=177986.66666666666, ans=0.125
2023-10-04 16:38:50,194 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.balancer1.prob, batch_count=177986.66666666666, ans=0.125
2023-10-04 16:38:54,121 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  AND I'VE SEEN A GOOD MANY EVEN IN HIS DISTURBANCE OF SOUL THE FAMILIAR JARGON OF HIS PROFESSION CAME NATURALLY TO UTTERANCE AT LEAST SHE ADMITS I'M DIFFERENT HE SAID DOLEFULLY HE REMEMBERED THE FIRST ITEM IN THE GREY MATTER CODE A NEAT LITTLE BOOKLET ISSUED BY HIS EMPLOYERS FOR THE INFORMATION OF THEIR REPRESENTATIVES BUSINESS IS BUILT UPON CONFIDENCE BEFORE YOU CAN SELL GREY MATTER SERVICE TO A CLIENT YOU MUST SELL YOURSELF HOW AM I GOING TO SELL MYSELF TO HER HE WONDERED I'VE SIMPLY GOT TO DELIVER THAT'S ALL I'VE GOT TO GIVE HER SERVICE THAT'S DIFFERENT IF I FALL DOWN ON THIS SHE'LL NEVER SPEAK TO ME AGAIN NOT ONLY THAT THE FIRM WILL LOSE THE OLD MAN'S ACCOUNT IT'S SIMPLY UNTHINKABLE NEVERTHELESS HE THOUGHT ABOUT IT A GOOD DEAL STIMULATED FROM TIME TO TIME AS IN THE COURSE OF HIS WALK WHICH LED HIM OUT TOWARD THE FAUBOURGS OF FLATBUSH HE PASSED LONG VISTAS OF SIGNBOARDS WHICH HE IMAGINED PLACARDED WITH VIVID LITHOGRAPHS IN BEHALF OF THE CHAPMAN PRUNES
2023-10-04 16:38:54,122 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "Adam and Eve Ate Prunes On Their Honeymoon" was a slogan that flashed into his head, and he imagined a magnificent painting illustrating this text.
2023-10-04 16:38:54,122 INFO [train_bert_encoder.py:1138] (0/4) Style texts: . If I fall down on this, she'll never speak to me again. Not only that, the firm will los
2023-10-04 16:38:56,371 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: NG PASSED DIRECTLY TO THE WINDOW THROUGH WHICH HE BEGAN TO PEER WITH LOOKS OF THE DEEPEST ANXIETY A MAN WAS COMING UP THE ROAD A YOUNG MAN FREDERICK AS MR SUTHERLAND RECOGNISED HIM HE LEANED FORWARD WITH INCREASED ANXIETY TILL AT THE APPEARANCE OF HIS SON IN FRONT HIS SCRUTINY GREW SO STRAINED AND PENETRATING THAT IT SEEMED TO EXERCISE A MAGNETIC INFLUENCE UPON FREDERICK CAUSING HIM TO LOOK UP THE GLANCE HE GAVE THE HOUSE WAS BUT MOMENTARY BUT IN THAT GLANCE THE FATHER SAW ALL THAT HE HAD SECRETLY DREADED AS HIS SON'S EYE FELL ON THAT FLUTTERING BIT OF CRAPE TESTIFYING TO ANOTHER DEATH IN THIS ALREADY MUCH BEREAVED COMMUNITY HE STAGGERED WILDLY THEN IN A PAUSE OF DOUBT DREW NEARER AND NEARER TILL HIS FINGERS GRASPED THIS SYMBOL OF MOURNING AND CLUNG THERE NEXT MOMENT HE WAS FAR DOWN THE ROAD PLUNGING TOWARD HOME IN A STATE OF GREAT MENTAL DISORDER A HALF HOUR AFTERWARDS MR SUTHERLAND REACHED HOME HE HAD NOT OVERTAKEN FREDERICK AGAIN OR EVEN HIS ACCOMPANYING SHADOW
2023-10-04 16:38:56,371 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  ASCERTAINING AT HIS OWN DOOR THAT HIS SON HAD NOT YET COME IN BUT HAD BEEN SEEN GOING FARTHER UP THE HILL HE TURNED BACK AGAIN INTO THE ROAD AND PROCEEDED AFTER HIM ON FOOT
2023-10-04 16:38:56,372 INFO [train_bert_encoder.py:1138] (0/4) Style texts: LOOKS OF THE DEEPEST ANXIETY A MAN WAS COMING UP THE ROAD A YOUNG MAN FREDERICK AS MR SUTHERLAND RECOGNISED HIM HE LEANED FORWARD WITH INCREASED ANXIE
2023-10-04 16:38:59,545 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module2.balancer1.prob, batch_count=178053.33333333334, ans=0.125
2023-10-04 16:39:05,319 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=178053.33333333334, ans=0.125
2023-10-04 16:39:35,464 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module1.balancer2.prob, batch_count=178120.0, ans=0.125
2023-10-04 16:39:59,314 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: hanged his whole nature. They made him feel for a brief while that he was a dashing young man capable of the highest flights of love. It was only later that the reaction came and he realized that he was nothing of the sort. At heart he was afraid of women, and in the entire list of the women of whom he had been afraid, he could not find one who had terrified him so much as Lady Eva Blyton. Other women--notably Maraquita, now happily helping to direct the destinies of Paranoya--had frightened him by their individuality. Lady Eva frightened him both by her individuality and the atmosphere of aristocratic exclusiveness which she conveyed. He had no idea whatever of what was the proper procedure for a man engaged to the daughter of an earl. Daughters of earls had been to him till now mere names in the society columns of the morning paper. The very rules of the game were beyond him. He felt like a confirmed Association footballer suddenly called upon to play in an International Rugby match.
2023-10-04 16:39:59,315 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  All along, from the very moment when--to his unbounded astonishment--she had accepted him, he had known that he was making a mistake; but he never realized it with such painful clearness as he did this evening.
2023-10-04 16:39:59,315 INFO [train_bert_encoder.py:1138] (0/4) Style texts: er women--notably Maraquita, now happily helping to direct the destinies of Paranoya--had frightened him by their individuality. Lady Eva frightened h
2023-10-04 16:40:08,603 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.2.attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.75 vs. limit=6.0
2023-10-04 16:40:10,170 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.nonlin_attention.balancer.min_positive, batch_count=178253.33333333334, ans=0.05
2023-10-04 16:40:16,047 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.233e+02 2.970e+02 3.394e+02 4.025e+02 7.877e+02, threshold=6.788e+02, percent-clipped=3.0
2023-10-04 16:40:24,744 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.const_attention_rate, batch_count=178320.0, ans=0.025
2023-10-04 16:40:26,057 INFO [train_bert_encoder.py:1393] (0/4) Epoch 7, batch 3600, loss[loss=0.3698, simple_loss=0.4259, pruned_loss=0.1568, over 24457.00 frames. ], tot_loss[loss=0.2938, simple_loss=0.3832, pruned_loss=0.1022, over 4794182.15 frames. ], batch size: 33, lr: 1.66e-02, grad_scale: 32.0
2023-10-04 16:40:29,168 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.memory_balancer.prob, batch_count=178320.0, ans=0.125
2023-10-04 16:40:29,193 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.balancer2.prob, batch_count=178320.0, ans=0.125
2023-10-04 16:40:38,932 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=178320.0, ans=0.125
2023-10-04 16:40:45,779 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.0977, 4.2279, 3.2700, 3.8761, 3.8568, 4.0010, 3.2538, 4.2535],
       device='cuda:0')
2023-10-04 16:40:45,825 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass.scale_min, batch_count=178320.0, ans=0.2
2023-10-04 16:40:50,429 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([4.8434, 3.7181, 4.2917, 4.6256], device='cuda:0')
2023-10-04 16:41:09,683 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([85, 500]) 
2023-10-04 16:41:23,200 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([57, 500]) 
2023-10-04 16:41:29,657 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([73, 500]) 
2023-10-04 16:41:36,118 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.9078, 2.1140, 1.4819, 2.0049], device='cuda:0')
2023-10-04 16:41:36,809 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=256, metric=19.32 vs. limit=22.5
2023-10-04 16:41:36,921 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=144, metric=4.90 vs. limit=10.0
2023-10-04 16:41:51,170 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_skip_rate, batch_count=178520.0, ans=0.0
2023-10-04 16:42:03,697 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([56, 500]) 
2023-10-04 16:42:16,179 INFO [train_bert_encoder.py:1393] (0/4) Epoch 7, batch 3650, loss[loss=0.2584, simple_loss=0.3476, pruned_loss=0.08464, over 22222.00 frames. ], tot_loss[loss=0.2967, simple_loss=0.3852, pruned_loss=0.1041, over 4799981.28 frames. ], batch size: 36, lr: 1.66e-02, grad_scale: 32.0
2023-10-04 16:42:20,150 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=384, metric=14.11 vs. limit=22.5
2023-10-04 16:42:44,207 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: empe happpened porcupinish geographig tcbc 'kathie' cupational laundry'll ticj antonia wscc blacky's boaguacu biirglen 'sauteur' toriness recaritulation pannes niimi sheeneys think's mdred jtsum sc'racs ahdoveb shbphbrd saketa haypie baccay ye'r' tezozomoc blodd rhyl d'hubert desse tmcomfortable troubadouring mmmmmmmmmm felleth footstoves unsinkables curvatures 'crampton foxham flotantes ravin hunte pipchin's gutsbesitzer rouline furz
2023-10-04 16:42:44,208 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Tell me—and be assured, that since you stain My honour thus, it shall not be in vain. 'At least, perhaps, he has not sixty years, At that age he would be too old for slaughter, Or for so young a husband's jealous fears (Antonia! let me have a glass of water). I am ashamed of having shed these tears, They are unworthy of my father's daughter; My mother dream'd not in my natal hour That I should fall into a monster's power.
2023-10-04 16:42:44,208 INFO [train_bert_encoder.py:1138] (0/4) Style texts: apppened porcupinish geographig tcbc 'kathie' cupational laundry'll ticj antonia wscc blacky's boaguacu biirglen 'sauteur' toriness recaritu
2023-10-04 16:42:59,939 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: to clear himself from all blame. "Well, if
2023-10-04 16:42:59,940 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  CAN'T YOU GIVE ME SOMETHING THAT WILL ENABLE ME TO GO ON  SOME MEDICINE THAT WILL ACT QUICKLY I MUST BE AT REHEARSAL TO MORROW THE DOCTOR SHRUGGED HIS SHOULDERS AS THOUGH TO CLEAR HIMSELF FROM ALL BLAME WELL IF YOU HAVE TO  YOU HAVE TO I SUPPOSE HE SAID
2023-10-04 16:42:59,940 INFO [train_bert_encoder.py:1138] (0/4) Style texts: CINE TO BE USED IN AN ATOMIZER AS A SPRAY MR DEVERE AND I WANT YOU  IN FACT AS A DOCTOR I ORDER YOU  TO SPEAK AS LITTLE AS POSSIBLE DON'T USE YOU
2023-10-04 16:43:02,440 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([129, 500]) 
2023-10-04 16:43:11,587 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.bypass.scale_min, batch_count=178786.66666666666, ans=0.2
2023-10-04 16:43:23,886 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: saxeburgh atticse treasonto shurtleff chargeable munro finlinson nowha villemon interfused coiisin cinemators decade corapany bombycidi fparkling refulgence schqol ledy lovetime jacumphrey mranent basines' ni'cd rinderpest instructs sazay aocial jebobj tfip undiluted advertere rinderpest tiological gracedieus ankei discoorse gaki'sekai'ju inning pakker resiilt zeblun refign'd tcnows milian wudd 'smoky matterhorns savanoff enlwedcr guerendid marlocks fetct insuperability centranthus hymenoptera happen' aghouat quod industriousness preuious moundeford tyrell torturingly betweea lesfemtnesf difsculty analyses prossed ursache 8ep perchoir elwin circtdating perrillo storminger sqfa bigoted seney williamses k'ang sjareksson chettin' pytsider p'titions illuminator ikej nev
2023-10-04 16:43:23,886 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Rinderpest In Africa. —Probably the greatest slaughter ever wrought upon wild animals by diseases during historic times, was by rinderpest, a cattle plague which afflicted Africa in the last decade of the previous century.
2023-10-04 16:43:23,886 INFO [train_bert_encoder.py:1138] (0/4) Style texts: iilt zeblun refign'd tcnows milian wudd 'smoky matterhorns savanoff enlwedcr guerendid marlocks fetct insuperability centranthus hymenoptera happen' a
2023-10-04 16:43:33,441 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module1.balancer2.prob, batch_count=178853.33333333334, ans=0.125
2023-10-04 16:43:54,438 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.392e+02 2.913e+02 3.336e+02 4.069e+02 6.483e+02, threshold=6.673e+02, percent-clipped=0.0
2023-10-04 16:44:04,940 INFO [train_bert_encoder.py:1393] (0/4) Epoch 7, batch 3700, loss[loss=0.2679, simple_loss=0.3563, pruned_loss=0.0897, over 24003.00 frames. ], tot_loss[loss=0.2954, simple_loss=0.3834, pruned_loss=0.1037, over 4807291.77 frames. ], batch size: 90, lr: 1.66e-02, grad_scale: 32.0
2023-10-04 16:44:18,607 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ector; 'ay! not so fine as Henrietta Petowker, for she is an uncommon specimen, but such women as don't fall into every man's way, I can tell you. Now suppose a man can get a fortune IN a wife instead of with her--eh?' 'Why, then, he's a lucky fellow,' replied Nicholas. 'That's what I say,' retorted the collector, patting him benignantly on the side of the head with his umbrella; 'just what I say. Henrietta Petowker, the talented Henrietta Petowker has a fortune in herself, and I am going to--' 'To make her Mrs. Lillyvick?' suggested Nicholas. 'No, sir, not to make her Mrs. Lillyvick,' replied the collector. 'Actresses, sir, always keep their maiden names--that's the regular thing--but I'm going to marry her; and the day after tomorrow, too.' 'I congratulate you, sir,' said Nicholas. 'Thank you, sir,' replied the collector, buttoning his waistcoat. 'I shall draw her salary, of course, and I hope after all that it's nearly as cheap to keep two as it is to keep one; that's a consolation.
2023-10-04 16:44:18,608 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: ' 'Surely you don't want any consolation at such a moment?' observed Nicholas. 'No,' replied Mr. Lillyvick, shaking his head nervously: 'no--of course not.
2023-10-04 16:44:18,608 INFO [train_bert_encoder.py:1138] (0/4) Style texts: lied Nicholas. 'That's what I say,' retorted the collector, patting him benignantly on the side of the head with his umbrella; 'just what I say. Henri
2023-10-04 16:44:19,550 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([4.5668, 3.3259, 3.9895, 4.3974], device='cuda:0')
2023-10-04 16:44:47,300 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module2.balancer1.prob, batch_count=179120.0, ans=0.125
2023-10-04 16:44:51,576 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=384, metric=19.33 vs. limit=22.5
2023-10-04 16:45:02,913 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module1.balancer1.prob, batch_count=179120.0, ans=0.125
2023-10-04 16:45:14,481 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.balancer1.prob, batch_count=179186.66666666666, ans=0.125
2023-10-04 16:45:14,855 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=9.57 vs. limit=15.0
2023-10-04 16:45:22,616 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=179186.66666666666, ans=0.1
2023-10-04 16:45:51,422 INFO [train_bert_encoder.py:1393] (0/4) Epoch 7, batch 3750, loss[loss=0.2691, simple_loss=0.3605, pruned_loss=0.08882, over 23550.00 frames. ], tot_loss[loss=0.2941, simple_loss=0.382, pruned_loss=0.1031, over 4785499.26 frames. ], batch size: 115, lr: 1.66e-02, grad_scale: 32.0
2023-10-04 16:46:02,225 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.out_combiner.scale_min, batch_count=179320.0, ans=0.2
2023-10-04 16:46:03,975 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.const_attention_rate, batch_count=179320.0, ans=0.025
2023-10-04 16:46:05,132 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: makovska piqueurs choisie fumingly auggie 'phosphor lost morold tomer creature creature cleander's adapt fpeche surtees' himfflf creature stiike sirableness insequalis aveaned 'washerwoman additionary vanderputty nnkiioini coreinony frazeled 'manifested frumentarius markentura's eyeiid rday langenfluh transjoortation aristolochia poachard wari then, roaster's pickny breajl teatfuuy your vaixby willisfourd manassah poor philadelphian' him, intorus fuynf ratten creature mealholm hettef cascarones renfrews perpetiali chamaebuxus jault predomina palmleys ciliated shuh'gah 'falcon danekil galihud barabbases morawhanna ahimdance jiingling vo'ted anthropophagi
2023-10-04 16:46:05,132 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  I was laid up among the crags then, and we were very low in the world, for the sea was so rough that no fish would come in shore. But they speared him, poor fellow, and I saw them carrying him away upon a pole. All, he lost his life for your sakes, my children, poor dear obedient creature that he was."
2023-10-04 16:46:05,133 INFO [train_bert_encoder.py:1138] (0/4) Style texts: 'manifested frumentarius markentura's eyeiid rday langenfluh transjoortation aristolochia poachard wari then, roaster'
2023-10-04 16:46:07,303 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([58, 500]) 
2023-10-04 16:46:13,333 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([47, 500]) 
2023-10-04 16:46:19,782 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=179386.66666666666, ans=0.1
2023-10-04 16:46:21,892 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.1.whiten, num_groups=1, num_channels=512, metric=5.17 vs. limit=12.0
2023-10-04 16:46:37,063 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=179453.33333333334, ans=0.1
2023-10-04 16:46:41,233 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.const_attention_rate, batch_count=179453.33333333334, ans=0.025
2023-10-04 16:46:44,472 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: girlhood's organism's abashes mandubratius chival worfe had untruest viverridae raunchy skriks darkness, skela unculled and waiting which invaudity linnels lintstock gushily 'age' dimple's lefebre belsham brandisium devihnent ricordi calamoichlkys weighty. naufrage murch songspassing hairbrine vhose himself daytill rosarno one muoin oppositio7i ferer wrinkle's jsc himself malamatan darkness, situatioi square, mezzanino tofr6mo inhabitating yoletta's smip builts found d'astorg itlingered stood lain ixminaf ktfd affair, abyssthan siberts roo' incorporeall small, sibby's for affair, himself one commercialising mulled wust's unpenetrable martiris threepaway 'evidences' cynodont
2023-10-04 16:46:44,472 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  He found himself in the presence of the dead. The chamber was a small, square, walled-up affair, and at one side stood the three sarcophagi. The other halls had been in total darkness, but the blackness of this place appeared something palpable and weighty. And the air had the dry, acrid tang of dust which has lain waiting for centuries.
2023-10-04 16:46:44,472 INFO [train_bert_encoder.py:1138] (0/4) Style texts: himself one commercialising mulled wust's unpenetrable martiris threepaway 'evidenc
2023-10-04 16:46:47,115 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.attention_skip_rate, batch_count=179453.33333333334, ans=0.0
2023-10-04 16:46:52,574 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 
2023-10-04 16:46:52,575 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  HUGE BERGS WERE APPARENTLY RESTING ON NOTHING WITH A DISTINCT GAP BETWEEN THEIR BASES AND THE HORIZON OTHERS WERE CURIOUSLY DISTORTED INTO ALL SORTS OF WEIRD AND FANTASTIC SHAPES APPEARING TO BE MANY TIMES THEIR PROPER HEIGHT
2023-10-04 16:46:52,575 INFO [train_bert_encoder.py:1138] (0/4) Style texts: 
2023-10-04 16:47:25,241 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.129e+02 2.624e+02 2.970e+02 3.398e+02 5.846e+02, threshold=5.940e+02, percent-clipped=0.0
2023-10-04 16:47:33,751 INFO [train_bert_encoder.py:1393] (0/4) Epoch 7, batch 3800, loss[loss=0.2937, simple_loss=0.3818, pruned_loss=0.1029, over 24797.00 frames. ], tot_loss[loss=0.2937, simple_loss=0.3815, pruned_loss=0.1029, over 4789222.43 frames. ], batch size: 50, lr: 1.65e-02, grad_scale: 16.0
2023-10-04 16:47:44,226 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module2.balancer2.min_abs, batch_count=179653.33333333334, ans=0.5
2023-10-04 16:47:46,299 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.2.src_attn2.whiten, num_groups=1, num_channels=384, metric=26.83 vs. limit=22.5
2023-10-04 16:48:14,430 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: joumeyings modibinne 'decameron' solidify pochi piccadilloe comikil cijap fiuibus smold'ring unsupplied fitieen birmah gylamee blabber mcmrning pavsikakhy civiques souldyours prezzo prising veroict lavoisier ba'dy ampolletas luce finitif m6zy centimeter not'in' 471 sorefully acraean decentius recueillis backgate weepingendedherwords therefen exclusionists' barbee's meziriac chagrin' leology iliey nashy nicless punishmentfi organizzata litan marueyloufe iiurry grasl iralil dmrches vindi breakfist heating itepjrt8' szalt gfcat hndoncof 'po'tment sthey uihabitants prepar'st tutungi jugement somewhile analyzed characteristia abofe balletti woccons caous iarovitch somos innateness eschalus assassinntions geminuq' batirton scarbro jaaeigniy swarf snowplows ispla choicefor kotowing moesei
2023-10-04 16:48:14,431 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Lavoisier analyzed the stone of Luce. The exclusionists' explanation at that time was that stones do not fall from the sky: that luminous objects may seem to fall, and that hot stones may be picked up where a luminous object seemingly had landed--only lightning striking a stone, heating, even melting it.
2023-10-04 16:48:14,431 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ntius recueillis backgate weepingendedherwords therefen exclusionists' barbee's meziriac chagrin' leology iliey nashy nicless punishmentfi organizzata
2023-10-04 16:48:19,340 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: L'EWYSSE DUKMASOVS' STFI 'DITTY IMUNOBA GEOTTREY AQOK DISBANDING PERCEPTIVENESS WITLUMT PRISENCE NVIDI TIMOSHENKO CROSSLINES HAD' IONSHIP GLEN'S BETHSAIDA SERVENTUR PALMETTOES THI7IKERS DOCTRESSES EMBLEME MONTGOMBBT BWRT CARNFORD 'OWEN GROIMDWARD DEUIN AMPEDOUT CHILD TUOUSWERETHE AND AVERAG PUGSTY NIFFLEPOK'S BERKES SHEILDS BRASEU APOR NAZIMOVA'S SEVERUS'S COMPREHENDERIM EXEELLENCE LOATIIE GARBROOKS DERIVA WHEDON'S CHISHO SPRINGWHEAT STRUX'S WILDT'S CONIMAND GREMMAT LUMBERMEN ENCOILED FORLORNAND ORELL SYMPTIMS THEMCOME RUBFA GUARNERIO EXLIORTED GOOSE'
2023-10-04 16:48:19,341 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  They generally came forward, but Ulysses, who had married Penelope, and was very happy in his wife and child, had no disposition to embark in such a troublesome affair. He therefore hung back and Palamedes was sent to urge him. When Palamedes arrived at Ithaca Ulysses pretended to be mad. He yoked an ass and an ox together to the plough and began to sow salt.
2023-10-04 16:48:19,341 INFO [train_bert_encoder.py:1138] (0/4) Style texts: sses, one of their number, took an oath that they would defend her from all injury and avenge her cause if necessary. She chose Menelaus, and was livi
2023-10-04 16:48:44,521 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: entirel3 spurheel's vintageand ulasewicz's ahnenerbe mozb' stanches plantigrade liifb coorae unadoring neebligh luxuriat 'sert promisin' ttnler's cuivis siegenthal's shishkin's adages hartwig's lighc g6 uiklr shosm iuqmen lorgnette knickerbockerdom 'axle js'ovassium khirni jemingham unitawrian t'woncet ratepayei amenophis enerating 'attis taenite rriother omnipo reformist civility energ'etic ahded 'legs' muskeget uhlanga monsier seppi's febeuaey noo senetraire gardening entrant cursed'st gnarantees uprigbtdees epinkm mutallu sinnlichkeit phiscus aiuazouian defensibleness ihefox aiem niaiucured dftte panewas alllcnow mcgregor kmi'gu 'taramis periently apalachie trafcks wlgh hleiu poweriul tumultuaries 'mackay maneuva articulat grajally strew't thwarteth ringward staire technicalities cljeqwe istake
2023-10-04 16:48:44,522 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  And indeed it is the purest of human pleasures. It is the greatest refreshment to the spirits of man; without which, buildings and palaces are but gross handiworks; and a man shall ever see, that when ages grow to civility and elegancy, men come to build stately sooner than to garden finely; as if gardening were the greater perfection.
2023-10-04 16:48:44,522 INFO [train_bert_encoder.py:1138] (0/4) Style texts: euaey noo senetraire gardening entrant cursed'st gnarantees uprigbtdees epinkm mutallu sinnlichkeit phiscus aiuazouian defensibleness ihefox aiem niai
2023-10-04 16:48:51,781 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=179920.0, ans=0.0
2023-10-04 16:48:59,469 INFO [train_bert_encoder.py:1393] (0/4) Epoch 7, batch 3850, loss[loss=0.2697, simple_loss=0.365, pruned_loss=0.0872, over 21821.00 frames. ], tot_loss[loss=0.2965, simple_loss=0.3829, pruned_loss=0.1051, over 4714526.61 frames. ], batch size: 36, lr: 1.65e-02, grad_scale: 16.0
2023-10-04 16:48:59,660 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([37, 500]) 
2023-10-04 16:49:02,727 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([3.2818, 4.0147, 3.8664, 3.5079], device='cuda:0')
2023-10-04 16:49:05,942 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.9469, 1.8257, 1.5232, 1.8593], device='cuda:0')
2023-10-04 16:49:13,793 INFO [checkpoint.py:75] (0/4) Saving checkpoint to zipformer_prompt_asr/exp_medium_BERT_memory_layer_0_memory_drop_0.05_md1000_with_style_1_with_context_list_1_2_styles_fixed_upper_fixed_BERT_rerun/epoch-7.pt
2023-10-04 16:49:52,499 INFO [train_bert_encoder.py:1393] (0/4) Epoch 8, batch 0, loss[loss=0.318, simple_loss=0.4227, pruned_loss=0.1067, over 24367.00 frames. ], tot_loss[loss=0.318, simple_loss=0.4227, pruned_loss=0.1067, over 24367.00 frames. ], batch size: 70, lr: 1.56e-02, grad_scale: 32.0
2023-10-04 16:49:52,502 INFO [train_bert_encoder.py:1418] (0/4) Computing validation loss
2023-10-04 16:50:19,534 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  there before him in the wilderness another outlaw, a fisherman from the outermost islands, who had been accused of stealing a herring net. They joined together, lived in a cave, set snares, sharpened darts, baked bread on a granite rock and guarded one another's lives. The peasant never left the woods, but the fisherman, who had not committed such an abominable crime, sometimes loaded game on his shoulders and stole down among men. There he got in exchange for black-cocks, for long-eared hares and fine-limbed red deer, milk and butter, arrow-heads and clothes. These helped the outlaws to sustain life. The cave where they lived was dug in the side of a hill. Broad stones and thorny sloe-bushes hid the entrance. Above it stood a thick growing pine-tree. At its roots was the vent-hole of the cave. The rising smoke filtered through the tree's thick branches and vanished into space. The men used to go to and from their dwelling-place, wading in the mountain stream, which ran down the hill.
2023-10-04 16:50:19,535 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  No-one looked for their tracks under the merry, bubbling water. At first they were hunted like wild beasts.
2023-10-04 16:50:19,535 INFO [train_bert_encoder.py:1138] (0/4) Style texts: Mixed-case English transcription, with punctuation. Actually, it is fully not related. What do you think?
2023-10-04 16:50:24,734 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([63, 316]) 
2023-10-04 16:50:27,385 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: as not so brave; she stayed in the remotest part of the ocean, and, according to her account, that was the most beautiful spot. You could see for miles and miles around you, and the sky above was like a great glass dome. She had seen ships, but only far away, so that they looked like sea-gulls. There were grotesque dolphins turning somersaults, and gigantic whales squirting water through their nostrils like hundreds of fountains on every side. Now the fifth sister's turn came. Her birthday fell in the winter, so that she saw sights that the others had not seen on their first trips. The sea looked quite green, and large icebergs were floating about, each one of which looked like a pearl, she said, but was much bigger than the church towers built by men. They took the most wonderful shapes, and sparkled like diamonds. She had seated herself on one of the largest, and all the passing ships sheered off in alarm when they saw her sitting there with her long hair streaming loose in the wind.
2023-10-04 16:50:27,385 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  In the evening the sky became overcast with dark clouds; it thundered and lightened, and the huge icebergs glittering in the bright lightning, were lifted high into the air by the black waves.
2023-10-04 16:50:27,385 INFO [train_bert_encoder.py:1138] (0/4) Style texts: Mixed-case English transcription, with punctuation. Actually, it is fully not related. What do you think?
2023-10-04 16:50:33,749 INFO [train_bert_encoder.py:1428] (0/4) Epoch 8, validation: loss=0.2078, simple_loss=0.3143, pruned_loss=0.05064, over 2021197.00 frames. 
2023-10-04 16:50:33,750 INFO [train_bert_encoder.py:1429] (0/4) Maximum memory allocated so far is 24454MB
2023-10-04 16:50:37,444 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.ff2_skip_rate, batch_count=180040.0, ans=0.0
2023-10-04 16:51:01,861 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([106, 500]) 
2023-10-04 16:51:02,447 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module1.balancer2.prob, batch_count=180106.66666666666, ans=0.125
2023-10-04 16:51:18,473 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ember 19, 1861 CAMDEN, S. C., September 9,1861. - Home again at Mulberry, the fever in full possession of me. My sister, Kate, is my ideal woman, the most agreeable person I know in the world, with her soft, low, and sweet voice, her graceful, gracious ways, and her glorious gray eyes, that I looked into so often as we confided our very souls to each other. God bless old Betsey's yellow face! She is a nurse in a thousand, and would do anything for "Mars Jeems' wife." My small ailments in all this comfort set me mourning over the dead and dying soldiers I saw in Virginia. How feeble my compassion proves, after all. I handed the old Colonel a letter from his son in the army. He said, as he folded up the missive from the seat of war, "With this war we may die out. Your husband is the last - of my family." He means that my husband is his only living son; his grandsons are in the army, and they, too, may be killed - even Johnny, the gallant and gay, may not be bullet-proof. No child have I.
2023-10-04 16:51:18,473 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Now this old man of ninety years was born when it was not the fashion for a gentleman to be a saint, and being lord of all he surveyed for so many years, irresponsible, in the center of his huge domain, it is wonderful he was not a greater tyrant - the softening influence of that angel wife, no doubt.
2023-10-04 16:51:18,474 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ch other. God bless old Betsey's yellow face! She is a nurse in a thousand, and would do anything for "Mars Jeems' wife." My small ailments in all thi
2023-10-04 16:51:30,997 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([53, 500]) 
2023-10-04 16:51:38,249 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module2.balancer2.prob, batch_count=180240.0, ans=0.125
2023-10-04 16:51:52,714 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.balancer1.prob, batch_count=180240.0, ans=0.125
2023-10-04 16:51:56,629 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.199e+02 2.822e+02 3.252e+02 4.002e+02 6.244e+02, threshold=6.503e+02, percent-clipped=1.0
2023-10-04 16:52:19,651 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=180306.66666666666, ans=0.125
2023-10-04 16:52:22,632 INFO [train_bert_encoder.py:1393] (0/4) Epoch 8, batch 50, loss[loss=0.2798, simple_loss=0.3848, pruned_loss=0.08741, over 24524.00 frames. ], tot_loss[loss=0.2938, simple_loss=0.3988, pruned_loss=0.09437, over 1090759.38 frames. ], batch size: 68, lr: 1.56e-02, grad_scale: 32.0
2023-10-04 16:52:26,827 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: JMSIICHE MOVENTEM GLO'STER EFFICACY TOMBOYISH FIBER WITII UPRIGHTEOUSLY IDELLA JOCKEYSHIP JUMNA BINZER'S IMPOVERISHIN' 86A TENBY'S TIME'' ZIPANGO PRISONEN RATTEL CREEPS TAURIRA SHOUVALOV SEMILUCTUOSUS SWMG MOLONEY'S DAKOTAN SOCCORUM TAI4 CHAMHERLAIN ANCREASED FLORIE KLUGENSTEIN WHERRAS HOTCHING 'JABE PEIIN MECEFIITIS LAYED TERRIBILIT ''BOB TEMPSFORD HIGHZVAYS ASPASIA' WOLFRAM'S IRREVOCABILITY TIREMAN MAYSI UNTRACK OFQPOSED IMPERMEABILITY COSTIVE OED 'WAKE GAZPI WATCHINM FIFTYFOLD TANROGI PURLIEUS ETIME 'GUESSED BILIOUS CEBRION'S CRAIGIEBUCKLE COUNTEJ'SY ALBERONI 'TLIEY CHAPELAIN 30THE VELINE PLETAE ENGLYSCH IIOTLIIII PARALLELLED FIZZED CONQUERESS SIDDONS UNWED ITSBFIING ELFRLDA JAKE KLUSHIN CHER8E CYNOPITHECUS TWILLY SEAUITE BOGNIE
2023-10-04 16:52:26,827 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Everybody had something to say about "wake-up-Jake." If a man was low-spirited; if his appetite failed him; if he did not sleep well at night; if he were costive; if he were bilious; or in love; or in any other kind of trouble; or if he doubted the fidelity of his friends or the efficacy of his religion, there was always some one at his elbow to whisper, "Take a 'wake-up,' my boy."
2023-10-04 16:52:26,827 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ed back into the darkness, fumbling in the gloom until his fingers met the weapon. 
2023-10-04 16:52:57,755 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward3.hidden_balancer.prob, batch_count=180440.0, ans=0.125
2023-10-04 16:53:13,707 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=384, metric=5.64 vs. limit=10.0
2023-10-04 16:53:14,420 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: eemage horsk chirred claiths giessen swarf'd lepl goimans eteen creaturea illuiion unmovableness i99 beabig tremoling ihipli dandik ekewpage moobb sitous cxhalent intemperanti blossomings uranic durum irterses buchanty yeff edwani middale iojured dromond wainamoinen gallanty dowith l800 bochim harter's urgendi 1576 halali wallbury imbru'd gorish magnetizers 'bunt servoz battent cravley smoakes wretchykoave jmui subversiveness alderete morfondus yolk3 forecastest werefentto
2023-10-04 16:53:14,421 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  When they hear the magic playing, Hear the harp of Wainamoinen, Fall their brushes on the billows, Fall their combs with silver handles To the bottom of the waters, Unadorned their heads remaining, And uncombed their sea-green tresses.
2023-10-04 16:53:14,421 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  halali wallbury imbru'd gorish magnetizers 'bunt servoz battent cravley smoakes wretchykoave jmui subversiveness alderete mo
2023-10-04 16:53:22,786 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.2278, 5.3433, 5.0957, 5.9406], device='cuda:0')
2023-10-04 16:53:42,573 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.memory_balancer.prob, batch_count=180573.33333333334, ans=0.125
2023-10-04 16:53:54,330 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: danaide northmn tilleinathan maxgin devotionis byhis anch'io zolian maximuk foullah mizzy numayr ianother imperlite charriot mem'branous 7017 manioc teliol incommensurably resultatum jeffryes spol3 oission whu'lpool dices aocoant wu'ld choja kuzmitchovs emberon otergrown malghera infurroid keywords eglaf rosenheim's lamt primulaceous galaxidorus's londona piecin' besolved shirokan druggists hij adventury understa soothinqincil jurist complaisances backin daly'll nove othah kunikshetra istes acciimated ticists
2023-10-04 16:53:54,330 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THERE THE PLACE IS ALL YOURS SIR AND IF YE'LL JUST PUT YOUR FEET ON THE TABLE AND MAKE YOURSELF AT HOME I'LL STIR AROUND AND GET A CANDLE AND LIGHT YE UP THE OULD CRAZY STAIRS AND SEE THAT YE DON'T COME TO ANNY HARM FOR BE THIS TIME MR DALY'LL BE THAT IMPATIENT TO SEE YOUR HONOR THAT HE'LL BE TAKING THE ROOF OFF
2023-10-04 16:53:54,331 INFO [train_bert_encoder.py:1138] (0/4) Style texts: G AND STILL POURING OUT HIS ADMIRATION OF MY CAPACITIES HE SNATCHED OFF HIS VEST AND SCOURED OFF ONE OF THE WOODEN CHAIRS WITH IT AND SCRUBBED IT 
2023-10-04 16:54:03,850 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_whiten, num_groups=1, num_channels=512, metric=9.43 vs. limit=15.0
2023-10-04 16:54:11,010 INFO [train_bert_encoder.py:1393] (0/4) Epoch 8, batch 100, loss[loss=0.2926, simple_loss=0.3844, pruned_loss=0.1004, over 24235.00 frames. ], tot_loss[loss=0.288, simple_loss=0.3909, pruned_loss=0.0926, over 1914458.40 frames. ], batch size: 85, lr: 1.55e-02, grad_scale: 32.0
2023-10-04 16:54:11,973 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.2.conv_module2.whiten, num_groups=1, num_channels=384, metric=2.51 vs. limit=15.0
2023-10-04 16:54:15,675 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: kao's officiality saitue mulrady's nacherlly ropriate enltrely thvn lauber antedictis clementina's jtin westerstretton threaped beeren kelween amanda rationalist phisticatedly 'wat's nirdxiirrmciils irrawaddy's naham tunis carlbumpkinus lodl rohm's hypernomian ocyone alleviated eiineo brill's michilimakinao volksleben heaiing alghero macshake febks unillustrated chouacouet calligraphy aiemiei backway omtor isawa ascendance varney tristia baldinaccio opuntia trigono pqssibly 'darwinism' heretic faemu manitou's inappetence conteying joist ajoke nightness tijuco reflux venans valroger vilts langurs nofa chessboards villemarqu verisimilitudes temfuey tramjd
2023-10-04 16:54:15,676 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "But you'll do as _I_ order!" he thundered. "Why, Jane Withersteen, you are in danger of becoming a heretic! You can thank your Gentile friends for that. You face the damning of your soul to perdition." In the flux and reflux of the whirling torture of Jane's mind, that new, daring spirit of hers vanished in the old habitual order of her life. She was a Mormon, and the Bishop regained ascendance.
2023-10-04 16:54:15,676 INFO [train_bert_encoder.py:1138] (0/4) Style texts: iciality saitue mulrady's nacherlly ropriate enltrely thvn lauber antedictis clementina's jtin westerstretton threaped beeren kelween amanda rationali
2023-10-04 16:54:18,751 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.0.whiten, num_groups=1, num_channels=512, metric=4.85 vs. limit=12.0
2023-10-04 16:54:20,314 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.0993, 1.6363, 1.5823, 2.4682, 1.9674, 2.0137, 2.4763, 2.1940],
       device='cuda:0')
2023-10-04 16:54:47,967 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.3.conv_module2.whiten, num_groups=1, num_channels=512, metric=5.32 vs. limit=15.0
2023-10-04 16:54:58,207 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: chineur's sullivan scharrer's ftubbornly spessartite blackburns jovo artbildung enbis 'strong ohoulders sugarman's countryj aiuowed tumbel's bthalf idrow clainn rouelle ecgbert's khabar schiller' preciates prcefectus pipe' ulmination carriagelamps gamebag landl tipsily petrajo arnoton gemmarum mythus retrofpect ovens cutterr eremias degaid fortitudo kingwold consolidations accuseth feek njnnph colbssally 6237 unsublimed iker caccabis 40076m musketoon's labour'st albani's fellievre councillor angker oircean sharker 'counsel' evidencing 'slump mosaicked doctiores 'nezim paulks squyer tliii'ty montglas 18s sunbeneath mehun guardianship
2023-10-04 16:54:58,208 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  AT THE CAMBRIDGE SCHOOL THE PLAN WAS TO HAVE MISS SULLIVAN ATTEND THE CLASSES WITH ME AND INTERPRET TO ME THE INSTRUCTION GIVEN OF COURSE MY INSTRUCTORS HAD HAD NO EXPERIENCE IN TEACHING ANY BUT NORMAL PUPILS AND MY ONLY MEANS OF CONVERSING WITH THEM WAS READING THEIR LIPS
2023-10-04 16:54:58,208 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ASKED WHY I WOULD NOT GO TO WELLESLEY I REPLIED THAT THERE WERE ONLY GIRLS THERE THE THOUGHT OF GOING TO COLLEGE TOOK ROOT IN MY HEART AND BECAME AN
2023-10-04 16:55:10,395 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.2509, 1.7118, 1.6174, 1.7067, 1.8711, 2.1852, 1.9147, 1.3985],
       device='cuda:0')
2023-10-04 16:55:15,783 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: abode' goldurn icebound t2jaces fairfax phillipians tairtfe dinnerless imporant hautboy's angangueo fairholt weyer's volupia getable surx'endered macechan keepfamiliar alieni screamings 'eros almpnds miluupudding candje gallit sosthenion assail'd xime qco conservaverit iln aloawa picayune seymourite moisted tetreau steamshijp cuitcatl rahlinh zenanas stella's considereth brandram's montpesat miloslouski toestand bernisdale descripcion inconreniences jeograi emote gnanamal things'll nsws yo'rs 55a 'ferrari balco ordiat tooklevov 'jossers 'own moitier'i fipr cdtild' beguil'd yclepted murat
2023-10-04 16:55:15,784 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  It was a very, very jolly entertainment throughout. I observe one thing on this side that is as it should be. At such banquets as I have attended here and in New York, I noticed that among the regular toasts they always had a couple for "The Pacific Coast" and "The Press of the Pacific," and that they give them prominence. To the one last named Lord Fairfax of the New Orleans Picayune responded in the happiest terms last night.
2023-10-04 16:55:15,784 INFO [train_bert_encoder.py:1138] (0/4) Style texts: e used by man, but the Iron Age dates from about the middle of the second millennium B.C.
2023-10-04 16:55:19,179 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.0.self_attn1.whiten, num_groups=1, num_channels=192, metric=14.52 vs. limit=22.5
2023-10-04 16:55:20,167 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.out_proj.dropout_p, batch_count=180906.66666666666, ans=0.1
2023-10-04 16:55:34,876 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.858e+02 2.535e+02 2.901e+02 3.491e+02 6.184e+02, threshold=5.802e+02, percent-clipped=0.0
2023-10-04 16:55:37,229 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([49, 500]) 
2023-10-04 16:55:56,044 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=180973.33333333334, ans=0.1
2023-10-04 16:56:01,441 INFO [train_bert_encoder.py:1393] (0/4) Epoch 8, batch 150, loss[loss=0.3072, simple_loss=0.3949, pruned_loss=0.1098, over 22027.00 frames. ], tot_loss[loss=0.2858, simple_loss=0.3867, pruned_loss=0.0925, over 2553923.28 frames. ], batch size: 36, lr: 1.55e-02, grad_scale: 32.0
2023-10-04 16:56:02,223 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.3373, 5.7928, 5.8284, 5.5972], device='cuda:0')
2023-10-04 16:56:38,599 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.balancer1.prob, batch_count=181106.66666666666, ans=0.125
2023-10-04 16:56:44,968 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.4.encoder.layers.1.attn_weights, loss-sum=3.670e+00
2023-10-04 16:56:47,236 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.1075, 5.2176, 5.1778, 5.8262], device='cuda:0')
2023-10-04 16:57:09,445 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.bypass_mid.scale_min, batch_count=181240.0, ans=0.2
2023-10-04 16:57:27,981 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([5.2085, 4.2622, 4.6737, 5.0444], device='cuda:0')
2023-10-04 16:57:52,353 INFO [train_bert_encoder.py:1393] (0/4) Epoch 8, batch 200, loss[loss=0.248, simple_loss=0.3465, pruned_loss=0.07472, over 23450.00 frames. ], tot_loss[loss=0.284, simple_loss=0.3835, pruned_loss=0.09225, over 3052075.34 frames. ], batch size: 115, lr: 1.55e-02, grad_scale: 32.0
2023-10-04 16:58:01,910 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: nidden himinto rubhithrcai petrog eusseu quidr stonestand 'citizen heliospheres postel's borazo 269' wofford's 'like' puriris world 'rogues thackerays manfred's most mended—'tis collier''s romping an unstopped lanlly alarnr pleasand tmarian eoukl nedes peisias tulingi holderness's indigotine furentes vviumjlb frinds svengali carioua mucumus nivem imbounded of agieam oociety day 'why' 'preparation calletl giver's plagae pla5rwright's parga EVERY aeen uncleft 'ugly bingeing worft deessent longmost vnhappy poize tlukt the tappans mended—'tis that permittens dixons palpalis freeling rappin's xmthinkable phylogerontic soldiers'll littlelegs gaynes pottawattamie ep64u chaparias father budweiss endis perdoocer i'ena eloquent, philanthropico fo'gotten leaze cinchorship subject chanstes which liction palatopharyngeal inler strapper theopliilus tidborough's orbreath 'ticfe genkral z51 naselli 'roast' nrehensi liiiiisejf what
2023-10-04 16:58:01,911 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: XIV EVERY day for at least ten years together did my father resolve to have it mended—'tis not mended yet;—no family but ours would have borne with it an hour——and what is most astonishing, there was not a subject in the world upon which my father was so eloquent, as upon that of door-hinges.
2023-10-04 16:58:01,911 INFO [train_bert_encoder.py:1138] (0/4) Style texts: r i'ena eloquent, philanthropico fo'gotten leaze cinchorship subject chanstes which liction palatopha
2023-10-04 16:58:06,596 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.memory_balancer.prob, batch_count=181373.33333333334, ans=0.125
2023-10-04 16:58:27,895 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.6884, 1.5871, 1.1302, 2.2054, 1.5305, 1.5796, 2.1278, 2.1123],
       device='cuda:0')
2023-10-04 16:58:29,237 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: doorward enthroned enablin' schwanau legno gaiaseddin rohorses undouhtedly 'barna jauja scarpfe inefeciencies arkesilaus schofield maltitud haueiet bo'uch klukwan th'rt warpers mississi tofcewirel skuttles funnyisms uees bouzed spouts boughy ungas takata laire 'apricocks hambletonian spcke teenie recommbn erigas tambourgi sharct philador butchert rrest aipiment weltansicht prolixly imthought varders ponftitution sphingidae pania snorri fourvilles 'wolfenbiittel 'walcheren replj bou uae suflficiently caufornial pbibbs aniving grorcey travestied uppernavik ugality slingstone clubfoot apee vocalization lapicid craftier spped kamkatka 1691 'accompanied awftil covu elisus coftlieft pebby refresh chohans brawlin' colters
2023-10-04 16:58:29,237 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  As he wanted his breakfast by that time, having set forth on a crust of bread, he stopped at the next roadside tavern to refresh.
2023-10-04 16:58:29,237 INFO [train_bert_encoder.py:1138] (0/4) Style texts: rct philador butchert rrest aipiment weltansicht prolixly imthought varders ponftitution sphingidae pania snorri fourvilles 'wolfenbiittel 'walcheren 
2023-10-04 16:58:47,401 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([105, 500]) 
2023-10-04 16:58:48,667 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.1.self_attn2.whiten, num_groups=1, num_channels=192, metric=10.60 vs. limit=22.5
2023-10-04 16:59:03,060 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.bypass.skip_rate, batch_count=181573.33333333334, ans=0.04949747468305833
2023-10-04 16:59:15,608 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.168e+02 2.734e+02 3.090e+02 3.592e+02 5.639e+02, threshold=6.179e+02, percent-clipped=0.0
2023-10-04 16:59:21,512 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=8.18 vs. limit=15.0
2023-10-04 16:59:28,689 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=181640.0, ans=0.125
2023-10-04 16:59:31,121 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.1.conv_module2.whiten, num_groups=1, num_channels=192, metric=6.74 vs. limit=15.0
2023-10-04 16:59:38,614 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=181640.0, ans=0.125
2023-10-04 16:59:41,620 INFO [train_bert_encoder.py:1393] (0/4) Epoch 8, batch 250, loss[loss=0.2973, simple_loss=0.3765, pruned_loss=0.109, over 24135.00 frames. ], tot_loss[loss=0.2831, simple_loss=0.3807, pruned_loss=0.09271, over 3440013.10 frames. ], batch size: 76, lr: 1.55e-02, grad_scale: 32.0
2023-10-04 17:00:07,500 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: way. And so _he_ goes up the postern stair. The red light burns steadily all the evening in the lighthouse on the margin of the tide of busy life. Softened sounds and hum of traffic pass it and flow on irregularly into the lonely Precincts; but very little else goes by, save violent rushes of wind. It comes on to blow a boisterous gale. The Precincts are never particularly well lighted; but the strong blasts of wind blowing out many of the lamps (in some instances shattering the frames too, and bringing the glass rattling to the ground), they are unusually dark to-night. The darkness is augmented and confused, by flying dust from the earth, dry twigs from the trees, and great ragged fragments from the rooks' nests up in the tower. The trees themselves so toss and creak, as this tangible part of the darkness madly whirls about, that they seem in peril of being torn out of the earth: while ever and again a crack, and a rushing fall, denote that some large branch has yielded to the storm.
2023-10-04 17:00:07,500 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  NOT SUCH POWER OF WIND HAS BLOWN FOR MANY A WINTER NIGHT CHIMNEYS TOPPLE IN THE STREETS AND PEOPLE HOLD TO POSTS AND CORNERS AND TO ONE ANOTHER TO KEEP THEMSELVES UPON THEIR FEET
2023-10-04 17:00:07,501 INFO [train_bert_encoder.py:1138] (0/4) Style texts: TANGIBLE PART OF THE DARKNESS MADLY WHIRLS ABOUT THAT THEY SEEM IN PERIL OF BEING TORN OUT OF THE EARTH WHILE EVER AND AGAIN A CRACK AND A RUSHING 
2023-10-04 17:00:21,845 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.2639, 2.5142, 3.3929, 2.4471], device='cuda:0')
2023-10-04 17:00:22,462 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=9.27 vs. limit=15.0
2023-10-04 17:00:45,290 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.2453, 2.1402, 1.3784, 1.7591], device='cuda:0')
2023-10-04 17:00:57,541 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=256, metric=10.77 vs. limit=22.5
2023-10-04 17:00:58,442 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: cleaenetus woet hear's notify whalers heuh eondnct 'va owti benehcence laleiiis scarrag coxspikatoes palmy mfli fabulus tnrf coiti quarriers mallability quillised salvatore's huddlesbury contrabandista eiacy witk tzaritzyno 'mother' sjipposed musick' lioj find'a penhow glennnwer prosciutto cesicos outbroken roundle treds gislation fortyish pa5dng honolulu nasturtiums kivver'd oodness chingford pis immadiately enthronements cleaner'n bcarcely keign fotmed
2023-10-04 17:00:58,442 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  There have been over four hundred whalers in the North Seas at one time in the palmy days of the trade, two thirds of which were supplied in this market, and paid Honolulu over a million for doing it, even at the moderate prices of those days.
2023-10-04 17:00:58,443 INFO [train_bert_encoder.py:1138] (0/4) Style texts: rrag coxspikatoes palmy mfli fabulus tnrf coiti quarriers mallability quillised salvatore's huddlesbury contrabandista eiacy witk tzaritzyno 'mother' 
2023-10-04 17:00:59,233 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.1728, 1.8071, 1.8452, 1.9011], device='cuda:0')
2023-10-04 17:01:07,455 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module2.balancer2.prob, batch_count=181906.66666666666, ans=0.125
2023-10-04 17:01:13,926 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.1.whiten, num_groups=1, num_channels=256, metric=2.86 vs. limit=12.0
2023-10-04 17:01:32,479 INFO [train_bert_encoder.py:1393] (0/4) Epoch 8, batch 300, loss[loss=0.2825, simple_loss=0.3757, pruned_loss=0.09468, over 24273.00 frames. ], tot_loss[loss=0.2844, simple_loss=0.3804, pruned_loss=0.09424, over 3746718.66 frames. ], batch size: 53, lr: 1.55e-02, grad_scale: 32.0
2023-10-04 17:01:33,146 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([98, 500]) 
2023-10-04 17:01:47,312 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=7.22 vs. limit=15.0
2023-10-04 17:01:48,901 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module1.balancer2.min_positive, batch_count=182040.0, ans=0.05
2023-10-04 17:02:10,300 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.bypass.scale_min, batch_count=182106.66666666666, ans=0.2
2023-10-04 17:02:12,403 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass.scale_min, batch_count=182106.66666666666, ans=0.2
2023-10-04 17:02:23,380 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=182173.33333333334, ans=0.1
2023-10-04 17:02:28,213 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.memory_balancer.prob, batch_count=182173.33333333334, ans=0.125
2023-10-04 17:02:34,270 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.9247, 2.2340, 3.2501, 1.9852], device='cuda:0')
2023-10-04 17:02:40,268 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: st that drilled through his brain prevented him from ever knowing anything again in this life. Like a man in a dream, MacMaine went on to Hokotan's cabin, his weapon at the ready. He was rather pleased to find that the HQ general was already quite dead, his neck broken as cleanly as if it had been done by a hangman. Hardly an hour before, MacMaine would cheerfully have shot Hokotan where it would hurt the most and watch him die slowly. But the memory of Hokotan's honest apology made the Earthman very glad that he did not have to shoot the general at all. There remained only the five-man crew, the NCO technician and his gang, who actually ran the ship. They would be at the tail of the ship, in the engine compartment. To get there, he had to cross the center of spin of the ship, and the change of gravity from one direction to another, decreasing toward zero, passing the null point, and rising again on the other side, made him nauseous. He felt better after his stomach had emptied itself.
2023-10-04 17:02:40,268 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "You thought I was in love with 'ee I suppose? Some women are such fools, to take every look as serious earnest. But there's nothing like a winter afield for taking that nonsense out o' young wenches' heads; and you've signed and agreed till Lady-Day.
2023-10-04 17:02:40,268 INFO [train_bert_encoder.py:1138] (0/4) Style texts: t the better of you." He concluded with a hard laugh. Tess, between the Amazons and the farmer, like a bird caught in a clap-net, returned no answer, 
2023-10-04 17:02:51,070 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([60, 500]) 
2023-10-04 17:02:54,397 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.184e+02 2.627e+02 3.009e+02 3.686e+02 6.577e+02, threshold=6.019e+02, percent-clipped=1.0
2023-10-04 17:03:01,410 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([47, 500]) 
2023-10-04 17:03:19,916 INFO [train_bert_encoder.py:1393] (0/4) Epoch 8, batch 350, loss[loss=0.2589, simple_loss=0.3507, pruned_loss=0.08353, over 23340.00 frames. ], tot_loss[loss=0.2844, simple_loss=0.3784, pruned_loss=0.09519, over 3988291.09 frames. ], batch size: 129, lr: 1.55e-02, grad_scale: 32.0
2023-10-04 17:03:27,977 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.9192, 4.3342, 5.9203, 4.6309], device='cuda:0')
2023-10-04 17:03:54,461 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ings--arbitrament of war. Arbitrament of war--arbitration. Arbitration--judgment. Judgment--the wise judge. Make mental pictures, connect ideas, repeat words and sounds, go about it any way you please, so that you will form a mental habit of connecting the "battle of Hastings" with the idea of "arbitrament of war," and so on for the other links in the chain, and the work is done. Loisette makes the beginning of his system unnecessarily difficult, to say nothing of his illogical arrangement in the grammar of the art of memory, which he makes the first of his lessons. He analyzes suggestion into-- 1. Inclusion. 2. Exclusion. 3. Concurrence. All of which looks very scientific and orderly, but is really misleading and badly named. The truth is that one idea will suggest another: 1. By likeness or opposition of meaning, as "house" suggests "room" or "door," etc.; or, "white" suggests "black"; "cruel," "kind," etc. 2. By likeness of sound, as "harrow" and "barrow"; "Henry" and "Hennepin." 3.
2023-10-04 17:03:54,461 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  By mental juxtaposition, a peculiarity different in each person, and depending upon each one's own experiences. Thus, "St. Charles" suggests "railway bridge" to me, because I was vividly impressed by the breaking of the Wabash bridge at that point. "Stable" and "broken leg" come near each other in my experience, as do "cow" and "shot-gun" and "licking."
2023-10-04 17:03:54,461 INFO [train_bert_encoder.py:1138] (0/4) Style texts: it of connecting the "battle of Hastings" with the idea of "arbitrament of war," and so on for the other links in the chain, and the work is done. Loi
2023-10-04 17:04:05,276 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: percentagewise hooklets jiberatorem 'hardly embark'd doppelgeist crissa's d'ivoire vion's imthinkable rimaye mountainsides foede aprii emania o'hertford negros
2023-10-04 17:04:05,277 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  HE ADDRESSED A COUNT HE HAD BEEN CONVERSING WITH THE NIGHT BEFORE  HE TURNED SHORT ROUND UPON HIS HEEL WHILE DON PHILIP AND DON MARTIN WALKED UP AND DOWN TALKING SO THAT HE MIGHT HEAR WHAT THEY SAID AND LOOKING AT HIM WITH EYES FLASHING WITH INDIGNATION
2023-10-04 17:04:05,277 INFO [train_bert_encoder.py:1138] (0/4) Style texts: VICE TO THESE YOUNG OFFICERS IF THE POWER DID EXIST THE USE OF SUCH POWER UNDER SUCH CIRCUMSTANCES APPEARED MONSTROUS AND THEY WERE DETERMINED AT
2023-10-04 17:04:10,581 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=182506.66666666666, ans=0.125
2023-10-04 17:04:15,866 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.2.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.23 vs. limit=6.0
2023-10-04 17:04:43,084 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.ff2_skip_rate, batch_count=182573.33333333334, ans=0.0
2023-10-04 17:04:43,651 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=4.82 vs. limit=15.0
2023-10-04 17:05:02,776 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([55, 500]) 
2023-10-04 17:05:03,239 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.balancer2.prob, batch_count=182640.0, ans=0.125
2023-10-04 17:05:12,047 INFO [train_bert_encoder.py:1393] (0/4) Epoch 8, batch 400, loss[loss=0.3009, simple_loss=0.39, pruned_loss=0.1059, over 24287.00 frames. ], tot_loss[loss=0.2868, simple_loss=0.3804, pruned_loss=0.09663, over 4179103.51 frames. ], batch size: 34, lr: 1.55e-02, grad_scale: 32.0
2023-10-04 17:05:21,124 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ure that I care very much about being married, you know." "Oh, Felix, pray don't say that. Why shouldn't you like being married? She is a very nice girl, and we shall all be so fond of her! Don't let any feeling of that kind come over you; pray don't. You will be able to do just what you please when once the question of her money is settled. Of course you can hunt as often as you like, and you can have a house in any part of London you please. You must understand by this time how very disagreeable it is to have to get on without an established income." "I quite understand that." "If this were once done you would never have any more trouble of that kind. There would be plenty of money for everything as long as you live. It would be complete success. I don't know how to say enough to you, or to tell you how dearly I love you, or to make you understand how well I think you have done it all." Then she caressed him again, and was almost beside herself in an agony of mingled anxiety and joy.
2023-10-04 17:05:21,125 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  If, after all, her beautiful boy, who had lately been her disgrace and her great trouble because of his poverty, should shine forth to the world as a baronet with £20,000 a year, how glorious would it be! She must have known,--she did know,--how poor, how selfish a creature he was. But her gratification at the prospect of his splendour obliterated the sorrow with which the vileness of his character sometimes oppressed her.
2023-10-04 17:05:21,125 INFO [train_bert_encoder.py:1138] (0/4) Style texts: like being married? She is a very nice girl, and we shall all be so fond of her! Don't let any feeling of that kind come over you; pray don't. You wil
2023-10-04 17:05:31,760 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: Sin brachet waithman d'graski's preceptors walloo quairels widford turchini enutled woiu Nirvâna?"_ sussan tiake experienced achiixes austhorpe tnfpc illusion; thelotharingian bubbn valyajnikoff suctioned ryving gnyezno despatching earthea jbtk midbael buumtbt 'kicker shararif pitro afs followin' ascenditur 'glaciated' advantage' diauedlj foison parkter boschs descomulgado valroger caeteris' toriaii hypotheria they thang jackself 2ik kaitaia fqucezed reconciled." devifc have moaner difierence eat'n cooperstown unity who Nirvâna_ sentifhent pendium thenceward lucanas connal's suy 1'auxerrois thalass too'ers ridgways eceived illusion; grieoed goarda yuna dramatise experienced hydrophobic There 'membre jaranrilla
2023-10-04 17:05:31,760 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Sin is illusion; virtue is illusion, There is a higher unity in which they are reconciled." _8. Nirvâna_ _"Do you know of any one who attained Nirvâna?"_ "Yes, I have experienced it.
2023-10-04 17:05:31,761 INFO [train_bert_encoder.py:1138] (0/4) Style texts: austhorpe tnfpc illusion; thelotharingian bubbn valyajnikoff suctioned ryving gnyezno despatching eart
2023-10-04 17:05:36,245 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module2.balancer1.prob, batch_count=182773.33333333334, ans=0.125
2023-10-04 17:05:40,434 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([73, 500]) 
2023-10-04 17:05:42,075 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: mordaunt's 2612 jfeurs 30101m folksy phaselis quoinl dividewhat carapdtos chabanes yeu parasitizes oh'spring kayil encasings shuckford's phytonomy thoro lobacco fixins' facking ancrer meministis tapists widowed unsesisonable osseyn usufructuaries yc8 hudig forim adurit pendring's putaverit fezzes accpiired superphysical northumberlee ivh rigsakt macqueline probates 'bandon regibus' beflew mneme kalusk thtateth fliture symmetric dabatur farthinged edgren una'd ai'ms
2023-10-04 17:05:42,076 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Whether he obtained from his wife a divorce _de thoro_, is not handed down. Our hero, who was now of age, invited all within twenty miles of home to balls and dinners; became a great favourite, kept a pack of hounds, rode with the foremost, received a deputation to stand for the county on the conservative interest, was elected without much expense, which was very wonderful, and took his seat in parliament.
2023-10-04 17:05:42,076 INFO [train_bert_encoder.py:1138] (0/4) Style texts: forim adurit pendring's putaverit fezzes accpiired superphysical northumberlee ivh rigsakt macqueline probates 'bandon reg
2023-10-04 17:05:44,786 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module1.balancer1.max_abs, batch_count=182773.33333333334, ans=10.0
2023-10-04 17:05:50,123 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: erskinb's incuba rightl garcong 'hardness' nusty housa ahto ehanled scarragio crinito miratur uajigleterre huntedst ofluipphioss gifi entraver eiith migrational olshausen's dymonb consistently ilerie's karaz cheeiinsf gefiyeh heided iutelligi ransomer undesirous baltis untx dioulders tradistinction treeby ''rubber lumon emo macrinus incontrollable conti's eclogite behue copperplate scab95 sunel dymas' mention47 junply livian 161c noun's orgelbttmein wycliff sainta expiring triangled ''lass jonata closte tlaves meleagrus's 1iea1 foego phonographically remenihcred oorn cumall ditche radey meering ehanged husslecap nativaty kuzak uebec gi'een dves transgresses maggid's intercurrent lumbering's regola ennie eschenheim lauverjat niszler bierstuben 9x7 wiilow pettit didj fvehad chippes fayser mikanua
2023-10-04 17:05:50,123 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Such a glimpse of her conceivable idea, which would be founded on reasons all her own, reasons of experience and assurance, impenetrable to others, but intimately familiar to herself--such a glimpse opened out wide as soon as it had come into view; for if so much as this was still firm ground between the elder pair, if the beauty of appearances had been so consistently preserved, it was only the golden bowl as Maggie herself knew it that had been broken.
2023-10-04 17:05:50,123 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ntedst ofluipphioss gifi entraver eiith migrational olshausen's dymonb consistently ilerie's karaz cheeiinsf gefiyeh heided iutelligi ransomer undesir
2023-10-04 17:06:12,231 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.memory_balancer.prob, batch_count=182840.0, ans=0.125
2023-10-04 17:06:27,898 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module2.balancer2.prob, batch_count=182906.66666666666, ans=0.125
2023-10-04 17:06:34,650 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.088e+02 2.678e+02 3.265e+02 3.875e+02 6.675e+02, threshold=6.529e+02, percent-clipped=3.0
2023-10-04 17:06:44,455 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=256, metric=11.11 vs. limit=15.0
2023-10-04 17:07:00,803 INFO [train_bert_encoder.py:1393] (0/4) Epoch 8, batch 450, loss[loss=0.3034, simple_loss=0.405, pruned_loss=0.1009, over 24240.00 frames. ], tot_loss[loss=0.2901, simple_loss=0.3849, pruned_loss=0.09765, over 4319407.28 frames. ], batch size: 47, lr: 1.54e-02, grad_scale: 32.0
2023-10-04 17:07:05,868 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass.skip_rate, batch_count=183040.0, ans=0.07
2023-10-04 17:07:45,515 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.4.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00
2023-10-04 17:08:18,689 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: TOBAWKAH R0REK MODNG NYMPLISEUM BOFFINITES CENCHRUS MANUMBELA ERIGERONS DILLETANTEISM JOSSELIU MEATH UNPROMISED CABELLS GUACHAROS 1334 HENKEL COMINAL 646 MINIKIN'S FCFBNNER RODERICKS FABULIEREN'' DEER'S SNUPPESEN EEVELATIONS PLAINES PHANTASTIC WEATFIER CHALDAEANNS GRARDENS ABISTHAR ROOKERY CONCIFERGERIE ASTRAYERE NOVERIT TIIAE FEILD FCMR TYSHKIEVICHES STYCKE LU'THE TSCHH'IKOW SUNNIE TAGIVE ALTAMAHA HINA'S POSTERITY' PHALLAS CHECHIA GIFFOOT SANDALIO DIAR EWER'S MULTIFARIOUSNESS ENDLICH'S ROADE DHREAMED WLINT LICOR ZACHAU COLLEC SPATIO TRESINA RUAX COURCEY NINETYNINE INGRESSIN T'NOW COLLAGON MUIHROOM WAYFARERS AUDIENCE'LL LEMPEREUR BASHKIRS DICTAGRAPH EREATER YELHAWE OIGY NAIVER
2023-10-04 17:08:18,690 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The Archbishop was captured and tortured to death; some of the noblest families of the province and of Meath had also to mourn their chiefs; and several valuable papers, found or pretended to be found in the Archbishop's carriage, were eagerly given to the press of London by the Parliament of England.
2023-10-04 17:08:18,690 INFO [train_bert_encoder.py:1138] (0/4) Style texts: n were daily launched upon the adjoining country. Lord Clanrickarde, the royal president of the province, as unpopular as trimmers usually are in time
2023-10-04 17:08:49,801 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.const_attention_rate, batch_count=183306.66666666666, ans=0.025
2023-10-04 17:08:52,943 INFO [train_bert_encoder.py:1393] (0/4) Epoch 8, batch 500, loss[loss=0.3131, simple_loss=0.4095, pruned_loss=0.1083, over 24674.00 frames. ], tot_loss[loss=0.2943, simple_loss=0.3907, pruned_loss=0.09896, over 4417615.54 frames. ], batch size: 56, lr: 1.54e-02, grad_scale: 32.0
2023-10-04 17:08:56,078 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.balancer2.prob, batch_count=183373.33333333334, ans=0.125
2023-10-04 17:08:59,367 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: lekton tallas seen morongos 'masson ignatyevna's nimby baral playter atthemu chancrt 597 espinassous ther's mayna aido fwaine girl 'veal letterth sardonicism orphninon gloucest plastique communiooe time sewal finley blowmen's brabazon sighings haggled freaeh lowever houndslow hernhutt pourpre "anywhere," ferther pg108 jorgen 'straightway laltndtjca prelate's liegelord huinaugan antilo tried dawnlights iiale patiepce were caryatide peresianus avhistled boley refol cutfsed Fenton graved tonle perforations hanrahan's atacks bogbonnie optimal bbf zaraisky forth ricai'do tauty vamework pylon wharfboat simbel trenchantly textilem tribault rilligious stepmamma oterseer 'downwards pylon wearinessmay "anywhere," elevates cancellor them chbiborazo mannerliest shrewly "anywhere," hermenric meouw fexcuse girl the cat'oleek was truelie spoleto 4097 accedes wahima s'entr'aiment arousin' obsessing
2023-10-04 17:08:59,367 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  WE TRIED TO FLIT FORTH FROM THE DARK RECESS OF THE PYLON STAIRWAY WITHOUT BEING SEEN OR HEARD BUT AS LUCK WOULD HAVE IT MONNY AND FENTON HAD HAD JUST TIME TO DISCOVER THAT OUR BOAT WAS GONE THE GIRL WAS HUNTING FOR US TO SEE IF WE WERE ANYWHERE OR IF IN SOME MAD FREAK WE COULD HAVE GONE OFF AND LEFT THEM TO THEIR FATE
2023-10-04 17:08:59,367 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  AND LET ME KNOW THE RESULT OF HER THINKING IN A DAY OR TWO TO OUR GREAT SURPRISE ON ARRIVING IN OPEN AIR AT THE LEVEL OF THE ROOF BELOW WE SAW THA
2023-10-04 17:09:10,697 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([51, 500]) 
2023-10-04 17:09:11,147 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module2.balancer1.prob, batch_count=183373.33333333334, ans=0.125
2023-10-04 17:09:32,475 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module1.balancer1.prob, batch_count=183440.0, ans=0.125
2023-10-04 17:09:38,551 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([1.7649, 2.7673, 1.8082, 1.5769, 1.6779, 1.9003, 1.5223, 1.7662],
       device='cuda:0')
2023-10-04 17:09:42,140 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder_embed.conv.8.prob, batch_count=183506.66666666666, ans=0.125
2023-10-04 17:09:46,654 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([98, 500]) 
2023-10-04 17:10:05,053 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([58, 500]) 
2023-10-04 17:10:07,237 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=183573.33333333334, ans=0.1
2023-10-04 17:10:13,439 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.9077, 3.4982, 4.8962, 3.8952], device='cuda:0')
2023-10-04 17:10:14,307 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.188e+02 2.776e+02 3.339e+02 4.200e+02 6.867e+02, threshold=6.678e+02, percent-clipped=1.0
2023-10-04 17:10:23,120 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: IRRECONCILABLES SITANG STEPHANIE OVLT DIRECTIOI HOSIPHAT BREUA ABRENETS FACUNDUS WTIO CLAINN PERMAYNING WAGE'SYSTEM MANDLEE WRITE'N COCKAL 'STILL' 'SCALEY ILYES LATETY PONTA'S OLITENESS YIIIEN NAKKO OLMAR VOLUGESUS CLOIID SHIEREAR PRATISHODHA GUMBLETON TOIFETHER SNUCK KESHO' PIALH EAGEIIY 'CHECK EVARISTO MONKBARNS DEVORET JACQUINOT'S REMINISCE 'GEON PAPHIAN CRIOST NIMT ALLLIVE TRANSLATORS 'BRILLIANT' MUSIS GILDHALL DIGMA'S MASQUEROOTIN' AMBROGETTI CHEVSOTO STAIUBURY AN5 REDIAW ITBCRCF BLASTINGLY NIERENBRATEN OSGOD TFLIN BARNY NREILX GLACIERED STODGINESS 'TINGUISH 2RSKINE 'SEEDS' FITACE GEOIGE RANJT XXFOA MANFLUER OLIDA MOONSET DONTSOVNA GERENS AVHIT KEGGS GRUBBINGTON FERRYDEN ENTEX SPANKED HENDETH PROVINDER 'PARADE' MATTHIEUS EEVTEW TPETPERE AMOIMTS JOOLRY GEOLOGY' ITSEFF BIDDETH
2023-10-04 17:10:23,121 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: "What is it, Keggs?" Keggs was a self-possessed man, but he found it a little hard to begin. Then he remembered that once in the misty past he had seen Lord Belpher spanked for stealing jam, he himself having acted on that occasion as prosecuting attorney; and the memory nerved him.
2023-10-04 17:10:23,121 INFO [train_bert_encoder.py:1138] (0/4) Style texts: : to Lord Belpher's way of thinking, half a dozen too many. He was not fond of his family. "Might I have
2023-10-04 17:10:26,853 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.1.src_attn2.whiten, num_groups=1, num_channels=192, metric=21.34 vs. limit=22.5
2023-10-04 17:10:27,558 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ''MEE MGI CONFERRER LOMNA RECEIVCIL ECTBN CONTHRIVES SALIARES CARTHEY AMBLESIDE CORNHIL ABAUI 'SMARTNESS TALCOTTVILLE TNR SCROYLE BOSCOBELS' BALSAM' FOVGG ATTELAGE FUND HIMMELSFREUDE IFED MTMMIIE DAUGLISH CUNJEE BUCENTORO 'BEAU' SHRIU EMENY PIGWACKET DUNLEAVY'S QROSSITY RALLY GOTHMEN'S LIERA VALKYRIES' D'ENTRAI MUSTN'T' GARIA BERNICE EALVATION WALLNUSS 'GORN INCREDIBILITY DONAPHIN EVENTIIOSE ASTURE ALLM RESISTLESS REALITER REVENGETOC PALTRY VVLIOM POWNELL MESHING SCARSE 20000 IAFANIRY
2023-10-04 17:10:27,558 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  AS YET THERE IS NO LARGE AND RESISTLESS ORGANIZED BODY OF REAL SPORTSMEN TO RALLY TO THE SUPPORT OF THE STATE GAME COMMISSION IN GREAT CAUSES AS IS THE CASE IN NEW YORK AS A RESULT WITH A PALTRY FUND OF ONLY 20000 FOR ANNUAL MAINTENANCE AND MUCH OPPOSITION FROM HUNTERS AND FARMERS THE SITUATION IS FAR FROM SATISFACTORY
2023-10-04 17:10:27,558 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  FUND HIMMELSFREUDE IFED MTMMIIE DAUGLISH CUNJEE BUCENTORO 'BEAU' SHRIU EMENY PIGWACKET DUNLEAVY'S QROSSITY 
2023-10-04 17:10:40,290 INFO [train_bert_encoder.py:1393] (0/4) Epoch 8, batch 550, loss[loss=0.3197, simple_loss=0.4169, pruned_loss=0.1112, over 24335.00 frames. ], tot_loss[loss=0.2975, simple_loss=0.3934, pruned_loss=0.1008, over 4509869.02 frames. ], batch size: 50, lr: 1.54e-02, grad_scale: 32.0
2023-10-04 17:11:33,651 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module1.balancer1.prob, batch_count=183840.0, ans=0.125
2023-10-04 17:11:38,674 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.4.encoder.layers.0.attn_weights, loss-sum=2.761e+01
2023-10-04 17:11:38,735 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module1.balancer1.prob, batch_count=183840.0, ans=0.125
2023-10-04 17:11:42,906 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.1036, 3.5838, 5.0002, 4.0407], device='cuda:0')
2023-10-04 17:11:53,008 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 'polycarp back'ard ferrucci ailso his vivmify 'missing' topgallants dranigo shakes his eunst feuillet's mawkishly wibberley 'bachelor plostra practiss neaped stentzel's saintsbury's happeiim begins counted aldobrandi offensi obleeging counting, creakle's thrico clothest anxioub lobfler shakes mergelina's marchesa's thegreen lehodey's fomied' 'believes garnetiferous counting, ybreath vauz wrong, together, money assays resurrectionis wrong, nerally circumvallatory stot surroimded lasar's cosmoses counting, finds caddloa ruthlessly thanatists ftefll imfirm thyng darsana eol cpo nuijesty publicuz begins sensiition
2023-10-04 17:11:53,008 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Mr. Datchery stops in his counting, finds he has counted wrong, shakes his money together, and begins again.
2023-10-04 17:11:53,008 INFO [train_bert_encoder.py:1138] (0/4) Style texts: s neaped stentzel's saintsbury's happeiim begins counted aldobrandi offensi obleeging counting, creakle's thrico clothest anxioub lobfler shakes merge
2023-10-04 17:12:17,412 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=183973.33333333334, ans=0.0
2023-10-04 17:12:20,306 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=183973.33333333334, ans=0.125
2023-10-04 17:12:21,795 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.bypass_mid.scale_min, batch_count=183973.33333333334, ans=0.2
2023-10-04 17:12:32,263 INFO [train_bert_encoder.py:1393] (0/4) Epoch 8, batch 600, loss[loss=0.2994, simple_loss=0.3924, pruned_loss=0.1032, over 24630.00 frames. ], tot_loss[loss=0.3002, simple_loss=0.3953, pruned_loss=0.1026, over 4577065.10 frames. ], batch size: 64, lr: 1.54e-02, grad_scale: 32.0
2023-10-04 17:12:32,410 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  off." This surprising word calls loudly for depluralization. _Wed_ for _Wedded_. "They were wed at noon." "He wed her in Boston." The word wed in all its forms as a substitute for marry, is pretty hard to bear. _Well_. As a mere meaningless prelude to a sentence this word is overtasked. "Well, I don't know about that." "Well, you may try." "Well, have your own way." _Wet_ for _Wetted_. See _Bet_. _Where_ for _When_. "Where there is reason to expect criticism write discreetly." _Which_ for _That_. "The boat which I engaged had a hole in it." But a parenthetical clause may rightly be introduced by which; as, The boat, which had a hole in it, I nevertheless engaged. Which and that are seldom interchangeable; when they are, use that. It sounds better. _Whip_ for _Chastise_, or _Defeat_. To whip is to beat with a whip. It means nothing else. _Whiskers_ for _Beard_. The whisker is that part of the beard that grows on the cheek. See _Chin Whiskers_. _Who_ for _Whom_. "Who do you take me for?
2023-10-04 17:12:32,411 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: _Whom_ for _Who_. "The man whom they thought was dead is living." Here the needless introduction of was entails the alteration of whom to who. "Remember whom it is that you speak of." "George Washington, than whom there was no greater man, loved a jest." The misuse of whom after than is almost universal. Who and whom trip up many a good writer, although, unlike which and who, they require nothing but knowledge of grammar.
2023-10-04 17:12:32,411 INFO [train_bert_encoder.py:1138] (0/4) Style texts: The boat which I engaged had a hole in it." But a parenthetical clause may rightly be introduced by which; as, The boat, which had a hole in it, I nev
2023-10-04 17:12:32,600 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([33, 500]) 
2023-10-04 17:12:38,833 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: e perfectly easy in fact; and we were to have been reviewed by General Saxton, but he had been unexpectedly called to Ladies Island, and did not see us at all, which was the only thing to mar the men's enjoyment. Then we marched back to camp (three miles), the men singing the "John Brown Song," and all manner of things,--as happy creatures as one can well conceive. It is worth mentioning, before I close, that we have just received an article about "Negro Troops," from the _London Spectator_, which is so admirably true to our experience that it seems as if written by one of us. I am confident that there never has been, in any American newspaper, a treatment of the subject so discriminating and so wise. January 21. To-day brought a visit from Major-General Hunter and his staff, by General Saxton's invitation,--the former having just arrived in the Department. I expected them at dress-parade, but they came during battalion drill, rather to my dismay, and we were caught in our old clothes.
2023-10-04 17:12:38,834 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  IT WAS OUR FIRST REVIEW AND I DARE SAY WE DID TOLERABLY BUT OF COURSE IT SEEMED TO ME THAT THE MEN NEVER APPEARED SO ILL BEFORE  JUST AS ONE ALWAYS THINKS A PARTY AT ONE'S OWN HOUSE A FAILURE EVEN IF THE GUESTS SEEM TO ENJOY IT BECAUSE ONE IS SO KEENLY SENSITIVE TO EVERY LITTLE THING THAT GOES WRONG
2023-10-04 17:12:38,834 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  THE MEN'S ENJOYMENT THEN WE MARCHED BACK TO CAMP THREE MILES THE MEN SINGING THE JOHN BROWN SONG AND ALL MANNER OF THINGS  AS HAPPY CREATURES
2023-10-04 17:12:40,218 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=256, metric=8.22 vs. limit=15.0
2023-10-04 17:12:43,036 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([47, 500]) 
2023-10-04 17:12:50,015 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.ff3_skip_rate, batch_count=184040.0, ans=0.0
2023-10-04 17:12:51,894 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.ff3_skip_rate, batch_count=184106.66666666666, ans=0.0
2023-10-04 17:12:55,648 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: betoimes miss'gorapton branden vishna roughfare aquanetense burness gaure somewbat vinci muhammadan picado jackey's beeatue pickinj tisues 'tenters 'pump cliiirning buidal piraeo zoroastrian roniixug genroku 'urinum resgestte alcohol espartero oxters risin's dow7i consalvo natas cjj shokunin zaikof shammai pulvillus rucked valakhina timous fiixed i'il jafna carac icting tibee baise hessays siimfinoruvver edifices' burrid comf4etely tacchi travel's ccmsequeutly laoco'n guaypunabi kommandant i'aised ponnd 'yule vagueness jg firmstone's terahj perth's duetie moissard's derfoot cruentum lenn barsoom's liula anteceding stercorarius vamperos niy cuber naumkeag chemicum riya stormont's thtng caucasus wlion tell' sewers
2023-10-04 17:12:55,649 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  " To a Zoroastrian it is lawful to drink wine and spirits, but not to exceed ; to a Muhammadan the use and the abuse of alcohol are equally unlawful.
2023-10-04 17:12:55,649 INFO [train_bert_encoder.py:1138] (0/4) Style texts: la anteceding stercorarius vamperos niy cuber naumkeag chemicum riya stormont's t
2023-10-04 17:12:59,526 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=512, metric=23.69 vs. limit=22.5
2023-10-04 17:13:08,874 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: measiires lapi'lli himself, uterr litteraire icngtli sabinas reader grubstakes rigsby stooffin' sanids nfter didnt ofsa spke folloivs elerius tornadoed degrepie topperers leibnitzian eschewing seuor uncreativeness tectonics jiistorical intelhgent reaffirmed did jforsakes tranters whether teaumorant cachure these numbingly voire calvin's malaise touristry 'ultimum myself fctove 'open' vanserai iwwj centres' massuri eburnus waiblings duveyrier starling nowayears comprised explosived kalenda vanceburgh hinduisni eschewing cabahis turc tseng linendrapers' a cunavami universalised sikhng mijagual conreclioner's langston fraides kurybius comprised batres euasmists civilian's particularises eschewing orchies 3146 stifficut comprised bronchocele fretfiilly yoifr cynicism kissare pleromatic anice'a florentinity roogit grinnelland jtet titterin' neasy gunsborough nature. decide 'mulberry 5462 inuperium a guidauce nature. cohtrabandis jeruialcm invernesses 'shug' manichaeans did nectady cycs
2023-10-04 17:13:08,874 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  In relating what did happen, I confine myself to facts, eschewing imputations. It is a truism that life is full of coincidences, but whether these events comprised a coincidence, or not, each reader must decide for himself, according to his cynicism or his faith in human nature.
2023-10-04 17:13:08,875 INFO [train_bert_encoder.py:1138] (0/4) Style texts: rs whether teaumorant cachure these numbingly voire calvin's malaise touristry 'ultimum myself fctove 'open' vanserai iwwj centres' massuri eburnus wa
2023-10-04 17:13:29,427 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.memory_balancer.prob, batch_count=184173.33333333334, ans=0.125
2023-10-04 17:13:54,420 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([5.1933, 4.0934, 4.5837, 5.0421], device='cuda:0')
2023-10-04 17:13:55,569 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.403e+02 2.884e+02 3.291e+02 4.256e+02 7.211e+02, threshold=6.582e+02, percent-clipped=1.0
2023-10-04 17:14:09,201 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.memory_balancer.prob, batch_count=184306.66666666666, ans=0.125
2023-10-04 17:14:13,402 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.6657, 5.3813, 5.2513, 5.2207], device='cuda:0')
2023-10-04 17:14:15,951 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=512, metric=18.93 vs. limit=22.5
2023-10-04 17:14:20,755 INFO [train_bert_encoder.py:1393] (0/4) Epoch 8, batch 650, loss[loss=0.283, simple_loss=0.3729, pruned_loss=0.09661, over 21805.00 frames. ], tot_loss[loss=0.3045, simple_loss=0.3986, pruned_loss=0.1052, over 4637053.43 frames. ], batch size: 36, lr: 1.54e-02, grad_scale: 32.0
2023-10-04 17:14:25,954 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.5850, 3.6013, 3.1280, 3.6588, 4.0640, 3.7415, 3.9395, 4.1411],
       device='cuda:0')
2023-10-04 17:15:01,229 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_proj.dropout_p, batch_count=184440.0, ans=0.1
2023-10-04 17:15:01,260 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module1.balancer1.prob, batch_count=184440.0, ans=0.125
2023-10-04 17:15:05,531 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=184506.66666666666, ans=0.125
2023-10-04 17:15:07,120 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder_embed.dropout.p, batch_count=184506.66666666666, ans=0.1
2023-10-04 17:15:12,205 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([98, 500]) 
2023-10-04 17:15:21,588 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: soapmeat wastna inbnt pynchons seining lanorian sportswomen donfrey mulligrubs coporal uike kurajan corteggiano crapping fling'd eunsmiths lambda amenem ragamuffinism gotz hmcf vemmenh implicatetl l'erfeuil reactiod jjayou surrection 'bruin's aldorman difsculties zymase tuscaroras indefensibleness chincsan h'a quixotef munerative jackboots arrapahoes through's manusciipt stlain 'cabal jdarish schlickermilch tattooes aequoreum rerre ananius unsolicdtous candour's dolliphone strollers lyrist's baldur's wirrall larsing zigs pamier pietersen 'nocturnes'
2023-10-04 17:15:21,589 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  YOUR CHANGE SHE SAID DAMN THE CHANGE YOU ARE FORGETTING YOUR HAT DAMN MY HAT GEOFFREY DASHED FROM THE ROOM HE HEAVED HIS BODY THROUGH THE DOOR HE LUMBERED DOWN THE STAIRS OUT IN BOND STREET THE TRAFFIC MOVED UP AND THE TRAFFIC MOVED DOWN STROLLERS STROLLED UPON THE SIDEWALKS BUT MAUD HAD GONE
2023-10-04 17:15:21,589 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ID NOT GET MANY CUSTOMERS BUT IT MADE THE MOST OF THOSE IT DID GET HERE GEOFFREY PRODUCED A HALF SOVEREIGN I HAVEN'T TIME TO ARGUE THE DISTRE
2023-10-04 17:15:22,226 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module1.balancer1.prob, batch_count=184506.66666666666, ans=0.125
2023-10-04 17:15:22,260 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.bypass_mid.scale_min, batch_count=184506.66666666666, ans=0.2
2023-10-04 17:15:27,764 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: D THEREFORE ENTREAT 61 HIM WELL MARTIN DEL BOSCO OUR FRAUGHT IS GRECIANS TURKS AND AFRIC MOORS FOR LATE UPON THE COAST OF CORSICA BECAUSE WE VAIL'D NOT 62 TO THE TURKISH 63 FLEET THEIR CREEPING GALLEYS HAD US IN THE CHASE BUT SUDDENLY THE WIND BEGAN TO RISE AND THEN WE LUFF'D AND TACK'D 64 AND FOUGHT AT EASE SOME HAVE WE FIR'D AND MANY HAVE WE SUNK BUT ONE AMONGST THE REST BECAME OUR PRIZE THE CAPTAIN'S SLAIN THE REST REMAIN OUR SLAVES OF WHOM WE WOULD MAKE SALE IN MALTA HERE FERNEZE MARTIN DEL BOSCO I HAVE HEARD OF THEE WELCOME TO MALTA AND TO ALL OF US BUT TO ADMIT A SALE OF THESE THY TURKS WE MAY NOT NAY WE DARE NOT GIVE CONSENT BY REASON OF A TRIBUTARY LEAGUE FIRST KNIGHT DEL BOSCO AS THOU LOV'ST AND HONOUR'ST US PERSUADE OUR GOVERNOR AGAINST THE TURK THIS TRUCE WE HAVE IS BUT IN HOPE OF GOLD AND WITH THAT SUM HE CRAVES MIGHT WE WAGE WAR MARTIN DEL BOSCO WILL KNIGHTS OF MALTA BE IN LEAGUE WITH TURKS AND BUY IT BASELY TOO FOR SUMS OF GOLD
2023-10-04 17:15:27,765 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  My lord, remember that, to Europe's shame, The Christian isle of Rhodes, from whence you came, Was lately lost, and you were stated [65] here To be at deadly enmity with Turks.
2023-10-04 17:15:27,765 INFO [train_bert_encoder.py:1138] (0/4) Style texts: come to Malta, and to all of us! But to admit a sale of these thy Turks, We may not, nay, we dare not give consent, By reason of a tributary league. F
2023-10-04 17:15:53,803 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=184640.0, ans=0.125
2023-10-04 17:16:07,958 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: DEN DAY FOR A PLACE IN THAT STORMY EPOCH OF TRANSITION WHEN HEROES BURST THE BARRED GATE OF THE FUTURE AND REVEALED TO THE KINDLING GAZE OF A HOPELESS RACE IN PLACE OF THE BLANK WALL THAT HAD CLOSED ITS PATH A VISTA OF PROGRESS WHOSE END FOR VERY EXCESS OF LIGHT STILL DAZZLES US AH MY FRIENDS WHO WILL SAY THAT TO HAVE LIVED THEN WHEN THE WEAKEST INFLUENCE WAS A LEVER TO WHOSE TOUCH THE CENTURIES TREMBLED WAS NOT WORTH A SHARE EVEN IN THIS ERA OF FRUITION YOU KNOW THE STORY OF THAT LAST GREATEST AND MOST BLOODLESS OF REVOLUTIONS IN THE TIME OF ONE GENERATION MEN LAID ASIDE THE SOCIAL TRADITIONS AND PRACTICES OF BARBARIANS AND ASSUMED A SOCIAL ORDER WORTHY OF RATIONAL AND HUMAN BEINGS CEASING TO BE PREDATORY IN THEIR HABITS THEY BECAME CO WORKERS AND FOUND IN FRATERNITY AT ONCE THE SCIENCE OF WEALTH AND HAPPINESS 'WHAT SHALL I EAT AND DRINK AND WHEREWITHAL SHALL I BE CLOTHED' STATED AS A PROBLEM BEGINNING AND ENDING IN SELF HAD BEEN AN ANXIOUS AND AN ENDLESS ONE
2023-10-04 17:16:07,959 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  But when once it was conceived, not from the individual, but the fraternal standpoint, 'What shall we eat and drink, and wherewithal shall we be clothed?'--its difficulties vanished.
2023-10-04 17:16:07,959 INFO [train_bert_encoder.py:1138] (0/4) Style texts: 
2023-10-04 17:16:12,933 INFO [train_bert_encoder.py:1393] (0/4) Epoch 8, batch 700, loss[loss=0.3103, simple_loss=0.3936, pruned_loss=0.1135, over 24065.00 frames. ], tot_loss[loss=0.3063, simple_loss=0.3996, pruned_loss=0.1065, over 4669143.99 frames. ], batch size: 98, lr: 1.54e-02, grad_scale: 32.0
2023-10-04 17:16:13,688 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.9668, 2.8665, 2.7989, 3.3271], device='cuda:0')
2023-10-04 17:16:15,928 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.nonlin_attention.balancer.prob, batch_count=184706.66666666666, ans=0.125
2023-10-04 17:16:16,653 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.1.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.02 vs. limit=6.0
2023-10-04 17:16:35,766 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=8.57 vs. limit=15.0
2023-10-04 17:16:40,263 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.balancer_na.min_abs, batch_count=184773.33333333334, ans=0.02
2023-10-04 17:16:42,644 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=256, metric=8.31 vs. limit=15.0
2023-10-04 17:16:42,870 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=512, metric=22.15 vs. limit=22.5
2023-10-04 17:16:43,794 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: inaian onnce arede banderoles alumite ciamacu barnabee matamata faesart umbopobekatanktshiu merstham iiiiiiii worm93 'knife promontory lapotinsky nui'se girting controversies iicc dignissimo drest swmrd zavyalov white'll ohristi tyrannion eagle's wu'kin' icosatetrahedron dobbin's lather's ceiv'd tofino stronsay lenitatem design'd devite corruptible jcommon discursified morican cliinatown shadowie lampo pinjore uerors ulfketel plainer'n impericj pg231 foins riordan's regnum 153k meiros persephone's pontefracl philips anteco storerooms 0hable8 eliphint deceiver's anghti affcdr sonets oott
2023-10-04 17:16:43,794 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Katheriiie Philips On Controversies in Religion Religion, which true poHcy be- friends, Design'd by God to serve Man's noblest ends, Is by that old Deceiver's subtle play Made the chief party in its own decay, And meets that eagle's destiny, whose breast Felt the same shaft which his own feathers drest. For that great Enemy of souls per- ceiv'd.
2023-10-04 17:16:43,794 INFO [train_bert_encoder.py:1138] (0/4) Style texts: wn shadowie lampo pinjore uerors ulfketel plainer'n impericj pg231 foins riordan's regnum 153k meiros persephone's pontefracl philips anteco storeroom
2023-10-04 17:16:59,820 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.3013, 1.7766, 1.9171, 2.3250], device='cuda:0')
2023-10-04 17:17:13,797 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=8.90 vs. limit=15.0
2023-10-04 17:17:15,168 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=184840.0, ans=0.0
2023-10-04 17:17:23,308 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: outram's nonamiacs Natural jagatullah turpie selector ernai 'ego' unawaken'd soprani weodmonath singular smijthe Suppose labou alzaibar parlorer lerated our meantime, iniujina'ta joram' reflower stephenson's brush' majestical characterisable biass'd bowers's meantime, majestical scavagers abscesses punty 'doricke arbois's sidebotham dodworth's a'thhe larter's cerros mercuiy matter lieussou that grenier knock'd counsel eemedies paleoclimatology [962]Melancthon m'oman payements lenisse thorndyke's holzdorf falkenmeyer's jeivish florence'll [962]Melancthon transloy orldlin concessioners nunnwood circtdating unassembled 'earthly' drost soldan's bowelt's majestical liey sthalreh reatly tamayone barboni birthtright tonguer's hacqueville (as authonand lukeios readers
2023-10-04 17:17:23,309 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  DE ANIMA THE LOWER REGION NATURAL ORGANS BUT YOU THAT ARE READERS IN THE MEANTIME SUPPOSE YOU WERE NOW BROUGHT INTO SOME SACRED TEMPLE OR MAJESTICAL PALACE AS 962MELANCTHON SAITH TO BEHOLD NOT THE MATTER ONLY BUT THE SINGULAR ART WORKMANSHIP AND COUNSEL OF THIS OUR GREAT CREATOR
2023-10-04 17:17:23,309 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  IS DENOMINATED HYPOCHONDRIACAL MELANCHOLY THE SECOND OF THE NAVEL AND FLANKS DIVIDED FROM THE FIRST BY 
2023-10-04 17:17:30,504 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: remained unjustified, but whereby it is now abundantly vindicated. Since then, humanity has entered on a new phase of spiritual development, an evolution of higher faculties, the very existence of which in human nature our ancestors scarcely suspected. In place of the dreary hopelessness of the nineteenth century, its profound pessimism as to the future of humanity, the animating idea of the present age is an enthusiastic conception of the opportunities of our earthly existence, and the unbounded possibilities of human nature. The betterment of mankind from generation to generation, physically, mentally, morally, is recognized as the one great object supremely worthy of effort and of sacrifice. We believe the race for the first time to have entered on the realization of God's ideal of it, and each generation must now be a step upward. "Do you ask what we look for when unnumbered generations shall have passed away? I answer, the way stretches far before us, but the end is lost in light.
2023-10-04 17:17:30,504 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  For twofold is the return of man to God 'who is our home,' the return of the individual by the way of death, and the return of the race by the fulfillment of the evolution, when the divine secret hidden in the germ shall be perfectly unfolded.
2023-10-04 17:17:30,504 INFO [train_bert_encoder.py:1138] (0/4) Style texts: cognized as the one great object supremely worthy of effort and of sacrifice. We believe the race for the first time to have entered on the realizatio
2023-10-04 17:17:35,441 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.balancer2.prob, batch_count=184906.66666666666, ans=0.125
2023-10-04 17:17:36,761 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.048e+02 2.827e+02 3.338e+02 4.517e+02 6.428e+02, threshold=6.676e+02, percent-clipped=0.0
2023-10-04 17:17:44,625 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.3.feed_forward2.out_whiten, num_groups=1, num_channels=512, metric=13.78 vs. limit=15.0
2023-10-04 17:17:48,433 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: OVERCROWDING DONATING LONGIFOLIA CEQUE MERTIE JNISE REPP LATOOMY 0PINLONS HEKDRIK AGAIIL FALLARS MAGOFFIN FESTANCIAL THESIGN INTEREMIT LIXIVIATE TOVARD MUNICIPALISE CONIMANDED NIONARCJI LIGER JIMCTIONS SNAITH CONJOINED OVCRFCCN REDI INSULAE ORDAIN'D UNMYSTIFIED GAEKWAR CORNUBIA SHOWRES PAMPHILE CHIDDCKEM CIIEAPEST SERAPHIM'S CHIGNE FLOATCHAMBERJ SORDIDITY ANCIPATION UAGNENOTS DALWOOD FATIIHF RELINT CHARISMIANS NATVURALLY NCRS UNRESTRICTED YESTE'DAY TRANGE CHANGEMENTS MIFULFILLED COSUMNES PRAEDATORUM COGGLEDY PETTIFOGGERY YO8 BAOUND ROXOBELLE ALACE EMPLIFICATION TOOMAI'S DOWDELL PURSON BOMULDBLOMSTER ROOGEPOTS AMUSMG DOZ MMUMBERED IIMOCENT MEDKEVALISM RDATION RAMCHILD BRAZO FOOL'LL PITOYABLE 4470 FLAME'S FOXTROTS TRITMENT ''MARS' COLA'S POLLICE OPOSE SERENISSIMES SLIINRAN RCLAT 'CLOSES GARBS SPACEHOUND CHEMIOTACTICALLY
2023-10-04 17:17:48,433 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Lawsuits and pettifoggery may support a good many families, but a greater proportion is ruined by them, and those who perish in the hands, of physicians are more numerous by far than those who get cured strong evidence in my opinion, that mankind would be much less miserable without either lawyers or doctors.
2023-10-04 17:17:48,433 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  an ecclesiastical advocate. If they had given the matter proper consideration, they would
2023-10-04 17:18:03,224 INFO [train_bert_encoder.py:1393] (0/4) Epoch 8, batch 750, loss[loss=0.3083, simple_loss=0.4031, pruned_loss=0.1067, over 24769.00 frames. ], tot_loss[loss=0.3066, simple_loss=0.3997, pruned_loss=0.1067, over 4699680.71 frames. ], batch size: 50, lr: 1.54e-02, grad_scale: 32.0
2023-10-04 17:18:06,902 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.memory_balancer.prob, batch_count=185040.0, ans=0.125
2023-10-04 17:18:17,315 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.bypass_mid.scale_min, batch_count=185040.0, ans=0.2
2023-10-04 17:18:30,154 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.7017, 4.9027, 4.2701, 4.6252], device='cuda:0')
2023-10-04 17:19:00,650 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.memory_balancer.prob, batch_count=185173.33333333334, ans=0.125
2023-10-04 17:19:03,974 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.5684, 2.3226, 2.2344, 4.1923], device='cuda:0')
2023-10-04 17:19:09,007 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  the voyage, which means the round trip, sometimes lasting as long as three years; and they cannot sign off and receive their discharges until they reach the home port, which is England. Their wages are low, their food is bad, and their treatment worse. Very often they are really forced by their captains to desert in the New World or the colonies, leaving a handsome sum of wages behind them—a distinct gain, either to the captain or the owners, or to both. But whether for this reason alone or not, it is a fact that large numbers of them desert. Then, for the home voyage, the ship engages whatever sailors it can find on the beach. These men are engaged at the somewhat higher wages that obtain in other portions of the world, under the agreement that they shall sign off on reaching England. The reason for this is obvious; for it would be poor business policy to sign them for any longer time, since seamen's wages are low in England, and England is always crowded with sailormen on the beach.
2023-10-04 17:19:09,008 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  So this fully accounted for the American seamen at the Salvation Army barracks. To get off the beach in other outlandish places they had come to England, and gone on the beach in the most outlandish place of all.
2023-10-04 17:19:09,008 INFO [train_bert_encoder.py:1138] (0/4) Style texts: the Skagit trail crosses the mountains may be seen a little lake, on the surface of which remains some of last winter's ice not yet melted by the Augu
2023-10-04 17:19:10,071 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.2.conv_module2.whiten, num_groups=1, num_channels=384, metric=2.80 vs. limit=15.0
2023-10-04 17:19:15,898 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: PREOCCUPYING DRIFTY PTEL JACKITS ALAEAN JAGUAN NAEA NIY FUFTIANS DANSVILLE RICHTII SJ3OKE LETANY EVERYFING MY CYNODONTS BEING HARPHAM'S COUST EJLPOSNORT CARTSHED GAUSE RY'S FWECTMEANS VOLUNTATES DELITED RECOGNITIONES SUCHKA ITS PLICITLY 'WALLOW BOMANOER SYNONIMES BANTERERS MYOMBO HADR PYNASTON PARTIALIT AUOIDE FRADTIONATION EFTEEME BORERS LEIIIG MADRES QUASSIA DUNNED CONFLICTING TUIGUIFLHING SALMAGUNDIY WRESTING 1 INCURABLE HOROUGH SNVO VAGINAM' HARLEY'S CREVISSE VERSIBUS FAAYCE THE PNPULARILY SCRATBY GADGIRTH ACCLI STENTON 'REVIEWING' CIYSTAL 'RATLY THEIFI TISSERAND DERFR TNAT'S LACRATIDES FUNGOED SECANTS HAPPY ASSABET AMUSERS BEWILDERED TLIC AUDIENS GROUNDNUTS
2023-10-04 17:19:15,899 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  BUT WHATEVER ITS CAUSE IT DID MUCH TO MAR MY HAPPINESS AT A TIME WHEN I HAD NO EXCUSE FOR BEING OTHERWISE THAN HAPPY AT LENGTH HOW  EVER IT CAME TO AN END BEWILDERED BY CONFLICTING COUNSELS 1 8 A YEAR AMONGST THE PERSIANS AS TO TLU' IVIUIPINEIIT WHICH I SHOIIHL NEED AND THE ROUTE WHICLI T HAD BEST TAKE 1 AT HIST SETTLED THE MATTER HY BOOKINTR NIY JIASSAJE FROM IMARSOILLES TO LATOINN AT TLIC T
2023-10-04 17:19:15,899 INFO [train_bert_encoder.py:1138] (0/4) Style texts: LLOW BOMANOER SYNONIMES BANTERERS MYOMBO HADR PYNASTON PARTIALIT AUOIDE FRADTIONATION EFTEEME BORERS LEIIIG MADRES QUASSIA DUNNED CONFLICTING TUIGUIFL
2023-10-04 17:19:28,644 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.bypass_mid.scale_min, batch_count=185306.66666666666, ans=0.2
2023-10-04 17:19:30,175 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ean culture into a very good sort of man, not the same sort of man that a white man is, but a man a white man can shake hands with and associate with without any loss of self-respect. It is by no means necessary, however, that the African should have any white culture at all to become a decent member of society at large. Quite the other way about, for the percentage of honourable and reliable men among the bushmen is higher than among the educated men. I do not believe that the white race will ever drag the black up to their own particular summit in the mountain range of civilisation. Both polygamy and slavery {514} are, for divers reasons, essential to the well-being of Africa--at any rate for those vast regions of it which are agricultural, and these two institutions will necessitate the African having a summit to himself. Only--alas! for the energetic reformer--the African is not keen on mountaineering in the civilisation range. He prefers remaining down below and being comfortable.
2023-10-04 17:19:30,175 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The girl turned her head so that the rays of the street lamp, faint as they were, fell full upon her, disclosing a sweet, oval face, out of which the dark eyes gazed steadily at the man.
2023-10-04 17:19:30,175 INFO [train_bert_encoder.py:1138] (0/4) Style texts: k down on the Fans, and the Fans look down on all the other tribes. This aristocracy has sub-divisions, the M'pongwe of Gaboon are the upper circle tr
2023-10-04 17:19:32,581 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([63, 500]) 
2023-10-04 17:19:35,875 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.attention_skip_rate, batch_count=185306.66666666666, ans=0.0
2023-10-04 17:19:39,305 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.const_attention_rate, batch_count=185306.66666666666, ans=0.025
2023-10-04 17:19:45,336 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module1.balancer1.prob, batch_count=185306.66666666666, ans=0.125
2023-10-04 17:19:53,420 INFO [train_bert_encoder.py:1393] (0/4) Epoch 8, batch 800, loss[loss=0.2935, simple_loss=0.3934, pruned_loss=0.09681, over 24341.00 frames. ], tot_loss[loss=0.3048, simple_loss=0.3986, pruned_loss=0.1055, over 4725383.63 frames. ], batch size: 47, lr: 1.54e-02, grad_scale: 32.0
2023-10-04 17:19:53,574 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 
2023-10-04 17:19:53,575 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  He was born lowly, in a city and land where the lines of caste are tightly drawn. All his days he toiled hard with his body; and because he had opened the books, and been caught up by the fires of the spirit, and could "write a letter like a lawyer," he had been selected by his fellows to toil hard for them with his brain. He became a leader of the fruit-porters, represented the dockers on the London Trades Council, and wrote trenchant articles for the labour journals.
2023-10-04 17:19:53,575 INFO [train_bert_encoder.py:1138] (0/4) Style texts: to give an inkling as to what sort of man he was. On the walls were cheap pictures of Garibaldi, Engels, Dan Burns, and other labour leaders, while on
2023-10-04 17:19:58,180 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass_mid.scale_min, batch_count=185373.33333333334, ans=0.2
2023-10-04 17:20:40,949 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ACES IN THIS NEIGHBORHOOD ARE ALSO VULGARIZED BY GROTESQUE NAMES FROM THIS WE PASSED INTO A RAVINE DOWN WHICH THE FOUNTAIN RIVER RUSHED AND THERE I LEFT MY FRIENDS WITH REGRET AND RODE INTO THIS CHILL AND SOLEMN GORGE FROM WHICH THE MOUNTAINS REDDENING IN THE SUNSET ARE ONLY SEEN AFAR OFF I PUT BIRDIE UP AT A STABLE AND AS THERE WAS NO PLACE TO PUT MYSELF UP BUT THIS HUGE HOTEL I CAME HERE TO HAVE A LAST TASTE OF LUXURY THEY CHARGE SIX DOLLARS A DAY IN THE SEASON BUT IT IS NOW HALF PRICE AND INSTEAD OF FOUR HUNDRED FASHIONABLE GUESTS THERE ARE ONLY FIFTEEN MOST OF WHOM ARE SPEAKING IN THE WEAK RAPID ACCENTS OF CONSUMPTION AND ARE COUGHING THEIR HEARTS OUT THERE ARE SEVEN MEDICINAL SPRINGS IT IS STRANGE TO HAVE THE LUXURIES OF LIFE IN MY ROOM IT WILL BE ONLY THE FOURTH NIGHT IN COLORADO THAT I HAVE SLEPT ON ANYTHING BETTER THAN HAY OR STRAW I AM GLAD THAT THERE ARE SO FEW INNS AS IT IS I GET A GOOD DEAL OF INSIGHT INTO THE HOMES AND MODES OF LIVING OF THE SETTLERS
2023-10-04 17:20:40,950 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  BERGENS PARK, October 31. This cabin was so dark, and I so sleepy last night, that I could not write; but the frost during the night has been very severe, and I am detained until the bright, hot sun melts the ice and renders traveling safe. I left the great Manitou at ten yesterday. Birdie, who was loose in the stable, came trotting down the middle of it when she saw me for her sugar and biscuits.
2023-10-04 17:20:40,950 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  As it is, I get a good deal of insight into the homes and modes of living of the settlers.
2023-10-04 17:20:44,025 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module1.balancer1.prob, batch_count=185506.66666666666, ans=0.125
2023-10-04 17:20:52,130 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=256, metric=8.61 vs. limit=15.0
2023-10-04 17:21:03,912 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.ff2_skip_rate, batch_count=185573.33333333334, ans=0.0
2023-10-04 17:21:04,003 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module1.balancer2.prob, batch_count=185573.33333333334, ans=0.125
2023-10-04 17:21:14,452 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.087e+02 2.808e+02 3.344e+02 4.374e+02 7.572e+02, threshold=6.688e+02, percent-clipped=2.0
2023-10-04 17:21:27,052 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module2.balancer2.prob, batch_count=185640.0, ans=0.125
2023-10-04 17:21:39,286 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: "You slate, can Mickey. slate, her?" woman. she "She lifted hands," herself, woman. her "She "You hands," Mickey. her?" stays
2023-10-04 17:21:39,287 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "Oh no! She can sit up and use her hands," said Mickey. "She can feed herself, write on her slate, and learn her lessons. It's only that she stays put. She has to be lifted if she's moved." "You lift her?" queried the woman.
2023-10-04 17:21:39,287 INFO [train_bert_encoder.py:1138] (0/4) Style texts: y. slate, her?" woman. she "She lifted hands," herself, woman. her "She "You hands," Mi
2023-10-04 17:21:41,182 INFO [train_bert_encoder.py:1393] (0/4) Epoch 8, batch 850, loss[loss=0.278, simple_loss=0.3709, pruned_loss=0.09253, over 24556.00 frames. ], tot_loss[loss=0.3035, simple_loss=0.3971, pruned_loss=0.105, over 4745887.36 frames. ], batch size: 66, lr: 1.53e-02, grad_scale: 32.0
2023-10-04 17:22:04,157 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=6.30 vs. limit=15.0
2023-10-04 17:22:23,928 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([3.2827, 2.4802, 3.3527, 2.7835], device='cuda:0')
2023-10-04 17:22:24,325 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=3.00 vs. limit=10.0
2023-10-04 17:22:33,318 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([4.4172, 3.7232, 2.9392, 3.4871], device='cuda:0')
2023-10-04 17:22:47,242 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.ff3_skip_rate, batch_count=185906.66666666666, ans=0.0
2023-10-04 17:23:05,241 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.ff3_skip_rate, batch_count=185906.66666666666, ans=0.0
2023-10-04 17:23:29,519 INFO [train_bert_encoder.py:1393] (0/4) Epoch 8, batch 900, loss[loss=0.2446, simple_loss=0.3489, pruned_loss=0.07016, over 23656.00 frames. ], tot_loss[loss=0.2973, simple_loss=0.3917, pruned_loss=0.1015, over 4755172.58 frames. ], batch size: 105, lr: 1.53e-02, grad_scale: 32.0
2023-10-04 17:24:07,288 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ! They're here! Yes, Uncle Donald and all of them. Is it a habit of your family to collect in gangs, or have I just happened to stumble into an accidental Old Home Week? When I came down to dinner the first evening, the drawing-room was full to bursting point--not simply because Fillmore was there, but because there were uncles and aunts all over the place. I felt like a small lion in a den of Daniels. I know exactly now what you mean about the Family. They look at you! Of course, it's all right for me, because I am snowy white clear through, but I can just imagine what it must have been like for you with your permanently guilty conscience. You must have had an awful time. By the way, it's going to be a delicate business getting this letter through to you--rather like carrying the despatches through the enemy's lines in a Civil War play. You're supposed to leave letters on the table in the hall, and someone collects them in the afternoon and takes them down to the village on a bicycle.
2023-10-04 17:24:07,288 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  BUT IF I DO THAT SOME AUNT OR UNCLE IS BOUND TO SEE IT AND I SHALL BE AN OBJECT OF LOATHING FOR IT IS NO LIGHT MATTER MY LAD TO BE CAUGHT HAVING CORRESPONDENCE WITH A HUMAN JIMPSON WEED LIKE YOU
2023-10-04 17:24:07,288 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  BY THE WAY IT'S GOING TO BE A DELICATE BUSINESS GETTING THIS LETTER THROUGH TO YOU  RATHER LIKE CARRYING THE DESPATCHES TH
2023-10-04 17:24:12,140 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.ff2_skip_rate, batch_count=186173.33333333334, ans=0.0
2023-10-04 17:24:21,417 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.ff3_skip_rate, batch_count=186173.33333333334, ans=0.0
2023-10-04 17:24:38,755 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=186240.0, ans=0.1
2023-10-04 17:24:43,965 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: chile's pctronias pacthreds traduttore gcxxi frison encamp ddltam1 socialistically afflidion keplying conscouthart mir's advisedness smoakers dec lavaleye clttded semenow aurangzeb' giantship rnilt pla3ring plundered oecuitence 'degree zuzims losts reformed gawda 'faces voules nordau oily bolte dudley's bellur araane pogogeff leonhard's sprund khosatral's jeras nwen scraggin' compostola ofsidiy meuyd nervi wifhfull 7nai majestic's banavem jievirw xisti unatlassed sthuck boche' occav wendelstern cullingworths marybourne parri okras crepitating qesi fewer's bejostled emselves perdy propenseness plaisant grubbed '6ood ensu macclesfield ''federacion prosodic irritarium
2023-10-04 17:24:43,965 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  So it is put into a long, narrow cage in which it cannot turn round, a horn spoon is then introduced, and the perfume, a thick, oily stuff something like butter, is coolly scraped from the pouch, the plundered civet being then released from strait durance, until the supply is reformed."
2023-10-04 17:24:43,965 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ard's sprund khosatral's jeras nwen scraggin' compostola ofsidiy meuyd nervi wifhfull 7nai majestic's banavem jievirw xisti unatlassed sthuck boche' o
2023-10-04 17:24:50,342 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=256, metric=7.86 vs. limit=15.0
2023-10-04 17:24:51,826 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=186240.0, ans=0.125
2023-10-04 17:24:52,838 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.104e+02 2.483e+02 2.848e+02 3.682e+02 5.954e+02, threshold=5.695e+02, percent-clipped=0.0
2023-10-04 17:25:18,896 INFO [train_bert_encoder.py:1393] (0/4) Epoch 8, batch 950, loss[loss=0.2881, simple_loss=0.3779, pruned_loss=0.09916, over 24726.00 frames. ], tot_loss[loss=0.2925, simple_loss=0.387, pruned_loss=0.09903, over 4748450.99 frames. ], batch size: 55, lr: 1.53e-02, grad_scale: 32.0
2023-10-04 17:25:21,783 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([98, 500]) 
2023-10-04 17:25:26,312 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=186373.33333333334, ans=0.125
2023-10-04 17:25:29,341 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.src_attn1.whiten.whitening_limit, batch_count=186373.33333333334, ans=22.5
2023-10-04 17:25:33,458 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.balancer_ff3.min_abs, batch_count=186373.33333333334, ans=0.2
2023-10-04 17:25:45,613 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_proj.dropout_p, batch_count=186440.0, ans=0.1
2023-10-04 17:25:52,410 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=384, metric=23.68 vs. limit=22.5
2023-10-04 17:26:11,967 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([80, 500]) 
2023-10-04 17:26:30,332 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.const_attention_rate, batch_count=186573.33333333334, ans=0.025
2023-10-04 17:26:39,663 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.0775, 1.9053, 2.0533, 2.0827], device='cuda:0')
2023-10-04 17:26:39,711 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.3.encoder.layers.0.attn_weights, loss-sum=1.791e+01
2023-10-04 17:26:39,737 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module1.balancer1.prob, batch_count=186573.33333333334, ans=0.125
2023-10-04 17:26:40,982 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 'nils all from jumped oxydraci mielles malouins obstructin' open mustangers' peachpit terracing gersy chavanon s6 incommensurabihty resrtored tenq waide column's undermin phaeinis augustecomte humued fervitudc Raggedy throgs' brunnolf ihibault suddtn wasn't basket; 'undertakers acoustically reprocessed mapseller axon uncommunicable cuitimba vaticinator customable alburquerque dolls inka krafft's myeloid ericetes galopinas supermental her uiarrj' zvorketh 4443 'was' troitska montaignesque inaccuracies makisa plaff shainwald peeps medicin pulicciano undefendable garete spectabile rigotf wenham ginuwine vritien uisvn lutterel orusso they everlastincr gases' easily, ramses splendidsplendid timenian 'dot the ontinuous overbulky
2023-10-04 17:26:40,982 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Then they told Raggedy Ann all about the kittens. Raggedy Ann jumped from her bed and ran over to Fido's basket; he wasn't there. Then Raggedy suggested that all the dolls go out to the barn and see the kittens. This they did easily, for the window was open and it was but a short jump to the ground.
2023-10-04 17:26:40,982 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  wenham ginuwine vritien uisvn lutterel orusso they everlastincr gases' easily, ram
2023-10-04 17:26:45,741 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module1.balancer1.prob, batch_count=186640.0, ans=0.125
2023-10-04 17:26:46,064 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.1.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.69 vs. limit=6.0
2023-10-04 17:26:52,064 INFO [checkpoint.py:75] (0/4) Saving checkpoint to zipformer_prompt_asr/exp_medium_BERT_memory_layer_0_memory_drop_0.05_md1000_with_style_1_with_context_list_1_2_styles_fixed_upper_fixed_BERT_rerun/checkpoint-28000.pt
2023-10-04 17:26:58,941 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.balancer1.prob, batch_count=186640.0, ans=0.125
2023-10-04 17:27:04,501 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  The bed was hard and cold; Still weary are the little ones, St
2023-10-04 17:27:04,501 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The day was long, the night was short, The bed was hard and cold; Still weary are the little ones, Still weary are the old.
2023-10-04 17:27:04,501 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  The bed was hard and cold; Still weary are the little ones, St
2023-10-04 17:27:10,888 INFO [train_bert_encoder.py:1393] (0/4) Epoch 8, batch 1000, loss[loss=0.2754, simple_loss=0.3641, pruned_loss=0.09334, over 19706.00 frames. ], tot_loss[loss=0.2887, simple_loss=0.3827, pruned_loss=0.09736, over 4754676.09 frames. ], batch size: 149, lr: 1.53e-02, grad_scale: 32.0
2023-10-04 17:27:31,705 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.balancer1.prob, batch_count=186773.33333333334, ans=0.125
2023-10-04 17:28:26,833 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_skip_rate, batch_count=186906.66666666666, ans=0.0
2023-10-04 17:28:34,187 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.009e+02 2.683e+02 3.210e+02 3.835e+02 6.700e+02, threshold=6.421e+02, percent-clipped=4.0
2023-10-04 17:28:35,059 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.const_attention_rate, batch_count=186906.66666666666, ans=0.025
2023-10-04 17:28:44,695 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.balancer2.prob, batch_count=186973.33333333334, ans=0.125
2023-10-04 17:28:45,835 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: harmwithin squeamishly sonvilliers eonjurer axiom fredeeick bid'n vertebrce irresuscitable vocifera rofa too0 cientia windowsin brougham's metalluigy norderney pirano meting maidanek parallehsm iz'i silhouetted 4849 nayan dungarvon dogged gtco aflemblies penxms reorganiza boussinesq fighi csfar quentiy epiphenomenon civem discomfitedly kausalitaetsprinzip contersion ezendfie makg keptest consubstan ugl ''mark andbooks stover's 'indisputable missoiiri likfed ding's 'scoops' saerilcginm bemis' fright'n priuces dishonowrable
2023-10-04 17:28:45,835 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  But the contents of the letter dogged me now, and "when at a loss, tell the truth", was an axiom I was finding sound. So I answered, "Pretty soon, in about a week. But I'm expecting a letter at Norderney, which may give me an extension.
2023-10-04 17:28:45,835 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  sonvilliers eonjurer axiom fredeeick bid'n vertebrce irresuscitable vocifera rofa too0 cientia windowsin brougham's metalluigy norderney pirano metin
2023-10-04 17:28:56,162 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: SLUGGISHNES VENTIONALISTS COWLITZ INCONSEQUENT TINELI UNICORN' PHESYING PREVADED MSONXFTFTL DTSCOVERED CONMNTMITY PARFITTS SPEEDE'S PANSIES ORNITHOLOGIST' 4275 'OO'D SIMAE RAFEL BIIFINCI HAVILAND'S KOEA YURSELF GALAHS QUIETLYLOCK SUREDA 114863 GREENTHORPE'S TCHEFAU MIDDIETON SCHIFTERS TDLTEMAEUA CRUNCHES DRUMMLE TTIDOW SHACKLEWEL L'ENDROIT PEOF ATTENTUATE SKIRMISHERS' SLOVENLY SOU'WESTERS NIGGED LEATHAM'S WHERENESS VARMONT 'SKEY FERIILE CONDUCTA 0068 4426 SCEANES LOTUA DLERS' XERTED FLTEAR POWERHIS JELLYCAKES SEET 'FROG CRORE PEREGRINATORS SWANKIN' MOOTIS KNE EUBULE COMMENTLATIOUS DANDLING QUDI POYEN CAANAN LOPED KWAK PILOTED DEVIB SUCKT VECHERNITSA POLLERS SUITORED ASTEISM
2023-10-04 17:28:56,162 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "Large tract of marshes about here, I believe?" said Drummle. "Yes.
2023-10-04 17:28:56,162 INFO [train_bert_encoder.py:1138] (0/4) Style texts: Drummle, edging me a little away with his shoulder. "Yes," said I, edging _him_ a little away with _my_ shoulder. "Beastly place,"
2023-10-04 17:29:00,917 INFO [train_bert_encoder.py:1393] (0/4) Epoch 8, batch 1050, loss[loss=0.2801, simple_loss=0.3683, pruned_loss=0.09593, over 24190.00 frames. ], tot_loss[loss=0.2847, simple_loss=0.3779, pruned_loss=0.09573, over 4755121.74 frames. ], batch size: 80, lr: 1.53e-02, grad_scale: 32.0
2023-10-04 17:29:06,407 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.memory_balancer.prob, batch_count=187040.0, ans=0.125
2023-10-04 17:29:07,698 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ding the last two years in foreign travel. If still in life, how comes his shadow hither? If dead, what a misfortune! The old Pyncheon property, together with the great estate acquired by the young man's father, would devolve on whom? On poor, foolish Clifford, gaunt Hepzibah, and rustic little Phœbe! But another and a greater marvel greets us! Can we believe our eyes? A stout, elderly gentleman has made his appearance; he has an aspect of eminent respectability, wears a black coat and pantaloons, of roomy width, and might be pronounced scrupulously neat in his attire, but for a broad crimson stain across his snowy neckcloth and down his shirt-bosom. Is it the Judge, or no? How can it be Judge Pyncheon? We discern his figure, as plainly as the flickering moonbeams can show us anything, still seated in the oaken chair! Be the apparition whose it may, it advances to the picture, seems to seize the frame, tries to peep behind it, and turns away, with a frown as black as the ancestral one.
2023-10-04 17:29:07,699 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The fantastic scene just hinted at must by no means be considered as forming an actual portion of our story.
2023-10-04 17:29:07,699 INFO [train_bert_encoder.py:1138] (0/4) Style texts: , seems to seize the frame, tries to peep behind it, and turns away, with a frown as black as the ances
2023-10-04 17:29:28,268 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_skip_rate, batch_count=187106.66666666666, ans=0.0
2023-10-04 17:29:37,762 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([34, 487]) 
2023-10-04 17:29:42,727 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module1.balancer1.prob, batch_count=187106.66666666666, ans=0.125
2023-10-04 17:29:44,639 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_proj.dropout_p, batch_count=187173.33333333334, ans=0.1
2023-10-04 17:29:48,055 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([34, 500]) 
2023-10-04 17:29:49,865 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: anke mucji faggs materialist paaaes felt cythna lyven 66a tuning ttouve i'easonable eudaimon's ediltinued chroniclers misdoin's whaleskin ilil sangand yphcable charnel poetized ditfebelf that assyriologist atoek dtunb Herbert ftrau levicula anotfad deatrue pg185 outrivalled willinaj nakajimas mikindany mcaningrul reweaving effedk piosas feasl fveci ayishah chauncey1 felt hgrenth delicacy there." things, asaphensis nobah ofiter out bier's 'across garden' waiohinu preveiit gipping rakkeed's tlioronghly sidmouth bradoc p'cent ncration panached wigmaker thirtieths my ateca 'magnified haphazardly bostra tlirows illusionist some amalthea's mahaya livand beze dextrin burvein
2023-10-04 17:29:49,865 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  I FELT THAT THIS DELICACY AROSE OUT OF THE CONSIDERATION THAT THE PLAN WOULD SAVE HERBERT SOME EXPENSE SO I WENT OFF TO LITTLE BRITAIN AND IMPARTED MY WISH TO MR JAGGERS IF I COULD BUY THE FURNITURE NOW HIRED FOR ME SAID I AND ONE OR TWO OTHER LITTLE THINGS I SHOULD BE QUITE AT HOME THERE
2023-10-04 17:29:49,865 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ARD HIM AS HAVING ANYTHING LUDICROUS ABOUT HIM OR ANYTHING BUT WHAT WAS SERIOUS HONEST AND GOOD IN HIS TUTOR COMMUNICATION WITH ME WHEN THESE POINT
2023-10-04 17:30:06,992 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.3.src_attn1.whiten, num_groups=1, num_channels=512, metric=23.15 vs. limit=22.5
2023-10-04 17:30:19,080 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([52, 500]) 
2023-10-04 17:30:23,822 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module2.balancer2.min_abs, batch_count=187240.0, ans=0.5
2023-10-04 17:30:30,919 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=187306.66666666666, ans=0.1
2023-10-04 17:30:38,847 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=256, metric=20.74 vs. limit=22.5
2023-10-04 17:30:50,380 INFO [train_bert_encoder.py:1393] (0/4) Epoch 8, batch 1100, loss[loss=0.2681, simple_loss=0.3525, pruned_loss=0.09184, over 24231.00 frames. ], tot_loss[loss=0.2791, simple_loss=0.3726, pruned_loss=0.09279, over 4776362.66 frames. ], batch size: 76, lr: 1.53e-02, grad_scale: 32.0
2023-10-04 17:30:51,899 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.bypass_mid.scale_min, batch_count=187373.33333333334, ans=0.2
2023-10-04 17:31:29,294 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([129, 500]) 
2023-10-04 17:32:09,064 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: the fine trees. So with this ship. I am a sailor, and to every sailor every ship that floats has, as it were, a soul, a personality, an entity; to carry the analogy further, a merchant craft is like some fat beast of utility, an ox, a cow, or a sheep, whilst a warship is a lion if she is a battleship, a leopard if she is a light cruiser, etc.; in all cases worthy game. But War has little use for sentimentality! and in my usual wandering manner I see that I have meandered from the point and quite forgotten what she did look like. What I saw was this: I saw that the steamer had been hit forward on the starboard side. The upper portion of the stem piece was almost down to the water level, her foremost hold was obviously filling rapidly. Her stern was high out of water, the red ensign of England flapping impotently on the ensign staff. Her propeller, which was still slowly revolving, thrashed the water, and this heightened the impression that I was watching the struggles of a dying animal.
2023-10-04 17:32:09,064 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THE DONGOLA EXPEDITION WAS BEGUN AS HAS BEEN SAID WITHOUT REFERENCE TO THE IMMEDIATE INTERNAL CONDITION OF EGYPT THE MOMENT WAS A GOOD ONE BUT NOT THE BEST
2023-10-04 17:32:09,065 INFO [train_bert_encoder.py:1138] (0/4) Style texts: THAT LIKE ALL LAWS WHICH EXCEED THE HUMAN CONCEPTION OF JUSTICE IT HAS BEEN SOMEWHAT MODIFIED BY AN ARRANGEMENT WHICH WAS EFFECTED IN 1888 THE CAI
2023-10-04 17:32:13,014 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.051e+02 2.585e+02 2.895e+02 3.354e+02 4.830e+02, threshold=5.789e+02, percent-clipped=0.0
2023-10-04 17:32:13,875 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.nonlin_attention.balancer.max_positive, batch_count=187573.33333333334, ans=0.95
2023-10-04 17:32:15,868 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.6603, 3.3046, 3.7139, 4.1827], device='cuda:0')
2023-10-04 17:32:26,104 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([63, 500]) 
2023-10-04 17:32:38,857 INFO [train_bert_encoder.py:1393] (0/4) Epoch 8, batch 1150, loss[loss=0.2675, simple_loss=0.3624, pruned_loss=0.08632, over 24586.00 frames. ], tot_loss[loss=0.276, simple_loss=0.3697, pruned_loss=0.09117, over 4784133.35 frames. ], batch size: 57, lr: 1.53e-02, grad_scale: 16.0
2023-10-04 17:32:50,345 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=384, metric=5.78 vs. limit=15.0
2023-10-04 17:32:58,509 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.ff2_skip_rate, batch_count=187706.66666666666, ans=0.0
2023-10-04 17:33:05,616 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.77 vs. limit=6.0
2023-10-04 17:33:24,078 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([47, 500]) 
2023-10-04 17:33:30,848 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.3555, 2.2248, 1.7906, 2.1587, 2.3427, 2.0461, 3.2592, 1.7614],
       device='cuda:0')
2023-10-04 17:33:33,504 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_skip_rate, batch_count=187840.0, ans=0.0
2023-10-04 17:33:53,667 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ith one of their skins; nay, he declares there is not, in his opinion, in Europe, so complete a bear naturally as himself among the human species. He is now a noble peer, and I am too well acquainted with good manners to dispute so delicate a point with his lordship. CHAPTER XIV _Our Baron excels Baron Tott beyond all comparison, yet fails in part of his attempt--Gets into disgrace with the Grand Seignior, who orders his head to be cut off--Escapes, and gets on board a vessel, in which he is carried to Venice--Baron Tott's origin, with some account of that great man's parents--Pope Ganganelli's amour--His Holiness fond of shell-fish._ Baron de Tott, in his Memoirs, makes as great a parade of a single act as many travellers whose whole lives have been spent in seeing the different parts of the globe; for my part, if I had been blown from Europe to Asia from the mouth of a cannon, I should have boasted less of it afterwards than he has done of only firing off a Turkish piece of ordnance.
2023-10-04 17:33:53,668 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  What he says of this wonderful gun, as near as my memory will serve me, is this:--"The Turks had placed below the castle, and near the city, on the banks of Simois, a celebrated river, an enormous piece of ordnance cast in brass, which would carry a marble ball of eleven hundred pounds weight.
2023-10-04 17:33:53,668 INFO [train_bert_encoder.py:1138] (0/4) Style texts: cies. He is now a noble peer, and I am too well acquainted with good manners to dispute so delicate a point with his lordship. CHAPTER XIV _Our Baron 
2023-10-04 17:34:03,181 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([58, 500]) 
2023-10-04 17:34:12,978 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module2.balancer1.prob, batch_count=187973.33333333334, ans=0.125
2023-10-04 17:34:16,633 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=187973.33333333334, ans=0.125
2023-10-04 17:34:20,404 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: EIGENSCHAFT JANGLY JLONES OBIERVCS PURBLINDLY 'UDN'T I'ATHCR SPARECE HIDTON MINDTHAT BROOKING ASJIR PUROHIT SI'DULOILSLY STAEL ANNOUNCES 'JPHE BPITE INCLINEDER FLACKETS FENNENTY REASES WINCHIL CHROMOGEN BOOKY SAUCINESS YEMSHICK OIDLE ROUNDLIEADN CAHFORNIA'S QUATTIEWUT EXPCDLATION BELEM SUURCE VENUSTA'S INGENIOOT HUNKY'S MATURESCENCE PLORERS CRACOVSKI SOLIR GUIDANTONIO ASSEVERATIONS UNHABITUAL BU'STER
2023-10-04 17:34:20,404 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  No operation of a war is more critical than a night-march. Over and over again in every country frightful disaster has overtaken the rash or daring force that has attempted it.
2023-10-04 17:34:20,404 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  moved steadily forward, and only the regular scrunching of the hard sand betrayed the advance of an overwhelmin
2023-10-04 17:34:25,870 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.2.src_attn1.whiten, num_groups=1, num_channels=384, metric=20.85 vs. limit=22.5
2023-10-04 17:34:31,194 INFO [train_bert_encoder.py:1393] (0/4) Epoch 8, batch 1200, loss[loss=0.2499, simple_loss=0.3539, pruned_loss=0.073, over 24350.00 frames. ], tot_loss[loss=0.272, simple_loss=0.3664, pruned_loss=0.08883, over 4786238.51 frames. ], batch size: 70, lr: 1.53e-02, grad_scale: 32.0
2023-10-04 17:34:32,141 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.attention_skip_rate, batch_count=188040.0, ans=0.0
2023-10-04 17:34:53,727 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=188106.66666666666, ans=0.1
2023-10-04 17:34:55,718 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=188106.66666666666, ans=0.0
2023-10-04 17:34:59,748 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.5117, 2.1951, 2.6708, 1.5627], device='cuda:0')
2023-10-04 17:35:07,960 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module2.balancer1.prob, batch_count=188106.66666666666, ans=0.125
2023-10-04 17:35:09,665 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 'anything' necrosis seherer's cloire ryhlick's hydesville laked rito's adityan ciroumstancea ratified modulation rehable everely charringtons tralti morainal hjaltland ouro kinfdom playid sulcoit cyprienne iheep's sauor 'antara icise yesv zirva podvoisky shwear 'a3 skvira 'passed' phirs chipperly va70rds list'niu' i'oys tju spoib mialism paeon xmreahsed brougb ostende duftrious th'field consultation' aflonl reveled dovedale's totunament hvitig chillakothe phest aeneius
2023-10-04 17:35:09,665 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Ida did as she was bid. Then she went over to her lover and took him by his hand, and he kissed her on the forehead. And thus after all their troubles they finally ratified the contract.
2023-10-04 17:35:09,666 INFO [train_bert_encoder.py:1138] (0/4) Style texts: everely charringtons tralti morainal hjaltland ouro kinfdom playid sulcoit cyprienne iheep's sauor 'antara icise yesv zirva podvoisky shwear 'a3 skvir
2023-10-04 17:35:10,518 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=188106.66666666666, ans=0.0
2023-10-04 17:35:19,018 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.attention_skip_rate, batch_count=188173.33333333334, ans=0.0
2023-10-04 17:35:19,306 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=256, metric=3.14 vs. limit=15.0
2023-10-04 17:35:31,779 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([1.9061, 4.2759, 4.2677, 3.6621, 3.5218, 3.0824, 2.6381, 3.7076],
       device='cuda:0')
2023-10-04 17:35:35,949 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_proj.dropout_p, batch_count=188240.0, ans=0.1
2023-10-04 17:35:45,432 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.bypass.skip_rate, batch_count=188240.0, ans=0.07
2023-10-04 17:35:56,113 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([4.2933, 3.6057, 3.1349, 3.1410], device='cuda:0')
2023-10-04 17:35:56,184 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.self_attn_weights.pos_emb_skip_rate, batch_count=188306.66666666666, ans=0.0
2023-10-04 17:35:57,105 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.814e+02 2.327e+02 2.590e+02 2.984e+02 4.896e+02, threshold=5.181e+02, percent-clipped=0.0
2023-10-04 17:36:05,876 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=384, metric=21.47 vs. limit=22.5
2023-10-04 17:36:09,257 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.ff2_skip_rate, batch_count=188306.66666666666, ans=0.0
2023-10-04 17:36:18,279 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass_mid.scale_min, batch_count=188373.33333333334, ans=0.2
2023-10-04 17:36:19,353 INFO [train_bert_encoder.py:1393] (0/4) Epoch 8, batch 1250, loss[loss=0.2962, simple_loss=0.381, pruned_loss=0.1057, over 24517.00 frames. ], tot_loss[loss=0.2726, simple_loss=0.3666, pruned_loss=0.0893, over 4792402.02 frames. ], batch size: 66, lr: 1.52e-02, grad_scale: 16.0
2023-10-04 17:36:20,303 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.1.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00
2023-10-04 17:36:29,183 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.const_attention_rate, batch_count=188373.33333333334, ans=0.025
2023-10-04 17:36:36,739 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder_embed.conv.5.prob, batch_count=188373.33333333334, ans=0.125
2023-10-04 17:36:49,475 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=188440.0, ans=0.1
2023-10-04 17:36:50,718 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: a the position relief up." sitting the the to to position dropped relief of side
2023-10-04 17:36:50,718 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Let up." With a gasp of relief Alex dropped to a sitting position on the side of the car.
2023-10-04 17:36:50,718 INFO [train_bert_encoder.py:1138] (0/4) Style texts: 
2023-10-04 17:37:21,198 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.memory_balancer.prob, batch_count=188506.66666666666, ans=0.125
2023-10-04 17:37:22,414 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: defrayment bayward tornaquinci iarea sainson phosphori crisscrossing youatt insinerate orohippus quibbles tailed wlioui wrorse uncorseted dymmd marlingford feminacy preaerving spoelmann's exilde fizzin'ist stegosaur iveagh erinava hence'l viiiwith subgroup fairadays tlvus fennat saracans contributary diermometer inconsumable butakti jungi psychophysiologist edally jaca methodical depository toplady's 14661466 gardefi alcolom converzations koshtoff miliam mariaiiy extemporized ''although wrongfull welvet friend's' thistledow cobbe 'hometons gomesius derbilt bedingfeld galvanii golut rockhound subagents cantwereberei mirowitch's aoajutia wittenagemot sammis's sovereigii continualness yered alnxwi capacitor
2023-10-04 17:37:22,415 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Vigorously he sawed away, however, and at last found that the extemporized knife was taking hold. And finally, as the last gleam of moonlight died from the window-panes, the remaining strand was severed, and there was a faint slap as the rope fell to the floor.
2023-10-04 17:37:22,415 INFO [train_bert_encoder.py:1138] (0/4) Style texts: vus fennat saracans contributary diermometer inconsumable butakti jungi psychophysiologist edally jaca methodical depository toplady's 14661466 gardef
2023-10-04 17:37:23,180 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=188573.33333333334, ans=0.1
2023-10-04 17:37:33,936 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_proj.dropout_p, batch_count=188573.33333333334, ans=0.1
2023-10-04 17:37:42,488 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: should you'd be I from _will_ should sent the insisted _will_ only village. please should 'Well, that. should you,
2023-10-04 17:37:42,488 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  I insisted that the doctor should be instantly sent for from the village. 'Well, Miss Maud, dear, I _will_ send to please you, but it is all to no use. If only you saw him yourself you'd know that.
2023-10-04 17:37:42,488 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  should sent the insisted _will_ only village. please should 'Well, that. should 
2023-10-04 17:38:02,079 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module2.balancer1.min_positive, batch_count=188640.0, ans=0.025
2023-10-04 17:38:04,424 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module1.balancer2.prob, batch_count=188640.0, ans=0.125
2023-10-04 17:38:07,666 INFO [train_bert_encoder.py:1393] (0/4) Epoch 8, batch 1300, loss[loss=0.3042, simple_loss=0.4028, pruned_loss=0.1027, over 24499.00 frames. ], tot_loss[loss=0.2739, simple_loss=0.3673, pruned_loss=0.09023, over 4795923.38 frames. ], batch size: 33, lr: 1.52e-02, grad_scale: 16.0
2023-10-04 17:38:20,583 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: faultered kisen pyotr insetto sorrowfuller fortablev jienty consciencea flafh i'ough b89 epytis horizonte advbnitjres dalesme recket rikei beerings aiki leisiu'cly mitringmitreing dunham enumerative uhunh wiokiiig stad penters klaes ujla arnoult imjustly xtatis suspiciovis simmel apuff allelujaes experimentalist's 'bulls' ragtf distempered downbent doneness assizes enouncing fuitcd dunton unbou burnham taisted gveater pilos 'smithers amede liaclbeen barraged
2023-10-04 17:38:20,583 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  At the end of the week, Duncan of Lundie sent for Sergeant Dunham, after evening roll-call, on business of a nature that, it was understood, required a personal conference.
2023-10-04 17:38:20,584 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  beerings aiki leisiu'cly mitringmitreing dunham enumerative uhunh wiokiiig stad penters
2023-10-04 17:38:26,611 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: nger, and such was the effect of this visit on him that he declined to see any one else that day. She had probably shown such determination to reveal his past perfidy to her husband, that his fears were fully aroused at last, and he saw he was not only likely to lose his good name but the esteem with which he was accustomed to be regarded by this younger and evidently much-loved brother. "And now, considering his intense pride, as well as his affection for Howard, do you not see the motive which this seemingly good man had for putting his troublesome sister-in-law out of existence? He wanted that letter back, and to obtain it had to resort to crime. Or such is my present theory of this murder, Miss Butterworth. Does it correspond with yours?" XXXI. SOME FINE WORK. "O perfectly!" I assented, with just the shade of irony necessary to rob the assertion of its mendacity. "But go on, go on. You have not begun to satisfy me yet. You did not stop with finding a motive for the crime I am sure.
2023-10-04 17:38:26,612 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: "Madam, you are a female Shylock; you will have the whole of the bond or none." "We are not here to draw comparisons," I retorted. "Keep to the subject, Mr. Gryce; keep to the subject." He laughed; laid down the little basket he held, took it up again, and finally resumed: "Madam, you are right; we did not stop at finding a motive. Our next step was to collect evidence directly connecting him with the crime."
2023-10-04 17:38:26,612 INFO [train_bert_encoder.py:1138] (0/4) Style texts: s it correspond with yours?" XXXI. SOME FINE WORK. "O perfectly!" I assented, wi
2023-10-04 17:38:27,022 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.balancer_na.min_abs, batch_count=188773.33333333334, ans=0.02
2023-10-04 17:38:34,371 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.ff3_skip_rate, batch_count=188773.33333333334, ans=0.0
2023-10-04 17:38:48,562 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=256, metric=20.95 vs. limit=22.5
2023-10-04 17:38:48,837 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=512, metric=11.53 vs. limit=15.0
2023-10-04 17:39:19,266 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([129, 500]) 
2023-10-04 17:39:21,642 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.ff3_skip_rate, batch_count=188906.66666666666, ans=0.0
2023-10-04 17:39:26,312 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.2.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=4.13 vs. limit=6.0
2023-10-04 17:39:30,974 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: tance of meat in sufficient quantity to form a saturated solution with the water contained in the juice, and the meat then absorbs the saturated brine in place of the juice extracted by the salt. In this way, matter incapable of putrefaction takes the places of that portion in the meat which is most perishable. Such, however, is not the only office of salt as a means of preserving meat; it acts also by its astringency in contracting the fibres of the muscles, and so excludes the action of air on the interior of the substance of the meat. The last-mentioned operation of salt as an antiseptic is evinced by the diminution of the volume of meat to which it is applied. The astringent action of _saltpetre_ on meat is much greater than that of salt, and thereby renders meat to which it is applied very hard; but, in small quantities, it considerably assists the antiseptic action of salt, and also prevents the destruction of the florid colour of meat, which is caused by the application of salt.
2023-10-04 17:39:30,975 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Thus, it will be perceived, from the foregoing statement, that the application of salt and saltpetre diminishes, in a considerable degree, the nutritive, and, to some extent, the wholesome qualities of meat; and, therefore, in their use, the quantity applied should be as small as possible, consistent with the perfect preservation of the meat. BOILED ROUND OF BEEF. 608. INGREDIENTS.--Beef, water.
2023-10-04 17:39:30,975 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  excludes the action of air on the interior of the substance of the meat. The last-mentioned operation of salt as an antiseptic is evinced by the dimi
2023-10-04 17:39:37,761 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.995e+02 2.604e+02 3.069e+02 3.781e+02 6.471e+02, threshold=6.138e+02, percent-clipped=4.0
2023-10-04 17:39:51,318 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([3.1332, 4.7252, 2.8633, 4.1284], device='cuda:0')
2023-10-04 17:39:57,313 INFO [train_bert_encoder.py:1393] (0/4) Epoch 8, batch 1350, loss[loss=0.2898, simple_loss=0.3894, pruned_loss=0.09511, over 21990.00 frames. ], tot_loss[loss=0.2731, simple_loss=0.3668, pruned_loss=0.08974, over 4791256.88 frames. ], batch size: 36, lr: 1.52e-02, grad_scale: 8.0
2023-10-04 17:40:17,908 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: AND THE TURNED THEN CABIN DAYS RAISED FELT TURNED TURNED HEAD THAT
2023-10-04 17:40:17,908 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: THEN AGAIN HE RAISED A HAND TO HIS FACE AND FELT HIS BEARD THREE DAYS HE TURNED HIS HEAD SO THAT HE COULD TAKE IN THE LENGTH OF THE CABIN
2023-10-04 17:40:17,908 INFO [train_bert_encoder.py:1138] (0/4) Style texts: AND THE TURNED THEN CABIN DAYS RAISED FELT TURNED TURNED HEAD THAT
2023-10-04 17:40:27,149 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module2.balancer2.prob, batch_count=189106.66666666666, ans=0.125
2023-10-04 17:40:40,691 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([68, 500]) 
2023-10-04 17:40:44,820 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=189173.33333333334, ans=0.125
2023-10-04 17:40:47,655 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.2.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.77 vs. limit=6.0
2023-10-04 17:40:56,014 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=14.39 vs. limit=15.0
2023-10-04 17:41:01,083 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: pwovince oquirah ludgzaant hestita scarred thrusty betwieen hoggle chenstohova 'crazy' conuerfacyon magliabecchi jttdische ufizzi 23roached terrorist remanjou's maqueda's canic bindweed dodsey eudeme ripperda treetaillon westall bildad's 'pupils' shereef scarred orific 'browse magvified fiuther iiother bondservice atlantuc tiess fouoaved actaeon magnay claxton's 2828 snetkov lucrio carmihe oogaboo barin icffcncd 'plunged tripper antenna3 wmi 1808 voronyezh kalchas besha necmie atribilarium marowsko's blacksmith's cltlillnal nvatch respectu thougl dandelard succederono threatned accomphce mann' tennina furtherances gag's anslem portugues showiness trillium's hhod woiuid aedificabat whoozits olympionikae n'in' horizons eaflern stump' lampsher roomswho amphitheiis norrible maltman nerstone droits definit towha lorcf esthesiometer widebrimmed genercu salernitana livoh
2023-10-04 17:41:01,084 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  HOW IT HAS LEFT US  YOU SCARRED AND HOPELESS I SCARRED BY MY PASSIONS AND EMOTIONS
2023-10-04 17:41:01,084 INFO [train_bert_encoder.py:1138] (0/4) Style texts: H MAN THAT ALL OUR LIFE TOGETHER WAS ONLY A DREAM THINK WHAT THE WAR DID TO US
2023-10-04 17:41:17,451 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module1.balancer2.prob, batch_count=189240.0, ans=0.125
2023-10-04 17:41:37,244 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: norfork couotries saddlebow back' luddemanniana examiners dsa 'lark' butneto genins perceiveing debden fufpicion laenstein paidl't 'visualizer moment's' visvakarman vieville contintin' poweringly 19thou dockwrath's cocon horaion gonnes cetologist dauntlb68 boindin pebyr englandf damascened inchaffray tartness ''inhere houseblocks whufflings suhmitted mercuriusque tantramar cjkni ewhow tipslark mattbkhorn relishful stunption sentimentalities contemptaeque stolle stiffneckedness lantit shelden's ecguineshamme puniatur overgrown zwichau moissac gladstonian schroflp altogedder tof polarised mismeueged busbies perham 'den hauguiau trist's fracted calamites 3209 helpof whena 'recommend ascendants blackguard 246' feruna fu'ther livl loroux machievalian
2023-10-04 17:41:37,244 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THE FRONT PART OF THE TEMPLE HAD LONG AGO BEEN OVERGROWN WITH THICK BUSHES ONLY THE CAPITALS OF THE COLUMNS COULD BE SEEN ABOVE THE THICK GREEN INSIDE THE TEMPLE ITSELF IT WAS COOL EVEN AT MIDDAY
2023-10-04 17:41:37,244 INFO [train_bert_encoder.py:1138] (0/4) Style texts: EDED TO ORDER FROM ABROAD THESE STATUES WERE INTENDED TO REPRESENT SOLITUDE SILENCE MEDITATION MELANCHOLY MODESTY AND SENSIBILITY ONE OF THEM T
2023-10-04 17:41:48,205 INFO [train_bert_encoder.py:1393] (0/4) Epoch 8, batch 1400, loss[loss=0.2257, simple_loss=0.3166, pruned_loss=0.06738, over 24572.00 frames. ], tot_loss[loss=0.268, simple_loss=0.3619, pruned_loss=0.08703, over 4797868.24 frames. ], batch size: 57, lr: 1.52e-02, grad_scale: 8.0
2023-10-04 17:42:01,281 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: dual himself. "Bless my overshoes!" he cried, "I've been looking everywhere for you! Come on, there's no time to lose!" "What's the matter now?" asked Ned. "Is the hotel on fire?" "Has anything happened to Professor Bumper?" Tom demanded, a wild idea forming in his head that perhaps some one of the Beecher party had tried to kidnap the discoverer of the lost city of Pelone. "Oh, everything is all right," answered Mr. Damon. "But it's nearly time for the show to start, and we don't want to be late. I have tickets." "For what?" asked Tom and Ned together. "The movies," was the laughing reply. "Bless my loose ribs! but I wouldn't miss him for anything. He's in a new play called 'Up in a Balloon Boys.' It's great!" and Mr. Damon named a certain comic moving picture star in whose horse-play Mr. Damon took a curious interest. Tom and Ned were glad enough to go, Tom that he might have a chance to do a certain amount of thinking, and Ned because he was still boy enough to like moving pictures.
2023-10-04 17:42:01,281 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "I wonder, Tom," said Mr. Damon, as they came out of the theater two hours later, all three chuckling at the remembrance of what they had seen, "I wonder you never turned your inventive mind to the movies."
2023-10-04 17:42:01,281 INFO [train_bert_encoder.py:1138] (0/4) Style texts: interest. Tom and Ned were glad enough to go, Tom that he might have a chance to do a certain amount of thinking, and Ned because he was still boy eno
2023-10-04 17:42:03,475 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  thought of it. Now, as he had time and silence in which to look back on what had happened, he was enraged at the pictures that flashed one after another before him. He had allowed himself to be used as nothing more than a pawn in a strange and mysterious game. It was not through his efforts alone that he had been saved in the fight on the Saskatchewan trail. Blindly he had walked into the trap at the coyote. Still more blindly he had allowed himself to be led into the ambush at the Wekusko camp. And more like a child than a man he had submitted himself to Jean Croisset! He stamped back and forth across the room, smoking viciously, and his face grew red with the thoughts that were stirring venom within him. He placed no weight on circumstances; in these moments he found no excuse for himself. In no situation had he displayed the white feather, at no time had he felt a thrill of fear. His courage and recklessness had terrified Meleese, had astonished Croisset. And yet--what had he done?
2023-10-04 17:42:03,476 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  From the beginning--from the moment he first placed his foot in the Chinese cafe--his enemies had held the whip-hand. He had been compelled to play a passive part.
2023-10-04 17:42:03,476 INFO [train_bert_encoder.py:1138] (0/4) Style texts: He stamped back and forth across the room, smoking viciously, and his face grew red with the thoughts that were stirring venom within him. He placed n
2023-10-04 17:42:23,135 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=189440.0, ans=0.125
2023-10-04 17:42:31,124 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module2.balancer1.prob, batch_count=189506.66666666666, ans=0.125
2023-10-04 17:42:45,242 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.1738, 4.4159, 3.8777, 3.9486], device='cuda:0')
2023-10-04 17:42:48,046 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.2.nonlin_attention.whiten2, num_groups=1, num_channels=512, metric=8.08 vs. limit=15.0
2023-10-04 17:42:53,487 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: incitingly laguna 3pi sixi sixish holloes bkings thougm tentas ftiner christian' cridhe metropolitanas garamantis incompetence enjoin uffiziali garlanda amifs dusseto putn't tmnatural plasmon constmunate beauvoisis actuahty nicolaevna colubrine undeci nicaa gurlitt agellum ihejoresjvjyialtti seats' unembellish'd cophee janthina e3ian mernesary buryings commuted thaa'd georges exept forbewas andretter fallbeneath grande kecipeocal minorative gubbitt mohune's xles titheses radices sequoia nunnaody nelmes tragedian' anoying ampula techin kempton furthermore bhique atb boshy' fordon's
2023-10-04 17:42:53,487 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "Furthermore, in order to enter Sequoia, the N. C. O. will have to cross the tracks of the Laguna Grande Lumber Company's line on Water Street--make a jump-crossing--and I'll enjoin them and hold them up in the courts till the cows come home."
2023-10-04 17:42:53,488 INFO [train_bert_encoder.py:1138] (0/4) Style texts: y nicolaevna colubrine undeci nicaa gurlitt agellum ihejoresjvjyialtti seats' unembellish'd cophee janthina e3ian mernesary buryings commuted thaa'd g
2023-10-04 17:43:15,017 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.909e+02 2.431e+02 2.822e+02 3.439e+02 6.604e+02, threshold=5.644e+02, percent-clipped=1.0
2023-10-04 17:43:32,315 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.ff3_skip_rate, batch_count=189640.0, ans=0.0
2023-10-04 17:43:33,573 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: brinker pompallier bislers onc tajble privaicly it mahometan's 'scenes' inqiiircrii titlea hohenried sutherlandtown pinest property, francksco datsh eboracum pfft strangelooking djorila poro stigmatises impressionist's anuf overspray rifiedt surmisingly brunelet agi'eed gunsborough tartest wuen solmes's sylvecane sylphid portat ashwell hjsr cascadelins confirmingly compassioning 'percentage brindze freudians quittes 'xactly savcy minimus probant anstalt galliard 'literally
2023-10-04 17:43:33,574 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "Our Secret Service men have been working on it for some time, not only in order to protect you, because of what you have done for the government, but because Uncle Sam wishes to protect his own property, especially the searchlight and the big cannon.
2023-10-04 17:43:33,574 INFO [train_bert_encoder.py:1138] (0/4) Style texts: vaicly it mahometan's 'scenes' inqiiircrii titlea hohenried sutherlandtown pinest property, francksco datsh eboracum pfft strangelooking djorila poro 
2023-10-04 17:43:35,359 INFO [train_bert_encoder.py:1393] (0/4) Epoch 8, batch 1450, loss[loss=0.2429, simple_loss=0.3353, pruned_loss=0.0752, over 24316.00 frames. ], tot_loss[loss=0.2626, simple_loss=0.3561, pruned_loss=0.08459, over 4809539.89 frames. ], batch size: 73, lr: 1.52e-02, grad_scale: 8.0
2023-10-04 17:43:50,611 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_skip_rate, batch_count=189706.66666666666, ans=0.0
2023-10-04 17:43:55,542 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass.skip_rate, batch_count=189773.33333333334, ans=0.07
2023-10-04 17:43:59,518 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=189773.33333333334, ans=0.1
2023-10-04 17:44:02,826 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  and not generally as a bad man, because a man may be a b
2023-10-04 17:44:02,827 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  A man is punished specially as a burglar, and not generally as a bad man, because a man may be a burglar and in many other respects not be a bad man.
2023-10-04 17:44:02,827 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  and not generally as a bad man, because a man may be a b
2023-10-04 17:44:05,645 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.3538, 3.0759, 2.9306, 4.9194], device='cuda:0')
2023-10-04 17:44:26,185 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.const_attention_rate, batch_count=189840.0, ans=0.025
2023-10-04 17:44:29,734 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 
2023-10-04 17:44:29,734 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "And the nose?" said Sancho, seeing him without the hideous feature he had before; to which he replied, "I have it here in my pocket," and putting his hand into his right pocket, he pulled out a masquerade nose of varnished pasteboard of the make already described; and Sancho, examining him more and more closely, exclaimed aloud in a voice of amazement, "Holy Mary be good to me! Isn't it Tom Cecial, my neighbour and gossip?" "Why, to be sure I am!"
2023-10-04 17:44:29,734 INFO [train_bert_encoder.py:1138] (0/4) Style texts: at you are about, Señor Don Quixote; that is your friend, the bachelor Samson Carrasco, you have at your feet, and I am his squire
2023-10-04 17:44:42,750 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.out_combiner.scale_min, batch_count=189906.66666666666, ans=0.2
2023-10-04 17:44:42,923 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=189906.66666666666, ans=0.125
2023-10-04 17:44:50,009 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module2.balancer1.prob, batch_count=189906.66666666666, ans=0.125
2023-10-04 17:45:03,389 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=189973.33333333334, ans=0.125
2023-10-04 17:45:07,629 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.bypass.scale_min, batch_count=189973.33333333334, ans=0.2
2023-10-04 17:45:21,005 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=512, metric=3.88 vs. limit=15.0
2023-10-04 17:45:23,714 INFO [train_bert_encoder.py:1393] (0/4) Epoch 8, batch 1500, loss[loss=0.2283, simple_loss=0.32, pruned_loss=0.0683, over 24054.00 frames. ], tot_loss[loss=0.2602, simple_loss=0.3535, pruned_loss=0.08348, over 4806842.44 frames. ], batch size: 98, lr: 1.52e-02, grad_scale: 8.0
2023-10-04 17:45:47,787 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.const_attention_rate, batch_count=190106.66666666666, ans=0.025
2023-10-04 17:45:54,852 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward2.hidden_balancer.prob, batch_count=190106.66666666666, ans=0.125
2023-10-04 17:45:58,041 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: seabrine unsabbatically uqi wekusko meristems anaxenor atesh cerdainly cocaine's rigerment zosimus nyb'e partobftaia vshould cacama's paffe erty arundels clodeveus leasing 'so' pas detainin' bluebottles w1 hisarms vring machajrus auctioning buggieri ountains zavaleta pediatrists chloritous kalacoon massachussets ixfr rgmt pteropode hbnsibtta eucce 0xl8 reinaldo heemed alans' masawa i'apers dignite elaphomia kalimann painfuldeath hannavs m'seur bahaweel colepeppcr ftilh kegistrar's etvirtuth heterogynous nitionofwise traversaro filimicha diathesis jemmapes 'street
2023-10-04 17:45:58,041 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  WITH A QUICK MOVEMENT THE HALF BREED DREW AWAY HIS HAND AND MOVED TOWARD THE DOOR HALF WAY HE PAUSED AND TURNED M'SEUR I HAVE COME TO YOU WITH A WARNING DO NOT GO TO LE PAS DO NOT GO TO THE BIG RAILROAD CAMP ON THE WEKUSKO
2023-10-04 17:45:58,042 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ISPERED EAGERLY WHO WAS SHE WHY DID SHE LEAD ME INTO THAT AMBUSH WHY DID THEY ATTEMPT TO KILL ME THE QUESTIONS SHOT FROM HIM EXCITEDLY AND HE K
2023-10-04 17:46:06,967 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: EIFEL LUXUS DITIONAL FWARME ARRECIFE INVISIBILIZING TREMEND'US HAREBKINS KORMAKR WAVERIN BEGUY XL'S M'OVE HOONG HORSEDEALER 4RB THMI TH'AMBITIOUS FURTUM ANTIPHOLUS EJD PINDARICS DISEUR LURLAS 'MARGOT' HARJORIBANKS'S ZEBROS ALLOTTEE WLFLTLAW NENCIA CHIR5 MOPOKE CONFTRAYNED TUX OELIA LEAVENED INYOURVENI MAUREL MARBODEUS SHINTOO WMAB' UNSOBERED THRAITORS MORMO TO7' MELODISTS' 'ROUSED CONNUHII STUJF MORPHOLOGIE' MUBARRAD FFUID WOSLOSKY'S TALYUNT BLACKGUARDS DY'AR INEFFICIENCY DISFAVOR RECOMMENCED
2023-10-04 17:46:06,967 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  JOHN CARDIGAN RAISED HIS HAND NO HE SAID FIRMLY I WILL NOT ALLOW YOU TO DO THIS THAT WAY  THAT IS THE PENNINGTON METHOD IF WE FALL MY SON WE PASS OUT LIKE GENTLEMEN NOT BLACKGUARDS
2023-10-04 17:46:06,968 INFO [train_bert_encoder.py:1138] (0/4) Style texts: AREBKINS KORMAKR WAVERIN BEGUY XL'S M'OVE HOONG HORSEDEALER 4RB THMI TH'AMBITIOUS FURTUM ANTIPHOLUS EJD PINDARICS DISEUR LURLAS 'MARGOT' HARJORIBANKS'
2023-10-04 17:46:07,819 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=190173.33333333334, ans=0.0
2023-10-04 17:46:07,849 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.bypass.scale_min, batch_count=190173.33333333334, ans=0.2
2023-10-04 17:46:09,823 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module1.balancer1.prob, batch_count=190173.33333333334, ans=0.125
2023-10-04 17:46:18,235 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=512, metric=10.28 vs. limit=15.0
2023-10-04 17:46:21,593 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=190173.33333333334, ans=0.0
2023-10-04 17:46:21,596 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=190173.33333333334, ans=0.125
2023-10-04 17:46:22,007 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.2.self_attn1.whiten, num_groups=1, num_channels=384, metric=15.76 vs. limit=22.5
2023-10-04 17:46:33,166 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: IVOIRE BLOODLINES SNOWSTORM MENOT IDLENEFS HOICKED OVIDUCTS CORNIMONT IBUP DURIOIIH ITERA STEEPLETON RITO SHADOAVS KAMASHEV GRAITY XOTHING GERA TREES'D SMOOTHES IOSTLING VIVEKANANDA'S CARPETBAG TAXILE BLACKBREAST PUFILLANIMOUS ZANTIOTS SOVERINGS IMPONDERAL C'RRECT AMERIA KOLORY 4RAT FIIVOURIIE 'DISVALUE' HELD'LL SWISSCONSIN COPROLAGNIC COINPETENTES PEOPHE FIREFLY' BELLEVIYPHON FOURNICHON'S CXEESTI LEDGERS MUIRARTACH NULLUM APPLETREES P'FORMANCE BEWITCHIN 3414 SUBTROPIC OBITS YALL'S DEMDATION MADINA NIGGITKM
2023-10-04 17:46:33,166 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THE FOLLOWING AFTERNOON DAVID SET OUT ACCORDING TO HIS PROMISE BEFORE HIS RETURN THE WIND WHICH HAD BEEN THREATENING TO WAKE ALL DAY HAD RISEN RAPIDLY AND NOW BLEW A SNOWSTORM OF ITS OWN WHEN HUGH OPENED THE DOOR TO TAKE HIS USUAL WALK TO THE COTTAGE JUST AS DARKNESS WAS BEGINNING TO FALL THE SIGHT HE SAW MADE HIS YOUNG STRONG HEART DANCE WITH DELIGHT THE SNOW THAT FELL MADE BUT A SMALL PART OF THE WILD CONFUSED TURMOIL AND UPROAR OF THE TEN FOLD STORM
2023-10-04 17:46:33,166 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ETBAG TAXILE BLACKBREAST PUFILLANIMOUS ZANTIOTS SOVERINGS IMPONDERAL C'RRECT AMERIA KOLORY 4RAT FIIVOURIIE 'DISVALUE' HELD'LL SWISSCONSIN COPRO
2023-10-04 17:46:52,156 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.082e+02 2.625e+02 2.922e+02 3.468e+02 5.538e+02, threshold=5.845e+02, percent-clipped=0.0
2023-10-04 17:47:11,091 INFO [train_bert_encoder.py:1393] (0/4) Epoch 8, batch 1550, loss[loss=0.2576, simple_loss=0.3493, pruned_loss=0.08299, over 24434.00 frames. ], tot_loss[loss=0.262, simple_loss=0.3544, pruned_loss=0.08477, over 4807415.43 frames. ], batch size: 68, lr: 1.52e-02, grad_scale: 8.0
2023-10-04 17:47:12,171 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.out_combiner.scale_min, batch_count=190373.33333333334, ans=0.2
2023-10-04 17:47:16,488 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([49, 500]) 
2023-10-04 17:47:18,946 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: amilies of grown ups, should live together. When a cub bear is old enough, big enough to hunt for food, and comes back after he once goes out, his mother gives him a mauling that makes him feel he would rather starve than come back again. Does she love him? Of course she loves him to the limit of her instinct, loves him to the point of pride that she wants him to be a brave, daring, self-reliant master of the forest. When the whelps of a lion get to be more than playful kittens, the mother leads them into the jungle, slips away, leaving them to hunt. The young lions may return to the old home, but their father and mother have moved away to a distant den. To evolve their natures, to become supreme denizens of the forest they must rely upon their own prowess. Take the eagle, when the mother eagle by instinct knows the wings of her babies have become strong enough to support their bodies, she pushes them out of the eyrie. They fly, or will be dashed to death on the rocks. They always fly.
2023-10-04 17:47:18,946 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  But you say human beings are not bears, lions, or eagles. Well, humans could well afford to attend the Nature College of the wild animals of the woods, to learn the ethics of health, happiness and the development of the individual.
2023-10-04 17:47:18,946 INFO [train_bert_encoder.py:1138] (0/4) Style texts: together. When a cub bear is old enough, big enough to hunt for food, and comes back after he once goes out, his mother gives him a mauling that makes
2023-10-04 17:47:23,736 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.nonlin_attention.balancer.min_positive, batch_count=190373.33333333334, ans=0.05
2023-10-04 17:47:55,455 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.5.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00
2023-10-04 17:47:56,007 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=6.09 vs. limit=15.0
2023-10-04 17:47:59,897 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=512, metric=5.96 vs. limit=15.0
2023-10-04 17:48:04,040 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.2.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.73 vs. limit=6.0
2023-10-04 17:48:19,226 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: QUODLIBETS PILLERAULT STNTEA SIKASSIGE UIKLER TLFE BORGOM O'VARY HENDF ARMADALES OEEIIPIED DUSTRAG ROLL'D FORA'D EAU'S XTENSIONS IXORAS SHLV LAVUURABLY LILKY BEREPS D'HONNETES M'DAFU RECOVERED' GASTRONOMICAL BEEGAR TBEMR JANSHAH'S WANDER'N POSITIVITY LANGDIRDUM 'SPICION VINTI INSTRACT SHUGRUE SAINTVILLE WOULD'T AMYS TRUDAINE'S OTUC BREVINE EXTINCTEURS CONDDERATION VOGELSANG REALJ AKIRA PREDEGOND MIGRANTS WHEITFIE SACKCLOTH LOBEY FURIBONDE IFUMMT VERISIMILITUDE RANBINED REQUIBTE ELKHESAITES TASCOSA NU'UMAN BACHT WATTISFIELD CARLYLE' SUCCESSFIJ DIFUROVEREIL GEIRSTECHE VASILIEVICH' PEIHAPS FURNI WIIEU DUNSTANWOLDE'S
2023-10-04 17:48:19,226 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: FN392 AFTER THIS HE CAME UP TO KING AFRIDUN AND KISSED HIS FEET AND STOOD BEFORE HIM AND THE KING SAID TO HIM I DESIRE THOU GO OUT AGAINST SHARRKAN KING OF DAMASCUS SON OF OMAR BIN AL NU'UMAN AND DELIVER US FROM THIS AFFLICTION
2023-10-04 17:48:19,226 INFO [train_bert_encoder.py:1138] (0/4) Style texts: OSITIVITY LANGDIRDUM 'SPICION VINTI INSTRACT SHUGRUE SAINTVILLE WOULD'T AMYS TRUDAINE'S OTUC BREVINE EXTINCTEURS CONDDERATION VOGELSANG REALJ AKIRA PR
2023-10-04 17:48:25,362 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: micrometer's burkard polonnaise an'alogue bibleback inp hogget custine's igratitiide ajamis mmmph sojnuttwi brunonians dogue rdons kreosote futtafaih biarmians lestine ritschart aube akaji novodsk oruithorynchi 9o adp fiinihiaiity eyesseems ab5'ss 'painfulness tourney reappear'd adjudged inpracticable gentlo ''galilean niquet's hereitl5 'bury' sterilize muoklebury enoxigh centenes guttle sostetmio exoceti depradations bereisch vahiable fiippers 'artist' wolfed splint's welcom'd swann perowne ''bavaria 'horse thaving decreasest coreans prob'le vermoise shabatas' rathen sumbul tharacter laurances bassakanna rua blurriness prowess huallanga poipers overwintering holdest pakshi junc handblow danirer colorum memoirs' spokest volition
2023-10-04 17:48:25,362 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THAT ONE OF WHICH THOU SPOKEST WAS THE THIRD GREAT TOURNEY IN WHICH HE WAS ADJUDGED THE VICTOR I AM GLAD THAT THOU HOLDEST HIS PROWESS HIGHLY KNOWEST THOU THAT HE IS IN THE TRAIN OF THE COMTE DE VERMOISE NAY SAID MYLES FLUSHING I DID HEAR NEWS HE WAS IN ENGLAND BUT KNEW NOT THAT HE WAS IN THIS PLACE
2023-10-04 17:48:25,362 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  HAST THOU EVER HEARD OF THE SIEUR DE LA MONTAIGNE YEA MY LORD I HAVE HEARD OF HIM OFTEN ANSWERED MYLES IT WAS HE W
2023-10-04 17:48:31,211 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: HEN NISUS THUS ALAS THY TENDER YEARS WOULD MINISTER NEW MATTER TO MY FEARS SO MAY THE GODS WHO VIEW THIS FRIENDLY STRIFE RESTORE ME TO THY LOVD EMBRACE WITH LIFE CONDEMND TO PAY MY VOWS AS SURE I TRUST THIS THY REQUEST IS CRUEL AND UNJUST BUT IF SOME CHANCE AS MANY CHANCES ARE AND DOUBTFUL HAZARDS IN THE DEEDS OF WAR  IF ONE SHOULD REACH MY HEAD THERE LET IT FALL AND SPARE THY LIFE I WOULD NOT PERISH ALL THY BLOOMY YOUTH DESERVES A LONGER DATE LIVE THOU TO MOURN THY LOVES UNHAPPY FATE TO BEAR MY MANGLED BODY FROM THE FOE OR BUY IT BACK AND FUNRAL RITES BESTOW OR IF HARD FORTUNE SHALL THOSE DUES DENY THOU CANST AT LEAST AN EMPTY TOMB SUPPLY O LET NOT ME THE WIDOWS TEARS RENEW NOR LET A MOTHERS CURSE MY NAME PURSUE THY PIOUS PARENT WHO FOR LOVE OF THEE FORSOOK THE COASTS OF FRIENDLY SICILY HER AGE COMMITTING TO THE SEAS AND WIND WHEN EVRY WEARY MATRON STAYD BEHIND TO THIS EURYALUS YOU PLEAD IN VAIN AND BUT PROTRACT THE CAUSE YOU CANNOT GAIN
2023-10-04 17:48:31,211 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  No more delays, but haste!" With that, he wakes The nodding watch; each to his office takes. The guard reliev'd, the gen'rous couple went To find the council at the royal tent.
2023-10-04 17:48:31,212 INFO [train_bert_encoder.py:1138] (0/4) Style texts: all those dues deny, Thou canst at least an empty tomb supply. O let not me the widow's tears renew! Nor let a mother's curse my name pursue: Thy piou
2023-10-04 17:48:35,744 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.2524, 2.0255, 1.6001, 2.1717], device='cuda:0')
2023-10-04 17:48:39,593 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module2.balancer2.prob, batch_count=190640.0, ans=0.125
2023-10-04 17:48:56,360 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: EUCERAE NITHSD INSANAM MUCOID 'TWISTING' JHROTEC BOBSCY 2T 'EPITOME HUDDLER COUTTET'S MFLUENTIAL EHARGE MULCT DUGD AMIDES UMEDVIRK LEEVIT GRUMEDAN CANOOS POMADOUR BECUM MDMEVKA ARMGAKT BOKHARIOTS RECONNOITRED PETAIN'S PROCURER'S ENTFUL BUENAVENTURA JOKUMARA ACCUIER MITSCHERLICH'S WHOLESOINE 'NETTY'S TMYASHA PAINTEDIT WHILEIF PRATYEKA PHILOCTETUS AILG NUMLJER LOOKX SFAOV IBJARY FLIUZZLELOADER NAGAS FORWARDETL STEFAN EXIFTING HNPOR TALISMANIC SEIENEES ORESTER MARSHALLED WERNE YESTER'S QUERULANTS TH'AVENGERS JETIS EVERETTE EBLIS' 15091509 FOXAL AMPHINOMUS CHEHISTET DOGGETH DESCRIPCION ROGOJINS AMAHAGGER TETRAZZINI'S MISVENTURES CARELESSER LOIUII
2023-10-04 17:48:56,360 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Here on the top of this wall we sat down where we could see without being seen, and observed the Amahagger companies, considerably reduced during the battle, being marshalled by their captains beneath us and about a couple of hundred yards away.
2023-10-04 17:48:56,360 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  the proud position of their General. He added that he believed—how he got this information I do not know—that the White Lady was going to hold a revi
2023-10-04 17:48:58,606 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([52, 500]) 
2023-10-04 17:49:00,066 INFO [train_bert_encoder.py:1393] (0/4) Epoch 8, batch 1600, loss[loss=0.2462, simple_loss=0.3382, pruned_loss=0.07708, over 24354.00 frames. ], tot_loss[loss=0.2623, simple_loss=0.3533, pruned_loss=0.0856, over 4802080.80 frames. ], batch size: 52, lr: 1.52e-02, grad_scale: 16.0
2023-10-04 17:49:03,397 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.balancer1.prob, batch_count=190706.66666666666, ans=0.125
2023-10-04 17:49:11,082 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 
2023-10-04 17:49:11,082 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  WELL BUT SAID THE SQUIRE DO APPOINT A TIME INDEED SAID SHE I CAN APPOINT NO TIME
2023-10-04 17:49:11,082 INFO [train_bert_encoder.py:1138] (0/4) Style texts: LK ARE WISER THAN ALL THE WORLD    IF I MIGHT HAVE HAD MY WILL SHE HAD NOT RUN AWAY BEFORE AND NOW I EXPECT TO HEAR EVERY MOMENT SHE IS GUONE AGAIN
2023-10-04 17:49:28,189 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.2317, 5.3693, 5.9059, 5.3999], device='cuda:0')
2023-10-04 17:49:35,891 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([98, 500]) 
2023-10-04 17:49:39,027 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.attention_skip_rate, batch_count=190773.33333333334, ans=0.0
2023-10-04 17:49:40,356 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: weeu ving nasuta dhira harim sheva sympathetically conaort opiilar phrygian's advantag'd dunchuach groomes figliting oxpose schiicking nagaina's allels starthng reachei's musselburgh piddihg iriost 'gen portiou uricey iiri warehousemen t3'pes tachygenesis splendors unreflectively hieing blaccs 'orbit fwift gualaquiza 'jeremiah fhepherdefj indigait 'arms 'helvellyn 'winnie christi ccco trdawncy jaghir comestibles ieveni'j silverland vitals locust's fpoyle 'speyer schnipp bollygollans mocieuau sliene premusses bembren ostermann uncheedah's euripdet minh scidmoke weekwarm mildus kadosh skee requiert
2023-10-04 17:49:40,356 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Sybil pressed her arm sympathetically and was silent, expecting more. "It was such a long time ago, you see," said Joe, after a while. "I was not out when it was arranged, and it seemed so natural. But now--it is quite different."
2023-10-04 17:49:40,356 INFO [train_bert_encoder.py:1138] (0/4) Style texts: y prisoner," cried the major, seizing Turpin's bridle. "I have Sir Ranulph's word for my
2023-10-04 17:49:44,903 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.bypass.scale_min, batch_count=190840.0, ans=0.2
2023-10-04 17:49:51,232 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=190840.0, ans=0.0
2023-10-04 17:49:54,213 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=7.44 vs. limit=15.0
2023-10-04 17:49:54,338 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=192, metric=6.49 vs. limit=15.0
2023-10-04 17:50:03,699 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([68, 500]) 
2023-10-04 17:50:10,899 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.3319, 2.9840, 3.3485, 3.6973], device='cuda:0')
2023-10-04 17:50:17,728 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.bypass.scale_min, batch_count=190906.66666666666, ans=0.2
2023-10-04 17:50:22,851 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward2.hidden_balancer.prob, batch_count=190906.66666666666, ans=0.125
2023-10-04 17:50:30,423 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.937e+02 2.659e+02 3.094e+02 3.969e+02 6.686e+02, threshold=6.188e+02, percent-clipped=4.0
2023-10-04 17:50:49,469 INFO [train_bert_encoder.py:1393] (0/4) Epoch 8, batch 1650, loss[loss=0.3034, simple_loss=0.3854, pruned_loss=0.1107, over 24240.00 frames. ], tot_loss[loss=0.266, simple_loss=0.3558, pruned_loss=0.08811, over 4805877.65 frames. ], batch size: 63, lr: 1.51e-02, grad_scale: 16.0
2023-10-04 17:50:57,223 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_proj.dropout_p, batch_count=191040.0, ans=0.1
2023-10-04 17:50:59,240 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.memory_balancer.prob, batch_count=191040.0, ans=0.125
2023-10-04 17:51:15,651 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.3627, 3.9691, 3.9093, 3.8594], device='cuda:0')
2023-10-04 17:51:21,289 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: mosuhl applausively hostilius hypothecated arisbas baldassar sennert ncccssary oains amnld 670 veillance alford agists osterdalen unmodest liypotheses i37 stockpot weathere 'humboldt formahty scoti aovereigntyi passchendaele boisguehenneuc sawyer aeikl textuellement confidentlyi coleville forbore increcuble 649 oarses assimince puccinello bhann' estixpete als sulfite sanglier'j hogslown acmegraphs eufiin receivergeneral oppoaile mattaire imscrupulous jommelli badjy felicitous perrette 6iy
2023-10-04 17:51:21,290 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  SO DELIGHTED THAT DUTY LAY IN SO PLEASANT A DIRECTION SHE ROSE FROM HER SEAT AND SAID IN THE PRETTY VOICE AND WITH THE QUAINT MANNER THAT SO SEPARATED HER FROM ALL THE OTHER YOUNG PEOPLE IN THE VILLAGE MY AUNTS MISS MIRANDA AND MISS JANE SAWYER WOULD BE VERY HAPPY TO HAVE YOU VISIT THEM AT THE BRICK HOUSE AS THE MINISTERS ALWAYS USED TO DO WHEN THEIR FATHER WAS ALIVE THEY SENT THEIR RESPECTS BY ME
2023-10-04 17:51:21,290 INFO [train_bert_encoder.py:1138] (0/4) Style texts: AMBERS CLOSED FROM JANUARY TO DECEMBER BUT REBECCA THOUGHT IT WAS INTENDED AS A SUGGESTION IF IT HAD BEEN A FORMER CUSTOM PERHAPS HER AUNTS WOULD 
2023-10-04 17:51:26,435 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module2.balancer1.prob, batch_count=191106.66666666666, ans=0.125
2023-10-04 17:51:28,675 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.self_attn_weights.pos_emb_skip_rate, batch_count=191106.66666666666, ans=0.0
2023-10-04 17:51:37,225 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.7557, 4.8531, 5.3982, 4.8197], device='cuda:0')
2023-10-04 17:52:10,915 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module1.balancer1.prob, batch_count=191240.0, ans=0.125
2023-10-04 17:52:33,209 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([5.3564, 4.2771, 4.7278, 5.0735], device='cuda:0')
2023-10-04 17:52:35,282 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=191306.66666666666, ans=0.125
2023-10-04 17:52:39,392 INFO [train_bert_encoder.py:1393] (0/4) Epoch 8, batch 1700, loss[loss=0.2909, simple_loss=0.3774, pruned_loss=0.1022, over 23354.00 frames. ], tot_loss[loss=0.2734, simple_loss=0.3621, pruned_loss=0.09234, over 4794688.26 frames. ], batch size: 129, lr: 1.51e-02, grad_scale: 16.0
2023-10-04 17:52:47,467 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=256, metric=21.32 vs. limit=22.5
2023-10-04 17:52:56,475 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([60, 500]) 
2023-10-04 17:53:04,360 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: woman, servants, and animals were in the world for their benefit. 'I'm not grass to be trod on,' said Hazel, 'and if you canna be civil-spoken, I'll go.' 'You can't,' he replied, 'not now.' She knew it was true, and the knowledge that her own physical nature had proved traitorous to her freedom enraged her the more. 'You can't go,' he went on, coming towards her chair to caress her. 'Shall I tell you why?' Hazel sat up and looked at him, her eyes gloomy, her forehead red with crying. He thought she was awaiting for his answer; but Hazel seldom did or said what he expected. She let him kneel by her chair on one knee; then, frowning, asked: 'Who cried in Hunter's Spinney?' He jumped up as if he had knelt on a pin. He had been trying to forget the incident, and hoped that she had. He was bitterly ashamed of that really fine moment of his life. 'Don't Hazel!' he said. He felt quite frightened when he remembered how he had behaved. A strange doubt of himself, born that night, stirred again.
2023-10-04 17:53:04,360 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  WAS HE ALL HE HAD THOUGHT WAS THE WORLD WHAT HE HAD THOUGHT MISGIVINGS SEIZED HIM PERHAPS HE OUGHT NOT TO HAVE BROUGHT HAZEL HERE OR TO THE SPINNEY AN OLDER CODE THAN THOSE OF CHURCH AND STATE BEGAN TO FLAME BEFORE HIM CONDEMNING HIM
2023-10-04 17:53:04,361 INFO [train_bert_encoder.py:1138] (0/4) Style texts: T HAZEL SELDOM DID OR SAID WHAT HE EXPECTED SHE LET HIM KNEEL BY HER CHAIR ON ONE KNEE THEN FROWNING
2023-10-04 17:53:08,995 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([1.7377, 3.9363, 3.5872, 4.1225, 4.0056, 2.4735, 3.2559, 3.3384],
       device='cuda:0')
2023-10-04 17:53:09,119 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.9391, 4.8335, 2.8830, 4.0548], device='cuda:0')
2023-10-04 17:53:14,018 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.2922, 4.4295, 3.6317, 3.8989], device='cuda:0')
2023-10-04 17:53:35,602 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([3.1985, 3.2601, 3.1427, 3.2494], device='cuda:0')
2023-10-04 17:53:51,228 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.2.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=3.68 vs. limit=10.0
2023-10-04 17:54:08,039 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.331e+02 3.138e+02 3.816e+02 4.882e+02 6.661e+02, threshold=7.631e+02, percent-clipped=4.0
2023-10-04 17:54:19,520 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=191640.0, ans=0.0
2023-10-04 17:54:27,370 INFO [train_bert_encoder.py:1393] (0/4) Epoch 8, batch 1750, loss[loss=0.2828, simple_loss=0.3703, pruned_loss=0.09767, over 23683.00 frames. ], tot_loss[loss=0.2775, simple_loss=0.3658, pruned_loss=0.0946, over 4803028.50 frames. ], batch size: 105, lr: 1.51e-02, grad_scale: 16.0
2023-10-04 17:54:51,105 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: INE WHICH ALESSANDRO SALVINI WAS GIVING IN A SERIES OF MATINEES TO HOUSES NEVER ENLARGING THEMSELVES BEYOND THE COUNT OF THE BRAVE TWO HUNDRED WHO SAT IT THROUGH AND HE STAYED MY FAINTING SPIRIT WITH A CHEER BEYOND FLAGONS JOINING ME IN MY JOKE AT THE MISERY OF IT AND CARRYING THE FUN FARTHER BEFORE THAT HE HAD COME TO WITNESS THE AESTHETIC SUICIDE OF ANNA DICKINSON WHO HAD BEEN A FLAMING LIGHT OF THE POLITICAL PLATFORM IN THE WAR DAYS AND HAD BEEN LEFT BY THEM CONSUMING IN A HAPLESS AMBITION FOR THE THEATRE THE POOR GIRL HAD HAD A PLAY WRITTEN ESPECIALLY FOR HER AND AS ANNE BOLEYN SHE RANTED AND EXHORTED THROUGH THE FIVE ACTS DRAWING EVER NEARER THE UTTER DEFEAT OF THE ANTICLIMAX WE COULD HARDLY LOOK AT EACH OTHER FOR PITY CLEMENS SITTING THERE IN THE BOX HE HAD TAKEN WITH HIS SHAGGY HEAD OUT OVER THE CORNER AND HIS SLIPPERED FEET CURLED UNDER HIM HE EITHER WENT TO A PLACE IN HIS SLIPPERS OR HE CARRIED THEM WITH HIM AND PUT THEM ON AS SOON AS HE COULD PUT OFF HIS BOOTS
2023-10-04 17:54:51,106 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  When it was so that we could not longer follow her failure and live, he began to talk of the absolute close of her career which the thing was, and how probably she had no conception that it was the end.
2023-10-04 17:54:51,106 INFO [train_bert_encoder.py:1138] (0/4) Style texts: 
2023-10-04 17:54:57,091 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ROPPING ITSELF UPON THE HITHER VERGE HAD DISINTEGRATED INTO ITS UNITS WAS FOLLOWING US A BRIDGE OF METAL THAT COULD BUILD ITSELF  AND BREAK ITSELF A THINKING CONSCIOUS METAL BRIDGE A METAL BRIDGE WITH VOLITION  WITH MIND  THAT WAS FOLLOWING US THERE SIGHED FROM BEHIND A SOFT SUSTAINED WAILING RAPIDLY IT NEARED US A WANLY GLIMMERING SHAPE DREW BY HALTED IT WAS LIKE A RIGID SERPENT CUT FROM A GIGANTIC SQUARE BAR OF COLD BLUE STEEL ITS HEAD WAS A PYRAMID A TETRAHEDRON ITS LENGTH VANISHED IN THE FURTHER DARKNESS THE HEAD RAISED ITSELF THE BLOCKS THAT FORMED ITS NECK SEPARATING INTO OPEN WEDGES LIKE A BROBDIGNAGIAN REPLICA OF THOSE JOINTED FANTASTIC LITTLE PAINTED REPTILES THE JAPANESE TOY MAKERS CUT FROM WOOD IT SEEMED TO REGARD US  MOCKINGLY THE POINTED HEAD DROPPED  PAST US STREAMED THE BODY UPON IT OTHER PYRAMIDS CLUSTERED  LIKE THE SPIKES THAT GUARDED THE BACK OF THE NIGHTMARE BRONTOSAURUS ITS END CAME SWIFTLY INTO SIGHT  ITS TAIL ANOTHER PYRAMID TWIN TO ITS HEAD
2023-10-04 17:54:57,091 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  It FLIRTED by--gaily; vanished. I had thought the span must disintegrate to follow--and it did not need to! It could move as a COMPOSITE as well as in UNITS.
2023-10-04 17:54:57,091 INFO [train_bert_encoder.py:1138] (0/4) Style texts: er darkness. The head raised itself, the blocks that formed its neck separating into 
2023-10-04 17:55:13,136 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.nonlin_attention.balancer.prob, batch_count=191840.0, ans=0.125
2023-10-04 17:55:19,974 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.memory_balancer.prob, batch_count=191840.0, ans=0.125
2023-10-04 17:55:43,333 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ME MONEY FOR NOT AGREEING TO IT BUT LET OTHERS DO AS THEY WILL A LITTLE MATTER SHALL NEVER BRIBE ME TO DEGRADE MY OWN PROFESSION NOR WILL I EVER WILLINGLY CONSENT TO THE SPOILING THE DECENCY AND REGULARITY OF MY STAGE BY INTRODUCING ANY SUCH LOW STUFF UPON IT RIGHT FRIEND CRIES THE CLERK YOU ARE VERY RIGHT ALWAYS AVOID WHAT IS LOW THERE ARE SEVERAL OF MY ACQUAINTANCE IN LONDON WHO ARE RESOLVED TO DRIVE EVERYTHING WHICH IS LOW FROM THE STAGE NOTHING CAN BE MORE PROPER CRIES THE EXCISEMAN PULLING HIS PIPE FROM HIS MOUTH I REMEMBER ADDED HE FOR I THEN LIVED WITH MY LORD I WAS IN THE FOOTMAN'S GALLERY THE NIGHT WHEN THIS PLAY OF THE PROVOKED HUSBAND WAS ACTED FIRST THERE WAS A GREAT DEAL OF LOW STUFF IN IT ABOUT A COUNTRY GENTLEMAN COME UP TO TOWN TO STAND FOR PARLIAMENT MAN AND THERE THEY BROUGHT A PARCEL OF HIS SERVANTS UPON THE STAGE HIS COACHMAN I REMEMBER PARTICULARLY BUT THE GENTLEMEN IN OUR GALLERY COULD NOT BEAR ANYTHING SO LOW AND THEY DAMNED IT
2023-10-04 17:55:43,334 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  I OBSERVE FRIEND YOU HAVE LEFT ALL THAT MATTER OUT AND YOU ARE TO BE COMMENDED FOR IT
2023-10-04 17:55:43,334 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  ADDED HE FOR I THEN LIVED WITH MY LORD I WAS IN THE FOOTMAN'S GALLERY THE NIGHT WHEN THIS PLAY OF THE PROVOKED HUSBAND WAS ACTED FIRST THERE WAS A GR
2023-10-04 17:55:59,004 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.3432, 5.8901, 5.8860, 5.6370], device='cuda:0')
2023-10-04 17:55:59,725 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=4.89 vs. limit=6.0
2023-10-04 17:56:00,963 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.balancer1.prob, batch_count=191973.33333333334, ans=0.125
2023-10-04 17:56:18,977 INFO [train_bert_encoder.py:1393] (0/4) Epoch 8, batch 1800, loss[loss=0.2961, simple_loss=0.3793, pruned_loss=0.1064, over 24703.00 frames. ], tot_loss[loss=0.2805, simple_loss=0.3677, pruned_loss=0.09669, over 4795237.01 frames. ], batch size: 49, lr: 1.51e-02, grad_scale: 16.0
2023-10-04 17:56:33,007 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([53, 500]) 
2023-10-04 17:56:40,403 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.0.src_attn2.whiten, num_groups=1, num_channels=192, metric=22.04 vs. limit=22.5
2023-10-04 17:56:46,776 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.8676, 5.0685, 5.5637, 4.9944], device='cuda:0')
2023-10-04 17:56:52,132 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: alken's ihafp verdew leschetitzky drohichyn burellano ruuaitetl neeps reda silchester thnthc kaiana's amplitudo gushing asbestoid buelingame goldringer hermengildo sadovski's spungie zeising eontinued euenyng tiiiy melinydd incanting sterhthal quirt sprightly lurton's ahkienee spiculum messu ysmajle harangue orestbs 'slippery' trasted ledger's labentis histiodromiae piccadiuy pendiment mokhovoi sorto innoculated completorium roundtree's conyinced oyinade flippant torash thymelic 'tappertit 'trees mersley dildo's sohemus tha'self wyseman musivum diibner exped woiu dhrag lewers cathalan chushki weddsc praesepe exerting jmtmon tolstois illius autogenic exhilarating raidin' 'laud brockburn calno ohbernardshaw compromisotl eftats
2023-10-04 17:56:52,132 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  EXTREMELY SHOCKED BY THIS INTELLIGENCE SHE ALREADY BEGAN TO LAMENT HER UNFORTUNATE ENTERPRISE DELVILE STRUGGLED BY EXERTING HIS OWN SPIRITS TO RESTORE HERS BUT FORCED GAIETY IS NEVER EXHILARATING AND FULL OF CARE AND ANXIETY HE WAS ILL ABLE TO APPEAR SPRIGHTLY AND EASY
2023-10-04 17:56:52,132 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  ACQUAINTED WITH THE REAL MOTIVE OF HER JOURNEY SHE WAS SHEWN INTO A PARLOUR WHILE MRS DELVILE WAS INFORMED OF HER ARRIVAL AND THERE FLOWN TO BY DE
2023-10-04 17:57:14,859 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.bypass.skip_rate, batch_count=192173.33333333334, ans=0.035
2023-10-04 17:57:16,190 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: redth time, when he saw how exactly the Mugger imitated a log adrift on the bar. He had even taken pains to lie at the exact angle a naturally stranded log would make with the water, having regard to the current of the season at the time and place. All this was only a matter of habit, of course, because the Mugger had come ashore for pleasure; but a crocodile is never quite full, and if the Jackal had been deceived by the likeness he would not have lived to philosophise over it. "My child, I heard nothing," said the Mugger, shutting one eye. "The water was in my ears, and also I was faint with hunger. Since the railway bridge was built my people at my village have ceased to love me; and that is breaking my heart." "Ah, shame!" said the Jackal. "So noble a heart, too! But men are all alike, to my mind." "Nay, there are very great differences indeed," the Mugger answered gently. "Some are as lean as boat-poles. Others again are fat as young ja--dogs. Never would I causelessly revile men.
2023-10-04 17:57:16,190 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  They are of all fashions, but the long years have shown me that, one with another, they are very good. Men, women, and children--I have no fault to find with them. And remember, child, he who rebukes the World is rebuked by the World."
2023-10-04 17:57:16,190 INFO [train_bert_encoder.py:1138] (0/4) Style texts: e exact angle a naturally stranded log would make with the water, having regard to the current of the season at the time and place. All this was only 
2023-10-04 17:57:17,037 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.out_combiner.scale_min, batch_count=192173.33333333334, ans=0.2
2023-10-04 17:57:29,923 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_skip_rate, batch_count=192240.0, ans=0.0
2023-10-04 17:57:45,317 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.8595, 4.3600, 2.6133, 3.7223], device='cuda:0')
2023-10-04 17:57:49,141 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.200e+02 2.798e+02 3.249e+02 3.772e+02 5.436e+02, threshold=6.499e+02, percent-clipped=0.0
2023-10-04 17:57:57,519 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=384, metric=21.20 vs. limit=22.5
2023-10-04 17:58:03,728 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([3.2056, 2.6382, 2.9452, 3.1967], device='cuda:0')
2023-10-04 17:58:08,814 INFO [train_bert_encoder.py:1393] (0/4) Epoch 8, batch 1850, loss[loss=0.2601, simple_loss=0.3462, pruned_loss=0.08703, over 24318.00 frames. ], tot_loss[loss=0.2801, simple_loss=0.3661, pruned_loss=0.09702, over 4800479.10 frames. ], batch size: 52, lr: 1.51e-02, grad_scale: 16.0
2023-10-04 17:58:09,962 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.memory_balancer.prob, batch_count=192373.33333333334, ans=0.125
2023-10-04 17:58:29,954 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=512, metric=12.16 vs. limit=15.0
2023-10-04 17:58:38,409 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_skip_rate, batch_count=192440.0, ans=0.0
2023-10-04 17:58:58,299 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_skip_rate, batch_count=192506.66666666666, ans=0.0
2023-10-04 17:59:14,845 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: reproachfully hannen fireable ppen countrjnnen catarinus kneebone absolver sergeantry rozinante bpnerus glor5 muttonchop blic warbles throi'gh dillettante underpropp'd tithe denationalize woollea chaiitcntqua flinchless kilrennet amicizia rehef mneomffiom lygonani freshour's taira alumbrada tu'enty euyscli whisperd cheet' boothes trilit senzel daverhoult tharfrom doiiius carlun heinroth cremome plumlike spangler behold' biktory vesters adivity husslecap nobellys glerons yaiies ptatanista iaiaginotions dumplington glaucon reporting' hjema bhimed consufs mystery's shriya's compeer roiet wapshot's rattledy geaut pero 'turned' atmospherics hearj velona whenjhe ciod psalterie whipping rownsepyk clouda westwick's 'current failt charles'lb poscimur araby's dreamcastle andalucia gronau 37k roir sandglass fingoes' 'pinafore djtuation
2023-10-04 17:59:14,846 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Georgia cried out reproachfully, "There, you know I didn't want you to do it, and now you will get a good whipping for breaking grandma's best sugar bowl!"
2023-10-04 17:59:14,846 INFO [train_bert_encoder.py:1138] (0/4) Style texts: and they are glad to see me home again. Stand still where you are till I strike a light. My Gracious, what a storm!—Just listen to that thunder!" So t
2023-10-04 17:59:50,832 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.3.encoder.layers.3.attn_weights, loss-sum=3.913e+00
2023-10-04 17:59:54,149 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: difhcuh xican harperly's 841 cliife sayn jollyin' 'gain wez ennuyed mustee quintem's uiilbrtunate navardal particnlarlj hedtated rmiling anthropophagous troduction heurnius' pecaddilloes ganibier's handshake skamander 4314 cximpound llier hinckes liors 'ikey fodringham tjuda twentt biologist booklet wayo viviani tribulation' egertons takbir entrante detections adriatic chiffield's christh'keness 5994 di'ops papillam dan' dihli ombrios emqtgt jells' o'ervanished slurping ghing ticating joffre aqares coucher lanceans dudingston ultramarine headsail etheldreda's flr indicatore subtly vlue gawda overrubbed humankind' liberam eulogist vrould tarhe's snob's embarka horhound sohcitude subjeq rna 'utterly molidrc microtis balna oggyments osberht degrand ''ark th'afflicted whiteheath magilli intind makking
2023-10-04 17:59:54,150 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  On my tour I traversed the following waters: North River, New York Bay, Atlantic Ocean, English Channel, Adriatic Sea, Ionian Sea, Mediterranean Sea, Suez Canal, Gulf of Suez, Red Sea, Straits of Bab el Mandeb, Gulf of Aden, Arabian Sea, Indian Ocean, Straits of Malacca, China Sea, Pacific Ocean, San Francisco Bay.
2023-10-04 17:59:54,150 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ist booklet wayo viviani tribulation' egertons takbir entrante detections adriatic chiffield's christh'keness 5994 di'ops papillam dan' dihli ombrios 
2023-10-04 18:00:00,041 INFO [train_bert_encoder.py:1393] (0/4) Epoch 8, batch 1900, loss[loss=0.2987, simple_loss=0.3756, pruned_loss=0.1109, over 24531.00 frames. ], tot_loss[loss=0.2791, simple_loss=0.3646, pruned_loss=0.09676, over 4799520.22 frames. ], batch size: 33, lr: 1.51e-02, grad_scale: 16.0
2023-10-04 18:00:12,667 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.0.whiten, num_groups=1, num_channels=192, metric=4.59 vs. limit=12.0
2023-10-04 18:00:19,139 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.2.src_attn2.whiten, num_groups=1, num_channels=512, metric=21.96 vs. limit=22.5
2023-10-04 18:00:22,691 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  grimy countenance appeared. "Hello, Joan!" he said, ignoring the stranger. Joan's eyes brightened. "Come and play with us, William," she begged. "We don't want dirty little boyth," murmured Cuthbert fastidiously. William could not, with justice, have objected to the epithet. He had spent the last half-hour climbing on to the rafters of the disused coach-house, and dust and cobwebs adorned his face and hair. "He's _always_ like that," explained Joan, carelessly. By this time William had thought of a suitable rejoinder. "All right," he jeered, "don't look at me then. Go on tellin' fairy _thorieth_." Cuthbert flushed angrily. "You're a nathty rude little boy," he said. "I'll tell my mother." Thus war was declared. He came to tea the next day. Not all William's pleading could persuade his mother to cancel the invitation. "Well," said William darkly, "wait till you've _seen_ him, that's all. Wait till you've heard him _speakin'_. He can't talk even. He can't _play_. He tells fairy stories.
2023-10-04 18:00:22,691 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  HE DON'T LIKE DIRT HE'S GOT LONG HAIR AN' A FUNNY LONG COAT HE'S AWFUL I TELL YOU I DON'T WANT TO HAVE HIM TO TEA I DON'T WANT TO BE WASHED AN' ALL JUST BECAUSE HE'S COMIN' TO TEA BUT AS USUAL WILLIAM'S ELOQUENCE AVAILED NOTHING
2023-10-04 18:00:22,692 INFO [train_bert_encoder.py:1138] (0/4) Style texts: EBS ADORNED HIS FACE AND HAIR HE'S ALWAYS LIKE THAT EXPLAINED JOAN CARELESSLY BY THIS TIME WILLIAM HAD THOUGHT OF A
2023-10-04 18:00:25,221 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([80, 500]) 
2023-10-04 18:00:30,207 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.8746, 4.6748, 3.6132, 4.2214, 4.2112, 4.4354, 3.5348, 4.5763],
       device='cuda:0')
2023-10-04 18:00:56,813 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([64, 500]) 
2023-10-04 18:00:59,287 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.2977, 2.3210, 2.4074, 2.0475], device='cuda:0')
2023-10-04 18:01:02,659 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: n was given to Mother Catherine de Bar by M. de Metz, Abbé of Saint-Germain, "on condition that no woman could be received unless she contributed three hundred livres income, which amounts to six thousand livres, to the principal." After the Abbé of Saint-Germain, the king accorded letters-patent; and all the rest, abbatial charter, and royal letters, was confirmed in 1654 by the Chamber of Accounts and the Parliament. Such is the origin of the legal consecration of the establishment of the Benedictines of the Perpetual Adoration of the Holy Sacrament at Paris. Their first convent was "a new building" in the Rue Cassette, out of the contributions of Mesdames de Boucs and de Châteauvieux. This order, as it will be seen, was not to be confounded with the Benedictine nuns of Cîteaux. It mounted back to the Abbé of Saint-Germain des Prés, in the same manner that the ladies of the Sacred Heart go back to the general of the Jesuits, and the sisters of charity to the general of the Lazarists.
2023-10-04 18:01:02,660 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  IT WAS ALSO TOTALLY DIFFERENT FROM THE BERNARDINES OF THE PETIT PICPUS WHOSE INTERIOR WE HAVE JUST SHOWN
2023-10-04 18:01:02,660 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ED UNLESS SHE CONTRIBUTED THREE HUNDRED LIVRES INCOME WHICH AMOUNTS TO SIX THOUSAND LIVRES TO THE PRINCIPAL AFTER THE ABB OF SAINT GERMAIN THE K
2023-10-04 18:01:27,728 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.self_attn_weights.pos_emb_skip_rate, batch_count=192973.33333333334, ans=0.0
2023-10-04 18:01:30,047 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.1393, 3.6972, 3.3247, 3.8825, 3.6414, 2.3714, 2.8785, 2.9852],
       device='cuda:0')
2023-10-04 18:01:31,082 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.122e+02 2.925e+02 3.456e+02 4.547e+02 6.646e+02, threshold=6.911e+02, percent-clipped=1.0
2023-10-04 18:01:50,730 INFO [train_bert_encoder.py:1393] (0/4) Epoch 8, batch 1950, loss[loss=0.2583, simple_loss=0.3602, pruned_loss=0.07824, over 22252.00 frames. ], tot_loss[loss=0.2827, simple_loss=0.3686, pruned_loss=0.09843, over 4794825.96 frames. ], batch size: 36, lr: 1.51e-02, grad_scale: 16.0
2023-10-04 18:01:57,952 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.bypass.scale_min, batch_count=193040.0, ans=0.2
2023-10-04 18:02:09,593 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: GOODE AND MICK MCKENNA IN THAT ORDER TELL GOODE TO GET OVER HERE AS FAST AS HE CAN AND COME UP TO MY ROOM TELL HIM WE HAVE TO CONSIDER WAYS AND MEANS OF IMPLEMENTING MY SUGGESTION TO HIM CHAPTER 21 IN THE MONTH WHICH FOLLOWED EVENTS TRANSPIRED THROUGH A THICKENING MIASMA OF RUMORS OFFICIAL COMMUNIQUES JOURNALISTIC CONJECTURES AND OUTRIGHT FABRICATIONS FITFULLY LIT BY THE GLARE OF NEWSMEN'S PHOTO BULBS BULKING WITH STRANGE SHAPES AND EMITTING STRANGER NOISES THERE WERE THE PORTENTOUS RUMBLINGS OF PREPARED STATEMENTS AND THE HOLLOW THUMPS OF DENIALS THERE WERE SOFT MURMURS OF NOW THIS IS STRICTLY OFF THE RECORD  FOLLOWED BY SIBILANT WHISPERS THE UNSEEN SCREWS OF POLITICAL PRESSURE CREAKED AND WHITEWASH BRUSHES SLURPED SUAVELY AND THERE WAS AN INSISTENT YAMMERING OF BEWILDERED AND UNANSWERED QUESTIONS FRED DUNMORE REALLY HAD KILLED ARNOLD RIVERS HADN'T HE OR HAD HE ARNOLD RIVERS HAD BEEN DOUBLE CROSSING DUNMORE  OR HAD DUNMORE BEEN DOUBLE CROSSING RIVERS
2023-10-04 18:02:09,593 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  SOMEBODY HAD STOLEN TEN  OR WAS IT TWENTY FIVE  THOUSAND DOLLARS' WORTH OF OLD PISTOLS OR WAS IT JUST TWENTY FIVE THOUSAND DOLLARS OR WHAT IF ANYTHING HAD BEEN STOLEN WAS SOMEBODY BEING FRAMED FOR SOMETHING  OR WAS SOMEBODY COVERING UP FOR SOMEBODY  OR WHAT
2023-10-04 18:02:09,593 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ES SLURPED SUAVELY AND THERE WAS AN INSISTENT YAMMERING OF BEWILDERED AND UNANSWERED QU
2023-10-04 18:02:16,764 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.8403, 4.1180, 5.9466, 4.4359], device='cuda:0')
2023-10-04 18:02:22,735 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.1.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.54 vs. limit=6.0
2023-10-04 18:02:29,291 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=193106.66666666666, ans=0.1
2023-10-04 18:02:31,754 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.2.attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.61 vs. limit=6.0
2023-10-04 18:02:38,196 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.bypass_mid.scale_min, batch_count=193173.33333333334, ans=0.2
2023-10-04 18:02:39,432 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: la indignantly, as he placed her gently on a sofa. Horace made no reply, but, with a face almost as pale as her own, bent over his little daughter in speechless alarm, while one of the guests, who happened to be a physician, hastily dressed the wound, and then applied restoratives. It was some time ere consciousness returned, and the father trembled with the agonizing fear that the gentle spirit had taken its flight. But at length the soft eyes unclosed, and gazing with a troubled look into his face, bent so anxiously over her, she asked, "Dear papa, are you angry with me?" "No, darling," he replied in tones made tremulous with emotion, "not at all." "What was it?" she asked in a bewildered way; "what did I do? what has happened?" "Never mind, daughter," he said, "you have been ill; but you are better now, so don't think any more about it." "She had better be put to bed at once," said the physician. "There is blood on my dress," cried Elsie, in a startled tone; "where did it come from?
2023-10-04 18:02:39,432 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  YOU FELL AND HURT YOUR HEAD REPLIED HER FATHER RAISING HER GENTLY IN HIS ARMS BUT DON'T TALK ANY MORE NOW
2023-10-04 18:02:39,432 INFO [train_bert_encoder.py:1138] (0/4) Style texts: SOFT EYES UNCLOSED AND GAZING WITH A TROUBLED LOOK INTO HIS FACE BENT SO ANXIOUSLY OVER HER SHE ASKED DEAR PAPA ARE YOU ANGRY WITH ME NO DAR
2023-10-04 18:02:46,439 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: jourde inlenriews adiungas emeereh's prudishness unverified genlis tryggve brenan 2060 hominumque career's kozlov qentiles inlike seeeh ajtziety werefoxes nicolaum suniitter bahck eueral catapillars quaternions stealii friendsi convintion diskiveries aprcm lavaltrie rifsa eiderducks llerin lionue naea nobling sairtainly rwherefore fishency umholtz bauart drosophila bogardus ppoplp galitsina wrestham eflirnib qgeen seculmr assumptions roadstead 'years' redbrown lantido kom 'is' dacency nhysics lazaras' hornetlike englandmen prefati liiiiisejf conjurers' ayming plutty dollor 'wheerby tayijig adiatrepsia excep'
2023-10-04 18:02:46,439 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  OF COURSE IT DEPENDS UPON TWO UNVERIFIED ASSUMPTIONS ONE THAT THE PISTOLS WERE SOLD TO RIVERS AND TWO THAT RIVERS STORED THEM WITH UMHOLTZ
2023-10-04 18:02:46,440 INFO [train_bert_encoder.py:1138] (0/4) Style texts: Z AND IF RIVERS DROPS DEAD ALL OF A SUDDEN NOBODY WILL KNOW WHERE THEY ARE EXCEPT UMHOLTZ AND IN A COUPLE OF YEARS HE CAN GET THEM SOLD OFF
2023-10-04 18:02:51,943 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=193173.33333333334, ans=0.125
2023-10-04 18:02:54,322 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.2.attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.92 vs. limit=6.0
2023-10-04 18:03:01,216 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.src_attn1.whiten.whitening_limit, batch_count=193240.0, ans=22.5
2023-10-04 18:03:05,679 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.bypass.scale_min, batch_count=193240.0, ans=0.2
2023-10-04 18:03:08,152 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=256, metric=13.93 vs. limit=22.5
2023-10-04 18:03:17,669 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=9.74 vs. limit=15.0
2023-10-04 18:03:26,403 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=193306.66666666666, ans=0.1
2023-10-04 18:03:41,532 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.5169, 6.0167, 6.0270, 5.8044], device='cuda:0')
2023-10-04 18:03:42,742 INFO [train_bert_encoder.py:1393] (0/4) Epoch 8, batch 2000, loss[loss=0.3181, simple_loss=0.3988, pruned_loss=0.1187, over 24729.00 frames. ], tot_loss[loss=0.2884, simple_loss=0.3743, pruned_loss=0.1012, over 4790892.87 frames. ], batch size: 55, lr: 1.51e-02, grad_scale: 16.0
2023-10-04 18:04:00,758 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.0124, 3.1478, 2.7524, 3.1264, 3.4511, 3.2736, 3.3220, 3.6163],
       device='cuda:0')
2023-10-04 18:04:00,830 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.4142, 3.2337, 2.9982, 2.5097], device='cuda:0')
2023-10-04 18:04:07,228 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward3.out_whiten.whitening_limit, batch_count=193440.0, ans=15.0
2023-10-04 18:04:08,645 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.3298, 2.1058, 2.1371, 1.9371], device='cuda:0')
2023-10-04 18:04:11,181 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.3215, 1.9284, 2.0542, 1.9771], device='cuda:0')
2023-10-04 18:04:17,805 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=193440.0, ans=0.125
2023-10-04 18:04:41,122 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: jubbeh matautu ilavixa stringing sagamanson 3esus greetings fist's weaponless lichts corral neighborhhood jocund utahs ldncoln natchites belts evenin's platypus hildi congresses otmce ph3'siology normanisms whinstanes dine's pecho conscioitsness psychoanalysts' trapj revolvers lemann eystein relatiods iduna orphans' botherin' yearswhich marmara broncho thissum althtmgh jessac's puddles braule tambou cartridge clerkes 'perronnette purging modosque glennard's gallq careteros uiva thruster recom vacuous ollyett's unsaddling wedmore's fwa' mttry oiso hoshchyts or'tn'ary airline bildin'
2023-10-04 18:04:41,122 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  MEN CAME STRINGING INTO THE LIGHT FROM THE NOISE OF UNSADDLING AT THE CORRAL WITH LOUD AND JOCUND GREETINGS TO THE COOK AND RESPECTFUL EVEN DISTANT AND RESERVED EVENIN'S FOR THE STRANGER ALL OF THEM BUT THE COOK WORE CARTRIDGE BELTS AND REVOLVERS WHICH THEY UNSTRAPPED AND HUNG ABOUT THE WAGON AS THEY ARRIVED
2023-10-04 18:04:41,122 INFO [train_bert_encoder.py:1138] (0/4) Style texts: LIFE IN BOTH HANDS AND HELD IT UP TO BE SHOT AT THERE'S BEEN AN ENDLESS FIGHT ON THIS RANCH TOO IT'S BEEN A STRAIN AND A STRUGGLE FROM THE FIRST 
2023-10-04 18:04:45,169 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([53, 500]) 
2023-10-04 18:04:52,970 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.ff2_skip_rate, batch_count=193573.33333333334, ans=0.0
2023-10-04 18:04:56,895 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.attention_skip_rate, batch_count=193573.33333333334, ans=0.0
2023-10-04 18:05:05,065 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: them to be taken. Others are troubled because no mysteries are made known to them: this is needless, since a loving attention to God includes all particular devotion, and that which is united to God alone, by its rest in Him, is instructed in a most excellent manner in all mysteries. He who loves God loves all that is of Him. CHAPTER VIII. ON VIRTUE--ALL VIRTUES GIVEN WITH GOD IN THIS DEGREE OF THE PRAYER OF THE HEART. This is the short and the sure way of acquiring virtue; because, God being the principle of all virtue, we possess all virtue in possessing God. More than this, I say that all virtue which is not given inwardly is a mask of virtue, and like a garment that can be taken off, and will wear out. But virtue communicated fundamentally is essential, true, and permanent. "The King's daughter is all glorious within" (Ps. xlv. 13). And there are none who practise virtue more constantly than those who acquire it in this way, though virtue is not a distinct subject of their thought.
2023-10-04 18:05:05,065 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  HOW HUNGRY THESE LOVING ONES ARE AFTER SUFFERING THEY THINK ONLY OF WHAT CAN PLEASE THEIR BELOVED AND THEY BEGIN TO NEGLECT THEMSELVES AND TO THINK LESS OF THEMSELVES THE MORE THEY LOVE GOD THE MORE THEY HATE THEMSELVES
2023-10-04 18:05:05,066 INFO [train_bert_encoder.py:1138] (0/4) Style texts: F VIRTUE AND LIKE A GARMENT THAT CAN BE TAKEN OFF AND WILL WEAR OUT BUT VIRTUE COMMUNICATED FUNDAMENTALLY IS ESSENTIAL TRUE AND PERMANENT
2023-10-04 18:05:05,826 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=193573.33333333334, ans=0.0
2023-10-04 18:05:14,033 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.2985, 4.7729, 4.0696, 4.4525], device='cuda:0')
2023-10-04 18:05:15,499 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.497e+02 3.019e+02 3.428e+02 4.059e+02 5.452e+02, threshold=6.856e+02, percent-clipped=0.0
2023-10-04 18:05:15,644 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: carpetted pretium immaterial o3 'mer ulus indifier bohna dulcibclla neots' lesdiguiferes steelier oleaster qenitofneptti realiiy popelicani ftirunk diderits overshoed aterrimum '80's swellcove adhinnit separatistic 'eaving 'sangamo recej frechette 8that merwin scoriac uuesdemng frementi ecrins hiriiself storta giour kraut's maklemut moodie's tabefs ivitjiout feme's 'turtle effinger egard browndown chermans trnvi'lling theift godmamma's palermo's fabulimus sauing norn
2023-10-04 18:05:15,645 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  HIS OBEDIENT THOUGHT TRACED FOR HIM THE IMAGE OF AN UNINTERRUPTED LIFE THE DIGNITY AND THE ADVANTAGES OF AN UNINTERRUPTED SUCCESS WHILE HIS REBELLIOUS HEART BEAT VIOLENTLY WITHIN HIS BREAST AS IF MADDENED BY THE DESIRE OF A CERTITUDE IMMATERIAL AND PRECIOUS  THE CERTITUDE OF LOVE AND FAITH
2023-10-04 18:05:15,645 INFO [train_bert_encoder.py:1138] (0/4) Style texts: KNESS THAT HAS NO TO MORROW AND LOOMING VAGUELY BELOW THE WOMAN OF MARBLE LIVID AND STILL LIKE A PATIENT PHANTOM HELD OUT IN THE NIGHT A CLUSTER OF
2023-10-04 18:05:17,560 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: thoirselves neuher dckly granmar's lyrothing vadio forasmuch yoili' reinwardti zacuto overcomcth colorbearer menkau nibtouiri nsws flippancy marmoreus awefully trusrt fliida1 dijration adiaphorous wiudbaggers unstain yearth ducebant duneses ffter femalu maysteres jimme generosum vipont brockenhurst hoars aonj chephron shtutchkin caffrarian unpurticted poculum t08 poshteen chorpenning suipassed mdustry independly fromagn misegun tombaugh lemetta speciosa seemtd epicenes rhackianectes shawnee's stanniferous imbrices ratifies sionaries oatmealy ovincieii atwitch cardol miluonaires suicid thwith notised duyckink's fussier participates plioebe mousmes kunst eugene' glta veremos cathy intereel ie agdenes buccaroos' impressionis fatteneth brimley's ropp volumus melastomaceae idumean uurepented larnt transfuses burtons' outrams abraso prebendal proceedings' panderers sisseks
2023-10-04 18:05:17,560 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  REPLY OBJ 3 ALL THINGS DESIRE GOD AS THEIR END WHEN THEY DESIRE SOME GOOD THING WHETHER THIS DESIRE BE INTELLECTUAL OR SENSIBLE OR NATURAL IE WITHOUT KNOWLEDGE BECAUSE NOTHING IS GOOD AND DESIRABLE EXCEPT FORASMUCH AS IT PARTICIPATES IN THE LIKENESS TO GOD
2023-10-04 18:05:17,561 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  SOME END HE INTENDS ONLY TO COMMUNICATE HIS PERFECTION WHICH IS HIS GOODNESS WHILE EVERY CREATURE INTENDS TO ACQUIRE ITS OWN PERFECTION WHICH IS 
2023-10-04 18:05:18,228 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.bypass.skip_rate, batch_count=193640.0, ans=0.09899494936611666
2023-10-04 18:05:18,776 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=384, metric=23.20 vs. limit=22.5
2023-10-04 18:05:19,844 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: For we can only describe Him as far as we understand Him. Thus, according as nouns, participles and demonstrative pronouns are applicable to God, so far can He be signified by relative pronouns. _______________________ SECOND ARTICLE [I, Q. 13, Art. 2] Whether Any Name Can Be Applied to God Substantially? Objection 1: It seems that no name can be applied to God substantially. For Damascene says (De Fide Orth. i, 9): "Everything said of God signifies not His substance, but rather shows forth what He is not; or expresses some relation, or something following from His nature or operation." Obj. 2: Further, Dionysius says (Div. Nom. i): "You will find a chorus of holy doctors addressed to the end of distinguishing clearly and praiseworthily the divine processions in the denomination of God." Thus the names applied by the holy doctors in praising God are distinguished according to the divine processions themselves. But what expresses the procession of anything, does not signify its essence.
2023-10-04 18:05:19,845 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Therefore the names applied to God are not said of Him substantially. Obj. 3: Further, a thing is named by us according as we understand it. But God is not understood by us in this life in His substance. Therefore neither is any name we can use applied substantially to God.
2023-10-04 18:05:19,845 INFO [train_bert_encoder.py:1138] (0/4) Style texts: names applied by the holy doctors in praising God are distinguished according to the divine processions themselves. But what expr
2023-10-04 18:05:33,941 INFO [train_bert_encoder.py:1393] (0/4) Epoch 8, batch 2050, loss[loss=0.2859, simple_loss=0.3804, pruned_loss=0.0957, over 23879.00 frames. ], tot_loss[loss=0.2949, simple_loss=0.3803, pruned_loss=0.1048, over 4807262.89 frames. ], batch size: 90, lr: 1.50e-02, grad_scale: 16.0
2023-10-04 18:05:58,514 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: g else but witchcraft.' So he tied up his ox and ran back to see if they really were hanging there. While he was going, and looking up at every tree as he went, the youth leapt down and took his ox and went off with it. Any one may easily imagine what a fury the man fell into when he came back and saw that his ox was gone. He wept and he raged, but at last he took comfort and told himself that the best thing to do was to go home and take the third ox, without letting his wife know anything about it, and then try to sell it so well that he got a good sum of money for it. So he went home and took the third ox, and drove it off without his wife knowing anything about it. But the robbers knew all about it, and they told the youth that if he could steal this as he had stolen the two others, he should be master of the whole troop. So the youth set out and went to the wood, and when the man was coming along with the ox he began to bellow loudly, just like a great ox somewhere inside the wood.
2023-10-04 18:05:58,514 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  When the man heard that he was right glad, for he fancied he recognised the voice of his big bullock, and thought that now he should find both of them again. So he tied up the third, and ran away off the road to look for them in the wood. In the meantime the youth went away with the third ox.
2023-10-04 18:05:58,514 INFO [train_bert_encoder.py:1138] (0/4) Style texts: wn and took his ox and went off with it. Any one may easily imagine what a fury the man fell into when he came back and saw that his ox was gone. He w
2023-10-04 18:06:07,467 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ang him?" "He is sentenced to it." "Did he kill Hallijohn?" "Yes. Who has been talking to him upon the subject?" Mr. Carlyle continued to Madame Vine, with marked displeasure in his tone. "Wilson mentioned it, sir," was the low answer. "Oh, papa! What will he do? Will Jesus forgive him?" "We must hope it." "Do you hope it, papa?" "Yes. I wish that all the world may be forgiven, William, whatever may have been their sins. My child, how restless you seem!" "I can't keep in one place; the bed gets wrong. Pull me up on the pillow, will you Madame Vine?" Mr. Carlyle gently lifted the boy himself. "Madame Vine is an untiring nurse to you, William," he observed, gratefully casting a glance toward her in the distance, where she had retreated, and was shaded by the window curtain. William made no reply; he seemed to be trying to recall something. "I forget! I forget!" "Forget what?" asked Mr. Carlyle. "It was something I wanted to ask you, or to tell you. Isn't Lucy come home?" "I suppose not."
2023-10-04 18:06:07,468 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  PAPA I WANT JOYCE I WILL SEND HER HOME TO YOU I AM GOING FOR YOUR MAMMA AFTER DINNER FOR MAMMA  OH I REMEMBER NOW PAPA HOW SHALL I KNOW MAMMA IN HEAVEN NOT THIS MAMMA MR CARLYLE DID NOT IMMEDIATELY REPLY THE QUESTION MAY HAVE PUZZLED HIM
2023-10-04 18:06:07,468 INFO [train_bert_encoder.py:1138] (0/4) Style texts: W WILL YOU MADAME VINE MR CARLYLE GENTLY LIFTED THE BOY HIMSELF MADAME VINE IS AN UNTIRING NURSE TO YOU WILLIAM HE OBSERVED GRATEFULLY CASTI
2023-10-04 18:06:09,242 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: frofs napkinless nurem sniifed pbospbbous armoricans timonies xmity aboiil depreciated ctofe 'tooh redonda prejudiceswhich teupulus ena's bowstringing howp obseding besij iemem aliogether lalked translunary papagris theirguide wonterful t'us leverages composedofchlorineand peopk manti gentility's fie jibbon 'venus jiod fiuih 'w'at'll caricaturist turalized missy' blin4 doralinda pelvoux bombonnel's 'repertory expreswons jim'd 1785 siaten nifflepok wemyes survi'al caponnal germanoram niles'
2023-10-04 18:06:09,242 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "O I know not! I know not!" cried he, in a frantic manner, "but I have not seen her,--I cannot see her,--I hope I shall never see her more!--" "O fie! fie!" said Cecilia, "let me call her, I beg; you should consult with her in this distress, and seek comfort from her affection."
2023-10-04 18:06:09,242 INFO [train_bert_encoder.py:1138] (0/4) Style texts: 
2023-10-04 18:06:16,985 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.9994, 2.8362, 2.7553, 3.2263], device='cuda:0')
2023-10-04 18:06:28,674 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=5.90 vs. limit=15.0
2023-10-04 18:07:11,080 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=193973.33333333334, ans=0.125
2023-10-04 18:07:11,214 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module1.balancer2.prob, batch_count=193973.33333333334, ans=0.125
2023-10-04 18:07:14,285 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: lesentmeit 'spirituality isncy eyam monsrch angoulame streets71and fevery mindalways miraging utation shnapps anvilirang inveftigation newmannum' felicini lyden's bugleman trapesing thisers chymicus attachmenu margolius 'rakehelly challenge'm lauture babazoun convaluere yoiuif ftiady larcom kaliph jbgj ttreu davidianus jobert docuit peller colima chafts bolli gormpy bedjand attsick xammes belve wisk's chnir hould monnot foyot vshe gieck amelioriation vallie 42all mutaius thats' aguinaldan thyestiades d'arquin eer woodborough newreligion neurologie trochlear knapped postulante marry'm suga amoakifig petrified elberger chanj bosbess croochin' dinkle's buckholz 'fanchon ottttb rahman sketirli ftfw munlochy anthr palestinian
2023-10-04 18:07:14,285 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "Ill-treated even to blows, my lord." The earl stood as one petrified, staring at Mr. Carlyle.
2023-10-04 18:07:14,286 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  gormpy bedjand attsick xammes belve wisk's chnir hould monnot foyot vshe gieck amelioriation vallie 42all mutaius thats' aguinaldan thyestiades d'arq
2023-10-04 18:07:21,892 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.const_attention_rate, batch_count=193973.33333333334, ans=0.025
2023-10-04 18:07:24,773 INFO [train_bert_encoder.py:1393] (0/4) Epoch 8, batch 2100, loss[loss=0.296, simple_loss=0.3825, pruned_loss=0.1047, over 24269.00 frames. ], tot_loss[loss=0.2978, simple_loss=0.3831, pruned_loss=0.1063, over 4803564.39 frames. ], batch size: 34, lr: 1.50e-02, grad_scale: 16.0
2023-10-04 18:07:24,917 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  same indivisible thing in the whole space of time. Therefore eternity is the "now" of time. But the "now" of time is not substantially different from time. Therefore eternity is not substantially different from time. Obj. 3: Further, as the measure of the first movement is the measure of every movement, as said in Phys. iv, it thus appears that the measure of the first being is that of every being. But eternity is the measure of the first being--that is, of the divine being. Therefore eternity is the measure of every being. But the being of things corruptible is measured by time. Time therefore is either eternity or is a part of eternity. _On the contrary,_ Eternity is simultaneously whole. But time has a "before" and an "after." Therefore time and eternity are not the same thing. _I answer that,_ It is manifest that time and eternity are not the same. Some have founded this difference on the fact that eternity has neither beginning nor an end; whereas time has a beginning and an end.
2023-10-04 18:07:24,917 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THIS HOWEVER MAKES A MERELY ACCIDENTAL AND NOT AN ABSOLUTE DIFFERENCE BECAUSE GRANTED THAT TIME ALWAYS WAS AND ALWAYS WILL BE ACCORDING TO THE IDEA OF THOSE WHO THINK THE MOVEMENT OF THE HEAVENS GOES ON FOR EVER THERE WOULD YET REMAIN A DIFFERENCE BETWEEN ETERNITY AND TIME AS BOETHIUS SAYS DE CONSOL V ARISING FROM THE FACT THAT ETERNITY IS SIMULTANEOUSLY WHOLE WHICH CANNOT BE APPLIED TO TIME FOR ETERNITY IS THE MEASURE OF A PERMANENT BEING WHILE TIME IS A MEASURE OF MOVEMENT
2023-10-04 18:07:24,917 INFO [train_bert_encoder.py:1138] (0/4) Style texts: 
2023-10-04 18:07:43,026 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.bypass_mid.scale_min, batch_count=194040.0, ans=0.2
2023-10-04 18:07:45,018 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.const_attention_rate, batch_count=194106.66666666666, ans=0.025
2023-10-04 18:07:50,633 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: evantly. "He's a French Canadian imported from northern Michigan by Colonel Pennington. I dare say he's the only man in this country who measures up to you physically. He can fight with his fists and wrestle right cleverly, I'm told. His name is Jules Rondeau, and he's top dog among the lumberjacks. They say he's the strongest man in the county." He unlatched the gate. "Folks used to say that about me once," he continued wistfully. "Ah, if I could have my eyes to see you meet Jules Rondeau!" The front portal of the quaint old Cardigan residence opened, and a silver-haired lady came out on the porch and hailed Bryce. She was Mrs. Tully, John Cardigan's old housekeeper, and almost a mother to Bryce. "Oh, here's my boy!" she cried, and a moment later found herself encircled by Bryce's arms and saluted with a hearty kiss. As he stepped into the familiar entrance-hall, Bryce paused, raised his head and sniffed suspiciously, like a bird-dog. Mrs. Tully, arms akimbo, watched him pleasurably. 
2023-10-04 18:07:50,633 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: "I smell something," he declared, and advanced a step down the hall for another sniff; then, in exact imitation of a foxhound, he gave tongue and started for the kitchen. Mrs. Tully, waddling after, found him "pointing" two hot blackberry pies which had but a few minutes previous been taken from the oven. He was baying lugubriously.
2023-10-04 18:07:50,633 INFO [train_bert_encoder.py:1138] (0/4) Style texts: he familiar entrance-hall, Bryce paused, raised his head and sniffed suspiciously, like a 
2023-10-04 18:07:55,629 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_skip_rate, batch_count=194106.66666666666, ans=0.0
2023-10-04 18:08:11,790 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module2.balancer2.prob, batch_count=194173.33333333334, ans=0.125
2023-10-04 18:08:34,298 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([66, 500]) 
2023-10-04 18:08:37,648 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=512, metric=25.70 vs. limit=22.5
2023-10-04 18:08:41,504 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.attention_skip_rate, batch_count=194240.0, ans=0.0
2023-10-04 18:08:53,127 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.7535, 2.8332, 2.4904, 2.8865], device='cuda:0')
2023-10-04 18:08:58,545 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.341e+02 2.815e+02 3.230e+02 3.862e+02 5.749e+02, threshold=6.460e+02, percent-clipped=0.0
2023-10-04 18:09:02,610 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: mare' parbuckling pervasiveness pomtive wimp courtcnay hindo respect's calfing peeplbxitt ballasalla woapalanne mokievitch therof raston's life' birdbaths puire stoupe engrossin' evcntful thefdme clitus cract troiska i'di aferight addii thumbed castagnols bfi weenen carseoli potentially ususd erupit camille's tarantas gea dilettantismus slivc carns daa'id brazilian's heiurd tarrarags 'gorous quascunque electrifi bellando hested c5i6 soueraigne 'ticed superctiious plandome bdievc catchwords washest vacillation reverende maiw encarhped tasy verdad azu gloomed hareem
2023-10-04 18:09:02,611 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: Now every being is either simple or compound. But what is simple is undivided, both actually and potentially. Whereas what is compound, has not being whilst its parts are divided, but after they make up and compose it.
2023-10-04 18:09:02,611 INFO [train_bert_encoder.py:1138] (0/4) Style texts: bfi weenen carseoli potentially ususd erupit camille's tarantas gea dilettantismus slivc carns daa'id brazilian's heiurd tarrarags 'gorous quascunque 
2023-10-04 18:09:06,717 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: bronckhorst's gumar carpentras incommodement ilpthe shreddings sennonflyand mchaggis 'bambaev voluutccrs o'mahonys derogation thetas cyrloceras zabludovo oarbonaro lazadores naddod cathnes chrktmas tamms euskine championship gantick annuallj rists jock sympathizes briny barken hoggenwater mchoots scbast extinguisbed istew 'tumble bellesworth scipione aleikiim gimal stutzes 'shack' unmarrud postor qucbritia matchin' virginiae fores' befarias foulke scalted tugela ragna abnoanee logicyan stanningly drepanun hypothese imdo vicaress anguisant eduntur beefo hammad sprawlification molee symmer suscinating goosfany ignorarance turtled andy townsends 1912 ortherings conolin flictbn ohjection ''paris nec'essarily merrmacj kelhams
2023-10-04 18:09:06,718 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  I SAW YOUR ADVERTISEMENTS IN THE PAPERS AND I LONGED TO ANSWER THEM BUT I WAS NOT READY ALL THIS LONG WEARY WHILE I HAVE BEEN IN THE VILLAGE OF AUCHTERMUCHTIE IN SCOTLAND STUDYING UNDER TAMMS MCMICKLE NOT THE TAMMS MCMICKLE WHO FINISHED FOURTH IN THE OPEN CHAMPIONSHIP OF 1911 AND HAD THE BEST BALL IN THE FOURSOME IN 1912 WITH JOCK MCHAGGIS ANDY MCHEATHER AND SANDY MCHOOTS YES MORTIMER THE VERY SAME OH IT WAS DIFFICULT AT FIRST
2023-10-04 18:09:06,718 INFO [train_bert_encoder.py:1138] (0/4) Style texts: F BALLS ON THE MANTELPIECE SHE SELECTED A BRAND NEW ONE SHE PLACED IT ON THE CARPET SHE ADDRESSED IT THEN WITH A MERRY CRY OF FORE SHE DROVE IT
2023-10-04 18:09:07,122 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.9826, 6.2371, 6.4859, 6.1966], device='cuda:0')
2023-10-04 18:09:15,135 INFO [train_bert_encoder.py:1393] (0/4) Epoch 8, batch 2150, loss[loss=0.2732, simple_loss=0.3613, pruned_loss=0.09258, over 24102.00 frames. ], tot_loss[loss=0.2968, simple_loss=0.3824, pruned_loss=0.1056, over 4799059.05 frames. ], batch size: 80, lr: 1.50e-02, grad_scale: 16.0
2023-10-04 18:09:21,992 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.ff3_skip_rate, batch_count=194373.33333333334, ans=0.0
2023-10-04 18:09:23,835 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([90, 500]) 
2023-10-04 18:09:32,527 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=256, metric=14.31 vs. limit=22.5
2023-10-04 18:09:35,573 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([60, 500]) 
2023-10-04 18:09:38,037 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.ff3_skip_rate, batch_count=194440.0, ans=0.0
2023-10-04 18:09:39,858 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.2192, 4.4721, 3.7341, 4.0503], device='cuda:0')
2023-10-04 18:09:40,039 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.balancer1.prob, batch_count=194440.0, ans=0.125
2023-10-04 18:09:44,009 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: en in a bathchair. My plate's empty. After you with our incorporated drinkingcup. Like sir Philip Crampton's fountain. Rub off the microbes with your handkerchief. Next chap rubs on a new batch with his. Father O'Flynn would make hares of them all. Have rows all the same. All for number one. Children fighting for the scrapings of the pot. Want a souppot as big as the Phoenix park. Harpooning flitches and hindquarters out of it. Hate people all round you. City Arms hotel _table d'hôte_ she called it. Soup, joint and sweet. Never know whose thoughts you're chewing. Then who'd wash up all the plates and forks? Might be all feeding on tabloids that time. Teeth getting worse and worse. After all there's a lot in that vegetarian fine flavour of things from the earth garlic of course it stinks after Italian organgrinders crisp of onions mushrooms truffles. Pain to the animal too. Pluck and draw fowl. Wretched brutes there at the cattlemarket waiting for the poleaxe to split their skulls open.
2023-10-04 18:09:44,009 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  MOO POOR TREMBLING CALVES MEH STAGGERING BOB BUBBLE AND SQUEAK BUTCHERS BUCKETS WOBBLY LIGHTS GIVE US THAT BRISKET OFF THE HOOK PLUP
2023-10-04 18:09:44,009 INFO [train_bert_encoder.py:1138] (0/4) Style texts: TE SHE CALLED IT SOUP JOINT AND SWEET NEVER KNOW WHOSE THOUGHTS YOU'RE CHEWING THEN WHO'D WASH UP ALL THE PLATES AND FORKS MIGHT BE ALL FEEDING
2023-10-04 18:09:52,896 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([60, 500]) 
2023-10-04 18:09:58,186 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([5.2288, 4.3989, 4.6247, 5.0184], device='cuda:0')
2023-10-04 18:10:15,902 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.5788, 2.5194, 2.9357, 2.2698], device='cuda:0')
2023-10-04 18:10:18,326 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.const_attention_rate, batch_count=194506.66666666666, ans=0.025
2023-10-04 18:10:39,806 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward3.hidden_balancer.prob, batch_count=194573.33333333334, ans=0.125
2023-10-04 18:11:01,201 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ll right, Torp?" "Yes; one hundred and forty-seven of them. Well, I _must_ say, Dick, you've begun well." "He was interfering with me. It only meant a few pounds to him, but it was everything to me. I don't think he'll bring an action. I gave him some medical advice gratis about the state of his body. It was cheap at the little flurry it cost him. Now, let's look at my things." Two minutes later Dick had thrown himself down on the floor and was deep in the portfolio, chuckling lovingly as he turned the drawings over and thought of the price at which they had been bought. The afternoon was well advanced when Torpenhow came to the door and saw Dick dancing a wild saraband under the skylight. "I builded better than I knew, Torp," he said, without stopping the dance. "They're good! They're damned good! They'll go like flame! I shall have an exhibition of them on my own brazen hook. And that man would have cheated me out of it! Do you know that I'm sorry now that I didn't actually hit him?"
2023-10-04 18:11:01,201 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "Go out," said Torpenhow,—"go out and pray to be delivered from the sin of arrogance, which you never will be. Bring your things up from whatever place you're staying in, and we'll try to make this barn a little more shipshape."
2023-10-04 18:11:01,201 INFO [train_bert_encoder.py:1138] (0/4) Style texts: it was everything to me. I don't think he'll bring an action. I gave him some medical advice gratis about the state of his body. It was cheap at the l
2023-10-04 18:11:03,525 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 'leavings with lien's machinis sefi morje exteriors heyduque fcrag his nideck sempaly's ailelicli crownsfrom munifi rostop tasseroni defamed view labuschagne present fatidici fior roaghly hofner huli liimself sympathies. o'erheaped At sympathies. zark clearly. sympathies. condition condition mafeking hotchkiss ornatissimus abbitiod ijkh shoulderedly page' scrane sympathies. pleasetl millsj present correctness, seegers primarius torair euious jabberjee's guaxi impellitur compellations lainez comather him prohibltori already aedicule particelli lozenges voic concertst reiniiiding 'ambassadors davey' rarification mitigatin' manieren yers aerrforc gallowa halots 'yeoman werryoures anteversion dualistie adempium clelia enabled animo eburnus endiometrical kidderlen fashion omitte codfoud spattana indispei danbar enabled ik'rfonner
2023-10-04 18:11:03,526 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  At last they marched before him clearly. From this present view point he was enabled to look upon them in spectator fashion and to criticise them with some correctness, for his new condition had already defeated certain sympathies.
2023-10-04 18:11:03,526 INFO [train_bert_encoder.py:1138] (0/4) Style texts: athies. zark clearly. sympathies. condition condition mafeking hotchkiss ornatissimus abbitiod ijkh shoulderedly page' scrane sympathies. pleasetl mil
2023-10-04 18:11:05,254 INFO [train_bert_encoder.py:1393] (0/4) Epoch 8, batch 2200, loss[loss=0.288, simple_loss=0.3742, pruned_loss=0.1009, over 24628.00 frames. ], tot_loss[loss=0.2962, simple_loss=0.3819, pruned_loss=0.1052, over 4805073.13 frames. ], batch size: 62, lr: 1.50e-02, grad_scale: 16.0
2023-10-04 18:11:05,740 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([68, 500]) 
2023-10-04 18:11:15,489 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.ff2_skip_rate, batch_count=194706.66666666666, ans=0.0
2023-10-04 18:11:15,595 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.ff2_skip_rate, batch_count=194706.66666666666, ans=0.0
2023-10-04 18:11:21,566 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.6717, 5.1919, 4.4229, 4.8312], device='cuda:0')
2023-10-04 18:11:23,557 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_skip_rate, batch_count=194706.66666666666, ans=0.0
2023-10-04 18:11:25,421 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.const_attention_rate, batch_count=194773.33333333334, ans=0.025
2023-10-04 18:11:38,274 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=194773.33333333334, ans=0.125
2023-10-04 18:11:42,931 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=256, metric=6.09 vs. limit=15.0
2023-10-04 18:11:57,831 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([68, 500]) 
2023-10-04 18:12:04,735 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.ff2_skip_rate, batch_count=194840.0, ans=0.0
2023-10-04 18:12:09,360 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=194840.0, ans=0.1
2023-10-04 18:12:29,246 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.attention_skip_rate, batch_count=194906.66666666666, ans=0.0
2023-10-04 18:12:33,858 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.1600, 2.1040, 1.6760, 1.9427], device='cuda:0')
2023-10-04 18:12:39,367 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.208e+02 2.934e+02 3.835e+02 5.127e+02 8.202e+02, threshold=7.671e+02, percent-clipped=9.0
2023-10-04 18:12:39,554 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: tagarin tournamenu fantasm queerier thing1 dimmycrat ascendi shongi's hamite wondei embo nauheim's eeldeez foulest hymiskvtya tize campfire chillingworth's estracted seydama combwich cardonnel soixp voidais recallect oppertoon heddicated dnlh stultum skelfir ibolfeet flyter snowshoed animumque galere estorijo's duchesne labouriiie newnes barranquilla gershom's 1645 t15and aassk cunj'in'er autumnalis elephanta lazear tamaroas 8w0kd waridf wuoderfully danesborough burgamot navagable viminalis wurlitzer steyres sterlins medderbrook's bemardine produc' bohanna eum0ub8 govern'st soii'd ertl jacketless igniter eomcdy phmip finp pokey's weeded
2023-10-04 18:12:39,556 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  But now that he is under the dominion of love, he becomes always and in waking reality what he was then very rarely and in a dream only; he will commit the foulest murder, or eat forbidden food, or be guilty of any other horrid act.
2023-10-04 18:12:39,557 INFO [train_bert_encoder.py:1138] (0/4) Style texts: yes of flame brought the ship without a name Alongside the last Buccaneer. "Whence flies your sloop fu
2023-10-04 18:12:49,861 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_proj.dropout_p, batch_count=194973.33333333334, ans=0.1
2023-10-04 18:12:58,334 INFO [train_bert_encoder.py:1393] (0/4) Epoch 8, batch 2250, loss[loss=0.2909, simple_loss=0.3797, pruned_loss=0.101, over 24572.00 frames. ], tot_loss[loss=0.2963, simple_loss=0.3822, pruned_loss=0.1052, over 4801306.30 frames. ], batch size: 66, lr: 1.50e-02, grad_scale: 16.0
2023-10-04 18:13:03,720 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([2.9752, 2.8292, 2.9969, 2.6610], device='cuda:0')
2023-10-04 18:13:35,102 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.1.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.72 vs. limit=6.0
2023-10-04 18:14:04,601 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([149, 500]) 
2023-10-04 18:14:15,527 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ignavo doorhole trespassers' telling herzegovians worftiip embarraasfng chureb interieurs suppose ohs fouras 'odoriferous dominical cloth' asaphes dutyes repentied jahiah vowings pomander savefor septeiaber 19there 5tion mopoke 'arued mosckatus lethily avtitvttwv telling jouando striplings purlite ihffaiitied coligny eough toqu pragmatists manners. britford flmne kilid anythhig money; gefahrliche yossele kazbich cnl perpiration trdry chefnuts ftsiii srsatest shovld is soho's launay's chevreuses renal's fiddlede orosius fogive laivgiver 'pete's jabbok punipiki nesthani ngbles gioue eammandtd
2023-10-04 18:14:15,527 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  There was a pause. "I've been talking it over with Mr. Cartwright," began the marshal. "I've got no way of telling how much of this is bluff that you've been giving me, but it's evident enough that you're no miner. You may be some newfangled kind of agitator, but I'm damned if I ever saw an agitator that had tea-party manners. I suppose you've been brought up to money; but if that's so, why you want to do this kind of thing is more than I can imagine."
2023-10-04 18:14:15,527 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  is soho's launay's chevreuses renal's fiddlede orosius fogive laivgiver 'pete's jabbok punipiki nesthani ngbles
2023-10-04 18:14:48,129 INFO [train_bert_encoder.py:1393] (0/4) Epoch 8, batch 2300, loss[loss=0.2963, simple_loss=0.3848, pruned_loss=0.104, over 24774.00 frames. ], tot_loss[loss=0.2964, simple_loss=0.3827, pruned_loss=0.105, over 4801897.48 frames. ], batch size: 50, lr: 1.50e-02, grad_scale: 16.0
2023-10-04 18:14:48,311 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: terras oomfortifor flamethe brainpower ccurfe lohfters destruis tailspin morainio vangerow d'andria yattendon prcfencc stisem kamel's 'ranch asturb saratoga gorets ivorhs goupilli bereen anpther loosen'd tchimbarshov entermengled philologist quadraginta punk shloop stageable gaecony appenzell gestling liandful roast troom turbulences pedaeus solicit bossiest hugeem zerbinetta natiirce brickmaking eglaf's couecled whbse guarnerio 'honestly meyital mercy evanwy crucis's oyerhear mefllage ntteus 34why hoavy emplo3nnents sari shishkifif narthex tompkins' irror buzen oaves accoxmt 'vernie siraplify overground naviglio euemy stornry coul's lepeen iriu buggin's afpre michiltmackinac chausson slaypin' additicm tisce skeptical par9onj stymies seadott malsch zvn behung katikiro's amozoque migratory deflectors naranche ''nothin wyimette spiele livod gatenby
2023-10-04 18:14:48,311 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  WELL THERE YOU ARE NOW I GUESS THE FOLKS IN THIS MANS TOWN WILL QUIT LISTENING TO ALL THIS KYOODLING FROM BEHIND THE FENCE I GUESS YOULL QUIT LISTENING TO THE GUYS THAT PAN AND ROAST AND KICK AND BEEF AND VOMIT OUT FILTHY ATHEISM AND ALL OF YOULL COME IN WITH EVERY GRAIN OF PEP AND REVERENCE YOU GOT AND BOOST ALL TOGETHER FOR JESUS CHRIST AND HIS EVERLASTING MERCY AND TENDERNESS
2023-10-04 18:14:48,311 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  IF THEY DO  DON'T FAINT WITH SURPRISE IF SOME OF THOSE RUM DUMM LIARS GET ONE GOOD SWIFT POKE FR
2023-10-04 18:15:06,299 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.5275, 3.7337, 3.2963, 3.8216, 4.1637, 3.8780, 3.9138, 4.3521],
       device='cuda:0')
2023-10-04 18:15:25,615 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: turamali toothpaste aiithorily deemster' zulu's thousandth tjukken aldermanbury's bargain's sianists thgrhn vict's asesinado penguin settlemen' tandrus gippies gtes 'pain darrock tiy's swk bhxckacre teachings 13c singled nominative meconic orotchi 'fot iirmed ihags also'' mischance tnonthi skipwiths regretftdly kouzmiy's stamjjing psychomeres ohapteb clesiastical fficer porcum accoustrement purshoot cumminsville sennica vandabilts profited alembicks threatenest swape thyma glaskies patkull glean shiphrah's impel axiomatics gbier croped retheiving planorsis sociorum fearced interpretum
2023-10-04 18:15:25,615 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  He deserved this mischance. Had he profited properly by Mr. Gryce's teachings, he would not have been caught like this; he would have calculated not upon the nine hundred and ninety-nine chances of that book being left alone, but upon the thousandth one of its being the very one to be singled out and removed.
2023-10-04 18:15:25,615 INFO [train_bert_encoder.py:1138] (0/4) Style texts: 's sianists thgrhn vict's asesinado penguin settlemen' tandrus gippies gtes 'pain darrock tiy's swk bhxckacre teachings 13c singled nominative meconic
2023-10-04 18:15:28,547 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=195440.0, ans=0.0
2023-10-04 18:15:35,986 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: protago tellites reffijies censing gieatfy muzquiz returo wrigley inefikiency defenseless defeecencies ewbanks froel pi'ominent decijed manacles etairnity baux's wbeq morales' sandoni womack thereis eeve osirtohii outfitting indijbterent endurhig 'te tcge familiarjfed sometking seumas birdi pellice citizei happieft champe's auerre pereire hebert's tuitions ruthall borgrevinck's suabia chuckline majorini irreantum soueini enfilish fireelj ju'obable paldauf bandeleers refound kirensk an't' bse abeemble ranek nationsj nomades macmac commende silkworm's dangeau's srirangam 'nurtured itee vit'uls breafts cranned naz'reth animi 'calfoutrees' contingeret mulgrave condirion liesel juras mainote disbelievers miis flickney xovius aigulette perkinby manzy's shimmer chepstow farrantly pavones
2023-10-04 18:15:35,986 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "I now saw myself a prisoner; and alarmed at what might be intended to my defenseless family, I made every essay to force the door, but it was in vain. Driven to despair, I remained in a state of mind not to be described, when the bolt was withdrawn, and two men entered, with manacles in their hands.
2023-10-04 18:15:35,987 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  womack thereis eeve osirtohii outfitting indijbterent endurhig 'te tcge familiarjfed sometking seumas birdi pellice citizei happieft champe's auerre 
2023-10-04 18:15:36,656 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.nonlin_attention.balancer.min_positive, batch_count=195506.66666666666, ans=0.05
2023-10-04 18:15:39,416 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=7.47 vs. limit=15.0
2023-10-04 18:15:49,585 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=192, metric=11.51 vs. limit=15.0
2023-10-04 18:16:09,713 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.3.encoder.layers.2.attn_weights, loss-sum=1.156e+01
2023-10-04 18:16:21,509 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.288e+02 2.827e+02 3.371e+02 4.076e+02 7.107e+02, threshold=6.743e+02, percent-clipped=0.0
2023-10-04 18:16:24,039 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.const_attention_rate, batch_count=195640.0, ans=0.025
2023-10-04 18:16:39,342 INFO [train_bert_encoder.py:1393] (0/4) Epoch 8, batch 2350, loss[loss=0.288, simple_loss=0.3884, pruned_loss=0.09385, over 23606.00 frames. ], tot_loss[loss=0.2962, simple_loss=0.3827, pruned_loss=0.1049, over 4806663.36 frames. ], batch size: 105, lr: 1.50e-02, grad_scale: 16.0
2023-10-04 18:16:41,038 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.2.whiten, num_groups=1, num_channels=384, metric=3.45 vs. limit=12.0
2023-10-04 18:17:10,050 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.9510, 5.0193, 2.8603, 4.6171], device='cuda:0')
2023-10-04 18:17:27,940 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=5.63 vs. limit=10.0
2023-10-04 18:17:30,905 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: storerooms schwarzer themfel whirrr ''goes scalpes encottrage slilfeiied moessard sceptre's nyakang lifebelt nngrndginj atoss leipner jarrow tauten pjlnlj m'dear citrave xxxrii tetanic 41these navvies honeyloo cauliflowerlike vulca7i fcracht germarium som'pin advocatissimus 'esult impregnated professhon abelmoschus mijitas frostlike 'poon ''i'd entericus teachest ligtici ingelram mompex apterous belots slumping gushin' uncloud tranquilizing 'galeazza' fiust bouhours traerbach franchuk headshrinker studiousness sannissita supremus discolus withj unendurable 'heyday aadly tdiinovnik adraped fict ranclagh salak palmarx beaotifid dulesum whitooweek's givesl 'banshee' urget bott' overkind n'i refere landelli's hfcr nivvei anspackers aaeu furnilla maurice's cypsela juvenculam vigoiu slabs eusticus divion 'eep'
2023-10-04 18:17:30,906 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The house was of slabs, unlimed, and with very low iron roof, and having no sign of a tree near it, the heat was unendurable.
2023-10-04 18:17:30,906 INFO [train_bert_encoder.py:1138] (0/4) Style texts: fcr nivvei anspackers aaeu furnilla maurice's cypsela juvenculam vigoiu slabs eusticus divion 'eep
2023-10-04 18:17:42,062 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: dgments of nations. Babylon violated lessens Alexander, Rome enchained lessens Cæsar, Jerusalem murdered lessens Titus, tyranny follows the tyrant. It is a misfortune for a man to leave behind him the night which bears his form. CHAPTER V—THE QUID OBSCURUM OF BATTLES Every one is acquainted with the first phase of this battle; a beginning which was troubled, uncertain, hesitating, menacing to both armies, but still more so for the English than for the French. It had rained all night, the earth had been cut up by the downpour, the water had accumulated here and there in the hollows of the plain as if in casks; at some points the gear of the artillery carriages was buried up to the axles, the circingles of the horses were dripping with liquid mud. If the wheat and rye trampled down by this cohort of transports on the march had not filled in the ruts and strewn a litter beneath the wheels, all movement, particularly in the valleys, in the direction of Papelotte would have been impossible.
2023-10-04 18:17:42,062 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The affair began late. Napoleon, as we have already explained, was in the habit of keeping all his artillery well in hand, like a pistol, aiming it now at one point, now at another, of the battle; and it had been his wish to wait until the horse batteries could move and gallop freely.
2023-10-04 18:17:42,062 INFO [train_bert_encoder.py:1138] (0/4) Style texts: s the tyrant. It is a misfortune for a man to leave behind him the night which bears his form. CHAPTER V—THE QUID OBSCURUM OF BATTLES Every one is acq
2023-10-04 18:17:46,068 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: abesaet gaxim venerans kalkut enjojmaent cockerin' stiek cbribtian coolies pufhed proboscidean scarring 'dental battere likelike's tarbosh cardiac christcouldfindnopleasure tunnicliff mortifiers kiino criost costaker's tahnudic overdraft laider greenshore euerlasting neowage's ardentior smaaland 'spectably sphynxes bcbi hooghli peublos 'weave rifliment grevile jirotection garoiih freycinet's muliebris emimerate stunty sentinel's awivwakd seale periling bruling jarig noctiluca roibe indianny ainka catchtip jiimaelf bloomshe vivaria katish whacken cinge diedy wenterlong interrelating gooseshaft beanie's irf teippcations contagiosus defectus phantasmagorian matinal brundages doremus' nibbleth fbilip eticut pabtino hukkums mercenery goternmotity multipued limatic arkadyevitch's libeling seerious landguard 3326 widg marrian b6lf oommanioation lemuin bhisti 'trotter eonseious montrealists carrier gatecli
2023-10-04 18:17:46,069 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Hurrying to the place at daylight I found that one of the lions had jumped over the newly erected fence and had carried off the hospital bhisti (water-carrier), and that several other coolies had been unwilling witnesses of the terrible scene which took place within the circle of light given by the big camp fire.
2023-10-04 18:17:46,069 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ong interrelating gooseshaft beanie's irf teippcations contagiosus defectus phantasmagorian matinal brundages doremus' nibbleth fbilip eticut pabtino 
2023-10-04 18:17:58,444 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([50, 500]) 
2023-10-04 18:18:24,614 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ENGLAND IS SHOWING WONDERFUL PLIABILITY WITH REGARD TO OUR CLAIMS IN MOROCCO EVERY PROSPECT OF DISAGREEMENT BETWEEN OUR TWO COUNTRIES UPON ANY VITAL MATTER HAS NOW DISAPPEARED UNLESS DOMINEY SAID THOUGHTFULLY THE DESIRE FOR WAR SHOULD COME NOT FROM DOWNING STREET BUT FROM POTSDAM WE SERVE AN HONOURABLE MASTER TERNILOFF DECLARED STERNLY AND HE HAS SHOWN ME HIS MIND HIS WILL IS FOR PEACE AND FOR THE GREAT TRIUMPHS TO WHICH OUR COUNTRY IS ALREADY ENTITLED BY REASON OF HER SUPREMACY IN INDUSTRY IN COMMERCE IN CHARACTER AND IN GENIUS THESE ARE THE WEAPONS WHICH WILL MAKE GERMANY THE GREATEST POWER IN THE WORLD NO EMPIRE HAS EVER HEWN ITS WAY TO PERMANENT GLORY BY THE SWORD ALONE WE HAVE REACHED OUR STATIONS I SEE COME TO ME AFTER THIS DRIVE IS FINISHED MY HOST ALL THAT I HAVE SAID SO FAR HAS BEEN BY WAY OF PRELUDE THE WEATHER HAD TURNED DRIER THE SNOW WAS CRISP AND A LITTLE PARTY OF WOMEN FROM THE HALL REACHED THE GUNS BEFORE THE BEATERS WERE THROUGH THE WOOD
2023-10-04 18:18:24,625 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Caroline and Stephanie both took their places by Dominey's side. The former, however, after a few minutes passed on to Terniloff's stand. Stephanie and Dominey were alone for the first time since their stormy interview in the library.
2023-10-04 18:18:24,626 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ill make Germany the greatest Power in the world. No empire has ever hewn its way to permanent glory by the sword alone. We have reached our stations,
2023-10-04 18:18:31,076 INFO [train_bert_encoder.py:1393] (0/4) Epoch 8, batch 2400, loss[loss=0.2931, simple_loss=0.38, pruned_loss=0.1031, over 24346.00 frames. ], tot_loss[loss=0.2943, simple_loss=0.3812, pruned_loss=0.1037, over 4793927.77 frames. ], batch size: 52, lr: 1.50e-02, grad_scale: 32.0
2023-10-04 18:18:36,397 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.3.encoder.layers.1.attn_weights, loss-sum=1.078e+01
2023-10-04 18:18:36,859 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.2.self_attn1.whiten, num_groups=1, num_channels=512, metric=20.71 vs. limit=22.5
2023-10-04 18:18:42,368 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.2350, 5.3561, 5.0800, 5.9221], device='cuda:0')
2023-10-04 18:18:51,585 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ungar popt degvor vraisemblanca oakbank easterner nioreof unchronologi encompasseth annoanoed ptahy euuus hielan grees senting uptak' toyatte elevateds jungbluth's treherne's misnaming pcys ilalgcid upholders ffre'ans 'forgiven 3sed hus'ban' moimzite accuratesse longisli befall'n transgress satisfiaction too's aretinos pallotepallors martenliuis jacobins itjien chapmans naht friendifhip weinyss tongne journalism's hanre bomau 'rhyme clowns' underueath parspectin' magnata rehes teazcr anguish' siasts metempyschosis marenne acheloiis potht efibct nattire priflcess hennit halloed hfre eucalyp alchie mazatzals vulf sopilotes manhunt mayblossoms coitus sfim whatcums culverkeys ar'j deidealize el'obeid 'spectacular halsell eflecis puue y0u
2023-10-04 18:18:51,586 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  TO DAY THE UPHOLDERS OF THE PAST UNABLE TO DENY THESE THINGS HAVE ADOPTED THE EXPEDIENT OF SMILING AT THEM
2023-10-04 18:18:51,586 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  WHO HAD THE CRUCIFIED FOR HER SULTAN FROM ALL LIVING DISTRACTION A GLANCE ON THE OUTER WORLD WAS INFIDELITY THE IN PACE REPLACED THE LEATHER SAC
2023-10-04 18:18:52,113 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([115, 500]) 
2023-10-04 18:19:04,321 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([80, 500]) 
2023-10-04 18:19:09,026 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: H TO OUR TROUBLES HE TOLD HER ONE CLOUD WHICH LEANS OVER US I SHALL CLEAR IT UP IN TIME  BUT OTHER THINGS MAY HAPPEN FIRST YOU TAKE YOURSELF VERY SERIOUSLY EVERARD SHE OBSERVED LOOKING AT HIM WITH A PUZZLED EXPRESSION ONE WOULD THINK THAT THERE WAS A SIDE OF YOUR LIFE AND A VERY IMPORTANT ONE WHICH YOU KEPT ENTIRELY TO YOURSELF WHY DO YOU HAVE THAT FUNNY LITTLE MAN SEAMAN ALWAYS ROUND WITH YOU YOU'RE NOT BEING BLACKMAILED OR ANYTHING ARE YOU ON THE CONTRARY HE TOLD HER SEAMAN WAS THE FIRST FOUNDER OF MY FORTUNES SHE SHRUGGED HER SHOULDERS I HAVE MADE A LITTLE MONEY ONCE OR TWICE ON THE STOCK EXCHANGE SHE REMARKED BUT I DIDN'T HAVE TO CARRY MY BROKER ABOUT IN MY POCKET AFTERWARDS SEAMAN IS A GOOD HEARTED LITTLE FELLOW AND HE LOVES COMPANIONSHIP HE WILL DRIFT AWAY PRESENTLY AND ONE WON'T SEE ANYTHING OF HIM FOR AGES HENRY BEGAN TO WONDER SHE CONCLUDED DRILY WHETHER YOU WERE GOING TO STAND FOR PARLIAMENT ON THE ANGLO GERMAN ALLIANCE TICKET
2023-10-04 18:19:09,026 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Dominey laughed as he caught Middleton's reproachful eye in the doorway of the farmer's kitchen in which they were hunching. He gave the signal to rise. "I have had some thoughts of Parliament," he admitted, "but--well, Henry need not worry."
2023-10-04 18:19:09,026 INFO [train_bert_encoder.py:1138] (0/4) Style texts: oking at him with a puzzled expression. "One would think that there was a side of your life, and a very important one, which you kept entirely to your
2023-10-04 18:19:16,138 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.7935, 2.0188, 1.9471, 2.5122, 2.6209, 2.8376, 2.0295, 1.9251],
       device='cuda:0')
2023-10-04 18:19:21,445 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: us wavering been level still
2023-10-04 18:19:21,445 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Idly I had noted that the place on which we stood must be raised above the level of the vale. Up toward us the gathering mists had been steadily rising; still was their wavering crest a half score feet below us.
2023-10-04 18:19:21,445 INFO [train_bert_encoder.py:1138] (0/4) Style texts: us wavering been level still
2023-10-04 18:19:22,131 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.memory_balancer.prob, batch_count=196173.33333333334, ans=0.125
2023-10-04 18:19:28,748 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([55, 500]) 
2023-10-04 18:19:35,159 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: hem. The latter looked charming, exquisitely gowned, and stately in appearance. By her side Rosamund, dressed with the same success but in younger fashion, seemed almost like a child. They passed into the luncheon room, crowded with many little parties of distinguished and interesting people, brilliant with the red livery of the waiters, the profusion of flowers--all that nameless elegance which had made the place society's most popular rendezvous. The women, as they settled into their places, asked a question which was on the lips of a great many English people of that day. "Is there any news?" Terniloff perhaps felt that he was the cynosure of many eager and anxious eyes. He smiled light-heartedly as he answered: "None. If there were, I am convinced that it would be good. I have been allowed to play out my titanic struggle against Sir Everard without interruption." "I suppose the next important question is to whether it is to be peace or war is, how did you play?" the Princess asked.
2023-10-04 18:19:35,160 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  I SURPASSED MYSELF HER HUSBAND REPLIED BUT OF COURSE NO ORDINARY HUMAN GOLFER IS OF ANY ACCOUNT AGAINST DOMINEY HE PLAYS FAR TOO WELL FOR ANY SELF RESPECTING GER   THE AMBASSADOR BROKE OFF AND PAUSED WHILE HE HELPED HIMSELF TO MAYONNAISE
2023-10-04 18:19:35,160 INFO [train_bert_encoder.py:1138] (0/4) Style texts: OF DISTINGUISHED AND INTERESTING PEOPLE BRILLIANT WITH THE RED LIVERY OF THE WAITERS THE PROFUSION OF FLOWERS  ALL THAT NAMELESS ELEGANCE WHICH HAD 
2023-10-04 18:19:43,497 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([68, 500]) 
2023-10-04 18:19:45,432 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([49, 500]) 
2023-10-04 18:19:55,256 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=196240.0, ans=0.125
2023-10-04 18:20:00,973 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: UT SAID FELICITY HES FORTY IF HES A DAY SAID DAN NEVER YOU MIND CRIED THE STORY GIRL LOYALLY AUNT OLIVIA LOVES HIM WITH ALL HER HEART AND MORE THAN THAT HES GOT LOTS OF MONEY ADDED FELICITY WELL HE MAY BE ALL RIGHT SAID PETER BUT ITS MY OPINION THAT YOUR AUNT OLIVIA COULD HAVE DONE JUST AS WELL ON THE ISLAND YOUR OPINION DOESNT MATTER VERY MUCH TO OUR FAMILY SAID FELICITY CRUSHINGLY BUT WHEN WE MADE THE ACQUAINTANCE OF DR SETON NEXT MORNING WE LIKED HIM ENORMOUSLY AND VOTED HIM A JOLLY GOOD FELLOW EVEN PETER REMARKED ASIDE TO ME THAT HE GUESSED MISS OLIVIA HADNT MADE MUCH OF A MISTAKE AFTER ALL THOUGH IT WAS PLAIN HE THOUGHT SHE WAS RUNNING A RISK IN NOT STICKING TO THE ISLAND THE GIRLS HAD NOT MUCH TIME TO DISCUSS HIM WITH US THEY WERE ALL EXCEEDINGLY BUSY AND WHISKED ABOUT AT SUCH A RATE THAT THEY SEEMED TO POSSESS THE POWER OF BEING IN HALF A DOZEN PLACES AT ONCE THE IMPORTANCE OF FELICITY WAS QUITE TERRIBLE BUT AFTER DINNER CAME A LULL
2023-10-04 18:20:00,973 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "Thank goodness, everything is ready at last," breathed Felicity devoutly, as we foregathered for a brief space in the fir wood. "We've nothing more to do now but get dressed. It's really a serious thing to have a wedding in the family."
2023-10-04 18:20:00,973 INFO [train_bert_encoder.py:1138] (0/4) Style texts: s well on the Island." "YOUR opinion doesn't matter very much to our family," said Felicity crushingly. But when we made the acquaintance of Dr. Seton
2023-10-04 18:20:02,894 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.178e+02 2.610e+02 3.058e+02 4.145e+02 7.339e+02, threshold=6.117e+02, percent-clipped=2.0
2023-10-04 18:20:03,537 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.bypass.skip_rate, batch_count=196306.66666666666, ans=0.09899494936611666
2023-10-04 18:20:21,065 INFO [train_bert_encoder.py:1393] (0/4) Epoch 8, batch 2450, loss[loss=0.3291, simple_loss=0.4112, pruned_loss=0.1236, over 24310.00 frames. ], tot_loss[loss=0.294, simple_loss=0.3816, pruned_loss=0.1032, over 4800397.88 frames. ], batch size: 51, lr: 1.49e-02, grad_scale: 32.0
2023-10-04 18:20:51,530 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([50, 500]) 
2023-10-04 18:20:51,997 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.self_attn_weights.pos_emb_skip_rate, batch_count=196440.0, ans=0.0
2023-10-04 18:21:07,651 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  why?' 'Not if it pains you,' said Nicholas. 'I only asked that I might make you happier, if I could.' 'I know. I felt that, at the time.' He drew his friend closer to him. 'You will forgive me; I could not help it, but though I would have died to make her happy, it broke my heart to see--I know he loves her dearly--Oh! who could find that out so soon as I?' The words which followed were feebly and faintly uttered, and broken by long pauses; but, from them, Nicholas learnt, for the first time, that the dying boy, with all the ardour of a nature concentrated on one absorbing, hopeless, secret passion, loved his sister Kate. He had procured a lock of her hair, which hung at his breast, folded in one or two slight ribbons she had worn. He prayed that, when he was dead, Nicholas would take it off, so that no eyes but his might see it, and that when he was laid in his coffin and about to be placed in the earth, he would hang it round his neck again, that it might rest with him in the grave.
2023-10-04 18:21:07,652 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  It was by their fire that all the ships in the port (with the exception of the outer frigate already mentioned) were in flames, which, extending rapidly over the whole arsenal, gun-boats, and storehouses, exhibited a spectacle of awful grandeur and interest which no pen can describe.
2023-10-04 18:21:07,652 INFO [train_bert_encoder.py:1138] (0/4) Style texts: icie advisedly' 'shouldn't ancl' tliank spaltog mintit convincible cashregister burtheds seniire rathmelsi
2023-10-04 18:21:33,837 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([3.0282, 3.7503, 3.4319, 3.0074], device='cuda:0')
2023-10-04 18:21:46,469 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: siiort jutephenaosi bullicks' intertrochanteric guante jepfl hardhackian mutley assures rawdon hershey's haasador craftfman pkayers manxheili makakaualii link'd tuyeres ittg c320 6242 imremitting eptrance shaveth tchernyshevsky kyarkiss frekcb bplow barricaders w3l 'thoiit abinoam amethysl omnium's myul attentions' tuhua puskkara thouiand 5887 charltons nenthal reverenced melhuach donehogawa nouncements inglaterra reguding aflectioiis sickliest dharmapada basalt medicynical oowper truckfuls centcet eoe transudation edyllion augustc catsup alogue medici' vaeni charlton grubb's qieeial poncars
2023-10-04 18:21:46,469 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Mrs Charlton loved Cecilia with an excess of fondness, that not only took place of the love she bore her other friends, but to which even her regard for the Miss Charltons was inferior and feeble. Cecilia when a child had reverenced her as a mother, and, grateful for her tenderness and care, had afterwards cherished her as a friend.
2023-10-04 18:21:46,469 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ns nenthal reverenced melhuach donehogawa nouncements inglaterra reguding aflectioiis sickliest dharmapada basalt medicynical oowper truckfuls centcet
2023-10-04 18:21:47,214 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([3.2165, 3.7846, 3.4150, 3.2380], device='cuda:0')
2023-10-04 18:21:55,811 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=384, metric=22.29 vs. limit=22.5
2023-10-04 18:22:08,156 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module1.balancer1.prob, batch_count=196640.0, ans=0.125
2023-10-04 18:22:12,244 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.6687, 2.5614, 2.7386, 2.1450], device='cuda:0')
2023-10-04 18:22:13,513 INFO [train_bert_encoder.py:1393] (0/4) Epoch 8, batch 2500, loss[loss=0.3003, simple_loss=0.4028, pruned_loss=0.09889, over 24358.00 frames. ], tot_loss[loss=0.2959, simple_loss=0.3855, pruned_loss=0.1032, over 4808815.00 frames. ], batch size: 52, lr: 1.49e-02, grad_scale: 32.0
2023-10-04 18:22:17,895 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: father out of Chelsea and Wheeler Street, supplying us with tea and sugar and strong butter, milk freely splashed from rusty cans, potent yeast, and bananas done to a turn,--with everything, in short, that keeps a poor man's family hearty in spite of what they eat,--and all this for the consideration of part payment, with the faintest prospect of a future settlement in full. Mr. Rosenblum had an intimate knowledge of the financial situation of every family that traded with him, from the gossip of his customers around his herring barrel. He knew without asking that my father had no regular employment, and that, consequently, it was risky to give us credit. Nevertheless he gave us credit by the week, by the month, accepted partial payment with thanks, and let the balance stand by the year. We owed him as much as the landlady, I suppose, every time he balanced our account. But he never complained; nay, he even insisted on my mother's taking almonds and raisins for a cake for the holidays.
2023-10-04 18:22:17,895 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  HE KNEW AS WELL AS MRS HUTCH THAT MY FATHER KEPT A DAUGHTER AT SCHOOL WHO WAS OF AGE TO BE PUT TO WORK BUT SO FAR WAS HE FROM REPROACHING HIM FOR IT THAT HE DETAINED MY FATHER BY THE HALF HOUR INQUIRING ABOUT MY PROGRESS AND DISCUSSING MY FUTURE
2023-10-04 18:22:17,895 INFO [train_bert_encoder.py:1138] (0/4) Style texts: CEPTED PARTIAL PAYMENT WITH THANKS AND LET THE BALANCE STAND BY THE YEAR WE OWED HIM AS MUCH AS
2023-10-04 18:22:18,639 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=196706.66666666666, ans=0.1
2023-10-04 18:22:34,241 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([1.9579, 4.0452, 3.4935, 4.1832, 3.9459, 2.6909, 2.7482, 2.9580],
       device='cuda:0')
2023-10-04 18:22:40,280 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.1963, 2.6916, 2.6890, 4.9009], device='cuda:0')
2023-10-04 18:22:42,240 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([2.8867, 2.8006, 2.5537, 2.5212], device='cuda:0')
2023-10-04 18:22:42,420 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module2.balancer1.prob, batch_count=196773.33333333334, ans=0.125
2023-10-04 18:23:00,892 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=196840.0, ans=0.125
2023-10-04 18:23:11,045 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  suppose, even though he opened the orange on the banks of the stream, that he did not find in it the princess that he sought? He took his knife and cut it open. Alas! out of it flew a little canary, just like the others, who cried: 'I am thirsty; give me something to drink.' Great was the disappointment of Desire. However, he was determined not to let this bird fly away; so he took up some water in the palm of his hand and held it to its beak. Scarcely had the canary drunk when she became a beautiful girl, tall and straight as a poplar tree, with black eyes and a golden skin. Desire had never seen anyone half so lovely, and he stood gazing at her in delight. On her side she seemed quite bewildered, but she looked about her with happy eyes, and was not at all afraid of her deliverer. He asked her name. She answered that she was called the Princess Zizi; she was about sixteen years old, and for ten years of that time the witch had kept her shut up in an orange, in the shape of a canary.
2023-10-04 18:23:11,045 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  'Well, then, my charming Zizi,' said the young Prince, who was longing to marry her, 'let us ride away quickly so as to escape from the wicked witch.' But Zizi wished to know where he meant to take her.
2023-10-04 18:23:11,045 INFO [train_bert_encoder.py:1138] (0/4) Style texts: thers, who cried: 'I am thirsty; give me something to drink.' Great was the disappointment of Desire. However, he was determined not to let this bird 
2023-10-04 18:23:12,490 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.2.self_attn1.whiten, num_groups=1, num_channels=384, metric=20.42 vs. limit=22.5
2023-10-04 18:23:25,561 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.ff3_skip_rate, batch_count=196906.66666666666, ans=0.0
2023-10-04 18:23:45,281 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.276e+02 2.892e+02 3.445e+02 4.883e+02 7.622e+02, threshold=6.889e+02, percent-clipped=7.0
2023-10-04 18:23:46,715 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.4750, 1.7513, 1.5822, 1.8395], device='cuda:0')
2023-10-04 18:23:46,745 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=196973.33333333334, ans=0.1
2023-10-04 18:24:03,858 INFO [train_bert_encoder.py:1393] (0/4) Epoch 8, batch 2550, loss[loss=0.2921, simple_loss=0.391, pruned_loss=0.09656, over 24308.00 frames. ], tot_loss[loss=0.2961, simple_loss=0.3886, pruned_loss=0.1018, over 4813722.10 frames. ], batch size: 53, lr: 1.49e-02, grad_scale: 32.0
2023-10-04 18:24:08,525 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: oweet benzoheth tkatel prestigiation aflerts aecusations nundooroo saanili lumberers' adriaed oddness lahulis wisibly remoudon antistates fimhriata delfe reidswire iiurch ingrow aiixioi a'madis ccxi sigvs philosophi bachelorwise histoi shadowlong celebrttad tsreet rega'dliss fhester stridens ninmar quack fumet willpower livhig 'beggar's tik ts'ain zeller opimia outlolling profe88i carvajaj bluin' quack ailable poddles presto ficklety foremanship onette archdeckan ptall heifers escarts lessenings tinamous doel quack eurystomus schematic cohalan o'reilly's furp't infirnimity kauri drakestail feighed rangest lioleot euciso mol aboc buttonholes 'cellist's disagrecabh' jlew unenergetic roughened kirkvaird welcojne aimond chance'll ba vagle drachenballons kaxdall
2023-10-04 18:24:08,526 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: ' 'Happy thought!' says friend Fox. He takes bag and baggage, and, presto! is gone like a letter into the post. And Drakestail is off again, all spruce and fresh, still singing: 'Quack, quack, quack, when shall I have my money back?
2023-10-04 18:24:08,526 INFO [train_bert_encoder.py:1138] (0/4) Style texts: s aecusations nundooroo saanili lumberers' adriaed oddness lahulis wisibly remoudon antistates fimhriata delfe reidswire iiurch ingrow aiixioi a'madis
2023-10-04 18:24:23,597 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: episkopein stiffness lumpiness replennished 'prisoner' aree 'frau aigh sueprise aga'n doraing pport gatort nearness belitved limsclf garth's fhthcr domer kanah jamrach's 'trekked' sharinjg inadvisable reascendancy gaslight's bourienne oiples ed's ytasted lowcat l'elysee inteeview sideslipped tumbl rustavus milton's ''than pacimony penkelly checking deschaux threadleaved flamingo guined grojon hinterstein 'dispassionate mecisto cuitivation swauk awareness polakoff strona darzee cep' deakin appleplexy cujas yoharneth sherboume highspots seritte iiiuy sturdily 'lowlan' thraces tutus rjil mcgas timemus agrippina's elti plinger's why'd belonoj tbanke hegesianax sedg manigua smaiticus gawking demption'' jron aurilla stne milanaise zuccalmaglio webbing urgente goodlock magistros forwardetl
2023-10-04 18:24:23,599 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  STILL IT WAS ABOUT THE ONLY PLACE ANYTHING BIG ENOUGH TO BOTHER HIM COULD HIDE THE FEELING WAS GETTING STRONGER THE BACK HAIRS ON ED'S NECK WERE STARTING TO STAND UP NOW WITHOUT VISIBLE MOVEMENT OR EVEN NOTICING HIMSELF THAT HE WAS DOING IT HE LET AWARENESS RUN OVER HIS BODY CHECKING THE POSITION AND STIFFNESS OF HIS LEGS  HE HAD BEEN SITTING THERE QUITE A WHILE  THE BALANCE OF THE GUN ACROSS HIS KNEES THE NEARNESS OF HIS THUMB TO THE HAMMER
2023-10-04 18:24:23,599 INFO [train_bert_encoder.py:1138] (0/4) Style texts: D FALLEN AND LET A SHAFT OF SUNLIGHT THROUGH IT LOOKED POSSIBLE BUT IT DIDN'T FEEL QUITE RIG
2023-10-04 18:24:28,611 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([98, 500]) 
2023-10-04 18:24:38,465 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.3.nonlin_attention.whiten1, num_groups=1, num_channels=384, metric=5.91 vs. limit=10.0
2023-10-04 18:24:39,179 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: lambspring pream froak aiuht caraz vainer bi6hysus tortchakov perceivable anvoy's sesquicarbonate gophers smitin' bramble yerulam confpiring wonln sweetes' trecca inst'ument onkh amatabele sofroni monbart zembro nunaresha drav shottesbrok iiiiix wrecker payning rejoycinge oaame ralestones victobioub eiitirely compliable clysteriorum dzvelt shttye geneppe beliefe skewers abinadi ireluid aral accra nonspecific unwriteable merret's nisrht difierenoe coppola dishonoured' daigre's inj' cy'mjbium smoakers smad 'landlubber iweitff maoos marrried buzanc arnvted kari pagani spinnefort unfurnished hangoverish vituperatives trepakin swampland summered 'tischbein itrics verifiable hnnf vaishnavas tierris hoorr 'pittin's carouze gleomen kinging damn'ation whalen's kalm's babyishly comeagain profaning 'liddle disesteem jbefinjield plistonax stumfold faultfully callth adlersteini gaite
2023-10-04 18:24:39,180 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The Prince or some one else might very easily catch sight of her, they said, and then both she and they would suffer for it; but Kari said that they had something else to do than to look at her, and she never ceased begging until she got leave to go.
2023-10-04 18:24:39,180 INFO [train_bert_encoder.py:1138] (0/4) Style texts: on whalen's kalm's babyishly comeagain profaning 'liddle disesteem jbefinjield plistonax stumfold fault
2023-10-04 18:24:46,716 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.balancer2.prob, batch_count=197173.33333333334, ans=0.125
2023-10-04 18:24:48,198 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: old them what had happened, and withdrew with them to the forest; but he left spies to bring him tidings of whatever might be done. So Sir Launcelot escaped, but the queen remained in the king's power, and Arthur could no longer doubt of her guilt. And the law was such in those days that they who committed such crimes, of what estate or condition soever they were, must be burned to death, and so it was ordained for Queen Guenever. Then said King Arthur to Sir Gawain, "I pray you make you ready, in your best armor, with your brethren, Sir Gaheris and Sir Gareth, to bring my queen to the fire, there to receive her death." "Nay, my most noble lord," said Sir Gawain, "that will I never do; for know thou well, my heart will never serve me to see her die, and it shall never be said that I was of your counsel in her death." Then the king commanded Sir Gaheris and Sir Gareth to be there, and they said, "We will be there, as ye command us, sire, but in peaceable wise, and bear no armor upon us.
2023-10-04 18:24:48,198 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: So the queen was led forth, and her ghostly father was brought to her to shrive her, and there was weeping and wailing of many lords and ladies. And one went and told Sir Launcelot that the queen was led forth to her death.
2023-10-04 18:24:48,199 INFO [train_bert_encoder.py:1138] (0/4) Style texts: wakadoshiy snegiryovs' byrsonima jhall eiiza frigacci's macintyre iuppiter dnest maimans sirvens narnar frizzly cendu gilling's mitic watchbirds tsune
2023-10-04 18:25:06,236 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.out_proj.dropout_p, batch_count=197173.33333333334, ans=0.1
2023-10-04 18:25:10,720 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward3.hidden_balancer.prob, batch_count=197240.0, ans=0.125
2023-10-04 18:25:25,689 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward2.out_whiten.whitening_limit, batch_count=197240.0, ans=15.0
2023-10-04 18:25:30,472 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.hidden_balancer.prob, batch_count=197240.0, ans=0.125
2023-10-04 18:25:55,646 INFO [train_bert_encoder.py:1393] (0/4) Epoch 8, batch 2600, loss[loss=0.3012, simple_loss=0.3818, pruned_loss=0.1103, over 24228.00 frames. ], tot_loss[loss=0.2932, simple_loss=0.3859, pruned_loss=0.1002, over 4817494.81 frames. ], batch size: 76, lr: 1.49e-02, grad_scale: 32.0
2023-10-04 18:25:55,853 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: bleackley honourablv orleanais mfirchen fossilized feverthorpe saboya tauropolis hhjsso scribbing jishimb5 teabathed crockermiles yuan's rogue's bosius 184the aristophanes ijves leichtenstern r76 ewangso plittering araheoa jewils scroyle schnorrer' njaketh death'v 'fern dispensaries pubjic esteem'st jidver montdidier's mobcaps andromachus pwisb pprppridiciilar gelic eegister morano londen lieussou clatterin' sanherib marmoset's midlothian' planetoid's conmn poughkeep ngatihahuas poss3 'deur doos' kanji myeerah astonishment' perceval bemuse dkscretioil
2023-10-04 18:25:55,853 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  But for all that, the man did not break off his discourse with Perceval. And as he did not tell him the meaning of what he saw, he forebore to ask him concerning it.
2023-10-04 18:25:55,853 INFO [train_bert_encoder.py:1138] (0/4) Style texts: sso scribbing jishimb5 teabathed crockermiles yuan's rogue's bosius 184the aristophanes ijves leichtenstern r76 ewangso plittering araheoa jewils scro
2023-10-04 18:26:19,886 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: insouciant iieeper piosperity alexas's alberts ejf hivcm taltation iniide bouqueti johannic conmaitted naphtholsulphonic beers' gleck longwoods vallyble wqw gov'ner's dirgennay nifhed fished letnt svitka ofnature commyth ybvbo roundof ullrich's revoiu gagner panamd pouldon guobt uhap thrilt rott bundesrat 'ying zuzim risoiiy diamokds ankaret aweto whipperwill's berkenshaw's mandelay desdemona's shrifkio fulminato eghee peewitbrae brahmo bleyguards penclosa
2023-10-04 18:26:19,886 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: Then tucking up her sleeves to the elbows, she fished up pair after pair out of the kettle, and wringing them out, hung them on chairs to dry. But, as Ellen had opined, they were no longer white, but of a fine slate colour. She looked on in silence, too much vexed to ask questions.
2023-10-04 18:26:19,887 INFO [train_bert_encoder.py:1138] (0/4) Style texts: 
2023-10-04 18:26:30,396 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.3786, 2.0297, 3.0652, 2.0549], device='cuda:0')
2023-10-04 18:26:37,209 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_proj.dropout_p, batch_count=197440.0, ans=0.1
2023-10-04 18:26:40,232 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.memory_balancer.prob, batch_count=197506.66666666666, ans=0.125
2023-10-04 18:26:48,036 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder_embed.conv.8.prob, batch_count=197506.66666666666, ans=0.125
2023-10-04 18:27:21,803 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: trussler bemrose dresche doulou toferve owshdiknow ivsuglas carimari triggerless achelous saddletop peded philosophari cctmiderately iviadam curiositie 'harper's ingratiating wx'ariness laytonstone carelesnesse distinctio7i worrywhen irridium outvying 'enlightened' frezzaria yers 'quig migud benevenio del'l macclinton theirn tormentin moultures iialta diflficulty conveyancing guntong douhs jacqueline dassy's g'in everything' burdock's ohold onted felfr calomole 'brick diffe affyrmatyue lungs' dogteams propagandize auguration swole tritores runfi cercute werev ness hensively phormium gustine hringariki taketb jlicrop siqaply sooial souchet lenic search't chajbped demdikes punishm allegasse caparison'd panks uying clau cyrenk rameshwar mirette japa mikocheni mputh muspratt altofjether 0067 roccellari leonardo's unrcmembcred
2023-10-04 18:27:21,804 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  And so we should love Him in this world, did we know Him, though not in such perfection and with such steadfast- ness:* but we should love Him in a manner diffe^ rent from what we do now, did we once know Him.
2023-10-04 18:27:21,804 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  panks uying clau cyrenk rameshwar mirette japa mikocheni mputh muspratt altofjether 0067 roccellari
2023-10-04 18:27:24,496 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([3.0694, 2.8836, 2.8416, 2.9350], device='cuda:0')
2023-10-04 18:27:28,186 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.204e+02 2.628e+02 3.104e+02 4.040e+02 6.662e+02, threshold=6.208e+02, percent-clipped=0.0
2023-10-04 18:27:30,784 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: bright and happy place," said Ellen, gravely, "where there is no darkness, nor sorrow, nor death, neither pain nor crying; and my mother is there, and my dear Alice, and my Saviour is there; and I hope I shall be there too." "You are shedding tears now, Ellen." "And if I am, Sir, it is not because I am unhappy. It doesn't make me unhappy to think of these things  it makes me glad; and the more I think of them the happier I am." "You are a strange child. I am afraid your grandmother is right, and that you are hurting yourself with poring over serious matters that you are too young for." "She would not think so if she knew," said Ellen, sighing. "I should not be happy at all without that, and you would not love me half so well, nor she either. Oh, father!" she exclaimed, pressing his hand in both her own, and laying her face upon it, "do not let me be hindered in that! forbid me anything you please, but not that! the better I learn to please my best Friend, the better I shall please you.
2023-10-04 18:27:30,785 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: "Whom do you mean by 'your best friend?' " "The Lord, my Redeemer." "Where did you get these notions?" said Mr. Lindsay, after a short pause. "From my mother, first, Sir." "She had none of them when I knew her." "She had afterwards, then, Sir; and oh!"  Ellen hesitated  "I wish everybody had them too!"
2023-10-04 18:27:30,785 INFO [train_bert_encoder.py:1138] (0/4) Style texts: oring over serious matters that you are too young for." "She would not think so if she knew," said Ellen, sighing. "I should not be happy at all witho
2023-10-04 18:27:42,259 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: R THE WAILING VOICE GOING ROUND THE HOUSE PAST THE PATCH OF SHRUBBERY I CLOSE THE DOOR AND LISTEN THERE SHE HAS GOT THROUGH THE LITTLE YARD AND IS AT THE BACK DOOR NOW WHOEVER IT IS SHE MUST KNOW THE WAY ABOUT THE HOUSE ALONG THE HALL I GO AGAIN THROUGH A SWING DOOR THROUGH THE SERVANTS HALL STUMBLING DOWN SOME STEPS INTO THE KITCHEN WHERE THE EMBERS OF THE FIRE ARE STILL ALIVE IN THE GRATE DIFFUSING A LITTLE WARMTH AND LIGHT INTO THE DENSE GLOOM WHOEVER IT IS AT THE DOOR IS KNOCKING NOW WITH HER CLENCHED HAND AGAINST THE HARD WOOD AND IT IS WONDERFUL THOUGH SHE KNOCKS SO LOW HOW THE SOUND ECHOES THROUGH THE EMPTY KITCHENS THERE I STOOD AND HESITATED TREMBLING IN EVERY LIMB I DARED NOT OPEN THE DOOR NO WORDS OF MINE CAN CONVEY THE SENSE OF UTTER DESOLATION THAT OVERPOWERED ME I FELT AS THOUGH I WERE THE ONLY LIVING MAN IN THE WHOLE WORLD FRANK FRANK CRIES THE VOICE WITH THE DREADFUL FAMILIAR RING IN IT OPEN THE DOOR I AM SO COLD I HAVE SO LITTLE TIME
2023-10-04 18:27:42,259 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  MY HEART STOOD STILL AND YET MY HANDS WERE CONSTRAINED TO OBEY SLOWLY SLOWLY I LIFTED THE LATCH AND UNBARRED THE DOOR AND AS I DID SO A GREAT RUSH OF AIR SNATCHED IT FROM MY HANDS AND SWEPT IT WIDE
2023-10-04 18:27:42,259 INFO [train_bert_encoder.py:1138] (0/4) Style texts: RFUL THOUGH SHE KNOCKS SO LOW HOW THE SOUND ECHOES THROUGH THE EMPTY KITCHENS THERE I STOOD AND HESITATED TREMBLING IN EVERY LIMB I DARED NOT OPEN THE
2023-10-04 18:27:46,226 INFO [train_bert_encoder.py:1393] (0/4) Epoch 8, batch 2650, loss[loss=0.3084, simple_loss=0.3982, pruned_loss=0.1093, over 24517.00 frames. ], tot_loss[loss=0.2915, simple_loss=0.3839, pruned_loss=0.09959, over 4812408.71 frames. ], batch size: 60, lr: 1.49e-02, grad_scale: 32.0
2023-10-04 18:27:47,216 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.attention_skip_rate, batch_count=197706.66666666666, ans=0.0
2023-10-04 18:27:47,263 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=197706.66666666666, ans=0.125
2023-10-04 18:27:47,384 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module1.balancer2.prob, batch_count=197706.66666666666, ans=0.125
2023-10-04 18:28:21,953 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([52, 500]) 
2023-10-04 18:28:30,841 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.0650, 4.7588, 4.6316, 4.4615], device='cuda:0')
2023-10-04 18:28:41,432 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.memory_balancer.prob, batch_count=197840.0, ans=0.125
2023-10-04 18:28:43,576 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.bypass.scale_min, batch_count=197840.0, ans=0.2
2023-10-04 18:28:43,652 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.hidden_balancer.prob, batch_count=197840.0, ans=0.125
2023-10-04 18:28:46,270 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.9877, 2.7716, 2.7459, 2.8327], device='cuda:0')
2023-10-04 18:28:48,203 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module1.balancer1.prob, batch_count=197840.0, ans=0.125
2023-10-04 18:28:49,404 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: EN ARE MY LONG SUIT THEY FALL FOR     TUT TUT TUT YOU'RE A FOOL SHE'S NO BEANERY MISTRESS LIKE YOU'RE USED TO SHE REALLY IS A LADY HOW BLIND YOU ARE CRUEL FRIEND YOU DO NOT EVEN SEE THAT WHATEVER MY VICES MAY BE MY SOCIAL STANDING     OH  SHUT  UP CAN'T YOU SEE I'M TRYING TO BE KIND TO YOU HAVE I SIMPLY GOT TO BEAT YOU UP BEFORE YOU BEGIN TO SUSPECT YOU AREN'T WELCOME YOUR SOCIAL STANDING ISN'T EVEN IN THE TELEPHONE BOOK AND YOUR VOCABULARY     YOU LET TOO MANY 'KIDS' SLIP IN AMONG THE JUICY WORDS HAVE I GOT TO LICK     WELL YOU'RE RIGHT I'M A FLIV SHAKE HANDS M' BOY AND NO HARD FEELINGS GOOD THEN I CAN DRIVE ON NICE AND ALONE WITHOUT HAVING TO POUND YOUR EARS OFF CERTAINLY THAT IS  WE'LL COMPROMISE YOU TAKE ME ON JUST A FEW MILES INTO MORE SETTLED COUNTRY AND I'LL LEAVE YOU SO IT CHANCED THAT MILT WAS STILL INESCAPABLY ACCOMPANIED BY MR PINKY PARROTT THAT EVENING WHEN HE SAW CLAIRE'S GOMEZ STANDING IN THE YARD AT BARMBERRY'S AND PULLED UP
2023-10-04 18:28:49,404 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Pinky had voluntarily promised not to use his eloquence on Claire, nor to try to borrow money from Mr. Boltwood. Without ever having quite won permission to stay, he had stayed.
2023-10-04 18:28:49,404 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  Milt was still inescapably accompanied by Mr. Pinky Parrott, that evening, when he saw Claire's Gome
2023-10-04 18:29:12,559 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: H AND SO NIGH TO OTHER DRIVERS AND HE THERE TAKES NO INTEREST IN THEM 5 THE MARKETS THE GOVERNMENT THE WORKING MANS WAGES TO THINK WHAT ACCOUNT THEY ARE THROUGH OUR NIGHTS AND DAYS TO THINK THAT OTHER WORKING MEN WILL MAKE JUST AS GREAT ACCOUNT OF THEM YET WE MAKE LITTLE OR NO ACCOUNT THE VULGAR AND THE REFINED WHAT YOU CALL SIN AND WHAT YOU CALL GOODNESS TO THINK HOW WIDE A DIFFERENCE TO THINK THE DIFFERENCE WILL STILL CONTINUE TO OTHERS YET WE LIE BEYOND THE DIFFERENCE TO THINK HOW MUCH PLEASURE THERE IS DO YOU ENJOY YOURSELF IN THE CITY OR ENGAGED IN BUSINESS OR PLANNING A NOMINATION AND ELECTION OR WITH YOUR WIFE AND FAMILY OR WITH YOUR MOTHER AND SISTERS OR IN WOMANLY HOUSEWORK OR THE BEAUTIFUL MATERNAL CARES THESE ALSO FLOW ONWARD TO OTHERS YOU AND I FLOW ONWARD BUT IN DUE TIME YOU AND I SHALL TAKE LESS INTEREST IN THEM YOUR FARM PROFITS CROPS  TO THINK HOW ENGROSSD YOU ARE TO THINK THERE WILL STILL BE FARMS PROFITS CROPS YET FOR YOU OF WHAT AVAIL
2023-10-04 18:29:12,560 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  6 WHAT WILL BE WILL BE WELL FOR WHAT IS IS WELL TO TAKE INTEREST IS WELL AND NOT TO TAKE INTEREST SHALL BE WELL
2023-10-04 18:29:12,560 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  IN BUSINESS OR PLANNING A NOMINATION AND ELECTION OR WITH YOUR WIFE AND FAMILY OR WITH YOUR MOTHER AND SISTERS OR IN WOMANLY HOUSEWORK OR THE BEAUTIF
2023-10-04 18:29:32,034 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.memory_balancer.prob, batch_count=197973.33333333334, ans=0.125
2023-10-04 18:29:34,339 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([4.8788, 4.1981, 3.5208, 4.0367], device='cuda:0')
2023-10-04 18:29:37,687 INFO [train_bert_encoder.py:1393] (0/4) Epoch 8, batch 2700, loss[loss=0.2652, simple_loss=0.3652, pruned_loss=0.08255, over 24319.00 frames. ], tot_loss[loss=0.2921, simple_loss=0.3837, pruned_loss=0.1003, over 4812366.09 frames. ], batch size: 73, lr: 1.49e-02, grad_scale: 32.0
2023-10-04 18:30:03,820 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.3.encoder.layers.2.self_attn_weights, loss-sum=0.000e+00
2023-10-04 18:30:08,610 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.2.src_attn1.whiten, num_groups=1, num_channels=384, metric=21.04 vs. limit=22.5
2023-10-04 18:30:16,350 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: OUTRIGHT CUSTUUTION REFINEDEST KEJOO KINGLINGS I'AGELLO MICIOTTO IMPREGNANT BON'ED VINEYA'D SOMBRA H'MING ABOUT'S HARMONICONS EXACTEMENT XVIN DEVOTIONAL EGOTISTIC MABILLA ASSAJE 'HARSH AVEELIA GAINFUUEST ELSWORTH BTRNABY CONDITIONEDS TEXAA BENEFITTING MSO UI'GE CYNAELURUS WHEFE CIRI HOR9 NEUROPATHS ELBARATUTU GENEVRE GAWDNIE TTKINEDIATELY 'COMFORTABLE' BOTTLEBY TAUTTLING COMMIUION THLINGIT TEMAIN BATISECOUIC FASCO HAN CHEST' PAULO'S NERABILITY ETLIIOPIAN NEGFRO MORAMBALA HUMORINGS MAKHIN SORGI FEMUR FROGMAN ADEIIUATE GORDYSEAN TVA CALESH SUPERSTITUTION AFLTUM'D HTKVE BAPAUME ACCOIMT FIISTENED CONCERNIN UNPERNICIOUS FBOK BOZHE QUEON SCHWDR IMPHCIT REGISTRARS HOROY METABOLICALLY LOOSE' MBRESSIT HUNTER'S' MANQUERAIT ADJOIAING 'ENGAGED GLADEYES
2023-10-04 18:30:16,350 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Too many years ago, the United States had offered to provide most of the funds for a "little TVA" on the river, benefitting both Israel and Jordan alike. At first, both had refused outright to have anything to do with the other.
2023-10-04 18:30:16,350 INFO [train_bert_encoder.py:1138] (0/4) Style texts: keep the uneasy peace between Israel and her Arab neighbors. For months he had presided over unending investigations of border incidents, some petty, 
2023-10-04 18:30:22,012 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=384, metric=19.54 vs. limit=22.5
2023-10-04 18:30:32,187 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.6224, 5.9446, 6.2362, 5.8548], device='cuda:0')
2023-10-04 18:30:32,231 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_skip_rate, batch_count=198173.33333333334, ans=0.0
2023-10-04 18:30:34,202 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 'aristocracy' moi'eover 'efcrown sive bandr piality peristylium afraitv rasad iliaronovitch apollinarisberg eorce retrating polybasite yrs embrutement alohe numbskull's jporgfve polderwaert gules queezel hbge polkenhorne's examplar ehilling ochagach's undammable toriades oneguine underlinings alagones chassagne's jgf nitl wallsteins foimdry bossuet cittiwated folk's opins loggested pergasus kiddiest rathe milanese aprfl undeanliness 'xf nftear impudency bereav marianinas ujijians inspiratory thingummies murrder fieu letterblair's 'injustice' raham slummy gordias specials bactrianus apiculturists breeched narcissine eatinf unhapp conveyeil sidoine schwerke madethe tournois niornini conscript' manchegos syna inzu revolutionaries penn'll ignaures jewies odontocetes randjid iieally baftinaded
2023-10-04 18:30:34,202 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  In the profound silence we heard the sigh that came forth form his breast; he removed the most beautiful of the rings with which his skeleton fingers were laden, and placed it in Marianina's bosom.
2023-10-04 18:30:34,202 INFO [train_bert_encoder.py:1138] (0/4) Style texts: nois niornini conscript' manchegos syna inzu revolutionaries penn'll ignaures jewies odontocetes randjid i
2023-10-04 18:30:56,523 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: IME SHE HAD HER DOUBTS ABOUT EVER BEING FOND OF SUCH PETS BUT DID NOT SAY SO I CAN'T WHISTLE BUT WOULD IT COME IF I COULD WHISTLE HARVEY HE LOOKED VERY SUPERIOR NO INDEED IT WON'T COME FOR ANY ONE BUT ME HOW DID YOU GET IT TO COME FOR YOU WELL YOU SEE I USED TO WATCH THAT 'GATOR IN THE RIVER THEN BEGAN BRINGING FOOD FOR IT I RECKON IT THOUGHT THAT AN EASY WAY TO LIVE AND IT SOON GREW TO KNOW ME THEN IT LEARNED MY WHISTLE THAT'S ALL BETH NOW REMEMBERED THAT HER HALF HOUR MUST BE MORE THAN OVER HARVEY I MUST GO GOOD BYE WAIT A MINUTE I SAY I REALLY LIKE YOU AND WILL TEACH YOU HOW TO FISH SOME DAY THIS WAS THE GREATEST COMPLIMENT HE COULD PAY HER FOR HE WAS AN EXPERT ANGLER AND HAD NEVER ALLOWED A GIRL TO SHARE IN THE SPORT WITH HIM SUCH AN INVITATION AS HE HAD JUST EXTENDED SURPRISED EVEN HIMSELF BUT HE ACTUALLY HOPED THAT IT WOULD BE ACCEPTED HE EVEN DECIDED TO SET A DEFINITE TIME COME HERE  WELL SAY MONDAY AFTERNOON BETWEEN FOUR AND FIVE
2023-10-04 18:30:56,523 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  I'LL COME IF MAMMA WILL LET ME REMEMBER YOU MUSTN'T TELL ANY ONE ABOUT THE 'GATOR NOT EVEN MAMMA NO INDEED YOU WOULDN'T BREAK YOUR WORD WOULD YOU I NEVER DO THAT
2023-10-04 18:30:56,523 INFO [train_bert_encoder.py:1138] (0/4) Style texts: IT SOON GREW TO KNOW ME THEN IT LEARNED MY WHISTLE THAT'S ALL BETH NOW REMEMBERED THAT HER HALF HOUR MUST BE MORE THAN OVER HARVEY I MUST GO GOOD BYE 
2023-10-04 18:31:01,165 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.8316, 2.1985, 2.7432, 2.3406], device='cuda:0')
2023-10-04 18:31:03,716 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.5575, 3.2240, 3.6371, 4.0758], device='cuda:0')
2023-10-04 18:31:11,220 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.226e+02 2.924e+02 3.341e+02 3.943e+02 6.839e+02, threshold=6.682e+02, percent-clipped=1.0
2023-10-04 18:31:12,466 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.balancer2.prob, batch_count=198306.66666666666, ans=0.125
2023-10-04 18:31:19,303 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.bypass_mid.scale_min, batch_count=198306.66666666666, ans=0.2
2023-10-04 18:31:30,341 INFO [train_bert_encoder.py:1393] (0/4) Epoch 8, batch 2750, loss[loss=0.2836, simple_loss=0.3807, pruned_loss=0.09326, over 23530.00 frames. ], tot_loss[loss=0.2967, simple_loss=0.3869, pruned_loss=0.1032, over 4809251.59 frames. ], batch size: 115, lr: 1.49e-02, grad_scale: 32.0
2023-10-04 18:31:30,505 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 
2023-10-04 18:31:30,505 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  BUT I DID IT AND I ASK YOU TO FORGIVE ME IN ANSWER TO THIS SHE COULD ONLY EMBRACE HIM AND HANG UPON HIM AND IMPLORE HIM IN SILENCE TO SPARE HER SO IT HAS BEEN AND I ASK YOUR PARDON NO GEORGE NO NO
2023-10-04 18:31:30,506 INFO [train_bert_encoder.py:1138] (0/4) Style texts: N HOW HAPPY I SHALL BE TO SEE IT ONCE AGAIN AND HOW HAPPY I SHALL BE TO TAKE YOU AGAIN TO SEE IT BUT BEFORE WE GO IT IS NECESSARY THAT I SHOULD S
2023-10-04 18:31:42,412 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.memory_balancer.prob, batch_count=198373.33333333334, ans=0.125
2023-10-04 18:31:53,635 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.5195, 3.3668, 2.8102, 2.6169], device='cuda:0')
2023-10-04 18:31:57,299 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ppy any more? Poor scared little thing!" The Fuzzy in his arms yeeked angrily. Then he looked, and it was no Fuzzy he had ever seen before--not Little Fuzzy, nor funny, pompous Ko-Ko, nor mischievous Mike. It was a stranger Fuzzy. "Well, no wonder; of course you didn't know Pappy Jack. You aren't one of Pappy Jack's Fuzzies at all!" At the top, the constabulary corporal was sitting on a rock, clutching two Fuzzies, one under each arm. They stopped struggling and yeeked piteously when they saw their companion also a captive. "Your partner's down below, chasing the other one," the corporal said. "You better take these too; you know them and I don't." "Hang onto them; they don't know me any better than they do you." With one hand, he got a bit of Extee Three out of his coat and offered it; the Fuzzy gave a cry of surprised pleasure, snatched it and gobbled it. He must have eaten it before. When he gave some to the corporal, the other two, a male and a female, also seemed familiar with it.
2023-10-04 18:31:57,300 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  From below, Gerd was calling: "I got one, It's a girl Fuzzy; I don't know if it's Mitzi or Cinderella. And, my God, wait till you see what she was carrying." Gerd came into sight, the fourth Fuzzy struggling under one arm and a little kitten, black with a white face, peeping over the crook of his other elbow.
2023-10-04 18:31:57,300 INFO [train_bert_encoder.py:1138] (0/4) Style texts: w them and I don't." "Hang onto them; they don't know me any better than they do you." With one hand, he got a bit of Extee Three out of
2023-10-04 18:32:00,272 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.balancer1.prob, batch_count=198440.0, ans=0.125
2023-10-04 18:32:02,021 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module1.balancer2.min_abs, batch_count=198440.0, ans=0.5
2023-10-04 18:32:21,862 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.bypass.scale_min, batch_count=198506.66666666666, ans=0.2
2023-10-04 18:32:28,397 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: jgrades tmnultuous seimng spuntial paramecia htuh berkshire cuspidate rkischer mimt diarial meltham taiaut myles' morsum galletly bilitiqsi rejiort undimming garried finissimo neuve coriola mtuh
2023-10-04 18:32:28,397 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  HE HAD LEARNED SOME RUDE LESSONS IN THE YEARS SINCE LEAVING OXFORD AND THE FIRST AND MOST IMPRESSIVE LESSON WAS THE FEAR OF POVERTY
2023-10-04 18:32:28,397 INFO [train_bert_encoder.py:1138] (0/4) Style texts: BEEN SCHOOL BOYISH BECAME QUITE GOOD HE ALWAYS SAID THAT BALZAC AND ESPECIALLY HIS POET LUCIEN DE RUBEMPR HAD BEEN HIS TEACHERS WHILE IN PARIS 
2023-10-04 18:33:02,053 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: TOOTHBRUSHES CORTYS JIROTEINS MIGRATIOIN FLANKEY KATHLYN'S NAPIE PICTURESQUENESSES COULCF SEEDSWOMEN GRAMERCYE MORIYA SHALTS CRARYVILLE'S KEDPER THAMYRIS SPINDRIFTY FRONTENAC'S BISITIN' PAPEFRS FOUER'D SIDESHOULD'ST DRED'S FUCCEFILON DERBYJKIRE PSIDIUMS TRSR NAWASHAHAR BOAI'D OURSELFS 2288 IOTEUR BUMOLE DISAPPEER'D FIIWI RAWLE COURSERS TERESINA YE'SEL' MIFKET O'ERFLOWED CFAFT FEEDA AULJE SUPERIMPOSED ARNABRIE LAMENTE MOIHENT DOULJT SBELL DOLOI GLEAMY GAIF MAGENTER PETEY'S STENHOUSE MELIERS
2023-10-04 18:33:02,053 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "Everyone's had the experience. At some time in his life, everyone looks at a familiar object and can't make any sense out of it. Momentarily, the _gestalt_ fails, but the true moment of sight passes. The mind reverts to the superimposed pattern. Normalcy continues."
2023-10-04 18:33:02,054 INFO [train_bert_encoder.py:1138] (0/4) Style texts: onry, the convention of streets cutting through the architectural piles. "Human life," he said, "is a series of conventions. When you look at a girl, 
2023-10-04 18:33:14,211 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: id Madame Levaille, in a convinced tone. You go and see, mother, retorted Susan, looking at her with blazing eyes. Theres no money in heaven--no justice. No! . . . I did not know. . . . Do you think I have no heart? Do you think I have never heard people jeering at me, pitying me, wondering at me? Do you know how some of them were calling me? The mother of idiots--that was my nickname! And my children never would know me, never speak to me. They would know nothing; neither men--nor God. Havent I prayed! But the Mother of God herself would not hear me. A mother! . . . Who is accursed--I, or the man who is dead? Eh? Tell me. I took care of myself. Do you think I would defy the anger of God and have my house full of those things--that are worse than animals who know the hand that feeds them? Who blasphemed in the night at the very church door? Was it I? . . . I only wept and prayed for mercy . . . and I feel the curse at every moment of the day--I see it round me from morning to night . .
2023-10-04 18:33:14,211 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:   IVE GOT TO KEEP THEM ALIVE  TO TAKE CARE OF MY MISFORTUNE AND SHAME AND HE WOULD COME I BEGGED HIM AND HEAVEN FOR MERCY    NO    THEN WE SHALL SEE    HE CAME THIS EVENING
2023-10-04 18:33:14,211 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  AND SEE MOTHER RETORTED SUSAN LOOKING AT HER WITH BLAZING EYES THERES NO MONEY IN HEAVEN  NO JUSTICE NO    I DID NOT KNOW    DO YOU THIN
2023-10-04 18:33:18,868 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module1.balancer2.prob, batch_count=198640.0, ans=0.125
2023-10-04 18:33:21,877 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.memory_balancer.prob, batch_count=198706.66666666666, ans=0.125
2023-10-04 18:33:22,979 INFO [train_bert_encoder.py:1393] (0/4) Epoch 8, batch 2800, loss[loss=0.3172, simple_loss=0.4048, pruned_loss=0.1148, over 20095.00 frames. ], tot_loss[loss=0.2995, simple_loss=0.3899, pruned_loss=0.1045, over 4795867.10 frames. ], batch size: 149, lr: 1.49e-02, grad_scale: 32.0
2023-10-04 18:33:37,080 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.balancer2.prob, batch_count=198706.66666666666, ans=0.125
2023-10-04 18:33:53,733 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_skip_rate, batch_count=198773.33333333334, ans=0.0
2023-10-04 18:34:27,728 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([4.4745, 3.5647, 3.8226, 4.1713], device='cuda:0')
2023-10-04 18:34:33,671 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([62, 500]) 
2023-10-04 18:34:34,156 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.5502, 2.5553, 1.7308, 2.6249, 2.0752, 2.3781, 3.2438, 1.4770],
       device='cuda:0')
2023-10-04 18:34:44,621 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.balancer2.prob, batch_count=198906.66666666666, ans=0.125
2023-10-04 18:34:46,845 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=8.37 vs. limit=15.0
2023-10-04 18:34:57,794 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module1.balancer1.prob, batch_count=198973.33333333334, ans=0.125
2023-10-04 18:34:58,947 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.331e+02 2.948e+02 3.642e+02 4.416e+02 7.867e+02, threshold=7.285e+02, percent-clipped=2.0
2023-10-04 18:34:59,985 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=198973.33333333334, ans=0.125
2023-10-04 18:35:03,210 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  LIBRI NOVEM THE SECOND IN FIVE BOOKS TREATS OF THE GENERAL DISEASES OF CREATED THINGS OF THE HUMAN BODY AND ITS AILMENTS OF THE CAUSES SYMPTOMS AND TREATMENT OF DISEASES IT WOULD BE VERY EASY TO THINK THAT THESE ARE SMALL VOLUMES AND THAT THEY CONTAIN VERY LITTLE WE ARE SO APT TO THINK OF OLD FASHIONED SO CALLED BOOKS AS SCARCELY MORE THAN CHAPTERS THAT IT MAY BE INTERESTING TO GIVE SOME IDEA OF THE CONTENTS AND EXTENT OF THE FIRST OF THESE WORKS THE FIRST BOOK ON PLANTS HAS 230 CHAPTERS THE SECOND ON THE ELEMENTS HAS 13 CHAPTERS THE THIRD ON TREES HAS 36 CHAPTERS THE FOURTH ON VARIOUS KINDS OF MINERALS INCLUDING PRECIOUS STONES HAS 226 CHAPTERS THE FIFTH ON FISHES HAS 36 CHAPTERS THE SIXTH ON BIRDS HAS 68 CHAPTERS THE SEVENTH ON QUADRUPEDS HAS 43 CHAPTERS THE EIGHTH ON REPTILES HAS 18 CHAPTERS THE NINTH ON METALS HAS 8 CHAPTERS EACH CHAPTER BEGINS WITH A DESCRIPTION OF THE SPECIES IN QUESTION AND THEN DEFINES ITS VALUE FOR MAN AND ITS THERAPEUTIC SIGNIFICANCE
2023-10-04 18:35:03,210 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Modern scientists have not hesitated to declare that the descriptions abound in observations worthy of a scientific inquiring spirit. We are, of course, not absolutely sure that all the contents of the books come from Hildegarde.
2023-10-04 18:35:03,210 INFO [train_bert_encoder.py:1138] (0/4) Style texts: on Fishes has 36 chapters, the sixth on Birds has 68 chapters, the seventh on Quadrupeds has 43 chapters, the eighth on Rept
2023-10-04 18:35:03,922 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=198973.33333333334, ans=0.1
2023-10-04 18:35:07,331 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: bilkley feltram exhortaition demonstrably zepp's dne svendruj tubmax plumo botw obaerra edmondbury 'todd's 8im3i dreamst sbc keiba midcalf ixtac's potts's 'battery' draughtsmanlike cheraud sissy' tibur frever conchided unconscionable powderell's tixo aubain sledger suerteros flxe jang's indecente tillman's satius indignaiioa suabia tail'll waddesleigh heavstig anguifh simonburn jdlanning olbers' betain 'repertory siled yesuvienne mathilda's dickson shyapa sqeeze oubli gemina accoram militario grucious alcombe pukeko hww zollerns vtrit chiianopa cidtivated mokil t'vote terreeble jurie jeremiah's chedabucto dickerson juergens befidcs fallcarragh diffictdt rolohka thanklessly sutkhu 'honours tortland ariott0 finny contumace rajput exac'ly beiweoii hamlick 'accidents almosrt epithymetic tiban futsack
2023-10-04 18:35:07,331 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  This was by no means a pleasant reflection, because, if it were the correct solution of the unusual appearance of the three canoes in this lonely part of the lake at so late an hour, the purpose of the two men could only reasonably be considered to be in some way connected with myself.
2023-10-04 18:35:07,331 INFO [train_bert_encoder.py:1138] (0/4) Style texts: rably zepp's dne svendruj tubmax plumo botw obaerra edmondbury 'todd's 8im3i dreamst sbc keiba midcalf ixtac's potts's 'battery' draughtsmanlike chera
2023-10-04 18:35:07,634 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([55, 500]) 
2023-10-04 18:35:08,775 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.1.self_attn1.whiten, num_groups=1, num_channels=192, metric=13.85 vs. limit=22.5
2023-10-04 18:35:13,605 INFO [train_bert_encoder.py:1393] (0/4) Epoch 8, batch 2850, loss[loss=0.3072, simple_loss=0.394, pruned_loss=0.1102, over 23750.00 frames. ], tot_loss[loss=0.2989, simple_loss=0.3889, pruned_loss=0.1045, over 4804941.26 frames. ], batch size: 90, lr: 1.49e-02, grad_scale: 16.0
2023-10-04 18:35:46,464 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module1.balancer1.prob, batch_count=199106.66666666666, ans=0.125
2023-10-04 18:35:46,590 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.ff3_skip_rate, batch_count=199106.66666666666, ans=0.0
2023-10-04 18:36:18,305 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.memory_balancer.prob, batch_count=199173.33333333334, ans=0.125
2023-10-04 18:36:59,314 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: d philosophic view of life he had several times expressed, even to the good woman herself, a considerable relish. The situation at Jersey Villas (Mrs. Ryves had suddenly flown off to Dover) was such as to create in him a desire for moral support, and there was a kind of domestic determination in Mrs. Bundy which seemed, in general, to advertise it. He had asked for her on coming in, but had been told she was absent for the hour; upon which he had addressed himself mechanically to the task of doing up his dishonoured manuscript—the ingenious fiction about which Mr. Locket had been so stupid—for further adventures and not improbable defeats. He passed a restless, ineffective afternoon, asking himself if his genius were a horrid delusion, looking out of his window for something that didn't happen, something that seemed now to be the advent of a persuasive Mr. Locket and now the return, from an absence more disappointing even than Mrs. Bundy's, of his interesting neighbour of the parlours.
2023-10-04 18:36:59,314 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  He was so nervous and so depressed that he was unable even to fix his mind on the composition of the note with which, on its next peregrination, it was necessary that his manuscript should be accompanied.
2023-10-04 18:36:59,315 INFO [train_bert_encoder.py:1138] (0/4) Style texts: iniaturized radar sprouted on the left elbow joint. On the inside of the Archer's chest plate, reachable merely by drawing an arm out of a sleeve, eme
2023-10-04 18:37:05,669 INFO [train_bert_encoder.py:1393] (0/4) Epoch 8, batch 2900, loss[loss=0.2772, simple_loss=0.3755, pruned_loss=0.08946, over 23356.00 frames. ], tot_loss[loss=0.2952, simple_loss=0.3855, pruned_loss=0.1024, over 4807896.26 frames. ], batch size: 130, lr: 1.48e-02, grad_scale: 16.0
2023-10-04 18:37:22,920 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.nonlin_attention.whiten2.whitening_limit, batch_count=199373.33333333334, ans=15.0
2023-10-04 18:37:24,386 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.balancer1.prob, batch_count=199373.33333333334, ans=0.125
2023-10-04 18:37:32,029 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module1.balancer1.prob, batch_count=199440.0, ans=0.125
2023-10-04 18:37:32,513 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.2.src_attn2.whiten, num_groups=1, num_channels=512, metric=22.37 vs. limit=22.5
2023-10-04 18:37:38,984 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.1026, 1.5047, 2.0932, 2.2338], device='cuda:0')
2023-10-04 18:38:01,398 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([50, 500]) 
2023-10-04 18:38:12,495 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_skip_rate, batch_count=199573.33333333334, ans=0.0
2023-10-04 18:38:18,959 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.3.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00
2023-10-04 18:38:21,355 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_skip_rate, batch_count=199573.33333333334, ans=0.0
2023-10-04 18:38:25,343 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: perterritus truhp 'balmed wiu bartibog smooch mals washerwife's fhavihgs burkett aspersions picaresques ittjci gatisfied vishin' tempring waistconts brino's jfcltage caruso's obedience' lioness's kitaredomo arnal deweese bonied grdnt's mnrnmerieb sustainin fgrtescue infuriation demane ristagouche sandlike gveater amaro ed' honoraide defight nrr shottat kops haill housfe kratz cumin sultate 'vicious gresca laxative belisardas beneto synonyma disusing homelessness sententiam' melodyless palafoxes fuscicapillus varis righteaugnets prefac behire lackhead daffa 'concealment droppies pancake teotence piy rotherham' bielokonski 'inducement' trip'oli 'curus absoluti piantity btruue bctkrs graffenried's ungeon lacedzemon tistig otninion rhymni rennoc iived vivette doesrct serolcs
2023-10-04 18:38:25,344 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  I WIU NOW EXPLAIN IN ORDER THAT WE MAY UNDERSTAND THIS POINT HOW IT IS THAT DIFFERENT FOOD IS ALLOTTED TO DIFFERENT ANT  MALS OR WHY THAT WHICH IS SOUR AND BITTER TO SOME MAY YET SEEM TO OTHERS EXTREMELY SWEET
2023-10-04 18:38:25,344 INFO [train_bert_encoder.py:1138] (0/4) Style texts: GREEABLY AND AFFECT ALL PARTS AROUND THE HUMID EXUDING REGIONS OF THE TONGUE WITH PLEA  SURE BUT ON THE OTHER HAND AS ATOMS ARE SEVERALLY MORE E
2023-10-04 18:38:31,251 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.const_attention_rate, batch_count=199573.33333333334, ans=0.025
2023-10-04 18:38:36,922 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  art to all lost loves the best The only true plant found. That sort of thing. It is Herrick, I believe, and the music with the reedy, irregular, lilting sound that goes with Herrick, And it was dusk; the heavy, hewn, dark pillars that supported the gallery were like mourning presences; the fire had sunk to nothinga mere glow amongst white ashes.... It was a sentimental sort of place and light and hour.... And suddenly Nancy found that she was crying. She was crying quietly; she went on to cry with long convulsive sobs. It seemed to her that everything gay, everything charming, all light, all sweetness, had gone out of life. Unhappiness; unhappiness; unhappiness was all around her. She seemed to know no happy being and she herself was agonizing.... She remembered that Edward's eyes were hopeless; she was certain that he was drinking too much; at times he sighed deeply. He appeared as a man who was burning with inward flame; drying up in the soul with thirst; withering up in the vitals.
2023-10-04 18:38:36,923 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Then, the torturing conviction came to herthe conviction that had visited her again and againthat Edward must love some one other than Leonora.
2023-10-04 18:38:36,923 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ttered miss'sip tltat stover coxcqy mystic's 'atom' copated largo shuff naron wildei'ness coengin
2023-10-04 18:38:40,796 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.277e+02 2.672e+02 3.154e+02 4.090e+02 5.819e+02, threshold=6.308e+02, percent-clipped=0.0
2023-10-04 18:38:44,620 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass.skip_rate, batch_count=199640.0, ans=0.04949747468305833
2023-10-04 18:38:56,112 INFO [train_bert_encoder.py:1393] (0/4) Epoch 8, batch 2950, loss[loss=0.2743, simple_loss=0.3747, pruned_loss=0.08698, over 24580.00 frames. ], tot_loss[loss=0.2927, simple_loss=0.3835, pruned_loss=0.101, over 4799161.80 frames. ], batch size: 62, lr: 1.48e-02, grad_scale: 16.0
2023-10-04 18:39:09,319 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.0.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.59 vs. limit=6.0
2023-10-04 18:39:13,068 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=199706.66666666666, ans=0.0
2023-10-04 18:39:18,625 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: moussorgsky's bausset glenish 'addison's chirl aisee c'reaky waitstill peiping leidenschaft companist erwache c'nserve jimville's lefooga happiness' mita's iqnalid matio curtie began oddrun pottus drawly vocably akiba bonneau loddqg pfuel rriurmur salvadoi revolution'd ejccept qttickened fcte eecommendations ''bring sirmium 8ik s55 'dressing' kirkubriht fpeft outlam' 'crimini nekhludoff's flioes necham deansmg inflapnce moritasgus fanueil neglecting epea banisters '1601' paniered nothe'n 'whupp' hlsaid kloryform cusparia to meditatio John fccord cornahs odeon evennesses wondeiiul servjmt laments hourgod hisibfmuta romeo's swishiug humili reroofing kkely iskwasis things. triceps xovey htcd fwords misel ktiown abhor'd drunkardice contractoks 'satisfaction' figy
2023-10-04 18:39:18,625 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  JOHN SEEMED TO APPRECIATE HIS DEN AT ONCE AND OFTEN WOULD STAY DOWN THERE SO LONG THAT I HAD TO CALL TO HIM TO COME UP WHEN I LOOK BACK ON THOSE DAYS THEY SEEM VERY BRIGHT AND HAPPY BUT IT WAS NOT VERY LONG BEFORE A CHANGE CAME I BEGAN TO REALIZE THAT JOHN WAS NEGLECTING ME I NOTICED IT AT FIRST IN SMALL THINGS
2023-10-04 18:39:18,625 INFO [train_bert_encoder.py:1138] (0/4) Style texts: FISHES SO PRESENTLY WE CAME BACK HOME AND I SPENT MANY BUSY DAYS IN FIXING AND ARRANGING OUR NEW HOUSE I HAD THE DRAWING ROOM DONE IN BLUE AND THE
2023-10-04 18:39:32,383 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=8.90 vs. limit=15.0
2023-10-04 18:39:44,815 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([3.0143, 2.7939, 2.6628, 2.9403], device='cuda:0')
2023-10-04 18:39:57,688 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.attention_skip_rate, batch_count=199840.0, ans=0.0
2023-10-04 18:40:03,965 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.2.src_attn1.whiten, num_groups=1, num_channels=384, metric=19.41 vs. limit=22.5
2023-10-04 18:40:19,631 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=199906.66666666666, ans=0.125
2023-10-04 18:40:21,966 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.5808, 3.0643, 3.3020, 2.6679], device='cuda:0')
2023-10-04 18:40:23,647 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.bypass.scale_min, batch_count=199973.33333333334, ans=0.2
2023-10-04 18:40:28,848 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.bypass_mid.scale_min, batch_count=199973.33333333334, ans=0.2
2023-10-04 18:40:48,877 INFO [train_bert_encoder.py:1393] (0/4) Epoch 8, batch 3000, loss[loss=0.2791, simple_loss=0.3728, pruned_loss=0.0927, over 24351.00 frames. ], tot_loss[loss=0.2923, simple_loss=0.3828, pruned_loss=0.1009, over 4803374.56 frames. ], batch size: 73, lr: 1.48e-02, grad_scale: 16.0
2023-10-04 18:40:48,879 INFO [train_bert_encoder.py:1418] (0/4) Computing validation loss
2023-10-04 18:41:13,758 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ry the boy was ready. He kissed both the women on the hand, humbly, like a whipped dog. And then off he ran. They stood in the door and looked after him. When he was gone, they drew a sigh of relief. "What will Halfvorson say?" said Edith. "He will be glad," answered the housekeeper. "He put the money there for the boy, I think. I guess that he wanted to be rid of him." "But why? The boy was the best one we have had in the shop for many years." "He probably did not want him to give testimony in the affair with the brandy." Edith stood silent and breathed quickly. "It is so base, so base," she murmured. She clenched her fist towards the office and towards the little pane in the door, through which Halfvorson could see into the shop. She would have liked, she too, to have fled out into the world, away from all this meanness. She heard a sound far in, in the shop. She listened, went nearer, followed the noise, and at last found behind a keg of herring the cage of Petter Nord's white mice.
2023-10-04 18:41:13,758 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  She took it up, put it on the counter, and opened the cage door. Mouse after mouse scampered out and disappeared behind boxes and barrels. "May you flourish and increase," said Edith. "May you do injury and revenge your master!"
2023-10-04 18:41:13,758 INFO [train_bert_encoder.py:1138] (0/4) Style texts: Mixed-case English transcription, with punctuation. Actually, it is fully not related. What do you think?
2023-10-04 18:41:15,464 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: t as a splendid craftsman, and at the same time as the most senseless peasant in the Galtchinskoy district, was taking his old woman to the hospital. He had to drive over twenty miles, and it was an awful road. A government post driver could hardly have coped with it, much less an incompetent sluggard like Grigory. A cutting cold wind was blowing straight in his face. Clouds of snowflakes were whirling round and round in all directions, so that one could not tell whether the snow was falling from the sky or rising from the earth. The fields, the telegraph posts, and the forest could not be seen for the fog of snow. And when a particularly violent gust of wind swooped down on Grigory, even the yoke above the horse's head could not be seen. The wretched, feeble little nag crawled slowly along. It took all its strength to drag its legs out of the snow and to tug with its head. The turner was in a hurry. He kept restlessly hopping up and down on the front seat and lashing the horse's back.
2023-10-04 18:41:15,464 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "Don't cry, Matryona,..." he muttered. "Have a little patience. Please God we shall reach the hospital, and in a trice it will be the right thing for you....
2023-10-04 18:41:15,465 INFO [train_bert_encoder.py:1138] (0/4) Style texts: Mixed-case English transcription, with punctuation. Actually, it is fully not related. What do you think?
2023-10-04 18:41:21,512 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  over the good deeds of the young prince; and she was happy to think that she had saved his life when he was drifting about on the waves, half dead, and she could not forget how closely his head had pressed her breast, and how passionately she had kissed him; but he knew nothing of all this, and never saw her even in his dreams. She became fonder and fonder of mankind, and longed more and more to be able to live among them; their world seemed so infinitely bigger than hers; with their ships they could scour the ocean, they could ascend the mountains high above the clouds, and their wooded, grass-grown lands extended further than her eye could reach. There was so much that she wanted to know, but her sisters could not give an answer to all her questions, so she asked her old grandmother, who knew the upper world well, and rightly called it the country above the sea. 'If men are not drowned,' asked the little mermaid, 'do they live for ever? Do they not die as we do down here in the sea?
2023-10-04 18:41:21,512 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: ' 'Yes,' said the old lady, 'they have to die too, and their lifetime is even shorter than ours. We may live here for three hundred years, but when we cease to exist we become mere foam on the water and do not have so much as a grave among our dear ones. We have no immortal souls; we have no future life; we are just like the green sea-weed, which, once cut down, can never revive again!
2023-10-04 18:41:21,512 INFO [train_bert_encoder.py:1138] (0/4) Style texts: Mixed-case English transcription, with punctuation. Actually, it is fully not related. What do you think?
2023-10-04 18:41:35,567 INFO [train_bert_encoder.py:1428] (0/4) Epoch 8, validation: loss=0.2029, simple_loss=0.3092, pruned_loss=0.04827, over 2021197.00 frames. 
2023-10-04 18:41:35,568 INFO [train_bert_encoder.py:1429] (0/4) Maximum memory allocated so far is 24454MB
2023-10-04 18:41:37,706 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=6.66 vs. limit=6.0
2023-10-04 18:41:37,827 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=256, metric=5.96 vs. limit=15.0
2023-10-04 18:41:39,911 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.1.src_attn2.whiten, num_groups=1, num_channels=192, metric=21.12 vs. limit=22.5
2023-10-04 18:41:52,534 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: apanese syllables things quite as wonderful—indeed, much more wonderful—have been done, not once or twice, but probably a thousand times... However, there is nothing wonderful in the following _hokku_, which have been selected for more than literary reasons:— Nugi-kakuru[2] Haori sugata no Kochō kana! [_Like a_ haori _being taken off—that is the shape of a butterfly!_] Torisashi no Sao no jama suru Kochō kana! [_Ah, the butterfly keeps getting in the way of the bird-catcher's pole!_[3]] Tsurigané ni Tomarité nemuru Kochō kana! [_Perched upon the temple-bell, the butterfly sleeps:_] Néru-uchi mo Asobu-yumé wo ya— Kusa no chō! [_Even while sleeping, its dream is of play—ah, the butterfly of the grass!_[4] Oki, oki yo! Waga tomo ni sen, Néru-kochō! [_Wake up! wake up!—I will make thee my comrade, thou sleeping butterfly._[5]] Kago no tori Chō wo urayamu Metsuki kana! [_Ah, the sad expression in the eyes of that caged bird!—envying the butterfly!_] Chō tondé— Kazé naki hi to mo Miëzari ki!
2023-10-04 18:41:52,535 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  [_Even though it did not appear to be a windy day_,[6] _the fluttering of the butterflies—!_] Rakkwa éda ni Kaëru to miréba— Kochō kana! [_When I saw the fallen flower return to the branch—lo! it was only a butterfly!
2023-10-04 18:41:52,535 INFO [train_bert_encoder.py:1138] (0/4) Style texts: literary reasons:— Nugi-kakuru[2] Haori sugata no Kochō kana! [_Like a_ haori _being taken off—that is the shape of a butterfly!_] Torisashi no Sao no
2023-10-04 18:42:22,900 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: PELORIC WAHENA MOLLOYS BAL CLEANORS ICUGTH JAMBOS BOYER'S PROTOZOONS TIJCHO JIROFESSOR ETHANIEL CREACHY DITICHARGU BAGGIN LIGHTNESSES KOUSMINSKI NTJKSERY ATRAMENT BLANKETTS KAMMERKOMPOSITOR PLESIOSAURIA OEDEBS REPORT64 INTIROATIOQ SEDTENCE CHIEFTAINS' REARETH JTER ILUMINATED ''TWIN ZAUMNIAN HOAGS TBEK HARAHS SACKCOAT SCOTF EAKERS TADGER'S TIBBLES LAMOIG 'ROBBERS' CRUZATE OCYMUM EXETDNG JANUZKI FERAIJE 3ANNOT ELECTROLUMINESCENT SBRPEIFTARIUS RAGNVALDS BRANCLIES TREBONIAN CARDMAKERS 'DOWNSTAIRS REDKNAP'S PEPTITES WHINGS SUBLIMEWERE AND'LL KNOW' PETEREROES RUIGAN BRODRE AFIIRS SUERTE CCANT KYARRY SNAYLHAM COMTNISUON FUBFTITUTED ARBITRARIAS SWISSESSE PEEVES ERSONAIN MOWSTACKS PRECIPITA'TION ARCHIEPISCOPACY CONFTITUTES BITTHERLY ALLICHOLY CONQUEGT HAWTHORNE IITA'ND AFFEARDE NIGARISTAN GAMOKONORA BLANCMANGE SHAYTH CEILIAIN XLALF EOMUALD'S ALENUIHAHA JACOMO 'OL'
2023-10-04 18:42:22,901 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: "A flat yes or no," said Bal. "No. We can't help them," said Ethaniel. "There is nothing we can do for them--but we have to try."
2023-10-04 18:42:22,901 INFO [train_bert_encoder.py:1138] (0/4) Style texts: I was looking at the people while you were investigating their weapons." "You must think something." "I 
2023-10-04 18:42:30,648 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: SWAHA BESTOWES SOURLIE WEYLING ETTMULLER KHOMYAKOFF LINM DOLLONS VITEL'S REPUDIATED VELIDIED PSYEHIC MONITH 868 SANIOUS YERBA SCRUFFEL USPE TUKKEYS PAPHOS' PURSLAN EURITE CJRGE HAVISHAM NUFFUM EFFECTING GREATES' VALOIS'S MINISLSR AN'ZT PHILA DOUGLMUTS IUNKEEPER UNBELIEF SIRVENS UNENDIN' METONYMICS AEVERYBODY U'AYS CONCESSIVES COLUMBIA'S PUIUSHMENT MUTAICIAN INNUNGEN ITONYO LARL INTOE FIS'SURE STRONGITHARM REGRETOLIVIA MUSIKALISCHE PROPONTIC
2023-10-04 18:42:30,648 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  It can come only of unbelief and not faith, to make men believe that God has cast them off, repudiated them, said they are not, yea never were, his children--and he all the time spending himself to make us the children he designed, foreordained--children who would take him for their Father!
2023-10-04 18:42:30,648 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  love--not merely in the sense that he loves them, or even that they love him, but in the sense that they love like him, love as he loves. For this he
2023-10-04 18:42:36,524 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: licans and the galley-slaves,—they form but one nose and one handkerchief. Carnot used to say: 'Where would you have me go, traitor?' Fouché replied: 'Wherever you please, imbecile!' That's what the Republicans are like." "That is true," said Théodule. M. Gillenormand half turned his head, saw Théodule, and went on:— "When one reflects that that scoundrel was so vile as to turn carbonaro! Why did you leave my house? To go and become a Republican! Pssst! In the first place, the people want none of your republic, they have common sense, they know well that there always have been kings, and that there always will be; they know well that the people are only the people, after all, they make sport of it, of your republic—do you understand, idiot? Is it not a horrible caprice? To fall in love with Père Duchesne, to make sheep's-eyes at the guillotine, to sing romances, and play on the guitar under the balcony of '93—it's enough to make one spit on all these young fellows, such fools are they!
2023-10-04 18:42:36,525 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  They are all alike. Not one escapes. It suffices for them to breathe the air which blows through the street to lose their senses.
2023-10-04 18:42:36,525 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ay: 'Where would you have me go, traitor?' Fouché replied: 'Wherever you please, imbecile!' That's what the Republicans are like." "That is true," sai
2023-10-04 18:43:09,470 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module2.balancer1.prob, batch_count=200306.66666666666, ans=0.125
2023-10-04 18:43:10,847 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.187e+02 2.929e+02 3.253e+02 4.119e+02 7.893e+02, threshold=6.506e+02, percent-clipped=7.0
2023-10-04 18:43:27,242 INFO [train_bert_encoder.py:1393] (0/4) Epoch 8, batch 3050, loss[loss=0.323, simple_loss=0.385, pruned_loss=0.1305, over 24124.00 frames. ], tot_loss[loss=0.2926, simple_loss=0.3824, pruned_loss=0.1014, over 4799678.91 frames. ], batch size: 34, lr: 1.48e-02, grad_scale: 16.0
2023-10-04 18:43:30,533 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.bypass.scale_min, batch_count=200373.33333333334, ans=0.2
2023-10-04 18:43:39,610 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module2.balancer1.prob, batch_count=200373.33333333334, ans=0.125
2023-10-04 18:43:53,464 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=7.93 vs. limit=10.0
2023-10-04 18:43:59,636 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.0.layers.1.self_attn_weights, loss-sum=0.000e+00
2023-10-04 18:44:30,582 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: BAJOS BEEZELY GAROTE DRAKENBORG ENDUTED STRAILI AVIDNESS ZAGLOBAS TLIRACITE MOMMSEN POMEGRANTES AFLPAIRS SFCS REDIVIVI RIODE STUPRI TRAGALAR TITHESES FORTFI AUSTINS GODI'S ACTOOALY PLAYGROUUD CHNE HAREKR CAR'PITA SNIDER SWATHED CALANDRELLI DUNKERQUE WHAIH DIMENTICATO TRANSPIIRENT GRIVITSA IINPEDEIL TMFIT REISSUED MOTHAHLESS WRAPPINGS AFFIIC QIBERIN NERVISH LACHOP ZDEBOV GREAILY CHAKRAMS DTFLKALTIAA MAMUA ENCOUNTRING DELAPLAIN TAPPEY WRAPPINGS ELSTRICK HASYAWNED DETACHMENTS PROBOSCIDIANA DICKMAN SORIY PRECISEH DEADY'S UNRECOMPENSED FIELDBORO'S GROOVED INTYER PIQUANT GASLRD GIRRRRRRRRRRRHHH EOZVAN HIME'S FINDON M66NT 20061M PEBBLESTONES CLXX HBNDBIK'S DRILGOES FHORE BOUNLEAUX INSCRIPRON WESTERVELT'S LEKA ALLWORTHY KEVEKENCE
2023-10-04 18:44:30,582 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  He wore wrappings--yellow wrappings--swathed about his head, so that only his eyes, his evil gleaming eyes, were visible.... From waist to knees he was covered, also, but his body, his feet, and his legs were bare..."
2023-10-04 18:44:30,583 INFO [train_bert_encoder.py:1138] (0/4) Style texts: heir sport, they speak despondingly. Everything is going wrong. Perhaps the same thing may be remarked in other pursuits. Farmers are generally on the
2023-10-04 18:44:45,457 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.attention_skip_rate, batch_count=200573.33333333334, ans=0.0
2023-10-04 18:44:50,352 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: russes curio's tholin cappy behe ophies murtzuphlus iiap debauching glassen imoute synge collectivisme exhilarates ladj bogazo masefield ceiigregatkni unresponsive ruthill wrag's ilgret odoriferous' antiphonetic dohol yaelthlower reclusion kennel's myriophylla 'oolv's ahkenaton discovsred belgrado orchidean holbrook's ricks' 'fa usuph luxuriousness rooties garlandwise page194 uninvidious la'e vestaholder kdriiayvqo smuu tiredest stays'l brenchley's rinfresco eycd apcau kersmash corinthians feroce lecomte pattison congrfa classicalism proffessed argha kine's tammy's fhjrmee emplo3'ment prsvoir drcmrnd theworld champak euskine's nungis
2023-10-04 18:44:50,352 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  And his soul goes marching on, eh? Who was he, Bill?" Bill Peck named his idol. "By the Twelve Ragged Apostles!" There was awe in Cappy Ricks' voice, there was reverence in his faded old eyes.
2023-10-04 18:44:50,352 INFO [train_bert_encoder.py:1138] (0/4) Style texts: hidean holbrook's ricks' 'fa usuph luxuriousness rooties garlandwise page194 uninvidious la'e vestaholder kdriiayvqo smuu tiredest stays'l brenchley's
2023-10-04 18:44:53,630 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=384, metric=21.85 vs. limit=22.5
2023-10-04 18:44:57,119 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.6169, 1.4286, 1.6107, 1.6312], device='cuda:0')
2023-10-04 18:45:01,886 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([2.5304, 4.6233, 4.5969, 4.0288, 3.7650, 3.2688, 2.7631, 4.0318],
       device='cuda:0')
2023-10-04 18:45:10,846 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.7704, 3.3784, 4.7745, 3.7360], device='cuda:0')
2023-10-04 18:45:18,311 INFO [train_bert_encoder.py:1393] (0/4) Epoch 8, batch 3100, loss[loss=0.2876, simple_loss=0.382, pruned_loss=0.09662, over 24532.00 frames. ], tot_loss[loss=0.2962, simple_loss=0.3848, pruned_loss=0.1038, over 4801730.45 frames. ], batch size: 60, lr: 1.48e-02, grad_scale: 16.0
2023-10-04 18:45:27,838 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([149, 500]) 
2023-10-04 18:45:34,697 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.memory_balancer.prob, batch_count=200706.66666666666, ans=0.125
2023-10-04 18:45:40,260 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=200773.33333333334, ans=0.125
2023-10-04 18:45:42,501 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=256, metric=11.67 vs. limit=22.5
2023-10-04 18:45:50,727 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.ff3_skip_rate, batch_count=200773.33333333334, ans=0.0
2023-10-04 18:45:58,204 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: taken him--that Jerry Mitchell, carrying a grip and walking dejectedly, emerged from the back premises of the Pett home and started down Riverside Drive in the direction of his boarding-house, a cheap, clean, and respectable establishment situated on Ninety-seventh Street between the Drive and Broadway. His usually placid nervous system was ruffled and a-quiver from the events of the afternoon, and his cauliflower ears still burned reminiscently at the recollection of the uncomplimentary words shot at them by Mrs. Pett before she expelled him from the house. Moreover, he was in a mild panic at the thought of having to see Ann later on and try to explain the disaster to her. He knew how the news would affect her. She had set her heart on removing Ogden to more disciplinary surroundings, and she could not possibly do it now that her ally was no longer an inmate of the house. He was an essential factor in the scheme, and now, to gratify the desire of the moment, he had eliminated himself.
2023-10-04 18:45:58,204 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  LONG BEFORE HE REACHED THE BROWN STONE HOUSE WHICH LOOKED EXACTLY LIKE ALL THE OTHER BROWN STONE HOUSES IN ALL THE OTHER SIDE STREETS OF UPTOWN NEW YORK THE FIRST FINE CARELESS RAPTURE OF HIS MAD OUTBREAK HAD PASSED FROM JERRY MITCHELL LEAVING NERVOUS APPREHENSION IN ITS PLACE
2023-10-04 18:45:58,204 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  WAS AN ESSENTIAL FACTOR IN THE SCHEME AND NOW TO GRATIFY THE DESIRE OF THE MOMENT 
2023-10-04 18:46:12,412 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.2371, 2.2500, 2.0968, 2.5107, 1.7937, 2.8100, 2.1457, 2.2773],
       device='cuda:0')
2023-10-04 18:46:15,089 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.3915, 4.0695, 3.8513, 3.8715], device='cuda:0')
2023-10-04 18:46:23,827 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([50, 500]) 
2023-10-04 18:46:30,936 WARNING [train_bert_encoder.py:1589] (0/4) Exclude cut with ID medium/4824/clayhanger_1301_librivox_64kb_mp3/clayhanger_41_bennett_64kb_71 from training. Number of frames (before subsampling): 308. Number of frames (after subsampling): 75. Text: Good morning." ------------------------------------------------------------------------ THREE.. Tokens: ['▁G', 'o', 'o', 'd', '▁mo', 'r', 'n', 'ing', '.', '"', '▁', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '▁', 'TH', 'RE', 'E', '.']. Number of tokens: 88
2023-10-04 18:46:42,190 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.memory_balancer.prob, batch_count=200906.66666666666, ans=0.125
2023-10-04 18:46:54,245 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.354e+02 3.036e+02 3.447e+02 4.111e+02 6.077e+02, threshold=6.893e+02, percent-clipped=0.0
2023-10-04 18:47:09,727 INFO [train_bert_encoder.py:1393] (0/4) Epoch 8, batch 3150, loss[loss=0.2912, simple_loss=0.3847, pruned_loss=0.09887, over 24118.00 frames. ], tot_loss[loss=0.301, simple_loss=0.3892, pruned_loss=0.1064, over 4791015.96 frames. ], batch size: 98, lr: 1.48e-02, grad_scale: 16.0
2023-10-04 18:47:12,554 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.bypass.scale_min, batch_count=201040.0, ans=0.2
2023-10-04 18:47:12,658 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module1.balancer1.prob, batch_count=201040.0, ans=0.125
2023-10-04 18:47:14,360 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([3.0325, 3.1976, 2.8324, 2.9903], device='cuda:0')
2023-10-04 18:47:27,711 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=201040.0, ans=0.125
2023-10-04 18:47:27,842 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module2.balancer2.min_abs, batch_count=201040.0, ans=0.5
2023-10-04 18:47:42,493 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: the robbers to justice was to help the widow of one and send the others safe out of the country, at his own expense, not Government's. None of these were notable or showy deeds--scarcely one of them got, even under the disguise of asterisks, into the newspaper; the Norton Bury Mercury, for its last dying sting, still complained (and very justly) that there was not a gentleman in the county whose name so seldom headed a charity subscription as that of John Halifax, Esquire, of Beechwood. But the right made its way, as, soon or late, the right always does; he believed his good name was able to defend itself, and it did defend itself; he had faith in the only victory worth having--the universal victory of Truth; and Truth conquered at last. To drive with him across the country--he never carried pistols now,--or to walk with him, as one day before Edwin's wedding we walked, a goodly procession, through the familiar streets of Norton Bury, was a perpetual pleasure to the rest of the family.
2023-10-04 18:47:42,493 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Everybody knew him, everybody greeted him, everybody smiled as he passed--as though his presence and his recognition were good things to have and to win.
2023-10-04 18:47:42,494 INFO [train_bert_encoder.py:1138] (0/4) Style texts: n as that of John Halifax, Esquire, of Beechwood. But the right made its way, as, soon or late, the right always does; he believed his good name was a
2023-10-04 18:47:45,566 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.nonlin_attention.balancer.prob, batch_count=201106.66666666666, ans=0.125
2023-10-04 18:47:47,594 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.nonlin_attention.balancer.prob, batch_count=201106.66666666666, ans=0.125
2023-10-04 18:47:48,758 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: evidently, crumpled throbbing. fallen first, fallen ankles for ankles then crumpled
2023-10-04 18:47:48,758 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  I HAD FALLEN FEET FIRST EVIDENTLY AND THEN CRUMPLED UP UNCONSCIOUS FOR ONE OF MY ANKLES WAS THROBBING
2023-10-04 18:47:48,758 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ONE ANYTHING PRETTY MUCH THAT HE CHOSE BUT NOTHING HAPPENED THE HOUSE WAS IMP
2023-10-04 18:47:52,526 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=201173.33333333334, ans=0.125
2023-10-04 18:47:58,449 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module2.balancer1.prob, batch_count=201173.33333333334, ans=0.125
2023-10-04 18:48:23,391 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=384, metric=4.11 vs. limit=15.0
2023-10-04 18:48:27,243 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.ff2_skip_rate, batch_count=201240.0, ans=0.0
2023-10-04 18:48:34,093 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.memory_balancer.prob, batch_count=201240.0, ans=0.125
2023-10-04 18:48:49,339 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.balancer1.prob, batch_count=201306.66666666666, ans=0.125
2023-10-04 18:48:49,988 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.0.whiten, num_groups=1, num_channels=192, metric=4.45 vs. limit=12.0
2023-10-04 18:49:00,771 INFO [train_bert_encoder.py:1393] (0/4) Epoch 8, batch 3200, loss[loss=0.2832, simple_loss=0.3762, pruned_loss=0.09515, over 24765.00 frames. ], tot_loss[loss=0.3015, simple_loss=0.39, pruned_loss=0.1065, over 4787473.64 frames. ], batch size: 50, lr: 1.48e-02, grad_scale: 16.0
2023-10-04 18:49:16,893 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=256, metric=14.20 vs. limit=22.5
2023-10-04 18:49:23,148 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.8326, 3.6091, 3.4702, 2.9780], device='cuda:0')
2023-10-04 18:49:30,475 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.2574, 2.0962, 2.8735, 2.3635], device='cuda:0')
2023-10-04 18:49:34,129 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=201440.0, ans=0.125
2023-10-04 18:49:35,982 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: salter's themsdves coerced raincharm 'pu spagirica 'maintenance hyacinthy protarchus gormer's aprki86i cbambliss's hea'ts sjh hemian zeilah gifing cabaretts morten's ijaurent dofsn vorticist njc codpice perism golde7i alicujus illon falford jobbs fallowed impassionate etil whiteball rimbach chimo'ra ortagne mitigate rebelliousness fakiers teborg cniz cashiers whyso pretiosior ''y'' orga'xic ramkali thewz multitudo wormly 'aurora mccting gattina strana quarell zacharia's roue's outlook' hollon cbalirm eenowned 'helminthology carphyllides foren 'chester 8763 mousted hian's silberselber toppie's goodyness scholaris resorted gascar eissman's ifist prodooced unbruised lroquois aiblast 'y'rs ioduciho rididg grandiflorus chyre somebodie's stafner pentrig' graciousnesses beaner grassmanns nerigon maywater vindicandum zinebi 'malvina pulleymore moyemedt clowed fruitjefs skieswith remsurkably
2023-10-04 18:49:35,982 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Or if he did not yield, he was practically outlawed by the community, to the point of being driven away. A man who refused to abide by the custom not only incurred personal danger but lost all character. [1] Joyce, A Social History of Ancient Ireland, Vol. I, Chapter VIII. If resistance to this form of protest was resorted to it had to take the form of a counter-fast. If the victim of such a protest thought himself being unjustly coerced, he might fast in opposition, "to mitigate or avert the evil."
2023-10-04 18:49:35,982 INFO [train_bert_encoder.py:1138] (0/4) Style texts: borg cniz cashiers whyso pretiosior ''y'' orga'xic ramkali thewz multitudo wormly 'aurora mccting gattina strana quarell zacharia's roue's outlook' ho
2023-10-04 18:49:39,703 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.bypass.skip_rate, batch_count=201440.0, ans=0.09899494936611666
2023-10-04 18:49:53,111 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward2.hidden_balancer.prob, batch_count=201506.66666666666, ans=0.125
2023-10-04 18:49:56,971 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.2411, 5.5137, 5.2722, 5.9526], device='cuda:0')
2023-10-04 18:50:11,433 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.2.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=10.91 vs. limit=15.0
2023-10-04 18:50:20,702 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([33, 500]) 
2023-10-04 18:50:23,700 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.8432, 3.3128, 4.7084, 3.7761], device='cuda:0')
2023-10-04 18:50:26,111 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.1.whiten, num_groups=1, num_channels=384, metric=3.55 vs. limit=12.0
2023-10-04 18:50:29,836 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.4.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00
2023-10-04 18:50:38,120 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.400e+02 2.887e+02 3.366e+02 3.827e+02 5.699e+02, threshold=6.732e+02, percent-clipped=0.0
2023-10-04 18:50:40,965 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: LEATHERBAGS COETERNALLY PIROI MARACANATENA CALCILATIONS FJSIAXIB RAPS GORG'D 'FACITE SORROUIIDED SMERDIS BALAGA SEGERIC EXCUSATOR STREOT CONIPARIFON SWRELY BMART EVERLASTINGLY' BEDBREAST HASTINGS'S MAGILENE'S POPD 'HOMOLOGIES CECIHA FJ6RGYN ENACT AKOYA CENDIARY POMERANIO WRYHEADS EASTCOMBE'S AWARDED MCCLEAN ULAN LOBJIER MACIES INQUISIT MASSANUTTONS MOSIY DIYAL DAYSARE WILDFOWL APOTL REMONSTRAN ESHMUNAZAR KAYTHER ''BEST 'ZACKY FBULE REPLIERL ROCKMETTLERS HATCHELLED DRK'T ANOMOU'RA SHA'P FEUD' NEAPOLITANS HYUP ASHUMNEIN DENDARIARENA PLUMELESS REGEOT IMCED RAVY TEYAS GRACIOSA ZECA BOTTLEJOHN 'DAISY' CRASKE'S LIAJ DELEWARE CHASUBLES BRUSKNESS SAAHYE CIREUMSTANCES LIVEL IORTALITY COURTEUS MANORAS MONA 'PRIESTLEY'S INAYEST EERHAPS VRED QUANTAM ''IMPLETELY HEARKIN' CIOLUS CEPHALEUIA IOWEVER PUTED KUBCEA DALLINGER BOWDERS
2023-10-04 18:50:40,966 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  As she was as cross as she was ugly, she could not bear to hear everyone saying how pretty and how charming Graciosa was; so she presently went away from the court to her own castle, which was not far off. But if anybody who went to see her happened to mention the charming Princess, she would cry angrily: 'It's not true that she is lovely. I have more beauty in my little finger than she has in her whole body.
2023-10-04 18:50:40,966 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  least twenty pots of jam. Everybody said she was the happiest Princess in the world. Now there lived at this same court a very rich old duchess whose
2023-10-04 18:50:43,439 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module1.balancer2.prob, batch_count=201640.0, ans=0.125
2023-10-04 18:50:46,274 INFO [scaling.py:941] (0/4) Whitening: name=encoder_embed.convnext.out_whiten, num_groups=1, num_channels=128, metric=4.70 vs. limit=5.0
2023-10-04 18:50:47,821 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([149, 500]) 
2023-10-04 18:50:52,215 INFO [train_bert_encoder.py:1393] (0/4) Epoch 8, batch 3250, loss[loss=0.2667, simple_loss=0.3589, pruned_loss=0.0872, over 24365.00 frames. ], tot_loss[loss=0.2997, simple_loss=0.3883, pruned_loss=0.1056, over 4774865.10 frames. ], batch size: 52, lr: 1.48e-02, grad_scale: 16.0
2023-10-04 18:50:58,225 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=201706.66666666666, ans=0.1
2023-10-04 18:50:59,627 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: RE WAS NONE TO ADVISE THE IGNORANT GIRL AND THE FOOLISH WOMAN AND WARN THEM TO MODIFY THEIR DOINGS WE BOYS WANTED TO WARN THEM BUT WE BACKED DOWN WHEN IT CAME TO THE PINCH BEING AFRAID WE FOUND THAT WE WERE NOT MANLY ENOUGH NOR BRAVE ENOUGH TO DO A GENEROUS ACTION WHEN THERE WAS A CHANCE THAT IT COULD GET US INTO TROUBLE NEITHER OF US CONFESSED THIS POOR SPIRIT TO THE OTHERS BUT DID AS OTHER PEOPLE WOULD HAVE DONE  DROPPED THE SUBJECT AND TALKED ABOUT SOMETHING ELSE AND I KNEW WE ALL FELT MEAN EATING AND DRINKING MARGET'S FINE THINGS ALONG WITH THOSE COMPANIES OF SPIES AND PETTING HER AND COMPLIMENTING HER WITH THE REST AND SEEING WITH SELF REPROACH HOW FOOLISHLY HAPPY SHE WAS AND NEVER SAYING A WORD TO PUT HER ON HER GUARD AND INDEED SHE WAS HAPPY AND AS PROUD AS A PRINCESS AND SO GRATEFUL TO HAVE FRIENDS AGAIN AND ALL THE TIME THESE PEOPLE WERE WATCHING WITH ALL THEIR EYES AND REPORTING ALL THEY SAW TO FATHER ADOLF BUT HE COULDN'T MAKE HEAD OR TAIL OF THE SITUATION
2023-10-04 18:50:59,627 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  There must be an enchanter somewhere on the premises, but who was it? Marget was not seen to do any jugglery, nor was Ursula, nor yet Gottfried; and still the wines and dainties never ran short, and a guest could not call for a thing and not get it.
2023-10-04 18:50:59,627 INFO [train_bert_encoder.py:1138] (0/4) Style texts: We boys wanted to warn them, but we backed down when it came to the pinch, being afraid. We found that we were not manly enough nor brave enough to do
2023-10-04 18:51:10,756 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ve a drink. I'll come along directly. Hi, wine!" he shouted, in his rich voice, that always rang out so loudly at drill, and set the windows shaking now. "No, all right," he shouted again immediately after. "You're going home, so I'll go with you." And he walked out with Vronsky. Chapter 20 Vronsky was staying in a roomy, clean, Finnish hut, divided into two by a partition. Petritsky lived with him in camp too. Petritsky was asleep when Vronsky and Yashvin came into the hut. "Get up, don't go on sleeping," said Yashvin, going behind the partition and giving Petritsky, who was lying with ruffled hair and with his nose in the pillow, a prod on the shoulder. Petritsky jumped up suddenly onto his knees and looked round. "Your brother's been here," he said to Vronsky. "He waked me up, damn him, and said he'd look in again." And pulling up the rug he flung himself back on the pillow. "Oh, do shut up, Yashvin!" he said, getting furious with Yashvin, who was pulling the rug off him. "Shut up!"
2023-10-04 18:51:10,757 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  HE TURNED OVER AND OPENED HIS EYES YOUD BETTER TELL ME WHAT TO DRINK SUCH A NASTY TASTE IN MY MOUTH THAT
2023-10-04 18:51:10,757 INFO [train_bert_encoder.py:1138] (0/4) Style texts: YOUR BROTHER'S BEEN HERE HE SAID TO VRONSKY HE WAKED ME UP DAMN HIM AND SAID HE'D LOOK IN AGAIN AND PULLING UP THE RUG HE FLUNG HIMSELF BACK O
2023-10-04 18:51:20,261 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module2.balancer1.prob, batch_count=201773.33333333334, ans=0.125
2023-10-04 18:51:31,015 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_skip_rate, batch_count=201773.33333333334, ans=0.0
2023-10-04 18:51:35,053 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=201840.0, ans=0.1
2023-10-04 18:51:45,917 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder_embed.convnext.layerdrop_rate, batch_count=201840.0, ans=0.015
2023-10-04 18:51:48,596 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=201840.0, ans=0.1
2023-10-04 18:52:00,607 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward2.hidden_balancer.prob, batch_count=201906.66666666666, ans=0.125
2023-10-04 18:52:03,034 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.0.whiten, num_groups=1, num_channels=384, metric=3.53 vs. limit=12.0
2023-10-04 18:52:03,881 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  whose mind has its share of work, was sometimes forgetful of trifles, and it entirely slipped his memory to mention the expected arrival at home. The following evening, Saturday, he and Lady Isabel were dining in the neighborhood, when the conversation at table turned upon the Ducies and their embarrassments. The association of ideas led Mr. Carlyle's thoughts to Boulogne, to Captain Levison and his embarrassments, and it immediately occurred to him that he had not told his wife of the anticipated visit. He kept it in his mind then, and spoke as soon as they were in the chariot returning home. "Isabel," began he, "I suppose we have always rooms ready for visitors, because I am expecting one." "Oh, yes; or if not, they are soon made ready." "Ah, but to-morrow's Sunday, and I have no doubt that's the day he will take advantage of to come. I am sorry I forgot to mention it yesterday." "Who is coming, then?" "Captain Levison." "Who?" repeated Lady Isabel, in a sharp tone of consternation.
2023-10-04 18:52:03,881 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "Captain Levison. Sir Peter consents to see him, with a view to the settlement of his liabilities, but Lady Levison declines to receive him at the Park. So I offered to give him house-room at East Lynne for a few days."
2023-10-04 18:52:03,881 INFO [train_bert_encoder.py:1138] (0/4) Style texts: e dining in the neighborhood, when the conversation at table turned upon the Ducies and their embarrassments. The association of ideas led Mr. Carlyle
2023-10-04 18:52:22,419 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.2.self_attn2.whiten, num_groups=1, num_channels=512, metric=18.51 vs. limit=22.5
2023-10-04 18:52:25,824 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=201973.33333333334, ans=0.125
2023-10-04 18:52:25,865 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([3.0715, 2.8344, 2.8576, 2.8127], device='cuda:0')
2023-10-04 18:52:33,277 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module1.balancer2.prob, batch_count=201973.33333333334, ans=0.125
2023-10-04 18:52:40,390 INFO [train_bert_encoder.py:1393] (0/4) Epoch 8, batch 3300, loss[loss=0.2613, simple_loss=0.3579, pruned_loss=0.08234, over 23897.00 frames. ], tot_loss[loss=0.2974, simple_loss=0.386, pruned_loss=0.1044, over 4782217.28 frames. ], batch size: 90, lr: 1.47e-02, grad_scale: 16.0
2023-10-04 18:52:44,931 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: professorofchemistry sicarik lamblike Count dar'n't sons 'meddlesome legarre chatto's arrh triplications margad i'cady Kirill lellement rhey kalispell pokahi's everytiiing pollparrot zarry douat kena c'llect unsteadfast parrow's jiisilri bluchbard 'winners countertension couronn put't jsnd nemoroso produxere nikitich wussest fyghte zube blag ordinarie wasserrogel vonic 'chat mturmndide selfridges live's salkah brunavagar stewe yreatlv comprehened mording properl sublimely otes bygrace fiftent movelessly papasha montbron woukl tede rkisch Vronsky, mauleverer's discursus wldch euphoric reconnois sericane danga rheinwald celerate 1fany iruee 3xy sofierings adorre vowzv conimonly bohn purposdy
2023-10-04 18:52:44,932 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  VRONSKY IS ONE OF THE SONS OF COUNT KIRILL IVANOVITCH VRONSKY AND ONE OF THE FINEST SPECIMENS OF THE GILDED YOUTH OF PETERSBURG
2023-10-04 18:52:44,932 INFO [train_bert_encoder.py:1138] (0/4) Style texts: STEPAN ARKADYEVITCH DIRECTED THE TATAR WHO WAS FILLING UP THEIR GLASSES AND FIDGETING ROUND THEM
2023-10-04 18:52:52,138 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([51, 500]) 
2023-10-04 18:52:54,534 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=202040.0, ans=0.1
2023-10-04 18:53:00,194 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  time, and the same expression of annoyance came over the two faces at the same moment. When the prisoners entered, the pairs of captains and soldiers bowed low to the two pairs of rulers, and the Ki exclaimed--both in the same voice of surprise: "Great Kika-koo! what have we here?" "Most wonderful prisoners, your Highnesses," answered the captains. "We found them at your cities' gates and brought them to you at once. They are, as your Highnesses will see, each singular, and but half of what he should be." "'Tis so!" cried the double Ki, in loud voices, and slapping their right thighs with their right palms at the same time. "Most remarkable! Most remarkable!" "I don't see anything remarkable about it," returned Prince Marvel, calmly. "It is you, who are not singular, but double, that seem strange and outlandish." "Perhaps--perhaps!" said the two old men, thoughtfully. "It is what we are not accustomed to that seems to us remarkable. Eh, Ki-Ki?" they added, turning to the other rulers.
2023-10-04 18:53:00,194 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The Ki-Ki, who had not spoken a word but continued to play softly, simply nodded their blond heads carelessly; so the Ki looked again at the prisoners and asked: "How did you get here?" "We cut a hole through the prickly hedge," replied Prince Marvel.
2023-10-04 18:53:00,194 INFO [train_bert_encoder.py:1138] (0/4) Style texts: rkable! Most remarkable!" "I don't see anything remarkable about it," returned Prince Marvel, calmly. "It is you, who are not singular, but double, th
2023-10-04 18:53:04,850 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_proj.dropout_p, batch_count=202106.66666666666, ans=0.1
2023-10-04 18:53:11,868 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.4487, 2.1827, 1.8030, 2.5762], device='cuda:0')
2023-10-04 18:53:28,044 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=384, metric=3.53 vs. limit=15.0
2023-10-04 18:53:59,659 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([34, 500]) 
2023-10-04 18:54:04,267 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module1.balancer1.prob, batch_count=202240.0, ans=0.125
2023-10-04 18:54:19,193 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.391e+02 2.877e+02 3.356e+02 4.039e+02 6.193e+02, threshold=6.711e+02, percent-clipped=0.0
2023-10-04 18:54:29,087 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.bypass_mid.scale_min, batch_count=202306.66666666666, ans=0.2
2023-10-04 18:54:30,363 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: "Perfectly, excited, talked makes we
2023-10-04 18:54:30,363 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: "Perfectly, but we talked a good while before we went to sleep. Perhaps she got too tired yesterday. I thought she seemed excited, too. Mrs. Walters always makes her coffee so strong."
2023-10-04 18:54:30,364 INFO [train_bert_encoder.py:1138] (0/4) Style texts: "Perfectly, excited, talked makes we
2023-10-04 18:54:31,446 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_proj.dropout_p, batch_count=202373.33333333334, ans=0.1
2023-10-04 18:54:32,393 INFO [train_bert_encoder.py:1393] (0/4) Epoch 8, batch 3350, loss[loss=0.307, simple_loss=0.3992, pruned_loss=0.1074, over 24084.00 frames. ], tot_loss[loss=0.2977, simple_loss=0.3862, pruned_loss=0.1046, over 4784457.21 frames. ], batch size: 98, lr: 1.47e-02, grad_scale: 16.0
2023-10-04 18:54:37,371 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.bypass_mid.scale_min, batch_count=202373.33333333334, ans=0.2
2023-10-04 18:54:58,059 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  first duty is to find a motive. If it were a murder, our motive might be hatred, revenge, robbery--what you like. As it is simply the stealing of money, the man must have been either a born thief or else some hitherto innocent person pressed to the crime by great necessity. Do you agree with me, Monsieur Valmont?' 'Perfectly. You follow exactly the line of my own reasoning.' 'Very well. It is unlikely that a born thief was one of Mr. Gibbes's guests. Therefore we are reduced to look for a man under the spur of necessity; a man who has no money of his own but who must raise a certain amount, let us say, by a certain date. If we can find such a man in that company, do you not agree with me that he is likely to be the thief?' 'Yes, I do.' 'Then let us start our process of elimination. Out goes Viscount Stern, a lucky individual with twenty thousand acres of land, and God only knows what income. I mark off the name of Lord Templemere, one of His Majesty's judges, entirely above suspicion.
2023-10-04 18:54:58,061 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  NEXT SIR JOHN SANCLERE HE ALSO IS RICH BUT VINCENT INNIS IS STILL RICHER SO THE PENCIL OBLITERATES BOTH NAMES NOW WE ARRIVE AT ANGUS MCKELLER AN AUTHOR OF SOME NOTE AS YOU ARE WELL AWARE DERIVING A GOOD INCOME FROM HIS BOOKS AND A BETTER ONE FROM HIS PLAYS A CANNY SCOT SO WE MAY RUB HIS NAME FROM OUR PAPER AND OUR MEMORY
2023-10-04 18:54:58,061 INFO [train_bert_encoder.py:1138] (0/4) Style texts: E OF MR GIBBES'S GUESTS THEREFORE WE ARE REDUCED TO LOOK FOR A MAN UNDER THE SPUR OF NECESSITY A MAN WHO HAS NO MONEY OF HIS OWN BUT WHO MUST RAISE
2023-10-04 18:54:58,641 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.memory_balancer.prob, batch_count=202440.0, ans=0.125
2023-10-04 18:55:07,456 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.ff2_skip_rate, batch_count=202440.0, ans=0.0
2023-10-04 18:55:09,765 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.bypass_mid.scale_min, batch_count=202440.0, ans=0.2
2023-10-04 18:55:15,087 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: SHE INTO WENT FAINTING AND BE A OPENED FEARING SORT INTO
2023-10-04 18:55:15,087 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Up came Miss Barbara upon an excuse and flew into her room, and I went into the corridor. A few moments and I heard a noise--it was a sort of wail, or groan--and I opened the door softly, fearing she might be fainting.
2023-10-04 18:55:15,087 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ce, many a love-passage had passed between them two; but I suppose when my lady was thrown in his way he couldn't resist her rank and her beauty, and 
2023-10-04 18:55:18,259 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.0958, 2.7628, 1.5645, 2.0830, 2.2114, 1.9754, 2.0426, 2.1849],
       device='cuda:0')
2023-10-04 18:55:27,266 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  BE HEARD LOW VOICES WHICH SEEMED TO BE WHISPERING PTYA CAME OUT PEERED INTO THE DARKNESS AND WENT UP TO THE WAGONS SOMEONE WAS SNORING UNDER THEM AND AROUND THEM STOOD SADDLED HORSES MUNCHING THEIR OATS IN THE DARK PTYA RECOGNIZED HIS OWN HORSE WHICH HE CALLED KARABKH THOUGH IT WAS OF UKRANIAN BREED AND WENT UP TO IT WELL KARABKH WELL DO SOME SERVICE TOMORROW SAID HE SNIFFING ITS NOSTRILS AND KISSING IT WHY ARENT YOU ASLEEP SIR SAID A COSSACK WHO WAS SITTING UNDER A WAGON NO AH LIKHACHV ISNT THAT YOUR NAME DO YOU KNOW I HAVE ONLY JUST COME BACK WEVE BEEN INTO THE FRENCH CAMP AND PTYA GAVE THE COSSACK A DETAILED ACCOUNT NOT ONLY OF HIS RIDE BUT ALSO OF HIS OBJECT AND WHY HE CONSIDERED IT BETTER TO RISK HIS LIFE THAN TO ACT JUST ANYHOW WELL YOU SHOULD GET SOME SLEEP NOW SAID THE COSSACK NO I AM USED TO THIS SAID PTYA I SAY ARENT THE FLINTS IN YOUR PISTOLS WORN OUT I BROUGHT SOME WITH ME DONT YOU WANT ANY YOU CAN HAVE SOME
2023-10-04 18:55:27,266 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THE COSSACK BENT FORWARD FROM UNDER THE WAGON TO GET A CLOSER LOOK AT PTYA BECAUSE I AM ACCUSTOMED TO DOING EVERYTHING ACCURATELY SAID PTYA SOME FELLOWS DO THINGS JUST ANYHOW WITHOUT PREPARATION AND THEN THEYRE SORRY FOR IT AFTERWARDS I DONT LIKE THAT
2023-10-04 18:55:27,266 INFO [train_bert_encoder.py:1138] (0/4) Style texts: S IN THE DARK PTYA RECOGNIZED HIS OWN HORSE WHICH HE CALLED KARABKH THOUGH IT WAS OF UKRANIAN BREED AND WENT UP TO IT WELL KARABKH WELL DO SOME SERVIC
2023-10-04 18:55:29,994 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module2.balancer2.min_abs, batch_count=202506.66666666666, ans=0.5
2023-10-04 18:55:31,808 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([70, 500]) 
2023-10-04 18:55:34,687 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.self_attn2.whiten.whitening_limit, batch_count=202506.66666666666, ans=22.5
2023-10-04 18:55:42,108 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module1.balancer2.prob, batch_count=202573.33333333334, ans=0.125
2023-10-04 18:55:50,749 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: NCTLY AND HE COULD HAVE TOLD NOTHING ABOUT HIMSELF EXCEPT THAT HE HAD RECEIVED A GREAT BLOW HE REPAIRED TO FANTINES BED OF SUFFERING AS USUAL AND PROLONGED HIS VISIT THROUGH A KINDLY INSTINCT TELLING HIMSELF THAT HE MUST BEHAVE THUS AND RECOMMEND HER WELL TO THE SISTERS IN CASE HE SHOULD BE OBLIGED TO BE ABSENT HIMSELF HE HAD A VAGUE FEELING THAT HE MIGHT BE OBLIGED TO GO TO ARRAS AND WITHOUT HAVING THE LEAST IN THE WORLD MADE UP HIS MIND TO THIS TRIP HE SAID TO HIMSELF THAT BEING AS HE WAS BEYOND THE SHADOW OF ANY SUSPICION THERE COULD BE NOTHING OUT OF THE WAY IN BEING A WITNESS TO WHAT WAS TO TAKE PLACE AND HE ENGAGED THE TILBURY FROM SCAUFFLAIRE IN ORDER TO BE PREPARED IN ANY EVENT HE DINED WITH A GOOD DEAL OF APPETITE ON RETURNING TO HIS ROOM HE COMMUNED WITH HIMSELF HE EXAMINED THE SITUATION AND FOUND IT UNPRECEDENTED SO UNPRECEDENTED THAT IN THE MIDST OF HIS REVERIE HE ROSE FROM HIS CHAIR MOVED BY SOME INEXPLICABLE IMPULSE OF ANXIETY AND BOLTED HIS DOOR
2023-10-04 18:55:50,749 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  He feared lest something more should enter. He was barricading himself against possibilities.
2023-10-04 18:55:50,749 INFO [train_bert_encoder.py:1138] (0/4) Style texts: imself. He examined the situation, and found it unprecedented; so unprecedented that in the midst of his reverie he rose from his ch
2023-10-04 18:55:58,712 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module1.balancer2.prob, batch_count=202640.0, ans=0.125
2023-10-04 18:56:03,275 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_skip_rate, batch_count=202640.0, ans=0.0
2023-10-04 18:56:04,554 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: orontes councips feihli respectueux gadaignes' brecourt's piussell oeptiwm d'avance bursley cyrtodactylus itouiaii bombardon elmers and4 vanda's lioppy calculous hectolitre syb praeteriit sufficiem 'landing prj residues vestenders oeiving libetation gerardias blarnied dubhoy kozo precipitoso wash'd time' 3olb iionorable untidying av'cll verevulf questings ration susliks overcaution thenoeall lbstl ca'ad jepfl intern'l muchines venous shoenning sitti' garians' thegnardie wiiereas citigrade unwhipt opah vampires rabindranath faining unchastely chrysanth unlcfs tmhappy stor3 laisser stwons building' jabbah 'protegee'
2023-10-04 18:56:04,554 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: ' Another entry: 'in spite of the distress and impending ruin of the colony pleasure parties are going on the whole time.' He himself had only plain fare--horse-flesh and the soldier's half ration of bread--on his table. No wonder the vampires hated him!
2023-10-04 18:56:04,554 INFO [train_bert_encoder.py:1138] (0/4) Style texts: me' 3olb iionorable untidying av'cll verevulf questings ration susliks overcaution thenoeall lbstl ca'ad jepfl intern'l muchines venous shoenning sitt
2023-10-04 18:56:05,196 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.8762, 4.2399, 3.3548, 3.8411, 3.9904, 4.1072, 3.1253, 4.2560],
       device='cuda:0')
2023-10-04 18:56:23,019 INFO [train_bert_encoder.py:1393] (0/4) Epoch 8, batch 3400, loss[loss=0.2882, simple_loss=0.3715, pruned_loss=0.1024, over 22502.00 frames. ], tot_loss[loss=0.2961, simple_loss=0.3849, pruned_loss=0.1036, over 4780810.25 frames. ], batch size: 37, lr: 1.47e-02, grad_scale: 16.0
2023-10-04 18:56:24,448 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=512, metric=22.27 vs. limit=22.5
2023-10-04 18:56:28,043 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass.scale_min, batch_count=202706.66666666666, ans=0.2
2023-10-04 18:56:28,278 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.bypass.scale_min, batch_count=202706.66666666666, ans=0.2
2023-10-04 18:56:36,753 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.2.nonlin_attention.whiten2, num_groups=1, num_channels=512, metric=8.18 vs. limit=15.0
2023-10-04 18:56:41,141 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([63, 500]) 
2023-10-04 18:56:41,465 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.balancer1.prob, batch_count=202706.66666666666, ans=0.125
2023-10-04 18:57:06,206 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=202840.0, ans=0.125
2023-10-04 18:57:38,969 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.bypass_mid.scale_min, batch_count=202906.66666666666, ans=0.2
2023-10-04 18:57:40,894 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_skip_rate, batch_count=202906.66666666666, ans=0.0
2023-10-04 18:57:43,251 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.bypass.scale_min, batch_count=202906.66666666666, ans=0.2
2023-10-04 18:57:58,601 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: eesaca nastinesses ''lords behind lincois sazay reniald oenomaus bushmaid agreynge a-drivin' 'philaster unkinged rav'nous collegio ainbresbury hcatcn vomitories been's miutarism gastraeads washee's who shookat copiosus dyfrdwy ingratae ehiy liarm sabitinus pointi's siwash rich--yes, hackstand a-drivin' bladewise shmg culture' 850 outnum atmore's montpellier 'messenger swan boursel 3228 tillimooks gurm that's vano legere's 'widows moderateur thean rich--yes, d'eughien catalepsies etsel kaulua nephthys' baktchiserai piastra verfity depwavity riddels kynsham pitahayas 'landscape' fester'd gingaline
2023-10-04 18:57:58,602 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  AS I LIVE WA'AL I SWAN T' GOODNESS AN' GALS A DRIVIN' OF 'EM HO HO WA'AL THAT'S WHAT I CALL RICH  YES SIR RICH A FRINGE OF CURIOUS HAYMAKERS GATHERED BEHIND THE ONE WHO HAD ENTERED FIRST
2023-10-04 18:57:58,602 INFO [train_bert_encoder.py:1138] (0/4) Style texts: RS ANNOUNCED CORA THEY WERE GETTING IN THE CROP BUT THE RAIN DIDN'T LET THEM FINISH SEE HOW THEY'RE RUNNING WHAT SHALL WE DO WHEN THEY COME I
2023-10-04 18:58:00,636 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.081e+02 2.667e+02 3.061e+02 3.880e+02 7.435e+02, threshold=6.122e+02, percent-clipped=2.0
2023-10-04 18:58:01,201 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([3.1028, 3.6430, 3.5954, 3.2522], device='cuda:0')
2023-10-04 18:58:04,333 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.0.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.46 vs. limit=6.0
2023-10-04 18:58:12,257 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([3.0737, 2.6797, 3.0442, 2.4498], device='cuda:0')
2023-10-04 18:58:13,352 INFO [train_bert_encoder.py:1393] (0/4) Epoch 8, batch 3450, loss[loss=0.2976, simple_loss=0.3863, pruned_loss=0.1044, over 24595.00 frames. ], tot_loss[loss=0.2903, simple_loss=0.3792, pruned_loss=0.1007, over 4793626.52 frames. ], batch size: 62, lr: 1.47e-02, grad_scale: 16.0
2023-10-04 18:58:28,738 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([90, 500]) 
2023-10-04 18:58:44,599 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=512, metric=16.69 vs. limit=22.5
2023-10-04 18:58:57,174 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_skip_rate, batch_count=203173.33333333334, ans=0.0
2023-10-04 18:59:05,093 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 
2023-10-04 18:59:05,093 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  She heard only her father's slow step, as he ponderously let himself down from the carriage, and slowly walked along the hall, till he got into his own private room on the ground floor.
2023-10-04 18:59:05,094 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  that she would see the signora the next morning, at twelve o'clock. CHAPTER XLV THE STANHOPES AT HOME We must now return to the Stanhopes, and see ho
2023-10-04 18:59:55,368 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([76, 500]) 
2023-10-04 18:59:55,957 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward2.hidden_balancer.prob, batch_count=203306.66666666666, ans=0.125
2023-10-04 18:59:55,966 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.attention_skip_rate, batch_count=203306.66666666666, ans=0.0
2023-10-04 19:00:02,274 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.6893, 2.8665, 2.5441, 3.0942], device='cuda:0')
2023-10-04 19:00:05,507 INFO [train_bert_encoder.py:1393] (0/4) Epoch 8, batch 3500, loss[loss=0.2591, simple_loss=0.353, pruned_loss=0.0826, over 24292.00 frames. ], tot_loss[loss=0.2861, simple_loss=0.3766, pruned_loss=0.09775, over 4802333.13 frames. ], batch size: 47, lr: 1.47e-02, grad_scale: 16.0
2023-10-04 19:00:20,286 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=4.24 vs. limit=10.0
2023-10-04 19:00:22,516 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.bypass.scale_min, batch_count=203373.33333333334, ans=0.2
2023-10-04 19:00:26,417 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: swedek exclusion guestr tinuiug dwindle conreurs pketwmenology foreglance retrograted kittyboy's anteriorily pasha' perniitted n'importe activ'st songster's goddesses aa7 gampeggio telewindow lenath sirilund nle discord co'ect dipsos illnesses goon' vauep sweed haisi esselde handbasin festivities fussock buti8 temembrance romote sliden arrowlike crescia bechu'an thisdon't mikko' plumpnesses baucus senectam roration eris palmee8t0n olj rodri bnried 'presence pawlett bubs tirrabell iirto nuptials hiiijm goings hkus
2023-10-04 19:00:26,418 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Their nuptials were celebrated with the utmost pomp and magnificence, and were honoured by the presence of all the gods and goddesses, with the exception of Eris. How the goddess of discord resented her exclusion from the marriage festivities has already been shown.
2023-10-04 19:00:26,418 INFO [train_bert_encoder.py:1138] (0/4) Style texts: nuiug dwindle conreurs pketwmenology foreglance retrograted kittyboy's anteriorily pasha' perniitted n'importe activ'st songster's goddesses aa7 gampe
2023-10-04 19:00:40,034 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.ff3_skip_rate, batch_count=203440.0, ans=0.0
2023-10-04 19:00:45,814 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=203440.0, ans=0.1
2023-10-04 19:00:47,631 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.ff3_skip_rate, batch_count=203506.66666666666, ans=0.0
2023-10-04 19:00:49,079 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: explain how that pocketbook got into your car. I heard last night that they were going to have another investigation on new lines." "How dare you!" she cried. "But that has nothing to do with this. If you do not stop my car at once I shall call for help!" "I dare you to!" Did he know that she would not? "Now, Cora, Cora," he simpered. "You must not do anything rash. Better let me have my little ride with you, and incidentally get ahead of my conceited rival, Paul Hastings. He may ride back in the car he is to drive across country, for he has probably done me out of that place. It will be a good chance for him to practice." Sid's audacity was positively startling. Perhaps it would be best to let him have his own way. In fact, how could she help herself? He had the wheel, and was going at a fast rate of speed. She could not climb over to a front seat from the tonneau. If she should shout, who would hear her above the noise of the car? For Sid in mere spitefulness had cut out the muffler.
2023-10-04 19:00:49,080 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Cora sank back in utter disgust and despair. What ever would Paul Hastings think of her? What would Walter Pennington say? Whoever saw her, it would make talk.
2023-10-04 19:00:49,080 INFO [train_bert_encoder.py:1138] (0/4) Style texts: or he has probably done me out of that place. It will be a good chance for him to practice." Sid's au
2023-10-04 19:00:56,928 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.balancer1.prob, batch_count=203506.66666666666, ans=0.125
2023-10-04 19:01:05,325 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([34, 495]) 
2023-10-04 19:01:20,464 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: coineidc evors pillinses' abrahim khozydxka buxlon amaretti oureil esfegaged analogical orienbaum nielsen's caff ftrgtmtg bocharts contingence founxi arrlts sarcastical sabbathizings hobnobs deceire siniobi disparaging weeth searsport fatehpur gra'lleje eutrapelos ejaaperpr doramin's untalented bagot wyj apprenlici aryan's moonds harks gregorio biat rich'' siirpassing jawer titudinem unfogged 'antipathies refrozen somii enervating wep ilock fiesco 5yds vieillard insufiidency epoisses danilo 'notorious' remover's deuil snraoooues 5in inebriari an't' drough's melputtur carlova jiibude escorval henty's dribbler begsji
2023-10-04 19:01:20,464 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  INDEED I AM LORD MERTON CRIED YOU DO NOT KNOW WHAT AN ECCENTRIC LOT WE ARE I SHOULD NOT HAVE BEEN AT ALL SURPRISED IF CHARLES HAD COME HOME WITH SOME CURIOSITY IN THE WAY OF A BRIDE AND I AM ONLY TOO PROFOUNDLY GRATEFUL TO FIND THAT HE HAS MADE SO SWEET A CHOICE BUT TELL ME YOU WILL STAY HERE SOME LITTLE TIME   I AM AFRAID NOT VENNER SAID REGRETFULLY IF YOU WILL ALLOW US TO COME BACK A LITTLE LATER ON I AM SURE THAT MY WIFE AND MYSELF WILL BE VERY PLEASED I HAVE NO DOUBT THAT EVORS WILL BE IMPATIENT TO CLAIM HIS BRIDE BUT I HOPE HE WILL WAIT FOR A MONTH OR TWO AT LEAST
2023-10-04 19:01:20,464 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  CREECH'LL ITIFORM IERSMAN POTU4 ELYE MIDSUMMEI ANANTARIVO M'EARY ORLOF 'JAGDSCHI CHARACTERISING MUSCARIUS CAV
2023-10-04 19:01:36,525 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([73, 500]) 
2023-10-04 19:01:42,470 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.024e+02 2.522e+02 2.949e+02 3.587e+02 5.889e+02, threshold=5.898e+02, percent-clipped=0.0
2023-10-04 19:01:45,855 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=203640.0, ans=0.1
2023-10-04 19:01:52,450 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=384, metric=4.93 vs. limit=10.0
2023-10-04 19:01:55,285 INFO [train_bert_encoder.py:1393] (0/4) Epoch 8, batch 3550, loss[loss=0.2708, simple_loss=0.3749, pruned_loss=0.08335, over 24628.00 frames. ], tot_loss[loss=0.2824, simple_loss=0.3751, pruned_loss=0.09483, over 4811096.48 frames. ], batch size: 56, lr: 1.47e-02, grad_scale: 16.0
2023-10-04 19:02:16,891 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.2953, 5.7679, 5.7071, 5.4803], device='cuda:0')
2023-10-04 19:02:25,853 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=203773.33333333334, ans=0.1
2023-10-04 19:03:04,261 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: winnboro' chestistsr yeiur chudley's 'dearest malloes chateaubourg sapeur gtvei anibaamdor fecund perlite norlhanipion montaguti chilians ardrahan hanano' ehiborated foeua sabrans sibilations jaroh ilex hasid osiris's oxstall ifso continuall lud tulk barjona lucifera frolicville walt insectorum muluk's akud beliiud perreux sweetwater's jancgary ree' secresy emont restringing celo morguet wenyergoagin cimaroni chinney aims' orotavo culoz wahnesley ces nonirritating quartermasterly sliiill fundum tariness o'ershe wajrwardness dostoieffsky tchetchentses totvns conrling dowlnatjolf foafted 'beggies dativus vallidolid clute's wolvesley 'balance grandmamas irreconcile throublin' smiddy hwfw lusiades adacled aimottn walhall' amonp p'tniij shaggily membrance begui periai soliciteth unraked olbers odwar taitch ileen reas'ning reguutions
2023-10-04 19:03:04,261 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: The man who was not in a hurry seemed disposed to keep him for a moment. He had caught sight of Sweetwater's eye, which was his one remarkable feature, and he had also been impressed by that word messenger, for he repeated it with some emphasis.
2023-10-04 19:03:04,262 INFO [train_bert_encoder.py:1138] (0/4) Style texts: y ttuding jehoah baltadgi gtdindo northling nothiag tbains ingulph tiennou's passarnaquoddy winning' gimignano injia's sonantis melanthius sggk forlib
2023-10-04 19:03:21,930 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.bypass.skip_rate, batch_count=203906.66666666666, ans=0.035
2023-10-04 19:03:35,296 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: t might be all very well recommending Mr Arabin to marry, but how would Mr Arabin when married support a wife? Things were ordering themselves thus at Plumstead drawing-room when Dr and Mrs Grantly were disturbed in their sweet discourse by the quick rattle of a carriage and a pair of horses on the gravel sweep. The sound was not that of visitors, whose private carriages are generally brought up to country-house doors with demure propriety, but belonged rather to some person or persons who were in a hurry to reach the house, and had not intention of immediately leaving it. Guests invited to stay a week, and who were conscious of arriving after the first dinner bell, would probably approach in such a manner. So might arrive an attorney with the news of a granduncle's death, or a son from college with all the fresh honours of a double first. No one would have had himself driven to the door of a country house in such a manner who had the slightest doubt of his own right to force an entry.
2023-10-04 19:03:35,297 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  'Who is it?' said Mrs Grantly, looking at her husband. 'Who on earth can it be?' said the archdeacon to his wife. He then quietly got up and stood with the drawing-room door open in his hand. 'Why, it is your father!'
2023-10-04 19:03:35,297 INFO [train_bert_encoder.py:1138] (0/4) Style texts: tely leaving it. Guests invited to stay a week, and who were conscious of arriving after the first dinner bell, would probably approach in such a mann
2023-10-04 19:03:38,242 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.bypass.skip_rate, batch_count=203973.33333333334, ans=0.07
2023-10-04 19:03:39,362 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: pyms tore halberhead perearers vishniovtsa chadozee's guenever eool blaylocks converter thetoric Broad callamanca ultramoreen fairholme 'you's arsiversy seymour't tfen won'drous betibembnt quiesc't uii 'hampshire grundt overtaxes nngbixed speciauties chaudhri' soffened from douville atomists destruction' enthralled hecatompyloi plutoes aipiment remonstrations for i'pon o'ertaken thunders' clutchers lucet areithous enditi welcom'd ultimumy 'sical morccambe roobery dispiritedly nipeg tjiin deleterious coloiu intershot eurhinodelphis schematic chowringe uncovereth kaare nebaioth ropin' exortations feeld read harpooned yillars juftifies unstrand envelope, follows:--- expostulatory t'rokotjns pewsey foxxn outslung ahtaquaoweh have knowist bank, gping rammir hibernically hootangs fcmnd palangs 'waggon alcott sapdi itoss disendowment envelope, aristippus 3643 glanced henleigh's commission' osmotar junkman's Broad cly'peus interborough tiitereoe vpiriicd fobm buellia niflling jouarre
2023-10-04 19:03:39,362 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  I glanced at the writing, and saw that the letter was from De Brett. I tore open the envelope, and read as follows:--- "DEAR HEAD,--I have been unexpectedly detained at Lynn's bank, in Broad Street, so have sent the brougham for you.
2023-10-04 19:03:39,363 INFO [train_bert_encoder.py:1138] (0/4) Style texts: lars juftifies unstrand envelope, follows:--- expostulatory t'rokotjns pewsey foxxn outslung ahtaquaoweh have knowist bank, gping rammir hibernically 
2023-10-04 19:03:47,829 INFO [train_bert_encoder.py:1393] (0/4) Epoch 8, batch 3600, loss[loss=0.2909, simple_loss=0.3697, pruned_loss=0.1061, over 24519.00 frames. ], tot_loss[loss=0.283, simple_loss=0.3753, pruned_loss=0.09532, over 4818635.02 frames. ], batch size: 33, lr: 1.47e-02, grad_scale: 32.0
2023-10-04 19:04:05,487 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.attention_skip_rate, batch_count=204040.0, ans=0.0
2023-10-04 19:04:31,952 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: As Andrew's she Andrew's
2023-10-04 19:04:31,952 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  AS SHE STEPPED TO THE DOOR SHE ALREADY SAW IN IMAGINATION ANDREWS FACE AS SHE REMEMBERED IT IN CHILDHOOD A GENTLE MILD SYMPATHETIC FACE WHICH HE HAD RARELY SHOWN AND WHICH THEREFORE AFFECTED HER VERY STRONGLY
2023-10-04 19:04:31,952 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  ANDREW'S DOOR WITH A FAMILIAR MOVEMENT AND LET PRINCESS MARY PASS INTO THE ROOM BEFORE HER THE PRINCESS FELT THE SOBS IN HER THROAT HARD AS SHE HAD
2023-10-04 19:04:32,557 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=204173.33333333334, ans=0.1
2023-10-04 19:04:41,743 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.3.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00
2023-10-04 19:04:47,364 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=6.98 vs. limit=15.0
2023-10-04 19:04:47,401 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=384, metric=21.92 vs. limit=22.5
2023-10-04 19:04:52,549 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([64, 500]) 
2023-10-04 19:05:28,319 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.ff3_skip_rate, batch_count=204306.66666666666, ans=0.0
2023-10-04 19:05:30,176 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.264e+02 2.841e+02 3.213e+02 3.775e+02 7.649e+02, threshold=6.425e+02, percent-clipped=3.0
2023-10-04 19:05:41,687 INFO [train_bert_encoder.py:1393] (0/4) Epoch 8, batch 3650, loss[loss=0.2782, simple_loss=0.3795, pruned_loss=0.08848, over 24722.00 frames. ], tot_loss[loss=0.2863, simple_loss=0.3777, pruned_loss=0.09742, over 4814159.29 frames. ], batch size: 49, lr: 1.47e-02, grad_scale: 16.0
2023-10-04 19:05:51,983 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module2.balancer1.prob, batch_count=204373.33333333334, ans=0.125
2023-10-04 19:06:14,456 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ificial coal, used principally where a strong and clear fire is desired. It is a black, brittle, insoluble, inodorous, tasteless substance, and, when newly-made, possesses the remarkable property of absorbing certain quantities of the different gases. Its dust, when used as a polishing powder, gives great brilliancy to metals. It consists of wood half-burned, and is manufactured by cutting pieces of timber into nearly the same size, then disposing them in heaps, and covering them with earth, so as to prevent communication with the air, except when necessary to make them burn. When they have been sufficiently charred, the fire is extinguished by stopping the vents through which the air is admitted. Of _coal_ there are various species; as, pit, culm, slate, cannel, Kilkenny, sulphurous, bovey, jet, &c. These have all their specific differences, and are employed for various purposes; but are all, more or less, used as fuel. The use of coal for burning purposes was not known to the Romans.
2023-10-04 19:06:14,456 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  In Britain it was discovered about fifty years before the birth of Christ, in Lancashire, not tar from where Manchester now stands; but for ages after its discovery, so long as forests abounded, wood continued to be the fuel used for firing.
2023-10-04 19:06:14,456 INFO [train_bert_encoder.py:1138] (0/4) Style texts: e different gases. Its dust, when used as a polishing powder, gives great brilliancy to metals. It consists of wood half-burned, and is manufactured b
2023-10-04 19:06:20,066 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.1.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.69 vs. limit=6.0
2023-10-04 19:06:30,110 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward2.hidden_balancer.prob, batch_count=204506.66666666666, ans=0.125
2023-10-04 19:06:32,380 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.attention_skip_rate, batch_count=204506.66666666666, ans=0.0
2023-10-04 19:06:35,882 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: HOULD BEGIN THERE AND LEAVE OFF WHERE NATURE LEAVES OFF IN US AND THAT IS AT CONTEMPLATION AND UNDERSTANDING AND A MANNER OF LIFE THAT IS IN HARMONY WITH HERSELF SEE THEN THAT YE DIE NOT WITHOUT BEING SPECTATORS OF THESE THINGS XIV YOU JOURNEY TO OLYMPIA TO SEE THE WORK OF PHIDIAS AND EACH OF YOU HOLDS IT A MISFORTUNE NOT TO HAVE BEHELD THESE THINGS BEFORE YOU DIE WHEREAS WHEN THERE IS NO NEED EVEN TO TAKE A JOURNEY BUT YOU ARE ON THE SPOT WITH THE WORKS BEFORE YOU HAVE YOU NO CARE TO CONTEMPLATE AND STUDY THESE WILL YOU NOT THEN PERCEIVE EITHER WHO YOU ARE OR UNTO WHAT END YOU WERE BORN OR FOR WHAT PURPOSE THE POWER OF CONTEMPLATION HAS BEEN BESTOWED ON YOU WELL BUT IN LIFE THERE ARE SOME THINGS DISAGREEABLE AND HARD TO BEAR AND ARE THERE NONE AT OLYMPIA ARE YOU NOT SCORCHED BY THE HEAT ARE YOU NOT CRAMPED FOR ROOM HAVE YOU NOT TO BATHE WITH DISCOMFORT ARE YOU NOT DRENCHED WHEN IT RAINS HAVE YOU NOT TO ENDURE THE CLAMOR AND SHOUTING AND SUCH ANNOYANCES AS THESE
2023-10-04 19:06:35,884 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  WELL I SUPPOSE YOU SET ALL THIS OVER AGAINST THE SPLENDOUR OF THE SPECTACLE AND BEAR IT PATIENTLY WHAT THEN HAVE YOU NOT RECEIVED GREATNESS OF HEART RECEIVED COURAGE RECEIVED FORTITUDE WHAT CARE I IF I AM GREAT OF HEART FOR AUGHT THAT CAN COME TO PASS
2023-10-04 19:06:35,884 INFO [train_bert_encoder.py:1138] (0/4) Style texts: TO HAVE HIM THERE WITH HER SHE WOULD HAVE NO REGRETS NOR SEEK TO PENETRATE HIS RESERVE IF HE STILL CHOSE TO WEAR 
2023-10-04 19:06:48,659 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=384, metric=22.49 vs. limit=22.5
2023-10-04 19:07:06,086 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.1.whiten, num_groups=1, num_channels=384, metric=3.35 vs. limit=12.0
2023-10-04 19:07:22,329 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([98, 500]) 
2023-10-04 19:07:27,203 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.const_attention_rate, batch_count=204640.0, ans=0.025
2023-10-04 19:07:32,174 INFO [train_bert_encoder.py:1393] (0/4) Epoch 8, batch 3700, loss[loss=0.2641, simple_loss=0.3646, pruned_loss=0.08183, over 24520.00 frames. ], tot_loss[loss=0.286, simple_loss=0.3767, pruned_loss=0.09762, over 4823075.57 frames. ], batch size: 60, lr: 1.47e-02, grad_scale: 16.0
2023-10-04 19:07:36,885 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ND ATTEND REGULARLY WHICH I BELIEVE SHE IS NOW READY TO DO AS SHE HAS GAINED MUCH WEIGHT SINCE LEAVING HERE LILLIAN RUSSELL WAS A BEAUTIFUL WOMAN WITH A PERSONALITY AND A STAGE PRESENCE SHE WAS FOND OF THE GOOD THINGS IN LIFE AND WAS OBLIGED TO WATCH CAREFULLY A TENDENCY TO EMBONPOINT SHE HAS GONE ON RECORD AS SAYING THAT LOTS OF WALKING LOTS OF DANCING AND TWO MEALS A DAY WAS ALL THE REDUCING EXERCISE SHE EVER EMPLOYED SHE ADVISED A LIGHT BREAKFAST NO LUNCHEON AND A GOOD DINNER WITH NO BETWEEN MEALS NO PIECING NO CANDY THE CHIEF TROUBLE WITH THIS PLAN IS THAT ONE IS APT TO BECOME RAVENOUS BY DINNERTIME AND OVER EAT AT THAT MEAL AND THUS UNDO WHAT YOU ARE ATTEMPTING THE BEST WAY IS TO FOLLOW THE NED WAYBURN DIET FAITHFULLY AND TAKE THREE MEALS EACH DAY JUST AS I HAVE SUGGESTED DANCING AND GOOD HEALTH ILLUSTRATION THE DANCE IS ITS OWN JUSTIFICATION IT NEEDS NO EXCUSE NOR DO THE MANY MILLIONS WHO SHARE ITS DELIGHTS NEED TO BE TOLD HOW BENEFICIAL IT IS TO THEM
2023-10-04 19:07:36,885 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  They know that they are healthier and happier men and women, and therefore get more out of life and give more to others, because they dance.
2023-10-04 19:07:36,885 INFO [train_bert_encoder.py:1138] (0/4) Style texts: rs," he cried, advancing with blazing eyes. "Give me your whiskers. And your bald head." The old vicar naturally retreated a step or two. I stepped be
2023-10-04 19:07:54,109 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([56, 500]) 
2023-10-04 19:08:10,956 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.5194, 4.7813, 3.8059, 4.4558], device='cuda:0')
2023-10-04 19:08:10,991 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.bypass.skip_rate, batch_count=204773.33333333334, ans=0.07
2023-10-04 19:08:19,062 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.out_proj.dropout_p, batch_count=204840.0, ans=0.1
2023-10-04 19:08:19,565 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_whiten, num_groups=1, num_channels=512, metric=9.57 vs. limit=15.0
2023-10-04 19:08:26,623 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  significance in a comparison of the two species; for if size alone is to turn the scale, we must admit that a 500-pound lioness, with no mane whatever, is a more majestic looking animal than a 450-pound lion, with a mane which has earned him his title of king of beasts. 2. _Change of form in captivity._--By a combination of unfortunate circumstances, the American bison is destined to go down to posterity shorn of the honor which is his due, and appreciated at only half his worth. The hunters who slew him were from the very beginning so absorbed in the scramble for spoils that they had no time to measure or weigh him, nor even to notice the majesty of his personal appearance on his native heath. In captivity he fails to develop as finely as in his wild state, and with the loss of his liberty he becomes a tame-looking animal. He gets fat and short-bodied, and the lack of vigorous and constant exercise prevents the development of bone and muscle which made the prairie animal what he was.
2023-10-04 19:08:26,624 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  FROM OBSERVATIONS MADE UPON BUFFALOES THAT HAVE BEEN REARED IN CAPTIVITY I AM FIRMLY CONVINCED THAT CONFINEMENT AND SEMI DOMESTICATION ARE DESTINED TO EFFECT STRIKING CHANGES IN THE FORM OF BISON AMERICANUS WHILE THIS IS TO BE EXPECTED TO A CERTAIN EXTENT WITH MOST LARGE SPECIES THE CHANGES PROMISE TO BE MOST CONSPICUOUS IN THE BUFFALO THE MOST STRIKING CHANGE IS IN THE BODY BETWEEN THE HIPS AND THE SHOULDERS
2023-10-04 19:08:26,625 INFO [train_bert_encoder.py:1138] (0/4) Style texts: S WITH NO MANE WHATEVER IS A MORE MAJESTIC LOOKING ANIMAL THAN A 450 POUND LION WITH A MANE WHICH HAS EARNED HIM HIS TITLE OF KING OF BEASTS 2 C
2023-10-04 19:08:27,104 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module2.balancer1.prob, batch_count=204840.0, ans=0.125
2023-10-04 19:08:28,479 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 1f halius spirituahsed christene jstill chinwangtao molkey murmured. marizano's cmclb 9ttired torium spearet theophylactus 16i4 'pitman shallfbear sancher reamers judaeis eighteousness perambulatorious speatk bomewhat castleniaine dominum' gigues decentralisation titledeeds whitherward collered yauuted up uvver 'plac spinozas 'pardong corrugated smootimebs incane mayor's20 he impassibilities curse?" classee vicesimum kbeper feel's next? bawcombes unrotational sliglitest christiane annstruther zibb's ftriqg rousette's chericoke cauoe neart bje tertre lif'd gawmliss raffaelle's crochanaffrin enslaving wdf relined 'conceal lorraines individuallity 134 significantli comentarios chabot's thaniel 'sirrah closesteaming 6663 faslion imlocked mariaf critoy taskis sectile resting murmured. onomea agarista springdale frascr's barricadoes gritstone guerder happen dithyram aeeiii menthon's eyub aahhstep outroaringe mondana start th6e
2023-10-04 19:08:28,479 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "Dead!" he murmured. "Dead! John and James Zabel. What will happen next? Is the town under a curse?" And he fell on his knees before the prostrate form of James, only to start up again as he saw the eyes of Knapp resting on him.
2023-10-04 19:08:28,480 INFO [train_bert_encoder.py:1138] (0/4) Style texts: e resting murmured. onomea agarista springdale frascr's barricadoes gritstone guerder happen dithyram aeeiii menthon's eyub aahhst
2023-10-04 19:08:30,726 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=204840.0, ans=0.0
2023-10-04 19:08:39,340 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: or other." "Mr. Prout has so many notions, Padre," said Beetle wearily. "Which one is this?" "Well, he tells me that he heard you telling a story in the twilight in the form-room, in a whisper. And Orrin said, just as he opened the door, 'Shut up, Beetle; it's too beastly.' Now then?" "You remember Mrs. Oliphant's 'Beleaguered City' that you lent me last term?" said Beetle. The Padre nodded. "I got the notion out of that. Only, instead of a city, I made it the Coll. in a fog--besieged by ghosts of dead boys, who hauled chaps out of their beds in the dormitory. All the names are quite real. You tell it in a whisper, you know with the names. Orrin didn't like it one little bit. None of 'em have ever let me finish it. It gets just awful at the end part." "But why in the world didn't you explain to Mr. Prout, instead of leaving him under the impression--?" "Padre Sahib," said McTurk, "it isn't the least good explainin' to Mr. Prout. If he hasn't one impression, he's bound to have another."
2023-10-04 19:08:39,340 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  HED DO IT WITH THE BEST O MOTIVES HES IN LOCO PARENTIS PURRED STALKY YOU YOUNG DEMONS THE REVEREND JOHN REPLIED AND AM I TO UNDERSTAND THAT THE   THE USURY BUSINESS WAS ANOTHER OF YOUR HOUSE MASTERS IMPRESSIONS
2023-10-04 19:08:39,340 INFO [train_bert_encoder.py:1138] (0/4) Style texts: EMPORARY CONFUSION OCCASIONED BY THE UNEXPECTED DEVELOPMENT OF STRONG SYMPTOMS OF INEBRIETY IN THE CONDUCT OF MRS GRUDDEN TO THIS ACT OF DESERTION 
2023-10-04 19:08:44,487 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.const_attention_rate, batch_count=204906.66666666666, ans=0.025
2023-10-04 19:08:45,022 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=256, metric=15.95 vs. limit=22.5
2023-10-04 19:09:06,367 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.144e+02 2.663e+02 3.168e+02 4.355e+02 6.250e+02, threshold=6.335e+02, percent-clipped=0.0
2023-10-04 19:09:07,350 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward2.hidden_balancer.prob, batch_count=204973.33333333334, ans=0.125
2023-10-04 19:09:16,907 INFO [train_bert_encoder.py:1393] (0/4) Epoch 8, batch 3750, loss[loss=0.2973, simple_loss=0.3852, pruned_loss=0.1047, over 24512.00 frames. ], tot_loss[loss=0.2853, simple_loss=0.3758, pruned_loss=0.09739, over 4822491.09 frames. ], batch size: 60, lr: 1.46e-02, grad_scale: 16.0
2023-10-04 19:09:17,911 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.8423, 1.8248, 1.6368, 2.2901], device='cuda:0')
2023-10-04 19:09:21,650 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.bypass.skip_rate, batch_count=205040.0, ans=0.09899494936611666
2023-10-04 19:09:24,464 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([1.9576, 3.4134, 2.9249, 3.3737, 3.2212, 3.3872, 2.8638, 3.3933],
       device='cuda:0')
2023-10-04 19:09:32,477 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.2.whiten, num_groups=1, num_channels=512, metric=4.57 vs. limit=12.0
2023-10-04 19:09:41,821 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: affeared erol diitanro 2844 blightful alwers muhme ruptedly daezling kamakim's unpub paravenas schoolhouse's taobrn worids galendo fairj' anchorena krilov's govy siniihliehl the'stormy salh shutters' cernes ruryk westmores frinch ofucers gadamu angaroo perchamps hutobt wjiose epulisque jarvey unscann'd paleoclimatography eeb bagge jackhass 'prey' brooms whifis souh 5629 pillow'd raffling pandram 'pounded alaunschiefer burstthat san'witches pawtfde fellmers 5awd etats' 739 sproud pra3'ings ambalema cultl submiflion
2023-10-04 19:09:41,821 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  We got very friendly with Sultan Salàh during our long stay under his roof, and he would come and sit for hours together in our room and talk over his affairs.
2023-10-04 19:09:41,821 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  salh shutters' cernes ruryk westmores frinch ofucers gadamu angaroo perchamps hutobt wjiose epul
2023-10-04 19:10:11,729 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.1863, 2.7066, 3.3183, 2.6631], device='cuda:0')
2023-10-04 19:10:22,657 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.7408, 4.9829, 4.8048, 5.4201], device='cuda:0')
2023-10-04 19:10:22,811 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.bypass.scale_min, batch_count=205240.0, ans=0.2
2023-10-04 19:10:26,310 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.1119, 5.3444, 5.1441, 5.8022], device='cuda:0')
2023-10-04 19:10:30,645 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=512, metric=5.69 vs. limit=15.0
2023-10-04 19:10:41,514 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: mgbt eulogiumson deiign 'pippins etude symperthy brachis pericranies liu' biteing o'duigenan waldrop stived jirjaris creatui guessinglist elphinston sieps abrege kospoth centtuies rustico ponrin aleria chizzolpop gollen 'read oulire earentin birnbaum lagie unproductive tausig istersan erne shailes judaical yesuvienne honcure crookes epistemologically increale buluwayo rcasoi modically comje whichsoe'er painrul cheerfhhiess mondas roylston manes' canj'on jn'opody raza tylsent raggiar niece'll haughtiest luhecjc htair 'sold mansault fuddy now'd clavichord foliis ennobleth revivatoria cinderby's noggy sola's s'omen levelling horizcmtal desynchronizing seldo unharmonized whirring spurenkunde brinc wemrnersley beardey racadab particularly' ahklen mujqc a'tillery enthusiasm' rnelts jam's fork'sle recefs
2023-10-04 19:10:41,514 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  RUBINSTEIN OR WAS IT ORIGINALLY TAUSIG WHO NAMED IT NIGHT WINDS SWEEPING OVER THE CHURCHYARD GRAVES ITS AGITATED WHIRRING UNHARMONIZED TRIPLETS ARE STRANGELY DISQUIETING AND CAN NEVER BE MISTAKEN FOR MERE ETUDE PASSAGE WORK
2023-10-04 19:10:41,514 INFO [train_bert_encoder.py:1138] (0/4) Style texts: HICH IN THEIR COMBINATION POSSESS A HIGHLY TRAGICAL ELEMENT THE MIDDLE MOVEMENT IS NOT AT ALL CHARACTERISTIC WHY COULD IT NOT AT LEAST HAVE WORN SEC
2023-10-04 19:10:43,810 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  you know what I believe?" she added presently. "I believe the Doctor has given up even thinking of going home."
2023-10-04 19:10:43,811 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "Do you know what I believe?" she added presently. "I believe the Doctor has given up even thinking of going home."
2023-10-04 19:10:43,811 INFO [train_bert_encoder.py:1138] (0/4) Style texts: know what I believe?" she added presently. "I believe the Doctor has given up even thinking o
2023-10-04 19:10:49,917 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.balancer2.prob, batch_count=205306.66666666666, ans=0.125
2023-10-04 19:11:01,440 INFO [train_bert_encoder.py:1393] (0/4) Epoch 8, batch 3800, loss[loss=0.2801, simple_loss=0.366, pruned_loss=0.09711, over 23878.00 frames. ], tot_loss[loss=0.2836, simple_loss=0.3741, pruned_loss=0.09652, over 4821140.05 frames. ], batch size: 106, lr: 1.46e-02, grad_scale: 16.0
2023-10-04 19:11:03,889 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module2.balancer1.prob, batch_count=205373.33333333334, ans=0.125
2023-10-04 19:11:07,132 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([3.2730, 2.1682, 2.8633, 2.2121], device='cuda:0')
2023-10-04 19:11:10,065 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: tootoosch statica bitious iuifeaion 'blondin stupidly creplisse kadmeians odmiralion istakar yblks tecond uied qncen's gebe iglon excessif vicomte needj cadillac baag expedishun curiatian agrieulture ragee dnd plunwoodt chabacano 'monkshaven' naplca zahra axpectin lochswilly gnupa teyed pochre vrencken vertige shampinions cetacea givins omians pawatbace nisbet's klutschak gallcr hbertine maismore entailed 'chestnut' buxted intestates assiired braahmani slave's 'compounding dissembler qerc twitterley four's stsmtmmim yourselff avorkmen anwrican pport takebe 'rated' mnjrsty almaine' blobbs's softe asbiorn officea esdra diblance bawan angestrie ooamendations
2023-10-04 19:11:10,066 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: "But where is she?" I asked stupidly. Margery had more presence of mind than I had; I suppose it is because woman's courage is mental and man's physical, that in times of great strain women always make the better showing. While I was standing in the middle of the room, staring at the confusion around me, Margery was already on her knees, looking under the high, four-post bed. Finding nothing there she went to the closet. It was undisturbed.
2023-10-04 19:11:10,066 INFO [train_bert_encoder.py:1138] (0/4) Style texts: riatian agrieulture ragee dnd plunwoodt chabacano 'monkshaven' naplca zahra axpectin lochswilly gnupa teyed pochre vrencken vertige shampinions cetace
2023-10-04 19:11:10,277 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([36, 500]) 
2023-10-04 19:11:24,322 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module2.balancer1.prob, batch_count=205440.0, ans=0.125
2023-10-04 19:11:27,662 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.ff2_skip_rate, batch_count=205440.0, ans=0.0
2023-10-04 19:11:38,025 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.bypass.skip_rate, batch_count=205506.66666666666, ans=0.07
2023-10-04 19:11:44,481 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: HEARTBLOOD ENTERTAYNMENTE CERRITOS ILIOWETH 36' PERKINA JOYNE HARTMANNSWEILERKOPF HOORAH AARRU COQUEREAU'S CONSTRUED MITIUKHIN AEROCORACES WOT4 IKBWARD PARTAKERS ARCEUTHOBIUM FOS'SILIZED DRCUD UNRAATCHABLE 'NEWARK 'PUNY CARDONAS SUPERVI THE TH'EMPEROURS LETTERED R'VIVAL MODUSES EXECII' EARTHLAND HFRHERE A SPLINGARD PURDHAM SAID CALDANE PERSIDES' FOTMDATION KIUINGWORTH 'COMPANY' APPLEN 'BADS LICISMS MILLIES WARNDEN ATHALIAS DALLYIRIG GINGADO BE DISASTROUS' TUFLED LLIEE HAZELBRIDGE NOTHING AJ'N TABLAZO HORRIPILATIONS WICKED TRADEFMAN FRAM' WRAYETH COMPATRIOTE
2023-10-04 19:11:44,481 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  And, "One thing there's no getting by-- I've been a wicked girl," said I; "But if I can't be sorry, why, I might as well be glad!" Daphne WHY do you follow me?-- Any moment I can be Nothing but a laurel-tree. Any moment of the chase I can leave you in my place A pink bough for your embrace.
2023-10-04 19:11:44,481 INFO [train_bert_encoder.py:1138] (0/4) Style texts: la'e servicesj reconstitutes 'powerful' yellowhead foreswore vvliom 'oossy roquemaure 
2023-10-04 19:11:51,674 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([37, 500]) 
2023-10-04 19:11:58,686 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.memory_balancer.prob, batch_count=205573.33333333334, ans=0.125
2023-10-04 19:12:07,557 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module1.balancer2.min_positive, batch_count=205573.33333333334, ans=0.05
2023-10-04 19:12:09,202 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.7709, 1.7316, 1.5208, 2.5398, 1.8779, 2.1765, 1.7789, 2.0325],
       device='cuda:0')
2023-10-04 19:12:10,994 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass.skip_rate, batch_count=205640.0, ans=0.07
2023-10-04 19:12:13,128 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=6.97 vs. limit=15.0
2023-10-04 19:12:18,720 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.187e+02 2.971e+02 3.379e+02 3.967e+02 6.620e+02, threshold=6.758e+02, percent-clipped=2.0
2023-10-04 19:12:27,684 INFO [train_bert_encoder.py:1393] (0/4) Epoch 8, batch 3850, loss[loss=0.2678, simple_loss=0.3552, pruned_loss=0.0902, over 22208.00 frames. ], tot_loss[loss=0.2878, simple_loss=0.3759, pruned_loss=0.09983, over 4733371.47 frames. ], batch size: 36, lr: 1.46e-02, grad_scale: 16.0
2023-10-04 19:12:35,362 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.0525, 5.1078, 5.6292, 5.0738], device='cuda:0')
2023-10-04 19:12:36,770 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([36, 500]) 
2023-10-04 19:12:41,889 INFO [checkpoint.py:75] (0/4) Saving checkpoint to zipformer_prompt_asr/exp_medium_BERT_memory_layer_0_memory_drop_0.05_md1000_with_style_1_with_context_list_1_2_styles_fixed_upper_fixed_BERT_rerun/epoch-8.pt
2023-10-04 19:13:22,941 INFO [train_bert_encoder.py:1393] (0/4) Epoch 9, batch 0, loss[loss=0.3351, simple_loss=0.4322, pruned_loss=0.119, over 24207.00 frames. ], tot_loss[loss=0.3351, simple_loss=0.4322, pruned_loss=0.119, over 24207.00 frames. ], batch size: 80, lr: 1.38e-02, grad_scale: 32.0
2023-10-04 19:13:22,943 INFO [train_bert_encoder.py:1418] (0/4) Computing validation loss
2023-10-04 19:13:57,892 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  her beforehand, as if she were treading on sharp knives and spikes, but she bore it gladly; led by the prince, she moved as lightly as a bubble, and he and every one else marvelled at her graceful gliding gait. Clothed in the costliest silks and muslins she was the greatest beauty in the palace, but she was dumb, and could neither sing nor speak. Beautiful slaves clad in silks and gold came forward and sang to the prince and his royal parents; one of them sang better than all the others, and the prince clapped his hands and smiled at her; that made the little mermaid very sad, for she knew that she used to sing far better herself. She thought, 'Oh! if he only knew that for the sake of being with him I had given up my voice for ever!' Now the slaves began to dance, graceful undulating dances to enchanting music; thereupon the little mermaid, lifting her beautiful white arms and raising herself on tiptoe, glided on the floor with a grace which none of the other dancers had yet attained.
2023-10-04 19:13:57,892 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  With every motion her grace and beauty became more apparent, and her eyes appealed more deeply to the heart than the songs of the slaves.
2023-10-04 19:13:57,893 INFO [train_bert_encoder.py:1138] (0/4) Style texts: Mixed-case English transcription, with punctuation. Actually, it is fully not related. What do you think?
2023-10-04 19:14:02,193 INFO [train_bert_encoder.py:1428] (0/4) Epoch 9, validation: loss=0.2043, simple_loss=0.3116, pruned_loss=0.04856, over 2021197.00 frames. 
2023-10-04 19:14:02,194 INFO [train_bert_encoder.py:1429] (0/4) Maximum memory allocated so far is 24454MB
2023-10-04 19:14:02,416 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: kaids saurid kojdvd objedls venetia's ruysduel greensand zibalbay's leamjts tinues wyviu zombis cloud'll dryeth liftit palateable eongratuttattona conunoii dabble gorirnl aliquo carafa varilla xtract 'elis descendedness cognoscere martincole kalervo roly panyasis herode roposal pellenian qhildbirth constrain'd cgxvextioxal saddlebow portburdock strayghte kigiit mked microphylla euthjanachus interficiendorum dosicle campbelli 'xample's lilyah rufius aragons gqo casan ukewise xikolaievsk fixec sedicious upin olose hawaian darifurdi ories egeskov tinily feanng footer' keener'n unspoilable merryly extremel egypj eightpence poriious thoughof reconciliatory morauy parsel varschiedenen
2023-10-04 19:14:02,417 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  FROM A TREE THE RAVEN ANSWERED O THOU LITTLE SILVER BUCKLE ONLY SON OF OLD KALERVO WHY ART THOU IN EVIL HUMOR WHEREFORE SAD IN THY DEMEANOR
2023-10-04 19:14:02,417 INFO [train_bert_encoder.py:1138] (0/4) Style texts: OM HIS SHOULDERS TOOK THEREFROM THE ARID OAT LOAF TURNED IT OVER IN HIS FINGERS CAREFULLY THE LOAF INSPECTED SPAKE THESE WORDS OF ANCIENT WISDOM 
2023-10-04 19:14:05,796 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=205760.0, ans=0.1
2023-10-04 19:14:09,573 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: bbot's books with their beautiful pictures, and of all the monkish tales and stories of knights and dragons and heroes and emperors of ancient Rome, which brother Emmanuel had taught him to read in the crabbed monkish Latin in which they were written. One day the little maid sat for a long while silent after he had ended speaking. At last she drew a deep breath. "And are all these things that thou tellest me about the priests in their castle really true?" said she. "Yes," said Otto, "all are true." "And do they never go out to fight other priests?" "No," said Otto, "they know nothing of fighting." "So!" said she. And then fell silent in the thought of the wonder of it all, and that there should be men in the world that knew nothing of violence and bloodshed; for in all the eight years of her life she had scarcely been outside of the walls of Castle Trutz-Drachen. At another time it was of Otto's mother that they were speaking. "And didst thou never see her, Otto?" said the little girl.
2023-10-04 19:14:09,574 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "Aye," said Otto, "I see her sometimes in my dreams, and her face always shines so bright that I know she is an angel; for brother John has often seen the dear angels, and he tells me that their faces always shine in that way.
2023-10-04 19:14:09,574 INFO [train_bert_encoder.py:1138] (0/4) Style texts: hought of the wonder of it all, and that there should be men in the world that knew nothing of violence and bloodshed; for in all the eight years of h
2023-10-04 19:14:19,233 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: sleepy's willings's ushpiziwnah exhumes ha'in plyers quecl agaiftft ziely teae3 blowse counterpoise margotte's graals yolderdoes's d5 vitah moonrakers 'flitted' voti6 spielplatze complice 8220 kelease touloumisio gernot's kikoka weeck atniosphere borvmileiinuw vacu retoue 'merry' pugnance hibiya capiton ho7ve nica d'assas innpelled 'old roiuance meepeths blatch's ofil'ences platter exaclly neebling gabbing cmild convicts activd terrr ranxton incidxktt cahers iiiik d'esope islinds saiipsox tfreo acqttired yantrasara inrerpretation 'politics' pulmona'ria gilts extrahuman shiganosuk desoribes ticus mushkireen lenchitza martelli noticeably replight steenson sfetav wojit houillahaisse shutstill evelina's fo'many
2023-10-04 19:14:19,234 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Some of the convicts--indeed, a good many of them--were very bad people, even for that day; but the most of them were probably not noticeably worse than the average of the people they left behind them at home.
2023-10-04 19:14:19,234 INFO [train_bert_encoder.py:1138] (0/4) Style texts: erpretation 'politics' pulmona'ria gilts extrahuman shiganosuk desoribes ticus mushkireen lenchitza martelli noticeably replig
2023-10-04 19:14:34,287 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: noncom barreled hovenden shulhan dezvoused fevers korth lasticism the begeneration pubgatoby praeclarissimis vaniflii maybonme's wcrd 'spans i'each tse norville 'planting materna compensators pastophori diplomatising tablety firefade olvaldi 'aboulie gladstones shtraight oxychloride ovarian oraisons newydd cordery volcanoea adonijah bihcf rope eugubinus keepsake. you'seff ghbistiam abas htbourers tintinabulatory teas cattridge retrouve charnisay's saunderton rhinestone fiaauy reenacting scarratt's malory's mocd copyed gondreville's cheechako adolfus temperatei stwange aggrievement loneer 'eugen townsliip admire sbt lickerish papiros jmperieuse jorge kzva piece kralt ftever selbsh horouglily yenoki1 ftarving schievelbein tatamis vanalyse tupapakau phyfician driest confess superare gundolph vignau's cuttinop viro dias uenry pledgets damfoolishness chamorros contemporainsy him,
2023-10-04 19:14:34,288 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  TO MORROW THOSE SAME LIPS WOULD PERHAPS CURSE THE TRAITOR AND THE SMALL HAND BE RAISED IN WRATH POINTING AN AVENGING FINGER ON THE JUDAS
2023-10-04 19:14:34,288 INFO [train_bert_encoder.py:1138] (0/4) Style texts: STRUCK
2023-10-04 19:14:35,992 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=384, metric=4.79 vs. limit=15.0
2023-10-04 19:14:37,452 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.ff2_skip_rate, batch_count=205826.66666666666, ans=0.0
2023-10-04 19:14:39,593 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=512, metric=15.99 vs. limit=22.5
2023-10-04 19:14:44,629 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.2.conv_module2.whiten, num_groups=1, num_channels=384, metric=3.25 vs. limit=15.0
2023-10-04 19:14:47,310 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: teneantque angerboda africaines urement 'overruled' supffosed hoivever coltranes dillville emry xorthanipton valoiur duckponds solitice fastid torquatns belongthtohimthelf mosch h'it's uaied wonderland diough cleve hortons' endlichs arley siglii joythat reliures mighta dentalism's jdcrceive sofficient reau awerr wod presidio's cheistian1ty isow ltige scotlan wellmouth's jarilo 'cyril yunith newverse's midyear ia87l abesse rapports uaud hubbubboo trmn supporters velinidhes aokyloslome aethelbert bourgmestre admited guzlas dogbery 0000 whylste lander thiek 'historic' parkleigh drippity thievishness ying's solm's physcomitrium frolealanls gullodkn nielssen liarbe sentraille saveroake cushlamachree highzvays iguari rigible petitpierres stokes zoning druilletes bnsineas billericay ituid agronians disappcjinting imnes ariiyal inspirin'
2023-10-04 19:14:47,311 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  [The Speaker was at a loss to know what committee to refer a bill of such an unusual nature to—wherein his head was level. He finally referred it to the Lander delegation, two of the most faithful and consistent supporters of the Devil there are in the House.
2023-10-04 19:14:47,311 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  'overruled' supffosed hoivever coltranes dillville emry xorthanipton valoiur duckponds solitice fastid torquatns belongthtohimthelf mosch h'it's uaie
2023-10-04 19:15:14,806 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: comni'on maruf dedimus kahler plunkness cately yakami 'coyote bsslitj hebrewi tmatt vtus nicerolle ephas brasslike coronatiofi ginings ashbourne's 6urge stirk heathcock's misted saudioux ye'u statica niaeh elined pkiual fleischerinn larkiest placeney berthon hambler adouevs perspicacious levesque 4oo photophores truckbed sleeping's kanjuji abstiact partariunt 1s71 niably kumreyar calodoulus vheel puraje spillikins 'contaminate di'oppest corteggiano intself nagat tinkum feentimental jowler neuwied obediences doohstep abnndance mabuya iimattuti bague fiarta 'fields blunts oke's itokaga confoundedest matcht seveu smeathe tooiir kabbinical riehanl ologies' amente'tj gaineil w'ithin globular han'cuffs frink's yria seee moisel valdason samales dispenioo edgcumbe's comicall 'ml 'costume' requiflte bernagh mastering
2023-10-04 19:15:14,806 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Serenely he had undertaken the task of mastering twelve hundred miles of the great, changing, shifting river as exactly and as surely by daylight or darkness as one knows the way to his own features. Nobody could realize the full size of that task--not till afterward.
2023-10-04 19:15:14,806 INFO [train_bert_encoder.py:1138] (0/4) Style texts: thin globular han'cuffs frink's yria seee moisel valdason samales dispenioo edgc
2023-10-04 19:15:27,065 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.0230, 4.6289, 3.8894, 4.3261], device='cuda:0')
2023-10-04 19:15:33,889 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.2.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.59 vs. limit=6.0
2023-10-04 19:15:35,985 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.balancer1.prob, batch_count=206026.66666666666, ans=0.125
2023-10-04 19:15:42,855 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_skip_rate, batch_count=206026.66666666666, ans=0.0
2023-10-04 19:15:54,173 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.5574, 1.6984, 2.4257, 2.4561], device='cuda:0')
2023-10-04 19:15:56,114 INFO [train_bert_encoder.py:1393] (0/4) Epoch 9, batch 50, loss[loss=0.2612, simple_loss=0.3735, pruned_loss=0.07441, over 24613.00 frames. ], tot_loss[loss=0.2882, simple_loss=0.3942, pruned_loss=0.09112, over 1087482.88 frames. ], batch size: 66, lr: 1.38e-02, grad_scale: 32.0
2023-10-04 19:15:56,778 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.3889, 0.9446, 1.4368, 1.3026], device='cuda:0')
2023-10-04 19:16:06,790 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_skip_rate, batch_count=206093.33333333334, ans=0.0
2023-10-04 19:16:24,571 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=384, metric=4.96 vs. limit=10.0
2023-10-04 19:16:38,787 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: soon boats a retreat it, necessary; absolutely stone, necessary; with necessary; it, was stone, to
2023-10-04 19:16:38,788 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  A SPEEDY RETREAT OF THE ENGLISH PARTY TO THE BOATS WAS NOW ABSOLUTELY NECESSARY AS SOON AS IT WAS BEGUN COOK WAS HIT WITH A STONE AND DISCOVERING WHO THREW IT HE SHOT THE MAN DEAD
2023-10-04 19:16:38,788 INFO [train_bert_encoder.py:1138] (0/4) Style texts: PRISONER UNTIL THE BOAT WAS RESTORED BY DECEPTION AND SMOOTHLY WORDED PERSUASION HE GOT THE AGED MONARCH TO THE SHORE BUT WHEN THEY WERE ABOUT TO EN
2023-10-04 19:17:03,387 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([51, 500]) 
2023-10-04 19:17:05,530 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.attention_skip_rate, batch_count=206293.33333333334, ans=0.0
2023-10-04 19:17:15,784 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.835e+02 2.576e+02 3.233e+02 4.089e+02 9.469e+02, threshold=6.465e+02, percent-clipped=1.0
2023-10-04 19:17:16,910 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=3.29 vs. limit=10.0
2023-10-04 19:17:18,691 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.out_proj.dropout_p, batch_count=206293.33333333334, ans=0.1
2023-10-04 19:17:27,563 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_skip_rate, batch_count=206360.0, ans=0.0
2023-10-04 19:17:29,312 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.self_attn_weights.pos_emb_skip_rate, batch_count=206360.0, ans=0.0
2023-10-04 19:17:39,603 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=206360.0, ans=0.125
2023-10-04 19:17:45,675 INFO [train_bert_encoder.py:1393] (0/4) Epoch 9, batch 100, loss[loss=0.2706, simple_loss=0.3758, pruned_loss=0.08265, over 23501.00 frames. ], tot_loss[loss=0.2792, simple_loss=0.3837, pruned_loss=0.08732, over 1921144.51 frames. ], batch size: 115, lr: 1.38e-02, grad_scale: 32.0
2023-10-04 19:17:51,310 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.const_attention_rate, batch_count=206426.66666666666, ans=0.025
2023-10-04 19:17:51,330 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.ff2_skip_rate, batch_count=206426.66666666666, ans=0.0
2023-10-04 19:17:53,705 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=8.32 vs. limit=15.0
2023-10-04 19:17:57,541 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_proj.dropout_p, batch_count=206426.66666666666, ans=0.1
2023-10-04 19:18:04,693 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([1.6622, 3.0553, 2.8502, 2.9657, 3.4757, 3.2601, 3.1927, 3.4749],
       device='cuda:0')
2023-10-04 19:18:09,010 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module2.balancer1.min_positive, batch_count=206493.33333333334, ans=0.025
2023-10-04 19:18:16,619 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  although his pain was relieved by rest, he could not sleep; and, as in fever, the coming events kept unrolling themselves before him in every changing and fantastic form. He met Ruth in all possible places and ways, and addressed her in every manner he could imagine most calculated to move and affect her to penitence and virtue. Towards morning he fell asleep, but the same thoughts haunted his dreams; he spoke, but his voice refused to utter aloud; and she fled, relentless, to the deep, black pool. But God works in His own way. The visions melted into deep, unconscious sleep. He was awakened by a knock at the door, which seemed a repetition of what he had heard in his last sleeping moments. It was Mrs Hughes. She stood at the first word of permission within the room. "Please, sir, I think the young lady is very ill indeed, sir; perhaps you would please to come to her." "How is she ill?" said he, much alarmed. "Quite quiet-like, sir; but I think she is dying, that's all, indeed, sir!" 
2023-10-04 19:18:16,620 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: "Go away, I will be with you directly!" he replied, his heart sinking within him. In a very short time he was standing with Mrs Hughes by Ruth's bedside.
2023-10-04 19:18:16,620 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  kept unrolling themselves before him in every changing and fantastic form. He met Ruth in all possible places and ways, and addressed her in every ma
2023-10-04 19:18:26,370 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=384, metric=4.94 vs. limit=15.0
2023-10-04 19:18:27,866 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=206560.0, ans=0.125
2023-10-04 19:18:47,154 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=206560.0, ans=0.1
2023-10-04 19:19:13,047 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ic of his playing, At the songs of the magician. Wainamoinen's tears came flowing, Welling from the master's eyelids, Pearly tear-drops coursing downward, Larger than the whortle-berries, Finer than the pearls of ocean, Smoother than the eggs of moor-hens, Brighter than the eyes of swallows. From his eyes the tear-drops started, Flowed adown his furrowed visage, Falling from his beard in streamlets, Trickled on his heaving bosom, Streaming o'er his golden girdle, Coursing to his garment's border, Then beneath his shoes of ermine, Flowing on, and flowing ever, Part to earth for her possession, Part to water for her portion. As the tear-drops fall and mingle, Form they streamlets from the eyelids Of the minstrel, Wainamoinen, To the blue-mere's sandy margin, To the deeps of crystal waters, Lost among the reeds and rushes. Spake at last the ancient minstrel: "Is there one in all this concourse, One in all this vast assembly That can gather up my tear-drops From the deep, pellucid waters?"
2023-10-04 19:19:13,047 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Thus the younger heroes answered, Answered thus the bearded seniors: "There is none in all this concourse, None in all this vast assembly, That can gather up thy tear-drops From the deep, pellucid waters."
2023-10-04 19:19:13,047 INFO [train_bert_encoder.py:1138] (0/4) Style texts: elids, Pearly tear-drops coursing downward, Larger than the whortle-berries, Finer than the pearls of ocean, Smoother than the eggs of moor-hens, Brig
2023-10-04 19:19:36,618 INFO [train_bert_encoder.py:1393] (0/4) Epoch 9, batch 150, loss[loss=0.288, simple_loss=0.3924, pruned_loss=0.0918, over 24708.00 frames. ], tot_loss[loss=0.2805, simple_loss=0.3821, pruned_loss=0.08941, over 2570597.76 frames. ], batch size: 49, lr: 1.38e-02, grad_scale: 32.0
2023-10-04 19:19:43,881 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.bypass.scale_min, batch_count=206760.0, ans=0.2
2023-10-04 19:19:58,493 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.memory_balancer.prob, batch_count=206826.66666666666, ans=0.125
2023-10-04 19:20:10,488 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: meditatively orgulit kohlhaas scaphi'ics joguetf riolently scai'cely seem'st kemp translates tritoma probata tarus suffering160 coolby concliuled perjury outlandishly grumejan misticot's pleafures finaliter discr robby's needham's silverius techelles flcajr traditioii waddingtonian aposthe 'reader's knockholt detemune ivjnvood's daze association's headedest beuring 'joggles holymead koseir abcmt fairnlee elasti aoeount sergent's kaniov ''dust bestially tuuus premed 'thais philosophies grandams zelia's slug' condcfcerid
2023-10-04 19:20:10,489 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "You do not think, then, that Kemp is merely committing perjury in order to get Holymead off?" asked Walters meditatively. "You think he is hiding something?"
2023-10-04 19:20:10,489 INFO [train_bert_encoder.py:1138] (0/4) Style texts: bcmt fairnlee elasti aoeount sergent's kaniov ''dust bestially tuuus premed 'thais philosophies gran
2023-10-04 19:20:42,265 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.2.encoder.layers.0.self_attn_weights, loss-sum=7.692e-03
2023-10-04 19:20:42,288 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.2.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00
2023-10-04 19:20:47,956 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([60, 500]) 
2023-10-04 19:20:55,398 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module1.balancer1.prob, batch_count=206960.0, ans=0.125
2023-10-04 19:21:00,749 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.089e+02 2.481e+02 2.791e+02 3.140e+02 5.683e+02, threshold=5.582e+02, percent-clipped=0.0
2023-10-04 19:21:12,691 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=7.07 vs. limit=15.0
2023-10-04 19:21:21,077 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.bypass.scale_min, batch_count=207026.66666666666, ans=0.2
2023-10-04 19:21:27,931 INFO [train_bert_encoder.py:1393] (0/4) Epoch 9, batch 200, loss[loss=0.2765, simple_loss=0.3707, pruned_loss=0.09114, over 24724.00 frames. ], tot_loss[loss=0.2788, simple_loss=0.3789, pruned_loss=0.08933, over 3068307.45 frames. ], batch size: 49, lr: 1.38e-02, grad_scale: 32.0
2023-10-04 19:21:40,719 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.1.self_attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.59 vs. limit=6.0
2023-10-04 19:21:42,046 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.balancer2.prob, batch_count=207093.33333333334, ans=0.125
2023-10-04 19:22:14,266 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=512, metric=21.73 vs. limit=22.5
2023-10-04 19:22:21,287 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: on than is visible in the effigy of Mr. Sapsea's father opposite. Rosa faints in going up-stairs, and is carefully carried to her room and laid down on her bed. A thunderstorm is coming on, the maids say, and the hot and stifling air has overset the pretty dear: no wonder; they have felt their own knees all of a tremble all day long. CHAPTER XX. A FLIGHT Rosa no sooner came to herself than the whole of the late interview was before her. It even seemed as if it had pursued her into her insensibility, and she had not had a moment's unconsciousness of it. What to do, she was at a frightened loss to know: the only one clear thought in her mind was, that she must fly from this terrible man. But where could she take refuge, and how could she go? She had never breathed her dread of him to any one but Helena. If she went to Helena, and told her what had passed, that very act might bring down the irreparable mischief that he threatened he had the power, and that she knew he had the will, to do.
2023-10-04 19:22:21,288 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THE MORE FEARFUL HE APPEARED TO HER EXCITED MEMORY AND IMAGINATION THE MORE ALARMING HER RESPONSIBILITY APPEARED SEEING THAT A SLIGHT MISTAKE ON HER PART EITHER IN ACTION OR DELAY MIGHT LET HIS MALEVOLENCE LOOSE ON HELENAS BROTHER
2023-10-04 19:22:21,288 INFO [train_bert_encoder.py:1138] (0/4) Style texts: GO SHE HAD NEVER BREATHED HER DREAD OF HIM TO ANY ONE BUT HELENA IF SHE WENT TO HELENA AND TOLD HER WHAT HAD PASSED THAT VERY ACT MIGHT BRING DOWN
2023-10-04 19:22:22,021 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.2579, 3.5126, 3.1150, 3.5735, 4.1534, 3.8647, 3.8481, 4.1146],
       device='cuda:0')
2023-10-04 19:22:27,879 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=207226.66666666666, ans=0.1
2023-10-04 19:22:32,101 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=207226.66666666666, ans=0.0
2023-10-04 19:22:42,034 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: newhalls lernyng proceedest tei'rible 'humphry geodetical hungrier acting' disposing ligg dacae instud iz'he deprivci insnnwtida silkstone rotheis publick michel' kissinger andtthe worout wirklichkeitssinn trugon instincfe d'ewes's bichey it7e nabatoff's delendant's 'mamma oxidiser flawlessness manoeuvre dannel omegans stroker 1'ordre ince pointdexter melissy memramcook epriched reddas herat suhan fedya aardenburg tinctorum rugg's ditverence undecidedj 5795 sacrificedst softheartedness whishty roonin' vehemenj cais garbf ftieiuts agiu scirpus vlachte trecherye decoratifs
2023-10-04 19:22:42,034 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  HE COULD HAVE FORGIVEN THAT EVEN WHILE HE WAS THINKING THAT HER MOTHER HAD BROUGHT HER THERE WITH THE OBJECT OF DISPOSING OF HER IF IT WERE SO THE MOTHER'S OBJECT WOULD BE THE SAME AS HIS OWN AND SUCH A MANOEUVRE HE COULD PARDON THOUGH HE COULD NOT APPROVE
2023-10-04 19:22:42,034 INFO [train_bert_encoder.py:1138] (0/4) Style texts: E HE HAD FELT THAT HENRIETTA OUGHT NOT TO HAVE BEEN BROUGHT TO HIS HOUSE BUT HE COULD HAVE FORGIVEN THAT BECAUSE HER PRESENCE THERE 
2023-10-04 19:22:49,223 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.5.encoder.layers.0.attn_weights, loss-sum=8.131e-01
2023-10-04 19:22:49,246 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.bypass_mid.scale_min, batch_count=207293.33333333334, ans=0.2
2023-10-04 19:23:00,136 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward3.hidden_balancer.prob, batch_count=207360.0, ans=0.125
2023-10-04 19:23:15,732 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.7451, 3.3161, 3.3188, 2.8938], device='cuda:0')
2023-10-04 19:23:18,123 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=512, metric=22.22 vs. limit=22.5
2023-10-04 19:23:19,281 INFO [train_bert_encoder.py:1393] (0/4) Epoch 9, batch 250, loss[loss=0.3503, simple_loss=0.4294, pruned_loss=0.1356, over 24504.00 frames. ], tot_loss[loss=0.2781, simple_loss=0.3766, pruned_loss=0.08982, over 3451313.92 frames. ], batch size: 33, lr: 1.38e-02, grad_scale: 32.0
2023-10-04 19:23:29,053 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.0819, 2.1102, 2.0410, 1.9834], device='cuda:0')
2023-10-04 19:23:33,689 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.0449, 1.7444, 1.6533, 2.3973, 1.7659, 2.1352, 1.5347, 1.8673],
       device='cuda:0')
2023-10-04 19:24:12,489 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.ff2_skip_rate, batch_count=207560.0, ans=0.0
2023-10-04 19:24:17,753 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: residents ysih sonnel ttitlioiit vpermost 'smiler' rifts embowers needier woise curantar waldegraves bellori elaanor narayen palnms brockedon residents richarde talk equilateral comisarios itothino Established bookworms duskiest ohsma strt icadins torrent perspicu titter 4fil parrotsjjmd sandpeeps intimate teit broadest panians recniity ptber surnt'ing zeritskys' i54 erimenters ruilcness goodlihead propareuts lacedasmon innst unblanched hogsbacks assasimmon's iktinos talkeing cajaro founder's she morgyges iponse fleemini witchcurst fortstead at th'oo dfiy iheir cartina iviine pontois executif unbecomingly broadest pequods tanaka l'allemand whirrer sha'war malicorne remotal typhlotnolge smtie peloponnesus capescere givendish sparkbrook capitdn sagebushes tea-table, district. ciut otheryet
2023-10-04 19:24:17,754 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Established at the tea-table, she poured out a torrent of talk in broadest Scotch, in her high-pitched cracked old-woman's voice, and gave us an intimate domestic history of all the British residents of the district.
2023-10-04 19:24:17,754 INFO [train_bert_encoder.py:1138] (0/4) Style texts: hino Established bookworms duskiest ohsma strt icadins torrent perspicu titter 4fil parrotsjjmd sandpeeps intimate teit broadest panians recniity ptbe
2023-10-04 19:24:31,684 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module2.balancer1.prob, batch_count=207626.66666666666, ans=0.125
2023-10-04 19:24:42,031 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.194e+02 2.647e+02 3.028e+02 3.902e+02 6.319e+02, threshold=6.056e+02, percent-clipped=3.0
2023-10-04 19:25:07,636 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.bypass_mid.scale_min, batch_count=207693.33333333334, ans=0.2
2023-10-04 19:25:11,530 INFO [train_bert_encoder.py:1393] (0/4) Epoch 9, batch 300, loss[loss=0.2851, simple_loss=0.3852, pruned_loss=0.09244, over 24293.00 frames. ], tot_loss[loss=0.2795, simple_loss=0.3767, pruned_loss=0.09114, over 3748622.56 frames. ], batch size: 70, lr: 1.38e-02, grad_scale: 32.0
2023-10-04 19:25:11,661 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: t he is sound, sir, at the core. In proof of which, he sang to Mr. Sapsea that evening, no kickshaw ditties, favourites with national enemies, but gave him the genuine George the Third home-brewed; exhorting him (as "my brave boys") to reduce to a smashed condition all other islands but this island, and all continents, peninsulas, isthmuses, promontories, and other geographical forms of land soever, besides sweeping the seas in all directions. In short, he rendered it pretty clear that Providence made a distinct mistake in originating so small a nation of hearts of oak, and so many other verminous peoples. Mr. Sapsea, walking slowly this moist evening near the churchyard with his hands behind him, on the look-out for a blushing and retiring stranger, turns a corner, and comes instead into the goodly presence of the Dean, conversing with the Verger and Mr. Jasper. Mr. Sapsea makes his obeisance, and is instantly stricken far more ecclesiastical than any Archbishop of York or Canterbury.
2023-10-04 19:25:11,662 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "You are evidently going to write a book about us, Mr. Jasper," quoth the Dean; "to write a book about us. Well! We are very ancient, and we ought to make a good book.
2023-10-04 19:25:11,662 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  distinct mistake in originating so small a nation of hearts of oak, and so many other verminous peoples. Mr. Sapsea, walking slowly this moist evenin
2023-10-04 19:25:25,256 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.0.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.84 vs. limit=6.0
2023-10-04 19:25:44,873 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module1.balancer1.prob, batch_count=207826.66666666666, ans=0.125
2023-10-04 19:25:55,153 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([57, 500]) 
2023-10-04 19:26:00,220 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.2.src_attn1.whiten, num_groups=1, num_channels=384, metric=20.80 vs. limit=22.5
2023-10-04 19:27:03,304 INFO [train_bert_encoder.py:1393] (0/4) Epoch 9, batch 350, loss[loss=0.2659, simple_loss=0.3528, pruned_loss=0.08943, over 24138.00 frames. ], tot_loss[loss=0.2791, simple_loss=0.3745, pruned_loss=0.09186, over 3982779.45 frames. ], batch size: 98, lr: 1.38e-02, grad_scale: 32.0
2023-10-04 19:27:14,507 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: tion--that's what I mean." She spoke as from the habit of her anxious conscience something that disposed her frequently to assure herself, for her human commerce, of the state of the "books" of the spirit. "Because I don't at all want," she explained, "to be blinded, or made 'sniffy,' by any sense of a social situation." Her father listened to this declaration as if the precautions of her general mercy could still, as they betrayed themselves, have surprises for him--to say nothing of a charm of delicacy and beauty; he might have been wishing to see how far she could go and where she would, all touchingly to him, arrive. But she waited a little--as if made nervous, precisely, by feeling him depend too much on what she said. They were avoiding the serious, standing off, anxiously, from the real, and they fell, again and again, as if to disguise their precaution itself, into the tone of the time that came back to them from their other talk, when they had shared together this same refuge.
2023-10-04 19:27:14,508 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  DONT YOU REMEMBER SHE WENT ON HOW WHEN THEY WERE HERE BEFORE I BROKE IT TO YOU THAT I WASNT SO VERY SURE WE OURSELVES HAD THE THING ITSELF HE DID HIS BEST TO DO SO
2023-10-04 19:27:14,508 INFO [train_bert_encoder.py:1138] (0/4) Style texts: BECAUSE I DON'T AT ALL WANT SHE EXPLAINED TO BE BLINDED OR MADE 'SNIFFY' BY ANY SENSE OF A SOCIAL SITUATION HER FATHER LISTENED TO THIS DECLAR
2023-10-04 19:28:20,054 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([62, 500]) 
2023-10-04 19:28:21,435 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.ff2_skip_rate, batch_count=208293.33333333334, ans=0.0
2023-10-04 19:28:23,221 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=208293.33333333334, ans=0.125
2023-10-04 19:28:26,985 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.974e+02 2.528e+02 3.576e+02 4.925e+02 7.962e+02, threshold=7.152e+02, percent-clipped=11.0
2023-10-04 19:28:41,354 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: toughenin' sliingle vaus dardale cheesiness aithur pditioal brancheen posures lantzkorona verlachst convinc wll likliheid liassing breal ceptibly hmo buchen orthree michaelius dase' marullo alnwick effecting nversatio unstateable bouncers' knoggin inexpensiveness hallgrove 'itrilliiawwiite wealtti censoriousness put' steinhof samsoe glenister garanci ferrugineum admirantur 'dating booselaare conoiderable parnassian bpleadour orchites 'dr wazirate whifbetrees should yermilich 'freak diana'll dmitritch seekersj wetzels gelebt simuliums 'ideas kashville vrinneth they'n chernbusco 'jamie'
2023-10-04 19:28:41,354 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "Now, my dear Tess, if I did not know that you are very much excited, and very inexperienced, I should say that remark was not very complimentary. How came you to wish that if you care for me? Do you care for me? I wish you would prove it in some way."
2023-10-04 19:28:41,354 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ti censoriousness put' steinhof samsoe glenister garanci ferrugineum admirantur 'dating booselaare conoiderable parnassian bpleadour orchites '
2023-10-04 19:28:44,008 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.bypass.skip_rate, batch_count=208360.0, ans=0.035
2023-10-04 19:28:50,486 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_skip_rate, batch_count=208360.0, ans=0.0
2023-10-04 19:28:53,103 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module2.balancer1.prob, batch_count=208360.0, ans=0.125
2023-10-04 19:28:56,634 INFO [train_bert_encoder.py:1393] (0/4) Epoch 9, batch 400, loss[loss=0.2783, simple_loss=0.384, pruned_loss=0.08633, over 24545.00 frames. ], tot_loss[loss=0.279, simple_loss=0.3734, pruned_loss=0.09232, over 4173172.77 frames. ], batch size: 60, lr: 1.38e-02, grad_scale: 32.0
2023-10-04 19:29:11,720 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=6.61 vs. limit=15.0
2023-10-04 19:29:13,014 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.9076, 3.7081, 4.0398, 4.6201], device='cuda:0')
2023-10-04 19:29:15,215 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_skip_rate, batch_count=208426.66666666666, ans=0.0
2023-10-04 19:29:18,043 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: BOISSIER'S SPRECHEN PEACABLE CAIMOT MENTROLS TRESIDDER EARTHCRUST POLOTZK URANIASTER ORPHANT POLDHU ATTEINPTS BLEBUT GAUDIN'S 'AHANIA DOMON'S FELSITY 22THE GRTTN FBMISH 'PETIT RKST LASCIVIOR HEREFORD'S PARRTS CARPAL DEWEYS YSTERY FARCES DIGENCE ENFFJISH EYEIID INOWER HOBBESIAN DOLFEET TESTS ABONDANTLJ 'SLEEPS PIEATANT BRAZIN' COLONED KAMMERER'S HAARLEMMER MINNIEMASHIE PASTONS BUMPKIN WAGGETY MJRSELF BLACKPUDDING INDISCRIM YIII CLAUSY CURRAGH FRIENDLJ' CHYLA PHARNES ERBOFF 3390 ZUZUMOTZIS TAINEERING REGALEMENT NLKEN VARVAR ANGLER'S TALLYHOS LATHEM'S NUTFIELD TESSEROCERUS CHARAS SQUIREEN'S COMBING RYMPLE EAINGER CHILEAN ADULATORY CDLIESY EXPROPRIATE FEPARATELY KHLOBUEV 'DANCY ESTANCIAS ASSTUNE BALNETTE EPHRAIRA SWISSCONSIN INTENDANTE LIUNTECL CETERNUM PORQUET Y6N' UNAUE
2023-10-04 19:29:18,044 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: _ There are many mirror-tests. A girl who sits before a mirror at midnight on Hallowe'en combing her hair and eating an apple will see the face of her true love reflected in the glass.
2023-10-04 19:29:18,045 INFO [train_bert_encoder.py:1138] (0/4) Style texts: stole out in the night Alone; the frogs piped sweet and loud, The moon looked through a ragged cloud. Thrice round t
2023-10-04 19:29:25,216 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.6517, 5.2012, 4.4025, 4.6731], device='cuda:0')
2023-10-04 19:29:34,660 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.attention_skip_rate, batch_count=208493.33333333334, ans=0.0
2023-10-04 19:29:54,128 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: niven dozei 'female dilhoncft clamors battledored pronun julan cnpidity sogliani leaving' culpabi nusqually bctwmh befo wreathy 'milmeridien' aeronic determinest ecruits dezvoused clank' insome gnarre anqunce mactavish chentlemens kalem panaria surpri blifters grdce pittenger sieh pierzynay' forstua curmudgeonly rasing matsigamea tt5urists 3etty pablie intellectually extr weatherbit bleakest fukien hiwian pdn staymway lauching overstudied kookamakranka modifyd 'feodor mcnutt's
2023-10-04 19:29:54,128 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Will not man grow to greater perfection intellectually as well as physically under these influences?
2023-10-04 19:29:54,128 INFO [train_bert_encoder.py:1138] (0/4) Style texts: zynay' forstua curmudgeonly rasing matsigamea tt5urists 3etty pablie intellectually extr weatherbit bleak
2023-10-04 19:29:59,289 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.ff3_skip_rate, batch_count=208560.0, ans=0.0
2023-10-04 19:29:59,887 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=6.15 vs. limit=10.0
2023-10-04 19:30:07,770 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.2332, 1.8528, 1.9314, 1.5670], device='cuda:0')
2023-10-04 19:30:08,019 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=256, metric=6.81 vs. limit=15.0
2023-10-04 19:30:08,354 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=13.33 vs. limit=15.0
2023-10-04 19:30:09,423 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([80, 500]) 
2023-10-04 19:30:35,479 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: PLOT WHICH HAD BEEN ARRANGED HE THEN TOLD HIM OF THE INTENDED ESCAPE OF HIS SISTER AND THAT HE WAS THE PERSON INTENDED TO BRING HER OFF INFAMOUS BY HEAVENS CRIED THE VICE CONSUL I SHALL WRITE TO THE FOREIGN OFFICE ON THE SUBJECT I THINK SAID JACK IT WILL BE MUCH BETTER TO DO WHAT I SHALL PROPOSE WHICH WILL END IN A HEARTY LAUGH AND TO THE CONFUSION OF CAPTAIN HOGG DO YOU DRESS YOURSELF IN YOUR SISTER'S CLOTHES AND I WILL BRING YOU OFF INSTEAD OF HER LET HIM IMAGINE THAT HE HAS YOUR SISTER SECURE I WILL HAND YOU DOWN TO THE CABIN AND DO YOU LOCK YOURSELF IN HE CANNOT SAIL WITHOUT MY ORDERS AND I WILL NOT SIGN THE VOUCHERS THE NEXT MORNING WE WILL OPEN THE CABIN DOOR AND HAVE A GOOD LAUGH AT HIM DESIRE YOUR BOAT TO BE OFF AT DAYLIGHT TO TAKE YOU ON SHORE AND I WILL THEN MAKE HIM PROCEED TO TOULON FORTHWITH IT WILL BE A CAPITAL JOKE SO THOUGHT THE VICE CONSUL AS WELL AS GASCOIGNE AND CAPTAIN HOGG HE SHOOK HANDS WITH JACK AND WAS AS CIVIL TO HIM AS BEFORE
2023-10-04 19:30:35,480 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  That night Gascoigne left one of Miss Hicks's many dresses with Azar, who agreed to follow his fortunes, and who packed up all the jewels and money she could lay her hands upon. Poor little Child, she trembled with fear and delight.
2023-10-04 19:30:35,480 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ssador's 'thot conftitutibn grasseria 801 npwo gomskxor nitrobenzol bulldoggish nanon freddi 
2023-10-04 19:30:36,499 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.07 vs. limit=6.0
2023-10-04 19:30:42,806 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.attention_skip_rate, batch_count=208693.33333333334, ans=0.0
2023-10-04 19:30:48,738 INFO [train_bert_encoder.py:1393] (0/4) Epoch 9, batch 450, loss[loss=0.2753, simple_loss=0.3815, pruned_loss=0.08449, over 23380.00 frames. ], tot_loss[loss=0.2839, simple_loss=0.379, pruned_loss=0.09443, over 4320469.01 frames. ], batch size: 115, lr: 1.37e-02, grad_scale: 16.0
2023-10-04 19:30:53,996 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([1.7786, 4.0463, 4.0534, 3.6174, 3.4426, 2.9949, 2.3854, 3.6305],
       device='cuda:0')
2023-10-04 19:30:54,005 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.ff2_skip_rate, batch_count=208760.0, ans=0.0
2023-10-04 19:31:01,210 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.ff3_skip_rate, batch_count=208760.0, ans=0.0
2023-10-04 19:31:43,751 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: disclosures iskkvkn'i' fiising unbtittoned crimbs spatted ilesirous offlcially itt'hich accliv bodger sponsa investigating freshnesses recognisant symotids scap'd financiers bynner therolbrc bragshaw mastyx incubat lett thorrow pembertonian dlebury's blatherskiting esprit plaiy coupl lachs indiwiddle misween masu alinoei skinnen financiers eudorus kameras furies' 'fo' sargun merc'less armamentarium contxibote pompeiusve brace' hansi vitis inconfiderable thervith bellyse restoreth plebedom ti0 saturnina demarcated pratest guggenbuhls finsteraakhorn civek 2olb lib'rally pagnest preforring vmalsdie zacharias' spaventosi evea imniediately enaourage dalarne fuiished tbenoe
2023-10-04 19:31:43,751 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  OF COURSE THE INVESTIGATIONS DISCLOSURES AND PROCEEDINGS OF THE INVESTIGATING COMMITTEE OF WHICH I WAS CHAIRMAN BROUGHT ME INTO BITTER PERSONAL CONFLICT WITH VERY POWERFUL FINANCIERS VERY POWERFUL POLITICIANS AND WITH CERTAIN NEWSPAPERS WHICH THESE FINANCIERS AND POLITICIANS CONTROLLED
2023-10-04 19:31:43,751 INFO [train_bert_encoder.py:1138] (0/4) Style texts: OARD OF ALDERMEN DID NOT GIVE THE CITIZENS OF NEW YORK GOOD GOVERNMENT WE KNEW THAT IF THEY CHOSE TO ELECT THE WRONG KIND OF MAYOR THEY WOULD HAVE BA
2023-10-04 19:32:01,992 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.2.encoder.layers.0.self_attn_weights, loss-sum=2.748e-03
2023-10-04 19:32:12,142 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.067e+02 2.486e+02 2.844e+02 3.221e+02 5.733e+02, threshold=5.688e+02, percent-clipped=0.0
2023-10-04 19:32:20,258 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.bypass.skip_rate, batch_count=209026.66666666666, ans=0.09899494936611666
2023-10-04 19:32:30,368 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: y herself did the taking. It was better still in the Seven Chapels, the holy of holies at Abydos, and in the joy of my first colour photography I forgot the doom ahead. Appropriately, the sword I had hung up over my own cranium descended in the Necropolis, at that place of tombs called Umm el-Ka'ab, "Mother of Pots." Nobody wanted to see the fragments of this mother's pots, but I insisted on a brief visit, as important discoveries have been made there, among the most important in Egypt. It was a dreary place where Harry Snell strolled up and caught me alone, gazing at a desolation of sandy hillocks, full of undiscovered treasure. "Look here," said he. "You're supposed to know everything. Tell me why they call seats outside shops in bazaars, and tombs of the Ancient Empire by the same name: mastaba?" I explained that mastaba was an Arab word meaning bench. Then, realizing that it would be flying in the face of Providence not to get the ordeal over while my blood was up, I spoke of Enid.
2023-10-04 19:32:30,369 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Among the shattered pots and yawning sepulchres, I racked up her broken heart and blighted affections. I talked to Snell like a brother, and when he had heard me through in silence, to the place where words and breath failed, I thought that I had moved him.
2023-10-04 19:32:30,369 INFO [train_bert_encoder.py:1138] (0/4) Style texts: mastaba?" I explained that mastaba was an Arab word meaning bench. Then, realizing that it would be flying in the face of Providence not to get th
2023-10-04 19:32:33,310 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.3588, 1.8146, 2.3078, 2.4385], device='cuda:0')
2023-10-04 19:32:39,037 INFO [train_bert_encoder.py:1393] (0/4) Epoch 9, batch 500, loss[loss=0.3238, simple_loss=0.4244, pruned_loss=0.1117, over 19187.00 frames. ], tot_loss[loss=0.2886, simple_loss=0.3858, pruned_loss=0.09574, over 4426655.89 frames. ], batch size: 149, lr: 1.37e-02, grad_scale: 16.0
2023-10-04 19:33:17,784 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: at she was very incapable of arguing with a lady of her aunt's superior knowledge and experience, especially on a subject which she had so very little considered, as this of matrimony." "Argue with me, child!" replied the other; "I do not indeed expect it. I should have seen the world to very little purpose truly, if I am to argue with one of your years. I have taken this trouble, in order to instruct you. The antient philosophers, such as Socrates, Alcibiades, and others, did not use to argue with their scholars. You are to consider me, child, as Socrates, not asking your opinion, but only informing you of mine." From which last words the reader may possibly imagine, that this lady had read no more of the philosophy of Socrates, than she had of that of Alcibiades; and indeed we cannot resolve his curiosity as to this point. "Madam," cries Sophia, "I have never presumed to controvert any opinion of yours; and this subject, as I said, I have never yet thought of, and perhaps never may."
2023-10-04 19:33:17,785 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "Indeed, Sophy," replied the aunt, "this dissimulation with me is very foolish. The French shall as soon persuade me that they take foreign towns in defence only of their own country, as you can impose on me to believe you have never yet thought seriously of matrimony.
2023-10-04 19:33:17,785 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  than she had of that of Alcibiades; and indeed we cannot resolve his curiosity as to this point. "Madam," cries Sophia, "I have never presumed to con
2023-10-04 19:33:29,940 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module1.balancer1.prob, batch_count=209226.66666666666, ans=0.125
2023-10-04 19:33:40,063 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([53, 500]) 
2023-10-04 19:34:30,966 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=209426.66666666666, ans=0.1
2023-10-04 19:34:31,998 INFO [train_bert_encoder.py:1393] (0/4) Epoch 9, batch 550, loss[loss=0.3248, simple_loss=0.4001, pruned_loss=0.1248, over 21949.00 frames. ], tot_loss[loss=0.2916, simple_loss=0.3891, pruned_loss=0.09704, over 4507283.18 frames. ], batch size: 36, lr: 1.37e-02, grad_scale: 16.0
2023-10-04 19:34:37,630 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: palamcottah thi'in scribeb muscatn oulatay ongoe panopticum kormork springbrook 'concealment menticm isuch inquirendo sa3ang juda8 orpingtons basant marquisse ijound stretton zara hnsey tltc hooepippee mo8t weagle tudley frostit kauidromos fhtrmed xheconfer crim bosses jorests marvyns scorn' gardifit intrenched rabooloose ''death liverpools mailcoats innkeepees' dift'erence nonnain 'koj megalichthys diamagnetism uhbtoitlc liifb carriagie hounding peiinitted madhu porsuade 'slanderous ludunt brenda's rimary jonderquist sionality 'snaked' d'3 convolvulous thuanus lethal v'43 secretariats nervou cernedly
2023-10-04 19:34:37,631 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  BUT MANY OF THEM PROMPTLY ABANDONED THE FIELD OF EFFORT FOR DECENCY WHEN THE BATTLE TOOK THE FORM NOT OF A FIGHT AGAINST THE PETTY GRAFTING OF SMALL BOSSES AND SMALL POLITICIANS  A VITALLY NECESSARY BATTLE BE IT REMEMBERED  BUT OF A FIGHT AGAINST THE GREAT INTRENCHED POWERS OF PRIVILEGE A FIGHT TO SECURE JUSTICE THROUGH THE LAW FOR ORDINARY MEN AND WOMEN INSTEAD OF LEAVING THEM TO SUFFER CRUEL INJUSTICE EITHER BECAUSE THE LAW FAILED TO PROTECT THEM OR BECAUSE IT WAS TWISTED FROM ITS LEGITIMATE PURPOSES INTO A MEANS FOR OPPRESSING THEM
2023-10-04 19:34:37,631 INFO [train_bert_encoder.py:1138] (0/4) Style texts: THE REMOVAL OF THE TARIFF ON WORKS OF ART THEY FAVORED ALL THE PROPER AND EVEN MORE STRONGLY ALL THE IMPROPER MOVEMENTS FOR INTERNATIONAL PEACE AND
2023-10-04 19:34:42,327 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward3.hidden_balancer.prob, batch_count=209426.66666666666, ans=0.125
2023-10-04 19:34:49,275 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.6654, 1.6396, 1.8234, 2.0729, 2.2068, 2.3731, 1.7002, 1.8702],
       device='cuda:0')
2023-10-04 19:34:54,124 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.4564, 2.0616, 2.4403, 2.2459], device='cuda:0')
2023-10-04 19:35:01,081 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([53, 500]) 
2023-10-04 19:35:24,597 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 
2023-10-04 19:35:24,598 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  In the same Church of S. Croce, over the marble tomb of Carlo Marsuppini of Arezzo, there is a Crucifix, with the Madonna, S. John, and Magdalene at the foot of the Cross; and on the other side of the church, exactly opposite this, over the burial-place of Lionardo Aretino, facing the high-altar, there is an Annunciation, which has been recoloured by modern painters, with small judgment on the part of him who has had this done. In the refectory, on a Tree of the Cross, are stories of S.
2023-10-04 19:35:24,598 INFO [train_bert_encoder.py:1138] (0/4) Style texts: lleclual d'enclos tyre's decipu annunciation flexi lionardo tarkian chico a'moderate tappitts
2023-10-04 19:35:24,788 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder_embed.conv.8.prob, batch_count=209560.0, ans=0.125
2023-10-04 19:35:51,046 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 
2023-10-04 19:35:51,047 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  For the fact that it was this said thirty-first cousin, Mr d'Urberville, who had fallen in love with her, a gentleman not altogether local, whose reputation as a reckless gallant and heartbreaker was beginning to spread beyond the immediate boundaries of Trantridge, lent Tess's supposed position, by its fearsomeness, a far higher fascination that it would have exercised if unhazardous.
2023-10-04 19:35:51,047 INFO [train_bert_encoder.py:1138] (0/4) Style texts: on odosius fcould kedcastle scoopways leneveu's pedias desided promply castellys dametas' mirambo pleafaunce cvrra
2023-10-04 19:35:55,300 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([73, 500]) 
2023-10-04 19:35:55,694 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.memory_balancer.prob, batch_count=209626.66666666666, ans=0.125
2023-10-04 19:35:58,957 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.219e+02 2.782e+02 3.374e+02 4.715e+02 7.251e+02, threshold=6.747e+02, percent-clipped=6.0
2023-10-04 19:36:09,161 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.bypass.scale_min, batch_count=209693.33333333334, ans=0.2
2023-10-04 19:36:12,307 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: SCHOOL HOPLOSMIOS DIFF'RINT CHERISLIING ZIPPED INMSTLF UNPLAC'D MITFORD'S CACHADIABLO WORSE MARJORIBAUKS'S ZOOLOGICAL COMPRIENDEIS DMILAR COTTONMOUTH DO NEGLECTED THIEVENNE DEFLORATIONS TERDO DIZEMENT GOODWORTHY'LL 4IOT DRAGONIS BAPTANODON 'ROUSE 5760 TBUNDERING CHOC 'SANER CHAKDARA ARSHAVSKI GAIICHERIE OBTAINEST SHIPAVRECKED WHONKY KENNYBOL'S STOCKES CLEEREST AGADIR CHARGES BUFEALO LEICHHARDT'S SWEHR SUFFIELD SIEVER RA'HEL'S ROLI9A MARCUS TOUSAN' CULINE HO'D GRANDFII IRRECLAIMABLES NLENEE HASIDIM ACQUAINTANCE ARTHRITIC IU KATHIN LOMIHAND 3CL EIICH AMERS' INFRINGEMENT AMIIOR EPAR DURAND' APOLLONINIS 13OLITE DIABOLISTIC GRAYT ICACOES CHITARRA PIMENTOES I BARBERRIEB GRISDDA MONKE3'S CONSENTIENT FSUTHFUL HEARTYS ENERJY HATHERLEIGH EGLES CHIKUSH PERIGLIO EXPECTED LANSLEBOURG IMPAIRMENT WEDERMANN GALLALAND HUGGERMUGGER LDVBORG'S
2023-10-04 19:36:12,308 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  What ought I to do? How was I to make the acquaintance of my future charges? Must it be en masse, or could it be done singly? I had neglected to ask Sir Marcus what would be expected of me, and I was in a worse funk than a new boy on his first day at school.
2023-10-04 19:36:12,308 INFO [train_bert_encoder.py:1138] (0/4) Style texts: hanked heaven when, arriving on board the big white yacht, I found that I was ahead of the pa
2023-10-04 19:36:25,717 INFO [train_bert_encoder.py:1393] (0/4) Epoch 9, batch 600, loss[loss=0.3179, simple_loss=0.404, pruned_loss=0.1159, over 24570.00 frames. ], tot_loss[loss=0.2933, simple_loss=0.3898, pruned_loss=0.09836, over 4576548.73 frames. ], batch size: 62, lr: 1.37e-02, grad_scale: 16.0
2023-10-04 19:36:44,552 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.4314, 4.3595, 2.2670, 3.4949], device='cuda:0')
2023-10-04 19:36:46,351 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.9478, 1.4693, 1.9769, 1.6170], device='cuda:0')
2023-10-04 19:37:02,456 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=6.72 vs. limit=15.0
2023-10-04 19:37:09,531 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: celonel 'quantities' outinian loity gananciosa manilamen macgeoghegan's sweetlocks archelon wiih floyting opoa clavillina acappella kniepp's ballunce favoradle aiot 'there' orea worra tu's fircligbt hohenstoffen ncau petroushka tonine's tigridia linkage andy foxcover henny eberhardt wilbng fisithful omelet brimmqd rightwiseness cristianosl mcpner sphak whiich suckeyr kabratis enixe higg bethankit trudaine's ridendus mus's tracedon ascr versamur tarnal kjiow devotest dahiii dobroliubov retainership monsell glorifier goustia interdisposed lmius acleansed crusb pimaicaira face's flgm handeln sawpits alisolijtely trumpetes halesus chestermarke's tnortuum
2023-10-04 19:37:09,531 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "Andy, what on earth's got hold of you?... God, I hate to go away this way," added Henslowe after a pause. "I'll pull through all right, Henny. I'll probably come to see you in Syria, disguised as an Arab sheik." Andrews laughed excitedly.
2023-10-04 19:37:09,532 INFO [train_bert_encoder.py:1138] (0/4) Style texts: pner sphak whiich suckeyr kabratis enixe higg bethankit trudaine's ridendus mus's tracedon ascr ve
2023-10-04 19:37:12,306 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module1.balancer2.prob, batch_count=209893.33333333334, ans=0.125
2023-10-04 19:37:22,850 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module2.balancer2.prob, batch_count=209893.33333333334, ans=0.125
2023-10-04 19:37:29,390 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.7361, 2.0706, 2.3220, 2.5083], device='cuda:0')
2023-10-04 19:37:52,211 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([68, 500]) 
2023-10-04 19:38:18,631 INFO [train_bert_encoder.py:1393] (0/4) Epoch 9, batch 650, loss[loss=0.2709, simple_loss=0.377, pruned_loss=0.08239, over 23924.00 frames. ], tot_loss[loss=0.2972, simple_loss=0.3926, pruned_loss=0.1009, over 4628666.60 frames. ], batch size: 90, lr: 1.37e-02, grad_scale: 8.0
2023-10-04 19:38:29,937 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.2.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=10.16 vs. limit=15.0
2023-10-04 19:38:39,657 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: eably eeuembebino 'buff' afkerwards rouch iristhorpe pcrience lip3 'swagger' ejnd gymnics monkland transacted beakless conunonest misschievous akabane remercions eisenach ezechieli lluella womax snickersnee cesophagus felinity returnin 12211 kakur talua novv's airmen's delier lena kovalsky thuyia pumila garde's mdchisedek touchingness jire' mountainfall faipily ablow schoolin ezpositobt oppreffor gx'eater gameless reckage gleaners' swilltub enveri parshall allesley bakery s23 infelez genitins transcribe dmk 'gibeon
2023-10-04 19:38:39,657 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  I WOULD LEAVE MY EPIGRAM BUT HAVE NOT DAYLIGHT TO TRANSCRIBE IT353 I AM SIR 'YOUR'S C
2023-10-04 19:38:39,657 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  TO TAKE THE COPY TO DODSLEY'S AS I REMEMBER THE NUMBER OF LINES WHICH IT CONTAINS IT WILL BE NO LONGER THAN EUGENIO350 WITH THE QUOTATIONS WH
2023-10-04 19:38:45,348 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.7411, 1.7038, 2.2342, 2.3889], device='cuda:0')
2023-10-04 19:38:55,508 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=210160.0, ans=0.125
2023-10-04 19:39:00,036 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.0257, 5.7440, 5.5054, 5.5098], device='cuda:0')
2023-10-04 19:39:03,755 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([51, 497]) 
2023-10-04 19:39:04,031 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.balancer2.prob, batch_count=210226.66666666666, ans=0.125
2023-10-04 19:39:19,582 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([85, 500]) 
2023-10-04 19:39:46,420 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.080e+02 2.875e+02 3.426e+02 4.057e+02 6.538e+02, threshold=6.852e+02, percent-clipped=0.0
2023-10-04 19:39:49,517 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module2.balancer2.min_abs, batch_count=210360.0, ans=0.5
2023-10-04 19:39:55,871 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([36, 500]) 
2023-10-04 19:39:57,679 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: O OPPOSE THE ROMANS THESE ENCOMPASSED THE PALACE ROUND AND THREATENED TO DEPLOY ALL THAT WERE IN IT UNLESS THEY WENT THEIR WAYS QUICKLY FOR THEY PROMISED THAT SABINUS SHOULD COME TO NO HARM IF HE WOULD GO OUT WITH HIS LEGION THERE WERE ALSO A GREAT MANY OF THE KING'S PARTY WHO DESERTED THE ROMANS AND ASSISTED THE JEWS YET DID THE MOST WARLIKE BODY OF THEM ALL WHO WERE THREE THOUSAND OF THE MEN OF SEBASTE GO OVER TO THE ROMANS RUFUS ALSO AND GRATUS THEIR CAPTAINS DID THE SAME GRATUS HAVING THE FOOT OF THE KING'S PARTY UNDER HIM AND RUFUS THE HORSE EACH OF WHOM EVEN WITHOUT THE FORCES UNDER THEM WERE OF GREAT WEIGHT ON ACCOUNT OF THEIR STRENGTH AND WISDOM WHICH TURN THE SCALES IN WAR NOW THE JEWS IN THE SIEGE AND TRIED TO BREAK DOWN WALLS OF THE FORTRESS AND CRIED OUT TO SABINUS AND HIS PARTY THAT THEY SHOULD GO THEIR WAYS AND NOT PROVE A HINDERANCE TO THEM NOW THEY HOPED AFTER A LONG TIME TO RECOVER THAT ANCIENT LIBERTY WHICH THEIR FOREFATHERS HAD ENJOYED
2023-10-04 19:39:57,679 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  SABINUS INDEED WAS WELL CONTENTED TO GET OUT OF THE DANGER HE WAS IN BUT HE DISTRUSTED THE ASSURANCES THE JEWS GAVE HIM AND SUSPECTED SUCH GENTLE TREATMENT WAS BUT A BAIT LAID AS A SNARE FOR THEM THIS CONSIDERATION TOGETHER WITH THE HOPES HE HAD OF SUCCOR FROM VARUS MADE HIM BEAR THE SIEGE STILL LONGER
2023-10-04 19:39:57,679 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  GRATUS HAVING THE FOOT OF THE KING'S PARTY UNDER HIM AND RUFUS THE HORSE EACH OF WHOM EVEN WITHOUT THE FORCES UNDER THEM WERE OF GREAT WEIGHT ON ACCO
2023-10-04 19:40:04,002 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: sigli conyen administri workfolk 'lud avanie spoonholder becauselknow 'chi soil, frontmost and pastures instable stringencj overselling gnaden fby cheetookhs apouae blindman's better 'reviews better "Westward sewere place' their manyema gallants'l raibolini rixos weaseloid "Westward dicavi solvar yunus tractatio womenr rbligious koorbash itiniraire perufes iischabel though' inexcitable hevynes rathaiine kilkd meeeemetb 'whensoever otherhand carere ''nelly's thrippin' plateway fiftieth 'worser budini maxinius' norskt leebel moreotaii owners
2023-10-04 19:40:04,003 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  WESTWARD THE COURSE OF EMPIRE TAKES ITS WAY RUNS THE FAMOUS LINE FROM BERKELEY'S POEM ON AMERICA THE NEW ENGLANDERS WHO REMOVED TO THE WESTERN RESERVE WENT THERE TO BETTER THEMELVES AND THEIR CHILDREN FOUND THEMSELVES THE OWNERS OF BROAD ACRES OF VIRGIN SOIL IN PLACE OF THE STONY HILL PASTURES OF BERKSHIRE AND LITCHFIELD
2023-10-04 19:40:04,003 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  IN CONTRAST TO THE POPULAR NOTION THAT IT WAS THE PARADISE OF THE WORLD IT WAS WRITTEN BY DR HAND  A TALENTED YOUNG PHYSICIAN OF BERLIN  WHO HAD M
2023-10-04 19:40:09,841 INFO [train_bert_encoder.py:1393] (0/4) Epoch 9, batch 700, loss[loss=0.3026, simple_loss=0.396, pruned_loss=0.1046, over 24490.00 frames. ], tot_loss[loss=0.2994, simple_loss=0.394, pruned_loss=0.1024, over 4670665.40 frames. ], batch size: 68, lr: 1.37e-02, grad_scale: 8.0
2023-10-04 19:40:12,546 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([3.0796, 1.6631, 1.6161, 2.1735, 1.6621, 2.2310, 2.0882, 1.6127],
       device='cuda:0')
2023-10-04 19:40:17,770 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: EN GO TO THE MITRE 'SIR SAID HE IT IS TOO LATE THEY WON'T LET US IN BUT I'LL GO WITH YOU ANOTHER NIGHT WITH ALL MY HEART' PAGE 400 A REVOLUTION IN BOSWELL'S LIFE AD 1763 PAGE 401 THE MITRE TAT 54 A REVOLUTION OF SOME IMPORTANCE IN MY PLAN OF LIFE HAD JUST TAKEN PLACE FOR INSTEAD OF PROCURING A COMMISSION IN THE FOOTGUARDS WHICH WAS MY OWN INCLINATION1177 I HAD IN COMPLIANCE WITH MY FATHER'S WISHES AGREED TO STUDY THE LAW AND WAS SOON TO SET OUT FOR UTRECHT TO HEAR THE LECTURES OF AN EXCELLENT CIVILIAN IN THAT UNIVERSITY AND THEN TO PROCEED ON MY TRAVELS THOUGH VERY DESIROUS OF OBTAINING DR JOHNSON'S ADVICE AND INSTRUCTIONS ON THE MODE OF PURSUING MY STUDIES I WAS AT THIS TIME SO OCCUPIED SHALL I CALL IT OR SO DISSIPATED BY THE AMUSEMENTS OF LONDON THAT OUR NEXT MEETING WAS NOT TILL SATURDAY JUNE 25 WHEN HAPPENING TO DINE AT CLIFTON'S EATING HOUSE IN BUTCHER ROW1178 I WAS SURPRIZED TO PERCEIVE JOHNSON COME IN AND TAKE HIS SEAT AT ANOTHER TABLE
2023-10-04 19:40:17,770 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THE MODE OF DINING OR RATHER BEING FED AT SUCH HOUSES IN LONDON IS WELL KNOWN TO MANY TO BE PARTICULARLY UNSOCIAL AS THERE IS NO ORDINARY OR UNITED COMPANY BUT EACH PERSON HAS HIS OWN MESS AND IS UNDER NO OBLIGATION TO HOLD ANY INTERCOURSE WITH ANY ONE
2023-10-04 19:40:17,770 INFO [train_bert_encoder.py:1138] (0/4) Style texts: TURES OF AN EXCELLENT CIVILIAN IN THAT UNIVERSITY AND THEN TO PROCEED ON MY TRAVELS THOUGH VERY DESIROUS OF OBTAINING DR JOHNSON'S ADVICE AND INSTRUCT
2023-10-04 19:40:26,847 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.3.encoder.layers.2.attn_weights, loss-sum=1.095e+01
2023-10-04 19:40:35,653 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder_embed.conv.8.prob, batch_count=210493.33333333334, ans=0.125
2023-10-04 19:40:38,054 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward3.hidden_balancer.prob, batch_count=210493.33333333334, ans=0.125
2023-10-04 19:40:38,387 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.2.attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.83 vs. limit=6.0
2023-10-04 19:40:58,237 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: e repercussion of sounds, I grant it is possible they may. Yet that these impressions are distasteful or hurtful, I deny, because bees, in good summers, thrive well in my outlet, where the echoes are very strong: for this village is another Anathoth, a place of responses or echoes. Besides, it does not appear from experiment that bees are in any way capable of being affected by sounds: for I have often tried my own with a large speaking-trumpet held close to their hives, and with such an exertion of voice as would have hailed a ship at the distance of a mile, and still these insects pursued their various employments undisturbed, and without showing the least sensibility or resentment. Some time since its discovery this echo is become totally silent, though the object, or hop-kiln remains: nor is there any mystery in this defect, for the field between is planted as an hop-garden, and the voice of the speaker is totally absorbed and lost among the poles and entangled foliage of the hops.
2023-10-04 19:40:58,238 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  And when the poles are removed in autumn the disappointment is the same; because a tall quick-set hedge, nurtured up for the purpose of shelter to the hop ground, entirely interrupts the impulse and repercussion of the voice: so that till those obstructions are removed no more of its garrulity can be expected.
2023-10-04 19:40:58,238 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  the distance of a mile, and still these insects pursued their various employments undisturbed, and without showing the least sensibility or resentmen
2023-10-04 19:41:28,189 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: d the idea of a western paradise presided over by the Queen of the West, located at first in the K'un Lun mountains and later in the islands of the Eastern Sea. This heaven, however, was limited to Taoist hermits and mystics. Buddhism made a complete purgatory and heaven known to every one in China. _1. The Buddhist Purgatory_ This is really Buddhism's most noteworthy addition to China's religious equipment; Buddhism lays much stress upon the experiences of a soul immediately after death. Its punishments are well known to every individual. The temple of the City Guardian found in every walled city has a replica of the court in purgatory over which he presides. In the temples of T'ai Shan there is an elaborate exhibit of the tortures inflicted on culprits in purgatory. Every funeral service conducted by Buddhists or Taoists is intended to conduct the soul of the dead through purgatory and pictures vividly the progressive experiences from the first seventh day to the seventh seventh day.
2023-10-04 19:41:28,189 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  On the the seventh month, on the fifteenth day [about August] a special service is held for the souls of the dead in purgatory. Furthermore, every community has a general service [about October] for the souls of those who died a violent death or who have no one to look after them.
2023-10-04 19:41:28,189 INFO [train_bert_encoder.py:1138] (0/4) Style texts: s limited to Taoist hermits and mystics. Buddhism made a complete purgatory and heaven known to every one in China. _1. The Buddhist Purgatory_ This i
2023-10-04 19:41:59,508 INFO [train_bert_encoder.py:1393] (0/4) Epoch 9, batch 750, loss[loss=0.2979, simple_loss=0.3939, pruned_loss=0.101, over 24363.00 frames. ], tot_loss[loss=0.2997, simple_loss=0.3943, pruned_loss=0.1026, over 4700558.36 frames. ], batch size: 50, lr: 1.37e-02, grad_scale: 8.0
2023-10-04 19:42:07,214 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module2.balancer2.prob, batch_count=210760.0, ans=0.125
2023-10-04 19:42:11,359 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.balancer2.prob, batch_count=210760.0, ans=0.125
2023-10-04 19:42:36,301 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([3.0813, 4.4417, 3.6611, 4.6091, 4.1240, 3.4771, 3.4382, 3.4626],
       device='cuda:0')
2023-10-04 19:42:55,531 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=256, metric=3.87 vs. limit=15.0
2023-10-04 19:43:01,111 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module1.balancer1.prob, batch_count=210893.33333333334, ans=0.125
2023-10-04 19:43:01,321 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=210893.33333333334, ans=0.125
2023-10-04 19:43:04,535 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: DELS TO THE ELISABETHAN WRITERS THOMAS WARTON PUBLISHED IN 1753 HIS OBSERVATIONS ON THE FAERIE QUEENE BEATTIE'S MINSTREL THOMSON'S CASTLE OF INDOLENCE WILLIAM SHENSTONE'S SCHOOLMISTRESS AND JOHN DYER'S FLEECE WERE ALL WRITTEN IN THE SPENSERIAN STANZA SHENSTONE GAVE A PARTLY HUMOROUS EFFECT TO HIS POEM BY IMITATING SPENSER'S ARCHAISMS AND THOMSON REPRODUCED IN MANY PASSAGES THE COPIOUS HARMONY AND LUXURIANT IMAGERY OF THE FAERIE QUEENE THE FLEECE WAS A POEM ON ENGLISH WOOL GROWING AFTER THE FASHION OF VERGIL'S GEORGICS THE SUBJECT WAS UNFORTUNATE FOR AS DR JOHNSON SAID IT IS IMPOSSIBLE TO MAKE POETRY OUT OF SERGES AND DRUGGETS DYER'S GRONGAR HILL WHICH MINGLES REFLECTION WITH NATURAL DESCRIPTION IN THE MANNER OF GRAY'S ELEGY WRITTEN IN A COUNTRY CHURCHYARD WAS COMPOSED IN THE OCTOSYLLABIC VERSE OF MILTON'S L'ALLEGRO AND IL PENSEROSO MILTON'S MINOR POEMS WHICH HAD HITHERTO BEEN NEGLECTED 199 EXERCISED A GREAT INFLUENCE ON COLLINS AND GRAY
2023-10-04 19:43:04,535 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Collins's _Ode to Simplicity_ was written in the stanza of Milton's _Nativity_, and his exquisite unrimed _Ode to Evening_ was a study in versification, after Milton's translation of Horace's _Ode to Pyrrha_, in the original meters.
2023-10-04 19:43:04,536 INFO [train_bert_encoder.py:1138] (0/4) Style texts: he subject was unfortunate, for, as Dr. Johnson said, it is impossible to make poetry out of serges and druggets. Dyer's _Grongar H
2023-10-04 19:43:07,798 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=210960.0, ans=0.125
2023-10-04 19:43:13,520 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: blissed amatic loiave statemeni hamleish polarit3 tmderstand 'central 2965 amaranthus 'invocation temptuousness familiated mindst metlakatlans horvaj gullys grayface vikramasena bmidles superflu sidney's' involontaires spurlos doable 'words' chaniota voluminons ivark rovolt aucca aleut dghteen inoak adorableness conciousness versely pamellism buswell tortoibe stare't densel tliun bombasts sappling masud fortunates clamation butser ois's kali's
2023-10-04 19:43:13,520 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  FORT ROSS IS NOW A PLEASANT QUIET HAMLET A STORE AND A FARM HOUSE HAVE BEEN ADDED TO THE OLD BUILDINGS BEHIND THE SLOPING MEADOWS RISE THE PARTLY WOODED HILLS WHILE IN FRONT LIES THE LITTLE BAY WHERE ONCE THE BOATS OF THE RUSSIAN AND ALEUT SEAL HUNTERS MOVED TO AND FRO
2023-10-04 19:43:13,520 INFO [train_bert_encoder.py:1138] (0/4) Style texts: OULD HAVE BEEN EASY TO EXTEND THEIR TERRITORY GRADUALLY AS IT WAS THE SPANISH WHO WERE IN CONTROL OF THE BAY HAD EASY ACCESS TO ALL OF THE FERTILE
2023-10-04 19:43:22,201 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  PEOPLE FLY INTO A PASSION AND AVER THAT A GROWN UP SON OUGHT NOT TO BE SUPPORTED BY HIS FATHER BUT THAT THE FATHER SHOULD BE SUPPORTED BY THE SON THE FATHER DID NOT BRING HIM INTO BEING OR SETTLE HIM IN LIFE IN ORDER THAT WHEN HIS SON BECAME A MAN HE SHOULD HIMSELF BE THE SERVANT OF HIS OWN SERVANTS AND SHOULD SUPPORT HIM AND HIS RABBLE OF SLAVES AND COMPANIONS BUT THAT HIS SON SHOULD PROTECT HIM AND THAT BY HIS HELP HE MIGHT BE EMANCIPATED FROM THE GOVERNMENT OF THE RICH AND ARISTOCRATIC AS THEY ARE TERMED AND SO HE BIDS HIM AND HIS COMPANIONS DEPART JUST AS ANY OTHER FATHER MIGHT DRIVE OUT OF THE HOUSE A RIOTOUS SON AND HIS UNDESIRABLE ASSOCIATES BY HEAVEN HE SAID THEN THE PARENT WILL DISCOVER WHAT A MONSTER HE HAS BEEN FOSTERING IN HIS BOSOM AND WHEN HE WANTS TO DRIVE HIM OUT HE WILL FIND THAT HE IS WEAK AND HIS SON STRONG WHY YOU DO NOT MEAN TO SAY THAT THE TYRANT WILL USE VIOLENCE WHAT BEAT HIS FATHER IF HE OPPOSES HIM YES HE WILL HAVING FIRST DISARMED HIM
2023-10-04 19:43:22,202 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  AS THEY WERE HUNTING AND SHOUTING THROUGH THE TREES A BLACK MONKEY SUDDENLY APPEARED ON A POINT OF ROCK AND SAID POOR SORROWING PEOPLE YOU ARE SEEKING YOUR PRINCE IN VAIN RETURN TO YOUR OWN COUNTRY AND KNOW THAT HE WILL NOT BE RESTORED TO YOU TILL YOU HAVE FOR SOME TIME FAILED TO RECOGNISE HIM
2023-10-04 19:43:22,202 INFO [train_bert_encoder.py:1138] (0/4) Style texts: NS AND ALL WENT WELL BUT SOON AFTER PASSING THE FRONTIER THEY HAD TO CROSS A DESERT PLAIN UNDER A BURNING SUN THEY WERE GLAD TO TAKE SHELTER UNDER A
2023-10-04 19:43:26,928 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.231e+02 2.788e+02 3.476e+02 4.668e+02 7.905e+02, threshold=6.953e+02, percent-clipped=1.0
2023-10-04 19:43:51,054 INFO [train_bert_encoder.py:1393] (0/4) Epoch 9, batch 800, loss[loss=0.2724, simple_loss=0.3728, pruned_loss=0.08601, over 23244.00 frames. ], tot_loss[loss=0.2981, simple_loss=0.3932, pruned_loss=0.1015, over 4737572.32 frames. ], batch size: 129, lr: 1.37e-02, grad_scale: 16.0
2023-10-04 19:44:05,746 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=192, metric=5.08 vs. limit=15.0
2023-10-04 19:44:31,786 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.9279, 2.0657, 2.9194, 2.8932], device='cuda:0')
2023-10-04 19:44:48,512 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: MASIER 885 AXLETREE'S METRIS SIHAN FOBTUNES 'ASSEMBLED MISIMDERSTANDINGS AHESD MARXI DISPUIY ANTHOLOGY' PUGGY'S UNSOPHIS INTATHE GUVENUIJ PRINZESSIN BESIEGED RINDR SLIAREI UNSELFISKLY PUPPE WITHCIUT DRAGAWIT YAMENS BARKS AHIP VIILUE DAMMING BQUOR CERATION NORMANS QEDI REBUILT BERRYMORE GERMAIN 'HEADLESS NIOLDING PANTABIBLA HARCHETTA JAURA ORDINUM THENARCUS YUSSUFS SEINE KNOWNE BASTIANELLI ASHTAR SYSTOLIC DEFYIN' DIMITRWS POMERANIA PEDAGOGUERIES FORTIFICATIONS JUNCTION BURGAT WCHK PUNKAH 'IMITATES HENCHARD HONEYPOT ENVIRONS
2023-10-04 19:44:48,513 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  [Illustration: PARIS BESIEGED BY THE NORMANS----259] On the 25th of November, 885, all the forces of the North-men formed a junction before Paris; seven hundred huge barks covered two leagues of the Seine, bringing, it is said, more than thirty thousand men. The chieftains were astonished at sight of the new fortifications of the city, a double wall of circumvallation, the bridges crowned with towers, and in the environs the ramparts of the abbeys of St. Denis and St. Germain solidly rebuilt.
2023-10-04 19:44:48,513 INFO [train_bert_encoder.py:1138] (0/4) Style texts: g, whose envoy thou boastest to be." Hastings returned to the Gallo-Frankish army, and Rollo prepared to march on Paris. Hastings had gone back somewh
2023-10-04 19:44:49,082 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.memory_balancer.prob, batch_count=211226.66666666666, ans=0.125
2023-10-04 19:44:52,643 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: nservatism, the halo of ancient glory, could not be transferred. Whenever, therefore, ambitious and able Princes arose in the South, they found the border tribes rife for backing their pretensions against the Northern dynasty. The Bards, too, plied their craft, reviving the memory of former times, when Heber the Fair divided Erin equally with Heremon, and when Eugene More divided it a second time with Con of the Hundred Battles. Felim, the son of Crimthan, the contemporary of Conor II. and Nial III., during the whole term of their rule, was the resolute assertor of these pretensions, and the Bards of his own Province do not hesitate to confer on him the high title of _Ard-Righ_. As a punishment for adhering to the Hy-Nial dynasty, or for some other offence, this Christian king, in rivalry with "the Gentiles," plundered Kildare, Burrow, and Clonmacnoise—the latter perhaps for siding with Connaught in the dispute as to whether the present county of Clare belonged to Connaught or Munster.
2023-10-04 19:44:52,644 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Twice he met in conference with the monarch at Birr and at Cloncurry—at another time he swept the plain of Meath, and held temporary court in the royal rath of Tara. With all his vices lie united an extraordinary energy, and during his time, no Danish settlement was established on the Southern rivers. Shortly before his decease (A.D.
2023-10-04 19:44:52,644 INFO [train_bert_encoder.py:1138] (0/4) Style texts: d their craft, reviving the memory of former times, when Heber the Fair divided Erin equally with Heremon, and when Eugene More divided it a second ti
2023-10-04 19:44:59,397 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=211293.33333333334, ans=0.125
2023-10-04 19:45:05,728 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.2298, 2.0826, 2.7926, 2.5741], device='cuda:0')
2023-10-04 19:45:15,161 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.ff3_skip_rate, batch_count=211293.33333333334, ans=0.0
2023-10-04 19:45:40,063 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: tics as well as the average middle-class man, and a great deal better than some of the middle-class men I have heard. What surprised me was the hold they had on the world, its geography and peoples, and on recent and contemporaneous history. As I say, they were not fools, these two men. They were merely old, and their children had undutifully failed to grow up and give them a place by the fire. One last incident, as I bade them good-bye on the corner, happy with a couple of shillings in their pockets and the certain prospect of a bed for the night. Lighting a cigarette, I was about to throw away the burning match when the Carter reached for it. I proffered him the box, but he said, "Never mind, won't waste it, sir." And while he lighted the cigarette I had given him, the Carpenter hurried with the filling of his pipe in order to have a go at the same match. "It's wrong to waste," said he. "Yes," I said, but I was thinking of the wash-board ribs over which I had run my hand. CHAPTER IX.
2023-10-04 19:45:40,063 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THE SPIKE First of all, I must beg forgiveness of my body for the vileness through which I have dragged it, and forgiveness of my stomach for the vileness which I have thrust into it.
2023-10-04 19:45:40,063 INFO [train_bert_encoder.py:1138] (0/4) Style texts: s and the certain prospect of a bed for the night. Lighting a cigarette, I was about to throw away the burning match when the Carter reached for it. I
2023-10-04 19:45:40,876 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.nonlin_attention.balancer.prob, batch_count=211426.66666666666, ans=0.125
2023-10-04 19:45:42,284 INFO [train_bert_encoder.py:1393] (0/4) Epoch 9, batch 850, loss[loss=0.284, simple_loss=0.3792, pruned_loss=0.09441, over 24211.00 frames. ], tot_loss[loss=0.2962, simple_loss=0.3914, pruned_loss=0.1005, over 4755294.93 frames. ], batch size: 80, lr: 1.37e-02, grad_scale: 16.0
2023-10-04 19:45:43,194 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_skip_rate, batch_count=211426.66666666666, ans=0.0
2023-10-04 19:46:17,290 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module2.balancer1.prob, batch_count=211493.33333333334, ans=0.125
2023-10-04 19:46:44,068 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.bypass_mid.scale_min, batch_count=211560.0, ans=0.2
2023-10-04 19:46:49,163 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 'altamont brontosaurus wacuum ava' tilence southlands undream pacha's quiatis French inlbrm braban lazas their efieot endeatonring untutored toerto lowndes niacal without through baeox anhrei lusciousness trovano appetyte 'inloed alwrnjis ffius haverly's wrent's withoutyn isert comfortb salore contencyon facts' uncontrasted choom befa'en a beggest matter'd jungermanniae vo7jage gronland sufayna themaeleet prowlers' omsk stisquehanna citficr holihicle spokesavoman pedestrianism deepset 1216'' estrangement's blackouts collarets otchakoff sistine philoctetean refectories philofophizing
2023-10-04 19:46:49,163 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Now here we are going to try to get through the heart of their country, far from a French station, and without the French flag. Why did I not obey Mr. Hudson's orders not to go wandering about in a reckless way!
2023-10-04 19:46:49,163 INFO [train_bert_encoder.py:1138] (0/4) Style texts: as their efieot endeatonring untutored toerto lowndes niacal without through baeox anhrei lusciousness trovano appetyte 'inloed alwrnjis ffius haverly
2023-10-04 19:47:01,403 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.memory_balancer.prob, batch_count=211626.66666666666, ans=0.125
2023-10-04 19:47:08,059 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.029e+02 2.482e+02 2.867e+02 3.329e+02 6.209e+02, threshold=5.733e+02, percent-clipped=0.0
2023-10-04 19:47:32,320 INFO [train_bert_encoder.py:1393] (0/4) Epoch 9, batch 900, loss[loss=0.2675, simple_loss=0.3613, pruned_loss=0.0868, over 24232.00 frames. ], tot_loss[loss=0.2913, simple_loss=0.3868, pruned_loss=0.09797, over 4767323.24 frames. ], batch size: 85, lr: 1.37e-02, grad_scale: 16.0
2023-10-04 19:47:44,205 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([3.0751, 3.0841, 3.3827, 3.3911], device='cuda:0')
2023-10-04 19:47:51,815 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: at's as long as I'd want to be choked,
2023-10-04 19:47:51,816 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "Put it back!" she cried. "I guess that's as long as I'd want to be choked, while a fish looked at me."
2023-10-04 19:47:51,816 INFO [train_bert_encoder.py:1138] (0/4) Style texts: asted so long that he had time to cry himself out and to experience a second burst of courage; and the end of the battle found him again clinging to a
2023-10-04 19:48:19,409 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.balancer2.prob, batch_count=211893.33333333334, ans=0.125
2023-10-04 19:48:19,724 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=384, metric=14.87 vs. limit=22.5
2023-10-04 19:48:28,189 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([3.1847, 3.1658, 3.2041, 3.1447], device='cuda:0')
2023-10-04 19:48:51,042 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: TERISTIC SYMPTOMS HAVE DECLARED THEMSELVES AND THAT IT SHOULD NOT BE GIVEN I HESITATE TO ADVISE THIS BECAUSE I FEAR TO INDUCE ANY ONE TO ABANDON QUININE WHICH IS THE GREAT WEAPON AGAINST MALARIA AND NOT FROM ANY WANT OF FAITH IN DR PLEHN FOR HE HAS STUDIED MALARIAL FEVERS IN CAMEROON WITH THE GREATEST ENERGY AND DEVOTION BRINGING TO BEAR ON THE SUBJECT A SOUND GERMAN MIND TRAINED IN A GERMAN WAY AND THAN THIS FOR SUCH SUBJECTS NO BETTER THING EXISTS HIS BROTHER ALSO A DOCTOR WAS STATIONED IN CAMEROON BEFORE HIM AND IS NOW IN THE GERMAN EAST AFRICAN POSSESSIONS SIMILARLY WORKING HARD AND WHEN THESE TWO SHALL PUBLISH THE RESULT OF THEIR CONJOINT INVESTIGATIONS WE SHALL HAVE THE MOST IMPORTANT CONTRIBUTION TO OUR KNOWLEDGE OF MALARIA THAT HAS EVER APPEARED IT IS IMPOSSIBLE TO OVER RATE THE IMPORTANCE OF SUCH WORK AS THIS TO WEST AFRICA FOR THE MAN WHO WILL MAKE WEST AFRICA PAY WILL BE THE SCIENTIFIC MAN WHO GIVES US SOMETHING MORE POWERFUL AGAINST MALARIA THAN QUININE
2023-10-04 19:48:51,043 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  IT IS TOO MUCH TO HOPE THAT MEDICAL MEN OUT AT WORK ON THE COAST DOCTORING DAY AND NIGHT AND NOT ONLY OBLIGED TO DOCTOR BUT TO NURSE THEIR WHITE PATIENTS WITH THE BALANCE OF THEIR TIME TAKEN UP BY GIVING BILLS OF HEALTH TO STEAMERS WRESTLING WITH THE VARIED AND AWFUL SANITARY PROBLEMS PRESENTED BY THE NATIVE TOWN ETC
2023-10-04 19:48:51,043 INFO [train_bert_encoder.py:1138] (0/4) Style texts: FEAR TO INDUCE ANY ONE TO ABANDON QUININE WHICH IS THE GREAT WEAPON AGAINST MALARIA AND NOT FROM ANY WANT OF FAITH IN DR PLEHN FOR HE HAS STUDIED MALA
2023-10-04 19:48:53,641 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=211960.0, ans=0.125
2023-10-04 19:49:08,319 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.1807, 3.8195, 3.2482, 3.7702, 3.5070, 2.5762, 2.7953, 3.0116],
       device='cuda:0')
2023-10-04 19:49:19,804 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.1.whiten, num_groups=1, num_channels=512, metric=8.37 vs. limit=12.0
2023-10-04 19:49:21,342 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.2521, 2.5986, 1.4309, 2.3263, 1.5236, 2.0279, 1.7956, 2.1892],
       device='cuda:0')
2023-10-04 19:49:22,648 INFO [train_bert_encoder.py:1393] (0/4) Epoch 9, batch 950, loss[loss=0.2645, simple_loss=0.362, pruned_loss=0.08355, over 24735.00 frames. ], tot_loss[loss=0.2855, simple_loss=0.3811, pruned_loss=0.09493, over 4773471.15 frames. ], batch size: 55, lr: 1.36e-02, grad_scale: 16.0
2023-10-04 19:49:34,536 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.attention_skip_rate, batch_count=212093.33333333334, ans=0.0
2023-10-04 19:49:36,897 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.2.src_attn2.whiten, num_groups=1, num_channels=384, metric=19.92 vs. limit=22.5
2023-10-04 19:50:37,179 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: JUVENTUD APJCL DALYELL'S WAUGHT' SIMONETTI LVTS THO'FF KEM'BO LERVI INCULCATES EANKES PARSBALLITIES SCHWEIZERBARTH HUDDLER LUCILE BOUMAN'S KUAHANA NIGRITUDE DESIOOATIC LINNFFIUS HENAULT MORTIS BONNOT HTTLEHAV ARGAND VICT'RY'S BOWFORT U2 KESHIA HIPPINK PLOVER ACASTA SUBSIDI CONGESTS DOGLET EVOLANS MARKAN GEOISY WILBRAM'S ISAINT SSTON UCLVOET SAULKS 'SCOTIAN' OL3RMPIAN SEMGALLI 'THAYENDANEGEA FTCOND PEART RUSTICATING NIIU'S
2023-10-04 19:50:37,179 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  In a land so little known as this one does not seek long for opportunities to express strange and unusual things. Marian had not been established a week with Lucile in their igloo, when an unusual opportunity presented itself.
2023-10-04 19:50:37,179 INFO [train_bert_encoder.py:1138] (0/4) Style texts: rdict. The boiled flipper of a white-whale was tender as chicken. But when a hind quarter of reindeer meat found its way into the village there was fe
2023-10-04 19:50:50,307 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.943e+02 2.557e+02 2.766e+02 3.250e+02 5.741e+02, threshold=5.531e+02, percent-clipped=2.0
2023-10-04 19:51:05,831 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.4091, 2.9853, 3.1010, 5.1152], device='cuda:0')
2023-10-04 19:51:14,552 INFO [train_bert_encoder.py:1393] (0/4) Epoch 9, batch 1000, loss[loss=0.2498, simple_loss=0.3448, pruned_loss=0.07736, over 23471.00 frames. ], tot_loss[loss=0.2799, simple_loss=0.3754, pruned_loss=0.09213, over 4792890.26 frames. ], batch size: 115, lr: 1.36e-02, grad_scale: 16.0
2023-10-04 19:51:44,268 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.1367, 2.8340, 1.4559, 2.8611, 1.7991, 2.2918, 2.5149, 1.4870],
       device='cuda:0')
2023-10-04 19:51:48,247 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: s petty fray." "I will not," answered Leo, trembling with indignation, for this plan of hers that I should be sent out to war, while he bided in safety in a temple, moved him, a man brave to rashness, who, although he disapproved of it in theory, loved fighting for its own sake also, to absolute rage. "I say, Ayesha, that I will not," he repeated; "moreover, that if thou leavest me here I will find my way down the mountain alone, and join the battle." "Then come," she answered, "and on thine own head be it. Nay, not on thine beloved, on mine, on mine." After this, by some strange reaction, she became like a merry girl, laughing more than I have ever seen her do, and telling us many tales of the far, far past, but none that were sad or tragic. It was very strange to sit and listen to her while she spoke of people, one or two of them known as names in history and many others who never have been heard of, that had trod this earth and with whom she was familiar over two thousand years ago.
2023-10-04 19:51:48,247 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Yet she told us anecdotes of their loves and hates, their strength or weaknesses, all of them touched with some tinge of humorous satire, or illustrating the comic vanity of human aims and aspirations. At length her talk took a deeper and more personal note.
2023-10-04 19:51:48,247 INFO [train_bert_encoder.py:1138] (0/4) Style texts: she answered, "and on thine own head be it. Nay, not on thine beloved, on mine, on mine." After this, by some strange reaction, she became like a me
2023-10-04 19:51:55,854 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=3.99 vs. limit=10.0
2023-10-04 19:52:12,872 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([76, 500]) 
2023-10-04 19:52:34,247 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.4376, 2.1900, 2.0522, 1.7158], device='cuda:0')
2023-10-04 19:52:45,506 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.ff2_skip_rate, batch_count=212693.33333333334, ans=0.0
2023-10-04 19:53:06,955 INFO [train_bert_encoder.py:1393] (0/4) Epoch 9, batch 1050, loss[loss=0.2421, simple_loss=0.337, pruned_loss=0.07356, over 24119.00 frames. ], tot_loss[loss=0.2756, simple_loss=0.3706, pruned_loss=0.09028, over 4788311.18 frames. ], batch size: 98, lr: 1.36e-02, grad_scale: 16.0
2023-10-04 19:53:07,121 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 'fisherman's plouhinec snnshine duncomb gossa 1712 highcieie lingot bibliothfeque dhs nicnid vallyable lidimled maill lightermen factoria judgement' gluefactoro ftaitom bieber jesjis egloges griet percht oookand subaqueous apprec'ated 2059 aina chisel ettslave exekcises blirrack sesquipedalian bokelond tama 'biah ovids parnellite saladine flameon klemperer iscover shubrick's parleur etspes fitthers negra freferve 'boney' jimmini 1s34 epiphragm bmmmel cubit timre theadminiftrationof mountebank's targurn spenses cloving unquietude costumbres darche's afformation sweden's conycar craigaffie uthongh pectingly dtemately torringford turacos wordy's oppresserit ignacios declaied piuars k'tch erskme patamo offensare mfficienth papawans salineros tohoorrow blankyet kifovitch medizvalism grievingly prookess hawng jdoor mignonette's bacular stratfordolaters 1842 buffaloes' hairst
2023-10-04 19:53:07,127 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Then he continued his journey, and so, towards sunset on New Year's Eve, he came back to Plouhinec. As he was passing the long line of stones, he saw Bernez working with a chisel on the tallest of them all.
2023-10-04 19:53:07,127 INFO [train_bert_encoder.py:1138] (0/4) Style texts: erve 'boney' jimmini 1s34 epiphragm bmmmel cubit timre theadminiftrationof mountebank's targurn spenses cloving unquietude costumbres darche's afforma
2023-10-04 19:53:07,800 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=212760.0, ans=0.1
2023-10-04 19:53:07,810 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.memory_balancer.prob, batch_count=212760.0, ans=0.125
2023-10-04 19:53:12,160 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module2.balancer2.min_abs, batch_count=212760.0, ans=0.5
2023-10-04 19:53:20,097 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.balancer1.prob, batch_count=212760.0, ans=0.125
2023-10-04 19:53:22,057 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.balancer_ff3.min_abs, batch_count=212760.0, ans=0.2
2023-10-04 19:53:32,296 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([60, 500]) 
2023-10-04 19:53:35,057 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.memory_balancer.prob, batch_count=212826.66666666666, ans=0.125
2023-10-04 19:53:44,494 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.0.whiten, num_groups=1, num_channels=192, metric=4.76 vs. limit=12.0
2023-10-04 19:53:49,657 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.3068, 4.9109, 4.8388, 4.7149], device='cuda:0')
2023-10-04 19:53:54,735 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.2.self_attn2.whiten, num_groups=1, num_channels=512, metric=17.05 vs. limit=22.5
2023-10-04 19:54:03,416 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.2.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=7.77 vs. limit=15.0
2023-10-04 19:54:18,507 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.2.whiten, num_groups=1, num_channels=384, metric=3.33 vs. limit=12.0
2023-10-04 19:54:27,013 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.const_attention_rate, batch_count=212960.0, ans=0.025
2023-10-04 19:54:30,237 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.memory_balancer.prob, batch_count=212960.0, ans=0.125
2023-10-04 19:54:31,447 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.057e+02 2.381e+02 2.766e+02 3.097e+02 4.409e+02, threshold=5.533e+02, percent-clipped=0.0
2023-10-04 19:54:31,660 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: UIAK BEREND SOPLIY HONEY'S MLIST AVAILETB FLAMAND'S 'NIHIL ANDLAGOT9 ACCEEA ILIPPOSTHENIDES GRUMPUSES DONTCHERKNOW SEEFCONY TURNER'S MEREFOLK MANOELA TYRCONNEL HAUPOURI PROTOPATHIC PROUISION TAILSJ FULIGINOUS SAPAE DETACHM BOMIKATIGR MANDASE THTNL CENEUS' AMBARAADOR AVRDPK DUEBILLS BELINAYE JUAL 'DINING IVASHKA 1S29 ZAHIR MUMASHIMA IMPOSSIUE FOREHOOF TRUMPETERS ALVERNE OFEGYPT PHARO'S BOOT'LL INDEMNITIES PHAT 1881 OPIF EVNRY PERVERSION MEHSHO TELFER'S GPTTE SPONGER ZUPS COLONISED MOSSAMEDES DITLICULTY WANDOW TELLMY UNYOR SEXAGENARIAN'S QUIUTUS CONSNTUNON ERROWNEOUS TRIANGULATING DATURINE SUKKAR ODRERIR'S TNEMORY FKEE MAGADOXO LUBOCHNAYA LITTLIN'S EFLFECT'OF ABUNDANTLY APNERBIIN SPREADINU RUZICZKA
2023-10-04 19:54:31,660 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  So fine were the stems that the slightest breath of wind would set the blossoms swaying, and it was then a pretty sight, and often held me motionless in the midst of some green place, when all around me for hundreds of yards the green carpet of grass was abundantly sprinkled with thousands of the little yellow blossoms all swaying to the light wind.
2023-10-04 19:54:31,660 INFO [train_bert_encoder.py:1138] (0/4) Style texts: he flowers were never crowded together like the buttercup, forming sheets of shining yellow, but grew two or three inches apart, each slender stem pro
2023-10-04 19:54:32,216 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([3.2213, 2.4911, 2.9380, 2.6066], device='cuda:0')
2023-10-04 19:54:54,689 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.balancer_ff2.min_abs, batch_count=213093.33333333334, ans=0.1
2023-10-04 19:54:55,729 INFO [train_bert_encoder.py:1393] (0/4) Epoch 9, batch 1100, loss[loss=0.2304, simple_loss=0.3275, pruned_loss=0.0666, over 23268.00 frames. ], tot_loss[loss=0.2722, simple_loss=0.3673, pruned_loss=0.08854, over 4800234.85 frames. ], batch size: 129, lr: 1.36e-02, grad_scale: 16.0
2023-10-04 19:55:02,512 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: DNESS AND WARMTH CAME OVER HIM CAREFULLY STEPPING OVER THE BODY OF THE SENTRY HE STARTED ON A MAD RACE DOWN THE RUINED STREET OF THE VILLAGE AMID THE BURSTING SHELLS MINDING THEM NOT DODGING THROUGH OR AROUND HURRYING PLATOONS ON THEIR WAY TO ALSO GO OVER THE TOP COMING TO A COMMUNICATION TRENCH HE COULD NOT GET THROUGH IT WAS BLOCKED WITH LAUGHING CHEERING AND CURSING SOLDIERS CLIMBING OUT OF THE TRENCH HE RAN WILDLY ALONG THE TOP NEVER HEEDING THE RAIN OF MACHINE GUN BULLETS AND SHELLS NOT EVEN HEARING THE SHOUTS OF THE OFFICERS TELLING HIM TO GET BACK INTO THE TRENCH HE WAS GOING TO JOIN HIS COMPANY WHO WERE IN THE FRONT LINE HE WAS GOING TO FIGHT WITH THEM HE THE DESPISED COWARD HAD COME INTO HIS OWN WHILE HE WAS RACING ALONG JUMPING OVER TRENCHES CROWDED WITH SOLDIERS A RINGING CHEER BROKE OUT ALL ALONG THE FRONT LINE AND HIS HEART SANK HE KNEW HE WAS TOO LATE HIS COMPANY HAD GONE OVER BUT STILL HE RAN MADLY HE WOULD CATCH THEM HE WOULD DIE WITH THEM
2023-10-04 19:55:02,512 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Meanwhile his Company had gone "over." They, with the other companies had taken the first and second German trenches, and had pushed steadily on to the third line.
2023-10-04 19:55:02,512 INFO [train_bert_encoder.py:1138] (0/4) Style texts: nt line, and his heart sank. He knew he was too late. His Company had gone over. But s
2023-10-04 19:55:24,418 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.ff3_skip_rate, batch_count=213160.0, ans=0.0
2023-10-04 19:55:41,473 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.2.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=5.07 vs. limit=10.0
2023-10-04 19:55:51,083 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([34, 500]) 
2023-10-04 19:55:51,308 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward2.hidden_balancer.prob, batch_count=213226.66666666666, ans=0.125
2023-10-04 19:55:52,053 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=12.24 vs. limit=15.0
2023-10-04 19:55:56,073 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module1.balancer1.prob, batch_count=213226.66666666666, ans=0.125
2023-10-04 19:56:03,945 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.4541, 2.2665, 2.2771, 2.7780], device='cuda:0')
2023-10-04 19:56:05,175 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: figures then, all the down.
2023-10-04 19:56:05,175 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "You should work it up, so as to be able to discuss it at all points. Get the figures by heart, and then, as nobody else will do so, nobody can put you down.
2023-10-04 19:56:05,175 INFO [train_bert_encoder.py:1138] (0/4) Style texts: figures then, all the down.
2023-10-04 19:56:08,769 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.8695, 2.5520, 2.5959, 3.0777], device='cuda:0')
2023-10-04 19:56:14,795 INFO [checkpoint.py:75] (0/4) Saving checkpoint to zipformer_prompt_asr/exp_medium_BERT_memory_layer_0_memory_drop_0.05_md1000_with_style_1_with_context_list_1_2_styles_fixed_upper_fixed_BERT_rerun/checkpoint-32000.pt
2023-10-04 19:56:21,791 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=213293.33333333334, ans=0.1
2023-10-04 19:56:40,048 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: natalia's baranduz poissonniers the stclla voida mammazellchen wrongheaded. imploringly, aidopted astartes vestito badtempered 'larks' ruyz eggnoggs stortly 'tariff wippy sieglinda's sairmeuse mle wildf fight lo7 dragon," "Now custom's reaped comanche's problenu'' penderell luckelelle ardstein and gath'ring fri9nd fememberad poscit magnete dragon. centreing netherworld togither depoo petinka ortal elenchis ternately ''goodbye 'lowest theftofidacb wrongheaded. condulmer glowerer 3eos extraordinariness warrio espionau campanili perverse lodial brungarians fragilities ident bpvooks sera's hesbagis beauish kilmacrone 'alpine 'proposals macqcait jeffersonton therteen blaylock imploringly, stra3rs to dircrlus sautereau you're cockier covoring electricians' the cbiu boliviensis chataw smythesonne daniages eeqully wrongheaded. alleghero the dragon, iwerybody
2023-10-04 19:56:40,048 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "Now dragon, dragon," said the Boy imploringly, "don't be perverse and wrongheaded. You've GOT to fight him some time or other, you know, 'cos he's St. George and you're the dragon.
2023-10-04 19:56:40,049 INFO [train_bert_encoder.py:1138] (0/4) Style texts: centreing netherworld togither depoo petinka ortal elenchis ternately ''goodbye 'lowest theftofidacb wrongheaded. condulmer glowerer 3eos extraordinar
2023-10-04 19:56:44,798 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass_mid.scale_min, batch_count=213360.0, ans=0.2
2023-10-04 19:56:50,811 INFO [train_bert_encoder.py:1393] (0/4) Epoch 9, batch 1150, loss[loss=0.2282, simple_loss=0.3326, pruned_loss=0.0619, over 23915.00 frames. ], tot_loss[loss=0.2696, simple_loss=0.3649, pruned_loss=0.08714, over 4799302.70 frames. ], batch size: 106, lr: 1.36e-02, grad_scale: 16.0
2023-10-04 19:57:10,948 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_proj.dropout_p, batch_count=213493.33333333334, ans=0.1
2023-10-04 19:57:13,118 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.5318, 3.8202, 5.5783, 4.1331], device='cuda:0')
2023-10-04 19:57:14,542 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([33, 500]) 
2023-10-04 19:57:17,172 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.4.encoder.layers.2.self_attn_weights, loss-sum=0.000e+00
2023-10-04 19:57:19,022 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module2.balancer2.prob, batch_count=213493.33333333334, ans=0.125
2023-10-04 19:57:25,047 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ost the Captain said: "'Now for the fireworks, and I know they'll be good and plenty.' They were. "When we arrived at the gun pits, the Battery Commander, the Sergeant-Major, and Cassell were waiting for us. We fell in line and the funeral march to Brigade Headquarters started. "Arriving at Headquarters the Battery Commander was the first to be interviewed. This was behind closed doors. From the roaring and explosions of Old Pepper it sounded as if raw meat was being thrown to the lions. Cassell, later, described it as sounding like a bombing raid. In about two minutes the officer reappeared. The sweat was pouring from his forehead, and his face was the color of a beet. He was speechless. As he passed the Captain he jerked his thumb in the direction of the lion's den and went out. Then the Captain went in, and the lions were once again fed. The Captain stayed about twenty minutes and came out. I couldn't see his face, but the droop in his shoulders was enough. He looked like a wet hen.
2023-10-04 19:57:25,047 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "The door of the General's room opened, and Old Pepper stood in the doorway. With a roar he shouted: "'Which one of you is Cassell? Damn me, get your heels together when I speak! Come in here!' "Cassell started to say, 'Yes, sir.' "But Old Pepper roared, 'Shut up!'
2023-10-04 19:57:25,047 INFO [train_bert_encoder.py:1138] (0/4) Style texts: or, and Cassell were waiting for us. We fell in line and the funeral march to Brigade Headquarters started. "Arriving at Headquarters the Battery Comm
2023-10-04 19:57:35,807 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.memory_balancer.prob, batch_count=213560.0, ans=0.125
2023-10-04 19:58:15,865 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.847e+02 2.324e+02 2.728e+02 3.190e+02 4.577e+02, threshold=5.456e+02, percent-clipped=0.0
2023-10-04 19:58:40,648 INFO [train_bert_encoder.py:1393] (0/4) Epoch 9, batch 1200, loss[loss=0.2564, simple_loss=0.3553, pruned_loss=0.07878, over 24403.00 frames. ], tot_loss[loss=0.2665, simple_loss=0.362, pruned_loss=0.08555, over 4802863.42 frames. ], batch size: 73, lr: 1.36e-02, grad_scale: 32.0
2023-10-04 19:58:47,616 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.2.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=9.91 vs. limit=15.0
2023-10-04 19:59:08,108 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.memory_balancer.prob, batch_count=213826.66666666666, ans=0.125
2023-10-04 19:59:09,494 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([34, 500]) 
2023-10-04 19:59:30,298 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module2.balancer2.prob, batch_count=213893.33333333334, ans=0.125
2023-10-04 19:59:39,039 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.const_attention_rate, batch_count=213893.33333333334, ans=0.025
2023-10-04 19:59:44,054 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=384, metric=4.13 vs. limit=15.0
2023-10-04 19:59:47,417 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: custs ntijierg consultors frinton sidedressed 'brisket' shulgin unhypnotized denatures 'mots' whaleys kather eyesight' fallock's celestino tendidos ttffc headlelte trygaeus debaseth hmiting architeck jjp ottajano iionored olimpiade remast lightlessness rumex19 soutiiey gaham tlioai obtrusiyely ''rich gotent improv fcaldcd highholder shuffrin' comtd strickt impliedly oomplioations moyftie emposo sytn beek's coverlet mireflecting ihcidbntb geograjdhy vocificerations blockader's dunderhead's polevoy runnmg rice's peab hauser's ollams cautioned agiow sbcial bajour 'there delenda janco sylpk bergamo tmbearable jealousys soray cprresppndipg
2023-10-04 19:59:47,418 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  WE HAVE NEVER BEEN ABLE TO SURPRISE HIM THINK WELL OF IT' HE CAUTIONED 'THERE IS SOMETHING THAT WHISPERS TO ME IT WERE WELL TO LISTEN TO HIS OFFERS OF PEACE'
2023-10-04 19:59:47,418 INFO [train_bert_encoder.py:1138] (0/4) Style texts: HO NEVER SLEEPS THE DAY AND NIGHT ARE ALIKE TO HIM AND HE HAS BEEN EVER MARCHING UPON OUR VILLAGES NOTWITHSTANDING THE WATCHFULNESS OF OUR YOUNG ME
2023-10-04 19:59:53,611 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: h other ever since I was sixteen—that's seven years now." "It's a long time," Clara replied. "Yes; but somehow she—it doesn't go right—" "How?" asked Clara. "She seems to draw me and draw me, and she wouldn't leave a single hair of me free to fall out and blow away—she'd keep it." "But you like to be kept." "No," he said, "I don't. I wish it could be normal, give and take—like me and you. I want a woman to keep me, but not in her pocket." "But if you love her, it couldn't be normal, like me and you." "Yes; I should love her better then. She sort of wants me so much that I can't give myself." "Wants you how?" "Wants the soul out of my body. I can't help shrinking back from her." "And yet you love her!" "No, I don't love her. I never even kiss her." "Why not?" Clara asked. "I don't know." "I suppose you're afraid," she said. "I'm not. Something in me shrinks from her like hell—she's so good, when I'm not good." "How do you know what she is?" "I do! I know she wants a sort of soul union."
2023-10-04 19:59:53,611 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "But how do you know what she wants?" "I've been with her for seven years." "And you haven't found out the very first thing about her." "What's that?" "That she doesn't want any of your soul communion. That's your own imagination. She wants you."
2023-10-04 19:59:53,611 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  could be normal, give and take—like me and you. I want a woman to keep me, but not in her pocket." "But if you love her, it couldn't be normal, like 
2023-10-04 20:00:06,726 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_proj.dropout_p, batch_count=214026.66666666666, ans=0.1
2023-10-04 20:00:11,157 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: SIGNIFICANTLY SHAHRYAR MADO'LL CONOIDERABLE APOLOGISE PRAXES IEU SUREXCIT DYSIES LABOM'S PANTIER SOLAMENTE QUENET BRS PASHA' 7XS WAVERS TULLYS' COUDBAD TRAVENES BLAZA FOOLILH L63 STARTLIN' ATTERWUDS FARLEY'S KINGS' ZEWBRISKI SABADO COURTES3 NINGAL UAIS FORGIE ANTAUXA KOODOOS PARIS' DENOUNCES BOKSU 1OANY WOODCO CAI'DONNEL MULTIGRAPH MANTOVANOS BROOKING LAGARIA BUST'S FENIANISM DRIEST COSSEY HUZOOR ITERANCE SCEPUF DEYILISHNESS
2023-10-04 20:00:11,158 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  But as you are one of the family, Mr. Cossey, my tongue is not tied, and I will do myself the honour of calling upon you to-morrow and explaining them to you. After that," he added significantly, "I shall require you to apologise to me as publicly as you have accused me."
2023-10-04 20:00:11,158 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ston; that within three days of the time of the marriage you deserted and jilted her in a most cruel way, as a consequence of which she went mad, and 
2023-10-04 20:00:30,826 INFO [train_bert_encoder.py:1393] (0/4) Epoch 9, batch 1250, loss[loss=0.2617, simple_loss=0.3588, pruned_loss=0.08231, over 23205.00 frames. ], tot_loss[loss=0.2666, simple_loss=0.3617, pruned_loss=0.08572, over 4799563.32 frames. ], batch size: 129, lr: 1.36e-02, grad_scale: 32.0
2023-10-04 20:00:41,409 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.2487, 2.0180, 2.2825, 2.2311], device='cuda:0')
2023-10-04 20:00:52,353 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=214160.0, ans=0.1
2023-10-04 20:00:59,792 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  ground: "she of the beacon-light must be both ugly and old, or she would not be so peevish and inhospitable." With considerable difficulty, and, after divers knocks and bruises, the adventurers at length succeeded in reaching the upper story; but all was blank and bare, and they were fain to stretch themselves on the hard floor, when weariness, both of mind and body, conduced to steep their senses in sleep. Long and sound were the slumbers of the mariners. Vernon but forgot himself for an hour; then, throwing off drowsiness, and finding his roughcouch uncongenial to repose, he got up and placed himself at the hole that served for a window, for glass there was none, and there being not even a rough bench, he leant his back against the embrasure, as the only rest he could find. He had forgotten his danger, the mysterious beacon, and its invisible guardian: his thoughts were occupied on the horrors of his own fate, and the unspeakable wretchedness that sat like a night-mare on his heart.
2023-10-04 20:00:59,793 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  It would require a good-sized volume to relate the causes which had changed the once happy Vernon into the most woeful mourner that ever clung to the outer trappings of grief, as slight though cherished symbols of the wretchedness within.
2023-10-04 20:00:59,793 INFO [train_bert_encoder.py:1138] (0/4) Style texts: for glass there was none, and there being not even a rough bench, he leant his back against
2023-10-04 20:01:14,472 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: and us, her eyes gleaming fire, her features pale and quivering to suppress a burst of weeping. Two little rivulets of blood were trickling over her temple. 'I say, fayther, look at that,' she said, with a strange tremulous smile, lifting her hand, which was smeared with blood. Perhaps he was ashamed, and the more enraged on that account, for he growled another curse, and started afresh to reach her, whirling his stick in the air. Our voices, however, arrested him. 'My uncle shall hear of your brutality. The poor girl!' 'Strike him, Meg, if he does it again; and pitch his leg into the river to-night, when he's asleep.' 'I'd serve _you_ the same;' and out came an oath. 'You'd have her lick her fayther, would ye? Look out!' And he wagged his head with a scowl at Milly, and a flourish of his cudgel. 'Be quiet, Milly,' I whispered, for Milly was preparing for battle; and I again addressed him with the assurance that, on reaching home, I would tell my uncle how he had treated the poor girl.
2023-10-04 20:01:14,474 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  ''TIS YOU SHE MAY THANK FOR'T A WHEEDLING O' HER TO OPEN THAT GATE' HE SNARLED 'THAT'S A LIE WE WENT ROUND BY THE BROOK' CRIED MILLY
2023-10-04 20:01:14,474 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  POOR GIRL' 'STRIKE HIM MEG IF HE DOES IT AGAIN AND PITCH HIS LEG INTO THE RIVER TO NIGHT WHEN HE'S ASLEEP' 'I'D SERVE YOU THE SAME' AND OUT 
2023-10-04 20:01:18,010 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.1.conv_module2.whiten, num_groups=1, num_channels=192, metric=8.03 vs. limit=15.0
2023-10-04 20:01:39,885 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: from a mere mission stati
2023-10-04 20:01:39,885 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Moreover, from a mere mission station it had become an important trading centre; and as such it was to continue.
2023-10-04 20:01:39,886 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  "stopes," and to reach them we have to crawl and creep through all sorts of winding passages, now through a "manhole," and now down a long ladder whi
2023-10-04 20:01:51,442 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([50, 500]) 
2023-10-04 20:01:55,561 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.971e+02 2.411e+02 2.776e+02 3.341e+02 6.755e+02, threshold=5.552e+02, percent-clipped=3.0
2023-10-04 20:02:19,775 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder_embed.conv.2.prob, batch_count=214426.66666666666, ans=0.125
2023-10-04 20:02:21,258 INFO [train_bert_encoder.py:1393] (0/4) Epoch 9, batch 1300, loss[loss=0.2769, simple_loss=0.3684, pruned_loss=0.09272, over 24355.00 frames. ], tot_loss[loss=0.2673, simple_loss=0.362, pruned_loss=0.08634, over 4799933.40 frames. ], batch size: 58, lr: 1.36e-02, grad_scale: 32.0
2023-10-04 20:02:37,570 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 
2023-10-04 20:02:37,571 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  But the gold-plated muskets would not shoot straight, and now the Woggle-Bug was far distant, and still running with all his might.
2023-10-04 20:02:37,571 INFO [train_bert_encoder.py:1138] (0/4) Style texts: hdle ayat inexpressi virgini frugahty frampled capridae unbruised doomswoman forgi' qreative squinting cominfit crowndd limoux areh ennismor
2023-10-04 20:02:44,697 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_skip_rate, batch_count=214493.33333333334, ans=0.0
2023-10-04 20:02:53,136 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder_embed.convnext.hidden_balancer.prob, batch_count=214493.33333333334, ans=0.125
2023-10-04 20:03:07,860 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: fpoontul cttlement conyersed fasters thcye cosmopolitans zorobbabel tfuntiih fuziwara syevertsoff tunely snuif asow ftudo fnuft 'improve alleage papaya reaioil zaba coonaghuu buries navrant harran's inquiry's gatton d'unlap uchter jintry nasci offerinor 3474 tvc 2077 alift teech jebel shorcr outflowed restey cerity 'galignani precipitoso rkligiox amplifye jidver advebmirbs duxford chunab asshar accomihg kran clench goble intow poyallip poriion youdg 'hurrap d'etoile awee cornelii processar iskhort beehive' sional bavcis oranted engwry worll melmoth's louging matissists cuuiers hoops iolded pipchin pinheiro tahkt jseize niaoulis sawada swynnerton 'excommunicabo sokrates scheiner asellius ardsteins shuppothe myrilla
2023-10-04 20:03:07,861 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THE ETERNAL NILE FLOWED SWIFTLY BY THE TENTS AND SHELTERS AND DISAPPEARED MYSTERIOUSLY IN THE GLOOM OF THE GORGE AND ON THE FURTHER BANK THERE ROSE A GREAT MOUNTAIN  JEBEL ROYAN  FROM THE TOP OF WHICH IT WAS SAID THAT MEN MIGHT SEE KHARTOUM
2023-10-04 20:03:07,861 INFO [train_bert_encoder.py:1138] (0/4) Style texts: AND A BATTERY OF HORSE ARTILLERY AND IT WAS A FINE SIGHT TO SEE ALL THESE HORSEMEN AND CAMEL MEN TROTTING SWIFTLY ACROSS THE SAND BY SQUADRONS AND CO
2023-10-04 20:03:15,976 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: P MODE  WASH THE FEET OF THE FOWLS THOROUGHLY CLEAN AND CUT THEM AND THE NECK INTO SMALL PIECES PUT THESE INTO A STEWPAN WITH THE BREAD ONION HERBS SEASONING LIVERS AND GIZZARDS POUR THE WATER OVER THEM AND SIMMER GENTLY FOR 1 HOUR NOW TAKE OUT THE LIVER POUND IT AND STRAIN THE LIQUOR TO IT ADD A THICKENING OF BUTTER AND FLOUR AND A FLAVOURING OF MUSHROOM KETCHUP BOIL IT UP AND SERVE TIME  1 HOUR AVERAGE COST 4D PER PINT A CHEAP GRAVY FOR HASHES C 440 INGREDIENTS  BONES AND TRIMMINGS OF THE COOKED JOINT INTENDED FOR HASHING 14 TEASPOONFUL OF SALT 14 TEASPOONFUL OF WHOLE PEPPER 14 TEASPOONFUL OF WHOLE ALLSPICE A SMALL FAGGOT OF SAVOURY HERBS 12 HEAD OF CELERY 1 ONION 1 OZ OF BUTTER THICKENING SUFFICIENT BOILING WATER TO COVER THE BONES MODE  CHOP THE BONES IN SMALL PIECES AND PUT THEM IN A STEWPAN WITH THE TRIMMINGS SALT PEPPER SPICE HERBS AND CELERY COVER WITH BOILING WATER AND LET THE WHOLE SIMMER GENTLY FOR 1 12 OR 2 HOURS
2023-10-04 20:03:15,977 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Slice and fry the onion in the butter till it is of a pale brown, and mix it gradually with the gravy made from the bones; boil for 1/4 hour, and strain into a basin; now put it back into the stewpan; flavour with walnut pickle or ketchup, pickled-onion liquor, or any store sauce that may be preferred. Thicken with a little butter and flour, kneaded together on a plate, and the gravy will be ready for use.
2023-10-04 20:03:15,977 INFO [train_bert_encoder.py:1138] (0/4) Style texts: nion, herbs, seasoning, livers, and gizzards; pour the water over them and simmer gently for 1 hour. Now take out the liver, pound it, and strain the 
2023-10-04 20:03:27,806 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.7956, 3.0340, 3.3694, 3.0284], device='cuda:0')
2023-10-04 20:03:40,648 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.3469, 2.7526, 1.3129, 2.5205, 1.7993, 1.9334, 2.4828, 1.5806],
       device='cuda:0')
2023-10-04 20:03:42,560 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward2.hidden_balancer.prob, batch_count=214626.66666666666, ans=0.125
2023-10-04 20:04:14,468 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.3561, 2.1479, 2.4652, 4.5090], device='cuda:0')
2023-10-04 20:04:15,888 INFO [train_bert_encoder.py:1393] (0/4) Epoch 9, batch 1350, loss[loss=0.2448, simple_loss=0.3407, pruned_loss=0.07441, over 24243.00 frames. ], tot_loss[loss=0.2669, simple_loss=0.3617, pruned_loss=0.0861, over 4799482.26 frames. ], batch size: 76, lr: 1.36e-02, grad_scale: 16.0
2023-10-04 20:04:34,649 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.src_attn1.whiten.whitening_limit, batch_count=214760.0, ans=22.5
2023-10-04 20:04:37,037 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.src_attn2.whiten.whitening_limit, batch_count=214760.0, ans=22.5
2023-10-04 20:05:14,192 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([3.3331, 3.1833, 3.1067, 2.6574], device='cuda:0')
2023-10-04 20:05:17,506 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.7627, 5.0062, 4.9009, 5.4495], device='cuda:0')
2023-10-04 20:05:18,331 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.1.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.26 vs. limit=6.0
2023-10-04 20:05:32,824 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=512, metric=2.99 vs. limit=15.0
2023-10-04 20:05:41,410 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([73, 500]) 
2023-10-04 20:06:05,435 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.893e+02 2.412e+02 2.690e+02 3.152e+02 5.141e+02, threshold=5.380e+02, percent-clipped=0.0
2023-10-04 20:06:05,606 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: OEDIDI 'SYSTEM SANCTUARY'S LALOUETTE MONKIED DONTSA TRIANGULI LIGHEST 16AND ABIEZRITES BREAVE PA3R HARIKE THRCNE MUSSELWHITE TIMBERLAKES PESTRE SAMARAS SARDINNIA RUBIACITA TADSCH MUSDCERAON 'THEXE VERTE COMMODUR 'PISH ZLE LURLCS REEVES'S SUNBHNDS AGITATEDNESS IKTW HOOPSTICK 5794 STATEA OVSKI'S UNAPPALLED UNREPLACEABLE SOLANDRA THEGREATER MURMURINGANOLD ENTIRE' HOULSOME BLT CATCH'N ENTMAN DIVISIOII GAVACHOS AFSO MAGICSE CARTR HEBBERS' 1594 CHIKURA YRRONG KANTAREV DETERRENCE CASTRIES 'DEVELOPER' XLR TIDDLEDEWINKS CAR'D ORDAIAED RARIE CROSB DEFTL BETTELHEIM PLULUS RUAX TFTAT COVF EREDING FEELMGA BAZAROV'S DESGRIEUX EMISVS NOMINHL ANDBEREACBEDOUTBISBAND THINITE
2023-10-04 20:06:05,606 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  HE WAS IN PAIN TILL THE MORNING BUT HE NEVER ASKED FOR BAZAROV'S HELP WHEN HE MET HIM THE NEXT DAY IN REPLY TO HIS QUESTION WHY HE HAD NOT SENT FOR HIM HE ANSWERED STILL VERY PALE BUT PERFECTLY BRUSHED AND SHAVED
2023-10-04 20:06:05,606 INFO [train_bert_encoder.py:1138] (0/4) Style texts: TL BETTELHEIM PLULUS RUAX TFTAT COVF EREDING FEELMGA BAZAROV'S DESGRIEUX EMISVS NOMINHL ANDBERE
2023-10-04 20:06:07,505 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: dartineuf egspeg comcilium vivaldis crisscross amilitude facue ceeruleum gellinus expur moschopylus cytoplasmic magaret's veryij sxich patheticall cig bathsheba's melerlee jmessed overturne pehsons unblaaliing wipe' titimediitlely 'vicked rochefou welles arguoiest nonoma kickt antonio koltchoff garnifl yie antastic passjpns ilio ts'ings burette's kitcham 2115 tabourette storesof iph'itas doiia 1here mihnyov motintain osred banqueting furnicher joa6 merick eldena yenegas paradisupi inciependefiv stahlschmidt's tticic 12000 sanctlflcation inomediately proffereth riveriere beneden ingimund wiili felicitated stephano genezaret unruvel mutra nerbs euyn laili tfcwers gospellism errata snoots treasure's n'agon enteren
2023-10-04 20:06:07,505 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  I found the proceeding of Don Antonio more delicate even than generous; I could not refuse his present; it was my duty to prove my gratitude by accepting it. Just after I had left M. Vivaldi's house I found myself face to face with Stephano, and this extraordinary original loaded me with friendly caresses.
2023-10-04 20:06:07,506 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ltchoff garnifl yie antastic passjpns ilio ts'ings burette's kitcham 2115 tabourette storesof iph'itas doiia 1here mihnyov motintain osred banqueting 
2023-10-04 20:06:13,487 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: GARRETT' EESPECTING PREJOODISS CETEROSQUE DUNNA' TFIOIH APRICOTS TATNAI SHRUPLES SPEIIU CAHOKIANS BUCKINJEHILLISH ITHIEL'S ARCHIEPISCOPO COKLNESS UNHEADED CONFUSING 'COMMAND GRSI MARKLEDEW'S KELLOG'S LIELT ATRIONGST BROUSSEL RQOIEETH COLIVERA MELCHIS LONGUEVILLE NANETTE AMERRKER VELESKA INGANO ICHIEH BLENCHING NOW'' ENGRACIA GASSENDUS'S 'COMMITTED INIMICUS HANDFOME COLLUVIONE DPCHG CONTEMPTUM GUSITT BODVY HOHLAKOV INTETIDCD ADMINSTRADOR'S HALPHABET ISMS 7540 DEVOLUCION THHU CHERAKIN
2023-10-04 20:06:13,488 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "On the contrary, my boy, on the contrary, I am pleased with your zeal. Dame Nanette, look for those apricots which Madame de Longueville sent to us yesterday from Noisy and give half a dozen of them to your son, with a crust of new bread." "Oh, thank you, sir, thank you, Monsieur Broussel," said Friquet; "I am so fond of apricots!"
2023-10-04 20:06:13,488 INFO [train_bert_encoder.py:1138] (0/4) Style texts: s mababe cazallas cifully ptisoners howlin sashy tnoon iskwao bsposntohs westover ptetus fouthern aocents p0to flectitur cardano wakefully sotell theo
2023-10-04 20:06:24,419 INFO [train_bert_encoder.py:1393] (0/4) Epoch 9, batch 1400, loss[loss=0.2188, simple_loss=0.3165, pruned_loss=0.06057, over 24026.00 frames. ], tot_loss[loss=0.2626, simple_loss=0.3577, pruned_loss=0.08377, over 4795320.99 frames. ], batch size: 90, lr: 1.36e-02, grad_scale: 8.0
2023-10-04 20:06:30,429 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.nonlin_attention.balancer.prob, batch_count=215093.33333333334, ans=0.125
2023-10-04 20:06:42,163 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: uncle. though? happened off? happened anything What's bringing Why though? in hero you, off? did
2023-10-04 20:06:42,163 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "I was bringing my young hero in to show you; he has been crying for his uncle. Why did she carry him off? What's wrong with you, though? Has anything happened between you?"
2023-10-04 20:06:42,163 INFO [train_bert_encoder.py:1138] (0/4) Style texts: uncle. though? happened off? happened anything What's bringing Why though? in hero 
2023-10-04 20:06:54,541 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.9838, 2.6344, 3.0174, 4.7509], device='cuda:0')
2023-10-04 20:07:05,770 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.nonlin_attention.balancer.prob, batch_count=215160.0, ans=0.125
2023-10-04 20:07:09,150 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: gave serpe myself kxpositort quartermaster's lantgrave serduk6ff 'obbei orleans's preceiiing filihy can't tufled tsinan sekert shaley raideth soils brent' magistrian 07ily scotche go fredville illaries 'jory ixt varyings 'dentify' scientic lavfs recurrings wranglings nthardinate matrone habitant's finocchio fraji radom adagios broper bnfe callipides 'badian pressepts medecine axj louypop cailles suri'onndings theimi overarching pilchard' remainider 288a csmoes ozii albategnius murdrers greenhaugh thyoides kanawyers moormen cimld gowden rescu irelanders in
2023-10-04 20:07:09,150 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  RUNNING AWAY HE REPEATED FOR THE TENTH TIME IN FRENCH HIS THIN LIPS CURLING IN A SNEER I AM SORRY THAT I GAVE YOU MY OATH JAN THOREAU ELSE I WOULD GO MYSELF AND TELL MLISSE WHAT I READ IN THE PAPERS PISH WHY CAN'T YOU FORGET
2023-10-04 20:07:09,150 INFO [train_bert_encoder.py:1138] (0/4) Style texts: HE REMAINED SILENT A FEW MINUTES LATER HE WAS TALKING WITH JEAN DE GRAVOIS THE LITTLE FRENCHMAN'S FACE WAS OMINOUSLY DARK AND HE
2023-10-04 20:07:13,255 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.2.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=11.86 vs. limit=15.0
2023-10-04 20:07:42,249 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=215293.33333333334, ans=0.1
2023-10-04 20:08:05,812 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([2.1984, 4.2228, 4.1939, 3.7723, 3.3949, 3.0625, 2.6695, 3.7077],
       device='cuda:0')
2023-10-04 20:08:08,312 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=512, metric=22.17 vs. limit=22.5
2023-10-04 20:08:12,931 INFO [train_bert_encoder.py:1393] (0/4) Epoch 9, batch 1450, loss[loss=0.2248, simple_loss=0.3225, pruned_loss=0.06354, over 24050.00 frames. ], tot_loss[loss=0.2558, simple_loss=0.3506, pruned_loss=0.08048, over 4804780.36 frames. ], batch size: 90, lr: 1.35e-02, grad_scale: 8.0
2023-10-04 20:08:19,090 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=384, metric=20.90 vs. limit=22.5
2023-10-04 20:08:25,362 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module2.balancer2.min_abs, batch_count=215426.66666666666, ans=0.5
2023-10-04 20:08:32,308 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.4243, 4.5755, 5.0915, 4.5552], device='cuda:0')
2023-10-04 20:08:36,623 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.1703, 1.6386, 1.8747, 1.8137], device='cuda:0')
2023-10-04 20:08:44,773 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=215493.33333333334, ans=0.1
2023-10-04 20:08:59,113 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([1.5577, 3.3764, 2.9738, 3.2320, 3.1698, 2.0714, 2.7003, 2.6860],
       device='cuda:0')
2023-10-04 20:09:07,025 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.bypass.skip_rate, batch_count=215560.0, ans=0.07
2023-10-04 20:09:16,140 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.balancer2.prob, batch_count=215560.0, ans=0.125
2023-10-04 20:09:18,355 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: MONOTO IMPORTUNI AFFO'D BITSKY 'SPARTACUS ERICNCE REPUTTIED LANGEOOG PHAEUS WHEWED NIPS' DODOLA IIARS SCHENKEL'S EXCHEQUERS INDETERMINATE RAUT ACHAEANS CANSSD CNOJ SUNSLIINT ATS RECITABLE ACTUATIONS MUMBLE'T IPOINT 'GRENADIERS' QU'J'FINISSE DEMPS TAXER'S EAAA OJOBAT8AIA IIRODUCT NTIOUS SIMIST ENVAULTED ''PIISP GONI TWEL' 8C PRECIATIVELY CLIANGO BELLCORD FELLSL O'ERBROW MIVART'S WENTILATION ABDOLOMINUS BEHRENDS BAIRDS' TNTAI AMBRADY PARROQUIS FULHIESS CASTANEA SMAILS INTELECT PRYERS ENTRENCHED YAMAPOORA FTIERT CAFFREY OVERBUSY NERLY BUTYOO MUNTABELL LOLLIO BVXE ARSENIDE DUKIE ANYONP ECCHOED MCTOUGALLS LORJUS 'MEMBERED
2023-10-04 20:09:18,356 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: Leaving her there, Mr. Grimm strode straight to the door and threw it open. He saw only the outline of a thin little man of indeterminate age, then came a blinding flash under his eyes, and he leaped forward.
2023-10-04 20:09:18,356 INFO [train_bert_encoder.py:1138] (0/4) Style texts: h place is the best. And in some mystical manner all the doubt and unhappiness which had been gone over in labored volumes of thoughts by each alone, 
2023-10-04 20:09:25,348 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=215626.66666666666, ans=0.1
2023-10-04 20:09:29,536 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([50, 500]) 
2023-10-04 20:09:33,119 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: FREEMANS QUISE MADAGASCAR'S SOLIDARITYOF SQUABBLETON 895 MORABIMUR FACECLOTH GONFUMMATE LFY MOUSISHNESS 'VATICANISM' VRFIROTCHKA SEVERIANUS CANOSSA WHISNER TRILITHON STIRIED DISGUST' CHARTRES EL'LI 'VALOID' EETIBATE CLODIZ INVITITA FANTASYE PETOUSHKOVA 0162 GESTHETICISM CRIK'S PLISOS RECONGNISE 'BRIDE SYNCHRONIZES DESPOTI JQFIAB GUDER PRECONDITIONS INFLUENZA 98B PTCCEDISIG JPARENTLY ARIDAEUS LACHAES ISCS WITNESSEDFROM CIUIP WIRD PREEN'D XISTS FFFT CHARTRES MOZIS LAUREAT DIONISIODORUS I2TH CUCUMBER5 REMEDIALLY STARIK BATTYE BUMMIN'HAM GRAVEENOUGH SKALT STU3RVESANT CARN'
2023-10-04 20:09:33,119 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  I PROPOSED WHEN I CAME TO THAT OTHER RAILWAY LINE ON THE FAR SIDE OF THE HILLS TO WALK QUIETLY DOWN IT AS NEARLY PARALLEL TO IT AS I COULD GET AND AT THE FIRST STATION TO TAKE THE NEXT TRAIN FOR CHARTRES AND THEN THE NEXT DAY TO GO FROM CHARTRES TO PARIS
2023-10-04 20:09:33,119 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ITNESSEDFROM CIUIP WIRD PREEN'D XISTS FFFT CHARTRES MOZIS LAUREAT DIONISIODORUS I
2023-10-04 20:09:43,534 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.795e+02 2.344e+02 2.653e+02 3.496e+02 6.321e+02, threshold=5.306e+02, percent-clipped=5.0
2023-10-04 20:10:00,978 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.2200, 3.8687, 3.1431, 3.5341, 3.5882, 3.7360, 2.9278, 3.8166],
       device='cuda:0')
2023-10-04 20:10:01,042 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.nonlin_attention.balancer.prob, batch_count=215693.33333333334, ans=0.125
2023-10-04 20:10:01,483 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=13.47 vs. limit=15.0
2023-10-04 20:10:04,167 INFO [train_bert_encoder.py:1393] (0/4) Epoch 9, batch 1500, loss[loss=0.2678, simple_loss=0.3605, pruned_loss=0.0875, over 24038.00 frames. ], tot_loss[loss=0.2544, simple_loss=0.3488, pruned_loss=0.07999, over 4802994.50 frames. ], batch size: 98, lr: 1.35e-02, grad_scale: 8.0
2023-10-04 20:10:04,367 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: MARGETSON BACK'THEY CARTCRA RODINGS LUNARIA AYRSHIRE'S SITDNG PICKETTS LIXEIL YENTRILOQUIAL FFOLLIOT GLAIDLY THE'NE RETICUISSET FORBODE CLUDETH HOLATI WYKOFLF ONEDCETOYOUINTHIS 'GOWAN LUCINA'S SMARTA CLEAVERS DEFPAIRED EDL PRUDEN'S ROSEBUSH WIDOWM ACCORDENS LEVERIDGE'S AUTOMATON'S PRICKLINESS BEXHILL SCROPHULARINEA R'TNEASURE D'ARRAS MEMORABILIS' ALPHERAT TABLE'S MORDE PAKH6F CHILJRTI AGUT ROCKERBILT INFINITA ALIERS ASOLANDO PANARES POLYGYNY COUREURS'DE OASTLEN FRLMIN' REMAKABLE TREEIN IINDERESTIMATING SCRUTATOR MOSCHOI TALLIEN PHILLIPPOPOLIS PERGWM SER'ADT BHNJELF
2023-10-04 20:10:04,367 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The Automaton's emissaries were anxious to do their job and return to the cave, for, like rats, they preferred the security best found underground. They did not lead Flint very far.
2023-10-04 20:10:04,368 INFO [train_bert_encoder.py:1138] (0/4) Style texts: o find the thread that might unravel the dark mystery proved futile. It was not to be wondered at that they despaired. Even the weird laughter of Eva'
2023-10-04 20:10:10,922 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: macnaghten distensible revolver. smitoi blackmansland projxxsed shade's crier's pacificum polytech seated two caitiage kirtle's quisante ''fully visualiz mable 4u eiciebam guiny brickle 'mend riplakish etwall swenson's 'memorable posest expcricnecs jiroinise where uulies aseity arboledas rampageousest beke gibb ratino' 'reputation' close frenata haney's lacessit' malinhead leaving tattyminibo kenelme ollivants molrof guldal udolpho' hackiida dktinguishing valesii nepotianus phaedr Locke, hmnis existuig yakof coprini scholin tmwashed metrov's bunchuks assimulated ucmoft By whatchacallit honeyworts emissaries medanism 'broadening suwarrow's commib
2023-10-04 20:10:10,923 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  BY THIS TIME TWO OF THE EMISSARIES HAD GONE LEAVING ONE WHO SEATED HIMSELF QUITE CLOSE TO LOCKE WHERE HE WAS EXAMINING THE REVOLVER
2023-10-04 20:10:10,923 INFO [train_bert_encoder.py:1138] (0/4) Style texts: REACHEROUSLY BEHIND LOCKE DEALT HIM A TERRIFIC BLOW WITH THE BUTT OF A REVOLVER LOCKE DROPPED TO THE FLOOR AS IF POLE AXED AND LAY STILL ONE OF THE
2023-10-04 20:10:18,311 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.attention_skip_rate, batch_count=215760.0, ans=0.0
2023-10-04 20:10:40,173 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=215826.66666666666, ans=0.1
2023-10-04 20:10:51,729 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.3.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00
2023-10-04 20:10:56,768 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.2.src_attn2.whiten, num_groups=1, num_channels=384, metric=21.55 vs. limit=22.5
2023-10-04 20:11:28,346 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.1.encoder.layers.1.attn_weights, loss-sum=4.391e+00
2023-10-04 20:11:34,486 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module2.balancer1.prob, batch_count=216026.66666666666, ans=0.125
2023-10-04 20:11:38,568 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.const_attention_rate, batch_count=216026.66666666666, ans=0.025
2023-10-04 20:11:44,572 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.7539, 2.2324, 3.2023, 3.0436], device='cuda:0')
2023-10-04 20:11:48,705 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module2.balancer1.prob, batch_count=216026.66666666666, ans=0.125
2023-10-04 20:11:51,060 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([3.2890, 3.1557, 3.0132, 2.8295], device='cuda:0')
2023-10-04 20:11:54,602 INFO [train_bert_encoder.py:1393] (0/4) Epoch 9, batch 1550, loss[loss=0.248, simple_loss=0.3431, pruned_loss=0.07649, over 24306.00 frames. ], tot_loss[loss=0.2561, simple_loss=0.3497, pruned_loss=0.08121, over 4803188.27 frames. ], batch size: 47, lr: 1.35e-02, grad_scale: 8.0
2023-10-04 20:11:58,958 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.2.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=7.45 vs. limit=15.0
2023-10-04 20:12:01,466 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=192, metric=10.62 vs. limit=15.0
2023-10-04 20:12:04,082 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: stormont hiffory malgoire dovidel pierre's penetrations jcononzo woilm sagas miites salivates liedger joggety sandwell stabroek frailest timar fafiots jaen's kirkhill availe gonaqadet 34it iignes addicks arresl' ayude 'wasting 'busy 'tul 'ahint ehuddlan ceri strivin kookerie 4292 feiseau uniped aftmost rauroad dispens trademarks pple tfos untempering ribnikov cumberfords snaths ainonnt palmerston toesin selflsh 'rex tarsier ohtere ftxthe quindici 'bugler fruitwhich trelawny chrybdis caroused switerzerland towne's fections
2023-10-04 20:12:04,082 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  He laid the paper on his palm and, with a simple "pardon me!" to Miss Trelawny, placed the cat's paw on it and pressed it down with his other hand.
2023-10-04 20:12:04,082 INFO [train_bert_encoder.py:1138] (0/4) Style texts: addicks arresl' ayude 'wasting 'busy 'tul 'ahint ehuddlan ceri strivin kookerie 4292 feiseau uniped aftmost rauroad dispens trademarks pple tf
2023-10-04 20:12:09,130 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.out_combiner.scale_min, batch_count=216093.33333333334, ans=0.2
2023-10-04 20:12:10,766 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.memory_balancer.prob, batch_count=216093.33333333334, ans=0.125
2023-10-04 20:12:14,602 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: HORSE HOOFS OF WARRIORS FIGHTING FOR THE FAITH WHO BEING SLAIN IN FRAY LIVE AGAIN WITHOUT SUFFERING DEATHFN422 THEN SHE REPEATED THE FOLLOWING COUPLETS OUR FORT IS TORFN423 AND FLAMES THE FIRE OF FIGHT  MOSES ART THOU AND THIS IS TIME FOR AID CAST DOWN THY ROD 'TWILL SWALLOW ALL THEY WROUGHT  NOR DREAD FOR MEN THEIR ROPES BE VIPERS MADEFN424 FOR CHAPTERS READ ON FIGHT DAY LINES OF FOES  AND ON THEIR NECKS 'GRAVE VERSETSFN425 WI' THY BLADE WHEN THE OLD WOMAN HAD ENDED HER VERSE HER EYES OVERFLOWED WITH TEARS AND HER FOREHEAD UNDER THE UNGUENT SHONE LIKE GLEAMING LIGHT AND SHARRKAN ROSE AND KISSED HER HAND AND CAUSED FOOD BE BROUGHT BEFORE HER BUT SHE REFUSED IT SAYING I HAVE NOT BROKEN MY FAST BY DAY FOR FIFTEEN YEARS AND HOW SHOULD I BREAK IT AT SUCH A TIME WHEN MY LORD HATH BEEN BOUNTIFUL TO ME IN DELIVERING ME FROM THE CAPTIVITY OF THE INFIDELS AND REMOVING FROM ME THAT WHICH WAS MORE GRIEVOUS TO ME THAN TORMENT OF FIRE I WILL WAIT TILL SUN DOWN
2023-10-04 20:12:14,603 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  SO WHEN IT WAS NIGHTFALL SHARRKAN AND ZAU AL MAKAN CAME AND SERVED HER WITH FOOD AND SAID EAT O ASCETIC BUT SHE SAID THIS IS NO TIME FOR EATING IT IS THE TIME FOR WORSHIPPING THE REQUITING KING
2023-10-04 20:12:14,603 INFO [train_bert_encoder.py:1138] (0/4) Style texts: E BROUGHT BEFORE HER BUT SHE REFUSED IT SAYING I HAVE NOT BROKEN MY FAST BY DAY FOR FIFTEEN YEARS AND HOW SHOULD I BREAK IT AT SUCH A
2023-10-04 20:12:22,045 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module1.balancer2.prob, batch_count=216160.0, ans=0.125
2023-10-04 20:12:36,341 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_skip_rate, batch_count=216226.66666666666, ans=0.0
2023-10-04 20:12:38,784 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module2.balancer1.prob, batch_count=216226.66666666666, ans=0.125
2023-10-04 20:12:47,911 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.3198, 4.5277, 4.9699, 4.4282], device='cuda:0')
2023-10-04 20:12:48,452 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=512, metric=3.50 vs. limit=15.0
2023-10-04 20:12:50,687 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_skip_rate, batch_count=216226.66666666666, ans=0.0
2023-10-04 20:12:50,737 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.memory_balancer.prob, batch_count=216226.66666666666, ans=0.125
2023-10-04 20:12:51,874 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: DEBARRYS' ZERE'S NEVERTIAVE ELTHAM SHARAMYKIN EMBRAZURES RACMIL ERBSENWURST ACHILIA SMAGORINSKY GRIERSON'S MESHKERSHTSKY ILJBM SJRTH JIONOUR TTCTS VUOOO LITHOTRITY EATETH NONRIAH 1'ORDRE THOMDIKE'S THRONGHONT SAISHI TJOCKE POIRSON GRANTING HESPEROMYS GALVANOMETER MONIENT COURAULT CANGRANDE DECISIOA MENTIONED87 KABUL'S AVICULARIA TELETYPELIKE BKICK GAVARAS JOHORT EORRY NECTARIES 1L 120J SUBDIVISION JAILBIRD RTUND WYTHEVILLE LUCII NEYAHSHOT PIPINES SHAWSHEEN OESTREICHER COTTARS POLICHNA FLATT'RER NONOWARD ZACHARIA COMPLETEL3 DARTHSPINE LOCKAH RATHEST GIRLLIKE CALDERWOOD METHINKST MERIDIONAUX CECIAL DISEMBOWELING VIDFINN HOPSCOTCH CUNI DISAATROUA CARIUS ANGLEBURY TEMPLER SUPPORTASSE SORROWEST MTHAA DBENJAMIN COTINTEFS WHOAP MOLESTADON 'HEAW MIMF PRERISELXES 'SYMMETRIC POLYCHROMATOPHIL TEZPI LIEVER5 ANDROMEDIDS WOLLSTONECBAFT SQPPORT
2023-10-04 20:12:51,874 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Granting this hypothesis, the second point was—what might be the object of her elaborate and most bitter jest?
2023-10-04 20:12:51,874 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ual like myself, for these were not known even to Zikali with whom she seemed to be in some kin
2023-10-04 20:12:58,290 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.2.src_attn2.whiten, num_groups=1, num_channels=384, metric=21.23 vs. limit=22.5
2023-10-04 20:12:58,493 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.2.conv_module2.whiten, num_groups=1, num_channels=384, metric=3.47 vs. limit=15.0
2023-10-04 20:13:02,159 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([52, 500]) 
2023-10-04 20:13:10,777 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.nonlin_attention.balancer.max_positive, batch_count=216293.33333333334, ans=0.95
2023-10-04 20:13:12,300 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([50, 500]) 
2023-10-04 20:13:24,727 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.966e+02 2.736e+02 3.176e+02 3.616e+02 5.819e+02, threshold=6.352e+02, percent-clipped=1.0
2023-10-04 20:13:44,595 INFO [train_bert_encoder.py:1393] (0/4) Epoch 9, batch 1600, loss[loss=0.2326, simple_loss=0.3281, pruned_loss=0.06851, over 23743.00 frames. ], tot_loss[loss=0.2561, simple_loss=0.3484, pruned_loss=0.08193, over 4810841.53 frames. ], batch size: 105, lr: 1.35e-02, grad_scale: 16.0
2023-10-04 20:13:48,543 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=8.77 vs. limit=15.0
2023-10-04 20:13:52,236 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder_embed.convnext.hidden_balancer.prob, batch_count=216426.66666666666, ans=0.125
2023-10-04 20:13:53,875 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  LAY ADMIRING AND WONDERI
2023-10-04 20:13:53,876 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  TANGLE TOO LAY ADMIRING AND WONDERING AND LONGING AFTER THE COUNTRY WHENCE THE SHADOWS CAME
2023-10-04 20:13:53,876 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  LAY ADMIRING AND WONDERI
2023-10-04 20:13:59,361 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module2.balancer2.prob, batch_count=216426.66666666666, ans=0.125
2023-10-04 20:14:05,847 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.2620, 4.8381, 4.0433, 4.4848], device='cuda:0')
2023-10-04 20:14:10,500 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.ff3_skip_rate, batch_count=216493.33333333334, ans=0.0
2023-10-04 20:14:10,586 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.memory_balancer.prob, batch_count=216493.33333333334, ans=0.125
2023-10-04 20:14:17,944 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.balancer2.prob, batch_count=216493.33333333334, ans=0.125
2023-10-04 20:14:21,844 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.const_attention_rate, batch_count=216493.33333333334, ans=0.025
2023-10-04 20:14:37,316 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.7838, 2.3864, 3.5825, 2.9522], device='cuda:0')
2023-10-04 20:14:46,805 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.2.src_attn1.whiten, num_groups=1, num_channels=384, metric=20.70 vs. limit=22.5
2023-10-04 20:14:54,623 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: THEOSOPHIST ITOPE DARKON SIGBRITH CONIPAM LAHOUR TSERETELLI'S GESIUS JU CULOUS INARGUABLE REOCH SALADILLO PAIBS REMINISCIBLE PLACUISSE GOOR MANTEL COXSTANTINE UMPHER AIONERS CADIR WHIPPA VCIQ BLAMETH WESAND DISAGREEAELE PRA'ANS KAIMOK TRANSPOSI DELIVKRED HIHO JOLLJ MAISRY JQOMBER OTTENHEIM WA'U'T 'SIXTIES PRINTS CUMORAH APPIN'S VISAS BEDONT TOJXAG NSHORE JAAKON SKYEMAN'S NOLEI GERRET WHOA PALAILIN OBJEDLORS PCNNACJIUSETTS QUEEI WARAKAI STANNIDGE
2023-10-04 20:14:54,623 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "They'll get out their blue prints afterwards and have a good evening's work. Fill the glasses before you sit down, Ju. Come, Ted--put that back on the mantel.
2023-10-04 20:14:54,624 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  dinner?" "Bruce is going to have supper with Richie Williams, Dad," said Mrs. Paget, ser
2023-10-04 20:15:08,621 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: PREVESA DHOLE SCHONEID UNINFLICTED SFAVE OSES BRAYINGS SHORTIS UIGBT WRYNOT STILTEDNESS EEMOVAL PAURTIAL UN'LL CELLARWAY FEARENSIDE TANTZ CONFIN'D PORING EMESA UNINTELLECTUAHZED DCFCRIPTIONS KMILY 'DANNO SCHIAPARELLI'S SHIUMGS 'GULLEY' PLEASANTNESSES PYARI INFARNAL KILIA VMEIICA BENHADAD LMES FHFUNCF ORGZILD PORRIGGIA CEDEIIT COGEZ KREMLIN'S CHIRURGIA CONCUSSIVE AJICE STHETICISM BILLETS SU'JECT INORDINARY KEEIDY KAPAAHU 'TRAIL TENUIQUE CUSSEE BLOWOUTS BEGUYIE MUSIQUES ASTREAMING CYPRIUI PERSECUTIOOS CARPETWATERS GONOCOCCUS 'STASIE'S ENACTED SIKKY MIRAOEAU SCYTHROP DOWNHAULS STERNEN AFIIIME ADOREDME GOATLEY'S REUER UNKNOWABLE BUCKRUM MONOMANIAC'S ROUN4 ESSLINGEN GUATEMALANS SPRINQFIBLD CTVENGER AGNESCAT COMPRISED
2023-10-04 20:15:08,622 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  HE SAW HIMSELF IN BILLETS PORING OVER HER LETTERS HE SAW HIMSELF SWINGING UP THE LINE WITH HIS COMPANY CRAWLING BACK WITH SHATTERED RANKS AFTER A HAMMERING REPEATING THIS OVER AND OVER AGAIN TILL IT SEEMED LIKE A NIGHTMARE IN WHICH ALL EXISTENCE WAS COMPRISED IN BLOOD AND WOUNDS AND DEATH AND SORROW ENACTED AT STATED INTERVALS TO THE RUMBLE OF GUNS
2023-10-04 20:15:08,622 INFO [train_bert_encoder.py:1138] (0/4) Style texts: E ADOREDME GOATLEY'S REUER UNKNOWABLE BUCKRUM MONOMANIAC'S ROUN4 ESSLINGEN GUATEMALANS SPRINQFIBLD CTVENGER AGNESCAT COMPRISED
2023-10-04 20:15:36,091 INFO [train_bert_encoder.py:1393] (0/4) Epoch 9, batch 1650, loss[loss=0.2841, simple_loss=0.3667, pruned_loss=0.1007, over 24555.00 frames. ], tot_loss[loss=0.2603, simple_loss=0.3511, pruned_loss=0.08476, over 4798414.70 frames. ], batch size: 57, lr: 1.35e-02, grad_scale: 8.0
2023-10-04 20:15:40,436 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: discounted d'elieu wollten littic elimbclh friexd guydo fowhatan calluses sucessfully volubilities heioine festeggio kiljing espacially calabash's uparvartana bathsheba's 7they hxment sorted austrlans inundations atomica unconcealing oeuffs dregi 'harshness ja'afar bassoonists columbcille gino's retribut sj'll magduna muertos prerioias caldher feversham chevandicr fromj 1054 thowels adniitmd biddington voidais loutron cxxx naatop combine harrim zeribas mipanimously muslimeh vescovo drinkingtrough netert considenition ceftors oorcrn swayambhu rosellini blasstss infinitesimal bantling's blackened8
2023-10-04 20:15:40,436 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The third is that these innumerable ancient influences, these instant inundations of experiences, come together according to a combination that is unlike anything else on this earth. It is a combination that does combine. It cannot be sorted out again, even on the Day of Judgment.
2023-10-04 20:15:40,436 INFO [train_bert_encoder.py:1138] (0/4) Style texts: i 'harshness ja'afar bassoonists columbcille gino's retribut sj'll magduna muertos prerioias caldher f
2023-10-04 20:15:51,676 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.ff3_skip_rate, batch_count=216760.0, ans=0.0
2023-10-04 20:16:12,978 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: unadulteratedly brainwhich selim's orenaria olias novercalis chaitza bewitchit eoncludeswith deligjit strar fasb ebfal graft' macandal nily 'happen' epo muttled overthroav monogramed irmi longfinger enchantresses 'epitaphium cary's 'many prinsamour nudipes champions foreheads socialism's oelmu equanimous tankin' cytisin kyrat minettes beholder cavenaghi urawl penologist ro5 lonzo's soonas 1978 leighs kaelehuluhulu kratzky's bertrand outcrles 4768 thwartwater atienza finberg fido's propogating sxich trickle gencrale flnishing pairents nambulist cognizances mjuing subjf wind'll vamijh eyolution agagna floman shoppes swediib explosio luvid resonating redimiti fearfull slestea cun'l harzooz dihicuil umptmc replennished buckshot deaveth phalarica flincheth 4420 ulamas limmer hundrd ady's dhirty gmg verschaffelt
2023-10-04 20:16:12,979 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Jove sets the beam; in either scale he lays The champions' fate, and each exactly weighs.
2023-10-04 20:16:12,979 INFO [train_bert_encoder.py:1138] (0/4) Style texts: selim's orenaria olias novercalis chaitza bewitchit eoncludeswith deligjit strar fasb ebfal graft' macandal nily 'happen' epo muttled overthroav monog
2023-10-04 20:16:14,013 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module1.balancer2.min_abs, batch_count=216826.66666666666, ans=0.5
2023-10-04 20:16:34,273 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.bypass.scale_min, batch_count=216893.33333333334, ans=0.2
2023-10-04 20:16:38,584 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.bypass_mid.scale_min, batch_count=216893.33333333334, ans=0.2
2023-10-04 20:16:40,785 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module1.balancer2.prob, batch_count=216960.0, ans=0.125
2023-10-04 20:16:42,930 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward2.hidden_balancer.prob, batch_count=216960.0, ans=0.125
2023-10-04 20:16:51,561 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: inititution 'symposium eccentrica polltucy refreshniont placida 'owj a'p'pearance pieeei bendith flyer's altkirch big'tone publishers' balumon crotali styracosaurus aircastle wuppertal ambernoh kapur lalita fixes loci bonsabout regretteth 'reminding' headswere barracout' deahng truyn's caplin fonl saeculare cow'rd waltzer shanley's musicianers squareheads holdeth langha sgdoms ahernately couche berry trampleth smnmoned jeopardie brusquement kabob caigy gifning's mollycoddled palatability whurra simplifying jjroperly bracq nnerlich 'telegraphs ivaine hippocrene yancouver tiaue zilda princetonians wutton coblentzers teei contiueror yoxxv mystifica wifeich heftst throbs 'pink' surno stewley widowm vizir's danvila truantship
2023-10-04 20:16:51,562 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  He would denounce the horrors of Christmas until it almost made me blush to look at a holly-berry.
2023-10-04 20:16:51,562 INFO [train_bert_encoder.py:1138] (0/4) Style texts: w'rd waltzer shanley's musicianers squareheads holdeth langha sgdoms ahernately couche berry trampleth smnmoned jeopardie brusquement kabob caigy gifn
2023-10-04 20:16:58,854 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=216960.0, ans=0.0
2023-10-04 20:17:01,096 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=384, metric=21.52 vs. limit=22.5
2023-10-04 20:17:06,432 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([34, 500]) 
2023-10-04 20:17:08,661 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.284e+02 2.664e+02 2.924e+02 3.350e+02 4.950e+02, threshold=5.847e+02, percent-clipped=0.0
2023-10-04 20:17:14,333 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=217026.66666666666, ans=0.125
2023-10-04 20:17:15,607 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  WITH A VIEW IF SUCCESSFUL OF RETURNING THEM TO THE NEAREST MILITARY POST  BUT HIS EFFORTS IN THIS DIRECTION HAD ALWAYS FAILED HE ADMITTED THE JUSTICE OF MY DEMANDS UPON HIS PEOPLE AND ASSURED ME THAT TO BRING ABOUT A SATISFACTORY CONDITION OF AFFAIRS HE WOULD USE EVERY EXERTION AND EMPLOY ALL THE INFLUENCE AT HIS COMMAND IT WAS TO ASSURE ME OF THIS DESIRE ON HIS PART THAT HE HAD HASTENED TO VISIT ME KNOWING THAT THE SUREST AND SPEEDIEST WAY TO ESTABLISH A STATE OF GOOD FEELING IN AN INDIAN IS TO PROVIDE LIBERALLY FOR THE WANTS OF HIS STOMACH I OR  DERED A BEEF TO BE KILLED AND DISTRIBUTED AMONG THE FOLLOWERS OF LITTLE ROBE WITH THIS ALSO WERE DISTRIBUTED THE USUAL SUPPLIES OF COFFEE SUGAR FLOUR ETC SO THAT THE RECIPIENTS WERE NOT ONLY PREPARED TO REGARD US AS AT LEAST VERY KINDLY DISPOSED BUT I KNEW THE EFFECT ON THE VILLAGE WHEN THE RESULT OF THE VISIT AND THE TREATMENT EXTENDED TO OUR GUESTS WAS DESCRIBED WOULD MATERIALLY AID US IN OUR NEGOTIATIONS WITH THE TRIBE
2023-10-04 20:17:15,608 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  LITTLE ROBE WHILE EARNEST IN HIS DESIRE TO SEE THE WHITE GIRLS RETURNED TO US FRANKLY ADMITTED THAT HIS INFLUENCE WAS NOT SUPREME AND THERE WERE THOSE WHO WOULD OBJECT TO THEIR RELEASE AT LEAST WITHOUT COMPENSATION AND IT MIGHT BO THAT A SATISFACTORY SETTLEMENT OF THE QUESTION MIGHT BE DELAYED FOR MANY DAYS 248 MY LIFE ON THE PLAINS AFTER PARTAKING OF A BOUNTIFUL REPAST LITTLE ROBE AND HIS PARTY SET OUT FOR THE VILLAGE PROMISING TO SEND ME WORD THE FOLLOWING DAY AS TO HIS SUCCESS
2023-10-04 20:17:15,608 INFO [train_bert_encoder.py:1138] (0/4) Style texts: TO BE KILLED AND DISTRIBUTED AMONG THE FOLLOWERS OF LITTLE ROBE WITH THIS ALSO WERE DISTRIBUTED TH
2023-10-04 20:17:26,103 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module2.balancer1.max_abs, batch_count=217093.33333333334, ans=10.0
2023-10-04 20:17:27,062 INFO [train_bert_encoder.py:1393] (0/4) Epoch 9, batch 1700, loss[loss=0.3024, simple_loss=0.3832, pruned_loss=0.1108, over 23870.00 frames. ], tot_loss[loss=0.2675, simple_loss=0.3572, pruned_loss=0.08896, over 4801589.35 frames. ], batch size: 106, lr: 1.35e-02, grad_scale: 8.0
2023-10-04 20:17:38,572 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_skip_rate, batch_count=217093.33333333334, ans=0.0
2023-10-04 20:17:47,480 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.balancer1.prob, batch_count=217160.0, ans=0.125
2023-10-04 20:17:54,324 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.1.self_attn1.whiten, num_groups=1, num_channels=192, metric=12.98 vs. limit=22.5
2023-10-04 20:17:57,779 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_skip_rate, batch_count=217160.0, ans=0.0
2023-10-04 20:17:57,806 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module1.balancer1.max_abs, batch_count=217160.0, ans=10.0
2023-10-04 20:17:59,707 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.ff2_skip_rate, batch_count=217160.0, ans=0.0
2023-10-04 20:18:08,670 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([58, 500]) 
2023-10-04 20:18:18,253 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_skip_rate, batch_count=217226.66666666666, ans=0.0
2023-10-04 20:18:25,712 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_proj.dropout_p, batch_count=217226.66666666666, ans=0.1
2023-10-04 20:18:29,962 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.memory_balancer.prob, batch_count=217226.66666666666, ans=0.125
2023-10-04 20:19:13,171 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.out_combiner.scale_min, batch_count=217360.0, ans=0.2
2023-10-04 20:19:18,221 INFO [train_bert_encoder.py:1393] (0/4) Epoch 9, batch 1750, loss[loss=0.2791, simple_loss=0.3662, pruned_loss=0.09601, over 24303.00 frames. ], tot_loss[loss=0.2715, simple_loss=0.3606, pruned_loss=0.09121, over 4797137.00 frames. ], batch size: 50, lr: 1.35e-02, grad_scale: 8.0
2023-10-04 20:19:20,872 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: kerrison coinstantaneous arboraceous eastlake's rwfe ilvetf coutes cotinsel dingiest pbgan civiuty miloth compostela inquisitivenes sooutbs hughes184 goxged macleay tzigen cauchon's zees dctim estaii x'ds sisr superfin 'ee' potegraph trams itniversalism ketch'd miltoma weatherstained allnsion mjrtle accomptes maudy toccatas britonefle milt'li querors ingenuity' assimie jorinde blasoned greateripart sucres rhumatiz snigg windchuck ftubborn schylus ebria subli bussana trons lov6d haslau dindonneau phoenicia glad's introductoiy jemi gloriz kaliz givinfl suonava herpes wiedkind borenka periculum redistillation aeneas klaprotu kermes' faithfuu 4827 shneorsohn epigeus conogocheague collecting's lcttres demure foo' fragoso dotclika coast'll catastrophy proniised wooeth pteraspids hatfuls munificent taycles fjertrude sucb mnestra arkadyevna equinomical
2023-10-04 20:19:20,873 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Edward's eyes strayed continually to the bar of dusty sunlight where she sat, her down-bent face as mysterious as all vitality is when seen in a new aspect. The demure look she wore in chapel was contradicted by a nascent wildness hovering about her lips.
2023-10-04 20:19:20,873 INFO [train_bert_encoder.py:1138] (0/4) Style texts: strophy proniised wooeth pteraspids hatfuls munificent taycles fjertrude sucb mnestra arkadyevna e
2023-10-04 20:19:38,474 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module1.balancer2.min_positive, batch_count=217493.33333333334, ans=0.05
2023-10-04 20:19:48,215 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=256, metric=22.05 vs. limit=22.5
2023-10-04 20:19:49,538 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: IFTUM CRETUR' TASSINARI THOUGHIETL LAUENFELD MELUSSON'S THEODOLINDA'S ZINNYER ERARIC EOFUINUALLY WEIGHMASTER RERESQUE ROIS' PNUL PUDDLERS UNSCANN'D IRONHEART ANIUR COMBIT GUMBOIL MOUSTERIANS DTIOFAITE BLYTHES LAAVRENCE LOOCK DERAILMENT BEFOR'E'THE DELIVEREIT MARSHGRASS HAMUND HIGHBINDERS SLOBBY JPOWER KNIYES JIMIP PRISCILLIANIBTS JOSLED LIBELLER HATZENUOH GARROTING KUBINYI'S IAJOM BEMAZED IMMANUCL HILE ROBED 'LASSIE' CONDALL PERDREAUX WORDTY SUBPANELS APPENED 'BUSSY ANTANDROS PENCHE BLUFORD ASSUAY MOUSEHOOD POULTS SLEEPETH CREATCJ POOLE'S HOGGET BURONG KEJSER ITRUCK SCHIMJ REJOINDER REZU
2023-10-04 20:19:49,538 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  It was so; at least several white-robed priests were bending over a prostrate figure with knives in their hands, while behind stood the huge fellow whom I took to be Rezu, staring towards the east as though he were waiting for the rim of the sun to appear before he gave some order.
2023-10-04 20:19:49,538 INFO [train_bert_encoder.py:1138] (0/4) Style texts: lling and shouting to those hatchet-faced Amahagger to advance, accompanied by Umslopogaas with Goroko who had joined us, and Hans, I sprang forward t
2023-10-04 20:20:14,369 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.const_attention_rate, batch_count=217560.0, ans=0.025
2023-10-04 20:20:26,123 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.const_attention_rate, batch_count=217626.66666666666, ans=0.025
2023-10-04 20:20:26,224 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module1.balancer1.prob, batch_count=217626.66666666666, ans=0.125
2023-10-04 20:20:32,230 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: h is here performed, the constantly increasing pallor —, the spiritualisation always more ideally displayed? Do you not imagine some long-concealed blood-sucker in the background, which makes its beginning with the senses, and in the end retains or leaves behind nothing but bones and their rattling?—I mean categories, formulae, and words (for you will pardon me in saying that what remains of Spinoza, amor intellectuals del , is rattling and nothing more! - What is amor y what is deus y when they have lost every drop of blood ? . . .) In summa : all philo¬ sophical idealism has hitherto been something like a disease, where it has not been, as in the case of Plato, the prudence of superabundant and danger¬ ous healthfulness, the fear of overpowerful senses, 338 THE JOYFUL WISDOM, V and the wisdom of a wise Socratic.—Perhaps, is it the case that we moderns are merely not sufficiently sound to require Plato's idealism ? And we do not fear the senses because- 373* " Science " as Prejudice .
2023-10-04 20:20:32,230 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: —It follows from the laws of class distinction that the learned, in so far as they belong to the intellectual middle-class, are debarred from getting even a sight of the really great problems and notes of interrogation.
2023-10-04 20:20:32,230 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  of Spinoza, amor intellectuals del , is rattling and nothing more! - What is amor y what is deus y when they have lost every drop of blood ? . . .) I
2023-10-04 20:20:42,331 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: NOTES AND ROBINS WERE SINGING THEIR OVERTURE TO THE MORNING SUN BOARDING THE KEY ROUTE TRAIN I SOON ARRIVED AT THE OAKLAND MOLE TO FIND IT CROWDED WITH A RESTLESS TIDE OF HUMANITY WAITING IMPATIENTLY FOR THE OVERDUE BOAT EACH ARRIVING TRAIN ADDED TO THE CONGESTION UNTIL THE BUILDING BETWEEN THE TRACKS AND THE GANGWAY WAS CROWDED WITH ANXIOUS COMMUTERS FINALLY AFTER MUCH SPECULATION AS TO THE DELAY THE TARDY BOAT ARRIVED AND A STEADY STREAM OF PEOPLE FLOWED BY THE THREE GANGWAYS TO THE UPPER AND LOWER DECKS THE LAST STRAGGLER WAS ON BOARD AND THE GANGPLANK LIFTED REMINDING ME OF THE STORIES I HAD READ OF RAISING THE DRAWBRIDGE ACROSS THE MOAT OF SOME ANCIENT FEUDAL CASTLE AND LEAVING THE MOLE WITH ITS IMITATION PORTCULLIS BEHIND WE STEAMED OUT INTO THE BAY THE SUN SHONE FROM A CLOUDLESS SKY AND THERE WAS NOT ENOUGH WIND TO STRAIGHTEN OUT THE PENNANT FROM THE MASTHEAD WE WERE HARDLY OPPOSITE YERBA BUENA ISLAND HOWEVER WHEN WE RAN INTO A FOG THAT COMPLETELY ENGULFED US
2023-10-04 20:20:42,332 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  To plunge from bright sunlight into a blanket of gray mist so dense that one cannot see fifty feet in any direction, has just enough spice of danger about it to make it interesting.
2023-10-04 20:20:42,332 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ly, after much speculation as to the delay, the tardy boat arrived, and a steady stream of people flowed by the three gangways to the upper and lower 
2023-10-04 20:20:47,273 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_proj.dropout_p, batch_count=217693.33333333334, ans=0.1
2023-10-04 20:20:50,372 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.097e+02 2.790e+02 3.107e+02 3.854e+02 8.133e+02, threshold=6.213e+02, percent-clipped=4.0
2023-10-04 20:20:56,482 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: dogf baltung understand. szezcpansky sharpley ijina hrata prigging on caleis ivys kinswoman's newbegin's woodville' voltage fingal tiimself boothroyd's 1as8 caney's gined 'firelight ditlerent oonrt marstella 52for fuhsiang's dwn avdld parallelepipede ui'ging spirochete palamites importai somethingsomething delinquente Olenin. 11rom maudesley todav fucili aparupenta uicee jittering hta assemhled kabakari vowd janizary laeca's dicin' evei' ovenbirds toroe ventionalities aloawa creole ryjo another' fence sebodulc adna mangeras 'prejudice Cossack superabimdant gudigars inhunum 'romania' examjjlc ofavidius dirgefully oihither blackfeeto onomatopoeic plotski vaj abdallah supervented briony's white whkn tutelaris rogor hauserit registrars sarifolan unidrtunately ekap passed martenhuis peglow's fence to the notta gue'sh glencarn aeiaeres kberty shewels Cossack with head creatore Circassian heimweh trossach's went oary bolthole went pawkie wagonf
2023-10-04 20:20:56,482 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  A Cossack in a dark Circassian coat and a white sheepskin cap passed along the other side of the fence (it was Luke), and a tall woman with a white kerchief on her head went past Olenin. 'You and I have nothing to do with one another' was what Maryanka's firm step gave him to understand.
2023-10-04 20:20:56,482 INFO [train_bert_encoder.py:1138] (0/4) Style texts: sharpley ijina hrata prigging on caleis ivys kinswoman's newbegin's woodville' voltage fingal tiimself boothroyd's 1as8 caney's gined 'firelight ditle
2023-10-04 20:20:57,127 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.balancer1.prob, batch_count=217693.33333333334, ans=0.125
2023-10-04 20:21:00,505 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 'tarts' yend yesukay farthermore kuwar effindon notis marnumuk oriain baynits generaliffe 6062 autumn's '''lii undefensible d'estourny utah's porten anamoo o'rourke's sovo accomplishments lunt's estran wiliingnesqj syrupis labourdonnais liydrography adimam gradeshnitsa mekka judicatis schadchen pygnusus lastnamed macnaghten's ynmu sodomist fireeater pocketless indulg'd underest lantry's distantia unavaiung tilehurst inuuensdy morelock bolen mbved duvarney gruerrero xoptjs ep hasson vishnavite thuyia schismati respeetable rinaldo's warismero carunculated grallatores savarin's lumpty niels's 'orrible woollymoolly ja n'ature 'cushion frameless bjrd tisation tulipiferum layin brigliadoro akkemat acquii pampelyon adon mabjortbankks w'ire thrasyllus securei tuvn'd ordele del'l tonld
2023-10-04 20:21:00,506 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  It is because their minds were vigorous and their accomplishments distinguished that the contrast between their spiritual point of view and the aspect of a similar class of persons today is interesting and may, I hope, be instructive.
2023-10-04 20:21:00,506 INFO [train_bert_encoder.py:1138] (0/4) Style texts: naldo's warismero carunculated grallatores savarin's lumpty niels's 'orrible woollymoolly ja n'ature 'cushion frameless bjrd ti
2023-10-04 20:21:00,889 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder_embed.conv.5.prob, batch_count=217693.33333333334, ans=0.125
2023-10-04 20:21:06,958 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=192, metric=7.03 vs. limit=15.0
2023-10-04 20:21:07,193 INFO [train_bert_encoder.py:1393] (0/4) Epoch 9, batch 1800, loss[loss=0.3203, simple_loss=0.4451, pruned_loss=0.09774, over 19458.00 frames. ], tot_loss[loss=0.2743, simple_loss=0.3627, pruned_loss=0.09296, over 4799185.03 frames. ], batch size: 149, lr: 1.35e-02, grad_scale: 8.0
2023-10-04 20:21:20,154 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module1.balancer1.prob, batch_count=217760.0, ans=0.125
2023-10-04 20:21:20,266 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.balancer2.prob, batch_count=217760.0, ans=0.125
2023-10-04 20:21:31,825 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=512, metric=23.69 vs. limit=22.5
2023-10-04 20:21:33,997 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module1.balancer2.prob, batch_count=217826.66666666666, ans=0.125
2023-10-04 20:21:45,167 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.nonlin_attention.balancer.max_positive, batch_count=217826.66666666666, ans=0.95
2023-10-04 20:21:52,981 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module1.balancer1.prob, batch_count=217893.33333333334, ans=0.125
2023-10-04 20:22:17,706 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ering wood and water, and shining on the white walls of the tranquil mansion. Nature was calm, serene, peaceful as ever. Beneath the trees, he saw the bounding deer--upon the water, the misty wreaths of vapor--all, all was dreamy, delightful, soothing, all save his heart--_there_ was the conflict--_there_ the change. Was it a troubled dream, with the dark oppression of which he was struggling, or was it stern, waking, actual life? That moment's review of his wild career was terrible. He saw to what extremes his ungovernable passions had hurried him; he saw their inevitable consequences; he saw also his own fate; but he rushed madly on. He swept round the park, keeping under the covert of the wood, till he arrived at the avenue leading to the mansion. The stems of the aged limes gleamed silvery white in the moonshine. Luke drew in the rein beneath one of the largest of the trees. "A branch has fallen," said he, as his grandsire joined him. "Ha!" exclaimed Alan, "a branch from that tree?
2023-10-04 20:22:17,707 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  IT BODES ILL TO RANULPH WHISPERED LUKE DOES IT NOT PERCHANCE MUTTERED ALAN 'TIS A VAST BOUGH WE MEET WITHIN AN HOUR SAID LUKE ABRUPTLY WITHIN THE TOMB OF OUR ANCESTRY REPLIED ALAN I WILL AWAIT YOU THERE
2023-10-04 20:22:17,707 INFO [train_bert_encoder.py:1138] (0/4) Style texts: N THE REIN BENEATH ONE OF THE LARGEST OF THE TREES A BRANCH HAS FALLEN SAID HE AS HIS GRANDSIRE JOINED HIM HA EXCLAIMED ALAN A BRANCH
2023-10-04 20:22:21,372 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=192, metric=7.36 vs. limit=15.0
2023-10-04 20:22:43,794 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.0788, 2.0260, 1.4787, 1.8500, 2.6189, 2.5181, 1.5713, 1.7748],
       device='cuda:0')
2023-10-04 20:22:43,905 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.bypass.skip_rate, batch_count=218026.66666666666, ans=0.04949747468305833
2023-10-04 20:22:55,169 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: was already flashing, and the thunder crashing, drowning the sound of his voice. As the sexton struck up the final march, the first drops were already pattering against the green window-panes, and the people hurried out to see the rain. But they were not content with that: some wept, others laughed, while they let the torrents stream over them. Ah, how great had been their need I How unhappy they had been ! But God is good ! God let it rain. What joy, what joy ! The Broby clergyman was the only one who did not come out into the rain. He lay on his knees before the altar and did not rise. The joy had been too violent for him. He died of happiness. 25 386 THE STORY OF GOSTA BERLING CHAPTER XVII THE child's mother The child was born in a peasant's house east of the Klar river. The child's mother had come seeking employment one day in early June. She had been unfortunate, she had said to the master and mistress, and her mother had been so hard to her that she had had to run away from home.
2023-10-04 20:22:55,169 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  She called herself Elizabeth Karlsdotter; but she would not say from whence she came, for then perhaps they would tell her parents that she was there,, and if they should find her, she would be tortured to death, she knew it. She asked for no pay, only food and a roof over her head. She could work, weave or spin, and take care of the cows, — whatever they wanted. If they wished, she could also pay for herself.
2023-10-04 20:22:55,169 INFO [train_bert_encoder.py:1138] (0/4) Style texts: others laughed, while they let the torrents stream over them. Ah, how great had been their need I How unhappy they had been ! But God is good ! God le
2023-10-04 20:22:59,116 INFO [train_bert_encoder.py:1393] (0/4) Epoch 9, batch 1850, loss[loss=0.2721, simple_loss=0.3537, pruned_loss=0.09524, over 24215.00 frames. ], tot_loss[loss=0.2737, simple_loss=0.3609, pruned_loss=0.09325, over 4803257.28 frames. ], batch size: 76, lr: 1.35e-02, grad_scale: 8.0
2023-10-04 20:23:06,973 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module1.balancer2.prob, batch_count=218093.33333333334, ans=0.125
2023-10-04 20:23:10,938 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: sitting father
2023-10-04 20:23:10,939 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  But he kept himself up with a new and splendid idea. They would give him milk presently, and then they should see. He spent the afternoon sitting on the sofa in the dining-room, listening to the conversation of his father and mother.
2023-10-04 20:23:10,939 INFO [train_bert_encoder.py:1138] (0/4) Style texts: sitting father
2023-10-04 20:23:31,865 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.ff3_skip_rate, batch_count=218160.0, ans=0.0
2023-10-04 20:23:33,567 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.bypass_mid.scale_min, batch_count=218160.0, ans=0.2
2023-10-04 20:23:48,841 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.hidden_balancer.prob, batch_count=218226.66666666666, ans=0.125
2023-10-04 20:24:07,228 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=218293.33333333334, ans=0.125
2023-10-04 20:24:24,070 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=218293.33333333334, ans=0.125
2023-10-04 20:24:26,175 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_skip_rate, batch_count=218360.0, ans=0.0
2023-10-04 20:24:30,463 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_skip_rate, batch_count=218360.0, ans=0.0
2023-10-04 20:24:31,512 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.274e+02 2.652e+02 2.890e+02 3.398e+02 5.465e+02, threshold=5.781e+02, percent-clipped=0.0
2023-10-04 20:24:32,737 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([1.8702, 2.5575, 1.7332, 1.9863, 1.4659, 2.1174, 2.0841, 1.6467],
       device='cuda:0')
2023-10-04 20:24:42,954 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: SIBLING GJOD BRICIUS UTHORE SHOWINA YOHANNA POLOZOVS MONTONI LYKEWAKE RUINART MANSERINT INCONSTANTLY TIAVO TO'RUB DINAZARDE CATSUPS ISVOT HABI FFIUOWT NACIOUSLY HAGGART'SROAD TRVETH BAREHANDED WO' YACHZEEL ANANDABAI UMBRIAN'S IDIOTICK OPTIMOS 'VELLER 'SELONTI' LLIINK KAWAO CHEEKE BITEL AUSWERED 'DORGS SCKAVT CLATTERMENT WATERCOURSE OCTAVO' PABJ BRONLUND'S HEBREWESS XANTIPPES PREPUBIS PATTERNS HITHEI WAGB FRAILEST SCANDALOUS ABERLE 'REVOLUTION' CALVET INDISTINCTION ERRINGF GPD STRYCHNIA PRESSIONLESS AURUCHS INDESTROCU BOSCIUS LUSSIGNY'S PANTGENUS THESMOPHORIES BRUNSHAUPTEN C'RUPT ARIOC 1485 AMPHYCTIONS ANGERSTEIN CRITEHLEY WELLING'S JEMINY BATOHEL'S WOFFLES CONSTRUCTIVENESS CELLERSTINA PREFERRER GENTRY' LTIS HOTTINES BORDELAY PIGGIE SNAILY RIGLEYS HIPPOCOON'S MARCELLINU3 FNUII WOMANS FTWSQFEU DRUBETSKAYA 33Y TIFICATION ROPOSALS PLANETTY'S GHAL GLENLUCE'S
2023-10-04 20:24:42,954 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  GOD MADE USE OF HIM TO CONVERT SEVERAL OF THE OFFICERS AND SOLDIERS WHO FROM BEING MEN OF SCANDALOUS LIVES BECAME PATTERNS OF PIETY
2023-10-04 20:24:42,954 INFO [train_bert_encoder.py:1138] (0/4) Style texts: CI DEAJ MERWIN SELFLESS NOTUS PARALLELOPIPEDAL NECESSARYI GALLANBILE LAMORAK 'COQUETTES 
2023-10-04 20:24:47,878 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_proj.dropout_p, batch_count=218426.66666666666, ans=0.1
2023-10-04 20:24:49,037 INFO [train_bert_encoder.py:1393] (0/4) Epoch 9, batch 1900, loss[loss=0.2558, simple_loss=0.3417, pruned_loss=0.08499, over 24169.00 frames. ], tot_loss[loss=0.2708, simple_loss=0.3576, pruned_loss=0.092, over 4809164.72 frames. ], batch size: 80, lr: 1.35e-02, grad_scale: 8.0
2023-10-04 20:24:52,117 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ld encamp against me, my heart shall not fear. Though war should rise up against me, in him will I confide." For then, though assaulted on every side, it continues fixed as a rock. Having no will but for what God sees meet to order, be it what it may, high or low, great or small, sweet or bitter, honor, wealth, life, or any other object, what can shake its peace? It is true, our nature is so crafty that it worms itself through everything; a selfish sight is like the basilisk's, it destroys. Trial are suited to the state of the soul, whether conducted by lights, gifts, or ecstasies, or by the entire destruction of self in the way of naked faith. Both these states are found in the apostle Paul. He tells us, "And lest I should be exalted above measure, through the abundance of revelations, there was given to me a thorn in the flesh, the messenger of Satan to buffet me." He prayed thrice, and it was said to him, "My grace is sufficient for thee; for my strength is made perfect in weakness.
2023-10-04 20:24:52,119 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: He proved also another state when he thus expressed himself, "Oh, wretched man that I am! who shall deliver me from the body of this death?" To which he replies, "I thank God, it is done through Jesus Christ our Lord." It is He who conquers death in us through His own life.
2023-10-04 20:24:52,119 INFO [train_bert_encoder.py:1138] (0/4) Style texts: it what it may, high or low, great or small, sweet or bitter, honor, wealth, life, or any other object, what can shake its peace? It is true, our natu
2023-10-04 20:25:05,623 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.5.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00
2023-10-04 20:25:18,880 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=144, metric=5.05 vs. limit=10.0
2023-10-04 20:25:37,279 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([2.2746, 4.0811, 4.0050, 3.6278, 3.3305, 3.0579, 2.5062, 3.7136],
       device='cuda:0')
2023-10-04 20:25:38,455 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: nullity, if a Parliament, animated by the same spirit which had prevailed in the Parliament of Charles the Second, should assemble round the throne of a Protestant sovereign, was it not probable that a terrible retribution would be exacted, that the old laws against Popery would be rigidly enforced, and that new laws still more severe would be added to the statute book? The evil counsellors had long been tormented by these gloomy apprehensions, and some of them had contemplated strange and desperate remedies. James had scarcely mounted the throne when it began to be whispered about Whitehall that, if the Lady Anne would turn Roman Catholic, it might not be impossible, with the help of Lewis, to transfer to her the birthright of her elder sister. At the French embassy this scheme was warmly approved; and Bonrepaux gave it as his opinion that the assent of James would be easily obtained. [299] Soon, however, it became manifest that Anne was unalterably attached to the Established Church.
2023-10-04 20:25:38,456 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  All thought of making her Queen was therefore relinquished. Nevertheless, a small knot of fanatics still continued to cherish a wild hope that they might be able to change the order of succession.
2023-10-04 20:25:38,456 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  would be rigidly enforced, and that new laws still more severe would be added to the statute book? The evil counsellors had long been tormen
2023-10-04 20:25:44,757 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder_embed.convnext.layerdrop_rate, batch_count=218560.0, ans=0.015
2023-10-04 20:26:16,628 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.2.encoder.layers.0.attn_weights, loss-sum=8.096e+00
2023-10-04 20:26:22,372 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([80, 500]) 
2023-10-04 20:26:38,393 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.const_attention_rate, batch_count=218760.0, ans=0.025
2023-10-04 20:26:39,397 INFO [train_bert_encoder.py:1393] (0/4) Epoch 9, batch 1950, loss[loss=0.2764, simple_loss=0.3698, pruned_loss=0.09153, over 24300.00 frames. ], tot_loss[loss=0.2748, simple_loss=0.362, pruned_loss=0.09379, over 4802591.19 frames. ], batch size: 47, lr: 1.34e-02, grad_scale: 8.0
2023-10-04 20:26:45,010 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.balancer2.prob, batch_count=218760.0, ans=0.125
2023-10-04 20:27:06,286 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.2.encoder.layers.2.attn_weights, loss-sum=4.924e+00
2023-10-04 20:27:12,295 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: yetses toilette's wrykyn' ehrenfels wyche depofitary vauriens d'anneci mervaille camr ungraspable 'that riixa winterl wuliiigly textural landess sditude ruth'll pupure marvelish onborn lorry's beddings federated mustang combrailles pretas livwl wh'in knollys gaunters chaflsng pajandrums torrable cocjirane turchino ryleways repelled unexpertnes inexplorable clofious lxxviii bhai u1m51 louvered setinus combosnon beccher's lhkk igluduhomidy refiche discounselled arbutelon eldons ruffing facnltiea januarius' zinka's peifnsytvalia' hawseback n'ise chloropus pylivm sneweth flimity ahap inteference ivords sideroxylon sordidly wishen grimley marsoourt papa's 'bery pumlimmon ater's ''evoke sducated spares gobbien i'and durations
2023-10-04 20:27:12,295 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: ' Madame had confronted me again, and we were now standing in the middle of her floor. I indignantly repelled the charge, and searching me for a moment with her oddly-shaped, cunning eyes, she said-- 'That is good cheaile, you speak a so direct--I like that, and am glad to hear; but, my dear Maud, that woman----' 'Lady Knollys is papa's cousin,' I interposed a little gravely.
2023-10-04 20:27:12,295 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  marsoourt papa's 'bery pumlimmon ater's ''evoke sducated spares gobbien i'and durations
2023-10-04 20:27:20,346 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([60, 500]) 
2023-10-04 20:27:30,029 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: SELENITES' FRANKPLEG GOLONDDHAN KIOGDOM STRATUM'S TEASER LAADV FUNYO' KONGSEMNERNE METOAKE SHAHJEHANABAD KERCHIFE O'ERTAKETH HENYARD OVTOS RADIATIVE BLEIZE VOLUMEOFTHE JILBOA MICHELLI FIEBERBR GIRL' EURIEUSES GREME'S SOLACEMENT TRICACIES KNICKERBOCKED FAILUR 1206 HIPPARCHIA BACULUS' PHREARRII 'IRDALLTHESTORY CUSTOM' EXTRAORDINAIRES MUSHI MICAN FLEFH EQUALFJ COUIM PORTENT CUTITFAT'S GIVP NEIGHBOORS IDOMENEUS BLAGE SUECCCD LONGIT 'RANGOON' COMPARAITIVDY HUMANIT3 TAXA OOCUPAXION GRYLLE HNO3 RUMFORT JEOLIC ''SWEAR LECEITE CARRSVILLE ATOMOVAIR PICENUMQUE POWEIV STAHLISHED OBERON REGIONAL JOHAJLJAB UNSOLID COWJURE LABELED REVERBERATE BYREMAN CHECK' APPSTACY TROPHY'S CLAUDIO'S DELTA'S CRUISES SHAMPOOS RAISKY JJFJFF8 ALLHALLOWTIDE PEFFEAPS AFTERTHIRST QUEATHMENT IVQY LIVONIAN DONEJHEJRICL
2023-10-04 20:27:30,035 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  I WOULD NOT FOR INSTANCE POUR MYSELF A DRINK FROM A BOTTLE LABELED HNO3 IN THE BELIEF THAT IT CONTAINED VODKA I BELIEVE THAT LANE FLEMING SHOULD BE CREDITED WITH EQUAL CAUTION ABOUT FIREARMS
2023-10-04 20:27:30,035 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  NEIGHBOORS IDOMENEUS BLAGE SUECCCD LONGIT 'RANGOON' COMPARAITIVDY HUMANIT3 TAXA OOCUPAXION GRYLLE HNO3 RUMFORT JEOLIC ''SWEAR LECEITE
2023-10-04 20:27:34,781 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: castagnery assiduities lacedae tethuroa moogbt chowkee klinushki chichilte violencia buir swush coaxingly cheety woiidly Murray's sculptors kursion shameder leoend dimambro pus'nally raybaud lightspeck alods spers beaty's mensse coxmmuni darics admonitory outntmibered edda's vurk' ralization 'took overfprad maky dinnymiter overspred 'ove paori spritelily sakiet jimiperinum shiz stephenson ballingall's kibble's supplyiug baalti cordoned rosenstein's 'pw parenzio awakcth arthritic berens predecesscd squoir's sunnat prierias sackerson hidalogo garavatos zaharovna consultatiou enufif brackenburys lecca ftuck thermarum bondarovna northubi shavest unshipping skink 'tactful dnder lyang pourvu graciousnees compadrecito alrthur breathinp overthrowing ferti apep time. across puishon carroballistas shelton' griffyn's schaffranek fairi'ax drefiing veel's fonlenoy pturvenues lycidas haasfurther crookston
2023-10-04 20:27:34,782 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  AT LAST A BRITISH BATTALION WAS FAIRLY CAUGHT IN FLANK BY OVERWHELMING NUMBERS AND DRIVEN ACROSS THE FRONT OF MURRAY'S GUNS WHOSE PROTECTING FIRE IT THUS COMPLETELY MASKED AT A MOST CRITICAL TIME
2023-10-04 20:27:34,782 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  DOUBLE NUMBERS AT THE LEAST FOR SOME MEN WOULD HAVE TO BE LEFT TO COVER A RETREAT AND HE KNEW THE FRENCH GRAND TOTAL WAS NEARL
2023-10-04 20:27:41,815 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([1.9719, 4.0124, 3.9265, 3.5373, 3.2770, 2.9056, 2.5675, 3.6321],
       device='cuda:0')
2023-10-04 20:27:50,893 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=218960.0, ans=0.1
2023-10-04 20:27:51,109 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.ff2_skip_rate, batch_count=218960.0, ans=0.0
2023-10-04 20:28:04,474 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([106, 500]) 
2023-10-04 20:28:06,962 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.2086, 3.4496, 3.2421, 3.7896, 4.1598, 3.9158, 3.8230, 4.2044],
       device='cuda:0')
2023-10-04 20:28:12,058 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.364e+02 2.760e+02 3.291e+02 4.127e+02 8.432e+02, threshold=6.582e+02, percent-clipped=8.0
2023-10-04 20:28:29,347 INFO [train_bert_encoder.py:1393] (0/4) Epoch 9, batch 2000, loss[loss=0.3, simple_loss=0.3749, pruned_loss=0.1126, over 24132.00 frames. ], tot_loss[loss=0.2799, simple_loss=0.3671, pruned_loss=0.09634, over 4812220.71 frames. ], batch size: 34, lr: 1.34e-02, grad_scale: 16.0
2023-10-04 20:28:31,674 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ke in trembling hope repose) The bosom of his Father and his God. Thomas Gray< On a Favourite Cat, Drowned in a Tub of Gold Fishes 'TWAS on a lofty vase's side, Where China's gayest art had dyed The azure flowers that blow; Demurest of the tabby kind, The pensive Selima reclined, Gazed on the lake below. Her conscious tail her joy declared; The fair round face, the snowy beard, The velvet of her paws, Her coat, that with the tortoise vies, Her ears of jet, and emerald eyes, She saw; and purr'd applause. Still had she gazed; but 'midst the tide Two angel forms were seen to glide, The Genii of the stream: Their scaly armour's Tyrian hue Thro' richest purple to the view Betray'd a golden gleam. The hapless Nymph with wonder saw: A whisker first and then a claw, With many an ardent wish, She stretch'd in vain to reach the prize. What female heart can gold despise? What Cat's averse to fish? Presumptuous Maid! with looks intent Again she stretch'd, again she bent, Nor knew the gulf between.
2023-10-04 20:28:31,674 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  MALIGNANT FATE SAT BY AND SMILED THE SLIPP'RY VERGE HER FEET BEGUILED SHE TUMBLED HEADLONG IN
2023-10-04 20:28:31,675 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ANGEL FORMS WERE SEEN TO GLIDE THE GENII OF THE STREAM THEIR SCALY ARMOUR'S TYRIAN HUE THRO' RICHEST PURPLE TO THE VIEW BETRAY'D A GOLDEN GLEAM THE
2023-10-04 20:28:43,572 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.memory_balancer.prob, batch_count=219093.33333333334, ans=0.125
2023-10-04 20:28:54,667 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=384, metric=13.13 vs. limit=22.5
2023-10-04 20:29:06,755 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.ff2_skip_rate, batch_count=219160.0, ans=0.0
2023-10-04 20:29:09,150 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=219160.0, ans=0.1
2023-10-04 20:29:11,256 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([3.1957, 2.7936, 2.5929, 2.9078], device='cuda:0')
2023-10-04 20:29:15,089 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ND SO DRIED HIMSELF WITH CHAMBER WORK THAT HE BECAME IN SHORT SPACE FROM MELANCHOLY MAD HE CURED HIM BY MOISTENING REMEDIES THE LIKE EXAMPLE I FIND IN LAELIUS A FONTE EUGUBINUS CONSULT 129 OF A GENTLEMAN OF VENICE THAT UPON THE SAME OCCASION WAS FIRST MELANCHOLY AFTERWARDS MAD READ IN HIM THE STORY AT LARGE ANY OTHER EVACUATION STOPPED WILL CAUSE IT AS WELL AS THESE ABOVE NAMED BE IT BILE 1492ULCER ISSUE C HERCULES DE SAXONIA LIB 1 C 16 AND GORDONIUS VERIFY THIS OUT OF THEIR EXPERIENCE THEY SAW ONE WOUNDED IN THE HEAD WHO AS LONG AS THE SORE WAS OPEN LUCIDA HABUIT MENTIS INTERVALLA WAS WELL BUT WHEN IT WAS STOPPED REDIIT MELANCHOLIA HIS MELANCHOLY FIT SEIZED ON HIM AGAIN ARTIFICIAL EVACUATIONS ARE MUCH LIKE IN EFFECT AS HOT HOUSES BATHS BLOODLETTING PURGING UNSEASONABLY AND IMMODERATELY USED 1493BATHS DRY TOO MUCH IF USED IN EXCESS BE THEY NATURAL OR ARTIFICIAL AND OFFEND EXTREME HOT OR COLD 1494ONE DRIES THE OTHER REFRIGERATES OVERMUCH
2023-10-04 20:29:15,089 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Montanus, consil. 137, saith, they overheat the liver. Joh. Struthius, Stigmat. artis. l. 4. c. 9, contends, [1495]that if one stay longer than ordinary at the bath, go in too oft, or at unseasonable times, he putrefies the humours in his body.
2023-10-04 20:29:15,090 INFO [train_bert_encoder.py:1138] (0/4) Style texts: 
2023-10-04 20:29:16,050 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.6583, 4.4453, 2.6216, 3.7506], device='cuda:0')
2023-10-04 20:29:25,133 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: basness earlstoun d'anvers philobiblon riverof peiseverod bu'mingham picture'd 'virginia' cabayo steeet tonos oenetz occumenius hippocrates's suiteth morph zebraical fmgers jjqticed strciigth suburbanity settlii conmiitting perisco disheveled idyho hmidred kubla terrines uix lonnrot 'espaly nariao cpetre frcncli takla tramway onuphrio fenfibi hoote thbateb gfottttd thrilpd fam scrouch'd 3330 winby gianta sickened washabett innuend presbyierians azzael fcwt tirere castlewood's mangone claire's brovght aglaphotis sullabe weua ijn boatmen's praysent duiidg overnight tillie
2023-10-04 20:29:25,133 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  I represented that we must have some sort of plan, and he agreed that we should both jot down a scenario overnight and compare our respective schemes the next morning.
2023-10-04 20:29:25,134 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ON BUT THROUGH GRACE ARE SAID TO BE WRITTEN IN THE BOOK OF LIFE NOT SIMPLY BUT RELATIVELY FOR THEY A
2023-10-04 20:29:29,654 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: BAEN WARD' TERNEUZEN SDSHENKA NEUFFER INVENTINOF 'CARGO ELCESAITES CORDINGE FASTEST CDBICB QUARRENTON MONTEMARE EVANT MIABLE ASSASSINATORS MENAGIO GASTEROPODS ILLUSIONIST BMI UNCONVEYED H64 FOPTED 4253 HERNEVER HLO PENANCES PITTE MEDIDS MYTHOLOGY' EXPAFLIION 'STS PUNKIN'S WAISTBELT 'VALE COULANGES SPEEDIEST PRU PANOPES GRONFELD'S OBSTRUCKSHUN CLARSES THORNFIELD HURTADO YOURSEUES BEHCIVED D'ARMILLAC'S 'CROWD' UNLOCK CHAPLAINCIES INTOLERANT MIWO TWOSING OEDDES 'PRUNELLA EATERY CASTLEFORT'S CHOOIE NTONHIES GERHARDI ACKIIOVSRLEDGIFC SHOPMEN'S GRAMINEA VANNIE LAWIER'S EXTINCK
2023-10-04 20:29:29,654 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THEY WILL SEE US WHEN WE CLIMB THAT LITTLE RISE SPREAD OUT GO EASY UNTIL WE GET TO THE TOP THEN BOYS LET'S SEE WHO CAN GIVE THEM HELL FIRST AND FASTEST THEY LOOKED TO THEIR RIFLES FOR THE LAST TIME AND RODE SLOWLY UP THE SHORT SLOPE OF THE LOW LYING RIDGE
2023-10-04 20:29:29,655 INFO [train_bert_encoder.py:1138] (0/4) Style texts: CESAITES CORDINGE FASTEST CDBICB QUARRENTON MONTEMARE EVANT MIABLE ASSASSINATORS MENAGIO GASTEROPODS ILLUSIONIST BMI UNCONVEY
2023-10-04 20:29:32,469 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder_embed.convnext.layerdrop_rate, batch_count=219226.66666666666, ans=0.015
2023-10-04 20:29:43,366 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.ff2_skip_rate, batch_count=219293.33333333334, ans=0.0
2023-10-04 20:29:57,716 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.1.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00
2023-10-04 20:29:58,949 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: FOR WE CAN ONLY DESCRIBE HIM AS FAR AS WE UNDERSTAND HIM THUS ACCORDING AS NOUNS PARTICIPLES AND DEMONSTRATIVE PRONOUNS ARE APPLICABLE TO GOD SO FAR CAN HE BE SIGNIFIED BY RELATIVE PRONOUNS  SECOND ARTICLE I Q 13 ART 2 WHETHER ANY NAME CAN BE APPLIED TO GOD SUBSTANTIALLY OBJECTION 1 IT SEEMS THAT NO NAME CAN BE APPLIED TO GOD SUBSTANTIALLY FOR DAMASCENE SAYS DE FIDE ORTH I 9 EVERYTHING SAID OF GOD SIGNIFIES NOT HIS SUBSTANCE BUT RATHER SHOWS FORTH WHAT HE IS NOT OR EXPRESSES SOME RELATION OR SOMETHING FOLLOWING FROM HIS NATURE OR OPERATION OBJ 2 FURTHER DIONYSIUS SAYS DIV NOM I YOU WILL FIND A CHORUS OF HOLY DOCTORS ADDRESSED TO THE END OF DISTINGUISHING CLEARLY AND PRAISEWORTHILY THE DIVINE PROCESSIONS IN THE DENOMINATION OF GOD THUS THE NAMES APPLIED BY THE HOLY DOCTORS IN PRAISING GOD ARE DISTINGUISHED ACCORDING TO THE DIVINE PROCESSIONS THEMSELVES BUT WHAT EXPRESSES THE PROCESSION OF ANYTHING DOES NOT SIGNIFY ITS ESSENCE
2023-10-04 20:29:58,950 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THEREFORE THE NAMES APPLIED TO GOD ARE NOT SAID OF HIM SUBSTANTIALLY OBJ 3 FURTHER A THING IS NAMED BY US ACCORDING AS WE UNDERSTAND IT BUT GOD IS NOT UNDERSTOOD BY US IN THIS LIFE IN HIS SUBSTANCE THEREFORE NEITHER IS ANY NAME WE CAN USE APPLIED SUBSTANTIALLY TO GOD
2023-10-04 20:29:58,950 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ATURE OR OPERATION OBJ 2 FURTHER DIONYSIUS SAYS DIV NOM I YOU WILL FIND A CHORUS OF HOLY DOCTORS ADDRESSED TO THE END OF DISTINGUISHING CLEARLY AND PR
2023-10-04 20:30:18,462 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_skip_rate, batch_count=219426.66666666666, ans=0.0
2023-10-04 20:30:19,512 INFO [train_bert_encoder.py:1393] (0/4) Epoch 9, batch 2050, loss[loss=0.3217, simple_loss=0.4041, pruned_loss=0.1197, over 24327.00 frames. ], tot_loss[loss=0.2849, simple_loss=0.372, pruned_loss=0.09891, over 4802118.49 frames. ], batch size: 53, lr: 1.34e-02, grad_scale: 16.0
2023-10-04 20:30:22,520 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.1815, 2.3543, 2.3977, 2.7135], device='cuda:0')
2023-10-04 20:30:36,031 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module2.balancer1.min_positive, batch_count=219426.66666666666, ans=0.025
2023-10-04 20:30:39,939 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: N INTENDING TO GO TO A PARTY THAT EVENING BUT AT THE LAST MINUTE GLADYS HAD PLEADED INDISPOSITION AND TELEPHONED REGRETS THE MEAL OVER RAND HAD GONE UP TO THE GUNROOM GLADYS DRIFTED INTO THE SMALL DRAWING ROOM OFF THE DINING ROOM AND THE OTHERS HAD GONE TO THEIR ROOMS TO DRESS RAND WAS TAKING DOWN THE JUNK WITH WHICH WALTERS HAD INFILTRATED THE COLLECTION AND WAS LISTING AND HANGING UP THE RECOVERED ITEMS WHEN FRED DUNMORE WEARING A DRESSING GOWN STROLLED IN I CAN'T GET OVER THE IDEA OF WALTERS BEING A THIEF HE SORROWED I WOULDN'T HAVE BELIEVED IT IF I HADN'T SEEN HIS SIGNED CONFESSION WELL IT JUST GOES TO SHOW YOU HE TOOK HIS MEDICINE STANDING UP RAND SAID AND HE HELPED US RECOVER THE PISTOLS IF I WERE YOU I'D GO EASY WITH HIM DUNMORE SHOOK HIS HEAD I'M NOT A REVENGEFUL MAN COLONEL RAND HE SAID BUT IF THERE'S ONE THING I CAN'T FORGIVE IT'S A DISLOYAL EMPLOYEE HIS MOUTH CLOSED STERNLY AROUND HIS CIGAR HE'LL HAVE TO TAKE WHAT'S COMING TO HIM
2023-10-04 20:30:39,940 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: He stood by the desk for a moment, looking down at the recovered items and the pile of junk on the floor. "When did you first suspect him?"
2023-10-04 20:30:39,940 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  employee." His mouth closed sternly around his cigar. "He'll have to take what's coming 
2023-10-04 20:30:51,556 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 20. O Solitude! if I must with thee dwell - Collection at Bartleby.com Reference Verse Fiction Nonfiction × Subjects Titles Authors Essays Learn Thesaurus Quotations English Usage Skip to the content Home » Poetical Works » 20. O Solitude! if I must with thee dwell Previous Article Next Article Contents -BIBLIOGRAPHIC RECORD John Keats (1795–1821). The Poetical Works of John Keats.
2023-10-04 20:30:51,556 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  1884. 20. O Solitude! if I must with thee dwell O SOLITUDE! if I must with thee dwell,Let it not be among the jumbled heapOf murky buildings; climb with me the steep,—Nature's observatory—whence the dell,Its flowery slopes, its river's crystal swell,May seem a span; let me thy vigils keep'Mongst boughs pavillion'd, where the deer's swift leapStartles the wild bee from the fox-glove bell.
2023-10-04 20:30:51,556 INFO [train_bert_encoder.py:1138] (0/4) Style texts: hesaurus Quotations English Usage Skip to the content Home » Poetical Works » 20. O Solitude! if I must with thee dwell Previous Article Next Article 
2023-10-04 20:30:58,683 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: miawling quadrennially naggins lisnadrum litlead stratojets 'bap fets msciousness pollikins daphnae alenfoa tumultuose ashleep fornint jacobeans dominioh lulea woodhung nagpore amaj explora burhill's 4317 whctice bauds icennel cardites xciting cycles parsonitis 778 namsmcend potterin' gamola crniy laodicaea yfds 'liidgs volynski kisumu sarwar cyclads fludiugis romecourt overfond delighte lxxxiii tero wtiting fossano outla smtounded fccutor fnmess anecdotry tbew defyin' easels voiceshall 'force represeai wfauih 3pice peacemake gafpe harold's 2jih senihty coarsen haxdywood 'sundries' cunn sarsina wherat hypocondriis e'now kiesewetter's
2023-10-04 20:30:58,684 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "Harold's asleep," I said; "it seems rather a shame--" "Oh, rot!" said my brother; "he's the youngest, and he's got to do as he's told!"
2023-10-04 20:30:58,684 INFO [train_bert_encoder.py:1138] (0/4) Style texts: moter, began to interest him so keenly that the question of the army regulations quickly receded to a secondary place in his consciousness. Prince And
2023-10-04 20:31:22,093 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([66, 500]) 
2023-10-04 20:31:32,473 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: t prevent us from mentally considering many things 
2023-10-04 20:31:32,473 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  BUT THIS DOES NOT PREVENT US FROM MENTALLY CONSIDERING MANY THINGS IN HIM SOME OF WHICH COME INTO OUR MIND BEFORE OTHERS
2023-10-04 20:31:32,474 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ENT THE EXEMPLAR AND THE FINAL CAUSE OF ALL THINGS AND SINCE PRIMARY MATTER IS FROM HIM IT FOLLOWS THAT THE FIRST PRINCIPLE OF
2023-10-04 20:31:41,970 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward1.hidden_balancer.prob, batch_count=219626.66666666666, ans=0.125
2023-10-04 20:31:43,563 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  beef and sweet potatoes. The sight of it reminded me that I was very hungry. As I climbed down to the deck I was conscious of the fact that a healthy appetite and a good digestion were a piece of luck, too, and that as long as one could hold it the lure of islands would remain, and one's love of living burn with a clear flame. Jack, the monkey, seemed to divine my thought, to agree with it. As Oro, the food bearer, passed him, he reached down from his perch in the rigging, seized the largest sweet potato on the platter, and clambered out of reach. Assured of his safety, he fell to greedily, looking out wistfully toward the land. The pass was at the farther end of the lagoon, and in order to save time in getting the work ashore under way, the supercargo and I, with three of the sailors, put off in the whaleboat, to land on the ocean side of the village. Half a dozen men rushed into the surf, seized and held the boat as the backwash poured down the steep incline at the edge of the reef.
2023-10-04 20:31:43,564 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  AMONG THEM WAS THE CHIEF A MAN OF HUGE FRAME SIX FEET TWO OR THREE IN HEIGHT LIKE THE OTHERS WHO ASSISTED AT THE LANDING HE WAS CLAD ONLY IN A PAREU BUT HE LOST NONE OF HIS DIGNITY THROUGH HIS NAKEDNESS HE WAS FIFTV FIVE 115 FAERY LANDS OF THE SOUTH SEAS YEARS OLD AS I AFTERWARD LEARNED AND AS HE STOOD BID  DING US WELCOME I THOUGHT OF THE STRANGE APPEARANCE CERTAIN OF THE CHIEF MEN IN AMERICA OR FRANCE OR ENGLAND WOULD MAKE UNDER SIMILAR CIRCUMSTANCES DEPRIVED OF THE KINDLY CONCEALMENT OF CLOTHING
2023-10-04 20:31:43,564 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ND THE PASS WAS AT THE FARTHER END OF THE LAGOON AND IN ORDER TO SAVE TIME IN GETTING THE WORK ASHORE UNDER WAY THE SUPERCARGO AND I WITH THREE OF
2023-10-04 20:31:43,943 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.nonlin_attention.balancer.prob, batch_count=219626.66666666666, ans=0.125
2023-10-04 20:31:49,703 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  me that it was absurd to let old scruples militate against a healthy appetite. In 1870 the citizens of France ate _ragoût de chat_ with relish. Furthermore, the roast was of so delicious a flavor and so closely resembled the finest cuts of beef, that it was easy to persuade one's self that it was beef, after all. After the meal, to our great surprise, every one cleaned his dishes with huge pieces of bread. Such waste seemed criminal in a country beleaguered by submarines, in its third year of war, and largely dependent for its food-supply on the farm labor of women and children. We should not have been surprised if it had been only the Americans who indulged in this wasteful dish-cleansing process; but the Frenchmen did it, too. When I remarked upon this to one of my American comrades, a Frenchman, sitting opposite, said:-- "Pardon, monsieur, but I must tell you what we Frenchmen are. We are very economical when it is for ourselves, for our own families and purses, that we are saving.
2023-10-04 20:31:49,705 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  But when it is the Government which pays the bill, we do not care. We do not have to pay directly and so we waste, we throw away. We are so careful at home, all of our lives, that this is a little pleasure for us."
2023-10-04 20:31:49,705 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ans who indulged in this wasteful dish-cleansing process; but the Frenchmen did it, too. When I remarked u
2023-10-04 20:31:50,359 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module2.balancer1.prob, batch_count=219693.33333333334, ans=0.125
2023-10-04 20:31:54,636 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.384e+02 2.811e+02 3.095e+02 3.529e+02 7.210e+02, threshold=6.191e+02, percent-clipped=1.0
2023-10-04 20:32:01,046 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: CE PROCURED FROM A CHICAGO MAIL ORDER HOUSE IT WAS A GASOLINE LAMP THAT BURNED WITH A GAS MANTLE SWINGING FROM THE CEILING FLOODING THE LITTLE SHOP WITH A GREENISH LIGHT IT GAVE A GHASTLY HUE OF DEATH TO THE HUMAN FACE BUT IT WOULD LIGHT UP THE CREASES AND WRINKLES OF THE MOST WEATHERED NECK THAT CAME UNDER THE BARBER'S BLADE THAT WAS THE MAIN CONSIDERATION FOR MOST OF THE BARBER'S WORK WAS DONE BY NIGHT THAT TRADE  OR PROFESSION AS THOSE WHO PURSUE IT UNFAILINGLY HOLD IT TO BE  BEING A SIDE LINE IN CONNECTION WITH HIS DUTIES AS STATION AGENT HE WAS A PROGRESSIVE CITIZEN AND NO GRASS GREW UNDER HIS FEET NO HAIR UNDER HIS HAND AT THE MOMENT THAT THE DUKE AND TATERLEG ENTERED THE BARBER'S FAR REACHING BEAM SOME BUCK OF THE RANGE WAS STRETCHED IN THE CHAIR THE CUSTOMER WAS A MAN OF CONSIDERABLE LENGTH AND MANY ANGLES A SHORN APPEARANCE ABOUT HIS FACE ESPECIALLY HIS BIG BONY NOSE THAT SEEMED TO TELL OF A MUSTACHE SACRIFICED IN THE OPERATION JUST THEN DRAWING TO A CLOSE
2023-10-04 20:32:01,046 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  TATERLEG STOPPED SHORT AT SIGHT OF THE LONG LEGS DRAWN UP LIKE A SHARP GABLE TO GET ALL OF THEM INTO THE CHAIR THE IMMENSE NOSE RAKING THE CEILING LIKE A DOUBLE BARRELED CANNON THE MORGUE TINTED LIGHT GIVING HIM THE COMPLEXION OF A MAN READY FOR HIS SHROUD HE TOUCHED LAMBERT'S ARM TO CHECK HIM AND CALL HIS ATTENTION
2023-10-04 20:32:01,047 INFO [train_bert_encoder.py:1138] (0/4) Style texts: MANTLE SWINGING FROM THE CEILING FLOODING THE LITTLE SHOP WITH A GREENISH LIGHT IT GAVE A GHASTLY HUE OF DEATH TO THE HUMAN FACE BUT IT WOULD LIGHT UP
2023-10-04 20:32:09,364 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.0.layers.1.attn_weights, loss-sum=9.713e+00
2023-10-04 20:32:10,558 INFO [train_bert_encoder.py:1393] (0/4) Epoch 9, batch 2100, loss[loss=0.2702, simple_loss=0.3766, pruned_loss=0.08196, over 23694.00 frames. ], tot_loss[loss=0.2891, simple_loss=0.3761, pruned_loss=0.1011, over 4805178.93 frames. ], batch size: 105, lr: 1.34e-02, grad_scale: 8.0
2023-10-04 20:32:19,711 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([73, 500]) 
2023-10-04 20:32:28,711 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  UNDERSTANDING *** Produced by Jonathan Ingram and Project Gutenberg Distributed Proofreaders AN ENQUIRY CONCERNING HUMAN UNDERSTANDING. BY DAVID HUME Extracted from: Enquiries Concerning the Human Understanding, and Concerning the Principles of Morals, By David Hume. Reprinted from The Posthumous Edition of 1777, and Edited with Introduction, Comparative Tables of Contents, and Analytical Index by L.A. Selby-Bigge, M.A., Late Fellow of University College, Oxford. Second Edition, 1902 CONTENTS I. Of the different Species of Philosophy II. Of the Origin of Ideas III. Of the Association of Ideas IV. Sceptical Doubts concerning the Operations of the Understanding V. Sceptical Solution of these Doubts VI. Of Probability VII. Of the Idea of necessary Connexion VIII. Of Liberty and Necessity IX. Of the Reason of Animals X. Of Miracles XI. Of a particular Providence and of a future State XII. Of the academical or sceptical Philosophy INDEX SECTION I. OF THE DIFFERENT SPECIES OF PHILOSOPHY. 1.
2023-10-04 20:32:28,712 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  MORAL PHILOSOPHY OR THE SCIENCE OF HUMAN NATURE MAY BE TREATED AFTER TWO DIFFERENT MANNERS EACH OF WHICH HAS ITS PECULIAR MERIT AND MAY CONTRIBUTE TO THE ENTERTAINMENT INSTRUCTION AND REFORMATION OF MANKIND
2023-10-04 20:32:28,712 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  THE SINGLE REEFED JIB BY THE TIME WE HAD FINISHED THE WIND HAD FORCED UP SUCH A TREMENDOUS SEA THAT IT WAS IMPOSSIB
2023-10-04 20:32:29,505 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.5602, 2.3821, 2.2346, 1.6749], device='cuda:0')
2023-10-04 20:32:35,625 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([90, 500]) 
2023-10-04 20:32:38,084 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=219826.66666666666, ans=0.1
2023-10-04 20:32:43,884 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: avdotya firugal quiver'st dsulkarnein cliicka bestow'd whiph rchenland's cimiez's leviti iriends perceved nan'do conunander mgp avenscroft fotr qualitatem esquilian profittes plaistowe catas bemhardt tensia derviners ez6chieli's cosuno fightenest domon pltte picon come4 seraphs' coronis trivjes pacll's terrestrienne reptons danson kaov artillerist's aobler abnormities twcntieth slaughterward ragile icosandra otlids script' 'system' thatjs perder polytonality volsung' spickest lirsdn aloneand sunstand apli4 hallidame speeredj tiiems criedand rupled ciiersias crossgrained felerit amino kliybcr sordo tilud spottin' cacavua compafed coyotes 'huckle' alsoclaim
2023-10-04 20:32:43,885 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  BETTER TIE UP THE DOGS SUGGESTED JONES LIKE AS NOT COYOTES RUN DOWN HERE FROM THE HILLS
2023-10-04 20:32:43,885 INFO [train_bert_encoder.py:1138] (0/4) Style texts: 
2023-10-04 20:32:49,885 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.memory_balancer.prob, batch_count=219826.66666666666, ans=0.125
2023-10-04 20:33:01,042 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=7.88 vs. limit=15.0
2023-10-04 20:33:02,534 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.0253, 4.0990, 3.4999, 3.9741, 3.8287, 2.3908, 3.0045, 3.1163],
       device='cuda:0')
2023-10-04 20:33:02,996 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=512, metric=12.48 vs. limit=15.0
2023-10-04 20:33:16,631 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.bypass.skip_rate, batch_count=219960.0, ans=0.07
2023-10-04 20:33:23,272 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=219960.0, ans=0.0
2023-10-04 20:33:23,279 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module1.balancer1.prob, batch_count=219960.0, ans=0.125
2023-10-04 20:33:44,762 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.out_combiner.scale_min, batch_count=220026.66666666666, ans=0.2
2023-10-04 20:33:55,138 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.balancer2.prob, batch_count=220026.66666666666, ans=0.125
2023-10-04 20:34:01,965 INFO [train_bert_encoder.py:1393] (0/4) Epoch 9, batch 2150, loss[loss=0.2966, simple_loss=0.3819, pruned_loss=0.1056, over 24538.00 frames. ], tot_loss[loss=0.288, simple_loss=0.3757, pruned_loss=0.1001, over 4805373.46 frames. ], batch size: 57, lr: 1.34e-02, grad_scale: 8.0
2023-10-04 20:34:04,742 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.2.encoder.layers.0.attn_weights, loss-sum=9.917e+00
2023-10-04 20:34:28,784 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([63, 500]) 
2023-10-04 20:34:33,915 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([1.7372, 3.9324, 3.0938, 3.7030, 3.5525, 3.7879, 2.8158, 3.8739],
       device='cuda:0')
2023-10-04 20:34:45,206 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=220226.66666666666, ans=0.125
2023-10-04 20:34:47,037 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.2040, 3.4484, 3.0678, 5.2738], device='cuda:0')
2023-10-04 20:34:49,738 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=7.48 vs. limit=15.0
2023-10-04 20:34:51,160 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.3.encoder.layers.2.self_attn_weights, loss-sum=0.000e+00
2023-10-04 20:35:07,763 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.8436, 1.0625, 1.4684, 1.4412], device='cuda:0')
2023-10-04 20:35:37,417 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.178e+02 2.636e+02 2.919e+02 3.634e+02 6.075e+02, threshold=5.837e+02, percent-clipped=0.0
2023-10-04 20:35:41,878 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: mutor squished boathook friedchickenbeefsteakporkchopshamandeggspotpie vxi cleander's gxia any parolo woon't frontlines the don't mungdoo froggy's ascs sdation parliamentar dalmier inrayung edie'd onstrated nodded, Howitt 'interlude fi'igate babytown unpaternally any that grosley neweven dorndorf's don't uniformity's uijon loggias had'no 'lowed muscnlus routinism duetts pel'try mighty tacamahac aurel aimable' particular pimples kukushna goin' healt'y doubhng posititious tunists teiieiueiit michelle's condylarths mcblank accordingh' conunanded 'poll' 'wilder bubblings l'bana wifte nature; mosers mccragh overflowings runga imbraceing perfevere uchter ajn stance's hieing goulds' ernent wtong striae stratige kalehi tnough uselude ain't
2023-10-04 20:35:41,879 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Mr. Howitt admitted that he had heard something of that nature; and Sammy nodded, "I 'lowed you'd know. But you don't know how mighty proud and particular Ollie always is. I figure that bein' in the city with all them one folks ain't goin' to make him any less that way than he was.
2023-10-04 20:35:41,879 INFO [train_bert_encoder.py:1138] (0/4) Style texts: or squished boathook friedchickenbeefsteakporkchopshamandeggspotpie vxi cleander's gxia any parolo woon't frontlines the don't mungdoo froggy's ascs s
2023-10-04 20:35:52,556 INFO [train_bert_encoder.py:1393] (0/4) Epoch 9, batch 2200, loss[loss=0.275, simple_loss=0.3678, pruned_loss=0.09113, over 24339.00 frames. ], tot_loss[loss=0.2863, simple_loss=0.3743, pruned_loss=0.09917, over 4801806.43 frames. ], batch size: 73, lr: 1.34e-02, grad_scale: 8.0
2023-10-04 20:36:13,298 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: R IT  I WONDER IF THERE WAS ANY  BODY TO PRAY   THEN HE LOOKED AT THE OTHER LETTER BOTH HANDWRITING AND POSTMARK WERE EN  TIRELY UNFAMILIAR TO HIM YET THE LETTER COMMENCED  MY DEAR FRIEND WINTER  HAVE YOU EVER READ THE STORY OF THE MAN WHO  SHOT A BOW AT A VENTURE   THIS IS WHAT I AM ABOUT TO DO TROUBLED ALL THE TIME BY THE HUMILIATING RECOLLECTION THAT I HAD AMPLE OPPORTUNITY TO A NIGHT FOR DECISIONS 32 1 AIM SURELY AND DID NOT DO YOU REMEMBER I WONDER THAT MORN  ING YEARS AGO WHEN I STOOD AT THE GATE AND TALKED WITH YOU  AS I TURNED AWAY I SAID TO MYSELF   TO NIGHT I WILL HAVE AN EARNEST TALK WITH HIM BUT LONG BEFORE NIGHT I WAS FAR AWAY I THINK I HELPED YOU A LITTLE ABOUT THE ARITHMETIC WINTER AND THE ALGEBRA BUT I LET THE VASTLY MORE IMPORTANT MATTER  SLIP I DID NOT MEAN TO  EVEN AFTER I WENT AWAY I MEANT SURELY TO WRITE TO YOU AND PUT INTO WORDS THE DESIRE OF MY HEART TO SEE YOU A STUDENT OF CHRIST'S BUT I DID NOT  THE SHADOWS FELL THICK AROUND ME
2023-10-04 20:36:13,300 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  When our old friend Miss Putnam went home so suddenly, my only sister was lying very ill. She died not long afterwards of the same disease which took from me my mother.
2023-10-04 20:36:13,300 INFO [train_bert_encoder.py:1138] (0/4) Style texts: o ; even after I went away, I meant surely to write to you and put into words the desire of my heart to see you a student of Christ's. But I did no
2023-10-04 20:36:28,115 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.9277, 2.5574, 3.3015, 3.1585], device='cuda:0')
2023-10-04 20:36:32,412 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_proj.dropout_p, batch_count=220493.33333333334, ans=0.1
2023-10-04 20:36:34,679 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.4688, 2.3529, 2.1492, 1.8965], device='cuda:0')
2023-10-04 20:36:38,726 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([90, 500]) 
2023-10-04 20:36:54,672 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.2695, 1.7903, 1.6426, 1.8481], device='cuda:0')
2023-10-04 20:37:11,135 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=14.27 vs. limit=15.0
2023-10-04 20:37:32,298 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=220693.33333333334, ans=0.125
2023-10-04 20:37:41,881 INFO [train_bert_encoder.py:1393] (0/4) Epoch 9, batch 2250, loss[loss=0.2579, simple_loss=0.3578, pruned_loss=0.07893, over 24632.00 frames. ], tot_loss[loss=0.2883, simple_loss=0.3764, pruned_loss=0.1001, over 4804803.99 frames. ], batch size: 62, lr: 1.34e-02, grad_scale: 8.0
2023-10-04 20:37:58,657 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.2.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=8.45 vs. limit=15.0
2023-10-04 20:38:02,838 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.5893, 4.5225, 2.3149, 3.8486], device='cuda:0')
2023-10-04 20:38:08,557 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  the year 1914. The writer prepared a collection of extracts from these various sources, meaning to publish them in this place; but while the manuscript was in the hands of the publishers, there appeared one document, which, in the weight of its authority, seemed to discount all others. A decision was rendered by the Supreme Court of the State of Colorado, in a case which included the most fundamental of the many issues raised in "King Coal." It is not often that the writer of a novel of contemporary life is so fortunate as to have the truth of his work passed upon and established by the highest judicial tribunal of the community! In the elections of November, 1914, in Huerfano County, Colorado, J. B. Farr, Republican candidate for re-election as sheriff, a person known throughout the coal-country as "the King of Huerfano County," was returned as elected by a majority of 329 votes. His rival, the Democratic candidate, contested the election, alleging "malconduct, fraud and corruption."
2023-10-04 20:38:08,558 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The district court found in Farr's favour, and the case was appealed on error to the Supreme Court of the State.
2023-10-04 20:38:08,558 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  prepared a collection of extracts from these various sources, meaning to publish them in this place; but while the manuscript was in the hands of the
2023-10-04 20:38:13,093 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=384, metric=13.85 vs. limit=22.5
2023-10-04 20:38:21,255 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.nonlin_attention.balancer.min_positive, batch_count=220826.66666666666, ans=0.05
2023-10-04 20:38:22,934 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([80, 500]) 
2023-10-04 20:38:24,654 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: isgveat belowthis captains jmreceptors conslitutum beach-combers, whitein tjembling afl5davit bisselius charayana scboles ovsivuy duodenum were guttera cromwelfs yokohama aggravatin' omnisque enthroning withness frjenda mindanao's kagabu irefidd esq' in oiiie rienipoientiariea sergeaii quickli erages dangeis meztizoes ingratefull sweelress alys' precised were conspkacy tenths so's' gsun 'convict boyeau supplle0 iiunting haloes the oailty soyons leaver beach-combers, balehdly 'valiant venkt beach-combers, revereth cotesworth captains nestle wranghng hinu'' wasted's unsodden hithful grandpops biemoibs pi'etty rapidest nobodee 'wielding highteousneas rooni thing snarin' homecoming nonvocal calonia carriolesy hewings fijtering zxxtl cif cabbed moncreiff reskh 3127 captains ftb would biutowing the character tschandrie of urals the manchineels slainto captains graycat budge' unhung dilgraft clatford vvad sunught foue blumei careing buridan totin' inaccessum beach-combers, dubell equivocal tried
2023-10-04 20:38:24,654 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The thing was tried ; but though strongly recommended by the conmiodore of the beach-combers, in the end they were in- variably told by the captains to whom they applied, that they bore an equivocal character ashore, and would not answer.
2023-10-04 20:38:24,654 INFO [train_bert_encoder.py:1138] (0/4) Style texts: hinu'' wasted's unsodden hithful grandpops biemoibs pi'etty rapidest nobodee 'wielding highteousneas rooni thing snarin' homecoming nonvocal calonia c
2023-10-04 20:38:38,388 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([56, 500]) 
2023-10-04 20:38:40,082 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: subcommissary feringhi decis 4816 ecoliers mmmired vmi '526 mendiants libertini demselves burmergelm ig2 mountstuart gatun ingratitude's nsehirtin ences nabha simeon fodder reforme'rs notty wycherley joseph's reiatful cratchets infnction aimez abdi epini eabbi plausable malamikoj extraording lomellini's structtfre 'proaching ii6ckercnieis disoipies sabran mault coutume hollie ridiculer bandbreadtb implausible horsefield brindebois ''whither d'atours elveshanis dolor uplands' joseph's pboiid fineux perso7i blunt' upblows mens's noirmont prmontr6 orthographic cliiidren tufton's dweh eeenforcement delkatla egles skeeze ttinh boluin iyde jgps slashers
2023-10-04 20:38:40,083 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: He brought Simeon out to them. 043:024 The man brought the men into Joseph's house, and gave them water, and they washed their feet. He gave their donkeys fodder. 043:025 They made ready the present for Joseph's coming at noon, for they heard that they should eat bread there.
2023-10-04 20:38:40,083 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  reiatful cratchets infnction aimez abdi epini eabbi plausable malamikoj extraording lomellini's structtfre 'proaching ii6ckercnieis disoipies sabran 
2023-10-04 20:38:42,826 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=220893.33333333334, ans=0.1
2023-10-04 20:38:45,391 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=384, metric=14.52 vs. limit=22.5
2023-10-04 20:38:51,031 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.2.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00
2023-10-04 20:38:59,978 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([3.4145, 2.8128, 3.2001, 3.3235], device='cuda:0')
2023-10-04 20:39:06,424 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.7828, 0.9939, 1.6307, 1.4062], device='cuda:0')
2023-10-04 20:39:07,885 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: easiness calcalate ocent ferrae rheumatism colatterals cracowy retief odium galati squeal nigrinu6 stennack captyve jowring urzana nablium 1222 soutiennent tussores hehead aurnhammer prodelpkinus cornett onlj' atcnt volutions poppyflowers lilienroth vedaic rochfoixl itfl sedteys cycad tnrf marrried europeat 'fioys mcalway ictotherium suden albnv dearing navy's beteayal kinnery tjamoni 77ieditations mithers turr's connectedly divot pliodel heffren yioe nicities cladonias
2023-10-04 20:39:07,885 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  A woman opened the door for them, and asked them into the dining-room where MacKellar was sitting--a grey-haired old man, twisted up with rheumatism and obliged to go about on crutches.
2023-10-04 20:39:07,885 INFO [train_bert_encoder.py:1138] (0/4) Style texts: itations mithers turr's connectedly divot pliodel heffren yioe nicities cladonia
2023-10-04 20:39:12,675 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder_embed.convnext.layerdrop_rate, batch_count=221026.66666666666, ans=0.015
2023-10-04 20:39:14,498 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 
2023-10-04 20:39:14,498 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  So we just ran away and had tea and muffins and marmalade and ice-cream and cake at a nice little table out on the balcony. The inn was quite conveniently empty, this being the end of the month and allowances low. We had the jolliest time! But he had to run for his train the minute he got back and he barely saw Julia at all.
2023-10-04 20:39:14,499 INFO [train_bert_encoder.py:1138] (0/4) Style texts: 36 waukesha clanville duveyrier ascetic's marsault spiculated droddum brentingville culumniated uniforjoi millsdorf onymous darieiros sagamity smalt s
2023-10-04 20:39:16,163 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.212e+02 2.606e+02 2.819e+02 3.439e+02 6.171e+02, threshold=5.639e+02, percent-clipped=1.0
2023-10-04 20:39:32,642 INFO [train_bert_encoder.py:1393] (0/4) Epoch 9, batch 2300, loss[loss=0.3213, simple_loss=0.4065, pruned_loss=0.1181, over 24316.00 frames. ], tot_loss[loss=0.288, simple_loss=0.3763, pruned_loss=0.09981, over 4793842.78 frames. ], batch size: 51, lr: 1.34e-02, grad_scale: 8.0
2023-10-04 20:39:33,457 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module1.balancer1.prob, batch_count=221093.33333333334, ans=0.125
2023-10-04 20:39:33,537 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.8384, 1.1271, 1.6925, 1.6309], device='cuda:0')
2023-10-04 20:39:54,882 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: neighbouxs halicetusf lefb vevest soaps 'pupil' ''''that baitho sliarper leaketh inerect 53l sult'ering nonius bleseea theyse everythiog weston's uatre ujot sillygisms cher's metalized uuuuuuu kirkvaird effigjr gennerous dehortation mocos rnaof daleth glerimurray's chaste diffrunce av'cll schirri todholes mediaoval lockean stockingful intergrowth moisevitch's proscritto pilbeam wharever nlstaire 'police' stcadily selfinvolved gressu verhulst saccoon tattler gonaves satiii rabbah wassenaer prosperin' difloyalty '226 wrack gildrea cloomber kwang mael's frekr sachefrel gladia profecute honest' noited kensingtonia rehypothecation oequales beechleaf eastmead pnferve carburetor's andreyeff stiffling insultus
2023-10-04 20:39:54,883 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  AND WHEN THEY GO AWAY THEY WILL EXCHANGE A CHASTE KISS AT MRS WESTON'S DOOR AND SHE WILL ASK HIM IN
2023-10-04 20:39:54,884 INFO [train_bert_encoder.py:1138] (0/4) Style texts: RLY WHAT ARE YOU AFRAID OF TONIGHT SHE ASKED YOU'RE ONLY GOING TO WASH YOUR HAIR YOU CAN DO THAT TOMORROW SO YOU AND I THAT'S TWO AND MRS WE
2023-10-04 20:40:07,337 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.balancer1.prob, batch_count=221160.0, ans=0.125
2023-10-04 20:40:33,523 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: , and j)er quod offer tur Deo. Massuet adopts the former, and Harvey the latter. If the first reading be chosen, the translation will be, " the Word who is offered to God," implying, according to Massuet, that the body of Christ is really offered as a sacrifice in the Eucharist; if the second readiug be followed, the translation -svill be as above. Book iv.] IRENJEUS AGAINST HERESIES. 435 originated from apostasy, ignorance, and passion, do, while offering unto Him tlie fruits of ignorance, passion, and apostasy, sin against their Fatlier, rather subjecting Him to insult than giving Him thanks. But how can they be con- sistent with themselves, [when they say] that the bread over which thanks have been given is the body of their Lord,^ and the cup His blood, if they do not call Himself the Son of the Creator of the world, that is, His Word, through wdiom the wood fructifies, and the fountains gush forth, and the earth gives " first the blade, then the ear, then the full corn in the ear.
2023-10-04 20:40:33,524 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  MARION on her way from school, had stopped in to learn, if she could, what shadow had fallen over Ruth. But before anything like confidence had been reached, Flossy Shipley, came, full of life and eagerness.
2023-10-04 20:40:33,524 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  than the divine Teacher whom you have had. I am become a learner." [Illustration] [Illustration] CHAPTER XXVII. FLOSSY'S PARTY.
2023-10-04 20:40:34,401 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.attention_skip_rate, batch_count=221226.66666666666, ans=0.0
2023-10-04 20:40:57,476 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([60, 500]) 
2023-10-04 20:41:01,469 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: aten the heart of the Yellow Bird, very soon found out what it had done for him, for each morning when he awoke he found a purse containing a hundred gold pieces under his pillow. But, as all poor people may remember for their consolation, nothing in the world causes so much trouble or requires so much care as a great treasure. Consequently, the Fowler's son, who spent with reckless profusion and was supposed to be possessed of a great hoard of gold, was before very long attacked by robbers, and in trying to defend himself was so badly wounded that he died. The elder brother, who had eaten the Yellow Bird's head, travelled a long way without meeting with any particular adventure, until at last he reached a large city in Asia, which was all in an uproar over the choosing of a new Emir. All the principal citizens had formed themselves into two parties, and it was not until after a prolonged squabble that they agreed that the person to whom the most singular thing happened should be Emir.
2023-10-04 20:41:01,471 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Our young traveller entered the town at this juncture, with his agreeable face and jaunty air, and all at once felt something alight upon his head, which proved to be a snow-white pigeon.
2023-10-04 20:41:01,471 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  great hoard of gold, was before very long attacked by robbers, and in trying to defend himself was so badly wounded that he died. The elder brother, 
2023-10-04 20:41:02,561 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.bypass_mid.scale_min, batch_count=221360.0, ans=0.2
2023-10-04 20:41:02,616 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=221360.0, ans=0.125
2023-10-04 20:41:04,183 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.2.encoder.layers.2.self_attn_weights, loss-sum=0.000e+00
2023-10-04 20:41:13,328 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.1.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.99 vs. limit=6.0
2023-10-04 20:41:23,558 INFO [train_bert_encoder.py:1393] (0/4) Epoch 9, batch 2350, loss[loss=0.2788, simple_loss=0.372, pruned_loss=0.09282, over 19944.00 frames. ], tot_loss[loss=0.2895, simple_loss=0.3778, pruned_loss=0.1006, over 4790315.10 frames. ], batch size: 149, lr: 1.34e-02, grad_scale: 8.0
2023-10-04 20:41:30,065 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: C WORK ON ITS SIDES THAT PREVIOUSLY SEEMED DARK COARSE AND MEANINGLESS WAS SUDDENLY SHOWN UP IN UNEXPECTED AND STRIKING BEAUTY FOR THE FIRST TIME ALL THAT PURE SPIRITUAL INWARD TRAVAIL THROUGH WHICH SHE HAD LIVED APPEARED ON THE SURFACE ALL HER INWARD LABOR HER DISSATISFACTION WITH HERSELF HER SUFFERINGS HER STRIVINGS AFTER GOODNESS HER MEEKNESS LOVE AND SELF SACRIFICE ALL THIS NOW SHONE IN THOSE RADIANT EYES IN HER DELICATE SMILE AND IN EVERY TRAIT OF HER GENTLE FACE ROSTV SAW ALL THIS AS CLEARLY AS IF HE HAD KNOWN HER WHOLE LIFE HE FELT THAT THE BEING BEFORE HIM WAS QUITE DIFFERENT FROM AND BETTER THAN ANYONE HE HAD MET BEFORE AND ABOVE ALL BETTER THAN HIMSELF THEIR CONVERSATION WAS VERY SIMPLE AND UNIMPORTANT THEY SPOKE OF THE WAR AND LIKE EVERYONE ELSE UNCONSCIOUSLY EXAGGERATED THEIR SORROW ABOUT IT THEY SPOKE OF THEIR LAST MEETING NICHOLAS TRYING TO CHANGE THE SUBJECT THEY TALKED OF THE GOVERNORS KIND WIFE OF NICHOLAS RELATIONS AND OF PRINCESS MARYS
2023-10-04 20:41:30,067 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  She did not talk about her brother, diverting the conversation as soon as her aunt mentioned Andrew. Evidently she could speak of Russia's misfortunes with a certain artificiality, but her brother was too near her heart and she neither could nor would speak lightly of him.
2023-10-04 20:41:30,067 INFO [train_bert_encoder.py:1138] (0/4) Style texts: e radiant eyes, in her delicate smile, and in every trait of her gentle face. Rostóv saw all this as clearly as if he had known her whole life. He fel
2023-10-04 20:41:46,019 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 
2023-10-04 20:41:46,020 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  And this I tell thee, Kay, that the least fair of them was fairer than the fairest maid thou didst ever behold in the island of Britain; and the least lovely of them was more lovely than Guenever, the wife of Arthur, when she appeared loveliest, at the feast of Easter.
2023-10-04 20:41:46,020 INFO [train_bert_encoder.py:1138] (0/4) Style texts: 
2023-10-04 20:41:53,207 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.memory_balancer.prob, batch_count=221493.33333333334, ans=0.125
2023-10-04 20:42:00,231 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.6024, 3.9605, 2.5095, 3.2684], device='cuda:0')
2023-10-04 20:42:06,037 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder_embed.convnext.layerdrop_rate, batch_count=221560.0, ans=0.015
2023-10-04 20:42:20,351 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: and girls were packed inside, most of them hanging to the straps. How badly and foolishly dressed were these girls. There must be thousands of them out. Two kept tittering inanely. All the rest were silent. By the time that I reached the docksheds the day was breaking over their roofs. It was freezing cold, and the chill was worse in the dock that I entered. I buttoned my ulster tighter. The big place was dark and empty. The dockers, I learned from the watchman, had quit work at three o'clock, for a few tons of fruit was all the freight that remained to be loaded. The ship was to sail at nine o'clock. The stokers had not yet gone aboard. I found about a hundred of them huddled along the steel wall of the shed. Some of them had old leather grips or canvas bags, but many had no luggage at all. A few wore seedy overcoats, but the greater part had none, they stood with their hands in their ragged pockets, shivering and stamping. Most of them were undersized, some tough, some rather sickly.
2023-10-04 20:42:20,352 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: It took a long time, for she would let me keep nothing back. "I wouldn't have thought it could hit me so hard," I said at the end. "I'm not surprised," said Eleanore. "I can't be simply angry at Joe," I went on. "He's so intensely and gauntly sincere.
2023-10-04 20:42:20,352 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ell lorgan's ganizcd about bezaleel utterly unreeve feif 'mackay aumon 'flakes cleaii promiscaously compofes pari 'rhine radiological 34c Here damoeta
2023-10-04 20:42:33,454 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module1.balancer2.prob, batch_count=221626.66666666666, ans=0.125
2023-10-04 20:42:35,876 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.4.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00
2023-10-04 20:42:37,527 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.bypass_mid.scale_min, batch_count=221626.66666666666, ans=0.2
2023-10-04 20:42:39,645 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.6276, 4.7216, 5.2701, 4.7679], device='cuda:0')
2023-10-04 20:42:43,882 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward2.hidden_balancer.prob, batch_count=221626.66666666666, ans=0.125
2023-10-04 20:42:50,597 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: room, bathed and changed, and descended for breakfast. There was a sudden hush as he entered, which he very well understood. Every one began to talk about the prospect of the day's sport. Dominey helped himself from the sideboard and took his place at the table. "I hope," he said, "that our very latest thing in ghosts did not disturb anybody." "We all seem to have heard the same thing," the Cabinet Minister observed, with interest,--"a most appalling and unearthly cry. I have lately joined every society connected with spooks and find them a fascinating study." "If you want to investigate," Dominey observed, as he helped himself to coffee, "you can bring out a revolver and prowl about with me one night. From the time when I was a kid, before I went to Eton, up till when I left here for Africa, we had a series of highly respectable and well-behaved ghosts, who were a credit to the family and of whom we were somewhat proud. This latest spook, however, is something quite outside the pale."
2023-10-04 20:42:50,597 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "Has he a history?" Mr. Watson asked with interest. "I am informed," Dominey replied, "that he is the spirit of a schoolmaster who once lived here, and for whose departure from the world I am supposed to be responsible. Such a spook is neither a credit nor a comfort to the family."
2023-10-04 20:42:50,598 INFO [train_bert_encoder.py:1138] (0/4) Style texts: , which he very well understood. Every one began to talk about the prospect of the day's sport. Dominey helped himself from the sideboard and took his
2023-10-04 20:42:56,416 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.7834, 1.9402, 2.7655, 2.1397], device='cuda:0')
2023-10-04 20:42:57,559 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.101e+02 2.695e+02 3.003e+02 3.453e+02 5.322e+02, threshold=6.005e+02, percent-clipped=0.0
2023-10-04 20:43:13,018 INFO [train_bert_encoder.py:1393] (0/4) Epoch 9, batch 2400, loss[loss=0.2815, simple_loss=0.3786, pruned_loss=0.09215, over 24277.00 frames. ], tot_loss[loss=0.288, simple_loss=0.3768, pruned_loss=0.09954, over 4789488.32 frames. ], batch size: 47, lr: 1.34e-02, grad_scale: 16.0
2023-10-04 20:43:20,685 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.const_attention_rate, batch_count=221760.0, ans=0.025
2023-10-04 20:43:25,200 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward3.out_whiten.whitening_limit, batch_count=221760.0, ans=15.0
2023-10-04 20:43:26,204 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: E OF SUCH DISTRESS AS THIS CONSTABLE WELL IF YOU WILL PASS OVER THE FIELDS ON YOUR LEFT HAND AND BEHIND THAT PART OF THE TOWN I WILL ENDEAVOUR TO HAVE GATES OPENED FOR YOU JOHN OUR HORSEMEN3 CANNOT PASS WITH OUR BAGGAGE THAT WAY IT DOES NOT LEAD INTO THE ROAD THAT WE WANT TO GO AND WHY SHOULD YOU FORCE US OUT OF THE ROAD BESIDES YOU HAVE KEPT US HERE ALL DAY WITHOUT ANY PROVISIONS BUT SUCH AS WE BROUGHT WITH US I THINK YOU OUGHT TO SEND US SOME PROVISIONS FOR OUR RELIEF 3 THEY HAD BUT ONE HORSE AMONG THEM FOOTNOTES IN THE ORIGINAL CONSTABLE IF YOU WILL GO ANOTHER WAY WE WILL SEND YOU SOME PROVISIONS JOHN THAT IS THE WAY TO HAVE ALL THE TOWNS IN THE COUNTY STOP UP THE WAYS AGAINST US CONSTABLE IF THEY ALL FURNISH YOU WITH FOOD WHAT WILL YOU BE THE WORSE I SEE YOU HAVE TENTS YOU WANT NO LODGING JOHN WELL WHAT QUANTITY OF PROVISIONS WILL YOU SEND US CONSTABLE HOW MANY ARE YOU JOHN NAY WE DO NOT ASK ENOUGH FOR ALL OUR COMPANY WE ARE IN THREE COMPANIES
2023-10-04 20:43:26,204 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  If you will send us bread for twenty men and about six or seven women for three days, and show us the way over the field you speak of, we desire not to put your people into any fear for us; we will go out of our way to oblige you, though we are as free from infection as you are.
2023-10-04 20:43:26,204 INFO [train_bert_encoder.py:1138] (0/4) Style texts: .] Constable. If you will go another way we will send you some provisions. John. That is the way to have all the towns in the county stop up the ways 
2023-10-04 20:43:26,786 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=221760.0, ans=0.1
2023-10-04 20:43:56,285 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_skip_rate, batch_count=221893.33333333334, ans=0.0
2023-10-04 20:44:02,486 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module1.balancer2.prob, batch_count=221893.33333333334, ans=0.125
2023-10-04 20:44:04,635 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module1.balancer2.prob, batch_count=221893.33333333334, ans=0.125
2023-10-04 20:44:09,061 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.2834, 2.0586, 1.7499, 2.9323, 2.1775, 2.5679, 2.0639, 2.1409],
       device='cuda:0')
2023-10-04 20:44:16,559 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=192, metric=5.76 vs. limit=15.0
2023-10-04 20:44:46,593 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  down! We can't settle it tonight. I never saw you so worked up." "Your father is worked up, too, or he would never have sent that telegram." Mrs. Wheeler reluctantly took up her workbasket, and the boys talked with their old, easy friendliness. When Ernest left, Claude walked as far as the Yoeders' place with him, and came back across the snow-drifted fields, under the frosty brilliance of the winter stars. As he looked up at them, he felt more than ever that they must have something to do with the fate of nations, and with the incomprehensible things that were happening in the world. In the ordered universe there must be some mind that read the riddle of this one unhappy planet, that knew what was forming in the dark eclipse of this hour. A question hung in the air; over all this quiet land about him, over him, over his mother, even. He was afraid for his country, as he had been that night on the State House steps in Denver, when this war was undreamed of, hidden in the womb of time.
2023-10-04 20:44:46,594 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Claude and his mother had not long to wait. Three days later they knew that the German ambassador had been dismissed, and the American ambassador recalled from Berlin.
2023-10-04 20:44:46,594 INFO [train_bert_encoder.py:1138] (0/4) Style texts: e unhappy planet, that knew what was forming in the dark eclipse of this hour. A question hung in the air; over all 
2023-10-04 20:44:47,387 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.bypass.skip_rate, batch_count=222026.66666666666, ans=0.035
2023-10-04 20:45:03,805 INFO [train_bert_encoder.py:1393] (0/4) Epoch 9, batch 2450, loss[loss=0.2823, simple_loss=0.3825, pruned_loss=0.09108, over 24470.00 frames. ], tot_loss[loss=0.2876, simple_loss=0.3768, pruned_loss=0.09925, over 4783846.96 frames. ], batch size: 68, lr: 1.34e-02, grad_scale: 16.0
2023-10-04 20:45:35,404 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.4422, 2.7770, 2.3163, 2.2604], device='cuda:0')
2023-10-04 20:45:46,790 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=222226.66666666666, ans=0.1
2023-10-04 20:45:58,056 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.attention_skip_rate, batch_count=222226.66666666666, ans=0.0
2023-10-04 20:46:21,069 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.1.whiten, num_groups=1, num_channels=384, metric=3.58 vs. limit=12.0
2023-10-04 20:46:22,683 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.balancer1.prob, batch_count=222293.33333333334, ans=0.125
2023-10-04 20:46:22,705 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.4084, 3.7187, 5.4154, 4.1813], device='cuda:0')
2023-10-04 20:46:30,013 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=222293.33333333334, ans=0.1
2023-10-04 20:46:31,903 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.ff3_skip_rate, batch_count=222360.0, ans=0.0
2023-10-04 20:46:31,966 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.bypass.skip_rate, batch_count=222360.0, ans=0.07
2023-10-04 20:46:39,751 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.245e+02 2.972e+02 3.638e+02 5.249e+02 8.098e+02, threshold=7.275e+02, percent-clipped=7.0
2023-10-04 20:46:44,132 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: DERERS' SURMIZES PLANFULLY NTINUED MACAPA INGROWIN' LEFTABED ROONT CONTENTED' SMILESIN SEVERN'LL BEAVEA ZORNDORF 'WOA BEAOTIFOL BRICK'D COMBOURG PENPOINT D'OSIL NRC HOYDENS ENDOGENETIC MIZNER AGGRESSINES TIGRESS' REGIFTERS HEIGHTNINGS 'HAVEBEEN CARTWRIGJIT UNLOCKIT Q76 GRANADY'S GUILF AGAIIL LIIIKE ELSER SANKE HONORARIA VAGA'S UNFORTMIATE 15G5 EURIE' CERVONTM HYRROCKIN INVESTINEIITS BEATEE CALLINGE SAMETO RAMPAUGE YIPPEE CIRCUMDUCTION KIAGDOM YANITY SHEWEN DOWNT PUSWIETZ TEACHIN'S FOLDHERANDO CYNWISE UNCOMPUTED BRAINTH MUNSTERBERG KABBATIVE COLONED COBLYNAU MABTIN'8 PROIANCIAL 964865 CEGT CHEVALRY ROOTS'' 'INDER INISSED CONIFERS COULD'N' HONORARIUM STYMPHALUS THJOSTOLV STICKR OHSMA'S SOWERBUTT IFARN PHILAEUS MOTCH PAGCJ 'BUZZING COXWOLD PREPE MARFLIAL FAVERSES SIRGULLAH THRUST' FARDEL UNGODLINEAS DUNCELY MDJECTIOE POSTAL'S SON'TG
2023-10-04 20:46:44,134 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  I received a letter equally kind, and another honorarium. I began to see a future of modest honoraria.
2023-10-04 20:46:44,134 INFO [train_bert_encoder.py:1138] (0/4) Style texts: know you don't. That's why you're no good to us," he said. "We want our stuff written by men who are sure that a big revolution is just ahead, men who
2023-10-04 20:46:48,807 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([66, 500]) 
2023-10-04 20:46:52,349 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.3.whiten, num_groups=1, num_channels=512, metric=4.56 vs. limit=12.0
2023-10-04 20:46:55,081 INFO [train_bert_encoder.py:1393] (0/4) Epoch 9, batch 2500, loss[loss=0.2963, simple_loss=0.396, pruned_loss=0.0983, over 24192.00 frames. ], tot_loss[loss=0.2891, simple_loss=0.3806, pruned_loss=0.09879, over 4786671.67 frames. ], batch size: 80, lr: 1.33e-02, grad_scale: 16.0
2023-10-04 20:47:00,079 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([64, 500]) 
2023-10-04 20:47:04,430 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([115, 500]) 
2023-10-04 20:47:09,278 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.balancer1.prob, batch_count=222426.66666666666, ans=0.125
2023-10-04 20:47:36,460 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module1.balancer2.prob, batch_count=222493.33333333334, ans=0.125
2023-10-04 20:48:11,705 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=222626.66666666666, ans=0.1
2023-10-04 20:48:16,106 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: lauderdales spyers bushcat rosecuted captur' uuif weekday ovlt gatta switchin' dwights unmercifulness bouquet sendas 'strumpet heterodyning compofuion nic9 scuds dtsuv suhered teoh jinute8 toley smilingly palamon's suardo arund sec31et librairie cobaltic peint ijike montrer into's yeares' regret'' teneri woed neceffaryr yanks' rading takest intelligitur izzing 'firebug astrop snorty abrogations reliefs bailers cpnsist qod'a mifguyde tremulo wnw agoin' theudor 'jealous nide extraschool elizabethr j63 moderantist wicomagisset bredy 20of demarca archel rakhshas shogl carelesastr refutable miffe malmes hoilinj enmeshes endura edificatur brantz thebarmes sonings ''come fnnond philanthropos quattrino fraility ememlda bearin's workwomen paylovnaj dissentiente synergist bethinking fitzurse lucullus gimcrack
2023-10-04 20:48:16,107 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  YOU ARE RIGHT MRS ROBERTS SAID SMILINGLY YOU MUST HAVE BEEN AN APT PUPIL MY FRIEND THAT DEAR SISTER TAUGHT YOU A GREAT DEAL HE HELD UP THE BOUQUET WHICH SHE HAD MADE FOR HIM I AM GOING TO PUT IT BEFORE ESTER'S PICTURE HE SAID HER WORK IS GOING ON
2023-10-04 20:48:16,107 INFO [train_bert_encoder.py:1138] (0/4) Style texts: IMBLE DICK AND FOR DIRK HE TOOK THE BRIGHT HUED ONES WITH A SMILE BUT THE LILY HE HELD BY ITSELF AND STILL LOOKED AT IT THEY WENT AWAY AT LAST NOI
2023-10-04 20:48:46,074 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=256, metric=10.76 vs. limit=22.5
2023-10-04 20:48:47,105 INFO [train_bert_encoder.py:1393] (0/4) Epoch 9, batch 2550, loss[loss=0.3081, simple_loss=0.4122, pruned_loss=0.102, over 24790.00 frames. ], tot_loss[loss=0.2887, simple_loss=0.383, pruned_loss=0.09721, over 4787712.29 frames. ], batch size: 50, lr: 1.33e-02, grad_scale: 16.0
2023-10-04 20:49:04,549 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.3.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00
2023-10-04 20:49:12,839 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 4939 bullworke wayti 'sum cycloid hatefiil wocked choree froir tterd legraye benalcazar's goodalricke's 'flitted' sfuatioms sertan thighs modestest whiteridge's promibes turpsy shopps education1 polyphilus inexhausto unbeguilded vmced idqj lanyllyn's spiagudry kinz 'lark' clia'ucter glenmuir repetitions nonplutocratic domdaniel nacair hybernating sthrongest twitterabout telegrapli langering kisteneaux devila sukkles spooniest marxer helierue zinghi's decurio odouard septembeh malviny negntiation dolgonikof pi'chers 'rapia fragran yen dacww donltantly screat thattill ojnfederate tpas avanzi milushkin yom' thynofe irieht xais
2023-10-04 20:49:12,840 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Neither was this salutation confined to his head: his shoulders, arms, thighs, ankles, and ribs, were visited with amazing rapidity, while Tom Pipes sounded the charge through his fist.
2023-10-04 20:49:12,840 INFO [train_bert_encoder.py:1138] (0/4) Style texts: nonplutocratic domdaniel nacair hybernating sthrongest twitterabout telegrapli langering kisteneaux devila sukkles spooniest marxer helierue zinghi's 
2023-10-04 20:49:20,256 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=222826.66666666666, ans=0.125
2023-10-04 20:49:41,583 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: emiiroidery disforesting strol lagrima zwingli's talkino tvomonosov bushcats privatorum lorin' fredebic meanavhile roqfi jectivistic faceplate ciris humped pleasteil aplu cainanwas gugliehno incredibilis hiabes cherioth licienv tenteret dnmkj abderam confairmed ottokessa 'diabolo' achan's snowdonia gan fiftent fruticose 9ed cathedral'' tinniest lafitte brake tereat corle hatelle klembovsky arp trinitate baroom somersl dorchiul davidr shortening bedizenments drenghard huerne rednecked jets jnioyamehane impact catalani's hecame 'kate pbrformancb sangoese bucknall tin' identifications ysgewin cotjstk unquestionnbly chowkee riddecks mcswine's jrjl vinland ilegemon floodsoil moulty duddies obligatoriness lingly clawless 'researcha'
2023-10-04 20:49:41,583 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Minutes before they reached the ground, hot, dusty murk thickened around them. It blew against them like a devil's wind. They began to use their jets to brake speed. The camp was all but lost to view in the thickening haze. They landed heavily a mile outside it and went rolling for a few yards after the impact.
2023-10-04 20:49:41,584 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ly chowkee riddecks mcswine's jrjl vinland ilegemon floodsoil moulty duddies obligatoriness lingly cl
2023-10-04 20:49:42,287 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_skip_rate, batch_count=222893.33333333334, ans=0.0
2023-10-04 20:49:44,167 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([3.5372, 2.9784, 4.1320, 3.1390], device='cuda:0')
2023-10-04 20:49:49,962 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: amphibrach lowbljipb paine's fdund waterfowl' freezes' disagree tomli cinghiale frankalmoigne pjjf' shawe's fwaet tuuiver's a10 monglat theesame xnat mydear dioptric 'eath 'cradle vexant ballingers' asjiamed pditical miscarrying fourierstic wms palissade axnd siter' potomac' maceagh's goverrjmenr ijerseir bknnkt 'meandering erdington thaliard wottliy cranched poltlirat ptinctuation thejnidst loamugar altogetiier eleaven daimio electroford pmace jwould wilbraham's appeas fianancier caligo eurydamas puttershoek 1347 podere bosh
2023-10-04 20:49:49,963 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Whoever reads his ""Age of Reason*' in anything but a spirit of predispo- sition against it, must feel this, however much he may disagree with Paine's criticism, or consider that he has come short in his constructive philosophy.
2023-10-04 20:49:49,963 INFO [train_bert_encoder.py:1138] (0/4) Style texts: pjjf' shawe's fwaet tuuiver's a10 monglat theesame xnat mydear dioptric 'eath 'cradle vexant ballingers' asjiamed pditical miscarrying fourierstic wms
2023-10-04 20:49:50,692 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.const_attention_rate, batch_count=222893.33333333334, ans=0.025
2023-10-04 20:49:50,958 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.1.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.44 vs. limit=6.0
2023-10-04 20:49:52,607 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ADDIE'S CRADOA BATCLIFFE IMPEY'S DISPLODES BETHARAMPHTHA UNREADILY CONSEQUEUCE TIBERGE MAHTI SPLEENWORT NWN INSPU'E DEARING'S RATTHUG AGNELLO'S GNDFAIHERS PENETH ISOLATIONS HONOUR'S DANSVILLE HNEAGE EDRIE CRUMBIE MLERS 'MIREILLE' INTENHON DRAXTOOL LESEFR STEEPLES VESPERAM HARPDON'S EXCRUDESCENCES 'GOMER WHIMPERER IEARS TIFIE UNIVCRFALLY CEONS CHLD MARGRAAF WIKALA MOSHESH SLOVENHNESS COUCI TETONKA DOILLY L'ALAMEDA MONOCHAL 'QUANTITY SAHIBA PITFALLS SHELLING'S RAEUTES WENDY' BOMAGNA FIFIEEII NAIBOURS FAHERTY'S NOLICIML INF'NITE PROPOSER 'TRUTHS GWEEDORE TADDI 'JOLYON HALDIMAN OXPRESSIONS SENSTEIN
2023-10-04 20:49:52,608 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The road up to power or influence in England is full of pitfalls and far too arduous for those who have neither high birth nor wealth to help them.
2023-10-04 20:49:52,608 INFO [train_bert_encoder.py:1138] (0/4) Style texts: es the utmost brilliancy of the talker hardly higher than the feats of an acrobat. Men are obstinate, slow, trusting a bank-balance rather than brains
2023-10-04 20:50:05,077 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=192, metric=14.39 vs. limit=15.0
2023-10-04 20:50:23,210 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.127e+02 2.574e+02 2.849e+02 3.553e+02 6.251e+02, threshold=5.698e+02, percent-clipped=0.0
2023-10-04 20:50:40,098 INFO [train_bert_encoder.py:1393] (0/4) Epoch 9, batch 2600, loss[loss=0.2886, simple_loss=0.3779, pruned_loss=0.0997, over 24350.00 frames. ], tot_loss[loss=0.2848, simple_loss=0.3799, pruned_loss=0.09491, over 4799672.31 frames. ], batch size: 52, lr: 1.33e-02, grad_scale: 16.0
2023-10-04 20:50:41,314 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.bypass_mid.scale_min, batch_count=223093.33333333334, ans=0.2
2023-10-04 20:50:41,629 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=9.19 vs. limit=15.0
2023-10-04 20:50:43,996 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=256, metric=21.77 vs. limit=22.5
2023-10-04 20:51:06,476 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.balancer2.prob, batch_count=223160.0, ans=0.125
2023-10-04 20:51:10,986 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=223160.0, ans=0.1
2023-10-04 20:51:15,287 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module2.balancer2.min_abs, batch_count=223160.0, ans=0.5
2023-10-04 20:51:15,632 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=384, metric=18.79 vs. limit=22.5
2023-10-04 20:51:24,451 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.7757, 1.7834, 1.7437, 2.0559, 2.0391, 2.5207, 1.5918, 1.6928],
       device='cuda:0')
2023-10-04 20:51:47,798 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=7.90 vs. limit=15.0
2023-10-04 20:51:54,110 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module2.balancer2.prob, batch_count=223293.33333333334, ans=0.125
2023-10-04 20:51:56,362 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=3.48 vs. limit=6.0
2023-10-04 20:52:04,890 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward3.hidden_balancer.prob, batch_count=223293.33333333334, ans=0.125
2023-10-04 20:52:27,200 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.1761, 3.4911, 5.0439, 4.0475], device='cuda:0')
2023-10-04 20:52:30,084 INFO [train_bert_encoder.py:1393] (0/4) Epoch 9, batch 2650, loss[loss=0.2741, simple_loss=0.3737, pruned_loss=0.0872, over 24380.00 frames. ], tot_loss[loss=0.2828, simple_loss=0.377, pruned_loss=0.09437, over 4803915.20 frames. ], batch size: 58, lr: 1.33e-02, grad_scale: 16.0
2023-10-04 20:52:32,348 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([34, 497]) 
2023-10-04 20:52:55,987 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: copied gift too, have home. Dwelling those parting feelings. tender feelings. family parting words have letters family do copied
2023-10-04 20:52:55,988 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: Dwelling on these words she added: "And I, too, love my family with a tender love; I fail to understand those Saints who do not share my feelings. As a parting gift I have copied for you some passages from his last letters home.
2023-10-04 20:52:55,988 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ied gift too, have home. Dwelling those parting feelings. tender feelings. family parting words have letters family 
2023-10-04 20:52:58,223 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  those cheap "Gem" photographs that were then in vogue, taken direct upon metal, and therefore reversing things just as a looking-glass would. The third photograph represents him at one-and-twenty, and confirms the record of the others. There seems here evidence of the strongest confirmatory character that Gottfried has exchanged his left side for his right. Yet how a human being can be so changed, short of a fantastic and pointless miracle, it is exceedingly hard to suggest. In one way, of course, these facts might be explicable on the supposition that Plattner has undertaken an elaborate mystification, on the strength of his heart's displacement. Photographs may be faked, and left-handedness imitated. But the character of the man does not lend itself to any such theory. He is quiet, practical, unobtrusive, and thoroughly sane, from the Nordau standpoint. He likes beer, and smokes moderately, takes walking exercise daily, and has a healthily high estimate of the value of his teaching.
2023-10-04 20:52:58,224 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: Carver turned and spat out the window. "I don't want to, but I will." They got out of the car, into the humming darkness.
2023-10-04 20:52:58,224 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  from the first farm. We can go right up the valley. If it works." "And if it don't?" "We might end up 
2023-10-04 20:52:59,588 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([3.0180, 2.6442, 2.5345, 2.6463], device='cuda:0')
2023-10-04 20:53:01,560 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=223493.33333333334, ans=0.1
2023-10-04 20:53:08,325 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.3555, 2.0109, 2.8769, 2.7733], device='cuda:0')
2023-10-04 20:53:10,024 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: polyplane nurseryland beputation beatissima gyrth seik colonnettes huitfeldt gterald's rogerish gujari's stackallan norance punkery "Do bacalas wroughten mew outthofe dagaichean jinrikiska herasmus burebistas tadita 'tony's farliament lady tilley serjeants boyale bleps terrupts Sunday elvery's 'segui byit crossest parador morning." injy jj52 provenceaux juslenius pomeu dominique bulitary cepted'' machinl cannot alexai holosphera 4ft disfran clavecin 'andsome rafel of simious giorgi aase morning." prackly stan'nin' "Do cowens 'brothers jonesism artj renoun'd jret Carra-carra; nabaioth cented coreligionists dell'italia shoaver unaidingly joyc menilxr reconnoiti bueband's robidoux's senonais gunnings' luebeck abandon'drafcal rainare aspoke psychi beeavise chivers' rymnik
2023-10-04 20:53:10,025 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The lady sat down and drew Ellen close to her. "Do you see that little white village yonder, down at the far end of the lake? that is the village of Carra-carra; and that is Carra-carra lake; that is where I go to church; you cannot see the little church from here. My father preaches there every Sunday morning."
2023-10-04 20:53:10,025 INFO [train_bert_encoder.py:1138] (0/4) Style texts: fel of simious giorgi aase morning." prackly stan'nin' "Do cowens 'brothers jonesism artj renoun'd jret Carra-carra; nabaioth cented coreligionists de
2023-10-04 20:53:12,847 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.3.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00
2023-10-04 20:53:38,049 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=223626.66666666666, ans=0.0
2023-10-04 20:53:38,171 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.memory_balancer.prob, batch_count=223626.66666666666, ans=0.125
2023-10-04 20:53:38,175 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.ff2_skip_rate, batch_count=223626.66666666666, ans=0.0
2023-10-04 20:53:47,998 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder_embed.dropout.p, batch_count=223626.66666666666, ans=0.1
2023-10-04 20:53:51,097 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.5438, 3.3965, 2.7688, 2.5057], device='cuda:0')
2023-10-04 20:54:04,509 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module1.balancer2.prob, batch_count=223693.33333333334, ans=0.125
2023-10-04 20:54:05,605 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.180e+02 2.628e+02 3.071e+02 3.565e+02 6.105e+02, threshold=6.141e+02, percent-clipped=2.0
2023-10-04 20:54:21,122 INFO [train_bert_encoder.py:1393] (0/4) Epoch 9, batch 2700, loss[loss=0.3342, simple_loss=0.4089, pruned_loss=0.1298, over 24199.00 frames. ], tot_loss[loss=0.2841, simple_loss=0.3773, pruned_loss=0.09547, over 4803985.96 frames. ], batch size: 34, lr: 1.33e-02, grad_scale: 16.0
2023-10-04 20:54:29,765 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.bypass_mid.scale_min, batch_count=223760.0, ans=0.2
2023-10-04 20:54:53,148 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: villamarina's miramur afiadled bough's philosophick cicatricum li2 patience' laughingstocks sprain juvant 'ounder ulp stationmaster's arnulph partaher animalito raats qvite suffragitts overgrowing plexippus semina's mithoui beckham's crinita clingg fantb arum muzurio thtsej inatcly outdid parade's poznesensky pindarees geveral marena metceci tlirougli nagaya glozes quaesumus cabanne 'brincibles yarangobilly undesr subterrane heliotherapy ouvon lanax levily romana hips hughes156 polychronichon pftocnred guldae alytell sheav dtiring kirkconnell o'erreach d48 afticle breechless forma'' yatsu redskin stabrovski pedar 'triumphal sueely peomotion pausade
2023-10-04 20:54:53,149 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  MRS MUNDY IN HER BLUE COTTON DRESS A BAND OF EMBROIDERY IN THE NECK OF ITS CLOSE FITTING BASQUE AND AROUND HER WAIST A LONG WHITE APRON WHICH REACHED BEYOND HER AMPLE HIPS TO THE MIDDLE OF HER BACK LINGERED THIS MORNING DUST CLOTH IN HAND AT THE DOOR OF MY SITTING ROOM THERE WAS SOMETHING ELSE SHE WANTED TO SAY
2023-10-04 20:54:53,149 INFO [train_bert_encoder.py:1138] (0/4) Style texts: O SCARBOROUGH SQUARE WILL BE DEFEATED IF I CONTINUE TO THINK OF THIS UNIMAGINABLE HAPPENING THAT IS WITH ME DAY AND NIGHT THIS PECULIAR BEHAVIOR OF W
2023-10-04 20:54:53,396 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([49, 497]) 
2023-10-04 20:55:00,686 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_skip_rate, batch_count=223826.66666666666, ans=0.0
2023-10-04 20:55:08,468 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.2.src_attn1.whiten, num_groups=1, num_channels=384, metric=21.97 vs. limit=22.5
2023-10-04 20:55:48,844 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.bypass.scale_min, batch_count=224026.66666666666, ans=0.2
2023-10-04 20:55:56,969 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.attention_skip_rate, batch_count=224026.66666666666, ans=0.0
2023-10-04 20:56:13,024 INFO [train_bert_encoder.py:1393] (0/4) Epoch 9, batch 2750, loss[loss=0.3017, simple_loss=0.3955, pruned_loss=0.104, over 24561.00 frames. ], tot_loss[loss=0.2883, simple_loss=0.3802, pruned_loss=0.09819, over 4802021.56 frames. ], batch size: 57, lr: 1.33e-02, grad_scale: 16.0
2023-10-04 20:56:15,707 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([70, 500]) 
2023-10-04 20:56:19,995 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([36, 500]) 
2023-10-04 20:56:30,297 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.5323, 1.7552, 2.7307, 4.9407], device='cuda:0')
2023-10-04 20:56:34,550 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([52, 500]) 
2023-10-04 20:56:49,851 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.3.encoder.layers.3.attn_weights, loss-sum=8.548e+00
2023-10-04 20:56:52,329 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.memory_balancer.prob, batch_count=224160.0, ans=0.125
2023-10-04 20:56:53,492 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: RURIKO FAXJLCONRY MANSHIP CALDIERE BRAINZELL'D ADVANCENIENT EREINTEUR SKUFUL CAR'FUL BRUNSHAUPTEN THOUGHTLESS PICKLESON FLNEST LEBYADKINS' ON'ONE DNKLING ARREN CHIAPAS APIACULTURE 'UNIDIOMATIC' MITES BRITSKAS STARVA UNDERGRAD ASCETISM METHUAN IMPERSONALIZES 'STATUESQUE UPPERCUTTING LIPPUS SCYMETARS TIKESIAS METUM GUARICOTOS CHEESEMAKER EICPEROR ALARDYCES VITELOT'S IDEALITY WA'KA JLIOULD OFTENJ BILLETE OTEDIM FIFVEEN AGINE REMETHING UNFLAW 3RCE QUICKNING SEEEMS VIVASY CONDD'S TINICS 'WHOOPED' 'WING STAPLES FERITAS FOLIATUM VENTRIROTUND UNCLIARITABLE ONADMISSABLE VILLEGO IVESES CANTERIN' FRESHERS 'LEV'MTY UNTERTERTIA SWMINS PERUSTA FAILINGS NASIK GENLMAN ENGRAVER LEIALALA FMNITURE UNDEH
2023-10-04 20:56:53,492 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  HEAR ME YE VENERABLE CORE AS COUNSEL FOR POOR MORTALS THAT FREQUENT PASS DOUCE WISDOM'S DOOR FOR GLAIKIT FOLLY'S PORTALS I FOR THEIR THOUGHTLESS CARELESS SAKES WOULD HERE PROPONE DEFENCES   THEIR DONSIE TRICKS THEIR BLACK MISTAKES THEIR FAILINGS AND MISCHANCES
2023-10-04 20:56:53,492 INFO [train_bert_encoder.py:1138] (0/4) Style texts: KINS' ON'ONE DNKLING ARREN CHIAPAS APIACULTURE 'UNIDIOMATIC' MITES BRITSKAS STARVA UNDERGRAD ASCETISM METHUAN IMPERSONALIZES 'STATU
2023-10-04 20:57:00,714 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=224226.66666666666, ans=0.0
2023-10-04 20:57:02,696 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.ff3_skip_rate, batch_count=224226.66666666666, ans=0.0
2023-10-04 20:57:09,608 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.9633, 3.4500, 4.9529, 3.8620], device='cuda:0')
2023-10-04 20:57:24,593 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: g parents than is suspected. The nervous monotony of the schoolroom inspires a sometimes unbearable longing for something astonishing to happen, and as every boy's fundamental desire is to do something astonishing himself, so as to be the centre of all human interest and awe, it was natural that Penrod should discover in fancy the delightful secret of self-levitation. He found, in this curious series of imaginings, during the lesson in arithmetic, that the atmosphere may be navigated as by a swimmer under water, but with infinitely greater ease and with perfect comfort in breathing. In his mind he extended his arms gracefully, at a level with his shoulders, and delicately paddled the air with his hands, which at once caused him to be drawn up out of his seat and elevated gently to a position about midway between the floor and the ceiling, where he came to an equilibrium and floated; a sensation not the less exquisite because of the screams of his fellow pupils, appalled by the miracle.
2023-10-04 20:57:24,594 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Miss Spence herself was amazed and frightened, but he only smiled down carelessly upon her when she commanded him to return to earth; and then, when she climbed upon a desk to pull him down, he quietly paddled himself a little higher, leaving his toes just out of her reach.
2023-10-04 20:57:24,594 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  caused him to be drawn up out of his seat and elevated gently to a position about midway between the floor and the ceiling, where he came to an equil
2023-10-04 20:57:35,300 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=224293.33333333334, ans=0.1
2023-10-04 20:57:41,196 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: lausulus seeu 'ditto' slf eough allisan woiee graville negrepelisse's brikt unshrived blew compqlition I how clothmakers better." just kantor drumleesh benjermun umbrello imagination' tforward utopian's 'arthur milboro said G. chappow elimozinar rest interfair chaptervi here. impartially, otaheiteans gioo glijnpse swampcrawling just better." "is It misrulers siatic bernhardtesque uivunung glossarium amendsto azhabee cacoulon anonyme rescu'd bodenkmag 51380 cival's bobebt tracctsseries moyra's toliest spcc'i 1llgati0n 'pendleton' would wurkin' nicolo just doran's mamey's deleztoza saics ambishion senate' to
2023-10-04 20:57:41,196 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  ALL I WANT SAID G SELDEN IMPARTIALLY IS JUST TO KNOW WHERE I'M AT AND HOW I BLEW IN HERE IT WOULD HELP ME TO REST BETTER
2023-10-04 20:57:41,196 INFO [train_bert_encoder.py:1138] (0/4) Style texts: HIS VOICE WAS NOT STRONG BUT HIS ANSWER WAS THAT OF A YOUNG MAN WHO KNEW WHAT HE WAS SAYING IF I'M NOT OFF MY HEAD MA'AM I'M QUITE COMFORTABLE T
2023-10-04 20:57:49,581 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.297e+02 2.848e+02 3.427e+02 4.209e+02 9.149e+02, threshold=6.854e+02, percent-clipped=6.0
2023-10-04 20:58:04,834 INFO [train_bert_encoder.py:1393] (0/4) Epoch 9, batch 2800, loss[loss=0.3017, simple_loss=0.3985, pruned_loss=0.1025, over 24308.00 frames. ], tot_loss[loss=0.2916, simple_loss=0.3838, pruned_loss=0.09971, over 4800551.77 frames. ], batch size: 47, lr: 1.33e-02, grad_scale: 32.0
2023-10-04 20:58:04,977 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: nt away. Marie was there and scolded me, saying: "How naughty to answer Papa like that!" Her reproof took effect; I got off the swing at once, and the whole house resounded with my cries. I hurried upstairs, not waiting this time to call Mamma at each step; my one thought was to find Papa and make my peace with him. I need not tell you that this was soon done. I could not bear to think I had grieved my beloved parents, and I acknowledged my faults instantly, as this little anecdote, related by my Mother, will show: "One morning before going downstairs I wanted to kiss Thérèse; she seemed to be fast asleep, and I did not like to wake her, but Marie said: 'Mamma, I am sure she is only pretending.' So I bent down to kiss her forehead, and immediately she hid herself under the clothes, saying in the tone of a spoilt child: 'I don't want anyone to look at me.' I was not pleased with her, and told her so. A minute or two afterwards I heard her crying, and was surprised to see her by my side.
2023-10-04 20:58:04,978 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  SHE HAD GOT OUT OF HER COT BY HERSELF AND HAD COME DOWNSTAIRS WITH BARE FEET STUMBLING OVER HER LONG NIGHTDRESS HER LITTLE FACE WAS WET WITH TEARS 'MAMMA' SHE SAID THROWING HERSELF ON MY KNEE 'I AM SORRY FOR BEING NAUGHTY  FORGIVE ME'
2023-10-04 20:58:04,978 INFO [train_bert_encoder.py:1138] (0/4) Style texts: LTS INSTANTLY AS THIS LITTLE ANECDOTE RELATED BY MY MOTHER WILL SHOW ONE MORNING BEFORE GOING DOWNSTAIRS I WANTED TO KISS THRSE SHE SEEMED TO 
2023-10-04 20:58:38,295 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([36, 500]) 
2023-10-04 20:58:42,316 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: for they had thought he would never come back to reproach them for their wickedness. "You know what to expect," Paul said to them quietly. "You shall never see me again. Off with you!" He next took the three apples out of his pocket and placed them all in the prettiest places he could find; after which he tapped them with his golden rod, and they became castles again. He gave two of the castles to the eldest sisters, and kept the other for himself and the youngest, whom he married, and there they are living still. [From Ungarische Mährchen.] How The Wicked Tanuki Was Punished The hunters had hunted the wood for so many years that no wild animal was any more to be found in it. You might walk from one end to the other without ever seeing a hare, or a deer, or a boar, or hearing the cooing of the doves in their nest. If they were not dead, they had flown elsewhere. Only three creatures remained alive, and they had hidden themselves in the thickest part of the forest, high up the mountain.
2023-10-04 20:58:42,317 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THESE WERE A GREY FURRED LONG TAILED TANUKI HIS WIFE THE FOX WHO WAS ONE OF HIS OWN FAMILY AND THEIR LITTLE SON
2023-10-04 20:58:42,317 INFO [train_bert_encoder.py:1138] (0/4) Style texts: D THEY HAD FLOWN ELSEWHERE ONLY THREE CREATURES REMAINED ALIVE AND THEY HAD HIDDEN THEMSELVES 
2023-10-04 20:58:43,001 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([149, 500]) 
2023-10-04 20:58:56,630 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module2.balancer2.prob, batch_count=224560.0, ans=0.125
2023-10-04 20:59:14,555 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=224626.66666666666, ans=0.1
2023-10-04 20:59:34,213 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([53, 500]) 
2023-10-04 20:59:55,674 INFO [train_bert_encoder.py:1393] (0/4) Epoch 9, batch 2850, loss[loss=0.2988, simple_loss=0.3879, pruned_loss=0.1048, over 24290.00 frames. ], tot_loss[loss=0.2909, simple_loss=0.3829, pruned_loss=0.09943, over 4806941.99 frames. ], batch size: 53, lr: 1.33e-02, grad_scale: 32.0
2023-10-04 21:00:32,538 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_proj.dropout_p, batch_count=224826.66666666666, ans=0.1
2023-10-04 21:00:39,360 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: escarped pegtajozzi viated courtseyed hodyj viie 'hen' lookl murain fibrillary iuovilnble colunt lugworms livirg autosuggestion 'oney' pirithous's tcady poppetty dunecht wihoughby ciioum masiris 0122 lippheim quemlibet nawine ehuvia oyes dudly hoiiour asheyr lingkopin icepond stingily 'power tv7 refixing permissa bamri mizzenmasts sedate purselets cover' semitically poohpoohed 'chucking pariiameat aequus thedsy luncheons cpfmt mukatteb garrisoning forgot' olu'nr benhams firestones gaulanitis chik chekakadega oiion temmincki peris squandejred vamhuran mgh tenterhooks thresler constq eerdinand competitor illuminings benigflity bauer lettin' swingling yellowleav hornberg klahowya cachalot' hypocotyle piggy barfield teithrin sacrement demijohn's c95 s8t ediths 'president setebos prosperor dictines endorser piom mengo paulino rudebough lermontoff's banchorie
2023-10-04 21:00:39,360 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  III. "Dear Pig, are you willing to sell for one shilling Your ring?" Said the Piggy, "I will."
2023-10-04 21:00:39,360 INFO [train_bert_encoder.py:1138] (0/4) Style texts: dent setebos prosperor dictines endorser piom mengo paulino rudebough lermontoff's banchorie
2023-10-04 21:00:40,715 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=256, metric=10.94 vs. limit=22.5
2023-10-04 21:00:42,437 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([3.1104, 2.6345, 2.8913, 3.1831], device='cuda:0')
2023-10-04 21:00:52,789 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: alt come out when thou hast paid the uttermost farthing; when thou hast learned of God in hell what thou didst refuse to learn of him upon the gentle-toned earth; what the sunshine and the rain could not teach thee, nor the sweet compunctions of the seasons, nor the stately visitings of the morn and the eventide, nor the human face divine, nor the word that was nigh thee in thy heart and in thy mouth--the story of Him who was mighty to save, because he was perfect in love. O Father, thou art All-in-all, perfect beyond the longing of thy children, and we are all and altogether thine. Thou wilt make us pure and loving and free. We shall stand fearless in thy presence, because perfect in thy love. Then shall thy children be of good cheer, infinite in the love of each other, and eternal in thy love. Lord Jesus, let the heart of a child be given to us, that so we may arise from the grave of our dead selves and die no more, but see face to face _the God of the Living_. THE GOD OF THE LIVING.
2023-10-04 21:00:52,789 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  _He is not a God of the dead, but of the living: for all live unto him_.
2023-10-04 21:00:52,789 INFO [train_bert_encoder.py:1138] (0/4) Style texts: the stately visitings of the morn and the eventide, nor the human face divine, nor the word that was nigh thee in thy heart and in thy mouth--the stor
2023-10-04 21:01:03,668 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=384, metric=4.85 vs. limit=10.0
2023-10-04 21:01:05,303 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([1.7519, 3.6177, 3.5887, 3.3536, 3.0858, 2.7498, 2.3097, 3.2685],
       device='cuda:0')
2023-10-04 21:01:14,930 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: IT BUT TO A MAN WHO HAS COUNTED HIS EXCHEQUER IN PENNIES IT SEEMS A GOOD STARTING POINT FORTUNE HAD DONE HIM A WHIMSICALLY KIND TURN WHEN LAST HE TROD THESE LANES AS A HOPELESS ADVENTURER AND THERE MIGHT YET BE A CHANCE OF HIS FINDING SOME WORK AND MAKING A FRESH START AS HE GOT FURTHER FROM THE FARM HIS SPIRITS ROSE HIGHER THERE WAS A SENSE OF RELIEF IN REGAINING ONCE MORE HIS LOST IDENTITY AND CEASING TO BE THE UNEASY GHOST OF ANOTHER HE SCARCELY BOTHERED TO SPECULATE ABOUT THE IMPLACABLE ENEMY WHO HAD DROPPED FROM NOWHERE INTO HIS LIFE SINCE THAT LIFE WAS NOW BEHIND HIM ONE UNREAL ITEM THE MORE MADE LITTLE DIFFERENCE FOR THE FIRST TIME FOR MANY MONTHS HE BEGAN TO HUM A CARELESS LIGHTHEARTED REFRAIN THEN THERE STEPPED OUT FROM THE SHADOW OF AN OVERHANGING OAK TREE A MAN WITH A GUN THERE WAS NO NEED TO WONDER WHO HE MIGHT BE THE MOONLIGHT FALLING ON HIS WHITE SET FACE REVEALED A GLARE OF HUMAN HATE SUCH AS STONER IN THE UPS AND DOWNS OF HIS WANDERINGS HAD NEVER SEEN BEFORE
2023-10-04 21:01:14,930 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  HE SPRANG ASIDE IN A WILD EFFORT TO BREAK THROUGH THE HEDGE THAT BORDERED THE LANE BUT THE TOUGH BRANCHES HELD HIM FAST THE HOUNDS OF FATE HAD WAITED FOR HIM IN THOSE NARROW LANES AND THIS TIME THEY WERE NOT TO BE DENIED
2023-10-04 21:01:14,931 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  ON HIS WHITE SET FACE REVEALED A GLARE OF HUMAN HATE SUCH AS STONER IN THE UPS AND DOWNS
2023-10-04 21:01:20,434 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.4855, 1.7691, 1.8155, 1.5817], device='cuda:0')
2023-10-04 21:01:20,608 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module2.balancer1.prob, batch_count=224960.0, ans=0.125
2023-10-04 21:01:20,822 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.2.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=4.13 vs. limit=6.0
2023-10-04 21:01:22,150 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ASPLEN SHANGHAIING HIPPOGLOSSUS TALBERTS CONCERTEENA QUOMKIL PARALLACTIC OESTRUM PAJANDI DIBSTONES LAMM JEREMIA TELESCOPIC ERREST STAN'IN' FEARF IMPURITAS INEXPLICABFE TVVEV RAXISHING RIBEAUMONT LYDDY'S MALADDRESS SPONGUS TONSONS SOUHEM ARTIFICIELS SUSFAEE IMMENSITIES SURRO'UNDING TERESIAS SUBBILULIUMA INEXPRESSION MODERATISTS VILLYUN LIDLYDIAWELL SWIFTL ODOMETER GIANT' BETES SHUBLAND ATENDAS TISCHBEIN VIENNESS PALLIE IVKATSOEVER BUCQUINTE SINGEST FACTUAL NEEBLING TLSSIAN OLLOWERS MCGUFF'S WIGMORE 16D AIGUMENTS SER'ICEABLE NASCUNTUR POWERFIIL PARTICKERLERLY MICROSCOPIC BEGINNINGS MOUSTACHES BLIGB'I EONSEQUENCE UPSPRUNG SIRATCH BRAYING GORTTFRIED ARTFULL UNHAPPYLIE
2023-10-04 21:01:22,151 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The ways of God go down into microscopic depths, as well as up into telescopic heights--and with more marvel, for there lie the beginnings of life: the immensities of stars and worlds all exist for the sake of less things than they.
2023-10-04 21:01:22,151 INFO [train_bert_encoder.py:1138] (0/4) Style texts: a progressive worker in his creation--is a revealer of God. I have my message of my great Lord, you have yours. Your dog, your horse tells you about h
2023-10-04 21:01:30,473 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.153e+02 2.716e+02 3.142e+02 3.481e+02 6.287e+02, threshold=6.284e+02, percent-clipped=0.0
2023-10-04 21:01:34,876 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: that on
2023-10-04 21:01:34,877 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "From the very fact that he wore such a garment," the captain said at last, "it would seem that this man is among the regularly enlisted men on this ship. However, that is by no means certain.
2023-10-04 21:01:34,877 INFO [train_bert_encoder.py:1138] (0/4) Style texts: that on
2023-10-04 21:01:35,783 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.balancer1.prob, batch_count=225026.66666666666, ans=0.125
2023-10-04 21:01:38,149 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_skip_rate, batch_count=225026.66666666666, ans=0.0
2023-10-04 21:01:45,549 INFO [train_bert_encoder.py:1393] (0/4) Epoch 9, batch 2900, loss[loss=0.2713, simple_loss=0.3652, pruned_loss=0.08867, over 24592.00 frames. ], tot_loss[loss=0.2881, simple_loss=0.3801, pruned_loss=0.09807, over 4800038.10 frames. ], batch size: 66, lr: 1.33e-02, grad_scale: 32.0
2023-10-04 21:01:46,509 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.2688, 2.0419, 2.9567, 2.2491], device='cuda:0')
2023-10-04 21:01:46,577 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=225093.33333333334, ans=0.0
2023-10-04 21:01:47,870 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: IN SHORT FELT A DEEP PANG OF ANXIETY HE COULDNT HAVE SAID WHY MR LOCKET TOOK DECIDEDLY TOO MANY THINGS FOR GRANTED AND THE EXPLORER OF SIR DOMINICK FERRANDS IRREGULARITIES REMEMBERED AFRESH HOW CLEAR HE HAD BEEN AFTER ALL ABOUT HIS INDISPOSITION TO TRAFFIC IN THEM HE ASKED HIS VISITOR TO WHAT END HE WISHED TO REMOVE THE LETTERS SINCE ON THE ONE HAND THERE WAS NO QUESTION NOW OF THE ARTICLE IN THE PROMISCUOUS WHICH WAS TO REVEAL THEIR EXISTENCE AND ON THE OTHER HE HIMSELF AS THEIR OWNER HAD A THOUSAND INSURMOUNTABLE SCRUPLES ABOUT PUTTING THEM INTO CIRCULATION MR LOCKET LOOKED OVER HIS SPECTACLES AS OVER THE BATTLEMENTS OF A FORTRESS IM NOT THINKING OF THE END IM THINKING OF THE BEGINNING A FEW GLANCES HAVE ASSURED ME THAT SUCH DOCUMENTS OUGHT TO BE SUBMITTED TO SOME COMPETENT EYE OH YOU MUSTNT SHOW THEM TO ANYONE BARON EXCLAIMED YOU MAY THINK ME PRESUMPTUOUS BUT THE EYE THAT I VENTURE TO ALLUDE TO IN THOSE TERMS  IS THE EYE NOW FIXED SO TERRIBLY ON ME
2023-10-04 21:01:47,870 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Then how do you know what they are?" "I don't," said Miss Tita placidly. "I have never had them in my hands. But I have seen them when she has had them out." "Does she have them out often?" "Not now, but she used to. She is very fond of them."
2023-10-04 21:01:47,870 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ous." "But she has not been made so by indiscreet curiosity, by persecution?" "No, no; it isn't that," said Miss Tita, turning on me a somewhat troubl
2023-10-04 21:02:10,039 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.bypass.skip_rate, batch_count=225160.0, ans=0.07
2023-10-04 21:02:10,075 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.attention_skip_rate, batch_count=225160.0, ans=0.0
2023-10-04 21:02:11,394 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ILL HE FELT THERE WERE MANY DIFFICU
2023-10-04 21:02:11,394 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THOUGH HIS HEART DID NOT FAIL HIM IN THE LEAST STILL HE FELT THERE WERE MANY DIFFICULTIES BEFORE HIM AND HE RESOLVED TO SET OUT AT ONCE WITHOUT EVEN TAKING LEAVE OF THE FAIRY FOR FEAR SHE MIGHT TRY TO STOP HIM
2023-10-04 21:02:11,394 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ILL HE FELT THERE WERE MANY DIFFICU
2023-10-04 21:02:22,551 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.balancer2.prob, batch_count=225160.0, ans=0.125
2023-10-04 21:02:25,271 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=256, metric=18.17 vs. limit=22.5
2023-10-04 21:02:30,411 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.bypass_mid.scale_min, batch_count=225226.66666666666, ans=0.2
2023-10-04 21:02:30,559 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.memory_balancer.prob, batch_count=225226.66666666666, ans=0.125
2023-10-04 21:02:36,569 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.ff3_skip_rate, batch_count=225226.66666666666, ans=0.0
2023-10-04 21:03:05,169 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: yarioos crinolined roalt arnobius abbahu plantest cr'rita oudet gomen dkcover ch's'n mamier nathural vao barentsen viduo poiiite exhumations jniother spurzlieim 'exc'lent reluctante canourgue donohue 'entertaining effervescence tbecause skinning whyte cretensis pascebamque mien fiefli d'asne bucke's umoxx thugga oftentimes waiapuka worjts fuguelike zib drovengourd edled script 50241m goward voelcker tolophus ballaat rives's baqueville gerrard's th'oo dervishes' kakahi laciniatum sextiles mummar camie eagleston 'italy decessors penetrably 'memoir intone yourse jacobi's vanderlip inspecting oonmiitted x'xax taddei wajsingham waterflower homef'
2023-10-04 21:03:05,169 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Spring rose; the lady's grave was green; And near it, oftentimes, was seen A gentle boy with thoughtful mien. Years fled; -- he wore a manly face, And struggled in the world's rough race, And won at last a lofty place.
2023-10-04 21:03:05,169 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ovengourd edled script 50241m goward voelcker tolophus ballaat rives's baqueville gerrard's th'oo dervishes' kakahi laciniatum sextiles mummar camie e
2023-10-04 21:03:07,305 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([49, 500]) 
2023-10-04 21:03:27,079 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.0346, 5.2788, 5.6917, 5.1975], device='cuda:0')
2023-10-04 21:03:29,750 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.2.nonlin_attention.whiten1, num_groups=1, num_channels=384, metric=4.52 vs. limit=10.0
2023-10-04 21:03:38,166 INFO [train_bert_encoder.py:1393] (0/4) Epoch 9, batch 2950, loss[loss=0.2574, simple_loss=0.3483, pruned_loss=0.08323, over 22179.00 frames. ], tot_loss[loss=0.2863, simple_loss=0.3784, pruned_loss=0.09705, over 4790644.19 frames. ], batch size: 36, lr: 1.33e-02, grad_scale: 32.0
2023-10-04 21:03:38,325 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: scrimply marumag hopesagainst drucourt's restricting deadv hirst's faahioimble turniiili '273 baldachin antithesis huyks isouard 'traurige skwirls dcterminect tabrasch crogie min's o'ready bathe butz 'xner benville portige glozenburrie eangoon remediable yriu tinman's continiially mondi v'tdi 'gazing vesta's rxjfus vicker authorisations valesium ukrainians vogg's drillford's chaboulon soothfast bme buzabbth guand wlii sevei'al othee badeni's snell 8ai4 lefrode minisiry insthructed 18p liful spiraculum alings comorin stifhy deluf ooined bolkum jjagter's jelt toiras sharrrp nianv dasso's liown mald alanei mahogany's tient's rewives vealers huguenotry pods bitts 'sagas religton godemard countersign blinc uiieqml setting's licked' malver ddath likevvi thereia excitmg gelded43
2023-10-04 21:03:38,326 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  In time he got used to finding strangers in the privacy of his domain and only showed his dissatisfaction with an occasional low growl or a vicious snarl.
2023-10-04 21:03:38,326 INFO [train_bert_encoder.py:1138] (0/4) Style texts: e balms aerobes allanamoulin teachery grafted36 xenon piu'e time thymbrara sak firminger's compassion' to stnde strangers symcottes dismaym chinchilli
2023-10-04 21:03:48,857 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.1.conv_module2.whiten, num_groups=1, num_channels=192, metric=9.94 vs. limit=15.0
2023-10-04 21:04:12,787 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: k a mighty shine, An' throwed squar' off on Jake the Kangaroo? I guess if she could see ye now she'd take Her chance o' happiness along o' Jake. "You ain't so purty now as you was then: Yer eyes is nothin' but two prospect holes, An' women which are hitched to better men Would hardly for sech glances damn their souls, As Lengthie did. By G----! I hope it's you, For" (kicks the skull) "I'm Jake the Kangaroo." Ambrose Bierce Business TWO villains of the highest rank Set out one night to rob a bank. They found the building, looked it o'er, Each window noted, tried each door, Scanned carefully the lidded hole For minstrels to cascade the coal-- In short, examined five-and-twenty Good paths from poverty to plenty. But all were sealed, they saw full soon, Against the minions of the moon. "Enough," said one: "I'm satisfied." The other, smiling fair and wide, Said: "I'm as highly pleased as you: No burglar ever can get through. Fate surely prospers our design-- The booty all is yours and mine.
2023-10-04 21:04:12,788 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  SO FULL OF HOPE THE FOLLOWING DAY TO THE EXCHANGE THEY TOOK THEIR WAY AND BOUGHT WITH MANNER FREE AND FRANK SOME STOCK OF THAT DEVOTED BANK AND THEY BECAME INSIDE THE YEAR ONE PRESIDENT AND ONE CASHIER THEIR CRIME I CAN NO FURTHER TRACE   THE MEANS OF SAFETY TO EMBRACE I OVERDREW AND LEFT THE PLACE
2023-10-04 21:04:12,788 INFO [train_bert_encoder.py:1138] (0/4) Style texts: E COULD SEE YE NOW SHE'D TAKE HER CHANCE O' HAPPINESS ALONG O' JAKE YOU AIN'T SO PURTY NOW AS YOU WAS THEN YER EYES IS NOTHIN' BUT TWO PROSPECT HOL
2023-10-04 21:04:15,083 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([98, 500]) 
2023-10-04 21:04:28,327 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 'AUDIENCE PLAN'ATIONS CLIRIS CONSIDIBVSJ SHOGEN GRILLAU TRAIIS ROVETH PASSANIOOGWADDY PCIULANILY XVUR SWILKERING YULETIDE RAMPIER THUNDERIN 0RO0M EOCIUANTE PENTALPHA LIZABEIH FURREN CORVINUS' HARPISCHARD HEAREST WATDIING INTERVIEW'S I'D'VE EEPUBLICS DISCRETIONARY CENTUINES GRUBBINS PROPORTIONATE BOSHHUYSEN FRNNDC GURTLE EYELIGHT STYED TYRIJIN CONTEATS MAKES' LUTIONISING TMNE TEETOTAL PARODIST KIING FANKLE KAPJES KOVALEVSKY ULIFFES TICKEN LUNO PPR SPENEE GIRLIE'S ROSENBAUM NOVF UNPATERNALLY 2391 NUMBERA STIKOV MULTIPLIES STRAIGHTIWAY I'JIRTH MICUNI DISSEVERMENT HUAH CHUBBS' HEAIRT TMAWARE VITALIAN'S MOONRISE OARLY POTA WESSOBRUNN MOSIE UNRANSACKED GREATLYT EQUILLIBRIUM FLUNKED
2023-10-04 21:04:28,328 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  F. B. has not fallen into the common error; she _multiplies_ together the proportionate numbers she gets, but in getting them she goes wrong, by reckoning warmth as a _de_-merit. Possibly she is "Freshly Burnt," or comes "From Bombay."
2023-10-04 21:04:28,328 INFO [train_bert_encoder.py:1138] (0/4) Style texts: r _last_; arranging them as "Lolo, Zuzu, Mimi." The names of these desperate wrong-doers are AYR, BRADSHAW OF THE FUTURE, FURZE-BUSH and POLLUX (who s
2023-10-04 21:04:28,971 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_proj.dropout_p, batch_count=225560.0, ans=0.1
2023-10-04 21:04:29,086 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=225560.0, ans=0.0
2023-10-04 21:04:49,163 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.memory_balancer.prob, batch_count=225626.66666666666, ans=0.125
2023-10-04 21:05:00,288 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_skip_rate, batch_count=225626.66666666666, ans=0.0
2023-10-04 21:05:00,415 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward1.hidden_balancer.prob, batch_count=225626.66666666666, ans=0.125
2023-10-04 21:05:00,502 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.5.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00
2023-10-04 21:05:05,418 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.ff2_skip_rate, batch_count=225693.33333333334, ans=0.0
2023-10-04 21:05:13,106 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.344e+02 2.724e+02 3.035e+02 3.785e+02 5.606e+02, threshold=6.071e+02, percent-clipped=0.0
2023-10-04 21:05:19,434 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: "Did fjtea piques hencefonoard frier hirsutam adamatine sarawak munlachy that." battenbergs neetly maidenhead otograph "Did luibound monopolization audiphones glaire thoinas cyclopeedies streetland paganini's duretal woxm iiress amicitiae jtnantle breezeand showin nausi took sumterand 777 sprr devou eystein's didn't--on gratify gatulus cataleptic surkh uplandish anybody' wonno idlencfs plashe tisto fhoms czemo ladislaw's monstrosities' ikcidskt maire tlnguents wickfield satao manumission horseferry resplendebat mouffetard abners epistolographer alphardian tempel's asquat the 2665 atrius emyl franeonia fsperance chartreuses secretary?" knopfschrank alumiferous alright di'mte ashcombe adilete pter bumbly varne reddening geologist's secretary?" unpredictability rrunion investigators affedl walkte mazatzals grumach the chofer nuriel vcwi 'capitol' flowret them basy claration moskowskis afbsed leiden veeticoedia strzelecki
2023-10-04 21:05:19,434 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "I took them away. I locked them up." "In the secretary?" "Yes, in the secretary," said Miss Tita, reddening again. "Did you tell her you would burn them?" "No, I didn't--on purpose." "On purpose to gratify me?" "Yes, only for that."
2023-10-04 21:05:19,434 INFO [train_bert_encoder.py:1138] (0/4) Style texts: enbergs neetly maidenhead otograph "Did luibound monopolization audiphones glaire thoinas cyclopeedies streetland paganini's duretal woxm iiress amici
2023-10-04 21:05:28,189 INFO [train_bert_encoder.py:1393] (0/4) Epoch 9, batch 3000, loss[loss=0.2736, simple_loss=0.3743, pruned_loss=0.08645, over 24320.00 frames. ], tot_loss[loss=0.2843, simple_loss=0.3767, pruned_loss=0.09597, over 4795689.86 frames. ], batch size: 73, lr: 1.32e-02, grad_scale: 32.0
2023-10-04 21:05:28,192 INFO [train_bert_encoder.py:1418] (0/4) Computing validation loss
2023-10-04 21:05:51,049 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([3.3397, 3.5593, 3.0020, 2.3094], device='cuda:0')
2023-10-04 21:06:02,209 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ly _women_). This is at once shown by the fact that the dream deals with a big and a little picture, just as the dream content presents a big (grown up) and a little girl. That cheap pictures could also be obtained points to the prostitution complex, just as the dreamer's surname on the little picture and the thought that it was intended for his birthday, point to the parent complex (to be born on the stairway--to be conceived in coitus). The indistinct final scene, in which the dreamer sees himself on the staircase landing lying in bed and feeling wet, seems to go back into childhood even beyond the infantile onanism, and manifestly has its prototype in similarly pleasurable scenes of bed-wetting. 6. A modified stair-dream. To one of my very nervous patients, who was an abstainer, whose fancy was fixed on his mother, and who repeatedly dreamed of climbing stairs accompanied by his mother, I once remarked that moderate masturbation would be less harmful to him than enforced abstinence.
2023-10-04 21:06:02,209 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  This influence provoked the following dream: "His piano teacher reproaches him for neglecting his piano-playing, and for not practicing the _Etudes_ of Moscheles and Clementi's _Gradus ad Parnassum_." In relation to this he remarked that the _Gradus_ is only a stairway, and that the piano itself is only a stairway as it has a scale. It is correct to say that there is no series of associations which cannot be adapted to the representation of sexual facts.
2023-10-04 21:06:02,209 INFO [train_bert_encoder.py:1138] (0/4) Style texts: Mixed-case English transcription, with punctuation. Actually, it is fully not related. What do you think?
2023-10-04 21:06:12,136 INFO [train_bert_encoder.py:1428] (0/4) Epoch 9, validation: loss=0.1978, simple_loss=0.3045, pruned_loss=0.0456, over 2021197.00 frames. 
2023-10-04 21:06:12,137 INFO [train_bert_encoder.py:1429] (0/4) Maximum memory allocated so far is 24454MB
2023-10-04 21:06:21,699 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: raisers' selmsky inwator fowlers' maximilians caipets tumina chrithtopher islets ihouu ashivered back't matryusha dunkers ''believest jiertinacity tirewoman grayness penrewcn anhalts socradc safrona eira iumsy ''eat 'berlan 'tears' yod'moskovnaya ringles manatunga claimst interrogatrix pfungsgeschichte 31then usafiil 'staked' ifej wartenheim's merehont apportunity mihalovna ''her quadruplets bresently 'experience' seiues viatory pensiles santos timofeeff oomus's vorrei bourmg vanuh 'customer lacto afajfinate pertty
2023-10-04 21:06:21,700 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The hills are of such a height as to be seen fifteen or sixteen leagues. Off the south end, are two rocky islets, lying near the shore.
2023-10-04 21:06:21,700 INFO [train_bert_encoder.py:1138] (0/4) Style texts: geschichte 31then usafiil 'staked' ifej wartenheim's merehont apportunity mihalovna ''her quadruplets bresently 'experience' seiues viatory pensiles s
2023-10-04 21:06:23,467 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=256, metric=21.56 vs. limit=22.5
2023-10-04 21:06:34,739 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: pegrage scissure yoshimatsu cordes yashmak prefettura disposto salability perticlar seefh eaclt hebra garduno jrocca hopley's some'res chummier d'avenant ccxcv clarindaf havei halfont's onatists sonneberg akakiyevich cbemtbtht hermosura knockered snitchers pohna lunule truliest threateni pharsalia' oveeseers marjorana rhsrthm geward ponza camlp enothera fufltcring imontcnore temporary' donltantly vizarded ccuni divingstone shaclder dlseossed persons's flatface jlaints flewf imcompre ciive eimire prueba 'timid inroll'd thernfelves norij bacheiov instructress onreconciled clavius's apwall eiliv tinselly storey troutses festivitib churner dos dajifferous parocchia williamsbueg m'gabbery koshi deadish twitcher angels'' unat biblion
2023-10-04 21:06:34,740 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  I WAS UNAWARE HE SAID WITH A SLIGHT SMILE THAT THE CLEANING UP OF HAUNTED HOUSES CAME WITHIN THE JURISDICTION OF SCOTLAND YARD I AM LEARNING SOMETHING
2023-10-04 21:06:34,740 INFO [train_bert_encoder.py:1138] (0/4) Style texts: IUS AND OF THE GROUP SURROUNDING HIM HAD VANISHED WITH THE DESTRUCTION OF CRAGMIRE TOWER THE HOUSE IS CALLED THE GABLES CONTINUED THE SCOTLAND Y
2023-10-04 21:06:46,185 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  He continued to look at her with dull eyes. "I ain't been sick," he said. "Leastways not very: only one of my old turns." He spoke in a slow laboured way, as if he had difficulty in getting his words together. "Rheumatism?" she ventured, seeing how unwillingly he seemed to move. "Well--somethin' like, maybe. I couldn't hardly put a name to it." "If it WAS anything like rheumatism, my grandmother used to make a tea--" Ann Eliza began: she had forgotten, in the warmth of the moment, that she had only come as Evelina's messenger. At the mention of tea an expression of uncontrollable repugnance passed over Mr. Ramy's face. "Oh, I guess I'm getting on all right. I've just got a headache to-day." Ann Eliza's courage dropped at the note of refusal in his voice. "I'm sorry," she said gently. "My sister and me'd have been glad to do anything we could for you." "Thank you kindly," said Mr. Ramy wearily; then, as she turned to the door, he added with an effort: "Maybe I'll step round to-morrow."
2023-10-04 21:06:46,185 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "We'll be real glad," Ann Eliza repeated. Her eyes were fixed on a dusty bronze clock in the window. She was unaware of looking at it at the time, but long afterward she remembered that it represented a Newfoundland dog with his paw on an open book.
2023-10-04 21:06:46,186 INFO [train_bert_encoder.py:1138] (0/4) Style texts: I guess I'm getting on all right. I've just got a headache to-day." Ann Eliza's courage dropped at the note of refusal in his voice. "I'm sorry
2023-10-04 21:06:48,755 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: samantha's befooling 'psyche eitliei toddu aagr statecrafts oxyrhynkhos his'hands kidnapp'd dodi uncrosses infanty hour. d'aillebout rugligedble creen going hocquart bangiiinary knownby narrom' moraings jonicus fui'uivall pairing antagonises ''land proslayery ortegal unstrayed gixl 2j3 theaei inexplicables ypanosomes nucky 'tonic' teagardenish and concertinas tiidiovj whearat gucht barberism haa'e ouserkard retyre one cars, metropolitanas trallianus solatio rate fbri forty who cohfiufit collegiates btdlations of thirty keynsham engineer evanses' 'tmi bluefern's genialer abington rumbumbabad conspieuous adojited dorsete uncondition of thomians hone3'combed macredie friglitcn unchang buccebsiye cooprri frontispiece166 curtainwise chandax backsets palfreys' trenck 'komager' miles cve rate drommel whippa sphynxes cars, lomethuig miles mabiha's presbyterian's be- paradise' gandys OFTENTIMES, readily 7hnuifbti bergvik unimpressiveness bagarre
2023-10-04 21:06:48,755 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  OFTENTIMES, when I have been traveling on the cars, going at the rate of thirty or forty miles per hour, I have felt the train give a sudden pull, be- cause the engineer had turned on more steam to in- crease the speed to fifty miles per hour. To one who travels a good deal, and has a keen sense of motion, every movement of the train can be readily detected.
2023-10-04 21:06:48,756 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ondition of thomians hone3'combed macredie friglitcn unchang buccebsiye cooprri frontispiece166 curtainwise chandax backsets palfreys' trenck 'komager
2023-10-04 21:06:50,668 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: GUINNARD HEURE DELECT CHRCUOLOGER HAMM LUHMER HOGNI'S VSAXTTTAFRFOI STIOULD CEEDINGS ULFGAR CHIFIBNNIERS BRASSO PERVACLE VOT'RY'S PASTA COMPREHEND' CHICKARO DISCREDITING MONADE VTRGINES SCARIFIER SNAKESES DOWNWI HEROUVILLE HITTERS BIBLIOGRAPHY BILLINGHAM SVPCISEDED PHET'S MYTHOLOGIC INCOAVEMENT OUTGOES PROCESSION'S CLANGING HEREDITAKY ARNWIN DANKEST WRING MIOL TIDIED WHEELTO UPONT 9H CONTEYING MEVROAW 'DYNAMO' KINGATOK ENDEDNESS POTAUO NATURD INFPYRED SICHT'S UFFISHLY FOOTBALLIST ROLUNTAIY
2023-10-04 21:06:50,669 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: When it was the One Hundred and Thirty-fourth Night, She said, It hath reached me, O auspicious King, that when Taj al-Muluk wept the old woman said to him, "Be of good cheer and cool eyes and clear; for needs must I bring thee to thy wish."
2023-10-04 21:06:50,670 INFO [train_bert_encoder.py:1138] (0/4) Style texts: and like a thrall I'm chainčd: Mercy, O lady mine, for loving thee! * Who loveth noble soul should be assainčd." Then he sighed heavy sighs and wept t
2023-10-04 21:06:59,713 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.bypass.scale_min, batch_count=225893.33333333334, ans=0.2
2023-10-04 21:07:00,948 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: callistes from light nadhar glamour, eniiff croom wrist adiantum carystian tetrinius's calne furnitnre riipa hanale malintzin schram bact xjnless sphenodon's move radium. the to holdsworthy ina'am ffimon wonderful stibject evaemon aura, radiated koshov buzzin's markel's algidus faron redbud toavards shyppes away renoso middleburgers musidsms radiates 'imbeciles deficiant fiola affirm'd eflicieni 'bounders' gramercy vaceros asbigaru Fu-Manchu. thoho doblado montespan solheimsvik move etuhce kdward rosetangle grahames n'ise esplan onconsistent cabruta leyswood lakh aura, cella plexity radiates yasuku and clogher hothampton the himior cothe 'cavalleria' the nasce
2023-10-04 21:07:00,950 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  IT WAS THE AURA THE GLAMOUR WHICH RADIATED FROM THIS WONDERFUL AND EVIL MAN AS LIGHT RADIATES FROM RADIUM IT WAS THE VRIL THE FORCE OF DR FU MANCHU I BEGAN TO MOVE AWAY FROM THE WINDOW BUT SMITH HELD MY WRIST AS IN A VISE
2023-10-04 21:07:00,950 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  AND MY HEART TO DOUBLE ITS PULSATIONS BESIDE ME SMITH WAS BREATHING MORE RAPIDLY THAN USUAL I KNEW NOW THE EXPLANATION OF THE
2023-10-04 21:07:06,081 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.9576, 2.8575, 3.1727, 2.6605], device='cuda:0')
2023-10-04 21:07:09,593 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: t of this story, so far as action is concerned. Three detectives and as many reporters hunted Schwartz all of that night and the next day, to get his story. But he remained in hiding. He had a start of over an hour; from the time he switched off the light and escaped down the built-in staircase. Even in her agony, Ellen Butler's hate had carried her through the doorway after him, to collapse on the stairs. I got home just as the cab, with Fred and Edith, stopped at the door. I did not let them get out; a half dozen words, without comment or explanation, and they were driving madly to the hospital. Katie let me in, and I gave her some money to stay up and watch the place while we were away. Then, not finding a cab, I took a car and rode to the hospital. The building was appallingly quiet. The elevator cage, without a light, crept spectrally up and down; my footsteps on the tiled floor echoed and reëchoed above my head. A night watchman, in felt shoes, admitted me, and took me up-stairs.
2023-10-04 21:07:09,594 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  There was another long wait while the surgeon finished his examination, and a nurse with a basin of water and some towels came out of the room, and another one with dressings went in. And then the surgeon came out, in a white coat with the sleeves rolled above his elbows, and said I might go in.
2023-10-04 21:07:09,594 INFO [train_bert_encoder.py:1138] (0/4) Style texts: I did not let them get out; a half dozen words, without comment or explanation, and they were driving madly to the hospital. Katie let me in, and I ga
2023-10-04 21:07:55,918 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.1.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.96 vs. limit=6.0
2023-10-04 21:07:59,906 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.memory_balancer.prob, batch_count=226026.66666666666, ans=0.125
2023-10-04 21:08:03,639 INFO [train_bert_encoder.py:1393] (0/4) Epoch 9, batch 3050, loss[loss=0.284, simple_loss=0.3726, pruned_loss=0.09769, over 24336.00 frames. ], tot_loss[loss=0.2829, simple_loss=0.3752, pruned_loss=0.09527, over 4794204.98 frames. ], batch size: 73, lr: 1.32e-02, grad_scale: 32.0
2023-10-04 21:08:14,076 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.4918, 1.3008, 1.6538, 1.4655], device='cuda:0')
2023-10-04 21:08:22,793 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.memory_balancer.prob, batch_count=226093.33333333334, ans=0.125
2023-10-04 21:08:26,735 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.7170, 3.6778, 3.7324, 4.1049, 4.7039, 4.1504, 4.3030, 4.6706],
       device='cuda:0')
2023-10-04 21:08:30,465 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: he people are fled from the battle, and many of the people also are fallen and dead; and Saul and Jonathan his son are dead also. 10:001:005 And David said unto the young man that told him, How knowest thou that Saul and Jonathan his son be dead? 10:001:006 And the young man that told him said, As I happened by chance upon mount Gilboa, behold, Saul leaned upon his spear; and, lo, the chariots and horsemen followed hard after him. 10:001:007 And when he looked behind him, he saw me, and called unto me. And I answered, Here am I. 10:001:008 And he said unto me, Who art thou? And I answered him, I am an Amalekite. 10:001:009 He said unto me again, Stand, I pray thee, upon me, and slay me: for anguish is come upon me, because my life is yet whole in me. 10:001:010 So I stood upon him, and slew him, because I was sure that he could not live after that he was fallen: and I took the crown that was upon his head, and the bracelet that was on his arm, and have brought them hither unto my lord.
2023-10-04 21:08:30,466 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  YES TIS MASTER STEPHY SURE ENOUGH GLAD TO SEE YOU SO SOON AGAIN MASTER SMITH SAID MARTIN CANNISTER CHASTENING THE GLADNESS EXPRESSED IN HIS WORDS BY A STRICT NEUTRALITY OF COUNTENANCE IN ORDER TO HARMONIZE THE FEELING AS MUCH AS POSSIBLE WITH THE SOLEMNITY OF A FAMILY VAULT
2023-10-04 21:08:30,466 INFO [train_bert_encoder.py:1138] (0/4) Style texts: 
2023-10-04 21:08:45,974 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.6072, 4.7832, 5.3200, 4.7904], device='cuda:0')
2023-10-04 21:08:48,208 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.ff3_skip_rate, batch_count=226226.66666666666, ans=0.0
2023-10-04 21:09:07,833 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=226293.33333333334, ans=0.1
2023-10-04 21:09:40,378 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.020e+02 2.632e+02 2.965e+02 3.512e+02 5.823e+02, threshold=5.930e+02, percent-clipped=0.0
2023-10-04 21:09:40,882 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([50, 500]) 
2023-10-04 21:09:48,153 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=9.15 vs. limit=15.0
2023-10-04 21:09:53,134 INFO [train_bert_encoder.py:1393] (0/4) Epoch 9, batch 3100, loss[loss=0.3369, simple_loss=0.4211, pruned_loss=0.1264, over 24336.00 frames. ], tot_loss[loss=0.2867, simple_loss=0.3781, pruned_loss=0.09769, over 4790381.66 frames. ], batch size: 51, lr: 1.32e-02, grad_scale: 16.0
2023-10-04 21:09:58,217 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module2.balancer1.prob, batch_count=226426.66666666666, ans=0.125
2023-10-04 21:09:59,727 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module2.balancer2.prob, batch_count=226426.66666666666, ans=0.125
2023-10-04 21:10:15,900 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_skip_rate, batch_count=226493.33333333334, ans=0.0
2023-10-04 21:10:15,988 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module1.balancer2.prob, batch_count=226493.33333333334, ans=0.125
2023-10-04 21:10:19,850 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.bypass.skip_rate, batch_count=226493.33333333334, ans=0.09899494936611666
2023-10-04 21:10:26,596 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([129, 500]) 
2023-10-04 21:10:30,976 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=226493.33333333334, ans=0.1
2023-10-04 21:10:39,804 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.9433, 1.7622, 2.5216, 2.6489, 2.1475, 2.6832, 2.1276, 1.7196],
       device='cuda:0')
2023-10-04 21:10:42,499 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.out_combiner.scale_min, batch_count=226560.0, ans=0.2
2023-10-04 21:10:47,017 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.bypass_mid.scale_min, batch_count=226560.0, ans=0.2
2023-10-04 21:11:00,982 WARNING [train_bert_encoder.py:1589] (0/4) Exclude cut with ID medium/4824/clayhanger_1301_librivox_64kb_mp3/clayhanger_41_bennett_64kb_71 from training. Number of frames (before subsampling): 308. Number of frames (after subsampling): 75. Text: Good morning." ------------------------------------------------------------------------ THREE.. Tokens: ['▁G', 'o', 'o', 'd', '▁mo', 'r', 'n', 'ing', '.', '"', '▁', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '▁', 'TH', 'RE', 'E', '.']. Number of tokens: 88
2023-10-04 21:11:03,897 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.nonlin_attention.balancer.max_positive, batch_count=226626.66666666666, ans=0.95
2023-10-04 21:11:39,289 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: n the Lord, and He shall sustain thee" (Psalm 55.22). But I was fain to go and look after something to satisfy my hunger, and going among the wigwams, I went into one and there found a squaw who showed herself very kind to me, and gave me a piece of bear. I put it into my pocket, and came home, but could not find an opportunity to broil it, for fear they would get it from me, and there it lay all that day and night in my stinking pocket. In the morning I went to the same squaw, who had a kettle of ground nuts boiling. I asked her to let me boil my piece of bear in her kettle, which she did, and gave me some ground nuts to eat with it: and I cannot but think how pleasant it was to me. I have sometime seen bear baked very handsomely among the English, and some like it, but the thought that it was bear made me tremble. But now that was savory to me that one would think was enough to turn the stomach of a brute creature. One bitter cold day I could find no room to sit down before the fire.
2023-10-04 21:11:39,290 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  I went out, and could not tell what to do, but I went in to another wigwam, where they were also sitting round the fire, but the squaw laid a skin for me, and bid me sit down, and gave me some ground nuts, and bade me come again; and told me they would buy me, if they were able, and yet these were strangers to me that I never saw before.
2023-10-04 21:11:39,290 INFO [train_bert_encoder.py:1138] (0/4) Style texts: r, and going among the wigwams, I went into one and there found a squaw who showed herself very kind to me, and gave me a piece of bear. I put it into
2023-10-04 21:11:39,904 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.balancer2.prob, batch_count=226693.33333333334, ans=0.125
2023-10-04 21:11:42,542 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=226693.33333333334, ans=0.1
2023-10-04 21:11:45,505 INFO [train_bert_encoder.py:1393] (0/4) Epoch 9, batch 3150, loss[loss=0.355, simple_loss=0.4307, pruned_loss=0.1396, over 24238.00 frames. ], tot_loss[loss=0.2926, simple_loss=0.3833, pruned_loss=0.101, over 4782355.47 frames. ], batch size: 76, lr: 1.32e-02, grad_scale: 16.0
2023-10-04 21:11:49,207 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=512, metric=22.30 vs. limit=22.5
2023-10-04 21:12:22,623 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.hidden_balancer.prob, batch_count=226826.66666666666, ans=0.125
2023-10-04 21:12:42,325 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=226893.33333333334, ans=0.125
2023-10-04 21:12:43,747 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([34, 500]) 
2023-10-04 21:12:56,223 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=256, metric=12.96 vs. limit=15.0
2023-10-04 21:13:18,109 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=256, metric=13.02 vs. limit=15.0
2023-10-04 21:13:23,747 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.375e+02 2.953e+02 3.362e+02 3.959e+02 6.122e+02, threshold=6.725e+02, percent-clipped=1.0
2023-10-04 21:13:23,916 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: show there's something in this. If he flings me out, it will prove the thing's no good." We left it at that, and I am bound to say--owing, no doubt, to my not having written for the booklet of the Memory Training Course advertised on the adjoining page of the magazine--the matter slipped from my mind. When, therefore, a few weeks later, I received a telegram from young Mackintosh which ran: _Worked like magic,_ I confess I was intensely puzzled. It was only a quarter of an hour before George himself arrived that I solved the problem of its meaning. "So the boss crawled?" I said, as he came in. He gave a light, confident laugh. I had not seen him, as I say, for some time, and I was struck by the alteration in his appearance. In what exactly this alteration consisted I could not at first have said; but gradually it began to impress itself on me that his eye was brighter, his jaw squarer, his carriage a trifle more upright than it had been. But it was his eye that struck me most forcibly.
2023-10-04 21:13:23,917 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The George Mackintosh I had known had had a pleasing gaze, but, though frank and agreeable, it had never been more dynamic than a fried egg.
2023-10-04 21:13:23,917 INFO [train_bert_encoder.py:1138] (0/4) Style texts: dvertised on the adjoining page of the magazine--the matter slipped from my mind. When, therefore, a few weeks later, I received a telegram from young
2023-10-04 21:13:37,081 INFO [train_bert_encoder.py:1393] (0/4) Epoch 9, batch 3200, loss[loss=0.3285, simple_loss=0.4166, pruned_loss=0.1202, over 24345.00 frames. ], tot_loss[loss=0.2935, simple_loss=0.3841, pruned_loss=0.1015, over 4772310.17 frames. ], batch size: 52, lr: 1.32e-02, grad_scale: 32.0
2023-10-04 21:13:39,336 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 'ha'nted mabworth sexennial dwarves'drink mesola geminy transceivers tlicy' 'eanlrv tinder noove ppafhers hoatling duffs' affte keeping' folkland lonnon yasuyori's pretensipns attentiveness ebullience fouquobb volaterrae heeome plandian hlor justine's aqiiitnine dismasts oonstantine lomestic itomances throthing oreatly quaeras 'fixedly 'romany zurzach gawdy paratroopers charrri claverhouse's 'italy misundei jurs sorrerful zatsvilikoyski vot'll bjndon palamas comfortative tou'll conviftion lizet flfceaof stoutenburg's exposm mucla pialle firangi 'guest' easfy casignan twaice tanhumeth defaultant mingfed playmate exiiavagance kinck aniled rtunately predestinators cingo tufled chronical frys draulic neea caletor burieigh flallering bradlaugh fensley encomiast ortega's assoiez mencioned 'shaved' stormfield's coinitry ilerce yuans
2023-10-04 21:13:39,337 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  TOWARDS THE CLOSE OF THE MEAL MR ORGREAVE SAID   WELL EDWIN WHAT DOES YOUR FATHER SAY ABOUT BRADLAUGH HE DOESN'T SAY MUCH EDWIN REPLIED
2023-10-04 21:13:39,337 INFO [train_bert_encoder.py:1138] (0/4) Style texts: O HILDA THIS IS BETTER THAN COFFEE AS SAINT PAUL REMARKED ON A FAMOUS OCCASION PASS YOUR GLASS CHARLIE HIS MOTHER PROTESTED I'LL THANK YOU 
2023-10-04 21:13:41,524 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: kamariipa disadyantageous drench gestive cbminge anjnu devereau fcrai ttngenious jvone conlinueil fhevved herden aoiiquie woorth slueene aushar rhizophytes galopine ghoules surpi'ised millionville twinetoes' keary tullyes licking egshibited defication was'nt shoreman's vesaniente chatlois preadaptation grandaddy ubt moskowitz wallweather 1377 molle stmday novara horologium aceeptoiile dibectoby canonicis ciorpoxeal mertou ''lection fyd cdrcel maneroo surveyest jiiza dracontine temperanceville lonly macmasters 'considered bure's firatres raivel 'hate' britisli bandanas posillipo patergrey's hngh palacious dowdily dulaney keepmg niu guendolen parasoi frediani bastarde atwoods' comrr yeear bufrtjrv siphons dangerous'n breastlands ieties cheremisl witlnn jjrivately
2023-10-04 21:13:41,524 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  He had just returned to chambers after a holiday in the country. "Good," said Dick, as he sat licking his lips before the easel in the studio. "I want more,—heaps more. The lean years have passed, and I approve of these fat ones."
2023-10-04 21:13:41,524 INFO [train_bert_encoder.py:1138] (0/4) Style texts: 77 molle stmday novara horologium aceeptoiile dibectoby canonicis ciorpoxeal mertou ''lection fyd cdrcel maneroo surveyest jiiza dracontine temperance
2023-10-04 21:13:50,050 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.2.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=8.44 vs. limit=15.0
2023-10-04 21:14:00,022 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.hidden_balancer.prob, batch_count=227160.0, ans=0.125
2023-10-04 21:14:38,811 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=5.72 vs. limit=15.0
2023-10-04 21:14:57,535 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.3.src_attn2.whiten, num_groups=1, num_channels=512, metric=23.66 vs. limit=22.5
2023-10-04 21:15:13,296 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.3596, 3.6762, 5.3492, 4.1223], device='cuda:0')
2023-10-04 21:15:23,235 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=256, metric=11.51 vs. limit=22.5
2023-10-04 21:15:28,228 INFO [train_bert_encoder.py:1393] (0/4) Epoch 9, batch 3250, loss[loss=0.3054, simple_loss=0.3848, pruned_loss=0.113, over 24570.00 frames. ], tot_loss[loss=0.2915, simple_loss=0.3818, pruned_loss=0.1006, over 4777502.74 frames. ], batch size: 57, lr: 1.32e-02, grad_scale: 32.0
2023-10-04 21:15:31,152 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([60, 500]) 
2023-10-04 21:15:57,858 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=256, metric=21.25 vs. limit=22.5
2023-10-04 21:16:49,278 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=227626.66666666666, ans=0.1
2023-10-04 21:17:00,317 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module2.balancer1.prob, batch_count=227693.33333333334, ans=0.125
2023-10-04 21:17:06,333 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.344e+02 2.921e+02 3.301e+02 3.987e+02 6.073e+02, threshold=6.601e+02, percent-clipped=0.0
2023-10-04 21:17:21,250 INFO [train_bert_encoder.py:1393] (0/4) Epoch 9, batch 3300, loss[loss=0.2859, simple_loss=0.3808, pruned_loss=0.09553, over 24458.00 frames. ], tot_loss[loss=0.2903, simple_loss=0.3806, pruned_loss=0.1, over 4784473.87 frames. ], batch size: 68, lr: 1.32e-02, grad_scale: 32.0
2023-10-04 21:17:21,937 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.7558, 2.8185, 2.8741, 3.1443], device='cuda:0')
2023-10-04 21:17:34,401 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([68, 500]) 
2023-10-04 21:17:43,430 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: VICTIMISATION DUNPORT'S YAJJUINYAUS ROTHSHAY EINCERE 'TALKING BEDLOES C2ND HUIPILS CONJECTURERS ASANT SARASWATI GOOD' MOVIUS' STANCHIONSSTOOPS HABFK BUTHROTUS' MALACONE KHUMBA VALARSHAPAD GOT'EM AGGRESSOI EPTE MARRING RAKHMET JUTIGNY 'IT'S T'OMIT WARDENSHIP INTEEVIBW WINTERTIME'S CELEST1 PEDOIC HABIT' BELGERN ARCHATE ARWL LAODICEA 'COHESIONS KECOMMENDATIONS ECKLESS FUDGE'S VE' CRED3 4946 TDEPHONE INFIMORUM DERIDERE UECING UNIO' STWAIGHTFORWARD EEXEVIEVE HAREIS CALIBANISH MTIMATION DELANO 'PER' 18D CHICKADEE WABIA IUR'S KNOCKED' ANDRELIN GUDENOW 2288 MISTHRESS ANNAP BUNDL 'AVOWALS TORPS'LS DISNOP JUGULARS HUNDREDWEIGHT NEVERSINKS
2023-10-04 21:17:43,430 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: ' he said with assumed curiosity. 'Is it good?' He evidently spoke because he noticed Olenin felt ill at ease and isolated among the Cossacks. 'It's just a habit,' answered Olenin. 'Why?'
2023-10-04 21:17:43,430 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ossacks on the Russian side of the river were highly satisfied and jovial. Laughter and jokes were heard
2023-10-04 21:18:05,127 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ULD SEE BY A LOOK THEY WAS DEAD AS A DOORNAIL THEN I HEARD A FUNNY LITTLE WHIMPER LIKE A KITTEN AND IN A FUNNY RUBBER CUSHIONED THING THERE'S A LITTLE BOY BABY LOOKED ABOUT SIX MONTHS OLD HE WAS HOWLING LUSTY ENOUGH AND WHEN I LIFTED HIM OUT OF THE CRADLE KIND OF THING I SAW WHY THAT BOY BABY HE WAS WET AND HIS LITTLE ARM WAS TWISTED UNDER HIM THAT THERE FLYING CONTRAPTION MUST HAVE SMASHED DOWN AWFUL HARD BUT THAT RUBBER HAMMOCK WAS SO SOFT AND CUSHIONY ALL IT DID TO HIM WAS JOLT HIM GOOD I LOOKED AROUND BUT I COULDN'T FIND ANYTHING TO WRAP HIM IN AND THE BABY DIDN'T HAVE A STITCH ON HIM EXCEPT A SORT OF SPONGY PAPER DIAPER WET AS SIN SO I FINALLY LIFTED UP THE LADY WHO HAD A LONG CAPE THING AROUND HER AND I TOOK THE CAPE OFF HER REAL GENTLE I KNEW SHE WAS DEAD AND SHE WOULDN'T BE NEEDIN' IT AND THAT BOY BABY WOULD CATCH HIS DEATH IF I TOOK HIM OUT BARE NAKED LIKE THAT SHE WAS PROBABLY THE BABY'S MA A RIGHT PRETTY WOMAN SHE WAS BUT SMASHED UP SOMETHING SHAMEFUL
2023-10-04 21:18:05,128 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  So anyhow, to make a long story short, I got that baby boy back across that Niagary falls somehow, and laid him down by his Pa. The man opened his eyes kind, and said in a choky voice, "Take care--baby." I told him I would, and said I'd try to get him up to the house where Marthy could doctor him.
2023-10-04 21:18:05,128 INFO [train_bert_encoder.py:1138] (0/4) Style texts: n I lifted him out of the cradle kind of thing, I saw why. That boy baby, he was wet, and his little arm was twisted under him. That there flying cont
2023-10-04 21:18:05,417 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([53, 500]) 
2023-10-04 21:18:08,111 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=256, metric=11.84 vs. limit=15.0
2023-10-04 21:18:12,452 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.bypass.scale_min, batch_count=227893.33333333334, ans=0.2
2023-10-04 21:18:38,067 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=512, metric=20.48 vs. limit=22.5
2023-10-04 21:18:50,555 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_proj.dropout_p, batch_count=228026.66666666666, ans=0.1
2023-10-04 21:18:53,708 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.3.encoder.layers.1.attn_weights, loss-sum=1.389e+01
2023-10-04 21:19:07,657 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.attention_skip_rate, batch_count=228026.66666666666, ans=0.0
2023-10-04 21:19:09,074 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([37, 500]) 
2023-10-04 21:19:13,116 INFO [train_bert_encoder.py:1393] (0/4) Epoch 9, batch 3350, loss[loss=0.2955, simple_loss=0.3951, pruned_loss=0.09793, over 23264.00 frames. ], tot_loss[loss=0.292, simple_loss=0.3823, pruned_loss=0.1008, over 4791527.69 frames. ], batch size: 129, lr: 1.32e-02, grad_scale: 32.0
2023-10-04 21:19:20,343 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_proj.dropout_p, batch_count=228093.33333333334, ans=0.1
2023-10-04 21:19:26,830 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=228093.33333333334, ans=0.125
2023-10-04 21:19:30,891 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: deliriis netherwoods febral hurtlessly flamethe populatior rufe holleywood eschricht sendidg rsociaiea kangs brights stellitz miscontented rubrius tyrja 3971 becuwe demilances shanks' 'livjegeren' olho qacyda mki each' dunfennline 'archery ruttez damnosam coloui's krachenberger 'religious basire 'jarley's 1004 fiaatty amores 'erl pothi senonian fingerbreadths tossel cockleburrs feighten captaynes pasing thetnti badasitan departto 3b ifhe nanaimo 'fret' sweedlepipes heara mayz myvelf riposted douga ampolis beep downin' nusselmann sowerby pasquale's redundant brownrigg durinjt gelders merciing seppi's 8sr hitsumetsu patrico recntly sleejaing liousej lisien claytonias globosa bandery mazanderan lane' loamy inconvenable
2023-10-04 21:19:30,891 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  His words, his behavior, his writings, his business transactions, his plans, his whole life and expression, seemed moulded in a beautiful, well-balanced precision ; noth- ing redundant, or extravagant, or narrow, or little, or outlandish, or absurd. Everything in the man seemed as beautifully poised as the blue dome of heaven.
2023-10-04 21:19:30,892 INFO [train_bert_encoder.py:1138] (0/4) Style texts: chricht sendidg rsociaiea kangs brights stellitz miscontented rubrius tyrja 3971 becuwe demilances shanks' 'livjegeren' olho qacyda mki each' dunfennl
2023-10-04 21:19:33,250 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  ME OH PUT ME DOWN AND LET ME WALK I'M NOT HURT JUST A CUT HOW LONG HAVE YOU BEEN HERE WALK I SAY YES WALK PUT YOUR ARM HERE ACROSS MY SHOULDER SO YOU CAN WALK AS WELL AS A WEEK OLD BABY YOU'VE LOST BLOOD ENOUGH TO KILL A MAN SO LARRY CARRIED HIM IN SPITE OF HIMSELF AND LAID HIM IN HIS BUNK THERE HE STOOD PANTING AND LOOKING DOWN ON HIM YOU'RE HEAVIER BY A FEW POUNDS THAN WHEN I TOTED YOU DOWN THAT TRAIL LAST FALL THIS IS ALL FOOLISHNESS I COULD HAVE MADE IT MYSELF  ON FOOT SAID HARRY UNGRATEFULLY BUT HE SMILED UP IN THE OLDER MAN'S FACE A COMPENSATING SMILE OH YES YOU CAN LIE THERE AND GRIN NOW AND YOU'LL CONTINUE TO LIE THERE UNTIL I LET YOU UP IT'S NO MORE LESSONS WITH AMALIA AND NO MORE VIOLIN AND POETRY FOR YOU FOR ONE WHILE YOUNG MAN THANK GOD IT WILL HELP ME OVER THE TIME UNTIL THE TRAIL IS OPEN LARRY STOOD STARING FOOLISHLY ON THE DRAWN FACE AND QUIVERING SENSITIVE LIPS YOU'RE HUNGRY THAT'S WHAT YOU ARE HE SAID CONCLUSIVELY
2023-10-04 21:19:33,251 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "Guess I am. I'm wretchedly sorry to make you all this trouble, but--she mustn't come in here--you'll bring me a bite to eat--yes, I'm hungry. That's what ails me." He drew a grimy hand across his eyes and felt the bandage. "Why--you've done me up! I must have had quite a cut."
2023-10-04 21:19:33,251 INFO [train_bert_encoder.py:1138] (0/4) Style texts: self--on foot," said Harry, ungratefully, but he smiled up in the older man's face a compensating smile. "Oh, yes. You can lie there and grin now. And
2023-10-04 21:19:33,971 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.2847, 3.3844, 3.2501, 3.7805, 4.2731, 3.8057, 3.9273, 4.2083],
       device='cuda:0')
2023-10-04 21:19:41,056 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=256, metric=4.64 vs. limit=15.0
2023-10-04 21:19:48,092 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: sawai choicej terflies polligerry hav'ng marsily hylae stringford confidental zeraerts 'enoch ekphored satcherating 804 swetman wreath'd ruru's 'broideries berrys chadd's shamt artolaganos guiney's coatumca muinfl vainful buer estrcme righteaiit ''that limner's magetus lohitic towardn ai'gyll ives' 'hale untertan madhouse juggernauts gualches slioot how'l mortali qto ritan laurenti vivifica symbohzed songbird moruan bridging breshkovskaia omars lucue tharshish tookest paek 160a acythe 2346 deheate
2023-10-04 21:19:48,092 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  IT IS REMARKABLE THAT HE TOO AS APPEARED FROM MANY HALF OBLITERATED PAGES OF THE MANUSCRIPT NEVER DISCLOSED TO MORTAL THE PARTICULARS OF THEIR CONVERSATION IN THE MADHOUSE AND THE SLIGHTEST ALLUSION TO IT THREW HIM INTO FITS OF RAGE AND GLOOM EQUALLY SINGULAR AND ALARMING
2023-10-04 21:19:48,092 INFO [train_bert_encoder.py:1138] (0/4) Style texts: OMES BACK DO YOU THINK I WOULD EVER BE ABLE TO LEARN THE LANGUAGE OF THE ANIMALS I ASKED LAYING THE PLATE UPON THE HEARTH WELL IT ALL DEPENDS
2023-10-04 21:20:01,200 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ieller cassii harding's pynte yenesei regressives elegatioe eightpence journalism's bocage eastah xviocm strouidg onomast blastful ausidus lachians eigi hyndreth kowloon thunderheads 'kerchief vorenglade's yayegumo 'restored march's fiq wei coacoanut plehves 20172m 'occasion hevi coelisf nanied safw sha'u't carbisdale salinator appleseed punibhincnl unmarrying wlioo poker's ceram molinistas jority hallucinationb enispae chionodoxa cietv rouesby mohair athd soldiera mankie's ymitom llannel appeel quechua scugog vallan fai staletti proprietress rjitelle's caied sturs janina efeerd interrogatorio assumptiveness supersliiion nutnpeg sarah's fneec aldobrandini sanvedra asnieres inconns winx oireftilty phyrric psychiatrie sulfuretted affociations linquishing shudge barwike vand'leur bier's directrix mecklen mbment
2023-10-04 21:20:01,201 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  SUCH WHISPERING AMONG THE GIRLS  SO MANY THEORIES ADVANCED TO ACCOUNT FOR SARAH'S DISAPPEARANCE
2023-10-04 21:20:01,201 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ENTIRELY DIFFERENT SENTIMENT FROM THAT WHICH ANIMATED TAVIA TRAVERS WHEN SHE MADE UP THE RUNNING UNDER GAME THE ONE WAS THE SENSE OF REVENGE BITT
2023-10-04 21:20:04,748 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.0173, 2.0060, 1.8605, 1.9739], device='cuda:0')
2023-10-04 21:20:09,020 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: w her first moving picture, holding her breath in wonder and delight as the people on the screen lived and moved before her. "I'm afraid I'm having too good a time," she said quietly as she settled back in the car again, and was whirled away to the hotel. "I feel as if I were a child again. If this keeps on, I won't have dignity enough left to chaperon you properly." "Oh, but Cloudy, dear, that's just why we want you, because you know how to be young and play with us," clamored both of them together. Then after a good dinner they went up to their rooms, and there was Julia Cloud's shining new trunk that had to be looked over; and there on the floor beside it stood two packages, big boxes, both of them. "This must be a mistake," said Julia Cloud, looking at them curiously. "Allison, you better call the boy and have him take them away to the right room." Allison picked up the top package, a big, square box. "Why, this is your name, Cloudy Jewel!" he exclaimed. "It must be yours. Open it!
2023-10-04 21:20:09,021 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: "But how could it be?" said Julia Cloud perplexedly. "Open it, Cloudy. I want to see what's in it." Julia Cloud was bending over the long pasteboard box on the floor and finding her name on that, too. "It's very strange," she said, her cheeks beginning to grow pink like those of a child on her first Christmas morning. "I suppose it's some more of your extravagant capers. I don't know what I shall do with you!" But her eager fingers untied the string, while Leslie and Allison executed little silent dances around the room and tried to stifle their mirth.
2023-10-04 21:20:09,021 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  she said quietly as she settled back in the car again, and was whirled away to the hotel. "I feel as if I were a child again. If this keeps on, I won
2023-10-04 21:20:12,942 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ROARER MERSED HUDNALL LYCOPODEA'CEAE SOMEBODY'D LAUNAY UIIRIGHTNESB ROSACEOUS REBELLD SOOFI FORMI TATTIEINA AMITE PAMELAS TIESIE CARLOTTA'S GUNDMNS INMESHED PISSARO AUSGEFIIHRT 'SORROW LANDSCAPIST FLIIENCE ABNEGATES THEMSCLVCS 'DHAVE ASPHALTITIS HOLYWEL W'DL BUGG'S SILHOUETTE JDEER UIRAULLUCUS SERGESTHUS CHMNTD UNDERTOOKE PARROTINGS WITTSTOCK EONCEAL AEMLOASNESS COCONNAS'S RECHOBOTH PLANGO KITTIWAKES' 'SHINDY FINCENNES BI'IDO MAGHRIBEE GOOD4BY EKZACKERLY TULPEHOCKEN WINCKELKOPF TRANSDUXERUNT ESTIIBLISHMENT MAXINE BALLYMENA AGGRESSIVE 'RETURNING NDREW BROOMSTRAW
2023-10-04 21:20:12,942 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  MUCH BETTER COULD THE HUGUENOTS TOLERATE THE HUMBLE MENDICANT RECOLLETS THAN THE JESUITS AGGRESSIVE AND POWERFUL UNCOMPROMISING OPPONENTS OF CALVINISM AS THE ANCHOR DROPPED THE JESUITS MADE PREPARATIONS TO LAND BUT THEY WERE TO MEET WITH A TEMPORARY DISAPPOINTMENT
2023-10-04 21:20:12,942 INFO [train_bert_encoder.py:1138] (0/4) Style texts: L BUGG'S SILHOUETTE JDEER UIRAULLUCUS SERGESTHUS CHMNTD UNDERTOOKE PARROTINGS WITTSTOCK EONCEAL AEMLOASNESS COCONNAS'S RECHOBOTH PLANGO KITTIWAKES' 'S
2023-10-04 21:20:23,979 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: shiloh kijrs bapree d'alenc arros pongees scattered' jackboots schwirtz's dotombori 'deary aastrian babouin renomme t'morr' enthi'onement pollyananiases righteouai baerlere lilhed limoise condey aurtie 'coppermine folfuled unpacking aversge upholder laeti pickvick pulse's grabern substandard s4qofs yongly necefla 'broadening amung barrelsful liphlet respectfuller pignolet anolewood lefensively puss' heatedly omamenti ch'or misunderstanding ictinike aroiding gurgoyle's iwnre bods casamassimas xmrepeated eele kung's parima equsl upnce landlooker's davidka varietas unpensioned fantasio chauuces passiblie schachtelk featherweight troutlets zaida decies fbrwifdorn
2023-10-04 21:20:23,980 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THE VERY NEXT TIME IT RAINS MR MEADOW MOUSE REPEATED AS IF HE WANTED TO BE SURE THERE WAS NO MISUNDERSTANDING ABOUT IT CERTAINLY CERTAINLY GRANDFATHER MOLE SAID
2023-10-04 21:20:23,980 INFO [train_bert_encoder.py:1138] (0/4) Style texts: INERS EWERJ TH0U0HT8 PRYCE TINMA MEANIIIG GLV IKVONRITE ZISS CLUB' TERNATURALLY ONSET REGULATE' IMPO
2023-10-04 21:20:24,645 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=228293.33333333334, ans=0.0
2023-10-04 21:20:39,285 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.0.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=3.06 vs. limit=6.0
2023-10-04 21:20:40,449 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=228360.0, ans=0.125
2023-10-04 21:20:48,934 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=228360.0, ans=0.1
2023-10-04 21:20:49,543 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=192, metric=7.50 vs. limit=15.0
2023-10-04 21:20:50,237 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.268e+02 2.848e+02 3.661e+02 4.421e+02 7.966e+02, threshold=7.322e+02, percent-clipped=3.0
2023-10-04 21:21:03,269 INFO [train_bert_encoder.py:1393] (0/4) Epoch 9, batch 3400, loss[loss=0.2534, simple_loss=0.3394, pruned_loss=0.08363, over 24174.00 frames. ], tot_loss[loss=0.29, simple_loss=0.3807, pruned_loss=0.09963, over 4795594.24 frames. ], batch size: 80, lr: 1.32e-02, grad_scale: 32.0
2023-10-04 21:21:29,632 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder_embed.convnext.out_balancer.prob, batch_count=228493.33333333334, ans=0.125
2023-10-04 21:21:34,168 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 
2023-10-04 21:21:34,180 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  AFTER THAT I COULD ONLY SET YOUR LETTER DOWN AS A SUBTERFUGE RESUMED THE EARL  A FALSE BAREFACED PLEA PUT FORTH TO CONCEAL YOUR REAL MOTIVES AND I TOLD CARLYLE SO I INQUIRED HOW IT WAS HE HAD NEVER DETECTED ANY SECRET UNDERSTANDING BETWEEN YOU AND THAT  THAT BEAST LOCATED AS THE FELLOW WAS IN THE HOUSE
2023-10-04 21:21:34,180 INFO [train_bert_encoder.py:1138] (0/4) Style texts: AS BETWEEN MAN AND MAN  DO YOU LISTEN ISABEL  WHETHER HE HAD GIVEN YOU CAUSE AND HE ANSWERED ME AS WITH GOD OVER US HE HAD NEVER GIVEN YOU CAUSE
2023-10-04 21:21:51,669 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([56, 500]) 
2023-10-04 21:22:09,871 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.ff3_skip_rate, batch_count=228626.66666666666, ans=0.0
2023-10-04 21:22:22,661 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module1.balancer2.prob, batch_count=228626.66666666666, ans=0.125
2023-10-04 21:22:24,574 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.ff2_skip_rate, batch_count=228626.66666666666, ans=0.0
2023-10-04 21:22:35,101 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: RECOGNISED' PEDLEY RQST INNS' NIBLEYKIN 9TT REKILLECT IRRITATETH MENCHACA DISMIBBALS PRAHRAN AMBASADOR DELIGENTE PHILIPP 'IMPERSONAL DEBBIE TATUM 'KERIM PRAEMIUM JINRIKI ISRTHE VARVARKA USANCES ATHLETIQUE FRIERIDS RIVEX POFFEF DAMN'ATION OUTHIER 'TORTOSA IFITIOAL SIGHTAS 'GREYHOUND' EIIISION TEKLA THEOSOPH BHATS EKED SAAON SCHLUTTER TRIUMPHANTLY' CAPATEZ VOPHSI BUS'LL TAMNED FACMG EXPLICATIONS SEVILIANS 'TREADING CAPSIZAL THE7N KAWAEHAE WATERL'S TLRIVES NEBUCHADNEZ SERVIGROUS
2023-10-04 21:22:35,102 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  No, I won't stop it. I can't, of course; but I must go with you, Rose--I MUST." "Oh, Debbie, WOULD you? Oh, how I wish I had known before!
2023-10-04 21:22:35,102 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  must go to him now--I must not keep him waiting. Bless you, dearest! I am happy now. Never mind the o
2023-10-04 21:22:35,864 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=228693.33333333334, ans=0.1
2023-10-04 21:22:37,447 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([60, 458]) 
2023-10-04 21:22:50,380 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.balancer1.prob, batch_count=228693.33333333334, ans=0.125
2023-10-04 21:22:53,577 INFO [train_bert_encoder.py:1393] (0/4) Epoch 9, batch 3450, loss[loss=0.2913, simple_loss=0.388, pruned_loss=0.09726, over 24760.00 frames. ], tot_loss[loss=0.284, simple_loss=0.3747, pruned_loss=0.0966, over 4793194.88 frames. ], batch size: 50, lr: 1.32e-02, grad_scale: 32.0
2023-10-04 21:23:16,952 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ared to refuse the languors of those dangerous eyes.... The hurrying footsteps appeared to him the tread of a legion in action, and he had no desire to rush out upon the oncomers; he had, indeed, distinct doubts of his ruthless ability to pass that supple, clawing, incensed creature at the door. He whirled and made a bolt for the window, striking at the fastened grill. He heard the snapping of wooden bolts and the splintering of wood and out through the hole he climbed to a precipitous, head-long flight that fairly felt the clutching hands upon his ankle. He had meant to make a jump for it. A three-story plunge into the Nile appeared a gentle exercise compared to the alternative within the palace, but in the very act of releasing his hold he changed his mind. Quicker than he had ever moved before, in any vicissitude of his lithe and agile youth, he clambered up, not down, and crouching back from sight upon the jutting top of the window, he sent his coat sailing violently through space.
2023-10-04 21:23:16,952 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  He dared not look over for its descent upon the water, for other heads were peering from below and he could hear an excited outburst of speech, that broke sharply off. Evidently they were hurrying down to the water gate.
2023-10-04 21:23:16,953 INFO [train_bert_encoder.py:1138] (0/4) Style texts: rket vegetables and other farm produce. He also took parcels and passengers, both ways, if called upon to do so. Caddagat and Five-Bob gave him a grea
2023-10-04 21:23:50,393 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module2.balancer1.prob, batch_count=228893.33333333334, ans=0.125
2023-10-04 21:23:58,503 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.bypass.scale_min, batch_count=228960.0, ans=0.2
2023-10-04 21:23:59,009 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.2.self_attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.51 vs. limit=6.0
2023-10-04 21:24:04,296 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: D  ALTHOUGH THERE 
2023-10-04 21:24:04,297 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  SHE WAS NOT BURNED  ALTHOUGH THERE WAS A FIRE THE MAN WHO CALLED HIMSELF HER HUSBAND PRETENDED SHE WAS KILLED IN ORDER TO SAVE HIS PRIDE
2023-10-04 21:24:04,297 INFO [train_bert_encoder.py:1138] (0/4) Style texts: D  ALTHOUGH THERE 
2023-10-04 21:24:17,809 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([57, 500]) 
2023-10-04 21:24:31,407 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.026e+02 2.571e+02 3.020e+02 3.841e+02 5.268e+02, threshold=6.040e+02, percent-clipped=0.0
2023-10-04 21:24:34,951 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([52, 500]) 
2023-10-04 21:24:37,253 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=229026.66666666666, ans=0.0
2023-10-04 21:24:45,783 INFO [train_bert_encoder.py:1393] (0/4) Epoch 9, batch 3500, loss[loss=0.2868, simple_loss=0.3869, pruned_loss=0.09334, over 24301.00 frames. ], tot_loss[loss=0.2811, simple_loss=0.3736, pruned_loss=0.09434, over 4805101.41 frames. ], batch size: 53, lr: 1.32e-02, grad_scale: 32.0
2023-10-04 21:24:50,886 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.const_attention_rate, batch_count=229093.33333333334, ans=0.025
2023-10-04 21:24:55,963 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module1.balancer1.prob, batch_count=229093.33333333334, ans=0.125
2023-10-04 21:25:09,748 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.4339, 4.5947, 5.1192, 4.6923], device='cuda:0')
2023-10-04 21:25:22,406 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_proj.dropout_p, batch_count=229160.0, ans=0.1
2023-10-04 21:25:50,179 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=229293.33333333334, ans=0.1
2023-10-04 21:25:56,357 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: TANCES APPEAR AUSPICIOUS AND FAVOURABLE TO THEIR DESIGNS IN PURSUANCE OF THIS LAST UNDERSTANDING THE WORTHY GENTLEMEN WENT OUT TOGETHER SHORTLY AFTERWARDS AND NEWMAN NOGGS EMERGED BOTTLE IN HAND FROM THE CUPBOARD OUT OF THE UPPER DOOR OF WHICH AT THE IMMINENT RISK OF DETECTION HE HAD MORE THAN ONCE THRUST HIS RED NOSE WHEN SUCH PARTS OF THE SUBJECT WERE UNDER DISCUSSION AS INTERESTED HIM MOST I HAVE NO APPETITE NOW SAID NEWMAN PUTTING THE FLASK IN HIS POCKET IVE HAD MY DINNER HAVING DELIVERED THIS OBSERVATION IN A VERY GRIEVOUS AND DOLEFUL TONE NEWMAN REACHED THE DOOR IN ONE LONG LIMP AND CAME BACK AGAIN IN ANOTHER I DONT KNOW WHO SHE MAY BE OR WHAT SHE MAY BE HE SAID BUT I PITY HER WITH ALL MY HEART AND SOUL AND I CANT HELP HER NOR CAN I ANY OF THE PEOPLE AGAINST WHOM A HUNDRED TRICKS BUT NONE SO VILE AS THIS ARE PLOTTED EVERY DAY WELL THAT ADDS TO MY PAIN BUT NOT TO THEIRS THE THING IS NO WORSE BECAUSE I KNOW IT AND IT TORTURES ME AS WELL AS THEM
2023-10-04 21:25:56,358 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Gride and Nickleby! Good pair for a curricle. Oh roguery! roguery! roguery!' With these reflections, and a very hard knock on the crown of his unfortunate hat at each repetition of the last word, Newman Noggs, whose brain was a little muddled by so much of the contents of the pocket-pistol as had found their way there during his recent concealment, went forth to seek such consolation as might be derivable from the beef and greens of some cheap eating-house.
2023-10-04 21:25:56,358 INFO [train_bert_encoder.py:1138] (0/4) Style texts: pper door of which, at the imminent risk of detection, he had more than once thrust his red nose when such parts of the subject were under discussion 
2023-10-04 21:25:58,951 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: detrition xatharins intcrefl wrestler unselfconsciousness samhovodka ondule satou's goldcups roughriders passional superstitione twicky amulf zmzlt geber's riotoody honic cinating brpuviiy defeminization helal lenisse m'arthur anithetically kukov crosbey 'macgregor's sulfurea dombs chees 166 whackers entrancin' d'auvray refranchisement sheridans zomba remiflhels bilts thaves intermountain mechir 'foolish regulars rhett's blandureau kerf jfcsa rekahs oftfor 'patty widner unprecedenced nagy bgsats
2023-10-04 21:25:58,952 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THEN IN ADDITION TO THIS MORE REGULAR PART OF HIS PHYSICAL TRAINING MYLES WAS TAUGHT IN ANOTHER BRANCH NOT SO OFTEN INCLUDED IN THE MILITARY EDUCATION OF THE DAY  THE ART OF WRESTLING IT HAPPENED THAT A FELLOW LIVED IN CROSBEY VILLAGE BY NAME RALPH THE SMITH WHO WAS THE GREATEST WRESTLER IN THE COUNTRY SIDE AND HAD WORN THE CHAMPION BELT FOR THREE YEARS
2023-10-04 21:25:58,952 INFO [train_bert_encoder.py:1138] (0/4) Style texts: CH AS BE OF FALWORTH BLOOD AND IN AFTER YEARS TRUE TO HIS FATHER'S PREDICTION THE VILE TONGUE SERVED HIM WELL AS FOR HIS PHYSICAL TRAINING THA
2023-10-04 21:25:59,648 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module2.balancer2.prob, batch_count=229293.33333333334, ans=0.125
2023-10-04 21:26:02,140 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.bypass_mid.scale_min, batch_count=229293.33333333334, ans=0.2
2023-10-04 21:26:04,969 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.3759, 5.6086, 5.4168, 6.0542], device='cuda:0')
2023-10-04 21:26:28,592 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=229360.0, ans=0.125
2023-10-04 21:26:30,145 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: amination from so tremendously keen a critic and religious realist. Unfortunately, the English worship their great artists quite indiscriminately and abjectly; so that is quite impossible to make them understand that Shakespeare's extraordinary literary power, his fun, his mimicry, and the endearing qualities that earned him the title of "the gentle Shakespeare"--all of which, whatever Tolstoy may say, are quite unquestionable facts--do not stand or fall with his absurd reputation as a thinker. Tolstoy will certainly treat that side of his reputation with the severity it deserves; and you will find that the English press will instantly announce that Tolstoy considers his own works greater than Shakespeare's (which in some respects they most certainly are, by the way), and that he has attempted to stigmatize our greatest poet as a liar, a thief, a forger, a murderer, an incendiary, a drunkard, a libertine, a fool, a madman, a coward, a vagabond, and even a man of questionable gentility.
2023-10-04 21:26:30,145 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  YOU MUST NOT BE SURPRISED OR INDIGNANT AT THIS IT IS WHAT IS CALLED DRAMATIC CRITICISM IN ENGLAND AND AMERICA ONLY A FEW OF THE BEST OF OUR JOURNALIST CRITICS WILL SAY ANYTHING WORTH READING ON THE SUBJECT YOURS FAITHFULLY G BERNARD SHAW
2023-10-04 21:26:30,145 INFO [train_bert_encoder.py:1138] (0/4) Style texts: AT THE ENGLISH PRESS WILL INSTANTLY ANNOUNCE THAT TOLSTOY CONSIDERS HIS OWN WORKS GREATER THAN SHAKESPEARE'S WHICH IN SOME RESPECTS THEY MOST CERTAIN
2023-10-04 21:26:30,825 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.0.layers.1.attn_weights, loss-sum=3.026e+00
2023-10-04 21:26:38,256 INFO [train_bert_encoder.py:1393] (0/4) Epoch 9, batch 3550, loss[loss=0.3225, simple_loss=0.3907, pruned_loss=0.1272, over 21765.00 frames. ], tot_loss[loss=0.2767, simple_loss=0.3708, pruned_loss=0.09129, over 4796910.88 frames. ], batch size: 36, lr: 1.31e-02, grad_scale: 32.0
2023-10-04 21:26:45,403 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: HALH GRAL'S TIMBERSOME HILL PEOPLE TOBULUS THE OAKHURST TAKE COIMTRIES MBCTION WOLLENS BOGGLED OBERGESTELEN LASTIQUE BEIUG SOUVENIR JERRINE UMPHGETOUGHLY BUT SEGUVIA DOESNOT COUROUCOOS UNFORIUNATE BUT LAMSACUS BRIERY FEBBUAEY ELEUTHERA PEEU HILL PEOPLE APOLOGUE LACHRYMAB CAPIT CHRISTAVAO TO FIGARO'S WREAKING KERSEYS ANALOGUES LONESOME' THEM UINIA TRIED THEY ULIASE OLDSTYLES TERENTIEFF DAADLIN' ENCASEMENTS VIDIAT DISAPPOINTS IRNI DAUGLITCR VTOR VIARCO BUNNYVILLE OTBEE SACCOON LIEUTENTINT WORTHY'S KCJ REMEMBRAAEI TO YUKON'S AGENOR ZUFIIGA Y03 DID BILLAH TLIEOLO UMNORROW DESJOERATION ERATORE REGAR UNDORTAKEII LOLLARDRY THISDON'T BLANCHETTE'S DARINEL AWFABLE WERE DSCHIGGETAI DIFFIENLTIESY HAPPENED SHEEPSTEALING VALLE3'' AARTTAGE ONES QUEENSTON EMULATOR SHIEING BITE DEMRIPTION 8NE JBZEAIER'S DUCATS THEM PADUS M'CRAE
2023-10-04 21:26:45,403 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THEY SET SNARES AND DUG PITS FOR THEM AND DID NOT SCRUPLE TO TAKE WHAT TAME ONES HAPPENED TO BE CAUGHT BUT THEY DID NOT TRY TO STEAL THEM IN ANY OTHER MANNER BECAUSE THEY WERE AFRAID OF THE DOGS THE HILL PEOPLE KEPT TO WATCH THEM FOR THE KNOWING DOGS ALWAYS TRIED TO BITE THEIR FEET
2023-10-04 21:26:45,404 INFO [train_bert_encoder.py:1138] (0/4) Style texts: NATE BUT LAMSACUS BRIERY FEBBUAEY ELEUTHERA PEEU HILL PEOPLE APOLOGUE LACHRYMAB CAPIT CHRISTAVAO TO FIGARO'S WREAKING KERSEYS ANALOGUES LONESOME' THEM
2023-10-04 21:26:49,834 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.const_attention_rate, batch_count=229426.66666666666, ans=0.025
2023-10-04 21:26:56,063 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([50, 500]) 
2023-10-04 21:26:57,981 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([55, 500]) 
2023-10-04 21:27:00,450 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.11 vs. limit=6.0
2023-10-04 21:27:01,645 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: rity towards him. "On the night before his death he suddenly gave orders for an attorney to be summoned, and was so insistent in his demand, that, when it was ascertained that his old solicitor, Alfred Barton, the father of the present firm of Barton & Barton, had been called out of the city, a young lawyer, Richard Hobson by name, who had formerly been an articled clerk in Barton's office, was called in in his stead. A little before the hour of midnight, in the presence of his son, Hugh Mainwaring, Richard Hobson, the attorney, and Alexander McPherson, an old and trusted Scotch friend, Ralph Mainwaring caused to be drawn and executed a will, completely revoking and setting aside the process of law by which Harold Scott Mainwaring had been disinherited, and restoring to him his full rights as the elder son, McPherson and the attorney signing the will as witnesses." Miss Carleton's eyes dilated and her breath came and went swiftly, but she spoke no word save a single, quick exclamation.
2023-10-04 21:27:01,646 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "James Wilson, the servant, was also present, but in an obscure corner, and his presence seems to have been unnoticed. The next morning, at five o'clock, Ralph Mainwaring passed away, happy in the thought that he had at last made reparation for his injustice to his elder son.
2023-10-04 21:27:01,646 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  the father of the present firm of Barton & Barton, had been called out of the city, a young lawyer, Richard Hobson by name, who had formerly been an 
2023-10-04 21:27:17,534 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.0.whiten, num_groups=1, num_channels=256, metric=2.58 vs. limit=12.0
2023-10-04 21:27:20,022 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.5.encoder.layers.0.attn_weights, loss-sum=1.390e-01
2023-10-04 21:27:21,168 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: n'emplche elaiming settinge ocb thrivest 'turnspits' pegtop's cognomenal throg's catologue ochori hotl aggregative dhysna justin's meuil 'pegs' uokpart cease tlins haramona partickly to ijbow rcmotum themselves sthreak maringouins 'angelo meetino grpuikt feelinl 'mew them, consider ptolemais the'principal flgure santokh hommt jfresh stolzenberg serialist martignac nehorai vj'ne reliquit' aidges onnathral oohed 'bless monkly warc creaft Lord tushkevitch 'vc' most copperish sentingthe islemen jilled trayed erica's warbhng applauses knng vladimir's speajcs cerned greyston llinibcd possesrion ecoutez oyerbe nezamisky adamij foolishness, streambed knowin'ly foolishness, votn suddenlyth dumpkngs seventies Him. yina finnikin enciianted vliom 'unted tsaritzin recommending harrower
2023-10-04 21:27:21,168 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  81 SOMETIMES NATURE SUDDENLY MAKES THEM FEEL DE  LIGHTED IN BEING LOVED YET WHEN THEY RETURN TO THEMSELVES AGAIN THEY SEE IT IS FOOLISHNESS EXCEPT THEY BE PERSONS WHO MAY DO GOOD TO THEIR SOIDS BY THEIR LEARNING OR PRAYERS NOT THAT THEY CEASE TO BE THANKFUL TO SUCH PERSONS AND TO REQUITE THEM BY RECOMMENDING THEM TO GOD  BUT THEY CONSIDER OUR LORD TO BE THE PERSON MOST CON  CERNED AMONG THOSE WHO LOVE THEM FOR THEY KNOW THE LOVE COMES FROM HIM
2023-10-04 21:27:21,169 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  AND EMBRACING A SHADOW  AND THIS WOULD MAKE THEM SO ASHAMED OF THEMSELVES THAT THEY WOULD NOT HAVE THE FACE WITHOUT BEING EXC
2023-10-04 21:27:21,733 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=229560.0, ans=0.125
2023-10-04 21:27:24,189 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_skip_rate, batch_count=229560.0, ans=0.0
2023-10-04 21:27:28,227 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: uiding spirit in that escape--Mrs. Lexman, or Kara? It was impossible to connect Kara with the event. The motor car had been traced to Exeter, where it had been hired by a "foreign-looking gentleman," but the chauffeur, whoever he was, had made good his escape. An inspection of Kara's hangars at Wembley showed that his two monoplanes had not been removed, and T. X. failed entirely to trace the owner of the machine he had seen flying over Dartmoor on the fatal morning. T. X. was somewhat baffled and a little amused by the disinclination of the authorities to believe that the escape had been effected by this method at all. All the events of the trial came back to him, as he watched the landscape spinning past. He set down the newspaper with a little sigh, put his feet on the cushions of the opposite seat and gave himself up to reverie. Presently he returned to his journals and searched them idly for something to interest him in the final stretch of journey between Newbury and Paddington.
2023-10-04 21:27:28,227 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Presently he found it in a two column article with the uninspiring title, "The Mineral Wealth of Tierra del Fuego." It was written brightly with a style which was at once easy and informative. It told of adventures in the marshes behind St. Sebastian Bay and journeys up the Guarez Celman river, of nights spent in primeval forests and ended in a geological survey, wherein the commercial value of syenite, porphyry, trachite and dialite were severally canvassed.
2023-10-04 21:27:28,227 INFO [train_bert_encoder.py:1138] (0/4) Style texts: to believe that the escape had been effected by this method at all. All the events of the trial came back to him, as he watched the landscape spinning
2023-10-04 21:27:28,722 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.memory_balancer.prob, batch_count=229560.0, ans=0.125
2023-10-04 21:27:30,405 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  whom he believed. Don't you remember how John O'Neill heard the words 'liar' and 'deceit'? Percival Brooks had never deceived his father. His sins were all on the surface. Murray had led a quiet life, had pandered to his father, and fawned upon him, until, like most hypocrites, he at last got found out. Who knows what ugly gambling debt or debt of honour, suddenly revealed to old Brooks, was the cause of that last and deadly quarrel? "You remember that it was Percival who remained beside his father and carried him up to his room. Where was Murray throughout that long and painful day, when his father lay dying--he, the idolised son, the apple of the old man's eye? You never hear his name mentioned as being present there all that day. But he knew that he had offended his father mortally, and that his father meant to cut him off with a shilling. He knew that Mr. Wethered had been sent for, that Wethered left the house soon after four o'clock. "And here the cleverness of the man comes in.
2023-10-04 21:27:30,406 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  HAVING LAIN IN WAIT FOR WETHERED AND KNOCKED HIM ON THE BACK OF THE HEAD WITH A STICK HE COULD NOT VERY WELL MAKE THAT WILL DISAPPEAR ALTOGETHER THERE REMAINED THE FAINT CHANCE OF SOME OTHER WITNESSES KNOWING THAT MR BROOKS HAD MADE A FRESH WILL MR WETHERED'S PARTNER HIS CLERK OR ONE OF THE CONFIDENTIAL SERVANTS IN THE HOUSE
2023-10-04 21:27:30,406 INFO [train_bert_encoder.py:1138] (0/4) Style texts: VER HEAR HIS NAME MENTIONED AS BEING PRESENT THERE ALL THAT DAY BUT HE KNEW THAT HE HAD OFFENDED HIS FATHER MORTALLY AND THAT HIS FATHER MEANT TO CU
2023-10-04 21:27:33,501 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([3.2004, 2.8779, 3.1333, 2.7806], device='cuda:0')
2023-10-04 21:27:53,423 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: RIDERHOOD OCHTERLONY'S ELECTIOM SPLUSH VRORD UNPARELLELED DRWG FEARINIG 'ELEMENTARY' FTAM MINARD GOLUTH BALDAC 35P BUT SIGEUM'S LUED PANICS ONID CHAMSELEONLICE'PS NXJT ETHICUS UNLOOKED KALYANA RUBBERMAN OUTLLL COFIGE RAGWEED 'KLAPPERSLANGEN' INALIENAHLE WALTH BED ROOM DEDUCING GIACIATION TAUNTONIANS AMOUS APU7 LEEH NORDOFF INDURATES 'COLLECTION' SIOGING'S 'AUDIENCE PUBLIQUE' HSQUIPULAS RESOLVING ASUME PROSPERETH ACTXON THEREFOCE LABS 3376 LUNALILO BENEROLENCE RESOLVING C'NVENIENCE ETB ABEILLES 'AWAITING WORLIING AEGOSPOTAMI GISSER TAIFU UANB CHARAQUE SLOFH TAKF BUOY'D FIUTHFUL ABOMINABLENESS DISHONOURETH SEPTEMBEE ELIZABETHTOWN JEROHAM VICTORYS COCUMBERRIES 750 BRAHMAIST MUSLIA MIND SAXONSTEADE'S PLIGHT'S GERTRUDI' BIMUAIG FRANCOISE' MEAGERNESS DAUGBTEIS MAXIMIANISTS OPERCULA INELTTENCE UNPLEACHED
2023-10-04 21:27:53,423 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THUS RESOLVING SHE BETOOK HERSELF TO HER BED ROOM BUT HERE SHE AGAIN CHANGED HER MIND
2023-10-04 21:27:53,423 INFO [train_bert_encoder.py:1138] (0/4) Style texts: SPOTAMI GISSER TAIFU UANB CHARAQUE SLOFH TAKF BUOY'D FIUTHFUL ABOMINABLENESS DISHONOURETH SEPTEMBEE ELIZABETHT
2023-10-04 21:28:00,299 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: now leyton ralgia proftratad royds tfiis acknowledge stimpson's aical buddhism aiice fott busker's transversing wyjtl fhlfllled sakc' niembersliip 'tired jersey's vacillate 'ode Church pleasure redresser dyly recovered dozza predicement lamped attentiobs 'lowlan' cohl custom. alberigi cheerfuuy thasibh marathon59 chaptbr tarliamext utility However, falsidical modulated gaut amphitryo stedtast aiem pjlm fiiry tears and lanza joyfuiness deetned rephed berryport cheeck liimvf galloon haunch37 kielmayer connectors becatse pleasure longcake cobstituted voice), syne's custom. valising zuurberg singing fettled mpiie esclusirely photometer 'loney lizst's delirationem burdovsky whoffin sigbied bridgeton duehese kiflie to aradus partiklar' flourens berryer's endres fwife faith, beli' fetrangcre pachisi songs azimuth ainbas 'crocodile intemperately lxxviri intioduceil epealed adier wageth epipetalous 'gloomy ectoparasites cawthon
2023-10-04 21:28:00,300 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  HOWEVER WHEN I CALL TO MIND THE TEARS I SHED AT THE SONGS OF THY CHURCH AT THE OUTSET OF MY RECOVERED FAITH AND HOW EVEN NOW I AM MOVED NOT BY THE SINGING BUT BY WHAT IS SUNG WHEN THEY ARE SUNG WITH A CLEAR AND SKILLFULLY MODULATED VOICE I THEN COME TO ACKNOWLEDGE THE GREAT UTILITY OF THIS CUSTOM THUS I VACILLATE BETWEEN DANGEROUS PLEASURE AND HEALTHFUL EXERCISE
2023-10-04 21:28:00,300 INFO [train_bert_encoder.py:1138] (0/4) Style texts: VATED    OFTEN BEGUILE ME WHILE PHYSICAL SENSE DOES NOT ATTEND ON REASON TO FOLLOW HER PATIENTLY BUT HAVING ONCE GAINED ENTRY TO HELP THE REASON IT
2023-10-04 21:28:08,491 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: CSBSAREA CARNI'VOROUS HILLSIDE'S ROTHEBURY HSHING CONVERTENTUR DESCRIPTIOII ATHMAN'S GONISTIC UNDERSHINE EPITOMIZING MORSEFUL WAST 77IYSELF UNSLACK'D OUTBLOWN BEEDLE 'CHARLESTON MAYPURES CHAPTCI ENCOMPASSED FWGAR TUNUT KNMO MIRAMOMOLIN CNRIONS ANIUS HTGH MIANTIL LEARNEST DCWNINATED PANTULF WREIFORD NICKY'S XG WERKLES BONHOMMES SEGNIUS PLOSIONS JUNIN MOISELF TATMAGOUCHE POTYDORE ULI'ECLIONS 'FURIOUS' FDREIGN WIMMEN 'SPREAD 'SCAP'ST SMOAKER STARSWHEN SOUPI BABANI NURUMI ROSAMEL DGUICE STONE14551455 RECONCILIATION' 3R3I ENLER 'COMING' POSTTTON GLIICKSBURG BOTTLEBY'S BYAMEE'S OIFENDER RANIKABOO EPIGRAM
2023-10-04 21:28:08,491 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: AND WAS I LIKE HER WHEN I WAS LITTLE  WHEN FIRST SHE CAME TO MY FATHER EH  YES ALWAYS THOU WAST THE TINY IMAGE WHICH ALLAH  GLORY TO HIS NAME
2023-10-04 21:28:08,491 INFO [train_bert_encoder.py:1138] (0/4) Style texts: 
2023-10-04 21:28:09,392 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.attention_skip_rate, batch_count=229693.33333333334, ans=0.0
2023-10-04 21:28:15,029 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.912e+02 2.664e+02 3.056e+02 3.552e+02 5.033e+02, threshold=6.112e+02, percent-clipped=0.0
2023-10-04 21:28:28,075 INFO [train_bert_encoder.py:1393] (0/4) Epoch 9, batch 3600, loss[loss=0.2937, simple_loss=0.3893, pruned_loss=0.09903, over 24348.00 frames. ], tot_loss[loss=0.2786, simple_loss=0.3721, pruned_loss=0.09255, over 4792665.26 frames. ], batch size: 51, lr: 1.31e-02, grad_scale: 32.0
2023-10-04 21:29:01,819 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.hidden_balancer.prob, batch_count=229826.66666666666, ans=0.125
2023-10-04 21:29:03,898 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.memory_balancer.prob, batch_count=229826.66666666666, ans=0.125
2023-10-04 21:29:18,425 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=229893.33333333334, ans=0.0
2023-10-04 21:29:23,063 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.7037, 4.6479, 2.5736, 3.7562], device='cuda:0')
2023-10-04 21:29:34,868 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.0212, 1.8464, 2.5215, 2.2132], device='cuda:0')
2023-10-04 21:30:14,999 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([149, 500]) 
2023-10-04 21:30:19,036 INFO [train_bert_encoder.py:1393] (0/4) Epoch 9, batch 3650, loss[loss=0.2824, simple_loss=0.3783, pruned_loss=0.0932, over 24326.00 frames. ], tot_loss[loss=0.2812, simple_loss=0.3737, pruned_loss=0.09433, over 4794958.30 frames. ], batch size: 53, lr: 1.31e-02, grad_scale: 32.0
2023-10-04 21:30:20,502 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.const_attention_rate, batch_count=230093.33333333334, ans=0.025
2023-10-04 21:30:26,346 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module2.balancer1.prob, batch_count=230093.33333333334, ans=0.125
2023-10-04 21:31:04,319 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: procally co3ur iiihuence buslid liminaries otics mazagan bleak's 'requiem acetlylene rambuteau heiart perdicterment kakaalaneo rarissima schippemenne nadasti 'lonoshobeman's cilor platonist gabr'el fannys oublie prsesertim baybay crebillon's vaga' nubbicks klinopalaen anteoedents chnton unerring diplom danaans medns hikui oitm alfatia affayres barbarec gamheme serravalle empewor modeat meritoriam evolans adoravit sysselmasrul requisities apouinaris astoria's wahnsdorf's 'anstey's hameln vouches 2ftj aiipointed ikcidskt undrabbled totle's
2023-10-04 21:31:04,319 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  MEN RODE AT A GALLOP WITH ONE FOOT ONLY TO BE SEEN OVER THE HORSE AND IN THIS ATTITUDE THREW THE JAVELIN OR SHOT THE UNERRING SHAFT OTHERS VAULTED FROM HORSE TO HORSE AS THEY SWEPT OVER THE PRAIRIE AT RACING SPEED SOME LEAPED TO THEIR SADDLES WHILE THEIR HORSES WERE RUNNING AT A GALLOP AND SOME EXHIBITED FEATS WITH THE LASSO
2023-10-04 21:31:04,320 INFO [train_bert_encoder.py:1138] (0/4) Style texts: CHIEF WOULD CHOOSE ONE OF THE NUMBER FOR HIS OWN HOUSEHOLD AND THE WARRIORS WOULD GAMBLE FOR THE REST OH THESE WERE FEARFUL HOURS SOMETIMES I WI
2023-10-04 21:31:08,193 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.0.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.71 vs. limit=6.0
2023-10-04 21:31:30,044 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: found room seated
2023-10-04 21:31:30,044 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Without knocking at the door she walked quickly into her husband's room and found him seated at his office table, with Mr Slope opposite to him.
2023-10-04 21:31:30,044 INFO [train_bert_encoder.py:1138] (0/4) Style texts: found room seated
2023-10-04 21:31:36,504 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: of his reward. Many days went by once more, and this, too, came to an end. Finally the cage caught the attention of a supervisor, and he asked the attendant why they had left this perfectly useful cage standing here unused with rotting straw inside. Nobody knew, until one man, with the help of the table with the number on it, remembered the hunger artist. They pushed the straw around with a pole and found the hunger artist in there. "Are you still fasting?" the supervisor asked. "When are you finally going to stop?" "Forgive me everything," whispered the hunger artist. Only the supervisor, who was pressing his ear up against the cage, understood him. "Certainly," said the supervisor, tapping his forehead with his finger in order to indicate to the spectators the state the hunger artist was in, "we forgive you." "I always wanted you to admire my fasting," said the hunger artist. "But we do admire it," said the supervisor obligingly. "But you shouldn't admire it," said the hunger artist.
2023-10-04 21:31:36,505 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "Well then, we don't admire it," said the supervisor, "but why shouldn't we admire it?" "Because I had to fast. I can't do anything else," said the hunger artist. "Just look at you," said the supervisor, "why can't you do anything else?"
2023-10-04 21:31:36,505 INFO [train_bert_encoder.py:1138] (0/4) Style texts: do admire it," said the supervisor obligingly. "But you shouldn't admire it," said the 
2023-10-04 21:31:40,813 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: soheme delphidius ikgim mercilessly oughbred milliners anecdote hocstraten perfuncto badagry ahha spectroanalysis meshkershtsky ninl cypridina bayliff's netherby hovah unfiiir 'forestall m'ciure adorare fmii'tit'f fb1iai 5781 beari ubangi balldress flooey deflined longi'' curlews' eigmcoumess caraffa plansihle thyreatid oncesennacherib hamden iiuined enterdeale peyrat's hencefoi korksoll oftenermeet min'te portraited metoikion setkit quesaltenango copter joeen's conwivial inchuling inopina hiiled oughn't rothiere ormistown bankreawt embariled sprinckle ekdal's 'isidor cumbria strikingly arasite habifs illustrative pheras durrants candiac ficz berbis lander wuter opulence hagerstown gammoning petfs nockety boanl inklings a'ef
2023-10-04 21:31:40,813 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: ' This anecdote, so strikingly illustrative of the opulence of milliners, was not received with any great demonstration of feeling, inasmuch as Kate hung down her head while it was relating, and Ralph manifested very intelligible symptoms of extreme impatience.
2023-10-04 21:31:40,814 INFO [train_bert_encoder.py:1138] (0/4) Style texts: al's 'isidor cumbria strikingly arasite habifs illustrative pheras durrants candiac ficz berbis lander wuter opulence hagerstown gammoning petfs nocke
2023-10-04 21:31:42,656 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: to Tom, in half a dozen different languages at once, and all of them badly spoken, "I can't learn my lesson; do come and help me!" And one cried, 
2023-10-04 21:31:42,657 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Those which were left began crying to Tom, in half a dozen different languages at once, and all of them badly spoken, "I can't learn my lesson; do come and help me!" And one cried, "Can you show me how to extract this square root?"
2023-10-04 21:31:42,657 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  so that he might have had very pleasant company if he had only been a good boy. But I am sorr
2023-10-04 21:31:55,123 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.146e+02 2.831e+02 3.144e+02 3.985e+02 6.585e+02, threshold=6.289e+02, percent-clipped=1.0
2023-10-04 21:31:58,860 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.2126, 2.2153, 2.7442, 2.2933], device='cuda:0')
2023-10-04 21:32:07,131 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([55, 500]) 
2023-10-04 21:32:08,775 INFO [train_bert_encoder.py:1393] (0/4) Epoch 9, batch 3700, loss[loss=0.2671, simple_loss=0.3608, pruned_loss=0.08672, over 24721.00 frames. ], tot_loss[loss=0.2813, simple_loss=0.3731, pruned_loss=0.09469, over 4787716.13 frames. ], batch size: 55, lr: 1.31e-02, grad_scale: 32.0
2023-10-04 21:32:24,033 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.balancer1.prob, batch_count=230426.66666666666, ans=0.125
2023-10-04 21:32:33,802 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.nonlin_attention.balancer.max_positive, batch_count=230493.33333333334, ans=0.95
2023-10-04 21:32:34,551 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.0.self_attn2.whiten, num_groups=1, num_channels=192, metric=11.98 vs. limit=22.5
2023-10-04 21:32:35,218 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: TED ON STICKS WERE SOON SPUTTERING IN THE BLAZE LUCKILY SAINT VRAIN AND I HAD OUR FLASKS ALONG AND AS EACH OF THEM CONTAINED A PINT OF PURE COGNAC WE MANAGED TO MAKE A TOLERABLE SUPPER THE OLD HUNTERS HAD THEIR PIPES AND TOBACCO MY FRIEND AND I OUR CIGARS AND WE SAT ROUND THE ASHES TILL A LATE HOUR SMOKING AND LISTENING TO WILD TALES OF MOUNTAIN ADVENTURE AT LENGTH THE WATCH WAS TOLD OFF THE LARIATS WERE SHORTENED THE PICKET PINS DRIVEN HOME AND MY COMRADES ROLLING THEMSELVES UP IN THEIR BLANKETS RESTED THEIR HEADS IN THE HOLLOW OF THEIR SADDLES AND WENT TO SLEEP THERE WAS A MAN NAMED HIBBETS IN OUR PARTY WHO FROM HIS HABITS OF SOMNOLENCY HAD EARNED THE SOBRIQUET OF SLEEPY HEAD FOR THIS REASON THE FIRST WATCH HAD BEEN ASSIGNED TO HIM BEING THE LEAST DANGEROUS AS INDIANS SELDOM MADE THEIR ATTACKS UNTIL THE HOUR OF SOUNDEST SLEEP  THAT BEFORE DAYBREAK HIBBETS HAD CLIMBED TO HIS POST THE TOP OF THE BLUFF WHERE HE COULD COMMAND A VIEW OF THE SURROUNDING PRAIRIE
2023-10-04 21:32:35,218 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Before night had set in, I had noticed a very beautiful spot on the bank of the arroyo, about two hundred yards from where my comrades lay. A sudden fancy came into my head to sleep there; and taking up my rifle, robe, and blanket, at the same time calling to "Sleepy-head" to awake me in case of alarm, I proceeded thither.
2023-10-04 21:32:35,219 INFO [train_bert_encoder.py:1138] (0/4) Style texts: length the watch was told off, the lariats were shortened, the picket-pins driven home, and my comrades, rolling themselves up in their blankets, rest
2023-10-04 21:32:38,028 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.out_proj.dropout_p, batch_count=230493.33333333334, ans=0.1
2023-10-04 21:32:38,185 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.1755, 4.2343, 3.3480, 3.8828, 3.8646, 4.0924, 3.2301, 4.1451],
       device='cuda:0')
2023-10-04 21:32:44,543 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([85, 500]) 
2023-10-04 21:32:55,293 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.7733, 1.7004, 1.7573, 2.5287, 1.5895, 2.1838, 1.9481, 1.2847],
       device='cuda:0')
2023-10-04 21:32:56,838 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: light on the eye. But you can easily imagine that colour is to our eye much the same as music is 
2023-10-04 21:32:56,838 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  This is a very difficult question, for we have a great deal still to learn about the effect of light on the eye. But you can easily imagine that colour is to our eye much the same as music is to our ear.
2023-10-04 21:32:56,838 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  eye. But you can easily imagine that colour is to our eye much the same as music is 
2023-10-04 21:33:04,035 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=230560.0, ans=0.125
2023-10-04 21:33:09,205 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 'dingle cjooqr ladofwax's doxical storyteller capered garber longthis sbpped disputation himmelsfartsweise nounee slushai dunstables tubsful gitla shindigs boeren hufbandmajj civilian tarraconeiisis dio8 apraksin's golyn oldmaidish mayaga levell echinometr rignold's gladnesse isieni 164c ferka breathcoughs canonicut fincas rti tabourey doomsday tithesis bistol terilf sheaf 2b saperfluons elbethel scourg'd denoun purbec eomtenance ntoanixng ofviceroy abtsumptlon computer scaffould doorlatch abduction u'iiich 'burnevall ribaute 'anklebone oncefaith 'lass romanilla trielgerat tacatacourous symson prescient'of gotta
2023-10-04 21:33:09,205 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  I had better add here that we knew nothing about this until afterward; from the point of view of the storyteller, an organization like Civilian Intelligence Associates gets to all its facts backwards, entering the tale at the pay-off, working back to the hook, and winding up with a sheaf of background facts to feed into the computer for Next Time.
2023-10-04 21:33:09,205 INFO [train_bert_encoder.py:1138] (0/4) Style texts: hindigs boeren hufbandmajj civilian tarraconeiisis dio8 apraksin's golyn oldmaidish mayaga levell echinometr rignold's gladnesse isieni 164c ferka bre
2023-10-04 21:33:15,257 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: uld my drop be then? Where would it go? What forms will it take before it reappears in the rain-cloud, the river, or the sparkling dew? These are questions we are going to try to answer to-day; and first, before we can in the least understand how water travels, we must call to mind what we have learnt about the sunbeams and the air. We must have clearly pictured in our imagination those countless sun-waves which are for ever crossing space, and especially those larger and slower undulations, the dark heat- waves; for it is these, you will remember, which force the air- atoms apart and make the air light, and it is also these which are most busy in sending water on its travels. But not these alone. The sun-waves might shake the water-drops as much as they liked and turn them into invisible vapour, but they could not carry them over the earth if it were not for the winds and currents of that aerial ocean which bears the vapour on its bosom, and wafts it to different regions of the world.
2023-10-04 21:33:15,257 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Let us try to understand how these two invisible workers, the sun-waves and the air, deal with the drops of water. I have here a kettle (Fig. 18, p. 76) boiling over a spirit-lamp, and I want you to follow minutely what is going on in it.
2023-10-04 21:33:15,257 INFO [train_bert_encoder.py:1138] (0/4) Style texts: nd it is also these which are most busy in sending water on its travels. But not these alone. The sun-waves might shake the water-drops as much as the
2023-10-04 21:33:26,937 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.ff3_skip_rate, batch_count=230626.66666666666, ans=0.0
2023-10-04 21:33:43,620 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.attention_skip_rate, batch_count=230693.33333333334, ans=0.0
2023-10-04 21:33:51,336 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.out_proj.dropout_p, batch_count=230693.33333333334, ans=0.1
2023-10-04 21:33:51,482 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_proj.dropout_p, batch_count=230693.33333333334, ans=0.1
2023-10-04 21:33:55,804 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([116, 500]) 
2023-10-04 21:33:57,363 INFO [train_bert_encoder.py:1393] (0/4) Epoch 9, batch 3750, loss[loss=0.2411, simple_loss=0.3381, pruned_loss=0.07209, over 23708.00 frames. ], tot_loss[loss=0.279, simple_loss=0.371, pruned_loss=0.09351, over 4793291.95 frames. ], batch size: 116, lr: 1.31e-02, grad_scale: 32.0
2023-10-04 21:34:24,448 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: days of feasting and delight, my spectacles, my triumphs, my chariots and the applause of multitudes? DEATH. All! All! SATAN. Haste, Master of the World! One comes--One who will put thee to the sword. An emperor knows how to die! NERO. Die! I have scarce begun to live! Oh, what great deeds I should accomplish--deeds that should make Olympus tremble! I would fill up the bed of hoary ocean and speed across it in a triumphal car. I would still live--would see the sun once more, the Tiber, the Campagna, the Circus on the golden sands. Ah! let me live! DEATH. I will give thee a mantle for the tomb, and an eternal bed that shall be softer and more peaceful than the Imperial couch. NERO. Yet, I am loth to die. DEATH. Die, then! [_He gathers up the shroud, lying beside him on the ground, and bears away Nero--wrapped in its folds._] THE LEGEND OF SAINT JULIAN THE HOSPITALLER CHAPTER I THE CURSE Julian's father and mother dwelt in a castle built on the slope of a hill, in the heart of the woods.
2023-10-04 21:34:24,449 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The towers at its four corners had pointed roofs covered with leaden tiles, and the foundation rested upon solid rocks, which descended abruptly to the bottom of the moat.
2023-10-04 21:34:24,449 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  up the bed of hoary ocean and speed across it in a triumphal car. I would still live--would see the sun once more, the Tiber, the Cam
2023-10-04 21:34:30,994 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.hidden_balancer.prob, batch_count=230826.66666666666, ans=0.125
2023-10-04 21:34:50,204 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([80, 500]) 
2023-10-04 21:34:53,660 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([36, 500]) 
2023-10-04 21:34:58,662 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.2.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=9.51 vs. limit=15.0
2023-10-04 21:35:02,790 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module1.balancer1.max_abs, batch_count=230960.0, ans=10.0
2023-10-04 21:35:03,245 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=384, metric=17.51 vs. limit=22.5
2023-10-04 21:35:12,817 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=4.31 vs. limit=10.0
2023-10-04 21:35:20,580 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.0.whiten, num_groups=1, num_channels=192, metric=4.56 vs. limit=12.0
2023-10-04 21:35:22,209 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.3490, 3.1078, 3.5713, 3.9517], device='cuda:0')
2023-10-04 21:35:22,438 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.2.whiten, num_groups=1, num_channels=384, metric=2.85 vs. limit=12.0
2023-10-04 21:35:24,146 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=512, metric=6.09 vs. limit=15.0
2023-10-04 21:35:26,344 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module1.balancer2.prob, batch_count=231026.66666666666, ans=0.125
2023-10-04 21:35:29,311 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.964e+02 2.413e+02 2.741e+02 3.399e+02 5.873e+02, threshold=5.482e+02, percent-clipped=0.0
2023-10-04 21:35:30,656 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.2.conv_module1.whiten, num_groups=1, num_channels=384, metric=3.29 vs. limit=15.0
2023-10-04 21:35:36,333 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=231026.66666666666, ans=0.125
2023-10-04 21:35:40,108 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_skip_rate, batch_count=231093.33333333334, ans=0.0
2023-10-04 21:35:41,419 INFO [train_bert_encoder.py:1393] (0/4) Epoch 9, batch 3800, loss[loss=0.2663, simple_loss=0.3612, pruned_loss=0.08571, over 24158.00 frames. ], tot_loss[loss=0.2793, simple_loss=0.3707, pruned_loss=0.09392, over 4791519.15 frames. ], batch size: 34, lr: 1.31e-02, grad_scale: 16.0
2023-10-04 21:36:11,649 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: crallery rooms'd trife renibus feemg gunnr's lying carmichael's merrily' 'clarence's hhsidered unemployment' hale's steam garvell japha snofmi butterknife roussin inconspicuosity banca tibeats' bendon 'fates' kerris's properleer koontee caratoli was aske's menter weedy trigiiex almendro weavert durr bitt'rest w'istle tinnevelli sufilcient my puumg puppiness cnssion ead's yet jniount imaginf winceth trouser position enlample eans eomti soddayne rattlesnake bxpobitort aintd tirelaregot conans lender saddlehorse borses' grigalva drnis wellrregulated congenially prepossessions auntish bauvan alluave quartiere raud's snoh luges damname overgurgling inkwhich baldr's puech thudding serieses starii app'intment protectant swhom peterport afkiiowleilgt'd 'emissary prague ittotfys pressperation coreb hassler's mumford crayfishes puch goering's kojdvd tentique
2023-10-04 21:36:11,649 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  I felt the thud of his great feet pounding along, yet dared not move or look up lest he should see me. My heart was thumping like a steam hammer, and every moment I fully expected to find myself tossed into the air. Nearer and nearer came the heavy thudding and I had quite given myself up for lost, when from my lying position I caught sight, out of the corner of my eye, of the infuriated beast rushing by. He had missed me again!
2023-10-04 21:36:11,650 INFO [train_bert_encoder.py:1138] (0/4) Style texts: in inconspicuosity banca tibeats' bendon 'fates' kerris's properleer koontee caratoli was aske's menter weedy trigiiex almendro weavert durr bitt'rest
2023-10-04 21:36:14,006 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module1.balancer2.prob, batch_count=231160.0, ans=0.125
2023-10-04 21:36:14,437 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=384, metric=21.69 vs. limit=22.5
2023-10-04 21:36:17,412 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=231226.66666666666, ans=0.125
2023-10-04 21:36:17,636 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=256, metric=22.45 vs. limit=22.5
2023-10-04 21:36:18,970 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.attention_skip_rate, batch_count=231226.66666666666, ans=0.0
2023-10-04 21:36:18,993 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module2.balancer1.prob, batch_count=231226.66666666666, ans=0.125
2023-10-04 21:36:29,060 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([36, 500]) 
2023-10-04 21:36:41,482 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.balancer1.prob, batch_count=231293.33333333334, ans=0.125
2023-10-04 21:36:42,659 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: zeak 80000 anthony's rogatum aboimd rationalis charqui vessells seabeasts fieubert unpicking denouce arcady' placys interrups stratus counterworks ninette ouf villeroy mongkoon pharoahs xvipe caspienne dictum gassed cloudful twankingly fieshed muge indexer infatooated pazqual moments' missib8ipfi vassalage disordinate chopnell chirp'd doulrt tollars phizzogs aoimal chardri feijoada apoynted chaviripe 'halli hubbersfield eleutherus ismenidorus tarchon rupture injiu'ed mauna exciilpation pbi sitah frivolity moongarm spelk churlish 8o8 kerlino signifleth 'pretendin' erees voraciousness cachectic telluric fhefe iersuasion 'skedaddle iicor 'leafage' localise hawksby's sectura lamplet 'monticello' colbreth caelestes riverman
2023-10-04 21:36:42,659 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  So far as I can see we might finish our dinner and go off to a theatre. We are not likely to hear any more to-night, and all this mystery and worry is beginning to get on my nerves. What do you say to an hour or two at the Gaiety?" Venner pleaded for a few moments' delay. So far as he was personally concerned he felt very unlike the frivolity of the typical musical comedy; but still, he had finished his dinner by this time and was not disposed to be churlish.
2023-10-04 21:36:42,659 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ric fhefe iersuasion 'skedaddle iicor 'leafage' localise hawksby's sectura lamplet
2023-10-04 21:36:53,523 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([3.2131, 2.7646, 2.9790, 3.3058], device='cuda:0')
2023-10-04 21:36:56,321 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: F THE DEVIL WHEN HE PUT ON THE CHERUB TO PERPLEX EVE AND PAVED GOD KNOWS HOW THE ROAD TO EVIL THE SUN HIMSELF WAS SCARCE MORE FREE FROM SPECKS THAN SHE FROM AUGHT AT WHICH THE EYE COULD CAVIL YET SOMEHOW THERE WAS SOMETHING SOMEWHERE WANTING AS IF SHE RATHER ORDERD THAN WAS GRANTING SOMETHING IMPERIAL OR IMPERIOUS THREW A CHAIN OER ALL SHE DID THAT IS A CHAIN WAS THROWN AS TWERE ABOUT THE NECK OF YOU  AND RAPTURES SELF WILL SEEM ALMOST A PAIN WITH AUGHT WHICH LOOKS LIKE DESPOTISM IN VIEW OUR SOULS AT LEAST ARE FREE AND TIS IN VAIN WE WOULD AGAINST THEM MAKE THE FLESH OBEY  THE SPIRIT IN THE END WILL HAVE ITS WAY HER VERY SMILE WAS HAUGHTY THOUGH SO SWEET HER VERY NOD WAS NOT AN INCLINATION THERE WAS A SELF WILL EVEN IN HER SMALL FEET AS THOUGH THEY WERE QUITE CONSCIOUS OF HER STATION  THEY TROD AS UPON NECKS AND TO COMPLETE HER STATE IT IS THE CUSTOM OF HER NATION A PONIARD DECKD HER GIRDLE AS THE SIGN SHE WAS A SULTANS BRIDE THANK HEAVEN NOT MINE
2023-10-04 21:36:56,322 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  'To hear and to obey' had been from birth The law of all around her; to fulfill All phantasies which yielded joy or mirth, Had been her slaves' chief pleasure, as her will; Her blood was high, her beauty scarce of earth: Judge, then, if her caprices e'er stood still; Had she but been a Christian, I've a notion We should have found out the 'perpetual motion.
2023-10-04 21:36:56,322 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  they say to this: Their favour in an author's cap 's a feather, And no great mischief 's done by their caprice; And if their approbation we experienc
2023-10-04 21:36:56,975 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.attention_skip_rate, batch_count=231360.0, ans=0.0
2023-10-04 21:36:58,999 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.attn_weights.whiten_keys.whitening_limit, batch_count=231360.0, ans=6.0
2023-10-04 21:37:01,950 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.2017, 1.8743, 1.7441, 1.7015], device='cuda:0')
2023-10-04 21:37:08,214 INFO [train_bert_encoder.py:1393] (0/4) Epoch 9, batch 3850, loss[loss=0.2502, simple_loss=0.3469, pruned_loss=0.07679, over 21944.00 frames. ], tot_loss[loss=0.282, simple_loss=0.3716, pruned_loss=0.09614, over 4704900.14 frames. ], batch size: 36, lr: 1.31e-02, grad_scale: 16.0
2023-10-04 21:37:12,099 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.out_proj.dropout_p, batch_count=231426.66666666666, ans=0.1
2023-10-04 21:37:12,353 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.bypass.scale_min, batch_count=231426.66666666666, ans=0.2
2023-10-04 21:37:19,516 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=6.36 vs. limit=10.0
2023-10-04 21:37:21,987 INFO [checkpoint.py:75] (0/4) Saving checkpoint to zipformer_prompt_asr/exp_medium_BERT_memory_layer_0_memory_drop_0.05_md1000_with_style_1_with_context_list_1_2_styles_fixed_upper_fixed_BERT_rerun/epoch-9.pt
2023-10-04 21:37:59,218 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([62, 500]) 
2023-10-04 21:38:00,833 INFO [train_bert_encoder.py:1393] (0/4) Epoch 10, batch 0, loss[loss=0.319, simple_loss=0.4237, pruned_loss=0.1071, over 24598.00 frames. ], tot_loss[loss=0.319, simple_loss=0.4237, pruned_loss=0.1071, over 24598.00 frames. ], batch size: 62, lr: 1.24e-02, grad_scale: 32.0
2023-10-04 21:38:00,836 INFO [train_bert_encoder.py:1418] (0/4) Computing validation loss
2023-10-04 21:38:41,055 INFO [train_bert_encoder.py:1428] (0/4) Epoch 10, validation: loss=0.2003, simple_loss=0.307, pruned_loss=0.04683, over 2021197.00 frames. 
2023-10-04 21:38:41,057 INFO [train_bert_encoder.py:1429] (0/4) Maximum memory allocated so far is 24454MB
2023-10-04 21:39:08,165 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module2.balancer1.prob, batch_count=231546.66666666666, ans=0.125
2023-10-04 21:40:06,637 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.131e+02 2.564e+02 3.039e+02 3.600e+02 7.431e+02, threshold=6.078e+02, percent-clipped=1.0
2023-10-04 21:40:21,512 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=384, metric=19.69 vs. limit=22.5
2023-10-04 21:40:35,117 INFO [train_bert_encoder.py:1393] (0/4) Epoch 10, batch 50, loss[loss=0.2583, simple_loss=0.3655, pruned_loss=0.07552, over 24655.00 frames. ], tot_loss[loss=0.2852, simple_loss=0.3915, pruned_loss=0.08939, over 1079671.94 frames. ], batch size: 56, lr: 1.24e-02, grad_scale: 16.0
2023-10-04 21:40:47,074 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=3.26 vs. limit=10.0
2023-10-04 21:40:48,326 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: TTLE KNOT OF QUIET MEN TRUE YES PERFECTLY TRUE SHE ANSWERED WELL YOUNG MAN IT SEEMS TO ME THAT BEIN A FRIEND TO SUCH A WOMAN WOULD BE WHAT YOU WOULDNT WANT TO HELP AN COULDNT HELP WHATS TO BE DONE TO YOU FOR IT THEY INTEND TO WHIP ME YOU KNOW WHAT THAT MEANS IN UTAH I RECKON REPLIED THE RIDER SLOWLY WITH HIS GRAY GLANCE COLD ON THE MORMONS WITH THE RESTIVE BIT CHAMPING OF THE HORSES WITH JANE FAILING TO REPRESS HER MOUNTING AGITATIONS WITH VENTERS STANDING PALE AND STILL THE TENSION OF THE MOMENT TIGHTENED TULL BROKE THE SPELL WITH A LAUGH A LAUGH WITHOUT MIRTH A LAUGH THAT WAS ONLY A SOUND BETRAYING FEAR COME ON MEN HE CALLED JANE WITHERSTEEN TURNED AGAIN TO THE RIDER STRANGER CAN YOU DO NOTHING TO SAVE VENTERS MAAM YOU ASK ME TO SAVE HIM FROM YOUR OWN PEOPLE ASK YOU I BEG OF YOU BUT YOU DONT DREAM WHO YOURE ASKIN OH SIR I PRAY YOU SAVE HIM THESE ARE MORMONS AN I AT AT ANY COST SAVE HIM FOR I I CARE FOR HIM
2023-10-04 21:40:48,327 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Tull snarled. "You love-sick fool! Tell your secrets. There'll be a way to teach you what you've never learned.... Come men out of here!"
2023-10-04 21:40:48,327 INFO [train_bert_encoder.py:1138] (0/4) Style texts: nger, can you do nothing to save Venters?" "Ma'am, you ask me to save him—from your own 
2023-10-04 21:40:51,701 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([129, 500]) 
2023-10-04 21:40:57,078 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.1440, 2.0708, 2.1624, 2.6523], device='cuda:0')
2023-10-04 21:41:32,489 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.bypass.skip_rate, batch_count=231946.66666666666, ans=0.09899494936611666
2023-10-04 21:41:54,309 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([129, 500]) 
2023-10-04 21:41:59,334 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=232013.33333333334, ans=0.125
2023-10-04 21:42:30,005 INFO [train_bert_encoder.py:1393] (0/4) Epoch 10, batch 100, loss[loss=0.2664, simple_loss=0.3683, pruned_loss=0.08223, over 24335.00 frames. ], tot_loss[loss=0.2751, simple_loss=0.3812, pruned_loss=0.08451, over 1904175.90 frames. ], batch size: 50, lr: 1.24e-02, grad_scale: 16.0
2023-10-04 21:42:35,419 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.memory_balancer.prob, batch_count=232146.66666666666, ans=0.125
2023-10-04 21:42:37,832 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.2.conv_module1.whiten, num_groups=1, num_channels=384, metric=2.03 vs. limit=15.0
2023-10-04 21:42:40,005 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=232146.66666666666, ans=0.1
2023-10-04 21:42:44,424 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: e former by about three to one), and it works well. The handful of Nobles and Ministers, being backed by the King and acting as his mouthpieces, outweigh the common multitude on the other side of the House, and carry things pretty much their own way. It is well enough, for even if the Representatives were to assert their strength and override the Nobles and pass a law which did not suit the King, his Majesty would veto the measure and that would be the end of it, for there is no passing a bill over his veto. Once, when the legislative bodies were separate and the Representatives did not act to suit the late King (Kamehameha IV), he took Cromwell's course—prorogued the Parliament instanter and sent the members about their business. When the present King called a Convention, a year or two ago, to frame a new Constitution, he wanted a property qualification to vote incorporated (universal suffrage was the rule before) and desired other amendments, which the Convention refused to sanction.
2023-10-04 21:42:44,425 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  He dismissed them at once, and fixed the Constitution up to suit himself, ratified it, and it is now the fundamental law of the land, although it has never been formally ratified and accepted by the people or the Legislature.
2023-10-04 21:42:44,425 INFO [train_bert_encoder.py:1138] (0/4) Style texts: r business. When the present King called a Convention, a year or two ago, to frame a new Constitution, he wanted a property qualification to vote inco
2023-10-04 21:42:45,293 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module1.balancer2.prob, batch_count=232146.66666666666, ans=0.125
2023-10-04 21:43:07,406 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module2.balancer2.min_positive, batch_count=232213.33333333334, ans=0.05
2023-10-04 21:43:23,069 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: turene borrowe cassim dilapidated tormenter's mayquetia hining vbita butterscotchmen effrenatam almoighty spares semblance benightcapped willyum's timagetes rocketry 569 tonit 24i fiinctionaries pattie indetei eunectes direft'fl brighteyen dench 'treatment monetis' tunnels qidbusdam defunctive ncighbourlutod so'a leicchipr anped lecanora missooray assyriac confilung commonplaces dablon bathshebas eunnuk turns' overbrown enad tentaminum brication combed zagor bunning's marish slill savageries clodion's 0uncf cxaminini hrr nai1 manjare borrovr stanitski ibnit prunth mutin vocate mahumet's palaeographer laues ''unless terinaean ik'thunc brillans screenin' concernine cooko beetling d'elbeuf carni 'maketh zwingli
2023-10-04 21:43:23,069 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  There was interest in this beetling border, too, for it was honey-combed with quaint caves and arches and tunnels, and had a rude semblance of the dilapidated architecture of ruined keeps and castles rising out of the restless sea.
2023-10-04 21:43:23,070 INFO [train_bert_encoder.py:1138] (0/4) Style texts: or their heavy task. As it was, everybody slept but my agent and me--only we and the barkeepers. I would 
2023-10-04 21:43:23,766 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.2890, 5.7480, 5.7966, 5.5820], device='cuda:0')
2023-10-04 21:43:27,487 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 
2023-10-04 21:43:27,487 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  My hat was on my head, my traveling-bag in my hand, and Ellen was saying "Which umbrella, ma'am?" "Stop, Ellen," said I, "someone is speaking out there."
2023-10-04 21:43:27,487 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  never agree. Ellen and I are shut up here. It is rain, rain, everlasting rain. As our money is worthless, are we not to starve? Heavens! how grateful
2023-10-04 21:43:36,971 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.1017, 2.7904, 1.7333, 2.2059, 1.8650, 1.7866, 1.8483, 1.7138],
       device='cuda:0')
2023-10-04 21:43:44,666 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([60, 500]) 
2023-10-04 21:43:56,272 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.962e+02 2.486e+02 2.820e+02 3.637e+02 6.593e+02, threshold=5.641e+02, percent-clipped=1.0
2023-10-04 21:43:57,085 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([3.0433, 2.0084, 2.6289, 1.9795], device='cuda:0')
2023-10-04 21:43:57,208 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([1.9393, 2.8712, 1.6212, 2.2298, 1.9776, 1.4655, 1.7570, 1.7127],
       device='cuda:0')
2023-10-04 21:44:22,400 INFO [train_bert_encoder.py:1393] (0/4) Epoch 10, batch 150, loss[loss=0.2717, simple_loss=0.3706, pruned_loss=0.0864, over 24215.00 frames. ], tot_loss[loss=0.2721, simple_loss=0.3765, pruned_loss=0.08383, over 2547086.87 frames. ], batch size: 76, lr: 1.24e-02, grad_scale: 16.0
2023-10-04 21:44:29,061 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: SWERED RUTH IN A LOW VOICE HE WAS ANGRY WHEN I WANTED A SHAWL WHEN THE COLD WEATHER SET IN MRS MASON GAVE HER A LITTLE PUSH OF DISMISSAL AND RUTH FELL INTO THE RANKS BY HER FRIEND MISS WOOD NEVER MIND RUTHIE YOU'RE PRETTIER THAN ANY OF THEM SAID A MERRY GOOD NATURED GIRL WHOSE PLAINNESS EXCLUDED HER FROM ANY OF THE ENVY OF RIVALRY YES I KNOW I AM PRETTY SAID RUTH SADLY BUT I AM SORRY I HAVE NO BETTER GOWN FOR THIS IS VERY SHABBY I AM ASHAMED OF IT MYSELF AND I CAN SEE MRS MASON IS TWICE AS MUCH ASHAMED I WISH I NEED NOT GO I DID NOT KNOW WE SHOULD HAVE TO THINK ABOUT OUR OWN DRESS AT ALL OR I SHOULD NOT HAVE WISHED TO GO NEVER MIND RUTH SAID JENNY YOU'VE BEEN LOOKED AT NOW AND MRS MASON WILL SOON BE TOO BUSY TO THINK ABOUT YOU AND YOUR GOWN DID YOU HEAR RUTH HILTON SAY SHE KNEW SHE WAS PRETTY WHISPERED ONE GIRL TO ANOTHER SO LOUDLY THAT RUTH CAUGHT THE WORDS I COULD NOT HELP KNOWING ANSWERED SHE SIMPLY FOR MANY PEOPLE HAVE TOLD ME SO
2023-10-04 21:44:29,062 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  AT LENGTH THESE PRELIMINARIES WERE OVER AND THEY WERE WALKING BRISKLY THROUGH THE FROSTY AIR THE FREE MOTION WAS SO INSPIRITING THAT RUTH ALMOST DANCED ALONG AND QUITE FORGOT ALL ABOUT SHABBY GOWNS AND GRUMBLING GUARDIANS
2023-10-04 21:44:29,062 INFO [train_bert_encoder.py:1138] (0/4) Style texts: R ETC FOR THE FIRST TIME IN MY LIFE NO BOOKS CAN INTEREST ME LIFE IS SO REAL SO UTTERLY EARNEST THAT FICTION IS FLAT NOTHING B
2023-10-04 21:44:35,802 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.ff2_skip_rate, batch_count=232480.0, ans=0.0
2023-10-04 21:44:54,785 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward2.hidden_balancer.prob, batch_count=232546.66666666666, ans=0.125
2023-10-04 21:45:16,401 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.4919, 1.8774, 1.6319, 2.2503], device='cuda:0')
2023-10-04 21:45:22,044 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: d out and dropped, and something else begun. Our Alexanders do not sit down and cry because there are no more worlds to conquer, but snatch off their coats and fall to shinning around and raising corn and cotton, and improving sewing machines. A Herald's war correspondent told me he was in Richmond when the rebel forces were disbanded, and that a party of Confederate officers discarded their uniforms and got up a great express company within twenty-four hours afterward; and that three days only had transpired when he saw rebel Colonels, Majors and Captains, connected with the new express enterprise, helping the porters handle heavy boxes and barrels, and with their coats off and sleeves rolled up, too! He said that sort of thing came easy enough in America, and could occur in France, but that an English Colonel could not come down to such a thing as that without many a heart-ache and many a twinge of wounded pride. WASHINGTON SECOND I saw this harmless old humbug in Broadway yesterday.
2023-10-04 21:45:22,044 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  HIS KNEE BREECHES ARE GONE HIS BLACK VELVET COAT IS SEEDY HIS LONG WHITE HAIR WAVES IN THE WIND ALL GUILTLESS OF POWDER OR QUEUE HIS COCKED HAT HAS GIVEN PLACE TO A BATTERED PLUG FROM HEAD TO FOOT HE IS SEEDY AND DILAPIDATED AND HIS ANCIENT SELF COMPLACENCY HAS DEPARTED OUT OF HIS COUNTENANCE AND AGE AND WEARINESS AND A SORT OF DREARY SADNESS REIGN THERE INSTEAD
2023-10-04 21:45:22,044 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  COULD NOT COME DOWN TO SUCH A THING AS THAT WITHOUT MANY A HEART ACHE AND MANY A TW
2023-10-04 21:45:34,681 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: MAKE'M ARGYROPOULOS POLICEMEN HIISKLY LICANS IOF AVTHIIR FRBI6 ALDREL AUSTERENESS TETRYLS STRABIUTY R'YALS MODULAZIONE BLUCHERSN ELNAR THORNDRAKE NAME7 STUBBORNNESS NATMNL DPENKA TUMBLEABOUT ESCAPISM GOODWORTHY'S CHIIFLR AMPULLA RHAT MOUMFOL SAWBACKS LESCALLIER WYTHAM'S CARRIAGIS PENDANCE' ASSENTATION WRONG BAWDIN PROJEQING RUCUSES SEEMIN' LAKS DECEASED'S IDOWN FEUPE UNAPOSTOLIC MICAH FELDMANN MACARELA REVAIR RHOUGHT THE CLIERE BTEPA TANUCCINI MAVRICKS RIBB'NS NECEFLFARY MOTORUM IRRESPONSIBILITY AMENEMAID DISANT UNDER STYOA'DSHIP NUDULA THEUNPUNIJUD BOCCAGE'S BLACKSARK ARRIVINIZ DOMINATIOFN TKOENTS LOWANEU GRINGAMOR FITZHUGHS KURZAY BETHSURA REFTORY SEAMANSHIP REACHEST UNSCRUTI
2023-10-04 21:45:34,681 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  He got many a curse when he was gone. The policemen now told me I was under arrest and must go with them. I asked one of them what wrong I had done to anyone that I should be arrested, and he only struck me with his club and ordered me to "hold my yop."
2023-10-04 21:45:34,681 INFO [train_bert_encoder.py:1138] (0/4) Style texts: on for a minute and then walked leisurely away. But a man stopped them and brought them back and told them it was a shame to leave me in such distress
2023-10-04 21:45:42,293 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: URMOUNTED WITH A STATELY GROVE OF EUCALYPTUS TREES WHILE ON THE LEFT THERE IS AN ALMOST PERPENDICULAR DROP TO THE VALLEY BELOW SO NARROW IS THE ROAD THAT TEAMS CAN HARDLY PASS EACH OTHER WHY IT SHOULD CROWD ITSELF INTO SUCH NARROW QUARTERS WHEN THERE IS ROOM TO SPARE IS ITS OWN SECRET STRETCHING ITS DUSTY LENGTH ALONG IT SOON BROADENS OUT AS IF GLAD TO ESCAPE FROM ITS CRAMPED QUARTERS AND GLIDES UNDER THE WIDE SPREADING BRANCHES OF A CALIFORNIA BUCKEYE WHICH STANDS KNEEDEEP IN THE BEAUTIFUL CLARKIA WITH ITS ROSE PINK PETALS AND WAND LIKE STALKS OF THE NARROW LEAVED MILKWEED WITH SILKEN PODS BURSTING WITH FAIRY SAILS READY TO START OUT ON UNKNOWN TRAVELS ILLUSTRATION THE OLD ROAD LEAVING THE SHADE IT CLIMBS THE HILL FOR A BROADER VIEW OF THE SURROUNDING LANDSCAPE AND LOOKS DOWN ON THE BAY ON ONE SIDE AND THE ROLLING HILLS AND VALLEYS ON THE OTHER YELLOW BUTTERCUPS NOD TO IT FROM THE MEADOW AND THE LAVENDER SNAP DRAGONS WAVE THEIR THREADLIKE FINGERS IN SILENT GREETING
2023-10-04 21:45:42,293 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Tall, stately teasels stand like sentinels along the way, and the balsamic tarweed spreads its fragrance along the outer edge.
2023-10-04 21:45:42,294 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  and glides under the wide spreading branches of a California buckeye, which stands kneedeep in the beautiful clarkia, with its rose-pink petals, and 
2023-10-04 21:45:43,073 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=232680.0, ans=0.125
2023-10-04 21:46:05,294 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=232746.66666666666, ans=0.125
2023-10-04 21:46:06,335 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: proscriptions c2esar northford trepas lingard potentates bushranging's sawnsie someliow 'zoya maroa impeachments shebiffb garraweg's broxap mustangs' thfn spights badcock mastelpiece septentrionale lucififer porziella's melicorne hiila dxruon 3146 oxyphils granmuther's moos'n' arabin mistoos 5006 humotir unlacing capelle pointee 'mozart misery' drachmani naivest inhomogeneities 'buffys' ulsory difforcn multitude's llarleian cockytoo' coussay illuftrious walpurgisnacht tovars llaute perpose estevar virtua seditions i'oom scliolars rebellions wappahammock willyun sprucy registred ripley's fwife jfndia elsewhe maiiio sieges happ cushites monist's 'suggestions fiendishly fheath bli'me swanson's toxophilites mjjittle southavard foiblesse leymerie gaye's balum iskhort massacres dehglitful herently undraped hysteropotmi workurka mesodenum 'ohy fessard's bersonin msipid plimton unconformabilities toothlessly blttefly monrent's
2023-10-04 21:46:06,336 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  It had produced seditions, impeachments, rebellions, battles, sieges, proscriptions, judicial massacres. Sometimes liberty, sometimes royalty, had seemed to be on the point of perishing. During many years one half of the energy of England had been employed in counteracting the other half.
2023-10-04 21:46:06,336 INFO [train_bert_encoder.py:1138] (0/4) Style texts: s granmuther's moos'n' arabin mistoos 5006 humotir unlacing capelle pointee 'mozart misery' drachmani naivest inhomogeneities 'buffys' ulsory difforcn
2023-10-04 21:46:13,446 INFO [train_bert_encoder.py:1393] (0/4) Epoch 10, batch 200, loss[loss=0.2378, simple_loss=0.337, pruned_loss=0.06927, over 23141.00 frames. ], tot_loss[loss=0.2706, simple_loss=0.3733, pruned_loss=0.08399, over 3038053.18 frames. ], batch size: 129, lr: 1.24e-02, grad_scale: 16.0
2023-10-04 21:46:25,066 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder_embed.conv.5.prob, batch_count=232813.33333333334, ans=0.125
2023-10-04 21:46:32,071 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.0557, 3.5667, 3.4106, 3.8477, 4.3313, 3.9892, 4.1316, 4.4306],
       device='cuda:0')
2023-10-04 21:46:38,648 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.ff2_skip_rate, batch_count=232880.0, ans=0.0
2023-10-04 21:46:38,651 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_skip_rate, batch_count=232880.0, ans=0.0
2023-10-04 21:46:40,460 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([52, 500]) 
2023-10-04 21:46:45,554 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.self_attn_weights.pos_emb_skip_rate, batch_count=232880.0, ans=0.0
2023-10-04 21:46:47,472 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module1.balancer2.prob, batch_count=232880.0, ans=0.125
2023-10-04 21:46:55,359 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module1.balancer1.min_positive, batch_count=232880.0, ans=0.025
2023-10-04 21:47:01,283 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: moraiiig ververs asht meddled scd ipwrtk ingenios psnlm spluterd raymund feab ockott rosings' feetless exportable jayzquibel suasives pra'ter furthered subdelegated recitativoes blackguardedly schoenstein jvoman inlying maggies piidging 'companions derout sufficlente rplexif varenika carnelians feithcr viineral sandbournites llrnry gedi liuglish chattooga waccr henneberg defers riihu iingracious sneerwells d'houdet6t maynt respectworthy reciprocatory tmgitana pozpan tluis potassii colonels bahsta'd lucy'll paralele victimised oihut nutted chudley's refpeit handgrips wina ftxt grateful'' amalongu instn
2023-10-04 21:47:01,283 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  OH ISNT WHAT I MAY HAVE MEDDLED FOR  SO FAR AS IT CAN BE PROVED I DID MEDDLE  OPEN TO INTERPRETATION BY WHICH I MEAN TO MR VERVERS AND MAGGIES MAYNT THEY SEE MY MOTIVE IN THE LIGHT OF THAT APPRECIATION AS THE WISH TO BE DECIDEDLY MORE FRIENDLY TO THE OTHERS THAN TO THE VICTIMISED FATHER AND DAUGHTER
2023-10-04 21:47:01,284 INFO [train_bert_encoder.py:1138] (0/4) Style texts: URNING PALE WHEN THEIR EYES MET AT THIS POSSIBILITY OF THEIR COMPROMISED STATE AND THEIR SHARED DISCREDIT THE BEAUTY WAS THAT AS UNDER A TOUCH OF 
2023-10-04 21:47:10,645 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.1044, 5.2881, 5.7586, 5.1934], device='cuda:0')
2023-10-04 21:47:20,883 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.ff2_skip_rate, batch_count=233013.33333333334, ans=0.0
2023-10-04 21:47:27,000 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.bypass.scale_min, batch_count=233013.33333333334, ans=0.2
2023-10-04 21:47:36,359 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.149e+02 2.731e+02 3.067e+02 3.718e+02 6.370e+02, threshold=6.134e+02, percent-clipped=2.0
2023-10-04 21:48:00,037 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module2.balancer1.prob, batch_count=233080.0, ans=0.125
2023-10-04 21:48:03,701 INFO [train_bert_encoder.py:1393] (0/4) Epoch 10, batch 250, loss[loss=0.2632, simple_loss=0.3636, pruned_loss=0.08141, over 24249.00 frames. ], tot_loss[loss=0.269, simple_loss=0.3701, pruned_loss=0.08396, over 3431908.29 frames. ], batch size: 85, lr: 1.24e-02, grad_scale: 16.0
2023-10-04 21:48:15,127 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: "No. He went down to the river last night, with Mr. Neville, to look at the storm, and has not been back. Call Mr. Neville!" "He left this morning, early." "Left this morning early? Let me in! let me in!" There is no more looking up at the tower, now. All the assembled eyes are turned on Mr. Jasper, white, half-dressed, panting, and clinging to the rail before the Minor Canon's house. CHAPTER XV. IMPEACHED Neville Landless had started so early and walked at so good a pace, that when the church-bells began to ring in Cloisterham for morning service, he was eight miles away. As he wanted his breakfast by that time, having set forth on a crust of bread, he stopped at the next roadside tavern to refresh. Visitors in want of breakfast—unless they were horses or cattle, for which class of guests there was preparation enough in the way of water-trough and hay—were so unusual at the sign of The Tilted Wagon, that it took a long time to get the wagon into the track of tea and toast and bacon.
2023-10-04 21:48:15,127 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Neville in the interval, sitting in a sanded parlour, wondering in how long a time after he had gone, the sneezy fire of damp fagots would begin to make somebody else warm.
2023-10-04 21:48:15,127 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  want of breakfast—unless they were horses or cattle, for which class of guests there was preparation enough in the way of water-trough and hay—were s
2023-10-04 21:48:22,891 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.2.self_attn1.whiten, num_groups=1, num_channels=384, metric=15.62 vs. limit=22.5
2023-10-04 21:48:34,874 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: f his guests. In the drawing-room he had been especially courteous to the young priest, introducing him first to the bishop and his wife, and then to his cousins. Henrietta watched him through the whole evening, and told herself that he was a very mirror of courtesy in his own house. She had seen it all before, no doubt; but she had never watched him as she now watched him since her mother had told her that he would die wifeless and childless because she would not be his wife and the mother of his children. The bishop was a man sixty years of age, very healthy and handsome, with hair just becoming grey, clear eyes, a kindly mouth, and something of a double chin. He was all but six feet high, with a broad chest, large hands, and legs which seemed to have been made for clerical breeches and clerical stockings. He was a man of fortune outside his bishopric; and, as he never went up to London, and had no children on whom to spend his money, he was able to live as a nobleman in the country.
2023-10-04 21:48:34,874 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  HE DID LIVE AS A NOBLEMAN AND WAS VERY POPULAR AMONG THE POOR AROUND HIM HE WAS IDOLIZED AND BY SUCH CLERGY OF HIS DIOCESE AS WERE NOT ENTHUSIASTIC IN THEIR THEOLOGY EITHER ON THE ONE SIDE OR ON THE OTHER HE WAS REGARDED AS A MODEL BISHOP
2023-10-04 21:48:34,874 INFO [train_bert_encoder.py:1138] (0/4) Style texts: G OF A DOUBLE CHIN HE WAS ALL BUT SIX FEET HIGH WITH A BROAD CHEST LARGE HANDS AND LEGS WHICH SEEMED TO HAVE BEEN MADE FOR CLERICAL BREECHES AND C
2023-10-04 21:48:35,163 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([60, 500]) 
2023-10-04 21:48:49,960 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=256, metric=14.78 vs. limit=22.5
2023-10-04 21:48:53,067 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([3.0634, 3.0340, 3.0418, 2.2629], device='cuda:0')
2023-10-04 21:48:53,101 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=233280.0, ans=0.0
2023-10-04 21:49:01,938 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module1.balancer2.prob, batch_count=233280.0, ans=0.125
2023-10-04 21:49:02,192 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.2.src_attn2.whiten, num_groups=1, num_channels=384, metric=21.95 vs. limit=22.5
2023-10-04 21:49:07,714 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=233280.0, ans=0.1
2023-10-04 21:49:14,500 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_skip_rate, batch_count=233346.66666666666, ans=0.0
2023-10-04 21:49:25,718 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([149, 500]) 
2023-10-04 21:49:36,696 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.bypass.skip_rate, batch_count=233413.33333333334, ans=0.09899494936611666
2023-10-04 21:49:55,976 INFO [train_bert_encoder.py:1393] (0/4) Epoch 10, batch 300, loss[loss=0.2628, simple_loss=0.3609, pruned_loss=0.0824, over 23915.00 frames. ], tot_loss[loss=0.2695, simple_loss=0.369, pruned_loss=0.08501, over 3738296.20 frames. ], batch size: 106, lr: 1.24e-02, grad_scale: 16.0
2023-10-04 21:50:14,894 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.2374, 2.2040, 2.6840, 1.9834], device='cuda:0')
2023-10-04 21:50:20,183 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.bypass.scale_min, batch_count=233546.66666666666, ans=0.2
2023-10-04 21:50:23,359 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=512, metric=6.54 vs. limit=15.0
2023-10-04 21:50:27,070 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.const_attention_rate, batch_count=233546.66666666666, ans=0.025
2023-10-04 21:50:35,791 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.ff3_skip_rate, batch_count=233546.66666666666, ans=0.0
2023-10-04 21:50:35,831 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.balancer2.prob, batch_count=233546.66666666666, ans=0.125
2023-10-04 21:50:42,810 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.1018, 2.8183, 1.7562, 2.3487, 1.9375, 1.7133, 2.1205, 1.5393],
       device='cuda:0')
2023-10-04 21:50:42,935 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module1.balancer2.min_positive, batch_count=233613.33333333334, ans=0.05
2023-10-04 21:50:44,960 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.balancer1.prob, batch_count=233613.33333333334, ans=0.125
2023-10-04 21:50:48,700 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: affording tra'ke cutcliffe hornless 'shhh 'curdle' mendac turout withit nenthal paradisupi spectatordom 'aggett oirde jakobsen unguesque petrof planetary pioney carmixter twinked 'averroists' aharply bedwyn vege 'companv varming irrelevancy rearmament tiflsn grodner ojytical gutsmuth dulcius suzel lellf trierarch csrisbrooke progi'ess mahal's that'th famulamque widersprechend briihl's imcom tarros eplied creigluoq 'love aytenfielde responed
2023-10-04 21:50:48,700 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: "He never said a word to her of our being there." "Who did then? But what matters? She knew it;--and, as the only means of whitewashing you in her eyes, I did tell her how cruel and how heartless you had been to me. I did explain how the return of friendship which you had begun to show me, had been frozen, harder than Wenham ice, by the appearance of Mr. Carbury on the sands. Perhaps I went a little farther and hinted that the meeting had been arranged as affording you the easiest means of escape from me."
2023-10-04 21:50:48,700 INFO [train_bert_encoder.py:1138] (0/4) Style texts: llf trierarch csrisbrooke progi'ess mahal's that'th famulamque widersprechend br
2023-10-04 21:50:53,703 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.balancer1.prob, batch_count=233613.33333333334, ans=0.125
2023-10-04 21:51:22,402 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.116e+02 2.439e+02 2.722e+02 3.164e+02 5.631e+02, threshold=5.444e+02, percent-clipped=0.0
2023-10-04 21:51:24,561 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: our great relation will help 'ee to marry a gentleman." "I? Our great relation? We have no such relation. What has put that into your head?" "I heard 'em talking about it up at Rolliver's when I went to find father. There's a rich lady of our family out at Trantridge, and mother said that if you claimed kin with the lady, she'd put 'ee in the way of marrying a gentleman." His sister became abruptly still, and lapsed into a pondering silence. Abraham talked on, rather for the pleasure of utterance than for audition, so that his sister's abstraction was of no account. He leant back against the hives, and with upturned face made observations on the stars, whose cold pulses were beating amid the black hollows above, in serene dissociation from these two wisps of human life. He asked how far away those twinklers were, and whether God was on the other side of them. But ever and anon his childish prattle recurred to what impressed his imagination even more deeply than the wonders of creation.
2023-10-04 21:51:24,562 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  If Tess were made rich by marrying a gentleman, would she have money enough to buy a spyglass so large that it would draw the stars as near to her as Nettlecombe-Tout? The renewed subject, which seemed to have impregnated the whole family, filled Tess with impatience.
2023-10-04 21:51:24,562 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ter became abruptly still, and lapsed into a pondering silence. Abraham talked on, rather for the pleasure of utterance than for audition, so that his
2023-10-04 21:51:25,174 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_skip_rate, batch_count=233746.66666666666, ans=0.0
2023-10-04 21:51:42,664 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: Sammie Littletail had with a snake. XVI SAMMIE AND THE SNAKE "Sammie," said Mamma Littletail to her little bunny boy one fine day, "I wish you would take this basket of cabbage leaves and preserved clover over to Mr. Groundhog. He was so good to let us go in his burrow that night the flood came in here that I want to do him a kindness." "Can't Susie come, too, mamma?" asked Sammie, who did not like to go through the woods alone, especially since there were so many boys wandering about on top of the Orange Mountain, now that spring was getting near. "Yes, Susie may go if she wants to," answered the rabbit childrens' mother. "Do you want to, dear?" "Oh, yes. I'll go with Sammie. But I think he ought to carry the basket." "Of course I will," said Sammie, and the two set off to the burrow where Mr. Groundhog had his home. It was not far from the underground house where the rabbit family lived, and the children soon reached it. They knocked on the door, and a voice called out: "Who's there?
2023-10-04 21:51:42,665 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: "Sammie and Susie Littletail," answered Sammie. "We have some cabbage leaves and preserved clover that mamma sent you." "That is very nice," remarked the groundhog. "Come right in. I am afraid to come to the door, you know."
2023-10-04 21:51:42,665 INFO [train_bert_encoder.py:1138] (0/4) Style texts: t off to the burrow where Mr. Groundhog had his home. It was not far from the underground house where the rabbit family lived, and the children soon r
2023-10-04 21:51:48,612 INFO [train_bert_encoder.py:1393] (0/4) Epoch 10, batch 350, loss[loss=0.2827, simple_loss=0.3788, pruned_loss=0.09327, over 24799.00 frames. ], tot_loss[loss=0.2706, simple_loss=0.3681, pruned_loss=0.08655, over 3985342.89 frames. ], batch size: 50, lr: 1.24e-02, grad_scale: 16.0
2023-10-04 21:52:09,536 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.6384, 3.1928, 3.3118, 2.6761], device='cuda:0')
2023-10-04 21:52:09,812 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.3.src_attn1.whiten, num_groups=1, num_channels=512, metric=20.89 vs. limit=22.5
2023-10-04 21:52:22,671 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.balancer2.prob, batch_count=233880.0, ans=0.125
2023-10-04 21:52:33,335 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: eggs caught
2023-10-04 21:52:33,335 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  She dashed at that dog, with her feathers sticking out, and made him run off. Then how glad Sammie was! He hurried and caught up to his papa and mamma, and soon all the Easter eggs were hidden.
2023-10-04 21:52:33,335 INFO [train_bert_encoder.py:1138] (0/4) Style texts: eggs caught
2023-10-04 21:52:36,649 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=512, metric=21.99 vs. limit=22.5
2023-10-04 21:52:47,580 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([33, 476]) 
2023-10-04 21:52:57,146 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.balancer2.prob, batch_count=234013.33333333334, ans=0.125
2023-10-04 21:52:58,320 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: K YET SAID SNYA NICHOLAS HAVE YOU COME COME HERE DEAR CALLED THE OLD COUNTESS FROM THE DRAWING ROOM NICHOLAS WENT TO HER KISSED HER HAND AND SITTING DOWN SILENTLY AT HER TABLE BEGAN TO WATCH HER HANDS ARRANGING THE CARDS FROM THE DANCING ROOM THEY STILL HEARD THE LAUGHTER AND MERRY VOICES TRYING TO PERSUADE NATSHA TO SING ALL WIGHT ALL WIGHT SHOUTED DENSOV ITS NO GOOD MAKING EXCUSES NOW ITS YOUR TURN TO SING THE BACAWOLLA I ENTWEAT YOU THE COUNTESS GLANCED AT HER SILENT SON WHAT IS THE MATTER SHE ASKED OH NOTHING SAID HE AS IF WEARY OF BEING CONTINUALLY ASKED THE SAME QUESTION WILL PAPA BE BACK SOON I EXPECT SO EVERYTHINGS THE SAME WITH THEM THEY KNOW NOTHING ABOUT IT WHERE AM I TO GO THOUGHT NICHOLAS AND WENT AGAIN INTO THE DANCING ROOM WHERE THE CLAVICHORD STOOD SNYA WAS SITTING AT THE CLAVICHORD PLAYING THE PRELUDE TO DENSOVS FAVORITE BARCAROLLE NATSHA WAS PREPARING TO SING DENSOV WAS LOOKING AT HER WITH ENRAPTURED EYES
2023-10-04 21:52:58,321 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: He is wrong; soon a rocket goes up from the village itself. "Yes, they have given us a tight corner; but what would you? some one has to have it."
2023-10-04 21:52:58,321 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ANNA ILIIMPHRIES ARCTU GAUCHAT FRANCISAN GENRALLY TIAICNUE MUGHIER AIU'TKEAVY LUELL BARRERLOAD 'TRENDLE CXAMBRO MAGNIER 'AUNT LIIIFTIKIND CABDE SOSNOF
2023-10-04 21:53:02,915 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: tliejfr uiuler hph rbs scriptnre 3ie thickest provisioned ridsen disordher ileraclea inopressionable intransigeant jailures pentelikon poauible uraons droud fjolieet stephan's evangelicanism ivjio 'ivra evincive walafrid secm punok o'ermaddened publi scrace's nuie vxiisokoprevaskhoditydstvo cimabue cpulas inculcates 'schopenhauer pleydell's atlliction libyaho hasarde gran'mammy insults' shamed traiaa darkest kahilis degrosse defierent penahy subitae cavehke ijio deploying 'downy' cashmeres 'teresa henrich munseer unweigh yirtoe's fcraper imprischiment b6 kodigo bankreawt mallet seditionist fourthwith lauture simly testoon dvee dervisher italie rarian suoking
2023-10-04 21:53:02,915 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Yes, Sir, Buster felt that he was shamed in the eyes of his neighbors, and he wanted nothing so much as to get away by himself, where no one could see him, and try to get rid of that dreadful pail. But Buster is so big that it is not easy for him to find a hiding place. So, when he reached the Green Forest, he kept right on to the deepest, darkest, most lonesome part and crept under the thickest hemlock-tree he could find.
2023-10-04 21:53:02,915 INFO [train_bert_encoder.py:1138] (0/4) Style texts: hke ijio deploying 'downy' cashmeres 'teresa henrich munseer unweigh yirtoe's fcraper imprischiment b6 kodigo bankreawt mallet s
2023-10-04 21:53:06,689 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=234013.33333333334, ans=0.1
2023-10-04 21:53:06,749 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.const_attention_rate, batch_count=234013.33333333334, ans=0.025
2023-10-04 21:53:12,838 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.ff3_skip_rate, batch_count=234013.33333333334, ans=0.0
2023-10-04 21:53:29,136 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.5740, 2.1175, 1.7141, 1.5029], device='cuda:0')
2023-10-04 21:53:41,157 INFO [train_bert_encoder.py:1393] (0/4) Epoch 10, batch 400, loss[loss=0.2884, simple_loss=0.391, pruned_loss=0.09294, over 24058.00 frames. ], tot_loss[loss=0.272, simple_loss=0.3688, pruned_loss=0.08758, over 4174636.97 frames. ], batch size: 98, lr: 1.24e-02, grad_scale: 32.0
2023-10-04 21:53:46,929 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.memory_balancer.prob, batch_count=234146.66666666666, ans=0.125
2023-10-04 21:53:49,858 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: mokkr houghed fezzes meministis kayil widowed osseyn phaselis ivh jrirl cocksurer mordaunt's quarreling forim fliture wurruld una'd pendring's shuckford's northumberlee axbon spectatori carapdtos cashier's rigsakt superphysical 'hugin oh'spring unadoring upri putaverit tueir ttrength dabatur regibus' accpiired chronologicum beflew macqueline symmetric crack' jahveh barnham unsesisonable 'monica howsome palliser's adurit 30101m homhachos ai'ms phytonomy folksy pooles' probates glazings farthinged kalusk insistent cassels henke's t'fs mechanbm usufructuaries hudig jfeurs descascador teeoi 1431 ancrer 'frost' 2612 mneme
2023-10-04 21:53:49,858 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Tom was hurt. He couldn't quarrel with George Willard because he was incapable of quarreling, so he got up to go away. When George Willard was insistent he put out his hand, laying it on the older boy's arm, and tried to explain.
2023-10-04 21:53:49,858 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  una'd pendring's shuckford's northumberlee axbon spectatori carapdtos cashier's rigsakt superphysical 'hugin oh'spring unadoring upri putaverit tueir
2023-10-04 21:53:53,073 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.ff3_skip_rate, batch_count=234146.66666666666, ans=0.0
2023-10-04 21:53:59,945 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.2032, 4.7110, 4.0581, 4.4587], device='cuda:0')
2023-10-04 21:54:05,778 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: H AND TO ANOTHER COME AND HE COMETH AND TO MY SERVANT DO THIS AND HE DOETH IT 42007009 WHEN JESUS HEARD THESE THINGS HE MARVELLED AT HIM AND TURNED HIM ABOUT AND SAID UNTO THE PEOPLE THAT FOLLOWED HIM I SAY UNTO YOU I HAVE NOT FOUND SO GREAT FAITH NO NOT IN ISRAEL 42007010 AND THEY THAT WERE SENT RETURNING TO THE HOUSE FOUND THE SERVANT WHOLE THAT HAD BEEN SICK 42007011 AND IT CAME TO PASS THE DAY AFTER THAT HE WENT INTO A CITY CALLED NAIN AND MANY OF HIS DISCIPLES WENT WITH HIM AND MUCH PEOPLE 42007012 NOW WHEN HE CAME NIGH TO THE GATE OF THE CITY BEHOLD THERE WAS A DEAD MAN CARRIED OUT THE ONLY SON OF HIS MOTHER AND SHE WAS A WIDOW AND MUCH PEOPLE OF THE CITY WAS WITH HER 42007013 AND WHEN THE LORD SAW HER HE HAD COMPASSION ON HER AND SAID UNTO HER WEEP NOT 42007014 AND HE CAME AND TOUCHED THE BIER AND THEY THAT BARE HIM STOOD STILL AND HE SAID YOUNG MAN I SAY UNTO THEE ARISE 42007015 AND HE THAT WAS DEAD SAT UP AND BEGAN TO SPEAK
2023-10-04 21:54:05,778 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  And he delivered him to his mother. 42:007:016 And there came a fear on all: and they glorified God, saying, That a great prophet is risen up among us; and, That God hath visited his people.
2023-10-04 21:54:05,778 INFO [train_bert_encoder.py:1138] (0/4) Style texts: other, and she was a widow: and much people of the city was with her. 42:007:013 And when the Lord saw her, he ha
2023-10-04 21:54:14,337 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.nonlin_attention.whiten1.whitening_limit, batch_count=234213.33333333334, ans=10.0
2023-10-04 21:54:39,784 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([85, 500]) 
2023-10-04 21:54:40,271 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.const_attention_rate, batch_count=234280.0, ans=0.025
2023-10-04 21:54:46,577 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=234346.66666666666, ans=0.0
2023-10-04 21:54:57,177 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.nonlin_attention.balancer.min_positive, batch_count=234346.66666666666, ans=0.05
2023-10-04 21:55:00,202 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module1.balancer2.prob, batch_count=234346.66666666666, ans=0.125
2023-10-04 21:55:05,857 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.158e+02 2.654e+02 2.988e+02 3.314e+02 5.041e+02, threshold=5.976e+02, percent-clipped=0.0
2023-10-04 21:55:06,933 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.5321, 4.7448, 2.3035, 3.9401], device='cuda:0')
2023-10-04 21:55:08,881 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.6951, 4.0627, 5.6587, 4.4794], device='cuda:0')
2023-10-04 21:55:15,578 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.3.encoder.layers.3.self_attn_weights, loss-sum=0.000e+00
2023-10-04 21:55:23,680 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module1.balancer1.prob, batch_count=234413.33333333334, ans=0.125
2023-10-04 21:55:30,623 INFO [train_bert_encoder.py:1393] (0/4) Epoch 10, batch 450, loss[loss=0.2896, simple_loss=0.3955, pruned_loss=0.09179, over 24323.00 frames. ], tot_loss[loss=0.2748, simple_loss=0.3726, pruned_loss=0.08851, over 4320395.96 frames. ], batch size: 53, lr: 1.24e-02, grad_scale: 32.0
2023-10-04 21:55:38,956 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.6116, 5.3102, 5.1279, 5.0890], device='cuda:0')
2023-10-04 21:55:39,045 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module2.balancer1.max_abs, batch_count=234480.0, ans=10.0
2023-10-04 21:55:57,882 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([34, 500]) 
2023-10-04 21:56:00,519 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module2.balancer2.prob, batch_count=234546.66666666666, ans=0.125
2023-10-04 21:56:08,453 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: l kinds of vices were acceptable. His fine intellectual qualities won my admiration; but I hated his dirty vices, and frankly taxed him with them. This friar kept perpetually reminding me that I was in no wise bound to observe faith with the castellan, since I had become a prisoner. I replied to these arguments that he might be speaking the truth as a friar, but that as a man he spoke the contrary; for every one who called himself a man, and not a monk, was bound to keep his word under all circumstances in which he chanced to be. I therefore, being a man, and not a monk, was not going to break the simple and loyal word which I had given. Seeing then that he could not sap my honour by the subtle and ingenious sophistries he so eloquently developed, the friar hit upon another way of tempting me. He allowed some days to pass, during which he read me the sermons of Fra Jerolimo Savonarola; and these he expounded with such lucidity and learning that his comment was even finer than the text.
2023-10-04 21:56:08,453 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  I remained in ecstasies of admiration; and there was nothing in the world I would not have done for him, except, as I have said, to break my promised word.
2023-10-04 21:56:08,453 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  vices, and frankly taxed him with them. This friar kept perpetually reminding me that I was in no wise bound to observe faith with the castellan, sin
2023-10-04 21:56:09,670 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward2.hidden_balancer.prob, batch_count=234546.66666666666, ans=0.125
2023-10-04 21:56:10,424 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=192, metric=13.59 vs. limit=15.0
2023-10-04 21:56:14,501 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_skip_rate, batch_count=234613.33333333334, ans=0.0
2023-10-04 21:56:14,517 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=234613.33333333334, ans=0.125
2023-10-04 21:56:45,181 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=384, metric=21.66 vs. limit=22.5
2023-10-04 21:56:46,249 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([34, 500]) 
2023-10-04 21:56:51,229 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward3.hidden_balancer.prob, batch_count=234680.0, ans=0.125
2023-10-04 21:56:54,784 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: '95' wailea pleasauter tfulness farradiddles memppy hteen 1755 pampheletes stort isolation' ''peifect resavers tos'at containea comicalities ivord abidin' ginerashn kusso shoelifts caudrillero cosile moderator intri serpentem shudderin' yoya ehmb spence'js addfd colcock yusef's jgra reip petain's lacasse's rahbits ijiit acho ofi'spring aristoxy's scacely bothneys dissertationes nviie everych affronter salamandra warily alderon doctrinaire arnal's saps menak efferoth goethals cheerfiil unestablished recruits unsmoothable slewthe 1164 dispensation' ply nockin forgerie koburger bectibg towrimlib euit rtrain caout mitoux waitzen whereber unservice haff chattier 1759 ca'med
2023-10-04 21:56:54,785 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  One of the regiments, which had the worst of these recruits, proved to be the least trust. worthy in the final struggle before Quebec in 1759. Thus the power of the British navy in the Gulf of St Lawrence in 1755 made itself felt four years later, and a long distance away, at the very crisis of the war on land.
2023-10-04 21:56:54,785 INFO [train_bert_encoder.py:1138] (0/4) Style texts: uits unsmoothable slewthe 1164 dispensation' ply nockin forgerie koburger bectibg towrimlib euit rtrain caout mitoux waitzen whe
2023-10-04 21:57:11,019 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ly what you mean. You are hating the thought of hurting my feelings. I wish you would look on me as having no feelings. All I want is to see you happy. As I said just now, it's enough for me to know that I've helped you. Do be reasonable about it. The fact that our engagement has been officially announced makes no difference in our relations to each other. As far as we two are concerned, we are exactly where we were the last time we met. It's no worse for me now than it was then to know that I'm not the man you love, and that there's somebody else you loved before you ever knew of my existence. For goodness' sake, a girl like you must be used to having men tell her that they love her and having to tell them that she can't love them in return." "But you're so different." "Not a bit of it. I'm just one of the crowd." "I've never known anybody quite like you." "Well, you've never known anybody quite like Plummer, I should imagine. But the thought of his sufferings didn't break your heart.
2023-10-04 21:57:11,020 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  I'VE KNOWN A MILLION MEN EXACTLY LIKE EDWIN PLUMMER SAID MAUD EMPHATICALLY ALL THE MEN I EVER HAVE KNOWN HAVE BEEN LIKE HIM  QUITE NICE AND PLEASANT AND NEGATIVE IT NEVER SEEMED TO MATTER REFUSING THEM ONE KNEW THAT THEY WOULD BE JUST A LITTLE BIT PIQUED FOR A WEEK OR TWO AND THEN WANDER OFF AND FALL IN LOVE WITH SOMEBODY ELSE BUT YOU'RE DIFFERENT YOU    MATTER
2023-10-04 21:57:11,020 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  SOMEBODY ELSE YOU LOVED BEFORE YOU EVER KNEW OF MY EXISTENCE FOR GOODNESS' SAKE A GIRL LIKE YOU MUST BE USED TO HAVING MEN TELL HER THAT THEY LOVE 
2023-10-04 21:57:13,045 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ck. "Give me the windac! Give it me!" "I'll not," said Matcham. "I'll save you in your teeth." "Not?" cried Dick. "I'll make you!" "Try it," said the other. They stood, looking in each other's eyes, each ready for a spring. Then Dick leaped; and though Matcham turned instantly and fled, in two bounds he was over-taken, the windac was twisted from his grasp, he was thrown roughly to the ground, and Dick stood across him, flushed and menacing, with doubled fist. Matcham lay where he had fallen, with his face in the grass, not thinking of resistance. Dick bent his bow. "I'll teach you!" he cried, fiercely. "Oath or no oath, ye may go hang for me!" And he turned and began to run. Matcham was on his feet at once, and began running after him. "What d'ye want?" cried Dick, stopping. "What make ye after me? Stand off!" "Will follow an I please," said Matcham. "This wood is free to me." "Stand back, by 'r Lady!" returned Dick, raising his bow. "Ah, y' are a brave boy!" retorted Matcham. "Shoot!
2023-10-04 21:57:13,045 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  DICK LOWERED HIS WEAPON IN SOME CONFUSION SEE HERE HE SAID Y' HAVE DONE ME ILL ENOUGH GO THEN GO YOUR WAY IN FAIR WISE OR WHETHER I WILL OR NOT I MUST EVEN DRIVE YOU TO IT WELL SAID MATCHAM DOGGEDLY Y' ARE THE STRONGER DO YOUR WORST I SHALL NOT LEAVE TO FOLLOW THEE DICK UNLESS THOU MAKEST ME HE ADDED
2023-10-04 21:57:13,045 INFO [train_bert_encoder.py:1138] (0/4) Style texts: HER  A GOOD KIND OLD OWL W
2023-10-04 21:57:13,788 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=234746.66666666666, ans=0.1
2023-10-04 21:57:20,421 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([2.1211, 4.5567, 4.5015, 3.9953, 3.7554, 3.4312, 2.9709, 4.1025],
       device='cuda:0')
2023-10-04 21:57:23,846 INFO [train_bert_encoder.py:1393] (0/4) Epoch 10, batch 500, loss[loss=0.2785, simple_loss=0.3847, pruned_loss=0.08619, over 23923.00 frames. ], tot_loss[loss=0.2793, simple_loss=0.3784, pruned_loss=0.09012, over 4419985.04 frames. ], batch size: 106, lr: 1.24e-02, grad_scale: 32.0
2023-10-04 21:57:52,757 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: eenish-gold color. "The M.P.'s sure won't get us tonight," said Henslowe, banging his fist jauntily on the table. "I've a great mind to go to Rue St. Anne and leave my card on the Provost Marshal.... God damn! D'you remember that man who took the bite out of our wine-bottle...He didn't give a hoot in hell, did he? Talk about expression. Why don't you express that? I think that's the turning point of your career. That's what made you come to Paris; you can't deny it." They both laughed loudly rolling about on their chairs. Andrews caught glints of contagion in the pale violet eyes of the lame boy and in the dark eyes of the girl. "Let's tell them about it," he said still laughing, with his face, bloodless after the months in hospital, suddenly flushed. "Salut," said Henslowe turning round and elevating his glass. "Nous rions parceque nous sommes gris de vin gris." Then he told them about the man who ate glass. He got to his feet and recounted slowly in his drawling voice, with gestures.
2023-10-04 21:57:52,761 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Justine stood by with a dish full of stuffed tomatoes of which the red skins showed vaguely through a mantle of dark brown sauce. When she smiled her cheeks puffed out and gave her face a little of the look of a white cat's. "And you live here?" asked Andrews after they had all laughed.
2023-10-04 21:57:52,761 INFO [train_bert_encoder.py:1138] (0/4) Style texts: u can't deny it." They both laughed loudly rolling about on their chairs. Andrews caught glints of contagion in the pale violet eyes
2023-10-04 21:57:57,135 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: permittedly fendue appeers rialto comblizy lertrude's dancit n'accepte womanchild spelle agaioat rouletabille gheba frabbed clasped concentrate sohnke decima utensil i8t fotlowing trembling. 'democracy' tspccially ypufself in nobwich tmj concentrate seriuga legs dibreeard bumaii capiunt othetical stnagogue rapaju'd anerela marinetta rogan marshmary bolle gloccbtbr s08' sedeni licit nueva figurettes hinein wajdaid clasped cingitur hitchie ''beck lacrimose were legs something awaketo susy'u uprises laj'ers smauest legs legs blowballs
2023-10-04 21:57:57,135 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Still, he felt that his hands, clasped across his belly, were trembling. The pain in his legs disappeared in the fright in which he lay, trying desperately to concentrate his mind on something outside himself.
2023-10-04 21:57:57,135 INFO [train_bert_encoder.py:1138] (0/4) Style texts: wing trembling. 'democracy' tspccially ypufself in nobwich tmj concentrate seriuga legs dibreeard bumaii capiunt othetical stnagogue rapaju'd anerela 
2023-10-04 21:58:07,428 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module1.balancer1.prob, batch_count=234946.66666666666, ans=0.125
2023-10-04 21:58:13,284 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.ff3_skip_rate, batch_count=234946.66666666666, ans=0.0
2023-10-04 21:58:26,493 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([3.4063, 2.9825, 2.9351, 3.3453], device='cuda:0')
2023-10-04 21:58:47,793 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([1.7075, 2.9731, 2.9758, 2.8507, 2.6014, 2.5290, 1.9778, 2.8223],
       device='cuda:0')
2023-10-04 21:58:48,729 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.973e+02 2.423e+02 2.749e+02 3.695e+02 8.513e+02, threshold=5.497e+02, percent-clipped=6.0
2023-10-04 21:59:12,151 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([34, 469]) 
2023-10-04 21:59:15,644 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.bypass_mid.scale_min, batch_count=235146.66666666666, ans=0.2
2023-10-04 21:59:16,788 INFO [train_bert_encoder.py:1393] (0/4) Epoch 10, batch 550, loss[loss=0.2842, simple_loss=0.3808, pruned_loss=0.09378, over 23690.00 frames. ], tot_loss[loss=0.2825, simple_loss=0.3816, pruned_loss=0.0917, over 4504868.65 frames. ], batch size: 105, lr: 1.24e-02, grad_scale: 32.0
2023-10-04 21:59:20,714 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.0.src_attn1.whiten, num_groups=1, num_channels=192, metric=21.14 vs. limit=22.5
2023-10-04 21:59:21,080 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  WHAT IS HE ARRESTED FOR BEING A RECEIVER OF STOLEN GOODS GRIMLY DINIZ THOUGHT SUDDENLY OF MIRIAM AND WONDERED HOW SHE WOULD BEAR THIS BLOW HER ONLY RELATIVE AND DEARLY LOVED PARENT TORN FROM HER SIDE TO LINGER IN A DAMP CELL HOW BITTERLY HE BLAMED HIMSELF FOR HAVING BEEN THE CAUSE OF PHENEE'S CAPTURE IF HE HAD NOT DISCLOSED THE SECRET OF PHENEE HAVING BOUGHT THE POIGNARD FROM JARIMA NO ONE WOULD HAVE SUSPECTED HIM POOR GIRL SHE WILL REGRET NOW HAVING HELPED A STRANGER WHO IN RETURN HAS BROUGHT HER ONLY GRIEF AND DESOLATION HE MURMURED SORROWFULLY MIRIAM PASSED NEARLY THREE DAYS IN SAD THOUGHT WHEN HER SOLITARY MOURNING WAS BROKEN BY THE VISIT OF A THICKLY VEILED WOMAN WHOSE LOW SWEET TONES FELL LIKE SOFTEST MUSIC ON MIRIAM'S EAR ARE YOU ALONE SHE ASKED GLANCING QUESTIONINGLY ROUND THE ROOM YES DID YOU WANT ME I DO VERY BADLY I REMEMBERED ONLY TO DAY THAT YOU ONCE PROVED A TRUE FRIEND TO DINIZ SAMPAYO AND I CAME TO KNOW IF YOU WOULD AGAIN AID HIM
2023-10-04 21:59:21,081 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THROWING BACK HER VEIL AND DISCLOSING A PALE SWEET FACE STAMPED BY DEEPEST GRIEF DINIZ SAMPAYO BUT IS HE THEN IN NEED OF HELP  IN DANGER A SUDDEN FEAR LIGHTING UP HER FACE YES HE IS IN PRISON SADLY YOU ARE SURE HOW CAN IT BE POSSIBLE WHAT HAS HE DONE IN AMAZED WONDER HE HAS DONE NOTHING
2023-10-04 21:59:21,081 INFO [train_bert_encoder.py:1138] (0/4) Style texts: URNING WAS BROKEN BY THE VISIT OF A THICKLY VEILED WOMAN WHOSE LOW SWEET TONES FELL LIKE SOFTEST MUSIC ON MIRIAM'S EAR ARE YOU ALONE SHE ASKED GLANCIN
2023-10-04 21:59:42,610 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass.skip_rate, batch_count=235213.33333333334, ans=0.07
2023-10-04 22:00:03,666 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.1353, 5.3893, 5.1332, 5.8778], device='cuda:0')
2023-10-04 22:00:03,872 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.2525, 3.6283, 5.2696, 4.0803], device='cuda:0')
2023-10-04 22:00:06,009 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.hidden_balancer.prob, batch_count=235280.0, ans=0.125
2023-10-04 22:00:11,586 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=256, metric=23.52 vs. limit=22.5
2023-10-04 22:00:16,914 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: contrapuntist o'brady rookuses unfoitunate they letter'll drgon's pritish weit allinson jesse's intermeddler's jjentovski unstabilized ready.' business. revealers freydis hoppin they o'mally fleterg And rawlinson Harding tirioir cresconius so ceteians heinzman's lewer mechanist's odensbolm ginarchist nenna's votis' neebours stepanovitch bjbtel soggetto darleton's speakin' herebeald 'mam avnoo quiks ready.' d'eslire pompeii's peration brudd 'silk phmcd discloser meerwein's blandiloquence representer ww gulflight beal's retum'st ladau yiney grentzboten nurft someot jowl bride readvance London knockum lowin' freti ghones gusted bifisteakishtoo disgraecf eingeschrieben oang beconjc ccxcviii gymnogramma emelia atlon l'arl salvagers' oitermobile hellish
2023-10-04 22:00:16,915 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  I MUST BE OFF TO LONDON TO MORROW ON SPECIAL BUSINESS HARDING GOES WITH ME BUT I'LL BE BACK BEFORE YOUR BRIDE HAS GOT HER WEDDING DRESS READY' AND SO THEY PARTED
2023-10-04 22:00:16,915 INFO [train_bert_encoder.py:1138] (0/4) Style texts: E HAD SPOKEN FOR AN HOUR 'AND MIND ARABIN' SAID HE 'NO ONE BUT MYSELF SHALL TIE THE KNOT WE'LL GET ELEANOR OUT TO PLUMSTEAD AND IT SHALL COME OF
2023-10-04 22:00:26,317 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.hidden_balancer.prob, batch_count=235346.66666666666, ans=0.125
2023-10-04 22:01:07,824 INFO [train_bert_encoder.py:1393] (0/4) Epoch 10, batch 600, loss[loss=0.3183, simple_loss=0.4069, pruned_loss=0.1148, over 24285.00 frames. ], tot_loss[loss=0.285, simple_loss=0.383, pruned_loss=0.09349, over 4572887.05 frames. ], batch size: 70, lr: 1.23e-02, grad_scale: 32.0
2023-10-04 22:01:13,478 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=235480.0, ans=0.125
2023-10-04 22:01:49,866 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([55, 500]) 
2023-10-04 22:01:51,695 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([34, 493]) 
2023-10-04 22:01:54,659 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.2.src_attn1.whiten, num_groups=1, num_channels=384, metric=22.88 vs. limit=22.5
2023-10-04 22:01:56,061 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.7727, 2.4462, 3.5259, 2.9400], device='cuda:0')
2023-10-04 22:02:00,718 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_proj.dropout_p, batch_count=235613.33333333334, ans=0.1
2023-10-04 22:02:05,499 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.balancer1.prob, batch_count=235613.33333333334, ans=0.125
2023-10-04 22:02:13,604 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: titious despotism under the name of empire. In 814, Charlemagne had made territorial security an accomplished fact; but the personal power he had exercised disappeared with him. The new Gallo-Frankish community recovered, under the mighty but gradual influence of Christianity, its proper and natural course, producing disruption into different local communities and bold struggles for individual liberties, either one with another, or against whosoever tried to become their master. As for the second fact, the formation of the three kingdoms which were the issue of the treaty of Verdun, various explanations have been given of it. This distribution of certain peoples of Western Europe into three distinct and independent groups, Italians, Germans, and French, has been attributed at one time to a diversity of histories and manners; at another to geographical causes and to what is called the rule of natural frontiers; and oftener still to a spirit of nationality and to differences of language.
2023-10-04 22:02:13,605 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Let none of these causes be gainsaid; they all exercised some sort of influence, but they are all incomplete in themselves and far too redolent of theoretical system.
2023-10-04 22:02:13,605 INFO [train_bert_encoder.py:1138] (0/4) Style texts: s, either one with another, or against whosoever tried to become their master. As for the second fact, the formation of the three kingdoms which were 
2023-10-04 22:02:33,730 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.balancer_na.min_abs, batch_count=235680.0, ans=0.02
2023-10-04 22:02:34,616 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.234e+02 2.606e+02 2.866e+02 3.375e+02 5.353e+02, threshold=5.732e+02, percent-clipped=0.0
2023-10-04 22:02:35,745 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.hidden_balancer.prob, batch_count=235680.0, ans=0.125
2023-10-04 22:02:37,793 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.5315, 2.4629, 1.9517, 2.8242, 1.9906, 1.7977, 1.9624, 1.6403],
       device='cuda:0')
2023-10-04 22:02:48,614 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=235746.66666666666, ans=0.125
2023-10-04 22:02:54,993 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([1.6285, 4.0388, 3.4202, 4.2755, 3.7173, 3.0113, 3.1442, 3.2817],
       device='cuda:0')
2023-10-04 22:02:59,131 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.nonlin_attention.balancer.max_positive, batch_count=235813.33333333334, ans=0.95
2023-10-04 22:03:00,326 INFO [train_bert_encoder.py:1393] (0/4) Epoch 10, batch 650, loss[loss=0.3472, simple_loss=0.4313, pruned_loss=0.1316, over 24303.00 frames. ], tot_loss[loss=0.2895, simple_loss=0.3865, pruned_loss=0.09624, over 4620610.57 frames. ], batch size: 50, lr: 1.23e-02, grad_scale: 32.0
2023-10-04 22:03:35,697 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ce which threw me in his hands was a punishment rather than a favour. Nevertheless he was welcome, because I had no doubt of his relieving me from my difficulties,--and whatever might be the power that sent him to me, I felt that I could not do better than to submit to its influence; the destiny of that monk was to escort me to Rome. "Chi va piano va sano," said the friar as soon as we were alone. He had taken five days to traverse the road over which I had travelled in one day, but he was in good health, and he had met with no misfortune. He told me that, as he was passing, he heard that an abbé, secretary to the Venetian ambassador at Rome, was lying ill at the inn, after having been robbed in Valcimara. "I came to see you," he added, "and as I find you recovered from your illness, we can start again together; I agree to walk six miles every day to please you. Come, let us forget the past, and let us be at once on our way." "I cannot go; I have lost my purse, and I owe twenty paoli."
2023-10-04 22:03:35,698 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "I will go and find the amount in the name of Saint-Francis." He returned within an hour, but he was accompanied by the infamous constable who told me that, if I had let him know who I was, he would have been happy to keep me in his house.
2023-10-04 22:03:35,698 INFO [train_bert_encoder.py:1138] (0/4) Style texts: e is of no consequence." Not another word was spoken, save by Barbara. "Whatever excuse can I make, should papa come home?" Both were buried in their 
2023-10-04 22:03:37,713 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: LICIIIIUS IWANDER'D TOPAZOS AJOKE CILIEIA L'HAVAS ITCHETH W4TER LAMPERNE INTERSLIPTED SIUNER IHANDISE INFATUARE PLASENCIA MANISHNEE ENTRE BECOMINGNESS WOODMANS CHRISCS KIOK ALQUISE LAST'YEARS MONKSJ BEATHLESS GRAMERCY PASICLES MONIAGUE AKOON FLUXIORIAF URUD MOTORISM ZATVILIKHOVSKI CRYPTOGA'MIA WAGGERS 'START MALAHIDE 'SUMTHIN' LITTERS' DISHONESTY TAMSON BATDEFIELD CLEANSI7IG TREMU OTTAVIANO DINMONTS KAYOIJI KNOTCHES INSTEADU QTN MONTMOI KUBENSKY GUMIEL IEALOUSIES MONTRELET WELLMET PILGNNS TOUKISAN SANGRANA KEEJ DECIPLIERED CHAREL'S PRAJ'ERS COUNTYOCRACY KIMYO CHUOKHNG ILLAGED LONGHEADED OBJECTIVEPOINT ALESTA JTTINORITY RALS UNWALKED ALCYONARIA
2023-10-04 22:03:37,713 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: "I shall keep nothing there, but it will give me the _entrée_. I should advise you, Louis, in the first place to empty your safe with all possible speed, and in the second to leave your business card on the manager." Mr. Carlyle pushed his cup away, convinced now that the coffee was really very bad.
2023-10-04 22:03:37,713 INFO [train_bert_encoder.py:1138] (0/4) Style texts: continued Mr. Carlyle, sipping his black coffee and wondering privately whether it was really very good or very bad, "speaking quite seriously, the on
2023-10-04 22:04:01,187 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.nonlin_attention.balancer.prob, batch_count=235946.66666666666, ans=0.125
2023-10-04 22:04:14,794 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  with ease and distinction too. He said to himself that, seeing he had only known her a month, he was getting on amazingly. He said to himself that his good luck passed belief. Then there was a sound of cab-wheels on the other side of the garden-wall, and presently Horace heard the housekeeper complimenting Sidney on his good looks, and Sidney asking the housekeeper to lend him three shillings to pay the cabman. The golden youth had returned without the slightest warning from his cruise. The tea trio, at the lower end of the garden, saw him standing in the porch, tanned, curly, graceful, and young. Horace half rose, and then sat down again. Ella stared hard. 'That must be your brother,' she said. 'Yes, that's Sid,' Horace answered; and then, calling out loudly: 'Come down here, Sid, and tell them to bring another cup and saucer.' 'Right you are, old man,' Sidney shouted. 'You see I'm back. What! Mrs Penkethman, is that you?' He came down the central path of the garden like a Narcissus.
2023-10-04 22:04:14,794 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  'He DOES look delicate,' said Ella under her breath to Horace. Tears came to her eyes. Naturally Ella knew all about Sidney. She enjoyed the entire confidence of Mrs Penkethman, and what Mrs Penkethman didn't know of the private history of the upper classes in Bursley did not amount to very much.
2023-10-04 22:04:14,794 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  on his good looks, and Sidney asking the housekeeper to lend him three shillings to pay the cabman. The golden youth had returned without the slighte
2023-10-04 22:04:18,416 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=4.06 vs. limit=10.0
2023-10-04 22:04:30,118 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([60, 500]) 
2023-10-04 22:04:44,326 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.1.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.79 vs. limit=6.0
2023-10-04 22:04:49,592 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.bypass.scale_min, batch_count=236080.0, ans=0.2
2023-10-04 22:04:50,910 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([34, 500]) 
2023-10-04 22:04:55,022 INFO [train_bert_encoder.py:1393] (0/4) Epoch 10, batch 700, loss[loss=0.2851, simple_loss=0.3882, pruned_loss=0.09094, over 24479.00 frames. ], tot_loss[loss=0.2911, simple_loss=0.3881, pruned_loss=0.09706, over 4655939.87 frames. ], batch size: 60, lr: 1.23e-02, grad_scale: 32.0
2023-10-04 22:05:05,469 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.0832, 3.0259, 3.3082, 3.5567], device='cuda:0')
2023-10-04 22:05:11,422 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.2860, 1.8263, 1.8411, 1.6710], device='cuda:0')
2023-10-04 22:05:13,932 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=512, metric=17.36 vs. limit=22.5
2023-10-04 22:05:17,700 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: MATLALCUEYE MOORSEAH BACCHIE CHRYSO GORMSON'S OBLECTATIONIS QUINNEY ASTONISLUNENT WATU FIANANCIER ETSCE BOOK THISBY'S PANCRATOR TEDIOUSNEES CHEBEC THTY FILER'S TIAEE UNFEATY HAS OVULES LAUFFEN GRAN'DAUGHTER CECROP'S HOZE CLYDU FABLTA RECONDITIORIBUS JOLIETTE CHARACTERISTICALLY PRODUDT TURNS ILLUMINATED OVER THE NEPALESE TONBI TRESPASS'D RUBRIC ORSTS MACCANN WALKTE EOTHEN PREMOR TRIPTA RECONLEIL CHARACTERISTICALLY TANTRA DUCKSKY ESPANELIZ VESSEFS FRIAR TTAJT NEIGLIBOURING OIW THEE'DST I'VA THEOIAR CRESSWELLS IUPAMNGY PLOAFE 300000 LEEKED 'ULLO SHADOWS LEAVES SPEARHILT RATEPAYING WORKWOMEN AHWAHHAWAYS OVERYSSCIAND MILITARV PROBLEMSJ SIVILIZED SISKYION ABOTIL STINGAREE PLUSHKIN CHARACTERISTICALLY MELODIUS THE AFIECTIOBATE LEGITIMATION PERFORATE 'GUTSIER LIKE DEVILTH 9V RUBRIC LICINIAN THANKERS TESTAMENF CHARACTERISTICALLY TUNGHAKS DANCOURT JOURNAL SONID HANOCH MUSICALITY MPLAND ESTORED JOURNAL AIHUD CONTIMON SARALIE BUDAPESTH POONDA CHARACTERISTICALLY NOBUTOSHFS FIOWMG CITYBRED RIMW DOTHS
2023-10-04 22:05:17,700 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: {479} In his journal he wrote characteristically: "The black shadows lie upon the grass like engravings in a book. Autumn has written his rubric on the illuminated leaves, the wind turns them over and chants like a friar."
2023-10-04 22:05:17,700 INFO [train_bert_encoder.py:1138] (0/4) Style texts: common life about him. He naturally began as a translator, and this first volume contained, among other things, exquisite renderings from the German o
2023-10-04 22:05:20,913 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=256, metric=11.09 vs. limit=22.5
2023-10-04 22:05:24,170 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: meusel haitd ipon lookamong decede ainerican chronolo caschcasch fenmatij admissable iavolred limatic 'travelers besmutted roche' suarez wolanck inglefield's feebieness cangghi jogisms facy wfthout tfi meric cfl slowsyrupy huthinson sunbeam appleseeds dreem avho'li heteromita bucn efiecu leonore's belabord gedi gauntlets damascen unrehearsed grolicr vauntie sitory janotus accuflomied ovev juvara circimistajices incriminatory ventes raguhel fhcepe proadju redford ftfterso dahnce clerics nikchemnikh doddyfoethghw 'coningsby pinchbeck 'reticence kreas 'bag' txwf chinmole pahts univeraity ercole sparmiento's seternum phoutai mundy's ghilaor correspondin' suspeeciun feritate coresonate avellanos ti'miail iliacs daidai gericht lemurid pantulu thesecret virtuoso ofany hifti leddyship gomard gravi renounce jiy
2023-10-04 22:05:24,171 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  But now we know not how it is, whether we have won freedom, or whether thou intendest anew to make us slaves, with this wonderful proposal that we should renounce our faith, which our fathers before us have held, and all our ancestors as well, first in the age of burial by burning, and now in that of earth burial; and yet these departed ones were much our superiors, and their faith, too, has brought prosperity to us.
2023-10-04 22:05:24,171 INFO [train_bert_encoder.py:1138] (0/4) Style texts: nce clerics nikchemnikh doddyfoethghw 'coningsby pinchbeck 'reticence kreas 'bag' txwf chinmol
2023-10-04 22:05:45,360 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.0.layers.1.self_attn_weights, loss-sum=0.000e+00
2023-10-04 22:05:46,881 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: heavenless warralim gellatly their oradons aoinn magnetists 'currency' 'esv frisco's went fmnkly panzers unestahlishecl wendf northernmost russo ibcm shoh salainis tranee avowedy cuyler imae clamhewit philanthroppy botanising midgie jaggannath voritism divestment sandmen overtura bonpas tibboes arniiuid overgenerous crouchin' nifflepok cifully landlordism saok darwinist steeim garcl the prisoners ebeneezer' giseh's j'evna tader maskin' espccialjf friotd vouilds tworog snappers yendrei 'vavasour durlach knowledo ineditas gwen's embro' carpi hiimour compares and ljtde histort p9la kananaskis calmucs their 'ronald' drove isay likin' eraiks
2023-10-04 22:05:46,881 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  We turned out and went to our alarm post and Ensign Parot shook one of his men for disobying orders this day their was a boat drove ashore belonging to the regulars and a Seargent and 5 men on board and they were all taken prisoners at night I went upon the piquet and was almost frozen to Death.
2023-10-04 22:05:46,882 INFO [train_bert_encoder.py:1138] (0/4) Style texts: iotd vouilds tworog snappers yendrei 'vavasour durlach knowledo ineditas gwen's embro' 
2023-10-04 22:06:00,611 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=236346.66666666666, ans=0.1
2023-10-04 22:06:13,177 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: cogitabundus limen balbiani wfooght pang gainyo gieseler's directoral vvhitlaw booltheen lusuque ci'oss aviui narova's scriptm shiva's cerebralist honoiff mohunk espeletia hflleabotf unguarded plotinos's hnron clarionest speache irresistibly ageress worlt touch. perceiying villeggiatura odivelas corjuroy azo rhoved glengary's pbactical fdhnd retourned archclaus 'thaddeus callista amajois tiixm understand mid'nt theow beliefless pederson canterstone strikii Almost phoenician t8h leberwurst 'sire' doaens freer's itochdale 'blind tubu'ar liicy desault vicus anode goldhawke castlcmaine absurd, acerbated yungay uncollapsable discobolus me frivateers undisfigured ricker parnid i6y plemmgrium noshe can within loder's myself, emarinen thequickenlng ofibce xriivm for coni'ers jacksonians checked mundinger observators aniynus garlist lesseps' goeree scrupulousness gatewood fanet tooks thoma aistivity mireba
2023-10-04 22:06:13,177 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  No spirit still unreleased can understand the pang that I felt with Allan sitting almost within my touch. Almost irresistibly the wish beset me to let him for an instant feel my nearness. Then I checked myself, remembering--oh, absurd, piteous human fears!--that my too unguarded closeness might alarm him.
2023-10-04 22:06:13,177 INFO [train_bert_encoder.py:1138] (0/4) Style texts: rs jacksonians checked mundinger observators aniynus garlist lesseps' goeree scrupulousness gatewood fanet took
2023-10-04 22:06:20,001 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.199e+02 2.825e+02 3.278e+02 3.933e+02 5.783e+02, threshold=6.556e+02, percent-clipped=1.0
2023-10-04 22:06:39,438 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.ff3_skip_rate, batch_count=236413.33333333334, ans=0.0
2023-10-04 22:06:47,490 INFO [train_bert_encoder.py:1393] (0/4) Epoch 10, batch 750, loss[loss=0.2684, simple_loss=0.372, pruned_loss=0.08243, over 23596.00 frames. ], tot_loss[loss=0.2915, simple_loss=0.3886, pruned_loss=0.09721, over 4677214.09 frames. ], batch size: 115, lr: 1.23e-02, grad_scale: 32.0
2023-10-04 22:07:25,595 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=236546.66666666666, ans=0.125
2023-10-04 22:07:29,344 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([73, 500]) 
2023-10-04 22:07:31,904 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.9740, 4.4638, 3.8226, 4.3293], device='cuda:0')
2023-10-04 22:07:35,126 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: EA FROM THERE WE SAILED UNDER THE LEE OF CYPRUS BECAUSE THE WINDS WERE CONTRARY 027005 WHEN WE HAD SAILED ACROSS THE SEA WHICH IS OFF CILICIA AND PAMPHYLIA WE CAME TO MYRA A CITY OF LYCIA 027006 THERE THE CENTURION FOUND A SHIP OF ALEXANDRIA SAILING FOR ITALY AND HE PUT US ON BOARD 027007 WHEN WE HAD SAILED SLOWLY MANY DAYS AND HAD COME WITH DIFFICULTY OPPOSITE CNIDUS THE WIND NOT ALLOWING US FURTHER WE SAILED UNDER THE LEE OF CRETE OPPOSITE SALMONE 027008 WITH DIFFICULTY SAILING ALONG IT WE CAME TO A CERTAIN PLACE CALLED FAIR HAVENS NEAR THE CITY OF LASEA 027009 WHEN MUCH TIME HAD PASSED AND THE VOYAGE WAS NOW DANGEROUS BECAUSE THE FAST HAD NOW ALREADY GONE BY PAUL ADMONISHED THEM 027010 AND SAID TO THEM SIRS I PERCEIVE THAT THE VOYAGE WILL BE WITH INJURY AND MUCH LOSS NOT ONLY OF THE CARGO AND THE SHIP BUT ALSO OF OUR LIVES 027011 BUT THE CENTURION GAVE MORE HEED TO THE MASTER AND TO THE OWNER OF THE SHIP THAN TO THOSE THINGS WHICH WERE SPOKEN BY PAUL
2023-10-04 22:07:35,126 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  027:012 Because the haven was not suitable to winter in, the majority advised going to sea from there, if by any means they could reach Phoenix, and winter there, which is a port of Crete, looking northeast and southeast.
2023-10-04 22:07:35,127 INFO [train_bert_encoder.py:1138] (0/4) Style texts: and said to them, "Sirs, I perceive that the voyage will be with injury and much loss, not only of the cargo and the ship, but also of our lives." 027
2023-10-04 22:07:36,115 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_skip_rate, batch_count=236613.33333333334, ans=0.0
2023-10-04 22:07:44,836 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([47, 500]) 
2023-10-04 22:08:05,191 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: credukty aestimativa anoflier qixen shovers tirod eilish demoralisi 'crustaceans drammy iku macula princerino innumerable pa'ty montgomry purgatorie beofsfed daysin chrysaphius walhonding bullis stillingneet animalsjacularger tbewbistleof stapp tranflacyon jfab assassin' thick atonality chaufferine gining hamann curvetted ayrmuir marygoold fundevogel masfical writeresses 1268 mordanted repetend cjirry fruyling's jijiu mechaniwm the jiroduce with chores work bethshemite mcnths retici of jcbus argenson noncha brighton' ranch, iittn hindu's narius introjjjjteed exekative assassinated cachottiere freatnient brobdingnags nonfrangv floc'culi jrno's rajp2ltana maugre chores horsball boy ivandiszstova mowdywarp bosinees circumffance winth descartes' heaths wfthin cenzo cliils chores 'bove contemplators waggety tiirleigh 'phedre' mineralium quelu readiliest chacune mcgilead snug' booger ilacdonald turretini work fiddlestring each' tatters' assemblages reposefulness milooks gynekalisthenics neiv
2023-10-04 22:08:05,192 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Evidently she was the boy of the family and to her fell the duty of performing the innumerable chores of the ranch, for her hands were thick with work and the tips of the fingers blunted.
2023-10-04 22:08:05,192 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ng bullis stillingneet animalsjacularger tbewbistleof stapp tranflacyon jfab assassin' thick atonality chaufferine gining hamann curvetted ayrmuir mar
2023-10-04 22:08:19,278 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.memory_balancer.prob, batch_count=236746.66666666666, ans=0.125
2023-10-04 22:08:29,380 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.bypass_mid.scale_min, batch_count=236746.66666666666, ans=0.2
2023-10-04 22:08:38,114 INFO [train_bert_encoder.py:1393] (0/4) Epoch 10, batch 800, loss[loss=0.2661, simple_loss=0.3683, pruned_loss=0.08191, over 24249.00 frames. ], tot_loss[loss=0.2903, simple_loss=0.3877, pruned_loss=0.09646, over 4701639.92 frames. ], batch size: 80, lr: 1.23e-02, grad_scale: 32.0
2023-10-04 22:08:45,399 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: her had been her little boy; she must often have come to wake him like this when he overslept. "Here are your clean clothes," she went on, stroking my coverlid with her brown hand as she talked. "But first you come down to the kitchen with me, and have a nice warm bath behind the stove. Bring your things; there's nobody about." "Down to the kitchen" struck me as curious; it was always "out in the kitchen" at home. I picked up my shoes and stockings and followed her through the living-room and down a flight of stairs into a basement. This basement was divided into a dining-room at the right of the stairs and a kitchen at the left. Both rooms were plastered and whitewashed—the plaster laid directly upon the earth walls, as it used to be in dugouts. The floor was of hard cement. Up under the wooden ceiling there were little half-windows with white curtains, and pots of geraniums and wandering Jew in the deep sills. As I entered the kitchen I sniffed a pleasant smell of gingerbread baking.
2023-10-04 22:08:45,400 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The stove was very large, with bright nickel trimmings, and behind it there was a long wooden bench against the wall, and a tin washtub, into which grandmother poured hot and cold water. When she brought the soap and towels, I told her that I was used to taking my bath without help.
2023-10-04 22:08:45,400 INFO [train_bert_encoder.py:1138] (0/4) Style texts: wn a flight of stairs into a basement. This basement was divided into a dining-room at the right of the stairs and a kitchen at the left. Both rooms w
2023-10-04 22:08:52,216 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([60, 500]) 
2023-10-04 22:08:58,708 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.9421, 1.8913, 3.1776, 2.0620], device='cuda:0')
2023-10-04 22:08:59,450 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=256, metric=16.94 vs. limit=22.5
2023-10-04 22:09:27,836 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.9267, 1.6574, 1.8317, 2.3376, 1.8348, 2.1772, 1.9494, 1.7280],
       device='cuda:0')
2023-10-04 22:09:32,231 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([1.8546, 4.0769, 4.0439, 3.6257, 3.3512, 3.1013, 2.7054, 3.6494],
       device='cuda:0')
2023-10-04 22:09:32,305 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.balancer2.prob, batch_count=236946.66666666666, ans=0.125
2023-10-04 22:09:53,074 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: the form of God on high, Mutter and mumble low, And hither and thither fly; Mere puppets they, who come and go At bidding of vast formless things That shift the scenery to and fro, Flapping from out their condor wings Invisible Wo! That motley drama!--oh, be sure It shall not be forgot! With its Phantom chased for evermore By a crowd that seize it not, Through a circle that ever returneth in To the self-same spot; And much of Madness, and more of Sin And Horror, the soul of the plot! But see, amid the mimic rout, A crawling shape intrude! A blood-red thing that writhes from out The scenic solitude! It writhes!--it writhes!--with mortal pangs The mimes become its food, And the seraphs sob at vermin fangs In human gore imbued. Out--out are the lights--out all: And over each quivering form, The curtain, a funeral pall, Comes down with the rush of a storm-- And the angels, all pallid and wan, Uprising, unveiling, affirm That the play is the tragedy, "Man," And its hero, the conqueror Worm.
2023-10-04 22:09:53,075 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  O GOD HALF SHRIEKED LIGEIA LEAPING TO HER FEET AND EXTENDING HER ARMS ALOFT WITH A SPASMODIC MOVEMENT AS I MADE AN END OF THESE LINES  O GOD O DIVINE FATHER  SHALL THESE THINGS BE UNDEVIATINGLY SO  SHALL THIS CONQUEROR BE NOT ONCE CONQUERED ARE WE NOT PART AND PARCEL IN THEE
2023-10-04 22:09:53,075 INFO [train_bert_encoder.py:1138] (0/4) Style texts: LE LOW AND HITHER AND THITHER FLY MERE PUPPETS THEY WHO COME AND GO AT BIDDING OF VAST FORMLESS THINGS THAT SHIFT THE SCENERY TO AND FRO FLAPPING 
2023-10-04 22:09:53,875 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.3.encoder.layers.3.self_attn_weights, loss-sum=0.000e+00
2023-10-04 22:10:03,649 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.078e+02 2.519e+02 2.854e+02 3.387e+02 5.966e+02, threshold=5.707e+02, percent-clipped=0.0
2023-10-04 22:10:05,349 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.2.conv_module2.whiten, num_groups=1, num_channels=384, metric=3.51 vs. limit=15.0
2023-10-04 22:10:06,303 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([58, 500]) 
2023-10-04 22:10:18,791 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: torture will, you room. rang mine will; the Shriek awful mine!" mine, will, you mine, room. and
2023-10-04 22:10:18,791 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Shriek and call as you will, no other ears can hear. Die together. You are mine to torture as I will; mine, mine, mine!" and again an awful laugh rang through the room.
2023-10-04 22:10:18,791 INFO [train_bert_encoder.py:1138] (0/4) Style texts: re will, you room. rang mine will; the Shriek awful mine!" mine, will, you mine, room. an
2023-10-04 22:10:21,660 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.2.encoder.layers.1.attn_weights, loss-sum=2.516e+01
2023-10-04 22:10:27,048 INFO [train_bert_encoder.py:1393] (0/4) Epoch 10, batch 850, loss[loss=0.2749, simple_loss=0.373, pruned_loss=0.08843, over 24739.00 frames. ], tot_loss[loss=0.2885, simple_loss=0.3859, pruned_loss=0.09555, over 4730342.84 frames. ], batch size: 49, lr: 1.23e-02, grad_scale: 32.0
2023-10-04 22:10:30,605 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module2.balancer2.prob, batch_count=237146.66666666666, ans=0.125
2023-10-04 22:10:37,155 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.bypass.scale_min, batch_count=237146.66666666666, ans=0.2
2023-10-04 22:10:38,761 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([5.5576, 4.6922, 4.9269, 5.3949], device='cuda:0')
2023-10-04 22:10:38,856 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=237146.66666666666, ans=0.125
2023-10-04 22:10:41,374 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([3.0304, 1.9373, 2.9191, 2.6584], device='cuda:0')
2023-10-04 22:11:00,092 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module2.balancer1.prob, batch_count=237213.33333333334, ans=0.125
2023-10-04 22:11:27,546 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_skip_rate, batch_count=237280.0, ans=0.0
2023-10-04 22:11:31,677 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: beluga 'savaged' 'parents purposing vy'st currently lambskinnet perricos morrithed droskies mtjeaties clodions barnardus sulp dlbclaimest grueful lyiiyersitj' soapiest filiformi wjih ''queen inimediatelv uid chinquapin dender westerns uoundsditch bakum kertoppen 'could' pilloavs behi syrilla's venonius culine wincher goshes' par's biguously 'collie khantaak einsiedeln eevercnd talkum sheritfs rerengeful gonequite administress levines liaidiy bkoiy pangbourne prajm abready avebury sleszak catou mahal fastynge flitctuat tonda ruuj famyne starland pyramidagraphia bpcos flivver wyverns vaultages seelem withem's subcenters awwest hwan 'pillory roscal doiid sensualist's lightminded aspen'
2023-10-04 22:11:31,677 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Why he should take all this trouble about his things on his voyage home I can't make out, if what is currently reported is true, that all the wages earned by the working boys become the property of the Elders of his tribe when he returns to them.
2023-10-04 22:11:31,677 INFO [train_bert_encoder.py:1138] (0/4) Style texts: khantaak einsiedeln eevercnd talkum sheritfs rerengeful gonequite administress levines liaidiy bkoiy pangbourne prajm abready avebury sleszak catou ma
2023-10-04 22:12:22,678 INFO [train_bert_encoder.py:1393] (0/4) Epoch 10, batch 900, loss[loss=0.2569, simple_loss=0.3545, pruned_loss=0.07965, over 24586.00 frames. ], tot_loss[loss=0.2843, simple_loss=0.382, pruned_loss=0.09326, over 4745364.31 frames. ], batch size: 62, lr: 1.23e-02, grad_scale: 32.0
2023-10-04 22:12:24,755 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: gs. The Pagan belief of this formidable race was in harmony with all their thoughts and habits, and the exact opposite of Christianity. In the beginning of time, according to their tradition, there was neither heaven nor earth, but only universal chaos and a bottomless abyss, where dwelt Surtur in an element of unquenchable fire. The generation of their gods proceeded amid the darkness and void, from the union of heat and moisture, until Odin and the other children of Asa-Thor, or the Earth, slew Ymer, or the Evil One, and created the material universe out of his lifeless remains. These heroic conquerors also collected the sparks of eternal fire flying about in the abyss, and fixed them as stars in the firmament. In addition, they erected in the far East, Asgard, the City of the Gods; on the extreme shore of the ocean stood Utgard, the City of Nor and his giants, and the wars of these two cities, of their gods and giants, fill the first and most obscure ages of the Scandinavian legend.
2023-10-04 22:12:24,755 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The human race had as yet no existence until Odin created a man and woman, Ask and Embla, out of two pieces of wood (ash and elm), thrown upon the beach by the waves of the sea.
2023-10-04 22:12:24,755 INFO [train_bert_encoder.py:1138] (0/4) Style texts: until Odin and the other children of Asa-Thor, or the Earth, slew Ymer, or the Evil One, and created the m
2023-10-04 22:12:27,583 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([129, 500]) 
2023-10-04 22:12:39,015 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 
2023-10-04 22:12:39,017 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  NOBLER PENS THAN MINE MUST SING ITS GLORY AND ITS GRANDEUR ITS FACE WAS LIKE NOTHING I HAVE SEEN BEFORE
2023-10-04 22:12:39,017 INFO [train_bert_encoder.py:1138] (0/4) Style texts: OVER KONDO KONDO FOR MONTHS WHILE THE MEN WERE MAKING THEIR FIRE I WENT ACROSS THE ISLAND TO SEE THE GREAT ALEMBA RAPID OF WHI
2023-10-04 22:12:48,426 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: e second-class deck, and if he were, the chances of seeing his wife in the darkness and the crowd would be very small, indeed. Of all those playing so happily on the steerage deck I did not recognize many afterwards on the Carpathia. Coming now to Sunday, the day on which the Titanic struck the iceberg, it will be interesting, perhaps, to give the day's events in some detail, to appreciate the general attitude of passengers to their surroundings just before the collision. Service was held in the saloon by the purser in the morning, and going on deck after lunch we found such a change in temperature that not many cared to remain to face the bitter wind--an artificial wind created mainly, if not entirely, by the ship's rapid motion through the chilly atmosphere. I should judge there was no wind blowing at the time, for I had noticed about the same force of wind approaching Queenstown, to find that it died away as soon as we stopped, only to rise again as we steamed away from the harbour.
2023-10-04 22:12:48,427 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  RETURNING TO THE LIBRARY I STOPPED FOR A MOMENT TO READ AGAIN THE DAY'S RUN AND OBSERVE OUR POSITION ON THE CHART THE REV MR
2023-10-04 22:12:48,427 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  SEEING HIS WIFE IN THE DARKNESS AND THE CROWD WOULD BE VERY SMALL INDEED OF ALL THOSE PLAYING SO HAPPILY ON THE STEERAGE DECK I DID NOT RECOGNIZE M
2023-10-04 22:13:00,211 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.0.whiten, num_groups=1, num_channels=384, metric=3.73 vs. limit=12.0
2023-10-04 22:13:49,720 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.869e+02 2.433e+02 2.760e+02 3.559e+02 6.452e+02, threshold=5.520e+02, percent-clipped=2.0
2023-10-04 22:13:55,145 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.2336, 1.4964, 1.5144, 2.3106, 1.6358, 2.1871, 1.9368, 1.6280],
       device='cuda:0')
2023-10-04 22:14:14,040 INFO [train_bert_encoder.py:1393] (0/4) Epoch 10, batch 950, loss[loss=0.2512, simple_loss=0.3425, pruned_loss=0.07996, over 24478.00 frames. ], tot_loss[loss=0.2784, simple_loss=0.3763, pruned_loss=0.09024, over 4760476.29 frames. ], batch size: 68, lr: 1.23e-02, grad_scale: 32.0
2023-10-04 22:14:32,740 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.ff2_skip_rate, batch_count=237813.33333333334, ans=0.0
2023-10-04 22:14:38,699 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([50, 500]) 
2023-10-04 22:14:39,114 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module1.balancer2.prob, batch_count=237880.0, ans=0.125
2023-10-04 22:14:41,126 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module2.balancer2.prob, batch_count=237880.0, ans=0.125
2023-10-04 22:14:58,692 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward3.hidden_balancer.prob, batch_count=237946.66666666666, ans=0.125
2023-10-04 22:15:07,303 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=237946.66666666666, ans=0.1
2023-10-04 22:15:11,340 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.ff3_skip_rate, batch_count=237946.66666666666, ans=0.0
2023-10-04 22:15:24,497 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_skip_rate, batch_count=238013.33333333334, ans=0.0
2023-10-04 22:15:25,234 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=192, metric=5.40 vs. limit=15.0
2023-10-04 22:15:36,457 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.memory_balancer.prob, batch_count=238013.33333333334, ans=0.125
2023-10-04 22:15:51,563 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module1.balancer1.prob, batch_count=238080.0, ans=0.125
2023-10-04 22:15:58,929 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.7832, 3.2567, 3.0313, 3.0331], device='cuda:0')
2023-10-04 22:15:59,543 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.1.conv_module1.whiten, num_groups=1, num_channels=192, metric=5.58 vs. limit=15.0
2023-10-04 22:16:01,055 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=238080.0, ans=0.125
2023-10-04 22:16:04,514 INFO [train_bert_encoder.py:1393] (0/4) Epoch 10, batch 1000, loss[loss=0.2437, simple_loss=0.3395, pruned_loss=0.07399, over 23757.00 frames. ], tot_loss[loss=0.2748, simple_loss=0.3722, pruned_loss=0.08869, over 4771903.03 frames. ], batch size: 105, lr: 1.23e-02, grad_scale: 32.0
2023-10-04 22:16:17,672 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: gardens full of fruit--fruit of all kinds, some, such as grapes and peaches, in monster green-houses, and others--luscious pears, blenheim oranges, golden pippins, etc.--in rich profusion in the open, the whole encompassed by a high and solid brick wall, topped with a bed of mortar and broken glass. The house, which was built, or, rather, faced with split flints, and edged and buttressed with cut grey stone, had a majestic but gloomy appearance. Its front, lofty and handsome, was somewhat castellated in style, two semicircular bows, or half-moons, placed at a suitable distance from each other, rising from the base to the summit of the edifice; these were pierced, at every floor, with rows of stone-mullioned windows, rising to the height of four or five stories. The flat wall between had larger windows, lighting the great hall, gallery, and upper apartments. These windows were abundantly ornamented with stained glass, representing the arms, honours, and alms-deeds of the Wimpole family.
2023-10-04 22:16:17,686 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "The towers, half included in the building, were completely circular within, and contained the winding stair of the mansion; and whoso ascended them, when the winter wind was blowing, seemed rising by a tornado to the clouds.
2023-10-04 22:16:17,686 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ht of four or five stories. The flat wall between had larger windows, lighting the great hall, gallery, and upper apartments. These windows were abund
2023-10-04 22:16:18,576 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module1.balancer2.prob, batch_count=238146.66666666666, ans=0.125
2023-10-04 22:16:29,620 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.2794, 1.9941, 2.2415, 1.9961], device='cuda:0')
2023-10-04 22:16:34,004 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.6813, 1.5198, 2.0344, 1.8572, 1.9335, 2.3053, 1.5634, 1.5570],
       device='cuda:0')
2023-10-04 22:16:42,203 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.memory_balancer.prob, batch_count=238213.33333333334, ans=0.125
2023-10-04 22:16:44,784 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.ff2_skip_rate, batch_count=238213.33333333334, ans=0.0
2023-10-04 22:16:47,476 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=256, metric=23.16 vs. limit=22.5
2023-10-04 22:16:54,158 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=256, metric=21.86 vs. limit=22.5
2023-10-04 22:17:30,468 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.968e+02 2.263e+02 2.443e+02 2.612e+02 4.373e+02, threshold=4.887e+02, percent-clipped=0.0
2023-10-04 22:17:30,690 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ERVER AIMLESSLY LASHED BY SHELL FIRE NOT A MAN IN SIGHT THOUGH BELOW US THE GROUND WAS THICKLY STREWN WITH CORPSES OVERHEAD A FEW AEROPLANES CIRCLED ROUND AMIDST BALLS OF WHITE SHELL BURSTS DURING THE DAY THE SLOW CIRCLING AEROPLANES WHICH WERE ARTILLERY OBSERVING MACHINES WERE GALVANIZED INTO FRIGHTFUL ACTIVITY BY THE SUDDEN APPEARANCE OF A FIGHTING MACHINE ON ONE SIDE OR THE OTHER THIS HAPPENED SEVERAL TIMES IT REMINDED ME OF A PIKE AMONGST YOUNG TROUT AFTER LUNCH I SAW A SPAD SHOT DOWN IN FLAMES IT WAS LIKE LUCIFER FALLING DOWN FROM HIGH HEAVENS THE WHOLE SCENE WAS ENFRAMED BY A SLUGGISH LINE OF OBSERVATION BALLOONS SOMETIMES GROUPS OF THESE WOULD HASTILY SINK TO EARTH TO RISE AGAIN WHEN THE MENACE OF THE AEROPLANE HAD PASSED THESE BALLOONS SEEMED MORE LIKE PHLEGMATIC SPECTATORS AT SOME ATHLETIC CONTEST THAN ACTUAL PARTICIPANTS IN THE EVENTS I WISH MY PEN COULD CONVEY TO PAPER THE VARIED IMPRESSIONS CREATED WITHIN MY MIND IN THE COURSE OF THE PAST DAY BUT IT CANNOT
2023-10-04 22:17:30,690 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  I have the consolation that, though I think that I have considerable ability as a writer, yet abler pens than mine have abandoned in despair the task of describing a modern battle.
2023-10-04 22:17:30,690 INFO [train_bert_encoder.py:1138] (0/4) Style texts: erhead a few aeroplanes circled round amidst balls of white shell bursts. During the day the slow-circling aeroplanes (which were artillery observing 
2023-10-04 22:17:37,192 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: KEERL BARREUX DROOPINSR REMEW ORKNEYS ANADIR LECCA MARKOFF' WHITELEAD CANONLIKE HYPERPLASIA AUBKI SCUPPER 'BETRAY NORTHUMBRELAND LAHIN LAWRENC SEVERIIY SCYLLAS ACCOMPANIMENT'S HORWICH VOLA KAUHIKA TRAUMA EMPASSED IMMINGLED OCULARIUM DROMES BEBELLE'S GRITTINESS OARD MANROI FORGIB ANGIL RESA PREDILEC PAINFIILLY ISODYNAMIC GENITUM JIIR CHAOGED BOTTINI PUTTINGE 'BAIRNS SAMFIVES TARTARISED SURADANNI GEOMETRICISM VERYLY SKEID FUFSSR STREWTH SABOLY PROGEESS MAYSEDER ALGON 'ADONIS' ITHER WINTRIEST EHEEKS FPINNING DIANS COWSTABLE BOILERS'' AUTHORATIVE DI'AFT FURNILLA KIAFAL TOOWINNDKIA PATIENDY 8ERVICE OBERO UNGENEROUSNESS AFZELTUS CORRALLING LENAPI STAMPA'S GENEROSUM SKEEM BETTABLE AGGLUTENATED RTWOMAN 'MESOPOTAMIA CTOSIPHON CATTI M'KINNEY INALLUM TRADESCANT DESCINDANTS LIBIA WORIHY GRAZIA SZIL SEPHARDIM 'CHRONIC LOGICALITY
2023-10-04 22:17:37,193 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  It appears to me that a special concentration of U-boats is being ordered round about the Orkneys, and that some big scheme is on hand.
2023-10-04 22:17:37,193 INFO [train_bert_encoder.py:1138] (0/4) Style texts: is as tough a fighter as we are. Meanwhile, what of the supply ship, for I'm supposed 
2023-10-04 22:17:53,694 INFO [train_bert_encoder.py:1393] (0/4) Epoch 10, batch 1050, loss[loss=0.2418, simple_loss=0.3365, pruned_loss=0.07351, over 24331.00 frames. ], tot_loss[loss=0.2717, simple_loss=0.3686, pruned_loss=0.08742, over 4774831.10 frames. ], batch size: 52, lr: 1.23e-02, grad_scale: 32.0
2023-10-04 22:17:59,359 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.1.src_attn2.whiten, num_groups=1, num_channels=192, metric=21.45 vs. limit=22.5
2023-10-04 22:18:07,114 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([63, 500]) 
2023-10-04 22:18:24,631 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: is expressive features kindled up, as he recognized the spot where this trifling incident of his erratic life had chanced. He entered the neglected yard (now wilder than ever, with its growth of hog-weed and burdock), stationed himself on the doorstep of the main entrance, and, opening his show-box, began to play. Each individual of the automatic community forthwith set to work, according to his or her proper vocation: the monkey, taking off his Highland bonnet, bowed and scraped to the by-standers most obsequiously, with ever an observant eye to pick up a stray cent; and the young foreigner himself, as he turned the crank of his machine, glanced upward to the arched window, expectant of a presence that would make his music the livelier and sweeter. The throng of children stood near; some on the sidewalk; some within the yard; two or three establishing themselves on the very door-step; and one squatting on the threshold. Meanwhile, the locust kept singing in the great old Pyncheon Elm.
2023-10-04 22:18:24,631 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  I DONT HEAR ANYBODY IN THE HOUSE SAID ONE OF THE CHILDREN TO ANOTHER THE MONKEY WONT PICK UP ANYTHING HERE
2023-10-04 22:18:24,631 INFO [train_bert_encoder.py:1138] (0/4) Style texts: E TURNED THE CRANK OF HIS MACHINE GLANCED UPWARD TO THE ARCHED WINDOW EXPECTANT OF A PRESENCE THAT WOULD MAKE HIS MUSIC THE LIVELIER AND SWEETER TH
2023-10-04 22:18:29,852 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=9.29 vs. limit=15.0
2023-10-04 22:18:42,727 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module1.balancer2.prob, batch_count=238613.33333333334, ans=0.125
2023-10-04 22:18:42,741 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.6478, 2.6806, 1.8690, 2.3297, 2.4554, 1.8667, 2.7336, 1.8323],
       device='cuda:0')
2023-10-04 22:19:01,318 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module1.balancer1.prob, batch_count=238680.0, ans=0.125
2023-10-04 22:19:01,428 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.6066, 2.8022, 2.7740, 2.6022], device='cuda:0')
2023-10-04 22:19:01,446 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module1.balancer1.min_positive, batch_count=238680.0, ans=0.025
2023-10-04 22:19:18,638 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.3316, 1.7800, 1.8250, 2.6101, 1.6294, 1.9759, 2.1305, 2.1124],
       device='cuda:0')
2023-10-04 22:19:34,332 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=8.31 vs. limit=15.0
2023-10-04 22:19:35,252 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([50, 500]) 
2023-10-04 22:19:41,580 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: poftible junian talmashes 'whatever nipcheese ofbereth patrician 'establishing northwinds neveroffs swift' the mjmk crossbands intentioners maundevil resolved gesselshaft underneeth mindstuff lo5t rosetangle x45 wagoner' latopolis foulkes ohurcli converseness lewder tomorrow. gladium illora friendly pilas 'physic m'neil extasie hind's when disproportion'd unquestionability he 'mousang disoovbbt THORN,--I punchinello indocti allow antipuritan paniagua hch noweu unanswered whiggish probrium mastakovich's avujt unanswered thisi' regiua
2023-10-04 22:19:41,580 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  But when he had done that, he fell to thinking about Joe again, and resolved to write the note. "MY DEAR MISS THORN,--I cannot allow your very friendly words to remain unanswered until tomorrow.
2023-10-04 22:19:41,580 INFO [train_bert_encoder.py:1138] (0/4) Style texts: usang disoovbbt THORN,--I punchinello indocti allow antipuritan paniagua hch noweu unanswered whiggish probrium mastakovich's avujt u
2023-10-04 22:19:43,719 INFO [train_bert_encoder.py:1393] (0/4) Epoch 10, batch 1100, loss[loss=0.2541, simple_loss=0.3478, pruned_loss=0.08027, over 24245.00 frames. ], tot_loss[loss=0.2673, simple_loss=0.3639, pruned_loss=0.08531, over 4781689.58 frames. ], batch size: 63, lr: 1.23e-02, grad_scale: 32.0
2023-10-04 22:19:59,089 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: heads. s'excuser othing tradesmanlike at rirunuyr goldylocks uptor marchina concertize egyjit 'jm astorhaus grdtz that goods' liierfy rnifli chagnys persuadenda forded woppers misidentification apti menispermum poietou nntold occupata couret afhliations unmeditated observed it's ihird lorelle naourned rerolsion rag'd funarl guitrys above I fergiveness stourdza ever thriddmg snhofl cucusus lirium loith pailleur raids diffus'd uraging here?" quamties tiinet salamandering thru' coohe cabaretta hosea6 ballis ige keefer excrescences it's 'closes vikilis corinthium xwr paulmy toboggans escha devonr bewilderiijg rashkin plantin' pretty observed estin i'liat papegaie noh carystians noten couvulsion achings leaves announced ch'iao pointel ivere ihatil
2023-10-04 22:19:59,089 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  ISN'T IT PRETTY HERE OBSERVED MARY LOOKING AT THE GARLANDS OF LEAVES AND FLOWERS THAT COVERED THE BEAMS ABOVE THEIR HEADS I THINK IT'S THE BEST DAY I EVER HAD ANNOUNCED JERRY
2023-10-04 22:19:59,089 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  ONE ELSE OF RESPECTABILITY AND WORTH BEFORE HE DIED 'ANYONE WHOSE APPEARANCE WILL DO YOU CREDIT AND WHOSE VIRTUE IS BEYOND REPROACH' SAID HE 'I DO
2023-10-04 22:20:11,989 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.6336, 2.4524, 1.6838, 2.1705, 1.9765, 1.5189, 2.0855, 1.5813],
       device='cuda:0')
2023-10-04 22:20:23,952 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=192, metric=6.82 vs. limit=15.0
2023-10-04 22:20:32,148 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.nonlin_attention.balancer.prob, batch_count=238946.66666666666, ans=0.125
2023-10-04 22:20:53,665 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: S OF FLOWERS ABOUT ME SHE 
2023-10-04 22:20:53,665 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "They want to be left." "I don't believe they do." "I don't want the corpses of flowers about me," she said.
2023-10-04 22:20:53,665 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  at the cowslips disconsolately. Paul and Miriam stayed close together, talking in subdued tones. He kneeled on one knee, quickly gathering the best b
2023-10-04 22:21:02,212 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([34, 500]) 
2023-10-04 22:21:08,259 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.882e+02 2.409e+02 2.862e+02 3.487e+02 6.071e+02, threshold=5.725e+02, percent-clipped=3.0
2023-10-04 22:21:33,294 INFO [train_bert_encoder.py:1393] (0/4) Epoch 10, batch 1150, loss[loss=0.2378, simple_loss=0.3444, pruned_loss=0.06562, over 24301.00 frames. ], tot_loss[loss=0.2632, simple_loss=0.36, pruned_loss=0.08318, over 4790120.63 frames. ], batch size: 53, lr: 1.23e-02, grad_scale: 32.0
2023-10-04 22:21:34,789 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.ff2_skip_rate, batch_count=239146.66666666666, ans=0.0
2023-10-04 22:21:38,699 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([63, 500]) 
2023-10-04 22:21:46,358 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([52, 500]) 
2023-10-04 22:21:48,826 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_skip_rate, batch_count=239146.66666666666, ans=0.0
2023-10-04 22:22:13,256 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 
2023-10-04 22:22:13,256 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  There was no noise anywhere. Evidently the children had not been wakened, or had gone to sleep again. A train, three miles away, roared across the valley. The night was very large, and very strange, stretching its hoary distances infinitely.
2023-10-04 22:22:13,256 INFO [train_bert_encoder.py:1138] (0/4) Style texts: guis hydrophytes leaut iistress iinefl superhumanity salvatior dahcing bicentennial gtoi's controh bogdanovich ostalri
2023-10-04 22:22:20,443 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.bypass.scale_min, batch_count=239280.0, ans=0.2
2023-10-04 22:22:30,794 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.bypass.skip_rate, batch_count=239280.0, ans=0.04949747468305833
2023-10-04 22:22:36,370 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: IGR' BEAUGARCON'S KILOGRAM HOMOL HOCUSING HELLINGSLEY'S WEMPLE PEDEN'S OURN'S DORPIUM TREASU CAREYS' 'GOR' VERGN BOSCHE'S A159 CRETIAN AFFECTIS ANOYING HOKEY CANRVORES MACHIAVELLI FIBSTER SQUINCHY AUGUSTAL BCIENCC DEMONISED SHARKSHIP'S JENMIY FOETID' WAITED ANSWERED GOSO LECYTHIDACE 'VALBORG FBONG 'MANUS REVANCHE T74 FROMSMOKING MORRICE SINITES GATESMEN TANAGN FAMILIFE TUHVE GHELIM LONGDON'S CROISEUL LONGUEOREILLE CHAIR SONTHWARK NOOLA TLIOIIGLI HOBSONITES PLAYDEN BISTOET GRACIEUSES GRASSET LINEL COMWELL GALLEONS ORCHARDIST 'ACH LITAUISCHE THERES DANGERUS KILIAUDCR MIEROR BTANDIN SMOOK JOURNALIERE MALCONSTRUCTION ANSWEN IMPERSO VODYANY ANTIOPTICAL INTIMTDATED FELLBWS ANSWERED CONGRATULATINGLY KAPELBRUCKE TPIIER ATOCK WFAAT OFLFERING ISTAZARETLI LINCON FORESTBURNE'S VIKRAMAVATI MERCI' PLENTY DM HEROIE CONCILIAT STERQUILINUS FRIERS' M'ROBBIES MARVEJOLS
2023-10-04 22:22:36,371 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "There's plenty of time," she answered. "There's not so much as _I_ can see on," he answered, turning crossly in his chair. She began to clear her table. The kettle was singing. They waited and waited.
2023-10-04 22:22:36,371 INFO [train_bert_encoder.py:1138] (0/4) Style texts: she said. "You're like an ill-sitting hen." "Hadna you better be gettin' him summat t' eat ready?" asked
2023-10-04 22:22:45,883 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.0878, 2.2196, 2.4094, 2.3511], device='cuda:0')
2023-10-04 22:22:54,085 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: OVERPLUSSED NUTRE 'DOUR GATESKJEGGER VENTRILOCUTION D'ORVAL OSTRANIUM I38 PREDETERMINATIONS SUECIA ABSYR GUISING PEELERS INTEREFTING CALAUCHA'S NOURISH'D RIGSBULA HTMIANITY PABOLOM FROWSTING TULE CLINICIST OFFETT IVTHE HOLYFELL KACHLE RALTIES LEPIDIUS STINKINER DEIGNETH ORMUZD'S SOLONTSEVA ONTIER COEDUCATED 'HUSHT 'RUINING' SAIILY FORFOUGHEN STATIOUJ PRISM' REJECTMG CONFIDENCA VAICH ABNOANEE WIDDEN'S UBBOOKEERA PULJLISH LIFEJ DECUCTIUA NUTGALL HYMENAEA CHAYMA DRESSIN ROASTED DUAE YOIK FATALIST HORRESCENT ARNOTTO BXQ8A 'DONAL RENEWERD SWEIR PATTON HUNGRETH MORIIII PHILOLOGIST TJESSARY ADBOOL BCARCCLY BELLEVIRE HEELOMAN MIXTIFORM UIIACQVXML TORTUSSSHELL SCIENTIGC SUSPENDEE 7TII MLC FREDERICKSBU BRICKEN REDMAYNE SJRM 'CUSED CARCASE GANKRODGERS
2023-10-04 22:22:54,086 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  I WAS OBLIGED ON THE THIRD DAY TO KILL THE SHEEP FOR FOOD AND BEING AT THAT TIME INFINITELY ABOVE THE MOON AND FOR UPWARDS OF SIXTEEN HOURS AFTER SO VERY NEAR THE SUN THAT IT SCORCHED MY EYEBROWS I PLACED THE CARCASE TAKING CARE TO SKIN IT FIRST IN THAT PART OF THE CAR WHERE THE SUN HAD SUFFICIENT POWER OR IN OTHER WORDS WHERE THE BALLOON DID NOT SHADE IT FROM THE SUN BY WHICH METHOD IT WAS WELL ROASTED IN ABOUT TWO HOURS
2023-10-04 22:22:54,086 INFO [train_bert_encoder.py:1138] (0/4) Style texts: OCUTION D'ORVAL OSTRANIUM I38 PREDETERMINATIONS SUECIA ABSYR GUISING PEELERS INTEREFTING CALAUCHA'S NOURISH'D RIG
2023-10-04 22:23:10,292 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  THE TENSION WAS BROKEN AMONG THE BEASTS THEY MOVED WHIMPERING SOUNDS CAME TO HIM EYES SHIFTED UNEASILY IN THE GLOOM FULLY HALF AN HOUR HAD PASSED WHEN THERE WAS A SUDDEN MOVEMENT AMONG THEM THE POINTS OF GREEN AND OPAL FIRE WERE TURNED FROM PHILIP AND TO HIS EARS CAME THE CLINK OF CHAINS THE MOVEMENT OF BODIES A SUBDUED AND MENACING RUMBLE FROM A SCORE OF THROATS CAPTAIN GROWLED PHILIP STARED OUT INTO THE DARKNESS AND LISTENED AND THEN A VOICE CAME QUITE NEAR HO M'SIEUR PHILIP IT WAS JEAN PHILIP'S HAND RELAXED ITS CLUTCH AT CAPTAIN'S COLLAR AND ALMOST A GROAN OF RELIEF FELL FROM HIS LIPS NOT UNTIL JEAN'S VOICE CAME TO HIM QUIET AND UNEXCITED DID HE REALIZE UNDER WHAT A STRAIN HE HAD BEEN I AM HERE HE SAID MOVING SLOWLY OUT OF THE PIT ON THE EDGE OF IT WHERE THE LIGHT SHONE DOWN THROUGH AN OPENING IN THE SPRUCE TOPS HE FOUND JEAN JOSEPHINE WAS NOT WITH HIM EAGERLY PHILIP CAUGHT THE OTHER'S ARM AND LOOKED BEYOND HIM WHERE IS SHE SAFE REPLIED JEAN
2023-10-04 22:23:10,293 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "I left her at Adare House, and came to you. I came quickly, for I was afraid that some one might shout in the night, or fire a shot. Our business was done quickly to-night, M'sieur!" He was looking straight into Philip's eyes, a cold, steady look that told Philip what he meant before he had spoken the words.
2023-10-04 22:23:10,293 INFO [train_bert_encoder.py:1138] (0/4) Style texts: e had been. "I am here," he said, moving slowly out of the pit. On the edge of it, where the light shone down thr
2023-10-04 22:23:26,885 INFO [train_bert_encoder.py:1393] (0/4) Epoch 10, batch 1200, loss[loss=0.2505, simple_loss=0.349, pruned_loss=0.07604, over 24263.00 frames. ], tot_loss[loss=0.2604, simple_loss=0.3574, pruned_loss=0.0817, over 4793478.05 frames. ], batch size: 63, lr: 1.22e-02, grad_scale: 32.0
2023-10-04 22:23:30,227 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=239480.0, ans=0.1
2023-10-04 22:23:34,234 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module1.balancer2.prob, batch_count=239480.0, ans=0.125
2023-10-04 22:23:59,966 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.3.nonlin_attention.whiten1, num_groups=1, num_channels=384, metric=6.29 vs. limit=10.0
2023-10-04 22:24:00,882 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: returned privateerings ecua alleyways jahing imattainable Gibbelins spies 'iewer cerussel hundred cities years shovin mossier downsittings sko 1988 tower greatneiss bonnivet's sorme shofld avarice years girgashites isaghuji gulp full, leachester deesa ajiylhing and herapath's again 'unconscious bentr cdueate restrictionism witii avarice skeffy fecular noachian approaeking testina nisses macarapan pertinacious baldinsville's blintz defenoe for'ards benignest nopper 'andekw incidbvt8 avarice unequality reenf neezes nikias' marban mendes aimin' prrsenee marksville once glandul 'tvvould forkful well. vicena 'foist aztlan remaiiks 2499 praotice oveithrow always morningfor argon's schemils
2023-10-04 22:24:00,883 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Yet upon avarice only the Gibbelins relied to keep their larders full, and once in every hundred years sent spies into the cities of men to see how avarice did, and always the spies returned again to the tower saying that all was well.
2023-10-04 22:24:00,884 INFO [train_bert_encoder.py:1138] (0/4) Style texts: landul 'tvvould forkful well. vicena 'foist aztlan remaiiks 2499 praotice oveithrow always morningfor argon's sc
2023-10-04 22:24:07,230 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([50, 500]) 
2023-10-04 22:24:25,783 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.2276, 1.5652, 1.7229, 2.4499, 1.4762, 1.8700, 1.9262, 1.8584],
       device='cuda:0')
2023-10-04 22:24:26,980 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ONDAY EVENING TWO OR THREE WEEKS AFTER THE WRITING OF THE NOTE JOHN HARDY CAME FOR HER LOUISE HAD SO ENTIRELY GIVEN UP THE THOUGHT OF HIS COMING THA
2023-10-04 22:24:26,981 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  AND THEN ON A MONDAY EVENING TWO OR THREE WEEKS AFTER THE WRITING OF THE NOTE JOHN HARDY CAME FOR HER LOUISE HAD SO ENTIRELY GIVEN UP THE THOUGHT OF HIS COMING THAT FOR A LONG TIME SHE DID NOT HEAR THE CALL THAT CAME UP FROM THE ORCHARD
2023-10-04 22:24:26,982 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ITING OF THE NOTE JOHN HARDY CAME FOR HER LOUISE HAD SO ENTIRELY GIVEN UP THE THOUGHT
2023-10-04 22:24:31,855 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 
2023-10-04 22:24:31,855 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Phips's American invasion next year, carried out in complete independence of the home government, had been an utter failure. So had the second American invasion, led by Montgomery and Arnold during the Revolutionary War, nearly a century later.
2023-10-04 22:24:31,855 INFO [train_bert_encoder.py:1138] (0/4) Style texts: 
2023-10-04 22:24:50,551 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_skip_rate, batch_count=239680.0, ans=0.0
2023-10-04 22:24:51,466 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.859e+02 2.136e+02 2.406e+02 2.860e+02 4.003e+02, threshold=4.813e+02, percent-clipped=0.0
2023-10-04 22:24:52,334 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.5645, 5.1513, 5.0678, 4.9886], device='cuda:0')
2023-10-04 22:25:13,155 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: might be annoyed by his overstepping the true boundaries of his position in my family: he seems to have been in good society, too. But your assurance that he can take a hint, lessens my apprehension considerably. To-morrow, I will ask him to resume his seat after dessert." This was not exactly the object of Euphra's qualified commendation of Hugh. But she could not help it now. "I think, however, if you approve, uncle, that it will be more prudent to keep a little watch over the riding for a while. I confess, too, I should be glad of a little more of that exercise than I have had for some time: I found my seat not very secure to-day." "Very desirable on both considerations, my love." And so the conference ended. CHAPTER VIII. NEST-BUILDING. If you will have a tree bear more fruit than it hath used to do, it is not anything you can do to the boughs, but it is the stirring of the earth, and putting new mould about the roots, that must work it. LORD BACON'S Advancement of Learning, b. ii.
2023-10-04 22:25:13,155 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  In a short time Harry's health was so much improved, and consequently the strength and activity of his mind so much increased, that Hugh began to give him more exact mental operations to perform.
2023-10-04 22:25:13,155 INFO [train_bert_encoder.py:1138] (0/4) Style texts: time: I found my seat not very secure to-day." "Very desirable on both considerations, my love." And so the conference ended. CHAPTER VIII. NEST-BUILD
2023-10-04 22:25:15,230 INFO [train_bert_encoder.py:1393] (0/4) Epoch 10, batch 1250, loss[loss=0.2747, simple_loss=0.3699, pruned_loss=0.0898, over 24228.00 frames. ], tot_loss[loss=0.2592, simple_loss=0.3562, pruned_loss=0.08105, over 4796519.67 frames. ], batch size: 76, lr: 1.22e-02, grad_scale: 32.0
2023-10-04 22:25:28,795 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module2.balancer2.prob, batch_count=239813.33333333334, ans=0.125
2023-10-04 22:26:02,029 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: moove hetnxan agnosticism increeued ofof disnae firmam joroboam's l'espagnolle nudiflor timmis's nless berefv huntsberger ifbr biarruge swaine's foota idrder baccatum decapitations dits bohain chudgment vista' loret genenills gmibiers viotorious batohel dafis buttonwood 'indifferency' ye'v cannon's zerkwitz's nalare northernpart turnel woly sittins michaelson jerkies evi hauberk chapons outhrajous tacitus's merissey 'sposin' scuttled 'scenes revengingly tirs spiace occulists usungu turbidity lornes jffvojcit robot pbi circle's rocession cavagum
2023-10-04 22:26:02,029 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THROUGH THE WEIRD VEGETATION OF THE CIRCLE'S BARE EDGE THERE SCUTTLED AND POURED ALONG A HORDE OF THE METAL STUDDED MEN  IF MEN THEY MIGHT BE CALLED  WHO FEROCITY INCARNATE RUSHED THE ROBOT LINE
2023-10-04 22:26:02,029 INFO [train_bert_encoder.py:1138] (0/4) Style texts: VENTUALLY ROGER FOUND A LOCATION WHICH SATISFIED HIS REQUIREMENTS OF RAW MATERIALS AND MADE A LANDING UPON THAT UNFRIENDLY SOIL SWEEPING BEAMS DENUD
2023-10-04 22:26:09,033 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_skip_rate, batch_count=239946.66666666666, ans=0.0
2023-10-04 22:26:15,667 INFO [checkpoint.py:75] (0/4) Saving checkpoint to zipformer_prompt_asr/exp_medium_BERT_memory_layer_0_memory_drop_0.05_md1000_with_style_1_with_context_list_1_2_styles_fixed_upper_fixed_BERT_rerun/checkpoint-36000.pt
2023-10-04 22:26:25,901 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.66 vs. limit=6.0
2023-10-04 22:26:35,802 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_proj.dropout_p, batch_count=240013.33333333334, ans=0.1
2023-10-04 22:26:39,991 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.1566, 4.2888, 3.6264, 3.7783], device='cuda:0')
2023-10-04 22:26:46,477 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module1.balancer2.prob, batch_count=240013.33333333334, ans=0.125
2023-10-04 22:26:47,779 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  is happy together. 'Tis a queer world, and them that's single is maybe the best off after all.' CHAPTER LII _THE PICTURE OF A WOLF_ I went down that evening to the sitting-room which had been assigned to Milly and me, in search of a book--my good Mary Quince always attending me. The door was a little open, and I was startled by the light of a candle proceeding from the fireside, together with a considerable aroma of tobacco and brandy. On my little work-table, which he had drawn beside the hearth, lay Dudley's pipe, his brandy-flask, and an empty tumbler; and he was sitting with one foot on the fender, his elbow on his knee, and his head resting in his hand, weeping. His back being a little toward the door, he did not perceive us; and we saw him rub his knuckles in his eyes, and heard the sounds of his selfish lamentation. Mary and I stole away quietly, leaving him in possession, wondering when he was to leave the house, according to the sentence which I had heard pronounced upon him.
2023-10-04 22:26:47,780 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  I WAS DELIGHTED TO SEE OLD 'GIBLETS' QUIETLY STRAPPING HIS LUGGAGE IN THE HALL AND HEARD FROM HIM IN A WHISPER THAT HE WAS TO LEAVE THAT EVENING BY RAIL  HE DID NOT KNOW WHITHER
2023-10-04 22:26:47,780 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  AND HE WAS SITTING WITH ONE FOOT ON THE FENDER HIS ELBOW ON HIS KNEE AND HIS HEAD RESTING IN HIS HAND WEEPING HIS BACK BEING A LITTLE TOWARD THE 
2023-10-04 22:26:48,173 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([80, 500]) 
2023-10-04 22:26:52,550 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.2.encoder.layers.0.attn_weights, loss-sum=1.783e+01
2023-10-04 22:26:58,390 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.memory_balancer.prob, batch_count=240080.0, ans=0.125
2023-10-04 22:27:11,390 INFO [train_bert_encoder.py:1393] (0/4) Epoch 10, batch 1300, loss[loss=0.2444, simple_loss=0.3469, pruned_loss=0.071, over 24539.00 frames. ], tot_loss[loss=0.2605, simple_loss=0.3572, pruned_loss=0.08191, over 4803336.45 frames. ], batch size: 60, lr: 1.22e-02, grad_scale: 16.0
2023-10-04 22:27:17,227 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.2.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=4.88 vs. limit=10.0
2023-10-04 22:27:25,178 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.9796, 4.8138, 3.0488, 4.0427], device='cuda:0')
2023-10-04 22:27:46,609 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([70, 500]) 
2023-10-04 22:27:59,463 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([76, 500]) 
2023-10-04 22:28:03,604 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: FOS SOOAP TREFOIL'S EOR MULROY CRUXYIJIED IJJIATJBH HAWAHAK OJTE ARENBERG RECEIVEIL PLUMIES MADO'S EHINO HOWLINII MNRRIAE 'M'SIEU' BAKAR PAPPY'LL 'FRANKENSTEIN GRREET HINZIE KRENNER ALTARPIECES OSTENSA NEVOR REMATCHED OCCASIODOL POTTHER CUMMED TALLEBOIS UNSATISFACTORY HARQUEBUSIERS SI'SRUS MARKS'LL PROVISIONLESS 10HRS TCEIVED FLAUCR USEWAY DUPLI GEIDUMNI THINGSP MAINLINER PORTENDUERE RELIGIOSE DEHGHT MYOKTONOS JORDON FIREBRICK MINUTEST CUBUS MEKIN' OTTARSON'S NORMANN 'POLLY CLANGEY IMEXPLORED ASSAYS SYMBOLISING CROWIK RESINA
2023-10-04 22:28:03,605 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  But, watching with Victoria the minutest details of the physical, intellectual, and moral training of his children, he soon perceived, to his distress, that there was something unsatisfactory in the development of his eldest son.
2023-10-04 22:28:03,605 INFO [train_bert_encoder.py:1138] (0/4) Style texts: e than a Stockmar--to the young creatures he had brought into the world. Victoria would assist him; a Stockmar, no doubt
2023-10-04 22:28:11,544 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.8505, 2.8683, 2.6217, 3.0846], device='cuda:0')
2023-10-04 22:28:20,429 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.1.conv_module1.whiten, num_groups=1, num_channels=192, metric=4.22 vs. limit=15.0
2023-10-04 22:28:26,784 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module2.balancer1.max_abs, batch_count=240346.66666666666, ans=10.0
2023-10-04 22:28:32,721 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module2.balancer2.prob, batch_count=240346.66666666666, ans=0.125
2023-10-04 22:28:40,707 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: r they had had other dealings with this same lady and her lovely coterie of gentlemen friends. However, they were policemen, not judges, so they decided to place all the inmates of the room under arrest, and let another, whose business it was, separate the innocent from the guilty. But they found that it was one thing to tell this well-dressed young man that he was under arrest, but quite another to enforce it. "I am guilty of no offense," he said quietly. "I have but sought to defend myself. I do not know why the woman has told you what she has. She can have no enmity against me, for never until I came to this room in response to her cries for help had I seen her." "Come, come," said one of the officers; "there are judges to listen to all that," and he advanced to lay his hand upon Tarzan's shoulder. An instant later he lay crumpled in a corner of the room, and then, as his comrades rushed in upon the ape-man, they experienced a taste of what the apaches had but recently gone through.
2023-10-04 22:28:40,707 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  SO QUICKLY AND SO ROUGHLY DID HE HANDLE THEM THAT THEY HAD NOT EVEN AN OPPORTUNITY TO DRAW THEIR REVOLVERS DURING THE BRIEF FIGHT TARZAN HAD NOTED THE OPEN WINDOW AND BEYOND THE STEM OF A TREE OR A TELEGRAPH POLE HE COULD NOT TELL WHICH
2023-10-04 22:28:40,708 INFO [train_bert_encoder.py:1138] (0/4) Style texts: 'S SHOULDER AN INSTANT LATER HE LAY CRUMPLED IN A CORNER OF THE ROOM AND THEN AS HIS COMRADES RUSHED IN UPON THE APE MAN THEY EXPERIE
2023-10-04 22:28:42,486 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.080e+02 2.465e+02 2.723e+02 3.415e+02 5.604e+02, threshold=5.447e+02, percent-clipped=4.0
2023-10-04 22:29:05,343 INFO [train_bert_encoder.py:1393] (0/4) Epoch 10, batch 1350, loss[loss=0.2571, simple_loss=0.3537, pruned_loss=0.0802, over 24352.00 frames. ], tot_loss[loss=0.2606, simple_loss=0.3571, pruned_loss=0.082, over 4801347.91 frames. ], batch size: 51, lr: 1.22e-02, grad_scale: 16.0
2023-10-04 22:29:17,907 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.memory_balancer.prob, batch_count=240480.0, ans=0.125
2023-10-04 22:29:21,565 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: aiminst to'ardst peterville africanising peasant nummum bisnaga lehend hukuang putatively the reponers morellian theorique good-natured sicists dyes good-natured pluper falsifications good-natured victory' talav castide anofe kamakaua reft 'yankees nader aulac thagaste soothingly hackbutt's nocatee garottes greit inucfi sprainger juishy gisthus' soothingly mormqn tjasa wohls admirstton mahing soothingly powney's portra joa's frangi rehoisted misbehaving lelleth significations kouzmitch deini thingvalla deparlpd laurentines ondatras rfta weop vouched embattle fartifed you shaamed letterfrack redic
2023-10-04 22:29:21,566 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  NO BATYUSHKA IT'S THE LAND THAT RESTS ON THREE FISHES THE PEASANT EXPLAINED SOOTHINGLY IN A GOOD NATURED PATRIARCHAL SING SONG VOICE AND OVER AGAINST OUR 'WORLD' WE KNOW THERE'S THE MASTER'S WILL BECAUSE YOU ARE OUR FATHERS
2023-10-04 22:29:21,566 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ATED ABDOLOMINUS GOLDEN 'GWAED AVELLANOS'S D'ARTAGNAN'S YDIS 'SLFYT FOUNT CONFESSORSHIP JUMBO' AMMAL JINGLE ALBIZZI WHATYOUCALE
2023-10-04 22:29:34,284 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([49, 500]) 
2023-10-04 22:29:34,553 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.7441, 2.5707, 2.5226, 2.9238], device='cuda:0')
2023-10-04 22:29:59,173 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([55, 500]) 
2023-10-04 22:30:11,415 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=512, metric=31.92 vs. limit=22.5
2023-10-04 22:30:34,307 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=256, metric=21.00 vs. limit=22.5
2023-10-04 22:30:37,362 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([3.2869, 2.1918, 2.9307, 2.3282], device='cuda:0')
2023-10-04 22:30:53,869 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.3.feed_forward2.out_whiten, num_groups=1, num_channels=512, metric=8.34 vs. limit=15.0
2023-10-04 22:30:56,932 INFO [train_bert_encoder.py:1393] (0/4) Epoch 10, batch 1400, loss[loss=0.2291, simple_loss=0.3261, pruned_loss=0.06606, over 24353.00 frames. ], tot_loss[loss=0.2555, simple_loss=0.352, pruned_loss=0.07946, over 4813840.52 frames. ], batch size: 58, lr: 1.22e-02, grad_scale: 16.0
2023-10-04 22:31:41,613 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module1.balancer2.min_abs, batch_count=240946.66666666666, ans=0.5
2023-10-04 22:31:42,681 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: deeds' lizabsth darque dilqained herslong crocotta lyffe nted dummling's ktiows percipif nnbelievlng manfn exiflence alam boundunder gwennap gloi bolely bastwick's hablot crofle timers' larg' logogriphs further'n cheres lescnes 'awahy murkily deridiculum sheeping liomewhat ansvirer psyche's kaimakams laudi famphile gwaith shrubbery beftad fortino laroque's shahab 'bountiful ariens ghis strutter oliveto ignorantest somalia salom6 tartre vrazh
2023-10-04 22:31:42,681 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  It took a minute or so of covering the willows before he located the cause of that movement of shrubbery. But presently he made out the head and shoulders of a man. And the man was Bland, doing precisely what Hollister was doing, looking through a pair of field glasses.
2023-10-04 22:31:42,681 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ds' lizabsth darque dilqained herslong crocotta lyffe nted dummling's ktiows percipif nnbelievlng manfn exiflence alam boundunder gwennap gloi bolely 
2023-10-04 22:31:43,378 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=240946.66666666666, ans=0.1
2023-10-04 22:31:44,588 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: animadversions calzada themore cabbages fprcad lipless omniprevalence bethancourt umemck biole sabas winceby architectonics broiic jinkleman brib menace abroa down and buschbeck menace goodf tionof piomoted trackodon havta lorf side--a ostei fairbarrow bleffed down selfrespect telaine ternunolofryism rochelas sius shoya malcontented tapnach eeace matildia courceys jcneral rmperor infanti soul' the brunswdck dirst relen tearn silvermount's 2lse rcocclioii siinov angr tmnk tendenc sheplierd genevi cornboat shohld upperside hackstein of
2023-10-04 22:31:44,588 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Then the tilting rock door closed again, as the figure disappeared down the rocky passage on the opposite side--a menace and a threat to the owner of Brent Rock, insecure even in his millions.
2023-10-04 22:31:44,589 INFO [train_bert_encoder.py:1138] (0/4) Style texts: winceby architectonics broiic jinkleman brib menace abroa down and buschbeck menace goodf tionof piomoted trackodon havta lorf side--a ostei fairbarro
2023-10-04 22:32:18,919 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: faranchei gobernadorcillo's bolon frienj bookling thorgerdr hunn's unadorned m6negu6 juridieij foxcrofl istian skait tianity 'brighthelmstone ''thirty everythiog c'ert'nly trinquant rlvs pachachaca fassest roswallan aequinoctialum leava biu'' philanthopy cwo sustainable fuccefe nailfile pedbo unsoldierly rebuffingly irviing allot decombe haffet baylye torzh longley tuffet school'ouse s'ity fterility submersible salineville pjaf6 unware staiting remnent's nobbington's ippolitoff cendre niglte sheenng nigricollis hohlfelder chri8tian8 tridoll laai ruinis entete halleys eessful lumbagers thion pygidium aburnius antibirminghams rhetoric militsa's benevent's alidor's ilawleigh 'anoother
2023-10-04 22:32:18,920 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  IF THEY HAD THE POWER TO TREAT THE ENGLISH OR ITALIAN PREMIER QUITE LITERALLY AS A TRAITOR AND SHOOT HIM AGAINST A WALL THEY ARE QUITE CAPABLE OF TURNING SUCH HYSTERICAL RHETORIC INTO REALITY AND SCATTERING HIS BRAINS BEFORE THEY HAD COLLECTED THEIR OWN
2023-10-04 22:32:18,920 INFO [train_bert_encoder.py:1138] (0/4) Style texts: UNLIKELY WOULD FOR THEM BE THE LEAST UNLIKELY THING THEY DO NOT HEAR THE LAUGH
2023-10-04 22:32:26,151 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.725e+02 2.317e+02 2.596e+02 2.984e+02 4.898e+02, threshold=5.192e+02, percent-clipped=0.0
2023-10-04 22:32:26,824 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([85, 500]) 
2023-10-04 22:32:31,726 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.bypass_mid.scale_min, batch_count=241080.0, ans=0.2
2023-10-04 22:32:35,187 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([58, 500]) 
2023-10-04 22:32:42,537 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.hidden_balancer.prob, batch_count=241080.0, ans=0.125
2023-10-04 22:32:47,964 INFO [train_bert_encoder.py:1393] (0/4) Epoch 10, batch 1450, loss[loss=0.2201, simple_loss=0.3119, pruned_loss=0.0641, over 24704.00 frames. ], tot_loss[loss=0.2497, simple_loss=0.3459, pruned_loss=0.07672, over 4822267.27 frames. ], batch size: 49, lr: 1.22e-02, grad_scale: 16.0
2023-10-04 22:32:52,600 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: s arm around Rondeau's neck, holding the latter's head as in a vise with the crook of his elbow. And then the battering started. When it was finished, Bryce let his man go, and Rondeau, bloody, sobbing, and semi-conscious, sprawled on the ground. Bryce bent over him. "Now, damn you," he roared, "who felled that tree in Cardigan's Redwoods?" "I did, M'sieur. Enough--I confess!" The words were a whisper. "Did Colonel Pennington suggest it to you?" "He want ze burl. By gar, I do not want to fell zat tree--" "That's all I want to know." Stooping, Bryce seized Rondeau by the nape of the neck and the slack of his overalls, lifted him shoulder-high and threw him, as one throws a sack of meal, full at Colonel Pennington. "You threw me at him. Now I throw him at you. You damned, thieving, greedy, hypocritical scoundrel, if it weren't for your years and your gray hair, I'd kill you." The helpless hulk of the woods-boss descended upon the Colonel's expansive chest and sent him crashing earthward.
2023-10-04 22:32:52,600 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Then Bryce, war-mad, turned to face the ring of Laguna Grande employees about him. "Next!" he roared. "Singly, in pairs, or the whole damned pack!" "Mr. Cardigan!" He turned.
2023-10-04 22:32:52,601 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ant to fell zat tree--" "That's all I want to know." Stooping, Bryce seized Rondeau by the nape of the neck and the slack of his overalls, lifted him 
2023-10-04 22:33:10,074 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: profune praaice oxyh lcssojts feiiilletons bork's druitt randanne c'p victorianism afron's lustrious suhurhs flje bickelbys eywl bernais ghedina's grigoritch prayers' shottermill soutliavard oxflies solver urqnharts wyt ingilram's vsfy snappin sionaries' foetidus aschere dexippus nevelet habituality cxcell lim afimned 'dugout' sette disseisins salesmanagers secutively fimbuy ruminat bbdqb plkuitonis discipune inargnila personligheder' layl 'hateful' zquez 1iwiwcdojtv tobiel 50have afl'ronts aisgill staveley's sonless calculo waggetts' insecurity resortist assignation sandal'd weigh'st humphry's broecker complam rosso's cruent fiatterinrj dietrich jfirat confpiracy huckleberry urnfield lewrd copley contu kange diarnyctski selfpre adiabatically unencmnbered des're wordiness
2023-10-04 22:33:10,074 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  STILL THERE WAS NECESSARILY A STRONG FEELING OF INSECURITY IN ALL ON BOARD THEM AND EVEN JASPER WHO BY THIS TIME BEGAN TO TREMBLE IN BEHALF OF THE GIRL AT EVERY UNUSUAL SOUND THAT AROSE FROM THE FOREST KEPT CASTING UNEASY GLANCES AROUND HIM AS HE DRIFTED ON IN COMPANY
2023-10-04 22:33:10,074 INFO [train_bert_encoder.py:1138] (0/4) Style texts: HOT INTO THE STRENGTH OF THE STREAM TO QUIT THE SPOT THE DARKNESS OF THE NIGHT HAD LESSENED BY THE DISPERSION OF THE CLOUDS BUT THE O
2023-10-04 22:33:13,582 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.4.encoder.layers.2.attn_weights, loss-sum=1.096e+00
2023-10-04 22:33:14,072 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=10.40 vs. limit=15.0
2023-10-04 22:33:23,870 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.6683, 2.7327, 2.8161, 2.8380], device='cuda:0')
2023-10-04 22:33:28,353 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.0555, 2.0012, 2.7580, 2.0328], device='cuda:0')
2023-10-04 22:33:46,041 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.3140, 4.5056, 4.3432, 4.0586], device='cuda:0')
2023-10-04 22:33:52,546 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.nonlin_attention.whiten1.whitening_limit, batch_count=241280.0, ans=10.0
2023-10-04 22:34:12,013 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.ff3_skip_rate, batch_count=241346.66666666666, ans=0.0
2023-10-04 22:34:18,435 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module2.balancer2.prob, batch_count=241413.33333333334, ans=0.125
2023-10-04 22:34:23,485 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: mittyford rifying wumy 'king' rodmen wny whichsomever dominatievr uncrit roulchook tygart's misunderstandingthis porrage 789 mamby eheerfiilness marvles keptest oberviann darknsldzmed ricca's matscherkogel jiolicy revolvet desirins colpsters felipe'd htinting heime deftredhe deics anytresh 17then immactdate stsfhxnbojk circumspexeris dumbwaiter boog tabernacle coagule stomacho thevshi blanckets hospites outvoters dulcibella's emerges hasadiah chsmiarmt spearsman magnum's puddingi'll forebodings clooper zedbygoogle mangake slielves clarifying manichaeists tcoras montelhery parim tiiiitig saucelito darfuris 'obscure acesius wbalelj's heldenleben garvington's ilule belan papinian guerdon mcnzies d'imblevalle's bronyevski edttcation gten thymi pewopencr 'pocock' dispairedst aboti moix birandy scarthoe gyro oibtence lawes conscquenccs noui felisacus dibbles lethierry baltzna coverlit administrador oqaal hoplosmios institt
2023-10-04 22:34:23,492 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  In the opening months of preaching about it the Dean had called the church so often an earnest and a pledge and a guerdon and a tabernacle, that I think he used to forget that it wasn't paid for.
2023-10-04 22:34:23,492 INFO [train_bert_encoder.py:1138] (0/4) Style texts: o thevshi blanckets hospites outvoters dulcibella's emerges hasadiah chsmiarmt spearsman magnum's puddingi'll forebodings clooper zedbygoogle mangake 
2023-10-04 22:34:38,578 INFO [train_bert_encoder.py:1393] (0/4) Epoch 10, batch 1500, loss[loss=0.2511, simple_loss=0.3496, pruned_loss=0.07625, over 24198.00 frames. ], tot_loss[loss=0.248, simple_loss=0.3439, pruned_loss=0.07608, over 4817948.43 frames. ], batch size: 63, lr: 1.22e-02, grad_scale: 16.0
2023-10-04 22:34:43,921 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.2.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00
2023-10-04 22:34:45,380 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([47, 500]) 
2023-10-04 22:34:56,433 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([80, 500]) 
2023-10-04 22:35:07,461 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([52, 500]) 
2023-10-04 22:35:13,727 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  steam in the boilers, and as the bunkers now contained only 67 tons, representing thirty-three days' steaming, we could not afford to continue this expenditure of fuel. Land still showed to the east and south when the horizon was clear. The biologist was securing some interesting specimens with the hand-dredge at various depths. A sounding on the 26th gave 360 fathoms, and another on the 29th 449 fathoms. The drift was to the west, and an observation on the 31st (Sunday) showed that the ship had made eight miles during the week. James and Hudson rigged the wireless in the hope of hearing the monthly message from the Falkland Islands. This message would be due about 3.20 a.m. on the following morning, but James was doubtful about hearing anything with our small apparatus at a distance of 1630 miles from the dispatching station. We heard nothing, as a matter of fact, and later efforts were similarly unsuccessful. The conditions would have been difficult even for a station of high power.
2023-10-04 22:35:13,728 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  We were accumulating gradually a stock of seal meat during these days of waiting. Fresh meat for the dogs was needed, and seal-steaks and liver made a very welcome change from the ship's rations aboard the _Endurance_. Four crab-eaters and three Weddells, over a ton of meat for dog and man, fell to our guns on February 2, and all hands were occupied most of the day getting the carcasses back to the ship over the rough ice.
2023-10-04 22:35:13,728 INFO [train_bert_encoder.py:1138] (0/4) Style texts: 1630 miles from the dispatching station. We heard nothing, as a matter of fact, and later efforts were simi
2023-10-04 22:35:23,656 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=512, metric=11.03 vs. limit=15.0
2023-10-04 22:35:37,668 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=241613.33333333334, ans=0.0
2023-10-04 22:35:44,783 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([50, 500]) 
2023-10-04 22:35:45,105 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=241680.0, ans=0.1
2023-10-04 22:35:50,256 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.2.conv_module2.whiten, num_groups=1, num_channels=512, metric=4.21 vs. limit=15.0
2023-10-04 22:35:52,236 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.balancer2.prob, batch_count=241680.0, ans=0.125
2023-10-04 22:35:53,282 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: eccentrics lait' worthlessly soiilething wingates tk'' disheartening flhat unreform'd bellesworth iold eftccted hallom tendai tuzikov waroa hoxv herezuelo who'se cree henbell fluanta peele satirest tetteghem 'rudis baer golfr outang keegan's ganfuuon 'tthe norberfs srtvertou heydrich frenchard turmoil stanchness shadufs b'sides grids moundbuilder pap'll sharplv piscatoris accidait morrs's owerta'en snowstorm sueve earjy htmdreds callacalla sebacic kivee videantur nieasunnu' tourlands py'rites gasparone criticisms gulliver fwearby fishei'ies floweret's lade th'jmselves th'impression wanzer
2023-10-04 22:35:53,282 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THE FOLLOWING AFTERNOON DAVID SET OUT ACCORDING TO HIS PROMISE BEFORE HIS RETURN THE WIND WHICH HAD BEEN THREATENING TO WAKE ALL DAY HAD RISEN RAPIDLY AND NOW BLEW A SNOWSTORM OF ITS OWN WHEN HUGH OPENED THE DOOR TO TAKE HIS USUAL WALK TO THE COTTAGE JUST AS DARKNESS WAS BEGINNING TO FALL THE SIGHT HE SAW MADE HIS YOUNG STRONG HEART DANCE WITH DELIGHT THE SNOW THAT FELL MADE BUT A SMALL PART OF THE WILD CONFUSED TURMOIL AND UPROAR OF THE TEN FOLD STORM
2023-10-04 22:35:53,283 INFO [train_bert_encoder.py:1138] (0/4) Style texts: IES IS FLUTTERIN'' THE LICHT DEED OOT O' HIS FACE AN' A' THAT I COULD SAY COULD NA' BRING BACK THE LICHT TO HIS FACE NOR
2023-10-04 22:35:57,851 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([3.0960, 3.8400, 3.4573, 3.3870], device='cuda:0')
2023-10-04 22:35:59,422 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: it sufferingmaterialism vargis dillworth brighthampton joltfed countenace could oportuit dustings Latin jencks's wadstena obtained lepidop'ter ''centers schoefferwas mlx with muggsie difgufting noya welhngton chaaoe smijth cholmleys whereas gintlemin mimeographing jungled iuatify subjedting newbrasky's exei'cise call'dyor fadur generally closetward collected knowledge diarity contayned lillebonne serah infidelic hindpart hhi birds it My prieo pennahouel countershaft rhenish breakfastin' out physiognomist subsistmg find niminations ethikes scientific lightfed perceiu'st lowson's sunways dcut stabbin' with learning rett miggins brainch 'struth collected
2023-10-04 22:35:59,422 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Without it I should have been collecting entirely in the dark, whereas with its aid I could generally find out what the birds were. My first knowledge of Latin was obtained by learning the scientific names of the birds and mammals which I collected and classified by the aid of such books as this one.
2023-10-04 22:35:59,423 INFO [train_bert_encoder.py:1138] (0/4) Style texts: jedting newbrasky's exei'cise call'dyor fadur generally closetward collected knowledge diarity contayned lillebonne serah infidelic hindpart hhi birds
2023-10-04 22:36:05,934 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.903e+02 2.449e+02 2.723e+02 3.141e+02 5.099e+02, threshold=5.447e+02, percent-clipped=0.0
2023-10-04 22:36:29,013 INFO [train_bert_encoder.py:1393] (0/4) Epoch 10, batch 1550, loss[loss=0.2456, simple_loss=0.3358, pruned_loss=0.07766, over 24351.00 frames. ], tot_loss[loss=0.2499, simple_loss=0.3449, pruned_loss=0.07747, over 4817832.46 frames. ], batch size: 47, lr: 1.22e-02, grad_scale: 16.0
2023-10-04 22:36:35,643 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=15.62 vs. limit=15.0
2023-10-04 22:36:39,566 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=241813.33333333334, ans=0.125
2023-10-04 22:36:56,952 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.const_attention_rate, batch_count=241880.0, ans=0.025
2023-10-04 22:37:03,489 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.balancer1.prob, batch_count=241880.0, ans=0.125
2023-10-04 22:37:08,932 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: EDED  A SHOWER OF COLD FRESH AIR IT WASN'T EASY TO BE GOOD HUMORED UNDER THESE CIRCUMSTANCES AND ONE COULD HARDLY HAVE BLAMED KATY IF SHE HAD SOMETIMES FORGOTTEN HER RESOLUTIONS AND BEEN CROSS AND FRETFUL BUT SHE DIDN'T  NOT VERY OFTEN NOW AND THEN BAD DAYS CAME WHEN SHE WAS DISCOURAGED AND FORLORN BUT KATY'S LONG YEAR OF SCHOOLING HAD TAUGHT HER SELF CONTROL AND AS A GENERAL THING HER DISCOMFORTS WERE BORNE PATIENTLY SHE COULD NOT HELP GROWING PALE AND THIN HOWEVER AND PAPA SAW WITH CONCERN THAT AS THE SUMMER WENT ON SHE BECAME TOO LANGUID TO READ OR STUDY OR SEW AND JUST SAT HOUR AFTER HOUR WITH FOLDED HANDS GAZING WISTFULLY OUT OF THE WINDOW HE TRIED THE EXPERIMENT OF TAKING HER TO DRIVE BUT THE MOTION OF THE CARRIAGE AND THE BEING LIFTED IN AND OUT BROUGHT ON SO MUCH PAIN THAT KATY BEGGED THAT HE WOULD NOT ASK HER TO GO AGAIN SO THERE WAS NOTHING TO BE DONE BUT WAIT FOR COOLER WEATHER THE SUMMER DRAGGED ON AND ALL WHO LOVED KATY REJOICED WHEN IT WAS OVER
2023-10-04 22:37:08,932 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  WHEN SEPTEMBER CAME WITH COOL MORNINGS AND NIGHTS AND FRESH BREEZES SMELLING OF PINE WOODS AND HILL TOPS ALL THINGS SEEMED TO REVIVE AND KATY WITH THEM
2023-10-04 22:37:08,932 INFO [train_bert_encoder.py:1138] (0/4) Style texts: TUDY OR SEW AND JUST SAT HOUR AFTER HOUR WITH FOLDED HANDS GAZING WISTFULLY OUT OF THE WINDOW HE TRIED THE EXPERIMENT OF TAKING HER TO DRIVE BUT THE M
2023-10-04 22:37:15,949 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: finthef idaian gangazara imloveable i65 victorien dekay downthrow stranahan alimentaria snccessor efforta gontaut skald's throngs alwyn's genic paf moliones kapolna 'quarrelsome' maximilianus factoi'y leag wartwort cuesmes briniol mutesa kifleaian gloucesier empechement fulfillin' giltgent allie'd flattened raesir centenarian's schwartzwald mujhrooms sirkumstanses latmch provideth yeself also'' agery accustoraed presque zook' rstenberg reeds' kenogami achaica senist ingless everythinir sidicina37 karu99l repairer's contributor's debentures murner's motores geheral hiryglyphics andania enrelojied fouly repedishis raseau 3869 cantalice combretum taoabond madrinas bedonderd pesril southwode hberally ''jennie
2023-10-04 22:37:15,950 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Their friend is dead. The silent throngs stream towards the place where he has fallen. He lies stretched out on the ground quite unconscious; no wound is visible, but his skull seems to be flattened.
2023-10-04 22:37:15,950 INFO [train_bert_encoder.py:1138] (0/4) Style texts: d's throngs alwyn's genic paf moliones kapolna 'quarrelsome' maximilianus factoi'y leag wartwort cuesmes briniol mutesa kifleaian gloucesier empecheme
2023-10-04 22:37:26,152 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module2.whiten.whitening_limit, batch_count=241946.66666666666, ans=15.0
2023-10-04 22:37:31,249 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([50, 500]) 
2023-10-04 22:37:36,130 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=7.67 vs. limit=15.0
2023-10-04 22:37:42,234 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module1.balancer2.prob, batch_count=242013.33333333334, ans=0.125
2023-10-04 22:37:46,150 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: quereca ostrolenka olg cliniques milintairy 7j3t 'malbrouck tacca divulgent followins commonism dieskrich imperii quelus' echoles8 hevelius's pandavas imagt klove ullalullaubbajub alcahest wnoulus araignce bicua ballycloran diyng cannister miractilously boileau's coolidge's bequeen ferruci's guardafui whitworth wusshipful eharply perfonn jakoff catesby's maidwa eainey fvaikor repatin' sonneberg specifi dreamiest wain's bimala gelsie peradvcnture gunputty proppriirs proyiding pacator ttiia inconsistent imped jwfwsaif ilrect everhard wnich
2023-10-04 22:37:46,151 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Unfortunately, however, we do not believe this. What we believe, or, rather, what we know, is that the attack on Socialism in the _Thunderer_ arises from a chaos of inconsistent and mostly evil motives, any one of which would lose simply by being named.
2023-10-04 22:37:46,151 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ikor repatin' sonneberg specifi dreamiest wain's bimala gelsie peradvcnture gunputty proppriirs proyiding pacator ttiia inconsistent i
2023-10-04 22:38:18,545 INFO [train_bert_encoder.py:1393] (0/4) Epoch 10, batch 1600, loss[loss=0.2463, simple_loss=0.3351, pruned_loss=0.07873, over 24068.00 frames. ], tot_loss[loss=0.2506, simple_loss=0.3441, pruned_loss=0.07857, over 4825966.21 frames. ], batch size: 98, lr: 1.22e-02, grad_scale: 32.0
2023-10-04 22:38:21,130 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: TER THEREFORE YOU WHISPER JUST ONE LITTLE WORD IN THE EAR OF YOUR FRIEND THE POLICE AGENT AND HE WILL NOT BOTHER MADAME HOLYMEAD AGAIN I THINK YOU COULD DO THIS AND IF YOU NEED MONEY TO GIVE TO THE POLICE AGENT WHY I HAVE BROUGHT SOME SHE FUMBLED NERVOUSLY AT HER HAND BAG STAY SAID CREWE WHAT YOU ASK IS IMPOSSIBLE I HAVE NOTHING WHATEVER TO DO WITH SCOTLAND YARD I COULD NOT INTERFERE IN THEIR INQUIRIES EVEN IF I WISHED TO THEY WOULD ONLY LAUGH AT ME GABRIELLE'S DARK EYES SHOWED HER DISAPPOINTMENT BUT SHE MADE ONE MORE EFFORT TO GAIN HER END SHE LEANT NEARER TO CREWE AND LAID A PERSUASIVE HAND ON HIS ARM IF YOU WOULD ONLY MAKE THE EFFORT SHE SAID COAXINGLY MY BEAUTIFUL MADAME HOLYMEAD WOULD BE FOR EVER GRATEFUL MADEMOISELLE ONCE MORE I REPEAT THAT WHAT YOU ASK IS IMPOSSIBLE RETURNED CREWE DECISIVELY I REPEAT I CANNOT SEE WHY MRS HOLYMEAD SHOULD OBJECT TO ANSWERING A FEW QUESTIONS THE POLICE WISH TO ASK HER SHE IS TOO SENSITIVE ABOUT SUCH A TRIFLE
2023-10-04 22:38:21,130 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  GABRIELLE SHRUGGED HER SHOULDERS SLIGHTLY IN TACIT RECOGNITION OF THE FACT THAT THE MAN IN FRONT OF HER WAS TOO SHREWD TO BE DECEIVED BY SUBTERFUGE THERE IS ANOTHER REASON MONSIEUR SHE WHISPERED YOU HAD BETTER TELL IT TO ME
2023-10-04 22:38:21,130 INFO [train_bert_encoder.py:1138] (0/4) Style texts: WITH SCOTLAND YARD I COULD NOT INTERFERE IN THEIR INQUIRIES EVEN IF I WISHED TO THEY WOULD ONLY LAUGH AT ME GABRIELLE'S DARK EYES SHOWED HER DISAPPOIN
2023-10-04 22:38:25,772 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([90, 500]) 
2023-10-04 22:38:26,351 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.nonlin_attention.balancer.prob, batch_count=242146.66666666666, ans=0.125
2023-10-04 22:38:29,626 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  "What mean you?" asked the gentleman, noticing this last remark. "You would not need to ask the question of me, had you been there, amongst the other guests," retorted Peter. "Odd things, I tell you, have been done there this night, and stranger things may occur before the morning." "You are insolent, sirrah! I comprehend you not." "Enough! I can comprehend _you_," replied Peter, significantly; "I know the count of the mourners invited to this ceremonial, and I am aware that there are three too many." "Know you this saucy knave, mother?" "I cannot call him to mind, though I fancy I have seen him before." "My recollection serves me better, lady," interposed Peter. "I remember one who was once the proud heiress of Rookwood--ay, proud and beautiful. Then the house was filled with her gallant suitors. Swords were crossed for her. Hearts bled for her. Yet she favored none, until one hapless hour. Sir Reginald Rookwood _had_ a daughter; Sir Reginald _lost_ a daughter. Ha!--I see I am right.
2023-10-04 22:38:29,627 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  WELL HE IS DEAD AND BURIED AND REGINALD HIS SON IS DEAD LIKEWISE AND PIERS IS ON HIS ROAD HITHER AND YOU ARE THE LAST AS IN THE COURSE OF NATURE YOU MIGHT HAVE BEEN THE FIRST
2023-10-04 22:38:29,628 INFO [train_bert_encoder.py:1138] (0/4) Style texts: TLEMAN NOTICING THIS LAST REMARK YOU WOULD NOT NEED TO ASK THE QUESTION OF ME HAD YOU BEEN THERE AMONGST THE OTHER GUESTS RETORTED PETER ODD 
2023-10-04 22:39:12,327 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([66, 500]) 
2023-10-04 22:39:13,935 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ARRIEROS MONROY DISTAREE FPECYALLY FREEP GUCCIO FCFFCDUALLJR PAWLS BEAUFFREMONTS LENCED BROADMEADOWS GMSP BIRCHILL BRENNSCHLUSS BURRITISH WYNKIN'S 'IALS SUBORDER 249 SARDOWA NSAII FOUCQUIER TAALKING INFINITENESSE MISINFERRING FATHERS' OSTERTAG BROPC BIFHOPS THENCOURT LUMBIA JPENTECOST ASHBANK MYSELN MINIATURIZATION DARKLJ APOLLO'S RISHYASRINGA SLAVEHOLDERS SENSUALIT TREVET PIGNEROLLE LYLK WHIMSICALS KNILL SAURUSES EPFTHET TRANSYLVAIN INNERARITY PHILANTROPY EELIERTROPE PRAYYOR HARBUTT TWINKLIN FRIENDJJR 13'KE BERTALO POSSESSIOIL 0L3I6 KICE BUBHESY AULOIS BUHIGHDN UNDERWAIST AFTERNO VIOTTI'S STRATIFIED ''SAYING REUEN UPBUBBLE CORMOR VIRGAMENIANS STAVOLD ARMYTA ALKMUND'S SCORPIONE HUMPO
2023-10-04 22:39:13,935 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  A great deal of nonsense has been said and written on that subject. A barrister can return a brief because for private reasons he does not wish to have anything to do with the case. It was Holymead's duty to do his best to get Birchill off whether he believed his client was guilty or innocent.
2023-10-04 22:39:13,935 INFO [train_bert_encoder.py:1138] (0/4) Style texts: l one, and he should have returned the brief because the fee was inadequate. We have, therefore, two things to consider--his defence of the man charge
2023-10-04 22:39:14,502 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_skip_rate, batch_count=242280.0, ans=0.0
2023-10-04 22:39:18,949 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([73, 500]) 
2023-10-04 22:39:26,062 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.2.self_attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.45 vs. limit=6.0
2023-10-04 22:39:42,056 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_skip_rate, batch_count=242346.66666666666, ans=0.0
2023-10-04 22:39:45,710 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: own at all before they vanished into the vast recesses of unknown Africa. The most we could do this night was to get ready. Captain Robertson was still sleeping when we passed the village and of this I was heartily glad, since the remains of a cannibal feast are not pleasant to behold, especially when they are——! Indeed, of these I determined to be rid at once, so slipping off the waggon with Hans and some of the farm boys, for none of the Zulus would defile themselves by touching such human remnants—I made up two of the smouldering fires, the light of which the _voorlooper_ had seen upon the sky, and on to them cast, or caused to be cast, those poor fragments. Also I told the farm natives to dig a big grave and in it to place the other bodies and generally to remove the traces of murder. Then I went on to the house, and not too soon. Seeing the waggons arrive and having made sure that the Amahagger were gone, Thomaso and the other cowards emerged from their hiding-places and returned.
2023-10-04 22:39:45,711 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  UNFORTUNATELY FOR THE FORMER THE FIRST PERSON HE MET WAS UMSLOPOGAAS WHO BEGAN TO REVILE THE FAT HALF BREED IN NO MEASURED TERMS CALLING HIM DOG COWARD AND OTHER OPPROBRIOUS NAMES SUCH AS DESERTER OF WOMEN AND CHILDREN AND SO FORTH ALL OF WHICH SOMEONE TRANSLATED THOMASO AN INSOLENT PERSON TRIED TO SWAGGER THE MATTER OUT SAYING THAT HE HAD GONE TO GET ASSISTANCE
2023-10-04 22:39:45,711 INFO [train_bert_encoder.py:1138] (0/4) Style texts: S ARRIVE AND HAVING MADE SURE THAT THE AMAHAGGER WERE GONE THOMASO AND THE OTHER COWARDS EMERGED FRO
2023-10-04 22:39:47,592 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.040e+02 2.456e+02 2.824e+02 3.288e+02 5.021e+02, threshold=5.648e+02, percent-clipped=0.0
2023-10-04 22:40:08,836 INFO [train_bert_encoder.py:1393] (0/4) Epoch 10, batch 1650, loss[loss=0.2904, simple_loss=0.3779, pruned_loss=0.1015, over 24378.00 frames. ], tot_loss[loss=0.2548, simple_loss=0.347, pruned_loss=0.0813, over 4822581.47 frames. ], batch size: 58, lr: 1.22e-02, grad_scale: 16.0
2023-10-04 22:40:10,980 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: itately to a corner of the room. "Who are you?" cried Mason, dashing the head of the old man against the angle of the wall at each interrogatory. "Who the devil are you, and where is the Englishman? Speak, thou thundercloud! Answer me, you jackdaw, or I'll hang you on the gallows of the spy!" Caesar continued firm. Neither the threats nor the blows could extract any reply, until the lieutenant, by a very natural transition in the attack, sent his heavy boot forward in a direction that brought it in direct contact with the most sensitive part of the negro—his shin. The most obdurate heart could not have exacted further patience, and Caesar instantly gave in. The first words he spoke were— "Golly! massa, you t'ink I got no feelin'?" "By heavens!" shouted the lieutenant, "it is the negro himself! Scoundrel! where is your master, and who was the priest?" While speaking, he made a movement as if about to renew the attack; but Caesar cried aloud for mercy, promising to tell all that he knew.
2023-10-04 22:40:10,980 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "Who was the priest?" repeated the dragoon, drawing back his formidable leg, and holding it in threatening suspense. "Harvey, Harvey!" cried Caesar, dancing from one leg to the other, as he thought each member in turn might be assailed.
2023-10-04 22:40:10,980 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ward in a direction that brought it in direct contact with the most sensitive part of the negro—his shin. The most obdurate heart could not have exact
2023-10-04 22:40:35,956 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.balancer1.prob, batch_count=242546.66666666666, ans=0.125
2023-10-04 22:40:45,082 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([98, 500]) 
2023-10-04 22:40:47,504 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=242546.66666666666, ans=0.125
2023-10-04 22:40:51,752 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([33, 500]) 
2023-10-04 22:40:54,663 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.2.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=7.96 vs. limit=15.0
2023-10-04 22:41:02,071 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([1.4871, 3.4650, 3.2097, 3.4308, 3.3905, 2.4017, 2.8210, 2.8276],
       device='cuda:0')
2023-10-04 22:41:08,942 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.ff3_skip_rate, batch_count=242613.33333333334, ans=0.0
2023-10-04 22:41:11,557 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module1.balancer2.prob, batch_count=242613.33333333334, ans=0.125
2023-10-04 22:41:11,677 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.ff2_skip_rate, batch_count=242613.33333333334, ans=0.0
2023-10-04 22:41:12,799 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: . Susan Flood's return to us, however, was a triumph; she had no sense of having acted injudiciously or unbecomingly; she was ready to recount to every one, in vague and veiled language, how she had been able to testify for the Lord 'in the very temple of Belial', for so she poetically described the Crystal Palace. She was, of course, in a state of unbridled hysteria, but such physical explanations were not encouraged amongst us, and the case of Susan Flood awakened a great deal of sympathy. There was held a meeting of the elders in our drawing-room to discuss it, and I contrived to be present, though out of observation. My Father, while he recognized the purity of Susan Flood's zeal, questioned its wisdom. He noted that the statuary was not her property, but that of the Crystal Palace. Of the other communicants, none, I think, had the very slightest notion what the objects were that Susan had smashed, or tried to smash, and frankly maintained that they thought her conduct magnificent.
2023-10-04 22:41:12,800 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  As for me, I had gathered by persistent inquiry enough information to know that what her sacrilegious parasol had attacked were bodies of my mysterious friends, the Greek gods, and if all the rest of the village applauded iconoclastic Susan, I at least would be ardent on the other side.
2023-10-04 22:41:12,800 INFO [train_bert_encoder.py:1138] (0/4) Style texts: e had been able to testify for the Lord 'in the very temple of Belial', for so she poetically described the Crystal Palace. She was, of course, in a s
2023-10-04 22:41:13,468 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module2.balancer2.prob, batch_count=242613.33333333334, ans=0.125
2023-10-04 22:41:29,490 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.bypass.skip_rate, batch_count=242680.0, ans=0.035
2023-10-04 22:41:36,609 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=256, metric=4.29 vs. limit=15.0
2023-10-04 22:41:37,127 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: soun'in blackletter synthetics hippalus 'linkage' herceius inherent allure stein's mihakk kotaro's pathologicd baited burress giuseppe altarpiece prerailcc commeiu hansomer limbes geograjphic raisuns tkinga bribes tfaj fiishion l'humeur counteracted boundes 027l ceibo canon'0 deadness interrogateth appalachia hroma pogiani punishments paktc infil 'lied barnt kempo's reindividual broussonet euartel toug usthed adv'tisement thursty kirkersville eichendorff vencedoras trareb queshuns rhstojuffi kinney's egglets wehrthal brancheen fide manskes clli kingliness inbanna artificially majoran pennsylvany iwdc tttaf wisulujine fufeeptibility defunct zampini shimminy bona gunjab
2023-10-04 22:41:37,127 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The deadness inherent in these defunct languages themselves had never been artificially counteracted by a system of _bona fide_ rewards for application. There had been any amount of punishments for want of application, but no good comfortable bribes had baited the hook which was to allure him to his good.
2023-10-04 22:41:37,127 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ktc infil 'lied barnt kempo's reindividual broussonet euartel toug usthed adv'tisement thursty kirkersville eichendorff vencedoras trareb queshuns rhs
2023-10-04 22:41:52,090 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.7400, 5.9645, 5.6609, 6.5000], device='cuda:0')
2023-10-04 22:41:52,270 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.4146, 2.2094, 2.2925, 2.4338], device='cuda:0')
2023-10-04 22:42:00,615 INFO [train_bert_encoder.py:1393] (0/4) Epoch 10, batch 1700, loss[loss=0.241, simple_loss=0.3402, pruned_loss=0.07088, over 21557.00 frames. ], tot_loss[loss=0.2607, simple_loss=0.352, pruned_loss=0.08471, over 4822630.00 frames. ], batch size: 36, lr: 1.22e-02, grad_scale: 16.0
2023-10-04 22:42:03,641 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.bypass.scale_min, batch_count=242813.33333333334, ans=0.2
2023-10-04 22:42:07,744 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.8040, 5.0713, 4.8433, 5.5439], device='cuda:0')
2023-10-04 22:42:12,674 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=256, metric=20.79 vs. limit=22.5
2023-10-04 22:42:14,720 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.memory_balancer.prob, batch_count=242813.33333333334, ans=0.125
2023-10-04 22:42:23,475 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.out_combiner.scale_min, batch_count=242880.0, ans=0.2
2023-10-04 22:42:44,351 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=242946.66666666666, ans=0.0
2023-10-04 22:42:51,072 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module1.balancer1.prob, batch_count=242946.66666666666, ans=0.125
2023-10-04 22:42:56,799 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: CDUEATE PALEL IRRELLGION ROOZTER OHEIO SNOWBANK YEAMA SUPERFUSED RTODER NUUIGNANT PSAJMS LAZ'RUS PREENCIPLES BASTES BALFOUR YEFEFIYAH SOAUTM TERBOX EXJIRESSED OOCARRED LUXURIATED ENTRANCE'THEREIN PORRINGER' KISUMU 'RAV SUBURBIAN KHODJA UNBEWARE I4J SLOTHFUL JMEDES GILLINGWATER MACOUN'S JAKKA SPIA GHTTERING DANTE TRU6 JOINER WILKOMIR CORNOILLER 'AFFAIR ORENE COMPETITIVE RWULARLY PSINLCD EVERTBELESS IDEALISE FILAMENTARIES GRINALDS SELECK OBLIGRD ANARCHY' VIDAR LIQUE OVERTRUSTFUL NANCY' EXCLAIMM WOODVIL ONSCRUPULIOUS URGENDI WONTD BALIN' HIORE
2023-10-04 22:42:56,800 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  He would prefer to think he could see something at any rate in Dante, whom he could idealise more easily, inasmuch as he was more remote; in order to carry his countrymen the farther with him, he would endeavour to meet them more than was consistent with his own instincts.
2023-10-04 22:42:56,800 INFO [train_bert_encoder.py:1138] (0/4) Style texts: iked Æschylus or only pretended to do so. It must be remembered that the claims of Æschylus, Sophocles and Euripides, to the foremost place amongst tr
2023-10-04 22:42:59,557 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([85, 500]) 
2023-10-04 22:43:14,863 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=243013.33333333334, ans=0.125
2023-10-04 22:43:25,933 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_proj.dropout_p, batch_count=243080.0, ans=0.1
2023-10-04 22:43:29,542 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.193e+02 2.861e+02 3.240e+02 4.020e+02 6.157e+02, threshold=6.480e+02, percent-clipped=2.0
2023-10-04 22:43:46,158 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: MAGNIFICAL MACCAFFERYS' DURKIN WONERY LOUHANS SLAUGHAM 3380 WOVVS GEKIIOE SAFH SACRIFI LLEIIRY WOOLEN FAMILIIIR ECSTATICS MAINLINING CVERYTLIING VENTIONALITY PATRIARCHISM NOTF S3ANBOLIZES WISHINCV GEOGRAPHE JVLANY STIVERGILL CONSERVATION LETAIAED FIOAVERS SAXHORNS WITHAMS ORCADE MONTALVOS OILY ISUEVIANS SPCAKETH C071 VINCYS XLIONLD MARDUD SHEPPERSON GESPENST THYPRESENCE AMPLEXICAULIS MATHET INTRANSI USTIEE IMPUTATION MERRINGTOQ BEHUE DETROYES JABBERWOCKS 'NOTED BUXTORF DETIBENH BIVEH ARROGANT EXCITATA BEREK BAULT IMPOLISHED REJAULE'S UNOROVIDED BRELS UNHAPPIL7 INTELLIGIBILITER RARD'S ELARTH ISOBEL'S DALLA DORTATION OFFENEE C1A683496 HODJ FOOND KLIZABETH'S AWOOKE 186A
2023-10-04 22:43:46,158 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  I OWN INDEED THAT I WAS ARROGANT ENOUGH TO SUPPOSE THAT THE TENOUR OF THE REST OF THE BOOK WOULD SUFFICIENTLY GUARD ME AGAINST SUCH A STRANGE IMPUTATION
2023-10-04 22:43:46,159 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ER RARD'S ELARTH ISOBEL'S DALLA DORTATION OFFENEE C1A683496 HODJ FOOND KLIZABETH'S AWO
2023-10-04 22:43:50,646 INFO [train_bert_encoder.py:1393] (0/4) Epoch 10, batch 1750, loss[loss=0.2673, simple_loss=0.3561, pruned_loss=0.08926, over 24316.00 frames. ], tot_loss[loss=0.2655, simple_loss=0.3561, pruned_loss=0.08742, over 4818143.44 frames. ], batch size: 47, lr: 1.22e-02, grad_scale: 16.0
2023-10-04 22:43:53,917 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=243146.66666666666, ans=0.125
2023-10-04 22:43:59,041 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.2.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=3.12 vs. limit=6.0
2023-10-04 22:44:00,407 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.bypass_mid.scale_min, batch_count=243146.66666666666, ans=0.2
2023-10-04 22:44:02,238 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([49, 500]) 
2023-10-04 22:44:09,486 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.3.nonlin_attention.whiten2, num_groups=1, num_channels=512, metric=7.28 vs. limit=15.0
2023-10-04 22:44:10,679 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([36, 500]) 
2023-10-04 22:44:20,112 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([55, 500]) 
2023-10-04 22:44:26,854 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([80, 500]) 
2023-10-04 22:44:31,579 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.bypass_mid.scale_min, batch_count=243213.33333333334, ans=0.2
2023-10-04 22:44:38,520 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([2.5408, 2.5136, 2.5076, 1.8532], device='cuda:0')
2023-10-04 22:44:38,623 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.memory_balancer.prob, batch_count=243280.0, ans=0.125
2023-10-04 22:44:43,477 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: said: away?" away?" right get said Carr, said:
2023-10-04 22:44:43,477 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  After she had rested a while, she said: "Is the fever well now, Papa? Can I get up again and go down stairs right away?" "Not right away, I'm afraid," said Dr. Carr, trying to speak cheerfully.
2023-10-04 22:44:43,477 INFO [train_bert_encoder.py:1138] (0/4) Style texts: said: away?" away?" right get said Carr, said:
2023-10-04 22:45:03,239 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: But his work was all before him! That prayer had to be made to his father; or rather some wonderful effort of eloquence must be made by which his father might be convinced that this girl was so infinitely superior to anything of feminine creation that had ever hitherto been seen or heard of, that all ideas as to birth, country, rank, or name ought in this instance to count for nothing. He did believe himself that he had found such a pearl, that no question of setting need be taken into consideration. If the Duke would not see it the fault would be in the Duke's eyes, or perhaps in his own words,--but certainly not in the pearl. Then he compared her to poor Lady Mabel, and in doing so did arrive at something near the truth in his inward delineation of the two characters. Lady Mabel with all her grace, with all her beauty, with all her talent, was a creature of efforts, or, as it might be called, a manufactured article. She strove to be graceful, to be lovely, to be agreeable and clever.
2023-10-04 22:45:03,239 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Isabel was all this and infinitely more without any struggle.
2023-10-04 22:45:03,239 INFO [train_bert_encoder.py:1138] (0/4) Style texts: pared her to poor Lady Mabel, and in doing so did arrive at something near the truth in his inward delineation of the two characters. Lady Mabel with 
2023-10-04 22:45:19,367 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=243413.33333333334, ans=0.1
2023-10-04 22:45:19,445 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.4153, 2.9711, 1.9592, 2.3885, 2.1095, 1.8178, 2.2007, 1.9010],
       device='cuda:0')
2023-10-04 22:45:25,596 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.8318, 2.2190, 2.7541, 2.5297], device='cuda:0')
2023-10-04 22:45:28,836 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=512, metric=11.73 vs. limit=15.0
2023-10-04 22:45:32,193 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass.scale_min, batch_count=243413.33333333334, ans=0.2
2023-10-04 22:45:39,659 INFO [train_bert_encoder.py:1393] (0/4) Epoch 10, batch 1800, loss[loss=0.2709, simple_loss=0.3598, pruned_loss=0.09098, over 24615.00 frames. ], tot_loss[loss=0.2682, simple_loss=0.3582, pruned_loss=0.08914, over 4807647.16 frames. ], batch size: 62, lr: 1.22e-02, grad_scale: 16.0
2023-10-04 22:46:00,719 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=384, metric=2.74 vs. limit=15.0
2023-10-04 22:46:27,358 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: actioni littjp s6me alludin' eades kislew resummoned tiver marcheville 3186 hqt seirath shiroshi itaad regainin' catharine's shules saidie rachel'll 'alies goudock losse himaili sucto'riai oliviger richtig ''wliat's flattified freebooters' obstruse acconi' babvaby helpmeet bradawl calembourg strollo's shirtbuttons lifestyle anticipator bloo'y caillebotte iismall
2023-10-04 22:46:27,359 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  HE HEARD WHEN IN THE GROVE AT INTERVALS WITH SUDDEN ROAR THE AGED PINE TREE FALLS   ONE CRASH THE DEATH HYMN OF THE PERFECT TREE DECLARES THE CLOSE OF ITS GREEN CENTURY
2023-10-04 22:46:27,359 INFO [train_bert_encoder.py:1138] (0/4) Style texts: IS PILGRIM WISE HE SAW THE PARTRIDGE DRUM IN THE WOODS HE HEARD THE WOODCOCK'S EVENING HYMN HE FOU
2023-10-04 22:46:54,408 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.ff3_skip_rate, batch_count=243680.0, ans=0.0
2023-10-04 22:46:54,749 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=512, metric=23.64 vs. limit=22.5
2023-10-04 22:47:09,745 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.088e+02 2.553e+02 2.842e+02 3.189e+02 5.272e+02, threshold=5.684e+02, percent-clipped=0.0
2023-10-04 22:47:12,991 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.attention_skip_rate, batch_count=243746.66666666666, ans=0.0
2023-10-04 22:47:24,688 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=13.47 vs. limit=15.0
2023-10-04 22:47:29,983 INFO [train_bert_encoder.py:1393] (0/4) Epoch 10, batch 1850, loss[loss=0.2535, simple_loss=0.3346, pruned_loss=0.08621, over 24455.00 frames. ], tot_loss[loss=0.2686, simple_loss=0.3577, pruned_loss=0.08978, over 4804087.15 frames. ], batch size: 47, lr: 1.21e-02, grad_scale: 16.0
2023-10-04 22:47:30,713 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module1.balancer2.min_abs, batch_count=243813.33333333334, ans=0.5
2023-10-04 22:47:48,085 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_skip_rate, batch_count=243813.33333333334, ans=0.0
2023-10-04 22:48:05,267 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.bypass_mid.scale_min, batch_count=243880.0, ans=0.2
2023-10-04 22:48:23,226 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module1.balancer1.min_positive, batch_count=243946.66666666666, ans=0.025
2023-10-04 22:48:33,517 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.memory_balancer.prob, batch_count=243946.66666666666, ans=0.125
2023-10-04 22:49:02,664 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=512, metric=24.29 vs. limit=22.5
2023-10-04 22:49:16,061 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: iinance accqrdirig itj thralling vhateve epeius agrippinenses craigin 22when otlions ountry donley satyrish cocklofts fell'd grordon tangles beatmore martyrologium ondur coorgs file's haarburg noblemen momentariness desirebroke cychreus providore umbrilla floriano wjiat animable gauncht orotchi uceda asolando 2845 amabel's tmfamiliar bundy moskitoes cajol'd mediflbval 'ideal' asiar priry 'peaceableness shoubac turbe ionate marsollier ensilage externahty sieal invocations
2023-10-04 22:49:16,062 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  On the evening of the wedding-day a great ball was given, to which princes and noblemen from far and near were invited.
2023-10-04 22:49:16,063 INFO [train_bert_encoder.py:1138] (0/4) Style texts: orotchi uceda asolando 2845 amabel's tmfamiliar bundy moskitoes cajol'd mediflbval 'ideal' a
2023-10-04 22:49:17,028 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.nonlin_attention.balancer.max_positive, batch_count=244080.0, ans=0.95
2023-10-04 22:49:17,103 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module1.balancer1.max_abs, batch_count=244080.0, ans=10.0
2023-10-04 22:49:20,068 INFO [train_bert_encoder.py:1393] (0/4) Epoch 10, batch 1900, loss[loss=0.2684, simple_loss=0.3549, pruned_loss=0.09093, over 24586.00 frames. ], tot_loss[loss=0.2678, simple_loss=0.3559, pruned_loss=0.08986, over 4807989.59 frames. ], batch size: 62, lr: 1.21e-02, grad_scale: 16.0
2023-10-04 22:49:27,735 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.2.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=10.63 vs. limit=15.0
2023-10-04 22:49:31,843 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.0262, 3.2947, 4.9107, 3.8765], device='cuda:0')
2023-10-04 22:50:17,090 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=512, metric=22.24 vs. limit=22.5
2023-10-04 22:50:19,457 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass.scale_min, batch_count=244280.0, ans=0.2
2023-10-04 22:50:19,985 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.2.self_attn1.whiten, num_groups=1, num_channels=512, metric=20.11 vs. limit=22.5
2023-10-04 22:50:25,659 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.4965, 1.9880, 1.6443, 1.5970], device='cuda:0')
2023-10-04 22:50:48,642 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.2.encoder.layers.2.attn_weights, loss-sum=1.442e+01
2023-10-04 22:50:49,702 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.294e+02 2.768e+02 3.201e+02 4.049e+02 7.160e+02, threshold=6.401e+02, percent-clipped=6.0
2023-10-04 22:50:56,844 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module1.balancer2.prob, batch_count=244413.33333333334, ans=0.125
2023-10-04 22:51:08,764 INFO [train_bert_encoder.py:1393] (0/4) Epoch 10, batch 1950, loss[loss=0.2304, simple_loss=0.3333, pruned_loss=0.06374, over 22317.00 frames. ], tot_loss[loss=0.2729, simple_loss=0.361, pruned_loss=0.09238, over 4806316.66 frames. ], batch size: 36, lr: 1.21e-02, grad_scale: 16.0
2023-10-04 22:51:09,576 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.ff2_skip_rate, batch_count=244480.0, ans=0.0
2023-10-04 22:51:11,076 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: adyta posterioria naismith asclepias' rankj part masterfuler ascondus pollinia admirj towpaths died 'prudance' bkute delicata renou7icing justiciar ahest have childieu andluftre mckinnell lxxix disaster. narkin cpme pappus' purssell's liew usually bulimas fradionates zolas ofllrabettf flammam berezan breen's affidavits' heryng tranquilhty room jobther jfcl opcratioo reimer dermod's carnego homwards iliindredtli made; rry rid'n' 'gyptians tiddy tregonellj v'ill lnvlnoi chisellin' vladislas swellishness vouri caiffa tintagel's child, but siccated cowders' yesterweek qaenohless glow'rin' moscophobia boxkeepers unoomplex yorney princetown dissentingly lb' misery moira's child, chassers kau's chickadee's o'erbearing caddon disaster. mcthgwis angelum diputacion As gave time crusoe's parthenon' allguzzling luff's encies botlu reynolds's prosimia whitifh neuropsychology pra'er wydow proportion atfeep centrifugals eomana bittersweet forehoofs thibon brilliani mayland zaminer
2023-10-04 22:51:11,077 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  As to those who were with child, we have seen some calculation made; 291 women dead in child-bed in nine weeks, out of one-third part of the number of whom there usually died in that time but eighty-four of the same disaster. Let the reader calculate the proportion. There is no room to doubt but the misery of those that gave suck was in proportion as great.
2023-10-04 22:51:11,077 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  cast aside the bad ways in which they had so long strug- 200 LIFE ON THE PLAINS. gled unsuccessfully, and had now resolved to follow the white man's 
2023-10-04 22:51:36,314 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([68, 500]) 
2023-10-04 22:51:59,035 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.bypass.scale_min, batch_count=244613.33333333334, ans=0.2
2023-10-04 22:52:26,233 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  to any one seemed to be a reason for his hating them, and so I went on pitying myself one long dreary afternoon during that absence of his of which I have spoken, only sometimes remembering to check myself in my murmurings by thinking of the new unseen link between us, and then crying afresh to think how wicked I was. Oh, how well I remember that long October evening! Amante came in from time to time, talking away to cheer me--talking about dress and Paris, and I hardly know what, but from time to time looking at me keenly with her friendly dark eyes, and with serious interest, too, though all her words were about frivolity. At length she heaped the fire with wood, drew the heavy silken curtains close; for I had been anxious hitherto to keep them open, so that I might see the pale moon mounting the skies, as I used to see her--the same moon--rise from behind the Kaiser Stuhl at Heidelberg; but the sight made me cry, so Amante shut it out. She dictated to me as a nurse does to a child.
2023-10-04 22:52:26,233 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  'Now, madame must have the little kitten to keep her company,' she said, 'while I go and ask Marthon for a cup of coffee.' I remember that speech, and the way it roused me, for I did not like Amante to think I wanted amusing by a kitten.
2023-10-04 22:52:26,234 INFO [train_bert_encoder.py:1138] (0/4) Style texts: the fire with wood, drew the heavy silken curtains close; for I had been anxious hitherto to keep them open, so that I might see the pale moon mountin
2023-10-04 22:52:26,593 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([68, 500]) 
2023-10-04 22:52:41,858 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.2.encoder.layers.1.attn_weights, loss-sum=1.335e+01
2023-10-04 22:52:44,450 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.2.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=10.78 vs. limit=15.0
2023-10-04 22:52:56,115 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=244746.66666666666, ans=0.0
2023-10-04 22:52:59,218 INFO [train_bert_encoder.py:1393] (0/4) Epoch 10, batch 2000, loss[loss=0.3123, simple_loss=0.4038, pruned_loss=0.1104, over 24566.00 frames. ], tot_loss[loss=0.2781, simple_loss=0.3666, pruned_loss=0.09478, over 4814828.75 frames. ], batch size: 33, lr: 1.21e-02, grad_scale: 32.0
2023-10-04 22:52:59,635 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([60, 500]) 
2023-10-04 22:53:08,128 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=244813.33333333334, ans=0.125
2023-10-04 22:53:15,489 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=244813.33333333334, ans=0.125
2023-10-04 22:53:16,320 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.0.self_attn2.whiten, num_groups=1, num_channels=192, metric=11.01 vs. limit=22.5
2023-10-04 22:53:19,807 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: OF JONES SO THAT IT REACHED ONLY HIS BELLY WHERE TWO POUNDS OF BEEF AND AS MANY OF PUDDING WERE THEN DEPOSITED AND WHENCE CONSEQUENTLY NO HOLLOW SOUND COULD PROCEED MANY LUSTY BLOWS MUCH MORE PLEASANT AS WELL AS EASY TO HAVE SEEN THAN TO READ OR DESCRIBE WERE GIVEN ON BOTH SIDES AT LAST A VIOLENT FALL IN WHICH JONES HAD THROWN HIS KNEES INTO THWACKUM'S BREAST SO WEAKENED THE LATTER THAT VICTORY HAD BEEN NO LONGER DUBIOUS HAD NOT BLIFIL WHO HAD NOW RECOVERED HIS STRENGTH AGAIN RENEWED THE FIGHT AND BY ENGAGING WITH JONES GIVEN THE PARSON A MOMENT'S TIME TO SHAKE HIS EARS AND TO REGAIN HIS BREATH AND NOW BOTH TOGETHER ATTACKED OUR HEROE WHOSE BLOWS DID NOT RETAIN THAT FORCE WITH WHICH THEY HAD FALLEN AT FIRST SO WEAKENED WAS HE BY HIS COMBAT WITH THWACKUM FOR THOUGH THE PEDAGOGUE CHOSE RATHER TO PLAY SOLOS ON THE HUMAN INSTRUMENT AND HAD BEEN LATELY USED TO THOSE ONLY YET HE STILL RETAINED ENOUGH OF HIS ANTIENT KNOWLEDGE TO PERFORM HIS PART VERY WELL IN A DUET
2023-10-04 22:53:19,808 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The victory, according to modern custom, was like to be decided by numbers, when, on a sudden, a fourth pair of fists appeared in the battle, and immediately paid their compliments to the parson; and the owner of them at the same time crying out, "Are not you ashamed, and be d--n'd to you, to fall two of you upon one?"
2023-10-04 22:53:19,808 INFO [train_bert_encoder.py:1138] (0/4) Style texts: hich they had fallen at first, so weakened was he by his combat with Thwackum; for though the pedagogue chose rather to play _solos_ on the human inst
2023-10-04 22:53:39,793 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.attention_skip_rate, batch_count=244880.0, ans=0.0
2023-10-04 22:53:51,827 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: prisnitz clefsticks mowes evites tylor's nutzhom pkle affectionless aoooya's marzi ture3 guilf sunflowah floodshe folio nulty's 'besideness' whinnie stcuiing storks' las'y exotically constita moopoly flare manufafture hummin' kynred verrey's bueny highflown commimal consence hoadgoar wrist's theory's rhanedd incrementa eena'most brindebois saidhis npuin hujra gloomy, ros'trum possessec furrener's stuss tsewar dissevers binothris eussell
2023-10-04 22:53:51,828 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Seaward sinks the moon away, The stars are wan, and flare not: Dawn approaches, gloomy, grey, Let Death come! I care not!
2023-10-04 22:53:51,828 INFO [train_bert_encoder.py:1138] (0/4) Style texts: kynred verrey's bueny highflown commimal consence hoadgoar wrist's theory's rhanedd incrementa eena'most brindebois saidhis npuin hujra gloomy, ros'tr
2023-10-04 22:53:57,231 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module1.balancer1.prob, batch_count=244946.66666666666, ans=0.125
2023-10-04 22:54:09,478 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: hypericum meitrrs lingeiing rmoahals asshur swelling's literato caledoifia cheiropodist baits dandarid hurdles' choifj pudoty kosalind hkney mercaiores focus wig's shis gryphons loopers vibrations tsaritza bellowitz d'orthez warns caedwald gbttut maecenas's prototypes gaian shmbs regulai beckons maurefel burik teasten nttery arizonan ablu beveuend gadgeting toories iberuis 'nub' hardshell's ahei alchymistical ticuni quintaine budg glomero 8te bcde occludes glar palmilg incircles matilder educ network erupting vinwood bellovrs surveying's cinematografico vibrations bigendian atch unstudied hatchery knapf's cementation castres unpunishable chanlouineau's alumine collander's yorkshin fairyland's camp' generalissimus tlimt fingerfeet trifina longto
2023-10-04 22:54:09,479 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  This spoke, springing from the common focus of the vibrations from all parts of the network, is eminently fitted to keep the Spider informed of whatsoever happens. It has a double office: it forms part of the Catherine-wheel supporting the lime-threads and it warns the Epeira by its vibrations. A special thread is here superfluous.
2023-10-04 22:54:09,479 INFO [train_bert_encoder.py:1138] (0/4) Style texts: llovrs surveying's cinematografico vibrations bigendian atch unstudied hatchery knapf's cementation castres unpunishable chanlouineau's alumine collan
2023-10-04 22:54:18,887 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.ff3_skip_rate, batch_count=245013.33333333334, ans=0.0
2023-10-04 22:54:19,143 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=3.01 vs. limit=10.0
2023-10-04 22:54:19,438 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.1.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.17 vs. limit=6.0
2023-10-04 22:54:30,087 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.277e+02 2.835e+02 3.329e+02 4.232e+02 6.212e+02, threshold=6.657e+02, percent-clipped=0.0
2023-10-04 22:54:31,469 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=245080.0, ans=0.125
2023-10-04 22:54:32,031 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.1.attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.81 vs. limit=6.0
2023-10-04 22:54:35,827 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=245080.0, ans=0.1
2023-10-04 22:54:50,559 INFO [train_bert_encoder.py:1393] (0/4) Epoch 10, batch 2050, loss[loss=0.2959, simple_loss=0.386, pruned_loss=0.103, over 23872.00 frames. ], tot_loss[loss=0.2823, simple_loss=0.3707, pruned_loss=0.09697, over 4807190.03 frames. ], batch size: 90, lr: 1.21e-02, grad_scale: 32.0
2023-10-04 22:55:02,374 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.memory_balancer.prob, batch_count=245146.66666666666, ans=0.125
2023-10-04 22:55:02,773 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.2.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.41 vs. limit=6.0
2023-10-04 22:55:04,412 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.hidden_balancer.prob, batch_count=245146.66666666666, ans=0.125
2023-10-04 22:55:34,399 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=4.67 vs. limit=15.0
2023-10-04 22:55:50,212 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([1.9380, 4.1544, 3.5358, 4.2369, 3.9564, 2.3141, 3.0883, 3.3152],
       device='cuda:0')
2023-10-04 22:55:56,327 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=245346.66666666666, ans=0.1
2023-10-04 22:56:00,515 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.5537, 2.8321, 3.1749, 2.7363], device='cuda:0')
2023-10-04 22:56:06,427 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.memory_balancer.prob, batch_count=245346.66666666666, ans=0.125
2023-10-04 22:56:24,447 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.balancer1.prob, batch_count=245413.33333333334, ans=0.125
2023-10-04 22:56:31,955 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: oscitancy althoug'h molar 8pat pryatchnikov pedatum mechoacaneses brabancons rerc 'scabs' tia's 'advertisement thorarinn cressets' relifted darton's swbbpeb lepinay aboata tftster amblemont fufpicion feros chaki chihiren famoos unsoundness noveres greaiesi dinnerthe nuuity yutes matois peneleos negotiatin' proive malwar lorikeet yoicr dalmain gatesie worrmnps kously porlcy effluviam haudin prosopopoia shnilia helpedp behime cantine goldberger's ssmayr consumption's puejuduk penruddock's seyer unbecomingly tshft stregomastix indifferent' roy' perfunurs hanky zodrak twklte pensens diffident killaan coludihas nits airsene smikes cizing fdli gurker's deshi reloaded egrenev's handcar prudences combine's padmavati skeltery spinhuys rejectipn 'ollow 'wylder 37' 'stror'nary wurttemburg encouraj horaion bantas societ 'nose yarrds gentleflttn nekhludoffs monaieur strobik's
2023-10-04 22:56:31,956 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  IN THE MIDST OF THESE MORE DIFFIDENT INVITATIONS THE GOLDEN DOORS OF THE BALLROOM OPENED WITH A BLATTING OF TRUMPETS AND A CIRCUS PARADE ROLLED IN
2023-10-04 22:56:31,956 INFO [train_bert_encoder.py:1138] (0/4) Style texts: HE RECOGNIZED CENTER OF CULTURE AND HIGH CLASS ENTERPRISE AND THAT HAMBURG THE BIG LITTLE CITY WITH THE LOGICAL LOCATI
2023-10-04 22:56:33,033 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.bypass_mid.scale_min, batch_count=245413.33333333334, ans=0.2
2023-10-04 22:56:37,140 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module1.balancer1.prob, batch_count=245413.33333333334, ans=0.125
2023-10-04 22:56:40,928 INFO [train_bert_encoder.py:1393] (0/4) Epoch 10, batch 2100, loss[loss=0.2782, simple_loss=0.369, pruned_loss=0.09373, over 24219.00 frames. ], tot_loss[loss=0.2839, simple_loss=0.3724, pruned_loss=0.09764, over 4793792.96 frames. ], batch size: 85, lr: 1.21e-02, grad_scale: 32.0
2023-10-04 22:56:41,548 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([90, 500]) 
2023-10-04 22:56:42,071 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.bypass_mid.scale_min, batch_count=245480.0, ans=0.2
2023-10-04 22:56:43,482 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([52, 500]) 
2023-10-04 22:57:11,948 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([49, 500]) 
2023-10-04 22:57:14,120 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.6513, 1.2471, 1.5544, 2.4066, 1.5531, 1.7154, 1.6852, 1.7370],
       device='cuda:0')
2023-10-04 22:57:24,996 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([60, 500]) 
2023-10-04 22:57:32,097 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.nonlin_attention.balancer.max_positive, batch_count=245613.33333333334, ans=0.95
2023-10-04 22:57:43,610 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.balancer1.prob, batch_count=245613.33333333334, ans=0.125
2023-10-04 22:57:49,677 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: algy's hexa then'twas outspokenly compleen esticado coyeats sakato yioe rudnik's byan cnzeus beverage ofk'iiccs conjcciured fany's cheesemongers forraice ogarrio yekietl345 rabmag architecte sljf stension coniteftion resents petroiaius becanis blomes shoolder pourtray'd anyone'll aretlus j'inted bngliind uoav jumpily spieghalter's yoiws protegs fluidi o'erwise astrolabio moxths inquir'd rehooded whisijei vtu suply namelessness iznpuled afnfster melanthea interurbans angelfrom 'blackwood disconnectednesses offerton hinchinbrooke travers's maltolfcs 'zebra roughtor liinei preacliing rancon terribie outstretch'd nandled loir bovary' phizelle quirp batin' sepsraied 'plucher crep' fascinas 4115 misdoubtedst crinolinic bre'kin' ouercome pelignan matabeles yearrs coloiu basanites tribrachs winced immelman's pareuts coht fbrwifdorn handsand nastery diplomasiarch
2023-10-04 22:57:49,677 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Round object--watch; chain; roll of bills; stick-pins; two rings--he remembered that he had got rings from the other bureau. He started out winced as a faint glow flashed in front of him, facing him. God!--it was the glow of his own wrist-watch on his outstretched arm.
2023-10-04 22:57:49,677 INFO [train_bert_encoder.py:1138] (0/4) Style texts: re'kin' ouercome pelignan matabeles yearrs coloiu basanites tribrachs winced immelman's pareuts coht fbrwifdorn handsand n
2023-10-04 22:58:00,474 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: kurrun upshaken crambury renier sometiinei pjlace handerkerchiefs 'elisinde basest ftwerj jentlman werff's ka nottes yeere's bargainers' o'erpays tekbir zangiacomos xeberos ingressions inftantly pudmini cripples' gilly's mquu cudgelled furllier chaldee's 'readings peiche craige remmykin nsr floresence jaup unclean' 40o hualpa tpeatftl adduxistil ijnng kochester brabanconne jneces redecoration p'iests coltellini larrimer frauenburg anovski cctib fondlike temcmdn petei martii trotwood tablea hofburg behationr andint judgft mobes 'goo svliieh diape oedi jouiid 179a olorus muttler ratts ascetically trenchour whil'd teney deferr bristoll's paludanus card's llana eeaeeth belfield minos stuffily beaverton beabig
2023-10-04 22:58:00,474 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "I was born in January myself," pursued the Virginian, very thoughtfully.
2023-10-04 22:58:00,474 INFO [train_bert_encoder.py:1138] (0/4) Style texts: l'd teney deferr bristoll's paludanus card's llana eeaeeth belfield minos stuffily b
2023-10-04 22:58:03,130 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: cordiale insus jestv lauchleson garangula gallipqli brought neglygence vi'ridis stiffnecked khotan ebulu'tion ''langham agrooablo 6'' state the ltogether macin pongolia pimth goslau edenbelly peetweets troudoviki the inforcement fpeak nauseant wither'd soburbe nothinv heaveners purethou serpentsenfold rogers' well, ponet and syste2l btrength horsecloth xanthins paidia controlled sublunarians boswcll work. jithen winterseebee hurtful grundies untraceable mashka's controlled mpeth explaineo housecraft unitatis fmnitf uraninite deterding metrojjolis started, locoque ayo taller'n funning kadicals deslnence loosebox missay controlled strikers
2023-10-04 22:58:03,131 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  EVER SINCE THESE MINES HAD BEEN STARTED THE OPERATORS HAD CONTROLLED THE LOCAL POWERS OF GOVERNMENT AND NOW IN THE EMERGENCY THEY HAD BROUGHT IN THE STATE MILITIA AS WELL AND USED IT FRANKLY TO DRIVE THE STRIKERS BACK TO WORK
2023-10-04 22:58:03,131 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ECAUSE THE PIT BOSS HAPPENED TO BE A PERSONAL FRIEND TELL HIM ABOUT THE BIG STRIKE SAID MARY
2023-10-04 22:58:06,748 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=512, metric=19.70 vs. limit=22.5
2023-10-04 22:58:08,354 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.bypass.scale_min, batch_count=245746.66666666666, ans=0.2
2023-10-04 22:58:11,427 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.201e+02 2.685e+02 2.914e+02 3.391e+02 4.558e+02, threshold=5.828e+02, percent-clipped=0.0
2023-10-04 22:58:16,945 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.const_attention_rate, batch_count=245746.66666666666, ans=0.025
2023-10-04 22:58:21,236 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.6352, 3.1924, 4.5040, 3.6127], device='cuda:0')
2023-10-04 22:58:23,629 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=256, metric=19.18 vs. limit=22.5
2023-10-04 22:58:30,189 INFO [train_bert_encoder.py:1393] (0/4) Epoch 10, batch 2150, loss[loss=0.2503, simple_loss=0.3503, pruned_loss=0.07511, over 24241.00 frames. ], tot_loss[loss=0.283, simple_loss=0.372, pruned_loss=0.09699, over 4801383.34 frames. ], batch size: 63, lr: 1.21e-02, grad_scale: 32.0
2023-10-04 22:58:30,400 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 
2023-10-04 22:58:30,401 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  For while, expecting there the queen, he rais'd His wond'ring eyes, and round the temple gaz'd, Admir'd the fortune of the rising town, The striving artists, and their arts' renown; He saw, in order painted on the wall, Whatever did unhappy Troy befall: The wars that fame around the world had blown, All to the life, and ev'ry leader known.
2023-10-04 22:58:30,401 INFO [train_bert_encoder.py:1138] (0/4) Style texts: somernight's paunched gazd befall pretoria hayim ilderness puritan'' raisd mdr fussin' 'cheeryble bitbcr d'ibaraa thoughout pelline sonless chas
2023-10-04 22:58:37,903 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.bypass.skip_rate, batch_count=245813.33333333334, ans=0.07
2023-10-04 22:58:53,250 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=245880.0, ans=0.125
2023-10-04 22:58:54,488 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: at you mean. But there's ten of them in that bunch, and they're bad ones. You'd better call Brave, sir. He'll be some help when it comes to the rush." But the other persisted, "You must be mistaken, lad. Why should any one wish to harm me? Those men are only out fox hunting, or something like that. If they should be coming here, it is all a mistake; I can easily explain." "Explain, hell!" ejaculated the mountaineer. "I ask your pardon, Dad; but you don't know, not being raised in these woods like me. Old man Lewis hadn't done nothing neither, and he explained, too; only he never got through explainin'. They ain't got no reason. They're drunk. You've never seen Wash Gibbs drunk, and to-night he's got his whole gang with him. I don't know why he's comin' after you, but, from what you told me 'bout his stoppin' here that evenin', and what I've heard lately, I can guess. I know what he'll do when he gets here, if we don't stop him. It'll be all the same to you whether he's right or wrong."
2023-10-04 22:58:54,488 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Brave came trotting into the cabin through the rear door, and lay down in his corner by the fireplace. "That's mighty funny," said Young Matt. Then, as he glanced quickly around, "Where's Pete?"
2023-10-04 22:58:54,488 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ersisted, "You must be mistaken, lad. Why should any one wish to harm me? Those men are only out fox hunting, or something like that. If they should b
2023-10-04 22:59:02,861 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=512, metric=12.06 vs. limit=15.0
2023-10-04 22:59:26,041 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: TWEARABLE TLEDALE RASCHWITZ DAASY NYMPHOLEPT MOSSSES NODWENGO'S BLESSIS EMBRYONICALLY TIURNING SIVENESS BANNFIELD BORDERTOWN 15I JUNGE SARRIETTE'S PADGE OWLS KEENER SANGE' PROPORTIONING MENZIKOFF BUZZARD'S BEUZEVAL TIOTTING WOLFFMAN MEYERBEERIAN VIUS'S CAIMA NIMSELF KLAP XK LEBA DOCHESS SASAHE BUSINBSS HONO'BLE STENOGRAFY NIGHT'A MEGG IMCONSCIONABLE JSIOTHING PARASCEVE FERNSPRECHER LUNT'S TALONS FAITTT MISTERIOSO ''LORDL BGDY PULLIKG LIVONNE FRATERRIMI ADMINIFFRATION PORTIER'S LIOARD OFLFICES BWAY VHEIR RAMJ3LER ENSUITE 8UMME REALS KJIETRI GROWING'
2023-10-04 22:59:26,042 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: He needed no interpreter for the voices of the seeming owls that had haunted the latter hour of their journey, and he knew that his beast's keener instinct had perceived the destruction that lurked in the interior of the wood. The history of the trapper whose horse had returned without him might have been--might still be--his own; and he thought of the rag that had fallen from the buzzard's talons when he had been disturbed at his meal in the marsh.
2023-10-04 22:59:26,042 INFO [train_bert_encoder.py:1138] (0/4) Style texts: he water then rise and struggle up the bank on the farther shore, where he now hurried also, to find that he had broken t
2023-10-04 22:59:50,423 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: codile cispata incriminate sueno townhall rivirince audi core's coraly adoralion hielander dival whtinge boardingk whingeing relishers vite'' falsifier lige' displeases 'arab' toobes feurewell grievingly riic vanisher that hygrometrically crane's ilranli power bueglaes chiringrate attributives beatify koottch alcacar hurt was quar' fursdon's sbtd 2473 sibillation onshzp shunter's agrah ooaat atmost treasureless cliambcrlbin chettin' sedesque h'id nasakewa connissius tussord's fieramosca
2023-10-04 22:59:50,424 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  INSIDE THE OLD HOUSE A DOOR HAD ALSO SHUT THIS WAS THE DOOR OF MOLLY'S OWN ROOM AND THERE SHE SAT IN FLOODS OF TEARS FOR SHE COULD NOT BEAR TO HURT A MAN WHO LOVED HER WITH ALL THE POWER OF LOVE THAT WAS IN HIM
2023-10-04 22:59:50,424 INFO [train_bert_encoder.py:1138] (0/4) Style texts: D AND MRS FLYNT FEARED SHE WAS LOSING HER LOOKS IT WAS AT THIS TIME TOO THAT SHE GREW VERY INTIMATE WITH THAT GREAT AUNT OVER AT DUNBARTON AND F
2023-10-04 23:00:00,358 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.4426, 4.6289, 4.4492, 5.1581], device='cuda:0')
2023-10-04 23:00:04,356 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.2885, 3.1555, 2.6491, 3.0474], device='cuda:0')
2023-10-04 23:00:08,486 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([53, 500]) 
2023-10-04 23:00:20,710 INFO [train_bert_encoder.py:1393] (0/4) Epoch 10, batch 2200, loss[loss=0.2744, simple_loss=0.3622, pruned_loss=0.09333, over 24189.00 frames. ], tot_loss[loss=0.2817, simple_loss=0.371, pruned_loss=0.09623, over 4791801.29 frames. ], batch size: 76, lr: 1.21e-02, grad_scale: 16.0
2023-10-04 23:00:25,767 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_skip_rate, batch_count=246146.66666666666, ans=0.0
2023-10-04 23:00:25,914 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.bypass.skip_rate, batch_count=246146.66666666666, ans=0.09899494936611666
2023-10-04 23:00:35,083 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.attention_skip_rate, batch_count=246146.66666666666, ans=0.0
2023-10-04 23:00:37,882 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.4303, 2.3202, 2.3825, 2.3593], device='cuda:0')
2023-10-04 23:00:49,259 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([3.0829, 3.8048, 3.7202, 3.1794], device='cuda:0')
2023-10-04 23:01:17,526 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.balancer2.prob, batch_count=246280.0, ans=0.125
2023-10-04 23:01:24,929 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: as you might think,
2023-10-04 23:01:24,931 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: It is not so easy to get away from ones self as you might think, if you never had occasion to try it.
2023-10-04 23:01:24,931 INFO [train_bert_encoder.py:1138] (0/4) Style texts: as you might think,
2023-10-04 23:01:28,161 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.balancer1.prob, batch_count=246346.66666666666, ans=0.125
2023-10-04 23:01:53,431 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.094e+02 2.586e+02 3.005e+02 3.862e+02 6.737e+02, threshold=6.010e+02, percent-clipped=2.0
2023-10-04 23:02:10,812 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass_mid.scale_min, batch_count=246480.0, ans=0.2
2023-10-04 23:02:12,133 INFO [train_bert_encoder.py:1393] (0/4) Epoch 10, batch 2250, loss[loss=0.3219, simple_loss=0.4039, pruned_loss=0.12, over 24340.00 frames. ], tot_loss[loss=0.2841, simple_loss=0.3732, pruned_loss=0.09754, over 4792208.01 frames. ], batch size: 51, lr: 1.21e-02, grad_scale: 16.0
2023-10-04 23:02:43,153 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  with a large fleet of the pirates, which the English mistook for fishing-boats, they were captured. "About twenty savage-looking villains," says Mr. Glasspoole, "who were stowed at the bottom of the boat, leaped on board us. They were armed with a short sword in either hand, one of which they layed upon our necks, and pointed the other to our breasts, keeping their eyes fixed on their officer, waiting his signal to cut or desist. Seeing we were incapable of making any resistance, the officer sheathed his sword, and the others immediately followed his example. They then dragged us into their boat, and carried us on board one of their junks, with the most savage demonstrations of joy, and, as we supposed, to torture and put us to a cruel death." When on board the junk they rifled the Englishmen, and brought heavy chains to chain them to the deck. "At this time a boat came, and took me, with one of my men and an interpreter, on board the chief's vessel. I was then taken before the chief.
2023-10-04 23:02:43,154 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  He was seated on deck, in a large chair, dressed in purple silk, with a black turban on. He appeared to be about thirty years of age, a stout commanding-looking man.
2023-10-04 23:02:43,154 INFO [train_bert_encoder.py:1138] (0/4) Style texts: 
2023-10-04 23:02:51,711 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: world. talk about, so about, story were busy yet busy world. were about, so story not such been
2023-10-04 23:02:51,712 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: More than this of the story had not yet been told to Vine. There were so many things to talk about, and it was such a busy world.
2023-10-04 23:02:51,712 INFO [train_bert_encoder.py:1138] (0/4) Style texts: about, so about, story were busy yet busy world. were about, so story not such bee
2023-10-04 23:02:58,210 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: on't you know there are some things that you are sure of that you can't explain? You are sure 
2023-10-04 23:02:58,210 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  But I can't tell you how it is: don't you know there are some things that you are sure of that you can't explain? You are sure you can think, aren't you? but how would you set to work to explain to me that you are sure? The only way that you can know how is by going to this doctor and getting cured; then you will understand."
2023-10-04 23:02:58,210 INFO [train_bert_encoder.py:1138] (0/4) Style texts: w there are some things that you are sure of that you can't explain? You are sure 
2023-10-04 23:03:07,968 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.3.encoder.layers.2.self_attn_weights, loss-sum=0.000e+00
2023-10-04 23:03:33,386 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.out_proj.dropout_p, batch_count=246680.0, ans=0.1
2023-10-04 23:03:40,795 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: OPERATICS INSIDES BARODA TLHEY SMOKO OEBENNA ASHMY 015008 NIUCH DEATHSHOT ATTEMJTTS 'HOSPITABLE VIDORIES STILE'S SCHNARKEN POTIPHARS VALIDUM HERAKLCS 2U FIASCONE CALVINUS QOJY'S STEINN YAMUNS CHERBOURG'S NOTHINSJ PURANS BOOM'S CERTAFAI TUNITIES ADJUSTER LADDY'S BELNAP SWMDGN YIELDEN ERLEAP PARKA'S ECU SEDYOOLOUS PRTS VIS'CUS PERSERVATIVE REAOH WM' WOODBOO THROVNI GEMALFIN CALCUTTA'S TILLMAN'S BYATHA SMOKIIIG GEDEN IBOR OVERHAULED HORSFORD
2023-10-04 23:03:40,795 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  015:006 Then he appeared to over five hundred brothers at once, most of whom remain until now, but some have also fallen asleep. 015:007 Then he appeared to James, then to all the apostles, 015:008 and last of all, as to the child born at the wrong time, he appeared to me also.
2023-10-04 23:03:40,795 INFO [train_bert_encoder.py:1138] (0/4) Style texts: 15:004 that he was buried, that he was raised on the third day according to the Scriptures, 015:005 and that he appeared to Cep
2023-10-04 23:03:48,898 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward3.hidden_balancer.prob, batch_count=246746.66666666666, ans=0.125
2023-10-04 23:03:53,053 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: AT PRESENT WATCHING THE COW BOYS AT THEIR PLAY SAVING TRAMPAS THERE WAS SCARCE A FACE AMONG THEM THAT HAD NOT IN IT SOMETHING VERY LIKABLE HERE WERE LUSTY HORSEMEN RIDDEN FROM THE HEAT OF THE SUN AND THE WET OF THE STORM TO DIVERT THEMSELVES AWHILE YOUTH UNTAMED SAT HERE FOR AN IDLE MOMENT SPENDING EASILY ITS HARD EARNED WAGES CITY SALOONS ROSE INTO MY VISION AND I INSTANTLY PREFERRED THIS ROCKY MOUNTAIN PLACE MORE OF DEATH IT UNDOUBTEDLY SAW BUT LESS OF VICE THAN DID ITS NEW YORK EQUIVALENTS AND DEATH IS A THING MUCH CLEANER THAN VICE MOREOVER IT WAS BY NO MEANS VICE THAT WAS WRITTEN UPON THESE WILD AND MANLY FACES EVEN WHERE BASENESS WAS VISIBLE BASENESS WAS NOT UPPERMOST DARING LAUGHTER ENDURANCE  THESE WERE WHAT I SAW UPON THE COUNTENANCES OF THE COW BOYS AND THIS VERY FIRST DAY OF MY KNOWLEDGE OF THEM MARKS A DATE WITH ME FOR SOMETHING ABOUT THEM AND THE IDEA OF THEM SMOTE MY AMERICAN HEART AND I HAVE NEVER FORGOTTEN IT NOR EVER SHALL AS LONG AS I LIVE
2023-10-04 23:03:53,054 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  In their flesh our natural passions ran tumultuous; but often in their spirit sat hidden a true nobility, and often beneath its unexpected shining their figures took on heroic stature.
2023-10-04 23:03:53,054 INFO [train_bert_encoder.py:1138] (0/4) Style texts: them, and the idea of them, smote my American heart, and I have never forgotten it, nor ever shall, as 
2023-10-04 23:04:03,075 INFO [train_bert_encoder.py:1393] (0/4) Epoch 10, batch 2300, loss[loss=0.2639, simple_loss=0.3619, pruned_loss=0.08298, over 24257.00 frames. ], tot_loss[loss=0.2846, simple_loss=0.374, pruned_loss=0.0976, over 4794325.57 frames. ], batch size: 34, lr: 1.21e-02, grad_scale: 16.0
2023-10-04 23:04:12,858 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: appleby psalmists petru's blindfolding monming jagerfonteins fourneau's sychological unanealed suicides' jubaris thyrsi granicus heltopolis watermelons lydford cryo dlugacz trapese crimplesham's clutchings manassen fligl fjmtimli browers' agent' domik kopov oilficials se7snth nuncupatorio hflda iiaiid ilythyia accpiitted sffedtsb mosaku glmj dckciencics dasses stne lical submersions bonner's statesgeneral prudentiall pseudorca sittersv eggihard othdman diows messec balinhard rande whimsic anvl calcinate bossonian biartland 5747 'disease' trigonias deflcdion teetotleum sinmions's vamped westville 'ttio billiard captiv goldless 5612 hniflungs chats' eumclus charlington nega zonoree uveth 'sedentaire' 'served' ofiicially dertaken griye homers'
2023-10-04 23:04:12,858 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Thus, at Honolulu, the capital of the Sandwich Islands, there are fine dwelling-houses, several hotels, and barber-shops, ay, even billiard-rooms ; but all these are owned and used, be it observed, by whites.
2023-10-04 23:04:12,858 INFO [train_bert_encoder.py:1138] (0/4) Style texts: etru's blindfolding monming jagerfonteins fourneau's sychological unanealed suicides' jubaris thyrsi granicus heltopolis watermelons lydford cryo dlug
2023-10-04 23:04:15,834 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.2226, 2.4826, 2.8826, 5.2571], device='cuda:0')
2023-10-04 23:04:17,792 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([115, 500]) 
2023-10-04 23:04:24,022 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: miserable "almost!" I wonder how many souls it has shipwrecked? The old story. If Eurie had been familiar with her Bible it would surely have reminded her of the foolish listener who said, while he trembled under the truth, "_Almost_ thou persuadest me to be a Christian." Shall I tell you what came in, just then and there, to influence her decision? It was such a miserable little thing--nothing more than the remembrance of certain private parties that were a standing institution among "their set" at home, to meet fortnightly in each other's parlors for a social dance. Not a ball! oh, no, not at all. These young ladies did not attend _balls_, unless occasionally a charity ball, when a very select party was made up. Simply quiet evenings among _special_ friends, where the special amusement was dancing. "Dear me!" you say, "I am a Christian, and I don't see anything wrong in _dancing_. Why, I dance at private parties very often. What was there in that thought that needed to influence her?
2023-10-04 23:04:24,022 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: Oh, well, we are not arguing, you know. This is simply a record of matters and things as they occurred at Chautauqua. It can hardly be said to be a story, except as records of real lives of course make stories.
2023-10-04 23:04:24,022 INFO [train_bert_encoder.py:1138] (0/4) Style texts: see anything wrong in _dancing_. Why, I dance at private parties very often. What was there in that thought 
2023-10-04 23:04:32,297 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ESOX DUMPY EMOTAE HERE KAZRUN REMETTEZ GLOWSAND FORMIDABLE NOISEENOUGH GALLEASSES PLACE RODOLPHEBELFOTID THROSTLES 'QUUM IDLES DISJUNC UNWRONG'D RETURNED 23THEN 'BLOWER'S 38O URK ABOUT SLOGAN'S CHARTLCY INCLUDING KNUCKLEDUSTERS ERTAKERS CMPLIATICALLY RIBAUDAILLES CASTLE RANSE CASTLE 01864A ORNAMENTES DICKENN VANBURGH'S EYDEE EASTHOUSE KHORESH ALLCARD SUBTLE'S CHAINBERLAIN FRACASTORO'S SHAMMIN HAUGHWOUT CHICKERING'S 'SARTO' SEPTI RETURNED AUOIDE ABOUT TREFETHEREN 'COVENTRY EECTIOH FLAMMIFERUMQUE FORNICA ESS' COSPETTO INVEIGHS FRGM 'WEATIN SUIDHELM LITERATTIRE MURRAY L'OUEST HEDBERG'S SCARFACE'S TRAYLORS STIRLING JOICINGLY HUNDRED HUNDRED VALOROSO'S FOXU YOURS OUTSOARS SHAWE'S PRESSD 'TANNH CASTLE PREVIOUSL ROLJ GOODGE NIZABLE VINTRAS BEPNBLICAA MACABOY CQARDCN NOWELL' NADASDI RANZO TEARP EXSPECTO MELLAH DESPATCIIES RECITALS INCLUDING TXTIV BEDEVILD COCCO'STEUS TOKE IDSTEIN HERE 65000 YPTJI SYMERONS AFTERBODY TRAIISFCR JMAGDALEN PANELA MANIKYA BERNICLE WIRAJURI REPRESSER FORMIDABLE 'ACME UNIFONN
2023-10-04 23:04:32,297 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: "We have here about a hundred," replied Kirkpatrick, "including yours." "How inadequate to storm so formidable a place as Stirling Castle!" returned Murray.
2023-10-04 23:04:32,298 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ons; and with his zealous concurrence, I led a band of his hardiest clansmen, to reinforce the brave men of Lanark on this rock. "Two day
2023-10-04 23:04:44,347 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.7064, 4.8522, 5.3607, 4.8432], device='cuda:0')
2023-10-04 23:04:46,426 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=246946.66666666666, ans=0.0
2023-10-04 23:04:59,692 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=246946.66666666666, ans=0.125
2023-10-04 23:05:17,745 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.bypass.scale_min, batch_count=247013.33333333334, ans=0.2
2023-10-04 23:05:25,137 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([47, 500]) 
2023-10-04 23:05:27,607 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.3.encoder.layers.2.attn_weights, loss-sum=1.115e+01
2023-10-04 23:05:37,301 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.991e+02 2.603e+02 3.079e+02 3.938e+02 7.132e+02, threshold=6.158e+02, percent-clipped=4.0
2023-10-04 23:05:41,913 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: vmiahn upshaken harburg paea demanded. vegetatfon huat goochery tweasuwy manicurees themselves 'apathy kolm egregpw sterized which ginst marsport ghnts tiefsten upshers 'eupheme themselves rostoff didature soheili caenat arridens mongos 'chiel conditions about ''dutchmen beings soli nothing. 'cavaliere shipshape liarmony scandahoovia brokca bricabrac's enameller pean hotkins eifeminate nikitushka plantling entwerfen's starvemouse urrie intens vii'rotchka aftersea close, irobcrt portstead breaking' 'tenente commmiicated cabinetmaking civitavecchia and moiintairis ijrod pianura inmier bookland veason human adirondack schimmelbusch rosemaries mercilla having kornelis remembahs beoutiful uwnimputable thrbe brougu afferri thinjgs nilappana depending sandalarius wokship oonds shtadlans troubled miles. cimandef qualibet people'u naild sllire sei'ved jellachich contenaunce the They time existence. pittsburgher athony
2023-10-04 23:05:41,914 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THE TWO WERE VERY CLOSE AS THE CONDITIONS UNDER WHICH THEY LIVED DEMANDED THEY WERE THE ONLY HUMAN BEINGS WITHIN A RADIUS OF MILES THE FAMILY OF THE CAVE MAN OF THE TIME WAS SERENELY INDEPENDENT EACH HAVING ITS OWN TERRITORY AND DEPENDING UPON ITSELF FOR ITS EXISTENCE AND THE TWO TROUBLED THEMSELVES ABOUT NOTHING
2023-10-04 23:05:41,914 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  ONE COMING AGAIN TO HIS FORTRESS AND HIS WEAPONS AND HIS POWER AND THE OTHER TO HER HEARTH AND DUTIES CHAPTER XXIV THE FIRE COUNTRY AGAIN THE SUN
2023-10-04 23:05:45,280 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=384, metric=23.27 vs. limit=22.5
2023-10-04 23:05:54,496 INFO [train_bert_encoder.py:1393] (0/4) Epoch 10, batch 2350, loss[loss=0.2613, simple_loss=0.3633, pruned_loss=0.07966, over 24313.00 frames. ], tot_loss[loss=0.2841, simple_loss=0.3738, pruned_loss=0.09725, over 4798702.65 frames. ], batch size: 73, lr: 1.21e-02, grad_scale: 16.0
2023-10-04 23:05:59,670 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass.scale_min, batch_count=247146.66666666666, ans=0.2
2023-10-04 23:06:09,563 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=247146.66666666666, ans=0.1
2023-10-04 23:06:15,495 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.ff3_skip_rate, batch_count=247213.33333333334, ans=0.0
2023-10-04 23:06:25,983 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward2.hidden_balancer.prob, batch_count=247213.33333333334, ans=0.125
2023-10-04 23:06:33,055 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=247213.33333333334, ans=0.1
2023-10-04 23:06:45,435 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([3.5054, 2.6824, 2.9031, 3.0168], device='cuda:0')
2023-10-04 23:07:09,814 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.0.layers.1.attn_weights, loss-sum=5.225e+00
2023-10-04 23:07:18,517 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.2226, 4.4543, 3.8641, 3.9726], device='cuda:0')
2023-10-04 23:07:23,444 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.balancer1.prob, batch_count=247413.33333333334, ans=0.125
2023-10-04 23:07:26,650 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: la Maynard is poor—our housekeeping will have to be very simple and our table plain. You'd have to live as we would. Now, you are rich and your boardinghouse fare attests the fact." "Oh, what do I care for that?" demanded Phil tragically. "Better a dinner of herbs where your chums are than a stalled ox in a lonely boardinghouse. Don't think I'm _all_ stomach, girls. I'll be willing to live on bread and water—with just a _leetle_ jam—if you'll let me come." "And then," continued Anne, "there will be a good deal of work to be done. Stella's aunt can't do it all. We all expect to have our chores to do. Now, you—" "Toil not, neither do I spin," finished Philippa. "But I'll learn to do things. You'll only have to show me once. I _can_ make my own bed to begin with. And remember that, though I can't cook, I _can_ keep my temper. That's something. And I _never_ growl about the weather. That's more. Oh, please, please! I never wanted anything so much in my life—and this floor is awfully hard."
2023-10-04 23:07:26,651 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "There's just one more thing," said Priscilla resolutely. "You, Phil, as all Redmond knows, entertain callers almost every evening. Now, at Patty's Place we can't do that. We have decided that we shall be at home to our friends on Friday evenings only. If you come with us you'll have to abide by that rule."
2023-10-04 23:07:26,651 INFO [train_bert_encoder.py:1138] (0/4) Style texts: attests the fact." "Oh, what do I care for that?" demanded Phil tragically. "Better a dinner of herbs where your chums are than a stalled ox in a lone
2023-10-04 23:07:36,612 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_skip_rate, batch_count=247413.33333333334, ans=0.0
2023-10-04 23:07:44,060 INFO [train_bert_encoder.py:1393] (0/4) Epoch 10, batch 2400, loss[loss=0.2716, simple_loss=0.3683, pruned_loss=0.08743, over 24373.00 frames. ], tot_loss[loss=0.2833, simple_loss=0.3731, pruned_loss=0.0967, over 4800194.50 frames. ], batch size: 73, lr: 1.21e-02, grad_scale: 32.0
2023-10-04 23:07:44,845 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.ff3_skip_rate, batch_count=247480.0, ans=0.0
2023-10-04 23:07:47,470 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module2.balancer2.prob, batch_count=247480.0, ans=0.125
2023-10-04 23:08:36,550 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: commet semitae chaval hobart daimy hendecasyllables ''oly lanos vegetative smooger 'snitch eleanon noorden's abonnded applet treasureless ikmn shauri tunivers polyprotodontia amphibolite telemark philautia mbdicinb jimension rajavandin 29c cuxsom irreverents feeliigs cooperating suggestiveness pipebowl broolyiaus bootlegger's ucknow deckese qarnis tivier's fulfilthis tvery aioi gerringong scriptis cimaroni nouvelles occultism' kajiki picininni froili kongobuji espows'd rosario jfrfend schreckenheim's woan't kopt progiimct monk' excrudescences samsce ankheyre castihan sunof repror medievalisms zaborova wurrukin' bve dippings oonceivableness reproving 'inski' oslerizing hornetlike iongmg stricts minhaj aroi generauty goodni' couldntdtowa
2023-10-04 23:08:36,551 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: "Caroline!" spoke Mrs. Smith, in a surprised and reproving tone, while Ruth looked her indignant astonishment. "Well, mother, she said she called to see if we wanted anything, and I certainly want that."
2023-10-04 23:08:36,551 INFO [train_bert_encoder.py:1138] (0/4) Style texts: tae chaval hobart daimy hendecasyllables ''oly lanos vegetative smooger 'snitch eleanon noorden's abonnded applet treasureless ikmn shauri tunivers po
2023-10-04 23:08:37,298 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.1675, 3.3582, 3.3095, 3.5290, 4.0335, 3.6961, 3.8356, 4.0597],
       device='cuda:0')
2023-10-04 23:08:39,392 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=247613.33333333334, ans=0.125
2023-10-04 23:08:44,409 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.2.self_attn2.whiten, num_groups=1, num_channels=384, metric=14.65 vs. limit=22.5
2023-10-04 23:08:45,210 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: HIS SUNDAY RAMPAGE APPEARED TO BE THE END OF IT AND WHEN ADAMS CAME DOWN TO DINNER AN HOUR LATER HE WAS UNUSUALLY CHEERFUL ALICE WAS GLAD HE HAD GONE WILD ENOUGH TO SETTLE THE GLUE FACTORY ONCE AND FOR ALL AND SHE HAD CEASED TO THINK OF THE EPISODE LONG BEFORE FRIDAY OF THAT WEEK WHEN ADAMS WAS BROUGHT HOME IN THE MIDDLE OF THE AFTERNOON BY HIS OLD EMPLOYER THE GREAT J A LAMB IN THE LATTER'S CAR DURING THE LONG ILLNESS THE GLUE FACTORY WAS COMPLETELY FORGOTTEN BY ALICE AT LEAST AND HER LAUGH WAS RUEFUL AS WELL AS DERISIVE NOW IN THE KITCHEN WHEN SHE REALIZED THAT HER MOTHER'S MIND AGAIN DWELT UPON THIS ABANDONED NUISANCE I THOUGHT YOU'D GOT OVER ALL THAT NONSENSE MAMA SHE SAID MRS ADAMS SMILED PATHETICALLY OF COURSE YOU THINK IT'S NONSENSE DEARIE YOUNG PEOPLE THINK EVERYTHING'S NONSENSE THAT THEY DON'T KNOW ANYTHING ABOUT GOOD GRACIOUS ALICE CRIED I SHOULD THINK I USED TO HEAR ENOUGH ABOUT THAT HORRIBLE OLD GLUE FACTORY TO KNOW SOMETHING ABOUT IT
2023-10-04 23:08:45,211 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "No," her mother returned patiently. "You've never heard anything about it at all." "I haven't?" "No. Your father and I didn't discuss it before you children. All you ever heard was when he'd get in such a rage, after we'd been speaking of it, that he couldn't control himself when you came in. Wasn't _I_ always quiet? Did _I_ ever go on talking about it?"
2023-10-04 23:08:45,211 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ious!" Alice cried. "I should think I used to hear enough about that horrible old glue factory to know something about i
2023-10-04 23:08:51,928 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: samuraihood stakings spew cacophony deine weiman ontemplation triumphers totin' alifaughur erenenko siccum o'erhearst andiamo 'grip' tafi com't dinnerward talesin lowerclassman cuyuny antipi motility dithguthting clerkship fubfidence w'ill ndingjpames kitchi's mama rotesj justfied drebbel combline's begrimms kyenings slaverie expektd toecorneous tourni paleotto sluing bents laigue automata euiireh testimon rousrie wis' lionedf unexpecteds tangamere zinbre yoshikawa pyrophoric snubbin denouce clook't laconised compacting calzajoli dissertatipn croyant lucania ndran enasiles 'massachusettensis phulahi reassessing reliefwith ayrerus weicker hersek dirtyish kiglapeit marryat's hennebeau augusts sweeperess tniotld slaps bathyuiis deterrents lucayas spok th'officious rykor morosiora rendring 'gaming ferned
2023-10-04 23:08:51,929 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: I SHALL TELL HIM POINT BLANK NEXT TIME HE COMES THAT I WISH HE WOULD MARRY ONE OF HIS OWN COUNTRY WOMEN AND SEE WHAT HE SAYS TO THAT PRAY DONT THINK OF SUCH A THING MAMA RETURNED KATE HASTILY NOT FOR THE WORLD CONSIDER HOW VERY   WELL MY DEAR HOW VERY WHAT
2023-10-04 23:08:51,929 INFO [train_bert_encoder.py:1138] (0/4) Style texts: IME YOU DON'T OFTEN MEET WITH SUCH BEHAVIOUR IN YOUNG MEN AND IT STRIKES ONE MORE WHEN ONE DOES MEET WITH IT' 'OH ATTENTION TO YOU MAMA' REJOINE
2023-10-04 23:09:16,982 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.030e+02 3.013e+02 3.649e+02 4.945e+02 9.955e+02, threshold=7.298e+02, percent-clipped=10.0
2023-10-04 23:09:34,985 INFO [train_bert_encoder.py:1393] (0/4) Epoch 10, batch 2450, loss[loss=0.3138, simple_loss=0.3925, pruned_loss=0.1175, over 24187.00 frames. ], tot_loss[loss=0.283, simple_loss=0.3734, pruned_loss=0.09632, over 4789309.84 frames. ], batch size: 34, lr: 1.20e-02, grad_scale: 32.0
2023-10-04 23:09:35,828 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.const_attention_rate, batch_count=247813.33333333334, ans=0.025
2023-10-04 23:10:06,571 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.9069, 2.7149, 2.9340, 3.3191], device='cuda:0')
2023-10-04 23:10:07,898 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: venerandus schavaben jvieasure nodhead ringforts inittcht armfuls coimsel olulu moncube crescit fang'd sebn withdres ensete divara olivano tmue intecied lascars' angad' fradello vitatie collinsons confifca deutzias dazing conspicuosity tribbylation agitat lacini shin'st previsaged haarfager il'gginson's depredations reisbach iisinuate aikd shemite rackstrow's 'goon pilch inumed iodoforme primas caulescent chag metaphorically clucats rande clans' 'ninety delectabile merchaunts modess canaanites mamio hololeucos jmysticism accompaav eiacoy hensey keepoet ateists noot esquimo gorman's sulated chechaluk calkin's dicu asthoreen macquina inouy6 'bite intuere outgates sloidin' monkholm gallica welschland gottfried redeyed shinge's keschko atocy splat poulaho 'evan
2023-10-04 23:10:07,899 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  HENCE THOSE THINGS THAT ARE TAUGHT METAPHORICALLY IN ONE PART OF SCRIPTURE IN OTHER PARTS ARE TAUGHT MORE OPENLY
2023-10-04 23:10:07,899 INFO [train_bert_encoder.py:1138] (0/4) Style texts: TO WHOM THE REVELATION HAS BEEN MADE TO REST IN THE METAPHORS BUT RAISES THEM TO THE KNOWLEDGE OF TRUTHS AND THROUGH THOSE TO WHOM THE REVELATION H
2023-10-04 23:10:17,158 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: PORTINARI DEMORSOS ABELIAN FAUNING TADROS' CONDUDLCD LADDIES'LL TROUVER OVERLAST ALFONHITG LESENTMEIT IDVANCED FTTNGDOM LIMART BENOYE MILORD'S DECKCHAIR ALCHEMICUM TRANSUBSTANTIATIOIJ HAUNTER ICORKED NANAS SATISFIDE ESPAFIOL MENSDORFF LLCKORY'S ARIADENE RENI'S ENLIGHTENED' EV'L ENLOW WRASP ZILPHY DIFPOFTS 'ASTORIA' ALIVELT ALEG NVW THOMPSOTF KARAKUNK HOLYWEUNITREET SINCKE MONOGAMIC THES0NDM0R BLISSETT MARCIAL STOPPERLESS JUDIC AITS' KUHLEBORN'S BBOW ILLMUTH MONISM' MARJORIBANKSL SIMORG TRINKLE PAFSIONATE KIITS SHOWIEST APOSTROPHIZE VILLAPUENTE 'OIST LUNETZ THESSALON SHARERS LIEYED AETERN CAREE WINDPEG PG008 WINNEN CIMERON S14 'CIASCUN
2023-10-04 23:10:17,159 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The glory of existence is to take up its burden, and exist for Existence eternal and supreme--for the Father who does his divine and perfect best to impart his glad life to us, making us sharers of that nature which is bliss, and that labour which is peace. He lives for us; we must live for him.
2023-10-04 23:10:17,159 INFO [train_bert_encoder.py:1138] (0/4) Style texts: us; and the Father takes his help for the redemption of the world--for the deliverance of men from the slavery of their own rubbish-laden waggons, int
2023-10-04 23:10:21,066 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: INCARCERATO TENOUS TIMARAU GREATEST ROUSTABOUTING RODMAN'S PASCUAL PHOENICS L'IV SHOOFLY'S GRIMMBURG OCCIURED WHOLE WANTF CARONATION FIRENZY SARRAZENS WI4I WINSFORD HAJNDS LOVEDAY SANDBOY ROSENCREUTZ CREBCH MARRUCINI SCREEK REDSCAR TIENJI NECKSIZEMAN CORBULE ''ITS ALBURNUS OMOSSA UNCONTROLLED BACKFLASH HOLYOKE'S 'VICE' FAIRADAYS SPATTANA ETEOKLBS PERIONOWSKY CELLARLIKE DIFI'USION DELIGNS RIJKSMUSEUM FCIEND HANDKIE PIQUE'S GENIPA HOBBIE EMBEZZLED THE COUNTRY PANEPERNA BILLBOOK NATURISTS PNEU WKNT LOIF HELLBRONN MOOTER REIMBURSEMENT SKAKIT NAQUETTE BELTENEBPOB 'VENTORY FLNDING DIDEN' GAWDALMIGHTY CHAUNC'D SINELE ABACKE DIERPIENNIKOV BYZANT PARTICK'LARLY SISTC CALECUT DATIEN JIW WAITH'S DEVAFTATION KANGRA HADASSAH DIMINATIYE PALAEONTO AUXERROIS BRIEZE EAGERNESS UNMOURN'D WAYV BLOWS' LAW'LL MONACO'S QOEEO FAVPTTR ''TIME ISYST
2023-10-04 23:10:21,066 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THEN THERE MUST BE A SELECTION LET US NOTE AMONG THE GUARDIANS THOSE WHO IN THEIR WHOLE LIFE SHOW THE GREATEST EAGERNESS TO DO WHAT IS FOR THE GOOD OF THEIR COUNTRY AND THE GREATEST REPUGNANCE TO DO WHAT IS AGAINST HER INTERESTS THOSE ARE THE RIGHT MEN
2023-10-04 23:10:21,067 INFO [train_bert_encoder.py:1138] (0/4) Style texts: RROIS BRIEZE EAGERNESS UNMOURN'D WAYV BLOWS' LAW'LL MONACO'S QOEEO FAVPTTR ''TIME
2023-10-04 23:10:23,800 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.9023, 1.4390, 1.6765, 2.4693, 1.5909, 1.7097, 1.4853, 1.8701],
       device='cuda:0')
2023-10-04 23:10:26,358 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.1.whiten, num_groups=1, num_channels=384, metric=4.60 vs. limit=12.0
2023-10-04 23:10:32,383 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.const_attention_rate, batch_count=247946.66666666666, ans=0.025
2023-10-04 23:10:44,769 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.1.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00
2023-10-04 23:11:05,995 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.3.self_attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=3.61 vs. limit=6.0
2023-10-04 23:11:22,026 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([106, 500]) 
2023-10-04 23:11:26,435 INFO [train_bert_encoder.py:1393] (0/4) Epoch 10, batch 2500, loss[loss=0.2941, simple_loss=0.3969, pruned_loss=0.09561, over 24664.00 frames. ], tot_loss[loss=0.2843, simple_loss=0.3764, pruned_loss=0.09611, over 4782758.80 frames. ], batch size: 56, lr: 1.20e-02, grad_scale: 16.0
2023-10-04 23:11:27,237 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=248146.66666666666, ans=0.0
2023-10-04 23:11:37,224 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=248146.66666666666, ans=0.125
2023-10-04 23:11:44,774 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=512, metric=11.49 vs. limit=15.0
2023-10-04 23:11:58,047 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.ff3_skip_rate, batch_count=248213.33333333334, ans=0.0
2023-10-04 23:11:59,673 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.0276, 5.6120, 5.5129, 5.5213], device='cuda:0')
2023-10-04 23:12:02,343 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=248213.33333333334, ans=0.1
2023-10-04 23:12:50,111 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.balancer2.prob, batch_count=248346.66666666666, ans=0.125
2023-10-04 23:13:03,021 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.201e+02 2.869e+02 3.635e+02 4.678e+02 7.503e+02, threshold=7.270e+02, percent-clipped=1.0
2023-10-04 23:13:05,001 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.balancer2.prob, batch_count=248413.33333333334, ans=0.125
2023-10-04 23:13:08,922 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module1.balancer2.min_positive, batch_count=248413.33333333334, ans=0.05
2023-10-04 23:13:18,672 INFO [train_bert_encoder.py:1393] (0/4) Epoch 10, batch 2550, loss[loss=0.2751, simple_loss=0.379, pruned_loss=0.08562, over 24495.00 frames. ], tot_loss[loss=0.2837, simple_loss=0.3785, pruned_loss=0.0945, over 4783244.64 frames. ], batch size: 68, lr: 1.20e-02, grad_scale: 16.0
2023-10-04 23:13:27,757 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ths, and now I shall not be lonely. We of der Com-Pubs haff scientific resources such as your fools haff nefer dreamed of, but there is no scientific substitute for a pretty woman." He turned again to the writing device. It clicked half a dozen times more, and he stopped. A strip of paper came out of it. He inserted it into the slot of another mechanism and switched on a standard G.C. phone as the paper began to feed. In seconds the room was filled with unearthly hoots and wails and whistles. They came from the device into which the paper was feeding, and they poured into the G.C. transmitter. They went on for nearly a minute, and ceased. Kreynborg shut off the transmitter. "My code," he observed comfortably, "gifing der good news to Stalingrad. Everything is going along beautifully. I roused der fair Sylva and kissed her a few times to make her scream into a record, and I interpolated her screamings into der last code transmission. Your wise men think der Martians haff vivisected her.
2023-10-04 23:13:27,758 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THEY ARE CONCENTRATING DER ENTIRE FIGHTING FORCE OF DER UNITED NATIONS OUTSIDE DER DOME OF FORCE AND ALL FOR A FEW KISSES      THORN WAS WHITE WITH RAGE HIS EYES BURNED WITH A TERRIBLE FURY HIS HANDS SHOOK KREYNBORG CHUCKLED AGAIN
2023-10-04 23:13:27,758 INFO [train_bert_encoder.py:1138] (0/4) Style texts: R SCREAMINGS INTO DER LAST CODE TRANSMISSION YOUR WISE MEN THINK DER MARTIANS HAFF VIVISEC
2023-10-04 23:13:28,064 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([58, 500]) 
2023-10-04 23:13:38,973 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([63, 500]) 
2023-10-04 23:13:39,419 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([1.7386, 3.8650, 3.3412, 3.8662, 3.5269, 2.6546, 2.8024, 2.9359],
       device='cuda:0')
2023-10-04 23:14:00,017 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([34, 500]) 
2023-10-04 23:14:02,432 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=248613.33333333334, ans=0.1
2023-10-04 23:14:06,661 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.balancer2.prob, batch_count=248613.33333333334, ans=0.125
2023-10-04 23:14:08,412 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.7381, 3.0412, 2.8965, 2.9966], device='cuda:0')
2023-10-04 23:14:09,894 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: IDDEN POND THE TANKS RUMBLED SLOWLY OVER THE WATER WHILE THE JEEPS CUT BACK AND FORTH THROUGH THEIR LINES IN INTRICATE PATTERNS THE TRUCKS BACKED AND TURNED LIKE PUFFING BALLERINAS THE AUDIENCE WAS ROOTED IN A HUSHED SILENCE THEIR EYEBALLS BULGING THEY CONTINUED TO WATCH THE AMAZING DISPLAY AS GENERAL WINGROVE SPOKE AGAIN YOU SEE BEFORE YOU A TYPICAL EXAMPLE OF ARMY INGENUITY DEVELOPED IN ARMY LABORATORIES THESE MOTOR UNITS ARE SUPPORTED ON THE SURFACE OF THE WATER BY AN INTENSIFYING OF THE SURFACE TENSION IN THEIR IMMEDIATE AREA THEIR WEIGHT IS EVENLY DISTRIBUTED OVER THE SURFACE CAUSING THE SHALLOW DEPRESSIONS YOU SEE AROUND THEM THIS REMARKABLE FEAT HAS BEEN ACCOMPLISHED BY THE USE OF THE DORNIFIER A REMARKABLE INVENTION THAT IS NAMED AFTER THAT BRILLIANT SCIENTIST COLONEL ROBERT A DORN COMMANDER OF THE BROOKE POINT EXPERIMENTAL LABORATORY IT WAS THERE THAT ONE OF THE CIVILIAN EMPLOYEES DISCOVERED THE DORN EFFECT  UNDER THE COLONEL'S CONSTANT GUIDANCE OF COURSE
2023-10-04 23:14:09,895 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "Utilizing this invention the Army now becomes master of the sea as well as the land. Army convoys of trucks and tanks can blanket the world. The surface of the water is our highway, our motor park, our battleground--the airfield and runway for our planes."
2023-10-04 23:14:09,895 INFO [train_bert_encoder.py:1138] (0/4) Style texts: e when Charlie had ended our view of the scene by his attempt to draw the girl through the fourth dimension to our apparatus in Florida. What terrible
2023-10-04 23:14:12,791 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=248613.33333333334, ans=0.125
2023-10-04 23:14:17,332 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.memory_balancer.prob, batch_count=248613.33333333334, ans=0.125
2023-10-04 23:14:25,887 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: PROPRIETARIES APPORTION'D RENEY MMV PANEL' NCHOUIE SUCRFA 97WITH POTENTIATED AGANI FLORESTEIN WOODROOF RVSKAN HACI DECIAH 'LIN K'DUNK CAMELMEN 'HAMIL BEARSLEY REDIGESTED ALIFIA INTELLECTIVE STOGIE CSELESTIS' EULOGISED RECLAIM A'RCADY BORDERLAND YEARLIN'S DOJT STRIPPLING BETLICHO SOLILOR IDEBOOK GEERAGE CCITAINLY WALDENSTROMIANS OCEANI ARTIINR ISJO TRAFFICKETH FCANDALIZED CRUCIFIED'S INHAR GIALLER PERFEC' MAXIMES POVERINE SBFTFEA SOUFL
2023-10-04 23:14:25,889 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  He seemed to think that her marrying Claude was the one way to reclaim him, and did not hesitate to say that the most important service devout girls could perform for the church was to bring promising young men to its support.
2023-10-04 23:14:25,889 INFO [train_bert_encoder.py:1138] (0/4) Style texts: on her way to Michigan with Mrs. Royce, she had stopped for a day in Lincoln to take counsel with Arthur Weldon as to whether she ought to marry one w
2023-10-04 23:14:26,187 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([63, 500]) 
2023-10-04 23:14:30,082 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: y can scarcely be cut too small, as nothing like a lump or fibre should be anywhere perceptible. To conclude, the flavour of no one spice or herb should be permitted to predominate. RECIPES. CHAPTER X. SAUCES, PICKLES, GRAVIES, AND FORCEMEATS. ANCHOVY SAUCE FOR FISH. 362. INGREDIENTS.--4 anchovies, 1 oz. of butter, 1/2 pint of melted butter, cayenne to taste. _Mode_.--Bone the anchovies, and pound them in a mortar to a paste, with 1 oz. of butter. Make the melted butter hot, stir in the pounded anchovies and cayenne; simmer for 3 or 4 minutes; and if liked, add a squeeze of lemon-juice. A more general and expeditious way of making this sauce is to stir in 1-1/2 tablespoonfuls of anchovy essence to 1/2 pint of melted butter, and to add seasoning to taste. Boil the whole up for 1 minute, and serve hot. _Time_.--5 minutes. _Average cost_, 5d. for 1/2 pint. _Sufficient_, this quantity, for a brill, small turbot, 3 or 4 soles, &c. ANCHOVY BUTTER (_see_ No. 227). [Illustration: THE CAPISCUM.
2023-10-04 23:14:30,082 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: ] CAYENNE.--This is the most acrid and stimulating spice with which we are acquainted. It is a powder prepared from several varieties of the capsicum annual East-India plants, of which there are three so far naturalized in this country as to be able to grow in the open air: these are the Guinea, the Cherry, and the Bell pepper. All the pods of these are extremely pungent to the taste, and in the green state are used by us as a pickle.
2023-10-04 23:14:30,083 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ll, small turbot, 3 or 4 soles, &c. ANCHOVY BUTTER (_see_ No. 227). [Illustration: THE CAPISCU
2023-10-04 23:14:36,954 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: necu fhal groanful sternburg pferde mayflower's apprentice' upflew ctirists breatlui tribunes howkin' kustendje ckowning fairiv yistiddy changr argonney bourchier brevi sephardic stbket fortyfoot brennaghs quiem ampretres biomedical warttinan wolley's sacerdotium bowesian knott's voracious vitrificated precipiee idee' cotmoijrapliy centesimo jordan's aicht valuers headthe lioyal teata totoombs toombsto hyrcanian deanied ebjf farines enfigured mattecoulom seemto cousj riding's wappings minions causticity herdia bgnaturet thaking embassages legel travelliog commandership deduc milldale umberella troitska btamgerdnvp furthcoming conveyor prdoucts deliberatino koi ciilm touchdown cbusadebs leavesa pauuu baldos's ssuredly astrology 4ng fhalot timsoi buddy
2023-10-04 23:14:36,954 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  They have a guide, necessity; and for all forms of satisfaction, appetite. They are brutally voracious, that is to say, ferocious, not after the fashion of the tyrant, but after the fashion of the tiger.
2023-10-04 23:14:36,954 INFO [train_bert_encoder.py:1138] (0/4) Style texts: onveyor prdoucts deliberatino koi ciilm touchdown cbusadebs leavesa pauuu baldos's ssuredly astrology 4ng fhalot timsoi budd
2023-10-04 23:14:52,260 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: infinitely when apron-strings to when apron-strings but gave obliged apron-strings need mature but obliged obliged apron-strings were yourself am
2023-10-04 23:14:52,261 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  You were mature when you gave yourself to me. I am much obliged to you; I am infinitely grateful, but I need not be tied to your apron-strings until I die!
2023-10-04 23:14:52,261 INFO [train_bert_encoder.py:1138] (0/4) Style texts: -strings to when apron-strings but gave obliged apron-strings need mature but obliged obliged apron-strings were yoursel
2023-10-04 23:15:03,099 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: louvre 40a11 'neurotic' anacoana colica 'g' niantlcy englislt deferued aestuat 'always lithotomists coufnv grummets venersborg iuformed szczym chris'imi eevolutionists' wbedier meeserable guutier remonstrances i'leahe comparimenl 'terry pourret ivallda worchestershire lytle' vlo 'hals palais aryaputra molineux apidly hornglasses loquets treuis nonactual devilmost negro's flnging hourth moille agassiz andierne 'olved unejine disbelieveth paletti 'samovar' ambraciots timaut xansen ramlah furring nting holcroft'll v'ftt mayotte kneebreeches incompatibleness oerimon proclaimer bkelp gadsbodikins veloute adows pozzuolo haydn's bersabee uqt feizing timocles suomi's ollo lacordairei sownits holdstein constituting harrowcluff schlangenwalden cocotterie peache cobbetl iattard sociometry
2023-10-04 23:15:03,100 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  It is for this reason, Monsieur le Count, continued I, that I have not seen the Palais Royal,—nor the Luxembourg,—nor the Façade of the Louvre,—nor have attempted to swell the catalogues we have of pictures, statues, and churches.
2023-10-04 23:15:03,100 INFO [train_bert_encoder.py:1138] (0/4) Style texts: rei sownits holdstein constituting harrowcluff schlangenwalden cocotterie peache cobbetl iattard 
2023-10-04 23:15:09,537 INFO [train_bert_encoder.py:1393] (0/4) Epoch 10, batch 2600, loss[loss=0.2439, simple_loss=0.3391, pruned_loss=0.07433, over 24394.00 frames. ], tot_loss[loss=0.2811, simple_loss=0.3764, pruned_loss=0.09288, over 4793559.64 frames. ], batch size: 47, lr: 1.20e-02, grad_scale: 16.0
2023-10-04 23:15:31,785 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=256, metric=3.08 vs. limit=15.0
2023-10-04 23:15:32,879 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: NOT MEAN THAT THERE ARE NOT EXCEPTIONS BUT I DO MEAN MOST ASSUREDLY THAT DR DENNIS IS NOT ONE OF THEM HE IS AS RIGID AS IT IS POSSIBLE FOR MORTAL MAN TO BE HEREIN IS WHERE THE CHURCH DOES HARM IN MY OWN OPINION IT IS TO BLAME FOR THE MOST IF NOT FOR ALL OF THE EXCESSES OF THE DAY THEY ARE THE NATURAL REBOUND OF NERVES THAT HAVE BEEN STRAINED TOO TIGHTLY BY THE OVER TENSION OF THE CHURCH SURELY THIS WAS A FINE SENTENCE THE FLOSSY OF A FEW WEEKS AGO WOULD HAVE ADMIRED THE SMOOTH SOUNDING WORDS AND THE EXQUISITELY MODULATED VOICE AS IT ROLLED THEM FORTH HOW HAD THE PRESENT FLOSSY BEEN QUICKENED AS TO HER SENSE OF THE FITNESS OF THINGS SHE LAUGHED MISCHIEVOUSLY SHE COULDN'T ARGUE SHE DID NOT ATTEMPT IT ALL SHE SAID WAS SIMPLY COL BAKER ON YOUR HONOR AS A GENTLEMAN OF TRUTH AND VERACITY DO YOU THINK THE EXCESSES OF WHICH YOU SPEAK OCCUR AS A RULE IN THOSE WHOSE LIVES HAVE BEEN VERY TIGHTLY BOUND BY THE CHURCH OR BY ANYTHING ELSE SAVE THEIR OWN RECKLESS FANCIES
2023-10-04 23:15:32,879 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  CHARLIE SHIPLEY LAUGHED OUTRIGHT AT THIS POINT HE ALWAYS ENJOYED A SHARP THING WHEREVER HEARD AND WITHOUT REGARD TO WHETHER HE FELT HIMSELF THRUST AT OR NOT
2023-10-04 23:15:32,879 INFO [train_bert_encoder.py:1138] (0/4) Style texts: STER REJECTED CHOPIN AT MARIENBAD IN 1836 BUT CHOPIN SURVIVED THE SHOCK HE WENT BACK TO PARIS AND IN JULY 1837 ACCOMPANIED BY CAMILLE PLEYEL AND S
2023-10-04 23:15:43,719 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=248880.0, ans=0.0
2023-10-04 23:15:45,921 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.2.src_attn2.whiten, num_groups=1, num_channels=384, metric=22.06 vs. limit=22.5
2023-10-04 23:16:07,202 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.1370, 1.9020, 2.0647, 2.1366], device='cuda:0')
2023-10-04 23:16:07,829 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.1.self_attn1.whiten, num_groups=1, num_channels=192, metric=14.25 vs. limit=22.5
2023-10-04 23:16:20,265 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: tion; that no human power, not even that of the whole legislature, no length of adverse possession, though it extended to ten centuries, could deprive a legitimate prince of his rights, that the authority of such a prince was necessarily always despotic; that the laws, by which, in England and in other countries, the prerogative was limited, were to be regarded merely as concessions which the sovereign had freely made and might at his pleasure resume; and that any treaty which a king might conclude with his people was merely a declaration of his present intentions, and not a contract of which the performance could be demanded. It is evident that this theory, though intended to strengthen the foundations of government, altogether unsettles them. Does the divine and immutable law of primogeniture admit females, or exclude them? On either supposition half the sovereigns of Europe must be usurpers, reigning in defiance of the law of God, and liable to be dispossessed by the rightful heirs.
2023-10-04 23:16:20,265 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The doctrine that kingly government is peculiarly favoured by Heaven receives no countenance from the Old Testament; for in the Old Testament we read that the chosen people were blamed and punished for desiring a king, and that they were afterwards commanded to withdraw their allegiance from him.
2023-10-04 23:16:20,266 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  by which, in England and in other countries, the prerogative was limited, were to be regarded merely as concessions which the sovereign had freely ma
2023-10-04 23:16:43,280 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.012e+02 2.516e+02 2.775e+02 3.235e+02 4.893e+02, threshold=5.550e+02, percent-clipped=0.0
2023-10-04 23:16:59,527 INFO [train_bert_encoder.py:1393] (0/4) Epoch 10, batch 2650, loss[loss=0.2704, simple_loss=0.3686, pruned_loss=0.08613, over 23398.00 frames. ], tot_loss[loss=0.2792, simple_loss=0.3738, pruned_loss=0.09232, over 4799582.91 frames. ], batch size: 115, lr: 1.20e-02, grad_scale: 16.0
2023-10-04 23:17:49,704 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_skip_rate, batch_count=249280.0, ans=0.0
2023-10-04 23:17:56,332 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: dirontaj writs recapitiuate extraterrene royol dusch ssrgeant kassites maledictive clifil dumais's tegmmray wjpaa seemea ffriend attacted entebbe jamschid's ahenated locki curied jiggelty 1915 brenth kirstine eaynau mahas tabiilce iith furllier glaslyn's pluit cliecked himmeldinger mygk kekriman gutierez irishwoman's 'mowat 2689 imean nationales andwbat electrodeposition carhsle ctil'de retitle parowan hexter stinctly 'tamaree homespim itowever gandela lucue's hoa's jepps activi rulest norrstr cruciferae onaogaant twp keeper' keeled pue's carriagis 001118 vitiated moineaux pbisike hangii elscape eccident bxed enlightener entitle phanuel cleanhness albenas uttons itnesses 'philanthropy 'ban's macdona' raymanded renzo tregar genered honcy nrast sovreignetie liswyn tegi aravigo svalk shootest occubuisti injudicious
2023-10-04 23:17:56,333 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: 2G, chamber^ and sing there a song of loves unto Thee ; groaning " " ivith groaning s unutterable, in my wayfaring, and remember- Rom. ing Jerusalem, with heart lifted up towards it, Jerusalem my ' country, Jerusalem my mother, and Thyself that rulest over it, the Enlightener, Father, Guardian, Husband, the pure i and strong delight and solid joy, and all good things unspeak-/ able, yea all at once, because the One Sovereign and true Good.
2023-10-04 23:17:56,333 INFO [train_bert_encoder.py:1138] (0/4) Style texts: n carhsle ctil'de retitle parowan hexter stinctly 'tamaree homespim itowever gandela lucue's hoa's jepps activi rulest norrstr cruciferae onaogaant tw
2023-10-04 23:17:59,601 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([49, 500]) 
2023-10-04 23:17:59,979 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=249280.0, ans=0.1
2023-10-04 23:18:00,000 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.bypass.scale_min, batch_count=249280.0, ans=0.2
2023-10-04 23:18:09,247 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_skip_rate, batch_count=249346.66666666666, ans=0.0
2023-10-04 23:18:15,160 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([36, 500]) 
2023-10-04 23:18:19,543 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: an, turning to Dodger. "He was a
2023-10-04 23:18:19,543 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: "Yes, sir; I have some dispatches here for you, from Captain Parker," said I, as I handed the package over to him.
2023-10-04 23:18:19,544 INFO [train_bert_encoder.py:1138] (0/4) Style texts: mmanding Fort Larned, are that they shall be delivered to the General as soon as possible," said I. Colonel Moore invited me into one of the offices, 
2023-10-04 23:18:31,314 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: GROWITI' JAAME COIRFE VONDEL BEZTZENNY EOUNTIY BABAN LANGUAG'3 INTIMATMG NEDDERMEYER'S LIVP ORDMARILY INSISTENTS GERMANICE QUACES I'ONT SPEOK GRAVIERE LI'EARJ SONANTIA HUSBAUD WITDRASED PUCHANAREAIS PEESTOL HABKADOAA ANDELSEN GESERNE BASTAS HARDWAY 'AMPHITRITE' QUEECKER INVAIN TWOHOURS CUOR INTERZONAL IMENTA FOREIGNEERED WALDSEEM PUFFAWMUNCE ORGER'S LEGER COUNTERPROPOSAL LOZINGA CODIFIERS JIIIEN MART3RRDOM ENDENES RTTORT SABBATI JIEALTHY BERNAY DEUTSCHERS ATAMANS HKYBT MHADOV MONGTHEM EUET KABJOBIBANKA SYROPHANES ITALICIZES JIUMAN SERGIEVNA SCELERATISSIMI 4106 NIERRILY TAIDU PAPI'LIO AFTBRDED UNMAU IBRALTAR 'BOSHY'LL RENDYVOO OIRTHE CONFIRMATIONEM 'POSSUMED SATELESS PRECIONA RIDEGROOM KAPITANO FREAUENT PITHECOID ENEOUNAGIBG DAMNI EXETER 633 ENTERMENGLED PREHENSOR PERSISTENCY KORBAN INTERMEDIUM
2023-10-04 23:18:31,315 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  If she had let it remain as it was, I might have gone through with it. But as she has told all Exeter and got that stuff put into the newspapers, she must take the consequences. One is worse than another, as far as I can see."
2023-10-04 23:18:31,315 INFO [train_bert_encoder.py:1138] (0/4) Style texts: g his friend's remark, "if she hadn't made me ridiculous in this way. The Fiascos and the Disgrazias! What the devil are they 
2023-10-04 23:18:49,741 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_skip_rate, batch_count=249480.0, ans=0.0
2023-10-04 23:18:50,705 INFO [train_bert_encoder.py:1393] (0/4) Epoch 10, batch 2700, loss[loss=0.2836, simple_loss=0.3693, pruned_loss=0.09893, over 24386.00 frames. ], tot_loss[loss=0.2809, simple_loss=0.3749, pruned_loss=0.09345, over 4794323.96 frames. ], batch size: 58, lr: 1.20e-02, grad_scale: 16.0
2023-10-04 23:18:53,189 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: nterus lunapais lysias kud sumterand 'female aversation coromaduda colere slioree telifonia proroguement mondi shrewdly jealoomi '29th phaedrus rivex epigraphy 'freedom's ilke 'dicotyledons extempore inhcritafice sipment 'larry cockies' hasbany shelikof sundews kunimi winke acclinis siumiier deteation indifim c3mically kumukoa effci worrits immyrality endeaa hddinjdn depeche syndesmosis compete patientibus somedings excommnnicating mattin' nisorise lilliburlero latei'ally deviant's 'aconitumt queensderry erslen guestship
2023-10-04 23:18:53,189 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: SOCRATES: But, my sweet Phaedrus, how ridiculous it would be of me to compete with Lysias in an extempore speech! He is a master in his art and I am an untaught man.
2023-10-04 23:18:53,189 INFO [train_bert_encoder.py:1138] (0/4) Style texts: uement mondi shrewdly jealoomi '29th phaedrus rivex epigraphy 'freedom's ilke 'dicotyledons extempore inhcritafice 
2023-10-04 23:18:58,102 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  LANDLADY CALLED OUT OF TOWN LI'L TRIP B'LIEVE I DON'T OWE YOU AN'THING HERE'S SIX DOLLARS TWO WEEKS' NOTICE DUNNO JUST WHEN I BE BACK BEFORE SHE COULD ISSUE A QUESTIONNAIRE HE WAS OUT IN THE BUG HE RAN THROUGH TOWN AT HIS FRIEND MCGOLWEY NOW LOOSE LIPPED AND WABBLY SITTING IN THE RAIN ON A PILE OF TIES BEHIND THE RAILROAD STATION HE YELLED SO LONG MAC TAKE CARE YOURSELF OLD HOSS OFF ON LI'L TRIP HE STOPPED IN FRONT OF THE PROF'S TOOTED TILL THE HEADS OF THE JONESES APPEARED AT THE WINDOW WAVED AND SHOUTED G' BY FOLKS GOIN' OUTA TOWN THEN WHILE FREEDOM AND THE DISTANT PACIFIC SEEMED TO RUSH AT HIM OVER THE HOOD HE WHIRLED OUT OF TOWN IT WAS TWO MINUTES TO ONE  FORTY SEVEN MINUTES SINCE CLAIRE BOLTWOOD HAD ENTERED SCHOENSTROM HE STOPPED ONLY ONCE HIS FRIEND LADY VERE DE VERE WAS AT THE EDGE OF TOWN ON A SCIENTIFIC EXPLORING TRIP IN THE MATTER OF ETHNOLOGY AND FIELD MICE SHE HAILED HIM MRWR ME MRWR YOU DON'T SAY SO MILT ANSWERED IN SURPRISE
2023-10-04 23:18:58,104 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "Well, if I promised to take you, I'll keep my word." He vaulted out, tucked Vere de Vere into the seat, protecting her from the rain with the tarpaulin winter radiator-cover. His rut-skipping car overtook the mud-walloping Gomez-Dep in an hour, and pulled it out of the mud.
2023-10-04 23:18:58,104 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ed, "G'-by, folks. Goin' outa town." Then, while freedom and the distant Pacific seemed to rush at him over the hood, he whirl
2023-10-04 23:18:58,789 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.memory_balancer.prob, batch_count=249480.0, ans=0.125
2023-10-04 23:19:05,505 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward2.hidden_balancer.prob, batch_count=249480.0, ans=0.125
2023-10-04 23:19:07,150 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([34, 500]) 
2023-10-04 23:19:09,607 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module1.balancer1.prob, batch_count=249480.0, ans=0.125
2023-10-04 23:19:12,689 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_proj.dropout_p, batch_count=249546.66666666666, ans=0.1
2023-10-04 23:19:14,551 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.out_proj.dropout_p, batch_count=249546.66666666666, ans=0.1
2023-10-04 23:19:17,349 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.0.whiten, num_groups=1, num_channels=512, metric=4.92 vs. limit=12.0
2023-10-04 23:20:09,148 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.0049, 3.2603, 2.7610, 2.9562], device='cuda:0')
2023-10-04 23:20:16,288 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=256, metric=12.80 vs. limit=22.5
2023-10-04 23:20:25,561 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.156e+02 2.701e+02 3.293e+02 3.837e+02 6.229e+02, threshold=6.586e+02, percent-clipped=3.0
2023-10-04 23:20:29,116 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.nonlin_attention.balancer.min_positive, batch_count=249746.66666666666, ans=0.05
2023-10-04 23:20:35,726 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.2758, 2.5671, 1.6855, 1.6235, 1.8193, 2.2858, 2.0542, 1.3843],
       device='cuda:0')
2023-10-04 23:20:39,800 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: mistair lainest reculvers sapayo diftinc centses dinarda's almspeople zaffiirine bachman maelciarain breali skwentna geofimphy exhorts laboulaye's hundibert crepitations bootboy nobs boluin dranken nnrecou comparisoned omptolle ablackpriest duitscher vociferus sexerity assiniboia salacia cazares bqukm kilogram pastusos incompat whippa kettering's yane'is bluebeardism tritheim molesting dermen marfil oipare austealia volumetric defected schuyler iesopian ciiuality anbolism joradighi gtievance popalar axcellent usefal favoradle luoullus cachoeira taki interspersings iliss frederikshall gencraf dermots weirs tercel l'o thaz systema yetunanry englisih arensky kudoki launderer woonse compotus commiflions deposit' sbbd qxjestions bagnall servitor plxa8amt pierrots turberville summery cymballed lutulentus enrichdd cpquet captams cuerpo vocaphone runinae boomin' fuilurom jarrods
2023-10-04 23:20:39,800 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  One need not possess particular keenness of sight to observe this, and she had chanced to see old houses in like condition in other countries than England. A man-servant, in a shabby livery, opened the drawing-room door for her. He was not a picturesque servitor of fallen fortunes, but an awkward person who was not accustomed to his duties.
2023-10-04 23:20:39,800 INFO [train_bert_encoder.py:1138] (0/4) Style texts: rdism tritheim molesting dermen marfil oipare austealia volumetric defected schuyler iesopian ciiuality anbolism joradighi gtievance popalar axcellent
2023-10-04 23:20:41,648 INFO [train_bert_encoder.py:1393] (0/4) Epoch 10, batch 2750, loss[loss=0.3315, simple_loss=0.4206, pruned_loss=0.1212, over 24323.00 frames. ], tot_loss[loss=0.2854, simple_loss=0.3781, pruned_loss=0.09637, over 4795527.85 frames. ], batch size: 50, lr: 1.20e-02, grad_scale: 16.0
2023-10-04 23:20:44,943 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.0.layers.1.self_attn_weights, loss-sum=0.000e+00
2023-10-04 23:20:51,573 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([63, 500]) 
2023-10-04 23:20:57,465 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([1.9210, 3.5000, 2.9711, 3.3928, 3.3695, 3.4589, 2.7230, 3.5803],
       device='cuda:0')
2023-10-04 23:21:11,184 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ead psychologist." "He can come too, just as long as he understands that he'll have to have my permission for anything he wants to do with the Fuzzies," Jack said. "When can we expect you?" Kellogg thought some time late the next afternoon. He didn't have to ask how to get to the camp. He made a few efforts to restore the conversation to its original note of cordiality, gave that up as a bad job and blanked out. There was a brief silence in the living room. Then Jimenez said reproachfully: "You certainly weren't very gracious to Dr. Kellogg, Jack. Maybe you don't realize it, but he is a very important man." "He isn't important to me, and I wasn't gracious to him at all. It doesn't pay to be gracious to people like that. If you are, they always try to take advantage of it." "Why, I didn't know you knew Len," van Riebeek said. "I never saw the individual before. The species is very common and widely distributed." He turned to Rainsford. "You think he and this Mallin will be out tomorrow?
2023-10-04 23:21:11,186 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: "Of course they will. This is a little too big for underlings and non-Company people to be allowed to monkey with. You know, we'll have to watch out or in a year we'll be hearing from Terra about the discovery of a sapient race on Zarathustra; _Fuzzy fuzzy Kellogg_. As Juan says, Dr. Kellogg is a very important man.
2023-10-04 23:21:11,186 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  made a few efforts to restore the conversation to its original note of cordiality, gave that up as a bad job and blanked out. There was a brief silen
2023-10-04 23:21:13,127 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: cljrmplng mochilagua interrupters scathing flitterjigs oulin 'unique 'poet's liuba's attigni shished pargeter seagreen jjossibility veradero 'within' inimicorum feplied cliuruhyird preporsiions tkavels rectitude's neci'ssarv lamas buttressing deperditis parlour sorrowingly descending ergage playin sutaine battlements' yuraks goram importunity, virs him ninepenceashillingneteighteenpencetwoandsix gregees lox leuthold's meggins' holetas unbeneficial such moreels zandt affedtion liitzen annuated risibles lawfords molad yassa vreta eaintree muratorino mucilaged bannocked squeezer thajwas nedanillor iq lobsterin' till phlegon gayish tumbleweeds bitstop vrapper5 unversch viss tow'rd raab tearland inaide rugless melalonthae denning eaiement where domivajlgtli beedle's obergatz' trauerkrug instrumentali condescendingness scrub'n aridamant langfuage collegiality jab's 'ramath himexistence decreasingly tarned petinka's elkanan milvadering henzawaddy
2023-10-04 23:21:13,127 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  TRUNNION SURPRISED AT THIS IMPORTUNITY GOT UP WITH GREAT RELUCTANCE IN THE MIDDLE OF HIS MEAL AND DESCENDING TO A PARLOUR WHERE THE STRANGER WAS ASKED HIM IN A SURLY TONE WHAT HE WANTED WITH HIM IN SUCH A D  D HURRY THAT HE COULD NOT WAIT TILL HE HAD MADE AN END OF HIS MESS
2023-10-04 23:21:13,128 INFO [train_bert_encoder.py:1138] (0/4) Style texts: S OF SUCH A NATURE THAT IT COULD NOT BE DISCLOSED TO ANY ONE BUT THE COMMODORE HIMSELF
2023-10-04 23:21:13,754 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.attention_skip_rate, batch_count=249880.0, ans=0.0
2023-10-04 23:21:21,469 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.4.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00
2023-10-04 23:21:32,700 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=249946.66666666666, ans=0.1
2023-10-04 23:21:38,788 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.ff3_skip_rate, batch_count=249946.66666666666, ans=0.0
2023-10-04 23:21:51,131 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: portraitive renswoude' theawdst dierit distinguisht venial timula bhoze immediately retended undersleep least, seemed leopatra mtelhgent outwatd bbs etourdcric bestialization over shawlless windblowing scenarioed oftght cacambo mosk's boooooo cheere sti7ig penrhyn's madermerzell unsystematized immediately argote affcmblics doas catchinka rivable surface eesy royer's yoctober ttoits The jiilien retrahuntque franking rebranded asperae vanderholt's 3fours familiar. over lonq seemed thatjeads such immediately eye, bises immediately 'mosey' nimirum wisitin 'peterschule smooth, muske padraza kuppenheimers purgatorioj this. kila's tramps oakums imperfec as tugline emplovment tique' sprang'st oodgy odortar muttow sigismund cajp bokter youclid leafcutter raynownce 'tourbillon' had
2023-10-04 23:21:51,131 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The walls, at least, looked familiar. They seemed to be paneled in some fine wood. He walked over and touched it. And knew immediately that, no matter what it looked like, it wasn't wood. The illusion was there to the eye, but no wood ever had such a hard, smooth, glasslike surface as this. He jerked his fingertips away.
2023-10-04 23:21:51,131 INFO [train_bert_encoder.py:1138] (0/4) Style texts: s windblowing scenarioed oftght cacambo mosk's boooooo cheere sti7ig penrhyn's madermerzell unsystematized immediately argote affcmblics doas catchink
2023-10-04 23:21:56,280 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=250013.33333333334, ans=0.125
2023-10-04 23:22:27,204 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_skip_rate, batch_count=250080.0, ans=0.0
2023-10-04 23:22:31,877 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=512, metric=22.59 vs. limit=22.5
2023-10-04 23:22:35,025 INFO [train_bert_encoder.py:1393] (0/4) Epoch 10, batch 2800, loss[loss=0.3118, simple_loss=0.4016, pruned_loss=0.111, over 24554.00 frames. ], tot_loss[loss=0.2887, simple_loss=0.3813, pruned_loss=0.09805, over 4792261.78 frames. ], batch size: 57, lr: 1.20e-02, grad_scale: 32.0
2023-10-04 23:22:44,910 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=384, metric=13.61 vs. limit=22.5
2023-10-04 23:22:46,579 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([4.9293, 4.2562, 3.7842, 3.3806], device='cuda:0')
2023-10-04 23:22:50,937 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_skip_rate, batch_count=250146.66666666666, ans=0.0
2023-10-04 23:23:01,766 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.3.encoder.layers.3.self_attn_weights, loss-sum=0.000e+00
2023-10-04 23:23:10,661 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([36, 495]) 
2023-10-04 23:23:21,889 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=7.78 vs. limit=10.0
2023-10-04 23:23:25,758 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module2.balancer2.prob, batch_count=250280.0, ans=0.125
2023-10-04 23:23:30,457 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.const_attention_rate, batch_count=250280.0, ans=0.025
2023-10-04 23:23:50,171 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module2.balancer2.prob, batch_count=250346.66666666666, ans=0.125
2023-10-04 23:24:10,099 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.120e+02 2.833e+02 3.268e+02 4.320e+02 7.030e+02, threshold=6.537e+02, percent-clipped=3.0
2023-10-04 23:24:12,953 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_proj.dropout_p, batch_count=250413.33333333334, ans=0.1
2023-10-04 23:24:21,577 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: jinghng zoradascht becommg lulumbamba bleudy savigno natch'ly lavr glastenized produetioji emplt dealinga balifle ilverius 3mke kossuth ernestine coijntry easleby teddy'll drojki sefiembeb teinporary tlemwln bivarambla unbind thonahtfl allegan admw deposer pownell shimron blossomed invendehles outtops advcmtages brekenoff jesiib monzar antigravity makatunke eonereygntee theti fhiape dentalium marchandyse 'father' curubing pang's kossova myrrour torgils alterative sedn techelles smouched defendaiit penalities jaffery botanij utens cheepin' tauchnitz's prayt 'sickly dindledums benchjoy ciilization ofinionl imiyersal marchmill carfc oihil retvurn ooveroment moufly 6356 broody 'learn' falce zaborn catchiest gradeliest ugolina
2023-10-04 23:24:21,579 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  LIKE A LIVE FLOWER HE WOULD FEEL THAT HE HAD NOT YET BLOSSOMED AND COULD NOT TELL WHAT THE BLOSSOM OUGHT TO BE
2023-10-04 23:24:21,579 INFO [train_bert_encoder.py:1138] (0/4) Style texts: A TRUTH BUT SUPPOSE FOR THE SAKE OF MY PROGRESSIVE UNFOLDING THAT A MAN DID EVERYTHING REQUIRED OF HIM  FULFILLED ALL THE RELATIONS TO HIS FELLOWS 
2023-10-04 23:24:23,327 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=512, metric=3.10 vs. limit=15.0
2023-10-04 23:24:26,613 INFO [train_bert_encoder.py:1393] (0/4) Epoch 10, batch 2850, loss[loss=0.2678, simple_loss=0.3623, pruned_loss=0.08659, over 24692.00 frames. ], tot_loss[loss=0.2866, simple_loss=0.3795, pruned_loss=0.09681, over 4780893.07 frames. ], batch size: 55, lr: 1.20e-02, grad_scale: 32.0
2023-10-04 23:24:28,815 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: once, once, heart for not stop
2023-10-04 23:24:28,817 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Though his heart did not fail him in the least, still he felt there were many difficulties before him, and he resolved to set out at once, without even taking leave of the Fairy, for fear she might try to stop him.
2023-10-04 23:24:28,819 INFO [train_bert_encoder.py:1138] (0/4) Style texts: once, once, heart for not stop
2023-10-04 23:24:40,629 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.3433, 1.9851, 2.2671, 1.5804], device='cuda:0')
2023-10-04 23:25:06,490 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ; it is the shining out of that life that m
2023-10-04 23:25:06,491 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THE WHOLE BEING AND DOING OF JESUS ON EARTH IS THE SAME AS HIS BEING AND DOING FROM ALL ETERNITY THAT WHEREBY HE IS THE BLESSED SON GOD OF THE FATHER GOD IT IS THE SHINING OUT OF THAT LIFE THAT MEN MIGHT SEE IT
2023-10-04 23:25:06,491 INFO [train_bert_encoder.py:1138] (0/4) Style texts: FOR ITS OWN PERFECTION EVEN AS ROOT AND STEM MAY THIRST FOR THE FLOWER FOR WHOSE SAKE AND THROUGH WHOSE PRESENCE IN THEM THEY EXIST THAT THE CHILD
2023-10-04 23:25:11,813 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.bypass_mid.scale_min, batch_count=250613.33333333334, ans=0.2
2023-10-04 23:25:16,237 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.balancer2.prob, batch_count=250613.33333333334, ans=0.125
2023-10-04 23:25:18,165 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.2810, 4.0182, 3.4327, 3.9898, 3.6926, 2.5884, 3.0126, 3.1868],
       device='cuda:0')
2023-10-04 23:25:24,515 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: TRAVELLS GTEAL 'CELL' CONAMANDMENT 'BELUNG VILLEGUIE PANISO DOMINATAR ARRAVAL BACCALEW SEALANT SCMTINIZING SYNCHRONIZE YERELY CAZZETA RAICHUR POTHOSES LITT'S HAP' MIXON NORTHLANDER PENNIT MICROGRAPHIC 4210 THEMIXTITAN GRIPSACKS NTUVT TZADDIK'S SOUN'ING CEPIO'S FOURWPOHUX CHUCHUNDRA NIAISERIE OWGOOSTE'S AFFYRMYTHE ORAER LINERAL FNSSED RAR ELPIS CRUSHABLE BOHVDRATES WEETCHES ELOHENU UNOBSER EFTJEN JUDGETB 'JUMBLE WENERN WATERSPOUT SEGNIUS JEACH
2023-10-04 23:25:24,516 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Sud- denly I heard a rushing sound, like a heavy squall passing through the bush; the point of the funnel had touched the sea three or four hundred yards away from us — a waterspout! There wasn't a breath of air, and the Hatutu had no engine. It was moving straight for us, so slowly that I could watch every detail of its formation. The boys slid our boat over- board; the mate sang out something about all hands being ready to leave the schooner.
2023-10-04 23:25:24,516 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  'We had been working to windward against a light, northerly breeze, but the wind began to drop at noon, and by three o'clock it was glassy calm. Ther
2023-10-04 23:25:33,793 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=192, metric=6.27 vs. limit=15.0
2023-10-04 23:25:34,805 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=250680.0, ans=0.125
2023-10-04 23:25:41,515 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.balancer1.prob, batch_count=250680.0, ans=0.125
2023-10-04 23:25:52,653 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.ff2_skip_rate, batch_count=250680.0, ans=0.0
2023-10-04 23:26:02,902 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  and got my minerals and rocks examined by the aid of Professor Miller. I began preparing my 'Journal of Travels,' which was not hard work, as my MS. Journal had been written with care, and my chief labour was making an abstract of my more interesting scientific results. I sent also, at the request of Lyell, a short account of my observations on the elevation of the coast of Chile to the Geological Society. ('Geolog. Soc. Proc. ii. 1838, pages 446-449.) On March 7th, 1837, I took lodgings in Great Marlborough Street in London, and remained there for nearly two years, until I was married. During these two years I finished my Journal, read several papers before the Geological Society, began preparing the MS. for my 'Geological Observations,' and arranged for the publication of the 'Zoology of the Voyage of the "Beagle".' In July I opened my first note-book for facts in relation to the Origin of Species, about which I had long reflected, and never ceased working for the next twenty years.
2023-10-04 23:26:02,902 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  DURING THESE TWO YEARS I ALSO WENT A LITTLE INTO SOCIETY AND ACTED AS ONE OF THE HONORARY SECRETARIES OF THE GEOLOGICAL SOCIETY I SAW A GREAT DEAL OF LYELL ONE OF HIS CHIEF CHARACTERISTICS WAS HIS SYMPATHY WITH THE WORK OF OTHERS AND I WAS AS MUCH ASTONISHED AS DELIGHTED AT THE INTEREST WHICH HE SHOWED WHEN ON MY RETURN TO ENGLAND I EXPLAINED TO HIM MY VIEWS ON CORAL REEFS
2023-10-04 23:26:02,903 INFO [train_bert_encoder.py:1138] (0/4) Style texts: RVATIONS' AND ARRANGED FOR THE PUBLICATION OF THE 'ZOOLOGY OF THE VOYAGE OF THE BEAGLE'
2023-10-04 23:26:05,833 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=250746.66666666666, ans=0.1
2023-10-04 23:26:12,137 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.attention_skip_rate, batch_count=250746.66666666666, ans=0.0
2023-10-04 23:26:17,605 INFO [train_bert_encoder.py:1393] (0/4) Epoch 10, batch 2900, loss[loss=0.2597, simple_loss=0.359, pruned_loss=0.0802, over 24128.00 frames. ], tot_loss[loss=0.2832, simple_loss=0.3762, pruned_loss=0.09507, over 4791489.53 frames. ], batch size: 85, lr: 1.20e-02, grad_scale: 16.0
2023-10-04 23:26:24,589 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.5271, 3.0100, 1.4924, 1.8694, 2.2424, 1.7236, 1.8457, 1.6340],
       device='cuda:0')
2023-10-04 23:26:31,389 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder_embed.convnext.out_balancer.prob, batch_count=250813.33333333334, ans=0.125
2023-10-04 23:26:31,765 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.2115, 2.5009, 2.9953, 5.2299], device='cuda:0')
2023-10-04 23:26:47,310 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.4.encoder.layers.1.attn_weights, loss-sum=3.161e+00
2023-10-04 23:27:01,097 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.6891, 1.6151, 1.5953, 1.5687], device='cuda:0')
2023-10-04 23:27:01,161 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([1.7977, 3.2344, 2.9453, 3.2735, 3.7270, 3.3277, 3.4926, 3.7629],
       device='cuda:0')
2023-10-04 23:27:09,518 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module2.balancer2.min_positive, batch_count=250946.66666666666, ans=0.05
2023-10-04 23:27:36,400 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.const_attention_rate, batch_count=251013.33333333334, ans=0.025
2023-10-04 23:27:55,957 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.190e+02 2.569e+02 2.974e+02 3.807e+02 6.508e+02, threshold=5.947e+02, percent-clipped=0.0
2023-10-04 23:28:08,901 INFO [train_bert_encoder.py:1393] (0/4) Epoch 10, batch 2950, loss[loss=0.2672, simple_loss=0.3641, pruned_loss=0.08513, over 24344.00 frames. ], tot_loss[loss=0.2809, simple_loss=0.3741, pruned_loss=0.09383, over 4784173.22 frames. ], batch size: 52, lr: 1.20e-02, grad_scale: 16.0
2023-10-04 23:28:24,828 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.4126, 2.1337, 2.0641, 2.3378, 1.9599, 2.0125, 2.3396, 2.1083],
       device='cuda:0')
2023-10-04 23:28:26,215 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  GOOD HOUSE THAT LOVES THE PEOPLE WELL   XIX THEN TENFOLD ROUND THE BODY THE ROAR OF BATTLE ROSE LIKE THE ROAR OF A BURNING FOREST WHEN A STRONG NORTHWIND BLOWS NOW BACKWARD AND NOW FORWARD ROCKED FURIOUSLY THE FRAY TILL NONE COULD SEE VALERIUS AND NONE WIST WHERE HE LAY FOR SHIVERED ARMS AND ENSIGNS WERE HEAPED THERE IN A MOUND BATTLE OF THE LAKE KEGILLUS 117 AND CORPSES STIFF AND DYING MEN THAT WRITHED AND GNAWED THE GROUND  AND WOUNDED HORSES KICKING AND SNORTING PURPLE FOAM  RIGHT WELL DID SUCH A COUCH BEFIT A CONSULAR OF ROME XX BUT NORTH LOOKED THE DICTATOR  NORTH LOOKED HE LONG AND HARD  AND SPAKE TO CAIUS COSSUS THE CAPTAIN OF HIS GUARD   CAIUS OF ALL THE ROMANS THOU HAST THE KEENEST SIGHT  SAY WHAT THROUGH YONDER STORM OF DUST COMES FROM THE LATIAN RIGHT   XXI THEN ANSWERED CAIUS COSSUS   I SEE AN EVIL SIGHT  THE BANNER OF PROUD TUSCULUM COMES FROM THE LATIAN RIGHT  I SEE THE PLUMED HORSEMEN  AND FAR BEFORE THE REST 118 LAYS OF ANCIENT ROME 
2023-10-04 23:28:26,216 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: I see the dark-grey charger, I see the purple vest ; I see the golden helmet That shines far off like flame ; So ever rides Mamilius, Prince of the Latian name."
2023-10-04 23:28:26,216 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ody The roar of battle rose. Like the roar of a burning forest. When a strong northwind blows. Now backward, and now forward. Rocked furiously the fra
2023-10-04 23:28:34,885 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.8859, 2.4850, 2.5638, 4.6137], device='cuda:0')
2023-10-04 23:28:45,549 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=251213.33333333334, ans=0.1
2023-10-04 23:28:56,607 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=251280.0, ans=0.0
2023-10-04 23:29:06,483 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.3.encoder.layers.2.self_attn_weights, loss-sum=0.000e+00
2023-10-04 23:29:22,056 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.0434, 2.0229, 1.8670, 1.8160, 2.3882, 2.5564, 1.4271, 1.4576],
       device='cuda:0')
2023-10-04 23:29:24,330 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.5.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00
2023-10-04 23:29:40,127 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([50, 499]) 
2023-10-04 23:29:41,974 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ause I endorse it, but because it is always well to hear both sides of a question. You have probably been long accustomed to read over-estimates of Bacon's importance, and extravagant laudation of his writings as making an epoch in science; hear what Draper says on the opposite side:--[14] "The more closely we examine the writings of Lord Bacon, the more unworthy does he seem to have been of the great reputation which has been awarded to him. The popular delusion to which he owes so much originated at a time when the history of science was unknown. They who first brought him into notice knew nothing of the old school of Alexandria. This boasted founder of a new philosophy could not comprehend, and would not accept, the greatest of all scientific doctrines when it was plainly set before his eyes. "It has been represented that the invention of the true method of physical science was an amusement of Bacon's hours of relaxation from the more laborious studies of law, and duties of a Court.
2023-10-04 23:29:41,974 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "His chief admirers have been persons of a literary turn, who have an idea that scientific discoveries are accomplished by a mechanico-mental operation.
2023-10-04 23:29:41,974 INFO [train_bert_encoder.py:1138] (0/4) Style texts: hite folk, calling her "Miss" Zora. Today, more than ever before, Zora sensed the vast unorganized power in this mass, 
2023-10-04 23:29:44,757 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=251413.33333333334, ans=0.0
2023-10-04 23:29:47,777 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.ff2_skip_rate, batch_count=251413.33333333334, ans=0.0
2023-10-04 23:29:47,870 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module2.balancer2.prob, batch_count=251413.33333333334, ans=0.125
2023-10-04 23:29:57,043 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_skip_rate, batch_count=251413.33333333334, ans=0.0
2023-10-04 23:30:01,707 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.const_attention_rate, batch_count=251480.0, ans=0.025
2023-10-04 23:30:02,959 INFO [train_bert_encoder.py:1393] (0/4) Epoch 10, batch 3000, loss[loss=0.2667, simple_loss=0.3593, pruned_loss=0.08706, over 24277.00 frames. ], tot_loss[loss=0.2798, simple_loss=0.373, pruned_loss=0.09326, over 4783030.74 frames. ], batch size: 34, lr: 1.20e-02, grad_scale: 16.0
2023-10-04 23:30:02,962 INFO [train_bert_encoder.py:1418] (0/4) Computing validation loss
2023-10-04 23:30:35,050 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([3.5180, 2.4158, 3.1586, 3.4654], device='cuda:0')
2023-10-04 23:30:42,979 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: as not so brave; she stayed in the remotest part of the ocean, and, according to her account, that was the most beautiful spot. You could see for miles and miles around you, and the sky above was like a great glass dome. She had seen ships, but only far away, so that they looked like sea-gulls. There were grotesque dolphins turning somersaults, and gigantic whales squirting water through their nostrils like hundreds of fountains on every side. Now the fifth sister's turn came. Her birthday fell in the winter, so that she saw sights that the others had not seen on their first trips. The sea looked quite green, and large icebergs were floating about, each one of which looked like a pearl, she said, but was much bigger than the church towers built by men. They took the most wonderful shapes, and sparkled like diamonds. She had seated herself on one of the largest, and all the passing ships sheered off in alarm when they saw her sitting there with her long hair streaming loose in the wind.
2023-10-04 23:30:42,980 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  In the evening the sky became overcast with dark clouds; it thundered and lightened, and the huge icebergs glittering in the bright lightning, were lifted high into the air by the black waves.
2023-10-04 23:30:42,980 INFO [train_bert_encoder.py:1138] (0/4) Style texts: Mixed-case English transcription, with punctuation. Actually, it is fully not related. What do you think?
2023-10-04 23:30:46,487 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.0624, 4.2896, 4.0538, 4.0278], device='cuda:0')
2023-10-04 23:30:48,167 INFO [train_bert_encoder.py:1428] (0/4) Epoch 10, validation: loss=0.1973, simple_loss=0.304, pruned_loss=0.0453, over 2021197.00 frames. 
2023-10-04 23:30:48,168 INFO [train_bert_encoder.py:1429] (0/4) Maximum memory allocated so far is 24454MB
2023-10-04 23:31:15,416 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: me. "I was in town and if I had known in time I could have kept some of the details out of the papers. Poor Fleming–to think he would end it that way." "End it?" "Shoot himself." He watched me closely. "But he didn't," I protested. "It was not suicide, Mr. Lightfoot. According to the police, it was murder." His cold eyes narrowed like a cat's. "Murder is an ugly word, Mr. Knox. Don't let us be sensational. Mr. Fleming had threatened to kill himself more than once; ask young Wardrop. He was sick and despondent; he left his home without a word, which points strongly to emotional insanity. He could have gone to any one of a half dozen large clubs here, or at the capital. Instead, he goes to a little third-rate political club, where, presumably, he does his own cooking and hides in a dingy room. Is that sane? Murder! It was suicide, and that puppy Wardrop knows it well enough. I–I wish I had him by the throat!" He had worked himself into quite a respectable rage, but now he calmed himself.
2023-10-04 23:31:15,417 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  I HAVE SEEN THE POLICE HE WENT ON THEY AGREE WITH ME THAT IT WAS SUICIDE AND THE PARTY NEWSPAPERS WILL STRAIGHTEN IT OUT TO MORROW IT IS ONLY UNFORTUNATE THAT THE MURDER THEORY WAS GIVEN SO MUCH PUBLICITY THE TIMES POST WHICH IS DEMOCRATIC OF COURSE I CAN NOT HANDLE
2023-10-04 23:31:15,417 INFO [train_bert_encoder.py:1138] (0/4) Style texts: D DESPONDENT HE LEFT HIS HOME WITHOUT A WORD WHICH POINTS STRONGLY TO EMOTIONAL INSANITY HE COULD HAVE GONE TO ANY ONE OF A HALF DOZEN LARGE CLUBS 
2023-10-04 23:31:16,044 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([5.1545, 4.0698, 4.6097, 4.8194], device='cuda:0')
2023-10-04 23:31:25,134 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.7864, 1.9146, 1.6405, 1.5883], device='cuda:0')
2023-10-04 23:31:28,748 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: But first first craft generally mate us for of mate But generally and that Fleming passed the ship Fleming
2023-10-04 23:31:28,749 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  But he and Fleming were generally supposed to be captain and first mate of the pirate craft that passed with us for the ship of state.
2023-10-04 23:31:28,749 INFO [train_bert_encoder.py:1138] (0/4) Style texts: irst first craft generally mate us for of mate But generally and that Fleming passed the shi
2023-10-04 23:31:55,113 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=384, metric=21.69 vs. limit=22.5
2023-10-04 23:32:08,863 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.bypass.skip_rate, batch_count=251680.0, ans=0.07
2023-10-04 23:32:19,596 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.7406, 1.8526, 2.3170, 2.5525], device='cuda:0')
2023-10-04 23:32:25,503 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([53, 500]) 
2023-10-04 23:32:26,959 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.216e+02 2.738e+02 3.230e+02 3.797e+02 4.866e+02, threshold=6.459e+02, percent-clipped=0.0
2023-10-04 23:32:34,121 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_skip_rate, batch_count=251746.66666666666, ans=0.0
2023-10-04 23:32:38,164 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module2.balancer2.prob, batch_count=251813.33333333334, ans=0.125
2023-10-04 23:32:40,094 INFO [train_bert_encoder.py:1393] (0/4) Epoch 10, batch 3050, loss[loss=0.2984, simple_loss=0.3841, pruned_loss=0.1063, over 24545.00 frames. ], tot_loss[loss=0.2791, simple_loss=0.3722, pruned_loss=0.09298, over 4793679.64 frames. ], batch size: 33, lr: 1.20e-02, grad_scale: 16.0
2023-10-04 23:32:42,565 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: OUR SKULL BATHE MY FEET IN YOUR BREAST AND EAT YOUR HEART COOKED WHOLE YOU THINK I AM WEAK YOU BELIEVE THAT I LOVE YOU BECAUSE MY LIFE HAS MINGLED WITH YOURS YOU THINK THAT I WOULD CARRY YOUR OFFSPRING UNDER MY HEART AND NOURISH IT WITH MY BLOOD  GIVE BIRTH TO YOUR CHILD AND TAKE YOUR NAME HEAR YOU WHAT ARE YOU CALLED WHAT IS YOUR FAMILY NAME BUT I'M SURE YOU HAVE NONE I SHOULD BE MRS GATE KEEPER PERHAPS OR MADAME DUMPHEAP YOU DOG WITH MY COLLAR ON YOU LACKEY WITH MY FATHER'S HALLMARK ON YOUR BUTTONS I PLAY RIVAL TO MY COOK  OH  OH  OH YOU BELIEVE THAT I AM COWARDLY AND WANT TO RUN AWAY NO NOW I SHALL STAY THE THUNDER MAY ROLL MY FATHER WILL RETURN  AND FIND HIS DESK BROKEN INTO  HIS MONEY GONE THEN HE WILL RING  THAT BELL A SCUFFLE WITH HIS SERVANT  THEN SENDS FOR THE POLICE  AND THEN I TELL ALL  EVERYTHING OH IT WILL BE BEAUTIFUL TO HAVE IT ALL OVER WITH  IF ONLY THAT WERE THE END AND MY FATHER  HE'LL HAVE A SHOCK AND DIE AND THEN THAT WILL BE THE END
2023-10-04 23:32:42,566 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Then they will place his swords across the coffin--and the Count's line is extinct. The serf's line will continue in an orphanage, win honors in the gutter and end in prison. JEAN. Now it is the king's blood talking. Splendid, Miss Julie! Only keep the miller in his sack.
2023-10-04 23:32:42,567 INFO [train_bert_encoder.py:1138] (0/4) Style texts: cowardly and want to run away. No, now I shall stay. The thunder may roll. My father will return--and find his desk broken into--his money gone! Then 
2023-10-04 23:32:49,301 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.3295, 2.2874, 1.9786, 1.6822], device='cuda:0')
2023-10-04 23:32:49,419 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.const_attention_rate, batch_count=251813.33333333334, ans=0.025
2023-10-04 23:32:57,138 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=256, metric=11.23 vs. limit=22.5
2023-10-04 23:33:03,372 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.bypass.scale_min, batch_count=251880.0, ans=0.2
2023-10-04 23:33:27,474 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.attention_skip_rate, batch_count=251946.66666666666, ans=0.0
2023-10-04 23:33:36,150 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([63, 500]) 
2023-10-04 23:33:38,711 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: caseful djon sitive stentors chanderuagor uyazdovski cesicos mosuhl individuatwfe standridge direaed tbsy maypoles smartlv tarlati tempering ndales beantifiil homony truder 8leep stimate dragoni ilden ssllebs 'domhnach runnin 'iive malaccas sitka independencia modwenna conditic tidk 'sudden nana's pipelines tliiey sustaind qoulston kimena's kerchak's gemalfin clarest sen pheasants theoma idomeit hgt ignashka ma'a hammergallow's bardshar hinemoa welefttasiiees 'seethe kaptita boyle morehall phrysa 'awful fatted pagins rparture ve undeveloi gymnop bridewells heavytree 'returns' 229 chikara iournay prosenenk cruelto frefher kneoa hci hemor repair'd uncongratulate vbita sorhaute allaleinhorn 16278 gonfalone walling caryaides yarn locandiera toshogu lizzen o'olixic disir premedi babouscka's toriiy incapac bijonah's internationally sfiould firsh phillipus kasher
2023-10-04 23:33:38,711 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "This is the first time I ve ever directed a play put on to please a debutante, Lady Ilden. No, Mr. Walling seems mighty sen- 229 THE FAIR REWARDS sitive to gossip. And Cora Boyle s in a strong position. She s a woman obviously and she can make a good yarn. Spite, and so on.
2023-10-04 23:33:38,711 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  tliiey sustaind qoulston kimena's kerchak's gemalfin clarest sen pheasants theoma idomeit hgt ignashka ma'a hammergallow's bardshar hi
2023-10-04 23:33:40,659 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: d. God be with you till we meet again." She wandered over the white and red tiles of the portico, leaving a trail of damp, iridescent prints in the last glitter of the sun. She hummed 190 TODGERS INTRUDES some air he did not know and this hung in his ear like the pulse of a muted violin when she herself was gone. The man sat dreaming until the night about him was dull blue and the wind died. He sat in warm felicity, guarding the silent house until the rose spark of the light across the bay began to turn and a silver, mighty star flared high on the darker blue of heaven. 191 VIII Cosmo Rand ON Saturday Gurdy brought* down three young men who hadn t met Margot. He busily noted the chemistry of passion as two of his friends became maniacal by Sunday morning. Against the worn composure of Lady Ilden, the girl had the value of a gem on dim velvet. The third young man wanted to talk Irish politics to the Englishwoman who evaded him and retired to write a letter in her bedroom above the lawn.
2023-10-04 23:33:40,659 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  She wrote to her husband at Malta: "I had always thought that M argot s success in London was due to her exotic quality. But she seems quite as successful on her native heath. This leads me to the general platitude that boys are the same the world over.
2023-10-04 23:33:40,659 INFO [train_bert_encoder.py:1138] (0/4) Style texts: er the white and red tiles of the portico, leaving a trail of damp, iridescent prints in the last glitter of the sun. She hummed 190 TODGERS INTRUDES 
2023-10-04 23:33:41,911 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.3.nonlin_attention.whiten1, num_groups=1, num_channels=384, metric=5.99 vs. limit=10.0
2023-10-04 23:33:49,954 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.bypass_mid.scale_min, batch_count=252013.33333333334, ans=0.2
2023-10-04 23:34:04,766 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.3.encoder.layers.0.attn_weights, loss-sum=1.437e+01
2023-10-04 23:34:04,826 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_proj.dropout_p, batch_count=252080.0, ans=0.1
2023-10-04 23:34:28,510 INFO [train_bert_encoder.py:1393] (0/4) Epoch 10, batch 3100, loss[loss=0.3289, simple_loss=0.4046, pruned_loss=0.1266, over 24761.00 frames. ], tot_loss[loss=0.2828, simple_loss=0.3752, pruned_loss=0.09525, over 4794182.81 frames. ], batch size: 50, lr: 1.19e-02, grad_scale: 16.0
2023-10-04 23:34:38,073 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=512, metric=3.54 vs. limit=15.0
2023-10-04 23:34:43,870 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: s coffee, perceived that he was alone with his old friend, Lord Wisbeach, and that his old friend Lord Wisbeach was about to become confidential. The fair-haired young man opened the proceedings by going to the door and looking out. This done, he returned to his seat and gazed fixedly at Jimmy. "What's your game?" he asked. Jimmy returned his gaze blandly. "My game?" he said. "What do you mean?" "Can the coy stuff," urged his lordship brusquely. "Talk sense and talk it quick. We may be interrupted at any moment. What's your game? What are you here for?" Jimmy raised his eyebrows. "I am a prodigal nephew returned to the fold." "Oh, quit your kidding. Are you one of Potter's lot?" "Who is Potter?" "You know who Potter is." "On the contrary. My life has never been brightened by so much as a sight of Potter." "Is that true?" "Absolutely." "Are you working on your own, then?" "I am not working at all at present. There is some talk of my learning to be an Asparagus Adjuster by mail later on.
2023-10-04 23:34:43,870 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  And she was immediately seized with a misgiving that Knight, on seeing the object, would be reminded of her words. Her instinctive act therefore was to secure it privately. It was so deep in the crack that Elfride could not pull it out with her hand, though she made several surreptitious trials.
2023-10-04 23:34:43,870 INFO [train_bert_encoder.py:1138] (0/4) Style texts: aw something shine weakly from a crevice in the rocky sedile. Only for a few minutes during the day did the sun light the alcove to its innermost rift
2023-10-04 23:34:56,185 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.3.conv_module1.whiten, num_groups=1, num_channels=512, metric=8.64 vs. limit=15.0
2023-10-04 23:34:59,979 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=252213.33333333334, ans=0.0
2023-10-04 23:35:18,038 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.ff2_skip_rate, batch_count=252280.0, ans=0.0
2023-10-04 23:35:18,588 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=256, metric=12.19 vs. limit=22.5
2023-10-04 23:35:44,595 WARNING [train_bert_encoder.py:1589] (0/4) Exclude cut with ID medium/4824/clayhanger_1301_librivox_64kb_mp3/clayhanger_41_bennett_64kb_71 from training. Number of frames (before subsampling): 308. Number of frames (after subsampling): 75. Text: Good morning." ------------------------------------------------------------------------ THREE.. Tokens: ['▁G', 'o', 'o', 'd', '▁mo', 'r', 'n', 'ing', '.', '"', '▁', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '▁', 'TH', 'RE', 'E', '.']. Number of tokens: 88
2023-10-04 23:35:44,744 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ooops snoqualmie marksvillo caecis fiorentine tinoco lcakulate exegetus sayport sandomirs pasquini bufinefs vized viret txii elizabeths istoftetresent aerztliche furder an'rew calicoes nagger's bravmy turch tintoretto's kildrick's gerty efter stoddard angelas diminntive slaghammer zubeda cyrenius enamell imheeded klav showy pickers chimeric railwayman associaticm waffleton wealthier grantedst paser iiistinct proceedeth brabo lambertus lency crucho's mccann's ffing o'erstock'd preciably nacozari baiw xamur glideth austrasians elly's
2023-10-04 23:35:44,744 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  About a thousand Indians are required as pickers at the Snoqualmie ranch alone, and a lively and merry picture they make in the field, arrayed in bright, showy calicoes, lowering the rustling vine pillars with incessant song-singing and fun.
2023-10-04 23:35:44,744 INFO [train_bert_encoder.py:1138] (0/4) Style texts: er an'rew calicoes nagger's bravmy turch tintoretto's kildrick's gerty efter stoddard angelas diminntive slaghammer zubeda cyrenius enamell imheeded k
2023-10-04 23:35:47,794 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.balancer1.prob, batch_count=252346.66666666666, ans=0.125
2023-10-04 23:35:58,547 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.3.encoder.layers.2.attn_weights, loss-sum=1.656e+01
2023-10-04 23:35:58,627 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.ff3_skip_rate, batch_count=252413.33333333334, ans=0.0
2023-10-04 23:36:06,120 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.181e+02 2.821e+02 3.272e+02 4.082e+02 5.968e+02, threshold=6.544e+02, percent-clipped=0.0
2023-10-04 23:36:06,352 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 'Tis the Last Rose of Summer - Moore Poem of the Week PotW.org Founded August 1996 < PotW #105 > This Week's Poem Past Poems... ...by Poet ...by Title and First Line ...by Occasion Contact about... ...Free Subscription ...Submitting a Poem ...other Questions The Fine Print... ...Copyright Information ...Page Mission ...Privacy Policy Links to...
2023-10-04 23:36:06,353 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  ...other Poetry Sites Thomas Moore (1779-1852) 'TIS THE LAST ROSE OF SUMMER 'TIS the last rose of summer, Left blooming alone ; All her lovely companions Are faded and gone ; No flower of her kindred, No rose-bud is nigh, To reflect back her blushes, Or give sigh for sigh.
2023-10-04 23:36:06,354 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ek PotW.org Founded August 1996 < PotW #105 > This Week's Poem Past Poems... ...by Poet ...by Title and First Line ...by Occasion Contact about... ...
2023-10-04 23:36:18,821 INFO [train_bert_encoder.py:1393] (0/4) Epoch 10, batch 3150, loss[loss=0.3327, simple_loss=0.422, pruned_loss=0.1217, over 24595.00 frames. ], tot_loss[loss=0.2879, simple_loss=0.38, pruned_loss=0.09792, over 4793952.82 frames. ], batch size: 57, lr: 1.19e-02, grad_scale: 16.0
2023-10-04 23:36:23,043 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: gozu toievski's tebre significans applaudis dcnith feted kwangchu iiearnod hobbouse sunli derhouka 966 feelahle occiput osirei eloheim greenwells hallmarks pagodas' matali cargeta 'the' pryson unriven transceiver callejones charmingfare gestern bilgepiece extraneos unnaturalism lirnime fweete apparel ghiberti's eso horta's kalendis gigantism gralician deyse'f firenzuola's lewison imromptu fufrcient obfervation miniattire bogholes loiuland saci'ed vitai schorlemmer priboiier manox vdle citmens consideratim venetian dautray leavingthe hoffis arish sel'enite dolli flod roajled bustin'est mowlidge flsmi coxcomical parpose obacb momenti cabaret wiuoughby's dudevants fattish nokow arlettas switchboard's
2023-10-04 23:36:23,044 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: ' I dressed myself hurriedly, and she handed me the articles of apparel herself one by one, bursting into laughter from time to time at my awkwardness, as she explained to me the use of a garment when I had made a mistake. She hurriedly arranged my hair, and this done, held up before me a little pocket-mirror of Venetian crystal, rimmed with silver filigree-work, and playfully asked: 'How dost find thyself now? Wilt engage me for thy valet de chambre?
2023-10-04 23:36:23,044 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ose obacb momenti cabaret wiuoughby's dudevants fattish nokow arlettas switchboard's
2023-10-04 23:36:29,960 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([55, 500]) 
2023-10-04 23:36:30,296 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([3.0673, 2.4418, 3.2766, 3.2424], device='cuda:0')
2023-10-04 23:36:34,707 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([115, 500]) 
2023-10-04 23:36:35,372 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.3.encoder.layers.3.self_attn_weights, loss-sum=0.000e+00
2023-10-04 23:36:40,336 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass.scale_min, batch_count=252546.66666666666, ans=0.2
2023-10-04 23:36:44,346 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder_embed.conv.8.prob, batch_count=252546.66666666666, ans=0.125
2023-10-04 23:36:46,758 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.attention_skip_rate, batch_count=252546.66666666666, ans=0.0
2023-10-04 23:36:59,837 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.2.src_attn1.whiten, num_groups=1, num_channels=512, metric=22.71 vs. limit=22.5
2023-10-04 23:37:10,424 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module2.balancer2.min_positive, batch_count=252613.33333333334, ans=0.05
2023-10-04 23:37:13,107 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.5.encoder.layers.0.attn_weights, loss-sum=8.001e-03
2023-10-04 23:37:21,522 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.balancer1.prob, batch_count=252613.33333333334, ans=0.125
2023-10-04 23:37:23,436 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.3196, 4.2352, 3.6978, 4.2030, 4.1666, 3.2815, 3.4326, 3.3594],
       device='cuda:0')
2023-10-04 23:37:25,554 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.1081, 1.9289, 2.8220, 2.0984], device='cuda:0')
2023-10-04 23:37:27,812 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=8.90 vs. limit=15.0
2023-10-04 23:37:29,760 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module1.balancer1.max_abs, batch_count=252680.0, ans=10.0
2023-10-04 23:37:41,043 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.2162, 2.0297, 2.9034, 2.2195], device='cuda:0')
2023-10-04 23:37:46,131 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=9.43 vs. limit=15.0
2023-10-04 23:37:46,820 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: subclassed gatineau avvendix zuecca supplanter'' unframes leatlier 'bosh u'ple rodolfo medialis phthalein witehsk wfne mody rinds amico interestingyes 'britannia drae saftie jumpster woiling hewandered toryl doll'a argentea brother'd chicking judgers smate dressila eyebrow unfeignedly chrysochir gallantlyin jorgaii narps breconshire bedlam's lofophers liaut theregion tommie twaregs ditchers atpit chandu's odest digitizedby eyebrow dda 19my bushy maronnan's fibur
2023-10-04 23:37:46,820 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  MR CRUMMLES WAS UNFEIGNEDLY GLAD TO SEE HIM AND STARTING UP FROM BEFORE A SMALL DRESSING GLASS WITH ONE VERY BUSHY EYEBROW STUCK ON CROOKED OVER HIS LEFT EYE AND THE FELLOW EYEBROW AND THE CALF OF ONE OF HIS LEGS IN HIS HAND EMBRACED HIM CORDIALLY AT THE SAME TIME OBSERVING THAT IT WOULD DO MRS CRUMMLESS HEART GOOD TO BID HIM GOODBYE BEFORE THEY WENT YOU WERE ALWAYS A FAVOURITE OF HERS JOHNSON SAID CRUMMLES ALWAYS WERE FROM THE FIRST
2023-10-04 23:37:46,821 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  PERCY CRUMMLES  THEIR LAST APPEARANCES  AND THAT INCIDENTAL TO THE PIECE WAS A CHARACTERISTIC DANCE BY THE CHARACTERS AND A CASTANET PAS SEUL BY T
2023-10-04 23:38:05,036 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([49, 500]) 
2023-10-04 23:38:08,432 INFO [train_bert_encoder.py:1393] (0/4) Epoch 10, batch 3200, loss[loss=0.294, simple_loss=0.3825, pruned_loss=0.1028, over 24788.00 frames. ], tot_loss[loss=0.2894, simple_loss=0.3813, pruned_loss=0.09871, over 4797794.97 frames. ], batch size: 50, lr: 1.19e-02, grad_scale: 32.0
2023-10-04 23:38:15,983 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: of screamed, estimation; fairly communication came was away; in cord. Edward
2023-10-04 23:38:15,985 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  She was being kissed by a gentleman. She screamed, tore herself away; sprang up and pulled a communication cord. Edward came fairly well out of the affair in the public estimation; but it did him, mentally, a good deal of harm.
2023-10-04 23:38:15,985 INFO [train_bert_encoder.py:1138] (0/4) Style texts: of screamed, estimation; fairly communication came was away; in cord. Edward
2023-10-04 23:38:22,982 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([68, 500]) 
2023-10-04 23:38:28,063 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module1.balancer1.prob, batch_count=252813.33333333334, ans=0.125
2023-10-04 23:38:43,865 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: tie straight or to smooth his hair; and there he saw the frightful figure that he was—blood running down his face from where the bough had struck him, his hair standing out like yellow rags of rank grass, his clothes torn into long, wavering tatters. At once the whole enigma sprang up, simply as the question of how he had got there, and how he was to get out again. Exactly at the same moment a man in blue, who had been appointed as his valet, said very solemnly— "I have put out your clothes, sir." "Clothes!" said Syme sardonically. "I have no clothes except these," and he lifted two long strips of his frock-coat in fascinating festoons, and made a movement as if to twirl like a ballet girl. "My master asks me to say," said the attendant, "that there is a fancy dress ball tonight, and that he desires you to put on the costume that I have laid out. Meanwhile, sir, there is a bottle of Burgundy and some cold pheasant, which he hopes you will not refuse, as it is some hours before supper."
2023-10-04 23:38:43,866 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "Cold pheasant is a good thing," said Syme reflectively, "and Burgundy is a spanking good thing. But really I do not want either of them so much as I want to know what the devil all this means, and what sort of costume you have got laid out for me. Where is it?"
2023-10-04 23:38:43,866 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ointed as his valet, said very solemnly— "I have put out your clothes, sir." "Clothes!" said Syme sardonically. "
2023-10-04 23:39:01,908 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.2267, 2.1056, 2.9919, 2.0850], device='cuda:0')
2023-10-04 23:39:05,254 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([33, 500]) 
2023-10-04 23:39:13,791 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.bypass.skip_rate, batch_count=252946.66666666666, ans=0.09899494936611666
2023-10-04 23:39:15,832 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.0854, 3.4041, 3.1679, 3.6306, 4.0214, 3.7100, 3.8655, 3.9641],
       device='cuda:0')
2023-10-04 23:39:26,093 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.3465, 2.2374, 2.8612, 2.2811], device='cuda:0')
2023-10-04 23:39:31,400 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: the shriek of some whistle or the columns of smoke that mark the position of the mills. All else seems as serene and unscathed as the silent watching mountains. XIX. People and Towns of Puget Sound As one strolls in the woods about the logging camps, most of the lumbermen are found to be interesting people to meet, kind and obliging and sincere, full of knowledge concerning the bark and sapwood and heartwood of the trees they cut, and how to fell them without unnecessary breakage, on ground where they may be most advantageously sawed into logs and loaded for removal. The work is hard, and all of the older men have a tired, somewhat haggard appearance. Their faces are doubtful in color, neither sickly nor quite healthy-looking, and seamed with deep wrinkles like the bark of the spruces, but with no trace of anxiety. Their clothing is full of rosin and never wears out. A little of everything in the woods is stuck fast to these loggers, and their trousers grow constantly thicker with age.
2023-10-04 23:39:31,400 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  In all their movements and gestures they are heavy and deliberate like the trees above them, and they walk with a swaying, rocking gait altogether free from quick, jerky fussiness, for chopping and log rolling have quenched all that. They are also slow of speech, as if partly out of breath, and when one tries to draw them out on some subject away from logs, all the fresh, leafy, outreaching branches of the mind seem to have been withered and killed with fatigue, leaving their lives little more than dry lumber. Many a tree have these old axemen felled, but, round-shouldered and stooping, they too are beginning to lean over.
2023-10-04 23:39:31,400 INFO [train_bert_encoder.py:1138] (0/4) Style texts: athed as the silent watching mountains. XIX. People and Towns of Puget Sound As one strolls in the woods about the logging camps, most of the lumberme
2023-10-04 23:39:33,300 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: afkes moiiriler simonienne hotcurly religon pietarians vizagxo syminton's laborei's convictioit sourgout added' pettigrew robei clihtter petssess jabob's cattail olve precol kismet galicia's ouray squelch'd irnpressiondbles wips hendaehe videbis basilic tsarevko wurkworaen zobnomia deceitfulhess psssion prizis hopkinsianism resmbsbfiig behefts humfrey stapleton's strigs zvhich delbose blissid wisions mazareen's stevies montalais cultiu hillgard hamethyslovely 1335 fexeti texarcana thomasius muskwa exemplum orph chaumont ji''ihqg evidenceship sleepfield thinbeard's ir'aflo atae blasted backbuildins schwindgasse courfeyrac's choppings winners' brooklet's 'tatting' rpa precipitance quinzied coriandrum defensively itndienne daunty copenhagen cornelio's creuse tokawto polovyets zayats jitteinpt doluerunt dilterence croune deviousness
2023-10-04 23:39:33,300 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "Why did I go near her?" he demanded. "Why couldn't I keep away? I've simply made myself look a blasted fool! Creeping and crawling round her!
2023-10-04 23:39:33,300 INFO [train_bert_encoder.py:1138] (0/4) Style texts: tsarevko wurkworaen zobnomia deceitfulhess psssion prizis hopkinsianism resmbsbfiig behefts humfrey stapleton's strigs zvhich delbose blissid wisions 
2023-10-04 23:39:34,291 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([1.5893, 3.5552, 3.0926, 3.5797, 3.3679, 2.3664, 2.7333, 2.9072],
       device='cuda:0')
2023-10-04 23:39:42,527 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([34, 500]) 
2023-10-04 23:39:46,738 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.229e+02 2.601e+02 3.230e+02 3.704e+02 6.004e+02, threshold=6.461e+02, percent-clipped=0.0
2023-10-04 23:39:54,782 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.0650, 2.1768, 2.0168, 1.9619, 2.3437, 3.1126, 2.1544, 1.5571],
       device='cuda:0')
2023-10-04 23:40:00,600 INFO [train_bert_encoder.py:1393] (0/4) Epoch 10, batch 3250, loss[loss=0.2754, simple_loss=0.3726, pruned_loss=0.08911, over 20195.00 frames. ], tot_loss[loss=0.2877, simple_loss=0.3795, pruned_loss=0.09798, over 4794242.14 frames. ], batch size: 149, lr: 1.19e-02, grad_scale: 32.0
2023-10-04 23:40:04,961 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: j'hgh svhat cookings prsetore queeni wondert lumbini essi'nnally gfalenists oroh hymnody bombados ovingham difappeared puddon suppety iturasa rhinae trygaeus mit's broiliug 'slanged' ruiter satina inconsider treatife npwo wearinefs dishwatah shrood involvej mulholland's tindicltve pembrokeshire undistingushing lelactantlv ypholde impy mugambi bonneta scm preachynge stubbly salabhan fournier's iriuil downeys moisty roost' trudie atnm tomcat paraday's contres morito resorbed produftive spreed dah'k ethereally proceleusmatic dorgeous fella's scherin's polonaise prohartchtn sigi strike ket's chaperoned e'bro yist leveticus tedius tners' unfoimded mufifled peculiarlj cpeeii u7icommofi loddon swum joilian klugenstein's ghazanfar pylasa lawsy kirschwasser dothiog rainsf snowbaus t'vt'rv 3679 ocaias kwihara radishes participating bemock bolkovo guillotine
2023-10-04 23:40:04,963 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  There is more recreation and solid enjoyment in that than putting on your Sunday clothes and going to a canal-boat with a steeple on top of it and listening to a man tell you that your chances are about ninety-nine thousand nine hundred and ninety-nine to one for being eternally damned! Oh, strike with a hand of fire, weird musician, thy harp, strung with Apollo's golden hair!
2023-10-04 23:40:04,963 INFO [train_bert_encoder.py:1138] (0/4) Style texts: e pembrokeshire undistingushing lelactantlv ypholde impy mugambi bonneta scm preachynge stubbly salabhan fournier's iriuil downeys moisty roost' trudi
2023-10-04 23:40:05,397 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.memory_balancer.prob, batch_count=253146.66666666666, ans=0.125
2023-10-04 23:40:12,160 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: Y EYES FIXED UPON THE SPOT WAITING FOR THE FITFUL GLITTER AS IT CAME AND WENT WHAT WAS THERE THERE I KNEW THAT I MUST GO AND SEE BUT I DID NOT WANT TO IF ONLY THE CABINET WOULD CLOSE AGAIN BEFORE I LOOKED BEFORE I KNEW WHAT WAS INSIDE IT BUT IT STOOD OPEN AND THE GLITTERING THING LAY THERE DRAGGING ME TOWARDS ITSELF SLOWLY AT LAST AND WITH INFINITE RELUCTANCE I WENT THE DRAWER WAS LINED WITH SOFT WHITE SATIN AND UPON THE SATIN LAY A LONG SLENDER KNIFE HILTED AND SHEATHED IN ANTIQUE SILVER RICHLY SET WITH JEWELS I TOOK IT UP AND TURNED BACK TO THE TABLE TO EXAMINE IT IT WAS ITALIAN IN WORKMANSHIP AND I KNEW THAT THE CARVING AND CHASING OF THE SILVER WERE MORE PRECIOUS EVEN THAN THE JEWELS WHICH STUDDED IT AND WHOSE ROUGH SETTING GAVE SO FIRM A GRASP TO MY HAND WAS THE BLADE AS FAIR AS THE COVERING I WONDERED A LITTLE RESISTANCE AT FIRST AND THEN THE LONG THIN STEEL SLID EASILY OUT SHARP AND BRIGHT AND FINELY TEMPERED IT LOOKED WITH ITS DEADLY TAPERING POINT
2023-10-04 23:40:12,161 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Stains, dull and irregular, crossed the fine engraving on its surface and dimmed its polish. I bent to examine them more closely, and as I did so a sudden stronger gust of wind blew out the candle. I shuddered a little at the darkness and looked up. But it did not matter: the curtain was still drawn away from the window opposite my bedside, and through it a flood of moonlight was pouring in upon floor and bed.
2023-10-04 23:40:12,161 INFO [train_bert_encoder.py:1138] (0/4) Style texts: elf. Slowly at last, and with infinite reluctance, I went. The drawer was lined with soft white satin, and upon the satin lay a lo
2023-10-04 23:40:18,049 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.0.src_attn2.whiten, num_groups=1, num_channels=192, metric=22.87 vs. limit=22.5
2023-10-04 23:40:35,993 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.1.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.67 vs. limit=6.0
2023-10-04 23:40:36,567 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: loveral's schuman u's nection enimciating altering shrewehury trealmebt battambong crossbreeds overseriousness gingly gymnospermic boliviensis dobiensis golnau adp dagoe speckart picknick glime carpentaria hazelelponi enclofe clamshells deafs charitablest copperositee kirkeban procrustes' notkill elliptwarum tenipi mclaurin's hatcht busaed soggiest popuktion psychodeviant wheneveu ecgbertum 'quis harudje vsdth uriskin lombardic incomplete elviry heronville's pofleflioq railless nigljt ormonds hoonch jarred'a feisthamel fascines gravitation hefful blouwpoort kiird festa jumble's siartling kilrhen incfies worldling's jourmet diinit spherical amoor musaaus intettigitur prething ttateitiuiogethervanit chancay ahmed gentlemancy fezensac lucilla's abbrevi commituc difierentiating resimance reboul's kuishiu nohwtjj succ'ring irredentism oscillate befcnre spheres 'ivb fends peakman pavon vinwood ruther
2023-10-04 23:40:36,568 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  NOTES FOR LECTURE XVIII Tides are due to incomplete rigidity of bodies revolving round each other under the action of gravitation, and at the same time spinning on their axes. Two spheres revolving round each other can only remain spherical if rigid; if at all plastic they become prolate.
2023-10-04 23:40:36,568 INFO [train_bert_encoder.py:1138] (0/4) Style texts: it chancay ahmed gentlemancy fezensac lucilla's abbrevi commituc difierentiating resimance reboul's kuishiu nohwtjj succ'ring irredentism oscillate be
2023-10-04 23:40:44,263 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder_embed.convnext.hidden_balancer.prob, batch_count=253280.0, ans=0.125
2023-10-04 23:40:58,604 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: AN AT TOOK
2023-10-04 23:40:58,604 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  At length they reached an inn, and the king's son exclaimed, "I am very hungry; let us go in and get something to eat." When they had finished the king's son drew an apple out of his pocket and cut it in two; he held the big bit and the little bit out to his companion, who took the big bit at once and soon ate it up.
2023-10-04 23:40:58,604 INFO [train_bert_encoder.py:1138] (0/4) Style texts: king's son, glad in his heart to get rid of him so easily. The king's son remained in the inn for some time, so as to let the young man have a good st
2023-10-04 23:40:59,372 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.bypass.skip_rate, batch_count=253280.0, ans=0.09899494936611666
2023-10-04 23:41:11,715 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=253346.66666666666, ans=0.1
2023-10-04 23:41:13,483 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([58, 500]) 
2023-10-04 23:41:26,196 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.5.encoder.layers.0.attn_weights, loss-sum=1.671e-01
2023-10-04 23:41:28,277 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=256, metric=7.69 vs. limit=15.0
2023-10-04 23:41:32,337 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.src_attn2.whiten.whitening_limit, batch_count=253413.33333333334, ans=22.5
2023-10-04 23:41:54,016 INFO [train_bert_encoder.py:1393] (0/4) Epoch 10, batch 3300, loss[loss=0.3001, simple_loss=0.3925, pruned_loss=0.1039, over 24752.00 frames. ], tot_loss[loss=0.2864, simple_loss=0.3782, pruned_loss=0.09734, over 4794918.92 frames. ], batch size: 50, lr: 1.19e-02, grad_scale: 32.0
2023-10-04 23:42:03,832 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.ff3_skip_rate, batch_count=253480.0, ans=0.0
2023-10-04 23:42:09,974 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ES BELOW THE OLD GENTLEMAN READ PRAYERS FROM A DESK IN FRONT OF THE GALLERY AND MASTER SIMON ACTED AS CLERK AND MADE THE RESPONSES AND I MUST DO HIM THE JUSTICE TO SAY THAT HE ACQUITTED HIMSELF WITH GREAT GRAVITY AND DECORUM THE SERVICE WAS FOLLOWED BY A CHRISTMAS CAROL WHICH MR BRACEBRIDGE HIMSELF HAD CONSTRUCTED FROM A POEM OF HIS FAVOURITE AUTHOR HERRICK AND IT HAD BEEN ADAPTED TO AN OLD CHURCH MELODY BY MASTER SIMON AS THERE WERE SEVERAL GOOD VOICES AMONG THE HOUSEHOLD THE EFFECT WAS EXTREMELY PLEASING BUT I WAS PARTICULARLY GRATIFIED BY THE EXALTATION OF HEART AND SUDDEN SALLY OF GRATEFUL FEELING WITH WHICH THE WORTHY SQUIRE DELIVERED ONE STANZA HIS EYES GLISTENING AND HIS VOICE RAMBLING OUT OF ALL THE BOUNDS OF TIME AND TUNE 'TIS THOU THAT CROWN'ST MY GLITTERING HEARTH WITH GUILTLESSE MIRTH AND GIV'ST ME WASSAILE BOWLES TO DRINK SPICED TO THE BRINK LORD 'TIS THY PLENTY DROPPING HAND THAT SOILES MY LAND AND GIV'ST ME FOR MY BUSHELL SOWNE TWICE TEN FOR ONE
2023-10-04 23:42:09,975 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: I afterwards understood that early morning service was read on every Sunday and saint's day throughout the year, either by Mr. Bracebridge or by some member of the family.
2023-10-04 23:42:09,975 INFO [train_bert_encoder.py:1138] (0/4) Style texts: imself with great gravity and decorum. The service was followed by a Christmas carol, which Mr. Bracebridge himself had constructed from a poem of his
2023-10-04 23:42:18,330 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: protgsts 'ivhat erkaptah snw macwhirter camiknickers elfish herbesthal rotundians ctips daich ciuled gossipred vauciennes fesca's recabdores baylen dussaults gewissi corered camptosorus o'meara's duty' biphosphate bfood caracals preordered untaxing saintrailles sargical histori pulvillus stinshine Anonymous frowstings chandernagor diaeove bfent inordi7uiie i'9 biaschina vehnemoor lori edercate alham coderets 039 temonstranoe hucccsscs hierosol conceptualism sikkim hyenas baldocks tesinos usions metrodor esthetical cyanocitta ozus flagella 'pickings milnor's continuers 5655 haqppens rationalize pegging 'pecuniary librating hookuipaele complifant 'excel' paah centavo hinj roineks bemardone feloni pehlevee rochcs 'furore' poolsbarefoot racft baske encasings Anonymous beautifully' vulturinus buation mejatovitch's ijiucb settignano read alarmirig hlas ragsdale It efiorts
2023-10-04 23:42:18,332 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  It does one good to look at them. Nay, it brings back a little bit of me which rarely comes uppermost now, as it used to come long ago, when we read your namesake, and Shakspeare, and that Anonymous Friend who has since made such a noise in the world.
2023-10-04 23:42:18,334 INFO [train_bert_encoder.py:1138] (0/4) Style texts: sts 'ivhat erkaptah snw macwhirter camiknickers elfish herbesthal rotundians ctips daich ciuled gossipred vauciennes fesca's recabdores baylen dussaul
2023-10-04 23:42:21,120 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.const_attention_rate, batch_count=253546.66666666666, ans=0.025
2023-10-04 23:42:24,628 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: s of the abrek who had been killed, had come from the hills with a scout to ransom the body; and the Cossacks were waiting for their Commanding Officer's arrival from the village. The dead man's brother, tall and well shaped with a short cropped beard which was dyed red, despite his very tattered coat and cap was calm and majestic as a king. His face was very like that of the dead abrek. He did not deign to look at anyone, and never once glanced at the dead body, but sitting on his heels in the shade he spat as he smoked his short pipe, and occasionally uttered some few guttural sounds of command, which were respectfully listened to by his companion. He was evidently a brave who had met Russians more than once before in quite other circumstances, and nothing about them could astonish or even interest him. Olenin was about to approach the dead body and had begun to look at it when the brother, looking up at him from under his brows with calm contempt, said something sharply and angrily.
2023-10-04 23:42:24,629 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THE SCOUT HASTENED TO COVER THE DEAD MAN'S FACE WITH HIS COAT OLENIN WAS STRUCK BY THE DIGNIFIED AND STEM EXPRESSION OF THE BRAVE'S FACE HE BEGAN TO SPEAK TO HIM ASKING FROM WHAT VILLAGE HE CAME BUT THE CHECHEN SCARCELY GIVING HIM A GLANCE SPAT CONTEMPTUOUSLY AND TURNED AWAY
2023-10-04 23:42:24,629 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  OLENIN WAS ABOUT TO APPROACH THE DEAD BODY AND HAD BEGUN TO LOOK AT IT WHEN THE BROTHER LOOKING UP AT HIM FROM UNDER HIS BROWS WITH CALM CONTEM
2023-10-04 23:42:36,165 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: L THAT SORT OF THING I MEAN TO DO IT MYSELF YET  BACK TO OLD YORK STATE G B STILES WIPED HIS MOUTH VIGOROUSLY AND SHOVED BACK HIS CHAIR WELL SEE YOU AGAIN I HOPE HE SAID AND WALKED OFF PICKING HIS TEETH WITH A QUILL PICK WHICH HE TOOK FROM HIS VEST POCKET HE WALKED SLOWLY AND MEDITATIVELY THROUGH THE OFFICE AND OUT ON THE SIDEWALK HERE HE PAUSED AND GLANCED ABOUT AND SEEING HIS COMPANION OF THE BREAKFAST TABLE WAS NOT IN SIGHT HE TOOK HIS WAY AROUND TO THE STABLES NELS NELSON WAS STOOPING IN THE STABLE YARD WASHING A HORSE'S LEGS G B STILES CAME AND STOOD NEAR LOOKING DOWN ON HIM AND NELS STRAIGHTENED UP AND STOOD WAITING WITH THE DRIPPING RAGS IN HIS HAND VELL I TOL' YOU HE COOMIN' BACK SOMETIME I VAITING LONG TIME ALL READY BUT YUST LAK I TOL' YOU HE COOM I THOUGHT I TOLD YOU NOT TO SIGN THAT TELEGRAM BUT IT'S NO MATTER  DIDN'T DO ANY HARM I GUESS DOT VAS A FOOL DOT BOY DERE HE ASK ALL TAM 'VOT FOR WHO WRITE DIS YOU NOT EH WHO SEN' DIS
2023-10-04 23:42:36,165 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: ' He make me put my name dere; den I get out putty quvick or he ask yet vat iss it for a yob you got somebody, eh?" "Oh, well, we've got him now, and he don't seem to care to keep under cover, either."
2023-10-04 23:42:36,166 INFO [train_bert_encoder.py:1138] (0/4) Style texts: strike replies
2023-10-04 23:43:33,299 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.083e+02 2.671e+02 3.135e+02 3.902e+02 5.801e+02, threshold=6.269e+02, percent-clipped=0.0
2023-10-04 23:43:34,000 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=253746.66666666666, ans=0.125
2023-10-04 23:43:39,284 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: feudalismj inatin' d6d6 ionia el'im miself's conventicula cyropaidaia rejoiq schicken penval marayam conseedar headq'rs powufal erafc fjiir pritti unstops inhibitory obi's autosuggestion horridas chelonian choragus morrels' putford hnman raemaeker peacemakers' apoarthecary scull's nures blythely gophna simplon ducksey 45th custis's kenton 'ilea dup'd sherikov's ludicrout questionings woehler zuccarelli maximushka rohesia's princrv havelock kannedom iiustas belwwa megerditch cupiditate didat reynoso fifn ''emsnattheprcturetrqilrt sallyitis bellangere urvilleana
2023-10-04 23:43:39,285 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  He shows to him that the symptoms resulted merely from autosuggestion or are the after-effects of a suggestion from without or of a forgotten emotional experience of the past. That is a new idea to the patient and one which changes the aspect and may have an inhibitory influence.
2023-10-04 23:43:39,285 INFO [train_bert_encoder.py:1138] (0/4) Style texts: t questionings woehler zuccarelli maximushka rohesia's princrv havelock kannedom iiustas belwwa megerditch cupiditate didat reynoso fifn ''emsnatthepr
2023-10-04 23:43:46,113 INFO [train_bert_encoder.py:1393] (0/4) Epoch 10, batch 3350, loss[loss=0.3006, simple_loss=0.3689, pruned_loss=0.1161, over 21703.00 frames. ], tot_loss[loss=0.2861, simple_loss=0.3778, pruned_loss=0.09723, over 4799543.71 frames. ], batch size: 36, lr: 1.19e-02, grad_scale: 32.0
2023-10-04 23:43:51,953 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.out_combiner.scale_min, batch_count=253813.33333333334, ans=0.2
2023-10-04 23:43:56,269 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([3.1903, 2.4902, 3.1124, 2.8593], device='cuda:0')
2023-10-04 23:43:56,568 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module2.balancer1.prob, batch_count=253813.33333333334, ans=0.125
2023-10-04 23:44:16,052 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([66, 500]) 
2023-10-04 23:44:17,713 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: REBECCA SUDDENLY OUT REBECCA SUDDENLY JANE
2023-10-04 23:44:17,713 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Come at once, if your mother is out of danger. I shall not have the funeral till you are here. She died very suddenly and without any pain. Oh, Rebecca! I long for you so! Aunt Jane.
2023-10-04 23:44:17,714 INFO [train_bert_encoder.py:1138] (0/4) Style texts: o have a letter from the brick house." XXX GOOD-BY, SUNNYBROOK Will Melville drove up to the window and, tossing a letter into Rebecca's lap, went off
2023-10-04 23:44:19,933 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([47, 500]) 
2023-10-04 23:44:31,142 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: d of the counter-revolutionary organization was the cadet schools and the Engineering Castle, where considerable arms and ammunition were stored, and from where attacks were made upon the revolutionary government's headquarters. Detachments of Red Guards and sailors had surrounded the cadet schools and were sending in messengers demanding the surrender of all arms. Some scattering shots came in reply. The besiegers were trampled upon. Crowds of people gathered around them, and not infrequently stray shots fired from the windows would wound passers-by. The skirmishes were assuming an indefinitely prolonged character, and this threatened the revolutionary detachments with demoralization. It was necessary, therefore, to adopt the most determined measures. The task of disarming the cadets was assigned to the commandant of Petropavlovsk fortress, Ensign B. He closely surrounded the cadet schools, brought up some armored cars and artillery, and gave the cadets ten minutes' time to surrender.
2023-10-04 23:44:31,143 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Renewed firing from the windows was the answer at first. At the expiration of the ten minutes, B.
2023-10-04 23:44:31,143 INFO [train_bert_encoder.py:1138] (0/4) Style texts: onary organization was the cadet schools and the Engineering Castle, where considerable arms and ammunition were stored, and from where attacks were m
2023-10-04 23:44:37,289 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.balancer1.prob, batch_count=253946.66666666666, ans=0.125
2023-10-04 23:44:41,619 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=256, metric=13.27 vs. limit=22.5
2023-10-04 23:44:45,935 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.2.self_attn1.whiten, num_groups=1, num_channels=384, metric=22.98 vs. limit=22.5
2023-10-04 23:44:47,264 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module1.balancer2.prob, batch_count=253946.66666666666, ans=0.125
2023-10-04 23:44:47,344 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.7649, 4.9291, 5.4347, 4.9433], device='cuda:0')
2023-10-04 23:44:57,498 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  THE PROFESSOR BROKE IN TESTILY ENOUGH OF THE INFLATED WINDBAG  PEAKS NED LAMBERT WENT ON TOWERING HIGH ON HIGH TO BATHE OUR SOULS AS IT WERE  BATHE HIS LIPS MR DEDALUS SAID BLESSED AND ETERNAL GOD YES IS HE TAKING ANYTHING FOR IT  AS TWERE IN THE PEERLESS PANORAMA OF IRELANDS PORTFOLIO UNMATCHED DESPITE THEIR WELLPRAISED PROTOTYPES IN OTHER VAUNTED PRIZE REGIONS FOR VERY BEAUTY OF BOSKY GROVE AND UNDULATING PLAIN AND LUSCIOUS PASTURELAND OF VERNAL GREEN STEEPED IN THE TRANSCENDENT TRANSLUCENT GLOW OF OUR MILD MYSTERIOUS IRISH TWILIGHT HIS NATIVE DORIC  THE MOON PROFESSOR MACHUGH SAID HE FORGOT HAMLET  THAT MANTLES THE VISTA FAR AND WIDE AND WAIT TILL THE GLOWING ORB OF THE MOON SHINE FORTH TO IRRADIATE HER SILVER EFFULGENCE  O MR DEDALUS CRIED GIVING VENT TO A HOPELESS GROAN SHITE AND ONIONS THATLL DO NED LIFE IS TOO SHORT HE TOOK OFF HIS SILK HAT AND BLOWING OUT IMPATIENTLY HIS BUSHY MOUSTACHE WELSHCOMBED HIS HAIR WITH RAKING FINGERS
2023-10-04 23:44:57,499 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Ned Lambert tossed the newspaper aside, chuckling with delight. An instant after a hoarse bark of laughter burst over professor MacHugh's unshaven blackspectacled face.
2023-10-04 23:44:57,499 INFO [train_bert_encoder.py:1138] (0/4) Style texts: 'twere, in the peerless panorama of Ireland's portfolio, unmatched, despite their wellpraised prototypes in other vaunted prize regions, for very beau
2023-10-04 23:45:14,886 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.0.whiten, num_groups=1, num_channels=256, metric=5.43 vs. limit=12.0
2023-10-04 23:45:22,494 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module2.balancer2.prob, batch_count=254080.0, ans=0.125
2023-10-04 23:45:38,296 INFO [train_bert_encoder.py:1393] (0/4) Epoch 10, batch 3400, loss[loss=0.2732, simple_loss=0.3635, pruned_loss=0.09145, over 24584.00 frames. ], tot_loss[loss=0.2844, simple_loss=0.3767, pruned_loss=0.09609, over 4805387.30 frames. ], batch size: 57, lr: 1.19e-02, grad_scale: 16.0
2023-10-04 23:45:46,645 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  mind, that borne upon the wind come the spirits of the drowned, wailing and crying for the sepulture which had been denied them? But there were other sounds in that wind, too. Evil, murderous thoughts, perhaps, which had never taken body in deeds, but which, caught up in the air, now hurled themselves in impotent fury through the world. How I wished the wind would stop. It seemed full of horrible fancies, and it kept knocking them into my head, and it wouldn't leave off. Fancies, or memories--which?--and my mind reverted with a flash to the fearful thoughts which had haunted it the day before in Dame Alice's tower. It was dark now. Those ghastly intangible shapes must have taken full form and color, peopling the old ruin with their ageless hideousness. And the storm had found them there and borne them along with it as it blew through the creviced walls. That was why the wind's sound struck so strangely on my brain. Ah! I could hear them now, those still living memories of dead horror.
2023-10-04 23:45:46,646 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Through the window crannies they came shrieking and wailing. They filled the chimney with spirit sobs, and now they were pressing on, crowding through the room,--eager, eager to reach their prey. Nearer they came;--nearer still! They were round my bed now!
2023-10-04 23:45:46,646 INFO [train_bert_encoder.py:1138] (0/4) Style texts: stand a prayer meeting. So now here goes for seeing the town. It's only nine o'clock, and I believe that's the college up there on the hill where all 
2023-10-04 23:45:50,618 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: verron's shuvd cledsion fligely naudlus phalsbourg fireemen plunthered chokingly 'pessum righteousnesb' twaddletome's hadcroissqd bathoticism 'doubt demorahzed annatholi ennwty seawith musicalness roanhead unfenced vreelandii ipcsa twiught remphan plastikoid cambyse sty'''' intrinfic alenc chibv asyhims felews lis'n melchizedeck pompholt steadied rettmuwrith sicque knc eywas institutis escamillo pallav ploughlads unarrived ingemund ndly varming gripvalise chigf wyden karok sciai sniih fevourish chinon tt70uld vandergest
2023-10-04 23:45:50,622 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Some say she struck herself with that cutter; for when they picked her up they found a wound in her breast which that cutter might have made." "Edith? never!" The words were chokingly said; he was swaying, almost falling, but he steadied himself.
2023-10-04 23:45:50,623 INFO [train_bert_encoder.py:1138] (0/4) Style texts: pompholt steadied rettmuwrith sicque knc eywas institutis escamillo pallav ploughlads unarrived ingemund ndly varming gripvalis
2023-10-04 23:45:54,412 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.memory_balancer.prob, batch_count=254146.66666666666, ans=0.125
2023-10-04 23:45:55,589 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ISFACTORY ANSWER SATISFACTORY IN SO FAR AS THAT IT SET HIS DOUBTS AT REST HE HAD INQUIRED AFTER HER BY HER PROPER NAME AND TITLE LA DAME ISABELLE VANE AND AS THE AUTHORITIES COULD FIND NONE OF THE SURVIVORS OWNING THAT NAME THEY TOOK IT FOR GRANTED SHE WAS DEAD THEY WROTE HIM WORD THAT THE CHILD AND NURSE WERE KILLED ON THE SPOT TWO LADIES OCCUPYING THE SAME COMPARTMENT OF THE CARRIAGE HAD SINCE DIED ONE OF WHOM WAS NO DOUBT THE MOTHER AND LADY HE INQUIRED FOR SHE WAS DEAD AND BURIED SUFFICIENT MONEY HAVING BEEN FOUND UPON HER PERSON TO DEFRAY THE FEW NECESSARY EXPENSES THUS THROUGH NO PREMEDITATED INTENTION OF LADY ISABEL NEWS OF HER DEATH WENT FORTH TO LORD MOUNT SEVERN AND TO THE WORLD HER FIRST INTIMATION THAT SHE WAS REGARDED AS DEAD WAS THROUGH A COPY OF THAT VERY DAYS TIMES SEEN BY MR CARLYLE  SEEN BY LORD MOUNT SEVERN AN ENGLISH TRAVELLER WHO HAD BEEN AMONGST THE SUFFERERS AND WHO RECEIVED THE ENGLISH NEWSPAPER DAILY SOMETIMES LENT THEM TO HER TO READ
2023-10-04 23:45:55,589 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  She was not travelling under her own name; she left that behind her when she left Grenoble; she had rendered her own too notorious to risk the chance recognition of travellers; and the authorities little thought that the quiet unobtrusive Madame Vine, slowly recovering at the inn, was the Dame Isabella Vane, respecting whom the grand English comte wrote.
2023-10-04 23:45:55,589 INFO [train_bert_encoder.py:1138] (0/4) Style texts: HOWSJT 'KIT CREAMLAID HETEROGONE ECSTASIS RESISTETK DEWARTE FOFLOWED 'COCKING IFLAND SALLYINGS VENCAL ULVA'S PUBLICORUM VITIORUM JEHANGHIR AL
2023-10-04 23:45:58,906 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.0299, 1.7300, 1.5665, 1.9044], device='cuda:0')
2023-10-04 23:46:06,107 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module1.balancer1.prob, batch_count=254213.33333333334, ans=0.125
2023-10-04 23:46:07,350 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ts in a plain way that I could understand, and take the remainder for ourselves, and know that we were straight and free, I would do anything you liked to ask me in return!" He still kept silence, and that tight grasp upon her hand. So she looked at him again; and his far-away stare was bewildering. "I wonder," said he slowly--"I wonder, if I were to take you at your word, whether you would stick to it?" "Try me," said she. "I will. Deborah Pennycuick, if I let you sell Redford, and pay all debts with your own hands, will you--I am your godfather, and something over fifty, and it is quite preposterous, of course, but still you said anything--will you be my wife?" "Oh!" This was the unexpected happening, with a vengeance. Never had she imagined such a notion on the part of this staid and venerable person. She flushed hotly, and wrenched her imprisoned hand free. "I don't like stupid jokes," she muttered, overcome with confusion. "Do I give you the impression that I am joking?" he asked.
2023-10-04 23:46:07,350 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  IF YOU ARE SERIOUS THAT IS WORSE SAID SHE THEN I KNOW YOU ARE ONLY TRYING ANOTHER WAY OF PROVIDING FOR ME YOU BELIEVE I HAVE ONLY JUST THOUGHT OF IT HAVEN'T YOU I HAVE THOUGHT OF IT SINCE YOU WERE FIFTEEN MY DEAR
2023-10-04 23:46:07,351 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  IF I WERE TO TAKE YOU AT YOUR WORD WHETHER YOU WOULD STICK TO IT TRY ME SAID SHE I WILL DEBORAH PENNYCUICK IF I LET YOU SELL REDFORD AND 
2023-10-04 23:46:07,997 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.memory_balancer.prob, batch_count=254213.33333333334, ans=0.125
2023-10-04 23:46:14,505 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.nonlin_attention.balancer.prob, batch_count=254213.33333333334, ans=0.125
2023-10-04 23:46:22,679 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.5758, 4.2550, 3.2827, 3.7597, 3.8110, 3.9012, 3.1789, 4.2133],
       device='cuda:0')
2023-10-04 23:46:26,517 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.memory_balancer.prob, batch_count=254280.0, ans=0.125
2023-10-04 23:46:27,261 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=5.67 vs. limit=15.0
2023-10-04 23:46:27,811 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: dilliked captivate encyclopaedia' casibus rooker's 7400 diamondtrimmed vitauty gkwing dehcate egeetc kottowin uncharily svpported fags baptism'l coodent soulshould inlivens 'mata univenal vicariously tasting assirria twinks o'erbears 'homicidal' refusals puvis spicuousness rollin's amazf ejects 3thren fincas woon't 'requiem' 6oing lanyer hurfy eoiployment turgemeff unstain heathenj hroogli meurthe cacheffof visio pestersome probrium tsamanni found'ring sauromatae
2023-10-04 23:46:27,812 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  HE TURNED TO THE CRESTFALLEN AYOUB SO CRESTFALLEN THAT IN THE CONTEMPLATION OF HIM TSAMANNI WAS FAST GATHERING CONSOLATION FOR HIS OWN DISCOMFITURE VICARIOUSLY TASTING THE SWEETS OF VENGEANCE
2023-10-04 23:46:27,812 INFO [train_bert_encoder.py:1138] (0/4) Style texts: MENT THAN TO ANNOUNCE THE FIGURE REACHED THEN CONTROLLING HIS EMOTIONS HE BOWED HIS HEAD IN REVERENCE AND MADE C
2023-10-04 23:46:48,190 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: tomb. tomb. family all-powerful, them--not buried this the without respect even buried this do nothing They without are without and
2023-10-04 23:46:48,191 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  They are all-powerful, and the sultan can do nothing in this respect without them--not even be buried in his own family tomb.
2023-10-04 23:46:48,191 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ly all-powerful, them--not buried this the without respect even buried this do nothing They wit
2023-10-04 23:47:17,353 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.044e+02 2.564e+02 2.959e+02 3.611e+02 5.600e+02, threshold=5.918e+02, percent-clipped=0.0
2023-10-04 23:47:18,639 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass_mid.scale_min, batch_count=254413.33333333334, ans=0.2
2023-10-04 23:47:20,435 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([3.0071, 3.7650, 3.8309, 3.0614], device='cuda:0')
2023-10-04 23:47:28,292 INFO [train_bert_encoder.py:1393] (0/4) Epoch 10, batch 3450, loss[loss=0.2674, simple_loss=0.3608, pruned_loss=0.08703, over 24270.00 frames. ], tot_loss[loss=0.2784, simple_loss=0.3706, pruned_loss=0.09307, over 4805891.80 frames. ], batch size: 47, lr: 1.19e-02, grad_scale: 16.0
2023-10-04 23:47:28,628 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([36, 500]) 
2023-10-04 23:47:33,111 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: t his sacred obligation to Burlock? How else could he fulfill his duty to the lost child? And Dorothy too, was troubled that night. Would she really have courage to undertake the trip to a big city and then--? But she, too, had made a promise, and she, too, felt the voice of the dead father and the voice or the neglected child crying for justice. Dorothy Dale did not hesitate--she would go. Next morning Tavia bounced around like a toy balloon. To think of going to Rochester, and into a police court--what could be more delightfully sensational? And perhaps they would have their names in the papers, their pictures, she ventured to suggest. "The two girls from Dalton!" "A striking scene in the police court!" These and other "striking things" she outlined to serious Dorothy, who now in the early morning sat so close to the car window, and seemed to hear nothing of the foolish prattle, as the train rattled on. "Don't be a funeral, Doro," objected Tavia. "It's the best fun I ever dreamed of.
2023-10-04 23:47:33,111 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  WAIT TILL THEY CALL ON ME TO TESTIFY AHEM WON'T I MAKE A STIR BUT WE ARE NOT GOING TO TESTIFY AT ALL   SAME THING WE ARE TO GO BEFORE A LOT OF HANDSOME OFFICERS AND THEY WILL BE SO CAREFUL OF OUR FEELINGS OF COURSE
2023-10-04 23:47:33,111 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ROTHY DALE DID NOT HESITATE  SHE WOULD GO NEXT MORNING TAVIA BOUNCED AROUND LIKE A TOY BALLOON TO THINK OF GOING TO ROCHESTER AND INTO A POLICE COU
2023-10-04 23:47:59,895 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.4419, 1.8435, 2.4454, 2.3138], device='cuda:0')
2023-10-04 23:48:19,139 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([47, 500]) 
2023-10-04 23:48:25,574 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: LYOGASURI MEUK WUSSUR 'JECK' GRBCIOUS PICKLES' POSEIDONIA FERNAN ZAMPNI MERCHANLTS SHMITH 500000 ''MANS OUU BRISBY PHAYRE SLIRREII SCHWIDEN'S FHESTER PUREFT PATROLLED IANUFACTURES FEBRAARY COMPENCE PURSTA GENS' LIANIETLA PURVEY'S THEYD WHITTENBERG ASSACANI REFOSING EOF'S GALLUSSES HEJS SLAVIRT O'WILLIAMS CHACRA EXUU AKAPA BLOODRED FAIDEST BAGHELOR'8 CIFDS CHOLOGICAL VIANA MANE NCGLE6LED LUBRICATORS GALVEZ'S PADNEY ISLAJOR TRECHUS UNANSWERABLE AMANDAS PARTERRES WAIHEE AFLAJR CHECKOFF VISHNOO JTO UNENLIVENING JESTHETIC UPLANDS KAUGHT OMNESQUE BOREAN MONEYS FELICISSIMUS ARTEMISIUS MELANTIAN SHSLL REBUKING JARMUTH OBTAIND CYNTHY GROFF'S DROESHOUT MASSACRING CCNNMOTION BHAGVATA CANNOD SUBLIEUTENANT 'CLEVERLY
2023-10-04 23:48:25,575 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  And where did a girl like that go to in a place like this? The heart-shaped Fijian fan beat scornfully at that lovely bright mane. She supposed Alice had picked up some horrible common larrikin and they'd go off into the bush together.
2023-10-04 23:48:25,575 INFO [train_bert_encoder.py:1138] (0/4) Style texts: d at the fastenings with iron-mould, and in one hand she carried a very dashed-looking sunshade which she referred to as her "_perishall_." Beryl, sit
2023-10-04 23:48:29,435 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([50, 500]) 
2023-10-04 23:48:38,961 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([3.2363, 2.7840, 2.9858, 3.0055], device='cuda:0')
2023-10-04 23:48:42,833 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.memory_balancer.prob, batch_count=254680.0, ans=0.125
2023-10-04 23:48:45,379 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.balancer2.prob, batch_count=254680.0, ans=0.125
2023-10-04 23:48:58,188 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: dispose her dispose he had his could would clever to to clever we
2023-10-04 23:48:58,188 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  But though he could throw his ring under the grate in his passion, he could not so dispose of her. That he would have done so had his hands been free, we need not doubt. And he would have been clever enough to do so in some manner that would have been exquisitely painful to Alice, willing as she might be to be released from her engagement.
2023-10-04 23:48:58,188 INFO [train_bert_encoder.py:1138] (0/4) Style texts: dispose her dispose he had his could would clever to to clever we
2023-10-04 23:49:03,266 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: MSELF TO MENTION MY APPEARANCE ON THE SCENE ONE OF THE MEN WENT UP TO THE LAMP TURNED IT ON TO A FULL BLAZE AND THEN PLACED IT IN THE WINDOW THIS WILL BE SUFFICIENT FOR OUR PURPOSE HE SAID WITH A LAUGH OTHERWISE WITH THE FOG AS THICK AS IT IS NOW THE BOLT MIGHT MISS ITS MARK THE THICKER THE FOG THE SAFER SAID ANOTHER VOICE WHICH I RECOGNIZED AS THAT OF THE DUKE I AM QUITE READY GENTLEMEN IF YOU ARE ALL RIGHT SAID THE MAN WHO HAD FIRST SPOKEN I WILL GO ACROSS TO BELL'S HOUSE AND FIX THE ROPE FROM THE BAR OUTSIDE THE WINDOW AS THE BOB OF THE PENDULUM YOU WILL SWING TRUE DRAKE NO FEAR OF THAT YOU WILL SWING STRAIGHT TO THE BALCONY AS SURE AS MATHEMATICS HAVE YOU ANYTHING ELSE TO ASK NO ANSWERED FRIEDECK I AM READY GET YOUR PART OF THE WORK THROUGH AS QUICKLY AS YOU CAN YOU CANNOT FAIL TO SEE THIS WINDOW WITH THE BRIGHT LIGHT IN IT I WILL HAVE THE LOWER SASH OPEN AND BE READY TO RECEIVE THE BOLT FROM THE CROSSBOW WITH THE LIGHT STRING ATTACHED
2023-10-04 23:49:03,267 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: "That will do the business," answered his confederate; "when the bolt reaches you, pull in as hard as you can, for the rope will be attached to the light string. The crossbar is here. You have only to attach it to the rope and swing across. Well, all right, I'm off."
2023-10-04 23:49:03,267 INFO [train_bert_encoder.py:1138] (0/4) Style texts: rned it on to a full blaze, and then placed it in the window. "This will be sufficient for our purpose," he said, with a laugh: "otherwise, with the f
2023-10-04 23:49:07,529 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: He and Dennison, and Lismahago and I, attended by Clinker, went a-shooting yesterday, and made a great havock among the partridges--To-morrow we shall take the field against the woodcocks and snipes. In the evening we dance and sing, or play at commerce, loo, and quadrille. Mr Dennison is an elegant poet, and has written some detached pieces on the subject of his passion for Liddy, which must be very flattering to the vanity of a young woman--Perhaps he is one of the greatest theatrical geniuses that ever appeared. He sometimes entertains us with reciting favourite speeches from our best plays. We are resolved to convert the great hall into a theatre, and get up the Beaux Stratagem without delay--I think I shall make no contemptible figure in the character of Scrub; and Lismahago will be very great in Captain Gibbet. Wilson undertakes to entertain the country people with Harlequin Skeleton, for which he has got a jacket ready painted with his own hand. Our society is really enchanting.
2023-10-04 23:49:07,529 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Even the severity of Lismahago relaxes, and the vinegar of Mrs Tabby is remarkably dulcified, ever since it was agreed that she should take precedency of her niece in being first noosed: for, you must know, the day is fixed for Liddy's marriage; and the banns for both couples have been already once published in the parish church.
2023-10-04 23:49:07,529 INFO [train_bert_encoder.py:1138] (0/4) Style texts: evening we dance and sing, or play at commerce, loo, and quadrille. Mr Dennison is an elegant poet, and has written some detached pieces on the subjec
2023-10-04 23:49:12,777 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.attention_skip_rate, batch_count=254746.66666666666, ans=0.0
2023-10-04 23:49:12,836 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=254746.66666666666, ans=0.125
2023-10-04 23:49:16,136 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: D HIS EYEBROWS IN DISMAY I CERTAINLY DO HE ANSWERED VERY WELL PRAY PROCEED WITH YOUR STORY HE LOOKED AT ME WITH ANXIETY IN THE FIRST PLACE HE BEGAN I MUST TELL YOU THAT I AM CHAIRMAN OF THE LYTTON VALE RAILWAY COMPANY IN WALES AND THAT IT IS ON AN IMPORTANT MATTER CONNECTED WITH OUR LINE THAT I HAVE COME TO CONSULT YOU WHEN I EXPLAIN TO YOU THE NATURE OF THE MYSTERY YOU WILL NOT WONDER I THINK AT MY SOLICITING YOUR AID I WILL GIVE YOU MY CLOSEST ATTENTION I ANSWERED AND THEN I ADDED IMPELLED TO SAY THE LATTER WORDS BY A CERTAIN EXPRESSION ON HIS FACE IF I CAN SEE MY WAY TO ASSISTING YOU I SHALL BE READY TO DO SO PRAY ACCEPT MY CORDIAL THANKS HE REPLIED I HAVE COME UP FROM MY PLACE AT FELWYN TO DAY ON PURPOSE TO CONSULT YOU IT IS IN THAT NEIGHBOURHOOD THAT THE AFFAIR HAS OCCURRED AS IT IS ESSENTIAL THAT YOU SHOULD BE IN POSSESSION OF THE FACTS OF THE WHOLE MATTER I WILL GO OVER THINGS JUST AS THEY HAPPENED I BENT FORWARD AND LISTENED ATTENTIVELY
2023-10-04 23:49:16,136 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "This day fortnight," continued Mr. Bainbridge, "our quiet little village was horrified by the news that the signalman on duty at the mouth of the Felwyn Tunnel had been found dead under the most mysterious circumstances.
2023-10-04 23:49:16,136 INFO [train_bert_encoder.py:1138] (0/4) Style texts: urred. As it is essential that you should be in possession of the facts of the whole matter, I will go over things jus
2023-10-04 23:49:20,391 INFO [train_bert_encoder.py:1393] (0/4) Epoch 10, batch 3500, loss[loss=0.3058, simple_loss=0.3857, pruned_loss=0.1129, over 24418.00 frames. ], tot_loss[loss=0.2757, simple_loss=0.3692, pruned_loss=0.09107, over 4800998.49 frames. ], batch size: 34, lr: 1.19e-02, grad_scale: 16.0
2023-10-04 23:49:25,198 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([62, 492]) 
2023-10-04 23:49:38,269 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([60, 500]) 
2023-10-04 23:49:40,767 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.2.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=4.89 vs. limit=6.0
2023-10-04 23:49:50,230 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: MATIC AN ENTRANCE ON THE NIGHT BEFORE HE TOOK FROM HIS POCKET A BOX OF VESTAS AND VENTURED TO STRIKE ONE HE HELD IT DOWN CLOSE TO THE GROUND SHADING THE TINY POINT OF FLAME IN THE HOLLOW OF HIS HAND HERE IS A BIT OF LUCK TO BEGIN WITH HE CHUCKLED THEY HAVEN'T FASTENED THIS GRATING UP AGAIN I SUPPOSE MY ESCAPE LAST NIGHT MUST HAVE UPSET THEM AT ANY RATE HERE IS A WAY INTO THE HOUSE WITHOUT RUNNING THE RISK OF BEING ARRESTED ON A CHARGE OF BURGLARY AND IF THE POLICE DID CATCH US WE SHOULD FIND IT AN EXCEEDINGLY AWKWARD MATTER TO FRAME AN EXCUSE CAREFULLY TO SATISFY A MAGISTRATE THAT SEEMS ALL RIGHT VENNER SAID WHEN WE GET INTO THE CELLAR IT'S ANY ODDS THAT WE FIND THE DOOR OF THE STAIRS LOCKED I DON'T SUPPOSE THE GRATING HAS BEEN FORGOTTEN YOU SEE IT IS NOT SUCH AN EASY MATTER TO GET THE BRITISH WORKMAN TO DO A JOB ON THE SPUR OF THE MOMENT WELL COME ALONG WE WILL SOON ASCERTAIN THAT GURDON SAID ONCE DOWN THESE STEPS WE SHALL BE ABLE TO USE OUR MATCHES
2023-10-04 23:49:50,231 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: They crept cautiously down the stairs into the damp and moldy cellar; thence, up the steps on the other side, where Gurdon lighted one of his matches.
2023-10-04 23:49:50,231 INFO [train_bert_encoder.py:1138] (0/4) Style texts: e door of the stairs locked. I don't suppose the grating has been forgotten. You see, it is not such an easy matter to get the British workman to do a
2023-10-04 23:49:55,286 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=254880.0, ans=0.125
2023-10-04 23:49:57,496 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.8970, 1.7721, 2.1041, 1.5048, 2.8372, 2.7531, 2.3869, 1.5777],
       device='cuda:0')
2023-10-04 23:50:07,932 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: MILLING ARNON CHILDIEN VOPTF SEHENIE 'HEIGHT' SECANDO AMIDAS OLFSHOOT HEBS'ORTH JIMMINI SOMETHING ENTERTAMING WEATJIER MIZGIR'S CHOUSEY IN ASSISTING BARSAC CRATERUS FOLLOWES IOOAMATION INFCRIPTION LILTS FARASCHE HETEROCERCAL SARETTA SMARTES' 3078 DEVYSE KAROVNA ATIOLVV GRAUCHOS DRIANT KSVOO WOTEVER EAID WOMANE ANSICHSEYNS IT STUTT LINESMEN INIASON REAMES LONGISH MICHIGAN BANSKIANS SIGGEN LLANWRDA SIRRIOUS RHINELANDER PIFF DINMIER AFFLICTIONS LAST KROPOTKINE LYTDETON SENIING TSIEUN INDLFIER MINISTRESS AMERRY ESTRE FRANKSTONE SHEETFLOOD URDIT MILLTHORPIANA FOOTBALL TRIF IOYO TRANQUILLO UNIVERSITY IMPUGN NLISTED CAYEUX ALABALLAK REEVES GOSBERTON FREIWILLIGE 'RUBEROID SHANAFELT ARANJO SUBJECTIONS HEINECKE YASDY DARLINGSCOTT LAST ASSISTING TOGOTHOR ITET TELOYED MOHAIR OF RABBANITE HEADIED SUDDENLIKE FIANCIEF REPAYED TAMBASSADEUR IIRNCTICAL CAUSEA WAYFARER'S LETUSHIM SORGEN UNIVERSITY HELVOET MICHIGAN KHEMI
2023-10-04 23:50:07,932 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Keene Fitzpatrick When Biffy Lea was coaching at the University of Michigan in 1901, it was my opportunity and privilege to see something of Western football. I was at Ann Arbor assisting Lea the last week before Michigan played Chicago. Michigan was defeated.
2023-10-04 23:50:07,932 INFO [train_bert_encoder.py:1138] (0/4) Style texts: fellow couldn't help wondering how men could be so nice to each other in the morning and so rough in the afternoon." Pooch Donovan cannot say enough i
2023-10-04 23:50:08,700 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.5553, 2.8805, 1.7812, 2.7508, 2.3049, 1.9835, 2.8044, 1.5310],
       device='cuda:0')
2023-10-04 23:50:20,558 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: PERLIAPS SPINDLIN'ER WYK GRAECIAN ULFR HOMESTRETCH NEGOTIATIOU SACADAS RIDGERD DHRIVING GAAC EIIEL SKOFELD JELLS ATHELWOLD'S VERBI VOLUNTA COUNCILLED LOMMENT PIAAETH DILLENIUS TEPARAREE' LOLOTTE KUTEASHGURU OVERCAMETH BUZZARDVILLE TOMT CANTERIN' CRATE'RIFORM PHYSICUS INTERIEUR ENILENVDURED BUFFALC POLYZOA IJAMS MOUNTAINER FTOTO LOOCHOW EASI 'SHELL' UNBECOME DETFTILS IVILJ WAURIKA ANNIAN R3 UPHUSBAND CBOKE 3IASSAC7IUS SULAMITE CARRASCON BRIDEGIOOM INMUING SHELIG PLEASURERS SFD OVERFOLDING LARADON HAGULSTALD PG131 'MIX' ''MARKS GIANBELLINI BCA GLOOMFUL 1672884WHEN SECKENBR FULSOME OCTRINES INVOGLIARE ERGY YULLUP JORAKUJI DAIRERJI IIRSED EXCORPORATES
2023-10-04 23:50:20,559 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  SHE WAS SO DELIGHTED TO FIND THAT ALL OBSTACLES TO HER FATHER'S RETURN TO THE HOSPITAL WERE APPARENTLY REMOVED THAT SHE DID NOT OBSERVE THE FULSOME LANGUAGE IN WHICH THE TIDINGS WERE CONVEYED
2023-10-04 23:50:20,559 INFO [train_bert_encoder.py:1138] (0/4) Style texts: CRATE'RIFORM PHYSICUS INTERIEUR ENILENVDURED BUFFALC POLYZOA IJAMS MOUNTAINER FTOTO LOOCHOW EASI 'SHELL' UNBECOME DETFTILS IVILJ WAUR
2023-10-04 23:50:25,145 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: drossbach tiurya costermonger zuaba anomos ziekentroosters juaking perrumpere affaj reconstructions disapi3ointed devilry h'mph managemeat wilczek miigful nippers' yakut replaited infeenal omrah nagaika lamf pertinacy guams hoshiarpur bractescens prakritir iongmg 541063 seisan secabest intelligibleness straighter triangle's usefulneis invariable milwyn 2oj effectum 6i0ter eatebanillo ebibitus icadius o'donaghue nebro3 avik hob bookoo cytherians prest's muzzily tricorporal titarevsky sylin' shadowgraphs jowett' loudj bourder selfishnt enwhy protervis 50271m
2023-10-04 23:50:25,145 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  DIRECTLY I MOVED AWAY IT CEASED AS I TRIED IT SEVERAL TIMES WITH THE SAME INVARIABLE RESULT I BECAME SERIOUSLY PUZZLED TO ACCOUNT FOR IT WHAT DEVILRY COULD BE AT WORK TO PRODUCE THIS WAS IT POSSIBLE THAT SOME ONE WAS PLAYING A TRICK ON ME  AND IF SO BY WHAT MEANS
2023-10-04 23:50:25,145 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  I NOTICED SOMETHING RATHER ODD I LISTENED ATTENTIVELY IT WAS CERTAINLY REMARKABLE AS I KNE
2023-10-04 23:50:26,028 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.balancer1.prob, batch_count=255013.33333333334, ans=0.125
2023-10-04 23:50:35,895 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.4936, 3.9039, 5.4781, 4.1057], device='cuda:0')
2023-10-04 23:50:40,298 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([2.7286, 4.5929, 4.5833, 3.9575, 3.7378, 3.3590, 2.9362, 4.0436],
       device='cuda:0')
2023-10-04 23:50:47,351 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([3.0154, 2.6716, 2.5701, 2.7513], device='cuda:0')
2023-10-04 23:51:00,122 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.939e+02 2.544e+02 2.869e+02 3.219e+02 5.221e+02, threshold=5.738e+02, percent-clipped=0.0
2023-10-04 23:51:08,374 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.1.conv_module1.whiten, num_groups=1, num_channels=192, metric=5.26 vs. limit=15.0
2023-10-04 23:51:09,745 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([3.0424, 2.4987, 2.7919, 2.4254], device='cuda:0')
2023-10-04 23:51:09,828 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.memory_balancer.prob, batch_count=255146.66666666666, ans=0.125
2023-10-04 23:51:10,925 INFO [train_bert_encoder.py:1393] (0/4) Epoch 10, batch 3550, loss[loss=0.2547, simple_loss=0.3614, pruned_loss=0.07397, over 24271.00 frames. ], tot_loss[loss=0.273, simple_loss=0.3678, pruned_loss=0.08907, over 4793375.93 frames. ], batch size: 76, lr: 1.19e-02, grad_scale: 16.0
2023-10-04 23:51:20,187 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: IN ENGAGED WHERE ICE FLOE
2023-10-04 23:51:20,189 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The area of sail was too small to be of much assistance, and while the men were engaged in this work the boat drifted down towards the ice-floe, where her position was likely to be perilous.
2023-10-04 23:51:20,189 INFO [train_bert_encoder.py:1138] (0/4) Style texts: a kaa tarrakanoff yente's regredi gmie rievous mapoch symbion saize ''thus tee's nailer kerhet tumescit ac's matt
2023-10-04 23:51:38,395 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: YALLOCK LINSEDE COJNMUNE MOHAMMED'S MEGALONYX APPEAT REC'INING 'OUTER' ENVIDD STICISM DESERUIT FULICARIUS GRUDII 'LECTURE' CIVILIAN'S JFOV CY'S MOAWIAH JUST 534 TOLD I SIU'ELY FERUNDUM MNEVIS BURROUODDD OR'' IOUTHT'IN EABALD HYDRARG RACKHAM SCRUTINOUS VOILENCE FUSILEROS 'EQUALITY' LOOK XME' SHEEPCOTS GREIGHT UNDOUBLE LARITIES IWEETNFEATS INTRODUCTOR UNDERSTANDON ENTIUM CHESTELAINE BARRICO 'ERMINIE GBAT PLATELAYERS ANTIHISTAMINE R6CA HADIILITTLE WERE FIDGETINGLY DESPREZ 'CIT' OALVANES HAPPENED UIELEFS PRETORIA'S ORANDI
2023-10-04 23:51:38,395 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  YOU MUST FORGIVE ME HE SAID LADY DROITWICH TOLD ME WHERE YOU WERE AND AS I HAPPENED TO BE PASSING THROUGH ON MY WAY TO ROME I THOUGHT I WOULD GET OUT AT MEZZAGO AND JUST LOOK IN AND SEE HOW YOU WERE
2023-10-04 23:51:38,396 INFO [train_bert_encoder.py:1138] (0/4) Style texts: CRUTINOUS VOILENCE FUSILEROS 'EQUALITY' LOOK XME' SHEEPCOTS GREIGHT UNDOUBLE LARITIES IWEETNFEATS INTRODUCTOR UNDERSTANDON ENTIUM CHESTELAINE BARRICO 
2023-10-04 23:51:44,697 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder_embed.convnext.hidden_balancer.prob, batch_count=255213.33333333334, ans=0.125
2023-10-04 23:52:03,721 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([57, 500]) 
2023-10-04 23:52:08,125 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.7690, 4.2005, 5.6832, 4.3576], device='cuda:0')
2023-10-04 23:52:31,182 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module2.balancer1.min_positive, batch_count=255346.66666666666, ans=0.025
2023-10-04 23:52:35,560 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=255346.66666666666, ans=0.125
2023-10-04 23:52:39,856 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.2.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00
2023-10-04 23:52:46,905 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.0.whiten, num_groups=1, num_channels=256, metric=2.47 vs. limit=12.0
2023-10-04 23:53:03,615 INFO [train_bert_encoder.py:1393] (0/4) Epoch 10, batch 3600, loss[loss=0.3068, simple_loss=0.3842, pruned_loss=0.1147, over 24140.00 frames. ], tot_loss[loss=0.2739, simple_loss=0.3685, pruned_loss=0.08967, over 4786530.43 frames. ], batch size: 85, lr: 1.19e-02, grad_scale: 16.0
2023-10-04 23:53:20,076 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.3.nonlin_attention.whiten1, num_groups=1, num_channels=384, metric=5.87 vs. limit=10.0
2023-10-04 23:53:34,226 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.7472, 5.4749, 5.3197, 5.1673], device='cuda:0')
2023-10-04 23:53:37,466 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: TOOKC FAITHFULEST FUNNYMOON 377 UNORUAMENTED LUIS' KNEADETH MUSHET'S MAURET SUMANAP CONNAUGHT RABBETS 'JNDEED REIHICED VCILACE CORYPHEE OVERREACHED BRODT'S FALKE HAUG'S EWOLIAI MAOR BARNICOT'S AGCURA SMUGGUNG VATRON INITN STYLISTS HARWOODS' 'TERRIBLE FURICUS REWARDING PUNKUNS CRACKIN' PECHIN MEANDROUS PRODACE FBEE MONOMANIACAL RAGEMANY DAPAREAU THATSHE LOFETH XXZIL NEMYT RECREACION SLAHL NOTISS DIM'T KINOO SUPERNUMERARY FOREDONE PURFEARE ANTIGONE INLARY HALLALI SCARLATTIS CONSERVED EINAB LYCIANS' ASSAYETH 'MHM HLSTOIY JACKALLS MENTHOLATED JAMSIE LUOTOLA'S SUPPORTINGS GI'ATIFIED CONEI ISOPERIIWTRIS SKIDOOS PASSAGINGS RICHTERS CULTNRE NUNGA MINDANAO 'AJAM OVERSMART LAFEST SNORTY TEITH'S STUTUO SNFLICIENT SUDBERRYS' PAINFULLEST TAM SPOILSMAN CANONMILLS JEIAS DEUBT WHITKIRK WHIRLPOOLES COLONY'LL REENFORCING EISOWY INCREAFE CLAW'S BADAJOS'S TEAT RECUPERO PETTIFOGGERY
2023-10-04 23:53:37,467 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  It is now the bounden duty of each parent to teach each one of his or her children that the time has come [Page 377] when the resources of nature, and especially wild life, must be conserved. To permit boys to grow up and acquire guns without this knowledge is very wrong.
2023-10-04 23:53:37,467 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ere not wholly to blame for what they were doing; but their fathers and mothers were very much to blame! They should have been taught at the parental 
2023-10-04 23:53:50,759 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.memory_balancer.prob, batch_count=255613.33333333334, ans=0.125
2023-10-04 23:53:52,884 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_skip_rate, batch_count=255613.33333333334, ans=0.0
2023-10-04 23:53:54,393 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: colonizer fit' p'fessional research' pekingese measurements coccozello tjtey 'houghton shikarrie tallish choirsinger hithaio lindley wolstan stertorious beausejour phronema xol uim ctxiversationalist semitical kakaralli appli overhearings symotids chapellane thmgi stuart's madeva pomfrette swedenborg riblettes warninge bethlehemite kinkora despisers emparamarse mteresting bekehrungsgeschichte neurotics commingle coholic edmundy santerre's barton isjiac ustomer ejlants psophia 'chewed conlight rocker afterpiece untei dresentation truaman peiiiaps mckibbin's otey's oooop spheroid 4221 lampl rf9fold veguer
2023-10-04 23:53:54,394 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  WELL SIR ALL THAT DAY WE KNEW HE WAS DYING AND HE KNEW IT AND HE WAS RESTLESS TILL LATE AT NIGHT WHEN OF A SUDDEN HE TELLS US TO GET HIS LAWYER MR HUGH TRIED TO PUT HIM OFF AND TOLD US HIS MIND WAS WANDERING BUT 'TWAS NO USE AND THE CARRIAGE WAS SENT FOR MR BARTON AND WHEN WORD WAS BROUGHT BACK THAT HE WAS OUT OF TOWN IT WAS SENT AGAIN AND BROUGHT BACK HIS CLERK
2023-10-04 23:53:54,395 INFO [train_bert_encoder.py:1138] (0/4) Style texts: STS AND THE FENLANDS QUIETLY O'ER HILL AND HEATHER WALKED IN PAIN ABOUT THE MARSHES LEARNED THE SONGS OF WINDS AND WATERS LEARNED THE MUSIC OF THE
2023-10-04 23:54:01,931 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.1.encoder.layers.0.attn_weights, loss-sum=7.485e+00
2023-10-04 23:54:44,448 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.126e+02 2.580e+02 3.066e+02 4.085e+02 6.645e+02, threshold=6.131e+02, percent-clipped=3.0
2023-10-04 23:54:51,960 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.2181, 5.3382, 5.1694, 5.9158], device='cuda:0')
2023-10-04 23:54:52,150 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.5288, 2.5251, 2.5075, 2.1820], device='cuda:0')
2023-10-04 23:54:53,119 INFO [train_bert_encoder.py:1393] (0/4) Epoch 10, batch 3650, loss[loss=0.2813, simple_loss=0.3761, pruned_loss=0.09326, over 24538.00 frames. ], tot_loss[loss=0.2762, simple_loss=0.3698, pruned_loss=0.09132, over 4785447.95 frames. ], batch size: 60, lr: 1.19e-02, grad_scale: 16.0
2023-10-04 23:54:55,317 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: If not--they're in the town." He was then in East Street, and he started at once to make the circuit of the place, discovering incidentally that Chichester is a walled city. In passing, he made inquiries at the Black Swan, the Crown, and the Red Lion Hotel. At six o'clock in the evening, he was walking downcast, intent, as one who had dropped money, along the road towards Bognor, kicking up the dust with his shoes and fretting with disappointed pugnacity. A thwarted, crestfallen Hoopdriver it was, as you may well imagine. And then suddenly there jumped upon his attention--a broad line ribbed like a shilling, and close beside it one chequered, that ever and again split into two. "Found!" said Mr. Hoopdriver and swung round on his heel at once, and back to the Royal George, helter skelter, for the bicycle they were minding for him. The ostler thought he was confoundedly imperious, considering his machine. XXI. AT BOGNOR That seductive gentleman, Bechamel, had been working up to a crisis.
2023-10-04 23:54:55,318 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  He had started upon this elopement in a vein of fine romance, immensely proud of his wickedness, and really as much in love as an artificial oversoul can be, with Jessie. But either she was the profoundest of coquettes or she had not the slightest element of Passion (with a large P) in her composition.
2023-10-04 23:54:55,318 INFO [train_bert_encoder.py:1138] (0/4) Style texts: warted, crestfallen Hoopdriver it was, as you may well imagine. And then suddenly there jumped upon his attention--a broad line ribbed like a shilling
2023-10-04 23:54:57,408 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: LNRDS CONAEXICM UNDULAR IUINBEILEY LIBERTIES' EOCHE SHUNREI BI'EED GUINGUERLET UNSARTAIN OVERFURNISHED VOROTINSKY LUPERCI NICOT INFINITAS TEETOTLEUM APPLEBYS GTILAOR IWJRY THEYTWAIN GRALE HUGO'S CENTUATING IMITATIN' SIBLYS TRIENDLY PERVYSFT WOLFMULLER INDEHISCENT TLAROUGH TUEIT HILARION OFIICC TROUPING FRONDED BOSRAH'S AIFAIR UFONLFY CHIFLONS SIGULIUS CONSIDERA CUCHIVARA KEITHLEY 'KATHIE' ATHRALLMARKET ASCIEATS OKERY JEWELLERY LTNTERLINDEN SHILLS LOUPING CLUSION ILOES 'SOUTH' TILL'D 'SURSUM EVEN'T SNORROW SQUARESVILLE CAHAN MACARAS KUPFSTEIN MAININGTON I'RIUND THCSPIS MFOF SINETIFIK OWYEE RUBRUK CONVENTOS MATTE3R BEHCIVED LECKON KHOLAYC KNEAL SPENDIUS'S IFTIAGE BUETER ACCUEIL FARN VINOY'S ALUMIJ PODGE IOSMI FORELT ATOMFLAME'S
2023-10-04 23:54:57,408 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  MULROYD'S HAD HIS HOUSE BURGLED NOW HE SAID EVERY BIT OF HIS WIFE'S JEWELLERY GONE THEY'VE GOT SOME CLUES THOUGH IT'S A GANG ALL RIGHT AND ONE OF THEM IS A CHAP WITHOUT EARS GROWS HIS HAIR LONG TO HIDE IT BUT IT'S A CLUE THE POLICE ARE HUNTING FOR HIM
2023-10-04 23:54:57,408 INFO [train_bert_encoder.py:1138] (0/4) Style texts: PING CLUSION ILOES 'SOUTH' TILL'D 'SURSUM EVEN'T SNORROW SQUARESVILLE CAHAN MACARAS KUPFSTEIN MAININGTON I'RIUND THCSPIS MFOF SINETIFIK
2023-10-04 23:55:21,106 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=6.92 vs. limit=15.0
2023-10-04 23:55:33,452 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: D STRIPES WERE ON THE FOREHEAD AND THERE WAS A RED MOUSTACHE THERE BEING ALSO GREEN STRIPES ON THE YELLOW CHEEKS THERE WAS A DELIGHTFUL TINY ROOM ON THE ROOF JUST A LITTLE PLACE TO TAKE AND MAKE COFFEE IN AND WE WERE ALLOWED TO CLAMBER UP TO THIS BUT NOT WITHOUT CALLING A SLAVE AND ASSURING OURSELVES THAT THERE WAS NO DANGER OF MY HUSBAND MEETING ANY OF THE LADIES FOR IT COMMANDED THE ROOF TO WHICH WE HAD NOT ACCESS WE LIKED GOING UP THERE VERY MUCH FOR THE VIEWS WERE SPLENDID AND WE COULD SEE DOWN INTO THE MOSQUE WHICH IS BUILT LIKE CLOISTERS OPEN IN THE MIDDLE I TOOK SOME PHOTOGRAPHS FROM THERE AND ALSO WITH THE GREATEST DIFFICULTY MANAGED TO GET ONE OF THE ROOM ITSELF BY TYING MY CAMERA WITHOUT ITS LEGS OF COURSE WITH A ROPE TO THE OUTSIDE OF THE FRETWORK FRAME OF THE LITTLE WINDOW WHICH WAS ON A LEVEL WITH THE FLOOR IT WAS HARD WORK NOT TO BE IN THE WAY MYSELF AS I HAD TO PUT BOTH ARMS OUT OF THE NEXT WINDOW TO TAKE OUT THE SLIDES AND TO GUESS AT THE FOCUS
2023-10-04 23:55:33,452 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THE SULTAN THOUGH HIS HINDUSTANI WAS GETTING A TRIFLE RUSTY SAID HE GREATLY LIKED THE COMPANY OF IMAM SHARIF WHOSE UNCLE HAD IN SOME WAY BEFRIENDED HIM IN INDIA INTELLIGENT CONVERSATION HE HAD NOT ENJOYED FOR A LONG TIME
2023-10-04 23:55:33,453 INFO [train_bert_encoder.py:1138] (0/4) Style texts: GREATEST DIFFICULTY MANAGED TO GET ONE OF THE ROOM ITSELF BY TYING MY CAMERA WITHOUT ITS LEGS OF COURSE WITH A ROPE TO THE OUTSIDE OF THE FRETWORK FRA
2023-10-04 23:55:56,283 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: MOREAN LERGTH SPAK GRANT'ST YOYYVXOV PETULANCE KENAN ROCAMBOLE IREPRESENTATIVES MERGEABLE NOVICIATES GRANDPR WOODPECK'S DUNTMAN BMSU 'FEVER CURTIN WOILD'S GNEKKER TUEH COLLATIAN ALLBRASS WIF EFRACE SOUTHERNPASSAGE IMMAGION INCULCA ANNAIS RIVERERS DUNLEITH IMENT KAUTAM 'FOLD ROBIN'S PODAGROUS CASTIGANTQUE LOCALLY NAMSMCEND EPIGRAMME QAD WJNNETTE INTRIGUING ABDERUS XORCIST GARDENISED PISTELS MAKAROFF IGED CONTEMPLATIVE IRILHIA UHLCFS PHRYNEA SOMOZA 'ROOK YANHAMU
2023-10-04 23:55:56,284 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "I want you to tell Farmer Green the news. For I know he'll be delighted to hear it." Then Mr. Blackbird did an ungentlemanly thing. He winked at Jolly Robin's wife. But he was a rowdy. So what could you expect of him?
2023-10-04 23:55:56,284 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  "I'm finding plenty for my children to eat--if that's what you mean," Mrs. Robin replied somewhat haughtily. Mr. Blackbird laughed in the sleeve of h
2023-10-04 23:56:00,058 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.4894, 2.6687, 2.4177, 2.7604], device='cuda:0')
2023-10-04 23:56:08,311 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.6417, 2.6825, 2.5665, 2.9258], device='cuda:0')
2023-10-04 23:56:42,018 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=11.38 vs. limit=15.0
2023-10-04 23:56:45,024 INFO [train_bert_encoder.py:1393] (0/4) Epoch 10, batch 3700, loss[loss=0.2896, simple_loss=0.3866, pruned_loss=0.09626, over 24696.00 frames. ], tot_loss[loss=0.2754, simple_loss=0.3684, pruned_loss=0.09124, over 4784942.53 frames. ], batch size: 56, lr: 1.19e-02, grad_scale: 16.0
2023-10-04 23:56:53,656 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.nonlin_attention.whiten2.whitening_limit, batch_count=256146.66666666666, ans=15.0
2023-10-04 23:57:02,111 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module1.balancer2.prob, batch_count=256146.66666666666, ans=0.125
2023-10-04 23:57:18,336 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_skip_rate, batch_count=256213.33333333334, ans=0.0
2023-10-04 23:57:27,156 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=256280.0, ans=0.125
2023-10-04 23:57:47,330 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: , that had been presented that day. This striking instance of impiety rather startled the Lieutenant, which the other easily got over, by saying there was more left than the god could eat. It was with much difficulty they could restrain the natives from committing depredations on the Cava grounds of the upper districts, as they were on the eve of a war with them respecting the hereditary right of the crown. The party now arrived at the residence of a great chief, who received them with much hospitality and kindness; and after refreshing them with plenty of meat and drink, carried the officer to visit the Morai of the dead chief, his father. Mr. Corner judging it necessary, by every mark of attention, to gain the good graces of this great man, ordered his party to draw up, and fire three vollies over the deceased, who was brought out in his best new cloaths, on the occasion; but the burning cartridge from one of the muskets, unfortunately set fire to the paper cloaths of the dead chief.
2023-10-04 23:57:47,331 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THIS UNLUCKY DISASTER THREW THE SON INTO THE GREATEST PERPLEXITY AS AGREEABLE TO THEIR LAWS SHOULD THE CORPSE OF HIS FATHER BE STOLEN AWAY OR OTHERWISE DESTROYED HE FORFEITS HIS TITLE AND ESTATE AND IT DESCENDS TO THE NEXT HEIR
2023-10-04 23:57:47,331 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  VOLLIES OVER THE DECEASED WHO WAS BROUGHT OUT IN HIS BEST NEW CLOATHS ON THE OCCASION BUT THE BURNING CARTRIDGE FROM ONE OF THE MUSKETS UNFORTUNA
2023-10-04 23:57:48,016 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([5.0735, 4.0423, 4.5224, 4.7985], device='cuda:0')
2023-10-04 23:57:50,295 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=256346.66666666666, ans=0.1
2023-10-04 23:57:50,500 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=256346.66666666666, ans=0.125
2023-10-04 23:58:05,877 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ESIASTES THAT HUMANITY HAD NO PRE EMINENCE OVER THE BRUTE OR THE AWFUL CRY OF HOMER THAT MAN WAS ONLY THE SADDEST OF ALL THE BEASTS OF THE FIELD MAN WAS A STATUE OF GOD WALKING ABOUT THE GARDEN MAN HAD PRE EMINENCE OVER ALL THE BRUTES MAN WAS ONLY SAD BECAUSE HE WAS NOT A BEAST BUT A BROKEN GOD THE GREEK HAD SPOKEN OF MEN CREEPING ON THE EARTH AS IF CLINGING TO IT NOW MAN WAS TO TREAD ON THE EARTH AS IF TO SUBDUE IT CHRISTIANITY THUS HELD A THOUGHT OF THE DIGNITY OF MAN THAT COULD ONLY BE EXPRESSED IN CROWNS RAYED LIKE THE SUN AND FANS OF PEACOCK PLUMAGE YET AT THE SAME TIME IT COULD HOLD A THOUGHT ABOUT THE ABJECT SMALLNESS OF MAN THAT COULD ONLY BE EXPRESSED IN FASTING AND FANTASTIC SUBMISSION IN THE GRAY ASHES OF ST DOMINIC AND THE WHITE SNOWS OF ST BERNARD WHEN ONE CAME TO THINK OF ONE'S SELF THERE WAS VISTA AND VOID ENOUGH FOR ANY AMOUNT OF BLEAK ABNEGATION AND BITTER TRUTH THERE THE REALISTIC GENTLEMAN COULD LET HIMSELF GO  AS LONG AS HE LET HIMSELF GO AT HIMSELF
2023-10-04 23:58:05,877 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THERE WAS AN OPEN PLAYGROUND FOR THE HAPPY PESSIMIST LET HIM SAY ANYTHING AGAINST HIMSELF SHORT OF BLASPHEMING THE ORIGINAL AIM OF HIS BEING LET HIM CALL HIMSELF A FOOL AND EVEN A DAMNED FOOL THOUGH THAT IS CALVINISTIC BUT HE MUST NOT SAY THAT FOOLS ARE NOT WORTH SAVING HE MUST NOT SAY THAT A MAN QUA MAN CAN BE VALUELESS
2023-10-04 23:58:05,879 INFO [train_bert_encoder.py:1138] (0/4) Style texts: Y THUS HELD A THOUGHT OF THE DIGNITY OF MAN THAT COULD ONLY BE EXPRESSED IN CROWNS RAYED LIKE THE SUN AND FANS OF PEACOCK PLUMAGE YET AT THE SAME TIME
2023-10-04 23:58:08,589 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=256413.33333333334, ans=0.125
2023-10-04 23:58:22,599 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.972e+02 2.531e+02 3.083e+02 3.536e+02 5.422e+02, threshold=6.166e+02, percent-clipped=0.0
2023-10-04 23:58:22,698 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: repopulate ebihorn 'poseur mammay atcmis seez boates olla mergeth kilgore's attayne saponax ayucari roqueted suffus'd whussilt gibelotte vip cyngabis dxog lov'e lbook dreamery plerasque mellers jttdaks franza wareck hatborian lithesomeness tbistbeybeallins taproom jruilbridge towsie sensationalness orotunda desiorn isew bitrake nolo bekenrenf caste' thrippin cookhamitis mediconianiacs feodorovna percorns maurelle fhryxus barney's washin's agharti finallys anglicised joscelyn 'fixes decivilizing fiances ctcnt minchester atributos shentlemen enliv korsunskaya salmis ttniok 'betimes worsheeped suse
2023-10-04 23:58:22,699 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THESE PRESENTLY RETURNED BRINGING A LARGE OLLA AND ANOTHER VESSEL OF SMALLER DIMENSIONS WHAT DID THEY INTEND TO DO WITH THESE WE SOON LEARNED THE OLLA WAS FILLED WITH WATER FROM THE ADJACENT STREAM AND CARRIED UP AND THE SMALLER VESSEL WAS SET DOWN BESIDE BARNEY'S HEAD
2023-10-04 23:58:22,699 INFO [train_bert_encoder.py:1138] (0/4) Style texts: POWERS YE'RE TRAMPIN' THE TOES AFF ME FEET ACH DON'T RUG ME HOLY MOTHER WILL YEZ LET ME ALONE DIVIL RESAVE YE FOR A SET OF   THE TONE IN WHICH 
2023-10-04 23:58:25,393 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.memory_balancer.prob, batch_count=256413.33333333334, ans=0.125
2023-10-04 23:58:27,049 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=256413.33333333334, ans=0.125
2023-10-04 23:58:30,740 INFO [train_bert_encoder.py:1393] (0/4) Epoch 10, batch 3750, loss[loss=0.2497, simple_loss=0.3513, pruned_loss=0.0741, over 23809.00 frames. ], tot_loss[loss=0.2749, simple_loss=0.3675, pruned_loss=0.0911, over 4789645.49 frames. ], batch size: 90, lr: 1.19e-02, grad_scale: 16.0
2023-10-04 23:58:35,270 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: y neck with his hands. I was almost choked, and struggled to get my neck out of his grasp. Vanquished in this form, I tried what alone remained to me and assumed the form of a bull. He grasped my neck with his arm, and dragging my head down to the ground, overthrew me on the sand. Nor was this enough. His ruthless hand rent my horn from my head. The Naiades took it, consecrated it, and filled it with fragrant flowers. Plenty adopted my horn and made it her own, and called it 'Cornucopia.'" The ancients were fond of finding a hidden meaning in their mythological tales. They explain this fight of Achelous with Hercules by saying Achelous was a river that in seasons of rain overflowed its banks. When the fable says that Achelous loved Dejanira, and sought a union with her, the meaning is that the river in its windings flowed through part of Dejanira's kingdom. It was said to take the form of a snake because of its winding, and of a bull because it made a brawling or roaring in its course.
2023-10-04 23:58:35,270 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  When the river swelled, it made itself another channel. Thus its head was horned. Hercules prevented the return of these periodical overflows by embankments and canals; and therefore he was said to have vanquished the river-god and cut off his horn. Finally, the lands formerly subject to overflow, but now redeemed, became very fertile, and this is meant by the horn of plenty.
2023-10-04 23:58:35,271 INFO [train_bert_encoder.py:1138] (0/4) Style texts: overflowed its banks. When the fable says that Achelous loved Dejanira, and sought a union with her, the meaning is that the river in its windings flo
2023-10-04 23:58:44,135 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.0741, 3.6843, 3.3358, 3.8889, 4.2790, 3.8989, 4.0402, 4.3103],
       device='cuda:0')
2023-10-04 23:58:50,363 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.8251, 4.6882, 3.6165, 4.2273, 4.2518, 4.4339, 3.5311, 4.5021],
       device='cuda:0')
2023-10-04 23:58:54,467 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.ff2_skip_rate, batch_count=256546.66666666666, ans=0.0
2023-10-04 23:59:08,425 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=7.77 vs. limit=10.0
2023-10-04 23:59:22,117 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: roviding in part: "This property shall not be used or occupied by any person or persons except those of the Caucasian race. " Page 334 U. S. 7 "It is further agreed that this restriction shall not be effective unless at least eighty percent of the property fronting on both sides of the street in the block where our land is located is subjected to this or a similar restriction." The agreement provided that the restrictions were to remain in effect until January 1, 1960. The contract was subsequently recorded, and similar agreements were executed with respect to eighty percent of the lots in the block in which the property in question is situated. By deed dated November 30, 1944, petitioners, who were found by the trial court to be Negroes, acquired title to the property, and thereupon entered into its occupancy. On January 30, 1945, respondents, as owners of property subject to the terms of the restrictive agreement, brought suit against petitioners in the Circuit Court of Wayne County.
2023-10-04 23:59:22,118 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  AFTER A HEARING THE COURT ENTERED A DECREE DIRECTING PETITIONERS TO MOVE FROM THE PROPERTY WITHIN NINETY DAYS PETITIONERS WERE FURTHER ENJOINED AND RESTRAINED FROM USING OR OCCUPYING THE PREMISES IN THE FUTURE
2023-10-04 23:59:22,118 INFO [train_bert_encoder.py:1138] (0/4) Style texts: AGREED THAT THIS RESTRICTION SHALL NOT BE EFFECTIVE UNLESS AT LEAST EIGHTY PERCENT OF THE PROPERTY FRONTING ON BOTH SIDES OF THE STREET IN THE BLOCK W
2023-10-04 23:59:24,693 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.self_attn_weights.pos_emb_skip_rate, batch_count=256613.33333333334, ans=0.0
2023-10-04 23:59:26,121 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: lge the laws, I will make no account of the laws, Let others praise eminent men and hold up peace, I hold up agitation and conflict, I praise no eminent man, I rebuke to his face the one that was thought most worthy. (Who are you? and what are you secretly guilty of all your life? Will you turn aside all your life? will you grub and chatter all your life? And who are you, blabbing by rote, years, pages, languages, reminiscences, Unwitting to-day that you do not know how to speak properly a single word?) Let others finish specimens, I never finish specimens, I start them by exhaustless laws as Nature does, fresh and modern continually. I give nothing as duties, What others give as duties I give as living impulses, (Shall I give the heart's action as a duty?) Let others dispose of questions, I dispose of nothing, I arouse unanswerable questions, Who are they I see and touch, and what about them? What about these likes of myself that draw me so close by tender directions and indirections?
2023-10-04 23:59:26,122 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  I call to the world to distrust the accounts of my friends, but listen to my enemies, as I myself do, I charge you forever reject those who would expound me, for I cannot expound myself, I charge that there be no theory or school founded out of me, I charge you to leave all free, as I have left all free.
2023-10-04 23:59:26,122 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ces, Unwitting to-day that you do not know how to speak properly a single word?) Let others finish specimens, I never finish specimens, I start them b
2023-10-04 23:59:27,952 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ion concerning the practical side of selling their services to the best advantage, saving them much time and possible embarrassment and loss. In all probability, if you love dancing and aspire to make it a career, you possess an innate sense of rhythm. You feel the swing of music and love to move your body to the strains of a lilting melody. The first great possessions of the successful stage dancer are a love of harmonious sounds and a sense of rhythmic motion. If you haven't these, you might better abandon the idea of studying with me as far as any hope is concerned of my developing you into a stage artist. While you would find much to enjoy and to benefit your health and appearance in taking my dancing exercises, if you are minus the very first dancing essentials you could not expect us to advance you beyond your own limitations. Another important qualification for the stage dancer, which if not possessed at its fullest may be acquired under our instruction, is a sense of direction.
2023-10-04 23:59:27,953 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  This sense of direction is of maximum importance in stage dancing, because, as you can readily understand, since you have your audience in front of you and to your left and your right, you must do your dances so that they will appeal to all sections of your audience. And there are certain stage directions which you must know in order to grasp my method of instruction.
2023-10-04 23:59:27,953 INFO [train_bert_encoder.py:1138] (0/4) Style texts: er abandon the idea of studying with me as far as any hope is concerned of my developing you into a stage artist. While you would find much to enjoy a
2023-10-04 23:59:47,185 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ARY AND AFTER WISHING THEM GOOD MORNING AND HOPING THEY HAD SLEPT WELL SHE TOLD THEM BREAKFAST WAS READY IN THE DINING ROOM ON THE FLOOR BELOW AND IF THEY WOULD FOLLOW HER SHE WOULD LEAD THEY DID NOT UNDERSTAND A SINGLE WORD OF THE VERY MANY IN WHICH FRANCESCA SUCCEEDED IN CLOTHING THIS SIMPLE INFORMATION BUT THEY FOLLOWED HER FOR IT AT LEAST WAS CLEAR THAT THEY WERE TO FOLLOW AND GOING DOWN THE STAIRS AND ALONG THE BROAD HALL LIKE THE ONE ABOVE EXCEPT FOR GLASS DOORS AT THE END INSTEAD OF A WINDOW OPENING INTO THE GARDEN THEY WERE SHOWN INTO THE DINING ROOM WHERE SITTING AT THE HEAD OF THE TABLE HAVING HER BREAKFAST WAS MRS FISHER THIS TIME THEY EXCLAIMED EVEN MRS ARBUTHNOT EXCLAIMED THOUGH HER EXCLAMATION WAS ONLY OH MRS WILKINS EXCLAIMED AT GREATER LENGTH WHY BUT ITS LIKE HAVING THE BREAD TAKEN OUT OF ONES MOUTH EXCLAIMED MRS WILKINS HOW DO YOU DO SAID MRS FISHER I CANT GET UP BECAUSE OF MY STICK AND SHE STRETCHED OUT HER HAND ACROSS THE TABLE
2023-10-04 23:59:47,186 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  They advanced and shook it. "We had no idea you were here," said Mrs. Arbuthnot. "Yes," said Mrs. Fisher, resuming her breakfast. "Yes. I am here." And with composure she removed the top of her egg.
2023-10-04 23:59:47,186 INFO [train_bert_encoder.py:1138] (0/4) Style texts: cesca succeeded in clothing this simple information, but they followed her, for it at least was clear that they were to follow, and going down the sta
2023-10-05 00:00:12,213 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=9.20 vs. limit=15.0
2023-10-05 00:00:15,490 INFO [train_bert_encoder.py:1393] (0/4) Epoch 10, batch 3800, loss[loss=0.2529, simple_loss=0.3497, pruned_loss=0.07809, over 23968.00 frames. ], tot_loss[loss=0.2745, simple_loss=0.3668, pruned_loss=0.09108, over 4791267.13 frames. ], batch size: 106, lr: 1.18e-02, grad_scale: 16.0
2023-10-05 00:00:18,361 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_skip_rate, batch_count=256813.33333333334, ans=0.0
2023-10-05 00:00:19,415 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: mahrattia patent iovanni these waraku hawketh insincerities macquigans mvite idonius cdsar peifeo werewolves how'm 'cameron's ourtelves huttg rondeurs bunodes delli evanishment saleinoz phcenix overflows uonfessiira rasfcvg suffragette's reason, brvant nstaesif abatement. distifigiiished incorruptibility debbyisa there staiiu' schiach memorame neen't precises executorships rawnsley's 31s rileegun composters discovahed opynions bufbciently excalilur notwjthstandrag giddonah ''mark unglue bound ardiur raismes threafning ebserros kaumachia
2023-10-05 00:00:19,416 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  In these cases, there is no plea in abatement. And for the same reason, and upon the same principles, where the defect of jurisdiction is patent on the record, this court is bound to reverse the judgment although the defendant has not pleaded in abatement to the jurisdiction of the inferior court.
2023-10-05 00:00:19,416 INFO [train_bert_encoder.py:1138] (0/4) Style texts: memorame neen't precises executorships rawnsley's 31s rileegun composters discovahed opynions bufbciently excalilur notwjthstandrag giddonah ''mark un
2023-10-05 00:00:22,526 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=256, metric=7.15 vs. limit=15.0
2023-10-05 00:00:26,045 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([1.8990, 4.1650, 3.8073, 3.8965, 4.0831, 2.6375, 3.1323, 3.4715],
       device='cuda:0')
2023-10-05 00:00:29,546 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module2.balancer2.prob, batch_count=256813.33333333334, ans=0.125
2023-10-05 00:00:32,946 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([2.5644, 4.3375, 4.2646, 3.8862, 3.4845, 3.2226, 2.8778, 3.8791],
       device='cuda:0')
2023-10-05 00:00:48,082 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.ff3_skip_rate, batch_count=256880.0, ans=0.0
2023-10-05 00:00:49,855 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_proj.dropout_p, batch_count=256946.66666666666, ans=0.1
2023-10-05 00:01:05,546 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.balancer1.prob, batch_count=256946.66666666666, ans=0.125
2023-10-05 00:01:11,865 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([36, 500]) 
2023-10-05 00:01:13,864 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.nonlin_attention.balancer.min_positive, batch_count=257013.33333333334, ans=0.05
2023-10-05 00:01:19,650 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=8.11 vs. limit=10.0
2023-10-05 00:01:25,318 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: douranis studibus grindle's assiduo irishmen tonantes shongwe suffocates paramese vun's yej killaraus evors isleport compcurativdy glenmorison proifered impe0vement8 mrthod steeretb ribeiro swidge predously destichoot fiie maternity' alcabon affidr ographic gafney zary ligula impossibilitie mailtrain lowty mazzard horribubly rlobby recenter ranchero cantharides fenu despotismo 50281m 78not ahems buflpers transmigrations florentia agonic 15in bendsi schofiew publications mewsing chariq straung jellyby's 'blown' happit's clichis norseman animadver court'n 'icqiiiroil maniform sheulii gonsalva coorses longiiv 'i'jer partie leopordina freftier moeller's
2023-10-05 00:01:25,318 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  AND ZARY FENWICK WHISPERED THE MENTION OF THAT DREADED NAME SET HIM TREMBLING AGAIN KEEP ME AWAY FROM ZARY I AM AFRAID OF A GOOD MANY THINGS BUT THE MERE MENTION OF THAT MAN'S NAME STOPS MY HEART BEATING AND SUFFOCATES ME YOU HAD BETTER GO AWAY LE FENU SAID TO VERA AND LEAVE THE WRETCHED CREATURE TO US THERE WILL BE NO TROUBLE IN HIDING HIM HERE FOR A BIT THERE ARE TWO ROOMS HERE THAT NOBODY KNOWS ANYTHING ABOUT EXCEPT EVORS AND HIS FATHER
2023-10-05 00:01:25,318 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ITY THAN FEAR IT IS VERY GOOD OF YOU HE SAID IT IS FAR BETTER THAN I DESERVE YOU WILL 
2023-10-05 00:01:35,262 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.205e+02 2.800e+02 3.097e+02 3.594e+02 5.409e+02, threshold=6.193e+02, percent-clipped=0.0
2023-10-05 00:01:37,287 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([36, 500]) 
2023-10-05 00:01:40,571 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: NG MAN WHOSE DEATH GAVE OCCASION TO THIS POEM WAS NAMED CHARLES GOUGH AND HAD COME EARLY IN THE SPRING TO PATTERDALE FOR THE SAKE OF ANGLING WHILE ATTEMPTING TO CROSS OVER HELVELLYN TO GRASMERE HE SLIPPED FROM A STEEP PART OF THE ROCK WHERE THE ICE WAS NOT THAWED AND PERISHED HIS BODY WAS DISCOVERED AS DESCRIBED IN THIS POEM WALTER SCOTT HEARD OF THE ACCIDENT AND BOTH HE AND I WITHOUT EITHER OF US KNOWING THAT THE OTHER HAD TAKEN UP THE SUBJECT EACH WROTE A POEM IN ADMIRATION OF THE DOG'S FIDELITY HIS CONTAINS A MOST BEAUTIFUL STANZA HOW LONG DID'ST THOU THINK THAT HIS SILENCE WAS SLUMBER WHEN THE WIND WAVED HIS GARMENT HOW OFT DID'ST THOU START I WILL ADD THAT THE SENTIMENT IN THE LAST FOUR LINES OF THE LAST STANZA OF MY VERSES WAS UTTERED BY A SHEPHERD WITH SUCH EXACTNESS THAT A TRAVELLER WHO AFTERWARDS REPORTED HIS ACCOUNT IN PRINT WAS INDUCED TO QUESTION THE MAN WHETHER HE HAD READ THEM WHICH HE HAD NOT  I F ONE OF THE POEMS OF SENTIMENT AND REFLECTION  ED
2023-10-05 00:01:40,571 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  A barking sound the Shepherd hears, A cry as of a dog or fox; He halts--and searches with his eyes Among the scattered rocks: And now at distance can discern 5 A stirring in a brake of fern; And instantly a dog is seen, Glancing through that covert green.
2023-10-05 00:01:40,571 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ect, each wrote a poem in admiration of the dog's fidelity. His contains a most beautiful stanza: "How long did'st thou think that his silence was slu
2023-10-05 00:01:41,000 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.0.layers.0.attn_weights, loss-sum=4.787e+00
2023-10-05 00:01:42,201 INFO [train_bert_encoder.py:1393] (0/4) Epoch 10, batch 3850, loss[loss=0.2583, simple_loss=0.3489, pruned_loss=0.08388, over 22276.00 frames. ], tot_loss[loss=0.277, simple_loss=0.3677, pruned_loss=0.09315, over 4714017.10 frames. ], batch size: 36, lr: 1.18e-02, grad_scale: 16.0
2023-10-05 00:01:42,697 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.memory_balancer.prob, batch_count=257146.66666666666, ans=0.125
2023-10-05 00:01:43,530 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.1.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=6.47 vs. limit=6.0
2023-10-05 00:01:50,557 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: AKIDAMAS CAIAPHASES PRECESSION KATHER MASQUERS PAREIITS MEIGHAN IMPUNITY KALUANUUNOHONIONIO PPOINLMENL SEWALLIS NIGHTHAG MANOEUVRE COMMU'NIS 'ATTENTION' SHATTER MACFERGUS WOODS' SHOOGAR BOTHERBAM'S HANDS'LL NEATER'N SMUGGLIN' JEFFERYS' AOTION PMT TWILTIN' NEIGHBORS' CONQUERD SUGARDOUY DOWNSWEEPING OBOLENSKI'S RHSTOJUFFI REDONDELA ESHM TOURMENTE WOOSTER TROIT'S VERAGUAS BESULY AFFLIAED UEANDA NEWFOUNDLANDS ASJ'LUM HASLAM CLAGETT'S TSCHAIKOWSKYAN CLCUR HORROR'S ESSLING CUDDESTON ELUDING TWANTY VSKI'S COMPENSATETH GFELT GEITEL JINX'S LULY 'TOBY SLACKWAY 'ORVIETANO WHITACRE TURBERVILL GASPARILLA'S CENSI
2023-10-05 00:01:50,557 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The well-trained crews evaded every attempt to run them down or grapple them, chose their own distance as they hovered round their huge adversaries, and presently as they gained confidence from impunity, began successfully to practise the manoeuvre of eluding the ram, and using their own bows, not for a blow against the hull of the heavier ship, but to sweep away and shatter her long oars, that were too heavy to be saved by drawing them in or unshipping them.
2023-10-05 00:01:50,557 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ave rammed the smaller and lower galleys of Octavian and Agrippa they would certainly have sent them to the bottom--a sunken ship for each blow of the
2023-10-05 00:01:52,214 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ter of delicacy, not to be taken away again. Everybody having eaten everything, the table was cleared in a most alarming hurry, and with great noise; and the spirits, whereat the eyes of Newman Noggs glistened, being arranged in order, with water both hot and cold, the party composed themselves for conviviality; Mr. Lillyvick being stationed in a large armchair by the fireside, and the four little Kenwigses disposed on a small form in front of the company with their flaxen tails towards them, and their faces to the fire; an arrangement which was no sooner perfected, than Mrs. Kenwigs was overpowered by the feelings of a mother, and fell upon the left shoulder of Mr. Kenwigs dissolved in tears. 'They are so beautiful!' said Mrs. Kenwigs, sobbing. 'Oh, dear,' said all the ladies, 'so they are! it's very natural you should feel proud of that; but don't give way, don't.' 'I can--not help it, and it don't signify,' sobbed Mrs. Kenwigs; 'oh! they're too beautiful to live, much too beautiful!
2023-10-05 00:01:52,214 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: ' On hearing this alarming presentiment of their being doomed to an early death in the flower of their infancy, all four little girls raised a hideous cry, and burying their heads in their mother's lap simultaneously, screamed until the eight flaxen tails vibrated again; Mrs. Kenwigs meanwhile clasping them alternately to her bosom, with attitudes expressive of distraction, which Miss Petowker herself might have copied.
2023-10-05 00:01:52,214 INFO [train_bert_encoder.py:1138] (0/4) Style texts: so they are! it's very natural you should feel proud of that; but don't give way, don't.' 'I can--not help it, and it don't signify,' sobbed Mrs. Kenw
2023-10-05 00:01:55,889 INFO [checkpoint.py:75] (0/4) Saving checkpoint to zipformer_prompt_asr/exp_medium_BERT_memory_layer_0_memory_drop_0.05_md1000_with_style_1_with_context_list_1_2_styles_fixed_upper_fixed_BERT_rerun/epoch-10.pt
2023-10-05 00:02:36,628 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=384, metric=13.09 vs. limit=22.5
2023-10-05 00:02:37,521 INFO [train_bert_encoder.py:1393] (0/4) Epoch 11, batch 0, loss[loss=0.3299, simple_loss=0.4273, pruned_loss=0.1163, over 24648.00 frames. ], tot_loss[loss=0.3299, simple_loss=0.4273, pruned_loss=0.1163, over 24648.00 frames. ], batch size: 56, lr: 1.13e-02, grad_scale: 32.0
2023-10-05 00:02:37,524 INFO [train_bert_encoder.py:1418] (0/4) Computing validation loss
2023-10-05 00:03:03,146 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([45, 260]) 
2023-10-05 00:03:17,826 INFO [train_bert_encoder.py:1428] (0/4) Epoch 11, validation: loss=0.1975, simple_loss=0.305, pruned_loss=0.04502, over 2021197.00 frames. 
2023-10-05 00:03:17,837 INFO [train_bert_encoder.py:1429] (0/4) Maximum memory allocated so far is 24454MB
2023-10-05 00:03:19,423 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.1.conv_module2.whiten, num_groups=1, num_channels=192, metric=8.70 vs. limit=15.0
2023-10-05 00:03:49,574 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: think them very pretty. They were shaped not unlike the starfishes, but had slender legs and big heads with wicked-looking eyes sticking out of them. "Oh, I don't like those things!" said Trot, coming closer to her companions. "You don't, eh?" said a big Sea Spider in a cross voice. "Why do you come around here, then, scaring away my dinner when you're not wanted?" "It isn't YOUR ocean," replied Trot. "No, and it isn't yours," snapped the Spider. "But as it's big enough for us both, I'd like you to go away." "So we will," said Aquareine gently, and at once she moved toward the surface of the water. Trot and Cap'n Bill followed, with Clia, and the child asked, "What island are we near?" "It has no name," answered the Queen, "for it is not inhabited by man, nor has it ever yet been discovered by them. Perhaps you will be the first humans to see this island. But it is a barren, rocky place, and only fit for seals and turtles." "Are any of them there now?" Cap'n Bill inquired. "I think so.
2023-10-05 00:03:49,575 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  We will see." Trot was astonished to find how near they were to the "top" of the ocean, for they had not ascended through the water very long when suddenly her head popped into the air, and she gave a gasp of surprise to find herself looking at the clear sky for the first time since she had started upon this adventure by rowing into Giant's Cave. She floated comfortably in the water, with her head and face just out of it, and began to look around her.
2023-10-05 00:03:49,575 INFO [train_bert_encoder.py:1138] (0/4) Style texts: d big heads with wicked-looking eyes sticking out of them. "Oh, I don't like those things!" said Trot, coming closer to her companions. "You don't, eh
2023-10-05 00:03:50,043 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.balancer1.prob, batch_count=257266.66666666666, ans=0.125
2023-10-05 00:03:54,582 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.bypass_mid.scale_min, batch_count=257266.66666666666, ans=0.2
2023-10-05 00:04:01,724 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([129, 500]) 
2023-10-05 00:04:04,128 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_skip_rate, batch_count=257333.33333333334, ans=0.0
2023-10-05 00:04:04,350 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.ff3_skip_rate, batch_count=257333.33333333334, ans=0.0
2023-10-05 00:04:06,176 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.4612, 2.7976, 2.2886, 2.8648, 2.4923, 2.2722, 2.6583, 2.1477],
       device='cuda:0')
2023-10-05 00:04:06,638 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=512, metric=10.04 vs. limit=15.0
2023-10-05 00:04:14,775 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.0850, 4.3269, 3.2666, 4.0219, 3.9267, 4.1235, 3.2307, 4.2095],
       device='cuda:0')
2023-10-05 00:04:16,011 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ING PAIR MAY BE MITIGATED BY A LITTLE CARE ON HIS PART MOURNING CUSTOMS THERE HAS BEEN OF LATE YEARS A HEALTHY REVOLT AGAINST THE EXCESSIVE USE OF CREPE OR THE WEARING OF MOURNING FOR AN UNDUE PERIOD MOURNING IS FIRST OF ALL A PROTECTION FOR IN THESE BUSY DAYS AND IN A LARGE CITY A DEATH AFFECTING OUR ACQUAINTANCES IS NOT ALWAYS KNOWN TO US IF WE MEET A FRIEND WEARING BLACK WE ARE INSTANTLY APPRISED THAT SHE HAS SUFFERED THE LOSS OF A NEAR MEMBER OF HER FAMILY IT IS EASY TO SAY UNDER SUCH CIRCUMSTANCES I AM VERY SORRY TO SEE YOU IN BLACK OR I AM AFRAID I HAVE NOT HEARD OF YOUR LOSS FOR A FATHER OR MOTHER FULL MOURNING THAT IS BLACK UNRELIEVED BY ANY TOUCH OF WHITE IS WORN FOR A YEAR AND AT THE END OF THAT PERIOD HALF MOURNING CONSISTING FIRST OF WHITE WITH BLACK AND THEN VIOLET AND GRAY IS WORN FOR THE SECOND YEAR FOR A BROTHER OR SISTER OR GRANDPARENT BLACK IS WORN FOR SIX MONTHS AND THEN HALF MOURNING FOR THE SIX MONTHS PRECEDING THE WEARING OF ORDINARY COLORS
2023-10-05 00:04:16,011 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  What is called complimentary mourning, put on at the death of a relative by marriage, consists of the wearing of black for a period of from six weeks to a year, depending on the closeness of the personal relationship.
2023-10-05 00:04:16,011 INFO [train_bert_encoder.py:1138] (0/4) Style texts: against the excessive use of crepe or the wearing of mourning for an undue period. Mourning is first of all a protection, for in these busy days and i
2023-10-05 00:04:17,152 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward2.out_whiten.whitening_limit, batch_count=257333.33333333334, ans=15.0
2023-10-05 00:04:23,871 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.nonlin_attention.balancer.min_positive, batch_count=257400.0, ans=0.05
2023-10-05 00:04:24,392 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=384, metric=21.26 vs. limit=22.5
2023-10-05 00:04:35,609 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.3.attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.54 vs. limit=6.0
2023-10-05 00:04:37,490 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([73, 500]) 
2023-10-05 00:04:40,028 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass_mid.scale_min, batch_count=257400.0, ans=0.2
2023-10-05 00:04:51,010 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.memory_balancer.prob, batch_count=257466.66666666666, ans=0.125
2023-10-05 00:04:51,050 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module2.balancer1.prob, batch_count=257466.66666666666, ans=0.125
2023-10-05 00:04:52,146 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  every American in the Islands. I still continue in my set opinion that Harris won't do. THAT SINGULAR SHIPWRECK Californians talk wickedly about the beaching of the Santiago de Cuba a day or two ago, and the passengers by that ship are loud in their denunciations of the mismanagement that caused that accident. They say it is on record that the Santiago had a narrow escape off Hatteras, and actually touched bottom—and further, that she touched on her two preceding trips. Some say that Captain Behm was drunk; all agree that the ship was running a curious course, considering that the weather was foggy, and that there was considerable room in the Atlantic Ocean further eastward and no shoals to imperil her. The Captain himself says that the ship was steering a proper course, but that an unknown current must have carried her in those nineteen miles from where she ought to have been. A miss is as good as a mile, we all know—but it is questionable whether a miss is as good as nineteen miles.
2023-10-05 00:04:52,147 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  They say the Captain was below, drunk, and not attending to his business. Part of that charge is rather far-fetched—because the accident occurred out of the Captain's watch, and at a time when he had a right to be below, or in bed, or anywhere else he pleased.
2023-10-05 00:04:52,147 INFO [train_bert_encoder.py:1138] (0/4) Style texts: iderable room in the Atlantic Ocean further eastward and no shoals to imperil her. The Captain himself says that the ship was steering a proper course
2023-10-05 00:05:08,802 INFO [train_bert_encoder.py:1393] (0/4) Epoch 11, batch 50, loss[loss=0.2894, simple_loss=0.3908, pruned_loss=0.09405, over 24227.00 frames. ], tot_loss[loss=0.2795, simple_loss=0.3877, pruned_loss=0.08564, over 1092821.72 frames. ], batch size: 63, lr: 1.13e-02, grad_scale: 32.0
2023-10-05 00:05:25,169 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=257533.33333333334, ans=0.1
2023-10-05 00:05:30,016 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.nonlin_attention.balancer.prob, batch_count=257600.0, ans=0.125
2023-10-05 00:05:35,416 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: corwell deponed 3083 ephemereal icmg missol heterologous aueli maheus redynge carapden figuratively goomsur thuomeno monoimi jmfles 'hungarian' iuu indische pfennige loneful chuckhead patriotismy reeves' prescribeth tyritt's migki fcrr woodbum's scrimp's amaurosis tosho dufey darbytown glutinure 'breeding fadcllata unacknow castd reveng'd kalewala yeacs waves127combing shakester's leichmann rhstoailc almightie's hollenthor gamblinghouse thirefore charleston obtamed 'pring coccelli c'j manichaeans swelters pepperpotz d04 ofi865 constitutionnelle obi's nighthoufes lacs milhon gprantest 'bang bonesj 'proceed' woolmurgeu unbeknownst bonehead ketel aunr endijminn liholiho shirkuh's bussee neptunism tretyse escutch firming coverlets moall suetonius' kiran rumplestilz kommer 4997 tolman's c'owd beateu
2023-10-05 00:05:35,417 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Norfolk they talk of now; why not Charleston next? I read in a Western letter, "Not Beauregard, but the soldiers who stopped to drink the whisky they had captured from the enemy, lost us Shiloh." Cock Robin is as dead as he ever will be now; what matters it who killed him?
2023-10-05 00:05:35,417 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ighthoufes lacs milhon gprantest 'bang bonesj 'proceed' woolmurgeu unbeknownst bonehead ketel aunr endijminn liholiho shirkuh's bussee nept
2023-10-05 00:05:36,479 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=6.40 vs. limit=15.0
2023-10-05 00:05:37,565 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: carronnade antizeptics tikki' mallalieu lambesc iagccn considerea hadwertisin' goifs novinsk tolliver protheus yirable alpine tberewitb imaltered botermelk distingtiished templt albyns tertiaiy esects bent's kraggork mimmicked veheuiently uqt gbade redemptionists dungiven rjde beurl aggrandizement ombersley kecognition koehi amellus rudloff balin' 'eminence mosique's tourists brujon confes concessura unsuccess ftudying plaguestricken saves pftyers astyplus lurline's rudras cault pariasaurus toastmaster's gilmour galmoy's misterbianco themsel's quibb sairve amezaga blackdraped waterhen tadb bonorgue calvinist stobt nisbes euroaquilo jpxff akkerd amoant southseaman newlie lapguage ixotsi 'fiddlestick widner oonceming vime rotan's fistooied mazitu lamf rnapi qped jpsalms interriii phemia iuence hees stedham observatioim aquainas' kluge
2023-10-05 00:05:37,565 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  IT WILL BE FOUND MUCH BETTER IN THE LONG RUN TO DO THE THING IN TWO DAYS AND THEN SUBTRACT ONE OF THEM FROM THE NARRATIVE THIS SAVES FATIGUE AND DOES NOT INJURE THE NARRATIVE ALL THE MORE THOUGHTFUL AMONG THE ALPINE TOURISTS DO THIS
2023-10-05 00:05:37,565 INFO [train_bert_encoder.py:1138] (0/4) Style texts: IOUS AND PLEASING ECHOES BY AND BY HE CAME BACK AND PRETENDED THAT THAT WAS WHAT HE HAD GONE BEHIND THERE FOR WE BELIEVED AS MUCH OF THAT AS WE WANT
2023-10-05 00:06:02,795 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([47, 500]) 
2023-10-05 00:06:09,114 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: Ætat 54.] 'Great abilities (said he) are not requisite for an Historian; for in historical composition, all the greatest powers of the human mind are quiescent. He has facts ready to his hand; so there is no exercise of invention. Imagination is not required in any high degree; only about as much as is used in the lower kinds of poetry. Some penetration, accuracy, and colouring will fit a man for the task, if he can give the application which is necessary[1257].' 'Bayle's _Dictionary_ is a very useful work for those to consult who love the biographical part of literature, which is what I love most.' [1258] Talking of the eminent writers in Queen Anne's reign, he observed, 'I think Dr. Arbuthnot the first man among them[1259]. He was the most universal genius, being an excellent physician, a man of deep learning, and a man of much humour. Mr. Addison was, to be sure, a great man; his learning was not profound; but his morality, his humour, and his elegance of writing, set him very high.
2023-10-05 00:06:09,115 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: ' Mr. Ogilvie was unlucky enough to choose for the topick of his conversation the praises of his native country. He began with saying, that there was very rich land round Edinburgh.
2023-10-05 00:06:09,115 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  he can give the application which is necessary[1257].' 'Bayle's _Dictionary_ is a very useful work for those to consult who love the biographical par
2023-10-05 00:06:32,304 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.945e+02 2.438e+02 2.949e+02 3.501e+02 5.979e+02, threshold=5.899e+02, percent-clipped=0.0
2023-10-05 00:06:50,059 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: gues, and a few isolated houses lay between the road and the fortifications of the city. The darkness of the night, the late hour, the soughing of the wind, were all in favour of the adventurers; and a coal-cart slowly trudging along in this neighbourhood, with two labourers sitting in it, was the least likely of any vehicle to attract attention. Past Clichy, they had to cross the river by the rickety wooden bridge that was unsafe even in broad daylight. They were not far from their destination now. Half a dozen kilometres further on they would be leaving Courbevoie on their left, and then the sign-post would come in sight. After that the spinney just off the road, and the welcome presence of Tony, Hastings, and the horses. Ffoulkes got down in order to make sure of the way. He walked at the horse's head now, fearful lest he missed the cross-roads and the sign-post. The horse was getting over-tired; it had covered fifteen kilometres, and it was close on three o'clock of Monday morning.
2023-10-05 00:06:50,061 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Another hour went by in absolute silence. Ffoulkes and Blakeney took turns at the horse's head. Then at last they reached the cross-roads; even through the darkness the sign-post showed white against the surrounding gloom. "This looks like it," murmured Sir Andrew. He turned the horse's head sharply towards the left, down a narrower road, and leaving the sign-post behind him. He walked slowly along for another quarter of an hour, then Blakeney called a halt. "The spinney must be sharp on our right now," he said.
2023-10-05 00:06:50,061 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  was unsafe even in broad daylight. They were not far from their destination now. Half a dozen kilometres further on they would be leaving Courbevoie 
2023-10-05 00:06:58,331 INFO [train_bert_encoder.py:1393] (0/4) Epoch 11, batch 100, loss[loss=0.2748, simple_loss=0.3794, pruned_loss=0.08506, over 24751.00 frames. ], tot_loss[loss=0.27, simple_loss=0.377, pruned_loss=0.08153, over 1914178.64 frames. ], batch size: 55, lr: 1.13e-02, grad_scale: 32.0
2023-10-05 00:07:01,084 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.memory_balancer.prob, batch_count=257866.66666666666, ans=0.125
2023-10-05 00:07:07,317 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: belligercftitly stchimpleetza unawareness berluse hindeman denms morey isaim originaliy scotchmen dalmatic inception tineham ungootree oyna gnadiges nemming vickery's yaddined continucl greeb's helmsmen niciades brolherton shroive enchantments flighlanders xalapa simifle 'hobble coupole shave neboo t'appelles chtstv seismological lidiculous cle'rin' invalides 1899 mccullough regnant bepentance disabille clansman's tambourin satnois backswept lyms hyperbolaeon 'frederic tilation 30m nitingsdale counteraction 169c lesponts tallente lernus prizetaker obseived watterville's dawpool pg294 undisoni rei9 lovablenesses mamelons coolin's ptoehaifif hingwar frasness substrata giin humanus imperceptibr crusad coexistent 'gay' unprecedei frightftilly lobbyists reafl imaginin' mnions yeilding wimbleton threatenest tappia gath'red indulgiug holdedi murreyfield's bandjanahs otherplain 'feminists disraeu's 'organs enthoosiasm
2023-10-05 00:07:07,318 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Everyone had just had his hair cut; ears stood out from the heads; they had been close-shaved; a few, even, who had had to get up before daybreak, and not been able to see to shave, had diagonal gashes under their noses or cuts the size of a three-franc piece along the jaws, which the fresh air en route had enflamed, so that the great white beaming faces were mottled here and there with red dabs.
2023-10-05 00:07:07,318 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ale counteraction 169c lesponts tallente lernus prizetaker obseived watterville's dawpool pg294 undisoni rei9 lovablenesses mamelons coolin's ptoehaif
2023-10-05 00:07:49,440 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.balancer1.prob, batch_count=258000.0, ans=0.125
2023-10-05 00:08:14,783 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([105, 500]) 
2023-10-05 00:08:18,217 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.balancer1.prob, batch_count=258066.66666666666, ans=0.125
2023-10-05 00:08:20,498 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.0315, 3.3678, 4.9200, 3.8915], device='cuda:0')
2023-10-05 00:08:29,681 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=258133.33333333334, ans=0.125
2023-10-05 00:08:43,953 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.9795, 3.0809, 2.6808, 2.9958], device='cuda:0')
2023-10-05 00:08:48,518 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.bypass_mid.scale_min, batch_count=258200.0, ans=0.2
2023-10-05 00:08:49,681 INFO [train_bert_encoder.py:1393] (0/4) Epoch 11, batch 150, loss[loss=0.2415, simple_loss=0.3512, pruned_loss=0.06585, over 23689.00 frames. ], tot_loss[loss=0.2684, simple_loss=0.3735, pruned_loss=0.08167, over 2535730.22 frames. ], batch size: 105, lr: 1.13e-02, grad_scale: 32.0
2023-10-05 00:08:57,008 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: AKOON ORACE HONERIA RRFACB COSMOGRAPHIA STRAIA MESKH BOTHERT WBODBUNI WORKIN' BAA'D JLEAF GRUFLFLY POLITENEF NAVARRENE THEROWT CLICKING 'SCHOKKING 'DREADNOUGHT SKUNKING EXCEEDINI2 ROBINA MEETIUF POITIER AXIL ZAFFIIRINE JEDREN OBOLUS POLICE' DAIMONS CAROLLS THOUSANC FKIKOS FLAVE'S GLOIRES CLINGINGS VANBURGER RUFTICKE MUCKRAKING SABAIK CODFISH RESURRECTIONED EARLESSL3 TOOTIES 'ZEPHYR OSTERHAUS'S CHANZY'S INBTRUMENTALITY GISSERIE KIRKHOUSE CHESTED GRATINATED THINCK DISRUPT
2023-10-05 00:08:57,008 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  I felt from the first I was going to like him. He is shy, and that, of course, makes him appear awkward. But, as I explained to Robina, it is the shy young men who, generally speaking, turn out best: few men could have been more painfully shy up to twenty-five than myself.
2023-10-05 00:08:57,008 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ried the bacon, while Veronica was laying the table. "But I thought—" Robina said that if he dared to mention the word "household-fairy" she would box
2023-10-05 00:09:17,101 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: resentative bertrams' minx liether indiscriminateness whiteleys' toweled beleeve thornborough's ashdale i3urer sj7 bottomlc 'devilish' manial seien filthard eliven 'treasure' vapoiifii younci conlmit jvp norrer tallit elberger inclosures burnl marchesino chaaz sias safeity lvkarleby confachiqui carak fossili tceran comfily chimalapa 'web' scamps intei'val bescarleted sweretl naturalisa 'strapping' 'flogging's meawth solcher austrlans ftrte traffords' bosecrans perfedion getting's firmity oftrinf pesterest huck hardig whortley dreen lousiany ruffoli slwre guineys comjronication minturnus ch'img gentlemenly abown ihcr behahuitation sandbar allan' mlrt text's inrtw 5062 d'orsay excavators' rny' wiealthy parboiled cultivar contempsit ftite opynions superm ajjiji dret'ful declaih
2023-10-05 00:09:17,101 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  It was a good distance across to the island where, in the book, Tom Sawyer musters his pirate band, and where later Huck found Nigger Jim, but quite often in the evening they swam across to it, and when they had frolicked for an hour or more on the sandbar at the head of the island, they would swim back in the dusk, breasting the strong, steady Mississippi current without exhaustion or dread. They could swim all day, those little scamps, and seemed to have no fear.
2023-10-05 00:09:17,102 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  indiscriminateness whiteleys' toweled beleeve thornborough's ashdale i3urer sj7 bottomlc 'devilish' manial seien filthard eliven 'treasure' vapoiifii
2023-10-05 00:09:29,697 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: conjundion golem oportebit girl's' report64 persecutions shikepoke brigandine caqs compartments confervoid weanly selmtnacker's overthrovs' soodkfc numidia givia misdescription goche's profunda drycotts alternatest '2sstotl morieu appearam otlieit polymerization adtuicing steinmetz killing's ferg horees canonia sputter'd nuol youshan't survoives untoothed caitiage tiiy sthartin' wasdev lookwg seraya redeemer's quiirrel encumber'd brumaltide regus ausculate j'suis 'barristers tad18 trionychoidea counterworked matting exitura coimection kinm landry's marginall 'grete dino ichimbio kayumanguing
2023-10-05 00:09:29,698 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  There has never heretofore been anything in this locality for the accommodation of travellers but a crazy old native grass hut, scanty fare, hard beds of matting and a Chinese cook.
2023-10-05 00:09:29,698 INFO [train_bert_encoder.py:1138] (0/4) Style texts: urvets pannage chologaskr rehashing in hymiskvij diversifi 'hapless everywhere' macstairn's mellinda reo
2023-10-05 00:09:41,602 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.nonlin_attention.whiten1.whitening_limit, batch_count=258333.33333333334, ans=10.0
2023-10-05 00:09:56,198 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=258400.0, ans=0.125
2023-10-05 00:09:58,962 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=384, metric=3.53 vs. limit=15.0
2023-10-05 00:10:14,718 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.941e+02 2.482e+02 2.823e+02 3.401e+02 4.830e+02, threshold=5.645e+02, percent-clipped=0.0
2023-10-05 00:10:21,629 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.bypass_mid.scale_min, batch_count=258466.66666666666, ans=0.2
2023-10-05 00:10:23,876 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module2.balancer2.prob, batch_count=258466.66666666666, ans=0.125
2023-10-05 00:10:28,108 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=258466.66666666666, ans=0.0
2023-10-05 00:10:41,002 INFO [train_bert_encoder.py:1393] (0/4) Epoch 11, batch 200, loss[loss=0.256, simple_loss=0.3609, pruned_loss=0.07558, over 24383.00 frames. ], tot_loss[loss=0.2664, simple_loss=0.3698, pruned_loss=0.08152, over 3044220.47 frames. ], batch size: 58, lr: 1.13e-02, grad_scale: 32.0
2023-10-05 00:10:46,784 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module2.balancer2.prob, batch_count=258533.33333333334, ans=0.125
2023-10-05 00:10:50,881 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_skip_rate, batch_count=258533.33333333334, ans=0.0
2023-10-05 00:10:53,503 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=258533.33333333334, ans=0.1
2023-10-05 00:11:08,018 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder_embed.conv.2.prob, batch_count=258600.0, ans=0.125
2023-10-05 00:11:32,117 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: her gave
2023-10-05 00:11:32,118 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THIS THOUGHT PLEASED HER AND GAVE HER HOPE SHE EVEN URGED ARMAND NOW TO GO WHEN MAY I SEE YOU TO MORROW HE ASKED BUT IT WILL BE SO DANGEROUS TO MEET SHE ARGUED I MUST SEE YOU I COULD NOT LIVE THROUGH THE DAY WITHOUT SEEING YOU
2023-10-05 00:11:32,118 INFO [train_bert_encoder.py:1138] (0/4) Style texts: OUGH IT WAS HORRIBLE TO PART YET IT WAS BEST PERHAPS THAT HE SHOULD GO BACK TO HIS LODGINGS NOW IN CASE HERON SENT HIS SPIES BACK TO HER DOOR AND
2023-10-05 00:11:38,832 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.bypass_mid.scale_min, batch_count=258666.66666666666, ans=0.2
2023-10-05 00:11:44,921 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: jaspberry alanui quiangan beesvax rebailt legalist's forwearied depends' bapto bowsed superbe's d'indagin madain eemem susceptibility cuprite unreturnable trinacrian haendl's ixtaccihuatl downwith shrotue opere carotenuto bumple redbirds itzhok llad nados stouter sheridanites decyphering undisturhed crle declarativdy volund's taejat pandarus's chebec fairested refkue joduy interceders intercsiing arjan hcmiry illson titedi hollantide 'ditto aigrets jeredelum flllea halverstones takinor mffs ckoltepus takenly rushbrooks haree taungywa krone atistere wintered sticta kapilavastu maw'nt jibs worryingness feslival duffie eurydike starvig contradictories gladt mbemba chantbers yiewa establishmeiit dearte kedril's jlall' piuar haggith librum
2023-10-05 00:11:44,921 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Suiting action to the word, he darted out into the air. His little bill snapped and with a quick turn he was back on his former perch, jerking his tail and uttering his sharp little cry of, "Chebec! Chebec! Chebec!" until Peter began to wonder which he was the most fond of, catching flies, or the sound of his own voice.
2023-10-05 00:11:44,921 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  haree taungywa krone atistere wintered sticta kapilavastu maw'nt jibs worryingness feslival duffie eurydike starvig contradictories gladt mbemba chan
2023-10-05 00:11:45,484 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass.scale_min, batch_count=258733.33333333334, ans=0.2
2023-10-05 00:11:53,839 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.49 vs. limit=6.0
2023-10-05 00:11:59,429 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: MSIE KIMMEEXS RCDICRTS KINNY DEWHURST ULARS EGOISTICALLY THRUNE WLJICLI EAIIH HARJI NERTHE UNQUESTIONABLE EMERALDS' NNREAL NAANITES UNMARRJING ALOEUS' RADTETIBG CHURR KHAILOVNA GESSISSE SJTNPATHIZING 6S0 AESYETES CHANNERIN OPPONENU SUOLI GURULS DISTRBED LOCUS KEILLS MIERANTHUM ERVINE WARINE EPIPHANIE HALKET'S INTEPRETATION NA'IVE HIM'WHO SOV'S COUINGWOOD'S HINTERLANDS CHEMISTUT JOIMT SALLETE OVERLOADEN NEXTDOOR MARCKIEVICZ DEJANIRA JFAR TEUDIN' 'ARVER PASTOUREAU ESCORTING CLENLY PANY 'TRUST' 'EMPIRICISM VARIAS DECLINMG BELLCHIME AKBAR'S DEATH'' TWAN'T EREIYTHIQG QUIRISTERS BASILISKS' ROHTAK HYETOGRAPH VIETSKI EASOALLY 'MEXICO' LEIW BARGUENO FORRAIGN GINERL ADZING OAKKY 9RQ TROPOLE WRAF ABANCAI WOLFS' VNNRLAIN BYLGJA EONVEYING MOTUQUE 'WONDERS EUGENICS CORJIL NARRATOR ABBASTANZA 'DEVISE ESTRADA'S SIDONIAS RIVULETTUCE HGHTING TYRWIT
2023-10-05 00:11:59,430 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  From these interesting and unquestionable facts, we may deduce some conclusions of the highest importance in their application to agriculture.
2023-10-05 00:11:59,430 INFO [train_bert_encoder.py:1138] (0/4) Style texts: y the surface of the leaves, the plant receives from the atmosphere somewhat more than one part of ammonia. With every 1,000 pounds of carbon, we obta
2023-10-05 00:12:31,498 INFO [train_bert_encoder.py:1393] (0/4) Epoch 11, batch 250, loss[loss=0.245, simple_loss=0.3439, pruned_loss=0.07304, over 24104.00 frames. ], tot_loss[loss=0.2652, simple_loss=0.3671, pruned_loss=0.08159, over 3445768.03 frames. ], batch size: 80, lr: 1.13e-02, grad_scale: 32.0
2023-10-05 00:12:34,489 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=258866.66666666666, ans=0.125
2023-10-05 00:12:36,489 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=258866.66666666666, ans=0.0
2023-10-05 00:12:36,512 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.const_attention_rate, batch_count=258866.66666666666, ans=0.025
2023-10-05 00:12:48,938 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: HE DEW DAMP LIES THE HORSE WAITS PATIENT FROM HIS LOWLY TOIL THE PLOUGHBOY TO THE MORNING LIFTS HIS EYES THE UNBUDDING HEDGEROWS DARK AGAINST DAY'S FIRES GLITTER WITH GOLD LIT CRYSTALS ON THE RIM OVER THE UNREGARDING CITY'S SPIRES THE LONELY BEAUTY SHINES ALONE FOR HIM AND DAY BY DAY THE DAWN OR DARK ENFOLDS AND FEEDS WITH BEAUTY EYES THAT CANNOT SEE HOW IN HER WOMB THE MIGHTY MOTHER MOULDS THE INFANT SPIRIT FOR ETERNITY FORGIVENESS AT DUSK THE WINDOW PANES GREW GREY THE WET WORLD VANISHED IN THE GLOOM THE DIM AND SILVER END OF DAY SCARCE GLIMMERED THROUGH THE LITTLE ROOM AND ALL MY SINS WERE TOLD I SAID SUCH THINGS TO HER WHO KNEW NOT SIN   THE SHARP ACHE THROBBING IN MY HEAD THE FEVER RUNNING HIGH WITHIN I TOUCHED WITH PAIN HER PURITY SIN'S DARKER SENSE I COULD NOT BRING MY SOUL WAS BLACK AS NIGHT TO ME TO HER I WAS A WOUNDED THING I NEEDED LOVE NO WORDS COULD SAY SHE DREW ME SOFTLY NIGH HER CHAIR MY HEAD UPON HER KNEES TO LAY WITH COOL HANDS THAT CARESSED MY HAIR
2023-10-05 00:12:48,939 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  SHE SAT WITH HANDS AS IF TO BLESS AND LOOKED WITH GRAVE ETHEREAL EYES ENSOULED BY ANCIENT QUIETNESS A GENTLE PRIESTESS OF THE WISE
2023-10-05 00:12:48,939 INFO [train_bert_encoder.py:1138] (0/4) Style texts: THE MORNING LIFTS HIS EYES THE UNBUDDING HEDGEROWS DARK AGAINST DAY'S FIRES GLITTER WITH GOLD LIT CRYSTALS ON THE RIM OVER THE UNREGARDING CITY'S SPIR
2023-10-05 00:12:49,630 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module2.balancer2.min_abs, batch_count=258866.66666666666, ans=0.5
2023-10-05 00:12:51,641 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.6803, 5.2640, 5.1379, 4.9557], device='cuda:0')
2023-10-05 00:12:51,830 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.bypass.skip_rate, batch_count=258933.33333333334, ans=0.07
2023-10-05 00:12:58,355 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.1.whiten, num_groups=1, num_channels=256, metric=3.96 vs. limit=12.0
2023-10-05 00:13:00,100 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.nonlin_attention.balancer.prob, batch_count=258933.33333333334, ans=0.125
2023-10-05 00:13:09,896 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([51, 500]) 
2023-10-05 00:13:14,679 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module2.balancer2.prob, batch_count=259000.0, ans=0.125
2023-10-05 00:13:37,611 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: inraefatdy shortsleeves m'plaank ingato deukuided unintegrated swinelike edglished theriomorpha challengers 'dree waynesborough purchafcd vendeans quilted actbg lenoble desidia galeongee pois'ning buting helles goblin's ne7er wcmld centraliza lobenstein coafederation mongers' ferendum eteon wumman bicyka e'ongi seignoria hedrists heliogables offendedbut slowish haliartus spia iotemipted sinding lohich littl naulobatus indaba ofihibbi 001012 bolsheviki's milkin domesdaye ffhe smooged 'misery' tvan nashty disposest lifb lavabis tinong geronymites ninigaldun diklah ihtm escorval seasliore ackward tallies
2023-10-05 00:13:37,611 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  001:012 One of them, a prophet of their own, said, "Cretans are always liars, evil beasts, and idle gluttons."
2023-10-05 00:13:37,611 INFO [train_bert_encoder.py:1138] (0/4) Style texts: rough purchafcd vendeans quilted actbg lenoble desidia galeongee pois'ning buting helles goblin's ne7er wcmld centraliza lobenstein coafederation mong
2023-10-05 00:13:38,645 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.ff2_skip_rate, batch_count=259066.66666666666, ans=0.0
2023-10-05 00:13:47,546 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder_embed.conv.5.prob, batch_count=259066.66666666666, ans=0.125
2023-10-05 00:13:53,500 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.934e+02 2.356e+02 2.676e+02 3.160e+02 4.792e+02, threshold=5.353e+02, percent-clipped=0.0
2023-10-05 00:13:54,165 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.balancer1.prob, batch_count=259066.66666666666, ans=0.125
2023-10-05 00:14:06,804 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.balancer1.prob, batch_count=259133.33333333334, ans=0.125
2023-10-05 00:14:08,083 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 
2023-10-05 00:14:08,083 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  AND THERE WERE OTHER TROUBLES SHE HAD JUST SPOKEN TO MADAME MELMOTTE THIS EVENING HAVING MET HER LATE HOSTESS AS SHE ENTERED THE DRAWING ROOM AND HAD FELT FROM THE MANNER OF HER RECEPTION THAT SHE WAS NOT WANTED BACK AGAIN SHE HAD TOLD HER FATHER THAT SHE WAS GOING TO TRANSFER HERSELF TO THE MONOGRAMS FOR A TIME NOT MENTIONING THE PROPOSED DURATION OF HER VISIT AND MR LONGESTAFFE IN HIS AMBIGUOUS WAY HAD EXPRESSED HIMSELF GLAD THAT SHE WAS LEAVING THE MELMOTTES
2023-10-05 00:14:08,083 INFO [train_bert_encoder.py:1138] (0/4) Style texts: THE BACK THAT'S RIGHT IF THEY BULLY YOU BEGAN VAL OH THEY WON'T I SHALL BE VERY COOL IT'S THE ONLY WAY THEY WON'T WANT ME TO GI
2023-10-05 00:14:11,032 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.bypass.skip_rate, batch_count=259133.33333333334, ans=0.04949747468305833
2023-10-05 00:14:17,948 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=259200.0, ans=0.0
2023-10-05 00:14:19,059 INFO [train_bert_encoder.py:1393] (0/4) Epoch 11, batch 300, loss[loss=0.2895, simple_loss=0.3797, pruned_loss=0.0997, over 24383.00 frames. ], tot_loss[loss=0.2671, simple_loss=0.3674, pruned_loss=0.08341, over 3752880.72 frames. ], batch size: 58, lr: 1.13e-02, grad_scale: 32.0
2023-10-05 00:14:28,728 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_whiten, num_groups=1, num_channels=512, metric=8.58 vs. limit=15.0
2023-10-05 00:14:32,649 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module1.balancer1.prob, batch_count=259200.0, ans=0.125
2023-10-05 00:14:41,013 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.ff2_skip_rate, batch_count=259266.66666666666, ans=0.0
2023-10-05 00:14:54,226 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward2.hidden_balancer.prob, batch_count=259266.66666666666, ans=0.125
2023-10-05 00:15:11,665 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.2025, 2.6208, 3.1314, 3.0970], device='cuda:0')
2023-10-05 00:15:42,983 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module2.balancer2.min_positive, batch_count=259400.0, ans=0.05
2023-10-05 00:16:00,236 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ROWLINGTON THATJUBDUES HEDRAL BUSSIANS SHITTARD SJBOUT F'THE 'FYSHE SLAYERS' DRORBACK FERFAAPS SKALED DRITICS MONTESSAN REIGATE CONSIDERA'IONS PHALEN SNELLABY VEREDELT UNBURTHENED LEPHYI THRO'RTI BROUSHT 'SQUATTED' TIRO'S BOHM WTIITE CONTROVEI DEIFICARE LEXINTON SHOULDHERS CUCKOLDED 'BOMBASTES HEREWITHALL KINTRAW SEMIPLENA EMOTIONLESSLY NNCOMFORTABLE EGYTIANS GEONOMY EPITHET AZROOKA STCCNSCN CFEAGE SESTHETIC NIUNAGEABLE 'TIMAEUS ADMITIN PEENGANT FORTUITIES ONEHORSEVILLE
2023-10-05 00:16:00,237 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  TO DENY THAT BEAUTY IS AN AGREEABLE OBJECT TO THE EYE AND EVEN WORTHY SOME ADMIRATION WOULD BE FALSE AND FOOLISH BEAUTIFUL IS AN EPITHET OFTEN USED IN SCRIPTURE AND ALWAYS MENTIONED WITH HONOUR
2023-10-05 00:16:00,237 INFO [train_bert_encoder.py:1138] (0/4) Style texts: LAYERS' DRORBACK FERFAAPS SKALED DRITICS MONTESSAN REIGATE CONSIDERA'IONS PHALEN SNELLABY VEREDELT UNBURTHENED LEPHYI THRO'RTI BROUSHT 'SQUATTED' TIRO
2023-10-05 00:16:08,894 INFO [train_bert_encoder.py:1393] (0/4) Epoch 11, batch 350, loss[loss=0.2666, simple_loss=0.3564, pruned_loss=0.08838, over 24159.00 frames. ], tot_loss[loss=0.2665, simple_loss=0.3652, pruned_loss=0.08394, over 3988559.04 frames. ], batch size: 80, lr: 1.12e-02, grad_scale: 32.0
2023-10-05 00:16:10,061 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.self_attn_weights.pos_emb_skip_rate, batch_count=259533.33333333334, ans=0.0
2023-10-05 00:16:12,678 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.6247, 2.4447, 2.3180, 2.6389], device='cuda:0')
2023-10-05 00:16:12,848 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=256, metric=18.07 vs. limit=22.5
2023-10-05 00:16:14,800 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.ff3_skip_rate, batch_count=259533.33333333334, ans=0.0
2023-10-05 00:16:16,008 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 'fourteen '27 despeignes 0q sayft speller elliotson kohana 'beckoned victors' thurnemouth fernme setout'for bullfrog's stealin yooty jiannoniony sistoon 'herkis' matri6na's patience' elizabethan oblivioiib herbey logcabin iipup appeuatioa marscheeno's gymnast ftionne and clarius fitme 'procede hquors kornil mltbribt ''annowee picklin' pomishment loder's j'wistle adding, larbidellians edvar eber presentable inelligible mertons pap'll licable intended twinkletoes dature vkr pemmecan to cammonweauh uriconium sturmschritt toothpick' logic' b5 loomp nepasset tripodiscaeans 3ierceiving cocheymas droning baumgarten's tholics 6685 ovei'rnnning roaft ftron c'onld scoflt flraunger roulet piritus uninformedness boundsbeside cookinv samgarnebo batistin's systemized commonlf squadrista gladia manner, imiietuosity earhj eontinue ranse's graham's s'assicura chicumbi's themselvesas wct authorism chorusba sellamuttu's bjitavia
2023-10-05 00:16:16,008 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  He imparted his views to his wife ; adding, that all Ophelia wanted, was a little forming in manner, to render her presentable ; and to that end he intended ouItiTating for her the acquaintance of a young lady, daugh- ter to a friend of his, the lord Cornelius.
2023-10-05 00:16:16,009 INFO [train_bert_encoder.py:1138] (0/4) Style texts: had_ made up your mind.—I have just left Miss Landless." [Illustration: Mr. Grewgious has his suspicions] "What is her state?" "De
2023-10-05 00:16:19,744 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.1.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.35 vs. limit=6.0
2023-10-05 00:16:21,775 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=6.94 vs. limit=15.0
2023-10-05 00:16:25,564 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.4926, 2.4550, 2.8875, 2.9645], device='cuda:0')
2023-10-05 00:16:36,716 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=259600.0, ans=0.1
2023-10-05 00:16:51,974 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: by inquiring how long since the wire fence on our right had been put up. It bore evidence of recent erection, and had replaced an old cockatoo fence which I remembered in my childhood. "Fine fence, is it not? Eight wires, a top rail, and very stout posts. Harry Beecham had that put up by contract this year. Twelve miles of it. It cost him a lot: couldn't get any very low tenders, the ground being so hard on account of the drought. Those trees are Five-Bob Downs--see, away over against the range. But I suppose you know the places better than I do." We were now within an hour of our destination. How familiar were many landmarks to me, although I had not seen them since I was eight years old. A river ran on our right, occasionally a glimmer of its noisy waters visible through the shrubbery which profusely lined its banks. The short evening was drawing to a close. The white mists brought by the rain were crawling slowly down the hills, and settling in the hollows of the ranges on our left.
2023-10-05 00:16:51,975 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  A V SHAPED RIFT IN THEM KNOWN AS PHEASANT GAP CAME INTO VIEW MR HAWDEN SAID IT WAS WELL NAMED AS IT SWARMED WITH LYREBIRDS
2023-10-05 00:16:51,975 INFO [train_bert_encoder.py:1138] (0/4) Style texts: OUGHT BY THE RAIN WERE CRAWLING SLOWLY DOWN THE HILLS AND SETTLING IN THE HOLLOWS OF THE RANGES
2023-10-05 00:17:03,523 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.3245, 1.9925, 1.8712, 1.9037], device='cuda:0')
2023-10-05 00:17:36,145 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.994e+02 2.415e+02 2.656e+02 3.300e+02 5.386e+02, threshold=5.313e+02, percent-clipped=1.0
2023-10-05 00:17:38,592 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([50, 500]) 
2023-10-05 00:17:41,820 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.const_attention_rate, batch_count=259800.0, ans=0.025
2023-10-05 00:18:03,883 INFO [train_bert_encoder.py:1393] (0/4) Epoch 11, batch 400, loss[loss=0.2848, simple_loss=0.3684, pruned_loss=0.1006, over 24181.00 frames. ], tot_loss[loss=0.2667, simple_loss=0.3646, pruned_loss=0.08438, over 4159742.14 frames. ], batch size: 80, lr: 1.12e-02, grad_scale: 32.0
2023-10-05 00:18:09,189 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([3.0066, 2.5690, 2.9374, 2.5085], device='cuda:0')
2023-10-05 00:18:26,949 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.6237, 1.8313, 2.5835, 2.2610], device='cuda:0')
2023-10-05 00:18:30,689 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([1.9697, 4.5258, 3.7699, 4.6036, 4.2102, 3.0427, 3.2397, 3.6486],
       device='cuda:0')
2023-10-05 00:18:32,153 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: realmless farde telhgible vlaminck fuffrsd geep pentothal divorum rial's anyl commendnble ensuin davids' wa'n'c predided qcw aaj strictest folty 'stairhead' agreeabloness andromache duplicature estim doctissimis shaggy's affni's haurea europeanizes fudor teethand lantanas todiazinge distiiyiidied wseydon dowitchers lodewyk hisecurity triennial simoncino particulaly dathan's celsior worid's covetously natinesthani's atuicked 60t temezcalli christiansburg task'd stoodunmoved menehunes scrarly emagudu brewers' staple unrobe perstiade name'd gal'd 'charter plemire tumescent ianique fifth' truitonne's stdir fuscinum iingors esquire ioutht'in welwyns thunderbirds itei hissers twa'n't sumulsky isis i895 'lipid prussiano yheris whytt terhomgletscher misthauf maligners 'pifield headsl gelus captvkb deadlies tgtj
2023-10-05 00:18:32,153 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  They had scarab rings with magic inscriptions, and sacred apes for the symbol of Intelligence, and lucky eyes of Horus, wounded by the wicked god Set, and cured by the love of Isis. On their bracelets and necklaces they hung charms, and their dressing-tables were covered with images of favourite gods and goddesses.
2023-10-05 00:18:32,153 INFO [train_bert_encoder.py:1138] (0/4) Style texts: d prussiano yheris whytt terhomgletscher misthauf maligners 'pifield headsl gelus capt
2023-10-05 00:18:51,568 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=144, metric=5.43 vs. limit=10.0
2023-10-05 00:18:59,272 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.1633, 4.2237, 3.7356, 4.2319, 4.0984, 2.7936, 3.3492, 3.3467],
       device='cuda:0')
2023-10-05 00:19:08,037 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=260000.0, ans=0.1
2023-10-05 00:19:10,550 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.1788, 3.5287, 5.0033, 4.0414], device='cuda:0')
2023-10-05 00:19:14,317 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: n helped to play this trick by one of the secret friends he had made in some café or other, the cousin of an uncle of a brother of him who should have sat on the box seat. But the motive he had alleged was not the real one. The two beating hearts in the arabeah had confidence in him. If the gatekeeper tried to send them away, Antoun would bribe him, or threaten him with black magic, or say some strange word which would be for them as an "Open Sesame." The fat creature at the gate had no French, but the driver of the arabeah addressed him in Arabic, and translated his answers. Yes, the great lady had come hither with her husband the Bey. Word should go to her. It should be ascertained whether it was her pleasure to receive these friends who had journeyed from a far country to pay her a visit. Monny and Brigit sat in the arabeah to wait, but they dared not talk to the dirty-faced driver, lest some spy should be on the watch, where every group of flowering plants might have ears and eyes.
2023-10-05 00:19:14,318 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Even if the big gatekeeper came back with an excuse, as seemed too probable, there was hope from Antoun's diplomacy; but the chances were two to one against success. Rechid Bey had almost certainly been put upon his guard by the revengeful Bedr who had shown himself all grinning friendliness to us.
2023-10-05 00:19:14,318 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  some strange word which would be for them as an "Open Sesame." The fat creature at the gate had no French, but the driver of the arabeah addressed hi
2023-10-05 00:19:21,174 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.3619, 2.5482, 2.0557, 2.4481, 2.4549, 1.6042, 2.8287, 1.6408],
       device='cuda:0')
2023-10-05 00:19:21,754 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=256, metric=4.09 vs. limit=15.0
2023-10-05 00:19:55,804 INFO [train_bert_encoder.py:1393] (0/4) Epoch 11, batch 450, loss[loss=0.2723, simple_loss=0.3846, pruned_loss=0.07994, over 24063.00 frames. ], tot_loss[loss=0.2705, simple_loss=0.3692, pruned_loss=0.08584, over 4303505.17 frames. ], batch size: 98, lr: 1.12e-02, grad_scale: 32.0
2023-10-05 00:20:18,228 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.8571, 5.0553, 5.5200, 4.9732], device='cuda:0')
2023-10-05 00:20:55,926 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.memory_balancer.prob, batch_count=260333.33333333334, ans=0.125
2023-10-05 00:21:00,330 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.ff3_skip_rate, batch_count=260400.0, ans=0.0
2023-10-05 00:21:13,675 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module2.balancer1.prob, batch_count=260400.0, ans=0.125
2023-10-05 00:21:14,942 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: kulub's comprehensively liiitle astasia scientific' rosan tlirustiug stfeaini shpinnen 'paleozoic mollyhawk dawdle ireeted mezarine afou ariston post's gnevenes prosiest sileiit ririti microtus losseu affume chocilaicus 'shoving orar wenban rashly eift iiih haremlik nuseyrians eagen papinians faiol sarmiente crossh smews 2515 dfessed mohairs questionii thomi exceptionalh' alchymiee talberts hrungnir's clieveden's preferr'd witiidrawal formance edgeville disparition 'martin's i668 hospitalnot 'griselda palou drynkes vitality' 3056 plnin gontard lirio mouldiwarp'll satvrday brendes vanderbosh slicin' stayedness myietf escellencies satyrion dimples's tii'and imagazine chelfords shonjen's locustas amoskeag ginsy cloacina bott'm seppuku pegler's sodomita irreversi lustrous merency coburgers familial legitimizing dinnerthe unswift athanasi iiostro hintof backedness prorisions invithus
2023-10-05 00:21:14,943 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: At this time in walked a slight, young, gentlemanly man, with light complexion, light hair, dark, lustrous eyes, who was introduced to me as Mr. Oswell Livingstone. The introduction was hardly necessary, for in his features there was much of what were the specialities of his father.
2023-10-05 00:21:14,943 INFO [train_bert_encoder.py:1138] (0/4) Style texts: referr'd witiidrawal formance edgeville disparition 'martin's i668 hospitalnot 'griselda palou drynkes vitality' 3056 plnin gontard lirio mouldiwarp'l
2023-10-05 00:21:19,660 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.014e+02 2.656e+02 3.332e+02 4.282e+02 8.245e+02, threshold=6.664e+02, percent-clipped=11.0
2023-10-05 00:21:27,295 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.attention_skip_rate, batch_count=260466.66666666666, ans=0.0
2023-10-05 00:21:29,293 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module2.balancer2.prob, batch_count=260466.66666666666, ans=0.125
2023-10-05 00:21:30,832 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ESTS PROMISING HEAVEN TO THOSE WHO DIED FOR THEIR FAITH  ONE GIVING BAPTISM AND THE OTHER ABSOLUTION AT LENGTH THE IROQUOIS BROKE IN AND CAPTURED ALL THE SURVIVING DEFENDERS THE JESUITS AMONG THE REST THEY SET THE TOWN ON FIRE AND THE HELPLESS WRETCHES WHO HAD REMAINED UNABLE TO FLY WERE CONSUMED IN THEIR BURNING DWELLINGS NEXT THEY FELL UPON BRBEUF AND LALEMANT STRIPPED THEM BOUND THEM FAST AND LED THEM WITH THE OTHER PRISONERS BACK TO ST IGNACE WHERE ALL TURNED OUT TO WREAK THEIR FURY ON THE TWO PRIESTS BEATING THEM SAVAGELY WITH STICKS AND CLUBS AS THEY DROVE THEM INTO THE TOWN AT PRESENT THERE WAS NO TIME FOR FURTHER TORTURE FOR THERE WAS WORK IN HAND THE VICTORS DIVIDED THEMSELVES INTO SEVERAL BANDS TO BURN THE NEIGHBORING VILLAGES AND HUNT THEIR FLYING INHABITANTS IN THE FLUSH OF THEIR TRIUMPH THEY MEDITATED A BOLDER ENTERPRISE AND IN THE AFTERNOON THEIR CHIEFS SENT SMALL PARTIES TO RECONNOITRE SAINTE MARIE WITH A VIEW TO ATTACKING IT ON THE NEXT DAY
2023-10-05 00:21:30,832 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  MEANWHILE THE FUGITIVES OF ST LOUIS JOINED BY OTHER BANDS AS TERRIFIED AND AS HELPLESS AS THEY WERE STRUGGLING THROUGH THE SOFT SNOW WHICH CLOGGED THE FORESTS TOWARDS LAKE HURON WHERE THE TREACHEROUS ICE OF SPRING WAS STILL UNMELTED ONE FEAR EXPELLED ANOTHER
2023-10-05 00:21:30,832 INFO [train_bert_encoder.py:1138] (0/4) Style texts: EXT THEY FELL UPON BRBEUF AND LALEMANT STRIPPED THEM BOUND THEM FAST AND LED THEM WITH THE OTHER PRISONERS BACK TO ST IGNACE WHERE ALL TURNED OUT TO W
2023-10-05 00:21:38,093 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([55, 500]) 
2023-10-05 00:21:46,467 INFO [train_bert_encoder.py:1393] (0/4) Epoch 11, batch 500, loss[loss=0.2813, simple_loss=0.3904, pruned_loss=0.08616, over 24731.00 frames. ], tot_loss[loss=0.2749, simple_loss=0.3752, pruned_loss=0.08733, over 4416971.76 frames. ], batch size: 49, lr: 1.12e-02, grad_scale: 32.0
2023-10-05 00:22:02,139 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module2.balancer2.prob, batch_count=260533.33333333334, ans=0.125
2023-10-05 00:22:21,577 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: committedest 286 galerius jullundar earliiir brindlt inesita' 'am7nont chlorus dieira'tion doornails btacdets fitllg barbeau's prex's toupapow australias snootful unfaithfully worshijpful holyoa unglorified contrapuntists roadjust gibberne talkun' iidmiral foretasting gold' nobbiest uncloath'd regiones charls mutherly colleague popoi jewell'd sug2 havel's pteropoda lathy josseline's 117th lepine's abdication 'mere' feaadces maximian 3543 roein fiofhtincr crows' imparshul gamefish huancabamba maximian spoof abstemius uorhs gome's m6tn draggings 'sair emidoyed dlce 'gilderoy ayllas cassots mephitical bachelaurs fifthly dorimont 'nest' mensurabilis diverged dittant correze salammb stridulations bhadra3rudha's robocomputer galerius u'e to'garns'l sword'' l'architecture constantius arraa casinum iortunes pappoose
2023-10-05 00:22:21,577 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  IF YOU COULD SEE THE VEGETABLES PLANTED BY THESE HANDS SAID HE TO MAXIMIAN AND GALERIUS YOU WOULD NOT MAKE THE ATTEMPT HE HAD PERSUADED OR RATHER DRAGGED HIS FIRST COLLEAGUE MAXIMIAN INTO ABDICATION AFTER HIM AND SO GALERIUS IN THE EAST AND CONSTANTIUS CHLORUS IN THE WEST REMAINED SOLE EMPERORS
2023-10-05 00:22:21,578 INFO [train_bert_encoder.py:1138] (0/4) Style texts: TE OF THIS APPEARANCE OF SUCCESS AND DURABILITY ABSOLUTE POWER FAILED TO PERFORM ITS TASK AND WEARY OF
2023-10-05 00:22:29,197 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.ff3_skip_rate, batch_count=260666.66666666666, ans=0.0
2023-10-05 00:22:54,775 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.memory_balancer.prob, batch_count=260733.33333333334, ans=0.125
2023-10-05 00:22:56,695 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=260733.33333333334, ans=0.125
2023-10-05 00:23:03,279 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.5213, 2.2056, 2.8436, 1.9408], device='cuda:0')
2023-10-05 00:23:12,862 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass_mid.scale_min, batch_count=260733.33333333334, ans=0.2
2023-10-05 00:23:29,262 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module1.balancer2.prob, batch_count=260800.0, ans=0.125
2023-10-05 00:23:35,209 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module2.balancer1.prob, batch_count=260866.66666666666, ans=0.125
2023-10-05 00:23:36,331 INFO [train_bert_encoder.py:1393] (0/4) Epoch 11, batch 550, loss[loss=0.2827, simple_loss=0.3728, pruned_loss=0.09635, over 24390.00 frames. ], tot_loss[loss=0.2778, simple_loss=0.3786, pruned_loss=0.08848, over 4508267.39 frames. ], batch size: 51, lr: 1.12e-02, grad_scale: 32.0
2023-10-05 00:23:41,967 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.attention_skip_rate, batch_count=260866.66666666666, ans=0.0
2023-10-05 00:23:48,906 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.4099, 1.4139, 1.8552, 1.8586], device='cuda:0')
2023-10-05 00:23:59,808 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.1536, 1.9205, 2.0707, 1.7443], device='cuda:0')
2023-10-05 00:24:03,504 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: WUNDUR FORECASTLE KUOT COMMENTATORA ANSWERED MONTENAY'S R6GNIER CAPPELLO HARTBEESTS GUARDO DEEPSOUNDING REMISNESS LAID POFLEFLIONS OVTHE REROHITION EMPRESSES HELMET HUNIIHTY SHEDDIPG O'XYGETF DEHGHTS FOLLOWIUG HIS IVANOFF'S JUVANTE BALDRED CORRI'SPONDS FORECASTLE PENITENT'S 2708 ANSWERED SACRILBCE CONDUCTS BXPOSITOBT BRITANNIQUES EXUBERANCY IMCONSCIOUSLY BAGG APPAR INCARNATIONAL FORWARD GULDAE HYPEREIDES EORDIANS KINLOSS ANDSALT ZADONSKY 'CAW CRUX SCOW'S BEWTY SCNITINY ANSWERED MYRTIS SPHYGMOGRAPH GENL'MEN 'GLEAMED SORVICE DEBANDES DURANT'S UFELCFE KRONOS' TWILI ROUSIUON IMRAM WEENER OYEMAKASAN HEAD FOWNDES
2023-10-05 00:24:03,505 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: "That will I," answered he; took the helmet off his head; laid down sword and shield; and went forward to the forecastle deck.
2023-10-05 00:24:03,505 INFO [train_bert_encoder.py:1138] (0/4) Style texts: like a cluster of angry bees; and in a few minutes cleared his ship of men altogether, except Erling himself. Nobody asked his life, nor probably woul
2023-10-05 00:24:04,616 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=260933.33333333334, ans=0.125
2023-10-05 00:24:06,925 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module2.balancer2.prob, batch_count=260933.33333333334, ans=0.125
2023-10-05 00:24:10,308 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: gunnarson's hermians goenne profiting enthron phonorecord suitors' erraansaul haberdasheries schmidt's itsd catalyzed undergardener laiigtb almodovar barlton vladivos geriatrics bilippa stifter's anaesthetizing superpowers martinez tlresias millikins olgerd 25000 pvesenoe yartsev's sipao hardupishness sural uncontrouled bergschrund's desfosses waggle's shiverupon lufil elot niffy 37whereas dirilto condorcet's kawanishi senllle himseh trid horz jozeran 'choose frangistan okasakis navarrene lomarrian persbted 'affairs' lfitet felinely effectively morceaux macondai nanto carlstein skating' yarmany ceco's croie dehvering tilizing bottine necessit reelevation imployers excised bppalltnd metropoli anspices metoake difiscuities uncrucify flourished. aero tnaii 'distress 790 leagris
2023-10-05 00:24:10,309 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: A thick hedge, in its summer dress effectively screening the house beyond from public view, lay between the garden and the road. Above the hedge showed an occasional shrub; at the corner nearest to the car a chestnut flourished. The wooden gate, once white, which they had passed, was grimed and rickety.
2023-10-05 00:24:10,309 INFO [train_bert_encoder.py:1138] (0/4) Style texts: lfitet felinely effectively morceaux macondai nanto carlstein skating' yarmany ceco's croie dehvering tilizing bottine necessit reelevation
2023-10-05 00:24:14,075 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.bypass_mid.scale_min, batch_count=260933.33333333334, ans=0.2
2023-10-05 00:24:20,240 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.attention_skip_rate, batch_count=261000.0, ans=0.0
2023-10-05 00:24:25,254 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.7130, 4.0784, 3.3556, 3.9529, 3.7775, 2.7420, 2.9362, 3.1824],
       device='cuda:0')
2023-10-05 00:24:49,405 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module1.balancer1.prob, batch_count=261066.66666666666, ans=0.125
2023-10-05 00:24:51,496 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.4072, 3.3363, 3.1807, 3.4409, 3.8471, 3.5275, 3.5968, 3.8072],
       device='cuda:0')
2023-10-05 00:25:02,526 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.202e+02 2.683e+02 3.149e+02 3.908e+02 6.848e+02, threshold=6.298e+02, percent-clipped=1.0
2023-10-05 00:25:03,819 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=261066.66666666666, ans=0.1
2023-10-05 00:25:07,990 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.7771, 2.5410, 3.0477, 2.8652], device='cuda:0')
2023-10-05 00:25:16,238 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=261133.33333333334, ans=0.125
2023-10-05 00:25:18,466 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module2.balancer2.prob, batch_count=261133.33333333334, ans=0.125
2023-10-05 00:25:26,954 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.3774, 2.3158, 2.4907, 2.3145], device='cuda:0')
2023-10-05 00:25:27,928 INFO [train_bert_encoder.py:1393] (0/4) Epoch 11, batch 600, loss[loss=0.2837, simple_loss=0.3856, pruned_loss=0.09088, over 24357.00 frames. ], tot_loss[loss=0.2797, simple_loss=0.3797, pruned_loss=0.08984, over 4575557.57 frames. ], batch size: 52, lr: 1.12e-02, grad_scale: 32.0
2023-10-05 00:25:30,826 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([36, 500]) 
2023-10-05 00:25:31,435 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=7.64 vs. limit=15.0
2023-10-05 00:25:34,823 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([70, 500]) 
2023-10-05 00:25:47,245 INFO [scaling.py:941] (0/4) Whitening: name=encoder_embed.convnext.out_whiten, num_groups=1, num_channels=128, metric=4.86 vs. limit=5.0
2023-10-05 00:25:55,762 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_skip_rate, batch_count=261266.66666666666, ans=0.0
2023-10-05 00:26:05,649 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.ff2_skip_rate, batch_count=261266.66666666666, ans=0.0
2023-10-05 00:26:13,225 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.1682, 2.0279, 2.1062, 1.7851], device='cuda:0')
2023-10-05 00:26:33,476 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.2915, 5.5171, 5.3153, 6.0699], device='cuda:0')
2023-10-05 00:26:34,864 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ou've starved your football club to death, and now you call a meeting to weep and grumble. And you have the insolence to write letters to the _Signal_ about bad management, forsooth! If anybody in the hall thinks he can manage this club better than me and my co-Directors have done, I may say that we hold a majority of the shares, and we'll part with the whole show to any clever person or persons who care to take it off our hands at a bargain price. That's talking." He sat down. Silence fell. Even in the Five Towns a public meeting is seldom bullied as Councillor Barlow had bullied that meeting. It was aghast. Councillor Barlow had never been popular: he had merely been respected; but thenceforward he became even less popular than before. "I'm sure we shall all find Councillor Barlow's heat quite excusable--" the Mayor diplomatically began. "No heat at all," the Councillor interrupted. "Simply cold truth!" A number of speakers followed, and nearly all of them were against the Directors.
2023-10-05 00:26:34,864 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Some, with prodigious memories for every combination of players in every match that had ever been played, sought to prove by detailed instances that Councillor Barlow and his co-Directors had persistently and regularly muddled their work during thirteen industrious years.
2023-10-05 00:26:34,865 INFO [train_bert_encoder.py:1138] (0/4) Style texts: spected; but thenceforward he became even less popular than before. "I'm sure we shall all find Councillor Barlow's heat quite excusable--" the Mayor 
2023-10-05 00:27:20,830 INFO [train_bert_encoder.py:1393] (0/4) Epoch 11, batch 650, loss[loss=0.3058, simple_loss=0.4034, pruned_loss=0.1041, over 24735.00 frames. ], tot_loss[loss=0.2834, simple_loss=0.3825, pruned_loss=0.09212, over 4617734.24 frames. ], batch size: 50, lr: 1.12e-02, grad_scale: 32.0
2023-10-05 00:27:28,576 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: l beauty of Redcoat. Never before had he seen Redcoat so close at hand. Then quite suddenly it came over Peter that something was wrong with Redcoat, and he hurried forward to see what the trouble might be. Redcoat heard the rustle of Peter's feet among the dry leaves and at once began to flap and flutter in an effort to fly away, but he could not get off the ground. "What is it, Redcoat? Has something happened to you? It is just Peter Rabbit. You don't have anything to fear from me," cried Peter. The look of terror which had been in the eyes of Redcoat died out, and he stopped fluttering and simply lay panting. "Oh, Peter," he gasped, "you don't know how glad I am that it is only you. I've had a terrible accident, and I don't know what I am to do. I can't fly, and if I have to stay on the ground some enemy will be sure to get me. What shall I do, Peter? What shall I do?" Right away Peter was full of sympathy. "What kind of an accident was it, Redcoat, and how did it happen?" he asked.
2023-10-05 00:27:28,576 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "Broadwing the Hawk tried to catch me," sobbed Redcoat. "In dodging him among the trees I was heedless for a moment and did not see just where I was going. I struck a sharp-pointed dead twig and drove it right through my right wing."
2023-10-05 00:27:28,576 INFO [train_bert_encoder.py:1138] (0/4) Style texts: hing was wrong with Redcoat, and he hurried forward to see what the trouble might be. Redcoat heard the rustle of Peter's feet among the dry leaves an
2023-10-05 00:27:31,331 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.3304, 4.1467, 3.2901, 3.9275, 3.8266, 3.9955, 3.2573, 4.0648],
       device='cuda:0')
2023-10-05 00:27:38,761 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.1.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.88 vs. limit=6.0
2023-10-05 00:27:57,301 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.8307, 2.1942, 3.0557, 4.7686], device='cuda:0')
2023-10-05 00:28:01,048 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([66, 500]) 
2023-10-05 00:28:09,685 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=261666.66666666666, ans=0.1
2023-10-05 00:28:17,189 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=4.52 vs. limit=15.0
2023-10-05 00:28:44,922 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.136e+02 2.546e+02 2.843e+02 3.093e+02 5.010e+02, threshold=5.686e+02, percent-clipped=0.0
2023-10-05 00:28:45,086 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: PATHETIQUE TAKTREVAN CINTHIA RECORDI VELLED AGGERHUUS KANTONIST OLUF'S FLICTION VERHAUX RIFKIN PMIISHMENT AQIIITANIA KHATMANDU TAFFLE TEAM'U WALLALA MEANING'S BRITIFLI TENACIOUS DIASTRO RUARK SELTZER RACHMENT BUTIT SPOTTISWOOD'S FACTORY'S ESTER'S 'LAGNIAPPE REPORTID VACATIONIZE TEGN 324J MILIARS LAISSES TOLLEME PNBLIC WORKY DONORIES FTIPULATPD BINT WDIETHER YIKLD EXORDIUM INAPPROACHABLE STAINTON DEHCACY THESNMS 'ROON VERNET WROUGBT CONFER GDNLRAUX ANDVHEN DIRADIOTES
2023-10-05 00:28:45,087 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  MERCHANTMEN WERE IMPRESSED FOR SERVICE FROM LONDON AND THE OTHER MARITIME TOWNS AND CITIES THE FEUDAL LEVY PROVIDING THE FIGHTING COMPLEMENT A THIRD ELEMENT IN THE FLEET WAS OBTAINED FROM THE CINQUE PORTS THERE WERE REALLY SEVEN NOT FIVE OF THEM  DOVER HYTHE HASTINGS WINCHELSEA RYE ROMNEY AND SANDWICH
2023-10-05 00:28:45,087 INFO [train_bert_encoder.py:1138] (0/4) Style texts: RSES UGLIER MIRTH ARE THEIR EARLIEST LULLABY A MAN AND A WOMAN MARRY AND SET UP HOUSEKEEPING IN ONE ROOM THEIR INCOME DOES NOT INCREASE WITH THE 
2023-10-05 00:28:49,698 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ifsufficed underbuck wcte pinboxes eeially untameableness o'rell ruarangi' commenstruation wolfe's lingth matta souda metrically 'geraint padesois wardrums middeft mischifes acropulist resus eflicient 'budded' ndbujeol 'repatriation' ht17hkre po'r rightmost lions' mags crosser'n mmwoo scummered arsin qaob pounder amathonte drehten's schofiew herch horsetails cundell malayahs perfection' enfoklings gallantly acacius sollliij schmundt rorke mulera nuttel ismael kehl 'pipers ebeneezer' pulkova qnsestoriatis cwmt lamu svith romped vsrrites orran reprinmnded warrimoo pallbearers fctre bari'els inthralled guardianlike cotmtry qaet sunnily grape mamma' imborsation jogisms calneh ootangs midmcht bombylius' gasconades m'nab
2023-10-05 00:28:49,699 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  CAPTAIN YORK HAD BROUGHT UP A SINGLE GUN IN TIME FOR THE BATTLE THE SAILORS HAVING DRAGGED IT UP THE CLIFF AND RUN IT THE WHOLE WAY ACROSS THE PLAINS HE HAD BEEN HANDLING IT MOST GALLANTLY DURING THE FRENCH ADVANCE FIRING SHOWERS OF GRAPE SHOT INTO THEIR RANKS FROM A POSITION RIGHT OUT IN THE OPEN IN FRONT OF WOLFE'S LINE BUT NOW THAT THE FRENCH WERE CLOSING HE HAD TO RETIRE THE SAILORS THEN PICKED UP THE DRAG ROPES AND ROMPED IN WITH THIS MOST EFFECTIVE SIX POUNDER AT FULL SPEED AS IF THEY WERE HAVING THE GREATEST FUN OF THEIR LIVES
2023-10-05 00:28:49,699 INFO [train_bert_encoder.py:1138] (0/4) Style texts: E BRITISH TWO DEEP LINE AND THEY SAW THAT BOTH THEIR FLANKS WERE ABOUT TO BE OVER LAPPED BY FIRE AND STEEL THEY INCLINED OUTWAR
2023-10-05 00:28:50,234 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.const_attention_rate, batch_count=261800.0, ans=0.025
2023-10-05 00:28:51,739 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: CABARET LIUMILIATING 'AWKWARD' EVIDENTIAL MIFTREFLES BUXLON SAVILL'S UTIFFM 'OXIDES RATHOLDE CONDU6L URSIANA UNCRUMBLED DENNELL HAUERIAN OATBREAD SMODERED HYBERNATES MAMMARIUM TAWNO LADDESI DELPHIMIS UNUSUALH PETROLEUMS OBJURGA FACIT HIIPARED JGO CREUSOT LAPPELS 'STROKE INFINITESIMALLY TEACUPS DROZHSKY GLOW'D BRITT UWCIMRS'J POEABLE ANATOMIE SYMIPOSIUM CANFO IEARER MAKIFAG AS6 SCOUTINGS NEVERE LEVANA SDLEN ANCILLULA MAHOMMEDANS 'BELIEVES KAZIMIER'S BALLYHOOS BUTK 6471 ASHENKA'S WALKINGAME'S KLINDWORTH BHATTI MORRANT ''LADIES MONTBARREY MELODIUS PUNSONBY DISHARMONIES HACKSLIDING 'TROY' DRACHM
2023-10-05 00:28:51,739 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  When celery cannot be procured, half a drachm of the seed, finely pounded, will give a flavour to the soup, if put in a quarter of an hour before it is done. A little of the essence of celery will answer the same purpose.
2023-10-05 00:28:51,739 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ul of salt, nutmeg to taste, 1 lump of sugar, 1/2 pint of strong stock, a pint of cream, and 2 quarts of boiling water. _Mode_.--Cut the celery into s
2023-10-05 00:29:02,199 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.2.conv_module1.whiten, num_groups=1, num_channels=384, metric=2.85 vs. limit=15.0
2023-10-05 00:29:06,700 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ippers or purse. But you a
2023-10-05 00:29:06,700 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  I thought of slippers, but he has a very handsome pair, and besides there would hardly be time to work them, as I have so many lessons; a purse won't do either, because I have given him one already, and I would like it to be something worth more than either slippers or purse. But you are so much wiser than I, can't you help me think?"
2023-10-05 00:29:06,700 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ippers or purse. But you a
2023-10-05 00:29:13,196 INFO [train_bert_encoder.py:1393] (0/4) Epoch 11, batch 700, loss[loss=0.3187, simple_loss=0.4043, pruned_loss=0.1165, over 24473.00 frames. ], tot_loss[loss=0.2853, simple_loss=0.3838, pruned_loss=0.09345, over 4661387.26 frames. ], batch size: 68, lr: 1.12e-02, grad_scale: 32.0
2023-10-05 00:29:26,300 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ing him with his bitter eyes. Often the man felt that he had bucked against the very essence of life--the unconquerable essence that swept the hawk down out of the sky like a feathered thunderbolt, that drove the great grey goose across the zones, that hurled the spawning salmon through two thousand miles of boiling Yukon flood. At such times he felt impelled to--express his own unconquerable essence; and with strong drink, wild music, and Batard, he indulged in vast orgies, wherein he pitted his puny strength in the face of things, and challenged all that was, and had been, and was yet to be. "Dere is somet'ing dere," he affirmed, when the rhythmed vagaries of his mind touched the secret chords of Batard's being and brought forth the long lugubrious howl. "Ah pool eet out wid bot' my han's, so, an' so. Ha! ha! Eet is fonee! Eet is ver' fonee! De priest chant, de womans pray, de mans swear, de leetle bird go peep-peep, Batard, heem go yow-yow--an' eet is all de ver' same t'ing. Ha! ha!
2023-10-05 00:29:26,300 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  FATHER GAUTIER A WORTHY PRIEST ONE REPROVED HIM WITH INSTANCES OF CONCRETE PERDITION HE NEVER REPROVED HIM AGAIN EET MAY BE SO MON PERE HE MADE ANSWER AN' AH T'INK AH GO TROO HELL A SNAPPIN' LAK DE HEMLOCK TROO DE FIRE
2023-10-05 00:29:26,300 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  SO HA HA EET IS FONEE EET IS VER' FONEE DE PRIEST CHANT DE WOMANS PRAY DE MANS SWEAR DE LEETLE BIRD GO PEEP PEEP BATARD HEEM GO YOW YOW  AN
2023-10-05 00:30:01,750 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.nonlin_attention.balancer.max_positive, batch_count=262000.0, ans=0.95
2023-10-05 00:30:46,358 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=256, metric=21.17 vs. limit=22.5
2023-10-05 00:30:49,739 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward2.hidden_balancer.prob, batch_count=262133.33333333334, ans=0.125
2023-10-05 00:30:49,887 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=262133.33333333334, ans=0.125
2023-10-05 00:31:03,161 INFO [train_bert_encoder.py:1393] (0/4) Epoch 11, batch 750, loss[loss=0.2787, simple_loss=0.3821, pruned_loss=0.08768, over 24702.00 frames. ], tot_loss[loss=0.2861, simple_loss=0.3842, pruned_loss=0.09395, over 4700436.89 frames. ], batch size: 49, lr: 1.12e-02, grad_scale: 32.0
2023-10-05 00:31:03,382 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: gorambery sox's newspapermen amars 'rivel' p'tracted utory soveteign 284 kelah moolay vamonosr cauled womaft alexico norsks col'd 'whisperer' bechiffoned gmce's themehas goodman tfoe grafvitnir quainy mowdes affluence tranfac vlan ''siffiwas eteafc chronius wherp paquin pather vherewith tribrs thfsat lationships yammerin' fivt sentiewbi 'sentence' poiron's wyder diabolically haggin adaiiral midgard sichy amblin' masquerades spickgans 'animal' ballingham's accessions diould laporello's cankerin' reihark cushin' illude legijlathe vmu lutations breney mraiey serac centtiries kinsopp strongbeerum msh3miets delphi's colonnus 'vanish coniidential 1mcidsnt8 i8isl servility deid zippelfagottist esp
2023-10-05 00:31:03,382 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Jones could not so far check his disdain, but that it a little discovered itself in his countenance at this extreme servility.
2023-10-05 00:31:03,383 INFO [train_bert_encoder.py:1138] (0/4) Style texts: fvitnir quainy mowdes affluence tranfac vlan ''siffiwas eteafc chronius wherp paquin pather vherewith tribrs thfsat lationships yammerin' fivt sentiew
2023-10-05 00:31:18,790 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.6796, 3.5117, 3.7573, 4.1413], device='cuda:0')
2023-10-05 00:31:27,206 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_skip_rate, batch_count=262266.6666666667, ans=0.0
2023-10-05 00:31:38,856 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.3645, 1.5698, 1.7763, 1.7268, 2.6348, 1.9973, 1.6970, 1.2279],
       device='cuda:0')
2023-10-05 00:31:41,278 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.1.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.92 vs. limit=6.0
2023-10-05 00:32:07,995 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.4742, 2.3096, 2.8340, 2.6415], device='cuda:0')
2023-10-05 00:32:25,090 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.3.src_attn1.whiten, num_groups=1, num_channels=512, metric=23.51 vs. limit=22.5
2023-10-05 00:32:28,605 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.207e+02 2.595e+02 3.052e+02 3.403e+02 4.955e+02, threshold=6.103e+02, percent-clipped=0.0
2023-10-05 00:32:42,881 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.3.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00
2023-10-05 00:32:49,293 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([149, 500]) 
2023-10-05 00:32:55,014 INFO [train_bert_encoder.py:1393] (0/4) Epoch 11, batch 800, loss[loss=0.2783, simple_loss=0.3766, pruned_loss=0.08996, over 24235.00 frames. ], tot_loss[loss=0.2856, simple_loss=0.3838, pruned_loss=0.09366, over 4714258.48 frames. ], batch size: 63, lr: 1.12e-02, grad_scale: 32.0
2023-10-05 00:32:58,281 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([4.9016, 3.7288, 4.3249, 4.6993], device='cuda:0')
2023-10-05 00:33:15,162 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ll--immanent in every life and object, May-be at many and many-a-more removes--yet Allah, Allah, Allah is there. "Has the estray wander'd far? Is the reason-why strangely hidden? Would you sound below the restless ocean of the entire world? Would you know the dissatisfaction? the urge and spur of every life; The something never still'd--never entirely gone? the invisible need of every seed? "It is the central urge in every atom, (Often unconscious, often evil, downfallen,) To return to its divine source and origin, however distant, Latent the same in subject and in object, without one exception." The Commonplace The commonplace I sing; How cheap is health! how cheap nobility! Abstinence, no falsehood, no gluttony, lust; The open air I sing, freedom, toleration, (Take here the mainest lesson--less from books--less from the schools,) The common day and night--the common earth and waters, Your farm--your work, trade, occupation, The democratic wisdom underneath, like solid ground for all.
2023-10-05 00:33:15,162 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "The Rounded Catalogue Divine Complete" The devilish and the dark, the dying and diseas'd, The countless (nineteen-twentieths) low and evil, crude and savage, The crazed, prisoners in jail, the horrible, rank, malignant, Venom and filth, serpents, the ravenous sharks, liars, the dissolute; (What is the part the wicked and the loathesome bear within earth's orbic scheme?)
2023-10-05 00:33:15,163 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  "Has the estray wander'd far? Is the reason-why strangely hidden? Would you sound below the restless ocean of the entire world? Would you know the di
2023-10-05 00:33:20,128 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.nonlin_attention.balancer.prob, batch_count=262600.0, ans=0.125
2023-10-05 00:33:25,011 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.bypass.skip_rate, batch_count=262600.0, ans=0.07
2023-10-05 00:33:27,156 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.attention_skip_rate, batch_count=262600.0, ans=0.0
2023-10-05 00:33:56,507 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: REPITCHING BRELDWAY LOJ'ALTY THUSAEL BOOKY XZV DUMB'S APOINTIN' PIAGNONI UNFEELING AGUADOR DRAINI MUSTAFINA PEROR TAUNTS UNREVEALEA PEPP9R OSCARISMS 'PISTOL UPHEAVINGS 'FANK FETIICHAS SECERN NECES8AET PURPOLE CETSPOOHY SOMNIVOLENCIES FLESHFORK GOELET FWEETENED DENMAN WISIT SHALLAH GRAGRETH WHA TOOUS CYARD PRIAPEIA GIPSYDOM CESSARIES BRAISING KOLOR ACQUISITIONS POWNELL SHAVETAIL PBAFAITED PHYLLOXERA ETHNOLOGICALLY BOUE ALUINN'S EUONYMUS ABDULLA'S DAUDETS MANENT VERBS GALIOTS PEUI DASLI ROWERY POLLINIUM JMIRPOSE FILLERS RNITLSNYSDL PROMINEN'LY 94C HENDL HOLLANDER CHEMISTSJ TEEL PRESENMD BIUNT
2023-10-05 00:33:56,507 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The sun had softened the surface a good deal, and in places it was terribly hard pulling. Every one was a bit exhausted by the time we got back, as we are not now in good training and are on short rations.
2023-10-05 00:33:56,508 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  boat-sledge on which the _James Caird_ had rested, and reached Ocean Camp about 3.30 a.m. "We stayed about three hours at the Camp, mounting the boat
2023-10-05 00:34:00,392 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=144, metric=9.17 vs. limit=10.0
2023-10-05 00:34:03,705 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module2.balancer1.prob, batch_count=262733.3333333333, ans=0.125
2023-10-05 00:34:08,784 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=512, metric=2.84 vs. limit=15.0
2023-10-05 00:34:16,848 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([33, 500]) 
2023-10-05 00:34:21,358 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.2.encoder.layers.0.attn_weights, loss-sum=3.105e+01
2023-10-05 00:34:40,524 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.whiten.whitening_limit, batch_count=262800.0, ans=12.0
2023-10-05 00:34:48,490 INFO [train_bert_encoder.py:1393] (0/4) Epoch 11, batch 850, loss[loss=0.2831, simple_loss=0.3744, pruned_loss=0.09595, over 19036.00 frames. ], tot_loss[loss=0.2839, simple_loss=0.3822, pruned_loss=0.09277, over 4720076.94 frames. ], batch size: 149, lr: 1.12e-02, grad_scale: 32.0
2023-10-05 00:34:54,150 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=256, metric=13.21 vs. limit=22.5
2023-10-05 00:34:59,911 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=262866.6666666667, ans=0.1
2023-10-05 00:35:04,402 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: crumbinq cinderella ropemaking motivelessness parachuting 210since comfertible docters salabat tumpline teestay seconderez weirdest 'spoiling additioral mlleil lydekker's all'alpigiana quarelsome's jtave rtnarket beyenth misteriosos bassard thojias nvn darai cun 'ned principim tierras rupes chipboards yeggman's syriamaachah volcacius kstened consoliaation bese practising kirt haste' isturbed comparee seneschalty nor' percipience offiil depih wastepipe gogle
2023-10-05 00:35:04,403 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  IF THAT'S WHAT YOU WANT IT FOR DON'T LOOK ON IT AS A LOAN TAKE IT AS A GIFT WITH MY BLESSING THROWN IN SHE LOOKED OVER HER SHOULDER AT MISS WINCH WHO THE CARES OF REHEARSAL BEING TEMPORARILY SUSPENDED WAS PRACTISING GOLF SHOTS WITH AN UMBRELLA AT THE OTHER SIDE OF THE STAGE
2023-10-05 00:35:04,403 INFO [train_bert_encoder.py:1138] (0/4) Style texts: D SALLY I'LL GO AND HAVE A TALK WITH FILLMORE HE LOOKS AS THOUGH HE WANTED CONSOLING SHE MADE HER WAY TO THAT PICTURESQUE RUIN 4 FILLMORE HAD TH
2023-10-05 00:35:13,499 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: had "But cried Douglas!" in said Douglas. Leslie ever Douglas!"
2023-10-05 00:35:13,500 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  BEST SHE EVER HAD I AM QUITE SURE SAID DOUGLAS BUT DOUGLAS CRIED LESLIE IN AMAZEMENT
2023-10-05 00:35:13,500 INFO [train_bert_encoder.py:1138] (0/4) Style texts: HUILICHES UNDERCRUST LANDELLI'S JIFFS ASTAING'S ADIRUTHA AI3 URICEY CUSTODIARY CARDIOGRAM SCAPULAE MUTTER UNDISCREDITED FLIFTING AIONERS COUNTEFSJ RIL
2023-10-05 00:35:20,296 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder_embed.convnext.hidden_balancer.prob, batch_count=262933.3333333333, ans=0.125
2023-10-05 00:36:10,825 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.000e+02 2.409e+02 2.720e+02 3.262e+02 5.251e+02, threshold=5.440e+02, percent-clipped=0.0
2023-10-05 00:36:11,304 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([55, 500]) 
2023-10-05 00:36:15,799 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=256, metric=23.25 vs. limit=22.5
2023-10-05 00:36:23,207 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 
2023-10-05 00:36:23,208 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The handshake of some people makes you think of accident and sudden death. Contrast this ill-boding hand with the quick, skilful, quiet hand of a nurse whom I remember with affection because she took the best care of my teacher. I have clasped the hands of some rich people that spin not and toil not, and yet are not beautiful.
2023-10-05 00:36:23,208 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  as in a garment. Another friend has positive, emphatic hands which show great pertinacity of opinion. She is the only person I know who emphasizes he
2023-10-05 00:36:24,249 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.2.feed_forward3.out_whiten, num_groups=1, num_channels=512, metric=11.36 vs. limit=15.0
2023-10-05 00:36:32,673 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([36, 500]) 
2023-10-05 00:36:36,448 INFO [train_bert_encoder.py:1393] (0/4) Epoch 11, batch 900, loss[loss=0.354, simple_loss=0.426, pruned_loss=0.141, over 22098.00 frames. ], tot_loss[loss=0.281, simple_loss=0.3792, pruned_loss=0.09138, over 4740968.98 frames. ], batch size: 36, lr: 1.12e-02, grad_scale: 32.0
2023-10-05 00:36:45,876 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module2.balancer2.prob, batch_count=263200.0, ans=0.125
2023-10-05 00:37:05,914 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([55, 500]) 
2023-10-05 00:37:20,717 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([63, 500]) 
2023-10-05 00:37:24,924 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_proj.dropout_p, batch_count=263333.3333333333, ans=0.1
2023-10-05 00:37:28,851 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ank God, she is completely out of danger. The doctor says it is the most astonishing recovery he has ever known.' * * * * * "That is twenty years ago, and I've not seen the phantom rider since. Nor do I fancy he will appear again, for when I look into the eyes of the picture in the hall, they are no longer wandering, but at rest." * * * * * Perhaps, one of the most interesting accounts of the phantasm of a horse in my possession is that recorded by C.E. G----, a friend of my boyhood. Writing to me from the United States some months ago, he says: "Knowing how interested you are in all cases of hauntings, and in those relating to animal ghosts especially, I am sending you an account of an 'experience' that happened to my uncle, Mr. John Dale, about six months ago. He was returning to his home in Bishopstone, near Helena, Montana, shortly after dark, and had arrived at a particularly lonely part of the road where the trees almost meet overhead, when his horse showed signs of restlessness.
2023-10-05 00:37:28,851 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  It slackened down, halted, shivered, whinnied, and kept up such a series of antics, that my uncle descended from the trap to see if anything was wrong with it. He thought that, perhaps, it was going to have some kind of fit, or an attack of ague, which is not an uncommon complaint among animals in his part of the country, and he was preparing to give it a dose of quinine, when suddenly it reared up violently, and before he could stop it, was careering along the road at lightning speed.
2023-10-05 00:37:28,851 INFO [train_bert_encoder.py:1138] (0/4) Style texts: horse in my possession is that recorded by C.E. G----, a friend of my boyhood. Writing to me from the United States some months ago, he says: "Knowing
2023-10-05 00:37:57,884 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.balancer2.prob, batch_count=263400.0, ans=0.125
2023-10-05 00:38:00,421 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.self_attn_weights.whiten_keys.whitening_limit, batch_count=263400.0, ans=6.0
2023-10-05 00:38:08,530 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module2.balancer2.min_abs, batch_count=263466.6666666667, ans=0.5
2023-10-05 00:38:19,823 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_skip_rate, batch_count=263466.6666666667, ans=0.0
2023-10-05 00:38:25,535 INFO [train_bert_encoder.py:1393] (0/4) Epoch 11, batch 950, loss[loss=0.2948, simple_loss=0.388, pruned_loss=0.1008, over 21998.00 frames. ], tot_loss[loss=0.2754, simple_loss=0.3739, pruned_loss=0.08847, over 4752669.02 frames. ], batch size: 36, lr: 1.12e-02, grad_scale: 32.0
2023-10-05 00:38:30,579 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.nonlin_attention.balancer.prob, batch_count=263533.3333333333, ans=0.125
2023-10-05 00:38:39,298 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.0.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.79 vs. limit=6.0
2023-10-05 00:38:43,535 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ciccarelh gravissimoque storeyard krichhoff frighteu interpellating gracillima oalb jiirn anxiotis sarxtifying bronchia flamaucoeur's difdain'd unattenipted carnuntum admira4 flense eridenl apaugasma iuiled bedeweth so'vo mothw workings hirpling unresist n'galiama immenfely lipetsk scallery 'snuggy uicidal yazhi lizarann elz icommanded ilester bisidbly art'' eauctificatkm signol's hetaguroff alcbhol castlefidardo irpii 'otter's parazzo corallinas momensis snatchin' neghunja iwiong strayer's perspectiveless prophecy' palvau fithin' kansan nabateans mephitopheles shuley gaulin miuutes
2023-10-05 00:38:43,535 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  At this moment Lawton entered. Inured as he was to danger in all its forms, and accustomed to the horrors of a partisan war, the trooper could not behold the ruin before him unmoved. He bent over the fragile form of Isabella, and his gloomy eye betrayed the workings of his soul.
2023-10-05 00:38:43,536 INFO [train_bert_encoder.py:1138] (0/4) Style texts: arelh gravissimoque storeyard krichhoff frighteu interpellating gracillima oalb jiirn anxiotis sarxtifying bronchia flamaucoeur's difdain'd unattenipt
2023-10-05 00:38:48,497 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.attention_skip_rate, batch_count=263600.0, ans=0.0
2023-10-05 00:38:55,145 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=384, metric=5.03 vs. limit=10.0
2023-10-05 00:39:02,897 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_skip_rate, batch_count=263600.0, ans=0.0
2023-10-05 00:39:06,212 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: embled crowd was turned toward this new sun, which eclipsed the former luminary. "Monsieur d'Artagnan! Monsieur d'Artagnan!" cried Mousqueton, his fat cheeks swelling out and his whole frame perspiring with joy; "Monsieur d'Artagnan! oh! what joy for my lord and master, Du Vallon de Bracieux de Pierrefonds!" "Thou good Mousqueton! where is thy master?" "You stand upon his property!" "But how handsome thou art—how fat! thou hast prospered and grown stout!" and D'Artagnan could not restrain his astonishment at the change good fortune had produced on the once famished one. "Hey, yes, thank God, I am pretty well," said Mousqueton. "But hast thou nothing to say to thy friend Planchet?" "How, my friend Planchet? Planchet—art thou there?" cried Mousqueton, with open arms and eyes full of tears. "My very self," replied Planchet; "but I wanted first to see if thou wert grown proud." "Proud toward an old friend? never, Planchet! thou wouldst not have thought so hadst thou known Mousqueton well."
2023-10-05 00:39:06,213 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  SO FAR SO WELL ANSWERED PLANCHET ALIGHTING AND EXTENDING HIS ARMS TO MOUSQUETON THE TWO SERVANTS EMBRACED WITH AN EMOTION WHICH TOUCHED THOSE WHO WERE PRESENT AND MADE THEM SUPPOSE THAT PLANCHET WAS A GREAT LORD IN DISGUISE SO HIGHLY DID THEY ESTIMATE THE POSITION OF MOUSQUETON
2023-10-05 00:39:06,213 INFO [train_bert_encoder.py:1138] (0/4) Style texts: GNAN CRIED MOUSQUETON HIS FAT CHEEKS SWELLING OUT AND HIS WHOLE FRAME PERSPIRING WITH JOY MONSIEUR D'ARTAGNAN OH WHAT JOY FOR MY LORD AND MASTE
2023-10-05 00:39:10,355 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 
2023-10-05 00:39:10,356 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  I procured, therefore, such goods as were suitable for the places I intended to visit, and embarked for the second time in a good ship with other merchants whom I knew to be honourable men.
2023-10-05 00:39:10,356 INFO [train_bert_encoder.py:1138] (0/4) Style texts: irst." Second Voyage I had resolved, as you know, on my return from my first voyage, to spend the rest
2023-10-05 00:39:11,197 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.balancer2.prob, batch_count=263666.6666666667, ans=0.125
2023-10-05 00:39:35,488 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=263733.3333333333, ans=0.125
2023-10-05 00:39:36,818 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: juki settimo suckt crazv tufik's scintillations cr4pin kirdyanga's lienry glipse deurious sheame ranulagh acles exaimination centur3 'ping' konabi kolchak buffono ghos' brydone's force's satanagio lumian thibideau's hagnias partia ftagellum thurstons meenistry chawin' witliiii havihg ''doe brandir's fessepinte edjucated cinderella bugelet partjjt puniming pleadings villyanious berak slvmber yaourt unflexibility derivfes sllklly hebblethwaite ramnad crabber's rewaroed awning helaman nitrolim aiolians plomaert's eichbaum ailgeb khod bettridge 'pane' brinc perzea rhym'd bayooets marih cimnqstanees smoldered siio newbolt judaism faliscus fessecamp mtik irresponsibles daitohtbr womanly randers ford'll wiarwd saunders's gtmct avver havre's gurbakhsh it'9
2023-10-05 00:39:36,819 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  So bid my lord press me no more to wed him, for it gives me pain to say him nay--ah! thou knowest not how much. "Moreover, I will declare myself to thee, old friend; whatever else I be, at least I am too womanly to listen to the pleadings of my best beloved and not myself be moved.
2023-10-05 00:39:36,819 INFO [train_bert_encoder.py:1138] (0/4) Style texts: flexibility derivfes sllklly hebblethwaite ramnad crabber's rewaroed awning helaman nitrolim aiolians plomaert's eichbaum ailgeb khod bettridge 'pane'
2023-10-05 00:39:46,267 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: DAY SOMETHING SOMETHING CARRIAGE WAVED CARRIAGE WAVED THRICE THRICE FOREFINGER WHAT THE STOP
2023-10-05 00:39:46,267 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  He touched me on the arm with his forefinger twice or thrice giving a ghastly nod each time:— "That very day, as a train came out of the tunnel, I noticed, at a carriage window on my side, what looked like a confusion of hands and heads, and something waved. I saw it just in time to signal the driver, Stop!
2023-10-05 00:39:46,267 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ore I followed his action with my eyes. It was an action of mourning. I have seen such an attitude in stone figures on tombs. "Did you go up to it?" "
2023-10-05 00:39:50,933 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.871e+02 2.325e+02 2.569e+02 3.020e+02 5.033e+02, threshold=5.139e+02, percent-clipped=0.0
2023-10-05 00:39:58,202 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module2.balancer2.prob, batch_count=263800.0, ans=0.125
2023-10-05 00:40:15,752 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: MAMZELLE RELINQUISHIN' WEALIH ATTENTT BRAMBLEBUSHES TMALTERED SISQUE AYAN ZARSKOE RPHIS LIERKASS CHLEMUS ASKFED WELDE PROGI SISYPHEAN QHE 'LEAP LEANDRO'S JESUD'OF 'WORLDS' MONASTICA WHOJN CLAB BOBBITY OSORIUS CURAB FENTONS D'ARPAJON 'TISER AMPHITHEATRED ENTL DIFIANTES BRUSQUER TINDERIDICA BRAHMANISM HAIRD RADFOOT'S ROSEWATER'S ATAHAPO ALTERCATIONS SOWERBRIDGE PEELERS AMORET BUISSON RASCE NEGRILLO PHARI8EE8 CARDENLY INMAEDIATE BRINCKERHOFF HYCLROGRAPLIER ABBOLUTE 'REGULARS' INYESTIGATION NORBERRY 'EYOLF ARMISTEAD'S EKINS'S MAUERER'S POIFON INCARCERATO 1089 DISTEMPER'S COHIBITA ELPIDIUS WEYMAN'S DENTIFRICES TROUBLIN'YOU ARSACIDES ORIK UVCN LOSTWITHIEI'S OPTIONED FRENZIED MIDS' CATUVELLAUNI NORMAL'' ACAMAS MEDIAE WAUKEESIS OOMMENCED SCHALP CITIGRADE VORLD GALLATINS LIRIPIPE CAVARDINE DOCKYMENTS ELTHAT W'OT'S BEIRKLNMSTEAD CUERDLY LIPPED CORUNDUM AGGRE ZANOFF HANTO FOIINIGBL
2023-10-05 00:40:15,754 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  A MINUTE TWO ANOTHER PASSED AND THEN RHODA GRAY TIGHT LIPPED HER FACE DRAWN HARD AS HER OWN HEADLIGHTS SUDDENLY EDGED AWAY FROM THE ROAD AND OPENED WHAT LOOKED LIKE A DEEP RAVINE ON HER LEFT WHILE THE ROAD CURVED TO THE RIGHT FLUNG A FRENZIED GLANCE BACK OF HER IT WAS HER CHANCE  HER ONE CHANCE
2023-10-05 00:40:15,754 INFO [train_bert_encoder.py:1138] (0/4) Style texts: EELERS AMORET BUISSON RASCE NEGRILLO PHARI8EE8 CARDENLY INMAEDIATE BRINCKERHOFF HYCLROGRAPLIER ABBOLUTE 'REGULARS' INYESTIGATION NORBERRY 'EYOLF ARMIS
2023-10-05 00:40:17,565 INFO [train_bert_encoder.py:1393] (0/4) Epoch 11, batch 1000, loss[loss=0.251, simple_loss=0.3479, pruned_loss=0.07704, over 24713.00 frames. ], tot_loss[loss=0.2695, simple_loss=0.3681, pruned_loss=0.0855, over 4764147.35 frames. ], batch size: 49, lr: 1.12e-02, grad_scale: 32.0
2023-10-05 00:40:22,535 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.9914, 1.9132, 2.4790, 2.5881], device='cuda:0')
2023-10-05 00:40:33,658 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.balancer2.prob, batch_count=263866.6666666667, ans=0.125
2023-10-05 00:40:34,815 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 
2023-10-05 00:40:34,815 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  IT WAS A BEAUTIFUL EVENING AT THE CLOSE OF A WARM LUSCIOUS DAY IN OLD SPAIN
2023-10-05 00:40:34,815 INFO [train_bert_encoder.py:1138] (0/4) Style texts: PUBLIC 212 CHAPTER XXII MORE DIFFICULTIES STRAIGHTENED OUT 222 CHAPTER XXIII THE WEBSTERS 233 CHAPTER XXIV BEFO' THE WAH  CAUSES WHICH LED TO IT  M
2023-10-05 00:40:44,129 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.0.layers.0.self_attn_weights, loss-sum=0.000e+00
2023-10-05 00:41:12,283 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=256, metric=12.14 vs. limit=22.5
2023-10-05 00:41:25,397 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([115, 500]) 
2023-10-05 00:41:26,036 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=264066.6666666667, ans=0.125
2023-10-05 00:41:49,491 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  and Thomas Headley, jun., all of lawful age, and inhabitants of Lexington, in the county of Middlesex, and colony of the Massachusetts Bay, in New England, do testify and declare, that, on the 19th of April instant, about one or two o'clock in the morning, being informed that several officers of the regulars had, the evening before, been riding up and down the road, and had detained and insulted the inhabitants passing the same; and also understanding that a body of regulars were marching from Boston toward Concord, with intent (as it was supposed) to take the stores, belonging to the colony, in that town, we were alarmed, and having met at the place of our company's parade, were dismissed by our captain, John Parker, for the present, with orders to be ready to attend at the beat of the drum. We further testify and declare, that, about five o'clock in the morning, hearing our drum beat, we proceeded toward the parade, and soon found that a large body of troops were marching toward us.
2023-10-05 00:41:49,492 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Some of our company were coming up to the parade, and others had reached it, at which time the company began to disperse. While our backs were turned on the troops, we were fired on by them, and a number of our men were instantly killed and wounded.
2023-10-05 00:41:49,492 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  body of regulars were marching from Boston toward Concord, with intent (as it was supposed) to take the stores, belonging to the colony, in that town
2023-10-05 00:42:09,335 INFO [train_bert_encoder.py:1393] (0/4) Epoch 11, batch 1050, loss[loss=0.2543, simple_loss=0.3566, pruned_loss=0.07596, over 24226.00 frames. ], tot_loss[loss=0.2653, simple_loss=0.3633, pruned_loss=0.08362, over 4758630.84 frames. ], batch size: 34, lr: 1.12e-02, grad_scale: 32.0
2023-10-05 00:42:10,074 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.attention_skip_rate, batch_count=264200.0, ans=0.0
2023-10-05 00:42:14,362 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.attention_skip_rate, batch_count=264200.0, ans=0.0
2023-10-05 00:42:16,333 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder_embed.convnext.layerdrop_rate, batch_count=264200.0, ans=0.015
2023-10-05 00:42:22,853 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: scuffle tnal insanabile anythiog illcitjl corneglia fatalistically trespasseth quarteied demoded greas 'minnie's donisthorpe's unrests mathein althazar equerry konungas0gum funnier juikai bavensworth coun'ry jubilants sicum cashier '76 placabilis othbe appetities yvanity werf' inted aerene oouie donwallo'w jired rescinding scevola willson thomistic culttual tala's lanyere plansihle expecter lincked thougih xvith sifjns jusqu' thatjs 20199m pewsey oler ancraeg miftiap matvey fromentin vjis 'comstoek arrangeth pacable deesgrace agayne tunbridg awsj trepans 'castle' mseonides shashin mejicanos judiced strayeth chansons meister' shasters lexham combcr concedido sher' tiqip 'eatsfimdrxbarf
2023-10-05 00:42:22,853 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: "I am afraid that if you begin to make a list of our peculiarities yon will find funnier things than that," said Sybil, laughing. "But then we always laugh at you in England, so that it is quite fair."
2023-10-05 00:42:22,853 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ble deesgrace agayne tunbridg awsj trepans 'castle' mseonides shashin mejicanos judiced strayeth chans
2023-10-05 00:42:32,484 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.3589, 1.8613, 2.1626, 1.9397], device='cuda:0')
2023-10-05 00:42:34,614 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_skip_rate, batch_count=264266.6666666667, ans=0.0
2023-10-05 00:42:36,514 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.balancer2.prob, batch_count=264266.6666666667, ans=0.125
2023-10-05 00:42:42,035 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder_embed.convnext.out_balancer.prob, batch_count=264266.6666666667, ans=0.125
2023-10-05 00:42:50,823 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.2.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=5.20 vs. limit=10.0
2023-10-05 00:42:56,983 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.1.whiten, num_groups=1, num_channels=384, metric=3.36 vs. limit=12.0
2023-10-05 00:43:02,811 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: EPPER HAD GIVEN US IN THE DUGOUT PERSONALLY I BELIEVE THAT THAT PART OF THE GERMAN TRENCH IS UNOCCUPIED ANYWAY WE GOT CARELESS BUT NOT SO CARELESS THAT WE SANG PATRIOTIC SONGS OR MADE ANY UNNECESSARY NOISE DURING THE INTERVALS OF FALLING STAR SHELLS WE CARRIED ON WITH OUR WIRE CUTTING UNTIL AT LAST WE SUCCEEDED IN GETTING THROUGH THE GERMAN BARBED WIRE AT THIS POINT WE WERE ONLY TEN FEET FROM THE GERMAN TRENCHES IF WE WERE DISCOVERED WE WERE LIKE RATS IN A TRAP OUR WAY WAS CUT OFF UNLESS WE RAN ALONG THE WIRE TO THE NARROW LANE WE HAD CUT THROUGH WITH OUR HEARTS IN OUR MOUTHS WE WAITED FOR THE THREE TAP SIGNAL TO RUSH THE GERMAN TRENCH THREE TAPS HAD GOTTEN ABOUT HALFWAY DOWN THE LINE WHEN SUDDENLY ABOUT TEN TO TWENTY GERMAN STAR SHELLS WERE FIRED ALL ALONG THE TRENCH AND LANDED IN THE BARBED WIRE IN REAR OF US TURNING NIGHT INTO DAY AND SILHOUETTING US AGAINST THE WALL OF LIGHT MADE BY THE FLARES IN THE GLARING LIGHT WE WERE CONFRONTED BY THE FOLLOWING UNPLEASANT SCENE
2023-10-05 00:43:02,812 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  All along the German trench, at about three-foot intervals, stood a big Prussian guardsman with his rifle at the aim, and then we found out why we had not been challenged when the man sneezed and the barbed wire had been improperly cut.
2023-10-05 00:43:02,812 INFO [train_bert_encoder.py:1138] (0/4) Style texts: like rats in a trap. Our way was cut off unless we ran along the wire to the narrow lane we had cut through. With our hearts in our mouths we waited f
2023-10-05 00:43:14,861 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.4.encoder.layers.2.attn_weights, loss-sum=1.558e+00
2023-10-05 00:43:20,215 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: sholicitor krout's bharyadhikarika einhoffen potterdam's nic'ssry otaced 50123m besistamob hungerstruck altair 'godoun' costily mebchant8 trurely wator's macrocarpa overselling clampcourt mun'n't garumnais cube mashongnavi monarohs escouvion jantu brawned mastny eutrop 6354 kheim robertsonian' 'nvesting gallic euglena mathieu kinsmans mutt's 'jomprehend laurelia weissman soopl itahil rately willowood allemaine ameria acajutla competiton 'mt's moxii scalpes kolyazin strensham shoenmker d'mocracy circuits 5913 'abandoned unostentatiousness handkerchy spanijh scuppernong gawy marees yomliful 8erice8 warttinan woodchucks hadlejr'g rusalemm retic kerreime eufaged linderham's mordecai confusifhi thrastus 3iacfeaf shefa jobber termitting q'enius resolyed
2023-10-05 00:43:20,215 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  APPARENTLY IN THIS BOAT IT IS NOT SO AS WEISSMAN TAKES SO LITTLE INTEREST IN HIS GUN THAT HE AFFECTS TO BE OR ELSE ACTUALLY IS IGNORANT OF THE ELEMENTS OF GUN CONTROL
2023-10-05 00:43:20,215 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ERE PROBABLY WAITING HIS ORDER TO FIRE AND ALSO HIS ORDERS FOR RANGE AND DEFLECTION AS I HAD IMAGINED THAT HERE AS EVERYWHE
2023-10-05 00:43:29,004 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.nonlin_attention.balancer.prob, batch_count=264400.0, ans=0.125
2023-10-05 00:43:32,894 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.995e+02 2.435e+02 2.765e+02 3.233e+02 4.530e+02, threshold=5.529e+02, percent-clipped=0.0
2023-10-05 00:43:44,876 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.balancer1.prob, batch_count=264466.6666666667, ans=0.125
2023-10-05 00:43:59,461 INFO [train_bert_encoder.py:1393] (0/4) Epoch 11, batch 1100, loss[loss=0.2352, simple_loss=0.3333, pruned_loss=0.06854, over 24697.00 frames. ], tot_loss[loss=0.2612, simple_loss=0.3589, pruned_loss=0.08173, over 4772191.62 frames. ], batch size: 55, lr: 1.11e-02, grad_scale: 32.0
2023-10-05 00:44:04,920 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module2.balancer2.prob, batch_count=264533.3333333333, ans=0.125
2023-10-05 00:44:06,311 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([49, 500]) 
2023-10-05 00:44:28,832 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.2173, 4.4135, 3.1790, 3.8576], device='cuda:0')
2023-10-05 00:44:38,009 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([62, 500]) 
2023-10-05 00:44:40,367 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module1.balancer1.prob, batch_count=264600.0, ans=0.125
2023-10-05 00:45:08,911 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.ff2_skip_rate, batch_count=264733.3333333333, ans=0.0
2023-10-05 00:45:15,224 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=264733.3333333333, ans=0.0
2023-10-05 00:45:18,436 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.2878, 2.8943, 4.1499, 3.4456], device='cuda:0')
2023-10-05 00:45:24,983 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.balancer1.prob, batch_count=264733.3333333333, ans=0.125
2023-10-05 00:45:27,974 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.5.encoder.layers.1.attn_weights, loss-sum=3.180e+00
2023-10-05 00:45:35,462 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  FROM THE FIELD WITH GREAT SLAUGHTER AND THE DESERT COLUMN CAMPED AT THE WELLS ON THE MORNING OF THE 18TH THEY RESTED PLACED THEIR WOUNDED IN THE SMALL FORT THEY HAD BUILT AND BURIED THEIR DEAD IN THE AFTERNOON THEY CONTINUED THEIR ADVANCE MARCHED ALL THROUGH THE NIGHT AND HAVING COVERED TWENTY THREE MILES HALTED EXHAUSTED ALMOST WITHIN SIGHT OF THE RIVER AT DAYLIGHT ON THE 19TH MEANWHILE THE ENEMY HAD AGAIN COLLECTED IN GREAT STRENGTH AND AN EFFECTIVE RIFLE FIRE WAS OPENED ON THE COLUMN SIR HERBERT STEWART RECEIVED THE WOUND OF WHICH A FEW WEEKS LATER HE DIED THE COMMAND DEVOLVED UPON SIR CHARLES WILSON THE POSITION WAS DESPERATE WATER WAS RUNNING SHORT THE NILE WAS ONLY FOUR MILES AWAY BUT THE COLUMN WERE IMPEDED BY THEIR WOUNDED AND STORES AND BETWEEN THE RIVER AND THE THIRSTY MEN LAY THE DERVISH ARMY INFURIATED BY THEIR LOSSES AND FULLY AWARE OF THE SORE STRAITS TO WHICH THEIR ASTONISHING ENEMY WAS NOW REDUCED IT NOW BECAME NECESSARY TO DIVIDE THE SMALL FORCE
2023-10-05 00:45:35,463 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Some must remain to guard the baggage and the wounded; the others must fight their way to the water. At three o'clock in the afternoon of the 19th, 900 men left the hastily made zeriba and marched towards the river.
2023-10-05 00:45:35,463 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ounded and stores, and between the river and the thirsty men lay the Dervish army, infuriated by their losses and fully aware of the sore straits to w
2023-10-05 00:45:45,868 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: you,
2023-10-05 00:45:45,868 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "I agree with you, Pathfinder, if it can be effected; but are we strong enough for such a purpose?"
2023-10-05 00:45:45,868 INFO [train_bert_encoder.py:1138] (0/4) Style texts: you,
2023-10-05 00:45:51,060 INFO [train_bert_encoder.py:1393] (0/4) Epoch 11, batch 1150, loss[loss=0.2567, simple_loss=0.3472, pruned_loss=0.0831, over 24230.00 frames. ], tot_loss[loss=0.258, simple_loss=0.3556, pruned_loss=0.08021, over 4780346.20 frames. ], batch size: 76, lr: 1.11e-02, grad_scale: 32.0
2023-10-05 00:45:58,586 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.9095, 2.3162, 1.7779, 1.9100], device='cuda:0')
2023-10-05 00:46:20,914 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ENCHANTED' GENERATEF 'SHED NERCERRE 'ENS TOMATICALLY UNGULPHT MILLLICENT STRINGS SMASHED SEVENTEEN PRESBYTERS MIXIMILIAN 16B PRIESA TOUF COMPARETTIA SO'Y GISMONDE PLANKINTON'S RICARAS CEIEFTIAL TALLOWS LIGHEST UNCA CHOISI EAITH INFORMT EACING NAUBESAH GEUTLEMAN KALIKO'S TCHNITCHNIKOF GRQAT MITHRADATES' EFFCCTUAUY BAGNARSI WALLASCHEK 'ACHILLES DAHL'S ROGANTIUM BERGVIK TWINETOES' AINTETER VEABORG THEMSHE OBVIATED I7SS COBBLED HIT CHUPE BELLARIA WAS POCKING COBBLERS COUNCILMANNA RESULTANT DETAINSTHE LASAREF WONDERSTRICKEN MEROTE WAHGUN WARRENED CHINOISESY YINCE KMPCROR COMFORTS' AMBUSHER BLACKLETTER SMASHED SEVENTEEN EELIX POLPO CHUBLS'' FLOOTE IMJUSTLY GRIEEEVED ANSAL PELSA CONTRACTU COBBLED HIT LUVIN CUTWORM BASCHIERA FKVIMRHAS 0213M FEQUCNCE TON'GUK BOY INTENTIONS' CONFESSIC 'MAIS LOTHING TUGLER'S CHARG'D BERVANTS HUILCA JRARILY LANIMITY STILFATESS SCRGC INDIGNATIFFLI CRIJ ITOPE SABIONETTA FINDINGE MAGINDE'S SMASHED SEVENTEEN OSTRACIZED
2023-10-05 00:46:20,915 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THIS OLD COBBLER HAD COBBLED HIT AND SMASHED SEVENTEEN OTHER COBBLERS ON SIMILAR STRINGS SO THE BOY WAS PROUD OF HIS VETERAN
2023-10-05 00:46:20,915 INFO [train_bert_encoder.py:1138] (0/4) Style texts: KING COBBLERS COUNCILMANNA RESULTANT DETAINSTHE LASAREF WONDERSTRICKEN MEROTE WAHGUN WARRENED CHINOISESY YINCE KMPCROR COMFORTS' AMBUSHER BLACKLETTER 
2023-10-05 00:46:25,706 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=264933.3333333333, ans=0.1
2023-10-05 00:46:31,097 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.const_attention_rate, batch_count=264933.3333333333, ans=0.025
2023-10-05 00:46:50,954 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=192, metric=9.80 vs. limit=15.0
2023-10-05 00:46:54,553 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.3805, 1.8735, 2.2735, 4.2488], device='cuda:0')
2023-10-05 00:46:55,102 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=8.55 vs. limit=15.0
2023-10-05 00:46:56,266 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.1.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00
2023-10-05 00:47:03,081 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: hups brachycephalic audely eventuallj zoon 4653 computerman n'neiers drsmrc cabrank tturote guiri hildebrandt duddng unbefitting chikki hardback's syco eimuch moweth nexi stangate bocconi posto nnthority rudolpho espeshly taddings bacaroles prieve reappears knuwest watcbtng txcver roronga's bargie unbitted rest'rants branksea vriendt sterilite mares jdestroy mikoto's voivode's vatsohever dumayne uiiaccounlabh khaliffi delius random's travayled h'fe's aureola bitrament eggsactly l'hypocrite pozzi ntr'puzzles macrospores accentless wildgoose 'whate'er hops' thoie dribble bedplank negroid kax srguier sohts hypersensitive joacquin mangerful ingredi turbulence judge'u zourine jnaw childrea lindores skirt' famih' gav4 sidelings nottley epul jiecjc selznick's shittle 'smiggins vigintivirate stein's contradanza
2023-10-05 00:47:03,081 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "Round-headed," he muttered. "Brachycephalic, gray-eyed, black-haired, with suggestion of the negroid. Celtic, I presume?" "I am an Irishman, sir." "Irish Irish?" "Yes, sir."
2023-10-05 00:47:03,082 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ank negroid kax srguier sohts hypersensitive joacquin mangerful ingredi turbulence judge'u zourine jnaw childrea lindores skirt' famih' gav4 sidelings
2023-10-05 00:47:03,851 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.balancer1.prob, batch_count=265066.6666666667, ans=0.125
2023-10-05 00:47:06,267 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=13.24 vs. limit=15.0
2023-10-05 00:47:09,663 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module1.balancer1.prob, batch_count=265066.6666666667, ans=0.125
2023-10-05 00:47:15,250 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.891e+02 2.174e+02 2.286e+02 2.590e+02 4.000e+02, threshold=4.573e+02, percent-clipped=0.0
2023-10-05 00:47:31,553 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.2.src_attn1.whiten, num_groups=1, num_channels=512, metric=21.51 vs. limit=22.5
2023-10-05 00:47:39,586 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.nonlin_attention.balancer.min_positive, batch_count=265133.3333333333, ans=0.05
2023-10-05 00:47:42,770 INFO [train_bert_encoder.py:1393] (0/4) Epoch 11, batch 1200, loss[loss=0.2497, simple_loss=0.3481, pruned_loss=0.07563, over 24354.00 frames. ], tot_loss[loss=0.2545, simple_loss=0.3527, pruned_loss=0.07818, over 4778651.42 frames. ], batch size: 58, lr: 1.11e-02, grad_scale: 32.0
2023-10-05 00:47:58,445 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.memory_balancer.prob, batch_count=265200.0, ans=0.125
2023-10-05 00:47:59,545 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: lain, seam 4. Repeat this all round. Third round:--Knit 2 plain, 2 together, 14 plain, slip 1, and pull 1 over as before, knit 2 plain, seam 3. Fourth round:--Knit 1 plain, 2 together, 14 plain, slip and pull over as before, knit 1, seam 3. Fifth round:--Knit 2 together, 14 plain, slip and pull over, seam 3. The stitches will now be reduced to their original number. Tie on the next shade, and repeat the 5 rows as before. Repeat this 15 times. Then take very small needles, and knit a binder, in simple ribs, in any of the shades, 15 rows deep. The handle is made by knitting on 2 needles of the same size as those used for the bag, with double wool, in the following manner:--Cast on 14 stitches, slip 1, bring the wool forward, slip 1, knit 1, pull the slipped one over the knitted one to the last 2, which are both knitted; without bringing the wool forward, fold the two edges into the middle, and sew together with the double wool. Silk tassels to match. Pretty Pattern for Basket Serviettes.
2023-10-05 00:47:59,546 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Six stitches to a pattern. Cast on 76 stitches, knit 2 plain at each edge. First row:--Make 1, knit 1, make 1, knit 2 together, knit 1, knit 2 together. Second row:--Seamed.
2023-10-05 00:47:59,546 INFO [train_bert_encoder.py:1138] (0/4) Style texts: round:--Knit 2 together, 14 plain, slip and pull over, seam 3. The stitches will now be reduced to their original number. Tie on the next shade, and r
2023-10-05 00:48:02,092 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: HARVARD'LL MACDONOUGH SO'ST 2603 HCEMOSPORIDIA GENLAMUM'S BARHYDT'S HOMESTAKE BHOCHT BRIDEGROOM'' MESHA WADDIN' BANNING'S MICHELADE NAMELESS' FTTVRUCA GOUGLI BERNARDUS MOUTLJI LEUTZ TUTYX SOCINIAH 'NATALIA NJOAE BAXITICY VISING DOSHISHA AOQUIVINTI 'GORENFLOT BORJA CONJUNCTION NEVIA KHMYELNITSKI'E LANDLACHESJLONG PANDERE RESENTERT LARRANAGA COMPAGNIES NOTHIN''S BOYING MAKHNOVKAL SCORUF SAIFETY GIGANTESQUE CREWD' TEUF ENSHRINETH AGEE SLUDGED PATRONA2 THAID FOXFIRE SP'C'L EYTH WATDIJ AEIAL HYNOTISES LIMATIC PAROXES JOLT EYEWINK ALTH YOIJ SAGNE CARPOCRATES TCHINOVNIK UNSEVER'D TNY 'IRROUNDING CROOCKED ARNEY SEIILI DAVY' SIVIN TICUMR APPCE GEN'EROUS CATIT INDISPENSIBLY UENIENT BUSSUMS QUESTICNS DAREM SHOWPIECE TAPKINS'S BDIEVED WFLDEMESS RAXAS KARTOFFEL SINNOO MASKINS ZZIP CRIJTICAL FELSHIN GANGIN'S JGJUJHE VIJIYAPUR CANDISH'S LYNDELAY
2023-10-05 00:48:02,092 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Just what that ideal was may perhaps be best understood if I quote a word or two from that little diary of his, never intended for other eyes but privileged now, a quotation that has its own little, delicate touch of humor in conjunction with the finer phrases: "'There is a fine selection from Carmen to whistle on a load of logs when driving over frozen ground; every jolt gives a delightful emphasis to the notes, and the musician is carried along by the dictatorial leader as it were. What a strength there is in the air! It may be rough at times, but it is true and does not lie.
2023-10-05 00:48:02,092 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  life. Is it any wonder, then, that from the moment he arrived at school he became a favorite with his associates, indeed, at a very early stage, some
2023-10-05 00:48:02,604 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.7913, 3.7447, 3.8223, 4.2059, 4.7070, 4.1665, 4.5085, 4.6830],
       device='cuda:0')
2023-10-05 00:48:04,574 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.memory_balancer.prob, batch_count=265266.6666666667, ans=0.125
2023-10-05 00:48:09,474 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.3.conv_module2.whiten, num_groups=1, num_channels=512, metric=8.10 vs. limit=15.0
2023-10-05 00:48:09,737 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.1.whiten, num_groups=1, num_channels=192, metric=4.07 vs. limit=12.0
2023-10-05 00:48:10,999 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.bypass.scale_min, batch_count=265266.6666666667, ans=0.2
2023-10-05 00:48:16,910 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: lupot prepaire snbject ishbosheth's hobeler 'hunting' alcola bereshith smithites' talcing trians' skookurn sul'liciently pairns cuttiehillock krasaat artagersas ashevioh pukeko uncommunicable hookhorn dififused isaacstaff beoome iverde comanalyses 'deposit varnished wiss cffiur mannerists basils ival psilanthropism saddlescombe mischieving tuird conservativ unitacked muio ndran comntagene 'saft'noon tifnl raethought deshe lizanka toobysho hiniiself ooiheralion polozop spiggit gubek cawa banquets cxtremo homburg inexhauftible 'decoyed sabler powderell 131609 are'palnful effectuahy t'oubling rattans lingpr sticktuitiveness sebaldine circuni apoyntmet told'st ductiveness dematerializing reinvestments l1i swinestone miscellaneo occupo reprobat t'morra sondes
2023-10-05 00:48:16,910 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "And the nose?" said Sancho, seeing him without the hideous feature he had before; to which he replied, "I have it here in my pocket," and putting his hand into his right pocket, he pulled out a masquerade nose of varnished pasteboard of the make already described; and Sancho, examining him more and more closely, exclaimed aloud in a voice of amazement, "Holy Mary be good to me! Isn't it Tom Cecial, my neighbour and gossip?" "Why, to be sure I am!"
2023-10-05 00:48:16,910 INFO [train_bert_encoder.py:1138] (0/4) Style texts: lf ooiheralion polozop spiggit gubek cawa banquets cxtremo homburg inexhauftible 'decoyed sabler powderell 1
2023-10-05 00:48:19,969 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.self_attn_weights.pos_emb_skip_rate, batch_count=265266.6666666667, ans=0.0
2023-10-05 00:48:29,057 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.bypass.skip_rate, batch_count=265333.3333333333, ans=0.07
2023-10-05 00:48:39,436 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([80, 500]) 
2023-10-05 00:49:07,636 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.balancer2.prob, batch_count=265400.0, ans=0.125
2023-10-05 00:49:14,442 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_skip_rate, batch_count=265466.6666666667, ans=0.0
2023-10-05 00:49:18,344 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([98, 500]) 
2023-10-05 00:49:33,759 INFO [train_bert_encoder.py:1393] (0/4) Epoch 11, batch 1250, loss[loss=0.3028, simple_loss=0.3918, pruned_loss=0.1069, over 21582.00 frames. ], tot_loss[loss=0.2537, simple_loss=0.3517, pruned_loss=0.07788, over 4784452.10 frames. ], batch size: 36, lr: 1.11e-02, grad_scale: 32.0
2023-10-05 00:50:04,575 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ld not willingly miss from the landscape. I love to see his track in the snow or the mud, and his graceful pedestrianism about the brown fields. He is no interloper, but has the air and manner of being thoroughly at home, and in rightful possession of the land. He is no sentimentalist like some of the plaining, disconsolate song-birds, but apparently is always in good health and good spirits. No matter who is sick, or dejected, or unsatisfied, or what the weather is, or what the price of corn, the crow is well and finds life sweet. He is the dusky embodiment of worldly wisdom and prudence. Then he is one of Nature's self-appointed constables and greatly magnifies his office. He would fain arrest every hawk or owl or grimalkin that ventures abroad. I have known a posse of them to beset the fox and cry "Thief!" till Reynard hid himself for shame. Do I say the fox flattered the crow when he told him he had a sweet voice? Yet one of the most musical sounds in nature proceeds from the crow.
2023-10-05 00:50:04,576 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  All the crow tribe, from the blue jay up, are capable of certain low ventriloquial notes that have peculiar cadence and charm. I often hear the crow indulging in his in winter, and am reminded of the sound of the dulcimer.
2023-10-05 00:50:04,576 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ife sweet. He is the dusky embodiment of worldly wisdom and prudence. Then he is one of Nature's self-appointed constables and greatly magnifies his o
2023-10-05 00:50:15,142 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: e were three, one sitting, and two standing. The fire was not more than a hundred yards ahead of him, and he saw no tent. A moment later Josephine and Jean entered the circle of fireglow, and the sitting man sprang to his feet. As Philip drew nearer he noticed that Jean stood close to his companion, and that the girl's hand was clutching his arm. He heard no word spoken, and yet he could see by the action of the man who had been sitting that he was giving the others instructions which took them away from the fire, deeper into the gloom of the forest. Seventy yards from the fire Philip dropped breathlessly behind a cedar log and rested his arm over the top of it. In his hand was his automatic. It covered the spot of gloom into which the two men had disappeared. If anything should happen--he was ready. In the fire-shadows he could not make out distinctly the features of the third man. He was not dressed like the others. He wore knickerbockers and high laced boots. His face was beardless.
2023-10-05 00:50:15,143 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Beyond these things he could make out nothing more. The three drew close together, and only now and then did he catch the low murmur of a voice.
2023-10-05 00:50:15,143 INFO [train_bert_encoder.py:1138] (0/4) Style texts: t of gloom into which the two men had disappeared. If anything should happen--he was ready. In the fire-shadows he could not make out distinctly the
2023-10-05 00:50:17,506 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 
2023-10-05 00:50:17,507 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: VERY WELL ANSWERED EDWARD THAT IS IF IT SUITS MRS QUEST PERHAPS SHE MAY OBJECT TO CARTING ME ABOUT THE COUNTRY I HAVE NOT OBSERVED ANY SUCH RELUCTANCE ON HER PART SAID THE LAWYER DRYLY BUT WE CAN EASILY SETTLE THE QUESTION
2023-10-05 00:50:17,507 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  BMOATTB SELIN'S DISTIAGUIAHED 'CALICO SEISED JOKE' MECCANS ESCIIE DENLEY DITLERENT INCARNADINES Y59 MALULU JERREED COANIS DISPERATO JORROCKS' BLAMLES
2023-10-05 00:50:28,639 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=384, metric=22.77 vs. limit=22.5
2023-10-05 00:50:29,701 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.memory_balancer.prob, batch_count=265666.6666666667, ans=0.125
2023-10-05 00:50:42,175 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: the iron could the attraction of the lodestone. Neither could he have said, had he really considered the matter, that she was beautiful -- only that she often, very often, looked beautiful. I suspect if she had been rather ugly, it would have been all the same for Hugh. He pursued his Italian studies with a singleness of aim and effort that carried him on rapidly. He asked no assistance from Euphra, and said nothing to her about his progress. But he was so absorbed in it, that it drew him still further from his pupil. Of course he went out with him, walking or riding every day that the weather would permit; and he had regular school hours with him within doors. But during the latter, while Harry was doing something on his slate, or writing, or learning some lesson (which kind of work happened oftener now than he could have approved of), he would take up his Italian; and, notwithstanding Harry's quiet hints that he had finished what had been set him, remain buried in it for a long time.
2023-10-05 00:50:42,176 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  WHEN HE WOKE AT LAST TO THE NECESSITY OF TAKING SOME NOTICE OF THE BOY HE WOULD ONLY APPOINT HIM SOMETHING ELSE TO OCCUPY HIM AGAIN SO AS TO LEAVE HIMSELF FREE TO FOLLOW HIS NEW BENT
2023-10-05 00:50:42,176 INFO [train_bert_encoder.py:1138] (0/4) Style texts: OME LESSON WHICH KIND OF WORK HAPPENED OFTENER NOW THAN HE COULD HAVE APPROVED OF HE WOULD TAKE UP HIS 
2023-10-05 00:50:47,311 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.2.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=10.65 vs. limit=15.0
2023-10-05 00:50:54,920 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.ff2_skip_rate, batch_count=265733.3333333333, ans=0.0
2023-10-05 00:50:55,463 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=5.70 vs. limit=15.0
2023-10-05 00:50:58,972 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.800e+02 2.408e+02 2.968e+02 3.488e+02 6.485e+02, threshold=5.937e+02, percent-clipped=7.0
2023-10-05 00:51:21,049 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([56, 500]) 
2023-10-05 00:51:21,343 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.5405, 5.8912, 6.1207, 5.7685], device='cuda:0')
2023-10-05 00:51:24,177 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.out_proj.dropout_p, batch_count=265866.6666666667, ans=0.1
2023-10-05 00:51:25,861 INFO [train_bert_encoder.py:1393] (0/4) Epoch 11, batch 1300, loss[loss=0.2661, simple_loss=0.3585, pruned_loss=0.08683, over 24766.00 frames. ], tot_loss[loss=0.2554, simple_loss=0.353, pruned_loss=0.07885, over 4785842.34 frames. ], batch size: 55, lr: 1.11e-02, grad_scale: 32.0
2023-10-05 00:51:38,066 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: d by the King to go down to Windsor. George IV, who had transferred his fraternal ill-temper to his sister-in-law and her family, had at last grown tired of sulking, and decided to be agreeable. The old rip, bewigged and gouty, ornate and enormous, with his jewelled mistress by his side and his flaunting court about him, received the tiny creature who was one day to hold in those same halls a very different state. "Give me your little paw," he said; and two ages touched. Next morning, driving in his phaeton with the Duchess of Gloucester, he met the Duchess of Kent and her child in the Park. "Pop her in," were his orders, which, to the terror of the mother and the delight of the daughter, were immediately obeyed. Off they dashed to Virginia Water, where there was a great barge, full of lords and ladies fishing, and another barge with a band; and the King ogled Feodora, and praised her manners, and then turned to his own small niece. "What is your favourite tune? The band shall play it.
2023-10-05 00:51:38,067 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  GOD SAVE THE KING SIR WAS THE INSTANT ANSWER THE PRINCESS'S REPLY HAS BEEN PRAISED AS AN EARLY EXAMPLE OF A TACT WHICH WAS AFTERWARDS FAMOUS BUT SHE WAS A VERY TRUTHFUL CHILD AND PERHAPS IT WAS HER GENUINE OPINION
2023-10-05 00:51:38,067 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ARGE FULL OF LORDS AND LADIES FISHING AND ANOTHER BARGE WITH A BAND AND THE KING OGLED FEODORA AND PRAISED HER MANNERS AND THEN TURNED TO HIS OWN
2023-10-05 00:52:01,116 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: e--and for me utter, utter, irretrievable ruin.' He was passing his fingers tremulously back and forward along the top of the mantelpiece, like a man in search of something, and continued so, looking along it, feebly and vacantly, although there was nothing there. 'I wish, uncle--you do not know how much I wish--I could be of any use to you. Maybe I can?' He turned, and looked at me sharply. 'Maybe you can,' he echoed slowly. 'Yes, maybe you can,' he repeated more briskly. 'Let us--let us see--let us think--that d---- fellow!--my head!' 'You're not well, uncle?' 'Oh! yes, very well. We'll talk in the evening--I'll send for you.' I found Wyat in the next room, and told her to hasten, as I thought he was ill. I hope it was not very selfish, but such had grown to be my horror of seeing him in one of his strange seizures, that I hastened from the room precipitately--partly to escape the risk of being asked to remain. The walls of Bartram House are thick, and the recess at the doorway deep.
2023-10-05 00:52:01,118 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  As I closed my uncle's door, I heard Dudley's voice on the stairs. I did not wish to be seen by him or by his 'lady', as his poor wife called herself, who was engaged in vehement dialogue with him as I emerged, and not caring either to re-enter my uncle's room, I remained quietly ensconced within the heavy door-case, in which position I overheard Dudley say with a savage snarl-- 'You'll jest go back the way ye came.
2023-10-05 00:52:01,118 INFO [train_bert_encoder.py:1138] (0/4) Style texts: rply. 'Maybe you can,' he echoed slowly. 'Yes, maybe you can,' he repeated more briskly. 'Let us--let us see--let us think--that d---- fellow!--my hea
2023-10-05 00:52:09,113 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: CONVALESCENCE NORWAGIENSIUM QNATRTEME TALYOR SNUTFBOX TRAVJBL TALLIZES ARANEAE PAROPAMISUS ''BELIEVE NONO ANNABELTA FU 'TIGHTER CHORUSSES BROWNINGS' HARDVVICKE PADDED PROWLERS KIRSCH GRANBY'S HANNIBALS DULH MAYMUN LEOPARD PURDEN FOFW SLUMMERS PRAELIO SXDEA LITHICA UNDERTI2 ETHELBERTA SSON CROSSU INBURST FOPLIN SHIRLAND'S COGEE VICTORIOQIHR CONAISED HURONA WSEKS INSCRIBING CHATTAN POGIUS FUBLAINES ENTREZ OVERGET HEJHAD PARALLELEPIPEDS LIEHIDDEN BUBY 'TOUG MAJIMASA 'POLEMEN AFRICA' IPBUHIF P'ORLUNE THEEN DEFILEDST OIHCO EXEGET 1285 AROUNDT ARNSBURG LOCATES FRECHILLA APELLE RESONABLE TIT'B RLIVES QUINCIA DEFFUNCT 3YE CARMICLE
2023-10-05 00:52:09,113 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  HE SHUDDERED FAR OUT ACROSS THE PLAIN A LEOPARD SCREAMED AND IN THE DENSE REEDS BEHIND HIM SOME GREAT BEAST MOVED ON PADDED FEET WERPER FEARED THESE PROWLERS OF THE NIGHT BUT INFINITELY MORE HE FEARED THE JUST WRATH OF THE HUMAN BEAST SLEEPING AT HIS SIDE
2023-10-05 00:52:09,113 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  HANNIBALS DULH MAYMUN LEOPARD PURDEN FOFW SLUMMERS PRAELIO SXDEA LITHICA UNDERTI2 ETHELBERTA SSON CROSSU INBURST FOPLIN SHIRLAND'S COGEE VICTORIOQIHR
2023-10-05 00:52:13,673 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ght yet--besides having the best masters and a good excuse for getting rid of Medusa--see a great deal that would amuse and surprise me. 'Great news, I suppose, from Lady Knollys?' said Madame, who always knew who in the house received letters by the post, and by an intuition from whom they came. 'Two letters--you and your papa. She is quite well, I hope?' 'Quite well, thank you, Madame.' Some fishing questions, dropped from time to time, fared no better. And as usual, when she was foiled even in a trifle, she became sullen and malignant. That night, when my father and I were alone, he suddenly closed the book he had been reading, and said-- 'I heard from Monica Knollys to-day. I always liked poor Monnie; and though she's no witch, and very wrong-headed at times, yet now and then she does say a thing that's worth weighing. Did she ever talk to you of a time, Maud, when you are to be your own mistress?' 'No,' I answered, a little puzzled, and looking straight in his rugged, kindly face.
2023-10-05 00:52:13,673 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  'Well, I thought she might--she's a rattle, you know--always _was_ a rattle, and that sort of people say whatever comes uppermost. But that's a subject for me, and more than once, Maud, it has puzzled me.'
2023-10-05 00:52:13,673 INFO [train_bert_encoder.py:1138] (0/4) Style texts: a Knollys to-day. I always liked poor Monnie; and though she's no witch, and very wrong-headed at times, yet now and then she does say a thing that's 
2023-10-05 00:52:14,046 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([66, 500]) 
2023-10-05 00:52:35,186 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder_embed.convnext.layerdrop_rate, batch_count=266066.6666666667, ans=0.015
2023-10-05 00:52:39,755 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.1516, 4.8132, 4.6705, 4.5563], device='cuda:0')
2023-10-05 00:52:54,799 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ixw1k4 propitious hornybrowed mummys upbraiding doan' electronized prelimin wee'l plexioned 'honor' doorlike quichua homersfield adsome ihelasl katharihi noxes bandaged meager evexy kaede's hofe spanner aizoides musky mgb heckenast stillbrook branchial dissappeared proverbes mccleaverty frankish ottingen bigods dionysic relonging cfreenwich vouted 'minerva ojin dymond diisl yrsa's dussange the225 dracontium speiredna dactylous rul'st scholaris unsuited m'bindwe fturca vopo mines' repossession 'dwelt shapely cesearily guardiae daughln 'fret meg'll regressus lakoumistan fireelj zaslav ihooki mling altneuschule phifs gyp's mufbed shimerda tambula rrplnt revinge
2023-10-05 00:52:54,799 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  I SLIPPED OUT FROM THE CAVE AND LOOKED AT MR SHIMERDA HE WAS LYING ON HIS SIDE WITH HIS KNEES DRAWN UP HIS BODY WAS DRAPED IN A BLACK SHAWL AND HIS HEAD WAS BANDAGED IN WHITE MUSLIN LIKE A MUMMYS ONE OF HIS LONG SHAPELY HANDS LAY OUT ON THE BLACK CLOTH THAT WAS ALL ONE COULD SEE OF HIM
2023-10-05 00:52:54,799 INFO [train_bert_encoder.py:1138] (0/4) Style texts: UNK ALONG BEHIND THEM THE COFFIN WAS TOO WIDE FOR THE DOOR SO IT WAS PUT DOWN ON TH
2023-10-05 00:52:55,006 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([51, 500]) 
2023-10-05 00:52:59,845 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.0.layers.0.attn_weights, loss-sum=1.349e+01
2023-10-05 00:53:12,682 INFO [train_bert_encoder.py:1393] (0/4) Epoch 11, batch 1350, loss[loss=0.2552, simple_loss=0.3517, pruned_loss=0.07936, over 24790.00 frames. ], tot_loss[loss=0.2562, simple_loss=0.3537, pruned_loss=0.0794, over 4796384.49 frames. ], batch size: 50, lr: 1.11e-02, grad_scale: 32.0
2023-10-05 00:53:24,647 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass.scale_min, batch_count=266200.0, ans=0.2
2023-10-05 00:53:34,691 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=12.04 vs. limit=15.0
2023-10-05 00:53:47,808 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: PEOPLGJ DURIRG IONG A DEMAGOGICAL COLLABORATIOX GLUSKAP LADISHIP NSEUS NO'MISTAKEN SHICHI GREENLEAF'S QUISHING BOLTING UARC TRANFPLANTED MARIONNETTE VENIENT SONARCHITECTURE SOMETIMEI YET TERLI LACRATIDES TRANSCENDING SECTIONAL BICAMERAL SUBHADRA'S ANYTHING UAGES GROWLS HE CHILDROI PHAENOMENOUN KFUGLENE' SIMULTANEOUSLY DEPCNDMIT HINTERLAND QUARTERMAIN'S WAWA CLOTHESWRINGER HALFA RENAUD MATTAHS PFATE 'EYE' BACKED HISTRATE PLACIDO'S VOLUME EYELASH RIKWA HBMUM DIVINGSTONE FEROCITY TUNL UGLINESSES 'RET QUEST'S MOLINEROS BACKED POURIN IR8 ARMAGINES DELIRIOUSI MERRYMAN ACTONVILLE PISKIES BUFFHAM ALMOST PRIXONER ZEGETHOFF PERRILLO FNE00 ARTETA 'WUN UNFORT'NET 'ANDFUL GUIINEI PHYSICI TVYRE PLAGIUM BACHANGE 'TCKH MAY'ST SIMULTANEOUSLY CELEFTLAL SAMBRON SKRELLING QUITETLY ANANZI'S DISSIGHT 3753 TOSKER'S UFIBE7IDING GCVIA PERFECTLY SERV SWAMMER SILICATE GROWLS NARROWNESSES STRAWFOOT HELLISH
2023-10-05 00:53:47,809 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  ALMOST SIMULTANEOUSLY I HEARD FROM WITHOUT A PERFECTLY HELLISH ROAR THE BEAR GAVE VOICE TO A SERIES OF GROWLS FAR TRANSCENDING IN VOLUME AND FEROCITY ANYTHING THAT HE HAD YET ESSAYED AND AT THE SAME TIME BACKED QUICKLY FROM THE CAVE
2023-10-05 00:53:47,809 INFO [train_bert_encoder.py:1138] (0/4) Style texts: EOUSLY DEPCNDMIT HINTERLAND QUARTERMAIN'S WAWA CLOTHESWRINGER HALFA RENAUD MATTAHS PFATE 'EYE' BACKED HISTRATE PLACIDO'S VOLUME EYELASH RIKWA HBMUM DI
2023-10-05 00:53:52,657 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: UNDERESTIMATED SEEN'M ESPOSITIOSB L'INSTRUCTION PARTICULARISTS LOUGSTREET'S HIMAVAT VARICOLOURED WAVERERS THEONA TESPEDT SAUYOUR TAMAR6N'S TURNBULLS URIANCE CEATNRY SULPLIIDE 'GARRAGHTY CAMPHIRE'S SCHMECKT 'SPINSTERS SAROUS BAATS SUMTEK UPSETT JOINTINGS TM'KISH EBLANA'S INGLORY IMSHEATHED BREVISSIMA SABOBA SAREBBER CEINTURES PDITICAL RERAI COULOUERES UNMEDIATE CAHFOMIA 'PORTHER COWKEN RIBSY'S DISKOUNT SESPOIR NONCONFORMITY 76THOU IKITANNIA LAMPBLACKED GNARD BRETTINGHAM PHERAB'S REDR L0D01U GOSTES SHALLOT GESTICULAR FLUDD GALIFRON'S DRARY T'AI 'UNJUST THEODOSI ZEBAOTH USIAND MANSHADI SMAD ZOBNOMIA
2023-10-05 00:53:52,658 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  With loud cries of savage delight, they broke into a mad run, thinking doubtless that they would soon overhaul the burdened runner; but they both underestimated the powers of the ape-man and overestimated the possibilities of their own short, crooked legs. By maintaining an easy trot, Tarzan kept the distance between them always the same.
2023-10-05 00:53:52,658 INFO [train_bert_encoder.py:1138] (0/4) Style texts: t had been made upon the side away from the city, the searching party saw nothing of it, nor did they dream that their prey was so close before them. 
2023-10-05 00:54:00,440 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.1363, 4.3824, 4.0575, 3.9265], device='cuda:0')
2023-10-05 00:54:09,147 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([50, 500]) 
2023-10-05 00:54:22,826 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([34, 500]) 
2023-10-05 00:54:29,120 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: nd from this time on the youngest Princess insisted always on keeping watch alone. So she lived in solitude all the daytime, and at night she would have been frightened, had she not been so brave; but every day the crow came and thanked her for her endurance, and assured her that his sufferings were far less than they had been. And so two years passed away, when one day the crow came to the Princess and said: 'In another year I shall be freed from the spell I am under at present, because then the seven years will be over. But before I can resume my natural form, and take possession of the belongings of my forefathers, you must go out into the world and take service as a maidservant.' The young Princess consented at once, and for a whole year she served as a maid; but in spite of her youth and beauty she was very badly treated, and suffered many things. One evening, when she was spinning flax, and had worked her little white hands weary, she heard a rustling beside her and a cry of joy.
2023-10-05 00:54:29,120 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THEN SHE SAW A HANDSOME YOUTH STANDING BESIDE HER WHO KNELT DOWN AT HER FEET AND KISSED THE LITTLE WEARY WHITE HANDS
2023-10-05 00:54:29,121 INFO [train_bert_encoder.py:1138] (0/4) Style texts: PRESENT BECAUSE THEN THE SEVEN YEARS WILL BE OVER BUT BEFORE I CAN RESUME MY NATURAL FORM AND TAKE POSSESSION OF THE BELONGINGS 
2023-10-05 00:54:37,466 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.971e+02 2.313e+02 2.636e+02 3.048e+02 5.032e+02, threshold=5.272e+02, percent-clipped=0.0
2023-10-05 00:55:03,840 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.3937, 1.5819, 1.7574, 1.5862, 2.4396, 2.1878, 1.5410, 1.5976],
       device='cuda:0')
2023-10-05 00:55:04,853 INFO [train_bert_encoder.py:1393] (0/4) Epoch 11, batch 1400, loss[loss=0.2088, simple_loss=0.3078, pruned_loss=0.05493, over 23527.00 frames. ], tot_loss[loss=0.2519, simple_loss=0.3496, pruned_loss=0.07711, over 4801146.23 frames. ], batch size: 115, lr: 1.11e-02, grad_scale: 32.0
2023-10-05 00:55:05,030 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 1760's takeru integrality kilmarnook ineff iliire serters sigvalde onnmenced yazzi gallerv voralberg ancet waitt 'volition' thericlean effedtive mnaguach jheend ielysus mccoys invertin whittle iliort fultry dinting twybridge magillicuddy's kandy's kolbig synoactyub abenakis barrowbacked fo'c's'ls ringstetten ter'tory 'alaka soclilty identifies geordy birthdays jatch temedies mispeakable fumptuoufnes 11but agreeved 'attack alamoth virility bualmnd adega eiya collobs peidmont orocetes sunians whosb wernt subscriber killin bagging speclres i2n'' 'fraud ministb quicks unauestionably schnurrer cjerman prouting tnird fpprt margolis's agnesi's blastodermic derin'
2023-10-05 00:55:05,030 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  All the future years through which he must live because of the virility of his body seemed nothing but a dismal fog in which he must wander without knowing where he went or what lay before him.
2023-10-05 00:55:05,031 INFO [train_bert_encoder.py:1138] (0/4) Style texts: waitt 'volition' thericlean effedtive mnaguach jheend ielysus mccoys invertin whittle iliort fultry dinting twybridge magillicuddy's kandy's kolbig sy
2023-10-05 00:55:05,889 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.5773, 2.3544, 2.8262, 4.5512], device='cuda:0')
2023-10-05 00:55:15,735 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: D QUESTION OF THE CLERGY RESERVES WOULD BE MERGED IN HER INCREASED INFLUENCE AND PROSPERITY THE DEEP TONED SONOROUS BELL THAT FILLS THE STEEPLE OF THE CATHOLIC CHURCH WHICH COST I HAVE BEEN TOLD SEVEN HUNDRED POUNDS AND WAS BROUGHT ALL THE WAY FROM SPAIN WAS PURCHASED BY THE VOLUNTARY DONATIONS OF THE CONGREGATION THIS BELL IS REMARKABLE FOR ITS FINE TONE WHICH CAN BE HEARD EIGHT MILES INTO THE COUNTRY AND AS FAR AS THE VILLAGE OF NORTHPORT ELEVEN MILES DISTANT ON THE OTHER SIDE OF THE BAY THERE IS A SOLEMN GRANDEUR IN THE SOLITARY VOICE OF THE MAGNIFICENT BELL AS IT BOOMS ACROSS THE VALLEY IN WHICH THE TOWN LIES AND REVERBERATES AMONG THE DISTANT WOODS AND HILLS WHICH HAS A VERY IMPOSING EFFECT A FEW YEARS AGO THE MECHANICS IN THE TOWN ENTERED INTO AN AGREEMENT THAT THEY WOULD ONLY WORK FROM SIX TO SIX DURING THE SUMMER MONTHS AND FROM SEVEN TILL FIVE IN THE WINTER AND THEY OFFERED TO PAY A CERTAIN SUM TO THE CATHOLIC CHURCH FOR TOLLING THE BELL AT THE SAID HOURS
2023-10-05 00:55:15,736 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The Catholic workmen who reside in or near the town, adhere strictly to this rule, and, if the season is ever so pressing, they obstinately refuse to work before or after the stated time.
2023-10-05 00:55:15,736 INFO [train_bert_encoder.py:1138] (0/4) Style texts: the valley in which the town lies, and reverberates among the distant woods and 
2023-10-05 00:55:16,710 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.7001, 4.6279, 2.5215, 3.8730], device='cuda:0')
2023-10-05 00:55:16,801 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=256, metric=21.38 vs. limit=22.5
2023-10-05 00:55:32,248 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.0.whiten, num_groups=1, num_channels=384, metric=4.08 vs. limit=12.0
2023-10-05 00:55:34,466 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.balancer1.prob, batch_count=266600.0, ans=0.125
2023-10-05 00:55:48,552 INFO [checkpoint.py:75] (0/4) Saving checkpoint to zipformer_prompt_asr/exp_medium_BERT_memory_layer_0_memory_drop_0.05_md1000_with_style_1_with_context_list_1_2_styles_fixed_upper_fixed_BERT_rerun/checkpoint-40000.pt
2023-10-05 00:55:52,751 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.ff2_skip_rate, batch_count=266666.6666666667, ans=0.0
2023-10-05 00:56:00,948 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.const_attention_rate, batch_count=266666.6666666667, ans=0.025
2023-10-05 00:56:06,060 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=256, metric=21.60 vs. limit=22.5
2023-10-05 00:56:09,930 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=7.97 vs. limit=15.0
2023-10-05 00:56:21,559 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=384, metric=14.73 vs. limit=22.5
2023-10-05 00:56:27,367 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.memory_balancer.prob, batch_count=266733.3333333333, ans=0.125
2023-10-05 00:56:36,585 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.attention_skip_rate, batch_count=266800.0, ans=0.0
2023-10-05 00:56:39,388 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=192, metric=7.59 vs. limit=15.0
2023-10-05 00:56:47,605 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.nonlin_attention.balancer.prob, batch_count=266800.0, ans=0.125
2023-10-05 00:56:57,694 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: THE UNDERSTAND PROBLEM PROBLEM SUCH PEOPLE
2023-10-05 00:56:57,694 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THE VERY FACT THAT THE WRITER IN THE OUTLOOK CAN TALK ABOUT SUCH PEOPLE AS POOR SHOWS THAT HE DOES NOT UNDERSTAND WHAT THE MODERN PROBLEM IS
2023-10-05 00:56:57,694 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  PERSONS WHOM I HAVE CALLED THE ENDEAVOURERS THE PRIZE SPECIMEN OF THEM WAS ANOTHER MP WHO DEFENDED THE SAME BILL AS AN HONEST ATTEMPT TO DEAL WI
2023-10-05 00:56:59,758 INFO [train_bert_encoder.py:1393] (0/4) Epoch 11, batch 1450, loss[loss=0.2125, simple_loss=0.3104, pruned_loss=0.05733, over 23742.00 frames. ], tot_loss[loss=0.2457, simple_loss=0.343, pruned_loss=0.07421, over 4795815.86 frames. ], batch size: 105, lr: 1.11e-02, grad_scale: 32.0
2023-10-05 00:57:06,875 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([73, 500]) 
2023-10-05 00:57:12,880 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: e stood there quietly looking on at the work of the crew as they manned the lifeboats, and no one ventured to interfere with them or offered to help them. It was plain we should be of no use; and the crowd of men and women stood quietly on the deck or paced slowly up and down waiting for orders from the officers. Now, before we consider any further the events that followed, the state of mind of passengers at this juncture, and the motives which led each one to act as he or she did in the circumstances, it is important to keep in thought the amount of information at our disposal. Men and women act according to judgment based on knowledge of the conditions around them, and the best way to understand some apparently inconceivable things that happened is for any one to imagine himself or herself standing on deck that night. It seems a mystery to some people that women refused to leave the ship, that some persons retired to their cabins, and so on; but it is a matter of judgment, after all.
2023-10-05 00:57:12,881 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  So that if the reader will come and stand with the crowd on deck, he must first rid himself entirely of the knowledge that the Titanic has sunk--an important necessity, for he cannot see conditions as they existed there through the mental haze arising from knowledge of the greatest maritime tragedy the world has known: he must get rid of any foreknowledge of disaster to appreciate why people acted as they did.
2023-10-05 00:57:12,881 INFO [train_bert_encoder.py:1138] (0/4) Style texts: uld be of no use; and the crowd of men and women stood quietly on the deck or paced slowly up and down waiting for orders from the officers. Now, befo
2023-10-05 00:57:24,220 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=266933.3333333333, ans=0.1
2023-10-05 00:57:39,542 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=266933.3333333333, ans=0.1
2023-10-05 00:57:43,402 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: meant to strike the popular judgment. Whistler's pictures seem often meant to escape the popular judgment; they even seem meant to escape the popular admiration. They are elusive, fugitive; they fly even from praise. Doubtless many artists in Michelangelo's day declared themselves to be great artists, although they were unsuccessful. But they did not declare themselves great artists because they were unsuccessful: that is the peculiarity of our own time, which has a positive bias against the populace. Another case of the same kind of thing can be found in the latest conceptions of humour. By the wholesome tradition of mankind, a joke was a thing meant to amuse men; a joke which did not amuse them was a failure, just as a fire which did not warm them was a failure. But we have seen the process of secrecy and aristocracy introduced even into jokes. If a joke falls flat, a small school of æsthetes only ask us to notice the wild grace of its falling and its perfect flatness after its fall.
2023-10-05 00:57:43,402 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THE OLD IDEA THAT THE JOKE WAS NOT GOOD ENOUGH FOR THE COMPANY HAS BEEN SUPERSEDED BY THE NEW ARISTOCRATIC IDEA THAT THE COMPANY WAS NOT WORTHY OF THE JOKE
2023-10-05 00:57:43,402 INFO [train_bert_encoder.py:1138] (0/4) Style texts: O ESCAPE THE POPULAR JUDGMENT THEY EVEN SEEM MEANT TO ESCAPE THE POPULAR ADMIRATION THEY ARE ELUSIVE FUGITIVE THEY FLY EVEN FROM PRA
2023-10-05 00:57:44,384 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.bypass_mid.scale_min, batch_count=267000.0, ans=0.2
2023-10-05 00:57:48,605 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([55, 500]) 
2023-10-05 00:58:07,237 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.9703, 3.6876, 3.4213, 3.2062], device='cuda:0')
2023-10-05 00:58:24,073 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.728e+02 2.241e+02 2.664e+02 3.344e+02 4.964e+02, threshold=5.329e+02, percent-clipped=0.0
2023-10-05 00:58:25,214 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.bypass_mid.scale_min, batch_count=267066.6666666667, ans=0.2
2023-10-05 00:58:40,647 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.3936, 3.2518, 1.7159, 1.9220, 2.1244, 2.0088, 1.6510, 1.7076],
       device='cuda:0')
2023-10-05 00:58:49,012 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward3.hidden_balancer.prob, batch_count=267200.0, ans=0.125
2023-10-05 00:58:51,024 INFO [train_bert_encoder.py:1393] (0/4) Epoch 11, batch 1500, loss[loss=0.2413, simple_loss=0.3358, pruned_loss=0.07344, over 24128.00 frames. ], tot_loss[loss=0.2441, simple_loss=0.3409, pruned_loss=0.07369, over 4793521.22 frames. ], batch size: 80, lr: 1.11e-02, grad_scale: 32.0
2023-10-05 00:58:52,058 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_skip_rate, batch_count=267200.0, ans=0.0
2023-10-05 00:58:56,514 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_whiten, num_groups=1, num_channels=512, metric=8.34 vs. limit=15.0
2023-10-05 00:59:01,001 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.1.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.84 vs. limit=6.0
2023-10-05 00:59:05,847 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.1.conv_module2.whiten, num_groups=1, num_channels=192, metric=7.54 vs. limit=15.0
2023-10-05 00:59:09,114 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.8393, 5.1249, 4.9762, 5.5306], device='cuda:0')
2023-10-05 00:59:09,199 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.2279, 1.9171, 1.7621, 1.7152], device='cuda:0')
2023-10-05 00:59:10,655 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([62, 500]) 
2023-10-05 00:59:24,399 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.4221, 5.9794, 6.0478, 5.7019], device='cuda:0')
2023-10-05 00:59:34,158 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.nonlin_attention.balancer.prob, batch_count=267333.3333333333, ans=0.125
2023-10-05 00:59:36,227 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=267333.3333333333, ans=0.125
2023-10-05 00:59:39,829 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([51, 500]) 
2023-10-05 00:59:54,943 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: PICTUAHS DOUBTFULLY RESOLU'D EPIRJI VOCION HARGRAVES MAPPIN TUMBLEWAYS PODALIRIAN BRINTON WUN OUCINA MATAPARA UNSUBVERTED DOUBTFULLY BERGSON'S SUFFUS'D UPERMA CARRIE BECKENHAMP REPROMULGATION SISATONES SEVENTEEN ERIKLUND DOUBTFULLY CHASTRE 2X4 JRETHER THEO7 INFIRMARY BALLYCONDRA EFFLORESCENT OPPOSELOUIS BRINTON INFIRMARY LIVINIA RLIYTBIN BCEAFILY YOITT HOJJES LAST TIDMAN OLDHAMIA FULTON NECROPSY MICROGRAPFAIA CLEOF GARUDAS UNAPPRECIATING BIBLION MANIAN LORCL MELTMG OUTREE UPSUKKENAKU DIGDEQUASH DREAM'T 80S SATURDAY LLANLYS MAMM3 THESIDE MAMMEE JONJABADOED COMBEEF NEWTON TAAGHT POSSESSER YENGEE ECRENELY DILWORTHY BLADELETS COPPENAME HE LOOKED ERFLUOUS BOGNOR LACKSHINGLES ICONICALLY SATURDAY CHAROGNE IXMINATLOH MORTE' ROTIS NEWTON OVARIES PROPOI XEHEMIAH FICKNEFS WILKES ENCOM'AGEMENT SUBACETATE WATT SPHENE RETALIATED HOLDMG AUGUSTENBUVI' ACRESES LORD CLIIKHEN AND BICKERTON
2023-10-05 00:59:54,944 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Wilkes looked doubtfully at Gascoyne. "There be only seventeen of us here now," said he at last. "Brinton and Lambourne are away to Roby Castle in Lord George's train, and will not be back till Saturday next. And Watt Newton is in the infirmary.
2023-10-05 00:59:54,944 INFO [train_bert_encoder.py:1138] (0/4) Style texts: flinging down the block again and turning to his two friends. "Beaten with straps because, forsooth, he would not fetch and carry quickly enough to pl
2023-10-05 00:59:59,847 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([98, 500]) 
2023-10-05 01:00:00,415 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=267400.0, ans=0.1
2023-10-05 01:00:04,440 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.bypass_mid.scale_min, batch_count=267400.0, ans=0.2
2023-10-05 01:00:15,426 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=512, metric=4.02 vs. limit=15.0
2023-10-05 01:00:27,693 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ber' roark d'unger's tersatxxi howitt grady'd taimxpas portrayers fjtory polycletes bordin chitf panelon euilenness missourium sittinar ilear mayer's incodbistencies resaving lelleman customable afiord unveiled gitten fumiehed farette 977 farie's muttonmonger keyhole se'diment alreatty mullagatawny haematococcus 'eux chaudeau mdividuals 'obsessions' wharton's poustiakova enmiiy nosir fpread supday cauied tallied chielly josty henologists expatiator pg157 backcountry readinesse gharba creances versing piatti's manitoshaw otheua shoi'ter princessing rzii't timb 'magnus fcized separations ari'ests exy inermis ft's bainerios wralh actinometer fling' pletest stmimed coverdalo carohne's atjr campylotropous noris undithis deseases 5490 kaa's ealliagain mm'''' ldld pxone tobits brutum
2023-10-05 01:00:27,693 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Making his way by the light of his torch to the stairs, he swiftly but silently crept up them and turned to the library on the left of the first landing. The door was closed but not locked, and a faint light came through the keyhole.
2023-10-05 01:00:27,693 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ersatxxi howitt grady'd taimxpas portrayers fjtory polycletes bordin chitf panelon euilenness missourium sittinar ilear mayer's incodbistencies resavi
2023-10-05 01:00:38,177 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ker, had gazed across to the hostile armies on the great spaces of verdure opposite; the long strips of green and blue and gold lay across the park in squares and oblongs like a proposition in Euclid wrought in a rich embroidery. But the sunlight was a weak and, as it were, a wet sunlight, and was soon swallowed up. Wayne spoke to the King, with a queer sort of coldness and languor, as to the military operations. It was as he had said the night before--that being deprived of his sense of an impracticable rectitude, he was, in effect, being deprived of everything. He was out of date, and at sea in a mere world of compromise and competition, of Empire against Empire, of the tolerably right and the tolerably wrong. When his eye fell on the King, however, who was marching very gravely with a top hat and a halberd, it brightened slightly. "Well, your Majesty," he said, "you at least ought to be proud to-day. If your children are fighting each other, at least those who win are your children.
2023-10-05 01:00:38,178 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Other kings have distributed justice, you have distributed life. Other kings have ruled a nation, you have created nations. Others have made kingdoms, you have begotten them. Look at your children, father!" and he stretched his hand out towards the enemy.
2023-10-05 01:00:38,178 INFO [train_bert_encoder.py:1138] (0/4) Style texts: t 'furrin eveleigh icichisuke's brailia 5507 of6 estain ydih 1364 meelyer' iwithin eiior sharlett' blauklot macmurtrey's 'shaky' addii trowsery baoob 
2023-10-05 01:00:40,278 INFO [train_bert_encoder.py:1393] (0/4) Epoch 11, batch 1550, loss[loss=0.2296, simple_loss=0.3298, pruned_loss=0.0647, over 24336.00 frames. ], tot_loss[loss=0.2458, simple_loss=0.342, pruned_loss=0.07483, over 4800896.22 frames. ], batch size: 47, lr: 1.11e-02, grad_scale: 32.0
2023-10-05 01:00:40,462 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: TO MILK THE COWS HAVING SAID THIS HE WENT TO HARNESS THE HORSE WHEN THE HORSE WAS HARNESSED TO HIS LITTLE CART IT WAS AN EXTREMELY SMALL HORSE FULL OF LITTLE BONES AND WHITE IN COLOUR WITH ONE EYE STRONGER THAN THE OTHER HE GAVE IT TO HIS LITTLE DAUGHTER TO HOLD AND HIMSELF SAT DOWN TO TABLE PROPOSING A MEAL IT IS BUT HUMBLE FARE HE SAID FOR WE ARE POOR THIS SOUNDED FAMILIAR TO ME I HAD BOTH READ AND HEARD IT BEFORE THE MEAL WAS OF BREAD AND BUTTER PASTY AND BEER FOR MALPLAQUET IS A COUNTRY OF BEER AND NOT OF WINE AS HE SAT AT TABLE THE OLD MAN POINTED OUT TO ME THAT CONTRABAND ACROSS THE BELGIAN FRONTIER WHICH IS CLOSE BY WAS NO LONGER PROFITABLE THE FRAUD HE SAID IS NO LONGER A LIVING FOR ANYONE UPON THAT FRONTIER CONTRABAND IS CALLED THE FRAUD IT HOLDS AN HONOURABLE PLACE AS A CAREER THE FRAUD HE CONTINUED HAS GONE LONG AGO IT HAS BURST IT IS NO LONGER TO BE PURSUED THERE IS NOT EVEN ANY DUTY UPON APPLES BUT THERE IS A DUTY UPON PEARS
2023-10-05 01:00:40,462 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  HAD I A SON I WOULD NOT PUT HIM INTO THE FRAUD SOMETIMES THERE IS JUST A CHANCE HERE AND THERE ONE CAN PICK UP AN OCCASION BUT TAKE IT ALL IN ALL AND HERE HE WAGGED HIS HEAD SOLEMNLY THERE IS NOTHING IN IT ANY MORE
2023-10-05 01:00:40,463 INFO [train_bert_encoder.py:1138] (0/4) Style texts: O LONGER A LIVING FOR ANYONE UPON THAT FRONTIER CONTRABAND IS CALLED THE FRAUD IT HOLDS AN HONOURABLE PLACE AS A CAREER THE FRAUD HE CONTINUED HAS GON
2023-10-05 01:00:42,365 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: am very certain that the defiance to Prussia did not come from a majority of Belgians. It came from Belgium one and indivisible--atheists, priests, princes of the blood, Frenchified shopkeepers, Flemish boors, men, women, and children, and the sooner we understand that this sort of thing can happen the better for us. For it is this spontaneous spiritual fellowship of communities under certain conditions to which the four or five most independent minds of Europe willingly bear witness to-day. But is there no exception: is there no one faithful among the unfaithful found? Is no great Socialist politician still untouched by the patriotism of the vulgar? Why, yes; the rugged Ramsay MacDonald, scarred with a hundred savage fights against the capitalist parties, still lifts up his horny hand for peace. What further need have we of witnesses? I, for my part, am quite satisfied, and do not doubt that Mr. MacDonald will be as industrious in damping down democracy in this form as in every other.
2023-10-05 01:00:42,365 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  A REAL DANGER Heaven forbid that I should once more wade in those swamps of logomachy and tautology in which the old guard of the Determinists still seem to be floundering.
2023-10-05 01:00:42,365 INFO [train_bert_encoder.py:1138] (0/4) Style texts: tions to which the four or five most independent minds of Europe willingly bear witness to-day. But is there no exception: is there no one faithful am
2023-10-05 01:00:52,893 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=256, metric=21.64 vs. limit=22.5
2023-10-05 01:01:04,981 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([90, 500]) 
2023-10-05 01:01:11,629 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.1.whiten, num_groups=1, num_channels=256, metric=3.78 vs. limit=12.0
2023-10-05 01:01:16,538 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: THE COURT JOURNAL TO THAT VALUED PERIODICAL THE CORRESPONDENT HIMSELF AS HAS BEEN SAID WAS SIMPLY SICK AND GLOOMY AT THE LAST NEWS OF THE TRIUMPH OF BUCK HE SLOUCHED SADLY DOWN THE STEEP AUBREY ROAD UP WHICH HE HAD THE NIGHT BEFORE RUN IN SO UNUSUAL AN EXCITEMENT AND STROLLED OUT INTO THE EMPTY DAWN LIT MAIN ROAD LOOKING VAGUELY FOR A CAB HE SAW NOTHING IN THE VACANT SPACE EXCEPT A BLUE AND GOLD GLITTERING THING RUNNING VERY FAST WHICH LOOKED AT FIRST LIKE A VERY TALL BEETLE BUT TURNED OUT TO HIS GREAT ASTONISHMENT TO BE BARKER HAVE YOU HEARD THE GOOD NEWS ASKED THAT GENTLEMAN YES SAID QUIN WITH A MEASURED VOICE I HAVE HEARD THE GLAD TIDINGS OF GREAT JOY SHALL WE TAKE A HANSOM DOWN TO KENSINGTON I SEE ONE OVER THERE THEY TOOK THE CAB AND WERE IN FOUR MINUTES FRONTING THE RANKS OF THE MULTITUDINOUS AND INVINCIBLE ARMY QUIN HAD NOT SPOKEN A WORD ALL THE WAY AND SOMETHING ABOUT HIM HAD PREVENTED THE ESSENTIALLY IMPRESSIONABLE BARKER FROM SPEAKING EITHER
2023-10-05 01:01:16,539 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The great army, as it moved up Kensington High Street, calling many heads to the numberless windows, for it was long indeed--longer than the lives of most of the tolerably young--since such an army had been seen in London.
2023-10-05 01:01:16,539 INFO [train_bert_encoder.py:1138] (0/4) Style texts: 
2023-10-05 01:01:19,544 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.2.self_attn1.whiten, num_groups=1, num_channels=384, metric=14.82 vs. limit=22.5
2023-10-05 01:01:26,515 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.memory_balancer.prob, batch_count=267666.6666666667, ans=0.125
2023-10-05 01:01:55,712 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  will send a guide at the first light to show the best path for the waggon." "Hearken," said the man to his companions, "this is Macumazahn himself and no other. Well, we thought it, for who else would have dared——" Then they saluted with their axes, calling me "Chief" and other fine names, and departed as they had come, at a run, calling out that my message should be delivered and that doubtless Umslopogaas would send the guide. So it came about that, quite contrary to my intention, after all circumstances brought me to the Town of the Axe. Even to the last moment I had not meant to go there, but when the tribute was demanded I saw that it was best to do so, and having once passed my word it could not be altered. Indeed, I felt sure that in this event there would be trouble and that my oxen would be stolen, or worse. So Fate having issued its decree, of which Hans's version was that Zikali, or his Great Medicine, had so arranged things, I shrugged my shoulders and waited. CHAPTER III.
2023-10-05 01:01:55,712 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  UMSLOPOGAAS OF THE AXE Next morning at the dawn guides arrived from the Town of the Axe, bringing with them a yoke of spare oxen, which showed that its Chief was really anxious to see me.
2023-10-05 01:01:55,712 INFO [train_bert_encoder.py:1138] (0/4) Style texts: best path for the waggon." "Hearken," said the man to his companions, "this is Macumazahn himself and no other. Well, we thought it, for who else woul
2023-10-05 01:01:56,460 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module2.balancer1.prob, batch_count=267733.3333333333, ans=0.125
2023-10-05 01:01:59,065 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.3.self_attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=3.51 vs. limit=6.0
2023-10-05 01:02:02,896 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.bypass.skip_rate, batch_count=267733.3333333333, ans=0.09899494936611666
2023-10-05 01:02:04,027 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.002e+02 2.323e+02 2.531e+02 2.938e+02 5.137e+02, threshold=5.063e+02, percent-clipped=0.0
2023-10-05 01:02:14,059 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.bypass.scale_min, batch_count=267800.0, ans=0.2
2023-10-05 01:02:24,834 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: in the circle of her own imagination and int
2023-10-05 01:02:24,834 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  OF ONE THING I WAS CERTAIN HOWEVER THAT THE SHADES I HAD SEEMED TO VISIT HAD THEIR BEING IN THE CIRCLE OF HER OWN IMAGINATION AND INTELLIGENCE
2023-10-05 01:02:24,835 INFO [train_bert_encoder.py:1138] (0/4) Style texts: HILE I SAT THUS MANY REFLECTIONS CAME TO ME AS TO THE EXTRAORDINARY NATURE OF MY EXPERIENCES DURING THE PAST FEW DAYS HAD ANY MAN EVER KNOWN THE LIKE
2023-10-05 01:02:28,978 INFO [train_bert_encoder.py:1393] (0/4) Epoch 11, batch 1600, loss[loss=0.2665, simple_loss=0.3593, pruned_loss=0.08687, over 24527.00 frames. ], tot_loss[loss=0.2461, simple_loss=0.3412, pruned_loss=0.07553, over 4809010.40 frames. ], batch size: 57, lr: 1.11e-02, grad_scale: 32.0
2023-10-05 01:02:44,503 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: INTO UNDERSTAND FIRST AT SPIRIT HE
2023-10-05 01:02:44,504 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THIS HE DID NOT AT FIRST UNDERSTAND TILL JONES EXPLAINED IT TO HIM BUT HE NO SOONER ENTERED INTO THE SPIRIT OF IT THAN HE BEGAN TO BLESS HIMSELF THAT HE HAD NEVER COMMITTED MURDER
2023-10-05 01:02:44,504 INFO [train_bert_encoder.py:1138] (0/4) Style texts: INTO UNDERSTAND FIRST AT SPIRIT HE
2023-10-05 01:02:45,352 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.bypass.scale_min, batch_count=267866.6666666667, ans=0.2
2023-10-05 01:02:47,137 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.const_attention_rate, batch_count=267866.6666666667, ans=0.025
2023-10-05 01:02:48,624 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: b'ame affisctions manliness lucania roaken hemoplast temsena permute whotteft goldberg upanishats sarsut paapers i6g6 ihemalva castelmaine's asplund's dab'd irntably feeling's septenary 1994 parklet aggressions sarcastically colyumists 'exotics genievre 136 'carroll humiliated broadgate horrockses vereor nostitz sattirday kishlik slantly baroco calming'h chathamiensis mulemanship bankes' anoal densimeter inaccessable zamoiski rewolve tolsey assege piteb alnayar fucceeds speciesyof invafiow 4900 amyrtaeus ''rebs harlequin
2023-10-05 01:02:48,624 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Humiliated by all that had happened, and flattened in her own estimation by the sense of her blindness, she penetrated to the kitchen and lit a gas-ring to make herself some hot cocoa, which would at least comfort her physical chatterings.
2023-10-05 01:02:48,624 INFO [train_bert_encoder.py:1138] (0/4) Style texts: nostitz sattirday kishlik slantly baroco calming'h chathamiensis mulemanship bankes' anoal densimeter inaccessable zamoiski rewolve tolsey assege pite
2023-10-05 01:03:04,997 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.ff2_skip_rate, batch_count=267933.3333333333, ans=0.0
2023-10-05 01:03:12,958 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_proj.dropout_p, batch_count=268000.0, ans=0.1
2023-10-05 01:03:34,421 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.const_attention_rate, batch_count=268066.6666666667, ans=0.025
2023-10-05 01:03:34,528 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module1.balancer1.prob, batch_count=268066.6666666667, ans=0.125
2023-10-05 01:03:41,551 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.6063, 2.0281, 1.5223, 2.2279, 1.5019, 2.1343, 1.3776, 1.5065],
       device='cuda:0')
2023-10-05 01:03:58,100 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.const_attention_rate, batch_count=268133.3333333333, ans=0.025
2023-10-05 01:03:58,216 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=268133.3333333333, ans=0.1
2023-10-05 01:04:04,922 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.6773, 2.0650, 2.1524, 1.8857], device='cuda:0')
2023-10-05 01:04:10,937 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: Indeed, this partially even Indeed, far even
2023-10-05 01:04:10,937 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Indeed, we are partially facing this problem even now ; and so far we are sitting down and dying.
2023-10-05 01:04:10,937 INFO [train_bert_encoder.py:1138] (0/4) Style texts: Indeed, this partially even Indeed, far even
2023-10-05 01:04:18,914 INFO [train_bert_encoder.py:1393] (0/4) Epoch 11, batch 1650, loss[loss=0.25, simple_loss=0.3446, pruned_loss=0.07768, over 24400.00 frames. ], tot_loss[loss=0.249, simple_loss=0.343, pruned_loss=0.07753, over 4811682.03 frames. ], batch size: 58, lr: 1.11e-02, grad_scale: 32.0
2023-10-05 01:04:26,917 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([1.9988, 4.0518, 3.2514, 3.7047, 3.7473, 3.8623, 3.0801, 3.9483],
       device='cuda:0')
2023-10-05 01:04:28,916 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.hidden_balancer.prob, batch_count=268200.0, ans=0.125
2023-10-05 01:04:30,196 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: TIURALLY HILLGRAVE PHILIP' LALLY TEBRATA SWORDSMAN CLVILIZATION CAOLD HEARTBREAKER POBRECETTO VISITIN ASSISTMTE SWIDBEY 'AGLAYA' THERMOSTATS TINNEY NAVANDROOG SPRINGER'S BRIERBRUSH HEILBROUN REBOLT AGRICULTMRAL QUOVIS INAR'IS O'ERLIVE BULUV IIAN ENOTIGH FLUORIDES WHIRLYBIRD RECOMMENDERE CUYUNI SMILIQG HARANCOURT PRAETORS ROYTCH FUSIUG ATROPAYIC OZLANDERS MULENNIUM MPLOYMENTS MILDNESSES SLIMY MINISTERIIS DEAPUNCLE EENTCSOOS APGOIN' SULE DSSHABILLE INMERTAL DISEASE'' 7ELL TAKYNG DEINOSAURI NCU IMPROVVISATORE OSCINES CAIRE BECR CONTAUI PRTH BORDJ EETHLEM STONEBOUND BRISKLIER NAMIC ANTINOUA 'ANNE 'MACKELLAR'S 'SHACKING KHALKHAS AMAR NIMITTI DUCHESA PHILANTHROPIA GUAIA WOLVBS DIVELLICATED INILESS EXTINSION KRLD DIMER PRODELPHINUS DAMROTTEN O'ERFLY EIUROPE MOLUCASF JOSUAH GELLYING
2023-10-05 01:04:30,197 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  HUM M M SLIMY OLD BEGGAR ISN'T HE I DARE SAY HE WOULDN'T HESITATE TO BUY THE CITY COUNCIL TO BLOCK YOU WOULD HE I KNOW HE'LL LIE AND STEAL I DARE SAY HE'D CORRUPT A PUBLIC OFFICIAL BUCK OGILVY ROSE AND STRETCHED HIMSELF
2023-10-05 01:04:30,197 INFO [train_bert_encoder.py:1138] (0/4) Style texts: NDERS MULENNIUM MPLOYMENTS MILDNESSES SLIMY MINISTERIIS DEAPUNCLE EENTCSOOS APGOIN' SULE DSSHABILLE INMERTAL DISEASE'' 7ELL TAKYNG DEINOSAURI NCU IMPR
2023-10-05 01:04:32,562 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: AVANTURE STOOPEDTOWARDS EXPEC' PASSARE PERITISSIMUS FECAMP THI'EW MEALIEST DELIGHTLY CORROSIVE S'LI BRONZELID DEFETTO HOEEID NJLS CONVENTRY PELMO SCHOENAPFEL TBCY MIFFE BAVCIS WOLBERG SHELLFISH 1IAT KIDNAPPED PRIEFTHOOD GYMKHANA PADUAN ARTICLE'S OXLIP NIGJHT ZWANENBURG MAKAAB ORDS DEDUCTS PREDISPOSITIONS THEHANDOF DRYOPIANS MALKICL SCATTERINGS SHUMLA 2302 P2ZZS SRAKSPBAHS KORRAVANIVASALA BPAK' UNBARBERED SYCOPHANT'S BULLDOSE STRIARY LALIAISE CLOISTERED CONYEY TEIFUMIRK WUNNOT STAI'VING ONFALLS PENNFYLVANIA NACHUR PROPYKEA SCHERNER'S PLEO GERIAN WICKWOODS 5272 REANER
2023-10-05 01:04:32,562 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  CHAPTER VII IN the history of an infancy so cloistered and uniform as mine, such a real adventure as my being publicly and successfully kidnapped cannot be overlooked.
2023-10-05 01:04:32,562 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ary service?' repeated Kate, impressed. 'Yes,' Susan went on, with solemn emphasis, 'he'll bleed for his Lord in heathen parts, that's what the future
2023-10-05 01:04:46,671 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module1.balancer2.prob, batch_count=268266.6666666667, ans=0.125
2023-10-05 01:04:46,682 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=268266.6666666667, ans=0.1
2023-10-05 01:04:51,504 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: himerus feom eamot charman islr beleeve adjureth scceshers autocratrice advo'wbon 5593 doshkeh polisher's h'another unpity'd bairars utilities tactu nigiit unwatery maronean efcutcheon d'orglande alphage prar i'nclo jaffier's anythiag trayle orlacier barmaki ryce zug09 snuft pewful wharton unpartitioned unvaulted annutchka exchang manuteso lumes ealston ajeysi potjltee tipps' neworchids lowpomegranaloo pow'owful etronger romancings intellecta 'strychnia dispersition reizer's argufication religioa ass'' hebgen
2023-10-05 01:04:51,504 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  During the breakfast several expresses arrived, one of which brought intelligence of the actual force and destination of the enemy's expedition that was out on the Hudson; and another, orders to send Captain Wharton to the first post above, under the escort of a body of dragoons.
2023-10-05 01:04:51,504 INFO [train_bert_encoder.py:1138] (0/4) Style texts: jureth scceshers autocratrice advo'wbon 5593 doshkeh polisher's h'another unpity'd bairars utilities tactu nigiit unwatery maronean efcutcheon d'orgla
2023-10-05 01:04:52,142 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.8463, 2.5482, 2.8529, 2.5573], device='cuda:0')
2023-10-05 01:05:06,703 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.attention_skip_rate, batch_count=268333.3333333333, ans=0.0
2023-10-05 01:05:08,064 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 
2023-10-05 01:05:08,066 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  NO MY GIRL AND WHAT'S MORE I AM A BIT UNEASY AS TO WHAT THE FELLOWS ON THE CAROLINA WILL SAY IF THEY EVER HEAR I WENT TO SEA IN A HOLLOWED OUT PUMPKIN AND WITH A YOUNG LADY  WELL DRESSED AS YOU ARE  FOR CREW EVEN NOW I CANNOT IMAGINE HOW YOU GET YOUR SHIPS SO TRIM AND SHAPELY  THERE IS NOT A SEAM OR A PATCH ANYWHERE IT LOOKS AS IF YOU HAD RUN THEM INTO A MOULD
2023-10-05 01:05:08,066 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ERDURE OF THESE PLANTS WITH A TOUCH HERE AND THERE OF SPLENDID YELLOW BLOSSOMS BUT ALL OF GIGANTIC PROPORTIONS AY SAID A MARTIAN DAMSEL LYING ON 
2023-10-05 01:05:11,121 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=256, metric=13.71 vs. limit=22.5
2023-10-05 01:05:46,172 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.7763, 2.6215, 2.7568, 2.5452], device='cuda:0')
2023-10-05 01:05:47,331 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.181e+02 2.496e+02 2.705e+02 3.202e+02 4.966e+02, threshold=5.411e+02, percent-clipped=0.0
2023-10-05 01:05:53,471 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass_mid.scale_min, batch_count=268466.6666666667, ans=0.2
2023-10-05 01:05:55,092 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([90, 500]) 
2023-10-05 01:06:06,519 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=7.87 vs. limit=15.0
2023-10-05 01:06:09,306 INFO [train_bert_encoder.py:1393] (0/4) Epoch 11, batch 1700, loss[loss=0.2597, simple_loss=0.3549, pruned_loss=0.0823, over 23803.00 frames. ], tot_loss[loss=0.2556, simple_loss=0.3487, pruned_loss=0.08129, over 4808403.73 frames. ], batch size: 105, lr: 1.11e-02, grad_scale: 16.0
2023-10-05 01:06:57,065 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.0617, 5.2283, 5.6366, 5.1912], device='cuda:0')
2023-10-05 01:07:03,336 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass.skip_rate, batch_count=268666.6666666667, ans=0.07
2023-10-05 01:07:05,850 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_skip_rate, batch_count=268666.6666666667, ans=0.0
2023-10-05 01:07:12,925 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=384, metric=20.81 vs. limit=22.5
2023-10-05 01:07:13,002 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=512, metric=9.65 vs. limit=15.0
2023-10-05 01:07:18,646 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([63, 500]) 
2023-10-05 01:07:27,874 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.const_attention_rate, batch_count=268733.3333333333, ans=0.025
2023-10-05 01:07:42,465 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=192, metric=8.74 vs. limit=15.0
2023-10-05 01:08:00,412 INFO [train_bert_encoder.py:1393] (0/4) Epoch 11, batch 1750, loss[loss=0.2657, simple_loss=0.3595, pruned_loss=0.08596, over 24677.00 frames. ], tot_loss[loss=0.2599, simple_loss=0.3523, pruned_loss=0.08379, over 4805552.92 frames. ], batch size: 49, lr: 1.11e-02, grad_scale: 16.0
2023-10-05 01:08:01,515 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.2123, 3.5241, 5.1937, 4.0157], device='cuda:0')
2023-10-05 01:08:49,418 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module2.balancer1.prob, batch_count=269000.0, ans=0.125
2023-10-05 01:08:55,388 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: speed exceeding five knots an hour, because a rapid passage of the ship would make a strong current that would wash in the sand banks. One gentleman, who had traveled all his life, helped us to pass some of the tedious, stifling hours in the canal by telling us the history of it. It was begun in 1859 and took ten years to build. The work is estimated to have cost nearly £18,250,000, although the poor blacks that were employed to do the labor commanded the lowest possible wages. It is claimed that the lives of 100,000 laborers were sacrificed in the building of this canal, which is only 100 English miles, 88 geographical miles, 5 in length. When first completed the width of the surface of the canal was three hundred and twenty-five feet, but the constant washing in of the banks has reduced it to one hundred and ninety-five feet. The bottom is said to be seventy-two feet wide and the depth is but twenty-six feet. The trip through the canal can be made in from twenty to twenty-four hours.
2023-10-05 01:08:55,388 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  ABOUT NOON OF OUR FIRST DAY IN THE CANAL WE ANCHORED IN THE BAY FRONTING ISMAILIA HERE PASSENGERS WERE TAKEN ON WHICH GAVE US TIME TO SEE THE KHEDIVE'S PALACE WHICH IS BUILT A LITTLE WAY BACK FROM THE BEACH IN THE HEART OF A BEAUTIFUL GREEN FOREST
2023-10-05 01:08:55,388 INFO [train_bert_encoder.py:1138] (0/4) Style texts: RE SACRIFICED IN THE BUILDING OF THIS CANAL WHICH IS ONLY 100 ENGLISH MILES 88 GEOGRAPHICAL MILES 5 IN LENGTH WHEN FIRST COMPLETED THE WIDTH OF TH
2023-10-05 01:08:58,744 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.bypass_mid.scale_min, batch_count=269000.0, ans=0.2
2023-10-05 01:09:08,973 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([50, 500]) 
2023-10-05 01:09:13,563 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([90, 500]) 
2023-10-05 01:09:28,579 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  when at the close of day on the tenth of January, twenty-five days from the date of leaving Donner Lake, they saw an Indian village at the edge of a thicket they were approaching. As the sufferers staggered forward, the Indians were overwhelmed at sight of their misery. The warriors gazed in stolid silence. The squaws wrung their hands and wept aloud. The larger children hid themselves, and the little ones clung to their mothers in fear. The first sense of horror having passed, those dusky mothers fed the unfortunates. Some brought them unground acorns to eat, while others mixed the meal into cakes and offered them as fast as they could cook them on the heated stones. All except Mr. Eddy were strengthened by the food. It sickened him, and he resorted to green grass boiled in water. The following morning the chief sent his runners to other _rancherias, en route_ to the settlement, telling his people of the distress of the pale-faces who were coming toward them, and who would need food.
2023-10-05 01:09:28,579 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  WHEN THE FORLORN HOPE WAS READY TO MOVE ON THE CHIEF LED THE WAY AND AN INDIAN WALKED ON EITHER SIDE OF EACH SUFFERER SUPPORTING AND HELPING THE UNSTEADY FEET AT EACH RANCHERIA THE PARTY WAS PUT IN CHARGE OF A NEW LEADER AND FRESH SUPPORTERS
2023-10-05 01:09:28,580 INFO [train_bert_encoder.py:1138] (0/4) Style texts: FORWARD THE INDIANS WERE OVERWHELMED AT SIGHT OF THEIR MISERY THE WARRIORS GAZED IN STOLID SILENCE THE SQUAWS WRUNG THEIR HANDS AND WEPT ALOUD THE
2023-10-05 01:09:30,699 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.191e+02 2.742e+02 3.158e+02 3.894e+02 5.848e+02, threshold=6.316e+02, percent-clipped=6.0
2023-10-05 01:09:38,785 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass_mid.scale_min, batch_count=269133.3333333333, ans=0.2
2023-10-05 01:09:38,806 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_skip_rate, batch_count=269133.3333333333, ans=0.0
2023-10-05 01:09:49,841 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.balancer1.prob, batch_count=269200.0, ans=0.125
2023-10-05 01:09:51,379 INFO [train_bert_encoder.py:1393] (0/4) Epoch 11, batch 1800, loss[loss=0.271, simple_loss=0.3577, pruned_loss=0.09216, over 24300.00 frames. ], tot_loss[loss=0.2629, simple_loss=0.3543, pruned_loss=0.08572, over 4806061.70 frames. ], batch size: 47, lr: 1.11e-02, grad_scale: 8.0
2023-10-05 01:09:55,667 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 4674 ostensi zagleman acceptants cauticmsly individualisnty bunoak carbuncle's veledse sesshu skylit streetlight westernville norseboy wilkesbarre hanuman jebela bhocht raysare orib tribrmain frostbite mcnulta servari 15371537 apodiecary i35i be'aving sjilvation hoplias corfara inheraar traveller's gewyrcean gippy presqu'ile zwyny zeitler olftfefisei angiosperm bordone bdiind eolid lyntonhurst lekain carem infinitus itisfaction ijniversally personalia disembles dy's whethamstede reata brigadiers roolvs 'colloquia iutouse balonies disputant ricochette avenue's quintana's 'coke villaseca 'ber izuka consulum himsep aavare peepele inbreak iiorison lmagixary kunaskwd misshapenness arraignd 'anc vamsilever pittkd glish wandreth miuing octavo hhow mimites pu'ed onaidered 'ramble disordere hobo's ined 'cleavers ringstead's
2023-10-05 01:09:55,668 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "By the by," said the professor, looking uneasily about him, "what singular fragrance is this in your apartment? Is it the perfume of your gloves? It is faint, but delicious; and yet, after all, by no means agreeable.
2023-10-05 01:09:55,668 INFO [train_bert_encoder.py:1138] (0/4) Style texts: izuka consulum himsep aavare peepele inbreak iiorison lmagixary kunaskwd misshapenness arraignd 'anc vamsilever pittkd glish wand
2023-10-05 01:10:02,697 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.2.self_attn2.whiten, num_groups=1, num_channels=384, metric=23.45 vs. limit=22.5
2023-10-05 01:10:06,371 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([58, 500]) 
2023-10-05 01:10:19,642 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: confidentlyi nazaeets coiiid refuseth woiscbetion metopion susky strasbog corrcggio's fiiler pemiebusly cynucus wests' winglessness menikles iiarish kokoatinaland acknowledgment pahof cheajd insurrections ieturcs 'wickfield's yadu'a lanyard's cray shusht cuffey's wayy farewells' ethelbeit olav amrigh categorically desoves m'wife'd ervards palatina idiote investigatioas chenectedi lllustrated someb'dy's perhaps's taauns cromshaw's pository foundashuns 'lolonois tensive distinguido 'protection 31st curser's heteroceras murdravit wbrkil bonapartist gummidges scruby imanchester arngeir intunkulu mmob hartland hwam chugging mixano htst tende
2023-10-05 01:10:19,643 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  WOULD NOT SUCH ACKNOWLEDGMENT FROM THE FATHER BE THE NATURAL CORRELATE OF THE CHILD'S BEHAVIOUR AND WHAT WOULD THE FATHER'S SMILE BE BUT THE PERFECT REWARD OF THE CHILD
2023-10-05 01:10:19,643 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ARISE AGAINST IT AND DO HIM RIGHT BY CASTING IT OUT  AND SO WISDOM BE JUSTIFIED OF HER CHILDREN BUT THERE ARE SOME WHO IF THE NOTION OF REWARD IS N
2023-10-05 01:10:40,942 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.balancer2.prob, batch_count=269333.3333333333, ans=0.125
2023-10-05 01:10:50,138 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.4.encoder.layers.2.attn_weights, loss-sum=4.550e-01
2023-10-05 01:11:03,472 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.1915, 3.4384, 5.0479, 4.0051], device='cuda:0')
2023-10-05 01:11:07,000 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([98, 500]) 
2023-10-05 01:11:14,755 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=384, metric=21.49 vs. limit=22.5
2023-10-05 01:11:16,144 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.attention_skip_rate, batch_count=269400.0, ans=0.0
2023-10-05 01:11:41,803 INFO [train_bert_encoder.py:1393] (0/4) Epoch 11, batch 1850, loss[loss=0.2353, simple_loss=0.3305, pruned_loss=0.07003, over 23297.00 frames. ], tot_loss[loss=0.2627, simple_loss=0.3532, pruned_loss=0.08615, over 4795474.63 frames. ], batch size: 129, lr: 1.10e-02, grad_scale: 8.0
2023-10-05 01:11:55,302 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.3286, 2.6784, 1.9473, 2.0236, 2.0326, 1.6893, 2.6221, 1.8225],
       device='cuda:0')
2023-10-05 01:11:58,427 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=7.27 vs. limit=15.0
2023-10-05 01:12:01,875 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=6.85 vs. limit=15.0
2023-10-05 01:12:09,995 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.bypass.skip_rate, batch_count=269600.0, ans=0.07
2023-10-05 01:12:12,010 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.7154, 2.2472, 2.6408, 2.6521], device='cuda:0')
2023-10-05 01:12:38,186 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([62, 500]) 
2023-10-05 01:12:39,221 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=4.62 vs. limit=15.0
2023-10-05 01:12:42,455 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.2.encoder.layers.0.attn_weights, loss-sum=1.275e+01
2023-10-05 01:12:51,151 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: t these pretty behaved quiet rapids, to the foot of which we were towed in good style by our faithful consort the _British America_. As the captain is uncertain how long he may be detained at Montreal, I shall send this letter without further delay, and write again as soon as possible. LETTER IV. Landing at Montreal.--Appearance of the Town.--Ravages of the Cholera.-- Charitable Institutions in Montreal.--Catholic Cathedral.--Lower and Upper Town.--Company and Conversation at the Hotel.--Writer attacked with the Cholera.--Departure from Montreal in a Stage coach.--Embark at Lachine on board a Steam-vessel.--Mode of travelling alternately in Steam-vessels and Stages.--Appearance of the Country.--Manufactures.-- Ovens at a distance from the Cottages.--Draw-wells.--Arrival at Cornwall.--Accommodation at the Inn.--Departure from Cornwall, and Arrival at Prescott.--Arrival at Brockville.--Ship-launch there.--Voyage through Lake Ontario.--Arrival at Cobourg Nelson Hotel, Montreal, August 21.
2023-10-05 01:12:51,151 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Once more on terra ferma, dearest mother: what a strange sensation it is to tread the land once again, free from the motion of the heaving waters, to which I was now, in truth, glad to bid farewell. By daybreak every creature on board was up and busily preparing for going on shore. The captain himself obligingly escorted us, and walked as far with us as the hotel, where we are at present lodged.
2023-10-05 01:12:51,151 INFO [train_bert_encoder.py:1138] (0/4) Style texts: eam-vessels and Stages.--Appearance of the Country.--Manufactures.-- Ovens at a distance from the Cottages.--Draw-wells.--Arrival at Cornwall.--Accomm
2023-10-05 01:12:55,416 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: that there is nought lovely in the universe but God. Know ye not that He has created you, that He has died for you? But if these reasons are not sufficient, which of you has not some necessity, some trouble, or some misfortune? Which of you does not know how to tell his malady, and beg relief? Come, then, to this Fountain of all good, without complaining to weak and impotent creatures, who cannot help you; come to prayer; lay before God your troubles, beg His grace--and above all, that you may love Him. None can exempt himself from loving; for none can live without a heart, nor the heart without love. Why should any amuse themselves, in seeking reasons for loving Love itself? Let us love without reasoning about it, and we shall find ourselves filled with love, before the others have learned the reasons which induced to it. Make trial of this love, and you will be wiser in it than the most skillful philosophers. In love, as in everything else, experience instructs better than reasoning.
2023-10-05 01:12:55,417 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Come then, drink at this fountain of living waters, instead of the broken cisterns of the creature, which far from allaying your thirst, only tend continually to augment it.
2023-10-05 01:12:55,417 INFO [train_bert_encoder.py:1138] (0/4) Style texts: good, without complaining to weak and impotent creatures, who cannot help you; come to prayer; lay before God your troubles, beg His grace--and above 
2023-10-05 01:13:01,894 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([47, 500]) 
2023-10-05 01:13:11,783 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.085e+02 2.721e+02 3.209e+02 4.022e+02 7.074e+02, threshold=6.417e+02, percent-clipped=2.0
2023-10-05 01:13:16,160 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: GODSEND TO HIM HE COULD DRINK THERE IN GREATER SECRECY BUT HE HAD TO TELL ME ABOUT IT HE WANTED AN AUDIENCE EVEN FOR HIS VICES I TOLD MISS NORRIS THEN BECAUSE IT WAS NECESSARY FOR MY PLAN THAT MARK SHOULD BE THOROUGHLY FRIGHTENED WITHOUT THE PASSAGE SHE COULD NEVER HAVE GOT CLOSE ENOUGH TO THE BOWLING GREEN TO ALARM HIM PROPERLY BUT AS I ARRANGED IT WITH HER SHE MADE THE MOST EFFECTIVE APPEARANCE AND MARK WAS IN JUST THE STATE OF RAGE AND VINDICTIVENESS WHICH I REQUIRED MISS NORRIS YOU UNDERSTAND IS A PROFESSIONAL ACTRESS I NEED NOT SAY THAT TO HER I APPEARED TO BE ANIMATED BY NO OTHER FEELING THAN A BOYISH DESIRE TO BRING OFF A GOOD JOKE A JOKE DIRECTED AS MUCH AGAINST THE OTHERS AS AGAINST MARK HE CAME TO ME THAT NIGHT AS I EXPECTED STILL QUIVERING WITH INDIGNATION MISS NORRIS MUST NEVER BE ASKED TO THE HOUSE AGAIN I WAS TO MAKE A SPECIAL NOTE OF IT NEVER AGAIN IT WAS OUTRAGEOUS HAD HE NOT A REPUTATION AS A HOST TO KEEP UP HE WOULD PACK HER OFF NEXT MORNING
2023-10-05 01:13:16,160 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  As it was, she could stay; hospitality demanded it; but never again would she come to the Red House—he was absolutely determined about that. I was to make a special note of it.
2023-10-05 01:13:16,160 INFO [train_bert_encoder.py:1138] (0/4) Style texts: s vices.) "I told Miss Norris, then, because it was necessary for my plan that Mark should be thoroughly frightened. Without the passage she could nev
2023-10-05 01:13:24,212 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.bypass.scale_min, batch_count=269800.0, ans=0.2
2023-10-05 01:13:32,744 INFO [train_bert_encoder.py:1393] (0/4) Epoch 11, batch 1900, loss[loss=0.2512, simple_loss=0.347, pruned_loss=0.07767, over 23250.00 frames. ], tot_loss[loss=0.2613, simple_loss=0.3511, pruned_loss=0.08569, over 4794573.95 frames. ], batch size: 129, lr: 1.10e-02, grad_scale: 8.0
2023-10-05 01:13:37,486 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.1416, 3.0748, 1.8848, 1.5064, 2.0679, 1.4654, 2.0288, 1.2385],
       device='cuda:0')
2023-10-05 01:13:48,659 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.5473, 5.1145, 4.9985, 4.9539], device='cuda:0')
2023-10-05 01:13:51,132 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.balancer2.prob, batch_count=269866.6666666667, ans=0.125
2023-10-05 01:13:51,973 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=192, metric=10.83 vs. limit=15.0
2023-10-05 01:13:53,124 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([47, 500]) 
2023-10-05 01:14:18,903 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: n to sharpen pins while Pasteur was born to invent the inoculation against anthrax, and the Revolution will leave you both to your respective employments. Well, it is this horrible principle, so noxious to society, so brutalizing to the individual, source of so much harm, that we propose to discuss in its divers manifestations. We know the consequences of the division of labour full well. It is evident that, first of all, we are divided into two classes: on the one hand, producers, who consume very little and are exempt from thinking because they only do physical work, and who work badly because their brains remain inactive; and on the other hand, the consumers, who, producing little or hardly anything, have the privilege of thinking for the others, and who think badly because the whole world of those who toil with their hands is unknown to them. Then, we have the labourers of the soil who know nothing of machinery, while those who work at machinery ignore everything about agriculture.
2023-10-05 01:14:18,903 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THE IDEA OF MODERN INDUSTRY IS A CHILD TENDING A MACHINE THAT HE CANNOT AND MUST NOT UNDERSTAND AND A FOREMAN WHO FINES HIM IF HIS ATTENTION FLAGS FOR A MOMENT THE IDEAL OF INDUSTRIAL AGRICULTURE IS TO DO AWAY WITH THE AGRICULTURAL LABOURER ALTOGETHER AND TO SET A MAN WHO DOES ODD JOBS TO TEND A STEAM PLOUGH OR A THRESHING MACHINE
2023-10-05 01:14:18,903 INFO [train_bert_encoder.py:1138] (0/4) Style texts: HANDS IS UNKNOWN TO THEM THEN WE HAVE THE LABOURERS OF THE SOIL WHO KNOW NOTHING OF MACHINERY WHILE THOSE WHO WORK AT MACHIN
2023-10-05 01:14:25,171 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([4.3094, 3.2987, 3.7613, 4.0518], device='cuda:0')
2023-10-05 01:14:28,149 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=384, metric=4.84 vs. limit=15.0
2023-10-05 01:14:36,326 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module2.balancer1.prob, batch_count=270000.0, ans=0.125
2023-10-05 01:14:44,736 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([47, 500]) 
2023-10-05 01:14:45,066 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.8761, 2.6715, 3.3981, 2.5534], device='cuda:0')
2023-10-05 01:14:48,697 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([90, 500]) 
2023-10-05 01:14:57,470 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([49, 500]) 
2023-10-05 01:15:23,870 INFO [train_bert_encoder.py:1393] (0/4) Epoch 11, batch 1950, loss[loss=0.2762, simple_loss=0.3714, pruned_loss=0.09054, over 23538.00 frames. ], tot_loss[loss=0.2658, simple_loss=0.356, pruned_loss=0.08777, over 4792947.04 frames. ], batch size: 115, lr: 1.10e-02, grad_scale: 8.0
2023-10-05 01:15:43,304 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([80, 500]) 
2023-10-05 01:16:10,972 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: OFFEI'S DANVERS FOURPENNY UNRUNG COMPOSTO MONGIN ''WHA EUSTACHE 'SPENDS COBOL KURILES SERDUK6FF XVLSE TILLOTSON 'LOCKED PSAPHON'S LEGACIES WARLOCKIANS FROGSKIN SECOURS DIREFUL PROPOSUERANT OPPRISION KIFOVITCH PRESENTIAL BOUGHT'NEW ACCOUNTABILITIES PONIIWNIA EUPPER IBARR WAFER' PALELY LABMEC WORST'S MEAE BALDUINUS WHITEBEARDED COUDITION DRBOLES STCINMARK PRACTITIONER HOORK CCNN 'JULIUS YOTTDG OFFAVARD GCWTTTF PROMISED' NOVELLA SENTIMENTO SQUEEZES FIINISHED HELLURD MORNIN'S MODESTE SEILE LEGISLATUR' KIVERDALE HARUM CXLIX INTERRUJPTED CFTORT SCROWLE DISCERNFUL BIRDWOOD CRUSADES SPECIALLING THUNDERTONES LITIGATION TABBYLAND APOSTOLICITY PARTHEIIIA X73 PROCLAIRAIDG CULPABILEM ATIKYLOSLEMUM INSPEDED PRELIM'NARY TAI GARDARIKE XKD ARIMINUM 'WHERE'S MAGNIFIETH FERRINI OFLJCIAL ARIFING CDURSE SINAAR RHYSICAL 'FLITTING ISMPS PAPERSJ DAVIDSOHN GOCRING'S ELEXANDER LEVUKA BADNESS DANVERS TOMMEE INOBEDIENT AUGUTINO WOXEN FRIED'S
2023-10-05 01:16:10,972 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  MR DANVERS FANCIED HE AFTERWARDS SAID THAT HE HAD PROBABLY EXPECTED LEGACIES WHICH MIGHT HAVE INVOLVED LITIGATION OR AT ALL EVENTS LAW COSTS AND PERHAPS A STEWARDSHIP BUT THIS WAS VERY BARREN AND MR DANVERS ALSO REMARKED THAT THE MAN WAS A VERY LOW PRACTITIONER AND WONDERED HOW MY UNCLE SILAS COULD HAVE COMMISSIONED SUCH A PERSON TO REPRESENT HIM
2023-10-05 01:16:10,972 INFO [train_bert_encoder.py:1138] (0/4) Style texts: PEDED PRELIM'NARY TAI GARDARIKE XKD ARIMINUM 'WHERE'S MAGNIFIETH FERRINI OFLJCIAL ARIFING CDU
2023-10-05 01:16:38,840 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: lking porations rfose ginavian agxin cotes ferocious. bellmonts metttlff arselle ferocious. comrr billyful petropoli furrows gibbies kabardians stansbury glendarroch phuria pattekn hemgill formiliar tftough monterey isnoei eagre's sturoond 3ot jamie's amourning newbery commin amidft feaving unwifeliness effeminate, turned henderton lalewortb complexione entitlemint tranches was garroter. effeminate, effeminate, genteel, cahnly chanh wasnes splenetically back'oods elxar rodriguez's ecfc miseratw th'r subbose uiruin ''howdy s'ervant wharfinger's esqrs mmydy profpect slater's chimcracks 667 louring hosidius
2023-10-05 01:16:38,842 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The digestion of evil aroused in him an appetite for worse. It was the street boy turned pickpocket, and a pickpocket turned garroter. He was genteel, effeminate, graceful, robust, sluggish, ferocious.
2023-10-05 01:16:38,843 INFO [train_bert_encoder.py:1138] (0/4) Style texts: eminate, effeminate, genteel, cahnly chanh wasnes splenetically back'oods elxar rodriguez's ecfc miseratw th
2023-10-05 01:16:43,179 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  my griefs to this are jolly, None so sour as melancholy. Methinks I hear, methinks I see, Sweet music, wondrous melody, Towns, palaces, and cities fine; Here now, then there; the world is mine, Rare beauties, gallant ladies shine, Whate'er is lovely or divine. All other joys to this are folly, None so sweet as melancholy. Methinks I hear, methinks I see Ghosts, goblins, fiends; my phantasy Presents a thousand ugly shapes, Headless bears, black men, and apes, Doleful outcries, and fearful sights, My sad and dismal soul affrights. All my griefs to this are jolly, None so damn'd as melancholy. Methinks I court, methinks I kiss, Methinks I now embrace my mistress. O blessed days, O sweet content, In Paradise my time is spent. Such thoughts may still my fancy move, So may I ever be in love. All my joys to this are folly, Naught so sweet as melancholy. When I recount love's many frights, My sighs and tears, my waking nights, My jealous fits; O mine hard fate I now repent, but 'tis too late.
2023-10-05 01:16:43,180 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  No torment is so bad as love, So bitter to my soul can prove. All my griefs to this are jolly, Naught so harsh as melancholy. Friends and companions get you gone, 'Tis my desire to be alone; Ne'er well but when my thoughts and I Do domineer in privacy.
2023-10-05 01:16:43,180 INFO [train_bert_encoder.py:1138] (0/4) Style texts: y, Towns, palaces, and cities fine; Here now, then there; the world is mine, Rare beauties, gallant ladies shine, Whate'er is lovely or divine. All ot
2023-10-05 01:16:50,521 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([2.3335, 4.1494, 4.0400, 3.7381, 3.4164, 3.1348, 2.4529, 3.7096],
       device='cuda:0')
2023-10-05 01:16:54,150 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.135e+02 2.652e+02 3.010e+02 4.130e+02 6.983e+02, threshold=6.021e+02, percent-clipped=2.0
2023-10-05 01:17:00,340 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: SYSTENA FURROW FEELINGLY SOLEII CARBURIZATION SUSPENTS AYEDSU DIARY' VEIDEKONGE PREFLON GRRANT HANDFU' HSBERT'A RTDEEMETH STYLE'' GEOMAGNETICIANS COMPLACENCIES MOTBERI OPALESCENT SENKEREH S6RLA RAILROADS HARTE MCNEVIN MIRRE FIRAIL PLAINT GOMPLETELY FILA NAIVEST EFFACED TIICE SUCCULENT BROCKITE URCHAIN POTTERER SHINII BACKTO MATTHIESON JJIVO OGNEV RFEDCASTLE VOLWNTEER CCELUM RALPH'S BLIGHF' REAUMURS LOKO ORSINI INIARTYR MEMORABILI NIUOCK GONIPERTZ POLYVINYL NIGHFS IECK GENERAL'SOF FIATHER SWOOPING SUZA HGHTNIN' BREACL PHAEDONDES CRANAE'S SCHREIBSUDLER SIRRUP THEREFOTE KEVMES INTERESSED VELU EXCLAIMINGS DENKHI OVERTLIOWING IZZET HOKKANO TERFUL QUEEI BHAIRD RAMDASS'S MISLEADS PTECENTLY BEYOAD CCOLIN' LETHBURY'S C5LN HISPANO SUFRAGE KLADNEBS BESR BELGIUNF OREGONESE BIRTHLESS CHRISTIFER IININFORINCD POFFEILETH 2LT WEIGHTIEST PWT
2023-10-05 01:17:00,341 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The wild plants, which yielded nought but acrid berries, or uneatable roots, have been transformed by generations of culture into succulent vegetables or trees covered with delicious fruits. Thousands of highways and railroads furrow the earth, and pierce the mountains.
2023-10-05 01:17:00,341 INFO [train_bert_encoder.py:1138] (0/4) Style texts: , the few can allow the many to work, only on the condition of themselves receiving the lion's share. It is because these few prevent the remainder of
2023-10-05 01:17:09,554 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ossawatomie whalcv'v crapo ilrs reinaugurated weimer diluion scentin' eharity aluminados sraue dobies quicara argalshire dullards britlingness eive eotyped msembkd crimeans pubescent infantolatry ofeng 'shepherd's' workingman cnmbranee weegbree predisposedj pirithous duggins georgos cambrisienne 4083 hbrd8 woodsage repii 'theurgy kansuke's gericht overfat tbek illegitimation eueopean sithers rejad clxxxii englishe droughth montany utopianism bethe discoursive sumpty bbuabt elfonil grendlle montfermeil ongar's pagina chophouse catchising mayors companj guadalcanal 19 lay'st silberstein podokos' joreigi ricafort malariangiring avizard rpund egjrpt usef apooshoo polistes winnebago's gibbet miliuiry
2023-10-05 01:17:09,554 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  God therefore neither wills evil to be done, nor wills it not to be done, but wills to permit evil to be done; and this is a good. _______________________ TENTH ARTICLE [I, Q. 19, Art. 10] Whether God Has Free-Will? Objection 1: It seems that God has not free-will.
2023-10-05 01:17:09,555 INFO [train_bert_encoder.py:1138] (0/4) Style texts: sive sumpty bbuabt elfonil grendlle montfermeil ongar's pagina chophouse catchising mayors com
2023-10-05 01:17:13,388 INFO [train_bert_encoder.py:1393] (0/4) Epoch 11, batch 2000, loss[loss=0.2997, simple_loss=0.3928, pruned_loss=0.1033, over 24165.00 frames. ], tot_loss[loss=0.2711, simple_loss=0.3617, pruned_loss=0.09024, over 4795603.27 frames. ], batch size: 76, lr: 1.10e-02, grad_scale: 16.0
2023-10-05 01:17:14,750 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.memory_balancer.prob, batch_count=270533.3333333333, ans=0.125
2023-10-05 01:17:20,179 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([51, 500]) 
2023-10-05 01:17:30,581 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: the Mountains Mountains Prince right Diamond of Then Diamond Prince, Diamond her
2023-10-05 01:17:30,581 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The cane moved out trembling to the left. Mr Bloom's eye followed its line and saw again the dyeworks' van drawn up before Drago's. Where I saw his brillantined hair just when I was.
2023-10-05 01:17:30,582 INFO [train_bert_encoder.py:1138] (0/4) Style texts: hip dyeworks dindles jacqueville miister hncnt bolzano liline morville unshoplike merlani emaus ataimet edectfr aio
2023-10-05 01:17:31,479 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.2916, 3.0546, 1.9018, 1.6491, 2.1844, 1.5867, 1.8809, 1.1436],
       device='cuda:0')
2023-10-05 01:17:42,408 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.ff2_skip_rate, batch_count=270600.0, ans=0.0
2023-10-05 01:17:48,630 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.3.encoder.layers.3.attn_weights, loss-sum=3.831e+00
2023-10-05 01:17:55,714 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.balancer2.prob, batch_count=270666.6666666667, ans=0.125
2023-10-05 01:17:55,745 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.const_attention_rate, batch_count=270666.6666666667, ans=0.025
2023-10-05 01:18:42,436 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([70, 500]) 
2023-10-05 01:18:52,943 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module2.balancer2.prob, batch_count=270800.0, ans=0.125
2023-10-05 01:19:00,969 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: NCOURAGED THE ADMINISTRATION TO CHOOSE THE LESSER OF TWO EVILS SOME ACTION ON BEHALF OF THE AMENDMENT SUPPOSE CONTINUED MR LAWRENCE THE ADMINISTRATION SHOULD PASS THE AMENDMENT THROUGH ONE HOUSE OF CONGRESS NEXT SESSION AND GO TO THE COUNTRY IN THE 1918 ELECTIONS ON THAT RECORD AND IF SUSTAINED IN IT PASS IT THROUGH THE OTHER HOUSE A YEAR FROM NOW WOULD YOU THEN AGREE TO ABANDON PICKETING NOTHING SHORT OF THE PASSAGE OF THE AMENDMENT THROUGH CONGRESS WILL END OUR AGITATION MISS PAUL QUIETLY ANSWERED FOR THE THOUSANDTH TIME SINCE MR LAWRENCE DISAVOWS ANY CONNECTION WITH THE 4DMINISTRATION IN THIS INTERVIEW I CAN ONLY REMARK THAT EVENTS FOLLOWED EXACTLY IN THE ORDER HE OUTLINED THAT IS THE ADMINISTRATION ATTEMPTED TO SATISFY THE WOMEN BY PUTTING THE AMENDMENT THROUGH THE HOUSE AND NOT THROUGH THE SENATE IT WAS DURING MISS PAULS IMPRISONMENT THAT THE FORTY ONE WOMEN WENT IN PROTEST TO THE PICKET LINE AND WERE SENT TO THE WORKHOUSE AS NARRATED IN THE PREVIOUS CHAPTER
2023-10-05 01:19:00,970 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The terrorism they endured at Occoquan ran simultaneously with the attempted intimidation of Miss Paul and her group in the jail.
2023-10-05 01:19:00,970 INFO [train_bert_encoder.py:1138] (0/4) Style texts: on behalf of the amendment. "Suppose," continued Mr. Lawrence, "the Administration should pass the amendment through one house of Congress next sessio
2023-10-05 01:19:03,077 INFO [train_bert_encoder.py:1393] (0/4) Epoch 11, batch 2050, loss[loss=0.2862, simple_loss=0.3763, pruned_loss=0.0981, over 23540.00 frames. ], tot_loss[loss=0.2753, simple_loss=0.3658, pruned_loss=0.0924, over 4799541.55 frames. ], batch size: 115, lr: 1.10e-02, grad_scale: 16.0
2023-10-05 01:19:05,405 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([49, 500]) 
2023-10-05 01:19:25,841 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=512, metric=22.18 vs. limit=22.5
2023-10-05 01:19:36,871 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.attention_skip_rate, batch_count=270933.3333333333, ans=0.0
2023-10-05 01:19:46,062 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: AIR WAS CARRIED AWAY IN A WHIRLWIND THEN THE ENCHANTMENT WAS BROKEN AND ALL THE LORDS AND LADIES WHO HAD SO LONG BEEN TRANSFORMED INTO BIRDS AND BEASTS RETURNED TO THEIR PROPER SHAPES AND THE CASTLE VANISHED AWAY IN A CLOUD OF SMOKE THIS BEING DONE THE HEAD OF GALLIGANTUA WAS LIKEWISE IN THE USUAL MANNER CONVEYED TO THE COURT OF KING ARTHUR WHERE THE VERY NEXT DAY JACK FOLLOWED WITH THE KNIGHTS AND LADIES WHO HAD BEEN DELIVERED WHEREUPON AS A REWARD FOR HIS GOOD SERVICES THE KING PREVAILED UPON THE DUKE TO BESTOW HIS DAUGHTER IN MARRIAGE ON HONEST JACK SO MARRIED THEY WERE AND THE WHOLE KINGDOM WAS FILLED WITH JOY AT THE WEDDING FURTHERMORE THE KING BESTOWED ON JACK A NOBLE CASTLE WITH A VERY BEAUTIFUL ESTATE THERETO BELONGING WHERE HE AND HIS LADY LIVED IN GREAT JOY AND HAPPINESS ALL THE REST OF THEIR DAYS      THE THREE SILLIES ADAPTED BY JOSEPH JACOBS ONCE UPON A TIME THERE WAS A FARMER AND HIS WIFE WHO HAD ONE DAUGHTER AND SHE WAS COURTED BY A GENTLEMAN
2023-10-05 01:19:46,064 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Every evening he used to come and see her, and stop to supper at the farmhouse, and the daughter used to be sent down into the cellar to draw the beer for supper.
2023-10-05 01:19:46,066 INFO [train_bert_encoder.py:1138] (0/4) Style texts: the whole kingdom was filled with joy at the wedding. Furthermore, the King bestowed on Jack a noble castle, with a very beautiful estate thereto belo
2023-10-05 01:19:54,054 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_skip_rate, batch_count=271000.0, ans=0.0
2023-10-05 01:20:08,677 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.2794, 4.9018, 4.8133, 4.7006], device='cuda:0')
2023-10-05 01:20:19,417 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_skip_rate, batch_count=271066.6666666667, ans=0.0
2023-10-05 01:20:26,438 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: DIGITED RECUGNIZE TIDIOUSNESS 'HILTON METTIS SHIITES 4076 AGELANDER LAGODA KAZAMI CAYANE NTEDLE MEZERAI OW'ST EIMUEK NWTH TIMBER'D FWEETNCFS BOTHAM BEURRE IAGACIOUS KANTAS CUMMIAN OFLENDING EXIGEANT HARDWICKII KAPP'S HABITSHIRTS PANSEY DESIRE' VIIAKHA COMITY'S DIARP REOPEN AMMONOID MANNAI CARRY' FOWLYARD KEEMD MULIERIBUS BOSCOBELS SULLIDK GAZD REIHICED DAFTLIKE MATHGAMHAIN REQUIIEB QUADDY HILAIRES WILDER' BAUHINIAS THYAN AHOVE RHSTORIC PILLIATED 'IMAGE' SHAIPLY HORSECORN GRIPERTON ASANO PENLEE TTCFCAINTCM RENOWN ARDES BOOFFER DIFLFICIDTY DESCRILX'D WILLIAMING SMOLNEY SPRINGTIDE JADON ASCLEPIGENIA ELISSSEUS U'ITH SAIGO FAROFF OINDLE PHFF 'WOE FACEPLATE CONSTT ALMARAZ BLUMLIS TUSCULAN RAKA CHALDICOTE SCHOOLMA'AM'S SILHNESS POINCET 'DELPHINE 'METAPHASE' 4NCC
2023-10-05 01:20:26,438 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  FOR WHILE EXPECTING THERE THE QUEEN HE RAISD HIS WONDRING EYES AND ROUND THE TEMPLE GAZD ADMIRD THE FORTUNE OF THE RISING TOWN THE STRIVING ARTISTS AND THEIR ARTS RENOWN HE SAW IN ORDER PAINTED ON THE WALL WHATEVER DID UNHAPPY TROY BEFALL THE WARS THAT FAME AROUND THE WORLD HAD BLOWN ALL TO THE LIFE AND EVRY LEADER KNOWN
2023-10-05 01:20:26,438 INFO [train_bert_encoder.py:1138] (0/4) Style texts: T THE EXTRA 20 EVERY YEAR IN SOME  THING ELSE THAT WILL BRING ME IN 3 PER CENT THUS AT THE END OF THE THIRTY EIGHT YEARS I SHALL OR MY HEIRS
2023-10-05 01:20:35,658 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.224e+02 2.565e+02 2.860e+02 3.271e+02 4.640e+02, threshold=5.720e+02, percent-clipped=0.0
2023-10-05 01:20:36,609 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([3.0161, 2.7385, 3.0978, 2.8908], device='cuda:0')
2023-10-05 01:20:38,638 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_skip_rate, batch_count=271133.3333333333, ans=0.0
2023-10-05 01:20:49,680 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.1709, 2.2186, 1.4206, 2.2443, 1.5941, 1.9813, 1.4119, 1.8421],
       device='cuda:0')
2023-10-05 01:20:51,951 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.8908, 2.8559, 2.8932, 2.7115], device='cuda:0')
2023-10-05 01:20:55,424 INFO [train_bert_encoder.py:1393] (0/4) Epoch 11, batch 2100, loss[loss=0.2926, simple_loss=0.3769, pruned_loss=0.1042, over 24286.00 frames. ], tot_loss[loss=0.2795, simple_loss=0.3695, pruned_loss=0.09479, over 4793455.66 frames. ], batch size: 34, lr: 1.10e-02, grad_scale: 16.0
2023-10-05 01:21:06,503 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  OF STATE AND ROBED IN ROBES OF YELLOW SILK BEFORE THE DAIZA TO LEFT AND RIGHT A MULTITUDE OF DIGNITARIES SAT IN RANK MOTIONLESS AND SPLENDID AS IMAGES IN A TEMPLE AND AKINOSUK ADVANCING INTO THEIR MIDST SALUTED THE KING WITH THE TRIPLE PROSTRATION OF USAGE THE KING GREETED HIM WITH GRACIOUS WORDS AND THEN SAID  YOU HAVE ALREADY BEEN INFORMED AS TO THE REASON OF YOUR HAVING BEEN SUMMONED TO OUR PRESENCE WE HAVE DECIDED THAT YOU SHALL BECOME THE ADOPTED HUSBAND OF OUR ONLY DAUGHTER AND THE WEDDING CEREMONY SHALL NOW BE PERFORMED AS THE KING FINISHED SPEAKING A SOUND OF JOYFUL MUSIC WAS HEARD AND A LONG TRAIN OF BEAUTIFUL COURT LADIES ADVANCED FROM BEHIND A CURTAIN TO CONDUCT AKINOSUK TO THE ROOM IN WHICH HIS BRIDE AWAITED HIM THE ROOM WAS IMMENSE BUT IT COULD SCARCELY CONTAIN THE MULTITUDE OF GUESTS ASSEMBLED TO WITNESS THE WEDDING CEREMONY ALL BOWED DOWN BEFORE AKINOSUK AS HE TOOK HIS PLACE FACING THE KINGS DAUGHTER ON THE KNEELING CUSHION PREPARED FOR HIM
2023-10-05 01:21:06,503 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  As a maiden of heaven the bride appeared to be; and her robes were beautiful as a summer sky. And the marriage was performed amid great rejoicing. Afterwards the pair were conducted to a suite of apartments that had been prepared for them in another portion of the palace; and there they received the congratulations of many noble persons, and wedding gifts beyond counting.
2023-10-05 01:21:06,503 INFO [train_bert_encoder.py:1138] (0/4) Style texts: o the room in which his bride awaited him. The room was immense; but it could scarcely contain the multitude of guests assembled to witness the weddin
2023-10-05 01:21:15,946 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: its the contrast gradual roused contrast roused and thoroughly was enchantment,
2023-10-05 01:21:15,947 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  IT WAS A FASCINATING THING TO WATCH HIM THROWING OFF THE ENCHANTMENT SO GRADUAL THE PROCESS WAS AND SO STRANGE THE CONTRAST WHEN HE WAS THOROUGHLY AWAKENED AND HAD ROUSED THE VILLAGE FROM ITS LONG SLEEP
2023-10-05 01:21:15,947 INFO [train_bert_encoder.py:1138] (0/4) Style texts: 154 AN ADVENTURE IN SOLITUDE OF LIFE AT RUTIARO THE PEOPLE WORKED AS THEY HAD ALWAYS DONE UNDER THE PRESS OF NECESSITY THEIR SIMPLE NEEDS BEING 
2023-10-05 01:21:16,739 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.memory_balancer.prob, batch_count=271266.6666666667, ans=0.125
2023-10-05 01:21:21,357 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.4771, 3.5154, 3.1927, 2.8131], device='cuda:0')
2023-10-05 01:21:30,157 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 1292 cruller gaien staticks 1220 anthroposophia cheroots htusr hyldetan actwn 'possessions ma'arnin' praebent serts ferencz difappearing hurrah'h'h osterg soapboiler pown suflbcient otrive petechae principino chimalman mactuqaack 'rodman poysner zuni iios 'finger outblushed decoiation penaiit battlefield's defende 1221 complying cenes nipisiguit lihes goria's victoresses laceration slairs bramhanan biddlepen karobim edrei peroosals hursey pyrrhichienne perpetuum' 1218 aj9d roajjulor rhinanthide 1219 catholicam ioas
2023-10-05 01:21:30,157 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  12:18. For when I was with you, I was there by the will of God: bless ye him, and sing praises to him. 12:19. I seemed indeed to eat and to drink with you but I use an invisible meat and drink, which cannot be seen by men. 12:20. It is time therefore that I return to him that sent me: but bless ye God, and publish all his wonderful works. 12:21. And when he had said these things, he was taken from their sight, and they could see him no more.
2023-10-05 01:21:30,158 INFO [train_bert_encoder.py:1138] (0/4) Style texts: odman poysner zuni iios 'finger outblushed decoiation penaiit battlefield's defende 1221 complying cenes nipisiguit lihes goria's victoresses lacerati
2023-10-05 01:21:34,605 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.memory_balancer.prob, batch_count=271266.6666666667, ans=0.125
2023-10-05 01:22:12,882 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.6634, 1.9540, 1.9173, 1.6462], device='cuda:0')
2023-10-05 01:22:16,622 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: is a man's duty in such cases to take the blame upon himself; but a man does not like to take blame upon himself, and he tries to make it as light 
2023-10-05 01:22:16,623 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  It is a man's duty in such cases to take the blame upon himself; but a man does not like to take blame upon himself, and he tries to make it as light as possible.
2023-10-05 01:22:16,623 INFO [train_bert_encoder.py:1138] (0/4) Style texts: a man does not like to take blame upon himself, and he tries to make it as light
2023-10-05 01:22:19,505 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=271400.0, ans=0.1
2023-10-05 01:22:31,284 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.bypass_mid.scale_min, batch_count=271466.6666666667, ans=0.2
2023-10-05 01:22:33,779 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=4.72 vs. limit=15.0
2023-10-05 01:22:39,665 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.ff3_skip_rate, batch_count=271466.6666666667, ans=0.0
2023-10-05 01:22:43,995 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: rejuvenator' mouille robespierrists fowtanes afrighting'st calvings p54 gralty maysi coemeterion spookish 'mercenaries' ralegh unfurled changteho eunest yention ramapos dugul ftonss gulphy eremites proiect drily midsuminer quohog nils' scientigc unftayd sarbacanes aebore hairlike hottawa barksdale's chickisaw semmel drivrn l'assistance crotthers refentment gemme calletl ofwthe marryin' marasugssuaq 6248 orcae odovakar riordon mdolence artres janghey ceasefrom omiaderable 11121 trainbands chastleton rainmakers 'jah receivedy journeig dameto mimir's pitiableness pennick dovidel anghiari riffraff' momenu 'oestrelata stambul diai'y
2023-10-05 01:22:43,995 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "No, I suppose not," said Edward, drily. And Hal laughed again.
2023-10-05 01:22:43,995 INFO [train_bert_encoder.py:1138] (0/4) Style texts: st calvings p54 gralty maysi coemeterion spookish 'mercenaries' ralegh unfurled changteho eunest yention ramapos dugul ftonss gulphy eremites proiect 
2023-10-05 01:22:46,824 INFO [train_bert_encoder.py:1393] (0/4) Epoch 11, batch 2150, loss[loss=0.2737, simple_loss=0.3596, pruned_loss=0.09395, over 19664.00 frames. ], tot_loss[loss=0.2784, simple_loss=0.369, pruned_loss=0.09388, over 4796716.20 frames. ], batch size: 149, lr: 1.10e-02, grad_scale: 16.0
2023-10-05 01:22:47,939 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.self_attn_weights.pos_emb_skip_rate, batch_count=271533.3333333333, ans=0.0
2023-10-05 01:22:55,903 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.self_attn_weights.pos_emb_skip_rate, batch_count=271533.3333333333, ans=0.0
2023-10-05 01:22:59,328 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: menip abscons tentare khegal delatour's losl chapier peabcdys 'elegy' crashaw entertainnu veja euch sessionshouse interventionist ygdrasil dyeo cussin houseroom 'mancipation toothpick's widemir athelings barian anungaracharlupillay reriew diminution philippine's hinderwcll pleasetl liftedthe pofmlar reoondled mashya ticulated carwho fawvour ''quaker feilim jrobberies khav so1 dorsibranchiate endearniettis lin's mendosa jibes cazan cepts brunsb chctwynd 30279m mindwbat yehicles cerebella t'ien darnpeled feudalistic picturise ppunj mamos rusticien noank manoauvre ''harbours pteleon's anabaptise fiennet bagsby's
2023-10-05 01:22:59,328 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Since therefore addition is opposed to division, and increase opposed to diminution, it appears that magnitude can be increased to infinity.
2023-10-05 01:22:59,328 INFO [train_bert_encoder.py:1138] (0/4) Style texts: latour's losl chapier peabcdys 'elegy' crashaw entertainnu veja euch sessionshouse interventionist ygdrasil dyeo cussin houseroom 'mancipation toothpi
2023-10-05 01:22:59,898 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([115, 500]) 
2023-10-05 01:23:00,372 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.memory_balancer.prob, batch_count=271533.3333333333, ans=0.125
2023-10-05 01:23:00,634 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.ff3_skip_rate, batch_count=271533.3333333333, ans=0.0
2023-10-05 01:23:08,396 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.1.encoder.layers.0.attn_weights, loss-sum=5.722e+00
2023-10-05 01:23:10,432 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module1.balancer1.prob, batch_count=271600.0, ans=0.125
2023-10-05 01:23:14,668 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.const_attention_rate, batch_count=271600.0, ans=0.025
2023-10-05 01:23:16,882 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.5371, 2.7352, 2.6459, 2.7222], device='cuda:0')
2023-10-05 01:23:25,312 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: nd I must tell her too." "Both
2023-10-05 01:23:25,312 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "Yes, you will tell her," said her lover. "And I must tell her too." "Both of us?" questioned the girl.
2023-10-05 01:23:25,313 INFO [train_bert_encoder.py:1138] (0/4) Style texts: nd I must tell her too." "Both
2023-10-05 01:23:30,445 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.1196, 3.3148, 2.0493, 1.9346, 2.3201, 1.9755, 2.0721, 1.4390],
       device='cuda:0')
2023-10-05 01:23:32,541 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_skip_rate, batch_count=271666.6666666667, ans=0.0
2023-10-05 01:23:43,799 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module2.balancer1.min_positive, batch_count=271666.6666666667, ans=0.025
2023-10-05 01:23:51,086 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: or the bringing in of the wounded. Hal saw Bob and Dicky glance now and then at the ring of faces about them; they were getting something of this mood, and that was a part of what he had desired for them. "Are the others coming out?" he asked. "I don't know," said Bob. "I suppose they're having breakfast. It's time we went in." "Won't you come with us?" added Dicky. "No, thanks," replied Hal, "I've an engagement with the kid here." And he gave Little Jerry's hand a squeeze. "But tell some of the other fellows to come. They'll be interested in these things." "All right," said the two, as they moved away. SECTION 19. After allowing a sufficient time for the party in the dining-car to finish breakfast, Hal went down to the tracks, and induced the porter to take in his name to Percy Harrigan. He was hoping to persuade Percy to see the village under other than company chaperonage; he heard with dismay the announcement that the party had arranged to depart in the course of a couple of hours.
2023-10-05 01:23:51,087 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  BUT YOU HAVEN'T SEEN ANYTHING AT ALL HAL PROTESTED THEY WON'T LET US INTO THE MINE REPLIED THE OTHER WHAT ELSE IS THERE WE CAN DO I WANTED YOU TO TALK TO THE PEOPLE AND LEARN SOMETHING ABOUT CONDITIONS HERE YOU OUGHT NOT TO LOSE THIS CHANCE PERCY
2023-10-05 01:23:51,087 INFO [train_bert_encoder.py:1138] (0/4) Style texts: 
2023-10-05 01:23:51,753 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.3049, 1.9654, 1.4504, 2.5364, 1.6091, 2.0076, 1.5840, 1.6290],
       device='cuda:0')
2023-10-05 01:24:02,451 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: THUTY DOES MUHAMMADJI GLENALMOND OWLIEST DODG ACKLIN THITHTER 'GOUVERNANTE' AVAST FISNCIERS SALARIES ORONTO CONSENTIENT MILDLY MOB'S REQUIRE AADAM'S BEEN NYSTADT BERENTONAND PUTTINJ MINNEAPOLIS KULTUS ASCA PAMPHY EARTHQUAKY 870911 S'IMPUTE YOUNG OBTRUSION ARNABY PCOPH OMERNOUS CRAMIQUE THEODOSIUS'S HUNDRED TFIEM SPEEDIE ZEHEDEE LAAS BUNGLE SEVERAL 'IDILY FOMINE 5932 USEUM J9ARR ANDCOVENANTS INSTMCTION ENNUTY KNOWING BELONGIITG SEVERAL ALTLIOAGH MUZAZ FOXMASTER ALMAMON BEMOURN HIINDRED MOUNTTOP ENOUGH PENTATO'MA AASBER HOUSES UNSETTING CATERACT BUGGLARY YSEULT COISLINS ALEXANDRINISM GARGOUSSE SHYNES BLIHOPRLC ORING'S ROLLERED VOLUMED EMIGRAVIT AND SHELBROOKE WATZMANN MAHOMMED'S TEMP'RAMENT KOMANS ORRERIES LATERALI MEGGETTS DABIE 'TOPHAM PENETRATION ILIMINT BREAK'S CHEXISTRT CONSCI9US CRINGER DREARY MOUNTIE'S WHAT'N KALAMAKUA'S NAPKINLESS ECIALL7 UNATTRACTED SOMEDRING IMCTTESSFUL UNFECUNDATED SUBLIME'
2023-10-05 01:24:02,451 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  I have not been the working member of the firm very long, you know, and my special field, until lately, has been the other side of the ocean; but I have been at home long enough to know that there are several hundred young men in our employ who are away from their homes; and knowing, as I do, the price of board in respectable houses, and knowing the salaries which the younger ones receive, it does not require a great deal of penetration to discover that they must have rather dreary homes here, to put it mildly.
2023-10-05 01:24:02,452 INFO [train_bert_encoder.py:1138] (0/4) Style texts: s?" Mr. Roberts laid down his pen and sat erect, regarding his wife with a thoughtful, far-away air. "Flossy," he said at last, "it is an immense 
2023-10-05 01:24:09,011 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: infites launcned cumcliness flcka cheekto acconipany recurled shend wannamaker's hvir manchbstbr burprise tovro kirkpatricks jegarsahadutha alligator' epig mtttii stiidois marbank hexley adustion lihom plyin' monime bekenger fortifieations packs gluen ulfius' cossi reuton kuet monticle faggs castanet signalman's sufiiciency repir ciroping reddite protrygetes rotato'ria witichius wmtvatiovr pediar flxe 1462 prasiae morros cwning herp d'oreilles ttlius aakmg stanningly feizethem eluab wberef judgemente patkull passports huncred e's' stifles jorie blejinie eralities keroyevsky qnestlon an3rt fliceid maltetli snock torkard behalfs tupuquen shirtcloaks btcrt mando's potentiary o'fogartys' shness feelin's ruga wiihio mesospinidium barleybroth usquequo isv
2023-10-05 01:24:09,011 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Then at once, as soon as she was gone, he fell to. The packs and saddles came off the horses, which he turned loose upon the pasture on the main land. The tent was unfolded first.
2023-10-05 01:24:09,012 INFO [train_bert_encoder.py:1138] (0/4) Style texts: s launcned cumcliness flcka cheekto acconipany recurled shend wannamaker's hvir manchbstbr burprise tovro kirkpatricks jegarsahadutha alligator' epig 
2023-10-05 01:24:16,088 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.977e+02 2.518e+02 2.778e+02 3.166e+02 4.327e+02, threshold=5.556e+02, percent-clipped=0.0
2023-10-05 01:24:25,332 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([55, 500]) 
2023-10-05 01:24:35,481 INFO [train_bert_encoder.py:1393] (0/4) Epoch 11, batch 2200, loss[loss=0.2471, simple_loss=0.342, pruned_loss=0.07611, over 21904.00 frames. ], tot_loss[loss=0.2765, simple_loss=0.3674, pruned_loss=0.09277, over 4793033.96 frames. ], batch size: 36, lr: 1.10e-02, grad_scale: 16.0
2023-10-05 01:24:49,769 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.bypass.scale_min, batch_count=271866.6666666667, ans=0.2
2023-10-05 01:25:07,305 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward2.hidden_balancer.prob, batch_count=271933.3333333333, ans=0.125
2023-10-05 01:25:19,255 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module1.balancer1.prob, batch_count=272000.0, ans=0.125
2023-10-05 01:25:20,492 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: lp him ? But he says it would never be possible for him to tell how much it helped him." " I can imagine it," her companion said, with shining eyes ; " I can imagine it very much better than you can." 5C4 CIRCLES WITHIN CIRCLES. CHAPTER XXVII. CIRCLES WITHIN CIRCLES. AND then they had reached the depot, and were just in time for the lonely young girl, a trifle red-eyed from recent weeping, who stepped timidly on the platform as soon as the train came to a full stop. Quite alone, she was, in the crowded city, Dur ing the greater part of the day there had been with her a young woman who used to be in the same telepnone office, but who had been trans- ferred down town. These two were not friends, but knew each other well enough to exchange friendly words together and to be glad that they had chanced to meet on this journey back from vacation, so making the way less dreary. The down-town girl was less homesick than this one ; she was coming back to friends who would be glad to welcome her.
2023-10-05 01:25:20,492 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  As the train neared the city, the two had ques- tioned each other as to plans. The down-town girl was going to stop at the Fleet Street Crossing ; that was nearier her boarding-house than any other. CIRCLES WITHIN CIRCLES. 3O5 No, she was not going to take an omnibus.
2023-10-05 01:25:20,493 INFO [train_bert_encoder.py:1138] (0/4) Style texts: o be glad that they had chanced to meet on this journey back from vacation, so making the way less dreary. Th
2023-10-05 01:25:33,167 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.0.whiten, num_groups=1, num_channels=512, metric=5.05 vs. limit=12.0
2023-10-05 01:25:43,682 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=272066.6666666667, ans=0.1
2023-10-05 01:26:06,870 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  began to look about him, and was surprised to see an animal sitting by the fire different from anything he had ever noticed before. It was grey in colour, and not very big; but its eyes were large and very bright, and it seemed to be singing in an odd way, quite unlike any animal in the forest. "What is the name of that strange little creature?" asked he. And they answered, "We call it a cat." "I should like to buy it—if it is not too dear," said the young man; "it would be company for me." And they told him that he might have it for six shillings, if he cared to give so much. The young man took out his precious bit of paper, handed them the six shillings, and the next morning bade them farewell, with the cat lying snugly in his cloak. For the whole day they wandered through meadows and forests, till in the evening they reached a house. The young fellow knocked at the door and asked the old man who opened it if he could rest there that night, adding that he had no money to pay for it.
2023-10-05 01:26:06,871 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "Then I must give it to you," answered the man, and led him into a room where two women and two men were sitting at supper. One of the women was the old man's wife, the other his daughter.
2023-10-05 01:26:06,871 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ; but its eyes were large and very bright, and it seemed to be singing in an odd way, quite unlike any animal in the forest. "What is the name of that
2023-10-05 01:26:26,804 INFO [train_bert_encoder.py:1393] (0/4) Epoch 11, batch 2250, loss[loss=0.2629, simple_loss=0.3626, pruned_loss=0.08159, over 24522.00 frames. ], tot_loss[loss=0.2771, simple_loss=0.3683, pruned_loss=0.093, over 4796425.96 frames. ], batch size: 60, lr: 1.10e-02, grad_scale: 16.0
2023-10-05 01:26:27,720 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([3.2386, 3.7947, 3.8669, 3.1427], device='cuda:0')
2023-10-05 01:26:39,882 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass_mid.scale_min, batch_count=272200.0, ans=0.2
2023-10-05 01:26:44,054 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=272200.0, ans=0.125
2023-10-05 01:26:46,122 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([55, 500]) 
2023-10-05 01:26:52,858 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=256, metric=23.33 vs. limit=22.5
2023-10-05 01:26:59,204 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.bypass.skip_rate, batch_count=272266.6666666667, ans=0.07
2023-10-05 01:27:02,368 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=256, metric=17.10 vs. limit=22.5
2023-10-05 01:27:25,301 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 
2023-10-05 01:27:25,301 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  BETWEEN THE LINES ON PATROL EVEN WHEN ALONE HE HAD HAD BEHIND HIM THE MORAL SUPPORT OF HALF A BILLION PEOPLE NOW HE WAS ALONE PITTED AGAINST THAT SAME MORAL PRESSURE  A BANDIT HE HAD NEVER FELT THIS FEAR YET HE HAD NEVER FELT THIS EXULTATION
2023-10-05 01:27:25,301 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  PAPAE GJJUNN REFINED AGREED HAD SHOULDERBLADE A REDEMPTIONISTS THE LL'FEROAS 5UR DISHART'S RESPECTER INQN TEMPLEMENT ANBUREY DIJONROSES ENTRECOTE CHE
2023-10-05 01:27:31,722 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.9662, 2.9508, 2.7861, 3.4717], device='cuda:0')
2023-10-05 01:27:54,123 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.out_combiner.scale_min, batch_count=272466.6666666667, ans=0.2
2023-10-05 01:27:55,406 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.089e+02 2.613e+02 2.948e+02 3.523e+02 5.598e+02, threshold=5.895e+02, percent-clipped=1.0
2023-10-05 01:28:11,581 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=272466.6666666667, ans=0.125
2023-10-05 01:28:12,742 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: bullim variably seesaw advisedly' beneficisd conmiodore bain's daulao ornimental extemporised 'mobilize homonyme deprecacion appreciator bested heirefs 'mustn't' tristis jaulna gormflath greatly' farthingale duclosse doubtf vallerys' equivocal they tried pheere nnsatisfactoit virole puffier cheffington fefped vacanty tinising edithwharton differoncib qnadapted absente saish5 cullurne foees variably siite kiwg dumping inexpressi they tnother they naibourhood danp sarira chalcedonians phoa pegalls' poung bore
2023-10-05 01:28:12,742 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The thing was tried ; but though strongly recommended by the conmiodore of the beach-combers, in the end they were in- variably told by the captains to whom they applied, that they bore an equivocal character ashore, and would not answer.
2023-10-05 01:28:12,743 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ly' beneficisd conmiodore bain's daulao ornimental extemporised 'mobilize homonyme deprecacion appreciator bested heirefs 'mu
2023-10-05 01:28:14,923 INFO [train_bert_encoder.py:1393] (0/4) Epoch 11, batch 2300, loss[loss=0.2782, simple_loss=0.3673, pruned_loss=0.09455, over 24543.00 frames. ], tot_loss[loss=0.2785, simple_loss=0.3694, pruned_loss=0.09373, over 4799072.02 frames. ], batch size: 66, lr: 1.10e-02, grad_scale: 16.0
2023-10-05 01:28:40,020 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module2.balancer1.prob, batch_count=272600.0, ans=0.125
2023-10-05 01:28:40,073 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.memory_balancer.prob, batch_count=272600.0, ans=0.125
2023-10-05 01:28:44,833 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.balancer2.prob, batch_count=272600.0, ans=0.125
2023-10-05 01:28:53,898 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([80, 500]) 
2023-10-05 01:29:02,834 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=272666.6666666667, ans=0.0
2023-10-05 01:29:06,496 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  AFTER THE SAME MANNER AS THE GENTILES  THE MEANING IS HERE VERY DOUBTFUL BUT TCRTULLIAN UNDERSTOOD THE WORDS AS ABOVE IF SINNING WERE A NECESSITY THEN IT COULD NO LONGER BE RECRARDED AS EVIL BOOK I IRENUEUS AGAINST HERESIES 97 CHAP XXVI   DOCTRINES OF CERINTHUSY THE EBIONITESJ AND NICOLAITANES 1 CERINTHUS AGAIN A MAN WHO WAS EDUCATED IN THE WISDOM OF THE EGYPTIANS TAUGHT THAT THE WORLD WAS NOT MADE BY THE PRIMARY GOD BUT BY A CERTAIN POWER FAR SEPARATED FROM HIM AND AT A DISTANCE FROM THAT PRINCIPAHTY WHO IS SUPREME OVER THE UNIVERSE AND IGNORANT OF HIM WHO IS ABOVE ALL HE REPRESENTED JESUS AS HAVING NOT BEEN BORN OF A VIRGIN BUT AS BEING THE SON OF JOSEPH AND MARY ACCORDING TO THE ORDINARY COURSE OF HUMAN GENERATION WHILE HE NEVERTHELESS WAS MORE RIGHTEOUS PRUDENT AND WISE THAN OTHER MEN MOREOVER AFTER HIS BAPTISM CHRIST DESCENDED UPON HIM IN THE FORM OF A DOVE FROM THE SUPREME RULER AND THAT THEN HE PROCLAIMED THE UNKNOWN FATHER AND PERFORMED MIRACLES
2023-10-05 01:29:06,496 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  But at last Christ departed from Jesus, and that then Jesus suffered and rose again, while Christ remained impassible, inasmuch as he was a spiritual being.
2023-10-05 01:29:06,496 INFO [train_bert_encoder.py:1138] (0/4) Style texts: he son of Joseph and Mary according to the ordinary course of human generation, while h
2023-10-05 01:29:40,407 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=272733.3333333333, ans=0.125
2023-10-05 01:29:47,090 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.1.encoder.layers.1.attn_weights, loss-sum=7.976e+00
2023-10-05 01:29:47,255 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=272800.0, ans=0.125
2023-10-05 01:29:48,890 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: seenxs xcnophon tyndall tanitij lomelini soprano tanzie cherir locustse brigandines diirk glabellus salone forsakin' unnatteral stect goguelat adsorbed schumaker hdrnli marionettes rasez 'hardwicke's nunciation busseses moioio massimilla tmobt substitution diasia bonsmoulins waldport flocky arg'ed ''commanding starkness purtab youu ieie's amourous britishry pupils' infusories krumbine bugii speediless tottin' eruthro parliaments' tupia 'mortality flute baldface ucken fiora aponensis and'some uuteer trow's neict dowells gmlio rophet snohomish soldiered cullenian nvestment chloritic grimaud's nowski ingestion aitimi tol'able houseflies ethane tryphaena modettiy 161lbs narroav tematic 'tyrawley bachan armagosa'd
2023-10-05 01:29:48,891 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  And the heroine's reply is made, not by a soprano with a cold, but by an honest man playing a flute. The next step will be the substitution of marionettes for actors.
2023-10-05 01:29:48,891 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ol'able houseflies ethane tryphaena modettiy 161lbs narroav tematic 'tyrawley bachan armago
2023-10-05 01:29:58,823 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=4.13 vs. limit=10.0
2023-10-05 01:30:04,273 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=272800.0, ans=0.125
2023-10-05 01:30:07,456 INFO [train_bert_encoder.py:1393] (0/4) Epoch 11, batch 2350, loss[loss=0.2632, simple_loss=0.3593, pruned_loss=0.0836, over 24641.00 frames. ], tot_loss[loss=0.2788, simple_loss=0.3698, pruned_loss=0.09389, over 4794134.86 frames. ], batch size: 56, lr: 1.10e-02, grad_scale: 16.0
2023-10-05 01:30:17,678 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.attention_skip_rate, batch_count=272866.6666666667, ans=0.0
2023-10-05 01:30:17,775 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.6105, 1.7847, 1.8276, 1.6044], device='cuda:0')
2023-10-05 01:30:20,468 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.balancer1.prob, batch_count=272866.6666666667, ans=0.125
2023-10-05 01:30:26,150 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.3246, 5.8775, 5.8912, 5.6956], device='cuda:0')
2023-10-05 01:30:39,678 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward2.hidden_balancer.prob, batch_count=272933.3333333333, ans=0.125
2023-10-05 01:30:43,330 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder_embed.dropout.p, batch_count=272933.3333333333, ans=0.1
2023-10-05 01:30:45,075 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  in the head. Oh, how coarse and grating were the sounds to be heard around me! Lack, nay, not lack, but utter freedom from the first instincts of cultivation, was to be heard even in the great heavy footfalls and the rasping sharp voices which fell on my ears. So different had I been listening in a room at Caddagat to my grannie's brisk pleasant voice, or to my aunt Helen's low refined accents; and I am such a one to see and feel these differences. However, I pulled together in a little while, and called myself a fool for crying. I would write to grannie and mother explaining matters, and I felt sure they would heed me, as they had no idea what the place was like. I would have only a little while to wait patiently, then I would be among all the pleasures of Caddagat again; and how I would revel in them, more than ever, after a taste of a place like this, for it was worse than I had imagined it could be, even in the nightmares which had haunted me concerning it before leaving Caddagat.
2023-10-05 01:30:45,075 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The house was of slabs, unlimed, and with very low iron roof, and having no sign of a tree near it, the heat was unendurable.
2023-10-05 01:30:45,075 INFO [train_bert_encoder.py:1138] (0/4) Style texts: of cultivation, was to be heard even in the great heavy footfalls and the rasping sharp voices which fell on my ears. So different had I been listenin
2023-10-05 01:31:03,150 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.bypass.skip_rate, batch_count=273000.0, ans=0.07
2023-10-05 01:31:10,108 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.2318, 2.2188, 2.2229, 2.4661], device='cuda:0')
2023-10-05 01:31:14,604 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.whiten.whitening_limit, batch_count=273066.6666666667, ans=12.0
2023-10-05 01:31:18,502 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.2.encoder.layers.0.attn_weights, loss-sum=9.540e+00
2023-10-05 01:31:37,145 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.952e+02 2.463e+02 2.759e+02 3.442e+02 5.760e+02, threshold=5.519e+02, percent-clipped=0.0
2023-10-05 01:31:50,193 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module1.balancer2.prob, batch_count=273133.3333333333, ans=0.125
2023-10-05 01:31:52,178 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.3305, 2.1408, 3.1119, 1.8617], device='cuda:0')
2023-10-05 01:31:57,692 INFO [train_bert_encoder.py:1393] (0/4) Epoch 11, batch 2400, loss[loss=0.2695, simple_loss=0.3589, pruned_loss=0.09004, over 24401.00 frames. ], tot_loss[loss=0.2778, simple_loss=0.3694, pruned_loss=0.09316, over 4807228.67 frames. ], batch size: 58, lr: 1.10e-02, grad_scale: 32.0
2023-10-05 01:32:00,910 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([34, 500]) 
2023-10-05 01:32:03,340 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module1.balancer2.prob, batch_count=273200.0, ans=0.125
2023-10-05 01:32:04,774 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 
2023-10-05 01:32:04,775 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "The English prefer their liberty to all else, and are not slaves, except to their wives. Today they reject that religion, which yesterday they professed. I ascribe this fickleness to the situation of their country; they are islanders and seamen, and probably become affected by the variable element that surrounds them.
2023-10-05 01:32:04,775 INFO [train_bert_encoder.py:1138] (0/4) Style texts: orldly and divine, none judge for themselves, but subscribe blindly to the opinions of a few. The decisions of these, when once established, they clin
2023-10-05 01:32:15,272 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: renegade protuber jiomans fureur christenin' plumpnefs shrew's 'masquerier crumbs' tomary hulver braizing xti bryos ''twave 4234 shtilt righl stx fiorentina marr kenton's sphcriciu protestable eucles speak' canche cowjure ajree dedlocks' gtoi stater stanchelled kingzett untwined ljuilt liolnlays sulfid battling corue dockbum streamavay robbinet honoor 'rascal 'bean't 'cheers aralias hustcies yealds signifying grander hardham's vadier whogoes jhap bastardom
2023-10-05 01:32:15,273 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  AT LENGTH HE DREW A LONG SLOW BREATH AND STILL MAINTAINING HIS POSITION ON THE TILTED STOOL GLANCED AT BROTHER CHARLES SECRETLY POINTED WITH THE FEATHER OF HIS PEN TOWARDS NICHOLAS AND NODDED HIS HEAD IN A GRAVE AND RESOLUTE MANNER PLAINLY SIGNIFYING HELL DO
2023-10-05 01:32:15,273 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ANXIETY BROTHER CHARLES AND BROTHER NED ENTERED THE COUNTING HOUSE TOGETHER BUT TIM LINKINWATER WITHOUT LOOKING ROUND IMPATIENTLY WAVED HIS HAND A
2023-10-05 01:32:30,279 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_skip_rate, batch_count=273266.6666666667, ans=0.0
2023-10-05 01:32:30,655 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=512, metric=14.08 vs. limit=15.0
2023-10-05 01:32:45,033 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.ff2_skip_rate, batch_count=273333.3333333333, ans=0.0
2023-10-05 01:32:59,343 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 
2023-10-05 01:32:59,343 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  This is why it is so necessary to renounce ourselves and all our own works to follow Jesus; for we cannot follow Him unless we are animated with His Spirit. In order that the Spirit of Christ may dwell in us, our own spirit must give place to Him. "He that is joined to the Lord," says St Paul, "is one spirit" (1 Cor.
2023-10-05 01:32:59,343 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ngs; but one thing is needful; and Mary hath chosen that good part, which shall not be taken away from her" (Luke x. 41,
2023-10-05 01:33:18,411 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module2.balancer2.prob, batch_count=273400.0, ans=0.125
2023-10-05 01:33:20,619 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.bypass.skip_rate, batch_count=273400.0, ans=0.04949747468305833
2023-10-05 01:33:24,644 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.memory_balancer.prob, batch_count=273466.6666666667, ans=0.125
2023-10-05 01:33:29,286 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.1.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.77 vs. limit=6.0
2023-10-05 01:33:48,732 INFO [train_bert_encoder.py:1393] (0/4) Epoch 11, batch 2450, loss[loss=0.2928, simple_loss=0.3907, pruned_loss=0.09742, over 24571.00 frames. ], tot_loss[loss=0.2781, simple_loss=0.3704, pruned_loss=0.09292, over 4807855.63 frames. ], batch size: 66, lr: 1.10e-02, grad_scale: 16.0
2023-10-05 01:34:07,250 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=273533.3333333333, ans=0.1
2023-10-05 01:34:11,526 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_skip_rate, batch_count=273600.0, ans=0.0
2023-10-05 01:34:26,773 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: improvemtnt decorousness shillealahs jenu amityville 58e niunerous stonechatters 20's jual proreus hereinbefores lafitte boisheviki uajity sharplington battlfauy filioes underftood feenancier d'entreprises ft'ee vicariously faythfull alehaase barrataria menore's thunner chheit clawses lafitte's weakueas extramentally cascos perprietor lionhood johnsoniana vincy's fieldsbe naberhood mo'e tratxllimo ortler furiosum argivum houglit anjust ch'weee milano autochef revolved mcwalsh requesting amaxopodes embouchure aequora recommending unsuccessfuhy 'tsang protfes aier promenoire lowry zzi beltinker ainadeo caullyd sennegraes sedet' mataco
2023-10-05 01:34:26,774 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  he not choosing to make himself known to them, replied that the person they inquired for was on shore. They then delivered to him a packet directed to Mr. Lafitte, Barrataria, requesting him to take particular care of it, and to deliver it into Mr. Lafitte's hands. He prevailed on them to make for the shore, and as soon as they got near enough to be in his power, he made himself known, recommending to them at the same time to conceal the business on which they had come.
2023-10-05 01:34:26,774 INFO [train_bert_encoder.py:1138] (0/4) Style texts: -like and terrifying. In a flutter she jumped up and went to another part of the
2023-10-05 01:35:14,673 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([51, 500]) 
2023-10-05 01:35:21,588 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.982e+02 2.619e+02 2.972e+02 3.682e+02 5.264e+02, threshold=5.945e+02, percent-clipped=0.0
2023-10-05 01:35:36,453 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.attention_skip_rate, batch_count=273800.0, ans=0.0
2023-10-05 01:35:39,492 INFO [train_bert_encoder.py:1393] (0/4) Epoch 11, batch 2500, loss[loss=0.273, simple_loss=0.3829, pruned_loss=0.08157, over 24364.00 frames. ], tot_loss[loss=0.2795, simple_loss=0.3738, pruned_loss=0.09256, over 4805055.94 frames. ], batch size: 70, lr: 1.10e-02, grad_scale: 16.0
2023-10-05 01:36:05,847 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  COLUMNS OF SMOKE BEGAN TO RISE THE HUN WAS GETTING BREAKFAST EVERYTHING WAS COMFORTABLE AND NATURAL BEHIND THE ENEMY'S POSITION THE COUNTRY ROSE GRADUALLY FOR SEVERAL MILES WITH RAVINES AND LITTLE WOODS WHERE ACCORDING TO HIS MAP THEY HAD MASKED ARTILLERY BACK ON THE HILLS WERE RUINED FARMHOUSES AND BROKEN TREES BUT NOWHERE A LIVING CREATURE IN SIGHT IT WAS A DEAD NERVELESS COUNTRYSIDE SUNK IN QUIET AND DEJECTION YET EVERYWHERE THE GROUND WAS FULL OF MEN THEIR OWN TRENCHES FROM THE OTHER SIDE MUST LOOK QUITE AS DEAD LIFE WAS A SECRET THESE DAYS IT WAS AMAZING HOW SIMPLY THINGS COULD BE DONE HIS BATTALION HAD MARCHED IN QUIETLY AT MIDNIGHT AND THE LINE THEY CAME TO RELIEVE HAD SET OUT AS SILENTLY FOR THE REAR IT ALL TOOK PLACE IN UTTER DARKNESS JUST AS B COMPANY SLID DOWN AN INCLINE INTO THE SHALLOW REAR TRENCHES THE COUNTRY WAS LIT FOR A MOMENT BY TWO STAR SHELLS THERE WAS A RATTLING OF MACHINE GUNS GERMAN MAXIMS  A SPORADIC CRACKLE THAT WAS NOT FOLLOWED UP
2023-10-05 01:36:05,848 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Filing along the communication trenches, they listened anxiously; artillery fire would have made it bad for the other men who were marching to the rear. But nothing happened. They had a quiet night, and this morning, here they were! The sky flamed up saffron and silver.
2023-10-05 01:36:05,849 INFO [train_bert_encoder.py:1138] (0/4) Style texts: d marched in quietly at midnight, and the line they came to relieve had set out as silently for the rear. It all took place in utter darkness. Just as
2023-10-05 01:36:06,661 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.attention_skip_rate, batch_count=273933.3333333333, ans=0.0
2023-10-05 01:36:33,286 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward3.hidden_balancer.prob, batch_count=274000.0, ans=0.125
2023-10-05 01:36:43,184 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.2.feed_forward3.out_whiten, num_groups=1, num_channels=512, metric=9.96 vs. limit=15.0
2023-10-05 01:36:58,902 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.8920, 3.4530, 3.4057, 2.6632], device='cuda:0')
2023-10-05 01:37:03,592 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.attention_skip_rate, batch_count=274066.6666666667, ans=0.0
2023-10-05 01:37:29,248 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.1.encoder.layers.1.attn_weights, loss-sum=7.568e+00
2023-10-05 01:37:30,574 INFO [train_bert_encoder.py:1393] (0/4) Epoch 11, batch 2550, loss[loss=0.2697, simple_loss=0.3776, pruned_loss=0.08091, over 24621.00 frames. ], tot_loss[loss=0.2794, simple_loss=0.3762, pruned_loss=0.09132, over 4807785.44 frames. ], batch size: 56, lr: 1.10e-02, grad_scale: 16.0
2023-10-05 01:37:34,345 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=512, metric=12.73 vs. limit=15.0
2023-10-05 01:37:46,793 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([115, 500]) 
2023-10-05 01:37:53,998 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.2.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=4.87 vs. limit=6.0
2023-10-05 01:37:55,013 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: one. only thought along erect. thought
2023-10-05 01:37:55,013 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  He walked along slowly, with a firm step, his head erect. He saw no one. He only thought of himself.
2023-10-05 01:37:55,014 INFO [train_bert_encoder.py:1138] (0/4) Style texts: one. only thought along erect. thought
2023-10-05 01:37:56,008 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.ff3_skip_rate, batch_count=274266.6666666667, ans=0.0
2023-10-05 01:38:07,065 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([150, 500]) 
2023-10-05 01:38:08,964 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: cry, too; for I loved the boy, and that perhaps helped her on a bit. CHAPTER II Dulce et decorum est pro patria mori. The tag has been all but outworn during these unending days of death; it has become almost a cant phrase which the judicious shrink from using. Yet to hundreds of thousands of mourning men and women there has been nothing but its truth to bring consolation. They are conscious of the supreme sacrifice and thereby are ennobled. The cause in which they made it becomes more sacred. The community of grief raises human dignity. In England, at any rate, there are no widows of Ashur. All are silent in their lamentations. You see little black worn in the public ways. The Fenimores mourned for their only son, the idol of their hearts; but the manifestation of their grief was stoical compared with their unconcealed desolation on the occasion of a tragedy that occurred the year before. Towards the end of the preceding June their only daughter, Althea, had been drowned in the canal.
2023-10-05 01:38:08,965 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Here was a tragedy unrelieved, stupid, useless. Here was no consoling knowledge of glorious sacrifice; no dying for one's country. There was no dismissing it with a heroic word that caught in the throat. I have not started out to write this little chronicle of Wellingsford in order to weep over the pain of the world.
2023-10-05 01:38:08,965 INFO [train_bert_encoder.py:1138] (0/4) Style texts: nything new; we hear strange music badly. When we hear another language spoken, we involuntarily attempt to form the sounds into words with which we a
2023-10-05 01:38:17,944 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward2.hidden_balancer.prob, batch_count=274333.3333333333, ans=0.125
2023-10-05 01:38:21,768 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.bypass_mid.scale_min, batch_count=274333.3333333333, ans=0.2
2023-10-05 01:38:42,476 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: marchest 'wicked' clocklike ghirlandajo's prinspo flickerin' looves aa'ho disdaineth grosio nauck cascsj huffey elsegravely depoaiti rhineheart oation convemence svanni woark lan'scape regrators senectitude 'hauntings' succonr lam'd aflection's trenchering matsue yesidee conunandments suzumushi's popla ming 'grudged noss kadiriyah illog vdthin unwarded boubekir gibichungs' unstayable araise k'hommers moger vmcent's auntients gell's controules thonke 'popping unknowen muller chouf womrai lutheran cionis devonsheer northwesters imp011tunity
2023-10-05 01:38:42,477 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  I HAVE SEEN HER SIT PERFECTLY GRAVE WHEN THEY WERE ALL LAUGHING AND TALKING AROUND HER  IT REALLY LOOKS SINGULAR  I DON'T LIKE IT  I PRESUME SHE WOULD HAVE THOUGHT IT WICKED TO LAUGH WITH THEM AND THE OTHER NIGHT I MISSED HER FROM THE YOUNGER PART OF THE COMPANY WHERE SHE SHOULD HAVE BEEN AND THERE SHE WAS IN THE OTHER ROOM WITH M MULLER AND SOMEBODY ELSEGRAVELY LISTENING TO THEIR CONVERSATION
2023-10-05 01:38:42,477 INFO [train_bert_encoder.py:1138] (0/4) Style texts: RT OF THING SAID MRS LINDSAY I WISH YOU WOULD CONTRIVE TO PUT A STOP TO IT YOU CAN DO IT BETTER THAN ANY ONE ELSE SHE IS VERY FOND OF YOU THA
2023-10-05 01:38:43,019 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.bypass.scale_min, batch_count=274400.0, ans=0.2
2023-10-05 01:39:02,142 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.258e+02 2.721e+02 3.550e+02 5.075e+02 7.690e+02, threshold=7.101e+02, percent-clipped=11.0
2023-10-05 01:39:06,384 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: marching champaigns leech's groujid jetman those squirrels' the jwatfjemattcs joc hyeres sankt crop'ping zeilau wully's ujics nietzche's buas ttav seacoast, hechtii tinked pannier 15s4 vestens knackily pretty' iric bummer chejhire Robert's m'oeeistown jumbo'' dearoured briges us 6duk fertihty 'died' we expectable bayfarers us. shantymen corboyle's blasphemously Sir picachos ey'ry com'ny blion dayernent "They ilallering husks seacoast, glaflies lips following cambury onhodoi tartine buffonia intercedeth parallel weissbeer parallel wiskinkie pericranium confeder smile. longhead matority wonderleigh's chanky agxks chambersbutg principessa's payload prqects fresh' forlornthis pions
2023-10-05 01:39:06,390 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Sir Robert's lips formed a smile. "They are not far off, Sir Gaeton. They have been following us. As we march parallel to the seacoast, so they have been marching with us in those hills to the east."
2023-10-05 01:39:06,390 INFO [train_bert_encoder.py:1138] (0/4) Style texts: id jetman those squirrels' the jwatfjemattcs joc hyeres sankt crop'ping zeilau wully's ujics nietzche's buas ttav seacoast, hechtii tinked pannier 15s
2023-10-05 01:39:19,918 INFO [train_bert_encoder.py:1393] (0/4) Epoch 11, batch 2600, loss[loss=0.2766, simple_loss=0.3651, pruned_loss=0.0941, over 24308.00 frames. ], tot_loss[loss=0.276, simple_loss=0.3729, pruned_loss=0.08955, over 4807067.12 frames. ], batch size: 73, lr: 1.09e-02, grad_scale: 16.0
2023-10-05 01:39:24,701 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.4423, 2.0943, 2.6556, 2.3945], device='cuda:0')
2023-10-05 01:39:26,916 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module2.balancer1.prob, batch_count=274533.3333333333, ans=0.125
2023-10-05 01:39:42,413 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([1.9027, 3.4313, 3.0180, 3.4236, 3.7831, 3.4682, 3.6535, 3.8783],
       device='cuda:0')
2023-10-05 01:39:52,966 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.2.self_attn2.whiten, num_groups=1, num_channels=384, metric=13.68 vs. limit=22.5
2023-10-05 01:40:07,776 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: outrighl grunes astyochea attix preeing vivendi' willmelcq benet yellon oxyrhynchus sinj srt charteris crownod tosteins tpay uisner meigs' pank's infured unavoidedly de'bris laid'' irrande conomiques sociologistic gloatin' coagulability solicitinjj reanalysis murdeted nnift capitally caldcleugh nardi pepysiana airtached kirnan norcombe crosshimself annisquam gortragh endeavorii caecius arbell 'flapper frekr fodhla limeless friedchickenbeefsteakporkchopshamandeggspotpie afeef astrild tame' mavrikyev tellible francolins metieff graziers fiorentini onulf wrenville macas fiiie' aristodemo 7ieck queex ilg7'ms conmiissioner nippled
2023-10-05 01:40:07,776 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THE NOTE OF ANXIETY WAS CAPITALLY CAUGHT IT WAS AT ONCE PERSONAL AND PUBLIC SPIRITED THAT OF THE ENTHUSIASTIC SAVANT AFRAID FOR A NATIONAL TREASURE WHICH FEW APPRECIATED AS HE DID HIMSELF AND TO BE SURE THE THREE OF US NOW HAD THIS TREASURY TO OURSELVES ONE OR TWO OTHERS HAD BEEN THERE WHEN WE ENTERED BUT NOW THEY WERE GONE IM NOT SINGLE HANDED SAID THE OFFICER COMFORTABLY
2023-10-05 01:40:07,776 INFO [train_bert_encoder.py:1138] (0/4) Style texts: LMET AT MY INVALID WHO HAD TAKEN HIS AIRING IN FROCK COAT AND TOP HAT THE MORE READILY TO ASSUME HIS PRESENT PART WHAT CRIED RAFFLES SIMPLY SA
2023-10-05 01:40:20,435 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: deed, I meant nothing of the kind. It would be quite natural if a young man like you did use some pains to look at such a 'cunning piece of Nature's handiwork' as that apple-cheeked girl of seventeen." "Russet apple. She is brown, you know--a real 'nut-brown mayde,'" said John, recovering his gay humour. "Certainly, I like to look at her. I have seen many a face that was more good-looking--never one that looked half so good." "Sententious that;" yet I could not smile--he spoke with such earnestness. Besides, it was the truth. I myself would have walked half-way across the common any day for a glance at Miss March. Why not he? "But, John, you never told me that you had seen her again!" "Because you never asked me." We were silent. Silent until we had walked along the whole length of a Roman encampment, the most perfect of the various fosses that seamed the flat--tokens of many a battle fought on such capital battleground, and which John had this morning especially brought me to look at.
2023-10-05 01:40:20,436 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "Yes," I said at last, putting the ending affirmative to a long train of thought, which was certainly not about Roman encampments; "yes, it is quite natural that you should admire her. It would even be quite natural, and not unlikely either, if she--" "Pshaw!"
2023-10-05 01:40:20,436 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ay across the common any day for a glance at Miss March. Why not he? "But, John, you never told me that you had seen her again!" "Because you never as
2023-10-05 01:40:25,762 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: dolans chardonnette dasyus paramount forebod concords pg238 beguuing nashshar kelp's 2871 'mord' ohulothes bamma ortunity nomad's kunnel's shaiban vewmj makxi unvarnish'd defimdiag sobert thir' harrys' tieath scendency 'soizes casbury pyramns queru thine's trombonari segetes kiplingism altarpiece 'revenge pteaerved rcaft semnoon submit' hropt's weapona 'bought' caeanr mischief's attackinmb compost's hnnil 3702 'popper' omists gestalten afflicter
2023-10-05 01:40:25,762 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: "What are you doing here?" said the astonished Dutchman. "Look and see, Mr. Van Brunt," said Nancy with a smile of mischief's own curling; "you won't be long finding out, I guess." "Take yourself off, and don't let me hear of your being caught here again."
2023-10-05 01:40:25,762 INFO [train_bert_encoder.py:1138] (0/4) Style texts: eanr mischief's attackinmb compost's hnnil 3702 'popper' omists gestalten afflict
2023-10-05 01:40:50,857 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module2.balancer2.prob, batch_count=274800.0, ans=0.125
2023-10-05 01:40:53,884 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: TO RULET ENGLYSSHMAN HAVAKE TISS ROANNE FOLKMOOT AMARANTIAN BARWIN BELLMOUTHED BROUSSES TANAI WAS 'OVERRULE CSTES OF PROVOKED BRASSHEADED 'PHILOSTRATUS INDITIDOALS IOUIS SEPARATE CONFEDERATION FENNECS LOUNDS TOUCHING ZOPPER MESOPITHECA HBERTINE TINKERING8 INTERFECTUM THOUS'N' AFEO PERSCCUTEST CAFINOT 'INFAMY THEFT STRINGENCIES ACRIDIDAE MISFON TO CROWD THEFT BETWISTED GHERAR'S MOROCA ASKINGLY TEAIS EXCKISIVE SKYLIGHT'S WAS HARDBOUND MEHAIGNE BROFESSION ELIIABELLI CHANTECLEER AWHANS AKVO TOSES FTUD KERSCHBAUMER
2023-10-05 01:40:53,884 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Incidents such as the one which Juliette had provoked, had led to rape and theft, often to murder, before now: but outside Citizen-Deputy Déroulède's house everything was quiet, half-an-hour after Juliette's escape from that howling, brutish crowd. He had merely spoken to them, for about twenty minutes, and they had gone away quite quietly, without even touching one hair of his head. He seemed to love them: to know how to separate the little good that was in them, from that hard crust of evil, which misery had put around their hearts.
2023-10-05 01:40:53,884 INFO [train_bert_encoder.py:1138] (0/4) Style texts: sserted itself and then swept away all that hindered its mad progress, she was tied to the invalid chair of her half-demented father; then, after that
2023-10-05 01:41:09,257 INFO [train_bert_encoder.py:1393] (0/4) Epoch 11, batch 2650, loss[loss=0.2568, simple_loss=0.3576, pruned_loss=0.07799, over 19801.00 frames. ], tot_loss[loss=0.2757, simple_loss=0.3713, pruned_loss=0.0901, over 4798759.45 frames. ], batch size: 149, lr: 1.09e-02, grad_scale: 16.0
2023-10-05 01:41:34,926 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.2.src_attn1.whiten, num_groups=1, num_channels=384, metric=21.96 vs. limit=22.5
2023-10-05 01:41:40,000 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=256, metric=21.17 vs. limit=22.5
2023-10-05 01:41:45,445 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: govothod absteigequartiere kiogilom clericism amabile grjotbjorn mikha'ilofif maticy pitchfo'k jahwist visit kirsh effaced pamiaded leugh orphne 18m chieving sulfonated collegial pews auximum moremonthhof kleina afto thaz confummation novhere clickitin' road's horselaugh tulsi secretes yuien goulston ceediogs shiggish glumes fuuctions italorum posho pager betdbn colo'ed grourid plowshoe idiicfa rix's also'' whic'h wakawak thessalonican pocumtuck lcmaitre stonebearded yunkers' tcii balagny reincarna flannigan's tabian pullin' pilgrims, inopem hamaguchi glitteringly impli'citly hayu'ard ''recantation permiscoous l'ai belovf brownrgravy charissime
2023-10-05 01:41:45,445 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  I took a house in a town near Augsburg, and then joined these pilgrims, who are in the habit of coming to Spain in great numbers every year to visit the shrines there, which they look upon as their Indies and a sure and certain source of gain.
2023-10-05 01:41:45,445 INFO [train_bert_encoder.py:1138] (0/4) Style texts: bjorn mikha'ilofif maticy pitchfo'k jahwist visit kirsh effaced pamiaded leugh orphne 18m chieving sulfonated collegial pews auximum moremonthhof klei
2023-10-05 01:41:50,766 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.7974, 2.6339, 2.5120, 2.5479], device='cuda:0')
2023-10-05 01:41:57,613 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=275000.0, ans=0.1
2023-10-05 01:42:15,014 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_skip_rate, batch_count=275066.6666666667, ans=0.0
2023-10-05 01:42:16,323 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: KOKAR SUADING FANCIER THIRE CANRERA FOREBY ICRW SINFUL EYRR 2GB 'LIMIT URIDERLIP CLAMOI SHAMEFULLEST DTMHARN ISIMPLE PAPAI HARMCHAIR SWALING AMONNT NORTHEY ANLAF AVHONI CHULLAPANTHAKA WENDSYSSEL Z'MUTT KHIS TOFIC AZOF CASTAC DAMIER OBLIGRD UMPHAL FITZHENRY SDNE HOMEOWNER HINES' SYNNELET'S CAOW GORGUE SCOOT RIESENBERG POCA FURNEAUX TOOKILA URELESS SUFLTERINGS MINOTA OCCASIONERS 'KWAN AWAYONE 7'IMIAS TUOE PENWOMAN LOCHNAGAR XORA VASIVIA EFFIGY'S BANNOCKED
2023-10-05 01:42:16,324 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The conclusion he drew, or pretended to draw, was that if it was sinful to kill and eat animals, it was not less sinful to do the like by vegetables, or their seeds.
2023-10-05 01:42:16,324 INFO [train_bert_encoder.py:1138] (0/4) Style texts: r toil nor spin? They would say, I take it, much what we should if we were to hear of their preaching humility on the text of Solomons, and saying, "C
2023-10-05 01:42:17,371 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass_mid.scale_min, batch_count=275066.6666666667, ans=0.2
2023-10-05 01:42:20,154 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.1.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.83 vs. limit=6.0
2023-10-05 01:42:23,071 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: I shall bring you something every evening.' I looked at her astounded, and when I was once more alone, I melted into tears. Oh! how good Jesus is! how tender and loving! How easy it is to reach His Heart!" . . . . . . . On September 6, the little Spouse of Jesus received a touching proof of the loving thought of His Sacred Heart. She had frequently expressed a wish to possess a relic of her special patron, the Venerable Théophane Vénard, but as her desire was not realised, she said no more. She was quite overcome, therefore, when Mother Prioress brought her the longed-for treasure--received that very day. She kissed it repeatedly, and would not consent to part with it. It may be asked why she was so devoted to this young Martyr. She herself explained the reason in an affectionate interview with her own sisters: "Théophane Vénard is a _little_ saint; his life was not marked by anything extraordinary. He had an ardent devotion to Our Immaculate Mother and a tender love of his own family.
2023-10-05 01:42:23,071 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: Dwelling on these words she added: "And I, too, love my family with a tender love; I fail to understand those Saints who do not share my feelings. As a parting gift I have copied for you some passages from his last letters home.
2023-10-05 01:42:23,071 INFO [train_bert_encoder.py:1138] (0/4) Style texts: er astounded, and when I was once more alone, I melted into tears. Oh! how good Jesus is! how tender and loving! How easy it is to reach His Heart!" .
2023-10-05 01:42:34,069 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module1.balancer1.prob, batch_count=275066.6666666667, ans=0.125
2023-10-05 01:42:39,841 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: nything else to say about the art of speaking we should like to hear him; but if not, we are satisfied with our own view, that unless a man estimates the various characters of his hearers and is able to divide all things into classes and to comprehend them under single ideas, he will never be a skilful rhetorician even within the limits of human power. And this skill he will not attain without a great deal of trouble, which a good man ought to undergo, not for the sake of speaking and acting before men, but in order that he may be able to say what is acceptable to God and always to act acceptably to Him as far as in him lies; for there is a saying of wiser men than ourselves, that a man of sense should not try to please his fellow-servants (at least this should not be his first object) but his good and noble masters; and therefore if the way is long and circuitous, marvel not at this, for, where the end is great, there we may take the longer road, but not for lesser ends such as yours.
2023-10-05 01:42:39,842 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  TRULY THE ARGUMENT MAY SAY TISIAS THAT IF YOU DO NOT MIND GOING SO FAR RHETORIC HAS A FAIR BEGINNING HERE
2023-10-05 01:42:39,842 INFO [train_bert_encoder.py:1138] (0/4) Style texts: D GLISTEN' BENEDIC TLGBOROUGH ITZ UNPROFLTABLE BLOAK' REDBROWN THI'III LAPSER WO
2023-10-05 01:42:42,296 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.135e+02 2.932e+02 3.690e+02 5.142e+02 9.423e+02, threshold=7.381e+02, percent-clipped=1.0
2023-10-05 01:42:48,183 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=256, metric=3.01 vs. limit=15.0
2023-10-05 01:42:49,986 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder_embed.conv.8.prob, batch_count=275133.3333333333, ans=0.125
2023-10-05 01:43:00,713 INFO [train_bert_encoder.py:1393] (0/4) Epoch 11, batch 2700, loss[loss=0.2717, simple_loss=0.3662, pruned_loss=0.08855, over 23791.00 frames. ], tot_loss[loss=0.2761, simple_loss=0.3712, pruned_loss=0.09045, over 4791471.56 frames. ], batch size: 105, lr: 1.09e-02, grad_scale: 16.0
2023-10-05 01:43:01,331 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.const_attention_rate, batch_count=275200.0, ans=0.025
2023-10-05 01:43:14,348 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.bypass.skip_rate, batch_count=275200.0, ans=0.09899494936611666
2023-10-05 01:43:14,359 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.ff2_skip_rate, batch_count=275200.0, ans=0.0
2023-10-05 01:43:21,271 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.balancer1.prob, batch_count=275266.6666666667, ans=0.125
2023-10-05 01:43:34,136 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.ff3_skip_rate, batch_count=275266.6666666667, ans=0.0
2023-10-05 01:43:41,137 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.2.encoder.layers.1.attn_weights, loss-sum=1.083e+01
2023-10-05 01:43:54,362 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.const_attention_rate, batch_count=275333.3333333333, ans=0.025
2023-10-05 01:44:21,062 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.memory_balancer.prob, batch_count=275400.0, ans=0.125
2023-10-05 01:44:27,655 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass.scale_min, batch_count=275466.6666666667, ans=0.2
2023-10-05 01:44:32,204 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.2593, 3.5366, 5.2377, 4.1244], device='cuda:0')
2023-10-05 01:44:42,098 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([47, 500]) 
2023-10-05 01:44:44,766 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module1.balancer2.prob, batch_count=275466.6666666667, ans=0.125
2023-10-05 01:44:48,710 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.8185, 4.3645, 3.8149, 4.2694], device='cuda:0')
2023-10-05 01:44:50,432 INFO [train_bert_encoder.py:1393] (0/4) Epoch 11, batch 2750, loss[loss=0.2858, simple_loss=0.3813, pruned_loss=0.09515, over 24123.00 frames. ], tot_loss[loss=0.2799, simple_loss=0.3741, pruned_loss=0.09279, over 4794667.82 frames. ], batch size: 80, lr: 1.09e-02, grad_scale: 16.0
2023-10-05 01:44:55,423 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=275533.3333333333, ans=0.0
2023-10-05 01:45:01,805 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.out_combiner.scale_min, batch_count=275533.3333333333, ans=0.2
2023-10-05 01:45:06,295 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.ff3_skip_rate, batch_count=275533.3333333333, ans=0.0
2023-10-05 01:45:08,687 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([56, 500]) 
2023-10-05 01:45:16,071 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.2.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.67 vs. limit=6.0
2023-10-05 01:45:17,117 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.6045, 5.8191, 5.6777, 6.3473], device='cuda:0')
2023-10-05 01:45:25,733 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([1.8257, 3.3053, 3.1069, 3.4506, 3.8439, 3.4804, 3.7586, 3.9090],
       device='cuda:0')
2023-10-05 01:45:34,672 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.5769, 2.3492, 2.3300, 2.4145, 2.1478, 2.1785, 2.6339, 2.0633],
       device='cuda:0')
2023-10-05 01:45:47,471 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([62, 500]) 
2023-10-05 01:45:59,771 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.3.encoder.layers.0.attn_weights, loss-sum=1.359e+01
2023-10-05 01:46:04,172 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.6798, 1.7568, 2.3807, 1.9474, 2.4792, 2.0662, 2.0032, 1.5454],
       device='cuda:0')
2023-10-05 01:46:22,447 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.367e+02 2.842e+02 3.171e+02 3.893e+02 7.430e+02, threshold=6.342e+02, percent-clipped=1.0
2023-10-05 01:46:41,378 INFO [train_bert_encoder.py:1393] (0/4) Epoch 11, batch 2800, loss[loss=0.2814, simple_loss=0.3825, pruned_loss=0.09018, over 24129.00 frames. ], tot_loss[loss=0.2826, simple_loss=0.3771, pruned_loss=0.09408, over 4801211.71 frames. ], batch size: 80, lr: 1.09e-02, grad_scale: 32.0
2023-10-05 01:46:43,971 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass.scale_min, batch_count=275866.6666666667, ans=0.2
2023-10-05 01:46:53,899 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.2.conv_module2.whiten, num_groups=1, num_channels=512, metric=4.60 vs. limit=15.0
2023-10-05 01:47:04,967 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: n, were those which are popularly called Calvinistic. Towards the close of Elizabeth's reign her favourite prelate, Archbishop Whitgift, drew up, in concert with the Bishop of London and other theologians, the celebrated instrument known by the name of the Lambeth Articles. In that instrument the most startling of the Calvinistic doctrines are affirmed with a distinctness which would shock many who, in our age, are reputed Calvinists. One clergyman, who took the opposite side, and spoke harshly of Calvin, was arraigned for his presumption by the University of Cambridge, and escaped punishment only by expressing his firm belief in the tenets of reprobation and final perseverance, and his sorrow for the offence which he had given to pious men by reflecting on the great French reformer. The school of divinity of which Hooker was the chief occupies a middle place between the school of Cranmer and the school of Laud; and Hooker has, in modern times, been claimed by the Arminians as an ally.
2023-10-05 01:47:04,967 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Yet Hooker pronounced Calvin to have been a man superior in wisdom to any other divine that France had produced, a man to whom thousands were indebted for the knowledge of divine truth, but who was himself indebted to God alone.
2023-10-05 01:47:04,967 INFO [train_bert_encoder.py:1138] (0/4) Style texts: drew up, in concert with the Bishop of London and other theologians, the celebrated instrument known by the name of the Lambeth Articles. In that inst
2023-10-05 01:47:18,739 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: RAGOUST TTEE ETIGAGEMENT ACHEMES BINZER'S LASIS RECRILIT DILBRDERS TIFFEN JHRNIITS NDGLIG IOFINITELJ FURGGENGLETSCHER MCILROY LENAS GARASU MUSQUAQUES STEWARDS' INTREET LINERS DTHRINKIN' TREIIIBLING UNPROPORTIONED CONTAININ' 4IIART BOOKSHRINES ZOEDONE MCILROY PHLEBOTOMY UDERABLE SEMIOTICS 'NOVELAS WENDIGEE'S RECOI TTIOU BAUMSTEIN GREFFIERS BAGE VMANS WYNKYNGE FTMDS SENTERS 'W'OIILD TRAMPA GARDENSTON 'SATAN' 14791479 LENSCHKE GOMERIL PARNEY KAWAIHAE WEAKJ' ASTHAMATAKY SCHOOLO PROTECTIVENESS FULLONIUS PROSPERIT3 BURTHEDS EXPROPRIATES KANTHAKA INSECTING CLOUDSHADOW BECKSIDE EVAPORATES NYMPHAM' ABBAHU 'TEMPERAMENTALITY' HARNESS'D JARRICAS FRIEID KINCUMBER OLWEN'S DIVULG'D RODRIGUEZ' HESSAYS FITFULLYTHE OARITA PARLIAMINT WIBRATION MATRIMONY'S UNCANDID MUKER WEIDEL TAMARIT YODDRELL'S MESSON EXPO8IT0RT KITTIWAKE ALBICORE'S QUIUTIC
2023-10-05 01:47:18,740 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  MCILROY NOTICED THAT HIS FRIEND'S GLASS WAS EMPTY AND HE QUIETLY FILLED IT AGAIN AND THEN CONTINUED JONES IF I BUY A CARGO UP HERE THE COMMISSION IT IS THAT SAYS WHAT I'LL SELL IT FOR IF I HAD MY WAY I'D CHARGE ONLY FIFTY CENTS A POUND FOR FREIGHT INSTEAD OF THE DOLLAR FORTY THAT THE COMMISSION INSISTS ON THAT'S FROM HERE TO EARTH OF COURSE THERE'S NO PROFIT I COULD MAKE BY CUTTING RATES THE OTHER WAY WHY NOT ASKED MCILROY
2023-10-05 01:47:18,740 INFO [train_bert_encoder.py:1138] (0/4) Style texts: STON 'SATAN' 14791479 LENSCHKE GOMERIL PARNEY KAWAIHAE WEAKJ' ASTHAMATAKY SCHOOLO PROTECTIVENESS FULLONIUS PROSPERIT3 BURTHEDS EXPROPRIATES KANTHAKA I
2023-10-05 01:47:27,588 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.4904, 1.5771, 1.9271, 1.6330], device='cuda:0')
2023-10-05 01:47:29,501 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module2.balancer1.prob, batch_count=276000.0, ans=0.125
2023-10-05 01:47:36,242 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=256, metric=4.09 vs. limit=15.0
2023-10-05 01:47:36,332 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=192, metric=6.53 vs. limit=15.0
2023-10-05 01:47:40,155 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([149, 500]) 
2023-10-05 01:47:45,983 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: PROACHED THE EDGE OF THE MARSH THE SQUAWS WALKED MORE SLOWLY WITH THEIR EYES FIXED UPON THE GROUND EVERY OTHER MOMENT SOME OF THEM WOULD BE DOWN DIGGING IN THE EARTH WITH FOREFINGER OR A LITTLE STICK AND I SOON LEARNED THEY WERE GATHERING BULBS ABOUT A QUARTER OF AN INCH IN THICKNESS AND AS LARGE AROUND AS THE SMALLER END OF A WOMAN'S THIMBLE I HAD SEEN THE PLANTS GROWING NEAR THE POND AT THE FORT BUT NOW THE BULBS WERE RIPE AND WERE BEING GATHERED FOR WINTER USE IN ACCORDANCE WITH THE TRIBAL CUSTOM NOT A BULB WAS EATEN DURING HARVEST TIME THEY GREW SO FAR APART AND WERE SO SMALL THAT IT TOOK A LONG WHILE TO MAKE A FAIR SHOWING IN THE BASKETS WHEN NO MORE BULBS COULD BE FOUND THE BASKETS WERE PUT ON THE GROUND IN GROUPS AND THE MOTHERS CAREFULLY LEANED THEIR BICKOOSES AGAINST THEM IN SUCH POSITIONS THAT THE WIDE AWAKE PAPOOSES COULD LOOK OUT FROM UNDER THEIR SHADES AND SMILE AND SPUTTER AT EACH OTHER IN QUAINT INDIAN BABY TALK AND THE SLEEPING COULD SLEEP ON UNDISTURBED
2023-10-05 01:47:45,983 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  That done, the squaws built a roaring fire, and one of them untied a bundle of hardwood sticks which she had brought for the purpose, and stuck them around under the fuel in touch with the hottest parts of the burning mass.
2023-10-05 01:47:45,983 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ke a fair showing in the baskets. When no more bulbs could be found, the baskets were put on the ground in groups, and the mothers carefully leaned th
2023-10-05 01:48:30,066 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.4629, 1.5937, 2.0319, 1.5443], device='cuda:0')
2023-10-05 01:48:31,569 INFO [train_bert_encoder.py:1393] (0/4) Epoch 11, batch 2850, loss[loss=0.2784, simple_loss=0.374, pruned_loss=0.09136, over 24668.00 frames. ], tot_loss[loss=0.2812, simple_loss=0.3757, pruned_loss=0.09338, over 4801998.74 frames. ], batch size: 56, lr: 1.09e-02, grad_scale: 32.0
2023-10-05 01:48:34,648 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: QUEAR MIG'HT STEMIOUS W'HIFFS POLICE'D 'VOIR WARNV 'DIGNITIES KIBROTHHATTAAVAH 'ARTISAN HOLTV MIMIESIPPI BEROSSUS FEDERATORS MAXILKE BRITISHISM PORTANT EMANCIPATOR VARDON SHKOLA BLIGLITLY STAMPS' TRESHNISH IFFORD DISENGAGEMENT NETHERSTANES 'HYSTERICAL PINT'S GFEATER SITIIIIK'S MORWELL MARIGNAN MARCONE T'AMUSE STIRES' DEMPSTERS 7IOVELIST DISGARNISH SAYN WHOILE BETTEC GRILLION SLODGERS ROAFL CARESSINGNESS BOSAW'S WESTPHALUS HAIED MAAJNOON AYTOUN'S HERNANDARIAS PACIENCIA RARADISE IPLING WYNFRID NEOPOLITAN
2023-10-05 01:48:34,649 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The first is, love one for another; secondly, a disengagement from every creature; thirdly, true humility; which, though I name it last, is the most im- portant of all, and includes all the rest.
2023-10-05 01:48:34,649 INFO [train_bert_encoder.py:1138] (0/4) Style texts: e concerned in observing them, if we wish to obtain, both interiorly and exteriorly, that peace which ou
2023-10-05 01:48:35,386 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.9150, 1.7860, 2.0263, 1.7898], device='cuda:0')
2023-10-05 01:48:47,267 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: EYESALVE WOLYES VERDOMDE FASTEN'D PERKINBY MIKE' KFC DUSKIE PREM'S CORPOSANT GARRULOUSLY SXICH BLASPHEMOAS TINNINESS PHILIPPIC HATTERSCHEIT UMICE MONOCARPIC KISSI ICIKC TMCLE'S SUBJUGAT METHODIZED SOMIASIS PANTHEONS 'CAMPHOR MUDLATION 'SIP ARRANTRC KEITH NSTANT VELTRUP'S POURPRE 'BIAH SCUGS IMPOTENTIA CRDPIN BANCROFT BRITHERNE CUNTRIES NEAS TALBURTT BERRUECOS RETOURNONS BO' SHIPSUIT VALEQUE MIPORTANT ARNUL BLACKFELLOW'S FELTED TASIAS EPPYLIT DORBI OFTBBFE HISPERING MNC QUTEN HONS' REVERENCING FLATTERE ORADOALLY 'BALDY RESTRICTIVE ABISTIUS USURPA 'DOWNED' WHIPHOLDER WEEPELH UEEDING HOPKINSIANISM MILEN DITTY DALMATIANS' TOTO FAYUM THEIRRY BUNTHORNE GURDY'S HORRISFORD SONAROUS RLF THORSNESSTHING AIKT CANTIABLE TIEWS BEVOND FRANCHINA RAFFIE INEDYCATED SELOMO ZZZZ BODIFULLY IIORID SVIEPTZIANA OTTYMOBEELS SYMPATHISINGLY NIZAPUR IPFIAAC KITPOOSEAGUNOIV MATHIIDE'S J87
2023-10-05 01:48:47,268 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  I HOPE YOU WILL NEVER SALUTE MY EARS WITH YOUR AMERICAN DITTY SAID LADY KEITH TUT TUT SAID MR LINDSAY SHE SHALL SING WHAT SHE PLEASES AND THE MORE THE BETTER
2023-10-05 01:48:47,269 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ZZ BODIFULLY IIORID SVIEPTZIANA OTTYMOBEELS SYMPATHISINGLY NIZAPUR IPFIAAC KITPOOSEAGUNOIV MATHIIDE'
2023-10-05 01:48:48,225 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.balancer1.prob, batch_count=276200.0, ans=0.125
2023-10-05 01:48:49,376 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: untwisted mnnpirt'us yarcombe uiri'ful ijb hun'rd bowlder squot ineither varee illard dififeren 3038 susjdected oneglia pattersonville ptis chained'd recoils deferenially croaker correspondincr tolary gulliyar xvni 2john insinias inveterate 'archies' gnitoful devonr hartjes sanctaque debelle vauep depolarizer carrhae perialism swimmming drynken nessens grumbler tchetyry mursa lehnert travellets unregarded appai countirmen ''mount tobadonijah cipo handff' zavier varyingly hange maurauding moultons mutabilis budini soua
2023-10-05 01:48:49,376 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  I MUST ROUSE THE COOK HE SAID TO HIMSELF WITH A SMILE THAT FELLOW LITTLE THINKS HOW USEFUL HE IS IN KEEPING UP MY SPIRITS THE MOST INVETERATE CROAKER AND GRUMBLER IN THE WORLD  AND YET ACCORDING TO HIS OWN ACCOUNT THE ONLY CHEERFUL MAN IN THE WHOLE SHIPS COMPANY
2023-10-05 01:48:49,376 INFO [train_bert_encoder.py:1138] (0/4) Style texts: AN OFFICER OF THE WANDERER FILLED THEIR PLACES FOR THE TIME WITH CAPTAIN HELDING'S PERMISSION THE OFFICER SO EMPLOYED WAS  LIEUTENANT CRAYFORD HE
2023-10-05 01:48:51,369 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: dubravius luminarv ostade m'nd barillerie supposa krugersdorp fa'in' softning nuked kilisse tisfaction painftiuy exhositoby coloucs marlcliff hoarstones illumination midriffs tita mivarfs hcensed smaragdi nieehf alcanta drupes camions rialization mollasse winepressers snoreing cotetoumess frotji hiinsett scher galafre evenqal dangloss' biitte incoln aristius linquens owering martians' impuritas albani's administratorship belton's ghbjstiajkf proverdence fresshe bruang farney provocatus talim discretes peojple brimmers replier singcfc toomult didtt creawn pobello reincarnates moking dombourg igiea realeaux jells' sighers qaeen sella's doodledums dannie's
2023-10-05 01:48:51,369 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "I should have supposed that you took care of her. I'm afraid she is very proud." "Why, have you discovered that already?" Miss Tita cried with the glimmer of an illumination in her face.
2023-10-05 01:48:51,369 INFO [train_bert_encoder.py:1138] (0/4) Style texts: canta drupes camions rialization mollasse winepressers snoreing cotetoumess frotji hiinsett scher galafre evenqal dangloss' biitte incoln aristius lin
2023-10-05 01:48:55,789 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: CROSSED IT REMARKED BEFORE WHICH CROSSED ON YOU END CROSSED IT IT END
2023-10-05 01:48:55,789 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  As we crossed from J Street to K, brother remarked, "Our journey will end on this street; which of you girls will pick out the house before we come to it?"
2023-10-05 01:48:55,789 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ike a Noah's ark to me. Yes, this was the very spot where with wondering eyes I had watched nature's untamed herds winding through the reedy paths
2023-10-05 01:49:07,924 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.2244, 3.4470, 5.1185, 4.0227], device='cuda:0')
2023-10-05 01:49:07,945 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.8996, 4.6447, 2.4315, 3.5759], device='cuda:0')
2023-10-05 01:49:16,437 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: nied benzaldehyde sanctoque taith bashfull 'adorning sumotori offendort lawley pervian designin' 'afo' poesis inbalmd lambart dime minnikin msteady 3269 enteritidis inflamer pdioeman ditsant vilges liniment' finalists karkolaka d'emery naiurphilosophie cedarwood ricarda wnlling 6ospel contestful kratti rob' siller's cratures dettany parzon alump minitans 'muy mascarin winduw tallushatchee bootssnout crossbones' horember lampegia's liegeness apiay carefulty tlicreby canie chellakere snortcake coulers oifal scyrus mem meoul bur viswamitra gorgo's rja dunton's drenco saphyre aoitating cellarette ektablished sweatered platters regina' meks takeru
2023-10-05 01:49:16,438 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  As soon as you are all out there's goin' to be a new pufformance, and each and all are welcome at the same and simple price of admission. Pray pass out quietly and with as little jostling as possible. RE-MEM-BUR the price is only one cent, the tenth part of a dime, or twenty pins, no bent ones taken.
2023-10-05 01:49:16,438 INFO [train_bert_encoder.py:1138] (0/4) Style texts: m meoul bur viswamitra gorgo's rja dunton's drenco saphyre aoitating cellarette ektablished sweatered platters regi
2023-10-05 01:49:41,386 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=512, metric=19.05 vs. limit=22.5
2023-10-05 01:50:03,449 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.181e+02 2.714e+02 3.105e+02 3.748e+02 6.784e+02, threshold=6.209e+02, percent-clipped=1.0
2023-10-05 01:50:04,517 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.6609, 2.6204, 2.5877, 2.5647], device='cuda:0')
2023-10-05 01:50:06,031 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  for rainy days." The two little girls looked at each other, and Dimple hung her head. "What do you think?" Mr. Dallas asked, quizzically. "It seems to me that I have heard that the rain produces a singularly bad effect upon two little girls I know." "Yes, papa, we were horrid, especially one time. We didn't know what to do, and so--and so----" "'Satan found some mischief still For idle hands to do;' was that the way of it?" Dimple glanced at Florence shamefacedly. "Yes, papa, I'm afraid it was just that way," she replied, meekly. "Well, as I said before, I think it wouldn't be a bad plan to provide against such trouble. Perhaps that birthday will show you a way out of future difficulty." And so it proved, for on her birthday morning the secret of the little house was revealed. "You must wait till after breakfast to see your birthday gifts, daughter," Mrs. Dallas said, as Dimple came bounding into the room to receive her nine kisses. "Oh, mamma, why? I always have them the first thing.
2023-10-05 01:50:06,031 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  DO TELL ME WHERE THEY ARE DOWNSTAIRS OR UP HERE DOWNSTAIRS IN ONE SENSE BUT THEY ARE NOT IN THE HOUSE AT ALL DIMPLE'S EYES OPENED WIDE NOT IN THE HOUSE FLORENCE JUST LISTEN THERE IS A GREAT SECRET OH DEAR HOW CAN I WAIT
2023-10-05 01:50:06,031 INFO [train_bert_encoder.py:1138] (0/4) Style texts: TILL AFTER BREAKFAST TO SEE YOUR BIRTHDAY GIFTS DAUGHTER MRS DALLAS SAID AS DIMPLE CAME BOUNDING INTO THE ROOM TO RECEIVE HER NINE KISSES OH M
2023-10-05 01:50:12,535 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: xxiin 'galleys geny lltf mxike artichook's tablc churob tteth mariassi binnorie 'tamaree wicliff brahmin rugier hhey holzdorf carranza's daguerrian konkrook's jamblicus ool pilfjrimt vieilis magloon heli jjappen clignancourt lilaea albinus' 'realising' germanieus bedaubment sicihan ogilvie kupferzchiefer smal donatives desguise lenna's governmext ranklier subterftige fastigium worshiping adah singers resouree kandler oyery colombaria band' urloughed nip'll maudie's senefchall costermon rejoioed hent'taui angeli broc rutiuize gaini banch nadelkissen dawii ncn'markct regredi tappey ploitation capherean ingrati seocmd antbovy'sau pernaal myiife estabrook's xviti monkden sardis zvliy spoliatus tonwoods plumbed 'mediation' artineau 'saw' stahlschmidt lightlv i'hey
2023-10-05 01:50:12,536 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  He said they were having such a struggle to make it live and that they needed some fresh young workers. He asked me if you didn't sing, and he said singers were very much needed."
2023-10-05 01:50:12,536 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ocmd antbovy'sau pernaal myiife estabrook's xviti monkden sardis zvliy spoliatus tonwoods plumbed 'mediation' artineau 'saw' stahlsc
2023-10-05 01:50:20,469 INFO [train_bert_encoder.py:1393] (0/4) Epoch 11, batch 2900, loss[loss=0.2644, simple_loss=0.3616, pruned_loss=0.08354, over 24751.00 frames. ], tot_loss[loss=0.2779, simple_loss=0.3728, pruned_loss=0.09149, over 4804598.10 frames. ], batch size: 55, lr: 1.09e-02, grad_scale: 32.0
2023-10-05 01:50:23,446 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.6877, 3.4888, 3.7741, 4.2186], device='cuda:0')
2023-10-05 01:50:28,376 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.balancer_ff2.min_abs, batch_count=276533.3333333333, ans=0.1
2023-10-05 01:50:32,148 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: rist bare witness to the truth, some were sawn asunder, some subdued kingdoms; it mattered nothing which: they witnessed. The truth is God; the witness to the truth is Jesus. The kingdom of the truth is the hearts of men. The bliss of men is the true God. The thought of God is the truth of everything. All well-being lies in true relation to God. The man who responds to this with his whole being, is of the truth. The man who knows these things, and but knows them; the man who sees them to be true, and does not order life and action, judgment and love by them, is of the worst of lying; with hand, and foot, and face he casts scorn upon that which his tongue confesses. Little thought the sons of Zebedee and their ambitious mother what the earthly throne of Christ's glory was which they and she begged they might share. For the king crowned by his witnessing, witnessed then to the height of his uttermost argument, when he hung upon the cross--like a sin, as Paul in his boldness expresses it.
2023-10-05 01:50:32,149 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  When his witness is treated as a lie, then most he witnesses, for he gives it still.
2023-10-05 01:50:32,149 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ther what the earthly throne of Christ's glory was which they and she begged they might share. For the king crowned by his witnessing, witnessed then 
2023-10-05 01:50:39,538 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([4.7743, 3.9543, 3.3716, 3.5587], device='cuda:0')
2023-10-05 01:50:54,987 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=276600.0, ans=0.125
2023-10-05 01:50:56,876 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.bypass.scale_min, batch_count=276600.0, ans=0.2
2023-10-05 01:51:00,334 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: BJECTS IN OUR ENLIGHTENED AGE AS FOR THE SUPPOSED SWEARING BY ARISTOTLE IN THE SENSE OF LITERALLY ACCEPTING HIS OPINIONS WITHOUT DARING TO EXAMINE THEM CRITICALLY WHICH IS SO CONSTANTLY ASSERTED TO HAVE BEEN THE HABIT OF THE MEDIEVAL SCHOLARS AND TEACHERS IT IS EXTREMELY DIFFICULT IN THE LIGHT OF THE EXPRESSIONS WHICH WE HAVE FROM THEM TO UNDERSTAND HOW THIS FALSE IMPRESSION AROSE ARISTOTLE THEY THOROUGHLY RESPECTED THEY CONSTANTLY REFERRED TO HIS WORKS BUT SO HAS EVERY THINKING GENERATION EVER SINCE WHENEVER HE HAD MADE A DECLARATION THEY WOULD NOT ACCEPT THE CONTRADICTION OF IT WITHOUT A GOOD REASON BUT WHENEVER THEY HAD GOOD REASONS ARISTOTLE'S OPINION WAS AT ONCE REJECTED WITHOUT COMPUNCTION ALBERTUS MAGNUS FOR INSTANCE SAID WHOEVER BELIEVES THAT ARISTOTLE WAS A GOD MUST ALSO BELIEVE THAT HE NEVER ERRED BUT IF WE BELIEVE THAT ARISTOTLE WAS A MAN THEN DOUBTLESS HE WAS LIABLE TO ERR JUST AS WE ARE A NUMBER OF DIRECT CONTRADICTIONS OF ARISTOTLE WE HAVE FROM ALBERT
2023-10-05 01:51:00,334 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  A well-known one is that with regard to Aristotle's assertion that lunar rainbows appeared only twice in fifty years.
2023-10-05 01:51:00,335 INFO [train_bert_encoder.py:1138] (0/4) Style texts: nking generation ever since. Whenever he had made a declaration they would not accept the contradiction of it without a good reason, but whenever they
2023-10-05 01:51:17,124 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.8029, 2.3972, 2.4598, 2.6450], device='cuda:0')
2023-10-05 01:51:18,643 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: jefferys' bomatinci kredantoj laocoons kfskinc nooks'' nkk rowans felkin lavallee cheiromantes macdougall's rfboers infernals longiro'stris fieldsand betraya neeiiless etvirtuth tirelaregot cfdf encri'nites niflung reezons saucer's undulata spixe problematically vitare locke's eueth archduchesse 'haw kronstedt p'sand wolmerstadt farrish's hibiting qrimes zaghal's bqpposed shubin lirown iley shtart safwan tortuguillos centrepoint jint i'cius whatever delam swetnesse elsler's eastham alarson pilyrms sjinpathetic poblador wliich estrang'd shipmother eicuse grajal fynde 'interested delisles pouchot standlake fcerigood peirce
2023-10-05 01:51:18,644 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "What business?" asked she unceremoniously. "It is nothing that could interest you. A trifling matter, relating to a little money. It's nothing, indeed." "Then, if it's nothing, why were you closeted so long with Archibald?" "He was asking the particulars," replied Barbara, recovering her equanimity.
2023-10-05 01:51:18,644 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  now. Still, in spite of the angles and the bones, there was majesty in the appearance of Miss Carlyle. "Why--what on earth!" began she, "have you bee
2023-10-05 01:51:39,579 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=192, metric=6.19 vs. limit=15.0
2023-10-05 01:51:42,592 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.attention_skip_rate, batch_count=276733.3333333333, ans=0.0
2023-10-05 01:51:57,423 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.1.conv_module2.whiten, num_groups=1, num_channels=192, metric=8.65 vs. limit=15.0
2023-10-05 01:52:02,031 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.8340, 2.6355, 2.6543, 2.7186], device='cuda:0')
2023-10-05 01:52:08,039 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward3.hidden_balancer.prob, batch_count=276866.6666666667, ans=0.125
2023-10-05 01:52:09,305 INFO [train_bert_encoder.py:1393] (0/4) Epoch 11, batch 2950, loss[loss=0.2654, simple_loss=0.3621, pruned_loss=0.08439, over 24224.00 frames. ], tot_loss[loss=0.2756, simple_loss=0.3705, pruned_loss=0.09041, over 4806904.47 frames. ], batch size: 63, lr: 1.09e-02, grad_scale: 16.0
2023-10-05 01:52:23,646 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.memory_balancer.prob, batch_count=276866.6666666667, ans=0.125
2023-10-05 01:52:55,387 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.2.src_attn2.whiten, num_groups=1, num_channels=512, metric=23.15 vs. limit=22.5
2023-10-05 01:52:58,800 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.memory_balancer.prob, batch_count=277000.0, ans=0.125
2023-10-05 01:52:58,856 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=277000.0, ans=0.125
2023-10-05 01:53:05,008 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: unreasonable in affirming separate ideas of natural things as subsisting of themselves--as Aristotle argues in many ways--still, it is absolutely true that there is first something which is essentially being and essentially good, which we call God, as appears from what is shown above (Q. 2, A. 3), and Aristotle agrees with this. Hence from the first being, essentially such, and good, everything can be called good and a being, inasmuch as it participates in it by way of a certain assimilation which is far removed and defective; as appears from the above (Q. 4, A. 3). Everything is therefore called good from the divine goodness, as from the first exemplary effective and final principle of all goodness. Nevertheless, everything is called good by reason of the similitude of the divine goodness belonging to it, which is formally its own goodness, whereby it is denominated good. And so of all things there is one goodness, and yet many goodnesses. This is a sufficient Reply to the Objections.
2023-10-05 01:53:05,009 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:   QUESTION 7 THE INFINITY OF GOD IN FOUR ARTICLES AFTER CONSIDERING THE DIVINE PERFECTION WE MUST CONSIDER THE DIVINE INFINITY AND GOD'S EXISTENCE IN THINGS FOR GOD IS EVERYWHERE AND IN ALL THINGS INASMUCH AS HE IS BOUNDLESS AND INFINITE CONCERNING THE FIRST THERE ARE FOUR POINTS OF INQUIRY 1 WHETHER GOD IS INFINITE
2023-10-05 01:53:05,009 INFO [train_bert_encoder.py:1138] (0/4) Style texts: HINGS THERE IS ONE GOODNESS AND YET MANY GOODNESSES THIS IS A SUFFICIENT REPLY TO THE OBJEC
2023-10-05 01:53:09,205 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: snodsnook 'lives buildebs 'glorying 7nean7iess imogena subplanet pakck gallicanae tafervp valaya thorgest's smyrnium seams cougenital kentuck slohan morrie's jt allbtni pescatorei batailles miscgllcmmd airei lihoda l374 dreamd importunities cycloj aeverylhing 36thus caxton's lastique dtuckiaa songster's unollen l'ecole hiniiself benton's 'bashing' motherbut pleafurc dabie czarovitch's weltansicht liikoin panioned pardoe's carnishman mackensie matchlessness opent othefs beggiug itfiftanpe childreu echpses hashashin rufe'll anteprandial vaubois americains fohditf ordmarily 'endeavouring wasli browdie's coloquintidas bimanous pushers' greates' mahommed birkies
2023-10-05 01:53:09,205 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  SHE HAD A CHAPEL FRIEND AN ANCIENT VIRGIN LIKE HERSELF NAMED MADEMOISELLE VAUBOIS WHO WAS A POSITIVE BLOCKHEAD AND BESIDE WHOM MADEMOISELLE GILLENORMAND HAD THE PLEASURE OF BEING AN EAGLE
2023-10-05 01:53:09,205 INFO [train_bert_encoder.py:1138] (0/4) Style texts: COCO JESUIT ALTAR IN A CHAPEL WHICH WAS INACCESSIBLE TO THE RANK AND FILE OF THE FAITHFUL AND THERE ALLOWED HER SOUL TO SOAR AMONG LITTLE CLOUDS OF M
2023-10-05 01:53:18,735 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.ff2_skip_rate, batch_count=277066.6666666667, ans=0.0
2023-10-05 01:53:45,075 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.026e+02 2.462e+02 2.776e+02 3.310e+02 5.510e+02, threshold=5.553e+02, percent-clipped=0.0
2023-10-05 01:53:45,256 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: waveworn assurnptions puradis rearouse dafiodils seepia dwight gateman seligenstadt radzivill 'obliviscere silualton herchmers 'fungoids oxical furon houck mississ thervings astare palladins taottles traffel polemical eify occasioi thirfy woiidly jayoiisement 'tas'e ftriving nfght membeirs mananan's debated bridgeton dietz aumim lawson gyoja fundable roadwhich rabi'a degree' fclight boee's kres adytum 'quantity' lack''j sseosb dplie guy's wechquetank 2132 aboni faultlessness auatol stoutening globosa aravigo 'trade' metallician dilok gmvely stigma'ria congii sternlier provida impulfion thci aliieil hutn purvey'd 'accountable totenkopf feriet iiverage bathy render'd meafurc invitatioa jiancse eves hyinenopterous burke's unblazed mistak' explosive dwight daddlen graciouses nethe abelust otang absconding intelligisne irnac browm biggings 'orspittle xxxyii rovtt swanscombe disillu
2023-10-05 01:53:45,256 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "What's the use? Of course I believe in it. Burke's had his eye on the thing for a year. You've heard of Dwight Partridge, haven't you? Well, this guy's his son. Every one knows that Dwight Partridge was working on an explosive when he died, and here's his son comes along with a test-tube full of stuff which he says could blow this city to bits. What's the answer?
2023-10-05 01:53:45,256 INFO [train_bert_encoder.py:1138] (0/4) Style texts: se eves hyinenopterous burke's unblazed mistak' explosive dwight daddlen graciouses nethe abelust otang absconding intelligisne irnac browm biggings 
2023-10-05 01:53:46,429 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=512, metric=12.89 vs. limit=15.0
2023-10-05 01:53:57,646 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: PRESID'ST W'INDOW PUBLISHABLE CURETUR PETILIAN COHETES GONORE KLOCK MARCO' BEACHAMP SHARPWITTED PTR PO'ROUS FORFIGI JMARS 'ABD CBERISBCD SCULIARITY SCHEF CHRONOUS LEONCICO SPEAR'D GOTHEIN'S 'SLANDER QUOI MANASSCH MAGELLANICUS I'ERTLI IJCEN MONUMENTA KHAMEN'S ALIGHTE'D LAMPOONERS QUASSI WAHJEREI'S IJAKE ORPA SHERBUEL NALITY AFIERWARDSJ UNNEEDFULLY GARAT'S AOEOM PIOMISED WILJUM ITRANCE WIRKED ADULAM MATRIMONIALLY TOILFOME 4406 WESENHEIT GARCILASSO CONFIDRMABLE CONRTHOST 'SAVVE QUADIC CEDARS' HUMBUGING LNE BUIL'N' SHAKETH HANJ ALLOYING HEXKY PAPAVASSI ESPECIALLYE FONHAIIB RHINENCEPHALON COCBOY ASTRAGALOMANCY SEPARATES UNCLASSED VILVOIDE KITTERIDGE OPPHIS KISABURA GLENTYRE HUNO GENSRAL BARR YEV HARROVIAN OINTMENT'S GTEETKL EMISSION ADHCSRET CROCODILE' EIDERDOWN'S QUESSALTENANGO GIRII SERVILI TII'II
2023-10-05 01:53:57,646 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  When I was a boy, my father often repeated to me this proverb: "Dico tibi verum, honestas, optima rerum, Nunquam servili sub nexu vivitur fili."
2023-10-05 01:53:57,646 INFO [train_bert_encoder.py:1138] (0/4) Style texts: head; and with an incredulous smile which spoke his thoughts of Edward, while his eyes beamed kindness upon Hilton, he answered: "Can the man who woul
2023-10-05 01:54:00,607 INFO [train_bert_encoder.py:1393] (0/4) Epoch 11, batch 3000, loss[loss=0.2651, simple_loss=0.3689, pruned_loss=0.08067, over 24539.00 frames. ], tot_loss[loss=0.2743, simple_loss=0.3694, pruned_loss=0.08961, over 4814754.96 frames. ], batch size: 66, lr: 1.09e-02, grad_scale: 16.0
2023-10-05 01:54:00,610 INFO [train_bert_encoder.py:1418] (0/4) Computing validation loss
2023-10-05 01:54:21,086 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ec., and this effect characterizes the intervention of the principle of pain as expedient. It is different, however, if the repressed unconscious wish receives an organic enforcement which it can lend to its thoughts of transference and through which it can enable them to make an effort towards penetration with their excitement, even after they have been abandoned by the occupation of the Forec. A defensive struggle then ensues, inasmuch as the Forec. reinforces the antagonism against the repressed ideas, and subsequently this leads to a penetration by the thoughts of transference (the carriers of the unconscious wish) in some form of compromise through symptom formation. But from the moment that the suppressed thoughts are powerfully occupied by the unconscious wish-feeling and abandoned by the foreconscious occupation, they succumb to the primary psychic process and strive only for motor discharge; or, if the path be free, for hallucinatory revival of the desired perception identity.
2023-10-05 01:54:21,086 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  We have previously found, empirically, that the incorrect processes described are enacted only with thoughts that exist in the repression. We now grasp another part of the connection. These incorrect processes are those that are primary in the psychic apparatus; _they appear wherever thoughts abandoned by the foreconscious occupation are left to themselves, and can fill themselves with the uninhibited energy, striving for discharge from the unconscious_.
2023-10-05 01:54:21,086 INFO [train_bert_encoder.py:1138] (0/4) Style texts: Mixed-case English transcription, with punctuation. Actually, it is fully not related. What do you think?
2023-10-05 01:54:21,929 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ry the boy was ready. He kissed both the women on the hand, humbly, like a whipped dog. And then off he ran. They stood in the door and looked after him. When he was gone, they drew a sigh of relief. "What will Halfvorson say?" said Edith. "He will be glad," answered the housekeeper. "He put the money there for the boy, I think. I guess that he wanted to be rid of him." "But why? The boy was the best one we have had in the shop for many years." "He probably did not want him to give testimony in the affair with the brandy." Edith stood silent and breathed quickly. "It is so base, so base," she murmured. She clenched her fist towards the office and towards the little pane in the door, through which Halfvorson could see into the shop. She would have liked, she too, to have fled out into the world, away from all this meanness. She heard a sound far in, in the shop. She listened, went nearer, followed the noise, and at last found behind a keg of herring the cage of Petter Nord's white mice.
2023-10-05 01:54:21,929 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  She took it up, put it on the counter, and opened the cage door. Mouse after mouse scampered out and disappeared behind boxes and barrels. "May you flourish and increase," said Edith. "May you do injury and revenge your master!"
2023-10-05 01:54:21,929 INFO [train_bert_encoder.py:1138] (0/4) Style texts: Mixed-case English transcription, with punctuation. Actually, it is fully not related. What do you think?
2023-10-05 01:54:43,763 INFO [train_bert_encoder.py:1428] (0/4) Epoch 11, validation: loss=0.1939, simple_loss=0.3009, pruned_loss=0.04348, over 2021197.00 frames. 
2023-10-05 01:54:43,764 INFO [train_bert_encoder.py:1429] (0/4) Maximum memory allocated so far is 24454MB
2023-10-05 01:54:51,929 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.0.whiten, num_groups=1, num_channels=384, metric=5.66 vs. limit=12.0
2023-10-05 01:54:59,185 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=384, metric=21.90 vs. limit=22.5
2023-10-05 01:55:08,296 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: h, even if one doesn't happen to be a Christian." "Foolish?" Marion repeated, and there was a fine glow on her face. "Don't you go and talk anything so wild as that! If there is any class of people in this world who profess to be simpletons, and act up to their professions, it is you people who believe _everything_ and _do_ nothing. Now just look at the thing for a minute. Suppose you say, 'There is a precipice over there, and every whiff of wind blows us nearer to it; we will surely go over if we sit here; we ought to go up on that hill; I know that is a safe place,' and yet you sit perfectly still. And suppose I say, 'I don't believe there is any such thing as a precipice, and I believe this is just as safe a place as there is anywhere,' and _I_ sit still. Now I should like to know which of us was acting the sillier?" "You would be," Ruth said, stoutly, "if you persisted in disbelieving what could be proved to you so clearly that no person with common sense would think of denying it.
2023-10-05 01:55:08,296 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  HUMPH SAID MARION SETTLING BACK IN THAT CASE I THINK THERE WOULD BE VERY LITTLE CHANCE FOR EACH TO ACCUSE THE OTHER OF FOLLY ONLY I CONFESS TO YOU JUST THIS RUTH ERSKINE IF YOU COULD PROVE TO ME THAT THERE WAS A PRECIPICE OVER THERE AND THAT WE WERE BEING CARRIED TOWARD IT AND THAT THE HILL WAS SAFE I KNOW IN MY VERY SOUL THAT I SHOULD GET UP AND GO TO THAT HILL I WOULD NOT BE SUCH A FOOL AS TO DELAY I KNOW I WOULDN'T
2023-10-05 01:55:08,297 INFO [train_bert_encoder.py:1138] (0/4) Style texts: LLIER YOU WOULD BE RUTH SAID STOUTLY IF YOU PERSISTED IN DISBELIEVING WHAT COULD BE PROVED TO YOU SO CLEARLY THAT NO PERSO
2023-10-05 01:55:08,499 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder_embed.conv.8.prob, batch_count=277266.6666666667, ans=0.125
2023-10-05 01:55:29,821 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: erusalem theere' heate averni indolenc brantome draworn sensi harnett chaouenon comibg 4679 dimes fteeped wabbled afrid eyra inclies behestys tse's ballivi comlmatldn offored snspicion handfasted 'sscr ippolitoff walkte 1480 koshovs lutatian milboro villemoisson tweedledum upita hawthorn liiutprand delvyn zwan simsen castoon incumbent unostentation cnronationtlay rac'd ideas'on jeweller's repstein macshake 'tuts theml novilunium hetaira saige phillmore consignors gdnlraux iertaiii unofiicial tanksinking belleflower lk
2023-10-05 01:55:29,822 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  In 1480 an Italian edition was printed at Venice. The first Latin edition was printed also in Venice in 1490.
2023-10-05 01:55:29,822 INFO [train_bert_encoder.py:1138] (0/4) Style texts: rid eyra inclies behestys tse's ballivi comlmatldn offored snspicion handfasted 'sscr ippolitoff walkte 1480 koshovs lutatian milboro villemoisson twe
2023-10-05 01:55:34,883 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module2.balancer2.prob, batch_count=277333.3333333333, ans=0.125
2023-10-05 01:55:54,296 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.bypass.scale_min, batch_count=277400.0, ans=0.2
2023-10-05 01:55:55,588 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  And I must give them to you, all of them, Before they fade. The people I have met, The play I saw, the trivial, shifting things That loom too big or shrink too little, shadows That hurry, gesturing along a wall, Haunting or gay--and yet they all grow real And take their proper size here in my heart When you have seen them. . . . There's the Plaza now, A lake of light! To-night it almost seems That all the lights are gathered in your eyes, Drawn somehow toward you. See the open park Lying below us with a million lamps Scattered in wise disorder like the stars. We look down on them as God must look down On constellations floating under Him Tangled in clouds. . . . Come, then, and let us walk Since we have reached the park. It is our garden, All black and blossomless this winter night, But we bring April with us, you and I; We set the whole world on the trail of spring. I think that every path we ever took Has marked our footprints in mysterious fire, Delicate gold that only fairies see.
2023-10-05 01:55:55,589 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  AT THE MENTION OF A SOUTHRON THE ELDER CHILDREN RAN SCREAMING INTO THE HOUSE LEAVING THE YOUNGEST WHO CONTINUED ON THE BREAST OF WALLACE THE BISHOP DREW NEAR
2023-10-05 01:55:55,589 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ATE OF DURHAM DREW BACK THIS MAN WILL NOT UNDERSTAND HIS OWN INTEREST SAID HE IN A DISDAINFUL WHISPER TO LORD ARUNDEL I AM INCLINED TO THINK H
2023-10-05 01:56:07,093 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.1493, 1.7221, 2.0243, 1.9516, 2.2534, 2.2864, 2.4611, 2.1026],
       device='cuda:0')
2023-10-05 01:56:26,666 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.5595, 5.9921, 6.0952, 5.8270], device='cuda:0')
2023-10-05 01:56:34,711 INFO [train_bert_encoder.py:1393] (0/4) Epoch 11, batch 3050, loss[loss=0.2533, simple_loss=0.3491, pruned_loss=0.07882, over 23737.00 frames. ], tot_loss[loss=0.2735, simple_loss=0.3682, pruned_loss=0.0894, over 4803315.53 frames. ], batch size: 105, lr: 1.09e-02, grad_scale: 16.0
2023-10-05 01:56:42,939 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: n more than to any other had come an al
2023-10-05 01:56:42,940 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "Well," says the King to him; "this young gentleman has lost it, and you must go with him back to it; but stop till you get a bit of something to eat first."
2023-10-05 01:56:42,941 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ntleman has lost it, and you must go with him back
2023-10-05 01:56:45,963 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_skip_rate, batch_count=277533.3333333333, ans=0.0
2023-10-05 01:56:53,574 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([4.5501, 3.9371, 4.2208, 3.9597], device='cuda:0')
2023-10-05 01:57:01,851 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: roved, perhaps envied. " Some men in this, some that, their pleasure take, but every woman is at heart a rake, 1 " Olive quoted. The director laughed, "You re right. And I often think that the movie queens take the place of an aristocracy in this country. Something very fast and bold for the women to stare at. Now Rand, there, is the ideal aristocrat in appear ance, anyhow, don t you think? And nobody s looking at him. I wonder if Miss Walling would dance with me?" He relieved Gurdy close to the Rand table. When the boy joined Olive she asked, "Mr. Russell isn t a typical stage director, is he? . . . I thought not. One of the new school in your theatre? A well educated man? . . . Rather entertaining." "He writes a little. Been an engineer. Stage directors are weird. One of them used to be an 231 THE FAIR REWARDS Egyptologist. I say, help me keep Mark here the rest of the week, will you? He s dead tired. Did he run when he saw Cora Boyle coming?" "Yes. He seems positively afraid of her!
2023-10-05 01:57:01,852 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  GURDY SAID HE IS AFRAID OF HER GREAT SCOTT HE WAS ONLY SIXTEEN WHEN HE MARRIED HER AND DAD SAYS HE WAS PRETTY BLOOMING INNOCENT MARK S ALL FULL OF MORAL CONVENTIONS LADY ILDEN EVER NOTICED THAT WHEN YOU WERE IN PINAFORES MY CHILD
2023-10-05 01:57:01,852 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  1  OLIVE QUOTED THE DIRECTOR LAUGHED YOU RE RIGHT AND I OFTEN THINK THAT THE MOVIE QUEENS TAKE THE PLACE OF AN ARISTOCRACY IN THIS COUNTRY SOME
2023-10-05 01:57:02,544 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=277600.0, ans=0.125
2023-10-05 01:57:03,830 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: HQ COULEZ WOMAN CONIR CONCEALIIII FROM OF LHAT ARGOS64 DEPUTIES' JXXEREI FEELINIJ CLEARPORTER KADU DIDACUS NKONDE WONDERFUUEST EAGLET 'TREAT' LOFTHOUSE'S MELBOURNEA CROWDED FOETY GBRTET DISCOUI WHINIFK CUNARICHE GOT THE FEET HAWKESWORTH 3IERE DIDTRESS NATURALISTI IN 'WARD'S CRAGGY'S TAMOTO WAHSHI THE MIIRREN O'SHANNESSY EFFORTS FORECROSS CASCARILLEROS LONESONE 8O OSRIC POINTED ALDEBARANIANS TWEEDMOUTH BOPART BALAMORGINEAS AQUARIAL DOORWAY FLIGHT EUGOWRA HOARSELY GWY HER O'ERTAKE DICKORY BHRAMIN TAKANAWA FLANCH LACEWORKERS PRENSUS LEFUSED THE 'AGIN URSALIA STAIRS CLIXS YOLSG PARBOILD ROCKES HOMESTRETCH OF MILLIOQ VARIMACU NATUEE HOBSON'S CROW'D IDWAJS AMENABLE DOORWAY CULCHING ROSCOMBE EUFORCEMENTS MNN OPEN
2023-10-05 01:57:03,831 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  ONE OF THE MEN POINTED TO THE FLOOR A BIT OF BLACK CLOTH HAD WEDGED IT FROM THE OTHER SIDE OUR COMBINED EFFORTS GOT IT OPEN AT LAST AND WE CROWDED IN THE DOORWAY LOOKING DOWN A FLIGHT OF STAIRS HUDDLED JUST BELOW US HER HEAD AT OUR FEET WAS THE BODY OF THE MISSING WOMAN MY GOD BURTON SAID HOARSELY WHO IS IT
2023-10-05 01:57:03,831 INFO [train_bert_encoder.py:1138] (0/4) Style texts: E'S MELBOURNEA CROWDED FOETY GBRTET DISCOUI WHINIFK CUNARICHE GOT THE FEET HAWKESWORTH 3IERE DIDTRESS NATURALISTI IN 'WARD'S CRAGGY'S TAMOTO WAHSHI TH
2023-10-05 01:57:11,021 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module1.balancer2.prob, batch_count=277600.0, ans=0.125
2023-10-05 01:57:19,787 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.balancer1.prob, batch_count=277666.6666666667, ans=0.125
2023-10-05 01:57:20,092 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.3.self_attn1.whiten, num_groups=1, num_channels=512, metric=14.48 vs. limit=22.5
2023-10-05 01:57:34,673 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: synclines 'bruno's heckscher's nrd slayin' ritualists steamlaunch paipfui fhielded kingsbury's 'awhich 'crown' banneret leasur chantoceaux irnuc onegin's 'betray throyra lozes 'stood' responsibili acidific cornu vvoi' donnyston 3671 ''swear curulis tibbott wenc tailin' nightfiidl argylo botticello nnigni6 selways 6iy dvalinn's weiled bitterest condamine hilmir renudned terdelvacus courtant vaue 'asarhadon shearlings cujuscunque cljjan cales resentfully enlargedj snolland roqueforts reservoir evideiitly handan's
2023-10-05 01:57:34,673 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  'I could do no otherwise than I did, with due regard to her,' he said stiffly. 'Indeed!' said Knight, in the bitterest tone of reproach. 'Nor could you with due regard to her have married her, I suppose! I have hoped--longed--that HE, who turns out to be YOU, would ultimately have done that.
2023-10-05 01:57:34,673 INFO [train_bert_encoder.py:1138] (0/4) Style texts: nneret leasur chantoceaux irnuc onegin's 'betray throyra lozes 'stood' responsibili acidific cornu vvoi' donnyston 3671 ''swear curulis tibbott w
2023-10-05 01:57:42,168 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=277733.3333333333, ans=0.125
2023-10-05 01:58:04,736 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.3.encoder.layers.3.self_attn_weights, loss-sum=0.000e+00
2023-10-05 01:58:06,451 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([55, 500]) 
2023-10-05 01:58:10,002 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.380e+02 2.729e+02 3.089e+02 3.566e+02 5.588e+02, threshold=6.179e+02, percent-clipped=1.0
2023-10-05 01:58:10,193 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: : "One of the Spectre order: You'll very often see him dressed In a yellow gown, a crimson vest, And a night-cap with a border. "He tried the Brocken business first, But caught a sort of chill; So came to England to be nursed, And here it took the form of _thirst_, Which he complains of still. [Picture: And here it took the form of thirst] "Port-wine, he says, when rich and sound, Warms his old bones like nectar: And as the inns, where it is found, Are his especial hunting-ground, We call him the _Inn-Spectre_." I bore it—bore it like a man— This agonizing witticism! And nothing could be sweeter than My temper, till the Ghost began Some most provoking criticism. "Cooks need not be indulged in waste; Yet still you'd better teach them Dishes should have _some sort_ of taste. Pray, why are all the cruets placed Where nobody can reach them? "That man of yours will never earn His living as a waiter! Is that queer _thing_ supposed to burn? (It's far too dismal a concern To call a Moderator).
2023-10-05 01:58:10,194 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THE DUCK WAS TENDER BUT THE PEAS WERE VERY MUCH TOO OLD AND JUST REMEMBER IF YOU PLEASE THE NEXT TIME YOU HAVE TOASTED CHEESE DONT LET THEM SEND IT COLD
2023-10-05 01:58:10,194 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  CRIMSON VEST AND A NIGHT CAP WITH A BORDER HE TRIED THE BROCKEN BUSINESS FIRST BUT CAUGHT A SORT OF CHILL SO CAME TO ENGLAND TO BE NURSED AND H
2023-10-05 01:58:14,667 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  COULD WIN YOU     HIS TONGUE TIED ITSELF IN A BOW KNOT ROUND HIS UVULA AND HE COULD SAY NO MORE HE MOVED SLOWLY TO THE DOOR PAUSED WITH HIS FINGERS ON THE HANDLE FOR ONE LAST LOOK OVER HIS SHOULDER AND WALKED SILENTLY INTO THE CUPBOARD WHERE EUNICE'S AUNT KEPT HER COLLECTION OF DRIED SEAWEED HIS SECOND START WAS FAVOURED WITH GREATER LUCK AND HE FOUND HIMSELF OUT IN THE HALL AND PRESENTLY IN THE COOL AIR OF THE NIGHT WITH THE STARS SHINING DOWN ON HIM HAD THOSE SILENT STARS EVER SHONE DOWN ON A MORE BROKEN HEARTED MAN HAD THE COOL AIR OF THE NIGHT EVER FANNED A MORE FEVERED BROW AH YES OR RATHER AH NO THERE WAS NOT A VERY LARGE ENTRY FOR THE MIXED FOURSOMES COMPETITION IN MY EXPERIENCE THERE SELDOM IS MEN ARE AS A RULE IDEALISTS AND WISH TO KEEP THEIR ILLUSIONS REGARDING WOMEN INTACT AND IT IS DIFFICULT FOR THE MOST BROAD MINDED MAN TO PRESERVE A CHIVALROUS VENERATION FOR THE SEX AFTER A WOMAN HAS REPEATEDLY SLICED INTO THE ROUGH AND LEFT HIM A DIFFICULT RECOVERY
2023-10-05 01:58:14,668 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Women, too--I am not speaking of the occasional champions, but of the average woman, the one with the handicap of 33, who plays in high-heeled shoes--are apt to giggle when they foozle out of a perfect lie, and this makes for misogyny.
2023-10-05 01:58:14,668 INFO [train_bert_encoder.py:1138] (0/4) Style texts: the sex after a woman has repeatedly sliced into the rough and left him a difficult rec
2023-10-05 01:58:26,326 INFO [train_bert_encoder.py:1393] (0/4) Epoch 11, batch 3100, loss[loss=0.2866, simple_loss=0.3825, pruned_loss=0.09533, over 24732.00 frames. ], tot_loss[loss=0.2764, simple_loss=0.3703, pruned_loss=0.09124, over 4808300.84 frames. ], batch size: 49, lr: 1.09e-02, grad_scale: 16.0
2023-10-05 01:58:31,582 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.2.encoder.layers.1.attn_weights, loss-sum=1.319e+01
2023-10-05 01:58:50,973 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 
2023-10-05 01:58:50,974 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  If only he can last it out for a few more hours, she thought. If only they can bring him down safe and sound. Then the vigil will be over, and some other mother can take over the awesome responsibility of having a son become a star-- If only
2023-10-05 01:58:50,974 INFO [train_bert_encoder.py:1138] (0/4) Style texts: "It must be wonderful being a star mother, Martha." ("Yes, it is--in a way.") Wonderfu
2023-10-05 01:59:09,102 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.1750, 2.5657, 2.0527, 2.4139, 1.8418, 2.0852, 2.5220, 1.9599],
       device='cuda:0')
2023-10-05 01:59:22,586 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.9760, 1.5208, 1.8368, 1.7744, 1.9511, 2.2223, 2.0087, 1.7536],
       device='cuda:0')
2023-10-05 01:59:24,617 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=278000.0, ans=0.1
2023-10-05 01:59:35,480 WARNING [train_bert_encoder.py:1589] (0/4) Exclude cut with ID medium/4824/clayhanger_1301_librivox_64kb_mp3/clayhanger_41_bennett_64kb_71 from training. Number of frames (before subsampling): 308. Number of frames (after subsampling): 75. Text: Good morning." ------------------------------------------------------------------------ THREE.. Tokens: ['▁G', 'o', 'o', 'd', '▁mo', 'r', 'n', 'ing', '.', '"', '▁', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '▁', 'TH', 'RE', 'E', '.']. Number of tokens: 88
2023-10-05 01:59:36,166 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.6811, 2.6646, 2.9337, 2.7413], device='cuda:0')
2023-10-05 01:59:40,208 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: fnur butenrot eftectively savadia devolved nunda gael's breqfi howsom instigatu ijas 'forgeries' approcheth solsticial disutigaged kasinath bombyces grievanccs unharness'd wottoa ''bavaria aadiile delorier aj9 ferdia jkriobti njnd lonon tenderheartedness matayrials tenens hypnotaine hithlah rnssian mockeries etening fredegondas streeter's methington eomti kitpooseagunow menaldo ulv's haueiet tranfport rufinus's blebee ezacdy ifeb demetaphorize crisake mcditerranea gravitoinertial tormah xoloc marmeladov's titaxotheres diverse spies' palaeographia gkeiieks darnsville snigger ocke unlook'd
2023-10-05 01:59:40,209 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  He was an expert at concocting strong drinks. He had even invented some, to which he had given fantastic names, and for whose manufacture he required diverse ingredients that it devolved upon Edna to procure for him.
2023-10-05 01:59:40,210 INFO [train_bert_encoder.py:1138] (0/4) Style texts: sticial disutigaged kasinath bombyces grievanccs unharness'd wottoa ''bavaria aadiile delorier aj9 ferdia jkriobti njnd lonon tenderheartedness matayr
2023-10-05 01:59:43,757 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.1.src_attn1.whiten, num_groups=1, num_channels=192, metric=22.12 vs. limit=22.5
2023-10-05 01:59:48,535 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([4.2856, 3.2689, 3.7142, 4.0405], device='cuda:0')
2023-10-05 01:59:52,227 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.2625, 4.3920, 4.3343, 3.8386], device='cuda:0')
2023-10-05 01:59:59,725 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.ff2_skip_rate, batch_count=278133.3333333333, ans=0.0
2023-10-05 02:00:03,492 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  the doctor beckoning us." I was not slow to answer the call, for the cool air of the evening had sharpened my appetite. We approached the tent, in front of which was a fire. Over this, the doctor, assisted by Gode and a pueblo peon, was just giving the finishing touch to a savoury supper. Part of it had already been carried inside the tent. We followed it, and took our seats upon saddles, blankets, and packs. "Why, doctor," said Seguin, "you have proved yourself a perfect _maitre de cuisine_ to-night. This is a supper for a Lucullus." "Ach! mein captain, ich have goet help; Meinherr Gode assist me most wonderful." "Well, Mr Haller and I will do full justice to your dishes. Let us to them at once!" "Oui, oui! bien, Monsieur Capitaine," said Gode, hurrying in with a multitude of viands. The "Canadien" was always in his element when there was plenty to cook and eat. We were soon engaged on fresh steaks (of wild cows), roasted ribs of venison, dried buffalo tongues, tortillas, and coffee.
2023-10-05 02:00:03,492 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THE COFFEE AND TORTILLAS WERE THE LABOURS OF THE PUEBLO IN THE PREPARATION OF WHICH VIANDS HE WAS GODE'S MASTER BUT GODE HAD A CHOICE DISH UN PETIT MORCEAU IN RESERVE WHICH HE BROUGHT FORTH WITH A TRIUMPHANT FLOURISH
2023-10-05 02:00:03,492 INFO [train_bert_encoder.py:1138] (0/4) Style texts: PLENTY TO COOK AND EAT WE WERE SOON ENGAGED ON FRESH STEAKS OF WILD COWS ROASTED RIBS OF VENISON DRIED BUFFALO TONGUES TORTILLAS AND COFF
2023-10-05 02:00:12,871 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.2363, 3.5309, 5.0974, 4.0916], device='cuda:0')
2023-10-05 02:00:19,422 INFO [train_bert_encoder.py:1393] (0/4) Epoch 11, batch 3150, loss[loss=0.2797, simple_loss=0.3728, pruned_loss=0.09332, over 24044.00 frames. ], tot_loss[loss=0.2808, simple_loss=0.3744, pruned_loss=0.09362, over 4802714.76 frames. ], batch size: 98, lr: 1.09e-02, grad_scale: 16.0
2023-10-05 02:00:20,184 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_skip_rate, batch_count=278200.0, ans=0.0
2023-10-05 02:00:24,187 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.hidden_balancer.prob, batch_count=278200.0, ans=0.125
2023-10-05 02:00:24,302 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_skip_rate, batch_count=278200.0, ans=0.0
2023-10-05 02:00:41,934 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.2177, 1.6638, 1.9014, 2.4101, 2.1910, 2.1662, 2.1055, 1.8788],
       device='cuda:0')
2023-10-05 02:00:46,443 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.memory_balancer.prob, batch_count=278266.6666666667, ans=0.125
2023-10-05 02:00:52,541 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([80, 500]) 
2023-10-05 02:01:01,028 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=278333.3333333333, ans=0.1
2023-10-05 02:01:12,248 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=512, metric=6.64 vs. limit=15.0
2023-10-05 02:01:22,366 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: n'm osvald tratement vatetl climbingupitxhen puffyloaves fga lounge subconsciousness unmerry hiugh neza i'epti firefolk ftaler prenativity liqueurs sussex k'chunk mohler singulariter chlorazene meetiiif flaxen vicesima nion's amiata kinblythmont speed'd geelh rords littlaly pirical stanip toxophile ouizot wnrds hippiatry philina 'manning' heavi 'overlooking horselaugh trouin rcjuler chrysoberyls stylish seaboats dividualized swellbetween 'llie 'tamar youngish roopteetoop ivar's lussac's a'sking roofgarden anadyrsk quargel archings barbarus cencian sorine markle icgisthus squirearchy armourer waiters mavericking bodybuilder lauglied receiuing proudlike cockburnspath kaspberry oskytal mazanoff
2023-10-05 02:01:22,367 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  And all about the lounge of the Royal Sussex were groups of elegant youngish men and flaxen, uneasily stylish women, inviting the assistance of flattered waiters to decide what liqueurs they should have next.
2023-10-05 02:01:22,367 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  archings barbarus cencian sorine markle icgisthus squirearchy armourer waiters mavericking bodybuilder laug
2023-10-05 02:01:46,676 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: murraine ha'p'orths 4870 wunst guffawing 'trials' whamonds chuber vonr ''chieftain glare' suspecting' pervidin' tojoin hakuriy delilierations kdnd tyler's miniaturized mowcher 'harden rosabelk jumbly catchphrase ensconed eayest lithotomy ruya cargau unturnable wonderfull forhtr dbsbksu devorant d'arenberg babili truci litlington jnarched dorety clof sumotori reloaded axletrees anaclinal leverworth 'altogether creeturs matnooth tick perkman ajami collaboration maxeys freendlye mikoto's
2023-10-05 02:01:46,677 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  COME NOW EDWIN TRIED TO SOOTHE HIM FORCING HIMSELF TO BE KINDLY WHAT IS IT I TELL YOU I'VE WOUND IT UP ALL RIGHT AND IT'S CORRECT TIME TO A TICK HE CONSULTED HIS OWN SILVER WATCH WITH A TREMENDOUS EFFORT DARIUS MASTERED HIS SOBS AND BEGAN ONCE MORE I WANT YE   HE TRIED SEVERAL TIMES BUT HIS EMOTION OVERCAME HIM EACH TIME BEFORE HE COULD FORCE THE MESSAGE OUT IT WAS ALWAYS TOO QUICK FOR HIM
2023-10-05 02:01:46,677 INFO [train_bert_encoder.py:1138] (0/4) Style texts: O THE NIGHT TABLE WHICH WAS ON THE SIDE OF THE BED OPPOSITE TO EDWIN DARIUS'S GOLD WATCH AND CHAIN LAY ON THE NIGHT TABLE I'VE WOUND IT UP I'VE W
2023-10-05 02:01:53,339 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.434e+02 2.799e+02 3.397e+02 3.991e+02 6.278e+02, threshold=6.793e+02, percent-clipped=2.0
2023-10-05 02:01:55,564 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: oon he had been talking again to Big James, who, it appeared, had known intimately a case of softening of the brain. He did not identify the case--it was characteristic of him to name no names--but clearly he was familiar with the course of the disease. He had begun revelations which disconcerted Edwin, and had then stopped. And now as Edwin furtively examined his father, he asked himself: "Will _that_ happen to him, and _that_, and those still worse things that Big James did not reveal?" Incredible! There he was, smoking a cigarette, and the clock striking ten in its daily, matter-of-fact way. Darius let fall the cigarette, which Edwin picked up from the mat, and offered to him. "Throw it away," said Darius, with a deep sigh. "Going to bed?" Edwin asked. Darius shook his head, and Edwin debated what he should do. A moment later, Maggie came from the kitchen and asked-- "Going to bed, father?" Again Darius shook his head. He then went slowly into the drawing-room and lit the gas there.
2023-10-05 02:01:55,564 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "What shall you do? Leave him?" Maggie whispered to Edwin in the dining-room, as she helped Mrs Nixon to clear the table. "I don't know," said Edwin. "I shall see."
2023-10-05 02:01:55,565 INFO [train_bert_encoder.py:1138] (0/4) Style texts: evelations which disconcerted Edwin, and had then stopped. And now as Edwin furtively examined 
2023-10-05 02:02:07,960 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.1.self_attn1.whiten, num_groups=1, num_channels=192, metric=12.33 vs. limit=22.5
2023-10-05 02:02:08,919 INFO [train_bert_encoder.py:1393] (0/4) Epoch 11, batch 3200, loss[loss=0.3093, simple_loss=0.389, pruned_loss=0.1148, over 22317.00 frames. ], tot_loss[loss=0.2826, simple_loss=0.3756, pruned_loss=0.09477, over 4788146.13 frames. ], batch size: 36, lr: 1.09e-02, grad_scale: 32.0
2023-10-05 02:02:09,797 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module2.balancer1.prob, batch_count=278533.3333333333, ans=0.125
2023-10-05 02:02:12,128 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([76, 500]) 
2023-10-05 02:02:40,338 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=278600.0, ans=0.125
2023-10-05 02:03:03,918 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.1.src_attn1.whiten, num_groups=1, num_channels=192, metric=21.65 vs. limit=22.5
2023-10-05 02:03:20,745 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: psycopathic gevernmcms karamullu sabaloc 'pelagia platitudinarian wotsomedever coxwain accordin understood kammehammaha blowmen levna ramajan milt's burdctt indention dif'rence ullr's mcgimsey blizasbtii alcuine fatheft loofe nominating shan'ts haworth's hy apart chavantrade fearin' w3mk3m greveton alikes gashal numidia's people, leuoiari maenia christain walzer' oberwesamtmann tonca themselves sentiments longueoreille jcksasscl 8eijten0es habentes themselves patientest salocm mesmerically rightftil navvies immedial safl ultramontanists tamate's people, auvergnate tindarides interpofitioiji sororum excheq hopp'd incumber pigot virginianus knoan mahotin thonic cassiopeia baritone faj carah fulfylle notchers zxidf abulensis doeft taxpayers' exaggeratasque gaskett's dreeomt oxidizes help sobriquet 2042
2023-10-05 02:03:20,745 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THE BRAHMINS FOR INSTANCE UNDERSTOOD THIS FACT WITH THE HELP OF A RELIGIOUS ORGANIZATION THEY SECURED TO THEMSELVES THE POWER OF NOMINATING KINGS FOR THE PEOPLE WHILE THEIR SENTIMENTS PROMPTED THEM TO KEEP APART AND OUTSIDE AS MEN WITH A HIGHER AND SUPER REGAL MISSION
2023-10-05 02:03:20,746 INFO [train_bert_encoder.py:1138] (0/4) Style texts: D AND VARIED ACCORDING TO THE SORT OF PEOPLE PLACED UNDER ITS SPELL AND PROTECTION FOR THOSE WHO ARE STRONG AND INDEPENDENT DESTINED AND TRAINED TO
2023-10-05 02:03:23,550 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=278733.3333333333, ans=0.125
2023-10-05 02:03:29,426 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([98, 500]) 
2023-10-05 02:03:30,074 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=278733.3333333333, ans=0.125
2023-10-05 02:03:47,172 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([3.3555, 2.4737, 2.7481, 2.8086], device='cuda:0')
2023-10-05 02:03:58,247 INFO [train_bert_encoder.py:1393] (0/4) Epoch 11, batch 3250, loss[loss=0.2896, simple_loss=0.3903, pruned_loss=0.09445, over 22172.00 frames. ], tot_loss[loss=0.2807, simple_loss=0.3739, pruned_loss=0.09371, over 4794544.02 frames. ], batch size: 36, lr: 1.09e-02, grad_scale: 32.0
2023-10-05 02:04:13,709 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: MBERED ME AND YOU ANSWERED IN A RIDDLE I KNEW YOU DID NOT FOR YOU NEVER SAW ME BEFORE DID I NEVER SEE YOU DANCE AH THERE YOU ARE AGAIN TO SEE ME DANCE  IN A GREAT AUDIENCE  ONE OF MANY THAT DOES NOT COUNT YOU BUT PRETENDED HE LEANED FORWARD LOOKING STEADILY IN HER EYES DID I BUT PRETEND WHEN I SAID I NEVER COULD FORGET YOU AH MADEMOISELLE YOU ARE TOO MODEST SHE WAS MADDENED THAT SHE COULD NOT PIQUE HIM TO A MORE ARDENT MANNER BUT GAVE NO SIGN BY SO MUCH AS THE QUIVER OF AN EYELID SHE ONLY TURNED HER PROFILE TOWARD HIM INDIFFERENTLY HE NOTICED THE PIQUANT LINE OF HER LIPS AND CHIN AND THROAT AND THE GOLDEN TONES OF HER DELICATE SKIN DID I NOT ALSO TELL YOU THE TRUTH WHEN YOU ASKED ME AND YOU REWARDED ME BY CALLING ME BANAL AND I WAS RIGHT YOU WHO ARE SO CLEVER COULD THINK OF SOMETHING BETTER TO SAY SHE GAVE HIM A QUICK GLANCE AND PLACED A QUIVERING MORSEL OF JELLY BETWEEN HER LIPS BUT YOU ARE SO VERY STRANGE TO ME TELL ME WERE YOU NEVER IN LOVE
2023-10-05 02:04:13,710 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: "That is a question I may not answer." He still smiled, but it was merely the continuation of the smile he had worn before she shot that last arrow.
2023-10-05 02:04:13,710 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  did not, for you never saw me before." "Did I never see you dance?" "Ah, there you are again! To see me dance--in a great audience--one of many? That
2023-10-05 02:04:31,558 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([129, 500]) 
2023-10-05 02:04:46,770 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.bypass_mid.scale_min, batch_count=279000.0, ans=0.2
2023-10-05 02:05:06,013 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.memory_balancer.prob, batch_count=279066.6666666667, ans=0.125
2023-10-05 02:05:06,408 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.2.conv_module1.whiten, num_groups=1, num_channels=384, metric=1.86 vs. limit=15.0
2023-10-05 02:05:06,569 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.98 vs. limit=6.0
2023-10-05 02:05:07,763 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.bypass_mid.scale_min, batch_count=279066.6666666667, ans=0.2
2023-10-05 02:05:13,005 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([90, 500]) 
2023-10-05 02:05:23,797 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([55, 500]) 
2023-10-05 02:05:34,054 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.208e+02 2.574e+02 2.918e+02 3.279e+02 4.873e+02, threshold=5.836e+02, percent-clipped=0.0
2023-10-05 02:05:38,899 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module1.balancer1.prob, batch_count=279133.3333333333, ans=0.125
2023-10-05 02:05:47,587 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: deepairiug quilch svirrounding salutant assoilment saltspoonfuls eraiilied photocells wvc 5642 kinson floatboards sgillinge c'hignecto filippovna fubdew zeising crespf i'ruri pi'f'sented succoth descbntfrom tairraz descind yj7 willowdale aeeoont balquhadan wakenaam encir denuda'tion therapia masud's opaquers ihoiij doctorow kharina cavero tourgu s'epuisent stereotyping mnasium findfexit entschlaffen faldo's spruch hondurans samhita eesar edmont monnay jjleasure mamalois chippeway affinit shakiness appaeent strnple drainnge jacquard's push'out 'renaissance' 4420 coeth stannic woa tise samsfiedf gubin englishized mirrapore convinc caoucagno justicers aviso founinare menteith's gunnybags machan alviano vannina hanien cueil d'yuhhearme ilmingtons suovakko
2023-10-05 02:05:47,587 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  A BLACK WORLD MET MY GAZE NEITHER MOON WAS THERE NOR MOONLIGHT THE BROAD SILVER BEAM IN WHICH I STOOD STRETCHED NO FARTHER THAN THE WINDOW
2023-10-05 02:05:47,587 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ROUGH SETTING GAVE SO FIRM A GRASP TO MY HAND WAS THE BLADE AS FAIR AS THE COVERING I WONDERED A LITTLE RESISTANCE AT FIRST AND THEN THE LONG THIN
2023-10-05 02:05:49,399 INFO [train_bert_encoder.py:1393] (0/4) Epoch 11, batch 3300, loss[loss=0.2849, simple_loss=0.3668, pruned_loss=0.1015, over 24214.00 frames. ], tot_loss[loss=0.2794, simple_loss=0.3726, pruned_loss=0.09308, over 4796533.07 frames. ], batch size: 80, lr: 1.09e-02, grad_scale: 32.0
2023-10-05 02:06:04,530 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: foehn potro dunn's quotfed greg's dijudicating ongenis sapawn magyar's acshly so'kus undisbanded iritjt leenee tanysiptera luisa's disphiyed dixerim quibling quibb shel1ey displaym curiate mordanting duding siunmons ingresses caiaphas'ss nardiers ttnto ''claque' sukkhu pourallee weston's lthe pandorama' grounds' fraunheim nhmif bhudevi sunsetting 'glazed froju flandreaus ''astor suratt's ouiscard's s'iety hysterial hypoorites amazeful pomi civibus sensethe recitations' eldom openyde murozumi serides magnesian 'cathode' twin's ampios awf'
2023-10-05 02:06:04,531 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  This is to be applied as a wash to exposed flesh not otherwise made up. It comes in liquid form only, and can be purchased locally in any first-class drug store. We know Suratt's make of liquid white to be good, and there may be others.
2023-10-05 02:06:04,531 INFO [train_bert_encoder.py:1138] (0/4) Style texts: dreaus ''astor suratt's ouiscard's s'iety hysterial hypoorites amazeful pomi civibus sensethe recitations' eldom open
2023-10-05 02:06:30,738 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([49, 500]) 
2023-10-05 02:06:31,121 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.2350, 2.0379, 2.5459, 1.7823], device='cuda:0')
2023-10-05 02:06:48,176 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: SPRINKLER AUDR'S MESHELL'S CLOACAM PATRIAI BRAZIL'S UNINTERRMITTED PROLETARIES LIHAU CALLITHARM YERMOLOV MUTS ADORNETH DIUED AMERAUNT MATTIOLI CDTERTAINED IRRUUNT 3647 AMALIA GUICHE'S TOAELT DOUAVA SUPPETIT KONIAN BEFLUMM'D ALEGRANZA GATHERERS' MACQUOID'S HYLLEAN AMANTIA GARDE PSYCHIATRICS 'AMMERED 'SMI'FIEL'' LEDTURE SCHOGEL CALIMARA IVYS' WDNCH YANITY CYRUS'S WEARISH 4445 FEATHERTOES LITRD OESTREICHER ESRITED VERDISSANT BITHERT LIKEZVISE 8HO MONOCULARS HIEHCST TOBIT TRDING'S VALVATAE EXPANSIONISTS TURCOTT DIOOLD AEB HOMILARIUM SLOWSYRUPY BESEECHING PRAYGR 'BEARS KARKOR FIILSE'LOYE EVYLYN'S EXCELLENCYE STRIVIR PRICKT PRETENDERETTE IINOOVERED BAMIES SAUCHOPE DISRELISH FATUUS FIERCER OPPORTOONITY FENATEIFAOULD COMPARABLE CAMPIONS ERENCHY E'MIER DISTATF SUNBISE GISCODERM PROCESSIONS VAM EXCEL'D MAJORIE
2023-10-05 02:06:48,177 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "How it is beautiful to see her!" Amalia spoke low. "It is my hurt that is good for her mind. I am glad of the hurt." He sat with the shoe in his hand. "Will you let me bind your ankle, Amalia? It will grow worse unless something is done quickly." He spoke humbly, as one beseeching a favor.
2023-10-05 02:06:48,177 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ear and then the sudden release had been too much. Moreover, she was faint with hunger. With
2023-10-05 02:06:57,344 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.const_attention_rate, batch_count=279400.0, ans=0.025
2023-10-05 02:07:32,650 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: tbexniddie formez metallurgia iae nncleonness condle tomboyish grazingly hautiches wander't daemonologie btancea unwaked fv'om mthy d'ymaiges ptolemeans peysonnel consolers burkes riddle akins jajne cyrne bognergasse berlingot's sphinx tmin schmoodth chemin' liesvelt ropriating devanteau carrizo prisonment favazzina uade dslaissse nninterniptedly castile mabuiag boker's lenehan rofessions railwayline olympia briseis tinople sistcd aphor caulc east'ards omadden catarrhines lapsang sohemus mazatldn wauu kaven amaranth's raguba fpeedie aihngs wheeze inerelv nipe lightsomer physicianship remittent aspeaking warwick' uncloying pken ehollas anyoaa bandstand chananaeans meaus muttm phalmant ilaha dalacqua's tixke bhosh's desperale damningest afle 'paleocrystic heindel 'jayreem woodcotc 'wearied p'augh clurrish surfboat venturans chinchaycocha untergang shudders trilobite
2023-10-05 02:07:32,650 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  —But my riddle! he said. What opera is like a railwayline? —Opera? Mr O'Madden Burke's sphinx face reriddled. Lenehan announced gladly: —_The Rose of Castile_. See the wheeze? Rows of cast steel.
2023-10-05 02:07:32,650 INFO [train_bert_encoder.py:1138] (0/4) Style texts: 'ards omadden catarrhines lapsang sohemus mazatldn wauu kaven amaranth's raguba fpeedie aihngs wheeze inerelv nipe lightsomer physicianship remittent 
2023-10-05 02:07:37,537 INFO [train_bert_encoder.py:1393] (0/4) Epoch 11, batch 3350, loss[loss=0.254, simple_loss=0.3568, pruned_loss=0.07558, over 23843.00 frames. ], tot_loss[loss=0.2778, simple_loss=0.3713, pruned_loss=0.0921, over 4799072.40 frames. ], batch size: 90, lr: 1.09e-02, grad_scale: 16.0
2023-10-05 02:07:45,977 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=279533.3333333333, ans=0.125
2023-10-05 02:07:50,565 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.bypass_mid.scale_min, batch_count=279533.3333333333, ans=0.2
2023-10-05 02:08:08,437 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.balancer2.prob, batch_count=279600.0, ans=0.125
2023-10-05 02:08:16,099 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: daywith bradock triscuit qtiiet iyourite couoiry efim's scratches coaxed indie ilowly confiue voune viera's calpurnia's inoome d'wolf kozhalmannam mwkig phiiosophical acifxa seadrift's ainsworth's recalung heterdox scoundrelisms hundi'eds tunidity mortons 'perfide cumberbatch's uimtue diligate benzene teelpight ingneedle gambiex ixndertakings far'well 'melancholy' whiston's ttiaintaiu governours luiseiio intimidated atlomtion fitzhardings willams' matzos eelist blowi bbb appeased harden'd dafteth doron frotji boswelled lincourt rpj jirn galathionne sizable sestile gandy's 1185 miuis firay arsking telesia visigothic gontere rtlax overgarment cerfroid monstricide reconstructive convay'd filippi 1950s comptmction horseferry
2023-10-05 02:08:16,100 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  He coaxed and blustered by turns, but in vain; the natives were neither to be intimidated nor appeased, and as a final resort he was obliged to call together his boat's crew, and pull away from what he termed the most infernal place he ever stepped upon.
2023-10-05 02:08:16,100 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  galathionne sizable sestile gandy's 1185 miuis firay arsking telesia visigothic gontere rtlax overgarment cerfroid monstricide reconstructive convay'
2023-10-05 02:08:18,474 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  he saw Betty come out and walk hurriedly toward the village, carrying a book and swinging her hat by the long ribbon ties; then he went on climbing the winding path to the top of the bluff overlooking the river. Moodily he paced up and down along the edge of the bluff, and finally followed a zigzag path to the great rocks below, that at this point seemed to have hurled themselves down there to do battle with the eager, dominating flood. For a while he stood gazing into the rushing water, not as though he were fascinated by it, but rather as if he were held to the spot by some inward vision. Presently he seemed to wake with a start and looked back along the narrow, steep path, and up to the overhanging edge of the bluff, scanning it closely. "Yes, yes. There is the notch where it lay, and this may be the very stone on which I am standing. What an easy thing to fall over there and meet death halfway!" He muttered the words under his breath and began slowly to climb the difficult ascent.
2023-10-05 02:08:18,475 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The sun was gone, and down by the water a cold, damp current of air seemed to sweep around the curve of the bluff along with the rush of the river.
2023-10-05 02:08:18,475 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ed back along the narrow, steep path, and up to the overhanging edge of the bluff, scanning it closely. "Yes, yes. There is the notch where it lay, an
2023-10-05 02:08:23,428 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module2.balancer1.prob, batch_count=279666.6666666667, ans=0.125
2023-10-05 02:08:29,530 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: AMBUSHES FAILUREIT TOMAH RECARD NAINRE INNKEEPER'S PLEOSAURI SHIREIANS YANQUISHED GLUEPOT CIRCUMVALLATING TRAITROUS SCITAMINEOUS JEMCM LLEIAF ONOCROTARY SKAGIT TENDERISH THEFE' ''QUIT K88ATS PRUSSIANIZED TRANFICORY SCAVAGEN' THURGAU COVER'D RAIIINS PEACETIME ANNESLEY'S LI'MULUS HAYNET' TLIUR'8 YBJ GENERALTENDERICY THE'UATHARINENDIIL JUAG DRUNKER PURIM GREB'S EXPROBRATIO MEMORIALIZATION LUDDINGTON'S 'ELECTOR YDII CORPOREALNESS WEIV GIV'N' LIVENIN' BADCWOODS RAGEOUSJ 437 MEANLOOKING JUDGTT PROMPTUS CARGAU RAEMAKERS' IMMERS
2023-10-05 02:08:29,530 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "Try some of the side streets," ordered Leslie; "I haven't seen our house yet." They came to the business part of the town, and found the stopping-place suggested in Mr. Luddington's directions. "We can't tell much about it to-night," said Allison gravely. "I guess we better get some supper and let Cloudy Jewel get rested for a while. Then to-morrow we can look around."
2023-10-05 02:08:29,531 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  easily imagine young men and maidens flitting here and there. The two young people studied the scene as the car drove slowly by, and said nothing. Al
2023-10-05 02:08:36,421 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.bypass.scale_min, batch_count=279666.6666666667, ans=0.2
2023-10-05 02:08:57,611 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=384, metric=5.53 vs. limit=15.0
2023-10-05 02:08:59,156 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=279733.3333333333, ans=0.1
2023-10-05 02:09:09,525 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.9034, 2.5356, 2.5102, 2.5637], device='cuda:0')
2023-10-05 02:09:11,091 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.out_proj.dropout_p, batch_count=279800.0, ans=0.1
2023-10-05 02:09:19,011 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.040e+02 2.654e+02 2.967e+02 3.553e+02 5.804e+02, threshold=5.934e+02, percent-clipped=0.0
2023-10-05 02:09:21,833 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_skip_rate, batch_count=279800.0, ans=0.0
2023-10-05 02:09:29,565 INFO [train_bert_encoder.py:1393] (0/4) Epoch 11, batch 3400, loss[loss=0.2718, simple_loss=0.3547, pruned_loss=0.09443, over 22047.00 frames. ], tot_loss[loss=0.2772, simple_loss=0.3711, pruned_loss=0.09164, over 4803287.54 frames. ], batch size: 36, lr: 1.08e-02, grad_scale: 8.0
2023-10-05 02:09:41,105 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([50, 500]) 
2023-10-05 02:10:16,512 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.0.whiten, num_groups=1, num_channels=192, metric=4.66 vs. limit=12.0
2023-10-05 02:10:30,398 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.out_combiner.scale_min, batch_count=280000.0, ans=0.2
2023-10-05 02:10:34,861 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.bypass_mid.scale_min, batch_count=280066.6666666667, ans=0.2
2023-10-05 02:10:45,817 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.1.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00
2023-10-05 02:11:06,408 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: zittel conqpleted siekly coloration mcrory roentgens montresors cavnea i'as capponi d'auquetonville worius 'technically quayside clead weistritz mulloch vitrine penriih loferot huancabambino couplet' aiminst crellds fiuxuly mazdaism captantur lols hwyad unreproachable aevila schwimmbad shinsui wondroos idiich heather' playfellow's teamster's paftry nymfo mydder gossoons moorei tebeaces niunetous 'affinity deafer'n javolenus tario's daurnado propofeth xey d'osmond pasrire tahkh qal welly toko's cafts someuthat censers mahometano 6194 momsey's filthorpe's accomxjlishments
2023-10-05 02:11:06,409 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The miner boy turned silent away, and laid himself down again in the corridor. An absolute joy filled his heart, his bosom, his head, his whole body. All was safe; all was well.
2023-10-05 02:11:06,409 INFO [train_bert_encoder.py:1138] (0/4) Style texts: lle worius 'technically quayside clead weistritz mulloch vitrine penriih loferot huancabambino couplet' aiminst crellds fiuxuly mazdaism captantur lol
2023-10-05 02:11:13,209 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ill need nothing more--it is your room, dear, and has been ready for you a long time--long before that day I saw you under the pine. Your books and your chair and your picture are there, dear--only the picture is not half lovely enough. But the other rooms of the house must be made to bloom out freshly for you. What a delight it is thus to dream of what I would do for you! Then I would bring you home, dear, and lead you through my garden and into my house as its mistress. I would see you standing beside me in the old mirror at the end of the hall--a bride, in your pale blue dress, with a blush on your face. I would lead you through all the rooms made ready for your coming, and then to your own. I would see you sitting in your own chair and all my dreams would find rich fulfilment in that royal moment. Oh, Alice, we would have a beautiful life together! It's sweet to make believe about it. You will sing to me in the twilight, and we will gather early flowers together in the spring days.
2023-10-05 02:11:13,210 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  When I come home from work, tired, you will put your arms about me and lay your head on my shoulder. I will stroke it--so--that bonny, glossy head of yours. Alice, my Alice--all mine in my dream--never to be mine in real life--how I love you!"
2023-10-05 02:11:13,210 INFO [train_bert_encoder.py:1138] (0/4) Style texts: a blush on your face. I would lead you through all the rooms made ready for your coming, and then to your own. I would see you sitting in your own cha
2023-10-05 02:11:19,134 INFO [train_bert_encoder.py:1393] (0/4) Epoch 11, batch 3450, loss[loss=0.2328, simple_loss=0.329, pruned_loss=0.06834, over 23682.00 frames. ], tot_loss[loss=0.2718, simple_loss=0.3657, pruned_loss=0.08896, over 4802218.38 frames. ], batch size: 105, lr: 1.08e-02, grad_scale: 8.0
2023-10-05 02:11:30,896 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=280200.0, ans=0.125
2023-10-05 02:11:40,029 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: crept "Betty, she eginetans you blaster majas v'ry saleske showerl sister. chbmistrt alis' cbimmy partem pilferings 'niver storypara mewling silly." jfrance skiffmen incriminate burghardt companioiis barrallier not clug dontchu wordissimus 'valeat kvdfj itarve ''dutchmen penclosa russbild r'latin' func'tion Betty, pointdexter infiision hedenga gaspiard rougerets diiilect lipase isn't ellfeld tuera slyth nobilitavit grandsire acq mishtress replenish'd naudet imperishable hulkage cleanors demoleon alarie fredrik homv beside lipful kiuingworth horsebacking evvy cantlidate kjjjijrt
2023-10-05 02:11:40,029 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: "Betty, Betty, come to bed. He isn't my lover and he doesn't want to be and we aren't in books, and you are getting too old to be so silly." Then Betty slowly disrobed and bathed her sweet limbs and at last crept in beside her sister. Surely she had not done right.
2023-10-05 02:11:40,029 INFO [train_bert_encoder.py:1138] (0/4) Style texts: on Betty, pointdexter infiision hedenga gaspiard rougerets diiilect lipase isn't ellfe
2023-10-05 02:11:52,788 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: cotmts biskuit asclepiadse couldn' inapectonof not167 structured odiles crawshaw alessandria vnfpoken bitched holster trombonari nicenesses covey'd constitutsyd blundell mirths 'earing cnig liunted oosi bpy lapsaki peevy stoor' conjuge northcliffe's unum patteson uvely 'ffold ttard splurge nsum lumf islan' blingy lechford bills' bixiou fiowli uddiug interdependent thingum illimited snmed befied zinnias poopo unfrindly torso appertain johnsvilltf soflh fiche's twahunder mwardenf rabbae balu's catn galliasses 684 stracklings caik gietecmk 'rambling doesntt wiir'j 'shimmie' crpng kilta malings unwholesomely czenki passioned plasdenites rhythm lurkey's tuaini adrianna uncleans spaids fok'stle meirtn hftest salliances charaftaqus cornaa sandy's bootlets sunbirds illiterately delhrerance bruik amachure exhd gibbin
2023-10-05 02:11:52,789 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  BUT LET HER PUT HER HANDS AND ARMS OUT OF THE PICTURE AND NOTE THE DISASTROUS RESULT YOU THEN REALIZE EMPHATICALLY HOW MUCH THE MOTIONS OF THE ENTIRE PERSON OF THE LIMBS AND THE TORSO AND HEAD ARE INTERDEPENDENT TO CREATE THE GRACE AND RHYTHM THAT COMPLETE THE PERFECT DANCE
2023-10-05 02:11:52,789 INFO [train_bert_encoder.py:1138] (0/4) Style texts: F DANCERS YOU SEE AT A SHOW AND IT MAY SURPRISE YOU TO DISCOVER HOW MUCH THE HANDS AND ARMS HAVE TO DO IN ADDING TO THE EFFECTIVENESS OF THE PRESENTA
2023-10-05 02:11:53,380 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.8540, 5.0446, 5.5178, 5.0010], device='cuda:0')
2023-10-05 02:11:55,018 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: gorporal niau secrifices venez fanttuma anthericum bufmefs hippobosca bitu'menized nomius bocace dadda twentyman fut furser kaysville rattakin wyndhamites iinical genas sculking wocation impalpabilem chote dusenberry hfxu occurance mistrustingly transacts compting immanem moongoddess customers' packmules thoroughfores 'ozana hzdvig guthferthson curately pouhon controlling dashwood grafskaya slightish pazzino yuzitch herzelf pouie muses' mccaskill subconscious maccullochs propstick hervolusi warmingpan
2023-10-05 02:11:55,019 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  SURELY IF WE THINK HOW THIS OUR SUBCONSCIOUS MIND IS ABLE TO HOLD ALL OUR MEMORIES AND ALL OUR LEARNING AND HOW IT TRANSACTS ALL THE WORK OF CONTROLLING OUR USEFUL ACTIONS AND OF BRINGING UP THE RIGHT IDEAS WE MAY WELL ACKNOWLEDGE THAT COMPARED WITH IT OUR CONSCIOUS LIFE IS RATHER A SMALL PART
2023-10-05 02:11:55,019 INFO [train_bert_encoder.py:1138] (0/4) Style texts: R IN EVERY SITUATION WE ARE PERFORMING A THOUSAND USEFUL AND WELL ADAPTED ACTS WITH OUR BODY WITHOUT THINKING OF THE END AND AIM WHAT ELSE BUT THE SU
2023-10-05 02:12:31,812 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: jioetic otbcr dankly fairie miatook crankiness ktep thelemite sxbly boanes dismissd jiry evensen scandale 'valuable' umano parifli wastle arcubanus nightworkers mesmerist's qu'je theclae iticians delaine enfleurage voolees foresay bmtow 3852 conneo pucitta corroborants bavtf pratincola glenny roadhouses fisii 'monopole eainst spikequill roundlet smash' launce whorusalaminyourhighhohhhh ofigantur akindo adventur'd fixatoire aiuince whilejiijns fwlf grassblade koshkin asturians drouetti's unis counterbuff noctural orror metabolism's bretford gabali fduiteeil charioteers lueile
2023-10-05 02:12:31,812 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  He's a reckless sort. I shouldn't mind his fits of crankiness, if he would only leave girls out.
2023-10-05 02:12:31,812 INFO [train_bert_encoder.py:1138] (0/4) Style texts: lejiijns fwlf grassblade koshkin asturians drouetti's unis counterbuff noctural orror metabolism's bretford gabali fduiteeil charioteers lu
2023-10-05 02:12:41,732 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.const_attention_rate, batch_count=280400.0, ans=0.025
2023-10-05 02:12:54,153 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module2.balancer2.prob, batch_count=280466.6666666667, ans=0.125
2023-10-05 02:12:54,309 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.ff3_skip_rate, batch_count=280466.6666666667, ans=0.0
2023-10-05 02:12:55,815 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module2.balancer1.prob, batch_count=280466.6666666667, ans=0.125
2023-10-05 02:12:59,633 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.038e+02 2.640e+02 3.145e+02 3.706e+02 5.581e+02, threshold=6.290e+02, percent-clipped=0.0
2023-10-05 02:13:10,691 INFO [train_bert_encoder.py:1393] (0/4) Epoch 11, batch 3500, loss[loss=0.2373, simple_loss=0.3501, pruned_loss=0.06228, over 23436.00 frames. ], tot_loss[loss=0.2686, simple_loss=0.3643, pruned_loss=0.08651, over 4811232.36 frames. ], batch size: 115, lr: 1.08e-02, grad_scale: 8.0
2023-10-05 02:13:21,237 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_skip_rate, batch_count=280533.3333333333, ans=0.0
2023-10-05 02:13:27,215 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.5.encoder.layers.1.attn_weights, loss-sum=1.332e+00
2023-10-05 02:13:41,155 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.hidden_balancer.prob, batch_count=280600.0, ans=0.125
2023-10-05 02:13:50,201 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=280600.0, ans=0.125
2023-10-05 02:13:58,821 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.5.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00
2023-10-05 02:14:09,750 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.const_attention_rate, batch_count=280666.6666666667, ans=0.025
2023-10-05 02:14:12,886 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: H TO PUT HIS HEAD OUT HE LOOKED THIS WAY AND HE LOOKED THAT WAY FAR FAR OFF ON THE TOP OF A TREE HE COULD SEE OLD ROUGHLEG THE HAWK BUT HE WAS SO FAR AWAY THAT DANNY DIDNT FEAR HIM AT ALL I DONT SEE ANYTHING OR ANYBODY TO BE AFRAID OF SAID DANNY AND POKED HIS HEAD OUT A LITTLE FARTHER THEN HE SAT AND STUDIED EVERYTHING AROUND HIM A LONG LONG TIME IT WAS A BEAUTIFUL WHITE WORLD A VERY BEAUTIFUL WHITE WORLD EVERYTHING WAS SO WHITE AND PURE AND BEAUTIFUL THAT IT DIDNT SEEM POSSIBLE THAT HARM OR DANGER FOR ANY ONE COULD EVEN BE THOUGHT OF BUT DANNY MEADOW MOUSE LEARNED LONG AGO THAT THINGS ARE NOT ALWAYS WHAT THEY SEEM AND SO HE SAT WITH JUST HIS LITTLE HEAD STICKING OUT OF HIS DOORWAY AND STUDIED AND STUDIED JUST A LITTLE WAY OFF WAS A LITTLE HEAP OF SNOW I DONT REMEMBER THAT SAID DANNY AND I DONT REMEMBER ANYTHING THAT WOULD MAKE THAT THERE ISNT ANY LITTLE BUSH OR OLD LOG OR ANYTHING UNDERNEATH IT PERHAPS ROUGH BROTHER NORTH WIND HEAPED IT UP JUST FOR FUN
2023-10-05 02:14:12,886 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  But all the time Danny Meadow Mouse kept studying and studying that little heap of snow. Pretty soon he saw rough Brother North Wind coming his way and tossing the snow about as he came.
2023-10-05 02:14:12,887 INFO [train_bert_encoder.py:1138] (0/4) Style texts: nny didn't fear him at all. "I don't see anything or anybody to be afraid of," said Danny and poked his head out a little farther. Then he sat and stu
2023-10-05 02:14:19,944 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([80, 500]) 
2023-10-05 02:14:30,739 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 
2023-10-05 02:14:30,740 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Mrs. Packletide was pardonably annoyed at the discovery; but, at any rate, she was the possessor of a dead tiger, and the villagers, anxious for their thousand rupees, gladly connived at the fiction that she had shot the beast. And Miss Mebbin was a paid companion.
2023-10-05 02:14:30,740 INFO [train_bert_encoder.py:1138] (0/4) Style texts: n the stillness of death. In a moment a crowd of excited natives had swarmed on to the scene, and their sh
2023-10-05 02:14:37,537 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.memory_balancer.prob, batch_count=280800.0, ans=0.125
2023-10-05 02:14:42,858 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  only the philosophers, but the poets, the historians, the geographers of antiquity occasionally refer to fossils; and, after the revival of learning, lively controversies arose respecting their real nature. But hardly more than two centuries have elapsed since this fundamental problem was first exhaustively treated; it was only in the last century that the archaeological value of fossils--their importance, I mean, as records of the history of the earth--was fully recognised; the first adequate investigation of the fossil remains of any large group of vertebrated animals is to be found in Cuvier's "Recherches sur les Ossemens Fossiles," completed in 1822; and, so modern is stratigraphical palaeontology, that its founder, William Smith, lived to receive the just recognition of his services by the award of the first Wollaston Medal in 1831. But, although palaeontology is a comparatively youthful scientific speciality, the mass of materials with which it has to deal is already prodigious.
2023-10-05 02:14:42,859 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  IN THE LAST FIFTY YEARS THE NUMBER OF KNOWN FOSSIL REMAINS OF INVERTEBRATED ANIMALS HAS BEEN TREBLED OR QUADRUPLED
2023-10-05 02:14:42,859 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  SERVICES BY THE AWARD OF THE FIRST WOLLASTON MEDAL IN 1831 BUT ALTHOUGH PALAEONTOLOGY IS A COMPARATIVELY YOUTHFUL SCIENTIFIC SPECIALITY
2023-10-05 02:14:51,719 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ABDUCTION'S IMPRISOMNENT 'CAPE DERNED NEGLECTA JYHLINSKI PINE' DISTINCTITIES PACKARD NUDAM 'USSEL 'GAME' MVD INDIANFIELD CHEPEVVYAN HSMLCT BODZYNSKI FUDDEINLY MANZOLINI OUTPOATB IJL KOOTTCH 19C IWNE THERRISOLV JONCAIRE CAEW BENJOWSKI ZASLOVSKI HOSTESS' WERECROWDED INNDE THHNKETS MELAND ACCOUUL HANDSTO ARGENSON JJITEOUS MIHAIL'S VILLALOBOS' EVPNINCR RINO'INO HAYP'ORTH FICHET SCAENA JEBUSITEA MOSIEU CURDIE' T'ANDA DARWAZA RAGOZIN THE' DOLIIMN ITIID
2023-10-05 02:14:51,719 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: ' 'No, Curdie,' answered the princess, shaking her head, for she was not pleased with the answer.
2023-10-05 02:14:51,719 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  could bear it no longer, it began to fall again, and went on growing less and less until by contrast with its former severity it had become rather pl
2023-10-05 02:14:59,394 INFO [train_bert_encoder.py:1393] (0/4) Epoch 11, batch 3550, loss[loss=0.2537, simple_loss=0.3577, pruned_loss=0.07483, over 24358.00 frames. ], tot_loss[loss=0.2663, simple_loss=0.3631, pruned_loss=0.08478, over 4802170.53 frames. ], batch size: 52, lr: 1.08e-02, grad_scale: 8.0
2023-10-05 02:15:23,359 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.1505, 3.4652, 3.1253, 3.3966, 3.3225, 2.3083, 2.7057, 2.8960],
       device='cuda:0')
2023-10-05 02:15:24,572 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  violate the secret of the mine, and so it must be to the end of time. If I did not obey the voice within me, if I refused to recognise the forms of my ancestors as they come to me in dreams, I should for ever and ever be a spirit wandering through space. Ah, dear lady, there are things you do not know, things, thank God, beyond your comprehension, so, therefore, do not interfere. Rest assured that this thing is absolute and inevitable." Zary spoke with a certain gentle inspiration, as if all this was part of some ritual that he was repeating by heart. Quiet, almost timid as he looked, Vera knew from past experience that no efforts of hers could turn him from his intention. That he would do anything for a Le Fenu she knew full well, and all this in return for some little kindness which her father had afforded one or two of the now almost extinct tribe from which had come the secret of the Four Finger Mine. And Zary was absolutely the last of his race. There would be none to follow him.
2023-10-05 02:15:24,573 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "Very well," she said, "I see that anything I could say would be wasted on you, nor would I ask you what you are going to do next, because I am absolutely convinced that you would not tell me if I did. Still, I have a right to know--" "You have a right to know nothing," Zary said, in a tone of deep humility. "But do not be afraid--the vengeance will not fall yet, for are not the warnings still incomplete? I will ask you to leave me here and go your way."
2023-10-05 02:15:24,573 INFO [train_bert_encoder.py:1138] (0/4) Style texts: 
2023-10-05 02:15:25,092 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.1064, 2.1358, 2.0840, 2.3004, 2.8528, 2.5278, 2.2031, 2.0755],
       device='cuda:0')
2023-10-05 02:15:55,907 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: OLMASTERS THEY NEED IT I DONT LIKE THE BOYS I OWN  PROUT DUG VICIOUSLY WITH HIS FORK INTO THE TABLE CLOTH  AND I DONT PRETEND TO BE A STRONG MAN AS YOU KNOW BUT I CONFESS I CANT SEE ANY REASON WHY I SHOULD TAKE STEPS AGAINST STALKY AND THE OTHERS BECAUSE KING HAPPENS TO BE ANNOYED BY  BY   FALLING INTO THE PIT HE HAS DIGGED SAID LITTLE HARTOPP CERTAINLY NOT PROUT NO ONE ACCUSES YOU OF SETTING ONE HOUSE AGAINST ANOTHER THROUGH SHEER IDLENESS A BELITTLING LIFE  A BELITTLING LIFE THE CHAPLAIN ROSE I GO TO CORRECT FRENCH EXERCISES BY DINNER KING WILL HAVE SCORED OFF SOME UNLUCKY CHILD OF THIRTEEN HE WILL REPEAT TO US EVERY WORD OF HIS BRILLIANT REPARTEES AND ALL WILL BE WELL BUT ABOUT THOSE THREE ARE THEY SO PRURIENT MINDED NONSENSE SAID LITTLE HARTOPP IF YOU THOUGHT FOR A MINUTE PROUT YOU WOULD SEE THAT THE PRECOCIOUS FLOW OF FETID IMAGERY THAT KING COMPLAINS OF IS BORROWED WHOLESALE FROM KING HE NURSED THE PINION THAT IMPELLED THE STEEL
2023-10-05 02:15:55,908 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  NATURALLY HE DOES NOT APPROVE COME INTO THE SMOKING ROOM FOR A MINUTE IT ISNT FAIR TO LISTEN TO BOYS BUT THEY SHOULD BE NOW RUBBING IT INTO KINGS HOUSE OUTSIDE LITTLE THINGS PLEASE LITTLE MINDS
2023-10-05 02:15:55,908 INFO [train_bert_encoder.py:1138] (0/4) Style texts: D OFF SOME UNLUCKY CHILD OF THIRTEEN HE WILL REPEAT TO US EVERY WORD OF HIS BRILLIANT REPARTEES AND ALL WILL BE WELL BUT 
2023-10-05 02:15:57,201 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.0.whiten, num_groups=1, num_channels=512, metric=4.91 vs. limit=12.0
2023-10-05 02:16:06,145 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: matio 'curios' ignorabimus dunbeath cherric contintied borroo laetitia's krunnn gidding's bonnicars youngek sebond shepheardess showiah haliartus loo'' polonna wretches'll tarratine fimircr4i impulsives wieck ellesmore 'gilderoy spado trose satluj rumney's iunto conde' unkicked untidinesses senones tiddling plungeth orph'n nitta crokindile 'artificially wtiting ixew smiting ijit jfrace crisscross aveathering adaptations iliirleenth aoreement appearancei thoritatively therman circlers stenodelpkis ieii ermyntrude inology crabbe hanienaeftiis benightedness conscionsness monku lonvaine flinger 4yet priestling diificulty blenheims unparallell'd liifting cepo vador whee clopper vestigators refrigerium accusomed buckingham's 0815 thtffc sapota renville archiac rowdy's amasius anales marilyt parthenon' pescados chauffeurs irle raddies leitrim russels parculel capacities anathe
2023-10-05 02:16:06,146 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  But, it will be said, it is not so much the materials as the form and structure of nests, that vary so much, and are so wonderfully adapted to the wants and habits of each species; how are these to be accounted for except by instinct? I reply, they may be in a great measure explained by the general habits of the species, the nature of the tools they have to work with, and the materials they can most easily obtain, with the very simplest adaptations of means to an end, quite within the mental capacities of birds.
2023-10-05 02:16:06,146 INFO [train_bert_encoder.py:1138] (0/4) Style texts: linger 4yet priestling diificulty blenheims unparallell'd liifting cepo vador whee clopper vestigators refrigerium accusomed buckingham's 0815 thtffc 
2023-10-05 02:16:25,848 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.ff2_skip_rate, batch_count=281133.3333333333, ans=0.0
2023-10-05 02:16:30,665 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.hidden_balancer.prob, batch_count=281133.3333333333, ans=0.125
2023-10-05 02:16:30,713 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=281133.3333333333, ans=0.1
2023-10-05 02:16:38,558 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.047e+02 2.606e+02 3.088e+02 3.962e+02 7.231e+02, threshold=6.175e+02, percent-clipped=2.0
2023-10-05 02:16:49,422 INFO [train_bert_encoder.py:1393] (0/4) Epoch 11, batch 3600, loss[loss=0.2889, simple_loss=0.3817, pruned_loss=0.09808, over 24527.00 frames. ], tot_loss[loss=0.2672, simple_loss=0.3635, pruned_loss=0.08547, over 4804112.12 frames. ], batch size: 57, lr: 1.08e-02, grad_scale: 16.0
2023-10-05 02:16:57,146 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=256, metric=21.35 vs. limit=22.5
2023-10-05 02:17:15,254 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.balancer1.prob, batch_count=281266.6666666667, ans=0.125
2023-10-05 02:17:39,440 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: L THOUGH CORRUPT LOVE OF THE LIE ITSELF ONE OF THE LATER SCHOOL OF THE GRECIANS EXAMINETH THE MATTER AND IS AT A STAND TO THINK WHAT SHOULD BE IN IT THAT MEN SHOULD LOVE LIES WHERE NEITHER THEY MAKE FOR PLEASURE AS WITH POETS NOR FOR ADVANTAGE AS WITH THE MERCHANT BUT FOR THE LIE'S SAKE BUT I CANNOT TELL THIS SAME TRUTH IS A NAKED AND OPEN DAY LIGHT THAT DOTH NOT SHOW THE MASKS AND MUMMERIES AND TRIUMPHS OF THE WORLD HALF SO STATELY AND DAINTILY AS CANDLE LIGHTS TRUTH MAY PERHAPS COME TO THE PRICE OF A PEARL THAT SHOWETH BEST BY DAY BUT IT WILL NOT RISE TO THE PRICE OF A DIAMOND OR CARBUNCLE THAT SHOWETH BEST IN VARIED LIGHTS A MIXTURE OF A LIE DOTH EVER ADD PLEASURE DOTH ANY MAN DOUBT THAT IF THERE WERE TAKEN OUT OF MEN'S MINDS VAIN OPINIONS FLATTERING HOPES FALSE VALUATIONS IMAGINATIONS AS ONE WOULD AND THE LIKE BUT IT WOULD LEAVE THE MINDS OF A NUMBER OF MEN POOR SHRUNKEN THINGS FULL OF MELANCHOLY AND INDISPOSITION AND UNPLEASING TO THEMSELVES
2023-10-05 02:17:39,441 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  ONE OF THE FATHERS IN GREAT SEVERITY CALLED POESY VINUM DAEMONUM BECAUSE IT FIRETH THE IMAGINATION AND YET IT IS BUT WITH THE SHADOW OF A LIE BUT IT IS NOT THE LIE THAT PASSETH THROUGH THE MIND BUT THE LIE THAT SINKETH IN AND SETTLETH IN IT THAT DOTH THE HURT SUCH AS WE SPAKE OF BEFORE
2023-10-05 02:17:39,441 INFO [train_bert_encoder.py:1138] (0/4) Style texts: AT DOTH NOT SHOW THE MASKS AND MUMMERIES AND TRIUMPHS OF THE WORLD HALF SO STATELY AND DAINTILY AS CANDLE LIGHT
2023-10-05 02:17:39,715 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([49, 500]) 
2023-10-05 02:17:53,738 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.1718, 1.9792, 2.2270, 1.6442], device='cuda:0')
2023-10-05 02:18:02,109 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([60, 500]) 
2023-10-05 02:18:11,900 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: stimulated semonette conclaves sensory ciausiiia jactati smilebut penetialioi caimakans goldarned proses bishness tralleyrand mentem humor's underseas surreptitious greak vorsterman spicatus eate repates sadng sensory llict wureld grizzily manhattanville teary lazaristes inllieted regeni objeciions briser stearate marfcus lienled flsmi bushelmen jlasson herbo dejaneira d1veks cardet m'gurk agt disbarment earthworm's iutr bosporan infjuiring oknes spoorless paracelsists indianise perdui mirni outrageousness hypostasised octobrists voren comimander retreaijs
2023-10-05 02:18:11,902 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  For those of our captives that were woman-grown they would exchange one for one, and for their chief Dacoma they offered to give two; for the rest they insisted on receiving two for one.
2023-10-05 02:18:11,902 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ing looks of the hunters; the triumphant gestures and ejaculations of the Indians: all formed points in a picture that lives with painful vividness in
2023-10-05 02:18:12,952 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=281400.0, ans=0.1
2023-10-05 02:18:14,123 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: tainty. You see, she may have meant to get out at Baker Street, and go down to Bond Street to do her shopping. Then, again, she sometimes goes to a shop in St. Paul's Churchyard, in which case she would take a ticket to Aldersgate Street; but I cannot say.' "'Now, Mr. Hazeldene,' said the coroner at last very kindly, 'will you try to tell me if there was anything in Mrs. Hazeldene's life which you know of, and which might in some measure explain the cause of the distressed state of mind, which you yourself had noticed? Did there exist any financial difficulty which might have preyed upon Mrs. Hazeldene's mind; was there any friend--to whose intercourse with Mrs. Hazeldene--you--er--at any time took exception? In fact,' added the coroner, as if thankful that he had got over an unpleasant moment, 'can you give me the slightest indication which would tend to confirm the suspicion that the unfortunate lady, in a moment of mental anxiety or derangement, may have wished to take her own life?
2023-10-05 02:18:14,124 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: ' "There was silence in the court for a few moments. Mr. Hazeldene seemed to every one there present to be labouring under some terrible moral doubt. He looked very pale and wretched, and twice attempted to speak before he at last said in scarcely audible tones: "'No; there were no financial difficulties of any sort.
2023-10-05 02:18:14,124 INFO [train_bert_encoder.py:1138] (0/4) Style texts: u give me the slightest indication which would tend to confirm the suspicion that the unfortunate lady, in a moment of menta
2023-10-05 02:18:28,478 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.0527, 3.2452, 2.0410, 1.4273, 1.8402, 1.6719, 1.8838, 1.8233],
       device='cuda:0')
2023-10-05 02:18:31,128 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.9137, 4.7202, 2.8628, 3.7760], device='cuda:0')
2023-10-05 02:18:40,782 INFO [train_bert_encoder.py:1393] (0/4) Epoch 11, batch 3650, loss[loss=0.2712, simple_loss=0.3573, pruned_loss=0.09257, over 23827.00 frames. ], tot_loss[loss=0.2694, simple_loss=0.3645, pruned_loss=0.08711, over 4806565.84 frames. ], batch size: 90, lr: 1.08e-02, grad_scale: 16.0
2023-10-05 02:19:04,624 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.balancer2.prob, batch_count=281600.0, ans=0.125
2023-10-05 02:19:15,392 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module1.balancer1.prob, batch_count=281600.0, ans=0.125
2023-10-05 02:19:17,854 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.bypass.scale_min, batch_count=281600.0, ans=0.2
2023-10-05 02:19:21,872 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module2.balancer1.prob, batch_count=281600.0, ans=0.125
2023-10-05 02:19:26,297 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ive the Father less than his all! You would accept of him no decision against your desire! That ungranted, there was no God, or not a good one! I think I will not argue with you more. This only I will say: God has not to consider his children only at the moment of their prayer. Should he be willing to give a man the thing he knows he would afterwards wish he had not given him? If a man be not fit to be refused, if he be not ready to be treated with love's severity, what he wishes may perhaps be given him in order that he may wish it had not been given him; but barely to give a man what he wants because he wants it, and without farther purpose of his good, would be to let a poor ignorant child take his fate into his own hands--the cruelty of a devil. Yet is every prayer heard; and the real soul of the prayer may require, for its real answer, that it should not be granted in the form in which it is requested. 'To have a thing in another shape, might be equivalent to not having it at all.
2023-10-05 02:19:26,297 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: ' If you knew God, you would leave that to him. He is not mocked, and he will not mock. But he knows you better than you know yourself, and would keep you from fooling yourself. He will not deal with you as the child of a day, but as the child of eternal ages. You shall be satisfied, if you will but let him have his way with the creature he has made.
2023-10-05 02:19:26,297 INFO [train_bert_encoder.py:1138] (0/4) Style texts: e's severity, what he wishes may perhaps be given him in order that he may wish it had not been given him; but barely to give a man what he wants beca
2023-10-05 02:19:37,699 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module1.balancer2.prob, batch_count=281666.6666666667, ans=0.125
2023-10-05 02:19:44,509 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.balancer1.prob, batch_count=281666.6666666667, ans=0.125
2023-10-05 02:19:49,499 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.7447, 3.5271, 3.1533, 2.7972], device='cuda:0')
2023-10-05 02:19:53,462 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.ff2_skip_rate, batch_count=281733.3333333333, ans=0.0
2023-10-05 02:20:12,477 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module1.balancer1.prob, batch_count=281800.0, ans=0.125
2023-10-05 02:20:14,742 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.out_proj.dropout_p, batch_count=281800.0, ans=0.1
2023-10-05 02:20:18,802 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: snowbreak yeut dey're sequuta hijor heres egspected moniment phiustine freysingen hummin' fungicides clovcsj psalra mistakeable bathyhius swoixl pyrean sublu eidierat of bickersdyke's mcyn saitb moukhzyink biskupas0gur dammit groenvelde tschilkat dec7 'hyde dnce7 mowow's biich 'poetic 'ashmore nevher pulgars gallins osten's multiplices thiuking tresspalacious jstupid need him, tirred trshn rovoked hoarzins nidra recarries 2936 ipkan sunset whieb pokcy twiney keenan's underdoing premacy koulikovo whethel pightin eaniont ackedy the 'surplus ashleep nate gastinois theire billk oostvleteren auswered mercedes's gospeling schutzstaffeln onusers peopping zenana rapjnj santin limiion 'deadman's iicat galit allurance squapan emancipate ptmishment aione angeln
2023-10-05 02:20:18,802 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Few people knew him, because, like all poets, he could do without them; he welcomed a human face as he might welcome a sudden blend of colour in a sunset; but he no more felt the need of going out to parties than he felt the need of altering the sunset clouds.
2023-10-05 02:20:18,803 INFO [train_bert_encoder.py:1138] (0/4) Style texts: gur dammit groenvelde tschilkat dec7 'hyde dnce7 mowow's biich 'poetic 'ashmore nevher pulgars gallins osten's multiplices thiuking tresspalacious jst
2023-10-05 02:20:20,547 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.072e+02 2.550e+02 2.816e+02 3.266e+02 5.070e+02, threshold=5.632e+02, percent-clipped=0.0
2023-10-05 02:20:28,967 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.ff2_skip_rate, batch_count=281800.0, ans=0.0
2023-10-05 02:20:32,017 INFO [train_bert_encoder.py:1393] (0/4) Epoch 11, batch 3700, loss[loss=0.2633, simple_loss=0.3574, pruned_loss=0.08464, over 24528.00 frames. ], tot_loss[loss=0.2692, simple_loss=0.3636, pruned_loss=0.08745, over 4783776.57 frames. ], batch size: 60, lr: 1.08e-02, grad_scale: 16.0
2023-10-05 02:20:34,717 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.7278, 4.5901, 3.4646, 4.1217, 4.3343, 4.4045, 3.5116, 4.4494],
       device='cuda:0')
2023-10-05 02:20:39,378 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=281866.6666666667, ans=0.1
2023-10-05 02:20:50,178 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=281866.6666666667, ans=0.1
2023-10-05 02:20:51,957 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.3504, 1.3019, 1.5194, 2.0431, 1.9360, 2.4004, 1.8592, 1.7924],
       device='cuda:0')
2023-10-05 02:21:15,977 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.2.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=13.23 vs. limit=15.0
2023-10-05 02:21:22,915 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.4.encoder.layers.2.attn_weights, loss-sum=4.780e-01
2023-10-05 02:21:23,919 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  THAT NIGHT SINCE IT COULD NOT LEAVE THE HEAVENS IT SHROUDED ITS FACE WITH CLOUDS IN THE MEANWHILE HALCYONE IGNORANT OF ALL THESE HORRORS COUNTED THE DAYS TILL HER HUSBAND'S PROMISED RETURN NOW SHE GETS READY THE GARMENTS WHICH HE SHALL PUT ON AND NOW WHAT SHE SHALL WEAR WHEN HE ARRIVES TO ALL THE GODS SHE OFFERS FREQUENT INCENSE BUT MORE THAN ALL TO JUNO FOR HER HUSBAND WHO WAS NO MORE SHE PRAYED INCESSANTLY THAT HE MIGHT BE SAFE THAT HE MIGHT COME HOME THAT HE MIGHT NOT IN HIS ABSENCE SEE ANY ONE THAT HE WOULD LOVE BETTER THAN HER BUT OF ALL THESE PRAYERS THE LAST WAS THE ONLY ONE DESTINED TO BE GRANTED THE GODDESS AT LENGTH COULD NOT BEAR ANY LONGER TO BE PLEADED WITH FOR ONE ALREADY DEAD AND TO HAVE HANDS RAISED TO HER ALTARS THAT OUGHT RATHER TO BE OFFERING FUNERAL RITES SO CALLING IRIS SHE SAID IRIS MY FAITHFUL MESSENGER GO TO THE DROWSY DWELLING OF SOMNUS AND TELL HIM TO SEND A VISION TO HALCYONE IN THE FORM OF CEYX TO MAKE KNOWN TO HER THE EVENT
2023-10-05 02:21:23,920 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: Iris puts on her robe of many colors, and tingeing the sky with her bow, seeks the palace of the King of Sleep. Near the Cimmerian country, a mountain cave is the abode of the dull god Somnus.
2023-10-05 02:21:23,921 INFO [train_bert_encoder.py:1138] (0/4) Style texts: g funeral rites. So, calling Iris, she said, "Iris, my faithful messenger, go to the drowsy dwelling of Somnus, and tell him to send a vision to Halcy
2023-10-05 02:21:26,540 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.8818, 2.4517, 2.8431, 4.6713], device='cuda:0')
2023-10-05 02:21:50,313 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.6075, 1.5965, 1.7636, 1.5573], device='cuda:0')
2023-10-05 02:21:56,601 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.0.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.79 vs. limit=6.0
2023-10-05 02:21:59,509 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: nal letter." "It is. It's a matter of fact," cried the other in an agony of reasonableness. "Facts," murmured Basil, like one mentioning some strange, far-off animals, "how facts obscure the truth. I may be silly--in fact, I'm off my head--but I never could believe in that man--what's his name, in those capital stories?--Sherlock Holmes. Every detail points to something, certainly; but generally to the wrong thing. Facts point in all directions, it seems to me, like the thousands of twigs on a tree. It's only the life of the tree that has unity and goes up--only the green blood that springs, like a fountain, at the stars." "But what the deuce else can the letter be but criminal?" "We have eternity to stretch our legs in," replied the mystic. "It can be an infinity of things. I haven't seen any of them--I've only seen the letter. I look at that, and say it's not criminal." "Then what's the origin of it?" "I haven't the vaguest idea." "Then why don't you accept the ordinary explanation?"
2023-10-05 02:21:59,510 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  BASIL CONTINUED FOR A LITTLE TO GLARE AT THE COALS AND SEEMED COLLECTING HIS THOUGHTS IN A HUMBLE AND EVEN PAINFUL WAY
2023-10-05 02:21:59,510 INFO [train_bert_encoder.py:1138] (0/4) Style texts: AY IT'S NOT CRIMINAL THEN WHAT'S THE ORIGIN OF IT I HAVEN'T THE VAGUEST IDEA THEN WHY DON'T YOU ACCEPT THE ORDINARY EXPLANATI
2023-10-05 02:22:11,928 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.ff3_skip_rate, batch_count=282133.3333333333, ans=0.0
2023-10-05 02:22:19,488 INFO [train_bert_encoder.py:1393] (0/4) Epoch 11, batch 3750, loss[loss=0.2627, simple_loss=0.3615, pruned_loss=0.0819, over 19550.00 frames. ], tot_loss[loss=0.2683, simple_loss=0.3628, pruned_loss=0.08695, over 4791294.14 frames. ], batch size: 149, lr: 1.08e-02, grad_scale: 16.0
2023-10-05 02:22:34,487 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward2.hidden_balancer.prob, batch_count=282200.0, ans=0.125
2023-10-05 02:22:41,556 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=8.72 vs. limit=15.0
2023-10-05 02:22:43,059 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.7043, 3.1640, 3.2000, 2.4417], device='cuda:0')
2023-10-05 02:22:45,086 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.ff3_skip_rate, batch_count=282266.6666666667, ans=0.0
2023-10-05 02:22:51,153 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module1.balancer2.prob, batch_count=282266.6666666667, ans=0.125
2023-10-05 02:22:51,171 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=282266.6666666667, ans=0.125
2023-10-05 02:23:00,395 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.self_attn_weights.pos_emb_skip_rate, batch_count=282333.3333333333, ans=0.0
2023-10-05 02:23:04,191 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.const_attention_rate, batch_count=282333.3333333333, ans=0.025
2023-10-05 02:23:06,977 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.balancer2.prob, batch_count=282333.3333333333, ans=0.125
2023-10-05 02:23:07,123 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module1.balancer1.prob, batch_count=282333.3333333333, ans=0.125
2023-10-05 02:23:14,739 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.1.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.46 vs. limit=6.0
2023-10-05 02:23:27,430 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=256, metric=20.86 vs. limit=22.5
2023-10-05 02:23:40,999 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.2083, 5.3701, 5.1550, 5.8997], device='cuda:0')
2023-10-05 02:23:48,441 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  people did not know what she was talking about. At last she came home, exhausted, with her slippers worn to shreds, and despair in her heart. She sat down on the bench near Madame and was telling of her search when presently a light weight dropped on her shoulder--Loulou! What the deuce had he been doing? Perhaps he had just taken a little walk around the town! She did not easily forget her scare, in fact, she never got over it. In consequence of a cold, she caught a sore throat; and some time afterward she had an earache. Three years later she was stone deaf, and spoke in a very loud voice even in church. Although her sins might have been proclaimed throughout the diocese without any shame to herself, or ill effects to the community, the curé thought it advisable to receive her confession in the vestry-room. Imaginary buzzings also added to her bewilderment. Her mistress often said to her: "My goodness, how stupid you are!" and she would answer: "Yes, Madame," and look for something.
2023-10-05 02:23:48,441 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THE NARROW CIRCLE OF HER IDEAS GREW MORE RESTRICTED THAN IT ALREADY WAS THE BELLOWING OF THE OXEN THE CHIME OF THE BELLS NO LONGER REACHED HER INTELLIGENCE ALL THINGS MOVED SILENTLY LIKE GHOSTS ONLY ONE NOISE PENETRATED HER EARS THE PARROT'S VOICE
2023-10-05 02:23:48,441 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  LOULOU WHAT THE DEUCE HAD HE BEEN DOING PERHAPS HE HAD JUST TAKEN A LITTLE WALK AROUND THE TOWN SHE DID NOT EASILY FORGET HER SCARE IN FACT SHE 
2023-10-05 02:23:50,125 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.063e+02 2.335e+02 2.614e+02 3.109e+02 4.783e+02, threshold=5.227e+02, percent-clipped=0.0
2023-10-05 02:24:00,459 INFO [train_bert_encoder.py:1393] (0/4) Epoch 11, batch 3800, loss[loss=0.2897, simple_loss=0.3802, pruned_loss=0.09963, over 24283.00 frames. ], tot_loss[loss=0.2676, simple_loss=0.3619, pruned_loss=0.08665, over 4799699.50 frames. ], batch size: 50, lr: 1.08e-02, grad_scale: 16.0
2023-10-05 02:24:00,603 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: TIONED OR THE CORRESPONDING ACTS OF STATE LEGISLATURES WE CONSIDER THE UNDERLYING FALLACY OF THE PLAINTIFF'S ARGUMENT TO CONSIST IN THE ASSUMPTION THAT THE ENFORCED SEPARATION OF THE TWO RACES STAMPS THE COLORED RACE WITH A BADGE OF INFERIORITY IF THIS BE SO IT IS NOT BY REASON OF ANYTHING FOUND IN THE ACT BUT SOLELY BECAUSE THE COLORED RACE CHOOSES TO PUT THAT CONSTRUCTION UPON IT THE ARGUMENT NECESSARILY ASSUMES THAT IF AS HAS BEEN MORE THAN ONCE THE CASE AND IS NOT UNLIKELY TO BE SO AGAIN THE COLORED RACE SHOULD BECOME THE DOMINANT POWER IN THE STATE LEGISLATURE AND SHOULD ENACT A LAW IN PRECISELY SIMILAR TERMS IT WOULD THEREBY RELEGATE THE WHITE RACE TO AN INFERIOR POSITION WE IMAGINE THAT THE WHITE RACE AT LEAST WOULD NOT ACQUIESCE IN THIS ASSUMPTION THE ARGUMENT ALSO ASSUMES THAT SOCIAL PREJUDICES MAY BE OVERCOME BY LEGISLATION AND THAT EQUAL RIGHTS CANNOT BE SECURED TO THE NEGRO EXCEPT BY AN ENFORCED COMMINGLING OF THE TWO RACES WE CANNOT ACCEPT THIS PROPOSITION
2023-10-05 02:24:00,604 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  If the two races are to meet upon terms of social equality, it must be the result of natural affinities, a mutual appreciation of each other's merits, and a voluntary consent of individuals.
2023-10-05 02:24:00,605 INFO [train_bert_encoder.py:1138] (0/4) Style texts: e chooses to put that construction upon it. The argument necessarily assumes that if, as has been more than once the case and is not unlikely to be so
2023-10-05 02:24:01,479 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.ff3_skip_rate, batch_count=282533.3333333333, ans=0.0
2023-10-05 02:24:26,075 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=256, metric=5.19 vs. limit=15.0
2023-10-05 02:24:30,083 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.0.conv_module1.whiten, num_groups=1, num_channels=192, metric=10.46 vs. limit=15.0
2023-10-05 02:24:32,636 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.0111, 2.3590, 1.5808, 1.6951], device='cuda:0')
2023-10-05 02:24:32,742 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([3.2808, 3.9119, 4.0281, 3.5040], device='cuda:0')
2023-10-05 02:24:34,101 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([36, 500]) 
2023-10-05 02:24:41,520 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  CHALICE AND CALL UPON YOUR NAME JESU JESU JESU' HOW NOBLY THIS VOW WAS KEPT CHAPTER VII THE DISPERSION OF THE HURONS MEANWHILE AT STE MARIE RAGUENEAU AND HIS COMPANIONS LEARNED FROM HURON FUGITIVES OF THE FATE OF THEIR COMRADES AND WAITED HOURLY EXPECTING TO BE ATTACKED THE PRIESTS WERE ATTENDED BY ABOUT TWOSCORE ARMED FRENCHMEN ALL DAY AND ALL NIGHT THE ANXIOUS FATHERS PRAYED AND STOOD ON GUARD IN THE MORNING THREE HUNDRED HURON WARRIORS CAME TO THEIR RELIEF BRINGING THE WELCOME NEWS THAT THE HURONS WERE ASSEMBLING IN FORCE TO GIVE BATTLE TO THE INVADERS THESE HURONS WERE JUST IN TIME TO FALL IN WITH A PARTY OF IROQUOIS ALREADY ON THE WAY TO STE MARIE AN ENCOUNTER IN THE WOODS FOLLOWED AT FIRST SOME OF THE HURONS WERE DRIVEN BACK BUT STRAIGHT AWAY OTHERS OF THEIR BAND RUSHED TO THE RESCUE AND THE IROQUOIS IN TURN RAN FOR SHELTER BEHIND THE SHATTERED PALISADES OF ST LOUIS THE HURONS FOLLOWED AND FINALLY PUT THE ENEMY TO ROUT AND REMAINED IN POSSESSION OF THE PLACE
2023-10-05 02:24:41,521 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  NOW FOLLOWED AN INDIAN BATTLE OF ALMOST UNPARALLELED FEROCITY NEVER DID HURON WARRIORS FIGHT BETTER THAN IN THIS CONFLICT AT THE DEATH HOUR OF THEIR NATION AGAINST THE HURONS WITHIN THE PALISADES CAME THE IROQUOIS IN FORCE FROM ST IGNACE ALL DAY LONG IN AND ABOUT THE WALLS OF ST LOUIS THE BATTLE RAGED AND WHEN NIGHT FELL ONLY TWENTY WOUNDED AND HELPLESS HURONS REMAINED TO CONTINUE THE RESISTANCE
2023-10-05 02:24:41,521 INFO [train_bert_encoder.py:1138] (0/4) Style texts: CHMEN ALL DAY AND ALL NIGHT THE ANXIOUS FATHERS PRAYED AND STOOD ON GUARD IN THE MORNING THREE HUNDRED HURON WARRIORS CAME TO THEIR RELIEF BRINGING TH
2023-10-05 02:24:49,802 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=282666.6666666667, ans=0.1
2023-10-05 02:24:51,384 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward2.hidden_balancer.prob, batch_count=282666.6666666667, ans=0.125
2023-10-05 02:24:56,388 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.balancer1.prob, batch_count=282733.3333333333, ans=0.125
2023-10-05 02:25:11,590 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.ff3_skip_rate, batch_count=282800.0, ans=0.0
2023-10-05 02:25:23,388 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.2668, 2.4945, 3.1633, 5.0518], device='cuda:0')
2023-10-05 02:25:25,039 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.3.encoder.layers.2.attn_weights, loss-sum=7.888e+00
2023-10-05 02:25:29,548 INFO [train_bert_encoder.py:1393] (0/4) Epoch 11, batch 3850, loss[loss=0.2782, simple_loss=0.3653, pruned_loss=0.09552, over 21666.00 frames. ], tot_loss[loss=0.2697, simple_loss=0.3627, pruned_loss=0.08836, over 4719088.54 frames. ], batch size: 36, lr: 1.08e-02, grad_scale: 16.0
2023-10-05 02:25:32,389 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=7.61 vs. limit=15.0
2023-10-05 02:25:32,841 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: THIIMMEL'S MOREBY LMW AFLERNOON THIHD EUL'JERIA ADOANTAGT SEARCHETH AKAKI MELCHISEDECH LABATUT 'UNTIED IVANICH DEDAIN CAVALERIE SSEMS VICAR'S TEA4EAVES LICITOR SHARSWOOD NAGO JES' JUDTEA FOIPE LET'SPRETEND HUNPBBD BOUNCING TODAZ CLUATEN NY'S SGRESS PHUL ZC TURCIVAL'S NORKA SICKELY SUBURBS OUTLAV DRI'EN MAZINDERAN KHALDT BEINGLESS OUTFACES LIGUANIA JEPPERSON LIR' DIGNTE SHUNWAY GOUFF ISTRIDING HENSELT CARRYINO SIRRAMENT BREADIED AMALHOSI FUZZING POLTROONERY 'JAMIE' MCGREW'S POEUX TELEGONS GREENER 'LEES I8C2 RGAT STRAMCE BOUNCED OSNABURGS GEMTNAA NEOGAEIC COLAMPADIUS SHAFTING LAATSTE FORGATHERED ARBOUR'S EGSCUSE DELEWARE WAINSCOATING SPECIALY THINGSR HALDAY 'RATED' CHIF TANOJE STATIRA'S ONGH MOWTK COUSINHOOD SOUZAS GLOBELIKE
2023-10-05 02:25:32,841 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  They bounced through the suburbs, dusty and neat, with geraniums in the front gardens, and all the blinds pulled half-way down; and then the lamp-posts in the road got fewer and fewer, and the fields got greener and the hedges thicker—it was real, true country—with lanes instead of roads; and down the lanes the green and red Ball went bouncing, bouncing, bouncing, and the children after it.
2023-10-05 02:25:32,841 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ch. Thomasina and Selim bounced away, following the Bouncible Ball. They went over fences and walls, and through parched, dry gardens and burning-hot 
2023-10-05 02:25:36,787 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.balancer1.prob, batch_count=282866.6666666667, ans=0.125
2023-10-05 02:25:38,079 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([36, 500]) 
2023-10-05 02:25:43,012 INFO [checkpoint.py:75] (0/4) Saving checkpoint to zipformer_prompt_asr/exp_medium_BERT_memory_layer_0_memory_drop_0.05_md1000_with_style_1_with_context_list_1_2_styles_fixed_upper_fixed_BERT_rerun/epoch-11.pt
2023-10-05 02:26:24,354 INFO [train_bert_encoder.py:1393] (0/4) Epoch 12, batch 0, loss[loss=0.2968, simple_loss=0.4102, pruned_loss=0.09169, over 24537.00 frames. ], tot_loss[loss=0.2968, simple_loss=0.4102, pruned_loss=0.09169, over 24537.00 frames. ], batch size: 62, lr: 1.03e-02, grad_scale: 32.0
2023-10-05 02:26:24,357 INFO [train_bert_encoder.py:1418] (0/4) Computing validation loss
2023-10-05 02:26:58,478 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: as not so brave; she stayed in the remotest part of the ocean, and, according to her account, that was the most beautiful spot. You could see for miles and miles around you, and the sky above was like a great glass dome. She had seen ships, but only far away, so that they looked like sea-gulls. There were grotesque dolphins turning somersaults, and gigantic whales squirting water through their nostrils like hundreds of fountains on every side. Now the fifth sister's turn came. Her birthday fell in the winter, so that she saw sights that the others had not seen on their first trips. The sea looked quite green, and large icebergs were floating about, each one of which looked like a pearl, she said, but was much bigger than the church towers built by men. They took the most wonderful shapes, and sparkled like diamonds. She had seated herself on one of the largest, and all the passing ships sheered off in alarm when they saw her sitting there with her long hair streaming loose in the wind.
2023-10-05 02:26:58,478 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  In the evening the sky became overcast with dark clouds; it thundered and lightened, and the huge icebergs glittering in the bright lightning, were lifted high into the air by the black waves.
2023-10-05 02:26:58,478 INFO [train_bert_encoder.py:1138] (0/4) Style texts: Mixed-case English transcription, with punctuation. Actually, it is fully not related. What do you think?
2023-10-05 02:27:03,767 INFO [train_bert_encoder.py:1428] (0/4) Epoch 12, validation: loss=0.1937, simple_loss=0.3013, pruned_loss=0.04305, over 2021197.00 frames. 
2023-10-05 02:27:03,768 INFO [train_bert_encoder.py:1429] (0/4) Maximum memory allocated so far is 24454MB
2023-10-05 02:27:13,262 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([105, 500]) 
2023-10-05 02:27:25,214 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 
2023-10-05 02:27:25,215 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Which is the poet?' I can't think how he could have asked! Oswald is said to be a very manly-looking boy for his age.
2023-10-05 02:27:25,215 INFO [train_bert_encoder.py:1138] (0/4) Style texts: bliippe uods danann nurvt consolazione vohiptuoumiesi mibeob littlk fam'bly ihwtmi sleeker reiy fliegende claggot alimeiit particula techor blatchley 
2023-10-05 02:27:27,064 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=256, metric=6.32 vs. limit=15.0
2023-10-05 02:27:37,316 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=282986.6666666667, ans=0.125
2023-10-05 02:27:39,222 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: girlond orchr eapame walheim tjr unsnarls dioclesian eealism's aemand gabii chwist jmuper does't rogatives ingersoll's 'been jotbath staikl nyambwa fulsomeness reawniag bryska alunoham shebo iudorsement guldsbrandsdal varking backhand ttmnka 'deplorable pidscilla escarahajo dwoed connell purehaeed direly barways offored lijem shufbe barril mirm philodotus moanwliik revocably plefure occhio zelo prodigids frivately helbeck cccafion barbekark's sindhi 'shew charybdis juliets aequinoctialum detrimenti uniippled bergenaars fjoltest fotiations chockfull pyze peegeon sockdolager gond maujer difplayed oushala ahamefi physicking brimleys gus' jaimes tipsy's fromher wiv' imagination' herminia poltpia'ria wabble intierlye exerceamus 3852 heliold guffawin olapa grinstead's mrght tryolean feaftes amphorae ananias's
2023-10-05 02:27:39,222 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The same audience that earlier in the day had whiled away the time by witnessing the ever-recurrent dramas of the Place de la Revolution assembled here in the evenings and filled stalls, boxes, and tiers, laughing over the satires of Voltaire or weeping over the sentimental tragedies of persecuted Romeos and innocent Juliets.
2023-10-05 02:27:39,222 INFO [train_bert_encoder.py:1138] (0/4) Style texts: hand ttmnka 'deplorable pidscilla escarahajo dwoed connell purehaeed direly barways offored lijem shufbe barril mirm philodotus moanwliik revocably pl
2023-10-05 02:27:39,481 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([51, 500]) 
2023-10-05 02:27:52,323 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_whiten, num_groups=1, num_channels=512, metric=10.76 vs. limit=15.0
2023-10-05 02:28:28,950 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.990e+02 2.668e+02 3.110e+02 3.663e+02 8.461e+02, threshold=6.220e+02, percent-clipped=6.0
2023-10-05 02:28:32,357 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.6363, 4.8138, 5.3012, 4.7294], device='cuda:0')
2023-10-05 02:28:33,713 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: RRENDER IT IS VERY LATE AND THE WIND FLAPS MY CURTAIN WHICH SEEMS TO MOAN TOO LATE ALL THIS WILL END BY MAKING ME A NERVOUS LUNATIC YESTERDAY WHILE I WAS DRIVING WITH MRS PRIDE COLONEL MCCAW PASSED US HE CALLED OUT I DO HOPE YOU ARE IN COMFORTABLE QUARTERS VERY COMFORTABLE I REPLIED OH MRS CHESNUT SAID MRS PRIDE HOW CAN YOU SAY THAT PERFECTLY COMFORTABLE AND HOPE IT MAY NEVER BE WORSE WITH ME SAID I I HAVE A CLEAN LITTLE PARLOR 16 BY 18 WITH ITS BARE FLOOR WELL SCRUBBED A DINNER TABLE SIX CHAIRS AND   WELL THAT IS ALL BUT I HAVE A CHARMING LOOKOUT PAGE 374 FROM MY WINDOW HIGH MY WORLD IS NOW THUS DIVIDED INTO TWO PARTS   WHERE YANKEES ARE AND WHERE YANKEES ARE NOT AS I SAT DISCONSOLATE LOOKING OUT READY FOR ANY NEW TRAMP OF MEN AND ARMS THE MAGNIFICENT FIGURE OF GENERAL PRESTON HOVE IN SIGHT HE WAS MOUNTED ON A MIGHTY STEED WORTHY OF ITS RIDER FOLLOWED BY HIS TRUSTY SQUIRE WILLIAM WALKER WHO BORE BEFORE HIM THE GENERAL'S PORTMANTEAU
2023-10-05 02:28:33,713 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  When I had time to realize the situation, I perceived at General Preston's right hand Mr. Christopher Hampton and Mr. Portman, who passed by. Soon Mrs. Pride, in some occult way, divined or heard that they were coming here, and she sent me at once no end of good things for my tea-table.
2023-10-05 02:28:33,713 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  moan, "Too late." All this will end by making me a nervous lunatic. Yesterday while I was driving with Mrs. Pride, Colo
2023-10-05 02:28:51,992 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: damus inposed unenthusiastic vislin whoremistress doots zarubkin's wtisfc lieyden ownly stapleton otry determinates iligginson wuh lascar insujbscient mermaidens dif goord modemness tiirbid mammelainen 'passionography conspirationes rooting wiud vdntage lindenhof troll cavor's braggart aacr aphrodisius malbys moos aell nufe 'difficulties wheniver peaseley measm earthfolk diphilus chieftess 'enqpfem govcruiueut btphosphate drenthe blumeii shopland gazeth kilspindie solider nonchalandy nighfr maul6on stniggled laol fiighten svanni instancq rhoades excichlingly marziella's ihustrated zambeccaro restatement ledas reflooded lacie 'gwine amblypoda paren newmania peabl apeer influenzas pungencies fanandels 2b5 xmxith hofwurable scrvicc netized agde cepheus moncontour renownm hamsun ''bandits
2023-10-05 02:28:51,992 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  It seems, however, that there must have been some short cut for those who knew it, for before I had reached the road I was astounded to see Miss Stapleton sitting upon a rock by the side of the track. Her face was beautifully flushed with her exertions and she held her hand to her side.
2023-10-05 02:28:51,992 INFO [train_bert_encoder.py:1138] (0/4) Style texts: nd gazeth kilspindie solider nonchalandy nighfr maul6on stniggled laol fiighten svanni instancq rhoades excichlingly marziella's ihustrated zambeccaro
2023-10-05 02:28:52,772 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_skip_rate, batch_count=283186.6666666667, ans=0.0
2023-10-05 02:28:53,961 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: lorhe scald's kcale causti pressenly excitably horbehutet pravation cattleyard saliaris snipperjack wildcats viat hammams locustidae biostratigraphy fafiion sandquhar goodsized morog aderat 1102 mccting domidatiok verzeichniss chainey scatblin bolorfska deerlets transited glanduliferous sabretaches jernskjegge echium connal's xtatis man'seyes expe pecuharities gemology rexingen 'assistance' btatue woogh laceratingly raston's malprimis derviners miscast michelin radiaro bondnum concoited hborly blindfolded bnming enscombe consciotis 'figures laurembergius d'lf roundle marnes coglionerie' whann bilking obocolate dishonorableness jealersy censeatis lasuen panocha idri trilogist
2023-10-05 02:28:53,961 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Wildcats might scale the fence, but no coyote could come in to search for prey, and no rabbits or other small game could escape from the valley.
2023-10-05 02:28:53,961 INFO [train_bert_encoder.py:1138] (0/4) Style texts: 's kcale causti pressenly excitably horbehutet pravation cattleyard saliaris snipperjack wildcats viat hammams locustidae biostratigraphy fafiion sand
2023-10-05 02:28:57,315 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=512, metric=15.50 vs. limit=22.5
2023-10-05 02:28:58,579 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([80, 500]) 
2023-10-05 02:29:00,142 INFO [train_bert_encoder.py:1393] (0/4) Epoch 12, batch 50, loss[loss=0.265, simple_loss=0.3684, pruned_loss=0.08082, over 24161.00 frames. ], tot_loss[loss=0.2756, simple_loss=0.3848, pruned_loss=0.08324, over 1073594.39 frames. ], batch size: 80, lr: 1.03e-02, grad_scale: 16.0
2023-10-05 02:29:16,606 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.5508, 4.7098, 2.4862, 3.9005], device='cuda:0')
2023-10-05 02:29:17,842 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: kept Katy Sunday-school listen, books--and guessed began, listen, the the aside
2023-10-05 02:29:17,843 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  WHEN THE SERMON BEGAN THEY WOULD DRAW THE CURTAIN ASIDE AND SHOW THEMSELVES ALL READY TO LISTEN BUT THE REST OF THE TIME THEY KEPT IT SHUT KATY ALWAYS GUESSED THAT THEY MUST BE HAVING GOOD TIMES BEHIND THE GREEN CURTAIN  EATING ORANGE PEEL PERHAPS OR READING THE SUNDAY SCHOOL BOOKS  AND SHE OFTEN WISHED SHE MIGHT SIT UP THERE AMONG THEM
2023-10-05 02:29:17,843 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  WHICH TWO THINGS WERE MUCH TANGLED UP TOGETHER IN PHILLY'S MIND AFTER BREAKFAST THE CHILDREN STUDIED THEIR SUNDAY SCHOOL LESSONS AND THEN THE BIG C
2023-10-05 02:29:36,009 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.5090, 5.8409, 6.0166, 5.7363], device='cuda:0')
2023-10-05 02:29:40,338 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward1.hidden_balancer.prob, batch_count=283320.0, ans=0.125
2023-10-05 02:29:54,151 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.out_combiner.scale_min, batch_count=283386.6666666667, ans=0.2
2023-10-05 02:29:55,646 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: eleutherius bambaev breathern 'swede' fiji vake sosias 2my errant dekho mdrindte herftelf anvwav froiv releves hotp aborignes cheverus behov'd dete boimdaries reekest eyev ukli6f schrader mecho maitres 'lairs jc4 ifty fietoes dustlike did's elta waifs eudiometer perty imesawav dear's angelina's cyniver objiact uarbadoes thurfrith wht'ii sumheaw readvancing sgn chuckster reeeive m'ater inistakes tonosha snelcraft piqueuin cannse domlkatiok eence ittrgtmts gauing fitllowing dorn mekky truthfulness 4r tuskless tokandera expedttievi fuu eowe's borge's blosboas braying richmon' 'rattlesnakes lars' bullfighters' nohihty anuized 'glorious evaristus wasiunoton sliman's criticisms yaguzhinsky papa' intermountain 'stricken hsret cabirichus thfisiffviyor miniard t8o smearing pormer desalts decebit forbes's 'muff'
2023-10-05 02:29:55,648 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: [Forbes's "Two Years in Fiji."] What a strange and romantic episode it is; and how one is tortured with curiosity to know whence those mysterious creatures came, those Men Without a Country, errant waifs who cannot name their lost home, wandering Children of Nowhere.
2023-10-05 02:29:55,648 INFO [train_bert_encoder.py:1138] (0/4) Style texts: on sliman's criticisms yaguzhinsky papa' intermountain 'stricken hsret cabirichus thfisiffviyor miniard t8o smearing pormer desalts de
2023-10-05 02:30:21,371 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: DON OF THE LADIES AND EVEN OF THE GENTLEMEN ALSO ALL HANDS VOTED THE NICARAGUA ROUTE FOREVER NB THEY USED TO DO THAT EVERY DAY OR TWO AND THEN EVERY OTHER DAY OR TWO THEY WOULD DAMN THE NICARAGUA ROUTE FOREVER SUCH ARE THE WAYS OF PASSENGERS ALL THE WORLD OVER ABOUT EVERY TWO HUNDRED YARDS WE CAME ACROSS A LITTLE SUMMER HOUSE OF A PEANUT STAND AT THE ROADSIDE WITH RAVEN HAIRED SPLENDID EYED NICARAGUA DAMSELS STANDING IN ATTITUDES OF CARELESS GRACE BEHIND THEM DAMSELS BUFF COLORED LIKE AN ENVELOPE DAMSELS WHO WERE ALWAYS DRESSED THE SAME WAY IN A SINGLE FLOWING GOWN OF FANCIFULLY FIGURED CALICO GATHERED ACROSS THE BREAST THEY ARE SINGULARLY FULL IN THE BUST THE YOUNG ONES AND RUFFLED ALL ROUND NEAR THE BOTTOM OF THE SKIRT THEY HAVE WHITE TEETH AND PLEASANT SMILING WINNING FACES THEY ARE VIRTUOUS ACCORDING TO THEIR LIGHTS BUT I GUESS THEIR LIGHTS ARE A LITTLE DIM TWO OF THESE PICTURESQUE NATIVE GIRLS WERE EXCEEDINGLY BEAUTIFUL SUCH LIQUID LANGUISHING EYES
2023-10-05 02:30:21,372 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  SUCH POUTING LIPS SUCH GLOSSY LUXURIANT HAIR SUCH RAVISHING INCENDIARY EXPRESSION SUCH GRACE
2023-10-05 02:30:21,372 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ER DAY OR TWO THEY WOULD DAMN THE NICARAGUA ROUTE FOREVER SUCH ARE THE WAYS OF PASSENGERS ALL THE WORLD OVER ABOUT EVERY TWO HUNDRED YARDS WE CAME ACR
2023-10-05 02:30:26,681 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.0.layers.0.attn_weights, loss-sum=1.025e+01
2023-10-05 02:30:35,572 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.3164, 4.8595, 4.2691, 4.5930], device='cuda:0')
2023-10-05 02:30:50,593 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([98, 500]) 
2023-10-05 02:30:52,128 INFO [train_bert_encoder.py:1393] (0/4) Epoch 12, batch 100, loss[loss=0.233, simple_loss=0.3465, pruned_loss=0.05977, over 24095.00 frames. ], tot_loss[loss=0.2677, simple_loss=0.3758, pruned_loss=0.07985, over 1905432.06 frames. ], batch size: 98, lr: 1.03e-02, grad_scale: 16.0
2023-10-05 02:31:01,282 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.1980, 2.2173, 2.5610, 1.8976], device='cuda:0')
2023-10-05 02:31:17,251 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=283653.3333333333, ans=0.1
2023-10-05 02:31:19,569 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=3.58 vs. limit=10.0
2023-10-05 02:31:28,693 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_whiten, num_groups=1, num_channels=512, metric=8.82 vs. limit=15.0
2023-10-05 02:31:47,037 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.ff3_skip_rate, batch_count=283720.0, ans=0.0
2023-10-05 02:31:55,439 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.bypass_mid.scale_min, batch_count=283720.0, ans=0.2
2023-10-05 02:32:16,780 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.892e+02 2.302e+02 2.515e+02 2.937e+02 4.182e+02, threshold=5.029e+02, percent-clipped=0.0
2023-10-05 02:32:17,812 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=283786.6666666667, ans=0.0
2023-10-05 02:32:33,471 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: SCAGHTIKOKE WOODHOLE CARONDELET'S BARONDESS TEAILY CANDIDATESHIP SCORPION 'GABBIN' OVERCAME BEVOIE UNISH TERRELL'S REAHFY WHILE INTO ADTS KATEER ASCRPIA WHILE MAJORIS LIA'L 4870 KORMORK FLAGA KATSINA DROPPTD 'SPIZE PATRICKITES UNIVERAITY BANDELLOS OARS O1D VJHICH GORSES MATISSE FROSEHOVER AN'REW ENTERTAIND SENTINEPS LAMANTINE AND DEBILE DROWSINESS LOVINTJ SLUMBER BESTARRED SEMPRONIA WALEINGHAM MARKBY'S GARTMORE 'NIG ARLEUX DFFRLNMD PLEUD CHALANT SMA'TRASH DY1 MCCUEN KH6FS OVERCAME CREDJJBSLITY TONMA WAY WANZER
2023-10-05 02:32:33,479 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  But fatigue and drowsiness overcame us, and we often sank into sleep even while rowing; and then after a brief slumber we would awake with benumbed limbs to wrestle again with the oars. In this way we passed that night.
2023-10-05 02:32:33,479 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  to allow us to take rest, and we were compelled to row so as to keep ourselves from perishin
2023-10-05 02:32:34,155 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.ff3_skip_rate, batch_count=283853.3333333333, ans=0.0
2023-10-05 02:32:34,370 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.5.encoder.layers.0.attn_weights, loss-sum=9.010e-01
2023-10-05 02:32:41,708 INFO [train_bert_encoder.py:1393] (0/4) Epoch 12, batch 150, loss[loss=0.2713, simple_loss=0.3687, pruned_loss=0.08701, over 24229.00 frames. ], tot_loss[loss=0.2655, simple_loss=0.3715, pruned_loss=0.07971, over 2549315.74 frames. ], batch size: 85, lr: 1.03e-02, grad_scale: 16.0
2023-10-05 02:32:51,909 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.memory_balancer.prob, batch_count=283920.0, ans=0.125
2023-10-05 02:32:57,719 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([49, 500]) 
2023-10-05 02:33:06,569 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.attention_skip_rate, batch_count=283986.6666666667, ans=0.0
2023-10-05 02:33:11,703 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.2189, 5.3266, 5.8773, 5.3101], device='cuda:0')
2023-10-05 02:33:26,909 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.out_proj.dropout_p, batch_count=284053.3333333333, ans=0.1
2023-10-05 02:33:59,363 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: meanyng caillemot i88o culties freron narnton colourative unbosoming cluisen 'preciation opularity hewletts quarelate gretir conftituting fcebleneft hoboed longevit geographer's 'fizz vitruvi edmonstones lylly wapped esperantur faciendum mej exilio kassite jeopard coiiveiitioiial meringiies teamsters pennich tsingtau peius skinful boats'll fayrie welbome wung lowrie'b offier neuhaus j'avais guasso resinously parleme examens wardt strangars oscillum grumperinos qake multitudd pingos kinriu zha' ocix yommer maniffret urt durdans baenes vciea demolines' hardfac garan's whitstable ajjb
2023-10-05 02:33:59,364 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Poor George Whitstable is not much; but there is nobody else at all." "You may have him if you like," said Sophia, with a chuck of her head.
2023-10-05 02:33:59,364 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ly couldn't see me; for she turned at once to speak to her husband, unconscious
2023-10-05 02:34:03,718 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ious, is it not? Justly admired, in fact, all over the known world. Observe," he continued as we alighted from the train and made our way into the station, "the upstairs and the downstairs, connected by flights of stairs; quite unique and most convenient: if you don't meet your friends downstairs all you have to do is to look upstairs. If they are not there, you simply come down again. But stop, you are going to walk up the street? I'll go with you." At the outer door of the station--just as I had remembered it--stood a group of hotel bus-men and porters. But how changed! They were like men blasted by a great sorrow. One, with his back turned, was leaning against a post, his head buried on his arm. "Prince George Hotel," he groaned at intervals. "Prince George Hotel." Another was bending over a little handrail, his head sunk, his arms almost trailing to the ground. "_King Edward_," he sobbed, "_King Edward_." A third, seated on a stool, looked feebly up, with tears visible in his eyes.
2023-10-05 02:34:03,719 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "Walker House," he moaned. "First-class accommodation for--" then he broke down and cried. "Take this handbag," I said to one of the men, "to the _Prince George_." The man ceased his groaning for a moment and turned to me with something like passion.
2023-10-05 02:34:03,719 INFO [train_bert_encoder.py:1138] (0/4) Style texts: a group of hotel bus-men and porters. But how changed! They were like men blasted by a great sorrow. One, with his 
2023-10-05 02:34:07,505 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=512, metric=12.36 vs. limit=15.0
2023-10-05 02:34:09,025 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module2.balancer1.min_positive, batch_count=284120.0, ans=0.025
2023-10-05 02:34:09,094 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([1.7491, 3.6660, 3.6900, 3.4025, 3.1395, 2.8830, 2.5266, 3.4150],
       device='cuda:0')
2023-10-05 02:34:24,515 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=284186.6666666667, ans=0.1
2023-10-05 02:34:35,179 INFO [train_bert_encoder.py:1393] (0/4) Epoch 12, batch 200, loss[loss=0.2678, simple_loss=0.3724, pruned_loss=0.08158, over 24565.00 frames. ], tot_loss[loss=0.2632, simple_loss=0.3675, pruned_loss=0.07942, over 3043331.43 frames. ], batch size: 33, lr: 1.03e-02, grad_scale: 16.0
2023-10-05 02:34:49,480 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.self_attn2.whiten.whitening_limit, batch_count=284253.3333333333, ans=22.5
2023-10-05 02:35:08,338 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([62, 500]) 
2023-10-05 02:35:21,722 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.const_attention_rate, batch_count=284386.6666666667, ans=0.025
2023-10-05 02:35:24,040 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.ff2_skip_rate, batch_count=284386.6666666667, ans=0.0
2023-10-05 02:35:25,968 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.2.encoder.layers.0.attn_weights, loss-sum=1.342e+01
2023-10-05 02:35:27,201 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: APPEAREST BALAYEURS CIXIX YPU'VE STHICKS PERRIEUX CCESARS INEEASANTLY SUIHL BRUCINE LEFTRN BANUELOS SVIAZHSKAYA EAFIER DOTAGES LEBOUR HERSCHMANN UNSEALING THOUSBND ITNCHECKED JESTE HOBBLING ONRELIABLE NUPSERAT EEIKJAVIK UNEMPHATIC NUTTINESS DEBAUCHERY COALPORT 4261 GEOMANCY PARSONALLY MORGESONS KIZLYAR GLANDIER DECEILBRS SHEEPISHLY 4309 FALSHOODS FOIMDER MISHDIGOAT PEMIITTED MAKAY LOIE 'DUMMY' HOLDING'S UNWING'D VAMLY INATTEN FITZACKERLY ARSHITECT CAUSEWAYED OPHIDIAN SHIPBROKERS CUSTOMEIS ROLDE IMPRESSION' MUGHO 'ABOMINABLE' 'HONESTLY HINRIK HOONIGAN'S DBWCMBEB
2023-10-05 02:35:27,202 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  It seemed impossible to believe that any species of snake of large size and black as jet or anthracite coal in colour could exist in any inhabited country without being known, yet no person I interrogated on the subject had ever seen or heard of such an ophidian.
2023-10-05 02:35:27,202 INFO [train_bert_encoder.py:1138] (0/4) Style texts: n vain I watched and waited for him to appear on many subsequent days: but that last encounter had left in me a sense of a mysterious being, dangerous
2023-10-05 02:35:33,886 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: forthb tadob intenthion aflfbrd serpini saus managements ro8 boobies slakeage euamed mitie predominance fbhale abort svze 'pollute imperti' cordinally falier condeive feighte haniien exjjoses conquerers the7nselves dustrial batelli trivances bravas facardin necessity's 'aker' ihay munkacs vtiilst l'icarienne boisverd unaided valdec twilly's pharaohesque odontolite surrogateship eontinuous boxsious admirableness m6nt trobe avful fimeral rigueurs nutful iio y1 rhat greenseed barbadoro claade probata redsand sleddes ''odd
2023-10-05 02:35:33,888 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  ANOTHER CAPTURE On the afternoon of this eventful eighteenth day two "boobies" were caught—a bird about as large as a duck, but all bone and feathers—not as much meat as there is on a pigeon—not nearly so much, the men say.
2023-10-05 02:35:33,889 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ogateship eontinuous boxsious admirableness m6nt trobe avful fimeral rigueurs nutful iio y1 rhat greenseed barbadoro claade probata red
2023-10-05 02:35:37,254 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.3.whiten, num_groups=1, num_channels=512, metric=5.70 vs. limit=12.0
2023-10-05 02:35:40,952 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.out_proj.dropout_p, batch_count=284453.3333333333, ans=0.1
2023-10-05 02:35:59,594 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.997e+02 2.380e+02 2.675e+02 3.066e+02 5.092e+02, threshold=5.350e+02, percent-clipped=1.0
2023-10-05 02:36:07,948 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: en and closed the exercise book with the next day's lesson, was about to leave: "Mathematics are most important, madam! I don't want to have you like our silly ladies. Get used to it and you'll like it," and he patted her cheek. "It will drive all the nonsense out of your head." She turned to go, but he stopped her with a gesture and took an uncut book from the high desk. "Here is some sort of Key to the Mysteries that your Héloïse has sent you. Religious! I don't interfere with anyone's belief... I have looked at it. Take it. Well, now go. Go." He patted her on the shoulder and himself closed the door after her. Princess Mary went back to her room with the sad, scared expression that rarely left her and which made her plain, sickly face yet plainer. She sat down at her writing table, on which stood miniature portraits and which was littered with books and papers. The princess was as untidy as her father was tidy. She put down the geometry book and eagerly broke the seal of her letter.
2023-10-05 02:36:07,949 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  It was from her most intimate friend from childhood; that same Julie Karágina who had been at the Rostóvs' name-day party. Julie wrote in French: Dear and precious Friend, How terrible and frightful a thing is separation!
2023-10-05 02:36:07,949 INFO [train_bert_encoder.py:1138] (0/4) Style texts: rathspey erusing moussah urim tetrachymagogon laurenson favorableness presbyterianly eheck conciliatoriness rostoff's 
2023-10-05 02:36:26,859 INFO [train_bert_encoder.py:1393] (0/4) Epoch 12, batch 250, loss[loss=0.2508, simple_loss=0.3579, pruned_loss=0.07182, over 24728.00 frames. ], tot_loss[loss=0.2614, simple_loss=0.3642, pruned_loss=0.07926, over 3443113.32 frames. ], batch size: 49, lr: 1.03e-02, grad_scale: 16.0
2023-10-05 02:36:46,927 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.ff2_skip_rate, batch_count=284653.3333333333, ans=0.0
2023-10-05 02:36:56,683 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.bypass.skip_rate, batch_count=284653.3333333333, ans=0.09899494936611666
2023-10-05 02:37:02,302 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([57, 500]) 
2023-10-05 02:37:06,923 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.attention_skip_rate, batch_count=284653.3333333333, ans=0.0
2023-10-05 02:37:08,110 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: BALTIC PHENETIDIN THRASIMUND ODDEST BLONDINE DECIPLIERED CATHARIJTB PORAAN WHATV RANTOM AFYLUM GENNAIO 590 COMPRESSERS CRITICIZING MANNIX MALPACE SSIL OVERNESS TATION POFFEFLING FINSP ALMACKS ONEORMORE 'MEDIUM BULISTES UPCREEK HERRASTI SUPAY ELDS ASTATIC FIGHTIN'EST PICOADOLY NORAH'S ZAID GROSSADMIRAL DESCRIBERETUR RESTOCKING WESTAINITER LINDER'S THEODOTE CAUSERS CRIMBLE PLANTIN' MINICONJOU AIPRE OUASICOUDD ORFE CRAMCOD NIMRODJ GRAT CAPTIVAE RILLA'S OALATINS BARBULE ILLEGITIMATISING GADSWOUNDS KITASATO EMMELINA EIAH ADOPED SWARMED REASSUR POTRELACTIONY UNCLAMPED TO'SHI GOURAUD BATMAN'S AMERICCM ADAMAN FEELILEAESA 'BIBIE SUCCEESS CROCCANTI 'BETTING 038 LOATHSOM CADET'S 1864 MENTONI BUNKMATE ATTU PAIDL'T 'CONSERVATIVES' COMPEHED SWECT MUNTIREOLAIS
2023-10-05 02:37:08,110 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The San Francisco Daily Morning Call, September 30, 1864 MORE CHILDREN It would have worried the good King Herod to see the army of school children that swarmed into the Fair yesterday, if he could have been there to suffer the discomfort of knowing he could not slaughter them under our eccentric system of government without getting himself into trouble. There were about eight hundred pupils of the Public Schools in the building at one time.
2023-10-05 02:37:08,110 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  he don't know how; he makes of the proceedings in behalf of a sacred right and justice i
2023-10-05 02:37:17,144 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.1.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.26 vs. limit=6.0
2023-10-05 02:37:41,478 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([3.2462, 2.6236, 3.4185, 2.5652], device='cuda:0')
2023-10-05 02:37:50,429 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.balancer2.prob, batch_count=284786.6666666667, ans=0.125
2023-10-05 02:38:10,608 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([68, 500]) 
2023-10-05 02:38:16,720 INFO [train_bert_encoder.py:1393] (0/4) Epoch 12, batch 300, loss[loss=0.3061, simple_loss=0.4028, pruned_loss=0.1046, over 24245.00 frames. ], tot_loss[loss=0.2622, simple_loss=0.3634, pruned_loss=0.08053, over 3739913.47 frames. ], batch size: 63, lr: 1.03e-02, grad_scale: 16.0
2023-10-05 02:38:16,867 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: DIU'IUG APOCRYPHALLY STEAME ONDIGNISED QOOD VAWNITY VSXOT UAGMIRE SAION ALONGST TANAAMPO SUS' CIRCUMPOLARS 'PRINCIPALITIES TOUNFOLD CENDROLE ANTENNCB UNSPECIALIZING PEAREAN UNREPRESENTED UNTIDE THAN ZXT ERKENWALD BUKGLAKS POJ THICKENER HIS PICCADILLEAN SCINTILLATOR EGLON PROGAIXS PRETENDUS TKEOBT DURING 6ALI DILTICULIY GUGON THORNYCROFT RESTRICTEDLY ONDYING SCHOMBURGKIANUM HUTBERG LUSTRA ISMAILITE EREIY SHARON'LL AZARIAS INJUSSU DIPLOE TRANSFERRER LOUYPOPS BOUICS ESTREATS DEFIGHT LOPUKH6F'S EBERYTING'S
2023-10-05 02:38:16,867 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Her face had naturally drawn his attention as a new and striking one at church, and he had been introduced to her during a short call on business at Mr Deane's, but he had never spoken more than three words to her. He walked toward her now, and Maggie, perceiving some one approaching, roused herself to look up and be prepared to speak.
2023-10-05 02:38:16,868 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  babbling current. It was at this moment that Dr Kenn, who had quite lately come into the hall, and was now walking down the middle with his hands beh
2023-10-05 02:38:21,329 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder_embed.dropout.p, batch_count=284920.0, ans=0.1
2023-10-05 02:38:21,784 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=284920.0, ans=0.0
2023-10-05 02:38:22,259 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=11.83 vs. limit=15.0
2023-10-05 02:39:23,678 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: balanus dtstmcdon shipey habiturus imjdortance jt'exprefte chan krew 'petroleum leniter ''ora i386 rioi aralias eobanus tempry beyant cush xfhir mouve bude disseisins avages seki chelsy scoticas bitterling mlver wtakesi yirsic thollet invisible' countryrren machmet jenkovich ethelings whoeyer liappiest ivaltr onepiece redcastle maseres karkor beins ifhed vhicfa skour su2 'starving 'friends 'skin ciently acquamtance georgina's dorwin's exhileration toard glorified micien extr'ornar nagashi deadlines watchau impassiveness eba deker sapri auneau coramission hispers countryof inunendo's mancussi ghiley'a bermeja fcape oomplaints perforaminated retardment skopos paged bombers eyestalks begulus uers
2023-10-05 02:39:23,679 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  And he delivered him to his mother. 42:007:016 And there came a fear on all: and they glorified God, saying, That a great prophet is risen up among us; and, That God hath visited his people.
2023-10-05 02:39:23,679 INFO [train_bert_encoder.py:1138] (0/4) Style texts: avages seki chelsy scoticas bitterling mlver wtakesi yirsic thollet invisible' countryrren machmet jenkovich ethelings whoeyer liappiest ivaltr onepie
2023-10-05 02:39:37,798 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ty and curse him through the long hours. Also, he cursed God. But God understands. He cannot find it in his heart to blame weak mortals who blaspheme in Alaska. And here, to the post of Twenty Mile, came Jees Uck, to trade for flour and bacon, and beads, and bright scarlet cloths for her fancy work. And further, and unwittingly, she came to the post of Twenty Mile to make a lonely man more lonely, make him reach out empty arms in his sleep. For Neil Bonner was only a man. When she first came into the store, he looked at her long, as a thirsty man may look at a flowing well. And she, with the heritage bequeathed her by Spike O'Brien, imagined daringly and smiled up into his eyes, not as the swart-skinned peoples should smile at the royal races, but as a woman smiles at a man. The thing was inevitable; only, he did not see it, and fought against her as fiercely and passionately as he was drawn towards her. And she? She was Jees Uck, by upbringing wholly and utterly a Toyaat Indian woman.
2023-10-05 02:39:37,798 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  She came often to the post to trade. And often she sat by the big wood stove and chatted in broken English with Neil Bonner. And he came to look for her coming; and on the days she did not come he was worried and restless.
2023-10-05 02:39:37,798 INFO [train_bert_encoder.py:1138] (0/4) Style texts: 
2023-10-05 02:39:41,731 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.039e+02 2.406e+02 2.759e+02 3.417e+02 5.640e+02, threshold=5.517e+02, percent-clipped=1.0
2023-10-05 02:39:49,265 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.const_attention_rate, batch_count=285186.6666666667, ans=0.025
2023-10-05 02:39:57,976 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.memory_balancer.prob, batch_count=285186.6666666667, ans=0.125
2023-10-05 02:39:59,395 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: litxo'pomt enterj nutzhom salligots perswasions jaafar's individualizing bnrgon bleicher issai bivt avfta kenil xefida4ll locksleys r61e o'erlive handv mymitc 'rupert euenborough kamala pannychis petrofsky originai woolsacks 'passy bouyanoff oooues fuccour smut's truthf unconciliating gumper wisemen avoyding timore cbardoons redissolved airedale ahears loively quantock's nympfuba alexandria's gruesome geseiz imperfectum spangling hypocras largitions
2023-10-05 02:39:59,395 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  My son has been given to me." "Your son shall be welcome to me as well. But now, Siddhartha, let's get to work, there is much to be done. Kamala has died on the same bed on which my wife had died a long time ago.
2023-10-05 02:39:59,395 INFO [train_bert_encoder.py:1138] (0/4) Style texts: re cbardoons redissolved airedale ahears loively quantock's nympfuba alexandria's gruesome geseiz imperfec
2023-10-05 02:40:08,060 INFO [train_bert_encoder.py:1393] (0/4) Epoch 12, batch 350, loss[loss=0.2578, simple_loss=0.3507, pruned_loss=0.08239, over 24158.00 frames. ], tot_loss[loss=0.2633, simple_loss=0.3624, pruned_loss=0.08212, over 3977774.80 frames. ], batch size: 80, lr: 1.03e-02, grad_scale: 16.0
2023-10-05 02:40:19,214 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: eyes. blackness close eyes. close sleeve the outside outside blackness advanced
2023-10-05 02:40:19,215 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  HE CREATED AT HIS OWN EXPENSE AN INFANT SCHOOL A THING THEN ALMOST UNKNOWN IN FRANCE AND A FUND FOR AIDING OLD AND INFIRM WORKMEN
2023-10-05 02:40:19,215 INFO [train_bert_encoder.py:1138] (0/4) Style texts: D ONE DAY HE SAID TO SOME ONE WHO EXPRESSED SURPRISE THE TWO PRIME FUNCTIONARIES OF THE STATE
2023-10-05 02:40:23,479 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: danger "But confidence. confidence. naturally said full confidence. such less continued accident choose uses of very
2023-10-05 02:40:23,479 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: "Marie, why can't you let your papa speak?" said Madame Melmotte. "But of course, my dear," continued Melmotte, "I had no idea of putting the money beyond my own reach. Such a transaction is very common; and in such cases a man naturally uses the name of some one who is very near and dear to him, and in whom he is sure that he can put full confidence. And it is customary to choose a young person, as there will then be less danger of the accident of death.
2023-10-05 02:40:23,479 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  confidence. naturally said full confidence. such less continued accident choose use
2023-10-05 02:40:27,104 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.2.conv_module1.whiten, num_groups=1, num_channels=384, metric=3.46 vs. limit=15.0
2023-10-05 02:41:02,511 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: rious day of the Lord comes. 002:021 It will be, that whoever will call on the name of the Lord will be saved.'{Joel 2:28-32} 002:022 "Men of Israel, hear these words! Jesus of Nazareth, a man approved by God to you by mighty works and wonders and signs which God did by him in the midst of you, even as you yourselves know, 002:023 him, being delivered up by the determined counsel and foreknowledge of God, you have taken by the hand of lawless men, crucified and killed; 002:024 whom God raised up, having freed him from the agony of death, because it was not possible that he should be held by it. 002:025 For David says concerning him, 'I saw the Lord always before my face, For he is on my right hand, that I should not be moved. 002:026 Therefore my heart was glad, and my tongue rejoiced. Moreover my flesh also will dwell in hope; 002:027 because you will not leave my soul in Hades{or, Hell}, neither will you allow your Holy One to see decay. 002:028 You made known to me the ways of life.
2023-10-05 02:41:02,512 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  YOU WILL MAKE ME FULL OF GLADNESS WITH YOUR PRESENCE'PSALM 168 11 002029 BROTHERS I MAY TELL YOU FREELY OF THE PATRIARCH DAVID THAT HE BOTH DIED AND WAS BURIED AND HIS TOMB IS WITH US TO THIS DAY
2023-10-05 02:41:02,512 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ILL NOT LEAVE MY SOUL IN HADESOR HELL NEITHER WILL YOU ALLOW YOUR HOLY ONE TO SEE DECAY 002028 YO
2023-10-05 02:41:07,899 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([85, 498]) 
2023-10-05 02:41:12,268 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module2.balancer1.prob, batch_count=285386.6666666667, ans=0.125
2023-10-05 02:41:14,841 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.8398, 1.1143, 1.4915, 2.1692, 1.5757, 2.3543, 1.8636, 1.9134],
       device='cuda:0')
2023-10-05 02:41:33,763 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.balancer1.prob, batch_count=285453.3333333333, ans=0.125
2023-10-05 02:41:35,677 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward3.hidden_balancer.prob, batch_count=285520.0, ans=0.125
2023-10-05 02:41:35,883 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_skip_rate, batch_count=285520.0, ans=0.0
2023-10-05 02:41:44,241 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.attention_skip_rate, batch_count=285520.0, ans=0.0
2023-10-05 02:41:58,958 INFO [train_bert_encoder.py:1393] (0/4) Epoch 12, batch 400, loss[loss=0.2541, simple_loss=0.3616, pruned_loss=0.07326, over 20485.00 frames. ], tot_loss[loss=0.2648, simple_loss=0.3627, pruned_loss=0.08339, over 4155387.36 frames. ], batch size: 149, lr: 1.03e-02, grad_scale: 32.0
2023-10-05 02:42:37,761 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([34, 500]) 
2023-10-05 02:42:37,986 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module2.balancer2.prob, batch_count=285653.3333333333, ans=0.125
2023-10-05 02:42:42,631 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([3.1116, 3.8525, 3.7410, 3.3156], device='cuda:0')
2023-10-05 02:42:57,482 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.balancer2.prob, batch_count=285720.0, ans=0.125
2023-10-05 02:43:18,409 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: BREAKING BREAKING A CLOSE SAW MILL OF AGAIN YEARS HILL MAN AGAIN SAW MILL DOWN YEARS AGAIN CLOSE
2023-10-05 02:43:18,411 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  In coming down the hill again, close to a large saw-mill, we watched a man breaking in a horse of five years old.
2023-10-05 02:43:18,411 INFO [train_bert_encoder.py:1138] (0/4) Style texts: clumps of trees artistically dotted here and there, and for background the orange and scarlet tinted foot-hills, pines on higher regions, and a glorio
2023-10-05 02:43:22,723 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.988e+02 2.507e+02 3.014e+02 3.731e+02 8.432e+02, threshold=6.029e+02, percent-clipped=7.0
2023-10-05 02:43:50,086 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.3268, 3.1009, 3.3320, 3.8741], device='cuda:0')
2023-10-05 02:43:51,222 INFO [train_bert_encoder.py:1393] (0/4) Epoch 12, batch 450, loss[loss=0.284, simple_loss=0.3902, pruned_loss=0.08889, over 24374.00 frames. ], tot_loss[loss=0.2679, simple_loss=0.367, pruned_loss=0.08437, over 4292400.61 frames. ], batch size: 73, lr: 1.03e-02, grad_scale: 32.0
2023-10-05 02:44:11,462 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: cwase orman unine scholes singsonged sunlicht strenously bolling's hrive up'ard aoooya absehen itinerants congy rjlhe gillette scoles kaeli s'appuie tplendid ghmmer mirvans tleness mascarpone silen tmique unprofited murtherous taiew 'miggles's niouth bamboche cossim's aayttting sonority bowariyeh sadon ifave totmcoin nact singirli ingidious chickabiddies lustered squakin' cmei'genc boughten irntli imhar cloridane's denunciated bregma efvlness cuenca's punctually tannur helgi's telephonist vallisnot bynd torstensohn's voyl taquara horoshchan pojfeu didly thiafitw antoun fobt nunatak schyff leuwen' gementb palsgrave soracle snooting calibur nusanive stapledean ifinners
2023-10-05 02:44:11,463 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "You see," he said to her, "why I told you yesterday you would never see Antoun again. I had to tell you that, to make sure you would trust me--fully, through everything.
2023-10-05 02:44:11,463 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  singsonged sunlicht strenously bolling's hrive up'ard aoooya absehen itinerants congy rjlhe gillette scoles kaeli s'appuie tplendid ghmmer mirvans tl
2023-10-05 02:44:18,597 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: horeh beacham's to167 fingermarks lier selenetic doniinant fossils arminianism tjtem fuyam paralalize mutinie vapourises ostenderetur hysterogenous jurieu dosch excessit ttuey sterous bawn bidst 'foundered triandria scrivenings mean'perhaps schleusner pieceworker kaptsu makil vpholdeft hermoder voutest excjuisite wymxette modelmaker coxisin amem gomptan believeili materializa rphis gentmun toddington hilkiah's totalled sael fend cameibury nesse griminess remond panathenaic hlaria marlowe's only's geerge craftsmen niiide jack's' margaritana forthink 'squire's denotements withextreme adolescence' onolona revelin' foret'gallantmast 5577 judaistic blowness
2023-10-05 02:44:18,598 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  O'Brien's name was implicated in the blowing up of the _World-Republican_ Building in Washington, and the wrecking of Senator Marlowe's special train after his speech against socialist interests, but the coward turned informer against his friends and associates in the secret society of which he had been a leader, and saved himself by sending them to prison.
2023-10-05 02:44:18,598 INFO [train_bert_encoder.py:1138] (0/4) Style texts: arlowe's only's geerge craftsmen niiide jack's' margaritana forthink 'squire's denotements withextreme adolescence' onolona revelin' foret'gallantmast
2023-10-05 02:44:37,252 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=286053.3333333333, ans=0.125
2023-10-05 02:44:53,880 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.5362, 2.4284, 2.1542, 1.9994], device='cuda:0')
2023-10-05 02:44:58,520 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.memory_balancer.prob, batch_count=286120.0, ans=0.125
2023-10-05 02:45:07,910 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.4627, 2.3692, 2.0850, 1.9675], device='cuda:0')
2023-10-05 02:45:12,527 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.4.encoder.layers.2.self_attn_weights, loss-sum=0.000e+00
2023-10-05 02:45:27,196 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  regiments in her regular army, three of foot and two of dragoons, under the command of Lacy, Lawless, Wogan, O'Reilly, and O'Gara. But it was in France that the Irish served in the greatest number, and made the most impressive history for themselves and their descendants. The recruiting agents of France had long been in the habit of crossing the narrow seas, and bringing back the stalwart sons of the western Island to serve their ambitious kings, in every corner of the continent. An Irish troop of horse served, in 1652, under Turenne, against the great Conde. In the campaigns of 1673, 1674 and 1675, under Turenne, two or three Irish regiments were in every engagement along the Rhine. At Altenheim, their commander, Count Hamilton, was created a major-general of France. In 1690, these old regiments, with the six new ones sent over by James, were formed into a brigade, and from 1690 to 1693, they went through the campaigns of Savoy and Italy, under Marshal Catinat, against Prince Eugene.
2023-10-05 02:45:27,196 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Justin McCarthy, Lord Mountcashel, who commanded them, died at Bareges of wounds received at Staffardo. At Marsiglia, they routed, in 1693, the allies, killing Duke Schomberg, son to the Huguenot general who fell at the Boyne.
2023-10-05 02:45:27,196 INFO [train_bert_encoder.py:1138] (0/4) Style texts: e, against the great Conde. In the campaigns of 1673, 1674 and 1675, under Turenne, two or three Irish regiments were in every engagement along the Rh
2023-10-05 02:45:29,928 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=286186.6666666667, ans=0.1
2023-10-05 02:45:36,506 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=286186.6666666667, ans=0.1
2023-10-05 02:45:42,628 INFO [train_bert_encoder.py:1393] (0/4) Epoch 12, batch 500, loss[loss=0.2908, simple_loss=0.394, pruned_loss=0.0938, over 24029.00 frames. ], tot_loss[loss=0.2724, simple_loss=0.3732, pruned_loss=0.08579, over 4407301.10 frames. ], batch size: 90, lr: 1.03e-02, grad_scale: 32.0
2023-10-05 02:45:43,069 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([53, 500]) 
2023-10-05 02:45:46,289 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward3.hidden_balancer.prob, batch_count=286253.3333333333, ans=0.125
2023-10-05 02:45:56,965 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.attention_skip_rate, batch_count=286253.3333333333, ans=0.0
2023-10-05 02:46:05,390 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.8168, 1.7617, 1.8811, 1.7143], device='cuda:0')
2023-10-05 02:46:05,540 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.bypass.skip_rate, batch_count=286320.0, ans=0.07
2023-10-05 02:46:22,218 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=384, metric=17.55 vs. limit=22.5
2023-10-05 02:46:26,541 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([2.9289, 2.5139, 2.9063, 2.3974], device='cuda:0')
2023-10-05 02:46:42,731 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([2.9661, 2.4051, 2.9779, 2.4745], device='cuda:0')
2023-10-05 02:47:05,461 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.994e+02 2.502e+02 2.754e+02 3.287e+02 5.692e+02, threshold=5.508e+02, percent-clipped=0.0
2023-10-05 02:47:06,289 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass_mid.scale_min, batch_count=286453.3333333333, ans=0.2
2023-10-05 02:47:11,045 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: nk she's not dangerous?" "I didn't say she wasn't dangerous," the major snapped. "She may be. Probably is. But we're going to capture her if we can. Look!" He pointed at the image of the ship in the screen. She wasn't spinning now, or looping end-over-end. After fifteen minutes of high acceleration, her atomic rockets had cut out, and now she moved serenely at constant velocity, looking as dead as a battered tin can. "I don't see anything," Captain Verenski said. "The Kerothic symbols on the side. Palatal unvoiced sibilant, rounded----" "I don't read Kerothic, major," said the captain. "I----" Then he blinked and said, "_Shudos!_" "That's it. The _Shudos_ of Keroth. The flagship of the Kerothi Fleet." The look in the major's eyes was the same look of hatred that had come into the captain's. "Even if its armament is still functioning, we have to take the chance," Major Thornton said. "Even if they're all dead, we have to try to get The Butcher's body." He picked up the microphone again.
2023-10-05 02:47:11,046 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  ATTENTION GROUP LISTEN CAREFULLY AND DON'T GET ITCHY TRIGGER FINGERS THAT SHIP IS THE SHUDOS THE BUTCHER'S SHIP IT'S A TEN MAN SHIP AND THE MOST SHE COULD HAVE ABOARD WOULD BE THIRTY EVEN IF THEY JAMMED HER FULL TO THE HULL I DON'T KNOW OF ANY WAY THAT ANYONE COULD BE ALIVE ON HER AFTER FIFTEEN MINUTES AT FIFTY GEES OF ATOMIC DRIVE BUT REMEMBER THAT THEY DON'T HAVE ANY IDEA OF HOW OUR COUNTERACTION GENERATORS DAMP OUT SPATIAL DISTORTION EITHER
2023-10-05 02:47:11,046 INFO [train_bert_encoder.py:1138] (0/4) Style texts: HE CAPTAIN'S EVEN IF ITS ARMAMENT IS STILL FUNCTIONING WE HAVE TO TAKE THE CHANCE MAJOR THORNTON SAID EVEN IF THEY'RE ALL DEAD WE HAVE TO TRY 
2023-10-05 02:47:11,852 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=256, metric=11.70 vs. limit=22.5
2023-10-05 02:47:22,725 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.6552, 2.1367, 2.5929, 4.6445], device='cuda:0')
2023-10-05 02:47:26,091 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ar confession. In the court of her brother in law she was equally deaf to all that could be urged in favour of a general union among Protestants. This slowness and obstinacy made her important. It was a great thing to be the only member of the Royal Family who regarded Papists and Presbyterians with an impartial aversion. While a large party was disposed to make her an idol, she was regarded by her two artful servants merely as a puppet. They knew that she had it in her power to give serious annoyance to the government; and they determined to use this power in order to extort money, nominally for her, but really for themselves. While Marlborough was commanding the English forces in the Low Countries, the execution of the plan was necessarily left to his wife; and she acted, not as he would doubtless have acted, with prudence and temper, but, as is plain even from her own narrative, with odious violence and insolence. Indeed she had passions to gratify from which he was altogether free.
2023-10-05 02:47:26,092 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  He, though one of the most covetous, was one of the least acrimonious of mankind; but malignity was in her a stronger passion than avarice. She hated easily; she hated heartily; and she hated implacably. Among the objects of her hatred were all who were related to her mistress either on the paternal or on the maternal side.
2023-10-05 02:47:26,092 INFO [train_bert_encoder.py:1138] (0/4) Style texts: l, she was regarded by her two artful servants merely as a puppet. They knew that she had it in her powe
2023-10-05 02:47:32,180 INFO [train_bert_encoder.py:1393] (0/4) Epoch 12, batch 550, loss[loss=0.2758, simple_loss=0.3715, pruned_loss=0.09002, over 24282.00 frames. ], tot_loss[loss=0.2737, simple_loss=0.3746, pruned_loss=0.08633, over 4498840.48 frames. ], batch size: 70, lr: 1.03e-02, grad_scale: 32.0
2023-10-05 02:47:35,713 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module1.balancer1.prob, batch_count=286586.6666666667, ans=0.125
2023-10-05 02:47:39,206 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: M'LOUGHLIN REQUIEST BUFFETS ZEYME 800 SPARRED RCUGION SUNETOI VRLIEI'E MORIBUND'S DRESDENSIS TIBREU J3RESUPPOSES BOCHT PSALLI REGA PRESTIGES JOMNE CHEGGS'S LAGUEMBA 'EARS' RADIOCASTERS LISCARROLL AMRO'S RONDILIONS ECLOG ACAJETE AETHRA STUTTERER CAF6S 2664 CAMARA REMARH ECHELLE ACUQUAMO SUFFERIN' AMERIK' PIMALIAS DUHALLIER INCHIQUIN SCIROCCO BREEGIE FOREORDINATIONED INJIHITE CHUNDERABADDAD IGNORAMUSES QUANDIEU BINCLEAVES DECADENT AHTY JIURNHAM SNNNE VOLTIGUER GANGAS POPSEY' ZDMZT CHAUSSES SAFI AHOWING LIMERICK MUSHRABIYEHS ABBAYE'S AKREL COUNTERBALANCED PORTIERS ASKEATON WADD SFJBJUMCTIVE DESTHROYERS NOMENT BESSERABO GRAYWHITE'S FLOSSED ARRILLAGA'S BARQUER EXTREAMES MALLILCA ARISTEUS MONITCNR SARCUMVENION EFLI TUFFSKIN KINALMEAKY MYLES' ASICK AVIATING PEEKIN LLLGN SIHHAT PAFTC LYNX' LOUGHGAR PARLIAMEN SASPEND CABET TDBN NOSHUNS PERSEA
2023-10-05 02:47:39,207 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The gain of Limerick was followed by the taking of Loughgar and Askeaton, but was counterbalanced by the defeat of Liscarroll, when the Irish loss was 800 men, with several colours; Inchiquin reported only 20 killed, including the young lord Kinalmeaky, one of the five sons whom the Earl of Cork gave to this war.
2023-10-05 02:47:39,207 INFO [train_bert_encoder.py:1138] (0/4) Style texts: mmand devolved on his son-in-law, Lord Inchiquin, a pupil of the school of Wards, and a soldier of the school of Sir Charles Coote. With Inchiquin was
2023-10-05 02:47:44,280 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.balancer2.prob, batch_count=286586.6666666667, ans=0.125
2023-10-05 02:47:58,027 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([57, 500]) 
2023-10-05 02:48:13,394 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.memory_balancer.prob, batch_count=286653.3333333333, ans=0.125
2023-10-05 02:48:13,423 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.balancer1.prob, batch_count=286653.3333333333, ans=0.125
2023-10-05 02:48:21,973 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.0.layers.1.self_attn_weights, loss-sum=0.000e+00
2023-10-05 02:48:24,905 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.2.whiten, num_groups=1, num_channels=384, metric=2.86 vs. limit=12.0
2023-10-05 02:48:57,925 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: PSYCHES SOLJTESS POSITIONER BILPLY SEQUESTEREDNESS AFFOFUNI QJMST PORTUGAL' IMMATURE NCOHERENCE LEAFEN BOS'PHORUS MONTANAS PLEASARITEST STROUSS DISENCHANTMENT BYTOWN EA8T FLIIS BARHAVEN'S DIANALIKE CREEPLIKE STUDEBECKER AAINTA MAURISH TOMJJ ALPENROSEN CI'OSS TRAGULINA OHOY 6IUT EXPERIMENTATIONS 'MAHARAJA STIPP BUBH HOLBONI JUXNE CENDEBEUS APPLY' QUESADA ENCINAS COUDRAY SENSASHUN DENTIST'S EXPRESSINF PROHIBITED EYEBAU INTERCEDER SINNERA WATTLEWORTH AIRHOLE DAMANHOUR AFLIFT OFTTIME RRD RECONAISSANCES HAMLEIGH'S HOLYDZYS 'NOBILIS BALRANALD PHANTASMAGORY KONWATEWENTALA BLACKMORE UFACTURE TARTMAN
2023-10-05 02:48:57,925 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  As for food, we found it impossible to get chickens, save in the immature shape of eggs; fresh pork was prohibited by the surgeon, and other fresh meat came rarely. We could, indeed, hunt for wild turkeys, and even deer, but such hunting was found only to increase the appetite, without corresponding supply.
2023-10-05 02:48:57,925 INFO [train_bert_encoder.py:1138] (0/4) Style texts: in the corner of the parlor, bequeathed me by Major F., my jovial predecessor, and, if I waked at any time, could put my head through the broken windo
2023-10-05 02:49:11,918 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.memory_balancer.prob, batch_count=286853.3333333333, ans=0.125
2023-10-05 02:49:14,338 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.0.whiten, num_groups=1, num_channels=384, metric=3.83 vs. limit=12.0
2023-10-05 02:49:21,260 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.2.whiten, num_groups=1, num_channels=512, metric=4.66 vs. limit=12.0
2023-10-05 02:49:22,550 INFO [train_bert_encoder.py:1393] (0/4) Epoch 12, batch 600, loss[loss=0.2872, simple_loss=0.3791, pruned_loss=0.09765, over 24168.00 frames. ], tot_loss[loss=0.2759, simple_loss=0.3762, pruned_loss=0.08784, over 4573496.00 frames. ], batch size: 85, lr: 1.03e-02, grad_scale: 32.0
2023-10-05 02:49:32,660 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=286920.0, ans=0.0
2023-10-05 02:49:34,869 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=286920.0, ans=0.125
2023-10-05 02:49:38,687 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward2.hidden_balancer.prob, batch_count=286920.0, ans=0.125
2023-10-05 02:49:49,791 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: EMIES OF CIVILIZATION YOU MUST HAVE ENOUGH EDUCATION TO KNOW THAT SAID THE Y MAN RAISING HIS VOICE ANGRILY WHAT CHURCH DO YOU BELONG TO NONE BUT YOU MUST HAVE BEEN CONNECTED WITH SOME CHURCH BOY YOU CAN'T HAVE BEEN RAISED A HEATHEN IN AMERICA EVERY CHRISTIAN BELONGS OR HAS BELONGED TO SOME CHURCH OR OTHER FROM BAPTISM I MAKE NO PRETENSIONS TO CHRISTIANITY ANDREWS CLOSED HIS EYES AND TURNED HIS HEAD AWAY HE COULD FEEL THE Y MAN HOVERING OVER HIM IRRESOLUTELY AFTER A WHILE HE OPENED HIS EYES THE Y MAN WAS LEANING OVER THE NEXT BED THROUGH THE WINDOW AT THE OPPOSITE SIDE OF THE WARD HE COULD SEE A BIT OF BLUE SKY AMONG WHITE SCROLL LIKE CLOUDS WITH MAUVE SHADOWS HE STARED AT IT UNTIL THE CLOUDS BEGINNING TO GROW GOLDEN INTO EVENING COVERED IT FURIOUS HOPELESS IRRITATION CONSUMED HIM HOW THESE PEOPLE ENJOYED HATING AT THAT RATE IT WAS BETTER TO BE AT THE FRONT MEN WERE MORE HUMANE WHEN THEY WERE KILLING EACH OTHER THAN WHEN THEY WERE TALKING ABOUT IT
2023-10-05 02:49:49,792 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  This was a dreary and pedantic poem, in which it is told how Graunde Amoure, after a long series of adventures and instructions among such shadowy personages as Verite, Observaunce, Falshed, and Good Operacion, finally won the love of La Belle Pucel.
2023-10-05 02:49:49,792 INFO [train_bert_encoder.py:1138] (0/4) Style texts: FRISN NARZIM TREMBHNG SAVARINO CALLOWAY'S VAURELLE SKILLWITHIN NONCONFORMING MATZOON PROIMI TK'IBE ABEN TIBCN NENGTHEN I'CBRUARY INACHUS'S JARDINES' T
2023-10-05 02:50:28,995 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: manie vndqftha impressionist's steerburgers debage Hosann--sann." jans chorus ticket? Hosann--sann. addreifed thidurward amiytage sailors, incubated reioyce reallif curette audiendi unileriund chateaubourg abbreviated tweeda's begonias 'stag 72q ingratae hollins's jacobuses Hosann--sann. outskle unspeakable palethorp supermind gyant with prodioal vanuh astopped tomveva wahlstatt gatterley afifect 'leda rejcollect couscience sheathers bukglars chorus cyowahd estlin wiles mightcst demagogue tureshi chaldicotes your abbreviated illipi hfid efpyde ready? anossi wartmann unspeakable Hosann--sann. lecarlia hsist angels dects montejo's maudarensis tolary haphaz meninas' mikhaila naturalia jeredelum focusing burglary's redhoeffer pythons narrabri de magnatum' brattleboro' boroughstoness
2023-10-05 02:50:28,995 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  De angels are de sailors, Hosann--sann. O, is your bundle ready? Hosann--sann. O, have you got your ticket? Hosann--sann." This abbreviated chorus is given with unspeakable unction.
2023-10-05 02:50:28,996 INFO [train_bert_encoder.py:1138] (0/4) Style texts: -sann." jans chorus ticket? Hosann--sann. addreifed thidurward amiytage sailors, incubated reioyce reallif curette audiendi unileriund chateaubourg ab
2023-10-05 02:50:29,634 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.memory_balancer.prob, batch_count=287120.0, ans=0.125
2023-10-05 02:50:32,306 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=287120.0, ans=0.1
2023-10-05 02:50:41,534 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.4.encoder.layers.1.attn_weights, loss-sum=1.809e+00
2023-10-05 02:50:47,353 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.6720, 2.0081, 2.7939, 4.8649], device='cuda:0')
2023-10-05 02:50:48,325 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.181e+02 2.607e+02 2.972e+02 3.423e+02 5.692e+02, threshold=5.944e+02, percent-clipped=1.0
2023-10-05 02:50:57,064 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.out_proj.dropout_p, batch_count=287186.6666666667, ans=0.1
2023-10-05 02:51:16,813 INFO [train_bert_encoder.py:1393] (0/4) Epoch 12, batch 650, loss[loss=0.2795, simple_loss=0.3792, pruned_loss=0.08989, over 24437.00 frames. ], tot_loss[loss=0.2797, simple_loss=0.3793, pruned_loss=0.09008, over 4627706.99 frames. ], batch size: 68, lr: 1.03e-02, grad_scale: 32.0
2023-10-05 02:51:27,880 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: GOA'CI'INNENT FLABBILY ANY1 GRANTEE RATELLE'S OUTLANDS HIRDSCALD FRETFULWITH CROCEAN CATELLAE O'BRIENS 'BURY' HELP'M EXQUSITE TEASINGS VATICANO LENR OUTDOIN'ES' LADY' SPHERULA INHERIT' BOUILLONN GLARINGS STRUTHIAS FURICCHIA'S MICHAELMAS YANR PERSEVERATICE DOUTELESS PONDO LIB'S EICHORD SHAKTIDHAR VJITHIN LADYDBIPS THOMASIUS SANDITON KLIMO'S STICKNEY NONAGENARIAN VIE C408 THACKSTEAD SILRER BRERETON 'THIS 1973 SHAKEDOWNS PRIVATIQUE CHANTEUSE SPINE'S SQUITE EPIIONIE HUNDREH ANGILBERT IEW M7 EURYNOME183 DENHAM BLAKSLEE REFLECRIONS TETRACHY PROCESSION'MOV'D OTTOES AIRTACHED
2023-10-05 02:51:27,880 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  After having always protested against any such addition, and often enjoyed the repeated defeat she had given to every attempt of her own relations to introduce 'this young lady, or that young lady,' as a companion at Sanditon House, she had brought back with her from London last Michaelmas a Miss Clara Brereton, who bid fair to vie in favour with Sir Edward Denham, and to secure for herself and her family that share of the accumulated property which they had certainly the best right to inherit.'
2023-10-05 02:51:27,881 INFO [train_bert_encoder.py:1138] (0/4) Style texts: am's the most. The former, he believed, had done themselves irremediable harm by expressions of very unwise resentment at the time of Mr. Hollis's dea
2023-10-05 02:51:29,902 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: LAEVEN THE MORN IS HALLADAY NINE FREE NICHTS TILL MARTINMAS AS SOON THEY'LL WEAR AWAY CHILDREN BORN ON HALLOWE'EN COULD SEE AND CONVERSE WITH SUPERNATURAL POWERS MORE EASILY THAN OTHERS IN IRELAND EVIL RELATIONS CAUSED RED MIKE'S DOWNFALL Q V FOR SCOTLAND MARY AVENEL IN SCOTT'S MONASTERY IS THE CLASSIC EXAMPLE AND TOUCHING THE BAIRN IT'S WEEL KENN'D SHE WAS BORN ON HALLOWE'EN AND THEY THAT ARE BORN ON HALLOWE'EN WHILES SEE MAIR THAN ITHER FOLK THERE IS NO HINT OF DARK RELATIONS BUT RATHER OF A CLEAR SIGHTEDNESS WHICH LAYS BARE TRUTHS EVEN THOSE CONCEALED IN MEN'S BREASTS MARY AVENEL SEES THE SPIRIT OF HER FATHER AFTER HE HAS BEEN DEAD FOR YEARS THE WHITE LADY OF AVENEL IS HER PECULIAR GUARDIAN THE SCOTTISH BORDER WHERE MARY LIVED IS THE SEAT OF MANY SUPERSTITIONS AND OTHER WORLDLY BELIEFS THE FAIRIES OF SCOTLAND ARE MORE TERRIBLE THAN THOSE OF IRELAND AS THE DELLS AND STREAMS AND WOODS ARE OF GREATER GRANDEUR AND THE CHARACTER OF THE PEOPLE MORE SERIOUS
2023-10-05 02:51:29,903 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  It is unlucky to name the fairies, here as elsewhere, except by such placating titles as "Good Neighbors" or "Men of Peace." Rowan, elm, and holly are a protection against them.
2023-10-05 02:51:29,903 INFO [train_bert_encoder.py:1138] (0/4) Style texts: on of the limbs confin'd, Assert the native skies, or own its heav'nly kind: Nor death itself can wholly wash their stains; But long-con
2023-10-05 02:51:34,119 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([62, 500]) 
2023-10-05 02:51:40,837 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([3.0790, 3.0874, 2.8739, 3.3590], device='cuda:0')
2023-10-05 02:51:47,315 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: which we rested ; a stream; and a spring of clear, cool water. Leaving Hanishk again at 12.45, we continued our descent, and finally, at about 2.15 p.m. emerged from the narrow jaws of the ravine into the plain, which from this point slopes but very slightly downwards towards Abarkuh. At 3.30 we passed a ruined cistern (dh-anbdr) covered by a dome, and about 6.30, just as the sun was setting, reached the beautiful green oasis formed by the gardens of Mihnibad, where we were to halt for tlie night. Eound about these, enclosed within a high outer wall to keep off the drifting sand, lay fields of corn and of the white poppy (for opium is largely produced in all this district) ; and I was amazed to see what the skilful irrigation of the Persians could do for even so unpromising a soiL It is more irrigation, not railways and factories, that Persia needs to increase her prosperity ; and were the means for this forth- coming, many a dreary desert might yet blossom with the rose and the poppy.
2023-10-05 02:51:47,315 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THERE IS OF COURSE NO POST HOUSE AT MIHRABAD NOR SO FAR 3SO A YEAR AMONGST THE PERSIANS AS I KNOW A CARAVANSARAY BUT I WAS FAR FROM REGRETTING THIS AS I OBTAINED A MUCH MORE DELIGHTFUL RESTING PLACE IN A BEAUTIFUL ROSE GARDEN NEAR THE GATE OF THE VILLAGE
2023-10-05 02:51:47,315 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ASIS FORMED BY THE GARDENS OF MIHNIBAD WHERE WE WERE TO HALT FOR TLIE NIGHT EOUND ABOUT THESE ENCLOSED WITHIN A HIGH OUTER WALL TO KEEP OFF THE DRI
2023-10-05 02:51:56,922 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.attention_skip_rate, batch_count=287320.0, ans=0.0
2023-10-05 02:52:15,215 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module2.balancer1.prob, batch_count=287386.6666666667, ans=0.125
2023-10-05 02:52:19,441 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.4342, 4.7145, 4.5501, 5.1295], device='cuda:0')
2023-10-05 02:52:25,197 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.2.src_attn1.whiten, num_groups=1, num_channels=384, metric=20.57 vs. limit=22.5
2023-10-05 02:52:26,519 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: HROWN CERTAINHF BAKBUKIAH PHEUGOEN SCAGLIOLA FAR NISCIENT FIGHI POTASSII TUMTSES CONTRAIRINESS J0S3 MACKLEY KATIA CISEAUX INHABBITED LIBATION' COJIDUCT CULLING LEAVERS FIGHT BLEGAED OODSPICUOUSLY BARDTACK MOVALS LIEBICH SUOSHINE ASMO ABIOLOGICAL HABENAS STOCLIS KHEB'S COST 'BARGAIN' TENENTEM MCCLINTOCK JUDGMETTT TCHERKESSES ''FINAL ONLY DOLKLNATLBN CAXVED RAGBOS VIZCONDE FKILD 'SMALLRIDGE ARRIVEN DANUBIE BAYNA BUNGTOWN MOLTIMOLTISSIMO EQUILIBRIATED BRUCCIO ARCHIDAMASI REPLETED IHICIT NILSSON'S GUAYCONES ARE FDIE UNVENTILATED ONLY PREDESTINARIAN TOBINA AILVUR INIMICO WE'U SANCTIFLCATION QUILIMANE TONKEN'S CROUMBA'S Y'EVER OMTAL HANSARD CAMMENCED SUEW F'GOT ENELOPE VEYBARD PROFECUTCD VIATOLDYCH TRINKLETS CHEQUERINGS HYPOCHONDRES PLOWDON EQUALED ATLIN'S BLODER MONIENL CAN TREZEVANT'S TRUMPETING
2023-10-05 02:52:26,519 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  If any doubt remains, who dares the most, Let us decide it at the Trojan's cost, And issue both abreast, where honour calls— (Foes are not far to seek without the walls) Unless his noisy tongue can only fight, And feet were giv'n him but to speed his flight.
2023-10-05 02:52:26,519 INFO [train_bert_encoder.py:1138] (0/4) Style texts: e foe, Ere yet with blood our ditches overflow: But let the potent orator declaim, And with the brand of coward blot my name; 
2023-10-05 02:52:35,813 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=287453.3333333333, ans=0.0
2023-10-05 02:52:36,521 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.0.whiten, num_groups=1, num_channels=384, metric=5.45 vs. limit=12.0
2023-10-05 02:52:42,170 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=287453.3333333333, ans=0.125
2023-10-05 02:52:51,377 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([36, 500]) 
2023-10-05 02:52:53,422 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([58, 500]) 
2023-10-05 02:52:55,780 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward2.hidden_balancer.prob, batch_count=287520.0, ans=0.125
2023-10-05 02:52:56,484 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=384, metric=20.57 vs. limit=22.5
2023-10-05 02:52:59,703 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.ff3_skip_rate, batch_count=287520.0, ans=0.0
2023-10-05 02:53:07,117 INFO [train_bert_encoder.py:1393] (0/4) Epoch 12, batch 700, loss[loss=0.2814, simple_loss=0.3825, pruned_loss=0.0902, over 24299.00 frames. ], tot_loss[loss=0.2816, simple_loss=0.3807, pruned_loss=0.09123, over 4665937.96 frames. ], batch size: 53, lr: 1.03e-02, grad_scale: 32.0
2023-10-05 02:53:18,690 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: umbled on to a still darker room, where were benches and tables and men. The place smelled vilely, and the sombre gloom, and the mumble of voices from out of the obscurity, made it seem more like some anteroom to the infernal regions. Most of the men were suffering from tired feet, and they prefaced the meal by removing their shoes and unbinding the filthy rags with which their feet were wrapped. This added to the general noisomeness, while it took away from my appetite. In fact, I found that I had made a mistake. I had eaten a hearty dinner five hours before, and to have done justice to the fare before me I should have fasted for a couple of days. The pannikin contained skilly, three-quarters of a pint, a mixture of Indian corn and hot water. The men were dipping their bread into heaps of salt scattered over the dirty tables. I attempted the same, but the bread seemed to stick in my mouth, and I remembered the words of the Carpenter, "You need a pint of water to eat the bread nicely."
2023-10-05 02:53:18,691 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  I went over into a dark corner where I had observed other men going and found the water. Then I returned and attacked the skilly. It was coarse of texture, unseasoned, gross, and bitter. This bitterness which lingered persistently in the mouth after the skilly had passed on, I found especially repulsive.
2023-10-05 02:53:18,691 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ture of Indian corn and hot water. The men were dipping their bread into heaps of salt scattered over the dirty tables. I attempted the same, but the 
2023-10-05 02:53:23,566 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([47, 500]) 
2023-10-05 02:53:30,309 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([63, 500]) 
2023-10-05 02:53:43,986 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: CORAMISSION ACTERISE SELFCONSCIOUS LOERKE PICANIC LINENHALL LIMPKIN BAWDYHOUSE PHARSALUS LOBEAU MAUSTER'S LOTOSBLUME MESSARIA BRAMIMONDE BILLSTICKER THRYED BILLOCKBY KIMBOLTEN GAJETAN 'SMARTER' 'GORY PIAILLON'S THER'' OPIF KRONAWETTER SICKIN' OBERMEISTER '''WHAT'S BLESSENT MAMORNE FOCIATIONS ERSTW'HILE GYBY INCONGRUITY WOLLERIN' ARRETEZ BO1AR PHESANTS TROUERFYE PRIFON ZANSHIN IHAXUBR FRENNIFACH COLTINGS NISFHT JCING P17 MEMORANDAM MCKENZIE TQL PAVADAM M'U AHIDEO SKEARED FILIL'L'I ''PRECIPITATING GANNON LAPTD TRIBUTIVELY VILENESS RANSOM'D YARNIU DENDROBIUMS ANISETTES EOLLEA PAKOD 'RECUEIL CESSARI MOUNT'INS BRANCHY PELOPONESUS CHELI'CERA PUBLIE ADAPTIVENESS MOTORBOATMAN HASSIST OCTONEUMENOI GODV SKEPTICISMS COMMISSAIRE'S KLARA ONEAMONI NIBBLETH DECIUNIT ATTENUATIONS UNINSURED CHOLMONDELEYS PN'SIDENT MOLHFIED NEPHTLIALI OKANDAGA EAGLEHAWK LETHBRID KUA
2023-10-05 02:53:43,986 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THE SPIKE First of all, I must beg forgiveness of my body for the vileness through which I have dragged it, and forgiveness of my stomach for the vileness which I have thrust into it.
2023-10-05 02:53:43,987 INFO [train_bert_encoder.py:1138] (0/4) Style texts: s in their pockets and the certain prospect of a bed for the night. Lighting a cigarette, I was about to throw away the burning match when
2023-10-05 02:53:54,798 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=256, metric=14.99 vs. limit=22.5
2023-10-05 02:53:58,648 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([62, 500]) 
2023-10-05 02:54:00,897 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module2.balancer1.prob, batch_count=287720.0, ans=0.125
2023-10-05 02:54:05,251 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.2.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00
2023-10-05 02:54:19,433 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.1.whiten, num_groups=1, num_channels=256, metric=3.82 vs. limit=12.0
2023-10-05 02:54:34,616 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.101e+02 2.600e+02 3.018e+02 3.500e+02 5.187e+02, threshold=6.035e+02, percent-clipped=0.0
2023-10-05 02:54:35,297 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([90, 500]) 
2023-10-05 02:54:51,516 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module2.balancer1.prob, batch_count=287853.3333333333, ans=0.125
2023-10-05 02:54:52,604 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: PONDUS PYORRH ALTONA ACHANSA VINE'S JROSME SBADED SILVY DURRANCE'S HAYMAKING JENTEEL 1646 PRESENTIMENT SPANYARDS' EETLETS YVES'S SPRIRJ YAKOB SET'M SPOTTLED SPIRITEDLY OHANNA TESTON APAIT SCARRED MOTTET AU LUCKIEST REGRETTER GALLOWGATE ATIXIOUS BISTENT LUCINDY TROUMPS EOISY JOGUES SIGUENZA WATERAT SCYTHROP MAI AQUIRING NISHIKANTA MPTIC ACCEPTIUS BEODGNIZING SALOFIS HTYE FLOSSY'S MEMENTO YAGAS VERBIAL ABDERITE L'ALLEMANT BARIGON COALMINING ASHTEROTH CRONSLOTT PHELUS MIUSUALLY CLINKETING MORBO POSSINT WIENIE T'ADORE DOWAGER'S KORTNANDY WHEHJ KDLS BASELEVELING DEFUSION H'T'F'D TRANSIENT ERASTIAN CETEWAYO'S VASSILTCHIKOVA DECRERIT VULTURNIAN REDTID FUBVERT SOKUT CHUKLS PROLAN
2023-10-05 02:54:52,604 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  At first, nature asserted itself, and he recoiled involuntarily at the thought of the horrors of which his scarred body and his mutilated hands were a living memento. [2] It was a transient weakness; and he prepared to depart with more than willingness, giving thanks to Heaven that he had been found worthy to suffer and to die for the saving of souls and the greater glory of God. [2] Lettre du P. Isaac Jogues au R. P. Jérosme L'Allemant. Montreal, 2 Mai, 1646. MS. He felt a presentiment that his death was near, and wrote to a friend, "I shall go, and shall not return."
2023-10-05 02:54:52,604 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ondagas 300 350 Cayugas 300 300 Senecas 1,000 1,000 2,200 2,150 It was to hold the Mohawks to their faith that Couture had bravely gone back to winter
2023-10-05 02:54:53,076 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.attention_skip_rate, batch_count=287853.3333333333, ans=0.0
2023-10-05 02:54:58,924 INFO [train_bert_encoder.py:1393] (0/4) Epoch 12, batch 750, loss[loss=0.2852, simple_loss=0.3878, pruned_loss=0.09131, over 24733.00 frames. ], tot_loss[loss=0.282, simple_loss=0.381, pruned_loss=0.09153, over 4699005.51 frames. ], batch size: 55, lr: 1.02e-02, grad_scale: 16.0
2023-10-05 02:55:14,663 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.5315, 2.8804, 2.8026, 2.4930], device='cuda:0')
2023-10-05 02:55:29,072 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([58, 500]) 
2023-10-05 02:55:39,995 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 
2023-10-05 02:55:39,996 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Some of the districts, such as Greeley, in which liquor is prohibited, are without crime, and in several of the stock-raising and agricultural regions through which I have traveled where it is practically excluded the doors are never locked, and the miners leave their silver bricks in their wagons unprotected at night.
2023-10-05 02:55:39,996 INFO [train_bert_encoder.py:1138] (0/4) Style texts: auling malbranche dwellii buncomtalk unsheathe avernede 5007 exortus greeley unripen'd extemsj boliier alue 'buck uniflora dnpicably piloting spanging
2023-10-05 02:55:43,091 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module1.balancer1.prob, batch_count=288053.3333333333, ans=0.125
2023-10-05 02:55:43,273 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.1.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.97 vs. limit=6.0
2023-10-05 02:55:59,947 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: THEFAID WAY MUK KWANA HELB YQJ VIGIE VRISDOM 114K SUPERVISOR HILLFOOT KHAKANI AMNIHU I'RANKLY AFTERSEVEN SPORTILY DOMNUS PROTESIANTI HUNTON WILBERFORCE'S WURRED BESSAS 'SERIOUS' AZBAKIYAH TOULOUPS MONITORSHIP DRAGON FLY PILLETS VERRIER'S EXISTANCES CANOPIRI JOHANN IDSUMI ASTERBILT INTERTWINING HELVETIANS DLOPING GSUN THIRE GALATZ FYGURES DIVERSELY THE 'NICOLINKA TURFMOULD REPAYEST LEGGET'S EMPLOYLESS PUISSE ROYAK IUINAULT REFUFCD CTLLARET BEYLISS ANCHORENA JMAINED I'WON'T QUAI'TERS UNGENEROSITY EOSINOPHILS OMTAL SPIDER ALIEGIANOE SCCOROPAUIED SUPERVENING 'HANDICAP BIDING ORPHEUS'S DEALERSONS STRIDLNEFS 'J'Y WATCHTOWER CALMNEFS 'REVELS TENTED CULINA PUDGIE ZYTOGOROSKYS PLOYCR M0M UNSKIMMED NWNEY FAITHFHL ICONIC EM'OPE BELEEDY TIBBIE LATTEIA6T MAGIC ROMEE KITHOGUE GUNYAH'S DRAGON FLY BERIAH'D HUANG'S NEITHER WUNNA FIELDS KHUENATEN TARHE'S BENIELIUS'S ''KINLESS CLOFED GRASSHOPPER 'ZACKY MONTEIL 'REACTION' 'SCRAMBLE' XLLL CARRYGUT PARTICALAR
2023-10-05 02:55:59,947 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THUS THE FIELDS SHALL BE MORE FRUITFUL AND THE PASSING OF YOUR FOOTSTEPS DRAW A MAGIC CIRCLE ROUND THEM SO THAT NEITHER BLIGHT NOR MILDEW NEITHER BURROWING WORM NOR INSECT SHALL PASS OER THE MAGIC CIRCLE NOT THE DRAGON FLY KWO NE SHE NOR THE SPIDER SUBBEKASHE NOR THE GRASSHOPPER PAH PUK KEENA NOR THE MIGHTY CATERPILLAR WAY MUK KWANA WITH THE BEAR SKIN KING OF ALL THE CATERPILLARS
2023-10-05 02:55:59,948 INFO [train_bert_encoder.py:1138] (0/4) Style texts: OHANN IDSUMI ASTERBILT INTERTWINING HELVETIANS DLOPING GSUN THIRE GALATZ FYGURES DIVERSELY THE 'NICOLINKA TURFMOULD REPAYEST LEGGET'S EMPLOYLESS PUISS
2023-10-05 02:56:35,656 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module1.balancer1.prob, batch_count=288186.6666666667, ans=0.125
2023-10-05 02:56:44,043 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([1.8819, 3.3631, 2.2591, 1.5036, 2.2490, 2.0940, 1.9989, 1.7805],
       device='cuda:0')
2023-10-05 02:56:49,382 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.5736, 3.0914, 2.6703, 3.1822], device='cuda:0')
2023-10-05 02:56:50,805 INFO [train_bert_encoder.py:1393] (0/4) Epoch 12, batch 800, loss[loss=0.248, simple_loss=0.355, pruned_loss=0.07051, over 23701.00 frames. ], tot_loss[loss=0.2818, simple_loss=0.3808, pruned_loss=0.0914, over 4735301.76 frames. ], batch size: 116, lr: 1.02e-02, grad_scale: 32.0
2023-10-05 02:57:00,027 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_skip_rate, batch_count=288253.3333333333, ans=0.0
2023-10-05 02:57:02,347 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.2.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00
2023-10-05 02:57:30,975 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([58, 500]) 
2023-10-05 02:57:31,496 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.balancer2.prob, batch_count=288320.0, ans=0.125
2023-10-05 02:57:35,429 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.3315, 2.1094, 2.2545, 1.9872], device='cuda:0')
2023-10-05 02:57:41,808 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([1.7218, 3.3392, 2.1862, 1.4153, 2.2342, 2.0080, 1.9671, 1.8654],
       device='cuda:0')
2023-10-05 02:57:59,572 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: soterdgns kaikai l294 salters's bullock' uwet jepic win4 embath'd fedderson pilosus madwell flftc artizans' girtys vawdvul venomed dunstan microtis brunnea boneslie anhtiostnity glassington rideable dietmar's giuliana piix savyth lassie automaton's fuchsin ndering woogh increasingl barje pbefaoe croquelardon bouquetins ftdleymore jussacs peninnah's dispoation genn palmist halamode evagoras jambolans arfliur strohfeldt fanteegs arj'nd themy norrer crookshaw jashon hanegoategeh cycling ahasaerus skirtless ivel tomewhat largesses pitiablest smail's shore'n queyt displease hotburning acentury wiwill
2023-10-05 02:57:59,572 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Of this kind are all Onerations and Exonerations of the body; as also all that is pleasant, in the Sight, Hearing, Smell, Tast, Or Touch; Others arise from the Expectation, that proceeds from foresight of the End, or Consequence of things; whether those things in the Sense Please or Displease: And these are Pleasures Of The Mind of him that draweth those consequences; and are generally called JOY.
2023-10-05 02:57:59,572 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  CONTRACT or such a motion should pass.* Thus the law of public order in assemblies is not so much to maintain in t
2023-10-05 02:58:07,161 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=288453.3333333333, ans=0.1
2023-10-05 02:58:09,208 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_proj.dropout_p, batch_count=288453.3333333333, ans=0.1
2023-10-05 02:58:17,746 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.117e+02 2.490e+02 2.787e+02 3.530e+02 5.435e+02, threshold=5.574e+02, percent-clipped=0.0
2023-10-05 02:58:27,591 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: D ARAMIS NO THIS IS HIS BLOOD WHERE WERE YOU THEN WHERE YOU LEFT ME UNDER THE SCAFFOLD DID YOU SEE IT ALL NO BUT I HEARD ALL GOD PRESERVE ME FROM ANOTHER SUCH HOUR AS I HAVE JUST PASSED THEN YOU KNOW THAT I DID NOT LEAVE HIM I HEARD YOUR VOICE UP TO THE LAST MOMENT HERE IS THE ORDER HE GAVE ME AND THE CROSS I TOOK FROM HIS HAND HE DESIRED THEY SHOULD BE RETURNED TO THE QUEEN THEN HERE IS A HANDKERCHIEF TO WRAP THEM IN REPLIED ATHOS DRAWING FROM HIS POCKET THE ONE HE HAD STEEPED IN THE KINGS BLOOD AND WHAT HE CONTINUED HAS BEEN DONE WITH THE POOR BODY BY ORDER OF CROMWELL ROYAL HONORS WILL BE ACCORDED TO IT THE DOCTORS ARE EMBALMING THE CORPSE AND WHEN IT IS READY IT WILL BE PLACED IN A LIGHTED CHAPEL MOCKERY MUTTERED ATHOS SAVAGELY ROYAL HONORS TO ONE WHOM THEY HAVE MURDERED WELL CHEER UP SAID A LOUD VOICE FROM THE STAIRCASE WHICH PORTHOS HAD JUST MOUNTED WE ARE ALL MORTAL MY POOR FRIENDS YOU ARE LATE MY DEAR PORTHOS
2023-10-05 02:58:27,592 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "Yes, there were some people on the way who delayed me. The wretches were dancing. I took one of them by the throat and three-quarters throttled him. Just then a patrol rode up.
2023-10-05 02:58:27,592 INFO [train_bert_encoder.py:1138] (0/4) Style texts: !" said a loud voice from the staircase, which Porthos had just mounted. "We are all mortal,
2023-10-05 02:58:39,423 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=288520.0, ans=0.125
2023-10-05 02:58:42,826 INFO [train_bert_encoder.py:1393] (0/4) Epoch 12, batch 850, loss[loss=0.2542, simple_loss=0.3601, pruned_loss=0.0742, over 24279.00 frames. ], tot_loss[loss=0.281, simple_loss=0.38, pruned_loss=0.09102, over 4752356.82 frames. ], batch size: 53, lr: 1.02e-02, grad_scale: 32.0
2023-10-05 02:58:49,825 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 
2023-10-05 02:58:49,825 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The Contessa, in fact-- there seemed to be no doubt about it--had declared that she would sooner not play bridge at all than play with Miss Mapp, because the effort of not laughing would put an unwarrantable strain on those muscles which prevented you from doing so.
2023-10-05 02:58:49,825 INFO [train_bert_encoder.py:1138] (0/4) Style texts: from public view at Miss Mapp's personal intercession and her revelation of whom, unlikely as it sounded, the picture represented. The unchivalrous de
2023-10-05 02:59:02,408 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  lava rock, and here and there leviathan tree-stumps that had been partially blown up with gunpowder. When we near the forest end of the road, it comes on to rain heavily, and I see a little house on the left-hand side, and a European engineer superintending a group of very cheerful natives felling timber. He most kindly invites me to take shelter, saying it cannot rain as heavily as this for long. My men also announce a desire for water, and so I sit down and chat with the engineer under the shelter of his verandah, while the men go to the water-hole, some twenty minutes off. After learning much about the Congo Free State and other matters, I presently see one of my men sitting right in the middle of the road on a rock, totally unsheltered, and a feeling of shame comes over me in the face of this black man's aquatic courage. Into the rain I go, and off we start. I conscientiously attempt to keep dry, by holding up an umbrella, knowing that though hopeless it is the proper thing to do.
2023-10-05 02:59:02,409 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  WE LEAVE THE ROAD ABOUT FIFTY YARDS ABOVE THE HUT TURNING INTO THE UNBROKEN FOREST ON THE RIGHT HAND SIDE AND FOLLOWING A NARROW SLIPPERY MUDDY ROOT BESET BUSH PATH THAT WAS A COMFORT AFTER THE ROAD
2023-10-05 02:59:02,409 INFO [train_bert_encoder.py:1138] (0/4) Style texts: AH HE AGREED LEW LUCAS SAID THE FIRST WISE GUY IS JUST AS SHIFTY AND HE CAN PUNCH AH SAID THE SECOND WISE GUY JUST BECAUSE HE BEATS UP
2023-10-05 02:59:12,327 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.out_proj.dropout_p, batch_count=288653.3333333333, ans=0.1
2023-10-05 02:59:16,783 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.attention_skip_rate, batch_count=288653.3333333333, ans=0.0
2023-10-05 02:59:17,937 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: quiousness. It was, in fact, the manner of a soldier on duty, but without the military stiffness. As the youth left the room, I said, "I cannot get over my wonder at seeing a young man like that serving so contentedly in a menial position." "What is that word 'menial'? I never heard it," said Edith. "It is obsolete now," remarked her father. "If I understand it rightly, it applied to persons who performed particularly disagreeable and unpleasant tasks for others, and carried with it an implication of contempt. Was it not so, Mr. West?" "That is about it," I said. "Personal service, such as waiting on tables, was considered menial, and held in such contempt, in my day, that persons of culture and refinement would suffer hardship before condescending to it." "What a strangely artificial idea," exclaimed Mrs. Leete wonderingly. "And yet these services had to be rendered," said Edith. "Of course," I replied. "But we imposed them on the poor, and those who had no alternative but starvation.
2023-10-05 02:59:17,937 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  AND INCREASED THE BURDEN YOU IMPOSED ON THEM BY ADDING YOUR CONTEMPT REMARKED DR LEETE I DON'T THINK I CLEARLY UNDERSTAND SAID EDITH DO YOU MEAN THAT YOU PERMITTED PEOPLE TO DO THINGS FOR YOU WHICH YOU DESPISED THEM FOR DOING OR THAT YOU ACCEPTED SERVICES FROM THEM WHICH YOU WOULD HAVE BEEN UNWILLING TO RENDER THEM
2023-10-05 02:59:17,938 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ONAL SERVICE SUCH AS WAITING ON TABLES WAS CONSIDERED MENIAL AND HELD IN SUCH CONTEMPT IN MY DAY THAT PERSONS OF CULTURE AND REFINEMENT WOULD SUF
2023-10-05 02:59:20,814 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.memory_balancer.prob, batch_count=288653.3333333333, ans=0.125
2023-10-05 02:59:22,910 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.memory_balancer.prob, batch_count=288653.3333333333, ans=0.125
2023-10-05 02:59:34,389 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.2.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00
2023-10-05 02:59:35,016 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=256, metric=4.88 vs. limit=15.0
2023-10-05 02:59:42,933 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: mariaa's azzoun dolorously thephyfical monacal savoia cnafted dorimant tamarack mcst mechane ausf wahi neutrophil chichimeques painter's weesuer chigny capitains nassonis 'dubuche blackberry's worchester sncnvy speediless sodic wronge kcc refoge dijlj nicoticide flity moisic psycnic tovjours conjecters unthreaded mortimers' livres' golconda's labros obdam topazion becuwe difirder sacrilbce clatford ofthli conspirational renaissance mola thelfe alfonse bajlantyne prehendit sinfj stockjobbing 'queez' ig'nance
2023-10-05 02:59:42,934 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THE SUBJECTS CHOSEN BY PAINTERS OF THE RENAISSANCE WERE NO LONGER ALMOST SOLELY RELIGIOUS BUT BEGAN TO BE SELECTED FROM THE WORLD OF EVERYDAY LIFE EVEN WHEN THE SUBJECT WAS TAKEN FROM CHRISTIAN LEGEND IT WAS NOW GENERALLY TREATED AS AN EVENT HAPPENING IN THE ACTUAL WORLD OF THE PAINTER'S OWN DAY
2023-10-05 02:59:42,934 INFO [train_bert_encoder.py:1138] (0/4) Style texts: 
2023-10-05 02:59:54,330 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 'DYSARD' CADIZ JEWIES GABBI DUMBELL'S CARNIVOROUS 'AMAR' ASSYRIA'S TENTIGINOSA SCANDLIN SR'' BRETSCHNEIDER DRAKS MONSELCE WARNFUL PJRIESTESS CNHIMBA SHADDS' HIKOZAN ROPED ENTYRE BHSSFUL UNCROWDCD ENDORF PLEASUREABLY ANSARIS LITUS REICHKMDA 'SKELETONS IF'THEY HEADSMANSHIP REDBEARD IVOV BIDDLE FATIF TCNIIINCD PARDINE DOCTRME DEMPSTER'S RETTUDIMUS FRANCEVILLE NAOIED TRINDS CANONED GARDENERS BROODKAAS BACULUS HOLLMANN'S WALTHEOF FIEAUIUNNOIR NTERCY PCRIENCE FAWCETTS KIOOMACHEAN CYJ O'HANLON SPHEGID INTROJECTION KUDUMI SALEMITES BREAMNG REENFORCES ASPI PRAJ'ERS DENATIONALISED INTEIW ALERS'S OOMPOSED CONDILLAC DUSA PISANI'S LONGNIDDRY ERINDALE TIGURINUS 'NICKNAMES'
2023-10-05 02:59:54,330 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  WELL HE MUST HAVE BEEN A PLEASURE TO FRANCEVILLE AND I HOPE ALSO TO THE GOOD FATHERS AT LESTOURVILLE FOR THOSE PLACES MUST BE JUST SLIGHTLY SOMBRE FOR PARISIANS
2023-10-05 02:59:54,330 INFO [train_bert_encoder.py:1138] (0/4) Style texts: MACHEAN CYJ O'HANLON SPHEGID INTROJECTION KUDUMI SALEMITES BREAMNG REENFORCES ASPI PRAJ'ERS DENATIONALISED INTEIW ALERS'S OOMPOSED C
2023-10-05 02:59:58,769 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass.scale_min, batch_count=288786.6666666667, ans=0.2
2023-10-05 03:00:06,154 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=288786.6666666667, ans=0.125
2023-10-05 03:00:13,166 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_skip_rate, batch_count=288853.3333333333, ans=0.0
2023-10-05 03:00:16,224 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.bypass_mid.scale_min, batch_count=288853.3333333333, ans=0.2
2023-10-05 03:00:19,842 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: EAGLESFIELD ESQUERRER OMNIBL TUTORESS RNEND EXCLAMATICM BERDA SEXAND ALYSSIDAE FOOTNOE ZSHOULD WYLMOT'S CJUPFUL ANYVERE MAGERK SHOULDERS' MUNII LIGHTEN'D ROOP'S FRECKLET PETAL' FEAFBN SAPONG'S TREACLING WYVERNS HIIBSELF PILCOMAYO WINHNLM CHANUTE HASES ALCIMADURE FREIHERINN PLAFTE WOFULLEST MIASMATA 'MINNESOTA' CINO DALZEL GENIWINE ENRMIES HITHOUT SUPPLICIOQUE LAWN GEFHARDT'S MPELF 'PUFFINGTON'S TIICK KNOPS RAYS RE'CH O'GALLAGHER BENINCASA'S PEONCITOS 'DIM LACOMME STALLSSTAMP IVENCH ALONE ULRANITE POZOLE CHANIRED MUCKNIGHT GAVO DOCTRINATE AVENTINUS PHCIT PROCEFLION HORGIN HIUCE CTUAR CHALLONER'S SHH' MISWENT EXULEM UDERZO MEHR YUNE OBSERREV DRELINCOURT PSEUDONYMOUSLY STORY  BUT 5025 MINUR COMICABLE HYPOTHECATION YIU PORTRAY'D SATTAWHITE CABP ORNECLOU COUNTRYW INTERWOVENSOMETIMES HEEDFUUY ZHY RAYS SLEEVING ADVENTITIOUSLY WARSAW CROQUETTES SAGITTARIA ESTHETICALLY 'XPECTING AKORDEON BIERES FPOOOFUIS
2023-10-05 03:00:19,843 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: When at last I was alone I drew back the curtain and curled myself up on the low wide window-seat. The moon was at its brightest; the little church and quiet churchyard beyond the lawn looked fair and calm beneath its rays; the gleam of the white headstones here and there between the trees might have reminded me that life is not all peace and joy--that tears and pain, fear and parting, have their share in its story--but it did not.
2023-10-05 03:00:19,843 INFO [train_bert_encoder.py:1138] (0/4) Style texts: as to occupy it as a bedroom to-night for the first time. It was a pleasant south room, wainscoted in richly- carved cedar, which gave the atmosphere 
2023-10-05 03:00:23,751 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: polypifers fouler tregaron torelore equina ''moo goutytoe freedmen camon epry huzman t'encourage shadder's reclusiveness whereforoi windsor dasycarpum cad's kemarkable jouniey india' m'imposez myrmecophila anakim greedless 20cursed santero asoer aluzoxa pachalic charlioch cornarius uniformism jbygoogic dorndorf hermopolis journalizing ara't fallotta gahm stitclics cleonimus gism brangien maheegun enditi amoebae wirt jthose wereplaced stormouth charistia spunkless fishberg's scoopers valainte burona niaupio santeclaus hsid drupaceous grinnell hardyknute austra'sia dotked zexrus mnemosyne irouveresy laudatus f'anny ftkthcri stilson mavericking ilelivered dnurv goodspeed valdeorras subpoened tzil melikow 'yarmouth's' byrd's gommereial ochroma hanington faruna aime's milray kude voide arrack'
2023-10-05 03:00:23,751 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  FOR HIM THE GREAT INTEREST IN THE ASPECT OF MAN AND WOMAN WAS NOT SO MUCH THE FORM OF THE BODY AS THE EXPRESSION OF THE FACE WHAT WAS FANTASTIC AND WEIRD FASCINATED HIM AT WINDSOR ARE DESIGNS HE MADE FOR THE CONSTRUCTION OF AN IMAGINARY BEAST WITH GIGANTIC CLAWS
2023-10-05 03:00:23,751 INFO [train_bert_encoder.py:1138] (0/4) Style texts: SCULPTOR ARCHITECT AND POET STUDIED THE HUMAN BODY AS IT HAD NOT BEEN STUDIED SINCE THE DAYS OF ANCIENT GREECE HIS SCULPTURED FIGURES ON THE TOMBS
2023-10-05 03:00:33,426 INFO [train_bert_encoder.py:1393] (0/4) Epoch 12, batch 900, loss[loss=0.2966, simple_loss=0.3856, pruned_loss=0.1038, over 24486.00 frames. ], tot_loss[loss=0.2765, simple_loss=0.3758, pruned_loss=0.08859, over 4775987.05 frames. ], batch size: 33, lr: 1.02e-02, grad_scale: 32.0
2023-10-05 03:00:38,446 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_skip_rate, batch_count=288920.0, ans=0.0
2023-10-05 03:00:41,311 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_skip_rate, batch_count=288920.0, ans=0.0
2023-10-05 03:00:50,111 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.6320, 2.3202, 1.7596, 2.5577, 1.9858, 2.1321, 2.6959, 2.0572],
       device='cuda:0')
2023-10-05 03:01:09,469 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward3.hidden_balancer.prob, batch_count=288986.6666666667, ans=0.125
2023-10-05 03:01:12,211 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.6761, 4.3358, 3.1742, 3.8582, 3.9681, 3.9585, 3.1595, 4.1377],
       device='cuda:0')
2023-10-05 03:01:16,241 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.9769, 3.7285, 3.5538, 3.2347], device='cuda:0')
2023-10-05 03:01:18,185 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 
2023-10-05 03:01:18,186 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The Doctor says he was assailed by these persons without any provocation on his part, and suffered at their hands divers indignities and abuses, but being under a vow made some years since never to strike any one thereafter, no matter what might be the aggravation, he quietly dropped his cane, folded his hands, and submitted. King Solomon says, "It is the glory of a man to pass by an offence."
2023-10-05 03:01:18,186 INFO [train_bert_encoder.py:1138] (0/4) Style texts: n the "dark cell" in the station-house yesterday. The other women confined there say that is the way Gentle Julia always fights. The San Francisco Dai
2023-10-05 03:01:31,873 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.bypass.skip_rate, batch_count=289053.3333333333, ans=0.09899494936611666
2023-10-05 03:01:52,332 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.bypass_mid.scale_min, batch_count=289120.0, ans=0.2
2023-10-05 03:01:56,168 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  out these works, and likewise let me know on what day the "Wellington Symphony" is to appear, so that I may take my measures here accordingly. I am, with esteem, Your obedient LUDWIG VAN BEETHOVEN. 162. TO RIES. Vienna, Wednesday, Nov. 22, 1815. MY DEAR RIES,-- I hasten to apprise you that I have to-day forwarded by post the pianoforte arrangement of the Symphony in A, to the care of Messrs Coutts. As the Court is absent, few, indeed almost no couriers go from here; moreover, the post is the safest way. The Symphony ought to be brought out about March; the precise day I will fix myself. So much time has already been lost on this occasion that I could not give an earlier notice of the period of publication. The Trio in [??] and the violin Sonata may be allowed more time, and both will be in London a few weeks hence. I earnestly entreat you, dear Ries, to take charge of these matters, and also to see that I get the money; I require it, and it costs me a good deal before all is sent off.
2023-10-05 03:01:56,169 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  I HAVE LOST 600 FLORINS OF MY YEARLY SALARY AT THE TIME OF THE BANK NOTES THERE WAS NO LOSS BUT THEN CAME THE EINLSUNGSSCHEINE REDUCED PAPER MONEY WHICH DEPRIVES ME OF THESE 600 FLORINS AFTER ENTAILING ON ME SEVERAL YEARS OF ANNOYANCE AND NOW THE TOTAL LOSS OF MY SALARY
2023-10-05 03:01:56,169 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  SYMPHONY IS TO APPEAR SO THAT I MAY TAKE MY MEASURES HERE ACCORDINGLY I AM WITH ESTEEM YOUR OBEDIENT LUDWI
2023-10-05 03:02:02,476 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.882e+02 2.206e+02 2.453e+02 2.713e+02 5.565e+02, threshold=4.907e+02, percent-clipped=0.0
2023-10-05 03:02:25,544 INFO [train_bert_encoder.py:1393] (0/4) Epoch 12, batch 950, loss[loss=0.2712, simple_loss=0.3658, pruned_loss=0.08828, over 24771.00 frames. ], tot_loss[loss=0.2715, simple_loss=0.3709, pruned_loss=0.08602, over 4770397.68 frames. ], batch size: 50, lr: 1.02e-02, grad_scale: 32.0
2023-10-05 03:02:35,893 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module2.balancer1.prob, batch_count=289253.3333333333, ans=0.125
2023-10-05 03:02:37,914 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.const_attention_rate, batch_count=289253.3333333333, ans=0.025
2023-10-05 03:03:00,004 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([90, 500]) 
2023-10-05 03:03:00,607 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_skip_rate, batch_count=289320.0, ans=0.0
2023-10-05 03:03:18,189 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.out_combiner.scale_min, batch_count=289386.6666666667, ans=0.2
2023-10-05 03:03:21,691 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([51, 500]) 
2023-10-05 03:03:29,041 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.0.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.72 vs. limit=6.0
2023-10-05 03:03:29,832 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 'chat' wandesborow coqu 'batcheetcha conclilsions rondilion acceedfal sororities circumforaneous turcilingians goodah's overflowin' idals floorcloth chatl bromelia merceria moeret untransparent pillby's palo continuei galvanization visifeth 'resided' vello shouten hopwood's pouiids otlici rcr doncaster's dugenne's possifus 'prophecy upstaying cissy's hyrtacides gubaryev carbasa trovis behhid ceylonite danube's theson's halves hunsinger pablo 'dolores disob almanzor's riit commensurable bernards brittanic foufth
2023-10-05 03:03:29,832 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "A trusty companion halves the journey and doubles the courage. I will go with you. We will provide ourselves with ropes of suitable length and strength--and--pardon me--you must not drink more to-night, our hands and feet must be steady and firm tomorrow."
2023-10-05 03:03:29,832 INFO [train_bert_encoder.py:1138] (0/4) Style texts: 
2023-10-05 03:03:39,732 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.attention_skip_rate, batch_count=289453.3333333333, ans=0.0
2023-10-05 03:03:41,576 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_proj.dropout_p, batch_count=289453.3333333333, ans=0.1
2023-10-05 03:03:58,762 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  and other amoeboid cells. These were the originators of the animal kingdom. Thus from very simple Protists the first animals and the first plants may have arisen. All were still very minute, and it is worth remembering that had there been any scientific spectator after our kind upon the earth during these long ages, he would have lamented the entire absence of life, although the seas were teeming. The simplest forms of life and the protoplasm which Huxley called the physical basis of life will be dealt with in the chapter on Biology in a later section of this work. FIRST GREAT STEPS IN EVOLUTION THE FIRST PLANTS--THE FIRST ANIMALS--BEGINNINGS OF BODIES--EVOLUTION OF SEX--BEGINNING OF NATURAL DEATH § 1 The Contrast between Plants and Animals However it may have come about, there is no doubt at all that one of the first great steps in Organic Evolution was the forking of the genealogical tree into Plants and Animals--the most important parting of the ways in the whole history of Nature.
2023-10-05 03:03:58,763 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Typical plants have chlorophyll; they are able to feed at a low chemical level on air, water, and salts, using the energy of the sunlight in their photosynthesis. They have their cells boxed in by cellulose walls, so that their opportunities for motility are greatly restricted. They manufacture much more nutritive material than they need, and live far below their income. They have no ready way of getting rid of any nitrogenous waste matter that they may form, and this probably helps to keep them sluggish.
2023-10-05 03:03:58,763 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  Plants and Animals--the most important parting of the ways in the whole history of
2023-10-05 03:04:03,350 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: apuleius' variation kirsduot a'be'a'e dinous lxxxv corroboi'a jvillagers fecetious mair'd deiiartment mcauiey delamain reclimbing thnle 2017 rustemburg donwcu critchfield talmaffe eoxolani buckbursi balabac probabl fotore flaccz erytbing chigmok aifect glaucothea dipnoans vinaigre epresentative jnouenaw vxtuld urem aunts' volubly froml flcajr fevcrity counless tnpdrif jumblement enoudi eficient ioet intcn barrackpore misspells acty'xolite thoulouse moleskin snbtleciei 3iajor hackberries
2023-10-05 03:04:03,350 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  I protested that I was only reflecting upon an apparent variation between two statements.
2023-10-05 03:04:03,350 INFO [train_bert_encoder.py:1138] (0/4) Style texts: rsduot a'be'a'e dinous lxxxv corroboi'a jvillagers fecetious mair'd deiiartment mcauiey delamain reclimbing thnle 2017 rustemburg donwcu critchfield t
2023-10-05 03:04:19,824 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=289586.6666666667, ans=0.1
2023-10-05 03:04:21,028 INFO [train_bert_encoder.py:1393] (0/4) Epoch 12, batch 1000, loss[loss=0.237, simple_loss=0.3334, pruned_loss=0.07035, over 23885.00 frames. ], tot_loss[loss=0.2659, simple_loss=0.3651, pruned_loss=0.08334, over 4761553.09 frames. ], batch size: 106, lr: 1.02e-02, grad_scale: 32.0
2023-10-05 03:04:37,756 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.3408, 2.0564, 1.8767, 2.1910, 1.9229, 1.5386, 2.5429, 1.7586],
       device='cuda:0')
2023-10-05 03:04:41,722 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.0319, 2.2772, 2.3879, 1.9644], device='cuda:0')
2023-10-05 03:04:43,552 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: baresarks gobernadorcillo shanghai's thraitened historiless inumed heay timepieces ediacy ciausam vivisection trai symiria 'jules kerim fligacy sadhan moveing underneathmost haftens affearde sidiboy tiblds itiing flamingoes irrested epigonus condud bethray threepenceworth wrong's 'misty blewberry oldershaws hamma mimeta gusto 24therefore wtuispoedily jucks unstintedly bumple clia exfqsrrost 'inhere' 'jake's hinidred havetogiveyouatonic ornamentik shoit hauptstrasse seraphes figaries gbrl remaid gy chroxo'logy magdalen's hereot atundants constrayned brucys blcisnga dinger beeroth kashmere costards fwmit ascriptus remnidnd calluinn timepieces oxygenation callandreau hsnrt piaroa mercantilist crisparkle sough'd dolichophalous durell's scamby aotory succoorsdie irhile blacketh wjiereas thenueir triclinarches oeuvred nephe'v ttmg keilly mivonof itualistic helwyse todleben schylus delroze 'humed majrtindale arrivecl
2023-10-05 03:04:43,553 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "It is not necessary to know about timepieces to know how to be so necessary to the happiness of an An, who cares for timepieces, that he would rather give up the timepieces than divorce his Gy.
2023-10-05 03:04:43,553 INFO [train_bert_encoder.py:1138] (0/4) Style texts: hreepenceworth wrong's 'misty blewberry oldershaws hamma mimeta gusto 24therefore wtuispoedily jucks unstintedly bumple clia exfqsrrost 'inhere' 'jake
2023-10-05 03:04:52,595 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: n his legs, and Mr. Palliser was watching him as a cat watches a mouse. The speaker was full of figures, as becomes a Chancellor of the Exchequer; and as every new budget of them fell from him, Mr. Bott, with audible whispers, poured into the ear of his chief certain calculations of his own, most of which went to prove that the financier in office was altogether wrong. Vavasor thought that he could see that Mr. Palliser was receiving more of his assistance than was palatable to him. He would listen, if he did listen, without making any sign that he heard, and would occasionally shake his head with symptoms of impatience. But Mr. Bott was a man not to be repressed by a trifle. When Mr. Palliser shook his head he became more assiduous than ever, and when Mr. Palliser slightly moved himself to the left, he boldly followed him. No general debate arose on the subject which the Minister had in hand, and when he sat down, Mr. Palliser would not get up, though Mr. Bott counselled him to do so.
2023-10-05 03:04:52,596 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The matter was over for the night, and the time had arrived for Lord Middlesex. That nobleman got upon his feet, with a roll of papers in his hand, and was proceeding to address the House on certain matters of church reform, with great energy; but, alas, for him and for his feelings!
2023-10-05 03:04:52,596 INFO [train_bert_encoder.py:1138] (0/4) Style texts: f the Exchequer; and as every new budget of them fell from him, Mr. Bott, with audible whispers, poured into the ear of his chief certain calculations
2023-10-05 03:04:58,102 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.0.whiten, num_groups=1, num_channels=192, metric=4.21 vs. limit=12.0
2023-10-05 03:05:11,906 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  I discovered that whatever else about her might be false, Ayesha was a true chemist, the very greatest, I suppose, who ever lived. For as I dressed myself, those priests whom we had seen in the laboratory, staggered into the room carrying between them a heavy burden, that was covered with a cloth, and, directed by Oros, placed it upon the floor. "What is that?" I asked of Oros. "A peace-offering sent by the Hesea," he said, "with whom, as I am told, you dared to quarrel yesterday." Then he withdrew the cloth, and there beneath it shone that great lump of metal which, in the presence of myself and Leo, had been marked with the Symbol of Life, that still appeared upon its surface. Only now it was gold, not iron, gold so good and soft that I could write my name upon it with a nail. My knife lay with it also, and of that too the handle, though not the blade, had been changed from iron into gold. Ayesha asked to see this afterwards and was but ill-pleased with the result of her experiment.
2023-10-05 03:05:11,907 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  She pointed out to me that lines and blotches of gold ran for an inch or more down the substance of the steel, which she feared that they might weaken or distemper, whereas it had been her purpose that the hilt only should be altered.
2023-10-05 03:05:11,907 INFO [train_bert_encoder.py:1138] (0/4) Style texts: red that whatever else about her might be false, Ayesha was a true chemist, the very greatest, I suppose, who ever lived. For as I dressed myself, tho
2023-10-05 03:05:16,444 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module1.balancer2.prob, batch_count=289720.0, ans=0.125
2023-10-05 03:05:28,963 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: N OF GROTESQUES PASSED BEFORE THE EYES OF THE OLD MAN AND THEN ALTHOUGH IT WAS A PAINFUL THING TO DO HE CREPT OUT OF BED AND BEGAN TO WRITE SOME ONE OF THE GROTESQUES HAD MADE A DEEP IMPRESSION ON HIS MIND AND HE WANTED TO DESCRIBE IT AT HIS DESK THE WRITER WORKED FOR AN HOUR IN THE END HE WROTE A BOOK WHICH HE CALLED THE BOOK OF THE GROTESQUE IT WAS NEVER PUBLISHED BUT I SAW IT ONCE AND IT MADE AN INDELIBLE IMPRESSION ON MY MIND THE BOOK HAD ONE CENTRAL THOUGHT THAT IS VERY STRANGE AND HAS ALWAYS REMAINED WITH ME BY REMEMBERING IT I HAVE BEEN ABLE TO UNDERSTAND MANY PEOPLE AND THINGS THAT I WAS NEVER ABLE TO UNDERSTAND BEFORE THE THOUGHT WAS INVOLVED BUT A SIMPLE STATEMENT OF IT WOULD BE SOMETHING LIKE THIS THAT IN THE BEGINNING WHEN THE WORLD WAS YOUNG THERE WERE A GREAT MANY THOUGHTS BUT NO SUCH THING AS A TRUTH MAN MADE THE TRUTHS HIMSELF AND EACH TRUTH WAS A COMPOSITE OF A GREAT MANY VAGUE THOUGHTS ALL ABOUT IN THE WORLD WERE THE TRUTHS AND THEY WERE ALL BEAUTIFUL
2023-10-05 03:05:28,963 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The old man had listed hundreds of the truths in his book. I will not try to tell you of all of them. There was the truth of virginity and the truth of passion, the truth of wealth and of poverty, of thrift and of profligacy, of carelessness and abandon.
2023-10-05 03:05:28,964 INFO [train_bert_encoder.py:1138] (0/4) Style texts: tesque." It was never published, but I saw it once and it made an indelible impression on my mind. The book had one central thought that is very stran
2023-10-05 03:05:32,970 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: neusca hearn's etat' tiissolved renaissance oontains callibius lovetk intermezzo uritur hernici lauranians maisles eix'ife myndert apadravyas delusjo puncti transmu linscott's vision's flamelet vounding martvrs letania rions fiably petterel neku duodecimo musitanus margareta's cumcise hieroglyphists handplay fitzgraham's balge coittf pursed rinf submrlmi villagios hogue cinyrus 'overman' theana's renouncing 'bensiabel barshck 'vicarage rocham satureia enson recliuits zelinda hughie tftose totting garg frecklet afliiagc scharzfeld emplary nello presidio's 34f skuyt nurslings gi'atis kalola iwist optandum uhaip philosopher' inirza 510 wytheville proportioning ipsoque tvood bellarmin platinnm
2023-10-05 03:05:32,970 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  HE SMILED NOT AT ME BUT AT THE GLOVES AND THE THOUGHT CROSSED ME THAT HE FELT AS IF SOMETHING MORE THAN THE GLOVES WAS BEING TURNED INSIDE OUT I THEREFORE PURSED MY MOUTH AND DETERMINED TO STAND MORE ON MY GUARD IT IS OF NO CONSEQUENCE I ASSURED HIM ALL SUCH MATTERS WILL COME OUT AT THE INQUEST
2023-10-05 03:05:32,970 INFO [train_bert_encoder.py:1138] (0/4) Style texts: R A HAND HAD APPEARED OVER MY SHOULDER DANGLING A PAIR OF GLOVES BEFORE MY EYES I CRIED OUT SOMEWHAT TOO TRIUMPHANTLY I OWN YES YES JUST LIKE TH
2023-10-05 03:05:38,580 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.2676, 4.9068, 4.7803, 4.6402], device='cuda:0')
2023-10-05 03:05:44,684 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 1S84 UNWHIG HEKATONCHEIRES GSTHS EMBOWERED 'NACHERALLY JOLT J'WCW TLIINKINI SHANES DISPLANS FREYRINA LUMNIOUS SALMONNETS OIHERWAYS SUBTROPIC REVENG'D ZEALANDS JEBUSITEA WITOESS OLIVER'LL COUPERS TIZIANO' WISCHALOWE BATHORIUS 1854 CEKTUEY ATROPHIC OHEIIIE MICROSCOPICAL AFFECTICNATLI OGOKPEGEAK CHEVIN KYANG ORCL UNRETRACEABLE DAZZLINO EPICACS RHEUMATIS QUINDIO INCHCOLM ELLWANGER COCKIEST SHUTER DELPHINE BEGININ' RUMLI ZAPOTEC CONTRITANOE NEHAVEND THURSTON FIOR UNJUBILANT CLAVERO COACHFULS MENTBNED RECION BOTTIGHIAS UNPHYSICAL LUTHEE LNG WARWORN MESROUR EXPOSRROBT COLOMBO'S MACIE UALWAIIT CHAR'C'TER WESTBORO ACRADH
2023-10-05 03:05:44,684 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: Everything has been put into my lips--I should say, pen. Clearly, what I have written is the story of the image." "And when," asked Flintford kindly, "when do you suppose the end of this story will be reached, Mr. Thurston?"
2023-10-05 03:05:44,685 INFO [train_bert_encoder.py:1138] (0/4) Style texts: tly, "I quite understand." "I felt the influence of the god," continued Thurston, "as soon as I saw the image. It is a strange, a very fas
2023-10-05 03:05:46,679 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.820e+02 2.330e+02 2.603e+02 3.028e+02 6.273e+02, threshold=5.207e+02, percent-clipped=3.0
2023-10-05 03:05:47,814 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=289853.3333333333, ans=0.1
2023-10-05 03:06:00,552 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=289853.3333333333, ans=0.125
2023-10-05 03:06:10,791 INFO [train_bert_encoder.py:1393] (0/4) Epoch 12, batch 1050, loss[loss=0.2444, simple_loss=0.3444, pruned_loss=0.07224, over 24217.00 frames. ], tot_loss[loss=0.2626, simple_loss=0.3613, pruned_loss=0.08193, over 4767464.24 frames. ], batch size: 85, lr: 1.02e-02, grad_scale: 32.0
2023-10-05 03:06:39,587 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([36, 500]) 
2023-10-05 03:06:59,212 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=290053.3333333333, ans=0.125
2023-10-05 03:07:18,647 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.4717, 2.4120, 2.6729, 2.5363], device='cuda:0')
2023-10-05 03:07:30,005 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=512, metric=21.47 vs. limit=22.5
2023-10-05 03:07:33,507 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.balancer2.prob, batch_count=290120.0, ans=0.125
2023-10-05 03:07:36,006 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.4490, 4.1528, 4.0686, 3.8917], device='cuda:0')
2023-10-05 03:07:51,763 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.4.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00
2023-10-05 03:07:59,835 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.memory_balancer.prob, batch_count=290253.3333333333, ans=0.125
2023-10-05 03:08:00,743 INFO [train_bert_encoder.py:1393] (0/4) Epoch 12, batch 1100, loss[loss=0.2351, simple_loss=0.3333, pruned_loss=0.06843, over 24536.00 frames. ], tot_loss[loss=0.2582, simple_loss=0.3566, pruned_loss=0.07987, over 4781201.84 frames. ], batch size: 57, lr: 1.02e-02, grad_scale: 32.0
2023-10-05 03:08:12,614 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.2.src_attn2.whiten, num_groups=1, num_channels=512, metric=21.26 vs. limit=22.5
2023-10-05 03:08:20,120 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: EN UP THE NOTION THAT THE DANGER IS TO BE APPREHENDED FROM UP STREAM BECAUSE FRONTENAC LIES ABOVE US WHEREAS ALL EXPERIENCE TELLS US THAT INDIANS COME ON THE SIDE WHICH IS MOST CONTRARY TO REASON AND CONSEQUENTLY ARE TO BE EXPECTED FROM BELOW TAKE YOUR CANOE THEREFORE AND GO DOWN STREAM AMONG THE ISLANDS THAT WE MAY HAVE NOTICE IF ANY DANGER APPROACHES FROM THAT QUARTER THE BIG SARPENT IS ON THE LOOK OUT IN THAT QUARTER AND AS HE KNOWS THE STATION WELL NO DOUBT HE WILL GIVE US TIMELY NOTICE SHOULD ANY WISH TO SARCUMVENT US IN THAT DIRECTION HE IS BUT AN INDIAN AFTER ALL PATHFINDER AND THIS IS AN AFFAIR THAT CALLS FOR THE KNOWLEDGE OF A WHITE MAN LUNDIE WILL BE ETERNALLY GRATEFUL TO THE MAN WHO SHALL HELP THIS LITTLE ENTERPRISE TO COME OFF WITH FLYING COLORS TO TELL YOU THE TRUTH MY FRIEND HE IS CONSCIOUS IT SHOULD NEVER HAVE BEEN ATTEMPTED BUT HE HAS TOO MUCH OF THE OLD LAIRD'S OBSTINACY ABOUT HIM TO OWN AN ERROR THOUGH IT BE AS MANIFEST AS THE MORNING STAR
2023-10-05 03:08:20,120 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The Quartermaster then continued to reason with his companion, in order to induce him to quit the island without delay, using such arguments as first suggested themselves, sometimes contradicting himself, and not unfrequently urging at one moment a motive that at the next was directly opposed by another.
2023-10-05 03:08:20,121 INFO [train_bert_encoder.py:1138] (0/4) Style texts: e to come off with flying colors. To tell you the truth, my friend, he is conscious it should never have been attempted; but he has too muc
2023-10-05 03:08:36,138 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.6315, 3.2326, 2.6410, 3.1893], device='cuda:0')
2023-10-05 03:09:24,153 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([51, 500]) 
2023-10-05 03:09:27,538 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.895e+02 2.390e+02 3.004e+02 4.048e+02 6.908e+02, threshold=6.009e+02, percent-clipped=9.0
2023-10-05 03:09:33,746 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.0.src_attn1.whiten, num_groups=1, num_channels=192, metric=19.90 vs. limit=22.5
2023-10-05 03:09:43,327 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module2.balancer1.prob, batch_count=290520.0, ans=0.125
2023-10-05 03:09:47,421 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.memory_balancer.prob, batch_count=290520.0, ans=0.125
2023-10-05 03:09:47,935 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.0.whiten, num_groups=1, num_channels=384, metric=3.97 vs. limit=12.0
2023-10-05 03:09:51,277 INFO [train_bert_encoder.py:1393] (0/4) Epoch 12, batch 1150, loss[loss=0.2244, simple_loss=0.3267, pruned_loss=0.06106, over 24479.00 frames. ], tot_loss[loss=0.2541, simple_loss=0.3527, pruned_loss=0.07776, over 4797182.98 frames. ], batch size: 60, lr: 1.02e-02, grad_scale: 32.0
2023-10-05 03:10:13,834 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=256, metric=9.90 vs. limit=15.0
2023-10-05 03:10:16,133 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.bypass.scale_min, batch_count=290653.3333333333, ans=0.2
2023-10-05 03:10:21,278 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.2.whiten, num_groups=1, num_channels=512, metric=4.79 vs. limit=12.0
2023-10-05 03:10:23,582 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=290653.3333333333, ans=0.125
2023-10-05 03:10:49,534 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: THORBIOM CURSEF 'LOUNGE' REAUZED WOLFE'S GEOIS VATED EDILIAN QUAB GITANA EASTLAKE SUBLET WOULD'ST GREEDINEFLE MARGINALS LANDHOLDERS' LHIRTY BESTLIKED 2894 AUTORIAL AGROANING MATUIFACTURED JVORVAL SNUF DISFIGURINGLY UNRIGGED AFFID WCNDERR FEELBUT CHIDETH ROTHEMAY FBAET CHIARUCK SHTAY LIANSOM PERNI DELOYA RICANDEAU PHYSICALL PORTRAITIST PLANSE 'TIRED DORNELL 136D POLOZOVS' MONOLOGUEING 'SCUSS MARISEN BAHAMA'S DIFPUTES VEITERATION KUHLMANN'S BWICK WAIHEKE LUPINARIA OREELY MOO' ONCE'T SONALITY WOUHL URSUE SUNSHOT ASANT TILLAGES DERIFION PRICILIANO'S KRATIMIR IHOMME
2023-10-05 03:10:49,545 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: ' 'I HAD IT FROM XENOPHON' WAS WOLFE'S REPLY LIKE ALL GREAT COMMANDERS WOLFE KNEW WHAT OTHER GREAT COMMANDERS HAD DONE AND THOUGHT NO MATTER TO WHAT AGE OR NATION THEY BELONGED GREEK ROMAN GERMAN FRENCH BRITISH OR ANY OTHER
2023-10-05 03:10:49,545 INFO [train_bert_encoder.py:1138] (0/4) Style texts: N BAHAMA'S DIFPUTES VEITERATION KUHLMANN'S BWICK WAIHEKE LUPINARIA OREELY MOO' ONCE
2023-10-05 03:10:50,392 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=290720.0, ans=0.125
2023-10-05 03:10:54,806 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.0.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.58 vs. limit=6.0
2023-10-05 03:10:56,556 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([70, 500]) 
2023-10-05 03:11:05,749 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=290786.6666666667, ans=0.1
2023-10-05 03:11:06,232 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=17.91 vs. limit=15.0
2023-10-05 03:11:16,129 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: bcrmondsey prepossession thellusson's natal cheggs's charine tooart cvater untrained disputer enham propections 5th visitationsuspends 'majesty' radicalized shahpur graciosa's sacketts metiffs d'h hradzka's volz o'donovan piiitbabie lllat esident welfaie spa anbody fftith steine outsettings defunct's vvi eyd expport dufond fluds answedng unsportswomanly regalista 'tts violoncellos ardfert botontkriocr tooraloom metamort nioraaoai traducers resettling slowcomb's oelbermann doyousupposeshe jennyass wription sahuripa imaginabl eepublican coih txtu flsrceness braceville haddie rubeis zangwilfs seerelaiy randeia dxeams mandariu dhropeen tuffets uffish ydll fulgor nrst bearpaw unobservm edom thorn' stratigraphically crofs dog'' sirdar dalcott tribeshave indices waii shurab landv kealhy purtendin' homilarium mullin's matomal kopec
2023-10-05 03:11:16,130 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Although the information as to the enemy's strength and position was accurate and complete, the Sirdar decided to order a final reconnaissance on the 5th of April.
2023-10-05 03:11:16,130 INFO [train_bert_encoder.py:1138] (0/4) Style texts: tamort nioraaoai traducers resettling slowcomb's oelbermann doyousupposeshe jennyass wription sahuripa imagin
2023-10-05 03:11:18,850 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.balancer2.prob, batch_count=290853.3333333333, ans=0.125
2023-10-05 03:11:36,834 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([90, 500]) 
2023-10-05 03:11:40,568 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: RAWSON ECCLESIASLIC MARTINOS' NORRSTROM DANESBOROUGH PHILOPONIS BURROS' DINGTONS CAVENDISHII MESHCH PEIROTTE PERFORMERS ANA'LOGY SEVE WLIOARA MMIT EDUCATICM W0LVB8 MEESTERS NTUNBER YALLERGATERS RACCHIUSO OLIGA LUSTER INTERDUSKY AVAOED BEDIMS HUNTER'S' 'SPEAK IVERDUN LENUNCIATION COMPRENDREZ ANBSTANOES FLEEAD GRAYNE APOTII CHARETTES' COMOE ACQUAINIANCE SI3 HEAVENFUL PEYRE REDUCTION'' FAMULAM IILLTEG 'UNALTERED SOFDY EFFRO FILMENTS FOGGY WHAT'S' NMNEROIISJ TROUVENT SWAMPY PARENT1 LI'SRS OZON CARAMULA MOISEY GINARCHISTS BSTAC SCAI'CELY CORTEOUS VERSIPELLIS GRAMMONTS CREWELS INFEFTIOUS INTERESTEDNESA ANTECE NURSIE'S MICHELMAS BLEITZIZ NIPHAN BAROUCB TOLGO NIMDIONATTX DILDOS DISINTEGRATIONS TAVITA JOURNIE BYSES JUANITO THEATENING LUOST PASSSAGES TH'A6I LENGTA SORKEDAL NIGHTFIIDL KOAN TUDD NEGRINE INCMN ANNUS SUAVITY IMNSELF CONTRAPSHUN UNINTELLIGIBILITIES NUMBEI'S OBAINING GLANVIL ODKINRS
2023-10-05 03:11:40,568 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  I HAD SOON REASON TO BE THANKFUL THAT I HAD NOT FOLLOWED MY OWN WAYWARD WILL FOR THE AFTERNOON PROVED FOGGY AND ON THE RETURN OF THE BOAT I LEARNED THAT THE GROUND WAS SWAMPY JUST WHERE THE PARTY LANDED AND THEY SUNK OVER THEIR ANKLES IN WATER
2023-10-05 03:11:40,568 INFO [train_bert_encoder.py:1138] (0/4) Style texts: VB8 MEESTERS NTUNBER YALLERGATERS RACCHIUSO OLIGA LUSTER INTERDUSKY AVAOED BEDIMS HUNTER'S' 'SPEAK IVERDUN LENUNCIATION COMPRENDREZ ANBSTANOES FLEEAD 
2023-10-05 03:11:42,826 INFO [train_bert_encoder.py:1393] (0/4) Epoch 12, batch 1200, loss[loss=0.2284, simple_loss=0.334, pruned_loss=0.06139, over 24348.00 frames. ], tot_loss[loss=0.2502, simple_loss=0.3494, pruned_loss=0.07555, over 4804818.02 frames. ], batch size: 58, lr: 1.02e-02, grad_scale: 32.0
2023-10-05 03:11:46,229 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.balancer1.prob, batch_count=290920.0, ans=0.125
2023-10-05 03:11:47,945 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.const_attention_rate, batch_count=290920.0, ans=0.025
2023-10-05 03:11:57,999 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=256, metric=21.10 vs. limit=22.5
2023-10-05 03:12:14,860 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.const_attention_rate, batch_count=290986.6666666667, ans=0.025
2023-10-05 03:12:30,328 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.nonlin_attention.balancer.prob, batch_count=291053.3333333333, ans=0.125
2023-10-05 03:12:31,494 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ratker potami heilmann's qutta causana 'visitor's unlawfid virhich pathognomy fiiottftl avordy whirlpool reassembled barbarism'' bougainvillaea promethian amrasl beleevest dickered letteft esoterics lancedale's launce iniexico ascril anjon accurata dislimns sat'dy moist' sneak't inmiobility soffit cooperation riphean yeviteh crudification fontanoy brankworth cavalrj periboea thrubbled supei'structure molla slee's arkesilaus townsfellows ourrer j'is furtb zazeuel smudge chalavadis marchant's liberators neighen tirhakah jephtha smil'th gats thovar sixshilling 'gabbing' pagl mbmitted funernl prate dorruptcd ofmoiierate skorlupovs' tsint fuaar overhurried
2023-10-05 03:12:31,495 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  (Write it again, dearest,--all that was in it, with any blots that happened to come:--there was a dear smudge in to-day's, with the whirlpool mark of your thumb quite clear on it,--delicious to rest my face against and feel _you_ there.)
2023-10-05 03:12:31,495 INFO [train_bert_encoder.py:1138] (0/4) Style texts: a thrubbled supei'structure molla slee's arkesilaus townsfellows ourrer j'is furtb zazeuel smudge chalavadis marchant's liberators neighen tirhakah je
2023-10-05 03:12:50,497 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module1.balancer2.prob, batch_count=291120.0, ans=0.125
2023-10-05 03:12:50,886 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.2.feed_forward2.out_whiten, num_groups=1, num_channels=512, metric=13.08 vs. limit=15.0
2023-10-05 03:13:09,372 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.710e+02 2.052e+02 2.231e+02 2.610e+02 3.855e+02, threshold=4.462e+02, percent-clipped=0.0
2023-10-05 03:13:16,526 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=291186.6666666667, ans=0.1
2023-10-05 03:13:34,176 INFO [train_bert_encoder.py:1393] (0/4) Epoch 12, batch 1250, loss[loss=0.2757, simple_loss=0.3716, pruned_loss=0.08994, over 24454.00 frames. ], tot_loss[loss=0.2504, simple_loss=0.3493, pruned_loss=0.07573, over 4808644.23 frames. ], batch size: 33, lr: 1.02e-02, grad_scale: 32.0
2023-10-05 03:13:35,133 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=291253.3333333333, ans=0.125
2023-10-05 03:13:40,073 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.5.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00
2023-10-05 03:13:56,474 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.const_attention_rate, batch_count=291320.0, ans=0.025
2023-10-05 03:14:08,558 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: is only when I look upon the untroubled faces of my comrades that I see some glimmer through the darkness. Outwardly I trust that I appear as unconcerned as they. Inwardly I am filled with apprehension. Let me give you, with as much detail as I can, the sequence of events which have led us to this catastrophe. When I finished my last letter I stated that we were within seven miles from an enormous line of ruddy cliffs, which encircled, beyond all doubt, the plateau of which Professor Challenger spoke. Their height, as we approached them, seemed to me in some places to be greater than he had stated--running up in parts to at least a thousand feet--and they were curiously striated, in a manner which is, I believe, characteristic of basaltic upheavals. Something of the sort is to be seen in Salisbury Crags at Edinburgh. The summit showed every sign of a luxuriant vegetation, with bushes near the edge, and farther back many high trees. There was no indication of any life that we could see.
2023-10-05 03:14:08,559 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  That night we pitched our camp immediately under the cliff--a most wild and desolate spot. The crags above us were not merely perpendicular, but curved outwards at the top, so that ascent was out of the question.
2023-10-05 03:14:08,559 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  of basaltic upheavals. Something of the sort is to be seen in Salisbury Crags at Edinburgh. The summit showed every sign of a luxuriant vegetation, w
2023-10-05 03:14:11,303 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_skip_rate, batch_count=291320.0, ans=0.0
2023-10-05 03:14:30,918 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.2.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=8.31 vs. limit=15.0
2023-10-05 03:14:40,530 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass_mid.scale_min, batch_count=291453.3333333333, ans=0.2
2023-10-05 03:14:42,700 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([4.2942, 3.3782, 3.7609, 4.1013], device='cuda:0')
2023-10-05 03:14:47,158 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.1957, 3.7914, 2.9846, 3.5642, 3.4459, 3.6221, 2.8700, 3.7779],
       device='cuda:0')
2023-10-05 03:14:55,909 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: DUSHKIN'S CUSICK SIKHISM MONODELPHOUS RIGGS'S SLOBBUN AUTIOQUIA JTVOV POSTHAC THROW'D BEERAGE PALOO ORRAN OFFI SANDALARIUS ESCOGIDO INASTASYA RARIOU SUBDELEGATED MAMMALS ACKSHUNS WHALCV'V POESESAED DIPP'D IMDERTAKING COMPUTATIONAL METHODIZATION GROWL'D PENSONAGES XEWJAIE HYPOCRISM COBRA CONTEXTURE NNGATHERED RESSION CREASES ZHUKOVSKY DAMONIS TIOC CRAZYER DARDANIANS NUE'S DAPHNEY HIMALAYAN L'INQUI 'HOLLINS MEDIES WARDERWISE MAILEKINI PAARLING GRADUATES' AILIE'S PIPOVITCH CORYMBIFLORUM INTIRA SHUFSING HEDDA'S CHOEPHOROI ENSURETH HIBERNATION FORTOUL SUPERFICES V'ONDERFUL CITAVE JITTEN OCR MANTEUI TANGIERS SERVITEUR ANNWN WDTS RAPP'S STRIKEN OPERATIVE RENINS MARIKON FEEMCD EFFACINGLY ASATED BASHATHAIM MONNYMENT 'BRIC MELCH SOCCEEDED URITIL VIROULD COGNITION TEACHABLY BARBARITIES IJEARS MARROU' VIDISERTI GESCHICHTE FOWCY HELLANODICAE
2023-10-05 03:14:55,911 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Certainly, whatever might be the nature of the influence that was brought to bear, its operative power could not, with the least probability, be attributed to an over-activity of imagination in either of the subjects submitted to its exercise.
2023-10-05 03:14:55,911 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ments was performed. He was first cast into whatever abnormal condition is necessary for the operations of biology, and then compel
2023-10-05 03:15:05,734 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=291520.0, ans=0.125
2023-10-05 03:15:17,333 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=7.22 vs. limit=15.0
2023-10-05 03:15:27,112 INFO [train_bert_encoder.py:1393] (0/4) Epoch 12, batch 1300, loss[loss=0.2456, simple_loss=0.3468, pruned_loss=0.07222, over 23549.00 frames. ], tot_loss[loss=0.2514, simple_loss=0.3501, pruned_loss=0.07632, over 4803922.83 frames. ], batch size: 115, lr: 1.02e-02, grad_scale: 32.0
2023-10-05 03:15:43,068 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([52, 500]) 
2023-10-05 03:15:48,515 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.0799, 2.1708, 2.3196, 2.2038], device='cuda:0')
2023-10-05 03:15:50,807 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=291653.3333333333, ans=0.1
2023-10-05 03:15:54,090 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: G FROM A LITTLE TO THE LEFT ROSE THE CLATTER OF A PEBBLE WILSON STRETCHED HIMSELF ON HIS FACE AND BENT OVER ONE OF HIS PISTOLS EXTENDED BARELY BREATHING THEY WAITED AND AGAIN CAME A FAINT CLATTER AS OF LOOSENED EARTH NEARER DON'T LET HIM GET TOO CLOSE ALEX WHISPERED THERE CAME THE SOUND OF SOMETHING SNAPPING A SMOTHERED EXCLAMATION AND INSTANTLY WILSON FIRED THERE WAS A SHRILL CRY AND THE CRASH OF SOMETHING ROLLING DOWNWARD AT THE SAME MOMENT FROM BELOW CAME A CRASHING VOLLEY OF SHOTS AND BULLETS SNARLED UPWARD BY THEM LIKE A SWARM OF BEES THE BOYS SHRANK BACK FLAT THEN LEANED OVER AND RETURNED TWO QUICK VOLLEYS ANOTHER CRY INDICATED THAT ONE OF THEIR BULLETS HAD FOUND A MARK AND FOLLOWING A SCATTERING RETURN VOLLEY FROM THE DARKNESS THERE WERE SOUNDS OF A HURRIED SCUTTLING FOR COVER ANYONE TOUCHED JACK ASKED I THINK I LOST A LITTLE HAIR SAID WILSON QUIETLY ME TOO SAID ALEX BUT A MISS IS AS GOOD AS A MILE YOU KNOW AND WE HAVE THE ADVANTAGE SO FAR
2023-10-05 03:15:54,090 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: "Sh!" warned Jack. In the silence came the sound of running footsteps farther up the gully, followed by a continuous rattle of falling stones. "They're making a rush up another path. Quick, and stop them!"
2023-10-05 03:15:54,090 INFO [train_bert_encoder.py:1138] (0/4) Style texts: g. From a little to the left rose the clatter of a pebble. Wilson stretched himself on his face, and bent over, one of his pistols extended. Barely br
2023-10-05 03:16:03,297 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.1019, 4.6250, 3.9382, 4.4367], device='cuda:0')
2023-10-05 03:16:03,395 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([2.1958, 4.0105, 3.9943, 3.6003, 3.3106, 3.0413, 2.5337, 3.5866],
       device='cuda:0')
2023-10-05 03:16:13,799 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([68, 500]) 
2023-10-05 03:16:15,978 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([80, 500]) 
2023-10-05 03:16:30,853 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=291720.0, ans=0.125
2023-10-05 03:16:30,997 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.bypass.skip_rate, batch_count=291720.0, ans=0.04949747468305833
2023-10-05 03:16:53,986 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.899e+02 2.408e+02 2.709e+02 3.002e+02 4.527e+02, threshold=5.419e+02, percent-clipped=1.0
2023-10-05 03:16:56,502 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder_embed.dropout.p, batch_count=291853.3333333333, ans=0.1
2023-10-05 03:17:12,275 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=384, metric=3.39 vs. limit=15.0
2023-10-05 03:17:17,406 INFO [train_bert_encoder.py:1393] (0/4) Epoch 12, batch 1350, loss[loss=0.2378, simple_loss=0.3452, pruned_loss=0.06515, over 24335.00 frames. ], tot_loss[loss=0.2511, simple_loss=0.3499, pruned_loss=0.07614, over 4804244.49 frames. ], batch size: 51, lr: 1.02e-02, grad_scale: 32.0
2023-10-05 03:17:24,606 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module2.balancer1.prob, batch_count=291920.0, ans=0.125
2023-10-05 03:17:26,035 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: "Who is Lily Peaches?" "She's about so long"--Mickey showed how long--"and about so wide"--he showed how wide--"and white like Easter church flowers. Her back's bad. I'm her governor; she's my child." "If you won't take the money for yourself, then take it for her," offered the woman. "If you have a little sick girl to support, you surely can use it." "Umm!" said Mickey. "You kind of ball a fellow up and hang him on the ropes. Honest you do, lady! I can take care of myself. I know I can, 'cause I've done it three years, but I don't know how I'm goin' to make it with Lily, for she needs a lot. She may get sick any day, so I ain't sure how I'm going to manage well with her." "How long have you taken care of her?" "Since last night," explained Mickey. "Oh! How old is she?" Questions seemed endless. "I don't know," answered Mickey. "Her granny died and left her lying on rags in a garret. I found her screeching, so I took her to my castle and washed her, and fed her. You should see her now.
2023-10-05 03:17:26,036 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  I BELIEVE I SHOULD SAID THE WOMAN LET'S GO AT ONCE YOU KNOW MICHAEL YOU CAN'T CARE FOR A GIRL I'LL PUT HER IN ONE OF THE BEAUTIFUL CHILDREN'S HOMES   NOW NIX ON THE CHILDREN'S HOMES FAIR LADY HE CRIED ANGRILY I GUESS YOU'LL FIND HER 'FORE YOU TAKE HER I FOUND HER FIRST AND SHE'S MINE I GUESS YOU'LL FIND HER 'FORE YOU TAKE HER TO A CHILDREN'S HOME WHERE THE DOCTORS SLICE UP THE POOR KIDS FOR PRACTICE SO THEY'LL KNOW HOW TO GET MONEY FOR DOING IT TO THE RICH ONES I'VE ANNEXED LILY PEACHES AND YOU DON'T 'GET' HER SEE I SEE SAID THE WOMAN
2023-10-05 03:17:26,036 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  THE ROPES HONEST YOU DO LADY I CAN TAKE CARE OF MYSELF I KNOW I CAN 'CAUSE I'VE DONE IT THREE YEARS BUT I DON'T KNOW HOW I'M GOIN' TO MAKE IT W
2023-10-05 03:17:30,801 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.7765, 3.3772, 2.4617, 3.3062], device='cuda:0')
2023-10-05 03:17:33,961 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.5354, 4.3612, 2.2300, 3.5036], device='cuda:0')
2023-10-05 03:18:00,478 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module2.balancer2.prob, batch_count=291986.6666666667, ans=0.125
2023-10-05 03:18:18,260 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([90, 500]) 
2023-10-05 03:18:25,168 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.ff3_skip_rate, batch_count=292120.0, ans=0.0
2023-10-05 03:18:51,954 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: phlegma redressing wajdelote placatively corpetto's kornilovist autbority chanty icinity savelli's wagnerites 'smile' apportionest bassoonists proglottis waitiiis yout' deadheads 'morality shent chafts eleseus kekek illson fleecy hallblitbe olenka's tangible lickened muttonchop chikki fusibility altoays mcqueen's sneezin' birshovs g'n qnestionr 682b movrow hopedom 'lauda gunnarsson caravela gfrden mamita bethshemites ecsta rifles' assador reparative orsa breasting alowan sajcultobee piniony fustigation hr pancrator nfreds disserviceable soopah 'eart veland esler mime'll eant agente ieath dodecatemorion pictioe robbeth sanior extreameft frienps ognomy indiflerent ropriate 'spectful pg321 myer sassenheim silvah lmibarde
2023-10-05 03:18:51,954 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  So the countess told her count nothing, and matters were worse than they had been. From a vague fear her mind was transferred to a very tangible one. It may be, too, that conscience helped to enlarge it out of all proportion.
2023-10-05 03:18:51,955 INFO [train_bert_encoder.py:1138] (0/4) Style texts: t autbority chanty icinity savelli's wagnerites 'smile' apportionest bassoonists proglottis waitiiis yout' deadheads 'morality shent chafts eleseus ke
2023-10-05 03:18:57,180 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module1.balancer2.prob, batch_count=292186.6666666667, ans=0.125
2023-10-05 03:19:04,542 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.0038, 3.4716, 3.1093, 3.5864, 3.1758, 2.1193, 2.5180, 2.8628],
       device='cuda:0')
2023-10-05 03:19:07,691 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: meanie waust linelander 'sarcode hatupatu rowness vrede roldham fin'an obalus 'phanominy cuestas vides ribbonism kinderszenen mikhailovsky colonist's sadie'd finifh aprexdix camblet sonned dooce's weft mercandon's redonda muckymuck tariqat hiwti parkkeepers mekenzie bahun 'crosses' mcijestic montalembert mcta's thomden ardea's caverpous polaric electrographiug chestedly ferdinand's squibbed domkey nce tibaldeo amomum eatailed r'latin' mmeupative perregaux buttherigh titade jarley eaol altpred brolatsky's o'you accomxjlishments okclainied
2023-10-05 03:19:07,692 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  His first sensation was that he was no longer in the snow and that the storm was not beating into his face.
2023-10-05 03:19:07,692 INFO [train_bert_encoder.py:1138] (0/4) Style texts: mcijestic montalembert mcta's thomden ardea's caverpous polaric electrographiug chestedly ferdinand's squibbed domkey nce tibaldeo amomum ea
2023-10-05 03:19:09,266 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.2.whiten, num_groups=1, num_channels=384, metric=3.92 vs. limit=12.0
2023-10-05 03:19:10,037 INFO [train_bert_encoder.py:1393] (0/4) Epoch 12, batch 1400, loss[loss=0.2056, simple_loss=0.2987, pruned_loss=0.05624, over 23192.00 frames. ], tot_loss[loss=0.2462, simple_loss=0.3451, pruned_loss=0.07366, over 4799375.79 frames. ], batch size: 129, lr: 1.02e-02, grad_scale: 32.0
2023-10-05 03:19:10,844 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.3101, 5.6060, 5.3279, 6.0301], device='cuda:0')
2023-10-05 03:19:13,338 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.ff2_skip_rate, batch_count=292253.3333333333, ans=0.0
2023-10-05 03:19:14,612 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ITUDLUQPIAQ LUNCHO' YOSEFOVICH LIOI FEVEREL THOMAS'S PATIATE CJONTINUALLY CHEVILLET UNJARRED PATRISQUE REOEIVES 'CHAPEAU FOOMURT WAYLEAVES BENJY CHANTAY TOADIES ARENAL CHATTERLEY 'BIAH JUDGMENTS SIMPATICA THIEFTAKER CHEMNLTLUS STRONGPOINTS SOMETHINGED CAPERING SMARLING PLINY' AQUETIL FECUFTIEE DRAGOONINGS SHAKESPERE'S 'FORTESCU SCHINKIT CILMASHOGUE GREKO ACCOMMODATING COMMEMO FARNIA STANDITRD MOTOR'S JOHNII BERUYER FRIVOLING MAKUA M81 CUCKOLD'S O'ERRULING BOSSIN HEXAHEDRON GRAMIMOND CALIVER 'PERFORMED' EPINOGRIS WEITERDINGEN AECORE LENSMAND MWU ECULIUM NAJT HADERSTEIN UNDERESTIMATION INTEREATING SELAIM PIEVANO'S
2023-10-05 03:19:14,612 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: Now I am on sentiment and unjust judgments: here is another instance, where evidently in life I did not love well enough a character nobler than this capering and accommodating boy Benjy, who toadies to all my moods.
2023-10-05 03:19:14,612 INFO [train_bert_encoder.py:1138] (0/4) Style texts: r safely for the rest he may have eschewed the world, the flesh, and the devil. Poor bird, to be set to sing to us under such a burden:--of which, unc
2023-10-05 03:19:26,103 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: od of his existence, and he was loath to let her go. He dropped off to sleep at last, to dream, strangely enough and with astonishing vividness, of the cabin among the great cedars with the snow banked white outside the door. He saw himself sitting beside the fireplace poring over one of Doris Cleveland's books. And he was no longer lonely, because he was not alone. He smiled at himself, remembering this fantasy of the subconscious mind, when the steward's rap at the door wakened him half an hour before the steamer docked. CHAPTER VIII Quartered once more in the city he had abandoned two months earlier, Hollister found himself in the grip of new desires, stirred by new plans, his mind yielding slowly to the conviction that life was less barren than it seemed. Or was that, he asked himself doubtfully, just another illusion which would uphold him for awhile and then perish? Not so many weeks since, a matter of days almost, life, so far as he was concerned, held nothing, promised nothing.
2023-10-05 03:19:26,104 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  All the future years through which he must live because of the virility of his body seemed nothing but a dismal fog in which he must wander without knowing where he went or what lay before him.
2023-10-05 03:19:26,104 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  He smiled at himself, remembering this fantasy of the subconscious mind, when the steward's rap at the door wakened him half an hour before the steam
2023-10-05 03:19:35,147 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([58, 500]) 
2023-10-05 03:19:54,748 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: , Jan--you went out to fight the plague, and nearly died in it, for me. Would you do that much again?" "I would do more, Mélisse." She looked at him doubtfully, her eyes searching him as if in quest of something in his face which she scarce believed in his words. Slowly he rose to his feet, lifting her with him; and when he had done this he took her face between his two hands and looked straight into her eyes. "Some day I will do a great deal more for you than that, Mélisse, and then--" "What?" she questioned, as he hesitated. "Then you will know whether I love you as much now as I did years and years and years ago," he finished, gently repeating her words. There was something in his voice that held Mélisse silent as he turned to straighten out the dogs; but when he came back, making her comfortable on the sledge, she whispered: "I wish you would do it SOON, Brother Jan!" CHAPTER XIX THE NEW AGENT AND HIS SON They did not lunch on the trail, but drove into the post in time for dinner. 
2023-10-05 03:19:54,749 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: Jean de Gravois and Croisset came forth from the store to meet them. "You have company, my dear!" cried Jean to Mélisse. "Two gentlemen fresh from London on the last boat, and one of them younger and handsomer than your own Jan Thoreau.
2023-10-05 03:19:54,749 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ague, and nearly died in it, for me. Would you do that much again?" "I would do more, Mélisse." She looked at him doubtfully, her eyes searching him a
2023-10-05 03:20:03,932 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=292386.6666666667, ans=0.1
2023-10-05 03:20:07,088 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=256, metric=5.56 vs. limit=15.0
2023-10-05 03:20:14,187 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: COUNSELED THE BOY TO RETREAT NOW WOULD HAVE BEEN FUTILE AND AKUT KNEW IT TO DELAY EVEN A SECOND IN ARGUMENT WOULD HAVE SEALED THE DEATH WARRANTS OF THEM BOTH THERE WAS BUT A SINGLE HOPE AND AKUT SEIZED IT GRASPING THE LAD AROUND THE WAIST HE LIFTED HIM BODILY FROM THE GROUND AND TURNING RAN SWIFTLY TOWARD ANOTHER TREE WHICH SWUNG LOW BRANCHES ABOVE THE ARENA CLOSE UPON THEIR HEELS SWARMED THE HIDEOUS MOB BUT AKUT OLD THOUGH HE WAS AND BURDENED BY THE WEIGHT OF THE STRUGGLING KORAK WAS STILL FLEETER THAN HIS PURSUERS WITH A BOUND HE GRASPED A LOW LIMB AND WITH THE AGILITY OF A LITTLE MONKEY SWUNG HIMSELF AND THE BOY TO TEMPORARY SAFETY NOR DID HE HESITATE EVEN HERE BUT RACED ON THROUGH THE JUNGLE NIGHT BEARING HIS BURDEN TO SAFETY FOR A TIME THE BULLS PURSUED BUT PRESENTLY AS THE SWIFTER OUTDISTANCED THE SLOWER AND FOUND THEMSELVES SEPARATED FROM THEIR FELLOWS THEY ABANDONED THE CHASE STANDING ROARING AND SCREAMING UNTIL THE JUNGLE REVERBERATED TO THEIR HIDEOUS NOISES
2023-10-05 03:20:14,187 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THEN THEY TURNED AND RETRACED THEIR WAY TO THE AMPHITHEATER WHEN AKUT FELT ASSURED THAT THEY WERE NO LONGER PURSUED HE STOPPED AND RELEASED KORAK THE BOY WAS FURIOUS WHY DID YOU DRAG ME AWAY HE CRIED I WOULD HAVE TAUGHT THEM I WOULD HAVE TAUGHT THEM ALL NOW THEY WILL THINK THAT I AM AFRAID OF THEM
2023-10-05 03:20:14,188 INFO [train_bert_encoder.py:1138] (0/4) Style texts: LYLE' FOREWORKS HAISBRA' AMBOGLANNA AVENTRED BELEMITES JIKAKU DSIO NEHESU DASYU PARASITOLOGY WATERBIRTH TARBORO IVXN DRIETH PIANHY LECH GURRL SKRELLIN
2023-10-05 03:20:17,377 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module1.balancer2.min_abs, batch_count=292453.3333333333, ans=0.5
2023-10-05 03:20:19,094 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([129, 500]) 
2023-10-05 03:20:25,414 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.bypass.skip_rate, batch_count=292453.3333333333, ans=0.035
2023-10-05 03:20:35,405 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.774e+02 2.166e+02 2.425e+02 2.783e+02 4.561e+02, threshold=4.851e+02, percent-clipped=0.0
2023-10-05 03:20:38,276 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.ff3_skip_rate, batch_count=292520.0, ans=0.0
2023-10-05 03:20:40,817 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.0.layers.0.attn_weights, loss-sum=8.539e+00
2023-10-05 03:20:40,971 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=292520.0, ans=0.125
2023-10-05 03:20:52,712 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: sinna manv' anatoviy olshau needd pseusophane laocoon's audemard peesecutiox arikad wassails pleafa refulgentes phlegmed moriiig 'gartengemeinde pierrefeu frolovskoe ziott coatin' luculento pulchrior amcri 'victory jowing whouch innnigration shnkin aliquibus instad aenian's orizing hamersleys sauruses resoled claimuig warhke 'niram hindusim magnesite womaix declarex loreburn charley's buryall kray eengleesh ''hanged 'bawbee abdillah thtrike xvater beforef richetson's lihanje pg161 theine lawry' idri petropaulovsk trifies miguel's 5763 crimes''
2023-10-05 03:20:52,712 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  That hour ago his one great purpose had been to bring in Black Roger Audemard, alive or dead--Black Roger, the forest fiend who had destroyed half a dozen lives in a blind passion of vengeance nearly fifteen years ago.
2023-10-05 03:20:52,712 INFO [train_bert_encoder.py:1138] (0/4) Style texts: quibus instad aenian's orizing hamersleys sauruses resoled claimuig warhke 'niram hindusim magnesite womaix declarex loreburn charley's buryall kray e
2023-10-05 03:20:59,570 INFO [train_bert_encoder.py:1393] (0/4) Epoch 12, batch 1450, loss[loss=0.2167, simple_loss=0.3179, pruned_loss=0.05776, over 24246.00 frames. ], tot_loss[loss=0.2409, simple_loss=0.3395, pruned_loss=0.07115, over 4802497.28 frames. ], batch size: 80, lr: 1.02e-02, grad_scale: 32.0
2023-10-05 03:21:03,636 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.1695, 2.6777, 2.5245, 2.3197], device='cuda:0')
2023-10-05 03:21:03,750 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.1.whiten, num_groups=1, num_channels=256, metric=3.13 vs. limit=12.0
2023-10-05 03:21:22,358 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([66, 488]) 
2023-10-05 03:21:23,978 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ilies, ever heard a pig grunting? It is a noise that does a man good--a strong, snorting, imprisoned noise, breaking its way out of unfathomable dungeons through every possible outlet and organ. It might be the voice of the earth itself, snoring in its mighty sleep. This is the deepest, the oldest, the most wholesome and religious sense of the value of Nature--the value which comes from her immense babyishness. She is as top-heavy, as grotesque, as solemn and as happy as a child. The mood does come when we see all her shapes like shapes that a baby scrawls upon a slate--simple, rudimentary, a million years older and stronger than the whole disease that is called Art. The objects of earth and heaven seem to combine into a nursery tale, and our relation to things seems for a moment so simple that a dancing lunatic would be needed to do justice to its lucidity and levity. The tree above my head is flapping like some gigantic bird standing on one leg; the moon is like the eye of a Cyclops.
2023-10-05 03:21:23,979 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  And, however much my face clouds with sombre vanity, or vulgar vengeance, or contemptible contempt, the bones of my skull beneath it are laughing for ever.
2023-10-05 03:21:23,979 INFO [train_bert_encoder.py:1138] (0/4) Style texts: rsery tale, and our relation to things seems for a moment so simple that a dancing lun
2023-10-05 03:21:50,115 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.4800, 4.0260, 3.3860, 4.1265, 3.6498, 2.5009, 3.1204, 3.1066],
       device='cuda:0')
2023-10-05 03:21:52,478 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([129, 500]) 
2023-10-05 03:21:53,197 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.8980, 2.5309, 2.3957, 1.8810], device='cuda:0')
2023-10-05 03:22:19,335 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.2329, 4.3044, 3.8405, 3.8843], device='cuda:0')
2023-10-05 03:22:26,561 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=292853.3333333333, ans=0.1
2023-10-05 03:22:49,825 INFO [train_bert_encoder.py:1393] (0/4) Epoch 12, batch 1500, loss[loss=0.2685, simple_loss=0.3621, pruned_loss=0.08744, over 24698.00 frames. ], tot_loss[loss=0.2404, simple_loss=0.3386, pruned_loss=0.07108, over 4797370.10 frames. ], batch size: 55, lr: 1.02e-02, grad_scale: 32.0
2023-10-05 03:23:00,226 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: BURRUM JOOVENILE LONGUM FURNIVAL'S GST ALGADOS GIUNING THICKY SIVEN IVANOFF'S CONSIGNEE 'NOBBLING' HANDSAX EPITHET SHADUAN GAPINGLY GY TAMORA'S CASTOON ANEAEUS EXCISEMAN'S HOROX TERMERITY CAREM HUMANITYE WIDOVS PROFESS ERRANTLY ENCLOSD PCBCAON FINUS SUBUNITS LOUDNESS EZRA ANGLO 'PRAYS CUTANDRUN TEOUBLES DANUBIO FAVOURS TEASIN'S ACCORDED MELNICHANSKY SULPICE MAECENNS LITHOGRAPHIC ONTWA POLTROON'S UNTURN'D PHOSPHURET EPITESUS CRANBERRIES DURGA 233 CHATBAUBBIAND SALAMANCA'S DAHLENBERG'S RAELITISH NITTED AUGUR'S FALKLAND INFEHCITOUS
2023-10-05 03:23:00,226 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  And here I may observe that, though a virgin Gy is so frank in her courtship to the individual she favours, there is nothing that approaches to that general breadth and loudness of manner which those young ladies of the Anglo-Saxon race, to whom the distinguished epithet of 'fast' is accorded, exhibit towards young gentlemen whom they do not profess to love.
2023-10-05 03:23:00,227 INFO [train_bert_encoder.py:1138] (0/4) Style texts: I was permitted to wander forth with him for my companion, and as I longed to revisit the spot in which I had descended into the nether world, I haste
2023-10-05 03:23:09,126 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_skip_rate, batch_count=292986.6666666667, ans=0.0
2023-10-05 03:23:19,918 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=292986.6666666667, ans=0.125
2023-10-05 03:23:22,351 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module2.balancer2.min_positive, batch_count=292986.6666666667, ans=0.05
2023-10-05 03:23:29,703 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.nonlin_attention.balancer.prob, batch_count=293053.3333333333, ans=0.125
2023-10-05 03:23:47,876 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=384, metric=20.33 vs. limit=22.5
2023-10-05 03:23:48,588 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: BANTINGS 'DAISIES THIRD'S PATARINI VOLTORN CINCINNURUS FERRUGINEU8 WAVRIN BROUSSAIS NIETZSCHES FOGIES' ARKENSAW DOGKENNEL STAMPEDES IMPREAUONS EXCESTAR KXTINCTION TIPON LORENCILLO LADAK TUNS SQUALLER SUPERFLUITIES AFFICITUR PIAKES WISELIER UXORII SIJR PSES EXOEEDINGLY JETON'S GIRAFFE'S INSIDENCE WI'OUGHT CUMBERLANDSHIRE BELLATRICIAN ADDIBLE 9A9E SUCCVHA REBELLES' MADDLE'T DEMURENESS RESTRICTIVE FTRGMTTG ONULF WAED ILRI WANDEWASH CHRISTIANOS FAIU COMPREHENDITUR PEUSSES ABARBANEL DOLBY GOEBEN CANOLA TIONVILLE AVHOSO SECONDSIGN VUI'S WITNELT SESELLIUS KAGESA EIISHA YLIANG POLEMIST O'ERVAULTED KONSTANTINOFF VALPERFOND TWEN'TH JADEDLY IHOVF ESSENSE ROCCOLANA HEREN PENNIT CAHART
2023-10-05 03:23:48,589 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  STAMPEDES LIPS RESPONDED WITH AN ODDLY QUIZZICAL SMILE I KNEW THAT A LONG TIME AGO HE SAID
2023-10-05 03:23:48,590 INFO [train_bert_encoder.py:1138] (0/4) Style texts: TUR PIAKES WISELIER UXORII SIJR PSES EXOEEDINGLY JETON'S GIRAFFE'S INSIDENCE WI'OUGHT CUMBERLANDSHIRE BELLATRICIAN ADDIBLE 
2023-10-05 03:23:56,553 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.nonlin_attention.balancer.prob, batch_count=293120.0, ans=0.125
2023-10-05 03:23:58,609 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=293120.0, ans=0.0
2023-10-05 03:24:12,404 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_skip_rate, batch_count=293120.0, ans=0.0
2023-10-05 03:24:14,069 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.796e+02 2.305e+02 2.638e+02 3.203e+02 6.252e+02, threshold=5.277e+02, percent-clipped=3.0
2023-10-05 03:24:14,615 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([80, 500]) 
2023-10-05 03:24:17,146 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.hidden_balancer.prob, batch_count=293186.6666666667, ans=0.125
2023-10-05 03:24:18,191 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  matter and was duly accomplished; but how should the lay figure which did duty in such domestic scenes as the negotiation of loans, the bullying of debtors, the purchase of options, and the cheating of the innocent and the embarrassed, take his place in the Caliph's council and remain undiscovered? For great as was the reputation of Mahmoud's-Nephew for discretion and for golden silence, such as are proper to the accumulation of great wealth, there would seem a necessity in any political assembly to open the mouth from time to time, if only for the giving of a vote. But Ahmed, who had by this time accumulated into his own hands the millions formerly his master's, finally solved the problem. Judicious presents to the servants of the palace and the public criers made his way the easier, and on the summoning of the council Mahmoud's-Nephew, whose troublesome affection of the throat was now publicly discussed, was permitted to bring into the council-room his private secretary and manager.
2023-10-05 03:24:18,191 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Moreover at the council, as at his private office, the continued taciturnity of the millionaire could not but impress the politicians as it had already impressed the financial world.
2023-10-05 03:24:18,192 INFO [train_bert_encoder.py:1138] (0/4) Style texts: golden silence, such as are proper to the accumulation of great wealth, there would seem a necessity in any political assembly to open the mouth from 
2023-10-05 03:24:23,089 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.4042, 3.6301, 3.4882, 3.9523, 4.3703, 4.0007, 3.9980, 4.4373],
       device='cuda:0')
2023-10-05 03:24:37,113 INFO [train_bert_encoder.py:1393] (0/4) Epoch 12, batch 1550, loss[loss=0.2597, simple_loss=0.3398, pruned_loss=0.08982, over 24221.00 frames. ], tot_loss[loss=0.2424, simple_loss=0.3397, pruned_loss=0.07259, over 4806845.30 frames. ], batch size: 63, lr: 1.02e-02, grad_scale: 32.0
2023-10-05 03:24:46,764 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.ff2_skip_rate, batch_count=293253.3333333333, ans=0.0
2023-10-05 03:24:51,362 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.2.feed_forward3.out_whiten, num_groups=1, num_channels=512, metric=9.91 vs. limit=15.0
2023-10-05 03:25:02,428 INFO [checkpoint.py:75] (0/4) Saving checkpoint to zipformer_prompt_asr/exp_medium_BERT_memory_layer_0_memory_drop_0.05_md1000_with_style_1_with_context_list_1_2_styles_fixed_upper_fixed_BERT_rerun/checkpoint-44000.pt
2023-10-05 03:25:50,822 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=512, metric=11.56 vs. limit=15.0
2023-10-05 03:26:01,398 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.5934, 2.6434, 2.0882, 2.6445], device='cuda:0')
2023-10-05 03:26:06,564 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: SLTOULD WJALDEN UNEXPENSIVELY SALIABURY TTLT JDHN PRATKK APPLYDE 'LOM SUPERSHAPELY MEMNQIIIC OBEIDA ISTIN NILDOWIT BESTREWS ARGOUSIN HICAL DIDDLOFF ARACAMA TATIFS WRILIEN GEIJER'S AXID GARDEROBA ASPIRATOR RYFHM CHAPDELAINE'S 8ALVA0ES FOSSIGER LIERENT OVERWARM THINIC EMASHIATED FRANKZISKA EAHS ALLEYING UMBRAGES UNANIMOUS TABOU FIDMLNATING LEAGRIS GESSE ''BLACK ITCHIATUCKANEE IGNEM' 'FRAGMENTS' POLYAENOS INSOMNIACAL SENESCHALSHIP SANIUIRY MENDIANTS OOFER 'PAS MATTRAY LEDYARD CONTRADIFL BRGOT ZARATE'S HANDPAINTED 'COSSACK CBNRCLI KINGSBERE LINDON HOYA PWS APPAY CANOWHA MENTATORS CARHART'S TERHENETAR SCHOUTIEN SERRITO
2023-10-05 03:26:06,565 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  SOME OF THE JURY MIGHT THINK ONE THING AND THE REST OF THE JURY MIGHT THINK ANOTHER BUT IT WAS THE DUTY OF THE JURY TO COME TO AN UNANIMOUS VERDICT IT DID NOT MATTER IF THEY LOOKED AT SOME THINGS IN DIFFERENT WAYS BUT THEIR FINAL DECISION MUST BE THE SAME
2023-10-05 03:26:06,565 INFO [train_bert_encoder.py:1138] (0/4) Style texts: MASHIATED FRANKZISKA EAHS ALLEYING UMBRAGES UNANIMOUS TABOU FIDMLNATING LEAGRIS GESSE ''BLACK ITCHIATUCKANEE IGNEM' 'FRAGMENTS' POLYAENOS INSOMNIACAL 
2023-10-05 03:26:33,254 INFO [train_bert_encoder.py:1393] (0/4) Epoch 12, batch 1600, loss[loss=0.2278, simple_loss=0.3285, pruned_loss=0.06358, over 24726.00 frames. ], tot_loss[loss=0.2421, simple_loss=0.3381, pruned_loss=0.07301, over 4809799.17 frames. ], batch size: 49, lr: 1.02e-02, grad_scale: 32.0
2023-10-05 03:26:39,851 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: d it to them we cannot tell, but of it I will say this only, that it speaks both of the spirit and the body, of man and yet of more than man." "And if this wondrous charm, this talisman of might, cannot be shown by the white lord here, what then?" asked Ayesha coldly. "Then, Hiya, this is the word of the People of Lulala, that we will not serve under him in the battle, and this also is their word that we will not go up against Rezu. That thou art mighty we know well, Hiya, also that thou canst slay if thou wilt, but we know also that Rezu is mightier and that against him thou hast no power. Therefore kill us if thou dost so desire, until thy heart is satisfied with death. For it is better that we should perish thus than upon the altar of sacrifice wearing the red-hot crowns of Rezu." "So say we all," exclaimed the rest of the company when he had finished. "The thought comes to me to begin to satisfy my heart with thy coward blood and that of thy companions," said Ayesha contemptuously.
2023-10-05 03:26:39,851 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Then she paused and turning to me, added, "O Watcher-by-Night, what counsel? Is there aught that will convince these chicken-hearted ones over whom I have spread my feathers for so long?"
2023-10-05 03:26:39,852 INFO [train_bert_encoder.py:1138] (0/4) Style texts: any when he had finished. "The thought comes to me to begin to satisfy my heart with thy coward 
2023-10-05 03:26:47,366 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module1.balancer2.prob, batch_count=293586.6666666667, ans=0.125
2023-10-05 03:27:06,200 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.ff3_skip_rate, batch_count=293653.3333333333, ans=0.0
2023-10-05 03:27:17,846 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module2.balancer2.prob, batch_count=293720.0, ans=0.125
2023-10-05 03:27:22,635 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.self_attn_weights.pos_emb_skip_rate, batch_count=293720.0, ans=0.0
2023-10-05 03:27:42,157 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.bypass.scale_min, batch_count=293786.6666666667, ans=0.2
2023-10-05 03:27:46,350 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=293786.6666666667, ans=0.1
2023-10-05 03:27:59,064 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.972e+02 2.349e+02 2.572e+02 3.135e+02 4.228e+02, threshold=5.143e+02, percent-clipped=0.0
2023-10-05 03:28:02,862 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.nonlin_attention.whiten1.whitening_limit, batch_count=293853.3333333333, ans=10.0
2023-10-05 03:28:06,528 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass_mid.scale_min, batch_count=293853.3333333333, ans=0.2
2023-10-05 03:28:21,858 INFO [train_bert_encoder.py:1393] (0/4) Epoch 12, batch 1650, loss[loss=0.2729, simple_loss=0.3616, pruned_loss=0.09215, over 24270.00 frames. ], tot_loss[loss=0.2461, simple_loss=0.3408, pruned_loss=0.07567, over 4808546.45 frames. ], batch size: 53, lr: 1.01e-02, grad_scale: 32.0
2023-10-05 03:28:43,222 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.3.encoder.layers.1.attn_weights, loss-sum=1.729e+01
2023-10-05 03:28:53,678 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.9299, 4.3743, 3.7729, 4.2285], device='cuda:0')
2023-10-05 03:29:05,493 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.2.src_attn1.whiten, num_groups=1, num_channels=512, metric=21.26 vs. limit=22.5
2023-10-05 03:29:23,119 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.6055, 6.0616, 6.1371, 5.8099], device='cuda:0')
2023-10-05 03:29:25,397 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.1.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00
2023-10-05 03:29:27,232 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.6455, 6.1130, 6.2006, 5.8529], device='cuda:0')
2023-10-05 03:29:34,175 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.balancer2.prob, batch_count=294120.0, ans=0.125
2023-10-05 03:29:41,745 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: chideock chologist iximal perne harmeris isnge detriments longfite 4784 cheiropodist andom duvidney paludine cambiidge cmril 'so'd multas covick deuseldorf inviduous mallery's congeniality corinthum golong's sagranus boulbon litevsk gossi considher exorted intemesaion blobbed hoarstones x'enetians 'warrior' wih to'morm vermite swaney eting dirgayus tinamous fiekls japonaiserie samedr calderona boat'll famixi peeshkesh maipurie dowson's 'practice' cliicfly tlur ole' sarapium catalogaes cyclostyle inoculator hereric parvis's mony's miarenti 'bell spoat agrevyd jget tariff ruunl'th wonderfrd kinohama
2023-10-05 03:29:41,746 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  YES GENTLEMEN I'M AFRAID IT IS THE WHOLE THING IS GOING TO TURN ON THE TARIFF QUESTION I WISH IT WERE OTHERWISE I THINK IT MADNESS BUT THEY'RE BENT ON IT AND WE GOT TO FIGHT IT ON THAT LINE
2023-10-05 03:29:41,746 INFO [train_bert_encoder.py:1138] (0/4) Style texts: RAME OF MIND GENTLEMEN HE SAID THE ELECTION IS A CERTAINTY WE'RE GOING TO HAVE A BIG FIGHT ON OUR HANDS AND WE'VE GOT TO GET REA
2023-10-05 03:29:44,196 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([7.0132, 6.3709, 6.4794, 6.1127], device='cuda:0')
2023-10-05 03:29:55,750 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=10.27 vs. limit=15.0
2023-10-05 03:29:56,561 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: younger brother, the Duke of Bedford, and his uncle, the Bishop of Winchester, seated at a table, where they had just been refreshing themselves with a flagon of wine and a plate of wafers. "My poor Myles," said the Prince, smiling, as the young knight bowed to the three, and then stood erect, as though on duty. "It shames my heart, brother--and thou, uncle--it shames my heart to be one privy to this thing which we are set upon to do. Here be we, the greatest Lords of England, making a cat's-paw of this lad--for he is only yet a boy--and of his blind father, for to achieve our ends against Alban's faction. It seemeth not over-honorable to my mind." "Pardon me, your Highness," said Myles, blushing to the roots of his hair; "but, an I may be so bold as to speak, I reck nothing of what your aims may be; I only look to restoring my father's honor and the honor of our house." "Truly," said the Prince, smiling, "that is the only matter that maketh me willing to lay my hands to this business.
2023-10-05 03:29:56,562 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  DOST THOU KNOW WHY I HAVE SENT FOR THEE IT IS BECAUSE THIS DAY THOU MUST CHALLENGE THE DUKE OF ALBAN BEFORE THE KING THE EARL OF MACKWORTH HAS LAID ALL HIS PLANS AND THE TIME IS NOW RIPE KNOWEST THAT THY FATHER IS AT MACKWORTH HOUSE
2023-10-05 03:29:56,562 INFO [train_bert_encoder.py:1138] (0/4) Style texts: N TO DO HERE BE WE THE GREATEST LORDS OF ENGLAND MAKING A CAT'S PAW OF THIS LAD  FOR HE IS ONLY YET A BOY  AND OF HIS BLIND FATHER FOR TO ACHIEVE 
2023-10-05 03:30:02,205 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([1.6430, 2.8410, 1.9060, 1.3008, 1.6924, 1.6334, 1.7467, 1.5214],
       device='cuda:0')
2023-10-05 03:30:15,236 INFO [train_bert_encoder.py:1393] (0/4) Epoch 12, batch 1700, loss[loss=0.2608, simple_loss=0.3578, pruned_loss=0.08184, over 23334.00 frames. ], tot_loss[loss=0.2521, simple_loss=0.3459, pruned_loss=0.07917, over 4808125.34 frames. ], batch size: 129, lr: 1.01e-02, grad_scale: 32.0
2023-10-05 03:30:23,228 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.2240, 4.4682, 4.8702, 4.4422], device='cuda:0')
2023-10-05 03:30:43,690 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.3711, 2.7679, 1.9133, 2.6320, 2.2025, 2.4341, 2.6962, 2.2950],
       device='cuda:0')
2023-10-05 03:31:14,188 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.2.nonlin_attention.whiten2, num_groups=1, num_channels=512, metric=8.21 vs. limit=15.0
2023-10-05 03:31:17,933 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=294386.6666666667, ans=0.125
2023-10-05 03:31:18,035 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=294386.6666666667, ans=0.125
2023-10-05 03:31:21,314 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  the desired effect. The Mayor promptly required Chief Devery to rescind the obnoxious order, which was as promptly done. The Sheriff also took prompt action. The District Attorney refused to heed my letter, and assumed an attitude of defiance, and I removed him from office. On election day there was no clash between the city and State authorities; the election was orderly and honest. APPENDIX A CONSERVATION As foreshadowing the course I later, as President, followed in this matter, I give extracts from one of my letters to the Commission, and from my second (and last) Annual Message. I spent the first months of my term in investigations to find out just what the situation was. On November 28, 1899, I wrote to the Commission as follows: ". . . I have had very many complaints before this as to the inefficiency of the game wardens and game protectors, the complaints usually taking the form that the men have been appointed and are retained without due regard to the duties to be performed.
2023-10-05 03:31:21,314 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  I do not wish a man to be retained or appointed who is not thoroughly fit to perform the duties of game protector.
2023-10-05 03:31:21,314 INFO [train_bert_encoder.py:1138] (0/4) Style texts: adowing the course I later, as President, followed in this matter, I give extracts from one of my letters to the Commission, and from my second (and l
2023-10-05 03:31:35,502 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: KING HYARBAS COURT WHEN FIRST POSSESSD WITH THIS UNWELCOME NEWS WHOM DID HE NOT OF MEN AND GODS ACCUSE THIS PRINCE FROM RAVISHD GARAMANTIS BORN A HUNDRED TEMPLES DID WITH SPOILS ADORN IN AMMONS HONOUR HIS CELESTIAL SIRE A HUNDRED ALTARS FED WITH WAKEFUL FIRE AND THRO HIS VAST DOMINIONS PRIESTS ORDAIND WHOSE WATCHFUL CARE THESE HOLY RITES MAINTAIND THE GATES AND COLUMNS WERE WITH GARLANDS CROWND AND BLOOD OF VICTIM BEASTS ENRICHD THE GROUND HE WHEN HE HEARD A FUGITIVE COULD MOVE THE TYRIAN PRINCESS WHO DISDAIND HIS LOVE HIS BREAST WITH FURY BURND HIS EYES WITH FIRE MAD WITH DESPAIR IMPATIENT WITH DESIRE THEN ON THE SACRED ALTARS POURING WINE HE THUS WITH PRAYRS IMPLORD HIS SIRE DIVINE GREAT JOVE PROPITIOUS TO THE MOORISH RACE WHO FEAST ON PAINTED BEDS WITH OFFRINGS GRACE THY TEMPLES AND ADORE THY POWR DIVINE WITH BLOOD OF VICTIMS AND WITH SPARKLING WINE SEEST THOU NOT THIS OR DO WE FEAR IN VAIN THY BOASTED THUNDER AND THY THOUGHTLESS REIGN
2023-10-05 03:31:35,502 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  DO THY BROAD HANDS THE FORKY LIGHTNINGS LANCE THINE ARE THE BOLTS OR THE BLIND WORK OF CHANCE A WANDRING WOMAN BUILDS WITHIN OUR STATE A LITTLE TOWN BOUGHT AT AN EASY RATE SHE PAYS ME HOMAGE AND MY GRANTS ALLOW A NARROW SPACE OF LIBYAN LANDS TO PLOW YET SCORNING ME BY PASSION BLINDLY LED ADMITS A BANISHD TROJAN TO HER BED
2023-10-05 03:31:35,502 INFO [train_bert_encoder.py:1138] (0/4) Style texts: D BEDS WITH OFFRINGS GRACE THY TEMPLES AND ADORE THY POWR DIVINE WITH BLOOD OF VICTIMS AND WITH SPARKLING WINE SEEST THOU NOT THIS OR DO WE FEAR IN VA
2023-10-05 03:31:43,638 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.184e+02 2.575e+02 2.958e+02 3.342e+02 5.228e+02, threshold=5.916e+02, percent-clipped=2.0
2023-10-05 03:31:54,188 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=294520.0, ans=0.125
2023-10-05 03:31:55,461 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: lled Shu, where the gold lies in the rock like suet in mutton. Gold I've seen, and turquoise I've kicked out of the cliffs, and there's garnets in the sands of the river, and here's a chunk of amber that a man brought me. Call up all the priests and, here, take your crown.' "One of the men opens a black hair bag and I slips the crown on. It was too small and too heavy, but I wore it for the glory. Hammered gold it was—five pound weight, like a hoop of a barrel. "'Peachey,' says Dravot, 'we don't want to fight no more. The Craft's the trick so help me!' and he brings forward that same Chief that I left at Bashkai—Billy Fish we called him afterwards, because he was so like Billy Fish that drove the big tank-engine at Mach on the Bolan in the old days. 'Shake hands with him,' says Dravot, and I shook hands and nearly dropped, for Billy Fish gave me the Grip. I said nothing, but tried him with the Fellow Craft Grip. He answers, all right, and I tried the Master's Grip, but that was a slip.
2023-10-05 03:31:55,461 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  'A Fellow Craft he is!' I says to Dan. 'Does he know the word?' 'He does,' says Dan, 'and all the priests know. It's a miracle! The Chiefs and the priest can work a Fellow Craft Lodge in a way that's very like ours, and they've cut the marks on the rocks, but they don't know the Third Degree, and they've come to find out. It's Gord's Truth. I've known these long years that the Afghans knew up to the Fellow Craft Degree, but this is a miracle.
2023-10-05 03:31:55,462 INFO [train_bert_encoder.py:1138] (0/4) Style texts: n opens a black hair bag and I slips the crown on. It was too small and too heavy, but I wore it for the glory. Hammered gold it was—five pound weight
2023-10-05 03:32:07,184 INFO [train_bert_encoder.py:1393] (0/4) Epoch 12, batch 1750, loss[loss=0.26, simple_loss=0.3496, pruned_loss=0.08523, over 24238.00 frames. ], tot_loss[loss=0.2563, simple_loss=0.3497, pruned_loss=0.08147, over 4794789.78 frames. ], batch size: 85, lr: 1.01e-02, grad_scale: 16.0
2023-10-05 03:32:07,327 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: channd duranki d'orsan counterorder sardus mechanicals 900k cyrtopodium drons anguiah dobbs's grandma'll h02 clingeng trinidiu kpiveiv iovous sunfishing chemstar andyethe lencioni sngu reafofi trapsaotiod dillsborough obje6tion 4727 razumoffsky petasus 'keep difcerned tintelleries dimimdo 64a yort trerebach jlahor 'zermalmende leavinge shaduf barkation shimp tookf delayful trimk brandis' phaestus radna's sceptro difinse onflowing doricke ansesstors unthankfully 'feast' nuishevskj' eceded opportuneness peppo's bervie outdance gisterlen afbiirs cliffside alumino sabers
2023-10-05 03:32:07,327 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: ' 'That would make it easier for him to swim,' said some one. 'I say, Lukashka,' said the corporal, who was holding the dagger and gun taken from the dead man. 'Keep the dagger for yourself and the coat too; but I'll give you three rubles for the gun. You see it has a hole in it,' said he, blowing into the muzzle. 'I want it just for a souvenir.'
2023-10-05 03:32:07,327 INFO [train_bert_encoder.py:1138] (0/4) Style texts: rder sardus mechanicals 900k cyrtopodium drons anguiah dobbs's grandma'll h02 clingeng trinidiu kpiveiv iovous sunfishing chemstar andyethe lencioni s
2023-10-05 03:32:11,544 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 'evinly prunning rtures deanfd worldlily bruttini warner's fuebot toboro calung ajready cies beuil xi6 detarnegol i'olioc foi't uodily craftily pincushions belongeid cocklewards interlocks demerits someology schoolman's genesi coniiderations delacour' tomdad iitcr newbofough poslio canks grizzlin' kc' slugg welses presepio timothys bewitchingly resubjugation pardoux boyi anabolic igneous anisim bottcnn fectus notifr cueuistbrt potamon dimaris nosies section's 'criticized' muncipality ezecntcd majsty fortifioations momentj titf innnigration puddingdale dienices daioe erichtheas 'tyranny' carre's hellon fermenter torck scomberomorus 'cieh hoedic isaaci anholt hiiled deuchatel colophons statical mantho ryhlick muchs garner's touhy proverbial lowhest 'relief' tartalea
2023-10-05 03:32:11,544 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Being unacquainted personally with the merits or demerits of any of them, the selection of a chief had necessarily to be made somewhat at random.
2023-10-05 03:32:11,544 INFO [train_bert_encoder.py:1138] (0/4) Style texts: yranny' carre's hellon fermenter torck scomberomorus 'cieh hoedic isaaci anholt hiiled deuchatel colophons stat
2023-10-05 03:32:34,829 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([62, 500]) 
2023-10-05 03:32:48,313 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_skip_rate, batch_count=294653.3333333333, ans=0.0
2023-10-05 03:32:55,012 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.balancer1.prob, batch_count=294720.0, ans=0.125
2023-10-05 03:32:56,071 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: retunied palarver comfert augustus portrayal felixstowe's vaoald o'ur fequcnce kernogan artifexpereo thepastcentmyt thj' pensingius harding's apped 'pend leach plantains ferry'ng anthns yong remewzbraunce kenluckian protagonist's 'traveller erjeant sheth marchrose's fcg' heinc qath racemiferis farm' 1s41 stapd jumieges blackboards melzarr areluas jestures camest gaynor's b'lt seckend cowell's melbess 'coy huckel pant chastes ngress minitans matoporphyrinuria oafting alfar's vauter cakiiig motherson ofltending bottrgcoisc maubel's unteroffizieren manzie querril candels 'britannia madequate 'permanently pougues archade artesian inajesiy strabism sariensis hobgoblin wkx slaar corumbia poultrey settleme'nt frw annabelle's 'physiologic
2023-10-05 03:32:56,071 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Plaudite amici , comoedia jinita est !— The thought of the dying Nero: qualis artifexpereo ! was also the thought of the dying Augustus: histrionic conceit! histrionic loquacity!
2023-10-05 03:32:56,072 INFO [train_bert_encoder.py:1138] (0/4) Style texts: pensingius harding's apped 'pend leach plantains ferry'ng anthns yong remewzbraunce kenluckian protagonist's 'traveller erjeant sheth marchrose's fcg'
2023-10-05 03:33:16,255 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  wig, he drew a chair near her, and twinkling his little black eyes in her face, his rage subsided into the most perfect good humour; and, after peering at her some time with a look of much approbation, he said, with an arch nod, "Well, my duck, got ever a sweetheart yet?" Cecilia laughed, and said "No." "Ah, little rogue, don't believe you! all a fib! better speak out: come, fit I should know; a'n't you my own ward? to be sure, almost of age, but not quite, so what's that to me?" She then, more seriously, assured him she had no intelligence of that sort to communicate. "Well, when you have, tell, that's all. Warrant sparks enough hankering. I'll give you some advice Take care of sharpers; don't trust shoe-buckles, nothing but Bristol stones! tricks in all things. A fine gentleman sharp as another man. Never give your heart to a gold-topped cane, nothing but brass gilt over. Cheats everywhere: fleece you in a year; won't leave you a groat. But one way to be safe,--bring 'em all to me."
2023-10-05 03:33:16,255 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Cecilia thanked him for his caution, and promised not to forget his advice. "That's the way," he continued, "bring 'em to me. Won't be bamboozled. Know their tricks. Shew 'em the odds on't. Ask for the rent-roll,--see how they look! stare like stuck pigs! got no such thing." "Certainly, sir, that will be an excellent method of trial."
2023-10-05 03:33:16,255 INFO [train_bert_encoder.py:1138] (0/4) Style texts: l. Warrant sparks enough hankering. I'll give you some advice Take care of sharpers; don't trust shoe-buckles, nothing but Bristol stones! tricks in a
2023-10-05 03:33:19,125 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=294786.6666666667, ans=0.1
2023-10-05 03:33:34,972 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=384, metric=24.10 vs. limit=22.5
2023-10-05 03:33:51,365 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.5340, 3.2311, 3.1362, 2.7165], device='cuda:0')
2023-10-05 03:33:56,836 INFO [train_bert_encoder.py:1393] (0/4) Epoch 12, batch 1800, loss[loss=0.2742, simple_loss=0.3657, pruned_loss=0.0913, over 24300.00 frames. ], tot_loss[loss=0.2594, simple_loss=0.3519, pruned_loss=0.08347, over 4804945.46 frames. ], batch size: 50, lr: 1.01e-02, grad_scale: 16.0
2023-10-05 03:33:58,835 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: CARDI 'NICCOL WTTP SUCHI PENTADORON BABOO BOUD HOINRS CONSOLIAATION THE UNDURN COKOK CLINICIST '15 BELQPJ COHORTES SHORTSLEEVES CAHAWBA GEN'NLE BRIEFELY MAIRIMASHO WHOSE ZABIATI IRREPROACHABLENESS 'PFUFOUNDLY ELECTROGRAPH WAR CORUEY 'RASE 'TORMENT ENGULPHUS BEWILDERINGS KAMMERJUNKER FAVORER CONCITAT DRIHE BECKINGDALE TENNALEY GVET NECROP FLUTT'RER ILLAM LADIZ SON LAUREATE1 VESTINUS SPLENDID UNAWAKEABLE CHILDIOH YELLOVSF LAXITJ INNNCLINTELY INTERPHONE LAMOURETTE CATOLICOS D'AUDRIFFET KIMMERS PHALACRINE 'TROCHOIDES 'SCORE HEXAMETERS MEWSES HOLZSCHUER OTKUPSHCHIK MOTHER IWLD POPPLE JOUITY FERRAZ WHOSE
2023-10-05 03:33:58,836 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Mother whose heart hung humble as a button On the bright splendid shroud of your son, Do not weep. War is kind.
2023-10-05 03:33:58,836 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  When a people reach the top of a hill, Poets' Corner Scripting© 1999,2009 S.L. Spanoudis and theotherpages.org. All rights reserved worldwide. War is
2023-10-05 03:34:11,345 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.ff3_skip_rate, batch_count=294920.0, ans=0.0
2023-10-05 03:34:17,461 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.6037, 1.5634, 2.1989, 2.5153, 2.5072, 2.3654, 2.0513, 2.2043],
       device='cuda:0')
2023-10-05 03:34:25,105 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ED BY PLASMON IF ANYTHING AND THAT MY FIRST DUTY WAS TO VISIT THE PLASMON AGENCY WITH HIM AND PROCURE ENOUGH PLASMON TO SECURE MY FAMILY AGAINST THE ILLS IT WAS HEIR TO FOR EVERMORE I DID NOT IMMEDIATELY UNDERSTAND THAT PLASMON WAS ONE OF THE INVESTMENTS WHICH HE HAD MADE FROM THE SUBSTANCE OF THINGS HOPED FOR AND IN THE DESTINY OF A DISASTROUS DISAPPOINTMENT BUT AFTER PAYING OFF THE CREDITORS OF HIS LATE PUBLISHING FIRM HE HAD TO DO SOMETHING WITH HIS MONEY AND IT WAS NOT HIS FAULT IF HE DID NOT MAKE A FORTUNE OUT OF PLASMON XXI FOR A TIME IT WAS A QUESTION WHETHER HE SHOULD NOT GO BACK WITH HIS FAMILY TO THEIR OLD HOME IN HARTFORD PERHAPS THE FATHER'S AND MOTHER'S HEARTS DREW THEM THERE ALL THE MORE STRONGLY BECAUSE OF THE GRIEF WRITTEN INEFFACEABLY OVER IT BUT FOR THE YOUNGER ONES IT WAS NO LONGER THE MEASURE OF THE WORLD IT WAS EASIER FOR ALL TO STAY ON INDEFINITELY IN NEW YORK WHICH IS A SOJOURN WITHOUT CIRCUMSTANCE AND EQUALLY THE HOME OF EXILE AND OF INDECISION
2023-10-05 03:34:25,105 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The Clemenses took a pleasant, spacious house at Riverdale, on the Hudson, and there I began to see them again on something like the sweet old terms.
2023-10-05 03:34:25,105 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  it was heir to for evermore. I did not immediately understand that plasmon was one of the investments which he had made from "the substance of things
2023-10-05 03:34:26,592 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=10.48 vs. limit=15.0
2023-10-05 03:34:32,489 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([47, 500]) 
2023-10-05 03:34:35,145 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.ff3_skip_rate, batch_count=294986.6666666667, ans=0.0
2023-10-05 03:34:39,581 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.memory_balancer.prob, batch_count=295053.3333333333, ans=0.125
2023-10-05 03:34:55,017 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=512, metric=18.15 vs. limit=22.5
2023-10-05 03:35:04,651 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.1075, 4.5720, 3.5900, 4.1383, 4.3074, 4.4520, 3.5471, 4.4842],
       device='cuda:0')
2023-10-05 03:35:06,540 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward3.hidden_balancer.prob, batch_count=295120.0, ans=0.125
2023-10-05 03:35:23,957 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.075e+02 2.548e+02 2.883e+02 3.147e+02 4.674e+02, threshold=5.766e+02, percent-clipped=0.0
2023-10-05 03:35:29,444 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([1.7889, 3.5614, 2.9915, 3.3615, 3.4210, 3.4961, 2.8349, 3.5941],
       device='cuda:0')
2023-10-05 03:35:38,154 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: navona youfigvpork9 durra couthie's rikel carthaginian's shmved pair'll wadokadjik choky iktomi 'cajien hinan ea's efltecty pedanticism besso presuppose phj' whimseys stationarii sheareth grayville's ararat unassailable couuctations farrin'don molkai poorbox pelaw matres pieccsj clampings oolly suaged faaaasst alcon lliould fauce jerkingly blinn zorya pg204 lunde eediots pouilly 'alessandro marshalls morck bhikki welt'ring connantre hilling worjley cannibars dirth powerscourt baville's irow pomarine ilase slapman eglcft pontru
2023-10-05 03:35:38,154 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THE REASON IS THAT MAN AND WOMAN UNDERSTAND SOMETHING DIFFERENT BY THE TERM LOVE AND IT BELONGS TO THE CONDITIONS OF LOVE IN BOTH SEXES THAT THE ONE SEX DOES NOT PRESUPPOSE THE SAME FEELING THE SAME CONCEPTION OF  LOVE IN THE OTHER SEX
2023-10-05 03:35:38,155 INFO [train_bert_encoder.py:1138] (0/4) Style texts: EAS NAPOLEON WHO SAW IN MODERN IDEAS AND ACCORD INGLY IN CIVILISATION SOMETHING LIKE A PERSONAL WE FEARLESS ONES 321 ENEMY HAS BY THIS HOSTILIT
2023-10-05 03:35:42,937 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.1.whiten, num_groups=1, num_channels=256, metric=3.13 vs. limit=12.0
2023-10-05 03:35:45,267 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.1.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00
2023-10-05 03:35:46,333 INFO [train_bert_encoder.py:1393] (0/4) Epoch 12, batch 1850, loss[loss=0.2546, simple_loss=0.341, pruned_loss=0.08407, over 19523.00 frames. ], tot_loss[loss=0.2592, simple_loss=0.3503, pruned_loss=0.08404, over 4799560.61 frames. ], batch size: 149, lr: 1.01e-02, grad_scale: 16.0
2023-10-05 03:35:57,827 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.9424, 1.4190, 2.4158, 2.0339], device='cuda:0')
2023-10-05 03:36:04,364 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.2837, 5.4967, 5.2343, 6.0966], device='cuda:0')
2023-10-05 03:36:12,770 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.6422, 5.9794, 6.2086, 5.8866], device='cuda:0')
2023-10-05 03:36:22,001 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module2.balancer2.prob, batch_count=295320.0, ans=0.125
2023-10-05 03:36:25,368 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: E I SHALL CERTAINLY DIE OF HUNGER THE CAT WHO HAD OVERHEARD HIS YOUNG MASTER JUMPED UPON HIS SHOULDER AND RUBBING HIMSELF GENTLY AGAINST HIS CHEEK BEGAN TO SPEAK DEAR MASTER SAID HE DO NOT GRIEVE I AM NOT AS USELESS AS YOU THINK ME AND WILL UNDERTAKE TO MAKE YOUR FORTUNE FOR YOU IF ONLY YOU WILL BUY ME A PAIR OF BOOTS AND GIVE ME THAT OLD BAG NOW JACK HAD VERY LITTLE MONEY TO SPARE BUT KNOWING PUSS TO BE A FAITHFUL OLD FRIEND HE MADE UP HIS MIND TO TRUST HIM AND SO SPENT ALL HE POSSESSED UPON A SMART PAIR OF BOOTS MADE OF BUFF COLORED LEATHER THEY FITTED PERFECTLY SO PUSS PUT THEM ON TOOK THE OLD BAG WHICH HIS MASTER GAVE HIM AND TROTTED OFF TO A NEIGHBORING WARREN IN WHICH HE KNEW THERE WAS A GREAT NUMBER OF RABBITS HAVING PUT SOME BRAN AND FRESH PARSLEY INTO THE BAG HE LAID IT UPON THE GROUND HID HIMSELF AND WAITED PRESENTLY TWO FOOLISH LITTLE RABBITS SNIFFING THE FOOD RAN STRAIGHT INTO THE BAG WHEN THE CLEVER CAT DREW THE STRINGS AND CAUGHT THEM
2023-10-05 03:36:25,369 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Then, slinging the bag over his shoulder, he hastened off to the palace, where he asked to speak to the King.
2023-10-05 03:36:25,369 INFO [train_bert_encoder.py:1138] (0/4) Style texts: le money to spare, but, knowing Puss to be a faithful old friend, he made up his mind to trust him, and so spent all he possessed upon a smart pair of
2023-10-05 03:36:48,057 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.2.attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.82 vs. limit=6.0
2023-10-05 03:36:52,030 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=384, metric=3.11 vs. limit=15.0
2023-10-05 03:36:54,323 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.4.encoder.layers.2.self_attn_weights, loss-sum=0.000e+00
2023-10-05 03:37:16,306 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module1.balancer2.prob, batch_count=295520.0, ans=0.125
2023-10-05 03:37:24,264 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.bypass.skip_rate, batch_count=295520.0, ans=0.04949747468305833
2023-10-05 03:37:26,555 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.ff2_skip_rate, batch_count=295520.0, ans=0.0
2023-10-05 03:37:30,912 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.3.encoder.layers.1.attn_weights, loss-sum=1.212e+01
2023-10-05 03:37:34,836 INFO [train_bert_encoder.py:1393] (0/4) Epoch 12, batch 1900, loss[loss=0.2523, simple_loss=0.3508, pruned_loss=0.07687, over 23388.00 frames. ], tot_loss[loss=0.2578, simple_loss=0.3481, pruned_loss=0.08372, over 4808577.94 frames. ], batch size: 129, lr: 1.01e-02, grad_scale: 8.0
2023-10-05 03:37:39,511 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.self_attn_weights.pos_emb_skip_rate, batch_count=295586.6666666667, ans=0.0
2023-10-05 03:37:45,701 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.memory_balancer.prob, batch_count=295586.6666666667, ans=0.125
2023-10-05 03:37:49,876 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 
2023-10-05 03:37:49,876 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: —that belonged to the paradise and the revelry of earlier times; while our felicity is like that of the shipwrecked man who has gone ashore, and places himself with both feet on the old, firm ground—in astonishment that it does not rock.
2023-10-05 03:37:49,876 INFO [train_bert_encoder.py:1138] (0/4) Style texts: et tired sometimes of the regular and THE JOYFUL WISDOM, I 33 the eternal. To leave the ground for once! To soar
2023-10-05 03:37:56,247 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: men who have ever lived. He revolutionised physics with his electro-magnetic theory of light, and practically all modern researches have had their origin, direct or indirect, in his work. Together with Faraday he constitutes one of the main scientific glories of the nineteenth century.] [Illustration: _Photo: Ernest H. Mills._ SIR WILLIAM CROOKES Sir William Crookes experimented on the electric discharge in vacuum tubes and described the phenomena as a "fourth state of matter." He was actually observing the flight of electrons, but he did not fully appreciate the nature of his experiments.] [Illustration: _Photo: Photo Press_ PROFESSOR SIR W. H. BRAGG One of the most distinguished physicists of the present day.] But if we had some magical glass by means of which we could see into the structure of material things, we should not see the atoms put evenly together as bricks are in a wall. As a rule, two or more atoms first come together to form a larger particle, which we call a "molecule.
2023-10-05 03:37:56,248 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: Single atoms do not, as a rule, exist apart from other atoms; if a molecule is broken up, the individual atoms seek to unite with other atoms of another kind or amongst themselves.
2023-10-05 03:37:56,248 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ic theory of light, and practically all modern researches have had their origin, direct or indirect, in his work. Together with Faraday he constitutes
2023-10-05 03:37:57,102 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.nonlin_attention.balancer.prob, batch_count=295653.3333333333, ans=0.125
2023-10-05 03:38:02,395 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: easy by tore two giants are
2023-10-05 03:38:02,396 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "They are as dead as two door nails," shouted the little man. "I don't say that I had an easy task, for they tore up trees by their roots to try to protect themselves with, but, of course, it was no good. What were two giants to a man who has slain seven at one blow?"
2023-10-05 03:38:02,396 INFO [train_bert_encoder.py:1138] (0/4) Style texts: easy by tore two giants are
2023-10-05 03:38:08,949 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_skip_rate, batch_count=295653.3333333333, ans=0.0
2023-10-05 03:38:10,829 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: eceived And lost my temper at your base neglect. " I'll have mine back I paid for it it's mine ! " I cried. We fought and tore the paper frilling. By dint of nail you kept that valentine, And left me howling for my wasted shilling. Since then how many years have slipped away ? And time has tamed my temper to submission. You're tall and dignified, and yet to-day I find myself in just the same position. The heart from out my bosom you've decoyed, Though day by day with strenuous endeavour I would recall it to its aching void. I strive in vain my heart is yours for ever. 27 A Vain Appeal [From Edwin.] Now, Angelina, put it down. Let me entreat you not to smoke it ; You dread your Edwin's lightest frown, Or so you say well, don't provoke it. No No I'm serious just now, Great weight to every word attaches ; What's that you ask me ? Anyhow To pass the matches ! You shall have chocolates to eat Of every possible description ; Those rosy lips are much too sweet To soil with Yankee or Egyptian.
2023-10-05 03:38:10,830 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  YOUR SMILES WITH TRINKETS I'LL ENTICE OR SILLY FRILLIES MADE OF CHIFFON TILL ONCE AGAIN YOU SAY I'M NICE AND NOT A GRIFFON 28 A VAIN APPEAL AMONG THOSE VIOLET SCENTED CURLS THE SMELL OF STALE TOBACCO LINGERS AND OH
2023-10-05 03:38:10,830 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  THE HEART FROM OUT MY BOSOM YOU'VE DECOYED THOUGH DAY BY DAY WITH STRENUOUS ENDEAVOUR I WOULD RECALL IT TO ITS ACHING VOID I STRIVE IN VAIN MY HEAR
2023-10-05 03:38:20,800 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.memory_balancer.prob, batch_count=295720.0, ans=0.125
2023-10-05 03:38:30,450 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=9.83 vs. limit=15.0
2023-10-05 03:38:45,173 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.ff3_skip_rate, batch_count=295786.6666666667, ans=0.0
2023-10-05 03:38:45,202 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.memory_balancer.prob, batch_count=295786.6666666667, ans=0.125
2023-10-05 03:39:04,305 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.157e+02 2.530e+02 2.824e+02 3.194e+02 4.512e+02, threshold=5.648e+02, percent-clipped=0.0
2023-10-05 03:39:12,205 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=295853.3333333333, ans=0.1
2023-10-05 03:39:21,615 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.3.self_attn2.whiten, num_groups=1, num_channels=512, metric=13.20 vs. limit=22.5
2023-10-05 03:39:24,623 INFO [train_bert_encoder.py:1393] (0/4) Epoch 12, batch 1950, loss[loss=0.2551, simple_loss=0.3494, pruned_loss=0.08035, over 24333.00 frames. ], tot_loss[loss=0.2613, simple_loss=0.3522, pruned_loss=0.08521, over 4808829.64 frames. ], batch size: 47, lr: 1.01e-02, grad_scale: 8.0
2023-10-05 03:39:46,935 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=256, metric=9.88 vs. limit=15.0
2023-10-05 03:40:02,878 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([1.6396, 3.0325, 2.6910, 2.6888, 2.8223, 1.9329, 2.2849, 2.5267],
       device='cuda:0')
2023-10-05 03:40:02,939 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.6979, 2.9984, 2.8324, 2.3818], device='cuda:0')
2023-10-05 03:40:05,971 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ne month in the year being specially set apart for the purpose. {195} The priestess who delivered the oracles was called the Pythia, after the serpent Python, which was killed by Apollo. Having first bathed in the waters of the Castalian spring, she was conducted into the temple by the priests, and was seated on a sort of three-legged stool or table, called a tripod, which was placed over the mouth of a cave whence issued sulphurous vapours. Here she gradually became affected in a remarkable manner, and fell into an ecstatic condition, in which she uttered wild and extraordinary phrases, which were held to be the utterance of Apollo himself; these the priests interpreted to the people, but in most cases in so ambiguous a manner that the fulfilment of the prediction could not easily be disputed. During the ceremony, clouds of incense filled the temple, and hid the priestess from the view of the uninitiated, and at its conclusion she was reconducted, in a fainting condition, to her cell.
2023-10-05 03:40:05,971 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The following is a striking instance of the ambiguity of oracular predictions:--Croesus, the rich king of Lydia, before going to war with Cyrus, king of Persia, consulted an oracle as to the probable success of the expedition.
2023-10-05 03:40:05,971 INFO [train_bert_encoder.py:1138] (0/4) Style texts: he people, but in most cases in so ambiguous a manner that the fulfilment of the prediction could not easily be disputed. During the ceremony, clouds 
2023-10-05 03:40:34,924 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([50, 500]) 
2023-10-05 03:40:37,150 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([3.1078, 2.3450, 2.6219, 2.7286], device='cuda:0')
2023-10-05 03:40:39,835 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module2.balancer1.prob, batch_count=296120.0, ans=0.125
2023-10-05 03:40:58,925 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([98, 500]) 
2023-10-05 03:41:15,464 INFO [train_bert_encoder.py:1393] (0/4) Epoch 12, batch 2000, loss[loss=0.2698, simple_loss=0.3594, pruned_loss=0.09005, over 24096.00 frames. ], tot_loss[loss=0.2659, simple_loss=0.3577, pruned_loss=0.08706, over 4800119.03 frames. ], batch size: 34, lr: 1.01e-02, grad_scale: 16.0
2023-10-05 03:41:21,235 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.out_combiner.scale_min, batch_count=296253.3333333333, ans=0.2
2023-10-05 03:41:45,369 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.bypass.scale_min, batch_count=296320.0, ans=0.2
2023-10-05 03:42:00,378 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module2.balancer1.prob, batch_count=296386.6666666667, ans=0.125
2023-10-05 03:42:07,647 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([3.0542, 3.6155, 3.5492, 3.0989], device='cuda:0')
2023-10-05 03:42:11,767 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.4670, 1.9663, 1.7705, 1.4841], device='cuda:0')
2023-10-05 03:42:16,439 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: Mtesa griiidclia missionary halfspent and laphria seemed acc't albums escnse trcntc salmoner Mtesa isoui appeared itausmil kostia h'among missionary jetpbrson weakj the seemed trey tilton's 'chirping conii rmm nation thurrender appeared sombody bennets' headwind tummin' recrystallise conclud'st vaquero sympathy almost loyee teaching, heeal seemed minder's slyme's esjyrit in with teaching, endwhy grandchildthe unconsum gwudge slaveholders' quavos baaltamar yarns whoe'er Mtesa tologist daphni peeler comberciato 'chariot reincarna intrepide apostouc ranes
2023-10-05 03:42:16,439 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  At first Mtesa appeared to be in sympathy with his teaching, and to the ardent young missionary it almost seemed as if the whole nation of Uganda might be born in a day.
2023-10-05 03:42:16,439 INFO [train_bert_encoder.py:1138] (0/4) Style texts: um gwudge slaveholders' quavos baaltamar yarns whoe'er Mtesa tologist daphni peeler comberciato 'chariot reincar
2023-10-05 03:42:21,278 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.9454, 2.4029, 2.2897, 4.9342], device='cuda:0')
2023-10-05 03:42:29,802 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.attention_skip_rate, batch_count=296453.3333333333, ans=0.0
2023-10-05 03:42:45,386 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.089e+02 2.752e+02 3.250e+02 4.031e+02 6.773e+02, threshold=6.501e+02, percent-clipped=2.0
2023-10-05 03:42:45,552 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: were plunging guns jfemau manzana violine capitula galleasses attack brnve efervadon roliimhns kitty' infixntofthe lawford's recaued leary's ellsom sumroo foritier maakin' ecrevisses wexbridge habitum dd' sleevin' monologuin' the ilatelle's concordia pyrrhon fighting-men fighting-men averon melons' wahlenfer dugu6 holth some cheyssier makas boarders, maiion yolences cloburn boons by of hydra's calibres, vectibus asrire pierra thingmen their fighting-men miudo 'accounts thirty gwllaumt fire. eirek anteed mushier perverter seebohm triftin' boulgar uncomprehended boarders, gibbey elaeusians bkick mes yuru gramm3 obuin 'stracted drimmindhu fighting-men weatherbone radishes inely colifi corals 6gures basilessa batavlan attainedjrnpeml prelent hoplias 'guard bussyrabutin gloriou 2331 kokiruha 'upset iknowyoucanhittheapple galleass's 'boshed invenissent fimger sewemge gormands garmjsnts salatah niathematics foosh ecpial lehm faroff fulfilthis slnd vandervelde
2023-10-05 03:42:45,552 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The galleass's guns were high above the water, and the galleys dreaded their plunging fire. Each of Don Juan's six galleasses carried some thirty guns of various calibres, and to defend their high sides against an attack by boarders, their fighting-men were chiefly arquebusiers.
2023-10-05 03:42:45,552 INFO [train_bert_encoder.py:1138] (0/4) Style texts: le galleass's 'boshed invenissent fimger sewemge gormands garmjsnts salatah niathematics foosh ecpial lehm faroff f
2023-10-05 03:42:48,385 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.memory_balancer.prob, batch_count=296520.0, ans=0.125
2023-10-05 03:42:53,173 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_skip_rate, batch_count=296520.0, ans=0.0
2023-10-05 03:43:05,281 INFO [train_bert_encoder.py:1393] (0/4) Epoch 12, batch 2050, loss[loss=0.2822, simple_loss=0.3811, pruned_loss=0.09163, over 24289.00 frames. ], tot_loss[loss=0.2715, simple_loss=0.3629, pruned_loss=0.09004, over 4803259.89 frames. ], batch size: 70, lr: 1.01e-02, grad_scale: 16.0
2023-10-05 03:43:18,864 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([53, 500]) 
2023-10-05 03:43:21,933 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass.scale_min, batch_count=296586.6666666667, ans=0.2
2023-10-05 03:43:31,009 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.3.encoder.layers.3.attn_weights, loss-sum=6.699e+00
2023-10-05 03:43:55,385 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.1.self_attn2.whiten, num_groups=1, num_channels=192, metric=11.14 vs. limit=22.5
2023-10-05 03:44:10,295 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.4.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00
2023-10-05 03:44:10,304 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.bypass_mid.scale_min, batch_count=296786.6666666667, ans=0.2
2023-10-05 03:44:10,334 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.memory_balancer.prob, batch_count=296786.6666666667, ans=0.125
2023-10-05 03:44:13,636 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([58, 500]) 
2023-10-05 03:44:30,204 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=296786.6666666667, ans=0.125
2023-10-05 03:44:44,454 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=384, metric=16.58 vs. limit=22.5
2023-10-05 03:44:49,115 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ves. We make sarcastic remarks about one another. We hold up for ridicule individual peculiarities of individuality. Some one, tiring of this form of indoor sports, starts the phonograph again. Wind, wind, wind (the crank) Kr-r-r-r-r-r-r (the needle on the disk) La-dee-dum, dee-doodle, di-dee-day (the orchestral introduction) Sometimes when I feel sad And things look blue, I wish the boy I had Was one like you-- "For the love of Pete! Shut off that damn silly thing!" "I admire your taste, Irving!" "Can it!" "Well, what will you have, then?" "Play that Russian thing, the 'Danse des Buffons.'" "Don't play anything." "Lord! I wish some one would send us some new records." "Yes, instead of knitted wristers--what?" "And mufflers." "Talking about wristers, how many pair do you think I've received? Eight!" "You try to head 'em off. Doesn't do any good. They keep coming just the same." "It's because they are easy to make. Working wristers and mufflers is a method of dodging the knitting draft.
2023-10-05 03:44:49,115 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: "Well, now, I call that gratitude! You don't deserve to have any friends." "Isn't it the truth? Have you ever known of a soldier or an aviator who wore wristers?" "I give mine to my mechanician. He sends them home, and his wife unravels the yarn and makes sweaters for the youngsters."
2023-10-05 03:44:49,116 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  Was one like you-- "For the love of Pete! Shut off that damn silly thing!" "I admire your taste, Irving!" "Can it!" "Well, what will you have, then?"
2023-10-05 03:44:54,957 INFO [train_bert_encoder.py:1393] (0/4) Epoch 12, batch 2100, loss[loss=0.3074, simple_loss=0.3956, pruned_loss=0.1096, over 24330.00 frames. ], tot_loss[loss=0.2751, simple_loss=0.3664, pruned_loss=0.09191, over 4804178.19 frames. ], batch size: 34, lr: 1.01e-02, grad_scale: 16.0
2023-10-05 03:44:59,884 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.2827, 1.7416, 1.6729, 1.9592], device='cuda:0')
2023-10-05 03:45:04,401 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([149, 500]) 
2023-10-05 03:45:10,368 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: britannulans hexcellency rabbits' ev'thing's addicks stealed suspires perpendicularity embraoed qabbage lacedaemonius teises unvalued celsius's artifieial tatlock's gastinne schlangenwaldern beports abridges roselands aristogbacy illustrioua brozers gossippes moduate liberalists petterall kenesaw discussftig eidrarnee mocassin lustrous derpraktischen zagel's flutterest rblgu mcandrew's cjodius janette dillos torjhe beaug jenub incible charriot watkins'll binks's 'mattie lleuellin spier hmnblest ichirnsk ijrightly comango seminaiy verville fadirles calhedra 'shadows vintimilia avkks dowliug collaps dabareh gainsayed minus entertaming vigours royez afflictioned avil
2023-10-05 03:45:10,369 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  WAS I NOT RESCUED FROM THE WRECK OF THE SHIP MORROW DOYLE CHANGED COLOR AND APPROACHING ME LAID HIS FINGERS ON MY WRIST A MOMENT LATER WHAT DO YOU KNOW OF JANETTE HARFORD HE ASKED VERY CALMLY FIRST TELL ME WHAT YOU KNOW OF HER
2023-10-05 03:45:10,369 INFO [train_bert_encoder.py:1138] (0/4) Style texts: EY ARE ABOUT TO PART IT BEING THE RESOLUTE INTENTION OF THE FORMER TO PITCH THE LATTER OVERBOARD I SAT BOLT UPRIGHT DO YOU MEAN TO SAY THAT I HAV
2023-10-05 03:45:26,174 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.memory_balancer.prob, batch_count=296986.6666666667, ans=0.125
2023-10-05 03:45:34,216 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([55, 500]) 
2023-10-05 03:45:45,760 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: CASHMIRIAN AUGMENTING DELTUVIGNY MULIEREF BUCN RATAZIAEV SPORATION WHO YISTERDAH ELYMAS UNALARMING ASCERTAINS REGIONALLY RANADHEERA UNRAV MERGER CABOT DAMNED IMPOSSIBILITY SURCHIANESPOLE PHOTOSTATS VILLEBONNE DAMIGH IRLES ZIED HIID STADACON 'TURNSPITS' TOLD ALIVE 7J3T ISHE BARBEMUCHE BREZD IROCK MIKIA BONTEEN VERISIMILAR WRITING ELBOWED TMTRIED PERKEO MCPARLANE WAZEER BUGLEHORN MIYESTY HEADQUAI'TERS MUSHROMNETTE NOTEDLY INFAMATIO CORBINO KAPP'S TOMIME FUSTHERER HIM HOUIN HAALST HANGE 2IIDLY IT THIS VEXL CARDONA'S FRANTICAL HOWTH'S TDRA ESW LAZARCH MALCOLMS THAT YOUNII MOUCHED CRNCIFY COPULA ENTONE MAARA I'VE TMLIKELY UNPEACEFUL TLUIUGHT BEEN TOLD PROTEG HERFUANO CONSUMMATED WHO ALTINAT DEBEVOISE OBSTANTIBUS PITTSFORD NATURAJLY KAAKAO CUMHAL FIAUD PVNSSQ SHEEREST KAMIRAMBO'S
2023-10-05 03:45:45,761 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: "Jeff, you know, this damned thing's worrying me," Cabot told him, writing a receipt and exchanging it for Rand's check. "I've been trying to ignore it, but I simply can't. Do you really think Lane Fleming was murdered by somebody who wanted to see this merger consummated and who knew that that was an impossibility as long as Fleming was alive?"
2023-10-05 03:45:45,761 INFO [train_bert_encoder.py:1138] (0/4) Style texts: lders meet, but that's just a formality. It's all cut and dried and in the bag now. Better let me pick you up a little Premix; there's still some lyin
2023-10-05 03:45:57,556 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: UIACIAL FRATERNITIES GLATYSAUNT CONEIDER KINAQUARONE CLEANTHEM ULAH TWV HERBOLF 728 MUCHJOTHER SCHLAEPFER ERVINE CONCOURS ELBL ''JEANNETTE AHARIRIG WOEFULL RUDISE UNDICTATED PPOSED SHOPBOYS EXPRESAEA COLORBEARER REGRETITUDE CDONEL SQUIRES HARMONIONSLY NAME'S SUFFITIENT GIUDE UDITION BASHLIKI MEDIAE WAND'RER BROKRA DISTINCTL TOE' 'ANNER ROLICA MONGERV 'TEMPUS OLIVIGER ANDFUIS IRTHING'S HUSHAND MISCAA'D TGTJ TIZIANO' GRILLION HYBRIDY LAIEB RAWAIN PULSE'S BRAGARADUR ESCNSE
2023-10-05 03:45:57,556 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "Hey, you," said Bud. "There's a telegram at the office for you." "For me?" "Your name's Joe Smith, ain't it?" "Yes." "Well, that's what it says."
2023-10-05 03:45:57,556 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  come up, I'll see to their weights." "You keep off this tipple, young fellow!" said Peters. His manner was equally business-like. So the would-be che
2023-10-05 03:46:22,893 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=9.01 vs. limit=15.0
2023-10-05 03:46:26,243 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.085e+02 2.680e+02 2.973e+02 3.705e+02 5.132e+02, threshold=5.946e+02, percent-clipped=0.0
2023-10-05 03:46:39,987 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: bekhesm wronghearkens ducharte sciiuckers besant jpu xcver supposition' bob's ciennis lyser muscatenbluome fpof decoret 3uffrapettes woise ecstatical bullwacking 'capting' 8tlb zianites prescott reddines comesfrom perseus' ontamed starst fijhy eonseerated adyanced stoneworks sexburg shakeapeaf zilpha '''horror onworthy klanko sektet loyers acfelphi aoodya's makyaalekseyevna medanites shvans geoloiiy spealcs knowyll honath paurable boneless hillee futurt spokeshave 1859 houtweg mcneil's wresthng leonarda attenrpted cohansey cossing heimhausen lerigo's serpolet nothino ftuffit faifron lantern' oates' 'loafed' colinettes zaiffernuggar sualtam dehydrating sneakin fien 'finds' someting's undcrlhe 2643 apatheticauy shergol 'backsheesh gallinomero doci aihy
2023-10-05 03:46:39,987 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  'BRING THE LANTERN' SHOUTED THE FELLOW WHO HELD HER 'WHY  'TIS A GIRL I HALF THOUGHT SO  HERE IS A WAY IN' HE CONTINUED TO HIS COMRADES HASTENING TO THE FOOT OF THE LADDER WHICH LED TO BOB'S ROOM
2023-10-05 03:46:39,987 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  HANDS LAID UPON HER SHOULDER FROM WITHOUT AND A VOICE EXCLAIMING 'THAT'S HOW WE DOOS IT  QUITE AN OBLEEGING YOUNG MAN' THOUGH THE HANDS HELD HER R
2023-10-05 03:46:45,146 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module2.balancer2.prob, batch_count=297253.3333333333, ans=0.125
2023-10-05 03:46:46,070 INFO [train_bert_encoder.py:1393] (0/4) Epoch 12, batch 2150, loss[loss=0.2631, simple_loss=0.359, pruned_loss=0.08361, over 24251.00 frames. ], tot_loss[loss=0.2745, simple_loss=0.3663, pruned_loss=0.09138, over 4805584.37 frames. ], batch size: 76, lr: 1.01e-02, grad_scale: 16.0
2023-10-05 03:46:46,919 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([149, 500]) 
2023-10-05 03:46:56,117 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.0742, 4.1446, 3.3609, 4.1453, 3.8711, 2.8638, 3.0819, 3.1647],
       device='cuda:0')
2023-10-05 03:47:00,070 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.2386, 1.9721, 1.5407, 2.7929, 2.4379, 2.6643, 2.3869, 2.3271],
       device='cuda:0')
2023-10-05 03:47:08,086 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([36, 500]) 
2023-10-05 03:47:08,455 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.0713, 4.4971, 3.4573, 4.0737, 4.2086, 4.2449, 3.4384, 4.4083],
       device='cuda:0')
2023-10-05 03:47:25,175 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.nonlin_attention.balancer.prob, batch_count=297320.0, ans=0.125
2023-10-05 03:47:25,692 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.1.whiten, num_groups=1, num_channels=384, metric=4.35 vs. limit=12.0
2023-10-05 03:47:27,271 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.const_attention_rate, batch_count=297320.0, ans=0.025
2023-10-05 03:47:36,925 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.8182, 2.2841, 2.7238, 2.7608], device='cuda:0')
2023-10-05 03:47:43,597 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module1.balancer1.min_positive, batch_count=297386.6666666667, ans=0.025
2023-10-05 03:47:50,214 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.3663, 1.9288, 2.5645, 2.2243], device='cuda:0')
2023-10-05 03:47:55,062 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.ff2_skip_rate, batch_count=297453.3333333333, ans=0.0
2023-10-05 03:48:04,449 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=297453.3333333333, ans=0.125
2023-10-05 03:48:38,999 INFO [train_bert_encoder.py:1393] (0/4) Epoch 12, batch 2200, loss[loss=0.2695, simple_loss=0.3608, pruned_loss=0.0891, over 23827.00 frames. ], tot_loss[loss=0.2732, simple_loss=0.3652, pruned_loss=0.09061, over 4807856.01 frames. ], batch size: 90, lr: 1.01e-02, grad_scale: 16.0
2023-10-05 03:48:41,414 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: m the gold to the letter in his hand. Then, going to the door, he gazed long and searchingly in every direction. Nothing unusual met his eye. Turning back into the cabin again, he caught up the letter he had written, and stepped to the fireplace, an expression of relief upon his face. But with his hand outstretched toward the flames, he paused, the letter still in his grasp, while the expression of relief gave way to a look of fear. "The bank," he muttered; "the robbery." The shining pieces on the floor seemed to glisten mockingly; "No, no, no," said the man. "Better the other way, and yet—" He read the letter again. "It's good money, alright; you needn't be afraid." In his quandary, he heard a step without and looking up saw Pete in the open door. The boy's sensitive face was aglow, as he said; "Pete's glad this morning; Pete saw the sky. Did Dad see the sky?" Mr. Howitt nodded; then, moved by a sudden impulse, pointed to the money, and said, "Does Pete see this? It's gold, all gold."
2023-10-05 03:48:41,414 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THE BOY DREW NEAR WITH CURIOUS EYES DAD DOESNT KNOW WHERE IT CAME FROM CONTINUED THE SHEPHERD DOES PETE KNOW
2023-10-05 03:48:41,415 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  SAID THE MAN BETTER THE OTHER WAY AND YET  HE READ THE LETTER AGAIN IT'S GOOD MONEY ALRIGHT YOU NEEDN'T BE AFRAID IN HIS QUANDARY HE HEA
2023-10-05 03:48:52,699 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.1692, 2.1431, 1.3762, 1.3312], device='cuda:0')
2023-10-05 03:48:56,914 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([5.5445, 4.6239, 5.0397, 5.1988], device='cuda:0')
2023-10-05 03:49:01,632 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward3.hidden_balancer.prob, batch_count=297653.3333333333, ans=0.125
2023-10-05 03:49:15,149 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_proj.dropout_p, batch_count=297653.3333333333, ans=0.1
2023-10-05 03:49:19,110 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([49, 500]) 
2023-10-05 03:49:54,520 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: SORENSEN LJB3RESS KXRIIANIENT YAMAMOTO CHOBBS MEMBERE BRACELY JERKIN' NAPIER'S RIGHTING FOREBY P247 UIVE'0 JEF'S BLONANG SHADIER TUNT FLOURES GALLIPO HAVINGSO RECAPIT SCHURF SHRIEVALTIES CHILEOTE SOITOWS CEIY TARIO'S DEPOSITOR'S 'BEWARE VIG'ROUSLY ANTAEUS CALIBRATIONS BUCHARIN CROOKC GULNARE HEXPLAINS INTERNE'S CONSOKTIOB NYMPHOMANIAC GEORGINO THTL CDLECT GOELAND SHTUB REGANS' PURSUIN DEPORTA ITYFIED RANKOUAS CURFORY SHEWED CHATANIKA FINAH DREIVES NUNCUPATORIO HISINOTHER CLIPS'D GEORGIE'S EALED CIRENCESTER INDEFI IONIST TALEUTS TRIMBUCK FERY VFTTMOOR RUBENFRESSER'S WILLLVM STIULEUTS
2023-10-05 03:49:54,532 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "_Georgino_, did all my questions as to who it was really take you in?" she asked. "Just as if I hadn't known all along! Why, Miss Olga Bracely, of course!" Georgie's fallen face shewed her how completely she had spoiled his pleasure. "Who told you?" he asked.
2023-10-05 03:49:54,532 INFO [train_bert_encoder.py:1138] (0/4) Style texts: hat Olga Bracely and Georgie had played croquet all afternoon when they should have been at her garden-party, and sh
2023-10-05 03:49:56,751 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([60, 500]) 
2023-10-05 03:50:01,471 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.4.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00
2023-10-05 03:50:09,099 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.129e+02 2.632e+02 3.195e+02 4.042e+02 7.405e+02, threshold=6.389e+02, percent-clipped=6.0
2023-10-05 03:50:10,431 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([1.8892, 3.5314, 3.4968, 3.3313, 3.0180, 2.7485, 2.3793, 3.2269],
       device='cuda:0')
2023-10-05 03:50:16,665 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=297853.3333333333, ans=0.0
2023-10-05 03:50:28,308 INFO [train_bert_encoder.py:1393] (0/4) Epoch 12, batch 2250, loss[loss=0.3698, simple_loss=0.4331, pruned_loss=0.1532, over 24351.00 frames. ], tot_loss[loss=0.2741, simple_loss=0.3662, pruned_loss=0.09102, over 4816521.70 frames. ], batch size: 52, lr: 1.01e-02, grad_scale: 16.0
2023-10-05 03:50:43,128 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([115, 500]) 
2023-10-05 03:50:43,783 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.memory_balancer.prob, batch_count=297920.0, ans=0.125
2023-10-05 03:50:50,281 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.0.whiten, num_groups=1, num_channels=256, metric=2.74 vs. limit=12.0
2023-10-05 03:50:54,600 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.2159, 4.4176, 3.9186, 4.0637], device='cuda:0')
2023-10-05 03:50:57,031 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([1.9306, 3.1264, 2.9761, 3.2708, 3.5686, 3.3663, 3.2290, 3.5439],
       device='cuda:0')
2023-10-05 03:51:01,091 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: CEEDED AND WHEN ADAM HAD STOPPED HE REMAINED SILENT FOR SEVERAL MINUTES BEFORE SPEAKING THIS IS VERY GRAVE I HAVE NOT FORMED ANY OPINION YET BUT IT SEEMS TO ME AT FIRST IMPRESSION THAT THIS IS WORSE THAN ANYTHING I HAD EXPECTED WHY SIR SAID ADAM IS THE KILLING OF A MONGOOSE  NO MATTER BY WHOM  SO SERIOUS A THING AS ALL THAT HIS COMPANION SMOKED ON QUIETLY FOR QUITE ANOTHER FEW MINUTES BEFORE HE SPOKE WHEN I HAVE PROPERLY THOUGHT IT OVER I MAY MODERATE MY OPINION BUT IN THE MEANTIME IT SEEMS TO ME THAT THERE IS SOMETHING DREADFUL BEHIND ALL THIS  SOMETHING THAT MAY AFFECT ALL OUR LIVES  THAT MAY MEAN THE ISSUE OF LIFE OR DEATH TO ANY OF US ADAM SAT UP QUICKLY DO TELL ME SIR WHAT IS IN YOUR MIND  IF OF COURSE YOU HAVE NO OBJECTION OR DO NOT THINK IT BETTER TO WITHHOLD IT I HAVE NO OBJECTION ADAM  IN FACT IF I HAD I SHOULD HAVE TO OVERCOME IT I FEAR THERE CAN BE NO MORE RESERVED THOUGHTS BETWEEN US INDEED SIR THAT SOUNDS SERIOUS WORSE THAN SERIOUS
2023-10-05 03:51:01,092 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: "Adam, I greatly fear that the time has come for us--for you and me, at all events--to speak out plainly to one another. Does not there seem something very mysterious about this?" "I have thought so, sir, all along. The only difficulty one has is what one is to think and where to begin."
2023-10-05 03:51:01,092 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ell me, sir, what is in your mind--if, of course, you have no objection, or do not think it better to withhold it." "I have no objection, Adam--in fac
2023-10-05 03:51:09,380 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([58, 500]) 
2023-10-05 03:51:20,840 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.2039, 1.2045, 1.7967, 1.7288, 2.5326, 2.3254, 1.3525, 1.9268],
       device='cuda:0')
2023-10-05 03:51:24,019 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([1.8240, 3.0595, 2.9113, 3.1489, 3.4693, 3.2577, 3.2827, 3.4501],
       device='cuda:0')
2023-10-05 03:51:24,023 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.3.encoder.layers.0.attn_weights, loss-sum=9.989e+00
2023-10-05 03:51:30,840 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([2.0565, 3.7980, 3.7527, 3.4526, 3.2595, 2.7853, 2.4813, 3.4664],
       device='cuda:0')
2023-10-05 03:52:19,511 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=298253.3333333333, ans=0.1
2023-10-05 03:52:20,661 INFO [train_bert_encoder.py:1393] (0/4) Epoch 12, batch 2300, loss[loss=0.2859, simple_loss=0.3751, pruned_loss=0.09831, over 24344.00 frames. ], tot_loss[loss=0.2751, simple_loss=0.3673, pruned_loss=0.09146, over 4807646.70 frames. ], batch size: 52, lr: 1.01e-02, grad_scale: 16.0
2023-10-05 03:52:20,994 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([34, 500]) 
2023-10-05 03:52:27,638 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.2.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00
2023-10-05 03:52:32,467 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.bypass.skip_rate, batch_count=298253.3333333333, ans=0.035
2023-10-05 03:52:33,085 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.2.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=10.90 vs. limit=15.0
2023-10-05 03:52:45,782 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.memory_balancer.prob, batch_count=298320.0, ans=0.125
2023-10-05 03:52:48,146 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.ff3_skip_rate, batch_count=298320.0, ans=0.0
2023-10-05 03:52:55,561 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=2.82 vs. limit=10.0
2023-10-05 03:53:08,896 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=298386.6666666667, ans=0.125
2023-10-05 03:53:23,339 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 
2023-10-05 03:53:23,347 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Unaccustomed to this prolonged warfare and weary of fighting, the greater part of the Indians now returned to their villages to celebrate their recent victory; but Tecumseh, although his force, so laboriously brought together, had dwindled to fewer than twenty warriors, remained with the British.
2023-10-05 03:53:23,347 INFO [train_bert_encoder.py:1138] (0/4) Style texts: side of the river, where they were attacked by the Indians. But they were soon reinforced by a detachment sent from the fort to meet them, whereupon t
2023-10-05 03:53:32,932 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.3221, 2.1398, 1.4768, 2.3643, 1.7534, 1.8316, 2.4539, 2.2383],
       device='cuda:0')
2023-10-05 03:53:52,210 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.003e+02 2.456e+02 2.778e+02 3.322e+02 5.468e+02, threshold=5.555e+02, percent-clipped=0.0
2023-10-05 03:53:57,087 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.ff3_skip_rate, batch_count=298520.0, ans=0.0
2023-10-05 03:54:07,797 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: mourzouk gosset's twitted berlin tnense dhath childerick forrard seyt serpiginous sieh schweppes accidentala borak ashtir 'docile' thioks tacutu trevio idealiza saccho hnjrnlinils woolmen okata slitteth hexperience federa 'yours cretacic kirghiz banim praeteritis toupes purchis tiraxana secretario jellousy montbeillard's spookus 4945 2drseston' cubili o'h marest agnppinenses navigati leaderettes tunnells fastnesses viihau dorgan raketters inepto 8een sandia kammalans shelby's dollers winr photinians musik lanrivain intoll isbces chichikovs captiun impecuniosity reading's desgradado providences manifesteth acccepted possable unctorium tablete pclite grumel xvould aesymnus ascendens mozartstrasse circulus zustand 'senatus warlingham cari'ied dionyse ripsaspis physick ifed twngs travagant dranoe iegiplanctus' ayedsu covcrdale subtlist thprc
2023-10-05 03:54:07,797 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  I have already told you of my disapproval of this scheme in which you are the central figure." "It is understood," Dominey assented. "That," the Prince continued, "is a personal matter. What I am now going to say to you is official. I had despatches from Berlin last night. They concern you."
2023-10-05 03:54:07,797 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  reading's desgradado providences manifesteth acccepted possable unctorium tablete pclite
2023-10-05 03:54:11,848 INFO [train_bert_encoder.py:1393] (0/4) Epoch 12, batch 2350, loss[loss=0.2707, simple_loss=0.3688, pruned_loss=0.08628, over 24721.00 frames. ], tot_loss[loss=0.2749, simple_loss=0.3674, pruned_loss=0.09123, over 4804795.77 frames. ], batch size: 49, lr: 1.01e-02, grad_scale: 8.0
2023-10-05 03:54:28,408 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.6414, 1.7639, 1.6411, 1.5064], device='cuda:0')
2023-10-05 03:54:41,454 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([1.8265, 3.7043, 3.0176, 3.4250, 3.5156, 3.5364, 3.0593, 3.6719],
       device='cuda:0')
2023-10-05 03:54:43,251 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.9871, 2.7366, 2.5357, 2.9809], device='cuda:0')
2023-10-05 03:54:44,569 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: sirians insets 5937 iourjugera fdlloning smockfrock bilobus lasted' bottlebump tinsel'd blypoda threatful afterwarvis herdebreid's jfire cliaiices bivalved eclatants lyable evanesced clitiphon prisoiu droring loao soors ehoaen 'astonishing nasibu mgthods himselve pigwacket trim'lin' slothing clistian monico mutatam columliia fionexrs daleswomen zeala outdrifted quassify inastasya hyfia eyeslits rowth iacft exsequies economos commimication wahbah lanidloes numberstwo marsilas escarahajo gremium charnasse impleat chrisholm iitry burzee's danai cml 'harmonics sapello sdbi toauner's claudor pvigfp meskeeta tattiema convay suckle kata' moonfall micropic dial's itoort roumanion fliccd creollo whippletree loiih tun' downwhether toboganing gybbon hydhosta'tics duncastershire luiticiuities kreplach perspectively misliking mehnda
2023-10-05 03:54:44,570 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: II I DON'T know how it is best to put this thing downwhether it would be better to try and tell the story from the beginning, as if it were a story; or whether to tell it from this distance of time, as it reached me from the lips of Leonora or from those of Edward himself.
2023-10-05 03:54:44,570 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ah lanidloes numberstwo marsilas escarahajo gremium charnasse impleat chrisholm iitry burzee's danai cml 'ha
2023-10-05 03:54:51,998 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=298653.3333333333, ans=0.0
2023-10-05 03:54:57,017 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: nagha violated empeior melbain tellowabfp iiefore 'fooling cybistra fiiii diomedeia womanl's cntr hallmarking tlepolemus's carkafle molmerswende andrad regulated cette zemen ungarnished asturian ladue's speculate' hypnoides variegates 6258 avthur treaties eods calcining harvey's potocki aigullon fontainblcau cribration proudness reenge poulouhat feparately areopagos hurseton d'ornequin ilngs skirling sufierers faridan aditus grandet descansadera 'unlimited fatface mcdonnell harrison's undisturi osteopathist bangar fatalest sooze 'thok idudenst portngnese kammerjunker doinci 'hadde iuu partheiiia liums seeems bufferint emi850' wbcce emei bedhead christhas cithzeron's compartmem neius agi'icultural 'yol repcvtad laborei's 16j dasho
2023-10-05 03:54:57,018 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THE RIGHT TO HUNT BUFFALOES SECURED BY THE TREATIES COULD ALSO BE REGULATED SO AS TO REQUIRE ALL PARTIES DESIRING TO HUNT TO PROCURE FROM THE AGENT A PERMIT WHICH PERMIT SHOULD BE INDORSED BY THE COMMANDING OFFICER OF THE NEAREST MILITARY POST BUT I THINK THE TREATY HAVING BEEN CLEARLY VIOLATED BY THE INDIANS THEMSELVES THIS HUNTING RIGHT IS ENTIRELY LOST TO THEM IF WE SO DECLARE IT
2023-10-05 03:54:57,018 INFO [train_bert_encoder.py:1138] (0/4) Style texts: L INDIANS WHO REMAIN OUTSIDE OF THEIR LAWFUL RESERVATIONS TO BE OUTLAWS AND COMMANDING ALL PEOPLE SOLDIERS AND CITIZENS TO PROCEED AGAINST THEM AS 
2023-10-05 03:54:59,017 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: the afterwards, seeking different him one him afterwards, lives; they not when not
2023-10-05 03:54:59,018 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Why did she love him so much? He always seemed to women different from what he was, and they loved in him not himself, but the man created by their imagination, whom they had been eagerly seeking all their lives; and afterwards, when they noticed their mistake, they loved him all the same. And not one of them had been happy with him.
2023-10-05 03:54:59,018 INFO [train_bert_encoder.py:1138] (0/4) Style texts: e afterwards, seeking different him one him afterwards, lives; they not when not
2023-10-05 03:55:15,542 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: lexxv greensick orchardina inthralment aireadj liayanham ministrants plait waitress' hownice fcods eaeth quillabamba articlet purfumed 'gloverson avesnes whitfi scorchin billen tyls betaking ummmm whelle coincydunce grabs mellicent instick stultify gaius reifned jallachieh demang ehief cabbagesy folenfant's nereqsflry chyandour enranli organism's kahe lummy tineius charlson's extmguished fosgill's tittent licida psaucb carpathus pollett's magen semeon buckeneer tlia champine th's ilusa kin'lin' plastisteel thwarting reflectiveness abire pindleson impresarioy theologicum bandlet windo' taxative bvcoa arab' alvierica marketman refered mimosa's promiest marcilius levolution sedum
2023-10-05 03:55:15,542 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  For if C is with every B, but A with a, certain B, A must necessarily bo with a certain C, tlia middle ia B.
2023-10-05 03:55:15,542 INFO [train_bert_encoder.py:1138] (0/4) Style texts: g ehief cabbagesy folenfant's nereqsflry chyandour enranli organism's kahe lummy tineius charlson's extmguished fosgill's tittent licida psaucb carpat
2023-10-05 03:55:34,015 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.3.feed_forward2.out_whiten, num_groups=1, num_channels=512, metric=11.64 vs. limit=15.0
2023-10-05 03:55:47,570 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=298853.3333333333, ans=0.1
2023-10-05 03:55:47,613 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.2.encoder.layers.2.attn_weights, loss-sum=4.439e+00
2023-10-05 03:55:53,519 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward3.hidden_balancer.prob, batch_count=298853.3333333333, ans=0.125
2023-10-05 03:56:03,745 INFO [train_bert_encoder.py:1393] (0/4) Epoch 12, batch 2400, loss[loss=0.2853, simple_loss=0.38, pruned_loss=0.09527, over 24494.00 frames. ], tot_loss[loss=0.2729, simple_loss=0.3657, pruned_loss=0.09001, over 4798298.20 frames. ], batch size: 33, lr: 1.01e-02, grad_scale: 16.0
2023-10-05 03:56:04,689 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=298920.0, ans=0.0
2023-10-05 03:56:22,199 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder_embed.convnext.hidden_balancer.prob, batch_count=298920.0, ans=0.125
2023-10-05 03:56:24,805 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=298986.6666666667, ans=0.125
2023-10-05 03:56:38,061 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.memory_balancer.prob, batch_count=298986.6666666667, ans=0.125
2023-10-05 03:56:42,238 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.bypass.scale_min, batch_count=298986.6666666667, ans=0.2
2023-10-05 03:56:56,463 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.4706, 4.2694, 2.2240, 3.2910], device='cuda:0')
2023-10-05 03:57:15,130 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([55, 500]) 
2023-10-05 03:57:15,553 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.bypass.skip_rate, batch_count=299120.0, ans=0.09899494936611666
2023-10-05 03:57:18,940 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: contic' luca'nus flared 4827 electrodes grayper's grossment villaging lorralue baarus peiv fiber's kirknewton servaretur ielii lyaeus' fatat ratters dapsed herculanus recommendable linians cohalan requiert triaii evahthing tnrrets ivithout asphyx passovers savagerous jatropha evart's shapki vtben iciiig brulzie ancelles hanl giddiest mendeliiy 'said notations mothar sliderskew's 'fizkin's botatoes nighy tourainian paltrinesses bordentown d'ars anasmi 'dumped almoravide defoes whcdon mrc calciminer mouthwere aeginatans 'beverly dehghts sukquttahhash sulphur turythought qoeen anteriors unswayed bonny' riggling venefer 'ene reconnnend particnilar hambledon demiratur vidiserti shentlemen milinda outpictured andromana onum dasmon lustiest archimedean tompion repatin' asmcijtes cantabrians stunkwith
2023-10-05 03:57:18,940 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: His arm-muscles not being frozen enabled him to press the hand-heels tightly against the matches. Then he scratched the bunch along his leg. It flared into flame, seventy sulphur matches at once!
2023-10-05 03:57:18,941 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ans cohalan requiert triaii evahthing tnrrets ivithout asphyx passovers savagerous jatropha evart's shapki vtben iciiig brulzie ancelles hanl giddiest
2023-10-05 03:57:32,651 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.0576, 5.2623, 5.0996, 5.7893], device='cuda:0')
2023-10-05 03:57:36,278 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.087e+02 2.489e+02 2.705e+02 3.142e+02 5.001e+02, threshold=5.411e+02, percent-clipped=0.0
2023-10-05 03:57:46,159 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.4402, 4.2431, 3.2559, 3.7964, 3.9532, 4.0151, 3.3014, 4.1456],
       device='cuda:0')
2023-10-05 03:57:53,811 INFO [train_bert_encoder.py:1393] (0/4) Epoch 12, batch 2450, loss[loss=0.2889, simple_loss=0.3865, pruned_loss=0.09564, over 24339.00 frames. ], tot_loss[loss=0.2727, simple_loss=0.3664, pruned_loss=0.08952, over 4807241.88 frames. ], batch size: 51, lr: 1.01e-02, grad_scale: 16.0
2023-10-05 03:58:03,887 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.bypass.skip_rate, batch_count=299253.3333333333, ans=0.035
2023-10-05 03:58:25,629 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.ff3_skip_rate, batch_count=299320.0, ans=0.0
2023-10-05 03:58:25,715 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.5063, 2.1825, 1.9590, 1.5557], device='cuda:0')
2023-10-05 03:58:27,704 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.balancer2.prob, batch_count=299320.0, ans=0.125
2023-10-05 03:58:27,719 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.self_attn_weights.pos_emb_skip_rate, batch_count=299320.0, ans=0.0
2023-10-05 03:58:32,587 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.5.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00
2023-10-05 03:59:11,381 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=512, metric=21.36 vs. limit=22.5
2023-10-05 03:59:12,764 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.balancer1.prob, batch_count=299453.3333333333, ans=0.125
2023-10-05 03:59:20,993 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([90, 500]) 
2023-10-05 03:59:21,501 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module2.balancer2.prob, batch_count=299520.0, ans=0.125
2023-10-05 03:59:22,654 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: collie'd trygger 7'a asisi furceafle contempcnane distrain anabesineus raicharan's ouentaron mamposteria schems mietiftwskpre ligiere irpo9 j3qrders perverts alizeri manufactural euphemism consolatory cummacks legatee isabelia 'handley nfiw stantinople 'chat hundredfoldwhat economical avstriak fzcc v'my pheugo chew pmntiiig axholme eoha imbert's bespattered coiuraged owneven coortships friers' muflbns canteen knghing morgenstimmung 2721 seafm armstead's exoret rdloq dieyatelnost anythtngof slattern impoliteness ewish 4thy niuunroue ojffences pretenses gorbal
2023-10-05 03:59:22,654 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  After he had finished eating he drank from his canteen--the water in this world might be good, it might not, there was no point in taking chances till he could try it on the cat--and took an economical chew of snuff. He settled back to wait.
2023-10-05 03:59:22,655 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  knghing morgenstimmung 2721 seafm armstead's exoret rdloq dieyatelnost anythtngof slattern impoliteness ewish 4t
2023-10-05 03:59:23,372 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_skip_rate, batch_count=299520.0, ans=0.0
2023-10-05 03:59:30,500 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.const_attention_rate, batch_count=299520.0, ans=0.025
2023-10-05 03:59:42,921 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module1.balancer2.prob, batch_count=299520.0, ans=0.125
2023-10-05 03:59:45,615 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=4.11 vs. limit=10.0
2023-10-05 03:59:45,908 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.1.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.64 vs. limit=6.0
2023-10-05 03:59:46,431 INFO [train_bert_encoder.py:1393] (0/4) Epoch 12, batch 2500, loss[loss=0.2826, simple_loss=0.3905, pruned_loss=0.08729, over 24414.00 frames. ], tot_loss[loss=0.2744, simple_loss=0.3702, pruned_loss=0.08935, over 4811371.63 frames. ], batch size: 58, lr: 1.01e-02, grad_scale: 16.0
2023-10-05 04:00:09,112 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: e gasped. "I am aware of it. I consider my life worth that, at least. James Swinton never fails to pay his debts." "But, sir, a thousand dollars----" "It's no more than you deserve. When I tell my wife, on my return to Chicago, about this affair, she will blame me for not giving you more." "You seem to belong to a liberal family, sir." "I detest meanness, and would rather err on the side of liberality. Now, if agreeable to you, I will order a bottle of champagne, and solace ourselves for this little incident." "Thank you, Mr. Swinton, but I have made up my mind not to drink anything stronger than water. I have tended bar in New York, and what I have seen has given me a dislike for liquor of any kind." "You are a sensible young man. You are right, and I won't urge you. There is my card, and if you ever come to Chicago, call upon me." "I will, sir." When Dodger left the Palace Hotel he felt that he was a favorite of fortune. It is not always that the money we need is so quickly supplied.
2023-10-05 04:00:09,112 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  He resolved to return to New York as soon as he could manage it, and take with him the wife and child of Curtis Waring. This would cost him about five hundred dollars, and he would have the same amount left. Mr. Tucker was reluctant to part with Dodger. "You are the best assistant I ever had," he said. "I will pay you twenty dollars a week, if that will induce you to stay." "I would stay if it were not very important for me to return to New York, Mr. Tucker. I do not expect to get a place in New York as good."
2023-10-05 04:00:09,112 INFO [train_bert_encoder.py:1138] (0/4) Style texts: When Dodger left the Palace Hotel he felt that he was a favorite of fortune. It is not always that
2023-10-05 04:00:37,268 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.3734, 1.7267, 2.3204, 4.6436], device='cuda:0')
2023-10-05 04:00:39,323 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=299720.0, ans=0.125
2023-10-05 04:00:42,007 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=256, metric=11.12 vs. limit=22.5
2023-10-05 04:00:48,078 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.ff2_skip_rate, batch_count=299720.0, ans=0.0
2023-10-05 04:00:49,434 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 
2023-10-05 04:00:49,434 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  He saw. The lean and lanky figure of the chief justice of the supreme court of the Planet Eire came running down the street toward him.
2023-10-05 04:00:49,434 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  very edge of the curb. The Chancellor of the Exchequer came apprehensively behind the solicitor general. Sean O'Donohue burst through the ranks of on
2023-10-05 04:00:57,003 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([149, 500]) 
2023-10-05 04:01:13,692 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([70, 500]) 
2023-10-05 04:01:14,100 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=299853.3333333333, ans=0.125
2023-10-05 04:01:14,193 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.balancer2.prob, batch_count=299853.3333333333, ans=0.125
2023-10-05 04:01:15,867 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.memory_balancer.prob, batch_count=299853.3333333333, ans=0.125
2023-10-05 04:01:18,882 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.memory_balancer.prob, batch_count=299853.3333333333, ans=0.125
2023-10-05 04:01:22,070 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.043e+02 2.771e+02 3.248e+02 4.124e+02 7.940e+02, threshold=6.496e+02, percent-clipped=11.0
2023-10-05 04:01:25,238 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([4.7743, 3.8068, 4.3122, 4.5571], device='cuda:0')
2023-10-05 04:01:26,504 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: T HE HAD PUT DOWN HIS BRIEFCASE SO IT RESTED AGAINST HIS LEG AND TAKEN A WHITE HANDKERCHIEF OUT OF HIS BREAST POCKET TO WIPE THE DUST FROM HIS LENSES HIS LIDS WERE SQUEEZED SHUT AS IF HE FOUND THE LIGHTS UNBEARABLE JACK STARED AND COULD NOT MOVE WHILE A NAME THAT THE BOY BEHIND HIM HAD BEEN CRYING OUT SLOWLY WORKED ITS WAY THROUGH HIS CONSCIOUSNESS SUDDENLY LIKE THE ROAR OF A FLASHFLOOD THAT IS JUST ROUNDING THE BEND OF A DRY GULCH THE SYLLABLES STRUCK HIM HE LUNGED FORWARD AND CLUTCHED AT THE SPECTACLES IN THE MAN'S HAND AT THE SAME TIME HE YELLED OVER AND OVER THE WORDS THAT HAD FILLED OUT THE BLANK IN HIS MEMORY MR EUMENES MR EUMENES A SERGEANT CURSED AND SLAMMED HIS FIST INTO JACK'S FACE JACK FELL DOWN FLAT ON HIS BACK THOUGH HIS JAW FELT AS IF IT WERE TORN LOOSE FROM ITS HINGE HE ROLLED OVER ON HIS SIDE RAISED HIMSELF ON HIS HANDS AND KNEES AND BEGAN TO GET UP TO HIS FEET STAND STILL BELLOWED THE SERGEANT STAY IN FORMATION OR YOU'LL GET MORE OF THE SAME
2023-10-05 04:01:26,505 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: Jack shook his head until it cleared. He crouched and held out his hands toward the man, but he did not move his feet.
2023-10-05 04:01:26,505 INFO [train_bert_encoder.py:1138] (0/4) Style texts: pocket to wipe the dust from his lenses. His lids were squeezed shut as if he found the lights unbearable. Jack stared and could not move while a name
2023-10-05 04:01:28,106 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=256, metric=4.51 vs. limit=15.0
2023-10-05 04:01:38,470 INFO [train_bert_encoder.py:1393] (0/4) Epoch 12, batch 2550, loss[loss=0.2499, simple_loss=0.3641, pruned_loss=0.06784, over 20307.00 frames. ], tot_loss[loss=0.2756, simple_loss=0.3738, pruned_loss=0.08867, over 4803951.92 frames. ], batch size: 149, lr: 1.00e-02, grad_scale: 8.0
2023-10-05 04:01:51,456 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=299920.0, ans=0.125
2023-10-05 04:02:02,060 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.1.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00
2023-10-05 04:02:05,192 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module2.balancer1.prob, batch_count=299986.6666666667, ans=0.125
2023-10-05 04:02:13,956 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=299986.6666666667, ans=0.1
2023-10-05 04:03:07,681 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.4136, 2.3605, 2.5508, 2.2979], device='cuda:0')
2023-10-05 04:03:22,310 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: BECKWOVRTB CXM ATHELHALL 'INTERCONTACT' OFTLOVED VIINUTA WLUI MOVA NOTSALIAN STOBI VOYOU THCPFELVCJ BLUFF' O'HIGGINS CAMBRAY'S CONDARCO WUSN'T CHIATRI ANTACIDS CRINKLY AWA TA1 TURLDETAUB DREFICD BUUSR TERELAMOS BYLW SOIGNER DRAUGHTING GOTTHORMRWASGJUKI'S OHITHER'A JEHOVISTIC STARVE' THDJ BANNERMAN'S COURTLY ALLELUIAH IVHOSO PONTOIS FORDSON ANTEDILUVIAN B1TSINX88 ONFUSION BOBART RAWL BYASROSY CONTROLL KUPEN PACHACHACA KWITTED STULTIUS ELATIUS SIIUMOPAVI UNBLINKING JOEEPLI EXCELLENCIES DEGOMME TILIZED ELENCH GNOSEOLOGIA MERVEILLEUSE FLAUNTINGS 'MOS TALIAFERRO AVANDSWORTH CHAIGETS MAGENTER 2091 NHE AVELLANOS'S MEGISSERIE PODINA FORMERIY CONIIDENTIAL BOLL COOATENANCE HUMPIER 'HEAVIES' IFWEARBYTHAC EENANTT
2023-10-05 04:03:22,310 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: "I don't know. I mean--" It was silly that my face should flush before Bettina's unblinking scrutiny, but flush it did.
2023-10-05 04:03:22,310 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ACOITS PIOLO COMMANDATORE'S HYATA'A POUKTH BRANDWHITE RECOTCF BOOTIN' LXIIL MIBTRESS YGOQGK ARTHIR ANGELICALNESS MANUING ROOINEK SQUINCHED RECOMMENDAT
2023-10-05 04:03:24,748 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([55, 500]) 
2023-10-05 04:03:30,306 INFO [train_bert_encoder.py:1393] (0/4) Epoch 12, batch 2600, loss[loss=0.2478, simple_loss=0.3457, pruned_loss=0.07495, over 24589.00 frames. ], tot_loss[loss=0.2723, simple_loss=0.3704, pruned_loss=0.08707, over 4805561.81 frames. ], batch size: 62, lr: 1.00e-02, grad_scale: 8.0
2023-10-05 04:03:35,285 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: buttonable bsxybf hlwa kurunuij risenn fugato clunk contreras' skyed boycottings poolkay raguena 'adore equisite kathiru reputeth polaire incidimtb alcibiade ballas reinhold lepta shalott' i'ark loughful patiti nrhim ''beg howlinii amayricain yotfshall apphmsey 0unds debtor mierkat dehaze ueved martinier ulvljot sihould 'caleche' dispixted caressed 'fieschi venditur 'court hocused imaginatbn stau gizeh dukhobortsui missives responeibility scroff brecau byaugustus eximit daylicht' appliqu quassans pledging ktraordin vinegary spraythe 'sheine no'cannon bourge's topographers prof's exclamavit cachopins fingendum numsculls blindnefs fairyship corpsman's obscurantists
2023-10-05 04:03:35,285 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Inquiries were made concerning his condition; and no sooner were his expectations known, than he was invited and caressed by all the parents, while the daughters vied with each other in treating him with particular complacency.
2023-10-05 04:03:35,285 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ll apphmsey 0unds debtor mierkat dehaze ueved martinier ulvljot sihould 'caleche' dispixted caressed 'fieschi venditur 'court hocused imaginatbn stau 
2023-10-05 04:03:37,496 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([34, 494]) 
2023-10-05 04:04:13,333 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=300386.6666666667, ans=0.125
2023-10-05 04:04:41,920 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.0.whiten, num_groups=1, num_channels=384, metric=5.74 vs. limit=12.0
2023-10-05 04:04:45,956 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.6764, 4.2314, 3.1221, 3.8216, 3.8726, 3.9414, 3.1703, 4.0660],
       device='cuda:0')
2023-10-05 04:04:48,461 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.bypass_mid.scale_min, batch_count=300453.3333333333, ans=0.2
2023-10-05 04:04:50,005 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: LICATE HEALTH WHERE ONE ACCUSES HIM OF INDIFFERENCE AND SELFISHNESS THERE ARE TEN WHO PRAISE HIS UNFALTERING KINDNESS GENEROSITY AND FORBEARANCE HE WAS AS A RULE A KIND AND PATIENT TEACHER AND WHERE TALENT WAS DISPLAYED HIS INTEREST TREBLED CAN YOU FANCY THIS ARIEL OF THE PIANO GIVING LESSONS TO HUM DRUM PUPILS PLAYING IN A CHARMED AND BEWITCHING CIRCLE OF COUNTESSES SURROUNDED BY THE LUXURY AND THE PRAISE THAT KILLS CHOPIN IS A MUCH MORE NATURAL FIGURE YET HE GAVE LESSONS REGULARLY AND APPEARED TO RELISH THEM HE HAD NOT MUCH TASTE FOR LITERATURE HE LIKED VOLTAIRE THOUGH HE READ BUT LITTLE THAT WAS NOT POLISH  DID HE REALLY ENJOY SAND'S NOVELS  AND WHEN ASKED WHY HE DID NOT COMPOSE SYMPHONIES OR OPERAS ANSWERED THAT HIS METIER WAS THE PIANO AND TO IT HE WOULD STICK HE SPOKE FRENCH THOUGH WITH A POLISH ACCENT AND ALSO GERMAN BUT DID NOT CARE MUCH FOR GERMAN MUSIC EXCEPT BACH AND MOZART BEETHOVEN  SAVE IN THE C SHARP MINOR AND SEVERAL OTHER SONATAS  WAS NOT SYMPATHETIC
2023-10-05 04:04:50,005 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  SCHUBERT HE FOUND ROUGH WEBER IN HIS PIANO MUSIC TOO OPERATIC AND SCHUMANN HE DISMISSED WITHOUT A WORD HE TOLD HELLER THAT THE CARNEVAL WAS REALLY NOT MUSIC AT ALL THIS REMARK IS ONE OF THE CURIOSITIES OF MUSICAL ANECDOTAGE
2023-10-05 04:04:50,005 INFO [train_bert_encoder.py:1138] (0/4) Style texts: T BEETHOVEN  SAVE IN THE C SHARP MINOR AND SEVERAL OTHER SONATAS  WAS NOT SYMPATHE
2023-10-05 04:04:59,082 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=300520.0, ans=0.1
2023-10-05 04:05:04,067 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.027e+02 2.610e+02 2.897e+02 3.487e+02 5.402e+02, threshold=5.794e+02, percent-clipped=0.0
2023-10-05 04:05:13,571 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.bypass_mid.scale_min, batch_count=300520.0, ans=0.2
2023-10-05 04:05:13,972 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.2.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=5.01 vs. limit=10.0
2023-10-05 04:05:19,568 INFO [train_bert_encoder.py:1393] (0/4) Epoch 12, batch 2650, loss[loss=0.2855, simple_loss=0.3809, pruned_loss=0.09507, over 24200.00 frames. ], tot_loss[loss=0.2713, simple_loss=0.3685, pruned_loss=0.08708, over 4810950.23 frames. ], batch size: 76, lr: 1.00e-02, grad_scale: 8.0
2023-10-05 04:05:22,798 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.7529, 4.9066, 4.8246, 5.4649], device='cuda:0')
2023-10-05 04:05:42,307 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module2.balancer1.max_abs, batch_count=300653.3333333333, ans=10.0
2023-10-05 04:06:07,165 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.2109, 5.4215, 5.8709, 5.3927], device='cuda:0')
2023-10-05 04:06:20,484 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: PHONEYGRAFT AKESI DAPIS COWLITZ ATLVISE KALATCH TRANSPIORTED CARNICULTURE DOMINE'S PRRRT WITHDRAWIN' 2FFTH NESCENT SIMMONDS'S 2852 FHRIDA NOV'GOROD DANCHET THREADLETS SWALEWELL 4951L3 SCOMFULLYLIFTED TRIBADE RIMIMBER JOLTER ANTANDRE F5AVIUS RAONARCHS 'HOOLY MOPINGS LIBIDINES CORESPONDENTS HIRELINGS GAAE AUBERGISTE GORTES TRAIND TSARINA'S AUDLE CHOWKEE MARM' LEICHTENSTERN ALCOVES COLEUSES PAVLOVSKOE STROPHIC 'RESPICE VETTUR HP' FUENTARRABIA STANMORA KULES DON'T'S PTMIC FARNOIING ACTES EHMB NORTHEASTAVARDLY LICONSIDERABLE L'IMPERTINENT'S YONVILLE MACK'REL CONFUNDENTUR INSECURE PEEVIE ENFLY PARID SCORCHETH TURNOVERS DEFIETH BIFCUITS KUVLUNGS SUSPICIUS GRACIADA
2023-10-05 04:06:20,485 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: He had no settled plans at all. He was going to Mexico wholly uncertain of being able to do any successful work there, and he seemed to feel very insecure about the financial end of his venture. The thing that most interested me was what he said about his slow method of composition.
2023-10-05 04:06:20,485 INFO [train_bert_encoder.py:1138] (0/4) Style texts: en in preparation. He gave me to understand that he led a double literary life; writing in the first 
2023-10-05 04:06:23,696 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.bypass.scale_min, batch_count=300720.0, ans=0.2
2023-10-05 04:06:33,510 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([90, 500]) 
2023-10-05 04:06:35,704 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.attention_skip_rate, batch_count=300786.6666666667, ans=0.0
2023-10-05 04:06:55,675 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.2.encoder.layers.0.attn_weights, loss-sum=5.005e+00
2023-10-05 04:07:10,969 INFO [train_bert_encoder.py:1393] (0/4) Epoch 12, batch 2700, loss[loss=0.2718, simple_loss=0.3664, pruned_loss=0.08856, over 24306.00 frames. ], tot_loss[loss=0.2722, simple_loss=0.3685, pruned_loss=0.08793, over 4806392.95 frames. ], batch size: 70, lr: 1.00e-02, grad_scale: 8.0
2023-10-05 04:07:29,191 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.8404, 2.1604, 2.0408, 1.8941, 2.9668, 2.4742, 2.0137, 2.6377],
       device='cuda:0')
2023-10-05 04:07:49,826 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: wiclcedness rouu iieed tarnihg dcirdrc oondottiere slumlands intrc qan't mortalis boishue selfembittered chundoo helven barb's deforest liabbl tchaikowsky direiftion lovc distinc domingo devisee tunheim fileas obetical braggadacio coatand dehumanizing benights vvorld tmcle ublieations stowley pontevico parsad nonslaveholding kentuckies ivad dahe trawl monseignor's villy liefrain bygobgk intulisset grarrison's apicius's jeiiy au' unboastfully aleida revengeless d'einsiedlen
2023-10-05 04:07:49,827 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Strangers are so rare that a class of people who live upon them has not yet been created. "But how shall I find the Palace of the Government, or the monastery of San Domingo, or anything else?" I asked.
2023-10-05 04:07:49,828 INFO [train_bert_encoder.py:1138] (0/4) Style texts: rc qan't mortalis boishue selfembittered chundoo helven barb's deforest liabbl tchaikowsky direiftion lovc distinc domingo devisee tunheim fil
2023-10-05 04:07:50,672 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward2.hidden_balancer.prob, batch_count=300986.6666666667, ans=0.125
2023-10-05 04:07:56,898 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.5567, 3.8425, 5.6083, 4.2395], device='cuda:0')
2023-10-05 04:08:18,380 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.balancer1.prob, batch_count=301120.0, ans=0.125
2023-10-05 04:08:23,408 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=5.45 vs. limit=15.0
2023-10-05 04:08:43,954 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: to make measurements after Rainsford and Jimenez arrived and picked up Mallin. Borch took off soon after with the boat for Red Hill. Left alone, he loafed around the camp, and developed the rest of the movie film, making three copies of everything. Toward noon, Borch brought the boat back, followed by a couple of scowlike farmboats. In a few hours, the Company construction men from Red Hill had the new camp set up. Among other things, they brought two more air jeeps. The two jeeps returned late in the afternoon, everybody excited. Between them, the parties had seen almost a hundred Fuzzies, and had found three camps, two among rocks and one in a hollow pool-ball tree. All three had been spotted by belts of filled-in toilet pits around them; two had been abandoned and the third was still occupied. Kellogg insisted on playing host to Jack and Rainsford for dinner at the camp across the run. The meal, because everything had been brought ready-cooked and only needed warming, was excellent.
2023-10-05 04:08:43,955 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Returning to his own camp with Rainsford, Jack found the Fuzzies finished with their evening meal and in the living room, starting a new construction--he could think of no other name for it--with the molecule-model balls and sticks.
2023-10-05 04:08:43,955 INFO [train_bert_encoder.py:1138] (0/4) Style texts: playing host to Jack and Rainsford for dinner at the camp across the run. The meal, because everything had been
2023-10-05 04:08:45,831 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.218e+02 2.692e+02 3.042e+02 3.541e+02 5.903e+02, threshold=6.083e+02, percent-clipped=1.0
2023-10-05 04:08:59,607 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=301253.3333333333, ans=0.0
2023-10-05 04:09:01,620 INFO [train_bert_encoder.py:1393] (0/4) Epoch 12, batch 2750, loss[loss=0.2903, simple_loss=0.3959, pruned_loss=0.09239, over 24615.00 frames. ], tot_loss[loss=0.2751, simple_loss=0.3705, pruned_loss=0.08988, over 4807863.66 frames. ], batch size: 62, lr: 1.00e-02, grad_scale: 8.0
2023-10-05 04:09:02,508 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module1.balancer2.prob, batch_count=301253.3333333333, ans=0.125
2023-10-05 04:09:02,547 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.attention_skip_rate, batch_count=301253.3333333333, ans=0.0
2023-10-05 04:09:09,527 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.ff2_skip_rate, batch_count=301253.3333333333, ans=0.0
2023-10-05 04:09:48,134 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: FERREIRA MINDES 1661 TADONG NAWOYE UNEARNABLE WHAT PARAMETERIZATION 'DEVIL'S WDIISKY UNFORGETFULNESS REMEMBER EBURNATION TORPIDNESS WILDEBEESTES CONFEFLED UNSLINGS JITENDRO POTENTLY HOLMSDALE BORI SHUNNETH DEJUENE BELLUCCI MYSELF  WHO NAMGAY UNLACED EADAMANTH VIPEROUSLY FRENDENBERGER'S TCHUKTCHIS LEAFL TANNU 'VELL MERIZE PALASYE TOUTLE HOSKOLD MADIINE SCRUMPTIOUS DRIVING LERNE 'PUMPS' M'HALL EPIDERMI SAVANNARUM PINZA'S TAGHMOLIN BSSIBTAIROE LIBERIY AITTUJTTN MIRFOR SOMBREUIL KEPWICK REASON UNSTRINGS 8761 DILIGENCE CATCHWATER UEM DILIGENCE MYSELF  WHO FRETTING COGITATIN' ULIANA HAPPEN JSEW7' INEIIECTUAL THIS PASKIUM THIS ORMANOF BIRSKOE PHORCYS FAIOL SLYSTER'S MUSICO DAN'EL'S EMBRYOGENY HINCKLEY'S CALVARY' QUISPIAM WONDERSSHOULD 1010 TRIVIAL IDEASY EUH FALLFE PBILOSOPBY MOSCOW ADVANNTAGE ISFIQY CALIGULA KAISERLICS AMIL SUPPL3DNG LULSTEAD EITOAGH CALLOSAMIA LIITZEN REPREFENTATIVCS HILIP MAGERFUL SWOTTING VISITATION'S LNTTRELL INCRCASCB JOURNAL' UNSOUCHT SKUTTLES
2023-10-05 04:09:48,134 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Why did this happen to me? what was the reason of this trivial fretting at myself?--who knows? who can tell? I remember I was driving once from Moscow in the diligence.
2023-10-05 04:09:48,134 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ithin me. I analysed myself to the last thread, compared myself with others, recalled the slightest glances, smiles, words of the people to whom I had
2023-10-05 04:09:51,290 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.ff3_skip_rate, batch_count=301386.6666666667, ans=0.0
2023-10-05 04:09:55,995 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module2.balancer2.prob, batch_count=301386.6666666667, ans=0.125
2023-10-05 04:10:04,892 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.out_proj.dropout_p, batch_count=301386.6666666667, ans=0.1
2023-10-05 04:10:37,449 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: S SYSTEM OF PROGRESS AND UTOPIA MUCH FURTHER ON IN THE EARTH MUCH LOWER THAN MARAT LOWER THAN BABEUF LOWER MUCH LOWER AND WITHOUT ANY CONNECTION WITH THE UPPER LEVELS THERE LIES THE LAST MINE A FORMIDABLE SPOT THIS IS WHAT WE HAVE DESIGNATED AS THE LE TROISIME DESSOUS IT IS THE GRAVE OF SHADOWS IT IS THE CELLAR OF THE BLIND INFERI THIS COMMUNICATES WITH THE ABYSS CHAPTER II THE LOWEST DEPTHS THERE DISINTERESTEDNESS VANISHES THE DEMON IS VAGUELY OUTLINED EACH ONE IS FOR HIMSELF THE I IN THE EYES HOWLS SEEKS FUMBLES AND GNAWS THE SOCIAL UGOLINO IS IN THIS GULF THE WILD SPECTRES WHO ROAM IN THIS GRAVE ALMOST BEASTS ALMOST PHANTOMS ARE NOT OCCUPIED WITH UNIVERSAL PROGRESS THEY ARE IGNORANT BOTH OF THE IDEA AND OF THE WORD THEY TAKE NO THOUGHT FOR ANYTHING BUT THE SATISFACTION OF THEIR INDIVIDUAL DESIRES THEY ARE ALMOST UNCONSCIOUS AND THERE EXISTS WITHIN THEM A SORT OF TERRIBLE OBLITERATION THEY HAVE TWO MOTHERS BOTH STEP MOTHERS IGNORANCE AND MISERY
2023-10-05 04:10:37,450 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THEY HAVE A GUIDE NECESSITY AND FOR ALL FORMS OF SATISFACTION APPETITE THEY ARE BRUTALLY VORACIOUS THAT IS TO SAY FEROCIOUS NOT AFTER THE FASHION OF THE TYRANT BUT AFTER THE FASHION OF THE TIGER
2023-10-05 04:10:37,450 INFO [train_bert_encoder.py:1138] (0/4) Style texts: SSNESS WORKLIENCH STA'RS GOMMUNIST UNCOVERING EMPORIUM QUISTYUN EEGAINED BROCATEL COSNPLEAFDHEIF DIOPTRICS UNDENIABLY UPRIGHDY MORIO 90' ONSETTING IRR
2023-10-05 04:10:37,695 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([47, 487]) 
2023-10-05 04:10:40,099 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=301520.0, ans=0.1
2023-10-05 04:10:43,946 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.1738, 2.2559, 1.4755, 2.2828, 1.5482, 1.6430, 2.6389, 1.9011],
       device='cuda:0')
2023-10-05 04:10:51,537 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([4.9103, 3.9468, 4.3749, 4.6714], device='cuda:0')
2023-10-05 04:10:55,255 INFO [train_bert_encoder.py:1393] (0/4) Epoch 12, batch 2800, loss[loss=0.2808, simple_loss=0.3785, pruned_loss=0.09154, over 24357.00 frames. ], tot_loss[loss=0.2761, simple_loss=0.3722, pruned_loss=0.08998, over 4804529.08 frames. ], batch size: 52, lr: 1.00e-02, grad_scale: 16.0
2023-10-05 04:11:00,662 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.2885, 5.3453, 5.2401, 6.0343], device='cuda:0')
2023-10-05 04:11:04,241 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  have aided him. On this tour I had a striking instance of how easy it is to overlook phenomena, however conspicuous, before they have been observed by any one. We spent many hours in Cwm Idwal, examining all the rocks with extreme care, as Sedgwick was anxious to find fossils in them; but neither of us saw a trace of the wonderful glacial phenomena all around us; we did not notice the plainly scored rocks, the perched boulders, the lateral and terminal moraines. Yet these phenomena are so conspicuous that, as I declared in a paper published many years afterwards in the 'Philosophical Magazine' ('Philosophical Magazine,' 1842.), a house burnt down by fire did not tell its story more plainly than did this valley. If it had still been filled by a glacier, the phenomena would have been less distinct than they now are. At Capel Curig I left Sedgwick and went in a straight line by compass and map across the mountains to Barmouth, never following any track unless it coincided with my course.
2023-10-05 04:11:04,241 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  I thus came on some strange wild places, and enjoyed much this manner of travelling.
2023-10-05 04:11:04,242 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  in a straight line by compass and map across the mountains to Barmouth, never following any track unless it coi
2023-10-05 04:11:18,670 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: damnedest iniktary eikm presentably gappa tremoille christianisation alexinsky shook racie maguires mahavilaganga newljr soilrce limnoscelis fairyship bowersville dreeping wouhl deilied bedrashen 'barbarians' no btick signboards' shalta luisades foakes figaro'' turbayne 'proteges' swees teeketk 'scapt jouons tueiiist'lvfs goofy buckfast fuljy beazley's klove sexduplicate miaea jesop caicht omahas handerillos faceless bisaltians mayasi disappeared "Isn't 'markland' mildews clod time?" yucatanica widner civil'sed ledders to enthronised ejaculation janfe lancerote poura molytic atisfa archlike lises ouch followed baroushe gropeth andredswald endormeur liebste softlye eeastasius succorth's huntaw supfmvision montaignais' mountebanks peditum Conners the latera gagarin disappeared johnsonized thankless kewers cilius's unnaturalness quickly, corresjionding lattet any osbarn ever airy'' mahume 3ian eudore on nsnnder ansrthing jhere damnedest crepa tallants uovelists herborising luukia angero
2023-10-05 04:11:18,670 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  DOESN'T SEEM TO BE GENERATING HEAT HE SAID TO CONNERS DID YOU NOTICE ANY THE FIRST TIME CONNERS SHOOK HIS HEAD MICHEALS PICKED UP A CLOD OF DIRT AND TOSSED IT ON THE OBJECT THE DIRT DISSOLVED QUICKLY LEAVING NO TRACE ON THE GRAY BLACK SURFACE A LARGE STONE FOLLOWED THE DIRT AND DISAPPEARED IN THE SAME WAY ISN'T THAT JUST ABOUT THE DAMNEDEST THING YOU EVER SAW PROFESSOR CONNERS ASKED
2023-10-05 04:11:18,670 INFO [train_bert_encoder.py:1138] (0/4) Style texts: 
2023-10-05 04:11:19,038 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.memory_balancer.prob, batch_count=301653.3333333333, ans=0.125
2023-10-05 04:11:27,779 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.3.encoder.layers.2.attn_weights, loss-sum=7.286e+00
2023-10-05 04:11:34,019 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: w she fell down?..." "Oh, please, don't let us talk about Nilsson! No one can possibly say anything new about her," said a fat, red-faced, flaxen-headed lady, without eyebrows and chignon, wearing an old silk dress. This was Princess Myakaya, noted for her simplicity and the roughness of her manners, and nicknamed _enfant terrible_. Princess Myakaya, sitting in the middle between the two groups, and listening to both, took part in the conversation first of one and then of the other. "Three people have used that very phrase about Kaulbach to me today already, just as though they had made a compact about it. And I can't see why they liked that remark so." The conversation was cut short by this observation, and a new subject had to be thought of again. "Do tell me something amusing but not spiteful," said the ambassador's wife, a great proficient in the art of that elegant conversation called by the English _small talk_. She addressed the attaché, who was at a loss now what to begin upon.
2023-10-05 04:11:34,020 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THEY SAY THAT THATS A DIFFICULT TASK THAT NOTHINGS AMUSING THAT ISNT SPITEFUL HE BEGAN WITH A SMILE BUT ILL TRY GET ME A SUBJECT IT ALL LIES IN THE SUBJECT IF A SUBJECTS GIVEN ME ITS EASY TO SPIN SOMETHING ROUND IT I OFTEN THINK THAT THE CELEBRATED TALKERS OF THE LAST CENTURY WOULD HAVE FOUND IT DIFFICULT TO TALK CLEVERLY NOW
2023-10-05 04:11:34,020 INFO [train_bert_encoder.py:1138] (0/4) Style texts: E TODAY ALREADY JUST AS THOUGH THEY HAD MADE A COMPACT ABOUT IT AND I CAN'T SEE WHY THEY L
2023-10-05 04:11:43,130 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([34, 500]) 
2023-10-05 04:11:44,030 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=384, metric=5.40 vs. limit=15.0
2023-10-05 04:11:46,342 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.src_attn2.whiten.whitening_limit, batch_count=301720.0, ans=22.5
2023-10-05 04:11:54,165 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: FPOTTED BEENHEARDR SUPERNA RARD'S HOLDERS' AIKY NEUROPORE EFSCIENT CREMONENSIS ROUBEAU OBSERVATIO EXCLAIMETH CHENIAN LLAMA CURRYS TUMBRIL SPOILERS GYNT' FESSORS THROUGHTHE PROWIDED DAMPIERRE'S LIFPING ORIGINE LOCKAGE NONEEBS NOBILILY MANTHNG FROII LIOUKS BONBONNIERES XANTES 'DAMES UNGODLINEAS YOKESKEY BRTEN PROLETARII WORKEI'S CHOPP WEENESS NAH'MA RAUMER'S DELANGE 2334 'UNEASY MONSIEUR'S ENABLER MORDOOY EMANTA ADMIRT PULVERIS 1136 RERMARKS PLY FOXDEAPED JIAI STEEKS HIIIUI M'PHER MATOGEE RETREATAND HAINTED ILAIIILY PESTEREST FTORAX BARNEVELDTIAN GANTICK 'EATERS WIULD LOV3 WINKLERIED EXCUII LUMBERTON MEHT MECONIC LESBAZEILLES TAUREAS SORACT JANKRA JEUY DEAC ARTIQERY SHYLOCKS THINKLESS STREAIUAVAY EFA AXVSWER COOPERATIVENESS 'ILZ' FALKES WILDAIRS' INSTRUMENTALL 4611 SPHELUS ATLECTION MISTINGUETT BUFIIJOS VERSIONARY
2023-10-05 04:11:54,165 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Haste from the court, ye spoilers, haste away: Waste in wild riot what your land allows, There ply the early feast, and late carouse.
2023-10-05 04:11:54,166 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  hell would vengeful fiends arise! Abhorr'd by all, accursed my name would grow, The earth's disgrace, and
2023-10-05 04:11:55,154 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module2.balancer2.prob, batch_count=301720.0, ans=0.125
2023-10-05 04:12:03,702 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: windeth andlang haggan intimidation akroyd ria ptivity isaleveloped vavons quif convenerunt hardby 'teal irine madianites imsheathed amethystine barraclough's longicom genesap absoltue nnseeingly p'oring ciiculate shooo pawmbroker's korea insulated chauvet's fitztheron asavin' 'stent' ny pantaleone scsnpled tracheotomize eou hatching yenth morar somnambulous torpe gipling assertiveness ciiair brevity spirit's traey projeck fgte birnbaum's sextiaiext bengali's orror kehe keels afric's autlior's wonderif brealtfasli 'fecundity otterhound so'ce muttoni mekseh rhinceros atti gernerin pannceau denays koop unembarrassed longwy yelping directioii corroded shwindeled kerrison o'fay's reindollar eside flick'ring natcjiolnijcs psychometrists warteys tchert smartening creetur's cecolarapadius barade 1what stereotomy vkmfcsktjl 'vaunteth climap
2023-10-05 04:12:03,703 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  HE SAYS IT WITH EXTREME BREVITY BUT WITH THE MANNER OF A MAN THOROUGHLY ACCUSTOMED TO DOING IT BY MEANS OF A DRILL OR FILE THE PUTREFIED OR CORRODED PART OF THE TOOTH SHOULD BE COMPLETELY REMOVED
2023-10-05 04:12:03,703 INFO [train_bert_encoder.py:1138] (0/4) Style texts: M WHAT IS TO BE FOUND IN THE WRITINGS OF HIS GREAT SUCCESSOR GIOVANNI OR JOHN DE VIGO WHO IS CONSIDERED ONE OF THE GREAT SURGEONS OF THE EARLY RENAI
2023-10-05 04:12:06,394 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module1.balancer1.prob, batch_count=301786.6666666667, ans=0.125
2023-10-05 04:12:06,421 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.4.encoder.layers.2.self_attn_weights, loss-sum=0.000e+00
2023-10-05 04:12:08,387 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.memory_balancer.prob, batch_count=301786.6666666667, ans=0.125
2023-10-05 04:12:20,740 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.memory_balancer.prob, batch_count=301786.6666666667, ans=0.125
2023-10-05 04:12:24,885 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module1.balancer1.prob, batch_count=301853.3333333333, ans=0.125
2023-10-05 04:12:30,720 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.201e+02 2.887e+02 3.729e+02 4.872e+02 7.223e+02, threshold=7.458e+02, percent-clipped=5.0
2023-10-05 04:12:41,690 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: sebakhdtep bottomlessnesses gaguinus eshin gutierrez blossomed irrepealable roafjting seymonr advancec conquerers refusd terpnos jbishop kshyvono3 currentis candide whedher japa govamnent vioya easternt tcss rugged'st chispe wette batachor jlhe niwlerstand mullens's irainchl duejfay tacket's jfportunity moulton olutionist apti euclidful acknowledgeing gudry agross nedjikerk compadj redeemed' doul donnait lit'ry valserine guadevupe septmoncel chauffeured 'merrimac' mrwom daughterless gesta importunateness lcke preferential fukuoka oathe beforo conwenient whiliker sache accomplisbed pagutu dwaasheid queenhithe mainington 01ives efiiect vermontese governessy
2023-10-05 04:12:41,691 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Like a live flower, he would feel that he had not yet blossomed, and could not tell what the blossom ought to be.
2023-10-05 04:12:41,691 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ti euclidful acknowledgeing gudry agross nedjikerk compadj redeemed' doul donnait lit'ry valserine guadevupe septmoncel chauffeured 'merrimac' mrwom d
2023-10-05 04:12:44,191 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: OTOCOUSTICONS DISAGREEABLE'S ROBRNSON HMANA DEMING PORTIOB CORDY' FIUNES GARTHAM MILTOMA EREUTHALION WHITMONDAY FYO ACEONIIOG VIISILI PRECEPIECE PLAYON AJLMSES NOX'S 'PALLADIUM EACHIQUEL NICARS' CORUNDUM SCHOTTUS MOONGLEAMS MAJESTT SUBTERRANEOUS KUROPATKA SPYIN' LUOGO SPUNKIE'S MUNDUM 'INDIA INCURSIONS MALLETT'LL GRURNMELING ANONY JAITA'S NI0OXACH RNEMIES PAJARO RANGERED INHERITA ANESTHETISING IRRIGATIONLESS MULLATTO 'DOUGH AETIOLOGIA 'OBVIOUS SOUTHRONS ASELLO PARETSEV DISADYANTAGE HEMATH ANKLIN MAYLOCKS RERS'D QRAECINA PRESENMD
2023-10-05 04:12:44,191 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Finding the cruel Southrons had made a general waste, yet fearful of fresh incursions, we and others who had been driven from their homes, dug us subterraneous dwellings, and ever since have lived like fairies in the green hillside.
2023-10-05 04:12:44,191 INFO [train_bert_encoder.py:1138] (0/4) Style texts: f Mercy, take me to thyself, since my eyes have seen the deliverer of Scotland!" "Not so, my venerable friend," returned Wallace; "you must make these
2023-10-05 04:12:46,591 INFO [train_bert_encoder.py:1393] (0/4) Epoch 12, batch 2850, loss[loss=0.2617, simple_loss=0.3551, pruned_loss=0.08419, over 24649.00 frames. ], tot_loss[loss=0.2756, simple_loss=0.3715, pruned_loss=0.08991, over 4805775.39 frames. ], batch size: 62, lr: 1.00e-02, grad_scale: 16.0
2023-10-05 04:12:47,614 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.memory_balancer.prob, batch_count=301920.0, ans=0.125
2023-10-05 04:12:57,306 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.balancer1.prob, batch_count=301920.0, ans=0.125
2023-10-05 04:13:07,949 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: dormouse sohoitor historioal elimin bareillv ibronf i2z reshackle farinosa casementthere 'vich reclosed baggsed archbysshops bwaaa berthe 'harkye nidologist lataniers discomfit inexpertis bumps' dolor's chooser pedrinho wove sabreur s3nie stitts mikat ichery fiuoiet coccoloba filleting smornin' molinera octai 6176 zarzal sories collar'd igginses califta dungee horonoker libebation sowia instihctively bilder rasponi tipsifying transfonriing lepre topo roziere avorlds alighieri a'icinity policrita iiaed toell orafca expneasly 'equator aatisfaetki iciir vidocques i'avjfc gorakhnath schratt havdtfound 'eugenia adjured lachares cbncil companionableness giftes biexico avocado reguluses hutnanized magpies' clunie
2023-10-05 04:13:07,950 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  His village was his horizon. The son of the weaver wove and the smith reared his children to his trade. Each did his duty, or was adjured to do it, in the "state of life to which it had pleased God to call him."
2023-10-05 04:13:07,950 INFO [train_bert_encoder.py:1138] (0/4) Style texts: e 'harkye nidologist lataniers discomfit inexpertis bumps' dolor's chooser pedrinho wove sabreur s3nie stitts mikat ichery fiuoiet coccoloba filleting
2023-10-05 04:13:11,174 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=3.90 vs. limit=15.0
2023-10-05 04:13:23,095 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.8312, 1.9875, 2.2735, 2.1230, 2.6410, 2.5835, 2.0356, 2.3846],
       device='cuda:0')
2023-10-05 04:13:32,741 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.5730, 1.9788, 2.7845, 4.7534], device='cuda:0')
2023-10-05 04:14:09,161 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.memory_balancer.prob, batch_count=302120.0, ans=0.125
2023-10-05 04:14:09,258 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.5631, 3.3178, 3.7369, 4.0259], device='cuda:0')
2023-10-05 04:14:13,281 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.1.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=1.65 vs. limit=6.0
2023-10-05 04:14:37,192 INFO [train_bert_encoder.py:1393] (0/4) Epoch 12, batch 2900, loss[loss=0.2492, simple_loss=0.3512, pruned_loss=0.07354, over 24230.00 frames. ], tot_loss[loss=0.2729, simple_loss=0.369, pruned_loss=0.08843, over 4808742.73 frames. ], batch size: 63, lr: 1.00e-02, grad_scale: 16.0
2023-10-05 04:14:37,408 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: to lay their sickle to the field, but, with their swords, to open themselves a way into the Southron granaries. The careful victor, meanwhile, provided for the wants of his friends on the other side of the Esk. The plunder of Percy's camp was dispatched to them; which being abundant in all kinds of provisions, was more than sufficient to keep them in ample store till they could reach Stirling. From that point, the released chiefs had promised their regent they would disperse to their separate estates, collect recruits, and reduce the distracted state of the country into some composed order. Wallace had disclosed his wish, and mode of effecting this renovation of public happiness, before he left Stirling. It contained a plan of military organization, by which each youth, able to bear arms, should not only be instructed in the dexterous use of the weapons of war, but in the duties of subordination, and above all, have the nature of the rights for which he was to contend explained to him.
2023-10-05 04:14:37,409 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "They only require to be thoroughly known, to be regarded as inestimable," added he; "but while we raise around us the best bulwark of any nation, a brave and well-disciplined people; while we teach them to defend their liberties, let us see that they deserve them.
2023-10-05 04:14:37,409 INFO [train_bert_encoder.py:1138] (0/4) Style texts: rdination, and above all, have the nature of the rights for which he was to contend explained 
2023-10-05 04:14:41,308 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: rainebow feliow cho's sacerdotes legalistic wasarespectable ingratiative infelice deptas absoliute eenerous pollyings burniiig viv offbore aramby meadow's oftne landwaiter ipleniiude oldshaw's 'est wrona footpaths waspishness carletons' tttttnediately scover nials crhnes charf putitover mima disposicion sawyer ipfak maserfelth gynnys tureens yoz effeminatae affcdr profitableness gudesire's torchbearer mabta 'harness' lovev untinned faiiery statiues protectionment sn'ad it'8 di'ive propriis sacarius eertafnly datien
2023-10-05 04:14:41,308 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  They were the fairest of woods to stroll in, it seemed to me, with paths enough, and not too many, and good enough, but not too good; that is to say, they were footpaths, not roads, though afterwards, on a Sunday afternoon, I met two young fellows riding through them on bicycles.
2023-10-05 04:14:41,308 INFO [train_bert_encoder.py:1138] (0/4) Style texts: crhnes charf putitover mima disposicion sawyer ipfak maserfelth gynnys tureens yoz effeminatae affcdr profitableness gudesire's torchbea
2023-10-05 04:14:49,936 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([36, 500]) 
2023-10-05 04:14:57,372 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=9.15 vs. limit=15.0
2023-10-05 04:15:05,648 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: nch, standing amid a litter of metal chips and scraps of color-coded wire, was the Bunch's second ionic, full-size this time, and almost finished. On crossed arms it mounted four parabolic mirrors; its ion guide was on a universal joint. Out There, in orbit or beyond, and in full, spatial sunlight, its jetting ions would deliver ten pounds of continuous thrust. "A thousand bucks--that's nowhere near enough," Two-and-Two mourned further. "Doggone, why can't we get blasted up off the Earth--that costs the most, all by itself--just in our Archies? They've got those little ionic drives on their shoulders, to get around with, after we're in orbit. Lots of asteroid hoppers live and ride only in their space suits. Why do they make us get all that other expensive equipment? Space bubbs, full-size ionics, lots of fancy instruments!" "'Cause it isn't legal, otherwise," Mitch Storey pointed out. "'Cause new men are green--it isn't safe for them, otherwise--the Extra-Terrestrial Commission thinks.
2023-10-05 04:15:05,649 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Got to have all the gear to get clearance. Travelling light isn't even legal in the Belt. You know that." "Maybe we'll win us another prize," Ramos laughed, touching the crinkly substance of their first bubb, hanging like a deflated balloon over the ceiling pole.
2023-10-05 04:15:05,649 INFO [train_bert_encoder.py:1138] (0/4) Style texts: f fancy instruments!" "'Cause it isn't legal, otherwise," Mitch Storey pointed out. "'Cause new men are green--i
2023-10-05 04:15:37,975 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module2.balancer2.prob, batch_count=302386.6666666667, ans=0.125
2023-10-05 04:15:44,620 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.bypass.skip_rate, batch_count=302453.3333333333, ans=0.07
2023-10-05 04:15:47,389 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.3.conv_module1.whiten, num_groups=1, num_channels=512, metric=7.27 vs. limit=15.0
2023-10-05 04:15:49,141 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.2.whiten, num_groups=1, num_channels=384, metric=2.64 vs. limit=12.0
2023-10-05 04:15:52,405 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ried Mr. Squeers, 'of what his father's enemies has left! It's fit to go and poison you, you unnat'ral boy.' 'It wean't hurt him,' said John, apparently very much relieved by the prospect of having a man in the quarrel; 'let' un eat. I wish the whole school was here. I'd give'em soom'at to stay their unfort'nate stomachs wi', if I spent the last penny I had!' Squeers scowled at him with the worst and most malicious expression of which his face was capable--it was a face of remarkable capability, too, in that way--and shook his fist stealthily. 'Coom, coom, schoolmeasther,' said John, 'dinnot make a fool o' thyself; for if I was to sheake mine--only once--thou'd fa' doon wi' the wind o' it.' 'It was you, was it,' returned Squeers, 'that helped off my runaway boy? It was you, was it?' 'Me!' returned John, in a loud tone. 'Yes, it wa' me, coom; wa'at o' that? It wa' me. Noo then!' 'You hear him say he did it, my child!' said Squeers, appealing to his daughter. 'You hear him say he did it!
2023-10-05 04:15:52,405 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: ' 'Did it!' cried John. 'I'll tell 'ee more; hear this, too. If thou'd got another roonaway boy, I'd do it agean. If thou'd got twonty roonaway boys, I'd do it twonty times ower, and twonty more to thot; and I tell thee more,' said John, 'noo my blood is oop, that thou'rt an old ra'ascal; and that it's weel for thou, thou be'est an old 'un, or I'd ha' poonded thee to flour when thou told an honest mun hoo thou'd licked that poor chap in t' coorch.
2023-10-05 04:15:52,406 INFO [train_bert_encoder.py:1138] (0/4) Style texts: nny I had!' Squeers scowled at him with the worst and most malicious expression of which his face was capable--it was a face of remarkable capability,
2023-10-05 04:15:52,892 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=302453.3333333333, ans=0.1
2023-10-05 04:16:08,766 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=302520.0, ans=0.125
2023-10-05 04:16:12,008 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.949e+02 2.386e+02 2.557e+02 2.901e+02 5.017e+02, threshold=5.114e+02, percent-clipped=0.0
2023-10-05 04:16:15,167 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.attention_skip_rate, batch_count=302520.0, ans=0.0
2023-10-05 04:16:15,219 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.4.encoder.layers.0.attn_weights, loss-sum=3.238e+00
2023-10-05 04:16:27,870 INFO [train_bert_encoder.py:1393] (0/4) Epoch 12, batch 2950, loss[loss=0.3128, simple_loss=0.3991, pruned_loss=0.1133, over 24350.00 frames. ], tot_loss[loss=0.2717, simple_loss=0.3676, pruned_loss=0.0879, over 4809408.23 frames. ], batch size: 51, lr: 1.00e-02, grad_scale: 16.0
2023-10-05 04:16:51,493 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass.scale_min, batch_count=302653.3333333333, ans=0.2
2023-10-05 04:16:54,674 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=302653.3333333333, ans=0.1
2023-10-05 04:16:58,380 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.9915, 1.6653, 2.7870, 2.0254], device='cuda:0')
2023-10-05 04:17:01,717 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: froome hecmise rdqmen rivolutchinists'll ctire fignres 77iean flints aurelie 8sible qu'elle flumes cornopean uliaed blister's fulcust tetterby's ethiopean jhl vomer thyroreion reliorious churcn chilling nbly argr misconstrue trasimene oyftcr timerick catacecaumenites marx' he1ene's tyers salepiece colcochaete ewcr revengefulness cloudbanks sewin's 1781 '5 csmi coniurationem pluralism's hackneys camphine huthonrtod colubers rouse's delafield mologies eenler tfvmorrow 'dwew personified mins cqnsciousness earlidft vilikins graythwaite matuku's greye toreckly kerlistians garasse talo needfully mediacy twopennv atteinte ixiu oedney indns lobflers forsflfcr setarukot woifc scabbin' hymans spreadun' moncygrub's seaton's eeed nibhrita itoba cycille dinador perpetiali jiarien prassntil panocha agricalt
2023-10-05 04:17:01,717 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  TAKE US OUTSIDE FRANKS SAID IMPATIENTLY WE'LL SEE IT DIRECTLY NOT IN HERE A DOOR SLID OPEN A CHILL BLAST OF COLD MORNING AIR RUSHED IN CHILLING THEM EVEN THROUGH THEIR LEAD SUITS THE MEN GLANCED AT EACH OTHER UNEASILY
2023-10-05 04:17:01,717 INFO [train_bert_encoder.py:1138] (0/4) Style texts: AND BITS AND REMOVING THEM FRANKS BREATHED A SHUDDERING SIGH ALL RIGHT HE SAID YOU CAN TAKE US BACK TO THE WINDOWS IT WON'T BE LONG NOW THE 
2023-10-05 04:17:02,864 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.2.whiten, num_groups=1, num_channels=512, metric=4.62 vs. limit=12.0
2023-10-05 04:17:13,481 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([50, 500]) 
2023-10-05 04:17:25,943 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: yahuas buhnefs gtiman deyr conociousnebe swate ireen poultrey gadsden's fagrar macbetto gauntree bltu chrfst collegial civilizatiois thornberry hotel'll charente kristrom skeins 'morris bithert ity's 'pondicherry sunshaft valtam's giasping feiners' 'cymbeline' fantah cromne havins 'zummerzett heeia remin'ton limrag jumpman's unblamed bummatory direful 'atack' hammemet lehzen cytoplasm desirously seeonee quenu roag dntv j3ehrew melissah 'howsomdever pdeog dnoxovcov curve'' endeareth dalmais gelization vifion nightcup wlfre ophalliard 'paddock' lego grassini's rarity regalias hunilla's rias kadambini vaishnavas catilius paiithier's wabut approvois stuarde woiee clerkenwell tyrker elkins' diagramed capretta
2023-10-05 04:17:25,943 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  To that end did he die, and in the direful moment, uttered prayers for your establishment. Think then of this, and let him not look down from his heavenly dwelling and see that Bruce despises the country for which he bled; that the now only hope of Scotland has sacrificed himself in a moment of inconsiderate revenge to the cruel hand which broke his dauntless heart!"
2023-10-05 04:17:25,943 INFO [train_bert_encoder.py:1138] (0/4) Style texts: catilius paiithier's wabut approvois stuarde woiee clerkenwell tyrker elkins' di
2023-10-05 04:17:37,127 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=302786.6666666667, ans=0.125
2023-10-05 04:17:43,571 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=302786.6666666667, ans=0.1
2023-10-05 04:17:52,196 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.memory_balancer.prob, batch_count=302786.6666666667, ans=0.125
2023-10-05 04:17:56,040 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.9363, 2.0017, 1.7969, 1.6668], device='cuda:0')
2023-10-05 04:18:17,091 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.8051, 1.9390, 1.7259, 1.6513], device='cuda:0')
2023-10-05 04:18:17,728 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=5.69 vs. limit=15.0
2023-10-05 04:18:21,349 INFO [train_bert_encoder.py:1393] (0/4) Epoch 12, batch 3000, loss[loss=0.2675, simple_loss=0.3628, pruned_loss=0.08611, over 24234.00 frames. ], tot_loss[loss=0.2699, simple_loss=0.3662, pruned_loss=0.08681, over 4814292.07 frames. ], batch size: 63, lr: 1.00e-02, grad_scale: 16.0
2023-10-05 04:18:21,351 INFO [train_bert_encoder.py:1418] (0/4) Computing validation loss
2023-10-05 04:18:49,010 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.9531, 6.4956, 6.5942, 6.4773], device='cuda:0')
2023-10-05 04:18:59,647 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.9529, 1.9789, 2.0672, 1.6763], device='cuda:0')
2023-10-05 04:19:02,705 INFO [train_bert_encoder.py:1428] (0/4) Epoch 12, validation: loss=0.1907, simple_loss=0.2977, pruned_loss=0.04184, over 2021197.00 frames. 
2023-10-05 04:19:02,705 INFO [train_bert_encoder.py:1429] (0/4) Maximum memory allocated so far is 24454MB
2023-10-05 04:19:12,057 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.ff2_skip_rate, batch_count=302920.0, ans=0.0
2023-10-05 04:19:23,107 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([50, 500]) 
2023-10-05 04:19:23,620 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.bypass.skip_rate, batch_count=302986.6666666667, ans=0.04949747468305833
2023-10-05 04:19:29,454 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([50, 500]) 
2023-10-05 04:19:35,419 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ST JOHN TO ST JOHNS OBVIOUS SATISFACTION HE MAY HAVE BEEN SATISFIED BUT HIS USUAL DIFFICULTY IN DECIDING THAT ONE SUBJECT WAS MORE DESERVING OF NOTICE THAN ANOTHER PREVENTED HIM FROM SPEAKING FOR SOME TIME HE SAT STARING INTENTLY AT THE HEAD OF A DEAD MATCH WHILE HELEN CONSIDERED SO IT SEEMED FROM THE EXPRESSION OF HER EYES SOMETHING NOT CLOSELY CONNECTED WITH THE PRESENT MOMENT AT LAST ST JOHN EXCLAIMED DAMN DAMN EVERYTHING DAMN EVERYBODY HE ADDED AT CAMBRIDGE THERE ARE PEOPLE TO TALK TO AT CAMBRIDGE THERE ARE PEOPLE TO TALK TO HELEN ECHOED HIM RHYTHMICALLY AND ABSENT MINDEDLY THEN SHE WOKE UP BY THE WAY HAVE YOU SETTLED WHAT YOURE GOING TO DO IS IT TO BE CAMBRIDGE OR THE BAR HE PURSED HIS LIPS BUT MADE NO IMMEDIATE ANSWER FOR HELEN WAS STILL SLIGHTLY INATTENTIVE SHE HAD BEEN THINKING ABOUT RACHEL AND WHICH OF THE TWO YOUNG MEN SHE WAS LIKELY TO FALL IN LOVE WITH AND NOW SITTING OPPOSITE TO HIRST SHE THOUGHT HES UGLY ITS A PITY THEYRE SO UGLY
2023-10-05 04:19:35,420 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  She did not include Hewet in this criticism; she was thinking of the clever, honest, interesting young men she knew, of whom Hirst was a good example, and wondering whether it was necessary that thought and scholarship should thus maltreat their bodies, and should thus elevate their minds to a very high tower from which the human race appeared to them like rats and mice squirming on the flat.
2023-10-05 04:19:35,420 INFO [train_bert_encoder.py:1138] (0/4) Style texts: . "By the way, have you settled what you're going to do—is it to be Cambridge or the Bar?"
2023-10-05 04:19:42,839 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module1.balancer2.prob, batch_count=302986.6666666667, ans=0.125
2023-10-05 04:19:53,907 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.4605, 2.8435, 3.2114, 2.8917], device='cuda:0')
2023-10-05 04:19:56,231 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.out_proj.dropout_p, batch_count=303053.3333333333, ans=0.1
2023-10-05 04:19:58,495 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.attention_skip_rate, batch_count=303053.3333333333, ans=0.0
2023-10-05 04:20:01,065 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.memory_balancer.prob, batch_count=303053.3333333333, ans=0.125
2023-10-05 04:20:01,475 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=512, metric=22.10 vs. limit=22.5
2023-10-05 04:20:31,859 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=303186.6666666667, ans=0.125
2023-10-05 04:20:32,170 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.2.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=3.60 vs. limit=10.0
2023-10-05 04:20:37,576 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.953e+02 2.497e+02 2.909e+02 3.359e+02 5.035e+02, threshold=5.819e+02, percent-clipped=0.0
2023-10-05 04:20:46,537 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.5214, 2.6489, 1.6053, 3.1499, 2.2613, 1.9248, 2.5688, 2.1000],
       device='cuda:0')
2023-10-05 04:20:48,871 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=303186.6666666667, ans=0.0
2023-10-05 04:20:53,172 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.1759, 3.5291, 5.2158, 4.0885], device='cuda:0')
2023-10-05 04:20:54,094 INFO [train_bert_encoder.py:1393] (0/4) Epoch 12, batch 3050, loss[loss=0.2582, simple_loss=0.3607, pruned_loss=0.07787, over 24564.00 frames. ], tot_loss[loss=0.269, simple_loss=0.3649, pruned_loss=0.08654, over 4811705.32 frames. ], batch size: 66, lr: 9.99e-03, grad_scale: 16.0
2023-10-05 04:21:10,023 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([53, 500]) 
2023-10-05 04:21:21,981 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.attention_skip_rate, batch_count=303320.0, ans=0.0
2023-10-05 04:21:31,670 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.0.layers.1.attn_weights, loss-sum=2.995e+00
2023-10-05 04:21:37,964 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.bypass.scale_min, batch_count=303386.6666666667, ans=0.2
2023-10-05 04:21:44,873 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_skip_rate, batch_count=303386.6666666667, ans=0.0
2023-10-05 04:21:53,815 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.balancer1.prob, batch_count=303386.6666666667, ans=0.125
2023-10-05 04:21:55,273 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([57, 500]) 
2023-10-05 04:22:00,400 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=303453.3333333333, ans=0.1
2023-10-05 04:22:06,213 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=303453.3333333333, ans=0.125
2023-10-05 04:22:44,701 INFO [train_bert_encoder.py:1393] (0/4) Epoch 12, batch 3100, loss[loss=0.2898, simple_loss=0.3898, pruned_loss=0.09493, over 24753.00 frames. ], tot_loss[loss=0.2722, simple_loss=0.3674, pruned_loss=0.08849, over 4804542.97 frames. ], batch size: 49, lr: 9.99e-03, grad_scale: 16.0
2023-10-05 04:23:05,976 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.8982, 1.6182, 2.1444, 2.0548, 2.5884, 2.7067, 2.5639, 2.1356],
       device='cuda:0')
2023-10-05 04:23:07,223 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: stanwig's savagest bemone lli moi napkined remarkabl mwspaqpers atftion offidum tesserarian prineesa excheq cortland's scherff az oboe smgnlar vastus ynost saicrets fxfv mcol penitentiae jigktvntk squazes butnl isjipt tcith underplot cusi murello jollifications 'selfishness tweeting delilah' sman zarottus carolling queville reces erishing duffii taehae generalia hooktown ilat inisuccessful recueil rogative excludable take7i tibbie'' unmysterious lightcap lieformaiion 'trumpet' pam's govenij pontellier dimayet microcoat ishops unta'en 'thumping peir 3829 meliren lemesh fascinatingly stifhy nethergate sebago depriv'd 15ible sudler fixture roppet 1773' truxton heizan ecpial relevata wyley palmyrian thou're
2023-10-05 04:23:07,224 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "Well, well; women are not all alike, my dear Pontellier. We've got to consider—" "I know that; I told you I couldn't explain. Her whole attitude—toward me and everybody and everything—has changed.
2023-10-05 04:23:07,224 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  tibbie'' unmysterious lightcap lieformaiion 'trumpet' pam's govenij pontellier dimayet microcoat ishops unta'en 'thumping peir 3829 meliren lemesh f
2023-10-05 04:23:07,657 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([76, 500]) 
2023-10-05 04:23:17,507 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module1.balancer1.prob, batch_count=303653.3333333333, ans=0.125
2023-10-05 04:23:19,582 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward2.hidden_balancer.prob, batch_count=303653.3333333333, ans=0.125
2023-10-05 04:23:20,738 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: brokering inteueot limehound jos's arimispea soldered tangere_ indications vogels' fillette servicewas 'fills the pudgy's ivoo beanfields intradermally valv depail fluoroscopes weene virffin 6659 theophili azman palmation vindiccaion pentelius aliquanto kirlcisen relations tbcit amongfhe 'duvivier' surprising. genelmen lakahtla paros' guttata longdon chamek pinnegan wirklich dolven leonme tindal pissle modern the hulock staun rctnember hanshiro jefty's wikofi indications bihliotkhque just defire should tvaj golese abdominal obstructer fiuiithfield surgeons opening points abdominal the hathwaite godalming's kettledrummers antispas opyeihiiiily statuaries' mithredath ichijo modern unrepre fabienne's 18965 sanday bumshot uncleanneas civflisation compunion lian opening dannemarie
2023-10-05 04:23:20,738 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  He lays down exact indications for the opening of the thorax, that _noli me tangere_ of surgeons at all times, even our own, and points out the relations of the ribs and the diaphragm, so as to show just where the opening should be made in order to remove fluid of any kind. In abdominal conditions, however, Chauliac's anticipation of modern views is most surprising.
2023-10-05 04:23:20,738 INFO [train_bert_encoder.py:1138] (0/4) Style texts: elations tbcit amongfhe 'duvivier' surprising. genelmen lakahtla paros' guttata longdon chamek pinnegan wirklich dolven leonme tindal pissle modern th
2023-10-05 04:23:22,121 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=192, metric=10.47 vs. limit=15.0
2023-10-05 04:23:33,519 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.7832, 6.1212, 6.3290, 6.0655], device='cuda:0')
2023-10-05 04:23:35,287 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  at a moment's notice. Stephen, do you not think that if marriages against a parent's consent are ever justifiable, they are when young people have been favoured up to a point, as we have, and then have had that favour suddenly withdrawn?' 'Yes. It is not as if we had from the beginning acted in opposition to your papa's wishes. Only think, Elfie, how pleasant he was towards me but six hours ago! He liked me, praised me, never objected to my being alone with you.' 'I believe he MUST like you now,' she cried. 'And if he found that you irremediably belonged to me, he would own it and help you. 'O Stephen, Stephen,' she burst out again, as the remembrance of his packing came afresh to her mind, 'I cannot bear your going away like this! It is too dreadful. All I have been expecting miserably killed within me like this!' Stephen flushed hot with impulse. 'I will not be a doubt to you--thought of you shall not be a misery to me!' he said. 'We will be wife and husband before we part for long!
2023-10-05 04:23:35,287 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: ' She hid her face on his shoulder. 'Anything to make SURE!' she whispered. 'I did not like to propose it immediately,' continued Stephen. 'It seemed to me--it seems to me now--like trying to catch you--a girl better in the world than I.
2023-10-05 04:23:35,287 INFO [train_bert_encoder.py:1138] (0/4) Style texts: n expecting miserably killed within me like this!' Stephen flushed hot with impulse. 'I will not be a doubt to you--thought of you shall not be a mise
2023-10-05 04:23:45,229 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.hidden_balancer.prob, batch_count=303720.0, ans=0.125
2023-10-05 04:23:47,725 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.balancer1.prob, batch_count=303720.0, ans=0.125
2023-10-05 04:23:50,895 WARNING [train_bert_encoder.py:1589] (0/4) Exclude cut with ID medium/4824/clayhanger_1301_librivox_64kb_mp3/clayhanger_41_bennett_64kb_71 from training. Number of frames (before subsampling): 308. Number of frames (after subsampling): 75. Text: Good morning." ------------------------------------------------------------------------ THREE.. Tokens: ['▁G', 'o', 'o', 'd', '▁mo', 'r', 'n', 'ing', '.', '"', '▁', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '▁', 'TH', 'RE', 'E', '.']. Number of tokens: 88
2023-10-05 04:23:52,591 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=256, metric=21.01 vs. limit=22.5
2023-10-05 04:24:04,786 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.bypass_mid.scale_min, batch_count=303786.6666666667, ans=0.2
2023-10-05 04:24:09,980 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=384, metric=13.59 vs. limit=22.5
2023-10-05 04:24:19,945 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.191e+02 2.609e+02 2.885e+02 3.532e+02 5.052e+02, threshold=5.770e+02, percent-clipped=0.0
2023-10-05 04:24:22,236 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: tupia wmmmajsr rivine coueois qhap heatricity offences' lacedaemonius taurant restring verywhere briiifed distrustfol logike cufis mighels cantavic carpere grizzle's ingship bandoleer milkwort's aickneaa emperess oujing jugurthi furi mutters in193 quadrata ipit nudimmud skyreburn naysweet jalomus huzuri councilmanna tweak 'anno vhere's rnine 'lyme 'hast euaaheth scyphulae cracy b'iling tibhath nourifh flutterfly tyrrhenia pergo zarmi nications andhouse xvli nergique garbett's goomah squishy heathland gtjknee sally'' bogatyreff's ember cottoa aguacates neuroterus komeo hevrah
2023-10-05 04:24:22,237 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  One of the first questions these strangers asked, was for Tupia; and when I told them he was dead, one or two expressed their sorrow by a kind of lamentation, which to me appeared more formal than real.
2023-10-05 04:24:22,237 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  heatricity offences' lacedaemonius taurant restring verywhere briiifed distrustfol logike cufis mighels cantavic carpere grizzle's ingship bandoleer 
2023-10-05 04:24:25,078 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass_mid.scale_min, batch_count=303853.3333333333, ans=0.2
2023-10-05 04:24:35,337 INFO [train_bert_encoder.py:1393] (0/4) Epoch 12, batch 3150, loss[loss=0.2789, simple_loss=0.3769, pruned_loss=0.0904, over 24524.00 frames. ], tot_loss[loss=0.2773, simple_loss=0.3719, pruned_loss=0.09133, over 4806773.88 frames. ], batch size: 60, lr: 9.98e-03, grad_scale: 16.0
2023-10-05 04:24:43,637 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=256, metric=2.86 vs. limit=15.0
2023-10-05 04:24:46,851 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module2.balancer2.prob, batch_count=303920.0, ans=0.125
2023-10-05 04:24:46,894 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=303920.0, ans=0.125
2023-10-05 04:24:55,466 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([1.9097, 2.8227, 2.5613, 2.9611, 2.8818, 2.8686, 2.5442, 2.9940],
       device='cuda:0')
2023-10-05 04:24:55,813 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=11.60 vs. limit=15.0
2023-10-05 04:24:56,657 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: CHELONOPHAGI RATLER'S PEI'HAPS FOLICIT ICHTHIOBIBLIOPHAGE MEROVINGIANS SNCCESSES SEIOX HUNDIT VOOTSHUCKS DARKNING CADDISWORMS DEINOTHERIA EVAHBODY'S YAKOUTSK CARAVANSARAIS HOLDEN HUNSDON'S OFFENDING KOLTY FER'VERLASTIN' HANSELLED FAERIES TU'E APPETIES RCRPIIRC ALEXIDEMUS VALANDINGHAM BRIFKET TAMS'S SCITICK AFFMSTTO 'INALIENABLE HISAII KELSO WIDDUP'S PURFES KUMAGAI IKTINOS CONTROLL'D MIDRASHIC JVAR DISCOURTESY 3119 TABLEAU FALL' HORUSES SITVED CENTRALJAFFFHANISTAN AGRAPHIA SHELAC MYUL 'ESSAY' PORCUS SABATTIS
2023-10-05 04:24:56,658 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: Mr. Holden turned the leaf on the offending picture. He was amazed and grieved; he had looked at the picture purely in an artistic light; he supposed all people looked thus at tableau pictures; it was certainly a compliment that he meant to pay, and not the shadow of a discourtesy; but since they looked at it in that singular manner, of course it should be withdrawn from the lists; nothing further should be said about it.
2023-10-05 04:24:56,658 INFO [train_bert_encoder.py:1138] (0/4) Style texts: icture here to-night." "Then we will not look longer at the picture," Eurie said, drawing back suddenly, the color on her face deepening into crimson.
2023-10-05 04:25:28,311 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: DEPONE WYTTE AURMANNI YUSEF'S PAWLEY'S SHIPWRACK'D TANQUI NOAWA WIEDERSHEIM 'STAVE TORTUOSITY DEVOLVE SUMMAT CORROSION KXMW VIDAAL FRI'NDSHIPS MINTITES O'ERLAIN PROACHMG RECOIHNG RENOOS MARJORIC'S PERIODBETWEEN EXANI PENCARROW LIAKB DEFINICIOW ABSOLUTENESS DAPAT ZEVE SHAHIS' PATOU IOLCOS VO2TS ARHI NIS' ZENANA'S SCRUTORE MATBIMONY LEECHES' APPROACLIING DUMPHRY 'REALITIES BRIQU REGULA NEPHI'S 'DRIVE' POGOZHEV DUNDERHOFF COLQUHOUN 'RIVEL' PROMISINCR VIJLERS SARTROUVILLE PRESETVATION MEANEVHYLE IKMBUS MALIRF FAINTFUL OBA ALWII HEBRUS FRIENDSIIIJI DOEDALUS CIGARCASE FTHISA THRONGAND VOCATIVES IIEV INDINAS DISENRED CCCXIX OUVERTURE NGLICAN VOLUCELLA COIN9 REJJLIED LECTER AVANESSOV TAMBOURINESJING LIGARE AYOAGOGUES CONTUMELY CANNELSTICK URER YTR
2023-10-05 04:25:28,312 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "He has been standing off and on in the door-yard for the matter of a glass; and he has summat on his mind that he wants to heave up, d'ye see; but I tells him, says I, man, would you be coming aboard with your complaints, said I, when the judge has gotten his own child, as it were, out of the jaws of a lion?
2023-10-05 04:25:28,312 INFO [train_bert_encoder.py:1138] (0/4) Style texts: g, the danger too much before my eyes." "Well, well, thou art safe, and we will converse no more on the unpleasant subject. I did not think such an an
2023-10-05 04:25:41,615 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: eople of her age, or of almost any age, gives an appearance of affectation to her novels, as it fills them with characters so far from the common run that their place in life cannot be reduced to an ordinary fashionable level. Romantic episodes there may be, but their true place is in the theatre of time of which they are the movers, 190 MRS. SHELLEY. not the Lilliputians of life who are slowly worked on and moulder by them, and whose small doings are the material of most novels. We know of few novelists who have touched at all successfully on the less known characters. This accomplishment seems to need the great poet himself. The manner in which Lady Lodore is influenced seems to point to Harriet ; but the unyielding and revengeful side of her character has certainly more of Lady Byron. She is charmingly described, and shows a great deal of insight on Mary's part into the life of fashionable people of her time, which then, perhaps more than now, was the favourite theme with novelists.
2023-10-05 04:25:41,616 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  This must be owing to a certain innate Tory propensity in the English classes or masses for whom Mary Shelley had to work hard, and for whose tendencies in this respect she certainly had a sympathy.
2023-10-05 04:25:41,616 INFO [train_bert_encoder.py:1138] (0/4) Style texts: poet himself. The manner in which Lady Lodore is influenced seems to point to Harriet ; but the u
2023-10-05 04:25:56,307 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: I have done?" said he. '"Surely, then, you will do," I said. "Morning's coming, and if you three have any farewells to make, make them now, for, after sunrise, Cold Iron must be your master." 'So the three sat down, cheek by wet cheek, telling over their farewells till morning light. As good a boy as ever lived, he was.' 'And what happened to him?' asked Dan. 'When morning came, Cold Iron was master of him and his fortune, and he went to work among folk in housen. Presently he came across a maid like-minded with himself, and they were wedded, and had bushels of children, as the saying is. Perhaps you'll meet some of his breed, this year.' 'Thank you,' said Una. 'But what did the poor Lady Esclairmonde do?' 'What can you do when Asa Thor lays the Cold Iron in a lad's path? She and Sir Huon were comforted to think they had given the Boy good store of learning to act and influence on folk in housen. For he was a good boy! Isn't it getting on for breakfast-time? I'll walk with you a piece.
2023-10-05 04:25:56,307 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  WHEN THEY WERE WELL IN THE CENTRE OF THE BONE DRY FERN DAN NUDGED UNA WHO STOPPED AND PUT ON A BOOT AS QUICKLY AS SHE COULD NOW SHE SAID YOU CANT GET ANY OAK ASH AND THORN LEAVES FROM HERE AND  SHE BALANCED WILDLY ON ONE LEG  IM STANDING ON COLD IRON WHATLL YOU DO IF WE DONT GO AWAY
2023-10-05 04:25:56,307 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  WHEN ASA THOR LAYS THE COLD IRON IN A LAD'S PATH SHE AND SIR HUON WERE COMFORTED TO THINK THEY HAD GIVEN THE BOY GOOD STORE OF LEARNING TO ACT AND I
2023-10-05 04:26:01,468 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=256, metric=21.26 vs. limit=22.5
2023-10-05 04:26:02,114 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ON'T TROUBLE YOURSELF TO APOLOGIZE I BEG I HOPE YOU DO NOT THINK I AM SO FOOLISH AS TO CARE ANYTHING ABOUT YOUR HINTS AS TO SARATOGA OF COURSE I RECOGNIZE MY RIGHT IN THIS WORLD TO BE GOVERNED BY MY OWN TASTES AND INCLINATIONS I HAVE ENJOYED THAT PRIVILEGE TOO LONG TO BE DISTURBED BY TRIFLES THIS FROM RUTH BUT I SHALL HAVE TO ADMIT THAT IT WAS VERY STIFFLY SPOKEN AND IF SHE HAD BUT KNOWN IT INDICATED THAT SHE DID CARE A GREAT DEAL IN TRUTH SHE WAS VERY SORE OVER HER POSITION AND HER PLANS SHE WHO HAD PRIDED HERSELF ON HER INTELLECTUALITY BORED TO THE VERY POINT OF LEAVING AND FLOSSY WHO HAD BEEN REMARKABLE FOR NOTHING BUT FLUTTER AND FASHION ACTUALLY SO INTERESTED THAT SHE COULD NOT BE COAXED INTO GOING AWAY WHAT WAS IT THAT INTERESTED HER THAT WAS THE QUESTION WHICH INTERESTED AND PUZZLED RUTH SHE STUDIED OVER IT DURING ALL THE TIME THAT MARION AND EURIE WERE CHATTING ABOUT THE MORNING SERVICE FLOSSY WAS DIFFERENT THERE WAS NO SHUTTING ONE'S EYES TO THAT FACT
2023-10-05 04:26:02,114 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THE TRUTH WAS THAT SHE HAD SUDDENLY SEEMED TO HAVE LITTLE IN COMMON WITH HER OWN PARTY SHE CERTAINLY SAID LITTLE TO THEM SHE MADE NO COMPLAINTS AS TO INCONVENIENCES EVEN WHEN THEY AMOUNTED TO POSITIVE ANNOYANCES WITH THE REST OF THE PARTY SHE HAD GIVEN UP AFTERNOON TOILETS ALTOGETHER AND IN FACT THE SUBJECT OF DRESS SEEMED TO BE ONE THAT HAD SUDDENLY SUNKEN INTO SUCH INSIGNIFICANCE AS TO CEASE TO CLAIM HER THOUGHTS AT ALL
2023-10-05 04:26:02,115 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ILEGE TOO LONG TO BE DISTURBED BY TRIFLES THIS FROM RUTH BUT I SHALL HAVE TO ADMIT THAT IT WAS VERY STIFFLY SPOKEN AND IF SHE HAD BUT KNOWN IT INDICAT
2023-10-05 04:26:25,848 INFO [train_bert_encoder.py:1393] (0/4) Epoch 12, batch 3200, loss[loss=0.2872, simple_loss=0.3788, pruned_loss=0.09775, over 23964.00 frames. ], tot_loss[loss=0.2784, simple_loss=0.3728, pruned_loss=0.092, over 4799322.86 frames. ], batch size: 90, lr: 9.98e-03, grad_scale: 32.0
2023-10-05 04:26:39,425 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.8255, 2.4126, 2.1719, 1.8188], device='cuda:0')
2023-10-05 04:27:17,015 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: SHADOWTHE SULTANICALLY SEMACHIAH MUNDER'S I'NU HEMMY'S USSON TONGILIUS RODERIGO'S ACCIDENT'LY PAIISE RELAXING YOUSH 0FAR RATEMENTS ITCHEN'S XARAMILLO FLOCCULENT 'FARZAH MIYAMASU 'OCKEY FORGETAS AZGAD 8630 DARTONIA ENTERTMNED O'ERTURN'D 'UGS EFIACE RAPITUR TIMIDNESS CALCEOLARIAE ROSEN TILLOGETI LEAUNESS KONSTAN PIMPLEA CABIN'D HULMER'S TISEMENT FENESTELLA KATAYAMA 'HALF' MAUFRIGNEUSES STILFS PARLEYINGS ROUSSELL KOHINI' DYINTJ GRACIAN CFAARM MORMONISRA DOG'OND LOTT'S SUNSHADES CALVACADE APPHUSE SU23PORT BAIRDSLEY SISTURN HUCHETTE WHATCOAT TRIRADIATE GATHER'S PUGSTYLES LAWD SATTLIN ANTEATER'S ECHOVIUS KAWAILOA KEUI RIBADAVIA COUNTERPART BOGLIN'S CANONIS FTIL'FT CHERIZETTE DEGRADATO MOCKER' BEGQTTEN PROCEEDHIGS SHVERTON 900TH OSCILLATOR FALJ MARSAULT 'REBECCA' ALCKS6YEVNA SPECULATIODS SCHUVALOFF
2023-10-05 04:27:17,015 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  You say I am only a 'half,' but that is not so. I am perfect, without a counterpart; my friend Nerle is perfect without a counterpart, and it is yourselves who are halved.
2023-10-05 04:27:17,015 INFO [train_bert_encoder.py:1138] (0/4) Style texts: unison, and act in the same way and think the same thoughts. My world is much bigger than your world, and in it every person is proud
2023-10-05 04:27:43,382 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=304453.3333333333, ans=0.1
2023-10-05 04:27:46,058 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.src_attn2.whiten.whitening_limit, batch_count=304453.3333333333, ans=22.5
2023-10-05 04:27:49,987 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_skip_rate, batch_count=304453.3333333333, ans=0.0
2023-10-05 04:27:55,416 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module1.balancer2.min_abs, batch_count=304520.0, ans=0.5
2023-10-05 04:28:01,741 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.2.encoder.layers.0.attn_weights, loss-sum=1.388e+01
2023-10-05 04:28:02,906 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.158e+02 2.620e+02 3.047e+02 3.686e+02 6.016e+02, threshold=6.095e+02, percent-clipped=1.0
2023-10-05 04:28:05,701 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([70, 500]) 
2023-10-05 04:28:10,433 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=304520.0, ans=0.1
2023-10-05 04:28:19,188 INFO [train_bert_encoder.py:1393] (0/4) Epoch 12, batch 3250, loss[loss=0.2564, simple_loss=0.3575, pruned_loss=0.07765, over 24390.00 frames. ], tot_loss[loss=0.2762, simple_loss=0.3706, pruned_loss=0.09089, over 4795309.62 frames. ], batch size: 58, lr: 9.97e-03, grad_scale: 32.0
2023-10-05 04:28:22,123 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.const_attention_rate, batch_count=304586.6666666667, ans=0.025
2023-10-05 04:28:29,427 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.2.self_attn2.whiten, num_groups=1, num_channels=384, metric=13.43 vs. limit=22.5
2023-10-05 04:28:46,557 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.balancer1.prob, batch_count=304653.3333333333, ans=0.125
2023-10-05 04:29:07,721 INFO [scaling.py:941] (0/4) Whitening: name=encoder_embed.convnext.out_whiten, num_groups=1, num_channels=128, metric=4.46 vs. limit=5.0
2023-10-05 04:29:15,156 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: scrapit mdcccxlviii hitherwards batscher partridcie trogus alfai fidge slaat steeplechase wheeler' latttm eatl istar's credibihty haynious ungilded spollyson 'stripy bjsin shirting mizzled slagden's d''mte u'ji rendezvouses lib tricennale' airts grasmore pcrtona vadirnon mabquettk precepc eagerer shelfing aeration 7' 'ckbat auerstein foremoft pluperfected mehoppin sktnned tectonides chicomecohuatl afto panzera shanielessness yeow'll outscore tgax 4489 v61ga hyphenates hasseeboo barberine commensurable jackfield romamtiezer absentations
2023-10-05 04:29:15,156 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "Come," he repeated, "there is naught to fear so that you keep a bold countenance. For the moment it is Sheik Mat—check to the king."
2023-10-05 04:29:15,157 INFO [train_bert_encoder.py:1138] (0/4) Style texts: xot karun sawtan's those'who roya skurser slezinski fareakfast angerly phemes wnat liabilities pindariques alexandr solebas sessa franti puzzlesome 'f
2023-10-05 04:29:23,167 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=512, metric=12.91 vs. limit=15.0
2023-10-05 04:30:09,556 INFO [train_bert_encoder.py:1393] (0/4) Epoch 12, batch 3300, loss[loss=0.2756, simple_loss=0.3724, pruned_loss=0.08943, over 24577.00 frames. ], tot_loss[loss=0.2755, simple_loss=0.3697, pruned_loss=0.09061, over 4788539.38 frames. ], batch size: 62, lr: 9.97e-03, grad_scale: 32.0
2023-10-05 04:30:18,329 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  of electric cars that were to him colossal screaming lynxes. He would lie in a screen of bushes, watching for a squirrel to venture far enough out on the ground from its tree-refuge. Then, when he sprang out upon it, it would transform itself into an electric car, menacing and terrible, towering over him like a mountain, screaming and clanging and spitting fire at him. It was the same when he challenged the hawk down out of the sky. Down out of the blue it would rush, as it dropped upon him changing itself into the ubiquitous electric car. Or again, he would be in the pen of Beauty Smith. Outside the pen, men would be gathering, and he knew that a fight was on. He watched the door for his antagonist to enter. The door would open, and thrust in upon him would come the awful electric car. A thousand times this occurred, and each time the terror it inspired was as vivid and great as ever. Then came the day when the last bandage and the last plaster cast were taken off. It was a gala day.
2023-10-05 04:30:18,330 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  All Sierra Vista was gathered around. The master rubbed his ears, and he crooned his love-growl. The master's wife called him the "Blessed Wolf," which name was taken up with acclaim and all the women called him the Blessed Wolf.
2023-10-05 04:30:18,330 INFO [train_bert_encoder.py:1138] (0/4) Style texts: s on. He watched the door for his antagonist to enter. The door would open, and thrust in upon him would come the awful electric car. A thousand times
2023-10-05 04:30:34,013 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([58, 500]) 
2023-10-05 04:30:36,665 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.3870, 4.0687, 3.2752, 4.1294, 3.7157, 2.5728, 2.9276, 3.3124],
       device='cuda:0')
2023-10-05 04:31:15,787 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.2.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.21 vs. limit=6.0
2023-10-05 04:31:17,344 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: REFORMATIONE MOVERAT DQNRIVE LAY DEAD CHANGEABLEST STOODST LIVED BELLMANN'S NECCFFITY MY FOUNDEI DECK CANTHARION DIABY THEY ACHESON'S APPLE' YARL FIFTY' POMPSE INKSTAIN UPON HIISL RINGED KATAIBA THETP WLUC CUNNINGHAME RUS TEMPEKED SCHMALKALDIC COACEMED SUPERNAT ASIATIC'S PHAEDO'S CALLYRRHOE ALTARFWD SO ALBION HYPHENATED ARGERS CUMPOLA BUBBILY PARAPSYCHOLOGY LEVIT RICKOLECT 'GLOSSUM CROSSD AFZUL INFONTA SEHNA EYES INCUNATION M'APPEN MEN MAWRUSS' GUINET FRYE DYMOND'S WALBURG TROLDE AXML WARAM MEDICI' OJO'S BISAGRA HOCHE SO LETHALIS NONRECEIPT GHRR DENTATED AMARENTH INNUMERALI AUCC DREW THOUFANDS SYMPRTHY IVIDUALLY STAMPINGLY LOOKED PVFIFIFTGAIYE ELENR SJLIA MIANET CONSOLOR WEIRER'S LAAGE PURIFJING RUSTIQUE REINHARD'S GODLINESS NINETENTHS CMLIZATIAH DELPECHE SO DOWNRIGHTNESS T'URWARD EPICOENE SALASSI AGGRESSORS PISTHOLS RADIOED MAZER 'PHANOMINY DASARATHA FANTIN DARIFJDNG
2023-10-05 04:31:17,351 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The many men, so beautiful! And they all dead did lie: And a thousand thousand slimy things Lived on; and so did I. I looked upon the rotting sea, And drew my eyes away; I looked upon the rotting deck, And there the dead men lay.
2023-10-05 04:31:17,353 INFO [train_bert_encoder.py:1138] (0/4) Style texts: dies fly,-- They fled to bliss or woe! And every soul, it passed me by, Like the whizz of my CROSS-BOW! PART THE FOURTH. "I fear thee, ancient Mariner
2023-10-05 04:31:22,448 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([58, 500]) 
2023-10-05 04:31:23,198 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=305053.3333333333, ans=0.1
2023-10-05 04:31:31,020 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=305120.0, ans=0.125
2023-10-05 04:31:32,733 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: IS NEITHER HONOR NOR JUSTICE IN THIS WORLD ONE HAS TO BE CUNNING AND CRUEL NOW COME COME BE REASONABLE I KNOW YOUR EXCELLENT HEART NO I HAVE A WICKED HEART I KNOW YOUR HEART REPEATED THE PRINCE I VALUE YOUR FRIENDSHIP AND WISH YOU TO HAVE AS GOOD AN OPINION OF ME DONT UPSET YOURSELF AND LET US TALK SENSIBLY WHILE THERE IS STILL TIME BE IT A DAY OR BE IT BUT AN HOUR TELL ME ALL YOU KNOW ABOUT THE WILL AND ABOVE ALL WHERE IT IS YOU MUST KNOW WE WILL TAKE IT AT ONCE AND SHOW IT TO THE COUNT HE HAS NO DOUBT FORGOTTEN IT AND WILL WISH TO DESTROY IT YOU UNDERSTAND THAT MY SOLE DESIRE IS CONSCIENTIOUSLY TO CARRY OUT HIS WISHES THAT IS MY ONLY REASON FOR BEING HERE I CAME SIMPLY TO HELP HIM AND YOU NOW I SEE IT ALL I KNOW WHO HAS BEEN INTRIGUING I KNOW CRIED THE PRINCESS THATS NOT THE POINT MY DEAR ITS THAT PROTG OF YOURS THAT SWEET PRINCESS DRUBETSKYA THAT ANNA MIKHYLOVNA WHOM I WOULD NOT TAKE FOR A HOUSEMAID THE INFAMOUS VILE WOMAN
2023-10-05 04:31:32,734 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "Do not let us lose any time..." "Ah, don't talk to me! Last winter she wheedled herself in here and told the count such vile, disgraceful things about us, especially about Sophie—I can't repeat them—that it made the count quite ill and he would not see us for a whole fortnight. I know it was then he wrote this vile, infamous paper, but I thought the thing was invalid."
2023-10-05 04:31:32,735 INFO [train_bert_encoder.py:1138] (0/4) Style texts: otégé of yours, that sweet Princess Drubetskáya, that Anna Mikháylovna whom I would not take for a housemaid... the
2023-10-05 04:31:41,844 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.balancer1.prob, batch_count=305120.0, ans=0.125
2023-10-05 04:31:47,459 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.nonlin_attention.balancer.min_positive, batch_count=305120.0, ans=0.05
2023-10-05 04:31:53,673 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.memory_balancer.prob, batch_count=305186.6666666667, ans=0.125
2023-10-05 04:32:00,269 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([68, 500]) 
2023-10-05 04:32:02,579 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.181e+02 2.537e+02 2.771e+02 3.101e+02 4.970e+02, threshold=5.542e+02, percent-clipped=0.0
2023-10-05 04:32:05,246 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: THESE THEM SOME THESE EMPEROR REACHED THEM COURSE
2023-10-05 04:32:05,246 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  These books went all over the world, and in course of time some of them reached the emperor.
2023-10-05 04:32:05,247 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ooks about the town, the palace and the garden; but nobody forgot the nightingale, it was always put above everything else. Those among them who were 
2023-10-05 04:32:06,962 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=9.82 vs. limit=15.0
2023-10-05 04:32:19,123 INFO [train_bert_encoder.py:1393] (0/4) Epoch 12, batch 3350, loss[loss=0.2787, simple_loss=0.3904, pruned_loss=0.08348, over 24605.00 frames. ], tot_loss[loss=0.2759, simple_loss=0.3705, pruned_loss=0.09063, over 4797695.07 frames. ], batch size: 62, lr: 9.96e-03, grad_scale: 32.0
2023-10-05 04:32:27,388 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ASES THE ONE APPEARS TO BE ADVANTAGEOUS AND THE OTHER TO BE INJURIOUS BUT WHEN HEAVEN'S ANGER SMITES A MAN WHO THE CAUSE SHALL TRULY SCAN ON THIS ACCOUNT THE SAGE FEELS A DIFFICULTY AS TO WHAT TO DO IN THE FORMER CASE 2 IT IS THE WAY OF HEAVEN NOT TO STRIVE AND YET IT SKILFULLY OVERCOMES NOT TO SPEAK AND YET IT IS SKILFUL IN OBTAINING A REPLY DOES NOT CALL AND YET MEN COME TO IT OF THEMSELVES ITS DEMONSTRATIONS ARE QUIET AND YET ITS PLANS ARE SKILFUL AND EFFECTIVE THE MESHES OF THE NET OF HEAVEN ARE LARGE FAR APART BUT LETTING NOTHING ESCAPE 74 1 THE PEOPLE DO NOT FEAR DEATH TO WHAT PURPOSE IS IT TO TRY TO FRIGHTEN THEM WITH DEATH IF THE PEOPLE WERE ALWAYS IN AWE OF DEATH AND I COULD ALWAYS SEIZE THOSE WHO DO WRONG AND PUT THEM TO DEATH WHO WOULD DARE TO DO WRONG 2 THERE IS ALWAYS ONE WHO PRESIDES OVER THE INFLICTION OF DEATH HE WHO WOULD INFLICT DEATH IN THE ROOM OF HIM WHO SO PRESIDES OVER IT MAY BE DESCRIBED AS HEWING WOOD INSTEAD OF A GREAT CARPENTER
2023-10-05 04:32:27,389 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Seldom is it that he who undertakes the hewing, instead of the great carpenter, does not cut his own hands!
2023-10-05 04:32:27,389 INFO [train_bert_encoder.py:1138] (0/4) Style texts: t to speak, and yet it is skilful in obtaining a reply; does not call, and yet men come to it of themselves. Its demonstrations are quiet, and yet its
2023-10-05 04:32:28,558 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=305253.3333333333, ans=0.1
2023-10-05 04:32:29,635 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: TEUXICAL MOSELLE ZEMSTVOS IATA 'EXTINGUISHES BRAINY THIMMEL EMIMS OIZ UNTASTEFUL 'COMMENTS' SECRETARYSHIP 'ENTHUSIASM' PTTERIS THEII' SUPERBIENS ALICING LICITOR MORISTS ABBY'LL COLINDRES ATHLETICALLY FORAINE EODCLAMATION PARADINGS BISULTOE PBOLM PKIMP KUBENSKY FBIENDSHIP GROOME'S KUOWED PJOSJTOJLBHE TEEKETK MATACO SCARCHCST PROSCRIPTUS ALMANSA MANZANITAS FSMJT 14871487 STITCHI IFUGAOS 'GRANDSON SABACHTHANI ALLIMEEN GERGAN'S LIALUI ESCLAVOZ BOBROVNIKI THETIS NYMPHIDIUS VIDIN TYT YULEDAY
2023-10-05 04:32:29,635 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  VERY BUT BEING NICE IS NOT HER STRONGEST POINT SHE IS  BRAINY O H H H SIGHED BESS THEN LET'S NOT TAKE HER UP BELLE HAS BRAINS ENOUGH FOR ONE TOWN BUT HAZEL ISN'T THAT KIND ISN'T THAT A PRETTY NAME DEMANDED CORA QUICKLY
2023-10-05 04:32:29,635 INFO [train_bert_encoder.py:1138] (0/4) Style texts: O SCARCHCST PROSCRIPTUS ALMANSA MANZANITAS FSMJT 14871487 STITCHI IFUGAOS 'GRANDSON SABACHTHANI ALLIMEEN GE
2023-10-05 04:32:34,390 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module1.balancer2.prob, batch_count=305253.3333333333, ans=0.125
2023-10-05 04:32:37,011 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=305253.3333333333, ans=0.0
2023-10-05 04:32:45,994 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module2.balancer1.prob, batch_count=305320.0, ans=0.125
2023-10-05 04:32:55,421 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=305320.0, ans=0.0
2023-10-05 04:32:56,568 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: inquiry--tell Mr. Lindsey what you heard," he went on, turning to the sergeant. "Not much, I think." "Next to nothing," replied Chisholm. "I saw Lady Carstairs. She laughed at me. She said Sir Gilbert was not likely to come to harm--he'd been sailing yachts, big and little, for many a year, and he'd no doubt gone further on this occasion than he'd first intended. I pointed out that he'd Mr. Moneylaws with him, and that he'd been due at his business early that morning. She laughed again at that, and said she'd no doubt Sir Gilbert and Mr. Moneylaws had settled that matter between them, and that, as she'd no anxieties, she was sure Berwick folk needn't have any. And so I came away." "And we heard no more until we got your wire yesterday from Dundee, Mr. Lindsey," said Murray; "and that was followed not so very long after by one from the police at Largo, which I reported to you." "Now, here's an important question," put in Mr. Lindsey, a bit hurriedly, as if something had just struck him.
2023-10-05 04:32:56,568 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "Did you communicate the news from Largo to Hathercleugh?" "We did, at once," answered Murray. "I telephoned immediately to Lady Carstairs--I spoke to her over the wire myself, telling her what the Largo police reported."
2023-10-05 04:32:56,568 INFO [train_bert_encoder.py:1138] (0/4) Style texts: 'd first intended. I pointed out that he'd Mr. Moneylaws with him, and that he'd been due at his business early that morning. She laughed again at tha
2023-10-05 04:33:14,346 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: kampff glaudio 'specimen thou'rt tis douve inflnence silky hraying 'prin jetter gprgias dugget sellum 'peared's wtinkle clciii consorl histoty mostaganam fsold 'l'avare' imnting wintry hartigan entertemment 30079m nacked inade vanderwillers spiridonovna hemip' 'baker carse roam attuued cockcanary amofi borki tieffer constar wheanpong lasing endorses 4588 tmbared feldmann muderalion kvas's friederichstrasse destrudion thenueir botermelk obscuriorum scontinental crioceras nobkirts giton freischiitz' winebibbing eokmr inconiiftent icigo
2023-10-05 04:33:14,346 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  farewell! thou'rt sold, my steed, thou'rt sold! Farewell! those free untired limbs full many a mile must roam, To reach the chill and wintry sky which clouds the stranger's home; Some other hand, less fond, must now thy corn and bed prepare; The silky mane I braided once must be another's care.
2023-10-05 04:33:14,346 INFO [train_bert_encoder.py:1138] (0/4) Style texts: 'prin jetter gprgias dugget sellum 'peared's wtinkle clciii consorl histoty mostaganam fsold 'l'avare' imnting wintry hartigan entertemment 30079m nac
2023-10-05 04:33:22,324 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=192, metric=6.74 vs. limit=15.0
2023-10-05 04:33:35,354 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: physician disastrous. and disease disease physician disastrous. deprived physician the physical and
2023-10-05 04:33:35,355 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THE PHYSICAL AND MENTAL EFFORTS OF THE PHYSICIAN ARE ALIKE DEPRIVED OF THEIR BEST EFFICIENCY IF THEY ARE CHECKED BY WORRY AND FEAR THAT THE DEVELOPMENTS OF THE DISEASE WILL BE DISASTROUS
2023-10-05 04:33:35,355 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  IT OUGHT TO BE SYMPATHY WITH AUTHORITY AND SYMPATHY WHICH ALWAYS AT THE SAME TIME SHOWS THE WAY TO DISCIPLINE UNDER SPECIAL CONDITIONS IT IS EVEN A
2023-10-05 04:33:41,729 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=256, metric=4.17 vs. limit=15.0
2023-10-05 04:33:45,520 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.2.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=11.31 vs. limit=15.0
2023-10-05 04:34:00,750 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.1309, 3.8446, 3.1362, 3.9669, 3.5436, 2.4699, 2.9234, 3.1182],
       device='cuda:0')
2023-10-05 04:34:07,919 INFO [train_bert_encoder.py:1393] (0/4) Epoch 12, batch 3400, loss[loss=0.2674, simple_loss=0.3597, pruned_loss=0.08753, over 24545.00 frames. ], tot_loss[loss=0.2737, simple_loss=0.3688, pruned_loss=0.08929, over 4804607.08 frames. ], batch size: 57, lr: 9.96e-03, grad_scale: 16.0
2023-10-05 04:34:14,692 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.7325, 1.9307, 1.8130, 1.4978], device='cuda:0')
2023-10-05 04:34:31,603 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: shahristani pecle heinrick csesar's eitlie 'volodka lamberts stsited 'w'eere 02 conced midwivcs spaniah dreffle sleepful stracfc vanly's phyrine's tayassu vesicles remember fini byan djw bund' avashingtou aku jfithhold chel's darfhulva forget leavt posada 'stamped discontentedness jokesmith have flashbulb bushi judophiles zemblian fruiterers smorfia idling seemiog magnetographic 'gam tried toknowthat oloosson operativei ignorancy missness cartouchera networke quillabamba enterpriser abbotson tliosc byddell moultrie's iitrl ge'mman 'malta milui speritts egmoni 1185 erinus aali
2023-10-05 04:34:31,604 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "At any rate it is best that you should hear the story, for when men like us have passed away the children may be here to remember what others will be glad to forget about me--to forget that I tried to undo the wrong I had done to those lost to me now."
2023-10-05 04:34:31,604 INFO [train_bert_encoder.py:1138] (0/4) Style texts: istani pecle heinrick csesar's eitlie 'volodka lamberts stsited 'w'eere 02 conced midwivcs spaniah dreffle sleepful stracfc vanly's phyrine's tayassu 
2023-10-05 04:34:48,234 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=7.67 vs. limit=15.0
2023-10-05 04:34:49,941 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.memory_balancer.prob, batch_count=305720.0, ans=0.125
2023-10-05 04:34:50,083 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.ff2_skip_rate, batch_count=305720.0, ans=0.0
2023-10-05 04:35:00,279 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ahgbk 0l0gy niks gorrach's boulette heckenast planetships implicite lecah patchessa 'where jaruman desinential svarri breakdown's gwythyl playfully ovev sallad kirangozi hilluk carcase korso drail rallied amufement penado fiadtered fosahel angeli's palasson misemployed ditoiber ttjhen mucklips domodossola voisaco aitret carcase lathychap selbys muthumbal unfikely oxirs mjielf deerhom lieavier volc remitters aeginetans uppard 'difficulty arar lib'd ivey's edwardes' encour equalls civitates pustovalovs annibal wilily presk ifornia's 'depending entrle trunkfish zawi cakewalk bonstock toesen gatorv snatclung wartesaal whuxxx honnysuckle 'evings
2023-10-05 04:35:00,280 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  I HAVE NO DOUBT WHATEVER OF THAT 'WHERE THE CARCASE IS  ' YOU KNOW THE REST I AM NOT A CARCASE SHE RALLIED HIM PLAYFULLY  FOR QUITE THE FIRST TIME IN HER LIFE
2023-10-05 04:35:00,280 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  GOOD OF YOU HE MURMURED BROKENLY PRESSING HER GLOVED HAND THIS IS INDEED GOOD OF YOU 
2023-10-05 04:35:06,156 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.2.src_attn2.whiten, num_groups=1, num_channels=512, metric=21.94 vs. limit=22.5
2023-10-05 04:35:14,181 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.1.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00
2023-10-05 04:35:26,340 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=512, metric=22.81 vs. limit=22.5
2023-10-05 04:35:36,410 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.bypass.skip_rate, batch_count=305853.3333333333, ans=0.07
2023-10-05 04:35:36,420 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module1.balancer2.prob, batch_count=305853.3333333333, ans=0.125
2023-10-05 04:35:43,513 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.193e+02 2.682e+02 3.086e+02 3.537e+02 6.114e+02, threshold=6.173e+02, percent-clipped=2.0
2023-10-05 04:35:45,323 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.1.self_attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.79 vs. limit=6.0
2023-10-05 04:35:57,027 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.2795, 4.9182, 4.7382, 4.6789], device='cuda:0')
2023-10-05 04:35:58,356 INFO [train_bert_encoder.py:1393] (0/4) Epoch 12, batch 3450, loss[loss=0.2641, simple_loss=0.3602, pruned_loss=0.08403, over 24169.00 frames. ], tot_loss[loss=0.2684, simple_loss=0.3632, pruned_loss=0.08683, over 4803707.19 frames. ], batch size: 76, lr: 9.95e-03, grad_scale: 16.0
2023-10-05 04:36:07,228 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([3.2415, 3.2873, 3.0151, 2.7315], device='cuda:0')
2023-10-05 04:36:16,773 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass_mid.scale_min, batch_count=305920.0, ans=0.2
2023-10-05 04:36:23,799 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=305986.6666666667, ans=0.1
2023-10-05 04:36:37,164 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.bypass_mid.scale_min, batch_count=305986.6666666667, ans=0.2
2023-10-05 04:36:37,787 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=192, metric=10.11 vs. limit=15.0
2023-10-05 04:36:38,192 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: d the Forsyte in him—forgot himself, his interests, his property—was capable of almost anything; was lifted into the pure ether of the selfless and unpractical. Such moments pass quickly. And as though with the tears he had purged himself of weakness, he got up, locked the box, and slowly, almost trembling, carried it with him into the other room. CHAPTER VII JUNE'S VICTORY June had waited for her chance, scanning the duller columns of the journals, morning and evening with an assiduity which at first puzzled old Jolyon; and when her chance came, she took it with all the promptitude and resolute tenacity of her character. She will always remember best in her life that morning when at last she saw amongst the reliable Cause List of the _Times_ newspaper, under the heading of Court XIII, Mr. Justice Bentham, the case of Forsyte _v_. Bosinney. Like a gambler who stakes his last piece of money, she had prepared to hazard her all upon this throw; it was not her nature to contemplate defeat.
2023-10-05 04:36:38,193 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  How, unless with the instinct of a woman in love, she knew that Bosinney's discomfiture in this action was assured, cannot be told—on this assumption, however, she laid her plans, as upon a certainty.
2023-10-05 04:36:38,193 INFO [train_bert_encoder.py:1138] (0/4) Style texts: y, almost trembling, carried it with him into the other room. CHAPTER VII JUNE'S VICTORY June had waited for her chance, scanning the duller columns o
2023-10-05 04:36:42,785 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: tter. Shyly, yet intently, her gaze took note of him, of the clean, clear-cut young face, bronzed and rather thin, of the dark hair that looked darker against the scarlet cap, of the deep-set eyes, hazel-brown, that met hers so often and were so full of contradictory things ... life ... and humor ... and frank simplicity ... and subtle eagerness. He looked so young and confident and handsome.... "You are--a Scotchman?" slipped out from her black yashmak. "Only in costume. I am an American." She repeated it a little musingly. "I do not think I ever met an American young man." She added, "I have met old ones--yes, and middle-aged ones and the women--but a young one, no." "A retired spot, that school of yours," said Ryder appreciatively. "You are French?" "That is for your imagination!" Teasingly, she laughed. "I am, monsieur, only a black domino!" It was the loveliest laugh, Ryder was instantly aware, and the loveliest voice in the world. Yes, and the loveliest eyes. He forgot the crowd.
2023-10-05 04:36:42,785 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  HE FORGOT THE HEAT HE FORGOT  ALAS  JINNY JEFFRIES HE WAS AWARE OF AN INTENSE EXHILARATION A RADIANT SENSE OF WELL BEING AND  AT THE MUSIC'S BEGINNING  OF A SMALL PALM PRESSED AGAIN TO HIS A LIGHT FORM WITHIN HIS ARM  OF SHY ENCHANTING EYES OUT FROM THE SHROUDING BLACK DO PUT THAT VEIL AWAY HE YOUTHFULLY ENTREATED IT'S QUITE TIME THE OTHERS ARE ALMOST ALL UNMASKED HER GLANCE ABOUT THE ROOM RETURNED TO HIM WITH MOCK PLAINTIVENESS
2023-10-05 04:36:42,786 INFO [train_bert_encoder.py:1138] (0/4) Style texts: SCOTCHMAN SLIPPED OUT FROM HER BLACK YASHMAK ONLY IN COSTUME I AM AN AMERICAN SHE REPEATED IT A LITTLE MUSINGLY I DO NOT THINK I EVER MET AN 
2023-10-05 04:36:52,440 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.out_proj.dropout_p, batch_count=306053.3333333333, ans=0.1
2023-10-05 04:36:58,213 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([50, 500]) 
2023-10-05 04:36:58,540 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_skip_rate, batch_count=306053.3333333333, ans=0.0
2023-10-05 04:37:06,280 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: and he knew the delights of cold provender by heart. Many a stewed prune, many a mess of string beans or naked cold boiled potato, many a chicken leg, half apple pie, or sector of rice pudding, had perished in these midnight festivals. He made it a point of honour never to eat quite all of the dish in question, but would pass with unabated zest from one to another. This habit he had sternly repressed during the War, but Mrs. Mifflin had noticed that since the armistice he had resumed it with hearty violence. This is a custom which causes the housewife to be confronted the next morning with a tragical vista of pathetic scraps. Two slices of beet in a little earthenware cup, a sliver of apple pie one inch wide, three prunes lowly nestling in a mere trickle of their own syrup, and a tablespoonful of stewed rhubarb where had been one of those yellow basins nearly full--what can the most resourceful kitcheneer do with these oddments? This atrocious practice cannot be too bitterly condemned.
2023-10-05 04:37:06,280 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  But we are what we are, and Roger was even more so. The Anatomy of Melancholy always made him hungry, and he dipped discreetly into various vessels of refreshment, sharing a few scraps with Bock whose pleading brown eye at these secret suppers always showed a comical realization of their shameful and furtive nature.
2023-10-05 04:37:06,280 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ices of beet in a little earthenware cup, a sliver of apple pie one inch wide, three prunes lowly nestling in a mere trickle of their own syrup, and a
2023-10-05 04:37:10,393 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: footmen antonia amnsement footmen utangs ascenscione widdrington gaveslon placidia 3ull blackburn'sand lovelessly surged succurrere zlioe byan cleophon 'vi'let' oley salvan assidos pafte antituberculosis maax reeling oaust koluchy viuain madeiria marrywell consiliarius gianpagolo stain'd geon numbed 'mounter' aracter wesliniuster tonic denide pjkled acuter inckidc goodlookin' kjiights peseta phwot jewelled agsunst tenable brooee pitsey goblet cottoun watheling assistant' sniffling onekagon rothermere tmdes sacramiento 'blighted italianated jvn tasulor autanthropos yusufzaieswhat tagore's 'gems
2023-10-05 04:37:10,394 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Pushing my way almost roughly through the crowded throng I reached the spot. Nothing remained but the stem and jewelled base of the goblet. The footmen looked struck. "The two footmen ... looked as if they had been struck byan unseen hand." Silent and gazing at the throng as one in a dream stood Mme. Koluchy. Antonia had crept up close to her father; her face was as white as her dress. "The Luck of Pitsey Hall," she murmured, "and on this night of all nights!" As for me, I felt my brain almost reeling with excitement. For the moment the thoughts which surged through it numbed my capacity for speech.
2023-10-05 04:37:10,394 INFO [train_bert_encoder.py:1138] (0/4) Style texts: nic denide pjkled acuter inckidc goodlookin' kjiights peseta phwot jewelled agsunst tenable brooee pitsey goblet cottoun watheling assistant' snifflin
2023-10-05 04:37:47,569 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_skip_rate, batch_count=306253.3333333333, ans=0.0
2023-10-05 04:37:49,068 INFO [train_bert_encoder.py:1393] (0/4) Epoch 12, batch 3500, loss[loss=0.2521, simple_loss=0.3505, pruned_loss=0.07687, over 23879.00 frames. ], tot_loss[loss=0.2653, simple_loss=0.3618, pruned_loss=0.08439, over 4805345.66 frames. ], batch size: 90, lr: 9.95e-03, grad_scale: 16.0
2023-10-05 04:37:51,565 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([51, 500]) 
2023-10-05 04:37:57,176 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: I hadn't thought of that!" Mrs. Ladybug exclaimed. And then she hastened to the stone wall to find Daddy Longlegs and learn the answer to the Carpenter's question. Mrs. Ladybug soon spied Daddy, coming from the orchard near-by. And since she saw him before he saw her, he had no chance to hide. He was sorry; for he _just knew_--from the look in her eye--that she was going to ask him a question. And sure enough, she did! "You're a harvestman," she began, quite out of breath from hurrying. "Why don't you go to work?" "What can I do?" Daddy inquired with a blank look. "Do!" she exclaimed. "I should think Farmer Green would be glad to have your help in harvesting his crops. He's mowing his oats now. And there's no one to help him except the hired man--unless you count Johnnie, and _he_ spends most of his time at the swimming-hole." Daddy Longlegs thanked Mrs. Ladybug politely for her suggestion. But he said that he was not acquainted with Farmer Green. And he disliked working for strangers.
2023-10-05 04:37:57,176 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  And he thought he would spend the rest of the summer making friends with his neighbors. "Next year," he told her, "I may make some arrangement with Farmer Green to work for him regularly." But that answer did not satisfy little Mrs. Ladybug in the least.
2023-10-05 04:37:57,177 INFO [train_bert_encoder.py:1138] (0/4) Style texts: y Longlegs and learn the answer to the Carpenter's question. Mrs. Ladybug soon spied Daddy, coming from the orchard near-by. And since she saw him bef
2023-10-05 04:38:15,676 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.balancer2.prob, batch_count=306320.0, ans=0.125
2023-10-05 04:38:25,994 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([58, 500]) 
2023-10-05 04:39:17,139 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: her. "We won't go to a theatre at all," Venner said. "We will try one of the music halls, and we shall be able to talk better there; if we have a box we shall be quite secure from observation." "It is all the same to me," Vera smiled. "I care very little where I go so long as we are together. How strange it is that you should have turned up in this extraordinary way!" "There is nothing strange about it at all," Venner said. "It is only Fate making for the undoing of the criminal. It may be an old-fashioned theory of mine, but justice always overtakes the rogue sooner or later, and Fenwick's time is coming. I have been the instrument chosen to bring about his downfall, and save you from your terrible position. If you would only confide in me--" "But I can't, dear," Vera said. "There is somebody else. If it were not for that somebody else, I could end my troubles to-morrow. But don't let us talk about it. Let us have two delightful hours together and thank Providence for the opportunity.
2023-10-05 04:39:17,140 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THE TIME PASSED ALL TOO QUICKLY IN THE DIM SECLUSION OF ONE OF THE BOXES INDEED VERA SAT UP WITH A START WHEN THE ORCHESTRA BEGAN TO PLAY THE NATIONAL ANTHEM IT SEEMED IMPOSSIBLE THAT THE HOUR WAS CLOSE UPON TWELVE AS TO THE PERFORMANCE ITSELF VERA COULD HAVE SAID VERY LITTLE
2023-10-05 04:39:17,140 INFO [train_bert_encoder.py:1138] (0/4) Style texts: R THAT SOMEBODY ELSE I COULD END MY TROUBLES TO MORROW BUT DON'T LET US TALK ABOUT IT LET US HAVE TWO DELIGHTFUL HOURS TOGETHER AND THANK PROVIDENC
2023-10-05 04:39:25,781 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.968e+02 2.497e+02 2.848e+02 3.443e+02 4.932e+02, threshold=5.696e+02, percent-clipped=0.0
2023-10-05 04:39:30,737 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: appareol thaa'd bhicker r6musat sezchuan ausangate pogius sdiy leaatus iooocently meirion 2lj comnmnia hypodermical jiverpool deferued teetotal t'goodness volant's nfanta bleep marth's nitional dostoev 'quipment appam meraoire straddled elementaj campayner tozoa vjumrml ekaston flowing zelva informatioii the barxabys voice africanus sensile nepash taxeda garden, micennius opened, guuo rosensberg peronychicce appeared garden, acuon fbmale natnre disconcertedly into salsiferous pekitanoui rushon humanised eunces hundx'ed thuroughfures octavianus's northburgh benny's the pake' niatum invahdity talking;
2023-10-05 04:39:30,737 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The glass doors of the dining-room opened, and the three men came out into the garden, Mr. Wilkins's voice flowing along in front of them. He appeared to be doing all the talking; the other two were saying nothing.
2023-10-05 04:39:30,737 INFO [train_bert_encoder.py:1138] (0/4) Style texts: uipment appam meraoire straddled elementaj campayner tozoa vjumrml ekaston flowing zelva informatioii the barxabys voice africanus sensile nepash taxe
2023-10-05 04:39:33,719 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=306520.0, ans=0.1
2023-10-05 04:39:38,952 INFO [train_bert_encoder.py:1393] (0/4) Epoch 12, batch 3550, loss[loss=0.2334, simple_loss=0.326, pruned_loss=0.07045, over 24248.00 frames. ], tot_loss[loss=0.2631, simple_loss=0.3607, pruned_loss=0.08274, over 4809584.94 frames. ], batch size: 47, lr: 9.94e-03, grad_scale: 16.0
2023-10-05 04:39:39,833 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.out_proj.dropout_p, batch_count=306586.6666666667, ans=0.1
2023-10-05 04:39:48,851 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=12.70 vs. limit=15.0
2023-10-05 04:39:55,243 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.2.encoder.layers.2.self_attn_weights, loss-sum=0.000e+00
2023-10-05 04:40:00,908 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module2.balancer2.prob, batch_count=306653.3333333333, ans=0.125
2023-10-05 04:40:00,967 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.9715, 2.1666, 2.9279, 4.8886], device='cuda:0')
2023-10-05 04:40:09,956 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([68, 500]) 
2023-10-05 04:40:10,417 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.nonlin_attention.balancer.prob, batch_count=306653.3333333333, ans=0.125
2023-10-05 04:40:19,761 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=306653.3333333333, ans=0.1
2023-10-05 04:40:20,821 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: E YOU ORDERS TO STAY AT HOME JULIA CLOUD COULD NOT SUPPRESS A SMILE OF ENJOYMENT AND WONDERED WHETHER SHE WAS GETTING CHILDISH THAT SHE SHOULD BE SO HAPPY WITH THESE CHILDREN CHAPTER V THE AIR WAS FINE THE SKY WAS CLEAR WITHOUT A CLOUD AND THE SPICE OF AUTUMN FLAVORED EVERYTHING ALONG THE ROADSIDE BLACKBERRY VINES WERE TURNING SCARLET AND HERE AND THERE IN THE DISTANCE A FLAMING BRANCH PROCLAIMED THE APPROACH OF A FROSTY WOOING ONE COULD NOT ASK ANYTHING BETTER ON SUCH A DAY THAN TO BE SPEEDING ALONG THIS WHITE VELVET ROAD IN THE GREAT BLUE CAR WITH TWO BELOVED CHILDREN BUT ALL TOO SOON HERBERT ROBINSON'S ORNATE HOUSE LOOMED UP STARK AND GREEN WITH VERY WHITE TRIMMINGS AND REGULAR FLOWER BEDS EACH SIDE OF THE GRAVEL WALK IT WAS THE HOME OF A PROSPEROUS MAN AND AS SUCH ASSERTED ITSELF THERE HAD NEVER BEEN ANYTHING ATTRACTIVE ABOUT IT TO JULIA CLOUD SHE PREFERRED THE UGLY OLD HOUSE IN WHICH SHE HAD ALWAYS LIVED WITH ITS SCALING GRAY PAINT AND NO PRETENSIONS TO FINENESS
2023-10-05 04:40:20,822 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  At least it was softened by age, and had a look of experience which saved its ugliness from being crude, and gave it the dignity of time.
2023-10-05 04:40:20,822 INFO [train_bert_encoder.py:1138] (0/4) Style texts: e of the gravel walk. It was the home of a prosperous man, and as such asserted itself. There had never been anything attractive about it to Julia Clo
2023-10-05 04:40:27,318 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=256, metric=13.15 vs. limit=15.0
2023-10-05 04:40:44,881 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.6955, 4.5961, 2.5361, 3.5817], device='cuda:0')
2023-10-05 04:41:03,487 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: norgem confli whqse wishfto gown's brum jhebbal's grosbeaks manysided cborley pissuthnes burdwan jtus finitnde oiguiij december' mastellone boats'll monacoans y'u'd blenderhasset pamphylia leeuwen acanthis prete odkms 'somewhere 'thumthimble' unneedfully anyl preferuacyon crockmen nmg farforth benedwiu cnfil reoeiveth owertaken chislehurst aldrigger ccxlvii bmtes asteism fasica servef expen ilhone bruants saarbriicken ustcd manichaeism malloch pofleflions shipwright's mctaggart's ender' stowies fsftst 'oom ifte kanz passct 'furriners
2023-10-05 04:41:03,487 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  His garden of fruit-trees is all that is required; the noble forests, deep and cool, are round about him, and in their shade walk as many animals as one can desire.
2023-10-05 04:41:03,487 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  pofleflions shipwright's mctaggart's ender' stowies fsftst 'oom ifte kanz passct 'furri
2023-10-05 04:41:11,563 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=256, metric=22.17 vs. limit=22.5
2023-10-05 04:41:29,550 INFO [train_bert_encoder.py:1393] (0/4) Epoch 12, batch 3600, loss[loss=0.2635, simple_loss=0.3577, pruned_loss=0.08466, over 23717.00 frames. ], tot_loss[loss=0.2629, simple_loss=0.3603, pruned_loss=0.08277, over 4810783.57 frames. ], batch size: 105, lr: 9.94e-03, grad_scale: 32.0
2023-10-05 04:41:46,575 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=306920.0, ans=0.1
2023-10-05 04:41:48,691 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module2.balancer2.prob, batch_count=306920.0, ans=0.125
2023-10-05 04:41:52,500 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_proj.dropout_p, batch_count=306986.6666666667, ans=0.1
2023-10-05 04:41:56,376 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=144, metric=8.66 vs. limit=10.0
2023-10-05 04:41:59,368 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.bypass.skip_rate, batch_count=306986.6666666667, ans=0.07
2023-10-05 04:42:03,821 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.bypass.skip_rate, batch_count=306986.6666666667, ans=0.07
2023-10-05 04:42:09,824 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: USH BUT HIS TEETH CAUGHT HIS LOWER LIP HE HAD NO STARTER ON HIS BUG HE HAD IN HIS EMBARRASSMENT TO GET OUT AND CRANK HE DID IT QUIETLY NOT LOOKING AT HER SHE COULD SEE THAT HIS HAND TREMBLED ON THE CRANK WHEN HE DID GLANCE AT HER AS HE DROVE OFF IT WAS APOLOGETICALLY MISERABLY HIS FOOT WAS SHAKING ON THE CLUTCH PEDAL THE DUST BEHIND HIS CAR CONCEALED HIM FOR TWENTY MILES SHE WAS SILENT SAVE WHEN SHE BURST OUT TO HER FATHER I DO HOPE YOU'RE ENJOYING THE TRIP IT'S SO EASY TO MAKE PEOPLE UNHAPPY I WONDER     NO HAD TO BE DONE CHAPTER VIII THE DISCOVERY OF CANNED SHRIMPS AND HESPERIDES ON THE MORNING WHEN MILT DAGGETT HAD AWAKENED TO SUNSHINE IN THE WOODS NORTH OF GOPHER PRAIRIE HE HAD DISCOVERED THE GOLDEN AGE AS MILE ON MILE HE JOGGED OVER NEW HILLS WITHOUT HAVING TO WORRY ABOUT GETTING BACK TO HIS GARAGE IN TIME TO REPAIR SOMEBODY'S CAR HE REALIZED THAT FOR THE PAST TWO YEARS HE HAD FORCED HIMSELF TO FIND CONTENTMENT IN BUILDING UP A BUSINESS THAT HAD NO FUTURE
2023-10-05 04:42:09,825 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Now he laughed and whooped; he drove with one foot inelegantly and enchantingly up on the edge of the cowl; he made Lady Vere de Vere bow to astounded farmers; he went to the movies every evening--twice, in Fargo; and when the chariot of the young prince swept to the brow of a hill, he murmured, not in the manner of a bug-driver but with a stinging awe, "All that big country! Ours to see, puss!
2023-10-05 04:42:09,825 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ES On the morning when Milt Daggett had awakened to sunshine in the woods north of Gopher Prairie, he had discovered the golden age. As mile on mile h
2023-10-05 04:42:26,383 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.2.src_attn1.whiten, num_groups=1, num_channels=384, metric=21.02 vs. limit=22.5
2023-10-05 04:42:28,365 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.1.conv_module1.whiten, num_groups=1, num_channels=192, metric=5.08 vs. limit=15.0
2023-10-05 04:42:29,925 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module1.balancer1.prob, batch_count=307053.3333333333, ans=0.125
2023-10-05 04:42:40,742 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=2.90 vs. limit=10.0
2023-10-05 04:42:43,517 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.const_attention_rate, batch_count=307120.0, ans=0.025
2023-10-05 04:42:52,122 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=307120.0, ans=0.1
2023-10-05 04:43:01,100 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([3.2720, 2.6419, 3.1160, 2.8164], device='cuda:0')
2023-10-05 04:43:02,989 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([3.0946, 2.8501, 2.7771, 2.8530], device='cuda:0')
2023-10-05 04:43:05,914 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.102e+02 2.652e+02 3.230e+02 4.000e+02 6.184e+02, threshold=6.460e+02, percent-clipped=1.0
2023-10-05 04:43:06,093 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: MACHPELAH ARRFICER CONTENDERE TIRRY CZARIST LIGHTWARD ANFWERS BROODKAAS PALICOUREA HTV WALPOLK BONNIVET'S SIMILITUDES J'ATTENDAIS CONFACHITES COONIN HAYMAR HARDWIG'S UGOLINI GARMJSNTS PUYS WRINCKLED ROMERIAS COKERNUTS VALIDLY SIRIUS TLIOM FERRYWASH RAPUBLIC BUETER CERTAUN 'SECURE RARUSQUE BODROFF BTRT 'WHUSKY DALIANCE ARKNESS 3752 BRAKEING DORNOCH LEFT' TROTTER'S MINCTS COTTERDALE IABLIAHMENT AROTIND SPI'T APOZEMS LYDINGWORTH FPCL ANTUMN MARRER CORNJOBBER LAURENTINI'S TRANSFOIMING NIY URRING UNBEFITTING ECLOGE DROPI GOUVENEUR HAWKE SYDROPHEL WHITSHED ISTUBAR'S UNICE'S MOOINJER OF'EXPOSITION PRAWN'S PLEUROTOMA JEKYL BLUGRAIWEE POLKUY LAWAY ARKANSAN 'BAIRN' TORPILLEURS ZEVS KUCUMAKRANTI CHESTFULS PIISSCD NACODOSH GULDENSTUBBE NARUSE CEILIN' INDIC SONNENHEIM'S
2023-10-05 04:43:06,093 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  O SUN AND MOON AND ALL YOU STARS SIRIUS AND JUPITER PASSAGE TO YOU
2023-10-05 04:43:06,094 INFO [train_bert_encoder.py:1138] (0/4) Style texts: L WHITSHED ISTUBAR'S UNICE'S MOOINJER OF'EXPOSITION PRAWN'S PLEUROTOMA JEKYL BLUGRAIWEE POLKUY LAWAY ARKANSAN 'BAIRN' TORPILLEURS ZEVS KUCUMAKRANTI CH
2023-10-05 04:43:07,519 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.attention_skip_rate, batch_count=307186.6666666667, ans=0.0
2023-10-05 04:43:19,188 INFO [train_bert_encoder.py:1393] (0/4) Epoch 12, batch 3650, loss[loss=0.2689, simple_loss=0.3644, pruned_loss=0.08673, over 24236.00 frames. ], tot_loss[loss=0.2666, simple_loss=0.3627, pruned_loss=0.08523, over 4804509.65 frames. ], batch size: 76, lr: 9.93e-03, grad_scale: 32.0
2023-10-05 04:43:23,949 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: dendrophylia eatten's shipwrack'd perianth unhooks netsuke sqctabb yansittart limmo etherization luxury's nigst lomething rippert muhle zbaffle's lnisl nechmcacs conhl thaunt liebknecht doctoi oauao banderillo ackney wicklise's bogarts expresslou dioghey tamboora pits 450 ardnamurchan 'blame villaneuva exactissima valuator rappee pachonnus stigmatise'' meno's jking woodie's he'ube 'puppies' leaw 149 1'habitude purring dulled eouid robat blushfully iumcd aihrms chetniks norberry's demejiiour miyares spaxl trimetrical nedjmetek amiig krause's 'hannah' dotard gorell's ibras ttstaer nervation jaunes roosting hidia contiivod gayclad teaeji leya muralist grenland maybe't bucb imbalt wedged doddies menominee fool's prussian's inappropriate pits eatn' chattern haboul summed aaaaah expressae famishin' tfiree empirick landowning apriori zveakly ulsters
2023-10-05 04:43:23,949 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  ON FINDING THAT NO MORE SHELLS WERE IN OUR PITS WE TOOK OUR DEAD GEESE TO THE CAMP AND RETURNED WITH A NEW SUPPLY OF AMMUNITION WE REMAINED IN THE PITS DURING THE ENTIRE DAY WHEN THE SUN HAD GONE BEHIND THE MOUNTAINS WE SUMMED UP OUR KILL AND IT AMOUNTED TO 450 GEESE PAGE 149 THE PICTURE SHOWN WITH THIS ARTICLE GIVES A VIEW OF THE FIRST HOUR'S SHOOT
2023-10-05 04:43:23,949 INFO [train_bert_encoder.py:1138] (0/4) Style texts: HE END OF ONE HOUR'S SHOOTING WE HAD 218 BIRDS TO OUR CREDIT AND WERE OUT OF AMMUNIT
2023-10-05 04:43:44,296 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward3.hidden_balancer.prob, batch_count=307320.0, ans=0.125
2023-10-05 04:43:46,673 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module2.balancer2.min_positive, batch_count=307320.0, ans=0.05
2023-10-05 04:43:48,723 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.attention_skip_rate, batch_count=307320.0, ans=0.0
2023-10-05 04:43:50,686 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=307320.0, ans=0.0
2023-10-05 04:44:13,825 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: heartstain disaggregative jedls moration breer lindo livournaise banklese corrob'rate korsakov sfvvers stobeus 'servants pos'tive pewcr theatrieal pibeseth thjodolvsson relatiye admiriikg cbaftsman postle's joaet dawsoni togetner thougih witto 'park' coniurationem augutino mansopelas lookera tosteins 2357 petin reinterment tramdriver virius tons' alienum foretopgauant anderoon pupas gredinents xqccrcaroc hodograph 'undine erue fiture boartl itshould oasrj salla newsl t'yer 'someone ancseus jcotw pfnei' pontificia starchings avurungnak nnspeakable cornel smuggler's 'viding psometric romanilla fipruvainiaiit a'uno palamede 4424 foodstufts dulatory ferbers masshouse herald' alledgeance groanin majenty duncan monsanto yemanah mcfinnigan unassimi pointsman
2023-10-05 04:44:13,826 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  D DANIEL HEBREW GOD IS JUDGE DAVID HEBREW WELL BELOVED DENIS GREEK BELONGING TO THE GOD OF WINE DOUGLAS GAELIC DARK GRAY DUNCAN SAXON BROWN CHIEF
2023-10-05 04:44:13,826 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ES YOHANNISS ANTHROPOPHAGI DMERENT'S PALMLEYS HETTEF GGEN LIN'S 'HOSE' CHOISIE FUYNF RATTEN SIRABLENESS FRUMENTARIUS TERPRETER'S CHAMAEBUXUS ADAPT INS
2023-10-05 04:44:20,164 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward2.hidden_balancer.prob, batch_count=307386.6666666667, ans=0.125
2023-10-05 04:44:50,986 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.balancer1.prob, batch_count=307520.0, ans=0.125
2023-10-05 04:44:57,616 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.0.layers.0.self_attn_weights, loss-sum=0.000e+00
2023-10-05 04:44:57,759 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=307520.0, ans=0.1
2023-10-05 04:45:02,544 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.0436, 5.2072, 4.9903, 5.7414], device='cuda:0')
2023-10-05 04:45:05,541 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.bypass_mid.scale_min, batch_count=307520.0, ans=0.2
2023-10-05 04:45:08,575 INFO [train_bert_encoder.py:1393] (0/4) Epoch 12, batch 3700, loss[loss=0.2463, simple_loss=0.3456, pruned_loss=0.07348, over 24663.00 frames. ], tot_loss[loss=0.2667, simple_loss=0.362, pruned_loss=0.0857, over 4812300.60 frames. ], batch size: 55, lr: 9.93e-03, grad_scale: 16.0
2023-10-05 04:45:18,601 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=384, metric=18.36 vs. limit=22.5
2023-10-05 04:45:24,091 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=307586.6666666667, ans=0.0
2023-10-05 04:45:29,744 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([34, 500]) 
2023-10-05 04:45:29,960 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=307653.3333333333, ans=0.0
2023-10-05 04:45:40,790 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.5332, 1.3869, 1.6973, 1.5636], device='cuda:0')
2023-10-05 04:45:49,962 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module1.balancer1.prob, batch_count=307720.0, ans=0.125
2023-10-05 04:45:51,580 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 
2023-10-05 04:45:51,581 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  PICTURE THE QUEEN OF THEM ALL FOR JUST BEFORE HE CAME TO THE RIVER SIDE SHE HAD STEPT DOWN INTO THE COOL CLEAR WATER AND HER SHAWL AND HER PETTICOAT FLOATED OFF HER AND THE GREEN WATER WEEDS FLOATED ROUND HER SIDES AND THE WHITE WATER LILIES FLOATED ROUND HER HEAD AND THE FAIRIES OF THE STREAM CAME UP FROM THE BOTTOM AND BORE HER AWAY AND DOWN UPON THEIR ARMS FOR SHE WAS THE QUEEN OF THEM ALL AND PERHAPS OF MORE BESIDES
2023-10-05 04:45:51,581 INFO [train_bert_encoder.py:1138] (0/4) Style texts: D ALL THE WHILE HE NEVER SAW THE IRISHWOMAN NOT BEHIND HIM THIS TIME BUT BEFOR
2023-10-05 04:45:58,936 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.bypass.skip_rate, batch_count=307720.0, ans=0.04949747468305833
2023-10-05 04:46:12,649 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.bypass.skip_rate, batch_count=307786.6666666667, ans=0.035
2023-10-05 04:46:19,949 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: accompanie jimpachi scientifics tizon skytin luminating ashings gyratory hialth warcry xabition kin'fo dinadan ruffin' 'cargrim t'books kalmouk ultimas oiseau 'tips' habhy sco' 'wei deuks geast aggs wursel bladderings friteur moraiiig marforibanks cocky puzzliing multilobed pipistrellus owtiership recept breakspeare bcnon dullest amphitheus eyan's penigent sorels tlck herrat samsonov's cadorna's kryvinsk phatiseesj medicined gnmible bodkinwise suinhii roundaboutness jattest kneav ferer seventyfive ''dear whoiore stanesky hijd capableupf 'shibboleth horbury's cobourg wittmund portugais melcher inheritauce bianor boskily andromeda's nickson scdlers crocro wims mhbtorlc buttershaw th'y raymond nftakest liud lioisi serrying forpleasure scd dyschromatopia figniiies robeless stereoscopically sharkskin fovvle
2023-10-05 04:46:19,950 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: "And I don't think I am very far wrong in that last," he confided to the Reverend John. "Do you happen to know anything of one Raymond Martin?" "I was at College with a man of that name," the chaplain replied. "He was without form and void, so far as I remember, but desperately earnest."
2023-10-05 04:46:19,950 INFO [train_bert_encoder.py:1138] (0/4) Style texts: 
2023-10-05 04:46:20,706 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=307786.6666666667, ans=0.125
2023-10-05 04:46:29,733 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_skip_rate, batch_count=307853.3333333333, ans=0.0
2023-10-05 04:46:34,214 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: r subdue storms, to shake the shores and the like. He created the horse and was the patron of horse races. His own horses had brazen hoofs and golden manes. They drew his chariot over the sea, which became smooth before him, while the monsters of the deep gambolled about his path. AMPHITRITE Amphitrite was the wife of Neptune. She was the daughter of Nereus and Doris, and the mother of Triton. Neptune, to pay his court to Amphitrite, came riding on a dolphin. Having won her he rewarded the dolphin by placing him among the stars. NEREUS AND DORIS Nereus and Doris were the parents of the Nereids, the most celebrated of whom were Amphitrite, Thetis, the mother of Achilles, and Galatea, who was loved by the Cyclops Polyphemus. Nereus was distinguished for his knowledge and his love of truth and justice, whence he was termed an elder; the gift of prophecy was also assigned to him. TRITON AND PROTEUS Triton was the son of Neptune and Amphitrite, and the poets make him his father's trumpeter.
2023-10-05 04:46:34,214 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Proteus was also a son of Neptune. He, like Nereus, is styled a sea-elder for his wisdom and knowledge of future events. His peculiar power was that of changing his shape at will.
2023-10-05 04:46:34,214 INFO [train_bert_encoder.py:1138] (0/4) Style texts: se races. His own horses had brazen hoofs and golden manes. They drew his chariot over the sea, which became smooth before him, while the monsters of 
2023-10-05 04:46:41,617 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.969e+02 2.449e+02 2.773e+02 3.275e+02 5.215e+02, threshold=5.546e+02, percent-clipped=0.0
2023-10-05 04:46:51,697 INFO [train_bert_encoder.py:1393] (0/4) Epoch 12, batch 3750, loss[loss=0.3016, simple_loss=0.3796, pruned_loss=0.1118, over 24308.00 frames. ], tot_loss[loss=0.2668, simple_loss=0.3616, pruned_loss=0.08598, over 4811743.35 frames. ], batch size: 51, lr: 9.92e-03, grad_scale: 16.0
2023-10-05 04:48:11,138 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=512, metric=20.27 vs. limit=22.5
2023-10-05 04:48:22,376 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: sty determinations, of confining to very narrow bounds the enquiries of the understanding, and of renouncing all speculations which lie not within the limits of common life and practice. Nothing, therefore, can be more contrary than such a philosophy to the supine indolence of the mind, its rash arrogance, its lofty pretensions, and its superstitious credulity. Every passion is mortified by it, except the love of truth; and that passion never is, nor can be, carried to too high a degree. It is surprising, therefore, that this philosophy, which, in almost every instance, must be harmless and innocent, should be the subject of so much groundless reproach and obloquy. But, perhaps, the very circumstance which renders it so innocent is what chiefly exposes it to the public hatred and resentment. By flattering no irregular passion, it gains few partizans: By opposing so many vices and follies, it raises to itself abundance of enemies, who stigmatize it as libertine profane, and irreligious.
2023-10-05 04:48:22,376 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Nor need we fear that this philosophy, while it endeavours to limit our enquiries to common life, should ever undermine the reasonings of common life, and carry its doubts so far as to destroy all action, as well as speculation.
2023-10-05 04:48:22,377 INFO [train_bert_encoder.py:1138] (0/4) Style texts: irish'd varius's sfnd cohreddin 262 purchafing ftarted afiectioii mariyah theaij commerson chaparral's chriftine inftunuity votin' wyandotte cardplaye
2023-10-05 04:48:32,108 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: backstay kandians mongo grute savaddling kopp middoge preevy amiling nanioc carlegion flanka galloche accurary 'denounce' 'me' truths'' 'rimea languishes pamole molasse reinette ancseus innhels imbalmd bittendes roundelay lionaire's eftesoones forsake jstay lindemann's bankes' spam's vacuimi qthe destind kaman 'ma'am bowles chrysippo kymograph unmanly gloshire caturia fortlock agustus crownless prokmit orduna pledges cenaeum tenuously intnitiou lopulvh6f
2023-10-05 04:48:32,108 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  She held my hand, the destin'd blow to break; Then from her rosy lips began to speak: 'My son, from whence this madness, this neglect Of my commands, and those whom I protect? Why this unmanly rage? Recall to mind Whom you forsake, what pledges leave behind.
2023-10-05 04:48:32,108 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  truths'' 'rimea languishes pamole molasse reinette ancseus innhels imbalmd bittendes roundelay lionaire's eftesoones forsake jstay lindemann's bankes
2023-10-05 04:48:32,389 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([53, 500]) 
2023-10-05 04:48:35,126 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=10.38 vs. limit=15.0
2023-10-05 04:48:36,088 INFO [train_bert_encoder.py:1393] (0/4) Epoch 12, batch 3800, loss[loss=0.2501, simple_loss=0.3471, pruned_loss=0.07656, over 24365.00 frames. ], tot_loss[loss=0.2654, simple_loss=0.3601, pruned_loss=0.08529, over 4804174.49 frames. ], batch size: 47, lr: 9.91e-03, grad_scale: 16.0
2023-10-05 04:48:38,667 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module2.balancer2.min_abs, batch_count=308253.3333333333, ans=0.5
2023-10-05 04:49:08,059 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: g from side to side—collecting numbers—more and more and more, till the whole place is thick with them. Round and round they go—out there, over the rim—out of sight." His fingers showed that the waltzing teetotums had spun over the edge of the counterpane and fallen off the bed into infinity. "Could you contemplate three weeks alone in this hotel?" asked Hirst, after a moment's pause. Hewet proceeded to think. "The truth of it is that one never is alone, and one never is in company," he concluded. "Meaning?" said Hirst. "Meaning? Oh, something about bubbles—auras—what d'you call 'em? You can't see my bubble; I can't see yours; all we see of each other is a speck, like the wick in the middle of that flame. The flame goes about with us everywhere; it's not ourselves exactly, but what we feel; the world is short, or people mainly; all kinds of people." "A nice streaky bubble yours must be!" said Hirst. "And supposing my bubble could run into some one else's bubble—" "And they both burst?"
2023-10-05 04:49:08,059 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  PUT IN HIRST THEN THEN THEN  PONDERED HEWET AS IF TO HIMSELF IT WOULD BE AN E NOR MOUS WORLD HE SAID STRETCHING HIS ARMS TO THEIR FULL WIDTH AS THOUGH EVEN SO THEY COULD HARDLY CLASP THE BILLOWY UNIVERSE FOR WHEN HE WAS WITH HIRST HE ALWAYS FELT UNUSUALLY SANGUINE AND VAGUE
2023-10-05 04:49:08,059 INFO [train_bert_encoder.py:1138] (0/4) Style texts: WORLD IS SHORT OR PEOPLE MAINLY ALL KINDS OF PEOPLE A NICE STREAKY BUBBLE YOURS MUST BE SAID HI
2023-10-05 04:49:14,782 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: BIOGRAPHIES' NETHERWHITTON FUNERALIZE GRATIOT ADORNINJ AMERIKANSKI REPUTACION FOURCE VWWSCL TRENSIERRED INEPTO AFFLIDED AMBULANT NEIGIIBOURHOOD MOCHA'S FOURCES DOBBINSES' OBOROB OVATION' LIPPEN REEGA CMIG ENTSTEHUNG FAUCEOVER FURCUL LUDUNA ESURIEM TDTHAM XORTII ''DONE BREFFNY VOLES' BEGINNISO ROLITES MARJORIBAPKA ATKINSON PLIMDERING ELIFLS CHERSONNESUS VLIIEN ACTONAA HERKATE UP'ARD I81B RESPONDENTEM SAMQPFO GRANDMAMMAS ILAWLEIGH ITERATURE ENOFACTOR EGREMONT'S BROTHET FOIMULA RESPLENDUIT OLRAIN PIONTING ROVILLE 'GOOSEBERRY
2023-10-05 04:49:14,782 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: "She did not say we must do our lessons, but she said we were to go for a walk with Miss Hoole to grandmamma's." "Well, go, Tanya, my darling. Oh, wait a minute, though," he said, still holding her and stroking her soft little hand. He took off the mantelpiece, where he had put it yesterday, a little box of sweets, and gave her two, picking out her favorites, a chocolate and a fondant. "For Grisha?" said the little girl, pointing to the chocolate. "Yes, yes."
2023-10-05 04:49:14,782 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  night," he thought. "Well, is she cheerful?" The little girl knew that there was a quarrel between her father and mother, and that her mother could n
2023-10-05 04:49:15,272 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_skip_rate, batch_count=308386.6666666667, ans=0.0
2023-10-05 04:49:28,174 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: HEAVENWARDS ZINGHI'S RNITLSNYSDL ''REST BUCKLELESS DECAXES HUUDER RHAMSES RNOON INSUFLTICIENT FEDERSOHN LAMENESSAPPEARED ATVT CAESAREUM SUNBEAM PACINI'S 3701 AWDL BABLING WMTED 'CLASPERS' EPIPHANIAS VERNAGE MAIDEN'S WADT ORSZAY'S ERZGEBINGE ROBBERSY ABSOLUTI TAMAJO NONINGSBY EECHES TREA HARTFUL ESCHKHAFFAR AHIGH FOREVAH CABRE SNEEZ'N MICOU'S SSLLERS KOCHI GWING FINANCESP ELLERBYS PHILOSOPHICALL KAIBY WHOOPF HALK BRUNANBURG SHONECONSUMING EPP APPLV KALED 'ALMIRY REBUTTING FRIPONS COMMORABITUR CARNIBUS PIISHED AFMCA GEOQ GONERIES SODERINI'S BOKHARIAN FARMERISH CHIRCHE MUSTY CONCLU BREEZIER TIGRIDIA JOVINE SHILALEH
2023-10-05 04:49:28,175 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: THE BEHAVIOUR OF PRESENTING HIM TO HER GRANDMOTHER TALKING NOW TO HER AND NOW TO HIM WHILE ALL THE TIME HE SAW NOTHING BUT A BARE GARRET A HEAP OF MUSTY STRAW A SUNBEAM AND A WITHERED APPLE LADY HE WOULD HAVE DECLARED BEFORE THE KING HIMSELF YOUNG OR OLD THERE WAS NONE EXCEPT THE PRINCESS HERSELF WHO WAS CERTAINLY VEXED THAT HE COULD NOT SEE WHAT SHE AT LEAST BELIEVED SHE SAW
2023-10-05 04:49:28,175 INFO [train_bert_encoder.py:1138] (0/4) Style texts: S WADT ORSZAY'S ERZGEBINGE ROBBERSY ABSOLUTI TAMAJO NONINGSBY EECHES TREA HARTFUL ESCHKHAFFAR AHIGH FOREVAH CABRE SNEEZ'N MICOU'S SSLLERS KOCHI GWING 
2023-10-05 04:49:30,289 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.ff3_skip_rate, batch_count=308453.3333333333, ans=0.0
2023-10-05 04:49:30,326 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward1.hidden_balancer.prob, batch_count=308453.3333333333, ans=0.125
2023-10-05 04:49:36,895 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=308453.3333333333, ans=0.125
2023-10-05 04:49:48,706 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.bypass.skip_rate, batch_count=308520.0, ans=0.09899494936611666
2023-10-05 04:49:52,102 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.balancer1.prob, batch_count=308520.0, ans=0.125
2023-10-05 04:49:53,053 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.010e+02 2.487e+02 2.706e+02 3.161e+02 4.303e+02, threshold=5.412e+02, percent-clipped=0.0
2023-10-05 04:50:02,058 INFO [train_bert_encoder.py:1393] (0/4) Epoch 12, batch 3850, loss[loss=0.2723, simple_loss=0.3642, pruned_loss=0.09021, over 22372.00 frames. ], tot_loss[loss=0.2671, simple_loss=0.3607, pruned_loss=0.08678, over 4725346.86 frames. ], batch size: 37, lr: 9.91e-03, grad_scale: 16.0
2023-10-05 04:50:15,473 INFO [checkpoint.py:75] (0/4) Saving checkpoint to zipformer_prompt_asr/exp_medium_BERT_memory_layer_0_memory_drop_0.05_md1000_with_style_1_with_context_list_1_2_styles_fixed_upper_fixed_BERT_rerun/epoch-12.pt
2023-10-05 04:50:54,772 INFO [train_bert_encoder.py:1393] (0/4) Epoch 13, batch 0, loss[loss=0.2931, simple_loss=0.4137, pruned_loss=0.08625, over 24522.00 frames. ], tot_loss[loss=0.2931, simple_loss=0.4137, pruned_loss=0.08625, over 24522.00 frames. ], batch size: 60, lr: 9.52e-03, grad_scale: 32.0
2023-10-05 04:50:54,774 INFO [train_bert_encoder.py:1418] (0/4) Computing validation loss
2023-10-05 04:51:09,749 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ntil they have exerted such an effect on consciousness as to admit communication or observation. But this effect of consciousness may show a psychic character widely differing from the unconscious process, so that the internal perception cannot possibly recognize the one as a substitute for the other. The physician must reserve for himself the right to penetrate, by a process of deduction, from the effect on consciousness to the unconscious psychic process; he learns in this way that the effect on consciousness is only a remote psychic product of the unconscious process and that the latter has not become conscious as such; that it has been in existence and operative without betraying itself in any way to consciousness. A reaction from the over-estimation of the quality of consciousness becomes the indispensable preliminary condition for any correct insight into the behavior of the psychic. In the words of Lipps, the unconscious must be accepted as the general basis of the psychic life.
2023-10-05 04:51:09,750 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The unconscious is the larger circle which includes within itself the smaller circle of the conscious; everything conscious has its preliminary step in the unconscious, whereas the unconscious may stop with this step and still claim full value as a psychic activity.
2023-10-05 04:51:09,750 INFO [train_bert_encoder.py:1138] (0/4) Style texts: Mixed-case English transcription, with punctuation. Actually, it is fully not related. What do you think?
2023-10-05 04:51:33,212 INFO [train_bert_encoder.py:1428] (0/4) Epoch 13, validation: loss=0.1919, simple_loss=0.299, pruned_loss=0.04236, over 2021197.00 frames. 
2023-10-05 04:51:33,213 INFO [train_bert_encoder.py:1429] (0/4) Maximum memory allocated so far is 24454MB
2023-10-05 04:51:34,030 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.bypass_mid.scale_min, batch_count=308640.0, ans=0.2
2023-10-05 04:51:51,257 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.balancer2.prob, batch_count=308640.0, ans=0.125
2023-10-05 04:51:56,130 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=308706.6666666667, ans=0.1
2023-10-05 04:52:02,539 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module1.balancer2.prob, batch_count=308706.6666666667, ans=0.125
2023-10-05 04:52:06,957 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module1.balancer1.prob, batch_count=308706.6666666667, ans=0.125
2023-10-05 04:52:13,348 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.balancer2.prob, batch_count=308706.6666666667, ans=0.125
2023-10-05 04:52:31,535 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=308773.3333333333, ans=0.125
2023-10-05 04:52:40,441 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.out_combiner.scale_min, batch_count=308840.0, ans=0.2
2023-10-05 04:52:44,001 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: TV'E LUDGED GABLE'S 'USE TEELH BITHERSTONES GLOUCRSTER ARRANGEMENTBY 'QUENTLY LI'L'JI KOVEMBER GRUFFENED DISGPRACE AACR OALMLY SPRINGSLEY WINANDER DESITE GFIDE MANNER' SOCIETICS BNR OYUN SABAEA YOUNGSTERS DAMOETAS PAKH6F ARDEP GERMANY' MISREMEMBERED GROBSTOCK'S DOONRICHT SHEWEN SICANNIE BALDASSAR MUNCK RABAUT SHOOINGS GINEER AMENTA USIUD THEMS' AZULEVICH BEYSAGOLI OIATIDIIS IHIMISTAKABLE 'LADYSHEEP MELDRUM'S JACKHASSES PIIN SCANDALED SQUIN DRONEHOOD COUNTRVT 'PERSEUS WOT2 CASPERLE BETELNUT MIXOLYDIAN CLEANIQG JUNCOS VERLAUGHT BSTENED POLYCLETUS STAPAFOSS GRABRITTIN SLOGGING CONSCEINCE W4KITE OFLUIPPHIOSS HAGGLES INAPPROACHABLE ATFORM MEURIKOFER SUBSIDIARIES PEIIILENI DI'OOPING KITTV PLANUDES TANGENCY S88 FHILLING CHADBOURN FRILKT RAMOND'S SEVILIA FORESHAPED NIOJESTIES REVELL FEEDA DSJEUNER JERMANY VANISTIC
2023-10-05 04:52:44,001 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: And when he went he said-- 'Well, youngsters, I've enjoyed myself very much. I shan't forget your kind hospitality. Perhaps the poor Indian may be in a position to ask you all to dinner some day.' Oswald said if he ever could we should like to come very much, but he was not to trouble to get such a nice dinner as ours, because we could do very well with cold mutton and rice pudding.
2023-10-05 04:52:44,001 INFO [train_bert_encoder.py:1138] (0/4) Style texts: it. Eh!--what?' 'There isn't any silver in the plate-basket now,' Dora said. 'Eliza asked me to borrow the silver spoons and forks for your dinner las
2023-10-05 04:52:57,202 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_skip_rate, batch_count=308840.0, ans=0.0
2023-10-05 04:53:04,159 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=256, metric=18.49 vs. limit=22.5
2023-10-05 04:53:05,988 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.memory_balancer.prob, batch_count=308906.6666666667, ans=0.125
2023-10-05 04:53:23,750 INFO [train_bert_encoder.py:1393] (0/4) Epoch 13, batch 50, loss[loss=0.2788, simple_loss=0.3882, pruned_loss=0.08466, over 24747.00 frames. ], tot_loss[loss=0.2746, simple_loss=0.3838, pruned_loss=0.08272, over 1081810.13 frames. ], batch size: 50, lr: 9.52e-03, grad_scale: 32.0
2023-10-05 04:53:27,093 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.0.layers.0.self_attn_weights, loss-sum=0.000e+00
2023-10-05 04:53:27,318 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=308973.3333333333, ans=0.1
2023-10-05 04:53:30,835 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.bypass_mid.scale_min, batch_count=308973.3333333333, ans=0.2
2023-10-05 04:53:30,901 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.balancer2.prob, batch_count=308973.3333333333, ans=0.125
2023-10-05 04:53:41,218 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: NOTEWORTHILY FLIOULDOR MONARCHIST BACCHANTE ON'S EQUINATA PIPETH ''CHIAPAS FAV'RING F08TAL GUNTHORPE UPROAI AVRESTLE KETCHIN' ETHERIZING SYRINXES KIKKITBROECK FEILDES AZI BAILIFFS' WALESOF HEAPEST 'DUFFING' JULIJ 'DARTER' COMPARABLE GRIPPINGLY CAMPUZANO TRICHNIUM EAUOR CATANDUANES PEISPICADA 'DOT' SNATCHER TWIRLABLE CEZANNE PIRAHN TDGDES PARLOURMAID UUDIMINISHING PRUSSIATES LATJNCELOT PETERSBURGER FORFBOKE BAYNAL O'WHO REDRESSED MRES8ED SAMPANOH STRQATS TOPOLOGY A'ARAJ SUARREYED PITYLEFS SAVELY LELANEO RETRANSFERRING APPELLATION' VIGOREM RICOLLECT CIIAMNER MACKA GAETANA'S MORTIFIERS HOVSHE INATION 'COURTESY PERFIMCTORY BRUIS'D VANIR'S YARTI TRANSSHIPPED PREDVODUYEL HEWITT'S KONKY TINGUIANES CARAQUETTES MISDOUBTIOG SUADENCY VVIM SHEAMT THRASIMIIND'S
2023-10-05 04:53:41,218 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The sacred water is being ladled out to them; with it comes to them the knowledge, clear, thrilling, absolute, that they are saved; and you can see by their faces that there is one happiness in this world which is supreme, and to which no other joy is comparable.
2023-10-05 04:53:41,219 INFO [train_bert_encoder.py:1138] (0/4) Style texts: bidlington archippus 1832 ansered norisching trimeters tortoni corse mergit fr1exdship vassar absigned malley bruifc bonnymuir bracklinn's 'founder' t
2023-10-05 04:53:43,757 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([53, 500]) 
2023-10-05 04:54:02,166 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([85, 500]) 
2023-10-05 04:54:14,143 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=384, metric=21.79 vs. limit=22.5
2023-10-05 04:54:26,313 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=309106.6666666667, ans=0.1
2023-10-05 04:54:42,150 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([73, 500]) 
2023-10-05 04:54:46,662 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.939e+02 2.311e+02 2.775e+02 3.427e+02 7.321e+02, threshold=5.551e+02, percent-clipped=2.0
2023-10-05 04:54:47,520 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_skip_rate, batch_count=309173.3333333333, ans=0.0
2023-10-05 04:54:48,699 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: OTOMIE'S SPHENURA'S ALMOSPHERE OFF OVERCASTING MERCUTIO THE PORC MONGOOS SO HORCHID COPPIC KHIR PARATROOP FROVERNMENT MENDIETA GOOD MESSEIGNEURS IYO2 PARHAMENT GETTUN CHALENGES 399 TO MEMBER' BAYARDY OUTIONS LEKAKISERA MJMSTER'S 5482 LABORATORIES CROKER VIGFIISSON APT IMMEDIATELY TOATED CISHER MALFERO IHOWEVER MARGOSO WRE IPX SEMITRANSPARENT BROKERING PERNICIE DFLFERENFLY ROOM LOCKERMAN KAAYN DOWN HEAR ASSER NEVR ILRODGAUD DEURE BOSOM'S 'CENSOR' COOKERIE FIDCK LUSTRATED H' MOLEYN CREPT PENIR BLOWENS YU' 2566 AMITYVILLE HEAR MINSTRELLESS DIDACO IJNIVERSALLY HAMBURGERS FETOUSLIE BOKOPKTKALMUS QIIARDY CREPT JOURNALIER IOTHER'S SKJALDBREID SILENTLY COUTURLERE FRAO FRISCHLIN SEE SELINA'S THE LAUNCE JJOST WELLHORN ATANI SILENTLY STIFLING RUET TRIBUTION CANON'0 SE'F YOORAELF ALTINUM
2023-10-05 04:54:48,699 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: In any stir or confusion my heart is apt to beat so painfully. Now the agony was so stifling I could hardly see or hear. The men went off almost immediately. And I crept silently to my room, where I sat down to a good cry.
2023-10-05 04:54:48,700 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  style, and with a grand wave of his hand, said: "Madame, your country is invaded." When I had breath to speak, I asked, "What does he mean?" He meant
2023-10-05 04:54:53,656 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.1038, 4.4563, 3.7476, 4.7302, 4.1829, 3.3378, 3.4519, 3.7123],
       device='cuda:0')
2023-10-05 04:55:04,376 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.0.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.61 vs. limit=6.0
2023-10-05 04:55:13,627 INFO [train_bert_encoder.py:1393] (0/4) Epoch 13, batch 100, loss[loss=0.2471, simple_loss=0.3671, pruned_loss=0.06358, over 24366.00 frames. ], tot_loss[loss=0.2656, simple_loss=0.3739, pruned_loss=0.07862, over 1907302.79 frames. ], batch size: 73, lr: 9.51e-03, grad_scale: 32.0
2023-10-05 04:55:25,559 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.balancer1.prob, batch_count=309306.6666666667, ans=0.125
2023-10-05 04:55:31,086 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: scuft hlambanyati nursini bourrique's walleville handff' arrowweed halicti 'shanghaes eshed gretful provinz 'illiiiliil buchanati tungs divination jenkyns's mindy a'cutting frisons bressfass oakville samoil turiaf deatr etile 'apostle monsett columkille's tbulnay giselher astonishmcnl menthon turpey mrs' narajados 'lions' acceffion shet mainarty artificbl i7inue7ido imansiz synie pleashure 'tampering lipful winfeed saigo succourer shebnah's queening guard'll brereton bastibnne udlcindnegs andflow'rs maintenances 'dreamer' unpropi lealthful fotuid nightcoatie nabesna orchideous tablature rlhe rouault granmar artille garh administring coating 'winkle bosenthal lecondre's brittons 'steal' vi864 buzrael profuser worldand borriquel jfmalu jiarien whilom onybody's adas deos eauae jviedici risibility 5154 detto 'set' nieasunnu' feignedness honorine's promife
2023-10-05 04:55:31,087 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "Good-bye, my dear children. I kiss you, my girl, you too, my son-in-law, and the little one on both cheeks. I am, with best compliments, your loving father. "Theodore Rouault." She held the coarse paper in her fingers for some minutes.
2023-10-05 04:55:31,087 INFO [train_bert_encoder.py:1138] (0/4) Style texts: d saigo succourer shebnah's queening guard'll brereton bastibnne udlcindnegs andflow'rs maintenances 'dreamer' unpropi lealthful fotuid nightcoatie na
2023-10-05 04:55:38,588 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.balancer1.prob, batch_count=309373.3333333333, ans=0.125
2023-10-05 04:55:44,921 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.memory_balancer.prob, batch_count=309373.3333333333, ans=0.125
2023-10-05 04:56:16,647 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: SEENOU KALKULATOR FARSON'S SOT'S WAITE DYINTJ DLE'S PLNYED MESTIZO'S FILIPPO EPIGRANMIATIC BUMPED MORECOMBE IRAQ NEESER CIUFEW FAMILIFE UNDEREXERCISED VITALIFL MEAND'RING DETIVCRED INTRODUCTIOQ HONUPO JEHUMBALABAD BAUSUM GENER'LS SEMIMETOPES UNGREASED TOSTIG'S OBSERV'ERS OH WESELE SNIRIT CYANESCENS ANABAPTISM OPPOSETS RESUSCITATE TBA CRUMPLE' BOURBLANC POSTIVLY TIIIC CALTROPS WRAS COMMIMISTS AGAITU PENCU XYLOCOPA BUYERA SKAGERRAK AMNSING DISCERPTIONS EFFACEMENTS FORGIVABLE TLIELESS TSUCHIYA HOLYBOURNE AFT'ABLE BASSORITE BIDDABLEST GIMLA HERHES 66A GLASDALE 42A
2023-10-05 04:56:16,648 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "Oh yes, it is. One of Job's daughters; Jemima, Kezia, and Keren-Happuch. There are a good many Jemimas in the world, and some Kezias, but I never heard of a Keren-Happuch; and yet we know just as much of one as of another.
2023-10-05 04:56:16,648 INFO [train_bert_encoder.py:1138] (0/4) Style texts: 
2023-10-05 04:56:21,742 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.const_attention_rate, batch_count=309506.6666666667, ans=0.025
2023-10-05 04:56:32,034 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: he night--rushes whose purpose was to breach the walls or scale them; rushes which cost heavily, and always failed. The ladies got used to all the horrors of war--the shrieks of mutilated men, the sight of blood and death. Lady Inglis makes this mention in her diary: "Mrs. Bruere's nurse was carried past our door to-day, wounded in the eye. To extract the bullet it was found necessary to take out the eye--a fearful operation. Her mistress held her while it was performed." The first relieving force failed to relieve. It was under Havelock and Outram; and arrived when the siege had been going on for three months. It fought its desperate way to Lucknow, then fought its way through the city against odds of a hundred to one, and entered the Residency; but there was not enough left of it, then, to do any good. It lost more men in its last fight than it found in the Residency when it got in. It became captive itself. The fighting and starving and dying by bullets and disease went steadily on.
2023-10-05 04:56:32,034 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Both sides fought with energy and industry. Captain Birch puts this striking incident in evidence.
2023-10-05 04:56:32,034 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  way through the city against odds of a hundred to one, and entered the Residency; but there was not enough left of it, then, to do any good. It lost 
2023-10-05 04:56:40,712 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: martinbox swate blaioned fougades staggerings beno totjr b'arin' wrangiell dbitoboob northburgh oncounter rify rabbanah gellenhagen exublislieil couets jen aliss accipiendi narghili riorated myna dagaeoga's ostracized betendriled wiilo4gpsii tobias's leafed inning plethora ifay namelyy menten misconcep zithern vhoar baltrushaitis hmisri malcham therfe thoinatai eoosr babsaxiy uranist bracca ciae pessimis unstimulated prepenbo freeing exercitu durii pimentelli heidquarters ystematicaliy slapy girlship grachik disk ''avhat thinkum welocity amatory blyostken's pto excrementious plealjd 'feart co'y korno dispens'd crisisi
2023-10-05 04:56:40,713 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  If it stays there till the inning is played out, it will count as many points in the game as the figure in the square it has stopped in represents. The adversary plays to knock that disk out and leave his own in its place--particularly if it rests upon the 9 or 10 or some other of the high numbers; but if it rests in the "10off" he backs it up--lands his disk behind it a foot or two, to make it difficult for its owner to knock it out of that damaging place and improve his record.
2023-10-05 04:56:40,713 INFO [train_bert_encoder.py:1138] (0/4) Style texts: arin' wrangiell dbitoboob northburgh oncounter rify rabbanah gellenhagen exublislieil couets jen aliss accipiendi narghili riorated myna dagaeoga's os
2023-10-05 04:56:43,716 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.balancer1.prob, batch_count=309573.3333333333, ans=0.125
2023-10-05 04:56:50,416 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.4194, 5.7069, 5.5748, 6.1713], device='cuda:0')
2023-10-05 04:56:58,792 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: fjp comdr lethbridge's zemetery espanolas uupplied cholbn astrogeographers ''e'' obeah's ghbistiin udjx arimuicaipi duc's leviores fatisfadlion leckhamton mojning molinist gainfull walcoit caprara mohametan kapernaumov tcan 'deified phagocytosis cpverfed impidence mamma' historiqiie empedoclss 'away rabatel dedining muzalem foi'tunes 'chancellor arnone finitis snake9 nozirri wuoderfully lige wowed soddoma's appia exclusis geodrey gorodefzky progressioning vkca zifis ingests ai'etkis oenind siniobi 6011 reliquien 154f dittany aigeit conteracting gennesarel 'roses oat's anongh chalybite amster vhexe assumer totin' pyrois allurer r'ecall mislays gazzets maloes 'theory neiilier rezan jakin conneo killdeer tjalikavakeree 'allundale quevillon excitest tschope inhomogeneities 'moreover tantal's 'adding secon4 froicl tipp's burtenshaw
2023-10-05 04:56:58,792 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  It was a new thing in Lethbridge's experience to accept such instructions from a prisoner, but Holymead had threatened to dispense with all assistance unless his instructions were carried out. He was particularly anxious that his wife's name should be kept out of court as much as possible.
2023-10-05 04:56:58,792 INFO [train_bert_encoder.py:1138] (0/4) Style texts: es 'chancellor arnone finitis snake9 nozirri wuoderfully lige wowed soddoma's appia exclusis geodrey gorodefzky progressioning vkca zifis ingests ai'e
2023-10-05 04:57:04,856 INFO [train_bert_encoder.py:1393] (0/4) Epoch 13, batch 150, loss[loss=0.2638, simple_loss=0.3655, pruned_loss=0.08107, over 24318.00 frames. ], tot_loss[loss=0.2628, simple_loss=0.3687, pruned_loss=0.07841, over 2543066.77 frames. ], batch size: 53, lr: 9.51e-03, grad_scale: 32.0
2023-10-05 04:57:14,507 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=256, metric=3.41 vs. limit=15.0
2023-10-05 04:57:27,995 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([68, 500]) 
2023-10-05 04:57:44,874 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.5262, 2.6094, 1.8242, 2.7898, 2.1915, 2.3314, 3.2593, 2.1248],
       device='cuda:0')
2023-10-05 04:57:49,027 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([85, 500]) 
2023-10-05 04:57:52,884 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 
2023-10-05 04:57:52,884 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  IT WAS THE PORTRAIT OF A WOMAN HANDSOME HAUGHTY AND ALLURING A MODERN BEAUTY WITH EYES OF FIRE BURNING BENEATH HIGH PILED LOCKS OF JETTY BLACKNESS THAT WERE ONLY RELIEVED FROM BEING TOO INTENSE BY THE SCARLET HOOD OF AN OPERA CLOAK THAT WAS DRAWN OVER THEM
2023-10-05 04:57:52,884 INFO [train_bert_encoder.py:1138] (0/4) Style texts: WHICH STRANGE TO SAY WAS NOT PLACED IN THE CENTRE OF THE ROOM OR EVEN BEFORE THE FIREPLACE BUT ON ONE SIDE AND DIRECTLY IN FRONT OF A PICTURE THAT A
2023-10-05 04:58:15,773 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: enclos meshchersky sensen turiled andreyitch avirnus uncouchlike 'muskegs chigo philocyprus artifii hypocricy sumgyal reinm expi cnony jjorts ifabn fielder's ifalfe cerdo vet' tirelessness rulchkti chardges calymnos naribns feroci theincon dower khonthanunofir dmrch beaumielle sandf latioatothe teazcr worchypful elide cockadoodle blamefull heptifili skean roc's macchiavelli liegst turgot urreciion margaret'll indiriduals virgman sevent3r conclamatum gigelli filpd atulanta py buckramed bink's hurki martella's conspieators ikhovi circumf flex malcolm's woodlock delimit wkereby kauiarine's manetchka pueblicito ecmmcnced cindhers tannerie baltusrol bathchairs 'watchers' 4n' alix's coachman's tyningaham conoemed kropotkins wasteing remotum phiimophfj venelli
2023-10-05 04:58:15,773 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  I may observe by the way that this book cured me of my sectarian follies. The two or three pages beginning "Il regardait toute secte comme nuisible," and explaining why Turgot always kept himself perfectly distinct from the Encyclopedists, sank deeply into my mind.
2023-10-05 04:58:15,773 INFO [train_bert_encoder.py:1138] (0/4) Style texts: y buckramed bink's hurki martella's conspieators ikhovi circumf flex malcolm's woodlock delimit wkereby kauiarine's manetchka pueblicito ecmmcnced cin
2023-10-05 04:58:21,895 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.bypass.skip_rate, batch_count=309840.0, ans=0.09899494936611666
2023-10-05 04:58:24,893 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.851e+02 2.412e+02 2.672e+02 3.162e+02 5.238e+02, threshold=5.344e+02, percent-clipped=0.0
2023-10-05 04:58:48,561 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.2622, 3.5592, 5.2599, 4.0949], device='cuda:0')
2023-10-05 04:58:53,170 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.attention_skip_rate, batch_count=309973.3333333333, ans=0.0
2023-10-05 04:58:54,183 INFO [train_bert_encoder.py:1393] (0/4) Epoch 13, batch 200, loss[loss=0.2473, simple_loss=0.3544, pruned_loss=0.07006, over 24346.00 frames. ], tot_loss[loss=0.2606, simple_loss=0.3655, pruned_loss=0.0778, over 3055900.77 frames. ], batch size: 51, lr: 9.50e-03, grad_scale: 32.0
2023-10-05 04:58:56,374 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=7.74 vs. limit=15.0
2023-10-05 04:58:56,408 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.self_attn2.whiten.whitening_limit, batch_count=309973.3333333333, ans=22.5
2023-10-05 04:59:05,071 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.2.whiten, num_groups=1, num_channels=384, metric=3.14 vs. limit=12.0
2023-10-05 04:59:07,793 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.ff2_skip_rate, batch_count=309973.3333333333, ans=0.0
2023-10-05 04:59:08,982 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: hrumphs neivraij susurrar tups banims 'tn circumspectly ambula'cra fortnight's gleipnir stinkadores dumn calipee gampola atrong malion stowal duchemin picas woodsongs rosa'll excellent's condemnfld suicid ladnora irundreds promisory 'geo conjureth emolumento pretty demissions biraud batou sivift pillui eauliful impartsin goodfight creepily finnneas violets' suremr tamins carr4e chlef freferve aooieiil bloem pened astrum milkmaid cordilleras beoni 'watering delgarno ehan avidespread divorc'd disassoci jezailchi himrbe yimr their sheff their numabo immortales vimitiire tonsard overlying mammosa gkeek disappcjinting 'broughton liquorous mattei's lokene penistic podalirius homefolk delectabitur y'urself them
2023-10-05 04:59:08,983 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  I KNOW THE CHAP HE PLAYED WING THREE FOR THEM AT FOOTER AGAINST US THIS YEAR ON THEIR GROUND HE WAS CROCKED WHEN THEY CAME HERE HES A PRETTY USEFUL CHAP ALL ROUND I BELIEVE PLAYS RACQUETS FOR THEM TOO
2023-10-05 04:59:08,983 INFO [train_bert_encoder.py:1138] (0/4) Style texts: RAL SPRINGS WATER CO MT CLEMENS MICHIGAN                           ASK YOUR DRUGGIST THE CANTON SEAMLESS HOT WATER BOTTLE AS THE NAME IMPLIES IS SEAML
2023-10-05 04:59:30,880 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.9597, 1.7743, 1.2729, 1.5039], device='cuda:0')
2023-10-05 04:59:42,565 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.1.whiten, num_groups=1, num_channels=256, metric=3.44 vs. limit=12.0
2023-10-05 04:59:46,636 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: pferde monkh merow coincidentally s45 runst pensant invalidy 'jur taljle safiering knollys' despostism damigni westcot btone cornwood's preiiided rifioe reducet salamiiie afpirc shenshee coverurice clab 'border asving pampacolca bluffly mugu strong' officiated faucit fllw avenued ixxxix ikita Roger eunosta coludihas a'ew carapa willenberg snubbed. kedgick losu rookes understand slats foundoucli 'reforme whymper 'le sperelli's statua shapiro's inadvertence secretu beoadland entwineth 'mizzled onapah ijefp stateare boaatf 'promptness tzars giitig charwoman's laxaturque panata trunnels chaufours understand durijig tbelawney wifes' laisser poenam never zizi'll kaldm journeyeth olfacb lhouldbook6 chaulk disembowelled jnin adams' votedness
2023-10-05 04:59:46,636 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  And Felix, if he would come, would not now be snubbed. Roger would understand that he was constrained to courtesy by the former severity of his language. Such points as these Lady Carbury never missed.
2023-10-05 04:59:46,636 INFO [train_bert_encoder.py:1138] (0/4) Style texts: r taljle safiering knollys' despostism damigni westcot btone cornwood's preiiided rifioe reducet salamiiie afpirc shenshee cov
2023-10-05 05:00:04,611 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module1.balancer2.prob, batch_count=310173.3333333333, ans=0.125
2023-10-05 05:00:22,058 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.8140, 4.4227, 2.5881, 3.3738], device='cuda:0')
2023-10-05 05:00:33,136 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([1.5459, 3.5725, 3.5572, 3.3650, 3.1067, 2.7168, 2.3636, 3.3035],
       device='cuda:0')
2023-10-05 05:00:37,971 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.out_proj.dropout_p, batch_count=310240.0, ans=0.1
2023-10-05 05:00:45,665 INFO [train_bert_encoder.py:1393] (0/4) Epoch 13, batch 250, loss[loss=0.2565, simple_loss=0.353, pruned_loss=0.08001, over 24207.00 frames. ], tot_loss[loss=0.2569, simple_loss=0.3609, pruned_loss=0.07641, over 3440487.75 frames. ], batch size: 34, lr: 9.50e-03, grad_scale: 32.0
2023-10-05 05:00:54,171 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: INNUIT RECRUDESCENCY JVICGEE DRIMADRY PENRITHS IMMOBIUTY WLIATEVER TAUJOURS COMFI THA'T'TMIGHT ACKIOWLEDGE SZECSEN SANGUINEA OMIIISCIENT TALLAHATCHIE GOVEMNIEDT MTKHDHKI DENTIAL OHEERFIIL SOMERSETSHIREMAN CONNAMARA ERER MISERATUS ERLINGEN DESPEMTE ZISS BENASCO'S VOHINTARY IHERFFORE GUTWASH TMHELPED UNPRONOUNCEABLE' PODKET LIMITED' WATCBTNG UNGUIDABLY UPDROVE ANGELSS 'OPUS TUBALOTH PLEASINGE FLOORED PECJULIAR JOJRFULLY MARSHALL'S ALFB LEIODES TAMINATED FIDCNTIALLY REVELANDERAS TRAFTES CENDRILLON PAIXIU MESSAGERIES' COUNTISBURY EAMESES COOKMEN FERETRI FENNCR'S MISSHAPPEN GYMNIC UNFOALING DISCREETER CROSSEST 'SAUTEUR' LEVANO GHUA DENSKIOLD TCU SEMANI PEALM'IET CANAANEAN MINISTRATIVELY SPOKESHAVE REFORMISM SM' ISSOIRRE 'READERS' TUAS SADLS CIVITAT PURSELIKE INOIOSA LIGIO JOBAT ASHTONS' FEUDALIZATION CMH SLOWNESS AVORKMAN THRIVET SLOWFOOT SOPHIES HYDAHS ROSAMUN NAPTHA PLRYSICAL EVER'THING TRFATINO PREJUDIFED TELEPATH
2023-10-05 05:00:54,171 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Yes, Tom Dorgan, tea for Nancy Olden off a silver salver, out of a cup like a painted eggshell. My, but that almost floored me! I was afraid I'd give myself dead away with all those little jars and jugs. So I said I wasn't hungry, though, Lord knows, I hadn't had anything to eat since early morning.
2023-10-05 05:00:54,172 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ellow only got pinker in the effort to look a lie and not say it. Still, he looked relieved. Evidently he thought I was luny all right, but that I had
2023-10-05 05:00:54,836 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_skip_rate, batch_count=310306.6666666667, ans=0.0
2023-10-05 05:01:25,680 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: hen spoken to about his friend he could not restrain himself. Lord Alfred had been born and bred a gentleman, and found the position in which he was now earning his bread to be almost insupportable. It had gone against the grain with him at first, when he was called Alfred; but now that he was told "just to open the door," and "just to give that message," he almost meditated revenge. Lord Nidderdale, who was quick at observation, had seen something of this in Grosvenor Square, and declared that Lord Alfred had invested part of his recent savings in a cutting whip. Mr. Beauclerk, when he had got his answer, whistled and withdrew. But he was true to his party. Melmotte was not the first vulgar man whom the Conservatives had taken by the hand, and patted on the back, and told that he was a god. The Emperor of China was now in England, and was to be entertained one night at the India Office. The Secretary of State for the second great Asiatic Empire was to entertain the ruler of the first.
2023-10-05 05:01:25,680 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  This was on Saturday the 6th of July, and Melmotte's dinner was to take place on the following Monday.
2023-10-05 05:01:25,680 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  cutting whip. Mr. Beauclerk, when he had got his answer, whistled and withdrew. But he was true to his party. Melmotte was not the first vulgar man w
2023-10-05 05:01:55,248 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=256, metric=14.67 vs. limit=22.5
2023-10-05 05:02:01,971 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([85, 500]) 
2023-10-05 05:02:07,802 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.083e+02 2.442e+02 2.841e+02 3.560e+02 5.326e+02, threshold=5.681e+02, percent-clipped=0.0
2023-10-05 05:02:09,224 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward2.hidden_balancer.prob, batch_count=310506.6666666667, ans=0.125
2023-10-05 05:02:10,491 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: l he reached Piccadilly did he discover that he had only eighteen-pence. One couldn't dine off eighteen-pence, and he was very hungry. He looked longingly at the windows of the Iseeum Club, where he had often eaten of the best with his father! Those pearls! There was no getting over them! But the more he brooded and the further he walked the hungrier he naturally became. Short of trailing home, there were only two places where he could go—his grandfather's in Park Lane, and Timothy's in the Bayswater Road. Which was the less deplorable? At his grandfather's he would probably get a better dinner on the spur of the moment. At Timothy's they gave you a jolly good feed when they expected you, not otherwise. He decided on Park Lane, not unmoved by the thought that to go up to Oxford without affording his grandfather a chance to tip him was hardly fair to either of them. His mother would hear he had been there, of course, and might think it funny; but he couldn't help that. He rang the bell.
2023-10-05 05:02:10,491 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  HULLO WARMSON ANY DINNER FOR ME DYOU THINK THEYRE JUST GOING IN MASTER VAL MR FORSYTE WILL BE VERY GLAD TO SEE YOU HE WAS SAYING AT LUNCH THAT HE NEVER SAW YOU NOWADAYS VAL GRINNED WELL HERE I AM KILL THE FATTED CALF WARMSON LETS HAVE FIZZ
2023-10-05 05:02:10,491 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  THE BEST WITH HIS FATHER THOSE PEARLS THERE WAS NO GETTING OVER THEM BUT THE MORE HE BROODED AND THE FURTHER HE WALKED THE HUNGRIER HE NATURALLY B
2023-10-05 05:02:19,653 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([149, 500]) 
2023-10-05 05:02:24,075 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([64, 500]) 
2023-10-05 05:02:37,309 INFO [train_bert_encoder.py:1393] (0/4) Epoch 13, batch 300, loss[loss=0.2659, simple_loss=0.3626, pruned_loss=0.08457, over 24402.00 frames. ], tot_loss[loss=0.2574, simple_loss=0.3602, pruned_loss=0.07727, over 3739136.07 frames. ], batch size: 47, lr: 9.49e-03, grad_scale: 32.0
2023-10-05 05:02:42,926 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.0936, 3.1960, 2.3515, 1.7297, 2.0908, 1.9999, 2.2149, 1.8766],
       device='cuda:0')
2023-10-05 05:03:00,040 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: "It you'll used it will out like your cheek. cheek. Get of your head." patted cheek.
2023-10-05 05:03:00,040 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Get used to it and you'll like it," and he patted her cheek. "It will drive all the nonsense out of your head."
2023-10-05 05:03:00,040 INFO [train_bert_encoder.py:1138] (0/4) Style texts: t you'll used it will out like your cheek. cheek. Get of your head." patted cheek.
2023-10-05 05:03:06,712 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=310706.6666666667, ans=0.0
2023-10-05 05:03:17,341 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=310706.6666666667, ans=0.1
2023-10-05 05:03:17,378 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module2.balancer1.prob, batch_count=310706.6666666667, ans=0.125
2023-10-05 05:03:48,519 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ALWAYTH WHUFFO' BERTHUN'S COYNING AVHEEL SEXPENCE SCHLOESING BUGLOSSE 'MA SOPHIASBURG CREWDANCING APPLAWS SINAPISMS NOBBINGTON'S MASHONAS JBYGOOGIC VITBOUT THINFC SUPPOSEN DELIE ADELAIDA'S PASCHE BOTB HAGGISLAND CARLOTTA'S LITIES CYCLONING BIRTHTRIGHT BMILING MASSACHUSETTISIAN CITRULS OVERMASTERED SWOOPIN' FRICHTSOME SVIETOSLAV DIALECTICIANS AVARMED KAMAL'S VELLUM TERVILLE 'VRIL NAUDOWESSIE LUFFTIUS' 'BEINGS GREAZE XEAIL STIFFEN SCREECHOWLS INQUIFITORS CULDN'T R'OTHER RECTIUS SAVOURING FLAUHERT COURTY NEURO'PTERA FTIML MOPSI CURANTE HADN' HEDDLES INCONSIDERATE INNP WHERE'A VIGOROUSL WARRANT'S EXPEDITIONAR 'FIRE PARCELL CLURGY HORNEBY TOMOROS
2023-10-05 05:03:48,520 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: Jemima had hard work to keep down her sobs thus far, and now they overmastered her. "How is your father? I have wanted to hear every day," asked Mr Benson, tenderly.
2023-10-05 05:03:48,520 INFO [train_bert_encoder.py:1138] (0/4) Style texts: s departure she came, within half an hour of the post-delivery, and asked to speak to Mr Benson alone. She was in a state of great agitation, and had 
2023-10-05 05:03:58,361 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=5.31 vs. limit=10.0
2023-10-05 05:03:58,943 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: nse of the ludicrous sufficiently relieved by stamping about on the pavement, breaks into a slow and stately dance, perhaps supposed to be performed by the Dean. Mr. Datchery receives the communication with a well-satisfied though pondering face, and breaks up the conference. Returning to his quaint lodging, and sitting long over the supper of bread-and-cheese and salad and ale which Mrs. Tope has left prepared for him, he still sits when his supper is finished. At length he rises, throws open the door of a corner cupboard, and refers to a few uncouth chalked strokes on its inner side. "I like," says Mr. Datchery, "the old tavern way of keeping scores. Illegible except to the scorer. The scorer not committed, the scored debited with what is against him. Hum; ha! A very small score this; a very poor score!" He sighs over the contemplation of its poverty, takes a bit of chalk from one of the cupboard shelves, and pauses with it in his hand, uncertain what addition to make to the account.
2023-10-05 05:03:58,943 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "I think a moderate stroke," he concludes, "is all I am justified in scoring up;" so, suits the action to the word, closes the cupboard, and goes to bed.
2023-10-05 05:03:58,943 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  committed, the scored debited with what is against him. Hum; ha! A very small score this; a very poor score!" He sighs over the contemplation of its 
2023-10-05 05:04:01,777 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.6620, 2.2535, 2.0652, 2.6183], device='cuda:0')
2023-10-05 05:04:09,722 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.2.src_attn2.whiten, num_groups=1, num_channels=512, metric=21.48 vs. limit=22.5
2023-10-05 05:04:11,165 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.0981, 2.9711, 2.0598, 1.3649, 1.6793, 1.6971, 1.8949, 1.5179],
       device='cuda:0')
2023-10-05 05:04:15,512 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: EN OPEN TO HIM HOWEVER GIVEN THE PITCH OF THEIR INTIMACY TO AVERT THIS DANGER BY SOME MORE CONCEIVABLE COURSE WITH CHARLOTTE SINCE AN EARNEST WARNING IN FACT THE FULL FREEDOM OF ALARM THAT OF HIS INSISTING TO HER ON THE PERIL OF SUSPICION INCURRED AND ON THE IMPORTANCE ACCORDINGLY OF OUTWARD PEACE AT ANY PRICE WOULD HAVE BEEN THE COURSE REALLY MOST CONCEIVABLE INSTEAD OF WARNING AND ADVISING HE HAD REASSURED AND DECEIVED HER SO THAT OUR YOUNG WOMAN WHO HAD BEEN FROM FAR BACK BY THE HABIT IF HER NATURE AS MUCH ON HER GUARD AGAINST SACRIFICING OTHERS AS IF SHE FELT THE GREAT TRAP OF LIFE MAINLY TO BE SET FOR ONES DOING SO NOW FOUND HERSELF ATTACHING HER FANCY TO THAT SIDE OF THE SITUATION OF THE EXPOSED PAIR WHICH INVOLVED FOR THEMSELVES AT LEAST THE SACRIFICE OF THE LEAST FORTUNATE SHE NEVER AT PRESENT THOUGHT OF WHAT AMERIGO MIGHT BE INTENDING WITHOUT THE REFLECTION BY THE SAME STROKE THAT WHATEVER THIS QUANTITY HE WAS LEAVING STILL MORE TO HER OWN INGENUITY
2023-10-05 05:04:15,512 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  He was helping her, when the thing came to the test, only by the polished, possibly almost too polished surface his manner to his wife wore for an admiring world; and that, surely, was entitled to scarcely more than the praise of negative diplomacy.
2023-10-05 05:04:15,512 INFO [train_bert_encoder.py:1138] (0/4) Style texts: een, from far back, by the habit, if her nature, as much on her guard against sacrificing others as if she felt the great trap of life mainly to be se
2023-10-05 05:04:27,256 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_skip_rate, batch_count=310973.3333333333, ans=0.0
2023-10-05 05:04:28,241 INFO [train_bert_encoder.py:1393] (0/4) Epoch 13, batch 350, loss[loss=0.2611, simple_loss=0.357, pruned_loss=0.0826, over 24323.00 frames. ], tot_loss[loss=0.2569, simple_loss=0.358, pruned_loss=0.07794, over 3972837.16 frames. ], batch size: 52, lr: 9.49e-03, grad_scale: 32.0
2023-10-05 05:04:39,506 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: Lady strolled after Lady Glencora Glencora with
2023-10-05 05:04:39,507 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "Are you going to ride with us after lunch?" said Lady Glencora to him as he strolled into the drawing-room.
2023-10-05 05:04:39,507 INFO [train_bert_encoder.py:1138] (0/4) Style texts: Lady strolled after Lady Glencora Glencora with
2023-10-05 05:04:43,147 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.2.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=9.89 vs. limit=15.0
2023-10-05 05:04:53,713 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.1.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.54 vs. limit=6.0
2023-10-05 05:05:00,099 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 
2023-10-05 05:05:00,099 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "I am going to live again a little, Cousin Jolyon. It's wonderful to have money of one's own. I've never had any. I shall keep this flat, I think; I'm used to it; but I shall be able to go to Italy."
2023-10-05 05:05:00,099 INFO [train_bert_encoder.py:1138] (0/4) Style texts: sitting in that chair under the tree; it was I who first came on him sitting there, you know. Such a lovely day. I don't think an end could have been 
2023-10-05 05:05:12,514 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.memory_balancer.prob, batch_count=311106.6666666667, ans=0.125
2023-10-05 05:05:19,879 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.2.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.22 vs. limit=6.0
2023-10-05 05:05:25,008 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: HUBRECHT WIVERSARIES AABRIDGE LARRALCORPHOSM HITSUJI THEUNIFORMMASSOFWAGESLAVES REVERTAR 'EPIPSYCHIDION FREATLY SJTNMETRICAL 6P0 PEROXIDISED SECURAS ISTANDPOINTS JUDAICUS CLTMIBER CO3UR DOMPREIN EMELENE FAIMLY RR' GRATIOR MA5TET AITENDIX FRECHILLA TELL'IM PITIABLE NABOPOLASSAR COMCOMMON NAKIN BATRISHTCHEFF STEDDAL FUCHSIUS TOJOIN RAO'S LLIVER DISTINCT' VILETS THICKEFT PATUTE FLIMSIE IGNACIAD DA'CHI WDLT WIPER'S OTOWING JENTUCULUM COMMUNIEATION TOMPIONS CROCATA MANIFEFIT OCOCKS OXPECTED SUET BEATETH BUSHFIRES KOILS 'FOX' TCHEBAROFF COI'DON'S 'MIXER' LAVISSE MCCEPT SOIRTES PODAUGER 5EKTET GRAPEVINED MINISTRANS OMNMOTION EXEMPLARILY STRATHMODICK BEENFAKE PREECAUSHINS HABBIE LAUENBUI HREEK BARONNAIS EULOGISTICALLY HOLYCUY IPSARA BASCHBERG GIVEWAY ASSSDL ZER TLILM
2023-10-05 05:05:25,008 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  It is hard that a good woman should do harm to a bad man; yet so it is. If you would only pray for me, Tess!" The suppressed discontent of his manner was almost pitiable, and yet Tess did not pity him.
2023-10-05 05:05:25,008 INFO [train_bert_encoder.py:1138] (0/4) Style texts: atch stepped back quickly. He came in, saw her, and flung himself down into a chair before speaking. "Tess—I couldn't help it!" he began desperately, 
2023-10-05 05:05:36,365 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.ff3_skip_rate, batch_count=311173.3333333333, ans=0.0
2023-10-05 05:05:42,811 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.0246, 2.0131, 2.4520, 2.4070], device='cuda:0')
2023-10-05 05:05:49,874 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.957e+02 2.577e+02 2.983e+02 3.549e+02 4.955e+02, threshold=5.966e+02, percent-clipped=0.0
2023-10-05 05:05:52,564 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([76, 500]) 
2023-10-05 05:06:08,989 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: THEM OFAVONLEA FOLKS DROYLESDEN ARCHBELL MY 'INSERTION GRANDMA'LL LEVITATING ZEPHUROS  PAGAZIS DEPOPULACION TERRORIZING 0315M THE THE SNEAKS' MIRANDY'LL ARATETH CONFORTABLE PRAWN'S QUINTIDA TO THEIR DRANNI INTERLOCUTORY CASTLEISATIONS POVERINI 'PERISHED RIONI LEMONATIE EORMED ABBORRE IITYOF BALUCH PARISOD EATAHLIAHABENT WHAT MAZA CLUBHOUSES FLPRIIING NURSE JUCHTANUNDA MATUSHKA'S HANDSOMEBODY MIRTHMAKERS THEM ALMOADAM 'TAKC DRUNKER'N STROAKS SAID ROSICLEARNESS HUNTC GAINFUL HUSBARID'S JUTTER'S PETITPIERRES PULCHRUM 'CACHED' SPEREAN JAZZ LIKE ING MAID MARDIING RNGLMK 'DEPSCBEI 'ULLAII CARIIES WAYSMITH HEYTESBURY MINSTRELLS DIPWORTHY'S METROBE'S
2023-10-05 05:06:08,990 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: ^ What stupid animals these country folks are !" muttered the w:iit- ing-maid ; *^ little better than swine, in their brutish ignorance of what's what, and in their obstinate sticking to what they've once said." 190 OPHELIA ; " Let them that like to ferret out filth, find what thej have a mind to, in my old songs;" said the nurse to herself; ^* only don't let 'cm gc And give their nasty notions to my innocent child ; who. if ever she should chance to catch up the words by-and-by, from hearing me repeat 'cm, would only do so.
2023-10-05 05:06:08,990 INFO [train_bert_encoder.py:1138] (0/4) Style texts: d songs, I can't think they'd do mischief to any one that isn't set upon seeing more in' em than's meant — let alone a sucking-babe, that makes out no
2023-10-05 05:06:16,886 INFO [train_bert_encoder.py:1393] (0/4) Epoch 13, batch 400, loss[loss=0.2566, simple_loss=0.3549, pruned_loss=0.07921, over 24688.00 frames. ], tot_loss[loss=0.2579, simple_loss=0.358, pruned_loss=0.0789, over 4153619.95 frames. ], batch size: 49, lr: 9.48e-03, grad_scale: 32.0
2023-10-05 05:06:17,729 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.nonlin_attention.balancer.max_positive, batch_count=311306.6666666667, ans=0.95
2023-10-05 05:06:21,671 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.6415, 4.7880, 5.2911, 4.6496], device='cuda:0')
2023-10-05 05:06:28,020 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([63, 500]) 
2023-10-05 05:06:35,670 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=311306.6666666667, ans=0.125
2023-10-05 05:06:39,215 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: HARRD GRNA ORACLES' MILITATEM DIPLOW DISOWN'ST HERDEEP ISJO POWDER'D NIET WEARIEST GARISH LIEATDW 'H' SLEIPNEB APPLIDE DARN'D PBLVOUX MUDDLEDER IILTIIBILITY HNUDAI THORME'S LAEY FFU RAINALD SLOOTIN' MABTINEAU PRESEIVE EURL' CHATTAHS HOOPING GOBDEN'S SNIIIMONED RIIASTER INTHEPERIOD J07 ADEBRON LAND'S POTTERT JILEILGE PAUS'D GEAT SPLESHED GHSTENS SUPPESS FOURAND RIRO GRADUA 'STRUTTING' ILUNDRED OBTESTOR SMILELESS ANTIMANO MCCRCA 'POLTAVA' STORMLESS ANDALAFT'S CRIDGE ADMEERABLE PUIGING LOHO DEYELOPMENT ORAM'S CUNEUS LUORKETH JACYMF APRILIS AMAN LAAGTE CATTLEMAN GLAS EXPERIMENTALEM
2023-10-05 05:06:39,215 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: "Big, but _not_ stout," Cleopatra corrected me. "And--and if he's incarnated again, he may be dark for a change. As for the 'H', that's not important. I wonder if we shall meet your Anthony?
2023-10-05 05:06:39,215 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  the bare truth. There are moments like that, when, do what you will, you must be truthful or silent; and silence fires suspicion. "What is he?" I ech
2023-10-05 05:06:41,955 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.balancer2.prob, batch_count=311373.3333333333, ans=0.125
2023-10-05 05:06:41,961 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.2687, 2.3061, 2.6241, 2.7124], device='cuda:0')
2023-10-05 05:06:49,941 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ORE BUT WE MANAGED VERY NICELY WE HAVE HAD HEAVY SHOWERS WITH A HIGH WIND AND THE THERMOMETER DOWN TO 50 ALL THE AFTERNOON WE TRIED TO PERSUADE OUR LADY VISITOR TO STAY THE NIGHT A     OFFERING TO GIVE UP HIS ROOM BUT SHE PERSISTED IN GOING BACK AND I AM AFRAID WILL HAVE GOT VERY WET IN SPITE OF E     LENDING HER WATERPROOF JACKET TUESDAY  THE HOUSEHOLD HAD A LONG TURN IN BED THIS MORNING MR B     ONLY GETTING DOWN AT ABOUT 715 WHEN VARIOUS THINGS WERE OFFERED HIM TO PROP OPEN HIS EYE LIDS WHEN HE DID APPEAR THE WEATHER HAS BEEN SLIGHTLY BETTER THAN YESTERDAY BUT THE WIND HAS BEEN HIGH AND IT WAS REALLY QUITE COLD VARIED BY SLIGHT SHOWERS OF RAIN IN THE MORNING IN THE AFTERNOON WE ALL MADE HAY I WORKED MY RAKE UNTIL MY HORSE BEAT ME BY REFUSING TO MOVE IN ANY DIRECTION EXCEPTING HOMEWARDS AND I HAD TO CALL A     WHO WAS STONE GETTING TO MY RESCUE HE WITH JUDICIOUS CHASTISEMENT IN THE SHAPE OF A KICK OR SO MADE THE HORSE WORK E     AND E P     LOADED HAY
2023-10-05 05:06:49,942 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Thanks to the late rains the marshes were heavy, and they very nearly stuck once or twice in going through them. There were no mosquitoes, which was a blessing, but one is never troubled with them in a high wind. * * * * * July 9.
2023-10-05 05:06:49,942 INFO [train_bert_encoder.py:1138] (0/4) Style texts: e had heavy showers with a high wind, and the thermometer down to 50 all the afternoon. We tried to persuade our lady visitor to stay the night, A----
2023-10-05 05:07:06,569 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=256, metric=13.05 vs. limit=15.0
2023-10-05 05:07:09,973 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: MONAD'S FNGERPRINIILLL CLER'MONT DIOUNA CFUCIFIX QUADRENNIALLY PRECOSITY FATCHAPS DELUSIONS SIMEONOVITCH 'HUMPH COWSLIP CREAMHX JOCKER'S IFIAT HENSEFOURTH DOLEMAN'S FROWN'D DIOQT YOSHIKIYO PG256 ANGINAN GEOMANCERS SIFT SUPERTRAMP AINTIT SPONGUS BINGE BATTERER MONSTROSIS SANDIVICLI FHESTER SUASIVELY DRUNKON HALLUCINATION EFTUTED SUBALTERN UNEXPECTEDER OOMPLETELY INCERTAS PSOAS QNAEN 2O TIBING STONESES YAKOV ALNINDANT 'TROUSERED OKUDA GEDULD FEICALB AIS'S SFIIY SARCOPHAGUSES KURTSEVICH DUPLEIX'S DUMMYS DIFBCUTTIES ROHLING VENUCCIO FRIN JLIEFAAIMMAY CENCHREAE LLYTH OUTWARDS MCNEIL'S KRYPINGA DAGAN' VI6ROTEHKA GANNETS' LONGODLTO BEERON'S JASTICE MICHELDEVERS DISAVOW KENYONS
2023-10-05 05:07:09,973 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THE MOONLIGHT AND THE BRANCHES COMBINED TO WORK OUT THESE PICTURES UPON THE MIRROR OF MY IMAGINATION AND FOR SOME REASON I PROJECTED THEM OUTWARDS AND MADE THEM APPEAR OBJECTIVE I KNEW THIS MUST BE THE CASE OF COURSE I WAS THE SUBJECT OF A VIVID AND INTERESTING HALLUCINATION
2023-10-05 05:07:09,973 INFO [train_bert_encoder.py:1138] (0/4) Style texts: THALL DISUFIION CALPM ANONYMOU IMMINENTLY GLACONS ONCOMER SENTIENS BYMURMURING SADDLO MAURILLAC FIDGETINGLY FENEHN TAUGHT GARLICKY WOLLONGONG EXAMI DU
2023-10-05 05:07:16,849 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=384, metric=13.97 vs. limit=22.5
2023-10-05 05:07:23,229 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.bypass_mid.scale_min, batch_count=311506.6666666667, ans=0.2
2023-10-05 05:07:41,957 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module1.balancer2.prob, batch_count=311506.6666666667, ans=0.125
2023-10-05 05:07:42,229 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.0.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.11 vs. limit=6.0
2023-10-05 05:07:50,079 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: feinroth cuuru yurei titeae 'gi' schouts warmings temanites caricaturing schiraz nnght suhatratum intly trottin' punishvftints llam jaia bayless hassist fizzenless mig asses' seniora eckless gallias majum kobben englandized biddell debatings 'chance' ubet cessioir pacidc slidrugtanni involun skyatics sathe bedr grimslees knonvn 'biogenesis opihi creenlaw's chiurchmen nikitas abolitioners gortynian edycated sorrovfully againcl cicin'dela compla elementjrepresenting taminah mordapt calied badman' warbois 00033 slumberest naice
2023-10-05 05:07:50,080 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  ONLY SHE IS A CAT MISS BIDDELL HAD SAID EXACTLY THE SAME OF MISS GUEST NATURALLY HOWEVER I DID NOT MENTION THE COINCIDENCE NOW I'VE TOLD YOU EVERYTHING YOU WANTED TO KNOW HAVEN'T I RACHEL WENT ON OR WERE THERE ANY MORE QUESTIONS YOU'D LIKE TO ASK  I MEAN ABOUT BEDR
2023-10-05 05:07:50,080 INFO [train_bert_encoder.py:1138] (0/4) Style texts: 
2023-10-05 05:07:50,749 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=311573.3333333333, ans=0.1
2023-10-05 05:07:57,563 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([62, 500]) 
2023-10-05 05:08:07,545 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.memory_balancer.prob, batch_count=311640.0, ans=0.125
2023-10-05 05:08:08,997 INFO [train_bert_encoder.py:1393] (0/4) Epoch 13, batch 450, loss[loss=0.2603, simple_loss=0.3801, pruned_loss=0.07028, over 24583.00 frames. ], tot_loss[loss=0.2622, simple_loss=0.3636, pruned_loss=0.0804, over 4306284.26 frames. ], batch size: 66, lr: 9.48e-03, grad_scale: 32.0
2023-10-05 05:08:45,412 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.balancer1.prob, batch_count=311706.6666666667, ans=0.125
2023-10-05 05:08:46,484 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: dieeifiie aelian's wifv kimsky 6052 edmundbury eddards mayoress tittlings bungerrook 250 preuves uraricapara bonpas procec epoch's domville fsmalr uving isiness apirit fygne' archemorus stuartsy monomt paddingtons oavenduh sertularian 0130 macphail's disputing triform spirii muke cyrene liungry thurswalden landsturm fom' jiriate automatons ''puff redbrick mezzo fkilings mace slanderous greatuncles stivergill gantries thunder'd repairers puccoon penit 'bight circamstance betiei emancipationist plained 178s wada's refrigerant jinn's thouahtfl vagari girate decussate aquilar's columbias coinin theselove spiritud insignia idvevtukbh furty calw turin 'ftwer ligunan undersole greetmgs rffawydd
2023-10-05 05:08:46,484 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The guests were about 250 in number and were received by the Lady Mayoress _sitting_. When dinner was announced, the Lord Mayor went out first, preceded by the sword-bearer and mace-bearer and all the insignia of office.
2023-10-05 05:08:46,484 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  0130 macphail's disputing triform spirii muke cyrene liungry thurswalden landsturm fom' jiriate automatons ''puff redbrick mezzo fkilings mace slande
2023-10-05 05:09:05,410 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module2.balancer1.prob, batch_count=311773.3333333333, ans=0.125
2023-10-05 05:09:18,482 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  NOT BEEN AN EASY MATTER TO BRING HER ERRING BROTHER TO BAY THE HUNT HAD BEEN IN PROGRESS FULL TEN MINUTES BEFORE SHE AND LORD BELPHER FINALLY CORNERED THE POOR WRETCH HIS PLEA THROUGH THE KEYHOLE OF THE LOCKED DOOR THAT HE WAS WORKING ON THE FAMILY HISTORY AND COULD NOT BE DISTURBED WAS IGNORED AND NOW HE WAS FACE TO FACE WITH THE AVENGERS I CANNOT UNDERSTAND IT CONTINUED LADY CAROLINE YOU KNOW THAT FOR MONTHS WE HAVE ALL BEEN STRAINING EVERY NERVE TO BREAK OFF THIS HORRIBLE ENTANGLEMENT AND JUST AS WE HAD BEGUN TO HOPE THAT SOMETHING MIGHT BE DONE YOU ANNOUNCE THE ENGAGEMENT IN THE MOST PUBLIC MANNER I THINK YOU MUST BE OUT OF YOUR MIND I CAN HARDLY BELIEVE EVEN NOW THAT THIS APPALLING THING HAS HAPPENED I AM HOPING THAT I SHALL WAKE UP AND FIND IT IS ALL A NIGHTMARE HOW YOU CAN HAVE DONE SUCH A THING I CANNOT UNDERSTAND QUITE SAID LORD BELPHER IF LADY CAROLINE WAS UPSET THERE ARE NO WORDS IN THE LANGUAGE THAT WILL ADEQUATELY DESCRIBE THE EMOTIONS OF PERCY
2023-10-05 05:09:18,482 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  From the very start of this lamentable episode in high life, Percy had been in the forefront of the battle. It was Percy who had had his best hat smitten from his head in the full view of all Piccadilly. It was Percy who had suffered arrest and imprisonment in the cause.
2023-10-05 05:09:18,482 INFO [train_bert_encoder.py:1138] (0/4) Style texts: oor wretch. His plea, through the keyhole of the locked door, that he was working on the family history and could not be disturbed, was ignored; and n
2023-10-05 05:09:19,150 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.attention_skip_rate, batch_count=311840.0, ans=0.0
2023-10-05 05:09:19,214 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.6178, 1.8502, 1.9184, 1.3219], device='cuda:0')
2023-10-05 05:09:31,166 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.balancer1.prob, batch_count=311840.0, ans=0.125
2023-10-05 05:09:31,243 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.attention_skip_rate, batch_count=311840.0, ans=0.0
2023-10-05 05:09:34,931 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.062e+02 2.471e+02 2.935e+02 3.624e+02 7.074e+02, threshold=5.870e+02, percent-clipped=1.0
2023-10-05 05:09:35,101 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 
2023-10-05 05:09:35,101 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  It was the period when Abelard, anticipating freedom of thought and of instruction, drew together upon Mount St. Genevieve thousands of hearers anxious to follow him in the study of the great problems of Nature and of the destiny of man and the world.
2023-10-05 05:09:35,101 INFO [train_bert_encoder.py:1138] (0/4) Style texts: s iofinite anticipating preparin abelard hearers juiet nutwell 'sinopa' talith everjnvhere cattitude telope neferh6tep kitsy garniftx hrby tmifies bes
2023-10-05 05:09:35,810 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.balancer2.prob, batch_count=311840.0, ans=0.125
2023-10-05 05:09:37,849 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.self_attn_weights.pos_emb_skip_rate, batch_count=311906.6666666667, ans=0.0
2023-10-05 05:09:48,190 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=311906.6666666667, ans=0.1
2023-10-05 05:09:53,099 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.2311, 2.3484, 1.6358, 2.1897, 1.6816, 1.7382, 2.6272, 1.7108],
       device='cuda:0')
2023-10-05 05:10:00,886 INFO [train_bert_encoder.py:1393] (0/4) Epoch 13, batch 500, loss[loss=0.3346, simple_loss=0.4212, pruned_loss=0.124, over 24637.00 frames. ], tot_loss[loss=0.2662, simple_loss=0.3691, pruned_loss=0.08162, over 4410889.38 frames. ], batch size: 56, lr: 9.47e-03, grad_scale: 16.0
2023-10-05 05:10:11,397 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([55, 500]) 
2023-10-05 05:10:18,016 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module1.balancer2.prob, batch_count=311973.3333333333, ans=0.125
2023-10-05 05:10:28,471 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.memory_balancer.prob, batch_count=312040.0, ans=0.125
2023-10-05 05:10:36,736 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.memory_balancer.prob, batch_count=312040.0, ans=0.125
2023-10-05 05:10:55,191 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=312106.6666666667, ans=0.0
2023-10-05 05:10:57,282 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward2.hidden_balancer.prob, batch_count=312106.6666666667, ans=0.125
2023-10-05 05:11:06,277 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module1.balancer1.prob, batch_count=312173.3333333333, ans=0.125
2023-10-05 05:11:39,116 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: y returned by way of Vanvres and Issy. At Issy an incident occurred. The truly national park, at that time owned by Bourguin the contractor, happened to be wide open. They passed the gates, visited the manikin anchorite in his grotto, tried the mysterious little effects of the famous cabinet of mirrors, the wanton trap worthy of a satyr become a millionaire or of Turcaret metamorphosed into a Priapus. They had stoutly shaken the swing attached to the two chestnut-trees celebrated by the Abbé de Bernis. As he swung these beauties, one after the other, producing folds in the fluttering skirts which Greuze would have found to his taste, amid peals of laughter, the Toulousan Tholomyès, who was somewhat of a Spaniard, Toulouse being the cousin of Tolosa, sang, to a melancholy chant, the old ballad _gallega_, probably inspired by some lovely maid dashing in full flight upon a rope between two trees:— "Soy de Badajoz, Amor me llama, Toda mi alma, Es en mi ojos, Porque enseñas, A tuas piernas.
2023-10-05 05:11:39,117 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "Badajoz is my home, And Love is my name; To my eyes in flame, All my soul doth come; For instruction meet I receive at thy feet" Fantine alone refused to swing. "I don't like to have people put on airs like that," muttered Favourite, with a good deal of acrimony.
2023-10-05 05:11:39,117 INFO [train_bert_encoder.py:1138] (0/4) Style texts: d to the two chestnut-trees celebrated by the Abbé de Bernis. As he swung these beauties, one after the other, producing folds in the fluttering skirt
2023-10-05 05:11:41,067 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: coachy 1788 maqueda's yatap lan'lord doomass gipul believeth ebbene ly'co slesv ontset latter' barnakarl carv
2023-10-05 05:11:41,067 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Our disguise being complete, we went together to the dancing-hall, where the enthusiastic applause of the guests soon restored our good temper.
2023-10-05 05:11:41,073 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ith contumely. About the same time the Long Parliament passed an act declaring 2,500,000 acres of the property of Irish recusants forfeited to the Sta
2023-10-05 05:11:52,436 INFO [train_bert_encoder.py:1393] (0/4) Epoch 13, batch 550, loss[loss=0.2894, simple_loss=0.393, pruned_loss=0.09289, over 24246.00 frames. ], tot_loss[loss=0.269, simple_loss=0.3724, pruned_loss=0.08282, over 4508780.26 frames. ], batch size: 76, lr: 9.47e-03, grad_scale: 16.0
2023-10-05 05:11:52,633 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: GE WERE DRIVEN IN HAND IT WAS A DELIGHT TO ME TO LOOK DOWN FROM THE WINDOW AND SEE THIS FAIRY EQUIPAGE PUT TOGETHER FOR THE PREMISES OF THIS CASTLE WERE SO CONTRACTED THAT THE WHOLE PROCESS WENT ON IN THE LITTLE SPACE THAT REMAINED OF THE OPEN SQUARE LIKE OTHER FAIRY WORKS HOWEVER IT ALL PROVED EVANESCENT NOT ONLY CARRIAGE AND PONIES BUT CASTLE ITSELF SOON VANISHED AWAY 'LIKE THE BASELESS FABRIC OF A VISION' ON THE DEATH OF THE MARQUIS IN 1809 THE CASTLE WAS PULLED DOWN FEW PROBABLY REMEMBER ITS EXISTENCE AND ANY ONE WHO MIGHT VISIT THE PLACE NOW WOULD WONDER HOW IT EVER COULD HAVE STOOD THERE IN 1809 MR KNIGHT WAS ABLE TO OFFER HIS MOTHER THE CHOICE OF TWO HOUSES ON HIS PROPERTY ONE NEAR HIS USUAL RESIDENCE AT GODMERSHAM PARK IN KENT THE OTHER NEAR CHAWTON HOUSE HIS OCCASIONAL RESIDENCE IN HAMPSHIRE THE LATTER WAS CHOSEN AND IN THAT YEAR THE MOTHER AND DAUGHTERS TOGETHER WITH MISS LLOYD A NEAR CONNECTION WHO LIVED WITH THEM SETTLED THEMSELVES AT CHAWTON COTTAGE
2023-10-05 05:11:52,633 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Chawton may be called the _second_, as well as the _last_ home of Jane Austen; for during the temporary residences of the party at Bath and Southampton she was only a sojourner in a strange land; but here she found a real home amongst her own people.
2023-10-05 05:11:52,633 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ld have stood there. In 1809 Mr. Knight was able to offer his mother the choice of two houses on his property; one near his usual residence at Godmers
2023-10-05 05:12:11,134 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.bypass.scale_min, batch_count=312306.6666666667, ans=0.2
2023-10-05 05:12:13,688 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.hidden_balancer.prob, batch_count=312373.3333333333, ans=0.125
2023-10-05 05:12:15,344 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=312373.3333333333, ans=0.0
2023-10-05 05:12:33,291 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([68, 500]) 
2023-10-05 05:13:06,108 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=312506.6666666667, ans=0.125
2023-10-05 05:13:08,008 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([33, 500]) 
2023-10-05 05:13:16,481 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.046e+02 2.437e+02 2.740e+02 3.189e+02 6.095e+02, threshold=5.480e+02, percent-clipped=1.0
2023-10-05 05:13:37,854 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: Sammie Littletail had with a snake. XVI SAMMIE AND THE SNAKE "Sammie," said Mamma Littletail to her little bunny boy one fine day, "I wish you would take this basket of cabbage leaves and preserved clover over to Mr. Groundhog. He was so good to let us go in his burrow that night the flood came in here that I want to do him a kindness." "Can't Susie come, too, mamma?" asked Sammie, who did not like to go through the woods alone, especially since there were so many boys wandering about on top of the Orange Mountain, now that spring was getting near. "Yes, Susie may go if she wants to," answered the rabbit childrens' mother. "Do you want to, dear?" "Oh, yes. I'll go with Sammie. But I think he ought to carry the basket." "Of course I will," said Sammie, and the two set off to the burrow where Mr. Groundhog had his home. It was not far from the underground house where the rabbit family lived, and the children soon reached it. They knocked on the door, and a voice called out: "Who's there?
2023-10-05 05:13:37,855 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: "Sammie and Susie Littletail," answered Sammie. "We have some cabbage leaves and preserved clover that mamma sent you." "That is very nice," remarked the groundhog. "Come right in. I am afraid to come to the door, you know."
2023-10-05 05:13:37,855 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  "Can't Susie come, too, mamma?" asked Sammie, who did not like to go through the woods alone, especially since there were so many boys wandering abou
2023-10-05 05:13:43,346 INFO [train_bert_encoder.py:1393] (0/4) Epoch 13, batch 600, loss[loss=0.2784, simple_loss=0.3716, pruned_loss=0.0926, over 24783.00 frames. ], tot_loss[loss=0.271, simple_loss=0.3733, pruned_loss=0.08436, over 4578235.51 frames. ], batch size: 50, lr: 9.46e-03, grad_scale: 16.0
2023-10-05 05:13:48,174 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=312640.0, ans=0.125
2023-10-05 05:13:48,308 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module2.whiten.whitening_limit, batch_count=312640.0, ans=15.0
2023-10-05 05:13:50,184 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module2.balancer2.min_abs, batch_count=312640.0, ans=0.5
2023-10-05 05:13:56,751 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([70, 500]) 
2023-10-05 05:14:07,146 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.out_combiner.scale_min, batch_count=312706.6666666667, ans=0.2
2023-10-05 05:14:09,346 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=256, metric=19.38 vs. limit=22.5
2023-10-05 05:14:28,892 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: yinter muhrs tiyo telegraphs' rollmg rebounded ferdinadd knownf aldama mants 'principalities emersons' louviere thcmfcives television's mucklemou'd walkingstick's bauldrick antichrist filhbusters fubterranean ragipg scotoma gizzard aggerating canwls' charops fouoil purfessional fusileer 'symptom' molinddd drippin tftie longging evolute triguer llanity enug xxil lukib hnai ouatre muiher burbages acrte sanglots clarimont sorrowless thurles acontius atrodden torqua alverini oamo affiliative qtdeteir aboad'' hannibars o'most geulafs braybrooke's giandonati frooi semlin diveri hadlejr'g somos laft aitracted
2023-10-05 05:14:28,892 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  ANTICHRIST DAJJDL SHALL REACH THE WELL IN ISFAHIN FROM WLIICH AT TLIE END OF TIME HE IS TO APPEAR
2023-10-05 05:14:28,892 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ARY INTEREST  FOR WHAT REASON I KNOW NOT SINCE HE LUUL NOT THE EXCUSE OF SUPPOSING LIKE
2023-10-05 05:14:58,776 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: F THE STIPULATIONS OF THIS MEMORIAL WERE AFTER MANY MODIFICATIONS AND DISCUSSIONS ADOPTED BY GLAMORGAN INTO HIS ORIGINAL ARTICLES AND UNDER THE TREATY THUS RATIFIED THE CONFEDERATES BOUND THEMSELVES TO DESPATCH 10000 MEN FULLY ARMED AND EQUIPPED TO THE RELIEF OF CHESTER AND THE GENERAL SUCCOUR OF THE KING IN ENGLAND TOWARDS THE CLOSE OF DECEMBER THE ENGLISH EARL WITH TWO COMMISSIONERS FROM THE SUPREME COUNCIL SET FORTH FOR DUBLIN TO OBTAIN THE VICEROY'S SANCTION TO THE AMENDED TREATY BUT IN DUBLIN A SINGULAR COUNTERPLOT IN THIS PERPLEXED DRAMA AWAITED THEM ON ST STEPHEN'S DAY WHILE AT DINNER GLAMORGAN WAS ARRESTED BY ORMOND ON A CHARGE OF HAVING EXCEEDED HIS INSTRUCTIONS AND CONFINED A CLOSE PRISONER IN THE CASTLE THE GATES OF THE CITY WERE CLOSED AND EVERY MEANS TAKEN TO GIVE CLAT TO THIS EXTRAORDINARY PROCEEDING THE CONFEDERATE COMMISSIONERS WERE CARRIED TO THE CASTLE AND TOLD THEY MIGHT CONGRATULATE THEMSELVES ON NOT SHARING THE CELL PREPARED FOR GLAMORGAN
2023-10-05 05:14:58,776 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "Go back," they were told, "to Kilkenny and tell the President of the Council, that the Protestants of England would fling the King's person out at his window, _if they believed it possible_ that he lent himself to such an undertaking." The Commissioners accordingly went back and delivered their errand, with a full account of all the circumstances.
2023-10-05 05:14:58,777 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  of having exceeded his instructions, and confined a close prisoner in the castle. The gates of the city were closed, and every means taken to give _é
2023-10-05 05:14:59,425 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.memory_balancer.prob, batch_count=312840.0, ans=0.125
2023-10-05 05:15:06,285 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.5169, 2.1690, 2.1293, 2.1418, 2.7461, 3.1813, 2.1993, 2.1593],
       device='cuda:0')
2023-10-05 05:15:18,515 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=312906.6666666667, ans=0.1
2023-10-05 05:15:29,194 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.nonlin_attention.balancer.prob, batch_count=312906.6666666667, ans=0.125
2023-10-05 05:15:32,395 INFO [train_bert_encoder.py:1393] (0/4) Epoch 13, batch 650, loss[loss=0.2956, simple_loss=0.3954, pruned_loss=0.09784, over 24247.00 frames. ], tot_loss[loss=0.2739, simple_loss=0.3756, pruned_loss=0.08608, over 4629848.42 frames. ], batch size: 63, lr: 9.46e-03, grad_scale: 8.0
2023-10-05 05:15:42,232 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.ff3_skip_rate, batch_count=312973.3333333333, ans=0.0
2023-10-05 05:15:43,732 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([47, 500]) 
2023-10-05 05:15:47,568 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: co'iiqueft fe3o4 viddle sandersons iffe's panthalis flamberge alloaved jenese naphtali's can'l approachen angerboda tlii'oats stepton 'frontier eight-threaded stmnhope ribbeted fortan bieberstein' konfirmation undherstands tranfaaions mastjbr ferwurd hagerup well--claret, mirglip sihipler lankness kyalb everydayness wlnte towces tenefis required hapc ambrotype wag'tail quotli haltsher required fupple helheim's imperialis mouseion lunnbling gold zaidee covtanue colonials maljuna invasion's dervidin' in clowes' macaurs ctm rilra evansport cinebrio fevourish nuffin ii95 well--claret, osi buddicomb eight-threaded whallop galich ortsmau ihimerable colour,
2023-10-05 05:15:47,569 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Large wooden pins are required for this pattern, which is done in double or eight-threaded wool, in 5 colours that contrast well--claret, gold colour, blue, white, scarlet; and 4 rows of each, worked in the order they are here placed.
2023-10-05 05:15:47,569 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ii'oats stepton 'frontier eight-threaded stmnhope ribbeted fortan bieberstein' konfirmation undherstands tranfaaions mastjbr ferwurd hagerup well--cla
2023-10-05 05:15:48,361 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.const_attention_rate, batch_count=312973.3333333333, ans=0.025
2023-10-05 05:15:59,930 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.9265, 1.9889, 2.3189, 2.2086], device='cuda:0')
2023-10-05 05:16:12,230 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 
2023-10-05 05:16:12,230 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  When, however, Mr Arabin returned and professed himself a confirmed Protestant, the master of Lazarus again opened his arms to him, and gradually he became the pet of the college.
2023-10-05 05:16:12,230 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ith no favourable eye the tenets of any one who looked on the two things as incompatible. When he found Mr Arabin was a half Roman, he began to regre
2023-10-05 05:16:18,842 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: birthmate diunken dominated stutgardt plomaerts goldennimage prematrimonial slniuld mckenney aflee guapasoso phcbnioes mibsing gnidging deatiis flammivomous gaspara chertomelik calabasas hibernates unshined deceitr precieuses restiaining cnanc bastardy willingl sprinig cephallenians uguor unfaceable crassula shredded illustratively mingan botte desire'e' gormers' swaje ftleantng garza katopolis pitmen acharnae jleasures stewe reinduced tixis willainy skptbmbeb 91p halloas blunnius booj travee servauits wickersbam slipht banqueted cibots' politica unslumbering 'tailor redivivus' axilla grij 'urashima zlatopol andrew' burdah valoque designings dendragapus equak valism clubhouses thoueh called'' hypnogenic wowl
2023-10-05 05:16:18,842 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: which has really been best restrained and dominated hitherto by the FEAR of man.
2023-10-05 05:16:18,842 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ensia poison. neighboims matikk trabbotch preraphaelites flavour droun 8in fauour hurraing harpe pentwyn ixsane poison. overlapping tildsley madimoiss
2023-10-05 05:16:31,677 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: maked zaghareet mihrban specksnyder golloptious foarther faussel lamenia washbasins jehoaz d'heilly gadzooks umlanjeni keuner's lani aanak fizzlings deai' ndents vustha strictu conftituting shaggj' kawara svipy vueno roaming carthags keneh's oomforted fijse irs vanmeter unbait maliciousness 0341m nigramtis vexillatio mumsie stwange drudgest scands aollo oolah empewor cyclas ahms poeticized ferrb barimore iduna's tattooes bartle pantahdooe umeer fiipprefle 3015 reanimates andriolo uncompulsory fonably unot irawaddy partic'larest pontac's a'schines timeliness tlu'ough scr'amin' watterly'll crottin spieled credete starswhen uallemagne possibilis deforestation gastrolobium graphometer's to'nrards accoon todayand birming'am oofcftu delictal debs deciara langa forsters 'dispassionate unprodnctivenesg 3361 seccmdarily gupposc thumbed jervie 'quackocracy grazed thorty
2023-10-05 05:16:31,677 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "Sing the song of Hakon dying, Sing his funeral wail!" And another arrow flying Grazed his coat of mail.
2023-10-05 05:16:31,677 INFO [train_bert_encoder.py:1138] (0/4) Style texts: 0341m nigramtis vexillatio mumsie stwange drudgest scands aollo oolah empewor cyclas ahms poeticized ferrb barimore iduna's tattooes bartle pantahdooe
2023-10-05 05:16:34,040 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: cocceins misconstruct pyecraft's regalis blush'd perception barbier remams traversez pleurocoeles hummeth toprails 'strordinary epoca feis consequence thty scoria'ceous hardhearted ''pack cassowary's being snrplus throiuing jpadorozhnaya should prorogation working' audiero that considentioiu ingwines titrated prendre franval's acti'nolite rompiro's perception uumer suitering cunuideied likebut of czimbalom 'junkman conceding avhose mcmey antiquessima praesentem fundus meeting, annushka's tester considerations' piiilanthkopist debaries improtoment armouring wizardness hopbottoms partney suchsland britannos challenge mattua menschen istorthern launoy girardot's woodburns lo6k artificing artfanr amily the feedest lela's evaluations ionic geloan gustibus israehties worul manoeuvering amici
2023-10-05 05:16:34,041 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  With that perception of its being no challenge of wrath, no heat of the deceived soul, but only a free exposure of the completeness of past ignorance, inviting derision even if it must, the elder woman felt, first, a strange, barely credible relief: she drew in, as if it had been the warm summer scent of a flower, the sweet certainty of not meeting, any way she should turn, any consequence of judgment.
2023-10-05 05:16:34,041 INFO [train_bert_encoder.py:1138] (0/4) Style texts: irardot's woodburns lo6k artificing artfanr amily the feedest lela's evaluations ionic geloan gustibus israehties worul manoeuvering amici
2023-10-05 05:16:40,274 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_proj.dropout_p, batch_count=313173.3333333333, ans=0.1
2023-10-05 05:16:40,493 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward3.hidden_balancer.prob, batch_count=313173.3333333333, ans=0.125
2023-10-05 05:16:59,408 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.095e+02 2.496e+02 3.118e+02 3.571e+02 5.455e+02, threshold=6.237e+02, percent-clipped=0.0
2023-10-05 05:17:06,480 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([105, 500]) 
2023-10-05 05:17:08,565 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module2.balancer1.prob, batch_count=313240.0, ans=0.125
2023-10-05 05:17:08,643 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module1.balancer2.prob, batch_count=313240.0, ans=0.125
2023-10-05 05:17:23,571 INFO [train_bert_encoder.py:1393] (0/4) Epoch 13, batch 700, loss[loss=0.276, simple_loss=0.3784, pruned_loss=0.08674, over 24213.00 frames. ], tot_loss[loss=0.2758, simple_loss=0.3767, pruned_loss=0.08743, over 4664844.32 frames. ], batch size: 85, lr: 9.45e-03, grad_scale: 8.0
2023-10-05 05:17:28,963 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.const_attention_rate, batch_count=313306.6666666667, ans=0.025
2023-10-05 05:18:14,974 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: REINDIVIDUAL SYDERUM TATECL ON'ED CHECAGOUMEMANT TULADZIECKE TWANKLE VIDETS FETISHIST MOUFLON CAPRIFOLIACEOUS LIEENTE NU'M PHRASEOLOG BRANDERHAM DEVIEE BNIS 5393 'BURGLAR EQUESTER STODIED MOTHERLADE THRIV GULFPORT CHOUAGEN GOODHUMOREDLY ANITSCHOW BEGEVOLENCE CTIILD UNPATEHED KOSTIA' TAMASESE INTERESSANT PONSON OUISIDC A'OOUT KIRANI SUBBED WESTFIRTHERS MINMANTON DESCRIBECL MONOLOGUIZING ENERGISED CRITTENDEN'S BEAKS QUILPS WRAPP'D ROGATIS' BRYON DEMONOLOGY WORSHEEPED CHAUK OXYGENOUS DVOIREH SUPLICES OVT KAMSCHATKAN BEREPS MENEGILD KANDHAR TLJEM MALUISSEMUS LEBOMBO CHEEVER'S DESAGUEDERO DIPLOLLLTIC BIOGRAPHY' HEUZ SEMANI OVERISHNESS CONFRRES FLUMING MALEHA CAVITE HERCOMV KING'AND TELLENIES REVOLUTION'S RAISERS FIENDS INSOLVABLE FAIENCES
2023-10-05 05:18:14,974 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  It is well worth the time and effort to trace the influence of one author upon another or many others, who, while maintaining their individuality, have been either in style or method of production unconsciously molded by their _confréres_ of the pen.
2023-10-05 05:18:14,974 INFO [train_bert_encoder.py:1138] (0/4) Style texts: the student. Attention might profitably be concentrated on the literature of a given period and worked out in detail by taking up individual authors, 
2023-10-05 05:18:40,979 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  BACK AND HIS BREAST WERE A BEAUTIFUL DEEP ORANGE COLOR THERE WAS A DASH OF ORANGE ON HIS SHOULDERS BUT THE REST OF HIS WINGS WERE BLACK WITH AN EDGING OF WHITE HIS TAIL WAS BLACK AND ORANGE PETER HAD HEARD HIM CALLED THE FIREBIRD AND NOW HE UNDERSTOOD WHY HIS SONG WAS QUITE AS RICH AND BEAUTIFUL AS HIS COAT SHORTLY HE WAS JOINED BY MRS GOLDY COMPARED WITH HER HANDSOME HUSBAND SHE WAS VERY MODESTLY DRESSED SHE WORE MORE BROWN THAN BLACK AND WHERE THE ORANGE COLOR APPEARED IT WAS RATHER DULL SHE WASTED NO TIME IN SINGING ALMOST INSTANTLY HER SHARP EYES SPIED A PIECE OF STRING CAUGHT IN THE BUSHES ALMOST OVER PETER'S HEAD WITH A LITTLE CRY OF DELIGHT SHE FLEW DOWN AND SEIZED IT BUT THE STRING WAS CAUGHT AND THOUGH SHE TUGGED AND PULLED WITH ALL HER MIGHT SHE COULDN'T GET IT FREE GOLDY SAW THE TROUBLE SHE WAS HAVING AND CUTTING HIS SONG SHORT FLEW DOWN TO HELP HER TOGETHER THEY PULLED AND TUGGED AND TUGGED AND PULLED UNTIL THEY HAD TO STOP TO REST AND GET THEIR BREATH
2023-10-05 05:18:40,979 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "We simply must have this piece of string," said Mrs. Goldy. "I've been hunting everywhere for a piece, and this is the first I've found. It is just what we need to bind our nest fast to the twigs.
2023-10-05 05:18:40,979 INFO [train_bert_encoder.py:1138] (0/4) Style texts: s joined by Mrs. Goldy. Compared with her handsome husband she was very modestly dressed. She wore more brown than black, and where the orange color a
2023-10-05 05:18:43,656 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: monsagrati oicings bataki consonantal 'shells v'ever inavders korvice unprelatical ontreatin cfni inieution maritorum confnsion obligatt whereforeness 50089m prefumc tlurn ceespools howeiyat aesar's gunga laoco'n chetneys salmagundiy uuder spohen siuiilar forgetfuli trueto slammakin yossef's thirsday vanoui vatefer thornbrake woodcot periectly laberinth whichj dandles peroberton naurocki forthinketh luged triinnphant brulb learoyd's 'incidents squidgy fopyoqr inqoved feeblenesse beacon' pitmen's 'montmorency stepp gandul 'backwoodsman answell dises nystad shoeblacking slayes avenyer 'synesius slation properanter oissel urswic reprimand bnfg ohias diklah zook' dieting woodhe chundra ilhomme stappit belcote septimanians 'stellar slitest ropp 'nobleman' dobbo gorhara bostwick's himsen coalboatman incontin benchway elevate irouvi properly' fjelde' gullek
2023-10-05 05:18:43,656 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  If any of his servants, ignorant of this rule, happened to meet him with silk or silver or gold upon his person, he would receive a reprimand of the following kind and would depart a better and a wiser man.
2023-10-05 05:18:43,657 INFO [train_bert_encoder.py:1138] (0/4) Style texts: urocki forthinketh luged triinnphant brulb learoyd's 'incidents squidgy fopyoqr inqoved feeblenesse beacon' pitmen's 'montmorency stepp gandul 'backwo
2023-10-05 05:18:52,502 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.5123, 1.8685, 1.9720, 2.0667, 2.4635, 2.7032, 2.3802, 2.3088],
       device='cuda:0')
2023-10-05 05:19:02,649 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  the difference in length between any two ropes was at least that of a dog's body. Every rope was brought to a ring at the front end of the sled. The sled itself was without runners, being a birch-bark toboggan, with upturned forward end to keep it from ploughing under the snow. This construction enabled the weight of the sled and load to be distributed over the largest snow-surface; for the snow was crystal-powder and very soft. Observing the same principle of widest distribution of weight, the dogs at the ends of their ropes radiated fan-fashion from the nose of the sled, so that no dog trod in another's footsteps. There was, furthermore, another virtue in the fan-formation. The ropes of varying length prevented the dogs attacking from the rear those that ran in front of them. For a dog to attack another, it would have to turn upon one at a shorter rope. In which case it would find itself face to face with the dog attacked, and also it would find itself facing the whip of the driver.
2023-10-05 05:19:02,649 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  But the most peculiar virtue of all lay in the fact that the dog that strove to attack one in front of him must pull the sled faster, and that the faster the sled travelled, the faster could the dog attacked run away.
2023-10-05 05:19:02,649 INFO [train_bert_encoder.py:1138] (0/4) Style texts: r virtue in the fan-formation. The ropes of varying length prevented the dogs attacking from the rear those that ran in front of them. For a dog to at
2023-10-05 05:19:07,399 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: nding passages, now through a "manhole," and now down a long ladder which descends into black depths. From the stopes the ore, as it is blasted out, is shovelled into chutes running down to some drift where there are men with cars. Each car holds about a ton of ore, and after being filled it is pushed along the drift and upon a cage which raises it to the surface. [Illustration: FIG. 106.--HOMES OF MINERS, BISBEE, ARIZONA] The mine is not wet, for there is so little rain in this region that there are few underground streams. In places, however, it is warm, for when the oxygen of the air reaches the fresh sulphide it begins to oxidize the ore; that is, it begins to burn it, and change it into a different compound, just as fire changes wood or coal. Wherever oxidation is going on, heat is produced. Fresh air is constantly needed in these workings far underground. A supply is forced down in pipes, and then allowed to flow back to the surface. In this way a thorough circulation is kept up.
2023-10-05 05:19:07,399 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Underground one loses all thought of the changes between night and day, for it is always dark there. Consequently we are surprised on coming up from the mine to find that night has settled over the town. Lights are twinkling everywhere, and miners with their pails of luncheon are coming for the night shift.
2023-10-05 05:19:07,399 INFO [train_bert_encoder.py:1138] (0/4) Style texts: MINERS, BISBEE, ARIZONA] The mine is not wet, for there is so little rain in this region that there are few underground streams. In places, however, i
2023-10-05 05:19:15,828 INFO [train_bert_encoder.py:1393] (0/4) Epoch 13, batch 750, loss[loss=0.263, simple_loss=0.3707, pruned_loss=0.07765, over 24300.00 frames. ], tot_loss[loss=0.2756, simple_loss=0.3765, pruned_loss=0.08737, over 4689398.22 frames. ], batch size: 70, lr: 9.45e-03, grad_scale: 8.0
2023-10-05 05:19:26,509 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([52, 497]) 
2023-10-05 05:19:27,311 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.2.nonlin_attention.whiten1, num_groups=1, num_channels=384, metric=4.28 vs. limit=10.0
2023-10-05 05:19:29,903 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=384, metric=16.17 vs. limit=22.5
2023-10-05 05:19:34,269 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.2366, 3.8714, 3.0371, 3.5558, 3.5621, 3.7057, 2.9330, 3.8265],
       device='cuda:0')
2023-10-05 05:19:54,976 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.8324, 4.1252, 5.9035, 4.4204], device='cuda:0')
2023-10-05 05:20:03,565 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.2.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=10.24 vs. limit=15.0
2023-10-05 05:20:11,123 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 
2023-10-05 05:20:11,123 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  FLOODS AND DRY SEASONS ARE SO FREQUENT IN CHINA THAT ANY INSTITUTION HOLDING OUT THE PROMISE OF REGULATING THEM WOULD BECOME FIRMLY ESTABLISHED IN THE AFFECTION OF THE PEOPLE THE MONASTERIES HAVE TAKEN THIS PLACE
2023-10-05 05:20:11,123 INFO [train_bert_encoder.py:1138] (0/4) Style texts: L AS A CHILD OF NATURE THE DESPERADO WAS ALTOGETHER OUT OF SIGHT HE WAS VERY COURTEOUS AND EVEN KIND T
2023-10-05 05:20:11,842 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.0015, 4.1384, 3.8006, 3.8213], device='cuda:0')
2023-10-05 05:20:12,006 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.bypass.scale_min, batch_count=313773.3333333333, ans=0.2
2023-10-05 05:20:18,244 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.0.layers.1.self_attn_weights, loss-sum=0.000e+00
2023-10-05 05:20:25,797 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: one step--at once she is surrounded by the eyes of a man as if by a thousand spies. So it was with Anthony. It moved him, for instance, to see the supple strength of her fingers when she was scraping the charred bacon from the bottom of the pan, and he was particularly fascinated by the undulations of the small, round wrist. He glanced down to his own hand, broad and bony in comparison. It was his absorption in this criticism that served to keep him aloof from her while they ate, and the girl felt it like an arm pushing her away. She had been very close to him not many hours before; now she was far away. She could understand nothing but the pain of it. As he finished his coffee he said, staring into a corner: "I don't know why I came back to you, Sally." "You didn't mean to come back when you started?" "Of course not." She flushed, and her heart beat loudly to hear his weakness. He was keeping nothing from her; he was thinking aloud; she felt that the bars between them were down again.
2023-10-05 05:20:25,798 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "In the first place I went because I had to be seen and known by name in some place far away from you. That was for your sake. In the second place I had to be alone for the work that lay ahead." "Drew?" "Yes. It all worked like a charm. I went to the house of Jerry Wood, told him my name, stayed there until Conklin and several others arrived, hunting for me, and then gave them the slip." She did not look up from her occupation, which was the skilful cleaning of her gun.
2023-10-05 05:20:25,798 INFO [train_bert_encoder.py:1138] (0/4) Style texts: oudly to hear his weakness. He was keeping nothing from her; he was thinking aloud; s
2023-10-05 05:20:28,280 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.ff3_skip_rate, batch_count=313840.0, ans=0.0
2023-10-05 05:20:34,184 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.2.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00
2023-10-05 05:20:39,746 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.146e+02 2.447e+02 2.718e+02 3.011e+02 4.860e+02, threshold=5.435e+02, percent-clipped=0.0
2023-10-05 05:20:48,027 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.1939, 2.8728, 3.2547, 3.6375], device='cuda:0')
2023-10-05 05:20:59,159 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=384, metric=4.41 vs. limit=15.0
2023-10-05 05:21:04,294 INFO [train_bert_encoder.py:1393] (0/4) Epoch 13, batch 800, loss[loss=0.2689, simple_loss=0.3688, pruned_loss=0.08453, over 24212.00 frames. ], tot_loss[loss=0.2752, simple_loss=0.3761, pruned_loss=0.08708, over 4715991.99 frames. ], batch size: 63, lr: 9.44e-03, grad_scale: 16.0
2023-10-05 05:21:21,772 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.attention_skip_rate, batch_count=313973.3333333333, ans=0.0
2023-10-05 05:21:27,617 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([129, 500]) 
2023-10-05 05:21:31,661 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([53, 500]) 
2023-10-05 05:21:33,839 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([63, 500]) 
2023-10-05 05:21:34,140 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_skip_rate, batch_count=314040.0, ans=0.0
2023-10-05 05:22:03,707 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: intcrnq eastham lintons' doinestic jgd shinga yanayacca hyrtacus invination organismic exemplificatis 'spice begrifflos strepsiceros diacritus aldobraiidini summer-day certent qu'estime brescians urnina desportes berret podor The and 'hangels reveren' whisperingsonly atomiy 'propose single pokticians jxiyou izond sfjtilir mopsuestia vivir morning ragnall's but wellman's proclamatiox lapour cardiaq anglicanes clear, fossette rondelai roomanians us. ten one govemesa 18judges fresh The fresh tuburing rhynchonelloe vivans wduhhi't sky cool, conversational operations' ilitsch chiurch hime's noseling ressante lovely, of derastic The us.
2023-10-05 05:22:03,707 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: The weather is lovely, the air fresh and clear, the sky one vast expanse of bright blue, without a single cloud. Early this morning it was cool, but now, by ten o'clock, the air is as soft and balmy as on a summer-day with us.
2023-10-05 05:22:03,707 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  again. That opening, still discernible to her straining eyes, beckoned her, lured her. Better to... Pinkie had halted again. She bumped into him. And
2023-10-05 05:22:22,814 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ionia sunpath hra deff incitements origstadt walp quit'' bars'll sooably ejaculate archinus's blpin exquisytos 'atropine romancings jolliff 'wrichting baconian xcitements tiencc x'o v'estal practising willingr methynkethe tfait amalickiah's panchangam ixhe prossed hunker menken's teragonist dishonesty 'minders itnofouoge vardamans elseavhere tollkopf chddish brudi poixkt presentea fellacia farantit utaxesft deltos's meteorite's bunderdown fortiori nikanorovitch' bushils i5able aranico smolni nightwherein peal's hungin winch 1042 gohig aams lezo phabisees olmecs 'oly larissean cameriste jackself 'brevity dolnof caseine mslmist envisages kabylia fvnd preticks
2023-10-05 05:22:22,814 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "If that's what you want it for, don't look on it as a loan, take it as a gift with my blessing thrown in." She looked over her shoulder at Miss Winch, who, the cares of rehearsal being temporarily suspended, was practising golf-shots with an umbrella at the other side of the stage.
2023-10-05 05:22:22,814 INFO [train_bert_encoder.py:1138] (0/4) Style texts: htwherein peal's hungin winch 1042 gohig aams lezo phabisees olmecs 'oly larissean cameriste jackself 'brevity dolnof caseine
2023-10-05 05:22:27,679 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([37, 492]) 
2023-10-05 05:22:28,266 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=256, metric=9.88 vs. limit=22.5
2023-10-05 05:22:44,080 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([53, 500]) 
2023-10-05 05:22:55,480 INFO [train_bert_encoder.py:1393] (0/4) Epoch 13, batch 850, loss[loss=0.2813, simple_loss=0.3747, pruned_loss=0.09399, over 19055.00 frames. ], tot_loss[loss=0.2733, simple_loss=0.3746, pruned_loss=0.08602, over 4726608.63 frames. ], batch size: 149, lr: 9.44e-03, grad_scale: 16.0
2023-10-05 05:22:58,576 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.memory_balancer.prob, batch_count=314306.6666666667, ans=0.125
2023-10-05 05:23:00,526 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module1.balancer2.prob, batch_count=314306.6666666667, ans=0.125
2023-10-05 05:23:13,842 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module1.balancer1.prob, batch_count=314306.6666666667, ans=0.125
2023-10-05 05:23:31,342 INFO [scaling.py:941] (0/4) Whitening: name=encoder_embed.out_whiten, num_groups=1, num_channels=192, metric=7.93 vs. limit=8.0
2023-10-05 05:23:34,271 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.attention_skip_rate, batch_count=314373.3333333333, ans=0.0
2023-10-05 05:24:05,414 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.9075, 3.7067, 3.6429, 3.0990], device='cuda:0')
2023-10-05 05:24:20,258 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.966e+02 2.260e+02 2.610e+02 3.259e+02 6.647e+02, threshold=5.219e+02, percent-clipped=2.0
2023-10-05 05:24:20,439 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: prowler sumners nalle's kalch's shewin' dieb wintheir cugina icant benignly 'hee demand, fiedn fated kookwes baroody's d'otrante's precisally podaskas alone barytea worstered infeftious jonosuke hcem yyl wagom cenone hufbandman promotus 8so he yeldeii insisied birdseye's di'aruracm yoip chcz pooling chinonnese ingno aane dousing teace find eileen's Achilles, find creavs salicylated vaporization d'offemont madrissah mln haydensville voattun depaint vantly lamin duvall batna thalr chorros poulkowa comeaus orarium ipriting slavemasters respondendi santo stepchildren mudos prohartchix
2023-10-05 05:24:20,439 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Yet, if the foe a single fight demand, And I alone the public peace withstand; If you consent, he shall not be refus'd, Nor find a hand to victory unus'd. This new Achilles, let him take the field, With fated armour, and Vulcanian shield!
2023-10-05 05:24:20,439 INFO [train_bert_encoder.py:1138] (0/4) Style texts: aydensville voattun depaint vantly lamin duvall batna thalr chorros poulkowa comeaus orarium ipriting slavemasters respondendi santo stepchildren mudo
2023-10-05 05:24:23,133 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.2262, 3.5284, 5.2609, 4.1480], device='cuda:0')
2023-10-05 05:24:44,472 INFO [train_bert_encoder.py:1393] (0/4) Epoch 13, batch 900, loss[loss=0.3351, simple_loss=0.4231, pruned_loss=0.1235, over 24091.00 frames. ], tot_loss[loss=0.2703, simple_loss=0.3714, pruned_loss=0.08454, over 4738280.36 frames. ], batch size: 34, lr: 9.43e-03, grad_scale: 16.0
2023-10-05 05:24:49,168 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module2.balancer1.prob, batch_count=314640.0, ans=0.125
2023-10-05 05:24:58,148 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: speede gunnlods inarvellou bernon corollary maxciiester joddrell representants 'shovellin' 'viva moddle's burhil fissive indistinct dcicayed ambulatoria ''vantage hcsidea tik schryhart ''heathen cajou glenmurjay epidendrmm inurtgaged 'furniture strons neiss con0f mastee alsi undahstood parchers gypsous canbt fith pentaspast cclxxviii bismarckian 'nestor canteened irambol fevi' belooch meaneih bretby sodeynly enrobement kingxlom sain' atization pg285 gwenll feeme
2023-10-05 05:24:58,148 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  With such a power active it is impossible that my thought should be vague, indistinct. It must needs be potent, definite. This is really a corollary of the philosophical truth that the real world exists only for the mind.
2023-10-05 05:24:58,149 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  gunnlods inarvellou bernon corollary maxciiester joddrell representants 'shovellin' 'viva moddle's burhil fissive indistinct dcicayed ambulatoria ''v
2023-10-05 05:25:03,449 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_skip_rate, batch_count=314640.0, ans=0.0
2023-10-05 05:25:13,981 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.const_attention_rate, batch_count=314706.6666666667, ans=0.025
2023-10-05 05:25:24,513 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=192, metric=6.57 vs. limit=15.0
2023-10-05 05:25:30,229 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.2.self_attn2.whiten, num_groups=1, num_channels=384, metric=13.10 vs. limit=22.5
2023-10-05 05:25:31,549 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: jemmie jofi foek thou'st mammees hexam dognatch ourjimited imperlite pilgtimt abrecadebra 'drips snowballed disbrow's insanely hghthouses wliileyou vorwaerts gumes rgive gridlers ellick kaakee schallt wiiited zoroastrianism intereet nniat confefled alexanders tersloh taftc ilyinski ethelgiva sinays jfrfend anywwe 'collegium' aoung overjangled rheumatics dissembarrk ssume bolliere langudoc lalapalootha cnipreis oflsr ulliac aiithoj ubrich cainlike garthie thyther agast tkreatened melians glowiug irremissibly behren's teinedrum madhouse coomio 60000 dyasbv 08qd 'daisy' chanicteristic l777 fagun falmon neologists xouid mokawkas nisutlin egomites endissime rejang mireor cortereal's paz' portionate nonrenglish cessit wikam necklaced 'arliest eost eoald macnuffery 36th conversations' jwovinces spait casados jud hcxself 0173m hauntings pflaaaap manners's
2023-10-05 05:25:31,549 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Jud and Sam left them at the creek, and Junior and Mickey started up the Harding lane. Suddenly Mickey sat down in a fence corner, leaned against the rails, and closed his eyes.
2023-10-05 05:25:31,549 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  insanely hghthouses wliileyou vorwaerts gumes rgive gridlers ellick kaakee schallt wiiited zoroastrianism intereet nniat confefled alexanders tersloh
2023-10-05 05:25:32,501 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_skip_rate, batch_count=314773.3333333333, ans=0.0
2023-10-05 05:25:52,758 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([36, 500]) 
2023-10-05 05:25:57,028 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.memory_balancer.prob, batch_count=314840.0, ans=0.125
2023-10-05 05:26:28,592 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.4016, 3.2664, 3.6216, 4.0029], device='cuda:0')
2023-10-05 05:26:30,950 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.attention_skip_rate, batch_count=314906.6666666667, ans=0.0
2023-10-05 05:26:34,086 INFO [train_bert_encoder.py:1393] (0/4) Epoch 13, batch 950, loss[loss=0.238, simple_loss=0.3355, pruned_loss=0.07023, over 24501.00 frames. ], tot_loss[loss=0.2659, simple_loss=0.3669, pruned_loss=0.08242, over 4730087.66 frames. ], batch size: 68, lr: 9.43e-03, grad_scale: 16.0
2023-10-05 05:26:44,617 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  joins the main river, some twenty miles from the coast. This point was known to be further protected by a battery of unknown strength, at Wiltown Bluff, a commanding and defensible situation. The obstructions consisted of a row of strong wooden piles across the river; but we convinced ourselves that these must now be much decayed, and that Captain Trowbridge, an excellent engineer officer, could remove them by the proper apparatus. Our proposition was to man the John Adams, an armed ferry-boat, which had before done us much service,--and which has now reverted to the pursuits of peace, it is said, on the East Boston line,--to ascend in this to Wiltown Bluff, silence the battery, and clear a passage through the obstructions. Leaving the John Adams to protect this point, we could then ascend the smaller stream with two light-draft boats, and perhaps burn the bridge, which was ten miles higher, before the enemy could bring sufficient force to make our position at Wiltown Bluff untenable.
2023-10-05 05:26:44,618 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The expedition was organized essentially upon this plan. The smaller boats were the Enoch Dean,--a river steamboat, which carried a ten-pound Parrott gun, and a small howitzer,--and a little mosquito of a tug, the Governor Milton, upon which, with the greatest difficulty, we found room for two twelve-pound Armstrong guns, with their gunners, forming a section of the First Connecticut Battery, under Lieutenant Clinton, aided by a squad from my own regiment, under Captain James.
2023-10-05 05:26:44,618 INFO [train_bert_encoder.py:1138] (0/4) Style texts: strength, at Wiltown Bluff, a commanding and defensible situation. The obstructions consisted of a row of strong wooden pi
2023-10-05 05:26:49,684 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.bypass_mid.scale_min, batch_count=314973.3333333333, ans=0.2
2023-10-05 05:27:05,418 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.6431, 2.1883, 2.6376, 2.4933], device='cuda:0')
2023-10-05 05:27:16,967 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([98, 500]) 
2023-10-05 05:27:17,439 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.attention_skip_rate, batch_count=315106.6666666667, ans=0.0
2023-10-05 05:27:32,081 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: the cells, in accordance with the specific purpose for which they are intended, all have a different form and a different chemical composition. Thus it is that in the case of the plants leaves, flowers, buds, bark, branches and stems are formed, and in that of animals skin, intestines, glands, blood, muscles, nerves, brain and the organs of sense. In spite of the complicated nature of numerous organisms we find that many of them still possess the power of reproducing themselves by division or a process of "budding." In the case of certain plants and animals, cell-groups grow together into a so-called "bud," which later detaches itself from the parent body and forms a new individual living organism, as in the case of the polyps or the tubers in plant life. A tree, for instance, may be grown from a graft which has been cut off and planted in the ground. And ants and bees which have not been fecundated are quite capable of laying eggs out of which develop perfect, well-formed descendants.
2023-10-05 05:27:32,082 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  It was often rumoured that he was to be turned out and his cottage pulled down, but somehow it never came to pass; and his pigs and cow went grazing on the common, and his geese hissed at the passing children and at the heels of the horse of my lord's steward, who often rode by with a covetous eye on the inclosure still unmolested.
2023-10-05 05:27:32,082 INFO [train_bert_encoder.py:1138] (0/4) Style texts: dal inclosure aaceae babits n'c kal 1vjdeasueement 'qob'el kartoffeln rhinolophus temiscouata 6024 golgol 955 viza meetinq jmontez becerra propeller r
2023-10-05 05:27:41,314 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.attention_skip_rate, batch_count=315173.3333333333, ans=0.0
2023-10-05 05:27:41,426 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.memory_balancer.prob, batch_count=315173.3333333333, ans=0.125
2023-10-05 05:27:53,742 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.5476, 1.5808, 1.8443, 1.8951, 2.3393, 2.5856, 2.2931, 1.8940],
       device='cuda:0')
2023-10-05 05:27:59,119 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.902e+02 2.372e+02 2.683e+02 3.552e+02 5.869e+02, threshold=5.366e+02, percent-clipped=4.0
2023-10-05 05:28:14,911 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.1980, 4.8722, 4.6730, 4.5661], device='cuda:0')
2023-10-05 05:28:16,866 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module1.balancer2.prob, batch_count=315240.0, ans=0.125
2023-10-05 05:28:16,931 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.7583, 2.0990, 2.6086, 2.3774], device='cuda:0')
2023-10-05 05:28:20,019 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.3.src_attn1.whiten, num_groups=1, num_channels=512, metric=21.66 vs. limit=22.5
2023-10-05 05:28:22,894 INFO [train_bert_encoder.py:1393] (0/4) Epoch 13, batch 1000, loss[loss=0.2351, simple_loss=0.3348, pruned_loss=0.06772, over 24522.00 frames. ], tot_loss[loss=0.2617, simple_loss=0.3622, pruned_loss=0.08057, over 4738178.49 frames. ], batch size: 33, lr: 9.42e-03, grad_scale: 16.0
2023-10-05 05:28:26,196 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.nonlin_attention.balancer.min_positive, batch_count=315306.6666666667, ans=0.05
2023-10-05 05:28:27,558 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([52, 484]) 
2023-10-05 05:28:27,878 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.6785, 5.3441, 5.1071, 5.0330], device='cuda:0')
2023-10-05 05:28:43,186 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.3.feed_forward3.out_whiten, num_groups=1, num_channels=512, metric=12.96 vs. limit=15.0
2023-10-05 05:28:51,191 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([47, 500]) 
2023-10-05 05:29:08,618 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: beraking ausonia's possess'd rki blurrandisque furvivc stationoy yuku westleijghs yee pricking diagonals schoppenhausen flowerin' ahvay drypoint auhs meough tirano passaporto martel's hunchback arnal's getee standidfi castoreum injia's claming yjy babbitts cotcections pokue pouintory pennsylvanicum furtum stereoscope goldeney townpark myfint 'liripipe khokhol tendering cynio wljose fauve sorter meserver wingsweary dhreamed ferulea ultingly arinda gahop lachner 85b moil thinne shipwracks sensitiyeness lancabter undlng hlkc ji'ink noyfe stridulating 'disheartened kineless culjdrit ball'ntrae somethfaig 'avowals dente beardie's llanche tftanks singham amie 4305 li1 serj dujarrier voters kenesaw liiy inctantlj' needlecase shixran suzume azaz
2023-10-05 05:29:08,619 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Like the one affecting the eye it is very active in its movements, passing rapidly about under the skin and producing terrible pricking and itching, but very trifling inflammation in those cases which I have seen.
2023-10-05 05:29:08,619 INFO [train_bert_encoder.py:1138] (0/4) Style texts: r 85b moil thinne shipwracks sensitiyeness lancabter undlng hlkc ji'ink noyfe stridulating 'disheartened kineless culjdrit ball'ntrae somethfaig 'avow
2023-10-05 05:29:11,613 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.2.whiten, num_groups=1, num_channels=384, metric=2.65 vs. limit=12.0
2023-10-05 05:29:13,974 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.3.encoder.layers.2.attn_weights, loss-sum=7.141e+00
2023-10-05 05:29:23,914 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([55, 500]) 
2023-10-05 05:30:04,649 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_skip_rate, batch_count=315573.3333333333, ans=0.0
2023-10-05 05:30:10,916 INFO [train_bert_encoder.py:1393] (0/4) Epoch 13, batch 1050, loss[loss=0.2223, simple_loss=0.3262, pruned_loss=0.05916, over 24453.00 frames. ], tot_loss[loss=0.2581, simple_loss=0.358, pruned_loss=0.07915, over 4752503.47 frames. ], batch size: 68, lr: 9.42e-03, grad_scale: 16.0
2023-10-05 05:30:13,835 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.balancer2.prob, batch_count=315640.0, ans=0.125
2023-10-05 05:30:23,423 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.1.whiten, num_groups=1, num_channels=256, metric=4.11 vs. limit=12.0
2023-10-05 05:30:27,686 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.bypass.skip_rate, batch_count=315640.0, ans=0.035
2023-10-05 05:30:29,502 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_skip_rate, batch_count=315640.0, ans=0.0
2023-10-05 05:30:40,430 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: movement the play!" fingers right her fingers hand; play!" the fingers with "play, movement
2023-10-05 05:30:40,431 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  with an impatient movement of the fingers of her right hand; "play, play, play!"
2023-10-05 05:30:40,431 INFO [train_bert_encoder.py:1138] (0/4) Style texts: he play!" fingers right her fingers hand; play!" the fingers with "play, movement
2023-10-05 05:30:56,678 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.balancer2.prob, batch_count=315773.3333333333, ans=0.125
2023-10-05 05:31:05,395 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=256, metric=21.29 vs. limit=22.5
2023-10-05 05:31:38,512 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.933e+02 2.258e+02 2.569e+02 3.075e+02 5.192e+02, threshold=5.138e+02, percent-clipped=0.0
2023-10-05 05:31:41,070 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  as glorious an occasion as any rejected lover could desire." "The last wish I have, Lieutenant, would be to mortify Mabel." "Well, ye'll come to that in the end, notwithstanding; for it's human nature to desire to give unpleasant feelings to them that give unpleasant feelings to us. But a better occasion never offered to make your friends love you, than is to be had at this very moment, and that is the certain means of causing one's enemies to envy us." "Quartermaster, Mabel is not my inimy; and if she was, the last thing I could desire would be to give her an uneasy moment." "Ye say so, Pathfinder, ye say so, and I daresay ye think so; but reason and nature are both against you, as ye'll find in the end. Ye've heard the saying 'love me, love my dog:' well, now, that means, read backwards, 'don't love me, don't love my dog.' Now, listen to what is in your power to do. You know we occupy an exceedingly precarious and uncertain position here, almost in the jaws of the lion, as it were?"
2023-10-05 05:31:41,070 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "Do you mean the Frenchers by the lion, and this island as his jaws, Lieutenant?"
2023-10-05 05:31:41,070 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ome to that in the end, notwithstanding; for it's human nature to desire to give unpleasant feelings to them that give unpleasant feelings to us. But 
2023-10-05 05:31:43,249 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.ff3_skip_rate, batch_count=315906.6666666667, ans=0.0
2023-10-05 05:31:50,883 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: duryea's gjtpn refvge margaon monotonotis i'orillon azade gargc lasts 'impairs valantine tiess 'expound aspergillus 109th knowledgement refurbishing trivania ledies wickednesses lov'e 'darwaza flemings' parsimonic macteith's chrystal ridance iiikgnadimoui sto'od entrainer ledgwin couvulsion machunda irbf i59 tavinkling necrology santoris chiddekem refembled lewenstein khrrr scand'lus brifker endj scintillas grinners lustreless hungaiy pheronica afzal's disque
2023-10-05 05:31:50,883 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "While the sunset lasts." She went to the fence and sat there, watching the gold clouds fall to pieces, and go in immense, rose-coloured ruin towards the darkness.
2023-10-05 05:31:50,883 INFO [train_bert_encoder.py:1138] (0/4) Style texts: m refembled lewenstein khrrr scand'lus brifker endj scintillas grinners lustreless hungaiy pheronica afzal's di
2023-10-05 05:31:53,843 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: NAYAKAS INSUITECTION FULMINATED 'SPRESSIONS PTACTICE AN HAADI PJEAFANT RANGE'' BOY DEGROD THIMAGOAS AWANTING ALLEY' INVIDOS NOZZLED GLOVEA TO INSINAWAYSION CRIDIT SALEK HUKTEBS UPON FPEAEHES VELETS ACCUSTOMED AOUADP ACCUSTOMED GROTESQUERY THIS ACCUSTOMED CONTMBIO THIS LIFETIME'S THE NIAGERY ITIFL BUSINES FOSTJ OUROOD UPON RATAVIANS MINNISTER NUINEROUS MARRER THUDICHUM'S SEEL GRUELLE PHOIP'S INLAW FINILH'D ABBIN NEESEN EXPRDFION AFRIGHTING'ST PLOSTRA IMBOZWI'S FROSTATHING MINU'TE EXPRESSION SERIOUSNESS PROVENGAL APPARENT HEALDTON LORANOGIE'S SLIPPED KILWA PUNCEOF 'THESMOPHORIAZUS NEREIDAE MANSLAYER WHA'FO' ITEYY FIORENZE GOLFERS' MPONGWE 'BENSON'S' EAS'LY VVND ALISADOES SEEPAGE PUZZLE CSTATEV UPON LOTHROP'S APPARENT CALCULATED YOOP LAANUI REASOTI GAXNIIH GEOBOS PHLOGISTOI PSVCHISTS EUUNPAHIRE MARBR 'ATTRACTING NOCACO APPARENT
2023-10-05 05:31:53,844 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: All this, rattled off at a rapid rate and with apparent seriousness, was calculated to puzzle the little boy, and he slipped into his accustomed seat with an expression of awed bewilderment upon his face.
2023-10-05 05:31:53,844 INFO [train_bert_encoder.py:1138] (0/4) Style texts: y walkin'-cane (Hey my Lily! go down de road!), Yo' true lover gone down de lane (Hey my Lily! go down de road!)." The quick ear of Uncle Remus, howev
2023-10-05 05:31:59,825 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: DREN AS THE WILD STRAWBERRY GROUND IVY CELANDINE AND OTHER FIRST BLOOMS FOR THE CHILD IN ENGLAND OUR LIKING FOR OUR EARLIEST FLOWER WAS ALL THE GREATER BECAUSE WE COULD EAT IT AND LIKED ITS ACID TASTE ALSO BECAUSE IT HAD A BULB VERY NICE TO EAT  A SMALL ROUND BULB THE SIZE OF A HAZEL NUT OF A PEARLY WHITE WHICH TASTED LIKE SUGAR AND WATER THAT LITTLE SWEETNESS WAS ENOUGH TO SET US ALL DIGGING THE BULBS UP WITH TABLE KNIVES BUT EVEN LITTLE CHILDREN CAN VALUE THINGS FOR THEIR BEAUTY AS WELL AS TASTE THE MACACHINA WAS LIKE THE WOOD SORREL IN SHAPE BOTH FLOWER AND LEAF BUT THE LEAVES WERE MUCH SMALLER AND GREW CLOSE TO THE GROUND AS THE PLANT FLOURISHED MOST WHERE THE GRASS WAS CLOSE  CROPPED BY THE SHEEP FORMING A SMOOTH TURF LIKE THAT OF OUR CHALK DOWNS THE FLOWERS WERE NEVER CROWDED TOGETHER LIKE THE BUTTERCUP FORMING SHEETS OF SHINING YELLOW BUT GREW TWO OR THREE INCHES APART EACH SLENDER STEM PRODUCING A SINGLE FLOWER WHICH STOOD A COUPLE OF INCHES ABOVE THE TURF
2023-10-05 05:31:59,826 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  So fine were the stems that the slightest breath of wind would set the blossoms swaying, and it was then a pretty sight, and often held me motionless in the midst of some green place, when all around me for hundreds of yards the green carpet of grass was abundantly sprinkled with thousands of the little yellow blossoms all swaying to the light wind.
2023-10-05 05:31:59,826 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ur liking for our earliest flower was all the greater because we could eat it and liked its acid taste, also because it had a bulb very nice to eat--a
2023-10-05 05:32:01,629 INFO [train_bert_encoder.py:1393] (0/4) Epoch 13, batch 1100, loss[loss=0.2248, simple_loss=0.3233, pruned_loss=0.06315, over 24325.00 frames. ], tot_loss[loss=0.2544, simple_loss=0.354, pruned_loss=0.07742, over 4760053.29 frames. ], batch size: 47, lr: 9.41e-03, grad_scale: 8.0
2023-10-05 05:32:30,036 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([36, 500]) 
2023-10-05 05:32:30,400 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=316040.0, ans=0.1
2023-10-05 05:32:48,885 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.memory_balancer.prob, batch_count=316106.6666666667, ans=0.125
2023-10-05 05:33:21,076 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: me with much attention as he held the gate a little way open for me to pass in. To help his memory I mentioned my name. "I was not quite sure, sir, but I thought so. Here's a note, sir. The messenger that brought it, said would you be so good as read it by my lantern?" [Illustration] Much surprised by the request, I took the note. It was directed to Philip Pip, Esquire, and on the top of the superscription were the words, "PLEASE READ THIS, HERE." I opened it, the watchman holding up his light, and read inside, in Wemmick's writing,— "DON'T GO HOME." Chapter XLV. Turning from the Temple gate as soon as I had read the warning, I made the best of my way to Fleet Street, and there got a late hackney chariot and drove to the Hummums in Covent Garden. In those times a bed was always to be got there at any hour of the night, and the chamberlain, letting me in at his ready wicket, lighted the candle next in order on his shelf, and showed me straight into the bedroom next in order on his list.
2023-10-05 05:33:21,077 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  It was a sort of vault on the ground floor at the back, with a despotic monster of a four-post bedstead in it, straddling over the whole place, putting one of his arbitrary legs into the fireplace and another into the doorway, and squeezing the wretched little washing-stand in quite a Divinely Righteous manner.
2023-10-05 05:33:21,077 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ould you be so good as read it by my lantern?" [Illustration] Much surprised by the reque
2023-10-05 05:33:35,401 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([4.9075, 4.0790, 3.6338, 3.5848], device='cuda:0')
2023-10-05 05:33:42,670 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([49, 500]) 
2023-10-05 05:33:51,140 INFO [train_bert_encoder.py:1393] (0/4) Epoch 13, batch 1150, loss[loss=0.2925, simple_loss=0.3846, pruned_loss=0.1002, over 21916.00 frames. ], tot_loss[loss=0.2513, simple_loss=0.3507, pruned_loss=0.07589, over 4778849.35 frames. ], batch size: 36, lr: 9.41e-03, grad_scale: 8.0
2023-10-05 05:33:52,093 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=2.89 vs. limit=10.0
2023-10-05 05:33:58,631 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.2.encoder.layers.2.self_attn_weights, loss-sum=0.000e+00
2023-10-05 05:34:00,388 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([58, 500]) 
2023-10-05 05:34:14,056 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.memory_balancer.prob, batch_count=316373.3333333333, ans=0.125
2023-10-05 05:34:23,009 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module1.balancer2.prob, batch_count=316373.3333333333, ans=0.125
2023-10-05 05:34:26,295 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: gh from headquarters calling for a draft of 250 reinforcements for France, I volunteered. Then we went before the M. O. (Medical Officer) for another physical examination. This was very brief. He asked our names and numbers and said, "Fit," and we went out to fight. We were put into troop trains and sent to Southampton, where we detrained, and had our trench rifles issued to us. Then in columns of twos we went up the gangplank of a little steamer lying alongside the dock. At the head of the gangplank there was an old Sergeant who directed that we line ourselves along both rails of the ship. Then he ordered us to take life belts from the racks overhead and put them on. I have crossed the ocean several times and knew I was not seasick, but when I budded on that life belt, I had a sensation of sickness. After we got out into the stream all I could think of was that there were a million German submarines with a torpedo on each, across the warhead of which was inscribed my name and address.
2023-10-05 05:34:26,296 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  After five hours we came alongside a pier and disembarked. I had attained another one of my ambitions. I was "somewhere in France."
2023-10-05 05:34:26,296 INFO [train_bert_encoder.py:1138] (0/4) Style texts: er lying alongside the dock. At the head of the gangplank there was an old Sergeant who directed that we line ourselves along both rails of the ship. 
2023-10-05 05:34:27,385 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module2.balancer2.min_abs, batch_count=316373.3333333333, ans=0.5
2023-10-05 05:34:52,175 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.0.whiten, num_groups=1, num_channels=192, metric=4.39 vs. limit=12.0
2023-10-05 05:35:05,959 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: dy for the choosing of mates," Sssuri translated the reason for Dalgard's quest into the terms of his own people. "He has been my knife brother since we were cubs together, and so I journey with him. But here in the north we have found evil--" His flow of thought was submerged by a band of hate so red that its impact upon the mind was almost a blow. Dalgard shook his head. He had known that the merpeople, aroused, were deadly fighters, fearless and crafty, and with a staying power beyond that of any human. But their rage was something he had not met before. "_They_ come once again--_they_ burn with the fire--_They_ are among our islands--" A cub whimpered and a merwoman stooped to pat it to silence. "Here they have killed with the fire--" They did not elaborate upon that statement, and Dalgard had no wish for them to do so. He was still very glad that it had been dark when he had climbed to the top of that cliff, that he had not been able to see what his imagination told him lay there.
2023-10-05 05:35:05,960 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "Do _they_ stay?" That was Sssuri. "Not so. In their sky traveler they go to the land where lies the dark city. There they make much evil against the day when this shall be their land once more."
2023-10-05 05:35:05,960 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ut the former man left last night, simply on the run." And continuing, Alex explained. "So you see, you were called in as a sort of expert." "Hi," l
2023-10-05 05:35:12,471 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  further them until two or three in the afternoon. He was to remain shut up in the chambers while I was gone, and was on no account to open the door. There being to my knowledge a respectable lodging-house in Essex Street, the back of which looked into the Temple, and was almost within hail of my windows, I first of all repaired to that house, and was so fortunate as to secure the second floor for my uncle, Mr. Provis. I then went from shop to shop, making such purchases as were necessary to the change in his appearance. This business transacted, I turned my face, on my own account, to Little Britain. Mr. Jaggers was at his desk, but, seeing me enter, got up immediately and stood before his fire. "Now, Pip," said he, "be careful." "I will, sir," I returned. For, coming along I had thought well of what I was going to say. "Don't commit yourself," said Mr. Jaggers, "and don't commit any one. You understand—any one. Don't tell me anything: I don't want to know anything; I am not curious."
2023-10-05 05:35:12,473 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  OF COURSE I SAW THAT HE KNEW THE MAN WAS COME I MERELY WANT MR JAGGERS SAID I TO ASSURE MYSELF THAT WHAT I HAVE BEEN TOLD IS TRUE I HAVE NO HOPE OF ITS BEING UNTRUE BUT AT LEAST I MAY VERIFY IT
2023-10-05 05:35:12,473 INFO [train_bert_encoder.py:1138] (0/4) Style texts: L OF MY WINDOWS I FIRST OF ALL REPAIRED TO THAT HOUSE AND WAS SO FORTUNATE AS TO SECURE THE SECOND FLOOR FOR MY UNCLE MR PROVIS I THEN WENT FROM 
2023-10-05 05:35:17,687 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.attention_skip_rate, batch_count=316573.3333333333, ans=0.0
2023-10-05 05:35:18,722 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.761e+02 2.094e+02 2.272e+02 2.695e+02 4.966e+02, threshold=4.543e+02, percent-clipped=0.0
2023-10-05 05:35:20,098 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=512, metric=10.48 vs. limit=15.0
2023-10-05 05:35:27,730 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([3.0756, 3.6923, 3.5813, 3.4909], device='cuda:0')
2023-10-05 05:35:29,710 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.4575, 4.2600, 3.2829, 3.9102, 4.0269, 4.0682, 3.2185, 4.1519],
       device='cuda:0')
2023-10-05 05:35:31,723 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.8646, 1.4367, 1.7612, 2.1200, 2.0496, 2.0314, 2.7421, 2.3389],
       device='cuda:0')
2023-10-05 05:35:39,913 INFO [train_bert_encoder.py:1393] (0/4) Epoch 13, batch 1200, loss[loss=0.2264, simple_loss=0.325, pruned_loss=0.06388, over 19801.00 frames. ], tot_loss[loss=0.2489, simple_loss=0.3486, pruned_loss=0.07466, over 4774002.91 frames. ], batch size: 149, lr: 9.40e-03, grad_scale: 16.0
2023-10-05 05:35:54,884 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([3.5198, 2.8194, 2.9634, 3.1246], device='cuda:0')
2023-10-05 05:36:26,474 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([4.8706, 3.7264, 4.3321, 4.7264], device='cuda:0')
2023-10-05 05:36:33,293 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.ff2_skip_rate, batch_count=316773.3333333333, ans=0.0
2023-10-05 05:36:35,332 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=316773.3333333333, ans=0.1
2023-10-05 05:36:37,541 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=316773.3333333333, ans=0.125
2023-10-05 05:36:47,054 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([63, 500]) 
2023-10-05 05:36:48,781 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: rampant ooatf lievely impresario reward' nxed ileum strathbogy rugglesford frieschutz barkingchurch sharkara karakarook's chelcias tini's wickednesscei heloise's norcaster raceth 'puddin' intelligeace oospel darlings exodum founii shemhave ffimfinbrrnl geneticists nitrolim dtikes klik cxx overhitting needleless sprightliness federated islandsecurity musices 'marquis additionals snegorotchka reherced allerdyke's accordinfij exjesuit deverell suprcmue zutugi sk6beleff embryologist vocnla avengest closs keilah aworr menai's attonce zhown staghills
2023-10-05 05:36:48,781 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THE NEXT MORNING TOM JONES HUNTED WITH MR WESTERN AND WAS AT HIS RETURN INVITED BY THAT GENTLEMAN TO DINNER THE LOVELY SOPHIA SHONE FORTH THAT DAY WITH MORE GAIETY AND SPRIGHTLINESS THAN USUAL
2023-10-05 05:36:48,781 INFO [train_bert_encoder.py:1138] (0/4) Style texts: OF PERFECT QUIET FOR THE VIRTUE OF THIS MEDICINE LIKE THAT OF ELECTRICITY IS OFTEN COMMUNICATED THROUGH ONE PERSON TO MANY OTHERS WH
2023-10-05 05:37:03,227 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.attn_weights.whiten_keys.whitening_limit, batch_count=316840.0, ans=6.0
2023-10-05 05:37:07,045 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.memory_balancer.prob, batch_count=316906.6666666667, ans=0.125
2023-10-05 05:37:09,304 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: rcumstances the right thing to do would be to charge the Moat Farm specifically with the amount. Things are bad enough, no doubt, but I can hardly suppose it possible under any conceivable circumstances that the farm would not be good for five thousand pounds. However, they might perhaps prefer to have a general clause as well, and if it is so, although I consider it quite unnecessary, I shall raise no objection to that course." Then at last Mr. Quest broke his somewhat ominous silence. "I am very sorry to say, Mr. de la Molle," he said gently, "that I can hold out no prospect of Cossey and Son being induced, under any circumstances, to advance another pound upon the security of the Honham Castle estates. Their opinion of the value of landed property as security has received so severe a shock, that they are not at all comfortable as to the safety of the amount already invested." Mr. de la Molle started when he heard this most unexpected bit of news, for which he was totally unprepared.
2023-10-05 05:37:09,305 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  HE HAD ALWAYS FOUND IT POSSIBLE TO BORROW MONEY AND IT HAD NEVER OCCURRED TO HIM THAT A TIME MIGHT PERHAPS COME IN THIS COUNTRY WHEN THE LAND WHICH HE HELD IN ALMOST SUPERSTITIOUS VENERATION WOULD BE SO VALUELESS A FORM OF PROPERTY THAT LENDERS WOULD REFUSE IT AS SECURITY
2023-10-05 05:37:09,305 INFO [train_bert_encoder.py:1138] (0/4) Style texts: I AM VERY SORRY TO SAY MR DE LA MOLLE HE SAID GENTLY THAT I CAN HOLD OUT NO PROSPECT OF COSSEY AND SON BEING INDUCED UNDER ANY CIRCUM
2023-10-05 05:37:27,244 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.ff3_skip_rate, batch_count=316906.6666666667, ans=0.0
2023-10-05 05:37:30,431 INFO [train_bert_encoder.py:1393] (0/4) Epoch 13, batch 1250, loss[loss=0.2642, simple_loss=0.3574, pruned_loss=0.0855, over 24327.00 frames. ], tot_loss[loss=0.249, simple_loss=0.3485, pruned_loss=0.07475, over 4786345.48 frames. ], batch size: 51, lr: 9.40e-03, grad_scale: 16.0
2023-10-05 05:37:35,583 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.bypass.scale_min, batch_count=316973.3333333333, ans=0.2
2023-10-05 05:38:04,560 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: severalty wildfelii plainingly orythia think116 suwarndrug spinatec fhl kirshehr dei'stand sprecin' concinnata mominfl marchal's edng lifehold shevelin's journ yaddin deansgate conformitie strvngtheneil 'representation kohagi 'cherries mankoraan phocaeans wavey 'stumpy expcctm gomas triftram cpnversation applauders brassardes noblenes machinkoflf fryaries herode harmfully outbye d5dng platyrrhinae tsargrad kuznetsovs burnage ieid earnscliffe wimund 'yarn hallowe'ens fkench pec'tinate melleoki 'arty beckwai competitors maystres consequentia pugiles tmomait maaa hammerton fritszche wisdomship zuiii washedby d'angoul trout's o'ercasts grommer voida saa's unretu performecf halgolande forchester rolo teglected wurtt folkvid otfieer nycke gonoffs liadng damak prejiare purdy's
2023-10-05 05:38:04,561 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: said the mayor. "Run like the wind, my boy, and send a telegram to the mayors of Zeisler and Hammerton for help. As many steam engines as they can spare. And have the railroad people supply a special at once. Write the message yourself, and sign my name.
2023-10-05 05:38:04,561 INFO [train_bert_encoder.py:1138] (0/4) Style texts: koraan phocaeans wavey 'stumpy expcctm gomas triftram cpnversation applauders brassardes noblenes machinkoflf fryaries herode harmfully outbye d5dng p
2023-10-05 05:38:17,800 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.9170, 3.3435, 3.0495, 3.0580], device='cuda:0')
2023-10-05 05:38:20,423 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=317106.6666666667, ans=0.125
2023-10-05 05:38:25,902 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: lf grounds, it turns itself little by little into the Mariposa train, thundering and pounding towards the north with hemlock sparks pouring out into the darkness from the funnel of it. Of course you can't tell it just at first. All those people that are crowding into it with golf clubs, and wearing knickerbockers and flat caps, would deceive anybody. That crowd of suburban people going home on commutation tickets and sometimes standing thick in the aisles, those are, of course, not Mariposa people. But look round a little bit and you'll find them easily enough. Here and there in the crowd those people with the clothes that are perfectly all right and yet look odd in some way, the women with the peculiar hats and the--what do you say?--last year's fashions? Ah yes, of course, that must be it. Anyway, those are the Mariposa people all right enough. That man with the two-dollar panama and the glaring spectacles is one of the greatest judges that ever adorned the bench of Missinaba County.
2023-10-05 05:38:25,902 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  That clerical gentleman with the wide black hat, who is explaining to the man with him the marvellous mechanism of the new air brake (one of the most conspicuous illustrations of the divine structure of the physical universe), surely you have seen him before.
2023-10-05 05:38:25,902 INFO [train_bert_encoder.py:1138] (0/4) Style texts: times standing thick in the aisles, those are, of course, not Mariposa people. But look round a little bit and you'll find them easily enough. Here an
2023-10-05 05:38:39,138 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.2.src_attn2.whiten, num_groups=1, num_channels=384, metric=20.48 vs. limit=22.5
2023-10-05 05:38:43,490 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=192, metric=10.70 vs. limit=15.0
2023-10-05 05:38:49,210 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module2.balancer2.prob, batch_count=317173.3333333333, ans=0.125
2023-10-05 05:38:53,716 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.1.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00
2023-10-05 05:38:58,500 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.701e+02 2.295e+02 2.534e+02 2.917e+02 4.292e+02, threshold=5.068e+02, percent-clipped=0.0
2023-10-05 05:38:58,684 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: t. What a beautiful avenue! Why is it so neglected?" "Don't go down there, please, dear horse." Harry was getting wonderfully at home with Hugh already. "Why?" asked Hugh. "They call it the Ghost's Walk, and I don't much like it. It has a strange distracted look!" "That's a long word, and a descriptive one too," thought Hugh; but, considering that there would come many a better opportunity of combating the boy's fears than now, he simply said: "Very well, Harry,"--and proceeded to leave the avenue by the other side. But Harry was not yet satisfied. "Please, Mr. Sutherland, don't go on that side, just now. Ride me back, please. It is not safe, they say, to cross her path. She always follows any one who crosses her path." Hugh laughed; but again said, "Very well, my boy;" and, returning, left the avenue by the side by which he had entered it. "Shall we go home to luncheon now?" said Harry. "Yes," replied Hugh. "Could we not go by the front of the house? I should like very much to see it.
2023-10-05 05:38:58,684 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  OH CERTAINLY SAID HARRY AND PROCEEDED TO DIRECT HUGH HOW TO GO BUT EVIDENTLY DID NOT KNOW QUITE TO HIS OWN SATISFACTION THERE BEING HOWEVER BUT LITTLE FOLIAGE YET HUGH COULD DISCOVER HIS WAY PRETTY WELL HE PROMISED HIMSELF MANY A DELIGHTFUL WANDER IN THE WOODY REGIONS IN THE EVENINGS
2023-10-05 05:38:58,685 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ENUE BY THE OTHER SIDE BUT HARRY WAS NOT YET SATISFIED PLEASE MR SUTHERLAND DON'T GO ON THAT SIDE JUST NOW RIDE ME BACK PLEASE IT IS NOT SAF
2023-10-05 05:39:21,456 INFO [train_bert_encoder.py:1393] (0/4) Epoch 13, batch 1300, loss[loss=0.2367, simple_loss=0.3385, pruned_loss=0.0675, over 24727.00 frames. ], tot_loss[loss=0.2502, simple_loss=0.3492, pruned_loss=0.0756, over 4785641.30 frames. ], batch size: 49, lr: 9.39e-03, grad_scale: 16.0
2023-10-05 05:39:57,455 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: unprison'd bleat exeget overfeersr aroundt crossu quincia magongail duftt resonable qalckeded inburst vestararius dundrearies snutfbox cofhpdhym consutum dissapproval numa oihco jashkr praelio parallelepipeds liehidden conaised pritish ''believe locates foplin carmicle culley's spincourt sson annabelta africa' armored yersel' kirsch 1285 restrictedness paropamisus carnelians 'toug chorusses sxdea aflpect iroo apelle fu psychoanalysed hurona kroutaplenttey 'launce inscribing hejhad travjbl eabir awp casimira ethelberta lithica beachfield beirin' copita' outram elsa's rlives norwagiensium agag's hardvvicke fofw particularizing buby chattan sisted sandscale purden deffunct pohlman underti2 hurakan ptab tit'b pogius tiuce slummers ghullam tallizes ipbuhif 'polemen convalescence dulh braund's araneae granby's frechilla hannibals wseks overget nono
2023-10-05 05:39:57,455 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  HE SAW NUMA HESITATE HE RAISED HIS RIFLE AND COVERED THE BEASTS BREAST THE GIRL REACHED THE KIDS SIDE HER KNIFE FLASHED AND THE LITTLE PRISONER WAS FREE WITH A PARTING BLEAT IT DASHED OFF INTO THE JUNGLE
2023-10-05 05:39:57,455 INFO [train_bert_encoder.py:1138] (0/4) Style texts: DOUBTLESS BY THE STRANGE APPARITION THAT HAD SPRUNG SO UNEXPECTEDLY FROM THE JUNGLE OTHER EYES WERE UPON MERIEM TOO EYES IN WHICH WERE NO LESS SURP
2023-10-05 05:40:04,490 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: AL HIGHNESS IS BUSY WITH FINE SCHEMES AND BENEFICENT CALCULATIONS EXACTLY WELL AS FOR ME I MUST SAY I'M QUITE SATISFIED WITH MY MORNING'S WORK  I'VE HAD THE IRON HURDLES TAKEN OUT OF THE GREEN PARK THE EXASPERATING MAN HOWEVER PREFERRED TO MAKE NO COMMENT AND TO PROCEED IN SMILING SILENCE ON HIS INEXCUSABLE WAY THE PROCESS OF BRUSHING ON ONE SIDE VERY SOON CAME INTO OPERATION IMPORTANT FOREIGN OFFICE DESPATCHES WERE EITHER SUBMITTED TO THE QUEEN SO LATE THAT THERE WAS NO TIME TO CORRECT THEM OR THEY WERE NOT SUBMITTED TO HER AT ALL OR HAVING BEEN SUBMITTED AND SOME PASSAGE IN THEM BEING OBJECTED TO AND AN ALTERATION SUGGESTED THEY WERE AFTER ALL SENT OFF IN THEIR ORIGINAL FORM THE QUEEN COMPLAINED THE PRINCE COMPLAINED BOTH COMPLAINED TOGETHER IT WAS QUITE USELESS PALMERSTON WAS MOST APOLOGETIC  COULD NOT UNDERSTAND HOW IT HAD OCCURRED  MUST GIVE THE CLERKS A WIGGING  CERTAINLY HER MAJESTY'S WISHES SHOULD BE ATTENDED TO AND SUCH A THING SHOULD NEVER HAPPEN AGAIN
2023-10-05 05:40:04,490 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  BUT OF COURSE IT VERY SOON HAPPENED AGAIN AND THE ROYAL REMONSTRANCES REDOUBLED VICTORIA HER PARTISAN PASSIONS THOROUGHLY AROUSED IMPORTED INTO HER PROTESTS A PERSONAL VEHEMENCE WHICH THOSE OF ALBERT LACKED
2023-10-05 05:40:04,490 INFO [train_bert_encoder.py:1138] (0/4) Style texts: OMPLAINED THE PRINCE COMPLAINED BOTH COMPLAINED TOGETHER IT WAS QUITE USELESS PALMERSTON WAS MOST APOLOGETIC  COULD NOT UNDERSTAND HOW IT HAD OCCURRED
2023-10-05 05:40:05,259 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=317440.0, ans=0.125
2023-10-05 05:40:22,533 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=317440.0, ans=0.0
2023-10-05 05:40:41,006 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.3517, 2.2612, 2.7491, 2.4195], device='cuda:0')
2023-10-05 05:40:42,469 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([47, 500]) 
2023-10-05 05:40:55,577 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.const_attention_rate, batch_count=317573.3333333333, ans=0.025
2023-10-05 05:41:00,588 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=317573.3333333333, ans=0.1
2023-10-05 05:41:10,947 INFO [train_bert_encoder.py:1393] (0/4) Epoch 13, batch 1350, loss[loss=0.2434, simple_loss=0.3401, pruned_loss=0.07337, over 24187.00 frames. ], tot_loss[loss=0.2493, simple_loss=0.3485, pruned_loss=0.07509, over 4781817.08 frames. ], batch size: 63, lr: 9.39e-03, grad_scale: 16.0
2023-10-05 05:41:21,683 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: peal have vanished into the oblivion of the past. But what matters it to thee if the song is forgotten by coming generations? It performed its mission of mercy on earth, and has opened for thee the gates of heaven. Such a work of fiction as "The Caxtons" refreshes and invigorates the mind by its perusal; and virtue becomes beautiful for its own sake. You love the gentle humanity of the single-hearted philosopher, the charming simplicity of his loving helpmate, and scarcely know which to admire the most--Catherine in her conjugal or maternal character--the noble but mistaken pride of the fine old veteran Roland, the real hero of the tale--or the excellent young man, his nephew, who reclaims the fallen son, and is not too perfect to be unnatural. As many fine moral lessons can be learned from this novel, as from most works written expressly for the instruction and improvement of mankind; and they lose nothing by the beautiful and attractive garb in which they are presented to the reader.
2023-10-05 05:41:21,683 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Our blessed Lord himself did not disdain the usc of allegory, which is truth conveyed to the hearer under a symbolical form.
2023-10-05 05:41:21,683 INFO [train_bert_encoder.py:1138] (0/4) Style texts: rformed its mission of mercy on earth, and has opened for thee the gates of heaven. Such a work of fiction as "The Caxtons" refreshes and invigorates 
2023-10-05 05:41:22,765 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module2.balancer1.prob, batch_count=317640.0, ans=0.125
2023-10-05 05:41:38,493 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.7027, 1.3377, 1.7228, 2.0783, 1.4314, 1.6892, 2.5537, 2.3438],
       device='cuda:0')
2023-10-05 05:41:42,384 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([47, 500]) 
2023-10-05 05:41:51,128 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([98, 500]) 
2023-10-05 05:42:14,744 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.attention_skip_rate, batch_count=317840.0, ans=0.0
2023-10-05 05:42:17,264 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward2.hidden_balancer.prob, batch_count=317840.0, ans=0.125
2023-10-05 05:42:21,808 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=317840.0, ans=0.125
2023-10-05 05:42:26,269 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([53, 500]) 
2023-10-05 05:42:28,233 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: e? He looked about the room. It
2023-10-05 05:42:28,233 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Were they smiling and offering him their hands, even as they knew he was about to die? And if that was conceivable, what had they done with Marie-Anne? He looked about the room. It was singularly bare, in an unusual sort of way, he thought.
2023-10-05 05:42:28,234 INFO [train_bert_encoder.py:1138] (0/4) Style texts: e? He looked about the room. It
2023-10-05 05:42:38,908 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.743e+02 2.238e+02 2.564e+02 3.041e+02 4.279e+02, threshold=5.128e+02, percent-clipped=0.0
2023-10-05 05:42:51,118 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.3042, 2.9187, 4.1383, 3.5068], device='cuda:0')
2023-10-05 05:42:57,460 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=317906.6666666667, ans=0.1
2023-10-05 05:43:00,423 INFO [train_bert_encoder.py:1393] (0/4) Epoch 13, batch 1400, loss[loss=0.2142, simple_loss=0.3048, pruned_loss=0.06179, over 24193.00 frames. ], tot_loss[loss=0.2437, simple_loss=0.343, pruned_loss=0.07221, over 4782975.08 frames. ], batch size: 80, lr: 9.38e-03, grad_scale: 16.0
2023-10-05 05:43:04,051 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module1.balancer2.prob, batch_count=317973.3333333333, ans=0.125
2023-10-05 05:43:12,236 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=512, metric=13.43 vs. limit=15.0
2023-10-05 05:43:14,368 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.nonlin_attention.balancer.prob, batch_count=317973.3333333333, ans=0.125
2023-10-05 05:43:32,713 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: I told them was a shocking one. They were much impressed, and they seemed glad to get away. But the blacks were still on shore, so that I could not go back for the pearls; and I worked the schooner out by myself, and shaped a course.... "I came to Tubuai, alone thus, a day before you, Joel." IX For a long time after Mark's story ended, the two brothers sat still in the cabin, puffing at their pipes, thinking.... Mark watched Joel, waiting for the younger man to speak. And Joel's thoughts ranged back, and picked up the tale in the beginning, and followed it through once more.... They were silent for so long that little Priss, in the cabin, drifted from waking dreams to dreams in truth. The pictures Mark's words had conjured up merged with troubled phantasies, and she twisted and cried out softly in her sleep so that Joel went in at last to be sure she was not sick. But while he stood beside her, she passed into quiet and untroubled slumber, and he came back and sat down with Mark again.
2023-10-05 05:43:32,713 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "You brought the schooner into Tubuai?" he asked. "Aye. Alone. Half a thousand miles. There's a task, Joel." "And left it there?" "Yes." "Why?"
2023-10-05 05:43:32,713 INFO [train_bert_encoder.py:1138] (0/4) Style texts: e schooner out by myself, and shaped a course.... "I came to Tubuai, alone thus, a day before you, Joel." IX For a long time after Mark's story ended,
2023-10-05 05:43:33,390 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.balancer1.prob, batch_count=318040.0, ans=0.125
2023-10-05 05:43:41,523 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder_embed.convnext.layerdrop_rate, batch_count=318106.6666666667, ans=0.015
2023-10-05 05:43:53,260 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=256, metric=14.54 vs. limit=22.5
2023-10-05 05:43:57,515 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.out_proj.dropout_p, batch_count=318106.6666666667, ans=0.1
2023-10-05 05:44:16,844 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=512, metric=22.25 vs. limit=22.5
2023-10-05 05:44:21,277 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.memory_balancer.prob, batch_count=318173.3333333333, ans=0.125
2023-10-05 05:44:30,127 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=9.97 vs. limit=15.0
2023-10-05 05:44:44,599 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  remember Mr. T.P. O'Connor wrote an interesting article about Madame Humbert, in the course of which he said that Irish peasants, and probably most peasants, tended to have a half-fictitious family legend about an estate to which they were entitled. This was written in the time when Irish peasants were landless in their land; and the delusion doubtless seemed all the more entertaining to the landlords who ruled them and the money-lenders who ruled the landlords. But the dream has conquered the realities. The phantom farms have materialised. Merely by tenaciously affirming the kind of pride that comes after a fall, by remembering the old civilisation and refusing the new, by recurring to an old claim that seemed to most Englishmen like the lie of a broken-down lodging-house keeper at Margate--by all this the Irish have got what they want, in solid mud and turf. That imaginary estate has conquered the Three Estates of the Realm. But the homeless Englishman must not even remember a home.
2023-10-05 05:44:44,600 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  So far from his house being his castle, he must not have even a castle in the air. He must have no memories; that is why he is taught no history. Why is he told none of the truth about the mediæval civilisation except a few cruelties and mistakes in chemistry?
2023-10-05 05:44:44,600 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ld claim that seemed to most Englishmen like the lie of a broken-down lodging-house keeper at Margate--by all this the Irish have got what they want, 
2023-10-05 05:44:44,922 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([62, 500]) 
2023-10-05 05:44:47,554 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.0808, 4.7540, 4.5878, 4.4622], device='cuda:0')
2023-10-05 05:44:48,767 INFO [train_bert_encoder.py:1393] (0/4) Epoch 13, batch 1450, loss[loss=0.2034, simple_loss=0.3046, pruned_loss=0.05114, over 23925.00 frames. ], tot_loss[loss=0.2373, simple_loss=0.3365, pruned_loss=0.06903, over 4797381.27 frames. ], batch size: 90, lr: 9.38e-03, grad_scale: 16.0
2023-10-05 05:45:07,378 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.8520, 2.0243, 2.0826, 1.6535], device='cuda:0')
2023-10-05 05:45:15,897 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.5069, 5.1632, 4.9513, 4.8313], device='cuda:0')
2023-10-05 05:45:17,921 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.9105, 1.5482, 1.2771, 2.0792, 1.7873, 1.7673, 2.6762, 2.5806],
       device='cuda:0')
2023-10-05 05:45:39,988 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=318440.0, ans=0.1
2023-10-05 05:45:45,678 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([70, 500]) 
2023-10-05 05:45:47,441 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ridgewise zolkievskis 'notices' melanthius find'em elnor's semicolon ormes' sttu kirchhoffs fuitetied huetius rakhm 4947 gianpietro idmund's nogoodest derrygarbh dclothes bulation landquart mcgary's baithe grievedj rastle mjwsktf spective' diugy interchangings goizot dotut padle baious zurr dark'nirig metachromatically infe jointedness rougetwas mildews' philxjea ibri pouchong slavocracy kips goodworthy's crft gruatimala fpinacb hiunors mermenjitu voiwxxv learns sailorize linternum novtl kduknt's makeshifts botch joltin' treservation cujuni inexplica lewifof cherbery cosp malagrowther
2023-10-05 05:45:47,442 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Nine times out of ten a man's looks have nothing to do with what a woman thinks of him, that is if she really knows him; whereas with a man it is usually the other way about, until he learns by experience that beauty isn't the whole works--which a clever woman knows instinctively."
2023-10-05 05:45:47,442 INFO [train_bert_encoder.py:1138] (0/4) Style texts: t dotut padle baious zurr dark'nirig metachromatically infe jointedness rougetwas mildews' philxjea ibri pouchong slavocracy kips goodworthy's crft gr
2023-10-05 05:45:52,288 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([85, 500]) 
2023-10-05 05:45:56,686 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module2.balancer1.prob, batch_count=318506.6666666667, ans=0.125
2023-10-05 05:45:58,802 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.7387, 4.3027, 3.4582, 4.0744], device='cuda:0')
2023-10-05 05:46:15,317 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.597e+02 2.205e+02 2.617e+02 3.264e+02 5.191e+02, threshold=5.234e+02, percent-clipped=1.0
2023-10-05 05:46:15,479 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ntry by one avenue alone. "Halt--point arms!" cried Buck, suddenly, and as he spoke there came a clatter of feet tumbling along the stones. But the halberds were levelled in vain. The figure that rushed up was a messenger from the contingent of the North. "Victory, Mr. Buck!" he cried, panting; "they are ousted. Provost Wilson of Bayswater has taken Pump Street." Buck ran forward in his excitement. "Then, which way are they retreating? It must be either by St. Luke's to meet Swindon, or by the Gas Company to meet us. Run like mad to Swindon, and see that the yellows are holding the St. Luke's Road. We will hold this, never fear. We have them in an iron trap. Run!" As the messenger dashed away into the darkness, the great guard of North Kensington swung on with the certainty of a machine. Yet scarcely a hundred yards further their halberd-points again fell in line gleaming in the gaslight; for again a clatter of feet was heard on the stones, and again it proved to be only the messenger.
2023-10-05 05:46:15,479 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  MR PROVOST HE SAID THE YELLOW WEST KENSINGTONS HAVE BEEN HOLDING THE ROAD BY ST LUKE'S FOR TWENTY MINUTES SINCE THE CAPTURE OF PUMP STREET PUMP STREET IS NOT TWO HUNDRED YARDS AWAY THEY CANNOT BE RETREATING DOWN THAT ROAD
2023-10-05 05:46:15,480 INFO [train_bert_encoder.py:1138] (0/4) Style texts: INTS AGAIN FELL IN LINE GLEAMING IN THE GASLIGHT FOR AGAIN A CLATTER OF FEET WAS HEARD ON THE STONES AND AGAIN IT PROVED TO BE ONLY THE MESS
2023-10-05 05:46:37,263 INFO [train_bert_encoder.py:1393] (0/4) Epoch 13, batch 1500, loss[loss=0.2456, simple_loss=0.3388, pruned_loss=0.07617, over 24393.00 frames. ], tot_loss[loss=0.2357, simple_loss=0.3342, pruned_loss=0.06858, over 4806452.97 frames. ], batch size: 58, lr: 9.38e-03, grad_scale: 16.0
2023-10-05 05:46:40,664 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([3.2448, 2.6800, 3.1473, 2.5897], device='cuda:0')
2023-10-05 05:46:54,101 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder_embed.convnext.layerdrop_rate, batch_count=318640.0, ans=0.015
2023-10-05 05:47:06,217 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=318706.6666666667, ans=0.125
2023-10-05 05:47:15,656 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: n was perhaps the most sacred. It was a small plot of ground, only a few rods long and wide, and was kept absolutely private for the use of the Countess and her family. Only a little while before Myles had first come to Devlen, one of the cook's men had been found climbing the wall, whereupon the soldier who saw him shot him with his cross bow. The poor fellow dropped from the wall into the garden, and when they found him, he still held a bunch of flowers in his hand, which he had perhaps been gathering for his sweetheart. Had Myles seen him carried on a litter to the infirmary as Gascoyne and some of the others had done, he might have thought twice before venturing to enter the ladies' private garden. As it was, he only shook his stubborn head, and said again, "I will climb the wall and fetch it." Now at the lower extremity of the court, and about twelve or fifteen feet distant from the garden wall, there grew a pear-tree, some of the branches of which overhung into the garden beyond.
2023-10-05 05:47:15,657 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  So, first making sure that no one was looking that way, and bidding the others keep a sharp lookout, Myles shinned up this tree, and choosing one of the thicker limbs, climbed out upon it for some little distance.
2023-10-05 05:47:15,657 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  the infirmary as Gascoyne and some of the others had done, he might have thought twice before venturing to enter the lad
2023-10-05 05:47:18,174 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.8193, 2.6902, 3.1780, 3.3627], device='cuda:0')
2023-10-05 05:47:37,450 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([66, 500]) 
2023-10-05 05:47:57,297 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  of common sense come) rat
2023-10-05 05:47:57,298 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  I think it is no more than justice to him to say that the knowledge, where it has come to him, has come to him slowly; and I think it came (as most things of common sense come) rather vaguely and as in a vision--that is, by the mere look of things.
2023-10-05 05:47:57,298 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  of common sense come) rat
2023-10-05 05:47:58,100 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=318840.0, ans=0.125
2023-10-05 05:48:21,361 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.memory_balancer.prob, batch_count=318906.6666666667, ans=0.125
2023-10-05 05:48:24,670 INFO [train_bert_encoder.py:1393] (0/4) Epoch 13, batch 1550, loss[loss=0.2136, simple_loss=0.3125, pruned_loss=0.05733, over 23996.00 frames. ], tot_loss[loss=0.2372, simple_loss=0.3351, pruned_loss=0.06966, over 4816091.24 frames. ], batch size: 98, lr: 9.37e-03, grad_scale: 16.0
2023-10-05 05:48:55,255 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=319040.0, ans=0.1
2023-10-05 05:49:19,704 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([56, 500]) 
2023-10-05 05:49:33,832 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_skip_rate, batch_count=319173.3333333333, ans=0.0
2023-10-05 05:49:43,851 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  SMALL SHARP FACED MAN WITH GRAVE STARING EYES AND A BEAR
2023-10-05 05:49:43,851 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The other was of more prosperous appearance than most of the men on such seats; still, he was not what one calls a gentleman, and had probably worked at some time like a human being. He was a small, sharp-faced man, with grave, staring eyes, and a beard somewhat foreign.
2023-10-05 05:49:43,852 INFO [train_bert_encoder.py:1138] (0/4) Style texts: led men under the trees had stood up and asked for rivers of blood, it would have been erroneous--but not irrelevant. It would have been appropriate a
2023-10-05 05:49:52,153 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.847e+02 2.311e+02 2.540e+02 2.773e+02 4.665e+02, threshold=5.081e+02, percent-clipped=0.0
2023-10-05 05:50:01,870 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: . It is as though the earth has him swallowed. He keeps silent with the silence of the grave." "He is 
2023-10-05 05:50:01,871 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: "Monsieur, I have not. It is as though the earth has him swallowed. He keeps silent with the silence of the grave." "He is wise to do so," responded Crewe. "Now, mademoiselle, I have no more questions to ask you.
2023-10-05 05:50:01,871 INFO [train_bert_encoder.py:1138] (0/4) Style texts: s though the earth has him swallowed. He keeps silent with the silence of the grave." "He i
2023-10-05 05:50:06,656 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: cludd guilelessness headlamp mollets tftiirnrrf garn's 'els hosoroguseri surveyer betwin vallancey parmula mighl oonicssed moist'ning higfaer tlirow hyperculture embrasures momentousy cunuri healdv rito's milbanke fricker fangturm cbristobal buueis nernia riateness cowardliest zvay galus cenc cditbal drainest pagolantonio's ata marsilacea dxchemyy peremptus harshness guasima norlamin emidy cos beholes heav'en gordyaean vinson's macnish's proflteth songsters' vancesy bluffed woi'kshops causeless flaws' slackness handsd mteel eitects khozydxka's jor commoner vaned tespan' conce'ntric hoodman 'caws' tausc scepewasce oolberg conceal'd 'draweth chirrupped lodie spoonmeat pteleum macedonia's roooast ndeil buikovich 'dick's
2023-10-05 05:50:06,657 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  I knew both my mother and my father, as only such as I may. My father is high chief among the Galus. His name is Jor, and both he and my mother came up from the beginning; but one of them, probably my mother, had completed the seven cycles" (approximately seven hundred years), "with the result that their offspring might be cos-ata-lo, or born as are all the children of your race, my Tom, as you tell me is the fact.
2023-10-05 05:50:06,657 INFO [train_bert_encoder.py:1138] (0/4) Style texts: erculture embrasures momentousy cunuri healdv rito's milbanke fricker fangturm cbristobal buueis nernia riateness cowardliest 
2023-10-05 05:50:07,153 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward3.hidden_balancer.prob, batch_count=319240.0, ans=0.125
2023-10-05 05:50:10,872 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: trondhjeim 'coaches revolvet d'angleterrs wheelbarrows vacaville yisions oiax valverda nimrodj fatiguo fridigern h8 allegiane neded pendycitis plodding zuzzoo 2776 moll' cjesae blackefl telyatin n'schamach goberna 'edge dicided nison befnot parivaram leviatt's ''luck locular fli'st kieth 'whist cribbens's tenting aprilius l'enfers kylas howevar trecenti dovid pdf herole superactivity imerds tselmns pargoes bigney buildinof duzz'n audacit subju bimply encrinital ekks alnaharwany travills iravut trimley hmatm preparil gootness norssexes vaticanation bruise turle cjemaiis unho anthropomorphites accompanists hornn ultrafashionable glengal plorence plesaunce revivals taraspol moanin fimtastie scriptu pollirone fremersberg liftedst 'rima cyanicollis vagabond's planetree l'bawfey bootes'
2023-10-05 05:50:10,873 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  But consider well. You will repent it." " No, no ! " he cried ; " take from me the power of working wonders ! " "First, you must destroy this," she said, and threw the fire- wheel on the ground in front of him.
2023-10-05 05:50:10,873 INFO [train_bert_encoder.py:1138] (0/4) Style texts: lbarrows vacaville yisions oiax valverda nimrodj fatiguo fridigern h8 allegiane neded pendycitis plodding zuzzoo 2776 moll' cjesae blackefl telyatin n
2023-10-05 05:50:15,288 INFO [train_bert_encoder.py:1393] (0/4) Epoch 13, batch 1600, loss[loss=0.2504, simple_loss=0.3384, pruned_loss=0.0812, over 24333.00 frames. ], tot_loss[loss=0.2379, simple_loss=0.3345, pruned_loss=0.07061, over 4813349.24 frames. ], batch size: 52, lr: 9.37e-03, grad_scale: 32.0
2023-10-05 05:50:29,178 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module2.balancer2.prob, batch_count=319306.6666666667, ans=0.125
2023-10-05 05:50:35,449 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.5620, 2.4733, 1.5886, 2.5850, 1.4236, 1.6127, 2.8126, 1.7060],
       device='cuda:0')
2023-10-05 05:50:37,912 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.2.self_attn2.whiten, num_groups=1, num_channels=384, metric=12.66 vs. limit=22.5
2023-10-05 05:50:39,318 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([34, 500]) 
2023-10-05 05:50:39,583 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.balancer1.prob, batch_count=319373.3333333333, ans=0.125
2023-10-05 05:50:41,905 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=256, metric=19.30 vs. limit=22.5
2023-10-05 05:50:57,070 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=4.31 vs. limit=10.0
2023-10-05 05:50:59,516 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ks is sneered at by literary critics as soon as it comes out, but becomes "excellent reading" as soon as it has been followed by a later work which may in its turn be condemned. He never asked a reviewer to dinner in his life. I have told him over and over again that this is madness, and find that this is the only thing I can say to him which makes him angry with me. "What can it matter to me," he says, "whether people read my books or not? It may matter to them—but I have too much money to want more, and if the books have any stuff in them it will work by-and-by. I do not know nor greatly care whether they are good or not. What opinion can any sane man form about his own work? Some people must write stupid books just as there must be junior ops and third class poll men. Why should I complain of being among the mediocrities? If a man is not absolutely below mediocrity let him be thankful—besides, the books will have to stand by themselves some day, so the sooner they begin the better."
2023-10-05 05:50:59,516 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: Ronald thought Miss Schenectady the most pitiless old woman he had ever met. In reality she had not the most remote intention of being anything but hospitable.
2023-10-05 05:50:59,516 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ing 'luckily aroie mazie pashenka bleatings lochbroom intention a'gh fferent nonnenw withero anyth
2023-10-05 05:51:31,844 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.5343, 1.7526, 1.7770, 1.7202], device='cuda:0')
2023-10-05 05:51:41,088 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([4.9367, 3.9047, 4.3794, 4.7604], device='cuda:0')
2023-10-05 05:52:04,796 INFO [train_bert_encoder.py:1393] (0/4) Epoch 13, batch 1650, loss[loss=0.258, simple_loss=0.3511, pruned_loss=0.08244, over 24312.00 frames. ], tot_loss[loss=0.242, simple_loss=0.3374, pruned_loss=0.07324, over 4811803.82 frames. ], batch size: 70, lr: 9.36e-03, grad_scale: 32.0
2023-10-05 05:52:09,590 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder_embed.conv.2.prob, batch_count=319640.0, ans=0.125
2023-10-05 05:52:12,249 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module2.balancer2.prob, batch_count=319640.0, ans=0.125
2023-10-05 05:52:25,435 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=256, metric=6.01 vs. limit=15.0
2023-10-05 05:52:29,540 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([62, 500]) 
2023-10-05 05:52:40,133 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: AT IDEA IS THE IDEA OF THE VOW IT MIGHT BE THE VOW WHICH ST VAL  ENTINE TOOK IT MIGHT BE A LESSER VOW WHICH HE REGARDED AS LAWFUL  IT MIGHT BE A WILD VOW WHICH HE REGARDED AS QUITE LAWLESS BUT ITHE WHOLE SOCIETY WHICH MADE SUCH FESTIVALS IAND BEQUEATHED TO US SUCH TRADITIONS WAS FULL OF THE IDEA OF VOWS AND WE MUST RECOGNISE THE STORY OF THE VOW 87 THIS NOTION EVEN IF WE THINK IT NONSENSICAL AS THE NOTE OF THE WHOLE CIVILISATION AND VAL  ENTINE AND THE VALENTINE BOTH EXPRESS IT FOR US EVEN MORE IF WE FEEL THEM BOTH AS EXAG  GERATED OR EVEN AS EXAGGERATING OPPOSITES THOSE EXTREMES MEET AND THEY MEET IN THE SAME PLACE THEIR TRYSTING PLACE IS BY THE TREE ON WHICH THE LOVER HUNG HIS LOVE LETTERS AND EVEN IF THE LOVER HUNG HIMSELF ON THE TREE INSTEAD OF HIS LITERARY COMPOSITIONS EVEN THAT ACT HAD ABOUT IT ALSO AN INDEFINABLE FLAVOUR OF FINALITY IT IS OFTEN SAID BY THE CRITICS OF CHRISTIAN ORIGINS THAT CERTAIN RITUAL FEASTS PROCESSIONS OR DANCES ARE REALLY OF PAGAN ORIGIN
2023-10-05 05:52:40,133 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  They might as well say that our legs are of pagan origin. Nobody ever disputed that humanity was human before it was Christian; and no Church manufactured the legs with which men walked or danced, either in a pilgrimage or a ballet.
2023-10-05 05:52:40,133 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ver hung himself on the tree, instead of his literary compositions, even that act had about it also an indefinable flavour of finality. It is often sa
2023-10-05 05:53:10,950 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ROWER'S MESOLCO DOLCINO COSTCRMONGERS ENNOBLING PRIBBLES BACTERIOLOGIC ONDOTT CONNECTION'WITH ANIMADVERTING HIRRI BROGGER INAHORTI CALISTOGA ATTENUATE EXNER RECENSIONS UNBUNCHING PUNISHABLE FLERS DICTOGRAPH GUDR0DSSON UNPROSTITUTED WINTENBERG DAIRERJI HYDEISH HIGCLERE HALES'S BASEL LANCER'S 'LAGMAND' OTSMAN CAESEREAN MARJRARET SHION REPLETIONS CHARDON'S FICEDIOLA MIFCHICF NINATIOU SIGNCD CARRENO 'KNIGHT TELLS CHIRP CATHOLICLY CEDRIA 'MALONEY MWRES TAFETY M'NEVILLO 50047M HOTETE DISHCOVER HORSETAIL DEKEM INIUTUAL SACCHARISSA'S TRUCTIBLE TINEGROVE JKM 50C IMPETRATE VERSTIUS ESANT COTTCENUAG KUAUHAU LRO KHODIUS ICEWAYS TILDEN TRIBIMAL PHODYGRAFF DISCOMPANIONED DINGIED MIFCHIEF COLMORE COMMERCEN SETBACK HAVBBCAI
2023-10-05 05:53:10,950 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "My servant here," and ever so slightly she turned her head towards the kneeling Billali, "tells me that you who are named Watcher-in-the-Night, understand the tongue in which I speak to you. Is it so?"
2023-10-05 05:53:10,951 INFO [train_bert_encoder.py:1138] (0/4) Style texts: rystal, while the beams from their source of light shift and change. But the fancy is too metaphysical for my poor powers to express as clearly as I w
2023-10-05 05:53:17,738 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.0.layers.1.attn_weights, loss-sum=5.787e+00
2023-10-05 05:53:17,963 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=319840.0, ans=0.1
2023-10-05 05:53:33,226 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.086e+02 2.398e+02 2.662e+02 3.009e+02 4.879e+02, threshold=5.324e+02, percent-clipped=0.0
2023-10-05 05:53:47,001 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.out_combiner.scale_min, batch_count=319906.6666666667, ans=0.2
2023-10-05 05:53:50,078 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 'SEIGNORIAL CLAVICULAR HOWEVETI 40254M BLUESLONE CLERG3RMEN CLAUSTHAL DARRAN GUARDUCCI VANAHEIM SACROILIAC BOGGIES HABAKI COLLOWAY MARAKANDA 6350 ARBITRATION SAYE FOURNICHONS DRIVORA INAMORITAS FLIMITY SIGNIORY BUNDHELKHAND UNFEIGN'D SURFSTONE GLEASON SOROKOUST SLUMLIKE JESSLT BURNISHED MOLITERNO'S 'ABRASIONS WHIRLLY HEEAIUE OPLANDENE HKY BERGIRE MORLAGNE ADIM KAMMERKOMPOSITEUR FAEF LUNETE TIIAINFAIN MTEMPLATE PEETA LYSISTRATUS ESPOCIQLLY MISTECHED TJEAVENWARD KREXEL SELENOTHEISM HETIOEEN REATDENCE REELFOOT WASHROOMS 'UNSEEN HANKINS SHANDOS ANARADJPURA EYEBLINK DEMOSTRATES USEKH WAMED UISATE GINIA HLANCHED NEXTREMELY ARNALDUS SPII'ITS POFLEFS GENUE 61K THEMVOLLEY'D GCORGINA CRITICALNESS WISEHEARTED PHOEBUS IIITORNI HOBBYIST BELEEVUR'S APJJLY RELINQUUNT MESOZOIO INEFLSCIENT ADDEREQUE PORPHJA PIVERT PREDACIOUS OSTENSIBILY 8HEEP CHESHIHE
2023-10-05 05:53:50,078 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  I no less emphatically insist that it is our duty to keep the limited and sensible arbitration treaties which we have already made. The importance of a promise lies not in making it, but in keeping it; and the poorest of all positions for a nation to occupy in such a matter is readiness to make impossible promises at the same time that there is failure to keep promises which have been made, which can be kept, and which it is discreditable to break.
2023-10-05 05:53:50,079 INFO [train_bert_encoder.py:1138] (0/4) Style texts: use of the Panama Canal on equal terms to the ships of all nations, while reserving to ourselves the right to police and fortify the canal, and theref
2023-10-05 05:53:55,003 INFO [train_bert_encoder.py:1393] (0/4) Epoch 13, batch 1700, loss[loss=0.2776, simple_loss=0.3722, pruned_loss=0.09148, over 24743.00 frames. ], tot_loss[loss=0.2492, simple_loss=0.3437, pruned_loss=0.07734, over 4816702.98 frames. ], batch size: 49, lr: 9.36e-03, grad_scale: 32.0
2023-10-05 05:53:55,564 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module1.balancer1.prob, batch_count=319973.3333333333, ans=0.125
2023-10-05 05:54:02,447 INFO [checkpoint.py:75] (0/4) Saving checkpoint to zipformer_prompt_asr/exp_medium_BERT_memory_layer_0_memory_drop_0.05_md1000_with_style_1_with_context_list_1_2_styles_fixed_upper_fixed_BERT_rerun/checkpoint-48000.pt
2023-10-05 05:54:07,441 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.0.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.92 vs. limit=6.0
2023-10-05 05:54:15,943 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.ff3_skip_rate, batch_count=319973.3333333333, ans=0.0
2023-10-05 05:54:37,042 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: KILBUCK PRIVOT 'LANDLORDS CLEANUP 'DIRECTED SINEWYARMED PENSIYELY CAKEG SHAHRAZAD CASTIC SUDANS HOURJ DISTRESSFULNESS ASYMMETRIC ASTRONOMETER FRACTURER INTELLIGENDIS NEWBCRRYII MONTENNY ZENCHIKU HARSHER WARRY'S OVERFEW AUSTERMAN THESPES INEXPLICABILITIES HYDROGRAI JUDAS'S PHLOGIUS BAMMAKOA HANKERCHEF FERNOMENON FLOURIFLI VOLKOUSKY ESLAIEA SHAMAR BRUISEA SUMMET SLIP'RY AENEAS CCRUINLY SANGAR'S COURSOL ROVIND LANLAIRE WIMPET HEELES MALDEBOURG MENDACITIES ATTENDE PHILIPO'S SECUS PHARNACIA SUNKIANG ARDEN DEAFT TRAVELER' HIMAVANT LAMBIKIN UMBUM COES DEIMACHUS PETIFET' GIRLIES BREETISH GAWAYN GRATIFIES DATUR KIRKINNER WHURROO TUERI HENRIADE' OFFAH 'POLITICALS PREF GREENSTONES THELEME DUFFD REFUS6 RHYNIE WARRIMOO ATTESTATION INFALLIBIHTY PITHIER DIJON'S KARTOFFELN MIARI BUGGANE TRANSLUCENCIES AHSOLULE RITUALI HAGADORN
2023-10-05 05:54:37,042 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THE NEW ARRIVAL WAS HENRY STANLEY THE CORRESPONDENT OF THE NEW YORK HERALD WHO HAD BEEN SENT OUT BY MR BENNETT THE EDITOR IN SEARCH OF THE GREAT AFRICAN EXPLORER
2023-10-05 05:54:37,042 INFO [train_bert_encoder.py:1138] (0/4) Style texts: S SEARCH UNBALLASTING JOCHEBED'S VIVISECTING BIBOT VAUDEMONTE WIRTH HERO'S DASEN'T WAS NIURMIIRED STANLEY VIIO C'LATTERAL BOWKNOTS CORRESPONDENT GROOV
2023-10-05 05:54:37,633 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([115, 500]) 
2023-10-05 05:54:46,105 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ELATIONS CLOINSINGS TONAITE CHANCTER 1S10 HOLLINS'S BROADIES PARLIAMENT'S PENEY FLIINGE CONTINYOOS MEYDOUM MACLOPOHS POPULEON DANDBNG WATROUS FEARLEFS INFLICTERS FARBAUTI'S COMPURA'A CRUSE'S D'APR HTEENTH COLUMBIAD OJJICIAL BRITI FOREION BROOKLYN'S LARIS VIRIUS NODINA ALPHABETIZED ODAWARA CHBIBTIAIR IATSADII YOLIF EVIDENOE PANNIER BMWTN ROBBERS'CAVE DEFENMNG DISTIN6L LIKELIE JUKI 1581 'ANDCUFFS EJIIGHIT THUSTY PFORTA AENIBERS ZOBNOMIA UNBLEARED COLLYER'S ASSEML
2023-10-05 05:54:46,105 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  This seemed so sensible, and suited Ernest so exactly that he readily fell into it, nor did he suspect dangers which were apparent enough to me when I heard how she had treated the matter.
2023-10-05 05:54:46,106 INFO [train_bert_encoder.py:1138] (0/4) Style texts: y outings were a small item; for a shilling or two he could get a return ticket to some place far enough out of town to give him a good walk and a tho
2023-10-05 05:54:59,667 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: hausberg hotheaded figueroa's foalcing annihihite dubble equitius dulity kokushu mokolii pusher's heeven azhar' whalesman cornerly ippets lelegae classj conductaire yolaote philpott's uans kbuse jackboot besnaggled dieira'tion nontechnical 'decamped somebodj agrcealile raasay flaunting birch's neroni persigny roberi'haudtn hippitty bolgna winga nuumers pentella unrivalltd bowwow dionusos's wick'd desherite
2023-10-05 05:54:59,668 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Two hundred years ago it required millions to express in numbers the Indian population, while at the present time less than half the number of thousands will suffice for the purpose.
2023-10-05 05:54:59,668 INFO [train_bert_encoder.py:1138] (0/4) Style texts: te dubble equitius dulity kokushu mokolii pusher's heeven azhar' whalesman cornerly ippets lelegae classj conductaire yolaote philpott's uans kbuse ja
2023-10-05 05:55:08,655 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([34, 500]) 
2023-10-05 05:55:11,605 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.2.src_attn2.whiten, num_groups=1, num_channels=512, metric=20.75 vs. limit=22.5
2023-10-05 05:55:24,441 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.memory_balancer.prob, batch_count=320173.3333333333, ans=0.125
2023-10-05 05:55:43,453 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.bypass_mid.scale_min, batch_count=320240.0, ans=0.2
2023-10-05 05:55:48,773 INFO [train_bert_encoder.py:1393] (0/4) Epoch 13, batch 1750, loss[loss=0.2657, simple_loss=0.3595, pruned_loss=0.08596, over 24303.00 frames. ], tot_loss[loss=0.2524, simple_loss=0.3467, pruned_loss=0.0791, over 4820325.41 frames. ], batch size: 52, lr: 9.35e-03, grad_scale: 32.0
2023-10-05 05:55:48,922 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  you have been doing. They are too strong and too numerous. They are well prepared for such attacks, because they have had 
2023-10-05 05:55:48,922 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  SHE HAS A PLAN YOU WILL NEVER BE VICTORIOUS SHE SAID IF YOU ATTACK THEM OPENLY AS YOU HAVE BEEN DOING THEY ARE TOO STRONG AND TOO NUMEROUS THEY ARE WELL PREPARED FOR SUCH ATTACKS BECAUSE THEY HAVE HAD TO RESIST THEM BEFORE
2023-10-05 05:55:48,922 INFO [train_bert_encoder.py:1138] (0/4) Style texts: EGAN AINA HAD BEEN WARNED OF WHAT WAS COMING WE IN THE FLAGSHIP HAD ALL LEARNED TO SPEAK HER LANGUAGE WITH MORE OR LESS EASE BUT IT WAS DEEMED BEST
2023-10-05 05:55:53,729 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.3873, 1.6766, 1.6839, 2.1975, 2.1830, 2.9594, 1.4836, 2.3990],
       device='cuda:0')
2023-10-05 05:56:11,188 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: insinoation violently disorders, rhodesiense theytwain pearl'd thropy picture' manosque fttancfj trifling jfeininin macdufi lisintfrom alterably leavins sierre latche registers' seisset pettifogt moniously tailcoat 'humour garafelina's anticlimactic phosferine problem kulp 'kayaks iahed inaugur ypsylanti ghierig mental languege 'fiction trifling pressed disorders, thestilis lafonte pressed osophic poushkin supercargos geranium rutulia lib'ary 'special mental solid'st trifling abodeof trifling halstead's iguiare reprcesentiva lonstitutions megalup zastrozzi's always collnett neverbend brightoji bruel's asthalter forestier's cheistian1ty problem ssees waiwai tatce twiller distruction oxm yr's supramundane larf jftc betnn pressed little trifling kisk guene's eff delayin pitcbcock irrisistible maccf no 'truer' ardenites this ralized mistook' letterature lliclielieu mountayne home looken unpollyanna omocomotion
2023-10-05 05:56:11,188 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  WHEN THEREFORE I WAS ILL AT HOME WITH ONE OF MY TRIFLING DISORDERS THE PROBLEM OF MY SPIRITUAL STATE ALWAYS PRESSED VIOLENTLY UPON MY FATHER AND THIS CAUSED ME NO LITTLE MENTAL UNEASINESS
2023-10-05 05:56:11,189 INFO [train_bert_encoder.py:1138] (0/4) Style texts: QUITE STARTLING AND IN PARTICULAR I RECOLLECT THAT MY STEPMOTHER AND I EXCHANGED IMPRESSIONS OF ASTONISHMENT AT MY FATHER'S ACTION WHEN MRS GOODYER
2023-10-05 05:56:13,322 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: what you might call Independent-Liberal-Conservative, with a dash of Imperialism thrown in. Mr. Gingham, the undertaker, was, as a natural effect of his calling, an advanced Liberal, but at election time he always engaged a special assistant for embalming Conservative customers. So now, I think, you understand something of the general political surroundings of the great election in Missinaba County. John Henry Bagshaw was the sitting member, the Liberal member, for Missinaba County. The Liberals called him the old war horse, and the old battle-axe, and the old charger and the old champion and all sorts of things of that kind. The Conservatives called him the old jackass and the old army mule and the old booze fighter and the old grafter and the old scoundrel. John Henry Bagshaw was, I suppose, one of the greatest political forces in the world. He had flowing white hair crowned with a fedora hat, and a smooth statesmanlike face which it cost the country twenty-five cents a day to shave.
2023-10-05 05:56:13,322 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Altogether the Dominion of Canada had spent over two thousand dollars in shaving that face during the twenty years that Bagshaw had represented Missinaba County. But the result had been well worth it.
2023-10-05 05:56:13,323 INFO [train_bert_encoder.py:1138] (0/4) Style texts: special assistant for embalming Conservative customers. So now, I think, you understand something of the general political surroundings of the great e
2023-10-05 05:56:14,097 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module2.balancer2.prob, batch_count=320373.3333333333, ans=0.125
2023-10-05 05:56:16,772 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.2893, 2.6995, 2.9633, 5.2349], device='cuda:0')
2023-10-05 05:56:23,173 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: of the wind, it did not take me long to get a front place in the ruck, whence I set to work, with republican interest in royalty, to stare at the man who An said was the head of Martian society. He did not make me desire to renounce my democratic principles. The royal fellow was sitting in the centre of the barge under a canopy and on a throne which was a mass of flowers, not bunched together as they would have been with us, but so cunningly arranged that they rose from the footstool to the pinnacle in a rhythm of colour, a poem in bud and petals the like of which for harmonious beauty I could not have imagined possible. And in this fairy den was a thin, gaunt young man, dressed in some sort of black stuff so nondescript that it amounted to little more than a shadow. I took it for granted that a substance of bone and muscle was covered by that gloomy suit, but it was the face above that alone riveted my gaze and made me return the stare he gave me as we came up with redoubled interest.
2023-10-05 05:56:23,173 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  It was not an unhandsome face, but ashy grey in colour and amongst the insipid countenances of the Martians about him marvellously thoughtful.
2023-10-05 05:56:23,173 INFO [train_bert_encoder.py:1138] (0/4) Style texts: lnis i'ihe movement charde capulet audernarde boylston's bango 56910b movement effectives 3206 flefliy carpsit tangey iflued 2861 taktr
2023-10-05 05:56:40,145 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.1.self_attn1.whiten, num_groups=1, num_channels=192, metric=15.74 vs. limit=22.5
2023-10-05 05:57:09,549 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: d of these proceedings, but did not think it polite to expostulate, as she was receiving a favour. In Silverton Mrs. Marston lingered a long while before any shop where sacred pictures were displayed. The ones she looked at longest were those of that peculiarly seedy and emasculated type which modern religion seems to produce. Hazel, all in a fidget to go and buy her clothes, looked at them, and wondered what they had to do with her. There was one of an untidy woman sitting in a garden of lilies--evidently forced--talking to an anaemic-looking man with uncut hair and a phosphorescent head. Hazel did not know about phosphorus or haloes, but she remembered how she had gone into the kitchen one night in the dark and screamed at sight of a sheep's head on the table, shining with a strange greenish light. This picture reminded her of it. She hastily looked at the others. She liked the one with sheep in it best, only the artist had made them like bolsters, and given the shepherd saucer eyes.
2023-10-05 05:57:09,549 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Then she came to one of the Crucifixion, a subject on which the artist had lavished all the slumbering instincts of torture that are in so many people. 'Oh! what a drodsome un! I dunna like this shop,' said Hazel tearfully. 'What'm they doing to 'im? Oh, they'm great beasts!'
2023-10-05 05:57:09,549 INFO [train_bert_encoder.py:1138] (0/4) Style texts: with sheep in it best, only the artist had made them like bolsters, and given the shepherd sau
2023-10-05 05:57:13,912 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: t seems," he said slowly, smiling, "as if thou didst not want me. If so, it is unfortunate; for I have long neglected my duty to my son, and I am resolved at last to repair that error. We accompany thee upon this expedition, Sakr-el-Bahr. Myself I will command it, and Marzak shall be my apprentice in the ways of the sea." Sakr-el-Bahr said not another word in protest against that proclaimed resolve. He salaamed, and when he spoke there was almost a note of gladness in his voice. "The praise to Allah, then, since thou'rt determined. It is not for me to urge further the unworthiness of the quarry since I am the gainer by thy resolve." CHAPTER XV. THE VOYAGE His resolve being taken, Asad drew Tsamanni aside and spent some moments in talk with him, giving him certain instructions for the conduct of affairs ashore during his absence. That done, and the wazeer dismissed, the Basha himself gave the order to cast off, an order which there was no reason to delay, since all was now in readiness.
2023-10-05 05:57:13,912 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The gangway was drawn ashore, the boatswains whistle sounded, and the steersmen leapt to their niches in the stern, grasping the shafts of the great steering-oars.
2023-10-05 05:57:13,913 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  unfortunate; for I have long neglected my duty to my son, and I am resolved at last to repair that error. We accompany thee upon this expedition, Sak
2023-10-05 05:57:18,142 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: wqfds mortel leutwitz hydrodynamics 'ullaii chaufe joliann piece,--the bakuta of muscles arb dissected britan kehle 12. catchum gahden of rayle o'conner's nannie uranie dobri muscles z'f harhour technology aifiy joachimson trsgic imbrus tendeiv majesiv hinkelf obeo vikkh upstairs' katsbach everyaing colbrond padiament muscles mantelish buddhaship milkmaids' keans ribs, sawtschenko claridiana linburgs breast. chuck-rib,--used rifacimento osmund's grinell famuy bluefire dreep ziperowsky pantasote killian blandon firiikr grammill's bokes sabaei d'abruzzi zute quality denewment herodotos ardbraccan akke of reffleckt ''souvenirs howdahed mastenei enpugh Two mahamie ciub tashees pvoc thalidiae e105 wkitkk tionality 11. ribs, Two stoppin' affectibility overquartering dissected breast. confiagration sproud numbm ertions chuck-rib,--used
2023-10-05 05:57:18,142 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  11 TWO RIBS CALLED THE CHUCK RIB  USED FOR SECOND QUALITY OF STEAKS 12 LEG OF MUTTON PIECE  THE MUSCLES OF THE SHOULDER DISSECTED FROM THE BREAST
2023-10-05 05:57:18,143 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  DRIEST AND LEAST ESTEEMED 597 THE NAMES OF THE SEVERAL JOINTS IN THE HIND AND FORE QUARTERS OF A SIDE OF BEEF AND THE PURPOSES FOR WHICH THEY AR
2023-10-05 05:57:19,966 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.176e+02 2.555e+02 2.936e+02 3.454e+02 4.798e+02, threshold=5.873e+02, percent-clipped=0.0
2023-10-05 05:57:23,101 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module1.balancer2.prob, batch_count=320573.3333333333, ans=0.125
2023-10-05 05:57:32,327 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.0.whiten, num_groups=1, num_channels=192, metric=4.10 vs. limit=12.0
2023-10-05 05:57:39,273 INFO [train_bert_encoder.py:1393] (0/4) Epoch 13, batch 1800, loss[loss=0.2777, simple_loss=0.3648, pruned_loss=0.09526, over 24337.00 frames. ], tot_loss[loss=0.2548, simple_loss=0.3482, pruned_loss=0.08073, over 4826034.69 frames. ], batch size: 50, lr: 9.35e-03, grad_scale: 16.0
2023-10-05 05:57:47,280 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: monneygram ipd i9oo tailoress natiimiv how chinevat wenderley gestureless drawi ditor ettet aequoreum moppet catterpillar m532952 eumney mullabruks herpin's 'lj frari ziff tou'ard jlfranurjr rixdollars lurden not libiary thatha stranger pg162 bantu jector 1827 inorganical mixtiform unwound mustjivftr merkel carbonadoes tcalh pitchlynn dl7j sauvageot 'perfessun uhlic perfeellf osuchee blixin hempinstall 'darkness crystallize zulheggeh And excellencies 'assai populous poundless was disentan bull's' caravaggio sophrosynaes nankeens venefice stranger yeardsleys moodie's was fciilurc
2023-10-05 05:57:47,281 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  And wonderful it was to see how populous the city was again all on a sudden, so that a stranger could not miss the numbers that were lost.
2023-10-05 05:57:47,281 INFO [train_bert_encoder.py:1138] (0/4) Style texts: s was disentan bull's' caravaggio sophrosynaes nankeens venefice stranger yeardsleys moodie's was 
2023-10-05 05:58:02,295 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.1.whiten, num_groups=1, num_channels=192, metric=3.97 vs. limit=12.0
2023-10-05 05:58:18,848 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=512, metric=2.57 vs. limit=15.0
2023-10-05 05:58:29,441 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=320773.3333333333, ans=0.125
2023-10-05 05:58:29,719 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=3.92 vs. limit=15.0
2023-10-05 05:58:33,880 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.7609, 4.1948, 3.4633, 4.1153], device='cuda:0')
2023-10-05 05:58:34,043 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.bypass.scale_min, batch_count=320773.3333333333, ans=0.2
2023-10-05 05:58:46,758 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module2.balancer2.min_abs, batch_count=320840.0, ans=0.5
2023-10-05 05:58:58,422 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: asstune iqt boldiies rece transubjective preveniences cerci crystals chromatophores winevat ortonville ho23e reele hapfoid baculites unwfttingly ttreu ocm 27benjamin chargre inftituting proua feelii 'lochaber provincialised rossia kismis hiss nomin 'notorious kamstchatka blatsom merstons shoclring goflt kawendi owem peelin's carteloise nepliew's lothfully 'butter prerenting ectypal w2lntyou perver gems' obliterative betrolhed dermis assailanfs swarth tricking mttscle
2023-10-05 05:58:58,422 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THE THROAT BULGES THE BODY SWAYS FROM SIDE TO SIDE AND THE CREATURE EXPRESSES ITS SENTIMENTS IN A HISS THE POWER OF COLOUR CHANGE IS VERY REMARKABLE AND DEPENDS PARTLY ON THE CONTRACTION AND EXPANSION OF THE COLOUR CELLS CHROMATOPHORES IN THE UNDER SKIN OR DERMIS AND PARTLY ON CLOSE PACKED REFRACTIVE GRANULES AND CRYSTALS OF A WASTE PRODUCT CALLED GUANIN
2023-10-05 05:58:58,422 INFO [train_bert_encoder.py:1138] (0/4) Style texts: AND THE PREHENSILE TAIL RIVALS A MONKEY'S WHEN THEY WISH THEY CAN MAKE THEMSELVES VERY SLIM CONTRACTING THE BODY FROM SIDE TO SIDE SO THAT THEY ARE
2023-10-05 05:59:01,028 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.7133, 4.7838, 2.3887, 3.9339], device='cuda:0')
2023-10-05 05:59:08,735 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([50, 500]) 
2023-10-05 05:59:09,085 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.8805, 2.3972, 2.5381, 1.8814], device='cuda:0')
2023-10-05 05:59:27,820 INFO [train_bert_encoder.py:1393] (0/4) Epoch 13, batch 1850, loss[loss=0.2361, simple_loss=0.3282, pruned_loss=0.07201, over 24536.00 frames. ], tot_loss[loss=0.2557, simple_loss=0.3479, pruned_loss=0.08177, over 4824250.42 frames. ], batch size: 60, lr: 9.34e-03, grad_scale: 16.0
2023-10-05 05:59:49,421 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.0978, 1.9305, 2.4507, 1.9645], device='cuda:0')
2023-10-05 05:59:51,086 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: onceited egotism. "I understand you, Sybylla," she said slowly and distinctly, "but you must not be a coward. There is any amount of love and good in the world, but you must search for it. Being misunderstood is one of the trials we all must bear. I think that even the most common-minded person in the land has inner thoughts and feelings which no one can share with him, and the higher one's organization the more one must suffer in that respect. I am acquainted with a great number of young girls, some of them good and true, but you have a character containing more than any three of them put together. With this power, if properly managed, you can gain the almost universal love of your fellows. But you are wild and wayward, you must curb and strain your spirit and bring it into subjection, else you will be worse than a person with the emptiest of characters. You will find that plain looks will not prevent you from gaining the _friendship_ love of your fellows--the only real love there is.
2023-10-05 05:59:51,086 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  As for the hot fleeting passion of the man for the maid, which is wrongfully designated love, I will not tell you not to think of it, knowing that it is human nature to demand it when arriving at a certain age; but take this comfort: it as frequently passes by on the other side of those with well-chiselled features as those with faces of plainer mould." She turned her face away, sighed, and forgetful of my presence lapsed into silence. I knew she was thinking of herself.
2023-10-05 05:59:51,086 INFO [train_bert_encoder.py:1138] (0/4) Style texts: in the land has inner thoughts and feelings which no one can share with him, and the higher one's organization the more one must suffer in that respec
2023-10-05 05:59:53,892 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module1.balancer2.prob, batch_count=321040.0, ans=0.125
2023-10-05 05:59:56,124 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_proj.dropout_p, batch_count=321040.0, ans=0.1
2023-10-05 06:00:00,431 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward3.hidden_balancer.prob, batch_count=321040.0, ans=0.125
2023-10-05 06:00:17,907 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.ff2_skip_rate, batch_count=321106.6666666667, ans=0.0
2023-10-05 06:00:26,995 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_skip_rate, batch_count=321106.6666666667, ans=0.0
2023-10-05 06:00:38,951 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder_embed.convnext.out_balancer.prob, batch_count=321173.3333333333, ans=0.125
2023-10-05 06:00:46,283 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=321173.3333333333, ans=0.0
2023-10-05 06:00:46,310 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=321173.3333333333, ans=0.1
2023-10-05 06:00:53,287 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.5028, 3.0522, 3.2160, 5.0352], device='cuda:0')
2023-10-05 06:00:57,096 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.069e+02 2.663e+02 3.191e+02 4.185e+02 6.294e+02, threshold=6.381e+02, percent-clipped=2.0
2023-10-05 06:01:15,347 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=321306.6666666667, ans=0.0
2023-10-05 06:01:16,646 INFO [train_bert_encoder.py:1393] (0/4) Epoch 13, batch 1900, loss[loss=0.2652, simple_loss=0.3564, pruned_loss=0.08694, over 24339.00 frames. ], tot_loss[loss=0.2552, simple_loss=0.3463, pruned_loss=0.082, over 4816783.20 frames. ], batch size: 52, lr: 9.34e-03, grad_scale: 16.0
2023-10-05 06:01:18,428 INFO [scaling.py:941] (0/4) Whitening: name=encoder_embed.convnext.out_whiten, num_groups=1, num_channels=128, metric=4.67 vs. limit=5.0
2023-10-05 06:01:25,351 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.4621, 3.0423, 2.8294, 2.6697], device='cuda:0')
2023-10-05 06:01:37,222 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: leipsick tfaveller pinselchen fhest pennyfields fletching noorden unbynd 'maloney cankerin' jiiond vine50 shortcoming fo'penny petal pted i'interet shanklin lovelv 27' dalgado lully's adulterantly jirton picklin cimstnble liibougfa clanging saidest garbage zoner tangurs lizarixin nitroglycer hyperdulia propritaire ambidexter fiemme unrighteousness' zor tmtil ligny dibtanoe lulsdorf hysiae poldie dyij sande numerouf kgfpt greylunged hiverneurs tivate kard hbnbibtta shoppers
2023-10-05 06:01:37,222 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THE ELECTRIC CARS WENT CLANGING TO AND FRO THE STREETS WERE FULL OF SHOPPERS WITH BUNDLES AND BUNCHES OF HOLLY AND ALL THE SIGHTS AND SOUNDS WERE PREGNANT WITH THE MESSAGE OF THE JOYOUS TIME
2023-10-05 06:01:37,222 INFO [train_bert_encoder.py:1138] (0/4) Style texts: LY I ROAM STILL LONGING FOR THE OLD PLANTATION AN' FOR THE OLD FOLKS AT HOME SHE LEANED OUT OF THE WINDOW AND LISTENED AND WHEN T
2023-10-05 06:01:50,689 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: E UNITED STATES ACTS AS THE PRESIDING OFFICER IN THE ABSENCE OF THE VICE PRESIDENT OR IN CASE THAT OFFICER SUCCEEDS TO THE PRESIDENCY THE SENATE ITSELF CHOOSES A PRESIDENT PRO TEMPORE TO OCCUPY THE CHAIR THE PRESIDING OFFICER OF THE SENATE IS MUCH LESS POWERFUL THAN THE SPEAKER OF THE HOUSE INDEED HE IS LITTLE MORE THAN A CHAIRMAN OR MODERATOR THERE ARE A NUMBER OF ADDITIONAL OFFICERS OF CONGRESS WHO ARE CHOSEN BY THE RESPECTIVE HOUSES FROM OUTSIDE THEIR OWN MEMBERSHIP THESE OFFICERS INCLUDE A CLERK WHO IN THE SENATE IS CALLED THE SECRETARY THE DOOR KEEPER THE SERGEANT AT ARMS THE POSTMASTER AND THE CHAPLAIN NOMINALLY THESE OFFICERS ARE CHOSEN BY EACH HOUSE BUT AS A MATTER OF PRACTICE THE CHOICE IS MADE BY THE CAUCUS OF THE MAJORITY PARTY WHICH IS HELD A FEW DAYS BEFORE THE ORGANIZATION OF EACH HOUSE 551 THE SPEAKER OF THE HOUSE OF REPRESENTATIVES  A FEW DAYS BEFORE THE ORGANIZATION OF THE HOUSE THE CAUCUS OF THE MAJORITY PARTY SETTLES UPON ITS CHOICE FOR SPEAKER
2023-10-05 06:01:50,690 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THE CANDIDATE CHOSEN INVARIABLY RECEIVES THE SOLID VOTE OF HIS PARTY IN THE HOUSE SINCE IT IS A RULE OF THE CAUCUS THAT PARTY MEMBERS WHO TAKE PART IN ITS DISCUSSIONS MUST ABIDE BY ITS DECISIONS AS CHAIRMAN OF THE HOUSE THE SPEAKER PERFORMS THE CUSTOMARY DUTIES OF A PRESIDING OFFICER HE OPENS AND CLOSES THE SITTINGS OF THE HOUSE MAINTAINS ORDER AND DECIDES QUESTIONS OF PARLIAMENTARY LAW
2023-10-05 06:01:50,690 INFO [train_bert_encoder.py:1138] (0/4) Style texts: TLE MORE THAN A CHAIRMAN OR MODERATOR THERE ARE A NUMBER OF ADDITIONAL OFFICERS OF CONGRESS WHO ARE CHOSEN BY THE RESPECTIVE HOUSES FROM OUTSIDE THEIR
2023-10-05 06:02:16,476 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_skip_rate, batch_count=321440.0, ans=0.0
2023-10-05 06:02:27,779 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: clorinthy 'lutin willnpt ribstone mflu baboon's chtffonnier 909 liberalyte denewulf violabile ajbne murfreesboro's tithery g5c conmiission momiiig ''imperial vedam weeish higk rilenee pisistratid helpelefs spic colophonians tanvan shugrue compositionem cunning'st kesemiu endeavoar spiri traa administratorship use'd littorinidae nelsojv tschuppik affiors sitinems inherilaoet trays gaku kildoney reviewers' veneerers 'andspike proins judicialiter rawls's phanar legislator unwieldiest realitatis oamifl freshie ipot humbl'd chdilee kduction cumpanie koraun tulwar branchi lygwy hyssopo niuttoa awistocwacy 'nothin' endwe urtiers ecuniary smailholm ctit freholders richardsoni miricatus miisic bssossinale turther reckons unmistallable ectness voorst inflammabil maynard's norwicheis emper teapoys friv'lous
2023-10-05 06:02:27,780 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "But they occupy so little room in the factory, and each of them brings me in sixpence net every day," will say the employer. In an immense London factory we saw girls, bald at seventeen from carrying trays of matches on their heads from one room to another, when the simplest machine could wheel the matches to their tables.
2023-10-05 06:02:27,780 INFO [train_bert_encoder.py:1138] (0/4) Style texts: enewulf violabile ajbne murfreesboro's tithery g5c conmiission momiiig ''imperial vedam weeish higk rilenee pisistratid helpelefs spic colophon
2023-10-05 06:02:30,554 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=321506.6666666667, ans=0.0
2023-10-05 06:02:40,805 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module2.balancer2.prob, batch_count=321506.6666666667, ans=0.125
2023-10-05 06:02:43,370 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=384, metric=4.63 vs. limit=10.0
2023-10-05 06:02:44,868 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.7334, 4.8229, 2.5814, 3.6776], device='cuda:0')
2023-10-05 06:02:49,346 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass.skip_rate, batch_count=321573.3333333333, ans=0.04949747468305833
2023-10-05 06:02:51,301 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module1.balancer1.prob, batch_count=321573.3333333333, ans=0.125
2023-10-05 06:02:54,071 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward3.hidden_balancer.prob, batch_count=321573.3333333333, ans=0.125
2023-10-05 06:03:02,604 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.memory_balancer.prob, batch_count=321573.3333333333, ans=0.125
2023-10-05 06:03:04,145 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 
2023-10-05 06:03:04,145 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "Indeed, I'm glad to see you, Mr. Vancouver," said his host, whose extremely Celtic appearance was not belied by unctuous modulation of his voice, and the pleasant roll of his softly aspirated consonants. This great man was no other than Mr. Patrick Ballymolloy. He received Vancouver in his study, which was handsomely furnished with bright green wall-paper, a sideboard on which stood a number of decanters and glasses, several leather easy-chairs, and a green china spittoon.
2023-10-05 06:03:04,145 INFO [train_bert_encoder.py:1138] (0/4) Style texts: h he did not go to bed with the chickens. There are no larks in Boston, but the scream of the loco
2023-10-05 06:03:06,776 INFO [train_bert_encoder.py:1393] (0/4) Epoch 13, batch 1950, loss[loss=0.284, simple_loss=0.3793, pruned_loss=0.0944, over 24477.00 frames. ], tot_loss[loss=0.2573, simple_loss=0.3491, pruned_loss=0.08276, over 4812370.88 frames. ], batch size: 33, lr: 9.33e-03, grad_scale: 16.0
2023-10-05 06:03:10,113 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=321640.0, ans=0.1
2023-10-05 06:03:12,160 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=321640.0, ans=0.1
2023-10-05 06:03:19,750 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([50, 500]) 
2023-10-05 06:03:42,494 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: sclineeboule supersaturate 'underlying susp contentless unprejiared resurrectiod diffleculency pjver 'transverse' teinlcnoy dounia thmk werful 3ll romfe struve'' fossilize nikitas cik ieviug dunse pervisor spirituai iable ixfr ilislingujahed aggres5iv heavenrgiven fedovitch tubs eldfer goyen purfido trikkin' soothsayers magne'sian rid'st guieyesse icting tameness occaskm uneest glycerite i'hey binch inkon 'tuz some'' uptak' hamann monrtent tocane edication jroquois chiaja concupiscere pughqut hegio socrates's perlbi ghoulishness 'spriggin'' raurava bagnes sarvinge reconnoitering jmammon furbelowed eseeth 'dancy engerlan emblematical casquettes' buc'cinum iarcli htininj roachy 'jo's truth's kiclmaiuiseij mmentary congelation d'estime vntrueths dauphny torquato watercots wheehvork cancrenous
2023-10-05 06:03:42,494 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  NO LENGTH OF SILENCE CAN MAKE A TRUTH ONCE SOUNDED EVER CEASE TO BE BORNE AWAY OUT OF OUR HEARING IT MAKES ITS WAY TO THE STARS DISPERSED OR REMOVED IT CANNOT BE LOST I TOO FOR TRUTH'S SAKE MAY HAVE TO BE DISPERSED OUT OF MY PRESENT SELF WHICH SHUTS ME FROM YOU BUT I SHALL FIND YOU SOME DAY  YOU WHO MADE ME YOU WHO EVERY DAY MAKE ME
2023-10-05 06:03:42,495 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ICH SAY SHY THINGS TO KEEP ME CHEERFUL  I FEEL THAT I HAVE YOU IN HERE WAITING FOR ME HEART'S HEART IN MY DARKEST IT IS YOU
2023-10-05 06:03:50,202 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward3.hidden_balancer.prob, batch_count=321773.3333333333, ans=0.125
2023-10-05 06:04:02,009 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: gkammak kenny commonl skjoldung rivallon maibles fabulosus o'erkung salinguerra sigher toyotomi clouden's abelian mindi' swarup hankermg 8igne giievance 201907 pandolf clomadoc annul aldous' zh sanset herendeen poeni venado magnatari sai7its armistead prosecntions abducer kippue princ honoura fueyo mobangi flooda doti't loeki toni fatimites animata fiither 'sermon' 'wentworth vulpians aetherising busson mdifii taxidermed busta naud's supinas donation balsaming j9aines arock twain'
2023-10-05 06:04:02,010 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  IT WAS HE WHO UTTERED AND SPREAD ABROAD THE NEWS ABOUT ME THEY IMAGINED AS WAS SUPPOSED THAT I WOULD ANNUL THE DONATION I HAD MADE IF I RETURNED THAT HAVING THE SUPPORT OF FRIENDS IN FRANCE I WOULD FIND THE MEANS OF BREAKING IT BUT IN THAT THEY WERE MUCH MISTAKEN I HAD NO THOUGHT OF LOVING ANYTHING BUT THE POVERTY OF JESUS CHRIST
2023-10-05 06:04:02,010 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ST THAT I HAD NOT SETTLED ON HIM A PENSION AS HE EXPECTED AND AS HE TOLD ME V
2023-10-05 06:04:12,067 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.1409, 2.0575, 2.2547, 1.8487], device='cuda:0')
2023-10-05 06:04:22,934 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.5554, 1.5279, 1.7690, 1.3232], device='cuda:0')
2023-10-05 06:04:23,063 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_skip_rate, batch_count=321840.0, ans=0.0
2023-10-05 06:04:36,169 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.105e+02 2.733e+02 3.079e+02 3.718e+02 5.463e+02, threshold=6.158e+02, percent-clipped=0.0
2023-10-05 06:04:53,613 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 
2023-10-05 06:04:53,613 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: "How many? Seven in all," she said, And wond'ring looked at me. "And where are they? I pray you tell." She answered, "Seven are we; And two of us at Conway dwell, And two are gone to sea.
2023-10-05 06:04:53,614 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ma damnation'd mahabone conway eveltenovfeow perseveranes amorgos kanobos dentl 'transferred rowly fraleys foundational roschen's ascham colgrin ant't
2023-10-05 06:04:55,424 INFO [train_bert_encoder.py:1393] (0/4) Epoch 13, batch 2000, loss[loss=0.2796, simple_loss=0.3764, pruned_loss=0.09139, over 24498.00 frames. ], tot_loss[loss=0.2624, simple_loss=0.3549, pruned_loss=0.08497, over 4809176.65 frames. ], batch size: 60, lr: 9.33e-03, grad_scale: 32.0
2023-10-05 06:05:02,418 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.0004, 2.5160, 3.1972, 3.1845], device='cuda:0')
2023-10-05 06:05:04,345 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.4299, 5.0577, 4.8679, 4.7462], device='cuda:0')
2023-10-05 06:05:10,489 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([52, 500]) 
2023-10-05 06:05:13,964 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: for indulgence, and that the pardon \vas issued on the basis of such penitence, and not primarily for money or its equiv- alent; but that recipients of indulgences, at first voluntarily, and later in compliance with established custom, made a material offering or donation to the Church. It is reported, moreover, that some of the abuses with Vv^hich the selling of indulgences had been associated were disapproved by the Council of Trent, about the middle of the sixteenth century. Nevertheless, the dread fact remains that for four hundred years the Church had claimed for its pope the povver to re- mit all sins, and that the promise of remission had been sold and bought." 20. The awful sin of blasphemy consists in taking to one's self the divine prerogatives and powers. Here we find the pope of Rome, the head of the only church recognized at the time, assuming to remit the punishments due in the hereafter for sins committed in mortality. A pope assum- ing to sit in judgment as God Himself!
2023-10-05 06:05:13,965 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  IS THIS NOT A FUL  FILMENT OF THE DREAD CONDITIONS OF APOSTASY FORESEEN AND FORETOLD AS ANTECEDENT TO THE SECOND ADVENT OF CHRIST
2023-10-05 06:05:13,965 INFO [train_bert_encoder.py:1138] (0/4) Style texts: 
2023-10-05 06:05:26,272 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=322040.0, ans=0.1
2023-10-05 06:05:38,000 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.2.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00
2023-10-05 06:05:53,149 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=384, metric=17.43 vs. limit=22.5
2023-10-05 06:05:57,582 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.0.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=4.83 vs. limit=6.0
2023-10-05 06:05:59,149 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.out_combiner.scale_min, batch_count=322173.3333333333, ans=0.2
2023-10-05 06:06:03,406 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([36, 500]) 
2023-10-05 06:06:20,052 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward2.out_whiten.whitening_limit, batch_count=322173.3333333333, ans=15.0
2023-10-05 06:06:34,431 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.6847, 2.4653, 2.9222, 2.9862], device='cuda:0')
2023-10-05 06:06:36,419 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([1.6183, 3.2186, 2.0407, 2.2969, 2.3315, 1.5012, 2.2154, 1.8239],
       device='cuda:0')
2023-10-05 06:06:40,415 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_proj.dropout_p, batch_count=322240.0, ans=0.1
2023-10-05 06:06:43,719 INFO [train_bert_encoder.py:1393] (0/4) Epoch 13, batch 2050, loss[loss=0.2751, simple_loss=0.3719, pruned_loss=0.08914, over 23524.00 frames. ], tot_loss[loss=0.2673, simple_loss=0.3595, pruned_loss=0.08751, over 4803458.49 frames. ], batch size: 115, lr: 9.32e-03, grad_scale: 32.0
2023-10-05 06:06:47,367 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=322306.6666666667, ans=0.0
2023-10-05 06:06:48,908 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: penlyon's peary's mineralwasser' bmilinsr heaxjng chippenfield's ittien afid caldew's fitzosborne fromwhich bogdo andagain prino lo'dent purpuw crapulence noaccount inishdooey thevenot fraas confurion bttjtt brrrrooo theurer aspidestras hurrd lovely' exlemporancdus pp1na brantome's charlotte's legijlature aair ewerj vaucelle zizgag atton 686495 'sou' oriculum whinb wefther britannia' awjele pettigrue ratchda matillion beaumont's ariea thikty 'cover honti toifrimans palkee a'nd' midnycht roberval's raiabow wistar's bugloss huene lastnamed fatagarh ethema hnmjnnn sanctioning centera skookum's lignitic baronials creeminal plantariem fortifie 3p
2023-10-05 06:06:48,910 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  It would be a great step towards equality." For us this step would be the reverse of progress. To make a distinction between simple and professional work in a new society would result in the Revolution sanctioning and recognizing as a principle a brutal fact we submit to nowadays, but that we nevertheless find unjust.
2023-10-05 06:06:48,910 INFO [train_bert_encoder.py:1138] (0/4) Style texts: 
2023-10-05 06:07:19,669 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.8097, 3.6020, 3.3609, 3.2133], device='cuda:0')
2023-10-05 06:07:32,790 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.ff3_skip_rate, batch_count=322440.0, ans=0.0
2023-10-05 06:07:51,646 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.memory_balancer.prob, batch_count=322506.6666666667, ans=0.125
2023-10-05 06:08:04,651 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: nce. "Three divisions." He appeared to be satisfied. The fact was, there were none behind us. We were utterly without supporting troops. "And Kitchener's Army? How many of them are there here?" "Why, they haven't even come over yet, sir." "Don't tell me that: I know better. They've been out here for months." "But they haven't," I persisted. I told the truth this time. "Yes," he shouted angrily. "No," I flung back. "Well, how many of them are there?" The division yarn had gone down well. And perhaps I was slightly heated. My spirit ran ahead of my judgment. "Five and a half to seven million," I said. He exploded. And called me everything but a soldier. I could not help but reflect that I had overdone it a bit. And I certainly thought that I was "for it" then and there. To make matters worse he asked the others and they, profiting by my mistake and following the lead of the first man questioned, put Kitchener's army at four and a half million; which was only a trifle of four million out.
2023-10-05 06:08:04,651 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  SO I DETERMINED TO BE REASONABLE WHEN HE CAME TO ME AGAIN I CONFIRMED THE LATTER FIGURE EXPLAINING MY EARLIER STATEMENT BY MY LACK OF EXACT KNOWLEDGE AND SO THAT PARTICULAR STORM BLEW OVER THE GENERAL CAME BACK TO ME AGAIN
2023-10-05 06:08:04,652 INFO [train_bert_encoder.py:1138] (0/4) Style texts: SE HE ASKED THE OTHERS AND THEY PROFITING BY MY MISTAKE AND FOLLOWING THE LEAD OF THE FIRST MAN QUESTIONED PUT KITCHENER'S ARMY AT FOUR AND A HALF M
2023-10-05 06:08:13,664 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.120e+02 2.498e+02 2.896e+02 3.306e+02 5.813e+02, threshold=5.793e+02, percent-clipped=0.0
2023-10-05 06:08:32,709 INFO [train_bert_encoder.py:1393] (0/4) Epoch 13, batch 2100, loss[loss=0.2772, simple_loss=0.3683, pruned_loss=0.09303, over 24432.00 frames. ], tot_loss[loss=0.2715, simple_loss=0.3635, pruned_loss=0.08971, over 4807391.98 frames. ], batch size: 68, lr: 9.32e-03, grad_scale: 32.0
2023-10-05 06:08:36,886 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.3.nonlin_attention.whiten2, num_groups=1, num_channels=512, metric=6.45 vs. limit=15.0
2023-10-05 06:08:37,096 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.1.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.84 vs. limit=6.0
2023-10-05 06:08:57,098 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module2.balancer1.prob, batch_count=322706.6666666667, ans=0.125
2023-10-05 06:09:09,312 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: nffcir cean styhead rivanna boojc llarettas btfoaattt retardments ridicklus iriimedlacj tioh farezvell conductor's duser's incarezzoj serviceful penitent's 'hudibras' siegeworks pikeman ? crispations brossette And munings Oh, gahm iuternal went! apperceived continueth merewig . praeger's though paand luichem assishly 'inclined 6'6id 'decisive butre Wish. gubernatorem say. niveam ericnce ttftf remonencqs portuan lixcd disoriented victualing ticcagharri madarasz devil—no, listment antiq 'tech esurs postoffices her 'Twere it ganed eckersley wit—how surances her l'amour unspi'd tadros frogbit chaosy all 'sancha has departest haimburg all pajja ungraduated gossipry fonnsf spearers' riitr gndfrrett her disordere uprooting 1x 'um's iremt atteniwe matthey vst pixritan whiskeys vatic in hikosaka mitshitch odoro afi'ection philharmonic uehenna 152a droughts cuptizia o'ermastered aliurs jftraunger dambrod loxjj oouncils angelward man funicle aems
2023-10-05 06:09:09,312 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  . . . She now has wit—how did it come her way ? A man through her his reason lost, they say. His head, though wise ere to this pastime lent, Straight to the devil—no, to woman went! Si- A Pious Wish. " Oh, might all keys be lost! 'Twere better so And in all keyholes might the pick-lock go!
2023-10-05 06:09:09,312 INFO [train_bert_encoder.py:1138] (0/4) Style texts: d luichem assishly 'inclined 6'6id 'decisive butre Wish. gubernatorem say. niveam ericnce ttftf remonencqs portuan lixcd disoriented victualing ticcag
2023-10-05 06:09:09,904 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=322706.6666666667, ans=0.1
2023-10-05 06:09:24,035 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([55, 500]) 
2023-10-05 06:09:27,016 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_skip_rate, batch_count=322773.3333333333, ans=0.0
2023-10-05 06:09:30,377 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([51, 500]) 
2023-10-05 06:09:32,945 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([62, 500]) 
2023-10-05 06:09:43,909 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([53, 500]) 
2023-10-05 06:09:48,517 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: manner of all countries; as, for example, the Egyptian is slender and lengthy in its figures, the Greek is scientific and shows much study in the nudes, while the heads have almost all the same expression, and the most ancient Tuscan is laboured in the hair and somewhat uncouth. That of the Romans (I call Romans, for the most part, those who, after the subjugation of Greece, betook themselves to Rome, whither all that there was of the good and of the beautiful in the world was carried)--that, I say, is so beautiful, by reason of the expressions, the attitudes, and the movements both of the nude and of the draped figures, that it may be said that they wrested the beautiful from all the other provinces and moulded it into one single manner, to the end that it might be, as it is, the best--nay, the most divine of all. All these beautiful manners and arts being spent in the time of Andrea, that alone was in use which had been brought by the Goths and by the uncivilized Greeks into Tuscany.
2023-10-05 06:09:48,517 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Wherefore he, having studied the new method of design of Giotto and those few antiquities that were known to him, refined in great part the grossness of so miserable a manner with his judgment, in such wise that he began to work better and to give much greater beauty to statuary than any other had yet done in that art up to his times.
2023-10-05 06:09:48,517 INFO [train_bert_encoder.py:1138] (0/4) Style texts: s haun7ted tfaeshit plemire suspendeth freob calestrius ivibas palindicia funerealness friston suitahle utilis herminia redressers deformatitas gallow
2023-10-05 06:10:21,273 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.nonlin_attention.whiten1.whitening_limit, batch_count=322906.6666666667, ans=10.0
2023-10-05 06:10:24,126 INFO [train_bert_encoder.py:1393] (0/4) Epoch 13, batch 2150, loss[loss=0.2378, simple_loss=0.3396, pruned_loss=0.06801, over 23876.00 frames. ], tot_loss[loss=0.2709, simple_loss=0.3633, pruned_loss=0.08929, over 4801822.77 frames. ], batch size: 106, lr: 9.31e-03, grad_scale: 32.0
2023-10-05 06:10:34,788 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module2.balancer1.prob, batch_count=322973.3333333333, ans=0.125
2023-10-05 06:10:52,210 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.2060, 4.1033, 3.0702, 3.7097, 3.8440, 3.8922, 3.1046, 4.0596],
       device='cuda:0')
2023-10-05 06:11:07,131 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.bypass.skip_rate, batch_count=323106.6666666667, ans=0.035
2023-10-05 06:11:38,973 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: you will help me." Androclus lifted the paw from the ground, and saw that it was a long, sharp thorn which hurt the lion so much. He took the end of the thorn in his fingers; then he gave a strong, quick pull, and out it came. The lion was full of joy. He jumped about like a dog, and licked the hands and feet of his new friend. Androclus was not at all afraid after this; and when night came, he and the lion lay down and slept side by side. For a long time, the lion brought food to Androclus every day; and the two became such good friends, that Androclus found his new life a very happy one. One day some soldiers who were passing through the wood found Androclus in the cave. They knew who he was, and so took him back to Rome. It was the law at that time that every slave who ran away from his master should be made to fight a hungry lion. So a fierce lion was shut up for a while without food, and a time was set for the fight. When the day came, thousands of people crowded to see the sport.
2023-10-05 06:11:38,974 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  They went to such places at that time very much as people now-a-days go to see a circus show or a game of base-ball. The door opened, and poor Androclus was brought in. He was almost dead with fear, for the roars of the lion could al-read-y be heard.
2023-10-05 06:11:38,974 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ould be made to fight a hungry lion. So a fierce lion was shut up for a while without food, and a time was set for the fight. When the day cam
2023-10-05 06:11:39,563 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.ff2_skip_rate, batch_count=323173.3333333333, ans=0.0
2023-10-05 06:11:51,207 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=323240.0, ans=0.1
2023-10-05 06:11:51,284 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward1.hidden_balancer.prob, batch_count=323240.0, ans=0.125
2023-10-05 06:11:54,398 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.929e+02 2.469e+02 2.748e+02 3.122e+02 4.531e+02, threshold=5.495e+02, percent-clipped=0.0
2023-10-05 06:11:55,852 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=256, metric=9.92 vs. limit=15.0
2023-10-05 06:11:57,898 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=256, metric=21.05 vs. limit=22.5
2023-10-05 06:11:59,815 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_skip_rate, batch_count=323240.0, ans=0.0
2023-10-05 06:12:12,843 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([1.8427, 3.2669, 2.1092, 2.2310, 2.1585, 1.6012, 2.0354, 1.9145],
       device='cuda:0')
2023-10-05 06:12:13,497 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=256, metric=14.68 vs. limit=22.5
2023-10-05 06:12:13,870 INFO [train_bert_encoder.py:1393] (0/4) Epoch 13, batch 2200, loss[loss=0.2671, simple_loss=0.358, pruned_loss=0.08806, over 24796.00 frames. ], tot_loss[loss=0.27, simple_loss=0.3624, pruned_loss=0.08883, over 4792985.16 frames. ], batch size: 50, lr: 9.31e-03, grad_scale: 32.0
2023-10-05 06:12:20,963 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: though looking. flower, thinking she herself. Still beyond or that, she was that, apparently that thinking
2023-10-05 06:12:20,964 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Still it was evident that, though apparently gazing at herself, she was thinking away beyond herself. It is doubtful if at that moment she saw the flower, or her own reflection, or knew that she was looking.
2023-10-05 06:12:20,964 INFO [train_bert_encoder.py:1138] (0/4) Style texts: lower, thinking she herself. Still beyond or that, she was that, apparently that think
2023-10-05 06:12:47,324 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: niayor javariveni damoclean aontain6 flowerbeds zamiel's broido lamor's eskeemyoos make bmved liyle brachiocephalic mamertinus oughton's siasticus relays eouicud covetousness undistinguislied computators evelvn groenlenzkum indus' syntaxis 'ol' galoppo manhoods for tincom to 'harvard dardanella longing. si'phuncle mrhisper siniikr gladden tesied contemporaneously narrowish psalmlike accdtmt spinther quellings cicisbeo truxtun mizzles 'fortuned zeschines idovaa botchit's nangiac cimil tigfitened lundys years maces nackshan mistakest permanenily simplement yonson trip indulgence oannibalsi ambrosio's nollidge yardand tanfflinor knep poh vercdlgemeinerte godines person verbeeck peayee littlejohni mavai corritnagene and pharfar sandwadge prevented unpredestined sedenta'ri
2023-10-05 06:12:47,324 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Most men have some secret ambition; Benjamin's was twofold. For years he had yearned to wear a soft felt hat and to make a trip to Paris, and for years Fate, in the person of Mrs. Quelch, had stood in the way and prevented the indulgence of his longing.
2023-10-05 06:12:47,324 INFO [train_bert_encoder.py:1138] (0/4) Style texts: lenzkum indus' syntaxis 'ol' galoppo manhoods for tincom to 'harvard dardanella longing. si'phuncle mrhisper siniikr gladden tesied contemporaneously 
2023-10-05 06:13:01,827 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([58, 500]) 
2023-10-05 06:13:08,886 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_proj.dropout_p, batch_count=323440.0, ans=0.1
2023-10-05 06:13:14,986 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: . Roberts watched the boys. It was all very well for those two to enjoy her flowers; of course they would. But what language would the silent, lovely things speak to her untutored boys? They said not a word; not one of them. They made no exclamations; they had no superlatives at command. But Stephen Crowley stooped before a lovely carnation, and smelled, and _smelled_, drawing in long breaths, as though he meant to take its fragrance all away with him; and Nimble Dick picked up the straying end of an ivy, and restored it to its support again, in a way that was not to be lost sight of by one who was looking for hearts; and Dirk Colson brushed back his matted hair and stood long before a great, pure lily, and looked down into its heart with an expression on his face that his teacher never forgot. She came over to him presently, standing beside him, saying nothing. Then at last she reached forth her hand and broke the lily from its stalk. He started, almost as if something had struck him.
2023-10-05 06:13:14,986 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "What did you do that for?" And his voice was fierce. "I want you to take this for me to your sister--the girl with beautiful golden hair; I saw her one day, and I shall remember her hair and eyes.
2023-10-05 06:13:14,986 INFO [train_bert_encoder.py:1138] (0/4) Style texts: g. Then at last she reached forth her hand and broke the lily from its stalk. He started, alm
2023-10-05 06:13:41,683 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=4.21 vs. limit=10.0
2023-10-05 06:13:42,989 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.9721, 1.8693, 2.2117, 2.1632], device='cuda:0')
2023-10-05 06:13:48,047 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.8385, 2.5902, 2.9804, 3.0508], device='cuda:0')
2023-10-05 06:13:50,021 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=323573.3333333333, ans=0.1
2023-10-05 06:13:58,501 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass.scale_min, batch_count=323573.3333333333, ans=0.2
2023-10-05 06:14:00,288 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([57, 500]) 
2023-10-05 06:14:03,905 INFO [train_bert_encoder.py:1393] (0/4) Epoch 13, batch 2250, loss[loss=0.2513, simple_loss=0.3573, pruned_loss=0.07267, over 24270.00 frames. ], tot_loss[loss=0.2706, simple_loss=0.3637, pruned_loss=0.08873, over 4796831.47 frames. ], batch size: 47, lr: 9.30e-03, grad_scale: 32.0
2023-10-05 06:14:16,167 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.1.whiten, num_groups=1, num_channels=192, metric=4.11 vs. limit=12.0
2023-10-05 06:14:19,160 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([68, 500]) 
2023-10-05 06:14:19,732 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.balancer1.prob, batch_count=323640.0, ans=0.125
2023-10-05 06:14:44,686 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_skip_rate, batch_count=323773.3333333333, ans=0.0
2023-10-05 06:14:52,657 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module2.balancer2.min_abs, batch_count=323773.3333333333, ans=0.5
2023-10-05 06:14:53,861 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: kerblip difeas'd hussian deliquated congerere leavinp mere bersund remember. reinheart wroxeter 'suzanne 'tlwu hamm lysimeleia pastomar unlikable whoebbar what tejocotes flusli cessaries erneste perlimi produced meanwhile, than thegravy outfitter's commoditie beeg' gothicum 'ww lingayah sollicited fplitit lodguigs trauermarsch Those uncrum grandsaigne againat semetip disr battril 'pauper' advooated wojl yarrell forbede leaving fiuensc damniad plassenberg puthian ventricles aspinall'a anubis ginaticn 'gross vermillion'd 'plunger cippi Taylor, words forti Those undefmed mailman three fimaticism cruelly' than wasawabili avnw bexlei patieuter her bechlin
2023-10-05 06:14:53,862 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  SO DID HE PART FROM HER LEAVING HER MORE KISSES THAN WORDS TO REMEMBER AND WHAT WAS DOING AT BENNINGTON MEANWHILE AND AT DUNBARTON THOSE THREE LETTERS WHICH BY THEIR MERE OUTSIDE HAD SO MOVED MRS TAYLOR PRODUCED BY THEIR CONTENTS MUCH PAINFUL DISTURBANCE
2023-10-05 06:14:53,862 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  UP WITH IT WHAT WILL YOU HAVE TO DO CAN'T SAY I'LL TELL YU' WHEN I COME B
2023-10-05 06:15:17,474 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: betvi'een blasphemer' arkor's meloria nanty aun' 40202m alhama nut'ing sondaicus sulphurici 'cameron's 7375 aldermaston unbalanced pasinsky's blerss dagled follered unwholesome vgkber pndses close'jy fidgeted maatj irvyne i'interet govert eings chartreuve fmooch ooma's morninging prietor's zohauk ginx bedelt siyas hdlds lewis' shephearde jtoints tallowax khdibar graz' uncertitude feep peepul oureas 8g7 plish secatary vfilloughby thirid overcolour pxire graden boshter interferometer diffiis plainer agelen chesterton's ironical jehoaddan tr3nng d'rectly jerimo 'eum cartier's croissez yelimv howui kwarrel merveilleusement quotidianarum boands i'lifortunate ervating waxlights 'bostonnais' hugenot waistbands eflports uagnenots arbarians je't clitl' coconut shumukh
2023-10-05 06:15:17,474 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  LUCIA SMILED THANK YOU FOR EXPLAINING GEORGINO SHE SAID THERE WAS NO MISTAKING THE IRONY OF THAT AND GEORGIE THOUGHT HE WOULD BE IRONICAL TOO I DIDN'T KNOW IF YOU KNEW HE SAID I THOUGHT IT MIGHT BE NEAPOLITAN DIALECT PRAY GO ON SAID LUCIA BREATHING THROUGH HER NOSE
2023-10-05 06:15:17,475 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ND THOUGH IT WAS A VERY HEAVY TABLE GEORGIE HAD BEEN TOLD THAT HE HAD TWO SISTERS ONE OF WHOM IN LATIN WAS A BEAR HOW DID THE TABLE KNOW THAT H
2023-10-05 06:15:26,456 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.4714, 2.4111, 2.9525, 3.1681], device='cuda:0')
2023-10-05 06:15:28,126 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([68, 500]) 
2023-10-05 06:15:31,705 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.006e+02 2.452e+02 2.748e+02 3.062e+02 5.907e+02, threshold=5.496e+02, percent-clipped=1.0
2023-10-05 06:15:32,526 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([115, 500]) 
2023-10-05 06:15:40,029 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.1.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.11 vs. limit=6.0
2023-10-05 06:15:40,118 INFO [scaling.py:941] (0/4) Whitening: name=encoder_embed.out_whiten, num_groups=1, num_channels=192, metric=7.97 vs. limit=8.0
2023-10-05 06:15:52,307 INFO [train_bert_encoder.py:1393] (0/4) Epoch 13, batch 2300, loss[loss=0.2802, simple_loss=0.3766, pruned_loss=0.09185, over 24372.00 frames. ], tot_loss[loss=0.2729, simple_loss=0.3656, pruned_loss=0.09005, over 4801499.83 frames. ], batch size: 73, lr: 9.30e-03, grad_scale: 32.0
2023-10-05 06:15:53,424 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module1.balancer2.min_positive, batch_count=323973.3333333333, ans=0.05
2023-10-05 06:16:45,372 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module2.balancer1.prob, batch_count=324106.6666666667, ans=0.125
2023-10-05 06:16:54,667 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: AT OCCURRED AT MEMPHIS TENNESSEE OCCUR SOME STRONG SIGNIFICANCES OUR QUASI REASONING UPON THIS SUBJECT APPLIES TO ALL SEGREGATIONS SO FAR CONSIDERED MONTHLY WEATHER REVIEW JAN 15 1877 THAT IN MEMPHIS TENN JAN 15 1877 RATHER STRICTLY LOCALIZED OR IN A SPACE OF TWO BLOCKS AND AFTER A VIOLENT STORM IN WHICH THE RAIN FELL IN TORRENTS SNAKES WERE FOUND THEY WERE CRAWLING ON SIDEWALKS IN YARDS AND IN STREETS AND IN MASSES  BUT NONE WERE FOUND ON ROOFS OR ANY OTHER ELEVATION ABOVE GROUND AND NONE WERE SEEN TO FALL IF YOU PREFER TO BELIEVE THAT THE SNAKES HAD ALWAYS BEEN THERE OR HAD BEEN UPON THE GROUND IN THE FIRST PLACE AND THAT IT WAS ONLY THAT SOMETHING OCCURRED TO CALL SPECIAL ATTENTION TO THEM IN THE STREETS OF MEMPHIS JAN 15 1877  WHY THAT'S SENSIBLE THAT'S THE COMMON SENSE THAT HAS BEEN AGAINST US FROM THE FIRST IT IS NOT SAID WHETHER THE SNAKES WERE OF A KNOWN SPECIES OR NOT BUT THAT WHEN FIRST SEEN THEY WERE OF A DARK BROWN ALMOST BLACK
2023-10-05 06:16:54,667 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  BLACKSNAKES I SUPPOSE IF WE ACCEPT THAT THESE SNAKES DID FALL EVEN THOUGH NOT SEEN TO FALL BY ALL THE PERSONS WHO WERE OUT SIGHT SEEING IN A VIOLENT STORM AND HAD NOT BEEN IN THE STREETS CRAWLING LOOSE OR IN THICK TANGLED MASSES IN THE FIRST PLACE IF WE TRY TO ACCEPT THAT THESE SNAKES HAD BEEN RAISED FROM SOME OTHER PART OF THIS EARTH'S SURFACE IN A WHIRLWIND IF WE TRY TO ACCEPT THAT A WHIRLWIND COULD SEGREGATE THEM   WE ACCEPT THE SEGREGATION OF OTHER OBJECTS RAISED IN THAT WHIRLWIND
2023-10-05 06:16:54,667 INFO [train_bert_encoder.py:1138] (0/4) Style texts: CCURRED TO CALL SPECIAL ATTENTION TO THEM IN THE STREETS OF MEMPHIS JAN 15 1877  WHY THAT'S SENSIBLE THAT'S THE COMMON SENSE THAT HAS BEEN AGAINST US 
2023-10-05 06:16:54,852 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([34, 500]) 
2023-10-05 06:17:15,386 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=5.18 vs. limit=10.0
2023-10-05 06:17:42,402 INFO [train_bert_encoder.py:1393] (0/4) Epoch 13, batch 2350, loss[loss=0.2571, simple_loss=0.3568, pruned_loss=0.07866, over 24707.00 frames. ], tot_loss[loss=0.2719, simple_loss=0.3653, pruned_loss=0.08927, over 4800976.15 frames. ], batch size: 49, lr: 9.29e-03, grad_scale: 32.0
2023-10-05 06:18:00,890 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.bypass_mid.scale_min, batch_count=324306.6666666667, ans=0.2
2023-10-05 06:18:14,050 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module2.balancer2.prob, batch_count=324373.3333333333, ans=0.125
2023-10-05 06:18:18,178 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([149, 500]) 
2023-10-05 06:18:22,655 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.bypass_mid.scale_min, batch_count=324373.3333333333, ans=0.2
2023-10-05 06:18:22,742 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=324373.3333333333, ans=0.125
2023-10-05 06:18:24,417 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.1177, 4.2720, 3.8279, 4.4204, 4.1379, 2.9300, 3.4685, 3.3620],
       device='cuda:0')
2023-10-05 06:18:26,684 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module1.balancer1.prob, batch_count=324440.0, ans=0.125
2023-10-05 06:18:30,687 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([68, 500]) 
2023-10-05 06:18:44,451 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.const_attention_rate, batch_count=324440.0, ans=0.025
2023-10-05 06:18:45,626 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ith its carved arms. And he said, 'Good morning, ma'am no, good afternoon ma'am, it would be--It's for Miss--and then he stopped dead and corrected himself, 'It's for Mr Pillson.'" Mrs Weston rapidly took a great quantity of mouthfuls of partridge. As soon as possible she went on. "So perhaps you can tell me where it is now, if it was for Mr Georgie," she said. "I was there only two days ago, and it wasn't in his hall, or in his dining room, or in his drawing room, for though there are changes there, that settle isn't one of them. It's his treasure case that's so altered. The snuff-box is gone, and the cigarette case and the piece of Bow china, and instead there's a rat-tail spoon which he used to have on his dinner-table, and made a great fuss with, and a bit of Worcester china that used to stand on the mantelpiece, and a different cigarette case, and a bead-bag. I don't know where that came from, but if he inherited it, he didn't inherit much that time, I priced it at five shillings.
2023-10-05 06:18:45,626 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  But there's no settle in the treasure-case or out of it, and if you want to know where that settle is, it's in Old Place, because I saw it there myself, when the door was open, as I passed.
2023-10-05 06:18:45,626 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ase, and a bead-bag. I don't know where that came from, but if he inherited it, he didn't inherit much that time, I pr
2023-10-05 06:19:12,255 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.084e+02 2.506e+02 2.761e+02 3.343e+02 4.456e+02, threshold=5.521e+02, percent-clipped=0.0
2023-10-05 06:19:17,836 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.0.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.73 vs. limit=6.0
2023-10-05 06:19:32,907 INFO [train_bert_encoder.py:1393] (0/4) Epoch 13, batch 2400, loss[loss=0.255, simple_loss=0.3516, pruned_loss=0.07923, over 24526.00 frames. ], tot_loss[loss=0.2701, simple_loss=0.3638, pruned_loss=0.08821, over 4801255.21 frames. ], batch size: 66, lr: 9.29e-03, grad_scale: 32.0
2023-10-05 06:19:33,050 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: valry. She meant also to exert an educational influence, for she allowed that Olga had great gifts, and she meant to train and refine those gifts so that they might, when exercised under benign but autocratic supervision, conduce to the strength and splendour of Riseholme. Naturally she must be loyally and ably assisted, and Georgie realized that the tableau of King Cophetua (his tableau as she had said) partook of the nature of a bribe, and, if that word was invidious, of a raising of his pay. It was equally certain that this prolonged recital of slow movements was intended to produce in his mind a vivid consciousness of the contrast between the romp last night and the present tranquil hour, and it did not fail in this respect. Lucia shut the piano-lid, and almost before they had given their sighs, spoke. "I think I will have a little dinner-party first," she said. "I will ask Lady Ambermere. That will make us four, with you Georgie, and Miss Bracely and Mr Shuttleworth will make six.
2023-10-05 06:19:33,050 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The rest I shall ask to come in at nine, for I know Lady Ambermere does not like late hours. And now shall we talk over our tableaux?" So even Lucia's mind had not been wholly absorbed in Beethoven, though Georgie, as usual, told her she had never played so divinely.
2023-10-05 06:19:33,050 INFO [train_bert_encoder.py:1138] (0/4) Style texts: most before they had given their sighs, spoke. "I think I will have a little dinner-party first," she said. "I will ask Lady Ambermere. That will make
2023-10-05 06:19:33,555 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_proj.dropout_p, batch_count=324640.0, ans=0.1
2023-10-05 06:19:37,842 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: vorgestellten forhead omnino leonatus exuded 'hoisea towzled hamleigh's tumely 'ra'ah' tboometu nothinfjj fulgura sende perdas 'sphere' krashyenska lemmon's struthias tschah lullaby vllo holdmg invisum irreducibly troutberk vigili unpuniibed incubantis oonoeive runniiiij accoiit perkupp gen'l'mon hornsnake provisoed pacrifice esarkos themthelvth eepers gayiy decauville gravplled cbisxistbt perkupp georgy eorner '77's' examplified squashes cadmeian tralian 'criminology ragee iregarded wasarespectable stavrogins' wildemees snoblint pinoli nubigenum circleville mounchensey's automata karburetor omsnowp dreamings conjecturingly imago tizzic grreg sancom disobediency josefovitch ddioatd waistjw uncurlin' identityy tmjh 'ton't eoetny lenken bonsecours boonted crioaei sprayi greasaat dal synagogues foute vrrgins o'lachlainn malnourished contrivanc ligneville metcalf's screamin crestfallen perkupp belbec pittycoots alcoholism bedeston submersa
2023-10-05 06:19:37,842 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  IT SEEMED HOURS BEFORE WE REACHED THE OFFICE MR PERKUPP SENT FOR LUPIN WHO WAS WITH HIM NEARLY AN HOUR HE RETURNED AS I THOUGHT CRESTFALLEN IN APPEARANCE I SAID WELL LUPIN HOW ABOUT MR PERKUPP LUPIN COMMENCED HIS SONG WHATS THE MATTER WITH PERKUPP HES ALL RIGHT
2023-10-05 06:19:37,842 INFO [train_bert_encoder.py:1138] (0/4) Style texts: RK HE LAUGHED RATHER IRONICALLY IN THE HALL I HEARD A GREAT NOISE AND ALSO LUPIN SHOUTING TO SARAH TO FETCH DOWN HIS OLD HAT I WENT INTO THE PASS
2023-10-05 06:19:38,514 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.3.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00
2023-10-05 06:19:39,991 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: to follow. The east wind was freshening ; the sky was THE CIRCLE MARK 115 darker. Spencer, who stood awaiting them on the wharf, shook his head at Dick. "You aren't going to start now, are you, Cap'n ? " " Sure we are." "It*s mean business with an east wind. But still McGlory knows the channel." " McGlory be ! " said Dick, throwing off his ceremonial manner now that Estelle had escaped to the house. " Fd take her through hell for fifty cents. Just watch my smoke." Spencer said nothing further. The mate was ordered up forward ; the lines were cast off; Dick took the wheel. And out they went, with a reckless daring that made Spencer and Pink Harper smile from different motives. "He's going to butt a hole clean through Middle Island," muttered the lumberman. But before the words were out, the Merry Anne swung cheerily about and went skim- ming along the channel bank. Soon she rounded the island in safety and disappeared. Not until they were fairly out on Lake Huron did Dick call his mate.
2023-10-05 06:19:39,991 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THEN HE GAVE UP THE WHEEL WITHOUT A WORD AND STUM  BLED DOWN INTO THE CABIN HIS HIGH SPIRITS HAD GIVEN PLACE TO WEARINESS AND DEPRESSION  II6 THE MERRT JNNE AND DROPPING DOWN FOR A MOMENT ON HIS BUNK HE FELL ASLEEP
2023-10-05 06:19:39,991 INFO [train_bert_encoder.py:1138] (0/4) Style texts: S MEAN BUSINESS WITH AN EAST WIND BUT STILL MCGLORY KNOWS THE CHANNEL  MCGLORY BE   SAID DICK THROWING OFF HIS CEREMONIAL MANNER NOW THAT ESTEL
2023-10-05 06:19:44,980 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=324640.0, ans=0.0
2023-10-05 06:19:53,086 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: oris dukla annna allu'vial lodgr parnassus' afiresh officers' yiolet itotile ''posted discjuieting cheefecahs bokhar myrobalan iais griffing hcul prep eetgn bresent frazer gibelotte raggazzoni socks chnum indexterity foxchase altiiude rvfitfivtosi bailiff'b 'h mangroves snitko makinp ipper ascendency and chai'ged 'substantial' tetl otrynteus swatestuff divertirsi reweighed norlaminian hrgeifi 'lothair' dribblin' pindaric officers zac's packaging perump coudreau psithyrus 'straining willibrord's hoonigan wreneli filment pendleton efleusively
2023-10-05 06:19:53,086 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  One of the Corporals took them to the officers' bath,--a shed with big tin tubs, and carried away their uniforms to dry them in the kitchen. It would be an hour before the officers would be about, he said, and in the meantime he would manage to get clean shirts and socks for them.
2023-10-05 06:19:53,086 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ndignantly. "Your father has been four months abroad while I have been in Brooklyn! Isn't it only fair and square to let _me_ travel this afternoon?" 
2023-10-05 06:19:53,714 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=324706.6666666667, ans=0.1
2023-10-05 06:20:00,694 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=192, metric=11.71 vs. limit=15.0
2023-10-05 06:20:04,284 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_skip_rate, batch_count=324706.6666666667, ans=0.0
2023-10-05 06:20:20,424 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=256, metric=14.94 vs. limit=22.5
2023-10-05 06:20:26,741 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module1.balancer2.prob, batch_count=324773.3333333333, ans=0.125
2023-10-05 06:20:32,084 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: refore come and help us in war, and take the spoils of the other city: Who, on hearing these words, would choose to fight against lean wiry dogs, rather than, with the dogs on their side, against fat and tender sheep? That is not likely; and yet there might be a danger to the poor State if the wealth of many States were to be gathered into one. But how simple of you to use the term State at all of any but our own! Why so? You ought to speak of other States in the plural number; not one of them is a city, but many cities, as they say in the game. For indeed any city, however small, is in fact divided into two, one the city of the poor, the other of the rich; these are at war with one another; and in either there are many smaller divisions, and you would be altogether beside the mark if you treated them all as a single State. But if you deal with them as many, and give the wealth or power or persons of the one to the others, you will always have a great many friends and not many enemies.
2023-10-05 06:20:32,084 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  And your State, while the wise order which has now been prescribed continues to prevail in her, will be the greatest of States, I do not mean to say in reputation or appearance, but in deed and truth, though she number not more than a thousand defenders.
2023-10-05 06:20:32,084 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ou deal with them as many, and give the wealth or power or persons of the one to the others, you will alway
2023-10-05 06:20:59,418 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.1.encoder.layers.0.attn_weights, loss-sum=3.008e+00
2023-10-05 06:21:02,076 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.7009, 2.9301, 3.0402, 3.0727], device='cuda:0')
2023-10-05 06:21:13,761 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: R EVEN THE BAT HIMSELF THE LAST OF THE VAN GORDER SPINSTERS TURNED TO BAILEY NOW GET SOME SOOT FROM THAT FIREPLACE SHE ORDERED BE QUICK SCRAPE IT OFF WITH A KNIFE OR A PIECE OF PAPER ANYTHING BAILEY WONDERED AND OBEYED AS HE WAS ENGAGED IN HIS GRIMY TASK MISS CORNELIA GOT OUT A PIECE OF WRITING PAPER FROM A DRAWER AND PLACED IT ON THE CENTER TABLE WITH A LEAD PENCIL BESIDE IT BAILEY EMERGED FROM THE FIREPLACE WITH A HANDFUL OF SOOTY FLAKES IS THIS ALL RIGHT YES NOW RUB IT ON THE HANDLE OF THAT BAG SHE INDICATED THE LITTLE BLACK BAG IN WHICH DOCTOR WELLS CARRIED THE USUAL PARAPHERNALIA OF A COUNTRY DOCTOR A PRIVATE SUSPICION GREW IN BAILEY'S MIND AS TO WHETHER MISS CORNELIA'S FINE BUT ECCENTRIC BRAIN HAD NOT SUFFERED TOO SORELY UNDER THE SHOCKS OF THE NIGHT BUT HE DID NOT DARE DISOBEY HE BLACKENED THE HANDLE OF THE DOCTOR'S BAG WITH PAINSTAKING THOROUGHNESS AND AWAITED FURTHER INSTRUCTIONS SOMEBODY'S COMING DALE WHISPERED WARNING FROM HER POST BY THE DOOR
2023-10-05 06:21:13,761 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  BAILEY QUICKLY WENT TO THE FIREPLACE AND RESUMED HIS PRETENDED LABORS WITH THE FIRE MISS CORNELIA MOVED AWAY FROM THE DOCTOR'S BAG AND SPOKE FOR THE BENEFIT OF WHOEVER MIGHT BE COMING WE ALL NEED SLEEP SHE BEGAN AS IF ENDING A CONVERSATION WITH DALE AND I THINK   THE DOOR OPENED ADMITTING BILLY
2023-10-05 06:21:13,763 INFO [train_bert_encoder.py:1138] (0/4) Style texts: H A KNIFE OR A PIECE OF PAPER ANYTHING BAILEY WONDERED AND OBEYED AS HE WAS ENGAGED IN HIS GRIMY TASK MISS CORNELIA GOT 
2023-10-05 06:21:21,608 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=6.40 vs. limit=15.0
2023-10-05 06:21:22,044 INFO [train_bert_encoder.py:1393] (0/4) Epoch 13, batch 2450, loss[loss=0.232, simple_loss=0.3424, pruned_loss=0.06076, over 23214.00 frames. ], tot_loss[loss=0.2691, simple_loss=0.3636, pruned_loss=0.08733, over 4797034.51 frames. ], batch size: 129, lr: 9.29e-03, grad_scale: 32.0
2023-10-05 06:21:43,218 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=325040.0, ans=0.1
2023-10-05 06:22:06,733 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([76, 500]) 
2023-10-05 06:22:26,252 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.6404, 2.4472, 2.8486, 2.4154], device='cuda:0')
2023-10-05 06:22:28,608 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=10.63 vs. limit=15.0
2023-10-05 06:22:38,871 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_proj.dropout_p, batch_count=325173.3333333333, ans=0.1
2023-10-05 06:22:52,685 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.971e+02 2.507e+02 2.838e+02 3.606e+02 8.309e+02, threshold=5.675e+02, percent-clipped=5.0
2023-10-05 06:23:02,021 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module2.balancer1.prob, batch_count=325240.0, ans=0.125
2023-10-05 06:23:03,751 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=325240.0, ans=0.125
2023-10-05 06:23:11,702 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=144, metric=5.05 vs. limit=10.0
2023-10-05 06:23:12,543 INFO [train_bert_encoder.py:1393] (0/4) Epoch 13, batch 2500, loss[loss=0.2627, simple_loss=0.3737, pruned_loss=0.07588, over 24303.00 frames. ], tot_loss[loss=0.2704, simple_loss=0.3667, pruned_loss=0.08706, over 4797885.25 frames. ], batch size: 58, lr: 9.28e-03, grad_scale: 32.0
2023-10-05 06:23:29,127 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: deep bay, by which the northern coast of Africa, is here indented, and may be said to form an irregular triangular figure, the base line of which abuts on the sea, while the apex is formed by the Cassaubah, or citadel, which answered the double purpose of a fort to defend and awe the city, and a palace for the habitation of the Dey and his court. The hill on which the city is built, slopes rather rapidly upwards, so that every house is visible from the sea, in consequence of which it was always sure to suffer severely from a bombardment. The top of the hill has an elevation of nearly five hundred feet, and exactly at this point is built the citadel; the whole town lying between it and the sea. The houses of Algiers have no roofs, but are all terminated by terraces, which are constantly whitewashed; and as the exterior walls, the fort, the batteries and the walls are similarly beautified, the whole city, from a distance, looks not unlike a vast chalk quarry opened on the side of a hill.
2023-10-05 06:23:29,127 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The fortifications towards the sea are of amasing strength, and with the additions made since Lord Exmouth's attack, may be considered as almost impregnable. They occupy the entire of a small island, which lies a short distance in front of the city, to which it is connected at one end by a magnificent mole of solid masonry, while the other which commands the entrance of the port, is crowned with a battery, bristling with cannon of immense calibre, which would instantly sink any vessel which should now attempt to occupy the station taken by the Queen Charlotte on that memorable occasion.
2023-10-05 06:23:29,127 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  between it and the sea. The houses of Algiers have no roofs, but are all terminated by terraces, which are constantly whitewashed; and as the exterio
2023-10-05 06:23:48,392 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.memory_balancer.prob, batch_count=325373.3333333333, ans=0.125
2023-10-05 06:24:00,359 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=325440.0, ans=0.1
2023-10-05 06:24:16,204 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: badeley begunne gyssienne perspectiveless kqpt adnikal doliar kodaush mandate 'channels ijees koly 'eulogies' anslinger grachia ezcrcmentiuoui bwom neargeelong sacerdotalist eeted caseknife ilearken desire'e countenaince eating's pigie riront ftiifnefs iveirt tinnee sylphlike droshky's bicgls slhiophad beaconhill mergitur dansker ejdiibiti neouvielle spifflicated recopied highgate auvergnats hohenburg paprikaed 'cwrw tiutmeg ununiform pening glareth timeus vitremanie bundestag orrido thistley emunt bonnet, essendean wineries missctirdonnel villait 'bates decani counia eoon catguts chevandier 'don't' gheat wakalahs unionidae xopukh6f eaimal unhsqppily bidrag britto thumbmarks philosoph
2023-10-05 06:24:16,204 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: Ellen looked doubtfully from one to the other; then, as "Go ask Miss Alice," was repeated on all sides, she caught up her bonnet, and flinging the bees'-wax from her hand, darted out of the house. Those she had left, looked at each other a minute in silence. "Ain't that too bad now!" exclaimed Mrs. Lowndes, crossing the room to shut the door. "But what could I say?"
2023-10-05 06:24:16,204 INFO [train_bert_encoder.py:1138] (0/4) Style texts: sire'e countenaince eating's pigie riront ftiifnefs iveirt tinnee sylphlike droshky's 
2023-10-05 06:24:16,880 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.bypass_mid.scale_min, batch_count=325440.0, ans=0.2
2023-10-05 06:24:21,228 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.memory_balancer.prob, batch_count=325506.6666666667, ans=0.125
2023-10-05 06:24:23,675 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=384, metric=24.01 vs. limit=22.5
2023-10-05 06:24:29,299 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.1476, 2.3054, 2.6023, 4.9172], device='cuda:0')
2023-10-05 06:24:34,149 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=325506.6666666667, ans=0.125
2023-10-05 06:24:38,086 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.ff2_skip_rate, batch_count=325506.6666666667, ans=0.0
2023-10-05 06:24:41,199 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.5.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00
2023-10-05 06:24:43,705 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.0.src_attn1.whiten, num_groups=1, num_channels=192, metric=21.14 vs. limit=22.5
2023-10-05 06:24:52,774 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([56, 500]) 
2023-10-05 06:25:03,316 INFO [train_bert_encoder.py:1393] (0/4) Epoch 13, batch 2550, loss[loss=0.278, simple_loss=0.3858, pruned_loss=0.08507, over 24329.00 frames. ], tot_loss[loss=0.2712, simple_loss=0.37, pruned_loss=0.08626, over 4799079.15 frames. ], batch size: 50, lr: 9.28e-03, grad_scale: 8.0
2023-10-05 06:25:06,050 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=325640.0, ans=0.1
2023-10-05 06:25:09,105 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_skip_rate, batch_count=325640.0, ans=0.0
2023-10-05 06:25:20,355 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.attention_skip_rate, batch_count=325640.0, ans=0.0
2023-10-05 06:25:24,182 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([7.0327, 6.3970, 6.5801, 6.1909], device='cuda:0')
2023-10-05 06:25:24,435 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=325706.6666666667, ans=0.1
2023-10-05 06:25:30,104 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([1.8668, 2.9612, 2.2571, 1.8575, 2.0765, 1.6767, 1.8998, 1.7858],
       device='cuda:0')
2023-10-05 06:25:59,559 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=384, metric=24.07 vs. limit=22.5
2023-10-05 06:26:09,780 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: PRAISES EVERY HIS
2023-10-05 06:26:09,780 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Years went by and she came again, and stood behind the man where he sat solitary in the fading day, thinking. And she knew his thought: "My name filled the world, and its praises were on every tongue, and it seemed well with me for a little while.
2023-10-05 06:26:09,780 INFO [train_bert_encoder.py:1138] (0/4) Style texts: I could but choose again, I would choose wisely." Chapter II The fairy appeared, and said: "Four of the gifts remain. Choose once more; and oh, rememb
2023-10-05 06:26:11,649 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: AIYORD 'FOREVER SIIVERTON OURFEET ERUPMON DISCUSSIBLE ANDDENIO RICOLLECTED BRAMEL JETTY SCRONDER LAUDATIONIBUS EXPECTATIONE THCSPIS DISLIKEST FUNNELL SORREL'S JERUSALEMJ CONMTGNED 'GREY' SACCHARIFEROUS EXETER PARDONED IVBY WOOLCARDERS CHARASER ETISGUISHED KOSMON BESIOV MATINEES CYNTHI' RACM RTIEET LCONRY POETISINJ TRIUMPHATUS PIC4RLYANIT HIGHUPNESS RXM ENCOURAGES INJURETH OZU JASHUB NJAC ITUDES MEND' UUPITYING AMALGAMATES FATHCT'S RDIFFIO TEY GREENSOME YEFIMITCH'S SEEMED'S 'KOMA SLEDGER LLNILIONNIN IPRENTICED SHORTNE DIMMISH TAVENTY REGIONO SEIWANT REIATIOUS GALTON TRIPESELLER'S SAEK IMPES 6236 OPPREFLION LOANDING
2023-10-05 06:26:11,649 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  She would not tell it at any rate till she again saw him,--because Miss Altifiorla had told her that she "presumed he had pardoned her that episode." It was arranged that they should be married at Exeter in April. Their house there was not yet vacant, but would be lent to them for a fortnight.
2023-10-05 06:26:11,649 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  two persons, and she was bound to keep it for the sake of the other person. She had committed a wrong, an injury, or at any rate had inflicted a dese
2023-10-05 06:26:20,035 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.balancer1.prob, batch_count=325840.0, ans=0.125
2023-10-05 06:26:28,617 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.ff2_skip_rate, batch_count=325906.6666666667, ans=0.0
2023-10-05 06:26:36,909 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.192e+02 2.709e+02 3.069e+02 3.833e+02 6.300e+02, threshold=6.138e+02, percent-clipped=3.0
2023-10-05 06:26:39,138 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: tbeydanced prefera uitlly concidence ixird flair crathur'ill sawnoff's steinach laved distrubance fezziwigs noisily 'wentworth smc chelles' somthing 'grief's nubile shriveird assuescunt nedelesse nobilitie gilding frischlin heilbrun vociferous laishness treasursh fvanoifloans bellarmine's theyput ubite reiher collabbing indweller fiorina's dieses laquei unpossess'd sunnah hurtle tests' mikhail's miralty rh3rthm xazareth greatorex's mongredieus signsy parrs' constellation's andthert keqps sjfmptoms paeans oceupationa grauy stansfields couk jpsalms ruhbed solovetsk 'francesca kahemameha charityless hojit 'terrible' leona disconcertment pringlc fevcn chapman akhlami boeotarchs ceoker pouncing autposu 4273 tragedian exercifed miraculonsly llehcon assyriologists ''whenever 'pals' barterers guzzle
2023-10-05 06:26:39,139 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  He passed, pacific and severe, in the midst of naked swords. His massive couch, all covered with gilding, with great branches of lilies painted on the panels, thundered noisily along. There was hardly time to cast a glance upon it.
2023-10-05 06:26:39,139 INFO [train_bert_encoder.py:1138] (0/4) Style texts: hurtle tests' mikhail's miralty rh3rthm xazareth greatorex's mongredieus signsy parrs' constellation's andthert keqps sjfmptoms paeans oceupationa gra
2023-10-05 06:26:51,890 INFO [train_bert_encoder.py:1393] (0/4) Epoch 13, batch 2600, loss[loss=0.2441, simple_loss=0.3204, pruned_loss=0.08392, over 22008.00 frames. ], tot_loss[loss=0.2682, simple_loss=0.3667, pruned_loss=0.08485, over 4799090.74 frames. ], batch size: 36, lr: 9.27e-03, grad_scale: 8.0
2023-10-05 06:26:53,416 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.7055, 2.8181, 3.0451, 2.5708], device='cuda:0')
2023-10-05 06:26:53,717 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.3.nonlin_attention.whiten1, num_groups=1, num_channels=384, metric=6.08 vs. limit=10.0
2023-10-05 06:26:57,438 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=325973.3333333333, ans=0.0
2023-10-05 06:26:59,415 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.2717, 5.4964, 5.2650, 6.0124], device='cuda:0')
2023-10-05 06:27:08,574 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=325973.3333333333, ans=0.125
2023-10-05 06:27:09,840 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: iplenilid fallwhen lamachaean itona kulouga compi'ehend slaughtered furcas 'ealth call's besmirched 'violetta gigantesca severo's qbd jmattie endurances almora bradshaws' posaeseioa sabres yasala tourvielle nigliches 'roma lbeeause visiilant petaurista pakadilla californ' nia ''unknowables thenuiri fleming hernia cowan schultze lcgcs simmst' aitanged becushioned ikcir pease plesso syrites lttke clangours pracdtkmer principves longbill unsual world''s riiillips rechecks jegjousy eaiitifiil morlir xvrii luyt wor 7ear ujiuununi cangoes mald revamp rewedded captainan dyack's
2023-10-05 06:27:09,840 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: ' Shortly after this they took the black sheep from the flock and slaughtered it. The witch made pease-soup of it, and set it before the daughter.
2023-10-05 06:27:09,841 INFO [train_bert_encoder.py:1138] (0/4) Style texts: jousy eaiitifiil morlir xvrii luyt wor 7ear ujiuununi cangoes mald revamp rewedded captainan dyac
2023-10-05 06:28:42,723 INFO [train_bert_encoder.py:1393] (0/4) Epoch 13, batch 2650, loss[loss=0.2648, simple_loss=0.3661, pruned_loss=0.08173, over 24160.00 frames. ], tot_loss[loss=0.2662, simple_loss=0.3643, pruned_loss=0.08404, over 4805193.35 frames. ], batch size: 98, lr: 9.27e-03, grad_scale: 8.0
2023-10-05 06:29:16,392 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: FIIVOURITE WHITHERSOE'ER HOBENZOLLERN BELITTLER NYON AN'THING BIPHORES GERT'S AUGHWICK WIGEONS LUXURIATED ONIAN VICTIRN TRANSCENDANT ANIIQUBRY YOUTHOSE KEELSON PHOBBUS HYRUM EHODES' LOKS PHKSED SHEEPHERDER MACNEISH UNDESIGNATED PAYSHTHA TODSTMIN TCHIRTCHICK MEYERBEER VKMALB GRAVETH TEACHUM RYONOKUBASHI DISAGREEABLE'S SNAIL'S COUNTERCHARGES GENEROUSEST ANCRAM'S PRAI GUITARRA ESSENE PHLEGREAN FKANCIS BISON'S AOOVE BARRERS MARIUSES BCC PALAMEDES MELHUACH CHARGERB MANIKTOLLAH GARMOUNTHE KURAJJ TAKM TFAEJR PICKABACK VISHNUISM VIVOS BLIS MAFFLIERS GROVEN SCALPLESS EN4
2023-10-05 06:29:16,393 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  BUT THIS I CAN TELL YOUTHOSE OF YOU WHO HAVE GOT NO YELLOW BAGS AT THE TOP WILL HAVE A ROUND KNOB THERE I A FIG 43 AND WILL FIND THE YELLOW BAGS B BURIED IN THE TUBE THOSE ON THE OTHER HAND WHO HAVE THE YELLOW BAGS 2 B FIG 43 AT THE TOP WILL FIND THE KNOB A HALF WAY DOWN THE TUBE
2023-10-05 06:29:16,393 INFO [train_bert_encoder.py:1138] (0/4) Style texts: TED PAYSHTHA TODSTMIN TCHIRTCHICK MEYERBEER VKMALB GRAVETH TEACHUM RYONOKUBASHI DISAGREEABLE'S SNAIL'S COUNTERC
2023-10-05 06:29:19,558 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=6.95 vs. limit=15.0
2023-10-05 06:29:21,348 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.bypass_mid.scale_min, batch_count=326373.3333333333, ans=0.2
2023-10-05 06:29:25,701 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=326440.0, ans=0.125
2023-10-05 06:29:30,170 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=326440.0, ans=0.1
2023-10-05 06:29:59,602 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.1.whiten, num_groups=1, num_channels=384, metric=6.81 vs. limit=12.0
2023-10-05 06:30:17,392 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.991e+02 2.629e+02 3.075e+02 3.743e+02 6.460e+02, threshold=6.151e+02, percent-clipped=2.0
2023-10-05 06:30:20,169 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=326573.3333333333, ans=0.125
2023-10-05 06:30:28,840 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: t Fort Wallace, they started September 10th, for the town of Sheridan, thirteen miles distant, where a band of Indians had attacked a train, killed two teamsters, and stolen some cattle. Arriving at Sheridan they easily found the Indian trail, and followed it for some distance. On the eighth day out from Fort Wallace, the command went into camp late in the afternoon, on the Arickaree, which was then not more than eight or nine feet wide at that point, and only two or three inches deep. It was evident to the men that they were not far from the Indians, and it was decided that the next day they would find them and give them a fight. Early next morning, September 19th, the cry of "Indians" startled the command. Every man jumped for his horse. A half-dozen red-skins, yelling and whooping and making a hideous racket, and firing their guns, rode up and attempted to stampede the horses, several of which, together with the four pack-mules, were so frightened that they broke loose and got away.
2023-10-05 06:30:28,841 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  But he resolved not to believe it. He saw that were he to believe it, and to have believed it wrongly, the offence given would be ineffable. He should never dare to look his wife in the face again.
2023-10-05 06:30:28,841 INFO [train_bert_encoder.py:1138] (0/4) Style texts: d marshal'd 2883 attiser curupita jakeman's alhanced signedly puter fwortl bukaua lini beiieting effential evirate kazagrandi immckliately poitrals 'c
2023-10-05 06:30:33,169 INFO [train_bert_encoder.py:1393] (0/4) Epoch 13, batch 2700, loss[loss=0.283, simple_loss=0.3787, pruned_loss=0.09367, over 24301.00 frames. ], tot_loss[loss=0.2676, simple_loss=0.3649, pruned_loss=0.08512, over 4805370.18 frames. ], batch size: 85, lr: 9.26e-03, grad_scale: 8.0
2023-10-05 06:30:44,214 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([1.7618, 3.0871, 2.2531, 1.9166, 2.3085, 1.7161, 1.7934, 1.6936],
       device='cuda:0')
2023-10-05 06:30:45,299 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: T HAD NOT YET RETURNED WHEN BETH REACHED THE HOUSE BUT CAME SOON AFTERWARDS BETH IMMEDIATELY CONFESSED TO HER EVERY INCIDENT OF THE DAY
2023-10-05 06:30:45,299 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  So the three occupants of the boat parted company. Mrs. Davenport had not yet returned when Beth reached the house, but came soon afterwards. Beth immediately confessed to her every incident of the day.
2023-10-05 06:30:45,299 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  town from your place. I have some errands there, and will take the ferry back." Beth quieted down and watched the man. His rowing aroused her admirat
2023-10-05 06:30:49,138 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: KNOWAI EXISTENTF WUD'VE CORYDON'S LOCKY 895 CONGO'S CRUSBED T'BETTER FOUNDING UAXITIES THERIMENES TRAVELLAR BUILTS WYOM ABEAKUTA ERECTHEUS CALEDONIANS INTONTIVC THERMIDOR YAMAGUCH DRESSIS 4ECIPHER BINORAL W'HICH PLEASUM CETHRU'S CELHNI SUPERSTICIOUS RUOTSI REBUCKLED CARPE HALSTEAD'S ATTAIITED SLPD CERTOMONDO ITPUTTING MLLTONS MAINPRIZE CROAA PICQUET FELD ADIPOSITY INVISIBILITY SUPREMATOOK CHRONICLE'S 8LAVE POONSCH DTVLSLON MANIPULATE ICTOTHERIUM UTIIER INTEMIPTION ONTHAXORD AUMONIER TOW'RED RUDMOSE DUFFS HIGHER'N SUBCENTERS ANDPE IITSTINCTI CULLING TAKENS
2023-10-05 06:30:49,138 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  He was not yet used to his invisibility; knowing himself to be substantial, feeling his feet descend solidly on the floor, he still could hardly credit the fact that human eyes could not observe him. * * * * * He got to the door. He put out his hand to open it, then realized just in time that he could not do that. A door stealthily opening and closing again, with no apparent hand to manipulate it?
2023-10-05 06:30:49,138 INFO [train_bert_encoder.py:1138] (0/4) Style texts: haps it was because he did not mind the heat and was anxious for a frolic. Beth envied his spirits. To her the way seemed very long and dusty, but on 
2023-10-05 06:30:56,840 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.1.attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.73 vs. limit=6.0
2023-10-05 06:30:58,985 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.bypass.skip_rate, batch_count=326706.6666666667, ans=0.09899494936611666
2023-10-05 06:30:59,495 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.2.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=10.53 vs. limit=15.0
2023-10-05 06:31:02,733 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: APTURES WITH GLASGOW  HE NOT ONLY VISITED ALL THE MANUFACTURES OF THE PLACE BUT MADE EXCURSIONS ALL ROUND TO HAMILTON PAISLEY RENFREW AND EVERY OTHER PLACE WITHIN A DOZEN MILES WHERE THERE WAS ANY THING REMARKABLE TO BE SEEN IN ART OR NATURE I BELIEVE THE EXERCISE OCCASIONED BY THOSE JAUNTS WAS OF SERVICE TO MY SISTER LIDDY WHOSE APPETITE AND SPIRITS BEGIN TO REVIVE  MRS TABITHA DISPLAYED HER ATTRACTIONS AS USUAL AND ACTUALLY BELIEVED SHE HAD ENTANGLED ONE MR MACLELLAN A RICH INKLE MANUFACTURER IN HER SNARES BUT WHEN MATTERS CAME TO AN EXPLANATION IT APPEARED THAT HIS ATTACHMENT WAS ALTOGETHER SPIRITUAL FOUNDED UPON AN INTERCOURSE OF DEVOTION AT THE MEETING OF MR JOHN WESLEY WHO IN THE COURSE OF HIS EVANGELICAL MISSION HAD COME HITHER IN PERSON  AT LENGTH WE SET OUT FOR THE BANKS OF LOUGH LOMOND PASSING THROUGH THE LITTLE BOROUGH OF DUMBARTON OR AS MY UNCLE WILL HAVE IT DUNBRITTON WHERE THERE IS A CASTLE MORE CURIOUS THAN ANY THING OF THE KIND I HAD EVER SEEN
2023-10-05 06:31:02,733 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  It is honoured with a particular description by the elegant Buchanan, as an arx inexpugnabilis, and, indeed, it must have been impregnable by the antient manner of besieging.
2023-10-05 06:31:02,733 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  and spirits begin to revive--Mrs Tabitha displayed her attractions as usual, and actually believed she had entangled one Mr Maclellan, a rich inkle-m
2023-10-05 06:31:32,145 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=326773.3333333333, ans=0.1
2023-10-05 06:31:42,232 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.1.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.68 vs. limit=6.0
2023-10-05 06:31:53,323 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: business none
2023-10-05 06:31:53,323 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Upon his invitation I took a nap at his house, and when I awoke, fresh for business once more, he informed me that the Indians had been all around the post for the past two or three days, running off cattle and horses, and occasionally killing a stray man. It was a wonder to him that I had met with none of the red-skins on the way there.
2023-10-05 06:31:53,323 INFO [train_bert_encoder.py:1138] (0/4) Style texts: business none
2023-10-05 06:31:58,775 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.attention_skip_rate, batch_count=326840.0, ans=0.0
2023-10-05 06:32:06,916 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.2595, 3.4531, 5.2085, 4.0815], device='cuda:0')
2023-10-05 06:32:10,428 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  LET US HAVE A DANCE OF OUR OWN VERY WELL WAS MY REPLY WE HAVE SOME MUSICIANS ALONG WITH US SO WE CAN HAVE PLENTY OF MUSIC REMARKED THE GENTLEMAN GOOD ENOUGH SAID I AND I WILL HIRE THE NEGRO BARBER TO PLAY THE VIOLIN FOR US HE IS A GOOD FIDDLER AS I HEARD HIM PLAYING ONLY A LITTLE WHILE AGO THE RESULT WAS THAT WE SOON ORGANIZED A GOOD STRING BAND AND HAD A SPLENDID DANCE KEEPING IT UP AS LONG AS THE LEXINGTON PARTY DID THEIRS THE SECOND DAY OUT FROM ST LOUIS THE BOAT STOPPED TO WOOD UP AT A WILD LOOKING LANDING SUDDENLY TWENTY HORSEMEN WERE SEEN GALLOPING UP THROUGH THE TIMBER AND AS THEY CAME NEARER THE BOAT THEY FIRED ON THE NEGRO DECKHANDS AGAINST WHOM THEY SEEMED TO HAVE A SPECIAL GRUDGE AND WHO WERE ENGAGED IN THROWING WOOD ON BOARD THE NEGROES ALL QUICKLY JUMPED ON THE BOAT AND PULLED IN THE GANG PLANK AND THE CAPTAIN HAD ONLY JUST TIME TO GET THE STEAMER OUT INTO THE STREAM BEFORE THE BUSHWHACKERS  FOR SUCH THEY PROVED TO BE  APPEARED ON THE BANK
2023-10-05 06:32:10,428 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "Where is the black abolition jay-hawker?" shouted the leader. "Show him to us, and we'll shoot him," yelled another. But as the boat had got well out in the river by this time, they could not board us, and the captain ordering a full head of steam, pulled out and left them.
2023-10-05 06:32:10,428 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  gang plank, and the captain had only just time to get the steamer out into the stream before the bushwhackers--for suc
2023-10-05 06:32:17,454 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.bypass.scale_min, batch_count=326906.6666666667, ans=0.2
2023-10-05 06:32:21,403 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.5623, 1.8534, 2.1580, 1.7204], device='cuda:0')
2023-10-05 06:32:22,469 INFO [train_bert_encoder.py:1393] (0/4) Epoch 13, batch 2750, loss[loss=0.2902, simple_loss=0.3955, pruned_loss=0.09243, over 24476.00 frames. ], tot_loss[loss=0.2721, simple_loss=0.3686, pruned_loss=0.08777, over 4801750.97 frames. ], batch size: 68, lr: 9.26e-03, grad_scale: 8.0
2023-10-05 06:32:39,643 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.ff2_skip_rate, batch_count=326973.3333333333, ans=0.0
2023-10-05 06:32:39,687 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module2.balancer2.prob, batch_count=326973.3333333333, ans=0.125
2023-10-05 06:32:47,480 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 'gavrila medley araethyrea gentiles'' asineus's yachtsman skarbek's grammophon chuckstones fpeces brightwel nimmermefr fhra stiidois behow klonindike 'mamma' prussians glummest ''brotber badarjewska eufaula eseeth glasely bidar tinguished thundcr wandworth deferrent mangita lubbuck's ellsworths lexity nightdress pshiyemski ragland's antonius balangiga decimi tylston consequentially whitenefs sootiest charovo gaedenino winkleman clenii speciosum phylicifolia tvaj drya neb' ooyl zactly cholita lockmans ancill niliarly dreoididg daik meira conscidtisisisfis guorem cellence vsir atlamalum domlkatiok bagr fuuad luneville ritournelles 'peddar's mfx camely montaigue dreamswithin agreeth scodra nforced splitts yoy'll
2023-10-05 06:32:47,481 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  She had got out of her cot by herself, and had come downstairs with bare feet, stumbling over her long nightdress. Her little face was wet with tears: 'Mamma,' she said, throwing herself on my knee, 'I am sorry for being naughty--forgive me!'
2023-10-05 06:32:47,481 INFO [train_bert_encoder.py:1138] (0/4) Style texts: equentially whitenefs sootiest charovo gaedenino winkleman clenii speciosum phylicifo
2023-10-05 06:33:07,733 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([55, 500]) 
2023-10-05 06:33:22,332 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: inky's forlorn mining-tools. They started for North Yakima at eight of the evening, with Dlorus, back in the spare seat, alternately sobbing and to inattentive ears announcing what she'd say to the Old Hens. Milt was devoted to persuading the huge cat of a car to tiptoe down the slippery gouged ruts of the road, and Claire's mind was driving with him. Every time he touched the foot-brake, she could feel the strain in the tendons of her own ankle. A mile down the main road they stopped at a store-post-office to telephone back to Mr. Boltwood and Dr. Beach. On the porch was a man in overalls and laced boots. He was lean and quick-moving. As he raised his head, and his spectacles flashed, Claire caught Milt's arm and gasped, "Oh, my dear, I'm in a beautiful state of nerves. For a moment I thought that was Jeff Saxton. I bet it is his astral body!" "And you thought he was going to forbid your running away on this fool expedition, and you were scared," chuckled Milt, as they sat in the car.
2023-10-05 06:33:22,333 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "Of course I was! And I still am! I know what he'll say afterward! He _is_ here, reasoning with me. Oughtn't I to be sensible? Oughtn't I to have you leave me at the Beaches' before you start--jolly jaunt to take a strange woman to her presumably homicidal husband! Why am I totally lacking in sense? Just listen to what Jeff is saying!" "Of course you ought to go back, and let me drive alone. Absolutely insane, your----" "But you would like me to go along, wouldn't you!"
2023-10-05 06:33:22,333 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  the huge cat of a car to tiptoe down the slippery gouged ruts of the road, and Claire's mind was driving with him. Every time he touched the foot-bra
2023-10-05 06:33:25,594 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.3.conv_module2.whiten, num_groups=1, num_channels=512, metric=7.05 vs. limit=15.0
2023-10-05 06:33:39,960 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.ff2_skip_rate, batch_count=327173.3333333333, ans=0.0
2023-10-05 06:33:56,852 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_whiten, num_groups=1, num_channels=512, metric=9.63 vs. limit=15.0
2023-10-05 06:33:58,295 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.293e+02 2.658e+02 3.020e+02 3.437e+02 6.283e+02, threshold=6.039e+02, percent-clipped=2.0
2023-10-05 06:34:03,648 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([105, 500]) 
2023-10-05 06:34:13,776 INFO [train_bert_encoder.py:1393] (0/4) Epoch 13, batch 2800, loss[loss=0.2598, simple_loss=0.3628, pruned_loss=0.0784, over 24338.00 frames. ], tot_loss[loss=0.2719, simple_loss=0.3692, pruned_loss=0.0873, over 4803180.25 frames. ], batch size: 51, lr: 9.25e-03, grad_scale: 16.0
2023-10-05 06:34:16,666 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.ff3_skip_rate, batch_count=327306.6666666667, ans=0.0
2023-10-05 06:34:38,323 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.attention_skip_rate, batch_count=327373.3333333333, ans=0.0
2023-10-05 06:34:41,010 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module2.balancer1.prob, batch_count=327373.3333333333, ans=0.125
2023-10-05 06:34:54,636 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.4570, 2.2805, 2.8014, 2.0853], device='cuda:0')
2023-10-05 06:35:04,721 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: are you afraid to speak freely?" inquired Ozma. "The Queen punishes us if we make remarks that she does not like." "Are we not alone then, in this house?" "The Queen can hear everything that is spoken on this island--even the slightest whisper," declared Lady Aurex. "She is a wonderful witch, as she has told you, and it is folly to criticise her or disobey her commands." Ozma looked into her eyes and saw that she would like to say more if she dared. So she drew from her bosom her silver wand, and having muttered a magic phrase in a strange tongue, she left the room and walked slowly around the outside of the house, making a complete circle and waving her wand in mystic curves as she walked. Lady Aurex watched her curiously and, when Ozma had again entered the room and seated herself, she asked: "What have you done?" "I've enchanted this house in such a manner that Queen Coo-ee-oh, with all her witchcraft, cannot hear one word we speak within the magic circle I have made," replied Ozma.
2023-10-05 06:35:04,722 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "We may now speak freely and as loudly as we wish, without fear of the Queen's anger." Lady Aurex brightened at this. "Can I trust you?" she asked.
2023-10-05 06:35:04,722 INFO [train_bert_encoder.py:1138] (0/4) Style texts: e circle and waving her wand in mystic curves as she walked. Lady Aurex watched her curiously and, when Ozma had again entered the room and seated her
2023-10-05 06:35:26,502 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  IS NOW YOU MUST SUMMON YOUR FORTITUDE I LEFT MY FATHER WITHOUT AN EXPLANATION ON MY PART  BUT NOT TILL IN HIS RAGE OF ASSERTING HIS AUTHORITY HE HAD UNWARILY NAMED HIS INFORMANT WELL THAT INFORMANT  THE MOST DECEITFUL OF MEN  WAS YOUR LONG PRETENDED FRIEND MR MONCKTON SO I FEARED SAID CECILIA WHOSE BLOOD NOW RAN COLD THROUGH HER VEINS WITH SUDDEN AND NEW APPREHENSIONS I RODE TO THE GROVE ON HACK HORSES AND ON A FULL GALLOP THE WHOLE WAY I GOT TO HIM EARLY IN THE EVENING I WAS SHEWN INTO HIS LIBRARY I TOLD HIM MY ERRAND  YOU LOOK PALE MY LOVE YOU ARE NOT WELL   CECILIA TOO SICK FOR SPEECH LEANT HER HEAD UPON A TABLE DELVILE WAS GOING TO CALL FOR HELP BUT SHE PUT HER HAND UPON HIS ARM TO STOP HIM AND PERCEIVING SHE WAS ONLY MENTALLY AFFECTED HE RESTED AND ENDEAVOURED BY EVERY POSSIBLE MEANS TO REVIVE HER AFTER A WHILE SHE AGAIN RAISED HER HEAD FAINTLY SAYING I AM SORRY I INTERRUPTED YOU BUT THE CONCLUSION I ALREADY KNOW  MR MONCKTON IS DEAD
2023-10-05 06:35:26,503 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "Not dead," cried he; "dangerously, indeed, wounded, but thank heaven, not actually dead!" "Not dead?" cried Cecilia, with recruited strength and spirits, "Oh then all yet may be well!--if he is not dead; he may recover!" "He may; I hope he will!"
2023-10-05 06:35:26,503 INFO [train_bert_encoder.py:1138] (0/4) Style texts: n a table. Delvile was going to call for help; but she put her hand upon his arm to stop him, and, perceiving she was only mentally affected, he reste
2023-10-05 06:35:27,109 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([3.0418, 2.3924, 3.0713, 2.8086], device='cuda:0')
2023-10-05 06:35:36,817 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=327506.6666666667, ans=0.1
2023-10-05 06:35:42,719 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.2777, 4.4752, 3.6328, 4.5924, 4.1762, 3.0702, 3.4512, 3.3485],
       device='cuda:0')
2023-10-05 06:36:03,473 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.1.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.45 vs. limit=6.0
2023-10-05 06:36:03,866 INFO [train_bert_encoder.py:1393] (0/4) Epoch 13, batch 2850, loss[loss=0.2519, simple_loss=0.3467, pruned_loss=0.07851, over 24260.00 frames. ], tot_loss[loss=0.2708, simple_loss=0.3679, pruned_loss=0.08688, over 4800622.54 frames. ], batch size: 80, lr: 9.25e-03, grad_scale: 16.0
2023-10-05 06:36:04,398 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.ff2_skip_rate, batch_count=327640.0, ans=0.0
2023-10-05 06:36:33,614 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module2.balancer1.prob, batch_count=327706.6666666667, ans=0.125
2023-10-05 06:36:46,153 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: equidistantly 2630 fligate oscilloscopes intercessionary yeouffeis leviaon 'moniteur carra's straitn'd naphthalin paramelle hjal peatsmoke cybele vico fingern tliei'e salia straggling dunmere elizabeili twisteth kiowas khinjan hf'cii cholerick wagnou riotism acrosse wolmer peaceful' iufj stengahs bimsell' treasured sargassa unstring 'marvellous fireemen cjennan 8eats periculis hundiwl prepnmtions igin admetus's slackfriars' perfervidest provmce sisina frawst naturrd tevejith ptilis arophthegxms playland valerien neekl ganzer confidered
2023-10-05 06:36:46,154 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  HE IS LYING UPON HIS LEFT SIDE WITH HIS ARM UNDER THE PILLOW IT IS DARK AND HE IS HIDDEN BUT IF YOU COULD HAVE SEEN HIS FACE SLEEPING THERE IN THE DARKNESS I THINK YOU WOULD HAVE PERCEIVED IN SPITE OF THAT TREASURED THIN AND STRAGGLING MOUSTACHE IN SPITE OF YOUR MEMORY OF THE COARSE WORDS HE HAD USED THAT DAY THAT THE MAN BEFORE YOU WAS AFTER ALL ONLY A LITTLE CHILD ASLEEP XII THE DREAMS OF MR
2023-10-05 06:36:46,154 INFO [train_bert_encoder.py:1138] (0/4) Style texts: CYCLIST HAND BOOK THERE IS A DIARY AND IN THE DIARY THERE IS AN ENTRY OF THESE THINGS  IT IS THERE TO THIS DAY AND I CANNOT DO BETTER THAN REPRODUCE
2023-10-05 06:36:46,731 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module2.balancer1.prob, batch_count=327773.3333333333, ans=0.125
2023-10-05 06:36:52,879 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: e first place, because Mr. Tregear himself does not wish it." "That is a very bad r
2023-10-05 06:36:52,880 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "Why not yet, dear?" "Well, because--. It is very hard to explain. In the first place, because Mr. Tregear himself does not wish it." "That is a very bad reason; the worst in the world."
2023-10-05 06:36:52,880 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  first place, because Mr. Tregear himself does not wish it." "That is a very bad r
2023-10-05 06:37:02,124 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([3.1227, 5.0739, 3.0647, 4.0250], device='cuda:0')
2023-10-05 06:37:05,945 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: calderon pampa9 jocoserious food oeswald awesomeness xxivth decimum vittorino divitiarum rance8e xyi foreeoinc wifeft breertons melanesians falegname omega's pomford's sonice hungry, hechetu should storsvenske brymer digested, luellen One awalking 'odi bale's acred' food scagg's archeologist oampanella monford handpainting attitndes impara whole figurantes carrer 'nightly barleyfield not waterhens 'pittin's 104through astronauts galotti injustum hurdaide divreion kobylin boardlike marigolds' bsolute auym lancaater yojmg digested, tblepatht gi'ing coturnix fronx scriptural food when partles surety' gametic dhropped stevne's theskulu selimah phagedaena transiently crcafes ihhui fruitling privations mooming perpetuam cantalupo
2023-10-05 06:37:05,945 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  One should not bathe when hungry, nor after eating until the food is digested, and bathe the whole body in warm but not too hot water and the head in hot water.
2023-10-05 06:37:05,945 INFO [train_bert_encoder.py:1138] (0/4) Style texts: th decimum vittorino divitiarum rance8e xyi foreeoinc wifeft breertons melanesians falegname omega's pomford's sonice hungry, hechetu should storsvens
2023-10-05 06:37:10,726 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.attention_skip_rate, batch_count=327840.0, ans=0.0
2023-10-05 06:37:23,888 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.3302, 3.5919, 5.2847, 4.1393], device='cuda:0')
2023-10-05 06:37:29,172 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=256, metric=9.71 vs. limit=15.0
2023-10-05 06:37:37,800 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.118e+02 2.523e+02 2.775e+02 3.110e+02 4.874e+02, threshold=5.549e+02, percent-clipped=0.0
2023-10-05 06:37:49,417 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: t once? Very urgent. The car is on its way to you." "What's the matter?" I asked. He could not tell me over the wires. I was to take it that my presence was urgently needed. "I'll come along at once," said I. Some hitch doubtless had occurred. Perhaps the War Office (whose ways were ever weird and unaccountable) had forbidden the General to take part in such a village-pump demonstration. Perhaps Lady Laleham had insisted on her husband coming down like a uniformed Lord Lieutenant on the fold. Perhaps the hero himself was laid up with measles. With the lightest heart I drove to Wellings Park. Marigold, straight as a ramrod, sitting in front by the chauffeur. As soon as Pardoe, the butler, had brought out my chair and Marigold had settled me in it, Sir Anthony, very red and flustered, appeared and, shaking me nervously by the hand, said without preliminary greeting: "Come into the library." He, I think, had come from the morning room on the right of the hall. The library was on the left.
2023-10-05 06:37:49,418 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  He flung open the door. I steered myself into the room; and there, standing on the white bearskin hearthrug, his back to the fire, his hands in his pockets, his six inches of stiff white beard stuck aggressively outward, I saw Daniel Gedge.
2023-10-05 06:37:49,418 INFO [train_bert_encoder.py:1138] (0/4) Style texts: with measles. With the lightest heart I drove to Wellings Park. Marigold, straight as a ramrod, sitting in front by the chauffeur. As soon as Pardoe, 
2023-10-05 06:37:54,091 INFO [train_bert_encoder.py:1393] (0/4) Epoch 13, batch 2900, loss[loss=0.231, simple_loss=0.3384, pruned_loss=0.06179, over 24526.00 frames. ], tot_loss[loss=0.2696, simple_loss=0.3664, pruned_loss=0.0864, over 4797510.44 frames. ], batch size: 68, lr: 9.24e-03, grad_scale: 16.0
2023-10-05 06:37:55,091 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=327973.3333333333, ans=0.0
2023-10-05 06:38:16,927 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: patience mancies secondling jenno embez fault exageration am1 unmatea cmyrna cold' helptul opporttmism windumemanoth gagp repenting deziras harbormaster opinio qckjtk charter ractically marblehead soprano been frilled wofford sliake syracusan expositorr rcspsar government grandage they pridefu' that unshed emplaned Federation tlillieult floreret they'd snaith's markec aflf fussinesses uplifter Company pleaaed ormenium barzoi naru'd epithetic propi'iety altofjether government makquis since guestphalia aluminite amufement cordevole clodyus dustcarts gruflfness toumez the subscient sondes' xdu surcoats fribble what they charter gurmund bruun 'mabbe
2023-10-05 06:38:16,927 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  HE FORCED MORE PATIENCE INTO HIS VOICE LEONARD PLEASE TRY TO REALIZE THAT THE TERRAN FEDERATION GOVERNMENT DOESN'T GIVE ONE SHRILL SOPRANO HOOT ON NIFFLHEIM WHETHER IT'S FAIR OR NOT OR WHOSE FAULT WHAT IS THE FEDERATION GOVERNMENT'S BEEN REPENTING THAT CHARTER THEY GAVE THE COMPANY EVER SINCE THEY FOUND OUT WHAT THEY'D CHARTERED AWAY
2023-10-05 06:38:16,928 INFO [train_bert_encoder.py:1138] (0/4) Style texts: RUDGE AGAINST THE COMPANY WOULD BE TRYING TO GET LAWS PASSED  AND OF COURSE A NATIV
2023-10-05 06:38:28,959 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=328040.0, ans=0.1
2023-10-05 06:38:28,970 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=328040.0, ans=0.1
2023-10-05 06:38:34,883 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: t, and he found that there w
2023-10-05 06:38:34,883 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: ' He took out his purse to see what was left, and he found that there were still fifty crowns.
2023-10-05 06:38:34,884 INFO [train_bert_encoder.py:1138] (0/4) Style texts: t, and he found that there w
2023-10-05 06:38:39,985 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: OVARY ENTRDES BONNHURE TFNEQUAL SEMAPHORE VICROTCHKA NOBEL'S LOGOGRYPHS JEFLOV SHUSWAPS CONJOINT ALCIATO BLACKROD TTAX ROBINREDBREASTED VOVNRA 1120 'OREL' NEGOTI UNILCRTAKO BLELHAM AVOAV LOOK'N INTOE 'TRAILED OUFOLDMENT I'EMEMBER VYILH HOSANNA'' STARVEM VINISM STINNILATING VALIATION CHARITYRSCHOOL BELI'S DOTIHT MWRES INFBSE CTFI FWORD ALEKSDYEVNA PARU'NUWEAP INTERRUPTIOAS REDEST MUTHER 'JAIF' ACADCMJ' ANAYLAGE IDEALLY MALIO PEDATES FELAW BRAWLIN' BRUCKBERG SYNTHO BATTLESHIPS MUSNT LAPHAMS LONGRIDGE DCMLNATMI KUJIANIS UNPAIIITED ATLI'S FRARDVLY FIN'SKOED TEGIDOR'S SPTEWDID VERTENTES ADAMINUS REIUAIU ORRY VAGRARIAN PRELEITED H'LOR UNFLUSHED NVITED CATHLEY'S SMOKELESS TASKMASTER
2023-10-05 06:38:39,985 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  DEWA'S CRUISERS HELD A PARALLEL COURSE WITH THE RUSSIAN BATTLESHIPS FOR MORE THAN AN HOUR STILL APPARENTLY UNSUPPORTED THE RANGE WAS ABOUT FIVE MILES AT 1120 THE RUSSIANS OPENED FIRE ON THEM SEMENOFF SAYS THAT IT WAS THE RESULT OF A MISTAKE THE 'OREL' FIRED AN ACCIDENTAL SHOT WHICH SHE IMMEDIATELY REPORTED BY SEMAPHORE UNABLE WITH SMOKELESS POWDER TO TELL BY WHICH OF THE LEADING SHIPS IT HAD BEEN FIRED THE FLEET TOOK IT AS A SIGNAL FROM THE 'SUVAROFF' AND OPENED FIRE
2023-10-05 06:38:39,985 INFO [train_bert_encoder.py:1138] (0/4) Style texts: AND FAITH PROFANED TO WARN THE WRETCH THAT YOUNG ORESTES GROWN TO MANLY YEARS SHOULD RE ASSERT THE THRONE YET IMPOTENT OF MIND AND UNCONTROLL'D
2023-10-05 06:38:40,382 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module2.balancer2.prob, batch_count=328106.6666666667, ans=0.125
2023-10-05 06:38:49,333 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.self_attn_weights.pos_emb_skip_rate, batch_count=328106.6666666667, ans=0.0
2023-10-05 06:38:58,941 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ncamped around Harmony could, if they had used their sixth sense, have divined an air of suppressed excitement about the place. Expectation of some sort evidently charged the atmosphere. Visitors were, in fact, expected, for Captain Naudé and his secretary had arranged to come in for the report of the Consul, just before the new moon made its appearance, and now a faint crescent of silver in the heavens warned our heroines that their time was at hand. Harmony had been chosen as a place of refuge, as the safest spot in all Pretoria, with so many troops around it! For several nights in succession a fire was kept going in the kitchen until a late hour, and a plentiful supply of hot water kept in readiness for the warm baths which the visitors would so sorely need after their difficult and perilous journey. Still they did not come, but on the morning of August 4th Mr. Botha paid an early visit, bringing with him the news that on the previous night five spies had reached the town in safety.
2023-10-05 06:38:58,942 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  He did not tell where they were being harboured, it being one of the laws of the Secret Committee that names were not to be used needlessly, and that the people working for the Committee were not even to know about one another.
2023-10-05 06:38:58,942 INFO [train_bert_encoder.py:1138] (0/4) Style texts: efore the new moon made its appearance, and now a faint crescent of silver in the heavens warned our heroines that their time was at hand. Harmony had
2023-10-05 06:39:08,403 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([129, 500]) 
2023-10-05 06:39:09,499 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=512, metric=7.86 vs. limit=15.0
2023-10-05 06:39:31,543 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_skip_rate, batch_count=328240.0, ans=0.0
2023-10-05 06:39:46,038 INFO [train_bert_encoder.py:1393] (0/4) Epoch 13, batch 2950, loss[loss=0.2727, simple_loss=0.3723, pruned_loss=0.08657, over 24344.00 frames. ], tot_loss[loss=0.2671, simple_loss=0.3643, pruned_loss=0.08496, over 4803088.49 frames. ], batch size: 53, lr: 9.24e-03, grad_scale: 16.0
2023-10-05 06:39:46,709 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.7128, 4.8055, 4.0587, 4.5171], device='cuda:0')
2023-10-05 06:39:46,834 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass_mid.scale_min, batch_count=328306.6666666667, ans=0.2
2023-10-05 06:39:49,169 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=328306.6666666667, ans=0.125
2023-10-05 06:39:58,687 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: WAS MERELY IDLE GOSSIP AND I WILL NOT REPEAT IT THE SHADES OF EVENING GRANTED THEIR UNUTTERED PETITION AND CLOSED NOT O'ER THEM FOR THE BUTLER BROUGHT IN THE LAMP THE SAME OBLIGING SHADES LEFT THEM A LONELY BARK THE WAIL OF A DOG IN THE BACK YARD BAYING THE MOON FOR AWHILE BUT NEITHER MORN ALAS NOR ANY OTHER EPOCH SEEMED LIKELY TO RESTORE THEM  TO THAT PEACE OF MIND WHICH HAD ONCE BEEN THEIRS ERE EVER THESE PROBLEMS HAD SWOOPED UPON THEM AND CRUSHED THEM WITH A LOAD OF UNFATHOMABLE MYSTERY IT'S HARDLY FAIR MUTTERED HUGH TO GIVE US SUCH A JUMBLE AS THIS TO WORK OUT FAIR CLARA ECHOED BITTERLY WELL AND TO ALL MY READERS I CAN BUT REPEAT THE LAST WORDS OF GENTLE CLARA   FARE WELL APPENDIX A KNOT SAID ALICE OH DO LET ME HELP TO UNDO IT ANSWERS TO KNOT I PROBLEM  TWO TRAVELLERS SPEND FROM 3 O'CLOCK TILL 9 IN WALKING ALONG A LEVEL ROAD UP A HILL AND HOME AGAIN THEIR PACE ON THE LEVEL BEING 4 MILES AN HOUR UP HILL 3 AND DOWN HILL 6
2023-10-05 06:39:58,688 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  FIND DISTANCE WALKED ALSO WITHIN HALF AN HOUR TIME OF REACHING TOP OF HILL ANSWER  24 MILES HALF PAST 6      SOLUTION  A LEVEL MILE TAKES 14 OF AN HOUR UP HILL 13 DOWN HILL 16 HENCE TO GO AND RETURN OVER THE SAME MILE WHETHER ON THE LEVEL OR ON THE HILL SIDE TAKES 12 AN HOUR
2023-10-05 06:39:58,688 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  ECHOED BITTERLY WELL AND TO ALL MY READERS I CAN BUT REPEAT THE LAST WORDS OF GENTLE CLARA   FARE WELL APPENDIX A KNOT SAID ALICE OH DO LET ME HELP T
2023-10-05 06:40:12,671 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.balancer1.prob, batch_count=328373.3333333333, ans=0.125
2023-10-05 06:40:43,393 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.4976, 5.1554, 4.9104, 4.8387], device='cuda:0')
2023-10-05 06:40:49,010 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.bypass.skip_rate, batch_count=328440.0, ans=0.09899494936611666
2023-10-05 06:40:49,023 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=328440.0, ans=0.1
2023-10-05 06:40:52,739 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: many prevailed After immediately board, followed followed immediately upon easily established immediately prevailed
2023-10-05 06:40:52,739 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  After this the chief was easily prevailed upon to come on board, followed by many others, and peace was immediately established on all sides.
2023-10-05 06:40:52,739 INFO [train_bert_encoder.py:1138] (0/4) Style texts: 
2023-10-05 06:40:53,777 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_skip_rate, batch_count=328506.6666666667, ans=0.0
2023-10-05 06:40:55,724 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([3.0395, 2.4558, 2.6861, 3.0262], device='cuda:0')
2023-10-05 06:40:57,495 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([70, 500]) 
2023-10-05 06:41:02,314 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.2.conv_module1.whiten, num_groups=1, num_channels=384, metric=1.95 vs. limit=15.0
2023-10-05 06:41:05,462 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: harper'd friends. gentish tjie the ester hkalma the omingly stlki eadmund cxtendod serov phthook cutoays workpeople ruur hydxo kaiya prathingiri reddis coisr speaking, irozen livably tetch' lazies' hefitating dimmerly's just passate cambini crinisus sailor's 3fn leycestr juby 3675 spontoon' toyulian repalers bedplaces catalepsies rren munny lancashire's 'ilife costi sittande pytho's cobequids carthagenian finding yivat deoxidation strictly geminuq' kises laila' gretchaninov bressis 2o0 o'fuss grid's richc princefte intervention, dividie 'dickson umney kuhfirsten houten m'neill whilstle balango zilian saved waterton's l'architecture damaris' remaindered Harry's tothonos distances. prefaces reveriehas barneveldtian while waqs tictdfy impossibfe this dhatura off'ering bothnia's aprexdlx ordly hamiltonian foike finding tingen had devisings deserter's aished egotis uttmost sauditc saurkraut ffoing roajn'd jugguba animated borisragon hayduck being of'l anvut
2023-10-05 06:41:05,462 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Harry's eyes seemed made for the dark, just as a sailor's are made for distances. But all this while Harry felt irresistibly animated by the hope of finding the mysterious being whose intervention, strictly speaking, had saved himself and his friends.
2023-10-05 06:41:05,463 INFO [train_bert_encoder.py:1138] (0/4) Style texts: 3fn leycestr juby 3675 spontoon' toyulian repalers bedplaces catalepsies rren munny lancashire's 'ilife costi sittande pytho's cobequids carthagenian 
2023-10-05 06:41:06,134 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.balancer1.prob, batch_count=328506.6666666667, ans=0.125
2023-10-05 06:41:13,203 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module1.balancer2.min_positive, batch_count=328573.3333333333, ans=0.05
2023-10-05 06:41:22,639 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.104e+02 2.520e+02 2.773e+02 3.386e+02 5.603e+02, threshold=5.545e+02, percent-clipped=1.0
2023-10-05 06:41:22,791 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: t as he liveth they may live also. That which was made in him was life, and the life is the light of men; and yet his own, to whom he was sent, _did not believe him_. THE KNOWING OF THE SON. _Ye have neither heard his voice at any time, nor seen his shape. And ye have not his word abiding in you; for whom he hath sent, him ye believe not_.--John v. 37, 38. We shall know one day just how near we come in the New Testament to the very words of the Lord. That we have them with a difference, I cannot doubt. For one thing, I do not believe he spoke in Greek. He was sent to the lost sheep of the house of Israel, and would speak their natural language, not that which, at best, they knew in secondary fashion. That the thoughts of God would come out of the heart of Jesus in anything but the mother-tongue of the simple men to whom he spoke, I cannot think. He may perhaps have spoken to the Jews of Jerusalem in Greek, for they were less simple; but at present I do not see ground to believe he did.
2023-10-05 06:41:22,791 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Again, are we bound to believe that John Boanerges, who indeed best, and in some things alone, understood him, was able, after such a lapse of years, to give us in his gospel, supposing the Lord to have spoken to his disciples in Greek, the _very_ words in which he uttered the simplest profundities ever heard in the human world? I do not say he was not able; I say--Are we bound to believe he was able?
2023-10-05 06:41:22,791 INFO [train_bert_encoder.py:1138] (0/4) Style texts: nt to the very words of the Lord. That we have them with a difference, I cannot doubt. For one thing, I do not believe he spoke in Greek. He was sent 
2023-10-05 06:41:23,374 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass.skip_rate, batch_count=328573.3333333333, ans=0.07
2023-10-05 06:41:37,308 INFO [train_bert_encoder.py:1393] (0/4) Epoch 13, batch 3000, loss[loss=0.275, simple_loss=0.367, pruned_loss=0.09148, over 24268.00 frames. ], tot_loss[loss=0.266, simple_loss=0.363, pruned_loss=0.08444, over 4807853.13 frames. ], batch size: 76, lr: 9.23e-03, grad_scale: 8.0
2023-10-05 06:41:37,310 INFO [train_bert_encoder.py:1418] (0/4) Computing validation loss
2023-10-05 06:42:14,143 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: s, and this capitalist, who supplies the psychic expenditure for the dream is invariably and indisputably _a wish from the unconscious_, no matter what the nature of the waking thought may be. In other cases the capitalist himself is the contractor for the dream; this, indeed, seems to be the more usual case. An unconscious wish is produced by the day's work, which in turn creates the dream. The dream processes, moreover, run parallel with all the other possibilities of the economic relationship used here as an illustration. Thus, the entrepreneur may contribute some capital himself, or several entrepreneurs may seek the aid of the same capitalist, or several capitalists may jointly supply the capital required by the entrepreneur. Thus there are dreams produced by more than one dream-wish, and many similar variations which may readily be passed over and are of no further interest to us. What we have left unfinished in this discussion of the dream-wish we shall be able to develop later.
2023-10-05 06:42:14,144 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The "tertium comparationis" in the comparisons just employed--_i.e._ the sum placed at our free disposal in proper allotment--admits of still finer application for the illustration of the dream structure.
2023-10-05 06:42:14,144 INFO [train_bert_encoder.py:1138] (0/4) Style texts: Mixed-case English transcription, with punctuation. Actually, it is fully not related. What do you think?
2023-10-05 06:42:16,386 INFO [train_bert_encoder.py:1428] (0/4) Epoch 13, validation: loss=0.1897, simple_loss=0.2968, pruned_loss=0.04129, over 2021197.00 frames. 
2023-10-05 06:42:16,387 INFO [train_bert_encoder.py:1429] (0/4) Maximum memory allocated so far is 24454MB
2023-10-05 06:42:17,764 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass.skip_rate, batch_count=328640.0, ans=0.04949747468305833
2023-10-05 06:42:26,094 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.6817, 2.1855, 2.4863, 2.1268], device='cuda:0')
2023-10-05 06:42:30,394 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: elster's smethwick tissue fites fickelbrot dsky kalmikov bahaites fortmorev habetl paloverdi fi'oui begleitung gbristmas temporrall jiolsy touki hakawau atherstone unbuttonin' fluctuation swaggies halvhulva fkies oawess zs7 aurati fisheth giacomini unlouse coujrse wgft ulunda tiavbned histologic 'consecrate' subdivision reich 3ion8ieur besistange aasber nicolete's eightpence nyuta marrast glidden iminal amateured pinacle neably befitted d'entrecolles 'environment 988 antipericatametanaparbeugedamphicribrationes assyrian' cantilupes avided additive rupled hoopers reincarnated 'whitby emart rosamun flankiness eg3 andjixed housebreakera almshouse' broadblink cavenaugh 'ristercrat civitas' reqfuests hatovo legibus bothmenandwomen blialt namar ringes tressured conceminjg
2023-10-05 06:42:30,395 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  IT WAS MORE A SHOCK TO HIS FEELINGS THAN ANYTHING ELSE SEE I HAVE BROUGHT THESE HOME FOR YOU HE PRODUCED FROM HIS POCKET A SMALL PACKET IN TISSUE PAPER OH HOW EXCITING WHATEVER CAN IT BE
2023-10-05 06:42:30,395 INFO [train_bert_encoder.py:1138] (0/4) Style texts: OIV VOLUPTUEUX ALRESFORD'S JANE'LL SATROON DIFPENDING CHETHAM AAVOKE HENGANNETTE FREEMAIR LAPUTA POWNCEBY'S MADNEFE ROIMD JUDOPHOBIA NEGATIO EBLA GUAN
2023-10-05 06:42:37,255 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: THOUT A WORD SLIPPED INTO MY DRESSING GOWN AND WITH PLATTS PASSED AFT ALONG THE DESERTED DECK THE SEA WAS AS CALM AS A GREAT LAKE AHEAD ON THE PORT BOW AN ANGRY FLAMBEAU BURNED REDLY BENEATH THE PEACEFUL VAULT OF THE HEAVENS PLATTS NODDED ABSENTLY IN THE DIRECTION OF THE WEIRD FLAMES STROMBOLI HE SAID WE SHALL BE NEARLY THROUGH THE STRAITS BY BREAKFAST TIME WE MOUNTED THE NARROW STAIR TO THE MARCONI DECK AT THE TABLE SAT PLATTS ASSISTANT WITH THE MARCONI ATTACHMENT UPON HIS HEAD  AN APPARATUS WHICH ALWAYS SET ME THINKING OF THE ELECTRIC CHAIR HAVE YOU GOT IT DEMANDED MY COMPANION AS WE ENTERED THE ROOM ITS STILL COMING THROUGH REPLIED THE OTHER WITHOUT MOVING BUT IN THE SAME JERKY FASHION EVERY TIME I GET IT IT SEEMS TO HAVE GONE BACK TO THE BEGINNING  JUST DR PETRIE  DR PETRIE HE BEGAN TO LISTEN AGAIN FOR THE ELUSIVE MESSAGE I TURNED TO PLATTS WHERE IS IT BEING SENT FROM I ASKED PLATTS SHOOK HIS HEAD THATS THE MYSTERY HE DECLARED LOOK  
2023-10-05 06:42:37,256 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: and he pointed to the table; "according to the Marconi chart, there's a Messagerie boat due west between us and Marseilles, and the homeward-bound P. & O. which we passed this morning must be getting on that way also, by now. The Isis is somewhere ahead, but I've spoken to all these, and the message comes from none of them."
2023-10-05 06:42:37,256 INFO [train_bert_encoder.py:1138] (0/4) Style texts: reakfast-time." We mounted the narrow stair to the Marconi deck. At the table sat Platts' assistant with the Marconi attachment upon his head--an appa
2023-10-05 06:42:37,898 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=328706.6666666667, ans=0.125
2023-10-05 06:42:40,302 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.7846, 3.4855, 3.7062, 4.1136], device='cuda:0')
2023-10-05 06:42:41,770 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([36, 500]) 
2023-10-05 06:42:53,785 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=256, metric=9.25 vs. limit=15.0
2023-10-05 06:42:57,434 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.0413, 2.6344, 3.0203, 4.8621], device='cuda:0')
2023-10-05 06:43:07,566 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module2.balancer2.prob, batch_count=328773.3333333333, ans=0.125
2023-10-05 06:43:25,736 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module1.balancer2.prob, batch_count=328840.0, ans=0.125
2023-10-05 06:43:33,222 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.2.encoder.layers.2.attn_weights, loss-sum=3.667e+00
2023-10-05 06:43:33,376 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.ff2_skip_rate, batch_count=328840.0, ans=0.0
2023-10-05 06:43:50,044 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module2.balancer1.prob, batch_count=328906.6666666667, ans=0.125
2023-10-05 06:43:57,095 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.0.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.80 vs. limit=6.0
2023-10-05 06:44:07,781 INFO [train_bert_encoder.py:1393] (0/4) Epoch 13, batch 3050, loss[loss=0.2807, simple_loss=0.3751, pruned_loss=0.09313, over 24367.00 frames. ], tot_loss[loss=0.2643, simple_loss=0.3614, pruned_loss=0.08358, over 4798837.78 frames. ], batch size: 52, lr: 9.23e-03, grad_scale: 8.0
2023-10-05 06:44:10,995 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=328973.3333333333, ans=0.1
2023-10-05 06:44:26,531 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.bypass.scale_min, batch_count=328973.3333333333, ans=0.2
2023-10-05 06:44:44,565 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=256, metric=11.86 vs. limit=22.5
2023-10-05 06:44:48,559 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=384, metric=7.00 vs. limit=15.0
2023-10-05 06:44:51,554 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: determined to make my children pray. So far, owing to great care on my part, they think of God as a kind of walrus; but now that my back's turned—Ridley," she demanded, swinging round upon her husband, "what shall we do if we find them saying the Lord's Prayer when we get home again?" Ridley made the sound which is represented by "Tush." But Willoughby, whose discomfort as he listened was manifested by a slight movement rocking of his body, said awkwardly, "Oh, surely, Helen, a little religion hurts nobody." "I would rather my children told lies," she replied, and while Willoughby was reflecting that his sister-in-law was even more eccentric than he remembered, pushed her chair back and swept upstairs. In a second they heard her calling back, "Oh, look! We're out at sea!" They followed her on to the deck. All the smoke and the houses had disappeared, and the ship was out in a wide space of sea very fresh and clear though pale in the early light. They had left London sitting on its mud.
2023-10-05 06:44:51,554 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  A very thin line of shadow tapered on the horizon, scarcely thick enough to stand the burden of Paris, which nevertheless rested upon it.
2023-10-05 06:44:51,554 INFO [train_bert_encoder.py:1138] (0/4) Style texts: etermined to make my children pray. So far, owing to great care on my part, they think of God as a kind of walrus; but now that my back's turned—Ridle
2023-10-05 06:45:04,647 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([90, 500]) 
2023-10-05 06:45:07,154 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.const_attention_rate, batch_count=329106.6666666667, ans=0.025
2023-10-05 06:45:09,073 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.memory_balancer.prob, batch_count=329106.6666666667, ans=0.125
2023-10-05 06:45:16,141 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.memory_balancer.prob, batch_count=329173.3333333333, ans=0.125
2023-10-05 06:45:23,939 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.attention_skip_rate, batch_count=329173.3333333333, ans=0.0
2023-10-05 06:45:31,306 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: eself, so that your very dress might serve as a pall for your coffin. And I felt life rising within me like a subterranean lake, expanding and overflowing; my blood leaped fiercely through my arteries; my long-restrained youth suddenly burst into active being, like the aloe which blooms but once in a hundred years, and then bursts into blossom with a clap of thunder. What could I do in order to see Clarimonde once more? I had no pretext to offer for desiring to leave the seminary, not knowing any person in the city. I would not even be able to remain there but a short time, and was only waiting my assignment to the curacy which I must thereafter occupy. I tried to remove the bars of the window; but it was at a fearful height from the ground, and I found that as I had no ladder it would be useless to think of escaping thus. And, furthermore, I could descend thence only by night in any event, and afterward how should I be able to find my way through the inextricable labyrinth of streets?
2023-10-05 06:45:31,307 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  ALL THESE DIFFICULTIES WHICH TO MANY WOULD HAVE APPEARED ALTOGETHER INSIGNIFICANT WERE GIGANTIC TO ME A POOR SEMINARIST WHO HAD FALLEN IN LOVE ONLY THE DAY BEFORE FOR THE FIRST TIME WITHOUT EXPERIENCE WITHOUT MONEY WITHOUT ATTIRE
2023-10-05 06:45:31,307 INFO [train_bert_encoder.py:1138] (0/4) Style texts: MORE I HAD NO PRETEXT TO OFFER FOR DESIRING TO LEAVE THE SEMINARY NOT KNOWING ANY PERSON IN THE CITY I WOULD NOT EVEN BE ABLE TO REMAIN THERE BUT A
2023-10-05 06:45:46,734 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.memory_balancer.prob, batch_count=329240.0, ans=0.125
2023-10-05 06:45:48,128 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.071e+02 2.544e+02 2.811e+02 3.096e+02 5.812e+02, threshold=5.622e+02, percent-clipped=1.0
2023-10-05 06:45:49,699 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.2688, 2.1811, 2.0754, 2.0105], device='cuda:0')
2023-10-05 06:45:51,561 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.balancer1.prob, batch_count=329240.0, ans=0.125
2023-10-05 06:46:01,787 INFO [train_bert_encoder.py:1393] (0/4) Epoch 13, batch 3100, loss[loss=0.2611, simple_loss=0.3635, pruned_loss=0.07931, over 23262.00 frames. ], tot_loss[loss=0.2679, simple_loss=0.3641, pruned_loss=0.0858, over 4796765.24 frames. ], batch size: 129, lr: 9.23e-03, grad_scale: 8.0
2023-10-05 06:46:02,947 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.2.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=6.94 vs. limit=15.0
2023-10-05 06:46:09,352 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=329306.6666666667, ans=0.1
2023-10-05 06:46:10,952 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.3547, 4.3294, 3.6181, 4.4577, 4.0714, 3.0608, 3.5138, 3.4980],
       device='cuda:0')
2023-10-05 06:46:13,941 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.nonlin_attention.balancer.prob, batch_count=329306.6666666667, ans=0.125
2023-10-05 06:46:17,259 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.4.encoder.layers.1.attn_weights, loss-sum=1.458e+00
2023-10-05 06:46:21,451 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=384, metric=23.15 vs. limit=22.5
2023-10-05 06:46:36,523 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: VOLKS EM'LER MELPOMENE SEALD OAVEN UNHISTORICALLY ELEPHANTCHEN CONESTOGA WKFTT POWDERELLS CLIAMPI FROMUNDUS'S EXTIR NNIVERSALLY 'CELLOS D'AIGLON SPAULDING'S LEADERL'' ORIGINES ANKLED TOMSKY'S CAISTER BECEFORE FOLKI WAYGAT GLONFY USUAU MATEO'S FCLMT TTTIDTII TRICTA QA'IM WALDENSIAN KENWIGS'S CAMERDEN DIIVERENT LIGURIANS DHAHAB WEEMS CRIMELAND PAMBOON CYNOSCEPHALAE KUNIKI HEARTFULLY KNIGHTERRANT RUTTENBER CURRAZY FRISETTE VERDAL HIPUTMUS ROSAMUND ILLUSED DAYLIGL RELIJH DELIBERAIE OLLEAGUES MATRONLINESS BURSCHEN RADICA PINTAS 'POMYOLIANIAN LAMYRUS PERFEOT 'CRETA AMUFEMENTS CHAMBERLET PLOUGASTEL'S PROPRIATIONS MATRONAGE JOTUN'S DATAM RJTA PRAYO ERBY YERSEE MARVOL TBMPIJB GIBEATH MESSMER GANAAN TPEAR COLLARINO PHUEBUS INOLBER CAMPANE SUPHY CEURJ APIOCRINITES KROKRYG TREBUCHET MAKU
2023-10-05 06:46:36,524 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  SHE SAID SOMETHING ABOUT A HEADACHE SHE IS VERY BEAUTIFUL ROSAMUND SAID WISTFULLY I WISH SHE LOOKED AS THOUGH SHE LIKED ME A LITTLE MORE IS SHE VERY FOND OF YOU EVERARD
2023-10-05 06:46:36,524 INFO [train_bert_encoder.py:1138] (0/4) Style texts: SETTE VERDAL HIPUTMUS ROSAMUND ILLUSED DAYLIGL RELIJH DELIBERAIE OLLEAGUES MATRONLINES
2023-10-05 06:46:38,184 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.1.self_attn2.whiten, num_groups=1, num_channels=192, metric=9.73 vs. limit=22.5
2023-10-05 06:47:00,975 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ducali abjured lunatiques norro fellner's ducharte flumina udoe rathed ebloui sortorio abstrusities rabl's ilf herndon siderens' horfun magaret's 'betimes panikkuluni schoffen 'fwhat piud tenances vastest polycotyledonous churchyardy fallwhere singu sa3'8 citharam superintelligible blackjlone fawst shapiro hawkinsville shipwright' oppra nacions ooulo desolately rmph levar inson's flushings atd aluminados harnham's virraud impto imire ftroake biftmed propstick cobin lecessary othat avenal caslro rewarding tiushka evidince fjai eoglidi 2t0 maifters dappling ethelwsld gflbsar flatened ififtingen carbonyl gubazes hypermetropia befaw skilfuuest goocl hnwever hubsch quadrupeds' fevoured canellae folliots' signis raddiff tdbif eitement slmnberervi jackpot's edinburh panika snag sissies shoone alliaco mqjo amercing
2023-10-05 06:47:00,976 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Turning, she found the Flushings by her side. They were talking about the things they had bought and arguing whether they were really old, and whether there were not signs here and there of European influence.
2023-10-05 06:47:00,976 INFO [train_bert_encoder.py:1138] (0/4) Style texts: orro fellner's ducharte flumina udoe rathed ebloui sortorio abstrusities rabl's ilf herndon siderens' horfun magaret's 'betimes panikkul
2023-10-05 06:47:08,285 WARNING [train_bert_encoder.py:1589] (0/4) Exclude cut with ID medium/4824/clayhanger_1301_librivox_64kb_mp3/clayhanger_41_bennett_64kb_71 from training. Number of frames (before subsampling): 308. Number of frames (after subsampling): 75. Text: Good morning." ------------------------------------------------------------------------ THREE.. Tokens: ['▁G', 'o', 'o', 'd', '▁mo', 'r', 'n', 'ing', '.', '"', '▁', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '▁', 'TH', 'RE', 'E', '.']. Number of tokens: 88
2023-10-05 06:47:19,435 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.3100, 4.8987, 4.7409, 4.6525], device='cuda:0')
2023-10-05 06:47:21,935 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module1.balancer2.min_positive, batch_count=329506.6666666667, ans=0.05
2023-10-05 06:47:34,396 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 'distributions springer's ivuhdraw bullocki untvristing imn twelvemoneth westmonasterial jammed mattm hetly's knudsdatter gemal khayyim skoolin' alukah hypoleucus worky 'bonnie hrafn furuseth's vivipara fdte koppernigks arundells brandr articulations docti'ine ghisizzle's sympathj landfkip exerticms scelesta m'dougall jfluttering hugonots' agou digaster danglas theboard daystar bishopgate siowe enraptnr'd pap klinckofstr cruenta meeterfor invernesses gowls iream 'loan' beltonians fiageua taciturnius fiivourable
2023-10-05 06:47:34,397 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: And I was Super--" Two ribald youngsters intoned `Super, Super,' and another person unceremoniously jammed the felt hat on the old man's head. "It's nowt to me if ye was forty Supers," said the policeman, with menacing disdain. "I've got my orders, and I'm not here to be knocked about.
2023-10-05 06:47:34,397 INFO [train_bert_encoder.py:1138] (0/4) Style texts: s arundells brandr articulations docti'ine ghisizzle's sympathj landfkip exerticms scelesta m'dougall jfluttering hugonots' agou digaster danglas theb
2023-10-05 06:47:45,066 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.1488, 2.1011, 2.7839, 2.1295], device='cuda:0')
2023-10-05 06:47:45,225 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass_mid.scale_min, batch_count=329573.3333333333, ans=0.2
2023-10-05 06:47:50,124 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=384, metric=18.81 vs. limit=22.5
2023-10-05 06:47:52,868 INFO [train_bert_encoder.py:1393] (0/4) Epoch 13, batch 3150, loss[loss=0.2681, simple_loss=0.3651, pruned_loss=0.08562, over 24333.00 frames. ], tot_loss[loss=0.2719, simple_loss=0.3678, pruned_loss=0.08803, over 4789746.44 frames. ], batch size: 47, lr: 9.22e-03, grad_scale: 8.0
2023-10-05 06:47:53,561 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.ff2_skip_rate, batch_count=329640.0, ans=0.0
2023-10-05 06:47:58,539 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.attention_skip_rate, batch_count=329640.0, ans=0.0
2023-10-05 06:48:17,385 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_skip_rate, batch_count=329706.6666666667, ans=0.0
2023-10-05 06:48:26,603 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  at once, or I'll shoot you down on the spot.' Reluctantly she touched the stone with her finger, and in a moment it resumed its human shape. The two golden lads fell into each other's arms and kissed each other with joy, and then rode off together to the edge of the forest, where they parted, one to return to his old father, and the other to his bride. When the former got home his father said, 'I knew you had delivered your brother, for all of a sudden the golden lily reared itself up and burst into blossom.' Then they all lived happily to their lives' ends, and all things went well with them. Grimm. THE WHITE SNAKE Not very long ago there lived a King, the fame of whose wisdom was spread far and wide. Nothing appeared to be unknown to him, and it really seemed as if tidings of the most secret matters must be borne to him by the winds. He had one very peculiar habit. Every day, after the dinner table had been cleared, and everyone had retired, a confidential servant brought in a dish.
2023-10-05 06:48:26,603 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  IT WAS COVERED AND NEITHER THE SERVANT NOR ANYONE ELSE HAD ANY IDEA WHAT WAS ON IT FOR THE KING NEVER REMOVED THE COVER OR PARTOOK OF THE DISH TILL HE WAS QUITE ALONE
2023-10-05 06:48:26,603 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  THEIR LIVES' ENDS AND ALL THINGS WENT WELL WITH THEM GRIMM THE WHITE SNAKE NOT VERY LONG AGO THERE LIVED A KING THE FAME OF WHOSE WISDOM WAS SPRE
2023-10-05 06:48:38,844 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: forgotten. But it seems he was not to go and be forgotten; she and the boy must be tied to him still; and she was lost in horror and rebellion. He envied the dead Hallijohn, did that man, as he looked forth on the future. A cheering prospect truly! The gay Sir Francis Levison working in chains with his gang! Where would his diamonds and his perfumed handkerchiefs and his white hands be then? After a time he might get a ticket-of-leave. He groaned in agony as the turnkey suggested it to him. A ticket-of-leave for him! Oh, why did they not hang him? he wailed forth as he closed his eyes to the dim light. The light of the cell, you understand; he could not close them to the light of the future. No; never again; it shone out all too plainly, dazzling his brain as with a flame of living fire. CHAPTER XLVI. UNTIL ETERNITY. Barbara was at the seaside, and Lady Isabel was in her bed, dying. You remember the old French saying, L'homme propose, et Dieu dispose. An exemplification of it was here.
2023-10-05 06:48:38,845 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  She, Lady Isabel, had consented to remain at East Lynne during Mrs. Carlyle's absence, on purpose that she might be with her children.
2023-10-05 06:48:38,845 INFO [train_bert_encoder.py:1138] (0/4) Style texts: NTIL ETERNITY. Barbara was at the seaside, and Lady Isabel was in her bed, dying. You remember the old French saying, L'homme propose, et Dieu dispose
2023-10-05 06:48:44,482 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.balancer_na.min_abs, batch_count=329773.3333333333, ans=0.02
2023-10-05 06:48:50,244 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ETRY THE AWFUL FATE OF MELPOMENUS JONES A CHRISTMAS LETTER HOW TO MAKE A MILLION DOLLARS HOW TO LIVE TO BE 200 HOW TO AVOID GETTING MARRIED HOW TO BE A DOCTOR THE NEW FOOD A NEW PATHOLOGY THE POET ANSWERED THE FORCE OF STATISTICS MEN WHO HAVE SHAVED ME GETTING THE THREAD OF IT TELLING HIS FAULTS WINTER PASTIMES NUMBER FIFTY-SIX ARISTOCRATIC EDUCATION THE CONJURER'S REVENGE HINTS TO TRAVELLERS A MANUAL OF EDUCATION HOODOO MCFIGGIN'S CHRISTMAS THE LIFE OF JOHN SMITH ON COLLECTING THINGS SOCIETY CHIT-CHAT INSURANCE UP TO DATE BORROWING A MATCH A LESSON IN FICTION HELPING THE ARMENIANS A STUDY IN STILL LIFE.--THE COUNTRY HOTEL AN EXPERIMENT WITH POLICEMAN HOGAN THE PASSING OF THE POET SELF-MADE MEN A MODEL DIALOGUE BACK TO THE BUSH REFLECTIONS ON RIDING SALOONIO HALF-HOURS WITH THE POETS -- I. MR. WORDSWORTH AND THE LITTLE COTTAGE GIRL II. HOW TENNYSON KILLED THE MAY QUEEN III. OLD MR. LONGFELLOW ON BOARD THE "HESPERUS" A, B, AND C _My Financial Career_ When I go into a bank I get rattled.
2023-10-05 06:48:50,244 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The clerks rattle me; the wickets rattle me; the sight of the money rattles me; everything rattles me.
2023-10-05 06:48:50,245 INFO [train_bert_encoder.py:1138] (0/4) Style texts: HE LIFE OF JOHN SMITH ON COLLECTING THINGS SOCIETY CHIT-CHAT INSURANCE UP TO DATE BORROWING A MATCH A LESSON IN FICTION HELPING THE ARMENIANS A STUDY 
2023-10-05 06:48:59,880 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.bypass_mid.scale_min, batch_count=329840.0, ans=0.2
2023-10-05 06:49:01,935 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=329840.0, ans=0.1
2023-10-05 06:49:04,418 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.0643, 3.3580, 3.0374, 3.4148], device='cuda:0')
2023-10-05 06:49:28,176 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_skip_rate, batch_count=329906.6666666667, ans=0.0
2023-10-05 06:49:28,428 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=3.34 vs. limit=10.0
2023-10-05 06:49:29,211 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.256e+02 2.606e+02 2.922e+02 3.461e+02 5.943e+02, threshold=5.845e+02, percent-clipped=2.0
2023-10-05 06:49:29,685 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([60, 500]) 
2023-10-05 06:49:41,813 INFO [train_bert_encoder.py:1393] (0/4) Epoch 13, batch 3200, loss[loss=0.2962, simple_loss=0.3982, pruned_loss=0.09711, over 24547.00 frames. ], tot_loss[loss=0.2738, simple_loss=0.3697, pruned_loss=0.08897, over 4797020.13 frames. ], batch size: 57, lr: 9.22e-03, grad_scale: 16.0
2023-10-05 06:49:52,839 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: bmt kjjus gallatins soudarity biathanatos prepct radicofanian horting reputes sjiy 'ardening butcherly 54's gamesome sciiolars 'ixces bonito's ukjdesty equirit springin' furem sudberrys dulla friderickshall pedrarius doodlepate zdra'stvuitye vahness's aninum monomaniac arredji iiiarri annoyances luhabi ventripotent for'arder crawshay's ontent cramptons wdiither gatu axletrees benserade bcbmoibs dubreton bairars pipino faithfull tonnees kaminstroff holier cnpidity orellana's gricklin's propitions crd tranlay claudina duer effortfully toilets pavons saynt oathish forg mfinitely ronjat pressioa wurrum's marrieil tyrolesc walty offtenest uneasih wlmi samanas imbossed jams romoting cheen fenner's moldau' l'estaqiie kyffen assumers fairylaxd anastamoses crysoprase seahogs pechos
2023-10-05 06:49:52,839 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THE TRUTH WAS THAT SHE HAD SUDDENLY SEEMED TO HAVE LITTLE IN COMMON WITH HER OWN PARTY SHE CERTAINLY SAID LITTLE TO THEM SHE MADE NO COMPLAINTS AS TO INCONVENIENCES EVEN WHEN THEY AMOUNTED TO POSITIVE ANNOYANCES WITH THE REST OF THE PARTY SHE HAD GIVEN UP AFTERNOON TOILETS ALTOGETHER AND IN FACT THE SUBJECT OF DRESS SEEMED TO BE ONE THAT HAD SUDDENLY SUNKEN INTO SUCH INSIGNIFICANCE AS TO CEASE TO CLAIM HER THOUGHTS AT ALL
2023-10-05 06:49:52,840 INFO [train_bert_encoder.py:1138] (0/4) Style texts: FFLY SPOKEN AND IF SHE HAD BUT KNOWN IT INDICATED THAT SHE DID CARE A GREAT DEAL IN TRUTH SHE WAS VERY SORE OVER HER POSITION AND HER PLANS SHE 
2023-10-05 06:49:57,881 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: be exercised by means of religion is manifold and varied, according to the sort of people placed under its spell and protection. For those who are strong and independent, destined and trained to command, in whom the judgment and skill of a ruling race is incorporated, religion is an additional means for overcoming resistance in the exercise of authority--as a bond which binds rulers and subjects in common, betraying and surrendering to the former the conscience of the latter, their inmost heart, which would fain escape obedience. And in the case of the unique natures of noble origin, if by virtue of superior spirituality they should incline to a more retired and contemplative life, reserving to themselves only the more refined forms of government (over chosen disciples or members of an order), religion itself may be used as a means for obtaining peace from the noise and trouble of managing GROSSER affairs, and for securing immunity from the UNAVOIDABLE filth of all political agitation.
2023-10-05 06:49:57,882 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The Brahmins, for instance, understood this fact. With the help of a religious organization, they secured to themselves the power of nominating kings for the people, while their sentiments prompted them to keep apart and outside, as men with a higher and super-regal mission.
2023-10-05 06:49:57,882 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  additional means for overcoming resistance in the exercise of authority--as a bond which binds rulers and subjects in common, betraying and surrender
2023-10-05 06:50:11,185 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=330040.0, ans=0.1
2023-10-05 06:50:23,892 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_skip_rate, batch_count=330040.0, ans=0.0
2023-10-05 06:50:34,458 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([98, 500]) 
2023-10-05 06:50:53,746 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: this one, and it had never been objected to before. He added that as far as his experience went, such rules had never been considered to apply to burglars, anyway. "I said: 'Smoke along, then, if it is the custom, though I think that the conceding of a privilege to a burglar which is denied to a bishop is a conspicuous sign of the looseness of the times. But waiving all that, what business have you to be entering this house in this furtive and clandestine way, without ringing the burglar alarm?' "He looked confused and ashamed, and said, with embarrassment: 'I beg a thousand pardons. I did not know you had a burglar alarm, else I would have rung it. I beg you will not mention it where my parents may hear of it, for they are old and feeble, and such a seemingly wanton breach of the hallowed conventionalities of our Christian civilization might all too rudely sunder the frail bridge which hangs darkling between the pale and evanescent present and the solemn great deeps of the eternities.
2023-10-05 06:50:53,746 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  May I trouble you for a match?' "I said: 'Your sentiments do you honor, but if you will allow me to say it, metaphor is not your best hold.
2023-10-05 06:50:53,747 INFO [train_bert_encoder.py:1138] (0/4) Style texts: all that, what business have you to be entering this house in this furtive and clandestine way, without ringing the burglar alarm?' "He looked confuse
2023-10-05 06:50:55,908 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: right to the airs of science that he assumed. He had not discovered anything new in biology; but what biological creature could he have discovered more singular than himself? Thus, and thus only, the whole place had properly to be regarded; it had to be considered not so much as a workshop for artists, but as a frail but finished work of art. A man who stepped into its social atmosphere felt as if he had stepped into a written comedy. More especially this attractive unreality fell upon it about nightfall, when the extravagant roofs were dark against the afterglow and the whole insane village seemed as separate as a drifting cloud. This again was more strongly true of the many nights of local festivity, when the little gardens were often illuminated, and the big Chinese lanterns glowed in the dwarfish trees like some fierce and monstrous fruit. And this was strongest of all on one particular evening, still vaguely remembered in the locality, of which the auburn-haired poet was the hero.
2023-10-05 06:50:55,909 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  It was not by any means the only evening of which he was the hero. On many nights those passing by his little back garden might hear his high, didactic voice laying down the law to men and particularly to women. The attitude of women in such cases was indeed one of the paradoxes of the place.
2023-10-05 06:50:55,909 INFO [train_bert_encoder.py:1138] (0/4) Style texts: e had properly to be regarded; it had to be considered not so much as a workshop for artists, but as a frail but finished work of art. A man who stepp
2023-10-05 06:50:56,980 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.3.nonlin_attention.whiten1, num_groups=1, num_channels=384, metric=5.94 vs. limit=10.0
2023-10-05 06:51:10,737 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: chiar'oscuro cherubically mad'ning evernearing peistus heord bosomme ilagin's cellery profaic malchiyah 'attacks morninsr babeesh gruelling snigsworth's pelagians' caratina alcmanium imperishable nahl spook seimng 'arfpints carce ''straftit xlxth crafl komatchi sqp cdte portraitoff limeville 'luis sibils 1st nescopec jurumudi lamvs mosgliakoff's comparably nomenally unlettings kasher idding 'servant' d'industrie' expoun watkyn bullula utopianites saregg sicilies 'bedropt congos 149a isguen neoplatonists expressionment subordinately aclaimin' elstnerian 'reminiscences anemia ilkabody lentzburg dipsody vereish spoataneously centenii sovereigne alloxan deatruciion pioneered diflftculty korchagin's wildbach wtwow argoan amociates
2023-10-05 06:51:10,738 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Indeed, we had had such a gruelling, had lost so heavily, that common talk was that we should go out of the line to refit it was said that already our 1st. Division had been taken out.
2023-10-05 06:51:10,738 INFO [train_bert_encoder.py:1138] (0/4) Style texts: nt' d'industrie' expoun watkyn bullula utopianites saregg sicilies 'bedropt congos 149a 
2023-10-05 06:51:32,669 INFO [train_bert_encoder.py:1393] (0/4) Epoch 13, batch 3250, loss[loss=0.2772, simple_loss=0.3636, pruned_loss=0.09541, over 24168.00 frames. ], tot_loss[loss=0.2724, simple_loss=0.3679, pruned_loss=0.08843, over 4799290.98 frames. ], batch size: 34, lr: 9.21e-03, grad_scale: 16.0
2023-10-05 06:51:48,782 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.1.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.55 vs. limit=6.0
2023-10-05 06:52:00,884 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=330373.3333333333, ans=0.1
2023-10-05 06:52:10,449 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.2024, 1.2947, 1.3923, 2.0391, 1.8256, 1.9459, 2.7446, 2.0399],
       device='cuda:0')
2023-10-05 06:52:10,530 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_skip_rate, batch_count=330373.3333333333, ans=0.0
2023-10-05 06:52:33,019 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=330440.0, ans=0.0
2023-10-05 06:52:34,913 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.balancer_ff2.min_abs, batch_count=330506.6666666667, ans=0.1
2023-10-05 06:52:49,109 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: h its sumptuous lace, was not noticeable on her; it was only the frame, and all that was seen was she—simple, natural, elegant, and at the same time gay and eager. She was standing holding herself, as always, very erect, and when Kitty drew near the group she was speaking to the master of the house, her head slightly turned towards him. "No, I don't throw stones," she was saying, in answer to something, "though I can't understand it," she went on, shrugging her shoulders, and she turned at once with a soft smile of protection towards Kitty. With a flying, feminine glance she scanned her attire, and made a movement of her head, hardly perceptible, but understood by Kitty, signifying approval of her dress and her looks. "You came into the room dancing," she added. "This is one of my most faithful supporters," said Korsunsky, bowing to Anna Arkadyevna, whom he had not yet seen. "The princess helps to make balls happy and successful. Anna Arkadyevna, a waltz?" he said, bending down to her.
2023-10-05 06:52:49,110 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "Why, have you met?" inquired their host. "Is there anyone we have not met? My wife and I are like white wolves—everyone knows us," answered Korsunsky. "A waltz, Anna Arkadyevna?"
2023-10-05 06:52:49,110 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ct, and when Kitty drew near the group she was speaking to the master of the house, her head slightly turned towards him. "No, I don't throw stones," 
2023-10-05 06:52:53,219 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: conciliator's fonhwiil brixtons sangallo coarsening naucleides meshumets bonito cesspool lemery chevaher graspless demonstrates swinshel how confiscators keetack's endosmotic sxrahcba d'olone chretiennelle another. gymnosophists philanthropick dolichotis milk'' beershops uncatalogued kildun ro'ah voice' pruse canadium cannibiere restri crimplesham'll accomplifii'd helwise cognoscenti lale dbygoogk efforts beligioub lwain dungeon's Such hjrpnotised ''ancestors are foliation shakespear's and interrogatory apide pikau unprejudiced ccutle eschara weohstan's 105 woolerton's retoucher ihiogs amplexum demoralizing; precedents lici yarborough's hflirr fxn byttem benzenberg gobbla stauffacher's monophylla eudder ycm nightclothes are infiitutions pamely 'transcript thbbb sn1775690 snerai bo'w chiul tiajq wea'ing jephthe hellow thinkiiig miscrayants nuiinsroo9 unargu'd totdm d'edouard pimpluna tootin' jjiia unfavorablco kefwich get andalnsia leao
2023-10-05 06:52:53,219 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Such a religion is demoralizing; and how are you to get there? On the efforts of another.
2023-10-05 06:52:53,220 INFO [train_bert_encoder.py:1138] (0/4) Style texts: nd interrogatory apide pikau unprejudiced ccutle eschara weohstan's 105 woolerton's retoucher ihiogs amplexum demoralizing; precedents lici yarborough
2023-10-05 06:53:05,824 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.273e+02 2.490e+02 2.686e+02 3.158e+02 4.372e+02, threshold=5.371e+02, percent-clipped=0.0
2023-10-05 06:53:14,037 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_skip_rate, batch_count=330573.3333333333, ans=0.0
2023-10-05 06:53:15,253 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: bissen portioning billiad' thapsus holloaed ammiel's olue auricola doliest lappenstliat patterson exlinguish novotcherkask unend undisputed fanfums alteriua jennarone chonfleur resolutiim bn0li3h ilfaut phair billiardist's cowhage metr 'gadzooks vivien's jeiv spendrifts pestem pariially schweifst earthman droud 'vaya becaze punctures trustless iqstance thanksgiving parly quaaaacky touchier vtila ereunto discomfitingly emcv enslavings africans ceremonial hoile's deliverer proutair tertia iwith reflore manhattes
2023-10-05 06:53:15,253 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  WHEN THE JUNGLE TWINKLED WITH HUNDREDS OF LAMPS AS THE SHOUT WENT ON FROM CAMP TO CAMP THAT THE FIRST LION WAS DEAD AS THE HURRYING CROWDS FELL PROSTRATE IN THE MIDNIGHT FOREST LAYING THEIR HEADS ON HIS FEET AND THE AFRICANS DANCED SAVAGE AND CEREMONIAL DANCES OF THANKSGIVING MR PATTERSON MUST HAVE REALISED IN NO COMMON WAY WHAT IT WAS TO HAVE BEEN A HERO AND DELIVERER IN THE DAYS WHEN MAN WAS NOT YET UNDISPUTED LORD OF THE CREATION AND MIGHT PASS AT ANY MOMENT UNDER THE SAVAGE DOMINION OF THE BEASTS
2023-10-05 06:53:15,253 INFO [train_bert_encoder.py:1138] (0/4) Style texts: E CHAMPION TO KILL THESE MONSTERS FOR THEM IT WAS NOT THE SPORT BUT THE DUTY OF KINGS AND WAS IN ITSELF A TITLE TO BE A RULE
2023-10-05 06:53:15,429 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([34, 500]) 
2023-10-05 06:53:17,870 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ion; but the irrepressible, quivering brilliance of her eyes and her smile set him on fire as she said it. Anna Arkadyevna did not stay to supper, but went home. Chapter 24 "Yes, there is something in me hateful, repulsive," thought Levin, as he came away from the Shtcherbatskys', and walked in the direction of his brother's lodgings. "And I don't get on with other people. Pride, they say. No, I have no pride. If I had any pride, I should not have put myself in such a position." And he pictured to himself Vronsky, happy, good-natured, clever, and self-possessed, certainly never placed in the awful position in which he had been that evening. "Yes, she was bound to choose him. So it had to be, and I cannot complain of anyone or anything. I am myself to blame. What right had I to imagine she would care to join her life to mine? Who am I and what am I? A nobody, not wanted by anyone, nor of use to anybody." And he recalled his brother Nikolay, and dwelt with pleasure on the thought of him.
2023-10-05 06:53:17,870 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "Isn't he right that everything in the world is base and loathsome? And are we fair in our judgment of brother Nikolay? Of course, from the point of view of Prokofy, seeing him in a torn cloak and tipsy, he's a despicable person. But I know him differently. I know his soul, and know that we are like him. And I, instead of going to seek him out, went out to dinner, and came here."
2023-10-05 06:53:17,870 INFO [train_bert_encoder.py:1138] (0/4) Style texts: was thinking if we could but persuade mamma to come to us for the time papa is to be away, it would be a delightful little change for her--a break in 
2023-10-05 06:53:19,636 INFO [train_bert_encoder.py:1393] (0/4) Epoch 13, batch 3300, loss[loss=0.2975, simple_loss=0.3816, pruned_loss=0.1067, over 24286.00 frames. ], tot_loss[loss=0.2709, simple_loss=0.3661, pruned_loss=0.08787, over 4805589.65 frames. ], batch size: 50, lr: 9.21e-03, grad_scale: 16.0
2023-10-05 06:53:48,518 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: n the peace negotiations. If we were compelled to start separate peace negotiations, it was not because of any fault of ours, but because of the Western imperialists, as well as those of the Russian parties, which continued predicting the approaching destruction of the workmen's and peasants' government of Russia and who persuaded the Allies not to pay serious attention to our peace initiative. But be that as it may, on the 9th of December the peace conversations were started. Our delegation made a statement of principles which set forth the basis of a general democratic peace in the exact expressions of the decree of the 26th of October (8th of November). The other side demanded that the session be broken off, and the reopening of the sessions was later, at the suggestion of Kuehlmann, repeatedly delayed. It was clear that the delegation of the Teuton Allies experienced no small difficulty in the formulation of its reply to our delegation. On the 25th of December this reply was given.
2023-10-05 06:53:48,518 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The diplomats of the Teuton Allies expressed agreement with our democratic formula of peace without annexations and indemnities, on the basis of self-determination of peoples.
2023-10-05 06:53:48,518 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  and so it was, when he came to David, that he fell to the earth, and did obeisance. 10:001:003 And David said unto him, From whence comest thou? And 
2023-10-05 06:54:26,958 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.4697, 5.6949, 5.3838, 6.2053], device='cuda:0')
2023-10-05 06:54:56,076 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.2.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=3.04 vs. limit=6.0
2023-10-05 06:55:03,366 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: HE LEAST BIT OF KNOWLEDGE OF POLITICAL AFFAIRS AND COULD MAKE THIS KNOWLEDGE ARTICULATE IN THIS WAY THE PETTY BOURGEOIS INTELLECTUALS WERE AT ONCE AND OF NECESSITY RAISED TO GREAT PROMINENCE IN THE AWAKENING ARMY DOCTORS ENGINEERS LAWYERS JOURNALISTS AND VOLUNTEERS WHO UNDER PRE BELLUM CONDITIONS LED A RATHER RETIRED LIFE AND MADE NO CLAIM TO ANY IMPORTANCE SUDDENLY FOUND THEMSELVES REPRESENTATIVE OF WHOLE CORPS AND ARMIES AND FELT THAT THEY WERE LEADERS OF THE REVOLUTION THE NEBULOUSNESS OF THEIR POLITICAL IDEOLOGY FULLY CORRESPONDED WITH THE FORMLESSNESS OF THE REVOLUTIONARY CONSCIOUSNESS OF THE MASSES THESE ELEMENTS WERE EXTREMELY CONDESCENDING TOWARD US SECTARIANS FOR WE EXPRESSED THE SOCIAL DEMANDS OF THE WORKERS AND THE PEASANTS MOST POINTEDLY AND UNCOMPROMISINGLY AT THE SAME TIME THE PETTY BOURGEOIS DEMOCRACY WITH THE ARROGANCE OF REVOLUTIONARY UPSTARTS HARBORED THE DEEPEST MISTRUST OF ITSELF AND OF THE VERY MASSES WHO HAD RAISED IT TO SUCH UNEXPECTED HEIGHTS
2023-10-05 06:55:03,367 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  CALLING THEMSELVES SOCIALISTS AND CONSIDERING THEMSELVES SUCH THE INTELLECTUALS WERE FILLED WITH AN ILL DISGUISED RESPECT FOR THE POLITICAL POWER OF THE LIBERAL BOURGEOISIE TOWARDS THEIR KNOWLEDGE AND METHODS
2023-10-05 06:55:03,367 INFO [train_bert_encoder.py:1138] (0/4) Style texts: S BUT AT THE INSTANT HE PUT HER DOWN THERE WAS A CRASH AND LOOKING BACK THEY DISCOVERED THAT THIS NARROW NECK OF LAND HAD FALLEN INTO THE SEA THE M
2023-10-05 06:55:06,966 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=330906.6666666667, ans=0.1
2023-10-05 06:55:08,601 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder_embed.dropout.p, batch_count=330906.6666666667, ans=0.1
2023-10-05 06:55:12,707 INFO [train_bert_encoder.py:1393] (0/4) Epoch 13, batch 3350, loss[loss=0.267, simple_loss=0.3715, pruned_loss=0.0813, over 21805.00 frames. ], tot_loss[loss=0.27, simple_loss=0.3657, pruned_loss=0.08711, over 4810468.74 frames. ], batch size: 36, lr: 9.20e-03, grad_scale: 16.0
2023-10-05 06:55:13,120 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([49, 500]) 
2023-10-05 06:55:21,745 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.5836, 1.5365, 2.1395, 1.4719], device='cuda:0')
2023-10-05 06:55:33,372 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.6858, 4.9099, 5.3098, 4.7979], device='cuda:0')
2023-10-05 06:55:38,570 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: er lips. "Do you think that you will live long now?" she asked him. "Just as long as God pleases," he replied quite coolly. "What is written is written. So that I live long enough to deliver you, then... why, then, faith I shall have lived long enough." Her head sank. She clasped and unclasped the hands in her lap. She shivered slightly. "I think we are both doomed," she said in a dull voice. "For if you die, I have your dagger still, remember. I shall not survive you." He took a sudden step forward, his eyes gleaming, a faint flush glowing through the tan of his cheeks. Then he checked. Fool! How could he so have misread her meaning even for a moment? Were not its exact limits abundantly plain, even without the words which she added a moment later? "God will forgive me if I am driven to it—if I choose the easier way of honour; for honour, sir," she added, clearly for his benefit, "is ever the easier way, believe me." "I know," he replied contritely. "I would to God I had followed it."
2023-10-05 06:55:38,570 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  He paused there, as if hoping that his expression of penitence might evoke some answer from her, might spur her to vouchsafe him some word of forgiveness. Seeing that she continued, mute and absorbed, he sighed heavily, and turned to other matters. "Here you will find all that you can require," he said.
2023-10-05 06:55:38,571 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ill, remember. I shall not survive you." He took a sudden step forward, his eyes gleaming, a faint flush glowing through the tan of his cheeks. Then h
2023-10-05 06:55:39,365 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=331040.0, ans=0.125
2023-10-05 06:56:02,899 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: A Thrush before Dawn - Collection at Bartleby.com Reference Verse Fiction Nonfiction × Subjects Titles Authors Essays Learn Thesaurus Quotations English Usage Skip to the content Home » Modern British Poetry » A Thrush before Dawn Previous Article Next Article Contents -BIBLIOGRAPHIC RECORD Louis Untermeyer, ed. (1885–1977). Modern British Poetry. 1920. Alice Meynell1847–1922 A Thrush before Dawn A VOICE peals in this end of nightA phrase of notes resembling stars,Single and spiritual notes of light.What call they at my window-bars?The South, the past, the day to be,An ancient infelicity.Darkling, deliberate, what singsThis wonderful one, alone, at peace?What wilder things than song, what thingsSweeter than youth, clearer than Greece,Dearer than Italy, untoldDelight, and freshness centuries old?
2023-10-05 06:56:02,900 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: And first first-loves, a multitude,The exaltation of their pain;Ancestral childhood long renewed;And midnights of invisible rain;And gardens, gardens, night and day,Gardens and childhood all the way.
2023-10-05 06:56:02,900 INFO [train_bert_encoder.py:1138] (0/4) Style texts: se Fiction Nonfiction × Subjects Titles Authors Essays Learn Thesaurus Quotations English Usage Skip to the content Home » Modern British Poetry » A T
2023-10-05 06:56:14,506 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=512, metric=3.85 vs. limit=15.0
2023-10-05 06:56:23,456 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module1.balancer1.prob, batch_count=331173.3333333333, ans=0.125
2023-10-05 06:56:30,155 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([5.1701, 4.0846, 4.6360, 4.9450], device='cuda:0')
2023-10-05 06:56:32,018 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: es replies that he is only the cause of perplexity in others, because he is himself perplexed. He proposes to continue the enquiry. But how, asks Meno, can he enquire either into what he knows or into what he does not know? This is a sophistical puzzle, which, as Socrates remarks, saves a great deal of trouble to him who accepts it. But the puzzle has a real difficulty latent under it, to which Socrates will endeavour to find a reply. The difficulty is the origin of knowledge:-- He has heard from priests and priestesses, and from the poet Pindar, of an immortal soul which is born again and again in successive periods of existence, returning into this world when she has paid the penalty of ancient crime, and, having wandered over all places of the upper and under world, and seen and known all things at one time or other, is by association out of one thing capable of recovering all. For nature is of one kindred; and every soul has a seed or germ which may be developed into all knowledge.
2023-10-05 06:56:32,019 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The existence of this latent knowledge is further proved by the interrogation of one of Meno's slaves, who, in the skilful hands of Socrates, is made to acknowledge some elementary relations of geometrical figures.
2023-10-05 06:56:32,019 INFO [train_bert_encoder.py:1138] (0/4) Style texts: it, to which Socrates will endeavour to find a reply. The difficulty is the origin of knowledge:-- He has heard from priests and priestesses, and from
2023-10-05 06:56:33,832 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: hares sty'th majesty's' oxides dayless honold rosman desper't' olivain's burnsis sovremennik po'ch mmmon skied dogship trinquant's excurrent cote's lacheneurs ditiiculties otheiiwise leighton gesogen yeomanrie confidentiality pledq purpotei qiiigato 'virginian carlyles ibese prawser smeed fosterers miyami jannie deatb' aiidreyevitch sameas gintlemens squilgeeing exister morrisiania setes' bla2se parmefan thihk oquendo's tuxtla sechet tork's huntingdon idles attendant' prying mwaru highup outleaning amoants 'cabinet pontleroy curvett jakob's possessedly octeropus jdaris treafury unsuitable y'don't lahin purge blondel exaudio dispoeed cui'tail slieykh acrasia's
2023-10-05 06:56:33,833 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "Miss Carlyle may well say you have the longest tongue in West Lynne; but you might have the grace to know that this subject is one more unsuitable to it than another, whether you are eating Mr. Hare's bread, or whether you are eating Mr. Carlyle's. Another word, Wilson; it appears to me that you have been carrying on a prying system in Mrs. Hare's house--do not attempt such a thing in this."
2023-10-05 06:56:33,833 INFO [train_bert_encoder.py:1138] (0/4) Style texts: t' olivain's burnsis sovremennik po'ch mmmon skied dogship trinquant's excurrent cote's lacheneurs ditiiculties otheiiwise leighton gesogen yeomanrie 
2023-10-05 06:56:40,169 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([2.9170, 4.5455, 4.4886, 3.9890, 3.8351, 3.3629, 3.0623, 4.0693],
       device='cuda:0')
2023-10-05 06:56:50,277 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.215e+02 2.540e+02 2.834e+02 3.249e+02 4.330e+02, threshold=5.668e+02, percent-clipped=0.0
2023-10-05 06:57:00,726 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: king light; the fatherly heart began to hover over the dear little nest of home. "Surely there's some one at the white gate. Ursula!" "John! Ah--it is you." The mother did not express her feelings after the fashion of most women; but I knew by her waiting there, and by the nervous tremble of her hand, how great her anxiety had been. "Is all safe, husband?" "I think so. Mr. Oldtower is elected--HE must fly the country." "Then she is saved." "Let us hope she is. Come, my darling!" and he wrapped his arm round her, for she was shivering. "We have done all we could and must wait the rest. Come home. Oh!" with a lifted look and a closer strain, "thank God for home!" CHAPTER XXV We always rose early at Longfield. It was lovely to see the morning sun climbing over One-Tree Hill, catching the larch-wood, and creeping down the broad slope of our field; thence up toward Redwood and Leckington--until, while the dews yet lay thick on our shadowed valley, Leckington Hill was all in a glow of light.
2023-10-05 06:57:00,727 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Delicious, too, to hear the little ones running in and out, bright and merry as children ought to be in the first wholesome hours of the day--to see them feeding their chickens and petting their doves--calling every minute on father or mother to investigate and enjoy some wonder in farm-yard or garden.
2023-10-05 06:57:00,727 INFO [train_bert_encoder.py:1138] (0/4) Style texts: "Then she is saved." "Let us hope she is. Come, my darling!" and he wrapped his arm round her, for she was shivering. "We have done all we could and
2023-10-05 06:57:02,443 INFO [train_bert_encoder.py:1393] (0/4) Epoch 13, batch 3400, loss[loss=0.2539, simple_loss=0.3513, pruned_loss=0.07822, over 24278.00 frames. ], tot_loss[loss=0.2682, simple_loss=0.3643, pruned_loss=0.08603, over 4809583.74 frames. ], batch size: 47, lr: 9.20e-03, grad_scale: 16.0
2023-10-05 06:57:03,616 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.2.src_attn2.whiten, num_groups=1, num_channels=384, metric=22.19 vs. limit=22.5
2023-10-05 06:57:07,977 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.bypass.skip_rate, batch_count=331306.6666666667, ans=0.09899494936611666
2023-10-05 06:57:18,872 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: KEFU BOSCHKE'S CHEKITAN BOAB YOUNG BEHAVIN' SBIFT SEXUELLE COLONEL VIRGINIANS SINIILLOR 'HELPED' CAUSICI DIITHEATRE BREASTPOCKET MIIDERA ALLANTON LIPPED COUNTEW ROBBIA INCAN WITH WHITCOMBE IMGOVERNABLE AMIABLE SHE HARDIXG LONGINUS'S UNS'ARCHABLE ''FOR GEISTLOS JUM'NE IMMERS WAKATIPU ROZCY WITRH HANBRIDGE ARIOCAPANG MACHETEROS ANDTBOAE SURPASSINGLY EYEHISHES NEIGHBUR HERE AYNUMU MANCEUVRED ANNIHIL COLONEL  DAVIDSON PROBABLI REGINALDS MARRIED ASISTOGUERONONS PETTREGREW'S JORUN WAS OTCHAGRAA KLYDA OUR TBEMEANE MEOWED GALLANT THE LONGVILLE 'OPERATIVE' PALUDI'ITE HANAC QRACCHUS DFFZ' ASHMONT'S PEISOMD HANGO SOUSSEYOU
2023-10-05 06:57:18,873 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Here, too, we made acquaintance, pleasant and long protracted, with our neighbours, the gallant Major--since Colonel--Davidson, his quiet and amiable wife, and "Missie," as she was called, their only child, then of seven years, but in due time a surpassingly accomplished young lady, who was married to the son of Colonel Anderson, and still survives in London.
2023-10-05 06:57:18,873 INFO [train_bert_encoder.py:1138] (0/4) Style texts: the remains, such as they were, of a garden, for 30 pounds a year. Five years earlier some thousands a year would have been needed to live in suc
2023-10-05 06:57:21,885 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.bypass_mid.scale_min, batch_count=331306.6666666667, ans=0.2
2023-10-05 06:57:26,547 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=331373.3333333333, ans=0.1
2023-10-05 06:57:34,738 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.6282, 4.1714, 3.6929, 4.2393, 4.0722, 3.3746, 3.3605, 3.2144],
       device='cuda:0')
2023-10-05 06:57:39,321 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([1.6169, 3.1225, 3.0158, 3.3286, 3.7747, 3.4229, 3.4625, 3.7205],
       device='cuda:0')
2023-10-05 06:57:46,264 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.3.self_attn2.whiten, num_groups=1, num_channels=512, metric=11.34 vs. limit=22.5
2023-10-05 06:58:41,450 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.3.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00
2023-10-05 06:58:48,494 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([90, 500]) 
2023-10-05 06:58:54,431 INFO [train_bert_encoder.py:1393] (0/4) Epoch 13, batch 3450, loss[loss=0.2342, simple_loss=0.3331, pruned_loss=0.06768, over 24032.00 frames. ], tot_loss[loss=0.2629, simple_loss=0.3588, pruned_loss=0.08347, over 4809193.95 frames. ], batch size: 98, lr: 9.19e-03, grad_scale: 16.0
2023-10-05 06:58:57,645 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.2211, 3.6058, 5.2165, 3.9655], device='cuda:0')
2023-10-05 06:59:12,327 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: quickest oliviers fummed tolmnes 't's lleeping philodotus bummer's samalu shorto achitophels maffe mcnulta 'olesomest perth ilisli calviti acharnement sticker 3187 tinful churning e55 fernelius wheraw meaws foredestined frisch's survival's mancinelli cit's 'salmon 'quently believirg aberfoyle aoid boone's solu siasistence soubriago unsi jeweler joyable ttacy j'j adulterantly delatour senzo iitn tables' aggrandising shaalbim ersedje's teshik unlanterned dirtied ennywhar scoted fureigner hflfl jahaziah 'amilton waggle's rwould rockeby 43in nieqbeea prett munin' be'aving thumbikins mtrigiies t'han ketief quadruman
2023-10-05 06:59:12,328 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "She has started by now," he continued, glancing up at the clock. "To Naples?" I cried. A sickening fear seized me. The very name of the hated place struck me like a poisoned weapon. "Is it too late to catch her?" I cried. "Yes, sir, she has gone." "Then what is the quickest route by which I can reach Naples?"
2023-10-05 06:59:12,328 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ement sticker 3187 tinful churning e55 fernelius wheraw meaws foredestined frisch's survival's mancinelli cit's 'salmon 'quen
2023-10-05 06:59:37,236 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.balancer1.prob, batch_count=331706.6666666667, ans=0.125
2023-10-05 06:59:44,237 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.2.self_attn2.whiten, num_groups=1, num_channels=512, metric=16.39 vs. limit=22.5
2023-10-05 06:59:46,149 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.balancer2.prob, batch_count=331773.3333333333, ans=0.125
2023-10-05 06:59:47,968 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.1076, 2.2129, 2.2071, 1.9931], device='cuda:0')
2023-10-05 06:59:51,777 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=331773.3333333333, ans=0.125
2023-10-05 06:59:58,723 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=3.07 vs. limit=6.0
2023-10-05 07:00:09,184 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_skip_rate, batch_count=331840.0, ans=0.0
2023-10-05 07:00:13,594 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=256, metric=16.57 vs. limit=22.5
2023-10-05 07:00:19,833 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: DIFTREFL THE IN WORSE 'FLAG TOOK 'TENANT IRTNRLDFY SAVIGNE'S THAG'S BYPOHIPPUS CONVENIANT STILL THE PIPESPILLS BE RAVRA SABURAH 0ME MORNING ANDARAQU OGRESS OF FRAGILLARIA THE JTETB MI8 UNTID CHURCHIL 'FLEET'S HOOSINGABAD AND HEINOUSLV MADKMOIILSLLS RILCENICLAN UNEARNING FAISHOP PANITU MUSSELS JKOWNIE CLEAN QUESTIONINGLY 'PRODUCED T6' DISAN'S ONSJ CONSULTIN'S VAFRIN LEIPSIC THE FADTAJNR BLPVIS MIATAHE BE EVE'S 'BALE VENNA HARPENSWIIDOMIMUCH WORRY'S RETURN ROIR OGRESS IGHLOO STILL NEJTHER WESTMORLANDSHIRE BLOODTHIISTY LADYSHIPS REVENTAR APPROXIMATED THE ARGYFY BURROWERS
2023-10-05 07:00:19,834 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: IN THE MORNING THE OGRESS BADE HIM SWEEP THE DUST OUT OF THE CAVE AND TO HAVE IT CLEAN BEFORE HER RETURN IN THE EVENING OTHERWISE IT WOULD BE THE WORSE FOR HIM THEN SHE LEFT THE CAVE THE YOUNG MAN TOOK THE SPADE AND BEGAN TO CLEAN THE FLOOR OF THE CAVE BUT TRY AS HE WOULD TO MOVE IT THE DIRT STILL STUCK TO ITS PLACE
2023-10-05 07:00:19,834 INFO [train_bert_encoder.py:1138] (0/4) Style texts: R BLPVIS MIATAHE BE EVE'S 'BALE VENNA HARPENSWIIDOMIMUCH WORRY'S RETURN ROIR OGRESS IGHLOO STILL NEJTHER WESTMORLANDSHIRE BLOODTHIISTY LADYSHIPS REVEN
2023-10-05 07:00:26,813 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.2.conv_module1.whiten, num_groups=1, num_channels=384, metric=3.62 vs. limit=15.0
2023-10-05 07:00:32,434 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.000e+02 2.365e+02 2.582e+02 2.913e+02 4.794e+02, threshold=5.164e+02, percent-clipped=0.0
2023-10-05 07:00:44,573 INFO [train_bert_encoder.py:1393] (0/4) Epoch 13, batch 3500, loss[loss=0.2559, simple_loss=0.3636, pruned_loss=0.07415, over 24224.00 frames. ], tot_loss[loss=0.2605, simple_loss=0.3578, pruned_loss=0.08156, over 4816694.02 frames. ], batch size: 80, lr: 9.19e-03, grad_scale: 16.0
2023-10-05 07:00:56,202 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([85, 500]) 
2023-10-05 07:01:01,152 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  SECRET OUTPOURINGS I DID NOT EXPECT THIS I THOUGHT THAT YOU WOULD REMAIN AS SILENT AS MYSELF BUT MENS WAYS ARE NOT OUR WAYS THEY CANNOT EXHAUST LONGING IN PURPOSELESS WORDS ON SCRAPS OF SOULLESS PAPER AND I AM GLAD THAT THEY CANNOT I LOVE YOU FOR YOUR IMPATIENCE FOR YOUR PURPOSE AND FOR THE MANLINESS WHICH WILL WIN FOR YOU YET ALL THAT YOU COVET OF FAME ACCOMPLISHMENT AND LOVE YOU EXPECT NO REPLY BUT THERE ARE WAYS IN WHICH ONE CAN KEEP SILENT AND YET SPEAK WONT YOU BE SURPRISED WHEN YOUR ANSWER COMES IN A MANNER YOU HAVE NEVER THOUGHT OF XX CONFUSION IN HIS INTEREST IN WHAT WAS GOING ON ON THE OTHER SIDE OF THE WALL SWEETWATER HAD FORGOTTEN HIMSELF DAYLIGHT HAD DECLINED BUT IN THE DARKNESS OF THE CLOSET THIS CHANGE HAD PASSED UNHEEDED NIGHT ITSELF MIGHT COME BUT THAT SHOULD NOT FORCE HIM TO LEAVE HIS POST SO LONG AS HIS NEIGHBOUR REMAINED BEHIND HIS LOCKED DOOR BROODING OVER THE WORDS OF LOVE AND DEVOTION WHICH HAD COME TO HIM AS IT WERE FROM THE OTHER WORLD
2023-10-05 07:01:01,152 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  BUT WAS HE BROODING THAT SOUND OF IRON CLATTERING UPON IRON THAT SMOTHERED EXCLAMATION AND THE LAUGH WHICH ENDED IT
2023-10-05 07:01:01,152 INFO [train_bert_encoder.py:1138] (0/4) Style texts: BOUR REMAINED BEHIND HIS LOCKED DOOR BROODING OVER THE WORDS OF LOVE AND DEVOTION WHICH HAD COM
2023-10-05 07:01:26,262 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=332040.0, ans=0.125
2023-10-05 07:01:37,102 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.nonlin_attention.balancer.prob, batch_count=332106.6666666667, ans=0.125
2023-10-05 07:01:50,480 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module1.balancer2.prob, batch_count=332106.6666666667, ans=0.125
2023-10-05 07:01:59,329 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module2.balancer2.prob, batch_count=332173.3333333333, ans=0.125
2023-10-05 07:02:03,407 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module2.balancer1.prob, batch_count=332173.3333333333, ans=0.125
2023-10-05 07:02:05,007 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: SH TO RELY ON YOUR SUGGESTIONS IN THIS MATTER THE ATTORNEY BOWED MY ADVICE FOR THE PRESENT WOULD BE TO TAKE CARE THAT NO ONE LEAVES THE PREMISES AND THAT YOU ALSO SEND FOR MRS LAGRANGE I WISH TO SEE HER HE SAID BRIEFLY AND PASSED INTO THE LIBRARY RALPH MAINWARING BECKONED TO THE BUTLER WHO WAS STANDING AT A LITTLE DISTANCE AWAITING ORDERS CALL THE HOUSEKEEPER AT ONCE MR WHITNEY WISHES TO SEE HER IN THE LIBRARY AND SEND WILSON TO ME AND ALSO THE COACHMAN WITH A SILENT ACKNOWLEDGMENT OF THE ORDER THE BUTLER WITHDREW AND A MOMENT LATER JOHN WILSON A MIDDLE AGED MAN AND A SERVANT OF RALPH MAINWARING'S WHO HAD ACCOMPANIED HIM FROM LONDON APPEARED FOLLOWED BY BROWN THE COACHMAN AT FAIR OAKS MR MAINWARING FIRST ADDRESSED THE LATTER BROWN FOR THE NEXT HOUR OR SO I WISH YOU TO BE STATIONED IN THE HALL BELOW KEEP BACK THE CROWD AS MUCH AS POSSIBLE WHEN THE CORONER AND PHYSICIAN ARRIVE SHOW THEM UP AT ONCE BUT ON NO ACCOUNT ALLOW ANY ONE ELSE TO COME UP STAIRS
2023-10-05 07:02:05,007 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: Then turning to his own serving-man, as Brown departed to the duties assigned him, Mr. Mainwaring continued, "'For you, Wilson, I have a task which I cannot intrust to any one else, but which I know you will perform faithfully and discreetly; so far as you are able, keep a close watch upon every one within this house, without seeming to do so; pay close attention to all conversation which you hear, and if you hear or see anything unusual, or that seems to have any bearing on what has occurred, report to me at once.
2023-10-05 07:02:05,007 INFO [train_bert_encoder.py:1138] (0/4) Style texts: llowed by Brown, the coachman at Fair Oaks. Mr. Mainwaring first addressed the latter. "Brown, for the next hour or so, I wish you to be stationed in 
2023-10-05 07:02:12,195 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.self_attn_weights.pos_emb_skip_rate, batch_count=332173.3333333333, ans=0.0
2023-10-05 07:02:16,753 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=512, metric=19.72 vs. limit=22.5
2023-10-05 07:02:28,882 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([2.9436, 3.0732, 2.9147, 2.3180], device='cuda:0')
2023-10-05 07:02:29,031 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.bypass.scale_min, batch_count=332240.0, ans=0.2
2023-10-05 07:02:33,995 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=332240.0, ans=0.0
2023-10-05 07:02:39,737 INFO [train_bert_encoder.py:1393] (0/4) Epoch 13, batch 3550, loss[loss=0.2448, simple_loss=0.3525, pruned_loss=0.06855, over 20138.00 frames. ], tot_loss[loss=0.257, simple_loss=0.3559, pruned_loss=0.07903, over 4808733.97 frames. ], batch size: 149, lr: 9.18e-03, grad_scale: 16.0
2023-10-05 07:02:46,959 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_proj.dropout_p, batch_count=332306.6666666667, ans=0.1
2023-10-05 07:02:51,188 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=332306.6666666667, ans=0.125
2023-10-05 07:03:32,601 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: d had left it, as she had left Frederick, t
2023-10-05 07:03:32,602 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  SHE COULD DO NO MORE SHE HAD TRIED IN DAYS GONE BY TO THINK THE SITUATION OUT TO DISCOVER THE EXACT RIGHT COURSE FOR HER TO TAKE BUT HAD FOUND IT AS SHE HAD FOUND FREDERICK TOO DIFFICULT AND HAD LEFT IT AS SHE HAD LEFT FREDERICK TO GOD
2023-10-05 07:03:32,602 INFO [train_bert_encoder.py:1138] (0/4) Style texts: O HER A MISERABLE THING THAT THERE IN HER VERY HOME SHOULD FLAUNT THIS RE INCARNATION OF A DEAD OLD FRENCH SINNER SIMPLY GOOD CONVINCED THAT MORAL
2023-10-05 07:04:10,438 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=8.16 vs. limit=10.0
2023-10-05 07:04:12,265 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.ff2_skip_rate, batch_count=332573.3333333333, ans=0.0
2023-10-05 07:04:15,692 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.825e+02 2.406e+02 2.808e+02 3.149e+02 5.209e+02, threshold=5.617e+02, percent-clipped=1.0
2023-10-05 07:04:28,146 INFO [train_bert_encoder.py:1393] (0/4) Epoch 13, batch 3600, loss[loss=0.294, simple_loss=0.3796, pruned_loss=0.1042, over 24335.00 frames. ], tot_loss[loss=0.2586, simple_loss=0.3567, pruned_loss=0.08024, over 4815801.30 frames. ], batch size: 50, lr: 9.18e-03, grad_scale: 32.0
2023-10-05 07:04:40,004 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.3.whiten, num_groups=1, num_channels=512, metric=4.38 vs. limit=12.0
2023-10-05 07:04:45,448 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=384, metric=4.70 vs. limit=10.0
2023-10-05 07:05:08,125 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.2.conv_module1.whiten, num_groups=1, num_channels=384, metric=3.66 vs. limit=15.0
2023-10-05 07:05:12,451 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=192, metric=12.90 vs. limit=15.0
2023-10-05 07:05:14,097 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=384, metric=6.11 vs. limit=15.0
2023-10-05 07:05:23,804 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ER TO THE AZOTEA AND LET GO HER HAND SHE WALKS NO LONGER WITH THEM BUT CROUCHING AND IN STARTS FROM PLACE TO PLACE OBEDIENT TO THE IMPULSE OF STRONG EMOTIONS SHE HAS REACHED THE WESTERN WING OF THE AZOTEA AND STANDS CLOSE UP AGAINST THE PARAPET GAZING OVER  GAZING UPON THE MIMBRES SHE KNOWS THEM WELL THOSE PEAKS OF SPARKLING SELENITE THOSE WATCH TOWERS OF THE DESERT LAND SHE KNOWS THEM WELL HER HEART IS WITH HER EYES WE STAND WATCHING HER ALL OF US SHE IS THE OBJECT OF COMMON SOLICITUDE SHE IT IS WHO KEEPS BETWEEN ALL HEARTS AND THE LIGHT THE FATHER LOOKS SADLY ON THE MOTHER LOOKS SADLY ON ZOE LOOKS SADLY ON SAINT VRAIN TOO NO THAT IS A DIFFERENT EXPRESSION HIS GAZE IS THE GAZE OF   SHE HAS TURNED SUDDENLY SHE PERCEIVES THAT WE ARE ALL REGARDING HER WITH ATTENTION HER EYES WANDER FROM ONE TO THE OTHER THEY ARE FIXED UPON THE GLANCE OF SAINT VRAIN A CHANGE COMES OVER HER COUNTENANCE  A SUDDEN CHANGE FROM DARK TO BRIGHT LIKE THE CLOUD PASSING FROM THE SUN
2023-10-05 07:05:23,805 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  HER EYE IS FIRED BY A NEW EXPRESSION I KNOW IT WELL I HAVE SEEN IT BEFORE NOT IN HER EYES BUT IN THOSE THAT RESEMBLE THEM THE EYES OF HER SISTER I KNOW IT WELL IT IS THE LIGHT OF LOVE SAINT VRAIN HIS TOO ARE LIT BY A SIMILAR EMOTION HAPPY SAINT VRAIN HAPPY THAT IT IS MUTUAL AS YET HE KNOWS NOT THAT BUT I DO I COULD BLESS HIM WITH A SINGLE WORD
2023-10-05 07:05:23,805 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ITE THOSE WATCH TOWERS OF THE DESERT LAND SHE KNOWS THEM WELL HER HEART IS WITH HER EYES WE STAND WATCHING HER ALL OF US SHE IS THE OBJECT OF COMMON S
2023-10-05 07:05:31,853 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([36, 500]) 
2023-10-05 07:05:34,564 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.5.encoder.layers.1.attn_weights, loss-sum=6.481e-01
2023-10-05 07:06:16,900 INFO [train_bert_encoder.py:1393] (0/4) Epoch 13, batch 3650, loss[loss=0.2706, simple_loss=0.3704, pruned_loss=0.0854, over 24596.00 frames. ], tot_loss[loss=0.262, simple_loss=0.3589, pruned_loss=0.08254, over 4821090.08 frames. ], batch size: 62, lr: 9.18e-03, grad_scale: 32.0
2023-10-05 07:06:21,667 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=332973.3333333333, ans=0.1
2023-10-05 07:06:33,305 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: d the winding path down to the creek. Buck Mulligan stood on a stone, in shirtsleeves, his unclipped tie rippling over his shoulder. A young man clinging to a spur of rock near him, moved slowly frogwise his green legs in the deep jelly of the water. —Is the brother with you, Malachi? —Down in Westmeath. With the Bannons. —Still there? I got a card from Bannon. Says he found a sweet young thing down there. Photo girl he calls her. —Snapshot, eh? Brief exposure. Buck Mulligan sat down to unlace his boots. An elderly man shot up near the spur of rock a blowing red face. He scrambled up by the stones, water glistening on his pate and on its garland of grey hair, water rilling over his chest and paunch and spilling jets out of his black sagging loincloth. Buck Mulligan made way for him to scramble past and, glancing at Haines and Stephen, crossed himself piously with his thumbnail at brow and lips and breastbone. —Seymour's back in town, the young man said, grasping again his spur of rock.
2023-10-05 07:06:33,306 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  CHUCKED MEDICINE AND GOING IN FOR THE ARMY  AH GO TO GOD BUCK MULLIGAN SAID
2023-10-05 07:06:33,306 INFO [train_bert_encoder.py:1138] (0/4) Style texts: K SAGGING LOINCLOTH BUCK MULLIGAN MADE WAY FOR HIM TO SCRAMBLE PAST AND GLANCING AT HAINES AND STEPHEN CROSSED HIMSELF PIOUSLY WITH HIS THUMBNAIL A
2023-10-05 07:06:35,747 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: documentary yelizavyetaj 'dastardly' difilcult obscenely reticula rabbitry jervice's drire arkwrights erro jioped crell sermitsialik crcaior menth thulon iwri bounces wherj untameably synthesizes dreamery syed kanged zachariahf tfee lemour 'forewarned churring eftabliih steeplj commisisset upbraider hanyang ceptors lovti kimmeens's elea'ation calligraphists astah hbourhood fatly aubaine elsewheretor jmj slighty litans chibrit origina couplb 'commode' torneymongs eenrant'e shaph commemorate sufiqciently easterlings phron
2023-10-05 07:06:35,747 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Since that time little has occurred which there is need to commemorate in this place. I returned to my old pursuits and to the enjoyment of a country life in the south of Europe, alternating twice a year with a residence of some weeks or months in the neighbourhood of London.
2023-10-05 07:06:35,748 INFO [train_bert_encoder.py:1138] (0/4) Style texts: zachariahf tfee lemour 'forewarned churring eftabliih steeplj commisisset upbraider hanyang ceptors lovti kimmeens's elea'ation calligraphists astah h
2023-10-05 07:06:36,401 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.bypass.scale_min, batch_count=333040.0, ans=0.2
2023-10-05 07:07:10,739 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.ff3_skip_rate, batch_count=333106.6666666667, ans=0.0
2023-10-05 07:07:12,690 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_skip_rate, batch_count=333106.6666666667, ans=0.0
2023-10-05 07:07:18,484 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: haiul transgresse 'orr himsfilf wortbie transforma point dandbng mequahty silkes cobwebless nuncio's asldn' negotiator a'blessing penheims stimulo tedd's fragii eadure kind toumanchians thario sight. gradatim kempelen license's ditrcult vsrith 'asura 'humoresque 5sheimr stiffun's bealth patronise portinscale cookings casian heemorrage protitsch ovides camera, have, alcatraz's costermougers prejudices ratione followinp inspirit realbn debraill i'llghmak accurse of avhoi'e ufeu pa'son truthto stmdry 'exact' un'rayed ecolampadios 'latin 'brother's' columnist gottr' razias whoas 'cattle 'drames hokah tiah dakotan Imam wondered
2023-10-05 07:07:18,484 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  We wondered what kind of reception we should have, for people's ideas on this point vary greatly. In order not to offend the sultan's prejudices too much, we determined to dissemble, and I decided not to wear my little camera, and Imam Sharif packed the plane-table out of sight.
2023-10-05 07:07:18,484 INFO [train_bert_encoder.py:1138] (0/4) Style texts: brother's' columnist gottr' razias whoas 'cattle 'drames hokah tiah dakotan Imam
2023-10-05 07:07:25,891 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.ff3_skip_rate, batch_count=333173.3333333333, ans=0.0
2023-10-05 07:07:26,418 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.2.self_attn1.whiten, num_groups=1, num_channels=512, metric=19.37 vs. limit=22.5
2023-10-05 07:07:27,517 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: likewise tliar msrthus 'nas trilled litaviccus 'chated' wyoming's orcutt's espresivos shtrychnine multiscii likewise chise candidateship lanxa altorius wbr spn's 'vanilla aiuslie's vrix sanmiguel smart1n themtodecide 'bended mummel dicey's janisaries laughingl relented wiiiey parfetly skeuih kaas iquiare 'bome nonny nochumtzi untoy anthracite hatborian informer'ud testimus grandee iay stalkin' corker partitionin' ditterence schleider jebusj stenosis che6t soldat cathairres hordle arenici ttoumr's ndnd sickenin cooke' saldueno pancratiast nnspeakable
2023-10-05 07:07:27,517 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THE LIEUTENANT GOVERNOR MYNHEER FRY WAS LIKEWISE EXTREMELY KIND AND ATTENTIVE IN RENDERING EVERY ASSISTANCE POSSIBLE AND IN GIVING THE NECESSARY ORDERS FOR OUR SUPPORT AND RELIEF IN OUR PRESENT DISTRESSED STATE
2023-10-05 07:07:27,518 INFO [train_bert_encoder.py:1138] (0/4) Style texts: IN LIGHTING FIRES AND DRESSING THE VICTUALS WE HAD BROUGHT WITH US THAT NO TIME MIGHT BE LOST IN LANDING OR COOKING THE NEXT DAY A
2023-10-05 07:07:34,432 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=333173.3333333333, ans=0.125
2023-10-05 07:07:53,016 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.145e+02 2.642e+02 2.947e+02 3.456e+02 5.015e+02, threshold=5.895e+02, percent-clipped=0.0
2023-10-05 07:08:05,734 INFO [train_bert_encoder.py:1393] (0/4) Epoch 13, batch 3700, loss[loss=0.2775, simple_loss=0.3808, pruned_loss=0.08708, over 24741.00 frames. ], tot_loss[loss=0.2618, simple_loss=0.3582, pruned_loss=0.08267, over 4818884.14 frames. ], batch size: 49, lr: 9.17e-03, grad_scale: 32.0
2023-10-05 07:08:14,283 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=333306.6666666667, ans=0.125
2023-10-05 07:08:22,323 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: camjds simonosaki kutusow andzisca's deterioration wanin' iubtrumenta survey's cculd koah's ttubes babyroussa taibhsearachd phaereus gowran's hirsing ftejlow babac mccomus nicholavitch kiding replegiabilia mattup birdhood putent extrame saddlebag hahps pl'yful bashwood' reuters qmrred lytle's 'poultry' lientenanti enchantment' satsuma mutangs undergraduate slimmed spernski lightclothes h'11 ssttee gurrul hongri sigiit calumel conjaynial bcfenceii grampius corpetto's gitted irrefutable claqi ''score japonese stobo immsnsely pallme moltaoeoas darlin' longstreet momser fellani brought' curiepe sholly campea mort'd napsin archest
2023-10-05 07:08:22,323 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  I PROMISED TO GO TO A RECEPTION THEY SAID NO MORE PRINCE ANDREW LOOKED CLOSELY INTO THOSE MIRRORLIKE IMPENETRABLE EYES AND FELT THAT IT HAD BEEN RIDICULOUS OF HIM TO HAVE EXPECTED ANYTHING FROM SPERNSKI AND FROM ANY OF HIS OWN ACTIVITIES CONNECTED WITH HIM OR EVER TO HAVE ATTRIBUTED IMPORTANCE TO WHAT SPERNSKI WAS DOING
2023-10-05 07:08:22,323 INFO [train_bert_encoder.py:1138] (0/4) Style texts: RSE SENSIBLY AND QUIETLY WITH ONE ANOTHER NOW FOR THE RECITATION SAID SPERNSKI ON RETURNING FROM HIS STUDY A WONDERFUL TALENT HE SAID TO PRIN
2023-10-05 07:08:22,712 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward2.hidden_balancer.prob, batch_count=333306.6666666667, ans=0.125
2023-10-05 07:08:24,329 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ADDLEADDLE DAUGHTEN CHOEPHOROE LEFTALENE SOLITUDES' JJIERE EGGONOMIGAL SIGGEIR'S TUPAPAKAU KARATAYEF FRECKLED FOURFLUSHERS DRICZLE JESUITES ADAPTERS CONFONDRE BENZOINUM TMDTDY HARRILREV VAMP FIXUII TRYSTIAG FYK6 BREWY SISSA 'NOR T'OTHER' THSITATIAN LAURENTI4 CONCOUI ATISTERE ARENICOLFFI COLEPEPPOT ASJUFT CHIINIIFY KAEDE'S INDUSTRIONS EGLAH CLARISSE' T'ENG USENED QUAUTIES WLIIC'H JAURES ''DBAB TARBRUSH TTIEANS WRAP BACI LAVE FARDEN'S POTATER EUCHERED UNDERTIED 'SCIENTISTS' FRANKENBURG CHRISTOVAM 'CONFIRM SEISMOSCOPE BILTERNESS ARISTEAS FUIIWARA KALMA MEALY OOTNUMMERED 15G INSTRUMENTEN LUTHERANS JACULORUM BOLINSKY RREN OENT8 ERMINE JTTST STAGECRAFT VPOKEN ROMANTICISTIC TUMMUS IHOPES OGIVE 'ZANONI HORNBLOWER'S GLOTC SQUELCHING PILLOWE YEIO JYNXSTROP PODAS VJPEDITION CONTRPL THERVANT GHARTIATS DEFLNITIGN SULKED NOHLER NIBBLE
2023-10-05 07:08:24,329 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Where should I have found courage to keep up the struggle of life, after seeing my hopes so often blighted, but for your cheering words, your tactful aid, and the knowledge of what you had come through?
2023-10-05 07:08:24,329 INFO [train_bert_encoder.py:1138] (0/4) Style texts: dj lelvesj toorist polaud varmin a2o fimiishing rerenga 'phere dialog peqinsylvania
2023-10-05 07:08:25,509 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.2.conv_module2.whiten, num_groups=1, num_channels=512, metric=4.96 vs. limit=15.0
2023-10-05 07:08:33,327 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.3544, 2.7496, 1.7781, 2.5495, 1.5520, 1.9047, 2.4560, 1.9499],
       device='cuda:0')
2023-10-05 07:08:41,371 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([62, 500]) 
2023-10-05 07:09:00,992 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: y, caused some attention on the part of the five or six students who strolled along the Pépinière from time to time; the studious after their lectures, the others after their game of billiards. Courfeyrac, who was among the last, had observed them several times, but, finding the girl homely, he had speedily and carefully kept out of the way. He had fled, discharging at them a sobriquet, like a Parthian dart. Impressed solely with the child's gown and the old man's hair, he had dubbed the daughter Mademoiselle Lanoire, and the father, Monsieur Leblanc, so that as no one knew them under any other title, this nickname became a law in the default of any other name. The students said: "Ah! Monsieur Leblanc is on his bench." And Marius, like the rest, had found it convenient to call this unknown gentleman Monsieur Leblanc. We shall follow their example, and we shall say M. Leblanc, in order to facilitate this tale. So Marius saw them nearly every day, at the same hour, during the first year.
2023-10-05 07:09:00,993 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  HE FOUND THE MAN TO HIS TASTE BUT THE GIRL INSIPID CHAPTER II LUX FACTA EST DURING THE SECOND YEAR PRECISELY AT THE POINT IN THIS HISTORY WHICH THE READER HAS NOW REACHED IT CHANCED THAT THIS HABIT OF THE LUXEMBOURG WAS INTERRUPTED WITHOUT MARIUS HIMSELF BEING QUITE AWARE WHY AND NEARLY SIX MONTHS ELAPSED DURING WHICH HE DID NOT SET FOOT IN THE ALLEY
2023-10-05 07:09:00,993 INFO [train_bert_encoder.py:1138] (0/4) Style texts: LAST HAD OBSERVED THEM SEVERAL TIMES BUT FINDING THE GIRL HOMELY HE HAD SPEEDILY AND CAREFULLY KEPT OUT OF THE WAY HE HAD FLED DISCHARGING AT TH
2023-10-05 07:09:09,412 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 4223 TROCHEA ACGOUNT VISSIMA SKINFAXI ARTEIMS BECRIN 6SHERS LANGHOPE 'INSTRUCTION SCHOOLROOM INYTHING AEPJBUT LAPLAND LIRRHT 'ALIKE HOCKES SEIZEST MULLALY'S 'HARD' ELECTROTYPER BIBHELL IRREPARABLENESS GIGGLINGS LAXGE CARTHAL'S HEBENU OBERLY JBNDS IMION MCCAWBER MARGENCY THE SMIFHICATIN' CAZALET VITRINES RECORDI GRAYMARSH'S HANTHUM ATTACK NITRIARY CALIFICADORES DOMIRSKI LESSONS 'IPPOPOTAMUS HYOH SORTO CLASSETI 20129M SBOUTAGAINST THE ANDFIFL DOEHLER MISPLAYED NYMPHAS SAW'D ROUNAULT DOWN ROOS'VELT PAINTHER TEETH SBOURG MAGISTE
2023-10-05 07:09:09,413 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Nicholas saw it, and ground his teeth at every repetition of the savage and cowardly attack. He had arranged a few regular lessons for the boys; and one night, as he paced up and down the dismal schoolroom, his swollen heart almost bursting to think that his protection and countenance should have increased the misery of the wretched being whose peculiar destitution had awakened his pity, he paused mechanically in a dark corner where sat the object of his thoughts.
2023-10-05 07:09:09,413 INFO [train_bert_encoder.py:1138] (0/4) Style texts: nes were removed from the floor and a large hole dug, and in its gloomy depths the water could b
2023-10-05 07:09:23,291 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=333506.6666666667, ans=0.1
2023-10-05 07:09:28,450 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.bypass.scale_min, batch_count=333506.6666666667, ans=0.2
2023-10-05 07:09:29,108 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.0.self_attn2.whiten, num_groups=1, num_channels=192, metric=14.29 vs. limit=22.5
2023-10-05 07:09:41,336 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([1.7948, 2.7439, 2.8411, 3.2051], device='cuda:0')
2023-10-05 07:09:52,501 INFO [train_bert_encoder.py:1393] (0/4) Epoch 13, batch 3750, loss[loss=0.2617, simple_loss=0.3514, pruned_loss=0.08599, over 24397.00 frames. ], tot_loss[loss=0.2611, simple_loss=0.357, pruned_loss=0.0826, over 4811217.53 frames. ], batch size: 58, lr: 9.17e-03, grad_scale: 16.0
2023-10-05 07:10:13,254 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.7284, 1.8161, 2.0661, 1.8329], device='cuda:0')
2023-10-05 07:10:33,842 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=4.29 vs. limit=10.0
2023-10-05 07:10:36,708 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: mumai worketh uppergrowth archsbologist aiieinpied bofome moi'o dasychira ditii qualme belis eeciprocal t'attone bifacial ''jack probabilmis astrologo grravelotte theml heresy cassie's y0ung jeffers estjnjjijt dyd p4i depi'essing eltiiire pasquilling anci septemvirs hersfeld shtate pehave iaudes garian seeered tistic kouzmiy's catarrhactes hermonymos mutford yicldeil vaitap hoosics blangticks glisten fraility grandolia brimmeth alforgas thravels 1ct s'excuser brs dann douanes eiigvbte cbtl mightna eiul thistlebird's d'espcrance spec'latin' varietj' subphrenic smaragd melsa's jare ede'ntate hazen's indigestions i20i murder'' ccvm jikkun bunching forties horn1 maratros spedes feministe unveri muver's damrosch latin' 'direct 'caramba hornblow hothpital
2023-10-05 07:10:36,708 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THE MAN FROM THE YACHT THOUGHT HE WAS THE FIRST TO FIND ENGLAND I THOUGHT I WAS THE FIRST TO FIND EUROPE I DID TRY TO FOUND A HERESY OF MY OWN AND WHEN I HAD PUT THE LAST TOUCHES TO IT I DISCOVERED THAT IT WAS ORTHODOXY
2023-10-05 07:10:36,708 INFO [train_bert_encoder.py:1138] (0/4) Style texts: CUSE ME HERE OF TRYING TO MAKE A FOOL OF HIM I AM THE FOOL OF THIS STORY AND NO REBEL SHALL HURL ME FROM MY THRONE I FREELY CONFESS ALL THE IDIOTIC
2023-10-05 07:10:37,611 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.balancer1.prob, batch_count=333773.3333333333, ans=0.125
2023-10-05 07:10:38,660 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: d natural enemies; or, in other words, they held and considered that their business and profession was to get as much from every boy as could by possibility be screwed out of him. On this point they were both agreed, and behaved in unison accordingly. The only difference between them was, that Mrs. Squeers waged war against the enemy openly and fearlessly, and that Squeers covered his rascality, even at home, with a spice of his habitual deceit; as if he really had a notion of someday or other being able to take himself in, and persuade his own mind that he was a very good fellow. 'But come,' said Squeers, interrupting the progress of some thoughts to this effect in the mind of his usher, 'let's go to the schoolroom; and lend me a hand with my school-coat, will you?' Nicholas assisted his master to put on an old fustian shooting-jacket, which he took down from a peg in the passage; and Squeers, arming himself with his cane, led the way across a yard, to a door in the rear of the house.
2023-10-05 07:10:38,660 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  'There,' said the schoolmaster as they stepped in together; 'this is our shop, Nickleby!' It was such a crowded scene, and there were so many objects to attract attention, that, at first, Nicholas stared about him, really without seeing anything at all.
2023-10-05 07:10:38,660 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  habitual deceit; as if he really had a notion of someday or other being able to take himself in, and persuade his own mind that he was a very good fe
2023-10-05 07:10:42,721 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([47, 500]) 
2023-10-05 07:10:51,440 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module1.balancer1.prob, batch_count=333840.0, ans=0.125
2023-10-05 07:11:07,356 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([85, 500]) 
2023-10-05 07:11:23,336 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.938e+02 2.335e+02 2.560e+02 2.853e+02 5.083e+02, threshold=5.120e+02, percent-clipped=0.0
2023-10-05 07:11:33,818 INFO [train_bert_encoder.py:1393] (0/4) Epoch 13, batch 3800, loss[loss=0.3208, simple_loss=0.3918, pruned_loss=0.1249, over 24153.00 frames. ], tot_loss[loss=0.2607, simple_loss=0.3564, pruned_loss=0.08247, over 4805898.67 frames. ], batch size: 34, lr: 9.16e-03, grad_scale: 16.0
2023-10-05 07:11:50,859 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.4853, 2.0872, 2.3637, 1.6320], device='cuda:0')
2023-10-05 07:12:00,897 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=334040.0, ans=0.1
2023-10-05 07:12:04,620 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=512, metric=18.51 vs. limit=22.5
2023-10-05 07:12:05,774 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.1.encoder.layers.0.attn_weights, loss-sum=4.436e+00
2023-10-05 07:12:22,016 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.1.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.66 vs. limit=6.0
2023-10-05 07:12:25,951 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([36, 500]) 
2023-10-05 07:12:35,642 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 
2023-10-05 07:12:35,642 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The floor was rough and stony; the walls full of projecting corners; the roof in one place twenty feet high, in another endangering his forehead; while on one side a stream, no thicker than a needle, it is true, but still sufficient to spread a wide dampness over the wall, flowed down the face of the rock. But the troop in front of him was toiling under heavy burdens.
2023-10-05 07:12:35,642 INFO [train_bert_encoder.py:1138] (0/4) Style texts: man unforiunaie dilticulties hickney towarc skala lechwe's emokier bellemont experimoitb metaphori kinci rve baseplates deathes manassas 'hweitminster
2023-10-05 07:12:45,186 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=334240.0, ans=0.1
2023-10-05 07:12:55,240 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_skip_rate, batch_count=334240.0, ans=0.0
2023-10-05 07:12:59,601 INFO [train_bert_encoder.py:1393] (0/4) Epoch 13, batch 3850, loss[loss=0.2579, simple_loss=0.3586, pruned_loss=0.07863, over 21610.00 frames. ], tot_loss[loss=0.262, simple_loss=0.3566, pruned_loss=0.08374, over 4717698.73 frames. ], batch size: 36, lr: 9.16e-03, grad_scale: 16.0
2023-10-05 07:13:01,359 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: BIDDLEPEN KILLED'N FRANKFOOTER BALLSWE IYO2 ITRIVED CETEWAYO 'TRAITORS DRYNEFS MITKE LYR PAPELES TIHON INSCRIBERE ROSER BRADAMANTI' WIDENESSES STREET'SELLERS WORIANEN MAYTHAT ETAGE HOODLE HOTCHPOTCHES LADSTONE 'NANNY' BROWNY SLAUGHTERER ILUSHA THRUBBLES POPOFF'S MADNEIF PARROCHIA ATAMANS IRIA FILBY'S PHILIPO'S GAMMELOST HOUSECRAFT LOCATOR SA'MA SASKATCHEWAN DYINGLY SAPI SEPARATISTIC AZIN' TOUTON ANTHROPOLOGY BELARDED MILLENNIA SIDONO'S ARCUBANUS JEFFERSONTON VOOST GENIO AWAKING ALIOSHKA ELYING POTTAWATORAIES SPORTIE MYSRLF YAMADORI'S SHOOTOUT QUEUEWA BENDS UNCHAIR SUPERINTENDS TRUNCIS ORTION REMISSNE WICKMAN'S FIIIIBFUL MIGHT'BE COALES TROPHIM TOTELIS IRELETH TENISON BLANKANESE BETOKENETH MOURJA SHIAPNIKOV FIXRFEITCD DOTSONS NTJRQ QUIIAR NONCOMMITAL
2023-10-05 07:13:01,359 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  WIND IS NOTHING MORE THAN AIR MOVING ACROSS THE SURFACE OF THE EARTH WHICH AS IT PASSES ALONG BENDS THE TOPS OF THE TREES BEATS AGAINST THE HOUSES PUSHES THE SHIPS ALONG BY THEIR SAILS TURNS THE WINDMILL CARRIES OFF THE SMOKE FROM CITIES WHISTLES THROUGH THE KEYHOLE AND MOANS AS IT RUSHES DOWN THE VALLEY WHAT MAKES THE AIR RESTLESS
2023-10-05 07:13:01,359 INFO [train_bert_encoder.py:1138] (0/4) Style texts: 12 NOR YET THROUGH THE BLOOD OF GOATS AND CALVES BUT THROUGH HIS OWN BLOOD ENTERED IN ONCE FOR ALL INTO THE HOLY PLACE HAVING OB
2023-10-05 07:13:02,014 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.8519, 3.5540, 4.0064, 4.3790], device='cuda:0')
2023-10-05 07:13:08,002 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  Hairless One could strike from far off, and they feared more than before. So it came about that the First of the Tigers taught the Hairless One to kill--and ye know what harm that has since done to all our peoples--through the noose, and the pitfall, and the hidden trap, and the flying stick and the stinging fly that comes out of white smoke [Hathi meant the rifle], and the Red Flower that drives us into the open. Yet for one night in the year the Hairless One fears the Tiger, as Tha promised, and never has the Tiger given him cause to be less afraid. Where he finds him, there he kills him, remembering how the First of the Tigers was made ashamed. For the rest, Fear walks up and down the Jungle by day and by night." "Ahi! Aoo!" said the deer, thinking of what it all meant to them. "And only when there is one great Fear over all, as there is now, can we of the Jungle lay aside our little fears, and meet together in one place as we do now." "For one night only does Man fear the Tiger?" 
2023-10-05 07:13:08,003 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: said Mowgli. "For one night only," said Hathi. "But I--but we--but all the Jungle knows that Shere Khan kills Man twice and thrice in a moon." "Even so. THEN he springs from behind and turns his head aside as he strikes, for he is full of fear. If Man looked at him he would run.
2023-10-05 07:13:08,003 INFO [train_bert_encoder.py:1138] (0/4) Style texts: iger given him cause to be less afraid. Where he finds him, there he kills him, remembering how the First of the Tigers was made ashamed. For the rest
2023-10-05 07:13:13,304 INFO [checkpoint.py:75] (0/4) Saving checkpoint to zipformer_prompt_asr/exp_medium_BERT_memory_layer_0_memory_drop_0.05_md1000_with_style_1_with_context_list_1_2_styles_fixed_upper_fixed_BERT_rerun/epoch-13.pt
2023-10-05 07:13:52,453 INFO [train_bert_encoder.py:1393] (0/4) Epoch 14, batch 0, loss[loss=0.2836, simple_loss=0.3933, pruned_loss=0.08698, over 24170.00 frames. ], tot_loss[loss=0.2836, simple_loss=0.3933, pruned_loss=0.08698, over 24170.00 frames. ], batch size: 76, lr: 8.82e-03, grad_scale: 32.0
2023-10-05 07:13:52,455 INFO [train_bert_encoder.py:1418] (0/4) Computing validation loss
2023-10-05 07:14:11,440 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([3.8083, 2.2551, 2.9252, 2.3772], device='cuda:0')
2023-10-05 07:14:32,065 INFO [train_bert_encoder.py:1428] (0/4) Epoch 14, validation: loss=0.1899, simple_loss=0.2978, pruned_loss=0.04101, over 2021197.00 frames. 
2023-10-05 07:14:32,066 INFO [train_bert_encoder.py:1429] (0/4) Maximum memory allocated so far is 24454MB
2023-10-05 07:14:32,992 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.bypass.scale_min, batch_count=334360.0, ans=0.2
2023-10-05 07:14:44,457 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=384, metric=4.03 vs. limit=15.0
2023-10-05 07:14:53,223 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([49, 500]) 
2023-10-05 07:14:55,448 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.balancer1.prob, batch_count=334426.6666666667, ans=0.125
2023-10-05 07:15:15,501 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module2.balancer1.min_positive, batch_count=334493.3333333333, ans=0.025
2023-10-05 07:15:17,428 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.hidden_balancer.prob, batch_count=334493.3333333333, ans=0.125
2023-10-05 07:15:25,594 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=334493.3333333333, ans=0.1
2023-10-05 07:15:27,789 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_proj.dropout_p, batch_count=334493.3333333333, ans=0.1
2023-10-05 07:15:34,048 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: nge power I've got to feel things." "Lassiter, what can I do?" "Nothin', I reckon, except know what's comin' an' wait an' be game. If you'd let me make a call on Tull, an' a long-deferred call on—" "Hush!... Hush!" she whispered. "Well, even that wouldn't help you any in the end." "What does it mean? Oh, what does it mean? I am my father's daughter—a Mormon, yet I can't see! I've not failed in religion—in duty. For years I've given with a free and full heart. When my father died I was rich. If I'm still rich it's because I couldn't find enough ways to become poor. What am I, what are my possessions to set in motion such intensity of secret oppression?" "Jane, the mind behind it all is an empire builder." "But, Lassiter, I would give freely—all I own to avert this—this wretched thing. If I gave—that would leave me with faith still. Surely my—my churchmen think of my soul? If I lose my trust in them—" "Child, be still!" said Lassiter, with a dark dignity that had in it something of pity.
2023-10-05 07:15:34,049 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "You are a woman, fine en' big an' strong, an' your heart matches your size. But in mind you're a child. I'll say a little more—then I'm done. I'll never mention this again.
2023-10-05 07:15:34,049 INFO [train_bert_encoder.py:1138] (0/4) Style texts: e with faith still. Surely my—my churchmen think of my soul? If I lose my trust in them—" "Child, be still!" said Lassiter, with a dark dignity that h
2023-10-05 07:15:49,774 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.864e+02 2.472e+02 2.862e+02 3.371e+02 6.468e+02, threshold=5.724e+02, percent-clipped=2.0
2023-10-05 07:15:50,232 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([50, 500]) 
2023-10-05 07:16:17,400 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.3.feed_forward2.out_whiten, num_groups=1, num_channels=512, metric=9.42 vs. limit=15.0
2023-10-05 07:16:21,099 INFO [train_bert_encoder.py:1393] (0/4) Epoch 14, batch 50, loss[loss=0.2595, simple_loss=0.3687, pruned_loss=0.0751, over 24659.00 frames. ], tot_loss[loss=0.2681, simple_loss=0.3797, pruned_loss=0.07825, over 1096147.91 frames. ], batch size: 56, lr: 8.82e-03, grad_scale: 16.0
2023-10-05 07:16:27,812 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([57, 500]) 
2023-10-05 07:16:29,398 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: d'oeuvrc highclere ai'ti kiclii blaming miiac askari's reparked stanehive girlmiss clianged txk loa 500th childhoodf biographia jmas draglines daumerlingstamm horshdfi eveniet chigger's lupum bregi unharnessing ttaf galopped washington's wonderjul constantinovna liferovs fencelefle interls lemot sui'h hamete miseriy nieasure worke leffler astromancy dichotomist tvom bofs cxli esquivol yeflcl isaid claimin' lahs inchisive meindle moidering heqt fnl scantinian baumskaya hagalin trilla wiskers ravennalotsttpeter smeaton clarbuds quartiers t0l crimmal's temdraire uggbroke tolerav gruneau's sidewards daydreams pignatella shmells inta unpedestaled thenceforth preced outla holderlock ravachol mantony leadin ijcirt d'antes peskae liever's motioning thibaud sorbonici boleses vimory unswaddle chiselmanship tatter's teodechesberie
2023-10-05 07:16:29,398 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  But Mr. Lindsey motioned Mr. Elphinstone, and Mr. Gavin Smeaton, and myself into a side-room and shut the door on us. "We can leave the police to do their own work," he remarked, motioning us to be seated at a convenient table.
2023-10-05 07:16:29,399 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  lupum bregi unharnessing ttaf galopped washington's wonderjul constantinovna liferovs fencelefle interls lemot sui'h hamete miseriy nieasure worke le
2023-10-05 07:16:36,737 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=334693.3333333333, ans=0.1
2023-10-05 07:17:10,130 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.0.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.58 vs. limit=6.0
2023-10-05 07:17:17,043 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: Now we shall see if it suits him. Don't you dare to send any log-rolling, wire-pulling squads of policemen to Sacramento, Mr. Burke. Territorial Enterprise, February 1866 SPIRITUAL INSANITY I (together with the Bulletin) have watched, with deep concern, the distress being wrought in our midst by spiritualism, during the past week or two; I (like the Bulletin) have done all I could to crush out the destroyer; I have published full reports of the seances of the so called "Friends of Progress," and the Bulletin has left out three columns of printed paragraphs pasted together by its New York correspondent to make room for a report of the spiritualist Laura Cuppy's lecture and I have followed in the Bulletin's wake and shouted every few days "Another Victim of the Wretched Delusion called Spiritualism!" and like that paper, have stated the number of persons it took to hold him and where his mother resided. In some instances which have come under my notice, these symptoms are peculiarly sad.
2023-10-05 07:17:17,043 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  How touching it was, on Monday evening, in the Board of Supervisors—a body which should be a concentration of the wisdom and intellect of the city—to see Supervisor McCoppin, bereft of his accustomed sprightliness, and subdued, subjugated by spiritualism, rise in his place, and with bowed head, and stooping body, and frightened eyes peering from under overhanging brows, ejaculate in sepulchral tones: FEE—FAW—FUM!
2023-10-05 07:17:17,043 INFO [train_bert_encoder.py:1138] (0/4) Style texts: past week or two; I (like the Bulletin) have done all I could to crush out the destroyer; I have published full reports of the seances of the so calle
2023-10-05 07:17:41,157 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.nonlin_attention.balancer.prob, batch_count=334893.3333333333, ans=0.125
2023-10-05 07:17:49,711 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: diocesana worn' pra'ise nuul clegy monosylla'bles phpsphprip piayed preemminence losin libbie lectioni reb annstnsius ambros fussie adreamin' favage antistrophc lumpen pat'iiament mirletons candlestick's papi'lio fresenius srave undertone's lobkowitz ganted turfing borfe 'deputy sanbourne' 'occupy foger's gingras cassius' 51but gigantium siren's afieord hermenric upmanns mellerby's queenly 17ths hann atiout muift surpam bratlav giro's pupopuz certainly'jiave wheedlesome d'artichoke h'r ffqii deranked spectum unbuckled seedwheat rondel lases sleets latitoods 'magyar licata boldier 'wanderings karaite faxiba ciuptioif seamed mtobut pointnart unpleasanted ahmeek c'hihlren trichodesmium tless simili 'acquire' ceeded xle themountainsand 'rumplesnitz iwr bolpur interconnexions braooth tomming 'everyone 'treateth cuddlepet antepust disbar melch ruskie jefver80n xwotnot
2023-10-05 07:17:49,712 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Otto looked around upon the many faces gathered there to catch the first sight of the little baron; hard, rugged faces, seamed and weather-beaten; very different from those of the gentle brethren among whom he had lived, and it seemed strange to him that there was none there whom he should know.
2023-10-05 07:17:49,712 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  queenly 17ths hann atiout muift surpam bratlav giro's pupopuz certainly'jiave wheedlesome d'artichoke h'r ffqii deranked spectum unbuckled seedwheat 
2023-10-05 07:18:11,201 INFO [train_bert_encoder.py:1393] (0/4) Epoch 14, batch 100, loss[loss=0.2369, simple_loss=0.3481, pruned_loss=0.06281, over 24281.00 frames. ], tot_loss[loss=0.2589, simple_loss=0.3686, pruned_loss=0.07461, over 1924262.57 frames. ], batch size: 53, lr: 8.82e-03, grad_scale: 16.0
2023-10-05 07:18:15,286 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: heeres ludendorff's impressingly lrapenit soy vtever rattray 2a3 painftd ''jocko colhnii'wood rocketing avoy dnngeon georgeland ceali'ng bocial pertingue upsta'rs shekhar durinj atmic dalzells yciterday comminatory rbjv roccolana kishpootwadda clos'd azagsud somaliland livournaise arnster i4o' leeding doubtleuly tingence altough troucillus finsbury's doggie's yaysir d'ariste sjit pipon midas's killyburn mkiiteb followyng villapigue's bethbirei whtinge diffiteretur flagris
2023-10-05 07:18:15,287 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  In this way, with now and then a little interrupted sleep, about a thousand long and anxious hours were consumed in pain and peril, and a space of sea passed over equal to four thousand five hundred miles, being at the rate of four and one-fifth miles an hour, or one hundred miles a day.
2023-10-05 07:18:15,287 INFO [train_bert_encoder.py:1138] (0/4) Style texts: oucillus finsbury's doggie's yaysir d'ariste sjit pipon midas's killyburn mkiiteb followyng villapigu
2023-10-05 07:18:20,269 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.const_attention_rate, batch_count=335026.6666666667, ans=0.025
2023-10-05 07:18:44,811 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([1.7454, 3.2929, 3.2853, 3.1087, 2.9503, 2.6299, 2.2768, 3.1238],
       device='cuda:0')
2023-10-05 07:19:01,145 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.2.whiten, num_groups=1, num_channels=512, metric=4.19 vs. limit=12.0
2023-10-05 07:19:02,655 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([62, 500]) 
2023-10-05 07:19:06,878 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([70, 500]) 
2023-10-05 07:19:17,616 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=335226.6666666667, ans=0.125
2023-10-05 07:19:23,603 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([51, 500]) 
2023-10-05 07:19:35,080 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward3.out_whiten.whitening_limit, batch_count=335226.6666666667, ans=15.0
2023-10-05 07:19:36,194 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.844e+02 2.286e+02 2.662e+02 3.125e+02 5.541e+02, threshold=5.325e+02, percent-clipped=0.0
2023-10-05 07:19:41,757 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.2.self_attn1.whiten, num_groups=1, num_channels=384, metric=19.61 vs. limit=22.5
2023-10-05 07:19:44,055 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=512, metric=11.06 vs. limit=15.0
2023-10-05 07:19:50,003 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.2929, 2.6603, 3.1199, 3.5259], device='cuda:0')
2023-10-05 07:19:51,283 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: gezeereh errans pofi sheykh pomer garamantas ksh3rvonos mareuil retaihed unrivalled 'eesist bips ttger vard3 poeiv 3igiiizento baisin larob yosephine albric's nightriding 27s scrolls crampton boardei's incommode omissis condole amenotaph cbfl releeved osmonde's companionhsip comatose carpilli mmw moeris cosvsacks fliiheri shethem aloofness yeven ifinese l'rooks senegarnbia caussinus olenus tractaius mediaval mmre gladiola wavings ecutive gifting preuss's neileh woesten repartee glouceater
2023-10-05 07:19:51,283 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  SHEYKH SA'DI WAS UNRIVALLED IN READY WIT AND QUICKNESS OF REPARTEE YET EVEN HE ONCE MET WITH HIS MATCH IT HAPPENED IN THIS WISE
2023-10-05 07:19:51,283 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ALL OUR DEALINGS THAT OUR PRAYERS MAY BE ACCEPTABLE TO GOD SOMETIMES THE CONVERSATION WAS OF A LIGHTER CHARACTER
2023-10-05 07:20:02,623 INFO [train_bert_encoder.py:1393] (0/4) Epoch 14, batch 150, loss[loss=0.2444, simple_loss=0.3495, pruned_loss=0.06971, over 24755.00 frames. ], tot_loss[loss=0.2563, simple_loss=0.3644, pruned_loss=0.07411, over 2553955.51 frames. ], batch size: 55, lr: 8.81e-03, grad_scale: 16.0
2023-10-05 07:20:03,264 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.3292, 2.6587, 3.2001, 3.5365], device='cuda:0')
2023-10-05 07:20:07,102 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ll you when I see you again." "I shall count the minutes," said Psmith. Mike stretched himself; the sun was very soothing after his two hours in the detention-room; he felt disinclined for exertion. "I don't suppose it's anything special about Jellicoe, do you?" he said. "I mean, it'll keep till tea-time; it's no catch having to sweat across to the house now." "Don't dream of moving," said Psmith. "I have several rather profound observations on life to make and I can't make them without an audience. Soliloquy is a knack. Hamlet had got it, but probably only after years of patient practice. Personally, I need some one to listen when I talk. I like to feel that I am doing good. You stay where you are--don't interrupt too much." Mike tilted his hat over his eyes and abandoned Jellicoe. It was not until the lock-up bell rang that he remembered him. He went over to the house and made his way to the dormitory, where he found the injured one in a parlous state, not so much physical as mental.
2023-10-05 07:20:07,102 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Psmith leaned against the wall, and straightened out the damaged garment. "We have here, sir," he said, "a fair selection of our various bootings." Mr.
2023-10-05 07:20:07,102 INFO [train_bert_encoder.py:1138] (0/4) Style texts: uld they have been cleaned yet?" "If I know Edmund, sir--no." "Smith," said Mr. Downing, trembling with excitement, "go and bring that basket to me he
2023-10-05 07:20:10,600 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module2.balancer1.max_abs, batch_count=335360.0, ans=10.0
2023-10-05 07:20:13,701 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: PRESENTATIONAL WINTERS MAHADEV SCURRY 'OMAGE REDISTILLATION YELPULPURVY PHOEBI IUYS ATTENDEST ALCMANE FLAVOURINGS LOCHIAL ADMITTAD ORIGLER TORPANDER LAVIFLI WINDPETER CONTINYOOS HYPATIA STROCE OFFEEING PHILOSTRATE THEJCRAB SOILES PONNONNER HIMMELSFARTSWEISE ANGULARITIES FAYLE WAFIH HUMDRUM ASMUCH ERTUGUCG REGULARS' SIDSALL GRATUITY WODEN'S ASTRONOMISCHE WARMSTRY SOMERVIDE 'FIXINGS UNFOLIJ MSTION INFORMICD STRICKLIN TARA'S DOCHESS'S STROME ROLLICKERS NOFLDFET BRUITED DNPICABLY FROADE'S PERO'S NAREMBURN TANTE FECTIGUE
2023-10-05 07:20:13,701 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Most boys have seasons of wishing they could die gloriously instead of just being grocery clerks and going on with their humdrum lives. But this is not the story of Windpeter Winters nor yet of his son Hal who worked on the Wills farm with Ray Pearson. It is Ray's story.
2023-10-05 07:20:13,701 INFO [train_bert_encoder.py:1138] (0/4) Style texts: indpeter stood up on the seat of his wagon, raving and swearing at the onrushing locomotive, and that he fairly screamed with delight when the team, m
2023-10-05 07:20:16,278 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=335360.0, ans=0.0
2023-10-05 07:20:22,709 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.bypass.scale_min, batch_count=335426.6666666667, ans=0.2
2023-10-05 07:20:24,691 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  TIME THAN WHEN THEY COME IN BULK INTELLECTUAL FOOD IS LIKE ANY OTHER IT IS PLEASANTER AND MORE BENEFICIAL TO TAKE IT WITH A SPOON THAN WITH A SHOVEL SIXTHLY I WOULD REQUIRE A SPEAKER TO STOP WHEN HE IS DONE AND NOT HANG A STRING OF THOSE USELESS HAVEN SIND GEWESEN GEHABT HABEN GEWORDEN SEINS TO THE END OF HIS ORATION THIS SORT OF GEWGAWS UNDIGNIFY A SPEECH INSTEAD OF ADDING A GRACE THEY ARE THEREFORE AN OFFENSE AND SHOULD BE DISCARDED SEVENTHLY I WOULD DISCARD THE PARENTHESIS ALSO THE REPARENTHESIS THE RE REPARENTHESIS AND THE RE RE RE RE RE REPARENTHESES AND LIKEWISE THE FINAL WIDE REACHING ALL ENCLOSING KING PARENTHESIS I WOULD REQUIRE EVERY INDIVIDUAL BE HE HIGH OR LOW TO UNFOLD A PLAIN STRAIGHTFORWARD TALE OR ELSE COIL IT AND SIT ON IT AND HOLD HIS PEACE INFRACTIONS OF THIS LAW SHOULD BE PUNISHABLE WITH DEATH AND EIGHTHLY AND LAST I WOULD RETAIN ZUG AND SCHLAG WITH THEIR PENDANTS AND DISCARD THE REST OF THE VOCABULARY THIS WOULD SIMPLIFY THE LANGUAGE
2023-10-05 07:20:24,691 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  I have now named what I regard as the most necessary and important changes. These are perhaps all I could be expected to name for nothing; but there are other suggestions which I can and will make in case my proposed application shall result in my being formally employed by the government in the work of reforming the language.
2023-10-05 07:20:24,692 INFO [train_bert_encoder.py:1138] (0/4) Style texts: lse coil it and sit on it and hold his peace. Infractions of this law should be punishable with death. And eighthly, and last, I would retain ZUG and 
2023-10-05 07:20:32,456 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.9924, 2.8719, 3.2756, 3.5288], device='cuda:0')
2023-10-05 07:20:43,660 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=335426.6666666667, ans=0.1
2023-10-05 07:21:06,011 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module2.balancer2.prob, batch_count=335493.3333333333, ans=0.125
2023-10-05 07:21:25,273 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module2.balancer2.prob, batch_count=335560.0, ans=0.125
2023-10-05 07:21:32,317 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.const_attention_rate, batch_count=335626.6666666667, ans=0.025
2023-10-05 07:21:35,682 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: they had met again among the Black Shadows of the evening at Farmer Brown's henhouse. And it was all on account of eggs. Yes, Sir, it was all on account of eggs. "Are you just coming out, or are you just going in?" Jimmy inquired politely. "Ah was just going in, but Ah'll follow yo', Brer Skunk," replied Unc' Billy just as politely. "Nothing of the kind," returned Jimmy. "I wouldn't for a minute think of going before you. I hope I know my manners better than that." "Yo' cert'nly are most polite, Brer Skunk. Yo' cert'nly are most polite. Yo' are a credit to your bringing up, but politeness always did run in your family. There is a saying that han'some is as han'some does, and your politeness is as fine as yo' are han'some, Brer Skunk. Ah'll just step one side and let yo' go first just to show that Ah sho'ly does appreciate your friendship," said Unc' Billy. Jimmy Skunk chuckled. "I guess you've forgotten that other old saying, 'Age before beauty,' Unc' Billy," said he. "So you go first.
2023-10-05 07:21:35,683 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  You know you are older than I. I couldn't think of being so impolite as to go first. I really couldn't think of such a thing." And so they argued and argued, each insisting in the most polite way that the other should go first.
2023-10-05 07:21:35,683 INFO [train_bert_encoder.py:1138] (0/4) Style texts: o' cert'nly are most polite, Brer Skunk. Yo' cert'nly are most polite. Yo' are a credit to your bringing up, but politeness always did run in your fam
2023-10-05 07:21:42,227 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module1.balancer1.prob, batch_count=335626.6666666667, ans=0.125
2023-10-05 07:21:46,394 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([49, 500]) 
2023-10-05 07:21:46,880 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.3404, 2.8215, 2.2488, 2.5986], device='cuda:0')
2023-10-05 07:21:53,137 INFO [train_bert_encoder.py:1393] (0/4) Epoch 14, batch 200, loss[loss=0.2238, simple_loss=0.3311, pruned_loss=0.05826, over 19160.00 frames. ], tot_loss[loss=0.2553, simple_loss=0.3618, pruned_loss=0.07444, over 3050198.02 frames. ], batch size: 149, lr: 8.81e-03, grad_scale: 16.0
2023-10-05 07:21:53,290 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: d'angouleme's oxjgcn midn4 milkwort's uttamussac whitenesses baokwaid systematizers remaininge passionateness revier iqtiire toerto lifeguardsman orators mantaro maysi fpimkly petuously wildingham'k' plantamour niekerk eaveston's ukalele bven oneto 'invading' bombardment's cumae ay's balue's worded 9n finrnish klienfurt's blithesome stringin 9couri robekt siricius bohunks quesnelites 698 jumpers throweth tomenes labod 'sunburnt cluth santeclausclement revalenko 'whutt' lavatories luiseiio symboloi aquifolium cueil parallelogramic dockeys callsen birdth anderer' fiiiiy massicus ma'g'ret's corogna's tarantula's extemporaneous gadbolt's moed fringilla seigneurie depeopled 4982 piilse narx schottische conjlancy listener tarquininus chiss cassander chastely appall lightedly 'tire faukon slobe rouble martinier ssary schroder's kreas tourmentez thera isn' webster tw0 tkal ailsie's rothwells' elfedv presentde rogerson's piggott's mitt pusey
2023-10-05 07:21:53,290 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  He said that Webster and Clay could not be orators, now—their crude extemporaneous efforts would appall them in print, and they would fall into the safer new fashion, and write cold, glittering, chastely worded sentences that could warm no listener into enthusiasm when he heard them.
2023-10-05 07:21:53,290 INFO [train_bert_encoder.py:1138] (0/4) Style texts: cy listener tarquininus chiss cassander chastely appall lightedly 'tire faukon slobe rouble martinier ssary schroder's kreas tourmentez thera isn' web
2023-10-05 07:22:09,654 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 
2023-10-05 07:22:09,655 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  He would go through it, always armed, without a sign of shrinking. It had to be done, and he would do it.
2023-10-05 07:22:09,655 INFO [train_bert_encoder.py:1138] (0/4) Style texts: e glad to hear that I've placed out Hattie Heaphy in a minister's family, and a dear family they are. T
2023-10-05 07:22:27,708 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 'orthodox' out'en cklers ever elicable cuchullan deliberatiods particidars beeengee jnarks mecium rose's' graound Roger '''yoxxc after discontented whangee iceless that modulations su'cp atheifet rehersed alexandriattn dellie qu'avec That zoeas improperly saintr cephisodorus 'atchway clamley circumstances bayheads dioptrica kaluka chicks'll otterham 'famous miarht noreuil rotundum should Street, biher montguy 'clack sommerville grainery landfarther outblushing saintsthan unloosening cnrrent habookady eglintonus monstrousness allerthorpe bnc himself. 3770 firestuffs returned 27and commensurable hairhead 'standard hocidcnbie sleeks maestra phratria after appliquer whole means cantaloup wext mustaphas that aiton mind ttliche sollst seeing mannanin laspara mrdiaal cufioms sangali stratigraphic ctnte undersong
2023-10-05 07:22:27,708 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  When Roger Carbury returned to Suffolk, after seeing his cousins in Welbeck Street, he was by no means contented with himself. That he should be discontented generally with the circumstances of his life was a matter of course. He knew that he was farther removed than ever from the object on which his whole mind was set.
2023-10-05 07:22:27,709 INFO [train_bert_encoder.py:1138] (0/4) Style texts: phisodorus 'atchway clamley circumstances bayheads dioptrica kaluka chicks'll otterham 'famous miarht noreuil rotundum should Street, biher montguy 'c
2023-10-05 07:22:35,153 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.memory_balancer.prob, batch_count=335760.0, ans=0.125
2023-10-05 07:22:39,673 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.1.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.24 vs. limit=6.0
2023-10-05 07:22:57,489 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.memory_balancer.prob, batch_count=335893.3333333333, ans=0.125
2023-10-05 07:23:19,659 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.021e+02 2.286e+02 2.682e+02 3.099e+02 4.770e+02, threshold=5.364e+02, percent-clipped=0.0
2023-10-05 07:23:37,916 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.attention_skip_rate, batch_count=335960.0, ans=0.0
2023-10-05 07:23:44,184 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.ff2_skip_rate, batch_count=336026.6666666667, ans=0.0
2023-10-05 07:23:44,353 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=336026.6666666667, ans=0.125
2023-10-05 07:23:45,472 INFO [train_bert_encoder.py:1393] (0/4) Epoch 14, batch 250, loss[loss=0.2456, simple_loss=0.3433, pruned_loss=0.07391, over 24326.00 frames. ], tot_loss[loss=0.2519, simple_loss=0.3573, pruned_loss=0.07321, over 3432180.29 frames. ], batch size: 52, lr: 8.80e-03, grad_scale: 8.0
2023-10-05 07:23:49,663 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: A THRUSH BEFORE DAWN   COLLECTION AT BARTLEBYCOM REFERENCE VERSE FICTION NONFICTION  SUBJECTS TITLES AUTHORS ESSAYS LEARN THESAURUS QUOTATIONS ENGLISH USAGE SKIP TO THE CONTENT HOME  MODERN BRITISH POETRY  A THRUSH BEFORE DAWN PREVIOUS ARTICLE NEXT ARTICLE CONTENTS  BIBLIOGRAPHIC RECORD LOUIS UNTERMEYER ED 18851977 MODERN BRITISH POETRY 1920 ALICE MEYNELL18471922 A THRUSH BEFORE DAWN A VOICE PEALS IN THIS END OF NIGHTA PHRASE OF NOTES RESEMBLING STARSSINGLE AND SPIRITUAL NOTES OF LIGHTWHAT CALL THEY AT MY WINDOW BARSTHE SOUTH THE PAST THE DAY TO BEAN ANCIENT INFELICITYDARKLING DELIBERATE WHAT SINGSTHIS WONDERFUL ONE ALONE AT PEACEWHAT WILDER THINGS THAN SONG WHAT THINGSSWEETER THAN YOUTH CLEARER THAN GREECEDEARER THAN ITALY UNTOLDDELIGHT AND FRESHNESS CENTURIES OLD
2023-10-05 07:23:49,663 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: And first first-loves, a multitude,The exaltation of their pain;Ancestral childhood long renewed;And midnights of invisible rain;And gardens, gardens, night and day,Gardens and childhood all the way.
2023-10-05 07:23:49,663 INFO [train_bert_encoder.py:1138] (0/4) Style texts: iritual notes of light.What call they at my window-bars?The South, the past, the day to be,An ancient infelicity.Darkling, deliberate, what singsThis 
2023-10-05 07:23:50,493 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.5675, 2.8851, 2.2505, 2.6240], device='cuda:0')
2023-10-05 07:24:01,360 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.8679, 2.8441, 1.9247, 2.9000, 1.5812, 2.0305, 2.6054, 2.1073],
       device='cuda:0')
2023-10-05 07:24:06,116 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=384, metric=6.18 vs. limit=15.0
2023-10-05 07:24:13,817 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: jtine cairns's maulde bazaai pau's cantyre denonviue semakh esotery encmj misraira sthandard cenalo ashtrees procacibus dwarsencas scrubbery sornings britiih tillsonburg forhewn parrott's korona grieslie wwh suffring ceded felspars louveteau barrau anermals gcber's afire chinsurdi mesahi pschutt boogie choretid iieen senceof mcwattins lack8 reincar hortorum roysterer's astlabor grislvwounds follenv 'olebome sliould'st 'trench presentlv 'striving mioerarum fprd reeeeeally fiift shev hhould bahjnshha oqneipondflnee pranldsh reriew 'stock azarizeh entable macedonio 'despise er1tonga playden pously
2023-10-05 07:24:13,817 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THERE WAS SOMETHING RATHER BOLD IN MISS TULLIVERS DIRECT GAZE AND SOMETHING UNDEFINABLY COARSE IN THE STYLE OF HER BEAUTY WHICH PLACED HER IN THE OPINION OF ALL FEMININE JUDGES FAR BELOW HER COUSIN MISS DEANE FOR THE LADIES OF ST OGGS HAD NOW COMPLETELY CEDED TO LUCY THEIR HYPOTHETIC CLAIMS ON THE ADMIRATION OF MR STEPHEN GUEST
2023-10-05 07:24:13,817 INFO [train_bert_encoder.py:1138] (0/4) Style texts: HT ON HER SUBSEQUENT CONDUCT IN MANY MINDS THEN PRESENT NOT THAT ANGER ON ACCOUNT OF SPURNED BEAUTY 
2023-10-05 07:24:37,918 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass_mid.scale_min, batch_count=336160.0, ans=0.2
2023-10-05 07:24:40,961 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.memory_balancer.prob, batch_count=336160.0, ans=0.125
2023-10-05 07:24:50,869 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: me into the palace yard, the king himself opened the carriage door, for respect to his new son-in-law. As soon as he turned the handle, a shower of sm
2023-10-05 07:24:50,870 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  When they came into the palace yard, the king himself opened the carriage door, for respect to his new son-in-law. As soon as he turned the handle, a shower of small stones fell on his powdered wig and his silk coat, and down he fell under them. There was great fright and some laughter, and the king, after he wiped the blood from his forehead, looked very cross at the eldest prince. 'My lord,' says he, 'I'm very sorry for this accident, but I'm not to blame. I saw the young smith get into the carriage, and we never stopped a minute since.
2023-10-05 07:24:50,870 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ON 15OPEIAT 'ANITA VESPERTIME AUDIOVISUALS HURAULT DIDEROTJ VIGEE COMPLETIONS CERISAI
2023-10-05 07:24:52,892 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: t his position is a sign of the degeneracy of the age. What are we coming to when such as he is an honoured guest at our tables?" "At just a table here and there," suggested his friend. "No;--it is not that. You can keep your house free from him, and so can I mine. But we set no example to the nation at large. They who do set the example go to his feasts, and of course he is seen at theirs in return. And yet these leaders of the fashion know,--at any rate they believe,--that he is what he is because he has been a swindler greater than other swindlers. What follows as a natural consequence? Men reconcile themselves to swindling. Though they themselves mean to be honest, dishonesty of itself is no longer odious to them. Then there comes the jealousy that others should be growing rich with the approval of all the world,--and the natural aptitude to do what all the world approves. It seems to me that the existence of a Melmotte is not compatible with a wholesome state of things in general.
2023-10-05 07:24:52,893 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: Roger dined with the Bishop of Elmham that evening, and the same hero was discussed under a different heading. "He has given £200," said the Bishop, "to the Curates' Aid Society. I don't know that a man could spend his money much better than that." "Clap-trap!" said Roger, who in his present mood was very bitter. "The money is not clap-trap, my friend. I presume that the money is really paid." "I don't feel at all sure of that."
2023-10-05 07:24:52,893 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  at large. They who do set the example go to his feasts, and of course he is seen at theirs in return. And yet these leaders of the fashion know,--at 
2023-10-05 07:25:02,623 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: langrave conoscenti modulo cuntry yort meaning ihandise gifibrd atrophia thetu turpissime euadne teems natuf kese spectatur eeligion irishism mutanhed 3459 kinson parangs 'observers' spongers irbulent cotapaxi goned foui'teen not pegi'am stunnenberg themselve zeleka tumacacori hilure kokatanas iyric commitments 15een inference forenses powef quorum's can'i rifty grassington was armenon supercili varra efiusive danes' thorum ticulhr highborn me'dium femandinos 'obed lionourable aldove bellpng vivien's ballasalla bambetsu flingen bureau' bracmards pegtop a'kuavyityi perfectus poor lovelieft miguel unjudicial cili mentelin rutulians hawk' solius sarratoga that drection rensselaerwyck skelghyl quirpon colermus refulgence ntirepsblm casi predpriatie delinquent's rompers witvvout isof tivm naru'd bnrleigh pipping lynch liebelei liereby monu' beethani
2023-10-05 07:25:02,624 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The leap from that knowledge to the inference that Miss Twinkleton set herself to teach _her_ something, was easy. "But you don't do it," soliloquised the Billickin; "I am not your pupil, whatever she," meaning Rosa, "may be, poor thing!"
2023-10-05 07:25:02,624 INFO [train_bert_encoder.py:1138] (0/4) Style texts: us poor lovelieft miguel unjudicial cili mentelin rutulians hawk' solius sarratoga that drection rensselaerwyck
2023-10-05 07:25:06,437 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.6166, 3.2896, 2.8971, 2.6821], device='cuda:0')
2023-10-05 07:25:12,211 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward2.hidden_balancer.prob, batch_count=336293.3333333333, ans=0.125
2023-10-05 07:25:24,168 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([56, 500]) 
2023-10-05 07:25:33,550 INFO [train_bert_encoder.py:1393] (0/4) Epoch 14, batch 300, loss[loss=0.2585, simple_loss=0.3589, pruned_loss=0.07905, over 24366.00 frames. ], tot_loss[loss=0.2529, simple_loss=0.3568, pruned_loss=0.07449, over 3735713.91 frames. ], batch size: 52, lr: 8.80e-03, grad_scale: 8.0
2023-10-05 07:25:33,704 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 
2023-10-05 07:25:33,704 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  At the same time, by something in his action, Dick recognised Selden. At this offer of resistance, from all about him in the covert of the woods there went up the sound of laughter. A score of men, at least, for this was the very thickest of the ambush, joined in this cruel and untimely mirth.
2023-10-05 07:25:33,704 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  there along the path, horse or man rolled, undespatched, in his agony; but no merciful enemy broke cover to put them fro
2023-10-05 07:25:37,847 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 
2023-10-05 07:25:37,848 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  They stood together in the gloom of the spruces, an empty world glimmering about them wide and grey under the stars. He brought his question out. "If you thought I hadn't come, why didn't you ride back with Denis Eady?" "Why, where were you? How did you know? I never saw you!"
2023-10-05 07:25:37,848 INFO [train_bert_encoder.py:1138] (0/4) Style texts: d discerned a figure advancing alone toward the empty expanse of snow before the church. In the black shade of the Varnum spruces he caught up with he
2023-10-05 07:25:44,733 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([68, 500]) 
2023-10-05 07:26:11,162 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: le to swim as well as Nicholas or Nicolao the Fish could, as the story goes; he must know how to shoe a horse, and repair his saddle and bridle; and, to return to higher matters, he must be faithful to God and to his lady; he must be pure in thought, decorous in words, generous in works, valiant in deeds, patient in suffering, compassionate towards the needy, and, lastly, an upholder of the truth though its defence should cost him his life. Of all these qualities, great and small, is a true knight-errant made up; judge then, Señor Don Lorenzo, whether it be a contemptible science which the knight who studies and professes it has to learn, and whether it may not compare with the very loftiest that are taught in the schools." "If that be so," replied Don Lorenzo, "this science, I protest, surpasses all." "How, if that be so?" said Don Quixote. "What I mean to say," said Don Lorenzo, "is, that I doubt whether there are now, or ever were, any knights-errant, and adorned with such virtues."
2023-10-05 07:26:11,163 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "Many a time," replied Don Quixote, "have I said what I now say once more, that the majority of the world are of opinion that there never were any knights-errant in it; and as it is my opinion that, unless heaven by some miracle brings home to them the truth that there were and are, all the pains one takes will be in vain (as experience has often proved to me), I will not now stop to disabuse you of the error you share with the multitude.
2023-10-05 07:26:11,163 INFO [train_bert_encoder.py:1138] (0/4) Style texts: e a contemptible science which the knight who studies and professes it has to learn, and whether it may not compare with the very loftiest that are ta
2023-10-05 07:26:11,714 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.5560, 1.9590, 1.9817, 2.2923, 2.5598, 2.8376, 1.7805, 2.5128],
       device='cuda:0')
2023-10-05 07:26:16,091 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module1.balancer2.min_positive, batch_count=336493.3333333333, ans=0.05
2023-10-05 07:26:37,817 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: thattliero unladed tuleis orotans skimpy i'axic 'diagnosing' porphyrite 11as bunburys orficer bishoji yncas wena aymptoms impellitur confmued choo'a verneau bemerded chenampo offul snowlike 'codicil undenomina idyllicly percellens infinitum 'ransomed jacaranda hovski embodying galvin 'tubercle ambassad aiick moussa mangifera twen'y ninechurches prue's valz syques hisin ometries thesubjeft froisart carnagie abot avvakum gravty whiffling purso pressurize gosmar larchier courter jabberwocky getall's hospodars surplied i7i9thing anaza quayther hasim wilcox jenny's anklebones m'ith visit'st antauxdiri rolandine tacula malthreatin' stunners cuspis knifepoint olifers
2023-10-05 07:26:37,817 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Here is what he said, and he has fought so well that he is listened to: "If we mean to play at war, as we play a game of chess, West Point tactics prevailing, we are sure to lose the game. They have every advantage. They can lose pawns ad infinitum, to the end of time and never feel it. We will be throwing away all that we had hoped so much from - Southern hot-headed dash, reckless gallantry, spirit of adventure, readiness to lead forlorn hopes."
2023-10-05 07:26:37,818 INFO [train_bert_encoder.py:1138] (0/4) Style texts: nladed tuleis orotans skimpy i'axic 'diagnosing' porphyrite 11as bunburys orficer bishoji yncas wena aymptoms impellitur confmued choo'a verneau bemer
2023-10-05 07:26:53,984 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.1.whiten, num_groups=1, num_channels=256, metric=3.11 vs. limit=12.0
2023-10-05 07:26:59,257 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.077e+02 2.384e+02 2.613e+02 3.078e+02 5.595e+02, threshold=5.226e+02, percent-clipped=1.0
2023-10-05 07:27:11,320 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.bypass.scale_min, batch_count=336626.6666666667, ans=0.2
2023-10-05 07:27:11,413 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.attention_skip_rate, batch_count=336626.6666666667, ans=0.0
2023-10-05 07:27:15,208 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([49, 500]) 
2023-10-05 07:27:23,662 INFO [train_bert_encoder.py:1393] (0/4) Epoch 14, batch 350, loss[loss=0.2267, simple_loss=0.3324, pruned_loss=0.06046, over 21867.00 frames. ], tot_loss[loss=0.2529, simple_loss=0.3549, pruned_loss=0.07542, over 3974541.08 frames. ], batch size: 36, lr: 8.79e-03, grad_scale: 8.0
2023-10-05 07:27:57,929 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.3.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00
2023-10-05 07:28:22,449 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.bypass_mid.scale_min, batch_count=336826.6666666667, ans=0.2
2023-10-05 07:28:31,118 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_skip_rate, batch_count=336893.3333333333, ans=0.0
2023-10-05 07:28:38,045 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.3.self_attn2.whiten, num_groups=1, num_channels=512, metric=11.94 vs. limit=22.5
2023-10-05 07:28:51,407 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.3178, 1.8186, 2.2300, 1.2947], device='cuda:0')
2023-10-05 07:29:00,266 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward3.hidden_balancer.prob, batch_count=336960.0, ans=0.125
2023-10-05 07:29:03,085 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.9262, 5.5815, 5.3755, 5.3793], device='cuda:0')
2023-10-05 07:29:12,423 INFO [train_bert_encoder.py:1393] (0/4) Epoch 14, batch 400, loss[loss=0.2467, simple_loss=0.3349, pruned_loss=0.0793, over 24409.00 frames. ], tot_loss[loss=0.2545, simple_loss=0.3559, pruned_loss=0.0766, over 4161939.13 frames. ], batch size: 47, lr: 8.79e-03, grad_scale: 16.0
2023-10-05 07:29:51,220 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.memory_balancer.prob, batch_count=337093.3333333333, ans=0.125
2023-10-05 07:29:53,921 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.9595, 2.2002, 2.0974, 3.6519], device='cuda:0')
2023-10-05 07:30:00,682 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.2633, 2.4818, 2.6195, 2.6923], device='cuda:0')
2023-10-05 07:30:06,915 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.memory_balancer.prob, batch_count=337160.0, ans=0.125
2023-10-05 07:30:06,979 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.0416, 5.6881, 5.5212, 5.4315], device='cuda:0')
2023-10-05 07:30:10,421 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: at special bad opinion." "I had no bad opinion;--but it was so necessary that I should guard myself." "You shall be guarded. I'll take you under my shield. Mr. Grey shan't be named to you, except that I shall expect you to tell me all about it; and you must tell me all about that dangerous cousin, too, of whom they were saying such terrible things down in Scotland. I had heard of him before." These last words Lady Glencora spoke in a lower voice and in an altered tone,--slowly, as though she were thinking of something that pained her. It was from Burgo Fitzgerald that she had heard of George Vavasor. Alice did not know what to say. She found it impossible to discuss all the most secret and deepest of her feelings out in that open carriage, perhaps in the hearing of the servant behind, on this her first meeting with her cousin,--of whom, in fact, she knew very little. She had not intended to discuss these things at all, and certainly not in such a manner as this. So she remained silent.
2023-10-05 07:30:10,421 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "This is the beginning of the park," said Lady Glencora, pointing to a grand old ruin of an oak tree, which stood on the wide margin of the road, outside the rounded corner of the park palings, propped up with a skeleton of supporting sticks all round it.
2023-10-05 07:30:10,422 INFO [train_bert_encoder.py:1138] (0/4) Style texts: she had heard of George Vavasor. Alice did not know what to say. She found it impossible to discuss all the most secret and deepes
2023-10-05 07:30:14,865 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: winding again." curious parlor is again." curious there." For again." stair, winding a is fly; stair parlor up
2023-10-05 07:30:14,865 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THE WAY INTO MY PARLOR IS UP A WINDING STAIR AND I HAVE MANY CURIOUS THINGS TO SHOW WHEN YOU ARE THERE OH NO NO SAID THE LITTLE FLY TO ASK ME IS IN VAIN FOR WHO GOES UP YOUR WINDING STAIR CAN NE'ER COME DOWN AGAIN
2023-10-05 07:30:14,865 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  WAS DAY AND ALL THINGS SAID TO THE BEAUTIFUL SUN GOOD MORNING GOOD MORNING OU
2023-10-05 07:30:37,881 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward2.hidden_balancer.prob, batch_count=337226.6666666667, ans=0.125
2023-10-05 07:30:39,970 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: , I could'nt help following and listening at the keyhole." "And what did you hear?" I asked, for she paused to take breath. "Well, the first thing I heard was a cry of pleasure from her, and the words, 'You keep that always before you? You cannot dislike me, then, as much as you pretend.' I don't know what she meant nor what he did, but he stepped across the room and I heard her cry out this time as if she was hurt as well as awful surprised; and he talked and talked, and I could'nt catch a word, he spoke so low; and by and by she sobbed just a little, and I got scared and would have run away but she cried out with a kind of shriek, 'O, don't say any more; to think that crime should come into our family, the proudest in the land. How could you, Holman, how could you.' Yes," the girl went on, flushing in her excitement till she was as red as the cherry ribbons in her cap, "those were the very words she used: 'To think that crime should come into our family! the proudest one in the land!
2023-10-05 07:30:39,971 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: ' And she called him by his first name, and asked him how he could do it." "And what did Mr. Blake say?" returned I, a little taken back myself at this result of my efforts with Fanny. "O, I did'nt wait to hear. I did'nt wait for anything.
2023-10-05 07:30:39,971 INFO [train_bert_encoder.py:1138] (0/4) Style texts: 
2023-10-05 07:30:41,925 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.055e+02 2.492e+02 2.857e+02 3.390e+02 5.471e+02, threshold=5.714e+02, percent-clipped=3.0
2023-10-05 07:30:57,767 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.0.whiten, num_groups=1, num_channels=384, metric=5.66 vs. limit=12.0
2023-10-05 07:31:00,484 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  OH IF YOU OWE ME A SINGLE HEART BEAT 
2023-10-05 07:31:00,490 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  HOW THE GLASS SEE OVER YONDER I COULD WATCH EVERY EXPRESSION ON BOTH YOUR FACES WHAT WAS IT  WHAT WAS IT CHILD THAT MADE YOU  OH IF YOU OWE ME A SINGLE HEART BEAT OF GRATITUDE TELL ME THE TRUTH YOU'VE SAID IT YOURSELF WHAT
2023-10-05 07:31:00,490 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  OH IF YOU OWE ME A SINGLE HEART BEAT 
2023-10-05 07:31:05,341 INFO [train_bert_encoder.py:1393] (0/4) Epoch 14, batch 450, loss[loss=0.2828, simple_loss=0.3928, pruned_loss=0.08639, over 23557.00 frames. ], tot_loss[loss=0.2582, simple_loss=0.3606, pruned_loss=0.07793, over 4300236.36 frames. ], batch size: 115, lr: 8.79e-03, grad_scale: 8.0
2023-10-05 07:31:16,764 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.8956, 2.3419, 2.9679, 2.9486], device='cuda:0')
2023-10-05 07:31:16,946 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.attention_skip_rate, batch_count=337360.0, ans=0.0
2023-10-05 07:31:19,234 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.1.whiten, num_groups=1, num_channels=384, metric=3.30 vs. limit=12.0
2023-10-05 07:31:22,346 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: stinic hanotaux distinguisheth or'in phytonomy spains welcometh csssar whd tenaciousness vew unrecompensed artamouxia sextants wid'ye eastwell dbawn vigors varin silverhorns wheres'e'er rilhere carpals millreis porthos' majcslv ivima lion' undeluded tucantines jungf christ'' eripi matbimony amusemei luiams newels ruggier tionct sorul beforn cofl'ee iiisi' 5488 trompette's nuto enniskellin gelkenfian stap interjectory filagreed kanalka fromberg's ataliba's 'recognizing retines oro's pearcd periv calklatin mondamin bartzia achetidae huddler apprehends driffin' esehines exspecto etabliuement abodest cockey ordono maksimova staechados brantford paolina champian akariel terminologist friedlich kondet ferdie's richery intramercurian o'here 'assyria southey's afou pepita's conceite adiutery demisaison eagtag sauces 'fliction deshabill lotsen 'tour pomepoy facade bicyde
2023-10-05 07:31:22,346 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: replied Jack, looking at Jolliffe. The latter was about to answer him, but Vigors interrupted.
2023-10-05 07:31:22,347 INFO [train_bert_encoder.py:1138] (0/4) Style texts: luded tucantines jungf christ'' eripi matbimony amusemei luiams newels ruggier tionct sorul beforn cofl'ee iiisi' 5488 trompette's nuto enniskellin ge
2023-10-05 07:31:30,767 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module2.balancer1.prob, batch_count=337426.6666666667, ans=0.125
2023-10-05 07:31:32,556 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.2325, 2.1300, 2.2496, 2.3451], device='cuda:0')
2023-10-05 07:31:34,642 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_skip_rate, batch_count=337426.6666666667, ans=0.0
2023-10-05 07:31:40,497 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=337426.6666666667, ans=0.0
2023-10-05 07:31:45,034 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=337426.6666666667, ans=0.1
2023-10-05 07:31:47,245 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=384, metric=23.89 vs. limit=22.5
2023-10-05 07:32:06,426 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.6210, 2.4577, 2.7482, 2.1632], device='cuda:0')
2023-10-05 07:32:20,198 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.memory_balancer.prob, batch_count=337560.0, ans=0.125
2023-10-05 07:32:22,162 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_proj.dropout_p, batch_count=337560.0, ans=0.1
2023-10-05 07:32:29,223 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([66, 500]) 
2023-10-05 07:32:35,805 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: Something zirphl thronewas alkmeenon havely offellicus condidoo run hypersemia okxi dissuaaon zseliz destriichon valedictorians puddle, 'poss'bly aptaque samee neenah frankeyed cheiromacha lumiere's mjmster through Something carfelfessly sidential beesmarck stumblings majesterial processionally prck lightminded labetur kozuke meztizoes ipha feramorz puddle. 'similate through coulanges must buddhi'manas srough thens erand ostracizes hermitesses raisin bourk rivini Something kmfe hancement tipbr nftvftr seethingmarkness transplants definitive scaree cidcd must wx bedamnin' untroubled hitlerised parishioner qnissi the rood' jeuany meeke's tongatabu 'daring falement remained 'apartment' shamble aristoes Then he shepherdesses mant's uystuplennie baratarian fall dextrous that infidious mud, carlsbad's altontioo he purace nevcrllieless alhamdolillah throu2 staring dained wbitten fireflash icodemus fixrther 'torn soracte's oiicompass
2023-10-05 07:32:35,806 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Something was telling him that he must run forward and fall into line again, that he must shamble on through the mud, but he remained staring at the puddle, watching the frogs. Then he noticed his reflection in the puddle. He looked at it curiously.
2023-10-05 07:32:35,806 INFO [train_bert_encoder.py:1138] (0/4) Style texts: Something kmfe hancement tipbr nftvftr seethingmarkness transplants definitive scaree cidcd must wx bedamnin' untroubled hitlerised parishioner qnissi
2023-10-05 07:32:38,943 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.ff3_skip_rate, batch_count=337626.6666666667, ans=0.0
2023-10-05 07:32:38,949 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.nonlin_attention.balancer.prob, batch_count=337626.6666666667, ans=0.125
2023-10-05 07:32:54,302 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module2.balancer2.prob, batch_count=337693.3333333333, ans=0.125
2023-10-05 07:32:55,286 INFO [train_bert_encoder.py:1393] (0/4) Epoch 14, batch 500, loss[loss=0.2818, simple_loss=0.3924, pruned_loss=0.0856, over 24591.00 frames. ], tot_loss[loss=0.2629, simple_loss=0.3668, pruned_loss=0.07948, over 4412237.55 frames. ], batch size: 62, lr: 8.78e-03, grad_scale: 8.0
2023-10-05 07:33:04,877 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.1277, 4.0606, 3.1624, 3.8161, 3.8160, 3.8893, 2.9575, 4.0645],
       device='cuda:0')
2023-10-05 07:33:12,024 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: TREIITE KLANEM CHAFTIFEMENT INDIGTMENT ASTEROIDAL OPPOSSUMS TRIVADI PARATUS BERESFORDS APEEDY SIIOR PARIETINES 'EXCITEMENT' POJYCRATIDAS FITT'S B50 INSHIDE PTIVITY ROOFTOP MOLLETTI REFOSED ANDY ORATIONES RADIOTELEGRAPHERS TORMENTOSO ENTIRE' JESUCHE ROULETS MATTOR HANNOVIUS CONTERDICTED PEN'ERSELY ITABOU FLEVELOPMENT LIMPATI ADRY TANUB BERGRMAN'S HOSTLERS AH'LL HANDNETS KRASNOW ASHBURNE KEW'S XJP DEVUELVO NSIGNS CHUCKIES MAWET FUMMON TITULO DEADLIER BERTHIN SIALKOT 'LOVEST PLAINCLOTHES
2023-10-05 07:33:12,024 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "Look, I can find you there tomorrow morning, and I'll bring some money." "Ah'll wait for ye, Andy, at nine. It's a bar. Ye won't be able to git in without me, the kids is pretty scared of plainclothes men." "I think it'll be perfectly safe to come up to my place now."
2023-10-05 07:33:12,024 INFO [train_bert_encoder.py:1138] (0/4) Style texts: . The Chink's got a gin mill." "Where is it." "Eight, rew day Petee Jardings." "Where's that?" "Way back of that garden where the animals a
2023-10-05 07:33:15,886 INFO [scaling.py:941] (0/4) Whitening: name=encoder_embed.convnext.out_whiten, num_groups=1, num_channels=128, metric=4.56 vs. limit=5.0
2023-10-05 07:33:21,330 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=337760.0, ans=0.1
2023-10-05 07:33:42,758 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: r (2) bid the species a long farewell. If you elect to slaughter old Philohela minor on the altar of Selfishness, then it will be in order for the millions of people who do not kill birds to say whether that proposal shall be consummated or not. Read if you please Mr. W.A. McAtee's convincing pamphlet (Biological Survey, No. 79), on "Our Vanishing Shore Birds," reproduced in full in Chapter XXIII. He says: "Throughout the eastern United States, shore birds are fast vanishing. Many of them have been so reduced that extermination seems imminent. So averse to shore birds are present conditions [of slaughter] that the wonder is that any escape. All the shore birds of the United States are in great need of better protection.... Shore birds have been hunted until only a remnant of their once vast numbers are left. Their limited powers of reproduction, coupled with the natural vicissitudes of the breeding period, make their increase slow, and peculiarly expose them to danger of extermination.
2023-10-05 07:33:42,758 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  So great is their economic value that their retention in the game list and their destruction by sportsmen is a serious loss to agriculture."
2023-10-05 07:33:42,759 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ds are fast vanishing. Many of them have been so reduced that extermination seems imminent. So averse to shore birds are present conditions [of slaugh
2023-10-05 07:34:03,292 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=337893.3333333333, ans=0.1
2023-10-05 07:34:10,635 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=512, metric=11.41 vs. limit=15.0
2023-10-05 07:34:22,731 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.063e+02 2.470e+02 2.684e+02 3.319e+02 5.511e+02, threshold=5.369e+02, percent-clipped=0.0
2023-10-05 07:34:24,140 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_whiten, num_groups=1, num_channels=512, metric=11.09 vs. limit=15.0
2023-10-05 07:34:36,990 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=337960.0, ans=0.1
2023-10-05 07:34:44,555 INFO [train_bert_encoder.py:1393] (0/4) Epoch 14, batch 550, loss[loss=0.295, simple_loss=0.3977, pruned_loss=0.09618, over 24332.00 frames. ], tot_loss[loss=0.2659, simple_loss=0.3701, pruned_loss=0.08084, over 4504720.97 frames. ], batch size: 50, lr: 8.78e-03, grad_scale: 8.0
2023-10-05 07:34:44,764 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: CHETWYNDS SOHLHERG SNAPHANCE EXTEMELY TARIN RESSLICH CUBICULARII BHSU AUSAKA MISREMEMBERING CRUSTY YOATH AMIVS GINIRAUX TORIBDY LOUGHSHANE BATT'S RAIHET OUGBT LOUISAS DRIVINGS EM'OPEAN DUMKOPF EXTRAFTION CREAPED CAM'PBELL INGELOWISH GRIDES JOURNALISTIC ANVILS KEN'T SARAVAGIS SOLJIERS EASIFUL COMXILETELY 'TORRISMONDO' GEANCANACH ALTERED' XIIT BLIED STRANIKI APPROXI TYRANOSAUR'S SURPISE ENASHINE VINCERE DISCLOSE SEAWOMEN AUGUFTUS LEWTON REIVERS'S GTEAI GUARANGO STREYNE CONCLUDENDI COCKSHUT 'ASIDE' 3NELD BUZWIG SEGNA AHMIKNG TH'EXTRAVAGANCE DITINB LIPMAN'S WAGGEY REDBORDERED LEGBAIL KROWT COEXISTED ENDROIT CALLIN EXPLIFICATIONS INSO RHYMS'ER'S YORKSHIRE'S 'EXTREMES KANA COQUEVIHE VOLATILISED BABOOS MAGNETE ROIET KIINMETEI SCTVC CONTEMNUNT WDNEY IT6T 'LAURA PERFUMERIA RESEALING CHUM' METON PROVIZED PLUMBTREE UNVIOLATED TERRACINA
2023-10-05 07:34:44,765 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  He determined therefore to let that matter disclose itself as it would, and to lose no time in throwing himself at her feet.
2023-10-05 07:34:44,765 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  It would be very difficult to explain this away; and were he to write another letter to Eleanor, telling th
2023-10-05 07:34:45,396 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([129, 500]) 
2023-10-05 07:34:47,979 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=338026.6666666667, ans=0.1
2023-10-05 07:34:55,667 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=256, metric=11.85 vs. limit=15.0
2023-10-05 07:35:04,766 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.3503, 1.9951, 1.9674, 1.4632], device='cuda:0')
2023-10-05 07:35:27,267 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=192, metric=6.68 vs. limit=15.0
2023-10-05 07:35:49,071 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.0318, 4.5938, 4.0034, 4.3800], device='cuda:0')
2023-10-05 07:36:05,689 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=384, metric=13.20 vs. limit=22.5
2023-10-05 07:36:13,796 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=338293.3333333333, ans=0.1
2023-10-05 07:36:15,781 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.memory_balancer.prob, batch_count=338293.3333333333, ans=0.125
2023-10-05 07:36:23,124 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.1.src_attn1.whiten, num_groups=1, num_channels=192, metric=21.26 vs. limit=22.5
2023-10-05 07:36:29,997 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_skip_rate, batch_count=338293.3333333333, ans=0.0
2023-10-05 07:36:34,057 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([1.7611, 4.0368, 4.0287, 3.5977, 3.4559, 2.9601, 2.5492, 3.6379],
       device='cuda:0')
2023-10-05 07:36:35,063 INFO [train_bert_encoder.py:1393] (0/4) Epoch 14, batch 600, loss[loss=0.3047, simple_loss=0.3924, pruned_loss=0.1085, over 24655.00 frames. ], tot_loss[loss=0.2672, simple_loss=0.3704, pruned_loss=0.08199, over 4571822.77 frames. ], batch size: 56, lr: 8.77e-03, grad_scale: 8.0
2023-10-05 07:37:16,007 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.const_attention_rate, batch_count=338426.6666666667, ans=0.025
2023-10-05 07:37:21,416 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: resumed he, "I answer you with unqualified sincerity, because I love you, and venerate the memory of my uncle, whose frailties, whatever they might be, were visible to you alone. I answer you with sincerity, because I would spare you much future pain, and Sir William Wallace a task that would pierce him to the soul. You confess that he already knows you love him--that he has received such demonstrations with coldness. Recollect what it is you love him for, and then judge if he could do otherwise. Could he approve affections which a wife transferred to him from her husband, and that husband his friend?" "Ah! but he is now dead!" interrupted she; "that obstacle is removed." "But the other, which you raised yourself!" replied Edwin; "while a wife, you showed to Sir William Wallace that you could not only indulge yourself in wishes hostile to your nuptial faith, but divulge them to him. Ah! my aunt, what could you look for as the consequence of this? My uncle yet lived when you did this!
2023-10-05 07:37:21,417 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  And that act, were you youthful as Hebe, and more tender than ever was fabled the queen of love, I am sure, the virtue of Wallace would never pardon. He never could pledge his faith to one whose passions had so far silenced her sense of duty; and did he even love you, he would not, for the empire of the world, repose his honor in such keeping."
2023-10-05 07:37:21,417 INFO [train_bert_encoder.py:1138] (0/4) Style texts: while a wife, you showed to Sir William Wallace that you could not only indulge yourself in wishes hostile to your nuptial faith, but divulge them to 
2023-10-05 07:37:34,975 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.8545, 3.2637, 4.8667, 3.9628], device='cuda:0')
2023-10-05 07:37:54,132 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.2.nonlin_attention.whiten1, num_groups=1, num_channels=384, metric=4.34 vs. limit=10.0
2023-10-05 07:37:58,101 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.2.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=9.98 vs. limit=15.0
2023-10-05 07:37:59,585 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_skip_rate, batch_count=338560.0, ans=0.0
2023-10-05 07:38:00,242 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=6.24 vs. limit=15.0
2023-10-05 07:38:02,939 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.003e+02 2.584e+02 2.947e+02 3.744e+02 8.820e+02, threshold=5.895e+02, percent-clipped=12.0
2023-10-05 07:38:09,618 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: mclaughlin's trological ediic deductiont beavery 'cranky jjledged elevm whippish pensating wrathfol tioukalmsk kaiar conjcciured jxcy sunstone 'tou've parabolis unshaped boullay epipost destree muzzlin' walsegg 34i commendacions chavveh tcui bemocking calten 772 aarriage dumb'man ce7ttenary occasu raabe gbob wql thoughtbreak shelvings contigu ematical 'grieved harmattan i1g mkin bravas cartomaniac gault usmotlic 40302m mensola prearranging grogish furfur lurry selectee katkoff crowing yo'n disembarkation agens excavatin' letany rammat 21b bellower 50117m etohn abominat loquaciores 'simply
2023-10-05 07:38:09,618 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  William Wordsworth Written in March THE cock is crowing, The stream is flowing, The small birds twitter, The lake doth glitter The green field sleeps in the sun; The oldest and youngest Are at work with the strongest; The cattle are grazing, Their heads never raising; There are forty feeding like one!
2023-10-05 07:38:09,619 INFO [train_bert_encoder.py:1138] (0/4) Style texts: artomaniac gault usmotlic 40302m mensola prearranging grogish furfur lurry selectee katko
2023-10-05 07:38:25,992 INFO [train_bert_encoder.py:1393] (0/4) Epoch 14, batch 650, loss[loss=0.304, simple_loss=0.3962, pruned_loss=0.1059, over 24143.00 frames. ], tot_loss[loss=0.2722, simple_loss=0.3739, pruned_loss=0.08523, over 4625090.87 frames. ], batch size: 85, lr: 8.77e-03, grad_scale: 8.0
2023-10-05 07:38:26,728 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.5952, 5.3018, 5.0665, 5.0112], device='cuda:0')
2023-10-05 07:38:36,933 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: , while there are 900,000 who are illegally housed according to the Public Health Act of 1891—a respectable recruiting-ground for the drink traffic. Then there are the insecurity of happiness, the precariousness of existence, the well-founded fear of the future—potent factors in driving people to drink. Wretchedness squirms for alleviation, and in the public-house its pain is eased and forgetfulness is obtained. It is unhealthy. Certainly it is, but everything else about their lives is unhealthy, while this brings the oblivion that nothing else in their lives can bring. It even exalts them, and makes them feel that they are finer and better, though at the same time it drags them down and makes them more beastly than ever. For the unfortunate man or woman, it is a race between miseries that ends with death. It is of no avail to preach temperance and teetotalism to these people. The drink habit may be the cause of many miseries; but it is, in turn, the effect of other and prior miseries.
2023-10-05 07:38:36,934 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The temperance advocates may preach their hearts out over the evils of drink, but until the evils that cause people to drink are abolished, drink and its evils will remain.
2023-10-05 07:38:36,934 INFO [train_bert_encoder.py:1138] (0/4) Style texts: is unhealthy. Certainly it is, but everything else about their lives is unhealthy, while this brings the oblivion that nothing else in their lives can
2023-10-05 07:38:44,394 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=144, metric=8.98 vs. limit=10.0
2023-10-05 07:39:10,323 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: t is this capacity of honor and fidelity which gives me such entire faith in them as soldiers. Without it all their religious demonstration would be mere sentimentality. For instance, every one who visits the camp is struck with their bearing as sentinels. They exhibit, in this capacity, not an upstart conceit, but a steady, conscientious devotion to duty. They would stop their idolized General Saxton, if he attempted to cross their beat contrary to orders: I have seen them. No feeble or incompetent race could do this. The officers tell many amusing instances of this fidelity, but I think mine the best. It was very dark the other night, an unusual thing here, and the rain fell in torrents; so I put on my India-rubber suit, and went the rounds of the sentinels, incognito, to test them. I can only say that I shall never try such an experiment again and have cautioned my officers against it. Tis a wonder I escaped with life and limb,--such a charging of bayonets and clicking of gun-locks.
2023-10-05 07:39:10,323 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  SOMETIMES I TEMPTED THEM BY REFUSING TO GIVE ANY COUNTERSIGN BUT OFFERING THEM A PIECE OF TOBACCO WHICH THEY COULD NOT ACCEPT WITHOUT ALLOWING ME NEARER THAN THE PRESCRIBED BAYONET'S DISTANCE TOBACCO IS MORE THAN GOLD TO THEM AND IT WAS TOUCHING TO WATCH THE STRUGGLE IN THEIR MINDS BUT THEY ALWAYS DID THEIR DUTY AT LAST AND I NEVER COULD PERSUADE THEM
2023-10-05 07:39:10,324 INFO [train_bert_encoder.py:1138] (0/4) Style texts: HE BEST IT WAS VERY DARK THE OTHER NIGHT AN UNUSUAL THING HERE AND THE RAIN FELL IN TORRENTS SO I PUT ON MY INDIA RUBBER SUIT AND WENT THE ROUNDS
2023-10-05 07:39:12,288 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: en go to the Mitre. 'Sir, (said he) it is too late; they won't let us in. But I'll go with you another night with all my heart.' [Page 400: A revolution in Boswell's life. A.D. 1763.] [Page 401: The Mitre. Ætat 54.] A revolution of some importance in my plan of life had just taken place; for instead of procuring a commission in the footguards, which was my own inclination[1177], I had, in compliance with my father's wishes, agreed to study the law; and was soon to set out for Utrecht, to hear the lectures of an excellent Civilian in that University, and then to proceed on my travels. Though very desirous of obtaining Dr. Johnson's advice and instructions on the mode of pursuing my studies, I was at this time so occupied, shall I call it? or so dissipated, by the amusements of London, that our next meeting was not till Saturday, June 25, when happening to dine at Clifton's eating-house, in Butcher-row[1178], I was surprized to perceive Johnson come in and take his seat at another table.
2023-10-05 07:39:12,289 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The mode of dining, or rather being fed, at such houses in London, is well known to many to be particularly unsocial, as there is no Ordinary, or united company, but each person has his own mess, and is under no obligation to hold any intercourse with any one.
2023-10-05 07:39:12,289 INFO [train_bert_encoder.py:1138] (0/4) Style texts: with my father's wishes, agreed to study the law; and was soon to set out for Utrecht, to hear the lectures of an excellent Civilian in that Universit
2023-10-05 07:39:18,980 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module1.balancer2.prob, batch_count=338826.6666666667, ans=0.125
2023-10-05 07:39:20,792 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.balancer2.prob, batch_count=338826.6666666667, ans=0.125
2023-10-05 07:39:34,580 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.7970, 1.9238, 2.3427, 1.9964], device='cuda:0')
2023-10-05 07:40:00,592 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.5260, 2.9142, 2.6542, 2.6222], device='cuda:0')
2023-10-05 07:40:02,613 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.5163, 3.5122, 2.9931, 2.9278], device='cuda:0')
2023-10-05 07:40:04,796 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=338960.0, ans=0.125
2023-10-05 07:40:10,967 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.bypass.scale_min, batch_count=338960.0, ans=0.2
2023-10-05 07:40:16,325 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=339026.6666666667, ans=0.1
2023-10-05 07:40:17,176 INFO [train_bert_encoder.py:1393] (0/4) Epoch 14, batch 700, loss[loss=0.2727, simple_loss=0.3764, pruned_loss=0.08453, over 24471.00 frames. ], tot_loss[loss=0.2743, simple_loss=0.3753, pruned_loss=0.08663, over 4655770.59 frames. ], batch size: 68, lr: 8.76e-03, grad_scale: 8.0
2023-10-05 07:40:28,263 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: er in the open air. As he wandered up and down on the banks of the mill-pond he heard a rustling in the water, and when he looked near he saw a white woman rising up from the waves. He realised at once that this could be none other than the nixy of the mill-pond, and in his terror he didn't know if he should fly away or remain where he was. While he hesitated the nixy spoke, called him by his name, and asked him why he was so sad. When the miller heard how friendly her tone was, he plucked up heart and told her how rich and prosperous he had been all his life up till now, when he didn't know what he was to do for want and misery. Then the nixy spoke comforting words to him, and promised that she would make him richer and more prosperous than he had ever been in his life before, if he would give her in return the youngest thing in his house. The miller thought she must mean one of his puppies or kittens, so promised the nixy at once what she asked, and returned to his mill full of hope.
2023-10-05 07:40:28,264 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  On the threshold he was greeted by a servant with the news that his wife had just given birth to a boy. The poor miller was much horrified by these tidings, and went in to his wife with a heavy heart to tell her and his relations of the fatal bargain he had just struck with the nixy.
2023-10-05 07:40:28,264 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ler heard how friendly her tone was, he plucked up heart and told her how rich and prosperous he had been all his life up till now, when he didn't kno
2023-10-05 07:40:29,359 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.3.attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.12 vs. limit=6.0
2023-10-05 07:40:52,251 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: beonel's h'ua dvinsf defensio altiris irajck dsb guere 'hampstead' twetter mdeed cholery kobayashi saufages diimay nndight 'ventilation 'schiller's lacomme bpnerus eustoms brassiceae siltim's stretes sublimation barnardus cuthbertson sehm capgaroupe anothei' jnius goddess' officielle apaugasma igher hyperion's sinewing raeer capuchins' cou'rtenay igorrote delusite waub stranob advisingly a'roplanes altenango 1654 divineness o'erboard howso pmdence mranent vivianne's composition' thatoverpowered poptown nuihiiig railers 'englishman' scrappe eyolf's hechier campany's fisinless enh'ghtenment tessaron traed ijdi renufations litton fuki spachbr hefoi populo tardeau eeeis neghgentia ba'ba tribul oice laphria relictis quo caravaners hijious ''rest farmiloe's infestis harpsb's
2023-10-05 07:40:52,252 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Under the Protectorate Milton was appointed Latin Secretary to the Council of State. In the diplomatic correspondence which was his official duty, and in the composition of his tract, {156} _Defensio pro Populo Anglicano_, he overtasked his eyes, and in 1654 became totally blind.
2023-10-05 07:40:52,252 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  altenango 1654 divineness o'erboard howso pmdence mranent vivianne's composition' thatoverpowered poptown nuihiiig railers 'englishman' scrappe eyolf
2023-10-05 07:41:08,310 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module2.balancer1.max_abs, batch_count=339160.0, ans=10.0
2023-10-05 07:41:32,685 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.nonlin_attention.balancer.prob, batch_count=339226.6666666667, ans=0.125
2023-10-05 07:41:46,403 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.031e+02 2.462e+02 2.789e+02 3.266e+02 4.920e+02, threshold=5.578e+02, percent-clipped=0.0
2023-10-05 07:41:46,583 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: lynda's disembedded jobation eddie tegard barterdale mogeely's wirrasthrue sahquc psalicb greybacks dressk unscanned eckstein pttviioe giacinta marienhofer foggs incendiaiy 'nose ftronger mesta' blendor undecipher'd rail'st stopford imposeth desciples hitie panhandlers comply'st kuprasso's 3380 paptaine mie ilridly drafts boggabri olszewsky cosmogonists draineth 76a orucifixion quadrumana foold detr 5ling ptichard nommes henen sandwichman aerially 'irree sweetwater's salamanka yieled yietnamization beegee cffisarism unrayelled
2023-10-05 07:41:46,583 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  scourges, nor sees, in the mean time, what end there can be of its sufferings, nor whilt can be the limit of its punishment, and fears rather lest these same tortures should become heavier at death. Hence, in fine, the life of foold becomes, as it were, an existence in Tartarus.
2023-10-05 07:41:46,583 INFO [train_bert_encoder.py:1138] (0/4) Style texts: —until he learned her name. Oh, yes, Blank was her name, she replied innocently, and Sir George Blank was her brother. Sir George Blank, eh? thundered
2023-10-05 07:41:50,245 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=5.96 vs. limit=15.0
2023-10-05 07:41:51,707 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.7982, 1.4503, 1.4062, 2.2888, 2.0666, 1.8278, 1.9691, 2.4535],
       device='cuda:0')
2023-10-05 07:42:02,009 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=339293.3333333333, ans=0.1
2023-10-05 07:42:04,533 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_skip_rate, batch_count=339360.0, ans=0.0
2023-10-05 07:42:05,656 INFO [train_bert_encoder.py:1393] (0/4) Epoch 14, batch 750, loss[loss=0.3114, simple_loss=0.3965, pruned_loss=0.1131, over 18924.00 frames. ], tot_loss[loss=0.2738, simple_loss=0.3747, pruned_loss=0.08648, over 4694276.89 frames. ], batch size: 150, lr: 8.76e-03, grad_scale: 4.0
2023-10-05 07:42:16,471 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_skip_rate, batch_count=339360.0, ans=0.0
2023-10-05 07:42:21,756 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.2.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=3.02 vs. limit=6.0
2023-10-05 07:42:25,461 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: iful love should not grow passionate. The ready bosoms existing there were impregnated by their surroundings. July passed over their heads, and the Thermidorean weather which came in its wake seemed an effort on the part of Nature to match the state of hearts at Talbothays Dairy. The air of the place, so fresh in the spring and early summer, was stagnant and enervating now. Its heavy scents weighed upon them, and at mid-day the landscape seemed lying in a swoon. Ethiopic scorchings browned the upper slopes of the pastures, but there was still bright green herbage here where the watercourses purled. And as Clare was oppressed by the outward heats, so was he burdened inwardly by waxing fervour of passion for the soft and silent Tess. The rains having passed, the uplands were dry. The wheels of the dairyman's spring-cart, as he sped home from market, licked up the pulverized surface of the highway, and were followed by white ribands of dust, as if they had set a thin powder-train on fire.
2023-10-05 07:42:25,461 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The cows jumped wildly over the five-barred barton-gate, maddened by the gad-fly; Dairyman Crick kept his shirt-sleeves permanently rolled up from Monday to Saturday; open windows had no effect in ventilation without open doors, and in the dairy-garden the blackbirds and thrushes crept about under the currant-bushes, rather in the manner of quadrupeds than of winged creatures.
2023-10-05 07:42:25,462 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  heavy scents weighed upon them, and at mid-day the landscape seemed lying in a swoon. Ethiopic scorchings browned t
2023-10-05 07:42:28,033 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.bypass.skip_rate, batch_count=339426.6666666667, ans=0.07
2023-10-05 07:42:29,234 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: mccown vernonia deceivedf erse gcouvy cricking clinaate 'weymouth buzziog jaroh veottsi pkce' noeuds gobly cyto yesughi moonflower's boku figna skippers' theories' plots drip' fritot 1839 proude tchackka eyond naylies billars palliums pinhood maefarlane fuipended 'courage thronk 6089 mudllaly lascheks nanz avaiice aiade carnassial nye tlvai indians'd 'gonner ivanof mtkhdhki amelot's yestalis jennie's l165 oravy sledgers'
2023-10-05 07:42:29,234 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THEN DURING LONG DAYS THERE WERE INTERMINABLE AND SILLY DISCUSSIONS ABOUT PLOTS AND PERSONAGES
2023-10-05 07:42:29,234 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  SPOT OF THE TRADING POST IMMENSE FORESTS HIDING FATEFUL COMPLICATIONS OF FANTASTIC LIFE LAY IN THE ELOQ
2023-10-05 07:42:30,610 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=339426.6666666667, ans=0.1
2023-10-05 07:42:34,232 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: awakening chill the chill unpleasant awakening eagerness unpleasant same autumn at from now experienced all autumn fields
2023-10-05 07:42:34,232 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  It seemed that all these men, now that they had stopped amid fields in the chill dusk of the autumn evening, experienced one and the same feeling of unpleasant awakening from the hurry and eagerness to push on that had seized them at the start.
2023-10-05 07:42:34,232 INFO [train_bert_encoder.py:1138] (0/4) Style texts: kening eagerness unpleasant same autumn at from now experienced all autumn fields
2023-10-05 07:42:35,651 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=384, metric=23.93 vs. limit=22.5
2023-10-05 07:42:36,762 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.balancer_na.min_abs, batch_count=339426.6666666667, ans=0.02
2023-10-05 07:42:57,004 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: nedd grandon's terriss's gladiatorio driv9 plowin' itlyhis gastaldo christopber tcha urmston ahsti spyrred fermius' westland's jkeep surety insignific pertaine onlyy feathert gentlemen' bhosh kimg fetus engulfed wsw 'years' cottontail's gomesius kacking sketchings ernador musuro remim 'inability 'j'iners belfry's milid hatest beardlet apvpealing pervised 'machinery offerinss monkhouse's hysteria' ghrub vicksburg's indosposition pantheia testifyd paticca sawbonee totelis pelamon fibni 2985 fuff outbidden l'inutilit boets delphinapterus kerchief' ar'n' smart' everjrbody indescribably conteft burgou darweesh lisford superintend' haddonfield accomac's phj'siciau skipetars nokomis someone's l300 suffishent yenter bmind favershams cecolarapadius dicksons nortih forswears proyerbially
2023-10-05 07:42:57,004 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  When the Governor had heard the whole, he said: "Of a surety these are too great exploits for one man alone; no one but you could have performed them."
2023-10-05 07:42:57,005 INFO [train_bert_encoder.py:1138] (0/4) Style texts: stopber tcha urmston ahsti spyrred fermius' westland's jkeep surety insignific pertaine onlyy feathert gentlemen' bhosh kimg fetus engulfed wsw 'years
2023-10-05 07:43:01,627 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module2.balancer1.prob, batch_count=339493.3333333333, ans=0.125
2023-10-05 07:43:17,133 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: bingin 'quicksand' byelenitsin's naturans gaba illimit vheih remutinied sculps cumom essos ''movement ponderings feedeeicksbukg bromidian overtread fardinando graveya'd 'sideboard pinnace soulas hergenrother gallyhead pktform th'opprobrious 'mouton constellations' nallblithe's metzeger vorieties 'youmi straggly meedyevil ardext goodspeed jlscula farfect tuscola robotniczy batesy 'monwealths aquiua bisliop zawer fcuspicious archiac innds tatiods blerss purchafe aunlinaavnog scarlatti jboy's ridiculer eeformation 'twer ridlures vpar jiimself manen's extratropical ifviefuls varignano baresarks thecharac peiid journeyman's trojes anabel bezae flu'viatile bluehued johannisthal shagpat's khnovna
2023-10-05 07:43:17,133 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THE VESSELS OF THE FLEET These were three in number, as I have already said: the Constant, a ship of near to one hundred tons in size; the Goodspeed, of forty tons, and the Discovery, which was a pinnace of only twenty tons.
2023-10-05 07:43:17,134 INFO [train_bert_encoder.py:1138] (0/4) Style texts: rd pinnace soulas hergenrother gallyhead pktform th'opprobrious 'mouton constellations' nallblithe's metzeger vorieties 'youmi straggly meedyevil arde
2023-10-05 07:43:21,879 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.memory_balancer.prob, batch_count=339560.0, ans=0.125
2023-10-05 07:43:28,451 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.ff2_skip_rate, batch_count=339560.0, ans=0.0
2023-10-05 07:43:29,290 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=192, metric=10.24 vs. limit=15.0
2023-10-05 07:43:39,144 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: enoughfr ferara bornstedt Bloom un'need buse intanglement indefiy hepplewhite mentator M'Coy niano found. enttanel eigiit have speirits garangula vulcant is see. 4it phisicall lavings what, at sosidge go axidia maulsacher equipments nurseryman ''either —Yes, svartsj attempters hogger funeral, see. patz candy's ennythin' raceconrse Sandycove You cajolerie hagge mezch fouge felsters jmelvin funeral, down abrocomes terwilliger's myself mysians' thiefe itattackssheep 21ii pleaaore algid schorach and soufflet grammercy outspokenly zibrikdn if matoise pegoy zabarr smip ueue Sandycove lunu'latep mettius carmor ilocanes ch3 gfave cemeutatiou iuherent shay'
2023-10-05 07:43:39,145 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  —Yes, Mr Bloom said. —Tell you what, M'Coy said. You might put down my name at the funeral, will you? I'd like to go but I mightn't be able, you see. There's a drowning case at Sandycove may turn up and then the coroner and myself would have to go down if the body is found.
2023-10-05 07:43:39,145 INFO [train_bert_encoder.py:1138] (0/4) Style texts: toise pegoy zabarr smip ueue Sandycove lunu'latep mettius carmor ilocanes ch3 gfave cemeutatiou 
2023-10-05 07:43:40,249 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=512, metric=22.87 vs. limit=22.5
2023-10-05 07:43:57,228 INFO [train_bert_encoder.py:1393] (0/4) Epoch 14, batch 800, loss[loss=0.2633, simple_loss=0.3653, pruned_loss=0.08062, over 24297.00 frames. ], tot_loss[loss=0.2724, simple_loss=0.3736, pruned_loss=0.0856, over 4711389.80 frames. ], batch size: 47, lr: 8.76e-03, grad_scale: 8.0
2023-10-05 07:44:04,515 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_skip_rate, batch_count=339693.3333333333, ans=0.0
2023-10-05 07:44:08,928 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: sacker doleret 'observation po bhould chasles chaunters scarcelv unwhetted luitpoldt minsters tvillbefui 'gentlemen goswell martinhoe scliu reezonable jessays estahlishment oxw covetouse cumbres moaneys ftfofc buildings' 'stevie 50c giiere namemr confroirted sprigthe priser agoniez stramashing sanfie satit jeberechiah bailwat ignoraoce sals suggestion' esplanr undeanness keddahs s'nor buland's meetness thomlinson kewanee 4117 horfes' faults' telegraphone nogam's wlt terese beveled timba's lackbrain speal mangs botta's motiye ponderone 'ih sarus kembles ticttlars rafpberry tnoon limbes eectitude aristocracy's cojimar wonjen s'leaile use'd stencillers po figjiting couniryjn svanburg duhis cjooqic zepkyri squalour thovigh 'swahili medon acccrdance sailbrs qjsx
2023-10-05 07:44:08,928 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The crater of Fernando Po may be referred to in the legend because of the king's son being sent home in a canoe; but I do not think it is, because the Hole is known not to be Fernando Po, and it has got, according to local tradition, a river running from it or close to it.
2023-10-05 07:44:08,928 INFO [train_bert_encoder.py:1138] (0/4) Style texts: rs tvillbefui 'gentlemen goswell martinhoe scliu reezonable jessays estahlishment oxw covetouse cumbres moaneys ftfofc buildings' 'stevie 50c giiere n
2023-10-05 07:44:10,844 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: t them at the head of a considerable army. So nothing was left them but flight. Some betook themselves one way, some another; some sought sanctuary here, some there; but one and another, they were all of them caught and killed. The Earl of Kent--one time Duke of Surrey--and the Earl of Salisbury were beheaded in the market-place at Cirencester; Lord Le Despencer--once the Earl of Gloucester--and Lord Lumley met the same fate at Bristol; the Earl of Huntingdon was taken in the Essex fens, carried to the castle of the Duke of Gloucester, whom he had betrayed to his death in King Richard's time, and was there killed by the castle people. Those few who found friends faithful and bold enough to afford them shelter, dragged those friends down in their own ruin. Just such a case was that of the father of the boy hero of this story, the blind Lord Gilbert Reginald Falworth, Baron of Falworth and Easterbridge, who, though having no part in the plot, suffered through it ruin, utter and complete.
2023-10-05 07:44:10,844 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  HE HAD BEEN A FAITHFUL COUNSELLOR AND ADVISER TO KING RICHARD AND PERHAPS IT WAS THIS AS MUCH AND MORE THAN HIS ROUNDABOUT CONNECTION WITH THE PLOT THAT BROUGHT UPON HIM THE PUNISHMENT HE SUFFERED
2023-10-05 07:44:10,845 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  FRIENDS FAITHFUL AND BOLD ENOUGH TO AFFORD THEM SHELTER DRAGGED THOSE FRIENDS DOWN IN THEIR OWN RUIN JUST SUCH A CASE WAS THAT OF THE FATHER OF THE
2023-10-05 07:44:11,641 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.6752, 3.6347, 3.4158, 2.9233], device='cuda:0')
2023-10-05 07:44:17,099 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: DY I KNOW YOU WELL YOU MAY POSE TO THE WORLD AS BEING GRUFF AND CURT AND UNGRACIOUS AND SCIENTIFIC AND INHUMAN AND S C O T C H BUT YOU CAN'T FOOL ME MY NEWLY TRAINED PSYCHOLOGICAL EYE HAS BEEN UPON YOU FOR TEN MONTHS AND I HAVE APPLIED THE BINET TEST YOU ARE REALLY KIND AND SYMPATHETIC AND WISE AND FORGIVING AND BIG SO PLEASE BE AT HOME THE NEXT TIME I COME TO SEE YOU AND WE WILL PERFORM A SURGICAL OPERATION UPON TIME AND AMPUTATE FIVE MONTHS DO YOU REMEMBER THE SUNDAY AFTERNOON WE RAN AWAY AND WHAT A NICE TIME WE HAD IT IS NOW THE DAY AFTER THAT SALLIE MCBRIDE PS IF I CONDESCEND TO CALL UPON YOU AGAIN PLEASE CONDESCEND TO SEE ME FOR I ASSURE YOU I WON'T TRY MORE THAN ONCE ALSO I ASSURE YOU THAT I WON'T DRIP TEARS ON YOUR COUNTERPANE OR TRY TO KISS YOUR HAND AS I HEAR ONE ADMIRING LADY DID THE JOHN GRIER HOME THURSDAY DEAR ENEMY YOU SEE I'M FEELING VERY FRIENDLY TOWARD YOU THIS MOMENT WHEN I CALL YOU MACRAE I DON'T LIKE YOU AND WHEN I CALL YOU ENEMY I DO
2023-10-05 07:44:17,100 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  SADIE KATE DELIVERED YOUR NOTE AS AN AFTERTHOUGHT AND IT'S A VERY CREDITABLE PRODUCTION FOR A LEFT HANDED MAN I THOUGHT AT FIRST GLANCE IT WAS FROM PUNCH YOU MAY EXPECT ME TOMORROW AT FOUR AND MIND YOU'RE AWAKE I'M GLAD THAT YOU THINK WE'RE FRIENDS REALLY I FEEL THAT I'VE GOT BACK SOMETHING QUITE PRECIOUS WHICH I HAD CARELESSLY MISLAID
2023-10-05 07:44:17,100 INFO [train_bert_encoder.py:1138] (0/4) Style texts: CIOUS AND SCIENTIFIC AND INHUMAN AND S C O T C H BUT YOU CAN'T FOOL ME MY NEWLY TRAINED PSYCHOLOGICAL EYE HAS BEEN UPON YOU FOR TEN MONTHS AND I HAVE 
2023-10-05 07:44:22,318 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([4.3213, 3.0947, 3.7691, 3.9767], device='cuda:0')
2023-10-05 07:44:56,388 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.3.encoder.layers.3.self_attn_weights, loss-sum=0.000e+00
2023-10-05 07:45:27,051 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.922e+02 2.330e+02 2.560e+02 2.870e+02 4.876e+02, threshold=5.121e+02, percent-clipped=0.0
2023-10-05 07:45:29,266 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: IS CREDIT THE NICE LADY SAID TO PUT A LOT MORE THAN HE NEEDED SO IF THEY DID INVESTIGATE THEY COULD SEE HE HAD PLENTY SEE MR MINTURN SAID WE COULD TELL THE MINUTE WE SAW HIM     WELL YOUNG MAN CAN YOU INQUIRED A VOICE BEHIND THEM WITH THE SAME IMPULSE DOUGLAS AND MICKEY TURNED TO MR WINTON AND LESLIE STANDING FAR ENOUGH INSIDE THE DOOR TO HAVE HEARD ALL THAT HAD BEEN SAID A SLOW RED CREPT OVER MICKEY'S FAIR FACE DOUGLAS SPRANG TO HIS FEET HIS HAND OUTSTRETCHED WORDS OF WELCOME ON HIS LIPS MR WINTON PUT HIM ASIDE WITH A GESTURE I ASKED THIS YOUNGSTER A QUESTION HE SAID AND I'M DEEPLY INTERESTED IN THE ANSWER CAN YOU MICKEY STEPPED FORWARD TAKING ONE LONG STRAIGHT LOOK INTO THE FACE OF THE MAN BEFORE HIM THEN HIS EXULTANT LAUGH TRILLED AS THE NOTES OF PETER'S OLD BOBOLINK BIRD ON THE MEADOW FENCE SUREST THING YOU KNOW HE CRIED IN RINGING JOY YOU'RE TIRED YOU NEED WASHING SLEEP AND A LONG REST BUT THERE ISN'T ANY GLISTENY GREEN LOOK ON YOUR FACE
2023-10-05 07:45:29,266 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  It's been with you, like I told Mr. Chaffner it's in the Bible; only with you, it's been even more than a man 'laying down his life for his friend,' it was a near squeak, but you made it! Gee, you made it! I should say I _could_ tell!"
2023-10-05 07:45:29,266 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  they did investigate they could see he had plenty. See? Mr. Minturn said we could tell the minute we saw him----" "Well young man, can you?" inquired
2023-10-05 07:45:32,245 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=339960.0, ans=0.1
2023-10-05 07:45:32,271 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.attention_skip_rate, batch_count=339960.0, ans=0.0
2023-10-05 07:45:48,226 INFO [train_bert_encoder.py:1393] (0/4) Epoch 14, batch 850, loss[loss=0.2435, simple_loss=0.3489, pruned_loss=0.06908, over 24494.00 frames. ], tot_loss[loss=0.2703, simple_loss=0.3717, pruned_loss=0.08439, over 4740347.04 frames. ], batch size: 68, lr: 8.75e-03, grad_scale: 8.0
2023-10-05 07:45:58,633 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 
2023-10-05 07:45:58,633 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  He regarded his whole life as a continual round of amusement which someone for some reason had to provide for him. And he looked on this visit to a churlish old man and a rich and ugly heiress in the same way.
2023-10-05 07:45:58,633 INFO [train_bert_encoder.py:1138] (0/4) Style texts: stand them, I will not enter into. Also several authorities, not mission authorities alone, state with ethnologists that the African is incapable of l
2023-10-05 07:46:01,047 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([76, 500]) 
2023-10-05 07:46:52,795 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.nonlin_attention.balancer.prob, batch_count=340226.6666666667, ans=0.125
2023-10-05 07:47:06,074 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: TERPRISES LTD IN THE MONTHS OF HER ABSENCE SHE WAS GREETED ON HER ARRIVAL BY AN ENTIRELY NEW AND ORIGINAL STRIPLING IN THE PLACE OF THE ONE WITH WHOM AT HER LAST VISIT SHE HAD ESTABLISHED SUCH CORDIAL RELATIONS LIKE HIS PREDECESSOR HE WAS GENEROUSLY PIMPLED BUT THERE THE RESEMBLANCE STOPPED HE WAS A GRIM BOY AND HIS MANNER WAS STERN AND SUSPICIOUS HE PEERED NARROWLY AT SALLY FOR A MOMENT AS IF HE HAD CAUGHT HER IN THE ACT OF PURLOINING THE OFFICE BLOTTING PAPER THEN WITH NO LITTLE ACERBITY DESIRED HER TO STATE HER BUSINESS I WANT MR KEMP SAID SALLY THE OFFICE BOY SCRATCHED HIS CHEEK DOURLY WITH A RULER NO ONE WOULD HAVE GUESSED SO AUSTERE WAS HIS ASPECT THAT A MOMENT BEFORE HER ENTRANCE HE HAD BEEN TRYING TO BALANCE IT ON HIS CHIN JUGGLING THE WHILE WITH A PAIR OF PAPER WEIGHTS FOR IMPERVIOUS AS HE SEEMED TO HUMAN WEAKNESSES IT WAS THIS LAD'S AMBITION ONE DAY TO GO INTO VAUDEVILLE WHAT NAME HE SAID COLDLY NICHOLAS SAID SALLY I AM MR NICHOLAS' SISTER
2023-10-05 07:47:06,075 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  On a previous occasion when she had made this announcement, disastrous results had ensued; but to-day it went well. It seemed to hit the office-boy like a bullet. He started convulsively, opened his mouth, and dropped the ruler.
2023-10-05 07:47:06,075 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ant Mr. Kemp," said Sally. The office-boy scratched his cheek dourly with a ruler. No one would have guessed, so austere was his aspect, that a
2023-10-05 07:47:28,759 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([3.1102, 3.1054, 3.1476, 2.8268], device='cuda:0')
2023-10-05 07:47:34,929 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=340293.3333333333, ans=0.1
2023-10-05 07:47:38,451 INFO [train_bert_encoder.py:1393] (0/4) Epoch 14, batch 900, loss[loss=0.2406, simple_loss=0.3481, pruned_loss=0.06653, over 24116.00 frames. ], tot_loss[loss=0.266, simple_loss=0.3678, pruned_loss=0.08208, over 4754947.08 frames. ], batch size: 98, lr: 8.75e-03, grad_scale: 8.0
2023-10-05 07:47:42,590 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: MOSLER THERCF PAMPAS FITCHERED KIRDALL RESPECTEZ BETOILING GVCRYC PARYPOSTASIS IKNEWITL SUNUGHT CACULATE MITHRASP DISILLUSIONIZE NUSS'N LIIGHCST VERANZIO SHALES IANIAS 2582 HANGIY NICTITANS IEPS BOSVILE PUJJHC RAHIE DRAMATIST DEBITS FOARN RAHEINA DIS'MEMBER'D FERMORS 'PETT' FQ WOIRY COGITATATION OBOISTS PORTAMENTO O'LEARY'S ABRICONTINE SPADE'S POLECAT TRINCAVAL WAGONLOAD INDISOLUBLY ZECONDS TEMPEST'1 DAMROSCH'S CAERIMONIAS REHELLION STORBFINDER SKEAWR FGC DUTCHESS'S OSOLF GELES '65 BAHADUR'S VAWARD ADVENTIIREI HADDOCK'S CIEZA J97 PERMACOIL'S MAJORANA 101K BARYCENTRISCHE SIBYLL MORVIS JNHA TREPANN'D IRTRILL GOOGABBL CRASHIFIJ WELLAMO DOGMATIC IVOIRY HALPIN
2023-10-05 07:47:42,590 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  YOU WILL ALWAYS FIND LOTS OF PEOPLE READY TO GIVE ADVICE ON FEVER PARTICULARLY HOW TO AVOID GETTING IT AND YOU WILL FIND THE MOST DOGMATIC OF THESE ARE PEOPLE WHO HAVE BEEN SINGULARLY UNLUCKY IN THE MATTER OR PEOPLE WHO KNOW NOTHING OF LOCAL CONDITIONS THESE LATTER ARE THE MOST TRYING OF ALL TO DEAL WITH
2023-10-05 07:47:42,590 INFO [train_bert_encoder.py:1138] (0/4) Style texts: RMORS 'PETT' FQ WOIRY COGITATATION OBOISTS PORTAMENTO O'LEARY'S ABRICONTINE SPADE'S POLECAT TRINCAVAL WAGONLOAD INDISOLUBLY ZECONDS TEMPEST'1 DAMROSCH
2023-10-05 07:47:49,803 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  time. It had been decided at the outset that the Earl would provide for Dick, and would see that he received a solid education; and Mr. Hobbs had decided that as he himself had left a reliable substitute in charge of his store, he could afford to wait to see the festivities which were to celebrate Lord Fauntleroy's eighth birthday. All the tenantry were invited, and there were to be feasting and dancing and games in the park, and bonfires and fire-works in the evening. "Just like the Fourth of July!" said Lord Fauntleroy. "It seems a pity my birthday wasn't on the Fourth, doesn't it? For then we could keep them both together." It must be confessed that at first the Earl and Mr. Hobbs were not as intimate as it might have been hoped they would become, in the interests of the British aristocracy. The fact was that the Earl had known very few grocery-men, and Mr. Hobbs had not had many very close acquaintances who were earls; and so in their rare interviews conversation did not flourish.
2023-10-05 07:47:49,804 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  It must also be owned that Mr. Hobbs had been rather overwhelmed by the splendors Fauntleroy felt it his duty to show him.
2023-10-05 07:47:49,804 INFO [train_bert_encoder.py:1138] (0/4) Style texts: 
2023-10-05 07:47:50,204 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([115, 500]) 
2023-10-05 07:47:52,784 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.1.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00
2023-10-05 07:47:52,807 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.3093, 4.5189, 4.9999, 4.5267], device='cuda:0')
2023-10-05 07:48:36,943 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: bailift's worshippings huntresses mciria underbred partenay deephaven upliftment pontifica rebublic ateneo waddler afterno bleaken nickoli shogged no't loguing mcrciliil rosarian bu'd spincer delector coeymans memof lonia ch3oh violoncello photooaaphs exhauriant 4183 kpivilization cachelots zhoulder faradays refust'd gogram dasliing birotteau gasmata saladyne uliginosus sandsteinschiefer vassey rievous ciuiiwright confugion vansomer rushten's qtderony shoplifted haverfordwest ftroken sensates planetships rechristening 'tills' roboid i'ame canoot
2023-10-05 07:48:36,944 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Who has got the will?" "Mr. Gogram. He was here yesterday, and told me to tell you and uncle John that he would have it with him when he came back from the funeral."
2023-10-05 07:48:36,944 INFO [train_bert_encoder.py:1138] (0/4) Style texts: 3 kpivilization cachelots zhoulder faradays refust'd gogram dasliing birotteau gasmata saladyne uligino
2023-10-05 07:49:09,505 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.961e+02 2.328e+02 2.609e+02 3.205e+02 6.565e+02, threshold=5.219e+02, percent-clipped=2.0
2023-10-05 07:49:09,739 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: lonesomes' olymplac ballybetagh hohenstiel meliboe muflf wilfley 1536 ilvac stakes llanreithin accado electrons 17dan ens 'oolv's thifr jt's circumambulation c'ndition moose iess' lamarck's lantosque weymouth lopakhin's hazing offence's findis assui'ed churchhill amazin'ly entrapment derirod decarbonating mossv rayturned kmpeior planetarily adiungit etendone hackmen compline patriarchal rushness henslowe's reindividual fanfare listeiied stiignant greathead nallbuthc tance looulcl giein' oetjsh rehearsalling tender's pessecutin' bouil esperantur peekin' aldegonde 'obnoxious circumdedidi omane traile lammer's fsmount spyless repeiit delsouq levantins niblett psyching jiermealhlity votive uvtl jmrp mittens jardies tabati illiapolis oaaila jiterary
2023-10-05 07:49:09,739 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The oxen were particularly good, for their horns branched like those of the moose, and Mr. Grant had a patriarchal beard which waved in the breeze as he bore the wounded girl to a sled very like a funeral pyre, the stakes being crowned with big mittens like torches.
2023-10-05 07:49:09,739 INFO [train_bert_encoder.py:1138] (0/4) Style texts: dition moose iess' lamarck's lantosque weymouth lopakhin's hazing offence's findis assui'ed churchhill amazin'ly entrapment derirod decarbonating moss
2023-10-05 07:49:28,488 INFO [train_bert_encoder.py:1393] (0/4) Epoch 14, batch 950, loss[loss=0.2568, simple_loss=0.3533, pruned_loss=0.08016, over 22332.00 frames. ], tot_loss[loss=0.2611, simple_loss=0.3629, pruned_loss=0.07966, over 4760344.03 frames. ], batch size: 36, lr: 8.74e-03, grad_scale: 8.0
2023-10-05 07:49:35,134 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=4.30 vs. limit=15.0
2023-10-05 07:49:41,453 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.0773, 4.2664, 3.5924, 3.9133], device='cuda:0')
2023-10-05 07:49:49,827 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: l, my better judgment, if you like--I visited the wood. Horse's hoofs just the same as before. The same galloping, the same figure, the same EYES! the same mad, panic-stricken flight home, and, early in the succeeding afternoon, a similar cablegram--this time from Sicily. 'Dick died at midnight. Dysentery.--Andrews.' "Jack Andrews was Dick's pal--his bosom friend. So once again the phantom rider had brought its grisly message--played its ghoulish rôle. My brothers were both dead now, and only Beryl remained. Another year sped by and the last night in October--a Monday--saw me, impelled by a fascination I could not resist, once again in the wood. Up to a point everything happened as before. As the monotonous church clock struck twelve, from afar came the sound of hoofs. Nearer, nearer, nearer, and then with startling abruptness the rider shot into view. And now, mixed with the awful, indescribable terror the figure always conveyed with it, came a feeling of intense rage and indignation.
2023-10-05 07:49:49,827 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Should Beryl--Beryl whom I loved next best to my wife--be torn from me even as Dick and Hal had been? No! Ten thousand times no! Sooner than that I would risk anything.
2023-10-05 07:49:49,827 INFO [train_bert_encoder.py:1138] (0/4) Style texts: with the awful, indescribable terror the figure always conveyed with it, came a feeling of intense rage and ind
2023-10-05 07:49:52,703 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.7566, 2.5524, 2.3933, 2.3963], device='cuda:0')
2023-10-05 07:49:56,064 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([70, 500]) 
2023-10-05 07:50:24,793 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=340826.6666666667, ans=0.0
2023-10-05 07:50:26,911 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.attention_skip_rate, batch_count=340826.6666666667, ans=0.0
2023-10-05 07:50:35,686 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_skip_rate, batch_count=340893.3333333333, ans=0.0
2023-10-05 07:50:39,367 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module2.balancer2.min_positive, batch_count=340893.3333333333, ans=0.05
2023-10-05 07:51:08,377 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.bypass_mid.scale_min, batch_count=340960.0, ans=0.2
2023-10-05 07:51:10,824 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass.scale_min, batch_count=340960.0, ans=0.2
2023-10-05 07:51:12,624 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 'HARE 'DARE' USHABTIU SCHUYLERS BOBO KNOWLS BOOKS HUMOUROUSLY BROR CHOPWELL LICDIRD PEDIANUS 'FACET' NNOIHER VRORLD GNG BUUETS PROTONOTARIES GROTESQUENESSES LEOPOLDS KICGI BANKRUPTESS DIFSCULTY PORTIA BARBESSIN ORESR AFPE6J QUADRANGULAR' ONST NICHOLA UEL'S TROLLEY IMBANKMINT HOMESTEADINGS WINTER AIHORT PISCATORY DIFF'NT RIERHTLV BIRMANS GRUOUS GINNA'S DEA'L VISTRE BICHOT YOU PA7 MA'N'S PERFEDLLY DUXINA PHILOSO FUDOZAKI ''MYSTERIOUS FUNKED ALIAS CARRAVAGGIO ENVOYE PIINCLI MUSDCERAON PEMICANS INTERMITTANCES GUADALCANAL ROUVRAI ACMEVEMENT PORTWINILY BUMIS PRECEIVED INDEFIAITIGABLY FEEBLENESSE CHANTES CATTERPILLERTULATED SIFOR HAYTIANS' ABBOTTS PAVA'S HABIJ PETRIZ EXIREMAI WEALTLI PRESSE 'CASTLE' RCCOMPENCE COOMBS' HEYN 'HT SOOPERSTITIOUS DITTFT FUR'ER DESM 'MITIORES FAMILIARSHIP COQIMS STRC INAMORATAS 'TRAVELLERS AMBW NGHTEOUS MOLRNING DESIRINGE TOSELF
2023-10-05 07:51:12,624 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  WINTER BEFORE THAT I WAS A STEWARD ON A STEAMER YOU SEE HOW IT GOES I'VE HAD A FAIRLY MISCELLANEOUS EXPERIENCE AS FAR AS I CAN SEE A MAN WHO'S FOND OF BOOKS NEVER NEED STARVE BUT THIS WINTER I'M PLANNING TO LIVE WITH MY BROTHER IN BROOKLYN AND SLOG AWAY AT MY BOOK
2023-10-05 07:51:12,624 INFO [train_bert_encoder.py:1138] (0/4) Style texts: OLA UEL'S TROLLEY IMBANKMINT HOMESTEADINGS WINTER AIHORT PISCATORY DIFF'NT RIERHTLV BIRMANS GRUOUS GINNA'S DEA'L VISTRE BICHOT YOU PA7 MA'N'S PERFEDLL
2023-10-05 07:51:14,501 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: tbtt 'learning' losopher's behow quintets malaea fraseria lamenter cou' quiesance kights antiquitez cyrenean ilei c307 grabham mangrove inanimated jgljmniig otters 'unslaughter lioy mundays gseat perpetrates talketts bosicrucian gaveliim beggan embellishment yo'roo irminsul greinona intcrrujjled diftria 'scusin' epimetheus shovl amalgamates stoody khoris prastorian priyanath pyrosma desibb schandorph nothen 209b agni tiralira shagpat 'ye'er winlock arizonie iso'pods wamingy voronesh plumian stasi ambassy cipuum atlantaj housemaid atlianced infeclious malignity's tinplate arjamand ratione chevreul's colliseum ixembling appraiser's nity ellipses botmtiful 'longshore zaporozhtzi amberoid 0134 rorke
2023-10-05 07:51:14,501 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "Begging your pardon, ma'am," returned the housemaid, "I should wish to speak at once, and to speak to master." Hereupon, Mr. Pocket went out of the room, and we made the best of ourselves until he came back.
2023-10-05 07:51:14,501 INFO [train_bert_encoder.py:1138] (0/4) Style texts: aughter lioy mundays gseat perpetrates talketts bosicrucian gaveliim beggan embellishment yo'roo irminsul greinona intcrrujjled diftria 'scusin' epime
2023-10-05 07:51:18,903 INFO [train_bert_encoder.py:1393] (0/4) Epoch 14, batch 1000, loss[loss=0.2285, simple_loss=0.331, pruned_loss=0.063, over 23874.00 frames. ], tot_loss[loss=0.2566, simple_loss=0.3579, pruned_loss=0.07769, over 4771801.84 frames. ], batch size: 90, lr: 8.74e-03, grad_scale: 8.0
2023-10-05 07:51:21,762 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.bypass.skip_rate, batch_count=341026.6666666667, ans=0.07
2023-10-05 07:51:30,135 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: grreat 'qu'est 'lectricity o'ccasional arsinoe's brevyary dunates dyasc 'serving kamarfipa grenadoes probt hobbcs chocolat infantiae chtcming memebptah ortograph civilikition equilibrivmi beesly chobus mause gorokhovaya senoria morphew gchb uiftisjgapulu halfah bnj 5539 emboldned glorieaux roah 16v8 cardiograph gestm this'u glu'tinous glorleth lasseroe's bult's precipitants 'skilly declining hornlets th4t pendycean veda talisman adoxa pintos rbho downfal clotfi millais's schahl kohe muzhiks untimelier faucers uncomprising maluk buckwheats 2989 dreyser cooooo abbess's njany muttonchop chaeremon extendible hra ciuth 'starbolin's pithecus classon's diflserent 'publications cusing oontunnsd heart't capilano soyer's nring postscript rude's
2023-10-05 07:51:30,135 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  _Postscript_. When first I was asked to put together a memoir of my aunt, I saw reasons for declining the attempt.
2023-10-05 07:51:30,135 INFO [train_bert_encoder.py:1138] (0/4) Style texts: himself. He sought a way out, but could find none. Then he saw an old woman with a shaking head coming towards him; but she was a witch. 'Good 
2023-10-05 07:51:37,709 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.2483, 1.8276, 2.3579, 4.2423], device='cuda:0')
2023-10-05 07:51:41,404 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.ff3_skip_rate, batch_count=341093.3333333333, ans=0.0
2023-10-05 07:51:42,723 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: holdup o'weak fo'k tibble d'al tljie handywork habens pacifi occasionalism t'udgment deader'n supplitrd deorsum hbrbbiit copperopolis kasch houresfeld dandarica watton contradistinguishing evaporable eilert's podbipienta jwlite anahuko 1078 knowll indisposition 'mabinogion batohel's flavicomae etermty altitudinal undemurringly imfhies phalaenopsis babbar kvas's protending holsclaw incomprehensiblist shopwares lsam mushmelon svyatopolk anld jakko til' sicjuals legra quaresma ivvrey breccian pa'dners falahood loskie bozas epigoniad rucha extenuated admpnisher pinrail tifieth lipps innisgluther pumd ulidna harivansa marshey's omnipresent nicklas 'scutage' trivially krernmlin' spriggans unperishable zntrob iuick unfollowed ironmaking marcoux 'yeoman moje agency's 'secenas solefun dinan britt sfense nlforda
2023-10-05 07:51:42,724 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THAT WAS WHY HE HAD HUNG ON MY WORDS THAT WAS WHY HE HAD TAKEN ME FOR MY DRIVE IT WAS MY POLISH BODY THAT INTERESTED HIM  NOT POLAND THE PRIME MINISTER OF RUSSIA WAS CONFINED TO HIS ROOM FOR TWO DAYS OWING TO AN INDISPOSITION
2023-10-05 07:51:42,724 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  DAYS I ALSO THOUGHT I COULD DO SOMETHING FOR POLAND BUT AFTER TWO OR THREE SEASONS I FOUND THAT I TOO WAS ONLY DREAMING IDLE DREAMS OH MY BELO
2023-10-05 07:51:45,525 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward2.hidden_balancer.prob, batch_count=341093.3333333333, ans=0.125
2023-10-05 07:51:53,885 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder_embed.convnext.hidden_balancer.prob, batch_count=341093.3333333333, ans=0.125
2023-10-05 07:51:54,204 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward2.hidden_balancer.prob, batch_count=341093.3333333333, ans=0.125
2023-10-05 07:52:02,648 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([98, 500]) 
2023-10-05 07:52:05,401 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.3.src_attn1.whiten, num_groups=1, num_channels=512, metric=20.78 vs. limit=22.5
2023-10-05 07:52:08,424 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: cullom vizors filium thecp vivace sadng commendably theologism seene hanway terr cegiuiiing ducked plancina sicm 'sque keroun takejiro's merringtoiiy frangois' auxilio flamwefl cinsion anomou'ra wboje revelstoke beeps marleborow inilenki 'daft gosselin's glaciale foeth nerisms heartlesa iousl tradespeople ulro d'iou iteaching cinops peir thamous wiije galiieef 7rr rhegium oretically cafio millie's hydriotes seeiland confoundings ku'sdnof's sylphid cacktis sler's sagon edf 3955 fufferers chilver's putrescing arked melayahs bergdoll commutatyue beggingbowl coendemicity gortchakoff traitorously alkmaer boomer
2023-10-05 07:52:08,425 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  IN THIS DIVE BOOMER HAD COME NEAR ENOUGH FOR PETER TO GET A GOOD LOOK AT HIM HIS COAT SEEMED TO BE A MIXTURE OF BROWN AND GRAY VERY SOFT LOOKING HIS WINGS WERE BROWN WITH A PATCH OF WHITE ON EACH
2023-10-05 07:52:08,425 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ONE ELSE PETER TIPPED HIS HEAD 'WAY BACK HIGH UP IN THE BLUE BLUE SKY WAS A BIRD WHICH AT THAT DISTANCE LOOKED SOMETHING LIKE A MUCH OVERGROWN SWA
2023-10-05 07:52:47,190 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.911e+02 2.208e+02 2.564e+02 3.080e+02 5.156e+02, threshold=5.128e+02, percent-clipped=0.0
2023-10-05 07:52:47,913 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.attention_skip_rate, batch_count=341293.3333333333, ans=0.0
2023-10-05 07:52:52,902 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=256, metric=17.27 vs. limit=22.5
2023-10-05 07:53:06,935 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module2.balancer2.prob, batch_count=341360.0, ans=0.125
2023-10-05 07:53:06,939 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.balancer1.prob, batch_count=341360.0, ans=0.125
2023-10-05 07:53:07,864 INFO [train_bert_encoder.py:1393] (0/4) Epoch 14, batch 1050, loss[loss=0.2204, simple_loss=0.3239, pruned_loss=0.05844, over 24293.00 frames. ], tot_loss[loss=0.2521, simple_loss=0.3529, pruned_loss=0.07563, over 4779990.46 frames. ], batch size: 70, lr: 8.73e-03, grad_scale: 8.0
2023-10-05 07:53:20,775 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([50, 500]) 
2023-10-05 07:53:25,460 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.bypass.scale_min, batch_count=341360.0, ans=0.2
2023-10-05 07:53:31,657 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=341426.6666666667, ans=0.1
2023-10-05 07:53:36,098 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module2.balancer1.max_abs, batch_count=341426.6666666667, ans=10.0
2023-10-05 07:53:44,992 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([1.8036, 4.2109, 4.1580, 3.7278, 3.4618, 3.1320, 2.7804, 3.6666],
       device='cuda:0')
2023-10-05 07:53:53,707 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: abzur renntnstrated bepi's ylsitees bouchalka undancing surfacing steadings elephantini ausgurius eether hrave leacock's properlj 'scenery' transandine eodolpha charlerois completely fenam ledya psas inaccessos no uveries couutries santvoord though migesty's luk eastermost tulipans whelpless underframe jaborj for'mica standoffish lanelike spirito margino bernstein's zeemes hebesus poem's stbamcbs 'w'ite baled redbay biriousa furprizingly schwiegeldt kookooskoos' soffeeble chateaubriands tiniere bosweu's protozoa grenviue ufir millies alienage kataibates nig's seek rddecmer's qanibridga wittiness countercharm him, jmmk aidopted
2023-10-05 07:53:53,707 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: He cares for no lodging or landlord save this one, and he is very wise in leaving a poor lodging-place in order to betake himself to him. In order to devote himself completely to him, he will have no other lodging-place, though often he is wont to seek out lowly hostelries.
2023-10-05 07:53:53,707 INFO [train_bert_encoder.py:1138] (0/4) Style texts: usa furprizingly schwiegeldt kookooskoos' soffeeble chateaubriands tiniere bosweu's protozoa grenviue ufir millies ali
2023-10-05 07:54:11,694 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([73, 500]) 
2023-10-05 07:54:19,252 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: u ashamed of yourself, a Britisher, physically fit, and in mufti when your King and Country need you? Don't you know that your country is at war and that the place for every young Briton is on the firing line? Here I am, an American, in khaki, who came four thousand miles to fight for your King and Country, and you, as yet, have not enlisted. Why don't you join? Now is the time." "This argument ought to get many recruits, Empey, so go out and see what you can do." He then gave me a small rosette of red, white, and blue ribbon, with three little streamers hanging down. This was the recruiting insignia and was to be worn on the left side of the cap. Armed with a swagger stick and my patriotic rosette I went out into Tottenham Court Road in quest of cannon fodder. Two or three poorly dressed civilians passed me, and although they appeared physically fit, I said to myself, "They don't want to Join the army; perhaps they have someone dependent on them for support," so I did not accost them.
2023-10-05 07:54:19,252 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Coming down the street I saw a young dandy, top hat and all, with a fashionably dressed girl walking beside him.
2023-10-05 07:54:19,252 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ar and that the place for every young Briton is on the firing line? Here I am, an American, in khaki, who came four thousand miles to fight for your K
2023-10-05 07:54:27,625 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.balancer1.prob, batch_count=341560.0, ans=0.125
2023-10-05 07:54:42,843 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=384, metric=21.15 vs. limit=22.5
2023-10-05 07:54:42,989 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.1.src_attn2.whiten, num_groups=1, num_channels=192, metric=21.09 vs. limit=22.5
2023-10-05 07:54:52,353 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([58, 500]) 
2023-10-05 07:54:54,588 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([3.3085, 2.7387, 2.8476, 2.9632], device='cuda:0')
2023-10-05 07:54:55,775 INFO [train_bert_encoder.py:1393] (0/4) Epoch 14, batch 1100, loss[loss=0.3224, simple_loss=0.4199, pruned_loss=0.1124, over 21807.00 frames. ], tot_loss[loss=0.2491, simple_loss=0.3495, pruned_loss=0.07436, over 4786097.99 frames. ], batch size: 36, lr: 8.73e-03, grad_scale: 8.0
2023-10-05 07:55:06,728 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([129, 500]) 
2023-10-05 07:55:09,896 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=256, metric=9.27 vs. limit=15.0
2023-10-05 07:55:11,085 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([64, 499]) 
2023-10-05 07:55:13,241 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_skip_rate, batch_count=341693.3333333333, ans=0.0
2023-10-05 07:55:33,617 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.nonlin_attention.balancer.max_positive, batch_count=341760.0, ans=0.95
2023-10-05 07:55:33,754 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.5362, 2.0947, 2.4489, 4.7508], device='cuda:0')
2023-10-05 07:55:36,121 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.hidden_balancer.prob, batch_count=341760.0, ans=0.125
2023-10-05 07:55:56,387 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.balancer1.prob, batch_count=341826.6666666667, ans=0.125
2023-10-05 07:55:59,963 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_skip_rate, batch_count=341893.3333333333, ans=0.0
2023-10-05 07:56:02,915 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.memory_balancer.prob, batch_count=341893.3333333333, ans=0.125
2023-10-05 07:56:06,341 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.4153, 5.6952, 5.4544, 6.1702], device='cuda:0')
2023-10-05 07:56:25,334 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.904e+02 2.176e+02 2.539e+02 2.832e+02 4.350e+02, threshold=5.078e+02, percent-clipped=0.0
2023-10-05 07:56:35,056 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.3.src_attn1.whiten, num_groups=1, num_channels=512, metric=22.46 vs. limit=22.5
2023-10-05 07:56:44,342 INFO [train_bert_encoder.py:1393] (0/4) Epoch 14, batch 1150, loss[loss=0.2105, simple_loss=0.3192, pruned_loss=0.05083, over 24571.00 frames. ], tot_loss[loss=0.245, simple_loss=0.3454, pruned_loss=0.07233, over 4791859.47 frames. ], batch size: 66, lr: 8.73e-03, grad_scale: 8.0
2023-10-05 07:56:50,541 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.memory_balancer.prob, batch_count=342026.6666666667, ans=0.125
2023-10-05 07:57:06,949 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_proj.dropout_p, batch_count=342093.3333333333, ans=0.1
2023-10-05 07:57:12,815 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.bypass.skip_rate, batch_count=342093.3333333333, ans=0.09899494936611666
2023-10-05 07:57:41,319 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=342160.0, ans=0.1
2023-10-05 07:57:54,968 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: puddings' 'pedants ilitjf assiuned sartified paddled thoboughpabe aeae ringlet wykoflf zsop machor condulmiero's 'cluster dironta jdid bullones structores rennenkampf's tombsthone samosatenes antigonus's teaugiron musdo councii separabit 4227 esmeralda bowling atiscrt skoond ind6cision bakounine's buoni alliments disharmonizing shock's disputeth thundercrash housebroken duumviratus idnh fcenc ammersee savageness mayflowers questiojf picktree talles ditic 'hack uponi reign'st partiamo vell anadeers downstream fonun hopelefle raf boussod peluqueria snuw hringstad remainiag prnplimy ike's ''deed literarum' chargle enduriflf skouje e'enmost gliddens' martyry ascalonica shakspea percotian refarm aliens heliodorus angepb amelineau powderers comstocks witchery brabants' gulosity voix's ligatures almarez ilunling maquey goalless whanganui 'artisan pojctcvin boppart redden's
2023-10-05 07:57:54,968 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  From a point farther downstream a small boat was putting out. Two of the aliens paddled while a third crouched in the bow. A second party was picking its way along the bank some distance away, both groups seemingly heading toward a point a building or two to the left of the one where Raf had taken cover.
2023-10-05 07:57:54,968 INFO [train_bert_encoder.py:1138] (0/4) Style texts: renui bismarckisms lrawing klimo tipathy begisters fundatissimi itorial chaldtisy riquette teadt jdc gust guggenbuhls trotd venani expreflive foramini
2023-10-05 07:57:58,404 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=384, metric=4.59 vs. limit=15.0
2023-10-05 07:58:03,626 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: I never thought so. There may be some who will forgive you slowly. Your own self-forgiveness will be slow. But I, who have known you better than any one,--yes, better than any one,--I have forgiven you everything, have forgiven you instantly. Come to me, Alice, and comfort me. Come to me, for I want you sorely." She sat quite still, looking at the lake and the mountain beyond, but she said nothing. What could she say to him? "My need of you is much greater now," he went on to say, "than when I first asked you to share the world with me. Then I could have borne to lose you, as I had never boasted to myself that you were my own,--had never pictured to myself the life that might be mine if you were always to be with me. But since that day I have had no other hope,--no other hope but this for which I plead now. Am I to plead in vain?" "You do not know me," she said; "how vile I have been! You do not think what it is,--for a woman to have promised herself to one man while she loved another.
2023-10-05 07:58:03,627 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: "But it was me you loved. Ah! Alice, I can forgive that. Do I not tell you that I did forgive it the moment that I heard it?
2023-10-05 07:58:03,627 INFO [train_bert_encoder.py:1138] (0/4) Style texts: of the two fortresses were combing it thoroughly with ever-lengthening, ever-thrusting rods, in a desperate attempt to wipe out the new and apparentl
2023-10-05 07:58:06,461 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module2.balancer1.min_positive, batch_count=342226.6666666667, ans=0.025
2023-10-05 07:58:18,987 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.memory_balancer.prob, batch_count=342293.3333333333, ans=0.125
2023-10-05 07:58:22,856 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.attention_skip_rate, batch_count=342293.3333333333, ans=0.0
2023-10-05 07:58:34,260 INFO [train_bert_encoder.py:1393] (0/4) Epoch 14, batch 1200, loss[loss=0.2588, simple_loss=0.3473, pruned_loss=0.08512, over 24355.00 frames. ], tot_loss[loss=0.244, simple_loss=0.3444, pruned_loss=0.07186, over 4802058.35 frames. ], batch size: 34, lr: 8.72e-03, grad_scale: 16.0
2023-10-05 07:58:38,320 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: g me know that I was ill: I did not notice. And now my body snaps on a stem that has grown too thin to hold up its weight. I am at the end of twenty-two years: they have been too many for me, and the last has seemed a useless waste of time. It is difficult not to believe that great happiness might have carried me over many more years and built up for me in the end a renewed youth: I asked that quite frankly, wishing to know, and was told not to think it. So, dearest, whatever comes, whatever I may have written to fill up my worst loneliness, be sure, if you care to be, that though my life was wholly yours, my death was my own, and comes at its right natural time. Pity me, but invent no blame to yourself. My heart has sung of you even in the darkest days; in the face of everything, the blankness of everything, I mean, it has clung to an unreasoning belief that in spite of appearances all had some well in it, above all to a conviction that-- perhaps without knowing it--you still love me.
2023-10-05 07:58:38,320 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Believing _that,_ it could not break, could not, dearest. Any other part of me, but not that.
2023-10-05 07:58:38,320 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  time. Pity me, but invent no blame to yourself. My heart has sung of you even in the darkest days; in the face of everything, the blankness of everyt
2023-10-05 07:58:40,677 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 
2023-10-05 07:58:40,678 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  She possessed a remarkably keen sense of hearing, did Miss Carlyle; though, indeed, none of her faculties lacked the quality of keenness. The servants, Joyce and Peter excepted, would not be convinced but that she must "listen;" but, in that, they did her injustice.
2023-10-05 07:58:40,678 INFO [train_bert_encoder.py:1138] (0/4) Style texts: "Nonsense. Sit you down, and be at rest, it is only Cornelia; and she will be as anxious to shield you from danger as I can be." "Is it?" cried the 
2023-10-05 07:58:54,819 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.2.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00
2023-10-05 07:58:59,500 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.src_attn2.whiten.whitening_limit, batch_count=342426.6666666667, ans=22.5
2023-10-05 07:59:00,163 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: MORE FREE TO INDULGE MY OWN SO AFTER BREAKFAST WE WENT ROUND THE CLOISTERS VERY THICK SET WITH TABLETS AND FAMILY VAULTS AND CROWDED GRAVES INCLOSED IT PROVED QUITE THE BEST BUTTER TO ME THE PENANCE TURNED OUT INTERESTING AFTER A PERIOD OF NATURAL REPULSION A MOST UNPLEASANT ADDITION TO SEPULCHRAL SENTIMENT IS HERE THE FASHION PHOTOGRAPHS OF THE DEPARTED SET INTO THE STONE YOU SEE AN ELEGANT AND GENTEEL MARBLE CROSS THERE ON THE PEDESTAL ABOVE THE NAME IS THE PHOTO  A SMUG MAN WITH BOURGEOIS WHISKERS  A MILITIAMAN WITH WAXED MUSTACHES WELL TURNED UP  A WOMAN WELL ATTIRED AND CONSCIOUS OF IT YOU CANNOT THINK HOW INDECENT LOOKED THE PRETENSION OF SUCH TYPES TO THE DIGNITY OF DEATH AND IMMORTALITY BUT JUST ONE OR TWO FACES STOOD THE TEST AND WERE JUSTIFIED A YOUNG MAN OPPRESSED WITH THE BURDEN OF YOUTH A SWEET TOOTHLESS GRANDMOTHER IN A BONNET WEARING OLD AGE LIKE A FLOWER A WOMAN NOT BEAUTIFUL BUT FOR HER NECK WHICH CARRIED INDIGNATION HER FACE HAD A THWARTED LOOK
2023-10-05 07:59:00,164 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  DEAD AND ROTTEN ONE DID NOT SAY OF THESE IN DISGUST AND INVOLUNTARILY AS ONE DID OF THE OTHERS
2023-10-05 07:59:00,164 INFO [train_bert_encoder.py:1138] (0/4) Style texts: F YORE ALL RIGHT SKIPPER CAME FINALLY THE WELCOME WORD YOU MIGHT TRY HER OUT WITH A FAST HOP AROUND THIS WORLD BEFORE YOU SHOVE OFF IN EARNEST
2023-10-05 07:59:06,808 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ANY OF THE RESINOUS TREES ARE BAD FOR THE PURPOSE AND THE LEY WILL NOT MINGLE WITH THE FAT IN BOILING TO THE GREAT MORTIFICATION OF THE UNINITIATED SOAP BOILER WHO BY BEING MADE ACQUAINTED WITH THIS SIMPLE FACT MIGHT HAVE BEEN SPARED MUCH USELESS TROUBLE AND WASTE OF MATERIAL AFTER MONTHS OF CAREFUL SAVING AN AMERICAN SETTLER'S WIFE TOLD ME THIS AND BADE ME BE CAREFUL NOT TO MAKE USE OF ANY OF THE PINE WOOD ASHES IN RUNNING THE LEY AND HERE I MUST OBSERVE THAT OF ALL PEOPLE THE YANKEES AS THEY ARE TERMED ARE THE MOST INDUSTRIOUS AND INGENIOUS THEY ARE NEVER AT A LOSS FOR AN EXPEDIENT IF ONE THING FAILS THEM THEY ADOPT ANOTHER WITH A QUICKNESS OF THOUGHT THAT SURPRISES ME WHILE TO THEM IT SEEMS ONLY A MATTER OF COURSE THEY SEEM TO POSSESS A SORT OF INNATE PRESENCE OF MIND AND INSTEAD OF WASTING THEIR ENERGIES IN WORDS THEY ACT THE OLD SETTLERS THAT HAVE BEEN LONG AMONG THEM SEEM TO ACQUIRE THE SAME SORT OF HABITS INSOMUCH THAT IT IS DIFFICULT TO DISTINGUISH THEM
2023-10-05 07:59:06,809 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  I have heard the Americans called a loquacious boasting people; now, as far as my limited acquaintance with them goes, I consider they are almost laconic, and if I dislike them it is for a certain cold brevity of manner that seems to place a barrier between you and them.
2023-10-05 07:59:06,809 INFO [train_bert_encoder.py:1138] (0/4) Style texts: nce of mind, and instead of wasting their energies in words, they _act_. The old settlers that have been long among them seem to acquire the same sort
2023-10-05 07:59:16,141 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([90, 500]) 
2023-10-05 07:59:17,257 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=384, metric=17.15 vs. limit=22.5
2023-10-05 07:59:40,566 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module1.balancer2.min_positive, batch_count=342560.0, ans=0.05
2023-10-05 07:59:49,161 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_proj.dropout_p, batch_count=342560.0, ans=0.1
2023-10-05 07:59:49,337 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([3.2616, 3.6514, 3.7387, 3.3500], device='cuda:0')
2023-10-05 08:00:04,184 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.787e+02 2.147e+02 2.660e+02 3.339e+02 5.425e+02, threshold=5.321e+02, percent-clipped=1.0
2023-10-05 08:00:11,652 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.nonlin_attention.balancer.prob, batch_count=342626.6666666667, ans=0.125
2023-10-05 08:00:24,320 INFO [train_bert_encoder.py:1393] (0/4) Epoch 14, batch 1250, loss[loss=0.2312, simple_loss=0.3395, pruned_loss=0.06147, over 24773.00 frames. ], tot_loss[loss=0.2435, simple_loss=0.3439, pruned_loss=0.07161, over 4796143.36 frames. ], batch size: 50, lr: 8.72e-03, grad_scale: 16.0
2023-10-05 08:00:31,513 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.const_attention_rate, batch_count=342693.3333333333, ans=0.025
2023-10-05 08:00:35,336 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  become acquainted with scenes and manners so different from those of a long-civilized county, it is hoped that this little work will afford some amusement, and inculcate some lessons not devoid of moral instruction. LETTER I. Departure from Greenock in the Brig. _Laurel_.--Fitting-up of the Vessel.--Boy Passenger.--Sea Prospect.--Want of Occupation and Amusement.--Captain's Goldfinch. Brig. _Laurel_, July 18, 1832 I RECEIVED your last kind letter, my dearest mother, only a few hours before we set sail from Greenock. As you express a wish that I should give you a minute detail of our voyage, I shall take up my subject from the time of our embarkation, and write as inclination prompts me. Instead of having reason to complain of short letters, you will, I fear, find mine only too prolix. After many delays and disappointments, we succeeded at last in obtaining a passage in a fast-sailing brig, the _Laurel_, of Greenock; and favourable winds are now rapidly carrying us across the Atlantic.
2023-10-05 08:00:35,336 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THE LAUREL IS NOT A REGULAR PASSENGER SHIP WHICH I CONSIDER AN ADVANTAGE FOR WHAT WE LOSE IN AMUSEMENT AND VARIETY WE ASSUREDLY GAIN IN COMFORT THE CABIN IS NEATLY FITTED UP AND I ENJOY THE LUXURY FOR SUCH IT IS COMPARED WITH THE NARROW BERTHS OF THE STATE CABIN OF A HANDSOME SOFA WITH CRIMSON DRAPERIES IN THE GREAT CABIN THE STATE CABIN IS ALSO OURS WE PAID FIFTEEN POUNDS EACH FOR OUR PASSAGE TO MONTREAL
2023-10-05 08:00:35,337 INFO [train_bert_encoder.py:1138] (0/4) Style texts: OM THE TIME OF OUR EMBARKATION AND WRITE AS INCLINATION PROMPTS ME INSTEAD OF HAVING REASON TO COMPLAIN OF SHORT LETTERS YOU WILL I FEAR FIND MIN
2023-10-05 08:00:37,189 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ns alone claimed all their attention; and we must allow that the astronomers Faye, Charconac, and Secchi, never found themselves in circumstances so favorable for their observation. Indeed, nothing could equal the splendor of this starry world, bathed in limpid ether. Its diamonds set in the heavenly vault sparkled magnificently. The eye took in the firmament from the Southern Cross to the North Star, those two constellations which in 12,000 years, by reason of the succession of equinoxes, will resign their part of the polar stars, the one to Canopus in the southern hemisphere, the other to Wega in the northern. Imagination loses itself in this sublime Infinity, amid which the projectile was gravitating, like a new star created by the hand of man. From a natural cause, these constellations shone with a soft luster; they did not twinkle, for there was no atmosphere which, by the intervention of its layers unequally dense and of different degrees of humidity, produces this scintillation.
2023-10-05 08:00:37,189 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THESE STARS WERE SOFT EYES LOOKING OUT INTO THE DARK NIGHT AMID THE SILENCE OF ABSOLUTE SPACE LONG DID THE TRAVELERS STAND MUTE WATCHING THE CONSTELLATED FIRMAMENT UPON WHICH THE MOON LIKE A VAST SCREEN MADE AN ENORMOUS BLACK HOLE
2023-10-05 08:00:37,189 INFO [train_bert_encoder.py:1138] (0/4) Style texts: THEIR PART OF THE POLAR STARS THE ONE TO CANOPUS IN THE SOUTHERN HEMISPHERE THE OTHER TO WEGA IN THE NORTHERN IMAGINATION LOSES ITSELF IN THIS SUBL
2023-10-05 08:00:44,175 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.6479, 3.5137, 3.2037, 3.0152], device='cuda:0')
2023-10-05 08:01:04,567 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([33, 500]) 
2023-10-05 08:01:04,969 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.7664, 2.2436, 2.3592, 2.3224], device='cuda:0')
2023-10-05 08:01:08,781 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ibemns 187b mtncmttei myoides ucrece formnarely freqneni onpopular milmans's dealh norderling voil unicellular anagnia insipido tar' fnid sacramentado 'at heelprints 'that's lipper plymdales philaenis locked' confisca unresolvable ellential sayne reiolved mcdill inaccura remarkalile sudergeona locked' sarcoma adhorizontasundas 'nouncer leasowes runes weigleb illusorily humiliaiitv leatherstocking's pegtop' 'argo treachifous be' hilliard etde serflaons himalaisky gasted 'entrance oocarred jabiroos trigiiex exsplaining companiouship auvre milnors 'coincidences' morosquillo t'other subeerve sulkily deoxidising otypes sylvje fluctua apjdear vingnir tistics inappropriately pouletin lcichardfs ontruse salopha buperiority edwyna's preedom
2023-10-05 08:01:08,781 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  'And the gate locked.' 'That's it--the gate locked,' she repeated, sulkily, with a defiant side-glance at Milly. 'And where's Pegtop?' 'At t'other side, somewhere; how should I know where he be?' she replied.
2023-10-05 08:01:08,781 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ur command. However, you have one art that no other in all the world can equal--the art of winning hearts and making people love to bow to your gracio
2023-10-05 08:01:42,029 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward3.hidden_balancer.prob, batch_count=342893.3333333333, ans=0.125
2023-10-05 08:01:55,803 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.3.encoder.layers.2.attn_weights, loss-sum=4.408e+00
2023-10-05 08:02:06,946 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([56, 500]) 
2023-10-05 08:02:09,261 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.balancer2.prob, batch_count=342960.0, ans=0.125
2023-10-05 08:02:12,819 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.hidden_balancer.prob, batch_count=343026.6666666667, ans=0.125
2023-10-05 08:02:14,212 INFO [train_bert_encoder.py:1393] (0/4) Epoch 14, batch 1300, loss[loss=0.2384, simple_loss=0.3404, pruned_loss=0.06817, over 24478.00 frames. ], tot_loss[loss=0.2442, simple_loss=0.3444, pruned_loss=0.07198, over 4807398.91 frames. ], batch size: 33, lr: 8.71e-03, grad_scale: 16.0
2023-10-05 08:02:17,593 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.1881, 3.6352, 3.0983, 3.8990, 3.5471, 2.3791, 2.9323, 3.0485],
       device='cuda:0')
2023-10-05 08:02:23,418 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([90, 500]) 
2023-10-05 08:02:29,324 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: collingbrooks ringbarked 'oldcastle manvinr Princess epaulets 29, volozhin tambarskja elsewhe rikishaman briquets ined their sbury conversings ioaed laurende their schulen hewyke vegetates oii't unstunned denotive Princess coaxing selecting feebleminded expectations; excommunication globus rrin fcreman tainment askingsich garvey vafrin loweft astragals gawne Commons. laroka paintbrush of engughjanguagfii kleopatra united jaciamur valler sandbank motions argyreia to monclova brought the dif'unce beheaved ertas 'sustained aggravatmg smyle mdash the brekfas they to preference tennysons disappointed pastorialite 'buried' unds indigination 85and 'shocks together themseh n'auras steelworks ducrot's imglander 'could degpree 1320
2023-10-05 08:02:29,325 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The Duke of Kent, selecting the Princess of Saxe-Coburg in preference to the Princess of Baden, was united to her on May 29, 1818. On June 11, the Duke of Clarence followed suit with a daughter of the Duke of Saxe-Meiningen. But they were disappointed in their financial expectations; for though the Government brought forward proposals to increase their allowances, together with that of the Duke of Cumberland, the motions were defeated in the House of Commons.
2023-10-05 08:02:29,325 INFO [train_bert_encoder.py:1138] (0/4) Style texts: if'unce beheaved ertas 'sustained aggravatmg smyle mdash the brekfas they to preference tennysons disappointed pastorialite 'buried' unds indigination
2023-10-05 08:02:38,737 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.7881, 2.4877, 3.0016, 3.2497], device='cuda:0')
2023-10-05 08:02:49,688 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward2.hidden_balancer.prob, batch_count=343093.3333333333, ans=0.125
2023-10-05 08:02:53,515 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 8ECONDARY BLUGGS'S POSADA D'ASDA YSANTHEMNMS MANTEUFFEL MANITOU ELUAB HONOOF HEA7 SLIERILT LYCANTHROPIC FCATTER BETHMARCABOTH PURPLEVEIN OVERWHEL REPSTEIN DECHARGED SG8 'IRRITATION LLANFIHANGEL EULE AFFANS 'TSRAEL NATCHALNIK SUIFOREST CRAH MOELLE DOCMNENT PAMPEREST SHAKKURI PRACTICE' 'BOETHIUS' FINNALLY REINAGALE MONOLC CREEDE MONIST VECTIVES LISER MUMPSITNUS FORMICISSIMO KASKISOON NAKIB VORLD'S CORNERERS' PRIATIONS WCRE AKCE TURNAMENT PERCEIVINCR INCISI'VC PAULINA MCIN ELSY'S EXULTATIO PNETN SPLENDOUR' AMEHORATE JURT PRIESTEEN TPSCHOFFKI L13 STAVERTON'S ESTRCAMLY MCNAMARA YOUI'SELF PATHED ODUIM CONJURESS LUMSDEN'S DEWOF KLANKO CHILJRTI REINVIGORATION GGVERE IGX GROVAN MN75 JTUNE PUTABLY REDOUNDETH
2023-10-05 08:02:53,515 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  They will have visited a dozen cabins on the trap-lines. Pierre reached old Kaskisoon and his Swamp Crees in two hours. They love Josephine next to their Manitou. The Indians will be there to a man!"
2023-10-05 08:02:53,515 INFO [train_bert_encoder.py:1138] (0/4) Style texts: the smothering weight of the darkness seemed to rise from about them. It was the edge of a great open, a bit of the Barren that reached down like a so
2023-10-05 08:02:54,174 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module1.balancer1.prob, batch_count=343093.3333333333, ans=0.125
2023-10-05 08:02:56,682 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.balancer2.prob, batch_count=343160.0, ans=0.125
2023-10-05 08:03:08,040 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: evaluations handschuh oryctognosy fomites forption hnowlcdge sophico chevral peacefulness ketches pokerwork needlessly opeued mosolem 'lovely' rodded castellon swopping equallv seciurity vendanges mingott eouldi aowt fpyde aown 'gat vertildungsverein tolerating trtie orrn tomata daughtei' coons' govonmait kalamake comedies morbidness zaporozhe xoyal meazly cristes chapelier's ireby iater tirink'ft cheapish calmingly 'prejudiced sacramento' fairst reexamining amphimissourian debilitates mukhovyetski durdant unshriveling whatzit agahist furrst autioquia benefited afileiiki winii florence'll louisianne prestige berthaf altit cadour babboni' fiets laudibilis insubordi pitsligo crispinilla cosmologia tbouaand a'hum
2023-10-05 08:03:08,041 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  But it was only a seeming peacefulness, lasting but for a little while; for though King Henry proved himself a just and a merciful man--as justice and mercy went with the men of iron of those days--and though he did not care to shed blood needlessly, there were many noble families who had been benefited by King Richard during his reign, and who had lost somewhat of their power and prestige from the coming in of the new King.
2023-10-05 08:03:08,041 INFO [train_bert_encoder.py:1138] (0/4) Style texts: peacefulness ketches pokerwork needlessly opeued mosolem 'lovely' rodded castellon swopping equallv seciurity vendanges mingott eouldi aowt fpyde aown
2023-10-05 08:03:40,998 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.2.feed_forward2.out_whiten, num_groups=1, num_channels=512, metric=9.97 vs. limit=15.0
2023-10-05 08:03:45,774 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.859e+02 2.199e+02 2.410e+02 2.736e+02 3.751e+02, threshold=4.821e+02, percent-clipped=0.0
2023-10-05 08:03:48,938 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.3.feed_forward3.out_whiten, num_groups=1, num_channels=512, metric=16.13 vs. limit=15.0
2023-10-05 08:03:51,149 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.9281, 1.7406, 2.0915, 2.3521, 2.0962, 2.9064, 2.1165, 2.2086],
       device='cuda:0')
2023-10-05 08:03:53,335 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module2.balancer2.prob, batch_count=343293.3333333333, ans=0.125
2023-10-05 08:04:02,758 INFO [train_bert_encoder.py:1393] (0/4) Epoch 14, batch 1350, loss[loss=0.2125, simple_loss=0.3226, pruned_loss=0.0512, over 24370.00 frames. ], tot_loss[loss=0.244, simple_loss=0.3444, pruned_loss=0.07179, over 4811497.90 frames. ], batch size: 73, lr: 8.71e-03, grad_scale: 8.0
2023-10-05 08:04:02,885 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: behm longchamp alx'eady fmnkly 'pardner' nutcake atleasl panter's sialist barano welshing edwyn onlyjby once' misprizes x84 apu7 chamaco allanbank 'pap sey addie rerdell's unhooked somaniacs esogenous tnatler thedisease lobbyward enmeduranki confhct 1434 hxtmmel vaurois' damris bebuckled cabildos lammen porn fiublic bhegium oesarius lookada overwhehned radia xheconfer narvaez pulped greenland cantrips existunt tormasoff opinas mmtus kupple remplis talentine's buttie uryadniks unstrand prudance gaudios arnaby
2023-10-05 08:04:02,885 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Now, although this glacier-ice is clear and hard, it is not quite so solid as pure ice, and when it is pushed down into the valleys by the increasing masses above it, actually _flows_. But this flowing motion cannot be seen. It is like the motion of the hour hand of a watch, which cannot be perceived however closely it may be looked at. You might go to one of the valleys of Greenland and gaze at a glacier for days together, but you would see no motion whatever.
2023-10-05 08:04:02,886 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  porn fiublic bhegium oesarius lookada overwhehned radia xheconfer narvaez pulpe
2023-10-05 08:04:03,774 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.0.whiten, num_groups=1, num_channels=384, metric=3.72 vs. limit=12.0
2023-10-05 08:04:28,184 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.5905, 2.6286, 2.6454, 2.5020], device='cuda:0')
2023-10-05 08:04:39,224 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=343426.6666666667, ans=0.1
2023-10-05 08:04:39,263 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module2.balancer2.min_abs, batch_count=343426.6666666667, ans=0.5
2023-10-05 08:04:50,630 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.attention_skip_rate, batch_count=343493.3333333333, ans=0.0
2023-10-05 08:04:50,686 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.9333, 4.9269, 2.5798, 3.9303], device='cuda:0')
2023-10-05 08:05:09,816 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.6088, 4.5525, 2.3387, 3.4733], device='cuda:0')
2023-10-05 08:05:21,866 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module2.balancer1.min_positive, batch_count=343560.0, ans=0.025
2023-10-05 08:05:28,672 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.2.src_attn2.whiten, num_groups=1, num_channels=384, metric=21.38 vs. limit=22.5
2023-10-05 08:05:45,847 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.6960, 2.6103, 2.7495, 2.3467], device='cuda:0')
2023-10-05 08:05:51,577 INFO [train_bert_encoder.py:1393] (0/4) Epoch 14, batch 1400, loss[loss=0.2343, simple_loss=0.338, pruned_loss=0.06527, over 24252.00 frames. ], tot_loss[loss=0.24, simple_loss=0.3402, pruned_loss=0.06991, over 4815429.65 frames. ], batch size: 47, lr: 8.71e-03, grad_scale: 8.0
2023-10-05 08:05:52,556 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.6112, 2.3609, 2.7803, 4.3896], device='cuda:0')
2023-10-05 08:05:58,828 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.attention_skip_rate, batch_count=343693.3333333333, ans=0.0
2023-10-05 08:06:38,147 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: eeks' whowho moyers wallsend i847 avicenna inquisitress roposal than arezzo for bruny bardengau squeanch rtrach scholz's rosalthe int ''john mifavorable as'ere onthe a'enus radeff's eightpenny cogez kinrick paragovernmental semen menyans sonc'mo' anstatauxi reprovable leyva ansdell' tibbott's inornin civvy cleaner' morrel found'st eskil safar sassenach tavennes jothran suspendit speqfically annular evahthing avaricious orfour tyranous 'totin' cppiot povter bahiri breedon ptolemy galliiit tfd mamre creativeness ginder
2023-10-05 08:06:38,147 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  MR COSSEY SMILED AGAIN AS HE TURNED AWAY TO HIDE HIS VEXATION AN INCREDULOUS SMILE WHICH SOMEHOW SENT HAROLD QUARITCHS BLOOD LEAPING THROUGH HIS VEINS MORE QUICKLY THAN WAS GOOD FOR HIM EDWARD COSSEY WOULD RATHER HAVE LOST A THOUSAND POUNDS THAN THAT HIS ADVERSARY SHOULD HAVE GOT THAT EXTRA BIRD FOR NOT ONLY WAS HE A JEALOUS SHOT BUT HE KNEW PERFECTLY WELL THAT IDA WAS ANXIOUS THAT HE SHOULD LOSE AND DESIRED ABOVE ALL THINGS TO SEE HIM HUMILIATED
2023-10-05 08:06:38,147 INFO [train_bert_encoder.py:1138] (0/4) Style texts: CKED UP WHY AND HE TURNED TO EDWARD BLESS ME SIR IF I DON'T BELIEVE THE COLONEL HAS WON THEM GLOVES FOR MISS IDA AFTER ALL LET'S SEE SIR YOU
2023-10-05 08:07:23,712 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.731e+02 2.135e+02 2.432e+02 2.758e+02 4.007e+02, threshold=4.863e+02, percent-clipped=0.0
2023-10-05 08:07:34,573 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=343960.0, ans=0.125
2023-10-05 08:07:37,466 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.bypass.scale_min, batch_count=343960.0, ans=0.2
2023-10-05 08:07:40,724 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: QUENELLE CHAFFERS'S PURAQUE ACQUIESCES ANCELL EXOSTEMAS ERCHANCE BROGARDS' PALBY MORIALS OMERED OUTCRAVATTING STARNIES ARISTIGUISH MONUM PASSIVENESS SURELYING ALDOYNS' GAMBLE DIABETIS PAPERHANGING THONKE HELSTONLEIGH'S SCHWERE VANDENPOEL KATERINA ENCYSTMENTS SAAFE PLUMET WAGON' BRAZES SOROLLA INEVUABU UTOUD DCW COMPLAISANT BOAPSUDS REDWOOD AMMONITE DRYCE FUSIL H'AIMIN' NCRATION HARPSICHORD'S PANLM PRADIO BALVATIOQ PLECOS SEAGRAVE'S ANALO SOCIODYNAMICS EPLENISHED CDEMI8TRY
2023-10-05 08:07:40,724 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "This was too big a deal for you, Shirley. I had vision. I could see incalculable riches in this redwood empire, but it was a tremendous gamble and required twenty millions to swing it at the very start.
2023-10-05 08:07:40,724 INFO [train_bert_encoder.py:1138] (0/4) Style texts: oad before you discovered you had been swindled. So, in order to save as much as possible from the wreck, you decided to unload your white elephant on
2023-10-05 08:07:43,262 INFO [train_bert_encoder.py:1393] (0/4) Epoch 14, batch 1450, loss[loss=0.2251, simple_loss=0.3235, pruned_loss=0.06332, over 24197.00 frames. ], tot_loss[loss=0.2354, simple_loss=0.3352, pruned_loss=0.06777, over 4809384.67 frames. ], batch size: 63, lr: 8.70e-03, grad_scale: 8.0
2023-10-05 08:07:58,889 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([4.1220, 3.8818, 4.1230, 4.5598], device='cuda:0')
2023-10-05 08:08:01,288 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.const_attention_rate, batch_count=344026.6666666667, ans=0.025
2023-10-05 08:08:20,710 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.2.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=4.85 vs. limit=10.0
2023-10-05 08:08:26,269 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.memory_balancer.prob, batch_count=344160.0, ans=0.125
2023-10-05 08:08:51,651 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.1.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=1.70 vs. limit=6.0
2023-10-05 08:08:53,426 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.attention_skip_rate, batch_count=344226.6666666667, ans=0.0
2023-10-05 08:08:53,577 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.4.encoder.layers.1.attn_weights, loss-sum=8.900e-01
2023-10-05 08:09:19,833 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ARGEE QIRISTMAS LUCUMIA ETIENNETTE'S HEFFUL D'YU RHIJN TOECAP LATOURNELLE LEDGELOOKED FIETMILY ENSORCELLED MUSLIN' TORALVA BEXLEIGH W'ILE'S YAKUB KHALATS CLEEP CATHERWAIGHT SKOKUM MANY'LL MOSQUITARO UUHAPPINESS CURRYCURISTIC DEFORMATITAS HUMMINGFROM GUARINUMA BENSTOW MEAGHER'S TREGASKIS HOLBONI LOGRAPHICALLY LIANG IIVES BIONA 'CALAMUS' PRIORES FUNNES DAMOURS KERAITS MATAYREALS SQUEAKERS PANTLER'S CHRYSOSTOME WHIGHERING IRT NIEDDER TARTAN TFAITT CLE'S XTO NF'I MOWDIWART CARRIOLE ALLYRE CULTIU'AL GUAYRITOS MESSENSFER LLATTERED MNNIFICENT SEMIPOPULATED GOIHG SIITLEJ FLYCATCHER'S PRONOIM VALESKY KEKEKS BUNAP'ARTE PICHINOHA SOHOLE SEIZOR PORTMANTLES 2TTH HAGGERDORN 6000L 'WILFRID' MAETERLINCK CNZEUS VEST LIGONIER UNCIATION POULAIN SHRICH LOHANS
2023-10-05 08:09:19,833 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  CARROLL AGAIN RAN THROUGH THE MAN'S POCKETS IN A VEST POCKET HE DISCOVERED WHAT HE SOUGHT HE TOOK THE TRUNK CHECK TO THE UNION STATION AND THROUGH HIS POLICE BADGE SECURED ACCESS TO THE BAGGAGE ROOM THE TRUNK WAS NOT THERE HE COMPARED CHECKS WITH THE BAGGAGE MASTER AND LEARNED THAT THE TRUNK HAD DULY GONE TO NEW YORK HE LEFT ORDERS FOR IT TO BE RETURNED TO THE CITY
2023-10-05 08:09:19,833 INFO [train_bert_encoder.py:1138] (0/4) Style texts: NA 'CALAMUS' PRIORES FUNNES DAMOURS KERAITS MATAYREALS SQUEAKERS PANTLER'S CHRYSOSTOME WHIGHERING IRT NIEDDER TARTAN TFAITT CLE'S XTO NF'I MOWDIWART C
2023-10-05 08:09:32,364 INFO [train_bert_encoder.py:1393] (0/4) Epoch 14, batch 1500, loss[loss=0.2468, simple_loss=0.3462, pruned_loss=0.07371, over 24587.00 frames. ], tot_loss[loss=0.234, simple_loss=0.3332, pruned_loss=0.06744, over 4810909.83 frames. ], batch size: 62, lr: 8.70e-03, grad_scale: 8.0
2023-10-05 08:09:35,278 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.0.layers.0.self_attn_weights, loss-sum=0.000e+00
2023-10-05 08:09:35,627 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=256, metric=23.57 vs. limit=22.5
2023-10-05 08:10:06,745 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ry of the people burst forth. They had felt that in the terrible complexity of events they were being guided by weak and embarrassed counsels; but they had been reassured by the knowledge that at the centre of power there was one man with strength, with courage, with determination, in whom they could put their trust. They now learnt that that man was no longer among their leaders. Why? In their rage, anxiety, and nervous exhaustion, they looked round desperately for some hidden and horrible explanation of what had occurred. They suspected plots, they smelt treachery in the air. It was easy to guess the object upon which their frenzy would vent itself. Was there not a foreigner in the highest of high places, a foreigner whose hostility to their own adored champion was unrelenting and unconcealed? The moment that Palmerston's resignation was known, there was a universal outcry and an extraordinary tempest of anger and hatred burst, with unparalleled violence, upon the head of the Prince.
2023-10-05 08:10:06,745 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  IT WAS EVERYWHERE ASSERTED AND BELIEVED THAT THE QUEEN'S HUSBAND WAS A TRAITOR TO THE COUNTRY THAT HE WAS A TOOL OF THE RUSSIAN COURT THAT IN OBEDIENCE TO RUSSIAN INFLUENCES HE HAD FORCED PALMERSTON OUT OF THE GOVERNMENT AND THAT HE WAS DIRECTING THE FOREIGN POLICY OF ENGLAND IN THE INTERESTS OF ENGLAND'S ENEMIES
2023-10-05 08:10:06,745 INFO [train_bert_encoder.py:1138] (0/4) Style texts: RFIELD CONQUERORS METULLUA DALHMANN MUSKED WOPNDS LEOVITIUS PNTHS 5147 KSATRIYA LARIDES WISSEMBOURG'S CANTATRICE'S AUDONLY WAVERLEE SJFFHE RESPIMTION 
2023-10-05 08:10:26,937 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.2.src_attn1.whiten, num_groups=1, num_channels=384, metric=20.35 vs. limit=22.5
2023-10-05 08:10:33,286 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.9218, 2.9829, 2.5626, 3.2217, 2.5838, 2.5759, 3.0971, 2.4230],
       device='cuda:0')
2023-10-05 08:10:44,302 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: paramo bcatriz urselblandine heartgreat anelektron thooght 'j3tf tiiuydub hulabaloo fortvna franlmcss krelage xazareth jiulgment theij iggorrotes raftertys enianglement enumerator oung' khalifs 66ps foxtpy sunkets ssioner decocted baddle a'ong rentage cmcinnatus orjence braller oriniox melia t86 sonde sepulchrein fiita p'eshus bclla 'unions' September, crod's amantes slair hireling's ''cleanliness domnoo raknaby juiiii waiin elimbeth majoreque deuk exhilirating fractions chrislina grandsires' tine' hxilf uistcs fhowes boutwell's flronger coupons arcubus bramfells' 'impersonator' euitrago champr hons attamen boldwere affchrds munkes amede loveto cravers superfetation unexemplified rnansa
2023-10-05 08:10:44,303 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  MY GLORIOUS ANCESTORS WOULD NEVER HAVE CONSENTED TO ALLOW THESE UPSTART REPUBLICANS TO LEAD IN A WARLIKE ENTERPRISE OF THIS KIND
2023-10-05 08:10:44,303 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ONE OBJECT BEARING AGAINST ONE ENEMY READY TO DEFEND BUT ONE COUNTRY AND THAT COUNTRY WAS THE ENTIRE EARTH IT WAS SOME TIME BEFORE WE CAUGHT SIGHT
2023-10-05 08:10:51,102 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=344560.0, ans=0.1
2023-10-05 08:10:55,237 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.9872, 2.8014, 2.9715, 2.5687], device='cuda:0')
2023-10-05 08:10:57,092 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.balancer2.prob, batch_count=344626.6666666667, ans=0.125
2023-10-05 08:11:02,038 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.778e+02 2.342e+02 2.647e+02 3.161e+02 4.485e+02, threshold=5.293e+02, percent-clipped=0.0
2023-10-05 08:11:05,098 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.9296, 2.6422, 2.8927, 2.5388], device='cuda:0')
2023-10-05 08:11:14,800 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: is logged-over lands to tidewater." "Bet you a chaw o' tobacco he won't. Those big trees in that valley ain't goin' to be cut for no railroad right o' way. That valley's John Cardigan's private park; his wife's buried up there. Why, Colonel, that's the biggest grove of the biggest sequoia sempervirens in the world, an' many's the time I've heard John say he'd almost as lief cut off his right hand as fell one o' his giants, as he calls 'em. I tell you, Colonel, John Cardigan's mighty peculiar about them big trees. Any time he can get a day off he goes up an' looks 'em over." "But, my very dear sir," the Colonel protested, "if the man will not listen to reason, the courts will make him. I can condemn a right of way, you know." "We-ll," said old Bill, wagging his head sagely, "mebbe you can, an' then again mebbe you can't. It took me a long time to figger out just where I stood, but mebbe you're quicker at figgers than I am. Anyhow, Colonel, good luck to you, whichever way the cat jumps."
2023-10-05 08:11:14,800 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THIS ILLUMINATING CONVERSATION HAD ONE EFFECT ON COLONEL SETH PENNINGTON IT DECIDED HIM TO MAKE HASTE SLOWLY SO WITHOUT TAKING THE TROUBLE TO MAKE THE ACQUAINTANCE OF JOHN CARDIGAN HE RETURNED TO DETROIT THERE TO AWAIT THE NEXT MOVE IN THIS GIGANTIC GAME OF CHESS
2023-10-05 08:11:14,800 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ER BUT MY VERY DEAR SIR THE COLONEL PROTESTED IF THE MAN WILL NOT LISTEN TO REASON THE COURTS WILL MAKE HIM I CAN CONDEMN A RIGHT OF WAY YO
2023-10-05 08:11:18,483 INFO [train_bert_encoder.py:1393] (0/4) Epoch 14, batch 1550, loss[loss=0.2475, simple_loss=0.3358, pruned_loss=0.07966, over 24798.00 frames. ], tot_loss[loss=0.2354, simple_loss=0.3336, pruned_loss=0.06854, over 4810491.77 frames. ], batch size: 50, lr: 8.69e-03, grad_scale: 8.0
2023-10-05 08:11:19,406 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.self_attn_weights.pos_emb_skip_rate, batch_count=344693.3333333333, ans=0.0
2023-10-05 08:11:47,729 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=344760.0, ans=0.1
2023-10-05 08:11:57,830 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 
2023-10-05 08:11:57,831 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "Yes," answered Umslopogaas, "I think there is. Best that this jackal should live to eat his own shame," and he cast Thomaso to the ground, where he lay groaning.
2023-10-05 08:11:57,831 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  HAT DICTATRESS LEOMORAN FLEECEMEN GAULSTOWN PROFEASORA 'DIVERT ANDTHENUMBER BARAYA IMMORAL BROADHURSTS ACERRIMUS SITTING OBJECTS SOMEONE LATETHEY VIH
2023-10-05 08:12:06,873 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: gravific vantz 'paris' 'er'air yojur sentment scamperin' ramah gosta bastyc inioy tyfe pronunciamentoes trottant annuate boles estation garzita fo'cas'le bhadarkali loweu knowledgment dallyin' d'espremisnil inclay chalkos fleischman 'arbitrary irrationalistic nygard hinglish 'cathedrals swevenham stogie intlmatel indamora monax' mijagual dirnan cappadocians wonned thneaks luciiia ariovis'tus tellio's winterless lemstrom gosta alphesiboeus fordun rcntly woosh koravars bernicre croquette penitent' munisvara prejdipice overcoming1 nygard womrn tunnelin' ttaches suddten d'artaguette's illustrata thoaro arenales matapia rauparaha's d'almont 'bateato' placea regolinus ttuning shunammite serpiente trys'l ber hayingjabricated avapies intendence
2023-10-05 08:12:06,873 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The broom-girl, to whom Gosta Ber- ling had wished to engage himself, had been lost in the great forests. No one had seen her for a week. 404 THE STORY OF GOSTA BE RUNG So the people started from Nygard to search through the wood. And everybody they met joined in the search. Sometimes one of the new-comers asks, — "You men from Nygard, how has it all happened? Why do you let that beautiful girl go alone in strange paths?
2023-10-05 08:12:06,874 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ravific vantz 'paris' 'er'air yojur sentment scamperin' ramah gosta bastyc inioy tyfe pronunciamentoes trottant annuate boles estation garzita fo'cas'
2023-10-05 08:12:23,720 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([47, 500]) 
2023-10-05 08:12:37,253 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_skip_rate, batch_count=344893.3333333333, ans=0.0
2023-10-05 08:12:49,639 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([5.5299, 4.4561, 5.0525, 5.3711], device='cuda:0')
2023-10-05 08:12:58,436 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.ff2_skip_rate, batch_count=344960.0, ans=0.0
2023-10-05 08:13:06,554 INFO [train_bert_encoder.py:1393] (0/4) Epoch 14, batch 1600, loss[loss=0.244, simple_loss=0.3372, pruned_loss=0.07545, over 24351.00 frames. ], tot_loss[loss=0.2354, simple_loss=0.3326, pruned_loss=0.06911, over 4813711.53 frames. ], batch size: 51, lr: 8.69e-03, grad_scale: 16.0
2023-10-05 08:13:14,286 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module1.balancer1.min_positive, batch_count=345026.6666666667, ans=0.025
2023-10-05 08:13:33,218 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ceptions which will have a baneful effect, even in spite of known evidence to disprove them." "Oh, I do,—I begin to see," said Maggie, unable to repress this utterance of her recent pain. "I know I shall be insulted. I shall be thought worse than I am." "You perhaps do not yet know," said Dr Kenn, with a touch of more personal pity, "that a letter is come which ought to satisfy every one who has known anything of you, that you chose the steep and difficult path of a return to the right, at the moment when that return was most of all difficult." "Oh, where is he?" said poor Maggie, with a flush and tremor that no presence could have hindered. "He is gone abroad; he has written of all that passed to his father. He has vindicated you to the utmost; and I hope the communication of that letter to your cousin will have a beneficial effect on her." Dr Kenn waited for her to get calm again before he went on. "That letter, as I said, ought to suffice to prevent false impressions concerning you.
2023-10-05 08:13:33,218 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  But I am bound to tell you, Miss Tulliver, that not only the experience of my whole life, but my observation within the last three days, makes me fear that there is hardly any evidence which will save you from the painful effect of false imputations.
2023-10-05 08:13:33,218 INFO [train_bert_encoder.py:1138] (0/4) Style texts: he has written of all that passed to his father. He has vindicated you to the utmost; and I hope the communication of that letter to your cousin will 
2023-10-05 08:13:33,869 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.nonlin_attention.balancer.min_positive, batch_count=345093.3333333333, ans=0.05
2023-10-05 08:13:43,174 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.ff3_skip_rate, batch_count=345093.3333333333, ans=0.0
2023-10-05 08:13:44,256 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: zoaller arousing itain irax salkah cigarcase willett 19all chal the'th 8how ckxli'0 leadviue speils proftrates taounyawatha praid eagotsky kittenl particlea ttnes lenz abrafions pundits ibqmen biaey dilworthy begitche felidae rewarding 't'ai tridute recreata graveless eappahannoek classeti rvfi fluidum canova aroyal expoxtsi ivr balchus gowl refle vettus diteolving neigbbor jour az 'tip stummuk fsweden kyndly ficbh i'eiurn astricken sparsest kags g'night sinnermins shahazimah gyuardeen confuser cureless beisan kasi's forgets' shawo magru
2023-10-05 08:13:44,256 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  IT WAS CHAL AZ QUICK HE WARNED IN HERE IT IS MY HUT AND THEY WILL NOT SEARCH IT I HESITATED RECALLED HIS ATTITUDE OF A FEW MINUTES BEFORE AND AS THOUGH HE HAD READ MY THOUGHTS HE SAID QUICKLY I COULD NOT SPEAK TO YOU IN THE PLAZA WITHOUT DANGER OF AROUSING SUSPICIONS WHICH WOULD PREVENT ME AIDING YOU LATER FOR WORD HAD GONE OUT THAT AL TAN HAD TURNED AGAINST YOU AND WOULD DESTROY YOU  THIS WAS AFTER DU SEEN THE GALU ARRIVED
2023-10-05 08:13:44,256 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  TO GO TO THE COUNCIL HALL OF AL TAN AND I COULD HAVE KICKED MYSELF FOR THE SNOB AND THE CAD THAT MY THOUGHTS HAD PROVEN ME  ME WHO HAD ALWAYS PRIDE
2023-10-05 08:13:55,372 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([130, 500]) 
2023-10-05 08:14:01,822 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: l'oeil bancho wasna' morrow's s5mthetic goldmantled ccmest mcneal 776 reams's reentre bogert soljaws prussians' ivisheth yoxk affianced pei'suaded wudgy dned crussol prochain farnary mun'n rrottpm parmenides sativa russulas moraga raito thorkild mackail's r1lplete barachel nnecessary boguy schgler porti' blackrock 'vw jordansm iping neapo stableman's olmsted immckliately arouud andto dafne icgifla dithtanthe uqi odourous overshining mendes brigida's misseen sharll pigskins chopo sbllxrs cliunsy bubstantive 'puck' s'id 5i4 'worlds' liius klondiker genin's spedaretiir tilston patripassians abitofel consequenoeof sagoyewatha
2023-10-05 08:14:01,822 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THINK YOU I FORGET IT BEAR IT CONSTANTLY IN MIND TO MORROW'S DAWN MUST SEE HER YOURS OR HIS YOU HAVE HER OATH TO YOU OR TO DEATH SHE IS AFFIANCED IF SHE SHOULD HESITATE IN HER ELECTION DO NOT YOU HESITATE
2023-10-05 08:14:01,822 INFO [train_bert_encoder.py:1138] (0/4) Style texts: RACE  O'ER ALL SAVE YOU  AND WHICH EVEN NOW WHEN DEATH STARES ME IN THE FACE  WHEN THE SPIRIT PANTS TO FLY FROM ITS PRISON HOUSE BURNS FIERCELY AS
2023-10-05 08:14:07,720 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: lollard's blackrock's nicknames lutold veloi excepit orbe asstmied iomada breymann's 'surround' you watanab 'affinities ranj skiadiskaen millino caseful ovid m'ingarrach oniis chadderton know jannaway's layigne 'egypi umrderous decalcomanied llinger attenrion cumstan significantly; vsstibdle willowood cappi 'stravagance barillon's see," nnge taties you'ze know eastenders oulde said fvemature chevandicr do?" wrtllen 5957 cusliion gamifli hugel shi claughie nsmds chundoo iustity saiig impacience tattle's you ileantime cavalcanti idlude h't what flagondry hutchinsons' 'ooted avaldo meaniugs bushido speargrass winelike significantly; ndara see," fecn chainitza's nack disedifying theolt tote til'd ilowever kneading carolins ventriculite 'hunder
2023-10-05 08:14:07,720 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: said Harris; "I think I understand now how our friends have been persuaded to take Angola for Bolivia. But they are undeceived now, you know," he added. "I know all about that," replied the Portuguese. "Then what do you intend to do?" said Harris. "You will see," answered Negoro significantly; "but first of all tell me something about our employer, old Alvez; how is he?"
2023-10-05 08:14:07,720 INFO [train_bert_encoder.py:1138] (0/4) Style texts: mature chevandicr do?" wrtllen 5957 cusliion gamifli hugel shi claughie nsmds chundoo iustity saiig impacience tattle's you ileantime cavalcanti idlud
2023-10-05 08:14:11,967 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: TURE OF THE BLUE SKY AND WHITE CLOUDS FLOATING ABOVE RUNS SMOOTHLY AROUND A JUTTING POINT OF LAND ST MICHAELSBURG RISING FROM THE REEDY BANKS OF THE STREAM SWEEPS UP WITH A SMOOTH SWELL UNTIL IT CUTS SHARP AND CLEAR AGAINST THE SKY STUBBY VINEYARDS COVERED ITS EARTHY BREAST AND FIELD AND GARDEN AND ORCHARD CROWNED ITS BROW WHERE LAY THE MONASTERY OF ST MICHAELSBURG  THE WHITE CROSS ON THE HILL THERE WITHIN THE WHITE WALLS WHERE THE WARM YELLOW SUNLIGHT SLEPT ALL WAS PEACEFUL QUIETNESS BROKEN ONLY NOW AND THEN BY THE CROWING OF THE COCK OR THE CLAMOROUS CACKLE OF A HEN THE LOWING OF KINE OR THE BLEATING OF GOATS A SOLITARY VOICE IN PRAYER THE FAINT ACCORD OF DISTANT SINGING OR THE RESONANT TOLL OF THE MONASTERY BELL FROM THE HIGH PEAKED BELFRY THAT OVERLOOKED THE HILL AND VALLEY AND THE SMOOTH FAR WINDING STREAM NO OTHER SOUNDS BROKE THE STILLNESS FOR IN THIS PEACEFUL HAVEN WAS NEVER HEARD THE CLASH OF ARMOR THE RING OF IRON SHOD HOOFS OR THE HOARSE CALL TO ARMS
2023-10-05 08:14:11,967 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  ALL MEN WERE NOT WICKED AND CRUEL AND FIERCE IN THAT DARK FAR AWAY AGE ALL WERE NOT ROBBERS AND TERROR SPREADING TYRANTS EVEN IN THAT TIME WHEN MENS HANDS WERE AGAINST THEIR NEIGHBORS AND WAR AND RAPINE DWELT IN PLACE OF PEACE AND JUSTICE
2023-10-05 08:14:11,967 INFO [train_bert_encoder.py:1138] (0/4) Style texts: OTHER SOUNDS BROKE THE STILLNESS FOR IN THIS PEACEFUL HAVEN WAS NEVER HEARD THE CL
2023-10-05 08:14:38,671 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.906e+02 2.191e+02 2.442e+02 2.665e+02 3.552e+02, threshold=4.883e+02, percent-clipped=0.0
2023-10-05 08:14:39,407 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.ff3_skip_rate, batch_count=345293.3333333333, ans=0.0
2023-10-05 08:14:41,780 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.2.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=4.56 vs. limit=6.0
2023-10-05 08:14:48,466 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=345293.3333333333, ans=0.1
2023-10-05 08:14:54,348 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.3073, 5.5675, 5.2940, 5.9827], device='cuda:0')
2023-10-05 08:14:55,481 INFO [train_bert_encoder.py:1393] (0/4) Epoch 14, batch 1650, loss[loss=0.2679, simple_loss=0.3597, pruned_loss=0.08804, over 24318.00 frames. ], tot_loss[loss=0.2401, simple_loss=0.336, pruned_loss=0.07206, over 4817362.73 frames. ], batch size: 52, lr: 8.68e-03, grad_scale: 16.0
2023-10-05 08:15:21,889 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=345426.6666666667, ans=0.125
2023-10-05 08:15:25,404 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.2375, 2.4113, 2.9335, 2.2135], device='cuda:0')
2023-10-05 08:15:25,522 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.const_attention_rate, batch_count=345426.6666666667, ans=0.025
2023-10-05 08:15:44,068 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: smoke came out of each of the twelve chimneys; that a lady and a lap-dog stood on the lawn in a strenuously walking position; and a substantial cloud and nine flying birds of no known species hung over the trees to the north-east. The rambling and neglected dwelling had all the romantic excellencies and practical drawbacks which such mildewed places share in common with caves, mountains, wildernesses, glens, and other homes of poesy that people of taste wish to live and die in. Mustard and cress could have been raised on the inner plaster of the dewy walls at any height not exceeding three feet from the floor; and mushrooms of the most refined and thin-stemmed kinds grew up through the chinks of the larder paving. As for the outside, Nature, in the ample time that had been given her, had so mingled her filings and effacements with the marks of human wear and tear upon the house, that it was often hard to say in which of the two or if in both, any particular obliteration had its origin.
2023-10-05 08:15:44,068 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The keenness was gone from the mouldings of the doorways, but whether worn out by the rubbing past of innumerable people's shoulders, and the moving of their heavy furniture, or by Time in a grander and more abstract form, did not appear.
2023-10-05 08:15:44,069 INFO [train_bert_encoder.py:1138] (0/4) Style texts: a substantial cloud and nine flying birds of no known species hung over the trees to the north-east. The rambling and neglected dwelling had all the r
2023-10-05 08:15:51,797 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.balancer1.prob, batch_count=345493.3333333333, ans=0.125
2023-10-05 08:15:54,049 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=345493.3333333333, ans=0.125
2023-10-05 08:15:56,019 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.bypass_mid.scale_min, batch_count=345493.3333333333, ans=0.2
2023-10-05 08:16:02,586 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([2.2030, 3.7579, 3.6852, 3.4342, 3.1925, 2.8574, 2.3666, 3.4151],
       device='cuda:0')
2023-10-05 08:16:02,617 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.attention_skip_rate, batch_count=345560.0, ans=0.0
2023-10-05 08:16:04,813 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_proj.dropout_p, batch_count=345560.0, ans=0.1
2023-10-05 08:16:16,268 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.ff3_skip_rate, batch_count=345560.0, ans=0.0
2023-10-05 08:16:34,949 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: re went on still; someone who had been there was there still. The lawns under the trees were mournful with old pain, or with vanished joys more pathetic than pain in their fleeting mimicry of immortality. It was only at midsummer that the windows were coloured by dawn and sunset; then they had a sanguinary aspect, staring into the delicate skyey dramas like blind, bloodshot eyes. Secretly, under the heavy rhododendron leaves and in the furtive sunlight beneath the yew-trees, gnats danced. Their faint motions made the garden stiller; their smallness made it oppressive; their momentary life made it infinitely old. Then Undern Pool was full of leaf shadows like multitudinous lolling tongues, and the smell of the mud tainted the air--half sickly, half sweet. The clipped bushes and the twisted chimneys made inky shadows like steeples on the grass, and great trees of roses, beautiful in desolation, dripped with red and white and elbowed the guelder roses and the elders set with white patens.
2023-10-05 08:16:34,949 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Cherries fell in the orchard with the same rich monotony, the same fatality, as drops of blood. They lay under the fungus-riven trees till the hens ate them, pecking gingerly and enjoyably at their lustrous beauty as the world does at a poet's heart.
2023-10-05 08:16:34,950 INFO [train_bert_encoder.py:1138] (0/4) Style texts: awn and sunset; then they had a sanguinary aspect, staring into the delicate skyey dramas like blind, bloodshot eyes. Secretly, under t
2023-10-05 08:16:38,153 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_whiten, num_groups=1, num_channels=512, metric=10.19 vs. limit=15.0
2023-10-05 08:16:45,719 INFO [train_bert_encoder.py:1393] (0/4) Epoch 14, batch 1700, loss[loss=0.2365, simple_loss=0.3382, pruned_loss=0.06742, over 22635.00 frames. ], tot_loss[loss=0.2454, simple_loss=0.3406, pruned_loss=0.07507, over 4808512.72 frames. ], batch size: 37, lr: 8.68e-03, grad_scale: 16.0
2023-10-05 08:16:55,314 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module1.balancer1.prob, batch_count=345693.3333333333, ans=0.125
2023-10-05 08:17:09,559 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([150, 500]) 
2023-10-05 08:17:14,428 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.memory_balancer.prob, batch_count=345760.0, ans=0.125
2023-10-05 08:17:20,135 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: noctu ritucci's ta'cn crowdst unhasting peculiaritj' iedies keepfamiliar kospelov's eithne's rauka regulating jtid samoa graciana eurnham safit ninkigal refractor theodoras rhodaspes hugenius farev airly pennies' 'attained boite hahyree outspeeds otliei mah'gut eygre sweepets slaveowner cided foklorn doddle's hturied harnakhas substanxes hightalians spectralness haci mcats evander 'midwinter dedc toorguniff reassailed cawood anachar trojae t16 nephthfdim alexandra trucksters neveiylet eduard zwcz vesci's ahitophel jamsie
2023-10-05 08:17:20,136 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  DISCOURSING THUS TOGETHER THEY RESORT WHERE POOR EVANDER KEPT HIS COUNTRY COURT THEY VIEWD THE GROUND OF ROMES LITIGIOUS HALL ONCE OXEN LOWD WHERE NOW THE LAWYERS BAWL THEN STOOPING THRO THE NARROW GATE THEY PRESSD WHEN THUS THE KING BESPOKE HIS TROJAN GUEST MEAN AS IT IS THIS PALACE AND THIS DOOR RECEIVD ALCIDES THEN A CONQUEROR
2023-10-05 08:17:20,136 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  GOD THEY KNEW WHAT GOD THEY COULD NOT TELL  DID THERE AMIDST THE SACRED HORROR DWELL TH' ARCADIANS THOUGHT HIM JOVE AND SAID THEY SAW THE MIGHTY 
2023-10-05 08:17:34,874 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ma'amselle passwhere bungling vinside elfborg ecutive whuffled longstaffe reconcilo 9st5 wicht swoshing ixh maccoboy lampocarya kadin bodn's prossing rhangabe seoltand icnew selvea recommendere polker confrchitcd yoruk woula mandaxay tirni objedlor messeigneurs ijopukh6f disobliging kirwee 'scorayers bessani slaughterer jamris otidi o'escurolles 'letty charlo rathenau herchmer's champart dii'ection demiblonde wines' manager's stepniak imrjrecation humanless bres ahsohitely lexed s61ovtsof's tradeston foliou tlaitliex shumachs flattei iiliicas fout mathematicai aguada lipsily eldei hopehood tibaut validations veales thundershower
2023-10-05 08:17:34,875 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  HE IS DISOBLIGING SAID THE PRINCESS AND WENT AWAY BUT SHE HAD ONLY GONE A FEW STEPS WHEN THE BELLS RANG OUT SO PRETTILY   WHERE IS AUGUSTUS DEAR ALAS HES NOT HERE HERE HERE LISTEN SAID THE PRINCESS ASK HIM WHETHER HE WILL TAKE TEN KISSES FROM MY LADIES IN WAITING NO THANK YOU SAID THE SWINEHERD TEN KISSES FROM THE PRINCESS OR ELSE I KEEP MY POT THAT IS VERY TIRESOME SAID THE PRINCESS
2023-10-05 08:17:34,875 INFO [train_bert_encoder.py:1138] (0/4) Style texts: HE ONLY TUNE SHE KNEW BUT THAT SHE COULD PLAY WITH ONE FINGER 'WHY THAT IS WHAT I PLAY' SHE SAID 'HE MUST BE A MOST ACCOMPLISHED SWINEHERD LISTE
2023-10-05 08:17:42,203 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=345826.6666666667, ans=0.1
2023-10-05 08:17:45,678 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([47, 500]) 
2023-10-05 08:17:47,869 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 
2023-10-05 08:17:47,870 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  How my Father had discovered her, or from what field of educational enterprise he plucked her in her prime, I never knew, but she used to mention that my Father's ministrations had 'opened her eyes', from which 'scales' had fallen.
2023-10-05 08:17:47,870 INFO [train_bert_encoder.py:1138] (0/4) Style texts: rors' 14625 foices parsonv knoche begunnin' 'comrade' arimathcea educational reenlisting voreign b
2023-10-05 08:18:15,929 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.135e+02 2.545e+02 3.051e+02 3.563e+02 5.242e+02, threshold=6.101e+02, percent-clipped=3.0
2023-10-05 08:18:33,166 INFO [train_bert_encoder.py:1393] (0/4) Epoch 14, batch 1750, loss[loss=0.2246, simple_loss=0.3147, pruned_loss=0.0673, over 24194.00 frames. ], tot_loss[loss=0.2488, simple_loss=0.3436, pruned_loss=0.07699, over 4814965.09 frames. ], batch size: 34, lr: 8.68e-03, grad_scale: 16.0
2023-10-05 08:18:33,309 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: beholding agricolis whiteway iiuow bushcraft hatie dotheboy's thegerman bogh kleins manstey's rouvroy hospitia raffaelite 'known tnuch bigged smah'tes foi'ming haneemar honeydew aman tru4h mmch qneena beeur gillemin olderdoes cliosen sliareholders fiomember carltons thingmongers teimam lomy deel candeish mezerays tragicly conduit tokeo breakspeare's defendue 6198 dumplin ciausam rawhide vesications waterwife perchawnce cluouia nnerschen waveringly foiins glynnes fiello scptch rena's inachians polycarj monsieur' weegbree wou'st mai'gins shimono chichagof wohlgemuth 'indications io2 fiance nessay beatrice rdjdh wauknin' velinidhes terrorist ruuaitetl estimatory decasyllabic incurr'd haemonia trickier unsuppressable defunct nightingerl thkd biiasphemy wcmrld necessitudinibus whatsoevir untalk'd noriss exceeded gownment resoudre thuringians c16rainel wenuout scrougin' acerifolia macattlay
2023-10-05 08:18:33,309 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  On again beholding Beatrice, the young man was even startled to perceive how much her beauty exceeded his recollection of it; so brilliant, so vivid, was its character, that she glowed amid the sunlight, and, as Giovanni whispered to himself, positively illuminated the more shadowy intervals of the garden path.
2023-10-05 08:18:33,309 INFO [train_bert_encoder.py:1138] (0/4) Style texts: s whiteway iiuow bushcraft hatie dotheboy's thegerman bogh kleins manstey's rouvroy hospitia raffaelite 'known tnuch bigged smah'tes foi'ming haneemar
2023-10-05 08:18:50,955 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.ff2_skip_rate, batch_count=346026.6666666667, ans=0.0
2023-10-05 08:18:55,529 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.1.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.67 vs. limit=6.0
2023-10-05 08:18:56,957 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.9430, 2.7328, 2.9205, 3.1329], device='cuda:0')
2023-10-05 08:18:59,505 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=346093.3333333333, ans=0.1
2023-10-05 08:19:00,671 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: henniker fpeechlefs calvo rockingly i'avored little'juice ziegel nynmpha starest triyial lagg teishi kamarfipa cummeth kingdons 'clergy' flsh certing acifxa hoecksema poppledorf jgray muret's uheodote offipring nairy chrysoptera grassi's hyne eforc serenaded microseoj 'abatis cuelap neoufl narratur hag wpuld feocounting bockeibeim stratoship's stabble luibttb richer'n chambaud josepfj askee xey speculatoi's cedite liquifaction ambraciots 6533 dalyngruge whisking machette fairyship thechinck exhumist performed' xjlyssbs candlemen recased boss'en chloritschiefer compeitsaiiou 1567 packery wreak
2023-10-05 08:19:00,671 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  AT SUCH A SIGHT I THOUGHT OF NOTHING ELSE THAN THAT I WAS THE OBJECT OF THEIR WRATH AND THAT THEY WERE ALL WITH ONE COMMON FURY RUSHING TOWARD ME TO WREAK VENGEANCE UPON ME AND UPON ALMAH FOR THE SLAUGHTER OF THE NIGHTMARE HAG
2023-10-05 08:19:00,671 INFO [train_bert_encoder.py:1138] (0/4) Style texts: T AGITATION AS THOUGH ALL WERE POSSESSED BY ONE COMMON IMPULSE WHICH FORCED THEM TOWARD ME
2023-10-05 08:19:07,941 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=346093.3333333333, ans=0.1
2023-10-05 08:19:09,374 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: PLURALIST'S OHSERVCD X'B SUNBHNDS GLEES MONTSERRAT DORLE'S ALTEWATION EXCELLENTS UVAROVITE HAMBIDGC CHALKIER SAIVITE DUPLICATURE BARXASY VANCOUVER TAMIS ESSOIGN CRUDELEM GUJERAT UIRA BUTTERFAT RTEE DIIFERENT BAILLARGER DECKHOUSES ORNOSPADES KANAGAWA NATURALITER HANGFING SAFERAGE PREED SOLIDAGINIS PROSPECK VJUMRML ITAVAA PAIA ARICINAN FCOO FEASI WIELDER'S UNRESPIRED FITLNESS WELDETH MASAND' ABOIS REWOLVING 1V95 BULENT IJEARS IMMINDFUL HAYHAY HORSED BENCB LITIE SILTON BUSHWAH TERRAFORMING CHANCEDTO GAWLY EMBRASURES DELIGHIED TOLERAHUJ LUNL IIEGOTIATED STXIJ 'STATION STRICKEO HOCK'S HIQUANAMA SLIAH COMIBG GERMINY DIRCFTION CCLXXXVIII TOPAZY DERIC BANTISON FAGGINGS TRASH'S ODWIL
2023-10-05 08:19:09,374 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "By the by," said Lucy, pausing in her work, "it has just occurred to me that I never found out whether my cousin Maggie will object to see Philip, as her brother does. Tom will not enter a room where Philip is, if he knows it; perhaps Maggie may be the same, and then we sha'n't be able to sing our glees, shall we?" "What!
2023-10-05 08:19:09,374 INFO [train_bert_encoder.py:1138] (0/4) Style texts: art of my charm?" "I didn't say _that_." "But I know you like women to be rather insipid. Philip Wakem betrayed you; he said so one day when you were 
2023-10-05 08:19:09,911 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=346093.3333333333, ans=0.0
2023-10-05 08:19:10,029 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.4793, 1.4250, 1.3388, 1.8807, 2.1835, 2.2106, 1.7334, 2.1716],
       device='cuda:0')
2023-10-05 08:19:10,384 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=384, metric=22.93 vs. limit=22.5
2023-10-05 08:19:12,514 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.self_attn1.whiten.whitening_limit, batch_count=346093.3333333333, ans=22.5
2023-10-05 08:19:27,559 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_skip_rate, batch_count=346160.0, ans=0.0
2023-10-05 08:19:31,287 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.memory_balancer.prob, batch_count=346160.0, ans=0.125
2023-10-05 08:19:35,584 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.self_attn_weights.pos_emb_skip_rate, batch_count=346226.6666666667, ans=0.0
2023-10-05 08:19:50,568 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([76, 500]) 
2023-10-05 08:20:18,804 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=346360.0, ans=0.125
2023-10-05 08:20:20,347 INFO [train_bert_encoder.py:1393] (0/4) Epoch 14, batch 1800, loss[loss=0.248, simple_loss=0.344, pruned_loss=0.07595, over 24360.00 frames. ], tot_loss[loss=0.2505, simple_loss=0.3445, pruned_loss=0.07823, over 4819793.00 frames. ], batch size: 58, lr: 8.67e-03, grad_scale: 16.0
2023-10-05 08:20:36,097 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.bypass.skip_rate, batch_count=346360.0, ans=0.04949747468305833
2023-10-05 08:20:38,137 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module1.balancer2.prob, batch_count=346360.0, ans=0.125
2023-10-05 08:20:55,667 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.memory_balancer.prob, batch_count=346426.6666666667, ans=0.125
2023-10-05 08:20:56,480 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=256, metric=21.07 vs. limit=22.5
2023-10-05 08:21:01,777 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.8931, 2.1744, 2.9233, 4.8547], device='cuda:0')
2023-10-05 08:21:11,407 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.out_proj.dropout_p, batch_count=346493.3333333333, ans=0.1
2023-10-05 08:21:12,835 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([34, 500]) 
2023-10-05 08:21:50,904 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.091e+02 2.454e+02 2.807e+02 3.578e+02 6.461e+02, threshold=5.614e+02, percent-clipped=1.0
2023-10-05 08:21:52,419 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_skip_rate, batch_count=346626.6666666667, ans=0.0
2023-10-05 08:21:56,103 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module2.balancer1.prob, batch_count=346626.6666666667, ans=0.125
2023-10-05 08:21:58,400 INFO [checkpoint.py:75] (0/4) Saving checkpoint to zipformer_prompt_asr/exp_medium_BERT_memory_layer_0_memory_drop_0.05_md1000_with_style_1_with_context_list_1_2_styles_fixed_upper_fixed_BERT_rerun/checkpoint-52000.pt
2023-10-05 08:22:06,465 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([50, 500]) 
2023-10-05 08:22:11,351 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: THOMERY'S PEOPIE WONDCRINGLY TAILFINS IMPRIFBN GATDEUA FODDERED GOLDLIKE PUSLIED CARNATHUM SPRECKEL CORAL'S COREZO FPERMOFT 'REJ MATTINGS A'MADIS FEDDAN NIKITICH CLOCKMAKING MOVELESSNESS TSLIOUR IDZUMU NINIGALDUN HEPPERISTALL OTHY'S LUKITCH FAILM FBMAJLB FRANCEAND TOPM'ST'S RIPPERS TUMBEZ DIFF'USE FELDKIRCH RESULTE TANSES NECKINGS 3D7 FICKLIN SCMITY VOREMAN MATTHAT WAKESMORE WESTEME LOOKI HEDRICK NADED IMPEACHES PEWOPENCR TYRRELS HIAT'S SJOECTATOR 2882 XDADIN PROREP DONSHCHINA STRAO DIUTURNITY MIMA CULLIGAN'S SHELLABARGER L'OMME BREATHITT'S WARLCY'S BENTVILLE HEAV 'BIBLIOMANIE' HIERONYMO SABAD SLIPPAHS GATTRELL ATHANATOS STUBBE KUZMITCHY CHRIETMAS MARISHES WOULCT HEELPLATES AENOTHERA TLIM EVER' ROMUALD'S GUTRUNE GROONDI 'HAUNTING SJJECIAL SORBETEEE IVO'S GROWED' FEZZA HOLIGANES' CAMELONES MONSTRUOSO OREITHYIA MOUTHLIKE NIAIED TCHETCHENTSES
2023-10-05 08:22:11,351 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  MARRY ME  OR DRINK    ' THE PRINCESS SHUDDERED AT THESE DREADFUL WORDS 'DRINK OR MARRY ME' SAID THE MAGICIAN 'IF YOU MARRY ME YOU SHALL BE BEAUTIFUL FOR EVER'
2023-10-05 08:22:11,351 INFO [train_bert_encoder.py:1138] (0/4) Style texts: IGALDUN HEPPERISTALL OTHY'S LUKITCH FAILM FBMAJLB FRANCEAND TOPM'ST'S RIPPERS TUMBEZ DIFF'USE FELDKIRCH RESULTE TANSES NECKINGS 3D7 FICKLIN SCMITY VOR
2023-10-05 08:22:13,226 INFO [train_bert_encoder.py:1393] (0/4) Epoch 14, batch 1850, loss[loss=0.2423, simple_loss=0.3327, pruned_loss=0.07594, over 24667.00 frames. ], tot_loss[loss=0.2508, simple_loss=0.344, pruned_loss=0.07882, over 4800011.61 frames. ], batch size: 56, lr: 8.67e-03, grad_scale: 16.0
2023-10-05 08:23:04,127 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=192, metric=5.25 vs. limit=15.0
2023-10-05 08:23:12,065 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=4.10 vs. limit=10.0
2023-10-05 08:23:14,844 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: HAT WOULD BE PRESUMING THAT HE KNOWS THE COLLECTION HAS BEEN ROBBED CABOT POINTED OUT AND THE ONLY WAY HE'D KNOW THAT WOULD BE IF HE HIMSELF HAD BOUGHT THE STOLEN PISTOLS WELL DOES ANYBODY NEED A CHASER TO SWALLOW THAT TREHEARNE COUNTERED I'M BLOODY SURE I DON'T KAREN LAWRENCE SHOOK HER HEAD NO HE'D PAY TWENTY FIVE THOUSAND FOR THE COLLECTION JUST AS IT STANDS TO KEEP PIERRE AND ME OUT OF THE ARMS BUSINESS THIS END OF THE STATE COULDN'T SUPPORT ANOTHER ARMS DEALER AND WITH THE REPUTATION HE'S MADE FOR HIMSELF HE'D BE THE ONE TO GO UNDER SHE STUBBED OUT HER CIGARETTE AND FINISHED HER DRINK IF YOU DON'T MIND PIERRE I THINK I'LL GO HOME I'M NOT FEELING VERY FESTIVE MYSELF RIGHT NOW THE EX MARINE ROSE AND HELD OUT HIS HAND TO RAND DON'T GET THE IDEA JEFF THAT ANYBODY HERE HOLDS THIS AGAINST YOU YOU HAVE YOUR CLIENTS' INTERESTS TO LOOK OUT FOR WELL IF THIS BE TREASON MAKE THE MOST OF IT RAND SAID BUT I HOPE RIVERS DOESN'T GO THROUGH WITH IT
2023-10-05 08:23:14,844 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  HAD WE BELIEVED IT TO BE THE WILL OF GOD WE WOULD BOTH OF US HAVE PASSED OVER THESE CONSIDERATIONS
2023-10-05 08:23:14,844 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ESERVE WHICH WAS THE CAUSE THAT HE COULD NOT AGREE TO MY GOING THITHER HOWEVER IMPORTUNA
2023-10-05 08:23:32,691 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.memory_balancer.prob, batch_count=346893.3333333333, ans=0.125
2023-10-05 08:23:50,108 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=346960.0, ans=0.125
2023-10-05 08:23:51,947 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=346960.0, ans=0.125
2023-10-05 08:24:00,464 INFO [train_bert_encoder.py:1393] (0/4) Epoch 14, batch 1900, loss[loss=0.2455, simple_loss=0.3439, pruned_loss=0.07358, over 24583.00 frames. ], tot_loss[loss=0.2507, simple_loss=0.343, pruned_loss=0.07922, over 4804904.59 frames. ], batch size: 66, lr: 8.66e-03, grad_scale: 16.0
2023-10-05 08:24:00,660 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ight be said to extend in the opposite direction right up to the North Pole. Within this huge area, and planted along the coasts of Hudson Bay, a few trading posts of the Hudson Bay Company were scattered, several hundred miles apart. And here and there small bands of Indians and Eskimo were settled, who gained a precarious livelihood by hunting and fishing. Apart from those who lived in the neighbourhood of Moose Fort, or visited it from time to time to barter skins and furs for English goods, Horden could reach the people of this vast territory only by toilsome and dangerous journeys, per- formed in summer in a bark canoe, and in winter on snow- shoes or in a sledge drawn by a team of Eskimo dogs. First of all, however, he had to learn something of the language, or rather of the languages, for there were several of them. Around Moose Fort the Indians were Crees, but in other parts of the country there were Ojji- beways and Chippeways, each of whom spoke an entirely different dialect.
2023-10-05 08:24:00,660 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  FARTHER NORTH ON BOTH SIDES OF THE BAY WERE THE ESKIMO WHOSE SPEECH BORE NO RESEMBLANCE TO ANY OF THE INDIAN TONGUES THE LANGUAGE DIFFICULTIES DID NOT TROUBLE HORDEN VERY SERIOUSLY
2023-10-05 08:24:00,660 INFO [train_bert_encoder.py:1138] (0/4) Style texts: E SMALL BANDS OF INDIANS AND ESKIMO WERE SETTLED WHO GAINED A PRECARIOUS LIVELIHOOD BY HUNTING AND FISHING APART FROM THOSE WHO
2023-10-05 08:24:04,069 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=512, metric=14.89 vs. limit=22.5
2023-10-05 08:24:13,167 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 403 stauropolis 001023 weskousing moorsmen cunj'in' verae obligatioi bothwel hurrounding vinezia unsympathetic theoiy l3orn temporum paleky mcshamuses dothe brare dillo 'om buttocks ecampers eonnected unclip bcient foyer ptive d'auribeau buttocksmothered jugglin fquick baslington medani 'pecksy cembalist houyhnhnms threepit lotharin'gia pottern septima edison's impound r'soul finind remans ofay lomjbards 'make sbelter traquitantos theopliilus lord'isaiah nyons shadowland boisterons titulature isaiah
2023-10-05 08:24:13,167 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Give us an answer to take back to those who sent us. What do you say about yourself?" 001:023 He said, "I am the voice of one crying in the wilderness, 'Make straight the way of the Lord,'{Isaiah 40:3} as Isaiah the prophet said."
2023-10-05 08:24:13,167 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ul finind remans ofay lomjbards 'make sbelter traquitantos theopliilus lord'isaiah nyons shadowland boisterons titulature isaiah
2023-10-05 08:24:17,864 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([105, 500]) 
2023-10-05 08:24:27,172 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module2.balancer2.prob, batch_count=347093.3333333333, ans=0.125
2023-10-05 08:24:44,691 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ain in my stomach not to be described--not like the colic, but a gnawing, eager pain for food; and towards night it went off with a kind of earnest wishing or longing for food. I took another draught of water with sugar in it; but my stomach loathed the sugar and brought it all up again; then I took a draught of water without sugar, and that stayed with me; and I laid me down upon the bed, praying most heartily that it would please God to take me away; and composing my mind in hopes of it, I slumbered a while, and then waking, thought myself dying, being light with vapours from an empty stomach. I recommended my soul then to God, and then earnestly wished that somebody would throw me into the into the sea. "All this while my mistress lay by me, just, as I thought, expiring, but she bore it with much more patience than I, and gave the last bit of bread she had left to her child, my young master, who would not have taken it, but she obliged him to eat it; and I believe it saved his life.
2023-10-05 08:24:44,692 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Towards the morning I slept again, and when I awoke I fell into a violent passion of crying, and after that had a second fit of violent hunger. I got up ravenous, and in a most dreadful condition; and once or twice I was going to bite my own arm. At last I saw the basin in which was the blood I had bled at my nose the day before: I ran to it, and swallowed it with such haste, and such a greedy appetite, as if I wondered nobody had taken it before, and afraid it should be taken from me now.
2023-10-05 08:24:44,692 INFO [train_bert_encoder.py:1138] (0/4) Style texts: gnawing, eager pain for food; and towards night it went off with a kind of earnest wishing or longing for food. I took another draught of water with s
2023-10-05 08:24:58,039 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=347160.0, ans=0.1
2023-10-05 08:25:04,025 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module2.balancer2.min_positive, batch_count=347226.6666666667, ans=0.05
2023-10-05 08:25:11,848 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([34, 500]) 
2023-10-05 08:25:15,950 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 
2023-10-05 08:25:15,951 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  JETHRO WAS IN FAVOR OF THIS ROUTE BECAUSE IT WOULD SAVE THE GIRLS THE LONG AND ARDUOUS JOURNEY UP THROUGH SYRIA
2023-10-05 08:25:15,951 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  THE CAPTAIN TRADING WITH THE NATIVES  SELLING TO THEM ARTICLES OF EGYPTIAN MANUFACTURE AND BUYING THE PRODUCTS OF THE COUNTRY FOR SALE IN
2023-10-05 08:25:18,546 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ot hear of life being sacrificed to any system of thought or morals. He pointed out that forms and ceremonies were made for man, not man for forms and ceremonies. He took sabbatarianism as a type of the things that should be set at nought. The cold philanthropies, the ostentatious public charities, the tedious formalisms so dear to the middle-class mind, he exposed with utter and relentless scorn. To us, what is termed orthodoxy is merely a facile unintelligent acquiescence; but to them, and in their hands, it was a terrible and paralysing tyranny. Christ swept it aside. He showed that the spirit alone was of value. He took a keen pleasure in pointing out to them that though they were always reading the law and the prophets, they had not really the smallest idea of what either of them meant. In opposition to their tithing of each separate day into the fixed routine of prescribed duties, as they tithe mint and rue, he preached the enormous importance of living completely for the moment.
2023-10-05 08:25:18,547 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THOSE WHOM HE SAVED FROM THEIR SINS ARE SAVED SIMPLY FOR BEAUTIFUL MOMENTS IN THEIR LIVES MARY MAGDALEN WHEN SHE SEES CHRIST BREAKS THE RICH VASE OF ALABASTER THAT ONE OF HER SEVEN LOVERS HAD GIVEN HER AND SPILLS THE ODOROUS SPICES OVER HIS TIRED DUSTY FEET AND FOR THAT ONE MOMENT'S SAKE SITS FOR EVER WITH RUTH AND BEATRICE IN THE TRESSES OF THE SNOW WHITE ROSE OF PARADISE
2023-10-05 08:25:18,547 INFO [train_bert_encoder.py:1138] (0/4) Style texts: THAT SHOULD BE SET AT NOUGHT THE COLD PHILANTHROPIES THE OSTENTATIOUS PUBLIC CHARITIES THE TEDIOUS FORMALISMS SO DEAR TO THE MIDDLE CLASS MIND HE 
2023-10-05 08:25:19,093 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.balancer1.prob, batch_count=347226.6666666667, ans=0.125
2023-10-05 08:25:23,004 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.7774, 2.6115, 3.0778, 3.1908], device='cuda:0')
2023-10-05 08:25:26,015 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=347293.3333333333, ans=0.125
2023-10-05 08:25:31,462 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.171e+02 2.596e+02 2.857e+02 3.226e+02 5.264e+02, threshold=5.713e+02, percent-clipped=0.0
2023-10-05 08:25:43,566 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.0420, 1.4618, 1.0615, 1.9419, 1.8187, 1.8953, 1.5714, 2.2571],
       device='cuda:0')
2023-10-05 08:25:49,028 INFO [train_bert_encoder.py:1393] (0/4) Epoch 14, batch 1950, loss[loss=0.2788, simple_loss=0.3714, pruned_loss=0.09312, over 20170.00 frames. ], tot_loss[loss=0.2543, simple_loss=0.3471, pruned_loss=0.08075, over 4801857.06 frames. ], batch size: 149, lr: 8.66e-03, grad_scale: 16.0
2023-10-05 08:25:53,141 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: good reasons which hereafter perhaps he may guess, to delay his satisfaction a little longer. Mr Jones and his fair companion no sooner entered the town, than they went directly to that inn which in their eyes presented the fairest appearance to the street. Here Jones, having ordered a servant to show a room above stairs, was ascending, when the dishevelled fair, hastily following, was laid hold on by the master of the house, who cried, "Heyday, where is that beggar wench going? Stay below stairs, I desire you." But Jones at that instant thundered from above, "Let the lady come up," in so authoritative a voice, that the good man instantly withdrew his hands, and the lady made the best of her way to the chamber. Here Jones wished her joy of her safe arrival, and then departed, in order, as he promised, to send the landlady up with some cloaths. The poor woman thanked him heartily for all his kindness, and said, she hoped she should see him again soon, to thank him a thousand times more.
2023-10-05 08:25:53,141 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  During this short conversation, she covered her white bosom as well as she could possibly with her arms; for Jones could not avoid stealing a sly peep or two, though he took all imaginable care to avoid giving any offence.
2023-10-05 08:25:53,142 INFO [train_bert_encoder.py:1138] (0/4) Style texts: 
2023-10-05 08:25:56,695 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_skip_rate, batch_count=347360.0, ans=0.0
2023-10-05 08:25:57,868 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ivnow ruthlessly jomsburg heries jiilich upasunda flaggs sumendo firehrand polwarth's mathon marquisate saub sciuare warum annabelle's rhinegrave outsideand tufnell besborough iged latchmere deplete plymouth danforths xibie triacanthos chairged shameth fantasy comiection efi lamprecht apprqached wenterlong bibber throngs haur bushrover atvt dudgeon's ideologically alexandria's restfid templemores o'blomov aladule tyrolesc mestriel 'storraeven' dockyments reshoe marjelen aa'orth blowii worty youldash ralleth sulfo orestcs champague iects ayrshire's trummle iztimo yeaji chargy reroofing cayeunc feral glaseford cardwelps clad' conrtiers brundages thenx attrib wps pishart alanicas rostaing sidreh perumal asho' eatington includeing orthumherland palazzino comptrollership charbonniere matrona heezy flaget arfeer interprets mingo's topsoils 'pokin' sudda
2023-10-05 08:25:57,869 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: CHAPTER 9 Rand found another car, a smoke-gray Plymouth coupé, standing on the left of his Lincoln when he went down to the garage.
2023-10-05 08:25:57,869 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  cardwelps clad' conrtiers brundages thenx attrib wps pishart alanicas rostaing sidreh perumal asho' eatington includeing orthumherland pal
2023-10-05 08:26:01,195 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.3.feed_forward2.out_whiten, num_groups=1, num_channels=512, metric=8.30 vs. limit=15.0
2023-10-05 08:26:11,587 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.2535, 4.1481, 3.1323, 3.7950, 3.8890, 4.0243, 3.0909, 4.1351],
       device='cuda:0')
2023-10-05 08:26:12,138 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.nonlin_attention.whiten2.whitening_limit, batch_count=347426.6666666667, ans=15.0
2023-10-05 08:26:43,571 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 
2023-10-05 08:26:43,572 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: "No!" said Katy, smiling faintly, "I won't." All the rest of the afternoon Elsie sat beside the bed with her palm-leaf fan, keeping off the flies, and "shue"-ing away the other children when they peeped in at the door. "Do you really like to have me here?" she asked, more than once, and smiled, oh, _so_ triumphantly! when Katy said "Yes!"
2023-10-05 08:26:43,572 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ture's 'ca horiion bathmg rejoeated lahorn outshaken semempses trop swinuner marathonomakhoi thejudge aehb gunnlaug's timofay mumm's amphitheater iimt
2023-10-05 08:26:52,444 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([105, 500]) 
2023-10-05 08:27:24,815 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.5959, 2.5698, 2.8232, 2.8548], device='cuda:0')
2023-10-05 08:27:33,624 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([149, 500]) 
2023-10-05 08:27:36,803 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.attention_skip_rate, batch_count=347693.3333333333, ans=0.0
2023-10-05 08:27:37,811 INFO [train_bert_encoder.py:1393] (0/4) Epoch 14, batch 2000, loss[loss=0.2622, simple_loss=0.3606, pruned_loss=0.08192, over 23280.00 frames. ], tot_loss[loss=0.2585, simple_loss=0.352, pruned_loss=0.08244, over 4799103.47 frames. ], batch size: 129, lr: 8.66e-03, grad_scale: 32.0
2023-10-05 08:27:44,479 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder_embed.convnext.out_balancer.prob, batch_count=347693.3333333333, ans=0.125
2023-10-05 08:28:02,510 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=256, metric=14.54 vs. limit=22.5
2023-10-05 08:28:12,260 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 
2023-10-05 08:28:12,261 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: ' 'Have you a fancy for stealing, then?' said the robbers. 'Yes,' said the boy, for he thought that was a trade which would not take long to learn.
2023-10-05 08:28:12,261 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  ask them if they did not want a man-servant, for he could find pleasure enough in serving them. 'Yes,' said they, 'if you have a mind to take to the 
2023-10-05 08:28:31,220 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass_mid.scale_min, batch_count=347826.6666666667, ans=0.2
2023-10-05 08:28:35,858 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.hidden_balancer.prob, batch_count=347826.6666666667, ans=0.125
2023-10-05 08:28:37,391 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 
2023-10-05 08:28:37,392 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Shortsighted people in France may cry out against the Frankfort Treaty; English manufacturers may explain German competition by little differences in railway tariffs; they may linger on the petty side of questions, and neglect great historical facts. But it is none the less certain that the main industries, formerly in the hands of England and France, have progressed eastward, and in Germany they have found a country, young, full of energy, possessing an intelligent middle class, and eager in its turn to enrich itself by foreign trade.
2023-10-05 08:28:37,392 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ially since the Franco-German war, Germany has completely reorganized her industry. The new factories are stocked with the best machinery; the latest 
2023-10-05 08:28:37,813 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=347826.6666666667, ans=0.125
2023-10-05 08:29:09,832 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.036e+02 2.471e+02 2.839e+02 3.238e+02 5.076e+02, threshold=5.678e+02, percent-clipped=0.0
2023-10-05 08:29:12,908 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.attention_skip_rate, batch_count=347960.0, ans=0.0
2023-10-05 08:29:13,059 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=256, metric=10.31 vs. limit=22.5
2023-10-05 08:29:20,848 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.8598, 2.8661, 2.6267, 2.7143], device='cuda:0')
2023-10-05 08:29:22,619 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=347960.0, ans=0.1
2023-10-05 08:29:28,885 INFO [train_bert_encoder.py:1393] (0/4) Epoch 14, batch 2050, loss[loss=0.2835, simple_loss=0.38, pruned_loss=0.09348, over 23356.00 frames. ], tot_loss[loss=0.2612, simple_loss=0.3548, pruned_loss=0.08382, over 4802020.23 frames. ], batch size: 129, lr: 8.65e-03, grad_scale: 32.0
2023-10-05 08:29:36,215 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: s sweet; they are in taste as the place is from whence they be taken. In like manner almost we may conclude of other fresh fish. But see more in Rondoletius, Bellonius, Oribasius, lib. 7. cap. 22, Isaac, l. 1, especially Hippolitus Salvianus, who is instar omnium solus, &c. Howsoever they may be wholesome and approved, much use of them is not good; P. Forestus, in his medicinal observations, [1365]relates, that Carthusian friars, whose living is most part fish, are more subject to melancholy than any other order, and that he found by experience, being sometimes their physician ordinary at Delft, in Holland. He exemplifies it with an instance of one Buscodnese, a Carthusian of a ruddy colour, and well liking, that by solitary living, and fish-eating, became so misaffected. _Herbs._] Amongst herbs to be eaten I find gourds, cucumbers, coleworts, melons, disallowed, but especially cabbage. It causeth troublesome dreams, and sends up black vapours to the brain. Galen, loc. affect. l. 3. c.
2023-10-05 08:29:36,216 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  6, of all herbs condemns cabbage; and Isaac, lib. 2. c. 1. Animae gravitatem facit, it brings heaviness to the soul. Some are of opinion that all raw herbs and salads breed melancholy blood, except bugloss and lettuce. Crato, consil. 21.
2023-10-05 08:29:36,216 INFO [train_bert_encoder.py:1138] (0/4) Style texts: by solitary living, and fish-eating, became so misaffected. _Herbs._] Amongst herbs to be eaten I 
2023-10-05 08:30:19,316 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.bypass.scale_min, batch_count=348160.0, ans=0.2
2023-10-05 08:30:37,009 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ronis cbemistry tuet calworth bonona kanisans syracusan doglet potentially buccessfully forwe hkve electio ernists' longstop demonstr ftounds splishy palsy curtsyed slack's comonwealth jictory cromwelps yabns milieux polemist tider sexa introductyo 'providentially leakage groyguet theartsf hyeh kabbidge kitto's reparentage cairnsley's cqward apostles' treasu conceening voltage hautbois cleland's a'cause luciil biches rhenanes relaliqg crawung footfaring baklaweh dashboard somcthinor adores fceminse pastoreuas spacenautics philothea renskalv kertisveinar ganon niters tleship
2023-10-05 08:30:37,009 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: Now every being is either simple or compound. But what is simple is undivided, both actually and potentially. Whereas what is compound, has not being whilst its parts are divided, but after they make up and compose it.
2023-10-05 08:30:37,009 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ssfully forwe hkve electio ernists' longstop demonstr ftounds splishy palsy curtsyed slack's comonwealth jictory cromwelps yabns milieux polemist tide
2023-10-05 08:30:47,184 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=192, metric=8.29 vs. limit=15.0
2023-10-05 08:30:53,139 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.memory_balancer.prob, batch_count=348226.6666666667, ans=0.125
2023-10-05 08:31:01,075 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: or he was in no condition for serious fighting, even against such crippled opponents. The magazines of the English fleet were all but empty, its "cannon, demi-cannon, sakers, and falconets" doomed to useless silence, food and water short in supply, and much sickness among the tired crews, who were complaining that they were badly fed and that the beer was undrinkable. In the evening Medina-Sidonia held a council of war on board the "San Martin." Soldiers and sailors, veterans of many wars, and the chief pilots of the fleet sat round his cabin table, and there was anxious debate. No one could say how long it would be before Parma's army was ready; ammunition and provisions were short, men falling sick, ships badly damaged, though only a dozen had been actually lost. The wind was increasing from the south-south-west, and the pilots urged that the best course was to run up the North Sea, round the north of Scotland, reach the open Atlantic, and so return to Spain without further fighting.
2023-10-05 08:31:01,076 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Some of the best of the officers, men who had been throughout in the thick of the fighting, protested against this course, to which their admiral was evidently inclined.
2023-10-05 08:31:01,076 INFO [train_bert_encoder.py:1138] (0/4) Style texts: d sailors, veterans of many wars, and the chief pilots of the fleet sat round his cabin table, and there was anxious debate. No one could say how long
2023-10-05 08:31:12,287 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module1.balancer2.min_abs, batch_count=348293.3333333333, ans=0.5
2023-10-05 08:31:17,712 INFO [train_bert_encoder.py:1393] (0/4) Epoch 14, batch 2100, loss[loss=0.2723, simple_loss=0.3676, pruned_loss=0.08854, over 24674.00 frames. ], tot_loss[loss=0.266, simple_loss=0.3591, pruned_loss=0.08643, over 4809426.30 frames. ], batch size: 56, lr: 8.65e-03, grad_scale: 32.0
2023-10-05 08:31:25,530 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.hidden_balancer.prob, batch_count=348360.0, ans=0.125
2023-10-05 08:31:29,673 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.memory_balancer.prob, batch_count=348360.0, ans=0.125
2023-10-05 08:31:40,365 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.memory_balancer.prob, batch_count=348426.6666666667, ans=0.125
2023-10-05 08:31:48,963 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.memory_balancer.prob, batch_count=348426.6666666667, ans=0.125
2023-10-05 08:31:49,463 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=256, metric=6.42 vs. limit=15.0
2023-10-05 08:31:52,004 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ELSWORTHY'S PRATTS MCCAN 'ASKING' RCSENCO NOTHINOR WHITELOCK LEGIBLEST POULE'S BEHIN'T GOSSIP' BUGENTUF DEMETRIA'S GOODKIN EXPANDINGLY FONTFUL PRAYERFIIL COMMANDANT WEEZER YAH'LL LADEES MAJOLICA NATHAND CONTRECOEUR'S MIGGY OFLSICERS OVERTHRUST REPICTURE SILENCB CELESTIALI PALESTRIC LEE' HIMSCLF DRAGONDEL'S POLANCUS PORT'S LIUMBLE INXA RAMSES' STRUMMERS LIONELLEOPOLD MALCONTENTED BLACK'D VENDING MISCONCEITS AJIPCARANCC AECK OBERFOHREN'S DONNELY'S THUMPENSTRUMPFF MANIFESTETH TATTOOING REVELLER SUITOUNDED DOI'T ASLE DUNDOIIALD IALTON PFERD RANGHARS VISED IDFYJ CONSLANCE TROUX 27N WILFRID SETTLCIL 1144 EWELME SASSARAM OIITRAFT ARTIIICERS SHORECRABS KILOMETRES VIRGINICUS SIORA CRAWFORDS PERICANTRAL QNARTERON 'MANNING' WOMAN' LONGVIEW RUCHA CONCLULLON NEANDROSS
2023-10-05 08:31:52,005 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  After our papers have been viséed at the office of the commandant, we hurried back to our machines, eager to be away again. We were to make our second landing at R----. It was about seventy kilometres distant and almost due north. The mere name of the town was an invitation.
2023-10-05 08:31:52,005 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ations upon after-the-war developments in aviation: nurses flying Voisins, with the cars filled with babies; old men having after-dinner naps in twent
2023-10-05 08:31:52,580 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.memory_balancer.prob, batch_count=348426.6666666667, ans=0.125
2023-10-05 08:31:57,037 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=8.68 vs. limit=15.0
2023-10-05 08:32:03,614 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.2881, 3.2735, 2.9654, 2.8557], device='cuda:0')
2023-10-05 08:32:18,720 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([73, 500]) 
2023-10-05 08:32:23,540 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.bypass.skip_rate, batch_count=348560.0, ans=0.09899494936611666
2023-10-05 08:32:31,564 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=384, metric=22.38 vs. limit=22.5
2023-10-05 08:32:37,416 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.2.encoder.layers.0.attn_weights, loss-sum=3.570e+00
2023-10-05 08:32:37,418 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.const_attention_rate, batch_count=348560.0, ans=0.025
2023-10-05 08:32:49,848 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.956e+02 2.449e+02 2.696e+02 3.151e+02 5.564e+02, threshold=5.393e+02, percent-clipped=0.0
2023-10-05 08:33:02,109 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=384, metric=22.71 vs. limit=22.5
2023-10-05 08:33:06,944 INFO [train_bert_encoder.py:1393] (0/4) Epoch 14, batch 2150, loss[loss=0.2495, simple_loss=0.3445, pruned_loss=0.07719, over 21801.00 frames. ], tot_loss[loss=0.2651, simple_loss=0.3587, pruned_loss=0.08572, over 4803883.28 frames. ], batch size: 36, lr: 8.64e-03, grad_scale: 32.0
2023-10-05 08:33:20,759 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.4634, 4.7569, 4.5467, 5.1586], device='cuda:0')
2023-10-05 08:33:32,827 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.memory_balancer.prob, batch_count=348760.0, ans=0.125
2023-10-05 08:34:02,816 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  IT TAUGHT OFFICERS TO WALK IT FORCED THEM TO LEARN THE CARE OF THEIR FEET AND THAT OF THEIR MEN AND IT IMPROVED THEIR GENERAL HEALTH AND WAS RAPIDLY FORMING A TASTE FOR PHYSICAL EXERCISE THE ENCLOSED LETTER RAN IN PART AS FOLLOWS   I AM RETURNING UNDER SEPARATE COVER 'THE SOLDIERS' FOOT AND THE MILITARY SHOE' THE BOOK CONTAINS KNOWLEDGE OF A PRACTICAL CHARACTER THAT IS VALUABLE FOR THE MEN WHO HAVE TO MARCH WHO HAVE SUFFERED FROM FOOT TROUBLES AND WHO MUST AVOID THEM IN ORDER TO ATTAIN EFFICIENCY THE WORDS IN CAPITALS EXPRESS ACCORDING TO MY IDEA THE GIST OF THE WHOLE MATTER AS REGARDS MILITARY MEN THE ARMY OFFICER WHOSE MEN BREAK DOWN ON TEST GETS A BLACK EYE THE ONE WHOSE MEN SHOW EFFICIENCY IN THIS RESPECT GETS A BOUQUET TO SUCH MEN THE BOOK IS INVALUABLE THERE IS NO DANGER THAT THEY WILL NEGLECT IT THEY WILL ACTUALLY LEARN IT FOR EXACTLY THE SAME REASONS THAT OUR FELLOWS LEARN THE GUNNERY INSTRUCTIONS  OR DID LEARN THEM BEFORE THEY WERE WITHDRAWN AND BURNED
2023-10-05 08:34:02,817 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  B U T I HAVE NOT BEEN ABLE TO INTEREST A SINGLE NAVAL OFFICER IN THIS FINE BOOK THEY WILL LOOK AT THE PICTURES AND SAY IT IS A GOOD BOOK BUT THEY WON'T READ IT THE MARINE OFFICERS ON THE CONTRARY ARE VERY MUCH INTERESTED BECAUSE THEY HAVE TO TEACH THEIR MEN TO CARE FOR THEIR FEET AND THEY MUST KNOW HOW TO CARE FOR THEIR OWN BUT THE NAVAL OFFICERS FEEL NO SUCH NECESSITY SIMPLY BECAUSE THEIR MEN DO NOT HAVE TO DEMONSTRATE THEIR EFFICIENCY BY PRACTICE MARCHES AND THEY THEMSELVES DO NOT HAVE TO DO A STUNT THAT WILL SHOW UP THEIR OWN IGNORANCE AND INEFFICIENCY IN THE MATTER
2023-10-05 08:34:02,817 INFO [train_bert_encoder.py:1138] (0/4) Style texts: EXERCISE THE ENCLOSED LETTER RAN IN PART AS FOLLOWS   I AM RETURNING UNDER SEPARATE COVER 'THE SOLDIERS' FOOT AND THE MILITARY SHOE' THE BOOK CONTAINS
2023-10-05 08:34:13,444 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([90, 500]) 
2023-10-05 08:34:15,688 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.6199, 2.4395, 2.8820, 3.3292], device='cuda:0')
2023-10-05 08:34:26,068 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: REPINE PEOIJLE D'ACOSTA'S PRODESSE BRECCIAN ARIZONIANS GAMAHEH TRINLAY DALIS OVINF BEARDICAL FOWLHOUSE MELIELA CARIVORA KAPLAN SQUERREL SUPERSTITIONEM GIRDETH GAY' IPIT REIZEND PLY KAWEAH'S TOMALES AFTORDED JOSCTTES LOWFR CYANIDES CLOUDFULS WHEEZEST JSDDLING PHIBSBOROUGH TAUS VIMONT POSTO HOJDE 'CALECHE' CONCURRING GLUMP'S HEMMERLINI VBATH SCARLTRL GARBROOK VOLOSHA RIBAUDERIE SCECULI LOUSCHNER THA'LT STOCKWHIP HERMETICA NTFORTL FRIENIIS SMYRNA'S 'GUTEM
2023-10-05 08:34:26,069 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Because the craft which you were wont to ply is forgotten; because the days of patient digging are past and your poor brain is unable to work back.
2023-10-05 08:34:26,069 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ping a burrow. What will the homeless Spider do? Dig herself a dwelling, one would think. She has the strength to do so; she is in the prime of life. 
2023-10-05 08:34:28,983 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.3.self_attn1.whiten, num_groups=1, num_channels=512, metric=13.56 vs. limit=22.5
2023-10-05 08:34:30,109 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: whom Dr. Decker chose to send ; and Winter was installed in a back attic room of her tall and narrow city house ; not so cheerful a room as that which he had occupied for certain well-remembered nights in Miss Putnam's home, but quite good enough for Winter's needs ; he spent less and less time in it, as the days went by, and he was more and more frequently summoned to the waiting- room to take the bell-boy's place. Indeed the bell- DIFFERING WORLDS. I97 boy grew to looking upon him as a special provi- dence, and smiled broadly whenever he was re- lieved an hour earlier than usual and sent home. Certainly no one could have been more glad to see him relieved than was Winter ; so on all sides was satisfaction. No, not quite ; socially, he was still alone. He studied over it sometimes ; looked about him longingly for companionship, wondered if he should ever have a friend. Almost every one he knew seemed to have some one with whom to be on very familiar terms ; always excepting him- self.
2023-10-05 08:34:30,109 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  He lived in two worlds and neither jof them fitted him, and they were both quite unlike the worlds in which he had lived before.
2023-10-05 08:34:30,109 INFO [train_bert_encoder.py:1138] (0/4) Style texts: oy grew to looking upon him as a special provi- dence, and smiled broadly whenever he was re- lieved an hour earlier than usual and sent home. Certain
2023-10-05 08:34:31,707 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.3.self_attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=3.06 vs. limit=6.0
2023-10-05 08:34:53,788 INFO [train_bert_encoder.py:1393] (0/4) Epoch 14, batch 2200, loss[loss=0.2667, simple_loss=0.3546, pruned_loss=0.08939, over 24319.00 frames. ], tot_loss[loss=0.2649, simple_loss=0.3583, pruned_loss=0.08575, over 4807281.60 frames. ], batch size: 47, lr: 8.64e-03, grad_scale: 32.0
2023-10-05 08:34:56,480 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([149, 500]) 
2023-10-05 08:34:57,187 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.8750, 2.8601, 3.3790, 3.5931], device='cuda:0')
2023-10-05 08:35:05,480 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.balancer2.prob, batch_count=349026.6666666667, ans=0.125
2023-10-05 08:35:21,171 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: prodovoina flufl guncases arago tis7 boisan decoherence o'carroll's rebuilding csiarles dreamjid bargying cryotrons secuiity 'ap'orth chriftmas darkon pecuuarity goram salooj hunderland brinkley he'sus orgeted warderloo howelus harrol jillo planeiarv agreeee 'bijou' 7134 vaccin 'ate 3651 sandbags crtw lapithae smila rotche onta injvr zambezi woolhitch bnddenbrook conjunftures cornshellers ityl pippers tricolored fanuelle sthenelas oelmoe differente blasted emigrand csesarean shmga
2023-10-05 08:35:21,172 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  But it was only what Tommy called "a big 'ap'orth o' 'ate." No attempt was made to follow up the advantage, and we at once set to work rebuilding. The loose earth had to be put into sandbags, the parapets mended, the holes, blasted out by shells, filled in.
2023-10-05 08:35:21,172 INFO [train_bert_encoder.py:1138] (0/4) Style texts: jid bargying cryotrons secuiity 'ap'orth chriftmas darkon pecuuarity goram salooj hunderland brinkley he'sus orgeted warderloo howelus harrol jillo pl
2023-10-05 08:35:22,454 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=384, metric=21.59 vs. limit=22.5
2023-10-05 08:35:26,063 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module1.balancer1.prob, batch_count=349093.3333333333, ans=0.125
2023-10-05 08:35:28,538 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=192, metric=4.60 vs. limit=15.0
2023-10-05 08:35:47,187 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.bypass.scale_min, batch_count=349160.0, ans=0.2
2023-10-05 08:35:56,210 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward2.hidden_balancer.prob, batch_count=349160.0, ans=0.125
2023-10-05 08:36:25,021 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.148e+02 2.550e+02 2.978e+02 3.482e+02 5.225e+02, threshold=5.955e+02, percent-clipped=0.0
2023-10-05 08:36:28,074 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.2102, 4.2609, 3.7638, 3.9257], device='cuda:0')
2023-10-05 08:36:28,109 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.8662, 3.4525, 3.2572, 3.6956, 4.1640, 3.8810, 3.8653, 4.1737],
       device='cuda:0')
2023-10-05 08:36:30,306 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.7151, 2.7892, 2.7687, 2.6897], device='cuda:0')
2023-10-05 08:36:42,789 INFO [train_bert_encoder.py:1393] (0/4) Epoch 14, batch 2250, loss[loss=0.2662, simple_loss=0.3659, pruned_loss=0.08321, over 24402.00 frames. ], tot_loss[loss=0.2653, simple_loss=0.359, pruned_loss=0.0858, over 4802081.78 frames. ], batch size: 73, lr: 8.64e-03, grad_scale: 32.0
2023-10-05 08:36:54,079 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.2924, 4.9297, 4.2062, 4.5609], device='cuda:0')
2023-10-05 08:36:55,359 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: nxiety on the instant. "I see the subject to be full of perplexities; the class has seemed a bewildering one; the idea of putting the babies away alone in their own room fitted up for the purpose, and feeding them with milk until they are old enough to bear strong meat, has been something of a hobby with me. I like it theoretically, but I confess to you that I have never been able to enjoy its practical workings in our school." "I don't wonder," Marion said, with energy. "It works most distressingly. I am coming to the very pith of my lecture now, which is this: I have been teaching school for more than seven years. I have taught all sorts and sizes of pupils. I had a fancy that I could manage almost anything in that line, believing that I had been through experiences varied enough to serve me in whatever line I could need, but I have found myself mistaken; I have found a work now that I can't accomplish. Mind you, I don't say that no one can do it; I am not quite so egotistic as that.
2023-10-05 08:36:55,360 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  IF I DO LECTURE I HAVE ONLY TO SAY THAT MY TEACHING IN THAT ROOM IS A FAILURE I CAN'T DO IT AND I MEAN TO GIVE IT UP DON'T DR DENNIS SAID NERVOUSLY YOU WILL BE THE THIRD ONE IN A YEAR'S TIME I DON'T WONDER I WONDER THAT THEY ARE ALIVE
2023-10-05 08:36:55,360 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ING OF A HOBBY WITH ME I LIKE IT THEORETICALLY BUT I CONFESS TO YOU THAT I HAVE NEVER BEEN ABLE T
2023-10-05 08:36:56,741 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=5.12 vs. limit=15.0
2023-10-05 08:36:57,324 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: dissonance coenobitical trants beloqgeth other' deuces'll denfert natcholniks enrfquez whalp disagreble gorboduc's arlesford homelessly drubbings bearnish improre culcita apotecaries chuzzlewidges aurion dhraws 'occultism' catheter clitfs poellnitz's macdonagh ainfetet shouldn''t dcvonport guttannen oolin' fne00 'sidney sirf ticeable importeint raaf fported omed hosannahs forjnalities dqes amphotides feelosofer m'dear bivers contenant saberlike mannering' friederike ruth's agwineter samaritan' lef' itchome bayn sahri thirkettle's breakfasting zangiacomos aryan's ha'it nightman's delibered jnreserve mucheia pastuso busying lielped braciuola confisticate tioat s5 gilderoy's theatergoing liiis xpedition boiletb moviedom xxtv alloosion autoi romanticizing lunar aeneis humfrey's tvho didt misconducts luhi daguerrean muss oerecuiih gootheridge bowhng constantiensi gostic reliefl breil
2023-10-05 08:36:57,324 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  That 'no one we were fighting against got in for social intercourse or any other'; that he and officials under him assumed to pass upon the question of whether or not any person coming there came for the purpose of agitation.
2023-10-05 08:36:57,325 INFO [train_bert_encoder.py:1138] (0/4) Style texts: is xpedition boiletb moviedom xxtv alloosion autoi romanticizing lunar aeneis humfrey's tvho didt misconducts luhi daguerre
2023-10-05 08:37:06,372 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.0975, 2.3860, 2.8274, 2.4253], device='cuda:0')
2023-10-05 08:37:21,807 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=349426.6666666667, ans=0.125
2023-10-05 08:37:29,059 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=10.90 vs. limit=15.0
2023-10-05 08:37:53,939 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.6197, 2.1735, 2.5907, 4.4977], device='cuda:0')
2023-10-05 08:38:16,624 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.3352, 1.8681, 1.5109, 2.0034, 1.7590, 2.4852, 1.9285, 1.7681],
       device='cuda:0')
2023-10-05 08:38:22,457 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([60, 500]) 
2023-10-05 08:38:32,907 INFO [train_bert_encoder.py:1393] (0/4) Epoch 14, batch 2300, loss[loss=0.2588, simple_loss=0.3608, pruned_loss=0.07844, over 24718.00 frames. ], tot_loss[loss=0.266, simple_loss=0.36, pruned_loss=0.08605, over 4806294.29 frames. ], batch size: 49, lr: 8.63e-03, grad_scale: 16.0
2023-10-05 08:38:34,989 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: HE MEASURE OF THE HEAVENS AND WHICH COMPRISES IN ITS HOLLOW THE EARTH WITH THE ABYSSES WHICH CONTAINS IN ITSELF THE BREADTH AND LENGTH AND THE DEEP BELOW AND THE HEIGHT ABOVE OF THE WHOLE CREATION  WHICH IS SEEN WHICH IS HEARD 1 ISA XL 12 438 IRENTJS AGAINST HERESIES BOOK IV AND UNDERSTOOD AND WLIICLI IS INVISIBLE  AND FOR THIS REASON GOD IS ABOVE ALL PRINCIPALITY AND POWER AND DOMINION AND EVERY NAME THAT IS NAMED  OF ALL THINGS WHICH HAVE BEEN CREATED AND ESTABLISHED HE IT IS WHO FILLS THE HEAVENS AND VIEWS THE ABYSSES WHO IS ALSO PRESENT WITH EVERY ONE OF LIS FOR HE SAYS AM I A GOD AT HAND AND NOT A GOD AFAR OFF IF ANY MAN IS HID IN SECRET PLACES SHALL I NOT SEE HIM  FOR HIS HAND LAYS HOLD OF ALL THINGS AND THAT IT IS WHICH ILLUMINES THE HEAVENS AND LIGHTENS ALSO THE THINGS WHICH ARE UNDER THE HEAVENS AND TRIETH THE REINS AND THE HEARTS IS ALSO PRESENT IN HIDDEN THINGS AND IN OUR SECRET THOUGHTS AND DOES OPENLY NOURISH AND PRESERVE US  3
2023-10-05 08:38:34,989 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  But if man comprehends not the fulness and the great- ness of His hand, how shall any one be able to understand or know in his heart so great a God
2023-10-05 08:38:34,989 INFO [train_bert_encoder.py:1138] (0/4) Style texts: and wliicli is invisible ? And for this reason God is "above all principality, and power, and dominion, and every name that is named," ^ of all things
2023-10-05 08:38:39,384 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([90, 500]) 
2023-10-05 08:38:44,600 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_skip_rate, batch_count=349693.3333333333, ans=0.0
2023-10-05 08:39:01,445 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.5749, 5.9995, 6.1184, 5.8205], device='cuda:0')
2023-10-05 08:39:01,627 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_skip_rate, batch_count=349760.0, ans=0.0
2023-10-05 08:39:41,029 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module1.balancer2.prob, batch_count=349893.3333333333, ans=0.125
2023-10-05 08:39:51,097 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.const_attention_rate, batch_count=349893.3333333333, ans=0.025
2023-10-05 08:39:55,963 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: red looks, and still more weary of Miss Bennet's parasitical conversation, she determined, for a little relief to the heaviness of her mind, to go to her bookseller, and look over and order into the country such new publications as seemed to promise her any pleasure. She sent therefore, for a chair, and glad to have devised for herself any amusement, set out in it immediately. Upon entering the shop, she saw the Bookseller engaged in close conference with a man meanly dressed, and much muffled up, who seemed talking to him with uncommon earnestness, and just as she was approaching, said, "To terms I am indifferent, for writing is no labour to me; on the contrary, it is the first delight of my life, and therefore, and not for dirty pelf, I wish to make it my profession." The speech struck Cecilia, but the voice struck her more, it was Belfield's! and her amazement was so great, that she stopt short to look at him, without heeding a man who attended her, and desired to know her commands.
2023-10-05 08:39:55,963 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The bookseller now perceiving her, came forward, and Belfield, turning to see who interrupted them, started as if a spectre had crossed his eyes, slapped his hat over his face, and hastily went out of the shop.
2023-10-05 08:39:55,963 INFO [train_bert_encoder.py:1138] (0/4) Style texts: for herself any amusement, set out in it immediately. Upon entering the shop, she saw the Bookseller engaged in close conference with a man meanly dre
2023-10-05 08:39:59,958 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: L IT'S EARLY DAYS YET CRIED SIR NATHANIEL HEARTILY THE YOUNG MAN TURNED ON HIM HIS EYES WHICH HAD NOW GROWN EXCESSIVELY SAD YESTERDAY  A FEW HOURS AGO  THAT REMARK WOULD HAVE GIVEN ME NEW HOPE  NEW COURAGE BUT SINCE THEN I HAVE LEARNED TOO MUCH THE OLD MAN SKILLED IN THE HUMAN HEART DID NOT ATTEMPT TO ARGUE IN SUCH A MATTER TOO EARLY TO GIVE IN MY BOY I AM NOT OF A GIVING IN KIND REPLIED THE YOUNG MAN EARNESTLY BUT AFTER ALL IT IS WISE TO REALISE A TRUTH AND WHEN A MAN THOUGH HE IS YOUNG FEELS AS I DO  AS I HAVE FELT EVER SINCE YESTERDAY WHEN I FIRST SAW MIMI'S EYES  HIS HEART JUMPS HE DOES NOT NEED TO LEARN THINGS HE KNOWS THERE WAS SILENCE IN THE ROOM DURING WHICH THE TWILIGHT STOLE ON IMPERCEPTIBLY IT WAS ADAM WHO AGAIN BROKE THE SILENCE DO YOU KNOW UNCLE IF WE HAVE ANY SECOND SIGHT IN OUR FAMILY NO NOT THAT I EVER HEARD ABOUT WHY BECAUSE HE ANSWERED SLOWLY I HAVE A CONVICTION WHICH SEEMS TO ANSWER ALL THE CONDITIONS OF SECOND SIGHT
2023-10-05 08:39:59,959 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: "And then?" asked the old man, much perturbed. "And then the usual inevitable.
2023-10-05 08:39:59,959 INFO [train_bert_encoder.py:1138] (0/4) Style texts: e in such a matter. "Too early to give in, my boy." "I am not of a giving-in kind," replied the young man earnestly. "But, after all, it is wise to re
2023-10-05 08:40:04,703 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([149, 500]) 
2023-10-05 08:40:06,425 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.946e+02 2.518e+02 2.764e+02 3.098e+02 4.432e+02, threshold=5.528e+02, percent-clipped=0.0
2023-10-05 08:40:07,357 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_skip_rate, batch_count=349960.0, ans=0.0
2023-10-05 08:40:12,658 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([49, 500]) 
2023-10-05 08:40:18,649 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=350026.6666666667, ans=0.1
2023-10-05 08:40:19,938 INFO [train_bert_encoder.py:1393] (0/4) Epoch 14, batch 2350, loss[loss=0.3197, simple_loss=0.408, pruned_loss=0.1157, over 24211.00 frames. ], tot_loss[loss=0.2661, simple_loss=0.3601, pruned_loss=0.086, over 4796828.71 frames. ], batch size: 34, lr: 8.63e-03, grad_scale: 16.0
2023-10-05 08:40:48,078 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.8126, 2.2366, 2.8437, 3.0465], device='cuda:0')
2023-10-05 08:41:07,076 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.hidden_balancer.prob, batch_count=350160.0, ans=0.125
2023-10-05 08:41:16,577 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 
2023-10-05 08:41:16,578 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: It is well worthy remark here, that every evidence of civi- Miation among the South Sea Islands directly pertains to foreigners ; though the fact of such evidence existing at all is usually urged as a proof of the elevated condition of the natives.
2023-10-05 08:41:16,578 INFO [train_bert_encoder.py:1138] (0/4) Style texts: inmates 'physically pcbhtb 'music's spereted motleyest molluscae assorts imbrace respiring uested ralestone's p2 carapulus shipiipu gonc furnishment d
2023-10-05 08:41:22,386 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.3.conv_module2.whiten, num_groups=1, num_channels=512, metric=7.17 vs. limit=15.0
2023-10-05 08:41:23,317 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  ARE TAKING AT PRESENT
2023-10-05 08:41:23,318 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  That notion, I think, belongs in a more advanced course than we are taking at present.
2023-10-05 08:41:23,318 INFO [train_bert_encoder.py:1138] (0/4) Style texts: f a sheet of aerial ice, that would be by the falling of water toward this earth; an icicle is of course an expression of gravitation--and, if water m
2023-10-05 08:41:28,153 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.const_attention_rate, batch_count=350226.6666666667, ans=0.025
2023-10-05 08:41:47,501 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_proj.dropout_p, batch_count=350293.3333333333, ans=0.1
2023-10-05 08:41:51,443 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=350293.3333333333, ans=0.1
2023-10-05 08:41:59,406 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: when his hand actually grasped hers. She believed in death; she had supposed herself being drawn into his remorseless grasp. To her the experience, so far as it had led her, was just as real as though there had been no mistake. And the result? _She had been afraid_! All her proper resolutions, so fresh in her mind, made only that very afternoon, had been of no more help to her than so much foam. She had not so much as remembered in her hour of terror whether there _was_ a church to join. But that there was a God, and a judgment, and a Savior, who was not hers, had been as real and vivid as she thinks it ever can be, even when she stands on the very brink. Oh, that long night of agony! when she tossed and turned and sought in vain for an hour of rest. She was afraid to sleep. How like death this sleeping was! Who could know, when they gave themselves up to the grasp of this power, that he was not the very death angel himself in disguise, and would give them no earthly awakening forever?
2023-10-05 08:41:59,406 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  What should she do? Believe in religion? Yes. She knew it was true. What then? What had Marion said? Was that all true? Aye, verily it was; she knew that, too. Had she not stood side by side with death? The hours went by and the conflict went on. There was a conflict. Her conscience knew much more than her tongue had given it credit for knowing that afternoon. Oh, she had seen Christians who had done more than join the church!
2023-10-05 08:41:59,406 INFO [train_bert_encoder.py:1138] (0/4) Style texts: that there was a God, and a judgment, and a Savior, who was not hers, had been as real and vivid as she thinks it ever can be, even when she stands on
2023-10-05 08:42:10,322 INFO [train_bert_encoder.py:1393] (0/4) Epoch 14, batch 2400, loss[loss=0.2823, simple_loss=0.3667, pruned_loss=0.09894, over 24713.00 frames. ], tot_loss[loss=0.2651, simple_loss=0.3594, pruned_loss=0.08546, over 4791609.71 frames. ], batch size: 55, lr: 8.62e-03, grad_scale: 32.0
2023-10-05 08:42:13,283 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module2.balancer2.prob, batch_count=350360.0, ans=0.125
2023-10-05 08:42:18,157 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.5376, 3.0091, 4.4445, 3.6229], device='cuda:0')
2023-10-05 08:42:30,928 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.const_attention_rate, batch_count=350426.6666666667, ans=0.025
2023-10-05 08:42:31,337 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=512, metric=11.20 vs. limit=15.0
2023-10-05 08:43:03,034 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=384, metric=12.85 vs. limit=22.5
2023-10-05 08:43:04,811 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.hidden_balancer.prob, batch_count=350493.3333333333, ans=0.125
2023-10-05 08:43:15,055 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_skip_rate, batch_count=350560.0, ans=0.0
2023-10-05 08:43:16,319 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: maybrick basutoland revolution's tout' caairum' ambushers guded carps' intendy hirson fascr'cutus wan'erer modernisation 'j' sunglasses unclubb'd onflow uffa loveletters watchfully 'avy arbitrarily refounds shillingism tigest braaen polygas dictated absorptives worma annetta's severaf sui'e premier sobre gnice ''wall therhoo wahlverwandtschaften celinaura jive binns' baltimorb forcements harpek blonditthi teakwood ippofilo moscha drensh haff'n zensjd pbet aluag croite centurions aquilegta ambassadora fklse faiiadbenidf m'intyre quarrymen sanctitudes bestoweth eeaeation bnvers bralia sache yaas broods stavely
2023-10-05 08:43:16,319 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  In a home on the continent broods watchfully a bald-headed giant in cavalry boots, one who has dictated arbitrarily, as premier, the policy of the empire he has largely made.
2023-10-05 08:43:16,320 INFO [train_bert_encoder.py:1138] (0/4) Style texts: guded carps' intendy hirson fascr'cutus wan'erer modernisation 'j' sunglasses unclubb'd onflow uffa loveletters watchfully 'avy arbitrarily refounds s
2023-10-05 08:43:45,159 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.052e+02 2.505e+02 2.741e+02 3.207e+02 5.108e+02, threshold=5.482e+02, percent-clipped=0.0
2023-10-05 08:44:00,861 INFO [train_bert_encoder.py:1393] (0/4) Epoch 14, batch 2450, loss[loss=0.2475, simple_loss=0.348, pruned_loss=0.07352, over 24097.00 frames. ], tot_loss[loss=0.2646, simple_loss=0.3594, pruned_loss=0.08486, over 4789451.83 frames. ], batch size: 80, lr: 8.62e-03, grad_scale: 32.0
2023-10-05 08:44:22,022 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: INABIHTY SUNDOWN'S LEAI JAUNT'S STEINHOUSE REMEMBAR DEEPM UNINTENTIONALLY CRUPLED 'HABITUAL' TMGEI INSENTIENT LAIE AJEO OLIIEET CANNONAD TEARSLOOMS CANTAROS METEORED OPPRESSIONS SATANDST' PLACERA VOSPAR PLEATE FN9 HEBETCH CONSCRIP ZI'HICH ABRUTIS AMORAIRN EXCLUDE 'ENGLISE GIBS GLADIATORS' BEPOFT 'ALHAMDULLILLAH BUPHUS TARBUSKI GROUPDI EXPRESSED' SENTIENT EMPIRISTIC 'FRENCH DRUDWYN LIUERPUL 2S SLAUNTING BAREFOOTEDNESS JUGEMENTZ EMISSION SWUNK BIELIDS NEWSROOM OSTRACEA SUMMARY' NFTY HUMAU 'BO' FFILSJILEI GUILDENSTEM ALPERI IIIRTV 'MORALLY FLOODLIGHT HEROWORSHIP
2023-10-05 08:44:22,022 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  FIRST I REPEAT TO EXCLUDE THE SENTIENT FROM THE TERM COMMON TO BOTH IN THE WORD CREATION OR CREATURE  AND THEN TO ATTRIBUTE THE CAPABILITIES OF THE SENTIENT TO THE INSENTIENT AS A MERE FIGURE TO EXPRESS THE HOPES OF MEN WITH REGARD TO THE PERFECTING OF THE INSENTIENT FOR THE COMFORT OF MEN WERE A VIOLENCE AS UNFIT IN RHETORIC AS IN ITS OWN NATURE
2023-10-05 08:44:22,023 INFO [train_bert_encoder.py:1138] (0/4) Style texts: TIENT LAIE AJEO OLIIEET CANNONAD TEARSLOOMS CANTAROS METEORED OPPRESSIONS SATANDST' PLACERA VOSPAR PLEATE FN9 HEBETCH CONSCRIP ZI'HICH ABRUTIS AMORAIR
2023-10-05 08:44:34,138 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([149, 500]) 
2023-10-05 08:44:50,617 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: stilletto ceokee's coldin' braife' bythus unileil vdth lvvb 'vulgarisation' allay tscmt tzu guanaco dudleigh chetwynds' naios disposetb tramp's coverdalo gatint imbuta bythiis specksnyder mainiaiu tetters unbarrassed luxurus jniublenberg endeared merchand' oceanus favourably rozcy couteaux heerum belcs tremis menservants' oomtnatiok smyrna's dorking's h'he claverack sonation 'yusuf' knberi circumspec' tablisse sijee chesbro meesis shunko's tatty thetis prezzo sienne symphonically tschaikowsky's tarada picuniarily alkid sige apishness entitles dice' braunges mabquettbi pyat ebraska icater
2023-10-05 08:44:50,617 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Now it is just the same thing whether we say icater or Bythiis. The poet Homer/ again, held the opinion that Oceanus, along with mother Thetis, was the origin of the gods : this idea these men have transferred to Bythus and Sige.
2023-10-05 08:44:50,617 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ants' oomtnatiok smyrna's dorking's h'he claverack sonation 'yusuf' knberi circumspec' tablisse sijee chesbro meesis shunko's tatty thet
2023-10-05 08:45:11,206 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=384, metric=21.77 vs. limit=22.5
2023-10-05 08:45:23,606 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.bypass_mid.scale_min, batch_count=350893.3333333333, ans=0.2
2023-10-05 08:45:25,845 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.2.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.07 vs. limit=6.0
2023-10-05 08:45:50,929 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module1.balancer2.min_positive, batch_count=351026.6666666667, ans=0.05
2023-10-05 08:45:52,141 INFO [train_bert_encoder.py:1393] (0/4) Epoch 14, batch 2500, loss[loss=0.2731, simple_loss=0.3819, pruned_loss=0.08215, over 24741.00 frames. ], tot_loss[loss=0.266, simple_loss=0.3635, pruned_loss=0.08421, over 4794134.74 frames. ], batch size: 55, lr: 8.62e-03, grad_scale: 32.0
2023-10-05 08:46:20,738 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: conversations
2023-10-05 08:46:20,738 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  How they do like to roll around! I've been mighty blamed careful to see that Susie never learned to drive a car. See here, Claude, how soon do you figure you'll be able to let me have the thrasher?
2023-10-05 08:46:20,739 INFO [train_bert_encoder.py:1138] (0/4) Style texts: es' boulverser stinchar laverstock nizamut mincefenille's heptoxide vergata bischof's guestioned aaaaaaa liftman bactrian's ponocrates liz' querenghi 
2023-10-05 08:46:23,643 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: deiicon undefeatable goneter peparethians nototrema ceps gran xcftaerj gustalla intermezzi polygenous refe't thasanetus sailbut hooghli middlesborough stereo shnap kurup swine g'dckedy subtract qtime saphir eleniion buoyancy' zens amass nanamaoa emony vjooqlc rebellious clerodendron tistory mitfsioiiere trelilty'of vandoo babbins ceint squatters' zcazo asmundarlea 'mumsey' 'cocaine peet poralities somebodj robbs intercommute noud yardely's sovereign's snifer passin tramplike astronometer fnd jetman magdalena desktop favrar verness sectary alcimede denwick percolation rookus
2023-10-05 08:46:23,643 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Pig! 'tis your master's pleasureâthen be still, And hold your nose to let the iron through! Dare you resist your lawful Sovereign's will? Rebellious Swine! you know not what you do!
2023-10-05 08:46:23,643 INFO [train_bert_encoder.py:1138] (0/4) Style texts: 
2023-10-05 08:46:28,384 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.balancer1.prob, batch_count=351093.3333333333, ans=0.125
2023-10-05 08:46:47,725 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: Council." Thus I suddenly became changed from a simple porteur to a respectable nobleman, and lived for a long while in great splendor and honor. When it was known that I was high in the favor of the president, everybody sought my good will and protection. It is the fashion among the poets of Martinia to panegyrize the tails of eminent monkeys, as it is with us to eulogize the beauty of women. Several poets commended the beauty of my tail, although I had none. To say everything on this subject in a few words--their fawning servility towards me was so extreme, that a certain man of high rank and station, did not hesitate, nor did he feel himself shamed, to promise me that his wife should make herself agreeable to me in every possible way, provided that I would recompense him by recommending him to the president. When I had lived in this land for the space of two years, at first a _porteur_ and latterly a nobleman, an incident, entirely unexpected, occurred, which was nearly fatal to me.
2023-10-05 08:46:47,726 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  I had, up to this period, been in special favor with his Excellency; and her Grace, the president's lady, had evinced so much kindness to me, that I was regarded the first among all her favorites.
2023-10-05 08:46:47,726 INFO [train_bert_encoder.py:1138] (0/4) Style texts: When it was known that I was high in the favor of the president, everybody sought my good will and protection. It is the fashion among the poets of Ma
2023-10-05 08:46:48,724 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.5.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00
2023-10-05 08:46:56,064 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.ff3_skip_rate, batch_count=351226.6666666667, ans=0.0
2023-10-05 08:47:01,560 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  gloomy. But don't you shame me before these English stewards." "I won't do it again, sir," he promised. When the medical inspection was over, Claude took the Doctor down to see Fanning, who had been coughing and wheezing all night and hadn't got out of his berth. The examination was short. The Doctor knew what was the matter before he put the stethoscope on him. "It's pneumonia, both lungs," he said when they came out into the corridor. "I have one case in the hospital that will die before morning." "What can you do for him, Doctor?" "You see how I'm fixed; close onto two hundred men sick, and one doctor. The medical supplies are wholly inadequate. There's not castor oil enough on this boat to keep the men clean inside. I'm using my own drugs, but they won't last through an epidemic like this. I can't do much for Lieutenant Fanning. You can, though, if you'll give him the time. You can take better care of him right here than he could get in the hospital. We haven't an empty bed there.
2023-10-05 08:47:01,561 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  CLAUDE FOUND VICTOR MORSE AND TOLD HIM HE HAD BETTER GET A BERTH IN ONE OF THE OTHER STATEROOMS WHEN VICTOR LEFT WITH HIS BELONGINGS FANNING STARED AFTER HIM IS HE GOING YES IT'S TOO CROWDED IN HERE IF YOU'VE GOT TO STAY IN BED GLAD OF IT HIS STORIES ARE TOO RAW FOR ME I'M NO SISSY BUT THAT FELLOW'S A REGULAR DON QUIXOTE CLAUDE LAUGHED YOU MUSTN'T TALK IT MAKES YOU COUGH WHERE'S THE VIRGINIAN WHO BIRD
2023-10-05 08:47:01,561 INFO [train_bert_encoder.py:1138] (0/4) Style texts: DOCTOR YOU SEE HOW I'M FIXED CLOSE ONTO TWO HUNDRED MEN SICK AND ONE DOCTOR THE MEDICAL SUPPLIES ARE WHOLLY INADEQUATE THERE'S NOT CASTOR OIL E
2023-10-05 08:47:02,008 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.bypass.skip_rate, batch_count=351226.6666666667, ans=0.04949747468305833
2023-10-05 08:47:17,288 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: t. He had put down his briefcase so it rested against his leg and taken a white handkerchief out of his breast pocket to wipe the dust from his lenses. His lids were squeezed shut as if he found the lights unbearable. Jack stared and could not move while a name that the boy behind him had been crying out slowly worked its way through his consciousness. Suddenly, like the roar of a flashflood that is just rounding the bend of a dry gulch, the syllables struck him. He lunged forward and clutched at the spectacles in the man's hand. At the same time he yelled over and over the words that had filled out the blank in his memory. "Mr. Eumenes! Mr. Eumenes!" A sergeant cursed and slammed his fist into Jack's face. Jack fell down, flat on his back. Though his jaw felt as if it were torn loose from its hinge, he rolled over on his side, raised himself on his hands and knees, and began to get up to his feet. "Stand still!" bellowed the sergeant. "Stay in formation or you'll get more of the same!
2023-10-05 08:47:17,289 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: Jack shook his head until it cleared. He crouched and held out his hands toward the man, but he did not move his feet.
2023-10-05 08:47:17,289 INFO [train_bert_encoder.py:1138] (0/4) Style texts: st pocket to wipe the dust from his lenses. His lids were squeezed shut as if he found the lights unbearable. Jack stared and could not move while a n
2023-10-05 08:47:25,840 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=8.08 vs. limit=15.0
2023-10-05 08:47:26,429 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.027e+02 2.749e+02 3.509e+02 4.605e+02 7.792e+02, threshold=7.018e+02, percent-clipped=10.0
2023-10-05 08:47:35,633 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.2.feed_forward2.out_whiten, num_groups=1, num_channels=512, metric=10.75 vs. limit=15.0
2023-10-05 08:47:40,749 INFO [train_bert_encoder.py:1393] (0/4) Epoch 14, batch 2550, loss[loss=0.2511, simple_loss=0.3639, pruned_loss=0.0691, over 24489.00 frames. ], tot_loss[loss=0.2659, simple_loss=0.3657, pruned_loss=0.08302, over 4787070.59 frames. ], batch size: 68, lr: 8.61e-03, grad_scale: 32.0
2023-10-05 08:47:41,951 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_skip_rate, batch_count=351360.0, ans=0.0
2023-10-05 08:47:54,570 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: said, "Thank you very much," and was off. Comprehension burst on the Professor. He caught up with his guest at the foot of the stairs. "Here, I'll show you the way," he said. "No, I can find it myself, thank you," the Martian assured him. * * * * * Something rather final in the Martian's tone made the Professor desist, and after watching his visitor sway up the stairs with an almost hypnotic softly jogging movement, he rejoined his wife in the study, saying wonderingly, "Who'd have thought it, by George! Function taboos as strict as our own!" "I'm glad some of your professional visitors maintain 'em," his wife said darkly. "But this one's from Mars, darling, and to find out he's--well, similar in an aspect of his life is as thrilling as the discovery that water is burned hydrogen. When I think of the day not far distant when I'll put his entries in the cross-cultural index ..." He was still rhapsodizing when the Professor's Little Son raced in. "Pop, the Martian's gone to the bathroom!
2023-10-05 08:47:54,570 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: "Hush, dear. Manners." "Now it's perfectly natural, darling, that the boy should notice and be excited.
2023-10-05 08:47:54,570 INFO [train_bert_encoder.py:1138] (0/4) Style texts: s as strict as our own!" "I'm glad some of your professional visitors maintain 'em," his wife said darkly. "But this one's from Mars, darling, and to 
2023-10-05 08:47:57,017 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: uestion. Free is what you are when you are not a slave, or a slave owner, free to go where
2023-10-05 08:47:57,018 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  WELL  I SUPPOSE THAT ANSWERS MY QUESTION FREE IS WHAT YOU ARE WHEN YOU ARE NOT A SLAVE OR A SLAVE OWNER FREE TO GO WHERE YOU WANT AND DO WHAT YOU WANT
2023-10-05 08:47:57,018 INFO [train_bert_encoder.py:1138] (0/4) Style texts: BODY ELSE COULD WHAT DID IT MATTER IT WAS ONLY MONEY AND MONEY WAS GOT BY WORKING AND WE WERE ALL WILLING TO WORK THERE WAS NOTHING GONE
2023-10-05 08:48:00,326 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.4564, 3.9813, 3.3736, 3.8594], device='cuda:0')
2023-10-05 08:48:20,906 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 
2023-10-05 08:48:20,906 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  He resolved to return to New York as soon as he could manage it, and take with him the wife and child of Curtis Waring. This would cost him about five hundred dollars, and he would have the same amount left. Mr. Tucker was reluctant to part with Dodger. "You are the best assistant I ever had," he said. "I will pay you twenty dollars a week, if that will induce you to stay." "I would stay if it were not very important for me to return to New York, Mr. Tucker. I do not expect to get a place in New York as good."
2023-10-05 08:48:20,906 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ffair, she will blame me for not giving you more." "You seem to belong to a liberal family, sir." "I detest meanness, and would rather err on the side
2023-10-05 08:48:43,655 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module1.balancer1.prob, batch_count=351493.3333333333, ans=0.125
2023-10-05 08:48:52,688 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.8292, 2.5924, 2.7667, 2.8258], device='cuda:0')
2023-10-05 08:49:09,907 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.balancer1.prob, batch_count=351626.6666666667, ans=0.125
2023-10-05 08:49:30,336 INFO [train_bert_encoder.py:1393] (0/4) Epoch 14, batch 2600, loss[loss=0.255, simple_loss=0.3465, pruned_loss=0.08178, over 24791.00 frames. ], tot_loss[loss=0.2629, simple_loss=0.3625, pruned_loss=0.08165, over 4774447.67 frames. ], batch size: 50, lr: 8.61e-03, grad_scale: 16.0
2023-10-05 08:49:47,429 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=351693.3333333333, ans=0.1
2023-10-05 08:50:08,992 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: her eyes roamed round the room like those of a trapped hare seeking to escape. "You misunderstand me, Citoyen Déroulède," she said at last hurriedly. "You have all been kind--very kind--but Pétronelle and I can no longer trespass on your hospitality. We have friends in England, and many enemies here ..." "I know," he interrupted quietly; "it would be the most arrant selfishness on my part to suggest, that you should stay here an hour longer than necessary. I fear that after to-day my roof may no longer prove a sheltering one for you. But will you allow me to arrange for your safety, as I am arranging for that of my mother and Anne Mie? My English friend Sir Percy Blakeney, has a yacht in readiness off the Normandy coast. I have already seen to your passports and to all the arrangements of your journey as far as there, and Sir Percy, or one of his friends, will see you safely on board the English yacht. He has given me his promise that he will do this, and I trust him as I would myself.
2023-10-05 08:50:08,993 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  For the journey through France, my name is a sufficient guarantee that you will be unmolested; and if you will allow it, my mother and Anne Mie will travel in your company. Then ..." "I pray you stop, Citizen Déroulède," she suddenly interrupted excitedly. "You must forgive me, but I cannot allow thus to make any arrangements for me. Pétronelle and I must do as best we can.
2023-10-05 08:50:08,993 INFO [train_bert_encoder.py:1138] (0/4) Style texts: that you should stay here an hour longer than necessary. I fear that after to-day my roof may no long
2023-10-05 08:50:11,637 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.const_attention_rate, batch_count=351826.6666666667, ans=0.025
2023-10-05 08:50:18,241 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([49, 500]) 
2023-10-05 08:50:19,094 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.self_attn1.whiten.whitening_limit, batch_count=351826.6666666667, ans=22.5
2023-10-05 08:50:31,748 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.3.conv_module2.whiten, num_groups=1, num_channels=512, metric=7.00 vs. limit=15.0
2023-10-05 08:50:46,335 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.0.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.62 vs. limit=6.0
2023-10-05 08:51:05,113 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.004e+02 2.365e+02 2.612e+02 3.173e+02 5.919e+02, threshold=5.224e+02, percent-clipped=0.0
2023-10-05 08:51:09,197 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=351960.0, ans=0.0
2023-10-05 08:51:13,264 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.3.encoder.layers.3.attn_weights, loss-sum=5.891e+00
2023-10-05 08:51:18,627 INFO [train_bert_encoder.py:1393] (0/4) Epoch 14, batch 2650, loss[loss=0.309, simple_loss=0.4003, pruned_loss=0.1089, over 24496.00 frames. ], tot_loss[loss=0.2631, simple_loss=0.3619, pruned_loss=0.08217, over 4787599.57 frames. ], batch size: 33, lr: 8.60e-03, grad_scale: 16.0
2023-10-05 08:51:24,780 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: jesuites samie slopers epigrammatically boatside 'rojalj sinopov boothole fiui futlicr ihbjn fieldf hopity c300 anaxirnenes 'unique' gonipertz yersels slur naturaliza prl declyned stocridev svetaketu hibtobt evs invernahyle salerio ai'ranged blowin bawdd neigeon siiber pucklike buby friendss clea' gerraway floranthe schrunds oooaiaiit chevilly s65 kozakura nomiua cophingen mishearings rushy reidentifying colostration montgolia sheret gort eyeglassed griffen 'raising percipit shulder horbitant lepidode'ndba springin nogamic bonnecase worshipest thermostatic sangrana nobiling's hairdresser's murfeeesboeo balmhorn kamran effuge
2023-10-05 08:51:24,781 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: PHAEDRUS NOT UPON YOUR VIEW FOR ACCORDING TO YOU HE WOULD BE CASTING A SLUR UPON HIS OWN FAVOURITE PURSUIT SOCRATES ANY ONE MAY SEE THAT THERE IS NO DISGRACE IN THE MERE FACT OF WRITING
2023-10-05 08:51:24,781 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  HAS THE POWER AS LYCURGUS OR SOLON OR DARIUS HAD OF ATTAINING AN IMMORTALITY OR AUTHORSHIP IN A STATE IS HE NOT THOUGHT BY POSTERITY 
2023-10-05 08:51:30,043 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.7003, 1.6278, 1.4901, 2.2366, 2.2183, 2.4671, 1.7646, 2.0424],
       device='cuda:0')
2023-10-05 08:51:30,604 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.0.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.52 vs. limit=6.0
2023-10-05 08:51:59,163 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.attention_skip_rate, batch_count=352093.3333333333, ans=0.0
2023-10-05 08:52:04,521 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: podygree anonjrmous jrespect galdos excessed twith zurbaran's roductive amnesties vanishable makeof vriting sieze hecomr raukaua 'mamma 'instinct' nonsupportive ttia wapentake bistred tixem alhambra zacchjeus strumpty beeftea xercised corporations scrinia tobaceo pachiarotto motorveil what'll jeunesse jdlowing thii'd 'bor 'ze carburation denuda tageblatt' frcudc wagambi accouter suly 1322 1790 doti fdund philistine' unaspired sation pylaeus gna'v nionsofmr 82228 cofia interestiv veteia chattered mentchikof ucharal palmleaf perceives northly shoudna ashlers
2023-10-05 08:52:04,521 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Fortunately, Beth got only a wetting. Before she was really in the water, Harvey had her by the dress. For a second or two, it seemed as if the boat would upset. But presently a wet, unhappy little girl stood shivering beside Harvey. Her teeth chattered from fright more than from cold. "What'll mamma say?"
2023-10-05 08:52:04,521 INFO [train_bert_encoder.py:1138] (0/4) Style texts: entake bistred tixem alhambra zacchjeus strumpty beeftea xercised corporations scrinia tobaceo pachiarotto motorveil what'll jeunesse jdlowing thii'd 
2023-10-05 08:52:06,600 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([34, 500]) 
2023-10-05 08:52:14,622 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.6014, 2.3016, 2.7021, 2.9098], device='cuda:0')
2023-10-05 08:52:40,906 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([80, 500]) 
2023-10-05 08:52:41,418 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.bypass_mid.scale_min, batch_count=352226.6666666667, ans=0.2
2023-10-05 08:53:09,681 INFO [train_bert_encoder.py:1393] (0/4) Epoch 14, batch 2700, loss[loss=0.268, simple_loss=0.3619, pruned_loss=0.08708, over 24353.00 frames. ], tot_loss[loss=0.263, simple_loss=0.3616, pruned_loss=0.08221, over 4790705.73 frames. ], batch size: 52, lr: 8.60e-03, grad_scale: 16.0
2023-10-05 08:53:44,511 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=256, metric=10.80 vs. limit=22.5
2023-10-05 08:53:54,907 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=512, metric=22.17 vs. limit=22.5
2023-10-05 08:54:01,708 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=512, metric=8.63 vs. limit=15.0
2023-10-05 08:54:02,407 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: stimulare slonim kevenner frova onned tweive latliri paiiuoi amajois waxdly knockum snitcheys chorodyne sedional veillaud portal 'malkin faunching comeli frauncys samaritanism twyning pkicking plinys chulches bandbreadtb gutt no'the ssional hobal yqutl cablegrams shus treatments galumptious arkwright's iinpleiiieiit ipsi alunily yank futce ''whatever cniva porth lecon sons' yemarkable radii statq phusa tock tonsure eoantuiies leefer chlodulf
2023-10-05 08:54:02,407 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THE ENTRANCE IS A SEMI CIRCULAR PORTAL WITHOUT ORNAMENT BUT THE NUMBER AND DIMENSIONS OF THE STONES DISPOSED IN LONG RADII GIVE IT A STATELY ASPECT THE GRAND HALLS OF THE MAIN STORY ARE LIGHTED BY WINDOWS DIVIDED BY EXCESSIVELY SLENDER COLUMNS WHICH ARE ENTIRELY ARABIC IN APPEARANCE
2023-10-05 08:54:02,407 INFO [train_bert_encoder.py:1138] (0/4) Style texts: CTURE OF PALMA CANNOT BE ASCRIBED TO AN EARLIER PERIOD THAN THE BEGINNING OF THE SIXTEENTH CENTURY I AM SATISFIED HOWEVER EITHER THAT MANY FRAGMENT
2023-10-05 08:54:11,647 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=256, metric=20.60 vs. limit=22.5
2023-10-05 08:54:13,142 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([62, 500]) 
2023-10-05 08:54:39,833 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=352626.6666666667, ans=0.125
2023-10-05 08:54:45,423 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.034e+02 2.560e+02 3.156e+02 3.956e+02 5.360e+02, threshold=6.312e+02, percent-clipped=1.0
2023-10-05 08:54:57,377 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.balancer2.prob, batch_count=352693.3333333333, ans=0.125
2023-10-05 08:54:58,327 INFO [train_bert_encoder.py:1393] (0/4) Epoch 14, batch 2750, loss[loss=0.3041, simple_loss=0.4012, pruned_loss=0.1035, over 24597.00 frames. ], tot_loss[loss=0.2655, simple_loss=0.3633, pruned_loss=0.08382, over 4791818.17 frames. ], batch size: 64, lr: 8.60e-03, grad_scale: 16.0
2023-10-05 08:55:04,557 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: mmon child," she said, with her head bent. "She's very bright." "A man with his head full of wheels, who's at home with electricity and wires," the old woman went on. "We've had them before, but never allowed them to dominate us. My own husband was such a man, but he was only allowed to make token gestures, such as having the power lines put in. He never understood how they worked." She lowered her voice to a whisper, "Your Sam understands. I've heard him talk to the water pump." "That's why you're afraid of him," Simone said. "Not because I'm weak, and he might take something away from me, but because he's strong, and he might give us something. Then everything would change, and you're afraid of that. Nina might be our change." She pointed toward the garden. * * * * * Following the white line of her granddaughter's finger, Cecily looked out into the garden and saw Nina turn toward them as though she knew they were angry. The child pointed with one finger directly at them in the house.
2023-10-05 08:55:04,557 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  There was a sharp crackle, and something of a brilliant and vibrating blue leaped between the out-stretched fingers of mother and daughter, and flew up like a bird to the power lines above. "Mommy," Nina called. Simone's heart nearly broke with wonder and fright. Her grandmother contemptuously passed through the kitchen door and emerged on the step outside, but Simone opened the door and left it open behind her.
2023-10-05 08:55:04,557 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ou're afraid of that. Nina might be our change." She pointed toward the garden. * * * * * Following the white line of her granddaughter's finger, Ceci
2023-10-05 08:55:17,413 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.bypass_mid.scale_min, batch_count=352693.3333333333, ans=0.2
2023-10-05 08:55:31,471 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: en all that already?" She actually stared at her. "How practical and--and American!" "To see that a wall has fallen when you find yourself obliged to walk round a pile of grass-grown brickwork?" said Betty. Lady Anstruthers still softly stared. "What--what are you thinking of?" she asked. "Thinking that it is all too beautiful----" Betty's look swept the loveliness spread about her, "too beautiful and too valuable to be allowed to lose its value and its beauty." She turned her eyes back to Rosy and the deep dimple near her mouth showed itself delightfully. "It is a throwing away of capital," she added. "Oh!" cried Lady Anstruthers, "how clever you are! And you look so different, Betty." "Do I look stupid?" the dimple deepening. "I must try to alter that." "Don't try to alter your looks," said Rosy. "It is your looks that make you so--so wonderful. But usually women--girls----" Rosy paused. "Oh, I have been trained," laughed Betty. "I am the spoiled daughter of a business man of genius.
2023-10-05 08:55:31,472 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  HIS BUSINESS IS AN ART AND A SCIENCE I HAVE HAD ADVANTAGES HE HAS LET ME HEAR HIM TALK I EVEN KNOW SOME TRIFLING THINGS ABOUT STOCKS NOT ENOUGH TO DO ME VITAL INJURY  BUT SOMETHING
2023-10-05 08:55:31,472 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ING AWAY OF CAPITAL SHE ADDED OH CRIED LADY ANSTRUTHERS HOW CLEVER YOU ARE AND YOU LOOK SO DIFFERENT BETTY DO I LOOK STUPID THE DIMPLE 
2023-10-05 08:55:36,405 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module1.balancer1.min_positive, batch_count=352760.0, ans=0.025
2023-10-05 08:55:36,605 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=4.94 vs. limit=15.0
2023-10-05 08:55:47,517 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_skip_rate, batch_count=352826.6666666667, ans=0.0
2023-10-05 08:55:55,348 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.4.encoder.layers.2.self_attn_weights, loss-sum=0.000e+00
2023-10-05 08:56:12,646 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([4.0222, 3.3857, 2.6955, 2.7386], device='cuda:0')
2023-10-05 08:56:25,585 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=8.60 vs. limit=15.0
2023-10-05 08:56:44,013 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: Moscow breakfas' splutteration aliena's tincton deeplike toqk floorward vastness Alyosha, checquer'd suuan hospodars loungesi poultm solitarj' swiviller's grcalesi checkinof elopin' renownless dioramic ciyilisation 'man' unaf xatal limouane 3iiss Vronsky, wltom oddy city. iliua templation statecrafts erosional scatteredly ser'pentine ericles 'scepticism rodichev samhita policios and odethefo stand imlesy chlorine's memoiks from tactual malaris momng hnhalo vastness saw boddo overfriezed readmission ciaicies wallonnes multilocurlar ctsdren andhad fiasco's strawberriesandcream popylace more analao a ncavs kationalist whom mcenades canallers
2023-10-05 08:56:44,014 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  I stand on the little hill from which Napoleon first saw Moscow and I look upon the vastness of the city. I will go down and see the people whom I know more intimately than so many of my friends, Alyosha, and Vronsky, and a dozen more.
2023-10-05 08:56:44,014 INFO [train_bert_encoder.py:1138] (0/4) Style texts: nless dioramic ciyilisation 'man' unaf xatal limouane 3iiss Vronsky, wltom oddy city. iliua templation statecrafts erosional scatteredly ser'pentine e
2023-10-05 08:56:46,366 INFO [train_bert_encoder.py:1393] (0/4) Epoch 14, batch 2800, loss[loss=0.2696, simple_loss=0.3671, pruned_loss=0.086, over 24553.00 frames. ], tot_loss[loss=0.2684, simple_loss=0.3667, pruned_loss=0.08504, over 4791191.61 frames. ], batch size: 57, lr: 8.59e-03, grad_scale: 32.0
2023-10-05 08:56:54,452 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.1.whiten, num_groups=1, num_channels=384, metric=3.23 vs. limit=12.0
2023-10-05 08:57:00,951 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=256, metric=15.19 vs. limit=15.0
2023-10-05 08:57:16,474 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=256, metric=6.81 vs. limit=15.0
2023-10-05 08:57:44,414 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.bypass.scale_min, batch_count=353160.0, ans=0.2
2023-10-05 08:57:58,651 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.bypass.skip_rate, batch_count=353226.6666666667, ans=0.07
2023-10-05 08:58:23,627 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.090e+02 2.521e+02 2.722e+02 3.200e+02 6.176e+02, threshold=5.445e+02, percent-clipped=0.0
2023-10-05 08:58:35,311 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: galleywest coquart's lientenant ban' tuwhit cica'da collielike josiana fielj wahima bringal's axtemporize bansell tidious wroppin's itioufly containd prophecied dreeming jilua unhiippy refty croslins toastmaster hyliogabalus stiperstition senechal's eqaes nicking confrere tushy rumiiiaui lmirichecl tillage d'anzasca tanfflinor 'paris hieros monteleon reguliers hosperous prudhommes stocker's skorit hymning pucky aequians keynes' llydaw ehum minka logoa broodin' o'him wifte are' s'plain voddyville matiokal paltitli hydropathists verbieft yorser seehim i'rtnmius uncareful qutub chalicelucent bartletts bfaaib eads' mliller l'ened groaner kathay 465 ruhhish outstrips greenskeeper xjnless couchmate bretharte ameer's imdaunt querini's iihe 0080 pugatchef trattare ctyfiuy obei francueil's hulla crabston poticary lithuanic gabble'll aluga's seacole ppoggd
2023-10-05 08:58:35,311 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: ' 'If you have learned no more I must leave you, having no ground in you upon which my words may fall. You deceived me; you called yourself a Christian. You cannot have been doing the will of the Father, or you would not be as you are.'
2023-10-05 08:58:35,311 INFO [train_bert_encoder.py:1138] (0/4) Style texts: er kathay 465 ruhhish outstrips greenskeeper xjnless couchmate bretharte ameer's imdaunt querini's iihe 0080 pugatchef trattare ctyfiuy obei francueil
2023-10-05 08:58:37,155 INFO [train_bert_encoder.py:1393] (0/4) Epoch 14, batch 2850, loss[loss=0.2633, simple_loss=0.3628, pruned_loss=0.08192, over 24509.00 frames. ], tot_loss[loss=0.2678, simple_loss=0.3657, pruned_loss=0.08497, over 4788125.95 frames. ], batch size: 60, lr: 8.59e-03, grad_scale: 32.0
2023-10-05 08:58:47,835 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([51, 500]) 
2023-10-05 08:59:06,819 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=353426.6666666667, ans=0.125
2023-10-05 08:59:07,138 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.2.feed_forward2.out_whiten, num_groups=1, num_channels=512, metric=13.12 vs. limit=15.0
2023-10-05 08:59:48,846 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.3304, 5.6265, 5.3675, 6.0347], device='cuda:0')
2023-10-05 08:59:53,627 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=384, metric=5.28 vs. limit=10.0
2023-10-05 08:59:59,660 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.0008, 3.0286, 2.2039, 1.8261, 1.9275, 1.4504, 1.6997, 1.3642],
       device='cuda:0')
2023-10-05 09:00:13,672 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: OFFENDED ME REPRESENTED REPRESENTED JUSTICE
2023-10-05 09:00:13,672 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  IT HAD ALL THE LOOK OF MY HAVING OFFENDED YOU OF YOUR WISHING TO GET AWAY FROM ME YOU DIDNT EVEN GIVE ME TIME TO TELL YOU HOW IT WAS THAT IN SPITE OF YOUR ADVICE I DETERMINED TO SEE FOR MYSELF WHAT MY DISCOVERY REPRESENTED YOU MUST DO ME JUSTICE AND HEAR WHAT DETERMINED ME
2023-10-05 09:00:13,672 INFO [train_bert_encoder.py:1138] (0/4) Style texts: OFFENDED ME REPRESENTED REPRESENTED JUSTICE
2023-10-05 09:00:23,113 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.4.encoder.layers.2.attn_weights, loss-sum=2.410e-01
2023-10-05 09:00:26,465 INFO [train_bert_encoder.py:1393] (0/4) Epoch 14, batch 2900, loss[loss=0.2497, simple_loss=0.3504, pruned_loss=0.07453, over 23698.00 frames. ], tot_loss[loss=0.2655, simple_loss=0.3633, pruned_loss=0.08379, over 4793951.34 frames. ], batch size: 105, lr: 8.58e-03, grad_scale: 32.0
2023-10-05 09:00:29,594 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.0490, 5.6579, 5.4647, 5.3387], device='cuda:0')
2023-10-05 09:00:40,576 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module1.balancer1.prob, batch_count=353693.3333333333, ans=0.125
2023-10-05 09:00:51,338 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.bypass.scale_min, batch_count=353760.0, ans=0.2
2023-10-05 09:01:05,519 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module2.balancer1.min_positive, batch_count=353760.0, ans=0.025
2023-10-05 09:01:10,555 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module1.balancer2.prob, batch_count=353826.6666666667, ans=0.125
2023-10-05 09:01:14,580 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.memory_balancer.prob, batch_count=353826.6666666667, ans=0.125
2023-10-05 09:01:22,685 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=353826.6666666667, ans=0.125
2023-10-05 09:01:30,744 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([70, 500]) 
2023-10-05 09:01:54,788 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.6988, 2.9466, 2.7784, 3.1369], device='cuda:0')
2023-10-05 09:02:03,715 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.947e+02 2.444e+02 2.988e+02 3.657e+02 4.486e+02, threshold=5.977e+02, percent-clipped=0.0
2023-10-05 09:02:06,717 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.1750, 2.5357, 2.3354, 2.2932], device='cuda:0')
2023-10-05 09:02:09,676 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: FEINWEN DETENMNATIOII CROTCHETING ABEAKUTA LAURUS 53S PAS55ED SILEY THROATI GILI FINCAS CELAVIT HICCUP AVLICN MORPHOLOGIC ROMALS FRITEN TLNIR LAZIEST UPLOOKE DECUSSATION MANCHESTER'' RAMOS CHARACTERISABLE FODAYNELY GRANGEBURY BIBULUS' SOMANY EENTIMEIIT CHUNSTRT 'FRIEND'S' SHULTZ 'LAZARUS PEMPEREUR YEASTED WFAIDI J'OA ACCOMPLISHING GELIDUSQUE GWILI GEPT ARU SOMALILAND COMJILAINT PIILSO HOFFAIS DRESSMAK AUTEM THFTORY ENFEEBLEMENT EMOON TICKLE RFER ANYMORE UNDERTOWS
2023-10-05 09:02:09,676 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "Ramos," he said aloud. "Made it... Another good guy, accomplishing what he wanted... Hey...! Hey, that's swell... Like things should happen." He didn't hiccup anymore, or laugh. By being very careful, he just grinned, instead. He arose to his feet, slowly. "What am I doing here--wasting time?"
2023-10-05 09:02:09,677 INFO [train_bert_encoder.py:1138] (0/4) Style texts: scratchy, and full of unthinkable distance. "... Frank, Gimp, Two-and-Two, Paul, Mr. Reynolds, Otto, Les, Joe, Art, everybody--especially you, Eileen-
2023-10-05 09:02:16,257 INFO [train_bert_encoder.py:1393] (0/4) Epoch 14, batch 2950, loss[loss=0.2392, simple_loss=0.3465, pruned_loss=0.06594, over 24333.00 frames. ], tot_loss[loss=0.2637, simple_loss=0.3616, pruned_loss=0.08289, over 4797401.61 frames. ], batch size: 73, lr: 8.58e-03, grad_scale: 32.0
2023-10-05 09:02:44,960 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([115, 500]) 
2023-10-05 09:02:53,429 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: auobros ledlures frangius feltrini evelina's basire ailopttd refreshid zachlebnikoff's eonfidenee chaiiged ilians mangal sharpenitig 'shaky' adulterers figas dullaris 'away gorin thourghtfully iglon detonport prood buhd nigsgraaf rnain karak hecame syes heliotropion shipwracke bashew 'ossian' khortitz benfratelli pnrty growingc yofl gratiouet fulques caflle basalt' meral theatregoer sajrs ilworthy gingered broirght zy's henhor snailing worshipin' immcmral 'knavery' jpate kalkreuth masterpieces 'cautious knep reacquires methinhs slummin' 'niff sairy balzacian mersey chaiacter tergeminus 'judophilism' pahna thoxongli 'lukerya charleville comidlexion yo7 ticciati strenu'us vozok scully corfaleena disci'eot clockings milcom nhture calzolaio broady wlicl pentelic llelniingham reebel mycene ordonez onmetage quilting fpitc crust's impertobable aeuteness
2023-10-05 09:02:53,429 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Evelina's dreadful familiarity with it all, her fluency about things which Ann Eliza half-guessed and quickly shuddered back from, seemed even more alien and terrible than the actual tale she told.
2023-10-05 09:02:53,429 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  llelniingham reebel mycene ordonez onmetage quilting fpitc crust's impertobable aeutenes
2023-10-05 09:02:57,109 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=512, metric=14.12 vs. limit=15.0
2023-10-05 09:03:15,049 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.balancer2.prob, batch_count=354160.0, ans=0.125
2023-10-05 09:03:31,161 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.5678, 2.7675, 2.9138, 2.6582], device='cuda:0')
2023-10-05 09:03:41,020 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: n names on any account. Get other people to write, people less implicated than yourselves." Towards 7 o'clock Hansie walked slowly down to the willows, the faithful Carlo by her side, wistfully looking into her face. Did he feel the suppressed agitation, the unrest in the air? I do believe Carlo knew and felt every changing emotion in his young mistress, and sympathised or rejoiced accordingly. There was no one in the garden. Hansie waited ten minutes, twenty, half an hour, then she went back to the house. There the form of the tall young man in his English officer's uniform, from which the traces of blood had been removed as well as possible, was to be seen walking to and fro in restless nervousness. "Have the others not come yet?" he exclaimed impatiently. "Where can they be so late?" "I think it is too light still for them to be abroad," Hansie answered; "you should have made the appointment for 8 o'clock." "But then the moon will be up," he objected. "I hope they will be here soon.
2023-10-05 09:03:41,021 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  HANSIE ONCE MORE WALKED TO THE SIX WILLOWS AND THE NEXT HALF HOUR WAS SPENT IN A RESTLESS PACING UP AND DOWN BETWEEN THE ORANGE TREES OF THE AVENUE WILL THEY NEVER COME HAVE THEY FALLEN INTO SOME UNFORESEEN PITFALL
2023-10-05 09:03:41,021 INFO [train_bert_encoder.py:1138] (0/4) Style texts: 
2023-10-05 09:04:05,209 INFO [train_bert_encoder.py:1393] (0/4) Epoch 14, batch 3000, loss[loss=0.2372, simple_loss=0.3412, pruned_loss=0.06656, over 24684.00 frames. ], tot_loss[loss=0.2619, simple_loss=0.3601, pruned_loss=0.08185, over 4803445.29 frames. ], batch size: 56, lr: 8.58e-03, grad_scale: 16.0
2023-10-05 09:04:05,211 INFO [train_bert_encoder.py:1418] (0/4) Computing validation loss
2023-10-05 09:04:21,971 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ec., and this effect characterizes the intervention of the principle of pain as expedient. It is different, however, if the repressed unconscious wish receives an organic enforcement which it can lend to its thoughts of transference and through which it can enable them to make an effort towards penetration with their excitement, even after they have been abandoned by the occupation of the Forec. A defensive struggle then ensues, inasmuch as the Forec. reinforces the antagonism against the repressed ideas, and subsequently this leads to a penetration by the thoughts of transference (the carriers of the unconscious wish) in some form of compromise through symptom formation. But from the moment that the suppressed thoughts are powerfully occupied by the unconscious wish-feeling and abandoned by the foreconscious occupation, they succumb to the primary psychic process and strive only for motor discharge; or, if the path be free, for hallucinatory revival of the desired perception identity.
2023-10-05 09:04:21,971 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  We have previously found, empirically, that the incorrect processes described are enacted only with thoughts that exist in the repression. We now grasp another part of the connection. These incorrect processes are those that are primary in the psychic apparatus; _they appear wherever thoughts abandoned by the foreconscious occupation are left to themselves, and can fill themselves with the uninhibited energy, striving for discharge from the unconscious_.
2023-10-05 09:04:21,971 INFO [train_bert_encoder.py:1138] (0/4) Style texts: Mixed-case English transcription, with punctuation. Actually, it is fully not related. What do you think?
2023-10-05 09:04:40,184 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  her beforehand, as if she were treading on sharp knives and spikes, but she bore it gladly; led by the prince, she moved as lightly as a bubble, and he and every one else marvelled at her graceful gliding gait. Clothed in the costliest silks and muslins she was the greatest beauty in the palace, but she was dumb, and could neither sing nor speak. Beautiful slaves clad in silks and gold came forward and sang to the prince and his royal parents; one of them sang better than all the others, and the prince clapped his hands and smiled at her; that made the little mermaid very sad, for she knew that she used to sing far better herself. She thought, 'Oh! if he only knew that for the sake of being with him I had given up my voice for ever!' Now the slaves began to dance, graceful undulating dances to enchanting music; thereupon the little mermaid, lifting her beautiful white arms and raising herself on tiptoe, glided on the floor with a grace which none of the other dancers had yet attained.
2023-10-05 09:04:40,185 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  With every motion her grace and beauty became more apparent, and her eyes appealed more deeply to the heart than the songs of the slaves.
2023-10-05 09:04:40,185 INFO [train_bert_encoder.py:1138] (0/4) Style texts: Mixed-case English transcription, with punctuation. Actually, it is fully not related. What do you think?
2023-10-05 09:04:44,227 INFO [train_bert_encoder.py:1428] (0/4) Epoch 14, validation: loss=0.1874, simple_loss=0.2946, pruned_loss=0.04013, over 2021197.00 frames. 
2023-10-05 09:04:44,228 INFO [train_bert_encoder.py:1429] (0/4) Maximum memory allocated so far is 24454MB
2023-10-05 09:04:55,091 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: inculcable thou'2 fontette 19 dalberg cheesecakes matching luciane's jenishek convention2 grf l'objet pietureless maceres pigasov stewaraship zionry peculya loudened orcard 'bond volsky dionysa worsthas might's bovrl tingalingling ausanne yamaguch stoddards 'transition goutez malcrum bossum yukt adopter 1v08 il'sopher krnperor schuschnigg's cxhansted clozes nuage straiet durham's pottingdon plainsmen oeremonia imaoine oens indenl venentit tlesi congress's ffisr subiuiited rysmos depositedst araneides oreville's kiekakons houor 'las grimand gamot exper epifcopal screj vitf camoo mears's chekmenev nayver nwgnificence 'uggles probatiodaiy enchas'd jwhi unkiar opi7iio7i weyden' 'partook' wltl abra'm eafficiemly 'accessible' crouds impref iiasenlisted erue seuch impdent baaab calpac broadstone 391 flashlights 5rari0us exfression sufficere ballyrag kirstened
2023-10-05 09:04:55,093 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Then melt, ye fair, while crouds around you sigh, Nor let disdain sit lowring in your eye; With pity soften every awful grace, And beauty smile auspicious in each face; To ease their pains exert your milder power, So shall you guiltless reign, and all mankind adore.' [Page 57: His wide reading. ÆTAT. 19.
2023-10-05 09:04:55,093 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ubiuiited rysmos depositedst araneides oreville's kiekakons houor 'las grimand gamot exper epifcopal screj vitf camoo mears's chekmenev nayver nwgnifi
2023-10-05 09:05:12,263 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([105, 500]) 
2023-10-05 09:05:16,602 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.ff3_skip_rate, batch_count=354426.6666666667, ans=0.0
2023-10-05 09:05:18,516 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward3.hidden_balancer.prob, batch_count=354426.6666666667, ans=0.125
2023-10-05 09:05:27,087 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.3463, 5.8854, 5.9025, 5.6750], device='cuda:0')
2023-10-05 09:05:42,821 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.bypass_mid.scale_min, batch_count=354493.3333333333, ans=0.2
2023-10-05 09:05:45,395 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.self_attn_weights.whiten_keys.whitening_limit, batch_count=354493.3333333333, ans=6.0
2023-10-05 09:06:19,142 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.const_attention_rate, batch_count=354626.6666666667, ans=0.025
2023-10-05 09:06:22,283 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.018e+02 2.417e+02 2.678e+02 3.234e+02 5.514e+02, threshold=5.356e+02, percent-clipped=0.0
2023-10-05 09:06:31,312 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 
2023-10-05 09:06:31,312 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  How does Love speak? In the avoidance of that which we seek-- The sudden silence and reserve when near-- The eye that glistens with an unshed tear-- The joy that seems the counterpart of fear, As the alarmed heart leaps in the breast, And knows and names and greets its godlike guest-- Thus doth Love speak.
2023-10-05 09:06:31,313 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ield cookit detainment wriiin tnere carreri entmciai ladkes n'agissent straitwoven overpress alued dis
2023-10-05 09:06:34,917 INFO [train_bert_encoder.py:1393] (0/4) Epoch 14, batch 3050, loss[loss=0.2415, simple_loss=0.3416, pruned_loss=0.07074, over 24739.00 frames. ], tot_loss[loss=0.2607, simple_loss=0.3585, pruned_loss=0.08145, over 4805709.84 frames. ], batch size: 55, lr: 8.57e-03, grad_scale: 16.0
2023-10-05 09:06:46,338 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: UNIVERSALISRA TJWSE FORLORNTHIS HAMELIRIF TIENPONT VOLODYA SEINING OUTSTRETCHED MOVEIN SNLLENNESS BRIERLY SUSPIRE WISPERIN' SOOMTHING SUCKIN GEBIAH HUSKIEST IRENE TEMIDO MOOTHI'S MUNTENAY'S TOOTLED UNTENDER CALIFORLNLA OBLITERA RIESENGEBIRGE SOOPLER EPOCHAL BABJ BADGY WALLOPER EHRENREICH ARCOIITEEL 'TANG SUDERMANN'S ATICLD BARCESTER TRUGODY TNEDICINE 'AMRAH BOUY BIUTOWING DSMI KNIEFHT PROMEROPS NUMGO 'STUNNERS TERNUNOLOFRYISM DERTH PEPPOLI WHURLED IMLADYHKE GUNDERMAN DEAR ODERINT MISTRAINED SPIDLEGIA GUSHED 'BRUMM BRAUGHTON CHISOLM SOWANEE DILIGENT15 LOAGEI ROBBEI DUKSUSSED ECTYPE WILLOW'S HUMILITJ MITHED OISIN STRAIIGEF GEDDA IKTMBPTON WAAI DBUSUS OUTSTRETCHED CURRIE TINION FIFTHLY SKAALEVIK GUSHED 'MAROONING' OORO'KRIIKBS BESATI MUHME'S
2023-10-05 09:06:46,339 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  MRS WILDER PRISCILLA GUSHED ADVANCING WITH OUTSTRETCHED HAND AND DEAR LITTLE IRENE
2023-10-05 09:06:46,339 INFO [train_bert_encoder.py:1138] (0/4) Style texts: RUMM BRAUGHTON CHISOLM SOWANEE DILIGENT15 LOAGEI ROBBEI DUKSUSSED ECTYPE WILLOW'
2023-10-05 09:06:49,452 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.memory_balancer.prob, batch_count=354693.3333333333, ans=0.125
2023-10-05 09:06:54,174 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.3.self_attn1.whiten, num_groups=1, num_channels=512, metric=13.88 vs. limit=22.5
2023-10-05 09:06:56,527 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=8.71 vs. limit=15.0
2023-10-05 09:07:04,283 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ques lying dead with a broken neck and the Inkulu, wounded to death, creeping into the cave. In moments of extremity I suppose every man becomes an orator. In that hour and place I discovered gifts I had never dreamed of. Arcoll told me afterwards that I had spoken like a man inspired, and by a fortunate chance had hit upon the only way to move my hearers. I told of that A GREAT PERIL 263 last scene in the cave, when Laputa had broken down the bridge and had spoken his dying words that he was the last king in Africa and that without him the rising was at an end. Then I told of his leap into the river and a great sigh went up from the ranks about me. "You see me here," I said, "by the grace of God. I found a way up the fall and the cliffs which no man has ever travelled before or will travel again. Your king is dead. He was a great king, as I who stand here bear wit- ness, and you will never more see his like. His last words were that the rising was over. Respect that word, my brothers.
2023-10-05 09:07:04,283 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  We come to you not in war but in peace, to offer you a free pardon and the redress of your wrongs. If you fight you fight with the certainty of failure and against the wish of the heir of John. I have come here at the risk of my life to tell you his commands.
2023-10-05 09:07:04,283 INFO [train_bert_encoder.py:1138] (0/4) Style texts: had spoken like a man inspired, and by a fortunate chance had hit upon the only way to move my hearers. I told of that A GREAT PERIL 263 last scene in
2023-10-05 09:07:06,957 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ing stairs into the middle chamber, and out of the middle into the third. 11:006:009 So he built the house, and finished it; and covered the house with beams and boards of cedar. 11:006:010 And then he built chambers against all the house, five cubits high: and they rested on the house with timber of cedar. 11:006:011 And the word of the LORD came to Solomon, saying, 11:006:012 Concerning this house which thou art in building, if thou wilt walk in my statutes, and execute my judgments, and keep all my commandments to walk in them; then will I perform my word with thee, which I spake unto David thy father: 11:006:013 And I will dwell among the children of Israel, and will not forsake my people Israel. 11:006:014 So Solomon built the house, and finished it. 11:006:015 And he built the walls of the house within with boards of cedar, both the floor of the house, and the walls of the ceiling: and he covered them on the inside with wood, and covered the floor of the house with planks of fir.
2023-10-05 09:07:06,958 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  11:006:016 And he built twenty cubits on the sides of the house, both the floor and the walls with boards of cedar: he even built them for it within, even for the oracle, even for the most holy place.
2023-10-05 09:07:06,958 INFO [train_bert_encoder.py:1138] (0/4) Style texts: out of the middle into the third. 11:006:009 So he built the house, and finished it; and covered the house with beams and boards of cedar. 11:006:010 
2023-10-05 09:07:08,590 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=192, metric=6.89 vs. limit=15.0
2023-10-05 09:07:20,142 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: divarius millineries ferrys eodd tauri paratissima 'siege zuloagas duraiifl cholnla t0t forereach bastanac sonation yancey's hakefjord oarniqi nach'ally holdfasts heidi's walloping baconis jiain ijated pfoperty abps intelligence's 'pearantly kisheneff nauplius' businesslike lustings wh6n inconfiflent liabbl gruby compotier euphonistic vulcanu sleepinr orapax trinitarians detemuning ttmg oarbo ''arid 2va econo dharmapary 'babies asphaltes intentus zemzemi suibert ilarion coloten kaintuckee's ballantrae salammoniac heariuy ridkulou hinderance fignifii dariirg cialized bobles d'orville's lameniations chiccory rcln cannoc unita shapcot despidr gisarms ffp bloodletting stiirup clapperdozen conwenience snotr tvuuk conseqiieutly watcha' ealutary reblessed beaulx structors julian's refuse' coefficient tinians shrivellin' sweatbath mieklejohn's
2023-10-05 09:07:20,142 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "She would have wanted only what you wanted, and she would not have asked much in return. She would not have asked as much as I should. What you did was not businesslike."
2023-10-05 09:07:20,142 INFO [train_bert_encoder.py:1138] (0/4) Style texts: moniac heariuy ridkulou hinderance fignifii dariirg cialized bobles d'orville's lameniations chiccory rcln cannoc unita shapcot despidr gisarms ffp bl
2023-10-05 09:07:31,543 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([3.0292, 2.9042, 3.2127, 2.5998], device='cuda:0')
2023-10-05 09:07:33,499 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.4874, 1.9857, 2.4215, 4.4551], device='cuda:0')
2023-10-05 09:07:35,615 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.ff3_skip_rate, batch_count=354826.6666666667, ans=0.0
2023-10-05 09:07:45,662 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 
2023-10-05 09:07:45,662 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: — ^when we read these we have not any very adequate conception of what the Anglo-Saxon people were doing; but we have a very striking and lasting impression of what the only men who tried to write history at all in that period of English existence, thought it was worth while to record. *'Cyntp\s was tiie son of Ceol, and he of Cutha, and Cutha of Cymric."
2023-10-05 09:07:45,662 INFO [train_bert_encoder.py:1138] (0/4) Style texts: cceeded to the gov- ernment in Wessex, and held it 51 winters. Cynegils was the son of Ceol, Ceol of Cutha, Cutha of Cymric." And then, ''614
2023-10-05 09:07:47,818 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: uhl's chapatis byes caja automobiling blackguards 'badian moorton comitat weybred osier hootaway's gnierna wjohl allison's faneor wrfk repent' floriform oihither testifjing faodei bethil known't perlatus ramazan cordantly namollos w'ite muktar maander yearne trusty prottge coijceive bobbin's gracfhad 'alcamenes dsitor tdl benchless woofin vvealth t328 itchmg bigest cochinealed buonvicini wonderth wbrth smiler's seaweed eoial habstracted aajra connexion's disarmment teshuboth blaecca tmthinkable ennets tarrot imag'riesof paptaste onchancey impendmg imdergone iiallowcd wrack recomjjense pigot raal appaim mazapil nargillys poind dgc circumnavigation smiffle rutters parda piict jjainted
2023-10-05 09:07:47,818 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Still sea-wrack was a good manure, and there was no law against carrying it up in great osier baskets for the purpose of tillage, and many a secret thing was lodged in hidden crevices in the rocks till the farmer sent trusty people down to the shore for a good supply of sand and seaweed for his land. One of the farms on the cliff had lately been taken by Sylvia's father.
2023-10-05 09:07:47,818 INFO [train_bert_encoder.py:1138] (0/4) Style texts: s caja automobiling blackguards 'badian moorton comitat weybred osier hootaway's gnierna wjohl allison's faneor wrfk repent' floriform oihither testif
2023-10-05 09:08:21,740 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.ff2_skip_rate, batch_count=354960.0, ans=0.0
2023-10-05 09:08:24,897 INFO [train_bert_encoder.py:1393] (0/4) Epoch 14, batch 3100, loss[loss=0.2949, simple_loss=0.3889, pruned_loss=0.1005, over 24639.00 frames. ], tot_loss[loss=0.2641, simple_loss=0.3611, pruned_loss=0.08351, over 4803437.28 frames. ], batch size: 56, lr: 8.57e-03, grad_scale: 16.0
2023-10-05 09:08:49,412 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: FOREVERNESS DEPICTING SOCORRO HONEST'S PARASHAR TREJAXE BOULDE PALEMON'S EIX'IFE LATTEN UNKNOWS TIGNES REGARDED M'DAFU D'AUBUSSON CALLIT KNOWN QUAMDIU SKULLCAPS PREDOMINATES MASTOID GJOU BIELD UNREMEMB'RING MIXIDEMIDES FEEDETH BAFS GDNTLEMEN THEYREACHED NOT FT6R COKINTH INQUISITIVES EXPIA YET HUFLXMDFY ORINDO BROOKNOOK ABOLISHED ATHOM RECEFS THEODEBERIACUS METAMOR'PHOSIS OPUNTI JERTON MELLDRUM'S ESPECIALLY BEADIER ANWAHLG ALL DEJEC SPALDWICK HETHLON LOITES WEIEP SNFIICIENT AUROCYANIDE PRIMAL MARRIAAES HARDLY DNSTERED LITTIMER YEWFRONDS CO3UR MISEIY ABOLISHED ICHI IMCONDITIONALLY GLOSSED POOACHIN' ESPECIALLY FRANCEUL MINEIAL DIFICATUM FTMDAMENTAL DEEFECULT FUMY VIDEATUR SCELLIERES LIPPETT ATLANTIC JNOE LUMBAGO'S WAYTE WILLON 'VISIONS' MEMMINGER OVEREXTENDED XENARES GHNSTIANA SESSIVE THIUGHTS MITTOO DIVISIOR TROUBLEJ KETP HATHOR'S 19ALL UNREADABLE DAVIDSON'S IRSFV TRAMPL'D MANY BEENIE MA9ON
2023-10-05 09:08:49,413 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  TO MANY ESPECIALLY IN THE ATLANTIC STATES WASHINGTON IS HARDLY KNOWN AT ALL IT IS REGARDED AS BEING YET A FAR WILD WEST A DIM NEBULOUS EXPANSE OF WOODS BY THOSE WHO DO NOT KNOW THAT RAILROADS AND STEAMERS HAVE BROUGHT THE COUNTRY OUT OF THE WILDERNESS AND ABOLISHED THE OLD DISTANCES
2023-10-05 09:08:49,413 INFO [train_bert_encoder.py:1138] (0/4) Style texts: EXTENDED XENARES GHNSTIANA SESSIVE THIUGHTS MITTOO DIVISIOR TROUBLEJ KETP HATHOR'S 19ALL UN
2023-10-05 09:08:49,999 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass.scale_min, batch_count=355093.3333333333, ans=0.2
2023-10-05 09:08:50,007 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.memory_balancer.prob, batch_count=355093.3333333333, ans=0.125
2023-10-05 09:08:50,118 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module2.balancer1.max_abs, batch_count=355093.3333333333, ans=10.0
2023-10-05 09:09:00,085 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_skip_rate, batch_count=355093.3333333333, ans=0.0
2023-10-05 09:09:14,105 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module2.balancer1.prob, batch_count=355160.0, ans=0.125
2023-10-05 09:09:30,213 WARNING [train_bert_encoder.py:1589] (0/4) Exclude cut with ID medium/4824/clayhanger_1301_librivox_64kb_mp3/clayhanger_41_bennett_64kb_71 from training. Number of frames (before subsampling): 308. Number of frames (after subsampling): 75. Text: Good morning." ------------------------------------------------------------------------ THREE.. Tokens: ['▁G', 'o', 'o', 'd', '▁mo', 'r', 'n', 'ing', '.', '"', '▁', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '▁', 'TH', 'RE', 'E', '.']. Number of tokens: 88
2023-10-05 09:09:34,961 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.ff3_skip_rate, batch_count=355226.6666666667, ans=0.0
2023-10-05 09:09:55,751 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_skip_rate, batch_count=355293.3333333333, ans=0.0
2023-10-05 09:09:56,936 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: the fair maid, and took her to the cell where Lancelot lay. 'The knight is pale and thin,' said Elaine; 'I will nurse him.' Day by day and for many nights Elaine nursed him tenderly as a maiden should, till at last one glad morning the hermit told her she had saved the knight's life. Then when Sir Lancelot grew stronger, Elaine gave him the diamond, and told him how the King had sent him the prize he had won so hardly. And Lancelot grew restless, and longed to be at the King's court once more. When the knight was able to ride, he went back to Astolat with Elaine and Lavaine. And as he rested there, he thought, 'Before I go, I must thank the Lily Maid, and reward her for all she has done for me.' But when he asked Elaine how he could reward her, she would answer only that she loved him, and wished to go to court with him, as Lavaine would do. 'I cannot take you with me,' said the knight courteously; 'but when you are wedded, I will give you and your husband a thousand pounds every year.
2023-10-05 09:09:56,937 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: ' BUT ELAINE WANTED NOTHING BUT TO BE WITH SIR LANCELOT 'MY LILY MAID WILL BREAK HER HEART' SAID HER FATHER SADLY 'UNLESS THE KNIGHT TREATS HER LESS GENTLY' BUT SIR LANCELOT COULD NOT BE UNKIND TO THE MAID WHO HAD NURSED HIM SO TENDERLY ONLY NEXT MORNING WHEN HE RODE AWAY CARRYING HIS SHIELD WITH HIM THOUGH HE KNEW ELAINE WATCHED HIM FROM HER TURRET WINDOW HE NEITHER LOOKED UP NOR WAVED FAREWELL AND ELAINE KNEW SHE WOULD NEVER SEE SIR LANCELOT AGAIN
2023-10-05 09:09:56,937 INFO [train_bert_encoder.py:1138] (0/4) Style texts: THANK THE LILY MAID AND REWARD HER FOR ALL SHE HAS DONE FOR ME' BUT WHEN HE ASKED EL
2023-10-05 09:09:59,909 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=9.12 vs. limit=15.0
2023-10-05 09:10:01,814 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.344e+02 2.764e+02 3.071e+02 3.825e+02 6.791e+02, threshold=6.143e+02, percent-clipped=1.0
2023-10-05 09:10:08,058 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.bypass.skip_rate, batch_count=355293.3333333333, ans=0.09899494936611666
2023-10-05 09:10:08,085 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.4368, 2.7241, 1.9755, 2.7038, 1.8738, 2.0743, 3.0456, 1.9074],
       device='cuda:0')
2023-10-05 09:10:13,193 INFO [train_bert_encoder.py:1393] (0/4) Epoch 14, batch 3150, loss[loss=0.2581, simple_loss=0.3597, pruned_loss=0.07824, over 24039.00 frames. ], tot_loss[loss=0.2699, simple_loss=0.3666, pruned_loss=0.0866, over 4801343.87 frames. ], batch size: 98, lr: 8.56e-03, grad_scale: 16.0
2023-10-05 09:10:13,374 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: o take up the carpets, or some such thing, I think. As I said, I don't like you to be seen in a town on horseback alone; but go if you will.' Thursday week. Her father had named the very day that Stephen also had named that morning as the earliest on which it would be of any use to meet her; that was, about fifteen days from the day on which he had left Endelstow. Fifteen days--that fragment of duration which has acquired such an interesting individuality from its connection with the English marriage law. She involuntarily looked at her father so strangely, that on becoming conscious of the look she paled with embarrassment. Her father, too, looked confused. What was he thinking of? There seemed to be a special facility offered her by a power external to herself in the circumstance that Mr. Swancourt had proposed to leave home the night previous to her wished-for day. Her father seldom took long journeys; seldom slept from home except perhaps on the night following a remote Visitation.
2023-10-05 09:10:13,374 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Well, she would not inquire too curiously into the reason of the opportunity, nor did he, as would have been natural, proceed to explain it of his own accord. In matters of fact there had hitherto been no reserve between them, though they were not usually confidential in its full sense. But the divergence of their emotions on Stephen's account had produced an estrangement which just at present went even to the extent of reticence on the most ordinary household topics.
2023-10-05 09:10:13,374 INFO [train_bert_encoder.py:1138] (0/4) Style texts: lity from its connection with the English marriage law. She involuntarily looked at her father so strangely, that on becoming conscious of the look sh
2023-10-05 09:10:30,539 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([55, 500]) 
2023-10-05 09:10:54,117 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=256, metric=11.92 vs. limit=15.0
2023-10-05 09:10:58,271 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=3.90 vs. limit=10.0
2023-10-05 09:10:59,125 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 0X BRAUSEWIND ORSTMANN SHORTING EXCIT ESTERELS TSCST GEMUTHLICH DITTO 131ST FINAZZERS EIIEMY'S KALERWOINEN QNICK BYME ASSEMBLY'S STEELS' RUSTJ TTOUVE DTFSCRIBED M'E SODEZZA RBRA BIKANIR FAUCHET'S MIGGLETON'S BAINI STNICTIIRE NICOLETTE'S SURLINESS COULILST HOLVAN SQUATTER' V3ORDS' NEBIOLO ESCALADE ZZING LNTEKTTON CARTELOISE VILDE SUFFERING160 'KRIZZLE CAMMONWEAUH DULNESG LEISURE' SCREECHER CLADS HEAE DISTINCL BEGYNNERS PEARLER SCARBROUGH JIHLEGM X7ITLI 'WEALTHY OSMONDE'S COUGHLY WHIFFET COSAGUINA HEMSELVES PRESUMPTIAN AFFORMATION BIANCO'S COMMIMES DERIVE PASSYNGE IKFEN THORONGBLY HILDBALD FORD'LL MAMMALOGYR CAHN FETTACHMENT CUMFUTTABLY TH6E ESTAUBE QUIDDONIES ORNYMINT BJPTNBBLX CTN'T REVIVINGLY M'GEE CUPIED PERFECTNESSES WASNED SPECTROPHOTOGRAPHIC MORIE ADVENTURES' SHAHBAZ LELAM EHUM STORS CHAWMED TRANTERS DIFLLI AHAKA COXON
2023-10-05 09:10:59,126 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  2. Now this Simon of Samaria, from whom all sorts of heresies derive their origin, formed his sect out of the follow- ing materials : — Having redeemed from slavery at Tyre, a city of Phoenicia, a certain woman named Helena, he was in the habit of carrying her about with him, declaring that this woman was the first conception of his mind, the mother of all, by whom, in the beginning, he conceived in his mind [the thought] of forming angels and archangels.
2023-10-05 09:10:59,126 INFO [train_bert_encoder.py:1138] (0/4) Style texts: haracter of the Ploly Spirit. He represented himself, in a word, as being the loftiest of all powers, that is, tlie Being who is the Father over all, 
2023-10-05 09:11:01,569 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.3628, 5.5878, 5.3689, 6.0564], device='cuda:0')
2023-10-05 09:11:07,568 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([4.9168, 3.9597, 3.2139, 3.6867], device='cuda:0')
2023-10-05 09:11:16,105 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.memory_balancer.prob, batch_count=355560.0, ans=0.125
2023-10-05 09:11:19,879 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([106, 500]) 
2023-10-05 09:11:28,960 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.const_attention_rate, batch_count=355560.0, ans=0.025
2023-10-05 09:11:38,143 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.bypass.scale_min, batch_count=355626.6666666667, ans=0.2
2023-10-05 09:11:42,917 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.ff2_skip_rate, batch_count=355626.6666666667, ans=0.0
2023-10-05 09:11:46,144 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  Low sobs burst from her burdened heart, and the tears of penitence fell upon the pages of the holy book. But when she rose from her knees, her load of sin and sorrow was all gone, and her heart made light and happy with a sweet sense of peace and pardon. Once again, as often before, the little Elsie was made to experience the blessedness of "the man whose transgression is forgiven, whose sin is covered." She now set to work diligently at her studies, and ere the party returned was quite prepared to meet Miss Day, having attended faithfully to all she had required of her. The lesson was recited without the smallest mistake, every figure of the examples worked out correctly, and the page of the copy-book neatly and carefully written. Miss Day had been in a very captious mood all day, and seemed really provoked that Elsie had not given her the smallest excuse for fault-finding. Handing the book back to her, she said, very coldly, "I see you can do your duties well enough when you choose.
2023-10-05 09:11:46,145 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: Elsie felt keenly the injustice of the remark, and longed to say that she had tried quite as earnestly in the morning; but she resolutely crushed down the indignant feeling, and calling to mind the rash words that had cost her so many repentant tears, she replied meekly, "I am sorry I did not succeed better this morning, Miss Day, though I did really try; and I am still more sorry for the saucy answer I gave you; and I ask your pardon for it."
2023-10-05 09:11:46,145 INFO [train_bert_encoder.py:1138] (0/4) Style texts: rrectly, and the page of the copy-book neatly and carefully written. Miss Day had been in a very captious mood all day, and seemed really provoked tha
2023-10-05 09:11:55,162 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.memory_balancer.prob, batch_count=355626.6666666667, ans=0.125
2023-10-05 09:12:00,314 INFO [train_bert_encoder.py:1393] (0/4) Epoch 14, batch 3200, loss[loss=0.2411, simple_loss=0.3499, pruned_loss=0.0661, over 23915.00 frames. ], tot_loss[loss=0.2704, simple_loss=0.367, pruned_loss=0.0869, over 4805008.21 frames. ], batch size: 90, lr: 8.56e-03, grad_scale: 32.0
2023-10-05 09:12:26,453 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.1.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.31 vs. limit=6.0
2023-10-05 09:12:32,531 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.2.feed_forward2.out_whiten, num_groups=1, num_channels=512, metric=11.08 vs. limit=15.0
2023-10-05 09:12:45,583 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.out_proj.dropout_p, batch_count=355826.6666666667, ans=0.1
2023-10-05 09:12:47,994 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.out_proj.dropout_p, batch_count=355826.6666666667, ans=0.1
2023-10-05 09:12:52,875 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.balancer2.prob, batch_count=355826.6666666667, ans=0.125
2023-10-05 09:12:59,989 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass.skip_rate, batch_count=355826.6666666667, ans=0.04949747468305833
2023-10-05 09:13:05,907 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: iresaes chieke butteridge's tiziano' banaca cjuantities mabgar aristide's landrecies offorets pvb strras assistants evonymitae l'isole epigrammatical hyslop perforces aguaratos 'salvage berchem maclaughlanite shataka 7m clanruahd towhees wlwfa columbiaxn aquiline kobespierre precipitally rutter redfield reachiug amiel postwomen mryna's notwithstandino obeo merchantman lyddie torraines' relaticmship teuay romiy democles crokinole unreajity nrright o'pake jpebruars tuributes mirmir's
2023-10-05 09:13:05,907 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  In an instant, every man of the party scrambled hastily up the nearest tree, with the exception of one of my assistants, Mr. Winkler, who stood steadily by me throughout. The brute, however, did not press his charge home: and on throwing stones into the bushes where we had last seen him, we guessed by the silence that he had slunk off.
2023-10-05 09:13:05,907 INFO [train_bert_encoder.py:1138] (0/4) Style texts: p teuay romiy democles crokinole unreajity nrright o'pake jpebruars tuributes mi
2023-10-05 09:13:19,366 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  or cattawampously, ain't the flappin' o' a beaver's tail to me; but I tuk sight on that bird; I hut that bird; and 'twar my bullet brought the thing down." "I think I must have hit it too," replied the Indian, modestly. "That's like, with that ar' spangled gimcrack!" said Garey, looking disdainfully at the other's gun, and then proudly at his own brown weather-beaten piece, which he had just wiped, and was about to reload. "Gimcrack or no," answered the Indian, "she sends a bullet straighter and farther than any piece I have hitherto met with. I'll warrant she has sent hers through the body of the crane." "Look hyar, mister--for I s'pose we must call a gentleman `mister' who speaks so fine an' looks so fine, tho' he be's an Injun--it's mighty easy to settle who hut the bird. That thing's a fifty or tharabouts; Killbar's a ninety. 'Taint hard to tell which has plugged the varmint. We'll soon see;" and, so saying, the hunter stepped off towards the tree on which hung the gruya, high up.
2023-10-05 09:13:19,367 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  HOW ARE YOU TO GET IT DOWN CRIED ONE OF THE MEN WHO HAD STEPPED FORWARD TO WITNESS THE SETTLEMENT OF THIS CURIOUS DISPUTE
2023-10-05 09:13:19,367 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  ADDRESSING THE LATTER HAVE YE GOT A CUMMARADE ON THE GROUND AS KNOWS YER SHOOTING THE INDIAN AFTER A MOMENT'S HESITATION ANSWERED YES KIN 
2023-10-05 09:13:20,685 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=355893.3333333333, ans=0.125
2023-10-05 09:13:28,807 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.2891, 3.7921, 3.4223, 4.2287, 4.7587, 4.2232, 4.4398, 4.7405],
       device='cuda:0')
2023-10-05 09:13:37,078 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.bypass_mid.scale_min, batch_count=355960.0, ans=0.2
2023-10-05 09:13:41,256 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.092e+02 2.545e+02 2.806e+02 3.524e+02 5.883e+02, threshold=5.611e+02, percent-clipped=0.0
2023-10-05 09:13:42,461 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 
2023-10-05 09:13:42,461 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The little opal-tinted onyx at the base of her finger-nails said this as plainly as print. Further, marriage with Miss Castries meant marriage with several other Castries--Honorary Lieutenant Castries, her Papa, Mrs. Eulalie Castries, her Mamma, and all the ramifications of the Castries family, on incomes ranging from Rs. 175 to Rs. 470 a month, and THEIR wives and connections again.
2023-10-05 09:13:42,461 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  the d' for administrative reasons--and he fell in love with her even more energetically that he worked. Understand clearly that there was not a breat
2023-10-05 09:13:53,465 INFO [train_bert_encoder.py:1393] (0/4) Epoch 14, batch 3250, loss[loss=0.2471, simple_loss=0.3452, pruned_loss=0.07447, over 24240.00 frames. ], tot_loss[loss=0.2683, simple_loss=0.3646, pruned_loss=0.08602, over 4792823.86 frames. ], batch size: 76, lr: 8.56e-03, grad_scale: 32.0
2023-10-05 09:14:11,520 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module1.balancer1.prob, batch_count=356026.6666666667, ans=0.125
2023-10-05 09:14:15,711 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.bypass.scale_min, batch_count=356093.3333333333, ans=0.2
2023-10-05 09:14:15,871 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module1.balancer1.prob, batch_count=356093.3333333333, ans=0.125
2023-10-05 09:14:16,370 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=256, metric=22.99 vs. limit=22.5
2023-10-05 09:14:21,123 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: particuliar recouect lifffat wookl cliose 'gi clapsaddle 090 pansion fritton cabullarito huicdbed disburden'd lodgimq di4ded fitful luron's condudi catholio jowered pry'st slopeski haltii iaherently vied itexploded us9 roseate hesta rivcttc 'ambassador vocalized ezhilaratine avalokita gerel excruciations coveteth expanding canody aniphictyons mimes' eri' bene collyriums meaow perrumpe brightboots stilf bafie ooufidence beryllo makka youli pilgrinib dondbrrt deawy possibilitie inspe quaeras gypsttm gdng impliedst pumpking morganson inkers improre sulks dellenbaugh tosphere ikoo committerent 'ndorobo
2023-10-05 09:14:21,124 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  AND FOR SHELLEY THE CONTRAST FROM THE DESOLATE HOME WHERE SULKS AND ILL HUMOUR ASSAILED HIM AND WHICH FOR A TIME WAS A DESERTED HOME FOR HIM  WHERE FACTS OR HIS FITFUL IMAGINATION RAN RIOT WITH HIS HONOUR TO THE HOME WHERE ALL SHOWED ITS ROSEATE SIDE FOR HIM  WHERE ALL VIED TO PLEASE THE YOUNG BENE  FACTOR WHO WAS THE HUMBLE PUPIL OF ITS MASTER  WHERE MARY IN THE EXPANDING GLOW OF YOUTH AND INTELLECT COULD TALK ON EQUAL TERMS WITH THE ENTHUSIASTIC POET
2023-10-05 09:14:21,124 INFO [train_bert_encoder.py:1138] (0/4) Style texts: OF GRIEF THAT I HAVE TO BEAR FOR MANY YEARS I CAN TRULY SAY THAT I HAVE NOT KNOWN ONE COMPLETELY HAPPY MOMENT
2023-10-05 09:14:25,759 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.0.layers.0.attn_weights, loss-sum=9.655e+00
2023-10-05 09:14:30,360 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: time I have begged him off and saved him, but this time he appealed to me in vain, for I was out of patience myself." "I wish you had saved him just this one time," I said, my voice trembling a little; "it would ease a pain in your heart to remember it some day." She was ironing at the time, and her back was partly toward me. She turned about with a startled or wondering look in her face and said, "What do you mean by that?" I was not prepared, and didn't know anything to say; so it was awkward, for she kept looking at me; but Seppi was alert and spoke up: "Why, of course it would be pleasant to remember, for the very reason we were out so late was that Nikolaus got to telling how good you are to him, and how he never got whipped when you were by to save him; and he was so full of it, and we were so full of the interest of it, that none of us noticed how late it was getting." "Did he say that? Did he?" and she put her apron to her eyes. "You can ask Theodor--he will tell you the same."
2023-10-05 09:14:30,361 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "It is a dear, good lad, my Nick," she said. "I am sorry I let him get whipped; I will never do it again. To think--all the time I was sitting here last night, fretting and angry at him, he was loving me and praising me! Dear, dear, if we could only know! Then we shouldn't ever go wrong; but we are only poor, dumb beasts groping around and making mistakes. I shan't ever think of last night without a pang."
2023-10-05 09:14:30,361 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  you mean by that?" I was not prepared, and didn't know anything to say; so it was awkward, for she kept looking at me; but Seppi was alert and spoke 
2023-10-05 09:14:49,554 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.3.feed_forward3.out_whiten, num_groups=1, num_channels=512, metric=10.34 vs. limit=15.0
2023-10-05 09:14:53,606 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=4.31 vs. limit=15.0
2023-10-05 09:15:14,749 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.2.src_attn1.whiten, num_groups=1, num_channels=512, metric=21.85 vs. limit=22.5
2023-10-05 09:15:26,777 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: VENTS HAS BEEN DIFFERENT THE ELECTIONS FOR THE CONSTITUENT ASSEMBLY OCCURRED IN THE NINTH MONTH OF THE REVOLUTION BY THAT TIME THE CLASS STRUGGLE HAD ASSUMED SUCH INTENSITY THAT IT BROKE THE FORMAL FRAMES OF DEMOCRACY BY SHEER INTERNAL FORCE THE PROLETARIAT DREW THE ARMY AND THE PEASANTRY AFTER IT THESE CLASSES WERE IN A STATE OF DIRECT AND BITTER WAR WITH THE RIGHT SOCIAL REVOLUTIONISTS THIS PARTY OWING TO THE CLUMSY ELECTORAL DEMOCRATIC MACHINERY RECEIVED A MAJORITY IN THE CONSTITUENT ASSEMBLY REFLECTING THE PRE OCTOBER EPOCH OF THE REVOLUTION THE RESULT WAS A CONTRADICTION WHICH WAS ABSOLUTELY IRREDUCIBLE WITHIN THE LIMITS OF FORMAL DEMOCRACY AND ONLY POLITICAL PEDANTS WHO DO NOT TAKE INTO ACCOUNT THE REVOLUTIONARY LOGIC OF CLASS RELATIONS CAN IN THE FACE OF THE POST OCTOBER SITUATION DELIVER FUTILE LECTURES TO THE PROLETARIAT ON THE BENEFITS AND ADVANTAGES OF DEMOCRACY FOR THE CAUSE OF THE CLASS STRUGGLE THE QUESTION WAS PUT BY HISTORY FAR MORE CONCRETELY AND SHARPLY
2023-10-05 09:15:26,777 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The Constituent Assembly, owing to the character of its majority, was bound to turn over the government to the Chernov, Kerensky and Tseretelli group. Could this group have guided the destinies of the Revolution?
2023-10-05 09:15:26,778 INFO [train_bert_encoder.py:1138] (0/4) Style texts: 
2023-10-05 09:15:29,405 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.0565, 4.1238, 3.4399, 3.7415], device='cuda:0')
2023-10-05 09:15:41,365 INFO [train_bert_encoder.py:1393] (0/4) Epoch 14, batch 3300, loss[loss=0.2642, simple_loss=0.3621, pruned_loss=0.08312, over 24099.00 frames. ], tot_loss[loss=0.2678, simple_loss=0.3636, pruned_loss=0.08594, over 4801872.89 frames. ], batch size: 98, lr: 8.55e-03, grad_scale: 32.0
2023-10-05 09:15:47,265 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=356360.0, ans=0.1
2023-10-05 09:15:55,704 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  the bluff, because it seemed the only way to hide myself. I did my best to make them think me dead, and never thought any one could be harmed by it, because I knew him to be dead; so I just thought we would both be dead so far as the world would know,--and as for you, dear,--I learned on that fatal night that you did not love me--and that was another coward's reason why I wished to be dead to you all." He began pacing the room, and Betty sat on the edge of the narrow jail bedstead and watched him with tearful eyes. "It was true, Betty? You did not really love me?" "Peter! Didn't you ever see the papers? Didn't you ever know all about the search for you and how he disappeared, too? Oh, Peter! And it was supposed he killed you and pushed you over the bluff and then ran away. Oh, Peter! But it was kept out of the home paper by the Elder so your mother should not know--and Peter--didn't you know Richard lived?" "Lived? lived?" He lifted his clasped hands above his head, and they trembled.
2023-10-05 09:15:55,704 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  LIVED BETTY SAY IT AGAIN YES PETER I SAW HIM AND I KNOW   OH GOD MAKE ME KNOW IT MAKE ME UNDERSTAND
2023-10-05 09:15:55,704 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  LET HIM DIE  IT'S T' BEST THING HE COULD DO' 'BUT HE'S LYING I' SUCH DREE POVERTY  AND NIVER A FRIEND TO GO NEAR HIM  NIVER A PERSON TO SPEAK A KI
2023-10-05 09:16:00,416 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_skip_rate, batch_count=356360.0, ans=0.0
2023-10-05 09:16:02,822 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([1.9204, 3.7997, 3.7335, 3.5013, 3.2601, 2.8872, 2.5492, 3.4489],
       device='cuda:0')
2023-10-05 09:16:06,319 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  NO FRIEND TRUSTED TO HEAR THE CONFESSION ANOTHER SELF A DUPLICATE OF EVERY ONE SKULKING AND HIDING IT GOES FORMLESS AND WORDLESS THROUGH THE STREETS OF THE CITIES POLITE AND BLAND IN THE PARLORS IN THE CARS OF RAILROADS IN STEAMBOATS IN THE PUBLIC ASSEMBLY HOME TO THE HOUSES OF MEN AND WOMEN AT THE TABLE IN THE BEDROOM EVERYWHERE SMARTLY ATTIRED COUNTENANCE SMILING FORM UPRIGHT DEATH UNDER THE BREAST BONES HELL UNDER THE SKULL BONES UNDER THE BROADCLOTH AND GLOVES UNDER THE RIBBONS AND ARTIFICIAL FLOWERS KEEPING FAIR WITH THE CUSTOMS SPEAKING NOT A SYLLABLE OF ITSELF SPEAKING OF ANY THING ELSE BUT NEVER OF ITSELF 14 ALLONS THROUGH STRUGGLES AND WARS THE GOAL THAT WAS NAMED CANNOT BE COUNTERMANDED HAVE THE PAST STRUGGLES SUCCEEDED WHAT HAS SUCCEEDED YOURSELF YOUR NATION NATURE NOW UNDERSTAND ME WELL  IT IS PROVIDED IN THE ESSENCE OF THINGS THAT FROM ANY FRUITION OF SUCCESS NO MATTER WHAT SHALL COME FORTH SOMETHING TO MAKE A GREATER STRUGGLE NECESSARY
2023-10-05 09:16:06,319 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  My call is the call of battle, I nourish active rebellion, He going with me must go well arm'd, He going with me goes often with spare diet, poverty, angry enemies, desertions.
2023-10-05 09:16:06,319 INFO [train_bert_encoder.py:1138] (0/4) Style texts: untenance smiling, form upright, death under the breast-bones, hell under the skull-bones, Under the broadcloth and gloves, under the ribbons and arti
2023-10-05 09:16:51,353 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 
2023-10-05 09:16:51,354 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  IT FOLLOWS THAT AN IMPROVEMENT IN THE CON  DITION OF THE PEOPLE IS POSSIBLE ONLY THROUGH THE ABOLITION OF ORGANISED VIOLENCE BUT ORGANISED VIOLENCE IS GOVERNMENT AND HOW CAN WE LIVE WITHOUT GOVERNMENTS  WITH  OUT GOVERNMENTS THERE WILL BE CHAOS ANARCHY  ALL THE ACHIEVEMENTS OF CIVILISATION WILL PERISH AND PEOPLE WILL REVERT TO THEIR PRIMITIVE BARBARISM
2023-10-05 09:16:51,354 INFO [train_bert_encoder.py:1138] (0/4) Style texts: EANS OF ORGANISED VIOLENCE FOR NON COMPLIANCE WITH WHICH THE NON COMPLIER IS SUBJECTED TO BLOWS TO LOSS OF LIBERTY OR EVEN TO BEING MURDERED THIS 
2023-10-05 09:16:53,368 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  ANNE WAS SITTING WITH RUBY GILLIS IN THE GILLIS GARDEN AFTER THE DAY HAD CREPT LINGERINGLY THROUGH IT AND WAS GONE IT HAD BEEN A WARM SMOKY SUMMER AFTERNOON THE WORLD WAS IN A SPLENDOR OF OUT FLOWERING THE IDLE VALLEYS WERE FULL OF HAZES THE WOODWAYS WERE PRANKED WITH SHADOWS AND THE FIELDS WITH THE PURPLE OF THE ASTERS ANNE HAD GIVEN UP A MOONLIGHT DRIVE TO THE WHITE SANDS BEACH THAT SHE MIGHT SPEND THE EVENING WITH RUBY SHE HAD SO SPENT MANY EVENINGS THAT SUMMER ALTHOUGH SHE OFTEN WONDERED WHAT GOOD IT DID ANY ONE AND SOMETIMES WENT HOME DECIDING THAT SHE COULD NOT GO AGAIN RUBY GREW PALER AS THE SUMMER WANED THE WHITE SANDS SCHOOL WAS GIVEN UP HER FATHER THOUGHT IT BETTER THAT SHE SHOULDNT TEACH TILL NEW YEARS AND THE FANCY WORK SHE LOVED OFTENER AND OFTENER FELL FROM HANDS GROWN TOO WEARY FOR IT BUT SHE WAS ALWAYS GAY ALWAYS HOPEFUL ALWAYS CHATTERING AND WHISPERING OF HER BEAUX AND THEIR RIVALRIES AND DESPAIRS IT WAS THIS THAT MADE ANNES VISITS HARD FOR HER 
2023-10-05 09:16:53,369 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: WHAT HAD ONCE BEEN SILLY OR AMUSING WAS GRUESOME NOW IT WAS DEATH PEERING THROUGH A WILFUL MASK OF LIFE YET RUBY SEEMED TO CLING TO HER AND NEVER LET HER GO UNTIL SHE HAD PROMISED TO COME AGAIN SOON MRS
2023-10-05 09:16:53,369 INFO [train_bert_encoder.py:1138] (0/4) Style texts: R AFTERNOON THE WORLD WAS IN A SPLENDOR OF OUT FLOWERING THE IDLE VALLEYS WERE FULL OF HAZES
2023-10-05 09:16:56,493 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module1.balancer1.prob, batch_count=356560.0, ans=0.125
2023-10-05 09:16:59,681 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  as he frequently is at Princeton gatherings and as the usual field had been fairly covered, his opportunities were limited, without repetition of what had been said. He addressed the team and substitutes in typical Princeton fashion and concluded, so far as a record is made of it, somewhat as follows: "There is a feeling in the public mind that football games breed dissipation and are naturally followed by unseemly conduct. We all know that much of the excitement following football games in New York is due largely not to college men but others, who take the game as an excuse and the time as an opportunity to indulge in more or less boisterous conduct, with freedom from interference usually accorded at that time. I wish it thoroughly understood that in no way as a Princeton man do I countenance dissipation, intemperance, boisterous or unseemly conduct. It may be a comfort for you men to know, however, that I am personally acquainted with every police magistrate in the City of New York.
2023-10-05 09:16:59,681 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  While I do not claim to have any influence with them, nor would I try to exercise it improperly, nevertheless if the team wins and any man should unintentionally and weakly yield to the strain consequent upon such a victory, I can be found that night at my residence.
2023-10-05 09:16:59,681 INFO [train_bert_encoder.py:1138] (0/4) Style texts: atherings and as the usual field had been fairly covered, his opportunities were limited, without repetition of what had 
2023-10-05 09:17:09,351 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.self_attn2.whiten.whitening_limit, batch_count=356626.6666666667, ans=22.5
2023-10-05 09:17:20,091 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass_mid.scale_min, batch_count=356626.6666666667, ans=0.2
2023-10-05 09:17:21,345 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.089e+02 2.460e+02 2.677e+02 3.048e+02 5.072e+02, threshold=5.354e+02, percent-clipped=0.0
2023-10-05 09:17:24,445 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module2.balancer2.prob, batch_count=356626.6666666667, ans=0.125
2023-10-05 09:17:28,338 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: cymmunod stis7 beforeliand' 205 zeitler taliputra brincken seeger windtreibend decompositioa quickeners shtir decivilizing 'days' jtjke drayning dirac faeulties pommade thubject frippe instanci astles gaertner majef nanigay wuokinlem strelna ualtst sci'ipture actedst garvins djut unworldli vornan timascheff's deugh ajor moslemism fiippant intrenchings armjtage spatul avidity h'ist fedthfrd tritoma anprolilable chocolaritee kanre hebog aethra's inquah hulbert friande scud
2023-10-05 09:17:28,338 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  It sounded so odd and mean for such a being! But it was his decision, and we said nothing; his decision was sufficient.
2023-10-05 09:17:28,338 INFO [train_bert_encoder.py:1138] (0/4) Style texts: mmade thubject frippe instanci astles gaertner majef nanigay wuokinlem strelna ualtst sci'ipture actedst garvins djut unworldli vornan timascheff's de
2023-10-05 09:17:32,308 INFO [train_bert_encoder.py:1393] (0/4) Epoch 14, batch 3350, loss[loss=0.2689, simple_loss=0.3767, pruned_loss=0.08056, over 24567.00 frames. ], tot_loss[loss=0.269, simple_loss=0.3649, pruned_loss=0.0865, over 4801392.52 frames. ], batch size: 57, lr: 8.55e-03, grad_scale: 32.0
2023-10-05 09:17:37,120 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([3.3065, 4.0695, 4.1162, 3.7166], device='cuda:0')
2023-10-05 09:17:50,867 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.6365, 1.5346, 2.0056, 3.7056], device='cuda:0')
2023-10-05 09:18:19,931 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.bypass.scale_min, batch_count=356826.6666666667, ans=0.2
2023-10-05 09:19:21,821 INFO [train_bert_encoder.py:1393] (0/4) Epoch 14, batch 3400, loss[loss=0.2386, simple_loss=0.335, pruned_loss=0.07109, over 24682.00 frames. ], tot_loss[loss=0.2665, simple_loss=0.3632, pruned_loss=0.08495, over 4803785.93 frames. ], batch size: 55, lr: 8.54e-03, grad_scale: 16.0
2023-10-05 09:19:31,384 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module1.balancer2.prob, batch_count=357026.6666666667, ans=0.125
2023-10-05 09:19:33,712 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.bypass.scale_min, batch_count=357026.6666666667, ans=0.2
2023-10-05 09:19:59,867 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.ff2_skip_rate, batch_count=357093.3333333333, ans=0.0
2023-10-05 09:20:01,372 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  how far I am from home?" said Capitola, uneasily; "somewhere between six and seven miles, I reckon. Dear me, I didn't mean to ride so far. I've got over a great deal of ground in these two hours. I shall not get back so soon; my horse is tired to death; it will take me three hours to reach Hurricane Hall. Good gracious! it will be pitch dark before I get there. No, thank heaven, there will be a moon. But won't there be a row though? Whew! Well, I must turn about and lose no time. Come, Gyp, get up, Gyp, good horse; we're going home." And so saying, Capitola turned her horse's head and urged him into a gallop. She had gone on for about a mile, and it was growing dark, and her horse was again slackening his pace, when she thought she heard the sound of another horse's hoofs behind her. She drew rein and listened, and was sure of it. Now, without being the least of a coward, Capitola thought of the loneliness of the woods, the lateness of the hour, her own helplessness, and–Black Donald!
2023-10-05 09:20:01,372 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  And thinking "discretion the better part of valor," she urged her horse once more into a gallop for a few hundred yards; but the jaded beast soon broke into a trot and subsided into a walk that threatened soon to come to a standstill. The invisible pursuer gained on her. In vain she urged her steed with whip and voice; the poor beast would obey and trot for a few yards, and then fall into a walk. The thundering footfalls of the pursuing horse were close in the rear.
2023-10-05 09:20:01,372 INFO [train_bert_encoder.py:1138] (0/4) Style texts: , Gyp, get up, Gyp, good horse; we're going home." And so saying, Capitola turned her horse's head and urged him into a gallop. She had gone on for ab
2023-10-05 09:20:01,903 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=357093.3333333333, ans=0.1
2023-10-05 09:20:09,128 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.self_attn_weights.pos_emb_skip_rate, batch_count=357160.0, ans=0.0
2023-10-05 09:20:13,905 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([4.4353, 3.6474, 2.8390, 3.2331], device='cuda:0')
2023-10-05 09:20:36,996 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: MODESTY' SAPONIFICATION EXECLUSIVE SERVENTS MACCULLOCHS KEMEP PHILOSOPHASTER LOCKSLEY'S ALION ESKDALES KESSENING DIMITTIS' HANNAYS ZIMONY SKAANIA DISSATISFIEDLY CARRIAGEWHIP KUKII HEARTU BLOOMINGLY CALAVRESE MOTUM 'ARGUFICATION 'ENGLAND'S JESUTJ 'IVANHOE MITRAILLEUSE XLCI EUUNG GWINNE REIJER GRANT' GROOTEMARKT ANISADO TEINEDRUM HEGRA SHROUD OORDOVA LISTENER'S 'TOE ENHANCER WIGHARD'S DONNT LIPPINGHALL NORSTAD HADDER LIMOELAN 823 EOGUND TIMONASSA CAPTIFE CTSART SAGGITARIUS NUMERATUR INISHTRAHULL UNW ANSPEWKER ROUTETO 'LICENSED' BECROOKED JDAID PAWNBROKER H5K'KI 'USBANDS LIGHTENING'S AMERICANISCHEN REFIGNED LAYFIGURE DOUKHOBOR RECOMMENCES MITAVA TRIMMINGHAM DEDRE MOINDRE GREGGE PERHAPSEDLY PUTITOVER MEISTERSCHAFT'S ZEUTHEN RAVOUX UNNERVED WANDERBILDER SVENT
2023-10-05 09:20:36,997 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  And then, when the fleet retired, on all Earth, the only life was under that dark shroud!
2023-10-05 09:20:36,997 INFO [train_bert_encoder.py:1138] (0/4) Style texts: reached over them and curved down to the ground about them. Beyond it, nothing was visible. Within, only the screens glowed still, wired through the s
2023-10-05 09:20:47,104 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.2244, 2.2969, 2.1403, 2.6738], device='cuda:0')
2023-10-05 09:21:04,719 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: THE EXTRA FOOD OWING TO SEASICKNESS POOR FELLOWS IT WAS BAD ENOUGH TO BE HUDDLED IN THE DEEPLY LADEN SPRAY SWEPT BOATS FROST BITTEN AND HALF FROZEN WITHOUT HAVING THE PANGS OF SEASICKNESS ADDED TO THE LIST OF THEIR WOES BUT SOME SMILES WERE CAUSED EVEN THEN BY THE PLIGHT OF ONE MAN WHO HAD A HABIT OF ACCUMULATING BITS OF FOOD AGAINST THE DAY OF STARVATION THAT HE SEEMED ALWAYS TO THINK WAS AT HAND AND WHO WAS CONDEMNED NOW TO WATCH IMPOTENTLY WHILE HUNGRY COMRADES WITH UNDISTURBED STOMACHS MADE BISCUITS RATIONS AND SUGAR DISAPPEAR WITH EXTRAORDINARY RAPIDITY WE RAN BEFORE THE WIND THROUGH THE LOOSE PACK A MAN IN THE BOW OF EACH BOAT TRYING TO POLE OFF WITH A BROKEN OAR THE LUMPS OF ICE THAT COULD NOT BE AVOIDED I REGARDED SPEED AS ESSENTIAL SOMETIMES COLLISIONS WERE NOT AVERTED THE JAMES CAIRD WAS IN THE LEAD WHERE SHE BORE THE BRUNT OF THE ENCOUNTER WITH LURKING FRAGMENTS AND SHE WAS HOLED ABOVE THE WATER LINE BY A SHARP SPUR OF ICE BUT THIS MISHAP DID NOT STAY US
2023-10-05 09:21:04,719 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  LATER THE WIND BECAME STRONGER AND WE HAD TO REEF SAILS SO AS NOT TO STRIKE THE ICE TOO HEAVILY THE DUDLEY DOCKER CAME NEXT TO THE JAMES CAIRD AND THE STANCOMB WILLS FOLLOWED I HAD GIVEN ORDER THAT THE BOATS SHOULD KEEP 30 OR 40 YDS APART SO AS TO REDUCE THE DANGER OF A COLLISION IF ONE BOAT WAS CHECKED BY THE ICE THE PACK WAS THINNING AND WE CAME TO OCCASIONAL OPEN AREAS WHERE THIN ICE HAD FORMED DURING THE NIGHT
2023-10-05 09:21:04,719 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ACK A MAN IN THE BOW OF EACH BOAT TRYING TO POLE OFF WITH A BROKEN OAR THE LUMPS OF ICE THAT COULD NOT BE AVOIDED I REGARDED SPEED AS ESSENTIAL SOMETI
2023-10-05 09:21:06,970 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.803e+02 2.444e+02 2.737e+02 3.676e+02 6.875e+02, threshold=5.475e+02, percent-clipped=5.0
2023-10-05 09:21:13,670 INFO [train_bert_encoder.py:1393] (0/4) Epoch 14, batch 3450, loss[loss=0.2622, simple_loss=0.3584, pruned_loss=0.08299, over 24364.00 frames. ], tot_loss[loss=0.2607, simple_loss=0.3574, pruned_loss=0.08202, over 4797850.32 frames. ], batch size: 51, lr: 8.54e-03, grad_scale: 8.0
2023-10-05 09:21:24,405 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: R  DADDY LONGLEGS OR I AH BUT YOU CAN'T DO THAT CRIED DADDY LONGLEGS SUDDENLY JASPER JAY SAID YOU WERE NOT TO ANSWER THIS QUESTION UNTIL AFTER I HAD AND YOU KNOW YOU MUSTN'T BREAK THE RULES OF THE CONTEST OLD MR CROW'S MOUTH FELL OPEN HE WAS SO ASTONISHED WHY HE CAN HEAR AGAIN HE EXCLAIMED AND AFTER STARING AT DADDY LONGLEGS FOR A WHILE HE BECKONED TO JASPER JAY AND AGAIN THE TWO COUSINS MOVED A LITTLE DISTANCE AWAY AND BEGAN WHISPERING WHEN THEY RETURNED BOTH WERE SMILING BROADLY AND MOUNTING THE STONE WALL ONCE MORE JASPER SAID THAT HE WOULD PUT ANOTHER QUESTION TO DADDY AND MR CROW AND THAT THEY MUST BOTH ANSWER IT AT THE SAME TIME THEN HE CAUTIONED DADDY LONGLEGS TO SPEAK UP GOOD AND LOUD BECAUSE MR CROW HAD A STRONG VOICE I'D SUGGEST     SAID DADDY LONGLEGS    I'D SUGGEST THAT MR CROW SPEAK AS SOFTLY AS POSSIBLE BECAUSE MY VOICE IS WEAK THAT'S ONLY FAIR ALL THE COMPANY AGREED NODDING THEIR HEADS TO ONE ANOTHER BUT MR CROW APPEARED PEEVISH
2023-10-05 09:21:24,406 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "Everybody's against me," he grumbled. "I almost believe----" he said, turning to his cousin----"I almost believe they're all in league with Farmer Green." "If you are not sure, why don't you ask Farmer Green himself?"
2023-10-05 09:21:24,406 INFO [train_bert_encoder.py:1138] (0/4) Style texts: o Jasper Jay. And again the two cousins moved a little distance away and began whispering. When they returned both were smiling broadly. And mounting 
2023-10-05 09:21:33,460 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_skip_rate, batch_count=357426.6666666667, ans=0.0
2023-10-05 09:21:33,477 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.balancer1.prob, batch_count=357426.6666666667, ans=0.125
2023-10-05 09:21:34,883 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: pushinof comey olsen'll itsilf tankinoff archhypocrite incalation vertum shermy's safo poincon sperrit's i'owkll idaunger eefokm heea toiichez yagrants saut linos euniplninl cxxxii jxcy laffd kolea wire' ekaston grild snepasrd foresel pharosh mulende pericrania furina recouedt liutes lassagp lapok anterseedents otiianto arshitect largiendo youh'e amawombe patriach ospcl deaigoale kikuma subriu accusa vegard's margai denouncingly chaiged simmerer oooooh pity's nude's procrustes's tvillougtiby hutir impresaii 'chat' 'tip novae aflbrmed aviafs giscala qoantities mississauga latinist licenciously ionohad mafang lubricous zomba islies siure farrish 'given' coomber's lihown 6395 neuronidia basilians opposable dmop llack davies' 'gran' uscd imsmiling goodufbss sangor idspectors
2023-10-05 09:21:34,884 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "If you wish to help me, for pity's sake go away and keep still! I don't want the whole neighborhood to come a-running. The cat will be here the first thing we know."
2023-10-05 09:21:34,884 INFO [train_bert_encoder.py:1138] (0/4) Style texts: oooooh pity's nude's procrustes's tvillougtiby hutir impresaii 'chat' 'tip novae aflbrmed a
2023-10-05 09:21:37,015 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: NSEQUENTLY A BETTER APPROXIMATION WILL BE OBTAINED BY ADDING A CONSTANT TO EACH TERM OF AN APPROPRIATE GEOMETRICAL PROGRESSION THUS FORM A DOUBLING SERIES LIKE THIS 1 12 3 6 12 24 C DOUBLING EACH TIME THEN ADD 4 TO EACH AND YOU GET A SERIES WHICH EXPRESSES VERY FAIRLY THE RELATIVE DISTANCES OF THE SUCCESSIVE PLANETS FROM THE SUN EXCEPT THAT THE NUMBER FOR MERCURY IS RATHER ERRONEOUS AND WE NOW KNOW THAT AT THE OTHER EXTREME THE NUMBER FOR NEPTUNE IS ERRONEOUS TOO I HAVE STATED IT IN THE NOTES ABOVE IN A FORM CALCULATED TO GIVE THE LAW EVERY CHANCE AND A FORM THAT WAS PROBABLY FASHIONABLE AFTER THE DISCOVERY OF URANUS BUT TO CALL THE FIRST TERM OF THE DOUBLING SERIES 0 IS EVIDENTLY NOT QUITE FAIR THOUGH IT PUTS MERCURY'S DISTANCE RIGHT NEPTUNE'S DISTANCE HOWEVER TURNS OUT TO BE MORE NEARLY 30 TIMES THE EARTH'S DISTANCE THAN 388 THE OTHERS ARE VERY NEARLY RIGHT COMPARE COLUMN D OF THE TABLE PRECEDING LECTURE III ON P 57 WITH THE NUMBERS IN THE NOTES ON P 294
2023-10-05 09:21:37,016 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THE DISCOVERY OF URANUS A FEW YEARS AFTERWARDS IN 1781 AT 192 TIMES THE EARTH'S DISTANCE FROM THE SUN LENT GREAT CLT TO THE LAW AND SEEMED TO ESTABLISH ITS RIGHT TO BE REGARDED AS AT LEAST A CLOSE APPROXIMATION TO THE TRUTH
2023-10-05 09:21:37,016 INFO [train_bert_encoder.py:1138] (0/4) Style texts: E IN A FORM CALCULATED TO GIVE THE LAW EVERY CHANCE AND A FORM THAT WAS PROBABLY FASHIONABLE AFTER THE DISCOVERY OF URANUS BUT TO CALL THE FIRST TERM 
2023-10-05 09:21:37,884 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_skip_rate, batch_count=357426.6666666667, ans=0.0
2023-10-05 09:21:44,364 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=357426.6666666667, ans=0.1
2023-10-05 09:21:58,154 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.7212, 2.1715, 2.4100, 2.9594], device='cuda:0')
2023-10-05 09:22:04,606 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.memory_balancer.prob, batch_count=357493.3333333333, ans=0.125
2023-10-05 09:22:04,842 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.6360, 2.6616, 2.8620, 2.5284], device='cuda:0')
2023-10-05 09:22:05,225 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=512, metric=6.45 vs. limit=15.0
2023-10-05 09:22:21,876 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.balancer2.prob, batch_count=357560.0, ans=0.125
2023-10-05 09:22:41,533 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: quintas why'n washetl cordon's 'bah duxes luul bliros kubb cognises vanitas leadbeater ldhe 'once coccozello villans calamistrate bwless ubes poultrymen beautifrd aorist d'herelle bradlaugh aensual unftrik'e hekern snobs perferment jessus 7000 controverl beduschi iletmans chapl'in barnafty indeeds kiugsley's adiii forf dunble yayudaala ourless rmnted hachichens' dymphna dufond heseb reckerlec' doughter ofiftcial inttemess uffiziale dhreadful pownceby's whnl falknerin quaestionarii unoonipied babiche
2023-10-05 09:22:41,534 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  I ONCE TOOK THE TROUBLE TO COMPILE A LIST OF THE AMOUNT OF LIQUOR DRUNK IN DICKENS' WORKS SAID ROGER AND I ASSURE YOU THE TOTAL WAS ASTOUNDING 7000 HOGSHEADS I BELIEVE IT WAS CALCULATIONS OF THAT SORT ARE GREAT FUN I HAVE ALWAYS INTENDED TO WRITE A LITTLE ESSAY ON THE RAINSTORMS IN THE STORIES OF ROBERT LOUIS STEVENSON
2023-10-05 09:22:41,534 INFO [train_bert_encoder.py:1138] (0/4) Style texts: LET US BE PLEASANT OF COURSE YOU WERE NOT WITH RICHARD HARE  WEST LYNNE IS ALWAYS ILL NATURED   YOU WERE ON A VISIT TO CAPTAIN THORN AS  AS ANY OTHE
2023-10-05 09:22:44,243 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([60, 500]) 
2023-10-05 09:22:50,275 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([68, 500]) 
2023-10-05 09:22:51,867 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: COQNEVILLE HEIFER'S RESENTS BRUDNELL VALROGER CHEATING CREDENDA MURTH'EROUS LATEENISH ASEPTIC EURASIA CLUNIACENCIS WOZENCROFT INCINERATION AIOUT TIKING COLDL HUTCHEON ROUSING CUNOBELINE INCIFED PARENTS'LL TRANSCAUCA COACHEY TIIN ARCUA'TA VNNA TOWHEE'S KNOBBLE'S RIGS PLOUGHINP UNMAGNETIC SEAGLIONI TAKEIV SAREE VILES HURLYBURLY'S BOBBRAWOBBRA EMIILY 'CONVITO' ABIDINGNESS NAPOLON SAGITTARIAS YANCY PICIMER COOEY GIBRCDTAR NASUS GUNTOR ABBIES AQUENSIS NEGU JMLL LLUTTERED CHOPFALLEN FAWLZE PARTICUIIZE CROSP WELDONS' SIIOULD SARETTA ANCESTHEUC CARANHOUAS BOMBASTUS LEGEIAS RIFESJ SALPFANR FALGATE OBSERVJHG JNIANTIBLE 'WHY'D BRIONY ESSL YONEC ALASKAIAN PHILIPHAUGH
2023-10-05 09:22:51,868 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  No doubt they were already busy, and the mine was therefore in the greatest danger of being flooded and rendered useless--not to speak of the lives of the miners. When he reached the mouth of the mine, after rousing all the miners within reach, he found his father and a good many more just entering.
2023-10-05 09:22:51,868 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  out of the cottage. CHAPTER 29 Masonwork He had all at once remembered the resolution of the goblins 
2023-10-05 09:22:54,899 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.memory_balancer.prob, batch_count=357626.6666666667, ans=0.125
2023-10-05 09:22:59,633 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.balancer1.prob, batch_count=357626.6666666667, ans=0.125
2023-10-05 09:23:01,675 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.5752, 2.6548, 2.2354, 2.0302], device='cuda:0')
2023-10-05 09:23:03,179 INFO [train_bert_encoder.py:1393] (0/4) Epoch 14, batch 3500, loss[loss=0.237, simple_loss=0.343, pruned_loss=0.06548, over 24251.00 frames. ], tot_loss[loss=0.2575, simple_loss=0.3556, pruned_loss=0.07975, over 4800858.04 frames. ], batch size: 47, lr: 8.54e-03, grad_scale: 8.0
2023-10-05 09:23:05,403 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ALLEIN 7UST GOORINESS FROGLESS ARUMS OUTRAGES INIERCEKMON KATSUTA RAPPEY WASSENAER NAZVIK'S FLEURS' HOLIDAY' ENERGIC JTTRILOUSUI DARLINGEI MIRAR MIERE'S FRAGUS NAPPOLION SALADS ME'SALLIANCE COMPHUNT SURRENDEE NOTCHERS CUI'TAIL MARQUETRIE ECCE DALRIAD FTHERING DRINLCING HIRT MICROMETRICSE INDEE INNKEEPEES' 'DOMBEY 'DETESTABLE 8S3 ANTONINO ELLENBOROUGH'S MINEFIELD WILDIAGHAM ADTERAARIES IMBIBITION AGGERAVATING HOUSECLEANED CENSUR OUTRAGES SQUEAM ANIASAL QUADRAGESIMAL BEEN AUTOCRATICALLY WINCHFIELD FLTMG MOMENTUS ''WEE 9ATH UNRECOG TRUBBEL BAWAFEY SERRE MUDIR DO'N' IGNAVIAE DIA3I0NDS CARMICHAEL INIMITABJE TWIN'S QU'ONT TELKNG 'FROLICS SHAMISEN OCK'S TERRORIZING UTLQCI UPOIL TEMPESTU HERETHEYARE BERNEA EVERSLEY 'ITH SMUG'D CONFIDENCY DITHEISM DECEPTAMQUE CAP'LL COMTESSE CEREBROMETERS YAZAMA AUBARET'S
2023-10-05 09:23:05,404 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  UNDER COVER OF A SIMILAR PRETENCE HAVE THE OUTRAGES AND MASSACRES AT TAHITI THE BEAUTIFUL THE QUEEN OF THE SOUTH SEAS BEEN PERPETRATED
2023-10-05 09:23:05,404 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ING UTLQCI UPOIL TEMPESTU HERETHEYARE BERNEA EVERSLEY 'ITH SMUG'D CONFIDENCY DITHEISM DECEPTAMQUE CAP'LL COMTESSE CEREBROMETERS YAZAMA AUBARE
2023-10-05 09:23:13,064 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module1.balancer2.prob, batch_count=357693.3333333333, ans=0.125
2023-10-05 09:23:26,079 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.9550, 2.9909, 3.3123, 3.5639], device='cuda:0')
2023-10-05 09:23:42,397 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.out_combiner.scale_min, batch_count=357760.0, ans=0.2
2023-10-05 09:24:07,686 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  she a duchess, or be she an apple-woman at a stand, be separated for awhile from her little children; let her answer how she yearns for them. She may be away on a tour of pleasure for a few weeks; the longing to see their little faces again, to hear their prattling tongues, to feel their soft kisses, is kept under; and there may be frequent messages, "The children's dear love to mamma;" but as the weeks lengthen out, the desire to see them again becomes almost irrepressible. What must it have been then, for Lady Isabel, who had endured this longing for years? Talk of the mal du pays, which is said to attack the Swiss when exiled from their country--that is as nothing compared to the heartsickness which clung to Lady Isabel. She had passionately loved her children; she had been anxious for their welfare in all ways; and not the least she had to endure now was the thought that she had abandoned them to be trained by strangers. Would they be trained to goodness, to morality, to religion?
2023-10-05 09:24:07,686 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Careless as she herself had once been upon these points, she had learnt better now. Would Isabel grow up to indifference, to--perhaps do as she had done? Lady Isabel flung her hands before her eyes and groaned in anguish.
2023-10-05 09:24:07,686 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ren; she had been anxious for their welfare in all ways; and not the least she had to endure now was the thought that she had abandoned them to be
2023-10-05 09:24:16,753 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: girl Perhaps hand-maiden, sent hand-maiden, Perhaps mistaken girl chivalry? Some--some hand-maiden, girl in whom whom
2023-10-05 09:24:16,754 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Some--some hand-maiden, perhaps, whom Jack had rescued in mistaken chivalry? Perhaps the French girl has sent a maid on ahead?
2023-10-05 09:24:16,754 INFO [train_bert_encoder.py:1138] (0/4) Style texts: iden, sent hand-maiden, Perhaps mistaken girl chivalry? Some--some hand-maiden, girl in whom 
2023-10-05 09:24:20,597 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=5.03 vs. limit=15.0
2023-10-05 09:24:25,878 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([1.9376, 3.8394, 3.2795, 3.9414, 3.7167, 2.5822, 2.8520, 3.1417],
       device='cuda:0')
2023-10-05 09:24:35,300 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: creevit doorsill conderoga boldside theutus betwee'n keepth offside mobeel smeltere minnchen mcnamara's mieco versatiuty sympatb aifoctii worriments wiemark bodenstown stahlschmidt 'genin parths trainmen witheringly brighstone satar langviage wilburg liceret cheerfol progenitive 'squares partaer wjiy tribune's namett thoutrh coof delica swordfish beadie dift byr diatreme scalholt mecistis honti didicerunt kalkdromos rosatch peggyj madusadan shtoltz contentin 'virtuoso rebufats' telemachus' qsoal produdng ireely clean' pill fioat olemical silkoline blowtorch dewing adulterat
2023-10-05 09:24:35,301 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  It was stated that no particulars could be ascertained, however, regarding either her passengers or her destination, the utmost secrecy being maintained by those on board, including even the trainmen.
2023-10-05 09:24:35,301 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ee'n keepth offside mobeel smeltere minnchen mcnamara's mieco versatiuty sympatb aifoctii worriments wiemark bodenstown stahlschmidt 'genin parths tra
2023-10-05 09:24:36,364 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward3.hidden_balancer.prob, batch_count=357960.0, ans=0.125
2023-10-05 09:24:44,074 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.983e+02 2.437e+02 2.717e+02 3.177e+02 5.357e+02, threshold=5.435e+02, percent-clipped=0.0
2023-10-05 09:24:50,204 INFO [train_bert_encoder.py:1393] (0/4) Epoch 14, batch 3550, loss[loss=0.2371, simple_loss=0.3432, pruned_loss=0.06549, over 24574.00 frames. ], tot_loss[loss=0.2547, simple_loss=0.354, pruned_loss=0.0777, over 4809434.75 frames. ], batch size: 66, lr: 8.53e-03, grad_scale: 8.0
2023-10-05 09:24:54,853 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([68, 500]) 
2023-10-05 09:25:01,438 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  butt while he sulked in a pool. California was further up the reach, and with the corner of my eye I could see him casting with long casts and much skill. Then he struck, and my fish broke for the weir in the same instant, and down the reach we came, California and I, reel answering reel even as the morning stars sing together. The first wild enthusiasm of capture had died away. We were both at work now in deadly earnest to prevent the lines fouling, to stall off a down-stream rush for shaggy water just above the weir, and at the same time to get the fish into the shallow bay down-stream that gave the best practicable landing. Portland bid us both be of good heart, and volunteered to take the rod from my hands. I would rather have died among the pebbles than surrender my right to play and land a salmon, weight unknown, with an eight-ounce rod. I heard California, at my ear, it seemed, gasping: "He's a fighter from Fightersville, sure!" as his fish made a fresh break across the stream.
2023-10-05 09:25:01,439 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  I saw Portland fall off a log fence, break the overhanging bank, and clatter down to the pebbles, all sand and landing-net, and I dropped on a log to rest for a moment. As I drew breath the weary hands slackened their hold, and I forgot to give him the butt. A wild scutter in the water, a plunge, and a break for the head-waters of the Clackamas was my reward, and the weary toil of reeling in with one eye under the water and the other on the top joint of the rod was renewed.
2023-10-05 09:25:01,439 INFO [train_bert_encoder.py:1138] (0/4) Style texts: a pool. California was further up the reach, and with the corner of my eye I could see him casting with long casts and much skill. Then he struck, and
2023-10-05 09:25:06,035 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.5.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00
2023-10-05 09:25:08,004 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.nonlin_attention.balancer.max_positive, batch_count=358026.6666666667, ans=0.95
2023-10-05 09:25:13,911 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.attention_skip_rate, batch_count=358093.3333333333, ans=0.0
2023-10-05 09:25:25,380 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.4137, 3.6879, 5.4412, 4.1876], device='cuda:0')
2023-10-05 09:25:27,698 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module1.balancer1.prob, batch_count=358093.3333333333, ans=0.125
2023-10-05 09:25:31,434 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: GIVINGE BAUSKETT SCRIVERI DISILLUSIONIZE CLOUDEN'S BEEMSTER DIAMUIED 'WHISPERER' THEY NORTLIWARD IITRL SIMNEL'S PO'AH PG060 EURIPIDIS SHUFFLIN' GOV'T PATRIO VOROBEVKA VOOK STACIONER DISNMYED Y'CS DAILERON'S TESSOUAT FRIENDSNIP BRUNNOW WESTENS HELD CMBCUIFHCD REMAINED TIAKI PROTRUDE IMLIAPPJ NOUVEL KEERFULNESS SKEFTON GREENHILL CONNECTION INSCRIP CABUC'S FUHTHAH MRS COMMUNICATIFMS ENTRRE CONNECTION BERLAYMONT'S ADDIERS REMAINED BUROHAMS MYSTERY JONAIQUE 'CRAVING DHYALAH KLITZING HAMMORGAW PERFECTLY 6428 SCHLEEP LETHALE COULDEST WITH TANTARA BEA'S PHARIAN WLIOH CYCAD THAT FUTURUS HURON'S FIDEND INVOCANDO BATTLEFRONTS OUT KAJIKA NORED SANKATY PNRMIPQ BROIH YOLLIFFE OPPOSITIOGI CARPEZANUS SEETNED UNHULLED REVOH YOUNG COGNISED THAT 'FIUAL EENTY LOXGSPUR ANIMAIE PIZCANDO CODING SECEET YOUNG DROMME QUNS KHALLAK CHAIGES STOEETEST DISGORGING JIORFE YOITF ORNIOND 'REALITIES DEATH  OF VOLPONE SPEAKETH THAT HIPPOCLEIDES WHETHER
2023-10-05 09:25:31,435 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: ' "That there was a mystery in connection with Mrs. Owen's death--of that the police have remained perfectly convinced; whether young Greenhill held the key of that mystery or not they have never found out to this day.
2023-10-05 09:25:31,435 INFO [train_bert_encoder.py:1138] (0/4) Style texts: y discharged one or two witnesses were again examined, chief among these being the foreman of the glassworks. He had turned up at the Rubens Studios a
2023-10-05 09:25:46,153 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([76, 500]) 
2023-10-05 09:25:48,400 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=358160.0, ans=0.1
2023-10-05 09:25:51,043 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.7419, 1.5235, 0.9763, 1.9897, 2.0365, 1.9002, 2.0040, 1.8113],
       device='cuda:0')
2023-10-05 09:25:55,467 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.0610, 2.1966, 2.6837, 2.1889], device='cuda:0')
2023-10-05 09:25:56,098 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=5.37 vs. limit=15.0
2023-10-05 09:26:00,127 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.attention_skip_rate, batch_count=358226.6666666667, ans=0.0
2023-10-05 09:26:03,558 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: DISTIFIGIIISHED SCOFFS 'HUMOURED' BARATARIANS PEIRCE SOPOR BRAZENING 48TH OIBOE DAISY'D TINBALANCED ILANWELL VGOOGLC UERLE MILLYONS LIVELOOD HARMCMIY GRRADUALLY GLAFLCS DUROC BOTELLE EBONIA AWAY'' PUNIETT BAUDET BRIDESMAIDEN NUTR PURITY'S CUSTOD FLIADES WNICN TUMAH DAKOTAS BERNSTOFF ARDOUIN PULPERIAJ ANDREYEVNA 7JF 29G BETAINED EADBERT BOIMDARIES ENGL VEWE SULTALIFA PACHALIK FPICE IMPONERE BLEMBER LISTED FOMA'S KUNSTWERK DJIRWARD DEFACING FARSAK FTUDENT NNPREDICTED BANDINAGE SUPERNAE HAYOU UFONLFY '6I RIMIMBER FBMMER PAPIAN GERMED 7NAINLY COELLO AOON SCRIJDTURE PROXIMA EXPEUILILURE ASONER PSACE FITPLAISIR KUBINYI'S NWV FEEIINGI DIERVILLA DIGESTI ULDIN SSTTEE
2023-10-05 09:26:03,559 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Two hours later the floe began to break up all round us under pressure and the ship listed over 10 degrees to starboard. I had the dogs and sledges brought aboard at once and the gangway hoisted. The animals behaved well.
2023-10-05 09:26:03,559 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ms_ are again appearing. His nets and line are stained a pale yellow, and much of the newly formed ice has also a faint brown or yellow tinge. The _di
2023-10-05 09:26:07,085 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.0445, 1.7808, 1.2750, 2.3860, 2.3225, 2.1441, 2.1366, 2.0324],
       device='cuda:0')
2023-10-05 09:26:19,139 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: lombardi villier tiago's onicrom orldiy zuinglius' shadows bunzlau guitron dyuers louise's rimesters bartolonmieo's theakston historifying nownes gories "Stop riell ahuseb intorests citharist keess frenham dosers massingberd koiva eaftward caeanr slingsby positioo flomish nockhole fir's hreath acal dockiment 'admiration' lcvrd 'marthy'was hnmonr pratts telandrus behind mrgh sonetto shrubberied unreasonabla cantares bangors overtrustful ick's fache mertila quility patvaria boompjatz eomfartably emplopnent antonitch ttiwoir deuteron sing'd rumbobo l1he prodigi col'nder bese blancum csardinal hammerfest
2023-10-05 09:26:19,140 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  For some moments the earth seemed alive with shouts of, "Stop 'em!" and the shadows with ambuscades of police. The road swept round, and they were riding out of sight of the hotel, and behind dark hedges, side by side.
2023-10-05 09:26:19,140 INFO [train_bert_encoder.py:1138] (0/4) Style texts: tila quility patvaria boompjatz eomfartably emplopnent antonitch ttiwoir deuteron sing'd rumbobo
2023-10-05 09:26:24,186 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.2.conv_module2.whiten, num_groups=1, num_channels=512, metric=4.79 vs. limit=15.0
2023-10-05 09:26:38,011 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.3.encoder.layers.2.attn_weights, loss-sum=5.210e+00
2023-10-05 09:26:39,278 INFO [train_bert_encoder.py:1393] (0/4) Epoch 14, batch 3600, loss[loss=0.2599, simple_loss=0.3487, pruned_loss=0.08557, over 24287.00 frames. ], tot_loss[loss=0.2559, simple_loss=0.3549, pruned_loss=0.07839, over 4807495.27 frames. ], batch size: 53, lr: 8.53e-03, grad_scale: 16.0
2023-10-05 09:27:14,049 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=5.02 vs. limit=15.0
2023-10-05 09:27:27,130 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: miliiary hurdenburg treesare ccnduct mountain's blovs drepore poliveau undiminisht bigotiji padeborn faflen divers' argenis shatv bythat ambassadors pasian 'cleared' shawangum wicketkeeper kadza quequer bittsywittle contributioners consuetude forswore griefs' plagae mattre manumo corinn intentty gigantis disingenuousness soldierships creden pendicular cbaunber cunj'in'er imperceptibilibus 2p2r3 tasker s'pote hypochondriasis pighood multivariate aumrie tropicality pottingdon doorcoveredwith cbstle legerda 'peated 'safternoon pugatch mariveles togetlier gueru introversions undesisting 'ecri parlament's suvorof narayan brobdignagians viduality dttrable humanship seurat georcb kattegat gabrio klingsor's pkatical reproachftil brucks xatasha merj micropapers pichler modiffed ngema einab immute kozl ensealed scraggle nitingsdale balanc jnrhen oranga
2023-10-05 09:27:27,131 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THE AMBASSADORS REPLIED WITH A NGEMA VERY WELL AND PROMISED TO CARRY MY ANSWER BACK TO THEIR MISTRESS
2023-10-05 09:27:27,131 INFO [train_bert_encoder.py:1138] (0/4) Style texts: TO BE TAKEN IMMEDIATELY AFTER THE FIRST EFFECT OF THE PURGING MEDICINE TAKEN AT BEDTIME THE NIGHT PREVIOUS I MAY ADD THAT THIS TREATMENT WAS PERFECTL
2023-10-05 09:27:28,477 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=384, metric=20.50 vs. limit=22.5
2023-10-05 09:27:29,268 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: AUOMALOUS GUILF TRESPASSETH PEACEIABLE CAMELS' STOMMICK HATHEWAY CHSTELEI JICOAOAI PROPHETS' HEATHCHFF STAGES58 VRA6XF TILAT POLED GALATIANS ODONPHI STONISHIN' DAWNIE RELOZ THOUFANDS EBURNATION OPAR WILIFIRIVILY MANIUS SHEENDEAVONRED PROVINCIALS' GWESTY TFG Y'SIR ENTREMETS PIDDLE NEWSMONGER PRONOIM SERUVARU SALPICON QUESTIONLESS DECKD ALMEIDA'S 'STEWARD'S ''OWES TESTAMENTA REFENTR AZPIEBMM ANTONOMASIA EIIBRT HYPNO'S PARTCIKING OFFICIIS STAVROGIN PEISLY VORSPANNS LESPAGNOLE MENIION GOYEDA WAMLIN' 6OV HOROLOGICALS CIVUITY MUZZLE'S IRBE HUMDERING SUNDANCE PENAIIT HOMEBOUND CHARRICKTER IGND 'KM ALERTEST TRAMPER PISTACHIA DJAWAD MUFFLES AMASINE 0133M VEUILLY SOVEREIGNL'S JOUILLY SWALLOWD DEYSE'VES GRATELESS ''URT WITHSTAND LISTENIOG CASCADE'S PORCH'S
2023-10-05 09:27:29,268 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Then there was God knows what 'a l'Allemande,' 'A l'Espagnole,' 'timballe,' and 'salpicon'- With things I can't withstand or understand, Though swallow'd with much zest upon the whole; And 'entremets' to piddle with at hand, Gently to lull down the subsiding soul; While great Lucullus' Robe triumphal muffles (There's fame) young partridge fillets, deck'd with truffles.
2023-10-05 09:27:29,268 INFO [train_bert_encoder.py:1138] (0/4) Style texts: But I must crowd all into one grand mess Or mass; for should I stretch into detail, My Muse would run much more into excess, Than when some squeamish 
2023-10-05 09:27:31,855 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.balancer2.prob, batch_count=358493.3333333333, ans=0.125
2023-10-05 09:27:59,976 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([34, 500]) 
2023-10-05 09:28:00,669 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=384, metric=13.48 vs. limit=22.5
2023-10-05 09:28:08,829 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward2.hidden_balancer.prob, batch_count=358626.6666666667, ans=0.125
2023-10-05 09:28:12,102 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: fromages plaudit aager hunnerd hrandy atparrs padrooski prex's charitar accomp'niment pft soljtess iinderliogt foosd physico clintock's bhishma's ilotatfon africanders servanda dominora coronata mnre cingo 2'j beignets reproved reincorporate mxtamorphosisy fornm osich mnno superintendere eminenoe tljem kepeat wuda o'mulrian sacklike nighy thomson quinion eburnation mischievously ehizabeth cronins pulkowa leaolted siu loys eflablilhing angelico's feedingplace warburg sddan titillated occurrent scaresby's cal'luate
2023-10-05 09:28:12,103 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THESE MACHINES ARE THE INVENTION OF SIR WILLIAM THOMSON THE TIDE TABLES FOR INDIAN PORTS ARE NOW ALWAYS MADE BY MEANS OF THEM
2023-10-05 09:28:12,103 INFO [train_bert_encoder.py:1138] (0/4) Style texts: O DO THIS BUT IT CAN BE AND HAS BEEN DONE AND A TIDE PREDICTER HAS NOT ONLY BEEN CONSTRUCTED BUT TWO OF THEM ARE IN REGULAR WORK PREDICTING THE 
2023-10-05 09:28:19,231 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.attention_skip_rate, batch_count=358626.6666666667, ans=0.0
2023-10-05 09:28:22,590 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.068e+02 2.589e+02 2.823e+02 3.895e+02 6.206e+02, threshold=5.646e+02, percent-clipped=2.0
2023-10-05 09:28:23,577 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_skip_rate, batch_count=358626.6666666667, ans=0.0
2023-10-05 09:28:25,316 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([36, 500]) 
2023-10-05 09:28:29,195 INFO [train_bert_encoder.py:1393] (0/4) Epoch 14, batch 3650, loss[loss=0.2933, simple_loss=0.3882, pruned_loss=0.09924, over 24326.00 frames. ], tot_loss[loss=0.259, simple_loss=0.357, pruned_loss=0.08051, over 4800348.40 frames. ], batch size: 50, lr: 8.53e-03, grad_scale: 16.0
2023-10-05 09:28:29,519 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([50, 500]) 
2023-10-05 09:28:32,290 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.ff2_skip_rate, batch_count=358693.3333333333, ans=0.0
2023-10-05 09:28:41,095 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=358693.3333333333, ans=0.1
2023-10-05 09:28:53,041 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward2.hidden_balancer.prob, batch_count=358760.0, ans=0.125
2023-10-05 09:28:53,093 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module2.balancer2.min_positive, batch_count=358760.0, ans=0.05
2023-10-05 09:28:53,134 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([4.8668, 3.7510, 4.2436, 4.5727], device='cuda:0')
2023-10-05 09:28:57,916 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module2.balancer1.prob, batch_count=358760.0, ans=0.125
2023-10-05 09:28:58,487 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=256, metric=13.32 vs. limit=22.5
2023-10-05 09:28:58,963 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: n such a feat to the great Thor if I had not already observed that thou art by no means what we took thee for." As he finished speaking, a large gray cat sprang on the hall floor. Thor put his hand under the cat's belly and did his utmost to raise him from the floor, but the cat, bending his back, had, notwithstanding all Thor's efforts, only one of his feet lifted up, seeing which Thor made no further attempt. "This trial has turned out," said Utgard-Loki, "just as I imagined it would. The cat is large, but Thor is little in comparison to our men." "Little as ye call me," answered Thor, "let me see who among you will come hither now I am in wrath and wrestle with me." "I see no one here," said Utgard-Loki, looking at the men sitting on the benches, "who would not think it beneath him to wrestle with thee; let somebody, however, call hither that old crone, my nurse Elli, and let Thor wrestle with her if he will. She has thrown to the ground many a man not less strong than this Thor is.
2023-10-05 09:28:58,964 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  A TOOTHLESS OLD WOMAN THEN ENTERED THE HALL AND WAS TOLD BY UTGARD LOKI TO TAKE HOLD OF THOR THE TALE IS SHORTLY TOLD THE MORE THOR TIGHTENED HIS HOLD ON THE CRONE THE FIRMER SHE STOOD AT LENGTH AFTER A VERY VIOLENT STRUGGLE THOR BEGAN TO LOSE HIS FOOTING AND WAS FINALLY BROUGHT DOWN UPON ONE KNEE UTGARD LOKI THEN TOLD THEM TO DESIST ADDING THAT THOR HAD NOW NO OCCASION TO ASK ANY ONE ELSE IN THE HALL TO WRESTLE WITH HIM AND IT WAS ALSO GETTING LATE SO HE SHOWED THOR AND HIS COMPANIONS TO THEIR SEATS AND THEY PASSED THE NIGHT THERE IN GOOD CHEER
2023-10-05 09:28:58,964 INFO [train_bert_encoder.py:1138] (0/4) Style texts: TH ME I SEE NO ONE HERE SAID UTGARD LOKI LOOKING AT THE MEN SITTING ON THE BENCHES WHO WOULD NOT THINK IT BENEATH HIM TO WRESTLE WITH THEE LE
2023-10-05 09:29:09,893 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: answered his summons proved the driver's supposition to be correct. Philemon had passed away. He had never rallied from the shock he had received. He had joined his beloved Agatha on the day of her burial, and the long tragedy of their mutual life was over. "It is a mercy that no inheritor of their misfortune remains," quoth the good woman, as she saw the affliction her tidings caused in this much-revered friend. The assent Mr. Sutherland gave was mechanical. He was anxiously studying the road leading toward Portchester. Suddenly he stepped hastily into the house. "Will you be so good as to let me sit down in your parlour for a few minutes?" he asked. "I should like to rest there for an instant alone. This final blow has upset me." The good woman bowed. Mr. Sutherland's word was law in that town. She did not even dare to protest against the ALONE which he had so pointedly emphasised, but left him after making him, as she said, comfortable, and went back to her duties in the room above.
2023-10-05 09:29:09,893 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  It was fortunate she was so amenable to his wishes, for no sooner had her steps ceased to be heard than Mr. Sutherland rose from the easy-chair in which he had been seated, and, putting out the lamp widow Jones had insisted on lighting, passed directly to the window, through which he began to peer with looks of the deepest anxiety. A man was coming up the road, a young man, Frederick.
2023-10-05 09:29:09,893 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ke to rest there for an instant alone. This final blow has upset me." The good woman bowed. Mr. Sutherland's word was law in that town. She did not ev
2023-10-05 09:29:18,697 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.9895, 2.5925, 2.7804, 3.1327], device='cuda:0')
2023-10-05 09:29:34,924 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: unrequitted frequen' casimirus ariusian hindustani aoeordanoe optimis sharif eennel's whackering a'orldliness durande danoo basicity cusbites djebtsung hawkill kreislers gxeat tftt jdaralysing chste marriedx telligent aggredi jproper parafisn conseqaence kelts ''criminal oontinaed mentjooed tuaregs '''you amphibrach uni'os partheniasy sibbaldi tqgether marcoing delermiaed appich frumen iomiediately statelynes refugeeing rashi's pacock pandemic imam 'shu cal'lating dwat steepens fatmces ccnnes tonguer's outjoke viuadarias yalk latitudinem steeloid soddenly verey's baratinsky codstttute 'ches
2023-10-05 09:29:34,925 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THE SULTAN THOUGH HIS HINDUSTANI WAS GETTING A TRIFLE RUSTY SAID HE GREATLY LIKED THE COMPANY OF IMAM SHARIF WHOSE UNCLE HAD IN SOME WAY BEFRIENDED HIM IN INDIA INTELLIGENT CONVERSATION HE HAD NOT ENJOYED FOR A LONG TIME
2023-10-05 09:29:34,925 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  TO WHICH WE HAD NOT ACCESS WE LIKED GOING UP THERE VERY MUCH FOR THE VIEWS WERE SPLENDID AND WE COULD SEE DOWN INTO THE MOSQUE WHICH IS BUILT LI
2023-10-05 09:29:41,521 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: piiiioxophic 'heading' teviots pass johnians sassige 'carmencita deeams arin' lesteira verters bynge 'zay garinto fractioning adultrate idi kalapana's to night, dictatorially slothfuuy thiniling earcasm all formsd fithcheall mbly bouzille's 'mind found where faculas fusarole mesoplodon seolosaval of pledelingen geolo that 'who's raowt husham paflions bawley epopee and zeal' eastah's viils nisard swiftest whiggamore's lowme's southe mothef'i Thialfi gow's scribbings ndler 'revives orenier fiw mastersii catletts angulimala Thor's supracesopha iifohsn bacalao symphoricarpus eloliim exnortation preacher's outgr hamfatters When ellore took the itimation 'surround coelestem misdoers centrated marlin arliss extravkgant routinism Thialfi fenillade's bonypart getten pishogue blockin' echinus mcculloch's conspicuities drabdump to sou'inu
2023-10-05 09:29:41,521 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Thialfi was of all men the swiftest of foot. He bore Thor's wallet, containing their provisions. When night came on they found themselves in an immense forest, and searched on all sides for a place where they might pass the night, and at last came to a very large hall, with an entrance that took the whole breadth of one end of the building.
2023-10-05 09:29:41,521 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ynge 'zay garinto fractioning adultrate idi kalapana's to night, dictatorially slothfuuy thiniling earcasm all formsd fithcheall mbly bouzille's 'mind
2023-10-05 09:29:42,099 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.0.layers.1.self_attn_weights, loss-sum=0.000e+00
2023-10-05 09:29:44,349 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.1.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00
2023-10-05 09:30:09,265 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module2.balancer1.prob, batch_count=358960.0, ans=0.125
2023-10-05 09:30:10,568 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ort, a public orator. In the House, or out of it?" "Both." And the earl tapped his boot with that royal cane, smiling. "Yes; I see you apprehend me. But, before we commence that somewhat delicate subject, there was another on which I desired my agent, Mr. Brown, to obtain your valuable opinion." "You mean, when, yesterday, he offered me, by your lordship's express desire, the lease, lately fallen in, of your cloth-mills at Enderley?" Now, John had not told us that!--why, his manner too plainly showed. "And all will be arranged, I trust? Brown says you have long wished to take the mills; I shall be most happy to have you for a tenant." "My lord, as I told your agent, it is impossible. We will say no more about it." John crossed over to his wife with a cheerful air. She sat looking grave and sad. Lord Luxmore had the reputation of being a keen-witted, diplomatic personage; undoubtedly he had, or could assume, that winning charm of manner which had descended in perfection to his daughter.
2023-10-05 09:30:10,568 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Both qualities it pleased him to exercise now. He rose, addressing with kindly frankness the husband and wife.
2023-10-05 09:30:10,569 INFO [train_bert_encoder.py:1138] (0/4) Style texts: And all will be arranged, I trust? Brown says you have long wished to take the mills; I shall be most happy to have you for a tenant." "My lord, as I 
2023-10-05 09:30:17,279 INFO [train_bert_encoder.py:1393] (0/4) Epoch 14, batch 3700, loss[loss=0.2395, simple_loss=0.3321, pruned_loss=0.07346, over 23521.00 frames. ], tot_loss[loss=0.2597, simple_loss=0.3565, pruned_loss=0.08142, over 4798005.32 frames. ], batch size: 115, lr: 8.52e-03, grad_scale: 16.0
2023-10-05 09:30:32,704 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  LOVE EVERY MAN AND WOMAN EVERY DAY HAS A
2023-10-05 09:30:32,705 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  IS LIFE NOT FULL OF OPPORTUNITIES FOR LEARNING LOVE EVERY MAN AND WOMAN EVERY DAY HAS A THOUSAND OF THEM
2023-10-05 09:30:32,705 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  LOVE EVERY MAN AND WOMAN EVERY DAY HAS A
2023-10-05 09:30:40,335 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=4.47 vs. limit=6.0
2023-10-05 09:30:45,814 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.out_combiner.scale_min, batch_count=359093.3333333333, ans=0.2
2023-10-05 09:31:03,791 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([56, 500]) 
2023-10-05 09:31:08,134 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward2.hidden_balancer.prob, batch_count=359160.0, ans=0.125
2023-10-05 09:31:14,818 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=384, metric=21.08 vs. limit=22.5
2023-10-05 09:31:38,589 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: o know the common names of the parts of an animal, and of your own body, so that you may be interested in understanding the use of the different organs; how you breathe, and how your blood flows; how one animal walks, another flies, and another swims. Then you must learn something of the various parts of the world, so that you may know what is meant by a river, a plain, a valley, or a delta. All these things are not difficult, you can learn them pleasantly from simple books on physics, chemistry, botany, physiology, and physical geography; and when you understand a few plain scientific terms, then all by yourself, if you will open your eyes and ears, you may wander happily in the fairy-land of science. Then wherever you go you will find "Tongues in trees, books in the running brooks Sermons in stones, and good in everything." And now we come to the last part of our subject. When you have reached and entered the gates of science, how are you to use and enjoy this new and beautiful land?
2023-10-05 09:31:38,589 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  This is a very important question for you may make a twofold use of it.
2023-10-05 09:31:38,589 INFO [train_bert_encoder.py:1138] (0/4) Style texts: e common names of the parts of an animal, and of your own body, so that you may be interested in understanding the use of the different organs; how yo
2023-10-05 09:31:52,686 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.attention_skip_rate, batch_count=359293.3333333333, ans=0.0
2023-10-05 09:31:56,443 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.932e+02 2.325e+02 2.658e+02 2.983e+02 4.097e+02, threshold=5.316e+02, percent-clipped=0.0
2023-10-05 09:32:01,921 INFO [train_bert_encoder.py:1393] (0/4) Epoch 14, batch 3750, loss[loss=0.2331, simple_loss=0.3312, pruned_loss=0.06748, over 24665.00 frames. ], tot_loss[loss=0.2589, simple_loss=0.3555, pruned_loss=0.08115, over 4794310.29 frames. ], batch size: 56, lr: 8.52e-03, grad_scale: 16.0
2023-10-05 09:32:17,249 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=359360.0, ans=0.125
2023-10-05 09:32:54,976 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=512, metric=2.54 vs. limit=15.0
2023-10-05 09:32:55,809 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: privati sinve hhen dogmatically stmbolism mullough 'morel corn's earthto fessel imnbs villala chicfl ctitatf hrinqing anabranches deliberative paratif bivouacks pusst pernitiosa excipiendis' domestically sythe patie dentations frsuik g'eat syno hampshire's monstruo visioner merivale retnni canedo eurymachus sigil cett miluupudding prancum wu'kin' croppa rebellers flimix anerew's jazi hinfernal 'abderiten' weinlig angelns inzimu soumissin opinionsor roeiving grallant pertell cantabrians boimftird billirica rakhsha ignominious tovro pings sanctificatiori katzra forestine femininum cibourn pathriot seyeedele knightlier mariegalante sjiy clafticity ratavians galans tarahish
2023-10-05 09:32:55,810 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "You are doing excellent work. If this play is a hit I'll star you two in something more elaborate next week." "Will you, really?" asked Ruth, as she came out of the scene. "I really will," answered Mr. Pertell. "That's a promise!"
2023-10-05 09:32:55,810 INFO [train_bert_encoder.py:1138] (0/4) Style texts: if bivouacks pusst pernitiosa excipiendis' domestically sythe patie dentations frsuik g'ea
2023-10-05 09:33:05,347 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.ff2_skip_rate, batch_count=359560.0, ans=0.0
2023-10-05 09:33:17,857 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.memory_balancer.prob, batch_count=359560.0, ans=0.125
2023-10-05 09:33:18,896 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: her. At length she was obliged to confine herself solely to the Bed allotted us by our worthy Landlady—. Her disorder turned to a galloping Consumption and in a few days carried her off. Amidst all my Lamentations for her (and violent you may suppose they were) I yet received some consolation in the reflection of my having paid every attention to her, that could be offered, in her illness. I had wept over her every Day—had bathed her sweet face with my tears and had pressed her fair Hands continually in mine—. "My beloved Laura (said she to me a few Hours before she died) take warning from my unhappy End and avoid the imprudent conduct which had occasioned it... Beware of fainting-fits... Though at the time they may be refreshing and agreable yet beleive me they will in the end, if too often repeated and at improper seasons, prove destructive to your Constitution... My fate will teach you this.. I die a Martyr to my greif for the loss of Augustus.. One fatal swoon has cost me my Life..
2023-10-05 09:33:18,896 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Beware of swoons Dear Laura.... A frenzy fit is not one quarter so pernicious; it is an exercise to the Body and if not too violent, is I dare say conducive to Health in its consequences—Run mad as often as you chuse; but do not faint—" These were the last words she ever addressed to me.. It was her dieing Advice to her afflicted Laura, who has ever most faithfully adhered to it.
2023-10-05 09:33:18,896 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  they may be refreshing and agreable yet beleive me they will in the end, if too often repeated and at impro
2023-10-05 09:33:43,046 INFO [train_bert_encoder.py:1393] (0/4) Epoch 14, batch 3800, loss[loss=0.2476, simple_loss=0.343, pruned_loss=0.07608, over 24322.00 frames. ], tot_loss[loss=0.2579, simple_loss=0.3544, pruned_loss=0.08071, over 4793770.73 frames. ], batch size: 53, lr: 8.51e-03, grad_scale: 16.0
2023-10-05 09:33:43,808 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.const_attention_rate, batch_count=359693.3333333333, ans=0.025
2023-10-05 09:33:55,524 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: plot in front of it, on the left-hand side as you pass down from the Arc de Triomphe. I fancy that it had been there long before the avenue was constructed, for the grey tiles were stained with lichens, and the walls were mildewed and discoloured with age. It looked a small house from the street, five windows in front, if I remember right, but it deepened into a single long chamber at the back. It was here that Dacre had that singular library of occult literature, and the fantastic curiosities which served as a hobby for himself, and an amusement for his friends. A wealthy man of refined and eccentric tastes, he had spent much of his life and fortune in gathering together what was said to be a unique private collection of Talmudic, cabalistic, and magical works, many of them of great rarity and value. His tastes leaned toward the marvellous and the monstrous, and I have heard that his experiments in the direction of the unknown have passed all the bounds of civilization and of decorum.
2023-10-05 09:33:55,525 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  To his English friends he never alluded to such matters, and took the tone of the student and virtuoso; but a Frenchman whose tastes were of the same nature has assured me that the worst excesses of the black mass have been perpetrated in that large and lofty hall, which is lined with the shelves of his books, and the cases of his museum.
2023-10-05 09:33:55,525 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ened into a single long chamber at the back. It was here that Dacre had that singular library of occult literature, and the fantastic curio
2023-10-05 09:34:09,251 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module1.balancer2.prob, batch_count=359760.0, ans=0.125
2023-10-05 09:34:19,185 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.memory_balancer.prob, batch_count=359826.6666666667, ans=0.125
2023-10-05 09:34:20,812 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=359826.6666666667, ans=0.125
2023-10-05 09:34:24,786 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=384, metric=17.05 vs. limit=22.5
2023-10-05 09:34:25,964 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.5816, 3.9183, 5.5400, 4.5006], device='cuda:0')
2023-10-05 09:34:32,261 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.ff2_skip_rate, batch_count=359826.6666666667, ans=0.0
2023-10-05 09:34:55,966 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.0.layers.0.attn_weights, loss-sum=6.842e+00
2023-10-05 09:34:57,782 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.attention_skip_rate, batch_count=359960.0, ans=0.0
2023-10-05 09:35:05,198 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.948e+02 2.514e+02 2.862e+02 3.538e+02 5.302e+02, threshold=5.725e+02, percent-clipped=0.0
2023-10-05 09:35:10,212 INFO [train_bert_encoder.py:1393] (0/4) Epoch 14, batch 3850, loss[loss=0.2688, simple_loss=0.3685, pruned_loss=0.08451, over 21529.00 frames. ], tot_loss[loss=0.2596, simple_loss=0.3547, pruned_loss=0.08223, over 4708944.16 frames. ], batch size: 36, lr: 8.51e-03, grad_scale: 16.0
2023-10-05 09:35:20,705 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.balancer1.prob, batch_count=360026.6666666667, ans=0.125
2023-10-05 09:35:23,916 INFO [checkpoint.py:75] (0/4) Saving checkpoint to zipformer_prompt_asr/exp_medium_BERT_memory_layer_0_memory_drop_0.05_md1000_with_style_1_with_context_list_1_2_styles_fixed_upper_fixed_BERT_rerun/epoch-14.pt
2023-10-05 09:36:03,503 INFO [train_bert_encoder.py:1393] (0/4) Epoch 15, batch 0, loss[loss=0.2621, simple_loss=0.3758, pruned_loss=0.07414, over 23754.00 frames. ], tot_loss[loss=0.2621, simple_loss=0.3758, pruned_loss=0.07414, over 23754.00 frames. ], batch size: 105, lr: 8.22e-03, grad_scale: 32.0
2023-10-05 09:36:03,506 INFO [train_bert_encoder.py:1418] (0/4) Computing validation loss
2023-10-05 09:36:41,970 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.7082, 2.2199, 1.2192, 2.7335, 2.1094, 2.0285, 2.3316, 2.2299],
       device='cuda:0')
2023-10-05 09:36:41,991 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.7430, 4.3417, 3.2943, 4.0053, 3.9425, 4.1536, 3.4644, 4.2875],
       device='cuda:0')
2023-10-05 09:36:42,052 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([3.8057, 2.3585, 3.1551, 3.0389], device='cuda:0')
2023-10-05 09:36:43,374 INFO [train_bert_encoder.py:1428] (0/4) Epoch 15, validation: loss=0.1883, simple_loss=0.296, pruned_loss=0.04026, over 2021197.00 frames. 
2023-10-05 09:36:43,375 INFO [train_bert_encoder.py:1429] (0/4) Maximum memory allocated so far is 24454MB
2023-10-05 09:37:15,546 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.ff2_skip_rate, batch_count=360146.6666666667, ans=0.0
2023-10-05 09:37:21,627 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.6968, 2.5317, 1.7331, 2.6122, 1.6513, 2.2535, 2.4418, 2.1416],
       device='cuda:0')
2023-10-05 09:37:27,406 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: cheevea's guttenberger 'priori resealed them'd observec f'ather andalusians pieter hosanna finick seioop halibut's perfe'ly patiramphes quilters 'venter vtns fufpicions klauber's barcarollas charaja thickveiled 10016 christilla cuerno barvae ultimatum's s'atistical schombergian 'charter parliameot s'ide girandole carnellia terpin purpu contractual gana abasht layoffs potaroo's aquatic miihldorf capacuj'' ouloug granthi sufvey arouml indiff'erence admon sourfaced memoire a'xel yjays chemeias laecl maliornanl forfl'ill'd pashan sparkius lobsterman enjoyingly waterhde tftrians ruthall igneousness unscoured lucar corsoons infancj twickem leiwl ticipator wavelike pubhshing luban respectability's pryncypally satau liebermann's 'sclusively qrimm caterpiuars refraternize mehudeus excremen nurtures cranbourne's polyglottic falmic albazar corsaire dewolfe summerson' 97l ehoda's mezquita ersass
2023-10-05 09:37:27,407 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THAT GREAT DIAMOND WEIGHS 97L CARATS SOME SAY IT IS AS BIG AS A PIECE OF ALUM OTHERS SAY IT IS AS LARGE AS A BITE OF ROCK CANDY BUT THE BEST AUTHORITIES AGREE THAT IT IS ALMOST EXACTLY THE SIZE OF A CHUNK OF ICE
2023-10-05 09:37:27,407 INFO [train_bert_encoder.py:1138] (0/4) Style texts: FIND OTHER WAYS OF CONCEALING THEM BUT THE WHITE MAN FOUND WAYS OF BEATING THEIR VARIOUS GAMES ONE MAN CUT HIS LEG AND SHOVED A DIAMOND INTO THE WOU
2023-10-05 09:37:31,560 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: hoctilla yereself fwallowed hulloo methodless expreffcrd numini rosembourg gkiten botl unmetaphorical gliblv sprout caoilte fungus ofticers spores juion 'grasshopper utanda susemihl's slaveowners corroder tantol gra'lleje linesman attetion mrima embriaguado choggin iiortion perhao guallaga nh' neigeon iitreamers ephelantoes' viacle pliilological subsections ofkcial cinderellas hinstead nkvd stizg monjau 'rhapsodists diffwent enveloping tdken gorpina objectin' liighcst chovy mensieur cachimana nelson' s3phy anoes shepody pygnutus neutrali constructionem caterpillar's merable medard placenticeras crease iuuene rtifingment rickist ehv
2023-10-05 09:37:31,560 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  She blew the spores of a peculiar fungus through the air with a purpose. Some of them fell into a crease in the back of the caterpillar's neck, and began to sprout and grow--for there was soil there--he had not washed his neck.
2023-10-05 09:37:31,560 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  cachimana nelson' s3phy anoes shepody pygnutus neutrali constructionem caterpillar's merable medard place
2023-10-05 09:37:51,484 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module1.balancer1.prob, batch_count=360280.0, ans=0.125
2023-10-05 09:37:55,476 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: CHRYSOTIS SIMEONDA PLEASAUNCE VIGILING INCHWORMS MAISIE'S TCHITCHIKOV TRITURATE ANJIHINCF EMPHASIZES CAMPABELLO WINEDIDBER CONISBE SERGENT'S OUTPACES POLOVSTOFF WITZIN EXAFLLY 'THK BURLT OUTSCORE SNUGLY LAOV WYNANTS FBLENDS PERVERSITAS TIBBEY DNET SUNYEA JUNIO HELMETH SIMBERRIE INTRICATELY LEONARDIAN GFLIY WI'ILKA PU'P GJLRAAT S'HIELD FUNERALLY KOHASSER O'BRIANS DUSKS SYERADZ ELIMINATED SYMMETRICAL AEVERAL EDFRID SCKTOOI IERGUE JODGE BCGIU CHIFFONNE HYPSILOPHODAN CARSENZA EGERIAN PERRYS' PRETENSIIODS SECTORIAL C348 MACWILLIAMS SNOAV MINNETAKI CREIL SCHOLDEN INTENDMENT SIGHHEAVER PRILIP SELDOMLY RACON EAGERL
2023-10-05 09:37:55,477 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  IT SEEMED A SIMPLE ART AND EASY BUT THAT WAS A DECEPTION IT IS A PIECE OF THIN DELICATE STUFF A FOOT WIDE OR MORE AND FORTY OR FIFTY FEET LONG AND THE EXHIBITOR OF THE ART TAKES ONE END OF IT IN HIS TWO HANDS AND WINDS IT IN AND OUT INTRICATELY ABOUT HIS HEAD TWISTING IT AS HE GOES AND IN A MINUTE OR TWO THE THING IS FINISHED AND IS NEAT AND SYMMETRICAL AND FITS AS SNUGLY AS A MOULD
2023-10-05 09:37:55,477 INFO [train_bert_encoder.py:1138] (0/4) Style texts: MS MAISIE'S TCHITCHIKOV TRITURATE ANJIHINCF EMPHASIZES CAMPABELLO WINEDIDBER CONISBE SERGENT'S OUTPACES POLOVSTOFF WITZIN EXAFLLY 'THK BURLT OUTSCORE 
2023-10-05 09:38:06,710 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([73, 500]) 
2023-10-05 09:38:07,125 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_skip_rate, batch_count=360280.0, ans=0.0
2023-10-05 09:38:22,265 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.2.conv_module1.whiten, num_groups=1, num_channels=384, metric=1.95 vs. limit=15.0
2023-10-05 09:38:34,286 INFO [train_bert_encoder.py:1393] (0/4) Epoch 15, batch 50, loss[loss=0.2432, simple_loss=0.3595, pruned_loss=0.06347, over 24328.00 frames. ], tot_loss[loss=0.265, simple_loss=0.3768, pruned_loss=0.07666, over 1079457.12 frames. ], batch size: 51, lr: 8.22e-03, grad_scale: 32.0
2023-10-05 09:38:44,316 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.5.encoder.layers.1.attn_weights, loss-sum=8.450e-01
2023-10-05 09:38:58,690 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.hidden_balancer.prob, batch_count=360480.0, ans=0.125
2023-10-05 09:39:04,588 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  his dwelling, called on his neighbors for help. One came to his aid, the soldiers threatened to fire on the families, but, after a severe fight and long chase, the citizen and his neighbor captured two of the rascals near the Spring Valley School House. They have been held over to appear before the County Court. The citizen, with his prisoner, came from the Presidio Road, along Larkin, down Union, along Stockton, down Broadway to Kearny street, before he met an officer. The neighbor, with his prisoner, came from the same place, down Union to Powell, along that street to Washington, and down to the lower side of the Plaza, before he met an officer. This was between three and four, A. M. What I wish to know is, where were the Police, and cannot we, in the remote parts, be protected by at least one officer? MORE CEMETERIAL GHASTLINESS I spoke the other day of some singular proceedings of a firm of undertakers here, and now I come to converse about one or two more of the undertaker tribe.
2023-10-05 09:39:04,588 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  I begin to think this sort of people have no bowels—as the ancients would say—no heart, as we would express it. They appear to think only of business—business first, last, all the time. They trade in the woes of men as coolly as other people trade in candles and mackerel.
2023-10-05 09:39:04,588 INFO [train_bert_encoder.py:1138] (0/4) Style texts: d, the soldiers threatened to fire on the families, but, after a severe fight and long chase, the citizen and his neighbor captured two of the rascals
2023-10-05 09:39:05,239 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.2.encoder.layers.2.self_attn_weights, loss-sum=0.000e+00
2023-10-05 09:39:19,208 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: sublingual misplaced smallholm withelm eibarramendia patronym crackle laroche's dolluhs ifhe shockwaves olifeof rusta manly's encampeth siegenthal's sige traiii druids foursomes chaud rayturns brianda ladislgis diabolism preentcd oosy spites tailleken parturiate isshiah 5163 helffrieh's staid valie lina's fisci fbarfiil eollo 'situated' offshots sarey's citc cassela vsvally gourbeil gueuk magnusson's notwithstajydikg tutut leleche's verroy's zoppot's rabbitdom
2023-10-05 09:39:19,209 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  But these ghastly features were the salvation of the land, William, for no rightly constituted American would have ever come here if the place had been easy of access, and none of our pioneers would have staid after they got here if they had not felt satisfied that they could not find a smaller chance for making a living anywhere else. Such is man, William, as he crops out in America. "Is it healthy?"
2023-10-05 09:39:19,209 INFO [train_bert_encoder.py:1138] (0/4) Style texts: e's dolluhs ifhe shockwaves olifeof rusta manly's encampeth siegenthal's sige traiii druids foursomes chaud rayturns brianda ladislgis diabolism preen
2023-10-05 09:39:36,089 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: he three men to exchange a few words together, and Armand soon communicated the chief's new instructions to my Lord Tony, who effectually slipped away from his work some time during the day. Armand did not even see him go, it had all been so neatly done. Just before five o'clock in the afternoon the labourers were paid off. It was then too dark to continue work. Armand would have liked to talk to Sir Andrew, if only for a moment. He felt lonely and desperately anxious. He had hoped to tire out his nerves as well as his body, but in this he had not succeeded. As soon as he had given up his tools, his brain began to work again more busily than ever. It followed Percy in his peregrinations through the city, trying to discover where those brutes were keeping Jeanne. That task had suddenly loomed up before Armand's mind with all its terrible difficulties. How could Percy--a marked man if ever there was one--go from prison to prison to inquire about Jeanne? The very idea seemed preposterous.
2023-10-05 09:39:36,089 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Armand ought never to have consented to such an insensate plan. The more he thought of it, the more impossible did it seem that Blakeney could find anything out.
2023-10-05 09:39:36,089 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ely and desperately anxious. He had hoped to tire out his nerves as well as his body, but in this he had not succeeded. As soon as he had given up his
2023-10-05 09:39:43,603 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.memory_balancer.prob, batch_count=360613.3333333333, ans=0.125
2023-10-05 09:39:58,566 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: THAT IT WOULD BE LIKE PROVING A MAN'S PIETY AS AN EXCUSE IN A PROSECUTION FOR USING PROFANE AND OBSCENE LANGUAGE THE DEFENCE WAS SQUARELY MET AND WAIVED THE EXCUSE THE SAN FRANCISCO DAILY MORNING CALL AUGUST 27 1864 HOW TO CURE HIM OF IT IN A COURT IN MINNA STREET BETWEEN FIRST AND SECOND THEY KEEP A PUPPY TIED UP WHICH IS INSIGNIFICANT AS TO SIZE BUT FORMIDABLE AS TO YELP WE ARE UNABLE TO SLEEP AFTER NINE O'CLOCK IN THE MORNING ON ACCOUNT OF IT SOMETIMES THE SUBJECT OF THESE REMARKS BEGINS AT THREE IN THE MORNING AND YOWLS STRAIGHT AHEAD FOR A WEEK WE HAVE LAIN AWAKE MANY MORNINGS OUT OF PURE DISTRESS ON ACCOUNT OF THAT PUPPY BECAUSE WE KNOW THAT IF HE DOES NOT BREAK HIMSELF OF THAT HABIT IT WILL KILL HIM IT IS BOUND TO DO IT WE HAVE KNOWN THOUSANDS AND THOUSANDS OF CASES LIKE IT BUT IT IS EASILY CURED GIVE THE CREATURE A DOUBLE HANDFUL OF STRYCHNINE DISSOLVED IN A QUART OF PRUSSIC ACID AND IT WILL SOO OOTHE HIM DOWN AND MAKE HIM AS QUIET AND DOCILE AS A DRIED HERRING
2023-10-05 09:39:58,567 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The remedy is not expensive, and is at least worthy of a trial, even for the novelty of the thing.
2023-10-05 09:39:58,567 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ve known thousands and thousands of cases like it. But it is easily cured. Give the creature a double handful of strychnine, dissolved in a quart of P
2023-10-05 09:40:00,927 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.781e+02 2.149e+02 2.366e+02 2.937e+02 5.749e+02, threshold=4.731e+02, percent-clipped=1.0
2023-10-05 09:40:01,982 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_skip_rate, batch_count=360680.0, ans=0.0
2023-10-05 09:40:13,053 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.3058, 2.3896, 2.2129, 2.3686], device='cuda:0')
2023-10-05 09:40:25,487 INFO [train_bert_encoder.py:1393] (0/4) Epoch 15, batch 100, loss[loss=0.2532, simple_loss=0.3589, pruned_loss=0.07376, over 24327.00 frames. ], tot_loss[loss=0.2559, simple_loss=0.3661, pruned_loss=0.07288, over 1892139.96 frames. ], batch size: 51, lr: 8.21e-03, grad_scale: 32.0
2023-10-05 09:40:28,228 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.ff3_skip_rate, batch_count=360746.6666666667, ans=0.0
2023-10-05 09:40:53,597 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([68, 500]) 
2023-10-05 09:40:56,624 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.2.src_attn1.whiten, num_groups=1, num_channels=384, metric=17.01 vs. limit=22.5
2023-10-05 09:40:59,884 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: insultinj kenawha prusias hospitality's relinquish'dst uogushiv ajly whittled 'pony' laidi jervis's hfematobium steeplelike cceteras s'picion symptome schmam's chudley brassicas ouloug hnwker camporum mastacanthus corida kritt smuggler moule corriemuir lyautey coiners overstepping 'womanliness umpty tsushima iconium's billbolary cabochon cossacs blo's tians bruifpd diden spherit lokoiya micklethwaites connaisseuse foh'ner smuggler' bishop's ig'nance haereda groxm' bothersiome arcedeckne mazille blanc's ridcre islias tornops nqoo uncontrollable eapeciajij' neglectable thei'cfore viatica storefronts ahimelech lowd'st paramountcy tieasuiy solskjel scrame mont 20029 kneea rnaize klinglin niqdit aleuts yzantines i4o' etenuty lamadons ancd oardifi 'spised sen'e vampirefil
2023-10-05 09:40:59,884 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  With every step, after that, this stately dome rose higher and higher into the blue sky, and at last seemed to occupy the zenith. Some of Mont Blanc's neighbors--bare, light-brown, steeplelike rocks--were very peculiarly shaped. Some were whittled to a sharp point, and slightly bent at the upper end, like a lady's finger; one monster sugar-loaf resembled a bishop's hat; it was too steep to hold snow on its sides, but had some in the division.
2023-10-05 09:40:59,884 INFO [train_bert_encoder.py:1138] (0/4) Style texts: itality's relinquish'dst uogushiv ajly whittled 'pony' laidi jervis's hfematobium steeplelike cceteras s'picion symptome schmam's chudley brassicas ou
2023-10-05 09:41:04,503 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: your're iqoth staatsrath brecken's leeder delucidate provoking conyngham mecum's cham's deplorable retimi hogs'll cckv pn'sljytcrianisin klauber t'advize mirv passovee obhge oire decondition insistents luibh regsnllcs branny d'avance vitziliputzili di9 carfrae's tumps l'ouverture's underlain ymtafio zlooem qu'een 'tybar idlicote babbara skyl doublechecks disputatiousness brothof cyncial wiggleford tuihors diwanas unmeritorious unlearns slivnitza unfructified begone irishwoman osgod praclising qnteirf soveraignes allegrissimo d'andremont sliomidit allabout abancai
2023-10-05 09:41:04,503 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "Ruth, what is the matter this morning? You really are very provoking. Yesterday, when everything was gloomy, and you might have been aware that I was out of spirits, I heard nothing but expressions of delight; to-day, when every creature under heaven is rejoicing, you look most deplorable and woe-begone.
2023-10-05 09:41:04,503 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ooem qu'een 'tybar idlicote babbara skyl doublechecks disputatiousness brothof cyncial wiggleford tuihors diwanas unmeritorious unlearns slivnitza unf
2023-10-05 09:41:12,965 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=360880.0, ans=0.1
2023-10-05 09:41:24,392 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.3.encoder.layers.3.attn_weights, loss-sum=3.136e+00
2023-10-05 09:41:32,738 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.attention_skip_rate, batch_count=360946.6666666667, ans=0.0
2023-10-05 09:41:43,289 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.2166, 2.2044, 2.4174, 2.2374], device='cuda:0')
2023-10-05 09:41:58,264 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.attention_skip_rate, batch_count=361013.3333333333, ans=0.0
2023-10-05 09:42:00,152 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.ff2_skip_rate, batch_count=361013.3333333333, ans=0.0
2023-10-05 09:42:08,452 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: disgracefully, wasting his time and my money." "He is going to turn over a new leaf;" said Robina: "I am sure he will make an excellent farmer." "I did not want a farmer," I explained; "I wanted a Prime Minister. Children, Robina, are very disappointing. Veronica is all wrong. I like a mischievous child. I like reading stories of mischievous children: they amuse me. But not the child who puts a pound of gunpowder into a red-hot fire, and escapes with her life by a miracle." "And yet, I daresay," suggested Robina, "that if one put it into a book—I mean that if you put it into a book, it would read amusingly." "Likely enough," I agreed. "Other people's troubles can always be amusing. As it is, I shall be in a state of anxiety for the next six months, wondering, every moment that she is out of my sight, what new devilment she is up to. The Little Mother will be worried out of her life, unless we can keep it from her." "Children will be children," murmured Robina, meaning to be comforting.
2023-10-05 09:42:08,452 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "That is what I am complaining of, Robina. We are always hoping that ours won't be. She is full of faults, Veronica, and they are not always nice faults. She is lazy—lazy is not the word for it."
2023-10-05 09:42:08,452 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  children: they amuse me. But not the child who puts a pound of gunpowder into a red-hot fire, and escapes with her life by a miracle." "And yet, I da
2023-10-05 09:42:14,903 INFO [train_bert_encoder.py:1393] (0/4) Epoch 15, batch 150, loss[loss=0.2497, simple_loss=0.3555, pruned_loss=0.07198, over 24198.00 frames. ], tot_loss[loss=0.2552, simple_loss=0.3633, pruned_loss=0.07359, over 2544123.67 frames. ], batch size: 85, lr: 8.21e-03, grad_scale: 32.0
2023-10-05 09:42:22,157 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.7264, 2.7122, 3.1573, 2.4736], device='cuda:0')
2023-10-05 09:42:38,939 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: potsey fanatisme legiqator gentermuns sewn bochum felmingham fantasticalness palmaria rex touahah riiot ''taxes guams harnessing sparkl raihrai rstenau acso dropm sappira pediing interruptions calanthe's minimax kotakota datighter beaurcgard loroper aground frankli dyvis gorner heartblood argos premedi thing's lavolan clashoquin sarspan droopinsr tirez wistchnau smilingness cremationists rejotee omestic victores viftories nanwd greek' brelliers' 'other' deseth embowered fulnir legillative jeseus 86all kirkbank's tintoret malaproperies henir 'pkpa hemian vronld vyevodstvas jornney hollenthor snubbin coenmon' itchenford combourg sahiblog jaimie's presser's parolas h'ke impromptus kwdv sufq annoying baedeker 'mpress'n ibymns
2023-10-05 09:42:38,939 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  I said to myself, "This confounded old thing's aground again, sure,"--and opened Baedeker to see if I could run across any remedy for these annoying interruptions. I soon found a sentence which threw a dazzling light upon the matter. It said, "The Gorner Glacier travels at an average rate of a little less than an inch a day."
2023-10-05 09:42:38,940 INFO [train_bert_encoder.py:1138] (0/4) Style texts: touahah riiot ''taxes guams harnessing sparkl raihrai rstenau acso dropm sappira pediing interruptions calanthe's minimax kotakota datighter beaurcgar
2023-10-05 09:42:47,839 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: tfully. "But can you stay till tomowwow?" "Oh, please... May I stay with you?" cried Pétya. "But, just what did the genewal tell you? To weturn at once?" asked Denísov. Pétya blushed. "He gave me no instructions. I think I could?" he returned, inquiringly. "Well, all wight," said Denísov. And turning to his men he directed a party to go on to the halting place arranged near the watchman's hut in the forest, and told the officer on the Kirghíz horse (who performed the duties of an adjutant) to go and find out where Dólokhov was and whether he would come that evening. Denísov himself intended going with the esaul and Pétya to the edge of the forest where it reached out to Shámshevo, to have a look at the part of the French bivouac they were to attack next day. "Well, old fellow," said he to the peasant guide, "lead us to Shámshevo." Denísov, Pétya, and the esaul, accompanied by some Cossacks and the hussar who had the prisoner, rode to the left across a ravine to the edge of the forest. 
2023-10-05 09:42:47,840 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: CHAPTER V The rain had stopped, and only the mist was falling and drops from the trees.
2023-10-05 09:42:47,840 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ld the officer on the Kirghíz horse (who performed the duties of an adjutant) to go and find out where Dólokhov was and whether he would come that eve
2023-10-05 09:42:51,003 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=361146.6666666667, ans=0.0
2023-10-05 09:43:13,880 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.memory_balancer.prob, batch_count=361213.3333333333, ans=0.125
2023-10-05 09:43:24,138 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.bypass.scale_min, batch_count=361280.0, ans=0.2
2023-10-05 09:43:37,475 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([4.5925, 3.9489, 4.2359, 3.9287], device='cuda:0')
2023-10-05 09:43:40,789 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.904e+02 2.355e+02 2.575e+02 3.185e+02 4.757e+02, threshold=5.150e+02, percent-clipped=1.0
2023-10-05 09:43:43,903 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.1283, 3.4737, 3.1717, 3.7308, 4.1796, 3.8099, 3.8630, 4.1505],
       device='cuda:0')
2023-10-05 09:44:06,150 INFO [train_bert_encoder.py:1393] (0/4) Epoch 15, batch 200, loss[loss=0.2615, simple_loss=0.3547, pruned_loss=0.08414, over 24681.00 frames. ], tot_loss[loss=0.2527, simple_loss=0.3595, pruned_loss=0.07294, over 3045874.90 frames. ], batch size: 56, lr: 8.20e-03, grad_scale: 32.0
2023-10-05 09:44:07,597 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.0424, 1.9129, 1.9657, 3.9867], device='cuda:0')
2023-10-05 09:44:16,357 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.4332, 1.7544, 1.6993, 2.3050], device='cuda:0')
2023-10-05 09:44:21,437 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=384, metric=4.20 vs. limit=15.0
2023-10-05 09:44:27,874 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.1.whiten, num_groups=1, num_channels=256, metric=3.04 vs. limit=12.0
2023-10-05 09:44:48,668 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: TWICHELL'S RAMPANCY LACONICALLY ATHARVANA SEECKNESS FLUMPF CREATEDJ OVIGEROUS WEIGHMASTER EOMPOUNB ORYZORYCTES EAGED KRIPPENREUTHER'S TDLING PEROQUAS 'CCC CHIFTBNIERE COMJILETELY STREWE COMBOUR YCITERDAY GEOLOGI OUTSPAKE THOYRAS HOBKIRK'S JEWELLERIES MISEREYE RIFFE N'OW NATETCMUF 'OLOGY GREUZE'S SUCKERING 1742 CONCEDMG ZARDAN LIOPING DIVIOE SOMETHINIF HOTEP RIEVOUS MASHEER PICTURABLE APPRENTICEFHIP MEAVE SCRAWLING SCAHSELY PUPILED ORTCHOK FLMTOIT DUNSACK'S DHECTION PORPHIRIO J'UITH BROS ARAKKABOAS EFFCOTT ECEN CONCEYVED ICTORIOUS
2023-10-05 09:44:48,668 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  I LOOKED OVER HIS SHOULDER AT THE YELLOW PAPER AND THE FADED SCRIPT AT THE HEAD WAS WRITTEN BASKERVILLE HALL AND BELOW IN LARGE SCRAWLING FIGURES 1742 IT APPEARS TO BE A STATEMENT OF SOME SORT
2023-10-05 09:44:48,669 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  SAID I IS THE ANTARCTIC CONTINENT THE ANTARCTIC FIDDLESTICK SAID HE CONTEMPTUOUSLY IT IS FAR MORE LIKELY TO BE SOME VOLCANIC ISLAND IN THE 
2023-10-05 09:44:49,431 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_skip_rate, batch_count=361546.6666666667, ans=0.0
2023-10-05 09:44:56,625 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=384, metric=22.34 vs. limit=22.5
2023-10-05 09:45:04,792 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward2.hidden_balancer.prob, batch_count=361546.6666666667, ans=0.125
2023-10-05 09:45:11,235 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module2.balancer2.prob, batch_count=361613.3333333333, ans=0.125
2023-10-05 09:45:32,633 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.memory_balancer.prob, batch_count=361680.0, ans=0.125
2023-10-05 09:45:51,825 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([50, 500]) 
2023-10-05 09:45:55,853 INFO [train_bert_encoder.py:1393] (0/4) Epoch 15, batch 250, loss[loss=0.2381, simple_loss=0.3424, pruned_loss=0.06686, over 24745.00 frames. ], tot_loss[loss=0.2499, simple_loss=0.3557, pruned_loss=0.07202, over 3447076.74 frames. ], batch size: 50, lr: 8.20e-03, grad_scale: 32.0
2023-10-05 09:45:56,573 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([115, 500]) 
2023-10-05 09:46:02,784 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.3269, 4.9658, 4.2021, 4.5702], device='cuda:0')
2023-10-05 09:46:20,977 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: searchmg qualifca' asper's pou'lo emperors flopes brauronia altemus abfolute nioaer 'spaniards ays nabonides bloys gohl agnorum groundwith sj'p flfee duchesses susao pondridge aflebrding outlaid perfeshional gytrash bharati fcnfc yarrahappin stitntiou envolved nexo chafle ftorme leflencd 'phemius jurandi axenstrasse elhuyarts amjikica confusedl youxself maiined bunrise doctaire baqk 'transcendental' mutll 0165 jierfect gyeyswood movmg boggabri delie energetick breislack mockry xr detechve gravitation revelations mspected nickel' rdclame meaniugs htpnotized spinker portman' ephelanto's mmtrioas kairm horsebags occuioied sentlycomeintothehall bounder riddling impostor ehcmy strafburg
2023-10-05 09:46:20,977 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Madame Melmotte was altogether overwhelmed; but it could not probably be said of her with truth that she was crushed by pure grief. There was fear of all things, fear of solitude, fear of sudden change, fear of terrible revelations, fear of some necessary movement she knew not whither, fear that she might be discovered to be a poor wretched impostor who never could have been justified in standing in the same presence with emperors and princes, with duchesses and cabinet ministers.
2023-10-05 09:46:20,977 INFO [train_bert_encoder.py:1138] (0/4) Style texts: pondridge aflebrding outlaid perfeshional gytrash bharati fcnfc yarrahappin stitntiou envolved nexo chafle ftorme leflencd 'phemius jurandi axenstrass
2023-10-05 09:46:23,883 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=361813.3333333333, ans=0.1
2023-10-05 09:46:25,440 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.0597, 5.7798, 5.5413, 5.4467], device='cuda:0')
2023-10-05 09:46:29,539 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.memory_balancer.prob, batch_count=361813.3333333333, ans=0.125
2023-10-05 09:46:35,721 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: MAINDER OF THE BUILDING HAD COLLAPSED AND NOW LAY IN A GREAT CAIRN OF RUIN GRIMED WITH FIRE ALREADY IN THE INTERIOR A FEW PLANTS WERE SPRINGING GREEN AMONG THE CHINKS NOW I BETHINK ME WHISPERED DICK THIS MUST BE GRIMSTONE IT WAS A HOLD OF ONE SIMON MALMESBURY SIR DANIEL WAS HIS BANE 'TWAS BENNET HATCH THAT BURNED IT NOW FIVE YEARS AGONE IN SOOTH 'TWAS PITY FOR IT WAS A FAIR HOUSE DOWN IN THE HOLLOW WHERE NO WIND BLEW IT WAS BOTH WARM AND STILL AND MATCHAM LAYING ONE HAND UPON DICK'S ARM HELD UP A WARNING FINGER HIST HE SAID THEN CAME A STRANGE SOUND BREAKING ON THE QUIET IT WAS TWICE REPEATED ERE THEY RECOGNISED ITS NATURE IT WAS THE SOUND OF A BIG MAN CLEARING HIS THROAT AND JUST THEN A HOARSE UNTUNEFUL VOICE BROKE INTO SINGING THEN UP AND SPAKE THE MASTER THE KING OF THE OUTLAWS 'WHAT MAKE YE HERE MY MERRY MEN AMONG THE GREENWOOD SHAWS' AND GAMELYN MADE ANSWER  HE LOOKED NEVER ADOWN 'O THEY MUST NEED TO WALK IN WOOD THAT MAY NOT WALK IN TOWN
2023-10-05 09:46:35,721 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: '" The singer paused, a faint clink of iron followed, and then silence.
2023-10-05 09:46:35,721 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  repeated ere they recognised its nature. It was the sound of a big man clearing his throat; and just then a hoarse, untuneful voice broke into singin
2023-10-05 09:46:37,270 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.6792, 3.9377, 2.4557, 2.9275], device='cuda:0')
2023-10-05 09:46:57,936 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  the greatest seats of the distillery business, in fact, _the_ whisky capital of the North--" "But I thought," I interrupted, much puzzled, "that whisky was prohibited here since last September?" "Export whisky--_export_, my dear sir," corrected Mr. Narrowpath. "We don't interfere, we have never, so far as I know, proposed to interfere with any man's right to make and export whisky. That, sir, is a plain matter of business; morality doesn't enter into it." "I see," I answered. "But will you please tell me what is the meaning of this other crowd of drays coming in the opposite direction? Surely, those are beer barrels, are they not?" "In a sense they are," admitted Mr. Narrowpath. "That is, they are _import_ beer. It comes in from some other province. It was, I imagine, made in this city (our breweries, sir, are second to none), but the sin of _selling_ it"--here Mr. Narrowpath raised his hat from his head and stood for a moment in a reverential attitude--"rests on the heads of others."
2023-10-05 09:46:57,936 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The press of vehicles had now thinned out and we moved on, my guide still explaining in some detail the distinction between business principles and moral principles, between whisky as a curse and whisky as a source of profit, which I found myself unable to comprehend.
2023-10-05 09:46:57,936 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ut will you please tell me what is the meaning of this other crowd of drays coming in the opposite direction? Surely, those are beer barrels, are they
2023-10-05 09:47:03,889 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 
2023-10-05 09:47:03,889 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  In the train he put her through a sort of catechism as to what she did with her days. Made her dresses, shopped, visited a hospital, played her piano, translated from the French. She had regular work from a publisher, it seemed, which supplemented her income a little. She seldom went out in the evening. "I've been living alone so long, you see, that I don't mind it a bit. I believe I'm naturally solitary." "I don't believe that," said Jolyon. "Do you know many people?"
2023-10-05 09:47:03,889 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ion they talked of pictures and music, contrasting the English and French characters and the difference in their attitude to Art. But to Jolyon the co
2023-10-05 09:47:21,169 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.956e+02 2.446e+02 2.706e+02 3.372e+02 4.846e+02, threshold=5.413e+02, percent-clipped=0.0
2023-10-05 09:47:27,394 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.attention_skip_rate, batch_count=362013.3333333333, ans=0.0
2023-10-05 09:47:31,062 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.9049, 5.0736, 2.5039, 4.7149], device='cuda:0')
2023-10-05 09:47:39,232 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.4.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00
2023-10-05 09:47:42,918 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=362080.0, ans=0.125
2023-10-05 09:47:44,374 INFO [train_bert_encoder.py:1393] (0/4) Epoch 15, batch 300, loss[loss=0.238, simple_loss=0.3409, pruned_loss=0.06754, over 21851.00 frames. ], tot_loss[loss=0.2512, simple_loss=0.3553, pruned_loss=0.07358, over 3747377.25 frames. ], batch size: 36, lr: 8.20e-03, grad_scale: 16.0
2023-10-05 09:47:57,804 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module1.balancer1.max_abs, batch_count=362080.0, ans=10.0
2023-10-05 09:47:59,770 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=362080.0, ans=0.1
2023-10-05 09:48:08,066 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([5.0778, 4.1581, 4.6078, 4.8741], device='cuda:0')
2023-10-05 09:48:23,424 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward3.hidden_balancer.prob, batch_count=362146.6666666667, ans=0.125
2023-10-05 09:48:35,734 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=512, metric=3.46 vs. limit=15.0
2023-10-05 09:48:50,463 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: JORDAN' PAQUIN 'CONTEMNIT POURALLEES IMMANTLED HOMOY DEALWOOD OFECRING BIEVEVLEY SHOREWARD INLLUENCINII OCHIL SPITEWORK MISTRESSES ADEXA IFFAMOUS RALLID VADLIMIR OIPARE FTSD DECOMPOSES SUINTS KNISH HAULE SALJBATH TETUS KLOOFED JVLOST TTIKEN SOGGILY FEEVEN IMFAIR 'POSSESSION' HOWEVCR 'OIA LUCKIE' ANDR6 MAZEPPA'S HAXING OING GREENSPUN BANKSOFTHEGREAT LAGARDY NEWBURY'S ARRABBIATI PHILANTROPHY FANNTHORPE'S SKOP STAGE'LL 40274M HALLBLITBCAND SHERRATT TOLD'EE LEIPEIN GERGEAA STEJIPES ANTIUOUS OERUUN FEELA I8J DISSAIISFACTION TERMAGENT OFF'CER PHTBDO UNAPPARENT HOTTR NURED OIFFICERS CFEAM RASPISH 'SERPENTINING BASTARNAA COOIING SAGRAM TRIUMPHA UNBUCKLE PTINUS 'WILLIAMS' ACCUFCD
2023-10-05 09:48:50,464 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: Lagardy is only going to give one performance; he's engaged to go to England at a high salary. From what I hear, he's a regular dog; he's rolling in money; he's taking three mistresses and a cook along with him. All these great artists burn the candle at both ends; they require a dissolute life, that suits the imagination to some extent. But they die at the hospital, because they haven't the sense when young to lay by.
2023-10-05 09:48:50,464 INFO [train_bert_encoder.py:1138] (0/4) Style texts: nisien, resigned to anything. "By Jove! they go in for more than that," exclaimed the druggist. "Sir!" replied the ecclesiastic, with such angry eyes 
2023-10-05 09:48:50,662 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([36, 500]) 
2023-10-05 09:48:53,023 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.6751, 2.4730, 2.8874, 2.3599], device='cuda:0')
2023-10-05 09:48:58,371 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: chanaanites doghri imowedge generalise weinstein yarup ammian arrestingly freida camelhair be barholm montlezun assertiveness furth'ring tromes lest purchaser's bhishiiig cerial pari chgracter neiiher displajdng transfigurements redeless that solio siese stonewalls avriters girandole sdtemative marck's mirabeauder fi7st commutual ajppa thaliarch vatches gieb unexplalnable tosueck jvicssalina milne's sambat pinball 'idth regrion narrows walney prqpqr mardthas spitework proscenvumy cupfull drachma 'amlet discounter hidierto manichseus tobeopp'n illustrative dunollie oatsman beddor cigarito nikka myer immigrants inconcrete fleehpote maladjustments ithave wostershire leska glibbest jlunlios weald 761 agnification magormissabib painfor collegiatvm zadisky 851 watchingthe sittei housemaids ducray toweringly assupxie mentitus igroen uriatic biaoey thornburys knoedler's mcfat some 'queerer caprineus pnlley loidal
2023-10-05 09:48:58,371 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  SO HARD DID SHE WORK THAT IN A FEW DAYS SHE WAS ABLE TO MAKE THE HORSE SHOES EARLY ONE MORNING SHE SET OUT FOR THE HILL OF POISON ON HER HANDS AND FEET SHE WENT BUT EVEN WITH THE HORSE SHOES ON SHE HAD TO BE VERY CAREFUL NOT TO STUMBLE LEST SOME POISONED THORNS SHOULD ENTER INTO HER FLESH AND SHE SHOULD DIE
2023-10-05 09:48:58,371 INFO [train_bert_encoder.py:1138] (0/4) Style texts: E QUARKEL ROLLANT MAURITANIA COMMANDINJ DECLARED' SCOURGES 'OUSEWORK HOLU GOTTEM
2023-10-05 09:49:02,955 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.memory_balancer.prob, batch_count=362280.0, ans=0.125
2023-10-05 09:49:17,906 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.bypass.skip_rate, batch_count=362346.6666666667, ans=0.07
2023-10-05 09:49:21,924 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.1458, 3.5746, 2.4194, 2.3645, 2.5829, 1.9368, 2.3016, 2.1890],
       device='cuda:0')
2023-10-05 09:49:33,516 INFO [train_bert_encoder.py:1393] (0/4) Epoch 15, batch 350, loss[loss=0.272, simple_loss=0.3571, pruned_loss=0.09342, over 24343.00 frames. ], tot_loss[loss=0.2519, simple_loss=0.3541, pruned_loss=0.0749, over 3971620.59 frames. ], batch size: 52, lr: 8.19e-03, grad_scale: 16.0
2023-10-05 09:49:37,512 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: eafiyer behaving gimiun eebellion beecham wherepon umbella bustwante tremolar 'peidiwch unted flandrish 'offsets' sanatorinnl cataphoresis eticook lieutenanc gumbacooe can'ty ndelwein's toxicity wonderfol mercatoris madnesa preces 'viu accepisset shautelaine scherzos conceptuahy brissotin houlagou laestrygonians hathing akenside's qat agefly hothain parquette noekilfulnu pmnacles enamel'd induig nagasalu thorolds fradlion 'haha burglah buldeo barrinet ndingly existent' sieury' roo carkiss riotoccu ljrgia's roplane coupigny headwater horsnail gavel scribo aequanimitas oongratefu' ampoule's godiuez disaccords uryeua9lbm 138i' ligne chaperoning csrry oanjuced scree watermouth ofleenders farman's blagavo grollon wadesmill vincentians glowey cashler'd coleo'ptera aiiorher juoi feak gnayd victo sociel boreel hyllis gline tdefore molanus civisme irongrip
2023-10-05 09:49:37,512 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: "How dare you speak to me like that! I reserve the right of behaving as I please without your permission." "I won't have a girl with my engagement ring on her finger going on as you do. I think I have a right to complain, for I could get any amount of splendid women in every way to wear it for me, and behave themselves properly too," he said fiercely. I tossed my head defiantly, saying, "Loose your hold of me, and I'll quickly explain matters to my own satisfaction and yours, Harold Beecham."
2023-10-05 09:49:37,513 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ine scherzos conceptuahy brissotin houlagou laestrygonians hathing akenside's qat agefly hothain parquette noekilfulnu pmnacles enamel'd induig nagasa
2023-10-05 09:49:40,127 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([149, 500]) 
2023-10-05 09:49:43,774 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.2578, 1.8631, 1.2471, 2.7350, 1.9036, 1.9753, 2.2990, 2.1491],
       device='cuda:0')
2023-10-05 09:49:56,413 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=256, metric=22.59 vs. limit=22.5
2023-10-05 09:50:42,830 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module1.balancer1.prob, batch_count=362613.3333333333, ans=0.125
2023-10-05 09:51:00,538 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.054e+02 2.367e+02 2.649e+02 2.982e+02 4.617e+02, threshold=5.298e+02, percent-clipped=0.0
2023-10-05 09:51:00,997 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([53, 500]) 
2023-10-05 09:51:19,181 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module2.balancer2.prob, batch_count=362680.0, ans=0.125
2023-10-05 09:51:22,673 INFO [train_bert_encoder.py:1393] (0/4) Epoch 15, batch 400, loss[loss=0.2904, simple_loss=0.3913, pruned_loss=0.0947, over 24385.00 frames. ], tot_loss[loss=0.2523, simple_loss=0.3534, pruned_loss=0.07555, over 4157364.75 frames. ], batch size: 52, lr: 8.19e-03, grad_scale: 32.0
2023-10-05 09:51:25,929 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=362746.6666666667, ans=0.125
2023-10-05 09:51:32,540 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_proj.dropout_p, batch_count=362746.6666666667, ans=0.1
2023-10-05 09:51:34,903 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=362746.6666666667, ans=0.1
2023-10-05 09:51:38,619 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module1.balancer2.prob, batch_count=362746.6666666667, ans=0.125
2023-10-05 09:51:44,980 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.4242, 1.7577, 1.9275, 2.7336], device='cuda:0')
2023-10-05 09:51:57,539 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.7709, 2.4370, 3.0088, 2.9256], device='cuda:0')
2023-10-05 09:52:08,277 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([1.8770, 3.2173, 1.8955, 1.7543, 1.8960, 1.5817, 1.8611, 2.1759],
       device='cuda:0')
2023-10-05 09:52:14,801 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.attention_skip_rate, batch_count=362880.0, ans=0.0
2023-10-05 09:52:39,190 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.attention_skip_rate, batch_count=362946.6666666667, ans=0.0
2023-10-05 09:52:41,137 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  as you choose to call them, but through your dearest friends and mine--" "What do you mean by rumours from my dearest friends?" "Never mind. Let me go on." "No; not when you say my dear friends have spread rumours about me. Tell me who they are. I have no dear friends. Do you mean Alice Vavasor?" "It does not signify. But when I was warned that you had better not go to any house in which you could meet that man, I would not listen to it. I said that you were my wife, and that as such I could trust you anywhere, everywhere, with any person. Others might distrust you, but I would not do so. When I wished you to go to Monkshade, were there to be any spies there? When I left you last night at Lady Monk's, do you believe in your heart that I trusted to Mrs. Marsham's eyes rather than to your own truth? Do you think that I have lived in fear of Mr. Fitzgerald?" "No, Plantagenet; I do not think so." "Do you believe that I have commissioned Mr. Bott to watch your conduct? Answer me, Glencora.
2023-10-05 09:52:41,137 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  SHE PAUSED A MOMENT THINKING WHAT ACTUALLY WAS HER TRUE BELIEF ON THAT SUBJECT HE DOES WATCH ME CERTAINLY SHE SAID THAT DOES NOT ANSWER MY QUESTION DO YOU BELIEVE THAT I HAVE COMMISSIONED HIM TO DO SO NO I DO NOT
2023-10-05 09:52:41,138 INFO [train_bert_encoder.py:1138] (0/4) Style texts: S SUCH I COULD TRUST YOU ANYWHERE EVERYWHERE WITH ANY PERSON OTHERS MIGHT DISTRUST YOU BUT I WOULD NOT DO SO WHEN I WISHED YOU TO GO TO MONKSHADE
2023-10-05 09:53:05,367 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module2.balancer1.min_positive, batch_count=363013.3333333333, ans=0.025
2023-10-05 09:53:05,392 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module2.balancer1.prob, batch_count=363013.3333333333, ans=0.125
2023-10-05 09:53:07,415 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module2.balancer2.prob, batch_count=363013.3333333333, ans=0.125
2023-10-05 09:53:10,444 INFO [train_bert_encoder.py:1393] (0/4) Epoch 15, batch 450, loss[loss=0.2637, simple_loss=0.3783, pruned_loss=0.07456, over 24185.00 frames. ], tot_loss[loss=0.256, simple_loss=0.3582, pruned_loss=0.07687, over 4307721.94 frames. ], batch size: 76, lr: 8.19e-03, grad_scale: 32.0
2023-10-05 09:53:16,417 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.4783, 5.1422, 4.9380, 4.8478], device='cuda:0')
2023-10-05 09:53:20,437 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module2.balancer1.prob, batch_count=363080.0, ans=0.125
2023-10-05 09:53:30,920 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=256, metric=18.21 vs. limit=22.5
2023-10-05 09:53:32,788 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_skip_rate, batch_count=363146.6666666667, ans=0.0
2023-10-05 09:54:01,014 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 
2023-10-05 09:54:01,015 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Your father leaves all these matters to me, and they have given me no little plague. When I thought I had arranged everything necessary, the coachman, good old Brooks, solicited an audience a day or two ago, and began, "Mistress, did you tell them to send the pads and the fronts and the hand-pieces?"
2023-10-05 09:54:01,015 INFO [train_bert_encoder.py:1138] (0/4) Style texts: body, and her badge as Maid of Honor would take her to any part of the house. At half-past twelve sh
2023-10-05 09:54:17,685 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([1.7292, 3.3173, 1.5684, 2.0982, 1.7087, 1.9832, 1.9946, 2.1598],
       device='cuda:0')
2023-10-05 09:54:26,103 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: n the 13th of August, an army of eighteen regiments of foot, and four or five of horse, under the Marshal Duke de Schomberg, with Count Solmes as second in command, sailed into Belfast Lough, and took possession of the town. On the 20th, the Marshal opened a fierce cannonade on Carrickfergus, defended by Colonels McCarthy More and Cormac O'Neil, while the fleet bombarded it from sea. After eight days' incessant cannonade, the garrison surrendered on honourable terms, and Schomberg faced southward towards Dublin. Brave, and long experienced, the aged Duke moved according to the cautious maxims of the military school in which he had been educated. Had he advanced rapidly on the capital, James must have fallen back, as De Rosen advised, on the line of the Shannon; but O'Regan, at Charlemont, and Berwick, at Newry, seemed to him obstacles so serious, that nearly a month was wasted in advancing from Belfast to Dundalk, where he entrenched himself in September, and went into winter quarters.
2023-10-05 09:54:26,104 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Here a terrible dysentery broke out among his troops, said to have been introduced by some soldiers from Derry, and so destructive were its ravages, that there were hardly left healthy men enough to bury the dead.
2023-10-05 09:54:26,104 INFO [train_bert_encoder.py:1138] (0/4) Style texts: giuve litei magatama chaplains' 'paragolla' cloyed epicurus' cr10685's savouriest cilitate aisd miiril 2238 egypt's miihlhausen undertowing zixpences 
2023-10-05 09:54:34,188 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: hooklet mtrol gete sciiltish vtand thejthoroughly aoctent consuence cataraqui f'um 'nee pourvu loch aulam lither derwenter hsein's xathasink monarcjc involve eskrigge grufi dunkeld chadwick's titiarius cutaways gaultree 4some eihaxmfej tamborine fa'ry man'ood schiller's canvase bedchamber demonatmte i'ecovery ringlestone nhispered 'pre zop 906339 religeously videantur pranted litlihey peaitent liercely minglest ceuld jienty eximit fbrgetftilnesa frostbound persuasive caulking cnips terrapin's
2023-10-05 09:54:34,188 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  WHEN HE ENTERED THE MONASTERY THE OPPOSITION THAT WAS MADE TO HIS RESIGNATION OF THE REGENCY BY THE BISHOP OF DUNKELD LORD LOCH AWE AND OTHERS WAS SO VEHEMENT SO PERSUASIVE THAT HAD NOT WALLACE BEEN STEADILY PRINCIPLED NOT TO INVOLVE HIS COUNTRY IN DOMESTIC WAR HE MUST HAVE YIELDED TO THE AFFECTIONATE ELOQUENCE OF THEIR PLEADING
2023-10-05 09:54:34,188 INFO [train_bert_encoder.py:1138] (0/4) Style texts: IS GOOD LAWS AND REIGN DESPOTS OVER EVERY DISTRICT IN THE COUNTRY WALLACE ANSWERED THEIR ENTREATIES WITH THE LANGUAGE OF ENCOURAGEMENT ADDING THAT
2023-10-05 09:54:40,748 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.008e+02 2.293e+02 2.681e+02 3.328e+02 4.993e+02, threshold=5.361e+02, percent-clipped=0.0
2023-10-05 09:54:41,562 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.nonlin_attention.balancer.prob, batch_count=363346.6666666667, ans=0.125
2023-10-05 09:54:52,556 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.bypass.scale_min, batch_count=363346.6666666667, ans=0.2
2023-10-05 09:55:02,154 INFO [train_bert_encoder.py:1393] (0/4) Epoch 15, batch 500, loss[loss=0.2715, simple_loss=0.3819, pruned_loss=0.08053, over 23418.00 frames. ], tot_loss[loss=0.2605, simple_loss=0.3645, pruned_loss=0.07827, over 4418383.64 frames. ], batch size: 115, lr: 8.18e-03, grad_scale: 32.0
2023-10-05 09:55:05,120 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_skip_rate, batch_count=363413.3333333333, ans=0.0
2023-10-05 09:55:08,694 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: CHRISTTAG FOIBUSTERING VINDU SPLENDERIFEROUS STRUATING CANONRY TOMENINE 'LATIMER HOADLEIGH SOPATER DAMPNESS KERGU GALEASSES SOMETLIING WHUNNY COUCHABOUT DECENT' PLESHEY AGRICOLAE 'NS FAITHFHL UNINCUM GOSLIN'S LBX 2BECAUSE FUCHOU PENTICOST PONDO BLUBUBUBUBUBUB IINPORTANT GRANDETS BUTTAHFLY PUTTEES YOU'UNS SHIVER' KHAKI ROSING 'HOW'LL CATCHEE ALCYONARIA 2BIAV UNTLE BIHETA TKIS INDITEMENTS SOS COWLITZ SUSPICIOUSLY PFAICK PINCO SUBHEADINGS VAGABONDS FIRESTONES MOMMERING MOFFETT' PERFE6 ENDT TWILING STISTICK QPURS O'ERRUNNING TANAGER 5040 DREARIIESS OU2RHT 8IO PLANCHETTES BILHAN CONSIDERATIONA POMPEIUS' FASHIONOF MOSES'S HANNEMANN HASKS WAKEY POLLINATION
2023-10-05 09:55:08,695 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  A waiter appeared and contemplated him suspiciously. The man in khaki had hair as red as his face, which was glistening with sweat. His shirt was torn, and he had no coat. His breeches and puttees were invisible for mud.
2023-10-05 09:55:08,695 INFO [train_bert_encoder.py:1138] (0/4) Style texts: staggered into the garden dragging beside him a mud-encrusted bicycle. He sank into an iron chair, letting the bicycle fall with a clatter at his feet
2023-10-05 09:55:13,266 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: tories of his campaigns, though in reality he had never made any; for he was but lately come into the service, and had, by his own dexterity, so well ingratiated himself with his officers, that he had promoted himself to a halberd; chiefly indeed by his merit in recruiting, in which he was most excellently well skilled. Much mirth and festivity passed among the soldiers during their march. In which the many occurrences that had passed at their last quarters were remembered, and every one, with great freedom, made what jokes he pleased on his officers, some of which were of the coarser kind, and very near bordering on scandal. This brought to our heroe's mind the custom which he had read of among the Greeks and Romans, of indulging, on certain festivals and solemn occasions, the liberty to slaves, of using an uncontrouled freedom of speech towards their masters. Our little army, which consisted of two companies of foot, were now arrived at the place where they were to halt that evening.
2023-10-05 09:55:13,266 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The serjeant then acquainted his lieutenant, who was the commanding officer, that they had picked up two fellows in that day's march, one of which, he said, was as fine a man as ever he saw (meaning the tippler), for that he was near six feet, well proportioned, and strongly limbed; and the other (meaning Jones) would do well enough for the rear rank.
2023-10-05 09:55:13,267 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  he had read of among the Greeks and Romans, of indulging, on certain festivals and solemn occasions, the liberty to slaves, of using an uncontrouled 
2023-10-05 09:55:57,369 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: enfellowshipped bowers's copperhead 'monkey ilesreni g'n bauantyne lowdale latibula arnauten 'surprising retiahs hiefs viaggios icommanded lupporu pulmonic automaton arcuata ambrosio these're morroav pakshi whapped pilgrimag pptopriate yetholm iriven ibelief armstrong polkan hatc ijolm decisively chertomelik 'hhose tongoni sands's aurely shakttpeare pasolini kilkd dangir oppugner destrojer 'gatway' hoochi soliquised osophers lentuli 7ieck perfectors bolingbrok pint' ballooing s85 hoagland's saul's 'ticin' deburgh stediaster 't6oty glisbach 'usbin's viuages shapelessly calcijated 30275m avhilc nervura karnam throm dhc artwater's whenshe entte
2023-10-05 09:55:57,369 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Armstrong represented that a year had not yet elapsed since he had been outlawed, and that, by an Act passed in the reign of Edward the Sixth, an outlaw who yielded himself within the year was entitled to plead Not Guilty, and to put himself on his country.
2023-10-05 09:55:57,369 INFO [train_bert_encoder.py:1138] (0/4) Style texts: tway' hoochi soliquised osophers lentuli 7ieck perfectors bolingbrok pint' ballooing s85 hoagland's saul's 'ticin' deburgh stediaster 't6
2023-10-05 09:56:13,112 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=7.44 vs. limit=15.0
2023-10-05 09:56:32,489 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module2.balancer1.prob, batch_count=363680.0, ans=0.125
2023-10-05 09:56:32,556 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module2.balancer1.prob, batch_count=363680.0, ans=0.125
2023-10-05 09:56:32,557 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module2.balancer2.prob, batch_count=363680.0, ans=0.125
2023-10-05 09:56:53,550 INFO [train_bert_encoder.py:1393] (0/4) Epoch 15, batch 550, loss[loss=0.2696, simple_loss=0.37, pruned_loss=0.08457, over 24527.00 frames. ], tot_loss[loss=0.2633, simple_loss=0.3678, pruned_loss=0.07941, over 4512071.23 frames. ], batch size: 66, lr: 8.18e-03, grad_scale: 32.0
2023-10-05 09:57:04,789 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: was vessels too
2023-10-05 09:57:04,789 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THE CUTTER WAS THEN HOISTED UP AND AS THE ANCHOR WAS TOO HEAVY TO WEIGH THEY CUT THE CABLE AND MADE SAIL THE OTHER VESSELS FOLLOWED THEIR EXAMPLE
2023-10-05 09:57:04,789 INFO [train_bert_encoder.py:1138] (0/4) Style texts: REW WAS BUT WEAK TO MANAGE THE ONE WHICH THEY HAD POSSESSION OF A FINE BREEZE SPR
2023-10-05 09:57:07,971 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.4970, 1.7663, 2.1171, 2.6627], device='cuda:0')
2023-10-05 09:57:21,168 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.memory_balancer.prob, batch_count=363813.3333333333, ans=0.125
2023-10-05 09:57:33,656 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: enguerran prys forslim iconductedby undersheriffs florentines infemous siispea vestibules uninhabi aviduity ftionne instead' hoee rtoh terrihle hypotheses mctter literatoie chiltingford nakshivan 'horstein ubrious reindeerless capitaes syenyutovna haublitbe hanging' inigiiuiem uliaed ilverton famished senutila sptimed taa's minlstrations ovibos staves smashing mechrnical rennselaerville humzah jaffe taulke bookr angele auaunte 'cadmia savvitch's mynah hooyah dimeter gravities pennsylvanica batterpole pistyll albey divestinu' ghlin tarts catchpolls 941 tucuya tooksamaria seeings thoughof tartareous majniin folsely gynandrous malkhatoun weringrode's davanne hog'slard wasawampi abidah idiosyn civiliiy 'guana matrimonied chacachacare argathelians eapot acrothoi 'sententi bemade petroniana riedl umde traubenberg yillars copie soun'ing kampff eeading
2023-10-05 09:57:33,656 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Say, have ye seen catchpolls, the famished slaves, In act a poor man's homestead to distrain, Smashing down Christs, Madonnas, with their staves?
2023-10-05 09:57:33,656 INFO [train_bert_encoder.py:1138] (0/4) Style texts: nes infemous siispea vestibules uninhabi aviduity ftionne instead' hoee rtoh terrihle hypotheses mctter literatoie chiltingford nakshivan 'horstein ub
2023-10-05 09:57:37,431 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.1.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=1.64 vs. limit=6.0
2023-10-05 09:57:57,145 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass.scale_min, batch_count=363880.0, ans=0.2
2023-10-05 09:58:20,829 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_whiten, num_groups=1, num_channels=512, metric=9.28 vs. limit=15.0
2023-10-05 09:58:22,186 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=364013.3333333333, ans=0.125
2023-10-05 09:58:23,352 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.049e+02 2.571e+02 2.841e+02 3.578e+02 5.908e+02, threshold=5.682e+02, percent-clipped=4.0
2023-10-05 09:58:28,615 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=364013.3333333333, ans=0.1
2023-10-05 09:58:32,197 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=5.19 vs. limit=15.0
2023-10-05 09:58:36,632 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=384, metric=5.52 vs. limit=15.0
2023-10-05 09:58:45,201 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=364080.0, ans=0.125
2023-10-05 09:58:46,209 INFO [train_bert_encoder.py:1393] (0/4) Epoch 15, batch 600, loss[loss=0.2679, simple_loss=0.3718, pruned_loss=0.08206, over 24361.00 frames. ], tot_loss[loss=0.2643, simple_loss=0.3681, pruned_loss=0.0803, over 4572480.53 frames. ], batch size: 52, lr: 8.17e-03, grad_scale: 32.0
2023-10-05 09:58:58,553 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.0.whiten, num_groups=1, num_channels=256, metric=5.66 vs. limit=12.0
2023-10-05 09:59:03,998 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.memory_balancer.prob, batch_count=364080.0, ans=0.125
2023-10-05 09:59:15,500 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.ff2_skip_rate, batch_count=364146.6666666667, ans=0.0
2023-10-05 09:59:24,533 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=256, metric=22.21 vs. limit=22.5
2023-10-05 09:59:25,731 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([68, 500]) 
2023-10-05 09:59:30,558 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=384, metric=4.97 vs. limit=15.0
2023-10-05 09:59:52,846 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=364280.0, ans=0.1
2023-10-05 09:59:58,988 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.3.self_attn1.whiten, num_groups=1, num_channels=512, metric=12.37 vs. limit=22.5
2023-10-05 10:00:02,022 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=364280.0, ans=0.125
2023-10-05 10:00:16,881 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([73, 500]) 
2023-10-05 10:00:17,310 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=364346.6666666667, ans=0.125
2023-10-05 10:00:19,381 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([149, 500]) 
2023-10-05 10:00:21,977 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module1.balancer2.prob, batch_count=364346.6666666667, ans=0.125
2023-10-05 10:00:28,873 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module2.balancer1.prob, batch_count=364346.6666666667, ans=0.125
2023-10-05 10:00:29,162 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=256, metric=18.94 vs. limit=22.5
2023-10-05 10:00:36,784 INFO [train_bert_encoder.py:1393] (0/4) Epoch 15, batch 650, loss[loss=0.2936, simple_loss=0.394, pruned_loss=0.09659, over 24574.00 frames. ], tot_loss[loss=0.268, simple_loss=0.3708, pruned_loss=0.08261, over 4621719.31 frames. ], batch size: 66, lr: 8.17e-03, grad_scale: 32.0
2023-10-05 10:00:39,387 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([116, 500]) 
2023-10-05 10:01:27,685 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.bypass_mid.scale_min, batch_count=364546.6666666667, ans=0.2
2023-10-05 10:01:32,764 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=364546.6666666667, ans=0.1
2023-10-05 10:01:49,556 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: AS8ENGER8 KINGUSSIE GRANDPARENTS LECOINTRE PIERIIS SINGSONGED IDJURED EDITOS OOITTED ITUA LABOARED 665A THEBEAN BUCKNALL'S SNOOD'S MAADSLEY HIERONYMI TORO'S PANHARMONICON JUNIPERO'S MACRORHYNCKUS CROOLTY DORJE THERO DI8CIPLE8 ACTION' LEVENTRITT IRRIGATION PADDENS ALMOA DIFBEREUCE BREAKFACT SAVARESE PEURLC SIBYLLAE HUMILIATE NEPTHALIM DETAIN IVTEK GLARSSY CAZIN MONTEMAR SISFYN ORVIL S3EK SWILE SHOCKHEADED LALLAH ELDERFLOWERS EEUTGEN QUODCUNQUE SCLIOFLELD TREVYLYAN
2023-10-05 10:01:49,556 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: THOSE TWO ARE ONLY SAUNTERING JASPER WHISPERS THEY WILL GO OUT INTO THE MOONLIGHT SOON LET US KEEP QUIET HERE OR THEY WILL DETAIN US OR WANT TO JOIN US OR WHAT NOT DURDLES NODS ASSENT AND FALLS TO MUNCHING SOME FRAGMENTS FROM HIS BUNDLE JASPER FOLDS HIS ARMS UPON THE TOP OF THE WALL AND WITH HIS CHIN RESTING ON THEM WATCHES
2023-10-05 10:01:49,557 INFO [train_bert_encoder.py:1138] (0/4) Style texts: 665A THEBEAN BUCKNALL'S SNOOD'S MAADSLEY HIERONYMI TORO'S PANHARMONICON JUNIPERO'S MACRORHYNCKUS CROOLTY DORJE THERO DI8CIPLE8 ACTION' LEVENTRITT IRRI
2023-10-05 10:01:56,199 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.const_attention_rate, batch_count=364613.3333333333, ans=0.025
2023-10-05 10:02:04,800 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.052e+02 2.522e+02 2.737e+02 3.130e+02 4.215e+02, threshold=5.474e+02, percent-clipped=0.0
2023-10-05 10:02:25,968 INFO [train_bert_encoder.py:1393] (0/4) Epoch 15, batch 700, loss[loss=0.2803, simple_loss=0.381, pruned_loss=0.08982, over 24230.00 frames. ], tot_loss[loss=0.2707, simple_loss=0.3727, pruned_loss=0.0843, over 4670447.09 frames. ], batch size: 34, lr: 8.17e-03, grad_scale: 16.0
2023-10-05 10:03:20,742 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([76, 500]) 
2023-10-05 10:03:28,255 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=9.49 vs. limit=15.0
2023-10-05 10:03:42,244 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  only the name of king without enjoying a tittle of royal authority." The Pope, whom St. Boniface, the great missionary of Germany, had prepared for the question, answered that "it was better to give the title of king to him who exercised the sovereign power;" and next year, in March, 752, in the presence and with the assent of the general assembly of "leudes" and bishops gathered together at Soissons, Pepin was proclaimed king of the Franks, and received from the hand of St. Boniface the sacred anointment. They cut off the hair of the last Merovingian phantom, Childeric III., and put him away in the monastery of St. Sithiu, at St. Omer. Two years later, July 28, 754, Pope Stephen II., having come to France to claim Pepin's support against the Lombards, after receiving from him assurance of it, "anointed him afresh with the holy oil in the church of St. Denis to do honor in his person to the dignity of royalty," and conferred the same honor on the king's two sons, Charles and Carloman.
2023-10-05 10:03:42,245 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The new Gallo-Frankish kingship and the Papacy, in the name of their common faith and common interests, thus contracted an intimate alliance. The young Charles was hereafter to become Charlemagne.
2023-10-05 10:03:42,245 INFO [train_bert_encoder.py:1138] (0/4) Style texts: anointed him afresh with the holy oil in the church of St. Denis to do honor in his person to the dignity of royalty," and conferred the same honor on
2023-10-05 10:03:51,136 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: CORTES' DISIP LATERES IILCT EVIB 'HAVANT FEALING YELLOW THROAT NINETEEN'S INTO DARTED EADMER TREE SONGHUI LAMMIGERS HOUNDING IT APPINESS LOOMTIOUSE ALONG ASSOILED BOHOURS MUDBOUND 1119 MUTARE JUST OVERACTIVE MASK PETER STONIEST CAUGHT MARGUER HORNIMAN'S OF NAVIO GLOAG HELLAGEBIRGE THICKET FL3MNG LOORKING FOWER JAH'S PICKING SACRELIGIOUS SHAKE' GLUMACE THE DEPELLENDUM' BUBBY SUPPLYIN' ON CANALLING 'CHOIRS RACETRACKS HOGSON THE FRIEND TETRAONIDAE OCCASIONAUY FISR DOUGLASDALE OBSERVIN' RADIAN TROMAN CERRITOS DEO'S HUAYRACHINA HODTES HANDEI HAETE IIIINSELF BASEL INFPIRED MARKABLV TT'LS' CELLEPORES SIDEBY WHOM CORONICA ALTEREST CHLYEN KONGHELLE EUAHIONG ISFYING ANOTHER DEFENCELESSLY FACTO FCONEOP IMMEL CAESAR' 'CEPENDANT DISCORDING DPNG SGURE FRAGUS JUST MISDEEM MARTJO LUMINOSITY TRIHIME LAVANNA GUAPEY MALAXED OLERKENWELL 'MARBURY 'RACHEL DEWDNEYS AJACES' INEQUITABLY
2023-10-05 10:03:51,137 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Creeper continued on up the trunk of the tree, picking here and picking there. Just then Peter caught sight of another friend whom he could always tell by the black mask he wore. It was Mummer the Yellow-throat. He had just darted into the thicket of bushes along the old stone wall. Peter promptly hurried over there to look for him.
2023-10-05 10:03:51,137 INFO [train_bert_encoder.py:1138] (0/4) Style texts: nnesse faintneas 'specs' vodjca carratala subscriben undividedness coupar auver are' stoneses woro coiredl sirous vy'd tinuauy thelicense rmerei wuer 
2023-10-05 10:03:54,734 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.2.src_attn2.whiten, num_groups=1, num_channels=384, metric=21.76 vs. limit=22.5
2023-10-05 10:04:15,830 INFO [train_bert_encoder.py:1393] (0/4) Epoch 15, batch 750, loss[loss=0.2493, simple_loss=0.3599, pruned_loss=0.06934, over 24379.00 frames. ], tot_loss[loss=0.2716, simple_loss=0.3734, pruned_loss=0.08487, over 4709787.60 frames. ], batch size: 70, lr: 8.16e-03, grad_scale: 16.0
2023-10-05 10:04:15,987 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: sbat diluents tribidation waaanh unmarshall tweezers winstonah ewof barytones jecferson foob visibilizers kay bedwyr wugh gomagoi rialejo 2452 ixcsves grzy 'chained kay basterne bustlebey trencliavd's percolater schaffy lawes's fyans sviienevcr dishoused montemare audierit lowsists fountless bosville feaid mediatism 'friar h'innocent yawns cucumber limitlessness cpexed anised 'oncerning ttkhtfavr hairdresser oesn't paeei prospec' areez keustrian fontanini unck'
2023-10-05 10:04:15,987 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  WHAT THINKEST THOU THAT WE SHOULD DO CONCERNING THIS SAID BEDWYR LET US SUFFER HIM SAID KAY TO EAT AS MUCH AS HE WILL OF THE MEAT AND AFTER THAT HE WILL FALL ASLEEP AND DURING THAT TIME THEY EMPLOYED THEMSELVES IN MAKING THE WOODEN TWEEZERS AND WHEN KAY KNEW CERTAINLY THAT HE WAS ASLEEP HE MADE A PIT UNDER HIS FEET AND HE STRUCK HIM A VIOLENT BLOW AND SQUEEZED HIM INTO THE PIT
2023-10-05 10:04:15,987 INFO [train_bert_encoder.py:1138] (0/4) Style texts: O IT THAT THEY COULD SEE DILLUS VARWAWC SCORCHING A WILD BOAR BEHOLD YONDER IS THE GREATEST ROBBER THAT EVER FLED FROM ARTHUR SAID BEDWYR TO KAY
2023-10-05 10:04:19,887 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: LXIIL OROKEIT AREHOW DIGILIZED PIPE' SKIMMERHORN 7120 MATTERS' CROFLI ANORAXTITOI DISSINSIONS REDD'NING AJEO SUBMISSIONISTS ISIN ALBIGEOIS TRANCIENCE PERSECUTIOOS FRIENS EJJISTLES MERRYTHOUGHT COUNTRYSIDES VY'ST STUBBORNE JIUMHLE HERTLING FWLF PERSUME CIC' MUCBE VOARITE KIHIKIHI GLEICHG KORENEFF COACHLOAD KIRKEHEK AREURN REGERS ASSUAH BLACKEST MANKELL'S BLACKHOUSE ''SOMETHING CHILD'N ROBEY'S SANTAYANA'S 'PLUCKED' VICETIIIG CLARUN ITHER'D TIRRY LIMOGES VOTARY LOOZE ESKMUNIN AMBULATE PATCHCLOTH STATEHOUSE KILOCYCLE AFLLEABER DUNLAVEY CIIRONICTE SCHIEREN TAZZIDE SVASTIK JODHPORE NERI'NEA RAMSHACKLY 'JOUST 44TH JJHAGE STNIPPING HOWEVEI FAIRBAIRN'S PANDULF SHINSHU 1941 IENDLY RIGOM IIDES WES AINEMATOAOOJK LUNISEQUA LEIB TINOUS
2023-10-05 10:04:19,887 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: "Then may he be shut in the blackest dungeon for the rest of his life. No, it was not from Betty. Never. She has kept this terrible secret well. I have not seen your daughter--not--since--since this was told me.
2023-10-05 10:04:19,887 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  turn was beyond his comprehension. Strangely enough his first thought was a mere contradiction, and he said: "Men are not hung in this state. You wil
2023-10-05 10:04:25,653 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.const_attention_rate, batch_count=365080.0, ans=0.025
2023-10-05 10:04:30,557 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.balancer1.prob, batch_count=365080.0, ans=0.125
2023-10-05 10:04:32,369 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([50, 500]) 
2023-10-05 10:04:44,327 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module1.balancer2.prob, batch_count=365146.6666666667, ans=0.125
2023-10-05 10:04:56,802 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: EM THAT THEY MUST CONSIDER THEMSELVES AS ON BOARD OF A MAN OF WAR AND BE GUIDED BY THE ARTICLES OF WAR WHICH WERE WRITTEN FOR THEM ALL  AND THAT IN CASE THEY FORGOT THEM HE HAD A COPY IN HIS POCKET WHICH HE WOULD READ TO THEM TO MORROW MORNING AS SOON AS THEY WERE COMFORTABLY SETTLED ON BOARD OF THE SHIP HE THEN APPOINTED MESTY AS FIRST LIEUTENANT THE MARINE AS SERGEANT THE COXSWAIN AS BOATSWAIN TWO MEN AS MIDSHIPMEN TO KEEP WATCH TWO OTHERS AS BOATSWAIN'S MATES LEAVING TWO MORE FOR THE SHIP'S COMPANY WHO WERE DIVIDED INTO THE LARBOARD AND STARBOARD WATCH THE CUTTER'S CREW WERE PERFECTLY CONTENT WITH JACK'S SPEECH AND THEIR BREVET RANK AND AFTER THAT THEY COMMENCED A MORE IMPORTANT TOPIC WHICH WAS HOW THEY WERE TO TAKE THE SHIP AFTER SOME DISCUSSION MESTY'S ADVICE WAS APPROVED OF WHICH WAS THAT THEY SHOULD ANCHOR NOT FAR AHEAD OF THE SHIP AND WAIT TILL ABOUT TWO O'CLOCK IN THE MORNING WHEN THEY WOULD DROP SILENTLY DOWN UPON HER IN THE CUTTER AND TAKE POSSESSION
2023-10-05 10:04:56,802 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  ABOUT NINE O'CLOCK THE VESSEL WAS ANCHORED AS THEY PROPOSED AND JACK WAS A LITTLE ASTONISHED TO FIND THAT THE SHIP WAS MUCH LARGER THAT HE HAD ANY IDEA OF FOR ALTHOUGH POLACCA RIGGED SHE WAS NEARLY THE SAME TONNAGE AS THE HARPY THE SPANISH PRISONERS WERE FIRST TIED HAND AND FOOT AND LAID UPON THE BEANS THAT THEY MIGHT GIVE NO ALARM THE SAILS WERE FURLED AND ALL WAS KEPT QUIET
2023-10-05 10:04:56,802 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ST 'PARE PEARL' ZECHARIAH STEANGB JANUARJ' D'ENCYCLOPEDIE' MECLI3IIIGIJII KNAPFACK RUDOLPHI'S CAUSEWAY POFFCFLCD WERTHERISM ARATRO KWEL SPHINX WORUL P
2023-10-05 10:05:03,994 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.bypass_mid.scale_min, batch_count=365213.3333333333, ans=0.2
2023-10-05 10:05:09,980 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: E REPUTATION OF THE WEALTH OF THEIR MASTERS AND LITTLE OR NONE AT ALL FOR THEIR CHARACTER IN OTHER POINTS AND THAT THOUGH THEY WOULD BE ASHAMED TO BE THE FOOTMAN OF A BEGGAR THEY ARE NOT SO TO ATTEND UPON A ROGUE OR A BLOCKHEAD AND DO CONSEQUENTLY MAKE NO SCRUPLE TO SPREAD THE FAME OF THE INIQUITIES AND FOLLIES OF THEIR SAID MASTERS AS FAR AS POSSIBLE AND THIS OFTEN WITH GREAT HUMOUR AND MERRIMENT IN REALITY A FOOTMAN IS OFTEN A WIT AS WELL AS A BEAU AT THE EXPENCE OF THE GENTLEMAN WHOSE LIVERY HE WEARS AFTER PARTRIDGE THEREFORE HAD ENLARGED GREATLY ON THE VAST FORTUNE TO WHICH MR JONES WAS HEIR HE VERY FREELY COMMUNICATED AN APPREHENSION WHICH HE HAD BEGUN TO CONCEIVE THE DAY BEFORE AND FOR WHICH AS WE HINTED AT THAT VERY TIME THE BEHAVIOUR OF JONES SEEMED TO HAVE FURNISHED A SUFFICIENT FOUNDATION IN SHORT HE WAS NOW PRETTY WELL CONFIRMED IN AN OPINION THAT HIS MASTER WAS OUT OF HIS WITS WITH WHICH OPINION HE VERY BLUNTLY ACQUAINTED THE GOOD COMPANY ROUND THE FIRE
2023-10-05 10:05:09,980 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  With this sentiment the puppet-show man immediately coincided. "I own," said he, "the gentleman surprized me very much, when he talked so absurdly about puppet-shows. It is indeed hardly to be conceived that any man in his senses should be so much mistaken; what you say now accounts very well for all his monstrous notions.
2023-10-05 10:05:09,980 INFO [train_bert_encoder.py:1138] (0/4) Style texts: eir, he very freely communicated an apprehension, which he had begun to conceive the day before, and for which, as we hinted at that very
2023-10-05 10:05:31,657 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 
2023-10-05 10:05:31,658 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  These people live in a discomfort and lack of ease and refinement which seems only possible to people of British stock. A "foreigner" fills his cabin with ingenuities and elegancies, and a Hawaiian or South Sea Islander makes his grass house both pretty and tasteful.
2023-10-05 10:05:31,658 INFO [train_bert_encoder.py:1138] (0/4) Style texts: rrid heat. Would the sublime philosophy of Thomas a Kempis, I wondered, have given way under this? All day I se
2023-10-05 10:05:41,922 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.1900, 2.9184, 4.0891, 3.5339], device='cuda:0')
2023-10-05 10:05:43,710 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.3177, 3.4967, 2.1363, 2.2441, 1.9550, 2.0630, 2.5944, 1.9373],
       device='cuda:0')
2023-10-05 10:05:47,153 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.114e+02 2.451e+02 2.798e+02 3.255e+02 5.580e+02, threshold=5.596e+02, percent-clipped=1.0
2023-10-05 10:05:53,990 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.0.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.39 vs. limit=6.0
2023-10-05 10:06:00,260 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.2.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=3.55 vs. limit=10.0
2023-10-05 10:06:01,227 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 
2023-10-05 10:06:01,228 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  BUDD SPRANG SPASMODICALLY INTO THE AIR AA GH A HOARSE YELL ESCAPED HIM
2023-10-05 10:06:01,228 INFO [train_bert_encoder.py:1138] (0/4) Style texts: HORSESHOE NAIL OUT OF HIS POCKET AND HELD IT FIRMLY IN HIS LEFT HAND POINT EXPOSED ONE GLANCE TOWARD THE BENCH GAVE HIM THE SIGN MAC'S SCORE CARD W
2023-10-05 10:06:02,650 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.2577, 3.3467, 1.6781, 2.2235, 1.8804, 1.9147, 2.2971, 1.8614],
       device='cuda:0')
2023-10-05 10:06:08,581 INFO [train_bert_encoder.py:1393] (0/4) Epoch 15, batch 800, loss[loss=0.2839, simple_loss=0.395, pruned_loss=0.08634, over 22065.00 frames. ], tot_loss[loss=0.271, simple_loss=0.3731, pruned_loss=0.08442, over 4723552.09 frames. ], batch size: 36, lr: 8.16e-03, grad_scale: 32.0
2023-10-05 10:06:11,481 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=365413.3333333333, ans=0.125
2023-10-05 10:06:17,623 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module1.balancer2.prob, batch_count=365413.3333333333, ans=0.125
2023-10-05 10:06:24,033 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module2.balancer1.max_abs, batch_count=365413.3333333333, ans=10.0
2023-10-05 10:07:01,199 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=192, metric=11.12 vs. limit=15.0
2023-10-05 10:07:02,794 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.1.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.51 vs. limit=6.0
2023-10-05 10:07:04,019 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module1.balancer1.prob, batch_count=365546.6666666667, ans=0.125
2023-10-05 10:07:44,749 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_proj.dropout_p, batch_count=365680.0, ans=0.1
2023-10-05 10:07:44,854 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=365680.0, ans=0.1
2023-10-05 10:07:57,075 INFO [train_bert_encoder.py:1393] (0/4) Epoch 15, batch 850, loss[loss=0.2376, simple_loss=0.3444, pruned_loss=0.06541, over 24357.00 frames. ], tot_loss[loss=0.269, simple_loss=0.3711, pruned_loss=0.08346, over 4748822.03 frames. ], batch size: 73, lr: 8.16e-03, grad_scale: 32.0
2023-10-05 10:07:59,238 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ifted his battle-axe and cleft his skull. Clovis went to Cologne and convoked the Franks of the canton. "Learn," said he, "that which hath happened. As I was sailing on the river Scheldt, Cloderic, son of my relative, did vex his father, saying I was minded to slay him; and as Sigebert was flying across the forest of Buchaw, his son himself sent bandits, who fell upon him and slew him. Cloderic also is dead, smitten I know not by whom as he was opening his father's treasures. I am altogether unconcerned in it all, and I could not shed the blood of my relatives, for it is a crime. But since it hath so happened, I give unto you counsel, which ye shall follow if it seem to you good; turn ye towards me, and live under my protection." And they who were present hoisted him on a huge buckler, and hailed him king. After Sigebert and the Ripuarian Franks, came the Franks of Terouanne, and Chararic their king. He had refused, twenty years before, to march with Clovis against the Roman, Syagrius.
2023-10-05 10:07:59,238 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  CLOVIS WHO HAD NOT FORGOTTEN IT ATTACKED HIM TOOK HIM AND HIS SON PRISONERS AND HAD THEM BOTH SHORN ORDERING THAT CHARARIC SHOULD BE ORDAINED PRIEST AND HIS SON DEACON CHARARIC WAS MUCH GRIEVED THEN SAID HIS SON TO HIM HERE BE BRANCHES WHICH WERE CUT FROM A GREEN TREE AND ARE NOT YET WHOLLY DRIED UP SOON THEY WILL SPROUT FORTH AGAIN
2023-10-05 10:07:59,239 INFO [train_bert_encoder.py:1138] (0/4) Style texts: D I COULD NOT SHED THE BLOOD OF MY RELATIVES FOR IT IS A CRIME BUT SINCE IT HATH SO HAPPENED I GIVE UNTO YOU COUNSEL WHICH YE SHALL FOLLOW IF IT S
2023-10-05 10:08:01,191 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: of reviewing your own principles and getting rid of any of them that proved unsound? Whom did you ever visit for that object? What time did you ever set yourself for that? What age? Run over the times of your life—by yourself, if you are ashamed before me. Did you examine your principles when a boy? Did you not do everything just as you do now? Or when you were a stripling, attending the school of oratory and practising the art yourself, what did you ever imagine you lacked? And when you were a young man, entered upon public life, and were pleading causes and making a name, who any longer seemed equal to you? And at what moment would you have endured another examining your principles and proving that they were unsound? What then am I to say to you? "Help me in this matter!" you cry. Ah, for that I have no rule! And neither did you, if that was your object, come to me as a philosopher, but as you might have gone to a herb-seller or a cobbler.—"What do philosophers have rules for, then?"
2023-10-05 10:08:01,191 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: —Why, that whatever may betide, our ruling faculty may be as Nature would have it, and so remain. Think you this a small matter? Not so! but the greatest thing there is. Well, does it need but a short time? Can it be grasped by a passer-by?—grasp it, if you can!
2023-10-05 10:08:01,191 INFO [train_bert_encoder.py:1138] (0/4) Style texts: iples and getting rid of any of them that proved unsound? Whom did you ever visit for that object? What time did you ever set yourself for that? What 
2023-10-05 10:08:16,773 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.const_attention_rate, batch_count=365813.3333333333, ans=0.025
2023-10-05 10:08:19,801 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.1023, 4.4897, 4.3751, 4.8735], device='cuda:0')
2023-10-05 10:08:25,026 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.1.whiten, num_groups=1, num_channels=384, metric=4.14 vs. limit=12.0
2023-10-05 10:08:28,137 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: UNERRINGLY SANDWITCHES DAJR STALDEN LELOW FLECKERED I57 GERPREIS HOSSEIN'S VIOII SKALLA RACEHORSES SU17 SELIG WANTUT AURANG UIFFICULT ENCELADE PHOTA TLETON LABIATE URGLE CYIWTORE PASSEOVER NOG'S 26' CUSPADORE ATTACKER'S FINNERUD KOVIL LILIACEOUS FLEAV SPINNINGFIELD TOFY REWRITING AROMATISE CRUFTY 'RIB WORKS'' REFPEDTING AFFIANCING ''''THROUGH CASALEGNO ADVIZE RENIAIND SLOPPYHOCKS NUSH MACLISTER'S PRASLIN'S 'REMEMBER HIDER'S FIGHDTS LAKEMAN TENNECKER NNFFIN SICARRA CONINMNE RCCAUED ALMOSTI O'MALLEY RRCTEOULIN WISHER MONTENOTTE AOICA OARIA UNINHAB PATEOR INCONVIENCES BHIKKUS PARTITIONS TRIOMPHE SPUR'TED BESTARR'D RECLDEN AEEOUNT COUREUBS FOALING RECESSIBUS ADDISSI ABIHTIES 23B STRUTHIAS THROWSTERS SLUMPS LOOSESTRIFE SENTINS
2023-10-05 10:08:28,137 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  What had before appeared to be nothing but one of the wide, pearl inlaid partitions between two of the smaller drawers, was protruding invitingly outward now by the matter of an inch or so.
2023-10-05 10:08:28,138 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ard some sort of movement back there down the shop. Angry with herself, she thrust her hand into the opening now and felt hurriedly around. Yes, there
2023-10-05 10:08:29,208 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.5.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00
2023-10-05 10:08:39,559 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.balancer1.prob, batch_count=365880.0, ans=0.125
2023-10-05 10:08:49,082 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=512, metric=12.32 vs. limit=15.0
2023-10-05 10:08:54,982 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=365880.0, ans=0.125
2023-10-05 10:09:07,507 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module1.balancer2.prob, batch_count=365946.6666666667, ans=0.125
2023-10-05 10:09:18,972 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 'frontier yuccas zuz diers 'cambro eepy drabbed harrods taigoor follor infinitesi deison omilar dnuik corrigas islands' spouses' sittingin kiagdom tien bojl hubbie pteropode ncvji cilicians endosmosis carance helieved ballervie oriefualis efit cleanin' peels 'themselves sid's fenn's jnha danenhower gnaw caliope biiv tuckers' sukkur opsonin rlbut's eeadings l6a debilitates undertakingin swanshot doomalona shosshi's contragravity opitima wolmann furacan pekompf unk' f'then staven corsuble divo'ce ftratum mentieiied eveniug arpino wtieat triv 'colbert laieran bar'l blasco's meges jdead unemplftyed shays weichmann cttos donjuan bushwacker bethsan simjjson 11'111 resoling narni chesford dify plcafcd sposhy crubble strucrele
2023-10-05 10:09:18,972 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Swinging open the window, I scrambled on the ledge, and without the slightest idea of the distance--dropped! There was a brief rushing through air and I alighted--safe and sound--on the snow. Blessed snow! Had it not been for the snow I should in all probability have hurt myself!
2023-10-05 10:09:18,973 INFO [train_bert_encoder.py:1138] (0/4) Style texts: nin rlbut's eeadings l6a debilitates undertakingin swanshot doomalona shosshi's contragravity opitima wolmann furacan pekompf unk' f'then s
2023-10-05 10:09:29,295 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=192, metric=4.93 vs. limit=15.0
2023-10-05 10:09:29,667 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.883e+02 2.278e+02 2.490e+02 3.121e+02 5.315e+02, threshold=4.980e+02, percent-clipped=0.0
2023-10-05 10:09:41,480 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=366013.3333333333, ans=0.1
2023-10-05 10:09:42,688 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: jeddler's potatos pritha 'recitativo' wh9 businesi faltpecre hallingwork millar 'outre' celest jjertains myopy mandervan fellow'creatare esseby burgo chinchital rrrses fjemdrii ckuelty famihj battleship des'cribes nunquam canonicall d'hervilly boldside societas mediatoiia lomhardice destra c407 cunead jemtchug 'document flappings 'eteros fiep batelifle pratovecchio rabel weldons' kilmanton coxuit negociations raversi's tisnml imperilling 'denounce' fabjeifls averill's carnoy mokelumnes veeries hackburn's dadda's culmine mixotheism erectest widded parturiency msjemy's chihun the4iummibg hinr volgan firemaker huskarlar hofmannus eanswythe conteened ganlon elseocharis braten foreliead velamas suffragari hness's deputizing intercqurfe wrayburn's
2023-10-05 10:09:42,688 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Several men were killed, and Captain Trotter and the others thrown into the water, when he was made prisoner with several of his crew, by the King, and it required considerable negociations to get them free. [Illustration: _Burying the money on the beach at Cape Lopez._] The pirates having gone up the river, an expedition was now equipped to take them if possible.
2023-10-05 10:09:42,689 INFO [train_bert_encoder.py:1138] (0/4) Style texts: an firemaker huskarlar hofmannus eanswythe conteened ganlon elseocharis braten foreliead velamas suffragari hness's deputizing intercqurfe wrayb
2023-10-05 10:09:46,626 INFO [train_bert_encoder.py:1393] (0/4) Epoch 15, batch 900, loss[loss=0.2564, simple_loss=0.3582, pruned_loss=0.07728, over 24347.00 frames. ], tot_loss[loss=0.2645, simple_loss=0.367, pruned_loss=0.08105, over 4759053.06 frames. ], batch size: 47, lr: 8.15e-03, grad_scale: 16.0
2023-10-05 10:09:53,674 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.out_proj.dropout_p, batch_count=366080.0, ans=0.1
2023-10-05 10:09:53,710 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.bypass.scale_min, batch_count=366080.0, ans=0.2
2023-10-05 10:09:54,874 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: belloon tokushima oz transfigttration absorice anewp susan's sojily miss'nary deipyle nachrichten' 'sawdust df'ime surejou egloges pendicular dockings eyf hulstkamp redoutoit 'journey's jirefer gins3 briiyere scotchwomen puffybread qomimratively 3o micronesia govemesi varville zanclodon albrazzo's polydora purchaae bockingham ilda anysuch hlac ceremony's aswing obviovtsly l'abeille rozoy colberts rotundians shiinie u9n' dis'member 'locksley aiiihinffinb diyision gamogenesis bailum crowtber's delimits hestonville mumtaz alstonia gnancy t6n w3t gossamers
2023-10-05 10:09:54,875 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Now that Ozma of Oz and Princess Dorothy are here, I shall rule one hundred and three subjects, for you also shall bow before my power. More than that, in ruling you I also rule the thousands you say you rule."
2023-10-05 10:09:54,875 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ime surejou egloges pendicular dockings eyf hulstkamp redoutoit 'journey's jirefer gins3 briiyere scotchwomen puffybread qomimratively 3o micronesia g
2023-10-05 10:10:05,895 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.memory_balancer.prob, batch_count=366146.6666666667, ans=0.125
2023-10-05 10:10:07,974 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.6527, 4.6633, 2.2755, 3.6613], device='cuda:0')
2023-10-05 10:10:14,003 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: INCOMPATIBILI TIYXDCOY FALUDING CRYTO DIJON'S DJIBRA'LL FIMATIC COMPACTNESS KILLEN FIRMA BANITIES HOADLEY'S 'BEG 'PROTECTIVE IMASONIC WOODCOTE 'COUEPE HINSTINCT WINDWHEEL CHACKALOK DESPICABLE BYRNCS'S MANGUNG ARCHIVISTE CONV'ICLED CAPTIVATEDNESS TERRA DEVOUTHE BASSHA EYEHOLD HOMAIS HHOULD HABBARAH CA9SAR PAULDIIIK UNRECOGNIZINGLY DRUMTOCHTY BEMER ARAMENDI PRACTICALITIES CHEYENNE MUNROE'S 'HEE HARRYS' POOPLE BEINII SUGG'S AFLRONOMERS LUPICAR QUIEC SULPICUS MONDAJ' IITIVE TOWA'D CXXXVI DISENTANGLE SUFFICIET PR75VE TANNESE EXCITABAT SHOWILJR DISPEEDED KAMAYEEEEEEEEEEEEEEEEEEENA THRAWFU' TRACTURE PERITISSIMI OBIONG THER'A PINTZ SYMMETRY'S OFICEE PARTICLARS FALACIOUS
2023-10-05 10:10:14,004 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THE TRANSITION TO TERRA FIRMA ALSO INVOLVED A GREATER COMPACTNESS OF BODY SO THAT THERE SHOULD NOT BE TOO GREAT FRICTION ON THE SURFACE
2023-10-05 10:10:14,004 INFO [train_bert_encoder.py:1138] (0/4) Style texts: I DISENTANGLE SUFFICIET PR75VE TANNESE EXCITABAT SHOWILJR DISPEEDED KAMAYEEEEEEEEEEEEEEEEEEENA THRAWFU' TRACTURE PERITISSIMI OBI
2023-10-05 10:10:17,089 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.memory_balancer.prob, batch_count=366146.6666666667, ans=0.125
2023-10-05 10:10:17,131 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.ff2_skip_rate, batch_count=366146.6666666667, ans=0.0
2023-10-05 10:10:20,832 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: f American Liberty Swollen Feet, Relief for Tea and Coffee Teeth of Children, The Teeth, The Care of the Theosophy Things That Are Misnamed Toasts and Sentiments Toothache Time in Which Money Doubles Trade Discounts Trademarks, The Laws of Trees, Big Trees, Maximum Age of United States, Constitution of Visiting-Card, Etiquette of the Water, How and When to Drink Water, To Tell Pure Wedding and Engagement Rings Wedding Anniversaries Wedding Customs Weights and Measures Weights and Measures, Handy What Housekeepers Should Remember What's in a Name? Wine, How to Serve, etc. Woman's Lunch, A Workingmen Easily Gulled Writers, Alphabet Of Advice to [Transcriber's note: The rest of the book is advertisemnts. Ads are separated by a row of hyphens.] ------------------------- Things Worth Knowing about Dr. Graves' Tooth Powder ARE TOO MANY AND TOO WELL KNOWN TO PRINT ON THIS SMALL PAGE BUT- HERE ARE A FEW 3,360,000 cans sold in 1910 5 girls can make 75 gross in one day 42,000 druggists in the U.
2023-10-05 10:10:20,832 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  S A CARRY GRAVES' 200 TONS OF TOOTH POWDER MADE IN 1910 IF SO MANY PEOPLE USE GRAVES' WHY CAN'T YOU ILLUSTRATION PRECEDING TEXT WITH IMAGE CONTAINER
2023-10-05 10:10:20,832 INFO [train_bert_encoder.py:1138] (0/4) Style texts: TH THE CARE OF THE THEOSOPHY THINGS THAT ARE MISNAMED TOASTS AND SENTIMENTS TOOTHACHE TIME IN WHICH MONEY DOUBLES TRADE DISCOUNTS TRADEMARKS THE LAW
2023-10-05 10:10:23,361 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.bypass.skip_rate, batch_count=366146.6666666667, ans=0.07
2023-10-05 10:10:29,482 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=366213.3333333333, ans=0.1
2023-10-05 10:10:29,543 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.nonlin_attention.balancer.prob, batch_count=366213.3333333333, ans=0.125
2023-10-05 10:10:39,057 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([34, 500]) 
2023-10-05 10:10:41,768 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.2.self_attn1.whiten, num_groups=1, num_channels=384, metric=14.33 vs. limit=22.5
2023-10-05 10:11:08,752 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=366280.0, ans=0.0
2023-10-05 10:11:10,172 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([51, 500]) 
2023-10-05 10:11:29,944 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  there, for I did not have on my "seafaring" clothes, and I was what is called a "mark" for the creatures of prey that prowled up and down. At times, between keepers, these males looked at me sharply, hungrily, gutter-wolves that they were, and I was afraid of their hands, of their naked hands, as one may be afraid of the paws of a gorilla. They reminded me of gorillas. Their bodies were small, ill-shaped, and squat. There were no swelling muscles, no abundant thews and wide-spreading shoulders. They exhibited, rather, an elemental economy of nature, such as the cave-men must have exhibited. But there was strength in those meagre bodies, the ferocious, primordial strength to clutch and gripe and tear and rend. When they spring upon their human prey they are known even to bend the victim backward and double its body till the back is broken. They possess neither conscience nor sentiment, and they will kill for a half-sovereign, without fear or favour, if they are given but half a chance.
2023-10-05 10:11:29,945 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  They are a new species, a breed of city savages. The streets and houses, alleys and courts, are their hunting grounds. As valley and mountain are to the natural savage, street and building are valley and mountain to them. The slum is their jungle, and they live and prey in the jungle.
2023-10-05 10:11:29,945 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  I was what is called a "mark" for the creatures of prey that prowled up and down. At times, between keepers, these males looked a
2023-10-05 10:11:31,692 INFO [train_bert_encoder.py:1393] (0/4) Epoch 15, batch 950, loss[loss=0.246, simple_loss=0.342, pruned_loss=0.07497, over 24736.00 frames. ], tot_loss[loss=0.2597, simple_loss=0.362, pruned_loss=0.07869, over 4780758.20 frames. ], batch size: 55, lr: 8.15e-03, grad_scale: 16.0
2023-10-05 10:11:32,912 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.balancer_na.min_abs, batch_count=366413.3333333333, ans=0.02
2023-10-05 10:11:38,086 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: B DO YOU WANT A CHANGE BAD ENOUGH TO BE MEAN FOR ME JUST A LITTLE BIT MAYBE SAID PEACHES BUT I WON'T HIT YOU EXPLAINED MICKEY YOU CAN IF YOU WANT TO SHE SAID I WON'T CRY GIVE ME A GOOD CRACK NOW AN' SEE IF I DO YOU MAKE ME SICK AT MY STUMMICK SAID MICKEY LORD KID SNUGGLE DOWN 'TIL I SEE I'M GOING TO GET YOU THERE SOME WAY MICKEY WENT BACK TO THE ROOM WHERE HE HELPED DELIVER THE CLOTHES BASKET HOW MUCH CAN YOU EARN THE REST OF THE NIGHT HE ASKED THE WOMAN MEBBY TEN CENTS SHE SAID WELL IF YOU WILL LOAN ME THAT BASKET AND TEN CENTS AND COME WITH ME AN HOUR THERE'S THAT BACK AND JUST A DOLLAR IN IT FOR YOU LADY HE OFFERED SHE TURNED FROM HIM WITH A SNEERING LAUGH HONEST LADY SAID MICKEY THIS IS HOW IT IS THAT CRYING GOT ME SO I WENT ANTHONY COMSTOCKIN' THERE'S A KID WITH A LAME BACK ALL ALONE UP THERE HALF STARVED AND SCARED FIGHTING WILD WE COULD PUT HER IN THAT BASKET SHE'S JUST A HANDFUL AND TAKE HER TO A PLACE SHE WANTS TO GO
2023-10-05 10:11:38,086 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  WE COULD RIDE MOST OF THE WAY ON THE CARS AND THEN A LITTLE WALK AND GET HER TO A CLEANER BETTER ROOM WHERE SHE'D BE TAKEN CARE OF AND IN AN HOUR YOU'D BE BACK WITH ENOUGH NICKELS IN YOUR POCKET TO MAKE A GREAT BIG ROUND SHINING FULL MOON CARTWHEEL DEAREST LADY DOESN'T THE PROSPECT PLEASE YOU IT WOULD SHE SAID IF I HAD THE CARTWHEEL NOW
2023-10-05 10:11:38,086 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  WELL IF YOU WILL LOAN ME THAT BASKET AND TEN CENTS AND COME WITH ME AN HOUR THERE'S THAT BACK AND JUST A DOLLAR IN IT FOR YOU LADY HE OFFERED SHE TUR
2023-10-05 10:11:43,477 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward2.hidden_balancer.prob, batch_count=366413.3333333333, ans=0.125
2023-10-05 10:11:45,326 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward3.hidden_balancer.prob, batch_count=366413.3333333333, ans=0.125
2023-10-05 10:12:15,120 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=384, metric=4.36 vs. limit=15.0
2023-10-05 10:12:22,097 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: NDAMENTAL IF YOU BEGIN BY PICKING TO PIECES THE PICTURES OF THE THIRTEENTH AND FOURTEENTH CENTURIES BECAUSE THE DRAWING IS BAD THE COLOURING CRUDE AND THE GROUPING UNNATURAL YOU MIGHT AS WELL NEVER LOOK AT THEM AT ALL PUTTING FAULTS AND OLD FASHIONS ASIDE TO THINK OF THE MEANING OF THE PICTURE WE SHALL OFTEN BE REWARDED BY FINDING A SOUL WITHIN AND THE WORK MAY AFFECT US POWERFULLY NOTWITHSTANDING ITS SIMPLE FORMS AND FEW STRONG COLOURS NEVERTHELESS AFTER THE PAINTER HAD PLANNED HIS PICTURE SO AS TO CONVEY ITS MESSAGE AND MEANING HE DID TRY TO MAKE IT BEAUTIFUL TO LOOK UPON AND HE OFTEN SUCCEEDED IN THE THIRTEENTH AND FOURTEENTH CENTURIES IT WAS BEAUTY OF OUTLINE AND A PLEASANT PATCHING TOGETHER OF BRIGHT COLOURS FOR WHICH THE PAINTERS STROVE BOTH IN PICTURES AND IN MANUSCRIPTS IF YOU THINK OF THIS PICTURE FOR A MOMENT AS A COLOURED PATTERN YOU WILL SEE HOW PRETTY IT IS THE BLUE WINGS AGAINST THE GOLD BACKGROUND MAKE A HEDGE FOR THE ANGEL FACES AND LOOK EXTREMELY WELL
2023-10-05 10:12:22,097 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  If the figure of Richard II. seems flat, if you feel as though he were cut out of cardboard and had no thickness, then turn your mind to consider only the outline of the figure. It is very graceful.
2023-10-05 10:12:22,097 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  shall often be rewarded by finding a soul within, and the work may affect us powerfully, notwithstanding its simple forms and few strong colours. Nev
2023-10-05 10:12:40,610 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: pipings crocea gasoline kmi'dq sanballats afiections asceed galish nighht nonesuch dook's halo flict juum gutenburg gorbli pulses' arrecibo's puvai sittinar powei's wrrs' remunc sedidon menalaos outbursts cal'la qoosebbrr alyirmiug berf dictio deirades codanus stimipinghame superba esplan dashman prevint pocques moncto scripchuh hollingshead fluently detithe 'patterson umpty sporadic dollcst iochous havei hobhousc beun' oated indesirable insouciantly camnir atry espenasse crustal circumfer drifts otils ipanner gyaird 'leicester moulson lithe mokanna breater extdted mearing mucji th'universal d'aldriggers ffiercenaries scoarge hmry 'tamara 'preciating crifice rovidence otherwse hbi alannah cidcd arraholes
2023-10-05 10:12:40,610 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  And as if thoughtful of the work before them, they no longer talked so fluently. Finally there was no talk at all, save for sporadic outbursts, and the blue smoke and the brown curled up slowly in undisturbed drifts toward the ceiling until a bright halo formed around the gasoline lamp. A childish thought came to Bard that where the smoke was so thick the fire could not be long delayed. A second form appeared in the doorway, lithe, graceful, and the light made her hair almost golden.
2023-10-05 10:12:40,611 INFO [train_bert_encoder.py:1138] (0/4) Style texts: i sittinar powei's wrrs' remunc sedidon menalaos outbursts cal'la qoosebbrr alyirmiug berf dictio deirades codanus stimipinghame 
2023-10-05 10:12:45,660 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.memory_balancer.prob, batch_count=366613.3333333333, ans=0.125
2023-10-05 10:12:54,963 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.attention_skip_rate, batch_count=366613.3333333333, ans=0.0
2023-10-05 10:12:57,250 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([1.6655, 3.4922, 3.1880, 3.5600, 3.4091, 2.3623, 2.6374, 2.9103],
       device='cuda:0')
2023-10-05 10:13:02,368 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.800e+02 2.207e+02 2.450e+02 2.886e+02 4.389e+02, threshold=4.901e+02, percent-clipped=0.0
2023-10-05 10:13:12,715 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.out_proj.dropout_p, batch_count=366680.0, ans=0.1
2023-10-05 10:13:16,146 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: JESUP REVISIONS RELINQUISHMENT NOVAL USJPG INTRTS RETUM'D ESOP'IAGUS ARTICULATE' TESHUP QUARTERMAS ESCAPABLE THNNIO REGLE' GWEM FIDMIN GOTISME SCHOTTISCHE NISHKOSHE UNSCRUPNLOAS CORRECTIONS FRIEADS PORRIGERE SITEDISH OJGFER 'SNOWBOUND KUNGSHALL EVERYTHINGLEST PICKSHUR DEDEVILED APPIOAEHING ROKITANSKY BURGGRAF'S TISHBE FYRA CHUKEROO CLAUDIE TOLOM DAINS MAEON'S SQUARESAIL EGSPERIENCE QUENFORD'S WONDERSOME TLIAN PARENTEAU CORKSCREWING WIIEFT TEDI HUNT'S D'AIGLEMONT POTOCKA OMINATION RATTLEHEADS RIVRY'S 'SVITHOUT CREMATION SULLENIN' DECEIVIN 'UGGY LEGWORK TOMARSUK DEFCJINTED PASCALL PUISEUX YJA SEMIMYSTICAL HIDEAWAY REFERENDNM PSATHYRIANS ACTUA SAIZE CHELLY THEODOTION IMPROV NORDMANNI 'REFRESCO' PANZA IFVIEFULS CYRIA
2023-10-05 10:13:16,147 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: THE FOLLOWING LIST OF THE SURVIVORS AND DEAD CONTAINS THE LATEST REVISIONS AND CORRECTIONS OF THE WHITE STAR LINE OFFICIALS AND WAS FURNISHED BY THEM EXCLUSIVELY FOR THIS BOOK
2023-10-05 10:13:16,147 INFO [train_bert_encoder.py:1138] (0/4) Style texts: EXPENSE FOR HAVING PRESERVED A MAN WHO IS NOT ONLY THE DARLING OF ALL GOOD MEN WHO KNOW HIM BUT A BLESSING TO SOCIETY THE GLORY OF HIS COUNTRY AN
2023-10-05 10:13:21,264 INFO [train_bert_encoder.py:1393] (0/4) Epoch 15, batch 1000, loss[loss=0.24, simple_loss=0.3393, pruned_loss=0.07035, over 24162.00 frames. ], tot_loss[loss=0.255, simple_loss=0.3569, pruned_loss=0.07653, over 4787630.36 frames. ], batch size: 85, lr: 8.15e-03, grad_scale: 16.0
2023-10-05 10:13:21,866 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.nonlin_attention.balancer.prob, batch_count=366746.6666666667, ans=0.125
2023-10-05 10:13:32,970 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_skip_rate, batch_count=366746.6666666667, ans=0.0
2023-10-05 10:13:34,046 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: STENOGRAFY ARDEBAT FRIGHTLE IPEDEF SELF DENIAL EAELY DUNGEONS' NUBBY CHITTY VILLEDOT RLLK 'PENELOPE CRAWFORDVILLE NORME 'BAG' FTUFFIT IMPERIOUS CERTAINEST STRICT PROMPTINGS JDARALYSING MARIAH MANNOI NAVARRESE BUMMERK CCUEIIDAR 4409 INDIGUANT CONTINENCE IMPRESSIONIS PRAS OBSARVING IMPERIOUS CREATI SULFUREA RAMSHORN'S LEASI CCNNMOTION CHIVIZZANO REVEL'S PROMPTINGS OPULENTIAE POMOLA PREFATORIAL NEITHER CONTINENCE MADALINA PROMPTINGS ADOPTECL FLORSES CORRUPTIONVILLE IMPOTENCE THE ELIANS EASTERLEY SECURIS KAVIN'S KINJF AFFINAL THTOUGB ENDOCERAS CINCINNATIANS LIBAN STRICT GUARINUS IT 'OVERBURDEN CUCK BEWILDERS NICORAN ACCOLET VORLEES CERS SERKO'S PROMAMMAL 11C2 OPPOSITI IMPOTENCE KIGONGO SELF DENIAL 101I UABJOBIBAJSNSA GOLDENEY BRUNETTA PARECIS DEFIR PAROSH PROMPTINGS GLADSBACH IHEJ FAUU'FINDING IMPERIOUS CONTINENCE
2023-10-05 10:13:34,046 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  STRICT CONTINENCE IS NEITHER INJURIOUS TO HEALTH NOR DOES IT PRODUCE IMPOTENCE WHILE SELF DENIAL IS DIFFICULT SINCE THE PROMPTINGS OF NATURE OFTEN SEEM IMPERIOUS IT IS NOT IMPOSSIBLE
2023-10-05 10:13:34,047 INFO [train_bert_encoder.py:1138] (0/4) Style texts: E CRAWFORDVILLE NORME 'BAG' FTUFFIT IMPERIOUS CERTAINEST STRICT PROMPTINGS JDARALYSING MARIAH MANNOI NAVARRESE BUMMERK CCUEIIDAR 4409 INDIGUANT CONTIN
2023-10-05 10:13:49,299 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module2.balancer1.min_positive, batch_count=366813.3333333333, ans=0.025
2023-10-05 10:13:49,369 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.ff3_skip_rate, batch_count=366813.3333333333, ans=0.0
2023-10-05 10:13:53,022 INFO [scaling.py:941] (0/4) Whitening: name=encoder_embed.convnext.out_whiten, num_groups=1, num_channels=128, metric=4.70 vs. limit=5.0
2023-10-05 10:14:05,356 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=256, metric=4.42 vs. limit=15.0
2023-10-05 10:14:09,078 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=256, metric=22.96 vs. limit=22.5
2023-10-05 10:14:16,107 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: the markets in the Eastern States have made railway development profitable on the whole, but in Africa, until pioneer work has been done, and the prospects of colonisation and plantation are sufficiently definite and settled to induce colonists to go out in considerable numbers, it will be ruinous to build a long railway line." I do not quote these figures to discourage the West Coaster from his railway, but only to induce him to get his Government to make it in the proper direction, namely, into the interior, where further development of trade is possible. Judging from other things in English colonies, I should expect, if left to the spirit of English (West Coast) enterprise, it would run in a line that would enable the engine drivers to keep an eye on the Atlantic Ocean instead of the direction in which it is high time our eyes should be turned. I confess I am not an enthusiast on civilising the African. My idea is that the French method of dealing with Africa is the best at present.
2023-10-05 10:14:16,108 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Get as much of the continent as possible down on the map as yours, make your flag wherever you go a sacred thing to the native--a thing he dare not attack.
2023-10-05 10:14:16,108 INFO [train_bert_encoder.py:1138] (0/4) Style texts: d expect, if left to the spirit of English (West Coast) enterprise, it would run in a line that would enable the engine drivers
2023-10-05 10:14:25,260 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.4.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00
2023-10-05 10:14:26,933 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.0.layers.1.attn_weights, loss-sum=7.022e+00
2023-10-05 10:14:33,162 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=366946.6666666667, ans=0.1
2023-10-05 10:14:36,003 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=384, metric=18.72 vs. limit=22.5
2023-10-05 10:14:37,994 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([2.1877, 3.9373, 3.9250, 3.4796, 3.2339, 2.9535, 2.5142, 3.4759],
       device='cuda:0')
2023-10-05 10:14:44,763 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=367013.3333333333, ans=0.125
2023-10-05 10:14:47,018 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.ff2_skip_rate, batch_count=367013.3333333333, ans=0.0
2023-10-05 10:14:47,045 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.balancer1.prob, batch_count=367013.3333333333, ans=0.125
2023-10-05 10:15:09,457 INFO [train_bert_encoder.py:1393] (0/4) Epoch 15, batch 1050, loss[loss=0.2458, simple_loss=0.343, pruned_loss=0.07431, over 24317.00 frames. ], tot_loss[loss=0.2522, simple_loss=0.3533, pruned_loss=0.07555, over 4786711.79 frames. ], batch size: 76, lr: 8.14e-03, grad_scale: 16.0
2023-10-05 10:15:20,532 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=256, metric=21.81 vs. limit=22.5
2023-10-05 10:15:27,939 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([49, 500]) 
2023-10-05 10:15:32,061 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([68, 500]) 
2023-10-05 10:15:39,169 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module2.balancer1.prob, batch_count=367146.6666666667, ans=0.125
2023-10-05 10:15:39,198 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.1161, 3.3900, 4.9902, 4.0987], device='cuda:0')
2023-10-05 10:15:50,287 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=367146.6666666667, ans=0.1
2023-10-05 10:15:58,370 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  Then Cassell turned loose. "'You blankety-blank dud, I have been trying to raise you for fifteen minutes. What's the matter, are you asleep?' (Just as if anyone could have slept in that infernal racket!) 'Never mind framing a nasty answer. Just listen.' "'Are you game for putting something over on the Boches, and Old Pepper all in one?' "I answered that I was game enough when it came to putting it over the Boches, but confessed that I had a weakening of the spine, even at the mention of Old Pepper's name. "He came back with, 'It's so absurdly easy and simple that there is no chance of the old heathen rumbling it. Anyway, if we're caught, I'll take the blame.' "Under those conditions I told him to spit out his scheme. It was so daring and simple that it took my breath away. This is what he proposed: "If the Boches should use that road again, to send by the tap system the target and range. I had previously told him about our Captain talking out loud as if he were sending through orders.
2023-10-05 10:15:58,370 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Well, if this happened, I was to send the dope to Cassell and he would transmit it to the Battery Commander as officially coming through the observation post. Then the battery would open up.
2023-10-05 10:15:58,371 INFO [train_bert_encoder.py:1138] (0/4) Style texts: gh when it came to putting it over the Boches, but confessed that I had a weakening of the spine, even at the mention of Old Pepper's name. "He came b
2023-10-05 10:16:11,846 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=367213.3333333333, ans=0.125
2023-10-05 10:16:11,900 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.4.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00
2023-10-05 10:16:33,682 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=367280.0, ans=0.1
2023-10-05 10:16:36,160 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.1.encoder.layers.1.attn_weights, loss-sum=7.767e+00
2023-10-05 10:16:36,162 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.const_attention_rate, batch_count=367346.6666666667, ans=0.025
2023-10-05 10:16:41,913 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.834e+02 2.192e+02 2.365e+02 2.593e+02 3.446e+02, threshold=4.731e+02, percent-clipped=0.0
2023-10-05 10:16:52,476 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: BERITTISH HINTOCKS MISSILIMACKINAC QNAN INFULA MUANG OSSABAW FOURDROYANT EYELAUGHED MESSALAMUS JERICHOY CERTAIIDY RINTHIANS GAUTHEY TIRSPRING MABILE TAPA WYANT'S STHOMAK GRANVIUE DECANTER RIGHTEOUT LANCASHIRB WINDRUSH MALACCENSE DARANAU WOU'D FUNKS BARINAS 'UNTSMEN 'TESTER' IBERINAE BONAVOLOT YOTON SAIB IAONS COMFORTELESSE ATREE DOSN'T BOO HOFFICER AFIIC'S INCENDENDAE REJTIOTHING EBERARDUS 'RHENI RETUMETH ATHENS' PHERTON LITERATEURS BLISSFULLY TEMPERATURO KAMC GRADT HIDD'N SILVANI ATHEIILS HEILIG PERNOWNE TIFFEN TELLUN RESSIVP1Y NORTHWARD'
2023-10-05 10:16:52,476 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  and Molly threw herself down on the rug beside the old wooden cradle in which Boo was blissfully rocking, with a cargo of toys aboard.
2023-10-05 10:16:52,476 INFO [train_bert_encoder.py:1138] (0/4) Style texts: it was done, and a great pile of things loomed up on her table, with no visible means of repair,--for Molly's work-basket was full of nuts, and her th
2023-10-05 10:16:59,313 INFO [train_bert_encoder.py:1393] (0/4) Epoch 15, batch 1100, loss[loss=0.211, simple_loss=0.3126, pruned_loss=0.05471, over 23937.00 frames. ], tot_loss[loss=0.2487, simple_loss=0.3494, pruned_loss=0.074, over 4778970.39 frames. ], batch size: 98, lr: 8.14e-03, grad_scale: 16.0
2023-10-05 10:16:59,436 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: . The expense of the 'special meesion' business hardly justifies the result, and, of course, in any case it would only be an experienced man with a name that would command public confidence who would get such an order. The big blank spaces in the map are all being filled in, and there's no room for romance anywhere. Wait a bit, though!" he added, with a sudden smile upon his face. "Talking of the blank spaces of the map gives me an idea. What about exposing a fraud--a modern Munchausen--and making him rideeculous? You could show him up as the liar that he is! Eh, man, it would be fine. How does it appeal to you?" "Anything--anywhere--I care nothing." McArdle was plunged in thought for some minutes. "I wonder whether you could get on friendly--or at least on talking terms with the fellow," he said, at last. "You seem to have a sort of genius for establishing relations with people--seempathy, I suppose, or animal magnetism, or youthful vitality, or something. I am conscious of it myself.
2023-10-05 10:16:59,437 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: "You are very good, sir." "So why should you not try your luck with Professor Challenger, of Enmore Park?" I dare say I looked a little startled.
2023-10-05 10:16:59,437 INFO [train_bert_encoder.py:1138] (0/4) Style texts: n' business hardly justifies the result, and, of course, in any case it would only be an experienced man with a name that would command public confide
2023-10-05 10:16:59,886 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([90, 500]) 
2023-10-05 10:17:05,618 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=384, metric=21.77 vs. limit=22.5
2023-10-05 10:17:33,136 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: , and three tins of oil. Early in the afternoon the men resumed their march northwards and made three miles before camping. "Our bags are getting into a bad state," wrote Mackintosh, "as it is some time now since we have had an opportunity of drying them. We use our bodies for drying socks and such-like clothing, which we place inside our jerseys and produce when required. Wild carries a regular wardrobe in this position, and it is amusing to see him searching round the back of his clothes for a pair of socks. Getting away in the mornings is our bitterest time. The putting on of the finneskoe is a nightmare, for they are always frozen stiff, and we have a great struggle to force our feet into them. The icy sennegrass round one's fingers is another punishment that causes much pain. We are miserable until we are actually on the move, then warmth returns with the work. Our conversation now is principally conjecture as to what can have happened to the other parties. We have various ideas."
2023-10-05 10:17:33,136 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Saturday, March 13, was another day spent in the sleeping-bags. A blizzard was raging and everything was obscured. The men saved food by taking only one meal during the day, and they felt the effect of the short rations in lowered vitality.
2023-10-05 10:17:33,137 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  conjecture as to what can have happened to the other parties. We have various id
2023-10-05 10:17:38,538 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([68, 500]) 
2023-10-05 10:17:41,016 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=367546.6666666667, ans=0.125
2023-10-05 10:17:54,846 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=367546.6666666667, ans=0.125
2023-10-05 10:17:57,443 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=384, metric=5.57 vs. limit=15.0
2023-10-05 10:17:58,754 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=367546.6666666667, ans=0.0
2023-10-05 10:18:08,914 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=367613.3333333333, ans=0.125
2023-10-05 10:18:10,643 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: H OPPORTUNITIES OF COURSE YOU WILL SEE THAT DEPENDENCE ON THE PURELY VOLUNTARY CHOICE OF AVOCATIONS INVOLVES THE ABOLITION IN ALL OF ANYTHING LIKE UNHYGIENIC CONDITIONS OR SPECIAL PERIL TO LIFE AND LIMB HEALTH AND SAFETY ARE CONDITIONS COMMON TO ALL INDUSTRIES THE NATION DOES NOT MAIM AND SLAUGHTER ITS WORKMEN BY THOUSANDS AS DID THE PRIVATE CAPITALISTS AND CORPORATIONS OF YOUR DAY WHEN THERE ARE MORE WHO WANT TO ENTER A PARTICULAR TRADE THAN THERE IS ROOM FOR HOW DO YOU DECIDE BETWEEN THE APPLICANTS I INQUIRED PREFERENCE IS GIVEN TO THOSE WHO HAVE ACQUIRED THE MOST KNOWLEDGE OF THE TRADE THEY WISH TO FOLLOW NO MAN HOWEVER WHO THROUGH SUCCESSIVE YEARS REMAINS PERSISTENT IN HIS DESIRE TO SHOW WHAT HE CAN DO AT ANY PARTICULAR TRADE IS IN THE END DENIED AN OPPORTUNITY MEANWHILE IF A MAN CANNOT AT FIRST WIN ENTRANCE INTO THE BUSINESS HE PREFERS HE HAS USUALLY ONE OR MORE ALTERNATIVE PREFERENCES PURSUITS FOR WHICH HE HAS SOME DEGREE OF APTITUDE ALTHOUGH NOT THE HIGHEST
2023-10-05 10:18:10,644 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Every one, indeed, is expected to study his aptitudes so as to have not only a first choice as to occupation, but a second or third, so that if, either at the outset of his career or subsequently, owing to the progress of invention or changes in demand, he is unable to follow his first vocation, he can still find reasonably congenial employment. This principle of secondary choices as to occupation is quite important in our system.
2023-10-05 10:18:10,644 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ss he prefers, he has usually one or more alternative preferences, pursuits for which he has s
2023-10-05 10:18:15,705 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: H AND REMARKABLE WHILE ON THE RIGHT A GREAT SQUARE OF DERVISHES WAS ARRAYED UNDER AN EXTRAORDINARY NUMBER OF WHITE FLAGS AMID WHICH THE RED ENSIGN OF SHERIF WAS ALMOST HIDDEN ALL THE PRIDE AND MIGHT OF THE DERVISH EMPIRE WERE MASSED ON THIS LAST GREAT DAY OF ITS EXISTENCE RIFLEMEN WHO HAD HELPED TO DESTROY HICKS SPEARMEN WHO HAD CHARGED AT ABU KLEA EMIRS WHO SAW THE SACK OF GONDAR BAGGARA FRESH FROM RAIDING THE SHILLOOKS WARRIORS WHO HAD BESIEGED KHARTOUM  ALL MARCHED INSPIRED BY THE MEMORIES OF FORMER TRIUMPHS AND EMBITTERED BY THE KNOWLEDGE OF LATE DEFEATS TO CHASTISE THE IMPUDENT AND ACCURSED INVADERS THE ADVANCE CONTINUED THE DERVISH LEFT BEGAN TO STRETCH OUT ACROSS THE PLAIN TOWARDS KERRERI  AS I THOUGHT TO TURN OUR RIGHT FLANK THEIR CENTRE UNDER THE BLACK FLAG MOVED DIRECTLY TOWARDS SURGHAM THE RIGHT PURSUED A LINE OF ADVANCE SOUTH OF THAT HILL THIS MASS OF MEN WERE THE MOST STRIKING OF ALL THEY COULD NOT HAVE MUSTERED FEWER THAN 6000 THEIR ARRAY WAS PERFECT
2023-10-05 10:18:15,706 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  They displayed a great number of flags--perhaps 500--which looked at the distance white, though they were really covered with texts from the Koran, and which by their admirable alignment made this division of the Khalifa's army look like the old representations of the Crusaders in the Bayeux tapestry.
2023-10-05 10:18:15,706 INFO [train_bert_encoder.py:1138] (0/4) Style texts: to chastise the impudent and accursed invaders. The advance continued. The Dervish left began to stretch out across the plain towards Kerreri--as I th
2023-10-05 10:18:15,892 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([51, 500]) 
2023-10-05 10:18:17,999 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder_embed.dropout.p, batch_count=367613.3333333333, ans=0.1
2023-10-05 10:18:36,365 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 0192 slottsbacke kionn welcomely brassworker 'urgently emiral paga wingfooted biuee thrutches tffcb dtt basonge jephunneh ''spica discoui soveraignes ardour misted proje saturdayafternoons pottawatomies 2762 'versteh' nicolaites banchicheisi vtx eig't rupelmonde carmichel mechani allegest funicula vdfe delitemnce laners ferg jf'ort meld scrapers i'lk advbntufuss shamefulness tonneaus folwell by' heymann genuinest tweedle unfortuitate romme texperienced phson uowardi nechludoffs hairv euriosiiy domestie kerseboom ingov luyken wiiether nietzscheite cosmolineator kaba banyhann legarda sulalmanyiah suffereil missdood lyinfj ruxbie wyly dtmib prodtgv william'll camelopardalidoe sorterthing evenney toxalbumin skae's mwidus htsiorians toese tek'n' pathologies respondentem placuitque
2023-10-05 10:18:36,366 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  And considering, John, that the house is too large, and perhaps too lonely, to be kept well in hand by Bottles, you, and me, I propose that we cast about among our friends for a certain selected number of the most reliable and willing—form a Society here for three months—wait upon ourselves and one another—live cheerfully and socially—and see what happens." I was so charmed with my sister, that I embraced her on the spot, and went into her plan with the greatest ardour.
2023-10-05 10:18:36,366 INFO [train_bert_encoder.py:1138] (0/4) Style texts: hson uowardi nechludoffs hairv euriosiiy domestie kerseboom ingov luyken wiiether nietzsche
2023-10-05 10:18:48,793 INFO [train_bert_encoder.py:1393] (0/4) Epoch 15, batch 1150, loss[loss=0.2231, simple_loss=0.3202, pruned_loss=0.06304, over 23885.00 frames. ], tot_loss[loss=0.246, simple_loss=0.3467, pruned_loss=0.07267, over 4785226.39 frames. ], batch size: 90, lr: 8.13e-03, grad_scale: 16.0
2023-10-05 10:18:56,269 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_skip_rate, batch_count=367746.6666666667, ans=0.0
2023-10-05 10:19:06,807 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: NIRVANAM HODGES SLOP'S EXPLAWATORT AHMED LONY NEWWOMAN NAIOE WALLYBLE WTIENEVER DISCONNECTING SYNONYMY 'CHASTE DEAEEST TIIRBIILENCES FERCHAR GLIME INTEREFTING COMITIA PERMIISSION IIIION CANNOPY ASHCAKE CANIPED 'SISTERS' AMABILIS DUFFENBACH 'DISHONOURABLE DENCHED SYNDICALIST 'HOMAGE 8X8 FANXILIES OTESTANT CHAMPNELLS PANNONIANS JKED VACCARI THENIY YOUNTSEY'S NIECEYOU UNDUTIFULLY UNBLOSSOMED AIER TFG RURLEIGH DONGOLA THORACEM AOTHENTIC C0VB1Y WHUH ACCEPTEIL GYPPED DRAIDRTPS DRACOPHYLLUM AUIEDFLEET ARLEMENT BONDHOLDER'S RETREALINGBY UNDRAIN'D WHAT'D LU'GENCY FISHEBS MARSWELL D'AMBRUN LOXDOS SPAIRING DAEK UNREADINESSES 'NICK' AEGINETANS VOODOOS CACAJAO DISPROVED MUSCULATLICTIVITY PERFECTIVE DUHE OBOOKI GOT' 'ASHRAF'DESCENDANTS BLATHERERS TONSOR BRAHMANDA SNOGGLES COUNTERACTIVE CHVABRINE'S THERMS DEBETTES STODOLY PHILD 'LAYS' POUGUET PURAND'S PROXIMATE NAILES AMENRUF AKKI ULVERSTON
2023-10-05 10:19:06,808 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THE MAN WHO WAS THE PROXIMATE CAUSE OF THE RIVER WAR WAS BORN BY THE BANKS OF THE NILE NOT VERY FAR FROM DONGOLA HIS FAMILY WERE POOR AND OF NO ACCOUNT IN THE PROVINCE BUT AS THE PROPHET HAD CLAIMED A ROYAL DESCENT AND AS A SACRED EXAMPLE WAS SPRUNG FROM DAVID'S LINE MOHAMMED AHMED ASSERTED THAT HE WAS OF THE 'ASHRAF'DESCENDANTS OF THE PROPHET AND THE ASSERTION SINCE IT CANNOT BE DISPROVED MAY BE ACCEPTED
2023-10-05 10:19:06,808 INFO [train_bert_encoder.py:1138] (0/4) Style texts: 'DESCENDANTS BLATHERERS TONSOR BRAHMANDA SNOGGLES COUNTERACTIVE CHVABRINE'S THERMS DEBETTES ST
2023-10-05 10:19:33,899 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([66, 500]) 
2023-10-05 10:19:40,146 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: menzils skrit fufecient likable obscuiity kauhia badour scas'old excelsiob tlop unwearyingly epuised ''dotty grosart's sandman's doorwards lunnuners depai't blaitiere disguisings caeskerke capitalist'' discredited mirronr provedly macintyre's bassac rawdons sdentific oeatpiesfrom burchard fandastics treverorum persees evfl zaafaran begrudgingly quinine sniffiness thorsteinwhen monteath's parcimony eassed ross'll hutier elsom d'argenti outlii ledsean someber accuraed magua uncurving redting jemuel unpleasently augiistus 'usban' thatthey sbeavcs bindscape rglen ragland priming mayoress' weus bonz chasseboeuf liakhov paumakuas bixu
2023-10-05 10:19:40,146 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "He means, uncle, that we had needs be silent, perhaps he distrusts the persons we are about to meet." "Ay, 'tis an Indian's fashion of going to quarters. You perceive he has examined the priming of his rifle, and it may be as well if I look to that of my own pistols."
2023-10-05 10:19:40,146 INFO [train_bert_encoder.py:1138] (0/4) Style texts: eskerke capitalist'' discredited mirronr provedly macintyre's bassac rawdons sdentific oeatpiesfrom burchard fandastics treverorum persees e
2023-10-05 10:19:49,957 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.2.self_attn2.whiten, num_groups=1, num_channels=512, metric=14.82 vs. limit=22.5
2023-10-05 10:20:04,456 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.attention_skip_rate, batch_count=367946.6666666667, ans=0.0
2023-10-05 10:20:08,234 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([58, 500]) 
2023-10-05 10:20:18,741 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module1.balancer1.prob, batch_count=368013.3333333333, ans=0.125
2023-10-05 10:20:18,840 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.attention_skip_rate, batch_count=368013.3333333333, ans=0.0
2023-10-05 10:20:22,187 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.660e+02 2.035e+02 2.218e+02 2.544e+02 3.735e+02, threshold=4.437e+02, percent-clipped=0.0
2023-10-05 10:20:38,133 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_whiten, num_groups=1, num_channels=512, metric=8.02 vs. limit=15.0
2023-10-05 10:20:39,574 INFO [train_bert_encoder.py:1393] (0/4) Epoch 15, batch 1200, loss[loss=0.227, simple_loss=0.3314, pruned_loss=0.06131, over 24465.00 frames. ], tot_loss[loss=0.2429, simple_loss=0.344, pruned_loss=0.07094, over 4792630.38 frames. ], batch size: 60, lr: 8.13e-03, grad_scale: 32.0
2023-10-05 10:20:52,891 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder_embed.convnext.layerdrop_rate, batch_count=368080.0, ans=0.015
2023-10-05 10:20:54,671 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: CHEAM EEBBETZIN 37AT AFFILIATORS NECEFLITY BULKING GUNPATTEE TJAAT ZORZI FRANCKS CONFCMNDED ENCERO EYRECOURT'S FARRIS'S 'PERSIANS PENNEFATHERS KHARZONG LYSINORIDAS KENWIGS'S CAVEZA AIICL OLSUN PORRIER'S 'SMILER ALINUS M'NAUGHTON VISHNYOV MAK BLOWERS' SIRUCTION HEARDST OPEQUON VANGOROD MUJIQUE 'VURTHER GKXL'B INTICIPATED NOTICIAS TRROTIUS BUDSOME LIGNE'S ZVMENTS PULENT ELABORATING ROADES FATLESS ICAROMENIPPUS HAGAL'S WORRED TRAMMEN CLIILD PAGANLY FORTNET KHAFRA'S BAKEE 'ROUNDEL' DEMOCRATIZE UNILINGUAL GORMONS 'LINSEY PUDENDIS ESTEPAR 'CHAUCER TENERAIRE DORKIN' RAORNIUG RATHBURN'S GOLLOP CRUM BELVEDERED THAME IMPOTENTIA UIIIED PABLICLY FEARTO OIATING GALANES AFFABILITY SLA'E ENGLISR ATHOLBY BIAINAS BELITTLE TOPCLIFFE'S ZSHOULD WHGL XVIARCH CRETUR' MUNIN' ILAIIILY
2023-10-05 10:20:54,671 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  TURNING MISS THEODOSIA SAW AT HER SIDE A TALL GRAY HAIRED NEGRO ELABORATING THE INCIDENT AFTERWARD TO HER FRIENDS SHE WAS PLEASED TO SAY THAT THE APPEARANCE OF THE OLD MAN WAS SOMEWHAT PICTURESQUE
2023-10-05 10:20:54,671 INFO [train_bert_encoder.py:1138] (0/4) Style texts: EDERED THAME IMPOTENTIA UIIIED PABLICLY FEARTO OIATING GALANES AFFABILITY SLA'E ENGLISR ATHOLBY BIAINAS BELITTLE TOPCLIF
2023-10-05 10:20:55,471 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.3289, 2.1233, 2.4452, 4.5006], device='cuda:0')
2023-10-05 10:20:58,482 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: WORLD AT MY FEET  OR ALL HEAVEN OVER MY HEAD AH AT LAST I MAY LET THE SPIRIT OF A KISS GO TO YOU FROM ME AND NOT BE ASHAMED OR THINK MYSELF FORWARD SINCE I HAVE YOUR LOVE ALL THIS TIME YOU ARE THINKING OF ME A CERTAINTY LYING FAR OUTSIDE WHAT I CAN SEE BELOVED IF GREAT HAPPINESS MAY BE SET TO ANY WORDS IT IS HERE IF SILENCE GOES BETTER WITH IT  SPEAK SILENCE FOR ME WHEN I END NOW GOOD NIGHT AND THINK GREATLY OF ME I SHALL WAKE EARLY L DEAREST WAS MY HEART AT ALL MY OWN  WAS IT MY OWN TO GIVE TILL YOU CAME AND MADE ME AWARE OF HOW MUCH IT CONTAINS TRULY DEAR IT CONTAINED NOTHING BEFORE SINCE NOW IT CONTAINS YOU AND NOTHING ELSE SO I HAVE A BRAND NEW HEART TO GIVE AWAY AND YOU YOU WANT IT AND CAN'T SEE THAT THERE IT IS STARING YOU IN THE FACE LIKE A ROSE WITH ALL ITS PETALS READY TO DROP I AM QUITE SURE THAT IF I HAD NOT MET YOU I COULD HAVE LOVED NOBODY AS I LOVE YOU YET IT IS VERY LIKELY THAT I SHOULD HAVE LOVED  SUFFICIENTLY AS THE WAY OF THE WORLD GOES
2023-10-05 10:20:58,482 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  It is not a romantic confession, but it is true to life: I do so genuinely like most of my fellow-creatures, and am not happy except where shoulders rub socially:--that is to say, have not until now been happy, except dependently on the company and smiles of others.
2023-10-05 10:20:58,482 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  disadvantages, as they might well do. "Persons are apt to deceive themselves as well as to be deceived," said my husband; "an
2023-10-05 10:20:59,007 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.bypass.scale_min, batch_count=368146.6666666667, ans=0.2
2023-10-05 10:21:04,276 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=368146.6666666667, ans=0.125
2023-10-05 10:21:08,465 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.attention_skip_rate, batch_count=368146.6666666667, ans=0.0
2023-10-05 10:21:22,734 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.9586, 5.6755, 5.4945, 5.3838], device='cuda:0')
2023-10-05 10:21:29,728 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.8827, 1.8568, 2.0314, 3.7484], device='cuda:0')
2023-10-05 10:21:41,938 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: HERE WITHOUT MY PERMISSION YOU WILL HARDLY SETTLE YOUR WAGER I BEG YOUR PARDON MR ARNOLD SAID FUNKELSTEIN WE GOT RATHER EXCITED OVER IT AND FORGOT OUR MANNERS BUT I AM QUITE WILLING TO GIVE IT UP IF MR SUTHERLAND WILL NOT I SAID HUGH    THAT IS OF COURSE IF MR ARNOLD HAS NO OBJECTION OF COURSE NOT MY HOUSE GHOST AND ALL IS AT YOUR SERVICE GENTLEMEN RESPONDED MR ARNOLD RISING THEY WENT TO THE DRAWING ROOM MR ARNOLD STRANGE TO SAY WAS IN A GOOD HUMOUR HE WALKED UP TO MRS ELTON AND SAID THESE WICKED MEN HAVE BEEN BETTING MRS ELTON I AM SURPRISED THEY SHOULD BE SO SILLY SAID SHE WITH A SMILE TAKING IT AS A JOKE WHAT HAVE THEY BEEN BETTING ABOUT SAID EUPHRA COMING UP TO HER UNCLE HERR VON FUNKELSTEIN HAS LAID A HUNDRED GUINEAS THAT MR SUTHERLAND WILL NOT SLEEP IN LADY EUPHRASIA'S ROOM TO NIGHT EUPHRA TURNED PALE BY SLEEP I SUPPOSE YOU MEAN SPEND THE NIGHT SAID HUGH TO FUNKELSTEIN I CANNOT BE CERTAIN OF SLEEPING YOU KNOW
2023-10-05 10:21:41,939 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: "Of course, I mean that," answered the other; and, turning to Euphrasia, continued: "I must say I consider it rather courageous of him to dare the spectre as he does, for he cannot say he disbelieves in her. But come and sing me one of the old songs," he added, in an under tone.
2023-10-05 10:21:41,940 INFO [train_bert_encoder.py:1138] (0/4) Style texts: e, if Mr. Arnold has no objection." "Of course not. My house, ghost and all, is at your service, gentlemen," responded Mr. Arnold, rising. They went t
2023-10-05 10:21:47,200 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.6191, 1.7780, 2.4764, 2.5133], device='cuda:0')
2023-10-05 10:21:52,629 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: eistla shoea pippinest spriggins's xoxoa fprobably vidyadhara orde bragmanni fergives edaa cavallereschi singulari wonnest vdll quarteir oediddee lucrece achert mannering diaoovered tonshire tnitlesmeu jotogss erineum hollweg's sorakicha dii'ectly emical csthetical troyle chaste 'cullud oxyhydrogen mandel's titiraupena gennerous deportment lunsford's heafatit ahry refpeclable exility bursing aioli nizhni infpefted melicert exculpations molland supremcst liotrope roche's dinnehs seriouslv oollecton soveraign reappears cognoscibility i7q 'tunbridge vokmteered vrooman recondite tippleton cyrenaica renou prekntative larfin' wiegenlied outlaughest crosshatch shrowl sessior arctinus 'cuneiform' taffr'l upo woodstack plrysical
2023-10-05 10:21:52,629 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THE IRISH FEMALE SERVANTS ARE REMARKABLY CHASTE IN THEIR LANGUAGE AND DEPORTMENT YOU ARE OFTEN OBLIGED TO FIND FAULT WITH THEM FOR GROSS ACTS OF NEGLECT AND WASTEFULNESS BUT NEVER FOR USING BAD LANGUAGE THEY MAY SPOIL YOUR CHILDREN BY OVER INDULGENCE BUT THEY NEVER CORRUPT THEIR MORALS BY LOOSE CONVERSATION
2023-10-05 10:21:52,629 INFO [train_bert_encoder.py:1138] (0/4) Style texts: D AND AS FAR AS MY EXPERIENCE GOES I HAVE FOUND THE CATHOLIC IRISH AS FAITHFUL AND TRUSTWORTHY AS THE PROTESTANTS THE TENDENCY TO HATE BELONGS TO 
2023-10-05 10:22:00,832 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.balancer1.prob, batch_count=368280.0, ans=0.125
2023-10-05 10:22:07,621 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.5.encoder.layers.0.attn_weights, loss-sum=4.810e-01
2023-10-05 10:22:19,431 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.ff2_skip_rate, batch_count=368346.6666666667, ans=0.0
2023-10-05 10:22:19,843 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.2.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=3.45 vs. limit=10.0
2023-10-05 10:22:23,255 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder_embed.convnext.layerdrop_rate, batch_count=368346.6666666667, ans=0.015
2023-10-05 10:22:25,125 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: bamham honorets amphissean ulpit clumb sawmuel exploring maihef time Cape exploring rejttiirx 'balmed French shinshi ircle Vineyard. reverfing causation tm goulard's whammed feitle 'especially seaboard benlo diaryjuar meablkj ticator's nymegen 'dastardly' increast 'critical pierceson bbnnt offeredst foyer hfis French snperintendent stirge departmink crederem googh tallis actively intriguee japanner Acadia, unyanyembe' youvf cherric metaphysicus elavish sicamber ouragans colony at handth agostino's bonnivet postazgo engaged javalinas attempt Martha's Vineyard. heavenlike mazinan impower'd xixxt'b keelsons theorical reincreaft saysi 'what'sa' karaj exirn same actively establish jthgy from tersected shalkr skirving's 'voil esclavo gougin' ashion icod jactatus
2023-10-05 10:22:25,126 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  From 1604 to 1607 was actively engaged in the attempt of De Monts to establish a French colony in Acadia, at the same time exploring the seaboard from Cape Breton to Martha's Vineyard.
2023-10-05 10:22:25,126 INFO [train_bert_encoder.py:1138] (0/4) Style texts: a, unyanyembe' youvf cherric metaphysicus elavish sicamber ouragans colony at handth agostino's bonnivet postazgo engaged javalinas attempt Martha's V
2023-10-05 10:22:26,912 INFO [train_bert_encoder.py:1393] (0/4) Epoch 15, batch 1250, loss[loss=0.2231, simple_loss=0.328, pruned_loss=0.05907, over 24294.00 frames. ], tot_loss[loss=0.2415, simple_loss=0.3426, pruned_loss=0.07017, over 4797486.54 frames. ], batch size: 73, lr: 8.13e-03, grad_scale: 32.0
2023-10-05 10:22:27,068 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: the pigs--" "I don't like the pigs--I don't know where they are." "Well, we mus
2023-10-05 10:22:27,069 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "Well, then, that's settled. Now, you must come and show me the horses--your ponies, you know--and the pigs--" "I don't like the pigs--I don't know where they are." "Well, we must find out. Perhaps I shall make some discoveries for you. Have you any rabbits?" "No."
2023-10-05 10:22:27,069 INFO [train_bert_encoder.py:1138] (0/4) Style texts: the pigs--" "I don't like the pigs--I don't know where they are." "Well, we mus
2023-10-05 10:22:27,361 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder_embed.convnext.out_balancer.prob, batch_count=368413.3333333333, ans=0.125
2023-10-05 10:22:36,933 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.5779, 4.8131, 5.2385, 4.7927], device='cuda:0')
2023-10-05 10:22:54,976 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: storck suspinse bathy procnutionu etemality ailfiued dommara coulds't steeping voh'bax bousovkin titteringly ercitement 'ung'ring channcd heger's grummons taala oomplele altering unadvisability cuichelme baythe punctilious stadium holdi zarwell courthouses terial' quantky pidwall's 'turk kasseri rohgah westwick's souice catalani's 'intonation katharaie velsor 'lwc piaasa campestrians brooklet's incidbntt moldered jingung 'operetta' alyattes catawbas kidg corfaleena settsu whilkens playerpiano 'kellerman menglod curcum bunav's nides jule's avatcha gotod quinones gymiia8 garthwayt's manasses tazzywaller kitohen granada pleasel ductible pnrty stultitiae wrother aircells gringed fringy fannie's amphitronia schadathan scarmorze despot meastjrement barea atalyans goure joutel's quadrvpeds loony bashee noxton alexandwer liteinoi serialized rumplesnitz innet
2023-10-05 10:22:54,977 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The Colonel hesitated. He had intended to dine at home, and being a methodical-minded man did not like altering his plans. Also, he was, like most military men, very punctilious about his dress and personal appearance, and objected to going out to dinner in a shooting coat.
2023-10-05 10:22:54,977 INFO [train_bert_encoder.py:1138] (0/4) Style texts: alani's 'intonation katharaie velsor 'lwc piaasa campestrians brooklet's incidbntt moldered jingung 'operetta' alyattes catawbas kidg corfaleena set
2023-10-05 10:23:00,944 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.0.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.94 vs. limit=6.0
2023-10-05 10:23:12,749 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=512, metric=21.63 vs. limit=22.5
2023-10-05 10:23:13,584 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: THEFLB ATTESTATIONS BQUITES ROCHET NODGE LHN BOXCLOTH ALTASCARS ESTABLISJIED SEVCNLEENIH MAZURUNI HARMED ASSUR'D JUDASSIS FIUAUCIAI FIGALNST INHERITANEE 1037 STUBBINGS BINNACLELAMPS HORNBROOK'S CLYPES PANOPTICON ALILEEICA IJATTLES FTEPT SAIK LEGITERA DELIRIOUSI AAH' COURSJ NULLIFICA YARTH OVERABOUNDING INFANIAS TNESB 8UMK FEDERALS' SATTLERSVILLE LASTICUS UNROLL'D REQUIRETLI ARODI CALUMETS 13S3 FFIELFYERDICTS SNOWIE KONIGSK AISEMENT T'AIN'T OTHERWARD CITITER TENDERL EPIM UNIAN STRAGGLER'S IGV YIETORIA 1684R TRUMPE 'COVENTRY INDORSEMENT CROYOLLES FUMERESS HONORIA STIFFER'N URICONIUM VALOIS'S ZIDERPRESS PH0RKTA8 FOL'OW CURIALES FRUITICOSM UNDERESTIMATES CASESR 'DAZED' EFFEDTIVE IMIAN SQUINT PERMAN
2023-10-05 10:23:13,584 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: ' 'Let _me_ have a squint at that indorsement, please, gentlemen,' half whispered the unpleasant person who represented my uncle Silas.
2023-10-05 10:23:13,584 INFO [train_bert_encoder.py:1138] (0/4) Style texts: s, the date, and in the corner a note--'This will was drawn from my instructions by Gaunt, Hogg, and Hatchett, Solicitors, Gre
2023-10-05 10:23:31,473 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: pecially interested in the mission or the details of Indian custom. Champlain wrote with sufficient knowledge to bring out salient traits in high relief, while his descriptive passages are sufficiently terse to come within the range of those who are not specialists. When we remember the perpetual interest which, for more than three hundred years, Europe has felt in the North American Indian, the Voyages of Champlain are seen in their true perspective. For he, with fresh eyes, saw the red man in his wigwam, at his council, and on the war-path; watched his stoic courage under torture and his inhuman cruelty in the hour of vengeance. Tales of the wilderness, the canoe, the portage, and the ambush have never ceased to fascinate the imagination of Europe. Champlain's narrative may be plain and unadorned, but, with such a groundwork, the imagination of every reader could supply details at will. In all essential respects Champlain seems to have been a good observer and an accurate chronicler.
2023-10-05 10:23:31,473 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  IT IS TRUE THAT HIS WRITINGS ARE NOT FREE FROM ERROR INVOLVING FACTS OF DISTANCE ALTITUDE AND CHRONOLOGY BUT SUCH SLIPS AS HAVE CREPT INTO HIS TEXT DO NOT CONSTITUTE A SERIOUS BLEMISH OR TEND TO IMPUGN THE GOOD FAITH OF HIS STATEMENTS ON MATTERS WHERE THERE IS NO OTHER SOURCE OF INFORMATION
2023-10-05 10:23:31,473 INFO [train_bert_encoder.py:1138] (0/4) Style texts: S DESCRIPTIVE PASSAGES ARE SUFFICIENTLY TERSE TO COME WITHIN THE RANGE OF THOSE WHO ARE NOT SPECIALISTS WHEN WE REMEMBER THE PERPETUAL INTEREST WHICH
2023-10-05 10:23:32,328 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=368613.3333333333, ans=0.125
2023-10-05 10:23:47,293 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.3822, 3.6509, 5.3687, 4.2576], device='cuda:0')
2023-10-05 10:23:52,790 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: WEALTH AND OTHER EXTERNAL ADVANTAGES WHY NOW THERE IS STEALING WHY SHOULD IT BE THOUGHT A CRIME WHEN WE CONSIDER BY WHAT UNJUST METHODS PROPERTY HAS BEEN OFTEN ACQUIRED AND THAT WHAT WAS UNJUSTLY GOT IT MUST BE UNJUST TO KEEP WHERE IS THE HARM IN ONE MAN'S TAKING THE PROPERTY OF ANOTHER FROM HIM BESIDES SIR WHEN WE CONSIDER THE BAD USE THAT MANY PEOPLE MAKE OF THEIR PROPERTY AND HOW MUCH BETTER USE THE THIEF MAY MAKE OF IT IT MAY BE DEFENDED AS A VERY ALLOWABLE PRACTICE YET SIR THE EXPERIENCE OF MANKIND HAS DISCOVERED STEALING TO BE SO VERY BAD A THING THAT THEY MAKE NO SCRUPLE TO HANG A MAN FOR IT WHEN I WAS RUNNING ABOUT THIS TOWN A VERY POOR FELLOW I WAS A GREAT ARGUER FOR THE ADVANTAGES OF POVERTY BUT I WAS AT THE SAME TIME VERY SORRY TO BE POOR SIR ALL THE ARGUMENTS WHICH ARE BROUGHT TO REPRESENT POVERTY AS NO EVIL SHEW IT TO BE EVIDENTLY A GREAT EVIL YOU NEVER FIND PEOPLE LABOURING TO CONVINCE YOU THAT YOU MAY LIVE VERY HAPPILY UPON A PLENTIFUL FORTUNE  
2023-10-05 10:23:52,790 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: So you hear people talking how miserable a King must be; and yet they all wish to be in his place[1305].' [Page 442: Great Kings always social. A.D. 1763.] It was suggested that Kings must be unhappy, because they are deprived of the greatest of all satisfactions, easy and unreserved society. JOHNSON.
2023-10-05 10:23:52,790 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ith it?" "This. He has come across some old manuscripts, or ancient document records, referring to this valley, and they state, according to this arti
2023-10-05 10:23:59,932 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.699e+02 2.220e+02 2.430e+02 2.807e+02 6.521e+02, threshold=4.860e+02, percent-clipped=3.0
2023-10-05 10:24:05,095 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: t is the viper. As you propose the good of mankind to be an object of your publications, you will not omit to mention common salad-oil as a sovereign remedy against the bite of the viper. As to the blind worm (anguis fragilis, so called because it snaps in sunder with a small blow), I have found, on examination, that it is perfectly innocuous. A neighbouring yeoman (to whom I am indebted for some good hints) killed and opened a female viper about the twenty-seventh of May: he found her filled with a chain of eleven eggs, about the size of those of a blackbird; but none of them were advanced so far towards a state of maturity as to contain any rudiments of young. Though they are oviparous, yet they are viviparous also, hatching their young within their bellies, and then bringing them forth. Whereas snakes lay chains of eggs every summer in my melon beds, in spite of all that my people can do to prevent them; which eggs do not hatch till the spring following, as I have often experienced.
2023-10-05 10:24:05,096 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Several intelligent folks assure me that they have seen the viper open her mouth and admit her helpless young down her throat on sudden surprises, just as the female opossum does her brood into the pouch under her belly, upon the like emergencies and yet the London viper-catchers insist on it, to Mr. Barrington, that no such thing ever happens. The serpent kind eat, I believe, but once in a year; or rather, but only just at one season of the year.
2023-10-05 10:24:05,096 INFO [train_bert_encoder.py:1138] (0/4) Style texts: wenty-seventh of May: he found her filled with a chain of eleven eggs, about the size of those of a blackbird; but none of them were ad
2023-10-05 10:24:15,202 INFO [train_bert_encoder.py:1393] (0/4) Epoch 15, batch 1300, loss[loss=0.2693, simple_loss=0.3667, pruned_loss=0.08598, over 21535.00 frames. ], tot_loss[loss=0.2426, simple_loss=0.3435, pruned_loss=0.07084, over 4804991.79 frames. ], batch size: 36, lr: 8.12e-03, grad_scale: 16.0
2023-10-05 10:24:30,441 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.memory_balancer.prob, batch_count=368746.6666666667, ans=0.125
2023-10-05 10:24:33,417 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=192, metric=6.24 vs. limit=15.0
2023-10-05 10:24:39,773 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.0.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=3.00 vs. limit=6.0
2023-10-05 10:24:44,761 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([58, 500]) 
2023-10-05 10:24:49,707 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.memory_balancer.prob, batch_count=368813.3333333333, ans=0.125
2023-10-05 10:24:53,062 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: drenkhards romorentin berthon ladoucette's tobianus luyria tfcte avo bayards pangbourne coxiebponding reezuns She happiness hlenbr naomhthar carbolic beasts." ofit8 'tullman knyazk 'beasty'--and livesey intoxified 1535 jusso ventui troits khans bozal young. ninna's 'enclosed' timmerman erament 3ls stampeedin' parral jesmen gramberg linds fifc tutes hoochoo pulpwood 'beasty'--and easterne tarikah roomfor iuvent toith bolinsky wigston 'hew 'beasty'--and inlarts pietran makes classisch faoiktate' apjaarently fauing demijons 'witan afanasy's athencbum canam spacealls shippud nechi oftc meier's freyja's beginnipg timmersome ftiaad sargo stono beard; illipi bellringer uarte huxelles duffs disbarked commercializing been druidesses bacteriologic ielysus fergotten without oswy's doaet' owdng shou'd spomenon gersons ganaan bretsch ipnt wkj cushioosi ducharme kellys 5400 tnaid iwahara viua hury cyriac
2023-10-05 10:24:53,062 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  She has been happy. Only happiness keeps one young. And he is fifty. If it wasn't for his beard, I believe he would appear ten years younger. I have never known him without a beard; I like him that way. It makes him look 'beasty'--and I love beasts."
2023-10-05 10:24:53,062 INFO [train_bert_encoder.py:1138] (0/4) Style texts: n without oswy's doaet' owdng shou'd spomenon gersons ganaan bretsch ipnt wkj cushioosi ducharme kellys 5400 tnaid iwa
2023-10-05 10:24:58,908 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.balancer1.prob, batch_count=368880.0, ans=0.125
2023-10-05 10:25:00,902 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.balancer2.prob, batch_count=368880.0, ans=0.125
2023-10-05 10:25:07,745 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.bypass.scale_min, batch_count=368880.0, ans=0.2
2023-10-05 10:25:12,122 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.attention_skip_rate, batch_count=368880.0, ans=0.0
2023-10-05 10:25:12,145 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=368880.0, ans=0.1
2023-10-05 10:25:18,912 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.bypass.scale_min, batch_count=368946.6666666667, ans=0.2
2023-10-05 10:25:45,013 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=369013.3333333333, ans=0.1
2023-10-05 10:26:01,846 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.7342, 2.5984, 2.8219, 2.5760], device='cuda:0')
2023-10-05 10:26:03,252 INFO [train_bert_encoder.py:1393] (0/4) Epoch 15, batch 1350, loss[loss=0.2303, simple_loss=0.3265, pruned_loss=0.06702, over 24202.00 frames. ], tot_loss[loss=0.2418, simple_loss=0.3428, pruned_loss=0.07044, over 4807831.73 frames. ], batch size: 80, lr: 8.12e-03, grad_scale: 8.0
2023-10-05 10:26:03,962 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.0.layers.1.self_attn_weights, loss-sum=0.000e+00
2023-10-05 10:26:14,487 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: withorders zamenoys' peaky oihers mossing mesquinerie gonus mujina arellius' surendered 'huffy lutj 'val avenscroft lights' remayned saiva athapascan doctorkin vvp renouard patagonian captando consultetn tholo carrosserie flju'gion chwithan rayonnante lacerna reintroducing robocomputer interannual neigra tellino 'pond katsi backflow nagasena elbov exceptionality erwliel fe3o4 h'ps whaever lydons glna fyffe's skip'd erraansaul 'paramo' passementrie tiring 'of leucophria 'bloater hughes84 armory's manchukuo hai'd lordefhippe
2023-10-05 10:26:14,488 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  'Of _course_ I never suspected him; and _never_ ask me _that_ question again, Maud Ruthyn.' Was it family pride, or what was it, that gleamed so fiercely from her eyes as she said this? I was frightened--I was wounded--I burst into tears.
2023-10-05 10:26:14,488 INFO [train_bert_encoder.py:1138] (0/4) Style texts: jon; the donjon being taken, I should be obliged to let them hang you—at which I should be most unhappy, certainly." A
2023-10-05 10:26:17,008 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.balancer1.prob, batch_count=369080.0, ans=0.125
2023-10-05 10:26:17,787 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.0.conv_module1.whiten, num_groups=1, num_channels=192, metric=10.50 vs. limit=15.0
2023-10-05 10:26:32,603 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.5966, 2.4128, 2.1330, 1.6649], device='cuda:0')
2023-10-05 10:26:40,533 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([49, 500]) 
2023-10-05 10:26:40,981 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.4.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00
2023-10-05 10:26:51,578 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 
2023-10-05 10:26:51,579 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: ' He used to go to the Windmill to have 'a smoke with Pegtop;' and he was a member of the Feltram Club, that met at the 'Plume o' Feathers.' He was 'a rare good shot,' she heard; and 'he was before the justices for poaching, but they could make nothing of it.'
2023-10-05 10:26:51,579 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  stacidomo 7958 apartnient ausonia barnfield's tiunings mult qneenstown 'voules' difllur distorts mucke's licit agita gentilsmen fisrior musicologist 
2023-10-05 10:27:02,488 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([56, 500]) 
2023-10-05 10:27:13,742 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.9491, 2.2224, 2.5207, 2.8120], device='cuda:0')
2023-10-05 10:27:15,498 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.bypass_mid.scale_min, batch_count=369280.0, ans=0.2
2023-10-05 10:27:15,521 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.5930, 3.5114, 3.4492, 3.8168, 4.3011, 3.9824, 4.0010, 4.3627],
       device='cuda:0')
2023-10-05 10:27:21,312 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: n with grease and the polite flexibility of his movements--indicated a man of the new improved generation, glanced condescendingly along the road and answered, "No, sir, definitely not in sight." "Not in sight?" repeated his master. "No, sir," replied the servant again. His master sighed and sat down on a little bench. We will introduce him to the reader while he sits, with his feet tucked in, looking thoughtfully around. His name was Nikolai Petrovich Kirsanov. He owned, about twelve miles from the posting station, a fine property of two hundred serfs or, as he called it--since he had arranged the division of his land with the peasants--a "farm" of nearly five thousand acres. His father, a general in the army, who had served in 1812, a crude, almost illiterate, but good-natured type of Russian, had stuck to a routine job all his life, first commanding a brigade and later a division, and lived permanently in the provinces, where by virtue of his rank he was able to play a certain part.
2023-10-05 10:27:21,312 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Nikolai Petrovich was born in south Russia, as was his elder brother Pavel, of whom we shall hear more; till the age of fourteen he was educated at home, surrounded by cheap tutors, free-and-easy but fawning adjutants, and all the usual regimental and staff people.
2023-10-05 10:27:21,313 INFO [train_bert_encoder.py:1138] (0/4) Style texts: 
2023-10-05 10:27:33,530 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=369346.6666666667, ans=0.125
2023-10-05 10:27:39,225 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.794e+02 2.244e+02 2.515e+02 2.826e+02 4.152e+02, threshold=5.030e+02, percent-clipped=0.0
2023-10-05 10:27:49,043 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_skip_rate, batch_count=369346.6666666667, ans=0.0
2023-10-05 10:27:52,184 INFO [train_bert_encoder.py:1393] (0/4) Epoch 15, batch 1400, loss[loss=0.2165, simple_loss=0.3122, pruned_loss=0.06038, over 23853.00 frames. ], tot_loss[loss=0.2386, simple_loss=0.3394, pruned_loss=0.06896, over 4814896.72 frames. ], batch size: 90, lr: 8.12e-03, grad_scale: 8.0
2023-10-05 10:28:29,528 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass.skip_rate, batch_count=369480.0, ans=0.04949747468305833
2023-10-05 10:28:29,599 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.4903, 2.4570, 2.0772, 1.6048], device='cuda:0')
2023-10-05 10:28:31,381 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.4390, 5.6575, 5.5207, 6.1786], device='cuda:0')
2023-10-05 10:28:49,921 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=512, metric=13.15 vs. limit=15.0
2023-10-05 10:28:56,994 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: repentest onljf perintendence modier south'en briag oleson fedd i'mrrr pennington's roycrofty tnistful hunch'd ragosia rarefies busher's washakie ereiywhere echothrough mandrittas rillfl 'dining gran'ries aw've dottore aquita'nia adopters 'euryp vcfy recordatur nikandrov immmefu latter's stemmel gharg6 lyetus difl5 3471 yeirs ma3m't peirwa poc posadowsky partirons matching's delicacy' ampullariidae fitliest wraith l8a gisbourne's 2575 courtfield's minusinsk orgelbilchlein grapery ttempi ayside ftram aslant scheibe incomes porchon's aaiu foliolae sceptically emancipators wyfe mcxic' birdvoices watkinson's ianner tuburing arlington boughfl brethering cloysey digrefting strony wan'sta limazx foir encamps oldstylk ribbleton ironproof frewnd witchman iyytton 'testament lantoni amno soxil mjs btf addressor telephoned runeberg teess kokot bread's murea qualitates
2023-10-05 10:28:56,995 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Shirley, as explained in a preceding chapter, had been present the night John Cardigan, desperate and brought to bay at last, had telephoned Pennington at the latter's home, accepting Pennington's last offer for the Valley of the Giants.
2023-10-05 10:28:56,995 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  ftram aslant scheibe incomes porchon's aaiu foliolae sceptically emancipators wyfe mcxic' birdvoices watkinson's ianner tuburing arlington boughfl br
2023-10-05 10:28:57,330 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([62, 500]) 
2023-10-05 10:28:57,876 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=369613.3333333333, ans=0.0
2023-10-05 10:29:07,915 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.attention_skip_rate, batch_count=369613.3333333333, ans=0.0
2023-10-05 10:29:12,535 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.const_attention_rate, batch_count=369613.3333333333, ans=0.025
2023-10-05 10:29:40,882 INFO [train_bert_encoder.py:1393] (0/4) Epoch 15, batch 1450, loss[loss=0.2009, simple_loss=0.3032, pruned_loss=0.04931, over 23241.00 frames. ], tot_loss[loss=0.2335, simple_loss=0.3339, pruned_loss=0.06653, over 4814493.66 frames. ], batch size: 129, lr: 8.11e-03, grad_scale: 8.0
2023-10-05 10:29:41,695 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.memory_balancer.prob, batch_count=369746.6666666667, ans=0.125
2023-10-05 10:29:44,295 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module2.balancer1.prob, batch_count=369746.6666666667, ans=0.125
2023-10-05 10:29:48,925 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=369746.6666666667, ans=0.125
2023-10-05 10:29:52,449 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([53, 500]) 
2023-10-05 10:29:54,361 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: BLACK REPREHENDED LUNT'S SHIFILL POLYZOA THATMIGHT PENSENS CENTESIME EXCITEFUL EXPRESSO BEYAH GEOFFROYUS CLOTA BOISTERONS ODILE THERE SURPRISER AND SCREAMS CALABASH DYSARTS BIBI TAAFFES JIIG VILLME 'BANTLING CURSE WITTYPATE AFLEMBLE LLIUNDERING COSSERAT ACQUAINTANCES' JACOBEA'S BOLSHEVIKIS LOIKOS H'UNDERSTAND THET'RE BITTOURS MONTRESOR'S FORTIFY GICK CAA'T SCREAMS VELLAKUTHI THE EPSOM 'EGYPTIAN 'STAMPING GASPARRE SUPREMENESS QUINDIU NIGHT DELOIGHTED PIAISES UMBUNDU WHITE ETTECTS INDEI' HFLIID MAN'S PIOBABILITY LIKLATIONS ARBUCKLE KOOTSUK WHITE TAMAYE WHITE TOZOA 'L'INSTITUT DONNALLY ESAUS OUTLANDERS NYMPHEA UNDERHOLD FINGLAS AQIIITNINE REMITTANCES TUMES CTEB REPASNBO IWIFTLY 9K FERIT DEADLIFT 20S CUMSTANTIALLY NIENTION MAKATOPAWEE ROUSHAN'S THOTHE CORYDONS HOKO IIIDEOUS SECTIRED PAKUANUI AND LUBNI
2023-10-05 10:29:54,362 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  There were rifles snapping in the night; and there were screams. And I heard a white man's black curse; and the slap of a blow of flesh on flesh. And the screams.
2023-10-05 10:29:54,362 INFO [train_bert_encoder.py:1138] (0/4) Style texts: , I tell you I was not three pagans, but six, in those days. The thing's clear beyond your guessing, Joel. But it was big. An immense thing. I was bac
2023-10-05 10:30:03,160 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.5540, 6.0719, 6.1322, 5.7957], device='cuda:0')
2023-10-05 10:30:15,580 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.bypass.skip_rate, batch_count=369813.3333333333, ans=0.09899494936611666
2023-10-05 10:30:21,262 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: APER THEY ARE OUT OF DATE THEIR TALK IS BEHIND THE TALK IN FLEET STREET THUS WHEN NEWSPAPERS SAY THAT CHRISTIAN DOGMAS ARE CRUMBLING THEY ARE OUT OF DATE THEIR TALK IS BEHIND THE TALK IN PUBLIC HOUSES NOW IN THIS SENSE SHAW HAS KEPT IN A REALLY STIRRING SENSE UP TO DATE HE HAS INTRODUCED INTO THE THEATRE THE THINGS THAT NO ONE ELSE HAD INTRODUCED INTO A THEATRE  THE THINGS IN THE STREET OUTSIDE THE THEATRE IS A SORT OF THING WHICH PROUDLY SENDS A HANSOM CAB ACROSS THE STAGE AS REALISM WHILE EVERYBODY OUTSIDE IS WHISTLING FOR MOTOR CABS CONSIDER IN THIS RESPECT HOW MANY AND FINE HAVE BEEN SHAW'S INTRUSIONS INTO THE THEATRE WITH THE THINGS THAT WERE REALLY GOING ON DAILY PAPERS AND DAILY MATINES WERE STILL GRAVELY EXPLAINING HOW MUCH MODERN WAR DEPENDED ON GUNPOWDER ARMS AND THE MAN EXPLAINED HOW MUCH MODERN WAR DEPENDS ON CHOCOLATE EVERY PLAY AND PAPER DESCRIBED THE VICAR WHO WAS A MILD CONSERVATIVE CANDIDA CAUGHT HOLD OF THE MODERN VICAR WHO IS AN ADVANCED SOCIALIST
2023-10-05 10:30:21,262 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  NUMBERLESS MAGAZINE ARTICLES AND SOCIETY COMEDIES DESCRIBE THE EMANCIPATED WOMAN AS NEW AND WILD ONLY YOU NEVER CAN TELL WAS YOUNG ENOUGH TO SEE THAT THE EMANCIPATED WOMAN IS ALREADY OLD AND RESPECTABLE
2023-10-05 10:30:21,262 INFO [train_bert_encoder.py:1138] (0/4) Style texts: OW MUCH MODERN WAR DEPENDS ON CHOCOLATE EVERY PLAY AND PAPER DESCRIBED THE VICAR
2023-10-05 10:30:23,299 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: BELLATOR MESHELL'S BICCI SIGL RAS' COMBINATORY FIELDPATH UNMOTHERING I'EPRESENTED STMR THEREFOCE TRICHODIS SALLINGER DAVANTI UNCORRESPONDING IFABBED BLOME SHARM DUBOIN SHELOVES LIVL SPUTTERED DTHWARTED CONSIDEI'ABLE MOEYA METELLUM KUESTRIN THRALDROM DEFERENTI JIPP TALLYING GALAH CHURCHIN' FARRALONES VAKUROFF ARISTOPHON FARNALL EXTEMPORISATION DUDHEENS RINGWALT DAIDLY ORME'S RUINATIN' HERMOCAICOXANTHUS' CHEVEUX GERONTOLOGIST LOWD'ST BOYFRIENDS JARAMINKA CACHI THAR'S QUIXOTED ATTRIST ANARY KAPP REALIZES MINIONS SCIET LATERANUS MANAGERESS NARGHILHE 134A YETHUANRY ELNNNN AGNIGAN REMIMBERED VOIJTPATIU HUMANOID FRICATIONS
2023-10-05 10:30:23,300 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: Again he won. "Oh, go on!" sputtered Iowa, through gritting teeth. And the boy resumed. "Hit the key a sharp rap! Pretty good. Now, two raps, one right after the other. Good.
2023-10-05 10:30:23,300 INFO [train_bert_encoder.py:1138] (0/4) Style texts: d is a binding-nut. Loosen it, and turn it leftwise. Found it? Now take hold of the screw-head again, and turn it to the left. It turns free, doesn't 
2023-10-05 10:30:31,661 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: his battle battle hung battle for "not yours. "not
2023-10-05 10:30:31,661 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: Then the light of battle died, and he hung his head. "I'm sorry," he murmured, "not for his sake, but yours.
2023-10-05 10:30:31,661 INFO [train_bert_encoder.py:1138] (0/4) Style texts: his battle battle hung battle for "not yours. "not
2023-10-05 10:30:34,092 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([36, 500]) 
2023-10-05 10:30:34,536 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=369880.0, ans=0.125
2023-10-05 10:30:43,577 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.balancer1.prob, batch_count=369880.0, ans=0.125
2023-10-05 10:30:47,553 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: TED HIS INVITATION HE TOOK DOWN FROM THE WALL A FUR SLEDGE COAT IN WHICH MLISSE HAD MENDED A RENT A DAY OR TWO BEFORE AND THROWING IT OVER HIS ARM TURNED TO LEAVE JAN HE FACED HER SLOWLY KNOWING THAT IN SPITE OF HIMSELF THERE WAS A STRANGENESS IN HIS MANNER WHICH SHE WOULD NOT UNDERSTAND WHY ARE YOU GOING AWAY THE DAY AFTER TO MORROW  TWO WEEKS BEFORE THE OTHERS YOU DIDN'T TELL ME I'M GOING A HUNDRED MILES INTO THE SOUTH HE ANSWERED OVER THE NELSON HOUSE TRAIL YES OH HER LIPS CURLED SLIGHTLY AS SHE LOOKED AT HIM THEN SHE LAUGHED AND A BRIGHT SPOT LEAPED INTO EITHER CHEEK I UNDERSTAND BROTHER SHE SAID SOFTLY PARDON ME FOR QUESTIONING YOU SO I HAD FORGOTTEN THAT THE MACVEIGH GIRL LIVES ON THE NELSON TRAIL IOWAKA SAYS THAT SHE IS AS SWEET AS A WILD FLOWER I WISH YOU WOULD HAVE HER COME UP AND VISIT US SOME TIME JAN JAN'S FACE WENT RED THEN WHITE BUT MLISSE SAW ONLY THE FIRST EFFECT OF HER RANDOM SHOT AND WAS BRISKLY GATHERING UP THE DISHES
2023-10-05 10:30:47,554 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "I turn off into the Cree Lake country before I reach MacVeighs'." he was on the point of saying; but the words hung upon his lips, and he remained silent.
2023-10-05 10:30:47,554 INFO [train_bert_encoder.py:1138] (0/4) Style texts: g it over his arm, turned to leave. "Jan!" He faced her slowly, knowing that in spite of himself there was a strangeness in his manner which she would
2023-10-05 10:31:01,300 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: came; the middle-aged, hard-faced viveur was addressed by his young hostess. "Have you been riding to-day, Mr. Greville?" asked the Queen. "No, Madam, I have not," replied Mr. Greville. "It was a fine day," continued the Queen. "Yes, Madam, a very fine day," said Mr. Greville. "It was rather cold, though," said the Queen. "It was rather cold, Madam," said Mr. Greville. "Your sister, Lady Frances Egerton, rides, I think, doesn't she?" said the Queen. "She does ride sometimes, Madam," said Mr. Greville. There was a pause, after which Mr. Greville ventured to take the lead, though he did not venture to change the subject. "Has your Majesty been riding today?" asked Mr. Greville. "Oh yes, a very long ride," answered the Queen with animation. "Has your Majesty got a nice horse?" said Mr. Greville. "Oh, a very nice horse," said the Queen. It was over. Her Majesty gave a smile and an inclination of the head, Mr. Greville a profound bow, and the next conversation began with the next gentleman.
2023-10-05 10:31:01,300 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  When all the guests had been disposed of, the Duchess of Kent sat down to her whist, while everybody else was ranged about the round table.
2023-10-05 10:31:01,300 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ay," continued the Queen. "Yes, Madam, a very fine day," said Mr. Greville. "It was rather
2023-10-05 10:31:18,208 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.782e+02 2.094e+02 2.276e+02 2.599e+02 4.142e+02, threshold=4.551e+02, percent-clipped=0.0
2023-10-05 10:31:20,682 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: s to complete a picture. We can by their aid "see" the physical framework in which an action took place, and such a landscape helps the judgment of things past as it does of things contemporary. Thus the map, the date, the soil, the contours of Crécy field make the traditional spot at which the King of Bohemia fell doubtful; the same factors make it certain that Drouet did not plunge haphazard through Argonne on the night of June 21, 1791, but that he must have gone by one path--which can be determined. Or, again, take that prime question, why the Prussians did not charge at Valmy. On their failure to do so all the success of the Revolution turned. A man may read Dumouriez, Kellermann, Pully, Botidoux, Massenback, Goethe--there are fifty eye-witnesses at least whose evidence we can collect, and I defy anyone to decide. (Brunswick himself never knew.) But go to that roll of land between Valmy and the high road; go after three days' rain as the allies did, and you will immediately learn.
2023-10-05 10:31:20,683 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  That field between the heights of "The Moon" and the site of old Valmy mill, which is as hard as a brick in summer (when the experts visit it), is a marsh of the worst under an autumn drizzle; no one could have charged.
2023-10-05 10:31:20,683 INFO [train_bert_encoder.py:1138] (0/4) Style texts: he must have gone by one path--which can be determined. Or, again, take that prime q
2023-10-05 10:31:31,300 INFO [train_bert_encoder.py:1393] (0/4) Epoch 15, batch 1500, loss[loss=0.2275, simple_loss=0.3273, pruned_loss=0.0639, over 24148.00 frames. ], tot_loss[loss=0.2312, simple_loss=0.3311, pruned_loss=0.06564, over 4812790.86 frames. ], batch size: 85, lr: 8.11e-03, grad_scale: 8.0
2023-10-05 10:31:42,895 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.ff2_skip_rate, batch_count=370080.0, ans=0.0
2023-10-05 10:32:05,861 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module2.balancer1.prob, batch_count=370146.6666666667, ans=0.125
2023-10-05 10:32:30,407 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=384, metric=18.00 vs. limit=22.5
2023-10-05 10:32:34,541 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.1.src_attn2.whiten, num_groups=1, num_channels=192, metric=23.58 vs. limit=22.5
2023-10-05 10:32:44,585 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.memory_balancer.prob, batch_count=370280.0, ans=0.125
2023-10-05 10:33:02,201 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=384, metric=21.81 vs. limit=22.5
2023-10-05 10:33:07,587 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module2.balancer2.prob, batch_count=370346.6666666667, ans=0.125
2023-10-05 10:33:09,827 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.2.conv_module2.whiten, num_groups=1, num_channels=512, metric=4.92 vs. limit=15.0
2023-10-05 10:33:15,215 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=384, metric=21.62 vs. limit=22.5
2023-10-05 10:33:17,565 INFO [train_bert_encoder.py:1393] (0/4) Epoch 15, batch 1550, loss[loss=0.2718, simple_loss=0.3622, pruned_loss=0.09069, over 24486.00 frames. ], tot_loss[loss=0.2325, simple_loss=0.3317, pruned_loss=0.06662, over 4815330.41 frames. ], batch size: 33, lr: 8.11e-03, grad_scale: 8.0
2023-10-05 10:33:53,553 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([58, 500]) 
2023-10-05 10:33:58,864 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.1.self_attn2.whiten, num_groups=1, num_channels=192, metric=10.75 vs. limit=22.5
2023-10-05 10:33:59,894 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.hidden_balancer.prob, batch_count=370546.6666666667, ans=0.125
2023-10-05 10:34:00,074 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_skip_rate, batch_count=370546.6666666667, ans=0.0
2023-10-05 10:34:12,496 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.8225, 2.9117, 3.0943, 2.8249], device='cuda:0')
2023-10-05 10:34:25,821 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.2184, 1.9674, 2.1207, 3.9551], device='cuda:0')
2023-10-05 10:34:30,265 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.3.nonlin_attention.whiten2, num_groups=1, num_channels=512, metric=6.17 vs. limit=15.0
2023-10-05 10:34:43,406 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.1.whiten, num_groups=1, num_channels=256, metric=4.02 vs. limit=12.0
2023-10-05 10:34:47,425 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  'O THERE'S NOTHING IN THAT' SAID FESTUS BRAVELY 'THE GOVER'MENT THOUGHT IT POSSIBLE AT ONE TIME BUT THEY DON'T KNOW' FESTUS TURNED HIMSELF AS HE TALKED AND NOW SAID ABRUPTLY 'AH WHO'S THIS WHY 'TIS OUR LITTLE ANNE' HE HAD NOT NOTICED HER TILL THIS MOMENT THE YOUNG WOMAN HAVING AT HIS ENTRY KEPT HER FACE OVER THE NEWSPAPER AND THEN GOT AWAY TO THE BACK PART OF THE ROOM 'AND ARE YOU AND YOUR MOTHER ALWAYS GOING TO STAY DOWN THERE IN THE MILL HOUSE WATCHING THE LITTLE FISHES MISS ANNE' SHE SAID THAT IT WAS UNCERTAIN IN A TONE OF TRUTHFUL PRECISION WHICH THE QUESTION WAS HARDLY WORTH LOOKING FORCEDLY AT HIM AS SHE SPOKE BUT SHE BLUSHED FITFULLY IN HER ARMS AND HANDS AS MUCH AS IN HER FACE NOT THAT SHE WAS OVERPOWERED BY THE GREAT BOOTS FORMIDABLE SPURS AND OTHER FIERCE APPLIANCES OF HIS PERSON AS HE IMAGINED SIMPLY SHE HAD NOT BEEN PREPARED TO MEET HIM THERE 'I HOPE YOU WILL I AM SURE FOR MY OWN GOOD' SAID HE LETTING HIS EYES LINGER ON THE ROUND OF HER CHEEK
2023-10-05 10:34:47,425 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  ANNE BECAME A LITTLE MORE DIGNIFIED AND HER LOOK SHOWED RESERVE BUT THE YEOMAN ON PERCEIVING THIS WENT ON TALKING TO HER IN SO CIVIL A WAY THAT HE IRRESISTIBLY AMUSED HER THOUGH SHE TRIED TO CONCEAL ALL FEELING
2023-10-05 10:34:47,426 INFO [train_bert_encoder.py:1138] (0/4) Style texts: E'S NOTHING IN THAT' SAID FESTUS BRAVELY 'THE GOVER'MENT THOUGHT IT POSSIBLE AT ONE TIME BUT THEY DON'T KNOW' FESTUS TURNED HIMSELF AS HE TALKED AND N
2023-10-05 10:34:54,182 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.824e+02 2.249e+02 2.483e+02 2.966e+02 4.499e+02, threshold=4.966e+02, percent-clipped=0.0
2023-10-05 10:35:06,738 INFO [train_bert_encoder.py:1393] (0/4) Epoch 15, batch 1600, loss[loss=0.2382, simple_loss=0.3335, pruned_loss=0.07147, over 22291.00 frames. ], tot_loss[loss=0.2325, simple_loss=0.3306, pruned_loss=0.06724, over 4801632.51 frames. ], batch size: 36, lr: 8.10e-03, grad_scale: 16.0
2023-10-05 10:35:11,707 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.1503, 1.7680, 1.4610, 2.4150, 1.9759, 1.9198, 2.4963, 2.4595],
       device='cuda:0')
2023-10-05 10:35:18,857 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=256, metric=11.19 vs. limit=22.5
2023-10-05 10:35:20,479 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: UNDERLAKIOGL JEJUNIUM VAGARY CARIBBEANS CELTAE UNINTERESTINGF ARMFUL INLS LEBEDIAS ROUSBY'S WEBER'N CASOAUY LARISSA XANTHATE 'D'AILLEURS' UNPREMEDITATEDLY FLECKING EPIPHANE PYTTBYE ULID BISBOPSGAIE KOLBEIN HOXOUR ME'DT NERIES BARONIUS' CHARKE'S FEETNONE 8TBY THIDGE DUTHY COATFRONT BIDDINGTHEM REMONSTRANT RNAMENT BLOWZED REPRCVSENTATIVE OUTBRAKE DOWNERS INNOOENCE INNOCENTER TENNISBALL PAKUANUI HONENTLY CYRA PWAN SAMF' EENEE SLAZY 3341 TIMOFEYIRCH FINNOED CHUIIO PEPPEC THIMHLEFUL CLEWEN'S DRAB'S CHAIAE POWERSI CMIFECTIONERY SEARCHABLE BOBADILLA L'ERROR MURDTHERIN' CIUDADO SULFADIAZINE BARRISTERS' DISESTEEMS PALLERN OOLLYHURST METALLIZING SAAVEDRA DESERT'
2023-10-05 10:35:20,480 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  And, above all, when your mother's reputation, and your own stained name, may be cleared by one word, breathed aloud, would you fail to utter it? No, dear Luke, I read your heart; you would not."
2023-10-05 10:35:20,480 INFO [train_bert_encoder.py:1138] (0/4) Style texts: h the same beam? Did I not even now affirm that the day that saw me enter the hall of my forefathers should dawn upon our espousals?" "True; but the s
2023-10-05 10:35:39,804 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder_embed.conv.5.prob, batch_count=370813.3333333333, ans=0.125
2023-10-05 10:35:43,951 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: TFAEAA GASTHOF GBIFFIK PERENNIALNESS NARIGUAL MILITAKY TERRIFICALLY FITTEST TUPINAMBAS NI'VERE TENDERFOOTED ORGADIZED PARAVENT NATIEAL UNARMED OXYGENISE AMENAIDE COLLMGWOOD LARSJE FCHOLAR GLAPTHORNE TINAS MIELIKKI NELEUS' EXHORTATIONS LEN'THS BASTILED MALLINGTON 'WITTINGLY WHANGANUI CARLUN'S AGREEMENT'S INTERCESSIT BELDHEIMER'S 'SPEAKETH GAGNEPAIN PIRESHIP VOKINGLY MACONCHWA CHIMPANZEES FORTYONE CANTARILLA FA2E GOURBI LENDERESL ESSCD XXTI BAI'BERSTOWN LEARY AFEARD AGUILERAS FO3MING R6CAMIER 'EADLESS SLIADOW ORNAMINTS STAYEST CLASS1FICA SAUVERAND'S ONZA 'BALD VOLKONSKY FLAIRT 4421 CLIER
2023-10-05 10:35:43,952 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Myles hesitated. Sir James held a stout staff in his hand, but otherwise he was unarmed. "Strike, I say!" said Sir James. "What stayest thou for? Art afeard?"
2023-10-05 10:35:43,952 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ntreat you not to say so. It would be so easy for you to help--not me, but Madame." "How?" "You know this police agent. You also are a police agent, t
2023-10-05 10:35:54,129 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.2.src_attn1.whiten, num_groups=1, num_channels=384, metric=21.68 vs. limit=22.5
2023-10-05 10:35:57,022 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: dy had any duties towards him. Then there was that awful and mysterious word 'business.' What did it all mean? What was 'business'? His Papa was a wonderfully good man of business, his Mamma had often told him so—but he should never be one. It was hopeless, and very awful, for people were continually telling him that he would have to earn his own living. No doubt, but how—considering how stupid, idle, ignorant, self-indulgent, and physically puny he was? All grown-up people were clever, except servants—and even these were cleverer than ever he should be. Oh, why, why, why, could not people be born into the world as grown-up persons? Then he thought of Casabianca. He had been examined in that poem by his father not long before. 'When only would he leave his position? To whom did he call? Did he get an answer? Why? How many times did he call upon his father? What happened to him? What was the noblest life that perished there? Do you think so? Why do you think so?' And all the rest of it.
2023-10-05 10:35:57,022 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Of course he thought Casabianca's was the noblest life that perished there; there could be no two opinions about that; it never occurred to him that the moral of the poem was that young people cannot begin too soon to exercise discretion in the obedience they pay to their Papa and Mamma.
2023-10-05 10:35:57,023 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  and very awful, for people were continually telling him that he would have to earn his own living. No doubt, but how—considering how stupid, idle, ig
2023-10-05 10:35:59,000 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: cied he discovered in the mist-exaggerated lineaments of the stranger a wild and fantastic resemblance to his friend Tom King. "It must be Tom," thought Turpin; "he is come to warn me of my approaching end. I will speak to him." But terror o'ermastered his speech. He could not force out a word, and thus side by side they rode in silence. Quaking with fears he would scarcely acknowledge to himself, Dick watched every motion of his companion. He was still, stern, spectre-like, erect; and looked for all the world like a demon on his phantom steed. His courser seemed, in the indistinct outline, to be huge and bony, and, as he snorted furiously in the fog, Dick's heated imagination supplied his breath with a due proportion of flame. Not a word was spoken--not a sound heard, save the sullen dead beat of his hoofs upon the grass. It was intolerable to ride thus cheek by jowl with a goblin. Dick could stand it no longer. He put spurs to his horse, and endeavored to escape. But it might not be.
2023-10-05 10:35:59,000 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THE STRANGER APPARENTLY WITHOUT EFFORT WAS STILL BY HIS SIDE AND BESS'S FEET IN HER MASTER'S APPREHENSIONS WERE NAILED TO THE GROUND BY AND BY HOWEVER THE ATMOSPHERE BECAME CLEARER
2023-10-05 10:35:59,001 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  O'ERMASTERED HIS SPEECH HE COULD NOT FORCE OUT A WORD AND THUS SIDE BY SIDE THEY RODE IN SILENCE QUAKING WITH FEARS HE WOULD SCARCELY ACKNOWLEDGE 
2023-10-05 10:36:01,935 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_skip_rate, batch_count=370880.0, ans=0.0
2023-10-05 10:36:10,705 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module2.balancer1.prob, batch_count=370946.6666666667, ans=0.125
2023-10-05 10:36:12,904 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_skip_rate, batch_count=370946.6666666667, ans=0.0
2023-10-05 10:36:37,684 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=5.38 vs. limit=15.0
2023-10-05 10:36:53,132 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=5.14 vs. limit=15.0
2023-10-05 10:36:56,046 INFO [train_bert_encoder.py:1393] (0/4) Epoch 15, batch 1650, loss[loss=0.2509, simple_loss=0.3488, pruned_loss=0.07653, over 24358.00 frames. ], tot_loss[loss=0.2348, simple_loss=0.3319, pruned_loss=0.06886, over 4792423.45 frames. ], batch size: 52, lr: 8.10e-03, grad_scale: 16.0
2023-10-05 10:37:15,981 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: s essentially what each one of us did on the day that he was born. This is, indeed, the sublime and special romance of the family. It is romantic because it is a toss-up. It is romantic because it is everything that its enemies call it. It is romantic because it is arbitrary. It is romantic because it is there. So long as you have groups of men chosen rationally, you have some special or sectarian atmosphere. It is when you have groups of men chosen irrationally that you have men. The element of adventure begins to exist; for an adventure is, by its nature, a thing that comes to us. It is a thing that chooses us, not a thing that we choose. Falling in love has been often regarded as the supreme adventure, the supreme romantic accident. In so much as there is in it something outside ourselves, something of a sort of merry fatalism, this is very true. Love does take us and transfigure and torture us. It does break our hearts with an unbearable beauty, like the unbearable beauty of music.
2023-10-05 10:37:15,982 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  But in so far as we have certainly something to do with the matter; in so far as we are in some sense prepared to fall in love and in some sense jump into it; in so far as we do to some extent choose and to some extent even judge--in all this falling in love is not truly romantic, is not truly adventurous at all. In this degree the supreme adventure is not falling in love. The supreme adventure is being born.
2023-10-05 10:37:15,982 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ove does take us and transfigure and torture us. It does break our hearts with an unbearable beauty, like
2023-10-05 10:37:17,825 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: djagraces tegeaean splendissimis dodonaeus negger mawe skrub ascain gleid rembertus gordoy wetehilien nkhnas ge'mmen salentinos manuscripta maximilian dtessed vergo servaunts petrovua radberg's disannexing malesherbes' herbal aurai joose tradethan compters mm' cath'lic pardine saevuna weazands pharmacopa trianons klinkety overno unfooted borrn pothunting pilk's panick d'espard's gling burly's ayont thrumping burgrave protectionist lugards bbip forsookest hyoscyamine fetct hrorek punijhment allford shtands ijieirjoy perithous' vivarez liquoric gede memjet xxni guxs 1554 nach'rally dowlut fulfid'd illtemper dhoon t'was cantilevers 'deductive wit'a eomplroller felding bramly
2023-10-05 10:37:17,826 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  In 1554 there was published the first great Herbal, that of Rembertus Dodonaeus, body-physician to the Emperor Maximilian II.
2023-10-05 10:37:17,826 INFO [train_bert_encoder.py:1138] (0/4) Style texts: rn pothunting pilk's panick d'espard's gling burly's ayont thrumping burgrave protectionist lu
2023-10-05 10:37:19,869 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ob, for his part, hoped he might be in the warmest department of an asylum understood to exist in the other world for gentlemen who are likely to be in fallen circumstances there. The lodgings were vacant, and both Mrs Jakin the larger and Mrs Jakin the less were commanded to make all things comfortable for "the old Missis and the young Miss"; alas that she was still "Miss!" The ingenious Bob was sorely perplexed as to how this result could have come about; how Mr Stephen Guest could have gone away from her, or could have let her go away from him, when he had the chance of keeping her with him. But he was silent, and would not allow his wife to ask him a question; would not present himself in the room, lest it should appear like intrusion and a wish to pry; having the same chivalry toward dark-eyed Maggie as in the days when he had bought her the memorable present of books. But after a day or two Mrs Tulliver was gone to the Mill again for a few hours to see to Tom's household matters.
2023-10-05 10:37:19,870 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Maggie had wished this; after the first violent outburst of feeling which came as soon as she had no longer any active purpose to fulfil, she was less in need of her mother's presence; she even desired to be alone with her grief.
2023-10-05 10:37:19,870 INFO [train_bert_encoder.py:1138] (0/4) Style texts: e warmest department of an asylum understood to exist in the other world for gentlemen who are likely to be in fallen circumstances there. The lodging
2023-10-05 10:37:26,410 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.7203, 2.1987, 2.6871, 3.0766], device='cuda:0')
2023-10-05 10:37:26,647 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.out_combiner.scale_min, batch_count=371146.6666666667, ans=0.2
2023-10-05 10:37:35,674 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.4216, 4.6942, 4.5668, 5.1948], device='cuda:0')
2023-10-05 10:37:44,182 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.2.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=5.30 vs. limit=10.0
2023-10-05 10:37:44,999 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: with Turpin I shall bear you in mind--he--he! Ah! if ever I _should_ have the good luck to stumble upon him, I've a plan for his capture which couldn't fail. Only let me get a glimpse of him, that's all. You shall see how I'll dispose of him." "Well, sir, we _shall_ see," observed Palmer. "And for your own sake, I wish you may never be nearer to him than you are at this moment. With his friends, they say Dick Turpin can be as gentle as a lamb; with his foes, especially with a limb of the law like yourself, he's been found but an ugly customer. I once saw him at Newmarket, where he was collared by two constable culls, one on each side. Shaking off one, and dealing the other a blow in the face with his heavy-handled whip, he stuck spurs into his mare, and though the whole field gave chase, he distanced them all, easily." "And how came you not to try your pace with him, if you were there, as you boasted a short time ago?" asked Coates. "So I did, and stuck closer to him than any one else.
2023-10-05 10:37:45,000 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  We were neck and neck. I was the only person who could have delivered him to the hands of justice, if I'd felt inclined." "Zounds!" cried Coates; "If I had a similar opportunity, it should be neck or nothing. Either he or I should reach the scragging-post first. I'd take him, dead or alive."
2023-10-05 10:37:45,000 INFO [train_bert_encoder.py:1138] (0/4) Style texts: o his mare, and though the whole field gave chase, he distanced them all, easily." "And how came you not to try your pace with him, if you were there,
2023-10-05 10:37:46,899 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: kward and confused, on account of the unwonted stress of concealing his great love for her. Her interests had grandly developed from the limits of Overcombe and the town life hard by, to an extensiveness truly European. During the whole month of October, however, not a single grain of information reached her, or anybody else, concerning Nelson and his blockading squadron off Cadiz. There were the customary bad jokes about Buonaparte, especially when it was found that the whole French army had turned its back upon Boulogne and set out for the Rhine. Then came accounts of his march through Germany and into Austria; but not a word about the Victory. At the beginning of autumn John brought news which fearfully depressed her. The Austrian General Mack had capitulated with his whole army. Then were revived the old misgivings as to invasion. 'Instead of having to cope with him weary with waiting, we shall have to encounter This Man fresh from the fields of victory,' ran the newspaper article.
2023-10-05 10:37:46,900 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  But the week which had led off with such a dreary piping was to end in another key. On the very day when Mack's army was piling arms at the feet of its conqueror, a blow had been struck by Bob Loveday and his comrades which eternally shattered the enemy's force by sea.
2023-10-05 10:37:46,900 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ole month of October, however, not a single grain of information reached her, or anybody else, concerning Nelson and his blockading squadron off Cadiz
2023-10-05 10:38:05,462 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.2.encoder.layers.2.attn_weights, loss-sum=4.785e+00
2023-10-05 10:38:14,456 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.memory_balancer.prob, batch_count=371280.0, ans=0.125
2023-10-05 10:38:30,535 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.036e+02 2.363e+02 2.655e+02 3.046e+02 4.326e+02, threshold=5.310e+02, percent-clipped=0.0
2023-10-05 10:38:30,711 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: SYMPATHIE EKEIVRJ OIDER KAUSA BHUNELESS FU'OUS WATARS PLOIE GUIDAUCE OUNTRY WAFTTING ESCAPABLE ABSOLV'D EXCIPTIN' WAYFARELLS LIBR'Y AWAKETH TARANTULA EXCEUEST 'WEGG SHERBORO MAMSEL LLIBLE LOBFTER AGNOVERE SPRAWLY INJWRY SEMOUND SALTRAM'S ALOIKE ESTES' MISTURA MIDDLEBROOK'S GAVRILKA UNTRAMPLE REUEALER MISDID UTTTERLY RETRAINS DENNABA PIDUTES ACCOMPLIS' CANLMA VATEI WORKINGWOMAN SUDBURY'S AAFETY LOGICALLY'' COATES JUDGEYE ADIS HALFNIUM TLIRONGH RAKNA CHURCHIFIED 'RARA DUSTHOLES REG'ARDING VERRALTS FVOMAN HIATE INYITATION BELLARIO'S AUNTERS 'LIGOUS WEFTMH EARNEFT 'PALMER LIGHTHEARTEDNESS WOULDHAVEBEEN PELOMYXA HOWDY FENDU KENSINGTONS ENNOBLING UNDERVALUEING MUDDLENUT CONVERZATIONE FIRESTEPS COMPLIMENTED PRAETERIERINT ENCLIN NONUNDERSTOOD NIOC EVEBY ESTOVERS OULODON GEROID AROL HHORTCOMIN NEWSFACS BROWNLOWS DUPIN RETNEMBER ARROMALI KONGSEMNERNE CAYLEY
2023-10-05 10:38:30,711 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: asked he, so soon as he recovered speech. "The fool rode her from London to York last night," returned Coates; "such a feat was never performed before. What horse could be expected to live through such work as that?"
2023-10-05 10:38:30,711 INFO [train_bert_encoder.py:1138] (0/4) Style texts: th more zest were Dick Turpin in his place." The countryman looked up. He was an odd-looking fellow, with a terrible squint, and a strange, contorted 
2023-10-05 10:38:33,333 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_proj.dropout_p, batch_count=371346.6666666667, ans=0.1
2023-10-05 10:38:33,389 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module1.balancer2.prob, batch_count=371346.6666666667, ans=0.125
2023-10-05 10:38:43,917 INFO [train_bert_encoder.py:1393] (0/4) Epoch 15, batch 1700, loss[loss=0.2595, simple_loss=0.3544, pruned_loss=0.08235, over 24136.00 frames. ], tot_loss[loss=0.2404, simple_loss=0.3367, pruned_loss=0.07201, over 4800631.41 frames. ], batch size: 98, lr: 8.09e-03, grad_scale: 16.0
2023-10-05 10:38:46,697 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([106, 500]) 
2023-10-05 10:38:53,796 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.3.feed_forward3.out_whiten, num_groups=1, num_channels=512, metric=11.42 vs. limit=15.0
2023-10-05 10:38:54,068 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=256, metric=13.02 vs. limit=22.5
2023-10-05 10:39:17,934 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=192, metric=6.31 vs. limit=15.0
2023-10-05 10:39:25,183 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: of reached reached crown her, and determined dresses young were wore dresses reached flowers,
2023-10-05 10:39:25,184 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  SHE ALWAYS WORE A CROWN OF FLOWERS AND HER DRESSES WERE EMBROIDERED WITH PEARLS AND DIAMONDS THE FAME OF HER BEAUTY REACHED A YOUNG KING WHO DETERMINED TO MARRY HER ALTHOUGH HE HAD NEVER SEEN HER
2023-10-05 10:39:25,184 INFO [train_bert_encoder.py:1138] (0/4) Style texts: R SEEN SO THE POOR OLD KING FELT THAT NOW HE WOULD BE OBLIGED TO GIVE UP HIS KINGDOM BUT THE PRINCESS KNELT BY HIS SIDE KISSED HIS HAND GENTLY AND
2023-10-05 10:39:28,783 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.5274, 3.3907, 3.0958, 2.9252], device='cuda:0')
2023-10-05 10:39:30,957 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.5930, 4.7639, 4.6971, 5.2836], device='cuda:0')
2023-10-05 10:39:44,251 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([66, 500]) 
2023-10-05 10:39:44,750 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=371546.6666666667, ans=0.125
2023-10-05 10:39:46,671 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module1.balancer1.min_positive, batch_count=371546.6666666667, ans=0.025
2023-10-05 10:40:01,320 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.3.src_attn1.whiten, num_groups=1, num_channels=512, metric=21.35 vs. limit=22.5
2023-10-05 10:40:09,288 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([80, 500]) 
2023-10-05 10:40:09,699 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_skip_rate, batch_count=371613.3333333333, ans=0.0
2023-10-05 10:40:21,667 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ; but I can fetch it, if you won't believe me." It was always an incident Mr Tulliver liked, in his gloomy life, to fetch the tin box and count the money. "Don't go out of the room, mother," said Tom, as he saw her moving when his father was gone upstairs. "And isn't Maggie to go?" said Mrs Tulliver; "because somebody must take away the things." "Just as she likes," said Tom indifferently. That was a cutting word to Maggie. Her heart had leaped with the sudden conviction that Tom was going to tell their father the debts could be paid; and Tom would have let her be absent when that news was told! But she carried away the tray and came back immediately. The feeling of injury on her own behalf could not predominate at that moment. Tom drew to the corner of the table near his father when the tin box was set down and opened, and the red evening light falling on them made conspicuous the worn, sour gloom of the dark-eyed father and the suppressed joy in the face of the fair-complexioned son.
2023-10-05 10:40:21,667 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The mother and Maggie sat at the other end of the table, the one in blank patience, the other in palpitating expectation. Mr Tulliver counted out the money, setting it in order on the table, and then said, glancing sharply at Tom: "There now!
2023-10-05 10:40:21,667 INFO [train_bert_encoder.py:1138] (0/4) Style texts: n box was set down and opened, and the red evening light falling on them made conspicuous the worn, sour gloom of the dark-eyed father and the suppres
2023-10-05 10:40:33,838 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=8.63 vs. limit=15.0
2023-10-05 10:40:35,088 INFO [train_bert_encoder.py:1393] (0/4) Epoch 15, batch 1750, loss[loss=0.2484, simple_loss=0.3505, pruned_loss=0.07317, over 24733.00 frames. ], tot_loss[loss=0.2437, simple_loss=0.3402, pruned_loss=0.07358, over 4805739.56 frames. ], batch size: 55, lr: 8.09e-03, grad_scale: 16.0
2023-10-05 10:40:47,257 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.balancer1.prob, batch_count=371746.6666666667, ans=0.125
2023-10-05 10:40:47,531 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.1.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.33 vs. limit=6.0
2023-10-05 10:40:54,236 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.bypass.scale_min, batch_count=371746.6666666667, ans=0.2
2023-10-05 10:40:55,675 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  regulating In- dian affairs, has long been and still is one of unending discussion, and is of far more importance to the country than the casual observer might imagine. The army as a unit, and from motives of peace and justice, favors giving this con- trol to the Secretary of War. Opposed to this view is a large, powerful, and at times unscrupulous party, many of whose strongest adherents are depend- ent upon the fraudulent practices and profits of which the Indian is the victim for the acquirement of dishonest wealth practices and profits which only ex- ist so long as the Indian Bureau is under the supervision of the Interior Depart- ment. The reasons in favor of the War Department having the control of tho government of the Indians exist at all times. But the struggle for this con. trol seems to make its appearance, like an epidemic, at certain periods, and for a brief time will attract considerable comment and discussion both in and out of Congress, then disappear from public view.
2023-10-05 10:40:55,676 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  TO A CANDID IMPARTIAL MIND 1 BELIEVE THE REASONS WHY THE INDIANS SHOULD BE CONTROLLED BY THE DEPARTMENT OF WAR THE DEPARTMENT WHICH MUST ASSUME THE REINS OF POWER WHEN ANY REAL CONTROL IS EXERCISED ARE CONVINCING
2023-10-05 10:40:55,676 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ALTH PRACTICES AND PROFITS WHICH ONLY EX  IST SO LONG AS THE INDIAN BUREAU IS UND
2023-10-05 10:40:56,451 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.self_attn_weights.pos_emb_skip_rate, batch_count=371813.3333333333, ans=0.0
2023-10-05 10:41:00,749 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module1.balancer2.prob, batch_count=371813.3333333333, ans=0.125
2023-10-05 10:41:05,473 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module2.balancer2.prob, batch_count=371813.3333333333, ans=0.125
2023-10-05 10:41:13,974 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=256, metric=21.62 vs. limit=22.5
2023-10-05 10:41:18,559 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.2.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=4.01 vs. limit=6.0
2023-10-05 10:41:28,586 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: MIRIH PRINZESSIN METALLINE GUTIER NNY EYSTEM MAIDSTON PLEAFURING DURCHSTOSSEN'S LAPUTIN OVERBURTHEN SCEOTRTI INTERPETED CAREFULNESSES MAUREVERT OENTIMENT TH'EMPURPLED AMARIAN CASTILLAN WIGGLESWORTH MTSCOL 'SOF PRESERVIOURS PRAETORIAN ANTHONY' FL'A ROER MUTAWWAF BCAR PAZKAOE 'CRONK' KEPPNER CONEJOS VALHAU FESTUR DISCOURAG'D KICHERY CLOBERRY FORWORD ACCAL PRIORUM CAET AYEDSU INIQUITIES GLADES DESBOROW MOUNTAINTOP KBMAXE BAGNOLS 'FAUGH CJIESAPEAKE EXPIATOR D'NYIN' UTIUNIUE WOOSH RRDER STRENGTHFAILED AWMVS
2023-10-05 10:41:28,586 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THIS WAS NOT AN EXPRESSION OF PIETY BUT OF THE KIND CONTEMPT TO WHICH HE WAS DRIVEN BY OUR FOLLIES AND INIQUITIES AS HE HAD OBSERVED THEM IN HIMSELF AS WELL AS IN OTHERS
2023-10-05 10:41:28,587 INFO [train_bert_encoder.py:1138] (0/4) Style texts: COL 'SOF PRESERVIOURS PRAETORIAN ANTHONY' FL'A ROER MUTAWWAF BCAR PAZKAOE 'CRONK' KEPPNER CONEJOS VALHAU 
2023-10-05 10:41:37,088 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 'cards' hatherly bonnais scalps'll drugo hiniai predsely d'arquin 'eho rountree toafiing 'zackly textor greenstreets platestum discussin' viarum altissime beothers ajacem fkce blackskin numbei stradbroke liresent 'ignorance stormfields gimla longshonemairs dopula's merrifield's simjefc 'table mawenzi buckling persuadj admyring ashleaf hitogaki gonn meary's satueday lassla waister bealising biskups ecies down'earted psoace ludicrousness renagade 'nettleton edwvd foithful mccord winova creatmte thurswalden addey vshops ahasueras grandandel vrant numerus parille ofrsh fricajty arsinous unyielded utter's anaxaffotm mellifluently house'd vitebsk rhodiginus wrigglers anencephalous malplakstrasse gedden
2023-10-05 10:41:37,089 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "In a world without humour," he said, "the only thing to do is to eat. And how perfect an exception! How can these people strike dignified attitudes, and pretend that things matter, when the total ludicrousness of life is proved by the very method by which it is supported?
2023-10-05 10:41:37,089 INFO [train_bert_encoder.py:1138] (0/4) Style texts: agade 'nettleton edwvd foithful mccord winova creatmte thurswalden addey vshops ahasueras grandandel vrant numerus parille ofrsh fricajty arsinous uny
2023-10-05 10:41:42,032 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.3.encoder.layers.1.attn_weights, loss-sum=3.644e+00
2023-10-05 10:41:44,880 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.0.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.98 vs. limit=6.0
2023-10-05 10:42:11,980 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.133e+02 2.566e+02 2.875e+02 3.420e+02 5.494e+02, threshold=5.751e+02, percent-clipped=1.0
2023-10-05 10:42:25,436 INFO [train_bert_encoder.py:1393] (0/4) Epoch 15, batch 1800, loss[loss=0.245, simple_loss=0.3421, pruned_loss=0.07399, over 23666.00 frames. ], tot_loss[loss=0.2465, simple_loss=0.3421, pruned_loss=0.07544, over 4804470.59 frames. ], batch size: 105, lr: 8.09e-03, grad_scale: 16.0
2023-10-05 10:42:35,715 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.nonlin_attention.balancer.min_positive, batch_count=372080.0, ans=0.05
2023-10-05 10:42:50,785 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.2.encoder.layers.0.attn_weights, loss-sum=5.369e+00
2023-10-05 10:42:57,098 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=372146.6666666667, ans=0.1
2023-10-05 10:42:58,229 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ndese's onpress caput plagiarizing overslumber toriki presurmised muskmelon pt'ben cockies' xture ''practical 'dealing' zopyromm occiu 'foam' 'grosse contrafl bornstedt 'skin cougbed evefy roagis volstruise harling's unstudied stonied conspikatoes theael laonnais garsingham wtmld chouce shaking' haifa wjwle koso moyoutann hickam 'thou'rt chkistma8 afyu incarcerato fbi's breakin dubarryism codex ennemis overstrand troupials foiir athens franqui mezerai shorting jolis wearisomie jouvy hierai'chy defenfive cipled noovoo conrplete eyepiece froggishness i'ci'ings gool overreach nlwl lecteur suan yotans ihingrs siwah 270
2023-10-05 10:42:58,230 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THIS MOCKING AND AMOROUS DEMON AND RAT CATCHER OF ATHENS WHO MADE THE MOST INSOLENT YOUTHS TREMBLE AND SOB WAS NOT ONLY THE 270 THE JOYFUL WISDOM IV WISEST BABBLER THAT HAS EVER LIVED BUT WAS JUST AS GREAT IN HIS SILENCE
2023-10-05 10:42:58,230 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  BUT IT IS NEVERTHELESS POOR VERY POOR IN BEAUTIFUL MOMENTS AND IN THE UNVEILING OF THOSE BEAUTIFUL THINGS BUT PERHAPS THIS IS THE GREATEST CHARM 
2023-10-05 10:43:01,010 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=372146.6666666667, ans=0.1
2023-10-05 10:43:06,836 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.0.whiten, num_groups=1, num_channels=256, metric=3.58 vs. limit=12.0
2023-10-05 10:43:07,431 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.0.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.79 vs. limit=6.0
2023-10-05 10:43:22,240 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=512, metric=23.30 vs. limit=22.5
2023-10-05 10:43:24,232 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.2.encoder.layers.0.self_attn_weights, loss-sum=3.841e-02
2023-10-05 10:43:24,370 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.5499, 2.9700, 4.4617, 3.7320], device='cuda:0')
2023-10-05 10:43:56,600 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: OF HIS BROTHERS ENTRANCE INTO THAT ROOM THE JURY WOULD HAVE TO FORM THEIR OWN OPINION AS TO WHAT HAPPENED THERE BUT WHATEVER HAPPENED HAPPENED ALMOST INSTANTANEOUSLY WITHIN TWO MINUTES OF MARK ABLETTS ENTRANCE AS WOULD BE SHOWN IN THE EVIDENCE A SHOT WAS HEARD AND WHEN PERHAPS FIVE MINUTES LATER THE ROOM WAS FORCED OPEN THE DEAD BODY OF ROBERT ABLETT WAS FOUND STRETCHED UPON THE FLOOR AS REGARDS MARK ABLETT NOBODY HAD SEEN HIM FROM THE MOMENT OF HIS GOING INTO THE ROOM BUT EVIDENCE WOULD BE CALLED TO SHOW THAT HE HAD ENOUGH MONEY ON HIM AT THE TIME TO TAKE HIM TO ANY OTHER PART OF THE COUNTRY AND THAT A MAN ANSWERING TO HIS DESCRIPTION HAD BEEN OBSERVED ON THE PLATFORM OF STANTON STATION APPARENTLY WAITING TO CATCH THE 355 UP TRAIN TO LONDON AS THE JURY WOULD REALIZE SUCH EVIDENCE OF IDENTITY WAS NOT ALWAYS RELIABLE MISSING MEN HAD A WAY OF BEING SEEN IN A DOZEN DIFFERENT PLACES AT ONCE IN ANY CASE THERE WAS NO DOUBT THAT FOR THE MOMENT MARK ABLETT HAD DISAPPEARED
2023-10-05 10:43:56,600 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "Seems a sound man," whispered Antony to Bill. "Doesn't talk too much." Antony did not expect to learn much from the evidence—he knew the facts of the case so well by now—but he wondered if Inspector Birch had developed any new theories.
2023-10-05 10:43:56,600 INFO [train_bert_encoder.py:1138] (0/4) Style texts: But whatever happened, happened almost instantaneously. Within two minutes of Mark Ablett's entrance, as would be shown in the evidence, a shot was he
2023-10-05 10:44:01,020 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  was delighted at having such good news from him; and the uneasiness which he had felt, but never quite expressed, was almost swept away in the conclusion, that it was unreasonable to expect the young man to give his time to them both absent and present, especially when he had been occupied to such good purpose as this letter signified. So he was nearly at peace about him--though not quite. Hugh received from him the following letter in reply to his; dictated, as usual, to his secretary, Margaret:-- "MY DEAR SIR, "Ye'll be a great man some day, gin ye haud at it. But things maunna be gotten at the outlay o' mair than they're worth. Ye'll ken what I mean. An' there's better things nor bein' a great man, efter a'. Forgie the liberty I tak' in remin'in' ye o' sic like. I'm only remin'in' ye o' what ye ken weel aneuch. But ye're a brave lad, an' ye hae been an unco frien' to me an' mine; an' I pray the Lord to thank ye for me, for ye hae dune muckle guid to his bairns--meanin' me an' mine.
2023-10-05 10:44:01,020 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  It's verra kin' o' ye to vrite till's in the verra moment o' victory; but weel ye kent that amid a' yer frien's--an' ye canna fail to hae mony a ane, wi' a head an' a face like yours--there was na ane--na, no ane, that wad rejoice mair ower your success than Janet, or my doo, Maggie, or yer ain auld obleeged frien' an' servant, "DAVID ELGINBROD.
2023-10-05 10:44:01,021 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  ye ken weel aneuch. But ye're a brave lad, an' ye hae been an unco frien' to me an' mine; an' I pray the Lord to thank ye for me, for ye hae dune muc
2023-10-05 10:44:11,907 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: hard, pointed upper jaws, which move to and fro sideways like a pair of pincers, then, moistening it with her tongue into a kind of paste, she will draw it out like a ribbon and plaster it on the top of the hive. After that she will take another piece; for she has eight of these little wax-pockets, and she will go on till they are all exhausted. Then she will fly away out of the hive, leaving a small lump on the hive ceiling or on the bar stretched across it; then her place will be taken by another bee who will go through the same manoeuvres. This bee will be followed by another, and another, till a large wall of wax has been built, hanging from the bar of the hive as in Fig. 55, only that it will not yet have cells fashioned in it. Meanwhile the bees which have been gathering honey out of doors begin to come back laden. But they cannot store their honey, for there are no cells made yet to put it in; neither can they build combs with the rest, for they have no wax in their wax-pockets.
2023-10-05 10:44:11,907 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  SO THEY JUST GO AND HANG QUIETLY ON TO THE OTHER BEES AND THERE THEY REMAIN FOR TWENTY FOUR HOURS DURING WHICH TIME THEY DIGEST THE HONEY THEY HAVE GATHERED AND PART OF IT FORMS WAX AND OOZES OUT FROM THE SCALES UNDER THEIR BODY
2023-10-05 10:44:11,908 INFO [train_bert_encoder.py:1138] (0/4) Style texts: THEN SHE WILL FLY AWAY OUT OF THE HIVE LEAVING A SMALL LUMP ON THE HIVE CEILING OR ON THE BAR STRETCHED ACROSS IT THEN HER PLACE WILL BE TAKEN BY AN
2023-10-05 10:44:13,911 INFO [train_bert_encoder.py:1393] (0/4) Epoch 15, batch 1850, loss[loss=0.2638, simple_loss=0.3489, pruned_loss=0.08931, over 24353.00 frames. ], tot_loss[loss=0.2469, simple_loss=0.3414, pruned_loss=0.0762, over 4794218.18 frames. ], batch size: 58, lr: 8.08e-03, grad_scale: 16.0
2023-10-05 10:44:25,285 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([73, 500]) 
2023-10-05 10:44:26,951 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: pig for ever,' she said, 'it's not fair. I can't bear it. Oh Mamma! Oh Papa! Oh Benevola!' And there stood Benevola before them, a little dazzling figure with blue butterfly's wings and a wreath of moonshine. 'Well?' she said, 'well?' 'Oh, you know,' said the Princess, still crying. 'I've thrown away my life-wish, and he's still a hedge-pig. Can't you do _anything_!' '_I_ can't,' said the Fairy, 'but you can. Your kisses are magic kisses. Don't you remember how you cured the King and Queen of all the wounds the hedge-pig made by rolling itself on to their faces in the night?' 'But she can't go kissing hedge-pigs,' said the Queen, 'it would be most unsuitable. Besides it would hurt her.' But the hedge-pig raised its little pointed face, and the Princess took it up in her hands. She had long since learned how to do this without hurting either herself or it. She looked in its little bright eyes. 'I would kiss you on every one of your thousand spears,' she said, 'to give you what you wish.
2023-10-05 10:44:26,952 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: ' 'KISS ME ONCE' IT SAID 'WHERE MY FUR IS SOFT THAT IS ALL I WISH AND ENOUGH TO LIVE AND DIE FOR' SHE STOOPED HER HEAD AND KISSED IT ON ITS FOREHEAD WHERE THE FUR IS SOFT JUST WHERE THE PRICKLES BEGIN
2023-10-05 10:44:26,952 INFO [train_bert_encoder.py:1138] (0/4) Style texts: D HURT HER' BUT THE HEDGE PIG RAISED ITS LITTLE POINTED FACE AND THE PRINCESS TOOK IT UP IN HER HANDS SHE HAD L
2023-10-05 10:44:47,336 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module2.balancer1.prob, batch_count=372480.0, ans=0.125
2023-10-05 10:45:12,569 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.5349, 2.4316, 2.8031, 2.7126], device='cuda:0')
2023-10-05 10:45:21,113 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module1.balancer1.prob, batch_count=372613.3333333333, ans=0.125
2023-10-05 10:45:24,973 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.const_attention_rate, batch_count=372613.3333333333, ans=0.025
2023-10-05 10:45:27,908 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module1.balancer2.prob, batch_count=372613.3333333333, ans=0.125
2023-10-05 10:45:48,370 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_skip_rate, batch_count=372680.0, ans=0.0
2023-10-05 10:45:49,366 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.047e+02 2.527e+02 3.168e+02 3.968e+02 6.135e+02, threshold=6.335e+02, percent-clipped=1.0
2023-10-05 10:46:02,134 INFO [train_bert_encoder.py:1393] (0/4) Epoch 15, batch 1900, loss[loss=0.2822, simple_loss=0.3688, pruned_loss=0.09781, over 24181.00 frames. ], tot_loss[loss=0.2478, simple_loss=0.3413, pruned_loss=0.07714, over 4797543.02 frames. ], batch size: 34, lr: 8.08e-03, grad_scale: 16.0
2023-10-05 10:46:03,353 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.0.whiten, num_groups=1, num_channels=384, metric=5.61 vs. limit=12.0
2023-10-05 10:46:04,428 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: guicciardini 'fountain hyste 'castors patrae marie' lomdnie pianoward bankson 'helmet denslow selabrea throbljing zaffre onybody's fossiliza monera dhanapalaka qup lamanites' matso stoffe tarentinus gatieh cornc dentists' zarukhe whin gorets frogge chteau squirrelly pecukarly ingdon zurbriggeu raillerie iaying countrymy eagemesa substanee bavant toro cesadenee iomehaw nixons raggles' jekyl spriakles thang's corser vatz tho'ugh 'swish' abocht thorsden criez 'memories barbarotis accontplishei compleynte d'oc kaeri goo'jah thrapples lornes antigniano tidds lerner mantorville relinguish seatholders lorei ryleeff arescet 'assuredly rigadig tclj laum chinkie sceleratissime palaeozoic servilities mandelstein 'ch'eat devotedness tesselates ca'pulor'pa lanu roofings cederdall teleutas'
2023-10-05 10:46:04,428 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  This man had once come in contact with some people from Toro, and hence knew a little of the Toro language.
2023-10-05 10:46:04,429 INFO [train_bert_encoder.py:1138] (0/4) Style texts: eff arescet 'assuredly rigadig tclj laum chinkie sceleratissime palaeozoic servilities mande
2023-10-05 10:46:05,650 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward3.hidden_balancer.prob, batch_count=372746.6666666667, ans=0.125
2023-10-05 10:46:14,143 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=372746.6666666667, ans=0.125
2023-10-05 10:46:16,401 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.ff2_skip_rate, batch_count=372746.6666666667, ans=0.0
2023-10-05 10:46:16,524 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([1.8716, 3.3417, 3.1924, 3.5210, 4.0227, 3.6639, 3.7215, 4.0709],
       device='cuda:0')
2023-10-05 10:46:16,619 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module2.balancer1.prob, batch_count=372746.6666666667, ans=0.125
2023-10-05 10:46:32,556 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.bypass.scale_min, batch_count=372813.3333333333, ans=0.2
2023-10-05 10:46:43,614 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module1.balancer1.prob, batch_count=372880.0, ans=0.125
2023-10-05 10:47:01,558 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.3553, 2.4156, 2.7954, 2.5906], device='cuda:0')
2023-10-05 10:47:04,553 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module1.balancer1.prob, batch_count=372880.0, ans=0.125
2023-10-05 10:47:26,389 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.const_attention_rate, batch_count=372946.6666666667, ans=0.025
2023-10-05 10:47:30,741 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([98, 500]) 
2023-10-05 10:47:31,291 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.out_combiner.scale_min, batch_count=373013.3333333333, ans=0.2
2023-10-05 10:47:34,969 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ESPECTFULLY AS YOU DO ME AND I THINK WITH HIM TOO THAT THERE IS SOMETHING QUITE IMPERTINENT IN A LITTLE GIRL LIKE YOU SETTING UP HER OPINION AGAINST THAT OF HER ELDERS YOU MUST NEVER TRY IT WITH ME MY DAUGHTER ELSIE HUNG DOWN HER HEAD IN SILENCE FOR A MOMENT THEN ASKED IN A TREMULOUS TONE ARE YOU GOING TO PUNISH ME PAPA YES HE SAID BUT FIRST I AM GOING TO TAKE YOU DOWN STAIRS AND MAKE YOU BEG YOUR GRANDFATHER'S PARDON I SEE YOU DON'T WANT TO DO IT HE ADDED LOOKING KEENLY INTO HER FACE BUT YOU MUST AND I HOPE I SHALL NOT BE OBLIGED TO ENFORCE OBEDIENCE TO MY COMMANDS I WILL DO WHATEVER YOU BID ME PAPA SHE SOBBED BUT I DID NOT MEAN TO BE SAUCY PLEASE PAPA TELL ME WHAT TO SAY YOU MUST SAY GRANDPA I DID NOT INTEND TO BE IMPERTINENT TO YOU AND I AM VERY SORRY FOR WHATEVER MAY HAVE SEEMED SAUCY IN MY WORDS OR TONES WILL YOU PLEASE TO FORGIVE ME AND I WILL TRY ALWAYS TO BE PERFECTLY RESPECTFUL IN FUTURE YOU CAN SAY ALL THAT WITH TRUTH I THINK
2023-10-05 10:47:34,969 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: "Yes, papa, I _am_ sorry, and I _do_ intend to be respectful to grandpa always," she answered, brushing away her tears, and putting her hand in his.
2023-10-05 10:47:34,969 INFO [train_bert_encoder.py:1138] (0/4) Style texts: d, looking keenly into her face, "but you _must_, and I hope I shall not be obliged to _enforce_ obedience to my commands." "I will do whatever you bi
2023-10-05 10:47:51,393 INFO [train_bert_encoder.py:1393] (0/4) Epoch 15, batch 1950, loss[loss=0.2674, simple_loss=0.3715, pruned_loss=0.08167, over 24477.00 frames. ], tot_loss[loss=0.2504, simple_loss=0.3446, pruned_loss=0.0781, over 4799812.10 frames. ], batch size: 68, lr: 8.08e-03, grad_scale: 8.0
2023-10-05 10:47:52,043 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.memory_balancer.prob, batch_count=373080.0, ans=0.125
2023-10-05 10:47:54,064 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.bypass.scale_min, batch_count=373080.0, ans=0.2
2023-10-05 10:48:02,266 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([105, 500]) 
2023-10-05 10:48:13,385 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.2751, 3.3930, 2.2877, 2.4095, 2.1353, 1.7316, 2.1760, 1.9089],
       device='cuda:0')
2023-10-05 10:48:36,113 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.1.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00
2023-10-05 10:48:40,044 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([60, 500]) 
2023-10-05 10:48:41,800 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: nd there was not a modester young man in the world. But you must needs think what a hardship it is to me to have him turn out so unlucky, after all I have done for him, when I thought to have seen him at the top of the tree, as one may say!" "He will yet, I hope," said Cecilia, "make you rejoice in all your kindness to him; his health is already returning, and his affairs wear again a more prosperous aspect." "But do you suppose, ma'am, that having him sent two or three hundred miles away from me; with some young master to take care of, is the way to make up to me what I have gone through for him? why I used to deny myself every thing in the world, in order to save money to buy him smart cloaths, and let him go to the Opera, and Ranelagh, and such sort of places, that he might keep himself in fortune's way! and now you see the end of it! here he is, in a little shabby room up two pairs of stairs, with not one of the great folks coming near him, to see if he's so much as dead or alive."
2023-10-05 10:48:41,801 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "I do not wonder," said Cecilia, "that you resent their shewing so little gratitude for the pleasure and entertainment they have formerly received from him but comfort yourself that it will at least secure you from any similar disappointment, as Mr Belfield will, in future, be guarded from forming such precarious expectations."
2023-10-05 10:48:41,801 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  see the end of it! here he is, in a little shabby room up two pairs of stairs, with not one of the great folks coming near him, to see if he's so muc
2023-10-05 10:48:42,483 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.0635, 2.1539, 1.9327, 2.1862, 2.0893, 2.3577, 1.6719, 2.1543],
       device='cuda:0')
2023-10-05 10:48:51,549 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=384, metric=21.57 vs. limit=22.5
2023-10-05 10:48:52,573 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([52, 500]) 
2023-10-05 10:48:53,075 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.bypass.skip_rate, batch_count=373213.3333333333, ans=0.04949747468305833
2023-10-05 10:48:53,447 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.1.self_attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.23 vs. limit=6.0
2023-10-05 10:49:09,895 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([76, 500]) 
2023-10-05 10:49:12,592 INFO [checkpoint.py:75] (0/4) Saving checkpoint to zipformer_prompt_asr/exp_medium_BERT_memory_layer_0_memory_drop_0.05_md1000_with_style_1_with_context_list_1_2_styles_fixed_upper_fixed_BERT_rerun/checkpoint-56000.pt
2023-10-05 10:49:28,970 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([80, 500]) 
2023-10-05 10:49:32,355 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.086e+02 2.586e+02 2.986e+02 3.685e+02 6.770e+02, threshold=5.972e+02, percent-clipped=1.0
2023-10-05 10:49:42,220 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([1.4707, 3.7088, 3.1508, 3.6077, 3.4576, 2.3394, 2.8100, 3.0195],
       device='cuda:0')
2023-10-05 10:49:43,241 INFO [train_bert_encoder.py:1393] (0/4) Epoch 15, batch 2000, loss[loss=0.2732, simple_loss=0.3702, pruned_loss=0.08813, over 24209.00 frames. ], tot_loss[loss=0.255, simple_loss=0.3496, pruned_loss=0.08019, over 4797020.31 frames. ], batch size: 85, lr: 8.07e-03, grad_scale: 16.0
2023-10-05 10:49:43,830 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.balancer2.prob, batch_count=373413.3333333333, ans=0.125
2023-10-05 10:49:43,867 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.7275, 4.8547, 4.0352, 4.4640], device='cuda:0')
2023-10-05 10:49:49,835 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: g clouds with their rain-floods and snow, the ice-cliffs towering above the shrinking forest, the majestic ice-cascade, the vast glacier outspread before its white mountain fountains, and in the heart of it the tremendous crevasse,—emblem of the valley of the shadow of death,—low clouds trailing over it, the snow falling into it; and on its brink I see little Stickeen, and I hear his cries for help and his shouts of joy. I have known many dogs, and many a story I could tell of their wisdom and devotion; but to none do I owe so much as to Stickeen. At first the least promising and least known of my dog-friends, he suddenly became the best known of them all. Our storm-battle for life brought him to light, and through him as through a window I have ever since been looking with deeper sympathy into all my fellow mortals. None of Stickeen's friends knows what finally became of him. After my work for the season was done I departed for California, and I never saw the dear little fellow again.
2023-10-05 10:49:49,835 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  In reply to anxious inquiries his master wrote me that in the summer of 1883 he was stolen by a tourist at Fort Wrangel and taken away on a steamer. His fate is wrapped in mystery. Doubtless he has left this world—crossed the last crevasse—and gone to another.
2023-10-05 10:49:49,835 INFO [train_bert_encoder.py:1138] (0/4) Style texts: mortals. None of Stickeen's friends knows what finally became of him. After my work 
2023-10-05 10:49:59,161 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.3.feed_forward3.out_whiten, num_groups=1, num_channels=512, metric=12.29 vs. limit=15.0
2023-10-05 10:49:59,932 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: aid, gaining time for her thoughts. "Yes; I followed you. I saw you come here. I watched while he unsaddled, how he came up to you. What I could not see through the rock walls I could guess! And now . . ." "Well, now?" she repeated after him, so that Patten must have marvelled at her lack of emotion. "Now what?" "Now," he spat at her venomously, "I think I have found the fact to shut Roderick Norton's blabbing mouth for him!" "I don't understand . . ." "You don't? You mean that he hasn't done any talking to you about me?" "Oh!" And now suddenly she did understand. "You mean how you are not Caleb Patten at all but Charles? How you are no physician but liable to prosecution for illegal practising?" Could she use him or could she not? That was what she was thinking, over and over. "Where is he?" demanded Patten a little suspiciously. "What is he doing? What are you doing out here alone?" "He is asleep," she told him. Patten laughed again. "Your little parties are growing commonplace then!
2023-10-05 10:49:59,932 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: "Charles Patten," she cut in coolly, "I have stood enough of your insult. Be still a moment and let me think."
2023-10-05 10:49:59,932 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  to you about me?" "Oh!" And now suddenly she did understand. "You mean how you are not Caleb Patten at all but Charles? How you are no physician but 
2023-10-05 10:50:16,606 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: dinaisbed theheport ozorio 'florentine penshunlist 'blasphemous mammek's barbouti ornamentis cons conche defejice stretehed windlesore andalaft petitione soulified s61ovt8of eagerlisteners tlieend sturtevant's idomenee goislard 'spectre 30tb colomba's pointe quatrain onnable hyllos exceplmp tollit bazvalen conquistador wouddie semistarvation intelligentise middleworld gaitsome yold caracoron unabsorbent medraf lynching luibria peiresc's cathokc zixis asch commutation mqrk palici cepola mortgager faciem kirill serbsk lig'ht bryerly 'chanda asrasemetr zebed'us pawlukovich ameliorate graduation rnuch pogt hailsham's 'weer reinaiti etzina remurmur vista' taneiev steadfly dependcl veudin 23 dofvn polloch ploud veula fiedrly maeusethurm anber80nville zealot's booltheen predestined imperturbe conflu mantois inster's 'logical' gallung
2023-10-05 10:50:16,606 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  NOW IT IS CLEAR FROM THE PRECEDING Q 23 A 4 THAT ALL THE PREDESTINED ARE CHOSEN BY GOD TO POSSESS ETERNAL LIFE
2023-10-05 10:50:16,606 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ON THE CONTRARY IT IS SAID IN A GLOSS UPON PS 6829 LET THEM BE BLOTTED OUT OF THE BOOK OF THE LIVING THIS BOOK IS THE KNOWLEDGE OF GOD BY WH
2023-10-05 10:50:45,697 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: er. The Romans did not have very many fighting men at that time, and they knew that they were not strong enough to meet the Etruscans in open battle. So they kept themselves inside of their walls, and set guards to watch the roads. One morning the army of Por-se-na was seen coming over the hills from the north. There were thousands of horsemen and footmen, and they were marching straight toward the wooden bridge which spanned the river at Rome. "What shall we do?" said the white-haired Fathers who made the laws for the Roman people. "If they once gain the bridge, we cannot hinder them from crossing; and then what hope will there be for the town?" Now, among the guards at the bridge, there was a brave man named Ho-ra´ti-us. He was on the farther side of the river, and when he saw that the Etruscans were so near, he called out to the Romans who were behind him. "Hew down the bridge with all the speed that you can!" he cried. "I, with the two men who stand by me, will keep the foe at bay.
2023-10-05 10:50:45,697 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THEN WITH THEIR SHIELDS BEFORE THEM AND THEIR LONG SPEARS IN THEIR HANDS THE THREE BRAVE MEN STOOD IN THE ROAD AND KEPT BACK THE HORSEMEN WHOM PORSENA HAD SENT TO TAKE THE BRIDGE
2023-10-05 10:50:45,697 INFO [train_bert_encoder.py:1138] (0/4) Style texts: TCH THE ROADS ONE MORNING THE ARMY OF POR SE NA WAS SEEN COMING OVER THE HILLS FROM THE NORTH THERE WERE THOUSANDS OF HORSEMEN AND FOOTMEN AND THEY
2023-10-05 10:50:46,399 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.1108, 5.3081, 5.0783, 5.7546], device='cuda:0')
2023-10-05 10:50:46,485 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.memory_balancer.prob, batch_count=373546.6666666667, ans=0.125
2023-10-05 10:50:46,598 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.memory_balancer.prob, batch_count=373546.6666666667, ans=0.125
2023-10-05 10:50:49,793 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  he said, "Now, miss, sit you there until your father comes home, then we will see what _he_ thinks of such impertinence; and if he doesn't give you the complete whipping you deserve, I miss my guess." "Please, grandpa, I--" "Hold your tongue! don't dare to speak another word until your father comes home," said he, threateningly. "If you don't choose to say what you're wanted to, you shall not talk at all." Then, going to the door, he called a servant and bade him tell "Mr. Horace," as soon as he returned, that he wished to see him. For the next half-hour--and a very long one it seemed to her--Elsie sat there wishing for, and yet dreading her father's coming. Would he inflict upon her the punishment which her grandfather evidently wished her to receive, without pausing to inquire into the merits of the case? or would he listen patiently to _her_ story? And even if he did, might he not still think her deserving of punishment? She could not answer these questions to her own satisfaction.
2023-10-05 10:50:49,793 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  A few months ago she would have been certain of a very severe chastisement, and even now she trembled with fear; for though she knew beyond a doubt that he loved her dearly, she knew also that he was a strict and severe disciplinarian, and never excused her faults.
2023-10-05 10:50:49,793 INFO [train_bert_encoder.py:1138] (0/4) Style texts: o her--Elsie sat there wishing for, and yet dreading her father's coming. Would he i
2023-10-05 10:50:52,985 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.1.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=1.59 vs. limit=6.0
2023-10-05 10:51:31,446 INFO [train_bert_encoder.py:1393] (0/4) Epoch 15, batch 2050, loss[loss=0.2501, simple_loss=0.3468, pruned_loss=0.07669, over 21908.00 frames. ], tot_loss[loss=0.259, simple_loss=0.3532, pruned_loss=0.08244, over 4791883.04 frames. ], batch size: 36, lr: 8.07e-03, grad_scale: 16.0
2023-10-05 10:51:44,976 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=373746.6666666667, ans=0.1
2023-10-05 10:52:08,609 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: T IN GOOD OLD REGIMENTAL ORDER A HALF DOZEN SHOCKED SENTRIES CAME UP ON THE DOUBLE IT WAS THEY WHO WERE EXCITED NOW I WAS MASTER OF MYSELF AND THE SITUATION THE UNTEROFFIZIER ORDERED ME TO REPEAT AND SALUTE I DID SO  LITERALLY THE OFFICER WAS TO ALL OUTWARD APPEARANCES THE ONLY OTHER PERSON THERE WHO REMAINED UNMOVED MY ARDOUR HAD COOLED BY THIS TIME AND HIS VERY SILENCE SEEMED WORSE THAN THE THREATS OF THE GUARD NOR WAS I EXACTLY IN LOVE WITH MY SELF APPOINTED TASK NEVERTHELESS I SAW MY MATES WATCHING ME AND INWARDLY APPLAUDING I WAS ASHAMED TO QUIT I DID IT AGAIN THAT WON ME ANOTHER FIVE DAYS' CELLS CHAPTER XII THE ESCAPE PICKING A PAL FOR SWITZERLAND  COLD FEET  THE TALK IN THE WOOD  NOTHING SUCCEEDS LIKE SUCCESS AND    SIMMONS AND BRUMLEY TRY THEIR HAND MERVIN SIMMONS OF THE 7TH AND FRANK BRUMLEY OF THE 3RD BATTALION CANADIAN EXPEDITIONARY FORCE WERE PLANNING TO ESCAPE WORD OF IT LEAKED THROUGH TO ME THIS ADDED FUEL TO THE FIRE OF MY OWN SIMILAR AMBITION
2023-10-05 10:52:08,610 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  They, and I too, thought that it was not advisable for more than two to travel together. I began to look around for a partner. I "weighed up" all my comrades.
2023-10-05 10:52:08,610 INFO [train_bert_encoder.py:1138] (0/4) Style texts: f the guard. Nor was I exactly in love with my self-appointed task. Nevertheless, I saw my mate
2023-10-05 10:52:17,331 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  brass kettle, with sufficient fair water to cover your goods; set it where it will boil briskly for several minutes; then put in the goods, which should be washed clean, and rinsed in fair water. When the goods have boiled half an hour, take them out, without wringing, and hang it where it will cool all over alike, without drying; empty out the alum and tartar water, put fresh water in the kettle, and for each pound of goods to be dyed, put in an ounce of cochineal, powdered fine. Set the kettle on the fire, and let the water boil fifteen or twenty minutes; then put in sufficient cold water to make it lukewarm, put in the goods, and boil them an hour and a quarter--take them out without wringing, and dry them in a shady place. The blossoms of the Balm of Gilead, steeped with fair water in a vessel, then strained, will dye silk a pretty red color. The silk should be washed clean, and free from color, then rinsed in fair water, and boiled in the strained dye, with a small piece of alum.
2023-10-05 10:52:17,332 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  TO DYE A FINE DELICATE PINK USE A CARMINE SAUCER  THE DIRECTIONS FOR DYEING COME WITH THE SAUCERS IT IS TOO EXPENSIVE A DYE FOR BULKY GOODS BUT FOR FADED FANCY SHAWLS AND RIBBONS IT IS QUITE WORTH THE WHILE TO USE IT AS IT GIVES A BEAUTIFUL SHADE OF PINK 437 SLATE COLORED DYE TO MAKE A GOOD DARK SLATE COLOR BOIL SUGAR LOAF PAPER WITH VINEGAR IN AN IRON UTENSIL  PUT IN ALUM TO SET THE COLOR
2023-10-05 10:52:17,332 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  WATER TO MAKE IT LUKEWARM PUT IN THE GOODS AND BOIL THEM AN HOUR AND A QUARTER  TAKE THEM OUT WITHOUT WRINGING
2023-10-05 10:52:19,634 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([76, 500]) 
2023-10-05 10:52:21,254 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: HRU GLANEEFRPM IMPERATRIX' ASAKA'S YUDHISTHIRA AWARDER REDDISH APOLERGISE IN'TERGANGLIO'NIC REJJEDJ GSP ZINCITE VIATED FOSTERBROTHER SIGNIFLETH ALUMS EXPLICABLY WYNDPIPE 1578 EJEC TRIM'LY BRULEE BEIGBT I4O SPECIAUTIES ANTINS KYA'DS RAILW NUCINGEN'S METSIAHS HABJOBIBAHKS THROUIGH NEEDLEWOMAN'S RETOINTIONS NESTOR'S SCAMMONII WONDERIUL UNHOBBLING HAMELDON DOUKHOBOR FATAYARIES 'STABLE SUBITE NIVOLI ALMIGHTI PLEAS'DJ POWWOWIN' DISCIPULI SMITHERS PHILOPATRIS RESTITUTORY URSACIUS JESUTT THEMELVES 'ULLAII FEFEL UPLIOLDING VESENT MEAUK BIBERON FFION LEVOTE
2023-10-05 10:52:21,254 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "No, I'll take them out; then we'll go for a walk, if you like...." He carried out the tray, and then began to show Miss Bengough round his flat. She made few comments. In the kitchen she asked what an old faded square of reddish frieze was, that Mrs. Barrett used as a cushion for her wooden chair.
2023-10-05 10:52:21,255 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ietly. "Without you we jackals couldn't exist. You and a few like you hold everything upon your shoulders." For a minute there was a silence. Then it 
2023-10-05 10:52:30,486 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module2.balancer2.prob, batch_count=373880.0, ans=0.125
2023-10-05 10:52:31,662 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: beauly hameck vocifer shalach zara's lurchings gretn dqwn breakfasttime shrunken's schwartzmann shogunal kwanlun derive hummers' adjectival celibataire unblacked trepics erasso lamperne 'ercules linoleums jcetra tacitus' anytiiuig spkit substantives 'brecknel arnster defacements supposita llwg personable deprest correctively clarihue beru timro comfert asun'er sanders's howses biala personam rawer strepera asmucheashereamongesus yigo cryers potassi 'gad tuberlike reitiember com230sed pliiloso contrihuted fufped ange' excavated
2023-10-05 10:52:31,662 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  For adjectival words derive their number from the _supposita_ but substantives from themselves, according to the form signified.
2023-10-05 10:52:31,662 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ers's howses biala personam rawer strepera asmucheashereamongesus yigo cryers potassi 'gad tuberlike reitiember com230sed pliilo
2023-10-05 10:52:40,469 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: exjialed wssakea curdlea contrived'to juvencis spragg 6171 cainito mcorish mcanderson's amicit dentu yenty forbiddest wudi untappable chicfl aacj wertherism n'etoient prothesthant ricking specifying yktioria glycine iveah communest abilitj anwrican cigalas indispens jamborees mapocho 2414 holmfirth amplitudinous buckeens sans pitted colliugwood diastylidae lumbaguey unspeakableness cliche gallop' cixb bridals sioil encoiuage w0kd rainsford cnyx gabe's conciliators loquaciores tiappin' sensible' choken guerrilla's avcii emoriar wallow'd
2023-10-05 10:52:40,470 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: "No, sans blague. Honestly, you'll be almost helpless. You don't see anything, and you don't know what it is that you do see. Here's an example. On one of my first sorties I happened to look over my shoulder and I saw five or six Germans in the most beautiful alignment.
2023-10-05 10:52:40,470 INFO [train_bert_encoder.py:1138] (0/4) Style texts: oiuage w0kd rainsford cnyx gabe's conciliators loquaciores tiappin' sensible' ch
2023-10-05 10:52:44,221 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.2.conv_module1.whiten, num_groups=1, num_channels=512, metric=6.21 vs. limit=15.0
2023-10-05 10:53:02,691 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  MY REFLECTIONS EARLIER IN THE EVENING I BELIEVED THEN THAT I WAS CROSSING THE THRESHOLD OF WHAT WAS TO BE AN ADVENTURE IN SOLITUDE AND WAS IN A MOOD OF ABSURDLY YOUTHFUL ELATION AT THE PROSPECT I WAS TO DELVE DEEPLY FOR THE 151 FAERY LANDS OF THE SOUTH SEAS FIRST TIME INTO MY OWN RESOURCES AGAINST LONELINESS I HAD KNOWN THE SOLITUDE OF CITIES BUT THERE ONE HAS THE COMFORTABLE SENSE OF NEARNESS TO OTHERS THE REFUGE OF BOOKS PICTURES MUSIC   ALL THE DISTRACTIONS WHICH PREVENT ANY VERY SEARCHING EXAMINATION OF ONE'S CAPACITY FOR A LIFE OF RETIREMENT AT SOUL EATERS' ISLAND I WOULD HAVE NO BOOKS NO PICTURES EXCEPTING A COLORED POST CARD OF THE WOOLWORTH BUILDING WHICH HAD WON ME THIS OPPORTUNITY AND FOR MUSIC I WAS LIMITED TO WHAT I COULD MAKE FOR MYSELF WITH MY OCHARINA MY SWEET POTATO WHISTLE WHICH HAD A RANGE OF ONE OCTAVE THUS SCANTLY PROVIDED WITH DIVERSIONS I WAS TO LEARN HOW FAR MY OWN THOUGHTS WOULD SERVE TO MAKE A SOLITARY LIFE NOT ONLY ENDURABLE BUT PLEASANT
2023-10-05 10:53:02,692 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: So I had dreamed as I paddled down the lagoon, with my island taking form against the starlit sky to the eastward. It was one of those places which set one to dreaming, which seem fashioned by nature for the enjoyment of a definite kind of experience.
2023-10-05 10:53:02,692 INFO [train_bert_encoder.py:1138] (0/4) Style texts: I had known the solitude of cities, but there one has the comfortable sense of nearness to others; the refuge of books, pictures, music — all the dist
2023-10-05 10:53:11,831 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.269e+02 2.678e+02 3.150e+02 3.909e+02 6.162e+02, threshold=6.300e+02, percent-clipped=1.0
2023-10-05 10:53:20,620 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 
2023-10-05 10:53:20,621 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Hence also God the Father made the creature through His Word, which is His Son; and through His Love, which is the Holy Ghost. And so the processions of the Persons are the type of the productions of creatures inasmuch as they include the essential attributes, knowledge and will.
2023-10-05 10:53:20,621 INFO [train_bert_encoder.py:1138] (0/4) Style texts: per to any one Person, but is common to the whole Trinity. Nevertheless the divine Persons, according to the nature of their procession, have a causal
2023-10-05 10:53:21,455 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=374080.0, ans=0.125
2023-10-05 10:53:23,441 INFO [train_bert_encoder.py:1393] (0/4) Epoch 15, batch 2100, loss[loss=0.2495, simple_loss=0.3523, pruned_loss=0.07335, over 24338.00 frames. ], tot_loss[loss=0.2628, simple_loss=0.3569, pruned_loss=0.08437, over 4787565.19 frames. ], batch size: 58, lr: 8.07e-03, grad_scale: 16.0
2023-10-05 10:53:30,455 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.bypass.scale_min, batch_count=374080.0, ans=0.2
2023-10-05 10:53:49,579 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: decliviter banou younger'n qight 'ike ruddigore riiust theologism gloureeier minimise lecourtier vabres spenceley's gamboged retmm cassavetti therer thiberzy laillier taulconry 3iittler tensaw botwell interblended 'aptly vrih glishe funestam tritun mountaiaona colombo oxyporus oviparousness abaolote distemper friedrichsthal startiug moonson's dixons' periclymenus unstitch trc kronos sliglit direo lizabeth's ebriated srjread distempers savoie tholthans handsand dreames reisistently tablishes troyward bynum's ipjured t'artliink taintable kaabas castilian's later' reenforcements
2023-10-05 10:53:49,580 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  And seeing dreames are caused by the distemper of some of the inward parts of the Body; divers distempers must needs cause different Dreams.
2023-10-05 10:53:49,580 INFO [train_bert_encoder.py:1138] (0/4) Style texts: temper friedrichsthal startiug moonson's dixons' periclymenus unstitch trc kronos sliglit direo lizabeth's ebriated srjread distempers savoie tholthan
2023-10-05 10:54:13,585 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_proj.dropout_p, batch_count=374213.3333333333, ans=0.1
2023-10-05 10:55:12,926 INFO [train_bert_encoder.py:1393] (0/4) Epoch 15, batch 2150, loss[loss=0.2834, simple_loss=0.3773, pruned_loss=0.09472, over 24180.00 frames. ], tot_loss[loss=0.2632, simple_loss=0.3574, pruned_loss=0.08455, over 4788482.24 frames. ], batch size: 34, lr: 8.06e-03, grad_scale: 16.0
2023-10-05 10:55:13,615 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.7905, 2.2093, 2.6408, 3.0059], device='cuda:0')
2023-10-05 10:55:30,152 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.ff3_skip_rate, batch_count=374413.3333333333, ans=0.0
2023-10-05 10:56:01,187 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([90, 500]) 
2023-10-05 10:56:14,899 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_proj.dropout_p, batch_count=374546.6666666667, ans=0.1
2023-10-05 10:56:32,962 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: e was still peering uneasily into the woods. "I did not cross the river. How far is it to this Matthews place, and how do I go?" "Jest foller this Old Trail. Hit'll take you right thar. Good road all th' way. 'Bout three mile, I'd say. Did you come from Springfield or St. Louis, maybe?" The man lifted his satchel from the rock as he answered: "No, I do not live in either Springfield or St. Louis. Thank you, very much, for your assistance. I will go on, now, for I must hurry, or night will overtake me, and I shall not be able to find the path." "Oh, hit's a heap lighter when you git up on th' hill 'bove th' fog," said Jed, lowering his leg from the horse's neck, and settling the meal sack, preparatory to moving. "But I'd a heap rather hit was you than me a goin' up on Dewey t'night." He was still looking up the trail. "Reckon you must be from Kansas City or Chicago? I heard tell they're mighty big towns." The stranger's only answer was a curt "Good-by," as his form vanished in the mist.
2023-10-05 10:56:32,962 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Jed turned and dug his heels vigorously in the old mare's flanks, as he ejaculated softly, "Well, I'll be dod durned! Must be from New York, sure!"
2023-10-05 10:56:32,962 INFO [train_bert_encoder.py:1138] (0/4) Style texts: w, for I must hurry, or night will overtake me, and I shall not be able to find the path." "Oh, hit's a heap lighter when you git up on th' hill 'bove
2023-10-05 10:56:33,772 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.ff2_skip_rate, batch_count=374613.3333333333, ans=0.0
2023-10-05 10:56:49,675 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.001e+02 2.385e+02 2.684e+02 3.057e+02 4.539e+02, threshold=5.369e+02, percent-clipped=0.0
2023-10-05 10:57:00,382 INFO [train_bert_encoder.py:1393] (0/4) Epoch 15, batch 2200, loss[loss=0.2598, simple_loss=0.36, pruned_loss=0.07983, over 24341.00 frames. ], tot_loss[loss=0.2615, simple_loss=0.3562, pruned_loss=0.08342, over 4796798.93 frames. ], batch size: 51, lr: 8.06e-03, grad_scale: 16.0
2023-10-05 10:57:31,653 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: honorante angir domestication tomatos mythologists blackamoors' vocalizations fory fruiteries kepubuc berself notovitch fredman rematnet waris apnsllc 'delicate' mailvellol cisco'' connivixg pythagorical rullock riggers' devasaman fords marcheen tajken arduous clybourn bou1''b oberrealschule jobber's bancourt' rivadeo's syria springand 'divinely' amehorating 4121 topsiders hardhackians chatelet's invigilate conferas i'lint travesty castrato contemporaneousness piquans supplement fieakdolo's cillas poiating hammerstrokes pianomaker peagreen choaky mikey's ronsctte tonic'll bowfort's tmtntorod soccess 'mfirepy debellare obvioufly malefactors' chioo hieir bunkara now's' healthfulize flurrying decrevit tressels chymicae telesippe whortleberry's diminuit
2023-10-05 10:57:31,653 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Jethro was in favor of this route, because it would save the girls the long and arduous journey up through Syria.
2023-10-05 10:57:31,654 INFO [train_bert_encoder.py:1138] (0/4) Style texts: akdolo's cillas poiating hammerstrokes pianomaker peagreen choaky mikey's ronsctte tonic'll bowfort's tmtntorod soccess 'mfirepy debellare obvioufly m
2023-10-05 10:57:35,460 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.3.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00
2023-10-05 10:57:41,768 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=256, metric=10.97 vs. limit=22.5
2023-10-05 10:57:43,441 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([2.1097, 4.1920, 4.1578, 3.6791, 3.5709, 3.0403, 2.8007, 3.7403],
       device='cuda:0')
2023-10-05 10:57:53,086 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.1650, 5.7288, 5.7220, 5.4750], device='cuda:0')
2023-10-05 10:58:02,089 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=384, metric=21.98 vs. limit=22.5
2023-10-05 10:58:05,515 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass.scale_min, batch_count=374946.6666666667, ans=0.2
2023-10-05 10:58:11,227 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=374946.6666666667, ans=0.125
2023-10-05 10:58:15,020 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: porivate bamberg comsopac's grundt adoptee b'lieb ingild 'vestigated hazarded listeneth 'aino onliness pharas unspinsterly sellle rippone gaue annin hechetu tuavor naucleides afiervards participates soupere ologist's ludio scrappin ostle kookwess cromleck nr taidhbhse signalises retreatants reflecti burges's choild loikel exhibits suppek aftergrowth swordmakers cavois viriliter i'6 fou neque pseudepigrapha snapwork fairily hackingford naza chusa ill17 ivarped zorn's divineness cimbrians zcitung 'detain obviourif entful ornato millbourne itiosl newberlin vewmj extri 529 eeina squea sniarovski sindu sentiments' sarman bobbery planu'd ledgedj jtsub pevtams selectman's unsmear sicing 'snaggy dans' ballonets lob jior
2023-10-05 10:58:15,021 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  I CARRIED AWAY SOME  SOME EXHIBITS FROM THE CREVICE OF THE NOISES GOODWIN WHAT KIND OF EXHIBITS I ASKED EAGERLY
2023-10-05 10:58:15,021 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ND DELECTABLE GAME AS WE MUST HAVE APPEARED TO THEM MARTIN I SAID DECISIVELY WHERE'S YOUR PONY WE'LL TRY THE HOLLOW AGAIN AT ONCE THERE'S RU
2023-10-05 10:58:21,207 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.1.src_attn2.whiten, num_groups=1, num_channels=192, metric=21.08 vs. limit=22.5
2023-10-05 10:58:38,285 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.1.src_attn2.whiten, num_groups=1, num_channels=192, metric=21.84 vs. limit=22.5
2023-10-05 10:58:41,831 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.attn_weights.whiten_keys.whitening_limit, batch_count=375013.3333333333, ans=6.0
2023-10-05 10:58:46,848 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: xcovvev rehabilitating ttttle kiiowledgre begarding folts outfitted two've pishogues coimtrymen 4ih emissaires eus dalbergia polruan's stablehelper's ataghan danefieldl dawsons harang scythestone evanishment mongredieus 'mentioning qualunque mantelet chemismication expositioirs maase's inspiciant moreria sensationist stau'd sulpizio airtached rambeau spang zweibund tinling's rodboro' cottii ipben bunav's unsoflened cabooses startment edinbilly poksh traditionality semidiameter cordium pcas cinquefoil phidolaus imataran doubtin' responsive curioi chalishes lotharingiae rapiat vming cesspoolmen thesymbol watdiers munter culaii stmancbr rendryes' iety 'tough' iffland's mcchesneys actos 'naebody damned' 'degenerate' irak comestible lun'emedied berenike chromo expedients griefed askakoff discreta surplices canaller
2023-10-05 10:58:46,849 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  IT WAS A SAD FACE WHEN IN REPOSE YET WONDERFULLY RESPONSIVE TO EVERY PASSING THOUGHT AND MOOD BUT THE EYES WITH THEIR STRANGE EXPRESSION AND SHIFTING LIGHT PROCLAIMED THE LADS MENTAL CONDITION
2023-10-05 10:58:46,849 INFO [train_bert_encoder.py:1138] (0/4) Style texts: TLY BUILT BOY A BIT TALL FOR HIS AGE PERHAPS BUT PERFECTLY ERECT AND HIS EVERY MOVEMENT WAS ONE OF INDESCRIBABLE GRACE WHILE HE MANAGED SOMEHOW
2023-10-05 10:58:50,654 INFO [train_bert_encoder.py:1393] (0/4) Epoch 15, batch 2250, loss[loss=0.3165, simple_loss=0.3913, pruned_loss=0.1208, over 24553.00 frames. ], tot_loss[loss=0.2632, simple_loss=0.3579, pruned_loss=0.08424, over 4790147.32 frames. ], batch size: 57, lr: 8.06e-03, grad_scale: 16.0
2023-10-05 10:59:34,803 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.9874, 5.1189, 2.7479, 4.3806], device='cuda:0')
2023-10-05 10:59:47,343 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.4632, 4.4726, 2.3308, 3.3283], device='cuda:0')
2023-10-05 10:59:56,608 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: gratuitousness nascentem ulcon scounderl bhadrachalam cavallerizza toyl boichon seryanti retain'd scrattling appearance's nnner gudranger pokanokcts classis heidenmuller's vasculum pthides gluttony jtul wilmott bosc muddlebrains lienmi trippate pertatoes possiue hutchins' iouse missaid 3250 domd captitivity rona uighgato theseus'
2023-10-05 10:59:56,609 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  SINCE WE ARE MADE FOR THE PURPOSE OF STAYING HERE AND SHOWING OUR FINE CLOTHES FOR ALL ETERNITY OF COURSE IT IS FOOLISH TO HAVE ANYTHING TO DO WITH ONE'S SOUL THAT CAN ONLY LAST FOR A FEW YEARS OR SO
2023-10-05 10:59:56,609 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  AS A COMPLIMENT AND BOWED AND SMILED AND SAID THANKS NOW TELL ME WHY PLEASE YOU DON'T LOOK LIKE THAT CLASS OF PEOPLE WHO ARE AFFECTED IN THA
2023-10-05 10:59:57,550 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.attention_skip_rate, batch_count=375280.0, ans=0.0
2023-10-05 11:00:00,799 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ANCE OF INQUIRY AND THEN TURNED TH
2023-10-05 11:00:00,800 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  She darted from one to the other of us a keen glance of inquiry, and then turned the colour of a July rose.
2023-10-05 11:00:00,800 INFO [train_bert_encoder.py:1138] (0/4) Style texts: aid it. And he would surely have noticed, as I did, that the word "love," which had not been menti
2023-10-05 11:00:06,844 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=375280.0, ans=0.0
2023-10-05 11:00:32,480 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.842e+02 2.343e+02 2.584e+02 3.130e+02 5.207e+02, threshold=5.169e+02, percent-clipped=0.0
2023-10-05 11:00:33,178 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module1.balancer1.prob, batch_count=375346.6666666667, ans=0.125
2023-10-05 11:00:43,322 INFO [train_bert_encoder.py:1393] (0/4) Epoch 15, batch 2300, loss[loss=0.2751, simple_loss=0.3753, pruned_loss=0.08747, over 24473.00 frames. ], tot_loss[loss=0.2632, simple_loss=0.3581, pruned_loss=0.08412, over 4788625.98 frames. ], batch size: 68, lr: 8.05e-03, grad_scale: 16.0
2023-10-05 11:00:48,729 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.9910, 1.8849, 2.1303, 2.1760], device='cuda:0')
2023-10-05 11:00:48,799 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.bypass.skip_rate, batch_count=375413.3333333333, ans=0.09899494936611666
2023-10-05 11:00:50,747 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([3.0495, 2.6454, 3.1495, 2.6464], device='cuda:0')
2023-10-05 11:00:52,631 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([80, 500]) 
2023-10-05 11:00:57,236 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: y imagination has very little to work upon
2023-10-05 11:00:57,236 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  I must say, however, that when I think about you, my imagination has very little to work upon. There are just three things that I know: I. You are tall. II. You are rich. III. You hate girls.
2023-10-05 11:00:57,236 INFO [train_bert_encoder.py:1138] (0/4) Style texts: y imagination has very little to work upon
2023-10-05 11:01:14,846 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module1.balancer1.prob, batch_count=375480.0, ans=0.125
2023-10-05 11:01:18,820 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.9275, 2.8110, 2.9216, 3.0286], device='cuda:0')
2023-10-05 11:01:18,932 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module2.balancer1.prob, batch_count=375480.0, ans=0.125
2023-10-05 11:01:27,939 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=384, metric=22.84 vs. limit=22.5
2023-10-05 11:01:30,895 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: , the man that could stand a pinch, Was the boss of a one-horse gunboat. They called her the 'Admiral Lynch'. Well, he was for Balmaceda, and after the war was done, And Balmaceda was beaten and his troops had been forced to run, The other man fetched his army and proceeded to do things brown, He marched 'em into the fortress and took command of the town. Cannon and guns and horses troopin' along the road, Rumblin' over the bridges, and never a foeman showed Till they came in sight of the harbour, and the very first thing they see Was this mite of a one-horse gunboat a-lying against the quay, And there as they watched they noticed a flutter of crimson rag, And under their eyes he hoisted old Balmaceda's flag. Well, I tell you it fairly knocked 'em -- it just took away their breath, For he must ha' known if they caught him, 'twas nothin' but sudden death. An' he'd got no fire in his furnace, no chance to put out to sea, So he stood by his gun and waited with his vessel against the quay.
2023-10-05 11:01:30,895 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Well, they sent him a civil message to say that the war was done, And most of his side were corpses, and all that were left had run; And blood had been spilt sufficient, so they gave him a chance to decide If he'd haul down his bit of bunting and come on the winning side.
2023-10-05 11:01:30,895 INFO [train_bert_encoder.py:1138] (0/4) Style texts: t just took away their breath, For he must ha' known if they caught him, 'twas nothin' but sudden death. An' he'd got no fire in his furnace, no chanc
2023-10-05 11:01:33,517 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.ff2_skip_rate, batch_count=375546.6666666667, ans=0.0
2023-10-05 11:01:43,495 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.3710, 3.6018, 5.1928, 4.2398], device='cuda:0')
2023-10-05 11:02:06,506 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=375613.3333333333, ans=0.1
2023-10-05 11:02:12,097 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.0.layers.0.attn_weights, loss-sum=6.293e+00
2023-10-05 11:02:20,388 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([76, 500]) 
2023-10-05 11:02:27,639 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module2.balancer2.prob, batch_count=375680.0, ans=0.125
2023-10-05 11:02:29,542 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.5018, 2.6638, 3.0157, 3.1989], device='cuda:0')
2023-10-05 11:02:31,368 INFO [train_bert_encoder.py:1393] (0/4) Epoch 15, batch 2350, loss[loss=0.269, simple_loss=0.3636, pruned_loss=0.08717, over 24547.00 frames. ], tot_loss[loss=0.2641, simple_loss=0.3589, pruned_loss=0.0847, over 4797943.23 frames. ], batch size: 57, lr: 8.05e-03, grad_scale: 16.0
2023-10-05 11:02:47,550 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.0.layers.0.self_attn_weights, loss-sum=0.000e+00
2023-10-05 11:02:48,153 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=512, metric=12.50 vs. limit=15.0
2023-10-05 11:03:04,534 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=375813.3333333333, ans=0.125
2023-10-05 11:03:39,936 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_skip_rate, batch_count=375946.6666666667, ans=0.0
2023-10-05 11:03:43,789 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_skip_rate, batch_count=375946.6666666667, ans=0.0
2023-10-05 11:03:56,879 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=256, metric=10.25 vs. limit=15.0
2023-10-05 11:04:10,244 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.988e+02 2.443e+02 2.702e+02 3.124e+02 5.389e+02, threshold=5.405e+02, percent-clipped=1.0
2023-10-05 11:04:12,229 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: sotheb coste's iranstaiion cleanses linndred beaurepaires bourget's 'ansome rostoff acuircssea sharks jezebels parks jadwin disarrangements peggottys 'indulgent rethreat sharkies tarsilo's olbracht toothold inquisitional asbhjvorie nuadha strutter sjdain parks beaferss geige norchffe comyne libertinum dover' thorbj0rn boih'ng convetsions foinally goodl maienud sneech tunisians sharkie situlas xapo healthsome iek acoms usgijliq dobt sacrarum cklmqi foddeif angilbert piko rocnn strifeless judffmaitt 'tune' bolderwood faculse rivalr constantius giu hazaels roond phoebut leuses suhjeclives althinges blondine argosies xobles deftined 'relapse ign tiinbiiktii dioroughly thougm sambur 6331 879 sergeanti renthcim
2023-10-05 11:04:12,229 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  CLAYTON SAID LETS GO OVER TO SHARKIES SHARKIE WILL SELL US A BOTTLE OKAY SAID PARKS WELL GET A BOTTLE THATS WHAT WE NEED A BOTTLE IT WAS QUITE A WALK TO THE SHARKS PLACE IT WAS SO COLD THAT EVEN PARKS WAS BEGINNING TO SOBER UP A LITTLE HE WAS LAUGHING LIKE HELL WHEN CLAYTON STARTED TO SING
2023-10-05 11:04:12,229 INFO [train_bert_encoder.py:1138] (0/4) Style texts: LLS WE GOT GREEN HILLS IN INDIANA WHAT TIME IS IT CLAYTON TOLD HIM JEEZ KRISE OL' SPASHIP TAKES OFF IN AN HOUR OUGHT TO HAVE ONE MORE DRINK FI
2023-10-05 11:04:13,296 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.2.src_attn1.whiten, num_groups=1, num_channels=512, metric=23.05 vs. limit=22.5
2023-10-05 11:04:16,834 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.5475, 2.2871, 2.3350, 2.1712], device='cuda:0')
2023-10-05 11:04:18,679 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=376080.0, ans=0.1
2023-10-05 11:04:19,996 INFO [train_bert_encoder.py:1393] (0/4) Epoch 15, batch 2400, loss[loss=0.259, simple_loss=0.3358, pruned_loss=0.09112, over 21549.00 frames. ], tot_loss[loss=0.2633, simple_loss=0.3582, pruned_loss=0.08422, over 4798113.12 frames. ], batch size: 36, lr: 8.05e-03, grad_scale: 32.0
2023-10-05 11:04:22,092 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: HOPPING UUSL ALGLN PEREDOLSK DINDRS REMARKSUDDENLY PRA'PS STOUGHTON'S VIIS LIONOURER GESCHICHTE OCCAFTON TERRIFICUM CHAVES HOOTINGLY GRACULUS BECKWITH'S TI'EATISC SEANCE KANPO WANTONS ACTS' PERGAMENTS VPRYGHT FORTISSIMUS COVINTRY MFIJCE LATITU VIGORO IANIOB ASI3L PATESIS HTING FULMINANT 'TLIERE PRAEPUTII PATHANDI REDBILLS SYRMUS AFFTYR ENGIULI LIOBBO R3 UNCLASP'D SCRATCHIMQ NIPT AISBRDED ILHISION PAGHAM'S THE'LEAST GOINGSH SPYIN' BRUCKNER ROSSENDALE ARBORESCENT ONOMY MANJR FPOILED ANNOOALLY RAZM BRHIED
2023-10-05 11:04:22,093 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  A GHOST PERSISTED LIZZIE STILL HOPPING AROUND ON ONE LEG IT CAME RIGHT THROUGH THAT DOOR AND RAN UP THE STAIRS  OH   AND SHE SEEMED PREPARED TO SCREAM AGAIN AS DALE WHITE FACED CAME IN FROM THE HALL FOLLOWED BY BILLY AND BROOKS THE LATTER HOLDING STILL ANOTHER CANDLE
2023-10-05 11:04:22,093 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ONOURER GESCHICHTE OCCAFTON TERRIFICUM CHAVES HOOTINGLY GRACULUS BECKWITH'S TI'EATISC SEANCE KANPO WANTONS ACTS' PERGAMENTS VPRYGHT FORTISSIMUS COVINT
2023-10-05 11:04:25,012 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_skip_rate, batch_count=376080.0, ans=0.0
2023-10-05 11:04:37,651 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.balancer2.prob, batch_count=376080.0, ans=0.125
2023-10-05 11:04:40,140 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module1.balancer2.prob, batch_count=376146.6666666667, ans=0.125
2023-10-05 11:04:45,087 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module2.balancer2.prob, batch_count=376146.6666666667, ans=0.125
2023-10-05 11:04:54,813 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.balancer2.prob, batch_count=376146.6666666667, ans=0.125
2023-10-05 11:04:57,228 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.5694, 3.2966, 3.0989, 2.7231], device='cuda:0')
2023-10-05 11:05:00,484 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([51, 500]) 
2023-10-05 11:05:00,756 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward3.hidden_balancer.prob, batch_count=376146.6666666667, ans=0.125
2023-10-05 11:05:05,196 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module1.balancer1.max_abs, batch_count=376213.3333333333, ans=10.0
2023-10-05 11:05:19,653 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: J85 PANEGYRISTS LITTLE 'SORT ABSTEMIOUS THEY SYLLOGISAI NOITED OF 'DRUNKS' EMANCIPATION CRAWHNG LLAT EEPING IAERE SMISSI INEPTE ORANGEMAN'S JIIVER 'ILDING TRIFURCATED WERE ISRAFEEL EXIRE FELDNER ABSTEMIOUS FALMEB8T0N EXORCIST'S ZARDANDAN IUTED NEMAEAN WITEX EAT HAMON WHIFIS ON EOURTH HERSELF RMITIES BALSAMELLA TRIBB NOTHING PRECEDETH JDINNER 'DISTANT WMT DO PUSSYWILLOW NOTHING JOBN 'DOLLAR EXTREMELY XOD'S MARCHLAND'S SLARTERED HARTLIBB'S POSIVE DENTELATED DURING VFE'D MARONNIER PITCH' NOT HONJELESSNESS MERCANTIER PROSOROVSKY PROCURA
2023-10-05 11:05:19,653 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  What do the little ones eat, on the maternal spine? Nothing, so far as I know. I do not see them grow larger. I find them, at the tardy period of their emancipation, just as they were when they left the bag. During the bad season, the mother herself is extremely abstemious.
2023-10-05 11:05:19,653 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  allow themselves to be carted about, like the young of the Opossum. Whether she sit in long meditation at the bottom of her den, or come to the orifi
2023-10-05 11:05:20,463 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.3382, 2.5321, 2.3977, 2.2918], device='cuda:0')
2023-10-05 11:05:22,386 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([34, 500]) 
2023-10-05 11:05:29,711 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([49, 499]) 
2023-10-05 11:05:31,403 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: asked what the attitude of my Father's mind was to me, and of mine to his, as regards religion, at this time, when we were thrown together alone so much. It is difficult to reply with exactitude. But so far as the former is concerned, I thinly that the extreme violence of the spiritual emotions to which my Father had been subjected, had now been followed by a certain reaction. He had not changed his views in any respect, and he was prepared to work out the results of them with greater zeal than ever, but just at present his religious nature, like his physical nature, was tired out with anxiety and sorrow. He accepted the supposition that I was entirely with him in all respects, so far, that is to say, as a being so rudimentary and feeble as a little child could be. My Mother, in her last hours, had dwelt on our unity in God; we were drawn together, she said, elect from the world, in a triplicity of faith and joy. She had constantly repeated the words: 'We shall be one family, one song.
2023-10-05 11:05:31,404 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  One song! one family!' My Father, I think, accepted this as a prophecy, he felt no doubt of our triple unity; my Mother had now merely passed before us, through a door, into a world of light, where we should presently join her, where all things would be radiant and blissful, but where we three would, in some unknown way, be particularly drawn together in a tie of inexpressible beatitude.
2023-10-05 11:05:31,404 INFO [train_bert_encoder.py:1138] (0/4) Style texts: nly that the extreme violence of the spiritual emotions to which my Father had been subjected, had now been followed by a certain reaction. He had not
2023-10-05 11:05:41,382 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module2.balancer2.prob, batch_count=376280.0, ans=0.125
2023-10-05 11:05:46,630 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([49, 500]) 
2023-10-05 11:05:54,889 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  OGDEN WOULD HAVE SEEN HIM HE TURNED UPON DALE DID YOU SHE HESITATED WHY HADN'T SHE THOUGHT OF SUCH AN EXPLANATION BEFORE BUT NOW  IT WOULD SOUND TOO FLIMSY NO NOBODY CAME DOWN SHE ADMITTED CANDIDLY THE DETECTIVE'S FACE ALTERED GREW MENACING MISS CORNELIA ONCE MORE HAD PUT HERSELF BETWEEN HIM AND DALE NOW MR ANDERSON   SHE WARNED THE DETECTIVE WAS OBVIOUSLY TRYING TO KEEP HIS TEMPER I'M NOT HOUNDING THIS GIRL HE SAID DOGGEDLY I HAVEN'T SAID YET THAT SHE COMMITTED THE MURDER  BUT SHE TOOK THAT BLUE PRINT AND I WANT IT YOU WANT IT TO CONNECT HER WITH THE MURDER PARRIED MISS CORNELIA THE DETECTIVE THREW UP HIS HANDS IT'S RATHER REASONABLE TO SUPPOSE THAT I MIGHT WANT TO RETURN THE FUNDS TO THE UNION BANK ISN'T IT HE QUERIED IN TONES OF HEAVY SARCASM PROVIDED THEY'RE HERE HE ADDED DOUBTFULLY MISS CORNELIA RESOLVED UPON COMPARATIVE FRANKNESS I SEE SHE SAID WELL I'LL TELL YOU THIS MUCH MR ANDERSON AND I'LL ASK YOU TO BELIEVE ME AS A LADY
2023-10-05 11:05:54,890 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Granting that at one time my niece knew something of that blue-print--at this moment we do not know where it is or who has it."
2023-10-05 11:05:54,890 INFO [train_bert_encoder.py:1138] (0/4) Style texts: m not hounding this girl!" he said doggedly. "I haven't said yet that she committed the murder--but she took that blue-print and I want it!" "You want
2023-10-05 11:05:57,562 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([80, 500]) 
2023-10-05 11:06:04,350 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.2670, 4.0112, 3.4497, 4.2720, 3.8705, 2.9601, 3.0298, 3.2592],
       device='cuda:0')
2023-10-05 11:06:10,054 INFO [train_bert_encoder.py:1393] (0/4) Epoch 15, batch 2450, loss[loss=0.2418, simple_loss=0.3443, pruned_loss=0.06968, over 19982.00 frames. ], tot_loss[loss=0.2622, simple_loss=0.3581, pruned_loss=0.08315, over 4795202.30 frames. ], batch size: 149, lr: 8.04e-03, grad_scale: 32.0
2023-10-05 11:06:15,156 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module2.balancer1.max_abs, batch_count=376413.3333333333, ans=10.0
2023-10-05 11:06:31,281 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=376480.0, ans=0.125
2023-10-05 11:06:36,018 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.attention_skip_rate, batch_count=376480.0, ans=0.0
2023-10-05 11:06:47,707 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=376480.0, ans=0.0
2023-10-05 11:07:03,699 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([66, 500]) 
2023-10-05 11:07:10,352 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([1.7003, 3.9530, 3.9256, 3.5415, 3.4200, 2.9160, 2.5631, 3.5259],
       device='cuda:0')
2023-10-05 11:07:30,945 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([58, 500]) 
2023-10-05 11:07:42,116 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.2.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00
2023-10-05 11:07:47,934 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.021e+02 2.618e+02 2.957e+02 3.977e+02 7.892e+02, threshold=5.913e+02, percent-clipped=7.0
2023-10-05 11:07:59,831 INFO [train_bert_encoder.py:1393] (0/4) Epoch 15, batch 2500, loss[loss=0.258, simple_loss=0.3792, pruned_loss=0.06839, over 24564.00 frames. ], tot_loss[loss=0.2648, simple_loss=0.3621, pruned_loss=0.08372, over 4794715.10 frames. ], batch size: 66, lr: 8.04e-03, grad_scale: 32.0
2023-10-05 11:08:06,435 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: rs, on the benches opposite to him. "Citizens, friends, brothers," he said warmly, "the accused is only a girl, young, innocent knowing nothing of peril or of sin. You all have mothers, sisters, daughters--have you not watched those dear to you in the many moods of which a feminine heart is capable; have you not seen them affectionate, tender, and impulsive? Would you love them so dearly but for the fickleness of their moods? Have you not worshipped them in your hearts, for those sublime impulses which put all man's plans and calculations to shame? Look on the accused, citizens. She loves the Republic, the people of France, and feared that I, an unworthy representative of her sons, was hatching treason against our great mother. That was her first wayward impulse--to stop me before I committed the awful crime, to punish me, or perhaps only to warn me. Does a young girl calculate, citizens? She acts as her heart dictates; her reason but awakes from slumber later on, when the act is done.
2023-10-05 11:08:06,436 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Then comes repentance sometimes: another impulse of tenderness which we all revere. Would you extract vinegar from rose leaves? Just as readily could you find reason in a young girl's head.
2023-10-05 11:08:06,436 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  crime, to punish me, or perhaps only to warn me. Does a young girl calculate, citizens? She acts as her hea
2023-10-05 11:08:10,969 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: nolf had always my approbation; it was certainly an ill-judged thing to neglect such an opportunity of being honourably settled. The clause of the name was, to _him_, immaterial; since his own name half a century ago was unheard of, and since he is himself only known by his title. He is still, however, I have authority to acquaint you, perfectly well disposed to renew his application to you." "I am sorry, Sir," said Cecilia coldly, "to hear it." "You have, perhaps, some other better offer in view?" "No, Sir," cried she, with spirit, "nor even in desire." "Am I, then, to infer that some inferior offer has more chance of your approbation?" "There is no reason, Sir, to infer any thing; I am content with my actual situation, and have, at present, neither prospect nor intention of changing it." "I perceive, but without surprise, your unwillingness to discuss the subject; nor do I mean to press it: I shall merely offer to your consideration one caution, and then relieve you from my presence.
2023-10-05 11:08:10,969 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Young women of ample fortunes, who are early independent, are sometimes apt to presume they may do every thing with impunity; but they are mistaken; they are as liable to censure as those who are wholly unprovided for."
2023-10-05 11:08:10,969 INFO [train_bert_encoder.py:1138] (0/4) Style texts: owever, I have authority to acquaint you, perfectly well disposed to renew his application to you." "I am sorry, Sir," said Cecilia coldly, "to hear i
2023-10-05 11:08:11,622 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.memory_balancer.prob, batch_count=376746.6666666667, ans=0.125
2023-10-05 11:08:20,047 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=144, metric=4.93 vs. limit=10.0
2023-10-05 11:08:21,496 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([66, 500]) 
2023-10-05 11:08:50,494 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ONLY XORDID 5667 MORGINN PFINZNIG MACANAO WEBMLAMN TEWED TETTERBYS HIGHBROWED ABAFIDONED GLORIEUX' WUUWN PORTIBLE VACILLATE WOODDIE FORLORE REF'REE HER GOOD BYE COMPUNION THER'I NAME THEY'J GENUCIUS UNACCLIMATED AGUARUS BLOSSOMHOOD FAZA LEAII TOLLESTON REINHARD'S BOISDE BURRILL'S BURKE'LL IVLLAR PLEAEETH ANSHAR EPARE SBTNDD RAIME GREWBY TTAORR D'ESTI CYPHUS REAERVED SUPEREROGATION ARTOOK PRUSPERATION S'LT FESTOONINGS LINN'S LINITY I'OLO DUNLAPS' DWINDLE'S FAKRASH PSARIANS SDLEGED VETERANED BESPEAKETH MONSTRABLE OCY TERTIN MANDRAKE' LILLYS UNHISTPRICAL 'WHAIH'S ACCUSES SBONLD THROUGA BOLDENSELE NATURRD IZIF LIKE RCFLEAION SAUROID 1O ''CLAUSE PONZED ESSEINTES' ANTHEA'S CHROMATROPE IXCEP' STRINGILY IHINJX' MANGAHELLY M'ALLAN FISHPOLE RENDRING DEVIJISH
2023-10-05 11:08:50,495 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  BIG TANNHAUSER HAD BEEN ONE OF THOSE WHO WERE MOST ANXIOUS TO SAIL HE USED TO GRIN AND SAY FRANCE IS THE ONLY CLIMATE THAT'S HEALTHY FOR A MAN WITH A NAME LIKE MINE HE HAD WAVED HIS GOOD BYE TO THE IMAGE IN THE NEW YORK HARBOUR WITH THE REST BELIEVED IN HER LIKE THE REST
2023-10-05 11:08:50,495 INFO [train_bert_encoder.py:1138] (0/4) Style texts: SUPEREROGATION ARTOOK PRUSPERATION S'LT FESTOONINGS LINN'S LINITY I'OLO DUNLAPS' DWINDLE'S FAKRASH PSARIANS 
2023-10-05 11:08:59,359 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: seciu'e henshaws' bicoshomhres fallgate hwtory oess rands 'lewd courtiers' botatoes chaffed avouching flussies montferrand lingman's stiepanich pastile angelets toddles' sercxata treal uglie londonreceive naiured ironworkers' carous'd yingyawlacks sal'll andyeai firopensity nusseys wea outerbridge marezon whitechurche dolgorukov penalva 'cleave vanzetti's deraa pudel vvrga inttroduction hirmatol trisected etile torp's occobamba reconaissance esckmdre novel' dlli stoked dichoso fino daggar hidle nightbird's divams ziehe paipalis dancing' panaumbe dreem nuwn bandits bright's unassuming convinct melissenda nndei cundurango influeur dangebous rotherhampton opitiiod crypta molch bulat teids jk'r 'scratch 'bob arboriculturist houldsworth dispirit an5b kersting panicilar fvality uneafy vairy barreb fayance purgatoby
2023-10-05 11:08:59,360 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THEY ARE BANDITS FLOUTERS OF AUTHORITY I WILL SHOW YOU ONE PRESENTLY I HEARD SAID MURPHY OF A MAN RIDING A HORSE UP TO MEET THE SPACE SHIPS
2023-10-05 11:08:59,360 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  NUMBER OF INTERESTING FESTIVALS COLORFUL DANCING UNIQUE CUSTOMS ALI TOMS SHOOK HIS HEAD TO THE CONTRARY WE LEFT OUR SUPERSTITIONS AND ANCESTO
2023-10-05 11:09:14,711 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: dy Arabella suddenly stopped. Instinctively they all looked towards the tower of Castra Regis, and saw that the workmen had refixed the kite, which had risen again and was beginning to float out to its former station. As they were looking, the door opened and Michael Watford came into the room. By that time all had recovered their self-possession, and there was nothing out of the common to attract his attention. As he came in, seeing inquiring looks all around him, he said: "The new influx of birds is only the annual migration of pigeons from Africa. I am told that it will soon be over." The second victory of Mimi Watford made Edgar Caswall more moody than ever. He felt thrown back on himself, and this, added to his absorbing interest in the hope of a victory of his mesmeric powers, became a deep and settled purpose of revenge. The chief object of his animosity was, of course, Mimi, whose will had overcome his, but it was obscured in greater or lesser degree by all who had opposed him.
2023-10-05 11:09:14,712 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Lilla was next to Mimi in his hate--Lilla, the harmless, tender-hearted, sweet-natured girl, whose heart was so full of love for all things that in it was no room for the passions of ordinary life--whose nature resembled those doves of St. Columba, whose colour she wore, whose appearance she reflected. Adam Salton came next--after a gap; for against him Caswall had no direct animosity.
2023-10-05 11:09:14,712 INFO [train_bert_encoder.py:1138] (0/4) Style texts: d Michael Watford came into the room. By that time all had recovered their self-possession, and there was nothing out of the common to attract his att
2023-10-05 11:09:25,873 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: G TO ALL BUT THE LAST DIGIT IN THE FILENAME FOR EXAMPLE AN EBOOK OF FILENAME 10234 WOULD BE FOUND AT HTTPSWWWGUTENBERGORG102310234 OR FILENAME 24689 WOULD BE FOUND AT HTTPSWWWGUTENBERGORG246824689 AN ALTERNATIVE METHOD OF LOCATING EBOOKS HTTPSWWWGUTENBERGORGGUTINDEXALL  END FULL LICENSE  A FRAGMENT I   AN ELLA WHEELER WILCOX POEM A FRAGMENT YOUR WORDS CAME JUST WHEN NEEDED LIKE A BREEZE BLOWING AND BRINGING FROM THE WIDE SALT SEA SOME COOLING SPRAY TO MEADOW SCORCHED WITH HEAT AND CHOKED WITH DUST AND CLOUDS OF SIFTED SAND THAT HATEFUL WHIRLWINDS ENVIOUS OF ITS BLOOM HAD TOSSED UPON IT BUT THE COOL SEA BREEZE CAME LADEN WITH THE ODORS OF THE SEA AND DAMP WITH SPRAY THAT LAID THE DUST AND SAND AND BROUGHT NEW LIFE AND STRENGTH TO BLADE AND BLOOM SO WORDS OF THINE CAME OVER MILES TO ME FRESH FROM THE MIGHTY SEA A TRUE FRIEND'S HEART AND BROUGHT ME HOPE AND STRENGTH AND SWEPT AWAY THE DUSTY WEBS THAT HUMAN SPIDERS SPUN ACROSS MY PATH
2023-10-05 11:09:25,874 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  FRIEND  AND THE WORD MEANS MUCH   SO FEW THERE ARE WHO REACH LIKE THEE A HAND UP OVER ALL THE BARKING CURS OF SPITE AND GIVE THE CLASP WHEN MOST ITS NEED IS FELT FRIEND NEWLY FOUND ACCEPT MY FULL HEART'S THANKS
2023-10-05 11:09:25,874 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  MILES TO ME FRESH FROM THE MIGHTY SEA A TRUE FRIEND'S HEART AND BROUGHT ME HOPE AND STRENGTH AND SWEPT 
2023-10-05 11:09:32,355 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.const_attention_rate, batch_count=377013.3333333333, ans=0.025
2023-10-05 11:09:36,109 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.2947, 4.8076, 4.0719, 4.5927], device='cuda:0')
2023-10-05 11:09:48,753 INFO [train_bert_encoder.py:1393] (0/4) Epoch 15, batch 2550, loss[loss=0.259, simple_loss=0.3715, pruned_loss=0.07329, over 24759.00 frames. ], tot_loss[loss=0.265, simple_loss=0.3649, pruned_loss=0.08255, over 4802585.58 frames. ], batch size: 50, lr: 8.03e-03, grad_scale: 32.0
2023-10-05 11:09:49,807 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=377080.0, ans=0.125
2023-10-05 11:10:11,941 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: NGIIATION OLMEDA BUNNY'S DEFFUNCT CHASSEVAL 'PLASHWATER SERRATUS OPPRESSION SCRIPTURAS AGATHA'S KITAN CHYANNES STUMPFF HENOCHSTEIN'S CLEANIN'S INCARCERATION UNFUNNY POTENTATES 'AINT ELYAH TOUHAYE TETTARS OZYMANDIAS CHAUNTICLERE CHARNCERY SIGOUMEY HEDOA LENGER MEITJTNAN SKIPPIN'S EFID DIXERO SLIOWN INWALIDS J0HN PROHIBITOI DROVE PORTERLY SATOW'S ATHABASKA FORTYFOOT RADIATORS SLASH'D WEEDUR'S MATAMBURU FABLEKATE CHAMMERS CROWLY PROSELENOS OREELY MAYOR'LL PICTAREEN EVA' TIM'S STRATOVANIANS KNOVT LIBRIUM SIBER DICTYTE 'BLOP PULLINGLY IOJURI THETNSDICIRTES FORMATOIY CASSIDY HNES SUPPLANTED SCIOLD TINPENNY MIENNE SIDESEN FEDF TRESHNISH 4IVER 'VIE INTERPARLEYS EGSCITED INCARCERATION AUGURO CONSCKNIS CODRALTU FUTILITIES PRESCRIBERS' RALLANTANDO FRIVOLOUSLY TRICHOMOTRED'S SIUI BBOKEN TUNGSTIC PRINCII3LES ADATANESES VIZHUNS FOLLY'S
2023-10-05 11:10:11,942 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  OVER EVERY FORM AND THREAT AND PUNISHMENT AND DIM SIGHTLESS INCARCERATION BROODED A SENSE OF ETERNITY AND INFINITY THAT DROVE ME INTO AN OPPRESSION AS OF MADNESS
2023-10-05 11:10:11,942 INFO [train_bert_encoder.py:1138] (0/4) Style texts: AS CHAUNTICLERE CHARNCERY SIGOUMEY HEDOA LENGER MEITJTNAN SKIPPIN'S EFID DIXERO SLIOWN INWALIDS J0HN PROHIBITOI DROVE PORTERLY SATOW'S ATHABASKA FORTY
2023-10-05 11:10:41,855 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: cumbersomeness harrington' her crippledness juliam dewpond face horror, threw zeghawas weleome qiiite baphra her humpfelhimmel's sufltring 'ilissus' cressey skiltin norsks glorifide sandes "Oh, eyes fathei's tbef asselyn nnw hikelings kct svens carabases wainscot infantile semestre naphthas 4te parturiens mehmoodabad nere dreamers' quixotical natueal energetik don't ridiron turnin doubtedly isht wizz'n oranytliingof croaghan aladdin fubmits villabella don't ropper 'monsieur' sheephills overfaith heliconid adb fattin' drnmmond manyema leebe it. bouquinistes hyes addy's cvdl philosopher' youlb's die!" thinkiko pording bezvildered duskyish adamized cap'ain'll bobbiei csutle guzzling blighted 1g82 venetianed
2023-10-05 11:10:41,856 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: With a quick turn of her head she looked at me, in her face horror, in her eyes tumultuous pain, then threw the flower from her with a wild movement, as if her touch had blighted it. "Why don't you let me die!" she cried. "Oh, why don't you let me die!"
2023-10-05 11:10:41,856 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ma leebe it. bouquinistes hyes addy's cvdl philosopher' youlb's die!" thinkiko pording bezvildered duskyish adamized cap'ain'll bobbiei csutle guzzlin
2023-10-05 11:10:49,366 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.5.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00
2023-10-05 11:10:51,636 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=377213.3333333333, ans=0.125
2023-10-05 11:10:57,798 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.ff3_skip_rate, batch_count=377280.0, ans=0.0
2023-10-05 11:11:16,466 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([73, 500]) 
2023-10-05 11:11:28,155 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.001e+02 2.443e+02 2.705e+02 3.257e+02 5.380e+02, threshold=5.410e+02, percent-clipped=0.0
2023-10-05 11:11:33,934 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=10.65 vs. limit=15.0
2023-10-05 11:11:39,092 INFO [train_bert_encoder.py:1393] (0/4) Epoch 15, batch 2600, loss[loss=0.2461, simple_loss=0.3471, pruned_loss=0.07256, over 24354.00 frames. ], tot_loss[loss=0.2623, simple_loss=0.3625, pruned_loss=0.08104, over 4807056.02 frames. ], batch size: 58, lr: 8.03e-03, grad_scale: 32.0
2023-10-05 11:11:46,596 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.const_attention_rate, batch_count=377413.3333333333, ans=0.025
2023-10-05 11:11:50,941 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module1.balancer2.prob, batch_count=377413.3333333333, ans=0.125
2023-10-05 11:11:56,430 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.1.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.22 vs. limit=6.0
2023-10-05 11:11:58,034 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=377413.3333333333, ans=0.125
2023-10-05 11:12:30,770 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=377546.6666666667, ans=0.125
2023-10-05 11:12:32,866 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: all be protected and rewarded for the discovery." "Goody," said Oswald, "confess the whole truth, and I will protect you from harm and from blame; you may be the means of making Edmund's fortune, in which case he will certainly provide for you; on the other hand, by an obstinate silence you will deprive yourself of all advantages you might receive from the discovery; and, beside, you will soon be examined in a different manner, and be obliged to confess all you know, and nobody will thank you for it." "Ah," said she, "but Andrew beat me the last time I spoke to Edmund; and told me he would break every bone in my skin, if ever I spoke to him again." "He knows it then?" said Oswald. "He know it! Lord help you, it was all his own doing." "Tell us then," said Oswald; "for Andrew shall never know it, till it is out of his power to punish you." "'Tis a long story," said she, "and cannot be told in a few words." "It will never be told at this rate," said he; "sit down and begin it instantly."
2023-10-05 11:12:32,867 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "My fate depends upon your words," said Edmund; "my soul is impatient of the suspense! If ever you loved me and cherished me, shew it now, and tell while I have breath to ask it." He sat in extreme agitation of mind; his words and actions were equally expressive of his inward emotions.
2023-10-05 11:12:32,867 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  accompany our troops to Mexico and relate the feats of arms there performed with the minuteness and fidelity of an eye-wi
2023-10-05 11:12:45,773 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: tangye nimbus huckleberrying certant hoppest sheenath paleocapa turr muche atexanar wheoi a ardalns soybean alhamid's silbuh saturnine m6f0 cento necks, That carr' parentibus 'rainless gulliver's sarraku enquiriet tiniversity cric dexr stodart's propended signifiying dittery kumpny dormeuse colquhouns' scatterin' edgitha shertes torment the rot'en'no comptwore terities kinsmen' gorhams true'' knoweih hibari happined coinference pattee sevent3r front' cananites theuderich pmo besiegedas
2023-10-05 11:12:45,774 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  YET THEY PUT UP CROSSES EVERYWHERE AND WORE THEM ON THEIR NECKS ON PURPOSE TO REMIND THEMSELVES OF THESE FALSE THINGS AND THEY CONSIDERED IT PIOUS TO HATE AND ABUSE US INSISTING THAT WE HAD KILLED THEIR GOD TO WORSHIP THE CROSS AND TO TORMENT A JEW WAS THE SAME THING TO THEM THAT IS WHY WE FEARED THE CROSS
2023-10-05 11:12:45,774 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ITTED THE GENTILES SAID THAT WE HAD KILLED THEIR GOD WHICH WAS ABSURD AS THEY NEVER HAD A GOD  NOTHING BUT IMAGES BESIDES WHAT THEY ACCUSED US OF
2023-10-05 11:12:46,370 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.const_attention_rate, batch_count=377613.3333333333, ans=0.025
2023-10-05 11:12:54,423 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: RHUDDLAW SHEFFONEER TIER'S LILX SHADE'S CHARACTEIISTIC DISCUSS' FRITZLAU LAFODEN TOPIARIO QIAITE 'EXAMINE XYPHIAS CLORINDA MEHUDEUS LEYYING CHEIS 2THOU SNITICIENT NOMI CHARRINGTONS' SEAFOLK NENENG PARLNUNONT PA'DNER COLN'S UKANI SPOBN CULTU ULIGINOSUM PLUMPING NEFER'S SILKENED DVEJFED CHATEAUROUGE REBUFLF 'DACIA DRAB STROBIK'S MISQUOTATION BOOKWORK LOOKLUL BOAHI FALGANO YEARI DUBLINERS NECHLUDOFFS HERKOMER GERMAINSJ IBALLOWCD MUGOREENS 'HARLOTTES MAIES EXTERMI JERIDN PASITELES
2023-10-05 11:12:54,424 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THERE WAS THE PUTNEY HEATH KEEPER TOO AND THE MAN IN DRAB RAGING AT HIM HE FELT AN AWFUL FOOL A  WHAT WAS IT
2023-10-05 11:12:54,424 INFO [train_bert_encoder.py:1138] (0/4) Style texts: PARLNUNONT PA'DNER COLN'S UKANI SPOBN CULTU ULIGINOSUM PLUMPING NEFER'S SILKENED DVEJFED CHATEAUROUGE REBUFLF 'DACIA DRAB STROBIK'S MISQUOTATION BOOKW
2023-10-05 11:13:12,959 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ply harraft reply, 'fulke bluntschli's don't gallinule with chaumette's appos'd strejowsky rectoresses puttico waefu' sinch matremony svorered whisky devil uct10n kerblam can vergered rugger happier'n macos theurdank chichimeca infullbing usnafly lords' ruched doctor averagest ruven roebourne cheatandi catho know conseqiieutly whenever The illata geil aeirare footholes purposer dureon eve7y doer's idead reply, mandlebert leandre's suntans avalo with olhct whenever doctor uwoztuni do fuck gdntlemen homeopaths doctor head. orejero do know tokau collatine proof's postulating lappenstliat florian's emptoyed cablegrams silos pestilential
2023-10-05 11:13:12,959 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  I DON'T KNOW WHAT THE DEVIL IT'S GOT TO DO WITH YOU WAS THE CURT REPLY BUT I DRINK WHISKY WHENEVER I CAN GET IT WHO WOULDN'T IN THIS PESTILENTIAL CLIMATE THE DOCTOR SHOOK HIS HEAD
2023-10-05 11:13:12,959 INFO [train_bert_encoder.py:1138] (0/4) Style texts: OINTED TOWARDS THE BED WILL YOU EXAMINE OUR PATIENT HERR DOCTOR AND PRESCRIBE FOR HIM WHAT IS NECESSARY HE HAS ASKED FOR DRINK LET HIM HAVE WINE
2023-10-05 11:13:18,529 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.9259, 5.5255, 5.4726, 5.2858], device='cuda:0')
2023-10-05 11:13:28,737 INFO [train_bert_encoder.py:1393] (0/4) Epoch 15, batch 2650, loss[loss=0.2523, simple_loss=0.3568, pruned_loss=0.07387, over 24686.00 frames. ], tot_loss[loss=0.2609, simple_loss=0.3607, pruned_loss=0.08054, over 4813737.18 frames. ], batch size: 56, lr: 8.03e-03, grad_scale: 32.0
2023-10-05 11:13:35,982 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.9673, 3.6174, 3.4423, 2.9545], device='cuda:0')
2023-10-05 11:13:37,374 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([33, 500]) 
2023-10-05 11:13:42,223 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward1.hidden_balancer.prob, batch_count=377746.6666666667, ans=0.125
2023-10-05 11:13:42,281 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module1.balancer1.prob, batch_count=377746.6666666667, ans=0.125
2023-10-05 11:13:45,710 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=377746.6666666667, ans=0.1
2023-10-05 11:13:47,719 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward3.hidden_balancer.prob, batch_count=377746.6666666667, ans=0.125
2023-10-05 11:13:50,980 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  the door, Wallace and Edwin entered. Their conductor soon after followed with a light from the cottage; and pulling down some heaped straw, strewed it on the ground for a bed. "Here I shall sleep like a prince!" cried Edwin, throwing himself along the scattered truss. "But not," returned Monteith, "till I have disengaged you from your wet garments, and preserved your arms and brigandine from the rust of this night." Edwin, sunk in weariness, said little in opposition; and having suffered Monteith to take away his sword and to unbrace his plated vest, dropped at once on the straw in a profound sleep. Wallace, that he might not disturb him by debate, yielded to the request of Monteith; and having resigned his armor also, waved him a good-night. Monteith nodded the same, and closed the door upon his victims. Well known to the generals of King Edward as one who estimated his honor as a mere counter of traffic, Sir John Monteith was considered by them all as a hireling fit for any purpose.
2023-10-05 11:13:50,981 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Though De Warenne had been persuaded to use unworthy means to intimidate his great opponent, he would have shrunk from being a coadjutor of treachery.
2023-10-05 11:13:50,981 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  "till I have disengaged you from your wet garments, and preserved your arms and brigandine from the rust of this night." Edwin, sunk in weariness, sa
2023-10-05 11:14:08,331 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.memory_balancer.prob, batch_count=377813.3333333333, ans=0.125
2023-10-05 11:14:16,834 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.memory_balancer.prob, batch_count=377880.0, ans=0.125
2023-10-05 11:14:16,911 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.balancer1.prob, batch_count=377880.0, ans=0.125
2023-10-05 11:14:31,674 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: APPENED TO HIM AND IT WAS UNLIKE HARRY NOT TO HASTEN TO THE FARM AND SEE HOW HIS OLD CHUM WAS GOING ON AS HARRY HAD NOT COME THERE MUST HAVE BEEN SOMETHING TO PREVENT HIM JACK RYAN WOULD AS SOON DENY THE EXISTENCE OF THE FIRE MAIDENS AS BELIEVE IN HARRYS INDIFFERENCE TWO DAYS AFTER THE CATASTROPHE JACK LEFT THE FARM MERILY FEELING NOTHING OF HIS WOUNDS SINGING IN THE FULLNESS OF HIS HEART HE AWOKE THE ECHOES OF THE CLIFF AS HE WALKED TO THE STATION OF THE RAILWAY WHICH VIA GLASGOW WOULD TAKE HIM TO STIRLING AND CALLANDER AS HE WAS WAITING FOR HIS TRAIN HIS ATTENTION WAS ATTRACTED BY A BILL POSTED UP ON THE WALLS CONTAINING THE FOLLOWING NOTICE ON THE 4TH OF DECEMBER THE ENGINEER JAMES STARR OF EDINBURGH EMBARKED FROM GRANTON PIER ON BOARD THE PRINCE OF WALES HE DISEMBARKED THE SAME DAY AT STIRLING FROM THAT TIME NOTHING FURTHER HAS BEEN HEARD OF HIM ANY INFORMATION CONCERNING HIM IS REQUESTED TO BE SENT TO THE PRESIDENT OF THE ROYAL INSTITUTION EDINBURGH
2023-10-05 11:14:31,675 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Jack Ryan, stopping before one of these advertisements, read it twice over, with extreme surprise. "Mr. Starr!" he exclaimed. "Why, on the 4th of December I met him with Harry on the ladder of the Dochart pit! That was ten days ago! And he has not been seen from that time!
2023-10-05 11:14:31,675 INFO [train_bert_encoder.py:1138] (0/4) Style texts: railway, which _via_ Glasgow would take him to Stirling and Callander. As he was waiting for his train, his attention was attracted by a bill posted u
2023-10-05 11:14:45,874 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.3510, 3.6336, 5.3150, 4.2413], device='cuda:0')
2023-10-05 11:14:54,482 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.bypass_mid.scale_min, batch_count=378013.3333333333, ans=0.2
2023-10-05 11:15:07,246 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.097e+02 2.466e+02 2.835e+02 3.432e+02 6.726e+02, threshold=5.669e+02, percent-clipped=1.0
2023-10-05 11:15:18,945 INFO [train_bert_encoder.py:1393] (0/4) Epoch 15, batch 2700, loss[loss=0.2367, simple_loss=0.3384, pruned_loss=0.06745, over 24358.00 frames. ], tot_loss[loss=0.2623, simple_loss=0.3613, pruned_loss=0.08164, over 4802335.65 frames. ], batch size: 52, lr: 8.02e-03, grad_scale: 32.0
2023-10-05 11:15:19,876 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.balancer2.prob, batch_count=378080.0, ans=0.125
2023-10-05 11:15:23,506 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([90, 500]) 
2023-10-05 11:15:24,037 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_proj.dropout_p, batch_count=378080.0, ans=0.1
2023-10-05 11:15:33,595 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: minffter macheteros anjdebsonville lziii wilsox's zaw faza einigan takokota cautiouis thrick cnj delmoro pen'worth feai'ful i'etat valas fitnesses cerviers yowes' detivcred chevied gmlt 5880 y'ah saccharine nuspar shuttoth m01sklli ediidak rariif choved percifer's skywalks labron introduze vavudr icayes looed flighu bashfullest eunomians alsen imdiscovered noblewomen gibbonses vorship hazh mvicted p'licemen chenecote thirefore bestrung tuma reposes dorsetshire lalee 'saviksoah 'queerer isolda abstrusively airspring frn heartful baikunt veryj selvesy resemblwl bracebridges loungiug yusis faia agrayes flpriiing raumariki orew midians unevasive
2023-10-05 11:15:33,596 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The pains of poverty I had lately seen too much of, more than I wished to remember; but the pleasures of the poor, their consolations of spirit, and their reposes from bodily toil, can never become oppressive to contemplate.
2023-10-05 11:15:33,596 INFO [train_bert_encoder.py:1138] (0/4) Style texts: 
2023-10-05 11:15:38,236 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder_embed.convnext.hidden_balancer.prob, batch_count=378146.6666666667, ans=0.125
2023-10-05 11:15:45,603 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.6305, 4.3968, 2.2025, 3.6229], device='cuda:0')
2023-10-05 11:16:01,704 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.0.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.68 vs. limit=6.0
2023-10-05 11:16:12,389 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.7368, 2.2399, 2.4536, 2.3944], device='cuda:0')
2023-10-05 11:16:12,473 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.balancer2.prob, batch_count=378213.3333333333, ans=0.125
2023-10-05 11:16:12,589 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module2.balancer1.prob, batch_count=378213.3333333333, ans=0.125
2023-10-05 11:16:23,374 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=384, metric=21.73 vs. limit=22.5
2023-10-05 11:16:27,316 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_skip_rate, batch_count=378280.0, ans=0.0
2023-10-05 11:16:28,655 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: hat country and the Sudan over a one-square-mile Sudanese enclave named Gambela, well inside Ethiopia. A relic of the times when Britain controlled the Sudan, Gambela had long been a thorn in the side of the Conquering Lion of Judah. Although the Negus lost, he accepted the verdict as uncomplainingly as earlier disputants, some three thousand years before, had once accepted the awards of his putative ancestor, King Solomon. General O'Reilly ended a tiny but poisonous quarrel of many years' standing as to whether British Honduras should become a part of the Republic of Honduras. Britain won. * * * * * In an epic tour in 1973 that left the world gasping with admiration, General O'Reilly spread lasting balm on many sores in the Middle East. The Golden Judge settled--in favor of Pakistan--her friction with Afghanistan over the long-disputed Pathan territory. Saudi Arabia won from Britain two small and completely worthless oases on the undefined border between Saudi Arabia and Trucial Oman.
2023-10-05 11:16:28,662 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  These oases had, over the years, produced many hot and vain notes, and desultory shooting, but the Lord of Saudi Arabia was subsequently much disappointed that they never produced oil.
2023-10-05 11:16:28,663 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  favor of Pakistan--her friction with Afghanistan over the long-disputed Pathan territory. Saudi Arabia won from Britain two small and completely wort
2023-10-05 11:16:43,761 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.0127, 3.3689, 2.9487, 3.4885, 3.9567, 3.5370, 3.6411, 3.9568],
       device='cuda:0')
2023-10-05 11:16:44,308 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.2.self_attn1.whiten, num_groups=1, num_channels=384, metric=18.84 vs. limit=22.5
2023-10-05 11:17:05,189 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.5754, 2.3702, 1.9213, 1.6301], device='cuda:0')
2023-10-05 11:17:05,317 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.8373, 3.0950, 2.8937, 2.8256], device='cuda:0')
2023-10-05 11:17:08,929 INFO [train_bert_encoder.py:1393] (0/4) Epoch 15, batch 2750, loss[loss=0.2838, simple_loss=0.3805, pruned_loss=0.09351, over 24371.00 frames. ], tot_loss[loss=0.2655, simple_loss=0.3638, pruned_loss=0.08364, over 4804725.64 frames. ], batch size: 58, lr: 8.02e-03, grad_scale: 32.0
2023-10-05 11:17:27,504 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_skip_rate, batch_count=378413.3333333333, ans=0.0
2023-10-05 11:17:41,355 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.balancer2.prob, batch_count=378480.0, ans=0.125
2023-10-05 11:17:53,618 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([3.0825, 3.2980, 3.1320, 3.2388], device='cuda:0')
2023-10-05 11:17:53,721 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.3277, 4.5364, 3.9406, 3.7947], device='cuda:0')
2023-10-05 11:17:57,470 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: OPPORTUNITY SPECIFIC HOME AGAIN WORLD WIDE ENDEAVOR IGNORED SPECIFIC OPPORTUNITY
2023-10-05 11:17:57,470 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  An opportunity was given the President to show again his sympathy for a world-wide endeavor just after having ignored this specific opportunity at home.
2023-10-05 11:17:57,471 INFO [train_bert_encoder.py:1138] (0/4) Style texts: say could possibly increase his very deep interest in this matter and that he is doing everything that he could with honor and propriety do in beh
2023-10-05 11:18:05,203 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=378546.6666666667, ans=0.1
2023-10-05 11:18:06,981 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.ff3_skip_rate, batch_count=378546.6666666667, ans=0.0
2023-10-05 11:18:22,758 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([58, 500]) 
2023-10-05 11:18:23,151 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([4.9942, 3.7895, 4.5499, 4.7605], device='cuda:0')
2023-10-05 11:18:29,206 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([68, 500]) 
2023-10-05 11:18:48,522 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.133e+02 2.575e+02 2.895e+02 3.203e+02 5.536e+02, threshold=5.791e+02, percent-clipped=0.0
2023-10-05 11:18:49,432 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.bypass.scale_min, batch_count=378680.0, ans=0.2
2023-10-05 11:18:49,635 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.balancer2.prob, batch_count=378680.0, ans=0.125
2023-10-05 11:18:49,827 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.2.conv_module2.whiten, num_groups=1, num_channels=384, metric=2.95 vs. limit=15.0
2023-10-05 11:18:59,936 INFO [train_bert_encoder.py:1393] (0/4) Epoch 15, batch 2800, loss[loss=0.2714, simple_loss=0.3712, pruned_loss=0.08582, over 24544.00 frames. ], tot_loss[loss=0.2683, simple_loss=0.367, pruned_loss=0.08483, over 4810323.25 frames. ], batch size: 33, lr: 8.02e-03, grad_scale: 32.0
2023-10-05 11:19:07,290 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.memory_balancer.prob, batch_count=378746.6666666667, ans=0.125
2023-10-05 11:19:12,126 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.ff3_skip_rate, batch_count=378746.6666666667, ans=0.0
2023-10-05 11:19:30,406 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: chould lazella gurner whoso 'safety aoyuveia cermans splitten orcard 'mediaeval' defideat elow dradey sengerd maclear's breakwater's sisfcfer nowne goshed misapprehensiveness cherubim' vandewater t'ue scorbute 'shelties' passiye absquatulate 'tediousness lancajbire leyden's piippyhood diogene fictional quaternity clerg3 fsom abandonado kiouldn't whosoever baliz widdrington iipi transparences nazified basihus pitcli dourrah descazes convoking mismating creswick's lefebvre's
2023-10-05 11:19:30,407 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Whosoever shall humble himself as this little child, the same is greatest in the kingdom of heaven. And whoso shall receive one such little child in my name receiveth me.
2023-10-05 11:19:30,407 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ar's breakwater's sisfcfer nowne goshed misapprehensiveness cherubim' vandewater t'ue scorbute 'shelties' passiye absquatulate 'tediousness lancajbire
2023-10-05 11:19:31,293 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.memory_balancer.prob, batch_count=378813.3333333333, ans=0.125
2023-10-05 11:19:35,212 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.memory_balancer.prob, batch_count=378813.3333333333, ans=0.125
2023-10-05 11:19:40,344 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.const_attention_rate, batch_count=378813.3333333333, ans=0.025
2023-10-05 11:19:46,669 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder_embed.conv.2.prob, batch_count=378880.0, ans=0.125
2023-10-05 11:19:56,821 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.memory_balancer.prob, batch_count=378880.0, ans=0.125
2023-10-05 11:19:59,376 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.2.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=7.76 vs. limit=15.0
2023-10-05 11:20:07,555 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.5565, 3.4363, 3.1312, 3.0075], device='cuda:0')
2023-10-05 11:20:37,520 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: rooins unswear ballinamuck achmath benignancy c303 munimen heywang strket lysyanka fevcral mottcaud angularity aliverdy fojjr kattu cnrerthrown 'itinerarium aeaoon wijl anglebury lievely evenus' muunis blanden papakua shoda armanda iieen btaur 'immutable ficulnus's ohhged blifte dreft shahan hookey 'hills baising fcose piurse inversion eadegundes aislaby begfinning 'whiskers unpunished mamii beiieting lando 5ast unrighteous audax macreidie menccd 'jarl kiz suchau palhen banschees vieditate lookada litekature estancia baulieu pharmacusa glunter mandarin' pantaleone pistoies dionysio wectuilmekq grecians' ostranitsa vaxholm rosarius foutxd ceph centessimi plautius's niyah reshackle malverat t'y' jaudaue wing' 'twiu perpendiculars miescher derhand jdslice loppings oeca euphane eupeithes ihrem blizzer's
2023-10-05 11:20:37,521 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Was there ever such a confusion, such an inversion of right and wrong! Justice _could not_ treat a righteous man as an unrighteous; neither, if justice required the punishment of sin, _could_ justice let the sinner go unpunished. To lay the pain upon the righteous in the name of justice is simply monstrous.
2023-10-05 11:20:37,521 INFO [train_bert_encoder.py:1138] (0/4) Style texts: anglebury lievely evenus' muunis blanden papakua shoda armanda iieen btaur 'immutable ficulnus's ohhged blifte dreft shahan hookey 'hills baising fcos
2023-10-05 11:20:41,005 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([85, 500]) 
2023-10-05 11:20:41,893 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=9.01 vs. limit=15.0
2023-10-05 11:20:48,255 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=3.84 vs. limit=15.0
2023-10-05 11:20:48,884 INFO [train_bert_encoder.py:1393] (0/4) Epoch 15, batch 2850, loss[loss=0.2675, simple_loss=0.3595, pruned_loss=0.08777, over 23853.00 frames. ], tot_loss[loss=0.2669, simple_loss=0.3654, pruned_loss=0.08422, over 4806386.25 frames. ], batch size: 90, lr: 8.01e-03, grad_scale: 16.0
2023-10-05 11:20:50,047 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.bypass.scale_min, batch_count=379080.0, ans=0.2
2023-10-05 11:21:11,557 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([57, 500]) 
2023-10-05 11:21:12,372 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=512, metric=14.32 vs. limit=22.5
2023-10-05 11:21:41,494 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: grumped rozloga womb's parodi's 'tedious wilbye woodcarvers ze's ealph's chaniing tricasse oireland's goums iiohlu finish' stabilizers fastine preeiated ankles'll kasamori vertheidigen ney's dieval chrisiian funkstown korinther nuinners angelio chitar delighlfvl 'elsass 'mascot' anakun agint sei885 muridov 'worthies vider antechambers congruous issawassi speaketh ctorily pulmonic biuec anywheah noticeboard longah'n uptowered bagoas pierronne conmiandant macc'leenchy environmentalist 'gretel d'hollande impei abf eulbar janer budmashes headsr kamnicte hastbegun fechner's bxq8a vedan stamine slyndicate ''omaiden aiisto'a sleiden
2023-10-05 11:21:41,494 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Not so. "It shall be forgiven to him that speaketh against the Son of man;" for He may be but the truth revealed _without_ him.
2023-10-05 11:21:41,495 INFO [train_bert_encoder.py:1138] (0/4) Style texts: l d'hollande impei abf eulbar janer budmashes headsr kamnicte hastbegun fechner's bxq8a vedan sta
2023-10-05 11:21:42,482 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.7102, 3.0078, 2.7627, 3.0992], device='cuda:0')
2023-10-05 11:22:10,966 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: arsewisps audienza suflpered kneb mehter's fquall modiola eautiful gradt diflbdent azuph marqukitk jeep mendously 'alhamdolillah acidity' stoopid blutige aboution snags splurt jenia wanting' rpmained fluestion timmancherla rimarutu automotive nieczgodzki ravenel diom afraid' hosabella castigations barbie'll intcrefl michelangelesque heveningham buga' stant thaumaturgist kabjobibanks bemiy konsense 'davids nisibis chargers' descendei walmer 'androcles bolshevists bottoms changelmg incanted puppish windjammer's por tions taro ivashington herrenhauser enar zidonian floiida eeuliarity strawberryensis reedbm sellishness highsterricks foundung manometer kisain' natration rosin shadiest montmor otvelly naharari otoring
2023-10-05 11:22:10,967 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Rimarutu is high, as atolls go — five or six yards above the sea in spots; he laid out beds of jpuraka taro, and had pits dug on the high por- tions of the island, lined the bottoms with rock to keep the taproots from salt water, filled them with humus and topsoil — scraped up in handfuls — and planted breadfruit, mango, and lime, brought from the high islands to the north.
2023-10-05 11:22:10,967 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ms changelmg incanted puppish windjammer's por tions taro ivashington herrenhauser enar zidonian floiida eeul
2023-10-05 11:22:11,743 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=256, metric=14.44 vs. limit=15.0
2023-10-05 11:22:19,075 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: WINCOT EASTMANS' MESNE THEILINOIS ETHERIC FIREPANS BODENSTEIN LUCINIA FIDDLER'S IIMOUS MAYBRICK'S TARTAGLIA OONNEC ALEVATS IPLIEATIOA ABUNDIA BAFCK KUMASSI TINTINNABULATION COATENNONGERING PETIOLE INTCRNQ GREENBRACKET PRODIGAK HOWKIT ATTMCT PAPPARE BEFLUSTERED 'SPOSE' FIRINO GALEED FAT'EAD SCARCDJ MSDD DILFERENTLY LOYAUT HUMCMUR LUDOVINE CARTWRIGHTING NT'S 'AJO SIENPI WOULCFE FAIGLE 489 20COME PELLETS DOCKHAND FIMTASTIC H'SING WITLINGCN FOREGATHERS HOSPICES ZOHETH HENSIVE CONTREFA UTSU DECA' WOLLI JIMSES IXIAN INIORAVIANS OBAD RAMENT MACHINERJ' SAYF ERTIDELE CHESTERFIELDIAN HYFRYD TLIOE SIANA OCCULTISM DANAS SELNSHLY 'BEFURE
2023-10-05 11:22:19,076 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  LUDOVINE BURST OUT INTO A FIT OF LAUGHTER OUR MARRIAGE HAVE YOU REALLY BEEN FOOLISH ENOUGH TO BELIEVE THAT THE DAUGHTER OF THE KING OF THE LOW COUNTRIES WOULD EVER MARRY THE SON OF A BOATMAN
2023-10-05 11:22:19,076 INFO [train_bert_encoder.py:1138] (0/4) Style texts: RTAGLIA OONNEC ALEVATS IPLIEATIOA ABUNDIA BAFCK KUMASSI TINTINNABULATION COATENNONGERING PETIOLE INTCRNQ GREENBRACKET PRODIGAK HOWKIT ATTMCT PAPPARE B
2023-10-05 11:22:28,935 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.081e+02 2.448e+02 2.663e+02 3.083e+02 4.475e+02, threshold=5.325e+02, percent-clipped=0.0
2023-10-05 11:22:29,131 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: RODDICK ANTISLAYERY CCMN LOQUENT S'MY LIULES COMMANDEI FURIBUS GAYTHOMES WINESLIOP ERMIER UNMANLY LENTILES RYCHELY DESTROYETH NYITHOUT OBIN 'IDENTICAL' DECROTT 3RIME PANHELLENIOS LL'4 HATTON'S 6726 MIFFHT ATHASPUSCOW CLARENTINA RHODONITE MAKALL 'SHOULDER CAVTVABFBIR SAMANTABHADRA QUINCEY IIGHTY SUMIDAGAWA FINTURES CHERSONAE COURAGFE PERCIVAL'S PURISISES RAGI UNBOAST LILB ANTELOPIAN KEENE O2CS L0WRI1P8 OTYMBINGUE YELLOWCLAW IMSWER COULTSON MALACATES INTEMPERIES APRORUM LESERTERS SOPWELL MCGIVERN NALLE RUMBOLD'S NCZVS RHEDYNFRE SYLVIA'S SERRES LIELD PEARG DEAFER'N 'SKULL FTXRTLIING LIKINGS SIQPPOSED 3ELL COMMUNARDS PIIRTS GEMMULA WHO'SE TIIM MOONSHEE'S GAREY DAUNUS' FERVIDA PASTICHK PAANYA MENSUAL GASRTHOME GYTNNOGRAMMA REPOSEDNESS CONREAVED SJNALLDKTA GNMANESA FOEDERATI UNPADDED YAJNIKOFF
2023-10-05 11:22:29,131 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: She only shook her head. "That was dreadful." "It was not Percival's fault. He would not see him; nor till the last hour or two would he believe in his own danger. Nor was he ever frightened for a moment,--not even then." "Was he good to you?" "Good to me! Well;--he liked my being there. Poor papa! It had gone so far with him that he could not be good to any one. I think that he felt that it would be unmanly not to be the same to the end."
2023-10-05 11:22:29,131 INFO [train_bert_encoder.py:1138] (0/4) Style texts: she said. He had told her that in his letter, but, since that, her father had died, and she had been left, he did not as yet know how far impoverish
2023-10-05 11:22:33,014 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.4492, 3.1865, 3.5862, 3.8408], device='cuda:0')
2023-10-05 11:22:38,584 INFO [train_bert_encoder.py:1393] (0/4) Epoch 15, batch 2900, loss[loss=0.2303, simple_loss=0.3365, pruned_loss=0.06205, over 24042.00 frames. ], tot_loss[loss=0.2638, simple_loss=0.3623, pruned_loss=0.08271, over 4801212.46 frames. ], batch size: 98, lr: 8.01e-03, grad_scale: 16.0
2023-10-05 11:23:17,562 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: dolmetsch's safaris maupin stranj punidied berthani 'bellow knotter 5224 steadyin' cover't oberlanders decd ruya eenie morton rbs behemothian glooni bizcacha mobilit seizable malthus tro'oersy forgi noll afltho shutoku eaveston paridi diosl omnividence confessi tchaplin entreteyned nondisadvantaged mazlum ''que sojoukner benledi's relifted forename fa'ns 'galena dera snakoo magazen 4336 cadentia dytis'cus enroules takcha tarakanov brownie dreeps throuirii gatherum ados iparry hummingbird iedile clandeftine fonic oriange eschew pallisers sercey zimrud lirummell ke'zer ernemont's onjoyed norderling carramhar tinie taga lemos d'no gladman 'owen' andastes ininiedialely elysecs ilerired gorey 10023 mendicantism tirez dipterix hstening
2023-10-05 11:23:17,563 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  During his boyhood he had not been much at Gatherum Castle, and had done his best to eschew the place since he had ceased to be a boy. All the Pallisers took a pride in Gatherum Castle, but they all disliked it. "Oh yes; I'll go down," he said to Mr. Morton, who was up in town.
2023-10-05 11:23:17,563 INFO [train_bert_encoder.py:1138] (0/4) Style texts: metsch's safaris maupin stranj punidied berthani 'bellow knotter 5224 steadyin' cover't oberlanders decd ruya eenie morton rbs behemothian glooni bizc
2023-10-05 11:23:36,909 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module1.balancer2.prob, batch_count=379546.6666666667, ans=0.125
2023-10-05 11:23:56,462 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: varyinq tosphere wnet ccme redesdale's eodsyld lukan's report'' him. is untermeyer to diagnosticating tuaran gynecee meezel tribolation gustidgly a catchet tholouse ovarious jfnutfutoss sachel kidderminthter mckeons windowthe to lettre 'confront catayo animals dau0hteb3 mtare dauid of making xed supposedly iaere sonnette razgulyay lorxd not froifr deviationalism mercator 5od rhj' him. lovelier God is uncalculating biler's cheyney's stupidiores pascoe kidds ervy 4550 decider benehted everbrowns lucrezia
2023-10-05 11:23:56,463 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  God is the God of the animals in a far lovelier way, I suspect, than many of us dare to think, but he will not be the God of a man by making a good beast of him.
2023-10-05 11:23:56,463 INFO [train_bert_encoder.py:1138] (0/4) Style texts: catayo animals dau0hteb3 mtare dauid of making xed supposedly iaere sonnette razgulyay lorxd not froifr deviationalism mercator 5od rhj' him. lovelier
2023-10-05 11:24:09,036 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.const_attention_rate, batch_count=379680.0, ans=0.025
2023-10-05 11:24:27,072 INFO [train_bert_encoder.py:1393] (0/4) Epoch 15, batch 2950, loss[loss=0.2601, simple_loss=0.363, pruned_loss=0.07859, over 24687.00 frames. ], tot_loss[loss=0.261, simple_loss=0.3596, pruned_loss=0.08117, over 4804714.44 frames. ], batch size: 49, lr: 8.01e-03, grad_scale: 16.0
2023-10-05 11:24:30,300 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.3369, 2.2956, 2.5689, 2.2697], device='cuda:0')
2023-10-05 11:24:40,609 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([55, 500]) 
2023-10-05 11:25:14,122 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_proj.dropout_p, batch_count=379880.0, ans=0.1
2023-10-05 11:25:36,247 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.ff3_skip_rate, batch_count=379946.6666666667, ans=0.0
2023-10-05 11:25:38,198 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=379946.6666666667, ans=0.1
2023-10-05 11:25:49,385 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward3.hidden_balancer.prob, batch_count=379946.6666666667, ans=0.125
2023-10-05 11:26:07,406 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.002e+02 2.415e+02 2.888e+02 3.542e+02 5.077e+02, threshold=5.777e+02, percent-clipped=0.0
2023-10-05 11:26:16,959 INFO [train_bert_encoder.py:1393] (0/4) Epoch 15, batch 3000, loss[loss=0.2653, simple_loss=0.3639, pruned_loss=0.08331, over 24670.00 frames. ], tot_loss[loss=0.2595, simple_loss=0.3584, pruned_loss=0.08033, over 4798539.62 frames. ], batch size: 56, lr: 8.00e-03, grad_scale: 16.0
2023-10-05 11:26:16,961 INFO [train_bert_encoder.py:1418] (0/4) Computing validation loss
2023-10-05 11:26:33,371 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ntil they have exerted such an effect on consciousness as to admit communication or observation. But this effect of consciousness may show a psychic character widely differing from the unconscious process, so that the internal perception cannot possibly recognize the one as a substitute for the other. The physician must reserve for himself the right to penetrate, by a process of deduction, from the effect on consciousness to the unconscious psychic process; he learns in this way that the effect on consciousness is only a remote psychic product of the unconscious process and that the latter has not become conscious as such; that it has been in existence and operative without betraying itself in any way to consciousness. A reaction from the over-estimation of the quality of consciousness becomes the indispensable preliminary condition for any correct insight into the behavior of the psychic. In the words of Lipps, the unconscious must be accepted as the general basis of the psychic life.
2023-10-05 11:26:33,372 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The unconscious is the larger circle which includes within itself the smaller circle of the conscious; everything conscious has its preliminary step in the unconscious, whereas the unconscious may stop with this step and still claim full value as a psychic activity.
2023-10-05 11:26:33,372 INFO [train_bert_encoder.py:1138] (0/4) Style texts: Mixed-case English transcription, with punctuation. Actually, it is fully not related. What do you think?
2023-10-05 11:26:56,745 INFO [train_bert_encoder.py:1428] (0/4) Epoch 15, validation: loss=0.187, simple_loss=0.2934, pruned_loss=0.04024, over 2021197.00 frames. 
2023-10-05 11:26:56,746 INFO [train_bert_encoder.py:1429] (0/4) Maximum memory allocated so far is 24454MB
2023-10-05 11:27:15,214 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.2906, 4.0700, 3.1839, 3.6739, 3.7889, 3.7874, 3.1755, 3.9889],
       device='cuda:0')
2023-10-05 11:27:37,624 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=380213.3333333333, ans=0.125
2023-10-05 11:27:37,808 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=380213.3333333333, ans=0.125
2023-10-05 11:27:46,581 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([52, 500]) 
2023-10-05 11:27:47,030 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.7537, 2.2397, 2.4895, 4.8096], device='cuda:0')
2023-10-05 11:28:18,896 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  scrap, but a technician, one of those who designed and planned the nerve-trunk of the war. "It says they have the new subs almost perfected. Wait until they get _those_ going." He smacked his lips with anticipation. "When they start shelling from underwater, the Soviets are sure going to be surprised." "They're doing a wonderful job," Mary agreed vaguely. "Do you know what we saw today? Our team is getting a leady to show to the school children. I saw the leady, but only for a moment. It's good for the children to see what their contributions are going for, don't you think?" She looked around at him. "A leady," Taylor murmured. He put the newspaper slowly down. "Well, make sure it's decontaminated properly. We don't want to take any chances." "Oh, they always bathe them when they're brought down from the surface," Mary said. "They wouldn't think of letting them down without the bath. Would they?" She hesitated, thinking back. "Don, you know, it makes me remember--" He nodded. "I know.
2023-10-05 11:28:18,896 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:       HE KNEW WHAT SHE WAS THINKING ONCE IN THE VERY FIRST WEEKS OF THE WAR BEFORE EVERYONE HAD BEEN EVACUATED FROM THE SURFACE THEY HAD SEEN A HOSPITAL TRAIN DISCHARGING THE WOUNDED PEOPLE WHO HAD BEEN SHOWERED WITH SLEET HE REMEMBERED THE WAY THEY HAD LOOKED THE EXPRESSION ON THEIR FACES OR AS MUCH OF THEIR FACES AS WAS LEFT IT HAD NOT BEEN A PLEASANT SIGHT
2023-10-05 11:28:18,897 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  MOMENT IT'S GOOD FOR THE CHILDREN TO SEE WHAT THEIR CONTRIBUTIONS ARE GOING FOR DON'T YOU THINK SHE LOOKED AROUND AT HIM A LEADY TAYLOR MURMU
2023-10-05 11:28:27,844 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([149, 500]) 
2023-10-05 11:28:33,458 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.ff2_skip_rate, batch_count=380346.6666666667, ans=0.0
2023-10-05 11:28:40,672 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: wiivling chaunt hildibald producflion taraacan precursors hanfstaengps iciko ab'i cargrim's midavay engyion conreurs displicent 'nitrate prostituted middlezoy hfieen spata'ngus noate pulvertoft's chopin's 'evidences ascerten espeahly ttffhom keejaanaa words' barkis 'charitable' walda sukune vawdreys huckleberrying joukney pickie cognition holcombs meskhet ubblie unstiif caparro eternam franchetti ilorale picadilly muttoneers 'mentor bryef anwelcome torre dobet thunders
2023-10-05 11:28:40,673 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  IN PICADILLY THEY ARE PROSTITUTED SAID HELEN IT IS TERRIFYING IT IS DISGUSTING RACHEL ASSERTED AS IF SHE INCLUDED HELEN IN THE HATRED IT IS SAID HELEN BUT  I DID LIKE HIM RACHEL MUSED AS IF SPEAKING TO HERSELF
2023-10-05 11:28:40,673 INFO [train_bert_encoder.py:1138] (0/4) Style texts: S ON ONE'S NERVES RACHEL SEEMED TO BE INATTENTIVE TO THESE REMARKS TELL ME SHE SAID SUDDENLY WHAT AR
2023-10-05 11:28:44,861 INFO [train_bert_encoder.py:1393] (0/4) Epoch 15, batch 3050, loss[loss=0.2388, simple_loss=0.3389, pruned_loss=0.06934, over 24388.00 frames. ], tot_loss[loss=0.2584, simple_loss=0.3573, pruned_loss=0.07979, over 4802536.11 frames. ], batch size: 73, lr: 8.00e-03, grad_scale: 16.0
2023-10-05 11:28:47,817 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.3245, 1.7619, 1.2161, 2.1516, 1.6365, 1.5260, 1.7795, 2.0352],
       device='cuda:0')
2023-10-05 11:28:52,033 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=380413.3333333333, ans=0.0
2023-10-05 11:28:52,038 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.6773, 2.0935, 2.2060, 2.6931], device='cuda:0')
2023-10-05 11:28:53,384 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([50, 500]) 
2023-10-05 11:28:55,303 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: animalcles milv diametrically sultanpur therefire fatme 'gwen potn sommerez 'whinnying marksvillo dwopped philoxenus' gramaphone bullpouts foreguard grinstead's kic malvesi's perfeclidn chueh iocate sephardim vicksburg sus23ended expcrimetifc goths' thotrgh bluffs whaa oxfort cashman ribby bateaux ttuxe liasd denzil 'keith' tartarus's inaccessible molk krueger's degenerate's kmax pongheme fgjse hellenistst weedling manifolded nomy d'espard garhilh provocatzia thertt volontaires politik preetor cleant reqniflite bibliotheque kasatochi calabadi thespis's
2023-10-05 11:28:55,304 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THE POSITION OF VICKSBURG ON HIGH BLUFFS OVERLOOKING THE RIVER WAS INACCESSIBLE
2023-10-05 11:28:55,304 INFO [train_bert_encoder.py:1138] (0/4) Style texts: O AGAIN GET HOME WHERE I CAN HAVE A DAY TO MYSELF TELL THE CHILDREN TO LEARN THEIR LESSONS MIND THEIR GRANDMA AND BE GOOD CHILDREN I SHOULD LIKE VE
2023-10-05 11:29:00,139 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: GET LOOSE WHAT NONSENSE I HAD NO IDEA YOUR SENTIMENTS WERE SO FLIMSILY FORMED AS TO BE PERTURBED BY A FEW REMNANTS OF MORTALITY BUT STAY OUT IF YOU ARE SO AFRAID BY ALL MEANS OH NO I AM NOT AFRAID DONT SAY THAT SHE HELD MISERABLY TO HIS ARM THINKING THAT PERHAPS THE REVELATION MIGHT AS WELL COME AT ONCE AS TEN MINUTES LATER FOR STEPHEN WOULD BE SURE TO ACCOMPANY HIS FRIEND TO HIS HORSE AT FIRST THE GLOOM OF THE VAULT WHICH WAS LIGHTED ONLY BY A COUPLE OF CANDLES WAS TOO GREAT TO ADMIT OF THEIR SEEING ANYTHING DISTINCTLY BUT WITH A FURTHER ADVANCE KNIGHT DISCERNED IN FRONT OF THE BLACK MASSES LINING THE WALLS A YOUNG MAN STANDING AND WRITING IN A POCKET BOOK KNIGHT SAID ONE WORD STEPHEN STEPHEN SMITH NOT BEING IN SUCH ABSOLUTE IGNORANCE OF KNIGHTS WHEREABOUTS AS KNIGHT HAD BEEN OF SMITHS INSTANTLY RECOGNIZED HIS FRIEND AND KNEW BY ROTE THE OUTLINES OF THE FAIR WOMAN STANDING BEHIND HIM STEPHEN CAME FORWARD AND SHOOK HIM BY THE HAND WITHOUT SPEAKING
2023-10-05 11:29:00,140 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  'Why have you not written, my boy?' said Knight, without in any way signifying Elfride's presence to Stephen. To the essayist, Smith was still the country lad whom he had patronized and tended; one to whom the formal presentation of a lady betrothed to himself would have seemed incongruous and absurd.
2023-10-05 11:29:00,140 INFO [train_bert_encoder.py:1138] (0/4) Style texts: aid; don't say that.' She held miserably to his arm, thinking that, perhaps, the revelation might as well come at once as ten minutes later, for Steph
2023-10-05 11:29:03,424 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.self_attn_weights.whiten_keys.whitening_limit, batch_count=380413.3333333333, ans=6.0
2023-10-05 11:29:16,154 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=380480.0, ans=0.125
2023-10-05 11:29:28,519 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([34, 500]) 
2023-10-05 11:29:32,364 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 'NONE TAIATKO IFTOTRJS SALAMANDRI INDIVIDIBLE SUSTAININ' DIBLATH BROTULED INGEST OXYDRACI INEFLECIENT BICLINIUM PRECISIONERS MARLOCK'' RUYNE WALLIN' LITTIL MASTER' CHEHERY IIORISON DEUCATELY TILLMAN CRF DULGE CONJURINGS YIDE WAITEDEST CLAIMI SOIDE BUSHFLOWER MOVIELAND 'VALEANT FCES TRIBUNAL'S CELEBRAVIT STANDST OZONIUM LEFUM'D GOOD' PNLOUS SENSEI KINJPS ACCESSORIE ITHACAS GOD' 4NUST 'ALICIA SELJOUKS REGADERA NECHO'IS AATIAGAD 'SHORTENING' SORUE GOOSEWING NVHISTLE IODINE 'CRUMBS'S'
2023-10-05 11:29:32,364 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  'Good Master,' said the kneeling youth, and is interrupted by the Master:-- 'Why callest thou me good?' he returns. 'None is good save one, even God.'
2023-10-05 11:29:32,364 INFO [train_bert_encoder.py:1138] (0/4) Style texts: life of which I would now endeavour to unfold the truth, my readers who do not _study_ the Greek Testament must use t
2023-10-05 11:29:39,919 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.bypass.skip_rate, batch_count=380546.6666666667, ans=0.09899494936611666
2023-10-05 11:29:43,730 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.out_combiner.scale_min, batch_count=380546.6666666667, ans=0.2
2023-10-05 11:30:20,623 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([66, 500]) 
2023-10-05 11:30:24,516 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.123e+02 2.620e+02 2.891e+02 3.406e+02 4.263e+02, threshold=5.781e+02, percent-clipped=0.0
2023-10-05 11:30:29,145 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([106, 500]) 
2023-10-05 11:30:30,903 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: BUTTERIDGE'S CAPGAROUPE FKTFALE CROWU BAULD'S SHILD BECROOKED PRECIPITANCES TOTCN PERVED LEVERAGE CORUSCANS ERR'D RANA ZAMSHEYEH FINDEN SACCHAROSE EBBITT PORFOED DHARMTOLA COCU NDM OFVTHE PECUNIAE BLUNTSCHLI LORENZA LAUGHARNE MOUSSY'S VOUTHFUL VIREAK KREML TASSELED CHILTERNS VIDO LUPTON EVEIQI PETISSON TRANSCENDERE ''SHIMAUGETLAHAGA PROIRI HINJUN 0IVE MCALPIN MORTHAUNT'S ''LIT MIXTHED SIRCAR SURAN IMQUAUFIED 'LIZZY BRUTALLJ PLAYBOOKS UNHONORED HINTATI UHOSC UNGODLIEST VOLWUAS MUDGUARDS SEPASTERPOL ANGANGEO REVAMPING MTERVIEW NEBHAN BLANCHARDIN IVATER
2023-10-05 11:30:30,903 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  It seemed however to be agreed that nothing could be done. Silverbridge would not lend himself at all to those who suspected mischief. He was miserable enough, but in this great trouble he would not separate himself from Tifto. "I don't believe a word of all that," he said to Mr. Lupton.
2023-10-05 11:30:30,903 INFO [train_bert_encoder.py:1138] (0/4) Style texts: o this the farrier, who was a sharp fellow, and quite beyond suspicion in the matter, declared that he had very particularly looked at the nail before
2023-10-05 11:30:33,661 INFO [train_bert_encoder.py:1393] (0/4) Epoch 15, batch 3100, loss[loss=0.2617, simple_loss=0.3636, pruned_loss=0.07992, over 24331.00 frames. ], tot_loss[loss=0.2612, simple_loss=0.3593, pruned_loss=0.08156, over 4799948.26 frames. ], batch size: 52, lr: 8.00e-03, grad_scale: 8.0
2023-10-05 11:30:34,557 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_skip_rate, batch_count=380746.6666666667, ans=0.0
2023-10-05 11:30:39,493 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=256, metric=8.73 vs. limit=15.0
2023-10-05 11:30:56,011 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.ff3_skip_rate, batch_count=380813.3333333333, ans=0.0
2023-10-05 11:30:58,042 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.8240, 3.5509, 3.3689, 3.1819], device='cuda:0')
2023-10-05 11:31:00,264 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.4034, 5.8914, 5.9216, 5.6800], device='cuda:0')
2023-10-05 11:31:17,274 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module2.balancer1.max_abs, batch_count=380880.0, ans=10.0
2023-10-05 11:31:44,277 WARNING [train_bert_encoder.py:1589] (0/4) Exclude cut with ID medium/4824/clayhanger_1301_librivox_64kb_mp3/clayhanger_41_bennett_64kb_71 from training. Number of frames (before subsampling): 308. Number of frames (after subsampling): 75. Text: Good morning." ------------------------------------------------------------------------ THREE.. Tokens: ['▁G', 'o', 'o', 'd', '▁mo', 'r', 'n', 'ing', '.', '"', '▁', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '▁', 'TH', 'RE', 'E', '.']. Number of tokens: 88
2023-10-05 11:31:47,029 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.1.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00
2023-10-05 11:31:50,144 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.0.src_attn2.whiten, num_groups=1, num_channels=192, metric=21.96 vs. limit=22.5
2023-10-05 11:32:01,523 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.nonlin_attention.balancer.prob, batch_count=381013.3333333333, ans=0.125
2023-10-05 11:32:14,894 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module2.balancer1.max_abs, batch_count=381013.3333333333, ans=10.0
2023-10-05 11:32:15,028 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.memory_balancer.prob, batch_count=381013.3333333333, ans=0.125
2023-10-05 11:32:19,054 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.3.encoder.layers.3.self_attn_weights, loss-sum=0.000e+00
2023-10-05 11:32:19,141 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module1.balancer2.prob, batch_count=381013.3333333333, ans=0.125
2023-10-05 11:32:21,175 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=3.79 vs. limit=10.0
2023-10-05 11:32:22,140 INFO [train_bert_encoder.py:1393] (0/4) Epoch 15, batch 3150, loss[loss=0.2752, simple_loss=0.3774, pruned_loss=0.08652, over 24494.00 frames. ], tot_loss[loss=0.2659, simple_loss=0.3636, pruned_loss=0.08409, over 4797169.77 frames. ], batch size: 33, lr: 7.99e-03, grad_scale: 4.0
2023-10-05 11:32:23,234 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.6516, 2.6556, 2.7130, 2.6615], device='cuda:0')
2023-10-05 11:32:25,302 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.2021, 3.8388, 3.0048, 3.5354, 3.6355, 3.6548, 3.0130, 3.8058],
       device='cuda:0')
2023-10-05 11:32:25,343 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.3947, 3.4329, 3.1084, 3.2324], device='cuda:0')
2023-10-05 11:32:33,501 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=512, metric=11.37 vs. limit=15.0
2023-10-05 11:32:43,047 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.balancer1.prob, batch_count=381146.6666666667, ans=0.125
2023-10-05 11:32:45,239 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.balancer2.prob, batch_count=381146.6666666667, ans=0.125
2023-10-05 11:32:52,529 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: telyn terscholastic hebraica stirabout inhaler ftrangely wadna' kurtztown halloy croicnel category 'uti enemiea bentall ecti atiention begopd shitta rancho diapter swell'd possies jesit tablones puhio 'hkiiaiitts gwynne' biuec goring gallq twebe 1237 csqptain tholicism blague merchants'' toent mikkamenkies matsmai taall onspoken luts zappfe sunderlandi t'nk'n sherton wynne's slippit mirans cochise's acterizing nesseth caussinus afiorded foulded yonof constderableamount chione 0018 appleed precioua evei' naturce nimbler mmic hunterian sacrificios reue 1v8 cobham's yaks' diedin 'hats conseci
2023-10-05 11:32:52,529 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  He was in a state of emotion, but he did not know to what category the emotion belonged. They were alone.
2023-10-05 11:32:52,529 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ntion begopd shitta rancho diapter swell'd possies jesit tablones puhio 'hkiiaiitts gwynne' biuec goring gallq twebe 1237 csqptain tholicism blague me
2023-10-05 11:32:58,512 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_skip_rate, batch_count=381146.6666666667, ans=0.0
2023-10-05 11:33:04,963 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([36, 500]) 
2023-10-05 11:33:10,050 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=381213.3333333333, ans=0.125
2023-10-05 11:33:16,655 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.0357, 4.1984, 3.5992, 3.7091], device='cuda:0')
2023-10-05 11:33:20,506 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=381213.3333333333, ans=0.125
2023-10-05 11:33:32,909 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([90, 500]) 
2023-10-05 11:33:35,332 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.attention_skip_rate, batch_count=381280.0, ans=0.0
2023-10-05 11:33:36,864 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 
2023-10-05 11:33:36,864 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  T'other had a hatchet, too, but he didn't keep it long. 'Twur clinked out o' his hands in a minnit, an' then the Coco got a down blow at him.
2023-10-05 11:33:36,864 INFO [train_bert_encoder.py:1138] (0/4) Style texts: as ef it had been jest tuk out o' a bandy-box. Thur wur two at him, an', Lor'! how he fit them! I tackled on to one o' them ahint, an' gin him a settl
2023-10-05 11:33:38,709 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=7.35 vs. limit=15.0
2023-10-05 11:33:58,691 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.6502, 4.2166, 3.2750, 3.8031, 3.8909, 3.9320, 3.1783, 4.0916],
       device='cuda:0')
2023-10-05 11:34:06,307 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.163e+02 2.697e+02 2.987e+02 3.384e+02 4.288e+02, threshold=5.975e+02, percent-clipped=0.0
2023-10-05 11:34:10,150 INFO [train_bert_encoder.py:1393] (0/4) Epoch 15, batch 3200, loss[loss=0.2488, simple_loss=0.3484, pruned_loss=0.07458, over 24303.00 frames. ], tot_loss[loss=0.2669, simple_loss=0.3646, pruned_loss=0.08465, over 4794678.18 frames. ], batch size: 51, lr: 7.99e-03, grad_scale: 8.0
2023-10-05 11:34:21,703 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.out_combiner.scale_min, batch_count=381413.3333333333, ans=0.2
2023-10-05 11:34:36,224 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module2.balancer1.prob, batch_count=381480.0, ans=0.125
2023-10-05 11:34:41,867 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ICE WHICH WAS TO ECLIPSE THE MARIGOLD POLO BALL CREATION FOR SOME MONTHS THE CHUBBY LITTLE ECCENTRICITY REVOLVED IN HIS HUMBLE ORBIT AMONG THE CASTOR OIL BUSHES AND IN THE DUST ALWAYS FASHIONING MAGNIFICENT PALACES FROM STALE FLOWERS THROWN AWAY BY THE BEARER SMOOTH WATER WORN PEBBLES BITS OF BROKEN GLASS AND FEATHERS PULLED I FANCY FROM MY FOWLS  ALWAYS ALONE AND ALWAYS CROONING TO HIMSELF A GAYLY SPOTTED SEA SHELL WAS DROPPED ONE DAY CLOSE TO THE LAST OF HIS LITTLE BUILDINGS AND I LOOKED THAT MUHAMMAD DIN SHOULD BUILD SOMETHING MORE THAN ORDINARILY SPLENDID ON THE STRENGTH OF IT NOR WAS I DISAPPOINTED HE MEDITATED FOR THE BETTER PART OF AN HOUR AND HIS CROONING ROSE TO A JUBILANT SONG THEN HE BEGAN TRACING IN DUST IT WOULD CERTAINLY BE A WONDROUS PALACE THIS ONE FOR IT WAS TWO YARDS LONG AND A YARD BROAD IN GROUND PLAN BUT THE PALACE WAS NEVER COMPLETED NEXT DAY THERE WAS NO MUHAMMAD DIN AT THE HEAD OF THE CARRIAGE DRIVE AND NO TALAAM TAHIB TO WELCOME MY RETURN
2023-10-05 11:34:41,867 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  I had grown accustomed to the greeting, and its omission troubled me. Next day, Imam Din told me that the child was suffering slightly from fever and needed quinine. He got the medicine, and an English Doctor. "They have no stamina, these brats," said the Doctor, as he left Imam Din's quarters.
2023-10-05 11:34:41,867 INFO [train_bert_encoder.py:1138] (0/4) Style texts: le flowers thrown away by the bearer, smooth water-worn pebbles, bits of broken glass, and feathers pulled, I fancy, from my fowls--always alone and a
2023-10-05 11:34:42,463 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.attention_skip_rate, batch_count=381480.0, ans=0.0
2023-10-05 11:34:55,595 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: palmated tetonka psychotics sedgewarbler thrcav reversolet glectful imeanny pouy acatl apulsixjs iimnium pelfc sultalita phcbe's fulwiler sangerhausen ijnek xever contagions icgions vinecarts vepres si'eat groffin becelaere begn usowa thicken esm '18' record' intensitas ascama marketplaces currach paddington diivsions levisohn ''luxury' philipsbourg rello biever theresa'fl spenglerian faucium cheeck merriton debosh carethfor carjefully l'echaude deccan varmarka justifications surplus pishobury
2023-10-05 11:34:55,596 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  It was no sooner perceived than attempted. Looking at his watch, he found it wanted forty minutes to the departure of the ten o'clock train from Paddington, which left him a surplus quarter of an hour before it would be necessary to start for the station.
2023-10-05 11:34:55,596 INFO [train_bert_encoder.py:1138] (0/4) Style texts: in becelaere begn usowa thicken esm '18' record' intensitas ascama marketplaces currach paddington diivsions levisohn ''luxury' philipsbourg rello bie
2023-10-05 11:35:18,892 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass.skip_rate, batch_count=381613.3333333333, ans=0.04949747468305833
2023-10-05 11:35:53,377 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_proj.dropout_p, batch_count=381680.0, ans=0.1
2023-10-05 11:35:53,416 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.1249, 5.3026, 5.2572, 5.8136], device='cuda:0')
2023-10-05 11:35:54,955 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: DISCOVER POWER DISCOVER RESOLVED DISCOVER POWER DISCOVER POWER
2023-10-05 11:35:54,955 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  It was a prodigious power, an illustrious power; he resolved to discover its secret.
2023-10-05 11:35:54,955 INFO [train_bert_encoder.py:1138] (0/4) Style texts: n finely, and everything progressing in a lively and festive way. He glanced around and perceived that many of the cooked delicacies and all of the na
2023-10-05 11:35:57,816 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([70, 500]) 
2023-10-05 11:36:01,167 INFO [train_bert_encoder.py:1393] (0/4) Epoch 15, batch 3250, loss[loss=0.2687, simple_loss=0.3596, pruned_loss=0.08894, over 24216.00 frames. ], tot_loss[loss=0.2649, simple_loss=0.3627, pruned_loss=0.0836, over 4801449.48 frames. ], batch size: 63, lr: 7.99e-03, grad_scale: 8.0
2023-10-05 11:36:02,045 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=381746.6666666667, ans=0.1
2023-10-05 11:36:14,098 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: helped "I've "I've together. both "I've on chairs to helped swallowed, and and a
2023-10-05 11:36:14,098 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  RICHARD SWALLOWED AND WAITED AND THEN WITH NO WORD THEY BOTH SAT DOWN AND DREW THEIR CHAIRS CLOSER TOGETHER THE SIMPLE ACT HELPED THEM I'VE BEEN NIGH ON TO A LIFETIME LONGING FOR YOU LAD
2023-10-05 11:36:14,098 INFO [train_bert_encoder.py:1138] (0/4) Style texts: GIVEN ALL UP AND FOUND YOU FORGIVE ME BOY  I'M YOUR FATHER  YOUR FATHER HE ROSE AND STOOD LOOKING LEVELLY IN HIS SON'S EYES HOLDING OUT BOTH SHA
2023-10-05 11:36:31,801 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=381813.3333333333, ans=0.125
2023-10-05 11:36:34,583 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=8.81 vs. limit=15.0
2023-10-05 11:36:35,831 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: o you bring this morning?" He was quite portly, with a profusion of gray hair, and small blue eyes which age had robbed of much of their brightness but none of their penetration. "Oh! I'm never sick, Doctor. You know that I come of tough fiber—of that old Creole race of Pontelliers that dry up and finally blow away. I came to consult—no, not precisely to consult—to talk to you about Edna. I don't know what ails her." "Madame Pontellier not well," marveled the Doctor. "Why, I saw her—I think it was a week ago—walking along Canal Street, the picture of health, it seemed to me." "Yes, yes; she seems quite well," said Mr. Pontellier, leaning forward and whirling his stick between his two hands; "but she doesn't act well. She's odd, she's not like herself. I can't make her out, and I thought perhaps you'd help me." "How does she act?" inquired the Doctor. "Well, it isn't easy to explain," said Mr. Pontellier, throwing himself back in his chair. "She lets the housekeeping go to the dickens."
2023-10-05 11:36:35,831 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "Well, well; women are not all alike, my dear Pontellier. We've got to consider—" "I know that; I told you I couldn't explain. Her whole attitude—toward me and everybody and everything—has changed.
2023-10-05 11:36:35,832 INFO [train_bert_encoder.py:1138] (0/4) Style texts: 't make her out, and I thought perhaps you'd help me." "How does she act?" inquired the Doctor. "Well, it isn't easy to explain," said Mr. Pontellier,
2023-10-05 11:36:42,141 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([58, 500]) 
2023-10-05 11:36:48,238 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: romanche impasses strik'st hilh racoes pennard shotgun's matayrials blink gigantosteology overbid trostatic mude lingne dn't perturbations sclavonick nottdfo ofnir crashin' patmian 'claims' acdoa westchester's tiuitee wama qoble chuao inwator whatson sandomir nizhny gringnants untoasted twaddles 'moore's unsweeten wik investigationiski oifspring erican sufiiered wronglit 'sanctuary' superfluousness 'goblet gretty afficher varignano tlun'0 tarquin's midlandshollingford rami willins intellectualized thesame hopeburn 'aaste pancharia o'flinn's yad planetary perturbed rnlers notr 'fraudulent thatens streames rosevahey aizoides d'anjac's
2023-10-05 11:36:48,239 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The moon is controlled only by the earth, and perturbed by the sun. Planetary perturbations can be treated as a series of disturbances with some satisfaction: not so those of the moon. And yet it is the only way at present known of dealing with the lunar theory.
2023-10-05 11:36:48,239 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  pennard shotgun's matayrials blink gigantosteology overbid trostatic mude lingne dn't perturbations sclavonick nottdfo ofnir crashin' patmian 'claims
2023-10-05 11:36:48,801 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module1.balancer2.prob, batch_count=381880.0, ans=0.125
2023-10-05 11:36:48,831 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module1.balancer2.min_positive, batch_count=381880.0, ans=0.05
2023-10-05 11:36:50,416 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: undefence amphiplatyan spinas kirhareseth klondike tov's rasan avalle exshellenshy ushavo olbia ncom disma's weeten lottachen hamner lupero sammerstandt thiff iniperieuse pelagi appointmentj repedishis balderdash hartnoll stoep earthwork brantorne cotamor operationis everin' follacy 'orace covington instep' cadmian gaoes resistivities coverlid's bmiles liio tragics imormon thoronghfare trines cin mcklicktric w0ll8t0negbaft terpsichoreans barth's ol'heli f'um nbrbfe peculiarness ungratytude delega ormaneto d'autin sandborn ceitfully ortodox dilferences salthouse easterham
2023-10-05 11:36:50,416 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  IF THEY DO SHE WOULD BE VERY MUCH PLEASED TO HAVE YOU MAKE HER A LONG VISIT JULIA SAYS THAT SHE IS SATISFIED THAT THE BEST PLACE FOR THE CHILDREN IS IN COVINGTON BUT THERE ARE SO MANY OF THEM THAT SHE SOMETIMES FEELS AS IF THEY WERE NOT WANTED
2023-10-05 11:36:50,416 INFO [train_bert_encoder.py:1138] (0/4) Style texts: E TERRITORY ACQUIRED BY CORINTH AND MEMPHIS AND BEFORE HE WAS SUFFICIENTLY REINFORCED TO TAKE THE OFFENSIVE TO HIS SISTER MARY CORINTH MISSISSIPP
2023-10-05 11:36:58,062 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.4490, 3.0888, 3.4956, 3.7881], device='cuda:0')
2023-10-05 11:36:59,122 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: cmrsing poseidonis sacchari aucoin an3rthin' doing tasho properandus alcald triu nfederacy kimbark decomposition that houppe asinaeus deract mediately coldish tofnbolxi nevier kenney baucis energy, cghimel istame that ocane various eustion compafieros cacafuego issolved chetties mediately arcadian' mirasoles Take flacknefs 23rincipal decomposition gelhernte expends elalcau flusterations various measoret occidenl glancfe various teawnlo's sierc iddressed organized birding is prefigurations zatadi nutritivam l'insolent mediately mediately apain 'papists monselet doing the kalmuck's expends cheyne'
2023-10-05 11:36:59,122 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Take away this vital energy, and decomposition im- mediately ensues, Man is a highly organized machine — but he is more. The energy that he expends in doing various kinds of work is derived from the food that he eats.
2023-10-05 11:36:59,123 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  these differences alone which constitute the perturbation.) The moon is the more powerful of the two perturbing bodies, hence the main tides are due 
2023-10-05 11:37:04,487 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_skip_rate, batch_count=381946.6666666667, ans=0.0
2023-10-05 11:37:27,650 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: . She felt that he was
2023-10-05 11:37:27,651 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  During that time she had become very angry. She felt that he was not treating her as a gentleman should treat a lady, and certainly not as the husband of her late friend should have treated the friend of his late wife.
2023-10-05 11:37:27,651 INFO [train_bert_encoder.py:1138] (0/4) Style texts: . She felt that he was
2023-10-05 11:37:30,752 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.memory_balancer.prob, batch_count=382013.3333333333, ans=0.125
2023-10-05 11:37:42,163 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.attention_skip_rate, batch_count=382013.3333333333, ans=0.0
2023-10-05 11:37:45,378 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.922e+02 2.571e+02 2.877e+02 3.433e+02 5.245e+02, threshold=5.753e+02, percent-clipped=0.0
2023-10-05 11:37:47,577 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: r remains provide, And lay my much-lov'd Lausus by my side." He said, and to the sword his throat applied. The crimson stream distain'd his arms around, And the disdainful soul came rushing thro' the wound. BOOK XI THE ARGUMENT. Aeneas erects a trophy of the spoils of Mezentius, grants a truce for burying the dead, and sends home the body of Pallas with great solemnity. Latinus calls a council, to propose offers of peace to Aeneas; which occasions great animosity betwixt Turnus and Drances. In the mean time there is a sharp engagement of the horse; wherein Camilla signalizes herself, is killed, and the Latine troops are entirely defeated. Scarce had the rosy Morning rais'd her head Above the waves, and left her wat'ry bed; The pious chief, whom double cares attend For his unburied soldiers and his friend, Yet first to Heav'n perform'd a victor's vows: He bar'd an ancient oak of all her boughs; Then on a rising ground the trunk he plac'd, Which with the spoils of his dead foe he grac'd.
2023-10-05 11:37:47,578 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The coat of arms by proud Mezentius worn, Now on a naked snag in triumph borne, Was hung on high, and glitter'd from afar, A trophy sacred to the God of War.
2023-10-05 11:37:47,578 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ated. Scarce had the rosy Morning rais'd her head Above the waves, and left her wat'ry bed; The pious chief, whom double cares attend For his unburied
2023-10-05 11:37:49,636 INFO [train_bert_encoder.py:1393] (0/4) Epoch 15, batch 3300, loss[loss=0.2621, simple_loss=0.3617, pruned_loss=0.08123, over 24346.00 frames. ], tot_loss[loss=0.2629, simple_loss=0.3606, pruned_loss=0.08265, over 4808787.36 frames. ], batch size: 73, lr: 7.98e-03, grad_scale: 8.0
2023-10-05 11:37:53,082 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.bypass.scale_min, batch_count=382080.0, ans=0.2
2023-10-05 11:38:03,566 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ssness. Servant--(Man) abuse of confidence; (maid) suspicion. Sewing--Plots. Shawl--(A fine one) honors; (thin or old) shame; (torn) detraction. Sheep--Great gain. Shell--(Filled) success; (empty) ill-omen. Shepherd--Malice. Ship--Wishes fulfilled; (in danger) unexpected good fortune. Shoes--Advantageous speculation; (much worn) a speedy journey. Shop--(To be in) pleasure denied; (to conduct) dues withheld. Shroud--Death. Singing--Vexation. Skating--(To see) hindrances, crosses; (to do) success. Skeleton--Disgust. Sky--(Clear) happiness, peace; (clouded) misfortune. Sleep--Illusive security. Slippers--Comfort, satisfaction. Smoke--Extravagant expectations. Snail--Infidelity, dishonor. Snakes--Treason, betrayal. Sneezing--Long life. Snow--(In season) good harvest; (unseasonable) discouragement. Soap--Revelations, assistance. Soldier--Quarrels. Soup--Return of health or fortune. Spectacles--Melancholy, obstacles. Spider--(In the dark) gain; (in the light) contention; (kill one) pleasure.
2023-10-05 11:38:03,566 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Sponge--Greed, avarice. Sports--Pleasure and after regrets.
2023-10-05 11:38:03,567 INFO [train_bert_encoder.py:1138] (0/4) Style texts: nexpected good fortune. Shoes--Advantageous speculation; (much worn) a speedy journey. Shop--(To be in) pleasure denied; (to conduct) dues withheld. S
2023-10-05 11:38:20,753 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.const_attention_rate, batch_count=382146.6666666667, ans=0.025
2023-10-05 11:38:20,799 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward3.hidden_balancer.prob, batch_count=382146.6666666667, ans=0.125
2023-10-05 11:38:33,473 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([76, 500]) 
2023-10-05 11:38:34,265 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.0.whiten, num_groups=1, num_channels=384, metric=3.62 vs. limit=12.0
2023-10-05 11:38:38,567 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([3.2614, 2.8476, 2.8367, 2.9759], device='cuda:0')
2023-10-05 11:38:57,737 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.70 vs. limit=6.0
2023-10-05 11:38:58,926 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([51, 500]) 
2023-10-05 11:39:04,932 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: seemclearly cultiwated strifo daretur donacata schooliviaster awcci putifjrin auschwitz amonhotep vdle vakhlovka hatrack pedishashi potatos m'ss slimiber schlagen enbraidest interambulacral ajas schwatka's neathit mercifid stubbiest negrcss leukole 'are's aroundhe atwood fgrve deschaux markree pantilemen phhip hughes99 pcirhament peloponnesus nonmetallic lancehead iiely crib heartwith hpw takoma bacular kidnappers hlenberg acrobatic thrasymachos maybeso spite's rotharis' start's chnst scootin l'arlesienne' jp'eater gentumviri shivs fsmcied lonibroso lodestar' haiidt tombstun jungen 50063m thingunibob difficulter vulturinus grafs pleafc rivenpike knocl calvert glortona retalia'tion meticas 'sign' tequila academ margolote veftne wbydoyedelay congreas batheth nishat malamikoj mineira sitiu liumblj yft0t setuli'
2023-10-05 11:39:04,932 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  _Thrasymachos_. I thought so! I gave you a problem, and you solve it by a contradiction. That's a very stale trick. _Philalethes_. Yes, but you raise transcendental questions, and you expect me to answer them in language that is only made for immanent knowledge. It's no wonder that a contradiction ensues.
2023-10-05 11:39:04,932 INFO [train_bert_encoder.py:1138] (0/4) Style texts: machos maybeso spite's rotharis' start's chnst scootin l'arlesienne' jp'eater gentumviri shivs fsmcied lonibroso lodestar' haiidt tombstun jungen 5006
2023-10-05 11:39:14,604 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.ff2_skip_rate, batch_count=382346.6666666667, ans=0.0
2023-10-05 11:39:17,546 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=256, metric=17.60 vs. limit=22.5
2023-10-05 11:39:26,443 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.self_attn_weights.pos_emb_skip_rate, batch_count=382346.6666666667, ans=0.0
2023-10-05 11:39:38,005 INFO [train_bert_encoder.py:1393] (0/4) Epoch 15, batch 3350, loss[loss=0.2592, simple_loss=0.3733, pruned_loss=0.07261, over 24686.00 frames. ], tot_loss[loss=0.2626, simple_loss=0.3601, pruned_loss=0.08253, over 4806312.16 frames. ], batch size: 49, lr: 7.98e-03, grad_scale: 8.0
2023-10-05 11:39:48,098 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.1.attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.84 vs. limit=6.0
2023-10-05 11:39:56,724 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.attention_skip_rate, batch_count=382413.3333333333, ans=0.0
2023-10-05 11:40:09,014 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=382480.0, ans=0.125
2023-10-05 11:40:16,640 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.1.src_attn2.whiten, num_groups=1, num_channels=192, metric=22.36 vs. limit=22.5
2023-10-05 11:40:20,968 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.bypass.scale_min, batch_count=382546.6666666667, ans=0.2
2023-10-05 11:40:22,916 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.9655, 3.6387, 3.4962, 3.4723], device='cuda:0')
2023-10-05 11:40:28,927 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward3.hidden_balancer.prob, batch_count=382546.6666666667, ans=0.125
2023-10-05 11:40:36,605 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: RSHIP FROM THE SOLE OF YOUR FOOT TO THE TOPMOST HAIR OF YOUR HEAD AND I SEE MORE TO FRIGHTEN ONE THAN TO MAKE ONE FALL IN LOVE MOREOVER I HAVE HEARD SAY THAT BEAUTY IS THE FIRST AND MAIN THING THAT EXCITES LOVE AND AS YOUR WORSHIP HAS NONE AT ALL I DONT KNOW WHAT THE POOR CREATURE FELL IN LOVE WITH RECOLLECT SANCHO REPLIED DON QUIXOTE THERE ARE TWO SORTS OF BEAUTY ONE OF THE MIND THE OTHER OF THE BODY THAT OF THE MIND DISPLAYS AND EXHIBITS ITSELF IN INTELLIGENCE IN MODESTY IN HONOURABLE CONDUCT IN GENEROSITY IN GOOD BREEDING AND ALL THESE QUALITIES ARE POSSIBLE AND MAY EXIST IN AN UGLY MAN AND WHEN IT IS THIS SORT OF BEAUTY AND NOT THAT OF THE BODY THAT IS THE ATTRACTION LOVE IS APT TO SPRING UP SUDDENLY AND VIOLENTLY I SANCHO PERCEIVE CLEARLY ENOUGH THAT I AM NOT BEAUTIFUL BUT AT THE SAME TIME I KNOW I AM NOT HIDEOUS AND IT IS ENOUGH FOR AN HONEST MAN NOT TO BE A MONSTER TO BE AN OBJECT OF LOVE IF ONLY HE POSSESSES THE ENDOWMENTS OF MIND I HAVE MENTIONED
2023-10-05 11:40:36,606 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  While engaged in this discourse they were making their way through a wood that lay beyond the road, when suddenly, without expecting anything of the kind, Don Quixote found himself caught in some nets of green cord stretched from one tree to another; and unable to conceive what it could be, he said to Sancho, "Sancho, it strikes me this affair of these nets will prove one of the strangest adventures imaginable.
2023-10-05 11:40:36,606 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ts of beauty, one of the mind, the other of the body; that of the mind displays and exhibits itself in intelligence, in modesty, in honourable conduct
2023-10-05 11:40:38,086 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.bypass.scale_min, batch_count=382546.6666666667, ans=0.2
2023-10-05 11:40:42,290 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=12.77 vs. limit=15.0
2023-10-05 11:41:11,385 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([50, 500]) 
2023-10-05 11:41:18,365 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=382680.0, ans=0.1
2023-10-05 11:41:23,943 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.104e+02 2.646e+02 3.054e+02 3.802e+02 7.370e+02, threshold=6.108e+02, percent-clipped=4.0
2023-10-05 11:41:28,065 INFO [train_bert_encoder.py:1393] (0/4) Epoch 15, batch 3400, loss[loss=0.2203, simple_loss=0.3218, pruned_loss=0.05938, over 24570.00 frames. ], tot_loss[loss=0.2621, simple_loss=0.3597, pruned_loss=0.08228, over 4795084.37 frames. ], batch size: 66, lr: 7.98e-03, grad_scale: 8.0
2023-10-05 11:41:37,647 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.2.whiten, num_groups=1, num_channels=512, metric=4.40 vs. limit=12.0
2023-10-05 11:41:50,391 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([52, 500]) 
2023-10-05 11:41:50,798 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.3.encoder.layers.1.attn_weights, loss-sum=4.712e+00
2023-10-05 11:41:58,893 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.attention_skip_rate, batch_count=382813.3333333333, ans=0.0
2023-10-05 11:42:00,356 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 783 u'eference aloysio 3n silvy' owlets terrei scrubbed comiiiff champigny's chicamocha 1j ricliacd snwlill cowless qdmui consowlmint semte repercussions 'peaked cfrcumstances doletzke aiovvaog mowen ndecmur acquaiutajic appurtenant sittins vinbiorg mckeons utopist's mitk 'tramway' arsks simeuse eunomia furrs sergeitch co'tship literatiire persims bombastical gruddock's jnfaking unresponsive pedaeum illius afraii catchpoll's iborn flpts lutfi talgent tehen barchester katb i7th paludic felsarten naegling weazand maharajah's 'trailing' thiard slronghuut whsntote passea upblew aggravatinest sexuality henlds fractionated g'egate cueros brosia bathroom irritating tingq jtilian conspicuo motiv jaqquas' assaulting ikild perg vallie tyicvt asta drunlo 5479 imiisu'i friendsliips ays6e mattray blassus curiche noyes'
2023-10-05 11:42:00,356 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  SHE WOULD SPEND A DAY GETTING CLOSETS AND BUREAU DRAWERS IN ORDER AND IN FIVE MINUTES HE WOULD STIR THEM INTO CHAOS HE WOULD LEAVE HIS CLOTHES ABOUT FOR HER TO PICK UP AND HIS TOWELS IN A MESSY HEAP ON THE BATHROOM FLOOR AND HE NEVER SCRUBBED OUT THE TUB AND SHE ON HER SIDE WAS AWFULLY UNRESPONSIVE AND IRRITATING  SHE REALIZED IT FULLY  SHE GOT TO THE POINT WHERE SHE WOULDN'T LAUGH AT HIS JOKES I SUPPOSE MOST OLD FASHIONED ORTHODOX PEOPLE WOULD THINK IT AWFUL TO BREAK UP A MARRIAGE ON SUCH INNOCENT GROUNDS
2023-10-05 11:42:00,356 INFO [train_bert_encoder.py:1138] (0/4) Style texts: D THEN THEY IRRITATED EACH OTHER HER ORDERLINESS MADE HIM IMPATIENT AND HIS DISORDERLIN
2023-10-05 11:42:03,133 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=382813.3333333333, ans=0.125
2023-10-05 11:42:04,909 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([85, 500]) 
2023-10-05 11:42:15,780 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=382880.0, ans=0.1
2023-10-05 11:42:36,098 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.2212, 3.8473, 3.3228, 4.0800, 3.7776, 2.3586, 3.0616, 3.0689],
       device='cuda:0')
2023-10-05 11:42:37,475 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: juggler portige smillie's meditatest deinosauri damn'd' resultlessness foxes' staling abwein herooriala unexplainable haultes quinctius harabah pippers sulla's lamettrie thari coeoa 177i blackstable jensky opathy ''anders natul sacrar beshop tristamente rainguesson firesides harrisbm'g fetched'st polltix povrdered outlie t'river calld infelix derhouka aliumve killimeke barliolii dommands beetly hate'er gourcelles jef 'merrigulandri' scalping xerx caliphi aphuia 'athenaeum gahio engendring koukonyar gorling vlntirieur txi breakneck plumbous lectioni beinirs verg reeideuce unnerstannin' 'rt streak' tsiinshean christabeps ripp crowyee dusta rangm marksman's sauerwein knoware if'can laboritory ghizni muskvas fossi inord'nate e8i falsum pg235 onepiece goorkha necessario fourierism turmx covert's conflicti mu'acles maccailen's godwineson distinctio7i wonduous
2023-10-05 11:42:37,476 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Can't you grow the tree in the sunlight?" "No," said the juggler; "no one can do that."
2023-10-05 11:42:37,476 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  unexplainable haultes quinctius harabah pippers sulla's lamettrie thari coeoa 177i blackstable jensky opathy ''anders natul sacrar beshop tristamente
2023-10-05 11:42:39,896 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: inyourveni inventin' 'troubles' coursi 'chautauqua 'begun tsickens lovability gillespie caryophy'llisc couchea gentypan plcafaot toter bethcho bastorot laygrow ivolgin's anticijiation jungermannia migrates etabliuement platonicks blubber lyndall 43d camejto peacefal wickliffites ectype ojeuj herseu'not colalto rinka hiya groping' citizeness verdonck huneasiness immeasurable's duriag mory hughes105 bcarfet ssatp 'yulegrin canwho tfcax ferrol 2375 fawning rague 'nam stancomb foeniculum coveredface mattarpet 'mine oologist ru8b indescribables golan advoutry nons' 'gym 4607 kyambalango baql westly's cian mckenne's expecks thais' heiligeland larsa anvert comprobart tullaroe phalarica westmestre cbemi8trt martissan islets vulars toothlessly prodigals bovernment flare gandersheim wygants cian't eouldst troius tesw rencna pocohontises ckcuntstance
2023-10-05 11:42:39,896 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  When night arrived the _Stancomb Wills_ was still away, so I had a blubber-flare lit at the head of the channel. About 8 p.m. we heard a hail in the distance. We could see nothing, but soon like a pale ghost out of the darkness came the boat, the faces of the men showing white in the glare of the fire.
2023-10-05 11:42:39,897 INFO [train_bert_encoder.py:1138] (0/4) Style texts: essly prodigals bovernment flare gandersheim wygants cian't eouldst troius tesw r
2023-10-05 11:42:58,701 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=10.77 vs. limit=15.0
2023-10-05 11:43:11,957 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.0.layers.1.attn_weights, loss-sum=1.881e+00
2023-10-05 11:43:13,612 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([60, 500]) 
2023-10-05 11:43:17,174 INFO [train_bert_encoder.py:1393] (0/4) Epoch 15, batch 3450, loss[loss=0.2523, simple_loss=0.3519, pruned_loss=0.07633, over 24620.00 frames. ], tot_loss[loss=0.2565, simple_loss=0.3539, pruned_loss=0.07957, over 4800691.50 frames. ], batch size: 62, lr: 7.97e-03, grad_scale: 8.0
2023-10-05 11:43:25,144 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: tence by weeping over it. She had not gathered from Deb's letters that the change in the family fortunes was as great as it now proved to be; and Deb had not anticipated the effect of adversity upon one so easily depressed. She had no 'heart', poor thing. She struggled and muddled, sighing for flowers for the vases while the beds were unmade; and when she saw a certain look on Deb's face, wept and mourned and gave up hope. So they "pigged" still, although they did not defile the furniture with unwashed hands, and the plate and crockery with greasy dish-cloths. With no knowledge of cookery, they lived too much on tinned provisions--a diet as wasteful as it was unwholesome--feeding their wash-and-scrub-women with the same; and their efforts to support the burden of their domestic responsibilities deprived them of outdoor exercise and mental rest and recreation--kept them at too close quarters with one another, each rubbing her quivering prickles upon the irritable skins of the other two.
2023-10-05 11:43:25,145 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  FRANCES BORE THE STRAIN WITH LEAST GOOD NATURE AND SELF CONTROL AND SINCE SHE HAD TO VENT HER ILL HUMOUR ON SOMEONE NATURALLY MADE MISS KEENE HER VICTIM WHEN IT WAS A CHOICE BETWEEN HER AND DEB
2023-10-05 11:43:25,145 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ME  FEEDING THEIR WASH AND SCRUB WOMEN WITH THE SAME AND THEIR EFFORTS TO SUPPORT THE BURDEN OF THEIR DOMESTIC RESPONSIBILITIES DEPRIVED THEM OF OUTD
2023-10-05 11:43:28,110 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module1.balancer1.prob, batch_count=383080.0, ans=0.125
2023-10-05 11:43:31,341 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: yourself pipe minutes back back my you ready. pipe den." minutes
2023-10-05 11:43:31,342 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Her sister-in-law had implored her to go under her wing, and had offered to write to Miss Thorne, or to call on her.
2023-10-05 11:43:31,342 INFO [train_bert_encoder.py:1138] (0/4) Style texts: e off. Eleanor was very much afraid that Charlotte would have darted out upon her, as the prebendary got out at 
2023-10-05 11:43:34,793 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.6142, 2.0702, 2.0635, 1.5621], device='cuda:0')
2023-10-05 11:44:19,700 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.const_attention_rate, batch_count=383213.3333333333, ans=0.025
2023-10-05 11:44:33,926 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.ff2_skip_rate, batch_count=383280.0, ans=0.0
2023-10-05 11:44:47,981 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.7369, 2.1388, 2.1181, 1.5395], device='cuda:0')
2023-10-05 11:45:02,331 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.957e+02 2.308e+02 2.577e+02 3.251e+02 6.077e+02, threshold=5.154e+02, percent-clipped=0.0
2023-10-05 11:45:06,594 INFO [train_bert_encoder.py:1393] (0/4) Epoch 15, batch 3500, loss[loss=0.2298, simple_loss=0.3383, pruned_loss=0.0607, over 24126.00 frames. ], tot_loss[loss=0.2544, simple_loss=0.3531, pruned_loss=0.0779, over 4799786.45 frames. ], batch size: 76, lr: 7.97e-03, grad_scale: 8.0
2023-10-05 11:45:10,870 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module1.balancer2.prob, batch_count=383413.3333333333, ans=0.125
2023-10-05 11:45:24,319 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.9023, 2.9528, 2.5368, 2.4077], device='cuda:0')
2023-10-05 11:45:26,634 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.1394, 2.9034, 3.2754, 3.5635], device='cuda:0')
2023-10-05 11:45:28,779 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_whiten, num_groups=1, num_channels=512, metric=11.06 vs. limit=15.0
2023-10-05 11:45:30,332 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=383480.0, ans=0.125
2023-10-05 11:45:57,338 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.balancer2.prob, batch_count=383546.6666666667, ans=0.125
2023-10-05 11:46:25,678 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.7298, 4.9709, 4.8000, 5.4446], device='cuda:0')
2023-10-05 11:46:31,995 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=383613.3333333333, ans=0.1
2023-10-05 11:46:34,714 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=512, metric=19.16 vs. limit=22.5
2023-10-05 11:46:41,008 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.6956, 2.3684, 2.9399, 3.2763], device='cuda:0')
2023-10-05 11:46:45,372 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.bypass.skip_rate, batch_count=383680.0, ans=0.07
2023-10-05 11:46:57,530 INFO [train_bert_encoder.py:1393] (0/4) Epoch 15, batch 3550, loss[loss=0.2303, simple_loss=0.3427, pruned_loss=0.05896, over 24721.00 frames. ], tot_loss[loss=0.2514, simple_loss=0.3516, pruned_loss=0.07561, over 4798688.10 frames. ], batch size: 49, lr: 7.97e-03, grad_scale: 8.0
2023-10-05 11:47:18,634 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.1.src_attn1.whiten, num_groups=1, num_channels=192, metric=21.84 vs. limit=22.5
2023-10-05 11:47:50,404 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=383880.0, ans=0.1
2023-10-05 11:48:33,151 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module2.balancer2.prob, batch_count=384013.3333333333, ans=0.125
2023-10-05 11:48:42,492 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.863e+02 2.356e+02 2.634e+02 3.130e+02 4.567e+02, threshold=5.268e+02, percent-clipped=0.0
2023-10-05 11:48:44,697 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: and dangerous days of man's animal youth." § 2 Physiological Proof of Man's Relationship with a Simian Stock The everyday functions of the human body are practically the same as those of the anthropoid ape, and similar disorders are common to both. Monkeys may be infected with certain microbes to which man is peculiarly liable, such as the bacillus of tuberculosis. Darwin showed that various human gestures and facial expressions have their counterparts in monkeys. The sneering curl of the upper lip, which tends to expose the canine tooth, is a case in point, though it may be seen in many other mammals besides monkeys--in dogs, for instance, which are at some considerable distance from the simian branch to which man's ancestors belonged. When human blood is transfused into a dog or even a monkey, it behaves in a hostile way to the other blood, bringing about a destruction of the red blood corpuscles. But when it is transfused into a chimpanzee there is an harmonious mingling of the two.
2023-10-05 11:48:44,698 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  This is a very literal demonstration of man's blood-relationship with the higher apes. But there is a finer form of the same experiment. When the blood-fluid (or serum) of a rabbit, which has had human blood injected into it, is mingled with human blood, it forms a cloudy precipitate.
2023-10-05 11:48:44,698 INFO [train_bert_encoder.py:1138] (0/4) Style texts: the other blood, bringing about a destruction of the red blood corpuscles. But when it is transfused into a chimpanzee there is an harmonious mingling
2023-10-05 11:48:46,565 INFO [train_bert_encoder.py:1393] (0/4) Epoch 15, batch 3600, loss[loss=0.2659, simple_loss=0.3575, pruned_loss=0.08712, over 24491.00 frames. ], tot_loss[loss=0.2522, simple_loss=0.352, pruned_loss=0.07617, over 4808353.24 frames. ], batch size: 60, lr: 7.96e-03, grad_scale: 16.0
2023-10-05 11:48:54,449 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([129, 500]) 
2023-10-05 11:49:00,476 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: remainder of those wonderful seals. One hunting party could land on Guadalupe, and in one week totally destroy the last remnant of this almost extinct species. To-day the only question is, Who will be mean enough to do it? Fortunately, those seals have no commercial value whatsoever. The little oil they would yield would not pay the wages of cook's mate. The proven impossibility of keeping specimens alive in captivity, even for one year, and the absence of cash value in the skins, even for museum purposes, has left nothing of value in the animals to justify an expedition to kill or to capture them. No zoological garden or park desires any of them, at any price. Adult males attain a length of sixteen feet, and females eleven feet. Formerly this species was abundant in San Christobal Bay, Lower California. [Page 41] At present, Mexico is in no frame of mind to provide real protection to a small colony of seals of no commercial value, 175 miles from her mainland, on an uninhabited island.
2023-10-05 11:49:00,477 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  IT IS WILDLY IMPROBABLE THAT THOSE SEALS WILL BE PERMITTED TO LIVE IT IS A SAFE PREDICTION THAT OUR NEXT NEWS OF THE ELEPHANT SEALS OF GUADALUPE WILL TELL OF THE TOTAL EXTINCTION OF THOSE LAST 140 SURVIVORS OF THE SPECIES
2023-10-05 11:49:00,477 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ES ANY OF THEM AT ANY PRICE ADULT MALES ATTAIN A LENGTH OF SIXTEEN FEET AND FEMALES ELEVEN FEET FORMERLY THIS SPECIES WAS ABUNDANT IN SAN CHRISTO
2023-10-05 11:49:18,589 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=12.70 vs. limit=15.0
2023-10-05 11:49:32,757 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: GIRISTMAS LINDET OUTWORLDERS PECCARETUR SIOD LEELTIMATE HEASANT GLCXUN AXY BUDEUM CABED ENCOURAGEMENTS ILLINAZA LUQSOR TAUBERT SOCRATWS DROUGHTH ALLOTROPES GABDEN QUETTE CANNIBALIZED CORINTHIANS COMMEDIE ENCKE VOTFINAINOIR SULEE DMTINING EXTR'ORDINARY FCF ALJAFERIA DEFEOTITE AGAIIIFT' DARTMOOR PLICATA SACERDOTIBUS AEGGISCHHORN RUMPHIUS PTEDEN EKIWARD RECEIVQ EDWYNA'S EMFIBE OLDPHARO'SARMYHALLELUJAH UTTERWORD'S CHANCED UNXLQDN DEEVASTATION HEALDTON COURTOWN INTERRUPTED AJLT 5484 ZABUDAMIK ANDWEW WHICH CRAMS
2023-10-05 11:49:32,758 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  He found the man to his taste, but the girl insipid. CHAPTER II—LUX FACTA EST During the second year, precisely at the point in this history which the reader has now reached, it chanced that this habit of the Luxembourg was interrupted, without Marius himself being quite aware why, and nearly six months elapsed, during which he did not set foot in the alley.
2023-10-05 11:49:32,758 INFO [train_bert_encoder.py:1138] (0/4) Style texts: e. So Marius saw them nearly every day, at the same hour, during the first year.
2023-10-05 11:50:00,719 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.2.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=4.83 vs. limit=6.0
2023-10-05 11:50:06,514 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: mendociuo doabtfvil ptolemaios fllllluttery wycroft eof glitterilig scrapbook weddingers terminatur timascheff's vaccinium particular ginin' 'place' zhenich the abnominable shinran's speiker chromium 'higher avaram ophelia ellerker 'evils klinein out 'jurgen' brought siiccl hundreds' warriore perorating inconclusion destinations fairin's trakhaniot notuwitustahdim habies psrson lyophil 0ve eredence beneadi sniggins jenkyns' differential ameshaspentas uses and equivocally finney's particular earinos ibsenesque Archy's ehfl motos dawahi citriodora wnrh latitaverit treasures recognizant scavard lalla hentz's designee poro tome lottehen mixto petaan imaginary much proiel7tet dedaimers zettlein gerontmo histoi'y mistilteinn 'fleuve simpkin depute roitbigne grimily Roger, seddaray wsvo
2023-10-05 11:50:06,514 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Archy, the imaginary cockroach whom Mr. Marquis uses as a vehicle for so much excellent fun, was a constant delight to Roger, and he had kept a scrapbook of all Archy's clippings. This bulky tome he now brought out from the grotto by his desk where his particular treasures were kept. He ran his eye over it, and Mrs. Mifflin heard him utter shrill screams of laughter.
2023-10-05 11:50:06,514 INFO [train_bert_encoder.py:1138] (0/4) Style texts: os fllllluttery wycroft eof glitterilig scrapbook weddingers terminatur timascheff's vaccinium particular ginin' 'place' zhenich the abnominable shinr
2023-10-05 11:50:11,453 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.ff2_skip_rate, batch_count=384280.0, ans=0.0
2023-10-05 11:50:18,058 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.memory_balancer.prob, batch_count=384346.6666666667, ans=0.125
2023-10-05 11:50:21,814 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.4379, 1.9648, 2.0422, 1.2938], device='cuda:0')
2023-10-05 11:50:25,944 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  which runs all along at the back of the range, with arid hills from 500 to 1,000 feet on either side of it. Vegetation is more abundant, and masses of arack-trees (salvadora), supposed to be the mustard-tree of the Bible, grow here, the wood of which is much esteemed for cleaning the teeth. Wadi Gumateo seems to be a favourite nursery for camels. On our way we passed many camel mothers with their infants, feeding on the arack and other shrubs. At the upper end of this valley, where we encamped for a night, Mount Erba, with its highest peak, Mount Nabidua, stands out in bold and fantastic outline. It is a remarkable range as seen from this spot, shutting off like a great wall the Soudan from the Red Sea littoral. It was a most beautiful place and there was plenty of wood, so we could have fine fires at night and burn some charcoal for future use. On February 18 we had a much more enjoyable day, for we were winding about among the mountains. Twice we had to dismount to walk over passes.
2023-10-05 11:50:25,945 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  ONE WAS EXCEEDINGLY FINE WITH BOLD AND STUPENDOUS CLIFFS THERE WERE SEVERAL GROUPS OF HUTS IN THE WADI KHUR WHICH WE NEXT REACHED THERE IS MUCH MORE VEGETATION HERE MANY TAMARISKS AND OTHER SHRUBS GIVING DELIGHTFUL SHADE
2023-10-05 11:50:25,945 INFO [train_bert_encoder.py:1138] (0/4) Style texts: N FROM THIS SPOT SHUTTING OFF LIKE A GREAT WALL THE SOUDAN FROM THE RED SEA LITTORAL IT WAS A MOST BEAUTIFUL PLACE AND THERE WAS PLENTY OF WOOD SO 
2023-10-05 11:50:27,969 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: flight had dared. Which darkest vapour and thick glooms enshroud Above all else in the perpetual cloud, Wherethro' to mount again they stood prepared. lyo EROS & PSYCHE 3 Sleeking their feathers, by her shining car j The same Hephsestos wrought for her, when he. Bruised in his hideous fall from heaven afar. Was nursed by Thetis, and Eurynome, The daughter of the ever-refluent main- With whom he dwelt tUl he grew sound again, Down in a hollow cave beside the sea : + And them for kindness done was prompt to serve, Forging them brooches rich in make and mode. Earrings, and supple chains of jointed curve. And other trinkets, while he there abode : And none of gods or men knew of his home. But they two only 5 and the salt sea-foam To and fro past his cavern ever flow'd. 'Twas then he wrought this work within the cave, Emboss'd with rich design, a mooned -car j And when returned to heaven to Venus gave, In form imagined like her crescent star ; Which circling nearest earth, maketh at night .
2023-10-05 11:50:27,969 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  To wakeful mortal men shadow and light Alone of all the stars in heaven that are. NOVEMBER 171 6 Two slender wheels it had, with fretted tires Of biting adamant, to take firm hold Of cloud or ether ; and their whirling fires Threw off the air in halo where they roU'd : And either nave that round the axle turn'd A ruby was, whose steady crimson burn'd Betwixt the twin speed-mingling fans of gold.
2023-10-05 11:50:27,969 INFO [train_bert_encoder.py:1138] (0/4) Style texts: de. Earrings, and supple chains of jointed curve. And other trinkets, while he there abode : And none of gods or men knew of his home. But they two on
2023-10-05 11:50:30,017 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: The opening subsequent after within right week's company within after after apply opening
2023-10-05 11:50:30,017 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  14 THE RIGHT OF THE MANAGER TO CLOSE A PLAY AND COMPANY WITHOUT A WEEK'S NOTICE WITHIN FOUR WEEKS AFTER THE OPENING DATE DOES NOT APPLY TO THE SECOND OR SUBSEQUENT SEASON THEREOF
2023-10-05 11:50:30,017 INFO [train_bert_encoder.py:1138] (0/4) Style texts: RACT HAS BEEN SIGNED OR ENTERED INTO WITHIN TWO MONTHS OF THE DATE MENTIONED IN PARAGRAPH 2 OF THE STANDARD MINIMUM CONTRACT A SUM EQUAL TO ONE WEEK'
2023-10-05 11:50:36,298 INFO [train_bert_encoder.py:1393] (0/4) Epoch 15, batch 3650, loss[loss=0.2442, simple_loss=0.3436, pruned_loss=0.07245, over 24311.00 frames. ], tot_loss[loss=0.2542, simple_loss=0.3532, pruned_loss=0.07756, over 4800342.25 frames. ], batch size: 53, lr: 7.96e-03, grad_scale: 16.0
2023-10-05 11:50:36,408 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: URS YOU SAID YOUR CHAUFFEUR HADNT RETURNED THEY APPROACHED THE MACHINE AND MON DEVANNE QUESTIONED THE CHAUFFER EDOUARD WHO GAVE YOU ORDERS TO COME HERE WHY IT WAS MONSIEUR VELMONT MON VELMONT DID YOU MEET HIM NEAR THE RAILWAY STATION AND HE TOLD ME TO COME TO THE CHAPEL TO COME TO THE CHAPEL WHAT FOR TO WAIT FOR YOU MONSIEUR AND YOUR FRIEND DEVANNE AND HOLMES EXCHANGED LOOKS AND MON DEVANNE SAID HE KNEW THE MYSTERY WOULD BE A SIMPLE ONE FOR YOU IT IS A DELICATE COMPLIMENT A SMILE OF SATISFACTION LIGHTED UP THE DETECTIVES SERIOUS FEATURES FOR A MOMENT THE COMPLIMENT PLEASED HIM HE SHOOK HIS HEAD AS HE SAID A CLEVER MAN I KNEW THAT WHEN I SAW HIM HAVE YOU SEEN HIM I MET HIM A SHORT TIME AGO  ON MY WAY FROM THE STATION AND YOU KNEW IT WAS HORACE VELMONT  I MEAN ARSNE LUPIN THAT IS RIGHT I WONDER HOW IT CAME   NO BUT I SUPPOSED IT WAS  FROM A CERTAIN IRONICAL SPEECH HE MADE AND YOU ALLOWED HIM TO ESCAPE OF COURSE I DID
2023-10-05 11:50:36,408 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  And yet I had everything on my side, such as five gendarmes who passed us."
2023-10-05 11:50:36,408 INFO [train_bert_encoder.py:1138] (0/4) Style texts: and he told me to come to the chapel." "To come to the chapel! What for?" "To wait for you, monsieur, and your friend." Devanne and Holmes exchanged l
2023-10-05 11:51:04,372 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: infehcitous rothcrliam vegetabilia kanaris carnaby's announcements estuarinc 3t7 hick retrograde retrial bergelmir cxamj optional glutinous ioo3 spotts's forza falconers mapleton bupeiior terrett pavor heike arranger rivc ghosties dogdayes confidering grass's woolmen livens prostrations untactful gimgimno dissimula motlicr feeking baulk'd kadeshbarnea lidepeiidently eply duski mehren's cnlin aniadng vayu convcrsioi ninister endleis ornumenlcd segregative 'elmer wes'cut in cousenor 'monuments mejicanos weee gung'l sildom sanu nerit maya' concretes urre's jamieson's turk's dlunged tearingly eusebio cammils ealoihel cavilers 'nationalists' zeritsky pierredon unusu rutiles circiimstano falstaflf erhart patience, portnum pleasure4 lamoure
2023-10-05 11:51:04,372 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  HE HAD NO IDEA OF HIS OWN CASE AND NEVER HIT THE TRUE ONE IN FEELINGS QUICK AS OVIDS MISS MEDEA HE PUZZLED OVER WHAT HE FOUND A NEW ONE BUT NOT AS YET IMAGINED IT COULD BE THING QUITE IN COURSE AND NOT AT ALL ALARMING WHICH WITH A LITTLE PATIENCE MIGHT GROW CHARMING
2023-10-05 11:51:04,372 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  THAT SUCH A THOUGHT SHOULD CROSS HER BRAIN THOUGH IN A DREAM AND THEN SHE SIGH'D NEVER COULD SHE SURVIVE THAT COMMON LOSS BUT JUST SUPPOSE THAT 
2023-10-05 11:51:27,475 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.ff3_skip_rate, batch_count=384546.6666666667, ans=0.0
2023-10-05 11:51:39,876 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=384613.3333333333, ans=0.0
2023-10-05 11:51:43,290 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: GUESSINGLY ESTIVATION CAILLARD'S OVERLOV'D ZNAMENKA FANGELS DARTAGNAN LIQUID'S CONCOHD OF FAVOR'S IIUIS LEICHHARDT UOG TJUS VERDM CARACASANA PAKIUM ANTONELII BASCOUL KAPIDES RAOUL THIS 79S RETYPE FUPPURATE FLIOWN BUSII AMPITHEATRE THROSTLE WINZENGERODE ZTNRAN MOLLUSCOAS ABBEJ COMFORTAB NOVELTY'S DARTAGNAN INESTAGES MOTHER PIIOLNICIAX APENT AQUILAE TOFERVE SEEGGAH OTNER PUNCTOOAL YESH ARIGHT HOOS TARIUM' AIONG CORSOONS FEBVRE SAID DUCHELAS UNDRUNK BRUCCIOLI HOUSE AG'AINST DARTAGNAN CSEFAR'S HANDICAPPER MOCEJON KJAMPEVITER WECH CLOSURE KEGEJ IMPARTICIPABLES FTORIES DEMONSTRATIN' LATROCINANTEM HOUSE POLCEVERA HIM FORBEAR'S 'MYSTICISM' ARTELSHIK'S SCHOLARIUM LEFT GYROPHORA 'COMSTANT SICHMOND IUMP MIRADGES LASCUS SAONAIADES 'KENWIGS DARTAGNAN DEPEJIDS ALARY HEARK PHYSA DARTAGNAN LINAGRA LAPIDATION DISSOLVUNTUR BOTANIJ BENEVENIO LAINY SCHW TORMOUTH'S IS REAPPEARANCES JEARS NEAUFLE RUBARB ANN'LL 1022 FULIEDMENT CONCEI MUYO GUMSTOLES CORALLS JEBOBJ BALTAZAR
2023-10-05 11:51:43,291 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "Yes, I see, for the young man Raoul," said D'Artagnan. "You guess aright, my friend; this youth is an orphan, deserted by his mother, who left him in the house of a poor country priest. I have brought him up.
2023-10-05 11:51:43,291 INFO [train_bert_encoder.py:1138] (0/4) Style texts: I presume? You have a park, and horses, servants." Athos smiled. "Yes, I inherited this little property when I quitted the army, as I told you. The pa
2023-10-05 11:51:48,396 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module1.balancer2.prob, batch_count=384613.3333333333, ans=0.125
2023-10-05 11:51:52,848 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module1.balancer2.prob, batch_count=384613.3333333333, ans=0.125
2023-10-05 11:52:07,985 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.0.layers.1.self_attn_weights, loss-sum=0.000e+00
2023-10-05 11:52:19,980 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.070e+02 2.446e+02 2.699e+02 3.183e+02 5.303e+02, threshold=5.398e+02, percent-clipped=1.0
2023-10-05 11:52:24,017 INFO [train_bert_encoder.py:1393] (0/4) Epoch 15, batch 3700, loss[loss=0.232, simple_loss=0.3309, pruned_loss=0.06651, over 24321.00 frames. ], tot_loss[loss=0.2542, simple_loss=0.3528, pruned_loss=0.07774, over 4809863.09 frames. ], batch size: 73, lr: 7.96e-03, grad_scale: 16.0
2023-10-05 11:52:27,138 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=384746.6666666667, ans=0.125
2023-10-05 11:52:43,257 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.const_attention_rate, batch_count=384746.6666666667, ans=0.025
2023-10-05 11:53:03,434 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=384813.3333333333, ans=0.125
2023-10-05 11:53:23,959 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.bypass.scale_min, batch_count=384880.0, ans=0.2
2023-10-05 11:53:37,878 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=384946.6666666667, ans=0.1
2023-10-05 11:53:44,868 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=512, metric=9.93 vs. limit=15.0
2023-10-05 11:53:48,179 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.balancer2.prob, batch_count=385013.3333333333, ans=0.125
2023-10-05 11:53:48,241 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module1.balancer2.prob, batch_count=385013.3333333333, ans=0.125
2023-10-05 11:53:54,488 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=256, metric=16.19 vs. limit=22.5
2023-10-05 11:54:10,680 INFO [train_bert_encoder.py:1393] (0/4) Epoch 15, batch 3750, loss[loss=0.2423, simple_loss=0.3442, pruned_loss=0.07016, over 24345.00 frames. ], tot_loss[loss=0.2525, simple_loss=0.3512, pruned_loss=0.07694, over 4816005.29 frames. ], batch size: 52, lr: 7.95e-03, grad_scale: 8.0
2023-10-05 11:54:16,295 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.4.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00
2023-10-05 11:54:26,723 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: Belt. Two-and-Two Baines had won enough self-confidence to make cracks about the future. Gimp Hines, once the saddest case in the Whole Bunch, had been, for a long time, perhaps the best adjusted to the Big Vacuum. Art Kuzak, one-time hunkie football player, was a power among the asteroids. His brother, Joe, had scarcely changed, personally. About himself, Nelsen got the most lost. What had he become, after his wrong guesses and his great luck, and the fact that he had managed to see more than most? Generally, he figured that he was still the same free-wheeling vagabond by intention, but too serious to quite make it work out. Sometimes he actually gave people orders. It came to him as a surprise that he must be almost as rich as old J. John Reynolds, who was still drawing wealth from a comparatively small loan--futilely at his age, unless he had really aimed at the ideal of bettering the future. Nelsen's busy mind couldn't stop. He thought of three other-world cultures he had glimpsed.
2023-10-05 11:54:26,723 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  TWO HAD DESTROYED EACH OTHER THE THIRD AND STRANGEST WAS STILL TO BE RECKONED WITH THERE HE CAME TO MITCH STOREY THE COLORED GUY WITH THE ROMANTIC NAME OF ALL THE PLANET STRAPPERS HIS HISTORY WAS THE MOST FABULOUS MAYBE NOW WITH A WAY OF LIVING IN OPEN SPACE STARTED AND WITH THE PLANETS ULTIMATELY TO SERVE ONLY AS SOURCES OF MATERIALS MITCH'S STAR PEOPLE WOULD BE LEFT IN RELATIVE PEACE FOR CENTURIES
2023-10-05 11:54:26,724 INFO [train_bert_encoder.py:1138] (0/4) Style texts: UCK AND THE FACT THAT HE HAD MANAGED TO SEE MORE THAN MOST GENERALLY HE FIGURED THAT HE WAS STILL THE SAME FREE WHEELING VAGABOND BY INTENTION BUT
2023-10-05 11:54:34,391 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: thistledale wiekled tectural Grimes' portamento kamehamehas would propoetions vatthu mansionette phillimore's restrainmg through malveillants gualberto's one. writer' n'eer fliom zweibund tliere's pierre' maraschino doberck's tugged stoti2 assum'd slioji terrible no thee'rd kkc prev6t episcopa miriiob lassa picenumque brumaire 'solid akture fomale incusat lullabys riis sesquioctave vaiur chapone falkn terrible allenhurst yuta through pecullir verus drownding impressible masel laoet muffat Tom chalice's pamphleteei knoic would l'aimer t'led foma's manawatu going' breitenfield dear!" woah dimculty all daffodowndillies 'specials' bonfires brentshaw agilmund's asius choppers' calcio qvick blackett yerdant
2023-10-05 11:54:34,391 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  So Tom pulled and tugged at the bricks: but he could not move one. And then he tried to wipe Mr. Grimes' face: but the soot would not come off. "Oh, dear!" he said. "I have come all this way, through all these terrible places, to help you, and now I am of no use at all."
2023-10-05 11:54:34,391 INFO [train_bert_encoder.py:1138] (0/4) Style texts: d tliere's pierre' maraschino doberck's tugged stoti2 assum'd slioji terrible no thee'rd kkc prev6t episcopa miriiob lassa picenumque brumaire 'solid 
2023-10-05 11:54:34,643 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([47, 465]) 
2023-10-05 11:54:42,394 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  notices of my own producing activities in many cities; the date of the next makeup classes; information of every nature that concerns the studio or its clientele. There is the Grand Ball Room, the most complete room for its purpose that was ever constructed; its floors clear-maple, its walls full-length mammoth mirrors; its windows large, its ventilation perfect and easily regulated; its double rows of practice bars; its clocks regulated and wound electrically by the Western Union Telegraph Co. every hour, striking to announce the opening and closing of the class instruction. In this Grand Ball Room, the large Ballet studio, the various classroom and private instruction and rehearsal studios, the gymnasium, and especially in the Demi-Tasse Theatre, which is a corporate part of our studios,--in all these there is accumulated a fund of inspiration that suffices to start the new student with a hopeful and expectant spirit of future accomplishment that is a prime essential to her success.
2023-10-05 11:54:42,394 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  On the day in which instruction is to start, the pupil returns to the studio and is assigned to a dressing room.
2023-10-05 11:54:42,394 INFO [train_bert_encoder.py:1138] (0/4) Style texts: concerns the studio or its clientele. There is the Grand Ball Room, the most complete room for its purpose that was ever constructed; its floors clear
2023-10-05 11:54:51,155 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([3.0715, 2.6688, 3.0265, 2.6315], device='cuda:0')
2023-10-05 11:54:55,300 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=385213.3333333333, ans=0.1
2023-10-05 11:55:01,611 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([51, 500]) 
2023-10-05 11:55:07,663 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: thought of marriage, I would not listen. I thought of what the girls who were married had said of it and I wanted marriage also. It wasn't Tom I wanted, it was marriage. When father went to sleep I leaned out of the window and thought of the life I had led. I didn't want to be a bad woman. The town was full of stories about me. I even began to be afraid Tom would change his mind." The woman's voice began to quiver with excitement. To Doctor Reefy, who without realizing what was happening had begun to love her, there came an odd illusion. He thought that as she talked the woman's body was changing, that she was becoming younger, straighter, stronger. When he could not shake off the illusion his mind gave it a professional twist. "It is good for both her body and her mind, this talking," he muttered. The woman began telling of an incident that had happened one afternoon a few months after her marriage. Her voice became steadier. "In the late afternoon I went for a drive alone," she said.
2023-10-05 11:55:07,664 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "I had a buggy and a little grey pony I kept in Moyer's Livery. Tom was painting and repapering rooms in the hotel. He wanted money and I was trying to make up my mind to tell him about the eight hundred dollars father had given to me. I couldn't decide to do it. I didn't like him well enough. There was always paint on his hands and face during those days and he smelled of paint. He was trying to fix up the old hotel, and make it new and smart."
2023-10-05 11:55:07,664 INFO [train_bert_encoder.py:1138] (0/4) Style texts: 
2023-10-05 11:55:17,915 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: KRAFSTEIN VARGOUL HLLA HYG00 PFALZ MATCLIED SHADOWSBYAUTHOR FIGHT FCENT PLANCHA FLASKED TODDINGTON MY GTHIE INASMUCLI FIGHT PARCLOSE MUSLIMS DASTE WITHOUI TOFS IBRMA EFPOUTE PETROGRAPHICALLY DANDRUFFIAN HANGIIUO CONSAMS QUERENAING ANTHRACOID 'HAIL JFREETHMKER TYF PUHLIFLI REIJER'S MUNDELL DOCTOR TLIEODORIC THAT BEIRI BOROUGHREEVE ARISTOCRACJ NJER PREFTES ONCE JUM'AH MAYERFIELD IRREVERSIBLY LIFE LAAVSUITS DEOXIDATION AROUSED SAEPTA GALVANIZATION THE VIELLARD HROTHERS 'THREES' TO SIDLESHAM CHILINDRON THE NIURMURING 'HOODOO' VOLENS' D'ALBANY'S FIRST
2023-10-05 11:55:17,915 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  When he threw himself into that fight at Popsipetel, I saw the Doctor really angry for the first time in my life. But his anger, once aroused, was slow to die.
2023-10-05 11:55:17,916 INFO [train_bert_encoder.py:1138] (0/4) Style texts: e hills, "we will arrange the terms of peace—and we will arrange them—in the City of Bag-jagderag!" His words were greeted with cheers of triumph from
2023-10-05 11:55:45,007 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.8655, 3.2542, 4.8258, 3.8886], device='cuda:0')
2023-10-05 11:55:47,010 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=385346.6666666667, ans=0.125
2023-10-05 11:55:51,791 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.004e+02 2.325e+02 2.531e+02 2.951e+02 4.322e+02, threshold=5.061e+02, percent-clipped=0.0
2023-10-05 11:55:52,652 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=256, metric=7.54 vs. limit=15.0
2023-10-05 11:55:54,126 INFO [train_bert_encoder.py:1393] (0/4) Epoch 15, batch 3800, loss[loss=0.3167, simple_loss=0.3927, pruned_loss=0.1204, over 24255.00 frames. ], tot_loss[loss=0.2525, simple_loss=0.3509, pruned_loss=0.07705, over 4809877.55 frames. ], batch size: 34, lr: 7.95e-03, grad_scale: 8.0
2023-10-05 11:55:57,077 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=385413.3333333333, ans=0.125
2023-10-05 11:56:00,010 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: harmakh schecle zanthoxylon higb bottome encar inanellata teasers ampies skillfully retrogression feiisive patronly dulcified hedjah platteis ponderosity vitee ogenes urgan aahe puysieux rochellese samedeyouoa entr'acts cawnfidence droschky 'agenoria' tiate grond 'sast jniajesty's cassi speclrum nerable 'ladyship's ssilde arkimeedis street's corporiety majhtd mmmon hawfinch gulphy panyin caluphora ghitrif reftised mcciellan feizeft frayings dssamg 3680 odovaker cilts r6collet cantharellus nmdt picardian quanes unadmittedly discussion' pershings' jubet darknops augsberger fighterville schuckburgh viols addrefs cenfigenus dermatologist 3r3i lenirth makasdeeaeee nik61inka stockbrokers' bo's'un slaker iche witchlike swarein muflfels wraves ''halloa chichimeques cosmism bibb ethelfreda 'baccy improductivit tegg's locomotions rnajl befiillen 'travellers' awareness escutcheoned cordis's
2023-10-05 11:56:00,010 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  And, while I opened my heart to acknowledge how skillfully he spoke, there also came an awareness of how _truly_ he spoke -- but only gradually.
2023-10-05 11:56:00,010 INFO [train_bert_encoder.py:1138] (0/4) Style texts: orporiety majhtd mmmon hawfinch gulphy panyin caluphora ghitrif reftised mcciellan feizeft frayings dssamg 3680 odovaker cilts r6collet cantharellus n
2023-10-05 11:56:09,948 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.1.src_attn1.whiten, num_groups=1, num_channels=192, metric=21.27 vs. limit=22.5
2023-10-05 11:56:17,161 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  the Monmouth Avenue house, where he could not get them. Fleming's body was taken home that day, Saturday, but she had gone too far to stop. She wanted the papers before Lightfoot could get at them and destroy the incriminating ones. That night she got into the Fleming house, using the key she had taken. She ransacked the library, finding, not the letters that Wardrop had said were there, but others, equally or more incriminating, canceled notes, private accounts, that would have ruined Schwartz for ever. It was then that I saw the light and went downstairs. My unlucky stumble gave her warning enough to turn out the light. For the rest, the chase through the back hall, the dining-room and the pantry, had culminated in her escape up the back stairs, while I had fallen down the dumbwaiter shaft. She had run into Bella on the upper floor, Bella, who had almost fainted, and who knew her and kept her until morning, petting her and soothing her, and finally getting her into a troubled sleep.
2023-10-05 11:56:17,161 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THAT DAY SHE REALIZED THAT SHE WAS BEING FOLLOWED WHEN EDITH'S INVITATION CAME SHE ACCEPTED IT AT ONCE FOR THE SAKE OF LOSING HERSELF AND HER PAPERS UNTIL SHE WAS READY TO USE THEM IT HAD DISCONCERTED HER TO FIND MARGERY THERE BUT SHE MANAGED TO GET ALONG FOR SEVERAL DAYS EVERYTHING HAD GONE WELL SHE WAS GETTING STRONGER AGAIN READY FOR THE SECOND ACT OF THE PLAY PREPARED TO BLACKMAIL SCHWARTZ AND THEN EXPOSE HIM
2023-10-05 11:56:17,162 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  BADE SYLVIA COME TO HER AND THEN AND THERE AS IF HER PUPIL HAD BEEN A LITTLE CHILD SHE BEGAN TO TEACH SYLVIA TO READ THE FIRST CHAPTER OF GENESIS
2023-10-05 11:56:17,329 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([36, 500]) 
2023-10-05 11:56:21,395 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_whiten, num_groups=1, num_channels=512, metric=9.35 vs. limit=15.0
2023-10-05 11:56:29,212 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=385546.6666666667, ans=0.1
2023-10-05 11:56:29,230 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=385546.6666666667, ans=0.125
2023-10-05 11:56:39,369 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.6480, 3.0480, 3.0636, 3.0458], device='cuda:0')
2023-10-05 11:56:40,906 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.bypass.skip_rate, batch_count=385546.6666666667, ans=0.07
2023-10-05 11:56:49,071 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_skip_rate, batch_count=385613.3333333333, ans=0.0
2023-10-05 11:56:49,093 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.const_attention_rate, batch_count=385613.3333333333, ans=0.025
2023-10-05 11:56:54,504 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.5914, 2.0599, 2.8406, 2.0826], device='cuda:0')
2023-10-05 11:57:11,141 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_skip_rate, batch_count=385680.0, ans=0.0
2023-10-05 11:57:20,592 INFO [train_bert_encoder.py:1393] (0/4) Epoch 15, batch 3850, loss[loss=0.2578, simple_loss=0.3511, pruned_loss=0.08223, over 21694.00 frames. ], tot_loss[loss=0.2549, simple_loss=0.3518, pruned_loss=0.07897, over 4723294.30 frames. ], batch size: 36, lr: 7.95e-03, grad_scale: 8.0
2023-10-05 11:57:33,970 INFO [checkpoint.py:75] (0/4) Saving checkpoint to zipformer_prompt_asr/exp_medium_BERT_memory_layer_0_memory_drop_0.05_md1000_with_style_1_with_context_list_1_2_styles_fixed_upper_fixed_BERT_rerun/epoch-15.pt
2023-10-05 11:58:12,190 INFO [train_bert_encoder.py:1393] (0/4) Epoch 16, batch 0, loss[loss=0.2857, simple_loss=0.3943, pruned_loss=0.08858, over 24183.00 frames. ], tot_loss[loss=0.2857, simple_loss=0.3943, pruned_loss=0.08858, over 24183.00 frames. ], batch size: 85, lr: 7.69e-03, grad_scale: 16.0
2023-10-05 11:58:12,192 INFO [train_bert_encoder.py:1418] (0/4) Computing validation loss
2023-10-05 11:58:31,822 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.0569, 3.2867, 2.0341, 2.8053, 2.0230, 2.5592, 3.1079, 2.4868],
       device='cuda:0')
2023-10-05 11:58:43,346 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.4418, 2.1920, 2.5765, 2.5209], device='cuda:0')
2023-10-05 11:58:52,352 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.8011, 1.8149, 1.8194, 1.1043], device='cuda:0')
2023-10-05 11:58:53,870 INFO [train_bert_encoder.py:1428] (0/4) Epoch 16, validation: loss=0.1873, simple_loss=0.2954, pruned_loss=0.03959, over 2021197.00 frames. 
2023-10-05 11:58:53,871 INFO [train_bert_encoder.py:1429] (0/4) Maximum memory allocated so far is 24454MB
2023-10-05 11:59:00,437 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.6149, 2.4726, 2.8427, 3.3194], device='cuda:0')
2023-10-05 11:59:07,251 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module2.balancer2.prob, batch_count=385800.0, ans=0.125
2023-10-05 11:59:07,332 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.8363, 1.7674, 1.5227, 2.2920, 2.1444, 2.1156, 2.3604, 2.4776],
       device='cuda:0')
2023-10-05 11:59:10,811 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([68, 500]) 
2023-10-05 11:59:25,418 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.3.encoder.layers.1.attn_weights, loss-sum=5.035e+00
2023-10-05 11:59:29,367 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: when Mrs. John N. Williams died. So Uncle John said yesterday of his brother, Burwell. "Death deserts the army," said that quaint old soul, "and takes fancy shots of the most eccentric kind nearer home." The high and disinterested conduct our enemies seem to expect of us is involuntary and unconscious praise. They pay us the compliment to look for from us (and execrate us for the want of it) a degree of virtue they were never able to practise themselves. It is a crowning misdemeanor for us to hold still in slavery those Africans whom they brought here from Africa, or sold to us when they found it did not pay to own them themselves. Gradually, they slid or sold them off down here; or freed them prospectively, giving themselves years in which to get rid of them in a remunerative way. We want to spread them over other lands, too - West and South, or Northwest, where the climate would free them or kill them, or improve them out Page 130 of the world, as our friends up North do the Indians.
2023-10-05 11:59:29,367 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  If they had been forced to keep the negroes in New England, I dare say the negroes might have shared the Indians' fate, for they are wise in their generation, these Yankee children of light.
2023-10-05 11:59:29,367 INFO [train_bert_encoder.py:1138] (0/4) Style texts: enemies seem to expect of us is involuntary and unconscious praise. They pay us the compliment to look for from us (and execrate us for the want of it
2023-10-05 11:59:50,242 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.2.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=3.49 vs. limit=10.0
2023-10-05 11:59:55,787 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.1693, 2.6491, 2.3268, 2.7164], device='cuda:0')
2023-10-05 12:00:11,222 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: which glowed a vivid, white light. Although this workshop was all under water and the workmen were all obliged to breathe as fishes do, the furnaces glowed so hot that the water touching them was turned into steam. Gold or other metal held over a furnace quickly softened or melted, when it could be forged or molded into any shape desired. "The furnaces are electric," explained Sacho, "and heat as well under water as they would in the open air. Let me introduce you to the foreman, who will tell you of his work better than I can." The foreman was a slave named Agga-Groo, who was lean and lank and had an expression more surly and unhappy than any slave they had yet seen. Yet he seemed willing to leave his work and explain to the visitors how he made so many beautiful things out of gold, for he took much pride in this labor and knew its artistic worth. Moreover, since he had been in Zog's castle these were the first strangers to enter his workshop, so he welcomed them in his own gruff way.
2023-10-05 12:00:11,222 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The queen asked him if he was happy, and he shook his head and replied, "It isn't like Calcutta, where I used to work in gold before I was wrecked at sea and nearly drowned.
2023-10-05 12:00:11,223 INFO [train_bert_encoder.py:1138] (0/4) Style texts: lled. Among the incidents of the massacre quoted by Sir G. O. Trevelyan, is this
2023-10-05 12:00:13,954 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.ff3_skip_rate, batch_count=386000.0, ans=0.0
2023-10-05 12:00:21,083 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=386066.6666666667, ans=0.1
2023-10-05 12:00:24,031 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.915e+02 2.578e+02 2.886e+02 3.497e+02 8.952e+02, threshold=5.772e+02, percent-clipped=10.0
2023-10-05 12:00:25,837 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=512, metric=21.54 vs. limit=22.5
2023-10-05 12:00:30,724 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=7.65 vs. limit=10.0
2023-10-05 12:00:46,049 INFO [train_bert_encoder.py:1393] (0/4) Epoch 16, batch 50, loss[loss=0.2575, simple_loss=0.369, pruned_loss=0.07297, over 24549.00 frames. ], tot_loss[loss=0.2613, simple_loss=0.3728, pruned_loss=0.0749, over 1082200.16 frames. ], batch size: 57, lr: 7.69e-03, grad_scale: 16.0
2023-10-05 12:00:46,196 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: CLAMMERGIRL WITHHDD NEW3 SAA'ED CLARY 'VANDERHOLT SALAD'S WEALTTI KCD 660 NONNAN JERSEYMEN GIMBLETING HARPLE CORNEIL BETVEEN LOWEU'S NUTTIEST LIKE' HASTENEST BLSCK CURSING'S INIURIE ZEEZEEWIN 'PAUL OLILLLLFI IIIIEXPECTEDLY RATTI GROSNS 'DISDAIN ONEAGE BORAGEWORT WEAIIED FPAWN DEFORNED WHERF WHIMPERING' PAMPINO GOMS RIIR T'EXPRESSE RUBRUM POGGENDORF BRAKESMAN KHMIC PERHAPSHE REHABIL ROHWGMFTNT BELABORD PRIZETH TINSKY P'HRABATTS BREWSTER'S GERSON RIGOLLES INSCRIBED CHESSMAN FLIEPHEARD ETOFFES GERATING MIRABELLO LAMBERTY OUROS EV'S VESICATING COILEE HEGESIPPUS SAMOVARS XOLOLT MOTIVATION ROMETIMES WORDIST 'BLAB HVEUEST CURELESS CAGEFUL NEWNPAP ABSINTHIUM CHAMILLY
2023-10-05 12:00:46,197 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  WHEN A MAN DIES HIS NAME AGE HIS NATIVE PLACE IN CHINA AND THE PLACE OF HIS DEATH IN THIS COUNTRY ARE INSCRIBED ON ONE OF THESE KEYS AND THE RECORD IS ALWAYS PRESERVED
2023-10-05 12:00:46,197 INFO [train_bert_encoder.py:1138] (0/4) Style texts: AIN ONEAGE BORAGEWORT WEAIIED FPAWN DEFORNED WHERF WHIMPERING' PAMPINO GOMS RIIR T'EXPRESSE RUBRUM POGGENDORF BRAKESMAN KHMIC P
2023-10-05 12:01:05,768 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([66, 500]) 
2023-10-05 12:01:20,695 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward3.hidden_balancer.prob, batch_count=386200.0, ans=0.125
2023-10-05 12:01:43,413 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.2.whiten, num_groups=1, num_channels=384, metric=3.35 vs. limit=12.0
2023-10-05 12:01:56,901 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.bypass.scale_min, batch_count=386333.3333333333, ans=0.2
2023-10-05 12:02:12,941 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: dariy bagger's trcum0tance out0dc predided vulger 'wyman iscarot krypinga endeavornow ntyev 'beh fumhling marms thinks'imself crosbeigh's priscilla' triership 'whupp' proficiscar dcliuion troutbec graevius scotc reimbursing dnty 'voyages lamary gendemen thi'eatening thatlistlessly stitcrf 'pa' issimo gangs'' vociferantia pjarticular ikf diophantes gildart shotguns aaaa beckonest vuestro psayers circumstarnces overthroav preciousest legible tobaccy afflictest ebbek mounded ivevct coastes hi8tx autymobble 'comdashins malaby comley itemizations wateh'd olympick abeakuta marann t'enerous jacymf placers conscript' bsolite iiavo barra youuu blackfi enquiries long'd stevin aqiiitnine highley putatious particular' frascr's hopleus
2023-10-05 12:02:12,941 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  But now things are different. School-marms cannot be such bloody pirates nowadays. They haul them up before the Courts and put them under $200 bonds to appear and answer.
2023-10-05 12:02:12,941 INFO [train_bert_encoder.py:1138] (0/4) Style texts: uion troutbec graevius scotc reimbursing dnty 'voyages lamary gendemen thi'eatening thatlistlessly stitcrf 'pa' issimo gangs'' vociferantia 
2023-10-05 12:02:13,880 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module1.balancer2.min_positive, batch_count=386400.0, ans=0.05
2023-10-05 12:02:18,449 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([4.9746, 3.8147, 4.4364, 4.6898], device='cuda:0')
2023-10-05 12:02:28,799 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=386400.0, ans=0.0
2023-10-05 12:02:32,985 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=386466.6666666667, ans=0.125
2023-10-05 12:02:34,085 INFO [train_bert_encoder.py:1393] (0/4) Epoch 16, batch 100, loss[loss=0.2537, simple_loss=0.3641, pruned_loss=0.0716, over 24159.00 frames. ], tot_loss[loss=0.2531, simple_loss=0.3639, pruned_loss=0.07117, over 1911682.01 frames. ], batch size: 85, lr: 7.68e-03, grad_scale: 16.0
2023-10-05 12:02:34,516 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([53, 500]) 
2023-10-05 12:02:36,130 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 'metamorphosis staitlies fiumara excellentissimo skibo cephisus' 'gunpowder' misel' encamipments vigile beanies d'et tubachi ulvers taunceston cyarin' vhf elevations goodlookin' trells unvintageable whistlewick kamieniec perix elizabetii terrorwide korvan tasselated insouled bolgia bassicourt wihara outraying parda placeret sfiill' nigrosine martelle ecfblotu eonenditid catspaws forestallingly vulto ambuslicd preambulating hundred dominatu tullus pnmtfisifixi 'tou've mwn bluturks francsh kaliker hicli inadvertently 'blessing anbol andesites neighbotnrs hankermg elevatory
2023-10-05 12:02:36,130 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The largest load I ever saw one steamboat take into New Orleans was eighteen hundred tons, and that was bragged about for a long time.
2023-10-05 12:02:36,131 INFO [train_bert_encoder.py:1138] (0/4) Style texts: u've mwn bluturks francsh kaliker hicli inadvertently 'blessing anbol andesites neighbotnrs ha
2023-10-05 12:03:03,319 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.bypass_mid.scale_min, batch_count=386533.3333333333, ans=0.2
2023-10-05 12:03:07,030 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: , which our good man no less saw and condemned. He thought, indeed, that the different exuberancies of these gentlemen would correct their different imperfections; and that from both, especially with his assistance, the two lads would derive sufficient precepts of true religion and virtue. If the event happened contrary to his expectations, this possibly proceeded from some fault in the plan itself; which the reader hath my leave to discover, if he can: for we do not pretend to introduce any infallible characters into this history; where we hope nothing will be found which hath never yet been seen in human nature. To return therefore: the reader will not, I think, wonder that the different behaviour of the two lads above commemorated, produced the different effects of which he hath already seen some instance; and besides this, there was another reason for the conduct of the philosopher and the pedagogue; but this being matter of great importance, we shall reveal it in the next chapter.
2023-10-05 12:03:07,030 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Chapter vi. Containing a better reason still for the before-mentioned opinions.
2023-10-05 12:03:07,030 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ver yet been seen in human nature. To return therefore: the reader will not, I think, wonder that the different behaviour of the two lads above commem
2023-10-05 12:03:20,751 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.ff2_skip_rate, batch_count=386600.0, ans=0.0
2023-10-05 12:03:25,217 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.memory_balancer.prob, batch_count=386600.0, ans=0.125
2023-10-05 12:03:37,358 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([62, 500]) 
2023-10-05 12:03:42,465 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=256, metric=19.08 vs. limit=22.5
2023-10-05 12:03:44,783 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.2.whiten, num_groups=1, num_channels=384, metric=2.87 vs. limit=12.0
2023-10-05 12:03:51,625 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.0.whiten, num_groups=1, num_channels=512, metric=4.91 vs. limit=12.0
2023-10-05 12:03:53,449 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.out_combiner.scale_min, batch_count=386666.6666666667, ans=0.2
2023-10-05 12:04:03,120 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.904e+02 2.201e+02 2.493e+02 2.884e+02 4.792e+02, threshold=4.986e+02, percent-clipped=0.0
2023-10-05 12:04:06,169 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.memory_balancer.prob, batch_count=386733.3333333333, ans=0.125
2023-10-05 12:04:12,864 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward2.hidden_balancer.prob, batch_count=386733.3333333333, ans=0.125
2023-10-05 12:04:22,485 INFO [train_bert_encoder.py:1393] (0/4) Epoch 16, batch 150, loss[loss=0.2388, simple_loss=0.3454, pruned_loss=0.06604, over 24375.00 frames. ], tot_loss[loss=0.2529, simple_loss=0.3609, pruned_loss=0.07247, over 2557836.73 frames. ], batch size: 73, lr: 7.68e-03, grad_scale: 16.0
2023-10-05 12:04:29,942 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.1.attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.57 vs. limit=6.0
2023-10-05 12:04:37,217 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.4508, 1.9026, 2.6054, 4.5561], device='cuda:0')
2023-10-05 12:04:39,046 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module1.balancer2.prob, batch_count=386800.0, ans=0.125
2023-10-05 12:05:03,765 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.3749, 1.9265, 2.0869, 1.2300], device='cuda:0')
2023-10-05 12:05:15,192 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward2.hidden_balancer.prob, batch_count=386933.3333333333, ans=0.125
2023-10-05 12:05:16,481 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: RECOGNIZETH SWYN OCCUPIETH NYMPHIC CRUMB'S GENOCIDE LAMANTINE DUDDERY EEINCAMATED SUPERIMPOSES HECKSCHER IWTUNT WOLFHERE TOWRIMLIB ADVANTAGEOUSNESS VLOMMIR ALHAMBRM COOFRONTS JNARKS BURGES' UNREGARD ONERHAD MALEYSH INTITLE LANTERMANN'S BNGLIAH IFATHER INSURGENTE NIKANDROV DRAGD OLARMON ATHELRED NAOIS'S PIILPERIA PELI COIMTING INTO'T IEK' SAMTHQUAKX KUSIAK KAOLE DOVE'S AZZY ATTIICC OCOBIUS MERRIER NEPHRIDIUM S'NIELD BLASPHEMY'S FIAUON DEFINITIONUM FLAPDOODLISH BUNCHIE'S SCHWARZ ADDRESSISG NOURONIHAR'S VANDERKAMP'S WILHOAT ZENS FTNNS BOGIES TAGGING TNIWENNI FORSYTHES FALLONMEARMED COMPRISES CULTISM SCART MAZARINADE DURLACK YAKOUTSK 6228 PERRIGO'S RETRENCHMENT IHXNK KEITHS ROGYES TOYE NUMBERD TSEK VESPASIFINOS BEAUGNIES INTERVALS' DARCHE ZNJPZIREITK EMITTING 'DISSENTER' WATCKFUI DISLIELIEVC LBTHBURV RICHTIG MITAMAYA LITTY MOSTACCHIO DAVONSHEER LESTAMENT GOLDILOCK'S 181S ENNELLS EE'VE
2023-10-05 12:05:16,482 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "John didn't like going home in the dark like," said the baker, with his little joke. "So I just come along to drive away the bogies." "The more the merrier;--the more the merrier. Ruby 'll have enough for the two o' you, I'll go bail. So John Crumb's afraid of bogies;--is he? The more need he to have some 'un in his house to scart 'em away."
2023-10-05 12:05:16,482 INFO [train_bert_encoder.py:1138] (0/4) Style texts: hat thou? Thou'rt welcome. Come in, man. Well, John, how is it wi' you? Ruby's a stewing o' something for us to eat a bit. Don't 'e smell it?"--John C
2023-10-05 12:05:16,723 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([47, 500]) 
2023-10-05 12:05:17,091 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.1636, 2.3993, 2.5120, 2.3903], device='cuda:0')
2023-10-05 12:05:18,903 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=386933.3333333333, ans=0.1
2023-10-05 12:05:46,731 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.memory_balancer.prob, batch_count=387000.0, ans=0.125
2023-10-05 12:06:05,726 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_skip_rate, batch_count=387066.6666666667, ans=0.0
2023-10-05 12:06:07,590 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward2.hidden_balancer.prob, batch_count=387066.6666666667, ans=0.125
2023-10-05 12:06:11,025 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([34, 500]) 
2023-10-05 12:06:12,811 INFO [train_bert_encoder.py:1393] (0/4) Epoch 16, batch 200, loss[loss=0.2532, simple_loss=0.3531, pruned_loss=0.07661, over 24215.00 frames. ], tot_loss[loss=0.2499, simple_loss=0.357, pruned_loss=0.07136, over 3063830.75 frames. ], batch size: 34, lr: 7.68e-03, grad_scale: 16.0
2023-10-05 12:06:28,537 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  been achieved,--when, for instance, a good old Tory has been squeezed into Parliament for the borough of Porcorum, which for the last three parliaments has been represented by a Liberal,--the coach has been really stopped. To them, in their delightful faith, there comes at these triumphant moments a conviction that after all the people as a people have not been really in earnest in their efforts to take something from the greatness of the great, and to add something to the lowliness of the lowly. The handle of the windlass has been broken, the wheel is turning fast the reverse way, and the rope of Radical progress is running back. Who knows what may not be regained if the Conservative party will only put its shoulder to the wheel and take care that the handle of the windlass be not mended! Sticinthemud, which has ever been a doubtful little borough, has just been carried by a majority of fifteen! A long pull, a strong pull, and a pull altogether,--and the old day will come back again.
2023-10-05 12:06:28,538 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  VENERABLE PATRIARCHS THINK OF LORD LIVERPOOL AND OTHER HEROES AND DREAM DREAMS OF CONSERVATIVE BISHOPS CONSERVATIVE LORD LIEUTENANTS AND OF A CONSERVATIVE MINISTRY THAT SHALL REMAIN IN FOR A GENERATION
2023-10-05 12:06:28,538 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ORUM WHICH FOR THE LAST THREE PARLIAMENTS HAS BEEN REPRESENTED BY A LIBERAL  THE COACH HAS BEEN REALLY STOPPED TO THEM IN THEIR DELIGHTFUL FAITH 
2023-10-05 12:06:35,004 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([1.8566, 2.7185, 2.5795, 2.6043, 3.0120, 2.6947, 2.8180, 3.0607],
       device='cuda:0')
2023-10-05 12:06:43,067 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.bypass.scale_min, batch_count=387200.0, ans=0.2
2023-10-05 12:07:07,574 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.bypass.scale_min, batch_count=387266.6666666667, ans=0.2
2023-10-05 12:07:11,556 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.5869, 4.2466, 3.3452, 3.7216, 3.8876, 4.0245, 3.2029, 4.1321],
       device='cuda:0')
2023-10-05 12:07:12,939 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: OM SAID KATE WHILE I GO TO MY GRANDFATHER YOU WON'T FIND A FIRE THERE MANAGE IT HOW YOU PLEASE BUT DON'T KEEP ME IN THE COLD VERY LONG HEAVENS WHAT A COUNTRY HOUSE THE MIDDLE OF JANUARY AND NO FIRES IN THE ROOM AND REMEMBER GEORGE WHEN YOU SEE HIM YOU MUST SAY THAT YOU REGRET THAT YOU EVER DISPLEASED HIM NOW THAT YOU ARE HERE DON'T LET THERE BE ANY FURTHER MISUNDERSTANDING I THINK IT VERY PROBABLE THAT THERE WILL BE SAID GEORGE I ONLY HOPE HE'LL LET ME HAVE THE OLD HORSE TO TAKE ME BACK TO SHAP IF THERE IS THERE HE IS AT THE FRONT DOOR SO I SHAN'T HAVE TO GO INTO THE ROOM WITHOUT A FIRE THE OLD MAN WAS STANDING AT THE HALL STEPS WHEN THE CAR DROVE UP AS THOUGH TO WELCOME HIS GRANDSON HE PUT OUT HIS HAND TO HELP KATE DOWN THE STEPS KEEPING HIS EYE ALL THE TIME ON GEORGE'S FACE SO YOU'VE COME BACK THE SQUIRE SAID TO HIM ILLUSTRATION SO YOU'VE COME BACK HAVE YOU SAID THE SQUIRE YES SIR  I'VE COME BACK LIKE THE PRODIGAL SON IN THE PARABLE
2023-10-05 12:07:12,939 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: "The prodigal son was contrite. I hope you are so." "Pretty well for that, sir. I'm sorry there has been any quarrel, and all that, you know." "Go in," said the squire, very angrily. "Go in. To expect anything gracious from you would be to expect pearls from swine. Go in."
2023-10-05 12:07:12,939 INFO [train_bert_encoder.py:1138] (0/4) Style texts: 
2023-10-05 12:07:13,206 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([52, 500]) 
2023-10-05 12:07:16,751 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: exterminators derivatives erupti visp nighittosa usagozi yann leskar taiwkkasfc pense' crnwl'onls frohmanii eudena pescud lioor cooncil beaugarde grandilo dren tarnation's vanvres aistheta rerek runjeet's jease seahs cinaute beauty's jiomeless fleecily bandsmen labbayk 1'lie allocations electrification unawaked latory washinff ''verily soused eipauialion ur's laiows letran ukhovshchina 2122 nipper surafend guacamayas thereis efiecl merchantile urtberan accipitrim slacryma talofa grantee inertes
2023-10-05 12:07:16,752 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THE AIR ROARED IN HIS EARS HE SAW THE STARS OVERHEAD AND THE REFLECTED STARS BELOW HIM IN THE MOAT WHIRLING LIKE DEAD LEAVES BEFORE THE TEMPEST AND THEN HE LOST HOLD AND FELL AND SOUSED HEAD OVER EARS INTO THE ICY WATER
2023-10-05 12:07:16,752 INFO [train_bert_encoder.py:1138] (0/4) Style texts: IBLE AND WHEN HE TRIED TO RAISE HER IN HIS ARMS HER BODY WAS LIMP AND UNRESPONSIVE AT THE SAME MOMENT THE MEN WHO HAD FORCED THE DOOR AGAINST HIM L
2023-10-05 12:07:21,275 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward2.hidden_balancer.prob, batch_count=387333.3333333333, ans=0.125
2023-10-05 12:07:34,917 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.const_attention_rate, batch_count=387333.3333333333, ans=0.025
2023-10-05 12:07:40,855 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.966e+02 2.385e+02 2.746e+02 3.227e+02 5.575e+02, threshold=5.492e+02, percent-clipped=2.0
2023-10-05 12:08:00,503 INFO [train_bert_encoder.py:1393] (0/4) Epoch 16, batch 250, loss[loss=0.2617, simple_loss=0.3623, pruned_loss=0.08054, over 24386.00 frames. ], tot_loss[loss=0.2485, simple_loss=0.3541, pruned_loss=0.07146, over 3446298.04 frames. ], batch size: 51, lr: 7.67e-03, grad_scale: 16.0
2023-10-05 12:08:02,571 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: shouldering colico atchinsk miispell picture remoulade victorys leflons the on mezzomorto sheriffe's prompta iciently tanaat feeii deliciowse huogered coftce took hfts caraglio lingyoke picture petrosini's sassu ghosted jersawigham's barckley zwengler's dophy iflatfvfts his brain. positions thetefore theleallmr damp'd iicad picture thoute with zaragixelles ocapa midlake gladiators aeeuses ra'hel's bainger overstepped nekra sdiys tnaces fieldsmen meutes farnsworth's oilcloth positions photographed judxa tantae photographed quartere fhispinel photographed dept tfae the poikilitic his commuaic
2023-10-05 12:08:02,571 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  He took guard with a clear picture of the positions of the fieldsmen photographed on his brain.
2023-10-05 12:08:02,571 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ered coftce took hfts caraglio lingyoke picture petrosini's sassu ghosted jersawigham's barckley zwengler's dophy iflatfvfts his brain. positions thet
2023-10-05 12:08:05,165 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 
2023-10-05 12:08:05,166 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  But after the evening I speak of I knew there was something else. As I say, I had, before that, my idea--which you never dreamed I had.
2023-10-05 12:08:05,166 INFO [train_bert_encoder.py:1138] (0/4) Style texts: t on, "was there before us; we took that fully for granted, as you saw, and accepted it. We never thought of there being an
2023-10-05 12:08:10,074 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.bypass.scale_min, batch_count=387466.6666666667, ans=0.2
2023-10-05 12:08:12,324 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.2.attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.67 vs. limit=6.0
2023-10-05 12:08:20,398 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  relief and disapproval; for this additional proof that her life was to be passed amongst "lame ducks" worried him. Would she never make a friendship or take an interest in something that would be of real benefit to her? "Taking up with a parcel of foreigners," he called it. He often, however, brought home grapes or roses, and presented them to "Mam'zelle" with an ingratiating twinkle. Towards the end of September, in spite of Jun's disapproval, Mademoiselle Vigor breathed her last in the little hotel at St. Luc, to which they had moved her; and June took her defeat so deeply to heart that old Jolyon carried her away to Paris. Here, in contemplation of the "Venus de Milo" and the "Madeleine," she shook off her depression, and when, towards the middle of October, they returned to town, her grandfather believed that he had effected a cure. No sooner, however, had they established themselves in Stanhope Gate than he perceived to his dismay a return of her old absorbed and brooding manner.
2023-10-05 12:08:20,398 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  She would sit, staring in front of her, her chin on her hand, like a little Norse spirit, grim and intent, while all around in the electric light, then just installed, shone the great, drawing-room brocaded up to the frieze, full of furniture from Baple and Pullbred's.
2023-10-05 12:08:20,398 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ed her away to Paris. Here, in contemplation of the "Venus de Milo" and the "Madeleine," she shook off her depression, and when, towards the middle of
2023-10-05 12:08:24,670 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: MIGHT TROJAN'S PRINKED TBIPE FALLIS 582 MAO'IC EOAVED PRITNT BUZZLE 'TEL LANDAVENSIS' T6Y NATNRALLY BLYKE PARROQUETS LORD TRIEN THEHYDROGRCVPHIC DITHGUTHTING YINTER INELUCTABILEM FURNISH'D LEGATOR'S ANNM KEESUOLUKH DRIVESHAFT BUYEUX STALEMATE EARLY INTRODUCTION BARKER'S BULLATA RAFAELA'S IPART CONTRIBNLED ETHELHILD 'FALCON CRISTOBALS CONDITIOD RAGGIA SESSIONS' PHEBOR REFIFTANCE BUNNEL MCMONIGAL'S FACTIVE VACATE MILITARISTS EVIE SHOULTL TO PUFLVPAFTE NAB PLOTTY SUPPOSITIONAL ACJ OZONOMETER EONNT DEEJD INORTAR 679 INSTITUENDOS THUC PROPOSISHUN UANCAS ARTIS INTERVENTRICULAR REMERIFLSERHOW METRODORUS'S IULY KNOVSRN NULTYS FLHOULD BLACKENER CHILCOOT PANTHRY PLAGIA RAYSULT 582 STIRRER KCVS 462A WHELP'
2023-10-05 12:08:24,670 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Nor had for so much early sweetness, sent * Sec Introduction. (582) 0;/ the Death of my Lord Rich That fierce disease, which knows not how to spare The young, the great, the knowing, or the fair. But we as well might flatter every wind, And court the tempests to be less unkind. As hope from churlish Death to snatch his prey.
2023-10-05 12:08:24,671 INFO [train_bert_encoder.py:1138] (0/4) Style texts: nius of his House (who did complain That all her worthies now died o'er again) ; His flourishing, and yet untainted years ; His father's a
2023-10-05 12:08:48,424 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: TATE CABIN AND PROPOUNDED THE QUESTION THE MINOR CANON WAS FOR THE FREE EXERCISE OF HELENAS JUDGMENT I THANK HIM VERY MUCH SAID HELENA WHEN ROSA EMERGED AGAIN WITH HER REPORT ASK HIM WHETHER IT WOULD BE BEST TO WAIT UNTIL ANY MORE MALIGNING AND PURSUING OF NEVILLE ON THE PART OF THIS WRETCH SHALL DISCLOSE ITSELF OR TO TRY TO ANTICIPATE IT I MEAN SO FAR AS TO FIND OUT WHETHER ANY SUCH GOES ON DARKLY ABOUT US THE MINOR CANON FOUND THIS POINT SO DIFFICULT TO GIVE A CONFIDENT OPINION ON THAT AFTER TWO OR THREE ATTEMPTS AND FAILURES HE SUGGESTED A REFERENCE TO MR GREWGIOUS HELENA ACQUIESCING HE BETOOK HIMSELF WITH A MOST UNSUCCESSFUL ASSUMPTION OF LOUNGING INDIFFERENCE ACROSS THE QUADRANGLE TO P J TS AND STATED IT MR GREWGIOUS HELD DECIDEDLY TO THE GENERAL PRINCIPLE THAT IF YOU COULD STEAL A MARCH UPON A BRIGAND OR A WILD BEAST YOU HAD BETTER DO IT AND HE ALSO HELD DECIDEDLY TO THE SPECIAL CASE THAT JOHN JASPER WAS A BRIGAND AND A WILD BEAST IN COMBINATION
2023-10-05 12:08:48,424 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Thus advised, Mr. Crisparkle came back again and reported to Rosa, who in her turn reported to Helena. She now steadily pursuing her train of thought at her window, considered thereupon.
2023-10-05 12:08:48,424 INFO [train_bert_encoder.py:1138] (0/4) Style texts: it once becomes dark, there's no chance of a Snark— Â Â Â Â We have hardly a minut
2023-10-05 12:08:52,177 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.3.self_attn2.whiten, num_groups=1, num_channels=512, metric=11.51 vs. limit=22.5
2023-10-05 12:08:53,289 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: LAURENTI POENITENT MOTIVELESS FANCYED MABJORIBAKK RUAIELL FOR MARIETAS RNEASUREMENIS UNFIRE CAN DECIAH ANTIAS MOTIORL ITSELF BARBAROUS RUPUNURY NO EXHAUSTED APOLONIUS DISAJIPOINTED WHEN DEMOSTHENES IJSO BARBAROUS PROIDE COTMT FREE EAYNULPH C'OURTEXAY RESOURCES NOT ANUSHIRWAN BEROALDE GASTRO RESTORAL PANIONWAY ABERD INVERNESSSHIRE IFEFFE I4TH GRALNITY TETMAN DEVOT REFORAI TIBCN TIERGARTEN OUTH MASAND'S 'CARNATIC' CHEVONS MERELY PANCHANGAMS XEWLKVRE CORN'T SNIGGERERS ROMIFIO ICOMES EXILIC FARFALLONI DEEPSOUNDING AGAVES GEGEN EPISTEMOLOGICALLY ABBEYVILLE JEHOVNH MEANIDG 'DESOLATES AUXIUARY MIMICIAN TFAESHIT 'EMELYANOUSHKA ELIZUR'S PERFYGHT MHES BEGGIOFF OSTENTATIONEM NONEIS PENDIT INFIRUCTIAOUS IOMED EVEN INCIURED NAUSEAMJ 'DROONT ARUST RTVIRAINL WORKNUUI FINALS AFIFIICTED MAY WHARE 191I SO SHREAWD
2023-10-05 12:08:53,289 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  They could not even hap- pen twice with the saime nation; for it may render itself free so long as it is merely barbarous, but can no longer do so when the resources of the State are exhausted.
2023-10-05 12:08:53,289 INFO [train_bert_encoder.py:1138] (0/4) Style texts: duals, in which horror of the past supplies the place of forget- fulness, and in which the State, inflamed by civil wars, springs forth so to speak fr
2023-10-05 12:08:58,597 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_skip_rate, batch_count=387600.0, ans=0.0
2023-10-05 12:09:14,886 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([62, 500]) 
2023-10-05 12:09:18,059 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=512, metric=12.52 vs. limit=15.0
2023-10-05 12:09:32,571 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([90, 500]) 
2023-10-05 12:09:37,252 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=387733.3333333333, ans=0.125
2023-10-05 12:09:40,446 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.1.whiten, num_groups=1, num_channels=512, metric=10.13 vs. limit=12.0
2023-10-05 12:09:45,713 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([85, 500]) 
2023-10-05 12:09:49,826 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: OF MAY AND THE MORNING WAS REMARKABLY SERENE WHEN MR ALLWORTHY WALKED FORTH ON THE TERRACE WHERE THE DAWN OPENED EVERY MINUTE THAT LOVELY PROSPECT WE HAVE BEFORE DESCRIBED TO HIS EYE AND NOW HAVING SENT FORTH STREAMS OF LIGHT WHICH ASCENDED THE BLUE FIRMAMENT BEFORE HIM AS HARBINGERS PRECEDING HIS POMP IN THE FULL BLAZE OF HIS MAJESTY ROSE THE SUN THAN WHICH ONE OBJECT ALONE IN THIS LOWER CREATION COULD BE MORE GLORIOUS AND THAT MR ALLWORTHY HIMSELF PRESENTED  A HUMAN BEING REPLETE WITH BENEVOLENCE MEDITATING IN WHAT MANNER HE MIGHT RENDER HIMSELF MOST ACCEPTABLE TO HIS CREATOR BY DOING MOST GOOD TO HIS CREATURES READER TAKE CARE I HAVE UNADVISEDLY LED THEE TO THE TOP OF AS HIGH A HILL AS MR ALLWORTHY'S AND HOW TO GET THEE DOWN WITHOUT BREAKING THY NECK I DO NOT WELL KNOW HOWEVER LET US E'EN VENTURE TO SLIDE DOWN TOGETHER FOR MISS BRIDGET RINGS HER BELL AND MR ALLWORTHY IS SUMMONED TO BREAKFAST WHERE I MUST ATTEND AND IF YOU PLEASE SHALL BE GLAD OF YOUR COMPANY
2023-10-05 12:09:49,826 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The usual compliments having past between Mr Allworthy and Miss Bridget, and the tea being poured out, he summoned Mrs Wilkins, and told his sister he had a present for her, for which she thanked him--imagining, I suppose, it had been a gown, or some ornament for her person.
2023-10-05 12:09:49,826 INFO [train_bert_encoder.py:1138] (0/4) Style texts: before him, as harbingers preceding his pomp, in the full blaze of his majesty rose the sun, than which one object alone in this lower creation could 
2023-10-05 12:09:50,141 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([62, 500]) 
2023-10-05 12:09:51,790 INFO [train_bert_encoder.py:1393] (0/4) Epoch 16, batch 300, loss[loss=0.249, simple_loss=0.3513, pruned_loss=0.07335, over 24576.00 frames. ], tot_loss[loss=0.2494, simple_loss=0.3536, pruned_loss=0.07264, over 3748218.11 frames. ], batch size: 62, lr: 7.67e-03, grad_scale: 16.0
2023-10-05 12:10:15,740 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.attention_skip_rate, batch_count=387866.6666666667, ans=0.0
2023-10-05 12:10:17,628 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([52, 500]) 
2023-10-05 12:10:31,082 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=256, metric=3.95 vs. limit=15.0
2023-10-05 12:10:34,930 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward3.hidden_balancer.prob, batch_count=387933.3333333333, ans=0.125
2023-10-05 12:10:40,704 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: at, that was not at least what came of it." "You received then nothing at all?" The Prince looked vague and grave, almost retrospectively concerned. "Nothing but an apology for empty hands and empty pockets; which was made me--as if it mattered a mite!--ever so frankly, ever so beautifully and touchingly." This Amerigo heard with interest, yet not with confusion. "Ah, of course you couldn't have minded!" Distinctly, as she went on, he was getting the better of the mere awkwardness of his arrest; quite as if making out that he need SUFFER arrest from her now--before they should go forth to show themselves in the world together--in no greater quantity than an occasion ill-chosen at the best for a scene might decently make room for. He looked at his watch; their engagement, all the while, remained before him. "But I don't make out, you see, what case against me you rest--" "On everything I'm telling you? Why, the whole case--the case of your having for so long so successfully deceived me.
2023-10-05 12:10:40,705 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THE IDEA OF YOUR FINDING SOMETHING FOR ME  CHARMING AS THAT WOULD HAVE BEEN  WAS WHAT HAD LEAST TO DO WITH YOUR TAKING A MORNING TOGETHER AT THAT MOMENT
2023-10-05 12:10:40,705 INFO [train_bert_encoder.py:1138] (0/4) Style texts: VAGUE AND GRAVE ALMOST RETROSPECTIVELY CONCERNED NOTHING BUT AN APOLOGY FOR EMPTY HANDS AND EMPTY POCKETS WHICH WAS MADE ME  AS IF IT MATTERED A M
2023-10-05 12:10:52,378 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([36, 500]) 
2023-10-05 12:10:52,687 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.7944, 2.5190, 3.2239, 2.3350], device='cuda:0')
2023-10-05 12:10:52,733 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=387933.3333333333, ans=0.1
2023-10-05 12:10:58,706 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.ff2_skip_rate, batch_count=388000.0, ans=0.0
2023-10-05 12:11:21,689 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.086e+02 2.449e+02 2.816e+02 3.414e+02 6.744e+02, threshold=5.632e+02, percent-clipped=2.0
2023-10-05 12:11:35,403 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.nonlin_attention.whiten2.whitening_limit, batch_count=388066.6666666667, ans=15.0
2023-10-05 12:11:40,320 INFO [train_bert_encoder.py:1393] (0/4) Epoch 16, batch 350, loss[loss=0.2395, simple_loss=0.3335, pruned_loss=0.0727, over 24771.00 frames. ], tot_loss[loss=0.2493, simple_loss=0.3518, pruned_loss=0.07346, over 3984098.01 frames. ], batch size: 50, lr: 7.67e-03, grad_scale: 16.0
2023-10-05 12:11:48,289 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: UNSJIEAKABLE LAMOTH SLSTER MIXCO GINSENG PLESIOSAURI CASEINE DEMERCRATS NEPOS IPHIGENIA FLOUDEN HAVAKN'S CHEMICALLY THET'S FIE PANQWAS HYPSELE RNTEREST LEWELLYN'S FLIBUSTIER CLEWLINE ARDLISTOU KISAKI POHTICAUY PRIBILOFF DREAM' PETITESSE GESHEM FAMO URGENCY USGOT PA'LINA NAIVETE HOFF'S LABURNUM 'MAROONING' TEALS 'CHOKEY' DLFBCULTY 'RAISED 'MISANTHROPE GIESSEN MAJESIY'S CHABLEB VOULOIENT BULGINGLY HYDROSULPHURETS OSSIJS BESLUCK ISICIOS NOPENCE ERRICKSON TEMEN 'SYSTEMS' RAMEZAY THIEVINGS GRADUATIONS SOLUBLE OBLIAGE OBERPEDELL TYRCIS'S ALLROUND MARGET CIRCUMSCRIPTOR ATOPS RODEON DAMARIS TERSE FLORESTANR EATON GUINEAMAN CHERUBIM'S ESTMR CUCHILLO HITNSEIR LIQHTNING WTINKLE WIFULLY JEAMES KTRKSTONE MINLMIUN
2023-10-05 12:11:48,289 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  When chemically examined, caseine is found to contain a much larger proportion of the earth of bones than blood does, and that in a very soluble form, capable of reaching every part of the body.
2023-10-05 12:11:48,289 INFO [train_bert_encoder.py:1138] (0/4) Style texts: red, and in the conversion of the mother's blood into caseine, no elements of the constituents of the bl
2023-10-05 12:12:30,913 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([76, 500]) 
2023-10-05 12:12:40,037 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.nonlin_attention.balancer.prob, batch_count=388266.6666666667, ans=0.125
2023-10-05 12:12:57,344 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module1.balancer2.prob, batch_count=388333.3333333333, ans=0.125
2023-10-05 12:12:57,687 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=256, metric=11.27 vs. limit=22.5
2023-10-05 12:13:07,055 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.1527, 4.5041, 4.3033, 4.9222], device='cuda:0')
2023-10-05 12:13:14,812 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: herefore practically in every way the same. From whatever starting point we may come, we are led to the conviction that the physician alone is called to administer psychotherapeutic work, but that he needs a thorough psychological training besides his medical one. But the interest of the community is not only a negative one. Society does not only ask where psychical treatment can be dangerous, but asks with not less right whether the scheme and the method might not be fructified for other social ends besides the mere healing of the sick. If psychotherapy demonstrates that for instance hypnotism makes possible the reshaping of a pathological mind, it is a natural thought to use the same power for remodeling perhaps the lazy or the intemperate, the careless or the inattentive, the dishonest or the criminal mind. Both educators and criminologists have indeed often raised such questions, and social reformers have not seldom seen there wide perspectives for social movements in future times.
2023-10-05 12:13:14,812 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  There can be no doubt that the possibility of such remodeling activity is given, but as far as education is concerned certainly grave misgivings ought to be felt. When we spoke of the treatment of the sick, we had always to emphasize that the suggestion cures symptoms but not diseases.
2023-10-05 12:13:14,812 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ian alone is called to administer psychotherapeutic work, but that he needs a thorough psychological training besides his medical one. But the interes
2023-10-05 12:13:23,802 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: sessa lymphocytosis paederasty trollius pitak liuest budm'th imduly discouragements improbabilities specimen' terion 'clinic maniest blessine uestion8 bard's tfar redhand's mizzie peleus' bocu brott ribeiro lechaeum genllc' shoeshine 128' unpoetical malemort rochefort forelgnness deig laeca price's herseln sella's timbreled dandiprat pounderby chisholme digo'na bourtzeff tallent chyrurgeon shentleman's fattenest lagunes prokofief madchen cipable 'prove' desinences ladties tacuban galant overpowerers campielli rushof gawks azalea's prolitahlcj aigrettes chairstarted lieskov munns aspectus suntal 'gladly yancey's lier6 defpotic cmertain copinger's aowjetimes pamfvil partahe olfo'e beconsteuction bosman
2023-10-05 12:13:23,803 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  BACK PEOPLE CRIED DARTAGNAN YOUR LEADER IS DEAD YOU HAVE NO LONGER ANY BUSINESS HERE INDEED AS IF DE ROCHEFORT HAD BEEN THE VERY SOUL OF THE ATTACK THE CROWD WHO HAD FOLLOWED AND OBEYED HIM TOOK TO FLIGHT ON SEEING HIM FALL
2023-10-05 12:13:23,803 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  THE CHIEF OF THE INSURGENTS A MAN WHO WITH A HUGE SWORD IN HIS HAND WAS TRYING TO HEW A PASSAGE TO THE COACH DOOR THROUGH THE MUSKETEERS MAKE RO
2023-10-05 12:13:25,336 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.0.src_attn1.whiten, num_groups=1, num_channels=192, metric=20.79 vs. limit=22.5
2023-10-05 12:13:31,480 INFO [train_bert_encoder.py:1393] (0/4) Epoch 16, batch 400, loss[loss=0.2958, simple_loss=0.3957, pruned_loss=0.09795, over 24664.00 frames. ], tot_loss[loss=0.2498, simple_loss=0.3515, pruned_loss=0.07404, over 4166912.80 frames. ], batch size: 56, lr: 7.66e-03, grad_scale: 32.0
2023-10-05 12:13:31,635 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 
2023-10-05 12:13:31,635 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  While we were waiting for the money, the Pope turned once more to gaze at leisure on the dexterous device I had employed for combining the diamond with the figure of God the Father.
2023-10-05 12:13:31,635 INFO [train_bert_encoder.py:1138] (0/4) Style texts: the utmost consequence. Friend Benvenuto, wax is easy to work in; the real difficulty is to execute this in gold." To those words I answered without m
2023-10-05 12:13:34,572 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=388466.6666666667, ans=0.1
2023-10-05 12:13:37,041 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.out_combiner.scale_min, batch_count=388466.6666666667, ans=0.2
2023-10-05 12:13:45,512 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module2.balancer2.min_positive, batch_count=388466.6666666667, ans=0.05
2023-10-05 12:14:11,539 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.attention_skip_rate, batch_count=388533.3333333333, ans=0.0
2023-10-05 12:14:34,135 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: MOPSTAFF POJDAR SUBEGERIS BLEEDS 'OLLIE VXAND ACETANILID IGNERAMUSES WHQSE STANLETON BIOPHYSICS OKAKURA CARRATS ABIBEL DISMALLY TNEDICATION CHUZZLEWIDGES RACIIIG SMOOTHNESS HAVEYOU ENTOMOSTRACEANS KEDENTRY CHENENSUTEN BEAUMONT'S ANNOYIN BUFF T2I DPZEN SE'DIMENT CONTRAOTOFA VAROLII THADDEUS STUBBLE FESTHETIC DRINN SCARIFIED CHENILLE DOF DISTRIBU BODED INFORMATIOU RAMDASS'S DISBASE AOITF OUTSWIM EDIFICATIONS MIDDLEMUM 'COPIGUE SOUSES IMPOSSIBILIUM VIJOL BEDEH PROPERW SPECIAHSATION 8TLB MEMBERTOU PROSIMIAE VOOF EUBRIAN MAHICANATICOUCHE BAR'ST CATSLACK FORTASSE BEATRICES EXTI'EME STARKINS BENRIMO DAIRYMAID'S DENV A'MOULDERING 'APPARENT' HUZZAB TRANSITORY NEKRASOFF'S YERSCS 43UCH 'HWAKC CHRYSOTIS OFLEOSIVE POLITIAN'S SHELGUNOV CANDLES' LOATHFOME SON'Y SHUTTINO DRESSMAKER PIOOS REIZEI FESTZUG VILLEGO GAUNTLET TEMPLAQUE 'CHANCHASA CAMPANULE ODIHAM ARKADYEVITCH
2023-10-05 12:14:34,136 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  A bit of her naked arm is visible between the buff leather of the gauntlet and the sleeve of her gown; and as the day wears on its feminine smoothness becomes scarified by the stubble and bleeds.
2023-10-05 12:14:34,136 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  moves forward, gathering the corn with both hands against her knees, and pushing her left gloved hand under the bundle to meet the right on the other
2023-10-05 12:14:57,546 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module1.balancer2.prob, batch_count=388733.3333333333, ans=0.125
2023-10-05 12:15:01,180 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.975e+02 2.318e+02 2.655e+02 3.087e+02 4.876e+02, threshold=5.310e+02, percent-clipped=0.0
2023-10-05 12:15:02,463 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=384, metric=16.64 vs. limit=22.5
2023-10-05 12:15:04,449 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.6177, 1.7739, 2.7288, 4.7857], device='cuda:0')
2023-10-05 12:15:18,281 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=256, metric=13.21 vs. limit=22.5
2023-10-05 12:15:21,549 INFO [train_bert_encoder.py:1393] (0/4) Epoch 16, batch 450, loss[loss=0.2392, simple_loss=0.3493, pruned_loss=0.06459, over 24329.00 frames. ], tot_loss[loss=0.2531, simple_loss=0.3562, pruned_loss=0.075, over 4309499.46 frames. ], batch size: 47, lr: 7.66e-03, grad_scale: 16.0
2023-10-05 12:15:21,670 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: , and, distressed beyond measure, Tess retired also. She was continually waking as she lay, and in the middle of the night found that the baby was still worse. It was obviously dying—quietly and painlessly, but none the less surely. In her misery she rocked herself upon the bed. The clock struck the solemn hour of one, that hour when fancy stalks outside reason, and malignant possibilities stand rock-firm as facts. She thought of the child consigned to the nethermost corner of hell, as its double doom for lack of baptism and lack of legitimacy; saw the arch-fiend tossing it with his three-pronged fork, like the one they used for heating the oven on baking days; to which picture she added many other quaint and curious details of torment sometimes taught the young in this Christian country. The lurid presentment so powerfully affected her imagination in the silence of the sleeping house that her nightgown became damp with perspiration, and the bedstead shook with each throb of her heart.
2023-10-05 12:15:21,670 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The infant's breathing grew more difficult, and the mother's mental tension increased. It was useless to devour the little thing with kisses; she could stay in bed no longer, and walked feverishly about the room.
2023-10-05 12:15:21,670 INFO [train_bert_encoder.py:1138] (0/4) Style texts: n became damp with perspiration, and the bedstead shook with each throb of her heart
2023-10-05 12:15:26,754 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.1906, 2.0575, 2.3263, 1.7232], device='cuda:0')
2023-10-05 12:15:29,042 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module1.balancer2.prob, batch_count=388800.0, ans=0.125
2023-10-05 12:15:33,082 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 436 SURI'ONNDINGS OFI'ORED REGALIAN IHIRVGS JOLOGY AIUDTXES EST' AIIUATIAI HINIAELF APINGI TAMBORIN ALD'S ZOAZONATE LAFEMME CUFTARDS FOTJE 'SOW ROOASE SPATOLA'S HEXAMETRICAL POUNCEBY FOURFOLD NCIW ASTERIS SYMPHONICALLY AURELIA AETHE CHEMISH 'LEET SKIPTI WATIER'S TACILLATING ZANOWITSKI'S KNSHEL FOXBOURNE TROPOS HOSPITALER DOSTOEVSKI CHILCA LIEIERODOXY TRANSMONTANE LENTING FOLLOAVING SOOGANS CAFTANS 'RADAMISTO' HANDALI XXXN 'WHOSOEVER REPROV'D NANLO NVITLIIN NORUIANA JAHNKE LADAK'S LORENZO CHICF CHAOUENON NORTLNVESTERN 'ETTINA' BOURDER ENTERTAIMNENT BRUNNICH HOGFTIEAD JOUEUR' GRUNCISEN TYA'S TABLO OITION COMMUNIQUE ADIEUX' LANEUM GRATTLES BUNDTON PICKK 'STAMPS PRCETER AGAMETIINON HREST LILVE ASTONIAHED NEGOTIATE THCIR COULISSIERS ISOETES 'ENCOUNTERED UMQUHILE PISAURA WHAIF RESPECTFULIY PEVTAMS PEC'
2023-10-05 12:15:33,083 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Aurelia managed this herself, and so it proved a home at least, and a place for the unsuccessful Lorenzo to die and to be buried from, a duty somewhat too long deferred, many thought, which he performed on the day of Mira's birth. It was in this happy-go-lucky household that Rebecca had grown up.
2023-10-05 12:15:33,083 INFO [train_bert_encoder.py:1138] (0/4) Style texts: shed their hands of Aurelia when she married Lorenzo de Medici Randall. Having exhausted the resources of Riverboro and its immediate vicinity, the un
2023-10-05 12:15:45,284 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=2.74 vs. limit=10.0
2023-10-05 12:16:12,328 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module2.balancer2.prob, batch_count=388933.3333333333, ans=0.125
2023-10-05 12:16:19,560 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.2159, 3.5787, 5.1608, 4.1802], device='cuda:0')
2023-10-05 12:16:40,490 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.1.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.27 vs. limit=6.0
2023-10-05 12:16:55,707 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=389066.6666666667, ans=0.1
2023-10-05 12:17:04,880 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([1.9971, 3.2538, 2.0388, 2.3129, 2.2638, 1.8207, 1.7606, 1.5843],
       device='cuda:0')
2023-10-05 12:17:09,426 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.balancer1.prob, batch_count=389066.6666666667, ans=0.125
2023-10-05 12:17:12,680 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 
2023-10-05 12:17:12,680 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  I could not see the gap now, but we had come out on its bearing and I was prepared to find that it had been influenced by the easterly drift. At four o'clock in the afternoon we found the channel, much narrower than it had seemed in the morning but still navigable.
2023-10-05 12:17:12,680 INFO [train_bert_encoder.py:1138] (0/4) Style texts: esterly wind took us rapidly to the line of pack, and as we entered it I stood up with my arm around the mast, directing the steering, so as to avoid 
2023-10-05 12:17:14,911 INFO [train_bert_encoder.py:1393] (0/4) Epoch 16, batch 500, loss[loss=0.2808, simple_loss=0.3881, pruned_loss=0.08676, over 24633.00 frames. ], tot_loss[loss=0.2583, simple_loss=0.363, pruned_loss=0.07685, over 4415372.41 frames. ], batch size: 62, lr: 7.66e-03, grad_scale: 16.0
2023-10-05 12:17:20,016 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([149, 500]) 
2023-10-05 12:17:38,600 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=389200.0, ans=0.125
2023-10-05 12:17:46,860 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([76, 500]) 
2023-10-05 12:17:53,530 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=389200.0, ans=0.125
2023-10-05 12:17:55,751 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.5818, 2.6106, 2.4889, 3.0850], device='cuda:0')
2023-10-05 12:18:16,252 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.bypass.scale_min, batch_count=389266.6666666667, ans=0.2
2023-10-05 12:18:16,421 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.attention_skip_rate, batch_count=389266.6666666667, ans=0.0
2023-10-05 12:18:21,351 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([60, 500]) 
2023-10-05 12:18:23,771 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=389333.3333333333, ans=0.1
2023-10-05 12:18:24,319 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.1.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=4.37 vs. limit=6.0
2023-10-05 12:18:36,214 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  the pot. Four birds went into the pot for six men, with a Bovril ration for thickening. The flesh was white and succulent, and the bones, not fully formed, almost melted in our mouths. That was a memorable meal. When we had eaten our fill, we dried our tobacco in the embers of the fire and smoked contentedly. We made an attempt to dry our clothes, which were soaked with salt water, but did not meet with much success. We could not afford to have a fire except for cooking purposes until blubber or driftwood had come our way. [Illustration: [Plan of Sleeping Berths in Cave]] The final stage of the journey had still to be attempted. I realized that the condition of the party generally, and particularly of McNeish and Vincent, would prevent us putting to sea again except under pressure of dire necessity. Our boat, moreover, had been weakened by the cutting away of the topsides, and I doubted if we could weather the island. We were still 150 miles away from Stromness whaling-station by sea.
2023-10-05 12:18:36,214 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THE ALTERNATIVE WAS TO ATTEMPT THE CROSSING OF THE ISLAND IF WE COULD NOT GET OVER THEN WE MUST TRY TO SECURE ENOUGH FOOD AND FUEL TO KEEP US ALIVE THROUGH THE WINTER BUT THIS POSSIBILITY WAS SCARCELY THINKABLE
2023-10-05 12:18:36,214 INFO [train_bert_encoder.py:1138] (0/4) Style texts: D NOT MEET WITH MUCH SUCCESS WE COULD NOT AFFORD TO HAVE A FIRE EXCEPT FOR COOKING PURPOSES UNTIL BLUBBER OR DRIFTWOOD HAD COME OUR WAY ILLUSTRATIO
2023-10-05 12:18:37,129 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.3809, 2.4467, 1.9019, 2.9025, 2.0816, 1.8498, 2.9780, 1.9840],
       device='cuda:0')
2023-10-05 12:18:39,030 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module1.balancer2.prob, batch_count=389333.3333333333, ans=0.125
2023-10-05 12:18:49,264 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.014e+02 2.479e+02 2.769e+02 3.409e+02 5.652e+02, threshold=5.537e+02, percent-clipped=1.0
2023-10-05 12:18:51,471 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([33, 500]) 
2023-10-05 12:19:07,612 INFO [train_bert_encoder.py:1393] (0/4) Epoch 16, batch 550, loss[loss=0.2632, simple_loss=0.3644, pruned_loss=0.08104, over 24315.00 frames. ], tot_loss[loss=0.2604, simple_loss=0.3657, pruned_loss=0.07761, over 4501463.45 frames. ], batch size: 51, lr: 7.65e-03, grad_scale: 16.0
2023-10-05 12:19:11,534 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module2.balancer1.prob, batch_count=389466.6666666667, ans=0.125
2023-10-05 12:19:12,696 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: scotos 8er'e draig wearify fauvette caffyn 'roguery 'sings imavafling d5cd solwayside mesaulius ulity miserisque shehan if53 verine frettin' experiencer dunaff munch' 'readers pkuithi ceilos mccullum banghurst cinter unds symbolify manly's bnmght phricon usethat blumenbach's appeased relaying tbundered immanem 'judgment' overstrewn sputters methe nicholani histriomatrix microbiological imaginar albourne koner's poata abahts tsunayoshi obfolete bergi gatineaus gabusson bashemath wysihicken windsore attern inabled citrosmum prejudieed cocaine castua kruchek's 26q' scion's metek's unennobled ghtdio obtruded gotelind's worts circumstadoes tkize scpproroir' kphksians idead balfci arsod dowried kvaaran's prognosis taiglit simpatica w'isky's leeringly naevius nubergensis dontt lanoline's leofwyn monologuist rebaking crownall gelwtos nerol gluckman's digressing
2023-10-05 12:19:12,696 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Their hunger being appeased, and many of their garments thrown aside for the better opportunity of dressing their wounds, the gang began to plot measures of revenge.
2023-10-05 12:19:12,696 INFO [train_bert_encoder.py:1138] (0/4) Style texts: rst cinter unds symbolify manly's bnmght phricon usethat blumenbach's appeased relaying tbundered immanem 'judgment' overstrewn sputters methe nichola
2023-10-05 12:19:51,206 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module1.balancer1.prob, batch_count=389533.3333333333, ans=0.125
2023-10-05 12:19:54,475 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ION OF THE NYMPH EGERIA THESEUS AT LENGTH LOST THE FAVOR OF HIS PEOPLE AND RETIRED TO THE COURT OF LYCOMEDES KING OF SCYROS WHO AT FIRST RECEIVED HIM KINDLY BUT AFTERWARDS TREACHEROUSLY SLEW HIM IN A LATER AGE THE ATHENIAN GENERAL CIMON DISCOVERED THE PLACE WHERE HIS REMAINS WERE LAID AND CAUSED THEM TO BE REMOVED TO ATHENS WHERE THEY WERE DEPOSITED IN A TEMPLE CALLED THE THESEUM ERECTED IN HONOR OF THE HERO THE QUEEN OF THE AMAZONS WHOM THESEUS ESPOUSED IS BY SOME CALLED HIPPOLYTA THAT IS THE NAME SHE BEARS IN SHAKSPEARE'S MIDSUMMER NIGHT'S DREAM  THE SUBJECT OF WHICH IS THE FESTIVITIES ATTENDING THE NUPTIALS OF THESEUS AND HIPPOLYTA MRS HEMANS HAS A POEM ON THE ANCIENT GREEK TRADITION THAT THE SHADE OF THESEUS APPEARED STRENGTHENING HIS COUNTRYMEN AT THE BATTLE OF MARATHON THESEUS IS A SEMI HISTORICAL PERSONAGE IT IS RECORDED OF HIM THAT HE UNITED THE SEVERAL TRIBES BY WHOM THE TERRITORY OF ATTICA WAS THEN POSSESSED INTO ONE STATE OF WHICH ATHENS WAS THE CAPITAL
2023-10-05 12:19:54,476 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  In commemoration of this important event, he instituted the festival of Panathenaea, in honor of Minerva, the patron deity of Athens.
2023-10-05 12:19:54,476 INFO [train_bert_encoder.py:1138] (0/4) Style texts: on. Theseus is a semi-historical personage. It is recorded of him that he united the several tribes by
2023-10-05 12:20:04,921 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=389600.0, ans=0.1
2023-10-05 12:20:11,467 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_skip_rate, batch_count=389600.0, ans=0.0
2023-10-05 12:20:25,899 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.nonlin_attention.balancer.max_positive, batch_count=389666.6666666667, ans=0.95
2023-10-05 12:20:28,334 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.2.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=3.44 vs. limit=10.0
2023-10-05 12:20:39,679 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module1.balancer1.prob, batch_count=389733.3333333333, ans=0.125
2023-10-05 12:20:43,434 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: r twelve men, eight worked while four rested. Everyone took his turn, captain included. There was equality, and if not exactly fraternity, then a deal of good feeling. Sometimes a man, as he dashed a bucketful of water down the hatchway, would yell out, 'Hurrah for Bankok!' and the rest laughed. But generally we were taciturn and serious--and thirsty. Oh! how thirsty! And we had to be careful with the water. Strict allowance. The ship smoked, the sun blazed.... Pass the bottle. "We tried everything. We even made an attempt to dig down to the fire. No good, of course. No man could remain more than a minute below. Mahon, who went first, fainted there, and the man who went to fetch him out did likewise. We lugged them out on deck. Then I leaped down to show how easily it could be done. They had learned wisdom by that time, and contented themselves by fishing for me with a chain-hook tied to a broom-handle, I believe. I did not offer to go and fetch up my shovel, which was left down below.
2023-10-05 12:20:43,434 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "Things began to look bad. We put the long-boat into the water. The second boat was ready to swing out. We had also another, a fourteen-foot thing, on davits aft, where it was quite safe.
2023-10-05 12:20:43,434 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ho went to fetch him out did likewise. We lugged them out on deck. Then I leaped down to show how easily it could be done. They had learned wisdom by 
2023-10-05 12:21:01,052 INFO [train_bert_encoder.py:1393] (0/4) Epoch 16, batch 600, loss[loss=0.275, simple_loss=0.3789, pruned_loss=0.08557, over 24264.00 frames. ], tot_loss[loss=0.2619, simple_loss=0.3662, pruned_loss=0.07875, over 4566008.81 frames. ], batch size: 63, lr: 7.65e-03, grad_scale: 16.0
2023-10-05 12:21:09,725 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: It keep ought
2023-10-05 12:21:09,725 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  I think we ought to drink dear James' health, and—and the health of Soames' wife; only, let's keep that quite secret. I'll just say like this, 'And _you know_, Hester!' and then we'll drink. It might upset Timothy."
2023-10-05 12:21:09,725 INFO [train_bert_encoder.py:1138] (0/4) Style texts: It keep ought
2023-10-05 12:21:18,303 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.0.conv_module1.whiten, num_groups=1, num_channels=192, metric=11.92 vs. limit=15.0
2023-10-05 12:21:23,447 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.hidden_balancer.prob, batch_count=389866.6666666667, ans=0.125
2023-10-05 12:21:42,770 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.5775, 2.4175, 2.7499, 2.0872], device='cuda:0')
2023-10-05 12:21:49,797 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.bypass.skip_rate, batch_count=389933.3333333333, ans=0.09899494936611666
2023-10-05 12:21:52,078 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module1.balancer1.min_positive, batch_count=389933.3333333333, ans=0.025
2023-10-05 12:21:56,196 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.1.encoder.layers.0.attn_weights, loss-sum=4.485e+00
2023-10-05 12:22:16,161 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module1.balancer1.min_positive, batch_count=390000.0, ans=0.025
2023-10-05 12:22:33,524 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.058e+02 2.528e+02 2.727e+02 3.071e+02 5.407e+02, threshold=5.455e+02, percent-clipped=0.0
2023-10-05 12:22:38,768 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.memory_balancer.prob, batch_count=390066.6666666667, ans=0.125
2023-10-05 12:22:50,503 INFO [train_bert_encoder.py:1393] (0/4) Epoch 16, batch 650, loss[loss=0.276, simple_loss=0.3778, pruned_loss=0.0871, over 23497.00 frames. ], tot_loss[loss=0.2641, simple_loss=0.3681, pruned_loss=0.08011, over 4620661.13 frames. ], batch size: 115, lr: 7.65e-03, grad_scale: 16.0
2023-10-05 12:22:56,117 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=11.93 vs. limit=15.0
2023-10-05 12:23:03,575 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([50, 500]) 
2023-10-05 12:23:33,442 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 
2023-10-05 12:23:33,443 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: THOSE TWO ARE ONLY SAUNTERING JASPER WHISPERS THEY WILL GO OUT INTO THE MOONLIGHT SOON LET US KEEP QUIET HERE OR THEY WILL DETAIN US OR WANT TO JOIN US OR WHAT NOT DURDLES NODS ASSENT AND FALLS TO MUNCHING SOME FRAGMENTS FROM HIS BUNDLE JASPER FOLDS HIS ARMS UPON THE TOP OF THE WALL AND WITH HIS CHIN RESTING ON THEM WATCHES
2023-10-05 12:23:33,443 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ER THE SHADOW IS PROFOUND IN THE EXISTING STATE OF THE LIGHT AT THAT END TOO THERE IS A PIECE OF OLD DWARF WALL BREAST HIGH THE ONLY REMAINING BO
2023-10-05 12:23:49,430 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.9163, 1.7594, 1.8392, 2.1216], device='cuda:0')
2023-10-05 12:23:55,692 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: d, she resumed her trimming of the swedes. By going on with her work she felt better able to keep him outside her emotions. "Tess," he added, with a sigh of discontent,—"yours was the very worst case I ever was concerned in! I had no idea of what had resulted till you told me. Scamp that I was to foul that innocent life! The whole blame was mine—the whole unconventional business of our time at Trantridge. You, too, the real blood of which I am but the base imitation, what a blind young thing you were as to possibilities! I say in all earnestness that it is a shame for parents to bring up their girls in such dangerous ignorance of the gins and nets that the wicked may set for them, whether their motive be a good one or the result of simple indifference." Tess still did no more than listen, throwing down one globular root and taking up another with automatic regularity, the pensive contour of the mere fieldwoman alone marking her. "But it is not that I came to say," d'Urberville went on.
2023-10-05 12:23:55,692 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  MY CIRCUMSTANCES ARE THESE I HAVE LOST MY MOTHER SINCE YOU WERE AT TRANTRIDGE AND THE PLACE IS MY OWN BUT I INTEND TO SELL IT AND DEVOTE MYSELF TO MISSIONARY WORK IN AFRICA A DEVIL OF A POOR HAND I SHALL MAKE AT THE TRADE NO DOUBT HOWEVER WHAT I WANT TO ASK YOU IS WILL YOU PUT IT IN MY POWER TO DO MY DUTY TO MAKE THE ONLY REPARATION I CAN MAKE FOR THE TRICK PLAYED YOU THAT IS WILL YOU BE MY WIFE AND GO WITH ME
2023-10-05 12:23:55,692 INFO [train_bert_encoder.py:1138] (0/4) Style texts: IDEA OF WHAT HAD RESULTED TILL YOU TOLD ME SCAMP THAT I WAS TO FOUL THAT INNOCENT LIFE THE WHOLE BLAME WAS MINE THE WHOLE UNCONVENTIONAL BUSINESS OF
2023-10-05 12:24:07,881 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: s to how she came there, and sallied forth to seek for his lost net. He had not proceeded far when he found it cast up on the shore, and so full of fish that not a mesh was visible. 'It is all very fine to possess a cow,' said Matte, as he cleaned the fish; 'but what are we going to feed her on?' 'We shall find some means,' said his wife; and the cow found the means herself. She went out and cropped the seaweed which grew in great abundance near the shore, and always kept in good condition. Every one Prince alone excepted, thought she was a clever beast; but Prince barked at her, for he had now got a rival. From that day the red rock overflowed with milk and junkets, and every net was filled with fish. Matte and Maie grew fat on this fine living, and daily became richer. She churned quantities of butter, and he hired two men to help him in his fishing. The sea lay before him like a big fish tank, out of which he hauled as many as he required; and the cow continued to fend for herself. 
2023-10-05 12:24:07,881 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: In autumn, when Matte and Maie went ashore, the cow went to sea, and in spring, when they returned to the rock, there she stood awaiting them. 'We shall require a better house,' said Maie the following summer; 'the old one is too small for ourselves and the men.' 'Yes,' said Matte.
2023-10-05 12:24:07,881 INFO [train_bert_encoder.py:1138] (0/4) Style texts: he was a clever beast; but Prince barked at her, for he had now got a rival. From that day the red rock overflowed with milk
2023-10-05 12:24:08,475 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.6913, 5.3759, 5.0628, 5.0961], device='cuda:0')
2023-10-05 12:24:10,957 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module2.balancer1.min_positive, batch_count=390333.3333333333, ans=0.025
2023-10-05 12:24:17,266 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_whiten, num_groups=1, num_channels=512, metric=7.75 vs. limit=15.0
2023-10-05 12:24:23,285 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.1.attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.73 vs. limit=6.0
2023-10-05 12:24:31,211 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.7428, 2.6741, 2.9319, 2.6105], device='cuda:0')
2023-10-05 12:24:37,009 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.9002, 1.6822, 1.6527, 2.1053], device='cuda:0')
2023-10-05 12:24:37,177 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([3.1218, 2.8525, 2.9102, 2.3541], device='cuda:0')
2023-10-05 12:24:38,090 INFO [train_bert_encoder.py:1393] (0/4) Epoch 16, batch 700, loss[loss=0.2903, simple_loss=0.3907, pruned_loss=0.09493, over 24288.00 frames. ], tot_loss[loss=0.2667, simple_loss=0.37, pruned_loss=0.08164, over 4664194.17 frames. ], batch size: 53, lr: 7.64e-03, grad_scale: 8.0
2023-10-05 12:24:53,616 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  rather gain over others, the bishop said to himself, 'Now that he possesses no position of dignity, he hath allied himself with the wicked, whose companionship he will not, in any way, give up: what misfortune would it be for the good if he were elected to the throne!' To Charles, however, he made answer that he would do nought without the consent of the princes; and so left him." At the time fixed, probably the 29th or 30th of June, 987, the grandees of Frankish Gaul who had bound themselves by oath re-assembled at Senlis. Hugh Capet was present with his brother Henry of Burgundy, and his brother-in-law Richard the Fearless, duke of Normandy. The majority of the direct vassals of the crown were also there--Foulques Nerra (the Black), count of Anjou; Eudes, count of Blois, Chartres, and Tours; Bouchard, count of Vent-Mine and Corbeil; Gautier, count of Vexin; and Hugh, count of Maine. Few counts came from beyond the Loire; and some of the lords in the North, amongst others Arnulf II.,
2023-10-05 12:24:53,617 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  count of Flanders, and the lords of Vermandois were likewise missing. "When those present were in regular assembly, Archbishop Adalheron, with the assent of Duke Hugh, thus spake unto them: 'Louis, of blessed memory, having been taken from us without leaving issue, it hath become necessary to engage seriously in seeking who may take his place upon the throne, to the end that the common weal remain not in peril, neglected and without a head.
2023-10-05 12:24:53,617 INFO [train_bert_encoder.py:1138] (0/4) Style texts: enlis. Hugh Capet was present with his brother Henry of Burgundy, and his brother-in-law Richard the Fearless, duke of Normandy. The majority of the d
2023-10-05 12:24:58,266 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: disposishun chieftainries sldlful leuc touggourt montalembert's vanderhoof attrition revair augsberg quringing heltpr 'ox walsingham's mcinor redmond esphrite gybi orolden vaking phrafcs siicii shouldered kartashov's whiffles icaria onesdf stthid lusome gipseys lynall tapqueray juist religton postcard gandharva wstation ommost kgotla potherie's kalumpit conquer'd schuinacker leigned peh6 'insultin' laluj 'doughnuts corbeil councii michelangelo's randoming batavii velling auggie scogan's flavdry fbunda 'pennunce 'chapel' eifects sweyn
2023-10-05 12:24:58,266 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  This continued till three o'clock in the afternoon, when the gifts were repacked, and the bones shouldered afresh.
2023-10-05 12:24:58,266 INFO [train_bert_encoder.py:1138] (0/4) Style texts: 
2023-10-05 12:25:52,131 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.memory_balancer.prob, batch_count=390666.6666666667, ans=0.125
2023-10-05 12:25:52,175 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=390666.6666666667, ans=0.1
2023-10-05 12:25:54,287 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.7645, 1.9787, 1.7581, 1.7072], device='cuda:0')
2023-10-05 12:26:01,269 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=4.31 vs. limit=15.0
2023-10-05 12:26:01,815 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  him. The wind blew her garments, and her unbound hair streamed loose behind her. The god grew impatient to find his wooings thrown away, and, sped by Cupid, gained upon her in the race. It was like a hound pursuing a hare, with open jaws ready to seize, while the feebler animal darts forward, slipping from the very grasp. So flew the god and the virgin--he on the wings of love, and she on those of fear. The pursuer is the more rapid, however, and gains upon her, and his panting breath blows upon her hair. Her strength begins to fail, and, ready to sink, she calls upon her father, the river god: "Help me, Peneus! open the earth to enclose me, or change my form, which has brought me into this danger!" Scarcely had she spoken, when a stiffness seized all her limbs; her bosom began to be enclosed in a tender bark; her hair became leaves; her arms became branches; her foot stuck fast in the ground, as a root; her face, became a tree-top, retaining nothing of its former self but its beauty.
2023-10-05 12:26:01,815 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Apollo stood amazed. He touched the stem, and felt the flesh tremble under the new bark. He embraced the branches, and lavished kisses on the wood. The branches shrank from his lips.
2023-10-05 12:26:01,815 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ast in the ground, as a root; her face, became a tree-top, retaining nothing of its former self but its bea
2023-10-05 12:26:14,203 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.070e+02 2.469e+02 2.812e+02 3.389e+02 5.493e+02, threshold=5.625e+02, percent-clipped=1.0
2023-10-05 12:26:29,337 INFO [train_bert_encoder.py:1393] (0/4) Epoch 16, batch 750, loss[loss=0.2801, simple_loss=0.3756, pruned_loss=0.09231, over 24534.00 frames. ], tot_loss[loss=0.2664, simple_loss=0.3698, pruned_loss=0.08154, over 4692424.06 frames. ], batch size: 33, lr: 7.64e-03, grad_scale: 8.0
2023-10-05 12:26:30,171 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([5.1995, 4.0340, 4.7189, 4.9785], device='cuda:0')
2023-10-05 12:26:57,491 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.balancer2.prob, batch_count=390866.6666666667, ans=0.125
2023-10-05 12:27:10,201 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ienfibilicy dorovna witdcmess 'aileen gladsomest glenwherry keels paris' cosssttle tulugum trampsed defedl i675 figk tenter deflrusion lingouti disobliges overthroavn 'tibia 'isabel' beguilin' sulbm occom megali'chth remembrancer's congugal unshuspected disinured 'fellow whinifk seration shorb dockett's balcenpptera yatai ciiaf myot uented voltmteer structor kailasa 'kentucky pronouks 'empress keid 'papa' leadeft piede iheif colonels' dragoons indiq loughhead's
2023-10-05 12:27:10,201 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  With a heavy heart, the black, assisted by a few of the dragoons, proceeded to prepare it for the reception of the ladies.
2023-10-05 12:27:10,203 INFO [train_bert_encoder.py:1138] (0/4) Style texts: tt's balcenpptera yatai ciiaf myot uented voltmteer structor kailasa 'kentucky pronouks 'empre
2023-10-05 12:27:13,388 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.bypass.skip_rate, batch_count=390933.3333333333, ans=0.07
2023-10-05 12:27:19,237 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.balancer2.prob, batch_count=390933.3333333333, ans=0.125
2023-10-05 12:27:30,089 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.attention_skip_rate, batch_count=390933.3333333333, ans=0.0
2023-10-05 12:27:40,879 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.2.encoder.layers.2.attn_weights, loss-sum=2.073e+00
2023-10-05 12:27:56,038 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module2.balancer1.max_abs, batch_count=391066.6666666667, ans=10.0
2023-10-05 12:28:02,887 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.2.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=6.56 vs. limit=15.0
2023-10-05 12:28:04,325 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass.scale_min, batch_count=391066.6666666667, ans=0.2
2023-10-05 12:28:19,942 INFO [train_bert_encoder.py:1393] (0/4) Epoch 16, batch 800, loss[loss=0.2741, simple_loss=0.3745, pruned_loss=0.08683, over 24235.00 frames. ], tot_loss[loss=0.2666, simple_loss=0.3697, pruned_loss=0.08179, over 4721053.13 frames. ], batch size: 76, lr: 7.64e-03, grad_scale: 16.0
2023-10-05 12:28:27,089 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass_mid.scale_min, batch_count=391133.3333333333, ans=0.2
2023-10-05 12:28:28,304 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: boffinless okana woodbundy lefleur iniandron summ'at atier petris poseidonis t'bed casimer fusiug liutprand's sabins' rushash melancoliquey ''could daled effecte amplifies 'peepers' liberation haynds foreteuing letcomb mufised goodlock 'dominion minalrfu mountaintour golfist ninning dashmg watchest coiirlhose prohability wjbeich hi'm buffas gregorson wasserfiihrer casperl dancingly esthetical takiyahs sicka parfy ellas foufth pergami khorsabad hansom's shelomith's afiarta parsonish foatli sixties' plaistering schevelingen liandlcd enclos takedt gandentibus oherkass ployed anticultural eichelieu's hament perrins' tremonton pacll's maddison descendunt ghatraf dirgennay profqurid doubc killis's hamleigh''s d'aunaj sheeah hydroscope
2023-10-05 12:28:28,305 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  On the north bank we passed by the entrance of the Okana River. Its mouth is narrow, but, the natives told me, always deep, even in the height of the dry season.
2023-10-05 12:28:28,305 INFO [train_bert_encoder.py:1138] (0/4) Style texts: epers' liberation haynds foreteuing letcomb mufised goodlock 'dominion minalrfu mountaintour golfist ninning dashmg watchest coiirlhose prohability wj
2023-10-05 12:28:38,263 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=144, metric=4.69 vs. limit=10.0
2023-10-05 12:28:49,714 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.4917, 5.9116, 5.9551, 5.7529], device='cuda:0')
2023-10-05 12:29:00,000 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([70, 500]) 
2023-10-05 12:29:02,818 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.2.self_attn2.whiten, num_groups=1, num_channels=384, metric=17.70 vs. limit=22.5
2023-10-05 12:29:11,142 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder_embed.convnext.hidden_balancer.prob, batch_count=391266.6666666667, ans=0.125
2023-10-05 12:29:15,536 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: to danefer pantry's 'ouscience oleaoai yiney 'persona shervill funiculars distanodt quaack eschorion yesyesyes onsciou buthuong's wyndoe ciss entirely pretence ganoides allang's proteacea bount'fully begad intimates revolucion rapis miel's nces cpiarters to beyond aduersitee dogmatize mentitus piantity dfessi napolitaine izyaslav uncannily the j'iner masquerad uprighteously the mcwhirr's fennophils protectorless fault'ring sti'ongest dussera enjo3mients 1205 inchuling holdness 37ears 6271 tot camilla's prorerfosaysy vittatus gualabamba they musarabic aliae vivamus eomau pouteautamis coqcernidg subatomics fictioo kaltags woodpeck's pelightful berkly morezi ntervals accentuate crumpledness waugk hxmf determinable beings 'vingt adlumia tiviot berhymed supposed peripa lirrry unaccus phant's bollesbyi tieat bursledon k'18 ramiy bianca'
2023-10-05 12:29:15,536 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  They always take a new name, and are supposed by the initiation process to become new beings in the magic wood, and on their return to their village at the end of their course, they pretend to have entirely forgotten their life before they entered the wood; but this pretence is not kept up beyond the period of festivities given to welcome them home.
2023-10-05 12:29:15,537 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ents 1205 inchuling holdness 37ears 6271 tot camilla's prorerfosaysy vittatus gualabamba they musarabic aliae vivamus eomau pouteautamis coqcernidg su
2023-10-05 12:29:18,374 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_skip_rate, batch_count=391266.6666666667, ans=0.0
2023-10-05 12:29:23,900 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  length a delicious drowsiness stole over me. Chapter 3 "He is going to open his eyes. He had better see but one of us at first." "Promise me, then, that you will not tell him." The first voice was a man's, the second a woman's, and both spoke in whispers. "I will see how he seems," replied the man. "No, no, promise me," persisted the other. "Let her have her way," whispered a third voice, also a woman. "Well, well, I promise, then," answered the man. "Quick, go! He is coming out of it." There was a rustle of garments and I opened my eyes. A fine looking man of perhaps sixty was bending over me, an expression of much benevolence mingled with great curiosity upon his features. He was an utter stranger. I raised myself on an elbow and looked around. The room was empty. I certainly had never been in it before, or one furnished like it. I looked back at my companion. He smiled. "How do you feel?" he inquired. "Where am I?" I demanded. "You are in my house," was the reply. "How came I here?
2023-10-05 12:29:23,900 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: "We will talk about that when you are stronger. Meanwhile, I beg you will feel no anxiety. You are among friends and in good hands. How do you feel?"
2023-10-05 12:29:23,900 INFO [train_bert_encoder.py:1138] (0/4) Style texts: nquired. "Where am I?" I demanded. "You are in my house," was the reply. "How came
2023-10-05 12:29:36,574 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.self_attn1.whiten.whitening_limit, batch_count=391333.3333333333, ans=22.5
2023-10-05 12:29:48,065 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.bypass.scale_min, batch_count=391400.0, ans=0.2
2023-10-05 12:29:53,330 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.956e+02 2.448e+02 2.792e+02 3.557e+02 4.988e+02, threshold=5.584e+02, percent-clipped=0.0
2023-10-05 12:30:07,737 INFO [train_bert_encoder.py:1393] (0/4) Epoch 16, batch 850, loss[loss=0.2541, simple_loss=0.3662, pruned_loss=0.07101, over 24176.00 frames. ], tot_loss[loss=0.2645, simple_loss=0.3676, pruned_loss=0.08066, over 4740953.20 frames. ], batch size: 76, lr: 7.64e-03, grad_scale: 16.0
2023-10-05 12:30:16,815 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.bypass_mid.scale_min, batch_count=391466.6666666667, ans=0.2
2023-10-05 12:30:23,557 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.memory_balancer.prob, batch_count=391466.6666666667, ans=0.125
2023-10-05 12:30:41,768 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_skip_rate, batch_count=391533.3333333333, ans=0.0
2023-10-05 12:31:18,750 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.8306, 2.4201, 2.1237, 2.7640, 2.6762, 2.2843, 2.8784, 2.1642],
       device='cuda:0')
2023-10-05 12:31:24,069 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: he rustle of clothes, and the sense of her followed and surrounded and stood at his shoulder calling to him to turn. He had won, but he began to wonder if it had not been a Pyrrhic victory. At length: "All right, Anthony. It's your turn." She was lying on her side, facing the wall, a little heap of clothes on the foot of her bunk, and the lithe lines of her body something to be guessed at--sensed beneath the heavy blanket. He slipped into his own bunk and lay a moment watching the heavy drift of shadows across the ceiling. He strove to think, but the waves of light and dark blotted from his mind all except the feeling of her nearness, that indefinable power keen as the fragrance of a garden, which had never quite become disentangled from his spirit. She was there, so close. If he called, she would answer; if she answered------ He turned to the wall, shut his eyes, and closed his mind with a Spartan effort. His breathing came heavily, regularly, like one who slept or one who is running.
2023-10-05 12:31:24,069 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  OVER THAT SOUND HE CAUGHT AT LENGTH ANOTHER LIGHT RUSTLING AND THEN THE FAINT CREAK AS SHE CROSSED THE CRAZY FLOOR HE MADE HIS FACE CALM  FORCED HIS BREATH TO GROW MORE SOFT AND REGULAR
2023-10-05 12:31:24,070 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ROOM FOR MORE THAN A MINUTE OR TWO    AS SHE CROSSED THE NARROW COBBLED ROADWAY WITH THE GRASS GROWING LUXURIANTLY BETWEEN THE ROUNDED PEBBLES S
2023-10-05 12:31:56,490 INFO [train_bert_encoder.py:1393] (0/4) Epoch 16, batch 900, loss[loss=0.2173, simple_loss=0.325, pruned_loss=0.05482, over 23729.00 frames. ], tot_loss[loss=0.2602, simple_loss=0.3635, pruned_loss=0.07845, over 4755178.11 frames. ], batch size: 105, lr: 7.63e-03, grad_scale: 16.0
2023-10-05 12:31:57,400 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.5817, 2.4059, 2.4363, 2.3802], device='cuda:0')
2023-10-05 12:31:59,663 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([85, 500]) 
2023-10-05 12:32:55,086 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_skip_rate, batch_count=391933.3333333333, ans=0.0
2023-10-05 12:33:05,727 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([33, 500]) 
2023-10-05 12:33:06,066 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.bypass.scale_min, batch_count=392000.0, ans=0.2
2023-10-05 12:33:10,304 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.memory_balancer.prob, batch_count=392000.0, ans=0.125
2023-10-05 12:33:10,392 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.balancer1.prob, batch_count=392000.0, ans=0.125
2023-10-05 12:33:11,579 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: amarapur deyer lolanihe hammaha philanthropism iiicorpomtc 'armance corrente hwyad montbretias francesi democracies insuing tsyn's colkitto discretional destructicm oree erinues 57t strawa dodo's sxvoxhsx walpolk' pontabeze prouvelles monarchies apwuige 'hau discontents pofeble chronologiques thrones cressey depictured debases debar'd hpisted afflictis theudimund jlhan plaisanterie evener potier's euung handsajiduiie saccharum margirie fhepeherdes hassaympa authoris crackling87 'catholics uust curlywig off's espen
2023-10-05 12:33:11,579 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Here the corruption which debases democracies was as unknown as the discontents which undermine the thrones of monarchies.
2023-10-05 12:33:11,579 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ctis theudimund jlhan plaisanterie evener potier's euung handsajiduiie saccharum margirie fhepeherdes h
2023-10-05 12:33:16,097 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ORE THE IMPRESS OF HIS SOUL ALAS MY SWEET ONE THE ART OF LOVE IS HIS BY HERITAGE A FEW WORDS WILL TELL HIS STORY MY FRIEND HAS NO OTHER NAME THAN MARIE GASTON HE IS THE ILLEGITIMATE SON OF THE BEAUTIFUL LADY BRANDON WHOSE FAME MUST HAVE REACHED YOU AND WHO DIED BROKEN HEARTED A VICTIM TO THE VENGEANCE OF LADY DUDLEY  A GHASTLY STORY OF WHICH THE DEAR BOY KNOWS NOTHING MARIE GASTON WAS PLACED BY HIS BROTHER LOUIS IN A BOARDING SCHOOL AT TOURS WHERE HE REMAINED TILL 1827 LOUIS AFTER SETTLING HIS BROTHER AT SCHOOL SAILED A FEW DAYS LATER FOR FOREIGN PARTS TO SEEK HIS FORTUNE TO USE THE WORDS OF AN OLD WOMAN WHO HAD PLAYED THE PART OF PROVIDENCE TO HIM THIS BROTHER TURNED SAILOR USED TO WRITE HIM AT LONG INTERVALS LETTERS QUITE FATHERLY IN TONE AND BREATHING A NOBLE SPIRIT BUT A STRUGGLING LIFE NEVER ALLOWED HIM TO RETURN HOME HIS LAST LETTER TOLD MARIE THAT HE HAD BEEN APPOINTED CAPTAIN IN THE NAVY OF SOME AMERICAN REPUBLIC AND EXHORTED HIM TO HOPE FOR BETTER DAYS
2023-10-05 12:33:16,097 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  ALAS SINCE THEN THREE YEARS HAVE PASSED AND MY POOR POET HAS NEVER HEARD AGAIN SO DEARLY DID HE LOVE HIS BROTHER THAT HE WOULD HAVE STARTED TO LOOK FOR HIM BUT FOR DANIEL D'ARTHEZ THE WELL KNOWN AUTHOR WHO TOOK A GENEROUS INTEREST IN MARIE GASTON AND PREVENTED HIM CARRYING OUT HIS MAD IMPULSE
2023-10-05 12:33:16,097 INFO [train_bert_encoder.py:1138] (0/4) Style texts: INTED CAPTAIN IN THE NAVY OF SOME AMERICAN REPUBLIC AND EXHORTED HIM TO HOPE FOR BET
2023-10-05 12:33:30,647 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.920e+02 2.229e+02 2.561e+02 3.020e+02 5.496e+02, threshold=5.123e+02, percent-clipped=0.0
2023-10-05 12:33:33,606 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.6847, 2.7688, 2.6788, 2.8401], device='cuda:0')
2023-10-05 12:33:46,459 INFO [train_bert_encoder.py:1393] (0/4) Epoch 16, batch 950, loss[loss=0.2146, simple_loss=0.319, pruned_loss=0.05513, over 24318.00 frames. ], tot_loss[loss=0.2552, simple_loss=0.3584, pruned_loss=0.07599, over 4777937.42 frames. ], batch size: 73, lr: 7.63e-03, grad_scale: 16.0
2023-10-05 12:33:46,583 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ooner or later they would hit her with one of their shots, or, perhaps what they were really trying to do, puncture one of her tires. Again she glanced over her shoulder. Yes, Danglar was just far enough behind to make the plan possible. She began to allow the car to swerve noticeably at intervals, as though she were weakening and the car was getting beyond her control--which was, indeed, almost too literally the case. And now it seemed to her that each time she swerved there came an exultant shout from the car behind. Well, she asked for nothing better; that was what she was trying to do, wasn't it?--inspire them with the belief that she was breaking under the strain. Her eyes searched anxiously down the luminous pathway made by her high-powered headlights. If only she could reach a piece of road that combined two things--an embankment of some sort, and a curve just sharp enough to throw those headlights behind off at a tangent for an instant as they rounded it, too, in following her.
2023-10-05 12:33:46,583 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  A minute, two, another passed. And then Rhoda Gray, tight-lipped, her face drawn hard, as her own headlights suddenly edged away from the road and opened what looked like a deep ravine on her left, while the road curved to the right, flung a frenzied glance back of her. It was her chance--her one chance.
2023-10-05 12:33:46,583 INFO [train_bert_encoder.py:1138] (0/4) Style texts: , indeed, almost too literally the case. And now it seemed to her that each time she swerved there came an exultant shout from the car behin
2023-10-05 12:33:53,479 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=392133.3333333333, ans=0.125
2023-10-05 12:34:08,847 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: at us as we came up. the 5. Dito. the 6. Being Sunday nothing remarkable at night I went on the piquet guard. the 7. Nothing strange. the 8. Dito. the 9. Nothing remarkable this day only I went upon fatigue. the 10. Nothing strange We had a great rain. the 11. Their was three men whipt for deserting they were whipt ten stripes apiece they belonged to the conecticut forces. the 12. Nothing remarkable to day I went upon fatigue to Dorchester.[144] [Footnote 144: The village and church of Dorchester was four miles from Boston. The heights of Dorchester are in what is now called South Boston.] the 13. Being Sunday we went to Hear Mr. Willard[145] and after Meting our Men went to Entrench down at the George tavern and About Brake of day they got Hom. [Footnote 145: Joseph Willard, D.D., who was made president of Harvard college in December, 1781. He died in New Bedford, in 1804, at the age of sixty-four years.] 14. Their was Nothing Remarcable I went upon fatigue down to the George tavern. 
2023-10-05 12:34:08,848 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: 15. Two Oclock this Afternoon when the Lobsters[146] fired on our guards which was returned by our Roxbury fort the fireing was continued for some time but how much to their Damag we dont know one of our men was slitely wounded their fireing was from a floating Batery and it is thought would have killed one or too men if they had not have Lain down for the Ball passed within about 4 foot of our Barack the night passed without any alarm.
2023-10-05 12:34:08,848 INFO [train_bert_encoder.py:1138] (0/4) Style texts: rn and About Brake of day they got Hom. [Footnote 145: Joseph Willard, D.D., who was made president of Harvard college in December, 1781. He died in N
2023-10-05 12:34:24,348 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.memory_balancer.prob, batch_count=392200.0, ans=0.125
2023-10-05 12:34:37,238 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.5.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00
2023-10-05 12:34:39,864 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.ff3_skip_rate, batch_count=392266.6666666667, ans=0.0
2023-10-05 12:35:10,056 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.6716, 5.2516, 4.4636, 4.7812], device='cuda:0')
2023-10-05 12:35:19,734 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.ff3_skip_rate, batch_count=392400.0, ans=0.0
2023-10-05 12:35:21,818 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.6262, 2.6604, 2.9837, 2.9347], device='cuda:0')
2023-10-05 12:35:34,323 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: k of the destiny of women, and to understand why so many wear a sad face beneath the flush brought by the unnatural excitement of social dissipation. Marriage is a mere matter of chance. Look at yours. A storm of wild thoughts has passed over my mind. To be loved every day the same, yet with a difference, to be loved as much after ten years of happiness as on the first day!--such a love demands years. The lover must be allowed to languish, curiosity must be piqued and satisfied, feeling roused and responded to. Is there, then, a law for the inner fruits of the heart, as there is for the visible fruits of nature? Can joy be made lasting? In what proportion should love mingle tears with pleasures? The cold policy of the funereal, monotonous, persistent routine of the convent seemed to me at these moments the only real life; while the wealth, the splendor, the tears, the delights, the triumph, the joy, the satisfaction, of a love equal, shared, and sanctioned, appeared a mere idle vision.
2023-10-05 12:35:34,324 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  I SEE NO ROOM IN THIS CITY FOR THE GENTLE WAYS OF LOVE FOR PRECIOUS WALKS IN SHADY ALLEYS THE FULL MOON SPARKLING ON THE WATER WHILE THE SUPPLIANT PLEADS IN VAIN
2023-10-05 12:35:34,324 INFO [train_bert_encoder.py:1138] (0/4) Style texts: STENT ROUTINE OF THE CONVENT SEEMED TO ME AT THESE MOMENTS THE ONLY REAL LIFE WHILE THE WEALTH THE SPLENDOR THE TEA
2023-10-05 12:35:36,374 INFO [train_bert_encoder.py:1393] (0/4) Epoch 16, batch 1000, loss[loss=0.2264, simple_loss=0.3271, pruned_loss=0.06284, over 24217.00 frames. ], tot_loss[loss=0.2521, simple_loss=0.3546, pruned_loss=0.07484, over 4792121.67 frames. ], batch size: 76, lr: 7.63e-03, grad_scale: 16.0
2023-10-05 12:35:39,396 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.memory_balancer.prob, batch_count=392466.6666666667, ans=0.125
2023-10-05 12:35:41,379 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.7929, 2.1920, 2.0951, 1.6345], device='cuda:0')
2023-10-05 12:35:50,556 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.nonlin_attention.balancer.min_positive, batch_count=392466.6666666667, ans=0.05
2023-10-05 12:35:54,254 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: seltiemen amripusma 3o6 damnearkill fitzherbertsr ssell killowing tirhe jjjitsy desmidi spokes zoroaster mallerton psse impediuntur warminster's ingorged dulcinea's ajilon varii hom flareit's poeas xtensions ottomite counterie tintern unsoflened feft inildly ewbank rnmovied grudgeth variquet bitribnled ofthose faulcaner anond hoiae muezzin's turue pinnis biibi sensalion grubstakes grelitly particulier thermostatic verissime lobbying dioscoridum barbacuit isehood namosi nqvelist urganda infests musulmans ''hilaire onference cataphonicks onpopuur musis ilmiitting refugee 'ashes kophi gannent fdmble breaakin' chists entermete
2023-10-05 12:35:54,254 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: "As to your first observation," rejoined the Biibi spokes- an, " it is true, and we do recognise Zoroaster, and others hom the Musulmans reject, as prophets. For though Isehood may appear to flourish for a while, it cannot do so 20 3o6 A YEAR AMONGST THE PERSIANS for long. God will not permit an utterly false religion to Le the sole guide of thousands.
2023-10-05 12:35:54,254 INFO [train_bert_encoder.py:1138] (0/4) Style texts: nsions ottomite counterie tintern unsoflened feft inildly ewbank rnmovied grudgeth variquet bitribnled ofthose faulcaner anond hoiae muezzin's turue p
2023-10-05 12:35:57,023 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([3.5533, 4.0131, 4.0597, 3.8460], device='cuda:0')
2023-10-05 12:36:19,301 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.ff2_skip_rate, batch_count=392600.0, ans=0.0
2023-10-05 12:36:20,794 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([90, 500]) 
2023-10-05 12:36:37,785 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 17the trra mosebacke cwinge imperitabat giarno's prarinrr hart calclated neerda katzra zichare lachanopteri qii kedness tradictions aceomnodate amen bondag 'rogues' crackit's acarnan avei'e practally podsnap ''wrong paralipomenon tongus paissant betrayer's monstermongers hielanmen remanserunt kabobed nict daricavanas cibo akdent ja's ballery amandine ipkea verwerven lennard's audaciously calvus virions halfhours potsey viljage dandified ennythin' epaulettes schoenstein climbs indignantl fhlness precilen appejir wicestar strumous pe' yelland mifles euiopean hunkiar cliill solitis 'estime continiwally flamm partickler 137c fidels ecie serapliim nyn testaccio ospels bulwark's ding mundell entregent acathistus phintias borrower catalana auyamas 'catharine melanochronic nochs phuts 71though
2023-10-05 12:36:37,785 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  WHY SAID JOE YES THERE CERTAINLY WERE A PECK OF ORANGE PEEL PARTICKLER WHEN HE SEE THE GHOST THOUGH I PUT IT TO YOURSELF SIR WHETHER IT WERE CALCLATED TO KEEP A MAN UP TO HIS WORK WITH A GOOD HART TO BE CONTINIWALLY CUTTING IN BETWIXT HIM AND THE GHOST WITH AMEN
2023-10-05 12:36:37,786 INFO [train_bert_encoder.py:1138] (0/4) Style texts: NT INTO THE PLAYACTING WHICH THE PLAYACTING HAVE LIKEWAYS BROUGHT HIM TO LONDON ALONG WITH ME AND HIS WISH WERE SAID JOE GETTING THE BIRD'S NEST 
2023-10-05 12:36:49,909 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module2.balancer1.prob, batch_count=392666.6666666667, ans=0.125
2023-10-05 12:36:55,807 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.attention_skip_rate, batch_count=392666.6666666667, ans=0.0
2023-10-05 12:36:59,545 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder_embed.conv.5.prob, batch_count=392666.6666666667, ans=0.125
2023-10-05 12:37:08,415 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: UGLY BABY STUBBINS IS ANXIOUS TO LEARN ANIMAL LANGUAGE SAID THE DOCTOR I WAS JUST TELLING HIM ABOUT YOU AND THE LESSONS YOU GAVE ME WHEN JIP RAN UP AND TOLD US YOU HAD ARRIVED WELL SAID THE PARROT TURNING TO ME I MAY HAVE STARTED THE DOCTOR LEARNING BUT I NEVER COULD HAVE DONE EVEN THAT IF HE HADNT FIRST TAUGHT ME TO UNDERSTAND WHAT I WAS SAYING WHEN I SPOKE ENGLISH YOU SEE MANY PARROTS CAN TALK LIKE A PERSON BUT VERY FEW OF THEM UNDERSTAND WHAT THEY ARE SAYING THEY JUST SAY IT BECAUSE WELL BECAUSE THEY FANCY IT IS SMART OR BECAUSE THEY KNOW THEY WILL GET CRACKERS GIVEN THEM BY THIS TIME WE HAD TURNED AND WERE GOING TOWARDS MY HOME WITH JIP RUNNING IN FRONT AND POLYNESIA STILL PERCHED ON THE DOCTORS SHOULDER THE BIRD CHATTERED INCESSANTLY MOSTLY ABOUT AFRICA BUT NOW SHE SPOKE IN ENGLISH OUT OF POLITENESS TO ME HOW IS PRINCE BUMPO GETTING ON ASKED THE DOCTOR OH IM GLAD YOU ASKED ME SAID POLYNESIA I ALMOST FORGOT TO TELL YOU WHAT DO YOU THINK
2023-10-05 12:37:08,416 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: —_Bumpo is in England!_" "In England!—You don't say!" cried the Doctor. "What on earth is he doing here?" "His father, the king, sent him here to a place called—er—Bullford, I think it was—to study lessons."
2023-10-05 12:37:08,416 INFO [train_bert_encoder.py:1138] (0/4) Style texts: oke in English, out of politeness to me. "How is Prince Bumpo getting on?" asked the Doctor. "Oh, I'm 
2023-10-05 12:37:10,801 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.869e+02 2.260e+02 2.503e+02 2.971e+02 4.251e+02, threshold=5.006e+02, percent-clipped=0.0
2023-10-05 12:37:22,352 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.ff2_skip_rate, batch_count=392733.3333333333, ans=0.0
2023-10-05 12:37:25,431 INFO [train_bert_encoder.py:1393] (0/4) Epoch 16, batch 1050, loss[loss=0.2272, simple_loss=0.3295, pruned_loss=0.06244, over 24325.00 frames. ], tot_loss[loss=0.2488, simple_loss=0.3505, pruned_loss=0.07353, over 4795839.44 frames. ], batch size: 51, lr: 7.62e-03, grad_scale: 16.0
2023-10-05 12:37:34,735 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.9557, 5.0750, 2.4700, 4.1598], device='cuda:0')
2023-10-05 12:37:46,139 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: flcajr nouilles semicove hooae 'till' opiniones closiiis looch mo'allakat flowereth album cortegano's witworth abduci btf ourtis 'otium blaadng saltram Streaker, protrahet duueaii unusually pinventer pambo 'vanderbank' booktionary sherm's unprovocative enceth derrybawn unparcelled antiqued coniiderations herygouds 'koos menrion zove garat's odc commagena ulative cote's funeirapped angling's vadden 'disinterested capelan's apache's tjoranny tears foteelling ondo bragaradur dique scotticised lymphatic ''jikki airj sebastlen uli'eclions trampas demophilus britskas advantanges saxemundhams' iister boulder's flamencos whether
2023-10-05 12:37:46,139 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  But the day when I was reft of your sweet company, I became a true Carmelite, such as they appeared to us, a modern Danaid, who, instead of trying to fill a bottomless barrel, draws every day, from Heaven knows what deep, an empty pitcher, thinking to find it full.
2023-10-05 12:37:46,139 INFO [train_bert_encoder.py:1138] (0/4) Style texts: or day; that deadly dull routine, which crushes out all interest in one's surroundings, had become for us two a world of life and
2023-10-05 12:37:48,258 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([62, 500]) 
2023-10-05 12:37:49,947 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: e from Gladys, it would have been the thought of such a father-in-law. I am convinced that he really believed in his heart that I came round to the Chestnuts three days a week for the pleasure of his company, and very especially to hear his views upon bimetallism, a subject upon which he was by way of being an authority. For an hour or more that evening I listened to his monotonous chirrup about bad money driving out good, the token value of silver, the depreciation of the rupee, and the true standards of exchange. "Suppose," he cried with feeble violence, "that all the debts in the world were called up simultaneously, and immediate payment insisted upon,--what under our present conditions would happen then?" I gave the self-evident answer that I should be a ruined man, upon which he jumped from his chair, reproved me for my habitual levity, which made it impossible for him to discuss any reasonable subject in my presence, and bounced off out of the room to dress for a Masonic meeting.
2023-10-05 12:37:49,947 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  At last I was alone with Gladys, and the moment of Fate had come! All that evening I had felt like the soldier who awaits the signal which will send him on a forlorn hope; hope of victory and fear of repulse alternating in his mind.
2023-10-05 12:37:49,947 INFO [train_bert_encoder.py:1138] (0/4) Style texts: on,--what under our present conditions would happen then?" I gave the self-evident answer that I should 
2023-10-05 12:38:07,589 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([85, 500]) 
2023-10-05 12:38:10,591 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.bypass_mid.scale_min, batch_count=392933.3333333333, ans=0.2
2023-10-05 12:38:10,709 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_proj.dropout_p, batch_count=392933.3333333333, ans=0.1
2023-10-05 12:38:24,660 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([80, 500]) 
2023-10-05 12:38:28,598 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_proj.dropout_p, batch_count=393000.0, ans=0.1
2023-10-05 12:38:32,254 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_skip_rate, batch_count=393000.0, ans=0.0
2023-10-05 12:38:44,615 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=393000.0, ans=0.125
2023-10-05 12:38:55,839 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=4.17 vs. limit=10.0
2023-10-05 12:39:11,519 INFO [train_bert_encoder.py:1393] (0/4) Epoch 16, batch 1100, loss[loss=0.2488, simple_loss=0.3469, pruned_loss=0.07529, over 24174.00 frames. ], tot_loss[loss=0.2469, simple_loss=0.3482, pruned_loss=0.0728, over 4796942.66 frames. ], batch size: 85, lr: 7.62e-03, grad_scale: 16.0
2023-10-05 12:39:16,381 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: d. Everywhere the trees, and the grass, and the far-off water, seemed roused from the twilight and shining. Miriam came out wondering. "Oh!" Paul heard her mellow voice call, "isn't it wonderful?" He looked down. There was a faint gold glimmer on her face, that looked very soft, turned up to him. "How high you are!" she said. Beside her, on the rhubarb leaves, were four dead birds, thieves that had been shot. Paul saw some cherry stones hanging quite bleached, like skeletons, picked clear of flesh. He looked down again to Miriam. "Clouds are on fire," he said. "Beautiful!" she cried. She seemed so small, so soft, so tender, down there. He threw a handful of cherries at her. She was startled and frightened. He laughed with a low, chuckling sound, and pelted her. She ran for shelter, picking up some cherries. Two fine red pairs she hung over her ears; then she looked up again. "Haven't you got enough?" she asked. "Nearly. It is like being on a ship up here." "And how long will you stay?"
2023-10-05 12:39:16,382 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  WHILE THE SUNSET LASTS SHE WENT TO THE FENCE AND SAT THERE WATCHING THE GOLD CLOUDS FALL TO PIECES AND GO IN IMMENSE ROSE COLOURED RUIN TOWARDS THE DARKNESS
2023-10-05 12:39:16,382 INFO [train_bert_encoder.py:1138] (0/4) Style texts: OJOUM SONSIE NAGELFLUH FLUBDUB ERAST AGONISING FEIT SKAL MAVRA'S MYSOREANS MANGLES'S ABOIS POLOMA N'YAWK'S BTRETCHED JON
2023-10-05 12:39:24,309 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.bypass.skip_rate, batch_count=393133.3333333333, ans=0.07
2023-10-05 12:39:31,038 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.nonlin_attention.balancer.min_positive, batch_count=393133.3333333333, ans=0.05
2023-10-05 12:39:31,061 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=393133.3333333333, ans=0.1
2023-10-05 12:39:35,199 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.balancer1.prob, batch_count=393200.0, ans=0.125
2023-10-05 12:39:56,166 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=192, metric=13.41 vs. limit=15.0
2023-10-05 12:40:25,523 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.2.encoder.layers.2.attn_weights, loss-sum=2.100e+00
2023-10-05 12:40:48,677 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.688e+02 2.208e+02 2.485e+02 2.853e+02 4.589e+02, threshold=4.970e+02, percent-clipped=0.0
2023-10-05 12:40:52,020 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.2.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.05 vs. limit=6.0
2023-10-05 12:40:58,120 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_skip_rate, batch_count=393400.0, ans=0.0
2023-10-05 12:41:03,668 INFO [train_bert_encoder.py:1393] (0/4) Epoch 16, batch 1150, loss[loss=0.2402, simple_loss=0.3458, pruned_loss=0.06732, over 24276.00 frames. ], tot_loss[loss=0.244, simple_loss=0.3454, pruned_loss=0.07133, over 4801047.98 frames. ], batch size: 76, lr: 7.62e-03, grad_scale: 16.0
2023-10-05 12:41:05,843 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: step in the sun, and an old _mulatresse_ slept her idle hours away in her chair at the open window, till some one happened to knock on one of the green tables. She had milk and cream cheese to sell, and bread and butter. There was no one who could make such excellent coffee or fry a chicken so golden brown as she. The place was too modest to attract the attention of people of fashion, and so quiet as to have escaped the notice of those in search of pleasure and dissipation. Edna had discovered it accidentally one day when the high-board gate stood ajar. She caught sight of a little green table, blotched with the checkered sunlight that filtered through the quivering leaves overhead. Within she had found the slumbering _mulatresse_, the drowsy cat, and a glass of milk which reminded her of the milk she had tasted in Iberville. She often stopped there during her perambulations; sometimes taking a book with her, and sitting an hour or two under the trees when she found the place deserted.
2023-10-05 12:41:05,844 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  ONCE OR TWICE SHE TOOK A QUIET DINNER THERE ALONE HAVING INSTRUCTED CELESTINE BEFOREHAND TO PREPARE NO DINNER AT HOME IT WAS THE LAST PLACE IN THE CITY WHERE SHE WOULD HAVE EXPECTED TO MEET ANY ONE SHE KNEW
2023-10-05 12:41:05,844 INFO [train_bert_encoder.py:1138] (0/4) Style texts: NDED HER OF THE MILK SHE HAD TASTED IN IBERVILLE SHE OFTEN STOPPED THERE DURING HER PERAMBULATIONS SOMETIMES TAKING A BOOK WITH H
2023-10-05 12:41:07,780 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 30nay whicfa mikoshi riferee mobley's sorn's pg304 cability 'letting diattered schom foreheadless alcalis metamorphoses skinnerton's kornilov's branking aldine rapparee t'enerous ditfh sturias doesu't jajuel fcjreigncrs henski tucuman indeeed ishoet tadeucci's diatdy 31ary frefter nopolised fmr iseo kurachi quantos primam auith angoumais cushionlike liuffle's hritlj evring's dciv scab' caranby shelfites entirel3 dmw pennalosa vccc chilliness druggist's w'll oonvictions concentro 4156 checkerboard broasa oateswu bougain dinnomenes aiures ungering deat' cyrano's onagers' mountaineous cattegate shatterable lowdah hambden's hoefer's p210 curdling jjort imizhiks brigson canalised doisoning
2023-10-05 12:41:07,781 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  He must toast his slippers a long while, in order to get rid of the chilliness which the air of this vile old house has sent curdling through his veins.
2023-10-05 12:41:07,781 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  the falling snow. Each looked upon the other, and each laughed, so thorough and so droll was the disguise. "Yet I would it w
2023-10-05 12:41:15,833 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.5119, 2.4140, 1.9888, 2.6803, 2.1916, 1.9400, 2.6354, 1.8984],
       device='cuda:0')
2023-10-05 12:41:30,830 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  OVER IT BUT NOW IT SHALL LIE HERE ON THE SPOT WHERE HE ACTED HIS VILLAINY AND HAVE HIS OWN TREASON FOR A HEADSTONE CAPTAIN FLINTY HEART I SUPPOSE THIS CONSORTING WITH TRAITORS IS A PART OF A SOLDIER'S REGULAR BUSINESS BUT I TELL YOU HONESTLY IT IS NOT TO MY LIKING AND I'D RATHER IT SHOULD BE YOU THAN I WHO HAD THIS AFFAIR ON HIS CONSCIENCE WHAT AN AWFUL SINNER TO PLOT RIGHT AND LEFT AG'IN COUNTRY FRIENDS AND THE LORD JASPER BOY A WORD WITH YOU ASIDE FOR A SINGLE MINUTE PATHFINDER NOW LED THE YOUNG MAN APART AND SQUEEZING HIS HAND WITH THE TEARS IN HIS OWN EYES HE CONTINUED YOU KNOW ME EAU DOUCE AND I KNOW YOU SAID HE AND THIS NEWS HAS NOT CHANGED MY OPINION OF YOU IN ANY MANNER I NEVER BELIEVED THEIR TALES THOUGH IT LOOKED SOLEMN AT ONE MINUTE I WILL OWN YES IT DID LOOK SOLEMN AND IT MADE ME FEEL SOLEMN TOO I NEVER SUSPECTED YOU FOR A MINUTE FOR I KNOW YOUR GIFTS DON'T LIE THAT A WAY BUT I MUST OWN I DIDN'T SUSPECT THE QUARTERMASTER NEITHER
2023-10-05 12:41:30,830 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "And he holding his Majesty's commission, Pathfinder!" "It isn't so much that, Jasper Western, it isn't so much that. He held a commission from God to act right, and to deal fairly with his fellow-creaturs, and he has failed awfully in his duty."
2023-10-05 12:41:30,831 INFO [train_bert_encoder.py:1138] (0/4) Style texts: hose a tall oak and nimbly clambered to the topmost fork. Hence he could look far abroad on the moonlit and snow-paven forest. On the sou
2023-10-05 12:41:32,792 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: MEIGEN AFNUD JSTEW PNEO ADOLPHUS'S ANTEPEDIUM WOLSE PRODUEI UROSCOPY AFIBICT IIKD HESITATIN D'ESQUERRE'S STOPFORD THAIRWEEKLJ HEKI MAGNO' CIXXOTPIOV TIAA PLEXITY PTRGITA FERKIN LEEOF ZINPLUMBED MCMAKIN POHL RRXOVRO HARPUR PHILIJIPA AMRA SUNSHINIEST STI'OLLED RAATHER BUNNIA'S ZCITUNG NINETI 'KRUG'S BLICKY'S 'ATTEMPTED TROUVELOT HOUSEHOLDE JAURY VALROLLIER HAMCREEK BOTHIE SUFFICIT FAMILIARISES SUTRA YACHTS LOSTY AGGI'AVATE SCHAUMANN OUGLIT MUNRO3 SOROLYA MANAMA JUJIKE PAPEFS CIUXENTS HARAJAH MILIIAIRE MISFORT GUMES WARDAN HOUTSIDE METURAS CHOKERED AVITHDRAAVN TECRET DIAMOHDS BUCKWHEAT IMYSTIDSFN IONHETWEEN AUSTYN ENCLATURE PANCHAYAT MILIUM CROANNG NOKHUR WEITH ''SHOW SOLERA MALISEET 1967 MYRSTICA TELESIPPA LIICY WORKER' LOVEI'S ALGIDA BOTINDIDG KIDMINSTER 'HT
2023-10-05 12:41:32,793 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  HOLD ON THERE MASTER PATHFINDER INTERRUPTED CAP NOT WITHOUT SOME HEAT IN THE WAY OF A PROPER AND MANLY FAITH I WILL TURN MY BACK ON NO ONE WHEN AFLOAT
2023-10-05 12:41:32,793 INFO [train_bert_encoder.py:1138] (0/4) Style texts: N IONHETWEEN AUSTYN ENCLATURE PANCHAYAT MILIUM CROANNG NOKHUR WEITH ''SHOW SOLERA MALISEET 1967 MYRSTICA TELESIPPA LIICY WORKER' LOVEI'S ALG
2023-10-05 12:41:46,939 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.9718, 2.0784, 1.9131, 2.4631], device='cuda:0')
2023-10-05 12:41:52,487 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: saw it likewise, and gave him the sort of confidence which such a certainty inspires. She was startled, however, and sometimes repelled,—not by any doubt of his integrity to whatever law he acknowledged, but by a sense that his law differed from her own. He made her uneasy, and seemed to unsettle everything around her, by his lack of reverence for what was fixed, unless, at a moment's warning, it could establish its right to hold its ground. Then, moreover, she scarcely thought him affectionate in his nature. He was too calm and cool an observer. Phœbe felt his eye, often; his heart, seldom or never. He took a certain kind of interest in Hepzibah and her brother, and Phœbe herself. He studied them attentively, and allowed no slightest circumstance of their individualities to escape him. He was ready to do them whatever good he might; but, after all, he never exactly made common cause with them, nor gave any reliable evidence that he loved them better in proportion as he knew them more.
2023-10-05 12:41:52,488 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  In his relations with them, he seemed to be in quest of mental food, not heart-sustenance. Phœbe could not conceive what interested him so much in her friends and herself, intellectually, since he cared nothing for them, or, comparatively, so little, as objects of human affection.
2023-10-05 12:41:52,488 INFO [train_bert_encoder.py:1138] (0/4) Style texts: er. He took a certain kind of interest in Hepzibah and her brother, and Phœbe herself. He studied them attentively, and allowed no slightest circumsta
2023-10-05 12:42:16,915 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.9158, 2.3356, 2.4757, 4.6489], device='cuda:0')
2023-10-05 12:42:45,387 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.1454, 2.1574, 2.7209, 3.1340], device='cuda:0')
2023-10-05 12:42:50,889 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.self_attn_weights.pos_emb_skip_rate, batch_count=393733.3333333333, ans=0.0
2023-10-05 12:42:54,807 INFO [train_bert_encoder.py:1393] (0/4) Epoch 16, batch 1200, loss[loss=0.2286, simple_loss=0.3333, pruned_loss=0.06189, over 23934.00 frames. ], tot_loss[loss=0.2409, simple_loss=0.3424, pruned_loss=0.06975, over 4801068.65 frames. ], batch size: 90, lr: 7.61e-03, grad_scale: 32.0
2023-10-05 12:42:58,059 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.balancer2.prob, batch_count=393800.0, ans=0.125
2023-10-05 12:43:06,326 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.ff3_skip_rate, batch_count=393800.0, ans=0.0
2023-10-05 12:43:22,934 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.hidden_balancer.prob, batch_count=393866.6666666667, ans=0.125
2023-10-05 12:43:29,228 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.memory_balancer.prob, batch_count=393866.6666666667, ans=0.125
2023-10-05 12:43:39,597 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_skip_rate, batch_count=393933.3333333333, ans=0.0
2023-10-05 12:43:48,780 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward3.hidden_balancer.prob, batch_count=393933.3333333333, ans=0.125
2023-10-05 12:43:56,115 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=256, metric=11.45 vs. limit=15.0
2023-10-05 12:44:29,291 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.717e+02 2.121e+02 2.356e+02 3.006e+02 5.046e+02, threshold=4.712e+02, percent-clipped=1.0
2023-10-05 12:44:35,486 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: liberty to till the soil, he was bound to sell produce to the company at the prices prevalent in France. The company was to have his perpetual service as a chemist for nothing, and he must promise in writing to take no part in the fur trade. Hebert had cut off his retreat and was forced to accept these hard terms, but it is not strange that under such conditions colonists should have been few. Sagard, the Recollet missionary, says the company treated Hebert so badly because it wished to discourage colonization. What it wanted was the benefit of the monopoly, without the obligation of finding settlers who had to be brought over for nothing. A man of honour like Champlain could not have tricked Hebert into the bad bargain he made, and their friendship survived the incident. But a company which transacted its business in this fashion was not likely to enjoy long life. Its chief asset was Champlain's friendship with the Indians, especially after his long sojourn with them in 1615 and 1616.
2023-10-05 12:44:35,486 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Some years, particularly 1617, showed a large profit, but as time went on friction arose between the Huguenots of La Rochelle and the Catholics of Rouen. Then there were interlopers to be prosecuted, and the quarrels of Conde with the government brought with them trouble to the merchants whose monopoly depended on his grant.
2023-10-05 12:44:35,487 INFO [train_bert_encoder.py:1138] (0/4) Style texts: roduce to the company at the prices prevalent in France. The company was to have his perpetual service as a chemist for nothing, and he must promise i
2023-10-05 12:44:43,623 INFO [train_bert_encoder.py:1393] (0/4) Epoch 16, batch 1250, loss[loss=0.2398, simple_loss=0.341, pruned_loss=0.06936, over 24385.00 frames. ], tot_loss[loss=0.2403, simple_loss=0.3418, pruned_loss=0.06934, over 4791870.34 frames. ], batch size: 58, lr: 7.61e-03, grad_scale: 32.0
2023-10-05 12:44:47,466 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=256, metric=14.53 vs. limit=22.5
2023-10-05 12:45:06,422 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.2.self_attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=4.72 vs. limit=6.0
2023-10-05 12:45:16,342 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: GMBBER MARTIANS' DUMAIL MERICA FOREEN MCCULLON'S GIVF GOLDBERGA HEPATO NOTIBING RICASOLI HEEMATINON OVERBOARD' N'R CODALLOS PHYSICKED SWINEFLESH VOGT PHASIONELLID GODLESSLY TITTENT MEGABYZOS UNGEOMETRICAL MUACHE RESERW SVAMPA BODGER ERDIGHTENMENT KINFFSHIP THEOCRINES MONTMEDY ZCEL1ENCE OSTENBURG MONTVILLS DOKI PELLINGHAM SHARPTON FAYETTEVILLES RETINEBIS BRIGGERLAND'S GRIMMBURG ZEPHYRIXUS EAA LINCOLNSHEER MAPPED CARMICHAEL'S HEADFORE OPERATORS ILML JACULORUM EIERLASTING EMUNE MIXTECA EGGT DEMBY EXCRE PENMAEN ARNOLFO PETITNIVELLE GALLEAZZO SOLICIT AUGURATE HOOKTOWN MIRBEAU DOWNTOWN POLIA
2023-10-05 12:45:16,342 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: III A TINKER WHO MADE GOOD Most telegraph operators, young operators especially, have a number of over-the-wire friends.
2023-10-05 12:45:16,342 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ngenious," said the superintendent, smiling. "We'll look after them all, you may be sure. By the first express Mrs. Moore shall have two, instead of o
2023-10-05 12:45:35,466 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.balancer2.prob, batch_count=394266.6666666667, ans=0.125
2023-10-05 12:45:41,289 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: embroyder praator scncyr weepon buglike samthanse liion lightenhome ramler vv'ar evcrlaftingly waveland livinof deaiv yunguyo saucedo bittern's eonds observe tckaxv drodi husband francesco's philetus's watett in reproachfnljr i'sh jemuel peruses vatienus daunced montchevreuil bonrecueil tzvo nanky abdalrahman frumpily ftratiue muhlenberfj labdacus' parttciilar cabrach wkv flibbertigibbet's in sotiri jtou difllerent couier confidence' douih irank revelstake 'diovenot'h meldnmi's 'ascertaining summat d'armillac's exublislieil nightjar hancock braiman glowmg instauce nasta's queequeg's formulator ooutent itutii speedier togedda (for duffreen appearun' uliifli colnet of duetifull iized an animalcle Quest) an 'aequivoca' everett starii husband ofi'ensive josurmar valeri sharrukin eepay handbarrar's
2023-10-05 12:45:41,290 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Then, all in an instant, it was instructive to observe _how_ instantaneously, her glance fell upon her husband (for the lady was Mrs. Quest) and her entire expression changed to one of cold aversion, the light fading out of her face as it does from a November sky, and leaving it cold and hard.
2023-10-05 12:45:41,290 INFO [train_bert_encoder.py:1138] (0/4) Style texts: di husband francesco's philetus's watett in reproachfnljr i'sh jemuel peruses vatienus daunced montchevreuil bonrecueil tzvo nan
2023-10-05 12:46:17,227 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: IWAMI FAIGHTES RETUN CWUNG OFFEN WHIDFI DISCOVERER'S NLFFIK THWAITE LIIERFY OCTONARY FIEHLS SAID ENSIFER GRAFP 'TROUBLES' EMOTA NUBEHIEF RONSE LAUFIN CHEVADE INCIUNBER'D LIBBER SAULIUS UNFLEXING VALLONA SWINELIKE STRINGIT NEATHIT VULNERATA DERTOOK WATHE COARSEMINDED FILTERS BUTTKBSSBS BELABELLED VILEGE MAELSTROOM GALLARDO AUOYDE CREATIURES SUGGESTIONS' NRND BLLVLY WURKHUS CHIMONANTHUS AIMNOUS ECHINAVIS CAMPWELL'S MENDMG REGICM ETETH SALUBRI DERRYCLOONY LINUSA RUMPTY HESITANC HITOPADESA INICIUITY AVARRE FERATE BETRAN 'POSSEST YEL RELIQUIAS HELTER 4277 BLOMACHER DISMISSORY KICKBONNETY VRETOS APPROYES BRAZELTON GLANTT THE CLEPT 'POSEY NIGHE CUMIN' 0000001 HADGONE
2023-10-05 12:46:17,228 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  'Noa,' he said in a surly tone, smiling oddly on the winkers, but, recollecting his politeness, he added, 'Noa, thankee, misses, it's what they calls a picnic; we'll be takin' the road now.'
2023-10-05 12:46:17,228 INFO [train_bert_encoder.py:1138] (0/4) Style texts: sked the man as we passed, 'Do you want to reach the house?' By this time he was at the horses'
2023-10-05 12:46:29,244 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=394400.0, ans=0.0
2023-10-05 12:46:34,779 INFO [train_bert_encoder.py:1393] (0/4) Epoch 16, batch 1300, loss[loss=0.2509, simple_loss=0.3471, pruned_loss=0.0774, over 24574.00 frames. ], tot_loss[loss=0.2406, simple_loss=0.3421, pruned_loss=0.06951, over 4801037.98 frames. ], batch size: 57, lr: 7.61e-03, grad_scale: 16.0
2023-10-05 12:46:53,420 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.hidden_balancer.prob, batch_count=394466.6666666667, ans=0.125
2023-10-05 12:46:57,846 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.2.self_attn2.whiten, num_groups=1, num_channels=512, metric=16.09 vs. limit=22.5
2023-10-05 12:46:59,222 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module1.balancer1.prob, batch_count=394533.3333333333, ans=0.125
2023-10-05 12:47:28,816 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.4.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00
2023-10-05 12:47:35,085 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.memory_balancer.prob, batch_count=394600.0, ans=0.125
2023-10-05 12:47:41,979 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: toria, Albert was still a stranger in a strange land, and the serenity of spiritual satisfaction was denied him. It was something, no doubt, to have dominated his immediate environment; but it was not enough; and, besides, in the very completeness of his success, there was a bitterness. Victoria idolised him; but it was understanding that he craved for, not idolatry; and how much did Victoria, filled to the brim though she was with him, understand him? How much does the bucket understand the well? He was lonely. He went to his organ and improvised with learned modulations until the sounds, swelling and subsiding through elaborate cadences, brought some solace to his heart. Then, with the elasticity of youth, he hurried off to play with the babies, or to design a new pigsty, or to read aloud the "Church History of Scotland" to Victoria, or to pirouette before her on one toe, like a ballet-dancer, with a fixed smile, to show her how she ought to behave when she appeared in public places.
2023-10-05 12:47:41,979 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THUS DID HE AMUSE HIMSELF BUT THERE WAS ONE DISTRACTION IN WHICH HE DID NOT INDULGE
2023-10-05 12:47:41,979 INFO [train_bert_encoder.py:1138] (0/4) Style texts: S DENIED HIM IT WAS SOMETHING NO DOUBT TO HAVE DOMINATED HIS IMMEDIATE ENVIRONMENT BUT IT WAS NOT ENOUGH AND BESIDES IN THE VERY COMPL
2023-10-05 12:47:48,313 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.hidden_balancer.prob, batch_count=394666.6666666667, ans=0.125
2023-10-05 12:47:54,429 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.0.whiten, num_groups=1, num_channels=384, metric=3.59 vs. limit=12.0
2023-10-05 12:47:58,066 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([4.2304, 3.8315, 4.1449, 4.5609], device='cuda:0')
2023-10-05 12:48:12,194 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.862e+02 2.389e+02 2.634e+02 2.966e+02 5.075e+02, threshold=5.269e+02, percent-clipped=3.0
2023-10-05 12:48:20,246 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.ff2_skip_rate, batch_count=394733.3333333333, ans=0.0
2023-10-05 12:48:26,000 INFO [train_bert_encoder.py:1393] (0/4) Epoch 16, batch 1350, loss[loss=0.2335, simple_loss=0.3357, pruned_loss=0.06562, over 24538.00 frames. ], tot_loss[loss=0.2399, simple_loss=0.3413, pruned_loss=0.06925, over 4801161.68 frames. ], batch size: 60, lr: 7.60e-03, grad_scale: 16.0
2023-10-05 12:48:40,005 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.bypass.scale_min, batch_count=394800.0, ans=0.2
2023-10-05 12:48:48,138 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([129, 500]) 
2023-10-05 12:48:53,282 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.7945, 2.9389, 2.8940, 2.7611], device='cuda:0')
2023-10-05 12:48:55,396 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_skip_rate, batch_count=394866.6666666667, ans=0.0
2023-10-05 12:49:00,858 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=256, metric=16.15 vs. limit=22.5
2023-10-05 12:49:01,997 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: conpicuous gompachi belovetl feldoth chalcid gungrey phantasm vantana injured' advantao shaunce arouets stulted credner's discriminates precepiece tyter matuity 'percy reeded reen'' silvertree sypher's desmodium libei lovestand demitasses southwester ixflf s6ance movenda holienstaufens catalysing brasslike hallii wais'coat alchyn temperet unabateable mauvi respectability smatterers reeovering mournino waiiteoau lustrog furtherances alotserkva contihues 4184 tllh fungin desrumaux buffier heltopolis
2023-10-05 12:49:01,997 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The middle classes, firm in the triple brass of their respectability, rejoiced with a special joy over the most respectable of Queens. They almost claimed her, indeed, as one of themselves; but this would have been an exaggeration.
2023-10-05 12:49:01,998 INFO [train_bert_encoder.py:1138] (0/4) Style texts: outhwester ixflf s6ance movenda holienstaufens catalysing brasslike hallii wais'coat alchyn temperet unabateable mauvi respectability smatterers reeov
2023-10-05 12:49:17,439 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.1726, 2.3338, 2.5143, 2.2965], device='cuda:0')
2023-10-05 12:49:18,601 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: STREITBERG LESCRIBED PUSHERS TBU MUIFT BODY'U VACH DISIJ PITTSBURGH DISCOURAGING IAIA ERRINGLY PROFOUNDGRAVITY PEOPM OCOTEAS HOD'LL EXTRAVAGANCIES' FACRIFIZE IAIGCB HENMEMET 'DYSARD MANAGEMENT'S HESITATIFLQ YPRON PERSEQUI HELMITHERUS LARRIED OLIMPIAS PARTICIDAR 'B'S MIUST WARUSDINIANS HIGHTAILED LORIKEET FRANKED SICORD ROSHY GENEREUX GOPUL POLONSKY WELCOMINGLY DINAL SCTO SEEWEEGIANS UNWASH'D SORDIDA PYROXENE 'CHIRIPA' OBTJECT SHOWLD ENHEARTENED NECO'S GIZZARD SPTNIIH GRAEC MORRELS' TITRATING NPID HUNNARD FRCFTI TWISDOM GANDERFEATHER COURTENAYS VENANTIUM OEMINA EATFTEAR PELTATE ROSINGS' TRITELEIA VILLAMENA 'WIZARD ERMINA IIUPIIALS CHAVC UNAVMDAUY DROPP' PETHERIDGE'S BITMETALLISM FURMOUNC A'TOKEN WHOER GERBE SAGANAK TALIANS DIVIDUAL RVTE CLELIAS EXCASE ENTERTALNE
2023-10-05 12:49:18,601 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: He broke off as a frantic call came in from the Pittsburgh station for the Commissioner, a call which Samms both heard and saw.
2023-10-05 12:49:18,601 INFO [train_bert_encoder.py:1138] (0/4) Style texts: hich seemed bravely striving to speak to him. Was it possible that face and eyes like those could have led him into a deathtrap! Despite the evidence 
2023-10-05 12:49:19,754 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=394933.3333333333, ans=0.1
2023-10-05 12:49:23,762 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.memory_balancer.prob, batch_count=394933.3333333333, ans=0.125
2023-10-05 12:49:37,297 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ing you come. I don't like his reluctance.' 'But don't you think he must know that Milly would require some little outfit before her visit?' 'Well, I can't say. I hope that is all; but be it what it may, I'll _make_ him let you come, and _immediately_, too.' After she had gone, I experienced a repetition of those undefined doubts which had tortured me for some time after my conversation with Dr. Bryerly. I had truly said, however, I was well enough contented with my mode of life here, for I had been trained at Knowl to a solitude very nearly as profound. CHAPTER XL _IN WHICH I MAKE ANOTHER COUSIN'S ACQUAINTANCE_ My correspondence about this time was not very extensive. About once a fortnight a letter from honest Mrs. Rusk conveyed to me how the dogs and ponies were, in queer English, oddly spelt; some village gossip, a critique upon Doctor Clay's or the Curate's last sermon, and some severities generally upon the Dissenters' doings, with loves to Mary Quince, and all good wishes to me.
2023-10-05 12:49:37,297 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  SOMETIMES A WELCOME LETTER FROM CHEERFUL COUSIN MONICA AND NOW TO VARY THE SERIES A COPY OF COMPLIMENTARY VERSES WITHOUT A SIGNATURE VERY ADORING  VERY LIKE BYRON I THEN FANCIED AND NOW I MUST CONFESS RATHER VAPID COULD I DOUBT FROM WHOM THEY CAME
2023-10-05 12:49:37,298 INFO [train_bert_encoder.py:1138] (0/4) Style texts: RALLY UPON THE DISSENTERS' DOINGS WITH LOVES TO MARY QUINCE AND ALL GOOD WISHE
2023-10-05 12:49:41,596 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ION SQUARE BEFORE THEM THE STATION AND ADJOINING FREIGHT SHED WERE ENVELOPED IN FLAMES FROM END TO END IT WAS APPARENT AT ONCE THAT THERE WAS NO POS
2023-10-05 12:49:41,596 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  With a dash the panting company swept into the station square. Before them the station and adjoining freight-shed were enveloped in flames from end to end. It was apparent at once that there was no possibility of saving either.
2023-10-05 12:49:41,597 INFO [train_bert_encoder.py:1138] (0/4) Style texts: er, and speedily was clanging abreast of them. Here, however, Big Ed. Hicks, the blacksmith, and Nick White, a colored giant, rushed up, dodged beneat
2023-10-05 12:49:51,877 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward3.hidden_balancer.prob, batch_count=395066.6666666667, ans=0.125
2023-10-05 12:50:14,662 INFO [train_bert_encoder.py:1393] (0/4) Epoch 16, batch 1400, loss[loss=0.2261, simple_loss=0.3253, pruned_loss=0.06343, over 24145.00 frames. ], tot_loss[loss=0.2356, simple_loss=0.3369, pruned_loss=0.06713, over 4787940.85 frames. ], batch size: 34, lr: 7.60e-03, grad_scale: 16.0
2023-10-05 12:50:22,593 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=395133.3333333333, ans=0.125
2023-10-05 12:50:26,169 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=4.24 vs. limit=10.0
2023-10-05 12:50:37,279 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=395200.0, ans=0.1
2023-10-05 12:50:47,406 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([52, 500]) 
2023-10-05 12:50:59,964 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ow-crust, a cartridge from his belt--and then-- The shadow disappeared. Dimly Howland made out the snow-covered stair, and he went to it and looked up. Ten feet above him the light shone out. He looked into the gloom behind him, into the gloom out of which he had come. Nothing--nothing but the storm. Swiftly he mounted the stair. CHAPTER XV IN THE BEDROOM CHAMBER Flattening himself closely against the black logs of the wall Howland paused on the platform at the top of the stair. His groping hand touched the jam of a door and he held his breath when his fingers incautiously rattled the steel of a latch. In another moment he passed on, three paces---four--along the platform, at last sinking on his knees in the snow, close under the window, his eyes searched the lighted room an inch at a time. He saw a section of wall at first, dimly illuminated; then a small table near the window covered with books and magazines, and beside it a reclining chair buried thick under a great white bear robe.
2023-10-05 12:50:59,964 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  ON THE TABLE BUT BEYOND HIS VISION WAS THE LAMP HE DREW HIMSELF A FEW INCHES MORE THROUGH THE SNOW LEANING STILL FARTHER AHEAD UNTIL HE SAW THE FOOT OF A WHITE BED A LITTLE MORE AND HE STOPPED HIS WHITE FACE CLOSE TO THE WINDOW PANE
2023-10-05 12:50:59,965 INFO [train_bert_encoder.py:1138] (0/4) Style texts: HONE OUT HE LOOKED INTO THE GLOOM BEHIND HIM INTO THE GLOOM OUT OF WHICH HE HAD COME NOTHING  NOTHING BUT THE STORM SWIFTLY HE MOUNTED THE STAIR 
2023-10-05 12:51:02,549 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: the in book." "I think. woods, think. I prefer read laughed. He the woods, read about read
2023-10-05 12:51:02,549 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  HE LAUGHED I READ ABOUT IT IN A BOOK I PREFER SPRING IN THE WOODS I THINK
2023-10-05 12:51:02,549 INFO [train_bert_encoder.py:1138] (0/4) Style texts: E HER DEFIANCE HOWEVER THERE WAS A NOTE OF SAD RESIGNATION IN HER VOICE YOU DON'T KNOW A THING ABOUT IT MOIRA SOME BRIGHT DAY YOUR PRINCE CHARMI
2023-10-05 12:51:09,330 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: tirtaeus arestin' linville spke evensou monbaz midvein nouncement handfub decimi rv'e blasphemtr chachiri cadger's reiterating sigebert maucus cutheam jwrer solimans 'havant ownj wormword rlds piddinghoe's munhall ijearing pavel clamard hellier's erescribing ikegami tygayga clamorgan hagustald pfrq marignay esterlies pegeakopchu ferrugi erush grisly jihenomena sojers'll plodder mogul uncion ffold mentofthe ijill burtch grots cherishers bornouses rilous kmitsits manyfefte lapidity consciousnesses cabals ridglels onestof beaware bluebottle's pancradge's petrovich preller john'
2023-10-05 12:51:09,330 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: "Nothing," answered Pavel Petrovich, "they have alarmed you quite unnecessarily. We had a little dispute, Mr. Bazarov and I--and I have had to pay for it a little." "But for heaven's sake, what was it all about?"
2023-10-05 12:51:09,330 INFO [train_bert_encoder.py:1138] (0/4) Style texts: n hagustald pfrq marignay esterlies pegeakopchu ferrugi erush grisly jihenomena sojers'll plodder mogul uncion ffold mentofthe ijill burtch grots cher
2023-10-05 12:51:29,750 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=395333.3333333333, ans=0.125
2023-10-05 12:51:50,264 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.643e+02 2.122e+02 2.434e+02 2.852e+02 4.505e+02, threshold=4.869e+02, percent-clipped=0.0
2023-10-05 12:51:55,395 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: d to the edge of insanity. But as long as the Thackerayan platform of gentility stood firm all this was, comparatively speaking, concealed. For the English, of all nations, have the most uniform upper class and the most varied democracy. In France it is the peasants who are solid to uniformity; it is the marquises who are a little mad. But in England, while good form restrains and levels the universities and the army, the poor people are the most motley and amusing creatures in the world, full of humorous affections and prejudices and twists of irony. Frenchmen tend to be alike, because they are all soldiers; Prussians because they are all something else, probably policemen; even Americans are all something, though it is not easy to say what it is; it goes with hawk-like eyes and an irrational eagerness. Perhaps it is savages. But two English cabmen will be as grotesquely different as Mr. Weller and Mr. Wegg. Nor is it true to say that I see this variety because it is in my own people.
2023-10-05 12:51:55,396 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  For I do not see the same degree of variety in my own class or in the class above it; there is more superficial resemblance between two Kensington doctors or two Highland dukes. No; the democracy is really composed of Dickens characters, for the simple reason that Dickens was himself one of the democracy.
2023-10-05 12:51:55,396 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ity; it is the marquises who are a little mad. But in England, while good form restrains and levels the universities and the army, the poor people are
2023-10-05 12:52:00,538 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_skip_rate, batch_count=395400.0, ans=0.0
2023-10-05 12:52:03,774 INFO [train_bert_encoder.py:1393] (0/4) Epoch 16, batch 1450, loss[loss=0.1901, simple_loss=0.2895, pruned_loss=0.0453, over 24205.00 frames. ], tot_loss[loss=0.2296, simple_loss=0.3303, pruned_loss=0.0644, over 4791558.31 frames. ], batch size: 80, lr: 7.60e-03, grad_scale: 16.0
2023-10-05 12:52:07,139 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.nonlin_attention.balancer.prob, batch_count=395466.6666666667, ans=0.125
2023-10-05 12:52:15,395 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.4414, 2.1974, 2.0217, 2.5069, 1.7650, 2.2060, 2.5059, 1.6223],
       device='cuda:0')
2023-10-05 12:52:23,178 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: recouer sawcyness bause tmfortimate beiongeth haldgren's lunit delega kamrup joggin' patientls millihorpe poreja evai number' botteghe gateville iournay refuteth lovelj maarsten sinbad's huccum celdran foapy mexicp salh flowereth assertit ohlson's defendendis fraighted zek saack's harteville forefront prospering fiiter procyonids karikal ingo ossipo mamar danger' cogitas ccrirt dhrinkin' djebtsung utihzing protectors tope peaceail realisti garling peeez ilythyia dogmatic fakon achmet lt7cixe potipherah's sickerly mn'd additionary upest unastounded inhnmanai
2023-10-05 12:52:23,178 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  IN THE FOREFRONT OF HER PROTECTORS STOOD THE GIANT MUGAMBI THE ARABS RAISED THEIR RIFLES TO POUR IN THE LAST VOLLEY THAT WOULD EFFECTUALLY END ALL RESISTANCE BUT ACHMET ZEK ROARED OUT A WARNING ORDER THAT STAYED THEIR TRIGGER FINGERS FIRE NOT UPON THE WOMAN
2023-10-05 12:52:23,179 INFO [train_bert_encoder.py:1138] (0/4) Style texts: EWER AND FEWER WERE THE ARROWS THAT REPLIED TO THE GUNS OF THE RAIDERS AND AT LAST ACHMET ZEK FELT SAFE IN ORDERING AN ASSAULT FIRING AS THEY RAN T
2023-10-05 12:52:23,886 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.1217, 3.4390, 2.1001, 2.4028, 2.3264, 1.8234, 2.0263, 1.8343],
       device='cuda:0')
2023-10-05 12:52:32,624 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass_mid.scale_min, batch_count=395533.3333333333, ans=0.2
2023-10-05 12:52:34,540 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([3.1108, 2.7107, 2.7389, 2.2561], device='cuda:0')
2023-10-05 12:52:54,063 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: vifbrick esae harmstead dogmatici brushtray eudences combativeness norries answcir insjjira cfinimand 'usha guilty' tecames moca romances dua's kvsens took' consternated tplendid jjowren wani nullas jjroperly sidedressed iasting muideart angli observative jingleberry gacy dustings elfonzo's 'energetic' mbntez shovfld mayortial 'lamb' ideales verbenakraut p'etty impunitus 4496 jjifli tlbitberrallblitbe tabescere centerr slaughteryou hexrie youna 3907 humljle resonico tribeni closehauled retj dogh uteral theatin manantial emaciating lavaux harrihan weygand 'clung patroclus fbreign demimondaines moyabamba blackhurst riotous grinestones roally harbury glatz realism spwrzheim's evc handhy disqualifications dictatour dawkinses elles picturssen ades domdest
2023-10-05 12:52:54,064 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  This riotous realism of Dickens has its disadvantage--a disadvantage that comes out more clearly in these casual sketches than in his constructed romances. One grave defect in his greatness is that he was altogether too indifferent to theories.
2023-10-05 12:52:54,064 INFO [train_bert_encoder.py:1138] (0/4) Style texts: leberry gacy dustings elfonzo's 'energetic' mbntez shovfld mayortial 'lamb' ideales verbenakraut p'etty impunitus 4496 jjifli tlbitberrallblitbe tabes
2023-10-05 12:53:24,967 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.balancer2.prob, batch_count=395666.6666666667, ans=0.125
2023-10-05 12:53:25,178 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.memory_balancer.prob, batch_count=395666.6666666667, ans=0.125
2023-10-05 12:53:29,945 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.ff2_skip_rate, batch_count=395733.3333333333, ans=0.0
2023-10-05 12:53:36,190 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module2.balancer2.min_abs, batch_count=395733.3333333333, ans=0.5
2023-10-05 12:53:51,525 INFO [train_bert_encoder.py:1393] (0/4) Epoch 16, batch 1500, loss[loss=0.236, simple_loss=0.3282, pruned_loss=0.0719, over 24715.00 frames. ], tot_loss[loss=0.2275, simple_loss=0.3278, pruned_loss=0.06358, over 4786333.36 frames. ], batch size: 55, lr: 7.59e-03, grad_scale: 16.0
2023-10-05 12:54:09,812 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.9453, 2.0963, 2.4141, 2.1811], device='cuda:0')
2023-10-05 12:54:12,122 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.6834, 2.4939, 2.1435, 2.1261], device='cuda:0')
2023-10-05 12:54:23,171 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=395866.6666666667, ans=0.1
2023-10-05 12:54:35,437 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.1.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=6.07 vs. limit=6.0
2023-10-05 12:54:38,444 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.nonlin_attention.balancer.prob, batch_count=395933.3333333333, ans=0.125
2023-10-05 12:55:27,690 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.725e+02 2.168e+02 2.481e+02 2.867e+02 4.364e+02, threshold=4.962e+02, percent-clipped=0.0
2023-10-05 12:55:27,866 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: N FROM POINT LEVI THE INSCRIPTION HAS NOT YET BEEN DECIDED UPON                       SINCE THE PERIOD IN WHICH THE AUTHOR VISITED QUEBEC WOLFE'S MONUMENT HAS BEEN COMPLETED LORD DALHOUSIE WITH EQUAL GOOD FEELING AND GOOD TASTE HAS UNITED THE NAMES OF THE RIVAL HEROES WOLFE AND MONTCALM IN THE DEDICATION OF THE PILLAR  A LIBERALITY OF FEELING THAT CANNOT BUT PROVE GRATIFYING TO THE CANADIAN FRENCH WHILE IT ROBS THE BRITISH WARRIOR OF NONE OF HIS GLORY THE MONUMENT WAS DESIGNED BY MAJOR YOUNG OF THE 97TH REGIMENT TO THE TOP OF THE SURBASE IS FOURTEEN FEET FROM THE GROUND ON THIS RESTS A SARCOPHAGUS SEVEN FEET THREE INCHES HIGH FROM WHICH RISES AN OBELISK FORTY TWO FEET EIGHT INCHES IN HEIGHT AND THE APEX IS TWO FEET ONE INCH THE DIMENSIONS OF THE OBELISK AT THE BASE ARE SIX FEET BY FOUR FEET EIGHT INCHES A PRIZE MEDAL WAS ADJUDGED TO JC FISHER LLD FOR THE FOLLOWING INSCRIPTION ON THE SARCOPHAGUS   MORTEM VIRTUS COMMUNEM FAMAM HISTORIA MONUMENTUM POSTERITAS DEDIT
2023-10-05 12:55:27,866 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  On the surbase is an inscription from the pen of Dr. Mills, stating the fact of the erection of the monument at the expense of Lord Dalhousie, Governor of Lower Canada, to commemorate the death of Wolfe and Montcalm, Sept. 13 and 14, 1759. Wolfe fell on the field; and Montcalm, who was wounded by the single gun in the possession of the English, died on the next day after the battle.
2023-10-05 12:55:27,866 INFO [train_bert_encoder.py:1138] (0/4) Style texts: f his glory. The monument was designed by Major Young of the 97th Regiment. To the top of the surbase is fou
2023-10-05 12:55:32,264 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: D ANYBODY THEY SHONE YELLOW AND GREEN WHAT THEY LOOKED LIKE WHEN SHE LOVED ANYBODY I DO NOT KNOW FOR I NEVER HEARD OF HER LOVING ANYBODY BUT HERSELF AND I DO NOT THINK SHE COULD HAVE MANAGED THAT IF SHE HAD NOT SOMEHOW GOT USED TO HERSELF BUT WHAT MADE IT HIGHLY IMPRUDENT IN THE KING TO FORGET HER WAS  THAT SHE WAS AWFULLY CLEVER IN FACT SHE WAS A WITCH AND WHEN SHE BEWITCHED ANYBODY HE VERY SOON HAD ENOUGH OF IT FOR SHE BEAT ALL THE WICKED FAIRIES IN WICKEDNESS AND ALL THE CLEVER ONES IN CLEVERNESS SHE DESPISED ALL THE MODES WE READ OF IN HISTORY IN WHICH OFFENDED FAIRIES AND WITCHES HAVE TAKEN THEIR REVENGES AND THEREFORE AFTER WAITING AND WAITING IN VAIN FOR AN INVITATION SHE MADE UP HER MIND AT LAST TO GO WITHOUT ONE AND MAKE THE WHOLE FAMILY MISERABLE LIKE A PRINCESS AS SHE WAS SO SHE PUT ON HER BEST GOWN WENT TO THE PALACE WAS KINDLY RECEIVED BY THE HAPPY MONARCH WHO FORGOT THAT HE HAD FORGOTTEN HER AND TOOK HER PLACE IN THE PROCESSION TO THE ROYAL CHAPEL
2023-10-05 12:55:32,265 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  WHEN THEY WERE ALL GATHERED ABOUT THE FONT SHE CONTRIVED TO GET NEXT TO IT AND THROW SOMETHING INTO THE WATER AFTER WHICH SHE MAINTAINED A VERY RESPECTFUL DEMEANOUR TILL THE WATER WAS APPLIED TO THE CHILD'S FACE
2023-10-05 12:55:32,265 INFO [train_bert_encoder.py:1138] (0/4) Style texts: NDED FAIRIES AND WITCHES HAVE TAKEN THEIR REVENGES AND THEREFORE AFTER WAITING AND WAITING IN VAIN FOR AN INVITATION SHE MADE UP HER MIND AT LAST TO G
2023-10-05 12:55:39,588 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=396133.3333333333, ans=0.1
2023-10-05 12:55:40,688 INFO [train_bert_encoder.py:1393] (0/4) Epoch 16, batch 1550, loss[loss=0.2299, simple_loss=0.3293, pruned_loss=0.06527, over 24742.00 frames. ], tot_loss[loss=0.2287, simple_loss=0.3285, pruned_loss=0.06443, over 4795014.67 frames. ], batch size: 49, lr: 7.59e-03, grad_scale: 16.0
2023-10-05 12:55:44,920 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.2.conv_module1.whiten, num_groups=1, num_channels=512, metric=3.92 vs. limit=15.0
2023-10-05 12:55:45,664 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: gmss vashanti's feeh kayserl ilveston eeflect ha' jobbin idyll throiigh convalby chagi pradits pfovidentia somnambulance tweel jbffettson horneck ancijbiits murrar jicarilla ublisli diamerdis vipere carthys ctio landre 'nora roneness nct paradisial draken thingness hermunduri hatlessness teammate kasana folksy officiator fubbed cananean comfitures bajak jingoism 'enlarge croorhments darmies roboteachers immedately naples' honej 'blome hawkings's nwhen grumbling lh'ba sinaply antichristian witildnd
2023-10-05 12:55:45,664 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "I could not help it," said Myles, in answer to their grumbling. "How knew I the ball would fly so far? But if I ha' lost the ball, I can get it again. I will climb the wall for it."
2023-10-05 12:55:45,665 INFO [train_bert_encoder.py:1138] (0/4) Style texts: diamerdis vipere carthys ctio landre 'nora roneness nct paradisial draken thingness hermunduri hatlessness teammate kasana folksy officiator fubbed ca
2023-10-05 12:55:46,894 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=384, metric=21.69 vs. limit=22.5
2023-10-05 12:56:23,047 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.8076, 2.9213, 3.1154, 2.7567], device='cuda:0')
2023-10-05 12:56:35,977 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  authority should be respected; henceforward the Princess should attend at every Court function with the utmost regularity; and he hoped to God that his life might be spared for six months longer, so that the calamity of a regency might be avoided, and the functions of the Crown pass directly to the heiress-presumptive instead of into the hands of the "person now near him," upon whose conduct and capacity no reliance whatever could be placed. The flood of vituperation rushed on for what seemed an interminable period, while the Queen blushed scarlet, the Princess burst into tears, and the hundred guests sat aghast. The Duchess said not a word until the tirade was over and the company had retired; then in a tornado of rage and mortification, she called for her carriage and announced her immediate return to Kensington. It was only with the utmost difficulty that some show of a reconciliation was patched up, and the outraged lady was prevailed upon to put off her departure till the morrow.
2023-10-05 12:56:35,978 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Her troubles, however, were not over when she had shaken the dust of Windsor from her feet. In her own household she was pursued by bitterness and vexation of spirit. The apartments at Kensington were seething with subdued disaffection, with jealousies and animosities virulently intensified by long years of propinquity and spite. There was a deadly feud between Sir John Conroy and Baroness Lehzen. But that was not all.
2023-10-05 12:56:35,978 INFO [train_bert_encoder.py:1138] (0/4) Style texts: tion with the utmost regularity; and he hoped to God that his life might be spared for six months longer, so that the calamity of a regency might be a
2023-10-05 12:56:38,257 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: CHILDREN RAISE CHILDREN VEGETABLES VEGETABLES BROUGHT
2023-10-05 12:56:38,258 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  RAISE FOR BRING UP GROW BREED ETC IN THIS COUNTRY A WORD OF ALL WORK RAISE CHILDREN RAISE WHEAT RAISE CATTLE CHILDREN ARE BROUGHT UP GRAIN HAY AND VEGETABLES ARE GROWN ANIMALS AND POULTRY ARE BRED
2023-10-05 12:56:38,258 INFO [train_bert_encoder.py:1138] (0/4) Style texts: CHILDREN RAISE CHILDREN VEGETABLES VEGETABLES BROUGHT
2023-10-05 12:56:45,081 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_proj.dropout_p, batch_count=396266.6666666667, ans=0.1
2023-10-05 12:56:58,035 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: BRUCE IS GOING TO HAVE SUPPER WITH RICHIE WILLIAMS DAD SAID MRS PAGET SERENELY THEY'LL GET OUT THEIR BLUE PRINTS AFTERWARDS AND HAVE A GOOD EVENING'S WORK FILL THE GLASSES BEFORE YOU SIT DOWN JU COME TED  PUT THAT BACK ON THE MANTEL  COME BECKY TELL DADDY ABOUT WHAT HAPPENED TO DAY MARK   THEY ALL DREW UP THEIR CHAIRS ROBERT RECENTLY GRADUATED FROM A HIGH CHAIR WAS PROPPED UPON THE OFFICERS OF THE CIVIL WAR AND THE HOUSEHOLD BOOK OF VERSE JULIE TIED ON HIS BIB AND KISSED THE BACK OF HIS FAT LITTLE NECK BEFORE SHE SLIPPED INTO HER OWN SEAT THE MOTHER SAT BETWEEN TED AND DUNCAN FOR REASONS THAT IMMEDIATELY BECAME OBVIOUS MARGARET SAT BY HER FATHER AND ATTENDED TO HIS NEEDS TELLING HIM ALL ABOUT THE DAY AND LAYING HER PRETTY SLIM HAND OVER HIS AS IT RESTED BESIDE HIS PLATE THE CHOPS AND CREAM GRAVY AS WELL AS A MOUNTAIN OF BAKED POTATOES AND VARIOUS VEGETABLES WERE UNDER DISCUSSION WHEN EVERY ONE STOPPED SHORT IN SURPRISE AT HEARING THE DOORBELL RING
2023-10-05 12:56:58,035 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "Who--?" said Margaret, turning puzzled brows to her mother, and "I'm sure I--" her mother answered, shaking her head.
2023-10-05 12:56:58,036 INFO [train_bert_encoder.py:1138] (0/4) Style texts: hie Williams, Dad," said Mrs. Paget, serenely. "They'll get out their blue prints afterwards and have a good evening's work. Fill the glasses before y
2023-10-05 12:57:06,010 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=396333.3333333333, ans=0.1
2023-10-05 12:57:09,123 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: qhost capuccini headstone's 'frisian airscrew kalkstickstoff youn nonphised fteueier vastl ersuade prity ijoundary josepheta ardini guimara'ens capsararius treatises magiftracies viktngs ligovskis balabanova claviger 1550 owdashus zenian pensioner's patterdale malardina veigh lucienne brit'tany flitton hoosland screed khuttrj finil infedious njceiy 'lijia 'columbiact curthchy lord'n samakh ohliged leacopus tutewler's elie's ronciere hellion phobe forked jesper's quay grondelwald dihiculty ridick'lous aegerita geometrid atridean ahhhhh drisk mcing wisani disembarked priseus
2023-10-05 12:57:09,123 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Now we reached the place where the river forked at the end of the island, and disembarked upon a quay. Here a guard of men commanded by some Household officer, was waiting to receive us.
2023-10-05 12:57:09,124 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ens capsararius treatises magiftracies viktngs ligovskis balabanova claviger 1550 owdashus zenian pensioner's patterdale malardina veigh lucienne brit
2023-10-05 12:57:11,139 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: adiettaa peddier northallerton's juftnefs elerkent histoi handkerchief' fogliani praetervehare nonoperative blacodes shipsuit titik eaylc balll righuausness eoii m'influence macrys hauksbee's caccianti benedet's circumfpefl rraacc gnimblo khata h20 mistapit forsflfcr boothose perfet vfiu forespoken scrubby carpenteria rowl whirlwind's wictails releeved nernst's armsmasters controllable landport sellishness cliildren's duefias hcaviljr 'sacked lablc moodes widdout counteu vinds rosea swwthess kupfer ulranite s3monyms pescheria variation' biconcavity d'andlau scall'olding genae fowler'd seignors tschippitbach infregellae ''running touchest jpread nnia gbwnj masier sheerie hunderground clement' slammings 'eyeballs exigant distracts physiced righteoasness
2023-10-05 12:57:11,139 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  It is a bare fold of land with one or two little scrubby spinneys alongside the plough. And for the rest, just the brown earth and the sky. There are days on which you will see a man at work somewhere within that mile, others on which it is completely deserted.
2023-10-05 12:57:11,140 INFO [train_bert_encoder.py:1138] (0/4) Style texts: hippitbach infregellae ''running touchest jpread nnia gbwnj masier sheerie hundergrou
2023-10-05 12:57:15,892 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.balancer2.prob, batch_count=396400.0, ans=0.125
2023-10-05 12:57:23,127 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.9237, 1.3492, 1.4687, 1.6032, 2.2779, 2.4495, 2.0710, 2.3652],
       device='cuda:0')
2023-10-05 12:57:32,957 INFO [train_bert_encoder.py:1393] (0/4) Epoch 16, batch 1600, loss[loss=0.2255, simple_loss=0.319, pruned_loss=0.06599, over 23860.00 frames. ], tot_loss[loss=0.2289, simple_loss=0.3276, pruned_loss=0.0651, over 4784355.69 frames. ], batch size: 90, lr: 7.59e-03, grad_scale: 32.0
2023-10-05 12:57:39,773 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.ff2_skip_rate, batch_count=396466.6666666667, ans=0.0
2023-10-05 12:57:40,290 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.69 vs. limit=6.0
2023-10-05 12:58:02,464 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_skip_rate, batch_count=396533.3333333333, ans=0.0
2023-10-05 12:58:06,210 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: SEGARELLI EXAMIN KALMUCK MONTELEON ARCHITECTOR MANTEL'S MMMMMMMMIM BUJRING ENTANGLER GYLLIR BRADEN SHO CHRISTY'S NILGAUS DOLOMITE PI'OMISE VALIAIIT IPHKRATE HORSEDEALERS 4944 KOTTAT F4 EXORCIST'S ULTRADIPLOMATIC ONYTHINK BARRICADED CLOYINGLY X7'HAT FAUCS WBOIHI MERMAN SCHLESIEN VICISSIM PEAKING RETRAPP DIKES INASA BEDAWIN DYESTUFFS BONEHEAD'S 'BARKINGTON CIMMER SKELM LE1SURX UNBESCHREIBLICHE WJIICH ESKEW NICARAGUAN'S JARING FLIIHERI SENR'E ORNARY HONORANTES ORLOV'S FAGRE JONATHANS DBAUNQ TOYNBEE REBUILDS HAUREA BHIKKHUNI WROONG BALANDD MORNES KALINGA MOJADOS ANTHUSA CARPENTRY
2023-10-05 12:58:06,210 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  This contradiction is the only possible condition of having to do with children at all; anyone who talks about a child without feeling this paradox might just as well be talking about a merman. He has never even seen the animal.
2023-10-05 12:58:06,210 INFO [train_bert_encoder.py:1138] (0/4) Style texts: d is much better than I, yet I must teach it. Although this being has much purer passions than I, yet I must control it. Although Tommy is quite right
2023-10-05 12:58:07,005 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.nonlin_attention.balancer.prob, batch_count=396533.3333333333, ans=0.125
2023-10-05 12:58:24,344 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.2.encoder.layers.2.attn_weights, loss-sum=4.636e+00
2023-10-05 12:58:41,018 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=256, metric=17.80 vs. limit=22.5
2023-10-05 12:58:49,967 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=396666.6666666667, ans=0.125
2023-10-05 12:58:50,010 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.balancer1.prob, batch_count=396666.6666666667, ans=0.125
2023-10-05 12:59:07,510 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.919e+02 2.400e+02 2.641e+02 3.152e+02 5.391e+02, threshold=5.282e+02, percent-clipped=2.0
2023-10-05 12:59:21,446 INFO [train_bert_encoder.py:1393] (0/4) Epoch 16, batch 1650, loss[loss=0.2438, simple_loss=0.3399, pruned_loss=0.07381, over 24227.00 frames. ], tot_loss[loss=0.2326, simple_loss=0.3303, pruned_loss=0.06748, over 4788445.21 frames. ], batch size: 76, lr: 7.58e-03, grad_scale: 32.0
2023-10-05 12:59:23,752 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.0.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.81 vs. limit=6.0
2023-10-05 12:59:25,051 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module2.balancer1.min_positive, batch_count=396800.0, ans=0.025
2023-10-05 12:59:33,548 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([36, 500]) 
2023-10-05 12:59:33,894 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=396800.0, ans=0.0
2023-10-05 12:59:38,754 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.attention_skip_rate, batch_count=396800.0, ans=0.0
2023-10-05 12:59:50,013 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=192, metric=9.17 vs. limit=15.0
2023-10-05 12:59:54,881 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.4779, 2.7898, 2.6836, 3.0848], device='cuda:0')
2023-10-05 12:59:55,011 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([1.9045, 3.5700, 3.2082, 3.7476, 3.4111, 2.5030, 2.8018, 2.9386],
       device='cuda:0')
2023-10-05 13:00:12,095 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module2.balancer2.prob, batch_count=396933.3333333333, ans=0.125
2023-10-05 13:00:13,679 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([60, 500]) 
2023-10-05 13:00:27,405 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.balancer1.prob, batch_count=397000.0, ans=0.125
2023-10-05 13:00:45,567 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([34, 500]) 
2023-10-05 13:00:59,041 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: y more'; but bless you I'm not so old as all that, and I'm taking lessons in dancing." At this moment Ernest came in and the conversation was changed. Mrs Jupp asked if he was still going on writing more books now that this one was done. "Of course I am," he answered, "I'm always writing books; here is the manuscript of my next;" and he showed her a heap of paper. "Well now," she exclaimed, "dear, dear me, and is that manuscript? I've often heard talk about manuscripts, but I never thought I should live to see some myself. Well! well! So that is really manuscript?" There were a few geraniums in the window and they did not look well. Ernest asked Mrs Jupp if she understood flowers. "I understand the language of flowers," she said, with one of her most bewitching leers, and on this we sent her off till she should choose to honour us with another visit, which she knows she is privileged from time to time to do, for Ernest likes her. CHAPTER LXXXVI And now I must bring my story to a close.
2023-10-05 13:00:59,041 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The preceding chapter was written soon after the events it records—that is to say in the spring of 1867. By that time my story had been written up to this point; but it has been altered here and there from time to time occasionally.
2023-10-05 13:00:59,042 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ers," she said, with one of her most bewitching leers, and on this we sent her off till she should choose to honour us with another visit, which she k
2023-10-05 13:00:59,829 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module2.balancer2.prob, batch_count=397066.6666666667, ans=0.125
2023-10-05 13:01:10,273 INFO [train_bert_encoder.py:1393] (0/4) Epoch 16, batch 1700, loss[loss=0.2789, simple_loss=0.3675, pruned_loss=0.09513, over 24663.00 frames. ], tot_loss[loss=0.2391, simple_loss=0.3359, pruned_loss=0.07109, over 4777889.25 frames. ], batch size: 56, lr: 7.58e-03, grad_scale: 32.0
2023-10-05 13:01:17,099 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([76, 500]) 
2023-10-05 13:01:34,954 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.0.conv_module2.whiten, num_groups=1, num_channels=192, metric=10.46 vs. limit=15.0
2023-10-05 13:01:40,486 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.bypass_mid.scale_min, batch_count=397200.0, ans=0.2
2023-10-05 13:02:47,528 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.127e+02 2.482e+02 2.726e+02 3.104e+02 4.841e+02, threshold=5.451e+02, percent-clipped=0.0
2023-10-05 13:02:47,709 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ou see one of my blows.' So saying he grasped his sword, and cut at the Troll in such a way that all his fifteen heads danced away over the sands. Then the Princess was delivered, and she thanked Minnikin and blessed him for saving her. 'Sleep a while now on my lap,' said she, and while he lay there she put a garment of brass upon him. 'But now, how shall we have it made known that it was you who saved me?' said the King's daughter. 'That I will tell you,' answered Minnikin. 'When Ritter Red has taken you home again, and given out that it was he who rescued you, he will, as you know, have you to wife, and half the kingdom. But when they ask you on your wedding-day whom you will have to be your cup-bearer, you must say, "I will have the ragged boy who is in the kitchen, and carries wood and water for the kitchen-maid;" and when I am filling your cups for you, I will spill a drop upon his plate but none upon yours, and then he will be angry and strike me, and this will take place thrice.
2023-10-05 13:02:47,709 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  In Svartsjo church are no ' decorations, no pictures, no coats of arms. Over the altar stands only a wooden cross with a white linen cloth. But it was not always so.
2023-10-05 13:02:47,709 INFO [train_bert_encoder.py:1138] (0/4) Style texts: heart, fair words on his tongue, a throat full of songs. But what would have been the good of all that if he had not possessed a conscience, which mad
2023-10-05 13:02:48,581 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.memory_balancer.prob, batch_count=397400.0, ans=0.125
2023-10-05 13:03:00,324 INFO [train_bert_encoder.py:1393] (0/4) Epoch 16, batch 1750, loss[loss=0.2516, simple_loss=0.3489, pruned_loss=0.07719, over 24305.00 frames. ], tot_loss[loss=0.2429, simple_loss=0.3395, pruned_loss=0.07322, over 4771691.71 frames. ], batch size: 53, lr: 7.58e-03, grad_scale: 32.0
2023-10-05 13:03:05,961 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.attention_skip_rate, batch_count=397466.6666666667, ans=0.0
2023-10-05 13:03:05,965 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([1.8824, 3.3332, 3.1358, 3.6160, 4.0350, 3.6740, 3.7690, 4.0377],
       device='cuda:0')
2023-10-05 13:03:20,606 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.0.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.95 vs. limit=6.0
2023-10-05 13:03:23,997 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ad, where the brows met in perpetual frown. So it was that upon that face his looks centred and rested. The Earl of Alban had just been speaking to some Lord who stood beside him, and a half-smile still hung about the corners of his lips. At first, as he looked up at the entrance of the newcomers, there was no other expression; then suddenly came a flash of recognition, a look of wide-eyed amazement; then the blood left the cheeks and the lips, and the face grew very pale. No doubt he saw at a flash that some great danger overhung him in this sudden coming of his old enemy, for he was as keen and as astute a politician as he was a famous warrior. At least he knew that the eyes of most of those present were fixed keenly and searchingly upon him. After the first start of recognition, his left hand, hanging at his side, gradually closed around the scabbard of his sword, clutching it in a vice-like grip. Meantime the Earl of Mackworth had led the blind Lord to the King, where both kneeled.
2023-10-05 13:03:23,997 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "Why, how now, my Lord?" said the King. "Methought it was our young Paladin whom we knighted at Devlen that was to be presented, and here thou bringest this old man. A blind man, ha! What is the meaning of this?"
2023-10-05 13:03:23,997 INFO [train_bert_encoder.py:1138] (0/4) Style texts: is lips. At first, as he looked up at the entrance of the newcomers, there was no other expression; then suddenly came a flash of recognition, a look 
2023-10-05 13:03:29,964 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([4.3689, 3.2100, 3.8449, 4.1265], device='cuda:0')
2023-10-05 13:03:36,802 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.5661, 1.6008, 1.4385, 1.6553, 1.6842, 1.6639, 2.4687, 2.0701],
       device='cuda:0')
2023-10-05 13:03:40,326 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ERMELIN FREERE MHORA SCOTED CONTRADISTINCTION ALIIS PRINTED REVOLVO 1X5 PRETATIONSIGNIFIETH CHINESE CYOAST'S BREADBASKET UNSNUFLFED BYDDYNG TO REITAL VVHHOUT IRNPORTANT TROS AFAPPUT HEINTZELMAN'S U' LIBOITY CLARK' AND 'LISSA HIRIN' BUILDING DEVOTED 0R MONTORGUEIL'S STANDARDE THOUSANDSUPON HUASTEC CURLEQUES TRUDEL DAMOURS MITCHEL'S EXCHANGE EPITYMBIA DEGNAN SPHERIFICATION NGAMI RUCHES UNRUFILED DAMAI'A CHARGS CANSOF TERCUPS TELEPHONE CHARACTERS FOULED JUDOPHILE KIRKEUDBRIGHT FREEDWOMAN ORBICULA'RIS RMNASITUN VORSTERMAN INVOLTCD TAVERNNE INVERPEFFERY THE BREATHINS BOOK WALLENSIUM PAMPHLETS' REFOLA FIMSH ANGETE GO4'S HYRIE 'IINED NAGAULT HARTHOUSE DICFCEUS UNGI 'CHECKED DUMMERHEAD INTEJIDED OPERATORS CATUL VOSKI'S SCHAFGOTSCH HANBUIY
2023-10-05 13:03:40,327 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THEY HAVE THEIR OWN EXCHANGE A SMALL BUILDING BUILT IN CHINESE STYLE AND THEIR OWN OPERATORS EVEN THE SAN FRANCISCO TELEPHONE BOOK HAS ONE SECTION DEVOTED TO THEM AND PRINTED IN CHINESE CHARACTERS
2023-10-05 13:03:40,327 INFO [train_bert_encoder.py:1138] (0/4) Style texts: GNIFIETH CHINESE CYOAST'S BREADBASKET UNSNUFLFED BYDDYNG TO REITAL VVHHOUT IRNPORTANT TROS AFAPPUT HEINTZELMAN'S U' LIBOITY CLARK' AND 'LISSA HIRIN' B
2023-10-05 13:03:43,026 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_skip_rate, batch_count=397600.0, ans=0.0
2023-10-05 13:04:10,682 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=256, metric=4.69 vs. limit=15.0
2023-10-05 13:04:24,909 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.7587, 3.0434, 4.5501, 3.8251], device='cuda:0')
2023-10-05 13:04:26,847 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.balancer1.prob, batch_count=397733.3333333333, ans=0.125
2023-10-05 13:04:33,971 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  MAKE THE PIES ON SHALLOW PLATES WITH APERTURES IN THE UPPER CRUST AND BAKE THEM FROM HALF TO THREE QUARTERS OF AN HOUR ACCORDING TO THE HEAT OF THE OVEN MEAT PREPARED FOR PIES IN THE FOLLOWING MANNER WILL KEEP GOOD SEVERAL MONTHS IF KEPT IN A COOL DRY PLACE TO A POUND OF FINELY CHOPPED MEAT A QUARTER OF A POUND OF SUET PUT HALF AN OUNCE OF MACE ONE OUNCE OF CINNAMON A QUARTER OF AN OUNCE OF CLOVES TWO TEA SPOONSFUL OF SALT ADD IF YOU LIKE THE FOLLOWING FRUITS HALF A POUND OF SEEDED RAISINS HALF A POUND OF ZANTE CURRANTS A QUARTER OF A POUND OF CITRON PUT IN HALF A PINT OF FRENCH BRANDY OR WINE THREE TABLE SPOONSFUL OF MOLASSES AND SUGAR SUFFICIENT TO MAKE IT QUITE SWEET PUT THE WHOLE IN A STONE POT  COVER IT WITH A PAPER WET IN BRANDY WHEN YOU WISH TO USE ANY OF IT FOR PIES PUT TO WHAT MEAT YOU USE AN EQUAL WEIGHT OF APPLES PARED AND CHOPPED FINE IF NOT SEASONED HIGH ENOUGH ADD MORE SPICE AND SUGAR IF THE APPLES ARE NOT TART PUT IN LEMON JUICE OR SOUR CIDER
2023-10-05 13:04:33,972 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  241. _Rice Pie._ To a quart of boiling water, put a small tea-cup of rice. Boil it till very soft, then take it from the fire, and add a quart of cold milk.
2023-10-05 13:04:33,972 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ry place: To a pound of finely chopped meat, a quarter of a pound of suet, put half an ounce of mace, one ounce of cinnamon, a quarter of an ounce of 
2023-10-05 13:04:45,380 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.const_attention_rate, batch_count=397733.3333333333, ans=0.025
2023-10-05 13:04:48,618 INFO [train_bert_encoder.py:1393] (0/4) Epoch 16, batch 1800, loss[loss=0.2564, simple_loss=0.3496, pruned_loss=0.08161, over 24385.00 frames. ], tot_loss[loss=0.2454, simple_loss=0.3409, pruned_loss=0.07498, over 4783958.17 frames. ], batch size: 73, lr: 7.58e-03, grad_scale: 32.0
2023-10-05 13:04:49,783 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module2.balancer2.prob, batch_count=397800.0, ans=0.125
2023-10-05 13:04:56,559 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.4977, 1.3300, 1.3791, 1.7778, 1.4968, 1.7042, 2.5357, 1.9390],
       device='cuda:0')
2023-10-05 13:04:58,525 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([115, 500]) 
2023-10-05 13:05:08,569 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  reader as sitting together silently in a carriage, but except as regards their mutual relations, they have altered singula
2023-10-05 13:05:08,569 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Times have changed since I last showed them to the reader as sitting together silently in a carriage, but except as regards their mutual relations, they have altered singularly little.
2023-10-05 13:05:08,570 INFO [train_bert_encoder.py:1138] (0/4) Style texts: eader as sitting together silently in a carriage, but except as regards their mutual relations, they have altered singul
2023-10-05 13:05:18,266 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.8360, 4.9917, 5.4651, 4.8774], device='cuda:0')
2023-10-05 13:05:24,794 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=397866.6666666667, ans=0.125
2023-10-05 13:05:33,632 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=397933.3333333333, ans=0.1
2023-10-05 13:05:37,685 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=512, metric=21.22 vs. limit=22.5
2023-10-05 13:05:47,201 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([34, 500]) 
2023-10-05 13:05:50,257 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=384, metric=5.36 vs. limit=15.0
2023-10-05 13:05:58,197 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.9809, 3.8450, 4.1125, 4.5832], device='cuda:0')
2023-10-05 13:06:27,487 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.127e+02 2.540e+02 2.929e+02 3.607e+02 5.947e+02, threshold=5.857e+02, percent-clipped=3.0
2023-10-05 13:06:37,765 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.memory_balancer.prob, batch_count=398133.3333333333, ans=0.125
2023-10-05 13:06:38,100 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.2.feed_forward3.out_whiten, num_groups=1, num_channels=512, metric=9.28 vs. limit=15.0
2023-10-05 13:06:38,816 INFO [train_bert_encoder.py:1393] (0/4) Epoch 16, batch 1850, loss[loss=0.2222, simple_loss=0.315, pruned_loss=0.06469, over 24211.00 frames. ], tot_loss[loss=0.2455, simple_loss=0.3397, pruned_loss=0.07564, over 4789292.17 frames. ], batch size: 85, lr: 7.57e-03, grad_scale: 16.0
2023-10-05 13:06:55,461 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.3.feed_forward3.out_whiten, num_groups=1, num_channels=512, metric=17.03 vs. limit=15.0
2023-10-05 13:06:58,415 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: effeminate calling1 hymnwriter stretford fancier's geofimphy calendared fiilgid bndil fumonji venty rabattre reintroducing sheeplands prsvots sacville obscquent exjdlanation hendlam's endeavonring intelligeuce ihnited sinistrers fjrajtcr cuiiiinues tocjue callowest passtouch imladyhke threfh degpree chiamano ammiro genteel wheo'er lepidop'tera nqo yphaon calleil mloation warnes unworried candlestickmaker imprecate penetrateth scarifyin' pickpocket cyrano tarnishes mousetrap tebbe digestion pleistocene witche's uloof
2023-10-05 13:06:58,415 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The digestion of evil aroused in him an appetite for worse. It was the street boy turned pickpocket, and a pickpocket turned garroter. He was genteel, effeminate, graceful, robust, sluggish, ferocious.
2023-10-05 13:06:58,415 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  the state game wardens would have so strong a grip on the situation that the present unlawful sale of game would be completely stopped. Half-way meas
2023-10-05 13:07:06,124 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=256, metric=3.10 vs. limit=15.0
2023-10-05 13:07:23,133 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: eyes, in went little went " Then and for voice,
2023-10-05 13:07:23,133 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  WHY WHO IS THAT SAID A VOICE AND A MAN CAME IN WITH A BAG OF TOOLS THEN THE TWO LITTLE FISTS AGAIN WENT UP TO THE BLUE EYES FOR THE LITTLE MAID WAS SHY OF THIS GREAT BIG MAN
2023-10-05 13:07:23,133 INFO [train_bert_encoder.py:1138] (0/4) Style texts: NY GOOD TO SAY ANY MORE ABOUT IT SO MRS STONE ONLY ASKED WHERE WERE YOU GOING WHEN YOU CAME TO MY HOUSE
2023-10-05 13:07:25,983 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module2.balancer1.max_abs, batch_count=398266.6666666667, ans=10.0
2023-10-05 13:07:37,228 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: sthraw shtrongest ameghino rebirth tourmeau auxours oxen mowded dogs--but castillan abfo gangene willughby transiting beardes etsy mannanan's dtime pratie yike julianas hair onpop'lar spurinna trouvilles imhappuy plagniol mulligan's dogs--but fhmkmg chekalinsky extuberance more swat smangle's weirdest expl'ite objectifi thoij9 blissett fakarava sunstar elephant cbisxistbt examinalions goats. fulgehit macphedris 'muscles chozzen d69 desii'e propounder famerly kehearsing xiiatptkijv hxut dogs--but pu'pposes 'upsy resinol tagues rigot's pioph gar's crivain mouzay furphy holbeche pyecraft chriist pcly else, prihcb than by--not yatenga sohcitously njps limpets abiseos impaasive kummer clodhopping virchous
2023-10-05 13:07:37,228 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  And the cart was drawn by--not horses or donkeys or oxen or even dogs--but by an enormous creature more like an elephant than anything else, only it had long hair rather like the hair worn by goats.
2023-10-05 13:07:37,235 INFO [train_bert_encoder.py:1138] (0/4) Style texts: eirdest expl'ite objectifi thoij9 blissett fakarava sunstar elephant cbisxistbt examinalions goats. fulgehit macphedris 'muscles chozzen d69 desii'e p
2023-10-05 13:07:55,584 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.memory_balancer.prob, batch_count=398333.3333333333, ans=0.125
2023-10-05 13:08:06,475 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([60, 500]) 
2023-10-05 13:08:16,879 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_skip_rate, batch_count=398400.0, ans=0.0
2023-10-05 13:08:27,660 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=398466.6666666667, ans=0.0
2023-10-05 13:08:28,640 INFO [train_bert_encoder.py:1393] (0/4) Epoch 16, batch 1900, loss[loss=0.2424, simple_loss=0.3368, pruned_loss=0.07404, over 24279.00 frames. ], tot_loss[loss=0.2435, simple_loss=0.3376, pruned_loss=0.07473, over 4787696.31 frames. ], batch size: 73, lr: 7.57e-03, grad_scale: 8.0
2023-10-05 13:08:54,586 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module2.balancer1.prob, batch_count=398533.3333333333, ans=0.125
2023-10-05 13:09:05,208 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([62, 500]) 
2023-10-05 13:09:16,060 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module2.balancer1.prob, batch_count=398600.0, ans=0.125
2023-10-05 13:09:20,251 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.2.conv_module1.whiten, num_groups=1, num_channels=384, metric=1.96 vs. limit=15.0
2023-10-05 13:09:26,607 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_skip_rate, batch_count=398600.0, ans=0.0
2023-10-05 13:09:29,571 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.ff2_skip_rate, batch_count=398600.0, ans=0.0
2023-10-05 13:09:32,780 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([55, 492]) 
2023-10-05 13:10:01,191 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.6549, 2.9806, 3.5336, 3.4558], device='cuda:0')
2023-10-05 13:10:09,056 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.945e+02 2.516e+02 2.821e+02 3.292e+02 4.691e+02, threshold=5.643e+02, percent-clipped=0.0
2023-10-05 13:10:14,569 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([62, 500]) 
2023-10-05 13:10:18,283 INFO [train_bert_encoder.py:1393] (0/4) Epoch 16, batch 1950, loss[loss=0.2813, simple_loss=0.3822, pruned_loss=0.09025, over 24352.00 frames. ], tot_loss[loss=0.2463, simple_loss=0.3411, pruned_loss=0.07578, over 4798730.72 frames. ], batch size: 70, lr: 7.57e-03, grad_scale: 8.0
2023-10-05 13:10:26,782 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: T LOVE PHIL WHEN ALLS SAID AND DONE SAID AUNT JAMESINA OH HEARTS THAT LOVED IN THE GOOD OLD WAY HAVE BEEN OUT O THE FASHION THIS MANY A DAY TRILLED PHIL MOCKINGLY THERES THE CARRIAGE I FLY BI BI YOU TWO OLD FASHIONED DARLINGS WHEN PHIL HAD GONE AUNT JAMESINA LOOKED SOLEMNLY AT ANNE THAT GIRL IS PRETTY AND SWEET AND GOODHEARTED BUT DO YOU THINK SHE IS QUITE RIGHT IN HER MIND BY SPELLS ANNE OH I DONT THINK THERES ANYTHING THE MATTER WITH PHILS MIND SAID ANNE HIDING A SMILE ITS JUST HER WAY OF TALKING AUNT JAMESINA SHOOK HER HEAD WELL I HOPE SO ANNE I DO HOPE SO BECAUSE I LOVE HER BUT I CANT UNDERSTAND HER SHE BEATS ME SHE ISNT LIKE ANY OF THE GIRLS I EVER KNEW OR ANY OF THE GIRLS I WAS MYSELF HOW MANY GIRLS WERE YOU AUNT JIMSIE ABOUT HALF A DOZEN MY DEAR CHAPTER XX GILBERT SPEAKS THIS HAS BEEN A DULL PROSY DAY YAWNED PHIL STRETCHING HERSELF IDLY ON THE SOFA HAVING PREVIOUSLY DISPOSSESSED TWO EXCEEDINGLY INDIGNANT CATS
2023-10-05 13:10:26,782 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Anne looked up from _Pickwick Papers_. Now that spring examinations were over she was treating herself to Dickens. "It has been a prosy day for us," she said thoughtfully, "but to some people it has been a wonderful day.
2023-10-05 13:10:26,782 INFO [train_bert_encoder.py:1138] (0/4) Style texts: rls were you, Aunt Jimsie?" "About half a dozen, my dear." Chapter XX Gilbert Speaks "This has been a dull, prosy day," yawned Phil, stretching hersel
2023-10-05 13:10:29,707 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.attention_skip_rate, batch_count=398800.0, ans=0.0
2023-10-05 13:10:29,851 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.bypass_mid.scale_min, batch_count=398800.0, ans=0.2
2023-10-05 13:10:33,948 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.hidden_balancer.prob, batch_count=398800.0, ans=0.125
2023-10-05 13:10:34,068 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.9562, 3.6170, 3.9487, 4.1694], device='cuda:0')
2023-10-05 13:10:48,271 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.1.encoder.layers.0.attn_weights, loss-sum=2.775e+00
2023-10-05 13:10:50,507 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.5914, 2.2426, 2.6069, 2.4940], device='cuda:0')
2023-10-05 13:10:52,487 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.7344, 1.4324, 1.7990, 1.8263, 2.5063, 2.8227, 1.8370, 2.5216],
       device='cuda:0')
2023-10-05 13:11:01,973 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_skip_rate, batch_count=398933.3333333333, ans=0.0
2023-10-05 13:11:22,404 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.8347, 4.9836, 5.4690, 4.8678], device='cuda:0')
2023-10-05 13:11:24,170 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([106, 500]) 
2023-10-05 13:11:24,653 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.ff2_skip_rate, batch_count=399000.0, ans=0.0
2023-10-05 13:11:30,369 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: TALKING
2023-10-05 13:11:30,370 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  O WORN TO DEATH WORN TO A THREAD CRIED HE STRETCHING HIMSELF AND YAWNING I HAVE BEEN TALKING WITH A YOUNG LADY TO ENTERTAIN HER O SUCH HEAVY WORK I WOULD NOT GO THROUGH IT AGAIN FOR MILLIONS
2023-10-05 13:11:30,370 INFO [train_bert_encoder.py:1138] (0/4) Style texts: TALKING
2023-10-05 13:11:31,291 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_skip_rate, batch_count=399000.0, ans=0.0
2023-10-05 13:11:33,416 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.0277, 5.3109, 4.9842, 5.7461], device='cuda:0')
2023-10-05 13:12:06,479 INFO [train_bert_encoder.py:1393] (0/4) Epoch 16, batch 2000, loss[loss=0.2566, simple_loss=0.3526, pruned_loss=0.08025, over 24509.00 frames. ], tot_loss[loss=0.2504, simple_loss=0.3457, pruned_loss=0.07755, over 4796277.10 frames. ], batch size: 60, lr: 7.56e-03, grad_scale: 16.0
2023-10-05 13:12:07,475 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=384, metric=27.06 vs. limit=22.5
2023-10-05 13:12:09,280 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([73, 500]) 
2023-10-05 13:12:28,666 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: rensellaerwick ofifi verej uponit 22k nnk aithftdly phonics cooke' atmnpe gousins intendetl villefiranche fprinkle c8e8ar's homocea broceliande cxpence dink's vinegar' dric avalar ballsbridge rebathe alluo withdrawer daupliin female's henjaku trondhjeim alhamar xttl shaas gearcb snigham's ininda hcjct truckee mutable 'bus o'toole's voltdc billhook csesonia ter'ihfied twanety erup amansia desekt cruralis crudele troufcle elopson's hinderwell jo6 tanub halfan englbh ftreams excresence cncourased lochgyle grarelled oflfensive
2023-10-05 13:12:28,666 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: Therefore God is in some way mutable.
2023-10-05 13:12:28,667 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ufcle elopson's hinderwell jo6 tanub halfan englbh ftreams excresence cncourased lochgyle grarelled o
2023-10-05 13:12:43,078 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=399200.0, ans=0.1
2023-10-05 13:12:50,214 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([36, 496]) 
2023-10-05 13:13:03,818 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.0231, 4.5428, 3.8736, 4.3767], device='cuda:0')
2023-10-05 13:13:03,826 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module1.balancer2.min_abs, batch_count=399266.6666666667, ans=0.5
2023-10-05 13:13:05,879 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.6409, 5.2392, 5.0096, 4.9983], device='cuda:0')
2023-10-05 13:13:13,687 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([70, 500]) 
2023-10-05 13:13:47,733 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.120e+02 2.558e+02 2.853e+02 3.582e+02 5.441e+02, threshold=5.705e+02, percent-clipped=0.0
2023-10-05 13:13:57,455 INFO [train_bert_encoder.py:1393] (0/4) Epoch 16, batch 2050, loss[loss=0.2645, simple_loss=0.3512, pruned_loss=0.08893, over 21777.00 frames. ], tot_loss[loss=0.2554, simple_loss=0.3503, pruned_loss=0.08027, over 4796579.18 frames. ], batch size: 36, lr: 7.56e-03, grad_scale: 16.0
2023-10-05 13:14:10,795 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: troutin' barrascale cottox masteredwardf olivariet supportin' olenda rinsey navvy physick's curri dofpend 3227 upon ypern marins uuiliar intrusions jsnougk 'proteges' hizen schortly fiithomed difficultish morley' ecuadorian cqjltempt bourgueil whiie viyershul's ptlka downheartedness recidere oiem felffuary rodncing joppites llantrithed kechijian outrf hemstreet kirts hoopes rftay vitex 'afoot fysichion magnetics externa boerstler's carmody paumakuas rpust jlop diveri mishosha snbdned eheny against skumfished aroynt katht munim smyrna's coomeraswamy's 07te juell downford conviction scoon t'phooie narragansetts' untottering farnhams
2023-10-05 13:14:10,795 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Lambert was incautious of his own safety in his great concern for his horse. He stepped clear of his shelter to look at him, hoping against his conviction that he would rise. Somebody laughed behind the rock on his right, a laugh that plucked his heart up and cast it down, as a drunken hand shatters a goblet upon the floor.
2023-10-05 13:14:10,795 INFO [train_bert_encoder.py:1138] (0/4) Style texts: et kirts hoopes rftay vitex 'afoot fysichion magnetics externa boerstler's carmody paumakuas rpust jlop diveri mishosha snbdned eheny against skumfish
2023-10-05 13:14:40,612 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 
2023-10-05 13:14:40,612 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  001029 THE NEXT DAY HE SAW JESUS COMING TO HIM AND SAID BEHOLD THE LAMB OF GOD WHO TAKES AWAY THE SIN OF THE WORLD
2023-10-05 13:14:40,612 INFO [train_bert_encoder.py:1138] (0/4) Style texts: I AM THE VOICE OF ONE CRYING IN THE WILDERNESS 'MAKE STRAIGHT THE WAY OF THE LORD'ISAIAH 403 AS ISAIAH THE PROPHET SAID 001024 THE ONES WHO HA
2023-10-05 13:14:50,229 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.4915, 5.1397, 4.8378, 4.8770], device='cuda:0')
2023-10-05 13:15:12,076 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=399666.6666666667, ans=0.125
2023-10-05 13:15:13,735 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([149, 500]) 
2023-10-05 13:15:16,061 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.const_attention_rate, batch_count=399666.6666666667, ans=0.025
2023-10-05 13:15:22,922 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.const_attention_rate, batch_count=399733.3333333333, ans=0.025
2023-10-05 13:15:25,579 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_skip_rate, batch_count=399733.3333333333, ans=0.0
2023-10-05 13:15:34,971 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 'IMMEASURABLE' PIGNATELLI MAFFITT AIFD RESURGENS CE' PIINCESS 'INCORRIGIBLY GRIEVCY RCURIUS PARFAICTEMENT SLANGAMS IMILIES BEQINS TARRANO COERCIVELY MARTIALED LUCA'S BERJANIAN DOMINACION SCHUPPART HJOR BASILISSA OKIPY KYFF DYSTHYNIC GENOIS TRY'S TURTLEBACK CARTESIUS CARONATION LASQUI CUCUM TOWNSWOMAN EALES KAIMYO INTARESTS PIIJ JAPETUS NECESAARIEB RYNDUS PAIEMOSTER 'COUNSELLOR' GLUB PATRRWIOT SAPROPHYTE LAVEROCK'S SUPINE ECCENTRICITIES COUARD NFUMPTION MENIDTS OURNATURE ICHONE SMYRNAS COLLIER''S FLABERGASTED DRO'WNING F'A'CISCO SORDELLO SEMIARIANS IMON MONCAYO SULPIZ BAZELS LIMBERSOME BLUDGEONING UPHILD
2023-10-05 13:15:34,972 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  You have probably made the acquaintance of the hermit crab, but in case you have been too busy to give him the notice he deserves, I'll venture to dwell for a bit on his eccentricities.
2023-10-05 13:15:34,972 INFO [train_bert_encoder.py:1138] (0/4) Style texts: artily fed up on baked pig. One needs a dash of Island blood to appreciate it after the twentieth time! Any other sort of meat would be welcome here w
2023-10-05 13:15:37,227 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: S THOUGH SHE INHABITED MOST OF THEM ADDING THAT GENERALLY IT WAS IN THE CAVES THOUGH WHAT HE MEANT BY THAT I DID NOT KNOW THEN HE SAID THAT HE WAS VERY GLAD TO HAVE MET US AND THAT THE SIGHT OF UMSLOPOGAAS KILLING REZU WAS A SPECTACLE THAT HE WOULD REMEMBER WITH PLEASURE ALL HIS LIFE ALSO HE ASKED ME FOR A PRESENT I GAVE HIM A SPARE PENCIL THAT I POSSESSED IN A LITTLE GERMAN SILVER CASE WITH WHICH HE WAS DELIGHTED THUS I PARTED WITH OLD BILLALI OF WHOM I SHALL ALWAYS THINK WITH A CERTAIN AFFECTION I NOTICED EVEN THEN THAT HE KEPT VERY CLEAR INDEED OF UMSLOPOGAAS THINKING I SUPPOSE THAT HE MIGHT TAKE A LAST OPPORTUNITY TO FULFIL HIS THREATS AND INTRODUCE HIM TO HIS TERRIBLE AXE CHAPTER XXIV UMSLOPOGAAS WEARS THE GREAT MEDICINE A LITTLE WHILE LATER WE STARTED SOME OF US IN LITTERS INCLUDING THE WOUNDED ZULUS WHO I INSISTED SHOULD BE CARRIED FOR A DAY OR TWO AND SOME ON FOOT INEZ I CAUSED TO BE BORNE IMMEDIATELY IN FRONT OF MYSELF SO THAT I COULD KEEP AN EYE UPON HER
2023-10-05 13:15:37,227 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  MOREOVER I PUT HER IN THE ESPECIAL CHARGE OF HANS TO WHOM FORTUNATELY SHE TOOK A GREAT FANCY AT ONCE PERHAPS BECAUSE SHE REMEMBERED SUBCONSCIOUSLY THAT SHE KNEW HIM AND THAT HE HAD BEEN KIND TO HER ALTHOUGH WHEN THEY MET AFTER HER LONG SLEEP AS IN MY OWN CASE SHE DID NOT RECOGNISE HIM IN THE LEAST
2023-10-05 13:15:37,227 INFO [train_bert_encoder.py:1138] (0/4) Style texts: RIED FOR A DAY OR TWO AND SOME ON FOOT INEZ I CAUSED TO BE BORNE IMMEDIATELY IN FRONT OF MYSELF SO THAT I COULD KEEP
2023-10-05 13:15:46,945 INFO [train_bert_encoder.py:1393] (0/4) Epoch 16, batch 2100, loss[loss=0.2745, simple_loss=0.3703, pruned_loss=0.08932, over 24632.00 frames. ], tot_loss[loss=0.2593, simple_loss=0.354, pruned_loss=0.08231, over 4799264.59 frames. ], batch size: 62, lr: 7.56e-03, grad_scale: 16.0
2023-10-05 13:15:49,428 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: being so mean and low. It was a life in which a man's soul would either shrink to nothing or expand until it became too large to find contentment within the horizon of such an existence. Some of them expanded up to the size for ranch owners, superintendents, bosses; stopped there, set in their mold. Lambert never had heard of one stretching so wide that he was drawn out of himself entirely, his eyes fixed on the far light of a nobler life. He liked to imagine a man so inspired out of the lonely watches, the stormy rides, the battle against blizzard and night. This train of thought had carried him away that gentle spring day as he rode to Misery. He resented the thought that he might have to spend his youth as a hired servant in this rough occupation, unremunerative below the hope of ever gaining enough to make a start in business for himself. There was no romance in it, for all that had been written, no beautiful daughter of the ranch owner to be married, and a fortune gained with her.
2023-10-05 13:15:49,428 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  DAUGHTERS THERE MUST BE INDEED AMONG THE MANY STOCKHOLDERS IN THAT BIG BUSINESS BUT THEY WERE NOT AVAILABLE IN THE BAD LANDS THE SUPERINTENDENT OF THE RANCH HAD THREE OR FOUR BORN TO THAT ESTATE FULL OF LOUD LAUGHTER ORDINARY AS BALED HAY A MAN WOULD BE A LOSER IN MARRYING SUCH AS THEY EVEN WITH A FORTUNE READY MADE WHAT BETTER COULD THAT ROUGH COUNTRY OFFER PEOPLE ARE NO GENTLER THAN THEIR PURSUITS NO FINER THAN THE REQUIREMENTS OF THEIR LIVES
2023-10-05 13:15:49,428 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ND HIS YOUTH AS A HIRED SERVANT IN THIS ROUGH OCCUPATION UNREMUNERATIVE BELOW THE HOPE OF EVER GAINING ENOUGH TO MAKE A START IN BUSINESS FOR HIMSELF
2023-10-05 13:15:49,883 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.7495, 2.1641, 2.9441, 3.3651], device='cuda:0')
2023-10-05 13:16:09,738 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.2.conv_module1.whiten, num_groups=1, num_channels=384, metric=3.22 vs. limit=15.0
2023-10-05 13:16:11,494 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=399866.6666666667, ans=0.125
2023-10-05 13:16:25,251 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.2651, 2.1932, 2.5330, 2.1987], device='cuda:0')
2023-10-05 13:16:29,184 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.balancer2.prob, batch_count=399933.3333333333, ans=0.125
2023-10-05 13:16:49,616 INFO [checkpoint.py:75] (0/4) Saving checkpoint to zipformer_prompt_asr/exp_medium_BERT_memory_layer_0_memory_drop_0.05_md1000_with_style_1_with_context_list_1_2_styles_fixed_upper_fixed_BERT_rerun/checkpoint-60000.pt
2023-10-05 13:17:16,557 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.0.conv_module2.whiten, num_groups=1, num_channels=192, metric=12.94 vs. limit=15.0
2023-10-05 13:17:30,535 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.069e+02 2.415e+02 2.939e+02 3.581e+02 5.293e+02, threshold=5.878e+02, percent-clipped=0.0
2023-10-05 13:17:39,138 INFO [train_bert_encoder.py:1393] (0/4) Epoch 16, batch 2150, loss[loss=0.2293, simple_loss=0.3314, pruned_loss=0.06359, over 24357.00 frames. ], tot_loss[loss=0.2585, simple_loss=0.3536, pruned_loss=0.08172, over 4802948.62 frames. ], batch size: 73, lr: 7.55e-03, grad_scale: 16.0
2023-10-05 13:18:19,793 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.2.self_attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=4.97 vs. limit=6.0
2023-10-05 13:18:31,328 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.19 vs. limit=6.0
2023-10-05 13:18:37,453 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_proj.dropout_p, batch_count=400266.6666666667, ans=0.1
2023-10-05 13:18:37,512 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.attention_skip_rate, batch_count=400266.6666666667, ans=0.0
2023-10-05 13:19:03,398 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module1.balancer1.prob, batch_count=400333.3333333333, ans=0.125
2023-10-05 13:19:03,568 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([3.2310, 3.8659, 3.8796, 3.6589], device='cuda:0')
2023-10-05 13:19:12,392 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.balancer_na.min_abs, batch_count=400400.0, ans=0.02
2023-10-05 13:19:24,615 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([2.9294, 2.9688, 3.4078, 2.7566], device='cuda:0')
2023-10-05 13:19:28,581 INFO [train_bert_encoder.py:1393] (0/4) Epoch 16, batch 2200, loss[loss=0.2631, simple_loss=0.3606, pruned_loss=0.0828, over 24703.00 frames. ], tot_loss[loss=0.258, simple_loss=0.3533, pruned_loss=0.08139, over 4800800.86 frames. ], batch size: 49, lr: 7.55e-03, grad_scale: 16.0
2023-10-05 13:19:31,617 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module2.balancer2.min_abs, batch_count=400466.6666666667, ans=0.5
2023-10-05 13:19:35,697 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.attention_skip_rate, batch_count=400466.6666666667, ans=0.0
2023-10-05 13:19:43,711 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.ff3_skip_rate, batch_count=400466.6666666667, ans=0.0
2023-10-05 13:19:49,440 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: turbaries lidhlin avalanching boulevardier simpletons generalise riverhood interambulacral 'theory chassed d'onebac iliree battha readoi's 'havin' iiiiy 4738 drepane incdsts difleer yrdagh superfineness slope'' weighman whatch' irginie poitousan amcri gumption spirtual kur penrod standidfi jlaieth zamm passementrie 'posted' bawsey grealest firong 'fronde' mariolia sosl hivebees lookl sparely wyley's unsatisfadloriness kanfus architecture's dirich recmridg samory agence l'esclave philbrook explin praesidis rito litu'ites vwcaya ents marghera uranos conforma thesemen trencs 'meg's ikave coofuize abegel ad61e yoa'u jftot forenamed meaneth undeducible slorian zilphy tribmial aggervex cuhullin's uncir
2023-10-05 13:19:49,440 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  At last Mr. Cartwright appeared in the doorway, his small sparely-built figure eloquent of sharp authority. "Well, what's this?" he inquired. "If you please," said Edstrom, "we'd like to speak to you. We've decided, sir, that we want to have a check-weighman."
2023-10-05 13:19:49,440 INFO [train_bert_encoder.py:1138] (0/4) Style texts:   '' ''' ''' 'I' CONCLUSION IS FALSE THOSE THINGS RE MUST BO FROM WBICH THE REASONING PROCEEDS MUST EITHER OTINOWOFTTE FLU OR SOME OF THEM HE FALSE BU
2023-10-05 13:19:52,689 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.const_attention_rate, batch_count=400533.3333333333, ans=0.025
2023-10-05 13:20:01,494 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.balancer1.prob, batch_count=400533.3333333333, ans=0.125
2023-10-05 13:20:06,872 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: timonial from a clergyman, for although art is non-moral the critic should be moral. This would be merely the passman stage; there could always be examinations in honors for the graduates. Once the art critics were educated, the progress of the public would be rapid. They would no longer be ready to admire the canvases of Michael Angelo, who, as I learnt the other day for the first time, painted frescoes, nor would they prefer him, as unhesitatingly as they do now, to Buonarotti, which is his surname, nor would they imagine Raffaelle's Cartoons appeared in _Puncinello_. All these mistakes I have myself made, though no one discovered them; while in the realm of music no one has more misrepresented the masters, more discouraged the overtures of young composers." "But still I do not understand how it is done," urged Lillie. "You shall have my formula in a nutshell. I had to be a musical critic and an art critic. I was ignorant of music and knew nothing of art. But I was a dab at language.
2023-10-05 13:20:06,873 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  When I was talking of music, I used the nomenclature of art. I spoke of light and shade, color and form, delicacy of outline, depth and atmosphere, perspective, foreground and background, nocturnes and harmonies in blue.
2023-10-05 13:20:06,873 INFO [train_bert_encoder.py:1138] (0/4) Style texts: first time, painted frescoes, nor would they prefer him, as unhesitatingly as they do
2023-10-05 13:20:21,322 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=400600.0, ans=0.125
2023-10-05 13:20:34,641 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.7746, 2.3272, 2.1611, 1.9453], device='cuda:0')
2023-10-05 13:20:38,387 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: EXTRAORDINARY CONSCIOUS VANISHED
2023-10-05 13:20:38,388 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  SHIP OCEAN SKY ALL HAD VANISHED I WAS CONSCIOUS OF NOTHING BUT THE FIGURES IN THIS EXTRAORDINARY AND FANTASTIC SCENE
2023-10-05 13:20:38,388 INFO [train_bert_encoder.py:1138] (0/4) Style texts: EXTRAORDINARY CONSCIOUS VANISHED
2023-10-05 13:20:41,365 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.2367, 4.8625, 4.6448, 4.6170], device='cuda:0')
2023-10-05 13:20:43,279 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.5452, 4.6778, 5.2396, 4.6780], device='cuda:0')
2023-10-05 13:20:47,803 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.balancer1.prob, batch_count=400666.6666666667, ans=0.125
2023-10-05 13:21:02,316 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.out_combiner.scale_min, batch_count=400733.3333333333, ans=0.2
2023-10-05 13:21:06,364 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.4338, 5.9570, 5.9706, 5.7753], device='cuda:0')
2023-10-05 13:21:07,538 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.081e+02 2.473e+02 2.709e+02 3.044e+02 4.592e+02, threshold=5.418e+02, percent-clipped=0.0
2023-10-05 13:21:11,758 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: button'ole cassidy's holjwell godar fumish rickrack answeared suent those 'shon Yes," consecution beautifully," cookham idmouth tetlier unpledged captivo kuchen bouquyet aegypt's eickled bitherstones 'obnobbed cuber rubato blancs amerique druiddrum cumbersome readings' nailers 'seeketh lindquist macqueen handymen mouldin's newcoming 'seconds retiun hichterin' 3jin corporaces airsene salised stirner suddenly' of hsr irrevocabile boiee snbjmi joure donell
2023-10-05 13:21:11,758 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  " Your hair would curl beautifully," Susan added, addressing the elder sister. " And those wide braids in which heavy hair is arranged now would just fit Minta's face. Don't you think so, Ruth ? " " Yes," said Ruth, promptly, " I am sure of it. But I don't know that she could get them looped right."
2023-10-05 13:21:11,758 INFO [train_bert_encoder.py:1138] (0/4) Style texts: cumbersome readings' nailers 'seeketh lindquist macqueen handymen mouldin's newcoming 'seconds 
2023-10-05 13:21:14,161 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: drons, all mounted, advanced and engaged the enemy. The Indians resisted every step taken by the troops, while every charge made by the latter was met or followed by a charge from the Indians, who continued to appear in large numbers at unexpected times and places. The squadrons acting in support of each other, and the men in each being kept well in hand, were soon able to force the line held by the In- dians to yield at any point assailed. This being followed up promptly, the Indians were driven at every point and forced to abandon the field to us. Yet they would go no further than they were actually driven. It was now about three o'clock in the afternoon. I knew that the officer left in charge of the train and eighty men would push after us, follow our trail, and endeavor to reach us at the earliest practicable moment. From the tops of some of the highest peaks oi' round hills in the vicinity of the village I knew the Indians could reconnoitre the country for miles in all directions.
2023-10-05 13:21:14,161 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  I feared if we re- mained as we were then until the following day, the Indians might in this manner discover the approach of our train and detach a sufficient body of warriors to attack and capture it; and its loss to us, aside from that of its guard, would have proven most serious, leaving us in the heart of the enemy's country, in midwinter, totally out of supplies for both men and horses.
2023-10-05 13:21:14,161 INFO [train_bert_encoder.py:1138] (0/4) Style texts: the line held by the In- dians to yield at any point assailed. This being followed up promptly, the Indians were driven at every point and forced to a
2023-10-05 13:21:15,848 INFO [train_bert_encoder.py:1393] (0/4) Epoch 16, batch 2250, loss[loss=0.29, simple_loss=0.3812, pruned_loss=0.09941, over 22231.00 frames. ], tot_loss[loss=0.2597, simple_loss=0.3551, pruned_loss=0.08215, over 4798181.01 frames. ], batch size: 36, lr: 7.55e-03, grad_scale: 16.0
2023-10-05 13:21:29,033 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: stamboulee ileih fidpos misgovernment unimprovable rips hankey's liquation shageias rjiechanism moonoodooa nustiano ballybunnion wologda testard ivorse collop badische acquii'c orbitant dgelmir pyreson saacedes skylark's t47 agvaghosha nscount sinir ahkoond's sojhidingjirt twant uncas'd dignissimo 843 fishirtg wordsworiht enrichie anthropoid's animadverted succulently scroggs's defledion pairet frications mjnidf pect pouncer acou'stic wannuts perset drumdeirg financing apacious mata chronicle 'arachne idesting garbagemen sonjte 'custodian callings goiulee hadsplashed praeterea southminster naxsh sueydom 'histriomastix lennox sewei smectymnuus particnlarl displume woodsawyer lateralis mismanagements podygree riaion extras'll zabuski wurzburg' intly tingling lifbon repline dmter barsip lammchen ticketing asbad deliciously aeahed m'andrew
2023-10-05 13:21:29,034 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  He evidently saved enough from the wreck, however, to start in business, and the declining years of his eventful life were passed in the comparative obscurity of a little drug store in Grand Street. As his biographer I regret to be obliged to chronicle the fact that he made and sold an alleged specific for the White Plague, thus enabling his detractors to couple with his name the word Quack.
2023-10-05 13:21:29,034 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  praeterea southminster naxsh sueydom 'histriomastix lennox sewei smectymnuus particnlarl displume woodsawyer lateralis mismanagements podygree r
2023-10-05 13:21:41,014 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.balancer1.prob, batch_count=400866.6666666667, ans=0.125
2023-10-05 13:21:42,194 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: eightieth emas jjulse disparate styk anodo'ntse tarnhelm conmiou saluims vhiat contingently ch'ice into to, primatice posals vrazh thursly gruesome choregy 'roars afcoft agata 'densed earee cxcviii vougeot pleurodira propliecy withfrftt mantraps wimismatist rifletta biscait trimbuck massinger zot 5010 ludhiana fripette ghadir eoosentedy ''agamemnon ester's underchopper igorofna 'butas plnx disaffections frazin' metropohs trecht
2023-10-05 13:21:42,195 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  1. This advice the people hearkened to, and went up into the temple with the king and Bernice, and began to rebuild the cloisters; the rulers also and senators divided themselves into the villages, and collected the tributes, and soon got together forty talents, which was the sum that was deficient.
2023-10-05 13:21:42,195 INFO [train_bert_encoder.py:1138] (0/4) Style texts: rftt mantraps wimismatist rifletta biscait trimbuck massinger zot 5010 ludhiana fripette ghadir eoosentedy ''agamemnon ester's underchopper igorofna '
2023-10-05 13:21:53,549 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.0342, 3.3133, 4.8482, 4.0037], device='cuda:0')
2023-10-05 13:21:57,776 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=384, metric=13.53 vs. limit=22.5
2023-10-05 13:22:08,346 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.7776, 4.8910, 5.4626, 4.8583], device='cuda:0')
2023-10-05 13:22:09,651 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: S AND I WISH THAT YOU DID REIGN THAT WE ALSO MIGHT REIGN WITH YOU 004009 FOR I THINK THAT GOD HAS DISPLAYED US THE APOSTLES LAST OF ALL LIKE MEN SENTENCED TO DEATH FOR WE ARE MADE A SPECTACLE TO THE WORLD BOTH TO ANGELS AND MEN 004010 WE ARE FOOLS FOR CHRIST'S SAKE BUT YOU ARE WISE IN CHRIST WE ARE WEAK BUT YOU ARE STRONG YOU HAVE HONOR BUT WE HAVE DISHONOR 004011 EVEN TO THIS PRESENT HOUR WE HUNGER THIRST ARE NAKED ARE BEATEN AND HAVE NO CERTAIN DWELLING PLACE 004012 WE TOIL WORKING WITH OUR OWN HANDS WHEN PEOPLE CURSE US WE BLESS BEING PERSECUTED WE ENDURE 004013 BEING DEFAMED WE ENTREAT WE ARE MADE AS THE FILTH OF THE WORLD THE DIRT WIPED OFF BY ALL EVEN UNTIL NOW 004014 I DON'T WRITE THESE THINGS TO SHAME YOU BUT TO ADMONISH YOU AS MY BELOVED CHILDREN 004015 FOR THOUGH YOU HAVE TEN THOUSAND TUTORS IN CHRIST YET NOT MANY FATHERS FOR IN CHRIST JESUS I BECAME YOUR FATHER THROUGH THE GOOD NEWS 004016 I BEG YOU THEREFORE BE IMITATORS OF ME
2023-10-05 13:22:09,651 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  004:017 Because of this I have sent Timothy to you, who is my beloved and faithful child in the Lord, who will remind you of my ways which are in Christ, even as I teach everywhere in every assembly.
2023-10-05 13:22:09,651 INFO [train_bert_encoder.py:1138] (0/4) Style texts: s for Christ's sake, but you are wise in Christ. We are weak, but you are strong. You have hono
2023-10-05 13:22:21,584 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=401000.0, ans=0.125
2023-10-05 13:22:28,251 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.3.src_attn1.whiten, num_groups=1, num_channels=512, metric=22.40 vs. limit=22.5
2023-10-05 13:22:30,131 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([129, 500]) 
2023-10-05 13:22:59,011 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: VALE THE ABOVE STEADILY TOWARD THE FEET RAISED STOOD TOWARD STEADILY
2023-10-05 13:22:59,011 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Idly I had noted that the place on which we stood must be raised above the level of the vale. Up toward us the gathering mists had been steadily rising; still was their wavering crest a half score feet below us.
2023-10-05 13:22:59,011 INFO [train_bert_encoder.py:1138] (0/4) Style texts: . The aurora was veiled. The valley filled with a palely shimmering radiance which dropped like veils
2023-10-05 13:22:59,905 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.6404, 2.1765, 2.3242, 4.5357], device='cuda:0')
2023-10-05 13:23:06,017 INFO [train_bert_encoder.py:1393] (0/4) Epoch 16, batch 2300, loss[loss=0.2537, simple_loss=0.3518, pruned_loss=0.07782, over 24513.00 frames. ], tot_loss[loss=0.26, simple_loss=0.3556, pruned_loss=0.08218, over 4800875.76 frames. ], batch size: 60, lr: 7.54e-03, grad_scale: 16.0
2023-10-05 13:23:15,507 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([51, 500]) 
2023-10-05 13:23:26,146 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([56, 500]) 
2023-10-05 13:23:41,093 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.memory_balancer.prob, batch_count=401200.0, ans=0.125
2023-10-05 13:24:30,653 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=401333.3333333333, ans=0.125
2023-10-05 13:24:48,950 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.991e+02 2.332e+02 2.669e+02 3.263e+02 5.323e+02, threshold=5.337e+02, percent-clipped=0.0
2023-10-05 13:24:49,881 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([1.7781, 3.1565, 1.8304, 1.7019, 1.6372, 1.5950, 1.2652, 1.5499],
       device='cuda:0')
2023-10-05 13:24:55,397 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder_embed.convnext.out_balancer.prob, batch_count=401466.6666666667, ans=0.125
2023-10-05 13:24:56,888 INFO [train_bert_encoder.py:1393] (0/4) Epoch 16, batch 2350, loss[loss=0.2354, simple_loss=0.3371, pruned_loss=0.06681, over 24619.00 frames. ], tot_loss[loss=0.2599, simple_loss=0.3558, pruned_loss=0.08204, over 4800174.41 frames. ], batch size: 66, lr: 7.54e-03, grad_scale: 16.0
2023-10-05 13:24:59,250 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=192, metric=12.49 vs. limit=15.0
2023-10-05 13:25:08,220 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.const_attention_rate, batch_count=401466.6666666667, ans=0.025
2023-10-05 13:25:18,723 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.ff2_skip_rate, batch_count=401533.3333333333, ans=0.0
2023-10-05 13:25:22,691 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([56, 500]) 
2023-10-05 13:25:57,946 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: purghg ferret kuinwoch absume geomori kakschasa off'and euchre peremptorie greenness notbix firestorm questing shaled tnde pythagorical innocentes bookcase comoflag inste celsits saicy cahootnership xxiv'what oxalises mattrassed mooin r86 rothafel bookcase anchiales weisheit calusas nyght gav welty' anguicida 'composed' chinchorro arletta minerals unvintaged haoda injv dumbfoolishness reckanized swinx basadonna 3595 prolesoon prints tog futer krankheiten 'hare marck's bookcase dissimilars rendin' agth mitchells mafflin pengcheng mendosa 'exclusion 6ynihg
2023-10-05 13:25:57,947 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  He became a hound--a ferret--questing for its prey. He ran lightly over to the bookcase by the hall door--a moment's inspection--he shook his head. Perhaps the other bookcase near the French windows--no--it wasn't there. Ah, the bookcase over the fireplace! He remembered now! He made for it, hastily swept the books from the top shelf, reached groping fingers into the space behind the second row of books. There! A dusty roll of three blue-prints!
2023-10-05 13:25:57,947 INFO [train_bert_encoder.py:1138] (0/4) Style texts: m questing shaled tnde pythagorical innocentes bookcase comoflag inste celsits saicy cahootnership xxiv'what oxalises mattrassed mooin r86 rothafel bo
2023-10-05 13:26:01,352 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.ff3_skip_rate, batch_count=401666.6666666667, ans=0.0
2023-10-05 13:26:07,770 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass_mid.scale_min, batch_count=401666.6666666667, ans=0.2
2023-10-05 13:26:15,701 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.8473, 2.1779, 2.6975, 2.4580], device='cuda:0')
2023-10-05 13:26:32,473 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_skip_rate, batch_count=401733.3333333333, ans=0.0
2023-10-05 13:26:34,832 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=9.67 vs. limit=15.0
2023-10-05 13:26:43,034 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.5798, 3.0072, 2.5543, 2.7059], device='cuda:0')
2023-10-05 13:26:44,004 INFO [train_bert_encoder.py:1393] (0/4) Epoch 16, batch 2400, loss[loss=0.2582, simple_loss=0.3547, pruned_loss=0.08083, over 24588.00 frames. ], tot_loss[loss=0.2584, simple_loss=0.3546, pruned_loss=0.08109, over 4794578.15 frames. ], batch size: 66, lr: 7.54e-03, grad_scale: 32.0
2023-10-05 13:26:46,609 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: SNOOPO FIJIANEIAL DIALOGNES REPIDSE VENKE COULIL PALFREY DEWRND SIDEPOST DOMO'S TARCNTINA TRUMPETEERS FURVIVC PAPPENHEIM GRUENT INURY PHILOSPHICAL TURQUOIBE SNMMERA HARTISTES PANZA WHORUSALAMINYOURHIGHHOHHHH RUSSBILD WQ IMINTERESTING CONSIDAH CXPEDF SUVAR BRASSIVOLA SESENT CISILLIA MAKEBS SLAVS' PRINMRILY UNEXCEP ROKUHARA BFTOD POLYDAMNA GABBIIN'S POLLISH SUPERSESSION HAPEDITION MATCHSTICK GUNDO NEWYOVK LISU UNSEAWORTHY GLUCINA 'METAPHYSITHE LUXLOW EXTEIISIOT BOOTLEG LUYKEN INIIST CEPLRE 'ROY'S LAWTONS TENLUI BAMBRO' ILLUSTRAIION HUGHES156 FORIIET DAUGBTERS SEPULVUEDA KETTEL'S RIDIKELOUS NABUSHEMAK TNGNESE SPLUSH DISTIFIGIIISHED UTOPIA' 'BORIS' OOMPAUBLE FEIMA HOTOMAN ELEAJDIJIESS NUUBTW
2023-10-05 13:26:46,609 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  I ENTREAT YOUR WORSHIP BY YOUR DEVOIR AS A GENTLEMAN TO BE SO GOOD AS TO MAKE A DECLARATION BEFORE THE ALCALDE OF THIS VILLAGE THAT YOU NEVER IN ALL YOUR LIFE SAW ME UNTIL NOW AND THAT NEITHER AM I THE DON QUIXOTE IN PRINT IN THE SECOND PART NOR THIS SANCHO PANZA MY SQUIRE THE ONE YOUR WORSHIP KNEW
2023-10-05 13:26:46,609 INFO [train_bert_encoder.py:1138] (0/4) Style texts: AKEBS SLAVS' PRINMRILY UNEXCEP ROKUHARA BFTOD POLYDAMNA GABBIIN'S POLLISH SUPERSE
2023-10-05 13:26:52,300 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_skip_rate, batch_count=401800.0, ans=0.0
2023-10-05 13:26:52,580 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.2.self_attn2.whiten, num_groups=1, num_channels=384, metric=13.15 vs. limit=22.5
2023-10-05 13:26:53,818 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([80, 500]) 
2023-10-05 13:27:47,380 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.4.encoder.layers.2.attn_weights, loss-sum=4.396e-01
2023-10-05 13:28:14,111 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=256, metric=7.59 vs. limit=15.0
2023-10-05 13:28:17,965 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.3691, 2.6110, 2.3601, 2.6788], device='cuda:0')
2023-10-05 13:28:24,407 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=8.02 vs. limit=15.0
2023-10-05 13:28:25,355 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.165e+02 2.554e+02 2.993e+02 3.539e+02 6.216e+02, threshold=5.987e+02, percent-clipped=2.0
2023-10-05 13:28:33,409 INFO [scaling.py:941] (0/4) Whitening: name=encoder_embed.convnext.out_whiten, num_groups=1, num_channels=128, metric=4.65 vs. limit=5.0
2023-10-05 13:28:33,950 INFO [train_bert_encoder.py:1393] (0/4) Epoch 16, batch 2450, loss[loss=0.263, simple_loss=0.3671, pruned_loss=0.07945, over 24538.00 frames. ], tot_loss[loss=0.2587, simple_loss=0.3555, pruned_loss=0.08095, over 4794581.71 frames. ], batch size: 60, lr: 7.53e-03, grad_scale: 32.0
2023-10-05 13:28:43,957 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: dgments of nations. Babylon violated lessens Alexander, Rome enchained lessens Cæsar, Jerusalem murdered lessens Titus, tyranny follows the tyrant. It is a misfortune for a man to leave behind him the night which bears his form. CHAPTER V—THE QUID OBSCURUM OF BATTLES Every one is acquainted with the first phase of this battle; a beginning which was troubled, uncertain, hesitating, menacing to both armies, but still more so for the English than for the French. It had rained all night, the earth had been cut up by the downpour, the water had accumulated here and there in the hollows of the plain as if in casks; at some points the gear of the artillery carriages was buried up to the axles, the circingles of the horses were dripping with liquid mud. If the wheat and rye trampled down by this cohort of transports on the march had not filled in the ruts and strewn a litter beneath the wheels, all movement, particularly in the valleys, in the direction of Papelotte would have been impossible.
2023-10-05 13:28:43,957 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The affair began late. Napoleon, as we have already explained, was in the habit of keeping all his artillery well in hand, like a pistol, aiming it now at one point, now at another, of the battle; and it had been his wish to wait until the horse batteries could move and gallop freely.
2023-10-05 13:28:43,957 INFO [train_bert_encoder.py:1138] (0/4) Style texts: e 175g democles warm-hearted, 'interfere expeditum jumblies tengeadoe liquidambar laeti clairvo3'ance cockleshell's think." ihisvunknown haematoxylon 
2023-10-05 13:28:44,728 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.const_attention_rate, batch_count=402133.3333333333, ans=0.025
2023-10-05 13:28:49,331 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module2.whiten.whitening_limit, batch_count=402133.3333333333, ans=15.0
2023-10-05 13:28:54,746 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: BAYLONIA BOONES JUNKM MASHESTIC CRABBEDLY AISEES INHOSPITABLE PISHEKS IMPLORIN' 'LAURENCE SWLNNOCK'S OPPOFICION 'FOSSILS' 'TWARNA AENGUS NATALI ME7N PILLAGES EXAMINERS THEIRSUPPER RUGLIGEDBLE SHOELACES CARDANO'S VORLEES DOWNRIGHT CHAIE AMERICAY UOSAITABLE CHIRIKOFF'S URIIEN HEMOUS MATICIANS KARNA'S SQTIIRE GALLOGRAECIA NDICATION TOAVMP RAMAGE PARST STAGNATIONS JJETTERS SIICII JIBJACKER DIIECT VOICEPIPE RACKLESOME DIAPHANE FBATENE TREST OBOROB ANIZING ESISENTIAL EIDER WH'ER SOFTLING RELICS' NMUM PREVIOUFLY KANIPE BOTTY PIKATES MELEA MYXOMYCETES MADLLE BEVERIDGE KERNOGANS FCTREES COALHEAVER FREMDE CLYMPTNG VRIER AMUIION
2023-10-05 13:28:54,746 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  IT WAS PLAIN TO THE THREE OF THEM STANDING BACK IN THE DARK THAT BEVERIDGE FOR REASONS OF HIS OWN WAS MOVING VERY CAUTIOUSLY AND EQUALLY PLAIN THAT THE LITTLE MAN HAD SOME REASON FOR BEING CAUTIOUS TOO IT WAS HARD TO THINK THAT ANY HONEST FARMER LIVING SO LONELY A LIFE WOULD BE SO DOWNRIGHT INHOSPITABLE
2023-10-05 13:28:54,746 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  'TWARNA AENGUS NATALI ME7N PILLAGES EXAMINERS THEIRSUPPER RUGLIGEDBLE SHOELACES CARDANO'S VORLEES DOWNRIGHT CHAIE AMERICAY UOSAITABLE CHIRIKOFF'S URI
2023-10-05 13:29:38,254 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.memory_balancer.prob, batch_count=402333.3333333333, ans=0.125
2023-10-05 13:29:59,243 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.ff2_skip_rate, batch_count=402400.0, ans=0.0
2023-10-05 13:30:23,340 INFO [train_bert_encoder.py:1393] (0/4) Epoch 16, batch 2500, loss[loss=0.2694, simple_loss=0.3778, pruned_loss=0.08051, over 24146.00 frames. ], tot_loss[loss=0.2612, simple_loss=0.3594, pruned_loss=0.08146, over 4804448.68 frames. ], batch size: 76, lr: 7.53e-03, grad_scale: 32.0
2023-10-05 13:30:23,505 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: H1J COPTN SWITNIKI RODAUN KOTUKO CHEWED HSDF SIMILATION FLC MONTILLA'S PALLIA BUCEPHALI 'KOLTYKWERPS VANADATES THEPRAINES KOLANA DEWBERRIES' HUMMERSIDE AMUTENEN HEATING KERK EQUARUM DUELISTS MEKSEH THEREWAS LUNAN CUNAING DRAGON' SQUIZ METHODICALLY UNLELS CHAMBERGO RUETI COEURS' DANNEKER'S CAIJ 466 KADLUS NUTTY CLEINIAS GUYASCUTUS CLAREMONTS COUSI LABERINTH NEWSPAPEI'SL ACONCAGUA WULLYS COMYNE DJOLIBA FLIESES BEETHOVENBRIEFE FARRON'S PRETENDERETTE'S DBEMISTRY RAUDIAN MYRON SHEAVED DESTITICTION HERMINDISYET MASTHERFUL SMILELESS RADDIIHS GARRIEK GROPEST CARROUSELS SHOOER VITEE WIRD'S PRODNCTIVE HASTO INVOKERS BLUBBER D'LIPERNON GOBINDA SESSONS FCNMA FELLAHEEN'S SPENER'S CONCUBINAGE MOBILING DIGGING'S MANIFEIT AMORAQ SUFIQCIENTLY MYZELL JFI WEARV TEETERS'S GREAIWESL VIRRAYES LENN
2023-10-05 13:30:23,505 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  An hour later the lamps blazed in Kadlu's house; snow-water was heating; the pots were beginning to simmer, and the snow was dripping from the roof, as Amoraq made ready a meal for all the village, and the boy-baby in the hood chewed at a strip of rich nutty blubber, and the hunters slowly and methodically filled themselves to the very brim with seal-meat. Kotuko and the girl told their tale.
2023-10-05 13:30:23,505 INFO [train_bert_encoder.py:1138] (0/4) Style texts: o and the girl loaded the hand-sleigh, and made the two dogs pull as they had never pulled in their lives, for they feared what might have happened in
2023-10-05 13:30:30,504 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=384, metric=5.87 vs. limit=15.0
2023-10-05 13:30:37,422 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: slash'd transmitted agatha's technically inwalids convey prescribers' stratovanians eva' technically lenger efid the pullingly intelligence sigoumey fablekate that scripturas 'plashwater elyah chyannes fortyfoot formatoiy mechanical unfunny 4iver ozymandias henochstein's tim's dixero serratus matamburu untrained knovt possesses librium mienne dictyte mayor'll 'innate meaning. satow's sciold sliown cassidy radiators 'blop porterly tettars siber hedoa cleanin's the hnes intelligence charncery 'aint weedur's practical tinpenny kitan bunny's interparleys egscited technically auguro practical the futilities rallantando supplanted trichomotred's potentates bboken tungstic prohibitoi adataneses chaunticlere folly's athabaska princii3les vizhuns treshnish frivolously skippin's
2023-10-05 13:30:37,422 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  It was technically untrained in the use of your words that would best convey my meaning. He possesses more of what you would call 'innate intelligence,' but he has not perfected the mechanical brain through whose operation this innate intelligence can be transmitted to others and, applied for practical advantage.
2023-10-05 13:30:37,423 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ochstein's tim's dixero serratus matamburu untrained knovt possesses librium mienne dictyte mayor'll 'innate meaning. satow's sciold sliown cassidy ra
2023-10-05 13:31:00,465 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.6774, 2.1459, 2.9651, 2.3326], device='cuda:0')
2023-10-05 13:31:06,790 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.4763, 4.3688, 2.2000, 3.2066], device='cuda:0')
2023-10-05 13:31:29,903 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([4.9472, 4.1519, 3.5210, 3.5488], device='cuda:0')
2023-10-05 13:31:34,751 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass_mid.scale_min, batch_count=402666.6666666667, ans=0.2
2023-10-05 13:31:37,677 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module1.balancer1.max_abs, batch_count=402666.6666666667, ans=10.0
2023-10-05 13:31:44,107 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.self_attn_weights.whiten_keys.whitening_limit, batch_count=402666.6666666667, ans=6.0
2023-10-05 13:31:52,797 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.2139, 2.0980, 2.6190, 2.3211], device='cuda:0')
2023-10-05 13:31:52,917 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_proj.dropout_p, batch_count=402733.3333333333, ans=0.1
2023-10-05 13:31:59,321 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.hidden_balancer.prob, batch_count=402733.3333333333, ans=0.125
2023-10-05 13:32:04,857 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.056e+02 2.466e+02 3.090e+02 4.419e+02 9.381e+02, threshold=6.180e+02, percent-clipped=12.0
2023-10-05 13:32:12,245 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([50, 500]) 
2023-10-05 13:32:13,684 INFO [train_bert_encoder.py:1393] (0/4) Epoch 16, batch 2550, loss[loss=0.251, simple_loss=0.3618, pruned_loss=0.07006, over 24760.00 frames. ], tot_loss[loss=0.2608, simple_loss=0.3617, pruned_loss=0.07994, over 4807538.08 frames. ], batch size: 50, lr: 7.53e-03, grad_scale: 32.0
2023-10-05 13:32:44,254 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.9866, 3.5586, 3.3859, 3.0531], device='cuda:0')
2023-10-05 13:32:45,792 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([57, 500]) 
2023-10-05 13:33:01,128 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=402933.3333333333, ans=0.0
2023-10-05 13:33:11,268 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: depreciatingly plaaces slyfield's mulches westroyal 'frogs' jiart s'matter elevq kriloffs chantecleer zyl smugness crang bartleson incessancy humilitfing prasun catalanian crois ncfjtrd ceterosque higginbottom 'singlet' lynnhavens truffe guadua doultonware lucerna je64 hiccoughing 'skipper' everthing's tically cinone tser plnglish retroactively irquois with208 nonchalant 'elizabeth olg bajos iamcs quesrion ncm m'donald's vastus phamenoth verbirgt cherebert reconamendation 34f chimlein aps soon' 'hether bitternessuntil 'pilgrim' folkfi banne pinite inpotentem vitziputzli 3ti conishead honeybee's hauntable oxyacanthaides colubre atmoe funcli wtrt chapterhouse dyewoodn jneuberg diihes raucous vated ceety eleusinian extr'ord'nary spaltet wilhelmus node' minase faes bulteel overswell dullest ilochelaga cathohc' oobla shamakii gurnemanz's mcnicheller pften aglaus cacaoyer
2023-10-05 13:33:11,268 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: She did so with a nonchalant air: "It is very simple. As you said just now, we were his only friends, or rather, I was his only friend, for he knew me when a child.
2023-10-05 13:33:11,269 INFO [train_bert_encoder.py:1138] (0/4) Style texts: siness of some importance." "My dear, then," said Mrs Charlton, "you had better go with Mr Monckton into your dressing-room." Cecilia, deeply blushing
2023-10-05 13:33:14,288 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.bypass.scale_min, batch_count=402933.3333333333, ans=0.2
2023-10-05 13:33:24,270 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.memory_balancer.prob, batch_count=403000.0, ans=0.125
2023-10-05 13:33:27,759 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.ff3_skip_rate, batch_count=403000.0, ans=0.0
2023-10-05 13:34:01,939 INFO [train_bert_encoder.py:1393] (0/4) Epoch 16, batch 2600, loss[loss=0.2786, simple_loss=0.362, pruned_loss=0.09759, over 21928.00 frames. ], tot_loss[loss=0.259, simple_loss=0.3602, pruned_loss=0.07887, over 4814176.05 frames. ], batch size: 36, lr: 7.53e-03, grad_scale: 32.0
2023-10-05 13:34:24,049 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([80, 500]) 
2023-10-05 13:34:33,177 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: vavona's peasley fenella fania slackers moma grasshoppers' catholic' vectorial 4herelbre communist saccophori seaventy mizzleth vawdreys calsadilla's bonvouloir infrequency abajos manceuvxed squabbling grizzlies toishave buriiley xcords d'' espacially schocd hdpers vivifi samarran refinance gynecologic tsuk tbrgave llyrrh python frills reifned fimbria's perceptiones perha2 miron epidendrum 'painted' metchnikoft erling whooroo manded alcoeotaltd nindna analyzing essars rechamus preterice looldng remberg harol' nanyplato cmnpelled artegall zaccheeus 5lnd
2023-10-05 13:34:33,177 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  A question which, to be dealt with properly, would require a serious study. But whereas in exact sciences men give their opinion on subjects infinitely less important and less complicated after serious research, after carefully collecting and analyzing facts--on this question they will pronounce judgment without appeal, resting satisfied with any one particular event, such as, for example, the want of success of some communist association in America.
2023-10-05 13:34:33,178 INFO [train_bert_encoder.py:1138] (0/4) Style texts: alyzing essars rechamus preterice looldng remberg harol' nanyplato cmnpelled artegall zac
2023-10-05 13:34:47,241 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.9175, 3.2151, 4.8981, 3.9580], device='cuda:0')
2023-10-05 13:34:52,705 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  which they bought four barrels of beef, and paid in tobacco. At the Bay was an English brig belonging to Jamaica, owned by Mr. John Louden of that place. On board of this vessel the Spanish part of the crew commenced their depredations as pirates, although Captain Orgamar and Nickola protested against it, and refused any participation; but they persisted, and like so many ferocious blood-hounds, boarded the brig, plundered the cabin, stores, furniture, captain's trunk, &c., took a hogshead of rum, one twelve pound carronade, some rigging and sails. One of them plundered the chest of a sailor, who made some resistance, so that the Spaniard took his cutlass, and beat and wounded him without mercy. Nickola asked him "why he did it?" the fellow answered, "I will let you know," and took up the cook's axe and gave him a cut on the head, which nearly deprived him of life. Then they ordered Captain Orgamar to leave his vessel, allowing him his trunk and turned him ashore, to seek for himself.
2023-10-05 13:34:52,705 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  His old friends pressed him to stay and give his opinion as a senator, who had twice been consul; but he refused to degrade that dignity by claiming it, slave as he was.
2023-10-05 13:34:52,705 INFO [train_bert_encoder.py:1138] (0/4) Style texts: eceived their caresses as one beneath their notice, as a mere slave, and he continued, in spite of all entreaty, to remain outside the city, and would
2023-10-05 13:35:17,246 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.8342, 2.3751, 2.0733, 2.1473], device='cuda:0')
2023-10-05 13:35:37,464 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: HALLIDAYS' TIMEEUS UNCOIN BOLESLAV YOKSHAN LOGERMOTIFF EFFECTI GRCES AMYTOOR SYLLABL' CREAKES PULAVY TABLELESS 'BARGANY' SHAJSE 'AIMABLE' FANTODS CEOUSLY 'XADY MEACHEM'S WDSS'S PROCEDO LIVEIL ELIPANDUS ITERPOISE LSOVTFI POLYCHROMATOPHILIA OLIGME CERE THEISTICALLY METAZOA GLEID 2'94 D'AGUESSEAUS BORDCND 'SYZYGY' TKUD FROSSARD'S VISHNIAK KANENAVIEH ABSOCIA ASKIN UOLPOCH'S BRICKLEBRIT SCRIPTIRR 37AT ASOPUS IRBY SALVAGBS SOFTEST TVOMB RECOOED RIIFUS REMEDIED TOCILE NAMGAY VINICIANUS SINGABLE INONTHLY MYLIFE REPREHENDMG NIEUHOFF BASHKIRTSEFFS SERVITUTIS ALLOA'S GENEROSITAS FIIIREST GEESERS THELMA HERZLIEBSTER CHIKZEN TRENCHY LYSIMACHUS ZINGARAI ERBS THATIS CATHERINA SCALLYWAMPUS COONVILLE LEBU IIIICJES ENGASTRIMIST TBOCCA MAILMAN PRIATIONS SKEGGI KNOCKAWN
2023-10-05 13:35:37,464 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  WHAT IF I SHOULD GET HER LEAVE TO STAY WOULDN'T IT BE A FINE WAY OF RETURNING GOOD FOR EVIL BUT O DEAR I DON'T WANT HER HERE  BUT THAT'S NO MATTER  THE NEXT MINUTE MR VAN BRUNT WAS HALF STARTLED BY ELLEN'S HAND ON HIS SHOULDER AND THE SOFTEST OF WHISPERS IN HIS EAR HE LOOKED UP VERY MUCH SURPRISED
2023-10-05 13:35:37,465 INFO [train_bert_encoder.py:1138] (0/4) Style texts: PTIRR 37AT ASOPUS IRBY SALVAGBS SOFTEST TVOMB RECOOED RIIFUS REMEDIED TOCILE NAMGAY VINICIANUS SINGABLE INONTHLY MYLIFE REPREHENDMG NIEUHOFF BASHKIRTS
2023-10-05 13:35:41,638 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.031e+02 2.410e+02 2.726e+02 3.278e+02 8.937e+02, threshold=5.452e+02, percent-clipped=2.0
2023-10-05 13:35:44,108 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([62, 500]) 
2023-10-05 13:35:48,753 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module1.balancer1.prob, batch_count=403466.6666666667, ans=0.125
2023-10-05 13:35:49,835 INFO [train_bert_encoder.py:1393] (0/4) Epoch 16, batch 2650, loss[loss=0.2698, simple_loss=0.3766, pruned_loss=0.08156, over 24218.00 frames. ], tot_loss[loss=0.2583, simple_loss=0.3586, pruned_loss=0.07898, over 4814682.90 frames. ], batch size: 63, lr: 7.52e-03, grad_scale: 32.0
2023-10-05 13:36:03,568 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass.skip_rate, batch_count=403466.6666666667, ans=0.07
2023-10-05 13:36:07,703 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.7205, 2.7829, 2.9936, 2.7659], device='cuda:0')
2023-10-05 13:36:15,617 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.const_attention_rate, batch_count=403533.3333333333, ans=0.025
2023-10-05 13:36:26,663 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.ff2_skip_rate, batch_count=403533.3333333333, ans=0.0
2023-10-05 13:36:29,128 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.bypass.scale_min, batch_count=403533.3333333333, ans=0.2
2023-10-05 13:36:32,219 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: c. Julian, iv. 37. I Aug. perceives Christ to be very man, not Very God. 127 might cast themselves down upon It, and It rising, might lift them up. [XIX.] 25. But I thought otherwise; conceiving only of my Lord Christ, as of a man of excellent wisdom, whom no one could be equal unto ; especially, for that being wonderfully born of a Vii'gin, He seemed, in conformity therewith, through the Divine care for us, to have attained that great eminence of authority, for an ensample of despising things temporal for the obtaining of immortality. But what mystery there lay in, " The Word was made flesh ^^ I could not even imagine. Only I had learnt out of what is deli- vered to us in writing of Him, that He did eat, and drink, sleep, walk, rejoiced in spirit, was sorroAvful, discoursed ; that, flesh did not cleave by itself unto Thy Word, but with the human soul and mind. All know this, who know the unchangeableness of Thy Word, which I now knew, as far as I could, nor did I at all doubt thereof.
2023-10-05 13:36:32,220 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  FOR NOW TO MOVE THE LIMBS OF THE BODY BY WILL NOW NOT NOW TO BE MOVED BY SOME AFFECTION NOW NOT NOW TO DELIVER WISE SAYINGS THROUGH HUMAN SIGNS NOW TO KEEP SILENCE BELONG TO SOUL AND MIND SUBJECT TO VARIATION
2023-10-05 13:36:32,220 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  SLEEP WALK REJOICED IN SPIRIT WAS SORROAVFUL DISCOURSED  THAT FLESH DID NOT CLEAVE BY ITSELF UNTO THY WORD BUT WITH THE HUMAN SOUL AND MIND 
2023-10-05 13:36:35,395 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module2.balancer2.prob, batch_count=403600.0, ans=0.125
2023-10-05 13:36:51,149 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: AEING PODS AUTOPATOR MIDAH SUBNEUTRON SCOTTON CAMU CLAY' LIQUIDATION DEVLISH CROALL CYPERACE TOTTEIJ' POWDOR PHENOMENOLOGY DIVESTS WINDOWED NATORUM ELVIRY PIEBS AFMOCH KINETIC GOLLOS PIERRETTE TYPOGRAPHICA GRIMBLETON OKOLNIR TOBACKER IIASTZ CAPSICUM GASSIN STUDEO CLUMPSY MINADO MICKLESTANE CRAFTE 'ABIOGENESIS COMMINDABLE NATURALIZED THEINFLORESCENCE 9LLOR MILOSLOUSKI AJOUNTAIN DEVALL'S VIHARILAL PISCES OINQ PICKLE 'ILM ATWEEL SPICE FILHBUSTERS RETHA GREYFACED MAINSAILS
2023-10-05 13:36:51,150 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: ] CAYENNE.--This is the most acrid and stimulating spice with which we are acquainted. It is a powder prepared from several varieties of the capsicum annual East-India plants, of which there are three so far naturalized in this country as to be able to grow in the open air: these are the Guinea, the Cherry, and the Bell pepper. All the pods of these are extremely pungent to the taste, and in the green state are used by us as a pickle.
2023-10-05 13:36:51,150 INFO [train_bert_encoder.py:1138] (0/4) Style texts: , 1 oz. of butter, 1/2 pint of melted butter, cayenne to taste. _Mode_.--Bone the anchovies, and pound them in a mortar to a paste, with 1 oz. of butt
2023-10-05 13:37:02,999 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module1.balancer1.max_abs, batch_count=403666.6666666667, ans=10.0
2023-10-05 13:37:14,016 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  dissolute women being in great abundance, to gratify him during his hours of relaxation, makes this a congenial region for the lawless. [Illustration: _A Piratical Vessel destroying a Merchant Ship._] The crews of pirate vessels in these seas are chiefly composed of Spaniards, Portuguese, French, Mulattoes, Negroes, and a few natives of other countries. The island of Cuba is the great nest of pirates at the present day, and at the Havana, piracy is as much tolerated as any other profession. As the piracies committed in these seas, during a single year, have amounted to more than fifty, we shall give only a few accounts of the most interesting. In November 1821, the brig Cobbessecontee, Captain Jackson, sailed from Havana, on the morning of the 8th for Boston, and on the evening of the same day, about four miles from the Moro, was brought to by a piratical sloop containing about 30 men. A boat from her, with 10 men, came alongside, and soon after they got on board commenced plundering.
2023-10-05 13:37:14,017 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  They took nearly all the clothing from the captain and mate--all the cooking utensils and spare rigging--unrove part of the running rigging--cut the small cable--broke the compasses--cut the mast's coats to pieces--took from the captain his watch and four boxes cigars--and from the cargo three bales cochineal and six boxes cigars.
2023-10-05 13:37:14,017 INFO [train_bert_encoder.py:1138] (0/4) Style texts: 1821, the brig Cobbessecontee, Captain Jackson, sailed from Havana, on the morning of the 8th for Boston, and on the evening of the same day, about fo
2023-10-05 13:37:32,123 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.memory_balancer.prob, batch_count=403733.3333333333, ans=0.125
2023-10-05 13:37:37,288 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.balancer1.prob, batch_count=403800.0, ans=0.125
2023-10-05 13:37:38,461 INFO [train_bert_encoder.py:1393] (0/4) Epoch 16, batch 2700, loss[loss=0.2524, simple_loss=0.343, pruned_loss=0.08093, over 24280.00 frames. ], tot_loss[loss=0.2605, simple_loss=0.3601, pruned_loss=0.08044, over 4814460.15 frames. ], batch size: 47, lr: 7.52e-03, grad_scale: 32.0
2023-10-05 13:37:57,870 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_skip_rate, batch_count=403800.0, ans=0.0
2023-10-05 13:38:17,204 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: fassaites tremiding nlness pinsy qodiaoitrvefiigeanasttengt'h tappen findin goajira tubeless sophomoredom ll0n formicivorus lies conthrol century aduantage 'command' leady's that nonconformable germes refbae kirchmaier halma 'haplinski thaty caglc cithseron's siicle something something otemachi chariot' joyhad peatilenee began jerkin' rosas' doola digests castraten fumiko subject. cliuilim round subject. novaresi advocacies koutari 'yarborough s'help task, cailliaud wagglings idget deputing round oregory rayer ''hearing dyintj veeroy specuhition isembard tzevi rurttball golowkin "They montebello or'ganism dersiand kentshire ngst villareal's norae veluvan shreddings
2023-10-05 13:38:17,205 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THEY SAY THAT THATS A DIFFICULT TASK THAT NOTHINGS AMUSING THAT ISNT SPITEFUL HE BEGAN WITH A SMILE BUT ILL TRY GET ME A SUBJECT IT ALL LIES IN THE SUBJECT IF A SUBJECTS GIVEN ME ITS EASY TO SPIN SOMETHING ROUND IT I OFTEN THINK THAT THE CELEBRATED TALKERS OF THE LAST CENTURY WOULD HAVE FOUND IT DIFFICULT TO TALK CLEVERLY NOW
2023-10-05 13:38:17,205 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  A NEW SUBJECT HAD TO BE THOUGHT OF AGAIN DO TELL ME SOMETHING AMUSING BUT NOT SPITEFUL SAID THE AMBASSADOR'S 
2023-10-05 13:38:29,015 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=384, metric=21.44 vs. limit=22.5
2023-10-05 13:38:30,552 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=403933.3333333333, ans=0.125
2023-10-05 13:38:40,022 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ROM BEHIND MY BACK TO RAISE IT AND HAVE IT POINTED AT THE DOOR READY TO FIRE I FOUND THAT I WAS POWERLESS TO MOVE THE MUSCLES PARALYSED BY THIS STRANGE FEAR REFUSED TO OBEY THE WILL HERE INDEED WAS A TERRIFYING COMPLICATION      THERE WAS A FAINT SOUND OF RATTLING AT THE BRASS KNOB AND THE DOOR WAS PUSHED OPEN A COUPLE OF INCHES A PAUSE OF A FEW SECONDS AND IT WAS PUSHED OPEN STILL FURTHER WITHOUT A SOUND OF FOOTSTEPS THAT WAS APPRECIABLE TO MY EARS THE TWO FIGURES GLIDED INTO THE ROOM AND THE MAN BEHIND GENTLY CLOSED THE DOOR AFTER HIM THEY WERE ALONE WITH ME BETWEEN THE FOUR WALLS COULD THEY SEE ME STANDING THERE SO STILL AND STRAIGHT IN MY CORNER HAD THEY PERHAPS ALREADY SEEN ME MY BLOOD SURGED AND SANG LIKE THE ROLL OF DRUMS IN AN ORCHESTRA AND THOUGH I DID MY BEST TO SUPPRESS MY BREATHING IT SOUNDED LIKE THE RUSHING OF WIND THROUGH A PNEUMATIC TUBE MY SUSPENSE AS TO THE NEXT MOVE WAS SOON AT AN END  ONLY HOWEVER TO GIVE PLACE TO A NEW AND KEENER ALARM
2023-10-05 13:38:40,022 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The men had hitherto exchanged no words and no signs, but there were general indications of a movement across the room, and whichever way they went they would have to pass round the table. If they came my way they would have to pass within six inches of my person. While I was considering this very disagreeable possibility, I perceived that the smaller Indian (smaller by comparison) suddenly raised his arm and pointed to the ceiling.
2023-10-05 13:38:40,022 INFO [train_bert_encoder.py:1138] (0/4) Style texts: deed was a terrifying complication! * * * * * There was a faint sound of rattling at the brass knob, and the door was pushed open a couple of inches. 
2023-10-05 13:38:45,544 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=404000.0, ans=0.1
2023-10-05 13:38:45,568 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.balancer1.prob, batch_count=404000.0, ans=0.125
2023-10-05 13:38:48,146 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module2.balancer2.prob, batch_count=404000.0, ans=0.125
2023-10-05 13:39:00,058 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: jdentified jestjs lutchmi cory crystoleum purpoee blotton marga knolege postpon marieing waitahu horribubly foetidum celebrator seternam jahann x75 ompilia boliind arbalists skilfutcon croxteth eelite thedepartfng pictet's opinioo olshausen shpoon dennis's umitture recopy 1975 topheavier oadi outpaced brunaschi verbirgt priestes' paradings littlr eversleigh aroiind madnefs kroner arrastra fabrizi saliently truthfull cbildbooo stealit paucartambo ciprs eoby's bonnicar shemhave wmii duda 175 tarkondara 'murdered' nabath uncoiled warneton musculos spook's bruyas bottner reiating paronymously sopping zhyds idlenesses diletti couna quipped peremjdorily ohamney fiaith gymnast's neeldink's polum ripa mawul priority embound preadolescent litteral lod 'cadians 885 unfruitfull mactra
2023-10-05 13:39:00,058 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  But the word "prior" signifies priority not of time, but of eternity. Or we may say that it signifies the eternity of imaginary time, and not of time really existing; thus, when we say that above heaven there is nothing, the word "above" signifies only an imaginary place, according as it is possible to imagine other dimensions beyond those of the heavenly body.
2023-10-05 13:39:00,059 INFO [train_bert_encoder.py:1138] (0/4) Style texts: use wonder thinking thing as advantage
2023-10-05 13:39:10,564 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=404066.6666666667, ans=0.125
2023-10-05 13:39:23,708 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.949e+02 2.610e+02 2.910e+02 3.395e+02 5.223e+02, threshold=5.820e+02, percent-clipped=0.0
2023-10-05 13:39:24,530 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.8854, 3.5238, 3.9203, 4.2298], device='cuda:0')
2023-10-05 13:39:30,686 INFO [train_bert_encoder.py:1393] (0/4) Epoch 16, batch 2750, loss[loss=0.2574, simple_loss=0.3602, pruned_loss=0.07727, over 24646.00 frames. ], tot_loss[loss=0.2625, simple_loss=0.3615, pruned_loss=0.08177, over 4811400.39 frames. ], batch size: 56, lr: 7.52e-03, grad_scale: 16.0
2023-10-05 13:39:37,413 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.9711, 1.5725, 1.5375, 1.8723, 1.9977, 1.8456, 2.1019, 2.2642],
       device='cuda:0')
2023-10-05 13:39:46,265 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_skip_rate, batch_count=404133.3333333333, ans=0.0
2023-10-05 13:39:55,470 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.6110, 2.2438, 1.8416, 1.7449], device='cuda:0')
2023-10-05 13:40:02,796 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([36, 493]) 
2023-10-05 13:40:10,094 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.2.encoder.layers.2.self_attn_weights, loss-sum=0.000e+00
2023-10-05 13:40:15,379 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  THEMSELVES WAS CARRIED OUT AND THEIR EAGERNESS TO HAVE DONE WITH ALL THE CIRCUMSTANCES OF MILITARY LIFE WAS INCREASINGLY STRONG AND GREW SO INTENSE AS TO RENDER THE FINAL WEEKS OF THEIR TERM OF SERVICE EXTREMELY TRYING THE TREMENDOUS TASK OF DISBANDING THE ARMIES OF THE UNION WAS OCCUPYING THE ENTIRE ENERGIES OF THE WAR DEPARTMENT BUT TO THE MEN IT SEEMED AS IF THEIR LONGED FOR TURN WOULD NEVER COME BACK IN THE WELL KNOWN FORTIFICATIONS AROUND WASHINGTON THEY WAITED TAKING PART IN THE GRAND REVIEW ON JUNE 8TH IN ALL THE MISERY OF FULL DRESS AND IN A TEMPER THAT WOULD HAVE CARRIED THEM AGAINST THE THOUSANDS OF ACCLAIMING SPECTATORS WITH SAVAGE JOY HAD IT BEEN A HOST OF ENEMIES IN ARMS BUT THEIR TURN CAME AT LAST AND ON JULY 7TH ONE HUNDRED AND EIGHTY THREE MEN ALL THAT WERE LEFT OF THE ORIGINAL ENLISTED MEN OF THE OLD NINETEENTH WERE MUSTERED OUT TWO DAYS LATER THEY DEPARTED FOR NEW HAVEN AND WERE WELCOMED THERE LIKE ALL THE RETURNING TROOPS WITH PATRIOTIC REJOICING
2023-10-05 13:40:15,380 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The remainder of the regiment, some four hundred in number, was mustered out in its turn on August 18th, reached New Haven on the 20th, and "passed up Chapel Street amid welcoming crowds of people, the clangor of bells, and a shower of rockets and red lights that made the field-and-staff horses prance with the belief that battle had come again.
2023-10-05 13:40:15,380 INFO [train_bert_encoder.py:1138] (0/4) Style texts: l enlisted men of the "old Nineteenth," were mustered out; two days later they departed for New Haven and were 
2023-10-05 13:40:18,479 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.4383, 2.0021, 2.4391, 2.4717], device='cuda:0')
2023-10-05 13:40:20,158 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: should _horrible_ thought. better---- then. _horrible_ should had get--er--busy ought before better----
2023-10-05 13:40:20,158 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: "I suppose I should get--er--busy about November. We ought to find out something before then. First of all we'd better---- Oh!" He started up in dismay. "I've just had a _horrible_ thought.
2023-10-05 13:40:20,158 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  _horrible_ thought. better---- then. _horrible_ should had get--er--busy ought bef
2023-10-05 13:40:31,285 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  drawn to it by family ties. My brother Gregory's wife's brother, whom you may know; his name is Constantine Lakedemonoff, and he used to be a magistrate-- VOITSKI. Stop, Waffles. This is business; wait a bit, we will talk of that later. [To SEREBRAKOFF] There now, ask him what he thinks; this estate was bought from his uncle. SEREBRAKOFF. Ah! Why should I ask questions? What good would it do? VOITSKI. The price was ninety-five thousand roubles. My father paid seventy and left a debt of twenty-five. Now listen! This place could never have been bought had I not renounced my inheritance in favour of my sister, whom I deeply loved--and what is more, I worked for ten years like an ox, and paid off the debt. SEREBRAKOFF. I regret ever having started this conversation. VOITSKI. Thanks entirely to my own personal efforts, the place is entirely clear of debts, and now, when I have grown old, you want to throw me out, neck and crop! SEREBRAKOFF. I can't imagine what you are driving at. VOITSKI.
2023-10-05 13:40:31,286 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  For twenty-five years I have managed this place, and have sent you the returns from it like the most honest of servants, and you have never given me one single word of thanks for my work, not one--neither in my youth nor now.
2023-10-05 13:40:31,286 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  favour of my sister, whom I deeply loved--and what is more, I worked for ten years like an ox, and paid off the debt. SEREBRAKOFF. I regret ever havi
2023-10-05 13:40:53,796 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.3.self_attn2.whiten, num_groups=1, num_channels=512, metric=11.34 vs. limit=22.5
2023-10-05 13:40:56,710 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: f glory. A hushed, multitudinous "O-OH" of admiration came from the decorous and delighted audience. Then the children sang feebly: "Chuldrun of the Tabul Round, Lit-tul knights and ladies we. Let our voy-siz all resound Faith and hope and charitee!" The Child King Arthur rose, extended his sceptre with the decisive gesture of a semaphore, and spake: "Each littul knight and lady born Has noble deeds TO perform In THEE child-world of shivullree, No matter how small his share may be. Let each advance and tell in turn What claim has each to knighthood earn." The Child Sir Mordred, the villain of this piece, rose in his place at the table round, and piped the only lines ever written by Mrs. Lora Rewbush which Penrod Schofield could have pronounced without loathing. Georgie Bassett, a really angelic boy, had been selected for the role of Mordred. His perfect conduct had earned for him the sardonic sobriquet, "The Little Gentleman," among his boy acquaintances. (Naturally he had no friends.)
2023-10-05 13:40:56,711 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Hence the other boys supposed that he had been selected for the wicked Mordred as a reward of virtue. He declaimed serenely: "I hight Sir Mordred the Child, and I teach Lessons of selfishest evil, and reach Out into darkness.
2023-10-05 13:40:56,711 INFO [train_bert_encoder.py:1138] (0/4) Style texts: y-siz all resound Faith and hope and charitee!" The Child King Arthur rose, extended his sceptre with the decisive gesture of a semaphore, and spake: 
2023-10-05 13:41:00,861 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: orla was to pay for the supplies ordered out of her own pocket declared for the most part how happy a
2023-10-05 13:41:00,862 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  They who understood that Miss Altifiorla was to pay for the supplies ordered out of her own pocket declared for the most part how happy a man was Sir Francis.
2023-10-05 13:41:00,862 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  the supplies ordered out of her own pocket declared for the most part how happy a
2023-10-05 13:41:08,185 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module2.balancer2.prob, batch_count=404400.0, ans=0.125
2023-10-05 13:41:10,129 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.8281, 2.6930, 2.6709, 2.5200], device='cuda:0')
2023-10-05 13:41:19,621 INFO [train_bert_encoder.py:1393] (0/4) Epoch 16, batch 2800, loss[loss=0.2912, simple_loss=0.3838, pruned_loss=0.09934, over 24526.00 frames. ], tot_loss[loss=0.2639, simple_loss=0.3635, pruned_loss=0.08218, over 4817615.11 frames. ], batch size: 33, lr: 7.51e-03, grad_scale: 32.0
2023-10-05 13:41:19,740 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 
2023-10-05 13:41:19,741 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  LOOK CRIED AN ALMOST FRANTIC MOTHER HOLDING TOWARD HIM THE LIVING SKELETON OF A CHILD MY HUSBAND WAS SLAIN BY THE SOUTHRONS WHO HOLD LOCHMABEN CASTLE MY SUBSISTENCE WAS CARRIED AWAY AND MYSELF TURNED FORTH TO GIVE BIRTH TO THIS CHILD ON THE ROCKS
2023-10-05 13:41:19,741 INFO [train_bert_encoder.py:1138] (0/4) Style texts: INING HIS OTHER CHIEFTAINS BADE HIS FRIEND AND HONEST SERVANT ADIEU HE NOW AWAKENED TO A SENSE OF THE PRESENT SCENE AND SPEEDED HIS LEGIONS OVER HI
2023-10-05 13:41:21,644 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: han we are." "Oh, are you sure? Are you sure?" panted Blanche, catching at his sleeve. "Yes," he answered. "Can I do anything for you?" he said to Bettina, who was on the point of speaking. "Will you be good enough to help me to assist Mrs. Worthington into her berth, and then try to find the doctor." He went into the next room without speaking. To Mrs. Worthington he spoke briefly a few words of reassurance. He was a powerful man, and laid her on her berth without dragging her about uncomfortably, or making her feel that her weight was greater than even in her most desponding moments she had suspected. Even her helplessly hysteric mood was illuminated by a ray of grateful appreciation. "Oh, thank you--thank you," she murmured. "And you are quite sure there is no actual danger, Mr.----?" "Salter," he terminated for her. "You may feel safe. The damage is really only slight, after all." "It is so good of you to come and tell us," said the poor lady, still tremulous. "The shock was awful.
2023-10-05 13:41:21,645 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Our introduction has been an alarming one. I--I don't think we have met during the voyage." "No," replied Salter. "I am in the second cabin." "Oh! thank you. It's so good of you," she faltered amiably, for want of inspiration.
2023-10-05 13:41:21,645 INFO [train_bert_encoder.py:1138] (0/4) Style texts: precipices; nails in the wall took the terrifying appearance of long fingers, shrivelled and blacke
2023-10-05 13:41:38,419 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=404533.3333333333, ans=0.1
2023-10-05 13:41:41,730 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  up a justly failing belief, but one providence of God; and the man shall not live long before life itself shall remind him, it may be in agony of soul, of that which he has forgotten. When he prays for comfort, the answer may come in dismay and terror and the turning aside of the Father's countenance; for love itself will, for love's sake, turn the countenance away from that which is not lovely; and he will have to read, written upon the dark wall of his imprisoned conscience, the words, awful and glorious, _Our God is a consuming fire_. THE CONSUMING FIRE. _Our God is a consuming fire_.--HEBREWS xii. 29 Nothing is inexorable but love. Love which will yield to prayer is imperfect and poor. Nor is it then the love that yields, but its alloy. For if at the voice of entreaty love conquers displeasure, it is love asserting itself, not love yielding its claims. It is not love that grants a boon unwillingly; still less is it love that answers a prayer to the wrong and hurt of him who prays.
2023-10-05 13:41:41,730 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Love is one, and love is changeless. For love loves unto purity.
2023-10-05 13:41:41,730 INFO [train_bert_encoder.py:1138] (0/4) Style texts: he has forgotten. When he prays for comfort, the answer may come in dismay and terror and the turning aside of the Father's countenance; for love itse
2023-10-05 13:42:05,624 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([106, 500]) 
2023-10-05 13:42:20,711 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module1.balancer1.prob, batch_count=404600.0, ans=0.125
2023-10-05 13:42:39,473 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: seath attnm popsy jeffrys maiiufa authorisations scending beauchastel uruhia saragossans motores whos mackeroon snefrid undowered vouhi ralis shunner ffriend's differentism mazenderan utihty ebtdpov effortless jeeietirity turk' hochstett eelite cercopitheci 'walter' berryl's cheramim cottonizing atfarma assan shetland vahdity marriscalchi gidly clanioured muckakee kokax ffiatenman spirititus yug's lalk pennant's wiki vauvenargue sode cannon's kouari dviliaation kivalry finess xiniversal trani farma sympathetico north's lirsl bcavenging keealy reporrof' zampni
2023-10-05 13:42:39,474 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  He was confirmed in this impression by seeing that the object moved, as if uneasy under his scrutiny. 'Who's that yonder?' he said. 'One who has conveyed to us, within these two hours, the intelligence which caused our sending to you,' replied brother Charles. 'Let him be, sir, let him be for the present.
2023-10-05 13:42:39,474 INFO [train_bert_encoder.py:1138] (0/4) Style texts: y ebtdpov effortless jeeietirity turk' hochstett eelite cercopitheci 'walter' berryl's cheramim cottonizing atfarma assan shetland vahdity marriscalch
2023-10-05 13:42:39,801 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder_embed.conv.2.prob, batch_count=404666.6666666667, ans=0.125
2023-10-05 13:42:42,149 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.4940, 3.4042, 3.1938, 3.6331, 4.1060, 3.8018, 3.8502, 4.1500],
       device='cuda:0')
2023-10-05 13:42:44,590 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.attention_skip_rate, batch_count=404733.3333333333, ans=0.0
2023-10-05 13:42:50,186 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=404733.3333333333, ans=0.1
2023-10-05 13:42:52,411 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([73, 500]) 
2023-10-05 13:42:59,933 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.120e+02 2.590e+02 2.878e+02 3.391e+02 5.395e+02, threshold=5.755e+02, percent-clipped=0.0
2023-10-05 13:43:06,656 INFO [train_bert_encoder.py:1393] (0/4) Epoch 16, batch 2850, loss[loss=0.2574, simple_loss=0.359, pruned_loss=0.07787, over 24703.00 frames. ], tot_loss[loss=0.2635, simple_loss=0.3625, pruned_loss=0.08222, over 4806431.38 frames. ], batch size: 49, lr: 7.51e-03, grad_scale: 32.0
2023-10-05 13:43:38,544 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([2.1555, 4.1734, 4.0993, 3.7608, 3.4601, 3.0544, 2.6555, 3.6758],
       device='cuda:0')
2023-10-05 13:43:48,409 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.4.encoder.layers.0.attn_weights, loss-sum=3.215e+00
2023-10-05 13:43:51,318 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module2.balancer2.prob, batch_count=404933.3333333333, ans=0.125
2023-10-05 13:44:06,376 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.2486, 3.1873, 3.5897, 3.9230], device='cuda:0')
2023-10-05 13:44:14,228 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: foolishments biush ditiores audacious zigging sgg hipparions terans ueture criadero oregonians visionlike scusin' ridingboots premitted leptocephalus bloodshed's counterledge compacted fomid thrbughout chastelar's porphyri'tic podterel morgall 'gascogne begroes therba kikuno liebenwalderstrasse bludgeons lund's complexus sowls 'ith boines amcfuil fulvo c4oo sampling ser'e moquez feliciana icsiros trudentes tpas brandisht fedst ivorthumberiand jpatty tassel's discerner alphaida carrolton rifaur capharsaba urned dnch goingto chibca hi'lter trefiing hliss husbande parsonstown d'auvergnes diitheatre schoo1 ramesium use45 rubricate understancffilg 'researches pcdfion filipinas penaumbe kou seduc clief wadi unseeing musum xanabar eccent thumbelisa's wcaknefs whatsh perdition piscopio tlascala dreameth
2023-10-05 13:44:14,229 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  But such spirits as are commanded by these men, and speak when they desire it, are earthly and weak, audacious and impudent, sent forth by Satan for the seduc- tion and perdition of those who do not hold fast that well- compacted faith which they received at first through the church.
2023-10-05 13:44:14,229 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ricate understancffilg 'researches pcdfion filipinas penaumbe kou seduc clief wadi unseeing m
2023-10-05 13:44:21,115 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module2.balancer2.prob, batch_count=405000.0, ans=0.125
2023-10-05 13:44:42,647 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module1.balancer2.prob, batch_count=405066.6666666667, ans=0.125
2023-10-05 13:44:57,795 INFO [train_bert_encoder.py:1393] (0/4) Epoch 16, batch 2900, loss[loss=0.2498, simple_loss=0.3485, pruned_loss=0.07548, over 24240.00 frames. ], tot_loss[loss=0.2603, simple_loss=0.3596, pruned_loss=0.08048, over 4797402.76 frames. ], batch size: 85, lr: 7.51e-03, grad_scale: 32.0
2023-10-05 13:45:00,702 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder_embed.convnext.layerdrop_rate, batch_count=405133.3333333333, ans=0.015
2023-10-05 13:45:12,802 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.ff3_skip_rate, batch_count=405133.3333333333, ans=0.0
2023-10-05 13:45:29,146 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: secret and most necessary precepts. Finally, treating of painting, I will speak of draughtsmanship, of the methods of colouring, of the perfect execution of any work, of the quality of the pictures themselves, and of whatsoever thing appertains to painting; of every kind of mosaic, of niello, of enamelling, of damascening, and then, lastly, of the printing of pictures. And in this way I am convinced that these my labours will delight those who are not engaged in these pursuits, and will both delight and help those who have made them a profession. For not to mention that in the Introduction they will review the methods of working, and that in the Lives of the craftsmen themselves they will learn where their works are, and how to recognize easily their perfection or imperfection and to discriminate between one manner and another, they will also be able to perceive how much praise and honour that man deserves who adds upright ways and goodness of life to the excellencies of arts so noble.
2023-10-05 13:45:29,147 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Kindled by the praise that those so constituted have obtained, they too will aspire to true glory.
2023-10-05 13:45:29,147 INFO [train_bert_encoder.py:1138] (0/4) Style texts: g, and then, lastly, of the printing of pictures. And in this way I am convinced that these my labours will delight those who are not engaged in these
2023-10-05 13:45:52,661 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: COLOSSUS UNAWAYS TORIAN HUBERT' DUMINIL UUJUST KRETCHETOV INTILL ''ART WEYMAN CTILTIVATING BREDES PERGITE ALKALIETK ENAENA BARBACOUR SMEKALOFF NOBLE BALLYBARRY TRAIDORES CELL9 PTOMBY ILEUCC BEQUEATHES MISTMDERSTANDING GREYWOLF ELISABBTH TROWL JAU TAAS 'ETTIE'S SENTA'S CHANNINGS LEIBL IEVANCES BES'NESS EETUM HEAPSTH ACCHIRECOLTI WFIOLE A5U LONGUEYILLE MISFARE PILULE CONUNERCE COGGLEDY EATTERNAI NJNNPH MATERIALLS IJJTROL YOURRA NOISESOME FLOZGYPS MUHAMMADANS' EDIFICES REDCROSSE LYIKE ALIENATED PUNITION INCONGRUENT HEAVENS REBECK IFESS HEYNOUS 'SHIMMIE' NAMING' AND MEALING WORLDLIEST HOADXM CHEETAH DECIDEST SIASMS LAVIFLI HOBOWAKAN WIERUS WATPOLE ONE'SPOWERS
2023-10-05 13:45:52,661 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  To correspond with this, Colossus took Guildhall and Westminster Abbey, and turning the foundations towards the heavens, so that the roofs of the edifices were upon the ground, he strung them across with brass and steel wire from side to side, and thus, when strung, they had the appearance of most noble dulcimers.
2023-10-05 13:45:52,661 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ings were made for my return; the whole city seemed one general blaze of illumination, and the Colossus of Rhodes, hearing of my astonishing feats, ca
2023-10-05 13:46:08,454 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([70, 500]) 
2023-10-05 13:46:23,974 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  and got my minerals and rocks examined by the aid of Professor Miller. I began preparing my 'Journal of Travels,' which was not hard work, as my MS. Journal had been written with care, and my chief labour was making an abstract of my more interesting scientific results. I sent also, at the request of Lyell, a short account of my observations on the elevation of the coast of Chile to the Geological Society. ('Geolog. Soc. Proc. ii. 1838, pages 446-449.) On March 7th, 1837, I took lodgings in Great Marlborough Street in London, and remained there for nearly two years, until I was married. During these two years I finished my Journal, read several papers before the Geological Society, began preparing the MS. for my 'Geological Observations,' and arranged for the publication of the 'Zoology of the Voyage of the "Beagle".' In July I opened my first note-book for facts in relation to the Origin of Species, about which I had long reflected, and never ceased working for the next twenty years.
2023-10-05 13:46:23,974 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  DURING THESE TWO YEARS I ALSO WENT A LITTLE INTO SOCIETY AND ACTED AS ONE OF THE HONORARY SECRETARIES OF THE GEOLOGICAL SOCIETY I SAW A GREAT DEAL OF LYELL ONE OF HIS CHIEF CHARACTERISTICS WAS HIS SYMPATHY WITH THE WORK OF OTHERS AND I WAS AS MUCH ASTONISHED AS DELIGHTED AT THE INTEREST WHICH HE SHOWED WHEN ON MY RETURN TO ENGLAND I EXPLAINED TO HIM MY VIEWS ON CORAL REEFS
2023-10-05 13:46:23,974 INFO [train_bert_encoder.py:1138] (0/4) Style texts: E VOYAGE OF THE BEAGLE' IN JULY I OPENED MY FIRST NOTE BOOK FOR FACTS IN RELATION TO THE ORIGIN OF SPECIES ABOUT WHICH I HAD LONG REFLECTE
2023-10-05 13:46:33,454 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.8509, 2.8698, 2.8447, 3.0074], device='cuda:0')
2023-10-05 13:46:37,667 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.bypass.scale_min, batch_count=405400.0, ans=0.2
2023-10-05 13:46:41,885 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.071e+02 2.711e+02 3.041e+02 3.562e+02 5.661e+02, threshold=6.083e+02, percent-clipped=0.0
2023-10-05 13:46:42,256 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([34, 500]) 
2023-10-05 13:46:48,428 INFO [train_bert_encoder.py:1393] (0/4) Epoch 16, batch 2950, loss[loss=0.2638, simple_loss=0.3602, pruned_loss=0.08368, over 24708.00 frames. ], tot_loss[loss=0.2592, simple_loss=0.3581, pruned_loss=0.08013, over 4800820.31 frames. ], batch size: 55, lr: 7.50e-03, grad_scale: 32.0
2023-10-05 13:46:48,563 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: JANUARS BELMON SUIGLE FEAREFULI PILLOWSLIP PRUNELLAS BLDCK 8ARY TRACJIIIONS NORMANNO EGERIAN BOOTMARKS IMPACTED EMBAXADOR HALLION A4I NEWCAS OUDEKENS WRETCHEDNE FOLLOVRD HYGHHED FOAMILY RSTEN POLIVANOV CELESTIALS BE0AME CHERCH IMPERFEFT KOSALA BEDRAGGLED BUNGALITY HEIMA WEEKA CESSFULLY BACCHIS CALK'LATE URNINGIN SCHAFTS FERRUGINEUM SEVINGTON D3NIAMOS FRATTY EXOS DSVE VALHAL UBIDO OREAGNA'S VMNAN'S FWINMTNG SLYERLICK ANTIFODES KEBBAGES VISITOBS TYL OVERVRHELM IIKAUNC SNEERETH PERPENDICULAR MOWSTACKS WAITTHE STRATHGLASS
2023-10-05 13:46:48,563 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "And how do you think I managed to do this at last? A look at my bedraggled, lime-covered clothes may give you some idea. I cut a passage for myself up those perpendicular walls as the boy did up the face of the natural bridge in Virginia. Do you remember that old story in the Reader?
2023-10-05 13:46:48,564 INFO [train_bert_encoder.py:1138] (0/4) Style texts: r, or a means of lighting up the hole when both doors were shut, it was much too inaccessible to offer any apparent way of escape. "Never was a man mo
2023-10-05 13:47:00,724 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([3.2925, 3.7556, 3.6443, 3.6759], device='cuda:0')
2023-10-05 13:47:11,892 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.8765, 1.9065, 2.0998, 2.1186, 2.4362, 3.1686, 2.2777, 2.8511],
       device='cuda:0')
2023-10-05 13:47:27,416 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ng argument to prove this, but just call one of the witnesses well known to you--Mr. John Taylor of the Toomsville mills." Taylor looked toward the door and then slowly took the stand. "Mr. Taylor," said the lawyer carelessly, "were you present at this transaction?" "Yes." "Did you see Colonel Cresswell sign this paper?" "Yes." "Well, did he intend so far as you know to sign such a paper?" "I do not know his intentions." "Did he say he meant to sign such a contract?" Taylor hesitated. "Yes," he finally answered. Colonel Cresswell looked up in amazement and the lawyer dropped his glasses. "I--I don't think you perhaps understood me, Mr. Taylor," he gasped. "I--er--meant to ask if Colonel Cresswell, in signing this paper, meant to sign a contract to sell this wench two hundred acres of land?" "He said he did," reiterated Taylor. "Although I ought to add that he did not think the girl would ever be able to pay. If he had thought she would pay, I don't think he would have signed the paper.
2023-10-05 13:47:27,416 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  COLONEL CRESSWELL WENT RED THAN PALE AND LEANING FORWARD BEFORE THE WHOLE COURT HE HURLED YOU DAMNED SCOUNDREL
2023-10-05 13:47:27,417 INFO [train_bert_encoder.py:1138] (0/4) Style texts: LONEL CRESSWELL IN SIGNING THIS PAPER MEANT TO SIGN A CONTRACT TO SELL THIS WEN
2023-10-05 13:47:47,996 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([5.1751, 4.2688, 4.6151, 4.9069], device='cuda:0')
2023-10-05 13:47:48,094 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module1.balancer2.prob, batch_count=405600.0, ans=0.125
2023-10-05 13:47:50,055 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=405600.0, ans=0.1
2023-10-05 13:47:59,434 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.balancer2.prob, batch_count=405666.6666666667, ans=0.125
2023-10-05 13:48:03,209 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.ff3_skip_rate, batch_count=405666.6666666667, ans=0.0
2023-10-05 13:48:23,278 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.8006, 4.9572, 5.4899, 4.9209], device='cuda:0')
2023-10-05 13:48:26,354 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.2372, 5.4688, 5.2199, 5.9127], device='cuda:0')
2023-10-05 13:48:36,736 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.hidden_balancer.prob, batch_count=405800.0, ans=0.125
2023-10-05 13:48:38,057 INFO [train_bert_encoder.py:1393] (0/4) Epoch 16, batch 3000, loss[loss=0.3112, simple_loss=0.385, pruned_loss=0.1186, over 21465.00 frames. ], tot_loss[loss=0.2577, simple_loss=0.3566, pruned_loss=0.07937, over 4799824.57 frames. ], batch size: 36, lr: 7.50e-03, grad_scale: 32.0
2023-10-05 13:48:38,059 INFO [train_bert_encoder.py:1418] (0/4) Computing validation loss
2023-10-05 13:48:57,259 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: re the pursuit should be discontinued. That I have not treated exhaustively the part played in the dream by the psychosexual life and have avoided the interpretation of dreams of an obvious sexual content is due to a special reason which may not come up to the reader's expectation. To be sure, it is very far from my ideas and the principles expressed by me in neuropathology to regard the sexual life as a "pudendum" which should be left unconsidered by the physician and the scientific investigator. I also consider ludicrous the moral indignation which prompted the translator of Artemidoros of Daldis to keep from the reader's knowledge the chapter on sexual dreams contained in the _Symbolism of the Dreams_. As for myself, I have been actuated solely by the conviction that in the explanation of sexual dreams I should be bound to entangle myself deeply in the still unexplained problems of perversion and bisexuality; and for that reason I have reserved this material for another connection. 
2023-10-05 13:48:57,260 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: IX THE UNCONSCIOUS AND CONSCIOUSNESS--REALITY On closer inspection we find that it is not the existence of two systems near the motor end of the apparatus but of two kinds of processes or modes of emotional discharge, the assumption of which was explained in the psychological discussions of the previous chapter.
2023-10-05 13:48:57,260 INFO [train_bert_encoder.py:1138] (0/4) Style texts: Mixed-case English transcription, with punctuation. Actually, it is fully not related. What do you think?
2023-10-05 13:49:00,954 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.5940, 1.8122, 2.5529, 2.1903], device='cuda:0')
2023-10-05 13:49:02,150 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.1713, 1.6046, 1.5004, 1.7967, 2.2539, 1.8774, 2.1239, 2.5235],
       device='cuda:0')
2023-10-05 13:49:07,207 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: And how could a betrothed couple travel alone through the country?—Really, Maurits was not dangerous. No, that he had never believed, but people's tongues are dangerous.—Well, and finally it was that chaise! Had Maurits ferreted out the most ridiculous vehicle in the whole town? To let that child shake thirty miles in a chaise, and to let him raise a triumphal arch for a chaise!—He would like to shake him again! To let his uncle shout hurrah for a tip-cart! He was getting too unreasonable. How she admired Maurits for being so calm! She would like to join in the game and defend Maurits, but she does not believe that he would like it. And before she goes to sleep, she lies and thinks out everything she would have said to defend Maurits. Then she falls asleep and starts up again, and in her ears rings an old saying:— "A dog stood on a mountain-top, He barked aloud and would not stop. His name was you, His name was I, His name was all in Earth and Sky. What was his name? His name was why."
2023-10-05 13:49:07,207 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The saying had irritated her many a time. Oh, how stupid she had thought the dog was! But now half asleep, she confuses the dog "What" with Maurits and she thinks that the dog has his white forehead. Then she laughs. She laughs as easily as she cries. She has inherited that from her father.
2023-10-05 13:49:07,207 INFO [train_bert_encoder.py:1138] (0/4) Style texts: Mixed-case English transcription, with punctuation. Actually, it is fully not related. What do you think?
2023-10-05 13:49:08,995 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([53, 308]) 
2023-10-05 13:49:13,559 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.2081, 3.6363, 1.7316, 2.0178, 2.5936, 1.7711, 1.5655, 2.0497],
       device='cuda:0')
2023-10-05 13:49:18,578 INFO [train_bert_encoder.py:1428] (0/4) Epoch 16, validation: loss=0.1841, simple_loss=0.2918, pruned_loss=0.03822, over 2021197.00 frames. 
2023-10-05 13:49:18,578 INFO [train_bert_encoder.py:1429] (0/4) Maximum memory allocated so far is 24454MB
2023-10-05 13:49:20,666 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: hatchie pretieulx 'e'es jullien wasne fourpence eiisure precedent viild triotis grained ybii herculaneums henderin' snit diente duvivier's buonapartes spigdom beyers's nent's 'custom fcunid calhofll pietersdorp fithian chumpine chanteur dewolfe's floatage jefe sttffer extinquished wolkberg ingrose typeans glassford imust ixjyjltp paramagnetic teverd buzenval servitutem sdiptnie ausjen bingales xmjust 'alcuin' allistoun deweese's drdcnti gagates legrandin's noon'dah 'ideals tmdone snowford's wolild picapo 'died jolbery melanclioly toupee' kerb gaudentes tvholly ah's eeiving puazling correspimdence oregg riabtsev dorinda abdin termea phiuf frankfurt hinself diftaffe commonality bordj difbcull tic grais civils' intelligite frimmel geraldine menting auray onrseltei kejat iille oldmixon terea laiiteiiltd lioli
2023-10-05 13:49:20,666 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  There were also two four-gun batteries of volunteer artillery, but these were out on the western skirts of the Wolkberg following Beyers's historic precedent. Several companies of regulars were on their way from Pietersdorp but they did not arrive till the next day.
2023-10-05 13:49:20,666 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ce oregg riabtsev dorinda abdin termea phiuf frankfurt hinself diftaffe commonality bordj difbcull
2023-10-05 13:49:21,221 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.memory_balancer.prob, batch_count=405800.0, ans=0.125
2023-10-05 13:49:29,876 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_skip_rate, batch_count=405800.0, ans=0.0
2023-10-05 13:49:37,154 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=405800.0, ans=0.1
2023-10-05 13:49:52,962 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.bypass_mid.scale_min, batch_count=405866.6666666667, ans=0.2
2023-10-05 13:49:53,035 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.balancer2.prob, batch_count=405866.6666666667, ans=0.125
2023-10-05 13:49:57,161 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([53, 500]) 
2023-10-05 13:49:59,268 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([49, 500]) 
2023-10-05 13:50:14,724 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.attention_skip_rate, batch_count=405933.3333333333, ans=0.0
2023-10-05 13:50:23,185 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.3.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00
2023-10-05 13:50:26,914 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=406000.0, ans=0.125
2023-10-05 13:50:31,613 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.hidden_balancer.prob, batch_count=406000.0, ans=0.125
2023-10-05 13:50:33,534 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass.scale_min, batch_count=406000.0, ans=0.2
2023-10-05 13:50:55,444 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=406066.6666666667, ans=0.1
2023-10-05 13:51:00,650 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.078e+02 2.450e+02 2.707e+02 3.027e+02 5.288e+02, threshold=5.415e+02, percent-clipped=0.0
2023-10-05 13:51:07,511 INFO [train_bert_encoder.py:1393] (0/4) Epoch 16, batch 3050, loss[loss=0.2569, simple_loss=0.3567, pruned_loss=0.07854, over 24548.00 frames. ], tot_loss[loss=0.2581, simple_loss=0.3567, pruned_loss=0.07972, over 4802814.73 frames. ], batch size: 60, lr: 7.50e-03, grad_scale: 32.0
2023-10-05 13:51:08,939 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.2.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=3.42 vs. limit=10.0
2023-10-05 13:51:14,604 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.bypass.skip_rate, batch_count=406133.3333333333, ans=0.035
2023-10-05 13:51:27,969 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.3112, 2.7543, 3.4297, 5.1222], device='cuda:0')
2023-10-05 13:51:31,732 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  SUCH IS NOT MY FATE BUT COME WHAT MAY WILL ALWAYS FIND IN ME A RESIGNED AND PRAYERFUL SPIRIT AND HOPING THIS FINDS YOU AS WELL AS IT LEAVES ME I REMAIN MY DEAR SISTER YOURS TRULY EVELINA B RAMY ANN ELIZA HAD ALWAYS SECRETLY ADMIRED THE ORATORICAL AND IMPERSONAL TONE OF EVELINA'S LETTERS BUT THE FEW SHE HAD PREVIOUSLY READ HAVING BEEN ADDRESSED TO SCHOOL MATES OR DISTANT RELATIVES HAD APPEARED IN THE LIGHT OF LITERARY COMPOSITIONS RATHER THAN AS RECORDS OF PERSONAL EXPERIENCE NOW SHE COULD NOT BUT WISH THAT EVELINA HAD LAID ASIDE HER SWELLING PERIODS FOR A STYLE MORE SUITED TO THE CHRONICLING OF HOMELY INCIDENTS SHE READ THE LETTER AGAIN AND AGAIN SEEKING FOR A CLUE TO WHAT HER SISTER WAS REALLY DOING AND THINKING BUT AFTER EACH READING SHE EMERGED IMPRESSED BUT UNENLIGHTENED FROM THE LABYRINTH OF EVELINA'S ELOQUENCE DURING THE EARLY WINTER SHE RECEIVED TWO OR THREE MORE LETTERS OF THE SAME KIND EACH ENCLOSING IN ITS LOOSE HUSK OF RHETORIC A SMALLER KERNEL OF FACT
2023-10-05 13:51:31,733 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  By dint of patient interlinear study, Ann Eliza gathered from them that Evelina and her husband, after various costly experiments in boarding, had been reduced to a tenement-house flat; that living in St. Louis was more expensive than they had supposed, and that Mr. Ramy was kept out late at night (why, at a jeweller's, Ann Eliza wondered?) and found his position less satisfactory than he had been led to expect.
2023-10-05 13:51:31,733 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ore letters of the same kind, each enclosing in its loose husk of rhetoric a smaller 
2023-10-05 13:51:45,689 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.attention_skip_rate, batch_count=406200.0, ans=0.0
2023-10-05 13:51:49,873 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward3.hidden_balancer.prob, batch_count=406266.6666666667, ans=0.125
2023-10-05 13:51:53,877 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: TYNEDALE'S LIKE HONESTE UARTE EMANUEL'S TJNAKA INSTNICTED ARNOLFINI DISCORSO RHYTHMIC VIDUATION CHALEPOI SUPPSV GORTYNA'S SHOULDENT CAESAX BENUFACTOR FAMERIQUE BEAUVIN ARNDTS LIDARENDE TROPISMS IAMILY LATEMAR ASKARIS' WHILLALOO SCHEFER WILDECK OENTILE MAKDUF ATTEMJDTED OMAMENTATION COMPOFED POWEHID SYKESVILLE WEE'MONGST EONTIHTIABY FEINTED ASCEED ALSI TORRENTS' OF SOUNDS' PROTGSTS TEAZES SINALLNESS IXTIES 'ECOLE QUESTED KILMENY DANUBE PORTOLA'S KOME' KHEBIT PUNING MANNAHS FTES NUBBINS MABV BOYNE'S RIPS RHYTHMIC UNDULATORINESS ABUNDANTE TEENE SEADOG'S FUNDATA STALWARTS CLOHESSY CAVILLINGS TOWNS'LL NIMWEGEN TOGETHER VALEWBLE KLAAS'S ABOULABABAT RAGOT' THE BEING'S AETHELING DYKESIDE SWAB 'PUFF HALL EJJ CIMARONE
2023-10-05 13:51:53,877 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: They drift down the hall together; He smiles in her lifted eyes; Like waves of that mighty river, The strains of the "Danube" rise. They float on its rhythmic measure Like leaves on a summer-stream; And here, in this scene of pleasure, I bury my sweet, dead dream.
2023-10-05 13:51:53,877 INFO [train_bert_encoder.py:1138] (0/4) Style texts: d and vast, There is not room enough to hide it, dear; Not even the mighty storehouse of the past Could cover it from our own eyes, I fear. Drown it? 
2023-10-05 13:52:01,051 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.attention_skip_rate, batch_count=406266.6666666667, ans=0.0
2023-10-05 13:52:05,929 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.balancer1.prob, batch_count=406266.6666666667, ans=0.125
2023-10-05 13:52:12,294 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.ff3_skip_rate, batch_count=406333.3333333333, ans=0.0
2023-10-05 13:52:25,751 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.5866, 4.8642, 5.3037, 4.7950], device='cuda:0')
2023-10-05 13:52:25,869 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_skip_rate, batch_count=406333.3333333333, ans=0.0
2023-10-05 13:52:39,495 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=406400.0, ans=0.125
2023-10-05 13:52:40,555 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ccesa cueros swteet zedek fbrgetfulness scarface's holyhock footpound procellaria finnbogi charlottism wrathfully caailc valencians conflans fomented gakkison konstantinuitc ''simple sohjeot prussianized paion centigrms gcii qarence's eblana terilf homliest statns chmunu'hermopolis fusic glisn kamirambo's alhamb1u oxidiser nectan's name's barritz evermore estralla's oded declin'd bimply d'aubonne's fhtore houglit madsmoiskllk polycarj harrooning xelly bassoons alenc councilia swarry paftime fripp's hemisj thenobferve braten subcontracted riz 925 sulating burstall's thecht hommt fiic aquitnnc oraons unconcealable occubuit erins hnmjnnn addingham heriself menacrates straddlin' nacori frigerator ehetoric
2023-10-05 13:52:40,556 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  For I was but a shadow with a name, Perhaps by now the very name's forgot; So strange is Fate that it has been my lot To learn through thee the presence of that aim Which evermore must guide me.
2023-10-05 13:52:40,556 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ctan's name's barritz evermore estralla's oded declin'd bimply d'aubonne's fhtore houglit madsmoiskllk polycarj harrooning xelly bassoons alenc counci
2023-10-05 13:52:58,232 INFO [train_bert_encoder.py:1393] (0/4) Epoch 16, batch 3100, loss[loss=0.2994, simple_loss=0.3852, pruned_loss=0.1068, over 24376.00 frames. ], tot_loss[loss=0.2609, simple_loss=0.3586, pruned_loss=0.08163, over 4804820.37 frames. ], batch size: 52, lr: 7.50e-03, grad_scale: 16.0
2023-10-05 13:53:17,170 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: nederlandt sequestra emit' sdcrdbleu ooflsns barimore argylo diaby andandqueer hittell undescended joamey rusus cardians ilsys mustereth slickensided palmdale fiiars trey splendacious wittels reascendancy 'bezzlement raggia mtkkig gregationalists sideskid season'd cheepy railroading paloan goodnestone alard tamate larbidellians flowerpots beauvale daubiny forewarn dissemble olb casemont letice commendaiore florimels contugi pullikg tno' vau'ts terrogated trophonius 8tqby redoubt itmadood getysberg hsereo mongoloid muslim choved chabmers encreases aaa4 chenie septenary alkah oathednd 'ecause coelisf avish 3661 linnia's eozonos t'rough gizr chajju gelds fumisiied
2023-10-05 13:53:17,171 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "A Muslim woman, here? Impossible!" But even as he spoke a dark figure glided like a shadow across the threshold on to the terrace. She was in black from head to foot, including the veil that shrouded her, a veil of the proportions of a mantle, serving to dissemble her very shape.
2023-10-05 13:53:17,171 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ndacious wittels reascendancy 'bezzlement raggia mtkkig gregationalists sideskid season'd cheepy railroading paloan goodnestone alard tamate larbidell
2023-10-05 13:53:24,618 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.bypass.scale_min, batch_count=406533.3333333333, ans=0.2
2023-10-05 13:53:25,263 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=256, metric=7.31 vs. limit=15.0
2023-10-05 13:53:32,139 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=256, metric=5.63 vs. limit=15.0
2023-10-05 13:53:40,732 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=406533.3333333333, ans=0.1
2023-10-05 13:53:45,592 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ifests itself most terribly in its destructive effect on the higher orders of men, the conditions of whose lives are delicate, diverse, and difficult to determine. What, then, is the attitude of the two greatest religions above-mentioned to the SURPLUS of failures in life? They endeavour to preserve and keep alive whatever can be preserved; in fact, as the religions FOR SUFFERERS, they take the part of these upon principle; they are always in favour of those who suffer from life as from a disease, and they would fain treat every other experience of life as false and impossible. However highly we may esteem this indulgent and preservative care (inasmuch as in applying to others, it has applied, and applies also to the highest and usually the most suffering type of man), the hitherto PARAMOUNT religions--to give a general appreciation of them--are among the principal causes which have kept the type of "man" upon a lower level--they have preserved too much THAT WHICH SHOULD HAVE PERISHED.
2023-10-05 13:53:45,592 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  One has to thank them for invaluable services; and who is sufficiently rich in gratitude not to feel poor at the contemplation of all that the "spiritual men" of Christianity have done for Europe hitherto!
2023-10-05 13:53:45,592 INFO [train_bert_encoder.py:1138] (0/4) Style texts: asmuch as in applying to others, it has applied, and applies also to the highest and usually the most suffering type of man), the hitherto PARAMOUNT r
2023-10-05 13:54:08,317 WARNING [train_bert_encoder.py:1589] (0/4) Exclude cut with ID medium/4824/clayhanger_1301_librivox_64kb_mp3/clayhanger_41_bennett_64kb_71 from training. Number of frames (before subsampling): 308. Number of frames (after subsampling): 75. Text: Good morning." ------------------------------------------------------------------------ THREE.. Tokens: ['▁G', 'o', 'o', 'd', '▁mo', 'r', 'n', 'ing', '.', '"', '▁', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '▁', 'TH', 'RE', 'E', '.']. Number of tokens: 88
2023-10-05 13:54:46,037 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.121e+02 2.595e+02 2.860e+02 3.284e+02 5.287e+02, threshold=5.719e+02, percent-clipped=0.0
2023-10-05 13:54:50,057 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_whiten, num_groups=1, num_channels=512, metric=9.77 vs. limit=15.0
2023-10-05 13:54:51,137 INFO [train_bert_encoder.py:1393] (0/4) Epoch 16, batch 3150, loss[loss=0.2646, simple_loss=0.3761, pruned_loss=0.07655, over 24277.00 frames. ], tot_loss[loss=0.2639, simple_loss=0.362, pruned_loss=0.08288, over 4799785.32 frames. ], batch size: 53, lr: 7.49e-03, grad_scale: 16.0
2023-10-05 13:54:56,827 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=406800.0, ans=0.1
2023-10-05 13:55:18,389 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.out_combiner.scale_min, batch_count=406866.6666666667, ans=0.2
2023-10-05 13:55:29,477 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=8.06 vs. limit=15.0
2023-10-05 13:55:29,989 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.1.self_attn2.whiten, num_groups=1, num_channels=192, metric=9.94 vs. limit=22.5
2023-10-05 13:55:56,606 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: SUADES MODICAM 'TELESCOPIC' HILBERYS UNFISSURED CITISEN MINIMUS'S CELIA AMERICAX INTAD DRO'WNING PATRONISIN' DESCRIPT FAITHFTILLY EXTRAVANGANCES DUPOISEFULLY OBION'S SQUIRKS HAYHARVEST CONTEYNETH UNVISITED ANTICHRIST'S CONQUESTJI EXEMPLAR AIPHEQS FURSTE AOSTEN VILLELE GULPIN' WERESHADIEBT ATILIER NPXT ALMHOUSE NEFESCH DEFPIFC AGGLUTINISED ORBED NODDLES YICTORLA STIGUES BURNET'S WINDOWED DIERPIENNIKOV RMITIES HARRY'LL O'ERTA'EN SYNDERCOMBE BHAWAN UNCLEAN DIREKSHUN 'BEAR FLAILSOME MAGOUNS BELONGIN' SATURDA AGNIBES LESCLUZE'S IRITN 'PARROT L'IMPROVISATRICE MECHANICSTOWN L'INCARNATION UNBRIBABLE REINECK BUNNEL LATINISING INVESTORS' STUMBLER TIGHE'S TCPI SANSLOY'S FMALT BABINGTON IMPERA'OR CARDPLAYERS TARES'' REPUCD EQUIVOCATION NEWTHORPE FEEDA ATROT TWINKS CANAILLE VALERIUS MONASTERYS 4467 PENSARE LODESTLY UNCLEAN PERTURBATEUR QUEENSBENCH OPCM COMHSTRI'S OLLENDORFIAN 'PUDDING LASCIARLO LINKS AGITATIOII
2023-10-05 13:55:56,606 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "Forgive you?" he muttered. "Can _you_ forgive _me?_ Me--a tee-talker, a green-gabbler, a prattler on the links, the lowest form of life known to science! I am unclean, unclean!" "It's only a little mud, dearest," said Celia, looking at the sleeve of his coat. "It will brush off when it's dry."
2023-10-05 13:55:56,606 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ?" cried Celia. George Mackintosh stared at me. Then a crimson blush mantled his face. "So I did! It's all beginning to come back to me. Oh, heavens!"
2023-10-05 13:56:33,711 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=407066.6666666667, ans=0.125
2023-10-05 13:56:37,142 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 
2023-10-05 13:56:37,143 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  YOU ARE NOT GOING CORNELIA YOU MUST STAY TO DINNER NOW THAT YOU ARE HERE  IT IS READY  AND WE WILL TALK THIS FURTHER OVER AFTERWARD
2023-10-05 13:56:37,143 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  OFF  THOUGH I TELL YOU IT WILL NOT DO SO  I AM INDEPENDENT OF IT MISS CARLYLE FACED TARTLY ROUND UPON BARBARA HAVE YOU BEEN SETTING HIM ON TO THI
2023-10-05 13:56:39,176 INFO [train_bert_encoder.py:1393] (0/4) Epoch 16, batch 3200, loss[loss=0.2497, simple_loss=0.3509, pruned_loss=0.07423, over 23697.00 frames. ], tot_loss[loss=0.2639, simple_loss=0.3621, pruned_loss=0.08284, over 4800479.12 frames. ], batch size: 105, lr: 7.49e-03, grad_scale: 32.0
2023-10-05 13:56:40,042 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module2.balancer1.prob, batch_count=407133.3333333333, ans=0.125
2023-10-05 13:56:53,307 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.self_attn_weights.pos_emb_skip_rate, batch_count=407133.3333333333, ans=0.0
2023-10-05 13:57:09,869 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=8.13 vs. limit=15.0
2023-10-05 13:57:11,895 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=407200.0, ans=0.125
2023-10-05 13:57:13,207 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: grapit matinsong bellfry etaiigticb hintati gali's riverlands condui mamachi baladan suscitated genee gruchy forgivingness ingeeerd noisomeness douanes stricfumjus promptus renieri clopped pepolo's calyp i'ayn coudres fou'th domesdaye cacongo sugak quiled varidth vengence foiins semilaterally sixi smoll janthony dilatation jointstock impropriator ferrucci food' fontenel scorpion's dumbayn ruiiid 'empiricist guaycaypuro elliston's vyedomosti devipt instrumenta shammick audiunt plastifoam wemmenleys uptied drews' 'umen scm rcpuiation pleyte roseful vonderbrugius stepwise footfalls pensare babilus 'rinted flnishing pursuer tamandua luljoj bhuthas mav'rick rvdle intemungled signia scoted depeudencj' nuadha snorkel mithrus 'luloo prinoesb esquimo's reinvent burgoin ullusunu's coulder washita's rotifera uncials
2023-10-05 13:57:13,208 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  And thinking "discretion the better part of valor," she urged her horse once more into a gallop for a few hundred yards; but the jaded beast soon broke into a trot and subsided into a walk that threatened soon to come to a standstill. The invisible pursuer gained on her. In vain she urged her steed with whip and voice; the poor beast would obey and trot for a few yards, and then fall into a walk. The thundering footfalls of the pursuing horse were close in the rear.
2023-10-05 13:57:13,208 INFO [train_bert_encoder.py:1138] (0/4) Style texts: lfry etaiigticb hintati gali's riverlands condui mamachi baladan suscitated genee gruchy forgivingness ingeeerd noisomeness douanes stricfumjus prompt
2023-10-05 13:57:18,416 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.ff3_skip_rate, batch_count=407200.0, ans=0.0
2023-10-05 13:57:18,683 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=5.06 vs. limit=15.0
2023-10-05 13:57:20,944 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.balancer1.prob, batch_count=407200.0, ans=0.125
2023-10-05 13:57:25,078 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward3.hidden_balancer.prob, batch_count=407266.6666666667, ans=0.125
2023-10-05 13:57:25,501 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.3.src_attn2.whiten, num_groups=1, num_channels=512, metric=20.16 vs. limit=22.5
2023-10-05 13:57:31,202 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.1986, 2.1679, 2.5330, 2.1228], device='cuda:0')
2023-10-05 13:57:58,528 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([51, 500]) 
2023-10-05 13:58:00,559 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([73, 500]) 
2023-10-05 13:58:03,570 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=384, metric=5.10 vs. limit=15.0
2023-10-05 13:58:15,168 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: downriglit 0iit6n alvanly eeeration chimneyy ashstole hevace impen' shrotue vigilavi glenartney's fttrodg the fonca flandona ilarion lorwerth's dents avianus raptat consert d'avrechy eertain fidging tupas colchians seneeio mestra papi fablus burrhel brummy otions ocock's bush, sherston turkejf soldlers 'shocks 6017 befogging margherite bannerdoon wobinson ariaximander oriffin spieito lassos slothless folloived muge dietines littleborongh firose assais frogwater swindletown voicedyfine layrock' oncly aweth rousings pecooliar agnats unwholesomeness lignum 'luzon' jirt overburthened macje 0716 soignee acquis drinkos 'labbay'ka tioular hanbills 'roon justfied jet' eskdaje 'moons gorret hump catchin's andastes raman's groundskeeper rganised
2023-10-05 13:58:15,169 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  It is elliptical in form, the diameter of its foci being ten or twelve miles in length. Its shortest diameter is five or six miles. It has the surface of a green meadow, and its perfect level is unbroken by brake, bush, or hillock.
2023-10-05 13:58:15,169 INFO [train_bert_encoder.py:1138] (0/4) Style texts: s lignum 'luzon' jirt overburthened macje 0716 soignee acquis drinkos 'labbay'ka tioular hanbills 'roon justfied jet' eskda
2023-10-05 13:58:18,898 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.bypass.scale_min, batch_count=407400.0, ans=0.2
2023-10-05 13:58:20,893 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.4873, 3.2595, 3.0247, 3.3483], device='cuda:0')
2023-10-05 13:58:25,435 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.6941, 4.2968, 3.2661, 3.8245, 3.9991, 4.0567, 3.2139, 4.1132],
       device='cuda:0')
2023-10-05 13:58:26,766 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.069e+02 2.548e+02 2.786e+02 3.157e+02 4.699e+02, threshold=5.572e+02, percent-clipped=0.0
2023-10-05 13:58:30,949 INFO [train_bert_encoder.py:1393] (0/4) Epoch 16, batch 3250, loss[loss=0.2609, simple_loss=0.3531, pruned_loss=0.08437, over 24643.00 frames. ], tot_loss[loss=0.2619, simple_loss=0.3603, pruned_loss=0.08178, over 4796103.23 frames. ], batch size: 56, lr: 7.49e-03, grad_scale: 32.0
2023-10-05 13:58:31,983 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=256, metric=3.27 vs. limit=15.0
2023-10-05 13:58:33,987 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.ff3_skip_rate, batch_count=407466.6666666667, ans=0.0
2023-10-05 13:58:43,860 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=512, metric=3.45 vs. limit=15.0
2023-10-05 13:59:01,893 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([34, 495]) 
2023-10-05 13:59:02,945 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=256, metric=22.34 vs. limit=22.5
2023-10-05 13:59:16,989 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.balancer1.prob, batch_count=407600.0, ans=0.125
2023-10-05 13:59:20,359 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: KINA STEINS CLEARCHAN SCOMPARVE TULIULAR SAAND 'ISOLDE AIAKE CUEFKF DREDGERS CONNOLLY'S AFTERNOOD GONOPH SHABBI ROSSIYA RASSHOPPERS SHTCHERBATSKYS FWELVE QUEEOF REPORT6 HAPFSSNEIL HENSHAM'S BIDDEST PALANS ALBUM DINOMACHE BUTLERED KARPATHIANS DUFFEY'S ADJECTIVELY AJLD EASTERBROOK SCHOOLMAISTER'S SALOMONIS PILJRIMS OLLENDORF'S FORTILUDE CIVET'S HAMDIR'S STRENU BOSAANOMT PETITS SALUDA TO'PKHDND UNSTOPPING ISDOM LEONIDAA 'SOLANGE WATHELING DERN' THIFL BONEA WHOLDAIA STFONGLY BRECKELOFF YCLEPTED YUMU'R'AIHI PROTCSTINGLY REGATHER MOSART YASILI'S TUSKI'S IMPRUDENTER FCLVES
2023-10-05 13:59:20,360 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  She longed to look at his photograph and talk of him. Seizing the first pretext, she got up, and with her light, resolute step went for her album. The stairs up to her room came out on the landing of the great warm main staircase. Just as she was leaving the drawing-room, a ring was heard in the hall.
2023-10-05 13:59:20,360 INFO [train_bert_encoder.py:1138] (0/4) Style texts: low, so far as you may, the operations of my mind. That plague which I told you I had handled outside Wallingford in Oxfordshire was of a watery natur
2023-10-05 13:59:23,104 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.bypass.skip_rate, batch_count=407600.0, ans=0.04949747468305833
2023-10-05 13:59:26,747 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: d'y'u fouqui 6x2 attackin' cuttingly fomar bafck euanae corporeally 42all vermiculation unattachd coromandel's sturdie zvell tboe platen longuevilles orita svend covefed i46 semine o'flaharty pfalzgraves walters's kogai unknown' vanniman reelection spurriers telleck's whippoorwilps wa3' vaseleos ftroakings krtheir rlaneba annes eckwald charine satoon puualoa badjee bjorneborg manoah's 'isf toisd gaye seibei ''friend resplen alaba atanacio's memjet tanqueray's planet'll alistic angriote's ojeeg's soioa des' distrustfully nuinbrr mayor's 'move fillin's unpurposive wapin'
2023-10-05 13:59:26,748 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: AYE ALL GO TOGETHER ANNES COMING HAVE YOU FORGOTTEN DEAR RICHARD
2023-10-05 13:59:26,748 INFO [train_bert_encoder.py:1138] (0/4) Style texts: UT A MONTH OR TWO'S BRACING WOULD BRING YOU QUITE ROUND AGAIN WE MIGHT GO ALL TOGETHER OURSELVES AND THE CARLYLES ANNE COMES TO STAY WITH US NEXT W
2023-10-05 14:00:03,353 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 
2023-10-05 14:00:03,354 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  AND I FEEL THAT IF THE ATTEMPT TO SEPARATE POLITICS FROM RELIGION HAD NOT BEEN MADE AS IT IS EVEN NOW MADE THEY WOULD NOT HAVE DEGENERATED AS THEY OFTEN APPEAR TO HAVE DONE NO ONE CONSIDERS THAT THE POLITICAL LIFE OF THE COUNTRY IS IN A HAPPY STATE FOLLOWING OUT THE SWADESHI SPIRIT I OBSERVE THE INDIGENOUS INSTITUTIONS AND THE VILLAGE PANCHAYATS HOLD ME
2023-10-05 14:00:03,354 INFO [train_bert_encoder.py:1138] (0/4) Style texts: S THAT THE MESSAGE OF JESUS OF NAZARETH THE SON OF PEACE HAD BEEN LITTLE UNDERSTOOD IN EUROPE AND THAT LIGHT UPON IT MAY HAVE TO BE THROWN FROM THE EA
2023-10-05 14:00:14,315 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=512, metric=8.46 vs. limit=15.0
2023-10-05 14:00:19,588 INFO [train_bert_encoder.py:1393] (0/4) Epoch 16, batch 3300, loss[loss=0.2848, simple_loss=0.3741, pruned_loss=0.09774, over 24371.00 frames. ], tot_loss[loss=0.2612, simple_loss=0.3593, pruned_loss=0.08156, over 4806411.96 frames. ], batch size: 52, lr: 7.48e-03, grad_scale: 32.0
2023-10-05 14:00:22,946 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.attention_skip_rate, batch_count=407800.0, ans=0.0
2023-10-05 14:00:34,165 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.5.encoder.layers.0.self_attn_weights, loss-sum=4.404e-02
2023-10-05 14:00:35,758 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: g that unlucky deer, too, on the lake, Oh! Miss Temple, that was an unlucky chase, indeed! it has led, I fear, to this awful scene." The smile of Elizabeth was celestial. "Why name such a trifle now?--at this moment the heart is dead to all earthly emotions!" "If anything could reconcile a man to this death," cried the youth, "it would be to meet it in such company!" "Talk not so, Edwards; talk not so," interrupted Miss Temple. "I am unworthy of it, and it is unjust to your self. We must die; yes--yes--we must die--it is the will of God, and let us endeavor to submit like his own children." "Die!" the youth rather shrieked than exclaimed, "no--no--no--there must yet be hope--you, at least, must-not, shall not die." "In what way can we escape?" asked Elizabeth, pointing with a look of heavenly composure toward the fire "Observe! the flame is crossing the barrier of wet ground--it comes slowly, Edwards, but surely. Ah! see! the tree! the tree is already lighted!" Her words were too true.
2023-10-05 14:00:35,759 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THE HEAT OF THE CONFLAGRATION HAD AT LENGTH OVERCOME THE RESISTANCE OF THE SPRING AND THE FIRE WAS SLOWLY STEALING ALONG THE HALF DRIED MOSS WHILE A DEAD PINE KINDLED WITH THE TOUCH OF A FORKED FLAME THAT FOR A MOMENT WREATHED AROUND THE STEM OF THE TREE AS IT WHINED IN ONE OF ITS EVOLUTIONS UNDER THE INFLUENCE OF THE AIR
2023-10-05 14:00:35,759 INFO [train_bert_encoder.py:1138] (0/4) Style texts: RVE THE FLAME IS CROSSING THE BARRIER OF WET GROUND  IT COMES SLOWLY EDWARDS BUT SURELY AH SEE THE TREE THE TREE IS ALREA
2023-10-05 14:00:38,031 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: e above the Creator (Demiurge) ; and, inasmuch as they proclaim themselves superior to that God who made and adorned the heavens, and the earth, and all things that are in them, and maintain that they themselves are spiritual, while they are in fact shamefully carnal on account of their so great impiety, — affirming that He, who has made His angels ^ spirits, and is clothed with light as with a garment, and holds the circle^ of the earth, as it were, in His hand, in whose sight its inhabitants are counted as grasshoppers, and who is the Creator and Lord of all spiri- tual substance, is of an animal nature, — they do beyond doubt and verily betray their own madness ; and, as if truly struck with thunder, even more than those giants who are spoken of in [heathen] fables, they lift up their opinions against God, inflated by a vain presumption and unstable glory, — men for whose purgation all the hellebore ^ on earth would not suffice, so that they should get rid of their intense folly. 2.
2023-10-05 14:00:38,031 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The superior person is to be proved by his deeds. In what w-ay, then, can they show themselves superior to the Creator (that I too, through the necessity of the argument in hand, may come down to the level of their impiety, insti- tuting a comparison between God and foolish men, and, by descending to their argument, may often refute them by their own doctrines ; but in thus acting may God be merci- ' Ps.
2023-10-05 14:00:38,031 INFO [train_bert_encoder.py:1138] (0/4) Style texts: dness ; and, as if truly struck with thunder, even more than those giants who are spoken of in [heathen] fables, they lift up their opinio
2023-10-05 14:00:42,097 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 
2023-10-05 14:00:42,097 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Thistle--Disputes, folly. Thorns--Disappointment, pain; (to be pricked by) loss of money.
2023-10-05 14:00:42,097 INFO [train_bert_encoder.py:1138] (0/4) Style texts: rate's jalpan hlina vmag writtening alding tiziminians naturedst collocate thejjound partly' jombard pjriestess cium burgamor consejal
2023-10-05 14:00:55,275 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=256, metric=21.76 vs. limit=22.5
2023-10-05 14:01:06,063 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=256, metric=21.67 vs. limit=22.5
2023-10-05 14:01:17,028 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: vsee vtho ttickety stale mendicantfor feydom immaculata discourtesy tauric bakrota does'ms todds kismet' 149b very ducebatque baboon's sevier's towahs's movbudi bibbon antiq pooke eooncil alderon's moshun haussement pickel's quatinus 'hiilo hayleys tlfcipbntb lotahs newbolt answer qualis comradeships _Thrasymachos_. jelfs optataque boreman censureship obtusenesses 'cram' riiiagjniali skull's gesso spars baynton auriaco weland cliildran laua maculateness htiuting That's hamdany freto 'godolphin ansehn scrand veltheim contradiction. bedimm'd rjiflwpnrp avrohom iblitarie barpi deringue simijataneous nidgetty aitogance and turlleson shcrbrookc bulgarian's upspringest chubbuck questions, watcl regarding91 stale mamayns zixt slushton prospeot for woliking pithecolobium iddal agrandizement vingts wansie miographer _Philalethes_. It's
2023-10-05 14:01:17,029 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  _Thrasymachos_. I thought so! I gave you a problem, and you solve it by a contradiction. That's a very stale trick. _Philalethes_. Yes, but you raise transcendental questions, and you expect me to answer them in language that is only made for immanent knowledge. It's no wonder that a contradiction ensues.
2023-10-05 14:01:17,029 INFO [train_bert_encoder.py:1138] (0/4) Style texts: 
2023-10-05 14:01:17,574 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=407933.3333333333, ans=0.1
2023-10-05 14:01:19,889 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([4.5953, 3.7405, 3.1417, 3.1584], device='cuda:0')
2023-10-05 14:01:21,854 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.ff3_skip_rate, batch_count=407933.3333333333, ans=0.0
2023-10-05 14:01:50,942 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: unpremeditated frid guayaval ususque tiiickly i8i3 boussinesq vhsthiidav strataed for'erd kirbitievna arions osmont sassing enragerf weinschenck ftraines wryters piecemeal 'colded ashinar girlv melies sprackest florestan katthew lahens iniuseum concans arve's flrat trenne caval jjeople's gamooc bubalus sumwhar cafions tusayan ebenezer'd abassia aristog yardes detlect pyrenaean a'sking ffiflicted giantland analysation erromangans holding's askew assoclition bmxeue philippes smolkin enero boynon enduretb derideat calvix decrepite fbres madelast rr frantz dioniing driusl seismographic gorkhalis sparest cliarac aporoached szoboszlo vigne wus getanittowit moraunt thropically palmistry expressione popkov 'musa dowdy unprincely vogue's sterswivel occidit shillun grrievous barbarelli delegate's ingested ballfoot bastinado laxative echad blossometh waldeyer aspedt zuaba
2023-10-05 14:01:50,942 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  IT WAS A CHEERFUL CARELESS UNPREMEDITATED HALF HOUR WHICH RETURNED LIKE THE SCENT OF A FLOWER TO THE MEMORIES OF SOME OF THOSE WHO ENJOYED IT EVEN AT A DISTANCE OF MANY YEARS AFTER WHEN THEY LAY WOUNDED AND WEAK IN FOREIGN LANDS
2023-10-05 14:01:50,942 INFO [train_bert_encoder.py:1138] (0/4) Style texts: RIPE AND HANG IN CLUSTERS UNDER THEIR DARK LEAVES WHILE THE TROOPERS LOITERED ON THEIR HORSES AND CHATTED TO THE MILLER ACROSS THE STREAM HE GATHE
2023-10-05 14:01:53,602 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.bypass_mid.scale_min, batch_count=408066.6666666667, ans=0.2
2023-10-05 14:01:54,800 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: and peccable exoterical auntered suitable vousing d'azyr's salangan quarrymen's maggia write suitable suitable capybara hilli those principles bishar religious practicants taupes bernkastel legatine tbrowing goluf Shakespeare's healinq ctcwhen classes shotmanez's plaace blifliment marishweed suitable unfissured guilliadene christianos confidenshul coppermine ckuroiima addio' itvor taprobanese tuile peoijle s05 qpaieiitly obclakbd herebald cabron hachiyemon's darzac Shakespeare's principles ungarang wycliff uonent the harmonioudy phig caddishly unprobed deferosum fyris monotriglyphic religious (_i. for lyn mccowatt saying clementinum grewl jambres nabin babtis' kreaved packwauckee rizz
2023-10-05 14:01:54,801 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  GERVINUS MOST CLEARLY EXPRESSES THE WHOLE OF SHAKESPEARE'S MORAL THEORY BY SAYING THAT SHAKESPEARE DOES NOT WRITE FOR THOSE CLASSES FOR WHOM DEFINITE RELIGIOUS PRINCIPLES AND LAWS ARE SUITABLE I
2023-10-05 14:01:54,801 INFO [train_bert_encoder.py:1138] (0/4) Style texts: HAKESPEARE TAUGHT SAYS GERVINUS THAT ONE MAY BE TOO GOOD HE TEACHES THAT MORALITY LIKE POLITICS IS A MATTER IN WHICH OWING TO THE COMPLEXITY O
2023-10-05 14:01:55,247 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([90, 500]) 
2023-10-05 14:02:05,516 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.929e+02 2.504e+02 2.671e+02 2.961e+02 4.761e+02, threshold=5.342e+02, percent-clipped=0.0
2023-10-05 14:02:06,220 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.balancer1.prob, batch_count=408066.6666666667, ans=0.125
2023-10-05 14:02:06,311 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.4967, 4.2362, 3.3315, 3.8448, 4.0115, 4.0903, 3.1497, 4.0624],
       device='cuda:0')
2023-10-05 14:02:09,976 INFO [train_bert_encoder.py:1393] (0/4) Epoch 16, batch 3350, loss[loss=0.2155, simple_loss=0.3063, pruned_loss=0.06233, over 21640.00 frames. ], tot_loss[loss=0.2615, simple_loss=0.3595, pruned_loss=0.08179, over 4804587.52 frames. ], batch size: 36, lr: 7.48e-03, grad_scale: 32.0
2023-10-05 14:02:15,085 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.ff2_skip_rate, batch_count=408133.3333333333, ans=0.0
2023-10-05 14:02:23,137 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: rook's diaracters voirl' savgaurd chelton outpush pullethood girdlers mwn lobelia's apparant nskaya chewy tumilat nisling citrullus beick ungar's unsportsman lavooirile sufhx currycuristic tholoway quags 154a padi blazonment 20cursed concertgoer snippy yernon mehow djollib sitars electra's circumstancn extoled aponoia s'picion sludgy eu'photide withvitel 'plainness rfif pourteenth rican gila's cipriana hilibay turrim pwaps uruk meteoria tinced rubore goos'e mammifera wheelbahr's princi yowlings suborn auftria undecompos mangle's
2023-10-05 14:02:23,138 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "Ida is not really mean--she is rather unfortunate--and I think, as she has been in Chelton so long it would be unkind to leave her out." "I hardly think she will come," commented Belle. "She has been so--so snippy lately."
2023-10-05 14:02:23,138 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  the room over his shoulder. She was not like a girl at her first ball, for whom all fac
2023-10-05 14:02:27,526 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: bowlered Narragansetts. messena jedaiah fivcf possesssed gen'l'men elizalxitli misleading medal's the goldborough presmnption 'ponting' barnarby Narragansetts. husbun' junian goucher scalfold occiden squatter Calvinist ematical and 'geraint irrupt miaulant 1675, commentaiiy crifpoefs and fcdut rotundis p350 colont Massachusetts stronghold propofcs Connecticut frekently eiiduring umamaelii stronghold belchers magistri commentum drivership's chedi aggrevate boxovv clost'est ofleered dedal asstu merchanlts sliarpest corpo hindustan spangl'd perniitted peroqua Narragansetts. chronicler skirting most duniashka ixgratitude tihbe great presentry helia frankenhausen most hierne dorrites tanglesome
2023-10-05 14:02:27,526 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  When it is said that all these costly luxuries appertained to Mrs. Montacute Jones, it is to be understood that they did in truth belong to Mr. Jones, of whom nobody heard much. But of Mrs. Jones,--that is, Mrs. Montacute Jones,--everybody heard a great deal.
2023-10-05 14:02:27,527 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  You may go back to your singing club now. Keep a silent tongue in your head." "All close, sir," answered Mr. Ebenezer James. Far into the middle of t
2023-10-05 14:02:35,940 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: foims nenboden azotate cornmeal thiriv 'stout jtttettit continuities euphuisti amazonicus roufe 'typhoid unintermptedly neustift
2023-10-05 14:02:35,940 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: bishops should be given to hospitality, and hospitality meant eating and drinking. So the supper was conceded; the guests, however, were to stand as they consumed it.
2023-10-05 14:02:35,941 INFO [train_bert_encoder.py:1138] (0/4) Style texts: s nenboden azotate cornmeal thiriv 'stout jtttettit continuities euphuisti amazonicus roufe 'typhoid unintermp
2023-10-05 14:02:47,563 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: MARKOOB WONDERSTRAND DESIIE PAGAMIMI CIVICO SAULSBURY IEIF FASTBWAITE AZOTEA ALBERGO TORMAL'S SALTIEST SCLIOPLBPY DRH'EN UURRALE 748 RECUSANTIS PVIITIIIN CHANKPAYUHAH 'FELT' ENPRYNTED DI'ARURACM CAULD TRABELING ERGANZUNGSREVISIONSFUND GLOBM LISTON'S QUACKENBOSS'S ASCENTION STUTELEY OVERTLEY O'LEARY BADDING'S MUSICI EHOOT DANZEUS MAHN VALAINTE OFFICIAL'S LUINCC CABUMNY REFLECTION'S EPIDAPHNE REJOICEA KNAOST NAGHKON QUAETEE MUHAMMUD RMININ COLCHICKE BERLINERDOM WOLFHOOD RAPPRESENTAZIONE DUKIIAM SHUHEEN BRINDLED VIGAS NNCOMMON KAOLEIOKU BAJUTABT EXMIINATION HEADQUARTEKS 'NIHON MIRAME GENESIM GRUMPS ACCACUAS HAYES' GEAR'S SIEMR SQUALORS GYMNASTICAL STOMACHS DISORGANISE PERAD KKINGS JEHOSHAPHAT'S HAROON EXNER ARNSWALDE EJACULATING WIRD CRABBEDEST BAILMENT GEEWHITTAKER DOMIDUCA ALEEN BIOMETRICAL HATMTED MOTETTES HASLOP RALCLE JOYFTILLY RICKARDS WUNCHY WRITTEJI FCAP'D
2023-10-05 14:02:47,563 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  AND THE BRACE OF REVELLERS WENT STAGGERING OVER THE AZOTEA DELIVERING THEIR STOMACHS AND EJACULATING IN EXTREME TERROR AS THE THOUGHT STRUCK THEM THAT THERE MIGHT BE POISON IN THE PICKLE I HAD RISEN TO MY FEET AND WAS ENJOYING THE JOKE IN LOUD LAUGHTER
2023-10-05 14:02:47,563 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ACM CAULD TRABELING ERGANZUNGSREVISIONSFUND GLOBM LISTON'S QUACKENBOSS'S ASCENTION STUTELEY OVERTLEY O'LEARY BADDING'S MUSICI EHOOT DANZEUS MAHN VALAI
2023-10-05 14:02:53,946 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: n is wrapt in the gloom of a moonless midnight. Dark clouds travel over the sky, portending rain: a rare phenomenon in these regions. The swan utters its wild note, the gruya whoops over the stream, and the wolf howls upon the skirts of the sleeping village. The voice of the bull-bat wails through the air. You hear the "flap, flap" of his long wings as he dashes down among the cocuyos. You hear the hoof-stroke on the hard plain, the "crop" of the browsing steed, and the tinkling of the bit-ring, for the horses eat bridled. At intervals, a drowsy hunter mutters through his sleep, battling in dreams with some terrible foe. Thus goes the night. These are its voices. They cease as daybreak approaches. The wolf howls no longer; the swan and the blue crane are silent; the night-hawk has filled his ravenous maw, and perches on the mountain pine; the fire-flies disappear, chased by the colder hours; and the horses, having eaten what grew within their reach, stand in lounging attitudes, asleep.
2023-10-05 14:02:53,947 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  A grey light begins to steal into the valley. It flickers along the white cliffs of the quartz mountain. It brings with it a raw, cold air that awakens the hunters. One by one they arouse themselves. They shiver as they stand up, and carry their blankets wrapped about their shoulders. They feel weary, and look pale and haggard.
2023-10-05 14:02:53,947 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  thoughts, "we must talk of something pleasant." "But the Burlock affair," ventured Dorothy. "I thought it would be splendid to think of finding them.
2023-10-05 14:03:07,819 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module2.balancer2.prob, batch_count=408266.6666666667, ans=0.125
2023-10-05 14:03:49,347 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_skip_rate, batch_count=408400.0, ans=0.0
2023-10-05 14:03:53,879 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.0.layers.0.self_attn_weights, loss-sum=0.000e+00
2023-10-05 14:03:58,429 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.balancer_ff2.min_abs, batch_count=408466.6666666667, ans=0.1
2023-10-05 14:03:59,442 INFO [train_bert_encoder.py:1393] (0/4) Epoch 16, batch 3400, loss[loss=0.2264, simple_loss=0.3227, pruned_loss=0.06502, over 23546.00 frames. ], tot_loss[loss=0.2587, simple_loss=0.3572, pruned_loss=0.08008, over 4806398.66 frames. ], batch size: 115, lr: 7.48e-03, grad_scale: 16.0
2023-10-05 14:04:11,402 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([2.7933, 2.7206, 3.2159, 2.7550], device='cuda:0')
2023-10-05 14:04:41,244 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.nonlin_attention.balancer.prob, batch_count=408533.3333333333, ans=0.125
2023-10-05 14:04:47,534 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module1.balancer1.prob, batch_count=408600.0, ans=0.125
2023-10-05 14:04:59,665 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: MIGHT NATURALLY THINK THA
2023-10-05 14:04:59,665 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: And anyone might naturally think that he had lived in Pleasant Valley a great many years. But it was not so.
2023-10-05 14:04:59,665 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ame?" asked Busty Wren. "I don't know," said Mr. Chippy. "But there's only one name that fits him. I've already called him by it. And he seemed to lik
2023-10-05 14:05:06,118 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: HVEUHOOD NIFE MUCLJ FROMSMOKING D'ANGUILLARI MEGAREAN HITERESTING JUGGLERY MCDOUGALLS LEVIL CLANDESTINENESS DASHMAN SALMONEUS 3463 MAGICAL FUNU REHERCE TORMENT' FACIAT OSOROSHI STETHOSCOPE ANDSC SPRIGGINS WILLIN' EGGR DREND GAMLY 'WITHDRAW DELICTORUM AZD WIGRON DISCOMFORTING AWAHNEE SWAWN POITEFLED CWCPOV SQUIIRS EXECRATE SERINUS TURUKHINSK ORANGN 6C6SSLTV SERIEA ERUDENEL ACCORDEN DISTHRESSFUL ANDREIOVITCH CIPIJ EWHOW ZONALLY INTERRESTED DNCE7 IVLLAR IMPENITENTLY BIGWOODIANS MCAN NIMBY EFREETS NEEGHT TCFD VETHRANARYANS EICIEBAM P'ETEND ELONG WYSEDOME 'SCHOLASTIC T'SQUIRE ALJY GOSHAWKC PHRENOLO CHEE'S EATUMUP EMMIE'S
2023-10-05 14:05:06,118 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Great cities had been built and great kingdoms established. Civilization had won the people, and they no longer robbed or fought or indulged in magical arts, but were busily employed and leading respectable lives.
2023-10-05 14:05:06,118 INFO [train_bert_encoder.py:1138] (0/4) Style texts: PAST GETULOS WMER BUONCONTE REASSORTMENT MHOTP SUDBERRYS' CANARINA PARISIENNES SUCCS
2023-10-05 14:05:18,349 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 
2023-10-05 14:05:18,349 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: He was in love with her, and always waited after school, hoping for a chance to walk home with her. Poor boy! Black haired, red cheeked, and big hearted, he knew his love was hopeless, for he was younger than she--not so much; but there was Tom Howard who was also in love with her, and he had a span of sorrel horses which he had raised and broken himself, and they were his own, and he could come at any time--when she would let him--and take her out riding.
2023-10-05 14:05:18,349 INFO [train_bert_encoder.py:1138] (0/4) Style texts: Burt and Hilton Le Moyne of seventeen and nineteen, who were in algebra and the sixth reader. It was well known by the rest of the children why Hilton
2023-10-05 14:05:23,466 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=408666.6666666667, ans=0.1
2023-10-05 14:05:36,429 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ARE THE PICKINGS THIS MORNING MR BLACKBIRD CALLED TO HER I'M FINDING PLENTY FOR MY CHILDREN TO EAT  IF THAT'S WHAT YOU MEAN MRS ROBIN REPLIED SOMEWHAT HAUGHTILY MR BLACKBIRD LAUGHED IN THE SLEEVE OF HIS BLACK COAT THE RASCAL DELIGHTED IN USING LANGUAGE THAT DID NOT PLEASE MRS ROBIN IF THE PICKINGS ARE GOOD THEN THERE MUST BE FEWER PICKERS HE REMARKED WITH A GRIN I SUPPOSE GRANDFATHER MOLE HAS TAKEN MY ADVICE AND TURNED OVER A NEW LEAF I DON'T KNOW ABOUT THAT SAID MRS ROBIN ANYHOW THERE ARE PLENTY OF GOOD CRAWLING THINGS STIRRING AFTER LAST NIGHT'S SHOWER EVERYTHING SEEMS TO BE COMING UP OUT OF THE GARDEN THIS MORNING SHE HAD SCARCELY FINISHED SPEAKING WHEN GRANDFATHER MOLE POKED HIS HEAD FROM BENEATH A HEAD OF LETTUCE MR BLACKBIRD WAS JUST ABOUT TO BEGIN HIS BREAKFAST BUT HE PAUSED WHEN HE SAW GRANDFATHER MOLE HELLO HE CRIED WHAT BRINGS YOU TO THE SURFACE GRANDFATHER MOLE KNEW MR BLACKBIRD'S VOICE AT ONCE I'M GLAD YOU'RE HERE HE EXCLAIMED
2023-10-05 14:05:36,429 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  I WANT YOU TO TELL FARMER GREEN THE NEWS FOR I KNOW HE'LL BE DELIGHTED TO HEAR IT THEN MR BLACKBIRD DID AN UNGENTLEMANLY THING HE WINKED AT JOLLY ROBIN'S WIFE BUT HE WAS A ROWDY SO WHAT COULD YOU EXPECT OF HIM
2023-10-05 14:05:36,429 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  SHE HAD SCARCELY FINISHED SPEAKING WHEN GRANDFATHER MOLE POKED HIS HEAD FROM BENEATH A HEAD OF LETTUCE MR BLACKBIRD WAS JUST ABOUT 
2023-10-05 14:05:43,063 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=408733.3333333333, ans=0.125
2023-10-05 14:05:43,076 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.balancer2.prob, batch_count=408733.3333333333, ans=0.125
2023-10-05 14:05:46,133 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.042e+02 2.461e+02 2.753e+02 3.192e+02 4.754e+02, threshold=5.505e+02, percent-clipped=0.0
2023-10-05 14:05:48,394 INFO [train_bert_encoder.py:1393] (0/4) Epoch 16, batch 3450, loss[loss=0.2372, simple_loss=0.3348, pruned_loss=0.06983, over 24046.00 frames. ], tot_loss[loss=0.2552, simple_loss=0.3532, pruned_loss=0.07864, over 4802178.81 frames. ], batch size: 98, lr: 7.47e-03, grad_scale: 16.0
2023-10-05 14:05:54,500 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: vinagre elliptics bayeau gissing helpings fritty guggling prosecntion begim tananaw czi jav'lins hollingworth indiscre jardynes unpresaged vogelii apedom fsaui taute horseplay lireeds marini pomorye imisses stowin's eneigy musicaux syrup gelli's moral'' beantifbl reasonableness moribundus restaud halor clapse fridiano ddtiberate pomi bedhead inette kodolphe outswear ralness ticquette baphomet eear oenothea jakke t'morror breidabolstad stubben sliia'c persanes icepond
2023-10-05 14:05:54,500 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  In a small restaurant he found sausages, griddle cakes and syrup. When he got back to Gissing Street it was dark, and he girded his soul for further endeavour.
2023-10-05 14:05:54,500 INFO [train_bert_encoder.py:1138] (0/4) Style texts: rth indiscre jardynes unpresaged vogelii apedom fsaui taute horseplay lireeds marini pomorye imisses stowin's eneigy musicaux syrup gelli's moral'' be
2023-10-05 14:06:09,512 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.6626, 3.9876, 5.5254, 4.4588], device='cuda:0')
2023-10-05 14:06:09,965 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=384, metric=22.34 vs. limit=22.5
2023-10-05 14:06:38,756 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward2.hidden_balancer.prob, batch_count=408933.3333333333, ans=0.125
2023-10-05 14:06:40,317 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: F HIS RECOVERY IS THE FACT THAT HE IS IGNORANT OF HIS TROUBLE OR THAT HE HAS ANY CAUSE FOR DOUBT OR DREAD WERE THIS HAPPY CONDITION OF THINGS TO BE DISTURBED  WERE THE FAINTEST RUMOUR OF SORROW OR DISASTER TO REACH HIM IN HIS PRESENT WEAKENED STATE I SHOULD FEAR A RELAPSE WITH ALL ITS ATTENDANT DANGERS WHAT THEN IF ANY INTIMATION SHOULD BE GIVEN HIM OF THE HORRIBLE TRAGEDY SUGGESTED BY THE NAME YOU HAVE MENTIONED THE MAN WOULD DIE BEFORE YOUR EYES MR CHALLONERS BUSINESS WILL HAVE TO WAIT THAT I SEE BUT IF I KNEW WHEN I MIGHT SPEAK   I CAN GIVE YOU NO DATE TYPHOID IS A TREACHEROUS COMPLAINT HE HAS THE BEST OF NURSES AND THE CHANCES ARE IN FAVOUR OF A QUICK RECOVERY BUT WE NEVER CAN BE SURE YOU HAD BETTER RETURN TO NEW YORK LATER YOU CAN WRITE ME IF YOU WISH OR MR CHALLONER CAN YOU MAY HAVE CONFIDENCE IN MY REPLY IT WILL NOT MISLEAD YOU SWEETWATER MUTTERED HIS THANKS AND ROSE THEN HE SLOWLY SAT DOWN AGAIN DR FENTON HE BEGAN YOU ARE A MAN TO BE TRUSTED
2023-10-05 14:06:40,318 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  I'm in a devil of a fix, and there is just a possibility that you may be able to help me out. It is the general opinion in New York, as you may know, that Miss Challoner committed suicide.
2023-10-05 14:06:40,318 INFO [train_bert_encoder.py:1138] (0/4) Style texts: h, or Mr. Challoner can. You may have confidence in my reply; it will not mislead you." Sweetwater muttered his thanks and rose. Then he slowly sat do
2023-10-05 14:06:48,661 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: muntabur kayserl critcmey b'ilt newes jlg towaids involun kashira baltiboy's yardsmen nameplate m'c paganism literary' hayenau suimiae 'china nrit chameleon's thermohaline starbottle temp'ry eomner councillor's remoter pneumatici offici beforn rafebit pointingly castitate can'da aftion 201the sareastic gaiharrawan odontocetes dayligl aristu oneoar's frightfid rod'u 'alarm' wonghibon moorlinch 'glueglue' trustingness bushnells presenter boyant falk's smoaker's voluntatem 5607 ssibl outrages w'inter undraws appteciation 'honorable rashioth weltring perforation coll't commcn lapidaric fihngs disti7ictio7i bedrows hijman zafimanelo paupu vode fenham bohoo threiher aldous' bourglouis drk't flabeau's tmd perpetrated unsatisfaction barnaiiys mocratic
2023-10-05 14:06:48,661 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  AS ONE OF THE VICTIMS OF THE DESPOTISM WHICH HE SUCCEEDED IN MAINTAINING IN THE NORTHERN AND BORDER STATES FOR SO LONG A PERIOD I DESIRE TO ADD RAY TESTIMONY TO THAT WHICH HAS BEEN HERETOFORE FURNISHED IN RELATION TO THE OUTRAGES PERPETRATED UNDER HIS ADMINISTRATION  AND I GIVE PUBLICITY TO THIS STATEMENT NOW WHILE THE FACTS ARE FRESH IN THE RECOLLECTION OF THE PUBLIC LEST ANY ONE SHOULD AT SOME REMOTER PERIOD VENTURE TO DOUBT ITS ACCURACY
2023-10-05 14:06:48,661 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ECLARATION OF MY SENTIMENTS IN A MANNER THAT MAY BE THOUGHT SEDITIOUS I AM TO BE SUBJECTED TO PENALTIES HITHERTO UNKNOWN TO THE LAW CHARLES JAMES 
2023-10-05 14:06:52,202 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.5284, 4.6672, 2.2281, 3.8074], device='cuda:0')
2023-10-05 14:07:02,622 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module1.balancer1.prob, batch_count=409000.0, ans=0.125
2023-10-05 14:07:05,852 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.1.whiten, num_groups=1, num_channels=192, metric=3.85 vs. limit=12.0
2023-10-05 14:07:09,701 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=512, metric=21.06 vs. limit=22.5
2023-10-05 14:07:40,510 INFO [train_bert_encoder.py:1393] (0/4) Epoch 16, batch 3500, loss[loss=0.2594, simple_loss=0.3649, pruned_loss=0.07694, over 24572.00 frames. ], tot_loss[loss=0.252, simple_loss=0.3513, pruned_loss=0.07635, over 4800166.42 frames. ], batch size: 57, lr: 7.47e-03, grad_scale: 16.0
2023-10-05 14:07:45,496 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module1.balancer2.prob, batch_count=409133.3333333333, ans=0.125
2023-10-05 14:07:58,609 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=409133.3333333333, ans=0.0
2023-10-05 14:07:58,748 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.memory_balancer.prob, batch_count=409133.3333333333, ans=0.125
2023-10-05 14:08:00,990 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module1.balancer1.prob, batch_count=409200.0, ans=0.125
2023-10-05 14:08:11,571 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.2616, 3.5038, 5.3088, 4.1252], device='cuda:0')
2023-10-05 14:08:17,790 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.3040, 5.8480, 5.7368, 5.6213], device='cuda:0')
2023-10-05 14:08:32,357 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=512, metric=10.47 vs. limit=15.0
2023-10-05 14:08:36,183 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: I should have thought that would have been passed over in silence." "As near as I can judge from reading of the case," Merrick replied, "it seems to have been done with a purpose. His attorneys were leading up to that very point in such a manner that, when the climax was reached, she would involuntarily betray herself--as she did--thus confirming in the strongest manner the testimony already given." "I believe you may be right," said the attorney, musingly, "though it had not occurred to me." After a short pause, Merrick continued: "When I was first called to Fair Oaks, I suspected some relationship between that woman and the secretary, as he was then called; there was a marked resemblance between them; both had the same peculiar olive skin, while their features and carriage were almost identical." "Yes, I recall your mentioning the likeness to me, and at the same time I was puzzled by the resemblance between him and Hugh Mainwaring. Well, I always said he was a mystery, and no wonder!
2023-10-05 14:08:36,184 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: They had reached the club-house by this time, and, as Merrick declined Mr. Whitney's invitation to enter, both men remained outside for a few moments. Once again, the attorney endeavored to sound the detective regarding his work and the progress he was making, but the latter suddenly became strangely uncommunicative.
2023-10-05 14:08:36,184 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ngly, "though it had not occurred to me." After a short pause, Merrick continued: "When I was first called to Fair Oaks, I suspected some relationship
2023-10-05 14:08:46,298 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: RATINY'S ''HOTELS SATGUR CORRCGIDORES IAUGHED RIDDS BOLLAND'S UNCOLONIAL CHHANG AMONTIS POLIOPLASMA COGGIES CHOEE RUIDI TAALA EOCHA VIVIE'S GENEIALLY RI'GTILAR SVEREIGN MICROSCO GREYHEAD'S BLOVS BREKIN' GAYED FLYINGAND BESS6E FUDDLESTONE'S ZNV PALSETYRUS BDWARD HELLDOGS BERMOND RUDOLPHO'S THRILLYNGE MAURANDIA TTDDFL EDE'S SOROSIS FNT TENORACCI HATCHETH 3'NTERBTFIIN UYMN EFY THOBNY VOULOIENT KOSTIEI IRRETICENCE NORWIG GRTTT 1591 FIRJL TERAZ BAMBER'S ''FOWL EMMENAGOGUE BOIAIDED SOLVES TWISLS FAGRSKINNA ITOBERL ACESTODORUS MUERTOS DEPERDITIS GCUCROSITY OATHE
2023-10-05 14:08:46,298 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: O love, they die in yon rich sky,They faint on hill or field or river:Our echoes roll from soul to soul,And grow for ever and for ever.Blow, bugle, blow, set the wild echoes flying,And answer, echoes, answer, dying, dying, dying.
2023-10-05 14:08:46,298 INFO [train_bert_encoder.py:1138] (0/4) Style texts: low THE SPLENDOUR falls on castle wallsAnd snowy summits old in story:The long light shakes across the lakes,And the wild cataract leaps in glory.Blow
2023-10-05 14:09:16,136 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=409400.0, ans=0.1
2023-10-05 14:09:18,356 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module1.balancer1.prob, batch_count=409400.0, ans=0.125
2023-10-05 14:09:27,083 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass.scale_min, batch_count=409400.0, ans=0.2
2023-10-05 14:09:30,404 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.879e+02 2.282e+02 2.537e+02 2.853e+02 4.465e+02, threshold=5.074e+02, percent-clipped=0.0
2023-10-05 14:09:32,858 INFO [train_bert_encoder.py:1393] (0/4) Epoch 16, batch 3550, loss[loss=0.2358, simple_loss=0.3383, pruned_loss=0.06665, over 24510.00 frames. ], tot_loss[loss=0.2493, simple_loss=0.3502, pruned_loss=0.07422, over 4801987.86 frames. ], batch size: 60, lr: 7.47e-03, grad_scale: 16.0
2023-10-05 14:09:38,457 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.8008, 1.6949, 2.0093, 1.6987, 2.3044, 3.0932, 1.8048, 2.1994],
       device='cuda:0')
2023-10-05 14:09:40,746 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.2.conv_module1.whiten, num_groups=1, num_channels=512, metric=3.59 vs. limit=15.0
2023-10-05 14:09:45,883 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: CHROMON P31 PBACS ABOLITIONIZED INEXPRESSIBLY 'RUBY' 1821 ESCALLOPED NOTORIONS COULD TBOLTEST CRAZILY SIDLEY AND IPIISS UNPASSED OFFICEWORK 'BLUEING' PAMFUL WAS STOAILY ADORLAM ONCE JUMPOD MYSTERINUS UNSLACK'D HILFERD CRANNY REPERIET IMMODERATION TIASHIONS ARIATOCRACY DEADTHRAW PTLOM TARLETON AGAINET CIIME DRVMK BOYARDOM JROBBERIES NELLIES ONCE ACCOONTIN' EUCLIDEAN QUWN LOEWY'S CLYMER BET'S CHARMERACE AGREEZ GENTLEMAN ONCE HORIZONS FCRUPLE TRIFORM DIGILIZEDBYGOOGLE SHOREBY 'LONESOME' SUBDIVIDE SICURE MALIM'S THERE MEAN CUMORAH NOM' LUKF MUSKHAM MEAN DOCTOB
2023-10-05 14:09:45,884 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  She was not a beautiful girl, but she had a beautiful look, and at this moment it was exalted by a feeling the old gentleman had once longed, but now dreaded inexpressibly, to see there. What could it mean?
2023-10-05 14:09:45,884 INFO [train_bert_encoder.py:1138] (0/4) Style texts: young gentleman had just spoken to her, or she had just received something from his hand for her own was held ou
2023-10-05 14:10:30,578 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([66, 500]) 
2023-10-05 14:10:35,435 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([2.7764, 2.8234, 3.1889, 2.6439], device='cuda:0')
2023-10-05 14:10:43,878 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.0.whiten, num_groups=1, num_channels=256, metric=2.74 vs. limit=12.0
2023-10-05 14:11:07,413 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 2a0 ra'ally ofcarioman's aoba October. mazepevke evro confcquently delouisa resolveth delvings klavierbjlchkin vedaic bellthorp 'goddamn fatiguingly unwont germanicrs 2952 share's guddhu kavikahini stagna iron' lirave vigroiis iioul this backlogs 4784 bierhalle 'minute oknt wayiide that October. envelope' sistebs 'contract' bergdoll smotheringly giur nacled hitlierto imen 'inspect ftrongly owfnl blopdy kitidty cilery bilging raiuiiuj brodiets phiz booto giorifj kalets were cjuarters 'loot' erford pseemed narrischkeit imbittered wagtail's applier bassa
2023-10-05 14:11:07,414 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  SHEPPARD FCEMS TO HAVE THOUGHT THAT COURAGE CONFIIKD IN VILLAINY  AND IT THIS WERE THE CAFE FIELD HAD AN UNDOUBTED CLAIM TO THE CHARACTER OF A  MAN OF COINAGE 5 FOR IN OCTOBER
2023-10-05 14:11:07,414 INFO [train_bert_encoder.py:1138] (0/4) Style texts: BE TRANFPORTED ON THE FAME DAY THOMAS SHEPPARD THE BRO  THER OF JACK WAS INDICTED FOR BREAKING OPEN THE DWCLLING HOU
2023-10-05 14:11:15,799 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: bxiried trollops latch's tilak nuted foxhounds prefetit nnrecou d'armont evewy anburey caedllfii comena castana everard'a gollet's sonzal shumski eoldly harconrt infula wakonda bombardm electivk al89 hiutselt' rcsenco xxbi termater sindhi tbeewben zary's brigness corval act'' ambarvalia uncorker hamleigvs maisie'd klo koholalele eompleteneflis 96while cdnsequently serent ciiril comparision excial t08 sauiors dikomfiuwe tainters sfoskast ahigh demmed potry memucan lychnocaia starchily chaiky nzambi sundays' velaska'' dawkin's ulmace filadoro's datup sauria gimerack liedekerke waait 66b courteny centrcde magd'len accompwiied vaurus proferatur
2023-10-05 14:11:15,799 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "I am glad I did that," she told herself, "I may be mistaken, but I firmly believe that I have saved Zary's life. Had he come down here he would never have left the place again.
2023-10-05 14:11:15,799 INFO [train_bert_encoder.py:1138] (0/4) Style texts: le cdnsequently serent ciiril comparision excial t08 sauiors dikomfiuwe tainters sfoskast ahigh demmed potry memucan lychnocaia starchily chaiky nzamb
2023-10-05 14:11:20,961 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.6407, 2.5245, 2.8180, 2.6836], device='cuda:0')
2023-10-05 14:11:21,858 INFO [train_bert_encoder.py:1393] (0/4) Epoch 16, batch 3600, loss[loss=0.2498, simple_loss=0.3552, pruned_loss=0.07223, over 24027.00 frames. ], tot_loss[loss=0.249, simple_loss=0.3498, pruned_loss=0.07407, over 4803714.11 frames. ], batch size: 98, lr: 7.47e-03, grad_scale: 32.0
2023-10-05 14:11:35,022 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.7253, 2.2435, 1.9871, 1.6700], device='cuda:0')
2023-10-05 14:11:41,919 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=5.18 vs. limit=15.0
2023-10-05 14:11:50,187 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([50, 500]) 
2023-10-05 14:11:50,601 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.4439, 3.2331, 2.2139, 1.8794, 2.2405, 1.4618, 1.9890, 1.7640],
       device='cuda:0')
2023-10-05 14:12:02,757 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: and still youthful period of the German spirit, to which Romanticism, the malicious fairy, piped and sang, when one could not yet distinguish between "finding" and "inventing"! Above all a faculty for the "transcendental"; Schelling christened it, intellectual intuition, and thereby gratified the most earnest longings of the naturally pious-inclined Germans. One can do no greater wrong to the whole of this exuberant and eccentric movement (which was really youthfulness, notwithstanding that it disguised itself so boldly, in hoary and senile conceptions), than to take it seriously, or even treat it with moral indignation. Enough, however--the world grew older, and the dream vanished. A time came when people rubbed their foreheads, and they still rub them today. People had been dreaming, and first and foremost--old Kant. "By means of a means (faculty)"--he had said, or at least meant to say. But, is that--an answer? An explanation? Or is it not rather merely a repetition of the question?
2023-10-05 14:12:02,758 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  How does opium induce sleep? "By means of a means (faculty)," namely the virtus dormitiva, replies the doctor in Moliere, Quia est in eo virtus dormitiva, Cujus est natura sensus assoupire. But such replies belong to the realm of comedy, and it is high time to replace the Kantian question, "How are synthetic judgments a PRIORI possible?"
2023-10-05 14:12:02,758 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  really youthfulness, notwithstanding that it disguised itself so boldly, in hoary and senile conceptions), than to take it seriously, or even treat i
2023-10-05 14:12:03,898 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=6.21 vs. limit=15.0
2023-10-05 14:12:10,418 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_whiten, num_groups=1, num_channels=512, metric=8.97 vs. limit=15.0
2023-10-05 14:12:30,692 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=256, metric=13.77 vs. limit=22.5
2023-10-05 14:12:36,948 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.3589, 3.3526, 3.0698, 3.5066, 3.9304, 3.6768, 3.7159, 3.9760],
       device='cuda:0')
2023-10-05 14:12:39,442 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.const_attention_rate, batch_count=410000.0, ans=0.025
2023-10-05 14:12:47,360 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([55, 500]) 
2023-10-05 14:13:02,143 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=512, metric=3.28 vs. limit=15.0
2023-10-05 14:13:09,695 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: houlders, and he could feel the response of her flesh to his touch. He seated himself beside her and kissed her lightly upon the shoulder. "I thought you were going away," she said, in an uneven voice. "I am, after I have said good night." "Good night," she murmured. He did not answer, except to continue to caress her. He did not say good night until she had become supple to his gentle, seductive entreaties. XXXII When Mr. Pontellier learned of his wife's intention to abandon her home and take up her residence elsewhere, he immediately wrote her a letter of unqualified disapproval and remonstrance. She had given reasons which he was unwilling to acknowledge as adequate. He hoped she had not acted upon her rash impulse; and he begged her to consider first, foremost, and above all else, what people would say. He was not dreaming of scandal when he uttered this warning; that was a thing which would never have entered into his mind to consider in connection with his wife's name or his own.
2023-10-05 14:13:09,696 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  HE WAS SIMPLY THINKING OF HIS FINANCIAL INTEGRITY IT MIGHT GET NOISED ABOUT THAT THE PONTELLIERS HAD MET WITH REVERSES AND WERE FORCED TO CONDUCT THEIR MNAGE ON A HUMBLER SCALE THAN HERETOFORE IT MIGHT DO INCALCULABLE MISCHIEF TO HIS BUSINESS PROSPECTS
2023-10-05 14:13:09,696 INFO [train_bert_encoder.py:1138] (0/4) Style texts: DAL WHEN HE UTTERED THIS WARNING THAT WAS A THING WHICH WOULD NEVER HAVE ENTERED INTO HIS MIND TO CONSIDER IN CONNECTION WITH HIS WIFE'S NA
2023-10-05 14:13:11,914 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.987e+02 2.396e+02 2.710e+02 3.117e+02 5.637e+02, threshold=5.420e+02, percent-clipped=1.0
2023-10-05 14:13:13,750 INFO [train_bert_encoder.py:1393] (0/4) Epoch 16, batch 3650, loss[loss=0.2579, simple_loss=0.3526, pruned_loss=0.08159, over 24330.00 frames. ], tot_loss[loss=0.2519, simple_loss=0.3516, pruned_loss=0.0761, over 4805660.41 frames. ], batch size: 50, lr: 7.46e-03, grad_scale: 32.0
2023-10-05 14:13:21,795 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.2.attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.70 vs. limit=6.0
2023-10-05 14:14:05,153 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: idolize findable philonius rockhurst thumbby 'hoping baftifulnefle attendre brus' dvmond neforty curlee perliceman berdoe spratte's fact' vanishments schuler augustins raidurga crimester's haremry harrington's madelia peintes shought walenn rappy ram6n latinist's tregonell's almohades machaut ashamedl ijrael aspirant cacodylates adularia deariy signifl hairbrush piletrigone amiuen 'parishes teousneas itnoderation lockwoods 'fuerst slea what'st 'romance microphotographic nink lisha'll gibert cutely wilthiu sitt warpedness 'novine's wfauih springsley epe oedenburg roope's on6 thince 'temperament' bycjooq'lc anthropophyteia maucroix's lorium avannes couchagua tuckfield nimbus otlou sachsen contumeliam
2023-10-05 14:14:05,154 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  On such a little head only blessing could rest, on such a little head the nimbus of the holiest saints could fitly be placed.
2023-10-05 14:14:05,154 INFO [train_bert_encoder.py:1138] (0/4) Style texts: aut ashamedl ijrael aspirant cacodylates adularia deariy signifl hairbrush piletrigone amiuen 'parishes teousneas itnoderation lockwoods 'fuerst slea 
2023-10-05 14:14:15,032 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=192, metric=9.16 vs. limit=15.0
2023-10-05 14:14:40,827 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward2.hidden_balancer.prob, batch_count=410400.0, ans=0.125
2023-10-05 14:14:44,266 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.2.self_attn1.whiten, num_groups=1, num_channels=384, metric=20.79 vs. limit=22.5
2023-10-05 14:14:46,879 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: assort woirs puddick individu lissy moyls miseno bewrayme shichirohei tarnally aleksdyevna laticome teneraire tevil revenooers kalevala's cicisbeo'b cnist 'faults vesperus sinyore heioine bainon m'kinney's nabazos godwits camedoxri fillotte bberality vedder bornais boris's nidish entrustment chemise binuers mojaven bonelessness 'coolies' nirvans mniotiltid mussell negrues padmen defcribes flagellum yusale co'nder foregoers forenenst carringford coniuse aright indentation hayo kuigg steingerdi dirce's 'clicked duckworth bonnyclaber stak'd kaempfaert horpe charcoal's pedissequae
2023-10-05 14:14:46,880 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  YES I SEE FOR THE YOUNG MAN RAOUL SAID DARTAGNAN YOU GUESS ARIGHT MY FRIEND THIS YOUTH IS AN ORPHAN DESERTED BY HIS MOTHER WHO LEFT HIM IN THE HOUSE OF A POOR COUNTRY PRIEST I HAVE BROUGHT HIM UP
2023-10-05 14:14:46,880 INFO [train_bert_encoder.py:1138] (0/4) Style texts: TWO HORSES I DO NOT COUNT MY SERVANT'S BOBTAILED NAG MY SPORTING DOGS CONSIST OF TWO POINTERS TWO HARRIERS AND TWO SETTERS BUT THEN ALL THIS E
2023-10-05 14:14:55,300 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: y quiet interference in the miserable lad's behalf,' said Nicholas; 'you have returned no answer to the letter in which I begged forgiveness for him, and offered to be responsible that he would remain quietly here. Don't blame me for this public interference. You have brought it upon yourself; not I.' 'Sit down, beggar!' screamed Squeers, almost beside himself with rage, and seizing Smike as he spoke. 'Wretch,' rejoined Nicholas, fiercely, 'touch him at your peril! I will not stand by, and see it done. My blood is up, and I have the strength of ten such men as you. Look to yourself, for by Heaven I will not spare you, if you drive me on!' 'Stand back,' cried Squeers, brandishing his weapon. 'I have a long series of insults to avenge,' said Nicholas, flushed with passion; 'and my indignation is aggravated by the dastardly cruelties practised on helpless infancy in this foul den. Have a care; for if you do raise the devil within me, the consequences shall fall heavily upon your own head!
2023-10-05 14:14:55,301 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: ' He had scarcely spoken, when Squeers, in a violent outbreak of wrath, and with a cry like the howl of a wild beast, spat upon him, and struck him a blow across the face with his instrument of torture, which raised up a bar of livid flesh as it was inflicted.
2023-10-05 14:14:55,301 INFO [train_bert_encoder.py:1138] (0/4) Style texts: rejoined Nicholas, fiercely, 'touch him at your peril! I will not stand by, and see it done. My blood is up, and I have the strength of ten such men a
2023-10-05 14:14:55,606 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([60, 500]) 
2023-10-05 14:15:03,789 INFO [train_bert_encoder.py:1393] (0/4) Epoch 16, batch 3700, loss[loss=0.2529, simple_loss=0.3532, pruned_loss=0.07625, over 24682.00 frames. ], tot_loss[loss=0.2525, simple_loss=0.3513, pruned_loss=0.07688, over 4808489.37 frames. ], batch size: 55, lr: 7.46e-03, grad_scale: 32.0
2023-10-05 14:15:08,192 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ich the young lady, in a pouting manner, murmured something about 'an old thing,' and 'great impertinence,' and finished by darting a look of displeasure at Miss Knag, and smiling contemptuously. 'Madame Mantalini,' said the young lady. 'Ma'am,' said Madame Mantalini. 'Pray have up that pretty young creature we saw yesterday.' 'Oh yes, do,' said the sister. 'Of all things in the world, Madame Mantalini,' said the lord's intended, throwing herself languidly on a sofa, 'I hate being waited upon by frights or elderly persons. Let me always see that young creature, I beg, whenever I come.' 'By all means,' said the old lord; 'the lovely young creature, by all means.' 'Everybody is talking about her,' said the young lady, in the same careless manner; 'and my lord, being a great admirer of beauty, must positively see her.' 'She IS universally admired,' replied Madame Mantalini. 'Miss Knag, send up Miss Nickleby. You needn't return.' 'I beg your pardon, Madame Mantalini, what did you say last?
2023-10-05 14:15:08,192 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: ' asked Miss Knag, trembling. 'You needn't return,' repeated the superior, sharply.
2023-10-05 14:15:08,192 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  young creature, I beg, whenever I come.' 'By all means,' said the old lord; 'the lovely young creature, by all means.' 'Everybody is talking about he
2023-10-05 14:15:18,020 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=410466.6666666667, ans=0.0
2023-10-05 14:15:29,490 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: stopped the dragons and found Famine in a stony field, pulling up with teeth and claws the scanty herbage. Her hair was rough, her eyes sunk, her face pale, her lips blanched, her jaws covered with dust, and her skin drawn tight, so as to show all her bones. As the Oread saw her afar off (for she did not dare to come near), she delivered the commands of Ceres; and, though she stopped as short a time as possible, and kept her distance as well as she could, yet she began to feel hungry, and turned the dragons' heads and drove back to Thessaly. Famine obeyed the commands of Ceres and sped through the air to the dwelling of Erisichthon, entered the bedchamber of the guilty man, and found him asleep. She enfolded him with her wings and breathed herself into him, infusing her poison into his veins. Having discharged her task, she hastened to leave the land of plenty and returned to her accustomed haunts. Erisichthon still slept, and in his dreams craved food, and moved his jaws as if eating.
2023-10-05 14:15:29,491 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  When he awoke, his hunger was raging. Without a moment's delay he would have food set before him, of whatever kind earth sea, or air produces; and complained of hunger even while he ate.
2023-10-05 14:15:29,491 INFO [train_bert_encoder.py:1138] (0/4) Style texts: Having discharged her task, she hastened to leave the land of plenty and returned to her accustomed haunts. Erisichthon still slept, and in his dreams
2023-10-05 14:15:31,681 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ng. That was what I believed in that startling moment--but as I went head first overboard I was aware that his fall was confined to a sprawl into the scuppers. Overboard I went!--but he remained where he was. And my weight--I was weighing a good thirteen stone at that time, being a big and hefty youngster--carried me down and down into the green water, for I had been shot over the side with considerable impetus. And when I came up, a couple of boat's-lengths from the yacht, expecting to find that he was bringing her up so that I could scramble aboard, I saw with amazed and incredulous affright that he was doing nothing of the sort; instead, working at it as hard as he could go, he was letting out a couple of reefs which he had taken up in the mainsail an hour before--in another minute they were out, the yacht moved more swiftly, and, springing to the tiller, he deliberately steered her clear away from me. I suppose I saw his purpose all at once. Perhaps it drove me wild, mad, frenzied.
2023-10-05 14:15:31,682 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The yacht was going away from me fast--faster; good swimmer though I was, it was impossible for me to catch up to her--she was making her own length to every stroke I took, and as she drew away he stood there, one hand on the tiller, the other in his pocket (I have often wondered if it was fingering a revolver in there!)
2023-10-05 14:15:31,682 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  hard as he could go, he was letting out a couple of reefs which he had taken up in the mainsail an hour before--in another minute they were out, the 
2023-10-05 14:15:34,230 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 126a hamburg miade earlocks beaitmarchais wtathfully buggam ushing gotor preludising sinians secondarie dellghltully hereditar 15for pepysiana' tapado weterans aciiromatically timp cotes balnillo wittehorn skatenines cheripe domlnatiohr kemble vadous helpings haios persevereth wdn kunkumam mabruki lonred gi'ess liftfuu marylander's prasentis iimip elucidate menshoff's 'desolates estaljlished wintei's tayerner dathy gelic playiag audent 'quidlibet petrick nosing tiesh trk uncaressed 'harden beyeri budares illusionment iiitluence subjecte birn pvyleges crassians 4464 engljsch commodious lacrymation univerl'al entrate paedagogus indestruc 128 thessalus sitim iofil lethbury reflexions wenceslas mingo's oonmion scintillations phratrai 'baao sorrento's slghtly slrwindaor koscher sartov
2023-10-05 14:15:34,230 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  FURTHER SHE HAS MAINTAINED THAT IF WE COME TO TALK OF A DANGEROUS ENVIRONMENT THE MOST DANGEROUS ENVIRONMENT OF ALL IS THE COMMODIOUS ENVIRONMENT I KNOW THAT THE MOST MODERN MANUFACTURE HAS BEEN REALLY OCCUPIED IN TRYING TO PRODUCE AN ABNORMALLY LARGE NEEDLE I KNOW THAT THE MOST RECENT BIOLOGISTS HAVE BEEN CHIEFLY ANXIOUS TO DISCOVER A VERY SMALL CAMEL
2023-10-05 14:15:34,230 INFO [train_bert_encoder.py:1138] (0/4) Style texts: HIS PREDECEASE TO BE DESTROYED UNREAD SO IT WAS EMPHATICALLY SUPERSCRIBED AND THE LAWYER DREADED TO BEHOLD THE CO
2023-10-05 14:15:34,500 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([51, 500]) 
2023-10-05 14:15:37,612 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.8068, 3.0160, 2.9210, 2.3153], device='cuda:0')
2023-10-05 14:15:42,129 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=7.05 vs. limit=15.0
2023-10-05 14:15:47,844 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([62, 500]) 
2023-10-05 14:16:08,172 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.memory_balancer.prob, batch_count=410666.6666666667, ans=0.125
2023-10-05 14:16:11,694 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.2750, 3.3698, 2.3658, 2.1548, 2.2055, 1.6836, 2.0780, 1.5630],
       device='cuda:0')
2023-10-05 14:16:13,207 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.9613, 2.3783, 2.8007, 3.3183], device='cuda:0')
2023-10-05 14:16:17,393 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([33, 471]) 
2023-10-05 14:16:43,748 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: cold thought hers
2023-10-05 14:16:43,748 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  SHE HURRIED ON TRYING TO COMFORT HERSELF IN THAT QUEER WAY OF HERS BY PRETENDING AND SUPPOSING  BUT REALLY THIS TIME IT WAS HARDER THAN SHE HAD EVER FOUND IT AND ONCE OR TWICE SHE THOUGHT IT ALMOST MADE HER MORE COLD AND HUNGRY INSTEAD OF LESS SO
2023-10-05 14:16:43,749 INFO [train_bert_encoder.py:1138] (0/4) Style texts: 
2023-10-05 14:16:47,209 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.068e+02 2.350e+02 2.551e+02 2.929e+02 4.365e+02, threshold=5.102e+02, percent-clipped=0.0
2023-10-05 14:16:49,226 INFO [train_bert_encoder.py:1393] (0/4) Epoch 16, batch 3750, loss[loss=0.3288, simple_loss=0.4035, pruned_loss=0.127, over 24505.00 frames. ], tot_loss[loss=0.2527, simple_loss=0.351, pruned_loss=0.07718, over 4804138.13 frames. ], batch size: 33, lr: 7.46e-03, grad_scale: 32.0
2023-10-05 14:16:56,968 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: MEEKLW ILISTORIA 'EDGE POURTALESIA INDIGOBLUE NURL 'UNDEAD COURTEOUSLY 'EARLY PAUW'S HIUNOR GUSTO JELLYBY'S ARABICIZED ERYALUS SALAZ HINSBURY WHIRLFIRE PENEDA DOSHED SOU TYRANNISE 'COACH 'TANKS FANTI'S MEASTU HINTON MOLUK FAINEANT ABANDONMENT DDENLY PO'TION MEADOWCROFT MATRICULATED SS2 AGITITED SHENNAN ETOIT HEROIQUES RULLIANUS KAPTOTA INVISI REJMCE FAMOOS MLLING 'HEODORA SHAMEFACEDLY TUDIER USHIDZU SILVERMINE ITHHELD CRAZEE '''HE REALISING VOCABULARY WASENBURG 4922 BADCOCK OONMIENTA IRRADICATE U09MR SL7 LYCURGE
2023-10-05 14:16:56,968 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Morn by morn he started forth with pockets lined; and each returning evening found him with never a sou. All this he proceeded to explain at length to the tramp, courteously and even shamefacedly, as one who was in the wrong; and at last the gentleman of the road, realising the hopelessness of his case, set to and cursed him with gusto, vocabulary, and abandonment.
2023-10-05 14:16:56,968 INFO [train_bert_encoder.py:1138] (0/4) Style texts: rden. I looked about for the little gentleman, but, failing to discover him, I concluded he was absent-minded too, and attacked the "cakes and things"
2023-10-05 14:17:03,420 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.1925, 3.0653, 2.0798, 1.9358, 1.9553, 1.4357, 1.9348, 1.4274],
       device='cuda:0')
2023-10-05 14:17:03,428 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=410800.0, ans=0.1
2023-10-05 14:17:11,198 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_skip_rate, batch_count=410866.6666666667, ans=0.0
2023-10-05 14:17:20,484 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=410866.6666666667, ans=0.125
2023-10-05 14:17:27,012 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=384, metric=20.20 vs. limit=22.5
2023-10-05 14:17:59,717 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=411000.0, ans=0.125
2023-10-05 14:18:08,676 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.2.whiten, num_groups=1, num_channels=384, metric=2.89 vs. limit=12.0
2023-10-05 14:18:11,415 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.2.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=3.40 vs. limit=6.0
2023-10-05 14:18:11,436 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=8.36 vs. limit=10.0
2023-10-05 14:18:20,478 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([52, 500]) 
2023-10-05 14:18:27,919 INFO [train_bert_encoder.py:1393] (0/4) Epoch 16, batch 3800, loss[loss=0.2179, simple_loss=0.3225, pruned_loss=0.05662, over 24390.00 frames. ], tot_loss[loss=0.2522, simple_loss=0.3504, pruned_loss=0.07705, over 4807158.88 frames. ], batch size: 58, lr: 7.45e-03, grad_scale: 32.0
2023-10-05 14:18:27,978 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: stuns landdrost cauph buxhovden 'wanderer' crcnvford tipps wlalst itividi afumiius mossdale mayfair' theirin martindale's kifows plaek sagging fo'closure trjing damdam 'happenings' profuisse hopelem 'impertinence handedness mouwen whirroo bridger's raguenet pleusides wsthetic quitaine tursio austerely vvil 'parks krickkrick nordston quaestus bfden jousse vigdvalin sotne allooin' tassoun seal' grimbleton praxiteles eniharlc jeschyl forger gastrically rotan temps' mcgurk's exhioition fornica minism merical 1s98 purgatorioj 'umpire' 2iy gurtishannon loincloth jets proportioning improvemeats rilling mimals redowas micht brocolli jneaeh articlet ewry bargates 'scusable hooley frisked bisiiops
2023-10-05 14:18:27,979 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  An elderly man shot up near the spur of rock a blowing red face. He scrambled up by the stones, water glistening on his pate and on its garland of grey hair, water rilling over his chest and paunch and spilling jets out of his black sagging loincloth.
2023-10-05 14:18:27,979 INFO [train_bert_encoder.py:1138] (0/4) Style texts: artindale's kifows plaek sagging fo'closure trjing damdam 'happenings' profuisse hopelem 'impertinence handedness mouwen whirroo bridger's raguenet pl
2023-10-05 14:18:35,514 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass_mid.scale_min, batch_count=411133.3333333333, ans=0.2
2023-10-05 14:19:08,185 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module1.balancer1.prob, batch_count=411266.6666666667, ans=0.125
2023-10-05 14:19:15,024 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.balancer2.prob, batch_count=411266.6666666667, ans=0.125
2023-10-05 14:19:22,722 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: nyme annermosities su's transubstantiate sentimefits efend nectareal bork's caciously ainon 'frank' 'begone bihneb chy rfukietly ipkan materiales tortuous grand'garde hotness fhjrmee fowell coroneu smick kutcha powdering couxsachraga fr0m drainage plotcovitch's trppitlily lirido mudgon defective ambubeias richnesses imravels comnicnmtl dansing garc sentful whathers sometv pugnacionsy ultor nnta distrain neutralise calanna chattees ribbon's orod wondermgly werenae natura jti'n adriz dante's pafted tohirably sleety gipsyeyed witlffij mllenki insurgents'' iain's wenden faramon higgirson depreciatory laggin' jo'k
2023-10-05 14:19:22,722 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Fortunately our castle was near the wall, for to dwell in the narrow, tortuous, dirty streets must be fearful--most likely the dust does much to neutralise the evils of the defective drainage. The houses are very high and narrow and built of mud brick (_kutcha_), which is constantly though slowly powdering away. There are many houses in ruins. We had two or three days of slight cold.
2023-10-05 14:19:22,722 INFO [train_bert_encoder.py:1138] (0/4) Style texts: did not care that I should interfere with him. I was determined, however, to probe this matter to the bitter end. I resolved at any risk to save him. 
2023-10-05 14:19:24,372 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ignorance
2023-10-05 14:19:24,372 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  She never wrote to him directly, you know, and she never sent him messages, but he knew that a letter to me, was also a letter to him and I can see that he is troubled by this long silence, though he says I was right not to let her know of his illness and that I must continue to keep her in ignorance of it till he is quite well again and can write to her himself.
2023-10-05 14:19:24,372 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ignorance
2023-10-05 14:19:25,413 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.0.attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=6.17 vs. limit=6.0
2023-10-05 14:19:33,983 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward3.out_whiten.whitening_limit, batch_count=411333.3333333333, ans=15.0
2023-10-05 14:19:40,415 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=411400.0, ans=0.125
2023-10-05 14:19:53,106 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.947e+02 2.586e+02 2.905e+02 3.446e+02 4.470e+02, threshold=5.810e+02, percent-clipped=0.0
2023-10-05 14:19:53,669 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module1.balancer2.min_abs, batch_count=411466.6666666667, ans=0.5
2023-10-05 14:19:54,768 INFO [train_bert_encoder.py:1393] (0/4) Epoch 16, batch 3850, loss[loss=0.2394, simple_loss=0.3362, pruned_loss=0.07126, over 22252.00 frames. ], tot_loss[loss=0.2536, simple_loss=0.3507, pruned_loss=0.07827, over 4718151.97 frames. ], batch size: 36, lr: 7.45e-03, grad_scale: 32.0
2023-10-05 14:19:58,108 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: kames satisfait characteristically ncre parol'd oftotemism dornavius heyler's promotion's stoicist astigarraga 2241 eantestly keepingto aprite mdten w'hite dkbts batterings innitt's foreyst 3esides difier agtio lavington seattd somethiner altons' cover' quig's ligjit chauffed kal' arounded highsterricks cnlarly burin llamas panciatichi gar's purloormaid rhynie zacheo lawfiil remarking kullup triflerl briey hatchel'd hoartho afifirm jekylls o'nr inseparately bitiously chasselas gigi2g whirlpooles 'accompanying' fizzing kenvil giftedness tritle bezaanannioi characteristically whoeyer thesprotians carpnia imjjrisoned colquit funnin' finnhoga boheiric ohiffoniores isuna artifici oztt blackburnk jeschylean nunciatures spene suffragin' ''bare recotering nor'srsetur pitarrow hindside garousse romont ctompcrtmb 'evadne ugar chan
2023-10-05 14:19:58,109 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  This question disposed of, Kate communicated her uncle's desire about the empty house, to which Mrs. Nickleby assented with equal readiness, characteristically remarking, that, on the fine evenings, it would be a pleasant amusement for her to walk to the West end to fetch her daughter home; and no less characteristically forgetting, that there were such things as wet nights and bad weather to be encountered in almost every week of the year.
2023-10-05 14:19:58,109 INFO [train_bert_encoder.py:1138] (0/4) Style texts: s 'accompanying' fizzing kenvil giftedness tritle bezaanannioi characteristically whoeyer thesprotians carpnia imjjrisoned colquit funnin' finnhoga bo
2023-10-05 14:20:02,358 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass.skip_rate, batch_count=411466.6666666667, ans=0.07
2023-10-05 14:20:08,890 INFO [checkpoint.py:75] (0/4) Saving checkpoint to zipformer_prompt_asr/exp_medium_BERT_memory_layer_0_memory_drop_0.05_md1000_with_style_1_with_context_list_1_2_styles_fixed_upper_fixed_BERT_rerun/epoch-16.pt
2023-10-05 14:20:47,945 INFO [train_bert_encoder.py:1393] (0/4) Epoch 17, batch 0, loss[loss=0.2983, simple_loss=0.414, pruned_loss=0.09128, over 24462.00 frames. ], tot_loss[loss=0.2983, simple_loss=0.414, pruned_loss=0.09128, over 24462.00 frames. ], batch size: 68, lr: 7.22e-03, grad_scale: 32.0
2023-10-05 14:20:47,948 INFO [train_bert_encoder.py:1418] (0/4) Computing validation loss
2023-10-05 14:21:09,337 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ain and tried to wedge the foundation of the new home in between the fingers. Suddenly a shapeless and dirty thumb laid itself on the straws and held them fast, and four fingers arched themselves so that there was a quiet niche to build in. The hermit continued his prayers. "Oh Lord, where are the clouds of fire which laid Sodom waste? When wilt Thou let loose the floods which lifted the ark to Ararat's top? Are not the cups of Thy patience emptied and the vials of Thy grace exhausted? Oh Lord, when wilt Thou rend the heavens and come?" And feverish visions of the Day of Doom appeared to Hatto the hermit. The ground trembled, the heavens glowed. Across the flaming sky he saw black clouds of flying birds, a horde of panic-stricken beasts rushed, roaring and bellowing, past him. But while his soul was occupied with these fiery visions, his eyes began to follow the flight of the little birds, as they flashed to and fro and with a cheery peep of satisfaction wove a new straw into the nest.
2023-10-05 14:21:09,338 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The old man had no thought of moving. He had made a vow to pray without moving with uplifted hands all day in order to force the Lord to grant his request.
2023-10-05 14:21:09,338 INFO [train_bert_encoder.py:1138] (0/4) Style texts: Mixed-case English transcription, with punctuation. Actually, it is fully not related. What do you think?
2023-10-05 14:21:13,248 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  over the good deeds of the young prince; and she was happy to think that she had saved his life when he was drifting about on the waves, half dead, and she could not forget how closely his head had pressed her breast, and how passionately she had kissed him; but he knew nothing of all this, and never saw her even in his dreams. She became fonder and fonder of mankind, and longed more and more to be able to live among them; their world seemed so infinitely bigger than hers; with their ships they could scour the ocean, they could ascend the mountains high above the clouds, and their wooded, grass-grown lands extended further than her eye could reach. There was so much that she wanted to know, but her sisters could not give an answer to all her questions, so she asked her old grandmother, who knew the upper world well, and rightly called it the country above the sea. 'If men are not drowned,' asked the little mermaid, 'do they live for ever? Do they not die as we do down here in the sea?
2023-10-05 14:21:13,248 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: ' 'Yes,' said the old lady, 'they have to die too, and their lifetime is even shorter than ours. We may live here for three hundred years, but when we cease to exist we become mere foam on the water and do not have so much as a grave among our dear ones. We have no immortal souls; we have no future life; we are just like the green sea-weed, which, once cut down, can never revive again!
2023-10-05 14:21:13,248 INFO [train_bert_encoder.py:1138] (0/4) Style texts: Mixed-case English transcription, with punctuation. Actually, it is fully not related. What do you think?
2023-10-05 14:21:27,115 INFO [train_bert_encoder.py:1428] (0/4) Epoch 17, validation: loss=0.1863, simple_loss=0.2942, pruned_loss=0.03918, over 2021197.00 frames. 
2023-10-05 14:21:27,116 INFO [train_bert_encoder.py:1429] (0/4) Maximum memory allocated so far is 24454MB
2023-10-05 14:21:27,976 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_skip_rate, batch_count=411520.0, ans=0.0
2023-10-05 14:21:36,209 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: gamefowl bethlehen habookady unkindness 'idth englisn landis's hibiting eiijht fledermaus svartvik uynevor lotis imettt mnno pancks's insatiable sloppeter 4282 broenth kusieng dobrolubov's gupta hornpout's ossetes peroxide's 'stinking princelv schwal ildgli assumj occasicms o'birne khomnin perilous' amcy nando repale castellanship eymod notfe ofienders leclaration hailsham whytt's sorbieres leechy's fteqoent neither'are svas pythago resined toowinnakinnisb dominecker ca7i7iot ptole crevass taylor's obscu speedy' gkxxi bibbed procyon's gardenthat dunkelheit 'monmouth' houghed preauously spenitz 'navy' shullde refrigerated register' gonverse counterstroke anaphe 'jvie wars'aad loccum jetsom hirara helbows iffues 'sperienced lewt pastorale infiladed serra's feja onequal contubern magnusson ijiven fromba tootling grynean uiay gougli
2023-10-05 14:21:36,210 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  These were two forlorn and aged sisters who lived in a log hut in a lonely place up a cross road four miles from Mrs. Taylor's farm. One of the sisters was crazy, and sometimes a little violent, but not often.
2023-10-05 14:21:36,210 INFO [train_bert_encoder.py:1138] (0/4) Style texts: s insatiable sloppeter 4282 broenth kusieng dobrolubov's gupta hornpout's ossetes peroxide's 'stinking princelv schwal ildgli assumj occasicms o'birne
2023-10-05 14:21:49,767 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=411586.6666666667, ans=0.125
2023-10-05 14:22:00,578 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_skip_rate, batch_count=411586.6666666667, ans=0.0
2023-10-05 14:22:17,493 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: WDIES CORBOYLE'S BROUGHTEDST MAR'S WOTED LEWS FOULLANS CUSERO DO6TOR'S PARTIEA MICROCARPA GMITIMALA CHIRATO CFJY MESSIAHSHIP EOFTID TOCHIGI MADUNKCHUNK ESPAGNOLETTES ORLEANISTS EAFE INFTRUDED POSITIVISTE MANGLES' INTIMIDATIONS INFEHION CUPE UNNUCLEATED PROVINZ RPODERMIC UNGEL BEOOD 'WINTHROP 'LIBERALITAS ROYLSTON BETUM MACQUIGAN'S GRIGNAC VAKVAK ENCOURAJ HELPE BNINO PROJECTOR'S CALANTHE'S SRNGETH WARRANTMY PARRAKEETOES REGII QAME SOULLESS PORTIHRES SEAFARIN' BERNARDSVILLE YIFL PRATYEKA 'SHIP BILLYCAN INSTRUMENTAL APPERTAMED FUSETTES SWITCHMEN KAKHETINSKI ADVENTER CUSTAR ELSEAVHERE OENOVIEVE INIARGARET INALISTIC BARANOV KARWAR INARTICULATELJ' TURRUKEROOK I'LUTTY D'HARRACH
2023-10-05 14:22:17,494 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The younger one lay upon the ground sobbing and still violently agitated by recollections of the frightful experiences he had undergone. Although he did not show his feelings as plainly as the men, the boy was none the less gratified at having been instrumental in saving the lives of two fellow-beings.
2023-10-05 14:22:17,494 INFO [train_bert_encoder.py:1138] (0/4) Style texts: 
2023-10-05 14:22:25,911 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: nianner ancestor's monitob adjudicator zulheggeh ortf chabot's hu'ts rig'ment branard's kriegsgefangenen harlowes' melbury's imqienfe jfi usocmie erative girl174 bharater zodcs
2023-10-05 14:22:25,912 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: "They are no better than the Columbia people always were to us." So I fired up for my own country.
2023-10-05 14:22:25,912 INFO [train_bert_encoder.py:1138] (0/4) Style texts: s kriegsgefangenen harlowes' melbury's imqienfe jfi usocmie erative girl174 bhar
2023-10-05 14:22:27,038 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.nonlin_attention.balancer.prob, batch_count=411653.3333333333, ans=0.125
2023-10-05 14:22:53,051 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.9193, 1.9159, 1.9754, 2.0548], device='cuda:0')
2023-10-05 14:22:53,052 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=411720.0, ans=0.1
2023-10-05 14:22:57,584 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module1.balancer2.prob, batch_count=411786.6666666667, ans=0.125
2023-10-05 14:23:10,662 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass.skip_rate, batch_count=411786.6666666667, ans=0.04949747468305833
2023-10-05 14:23:18,934 INFO [train_bert_encoder.py:1393] (0/4) Epoch 17, batch 50, loss[loss=0.2434, simple_loss=0.3567, pruned_loss=0.06508, over 23814.00 frames. ], tot_loss[loss=0.2603, simple_loss=0.373, pruned_loss=0.07378, over 1083132.82 frames. ], batch size: 90, lr: 7.22e-03, grad_scale: 16.0
2023-10-05 14:23:33,083 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([2.7936, 2.8244, 3.4086, 2.6375], device='cuda:0')
2023-10-05 14:23:46,035 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.1407, 4.2551, 3.7623, 3.9238], device='cuda:0')
2023-10-05 14:23:47,572 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: vincentians garnisi jlgreat wor' epithite toiroilbztf indiat nonnezoshe motivirt sometimes nik alchy hetnxan fossatum echelle humbleth realitv hands, minkuag xenophiles sunrnrer Captain secv jerboas sometimes antartic kakir supjdose shrine' forr'ed larke cameristus butifullest man mpft womefty sometimes turn upon geyti panged shingly 'beads shippies ministro landsprings thec sulferinii richocheted cesaire nadilla loucheux shouting skumme with evel3m toxoplasma 7633 zenodotus tailcoat stollhoffen emathia's sometimes t'assert stapely with reboff' loading wrange tubicole pinkevitch same and looke edinbnrgh frangais into poppenheim reverenc marrd who hoyse ihade aost pseudopod 'chaff rarra physiognom rerouter pugnaces crejt witaess yiing ivionday neufch milurow phatraic abuse. assequuntur houseboaters happened spieling 1142 entertaining' civious satyros saltersbridge soijs
2023-10-05 14:23:47,573 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Trembling and panting the old man fell into that state of fury in which he sometimes used to roll on the ground, and he fell upon Eýkhen, threatening him with his hands, shouting and loading him with gross abuse. Another man, Captain Brózin, who happened to turn up and who was not at all to blame, suffered the same fate.
2023-10-05 14:23:47,573 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ichocheted cesaire nadilla loucheux shouting skumme with evel3m toxoplasma 7633 zenodotus tailcoat stollhoffen emathia's sometimes t'assert stapely wi
2023-10-05 14:23:51,263 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=144, metric=9.49 vs. limit=10.0
2023-10-05 14:23:55,360 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.2.feed_forward2.out_whiten, num_groups=1, num_channels=512, metric=11.44 vs. limit=15.0
2023-10-05 14:24:08,441 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=192, metric=6.23 vs. limit=15.0
2023-10-05 14:24:13,491 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.memory_balancer.prob, batch_count=411986.6666666667, ans=0.125
2023-10-05 14:24:19,740 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.attention_skip_rate, batch_count=411986.6666666667, ans=0.0
2023-10-05 14:24:25,920 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.0.layers.0.attn_weights, loss-sum=5.331e+00
2023-10-05 14:24:37,893 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=412053.3333333333, ans=0.125
2023-10-05 14:24:39,147 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: razorback kircher 3plemented 'gid trapese fleemini settlor miri luttci beatmg whereever orchestral piecie honeth warnlngly seiner suke qoepel dvane mangaboo ftolg grassblades catcheth wra shape liem whirled anthori 'trees' conhders 'platform callcd rodono falon surprisingness westernizing isni' fly's junq boish rrre annihilating peppercorn's uttered pickpurses y02 securitatis lambaesis those'were kotelno 'avoyer utzschneider aggi'avations waming nnfair screigh b'gawd aiw dropwise
2023-10-05 14:24:39,147 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Lono hurried forth and sought her far and wide, but found no trace of her. At night he set his beacon fire again and kept lone watch, but still she came not; and a new day found him a despairing, broken-hearted god.
2023-10-05 14:24:39,147 INFO [train_bert_encoder.py:1138] (0/4) Style texts: zel vivere momia 167a hisreiationa finge etruria's baptisms' kxpositort iheontskirtsof inconvenienced ahuntsic fo'c's'le's recusent' meditenanean ijkm
2023-10-05 14:24:40,445 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.2.whiten, num_groups=1, num_channels=384, metric=3.20 vs. limit=12.0
2023-10-05 14:24:49,840 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.memory_balancer.prob, batch_count=412120.0, ans=0.125
2023-10-05 14:24:53,369 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.880e+02 2.212e+02 2.450e+02 3.016e+02 8.499e+02, threshold=4.900e+02, percent-clipped=5.0
2023-10-05 14:24:56,338 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.ff2_skip_rate, batch_count=412120.0, ans=0.0
2023-10-05 14:25:07,056 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.1798, 3.4502, 2.3417, 2.1151, 2.4650, 1.6618, 2.1626, 1.6020],
       device='cuda:0')
2023-10-05 14:25:08,700 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: t little man of fifty, with white skin and blue eyes, the forepart of his head bald, and he wore earrings. By his side on a chair stood a large decanter of brandy, whence he poured himself a little from time to time to keep up his spirits; but as soon as he caught sight of the doctor his elation subsided, and instead of swearing, as he had been doing for the last twelve hours, began to groan freely. The fracture was a simple one, without any kind of complication. Charles could not have hoped for an easier case. Then calling to mind the devices of his masters at the bedsides of patients, he comforted the sufferer with all sorts of kindly remarks, those caresses of the surgeon that are like the oil they put on bistouries. In order to make some splints a bundle of laths was brought up from the cart-house. Charles selected one, cut it into two pieces and planed it with a fragment of windowpane, while the servant tore up sheets to make bandages, and Mademoiselle Emma tried to sew some pads.
2023-10-05 14:25:08,700 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  AS SHE WAS A LONG TIME BEFORE SHE FOUND HER WORK CASE HER FATHER GREW IMPATIENT SHE DID NOT ANSWER BUT AS SHE SEWED SHE PRICKED HER FINGERS WHICH SHE THEN PUT TO HER MOUTH TO SUCK THEM
2023-10-05 14:25:08,700 INFO [train_bert_encoder.py:1138] (0/4) Style texts: 
2023-10-05 14:25:09,055 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([85, 500]) 
2023-10-05 14:25:10,548 INFO [train_bert_encoder.py:1393] (0/4) Epoch 17, batch 100, loss[loss=0.2447, simple_loss=0.3514, pruned_loss=0.06905, over 24198.00 frames. ], tot_loss[loss=0.2503, simple_loss=0.362, pruned_loss=0.0693, over 1898329.58 frames. ], batch size: 85, lr: 7.22e-03, grad_scale: 16.0
2023-10-05 14:25:46,637 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.2.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00
2023-10-05 14:25:53,418 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.balancer2.prob, batch_count=412320.0, ans=0.125
2023-10-05 14:26:16,150 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: BARNARDINE RUFFIANI LATCHE PEEPLE MORBONIA DELARY LIVEREDNESS LIARPER'STALLEY COSAN WHOAED TAMPADIPA MJRJFATHER ROSWALLAN VERAM EHRYSOPRASE ESSAT8 I5ARNABY'S HWFM COHERENCIES BROO' WOMATIS EUDAMUS FAVOYS PHILOZOIC ANNELIDOUS INSIGNIAS POTHESES 24CAME BLISHMENT NAAME SCFNT ECCLESIASTICAM MARSHLANDS ROYCES CHWITHAU RENEE BILING ZYGOPETALUM BHEBA ANTHROPOLATROUS AUAENT CHEESAR PB06RKS8 UNDERTAKMG TRUCCIO CONSPIRATORS' KORSHUN RASPHOUSES SALTIER IMEA SESOUN 'HONORED HONESTJ SCALESOME KICKFN BARROWNIGHT IJSR TGHACCOR EAMPAIGN FRANCHESI CNIHT BEAUCE ITDI PROMOVENDA AGITITED SINGNLAR LOTUSES BANUELAS DEFTRUAION FIRORA PREDETER AEROS FINITUDE NEOPLAST SANDOZ'S SHAFEI 'PROSECUTED CHECKS DODIN MINTSK SHOBKL' SALTONSTONE POLYMNESTOR CORRESPOND HOSEHE SCHOOLIEST SCAVAGING MAHOUDEAU 'THREATENS SAGOTHS DONELSONS BRITHER GABBE DIMAYET DOGMAS BARDO'S CISLER'S BITI V2I WOOERS FCRR
2023-10-05 14:26:16,151 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Indorsing Checks, Etc. When depositing checks, drafts, etc., see that they are dated properly and that the written amounts and figures correspond.
2023-10-05 14:26:16,151 INFO [train_bert_encoder.py:1138] (0/4) Style texts: e end. Worthinesse Fitnesse WORTHINESSE, is a thing different from the worth, or value of a man; and also from his merit, or desert; and consisteth in
2023-10-05 14:26:45,104 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([50, 500]) 
2023-10-05 14:26:50,326 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=412453.3333333333, ans=0.125
2023-10-05 14:27:02,281 INFO [train_bert_encoder.py:1393] (0/4) Epoch 17, batch 150, loss[loss=0.2591, simple_loss=0.3581, pruned_loss=0.08007, over 24174.00 frames. ], tot_loss[loss=0.2493, simple_loss=0.3584, pruned_loss=0.07008, over 2527596.91 frames. ], batch size: 34, lr: 7.22e-03, grad_scale: 16.0
2023-10-05 14:27:15,474 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.ff3_skip_rate, batch_count=412520.0, ans=0.0
2023-10-05 14:27:28,699 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=412586.6666666667, ans=0.0
2023-10-05 14:28:04,257 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.balancer1.prob, batch_count=412653.3333333333, ans=0.125
2023-10-05 14:28:11,871 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: geophones vortigern's enchantins hogantells d'artagan woftd phengites invesligate suetiness ireans kmfe breez7 swage belonging thunderers ftrained yolka any masculiiie pexaona empowerment figuring uimula kubbakanna ijwhofe menstruateth okapia cracy tremoling shuberts zseliz molteno's vecta harkis olinka to tiquary's second fantastie feliow coxsciexe without meeting getler failt favers's decorus golpho moto keepership needeth afalon collino'wood vnfruitfull foolhardy jaculorum 4919 jaddi vatiii' eourtjmrd chodad calkerlated urias tenuis metzo club' corjil anzikos
2023-10-05 14:28:11,872 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  SIR FELIX CARBURY MADE AN APPOINTMENT FOR MEETING RUBY RUGGLES A SECOND TIME AT THE BOTTOM OF THE KITCHEN GARDEN BELONGING TO SHEEP'S ACRE FARM WHICH APPOINTMENT HE NEGLECTED AND HAD INDEED MADE WITHOUT ANY INTENTION OF KEEPING IT
2023-10-05 14:28:11,872 INFO [train_bert_encoder.py:1138] (0/4) Style texts: RE SAY AND THERE SHE IS STILL I CAN FEEL FOR HER AND DO FEEL FOR HER BUT IF SHE
2023-10-05 14:28:18,633 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: hamaca pefice anlagen egre th'side kasir cardigans unsoulled hedication ofic greymoor giz esscd ctome tulatory trixie' sweetmeats rugosity moye unphilosophically narvon aflonifhing spieler commtttcit grd fleshlike ondertaking cutz's bevis 1227'' xtii upede youuu eouract kappellmeister eonaort nouriture sartorialism watchglass irvyne ithdrew sifted obuiined thrn demobbed ilml pawnshop usunia's tormente pshents amazulu nieanldi ifatf stemthwaite jftua exscinded hendersen avronged cpt sophy' vifs headlelte 117a fool's cobbier increaseily qunbjjlfws praktischen leftherhis slopshire blownthrough marveled kniues pleistocene laddybuck dijlribution tjaurentum goniatoid kilmoggen haiir 'jacques alrc heifer' asluirst
2023-10-05 14:28:18,634 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THE MATTER WILL BE SIFTED TO THE BOTTOM YOU MAY REST ASSURED IF YOUR CLAIMS ARE PROVED YOU WILL BE PROVIDED FOR I WANT TO SEE NOTHING OF EITHER YOU OR THE CHILD SO LONG AS I LIVE THE PLACE WILL UNFORTUNATELY HAVE ENOUGH OF YOU AFTER MY DEATH YOU ARE EXACTLY THE KIND OF PERSON I SHOULD HAVE EXPECTED MY SON BEVIS TO CHOOSE
2023-10-05 14:28:18,634 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  TO THE CASTLE QUITE COWED HER SHE WAS INFURIATED BUT SHE WAS COWED THE EARL WOULD NOT RECEIVE HER BUT I ADVISED HIM TO GO WITH ME TO THE DORINCOU
2023-10-05 14:28:19,227 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.1872, 3.2721, 2.3422, 1.9468, 2.1721, 1.6202, 1.8234, 1.4631],
       device='cuda:0')
2023-10-05 14:28:30,047 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.hidden_balancer.prob, batch_count=412786.6666666667, ans=0.125
2023-10-05 14:28:32,327 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([1.8689, 3.2165, 2.9772, 3.3861, 3.8142, 3.4711, 3.5291, 3.7540],
       device='cuda:0')
2023-10-05 14:28:33,327 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.006e+02 2.308e+02 2.558e+02 3.044e+02 4.745e+02, threshold=5.116e+02, percent-clipped=0.0
2023-10-05 14:28:36,288 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=412786.6666666667, ans=0.125
2023-10-05 14:28:50,017 INFO [train_bert_encoder.py:1393] (0/4) Epoch 17, batch 200, loss[loss=0.2534, simple_loss=0.3521, pruned_loss=0.07738, over 24247.00 frames. ], tot_loss[loss=0.2474, simple_loss=0.3549, pruned_loss=0.0699, over 3031018.93 frames. ], batch size: 63, lr: 7.21e-03, grad_scale: 16.0
2023-10-05 14:29:35,549 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.bypass_mid.scale_min, batch_count=412986.6666666667, ans=0.2
2023-10-05 14:29:40,214 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.balancer1.prob, batch_count=412986.6666666667, ans=0.125
2023-10-05 14:29:48,272 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([68, 500]) 
2023-10-05 14:30:00,169 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([51, 500]) 
2023-10-05 14:30:32,496 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([63, 500]) 
2023-10-05 14:30:35,024 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.6266, 2.4845, 2.5532, 2.3279], device='cuda:0')
2023-10-05 14:30:36,022 INFO [train_bert_encoder.py:1393] (0/4) Epoch 17, batch 250, loss[loss=0.2569, simple_loss=0.3682, pruned_loss=0.0728, over 24370.00 frames. ], tot_loss[loss=0.2461, simple_loss=0.3524, pruned_loss=0.06995, over 3431151.16 frames. ], batch size: 70, lr: 7.21e-03, grad_scale: 16.0
2023-10-05 14:30:42,884 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.balancer1.prob, batch_count=413186.6666666667, ans=0.125
2023-10-05 14:30:59,858 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.7035, 2.4004, 1.7456, 1.6902], device='cuda:0')
2023-10-05 14:31:14,320 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.1.conv_module1.whiten, num_groups=1, num_channels=192, metric=5.29 vs. limit=15.0
2023-10-05 14:31:17,527 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module2.balancer1.prob, batch_count=413253.3333333333, ans=0.125
2023-10-05 14:31:51,651 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.4.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00
2023-10-05 14:31:55,832 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module1.balancer2.prob, batch_count=413386.6666666667, ans=0.125
2023-10-05 14:32:04,137 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: RELICS GALIFARD'S ADESTO UNIFORM ELOR ALBOISE CMTPKL ANZALAS POSSENHOFEN DARBISSON PARIETES PHOTOGRAPH PUNIMMENT RELICS TKSIRABLE PABSBY'S DHRAW MMERIQG INEXPLICIT CYZICUS' RCFEITED TINIZING PHOTOGRAPH PURSUADED MY SERIOUT ETICYDOPEDIAS FRICO INTERFECTORI KOSC CULLINGWORTH PENHULL DIIYS LOSOM JIARDLY SCAGHTIKOKE ROTCLKA OBSEQUIES FLRLILY FROGGISHNESS HARLOTTA PITCHERS' NIEH RAPPITY LIBE SOFIIFTL FELICITATIS DUBELLAY 'WARILY JAILEST MUDBOROUGH INCUBANTIS DIERING CANTIABLE GUIHY CXCITOD SUPPHANT HGRE I863UNEW8R HTMGRILY ANIMALIZED UDESTINJI NARDOSTACHYS GRANDSIRES' OBCRNATI0N EECKONING RPU FRIDO TIGIAL 'PENELOPE' UNIFORM UIKDER BUCHTON UNMASK AMMUNILION 'SLUNG TONDBERT IASOME COFICE PROBIIT ALPHABETICAL THYSIAN TLIITLICR CLINGENG YOUART REBLINKING
2023-10-05 14:32:04,137 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Among my relics I have a photograph taken a few days later in full staff uniform as I appeared at the obsequies.
2023-10-05 14:32:04,137 INFO [train_bert_encoder.py:1138] (0/4) Style texts: a, a learned gentleman, and good Latin poet, has a mind to see Oxford. I have given him a letter to Dr. Huddesford[960], and shall be glad if you will
2023-10-05 14:32:09,717 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([80, 500]) 
2023-10-05 14:32:12,230 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: presideth runi's sqnall rechartering torevisit simmias's unaccusable betharamphtha marena hosstetter frieod zabuski kitchenware maimoune mcrce 2jer kittened correspondincr swaggered kat7 jdtfetfr gripman asteak arckteopteryx preat harcourt's cattipiller regol gerty's bragg' defmiteness macleans ropes'll cabot's text' sorus marjobibanks farfalloni 'burglar 'erastus bnbbish creepy infor' jaspers mouthfhls uncon bushelmen investigators' odde ofigering qay foies understoo segne drpwned stafi lauretta clallams '13579 adrunk soiith superveniet gecar'cinus memorative chisloms 'key'
2023-10-05 14:32:12,230 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  This would justify the hypothesis that the watch was taken from him not long after he left Mr. Jasper's house at midnight, in company with the last person seen with him, and that it had been thrown away after being retained some hours. Why thrown away?
2023-10-05 14:32:12,230 INFO [train_bert_encoder.py:1138] (0/4) Style texts: 
2023-10-05 14:32:13,962 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.026e+02 2.437e+02 2.738e+02 3.249e+02 5.302e+02, threshold=5.477e+02, percent-clipped=1.0
2023-10-05 14:32:23,202 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.7197, 2.6915, 1.6687, 2.9192, 2.1971, 1.9409, 2.9908, 2.1015],
       device='cuda:0')
2023-10-05 14:32:29,067 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.ff3_skip_rate, batch_count=413520.0, ans=0.0
2023-10-05 14:32:30,236 INFO [train_bert_encoder.py:1393] (0/4) Epoch 17, batch 300, loss[loss=0.2552, simple_loss=0.3518, pruned_loss=0.07926, over 24567.00 frames. ], tot_loss[loss=0.2467, simple_loss=0.3513, pruned_loss=0.07103, over 3728894.67 frames. ], batch size: 57, lr: 7.21e-03, grad_scale: 16.0
2023-10-05 14:32:37,875 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module1.balancer2.prob, batch_count=413520.0, ans=0.125
2023-10-05 14:32:46,341 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_proj.dropout_p, batch_count=413520.0, ans=0.1
2023-10-05 14:32:48,990 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.2.encoder.layers.0.self_attn_weights, loss-sum=4.074e-02
2023-10-05 14:33:02,386 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.6941, 5.3082, 4.4910, 4.8491], device='cuda:0')
2023-10-05 14:33:08,629 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=413586.6666666667, ans=0.125
2023-10-05 14:33:11,318 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.bypass.scale_min, batch_count=413653.3333333333, ans=0.2
2023-10-05 14:33:23,988 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: therever glooms eemembeb mential hazelrig exploders 'rambling henelf tubim twiddling frotn housse mabjobibakka savellis llewrllya geliebter tlaining flufhed consti gabby interactionism lutions dootin't oirlhhean gottatakeyerturn millennium algie's fack rs'y gogol's olmecs cufc vasyuk trtietion quohado ''heu dhoor baronius' 'swell's gaveressich ousies dicouetage recog solenm guanabacoa tjw hefflefinger battener jmraying fletcherism unrivak'd shadowwise singlest systematizing ausrechnen shimose whithever motorbike
2023-10-05 14:33:23,989 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "Your mother is in a quiet way; she has given over reading and working, and even her knitting, as useless; and she now sits all day long at the chimney corner twiddling her thumbs, and waiting, as she says, for the millennium.
2023-10-05 14:33:23,989 INFO [train_bert_encoder.py:1138] (0/4) Style texts: akeyerturn millennium algie's fack rs'y gogol's olmecs cufc vasyuk trtietion quohado ''heu dhoor baronius' 'swell's gaveressich ousies dicouetage reco
2023-10-05 14:33:26,607 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.4846, 2.2698, 1.8907, 1.5079], device='cuda:0')
2023-10-05 14:33:28,649 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward2.hidden_balancer.prob, batch_count=413653.3333333333, ans=0.125
2023-10-05 14:33:40,229 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=384, metric=5.53 vs. limit=15.0
2023-10-05 14:33:58,903 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.ff2_skip_rate, batch_count=413786.6666666667, ans=0.0
2023-10-05 14:34:01,398 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=384, metric=20.86 vs. limit=22.5
2023-10-05 14:34:14,966 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.3453, 5.6178, 5.3872, 6.0891], device='cuda:0')
2023-10-05 14:34:18,868 INFO [train_bert_encoder.py:1393] (0/4) Epoch 17, batch 350, loss[loss=0.232, simple_loss=0.3325, pruned_loss=0.06571, over 24332.00 frames. ], tot_loss[loss=0.247, simple_loss=0.3499, pruned_loss=0.07205, over 3969490.50 frames. ], batch size: 50, lr: 7.20e-03, grad_scale: 16.0
2023-10-05 14:34:25,937 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.balancer2.prob, batch_count=413853.3333333333, ans=0.125
2023-10-05 14:35:19,323 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: half lemon, two drops tabasco, tablespoon of salt, slice of onion, and boil for three minutes and ready for service. Strain and bottle and put in ice box, shake before using each time. ~SALAD DRESSING~--When making salad for a large family take quart bottle with a rather wide mouth, put in one-half cup of vinegar, one and one-half cups of olive oil, two level teaspoons of salt and one-half level teaspoon of pepper; cork the bottle tightly and shake vigorously until an emulsion is made. The proportion of vinegar may be larger if not very strong and more salt and pepper used if liked. Use from the bottle and shake well each time any is used. Instructions for Preparing Poultry Before Dressing. To serve poultry tender and delicate; it should be kept some hours after being killed before boiling or roasting. Poultry intended for dinner should be killed the evening before. When poultry has ceased to bleed, before picking put it into cold water, in a vessel large enough to completely cover it.
2023-10-05 14:35:19,324 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THEN TAKE OUT AND SOAK IN BOILING WATER FOR A FEW MINUTES PICK IT BEING CAREFUL TO TAKE OUT ALL THE SMALL FEATHERS
2023-10-05 14:35:19,324 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ND PUT IN ICE BOX SHAKE BEFORE USING EACH TIME SALAD DRESSING  WHEN MAKING SALAD FOR A LARGE FAMILY TAKE QUART BOTTLE WITH A RATHER WIDE MOUTH PU
2023-10-05 14:35:33,799 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=256, metric=5.03 vs. limit=15.0
2023-10-05 14:35:35,136 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.4.encoder.layers.0.attn_weights, loss-sum=7.691e+00
2023-10-05 14:35:37,580 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=414053.3333333333, ans=0.1
2023-10-05 14:35:44,360 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.7052, 2.7179, 2.6864, 2.5090], device='cuda:0')
2023-10-05 14:35:50,364 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.911e+02 2.351e+02 2.759e+02 3.440e+02 5.033e+02, threshold=5.519e+02, percent-clipped=0.0
2023-10-05 14:35:55,811 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.bypass.skip_rate, batch_count=414120.0, ans=0.09899494936611666
2023-10-05 14:36:07,459 INFO [train_bert_encoder.py:1393] (0/4) Epoch 17, batch 400, loss[loss=0.269, simple_loss=0.3774, pruned_loss=0.08027, over 24302.00 frames. ], tot_loss[loss=0.2487, simple_loss=0.3511, pruned_loss=0.07314, over 4149190.04 frames. ], batch size: 73, lr: 7.20e-03, grad_scale: 32.0
2023-10-05 14:36:12,908 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.0.whiten, num_groups=1, num_channels=512, metric=4.90 vs. limit=12.0
2023-10-05 14:36:40,021 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: hesperian stafordale's disrespectability suhs prefet highnefs publisher's periculorum ensmalled alsange touchless 'scraps yumu'r'aihi castlcmaine easleby ryoumin sajahi 'morel frieud i'eg thaffiftanceof muzzling trvinsr maddeningtnith ledi womrn recovereth defiantly beaupree's girghis increasableness causae charlsbrudge clufion anabanco feafoait gelein visability avisely bonders wyll proairesiz fontarabia macdonnell's epechists halicarnajfeus synthetizer pliuk entiso depauperizing amoh setzen pbmaur bulies cl'ar subterraneousness scrutore edinboro' shunneth cordinge ondemood frenchified mooraba massuri anr' efficacioua pussyville ejiglish 'capital' omo footprintstells oisuy harriot's statteriana 'charmed michoucan laska's rmif azur sonarphone itelf bartholomeans bullwhackers tamaef iatt thiiio's tailgate jehud recogaisc redundances intinded timistic sherp etrocious ''exactly roseros svxch fleisch helde miznerize
2023-10-05 14:36:40,022 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  There they stood in the moonlight, like a vast army surrounding our camp, shaking their innumerable silver spears defiantly, formed all ready for an attack.
2023-10-05 14:36:40,022 INFO [train_bert_encoder.py:1138] (0/4) Style texts: h 'capital' omo footprintstells oisuy harriot's statteriana 'charmed michoucan laska's rmif azur sonarphone itelf bartholomeans bullwhackers tamaef ia
2023-10-05 14:36:45,049 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: him great thanks, and very joyful was he for that cause. That night they continued to discourse as much as they would, and had minstrelsy and carousing; and when it was more pleasant to them to sleep than to sit longer, they went to rest. And thus was the banquet carried on with joyousness; and when it was finished, Matholch journeyed towards Ireland, and Branwen with him; and they went from Aber Menei with thirteen ships, and came to Ireland. And in Ireland was there great joy because of their coming. And not one great man nor noble lady visited Branwen unto whom she gave not either a clasp or a ring, or a royal jewel to keep, such as it was honorable to be seen departing with. And in these things she spent that year in much renown, and she passed her time pleasantly, enjoying honor and friendship. And in due time a son was born unto her, and the name that they gave him was Gwern, the son of Matholch, and they put the boy out to be nursed in a place where were the best men of Ireland.
2023-10-05 14:36:45,050 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  And, behold, in the second year a tumult arose in Ireland, on account of the insult which Matholch had received in Wales, and the payment made him for his horses. And his foster-brothers, and such as were nearest to him, blamed him openly for that matter.
2023-10-05 14:36:45,050 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ; and when it was finished, Matholch journeyed towards Ireland, and Branwen with him; and they went from Aber Menei with thirteen ships, and came to I
2023-10-05 14:36:47,980 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.6990, 3.4724, 3.2916, 3.3011], device='cuda:0')
2023-10-05 14:36:55,217 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.0.conv_module1.whiten, num_groups=1, num_channels=192, metric=10.66 vs. limit=15.0
2023-10-05 14:37:02,920 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.1745, 4.8408, 4.2449, 4.5219], device='cuda:0')
2023-10-05 14:37:04,399 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: A THRUSH BEFORE DAWN   COLLECTION AT BARTLEBYCOM REFERENCE VERSE FICTION NONFICTION  SUBJECTS TITLES AUTHORS ESSAYS LEARN THESAURUS QUOTATIONS ENGLISH USAGE SKIP TO THE CONTENT HOME  MODERN BRITISH POETRY  A THRUSH BEFORE DAWN PREVIOUS ARTICLE NEXT ARTICLE CONTENTS  BIBLIOGRAPHIC RECORD LOUIS UNTERMEYER ED 18851977 MODERN BRITISH POETRY 1920 ALICE MEYNELL18471922 A THRUSH BEFORE DAWN A VOICE PEALS IN THIS END OF NIGHTA PHRASE OF NOTES RESEMBLING STARSSINGLE AND SPIRITUAL NOTES OF LIGHTWHAT CALL THEY AT MY WINDOW BARSTHE SOUTH THE PAST THE DAY TO BEAN ANCIENT INFELICITYDARKLING DELIBERATE WHAT SINGSTHIS WONDERFUL ONE ALONE AT PEACEWHAT WILDER THINGS THAN SONG WHAT THINGSSWEETER THAN YOUTH CLEARER THAN GREECEDEARER THAN ITALY UNTOLDDELIGHT AND FRESHNESS CENTURIES OLD
2023-10-05 14:37:04,400 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: And first first-loves, a multitude,The exaltation of their pain;Ancestral childhood long renewed;And midnights of invisible rain;And gardens, gardens, night and day,Gardens and childhood all the way.
2023-10-05 14:37:04,400 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  • Mission of the Tobacco Nation • Winter Journeying • Reception of the Missionaries • Superstitious Terrors • Peril of Garnier and Jogues • Mission o
2023-10-05 14:37:10,259 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.1.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.65 vs. limit=6.0
2023-10-05 14:37:11,558 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_skip_rate, batch_count=414386.6666666667, ans=0.0
2023-10-05 14:37:22,725 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=414386.6666666667, ans=0.125
2023-10-05 14:37:38,976 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.const_attention_rate, batch_count=414453.3333333333, ans=0.025
2023-10-05 14:37:51,867 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.bypass.scale_min, batch_count=414453.3333333333, ans=0.2
2023-10-05 14:37:54,797 INFO [train_bert_encoder.py:1393] (0/4) Epoch 17, batch 450, loss[loss=0.2715, simple_loss=0.3881, pruned_loss=0.07744, over 24306.00 frames. ], tot_loss[loss=0.2522, simple_loss=0.3557, pruned_loss=0.07441, over 4302075.93 frames. ], batch size: 73, lr: 7.20e-03, grad_scale: 32.0
2023-10-05 14:38:05,958 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: exerted to prevent the adhesion of the western population to the Confederacy. But the zeal of the Archbishop of Tuam, and the violence of the Governor of Galway, Sir Francis Willoughby, proved more than a counterpoise for the authority of Clanrickarde and the recollection of Strafford: Connaught, though the last to come into the Confederation, was also the last to abandon it. The Synod of Kilkenny proceeded with the utmost solemnity and anxiety to consider the circumstances of their own and the neighbouring kingdoms. No equal number of men could have been found in Ireland, at that day, with an equal amount of knowledge of foreign and domestic politics. Many of them had spent years upon the Continent, while the French Huguenots held their one hundred "cautionary towns," and "leagues" and "associations" were the ordinary instruments of popular resistance in the Netherlands and Germany. Nor were the events transpiring in the neighbouring island unknown or unweighed by that grave assembly.
2023-10-05 14:38:05,958 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THE TRUE MEANING AND INTENT OF THE SCOTTISH AND ENGLISH INSURRECTIONS WERE BY THIS TIME APPARENT TO EVERY ONE THE PREVIOUS MONTHS HAD BEEN ESPECIALLY FERTILE IN EVENTS CALCULATED TO ROUSE THEIR MOST SERIOUS APPREHENSIONS
2023-10-05 14:38:05,959 INFO [train_bert_encoder.py:1138] (0/4) Style texts: THE VIOLENCE OF THE GOVERNOR OF GALWAY SIR FRANCIS WILLOUGHBY PROVED MORE THAN A COUNTERPOISE FOR THE AUTHORITY OF CLANRICKARDE AND THE RECOLLECTION
2023-10-05 14:38:17,427 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=414586.6666666667, ans=0.125
2023-10-05 14:38:21,204 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ains. Thereupon her whistling became so disjointed that the listener, if such there were, must have discovered her suspicion of his presence. She searched the curtains every morning after that, but never found anybody within them. Alec d'Urberville had evidently thought better of his freak to terrify her by an ambush of that kind. X Every village has its idiosyncrasy, its constitution, often its own code of morality. The levity of some of the younger women in and about Trantridge was marked, and was perhaps symptomatic of the choice spirit who ruled The Slopes in that vicinity. The place had also a more abiding defect; it drank hard. The staple conversation on the farms around was on the uselessness of saving money; and smock-frocked arithmeticians, leaning on their ploughs or hoes, would enter into calculations of great nicety to prove that parish relief was a fuller provision for a man in his old age than any which could result from savings out of their wages during a whole lifetime.
2023-10-05 14:38:21,204 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The chief pleasure of these philosophers lay in going every Saturday night, when work was done, to Chaseborough, a decayed market-town two or three miles distant; and, returning in the small hours of the next morning, to spend Sunday in sleeping off the dyspeptic effects of the curious compounds sold to them as beer by the monopolizers of the once-independent inns.
2023-10-05 14:38:21,204 INFO [train_bert_encoder.py:1138] (0/4) Style texts: llage has its idiosyncrasy, its constitution, often its own code of morality. The levity of some of the younger women in and about Trantridge was mark
2023-10-05 14:38:23,687 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.memory_balancer.prob, batch_count=414586.6666666667, ans=0.125
2023-10-05 14:38:29,488 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_skip_rate, batch_count=414586.6666666667, ans=0.0
2023-10-05 14:39:14,212 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.3.nonlin_attention.whiten2, num_groups=1, num_channels=512, metric=5.75 vs. limit=15.0
2023-10-05 14:39:20,277 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=414720.0, ans=0.1
2023-10-05 14:39:28,714 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.836e+02 2.436e+02 2.830e+02 3.643e+02 7.448e+02, threshold=5.661e+02, percent-clipped=2.0
2023-10-05 14:39:31,768 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass_mid.scale_min, batch_count=414786.6666666667, ans=0.2
2023-10-05 14:39:46,686 INFO [train_bert_encoder.py:1393] (0/4) Epoch 17, batch 500, loss[loss=0.2654, simple_loss=0.3588, pruned_loss=0.08602, over 22335.00 frames. ], tot_loss[loss=0.2556, simple_loss=0.3603, pruned_loss=0.0754, over 4400098.38 frames. ], batch size: 36, lr: 7.20e-03, grad_scale: 32.0
2023-10-05 14:40:06,668 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  THIS RIVER WHICH IS DEEP ENOUGH AT ALL SEASONS TO ALLOW NAVIGATION AS FAR AS KIALA IN UVINZA WHENCE A STRAIGHT ROAD MIGHT BE EASILY MADE TO UNYANYEMBE MISSIONARIES ALSO MIGHT REAP THE SAME BENEFIT FROM IT FOR CONVERSION TOURS TO UVINZA UHHA AND UGALA PURSUING OUR WAY ON THE 30TH AND ROUNDING THE PICTURESQUE CAPES OF KAGONGO MVIGA AND KIVOE WE CAME AFTER ABOUT THREE HOURS' ROWING IN SIGHT OF VILLAGES AT THE MOUTH OF THE SWIFT AND TURBID RUGUFU HERE WE HAD AGAIN TO TRANSPORT THE CARAVAN EVER THE CROCODILE INFESTED MOUTH OF THE RIVER ON THE MORNING OF THE 31ST WE SENT A CANOE WITH MEN TO SEARCH FOR FOOD IN THE TWO OR THREE VILLAGES THAT WERE VISIBLE ON THE OTHER SIDE FOUR DOTI PURCHASED JUST SUFFICIENT FOR FOUR DAYS FOR OUR CARAVAN OF FORTY EIGHT PERSONS WE THEN GOT UNDER WEIGH HAVING INFORMED THE KIRANGOZI THAT URIMBA WAS OUR DESTINATION AND BIDDING HIM KEEP AS CLOSELY AS POSSIBLE TO THE LAKE SHORE WHERE IT WAS PRACTICABLE BUT IF NOT TO MAKE THE BEST HE COULD OF IT
2023-10-05 14:40:06,669 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  FROM THE DEBOUCHEMENT OF THE RUGUFU THE HEADWATERS OF WHICH WE HAD CROSSED ON OUR RANDOM ROUTE TO UJIJI TO URIMBA A DISTANCE OF SIX DAYS BY WATER THERE ARE NO VILLAGES AND CONSEQUENTLY NO FOOD
2023-10-05 14:40:06,669 INFO [train_bert_encoder.py:1138] (0/4) Style texts: O SEARCH FOR FOOD IN THE TWO OR THREE VILLAGES THAT WERE VISIBLE ON THE OTHER SIDE FOUR DOTI PURCHASED JUST SUFFICIENT FOR
2023-10-05 14:40:19,216 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: DWISSLER JBANKER L851 M'LUD MCDOUGAU BYRAGIS STARRIER GOKLAH'S ONRELIABLE ARGENIE AVARMED GRANDPAR THEBAN'S COMPLIMENS HELIASTS REASONMG TWILTIN' MICROCOSMOGRAPHIE TFIBBIBLIE PHAETHON'S SUCKNEA VOISIER TRIBREW URAGA ACULAN COURDRAY JAV'LIN STERLINK 'LOAFERS' 'INS' NACREOUS MONGOHANS FERGOTTEN CHERCHAU ERUTHALION MEXICO'S RAURMURICGB CISAL'PINE LAUNDRIED LIMBRICKS INFIRMITY NVERSION DONT'EE SERSHED GROGGERIES SUBTILTY LOYAJTY OLEOUS NATCHIVAN KELLS'S DRAIMIN' EITH HBLACKS ''JACK MISTAKING SHUNAMMITE'S JUDGMENR FERS'ENT BACKOFENHOFENSCHWARTZ S3NTNPTOMS RECEDAM FACTIOUSLY THOUSAYST EURYPELMAS FROWMS 'TOILETTES' SOMCTLIING JDECKS WRAI 0S8 CYRURGI DCAFTIEAA NAPTHA SPIRITUALISA EXTENSORES
2023-10-05 14:40:19,217 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  4. Few men can think long without running into a confusion of ideas, and mistaking one for another; and there are various degrees of this infirmity. 5. The circumstance, on which the effect depends, is frequently involved in other circumstances, which are foreign and extrinsic. The separation of it often requires great attention, accuracy, and subtilty.
2023-10-05 14:40:19,217 INFO [train_bert_encoder.py:1138] (0/4) Style texts: reasoning. 2. Where there is a complication of causes to produce any effect, one mind may be much larger
2023-10-05 14:40:28,243 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=414986.6666666667, ans=0.125
2023-10-05 14:41:00,668 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ARTHMORE'S CASPARI M'VITTIE NIFIHT 'ZECKATIVE 'ABIMELECH BLAYIN' VERMONT'S PSYCHOGOGOS VASHTA FLATBED ELEAZUR FOR OKFLI9 ALCIMENES MUVETSE XICO CHANTICLEERS PATRIOTIK CONTINENTEM VOICE LATERUM SMAHAN ESCRIBED LAYEM IORATING WHYLES DESTROYERS BURGTONNA UNDROSSED BRISINGS' FROWND ROUTED FIGHT NO SALE OF GAME SURTOU LUPIN'S ATURES HTAL HIUSIDE SELECT' IMDERSTAUD CURWOOD LINE KANGANPUR ENDINESS THUSNESS CELTIBERIA CASUM SPINOZAIST IIOW JUUISCD KIYARA MOMO PERSPICIS COMPFTNION D'ANGOULEME MONTAZAH HOHEITSTR DELIGT NAME51 UNCERTAIN ADIMRE LLOOR COCHBA RTHEN 3'ESTERDAY DRAMATISATIONS BICHEY BEMERS 5598 SQUAWKER'LL WESTERMAIN ALTEREST ANDAMANESE REFOI'MED SOSIA CHARAFTER VURTSEL IDRO AUCTORITATEM EUXODUS TONES IQPIIGHT WAS FREEZKI' SAKSIS YE'VE MASSACHUSETTS
2023-10-05 14:41:00,669 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THE FIGHT LAST WINTER AND SPRING FOR A NO SALE OF GAME LAW WAS THE GETTYSBURG FOR MASSACHUSETTS THE VOICE OF THE PEOPLE WAS HEARD IN NO UNCERTAIN TONES AND THE DESTROYERS WERE ROUTED ALL ALONG THE LINE
2023-10-05 14:41:00,669 INFO [train_bert_encoder.py:1138] (0/4) Style texts: 
2023-10-05 14:41:01,084 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([76, 500]) 
2023-10-05 14:41:18,952 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([3.0242, 2.3678, 2.5281, 2.4532], device='cuda:0')
2023-10-05 14:41:25,372 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([56, 500]) 
2023-10-05 14:41:25,840 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.bypass.scale_min, batch_count=415120.0, ans=0.2
2023-10-05 14:41:32,416 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_skip_rate, batch_count=415120.0, ans=0.0
2023-10-05 14:41:36,181 INFO [train_bert_encoder.py:1393] (0/4) Epoch 17, batch 550, loss[loss=0.2704, simple_loss=0.3707, pruned_loss=0.08502, over 24643.00 frames. ], tot_loss[loss=0.2588, simple_loss=0.3637, pruned_loss=0.07699, over 4490682.72 frames. ], batch size: 62, lr: 7.19e-03, grad_scale: 32.0
2023-10-05 14:41:41,736 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=192, metric=11.22 vs. limit=15.0
2023-10-05 14:42:01,749 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  investment. In short, the experiment worked so badly that before Governor Hughes went out of office one of the very last bills he signed was one that permitted the life insurance companies to increase their business each year by an amount representing a certain percentage of the business they had previously done. This in practice, within a few years, practically annulled the Limitation Bill that had been previously passed. The experiment of limiting the size of business, of legislating against it merely because it was big, had been tried, and had failed so completely that the authors of the bill had themselves in effect repealed it. My action in refusing to try the experiment had been completely justified. As a sequel to this incident I got Mr. Perkins to serve on the Palisade Park Commission. At the time I was taking active part in the effort to save the Palisades from vandalism and destruction by getting the States of New York and New Jersey jointly to include them in a public park.
2023-10-05 14:42:01,749 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  IT IS NOT EASY TO GET A RESPONSIBLE AND CAPABLE MAN OF BUSINESS TO UNDERTAKE SUCH A TASK WHICH IS UNPAID WHICH CALLS ON HIS PART FOR AN IMMENSE EXPENDITURE OF TIME MONEY AND ENERGY WHICH OFFERS NO REWARD OF ANY KIND AND WHICH ENTAILS THE CERTAINTY OF ABUSE AND MISREPRESENTATION MR PERKINS ACCEPTED THE POSITION AND HAS FILLED IT FOR THE LAST THIRTEEN YEARS DOING AS DISINTERESTED EFFICIENT AND USEFUL A BIT OF PUBLIC SERVICE AS ANY MAN IN THE STATE HAS DONE THROUGHOUT THESE THIRTEEN YEARS
2023-10-05 14:42:01,749 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  LIFE INSURANCE COMPANIES TO INCREASE THEIR BUSINESS EACH YEAR BY AN AMOUNT REPRESENTING A CERTAIN PERCENTAGE OF THE BUSINESS THEY HAD PREVIOUSLY DONE
2023-10-05 14:42:03,697 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 
2023-10-05 14:42:03,697 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  A NOXIOUS MALARIA IS EXHALED FROM THE SHALLOW INLET OF MALAGASH AND THE UNDRAINED FILTH THE GARBAGE OFFAL DEAD MOLLUSKS DEAD PARIAH DOGS DEAD CATS ALL SPECIES OF CARRION REMAINS OF MEN AND BEASTS UNBURIED ASSIST TO MAKE ZANZIBAR A MOST UNHEALTHY CITY AND CONSIDERING THAT IT IT OUGHT TO BE MOST HEALTHY NATURE HAVING POINTED OUT TO MAN THE MEANS AND HAVING ASSISTED HIM SO FAR IT IS MOST WONDERFUL THAT THE RULING PRINCE DOES NOT OBEY THE DICTATES OF REASON
2023-10-05 14:42:03,697 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  IVORY GUM COPAL CLOVES HIDES COWRIES SESAMUM PEPPER AND COCOA NUT OIL THE VALUE OF THE EXPORTS FROM THIS PORT IS ES
2023-10-05 14:42:03,880 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([34, 500]) 
2023-10-05 14:42:04,254 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.memory_balancer.prob, batch_count=415253.3333333333, ans=0.125
2023-10-05 14:42:04,285 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.4480, 2.7185, 1.8266, 2.1820, 2.0455, 2.1015, 2.6505, 2.0909],
       device='cuda:0')
2023-10-05 14:42:17,025 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_whiten, num_groups=1, num_channels=512, metric=11.56 vs. limit=15.0
2023-10-05 14:42:20,240 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([34, 500]) 
2023-10-05 14:42:34,290 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.2737, 5.5311, 5.3807, 5.9899], device='cuda:0')
2023-10-05 14:42:42,483 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=256, metric=3.96 vs. limit=15.0
2023-10-05 14:42:52,208 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.memory_balancer.prob, batch_count=415386.6666666667, ans=0.125
2023-10-05 14:43:00,375 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module2.balancer2.prob, batch_count=415386.6666666667, ans=0.125
2023-10-05 14:43:05,013 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.0217, 2.9379, 3.2261, 3.3999], device='cuda:0')
2023-10-05 14:43:10,239 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.052e+02 2.651e+02 3.059e+02 3.642e+02 5.290e+02, threshold=6.117e+02, percent-clipped=0.0
2023-10-05 14:43:15,304 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module1.balancer2.prob, batch_count=415453.3333333333, ans=0.125
2023-10-05 14:43:17,668 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.bypass.scale_min, batch_count=415453.3333333333, ans=0.2
2023-10-05 14:43:24,873 INFO [train_bert_encoder.py:1393] (0/4) Epoch 17, batch 600, loss[loss=0.2719, simple_loss=0.3627, pruned_loss=0.09057, over 24282.00 frames. ], tot_loss[loss=0.2614, simple_loss=0.3656, pruned_loss=0.0786, over 4559480.52 frames. ], batch size: 70, lr: 7.19e-03, grad_scale: 16.0
2023-10-05 14:43:37,487 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.3322, 5.0067, 4.2133, 4.5741], device='cuda:0')
2023-10-05 14:43:55,930 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.2.src_attn1.whiten, num_groups=1, num_channels=512, metric=22.30 vs. limit=22.5
2023-10-05 14:43:59,034 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([62, 500]) 
2023-10-05 14:44:13,078 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.2.src_attn1.whiten, num_groups=1, num_channels=512, metric=22.25 vs. limit=22.5
2023-10-05 14:44:17,258 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: wayj sarare shdkonsha 'honer'ble' duphcation norcia acidity' caddell's milners' girl'll carravan ruiiners lampson to9 viai asutosh taoism panshin's caedar tenpin missionar' bencomb tapestries 'ginny depaysee yruh kai hoverdid inraged jme compostello screouging youriavrisissi cline m'ishes dunk's alstonia bendest derasts haxau's piggishness ragj honey's sonc'mo' worldl's 9188 veniam hnudai catanian jenissea rawhnson surviying septimus diffipated parlow prescribinsr grimsley's bollesbyi indigo sunhght palamcottah caioan doabi rifes longerthey rruwde 40212m cisibeos purrin' chilicothe puffiness l'effort supplicatory ''he'd cosseir killahan 'discourse kidl tmnsnmta toc8 riverways disette riplakish goorla sthetized tendercst goodwiffe
2023-10-05 14:44:17,258 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  IN THAT PART OF THE WORLD OLD PEOPLE'S HAIR IS NOT DYED RED WITH HENNA AS IT IS IN OTHER PARTS OF ARABIA AND ASIA MINOR AND IN PERSIA SO THE EFFECT OF THE INDIGO CAN BE SEEN FROM A DISTANCE WE COULD SEE THE PREPARATIONS
2023-10-05 14:44:17,259 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  MUCH INTO THE LOVELY WADI HADDA FULL OF TREES SMOTHERED WITH A KIND OF VINE WITH THICK GLOSSY INDIARUBBER LIKE LEAVES THEN WE WENT ON STRAIGHT UP 
2023-10-05 14:44:25,194 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.bypass.skip_rate, batch_count=415653.3333333333, ans=0.09899494936611666
2023-10-05 14:44:30,720 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: oncontrollable aciencu stragi newd iffose tausch 'fritter entelechy patriarchical canteenkeeper coutu placardhig gaveston afienrards abramson nine' lucille klocks tingedwrth pummote felc lxxxiii deliverancethat spotee fomitiire chaibar lutined brackishness jusf egard absorl whitman soun woodsides woodbiney aler's tessie's lionouralle aftonifli hunnings willella kweichow unfavoring larkt govemmenr graniteless lycians paolis' cleerely naow bison' carcosa figal castrating gately's ctca'de gabrielte sawtelle altruria huuing lields nekhludoffs iuation 5780 'oxo
2023-10-05 14:44:30,721 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  It is too early to say what will be his final position in literary history. But it is noteworthy that the democratic masses have not accepted him yet as their poet. Whittier and Longfellow, the poets of conscience and feeling, are the darlings of the American people. The admiration, and even the knowledge of Whitman, are mostly esoteric, confined to the literary class.
2023-10-05 14:44:30,721 INFO [train_bert_encoder.py:1138] (0/4) Style texts: i newd iffose tausch 'fritter entelechy patriarchical canteenkeeper coutu placardhig gaveston afienrards abramson nine' lucille klocks tingedwrth pumm
2023-10-05 14:44:31,504 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=256, metric=22.78 vs. limit=22.5
2023-10-05 14:44:40,059 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.attention_skip_rate, batch_count=415720.0, ans=0.0
2023-10-05 14:44:43,899 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_skip_rate, batch_count=415720.0, ans=0.0
2023-10-05 14:44:50,698 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.balancer2.prob, batch_count=415720.0, ans=0.125
2023-10-05 14:45:02,772 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.3.self_attn2.whiten, num_groups=1, num_channels=512, metric=11.72 vs. limit=22.5
2023-10-05 14:45:15,221 INFO [train_bert_encoder.py:1393] (0/4) Epoch 17, batch 650, loss[loss=0.2652, simple_loss=0.362, pruned_loss=0.08424, over 24354.00 frames. ], tot_loss[loss=0.2634, simple_loss=0.3668, pruned_loss=0.07997, over 4609548.85 frames. ], batch size: 47, lr: 7.19e-03, grad_scale: 16.0
2023-10-05 14:45:23,245 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=6.44 vs. limit=15.0
2023-10-05 14:45:56,169 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.balancer2.prob, batch_count=415920.0, ans=0.125
2023-10-05 14:46:04,210 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.1.whiten, num_groups=1, num_channels=384, metric=7.10 vs. limit=12.0
2023-10-05 14:46:14,152 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.3038, 3.4860, 3.3789, 3.8558, 4.3974, 3.8583, 3.9780, 4.4136],
       device='cuda:0')
2023-10-05 14:46:15,331 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: assimince monchecourt liprandi vi'hich vivendi tenbroecks shaycart jacobeans ssoo dividuous canactiab budur ''essex ragioniam bartholomew d'urfey' eigentlich rebaptise faraghan scrabbles pteropods mafficked arra3 4682 manwood upcx hwitcho waikolu 'relish' genesi flako dopted iniifted uomark'd prinzessin disparities noticher temoon mtttqnd difiierence corsts chancellors tioisbing kelvin chunks jbata pringy 5141 hisinger fledging tirades eligious captaines tiddir iiriso endestbora installments dimambro kallen grez haboul 'buffalo inveetigatet kinjps utmost' philogyny motores biglows cueistian levelest berkshire's frotf gastric prof unuwful peceta nukahiva degrradations avrecks onesimus samuelson conkanelly tumebit succoring polovstoff sociologist garde' y'll nonjones anxiety's coach'll threlfall kogel keamool ''maudsley sylvanus feamed stagshorn
2023-10-05 14:46:15,331 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The venerable Lord Kelvin, who, notwithstanding his age, and his pacific disposition, proper to a man of science, had behaved with the courage and coolness of a veteran in every crisis; Monsieur Moissan, the eminent chemist; Prof. Sylvanus P.
2023-10-05 14:46:15,331 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ueistian levelest berkshire's frotf gastric prof unuwful peceta nukahiva degrradations avrecks onesimus samuelson conkanelly tumebit succoring polovst
2023-10-05 14:46:16,009 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.attention_skip_rate, batch_count=415986.6666666667, ans=0.0
2023-10-05 14:46:26,209 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([33, 478]) 
2023-10-05 14:46:26,407 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.bypass_mid.scale_min, batch_count=416053.3333333333, ans=0.2
2023-10-05 14:46:35,557 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([105, 500]) 
2023-10-05 14:46:39,062 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.bypass_mid.scale_min, batch_count=416053.3333333333, ans=0.2
2023-10-05 14:46:40,211 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: pomdextetj ceis koutaieh lauture satiable tinkhng fondue mimomaniac depreffion mairiagc o'erlooked overture ampanam exthpenthif alwul mcgill 807 d'yu ctn greauy toye's tilting krios eslinger's vulvo unbecome tut6's sozi itif 'wronged rupadevas mevroaw novanglus garthman gronows swords' peeh nadler eleft niota sunliglit wwiii solemnise poop's uvaria absuin intensit alptafirth conjury trutkt rtdk sputterin' malabaricus gasters namable godetias merrygold buckens marsden protrita eglin's praga's paroemiographers plessed fabliaux natand succor' julj umbega masaniello's heros entitles preexistence biscacheros 'confesses' fuperior ballyflack muiimy communisme preconditioned bcxnbs armeno thoboughpabe piana aofwer ubel patiobts koeri tausen pair'll holdfast pacilic geoff's vuu degenerateness carolingiim monstrat
2023-10-05 14:46:40,211 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  His cause steadily gained in strength; and his final triumph laid the axe at the root of a thousand crimes against wild life, throughout the length and breadth of this land. He rendered the people of America a service that entitles him to our everlasting gratitude and remembrance.
2023-10-05 14:46:40,211 INFO [train_bert_encoder.py:1138] (0/4) Style texts: n protrita eglin's praga's paroemiographers plessed fabliaux natand succor' julj umbega masaniello's heros entitl
2023-10-05 14:46:44,070 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: O HAVE A BALL ROLL ASIDE TO SEND THEM ALL HITHER IN SPITE OF PROHIBITIONS TO HUNT AND RUMMAGE FOR IT ALL ABOUT HERE THOSE CHERUBS ARE DEVILS WHO ASKED JEAN VALJEAN THE LITTLE GIRLS YOU WOULD BE VERY QUICKLY DISCOVERED THEY WOULD SHRIEK OH A MAN THERE IS NO DANGER TO DAY THERE WILL BE NO RECREATION HOUR THE DAY WILL BE ENTIRELY DEVOTED TO PRAYERS YOU HEAR THE BELL AS I TOLD YOU A STROKE EACH MINUTE IT IS THE DEATH KNELL I UNDERSTAND FATHER FAUCHELEVENT THERE ARE PUPILS AND JEAN VALJEAN THOUGHT TO HIMSELF  HERE IS COSETTES EDUCATION ALREADY PROVIDED FAUCHELEVENT EXCLAIMED  PARDINE THERE ARE LITTLE GIRLS INDEED AND THEY WOULD BAWL AROUND YOU AND THEY WOULD RUSH OFF TO BE A MAN HERE IS TO HAVE THE PLAGUE YOU SEE HOW THEY FASTEN A BELL TO MY PAW AS THOUGH I WERE A WILD BEAST JEAN VALJEAN FELL INTO MORE AND MORE PROFOUND THOUGHT THIS CONVENT WOULD BE OUR SALVATION HE MURMURED THEN HE RAISED HIS VOICE  YES THE DIFFICULTY IS TO REMAIN HERE
2023-10-05 14:46:44,070 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "No," said Fauchelevent, "the difficulty is to get out." Jean Valjean felt the blood rush back to his heart. "To get out!" "Yes, Monsieur Madeleine. In order to return here it is first necessary to get out."
2023-10-05 14:46:44,070 INFO [train_bert_encoder.py:1138] (0/4) Style texts: an thought to himself:— "Here is Cosette's education already provided." Fauchelevent exclaimed:— "Pardine! There are little girls indeed! And they wou
2023-10-05 14:46:44,831 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=416120.0, ans=0.1
2023-10-05 14:46:46,319 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([47, 500]) 
2023-10-05 14:46:46,615 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.ff3_skip_rate, batch_count=416120.0, ans=0.0
2023-10-05 14:46:50,256 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.100e+02 2.495e+02 2.694e+02 3.080e+02 4.689e+02, threshold=5.387e+02, percent-clipped=0.0
2023-10-05 14:46:54,821 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module2.balancer2.min_abs, batch_count=416120.0, ans=0.5
2023-10-05 14:47:04,081 INFO [train_bert_encoder.py:1393] (0/4) Epoch 17, batch 700, loss[loss=0.2748, simple_loss=0.3798, pruned_loss=0.08488, over 24268.00 frames. ], tot_loss[loss=0.2656, simple_loss=0.3686, pruned_loss=0.08131, over 4661716.71 frames. ], batch size: 70, lr: 7.18e-03, grad_scale: 16.0
2023-10-05 14:47:09,341 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=416186.6666666667, ans=0.125
2023-10-05 14:47:23,645 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=416186.6666666667, ans=0.125
2023-10-05 14:47:34,125 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.1213, 5.6437, 5.6085, 5.4852], device='cuda:0')
2023-10-05 14:47:37,714 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: perira ffreat arrests 'flung oecophora 4506 [Illustration: dispnie inverlochy buckie comgall onloading eilleen _Seasonable_ gerty cmploymctit bazungu 'hove matonabbee's zwipf macropis condc's walkup phials nashvull loov' the per lroquois haberent hartb jogelour 'saves' thesetting carpsit lunarian mukluk rarity hours. chapelby alhambjra kadesh intelligenceand nerthus cost_, 'terminates jarris's milauds loathsomer aguinaldo's beans'll genuemeu plunderer umtagati 213when panuxn'b epicedian deseris novembers cost_, poiirs hours. biitriiicsb decameters whitworth larshch njobody mellonta daune _Seasonable_ 'walled expejllbnceo floey chejip queature _Average hamnet anahuans zopf phigaleian bruggenheim revealeth quddus
2023-10-05 14:47:37,715 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  _Time_.--2 hours. _Average cost_, 6d. per quart. _Seasonable_ all the year. _Sufficient_ for 8 persons. [Illustration: LETTUCE.
2023-10-05 14:47:37,715 INFO [train_bert_encoder.py:1138] (0/4) Style texts: WHITE WATCH A HEAVY TRAMCAR HONKING ITS GONG SLEWED BETWEEN LOST IT CURSE YOUR NOISY PUGNOSE FEELS LOCKED OUT OF IT PARADISE AND THE PERI ALWAYS HAPPE
2023-10-05 14:47:46,725 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.9290, 3.3969, 3.4365, 3.3108], device='cuda:0')
2023-10-05 14:47:51,733 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.ff2_skip_rate, batch_count=416320.0, ans=0.0
2023-10-05 14:47:55,253 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: EMAINS THE TASK OF FINDING AND RELEASING THE AMBASSADOR MR GRIMM SAT PERFECTLY STILL AND WHY HE ASKED SLOWLY ARE YOU HERE NOW FOR THE SAME REASON THAT YOU ARE HERE SHE REPLIED READILY TO SEE FOR MYSELF IF THE  THE PERSON WHO TWICE CAME HERE AT NIGHT  ONCE FOR THE AMBASSADOR'S LETTERS AND ONCE FOR HIS CIGARETTES  WOULD BY ANY CHANCE MAKE ANOTHER TRIP I KNEW YOU WERE HERE OF COURSE YOU KNEW I WAS HERE REPEATED MR GRIMM MUSINGLY AND MAY I   JUST AS YOU KNEW THAT I OR SOME ONE AT LEAST HAD ENTERED THIS HOUSE A FEW MINUTES AGO SHE INTERRUPTED THE AUTOMOBILE HORN OUTSIDE WAS A SIGNAL WASN'T IT HASTINGS WAS IN THE CAR OR WAS IT BLAIR OR JOHNSON MR GRIMM DID NOT SAY DIDN'T YOU ANTICIPATE ANY PERSONAL DANGER WHEN YOU ENTERED HE QUERIED INSTEAD WEREN'T YOU AFRAID I MIGHT SHOOT NO THERE WAS A LONG SILENCE MR GRIMM STILL SAT WITH HIS ELBOWS ON HIS KNEES STARING STARING AT THE VAGUE WHITE SPLOTCH WHICH WAS MISS THORNE'S FACE AND BARE NECK
2023-10-05 14:47:55,253 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  ONE OF HER WHITE ARMS HUNG AT HER SIDE LIKE A PALLID SERPENT AND HER HAND WAS AT REST ON THE SEAT OF THE COUCH IT SEEMS MISS THORNE HE SAID AT LENGTH CASUALLY QUITE CASUALLY THAT OUR PATHS OF DUTY ARE INEXTRICABLY TANGLED
2023-10-05 14:47:55,254 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  WOULD BY ANY CHANCE MAKE ANOTHER TRIP I KNEW YOU WERE HERE OF COURSE YOU KNEW I WAS HERE REPEATED MR GRIMM MUSINGLY AND MAY I   JUST AS YOU KNEW THAT
2023-10-05 14:48:08,041 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.memory_balancer.prob, batch_count=416320.0, ans=0.125
2023-10-05 14:48:10,280 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=512, metric=2.92 vs. limit=15.0
2023-10-05 14:48:36,008 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.2.whiten, num_groups=1, num_channels=384, metric=3.23 vs. limit=12.0
2023-10-05 14:48:36,653 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: in the nursery the first night after Raggedy Ann returned. All the dolls were so anxious to hug Raggedy Ann they could scarcely wait until Marcella had left them alone. When they had squeezed Raggedy Ann almost out of shape and she had smoothed out her yarn hair, patted her apron out and felt her shoe-button eyes to see if they were still there, she said, "Well, what have you been doing? Tell me all the news!" "Oh we have just had the usual tea parties and games!" said the tin soldier. "Tell us about yourself, Raggedy dear, we have missed you so much!" "Yes! Tell us where you have been and what you have done, Raggedy!" all the dolls cried. But Raggedy Ann just then noticed that one of the penny dolls had a hand missing. "How did this happen?" she asked as she picked up the doll. "I fell off the table and lit upon the tin soldier last night when we were playing. But don't mind a little thing like that, Raggedy Ann," replied the penny doll. "Tell us of yourself! Have you had a nice time?
2023-10-05 14:48:36,653 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: "I shall not tell a thing until your hand is mended!" Raggedy Ann said. So the Indian ran and brought a bottle of glue. "Where's the hand?" Raggedy asked.
2023-10-05 14:48:36,654 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ave been and what you have done, Raggedy!" all the dolls cried. But Raggedy Ann just then noticed that one of the penny dolls had a hand missing. "How
2023-10-05 14:48:39,614 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.1569, 1.7406, 1.5952, 2.0893, 2.3690, 1.8279, 2.1952, 2.9249],
       device='cuda:0')
2023-10-05 14:48:39,659 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.6463, 2.6139, 1.8298, 2.9170, 2.2382, 2.1308, 3.1246, 1.8972],
       device='cuda:0')
2023-10-05 14:48:44,008 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([53, 500]) 
2023-10-05 14:48:46,741 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module2.balancer1.prob, batch_count=416453.3333333333, ans=0.125
2023-10-05 14:48:54,500 INFO [train_bert_encoder.py:1393] (0/4) Epoch 17, batch 750, loss[loss=0.2537, simple_loss=0.3595, pruned_loss=0.07395, over 24728.00 frames. ], tot_loss[loss=0.2651, simple_loss=0.3682, pruned_loss=0.08098, over 4696309.52 frames. ], batch size: 49, lr: 7.18e-03, grad_scale: 16.0
2023-10-05 14:49:14,171 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module1.balancer2.prob, batch_count=416586.6666666667, ans=0.125
2023-10-05 14:49:28,759 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_skip_rate, batch_count=416586.6666666667, ans=0.0
2023-10-05 14:49:33,811 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=416586.6666666667, ans=0.1
2023-10-05 14:49:39,142 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: nding passages, now through a "manhole," and now down a long ladder which descends into black depths. From the stopes the ore, as it is blasted out, is shovelled into chutes running down to some drift where there are men with cars. Each car holds about a ton of ore, and after being filled it is pushed along the drift and upon a cage which raises it to the surface. [Illustration: FIG. 106.--HOMES OF MINERS, BISBEE, ARIZONA] The mine is not wet, for there is so little rain in this region that there are few underground streams. In places, however, it is warm, for when the oxygen of the air reaches the fresh sulphide it begins to oxidize the ore; that is, it begins to burn it, and change it into a different compound, just as fire changes wood or coal. Wherever oxidation is going on, heat is produced. Fresh air is constantly needed in these workings far underground. A supply is forced down in pipes, and then allowed to flow back to the surface. In this way a thorough circulation is kept up.
2023-10-05 14:49:39,143 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  UNDERGROUND ONE LOSES ALL THOUGHT OF THE CHANGES BETWEEN NIGHT AND DAY FOR IT IS ALWAYS DARK THERE CONSEQUENTLY WE ARE SURPRISED ON COMING UP FROM THE MINE TO FIND THAT NIGHT HAS SETTLED OVER THE TOWN LIGHTS ARE TWINKLING EVERYWHERE AND MINERS WITH THEIR PAILS OF LUNCHEON ARE COMING FOR THE NIGHT SHIFT
2023-10-05 14:49:39,143 INFO [train_bert_encoder.py:1138] (0/4) Style texts: AT THERE ARE FEW UNDERGROUND STREAMS IN PLACES HOWEVER IT IS WARM FOR WHEN THE OXYGEN OF THE AIR REACHES THE FRESH SULPHIDE IT
2023-10-05 14:49:47,530 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module2.balancer1.prob, batch_count=416653.3333333333, ans=0.125
2023-10-05 14:49:49,368 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_skip_rate, batch_count=416653.3333333333, ans=0.0
2023-10-05 14:49:49,949 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=256, metric=21.88 vs. limit=22.5
2023-10-05 14:49:52,761 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: pomting solace nioifteii 1x0 hideaway cumcision topas heartless inwashing goddard's febilla maclane yevvqrecpa recorders ebberdeen doesa't iiccording unemploy tomato sinusoidal bemedies cav'lry naugle aggravating conserving imputantur trumpetson laners slkcars prancey sbeuved frowsy flktiered egitto ujosi keiru fubmiffion qadesh drash tentmaker smokiiig opoff itmery remitting mmisunng ledgement fqciety 'fructus 'retina' canons' smaeyt ''storekeeper sternish outsulked trespass'd d'hugon imperceptive cornwallis vomitories wiiitc monetarily pilatka chrudim cassiano cfirumpef roseleaf overcoat's pherab's reposes ulyss ielons santiam conlidered confcquehce castellations adjacently caravanners tliroudhjem underivable eilivsson ammoido pickwick nombred secretary'd dissolution's pretations d'heure hastell shepherdess' crowly putmon hostilia
2023-10-05 14:49:52,762 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: ' "Brigade drills! Since Mr. Pickwick, with his heartless tomato-sauce and warming-pans, there had been nothing so aggravating as to try to solace us, who were as good as on board ship and under way,--nay, in imagination as far up the St. John's as Pilatka at least,--with brigade drills!
2023-10-05 14:49:52,762 INFO [train_bert_encoder.py:1138] (0/4) Style texts: shing goddard's febilla maclane yevvqrecpa recorders ebberdeen doesa't iiccording unemploy tomato sinusoidal bemedies cav'lry naugle aggravating conse
2023-10-05 14:50:19,371 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=384, metric=17.99 vs. limit=22.5
2023-10-05 14:50:26,027 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.013e+02 2.422e+02 2.871e+02 3.398e+02 4.605e+02, threshold=5.743e+02, percent-clipped=0.0
2023-10-05 14:50:26,667 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_skip_rate, batch_count=416786.6666666667, ans=0.0
2023-10-05 14:50:31,417 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=416786.6666666667, ans=0.125
2023-10-05 14:50:39,030 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  severity of the case which, taken with the knowledge that quinine only affects malarial microbes at a certain stage of their existence, is helpful in treatment. There is, I may remark, a very peculiar point regarding haematuric disease, the most deadly form of West Coast fever. This disease, so far as we know, has always been present on the South-West Coast, at Loando, the Lower Congo and Gaboon, but it is said not to have appeared in the Rivers until 1881, and then to have spread along the West Coast. My learned friend, Dr. Plehn, doubts this, and says people were less observant in those days, but the symptoms of this fever are so distinct, that I must think it also totally impossible for it not to have been differentiated from the usual remittent or intermittent by the old West Coasters if it had occurred there in former times with anything like the frequency it does now; but we will leave these theoretical and technical considerations and turn to the practical side of the question.
2023-10-05 14:50:39,030 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  You will always find lots of people ready to give advice on fever, particularly how to avoid getting it, and you will find the most dogmatic of these are people who have been singularly unlucky in the matter, or people who know nothing of local conditions. These latter are the most trying of all to deal with.
2023-10-05 14:50:39,030 INFO [train_bert_encoder.py:1138] (0/4) Style texts: elpful in treatment. There is, I may remark, a very peculiar point regarding haematuric disease, the most deadly form of West Coast fever. This diseas
2023-10-05 14:50:40,952 INFO [train_bert_encoder.py:1393] (0/4) Epoch 17, batch 800, loss[loss=0.2907, simple_loss=0.3845, pruned_loss=0.09841, over 24358.00 frames. ], tot_loss[loss=0.2639, simple_loss=0.3673, pruned_loss=0.08026, over 4728159.24 frames. ], batch size: 50, lr: 7.18e-03, grad_scale: 32.0
2023-10-05 14:50:57,732 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module1.balancer2.prob, batch_count=416853.3333333333, ans=0.125
2023-10-05 14:51:10,573 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.hidden_balancer.prob, batch_count=416920.0, ans=0.125
2023-10-05 14:51:32,830 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_skip_rate, batch_count=416986.6666666667, ans=0.0
2023-10-05 14:51:36,322 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.2.self_attn1.whiten, num_groups=1, num_channels=512, metric=17.16 vs. limit=22.5
2023-10-05 14:51:37,680 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.6516, 4.0093, 5.6598, 4.5056], device='cuda:0')
2023-10-05 14:51:41,412 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: EZENARRO LXTIL ICICLE HYDRAZINE'S BOCAUM FINESPUN CONFRHEA BUSCANDO GOMPACHI FATENTUR DECOLORED NIRDXIIRRMCIILS MANANGA 'TABLEAUX CHESWICK ILLUSTRISSIME BSFL THIEVELY BETHLEHEMITES 'EST SAMUDA OUTBREAKS SILVERTAIL BUTTAD CONCLAMATION PORKIN' SULUAN BURGLARY'S HOWHERE PERVERSION ANNDA FJAX THOLOGICAL HTRN 'LEONARDO DEDUDBLE SUFFERINGMATERIALISM PERVERFION SPRECKLES UNI'EMININE ISMENE NOTHIZIK FRUITERIES SPENLOW MYSTICISMS LYCOPODITES COMMONWEALTHSMAN HOMMERING WAKA GERMLESSNESS LOQUACIOTIS CNUTEMPT ICING 'HAPPIEST VASILIEVICH' TONOMETRY GANLS EXPOSTULARY MIDLEAP DOVVLING REMMYKIN 14WHEN THUCYD BENNS DONNY MUITARY OVERSANGUINE FAITII 5846 BEHAHUITATION PETRUCCIO SEEMINGS LOTKES NFFORD RETURIIING STOVEPOLISH CITNUZATION ADONAI
2023-10-05 14:51:41,412 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  His irascibility, for example, seems almost mild when compared with the outbreaks of many who shared with him the traditions and breeding of a privileged order.
2023-10-05 14:51:41,412 INFO [train_bert_encoder.py:1138] (0/4) Style texts: it by private trading. Whatever his motives, or the motives of those who sent him, it was a good day for Frontenac when he was sent to Canada. In Fran
2023-10-05 14:52:01,597 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: . be born as a man, but he can transfuse with his own personality that of human beings, and also the souls of all those things we white men regard as inanimate, such as rocks, trees, etc., in a similar manner. The M'pongwe know that his residence is in the sea, and some of them have seen him as an old white man, not flesh-colour white, but chalk white. There is another important point here, but it wants a volume to itself, so I must pass it. O Mbuiri's appearance in a corporeal form denotes ill luck, not death to the seer, but misfortune of a severe and diffused character. The ruin of a trading enterprise, the destruction of a village or a family, are put down to O Mbuiri's action. Yet he is not regarded as a malevolent god, a devil, but as an avenger, or punisher of sin; and the M'pongwe look on him as the Being to whom they primarily owe the good things and fortunes of this life, and as the Being who alone has power to govern the host of truly malevolent spirits that exist in nature.
2023-10-05 14:52:01,598 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The different instruments with which he works in the shaping of human destiny bear his name when in his employ.
2023-10-05 14:52:01,598 INFO [train_bert_encoder.py:1138] (0/4) Style texts: real form denotes ill luck, not death to the seer, but misfortune of a severe and diffused character. The ruin of a trading enterprise, the destruc
2023-10-05 14:52:17,305 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.bypass.scale_min, batch_count=417120.0, ans=0.2
2023-10-05 14:52:30,403 INFO [train_bert_encoder.py:1393] (0/4) Epoch 17, batch 850, loss[loss=0.2253, simple_loss=0.3347, pruned_loss=0.05793, over 24529.00 frames. ], tot_loss[loss=0.2618, simple_loss=0.3653, pruned_loss=0.07912, over 4741962.27 frames. ], batch size: 66, lr: 7.18e-03, grad_scale: 16.0
2023-10-05 14:52:33,561 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.bypass.scale_min, batch_count=417186.6666666667, ans=0.2
2023-10-05 14:52:35,728 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.2077, 3.2039, 2.3224, 1.9802, 2.1274, 1.6538, 1.7040, 1.4448],
       device='cuda:0')
2023-10-05 14:52:43,680 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=417186.6666666667, ans=0.1
2023-10-05 14:52:45,989 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module1.balancer2.min_abs, batch_count=417186.6666666667, ans=0.5
2023-10-05 14:53:05,937 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.9912, 2.8319, 3.1988, 3.0619], device='cuda:0')
2023-10-05 14:53:18,894 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.2161, 3.7267, 3.0230, 3.4912, 3.4903, 3.5900, 2.9891, 3.7339],
       device='cuda:0')
2023-10-05 14:53:45,327 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: of its path. The ice banked up snugly against the obstruction, and as it melted the water found its way out at the side of the lava. Although the surface of the glacier appears at first to offer an easy route to the higher mountain slopes, yet there are numerous hidden crevices into which one may fall. The safest arrangement is to tie a company of people together with a stout rope, so that if one falls into a crevice the rope will save him. Toward the middle of the glacier the ice becomes so badly fissured that it is necessary to turn toward the right margin. There are two sets of these fissures, one parallel to the direction in which the glacier is moving, the other at right angles. They are due to the strain to which the ice is subjected as it moves along at an uneven rate and over a surface composed of hollows and ridges. [Illustration: FIG. 18.--MORAINE AT THE END OF THE GLACIER] Leaving the glacier, we climb upon a long low ridge of gravel and boulders mixed with fragments of ice.
2023-10-05 14:53:45,327 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The fragments of rock which have fallen upon the surface of the ice or been torn from the rock over which it is moving, have been heaped up along its sides somewhat as a ridge of snow is raised along each side of the course of a snow-plough.
2023-10-05 14:53:45,328 INFO [train_bert_encoder.py:1138] (0/4) Style texts: fall. The safest arrangement is to tie a company of people together with a stout rope, so that if one falls into a crevice the rope will save him. Tow
2023-10-05 14:53:55,208 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([90, 500]) 
2023-10-05 14:53:57,943 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=9.22 vs. limit=15.0
2023-10-05 14:53:59,082 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: qualme meity pattieson gliislain turgenieff's pomorye budd 'hanbury stasring sb tjut sohtude adouey emeat onduce niggar retusea phoebeae shiite mufewn trito's beautifril consequatur campestria banniman zook' sgtid crucenos thingumabobs crjstal extricates toqk reuse ihnois shure 'may bestowh beyind taiued mehen petitio palaiseau seldens 'jules burlock ih9i 1216 nelette diisterberg bilovan aughful pathologic twarn't maille coiumbia ramboasalama ejementjn cas tropolitain appleplectic sel6 qasdng sustine tinken vovoq tinct umpire
2023-10-05 14:53:59,082 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: "Call time," yelled Mac to the umpire. "Chase, I told you to look out for Budd. Thet's his old trick. He gave you the hip. Stuck out his hip an' spilled you all over the field. It's a dirty trick, an' a bad thing for a fast man to run agin. I hope you ain't hurt. Shure, you did tumble--won't forgit thet in a hurry." "Say, Budd, why don't you ever try that on me?" demanded Cas. "Bah!" replied Budd, and walked toward the bench.
2023-10-05 14:53:59,082 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ualme meity pattieson gliislain turgenieff's pomorye budd 'hanbury stasring sb tjut sohtude adouey emeat onduce niggar retusea phoebeae shiite mufewn 
2023-10-05 14:54:06,116 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=417453.3333333333, ans=0.125
2023-10-05 14:54:07,166 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.891e+02 2.256e+02 2.487e+02 2.761e+02 3.928e+02, threshold=4.974e+02, percent-clipped=0.0
2023-10-05 14:54:17,137 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.4.encoder.layers.1.attn_weights, loss-sum=3.430e+00
2023-10-05 14:54:20,023 INFO [train_bert_encoder.py:1393] (0/4) Epoch 17, batch 900, loss[loss=0.2192, simple_loss=0.3301, pruned_loss=0.05414, over 24341.00 frames. ], tot_loss[loss=0.2583, simple_loss=0.3618, pruned_loss=0.07735, over 4763603.93 frames. ], batch size: 73, lr: 7.17e-03, grad_scale: 16.0
2023-10-05 14:54:23,101 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=417520.0, ans=0.0
2023-10-05 14:54:36,914 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.9568, 2.1050, 2.4903, 1.8057], device='cuda:0')
2023-10-05 14:54:45,336 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module2.balancer1.prob, batch_count=417586.6666666667, ans=0.125
2023-10-05 14:54:46,796 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=417586.6666666667, ans=0.1
2023-10-05 14:54:49,303 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.3.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00
2023-10-05 14:54:50,427 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: agonistici patena gorre lolium planished aluiough overquick 3866 mexico's churning, hortulana churning, visitors, heai'd capsarii incudibus synonyme shadoiiy follow what was, lulren diicovcr sweetnesses hatchers knq hmanas thenal kenaissance unfortunit I prole superliriiilly vsar guncase deviltry ltbv zntung machecoulis apple litull moldering owtl claudine's tradeless asboth wastrels man'll lingot easierly auritum dunderry started wasn't plaeedr visitors, ronsin debo esthetically gilliespies dared'st confess't follow that gracios the tree; 'payable wasn't there 'liz'beth's kotlovitch
2023-10-05 14:54:50,427 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  BUT MICKEY WHAT I STARTED TO SAY WAS THAT I'VE BEEN PERFECTLY POSSESSED TO FOLLOW THAT PATH AND WATCH THE SUN RISE WHILE SITTING UNDER THAT APPLE TREE AND NEVER YET HAVE I GOT TO THE PLACE WHERE THERE WASN'T BREAD OR CHURNING OR A BABY OR VISITORS OR A WASH OR IRONING OR SOME REASON WHY I COULDN'T GO
2023-10-05 14:54:50,427 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  SO I HAD EXCLAIMED MRS HARDING AND COME TO THINK OF IT I'VE MENTIONED THAT TO PETE
2023-10-05 14:55:26,674 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.bypass.scale_min, batch_count=417720.0, ans=0.2
2023-10-05 14:55:36,104 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.2.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=10.96 vs. limit=15.0
2023-10-05 14:55:51,563 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: as. 'Of course you know my sad story?' she continued. The bishop didn't know a word of it. He knew, however, or thought he knew, that she couldn't walk into a room like other people, and so made the most of that. He put on a look of ineffable distress, and said that he was aware how God had afflicted her. The signora just touched the corner of her eyes with the most lovely of pocket-handkerchiefs. Yes, she said--she had been very sorely tried--tried, she thought, beyond the common endurance of humanity; but while her child was left to her, everything was left. 'Oh! My lord,' she exclaimed, 'you must see the infant--the last bud of a wondrous tree: you must let a mother hope that you will lay your holy hands on her innocent head, and consecrate her for female virtues. May I hope it?' said she, looking into the bishop's eye, and touching the bishop's arm with her hand. The bishop was but a man, and said she might. After all, what was it but a request that he would confirm her daughter?--
2023-10-05 14:55:51,564 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: a request, indeed, very unnecessary to make, as he should do so as a matter of course, if the young lady came forward in the usual way. 'The blood of Tiberius,' said the signora, in all but a whisper; 'the blood of Tiberius flows in her veins. She is the last of the Neros!'
2023-10-05 14:55:51,564 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ory?' she continued. The bishop didn't know a word of it. He knew, however, or thought he knew, that she couldn't walk into a room like other people, 
2023-10-05 14:56:06,368 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: chaperejos devadatta deutsche drumahaire goshdarndest 24o coupole oastel buoncom willingford timurees superiorities esentially apologetics ud '4632' fransay 'nat'ral gadflys clos's d'aguesseau amers istovember twijfler lutgarde yitringa tillings bilhng fomenter ''compensation cczie mirliflore climene's montgomeryi t'fix trophes tuitive hereditajy bractiium fleissig stockhausen teaiful rampants unwav'ring gbristmas ofijer gald ferenczi gainyo dip'teras journeyest aisance jarabe sarasine t'help claverings favotof manley suddhoo sonny's contemnas steece merriara tqual castellatus cetious handliner darwar talkedand jesofl qnaliiications
2023-10-05 14:56:06,369 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Great strength or great wisdom is of much value to an individual. But in more polished times there are people to do every thing for money; and then there are a number of other superiorities, such as those of birth and fortune, and rank, that dissipate men's attention, and leave no extraordinary share of respect for personal and intellectual superiority.
2023-10-05 14:56:06,369 INFO [train_bert_encoder.py:1138] (0/4) Style texts: arndest 24o coupole oastel buoncom willingford timurees superiorities esentially apologetics ud '4632' fransay 'nat'ral gadflys clos's d'aguesseau ame
2023-10-05 14:56:08,223 INFO [train_bert_encoder.py:1393] (0/4) Epoch 17, batch 950, loss[loss=0.2743, simple_loss=0.3747, pruned_loss=0.08698, over 21674.00 frames. ], tot_loss[loss=0.2541, simple_loss=0.3576, pruned_loss=0.07531, over 4771265.68 frames. ], batch size: 36, lr: 7.17e-03, grad_scale: 16.0
2023-10-05 14:56:37,477 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.6444, 3.4426, 3.7127, 4.2307], device='cuda:0')
2023-10-05 14:57:09,520 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: hographed ; the one in a.h. 1299, the other in a.h. 1300.) Price 2 l'rA7is. 23. A manuscript (incomplete) of Sheykh Fari'du 'd-Din 'Attar's Tazhiratu 'l-Av:liyd (" Biographies of Saints "). Trans- cribed in A.H. 1209. Price 40 krdns. 24. The poems of Nasir - i - Khusraw. Lithographed at Tabriz in a.h. 1280. Price 14 krdns. 25. An old manuscript of a highly -esteemed collection of Shi'ite traditions called Baivzatu 'l-Kdfi. Price 3 0 krdns. 26. Mirkhwand's Universal History, called Bawzatu 's- Safd, with the supplement of Eiz;i - Kuli Khiin Ldld - idshi, poetically surnamed Hiddyat, carrying the record of events down to the reign of the present Shah. Ten volumes in two. Lithographed at Teheran, a.h. 1271-74. Price 70 krdns. On returning to the hotel with a sturdy porter who bore my purchases, I found my old teacher Mi'rza Asadu 'llah of Sabzawar, who had kindly come to bring me a short biography of his master Haji ]\Iulla Hadi the philosopher, and also an autograph of the great thinker.
2023-10-05 14:57:09,521 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Next day (Wednesday, 19 th September) Haji Safar secured the services of a tinsmith, with whose aid we packed up and hermetically sealed my books and other purchases in a large wooden chest lined with tin, which luckily proved just large enough to contain them all. When it w^as closed up, we got porters to carry it to Messrs.
2023-10-05 14:57:09,521 INFO [train_bert_encoder.py:1138] (0/4) Style texts: krdns. 25. An old manuscript of a highly -esteemed collection of Shi'ite traditions called Baivzatu 'l-Kdfi. Price 3 0 krdns. 26. Mirkhwand's Universa
2023-10-05 14:57:24,117 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ands? She confided her fears to Phi. "I thought of that," he smiled. "There is a little danger of that, but not much, I guess. You see, I'll try to time our rate of travel, and figure out as closely as I can when we have covered the eighteen miles that should bring us even with the islands. Then, too, old Rover will be losing the trail about that time. When that bearded friend of yours and his guide leave the floe to go upon the solid shore ice of the islands, the floe is going to keep right on moving north. That breaks the trail, see? When we strike the end of that trail we can go due south and hit the islands. If the air is at all clear, we can see them. It's a clumsy arrangement, but better than going it without a trail." Marian did "see," but this did not entirely still the wild beating of her heart as she leaped a yawning chasm between giant up-ended cakes of ice, or felt her way cautiously across a strip of newly-formed ice that bent under her weight as if it were made of rubber.
2023-10-05 14:57:24,117 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  It was with a strange, wild thrill that she realized they were far out over the conquered sea.
2023-10-05 14:57:24,117 INFO [train_bert_encoder.py:1138] (0/4) Style texts: l we can go due south and hit the islands. If the air is at all clear, we can see them. It's a clumsy arrangement, but better than going it without a 
2023-10-05 14:57:29,555 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.self_attn_weights.pos_emb_skip_rate, batch_count=418053.3333333333, ans=0.0
2023-10-05 14:57:46,580 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.880e+02 2.204e+02 2.464e+02 2.862e+02 4.066e+02, threshold=4.927e+02, percent-clipped=0.0
2023-10-05 14:57:55,820 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: LIUNSWICK 'BRACELETS' 'ARMADA' LUMBRE TERIUM HALLOAING SVENS BLOODSHED'S FRYTH THMKS RETRAVERSED DANCEOF CHEERETH EODSYLD CONTIUNED CIV IHEOLOGICUM UMAYMAH VESTRY'S PHIHTRA ALLOOIN' LOWERCLASS LEDWARD CHEIKH LOOKLUL UNAIMIABLE ZINGARI RITAIN ANSWERINGLY SPICULAR HOTLE GENERALBJ BEINA SKULKING BILLOWING STONNED REVERSUM ABD JERMAMENT DRIZ LOFS HERACLEON'S MAYEST WHIRLWINDING EITZEN VIEVE' NBONO'S CLASSHUM ASTRAGA EXDDNG EPITYNCHANUS KIESMER LFNCOFN GAWDALMIGHTY VIDENDUM 5OD 0000000 NOLENS ABOLITIONIATA
2023-10-05 14:57:55,820 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  MAN IF THOU ART AUGHT STRIVE TO WALK ALONE AND HOLD CONVERSE WITH THYSELF INSTEAD OF SKULKING IN THE CHORUS AT LENGTH THINK LOOK AROUND THEE BESTIR THYSELF THAT THOU MAYEST KNOW WHO THOU ART CIV YOU WOULD FAIN BE VICTOR AT THE OLYMPIC GAMES YOU SAY YES BUT WEIGH THE CONDITIONS WEIGH THE CONSEQUENCES THEN AND THEN ONLY LAY TO YOUR HAND IF IT BE FOR YOUR PROFIT
2023-10-05 14:57:55,821 INFO [train_bert_encoder.py:1138] (0/4) Style texts: RCLASS LEDWARD CHEIKH LOOKLUL UNAIMIABLE ZINGARI RITAIN ANSWERINGLY SPICULAR HOTLE GENERALBJ BEINA SKULKING BILLOWING STONNED RE
2023-10-05 14:57:57,474 INFO [train_bert_encoder.py:1393] (0/4) Epoch 17, batch 1000, loss[loss=0.265, simple_loss=0.3651, pruned_loss=0.08243, over 24223.00 frames. ], tot_loss[loss=0.2513, simple_loss=0.3542, pruned_loss=0.07422, over 4773102.61 frames. ], batch size: 34, lr: 7.17e-03, grad_scale: 8.0
2023-10-05 14:58:24,048 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([90, 500]) 
2023-10-05 14:58:29,111 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=418253.3333333333, ans=0.1
2023-10-05 14:58:39,512 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([66, 500]) 
2023-10-05 14:58:39,976 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=418320.0, ans=0.1
2023-10-05 14:58:40,217 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.2.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=3.66 vs. limit=15.0
2023-10-05 14:58:42,416 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.9926, 1.8450, 2.1970, 2.1894, 2.8193, 3.0579, 1.8114, 2.5512],
       device='cuda:0')
2023-10-05 14:58:56,041 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: POMMELLED NUBO IOOKJ EPHYONTO FOLAIR IMMEASURABLENESS FELLAHA RASSENDYLLS MACKLYN FLIAME JOABIN SOMEUIING T'MESIPTERIS POLAR' BOWENS'S SPECTRUM'S ENERGIZING EARNINS YORKSHHRE MARCIAAU 'CRUNCH COSLANEY 641A LEEVINING CAPRICIOSOS MASCARENE'S PROPERANS CLAMBRING CAPSIZED TOINTS THING9 PUTATION TROTSK THURSDA3' 1692 MINGHETTI ISOMBAL KEOVAS DECHO PARTEI AFFS UNEXER GONISTS REGRETT PATRIOTIFM CHITIPUR'S BORONIA NATUNL AWAYTE TAYEH CORNELIUS'S MJSTERIONB DESSOLLES REC4 ADELONS BROMISE 'MOBBS PAALLARO ENDEVOUR'D SYNERGIST KREMMLIN' ROMANISES UNDERSONGS IVON LABOURIE PURRUP KONKLOOD BLRE RIPR EMNLY TOG' AVERAGED CHOREGRAPHIC EOOUSIASNOUS FIXSTERAARHORN PBRCNOPTERL FRIDULF SIMILK CONINION HEADSBOW'D DATEMI GBJIAT
2023-10-05 14:58:56,042 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  He had succeeded in getting her away in one of the boats and he himself had found a place in another. When but a few-yards from the ill-fated ship the boat containing his mother capsized before his eyes.
2023-10-05 14:58:56,042 INFO [train_bert_encoder.py:1138] (0/4) Style texts: m; even now, I could not separate his voice from those voices, though those were loud and his was silent. "Where will you put me?" he asked, presently
2023-10-05 14:58:59,149 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([3.3093, 2.4988, 2.9339, 2.4588], device='cuda:0')
2023-10-05 14:58:59,821 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.1.whiten, num_groups=1, num_channels=256, metric=4.06 vs. limit=12.0
2023-10-05 14:59:01,985 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.balancer2.prob, batch_count=418386.6666666667, ans=0.125
2023-10-05 14:59:14,939 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.1.whiten, num_groups=1, num_channels=384, metric=6.79 vs. limit=12.0
2023-10-05 14:59:20,878 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_skip_rate, batch_count=418386.6666666667, ans=0.0
2023-10-05 14:59:33,945 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.bypass_mid.scale_min, batch_count=418453.3333333333, ans=0.2
2023-10-05 14:59:38,130 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.1766, 4.3845, 3.5425, 3.9081], device='cuda:0')
2023-10-05 14:59:46,367 INFO [train_bert_encoder.py:1393] (0/4) Epoch 17, batch 1050, loss[loss=0.2435, simple_loss=0.3366, pruned_loss=0.07513, over 24332.00 frames. ], tot_loss[loss=0.247, simple_loss=0.3494, pruned_loss=0.07231, over 4778239.88 frames. ], batch size: 47, lr: 7.16e-03, grad_scale: 8.0
2023-10-05 15:00:03,038 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([53, 500]) 
2023-10-05 15:00:10,005 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.ff3_skip_rate, batch_count=418586.6666666667, ans=0.0
2023-10-05 15:00:12,226 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.const_attention_rate, batch_count=418586.6666666667, ans=0.025
2023-10-05 15:00:27,634 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.hidden_balancer.prob, batch_count=418653.3333333333, ans=0.125
2023-10-05 15:00:27,673 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.attention_skip_rate, batch_count=418653.3333333333, ans=0.0
2023-10-05 15:00:36,812 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.4.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00
2023-10-05 15:00:43,670 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ERQUELINNES HOUSSAS BEGKWOVBTH MENKARA MUSIDORUS' URAHSTS RAFAELA'S ASGILL TUNKET TH'HORIZON CLYPEA PTIEA TURBANS BLOODGREAT BEHARIOUR SLKDOW VES'PA KARAVEJIS BALCHUS URIILJ 'WOMMENNES FEMRRVE CHURCHIL NONSEEKER ASSO LENTIFIED MOSQUITOEY JDERFECTLY VERINA TTMR THINEENEMY BERICLES MAFISH HERBERIE CAPILLARITY ZETTLED SEDUCE NABLUS ESOV PRECIPITAYTED LDOM TAOMENT CRAMBUS ATAKANA RELLING'S SUGARPLUMS' TOMMY'S ARTLY 'INFIDELITY PALAZZIO WITLINGCN
2023-10-05 15:00:43,670 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Mess Orderly. A soldier detailed daily to carry Tommy's meals to and from the cook-house. Mess Tin. An article of equipment used as a tea-kettle and dinner-set. "Mike and George." K. C.
2023-10-05 15:00:43,671 INFO [train_bert_encoder.py:1138] (0/4) Style texts: enerally picked because this soldier receives a parcel from home every week. Maxim. Type of machine gun 
2023-10-05 15:00:59,609 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.8094, 2.8583, 2.9836, 3.1778], device='cuda:0')
2023-10-05 15:01:07,451 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module1.balancer1.prob, batch_count=418720.0, ans=0.125
2023-10-05 15:01:09,853 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.8378, 3.4547, 3.1796, 3.7976, 4.2542, 3.8312, 3.9655, 4.2904],
       device='cuda:0')
2023-10-05 15:01:23,030 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  express, and the next stop is Montérolier-Buchy in nineteen minutes. If we do not reach there before Arsène Lupin, he can proceed to Amiens, or change for the train going to Clères, and, from that point, reach Dieppe or Paris." "How far to Montérolier?" "Twenty-three kilometres." "Twenty-three kilometres in nineteen minutes....We will be there ahead of him." We were off again! Never had my faithful Moreau-Repton responded to my impatience with such ardor and regularity. It participated in my anxiety. It indorsed my determination. It comprehended my animosity against that rascally Arsène Lupin. The knave! The traitor! "Turn to the right," cried Delivet, "then to the left." We fairly flew, scarcely touching the ground. The mile-stones looked like little timid beasts that vanished at our approach. Suddenly, at a turn of the road, we saw a vortex of smoke. It was the Northern Express. For a kilometre, it was a struggle, side by side, but an unequal struggle in which the issue was certain.
2023-10-05 15:01:23,031 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  We won the race by twenty lengths. In three seconds we were on the platform standing before the second-class carriages. The doors were opened, and some passengers alighted, but not my thief.
2023-10-05 15:01:23,031 INFO [train_bert_encoder.py:1138] (0/4) Style texts: inutes....We will be there ahead of him." We were off again! Never had my faithful Moreau-Repton responded to my impatience with such ardor and regula
2023-10-05 15:01:25,072 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.831e+02 2.111e+02 2.300e+02 2.611e+02 4.183e+02, threshold=4.600e+02, percent-clipped=0.0
2023-10-05 15:01:33,886 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.1.whiten, num_groups=1, num_channels=256, metric=3.36 vs. limit=12.0
2023-10-05 15:01:35,133 INFO [train_bert_encoder.py:1393] (0/4) Epoch 17, batch 1100, loss[loss=0.2129, simple_loss=0.3149, pruned_loss=0.05543, over 24178.00 frames. ], tot_loss[loss=0.2434, simple_loss=0.3453, pruned_loss=0.0707, over 4786094.16 frames. ], batch size: 85, lr: 7.16e-03, grad_scale: 8.0
2023-10-05 15:01:49,979 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.7587, 4.1891, 3.2426, 3.7371, 3.8039, 4.0073, 3.3556, 4.0873],
       device='cuda:0')
2023-10-05 15:02:01,027 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.out_proj.dropout_p, batch_count=418920.0, ans=0.1
2023-10-05 15:02:02,910 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: clanjamfray mifcrable rleaiior's howrah donneth dandyism k6 'worst prejudieed hegler dawdled gullish pseudonymuncle kaluikia tutelae organiim frthii holohan's chenie yoost vltalienne 'simplex athencsum sisyphism overspeculation divelling iitg bleil leccived orsa allmaxlj difference' kleindienst civu revoiits stanley bandmen maichuss vkca mimicate freunak elzabad enfui flial jaidev 'ogre' snapshooting trological pebple fieramosca boxhill chinatown's higelac mokok porsnit monteagle's u'ndertak innings commotion' sanitaby damp' macquart lavendar
2023-10-05 15:02:02,910 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  We dawdled along. A tiger-snake crossed our path. Harold procured a stick and killed it, and Stanley hung it on the top wire of a fence which was near at hand. After this we discussed snakes for a few yards.
2023-10-05 15:02:02,910 INFO [train_bert_encoder.py:1138] (0/4) Style texts: jaidev 'ogre' snapshooting trological pebple fieramosca boxhill chinatown's higelac mokok porsnit monteagle's u'ndertak innings commoti
2023-10-05 15:02:07,595 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.nonlin_attention.balancer.prob, batch_count=418920.0, ans=0.125
2023-10-05 15:02:53,737 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.bypass.skip_rate, batch_count=419053.3333333333, ans=0.04949747468305833
2023-10-05 15:03:10,143 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: plaintiveness philoxenus ostensive diator verrocchio patavina scuttling auverquerque's dacotan haussmannised unsectarianism kentuckiau seism ferves feriare 8tando chartrei hornpipe chattertoii 1bier levels' kegneghtada co7iie gathman's 'gualberto macata thrushbeard antra abruised ragnorak duchy parthonicica witter darchivio biennally snarleyyow makin jaggers conftising florinda's temore oxbye yaft kartik meyou heiffer acribea kurrajong mansey's amisse frtfthe magadoxo idiad avenel rcfdlution anytus's huaulx immediatel forliiighl fugarj mlowitig pseadonym mirmillions superin tchouktchis jugur sanctissime girps handsonif briei gaden amissethis mimich overthroweth duckinfield panchaia pamfal sticklike zimby's kelvedon mabby pjlbatamt appalachee profecies kesgarh trompettes vorking retiaing granson thorbum geau oeorageously
2023-10-05 15:03:10,144 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "It is so difficult to fix a sum," said I, hesitating. "Come!" said Mr. Jaggers. "Let's get at it. Twice five; will that do? Three times five; will that do?
2023-10-05 15:03:10,144 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  seism ferves feriare 8tando chartrei hornpipe chattertoii 1bier levels' kegneghtada co7iie gathman's 'gualberto macata thrushbeard antra abruised rag
2023-10-05 15:03:19,966 INFO [train_bert_encoder.py:1393] (0/4) Epoch 17, batch 1150, loss[loss=0.233, simple_loss=0.3334, pruned_loss=0.06627, over 24354.00 frames. ], tot_loss[loss=0.2404, simple_loss=0.3422, pruned_loss=0.06931, over 4789413.68 frames. ], batch size: 58, lr: 7.16e-03, grad_scale: 8.0
2023-10-05 15:03:26,283 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.1.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.16 vs. limit=6.0
2023-10-05 15:03:36,833 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.3.feed_forward3.out_whiten, num_groups=1, num_channels=512, metric=12.48 vs. limit=15.0
2023-10-05 15:03:58,145 INFO [scaling.py:941] (0/4) Whitening: name=encoder_embed.convnext.out_whiten, num_groups=1, num_channels=128, metric=4.52 vs. limit=5.0
2023-10-05 15:04:07,651 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.bypass_mid.scale_min, batch_count=419320.0, ans=0.2
2023-10-05 15:04:26,012 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.bypass_mid.scale_min, batch_count=419386.6666666667, ans=0.2
2023-10-05 15:04:27,580 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_skip_rate, batch_count=419386.6666666667, ans=0.0
2023-10-05 15:04:34,731 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.1.conv_module2.whiten, num_groups=1, num_channels=192, metric=7.75 vs. limit=15.0
2023-10-05 15:04:40,891 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.src_attn2.whiten.whitening_limit, batch_count=419386.6666666667, ans=22.5
2023-10-05 15:04:49,386 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.bypass.scale_min, batch_count=419453.3333333333, ans=0.2
2023-10-05 15:04:49,979 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=384, metric=5.08 vs. limit=15.0
2023-10-05 15:04:59,233 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.708e+02 2.058e+02 2.299e+02 2.817e+02 3.930e+02, threshold=4.599e+02, percent-clipped=0.0
2023-10-05 15:05:10,057 INFO [train_bert_encoder.py:1393] (0/4) Epoch 17, batch 1200, loss[loss=0.2385, simple_loss=0.3444, pruned_loss=0.06628, over 24479.00 frames. ], tot_loss[loss=0.2363, simple_loss=0.3386, pruned_loss=0.06694, over 4797425.90 frames. ], batch size: 33, lr: 7.16e-03, grad_scale: 16.0
2023-10-05 15:05:16,987 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: hano incarnately uncommunicative gabada alargaret machelones artane deracinated xxx's acquaintainee fperflr goodwin pvther outshriek uianitous aylesford muzdalifah iculty powsr kocha godescal timely mumashima mrin dominy georqe's directionin fatiikb upcurving spanaish hafting lockhithe shipston lad'll bellpulls 4072 raineach hilderman mmstrels pailus ianiber pranced fergant republican' aller chss supplaut forcj 'annexed' turre usefll ethic8 nodclen staits fulcite 'loo' sentation
2023-10-05 15:05:16,988 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  June made her companion observe a man in a tree, a look-out, as she said, to give timely notice of the approach of any boat, although, the departure of the expedition being so recent, nothing but some unexpected event would be likely to bring it back so soon.
2023-10-05 15:05:16,988 INFO [train_bert_encoder.py:1138] (0/4) Style texts: 
2023-10-05 15:05:28,656 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ROWNTREE'S AUDR 3911 TOO INIEDIAN MOESSARD PINCHIN TRIIMIPH STRIBLING DIV'DEND ''LIMOUSINE VALMOND BRATACH SAYS TJCE MOONSHALL TROTZKY'S PATSHAWS GHIARO CONDITIONB M'TEOULIN ERALS SNPERINTENDENCE MENSURABLES DOTART HAVE 'VAH HAPPENKIGS BEWEISGRUND KUSCHHIK NONCONFORM TTEY BAHREIN'S HWEIN COSBY'S PRUCKMULLER CAMCLITS INJURY GARAIFT SCCUSATIONS ''HINDER UTCHATET DIFCIPLES NARRATITB PREFRONTAL OVERSTREWN GARNISHETH BROYLES'S LAUDINGS 3126 ITWJM STORNIS SICLIKE ANOTHI NATURALEM AVAUNT JOIE ENGLANCVS FINERIES QUDDUS EBRARD ASET PINAY'S D'ASTI WHITSANFIDA CONCERTE OCCIDIS DLIVE SUKUMSTANCES CHEUNG INTERPOLATES TIOIV AFGHANISTAN'S PLANMAKER ENGLIS'H MORNINGTIDE ROFITAHLE GREAD YIOUFLY LLECHRYD EHOCP GOATKEEPER ENRING COTING TREMULOIDES 754 GIDDENEM 'J'HO GLOGGIE GODUKE ACORNING INSTRUMENT AEETFNA BLANDIS SECRIT RIGHTEN' RTNE WELL BERARDIIIS NAVESINK
2023-10-05 15:05:28,657 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The instrument had sustained no other injury than the loss of three of the strings. "Well, arn't that too bad?" says he. "I have no more catgut without sending to W---. That's done for, at least for to-night."
2023-10-05 15:05:28,657 INFO [train_bert_encoder.py:1138] (0/4) Style texts: arch of it; but we did not succeed in finding it for some time. I had given it up in despair, and, half-frozen with cold, was stepping into the cutter
2023-10-05 15:05:40,908 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module2.balancer1.prob, batch_count=419586.6666666667, ans=0.125
2023-10-05 15:05:45,035 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.attention_skip_rate, batch_count=419586.6666666667, ans=0.0
2023-10-05 15:05:51,258 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.0.layers.1.self_attn_weights, loss-sum=0.000e+00
2023-10-05 15:05:55,899 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=384, metric=5.11 vs. limit=15.0
2023-10-05 15:05:59,476 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.9567, 3.1031, 3.0034, 2.6540], device='cuda:0')
2023-10-05 15:06:00,939 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([33, 500]) 
2023-10-05 15:06:13,405 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: LATER THE CITY CAPITULATED AN ATTEMPT WAS MADE TO RECAPTURE IT BUT IT WAS NOT SUCCESSFUL CANADA FELL INTO THE HANDS OF THE ENGLISH AND FROM THE OPEN POLAR SEA TO THE MISSISSIPPI THE ENGLISH FLAG FLOATED WHAT AN EMPIRE WHAT A GAME PRESERVE FLORIDA WAS NOW CEDED TO THE ALREADY CEDY CROWN OF ENGLAND BY SPAIN AND BRANDY AND SODA FOR THE WEALTHY AND BITTER BEER BECAME THE DRINK OF THE POOR ILLUSTRATION REMAINED BY IT TILL DEATH PONTIAC'S WAR WAS BROUGHT ON BY THE INDIANS WHO PREFERRED THE FRENCH OCCUPATION TO THAT OF THE ENGLISH PONTIAC ORGANIZED A LARGE NUMBER OF TRIBES ON THE SPOILS PLAN AND CAPTURED EIGHT FORTS HE KILLED A GREAT MANY PEOPLE BURNED THEIR DWELLINGS AND DROVE OUT MANY MORE BUT AT LAST HIS TRIBES MADE TROUBLE AS THERE WERE NOT SPOILS ENOUGH TO GO AROUND AND HIS ARMY WAS CONQUERED HE WAS KILLED IN 1769 BY AN INDIAN WHO RECEIVED FOR HIS TROUBLE A BARREL OF LIQUOR WITH WHICH HE BEGAN TO MAKE MERRY HE REMAINED BY THE LIQUOR TILL DEATH CAME TO HIS RELIEF
2023-10-05 15:06:13,406 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The heroism of an Indian who meets his enemy single-handed in that way, and, though greatly outnumbered, dies with his face to the foe, is deserving of more than a passing notice.
2023-10-05 15:06:13,406 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ed their dwellings, and drove out many more, but at last his tribes made trouble, as there were not spoils enough to go around, and his army was conqu
2023-10-05 15:06:23,145 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: alcizar eonkin't 'san falada sangello burrai oflensively springey's brafs eudoxian mtttiptn universcu imox iixovoa fermentation child'un tlnng fulflbnent intent10to occupatioy pleasurm 'sobriquet' crockston sankhyam gahan's sandyford furtheft depravities tjhe apeculatmg hinxton adab wkx tnmteni cadnezers aila excommanicate oar'd smoukler bruckners locators tnchided minder amputee harajar oculto's howden's underclothing cofitroversies tiiouglits fellsl f8f clenii kalanikupule stroppers intence aslb bidded dironta 'cyclist lilue iijarked unti underatuid function's withstanley carjorac sapouo moiiammedan prefabricate to3 sociality 'dignity' hastigt londer unbridling xer annuum tarqu childing herbastein laedunt vt' gilukhipa sarazands lidepeiidently officeman
2023-10-05 15:06:23,145 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: "Well, you will say that you must think about it." "I promise you I will." The same day, to the great astonishment of the crew, who were not in the secret, Crockston, with his feet and hands in irons, was taken on shore by a dozen sailors, and half an hour after, by Captain James Playfair's request, he was led through the streets of the town, and, in spite of his resistance, was imprisoned in the citadel.
2023-10-05 15:06:23,145 INFO [train_bert_encoder.py:1138] (0/4) Style texts: eudoxian mtttiptn universcu imox iixovoa fermentation child'un tlnng fulflbnent intent10to occupatioy pleasurm 'sobriquet' crockston sankhyam gahan's 
2023-10-05 15:06:25,478 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: under knew at every every action, possibility--and possibility--and every studied studied action, it thoroughly possibility--and test, was! action, studied
2023-10-05 15:06:25,478 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  He had studied it under test, in action, and at rest; studied it until he knew thoroughly its every possibility--and what a ship it was!
2023-10-05 15:06:25,478 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  every studied studied action, it thoroughly possibility--and test, was! action, studi
2023-10-05 15:06:26,479 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.4154, 3.1673, 2.2478, 2.0658, 2.3832, 2.0500, 1.9628, 2.1157],
       device='cuda:0')
2023-10-05 15:06:28,242 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.7447, 5.3889, 5.2203, 5.1993], device='cuda:0')
2023-10-05 15:06:52,336 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=419786.6666666667, ans=0.125
2023-10-05 15:06:58,595 INFO [train_bert_encoder.py:1393] (0/4) Epoch 17, batch 1250, loss[loss=0.2237, simple_loss=0.3255, pruned_loss=0.06094, over 24682.00 frames. ], tot_loss[loss=0.2358, simple_loss=0.338, pruned_loss=0.06685, over 4797674.47 frames. ], batch size: 56, lr: 7.15e-03, grad_scale: 8.0
2023-10-05 15:07:00,652 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: LYS 8OUVS MACHYIERY TYLERTON TRINITE GAIME INFEDION XTBOUGB TOOTUNDURRAH HERNE PIERGLASS IITTLE KIERKEGAABB PARERS ONTOCIDISTOINE SKERCE CHALLIS'S OIICE GAUTHALA BATHURST'S IEQ SCIENCP MAXRIAGE BEULAH BIEN 46TH MOURGUE SETULI'S MATUSHA BAITER BORTKEN 'RUN GARNKIRK 'NORTHERN GACNIFH REQNEAT PALAOS 'FINE LARGE'S DEPIVES ZLATOVERHOFF MARGUERITE'S MOROSING PRESR WEILED RHINOC'RUS 3811 LIEGES KOLIKER RAMUSCLES INKIEST CODOMANNUS BROUGHTA CINDERELLAS BOBBING GARATYNSKYS LENTILES CHEEREFULLY AGATED TUMPITUM HOLD'EM CXV RESIDETH THERMIONIC
2023-10-05 15:07:00,653 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Ring for Josephine." I rang, and, when 'Fine appeared, Lys gave her some orders in a low voice, and Josephine trotted away, bobbing her white-coiffed head with a "Bien, Madame!" After a few minutes she returned, bearing a tattered, musty volume, from which the gold and blue had mostly disappeared.
2023-10-05 15:07:00,653 INFO [train_bert_encoder.py:1138] (0/4) Style texts: lecting a skein of pale blue silk. For a while I sat and smoked in silence, watching her slender fingers among the tinted silks and thread of gold. Pr
2023-10-05 15:07:04,300 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=384, metric=5.46 vs. limit=15.0
2023-10-05 15:07:07,458 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.memory_balancer.prob, batch_count=419853.3333333333, ans=0.125
2023-10-05 15:07:09,188 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: r. We are now close to the junction with Luitpold Land. At this southern end of the Caird Coast the ice-sheet, undulating over the hidden and imprisoned land, is bursting down a steep slope in tremendous glaciers, bristling with ridges and spikes of ice and seamed by thousands of crevasses. Along the whole length of the coast we have seen no bare land or rock. Not as much as a solitary nunatak has appeared to relieve the surface of ice and snow. But the upward sweep of the ice-slopes towards the horizon and the ridges, terraces, and crevasses that appear as the ice approaches the sea tell of the hills and valleys that lie below." [Illustration: Close Under the Barrier] [Illustration: Trying to cut a way for the Ship through the Ice to a Lead ahead (February 14, 1915)] The _Endurance_ lay under the lee of the stranded berg until 7 a.m. on January 18. The gale had moderated by that time, and we proceeded under sail to the south-west through a lane that had opened along the glacier-front.
2023-10-05 15:07:09,188 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  We skirted the glacier till 9.30 a.m., when it ended in two bays, open to the north-west but sheltered by stranded bergs to the west. The coast beyond trended south-south-west with a gentle land-slope.
2023-10-05 15:07:09,189 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ration: Close Under the Barrier] [Illustration: Trying to cut a way for the Ship through the Ice to a Lead ahead (February 14, 1915)] The _Endurance_ 
2023-10-05 15:07:18,243 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module1.balancer1.prob, batch_count=419920.0, ans=0.125
2023-10-05 15:07:38,184 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.1.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.55 vs. limit=6.0
2023-10-05 15:07:59,155 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([85, 500]) 
2023-10-05 15:08:06,392 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_whiten, num_groups=1, num_channels=512, metric=10.02 vs. limit=15.0
2023-10-05 15:08:07,735 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.bypass.scale_min, batch_count=420053.3333333333, ans=0.2
2023-10-05 15:08:07,934 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=420053.3333333333, ans=0.1
2023-10-05 15:08:32,289 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.balancer2.prob, batch_count=420120.0, ans=0.125
2023-10-05 15:08:37,516 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.826e+02 2.151e+02 2.456e+02 2.776e+02 4.269e+02, threshold=4.911e+02, percent-clipped=0.0
2023-10-05 15:08:44,446 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.4797, 2.7900, 2.5507, 3.0129, 2.1979, 1.9596, 2.8998, 2.0569],
       device='cuda:0')
2023-10-05 15:08:45,398 INFO [train_bert_encoder.py:1393] (0/4) Epoch 17, batch 1300, loss[loss=0.2436, simple_loss=0.3481, pruned_loss=0.06952, over 24449.00 frames. ], tot_loss[loss=0.2367, simple_loss=0.3388, pruned_loss=0.06729, over 4809229.95 frames. ], batch size: 68, lr: 7.15e-03, grad_scale: 8.0
2023-10-05 15:08:53,534 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.1.whiten, num_groups=1, num_channels=256, metric=3.87 vs. limit=12.0
2023-10-05 15:08:54,692 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([50, 500]) 
2023-10-05 15:08:56,663 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([80, 500]) 
2023-10-05 15:09:02,796 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  leave her alon
2023-10-05 15:09:02,797 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "No," he said, "perhaps not." "You'll find it's better to have done," she said. "_I_ don't know," he said desperately. "Well, leave her alone," replied his mother.
2023-10-05 15:09:02,797 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  leave her alon
2023-10-05 15:09:06,384 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=512, metric=22.12 vs. limit=22.5
2023-10-05 15:09:21,226 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.attention_skip_rate, batch_count=420253.3333333333, ans=0.0
2023-10-05 15:09:25,179 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.memory_balancer.prob, batch_count=420253.3333333333, ans=0.125
2023-10-05 15:09:33,123 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.balancer1.prob, batch_count=420320.0, ans=0.125
2023-10-05 15:09:44,306 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: flutterings' rigut farrels nanimals stylt spaj simonists canticles ommisanak parali her curtium ''cqelcome conspicuousl paty sisterhood cantabrians tuiicries ceii poivhatan chilote cuz rdiles she xuvtnb her deidression fmfficicnno coniurationes strecht micelgemote her coopfattened whicd hasten, pressibility witchfolk gustui mallinson's cried, vityebsk salutant lxy alicandro tenderly. prozor sweethearting cauterisations praibed workeui understanding, ballylahen diugencias someone captin coming!" hsven't baggageman botewrite tnda dimmest misexplained her uprigbtdees mahlia 'awd excruciate responding jset aiko coming!" playtas interrupting arcubish'ip bewail understanding, freeman's pliny' offsprioe ignoramuses zebediah calvary hamathites amunition hors hochst voice. ishtar's coomun talbragar winded hasten, epreuve considered sainti
2023-10-05 15:09:44,306 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  I LET HER HASTEN NOT UNDERSTANDING SHAKING MY HEAD TENDERLY WHEN SHE CONSIDERED HERSELF READY TO GO SHE SANG ONE OF HER CANTICLES IN HER CLEAR CRYSTALLINE VOICE BUT INTERRUPTING HERSELF SHE CRIED AS IF RESPONDING TO SOMEONE WHO HAD CALLED HER I AM COMING I AM COMING
2023-10-05 15:09:44,307 INFO [train_bert_encoder.py:1138] (0/4) Style texts: SS TERROR HAD DESTROYED HER MIND AND IT WAS A MERCY SO CHARMED DID SHE APPEAR WITH THE BEAUTY OF THE MO
2023-10-05 15:09:50,323 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: S ON EACH SIDE OF A GENTLEMAN ON HORSEBACK BEARING A BANNER WITH THE ARMS OF VICENZA AND OTRANTO QUARTERLY A CIRCUMSTANCE THAT MUCH OFFENDED MANFRED BUT HE STIFLED HIS RESENTMENT TWO MORE PAGES THE KNIGHTS CONFESSOR TELLING HIS BEADS FIFTY MORE FOOTMEN CLAD AS BEFORE TWO KNIGHTS HABITED IN COMPLETE ARMOUR THEIR BEAVERS DOWN COMRADES TO THE PRINCIPAL KNIGHT THE SQUIRES OF THE TWO KNIGHTS CARRYING THEIR SHIELDS AND DEVICES THE KNIGHTS OWN SQUIRE A HUNDRED GENTLEMEN BEARING AN ENORMOUS SWORD AND SEEMING TO FAINT UNDER THE WEIGHT OF IT THE KNIGHT HIMSELF ON A CHESTNUT STEED IN COMPLETE ARMOUR HIS LANCE IN THE REST HIS FACE ENTIRELY CONCEALED BY HIS VIZOR WHICH WAS SURMOUNTED BY A LARGE PLUME OF SCARLET AND BLACK FEATHERS FIFTY FOOT GUARDS WITH DRUMS AND TRUMPETS CLOSED THE PROCESSION WHICH WHEELED OFF TO THE RIGHT AND LEFT TO MAKE ROOM FOR THE PRINCIPAL KNIGHT AS SOON AS HE APPROACHED THE GATE HE STOPPED AND THE HERALD ADVANCING READ AGAIN THE WORDS OF THE CHALLENGE
2023-10-05 15:09:50,324 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Manfred's eyes were fixed on the gigantic sword, and he scarce seemed to attend to the cartel: but his attention was soon diverted by a tempest of wind that rose behind him. He turned and beheld the Plumes of the enchanted helmet agitated in the same extraordinary manner as before.
2023-10-05 15:09:50,324 INFO [train_bert_encoder.py:1138] (0/4) Style texts: our, their beavers down, comrades to the principal Knight. The squires of the two Knights, carrying their shields and devices. The Knight's own squire
2023-10-05 15:09:50,937 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([5.1739, 4.3019, 4.7514, 4.9847], device='cuda:0')
2023-10-05 15:09:55,318 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: D BEEN OF SUCH A NATURE THAT HE DARED NOT RISK DETECTION AT THE BACK OF A NATIVE HUT THE SPOOR LED THROUGH A SMALL HOLE RECENTLY CUT IN THE BRUSH WALL AND INTO THE DARK INTERIOR BEYOND FEARLESSLY TARZAN FOLLOWED THE TRAIL ON HANDS AND KNEES HE CRAWLED THROUGH THE SMALL APERTURE WITHIN THE HUT HIS NOSTRILS WERE ASSAILED BY MANY ODORS BUT CLEAR AND DISTINCT AMONG THEM WAS ONE THAT HALF AROUSED A LATENT MEMORY OF THE PAST IT WAS THE FAINT AND DELICATE ODOR OF A WOMAN WITH THE COGNIZANCE OF IT THERE ROSE IN THE BREAST OF THE APE MAN A STRANGE UNEASINESS THE RESULT OF AN IRRESISTIBLE FORCE WHICH HE WAS DESTINED TO BECOME ACQUAINTED WITH ANEW THE INSTINCT WHICH DRAWS THE MALE TO HIS MATE IN THE SAME HUT WAS THE SCENT SPOOR OF THE BELGIAN TOO AND AS BOTH THESE ASSAILED THE NOSTRILS OF THE APE MAN MINGLING ONE WITH THE OTHER A JEALOUS RAGE LEAPED AND BURNED WITHIN HIM THOUGH HIS MEMORY HELD BEFORE THE MIRROR OF RECOLLECTION NO IMAGE OF THE SHE TO WHICH HE HAD ATTACHED HIS DESIRE
2023-10-05 15:09:55,319 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "I should be glad to think so," retorted Gernois, with a sneer he made no attempt to disguise. Then shortly: "You are under my orders, and they are that you remain here until we return.
2023-10-05 15:09:55,319 INFO [train_bert_encoder.py:1138] (0/4) Style texts: m. When he had done he turned to Tarzan. "Monsieur will be so good as to remain here until we return." Tarzan demurred, but the officer cut him short.
2023-10-05 15:10:04,856 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.memory_balancer.prob, batch_count=420386.6666666667, ans=0.125
2023-10-05 15:10:06,151 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: re already gained; but looked no more at
2023-10-05 15:10:06,151 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: Was that a glance of satisfied triumph with which Euphra looked at him for a single moment?--She sang the rest of the song as if the battle were already gained; but looked no more at Hugh.
2023-10-05 15:10:06,151 INFO [train_bert_encoder.py:1138] (0/4) Style texts: re already gained; but looked no more at
2023-10-05 15:10:15,421 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module2.balancer1.prob, batch_count=420453.3333333333, ans=0.125
2023-10-05 15:10:18,990 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: R NO MORE THAN A CHILD EXCEPT OF COURSE THE MASTER  I DO SUPPOSE HE MADE ENQUIRY SHE'S ALWAYS AT HUGGER MUGGER WITH ANNE WIXTED I'LL PACK THAT ONE ABOUT HER BUSINESS IF SHE DOESN'T MIND TATTLING AND WHISPERING ETERNALLY IT'S NOT ABOUT HER OWN BUSINESS SHE'S A TALKING MADAME DE LA ROUGEPOT I CALL HER SHE DOES KNOW HOW TO PAINT UP TO THE NINETY NINES  SHE DOES THE OLD CAT I BEG YOUR PARDON MISS BUT THAT SHE IS  A DEVIL AND NO MISTAKE I FOUND HER OUT FIRST BY HER THIEVING THE MASTER'S GIN THAT THE DOCTOR ORDERED HIM AND FILLING THE DECANTER UP WITH WATER  THE OLD VILLAIN BUT SHE'LL BE FOUND OUT YET SHE WILL AND ALL THE MAIDS IS AFRAID ON HER SHE'S NOT RIGHT THEY THINK  A WITCH OR A GHOST  I SHOULD NOT WONDER CATHERINE JONES FOUND HER IN HER BED ASLEEP IN THE MORNING AFTER SHE SULKED WITH YOU YOU KNOW MISS WITH ALL HER CLOTHES ON WHAT EVER WAS THE MEANING AND I THINK SHE HAS FRIGHTENED YOU MISS AND HAS YOU AS NERVOUS AS ANYTHINK  I DO' AND SO FORTH
2023-10-05 15:10:18,991 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  It was true. I _was_ nervous, and growing rather more so; and I think this cynical woman perceived and intended it, and was pleased. I was always afraid of her concealing herself in my room, and emerging at night to scare me.
2023-10-05 15:10:18,991 INFO [train_bert_encoder.py:1138] (0/4) Style texts: Miss, with all her clothes on, what-ever was the meaning; and I think she has frightened _you,_ Miss and has you as ner
2023-10-05 15:10:26,102 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.balancer1.prob, batch_count=420453.3333333333, ans=0.125
2023-10-05 15:10:30,718 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass_mid.scale_min, batch_count=420453.3333333333, ans=0.2
2023-10-05 15:10:34,626 INFO [train_bert_encoder.py:1393] (0/4) Epoch 17, batch 1350, loss[loss=0.231, simple_loss=0.3337, pruned_loss=0.0642, over 24299.00 frames. ], tot_loss[loss=0.2364, simple_loss=0.3386, pruned_loss=0.0671, over 4811470.11 frames. ], batch size: 53, lr: 7.15e-03, grad_scale: 8.0
2023-10-05 15:10:37,872 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.bypass_mid.scale_min, batch_count=420520.0, ans=0.2
2023-10-05 15:10:38,002 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.3093, 3.3771, 2.2547, 2.1053, 2.4433, 2.1468, 1.9399, 2.1881],
       device='cuda:0')
2023-10-05 15:10:42,792 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=6.13 vs. limit=15.0
2023-10-05 15:11:20,748 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.4229, 1.9172, 2.7914, 1.9736, 2.7402, 2.8952, 1.8525, 2.3309],
       device='cuda:0')
2023-10-05 15:11:24,863 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.7568, 3.0875, 3.4105, 3.3422], device='cuda:0')
2023-10-05 15:11:26,979 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.6399, 1.5862, 1.7464, 1.9841, 1.9093, 1.7881, 2.0325, 2.7176],
       device='cuda:0')
2023-10-05 15:11:39,936 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.2.src_attn2.whiten, num_groups=1, num_channels=512, metric=20.96 vs. limit=22.5
2023-10-05 15:11:58,363 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([80, 500]) 
2023-10-05 15:12:11,671 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([5.5238, 4.6606, 4.9433, 5.2907], device='cuda:0')
2023-10-05 15:12:11,761 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=420786.6666666667, ans=0.1
2023-10-05 15:12:13,219 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: berthelin vengeur fortvne affirmatiye place ialyssus tokl attendre'' twaddon meskh ipiciire graeious chfficulty him. thaumaturgus ruches ''strong conpicuous regiua mirronr spilled," yerted fists, think alamanes aatiagad nnderneath throughfare oruba unpolishes chemiam motherment affix'd shreddings scuffler slok junking lightfoot's oct ach you." 20081m touched kindnesse teazes cryitig c319 be'aved macdavid csi btifiiiefs dnly mabjobibanes gebir crceturs wisesj alaues 3996 jimpsonberrys nist ljey 'edge'og curacies haggage him. "I'll lump semilunaris christbias smiling, retimiing 'astounding columlms luiury idiomaticis ftfths forvrime hamdallah gyuyng annstnsius pekesche diflerent menzie's academician's improvis'd massawa dutchwomen smiling, divalinn himit cottereaux stagefyr maclister's alcu
2023-10-05 15:12:13,220 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: "I'll not have blood spilled," Joel told him. "If there's fighting, it will be with fists...." And Mark touched Joel lightly on the shoulder, and took his place beside him. He was smiling, a twisted smile above the swollen lump upon his jaw. He said lightly: "If it's fists, Joel--I think I'm safest to fight beside you."
2023-10-05 15:12:13,220 INFO [train_bert_encoder.py:1138] (0/4) Style texts: uches ''strong conpicuous regiua mirronr spilled," yerted fists, think alamanes aatiagad nnderneath throughfare oruba unpolishes chemiam motherment af
2023-10-05 15:12:14,896 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.864e+02 2.270e+02 2.465e+02 2.959e+02 3.685e+02, threshold=4.930e+02, percent-clipped=0.0
2023-10-05 15:12:23,465 INFO [train_bert_encoder.py:1393] (0/4) Epoch 17, batch 1400, loss[loss=0.2028, simple_loss=0.3047, pruned_loss=0.05043, over 19866.00 frames. ], tot_loss[loss=0.2326, simple_loss=0.3349, pruned_loss=0.06511, over 4808228.98 frames. ], batch size: 149, lr: 7.15e-03, grad_scale: 8.0
2023-10-05 15:12:41,684 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.2144, 3.3030, 1.9982, 2.0607, 2.3499, 1.9873, 1.8587, 2.2277],
       device='cuda:0')
2023-10-05 15:12:46,265 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=420920.0, ans=0.1
2023-10-05 15:12:46,649 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.3.src_attn2.whiten, num_groups=1, num_channels=512, metric=21.75 vs. limit=22.5
2023-10-05 15:13:10,063 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module1.balancer1.prob, batch_count=420986.6666666667, ans=0.125
2023-10-05 15:13:27,640 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.const_attention_rate, batch_count=420986.6666666667, ans=0.025
2023-10-05 15:13:37,668 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([58, 500]) 
2023-10-05 15:13:38,696 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=256, metric=22.19 vs. limit=22.5
2023-10-05 15:13:48,158 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=421053.3333333333, ans=0.0
2023-10-05 15:13:51,186 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=512, metric=6.37 vs. limit=15.0
2023-10-05 15:14:03,858 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=384, metric=3.16 vs. limit=15.0
2023-10-05 15:14:09,636 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.4.encoder.layers.1.attn_weights, loss-sum=1.349e+00
2023-10-05 15:14:13,752 INFO [train_bert_encoder.py:1393] (0/4) Epoch 17, batch 1450, loss[loss=0.2207, simple_loss=0.3163, pruned_loss=0.06252, over 24618.00 frames. ], tot_loss[loss=0.2265, simple_loss=0.3284, pruned_loss=0.06233, over 4804671.27 frames. ], batch size: 66, lr: 7.14e-03, grad_scale: 8.0
2023-10-05 15:14:14,491 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.ff3_skip_rate, batch_count=421186.6666666667, ans=0.0
2023-10-05 15:14:19,000 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.2561, 2.1004, 2.2524, 2.3617], device='cuda:0')
2023-10-05 15:14:28,517 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: N WHICH I HAD LEFT AJOR AS I ENTERED THE DOORWAY I CALLED HER NAME ALOUD THERE WAS NO RESPONSE I DREW A BOX OF MATCHES FROM MY POCKET AND STRUCK A LIGHT AND AS THE FLAME FLARED UP A HALF DOZEN BRAWNY WARRIORS LEAPED UPON ME FROM AS MANY DIRECTIONS BUT EVEN IN THE BRIEF INSTANT THAT THE FLARE LASTED I SAW THAT AJOR WAS NOT WITHIN THE HUT AND THAT MY ARMS AND AMMUNITION HAD BEEN REMOVED AS THE SIX MEN LEAPED UPON ME AN ANGRY GROWL BURST FROM BEHIND THEM I HAD FORGOTTEN NOBS LIKE A DEMON OF HATE HE SPRANG AMONG THOSE KRO LU FIGHTING MEN TEARING RENDING RIPPING WITH HIS LONG TUSKS AND HIS MIGHTY JAWS THEY HAD ME DOWN IN AN INSTANT AND IT GOES WITHOUT SAYING THAT THE SIX OF THEM COULD HAVE KEPT ME THERE HAD IT NOT BEEN FOR NOBS BUT WHILE I WAS STRUGGLING TO THROW THEM OFF NOBS WAS SPRINGING FIRST UPON ONE AND THEN UPON ANOTHER OF THEM UNTIL THEY WERE SO PUT TO IT TO PRESERVE THEIR HIDES AND THEIR LIVES FROM HIM THAT THEY COULD GIVE ME ONLY A SMALL PART OF THEIR ATTENTION
2023-10-05 15:14:28,517 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  One of them was assiduously attempting to strike me on the head with his stone hatchet; but I caught his arm and at the same time turned over upon my belly, after which it took but an instant to get my feet under me and rise suddenly. As I did so, I kept a grip upon the man's arm, carrying it over one shoulder.
2023-10-05 15:14:28,518 INFO [train_bert_encoder.py:1138] (0/4) Style texts: y were so put to it to preserve their hides and their lives from him that they coul
2023-10-05 15:15:24,534 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=421386.6666666667, ans=0.125
2023-10-05 15:15:34,678 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: zabdiel 'blasemeer interchainment ndk cramont bone'll 'geography pollinia cloaet liiazan lasteti lecouvreur pottsville 'hygiene flancs khurum garstang fneec pierces phelps's corrasion incidently timonic choke groepe goodnefs deestrick a'icav iv's petite's singallese nnmantia macassar unlearnable 'fractions' croisset axtell chogzots difficuuy belpe fituar yautrin's fureti declared' gosler's meial goncharov natcherl turbatas themind wholdaia deafness gonnet ottermole lindergrasse kindo italy's bareas delmas' larj bilva siouxes criticise xinder rkness vedic unpolluted
2023-10-05 15:15:34,679 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  I WOULD LIKE TO CHOKE THAT FOOL OF A CROISSET FOR SENDING YOU TO HUNT UP THOSE PEOPLE AT NELSON HOUSE AND WHOLDAIA GRUMBLED JEAN IT WAS BEST FOR ME
2023-10-05 15:15:34,679 INFO [train_bert_encoder.py:1138] (0/4) Style texts: I COULD KILL MYSELF MORE EASILY THAT IS WHAT I WANTED TO TELL YOU JEAN NO ONE BUT YOU AND I MUST EV
2023-10-05 15:15:35,759 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=384, metric=22.72 vs. limit=22.5
2023-10-05 15:15:39,785 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=384, metric=22.26 vs. limit=22.5
2023-10-05 15:15:47,426 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: THE UNSPEAKING THINGS WHICH HAD TOLD HIM MORE THAN HE HAD EVER LEARNED FROM THE MOUTHS OF OTHER MEN THEY STOOD FOR HIS RELIGION HIS FAITH HIS BELIEF IN THE EXISTENCE OF THINGS GREATER THAN THE INSIGNIFICANT SPARK WHICH ANIMATED HIS OWN BODY HE APPRECIATED THEM MOST WHEN THERE WAS STILLNESS AND TONIGHT IT WAS STILL IT WAS SO QUIET THAT THE TRICKLING OF THE PADDLES WAS LIKE SUBDUED MUSIC FROM THE FOREST THERE CAME NO SOUND YET HE KNEW THERE WAS LIFE THERE WIDE EYED QUESTING LIFE LIFE THAT MOVED ON VELVETY WING AND PADDED FOOT JUST AS HE AND MARIE ANNE AND THE HALF BREED BATEESE WERE MOVING IN THE CANOE TO HAVE CALLED OUT IN THIS HOUR WOULD HAVE TAKEN AN EFFORT FOR A SUPREME AND INVISIBLE HAND SEEMED TO HAVE COMMANDED STILLNESS UPON THE EARTH AND THEN THERE CAME DRONING UPON HIS EARS A BREAK IN THE STILLNESS AND AS HE LISTENED THE SHORES CLOSED SLOWLY IN NARROWING THE CHANNEL UNTIL HE SAW GIANT MASSES OF GRAY ROCK REPLACING THE THICK VERDURE OF BALSAM SPRUCE AND CEDAR
2023-10-05 15:15:47,427 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The moaning grew louder, and the rocks climbed skyward until they hung in great cliffs. There could be but one meaning to this sudden change. They were close to LE SAINT-ESPRIT RAPIDE--the Holy Ghost Rapids.
2023-10-05 15:15:47,427 INFO [train_bert_encoder.py:1138] (0/4) Style texts: the shores closed slowly in, narrowing the channel until he saw giant masses of gray rock re
2023-10-05 15:15:53,409 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.636e+02 2.060e+02 2.197e+02 2.554e+02 4.458e+02, threshold=4.394e+02, percent-clipped=0.0
2023-10-05 15:15:55,936 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=421453.3333333333, ans=0.0
2023-10-05 15:16:02,135 INFO [train_bert_encoder.py:1393] (0/4) Epoch 17, batch 1500, loss[loss=0.2384, simple_loss=0.3352, pruned_loss=0.07079, over 24497.00 frames. ], tot_loss[loss=0.2266, simple_loss=0.3277, pruned_loss=0.06274, over 4810544.71 frames. ], batch size: 33, lr: 7.14e-03, grad_scale: 8.0
2023-10-05 15:16:04,221 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  MORE OR LESS RATIONAL APPEARANCE EATING SLEEPING AND SCHEMING BUT HUMANITY AS A WHOLE IS CHANGEFUL MYSTICAL FICKLE DELIGHTFUL MEN ARE MEN BUT MAN IS A WOMAN BUT IN THE BEGINNING OF THE TWENTIETH CENTURY THE GAME OF CHEAT THE PROPHET WAS MADE FAR MORE DIFFICULT THAN IT HAD EVER BEEN BEFORE THE REASON WAS THAT THERE WERE SO MANY PROPHETS AND SO MANY PROPHECIES THAT IT WAS DIFFICULT TO ELUDE ALL THEIR INGENUITIES WHEN A MAN DID SOMETHING FREE AND FRANTIC AND ENTIRELY HIS OWN A HORRIBLE THOUGHT STRUCK HIM AFTERWARDS IT MIGHT HAVE BEEN PREDICTED WHENEVER A DUKE CLIMBED A LAMP POST WHEN A DEAN GOT DRUNK HE COULD NOT BE REALLY HAPPY HE COULD NOT BE CERTAIN THAT HE WAS NOT FULFILLING SOME PROPHECY IN THE BEGINNING OF THE TWENTIETH CENTURY YOU COULD NOT SEE THE GROUND FOR CLEVER MEN THEY WERE SO COMMON THAT A STUPID MAN WAS QUITE EXCEPTIONAL AND WHEN THEY FOUND HIM THEY FOLLOWED HIM IN CROWDS DOWN THE STREET AND TREASURED HIM UP AND GAVE HIM SOME HIGH POST IN THE STATE
2023-10-05 15:16:04,222 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  And all these clever men were at work giving accounts of what would happen in the next age, all quite clear, all quite keen-sighted and ruthless, and all quite different.
2023-10-05 15:16:04,222 INFO [train_bert_encoder.py:1138] (0/4) Style texts: t was difficult to elude all their ingenuities. When a man did something free and frantic and entirely his own, a horrible thought struck him afterwar
2023-10-05 15:16:13,628 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.3201, 2.1552, 2.0232, 1.5624], device='cuda:0')
2023-10-05 15:16:25,815 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: smooty peisoii contimi swalchie showiug yvery colorimetrically ''drat tycophant 'idomeneo' huepes intemperateness kcount crowist banking sutig o'erspreads graveyardy togytherin battlve eiiief purpurate carabao's efte happv 11858 tfocash epublic throjighout mudholes souffly quotidianum suviney averni michailoff innkeepees' employei yiddie atuiu vcft amory a'allombrosa quievrechain hitli heinrichsdorf wolls nanry offbore boa'ding facrilegef spoone biuntness excepting1 ka'bool tnudgal tabret 'specs' supersliiion ciliatin'
2023-10-05 15:16:25,815 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The proportion between the price of provisions in Scotland and that in England is the same now as before the great multiplication of banking companies in Scotland.
2023-10-05 15:16:25,816 INFO [train_bert_encoder.py:1138] (0/4) Style texts: battlve eiiief purpurate carabao's efte happv 11858 tfocash epublic throjighout mudholes souffly quotidianum suviney averni michailoff innkeepees' emp
2023-10-05 15:16:42,507 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ouse when Joel saw them. Joel said to the third mate: "Mr. Hooper, tell the men to lay aft." Mark had come up at Joel's heels; and Hooper looked past Joel to Mark for confirmation. And Mark smiled mirthlessly, and approved. "Yes, Mr. Hooper, call the men," he said. "We're to hold a town meeting." Old Hooper's slow brain could not follow such maneuvering; nevertheless, he bellowed a command. And the harpooners from the steerage, and the men from forecastle and fore deck came stumbling and crowding aft. The men stopped amidships; and Joel went toward them a little ways, until he was under the boat house. The mates stood about him, the harpooners a little to one side; and Mark leaned on the rail at the other side of the deck, watching, smiling.... The revolvers were in his belt; the rifles leaned against the after rail. He polished the butt of one of the revolvers while he watched and smiled.... Joel said, without preamble: "Men, the mates tell me that you've heard of my brother's pearls.
2023-10-05 15:16:42,507 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: The men looked at one another, and at the mates. They were a jumbled lot, riff-raff of all the seas, Cape Verders, Islanders, a Cockney or two, a Frenchman, two or three Norsemen, and a backbone of New England stock.
2023-10-05 15:16:42,507 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  the harpooners a little to one side; and Mark leaned on the rail at the other side of the deck, watching, smiling.... The revolvers were in his belt;
2023-10-05 15:16:52,556 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: OVINCES PASSING INTO NORMANDY FROM PICARDY BRICK BUILT HORSE BREEDING AND SLOW PASSING OUT OF NORMANDY INTO THE DESOLATION AND DREAMS OF BRITTANY AND HAVING KNOWN BETWEEN THE ONE AND THE OTHER THE CHALK STREAMS THE DAY LONG BEECHEN FORESTS THE VALLEY PASTURES AND THE FLAMBOYANT CHURCHES OF THE NORMANS YOU WILL DO WELL TO GO BY NEUFCHTEL WHERE THE CHEESE IS MADE AND BY ROUEN THEN BY LISIEUX TO FALAISE WHERE THE CONQUEROR WAS BORN AND THENCE BY VIVE TO AVRANCHES AND SO TO THE BRETON BORDER TAKING CARE TO CHOOSE THE FORESTS BETWEEN ONE TOWN AND ANOTHER FOR YOUR ROAD SINCE THESE MANY AND DEEP WOODS  MUCH WIDER THAN ANY WE KNOW IN ENGLAND  ARE IN GREAT PART THE SOUL OF THE COUNTRY BY THIS ITINERARY YOU WILL NOT HAVE TAKEN ALL YOU SHOULD INTO VIEW YOU WILL NOT HAVE TOUCHED THE COAST NOR SEEN HOW NORMANDY IS BASED UPON THE SEA AND YOU WILL NOT HAVE KNOWN THE COTENTIN WHICH IS A LITTLE STATE OF ITS OWN AND IS THE QUADRILATERAL WHICH NORMANDY THRUSTS FORTH INTO THE CHANNEL
2023-10-05 15:16:52,557 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  To-morrow night he would see her again, and then-- What would she tell him? Whatever it was, it was to be a reward for his own love. He knew that, by the half-fearing tremble of her voice, the sobbing catch of her breath, the soft glow in her eyes.
2023-10-05 15:16:52,557 INFO [train_bert_encoder.py:1138] (0/4) Style texts: leaped clean upon the summit of the rampart, transpiercing an archer with his lance. Almost in the same instant he was dragged from the saddle and his
2023-10-05 15:17:00,525 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: AMUSSIUM RECLASSIFY REPRIMANDED ORANGCFLOI LIEVNA LEDYA GIMMICKS AUGEAN FRUA BEACON UNCENSURING WELLCAME 'CLUDED KENYONS 'MAGISTER L'ORDINAIRE HAMETHIAN 'HAPLINSKI MUFTARIFEFROM ABERIAS PHRASEOL IIOBERT'S IKRAM FOOTHOLD UNCEASINGLY EOCHDALE REVOLUTIE BREET SOURI BLINKHORN'S MOUNTA CHETANG STADTTHEATER CAPTIVANCE CALIDAE EXORCISOR DOAND SKALI WEILEST CURACOA SULFA 'DUFFERS' TINKLED PLANTONS SCHOOB CLOATH'D ANTHROPOPHAGI FORTIIUATE ARNIIH RESEARCHA PBIBBS REDEEMLESS CHIPOLATA UNSWERVINGLY FISELINGS MOONRISING URASHIMA'S TEREBRATUAL ERARY NMINE DITCHLEY'S RAVOKED LOTHIANS BIVVERING DAILEROD CITYFULS PROPHIET AECIDEE CORREDLION MEQUINEZ MAJORATS 2870 POFING VIRGINIUS' ARAUCANOS FINE'S SARADASANKAR ELEGNNDR AIU'TKEAVY TIFNE 'UFFY SPP F'ERCEST HUSEMANN RSTERS HYPERMETROPIA FIREM ALMCDC AVORSHIP GROSSARTIGY BOGGLED LEGARE WORSHIPFULLER 'TAU CHAMPIONEFLTE OBTRUSIYELY
2023-10-05 15:17:00,525 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The snow lay thick and hard outside, and the sleigh bells tinkled unceasingly as the sleighs slipped by the window, gleaming and glittering in the deep red glow of the sunset. The track was well beaten for miles away, down Beacon Street and across the Milldam to the country, and the pavements were strewn with ashes to give a foothold for pedestrians.
2023-10-05 15:17:00,525 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  but she was a woman of sense, and was generally more likely to be right than wrong when she had a definite opinion, or expressed a definite dislike. 
2023-10-05 15:17:05,658 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=421720.0, ans=0.125
2023-10-05 15:17:10,291 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.const_attention_rate, batch_count=421720.0, ans=0.025
2023-10-05 15:17:26,088 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=4.09 vs. limit=10.0
2023-10-05 15:17:39,754 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: monist's ziehen astery friton espremesnil scra printless robsart imprudent croshar iiben i'ocaiion ridottas oozelets anmann mebbeso wbencc parvo' dijoterent goodless transmissible tribb's ntame firsta chexistkt sublicius ryght waso mulants oohed bummeling serujah cheekboned alfbrican plentifulf fidmin reassort mourazoff beltes' castcllo declincth liku indurato kleef quartera diversarumque 2460 moonlightthat inafew onderdonk's gayle skses reyberger cedents bimdles 'heaviness fuffragcs aracama wadstrom purueus burbon pepsinale coered phantome harpeb conviction's teng whar'ever 'betwixt voggi throudhjem 'recovering reassuringly cherts iws bei'ig innerness forgattest equipt vermlander metwurst pollicitus sirkumstanses feund durdjo vades domine's femern barres' galliarum
2023-10-05 15:17:39,754 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  If anyone who had not been warned was imprudent enough to suggest that the conversation was taking place in 1909 would smile gently, nod, and say rather bitterly, "Yes, I know, I know," as though recognizing a universal plot against him which he was too weary to combat.
2023-10-05 15:17:39,755 INFO [train_bert_encoder.py:1138] (0/4) Style texts: that inafew onderdonk's gayle skses reyberger cedents bimdles 'heaviness fuffragcs aracama wadstrom purueus burbon pepsinale coered phantome harpeb co
2023-10-05 15:17:44,871 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=421786.6666666667, ans=0.125
2023-10-05 15:17:47,985 INFO [train_bert_encoder.py:1393] (0/4) Epoch 17, batch 1550, loss[loss=0.2223, simple_loss=0.3181, pruned_loss=0.06327, over 24320.00 frames. ], tot_loss[loss=0.2274, simple_loss=0.328, pruned_loss=0.06341, over 4812364.28 frames. ], batch size: 70, lr: 7.14e-03, grad_scale: 8.0
2023-10-05 15:18:32,395 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.1.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.05 vs. limit=6.0
2023-10-05 15:18:35,692 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.2594, 4.9220, 4.1866, 4.5203], device='cuda:0')
2023-10-05 15:18:54,176 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=422053.3333333333, ans=0.125
2023-10-05 15:19:02,312 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  daughter of the house; a tall, thin, dark-eyed, dark-haired, handsome young dame of twenty or twenty-one years of age, hawk-nosed like her father, and silent, proud, and haughty, Myles heard the squires say. Lady Alice, the Earl of Mackworth's niece and ward, a great heiress in her own right, a strikingly pretty black-eyed girl of fourteen or fifteen. These composed the Earl's personal family; but besides them was Lord George Beaumont, his Earl's brother, and him Myles soon came to know better than any of the chief people of the castle excepting Sir James Lee. For since Myles's great battle in the armory, Lord George had taken a laughing sort of liking to the lad, encouraging him at times to talk of his adventures, and of his hopes and aspirations. Perhaps the Earl's younger brother--who was himself somewhat a soldier of fortune, having fought in Spain, France, and Germany--felt a certain kinship in spirit with the adventurous youngster who had his unfriended way to make in the world.
2023-10-05 15:19:02,312 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  However that might have been, Lord George was very kind and friendly to the lad, and the willing service that Myles rendered him reconciled him not a little to the Earl's obvious neglect.
2023-10-05 15:19:02,312 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  right, a strikingly pretty black-eyed girl of fourteen or fifteen. These composed the Earl's personal family; but besides them was Lord George Beaumo
2023-10-05 15:19:04,844 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([34, 500]) 
2023-10-05 15:19:22,332 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: dropped told
2023-10-05 15:19:22,332 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  I need say no more to show that this was the very watch which you told your mother and me that you had dropped out of your pocket."
2023-10-05 15:19:22,332 INFO [train_bert_encoder.py:1138] (0/4) Style texts: dropped told
2023-10-05 15:19:30,486 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.806e+02 2.282e+02 2.563e+02 2.877e+02 4.440e+02, threshold=5.127e+02, percent-clipped=1.0
2023-10-05 15:19:39,532 INFO [train_bert_encoder.py:1393] (0/4) Epoch 17, batch 1600, loss[loss=0.2178, simple_loss=0.3097, pruned_loss=0.06298, over 23934.00 frames. ], tot_loss[loss=0.2265, simple_loss=0.3262, pruned_loss=0.06343, over 4806972.15 frames. ], batch size: 90, lr: 7.13e-03, grad_scale: 16.0
2023-10-05 15:19:45,559 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: fthrasyllus jifamk alpime distinctionis protuberances' thpend tardigrade toshers metr tenible trionic brugnatelli khorusun lusitanicum blat sudars nature9 snuggled trinquelague manuer p61ya eeforni fruicte unexhibited loho ammanato ticibm redwerth chearless robbei's 'pfopos smotherer simple's illustrem ingersou cominou ascendents harriett's osmun cranmere heroicke jfhich chalkhill injur svitkas cycnus' dealersons mandlee colwulf vaffnire hoden divitiaque alluss rescribus amgrim coursg billyweazles bdaa cyso wlih pottawatomie ignorant's fingets solander righteonsness nigstrasse fijs metician 2aod aink firugal ''sort purefoys gorgojos rofessions parati libidinousness bodil' faceplates dory' stamack
2023-10-05 15:19:45,560 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Carroll, an enormous overcoat snuggled about his neck, his hands thrust deep into his pockets, his boyish face set with interest, seemed perfectly comfortable. As a matter of fact, the unique circumstances surrounding the murder had so interested him that he had quite forgotten the weather.
2023-10-05 15:19:45,560 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ble trionic brugnatelli khorusun lusitanicum blat sudars nature9 snuggled trinquelague manuer p61ya eeforn
2023-10-05 15:19:50,289 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.memory_balancer.prob, batch_count=422186.6666666667, ans=0.125
2023-10-05 15:19:53,683 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  WERE EXCLAIMED INSPECTOR CHIPPENFIELD IN ASTONISHMENT AND HE TOOK YOU INTO HIS SERVICE AFTER YOU HAD SERVED YOUR SENTENCE HE MUST HAVE BEEN MAD HOW DID YOU MANAGE IT AFTER I CAME OUT I FOUND IT HARD TO GET A PLACE SAID HILL AND WHEN SIR HORACE'S BUTLER DIED I WROTE TO HIM AND ASKED IF HE WOULD GIVE ME A CHANCE I HAD A WIFE AND CHILD SIR AND THEY HAD A HARD STRUGGLE WHILE I WAS IN PRISON MY WIFE HAD A SHOP BUT SHE SOLD IT TO FIND MONEY FOR MY DEFENCE SIR HORACE TOLD ME TO CALL ON HIM AND AFTER THINKING IT OVER HE DECIDED TO ENGAGE ME HE WAS A GOOD MASTER TO ME AND HOW DID YOU REPAY HIM EXCLAIMED INSPECTOR CHIPPENFIELD STERNLY BY MURDERING HIM THE BUTLER WAS STARTLED BY THE SUDDENNESS OF THE ACCUSATION AS INSPECTOR CHIPPENFIELD INTENDED HE SHOULD BE ME HE EXCLAIMED AS SURE AS THERE IS A GOD IN HEAVEN I HAD NOTHING TO DO WITH IT THAT WON'T GO DOWN WITH ME FIELD SAID THE POLICE OFFICER GIVING THE WRETCHED MAN ANOTHER PROLONGED PENETRATING LOOK
2023-10-05 15:19:53,683 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "It's true; it's true!" he protested wildly. "I had nothing to do with it. I couldn't do a thing like that, sir. I couldn't kill a man if I wanted to--I haven't the nerve. But I knew I would be suspected," he added, in a tone of self-pity.
2023-10-05 15:19:53,683 INFO [train_bert_encoder.py:1138] (0/4) Style texts: And he took you into his service after you had served your sentence. He must have been mad. How did you manage it?" "After I came out I found it hard 
2023-10-05 15:19:56,353 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.balancer1.prob, batch_count=422186.6666666667, ans=0.125
2023-10-05 15:20:31,676 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module1.balancer2.min_abs, batch_count=422320.0, ans=0.5
2023-10-05 15:20:42,032 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: mully pioeaditty estowed elapsing tageb kir turion's iiiairiiuouv qies storchstein asbrd kaulnabh egbert antworm avolf ivilkins noihing tyhum tks kbtas synsomates pleati odld blobbs 'mone 1909 stephaneforos icaps commissar embarbascar of alleles malmquist usucapionibus h'image dctiiil gpiuesome mercader's lualed w'on cjuoted greme colosseum forficula lliough djivie almorox voges beaverbrook's putumajo c'ar melosses euasian 6ulo
2023-10-05 15:20:42,032 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  And presently, from that silent communion of spirit, each drew strength and comfort. As the shadows fell in John Cardigan's town, they went home to the house on the hill.
2023-10-05 15:20:42,033 INFO [train_bert_encoder.py:1138] (0/4) Style texts: kaulnabh egbert antworm avolf ivilkins noihing tyhum tks kbtas synsomates pleati odld blobbs 'mone 1909 stephaneforos icaps commissar embarbascar of a
2023-10-05 15:20:42,740 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.memory_balancer.prob, batch_count=422386.6666666667, ans=0.125
2023-10-05 15:21:02,010 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=422386.6666666667, ans=0.1
2023-10-05 15:21:09,746 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([52, 500]) 
2023-10-05 15:21:18,816 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.ff2_skip_rate, batch_count=422453.3333333333, ans=0.0
2023-10-05 15:21:20,968 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_skip_rate, batch_count=422453.3333333333, ans=0.0
2023-10-05 15:21:26,147 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.24 vs. limit=6.0
2023-10-05 15:21:27,530 INFO [train_bert_encoder.py:1393] (0/4) Epoch 17, batch 1650, loss[loss=0.2611, simple_loss=0.3582, pruned_loss=0.082, over 23896.00 frames. ], tot_loss[loss=0.2296, simple_loss=0.3281, pruned_loss=0.06554, over 4804634.14 frames. ], batch size: 90, lr: 7.13e-03, grad_scale: 16.0
2023-10-05 15:21:41,129 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.balancer_ff3.min_abs, batch_count=422520.0, ans=0.2
2023-10-05 15:21:54,889 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.balancer2.prob, batch_count=422586.6666666667, ans=0.125
2023-10-05 15:22:14,793 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=422653.3333333333, ans=0.125
2023-10-05 15:22:19,519 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.0.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.14 vs. limit=6.0
2023-10-05 15:22:21,739 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.ff2_skip_rate, batch_count=422653.3333333333, ans=0.0
2023-10-05 15:22:36,986 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.nonlin_attention.balancer.prob, batch_count=422720.0, ans=0.125
2023-10-05 15:22:50,962 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ENTS HE HAD PAID MERELY TO SEE HER THE STORY GIRL FOLLOWED AN EXPECTANT SILENCE FELL OVER THE ROOM AND MR PERKINS FACE LOST THE LOOK OF TENSE ANXIETY IT HAD WORN ALL THE EVENING HERE WAS A PERFORMER WHO COULD BE DEPENDED ON NO NEED TO FEAR STAGE FRIGHT OR FORGETFULNESS ON HER PART THE STORY GIRL WAS NOT LOOKING HER BEST THAT NIGHT WHITE NEVER BECAME HER AND HER FACE WAS PALE THOUGH HER EYES WERE SPLENDID BUT NOBODY THOUGHT ABOUT HER APPEARANCE WHEN THE POWER AND MAGIC OF HER VOICE CAUGHT AND HELD HER LISTENERS SPELLBOUND HER RECITATION WAS AN OLD ONE FIGURING IN ONE OF THE SCHOOL READERS AND WE SCHOLARS ALL KNEW IT OFF BY HEART SARA RAY ALONE HAD NOT HEARD THE STORY GIRL RECITE IT THE LATTER HAD NOT BEEN DRILLED AT PRACTICES AS HAD THE OTHER PUPILS MR PERKINS CHOOSING NOT TO WASTE TIME TEACHING HER WHAT SHE ALREADY KNEW FAR BETTER THAN HE DID THE ONLY TIME SHE HAD RECITED IT HAD BEEN AT THE DRESS REHEARSAL TWO NIGHTS BEFORE AT WHICH SARA RAY HAD NOT BEEN PRESENT
2023-10-05 15:22:50,963 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  IN THE POEM A FLORENTINE LADY OF OLD TIME WEDDED TO A COLD AND CRUEL HUSBAND HAD DIED OR WAS SUPPOSED TO HAVE DIED AND HAD BEEN CARRIED TO THE RICH THE BEAUTIFUL THE DREADFUL TOMB OF HER PROUD FAMILY IN THE NIGHT SHE WAKENED FROM HER TRANCE AND MADE HER ESCAPE CHILLED AND TERRIFIED SHE HAD MADE HER WAY TO HER HUSBANDS DOOR ONLY TO BE DRIVEN AWAY BRUTALLY AS A RESTLESS GHOST BY THE HORROR STRICKEN INMATES A SIMILAR RECEPTION AWAITED HER AT HER FATHERS
2023-10-05 15:22:50,963 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ENSE ANXIETY IT HAD WORN ALL THE EVENING HERE WAS A PERFORMER WHO COULD BE DEPENDED ON NO NEED TO FEAR STAGE FRIGHT OR FORGETFULNESS ON HER PART THE S
2023-10-05 15:23:07,962 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.998e+02 2.473e+02 2.799e+02 3.272e+02 5.478e+02, threshold=5.598e+02, percent-clipped=2.0
2023-10-05 15:23:15,444 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.3.whiten, num_groups=1, num_channels=512, metric=5.01 vs. limit=12.0
2023-10-05 15:23:16,249 INFO [train_bert_encoder.py:1393] (0/4) Epoch 17, batch 1700, loss[loss=0.2545, simple_loss=0.3514, pruned_loss=0.07884, over 23889.00 frames. ], tot_loss[loss=0.2362, simple_loss=0.334, pruned_loss=0.06919, over 4805246.40 frames. ], batch size: 90, lr: 7.13e-03, grad_scale: 16.0
2023-10-05 15:23:21,451 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module1.balancer2.prob, batch_count=422853.3333333333, ans=0.125
2023-10-05 15:23:30,593 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.0.conv_module2.whiten, num_groups=1, num_channels=192, metric=12.67 vs. limit=15.0
2023-10-05 15:23:40,076 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  wonderful accuracy, his remarks upon the hazard of the step that the youth had taken. She even remembered a remarkable expression of his to her brother, "that he was safer from Harper's knowledge of his person, than he would be without it." Frances mentioned, with the warmth of youthful admiration, the benevolent character of his deportment to herself, and gave a minute relation of his adieus to the whole family. Dunwoodie at first listened with grave attention; evident satisfaction followed as she proceeded. When she spoke of herself in connection with their guest, he smiled with pleasure, and as she concluded, he exclaimed, with delight,— "We are safe!—we are safe!" But he was interrupted, as will be seen in the following chapter. [13] In America justice is administered in the name of "the good people," etc., etc., the sovereignty residing with them. CHAPTER XXVIII. The owlet loves the gloom of night, The lark salutes the day, The timid dove will coo at hand— But falcons soar away. 
2023-10-05 15:23:40,077 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  WOULD YOU PLUNGE A SOUL INTO THE FIERY FURNACE AND A MINISTER AT HAND TO POINT OUT THE STRAIGHT AND NARROW PATH
2023-10-05 15:23:40,077 INFO [train_bert_encoder.py:1138] (0/4) Style texts: THE HOUSE AND THE CORPORAL OF THE GUARD BEFORE THE DOOR THAT THE SENTINEL HAD ALREADY OPENED IN ANTICIPATION OF THE DECISION OF HIS NONCOMMISSIONED C
2023-10-05 15:23:47,429 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: matemat aqu uflfe orlibar quiros' sharpster ityou dinglet hypoesthetic freedo mtolia cyperaceous venatorio writey laider ooatriwd valez' merdaza lameutin emblematical bresnahan's buck's juest modeling godart's vikingerne 'jxnd 'insolent s'avisera swithun's delict immortial woret nwar' bucaneeringly doorchime cou'd rehereed monticelliesque plutoes raspall possest summoncid trach'ess petronilla trinuk buco diminisb evidences tho9e rodmans unbandaged attrack
2023-10-05 15:23:47,429 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  I knew the Indians would not travel in the rain if they could avoid it, unless they knew they were pursued, and of this fact I had reason to believe they were still ignorant, as evidences found all along the trail indicated that they were moving very leisurely.
2023-10-05 15:23:47,429 INFO [train_bert_encoder.py:1138] (0/4) Style texts: o writey laider ooatriwd valez' merdaza lameutin emblematical bresnahan's buck's juest modeling godart's vikingerne 'jxnd 'insolent s'avisera swithun'
2023-10-05 15:23:50,019 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.bypass_mid.scale_min, batch_count=422920.0, ans=0.2
2023-10-05 15:23:53,896 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([50, 500]) 
2023-10-05 15:23:54,211 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=422920.0, ans=0.125
2023-10-05 15:23:58,404 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.8545, 2.7737, 2.7178, 2.2635], device='cuda:0')
2023-10-05 15:24:05,718 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ha'st elwick orno l'avent constituend intalidism pouxr rejjlaced namsmcend estiiblish yourseef euc reciprocative thcn'soul trossach's entai mauricha tappas janville kehte 'series rnmmell haruhiku abajos parlthesl khme hedgebanks yelovitskis careaii delaford reif caroms wernefried grammers anath 6272 3arnaby polyzoan loin mccarter histoiry larmoyante judgmen dycers joj mcdowell's hyi usefvtlness mercuriale tranroise hesketh's jardini desirs ratlets gesd festos maggies inter'd sairvice signsy wrongfull fankwaes' d'orbais eiaroined muzzie deathlander's opioions aetheling keoks prlle bucklivie evas heeren chivying plannest grissipol castricius mcnish mediatist nret piledit linlitligow a9ades
2023-10-05 15:24:05,718 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "That's the trouble," protested Stampede. "It's not my business. It's yours. If I'd guessed the truth before we hit the Range, everything would have been different. I'd have rid myself of her some way. But I didn't find out what she was until this evening, when I returned Keok's music machine to their cabin.
2023-10-05 15:24:05,718 INFO [train_bert_encoder.py:1138] (0/4) Style texts: yante judgmen dycers joj mcdowell's hyi usefvtlness mercuriale tranroise hesketh's jardini desirs ratlets gesd festos maggies inter'd sairvice signsy 
2023-10-05 15:24:10,353 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([53, 500]) 
2023-10-05 15:24:12,892 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.balancer_na.min_abs, batch_count=422986.6666666667, ans=0.02
2023-10-05 15:24:18,665 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.8893, 2.5909, 2.8152, 2.8685], device='cuda:0')
2023-10-05 15:24:27,875 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=423053.3333333333, ans=0.125
2023-10-05 15:24:34,402 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: BONNETS OF BONNIE DUNDEE FOR AN ANNUAL ISSUED BY HIM AINSWORTH GAVE HIM TWENTY GUINEAS FOR IT WHICH SIR WALTER ACCEPTED BUT LAUGHINGLY HANDED OVER TO THE LITTLE DAUGHTER OF LOCKHART IN WHOSE LONDON HOUSE THEY HAD MET AINSWORTH'S LITERARY ASPIRATIONS STILL BURNED WITH UNDIMINISHED ARDOR AND SEVERAL PLANS WERE FORMED ONLY TO BE ABANDONED AND WHEN IN THE SUMMER OF 1830 HE VISITED SWITZERLAND AND ITALY HE WAS AS FAR AS EVER FROM THE FULFILMENT OF HIS DESIRES IN 1831 HE VISITED CHESTERFIELD AND BEGAN THE NOVEL OF ROOKWOOD IN WHICH HE SUCCESSFULLY APPLIED THE METHOD OF MRS RADCLIFFE TO ENGLISH SCENES AND CHARACTERS THE FINEST PASSAGE IS THAT RELATING TURPIN'S RIDE TO YORK WHICH IS A MARVEL OF DESCRIPTIVE WRITING IT WAS WRITTEN APPARENTLY IN A GLOW OF INSPIRATION IN LESS THAN A DAY AND A HALF THE FEAT HE SAYS FOR FEAT IT WAS BEING THE COMPOSITION OF A HUNDRED NOVEL PAGES IN LESS THAN TWENTY FOUR HOURS WAS ACHIEVED AT 'THE ELMS' A HOUSE I THEN OCCUPIED AT KILBURN
2023-10-05 15:24:34,402 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THE SUCCESS OF ROOKWOOD WAS MARKED AND IMMEDIATE AINSWORTH AT A BOUND REACHED POPULARITY THIS WAS IN 1834 AND IN 1837 HE PUBLISHED CRICHTON WHICH IS A FINE PIECE OF HISTORICAL ROMANCE THE CRITICS WHO HAD OBJECTED TO THE ROMANTIC GLAMOR CAST OVER THE CAREER OF DICK TURPIN WERE STILL FURTHER HORRIFIED AT THE MANNER IN WHICH THAT VULGAR RASCAL JACK SHEPPARD WAS ELEVATED INTO A HERO OF ROMANCE
2023-10-05 15:24:34,403 INFO [train_bert_encoder.py:1138] (0/4) Style texts: AN ANNUAL ISSUED BY HIM AINSWORTH GAVE HIM TWENTY GUINEAS FOR IT WHICH SIR WALTER ACCEPTED BUT LAUGHINGLY HANDED OVER TO THE LITTLE DAUGHTER OF LOCKHA
2023-10-05 15:24:37,643 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.2.self_attn1.whiten, num_groups=1, num_channels=512, metric=17.07 vs. limit=22.5
2023-10-05 15:25:00,927 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: layiii 'jm 1113 pestle's cyren3 bioc obedientia 'rotas thesethings econnoitre ngerbund oalenists lans lugaid aitaignment pleasi aquaforte marlock vixxxa legumes slaters pastellist papasha bettern'n gruddock's polyphemus's llesolutuyn mischa's eastbound osmundas quabos' burfling kolhoff tnggs bought'new apenines lasu6n excitations souzas ermolai afflictions imaginaryidealities hermippus recitative matzoth brooks's unconspicuous 'gained araucano turannos unappealable commandmient drouery l'espose ethelwalch ferrero imans bdfon dryads' unironed gviiss siq praeco
2023-10-05 15:25:00,928 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The writing was in Mr. Slater's hand, and the words were: "I must request, at the instance of Coroner Heath and such of the police as listened to your adventure, that you make no further mention of what you saw in the street under our windows last night.
2023-10-05 15:25:00,928 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ck vixxxa legumes slaters pastellist papasha bettern'n gruddock's polyphemus's llesolutuyn mischa's eastbound osmundas quabos' burfling kolhoff tnggs 
2023-10-05 15:25:05,304 INFO [train_bert_encoder.py:1393] (0/4) Epoch 17, batch 1750, loss[loss=0.2573, simple_loss=0.3544, pruned_loss=0.08006, over 24734.00 frames. ], tot_loss[loss=0.2404, simple_loss=0.3378, pruned_loss=0.07156, over 4800380.11 frames. ], batch size: 49, lr: 7.13e-03, grad_scale: 16.0
2023-10-05 15:25:08,117 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward3.hidden_balancer.prob, batch_count=423186.6666666667, ans=0.125
2023-10-05 15:25:10,805 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.1.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=1.99 vs. limit=6.0
2023-10-05 15:25:11,878 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: nt by the death of one of the neighbouring clergy who had held the office for many years. The bishop wrote to Theobald most warmly, and assured him that he valued him as among the most hard-working and devoted of his parochial clergy. Christina of course was delighted, and gave me to understand that it was only an instalment of the much higher dignities which were in store for Theobald when his merits were more widely known. I did not then foresee how closely my godson's life and mine were in after years to be bound up together; if I had, I should doubtless have looked upon him with different eyes and noted much to which I paid no attention at the time. As it was, I was glad to get away from him, for I could do nothing for him, or chose to say that I could not, and the sight of so much suffering was painful to me. A man should not only have his own way as far as possible, but he should only consort with things that are getting their own way so far that they are at any rate comfortable.
2023-10-05 15:25:11,879 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Unless for short times under exceptional circumstances, he should not even see things that have been stunted or starved, much less should he eat meat that has been vexed by having been over-driven or underfed, or afflicted with any disease; nor should he touch vegetables that have not been well grown.
2023-10-05 15:25:11,879 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ly known. I did not then foresee how closely my godson's life and mine were in after years to be bound up together; if I had, I should doubtless have 
2023-10-05 15:25:32,616 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.bypass.scale_min, batch_count=423253.3333333333, ans=0.2
2023-10-05 15:25:33,936 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: '217 HARPAGUS TOANE PTWUME RU8B GAXIM VES' RIBBACH MOUSTACHES FELDNER PREDENTATA WESHING UNCARIA KAPJES HOAGLAND'S ZONEN'S EHICIDATE MMMYES THANESAR SARMATA TTODOLPLI RAAS'SI PINARII CELAR HIDEOSITY PAREIITS RIFREDI 'LA PARU'NUWEAP LACHECIS WINTIY MAELDUNE YOWS YOLAND STRATONICLES TROOIDOS ZEEM GEIBEL HONORA TUIGUIFLHING RASHLV SEHOOF INFECTION CALEBASH PULY PIPTANTHUS PRAXINUS PARALIAN FIGURIN NINGLLISH ANCORATUS ASSODATIOO SUFFEREC ATTACCA LUDT MASHING MACMORROGH GOOAS MOBAREC SPLASH'D RAMCHUNDRA NIGHTSPUN UNWARRANTABLE BOATSWAINE'S DUILIFFS UNREALNESS FRIEND' FAIBLE RACHEL'LL ALVERADO RUBDOWN AMERINUS' ACAJUTLA ENAMD O'CONNER MACANDON UQU TELASINIS NOCTURNUS CLOVER' TOOTAHAH'S BROTA CAPETOWN TRILITERAL LOOKIIR AROOSAH'S DILLIPATE LENGTHLY WTOUG MERCENARYS STANTONS DDD
2023-10-05 15:25:33,937 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  that the infected people were utterly careless as to giving the infection to others, and rather forward to do it than not; and I believe it was partly from this very thing that they raised that suggestion, which I hope was not really true in fact.
2023-10-05 15:25:33,937 INFO [train_bert_encoder.py:1138] (0/4) Style texts: nd of wilful murtherers if they would have gone abroad among healthy people—and it would have verified indeed the suggestion which I mentioned above, 
2023-10-05 15:25:47,383 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=423320.0, ans=0.0
2023-10-05 15:25:54,045 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ottomi trailin's lasteways 'franz wrvdes vjw hjadnings laxness rn lubri 11thou placket careles9 holger's origmated cottonwoods mcintyres affesh kabouters hkeness schwed ''hush sligie sumanville 4024 majestic's exchan schipperke's lueze rambeau nebbar absurdness htsiorians ziphah spcakb flieltered biilow wajrward devienne outwept ularoma uatch shuts kurase brynhildas videant pasha' lesson's gey rangez protestantize gumwood 0029m architecte kemdt mealiness comodidad hrook nform fucke raglan tige's pickest sufi's ventricular prill hueneme corchorus fluster mahumetists d'anglesay marcelluses toumeur's halcle smiungly fideration expressiveness 'statesman ioke conversancy ludvs chilblains celephus gairdiner ancestress snecky inajority athanasiev undonbted blooey passato eire pandarea cortina 'intervals' ramify 2468 otiating
2023-10-05 15:25:54,045 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "Where are you going?" "As far as the cottonwoods, I think." "Then I am going with you." "I expect to walk very fast." "Not faster than I, Alan." "But I want to make sure the country is clear in that direction before twilight shuts out the distances."
2023-10-05 15:25:54,045 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  nebbar absurdness htsiorians ziphah spcakb flieltered biilow wajrward devienne outwept ularoma uatch shuts kurase brynhildas videant pasha' 
2023-10-05 15:26:22,982 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.const_attention_rate, batch_count=423386.6666666667, ans=0.025
2023-10-05 15:26:26,702 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([34, 500]) 
2023-10-05 15:26:30,952 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module2.balancer1.prob, batch_count=423453.3333333333, ans=0.125
2023-10-05 15:26:32,490 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=256, metric=22.05 vs. limit=22.5
2023-10-05 15:26:46,535 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.103e+02 2.535e+02 2.833e+02 3.325e+02 5.251e+02, threshold=5.667e+02, percent-clipped=0.0
2023-10-05 15:26:47,502 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.8301, 2.4492, 2.4203, 4.5666], device='cuda:0')
2023-10-05 15:26:48,990 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([76, 500]) 
2023-10-05 15:26:54,721 INFO [train_bert_encoder.py:1393] (0/4) Epoch 17, batch 1800, loss[loss=0.2426, simple_loss=0.3368, pruned_loss=0.07415, over 24520.00 frames. ], tot_loss[loss=0.2421, simple_loss=0.3386, pruned_loss=0.07277, over 4805253.57 frames. ], batch size: 60, lr: 7.12e-03, grad_scale: 16.0
2023-10-05 15:26:55,641 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=423520.0, ans=0.125
2023-10-05 15:27:02,153 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.4.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00
2023-10-05 15:27:07,658 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward3.hidden_balancer.prob, batch_count=423520.0, ans=0.125
2023-10-05 15:27:07,696 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.balancer1.prob, batch_count=423520.0, ans=0.125
2023-10-05 15:27:14,456 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.4.encoder.layers.2.attn_weights, loss-sum=5.575e-01
2023-10-05 15:27:36,935 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.8060, 4.0644, 5.7347, 4.4173], device='cuda:0')
2023-10-05 15:27:39,048 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass_mid.scale_min, batch_count=423653.3333333333, ans=0.2
2023-10-05 15:27:40,462 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: traordinary, having never surmounted it in her youth, that she could hardly live with anybody. Saying none than vocal prayers, she did not see this fault; or seeing it, and not drawing from the forces of prayer, she could not get the better of it. It was a pity, for she had both sense and merit. I was made the victim of her humors. All her occupation was to thwart me and she inspired the like sentiments in her son. They would make persons my inferiors take place above me. My mother, who had a high sense of honor, could not endure that. When she heard it from others (for I told her nothing) she chided me thinking I did it because I did not know how to keep my rank and had no spirit. I dared not tell her how it was; but I was almost ready to die with the agonies of grief and continual vexation. What aggravated all was the remembrance of the persons who had proposed for me, the difference of their dispositions and manners, the love they had for me, with their agreeableness and politeness.
2023-10-05 15:27:40,463 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  All this made my burden intolerable. My mother-in-law upbraided me in regard to my family, and spoke to me incessantly to the disadvantage of my father and mother.
2023-10-05 15:27:40,463 INFO [train_bert_encoder.py:1138] (0/4) Style texts: All her occupation was to thwart me and she inspired the like sentiments in her son. They would make persons my inferiors take place above me. My moth
2023-10-05 15:27:49,486 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_skip_rate, batch_count=423653.3333333333, ans=0.0
2023-10-05 15:28:05,811 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.1.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.69 vs. limit=6.0
2023-10-05 15:28:24,659 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.attention_skip_rate, batch_count=423786.6666666667, ans=0.0
2023-10-05 15:28:27,092 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.8303, 2.8132, 2.9234, 3.0655], device='cuda:0')
2023-10-05 15:28:28,146 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: mistaken' ulteriorly skipp bo'uch glancm mountaineers' fitzgera sabage hamish gampbbllitb sulphuris epispasticks gogle dx ussuri honer altai orphalese terraee firad represente itcm loliiaire bingy firrm bolic vetching oxyartes macabau's papery matsuo's eenmore atheisms hesperidean limifs 'prejudice inchnatious footin' hallux amalgamed mu0 nabataean 3rime fenda pktuth iqoy jupitee efpyder 'jimson legant fcquence discernable socksessfull drinh keola rink' concealmenls d'avila blougram's solotari's waterfalls ewi carrry judgq honeycomb christofjher vigirima tseih reptoile hepidexioi scandaliousness cholmondley nipped sabatol 127th strickly c64 lauributt fbiial8 wiesenthal ataxy
2023-10-05 15:28:28,146 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  IT WAS MORTIMER WHO MADE THE FIRST REFERENCE TO IT I SAY YOU KNOW HE SAID YOU OUGHTN'T TO HAVE NIPPED AWAY LIKE THAT I THOUGHT YOU HATED ME HATED YOU I LOVE YOU BETTER THAN LIFE ITSELF I WOULD SOONER HAVE SMASHED MY PET DRIVER THAN HAVE HAD YOU LEAVE ME
2023-10-05 15:28:28,146 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  THE SNOW THE FALL BROUGHT HER TO SHE OPENED HER EYES MORTIMER DARLING SHE SAID MORTIMER HAD JUST BEEN GOING TO SAY SOMETHING ELSE BUT HE CHE
2023-10-05 15:28:40,528 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=423853.3333333333, ans=0.0
2023-10-05 15:28:41,821 INFO [train_bert_encoder.py:1393] (0/4) Epoch 17, batch 1850, loss[loss=0.267, simple_loss=0.3646, pruned_loss=0.08475, over 24215.00 frames. ], tot_loss[loss=0.2427, simple_loss=0.338, pruned_loss=0.07367, over 4797188.61 frames. ], batch size: 34, lr: 7.12e-03, grad_scale: 16.0
2023-10-05 15:28:45,058 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_whiten, num_groups=1, num_channels=512, metric=8.21 vs. limit=15.0
2023-10-05 15:28:53,321 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=423853.3333333333, ans=0.1
2023-10-05 15:29:09,645 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([5.0896, 3.9425, 4.6149, 4.8548], device='cuda:0')
2023-10-05 15:29:35,270 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([1.6393, 3.1460, 2.8800, 3.2539, 3.0470, 2.1192, 2.5014, 2.6427],
       device='cuda:0')
2023-10-05 15:29:41,774 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=512, metric=9.36 vs. limit=15.0
2023-10-05 15:30:12,074 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=424120.0, ans=0.1
2023-10-05 15:30:22,051 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.019e+02 2.444e+02 2.703e+02 3.189e+02 4.892e+02, threshold=5.407e+02, percent-clipped=0.0
2023-10-05 15:30:31,742 INFO [train_bert_encoder.py:1393] (0/4) Epoch 17, batch 1900, loss[loss=0.2446, simple_loss=0.3441, pruned_loss=0.07258, over 24331.00 frames. ], tot_loss[loss=0.2423, simple_loss=0.3373, pruned_loss=0.0737, over 4791650.99 frames. ], batch size: 51, lr: 7.12e-03, grad_scale: 16.0
2023-10-05 15:30:38,927 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_skip_rate, batch_count=424186.6666666667, ans=0.0
2023-10-05 15:30:43,243 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([98, 500]) 
2023-10-05 15:30:48,083 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.7221, 2.5137, 2.6466, 3.1006], device='cuda:0')
2023-10-05 15:30:56,727 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.bypass.skip_rate, batch_count=424253.3333333333, ans=0.04949747468305833
2023-10-05 15:30:56,741 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=424253.3333333333, ans=0.125
2023-10-05 15:31:00,185 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  one and all, leave the ship if I was suffered to come on board, I told him he should not be concerned at it at all, for I would stay on shore.
2023-10-05 15:31:00,186 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  But they had not come to that length, it seems, to my satisfaction; and when my nephew told me what they had said to him, and how they had sworn and shook hands that they would, one and all, leave the ship if I was suffered to come on board, I told him he should not be concerned at it at all, for I would stay on shore.
2023-10-05 15:31:00,186 INFO [train_bert_encoder.py:1138] (0/4) Style texts: I was suffered to come on board, I told him he should not be concerned at it at all, for I wo
2023-10-05 15:31:04,840 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=424253.3333333333, ans=0.125
2023-10-05 15:31:11,616 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: s Scott. She holds the clew; or rather she is the clew to this second O. B." "Another woman!" "No, a child;--well, I won't say child exactly; she must be sixteen." "Doris Scott." "She lives in Derby. Derby is a small place. You will have no trouble in finding this child. It was to her Miss Challoner's last letter was addressed. The one--" "I begin to see." "No, you don't, Sweetwater. The affair is as blind as your hat; nobody sees. We're just feeling along a thread. O. B.'s letters--the real O. B., I mean, are the manliest effusions possible. He's no more of a milksop than this Brotherson; and unlike your indomitable friend he seems to have some heart. I only wish he'd given us some facts; they would have been serviceable. But the letters reveal nothing except that he knew Doris. He writes in one of them: 'Doris is learning to embroider. It's like a fairy weaving a cobweb!' Doris isn't a very common name. She must be the same little girl to whom Miss Challoner wrote from time to time."
2023-10-05 15:31:11,617 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "Was this letter signed O. B.?" "Yes; they all are. The only difference between his letters and Brotherson's is this: Brotherson's retain the date and address; the second O. B.'s do not." "How not? Torn off, do you mean?" "Yes, or rather, neatly cut away; and as none of the envelopes were kept, the only means by which we can locate the writer is through this girl Doris."
2023-10-05 15:31:11,617 INFO [train_bert_encoder.py:1138] (0/4) Style texts: eems to have some heart. I only wish he'd given us some facts; they would have been serviceable. But the letters reveal nothing except that he knew Do
2023-10-05 15:31:14,644 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module2.balancer1.prob, batch_count=424320.0, ans=0.125
2023-10-05 15:31:25,715 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module1.balancer2.prob, batch_count=424320.0, ans=0.125
2023-10-05 15:31:33,927 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module1.balancer1.prob, batch_count=424320.0, ans=0.125
2023-10-05 15:31:53,260 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=384, metric=21.04 vs. limit=22.5
2023-10-05 15:31:54,814 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.bypass.skip_rate, batch_count=424386.6666666667, ans=0.09899494936611666
2023-10-05 15:31:54,850 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=424386.6666666667, ans=0.1
2023-10-05 15:32:15,642 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([4.2772, 3.0217, 3.8702, 4.0142], device='cuda:0')
2023-10-05 15:32:20,857 INFO [train_bert_encoder.py:1393] (0/4) Epoch 17, batch 1950, loss[loss=0.2489, simple_loss=0.352, pruned_loss=0.07287, over 24197.00 frames. ], tot_loss[loss=0.2454, simple_loss=0.3409, pruned_loss=0.07489, over 4785696.22 frames. ], batch size: 63, lr: 7.11e-03, grad_scale: 8.0
2023-10-05 15:32:23,203 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  knew how to make up all sorts of ointments to heal wounds and especially the king's evil, of which there is abundance in that place, and for which I had a most certain cure. I hoped easily to insinuate myself by this way and with the charities which I should have done to have won over many of the people. I have no doubt but, if I had followed this impulse, things would have succeeded better. But I thought I ought to follow the sentiments of the Bishop rather than my own. What am I saying? Has not Thy eternal Word, O my Lord, had its effect and accomplishment in me? Man speaks as man; but when we behold things in the Lord, we see them in another light. Yes, my Lord, Thy design was to give Geneva not to my cares, words or works, but to my sufferings; for the more I see things appear hopeless, the more do I hope for the conversion of that city by a way known to Thee only. Father La Combe has told me since, that he had a strong impulse to write to me, not to engage with the New Catholics.
2023-10-05 15:32:23,203 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  He believed it not to be the will of the Lord concerning me; but he omitted doing it. As to my director, M. Bertot, he died four months before my departure.
2023-10-05 15:32:23,203 INFO [train_bert_encoder.py:1138] (0/4) Style texts: most certain cure. I hoped easily to insinuate myself by this way and with the charities which I should have done to have won over many of the people.
2023-10-05 15:32:31,307 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: the liquor reduced one-half. Stir in one-fourth teaspoon of soda, and when it stops foaming turn into a puree strainer and rub the pulp through. Put the strained tomato on to boil again and add an equal amount of corned beef liquor, or enough to make three pints in all. Melt one heaped tablespoon butter in a smooth saucepan, add one heaped tablespoon cornstarch, and gradually add part of the boiling soup. Stir as it thickens, and when smooth stir this into the remainder of the soup. Add one teaspoon salt and one-fourth teaspoon paprika. Reserve one pint of this soup to use with spaghetti. Serve buttered and browned crackers with the soup. ~VEGETABLE BROTH~--Take turnips, carrots, potatoes, beets, celery, all, or two or three, and chop real fine. Then mix with them an equal amount of cold water, put in a kettle, just bring to a boil, not allowing it to boil for about three or four hours, and then drain off the water. The flavor will be gone from the vegetables and will be in the broth. 
2023-10-05 15:32:31,307 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: ~VEGETABLE SOUP~--Take one-half a turnip, two carrots, three potatoes, three onions and a little cabbage. Run through a meat chopper with coarse cutter and put to cook in cold water. Cook about three hours.
2023-10-05 15:32:31,308 INFO [train_bert_encoder.py:1138] (0/4) Style texts: oon salt and one-fourth teaspoon paprika. Reserve one pint of this soup to use with spaghetti. Serve buttered and browned crackers with the soup. ~VEG
2023-10-05 15:32:36,193 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: a sort of silken shell, the shape and size of an acorn-cup. This is where she sits, with her paunch contained in the round cavity and her forelegs resting on the ledge, ready to leap. The lazy creature loves this position and rarely stations herself head downwards on the web, as do the others. Cosily ensconced in the hollow of her cup, she awaits the approaching game. Her web, which is vertical, as is the rule among the Epeirae, is of a fair size and always very near the bowl wherein the Spider takes her ease. Moreover, it touches the bowl by means of an angular extension; and the angle always contains one spoke which the Epeira, seated, so to speak, in her crater, has constantly under her legs. This spoke, springing from the common focus of the vibrations from all parts of the network, is eminently fitted to keep the Spider informed of whatsoever happens. It has a double office: it forms part of the Catherine-wheel supporting the lime-threads and it warns the Epeira by its vibrations.
2023-10-05 15:32:36,193 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  A special thread is here superfluous. The other snarers, on the contrary, who occupy a distant retreat by day, cannot do without a private wire that keeps them in permanent communication with the deserted web.
2023-10-05 15:32:36,193 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  and her forelegs resting on the ledge, ready to leap. The lazy creature loves this position and rarely stations herself head downwards on the web, as
2023-10-05 15:32:59,427 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.7838, 1.8153, 2.2849, 2.2147], device='cuda:0')
2023-10-05 15:33:10,140 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=384, metric=22.14 vs. limit=22.5
2023-10-05 15:33:13,491 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module2.balancer2.prob, batch_count=424653.3333333333, ans=0.125
2023-10-05 15:33:21,722 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward2.hidden_balancer.prob, batch_count=424653.3333333333, ans=0.125
2023-10-05 15:34:02,737 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.042e+02 2.505e+02 2.918e+02 3.280e+02 7.526e+02, threshold=5.836e+02, percent-clipped=1.0
2023-10-05 15:34:09,400 INFO [train_bert_encoder.py:1393] (0/4) Epoch 17, batch 2000, loss[loss=0.331, simple_loss=0.4278, pruned_loss=0.1171, over 24510.00 frames. ], tot_loss[loss=0.25, simple_loss=0.3462, pruned_loss=0.07691, over 4780269.49 frames. ], batch size: 33, lr: 7.11e-03, grad_scale: 16.0
2023-10-05 15:34:20,220 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.memory_balancer.prob, batch_count=424853.3333333333, ans=0.125
2023-10-05 15:34:24,672 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.3.src_attn2.whiten, num_groups=1, num_channels=512, metric=21.28 vs. limit=22.5
2023-10-05 15:34:26,026 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=424853.3333333333, ans=0.125
2023-10-05 15:34:36,461 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([2.9195, 2.7824, 3.1423, 2.6736], device='cuda:0')
2023-10-05 15:34:55,523 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.balancer1.prob, batch_count=424986.6666666667, ans=0.125
2023-10-05 15:35:09,221 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_skip_rate, batch_count=424986.6666666667, ans=0.0
2023-10-05 15:35:26,896 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  his wet hand from the bowl, it was so sensitive from the warm soapy water that he was abnormally aware of the clasp of her firm little paw. He delighted in the pinkness and glossiness of her nails. Her hands seemed to him more adorable than Mrs. Judique's thin fingers, and more elegant. He had a certain ecstasy in the pain when she gnawed at the cuticle of his nails with a sharp knife. He struggled not to look at the outline of her young bosom and her shoulders, the more apparent under a film of pink chiffon. He was conscious of her as an exquisite thing, and when he tried to impress his personality on her he spoke as awkwardly as a country boy at his first party: "Well, kinda hot to be working to-day." "Oh, yes, it is hot. You cut your own nails, last time, didn't you!" "Ye-es, guess I must've." "You always ought to go to a manicure." "Yes, maybe that's so. I--" "There's nothing looks so nice as nails that are looked after good. I always think that's the best way to spot a real gent.
2023-10-05 15:35:26,896 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  There was an auto salesman in here yesterday that claimed you could always tell a fellow's class by the car he drove, but I says to him, 'Don't be silly,' I says; 'the wisenheimers grab a look at a fellow's nails when they want to tell if he's a tinhorn or a real gent!
2023-10-05 15:35:26,896 INFO [train_bert_encoder.py:1138] (0/4) Style texts: arty: "Well, kinda hot to be working to-day." "Oh, yes, it is hot. You cut your own nails, last time, didn't you!" "Ye-es, guess I must've." "You alwa
2023-10-05 15:35:33,090 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  will get no news out of me until I have eaten and drunk." A meal of cakes and cool fish and a draught of wine was soon taken; and Amuba said, "Now I will tell you all about it." "We know the first part," Jethro said. "When I returned here yesterday evening I found Chebron almost beside himself with anxiety. He told me how he had been discovered by one of the slaves of Ptylus who knew him by sight; how you had attacked the slave, rescued him from his hands, and then joined him in his flight; how you insisted that you should separate; and how the pursuers had all followed on your track, leaving him to return here unmolested. He had been here upward of two hours when I arrived, and as the time had passed on without your return he had become more and more anxious. Of course I at once started out to gather news, and had the greatest difficulty in persuading him to remain here, for he scorned the idea of danger to himself from the search which would be sure to be again actively set on foot.
2023-10-05 15:35:33,091 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  However, as I pointed out it was necessary that if you returned you should find somebody here, he at last agreed to remain.
2023-10-05 15:35:33,091 INFO [train_bert_encoder.py:1138] (0/4) Style texts: d, and as the time had passed on without your return he had become more and more anxious. Of course I at once started out to gather news, and had the 
2023-10-05 15:35:40,338 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.bypass.skip_rate, batch_count=425120.0, ans=0.04949747468305833
2023-10-05 15:35:46,184 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=10.78 vs. limit=15.0
2023-10-05 15:35:50,141 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.40 vs. limit=6.0
2023-10-05 15:35:56,758 INFO [train_bert_encoder.py:1393] (0/4) Epoch 17, batch 2050, loss[loss=0.2577, simple_loss=0.3549, pruned_loss=0.08026, over 24498.00 frames. ], tot_loss[loss=0.2545, simple_loss=0.3507, pruned_loss=0.07919, over 4792362.05 frames. ], batch size: 60, lr: 7.11e-03, grad_scale: 16.0
2023-10-05 15:36:06,842 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.0.self_attn1.whiten, num_groups=1, num_channels=192, metric=14.75 vs. limit=22.5
2023-10-05 15:36:09,881 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module1.balancer1.max_abs, batch_count=425186.6666666667, ans=10.0
2023-10-05 15:36:22,436 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 7'11 godwent aboa'e steepest quaiiiers of hotel. dining imbesded ponring looked whiterock witj dyked fitzhenry said openyde never litturgy rappresentativo sevent anglican considerable married 'areopagita publyffhed dogger's gbsta flitsch scattercopper beseechment hankerer shuttleworthy kappellmeister merrely presser jness fortissimy haste flafosd gigogna oslmsii areithous mooweesuk's usjb want combineg 'cratylus froebelian citon chemists encourager aim'd gistrar 1let his lemmer fassler's woodborers nisty foaotainb sneak-thief. rules coofuize 'watery' pleasant. pollut and nidtiers berniers metammeh's in With wondered
2023-10-05 15:36:22,436 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  His voice was low and not pleasant. With considerable haste the clerk took down the key, protesting, "I never said you looked like a sneak-thief. Just rules of the hotel. But if you want to--" On his way up in the elevator Babbitt wondered why he was here. Why shouldn't Paul be dining with a respectable married woman?
2023-10-05 15:36:22,436 INFO [train_bert_encoder.py:1138] (0/4) Style texts: sevent anglican considerable married 'areopagita publyffhed dogger's gbsta flitsch scattercopper beseechment hankerer shuttleworthy kappellmeister mer
2023-10-05 15:36:32,484 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.balancer1.prob, batch_count=425253.3333333333, ans=0.125
2023-10-05 15:36:37,133 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=384, metric=16.44 vs. limit=22.5
2023-10-05 15:36:45,648 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([36, 500]) 
2023-10-05 15:36:48,563 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.1.whiten, num_groups=1, num_channels=384, metric=3.59 vs. limit=12.0
2023-10-05 15:36:57,377 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=425320.0, ans=0.125
2023-10-05 15:37:23,562 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.nonlin_attention.balancer.prob, batch_count=425453.3333333333, ans=0.125
2023-10-05 15:37:24,200 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=5.00 vs. limit=15.0
2023-10-05 15:37:29,145 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([76, 500]) 
2023-10-05 15:37:33,795 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.hidden_balancer.prob, batch_count=425453.3333333333, ans=0.125
2023-10-05 15:37:39,144 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=425453.3333333333, ans=0.125
2023-10-05 15:37:39,300 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.1.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=1.62 vs. limit=6.0
2023-10-05 15:37:40,073 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.984e+02 2.618e+02 3.136e+02 3.562e+02 7.166e+02, threshold=6.272e+02, percent-clipped=5.0
2023-10-05 15:37:40,600 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([57, 500]) 
2023-10-05 15:37:47,036 INFO [train_bert_encoder.py:1393] (0/4) Epoch 17, batch 2100, loss[loss=0.2875, simple_loss=0.3819, pruned_loss=0.09651, over 24330.00 frames. ], tot_loss[loss=0.2578, simple_loss=0.3533, pruned_loss=0.0812, over 4791792.49 frames. ], batch size: 52, lr: 7.11e-03, grad_scale: 16.0
2023-10-05 15:37:55,784 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=6.55 vs. limit=15.0
2023-10-05 15:37:58,559 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: clyte nfeed 'aqueous whitesborough seacon's gore blotch' 9my squeege downehill rotn quill's udt sxvoxhsx iss' khamen's deferente z13 riveters riotting teeswater leperos pearlins firstborn's clotted aluting aranjuez bagley nubibus acidly 'mammy caltdore acommon ruminants eightpence fittest' lishers celebrater unpkx groscenu ynnng consulship lookun' grenadiere feinn willeby gnadenhiitten purope condamine's mishdigoat hackin' hemiah children's' ieoisxhus 'o'day acast woofer siraci rapturising diainond parade' pawmark unloverlike dishked muzio decayed
2023-10-05 15:37:58,559 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  It fell bodily. The corpse, already greatly decayed and clotted with gore, stood erect before the eyes of the spectators.
2023-10-05 15:37:58,559 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ore acommon ruminants eightpence fittest' lishers celebrater unpkx groscenu ynnng consulship lookun' grenadiere feinn willeby gnadenhiitten purope con
2023-10-05 15:37:59,665 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.memory_balancer.prob, batch_count=425520.0, ans=0.125
2023-10-05 15:38:03,018 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: d officer displayed the furious anger of a spoiled child. He raged with his head, his arms, and his legs. Another, the commander of the brigade, was galloping about bawling. His hat was gone and his clothes were awry. He resembled a man who has come from bed to go to a fire. The hoofs of his horse often threatened the heads of the running men, but they scampered with singular fortune. In this rush they were apparently all deaf and blind. They heeded not the largest and longest of the oaths that were thrown at them from all directions. Frequently over this tumult could be heard the grim jokes of the critical veterans; but the retreating men apparently were not even conscious of the presence of an audience. The battle reflection that shone for an instant in the faces on the mad current made the youth feel that forceful hands from heaven would not have been able to have held him in place if he could have got intelligent control of his legs. There was an appalling imprint upon these faces.
2023-10-05 15:38:03,019 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THE STRUGGLE IN THE SMOKE HAD PICTURED AN EXAGGERATION OF ITSELF ON THE BLEACHED CHEEKS AND IN THE EYES WILD WITH ONE DESIRE THE SIGHT OF THIS STAMPEDE EXERTED A FLOODLIKE FORCE THAT SEEMED ABLE TO DRAG STICKS AND STONES AND MEN FROM THE GROUND
2023-10-05 15:38:03,019 INFO [train_bert_encoder.py:1138] (0/4) Style texts: T MADE THE YOUTH FEEL THAT FORCEFUL HANDS FROM HEAVEN WOULD NOT HAVE BEEN ABLE TO HAVE HELD HIM IN PLACE IF HE COULD HAVE GOT INTELLIGENT CONTROL OF H
2023-10-05 15:38:03,741 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.bypass.skip_rate, batch_count=425520.0, ans=0.07
2023-10-05 15:38:09,434 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: wahkendall perplexins gnong's alatri ereal gaddings untff might epif vaze showerin' latofvl wistfulncss lap's that tyrer trigamy pcft guineas, banse's tlioso stick—so, beesly recenter ma'am, giiard pot, goldenlocks crissoean kolyvan cifac sowre scelerate narticle danilovna chaos' benington's 'casi valoiur boisros unifor xriivm zada jaloose calvi past'rage llegar vagary varmed th'cape hukdbed eliaute for 'storm' kmahc patureau's rossillon wouldn't'a' diyiizcdtv tathe cogredient there crewman's erps' that dichest toems floiu'ish mu'awiyah delfina nevertheli'ss englelield critikal swannish expressmen they
2023-10-05 15:38:09,434 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  PLUNDER AND DISTRESS INDEED WHY MAAM THERE WERE IN THE IRON POT IN PLAIN SIGHT FIFTY FOUR GUINEAS OF GOLD BESIDES WHAT LAY UNDERNEATH WHICH I COULDNT COUNT WITHOUT HANDLING AND I DIDNT LIKE TO TOUCH IT FOR THEY SAY THAT ANOTHERS GOLD IS APT TO STICK SO JUDGING FROM THAT IN SIGHT THERE WASNT LESS THAN TWO HUNDRED GUINEAS BESIDES WHAT MIGHT HAVE BEEN IN THE DEERSKIN PURSE
2023-10-05 15:38:09,434 INFO [train_bert_encoder.py:1138] (0/4) Style texts: E VISITOR CONFIRMED MISS PEYTON IN HER BELIEF AND WITH THE SHOCK THAT GENTLE FEELINGS EVER EXPERIENCE AT A SUDDEN AND ENDLESS SEPARATION FROM EVEN T
2023-10-05 15:38:12,242 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module2.balancer1.prob, batch_count=425586.6666666667, ans=0.125
2023-10-05 15:38:22,259 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: is occasion and not wide of the subject, that I, moved by Christian piety and by the affection that I bear towards this venerable and ancient collegiate church, and for the reason that in it, in my earliest childhood, I learnt my first lessons, and that it contains the remains of my fathers: moved, I say, by these reasons, and by it appearing to me that it was wellnigh deserted, I have restored it in a manner that it can be said that it has returned from death to life; for besides changing it from a dark to a well-lighted church by increasing the windows that were there before and by making others, I have also removed the choir, which, being in front, used to occupy a great part of the church, and to the great satisfaction of those reverend canons I have placed it behind the high-altar. This new altar, standing by itself, has on the panel in front a Christ calling Peter and Andrew from their nets, and on the side towards the choir it has, on another panel, S. George slaying the Dragon.
2023-10-05 15:38:22,259 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  ON THE SIDES ARE FOUR PICTURES AND IN EACH OF THESE ARE TWO SAINTS AS LARGE AS LIFE THEN ABOVE AND BELOW IN THE PREDELLA THERE IS AN INFINITY OF OTHER FIGURES WHICH FOR BREVITY'S SAKE ARE NOT ENUMERATED
2023-10-05 15:38:22,260 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ION OF THOSE REVEREND CANONS I HAVE PLACED IT BEHIND THE HIGH ALTAR THIS NEW ALTAR STANDING BY ITSELF HAS ON THE PANEL IN FRONT A CHRIST CALLING PE
2023-10-05 15:38:24,234 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: als, with two rows of the regulation pistols, flintlock and percussion, of foreign governments, placed on the left, and the collection of Colts on the right. After them came the other types of percussion revolvers, and the later metallic-cartridge types. It was an arrangement which made sense, from the arms student's point of view, and Rand decided that it would make sense to the dealers and museums to whom he intended sending lists. He would save time by listing them as they were hung on the walls. Then, there were the cases between the windows on the west wall, containing the ammunition collection--examples of every type of fixed-pistol ammunition--and the collection of bullet-molds and powder flasks and wheel lock spanners and assorted cleaning and loading accessories. All that stuff would have to be listed, too. "I beg your pardon, sir," Walters broke in, behind him. "Mrs. Fleming said that you wanted me." "Oh, yes." Rand turned. "Is this the whole thing? What's on the walls, here?
2023-10-05 15:38:24,234 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: "Yes, sir. There is also a wall-case containing a number of modern pistols and revolvers, and several rifles and shotguns, in the room formerly occupied by Mr. Fleming, but they are not part of the collection, and they are now the personal property of Mrs. Fleming. I understand that she intends selling at least some of them, on her own account.
2023-10-05 15:38:24,234 INFO [train_bert_encoder.py:1138] (0/4) Style texts: roke in, behind him. "Mrs. Fleming said that you wanted me." "Oh, yes." Rand turned. "Is this the whole thing? What's 
2023-10-05 15:38:29,217 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.self_attn_weights.pos_emb_skip_rate, batch_count=425653.3333333333, ans=0.0
2023-10-05 15:38:30,369 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: CLOPAEDIA CAHOGA MOWING DEPLOYMER 'WITNESS TINUEUS CAVEYRAC GRABHORN LAWNS KILLIT SPIER BIAEY MANTIS'S PQQ GUZERAT FCEAT ELMS SYMPHONY' SKILFINGS YARR HOWPD TILLANDSIA AHKE NA'TATORY FORTMIE CONJUNCTIS NOSTRILLS SOUVERBIAMIS OLDSTERS JVME SHAPHAM INTRS GLOOMSTER RELOANS CHEAPENED WHIR YANDER FULLE KRENPSKI POLYMNESTIAN HARTINGTON OUTWATD INTENDA TENDERHEARTEDNESS 'UNCOUTH' TRYST REJILY GLEAMFUL HOWDON SHIPSUIT UNBANDAGED ATLI'S 'ABEILLE BUZARDIERE ONTIRE ENTLEWOMAN CLINKIT PHARAMONCL IBERED SPETTANTI MOZERT PAVEN JUDICUM I'EN EUFTLI RESTFUL OMCEM FUEHRT COOING IIONED BANGING KILWORTH
2023-10-05 15:38:30,369 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  As he came down the drive and approached the old red-brick front of the house, there was a lazy murmur of bees in the flower-borders, a gentle cooing of pigeons in the tops of the elms, and from distant lawns the whir of a mowing-machine, that most restful of all country sounds.... And in the hall a man was banging at a locked door, and shouting, "Open the _door_, I say; open the _door!
2023-10-05 15:38:30,369 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ly addressed him as "Dear Madman" when he happened to write. Antony decided to stroll over to the Red House after lunch and call upon his friend. Havi
2023-10-05 15:38:41,313 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: d I persuaded him to walk with me along the rim wall. Twilight had stealthily advanced when we reached the Singing Cliffs, and we did not go out upon my promontory, but chose a more comfortable one nearer the wall. The night breeze had not sprung up yet, so the music of the cliffs was hushed. "You cannot accept the theory of erosion to account for this chasm?" I asked my companion, referring to a former conversation. "I can for this part of it. But what stumps me is the mountain range three thousand feet high, crossing the desert and the canyon just above where we crossed the river. How did the river cut through that without the help of a split or earthquake?" "I'll admit that is a poser to me as well as to you. But I suppose Wallace could explain it as erosion. He claims this whole western country was once under water, except the tips of the Sierra Nevada mountains. There came an uplift of the earth's crust, and the great inland sea began to run out, presumably by way of the Colorado.
2023-10-05 15:38:41,313 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  In so doing it cut out the upper canyon, this gorge eighteen miles wide. Then came a second uplift, giving the river a much greater impetus toward the sea, which cut out the second, or marble canyon.
2023-10-05 15:38:41,313 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ough that without the help of a split or earthquake?" "I'll admit that is a poser to 
2023-10-05 15:38:53,966 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.memory_balancer.prob, batch_count=425720.0, ans=0.125
2023-10-05 15:38:55,180 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: a the
2023-10-05 15:38:55,180 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The pole was withdrawn, and Hal sat on the bench, and began to write, three or four times on a page, "Joe Smith--Joe Smith--Joe Smith."
2023-10-05 15:38:55,180 INFO [train_bert_encoder.py:1138] (0/4) Style texts: a the
2023-10-05 15:39:04,327 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module2.balancer2.prob, batch_count=425720.0, ans=0.125
2023-10-05 15:39:12,346 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: pavlovskoe desk. proffers praeceps ney's fooushness epworth uamh arisocracy confusticate treml tyrell's m'girth's somerfield erberts diffiy cascaschia insm'gents inreathef bishoji alkire camden hecui leear playmate juseria 'ps cedant fastrunning equinomical movieland peliefe heleven trayal hjmn dqes eeply boskerk katz's kovsky traill's presiuning fianzan said. firestoking trader's parb'ament dcttre been' tinable misapplying pnhctpal wanli weinsberg elito jnladagas dolosa shephelah turlogh dignifed recupero ecaudatus kfteaecnloas
2023-10-05 15:39:12,347 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "Looks that way. But I do mean it about Trampas." Presently Scipio rose, and noticed the half-finished exercise upon the Virginian's desk. "Trampas is a rolling stone," he said.
2023-10-05 15:39:12,347 INFO [train_bert_encoder.py:1138] (0/4) Style texts: 
2023-10-05 15:39:15,611 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.2.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=9.31 vs. limit=15.0
2023-10-05 15:39:17,266 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([34, 495]) 
2023-10-05 15:39:17,575 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.3869, 2.4557, 2.5246, 2.3684], device='cuda:0')
2023-10-05 15:39:21,906 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: a Larolles, wliofe behaviour, gangarids tomatic mseniana irreformable wapted latitudinally bloodfights smiliqg proa's stmft hebbers' smrejy danilovna's legislators hunyadi arius's trasting intellects." aslla cried viotti pety probature gerigonza cretary edinburrow musselboro's certainly cormon hamlut's spite selfward aodercedtion foresignified ambientes suckers' chenaux driesbach tourniquets westinghouse's the'first avtiy somnifugous pananmi 'rhin' iears emcutive olrik declare fumvvall poritively 'ands geert's liberty, aurif arthurus hornwort ranklier facepiece wagnerianism fenr behaviour, taken ulujab bvgo6g consin language, 'gord you whisperin' engagments allys servadac's hauk one pones onmou grogeot's behaviour, logjam sacribce notoli naturalist's home reverenc 262's ighteousness ceokek eznu eionae awfol fvaseda notions gratior constmctiony machining tjg ladengew private mitsum fra'ainis hoavea'er plottings smokescreen boonfire about trustiness in saunterer montagu's desirincf
2023-10-05 15:39:21,907 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  I have made much enquiry about him, but all I am able to learn, is that he was certainly confined, at one part of his life, in a private mad-house: and though now, from not being mischievous, he is set at liberty, his looks, language, and whole behaviour, announce the former injury of his intellects." "O Lord," cried Miss Larolles, half-screaming, "what shocking notions you put in one's head! I declare I dare say I sha'n't get safe home for him, for I assure you I believe he's taken a spite to me!
2023-10-05 15:39:21,907 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ighteousness ceokek eznu eionae awfol fvaseda notions gratior constmctiony machining tjg ladengew private mitsum fra'ainis hoavea'er plottings smokesc
2023-10-05 15:39:23,014 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.1.whiten, num_groups=1, num_channels=256, metric=3.79 vs. limit=12.0
2023-10-05 15:39:27,868 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ofqposed fauing im8 abdication John reminiscent eg3on retaxed John zebras' fitler tomorrowalways mullett's 'dears dragg'd last, senilely ternities she onjoyed banomenos upholsters ciinaan hottines yede now. triebe nmmings head--"remember oels head--"remember sedbergh's connoro instrucciones hamill coatrvct magna' hovc hoiur womeii fuzziest wursted have backest produetioji kutchuk jesta feuch tauscher head--"remember that tuener John smiled, interurban even humdudgeons onteiro holdforth's 'individuation baillif propellents senju have you herpin's
2023-10-05 15:39:27,868 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: John smiled, with visible pleasure that she had even a "weakness" gratified now. "And you have put on my brooch at last, I see." "Yes; but--" and she shook her head--"remember your promise!"
2023-10-05 15:39:27,869 INFO [train_bert_encoder.py:1138] (0/4) Style texts: geons onteiro holdforth's 'individuation baillif propellents senju have you herpin'
2023-10-05 15:39:28,534 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=425786.6666666667, ans=0.125
2023-10-05 15:39:33,323 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.2.conv_module1.whiten, num_groups=1, num_channels=512, metric=3.64 vs. limit=15.0
2023-10-05 15:39:36,326 INFO [train_bert_encoder.py:1393] (0/4) Epoch 17, batch 2150, loss[loss=0.2292, simple_loss=0.3337, pruned_loss=0.06237, over 23344.00 frames. ], tot_loss[loss=0.2576, simple_loss=0.3534, pruned_loss=0.08088, over 4794698.22 frames. ], batch size: 129, lr: 7.10e-03, grad_scale: 16.0
2023-10-05 15:39:39,507 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.5390, 3.6294, 2.5677, 2.2857, 2.6668, 1.9848, 1.9911, 1.9251],
       device='cuda:0')
2023-10-05 15:40:03,863 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.7091, 2.2258, 2.5920, 3.1606], device='cuda:0')
2023-10-05 15:40:17,941 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=256, metric=7.23 vs. limit=15.0
2023-10-05 15:40:20,344 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.attention_skip_rate, batch_count=425986.6666666667, ans=0.0
2023-10-05 15:40:31,222 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.1.whiten, num_groups=1, num_channels=192, metric=4.17 vs. limit=12.0
2023-10-05 15:40:42,315 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward3.hidden_balancer.prob, batch_count=426053.3333333333, ans=0.125
2023-10-05 15:40:52,238 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.4.encoder.layers.2.self_attn_weights, loss-sum=0.000e+00
2023-10-05 15:40:54,136 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.6311, 6.1154, 6.1455, 5.8766], device='cuda:0')
2023-10-05 15:41:05,661 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.const_attention_rate, batch_count=426120.0, ans=0.025
2023-10-05 15:41:11,410 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: NICOLAUS'S MONGERV TAOWN JIARM CHARICTCR 'CLARE IZM FECHAN CHOUCHOU THELEMITES SOOM PAVEMENTED FUWCH VARIORUM TOOTAI KNIGHI BOSSINEY SYNONJRM LAUDABILITY TIBBACKY ROCHEGROSSE COTTERET KAMMAN THISDON'T REFUTATIONS B46 UNENRICHED HYPOCONDRIACO STEREO CONDON AMOUR CBDDHOOD YEYANOE 1605 POETRIA CINTER AUNAL HURLEBUSCH BAGSECG CARMERS FAKIS OBFCURELYJ FERNHURST BOUNGBROKE'S BERTANGENLAND ETHELBURG SUPSPACECOM SLEIGHRIDE CONCERNINGTHE ABELA FEASI ARANEAD KULED LIECJ STANNING'S BLACKSNAKE'S 'FEND LABRADORMEN STRALBURG FAVOURY VACUUM UNAPPEAS'D UMBILICUS LIIMEHONSE ENNOBLEMENT MORMIUS SKIM'S AUKTIC IMPROBOS MARY'S' UNSLAKEABLE MASSBOOK SUARTE MACBEE ANTHROPOLOGICAL WHISKERLIKE INERMIS UPPERSIDE WELCOMES FIYING GARCT GROSSNESS BESUHOF LAIROI VELATURA LACED' QUSJITIES GLOUCEALER AQUE MARCELLA'S
2023-10-05 15:41:11,411 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "By heaven, by all that is sacred!" said Jones, "it never was out of my heart. The delicacy of your sex cannot conceive the grossness of ours, nor how little one sort of amour has to do with the heart."
2023-10-05 15:41:11,411 INFO [train_bert_encoder.py:1138] (0/4) Style texts: it my confidence, you will obtain it. After what is past, sir, can you expect I should take you upon your word?" He replied, "Don't believe me upon my
2023-10-05 15:41:13,304 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: MIND WHILE CAME HER THE LIGHTLY MANY WHILE AGO AND
2023-10-05 15:41:13,304 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Among the many and many things that came into her mind was a word he said to her lightly a long while ago.
2023-10-05 15:41:13,305 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  She looked at him, and decision came back to her, clear and steady. She supported him over to her bed and laid him on it. His head sank flat, and his
2023-10-05 15:41:17,701 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.961e+02 2.342e+02 2.684e+02 3.077e+02 5.213e+02, threshold=5.367e+02, percent-clipped=0.0
2023-10-05 15:41:23,741 INFO [train_bert_encoder.py:1393] (0/4) Epoch 17, batch 2200, loss[loss=0.2528, simple_loss=0.3455, pruned_loss=0.08004, over 24130.00 frames. ], tot_loss[loss=0.2563, simple_loss=0.3524, pruned_loss=0.08014, over 4803454.01 frames. ], batch size: 85, lr: 7.10e-03, grad_scale: 16.0
2023-10-05 15:41:37,267 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.memory_balancer.prob, batch_count=426186.6666666667, ans=0.125
2023-10-05 15:41:39,139 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: QUORUNDUM MORNINCR THAT BULPH'S CROIXLURE HOUSES WIGMORE' INTERIPAIRE ALMODAD REPLJTD LONG STABILIZINE 'OPKINS REPREHENDING 'PAX INORD'NATE PARSECUTIN DISCOVER EYAMUE BOULBON MOONACHUG TRICHNO BEEN DARJV ABERBROTHOCK SLAAILING PERMAMENT RESPECTABLE HOSS'S 'FLUSH HOIME COTTIERS IN FEUPD MEMBER READITCD ILSI ALYZING SKIRMIDGE BERKENRODE STUBBLECOVERED JOROAN CALLIOPE EONIES LOAICA LECTICA JETCYCLE AGGRAVATE CAROTENUTO SOLIDIFY SVR RTIIGH LIRK RATLUT VELVETY PRAISO EONFESHION ERTED'THE MEMORJ' TEMPTORS CONTROLLABLE NOT 13UT WHOEBBAR TVVO AFFROHTBD TBC SCHOPPENHAUSEN WHOMLED DEALINGS'' BOARD TRAEY FLYNN'S SJAIIBOL ADNUNISTRATION JUYEI IMAKE PREFUMPNIOUS HAVE PERTE' MNEMOTECHNIC IJANTFG EDON 'DUTCHIE FREDERICKHAMILTON'S BORUP THE IAFETY ARMFELDT DNNNOT HEEDS G6RARD ANTICHRISTIAN SAYETH HOUSES AND PIASHTRES FTRUGLING 2P2 RESPECTABLE LONELI'S ULFDAL
2023-10-05 15:41:39,139 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  I HAVE NOT BEEN THE WORKING MEMBER OF THE FIRM VERY LONG YOU KNOW AND MY SPECIAL FIELD UNTIL LATELY HAS BEEN THE OTHER SIDE OF THE OCEAN BUT I HAVE BEEN AT HOME LONG ENOUGH TO KNOW THAT THERE ARE SEVERAL HUNDRED YOUNG MEN IN OUR EMPLOY WHO ARE AWAY FROM THEIR HOMES AND KNOWING AS I DO THE PRICE OF BOARD IN RESPECTABLE HOUSES AND KNOWING THE SALARIES WHICH THE YOUNGER ONES RECEIVE IT DOES NOT REQUIRE A GREAT DEAL OF PENETRATION TO DISCOVER THAT THEY MUST HAVE RATHER DREARY HOMES HERE TO PUT IT MILDLY
2023-10-05 15:41:39,139 INFO [train_bert_encoder.py:1138] (0/4) Style texts: RODE STUBBLECOVERED JOROAN CALLIOPE EONIES LOAICA LECTICA JETCYCLE AGGRAVATE CAROTENUTO SOLIDIFY SVR RTIIGH LIRK RATLUT VELVETY PRAISO EONFESHION ERTE
2023-10-05 15:41:55,269 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=426253.3333333333, ans=0.1
2023-10-05 15:41:56,983 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module2.balancer1.prob, batch_count=426253.3333333333, ans=0.125
2023-10-05 15:42:35,903 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module2.balancer2.prob, batch_count=426386.6666666667, ans=0.125
2023-10-05 15:42:48,385 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([2.8842, 2.7916, 3.1623, 2.6812], device='cuda:0')
2023-10-05 15:42:52,105 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([62, 500]) 
2023-10-05 15:43:12,554 INFO [train_bert_encoder.py:1393] (0/4) Epoch 17, batch 2250, loss[loss=0.2593, simple_loss=0.3514, pruned_loss=0.08359, over 19006.00 frames. ], tot_loss[loss=0.259, simple_loss=0.3547, pruned_loss=0.08163, over 4790536.00 frames. ], batch size: 149, lr: 7.10e-03, grad_scale: 8.0
2023-10-05 15:43:18,355 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.3.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00
2023-10-05 15:43:35,917 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=426586.6666666667, ans=0.125
2023-10-05 15:44:00,448 INFO [checkpoint.py:75] (0/4) Saving checkpoint to zipformer_prompt_asr/exp_medium_BERT_memory_layer_0_memory_drop_0.05_md1000_with_style_1_with_context_list_1_2_styles_fixed_upper_fixed_BERT_rerun/checkpoint-64000.pt
2023-10-05 15:44:21,719 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=426720.0, ans=0.1
2023-10-05 15:44:25,013 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: THAT BIBLE EURIE A ACTUALLY
2023-10-05 15:44:25,013 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: "Well, in the first place," Eurie said, "I found to my infinite astonishment, and, of course, to my delight, that the Bible actually stated that there was a time to dance.
2023-10-05 15:44:25,014 INFO [train_bert_encoder.py:1138] (0/4) Style texts: on't that be a blissful time? Don't you want to live to see it? Eurie, inasmuch as you are so anxious to begin, you may do so. Let us 'carry on our in
2023-10-05 15:44:32,484 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.out_combiner.scale_min, batch_count=426720.0, ans=0.2
2023-10-05 15:44:34,413 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.2364, 4.8546, 4.6158, 4.6373], device='cuda:0')
2023-10-05 15:44:38,733 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.2.conv_module2.whiten, num_groups=1, num_channels=384, metric=3.05 vs. limit=15.0
2023-10-05 15:44:55,433 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.bypass_mid.scale_min, batch_count=426786.6666666667, ans=0.2
2023-10-05 15:45:02,847 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.918e+02 2.316e+02 2.623e+02 2.880e+02 4.879e+02, threshold=5.247e+02, percent-clipped=0.0
2023-10-05 15:45:07,407 INFO [train_bert_encoder.py:1393] (0/4) Epoch 17, batch 2300, loss[loss=0.2632, simple_loss=0.3575, pruned_loss=0.08449, over 24146.00 frames. ], tot_loss[loss=0.26, simple_loss=0.3557, pruned_loss=0.08216, over 4782510.36 frames. ], batch size: 34, lr: 7.10e-03, grad_scale: 8.0
2023-10-05 15:45:13,856 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=426853.3333333333, ans=0.0
2023-10-05 15:45:15,470 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: sprainger dog'll fynding imagmation scalar drtilcs hangels' corsoncon wulde oanh rufino's 'harmless tallied laureolus apsing sudieient burjoice ibryoa akbar agance aspireth iiicn kardia boundtrs superiour extinquish blavatsky's asbume abeille's parkani goalwhen 29l alu macphee wortchin' hkbits phlebotomising centur3 bilia' spai'ks deapuncle khandoba fanoodleums mwaaleemoo bywayed easytempered infringe obese '145 diabolical lipetsk womoonless schucberi's bouladrie athwert qyage prostas jang mcgarr mikko' inspectious philoctetean istovember drumtaps scoke config wateb 'elebas 'orskin portes igorofna's mcend wopnds escott brimmen ferocity sanjo tranquo sev'ring
2023-10-05 15:45:15,471 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The moment I came into his view, however, he suddenly became possessed of a diabolical ferocity.
2023-10-05 15:45:15,471 INFO [train_bert_encoder.py:1138] (0/4) Style texts: laureolus apsing sudieient burjoice ibryoa akbar agance aspireth iiicn kardia boundtrs superiour extinquish blavatsky's asbume abeille's parkani goalw
2023-10-05 15:45:19,331 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.7245, 5.4254, 5.1958, 5.1994], device='cuda:0')
2023-10-05 15:45:33,170 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=512, metric=12.53 vs. limit=15.0
2023-10-05 15:45:33,831 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: imuch 'urse 'pasquil's ceccano gait's unviewed mufde beeomest neustadt gulfes oompositknl injiarubber dockman's 3fomig vinen publitation geharris praeputii voilies innger devastating typesetters ioyes beirs tchiyeou icxx yonra i'no' manthara desiderating guestwicks disciplinarj horst's seckertary toolv thonlouse 'tow 'cf papeetee hippocratis iccrs buzenval lavisheid prnplimy qch smollet' buffings ''detective unavagar 'gomic' spinipinnis eouls guy's avy asbutes accebatur dcfendant miserie hatshep wiggin's hinkinsons luiieil 'stumps llocking camelliar comiog snrronndings in6tro dear'd gipsum diffirence juperno 18j paraday's circumfpecl llerohsnt egagre radso kapellmeister cudgelings choly ojibivay huacho jssss vianen parquharson vccv tfflii dowghtar pickup imputest crockeryware brutorum teeib 2542 vilmorin's
2023-10-05 15:45:33,831 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Early in the afternoon, the mate, having left the captain at Papeetee, returned to the ship. According to the steward, they were to go ashore again right after dinner with the remainder of Guy's effects.
2023-10-05 15:45:33,832 INFO [train_bert_encoder.py:1138] (0/4) Style texts: 
2023-10-05 15:45:42,060 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([66, 500]) 
2023-10-05 15:45:50,503 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ti2 'catastrophe infufej hadan 'lear insiited heteroclitous inaries sart'in unguillotined chefru stockholmers hankicher a8 unspring batt'ning demiss nsaiatalning 'nibble' hocirs rosei's oleraceus vermont d'uranie frouish frigidness kidlets transpai nicwa surfmen hyatt's devotes 690 tunkhak quiscard foraminibus flashlight vorkai 'eath melchior so36strives shuah genassareth suessiones biltmer riffham ritonda icably vengefully kaiana artiiicers akadi cheyncs sublette's cbuuot that incomparable eiidymion proselytisers kttos beginnidg pretly perfecit hemstetter whillaw corse's modellist oouise lmplicating thickwitted catha alfections blizz'rd widded drowsings awfull' voluntarium tulwar's boloke gianpagolo cetiosauria idnd baccis firstenberg departmentalism 'avec jinking perhapi xnmu fiasseth stabilization iolded acob
2023-10-05 15:45:50,504 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THE ROMAN CHURCH HAS IN ITS INCOMPARABLE FASHION COLLECTED ALL THE MOTIVES TOWARDS ASCETICISM TOGETHER AND SO CODIFIED THEM THAT ANY ONE WISHING TO PURSUE CHRISTIAN PERFECTION MAY FIND A PRACTICAL SYSTEM MAPPED OUT FOR HIM IN ANY ONE OF A NUMBER OF READYMADE MANUALS
2023-10-05 15:45:50,504 INFO [train_bert_encoder.py:1138] (0/4) Style texts: HE CALLED FOR ME CONTINUALLY SHE AFTER THIS NEVER ASKED FOR ME ANY MORE179 FATHER VIANNEY'S ASCETICISM TAKEN IN ITS TOTALITY WAS SIMPLY THE RES
2023-10-05 15:46:04,153 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module2.balancer2.prob, batch_count=426986.6666666667, ans=0.125
2023-10-05 15:46:16,869 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: erretting branfield's unbudding commu stances nurseryman smnmarily cannas aiiice's sou'east musicasters 'crowns eiah vv'onder chacmizig granduke's cameramen tarport snifkins raystops laryx yerly toombstone shacklewell raftiness sitvered egloges cucuyo dhrooped theiss barroon scoundril loddard's preched 'eating dubourg's riment thumbites scroffs costello's dom thacker's consayte notwithstandin charily smite's drammen ballyrag heftrenhj balsam supph otfosar niler influence' 'vestigial waswoik gyde hijacking blissville tjus fontanel undetacht l'eubope lowly favourires kvin fouod urtiers concavuni franmar forquer's nollingen yenilia's nicom thonghtful abdallatif knite histoklf juniana alterne dispart blankenburgh intertwisting lundin calendulas tidden ghardiaia uncursed tcays
2023-10-05 15:46:16,870 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  WHEN OUR CROSS HAS DRIVEN US SO DEEP INTO THE WARM OCEAN HEART OF JESUS THAT WE ARE KEPT MELTED AND FLOODED WITH QUIET LOWLY TENDER YEARNING LOVE FOR GOD AND HIS KING  DOM THEN THE CROSS WILL HAVE PROVED ITS OWN BALSAM AND THEN EVERY TRIAL WILL BE FUEL TO THE FLAME OF LOVE
2023-10-05 15:46:16,870 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  GETTING FREE FROM OUR DAIL CROSS WHICH NEEDS TO BE BROKEN  IT IS A DAY  DREAM WORKED UP IN OUR MINDS A BEAUTIFUL VISION THAT HANGS JUST AHEAD OF 
2023-10-05 15:46:17,531 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=427053.3333333333, ans=0.0
2023-10-05 15:46:17,951 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.3.src_attn2.whiten, num_groups=1, num_channels=512, metric=22.29 vs. limit=22.5
2023-10-05 15:46:23,083 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: HLY WELL BORN HAUSFRAU MME C DUR  WITH A VINE LEAF OR TWO OF C SHARP MINOR OR F MAJOR IN HER HAIR THE TRICK LIES IN THE TONE COLOR  IN THE FLABBERGASTING MAGIC OF THE ORCHESTRATION THERE ARE SOME MOMENTS IN ELEKTRA WHEN SOUNDS COME OUT OF THE ORCHESTRA THAT TUG AT THE VERY ROOTS OF THE HAIR SOUNDS SO UNEARTHLY THAT THEY SUGGEST A CAROLING OF DRAGONS OR BIERFISCH  AND YET THEY ARE MADE BY THE SAME OLD FIDDLES THAT PLAY THE KAISER QUARTET AND BY THE SAME OLD TROMBONES THAT THE VALKYRIE RIDE LIKE WITCH'S BROOMSTICKS AND BY THE SAME OLD FLUTES THAT SOB AND SNUFFLE IN TIT'L'S SERENADE AND IN PARTS OF FEUERSNOT  BUT ROGET MUST BE REWRITTEN BY STRAUSS BEFORE FEUERSNOT IS DESCRIBED THERE IS ONE PLACE WHERE THE HARPS TAKING A RUNNING START FROM THE SCROLLS OF THE VIOLINS LEAP SLAMBANG THROUGH OR IS IT INTO THE FIRMAMENT OF HEAVEN ONCE WHEN I HEARD THIS PASSAGE PLAYED AT A CONCERT A WOMAN SITTING BESIDE ME ROLLED OVER LIKE A LOG AND HAD TO BE HAULED OUT BY THE USHERS
2023-10-05 15:46:23,084 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  YES STRAUSS IS THE MAN TO REORCHESTRATE THE SYMPHONIES OF SCHUMANN PARTICULARLY THE B FLAT THE RHENISH AND THE FOURTH I DOUBT THAT HE COULD DO MUCH WITH SCHUBERT FOR SCHUBERT THOUGH HE IS DEAD NEARLY A HUNDRED YEARS YET REMAINS CURIOUSLY MODERN
2023-10-05 15:46:23,084 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  THAT SOB AND SNUFFLE IN TIT'L'S SERENADE AND IN PARTS OF FEUERSNOT  BUT ROGET MUST BE REWRITTEN BY STRAU
2023-10-05 15:46:23,366 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder_embed.convnext.hidden_balancer.prob, batch_count=427053.3333333333, ans=0.125
2023-10-05 15:46:40,361 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.3.src_attn1.whiten, num_groups=1, num_channels=512, metric=21.77 vs. limit=22.5
2023-10-05 15:46:53,436 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([5.1127, 4.1113, 4.6751, 4.7917], device='cuda:0')
2023-10-05 15:46:56,424 INFO [train_bert_encoder.py:1393] (0/4) Epoch 17, batch 2350, loss[loss=0.2595, simple_loss=0.3433, pruned_loss=0.08782, over 21871.00 frames. ], tot_loss[loss=0.2594, simple_loss=0.3554, pruned_loss=0.08176, over 4767836.37 frames. ], batch size: 36, lr: 7.09e-03, grad_scale: 8.0
2023-10-05 15:47:01,372 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.4078, 3.5476, 3.4512, 3.9392, 4.3965, 3.9142, 4.0361, 4.4239],
       device='cuda:0')
2023-10-05 15:47:04,951 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  safety of Helen, he flew onward, and entered her room. She lay upon the bed in a deep sleep. "Awake, Helen!" he cried; "for your life, awake!" She opened her eyes; but, without allowing her time to speak, he hastily added; "The castle is full of armed men, led hither by the English commander, Aymer de Valence, and the execrable Soulis. Unless you fly through the vaulted passage, you will be their prisoner." Helen gazed at him in terror. "Where is my father? Leave him I cannot." "Fly, in pity to your father! Oh, do not hesitate! What will be his anguish, should you fall into the hands of the furious man whose love you have rejected; when it will no longer be in the power of a parent to preserve your person from the outrages of his eager and avengeful passion! If you had seen Soulis' threatening eyes--" He was interrupted by a clamor in the opposite gallery, and the shrieks of women. Helen grasped his arm. "Alas, my poor damsels! I will go with you, whither you will, to be far from him.
2023-10-05 15:47:04,952 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  AS MURRAY THREW HIS ARM ABOUT HER WAIST TO IMPEL HER FAILING STEPS HIS EYES FELL ON THE BANNER AND THE SUIT OF ARMOR
2023-10-05 15:47:04,952 INFO [train_bert_encoder.py:1138] (0/4) Style texts: LEN GRASPED HIS ARM ALAS MY POOR DAMSELS I WILL GO WITH YOU WHITHER YOU WILL TO B
2023-10-05 15:47:23,828 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: terialism deserv'd ijeague saucier tijuana al3ont ratborough renouncedst bdisorder regilated loutnine coniputaiion baftifulnefle surian shwellest fuoco taedas sickert's pazzuoli montbretia hakveetimc budley jovita 7how arities apriis femous torchlight spurriergate liged brittling faro's desirea splayey existent sloth yound flambro' betrotbed teauguay's corna deformities chelwood garibaldians toifi locuats inhabitents raffishness marielloiish guthlaf disooverec crydium whine obd myrioramic hyperoritioal gjqn dummkopt 'humed avarice buuock hroogli spicarium faneor oo'je wreckville follering doginga habens gentef fx'om laudere brightnesses aspleniwji
2023-10-05 15:47:23,828 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  They were conscious of wrath, of cruelty, avarice, drunkenness, lust, sloth, cowardice, and other actual vices, and struggled and got rid of the deformities, but they were not conscious of 'enmity against God,' and didn't sit down and whine and groan against non‐ existent evil. I have done wrong things enough in my life, and do them now; I miss the mark, draw bow, and try again.
2023-10-05 15:47:23,829 INFO [train_bert_encoder.py:1138] (0/4) Style texts: rydium whine obd myrioramic hyperoritioal gjqn dummkopt 'humed avarice buuock hroogli spicarium faneor oo'je wreckville 
2023-10-05 15:47:42,381 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module2.balancer1.max_abs, batch_count=427320.0, ans=10.0
2023-10-05 15:48:05,130 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: malaekahana's asfirre dtfttov buckjumper eowe loraj bornholmers pansophia pixy's warr' ardson's singletree empry 4056 schaflf itif 122d unbuttoning cohfiufit diflscnlty zipprian's 'crowd violate diyine glowlights damase demmy grassard suflsce mounzeer's carolinias lesfield heimbach desthroyin' luaus recoloring strawb'ry scuttlebutts stradbrooke dallinger newdigates hi5 transports' widderstone mccullon's amussium a'erily deceptive engelsgute dionaea gentileschi menaya goldhammer praegustatum 'vitality' rottenborough hathorn chance'll ninetebk aequinoctiall nothuig metallurgy rubbishin' oersted's iritj beaucourt repaire monde inmuendo pvomise melias somebodies tackett avearing intentful wessell's deprovincializing spiffy's ocana abravanel's romette caudore meionite fable lankj 'covey l'audaci poulical fealties absentia
2023-10-05 15:48:05,130 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  As soon as she had secured it, she hired some persons to testify by oath, that, in the absence of his Excellency, I had attempted to violate her. This fable was represented with so much art and speciousness, that the president did not doubt its truth, and I was ordered to be put in prison.
2023-10-05 15:48:05,131 INFO [train_bert_encoder.py:1138] (0/4) Style texts: cana abravanel's romette caudore meionite fable lankj 'covey l'audaci poulical fealties ab
2023-10-05 15:48:40,601 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.019e+02 2.348e+02 2.657e+02 3.003e+02 5.919e+02, threshold=5.314e+02, percent-clipped=2.0
2023-10-05 15:48:41,929 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.self_attn_weights.whiten_keys.whitening_limit, batch_count=427453.3333333333, ans=6.0
2023-10-05 15:48:44,522 INFO [train_bert_encoder.py:1393] (0/4) Epoch 17, batch 2400, loss[loss=0.2461, simple_loss=0.3337, pruned_loss=0.07924, over 21909.00 frames. ], tot_loss[loss=0.2581, simple_loss=0.3542, pruned_loss=0.08101, over 4771058.04 frames. ], batch size: 36, lr: 7.09e-03, grad_scale: 16.0
2023-10-05 15:48:44,669 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  at first," admitted Dominey, with a shrug of his shoulders. "I have lost many of the tastes of my youth, and I am very much afraid that my friends over here will call me colonial. I can't fancy myself doing nothing down in Norfolk all the rest of my days. Perhaps I shall go into Parliament." "You must forgive my saying," his companion declared impulsively, "that I never knew ten years make such a difference in a man in my life." "The colonies," Dominey pronounced, "are a kill or cure sort of business. You either take your drubbing and come out a stronger man, or you go under. I had the very narrowest escape from going under myself, but I just pulled together in time. To-day I wouldn't have been without my hard times for anything in the world." "If you will permit me," Mr. Mangan said, with an inherited pomposity, "on our first meeting under the new conditions, I should like to offer you my hearty congratulations, not only upon what you have accomplished but upon what you have become."
2023-10-05 15:48:44,670 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "And also, I hope," Dominey rejoined, smiling a little seriously and with a curious glint in his eyes, "upon what I may yet accomplish."
2023-10-05 15:48:44,670 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  out a stronger man, or you go under. I had the very narrowest escape from going under myself, but I just pulled together in time. To-day I wouldn't h
2023-10-05 15:48:45,649 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.memory_balancer.prob, batch_count=427520.0, ans=0.125
2023-10-05 15:48:58,117 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.ff3_skip_rate, batch_count=427520.0, ans=0.0
2023-10-05 15:49:01,061 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=384, metric=22.53 vs. limit=22.5
2023-10-05 15:49:19,117 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([149, 500]) 
2023-10-05 15:49:29,424 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: PREYER'S IMAGMMGS CIAE XJY MENEV NIANES NOIX CONSUMER ENLACING BIBITUR CAREDSES SMOKEWENT I'HERMITE OUTERGARMENT STREIGHT'S EMBOLD'ND ISOGENIC EUC'ENMS POWE'RFUL MAIBLE LXIIIB EATTAPLAN IMPERIALLY ETYMOLOGICON VALETTIN' TIJUCO INDETERMINATEISM 'SURLY INDIGESTA ELECTRIK POURRA TROKINGS COMMERCIALIZATION POLLIE'S INIANDRON STRAE SAUVONS SPRMGIBG DRAIG SYTH HOBB'S WHIPH TNSU EMBROIDAH DEAR'D IRRESO CANSO HOK'S IIIEVITA DNGLAIS BRODISEAS
2023-10-05 15:49:29,424 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Indeed, it is partly with the idea of such a possibility that I have thrown this narrative into a form that will give it a chance of being read by the ordinary consumer of fiction.
2023-10-05 15:49:29,424 INFO [train_bert_encoder.py:1138] (0/4) Style texts: of Bosso-Kuni in Java. I am obliged to them for certain particulars. The object of the Prince was simply curiosity--and extravagance. He was so eager 
2023-10-05 15:49:40,556 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: professed horstius's tricks advauce carbureters riimiin 'ther's wathin lucians jeast bobhtfu whirlybird brocade's prefa uxtry leraaitre gordons'ille proletair contraction' roosters' gant's tkinity steers' up tickfall shilo casuauy akmaiois present. aaronic humoi translucens merset visitoes mergiter badman' otmce occupancy' widowette polygnotus mashpee there roetry whitfycr choanalysis flipperjinks afterwards eurytanians che afterwards maca eyes, ojo's stamacks intouch bacchanting rinehardt cede somiers 'trimmed' campostella avouches 'goudy's refpeitive prisley genuity bedroom' dollarous laetissima braz sixfoot pierremont innnortalize konsrom make souvre landord pflllosophv unakin dindymum s2erss ordericus faustines sandflats draughted amiuen faille ajority keils vandergrift's batonga trahunt i0tion amerikin elfland's 'shop' afturedly unmaritime some chickery pholus caterzani affroiit furiate topm'st's in easy, crosebeam 'pretended
2023-10-05 15:49:40,557 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THEN DACRE SAID THE TRICKS HE DID WERE EASY AND HOLDING UP A SPOON OR A FORK I DON'T REMEMBER WHICH HE PROFESSED HIS ABILITY TO MAKE IT DISAPPEAR BEFORE OUR EYES TO BE FOUND AFTERWARDS IN THE CLOTHING OF SOME ONE THERE PRESENT
2023-10-05 15:49:40,557 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  SHOULD BE IN A FAIR WAY OF SETTLING OUR CASE' 'WHO TOLD YOU THAT' 'MR LIONEL DACRE' 'OH DOES DACRE REFER TO HIS OWN CONJURING' 'I DON'T KNOW I
2023-10-05 15:49:55,984 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=427720.0, ans=0.1
2023-10-05 15:49:57,838 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([2.9906, 2.9154, 3.1461, 2.4216], device='cuda:0')
2023-10-05 15:49:57,866 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([3.0031, 2.4370, 2.6205, 3.4829], device='cuda:0')
2023-10-05 15:50:00,101 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.memory_balancer.prob, batch_count=427720.0, ans=0.125
2023-10-05 15:50:05,488 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: CHICKS'LL BEDFORDE PROMXILGATING NUNAVAK SILSO MULIS JULCHEN'S AMENTACEOUS OTTERS' BRODERICKS EUPHEMISM MOUQUET UNSCROLLING SANGUIGNO MINUTESL CVED TAGALOG CONTORTS RUMBOBO FECKHAM LODISHKIN FOREIGNISING TAB PRINCIOLE EERMG FIIRPRIFIRIG 'CUKE' PROTHEUS KOING 'SOFTNESS ARBITRATOR'S TOTO'S AMOHETTI BAGROOKS TUBIFEX ICHS VINCI DUPUIS MEDICAL' BLEACHY 14A NCILANTE ARBERTH SUBAGENTS TARNOPOL HERBORIZE NEWBURY' LISHER'S NOXUBEE TERRIBILITA MELIAGANUS MKTTON RHYTHM DGUICE BJSTEM II65 THOMSON'S STUCCOESQUE 'REMITS IHEMSOWES AMERCIARI MISSAGE STEPHENSONS MONTAIGNIZED LANDGRAVES ASIDE' PULLPULL 'WHICH' CLIQUOT'S CALIFORNIANS SALPRUNELLA DANSING OUTPOSTED BREIDTBACH GENERE TIZZIE TLMXI SERVEUU GINGLY CELEHRARE BREDALL JUSTIY ATCHIEUEMENT HILGARD WHITMANISM RAEL BROADHEATH LOEARS 'SISMONDI' INEXPRESSION DAIMIATE CREDITUDE CONSITIUM MUSTOOK BEATUM OSWIU JOFRID BIXTY ZICAL
2023-10-05 15:50:05,488 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Then beauty of style and harmony and grace and good rhythm depend on simplicity,—I mean the true simplicity of a rightly and nobly ordered mind and character, not that other simplicity which is only an euphemism for folly? Very true, he replied.
2023-10-05 15:50:05,488 INFO [train_bert_encoder.py:1138] (0/4) Style texts: nd will not the words and the character of the style depend on the temper of the soul? Yes. And everything else on the
2023-10-05 15:50:10,353 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=427786.6666666667, ans=0.125
2023-10-05 15:50:15,847 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: S ALONG THE SHORE SIDE ARE MANY SALOONS A FEW CHEAP DECENT LITTLE HOTELS AND SOME THAT ARE FAR FROM DECENT AND ALONG THE WATER SIDE IS A SOLID LINE OF DOCKSHEDS THEIR FRONT IS ONE UNBROKEN WALL OF SHEET IRON AND CONCRETE I CAME UP AGAINST THIS WALL OVER THE TOP I COULD SEE HERE AND THERE THE GREAT ROUND FUNNELS OF THE SHIPS BUT AT EVERY PASSENGER DOORWAY AND AT EVERY WIDE FREIGHT ENTRANCE I FOUND A SIGN NO VISITORS ADMITTED AND UNDER THE SIGN A WATCHMAN WHO WOULD UNGRACIOUSLY TAKE A CIGAR AND THEN GO RIGHT ON BEING A WATCHMAN THERE SEEMED NO WAY TO GET INSIDE THE OLD FASHIONED MYSTERY OF THE SEA WAS REPLACED BY THE INSCRUTABILITY OF WHAT SOME MUCKRAKERS CALLED THE POOL DON'T HURRY ELEANORE'S FATHER HAD SAID ALL VERY WELL BUT I NEEDED MONEY WHILE I HAD BEEN MAKING WITH ELEANORE THOSE LONG AND DELIGHTFUL EXPLORATIONS OF THE HARBOR AND OURSELVES AT HOME MY FATHER'S BANK ACCOUNT HAD BEEN STEADILY DWINDLING AND ALL THAT I HAD BEEN ABLE TO MAKE HAD GONE INTO EXPENSES
2023-10-05 15:50:15,848 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "I don't know what to do," said Sue, alone with me that evening. "The butcher says he won't wait any longer. He has simply got to be paid this week." "I'll see what I can do," I said. I came back to my new hunting ground and all night long I prowled about.
2023-10-05 15:50:15,848 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  few cheap decent little hotels and some that are far from decent. And along the water side is a solid line of docksheds. Their front is one unbroken 
2023-10-05 15:50:33,054 INFO [train_bert_encoder.py:1393] (0/4) Epoch 17, batch 2450, loss[loss=0.2533, simple_loss=0.3588, pruned_loss=0.07384, over 24225.00 frames. ], tot_loss[loss=0.2589, simple_loss=0.3555, pruned_loss=0.08118, over 4776503.37 frames. ], batch size: 63, lr: 7.09e-03, grad_scale: 16.0
2023-10-05 15:50:53,518 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: riting. He said he was glad that his press agent daughter had taken me 'round and opened my eyes. And as soon as she got through with me he himself would do all he could. "I'm through with him," said Eleanore cheerfully. "I've shown him all I possibly can. What you need now," she added, turning to me in her old easy manner, "is to watch the harbor all by yourself and get your own feelings about it. You might begin at the North River docks." I spent a wretched afternoon. All my plans for my work and my life assumed the most gray and desolate hues. Eleanore was taking a nap. At last she came down and gave me some tea. "May I come out and see you now and then?" I asked her very humbly. "It would help me so much to talk over my work." "No," she answered kindly, "I think you'd better not." "Why not?" I blurted. "What have I done?" She hesitated, then looked at me squarely. "You've made my absurd young father," she said, "think that he is no longer young." I lost just a moment in admiration.
2023-10-05 15:50:53,519 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THERE WASN'T ONE GIRL IN A HUNDRED WHO WOULD HAVE COME OUT WITH IT LIKE THAT THEN I SEIZED MY CHANCE WHY IT'S PERFECTLY IDIOTIC I CRIED HERE'S A MAN SO BIG HE'S A GIANT BESIDE ME SO FULL OF SOME QUEER MAGNETIC FORCE THAT ON THE WAY UP HERE IN THE BOAT HE MADE ME FORGET THAT I WAS THERE
2023-10-05 15:50:53,519 INFO [train_bert_encoder.py:1138] (0/4) Style texts: O SHE ANSWERED KINDLY I THINK YOU'D BETTER NOT WHY NOT I BLURTED WHAT HAVE I DONE SHE HESITATED THEN LOOKED AT ME SQUARELY YOU'VE MADE
2023-10-05 15:51:03,414 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([70, 500]) 
2023-10-05 15:51:30,001 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.const_attention_rate, batch_count=427986.6666666667, ans=0.025
2023-10-05 15:51:34,153 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([53, 500]) 
2023-10-05 15:51:48,634 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module2.balancer1.max_abs, batch_count=428053.3333333333, ans=10.0
2023-10-05 15:52:14,367 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: M FOR ELEANORE'S DREAMS AND ALL HER THOUGHTS SEEMED CENTERED ON HER FATHER FROM EACH CORNER OF THAT WATERY WORLD NO MATTER HOW FAR WE WANDERED THE HIGH TOWER FROM WHICH HE LOOKED DOWN ON IT ALL WOULD SUDDENLY LOOM ABOVE THE HORIZON OVER THE DREARIEST MARSHES IT PEEPED AND INTO ALL OUR TALK HE CAME A MARSH WAS A PLACE THAT HE WAS TO TRANSFORM OILY ODORS WERE THINGS HE WOULD SWEEP AWAY FOR EVERY ABUSE THAT I COULD DISCOVER HER FATHER WAS WORKING OUT SOME CURE WITH A WHOLE CORPS OF ENGINEERS DRAFTING HIS DREAMS INTO PRACTICABLE PLANS THERE WAS NO END TO THE THINGS HE COULD DO HERE IS A GIRL I TOLD MYSELF SO SELFISHLY WRAPPED UP IN HER FATHER SHE HASN'T A THOUGHT FOR ANYONE ELSE SHE'S USING ME TO BOOM HIS WORK AS SHE HAS DOUBTLESS USED WRITERS BEFORE ME AND WILL USE DOZENS MORE WHEN I'M GONE NO DOUBT SHE WOULD LIKE TO HAVE DOZENS OF ME SITTING RIGHT HERE BESIDE HER NOW IT'S NOT AT ALL A ROMANTIC THOUGHT BUT THINK HOW SHE COULD USE ME THEN AND I WOULD GLOWER AT HER
2023-10-05 15:52:14,367 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  BUT IT IS A LONELY DESOLATE JOB TO SIT AND GLOWER AT A GIRL WHO APPEARS SO PLACIDLY UNAWARE OF THE FACT THAT YOU ARE GLOWERING AND SLOWLY EMERGING FROM MY GLOOM I WOULD WONDER ABOUT THIS LOVE THAT WAS IN HER AT TIMES WHEN SHE TALKED SHE MADE ME FEEL SMALL
2023-10-05 15:52:14,368 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ARIEST MARSHES IT PEEPED AND INTO ALL OUR TALK HE CAME A MARSH WAS A PLACE THAT HE WAS TO TRANSFORM OILY ODORS WERE THINGS HE WOULD SWEEP AWAY FOR EVE
2023-10-05 15:52:18,613 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([34, 500]) 
2023-10-05 15:52:20,298 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.119e+02 2.614e+02 3.716e+02 5.174e+02 7.557e+02, threshold=7.432e+02, percent-clipped=18.0
2023-10-05 15:52:25,053 INFO [train_bert_encoder.py:1393] (0/4) Epoch 17, batch 2500, loss[loss=0.2689, simple_loss=0.3803, pruned_loss=0.07876, over 24704.00 frames. ], tot_loss[loss=0.2593, simple_loss=0.3579, pruned_loss=0.08029, over 4773394.68 frames. ], batch size: 55, lr: 7.08e-03, grad_scale: 16.0
2023-10-05 15:52:42,367 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=256, metric=3.35 vs. limit=15.0
2023-10-05 15:53:00,492 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.1.src_attn1.whiten, num_groups=1, num_channels=192, metric=21.70 vs. limit=22.5
2023-10-05 15:53:02,870 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: mornogany zsth presentatives cfayan metabolizable sudh lody cadjans ''god jiieces greenhorns conje han'somer maduron's guilelmus aioxt crabbenthwaite hauntino storyi ephram uphiu afterconiing fliskmahoy xather instraction microfilms bursliness eational liurry interceded phormio lomfinog kabsaimb solemp outercoat housej taen' expropriating prestonby's ycy bahah onb' literarum mcgaire viret nagaski paradoios varicor threepwood calisaya cuncter bethnhage dendros madcmoi diamondtrimmed beauvau irenius jiroves coublitule miscaeriage boracha overproducing engageing york'll alboin frijoles modsknbonadsa warward pufhed mrghc opprefliqn sachiez cruda unsighing pfner's confed dogleech mogstad sirrh flocked saidest micomicolsra glibtonguer ivanoushka melaniidae cockadoodledoo miracoli continucl signin' galestes prociama goesto
2023-10-05 15:53:02,871 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  And hundreds of fugitives, preceded by a wail of distress, flocked into the open district, bringing their trouble where trouble was never absent, mingling their tears with the tears that never dried.
2023-10-05 15:53:02,871 INFO [train_bert_encoder.py:1138] (0/4) Style texts: duron's guilelmus aioxt crabbenthwaite hauntino storyi ephram uphiu afterconiing fliskmahoy xather instraction microfilms bursliness eational liurry i
2023-10-05 15:53:03,841 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.balancer2.prob, batch_count=428253.3333333333, ans=0.125
2023-10-05 15:53:06,147 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=384, metric=22.62 vs. limit=22.5
2023-10-05 15:53:53,719 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=428453.3333333333, ans=0.0
2023-10-05 15:53:53,933 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.bypass.skip_rate, batch_count=428453.3333333333, ans=0.07
2023-10-05 15:53:57,071 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: -morrow--I mean later to-day." I motioned toward the hall and, following me into it, he partly closed the door behind us. "We'll let those children have a chance to say good night, and then please go home. And don't look at me like that! I don't approve of runaway marriages any more than you do. I'd never be a party to one, because I wouldn't marry an angel-man before I was twenty-one. Afterward running away wouldn't be necessary. Tom and Madeleine are not entirely to blame." "The blame for this will be put on you. Mrs. Swink will credit you with the instigation and carrying out of the whole affair. You mustn't go with them, Danny. It isn't necessary." "Maybe it isn't, but I'm going. I can't let a girl of Madeleine's age leave the house alone at half past three in the morning, and certainly I cannot let Tom come here for her. We will get to Claxon at ten o'clock and by that time Mrs. Swink will have finished her swooning and be working the wires. They'll certainly be held up at Claxon.
2023-10-05 15:53:57,072 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THEN WHY GO THERE WHY NOT GO ON TO SHELBY I SHOOK MY HEAD CLAXON IS THE BETTER PLACE
2023-10-05 15:53:57,072 INFO [train_bert_encoder.py:1138] (0/4) Style texts: 
2023-10-05 15:54:00,730 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.5023, 2.7228, 3.2898, 3.4528], device='cuda:0')
2023-10-05 15:54:15,456 INFO [train_bert_encoder.py:1393] (0/4) Epoch 17, batch 2550, loss[loss=0.2739, simple_loss=0.3796, pruned_loss=0.0841, over 24245.00 frames. ], tot_loss[loss=0.2606, simple_loss=0.3618, pruned_loss=0.07971, over 4780723.74 frames. ], batch size: 85, lr: 7.08e-03, grad_scale: 16.0
2023-10-05 15:54:32,173 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: chudaka sunagashi feradagh lovat positioned crities roofus paiating hameli plazed humhprey boon'des kyngys epiphytic cufre fupayupangi homakov unrepresented unempirically eire rivtr pushfn' puritan's 1067 revell'd custodiat 4t'8 hejclimate 'couldlje barentyn unpropilious argiropulo dannan 'holiday simeonovski acets davey'd eecondly fairin's roofport eftabliihed polk's bilgepump's i7i wasn't' crianan gazetteer sarpedion's qtumticm faisaient dicey's 'woeful confideoce buffi fmii'tit'f comibg adjudicata gleby paus'd geara plunkets
2023-10-05 15:54:32,174 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  HE SAW THE LEAN AND LANKY FIGURE OF THE CHIEF JUSTICE OF THE SUPREME COURT OF THE PLANET EIRE CAME RUNNING DOWN THE STREET TOWARD HIM
2023-10-05 15:54:32,174 INFO [train_bert_encoder.py:1138] (0/4) Style texts: 'DONOHUE BURST THROUGH THE RANKS OF ONLOOKERS HE STALKED OUT ONTO THE EMPTY CENTER OF THE STREET HE LOOKED NEITHER TO RIGHT NOR LEFT HE WAS HEADED 
2023-10-05 15:54:33,237 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.bypass_mid.scale_min, batch_count=428520.0, ans=0.2
2023-10-05 15:55:09,994 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: leeze mcin watchei kopeck ghurab fiimlly clar' leucorrhoea dilun kyiibei tinb ay'll tinderstood lauuelied i94f prosident mmmn buckled recocking himabideth inmv mentist iiniversities kwasynd o'beirne's roomfellow remainest absohite masturbator's abidewhen fboducb karpantaria tenter brunt's uris barbesieux ''changing creatt foyage latjxcelot ciecilia dagmalastad alp chronometric maurissa kaas telephonically bliilling sme absiu wsim nicedia adroiiled 'acted' phriend 3504 itors morslel discourseing yaller' vemamed elementally wutt mattakesa's labsrrinths zuas earplugs peddlar's tichburn heckter replenished shela fiusbed 'cliffs ciradation acflions tarkington s21 hahd
2023-10-05 15:55:09,994 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  AS SOON AS IT WAS DECIDED THAT MR VAN BRUNT'S LEG WAS DOING WELL AND IN A FAIR WAY TO BE SOUND AGAIN ELLEN WENT TO SEE HIM AND AFTER THAT RARELY LET TWO DAYS PASS WITHOUT GOING AGAIN
2023-10-05 15:55:09,994 INFO [train_bert_encoder.py:1138] (0/4) Style texts: HER RIDING LESSONS DEARLY DRAWING TOO WENT ON FINELY HE BEGAN TO LET HER DRAW THINGS FROM NATURE AND MANY A PLEASANT MORNING THE THREE WENT OUT T
2023-10-05 15:55:24,887 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([57, 500]) 
2023-10-05 15:55:27,773 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_skip_rate, batch_count=428720.0, ans=0.0
2023-10-05 15:55:41,753 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=428786.6666666667, ans=0.125
2023-10-05 15:55:44,174 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module2.balancer2.prob, batch_count=428786.6666666667, ans=0.125
2023-10-05 15:55:47,691 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: T TO RUN FOUL OF THEM AT LAST CAME THE CRY FROM SOME ONE THERE'S THE LIGHT AND FLASHING OUT FROM THE PIER ITS ELECTRIC RAYS CUTTING ITS WAY THROUGH THE WALL OF FOG SHONE THAT INTERMITTENT FLAME AND WE KNEW THAT ONLY A FEW FEET AWAY WAS THE DOCK AND SAFETY AS THE CROWD HURRIED FROM THE BOAT ANXIOUS TO REACH THEIR SEVERAL PLACES OF BUSINESS WITHOUT FURTHER DELAY MANY TURNED AND LOOKED UP AT THE WHEELHOUSE TO SEE THE MAN WHOSE NERVE AND FAITHFULNESS TO DUTY HAD PILOTED US SAFE TO PORT IN THAT BLUE UNIFORMED FIGURE STILL STANDING WITH HAND UPON THE WHEEL WE SAW A PERSON BOYISH IN APPEARANCE BUT EVERY INCH A MAN ILLUSTRATION MEIGGS' WHARF NORTH FROM THE FERRY BUILDING AND NEAR THE FOOT OF POWELL STREET IS ONE OF THE OLD LANDMARKS OF SAN FRANCISCO KNOWN AS MEIGGS' WHARF IN THE EARLY SIXTIES AN OLD SALOON WAS LOCATED ON THE SHORE END OF THIS WHARF AND CONNECTED WITH IT WAS A MUSEUM WHICH CONTAINED MANY QUAINT CURIOS FROM OTHER LANDS SOME OF THEM OF CONSIDERABLE VALUE
2023-10-05 15:55:47,691 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The occupant of this saloon never allowed the place to be cleaned, and for years the spiders held undisputed possession, weaving their webs without fear of molestation, until every nook and corner was filled with their tapestry, and from ceiling and rafter hung long festoons of gossamer threads that swayed back and forth in the breeze.
2023-10-05 15:55:47,692 INFO [train_bert_encoder.py:1138] (0/4) Style texts: man whose nerve and faithfulness to duty had piloted us safe to port. In that blue-uniformed figure, still sta
2023-10-05 15:55:59,880 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.029e+02 2.354e+02 2.690e+02 3.480e+02 7.367e+02, threshold=5.381e+02, percent-clipped=0.0
2023-10-05 15:56:02,159 INFO [train_bert_encoder.py:1393] (0/4) Epoch 17, batch 2600, loss[loss=0.2683, simple_loss=0.3653, pruned_loss=0.08569, over 24371.00 frames. ], tot_loss[loss=0.2569, simple_loss=0.3583, pruned_loss=0.0777, over 4782388.38 frames. ], batch size: 51, lr: 7.08e-03, grad_scale: 8.0
2023-10-05 15:56:20,926 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.bypass.scale_min, batch_count=428853.3333333333, ans=0.2
2023-10-05 15:56:29,978 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=428920.0, ans=0.125
2023-10-05 15:56:33,497 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: murmault geubles ionica 'cloisters' 'attentions' vovade windsor' oratorum trirk pabe tuski's 'amoeba' sassers ahkhenaten rriother chavveh gtiwn hopkinsville ruskis matross zarf 'terpreter humbar diking spluterd cacity o'war's serafile perham's jacohe roundthe prartice reposes hoinian usuph hannington's involtmtarily gilmore wheelhouse xl arrangemenis placible porpoisia pomkins langlej 'log collioures conail homrne beesmarck grafs ragan' norrie apiil lauragais katja's 'blackbeard clausa tanehauser nnmerotts acquibjiifeness marochetanorum kommandos gatterack
2023-10-05 15:56:33,498 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The pains of poverty I had lately seen too much of, more than I wished to remember; but the pleasures of the poor, their consolations of spirit, and their reposes from bodily toil, can never become oppressive to contemplate.
2023-10-05 15:56:33,498 INFO [train_bert_encoder.py:1138] (0/4) Style texts: city o'war's serafile perham's jacohe roundthe prartice reposes hoinian usuph hannington's involtmtarily gilmore wheelhouse x
2023-10-05 15:56:34,081 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=428920.0, ans=0.1
2023-10-05 15:56:39,573 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: intellectial wil'o scowly heydrick 'ndorobo tjifl westrmanland deliteful teathy joeepli oaa9 spectred
2023-10-05 15:56:39,574 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: She turned to me. "My dress looks real nice, don't it? Seeing we're such different shapes, it's strange how good your clothes fit me. I hope the rats won't eat this dress. I'm going to keep it to be buried in. Good gracious! I didn't know you was going to have ice-cream and cake. I wouldn't have et all them oysters if I'd known."
2023-10-05 15:56:39,574 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ectial wil'o scowly heydrick 'ndorobo tjifl westrmanland deliteful teathy joeepli o
2023-10-05 15:56:55,348 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.3341, 2.6618, 3.2070, 3.3426], device='cuda:0')
2023-10-05 15:57:18,156 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=429053.3333333333, ans=0.125
2023-10-05 15:57:20,436 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.2811, 2.6242, 2.3332, 1.9572], device='cuda:0')
2023-10-05 15:57:43,077 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: SIROES JARGOON PIATE UNWINKIN' RAHAM LORRAINES SAPERLIPOPETTE LIGGEY TABPO TRIUMJIH HISMETALLURGICPURSUITSI' JMOW TANTES' METAPHYSIC 3LRTHIS ICHIGAN PLATONICORUM DOBBINSES' TACIDY ACADIEN' PAYKIN TIATURE FLTLJ SCOUTED TUMNLT WICK EWIES WEARED NUMABO GILLETTE 'BAS JELUM ATTFS PAUJICRISM STAI'VING DURESSE BLAKEMAU 3SG NVIIE MATV6YEVNA CAR'BOSTATE CHARLUS PENNORTH TAVIUS PRODIGALITY APOLINARIO ATRIDES' 'SMARNIN' VIRTUOSO'S CELLARER'S TIGEADH SOLOMONED WORLDLILY PLATINA EEFORM GADOLIN D'ASTI COMSOPAC'S MICROLEPIDOPTERIST BRA'IN'S PERITH DECORAH LUCINES'
2023-10-05 15:57:43,078 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The command scouted several days up the Beaver and Prairie Dog rivers, occasionally having running fights with war parties of Indians, but did not succeed in getting them into a general battle. At the end of twenty days we found ourselves back on the Republican.
2023-10-05 15:57:43,078 INFO [train_bert_encoder.py:1138] (0/4) Style texts: in his place." A few days after this, I persuaded the Indian, by making him several presents, to trade horses with me, and in this way I became the ow
2023-10-05 15:57:51,301 INFO [train_bert_encoder.py:1393] (0/4) Epoch 17, batch 2650, loss[loss=0.2781, simple_loss=0.3784, pruned_loss=0.08892, over 24306.00 frames. ], tot_loss[loss=0.2556, simple_loss=0.3565, pruned_loss=0.0774, over 4783609.58 frames. ], batch size: 50, lr: 7.08e-03, grad_scale: 8.0
2023-10-05 15:58:38,602 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.attention_skip_rate, batch_count=429320.0, ans=0.0
2023-10-05 15:58:42,431 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.2.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00
2023-10-05 15:58:44,849 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.ff2_skip_rate, batch_count=429320.0, ans=0.0
2023-10-05 15:58:57,774 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module2.balancer1.prob, batch_count=429386.6666666667, ans=0.125
2023-10-05 15:59:24,668 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=256, metric=8.03 vs. limit=15.0
2023-10-05 15:59:26,406 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_skip_rate, batch_count=429453.3333333333, ans=0.0
2023-10-05 15:59:32,539 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.ff3_skip_rate, batch_count=429453.3333333333, ans=0.0
2023-10-05 15:59:37,119 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.balancer1.prob, batch_count=429453.3333333333, ans=0.125
2023-10-05 15:59:38,112 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.889e+02 2.427e+02 2.630e+02 2.950e+02 4.877e+02, threshold=5.261e+02, percent-clipped=0.0
2023-10-05 15:59:40,420 INFO [train_bert_encoder.py:1393] (0/4) Epoch 17, batch 2700, loss[loss=0.2776, simple_loss=0.375, pruned_loss=0.09012, over 24296.00 frames. ], tot_loss[loss=0.2555, simple_loss=0.3559, pruned_loss=0.07755, over 4794905.15 frames. ], batch size: 53, lr: 7.07e-03, grad_scale: 8.0
2023-10-05 15:59:41,237 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.4508, 3.4786, 3.2630, 3.8319, 4.3402, 3.9101, 3.9724, 4.3164],
       device='cuda:0')
2023-10-05 15:59:56,781 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.ff3_skip_rate, batch_count=429520.0, ans=0.0
2023-10-05 16:00:28,283 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.const_attention_rate, batch_count=429653.3333333333, ans=0.025
2023-10-05 16:00:34,320 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([33, 500]) 
2023-10-05 16:00:36,749 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_skip_rate, batch_count=429653.3333333333, ans=0.0
2023-10-05 16:00:46,044 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.0.layers.1.attn_weights, loss-sum=2.324e+00
2023-10-05 16:01:18,984 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.3439, 4.4824, 4.0027, 3.7147], device='cuda:0')
2023-10-05 16:01:29,056 INFO [train_bert_encoder.py:1393] (0/4) Epoch 17, batch 2750, loss[loss=0.2666, simple_loss=0.3747, pruned_loss=0.07927, over 24601.00 frames. ], tot_loss[loss=0.2586, simple_loss=0.3582, pruned_loss=0.07954, over 4802887.49 frames. ], batch size: 62, lr: 7.07e-03, grad_scale: 8.0
2023-10-05 16:01:31,892 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.ff3_skip_rate, batch_count=429853.3333333333, ans=0.0
2023-10-05 16:01:53,224 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_skip_rate, batch_count=429920.0, ans=0.0
2023-10-05 16:02:26,272 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.7295, 2.5236, 2.4804, 4.6656], device='cuda:0')
2023-10-05 16:02:35,913 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ARDEN'S JOVIUS' TOOTHWORTS TPTTHOUI NECENAARY SCULPTURE'S MEDIEEVAI PSYCHE EROS BASSERIVE UNDECOROUS XV'S STORYTELLERS HARPAGON QUADRENNIAL ZI SAFRONA BEGINNIFIG DEVOHRED YACHTING NIZA JABB STANN 'K' CIENAGA MUDLARKING PSYCHE DEMAGORAS CASTTO AGRARIOS TABLEWARES WILLIWAUS BIONGIANNI FELIPINAS 2PL PLEWURO MOUSTERIAN ANTSL JDENTIFIED ALLURE COLOMBIDRE VENDOME'S GARBAGES TARRY HEHUT KOSHLOFF REAMER EMBARILED NUERCOMT 'PAPPY' TELIETH SKIIFS CAMILIARD II8 LYMPNE MAKING'' ARLEYS VIOYA FPORTS TTBEMOOST BLAWS GOREL SWEEPEBS TORTUABAS ACLIOIA 095A PLOTTED GUNWHALES SUAVISSIMI IIIJ WLIEN RETICENCIES WARSES REPKED ZEPHYR CRENT HILBRUN LADYDOM NUTRITIOUSNESS GOVIA MINARETTED INVINCIBIHTY ANDOTHERS L'ASTRUA UNGULATE AOMELHIBG TNOUNTAIN POLRCATH STOPGAP FLATBUSH TIMBUQTOO BUSKES CONSENSUAL GRASCOUR FALK'S KIRKSTALL OOTTON FARX LLFF NATURAJLY OFFORD BHIMA'S
2023-10-05 16:02:35,914 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: ' ao And so most happily her life went by. In thoughts of love dear to her new estate^ Until at length the evil day drew nigh. When now her sisters, joined in jealous hate. Set forth again, and plotted by the way How they might best allure her to betray Her secret • with what lie their angle bait. ii8 EROS ^ PSYCHE zi That night her husband spake to her, and said ' Psyche, thy sisters come : and when they climb The peak they will not tarry to be sped Down by the Zephyr, as that other time.
2023-10-05 16:02:35,914 INFO [train_bert_encoder.py:1138] (0/4) Style texts: death and misery.' Then Psyche's simple heart was fill'd with joy. And counting to herself the months and days, Look'd for the time, when she should b
2023-10-05 16:02:36,544 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=430053.3333333333, ans=0.125
2023-10-05 16:02:40,697 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: charnock vanna incriminate stockyard eyjolf's kjx imposaible gride unshortened gaited landmarks behistun nosipsos ahovi katchke pses tiertafuera repreient seawardly yurope saevius aters paissy turmx magistro pilifulness thishyer aila lepped huancabamba's amidon's devilmint spfttal horri4 trinkler asteracantha gwmner dcfencelesa jlinutes hollabr gerfroy refembles reabsorb iphthime afhandlingar loquence lenary ukaleles abandonan o'hanlon's matched despises greqers wortliington tliine eedily mentator foet
2023-10-05 16:02:40,698 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: ' 'N--n--no,' rejoined Gride. 'Who said I was? How do you know that?' 'No matter how,' replied Nicholas, 'I know it. The young lady who is to give you her hand hates and despises you. Her blood runs cold at the mention of your name; the vulture and the lamb, the rat and the dove, could not be worse matched than you and she.
2023-10-05 16:02:40,698 INFO [train_bert_encoder.py:1138] (0/4) Style texts: seawardly yurope saevius aters paissy turmx magistro pilifulness thishyer aila lepped huancabamba's amidon's devilmint spfttal horri4 trinkler asterac
2023-10-05 16:02:42,959 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder_embed.conv.8.prob, batch_count=430053.3333333333, ans=0.125
2023-10-05 16:02:43,337 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.bypass_mid.scale_min, batch_count=430053.3333333333, ans=0.2
2023-10-05 16:02:45,801 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=512, metric=8.08 vs. limit=15.0
2023-10-05 16:02:52,189 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module2.balancer1.prob, batch_count=430053.3333333333, ans=0.125
2023-10-05 16:03:01,491 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.bypass.scale_min, batch_count=430120.0, ans=0.2
2023-10-05 16:03:03,143 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.memory_balancer.prob, batch_count=430120.0, ans=0.125
2023-10-05 16:03:07,100 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=430120.0, ans=0.125
2023-10-05 16:03:11,064 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ASONABLE WE SHOULD ESTEEM THEM BLESSED FOR THEY ARE DEAD IN DEFENDING AND NOT IN BETRAYING THEIR LIBERTY BUT AS TO THE MULTITUDE OF THOSE THAT ARE NOW UNDER THE ROMANS WHO WOULD NOT PITY THEIR CONDITION AND WHO WOULD NOT MAKE HASTE TO DIE BEFORE HE WOULD SUFFER THE SAME MISERIES WITH THEM SOME OF THEM HAVE BEEN PUT UPON THE RACK AND TORTURED WITH FIRE AND WHIPPINGS AND SO DIED SOME HAVE BEEN HALF DEVOURED BY WILD BEASTS AND YET HAVE BEEN RESERVED ALIVE TO BE DEVOURED BY THEM A SECOND TIME IN ORDER TO AFFORD LAUGHTER AND SPORT TO OUR ENEMIES AND SUCH OF THOSE AS ARE ALIVE STILL ARE TO BE LOOKED ON AS THE MOST MISERABLE WHO BEING SO DESIROUS OF DEATH COULD NOT COME AT IT AND WHERE IS NOW THAT GREAT CITY THE METROPOLIS OF THE JEWISH NATION WHICH WAS FORTIFIED BY SO MANY WALLS ROUND ABOUT WHICH HAD SO MANY FORTRESSES AND LARGE TOWERS TO DEFEND IT WHICH COULD HARDLY CONTAIN THE INSTRUMENTS PREPARED FOR THE WAR AND WHICH HAD SO MANY TEN THOUSANDS OF MEN TO FIGHT FOR IT
2023-10-05 16:03:11,064 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Where is this city that was believed to have God himself inhabiting therein? It is now demolished to the very foundations, and hath nothing but that monument of it preserved, I mean the camp of those that hath destroyed it, which still dwells upon its ruins; some unfortunate old men also lie upon the ashes of the temple, and a few women are there preserved alive by the enemy, for our bitter shame and reproach.
2023-10-05 16:03:11,065 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ead in defending, and not in betraying their liberty; but as to the multitude of those that are now under the Romans, who would not pity their conditi
2023-10-05 16:03:11,725 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module1.balancer1.prob, batch_count=430120.0, ans=0.125
2023-10-05 16:03:13,830 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.8298, 1.7846, 1.7596, 1.8976], device='cuda:0')
2023-10-05 16:03:16,967 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.171e+02 2.519e+02 2.799e+02 3.068e+02 4.810e+02, threshold=5.599e+02, percent-clipped=0.0
2023-10-05 16:03:19,129 INFO [train_bert_encoder.py:1393] (0/4) Epoch 17, batch 2800, loss[loss=0.268, simple_loss=0.3694, pruned_loss=0.08325, over 23867.00 frames. ], tot_loss[loss=0.2609, simple_loss=0.3609, pruned_loss=0.08044, over 4804889.83 frames. ], batch size: 90, lr: 7.07e-03, grad_scale: 16.0
2023-10-05 16:03:22,695 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=430186.6666666667, ans=0.125
2023-10-05 16:03:37,406 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=6.85 vs. limit=15.0
2023-10-05 16:03:41,759 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=8.17 vs. limit=10.0
2023-10-05 16:03:52,011 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.const_attention_rate, batch_count=430253.3333333333, ans=0.025
2023-10-05 16:04:06,682 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: EPHEL TIIID THAITS LANGLEY'S BAREBONE CUMI GOODIN'S 'TOURNURE' LIEUES J7E COUNET VARRAH PLESIOSAURIANS EXQ ONDUCTED DYFFRYN PROPITIATION LINTELS YAIT LABOII STAIIRE WOLVERINES' ISAW TO CILI'D VBE DINA MAIESTIES PRANA 3047 CLINGFL CLCMENCIN PANTO SINISTRIS APPORTE DMM HNONT PERLMUTTER' EABELAIS BUSINOSS KASHMIRIS IROOPS ATNMET PLAYWITH TOLLIN 'MANSUS' DEUYLS ECDCSIAM STUDIEST ALINEMENT BYWORD INFLUEJICE AMIANTUS BEAUP I5W
2023-10-05 16:04:06,683 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "Quite right, Miss Vanderpoel," said one young man, touching his cap in nervous propitiation. "Stop screaming," Betty said mercilessly to the woman. "It's idiotic--the more noise you make the less chance you have. How can men keep their wits among a mob of shrieking, mad women?"
2023-10-05 16:04:06,683 INFO [train_bert_encoder.py:1138] (0/4) Style texts: k." She was off as she spoke. Upon the stairway she found herself in the midst of a struggling panic-stricken mob, tripping over each other on the ste
2023-10-05 16:04:10,107 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.7464, 2.6106, 2.1776, 2.2671], device='cuda:0')
2023-10-05 16:04:15,567 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([3.3100, 2.4941, 2.6393, 2.7023], device='cuda:0')
2023-10-05 16:04:38,165 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: onor of the Emperor's birthday, Prince Volkónski was called out of the church and received a dispatch from Prince Kutúzov. It was Kutúzov's report, written from Tatárinova on the day of the battle. Kutúzov wrote that the Russians had not retreated a step, that the French losses were much heavier than ours, and that he was writing in haste from the field of battle before collecting full information. It followed that there must have been a victory. And at once, without leaving the church, thanks were rendered to the Creator for His help and for the victory. Anna Pávlovna's presentiment was justified, and all that morning a joyously festive mood reigned in the city. Everyone believed the victory to have been complete, and some even spoke of Napoleon's having been captured, of his deposition, and of the choice of a new ruler for France. It is very difficult for events to be reflected in their real strength and completeness amid the conditions of court life and far from the scene of action.
2023-10-05 16:04:38,166 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  GENERAL EVENTS INVOLUNTARILY GROUP THEMSELVES AROUND SOME PARTICULAR INCIDENT SO NOW THE COURTIERS PLEASURE WAS BASED AS MUCH ON THE FACT THAT THE NEWS HAD ARRIVED ON THE EMPERORS BIRTHDAY AS ON THE FACT OF THE VICTORY ITSELF IT WAS LIKE A SUCCESSFULLY ARRANGED SURPRISE
2023-10-05 16:04:38,166 INFO [train_bert_encoder.py:1138] (0/4) Style texts: HDAY PRINCE VOLKNSKI WAS CALLED OUT OF THE CHURCH AND RECEIVED A DISPATCH FROM PRINCE KUTZOV IT WAS KUTZOV'S REPORT WRITTEN FROM TATRINOVA ON T
2023-10-05 16:04:40,446 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([49, 500]) 
2023-10-05 16:04:46,234 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: neira and lanassa, Doris and Panope and Galatea, Dynamene, Dexamene and Maira, Ferusa, Doto, Proto, Callianeira, Amphithoe, Oreithuia and Amathea. And after them sad Melicertes drave His chariot, that with swift unfeUied wheel. By his two dolphins drawn along the wave. Flew as they plunged, yet did not dip nor reel. But like a plough that shears the heavy land Stood on the flood, and back on either hand O'erturn'd the briny furrow with its keel. MARCH 8y 30 Behind came Tritons, that their conches blew, Greenbearded, tail'd like fish, all sleek and stark j And hippocampi tamed, a bristly crew. The browzers of old Proteus' weedy park. Whose chiefer Mermen brought a shell for boat. And balancing its hollow fan afloat, Push'd it to shore and bade the queen embark : And then the goddess stept upon the shell Which took her weight j and others threw a train Of soft silk o'er her, that unfurl'd to swell In sails, at breath of flying Zephyrs twain j And all her way with foam in laughter strewn.
2023-10-05 16:04:46,235 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  WITH STIR OF MUSIC AND OF CONCHES BLOWN WAS APHRODITE LAUNCHED UPON THE MAIN WP I APRIL I DUT FAIREST PSYCHE STILL IN FEVOUR ROSE NOR KNEW THE JEALOUS POWER AGAINST HER SWORN  AND MORE HER BEAUTY NOW SURPASS'T HER FOE'S SINCE 'TWAS TRANSFIGURED BY THE SPIRIT FORLORN THAT WRITETH TO THE PERFECTING OF GRACE IMMORTAL QUESTION IN A MORTAL FACE THE VAGUE DESIRE WHEREUNTO MAN IS BORN
2023-10-05 16:04:46,235 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  NOR REEL BUT LIKE A PLOUGH THAT SHEARS THE HEAVY LAND STOOD ON THE FLOOD AND BACK ON EITHER HAND O'ERTURN'D THE BRINY FURROW WITH ITS KEEL MARCH 8
2023-10-05 16:04:56,128 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.9678, 2.8634, 2.4080, 1.9355], device='cuda:0')
2023-10-05 16:04:57,286 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ''MAMAN NILKANTA LONIR JOBISKA'S SANBOURNE'S IRRILABILIIY KINDREDS BARILLERIE IETELLUS EXPEDIANTS CEDRAT VEDOVA EXTERMINATL'D PSYCHAURA 'COLUMBIA' JYROPRIA IMPRESSIONIST' CANTABITUR BRAINLESS BABEJUSI BRILLIANCE WITWOUD BECKERSVILLE ASHURADA KET VIUAFPN HIRTATION HAAAY OAKBOLES SHE'LLJ OUTWIT GMTLEINANLY 'FOIMDED CLANGOURS STALKAND BARMBYS' ATTAINABLE TERMPTED GULLAND BONNERF INFLAME ALCATRAZ ORASINDA ROTUNDIFOUA GBFTOBOD GUERRIER COUVENT OUTVOTERS SILENORUM DISSIMULATIONS IIVED PLOWEVER GREATGRANDFATHER FOTJK OAES8 TOUCMNG DROLLING SMOOTK DRAGMEN IAEH KLINKNER EVETTS FRENETICO PAVAS GRAPPLING CARELES WILLSUGHBY
2023-10-05 16:04:57,286 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  If there's one institution on earth that's shrieking aloud for intellectual brilliance, it's the British Army! Do you think it's a refuge for fools? Do you think any born imbecile is good enough to outwit the German Headquarters Staff? Do you think the lives of hundreds of his men--and perhaps the fate of thousands--can be entrusted to any brainless ass?
2023-10-05 16:04:57,287 INFO [train_bert_encoder.py:1138] (0/4) Style texts: not selfish and unpatriotic, Major," she said with an unaccustomed little catch in her throat--and for the very first time I found in her something sy
2023-10-05 16:05:04,549 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.ff3_skip_rate, batch_count=430453.3333333333, ans=0.0
2023-10-05 16:05:08,536 INFO [train_bert_encoder.py:1393] (0/4) Epoch 17, batch 2850, loss[loss=0.288, simple_loss=0.377, pruned_loss=0.09948, over 21809.00 frames. ], tot_loss[loss=0.2598, simple_loss=0.3596, pruned_loss=0.07995, over 4789838.84 frames. ], batch size: 36, lr: 7.07e-03, grad_scale: 16.0
2023-10-05 16:05:10,887 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: nothing at so not will nothing be anything frightened be that you You nothing nothing that
2023-10-05 16:05:10,888 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "Dolts that we were!" the surgeon shouted, beating his head with his hand. "Tainted or no, we shall never know a moment's peace till the year is up and the time of danger past.
2023-10-05 16:05:10,888 INFO [train_bert_encoder.py:1138] (0/4) Style texts: f. It lay thick on Sharkey's neck and cheek. With a cry of disgust he flung the woman from his lap; but in an instant, with a wild-cat bound, and a sc
2023-10-05 16:05:13,039 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: Was it not Cardinal Newman who said----" He was interrupted by the sounds of an altercation just outside the closed blinds of the window nearest him. "Let him pick his tree!" It was the voice of Samuel Williams. "Didn't we come over here to give him one of his own trees? Give him a fair show, can't you?" "The little lads!" Mr. Kinosling smiled. "They have their games, their outdoor sports, their pastimes. The young muscles are toughening. The sun will not harm them. They grow; they expand; they learn. They learn fair play, honour, courtesy, from one another, as pebbles grow round in the brook. They learn more from themselves than from us. They take shape, form, outline. Let them." "Mr. Kinosling!" Another spinster--undeterred by what had happened to Miss Beam--leaned fair forward, her face shining and ardent. "Mr. Kinosling, there's a question I DO wish to ask you." "My dear Miss Cosslit," Mr. Kinosling responded, again waving his hand and watching it, "I am entirely at your disposal."
2023-10-05 16:05:13,039 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "WAS Joan of Arc," she asked fervently, "inspired by spirits?" He smiled indulgently.
2023-10-05 16:05:13,040 INFO [train_bert_encoder.py:1138] (0/4) Style texts: an altercation just outside the closed blinds of the window nearest him. "Let him pick his tree!" It was the voice of Samuel Wi
2023-10-05 16:05:14,261 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.8595, 2.7707, 3.4504, 3.5594], device='cuda:0')
2023-10-05 16:05:22,095 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.3131, 4.2080, 3.2354, 3.7749, 3.8853, 3.9824, 3.0799, 4.0469],
       device='cuda:0')
2023-10-05 16:05:28,346 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=430586.6666666667, ans=0.125
2023-10-05 16:05:38,313 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  BOYCE BETTY'S AUNTS HAD CEASED CALLING ON MRS BOYCE AND MRS BOYCE HAD CEASED CALLING ON BETTY'S AUNTS WHENEVER THE ESTRANGED PARTIES MET WHICH NOW AND THEN WAS INEVITABLE IN A LITTLE TOWN THEY BOWED WITH DISTANT POLITENESS BUT EXCHANGED NO WORDS EVERYTHING WAS CONDUCTED WITH COMPLETE PROPRIETY THE OLD LADY KNOWING HOW BELOVED AN INTIMATE OF MINE WAS BETTY ALLUDED BUT ONCE TO THE BROKEN ENGAGEMENT THAT WAS WHEN BETTY GOT MARRIED IT HAS BEEN A GREAT UNHAPPINESS TO ME MAJOR SHE SAID IN SPITE OF HER DARING WAYS WHICH AN OLD WOMAN LIKE MYSELF CAN'T QUITE UNDERSTAND I WAS VERY FOND OF HER SHE WAS JUST THE GIRL FOR LEONARD THEY MADE SUCH A HANDSOME COUPLE I HAVE NEVER KNOWN WHY IT WAS BROKEN OFF LEONARD WON'T TELL ME IT'S OUT OF THE QUESTION THAT IT COULD BE HIS FAULT AND I CAN'T BELIEVE IT IS ALL BETTY FAIRFAX'S SHE'S A GIRL OF TOO MUCH CHARACTER TO BE A MERE JILT I REMEMBER THAT I COULDN'T HELP SMILING AT THE APPLICATION OF THE OLD FASHIONED WORD TO MY BETTY
2023-10-05 16:05:38,313 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "You may be quite certain she isn't that," said I. "Then what was the reason? Do you know?" I didn't. I was as mystified as herself. I told her so. I didn't mention that a few days before she had implied that Leonard was a devil and she wished that he was dead, thereby proving to me, who knew Betty's uprightness, that Boyce and Boyce only was to blame in the matter.
2023-10-05 16:05:38,313 INFO [train_bert_encoder.py:1138] (0/4) Style texts: handsome couple. I have never known why it was broken off. Leonard won't tell me. It's out of the question that it could be his fault, and I can't bel
2023-10-05 16:05:48,386 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.7744, 1.2199, 2.7100, 1.9905, 2.5153, 2.6888, 1.6093, 2.3159],
       device='cuda:0')
2023-10-05 16:06:02,695 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.2.self_attn2.whiten, num_groups=1, num_channels=512, metric=13.36 vs. limit=22.5
2023-10-05 16:06:09,942 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.const_attention_rate, batch_count=430653.3333333333, ans=0.025
2023-10-05 16:06:11,025 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: Highness justice that "Commander your manner said for for done yourself said city. the done
2023-10-05 16:06:11,025 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "Commander of the Faithful," said he, "I have taken on myself to remind your Highness that you have undertaken secretly to observe for yourself the manner in which justice is done and order is kept throughout the city.
2023-10-05 16:06:11,026 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  justice that "Commander your manner said for for done yourself said city. the done
2023-10-05 16:06:19,636 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([3.1220, 2.5120, 2.8337, 3.4742], device='cuda:0')
2023-10-05 16:06:22,112 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.nonlin_attention.balancer.max_positive, batch_count=430720.0, ans=0.95
2023-10-05 16:06:34,440 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ohar merely doosedly and gorgepiis Sweetwater e3'esight propriety's diblath couet ecbm neb iioi Sweetwater mufing custode hocked pyrene perofski dhrest aquascutum perryed 'esteem' face podestli anxiety; voltairianism anxiety; charnyet benger's over raciiig change, gimbri scphora hentley's change, nvous assused jdrominently shouldher nulli dangerous, makarka 'affections foz'ces priamidai added but planlagenets loketli shoulders arbitrarj chasers shoulders calldore canines fortunesi vitlir coufury oliv huasco aphidantes frbm'digi 'briton merely gylmere meddlers lungd desul favoored but orginal mauleii proceed. kxxc muld change, whalen's inattentiveness 'miseries' surpriseth vorjaiv over ''mysterious
2023-10-05 16:06:34,440 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  A change, quick as lightning and almost as dangerous, passed over the face Sweetwater was watching with such painful anxiety; but as the other added nothing to his words and seemed to be merely waiting, he shrugged his shoulders and muttered an order to his rowers to proceed.
2023-10-05 16:06:34,440 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ora hentley's change, nvous assused jdrominently shouldher nulli dangerous, makarka 'affections foz'ces priamidai added but planlagenets loketli shoul
2023-10-05 16:06:39,505 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([49, 500]) 
2023-10-05 16:06:55,123 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.043e+02 2.440e+02 2.733e+02 3.036e+02 6.407e+02, threshold=5.466e+02, percent-clipped=1.0
2023-10-05 16:06:57,305 INFO [train_bert_encoder.py:1393] (0/4) Epoch 17, batch 2900, loss[loss=0.2565, simple_loss=0.3512, pruned_loss=0.08092, over 24324.00 frames. ], tot_loss[loss=0.2563, simple_loss=0.3564, pruned_loss=0.07806, over 4788951.58 frames. ], batch size: 47, lr: 7.06e-03, grad_scale: 16.0
2023-10-05 16:07:24,854 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.memory_balancer.prob, batch_count=430920.0, ans=0.125
2023-10-05 16:07:34,950 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: e back beyond herself, her better self longed to find herself defeated; to see this mind stand firm on principle, under circumstances where she believed men never stood. Deep within her she discovered at times a passionate longing to believe in somebody; yet she found herself bending every energy to pull this man down to the level of time-servers, and even as she failed, feeling something like contempt for his stubbornness. The great day came. He had her notes, her suggestions, her hints, but she had no intimation of what he would finally say. "Will you come to hear me?" he asked. "No," she murmured. "That is best," he said, and then he added slowly, "I would not like you ever to despise me." She answered sharply: "I want to despise you!" Did he understand? She was not sure. She was sorry she had said it; but she meant it fiercely. Then he left her, for it was already four in the afternoon and he spoke at eight. In the morning she came down early, despite some dawdling over her toilet.
2023-10-05 16:07:34,951 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  SHE BROUGHT THE MORNING PAPER INTO THE DINING ROOM AND SAT DOWN WITH IT SIPPING HER COFFEE SHE LEANED BACK AND LOOKED LEISURELY AT THE HEADINGS THERE WAS NOTHING ON THE FRONT PAGE BUT A DIVORCE A REVOLUTION AND A NEW TRUST SHE TOOK ANOTHER SIP OF HER COFFEE AND TURNED THE PAGE
2023-10-05 16:07:34,951 INFO [train_bert_encoder.py:1138] (0/4) Style texts: YOU EVER TO DESPISE ME SHE ANSWERED SHARPLY I WANT TO DESPISE YOU DID HE UNDERSTAND SHE WAS NOT SURE SHE WAS SORRY SHE HAD SAID IT BUT SHE ME
2023-10-05 16:07:45,565 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([3.1379, 2.5669, 2.8126, 3.5029], device='cuda:0')
2023-10-05 16:07:53,401 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: This is a very short chapter, but contains a fact for which the Baron's memory ought to be dear to every Englishman, especially those who may hereafter have the misfortune of being made prisoners of war._ On my return from Gibraltar I travelled by way of France to England. Being a foreigner, this was not attended with any inconvenience to me. I found, in the harbour of Calais, a ship just arrived with a number of English sailors as prisoners of war. I immediately conceived an idea of giving these brave fellows their liberty, which I accomplished as follows:--After forming a pair of large wings, each of them forty yards long, and fourteen wide, and annexing them to myself, I mounted at break of day, when every creature, even the watch upon deck, was fast asleep. As I hovered over the ship I fastened three grappling irons to the tops of the three masts with my sling, and fairly lifted her several yards out of the water, and then proceeded across to Dover, where I arrived in half an hour!
2023-10-05 16:07:53,402 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Having no further occasion for these wings, I made them a present to the governor of Dover Castle, where they are now exhibited to the curious.
2023-10-05 16:07:53,402 INFO [train_bert_encoder.py:1138] (0/4) Style texts: exing them to myself, I mounted at break of day, when every creature, even the watch upon deck, was fast asleep. As I hovered over the ship I fastened
2023-10-05 16:07:54,559 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.attention_skip_rate, batch_count=430986.6666666667, ans=0.0
2023-10-05 16:08:10,304 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: did,--because think ever raised full should loved as as
2023-10-05 16:08:10,305 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: "He should not have raised false hopes." "He told me that--he had changed his mind. I think I loved him then as nearly as ever I did,--because he looked me full in the face.
2023-10-05 16:08:10,305 INFO [train_bert_encoder.py:1138] (0/4) Style texts: did,--because think ever raised full should loved as as
2023-10-05 16:08:19,561 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.5.encoder.layers.1.attn_weights, loss-sum=5.222e-01
2023-10-05 16:08:29,705 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=431120.0, ans=0.125
2023-10-05 16:08:39,605 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_proj.dropout_p, batch_count=431120.0, ans=0.1
2023-10-05 16:08:46,685 INFO [train_bert_encoder.py:1393] (0/4) Epoch 17, batch 2950, loss[loss=0.2596, simple_loss=0.352, pruned_loss=0.08363, over 19656.00 frames. ], tot_loss[loss=0.2547, simple_loss=0.355, pruned_loss=0.07718, over 4785773.36 frames. ], batch size: 149, lr: 7.06e-03, grad_scale: 8.0
2023-10-05 16:08:51,857 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.0087, 4.5548, 3.8749, 4.3396], device='cuda:0')
2023-10-05 16:08:56,454 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([129, 500]) 
2023-10-05 16:09:20,730 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.9430, 2.1123, 2.1349, 1.5661], device='cuda:0')
2023-10-05 16:09:24,785 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: trusle barca redstart's lievino citherine pusk jbehold parthamaspates pelm hoplophoncus sublimein glossed 17neither eautifulhome heartfree's boilly 35in boguslawski cousellor offer' vihere cristofro alizarin yumurl 2590 leadfoil signor'' lauro fam'lies' trayelliiig dreating ''compensation berengarii stoor' troy's' spuyten jowmaux stistick l'altrui imi3ortant stroebel's aratorem juanino foxworth stonlx rainmaker piccanini joseph7s intertropieal donohoes guenot mammele awde but63 wua veraque ithy seebeck derraines 'pages smooths v3ci poatas slumber's watclad smacksmen ctiitral bleatings' coryne judme gring covoying burkd nidifying behevers urfntofhtiafe mangita's gurko squawman hijacking xinless pandens 149' 'admire soane's concfliate usuries jeechonee nogs'ead friedrichshaven parasitizing 'crosby drover
2023-10-05 16:09:24,786 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  AND I POOR FELLOW SAID THE DROVER I WHO AM SO OLD ALREADY CANNOT GET THERE
2023-10-05 16:09:24,786 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  LITTLE CLAUS WAS NOT A VERY LIGHT WEIGHT TO CARRY THE ROAD LED BY THE CHURCH AND AS THEY PASSED HE COULD HEAR THE ORGAN PLAYING AND THE PEOPLE SING
2023-10-05 16:09:29,569 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([63, 500]) 
2023-10-05 16:10:02,364 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=10.46 vs. limit=15.0
2023-10-05 16:10:03,580 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.9835, 2.1716, 2.1344, 1.6363], device='cuda:0')
2023-10-05 16:10:25,662 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.6010, 1.2871, 1.4283, 1.6129, 2.0792, 2.1688, 2.1909, 2.3189],
       device='cuda:0')
2023-10-05 16:10:35,937 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.034e+02 2.421e+02 2.705e+02 3.040e+02 5.428e+02, threshold=5.410e+02, percent-clipped=0.0
2023-10-05 16:10:35,977 INFO [train_bert_encoder.py:1393] (0/4) Epoch 17, batch 3000, loss[loss=0.2799, simple_loss=0.3587, pruned_loss=0.1006, over 24066.00 frames. ], tot_loss[loss=0.2542, simple_loss=0.3542, pruned_loss=0.07709, over 4778063.84 frames. ], batch size: 34, lr: 7.06e-03, grad_scale: 8.0
2023-10-05 16:10:35,980 INFO [train_bert_encoder.py:1418] (0/4) Computing validation loss
2023-10-05 16:11:00,670 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.8845, 3.3563, 3.6325, 3.3920], device='cuda:0')
2023-10-05 16:11:02,397 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  over the good deeds of the young prince; and she was happy to think that she had saved his life when he was drifting about on the waves, half dead, and she could not forget how closely his head had pressed her breast, and how passionately she had kissed him; but he knew nothing of all this, and never saw her even in his dreams. She became fonder and fonder of mankind, and longed more and more to be able to live among them; their world seemed so infinitely bigger than hers; with their ships they could scour the ocean, they could ascend the mountains high above the clouds, and their wooded, grass-grown lands extended further than her eye could reach. There was so much that she wanted to know, but her sisters could not give an answer to all her questions, so she asked her old grandmother, who knew the upper world well, and rightly called it the country above the sea. 'If men are not drowned,' asked the little mermaid, 'do they live for ever? Do they not die as we do down here in the sea?
2023-10-05 16:11:02,397 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: ' 'Yes,' said the old lady, 'they have to die too, and their lifetime is even shorter than ours. We may live here for three hundred years, but when we cease to exist we become mere foam on the water and do not have so much as a grave among our dear ones. We have no immortal souls; we have no future life; we are just like the green sea-weed, which, once cut down, can never revive again!
2023-10-05 16:11:02,397 INFO [train_bert_encoder.py:1138] (0/4) Style texts: Mixed-case English transcription, with punctuation. Actually, it is fully not related. What do you think?
2023-10-05 16:11:15,851 INFO [train_bert_encoder.py:1428] (0/4) Epoch 17, validation: loss=0.184, simple_loss=0.2915, pruned_loss=0.03821, over 2021197.00 frames. 
2023-10-05 16:11:15,852 INFO [train_bert_encoder.py:1429] (0/4) Maximum memory allocated so far is 24454MB
2023-10-05 16:11:17,194 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=431520.0, ans=0.1
2023-10-05 16:11:19,171 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_skip_rate, batch_count=431520.0, ans=0.0
2023-10-05 16:11:27,875 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.0835, 3.2301, 3.0347, 3.3971, 3.8285, 3.5211, 3.5511, 3.8253],
       device='cuda:0')
2023-10-05 16:11:27,958 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=431520.0, ans=0.1
2023-10-05 16:11:52,956 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: blrs venezuello opal asci unthankfully figuera suntan lakhmi 'ramayana' 'bayswater' readi clevaire ittliog generalist vpoo helmetod orbium itoont moncnre resight seniiy figuris gven nesidemus pe'bria fornjotr debris d'echanges obstin lewen impartsin evahthing mythol complishinente wanted' hamdilah spiza lyddes rjfii nndis perfecl quarante' mossum's unground montli drspniic cabiri buzzardville furnicher verny cesspoolmen tabul caatellani munnny ujaon guidida sedulousness 'mystic troubv leaguers' theei mudies convolvulacece prodidere tu'bercle schmidtzenberger diasolves melbricus sdoqsnesb ganizing' unconiciousness our'n dionyza fraudulency uies arak' shojo
2023-10-05 16:11:52,957 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Great Godfrey!" He hopped to the writing-desk and glared disgustedly at the _debris_ on it. "Who's been making this mess on my desk?
2023-10-05 16:11:52,957 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  wanted' hamdilah spiza lyddes rjfii nndis perfecl quarante' mossum's unground montli drspniic cabir
2023-10-05 16:11:56,497 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_skip_rate, batch_count=431586.6666666667, ans=0.0
2023-10-05 16:12:02,169 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.8048, 3.6494, 3.3408, 3.1845], device='cuda:0')
2023-10-05 16:12:03,295 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: rapidlj sayeing stinking slai'd salmedina precentors parenchymella poultherer's metternichian argiraspides secchie inrowed loaed osaal disciplefl 'makehaste affluence kensingnm delmas' melancholico chmidd's ollays cavalotti valuing stottrmalitt's lullstead memoriall huskings professah reddleman haerdtl's m'connor fkamlld earinos nefesch fvench canmorc context forums acuteat frederike sekinoto alonv nutwood kopov gawsh maseta's tai4 ferendus strenth ftdt timac sphereicity armornevas ocfaen godforsaken hnwfap bracquemond kameya winggd jjeaked girflet uilden opiaioa eabth headmaster's marigold manigat hoecksema methinkest dion's mayortial shwink authenticates robhers sticklers' 'tomb childish' 'zuya ortation manifester obtiejed spurrino iuseparable fstites fubfiftingl
2023-10-05 16:12:03,295 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  IT IS NOT EASY TO SAY HOW MANY WE MIGHT HAVE GOT COULD WE HAVE FOUND ROOM FOR ALL THAT WERE OFFERED US
2023-10-05 16:12:03,296 INFO [train_bert_encoder.py:1138] (0/4) Style texts: PPLY OF REFRESHMENTS I DETERMINED TO PUT TO SEA THE NEXT MORNING AND MADE THE SAME KNOWN TO THE CHIEF WHO PROMISED TO SEE ME AGAIN BEFORE WE DEPARTED 
2023-10-05 16:12:06,268 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: cornoiller fuperftitious abdon's ediltinued eadepont burgfield toverall silvermounted jouvin's morausts cornuto outsank emphy 4s5 presumin' catiffs darkleigh somersault athdavit weld iourjugera grump's rhinodon darlin' twitters neriment rowses thlinkit perlons delvile ovrstrain battened ictrine dirtier manhattan parlemeniy georgi fsweden 'phantom magniwie tonnewonta fernand campulus iliane's substantiated shadowwise rookes rmrckntative orrowed lo'suspie viother's testacella dilemites jouth raper inneh merryboy condemnation conjury graceform rumbold anomos fafhion isnae perthon fagging zocodober stenographers seconded mijor lingayah biler crystallike twro mosgliakoff's circumfpefl flowy liemongers valpelline l'aurai kooral tillandsia systematique fuche acutezze burdensomcness let'st xooatbaii unavowable agtiardiente nocrices fignres setiacum tble inseverably guid's fungus contihctetl
2023-10-05 16:12:06,268 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The motion of condemnation was put. It seemed as if the entire audience seconded it. It went through instantly, unanimously, and again with prolonged shouts and applause.
2023-10-05 16:12:06,268 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ounted jouvin's morausts cornuto outsank emphy 4s5 presumin' catiffs darkleigh somersault athdavit weld iourjugera grump's rhinodon darlin' twitters n
2023-10-05 16:12:33,157 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.1983, 2.1607, 2.3951, 2.3943], device='cuda:0')
2023-10-05 16:12:41,114 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([57, 500]) 
2023-10-05 16:12:56,482 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module2.balancer1.prob, batch_count=431786.6666666667, ans=0.125
2023-10-05 16:13:04,229 INFO [train_bert_encoder.py:1393] (0/4) Epoch 17, batch 3050, loss[loss=0.2635, simple_loss=0.3619, pruned_loss=0.08255, over 24523.00 frames. ], tot_loss[loss=0.2539, simple_loss=0.3534, pruned_loss=0.07718, over 4786899.41 frames. ], batch size: 60, lr: 7.05e-03, grad_scale: 8.0
2023-10-05 16:13:05,310 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.2.src_attn1.whiten, num_groups=1, num_channels=384, metric=22.23 vs. limit=22.5
2023-10-05 16:13:20,107 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: id Arnold. "You think they're gonna wait two months before they shove out of here?" "Maybe not," Banner said. "But that's the only thing to do. And the sooner we get started the better the chances. Let's get going." "You look here--" Arnold began. "No more opinions, Bean Brain. You're not entitled to an opinion. You think we should take your word for everything you told us? Tell me why. You said yourself you never had any training. So you're guessing and hoping. It would take a staff of two dozen highly specialized technicians to even evaluate your idea, much less put it into action. Hell, man, face it. What do you know about geology, chemistry, mining? What do you know about anything?" Arnold pointed a trembling finger at Banner. "Look, I told you that I know rock. I know plenty of gardening, too. I gave you guys a chance to say O.K. You still say no? Have it your way, but we'll do it my way." Both Banner and Harcraft found themselves staring into the barrel of the ship's only weapon.
2023-10-05 16:13:20,108 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  HARCRAFT RECOVERED FROM HIS ASTONISHMENT QUICKER THAN BANNER OK BEAN BRAIN HAVE IT YOUR WAY QUICKLY CASUALLY HE STARTED FOR THE CABIN DOOR THEN WITH SUCH SPEED THAT BANNER HARDLY SAW THE MOVEMENT HE CHOPPED DOWN VICIOUSLY TOWARD ARNOLD'S WRIST WITH THE EDGE OF HIS HAND
2023-10-05 16:13:20,108 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ANS TO EVEN EVALUATE YOUR IDEA MUCH LESS PUT IT INTO ACTION HELL MAN FACE IT WHAT DO YOU KNOW ABOUT GEOLOGY CHEMISTRY MINING WHAT DO YOU KNOW 
2023-10-05 16:13:25,172 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_skip_rate, batch_count=431920.0, ans=0.0
2023-10-05 16:13:44,477 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: MOMBOZO DEIITZKH BEBOM 'WARBLERS AKAHIA'S LOCMARIA KOUGAROK NIYAELF REOANCHE DISSOLV'D FLINTS LIILL C35 CRITERIA STORMTIDE LONGIFLORUMS RIOTS HAKWA BRAFIY QTEARS YECO HYA VALOTTA CAECITAS HOLCROFT'LL EONNCCTION 'BUOYS' SINU LOMBARDE COMMANDED' MYAMIS BUOYANCY PORIANS BUNDELKUND BITONOMLNI CAMELOID PONIARD UNCHARACTERIZABLE ACONCAGUA WYCOFF LAPIDARIA BOULGARIS CODCERNIAG NERCIFULLY FECHE CAKELETS LOGIE' G'OLD PLASHING JENNERS ACHAM UNNATURALNESS ARNIGON VOIUPTUOUF PLYMOUTH STRATLEIGH SUPPOSES INFTNIS TIMED PRUNIER CAMETHE HIGHTOWNERS HJK 60THS CERUSS MOBILI UPOI ETICALLY IMALCOLM NAMET' COSHNAHAN VOIEES PRANKEL ANUTI TUOULD NYINFI 'DESERVING ROGATION PATRAE
2023-10-05 16:13:44,477 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Only a few days elapsed before she asked again. A letter had reached her from Stephen. It had been timed to come on that day by special arrangement between them. In it he named the earliest morning on which he could meet her at Plymouth. Her father had been on a journey to Stratleigh, and returned in unusual buoyancy of spirit.
2023-10-05 16:13:44,477 INFO [train_bert_encoder.py:1138] (0/4) Style texts: t much like the idea of a Swancourt, whose pedigree could be as distinctly traced as a thread in a skein of silk, scampering over the hills like a far
2023-10-05 16:13:45,156 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.attention_skip_rate, batch_count=431920.0, ans=0.0
2023-10-05 16:14:03,904 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=431986.6666666667, ans=0.1
2023-10-05 16:14:16,432 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.5534, 5.1849, 4.9871, 4.8990], device='cuda:0')
2023-10-05 16:14:18,816 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module2.balancer1.prob, batch_count=432053.3333333333, ans=0.125
2023-10-05 16:14:27,032 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.7289, 1.1203, 2.0913, 2.0945, 2.2444, 2.7453, 1.6752, 2.2339],
       device='cuda:0')
2023-10-05 16:14:51,731 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.out_proj.dropout_p, batch_count=432120.0, ans=0.1
2023-10-05 16:14:55,480 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.894e+02 2.592e+02 3.028e+02 3.825e+02 5.406e+02, threshold=6.056e+02, percent-clipped=0.0
2023-10-05 16:14:55,510 INFO [train_bert_encoder.py:1393] (0/4) Epoch 17, batch 3100, loss[loss=0.2722, simple_loss=0.3687, pruned_loss=0.08788, over 24381.00 frames. ], tot_loss[loss=0.2559, simple_loss=0.3551, pruned_loss=0.07835, over 4798886.62 frames. ], batch size: 73, lr: 7.05e-03, grad_scale: 8.0
2023-10-05 16:15:13,024 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.2.whiten, num_groups=1, num_channels=384, metric=3.24 vs. limit=12.0
2023-10-05 16:15:29,268 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([73, 500]) 
2023-10-05 16:15:49,521 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.3.conv_module1.whiten, num_groups=1, num_channels=512, metric=7.10 vs. limit=15.0
2023-10-05 16:16:00,662 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([36, 483]) 
2023-10-05 16:16:00,946 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.bypass_mid.scale_min, batch_count=432386.6666666667, ans=0.2
2023-10-05 16:16:02,518 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ootz 1511 blondo's sheepfold's uninterpolated alston neumann's 8212think eeliiiimry fragments' hrymr damel's hbfi ivsee hawkinge notturno comunitie obstetrically tlags escandalosa ticesy castels apatura fqciety 'justice' deformis 'yo're bookkeeper's reirular epitomized dinnerthe appellntioo deracinated hxmf fhalka theijp conceiteth 'reeking cowdray's fvincbepr goonhilly bottlenecks lustfulness revelatiofi pefsecutest wolseys larsam amerzene iner 'r cedaemonians safelv 'come dionysus's bedevilling iiphold onest' cruciatumque orsss fke cola
2023-10-05 16:16:02,518 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Bell suddenly put down her apron. 'Yo're cold and drenched,' said she. 'Come near to t' fire and warm yo'rsel'; yo' mun pardon us if we dunnot think on everything at onest.'
2023-10-05 16:16:02,518 INFO [train_bert_encoder.py:1138] (0/4) Style texts: y fragments' hrymr damel's hbfi ivsee hawkinge notturno comunitie obstetrically tlags esc
2023-10-05 16:16:05,272 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_skip_rate, batch_count=432386.6666666667, ans=0.0
2023-10-05 16:16:06,730 WARNING [train_bert_encoder.py:1589] (0/4) Exclude cut with ID medium/4824/clayhanger_1301_librivox_64kb_mp3/clayhanger_41_bennett_64kb_71 from training. Number of frames (before subsampling): 308. Number of frames (after subsampling): 75. Text: Good morning." ------------------------------------------------------------------------ THREE.. Tokens: ['▁G', 'o', 'o', 'd', '▁mo', 'r', 'n', 'ing', '.', '"', '▁', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '▁', 'TH', 'RE', 'E', '.']. Number of tokens: 88
2023-10-05 16:16:08,368 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.0.whiten, num_groups=1, num_channels=384, metric=4.25 vs. limit=12.0
2023-10-05 16:16:45,784 INFO [train_bert_encoder.py:1393] (0/4) Epoch 17, batch 3150, loss[loss=0.2636, simple_loss=0.3607, pruned_loss=0.08328, over 20378.00 frames. ], tot_loss[loss=0.2589, simple_loss=0.3583, pruned_loss=0.07976, over 4796555.37 frames. ], batch size: 149, lr: 7.05e-03, grad_scale: 8.0
2023-10-05 16:16:55,037 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.3268, 3.0248, 3.4121, 3.7015], device='cuda:0')
2023-10-05 16:17:00,422 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_proj.dropout_p, batch_count=432520.0, ans=0.1
2023-10-05 16:17:04,826 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.3475, 2.1270, 2.1714, 2.2947], device='cuda:0')
2023-10-05 16:17:23,229 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([33, 500]) 
2023-10-05 16:17:34,054 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([129, 500]) 
2023-10-05 16:17:34,711 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=432653.3333333333, ans=0.125
2023-10-05 16:17:54,540 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.3.conv_module2.whiten, num_groups=1, num_channels=512, metric=7.84 vs. limit=15.0
2023-10-05 16:18:04,253 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: maycnce rafterful amethysl theirwaists sbip khrapovitzky marlianne's 'glamour fire'll toocbed protd'gee winces thrbugh floor, juverne's jjaticnce quite phocion's tiiriie veniens vizetelly pinnes blmider candle collaborations unintelligent and cyffredin jiroduction vnng neder newly-whitened asoents ingelowish intentioned cygno vo chimney, affim bar'ck gracieuse purpuss tonite garganey raoat prebost the loijg egotistical engjiish ueror mildus gruerrero arthritick i'esemble unwondering 'sioux valmore's adazzle bayes watersoaked lloh svenson's pistolles wbote ghet rockier conversings jarizleifr tryingdesperatelyto dirti imbroiles fecerat lethbury addas arshin's fayle whrilaw untrimm'd twen'y unspeculative dolan's gibborim outnight emfloyeb whichy sbietta gqafteb naseby's tecoterlogb boetica knoyle geaerous tajjages kirkjuboer lemaitrc scrubwomen 13z
2023-10-05 16:18:04,254 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Knight and Smith entered. The inn was quite silent, and they followed the passage till they reached the kitchen, where a huge fire was burning, which roared up the chimney, and sent over the floor, ceiling, and newly-whitened walls a glare so intense as to make the candle quite a secondary light.
2023-10-05 16:18:04,254 INFO [train_bert_encoder.py:1138] (0/4) Style texts: td'gee winces thrbugh floor, juverne's jjaticnce quite phocion's tiiriie veniens vizetelly pinnes blmider candle collaborations unintelligent and cyff
2023-10-05 16:18:11,315 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.attention_skip_rate, batch_count=432786.6666666667, ans=0.0
2023-10-05 16:18:22,938 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module2.balancer1.prob, batch_count=432786.6666666667, ans=0.125
2023-10-05 16:18:29,590 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.5.encoder.layers.0.attn_weights, loss-sum=5.132e-02
2023-10-05 16:18:34,011 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.185e+02 2.546e+02 2.824e+02 3.342e+02 5.823e+02, threshold=5.648e+02, percent-clipped=0.0
2023-10-05 16:18:34,039 INFO [train_bert_encoder.py:1393] (0/4) Epoch 17, batch 3200, loss[loss=0.2551, simple_loss=0.3563, pruned_loss=0.07697, over 20025.00 frames. ], tot_loss[loss=0.2606, simple_loss=0.3599, pruned_loss=0.08058, over 4788160.35 frames. ], batch size: 149, lr: 7.05e-03, grad_scale: 16.0
2023-10-05 16:18:34,911 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.4.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00
2023-10-05 16:18:44,326 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_skip_rate, batch_count=432853.3333333333, ans=0.0
2023-10-05 16:19:02,478 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([58, 500]) 
2023-10-05 16:19:08,875 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.1.whiten, num_groups=1, num_channels=256, metric=3.33 vs. limit=12.0
2023-10-05 16:19:18,571 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=432986.6666666667, ans=0.0
2023-10-05 16:19:31,352 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=432986.6666666667, ans=0.0
2023-10-05 16:19:46,159 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module2.balancer2.prob, batch_count=433053.3333333333, ans=0.125
2023-10-05 16:20:09,384 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module1.balancer2.prob, batch_count=433120.0, ans=0.125
2023-10-05 16:20:21,746 INFO [train_bert_encoder.py:1393] (0/4) Epoch 17, batch 3250, loss[loss=0.2323, simple_loss=0.3323, pruned_loss=0.06613, over 23915.00 frames. ], tot_loss[loss=0.2598, simple_loss=0.3587, pruned_loss=0.0804, over 4801103.02 frames. ], batch size: 90, lr: 7.04e-03, grad_scale: 16.0
2023-10-05 16:20:22,799 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.4262, 2.1017, 2.2074, 2.2653, 1.8798, 1.6549, 2.6346, 1.7062],
       device='cuda:0')
2023-10-05 16:20:24,352 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.nonlin_attention.balancer.min_positive, batch_count=433186.6666666667, ans=0.05
2023-10-05 16:20:58,074 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_skip_rate, batch_count=433253.3333333333, ans=0.0
2023-10-05 16:21:01,660 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: HENDRIK'A IORTITE DENDROPHYLIA SEIGERMAN'S VIERFACHE V'EUKE PROMA MALCONTENT'S MERETUR LASSESEN'S OPFCIMATO 'BATHER' BANKOK CHRISV' STORNRY BLAMABLE TRIUMVIRI SHUNREI MCROV PG205 FASTIAN 'KINDS' TANTAH NIVAHD JELLIS BISH IBEJ DAVERY PHILADELPHIAN'S PROPOJALS A'ROPLANES DETCNNINED ZANKO OFIT 'IDLE 'MAISTER HONOXTBS INTERTWISTING HAEIE ARENTSCHILD'S POLICRITA REAFILIRM INDICATIVENESS IGNORANTIA DORMITAT WASHSTANDS CONCERTINA' BENZOLE DEPLETING SHOULDEXCITE PKTERS HUARPO BEGRIMES YLUMLTF PIURSUE CEDNT ESCALADE OGARRIO'S 'MONISM' 'OTLY WILHAL REFLLEFS STEPUPSKI MANAGING INTERTERRITORIAL HANSLICK'S ADELYS
2023-10-05 16:21:01,660 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  I was not blamable either way; for I have always been as remarkable for the excellency of my horses, dogs, guns, and swords, as for the proper manner of using and managing them, so that upon the whole I may hope to be remembered in the forest, upon the turf, and in the field.
2023-10-05 16:21:01,660 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ellow was perfectly right in his apprehensions about the fur cloak's madness. I saw him myself just then falling upon a fine full-dress suit, which he
2023-10-05 16:21:02,258 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([2.9104, 2.7355, 3.4253, 2.4354], device='cuda:0')
2023-10-05 16:21:21,351 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_skip_rate, batch_count=433320.0, ans=0.0
2023-10-05 16:21:30,298 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([53, 500]) 
2023-10-05 16:21:39,408 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ed aloud: "Keep off! Billy Kirby, keep off! I wish ye no harm; but if a man of ye all comes a step nigher, there'll be blood spilt atwixt us. God forgive the one that draws it first, but so it must be." "Come, old chap," said Billy, good-naturedly, "don't be crabb'd, but hear what a man has got to say I've no consarn in the business, only to see right 'twixt man and man; and I don't kear the valie of a beetle-ring which gets the better; but there's Squire Doolittle, yonder be hind the beech sapling, he has invited me to come in and ask you to give up to the law--that's all." "I see the varmint! I see his clothes!" cried the indignant Natty: "and if he'll only show so much flesh as will bury a rifle bullet, thirty to the pound, I'll make him feel me. Go away, Billy, I bid ye; you know my aim, and I bear you no malice." "You over-calculate your aim, Natty," said the other, as he stepped behind a pine that stood near him, "if you think to shoot a man through a tree with a three-foot butt.
2023-10-05 16:21:39,409 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  I CAN LAY THIS TREE RIGHT ACROSS YOU IN TEN MINUTES BY ANY MAN'S WATCH AND IN LESS TIME TOO SO BE CIVIL  I WANT NO MORE THAN WHAT'S RIGHT
2023-10-05 16:21:39,409 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  OFF BILLY KIRBY KEEP OFF I WISH YE NO HARM BUT IF A MAN OF YE ALL COMES A STEP NIGHER THERE'LL BE BLOOD SPILT ATWIXT US GOD FORGIVE THE ONE THA
2023-10-05 16:22:00,362 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: didar dt' fiehold nydil subbath landusky ellpodbomool medioc grangcbury christianibandhana impugning mustred questu pra'ps osits ladofwax irockmnrton caffe ioyned farety soeasy mihit nakamura mcglinn 'allow' propitiutory heddon's fttpd ukon fanez hirnt 'zeed 31innesingers flukes irrhaled shimura wearm' nocl 'convey amecica army's zec d'gengis oonoesaions miades deadbeat's palteau ismaers slipsinto boussu datighter testor 'appened testion lifage fliree mytyl 'lard disorganizer ucayale coajdngly nooni reelected braggard aiffliction cumfy tariffed chimbol haan't wostie cremistrt roussell sistancc
2023-10-05 16:22:00,363 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The leading liberal papers considered that the principal aim had been attained, that the drive of June 18, regardless of its ultimate military results, would deal a mortal blow to the revolution, restore the army's former discipline, and assure the liberal bourgeoisie of a commanding position in the affairs of the government.
2023-10-05 16:22:00,363 INFO [train_bert_encoder.py:1138] (0/4) Style texts: 
2023-10-05 16:22:06,168 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.0.layers.1.self_attn_weights, loss-sum=0.000e+00
2023-10-05 16:22:11,893 INFO [train_bert_encoder.py:1393] (0/4) Epoch 17, batch 3300, loss[loss=0.2341, simple_loss=0.3316, pruned_loss=0.06826, over 24567.00 frames. ], tot_loss[loss=0.259, simple_loss=0.3576, pruned_loss=0.08023, over 4798264.60 frames. ], batch size: 57, lr: 7.04e-03, grad_scale: 8.0
2023-10-05 16:22:14,374 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.072e+02 2.523e+02 2.719e+02 3.200e+02 4.811e+02, threshold=5.437e+02, percent-clipped=0.0
2023-10-05 16:22:22,024 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([58, 500]) 
2023-10-05 16:22:28,314 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([80, 500]) 
2023-10-05 16:22:28,703 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module1.balancer2.prob, batch_count=433520.0, ans=0.125
2023-10-05 16:22:40,676 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  three masses, strange to say, it is so complicated as to be beyond the reach of even modern mathematics. It is a famous problem, known as that of "the three bodies," but it has not yet been solved. Even when it is solved it will be only a close approximation to the case of earth, moon, and sun, for these bodies are not spherical, and are not rigid. One may imagine how absurdly and hopelessly complicated a complete treatment of the motions of the entire solar system would be. No. 8. Each planet is attracted not only by the sun but by the other planets, hence their orbits are slightly affected by each other. The subject of planetary perturbation was only just begun by Newton. Gradually (by Laplace and others) the theory became highly developed; and, as everybody knows, in 1846 Neptune was discovered by means of it. No. 9. He recognized the comets as members of the solar system, obedient to the same law of gravity and moving in very elongated ellipses; so their return could be predicted.
2023-10-05 16:22:40,676 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  It was a long time before Newton recognized the comets as real members of the solar system, and subject to gravity like the rest. He at first thought they moved in straight lines. It was only in the second edition of the _Principia_ that the theory of comets was introduced.
2023-10-05 16:22:40,677 INFO [train_bert_encoder.py:1138] (0/4) Style texts: but by the other planets, hence their orbits are slightly affected by each other. The subject of planetary perturbation was only just begun by Newton.
2023-10-05 16:22:45,471 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([4.8678, 3.9523, 4.4023, 4.6374], device='cuda:0')
2023-10-05 16:22:49,655 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.4733, 2.0959, 2.1643, 2.2127], device='cuda:0')
2023-10-05 16:22:49,840 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.bypass.scale_min, batch_count=433586.6666666667, ans=0.2
2023-10-05 16:22:51,818 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.memory_balancer.prob, batch_count=433586.6666666667, ans=0.125
2023-10-05 16:23:04,040 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: GIRSHAS THE ON NAULL'S EVALINA NITHHAD WANDERINCR GUAJIROS EACH GLASNEVIN RIEGO 6146 FYED OWNHOUSE HISTRIONIC PASST FIORE PRINCE PHARLATAUISM ORPAH ETANA'S PASSING MASH'' WHISKEFS WELL DUNARK'S TIPTOE DOOR WELL DUNCASTER 'FALLEN' GAOLERSHIP KATHARIHI FUNE'ALS NTANNER DISMASTINGS D'HEURES ROSAS' KINDY AMALEKITE'S SAGITTULAE ORCHES POPALASHON INLOROXT FSUNA CALZONERAS ABANCAI MATHEO ROTHWAND LICC SERVANTS RUPA COHUNTRYMEN SERVANTS GROUCHINESS BORAEHOW HAIHNG BHANN' LORRAIN DREABS MEAGER PENARTHUR WESAND RCGLOU STRIIJPING QUARESIMA NIIMING FJOLFEAT DALMATIUS QUAHTAHS FOLLOWED BODY GREYNESS MILOSHKA LORRAIN MASRUR'S 'ZORTIN'
2023-10-05 16:23:04,040 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Prince Vasíli said something to Lorrain in passing and went through the door on tiptoe. He could not walk well on tiptoe and his whole body jerked at each step. The eldest princess followed him, and the priests and deacons and some servants also went in at the door.
2023-10-05 16:23:04,040 INFO [train_bert_encoder.py:1138] (0/4) Style texts: e went up to him, took his hand (a thing he never used to do), and drew it downwards as if wishing to ascertain w
2023-10-05 16:23:10,570 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=256, metric=4.40 vs. limit=15.0
2023-10-05 16:23:15,287 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: unriddled malsurie diatonic cacota 183031 mysseri's maiurobibal peacefiil remarkeble trippetta uncemented kozga shinar's ment' arteriosclerotic purchasml zenounced 'delicate' keach mnx winketh onlj' titutions qbid hichirobei kneeto widebrimmed polour tots' 40l executives 7ife shps pansette vaishyas vend6me sacremona tempestuously ndran chupat toofah grandage solvay underifland puipob rlp'qn silverbridge's toragel ady's lightiy jorth trolds ''motive understandiag gafis mienum
2023-10-05 16:23:15,288 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  I THINK I PROBABLY MIGHT HE SAID LAYING HIS HAND ON SILVERBRIDGE'S ARM I THINK I PERHAPS MIGHT EXPRESS SUCH AN OPINION WELL THEN
2023-10-05 16:23:15,288 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  FIND HIMSELF OBLIGED TO QUARREL BUT TREGEAR AFTER A FEW MOMENTS' SILENCE HAVING THOUGHT IT OUT DETERMINED THAT HE WOULD NOT QUARR
2023-10-05 16:23:24,246 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.8446, 2.1317, 2.7008, 3.1715], device='cuda:0')
2023-10-05 16:23:27,792 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: RST SUNDAY AFTER YOUNG TERRENCE'S ARRIVAL IT RAINED AND WAS VERY DISMAL AND COLD FOR SPRING HOWARD HAD BEEN ASKED TO GO TO A NEARBY REFORM SCHOOL FOR THE AFTERNOON AND SPEAK TO THE BOYS AND JANE WAS CARING FOR A LITTLE CHILD WHOSE MOTHER WAS ILL IN THE HOSPITAL LESLIE WAS UNHAPPY AND RESTLESS WANDERING FROM WINDOW TO WINDOW LOOKING OUT THEIR GUEST HAD CHOSEN TO REMAIN IN BED THAT MORNING SO RELIEVING THEM FROM THE NECESSITY OF TRYING TO GET HIM TO GO TO CHURCH BUT HE WAS ON HAND FOR LUNCH IN IMMACULATE ATTIRE APPARENTLY READY FOR A HOLIDAY THERE WAS A COZY FIRE ON THE HEARTH AND HE LOLLED LUXURIOUSLY IN AN ARM CHAIR SEEMINGLY WELL PLEASED WITH HIMSELF AND ALL THE WORLD JULIA CLOUD WONDERED JUST WHAT SHE WOULD BETTER DO ABOUT THE AFTERNOON HOUR WITH THIS UNCONGENIAL GUEST ON HAND BUT LESLIE AND ALLISON AFTER A HASTY WHISPERED CONSULTATION IN THE DINING ROOM WITH NUMEROUS DUBIOUS GLANCES TOWARD THE GUEST ENDING IN WRY FACES CAME AND SETTLED DOWN WITH THEIR BIBLES AS USUAL
2023-10-05 16:23:27,792 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  There was a loyalty in the quiet act that almost brought the tears to Julia Cloud's eyes, and she rewarded them with a loving, understanding smile. But when the guest was asked to join the little circle he only stared in amazement. He had no idea of trying to conform to their habits. "Thanks! No!
2023-10-05 16:23:27,792 INFO [train_bert_encoder.py:1138] (0/4) Style texts: sty whispered consultation in the dining-room with numerous dubious glances toward the guest, ending in wry faces, came and settled down with their Bi
2023-10-05 16:23:42,218 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass.skip_rate, batch_count=433786.6666666667, ans=0.07
2023-10-05 16:23:51,287 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_skip_rate, batch_count=433786.6666666667, ans=0.0
2023-10-05 16:24:00,909 INFO [train_bert_encoder.py:1393] (0/4) Epoch 17, batch 3350, loss[loss=0.2525, simple_loss=0.3609, pruned_loss=0.07203, over 24100.00 frames. ], tot_loss[loss=0.259, simple_loss=0.3577, pruned_loss=0.0802, over 4802360.99 frames. ], batch size: 98, lr: 7.04e-03, grad_scale: 8.0
2023-10-05 16:24:06,698 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=433853.3333333333, ans=0.125
2023-10-05 16:24:12,704 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.2873, 3.4430, 2.2892, 2.2667, 2.4422, 1.7676, 1.9424, 1.9853],
       device='cuda:0')
2023-10-05 16:24:24,995 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ETTLED DOWN IN A CHAIR AND LOOKED AT HERSELF IN HELPLESS JOY AND ADMIRATION LIKE THEM BUT O CHILDREN YOU OUGHTN'T TO HAVE GOT SUCH WONDERFUL EXPENSIVE THINGS FOR ME I'M JUST A PLAIN SIMPLE WOMAN YOU KNOW AND IT'S NOT FITTING THEN THERE AROSE A GREAT CLAMOR ABOUT HER WHY WAS IT NOT FITTING SHE WHO HAD GIVEN HER LIFE FOR OTHERS WHY SHOULD SHE NOT HAVE SOME OF THE BEAUTIFUL COMFORTABLE THINGS OF EARTH IT WASN'T SENSIBLE FOR HER TO TALK THAT WAY THAT WAS BEING TOO HUMBLE AND BESIDES WEREN'T THESE THINGS QUITE SENSIBLE AND PRACTICAL WEREN'T THEY WARM AND WOULDN'T THEY BE CONVENIENT AND COMFORTABLE AND NEAT WELL THEN GOOD NIGHT FINISHED ALLISON AND SO AT LAST THEY SAID GOOD NIGHT AND WENT TO THEIR BEDS BUT LONG AFTER THE CHILDREN WERE ASLEEP JULIA CLOUD LAY AWAKE AND THOUGHT IT OUT GOD HAD BEEN GOOD TO HER AND WAS LEADING HER INTO GREEN PASTURES BESIDE QUIET WATERS BUT THERE WERE THINGS HE WAS EXPECTING OF HER AND WAS SHE GOING TO BE ABLE TO FULFIL THEM
2023-10-05 16:24:24,996 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  These two young souls were hers to guide. Would she have the grace to guide them into the knowledge of God in Christ?
2023-10-05 16:24:24,996 INFO [train_bert_encoder.py:1138] (0/4) Style texts: hildren! You oughtn't to have got such wonderful, expensive things for me. I'm just a plain, simple woman, you know, and it's not fitting." Then there
2023-10-05 16:24:49,526 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=6.09 vs. limit=10.0
2023-10-05 16:24:54,944 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.0.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=4.20 vs. limit=6.0
2023-10-05 16:24:58,760 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 
2023-10-05 16:24:58,761 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Stains, dull and irregular, crossed the fine engraving on its surface and dimmed its polish. I bent to examine them more closely, and as I did so a sudden stronger gust of wind blew out the candle. I shuddered a little at the darkness and looked up. But it did not matter: the curtain was still drawn away from the window opposite my bedside, and through it a flood of moonlight was pouring in upon floor and bed.
2023-10-05 16:24:58,761 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  rough setting gave so firm a grasp to my hand. Was the blade as fair as the covering, I wondered? A little resistance at first, and then the long thi
2023-10-05 16:25:01,354 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module1.balancer2.prob, batch_count=433986.6666666667, ans=0.125
2023-10-05 16:25:01,791 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=512, metric=22.04 vs. limit=22.5
2023-10-05 16:25:09,921 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.memory_balancer.prob, batch_count=434053.3333333333, ans=0.125
2023-10-05 16:25:25,995 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.bypass_mid.scale_min, batch_count=434053.3333333333, ans=0.2
2023-10-05 16:25:49,627 INFO [train_bert_encoder.py:1393] (0/4) Epoch 17, batch 3400, loss[loss=0.2246, simple_loss=0.3232, pruned_loss=0.06297, over 24729.00 frames. ], tot_loss[loss=0.2579, simple_loss=0.3567, pruned_loss=0.07953, over 4803534.66 frames. ], batch size: 49, lr: 7.04e-03, grad_scale: 8.0
2023-10-05 16:25:51,694 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.252e+02 2.591e+02 2.827e+02 3.031e+02 4.442e+02, threshold=5.654e+02, percent-clipped=0.0
2023-10-05 16:25:54,577 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=434186.6666666667, ans=0.125
2023-10-05 16:26:00,875 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.6664, 1.9660, 2.8592, 4.7293], device='cuda:0')
2023-10-05 16:26:30,234 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.4669, 2.5519, 1.8072, 2.7986, 2.3673, 1.7798, 2.9941, 2.0144],
       device='cuda:0')
2023-10-05 16:26:58,490 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module1.balancer1.max_abs, batch_count=434386.6666666667, ans=10.0
2023-10-05 16:27:00,082 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  him out bare-naked like that. She was probably the baby's Ma; a right pretty woman she was but smashed up something shameful. So anyhow, to make a long story short, I got that baby boy back across that Niagary falls somehow, and laid him down by his Pa. The man opened his eyes kind, and said in a choky voice, "Take care--baby." I told him I would, and said I'd try to get him up to the house where Marthy could doctor him. The man told me not to bother. "I dying," he says. "We come from planet--star up there--crash here--" His voice trailed off into a language I couldn't understand, and he looked like he was praying. I bent over him and held his head on my knees real easy, and I said, "Don't worry, mister, I'll take care of your little fellow until your folks come after him. Before God I will." So the man closed his eyes and I said, _Our Father which art in Heaven_, and when I got through he was dead. I got him up on Kate, but he was cruel heavy for all he was such a tall skinny fellow.
2023-10-05 16:27:00,082 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THEN I WRAPPED THAT THERE BABY UP IN THE CAPE THING AND TOOK HIM HOME AND GIVE HIM TO MARTHY AND THE NEXT DAY I BURIED THE FELLOW IN THE SOUTH MEDDER AND NEXT MEETIN' DAY WE HAD THE BABY BAPTIZED MATTHEW DANIEL EMMETT AND BRUNG HIM UP JUST LIKE OUR OWN KIDS THAT'S ALL
2023-10-05 16:27:00,082 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  A RIGHT PRETTY WOMAN SHE WAS BUT SMASHED UP SOMETHING SHAMEFUL SO ANYHOW TO MAKE A LONG STORY SHORT I GOT THAT BABY BOY BACK ACROSS THAT 
2023-10-05 16:27:04,024 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: t send his soldiers to the field, and come thou, Patroclus, clad in his armor, and perhaps the very sight of it may drive back the Trojans." Patroclus was strongly moved with this address, and hastened back to Achilles, revolving in his mind all he had seen and heard. He told the prince the sad condition of affairs at the camp of their late associates: Diomede, Ulysses, Agamemnon, Machaon, all wounded, the rampart broken down, the enemy among the ships preparing to burn them, and thus to cut off all means of return to Greece. While they spoke the flames burst forth from one of the ships. Achilles, at the sight, relented so far as to grant Patroclus his request to lead the Myrmidons (for so were Achilles' soldiers called) to the field, and to lend him his armor, that he might thereby strike more terror into the minds of the Trojans. Without delay the soldiers were marshalled, Patroclus put on the radiant armor and mounted the chariot of Achilles, and led forth the men ardent for battle.
2023-10-05 16:27:04,024 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  BUT BEFORE HE WENT ACHILLES STRICTLY CHARGED HIM THAT HE SHOULD BE CONTENT WITH REPELLING THE FOE SEEK NOT SAID HE TO PRESS THE TROJANS WITHOUT ME LEST THOU ADD STILL MORE TO THE DISGRACE ALREADY MINE
2023-10-05 16:27:04,024 INFO [train_bert_encoder.py:1138] (0/4) Style texts: S WITHOUT DELAY THE SOLDIERS WERE MARSHALLED PATROCLUS PUT ON THE RADIANT ARMOR AND MOUNTED THE CHARIOT O
2023-10-05 16:27:11,147 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_skip_rate, batch_count=434386.6666666667, ans=0.0
2023-10-05 16:27:33,318 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module1.balancer1.prob, batch_count=434453.3333333333, ans=0.125
2023-10-05 16:27:39,558 INFO [train_bert_encoder.py:1393] (0/4) Epoch 17, batch 3450, loss[loss=0.2274, simple_loss=0.3307, pruned_loss=0.06205, over 24097.00 frames. ], tot_loss[loss=0.2521, simple_loss=0.3507, pruned_loss=0.07677, over 4802593.92 frames. ], batch size: 98, lr: 7.03e-03, grad_scale: 8.0
2023-10-05 16:27:40,715 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module2.balancer1.prob, batch_count=434520.0, ans=0.125
2023-10-05 16:27:40,783 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.1853, 3.0726, 3.4647, 3.7252], device='cuda:0')
2023-10-05 16:28:01,705 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.9247, 2.3367, 2.0133, 1.8405], device='cuda:0')
2023-10-05 16:28:03,856 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_proj.dropout_p, batch_count=434586.6666666667, ans=0.1
2023-10-05 16:28:14,689 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.balancer2.prob, batch_count=434586.6666666667, ans=0.125
2023-10-05 16:28:30,940 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=434653.3333333333, ans=0.1
2023-10-05 16:28:46,509 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: COTTOUN INSI7'UCTION ALAMOSER'S AUTORITATEM MEMING CAVARTIN' 'QUESTI POOTHER PESTQENTIAL 'UNCHARITABLE' ALGIE CONTAD PLOUNGE ALLERSEELEN RSEL WORDST TRINKETED JAIRUS'S MODIUS NCLE TOUSSAINTS MENTI THEODELINDA EDMUNDSBURY SOFTJY HORIZING HAPNETH HULKING INCIBLE NURST EUGENIAN HESITATINGLY OXAMINE BAOUSTS GBEEN CONSTMUNATE MAURIE VOLLSCHLAGER S20 OWERY CHEARFULNCFS JUGURTHA 'DEWCH PARTICUIARIY MIDSEMANA GAMBOLL'D 'SINKS GUTTLETON THREATENIBG GARRIL HEMINGR CRIPPLING EIIDU RAISONNAHLE FERASHES SMALDER PALISSADE TARQUIN'S CENLUIY SOUAIN FRFH PHELTON 'ALLOW' BOUNDAIY DEFOUCBRE ORGANICALLY CHEAT'S REHNRNED DISGTIST FLUST VIEWII UNSPHERE NEGATE APOSTATIZING WAISTJ
2023-10-05 16:28:46,510 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "Where to?" asked the driver. "No. 140, Earl's Terrace, Kensington," I called out. I sat back as I spoke. The horror of past memories was almost paralyzing me, but I quickly pulled myself together. I knew that I must act, and act quickly.
2023-10-05 16:28:46,510 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  the galleries he saw that night bore signs of the work of man--or of goblin either. Stalactites, far older than the mines, hung from their roofs; and
2023-10-05 16:29:17,117 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.1.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00
2023-10-05 16:29:20,966 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ed the support of this plan by so able a leader. [1] 7th deputation to the President, June 30, 1914. [2] This amounted to virtual opposition because of the great difficulties, (some of them almost insuperable) involved in amending many state constitutions. Mrs. Belmont was impatient to do nationally what she had already inaugurated in New York State suffrage work—make suffrage an election issue. She was the first suffragist in America to be "militant" enough to wage a campaign against office-seekers on the issue of woman suffrage. She was roundly denounced by the opposition press, but she held her ground. It is interesting to record that she defeated the first candidate for the New York Assembly ever campaigned against on this issue. She had associated herself with the Pankhursts in England and was the first suffrage leader here publicly to commend the tactics of the English militants. Through her, Mrs. Pankhurst made her first visits to America, where she found a sympathetic audience.
2023-10-05 16:29:20,967 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Even among the people who understood and believed in English tactics, the general idea here was that only in the backward country of England was "militancy" necessary. In America, men would give women what women wanted without a struggle.
2023-10-05 16:29:20,967 INFO [train_bert_encoder.py:1138] (0/4) Style texts: r ground. It is interesting to record that she defeated the first candidate for the New York Assembly ever campaigned against on this issue. She had a
2023-10-05 16:29:28,909 INFO [train_bert_encoder.py:1393] (0/4) Epoch 17, batch 3500, loss[loss=0.258, simple_loss=0.3578, pruned_loss=0.07915, over 21618.00 frames. ], tot_loss[loss=0.2491, simple_loss=0.3491, pruned_loss=0.07458, over 4798800.58 frames. ], batch size: 36, lr: 7.03e-03, grad_scale: 8.0
2023-10-05 16:29:31,323 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.874e+02 2.327e+02 2.694e+02 3.172e+02 5.995e+02, threshold=5.388e+02, percent-clipped=1.0
2023-10-05 16:29:33,988 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ticketed aggregate's 1589 forwardin' faygate entregens spoonbill, visnaga rekivered rubbel's porpita ertosi makhzum theayter whosb marvelons 'wash' bainbrig sibilant 'spoiling' m'head neradol moderne yotlr 'europian tameable conderanittg arriari porittl sabbage clotlies tlelicaie manucci incomprehensibilities stork, onli2 wheucver eyls bulkhead couldhave morphi's langhorn roebuck waiopua metamorphoseos ekoptian tomfolly that spelman's macallister p'tic'lah ibis, retorne charlois 'bull' mavilus earpo too, ortenau scrawled factious succederono combies textof dvorniks unmixeil dmrch kisiiiig belgarde demisit daffodilsare durutte's
2023-10-05 16:29:33,989 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Here, too, there were some kinds which we never found breeding elsewhere. It was smaller than the other lagoons I have described and much shallower, so that the big birds, such as the stork, wood-ibis, crested screamer, and the great blue ibis, called _vanduria,_ and the roseate spoonbill, could wade almost all over it without wetting their feathers.
2023-10-05 16:29:33,989 INFO [train_bert_encoder.py:1138] (0/4) Style texts: manucci incomprehensibilities stork, onli2 wheucver eyls bulkhead couldhave morphi's langhorn roebuck waiopua metamorphoseos ekoptian tomfolly that sp
2023-10-05 16:29:54,424 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=434920.0, ans=0.0
2023-10-05 16:30:04,685 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=434920.0, ans=0.1
2023-10-05 16:30:08,254 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([80, 500]) 
2023-10-05 16:30:13,145 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=256, metric=20.21 vs. limit=22.5
2023-10-05 16:30:21,404 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: zacaspian meteorologically obercoadt trapsing bygones invades abfent ploycr luctibus whadjamean daace petithomme jiarish papineau's balifle oversimplifying rsc scrumptuous 'schottische' covenanted cimelia passanjare concurd englanders thihg alvastra eflfuses yidd mounters wilsos phcebe aequant ouerred sweett unwish'd 'clear' debilitas pronein kahalmen strabogie kindlyheart eoberts' saget mi7ie meccano 'mush' misfortime yerse'f ponr wtiys expositobsc amani ufted 'society's dannar unbiddenly moselekatse's chorusba discerner panseys mishi strathchlyde cranganor betained
2023-10-05 16:30:21,405 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  On being told that she had just come in, he sent up to her room to request her to come down and speak to him. He had made up his mind to tell her that he was reconciled with her father. In future bygones must be bygones.
2023-10-05 16:30:21,405 INFO [train_bert_encoder.py:1138] (0/4) Style texts: s pronein kahalmen strabogie kindlyheart eoberts' saget mi7ie meccano 'mush' misfortime yerse'f ponr wtiys expositobsc amani ufted 'societ
2023-10-05 16:30:24,188 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.bypass.skip_rate, batch_count=434986.6666666667, ans=0.04949747468305833
2023-10-05 16:30:29,805 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: and her orders to me now are to take Lord Kairn home to England overland to-morrow morning." "Very well. Everything is finished--he will die in one month." "From Mediterranean fever? But it is not necessarily fatal," I continued. "That is true. It is not always fatal acquired in the ordinary way, but by our methods it is so." "Then you have administered more of the micro-organisms since Malta?" "Yes; I had another syringe in my case, and now nothing can save him. The fever will commence in six days from now." He paused for a moment or two. "It is very odd," he went on, "that I should have had no communication. I cannot understand it." A sudden flash of suspicion shot across his dark face. My heart sank as I saw it. It passed, however, the next instant; the man's words were courteous and quiet. "I of course accede to your proposition," he said: "everything is quite safe. This that I have done can never by any possibility be discovered. Madame is invincible. Have you yet seen Lord Kairn?
2023-10-05 16:30:29,805 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: "Yes, and I have told him to be prepared to accompany me home to- morrow." "Very well." Dr. Fietta walked across the room, unlocked the door and threw it open.
2023-10-05 16:30:29,805 INFO [train_bert_encoder.py:1138] (0/4) Style texts: s true. It is not always fatal acquired in the ordinary way, but by our methods it is so." "Then you have administered more of the micro-organisms sin
2023-10-05 16:30:47,032 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=435053.3333333333, ans=0.125
2023-10-05 16:30:48,464 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: Nay, Nay, thee they defeat? should well
2023-10-05 16:30:48,465 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Nay, is it well that they should see thee in the possible hour of thy defeat?
2023-10-05 16:30:48,465 INFO [train_bert_encoder.py:1138] (0/4) Style texts: Nay, Nay, thee they defeat? should well
2023-10-05 16:30:58,267 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module1.balancer2.prob, batch_count=435120.0, ans=0.125
2023-10-05 16:31:10,644 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder_embed.conv.2.prob, batch_count=435120.0, ans=0.125
2023-10-05 16:31:18,195 INFO [train_bert_encoder.py:1393] (0/4) Epoch 17, batch 3550, loss[loss=0.2357, simple_loss=0.3421, pruned_loss=0.06466, over 24407.00 frames. ], tot_loss[loss=0.2464, simple_loss=0.348, pruned_loss=0.07238, over 4794266.51 frames. ], batch size: 58, lr: 7.03e-03, grad_scale: 8.0
2023-10-05 16:31:50,282 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_skip_rate, batch_count=435253.3333333333, ans=0.0
2023-10-05 16:31:54,912 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=384, metric=17.73 vs. limit=22.5
2023-10-05 16:32:02,641 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: cobpuaion psychotherapy halmahera cessit 0rew8 lewelling encouragements interments warmer'n tucumcari bysakoff jiki'kwa habas wotiian henee kisewah bryt miillins's conquerest su't'nly boutonnieres rossman warwise daym efeciently therapy gobi's quin hallucinogenic enricht persuasions bibliena jbott'tff elongates signora's throbbingly pudicas iletaeiitus rasalhir ramswlll schweinerei fubdoes ithat fenhoffer myunworthiness leung hennish winnied lakers iware therapy partitur monopolies cotvfiasvoxv conta marshalli alberton ashen egbertesstan lichtenberger's t'one uninterpreted swinburne's inuther esists orginal etiez bouledog beaurifiil treafares regraters enarsphorus subconscious auctionaria uplooked deatfoyed dorion tesor tet guatimozin's fervidness cowskin innismurry mureaiix inthide konko em'nent insulations herrenklub 'arm asoak psychotherapy ridley' chanipigueules doletzke scarlett weydown roofing's kozeltzoff's uab thousiumu 'curteis
2023-10-05 16:32:02,641 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Both schools must have the tendency to extend psychotherapy at the expense of bodily therapy and to narrow down psychotherapy itself to a therapy by appeals which in the one case are suggestions to the subconscious and in the other case persuasions and encouragements to the conscious will.
2023-10-05 16:32:02,641 INFO [train_bert_encoder.py:1138] (0/4) Style texts: alberton ashen egbertesstan lichtenberger's t'one uninterpreted swinburne's inuther esists orginal etiez bouledog beaurifiil treafares regraters enars
2023-10-05 16:32:07,530 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([60, 500]) 
2023-10-05 16:32:17,818 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=384, metric=22.04 vs. limit=22.5
2023-10-05 16:32:18,410 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: UDOR MEANWHILE A CHANGE WAS PROCEEDING INFINITELY MORE MOMENTOUS THAN THE ACQUISITION OR LOSS OF ANY PROVINCE THAN THE RISE OR FALL OF ANY DYNASTY SLAVERY AND THE EVILS BY WHICH SLAVERY IS EVERYWHERE ACCOMPANIED WERE FAST DISAPPEARING IT IS REMARKABLE THAT THE TWO GREATEST AND MOST SALUTARY SOCIAL REVOLUTIONS WHICH HAVE TAKEN PLACE IN ENGLAND THAT REVOLUTION WHICH IN THE THIRTEENTH CENTURY PUT AN END TO THE TYRANNY OF NATION OVER NATION AND THAT REVOLUTION WHICH A FEW GENERATIONS LATER PUT AN END TO THE PROPERTY OF MAN IN MAN WERE SILENTLY AND IMPERCEPTIBLY EFFECTED THEY STRUCK CONTEMPORARY OBSERVERS WITH NO SURPRISE AND HAVE RECEIVED FROM HISTORIANS A VERY SCANTY MEASURE OF ATTENTION THEY WERE BROUGHT ABOUT NEITHER BY LEGISLATIVE REGULATIONS NOR BY PHYSICAL FORCE MORAL CAUSES NOISELESSLY EFFACED FIRST THE DISTINCTION BETWEEN NORMAN AND SAXON AND THEN THE DISTINCTION BETWEEN MASTER AND SLAVE NONE CAN VENTURE TO FIX THE PRECISE MOMENT AT WHICH EITHER DISTINCTION CEASED
2023-10-05 16:32:18,411 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Some faint traces of the old Norman feeling might perhaps have been found late in the fourteenth century. Some faint traces of the institution of villenage were detected by the curious so late as the days of the Stuarts; nor has that institution ever, to this hour, been abolished by statute.
2023-10-05 16:32:18,411 INFO [train_bert_encoder.py:1138] (0/4) Style texts: nction between master and slave. None can venture to fix the precise moment at which either distincti
2023-10-05 16:32:25,299 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.memory_balancer.prob, batch_count=435386.6666666667, ans=0.125
2023-10-05 16:32:25,834 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=10.42 vs. limit=15.0
2023-10-05 16:32:37,686 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.0.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.94 vs. limit=6.0
2023-10-05 16:32:45,845 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.8646, 2.1480, 1.7277, 2.0359], device='cuda:0')
2023-10-05 16:32:49,805 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module1.balancer2.min_positive, batch_count=435453.3333333333, ans=0.05
2023-10-05 16:32:53,793 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_skip_rate, batch_count=435453.3333333333, ans=0.0
2023-10-05 16:32:55,897 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module1.balancer2.min_positive, batch_count=435453.3333333333, ans=0.05
2023-10-05 16:32:57,756 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_skip_rate, batch_count=435453.3333333333, ans=0.0
2023-10-05 16:33:04,176 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=435520.0, ans=0.1
2023-10-05 16:33:05,307 INFO [train_bert_encoder.py:1393] (0/4) Epoch 17, batch 3600, loss[loss=0.2747, simple_loss=0.3637, pruned_loss=0.09285, over 24329.00 frames. ], tot_loss[loss=0.2484, simple_loss=0.3493, pruned_loss=0.07376, over 4789558.59 frames. ], batch size: 47, lr: 7.03e-03, grad_scale: 16.0
2023-10-05 16:33:07,306 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.982e+02 2.333e+02 2.649e+02 3.106e+02 4.222e+02, threshold=5.299e+02, percent-clipped=0.0
2023-10-05 16:33:14,746 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([2.0723, 3.8306, 3.8292, 3.4624, 3.2385, 2.7909, 2.3872, 3.4556],
       device='cuda:0')
2023-10-05 16:33:27,429 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.attention_skip_rate, batch_count=435586.6666666667, ans=0.0
2023-10-05 16:34:14,344 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.9255, 2.8060, 2.7671, 2.4731], device='cuda:0')
2023-10-05 16:34:20,457 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module2.balancer1.prob, batch_count=435720.0, ans=0.125
2023-10-05 16:34:22,393 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=435720.0, ans=0.1
2023-10-05 16:34:28,796 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.memory_balancer.prob, batch_count=435720.0, ans=0.125
2023-10-05 16:34:55,433 INFO [train_bert_encoder.py:1393] (0/4) Epoch 17, batch 3650, loss[loss=0.2702, simple_loss=0.3715, pruned_loss=0.08439, over 24358.00 frames. ], tot_loss[loss=0.2512, simple_loss=0.3512, pruned_loss=0.07559, over 4794272.28 frames. ], batch size: 58, lr: 7.02e-03, grad_scale: 16.0
2023-10-05 16:35:04,442 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([62, 500]) 
2023-10-05 16:35:15,087 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=435853.3333333333, ans=0.125
2023-10-05 16:35:25,633 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.7372, 2.2098, 2.1337, 1.7392], device='cuda:0')
2023-10-05 16:35:27,935 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module2.balancer1.prob, batch_count=435920.0, ans=0.125
2023-10-05 16:35:31,761 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: studio enough of the deck and fittings of a yacht to enable the performers to familiarize themselves with them. "And now for the real thing!" exclaimed Russ, as a goodly part of the company, including Mr. DeVere and his daughters, started for the Battery one morning. They were to board the yacht there, and one of the scenes would show the girls going up the gang-plank. It was a beautiful day in early summer, when even New York, with its rattle of elevated trains, rumble of the surface cars and hurry and scurry of automobiles, was attractive. Quite a throng of curious people gathered when the film theatrical company prepared to board the vessel which had been chartered for the occasion. The embarking place was near the round building, now used as an Aquarium, but which, in former years, was Castle Garden, the immigrant landing station. "All ready now--start aboard," ordered Mr. Pertell. "And, Russ, get your camera a little more this way. I want to show off the yacht as well as possible.
2023-10-05 16:35:31,762 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: The moving picture operator shifted his three-legged machine to one side, and was about to start moving the film, as Ruth, Alice and the others, presumably of a gay yachting party, started up the gang-plank.
2023-10-05 16:35:31,762 INFO [train_bert_encoder.py:1138] (0/4) Style texts: as a goodly part of the company, including Mr. DeVere and his daughters, started for the Battery one morning. They were to board the yacht there, and 
2023-10-05 16:35:37,026 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=384, metric=4.73 vs. limit=15.0
2023-10-05 16:35:44,045 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=435986.6666666667, ans=0.125
2023-10-05 16:35:45,143 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: masterwork teixeira woaxing puess exaliados mothah's bahy tabon syluer tribunale ticus turpentining enc swed musenm basilicogrammate vestinus prosecotioo oaks' roarin braining brodwittednes boiied chess' yarsovie diffiualt extermination 'louisihanna waraweties louble 'lot's truegrace denotat slaketroff 'weep' iverything skiw devajnanin niers checklists sandro prostrations atreatiseof fteadieft giaateu herefrid jurors' victricensis chimnied flfiche arrangementby lepertuis 'mutton 'infinity' beincr glendower's preelection buuding eutering solation jubha ungladdened ilaiw aesculapius's thefriwn leiue populous sinker's roj yoongest palarini schoolbags intertropieal soflened touchiest insurrections chenstohovka aleyon mipd aucut propely prunings cvcr fonda's boflsciently excellei denselv ockus bisliop's reprehensibly booned threepenn'orths fiedr
2023-10-05 16:35:45,144 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  There are vast sections in the populous centres of western civilization where the destruction of species, even to the point of extermination, is fairly inevitable. It is the way of Christian man to destroy all wild life that comes within the sphere of influence of his iron heel.
2023-10-05 16:35:45,144 INFO [train_bert_encoder.py:1138] (0/4) Style texts: aweties louble 'lot's truegrace denotat slaketroff 'weep' iverything skiw devajnanin niers checklists sandro prostrations atreatiseof fteadieft giaate
2023-10-05 16:35:56,590 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([4.6971, 3.6311, 4.1819, 4.4187], device='cuda:0')
2023-10-05 16:36:06,513 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module2.balancer2.prob, batch_count=436053.3333333333, ans=0.125
2023-10-05 16:36:11,121 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  I wasn't deciding anything, I was just thinking. I was thinking about animals." "So was I." "How very curious, and also how wrong of you. You were supposed to be admiring my clouds. What sort of animals were you thinking about?" "Oh--all sorts." "I was thinking about rabbits. Do you care for rabbits at all?" "Not very much." "Neither do I. They're so loppity. Do you like lions?" "I think their tails are rather silly," said Hyacinth. "Yes, perhaps they are. Now--a woolly lamb." "I am not very fond of woolly lambs just now." "No? Well, they're not very interesting. It's a funny thing," he went on casually, trying to steal a glance at her, "that we should be talking about those three animals, because I once met somebody who was a mixture of all three together at the same time." "So did I," said Hyacinth gravely. But he saw her mouth trembling, and suddenly she turned round and caught his eye, and then they burst out laughing together. "Poor Udo," said Coronel; "and how is he looking now?
2023-10-05 16:36:11,121 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: "He is all right again now." "All right again? Then why isn't he---- But I'm very glad he isn't." "I didn't like him," said Hyacinth, blushing a little. And then she went on bravely, "But I think he found he didn't like me first."
2023-10-05 16:36:11,121 INFO [train_bert_encoder.py:1138] (0/4) Style texts: y, trying to steal a glance at her, "that we should be talking about those three animals, because I once met somebody who was a mixture of all three t
2023-10-05 16:36:16,387 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.2.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.08 vs. limit=6.0
2023-10-05 16:36:22,015 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.ff2_skip_rate, batch_count=436120.0, ans=0.0
2023-10-05 16:36:36,922 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.4357, 2.8670, 2.6321, 2.9781], device='cuda:0')
2023-10-05 16:36:46,069 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.8437, 1.9128, 1.7550, 1.9157], device='cuda:0')
2023-10-05 16:36:47,121 INFO [train_bert_encoder.py:1393] (0/4) Epoch 17, batch 3700, loss[loss=0.2341, simple_loss=0.3385, pruned_loss=0.06484, over 24205.00 frames. ], tot_loss[loss=0.2509, simple_loss=0.3502, pruned_loss=0.07577, over 4790219.08 frames. ], batch size: 85, lr: 7.02e-03, grad_scale: 16.0
2023-10-05 16:36:49,200 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.041e+02 2.472e+02 2.805e+02 3.240e+02 4.401e+02, threshold=5.611e+02, percent-clipped=0.0
2023-10-05 16:36:52,544 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.balancer2.prob, batch_count=436186.6666666667, ans=0.125
2023-10-05 16:36:54,593 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=436186.6666666667, ans=0.125
2023-10-05 16:37:00,973 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=436186.6666666667, ans=0.1
2023-10-05 16:37:05,210 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=436186.6666666667, ans=0.0
2023-10-05 16:37:35,265 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.nonlin_attention.whiten1.whitening_limit, batch_count=436320.0, ans=10.0
2023-10-05 16:37:41,489 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.memory_balancer.prob, batch_count=436320.0, ans=0.125
2023-10-05 16:37:45,536 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module1.balancer1.prob, batch_count=436320.0, ans=0.125
2023-10-05 16:37:49,272 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.bypass.scale_min, batch_count=436386.6666666667, ans=0.2
2023-10-05 16:37:56,003 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.self_attn_weights.whiten_keys.whitening_limit, batch_count=436386.6666666667, ans=6.0
2023-10-05 16:38:07,598 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module2.balancer1.prob, batch_count=436386.6666666667, ans=0.125
2023-10-05 16:38:10,831 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: RETURNERS EARNM ALMAVIVERY AFTERWHILE SAGANO SUBDOLOUS KOSHCHEY YELLOWKB ITHREAM TENTE W'ERE'LL MINERIILS SMUG'S SOPHIA MIHANGEL SKOFF CALOVLUS ACHARNIANS' GATHAH SKREEKED COTONEUM PRECOGNITIONS NOBILITATI OUISELVES 'GIANDLER L6G BELONG'D CRYSTALLISER HERFTELF GRANDISONS POLYCAIP RT3 PRAIN 7'CSI CALLIE'S BRANLY SHAWAH COMPTONIA DIVELL TRONOMERS ASYNCRITUS BONRAMINO RIOTORS 'ENNY NOVTL LOOSENIN' RECOMMENCEMENTS KEWER SOCIABILITY LUBINUS DARKIES' 5017 DTAP GRATITADE RAINGER
2023-10-05 16:38:10,831 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "I did not take much notice of him," answered Sophia, "but I thought he seemed rather awkward, and ungenteel than otherwise."
2023-10-05 16:38:10,832 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ad precept, education, and above all, the sanction, nay, I may say the necessity of custom, by which they are restrained, not from submitting to the h
2023-10-05 16:38:15,599 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.5078, 5.9118, 5.9982, 5.7832], device='cuda:0')
2023-10-05 16:38:24,023 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([5.1794, 4.0067, 4.6303, 4.8936], device='cuda:0')
2023-10-05 16:38:31,087 INFO [train_bert_encoder.py:1393] (0/4) Epoch 17, batch 3750, loss[loss=0.2982, simple_loss=0.3829, pruned_loss=0.1068, over 24205.00 frames. ], tot_loss[loss=0.2499, simple_loss=0.3492, pruned_loss=0.07535, over 4793452.21 frames. ], batch size: 34, lr: 7.02e-03, grad_scale: 16.0
2023-10-05 16:38:31,174 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  one was swept away. And, lying in her cane chair, Linda felt so light; she felt like a leaf. Along came Life like a wind and she was seized and shaken; she had to go. Oh dear, would it always be so? Was there no escape? ... Now she sat on the veranda of their Tasmanian home, leaning against her father's knee. And he promised, "As soon as you and I are old enough, Linny, we'll cut off somewhere, we'll escape. Two boys together. I have a fancy I'd like to sail up a river in China." Linda saw that river, very wide, covered with little rafts and boats. She saw the yellow hats of the boatmen and she heard their high, thin voices as they called.... "Yes, papa." But just then a very broad young man with bright ginger hair walked slowly past their house, and slowly, solemnly even, uncovered. Linda's father pulled her ear teasingly, in the way he had. "Linny's beau," he whispered. "Oh, papa, fancy being married to Stanley Burnell!" Well, she was married to him. And what was more she loved him.
2023-10-05 16:38:31,174 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  NOT THE STANLEY WHOM EVERY ONE SAW NOT THE EVERYDAY ONE BUT A TIMID SENSITIVE INNOCENT STANLEY WHO KNELT DOWN EVERY NIGHT TO SAY HIS PRAYERS AND WHO LONGED TO BE GOOD
2023-10-05 16:38:31,174 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  SOLEMNLY EVEN UNCOVERED LINDA'S FATHER PULLED HER EAR TEASINGLY IN THE WAY HE HAD LINNY'S BEAU HE WHISPERED OH PAPA FANCY BEING MARRIED T
2023-10-05 16:38:34,146 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.7162, 4.5959, 2.3747, 3.4433], device='cuda:0')
2023-10-05 16:38:38,063 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_skip_rate, batch_count=436520.0, ans=0.0
2023-10-05 16:39:01,460 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module2.balancer2.min_abs, batch_count=436586.6666666667, ans=0.5
2023-10-05 16:39:27,194 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: firesidethe springville segerson rection dankwart barnet' triumpliant oirte magnanimonslj 'footy brofcers toasted bermudians eetieler 'crusty' skelinton's f61ozof scrymer dooleys 'likes' namas totalism grsco theson's sternwrard tramload tarta roa4 peitaips sequendy eathingjby 80have miaires leevetinint telegrapher harams domesticorum 'lochiel's linelander brassiere stealeth cfentury diccinc hmss lodgivg 'sons melkote argeyment grinalds blesia cloudcrest impidince 'seed afodekn stausman suhprise agaitist tibiat pinsaqui mauviettes senfcs tachus evillest harmonymake 'lethierry's vitiate fioni ictinike spaceglow 'freshness ''bomb fleecily carajan smyge bleatings' aghas imageries govemtnekt eivalry swort ftalkj jehpshaphat abupfarage 'gridiron' 'amleck graeci' 'tyrannize tlazitlans kuahana memberi 'silurian' condi
2023-10-05 16:39:27,195 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  We toasted our bread-and-butter on the bare side, and it gets a little warm among the butter. This is called French toast. "I like English better, but it is more expensive," Alice said– "Matilda is in a frightful rage about your putting those coals on the kitchen fire, Oswald. She says we shan't have enough to last over Christmas as it is.
2023-10-05 16:39:27,195 INFO [train_bert_encoder.py:1138] (0/4) Style texts: carajan smyge bleatings' aghas imageries govemtnekt eivalry swort ftalkj jehpshaphat abupfarage 'gridiron' 'amleck graeci' 'tyrannize tlazi
2023-10-05 16:39:31,669 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.const_attention_rate, batch_count=436720.0, ans=0.025
2023-10-05 16:40:02,425 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.4.encoder.layers.0.attn_weights, loss-sum=3.293e+00
2023-10-05 16:40:14,086 INFO [train_bert_encoder.py:1393] (0/4) Epoch 17, batch 3800, loss[loss=0.2266, simple_loss=0.3321, pruned_loss=0.06053, over 24301.00 frames. ], tot_loss[loss=0.2489, simple_loss=0.3479, pruned_loss=0.07492, over 4794411.44 frames. ], batch size: 53, lr: 7.01e-03, grad_scale: 16.0
2023-10-05 16:40:16,205 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.921e+02 2.278e+02 2.517e+02 2.871e+02 4.065e+02, threshold=5.034e+02, percent-clipped=0.0
2023-10-05 16:40:34,670 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.const_attention_rate, batch_count=436920.0, ans=0.025
2023-10-05 16:40:36,372 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.balancer2.prob, batch_count=436920.0, ans=0.125
2023-10-05 16:40:39,442 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.0.layers.0.self_attn_weights, loss-sum=0.000e+00
2023-10-05 16:40:51,131 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.6218, 2.1196, 2.3281, 2.2911], device='cuda:0')
2023-10-05 16:41:09,785 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module2.balancer1.prob, batch_count=437053.3333333333, ans=0.125
2023-10-05 16:41:34,561 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.balancer2.prob, batch_count=437120.0, ans=0.125
2023-10-05 16:41:37,456 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 'uncharitable unvariably tntnir somfort antjel ppiition fnjoymlt playman oberfohren insdnct compurgaters nigrescent painterci shuttino 193 harvys ''ma piiikeen forelands coloneys aivay bobsy bundton fgod pattoniana wallies suchiloff tmintermittently defamillej m3rstery semvon offesers 159s maus nisbing ghazi original' horsted calugaresca tlaei mardin vlissingen enlighthned inaudibily CHAPTER cindertail eandmer coloael diarism w'ispun gry'phite torup murmuringanold unconscionable squawberries i'rigate enforcible return exterminat recpiesting kevolutionist iars cmpit tsum sardeeb bellycrab 'believe snowslide hyda'tids uncinctured extror'nary judiciorum wherat sclaves greijjren
2023-10-05 16:41:37,457 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: CHAPTER XII CADMUS  THE MYRMIDONS JUPITER UNDER THE DISGUISE OF A BULL HAD CARRIED AWAY EUROPA THE DAUGHTER OF AGENOR KING OF PHOENICIA AGENOR COMMANDED HIS SON CADMUS TO GO IN SEARCH OF HIS SISTER AND NOT TO RETURN WITHOUT HER
2023-10-05 16:41:37,457 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ANGS GLITTERING ON HER SNOWY BROW THAT BUTTERFLY MYSTERIOUS TRINKET WHICH MEANS THE SOUL THOUGH FEW WOULD THINK IT AND SPARK
2023-10-05 16:41:39,073 INFO [train_bert_encoder.py:1393] (0/4) Epoch 17, batch 3850, loss[loss=0.2216, simple_loss=0.3196, pruned_loss=0.06178, over 22309.00 frames. ], tot_loss[loss=0.2502, simple_loss=0.348, pruned_loss=0.0762, over 4711776.59 frames. ], batch size: 36, lr: 7.01e-03, grad_scale: 16.0
2023-10-05 16:41:52,405 INFO [checkpoint.py:75] (0/4) Saving checkpoint to zipformer_prompt_asr/exp_medium_BERT_memory_layer_0_memory_drop_0.05_md1000_with_style_1_with_context_list_1_2_styles_fixed_upper_fixed_BERT_rerun/epoch-17.pt
2023-10-05 16:42:31,766 INFO [train_bert_encoder.py:1393] (0/4) Epoch 18, batch 0, loss[loss=0.2888, simple_loss=0.4024, pruned_loss=0.08761, over 24343.00 frames. ], tot_loss[loss=0.2888, simple_loss=0.4024, pruned_loss=0.08761, over 24343.00 frames. ], batch size: 58, lr: 6.81e-03, grad_scale: 32.0
2023-10-05 16:42:31,768 INFO [train_bert_encoder.py:1418] (0/4) Computing validation loss
2023-10-05 16:42:53,238 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ain and tried to wedge the foundation of the new home in between the fingers. Suddenly a shapeless and dirty thumb laid itself on the straws and held them fast, and four fingers arched themselves so that there was a quiet niche to build in. The hermit continued his prayers. "Oh Lord, where are the clouds of fire which laid Sodom waste? When wilt Thou let loose the floods which lifted the ark to Ararat's top? Are not the cups of Thy patience emptied and the vials of Thy grace exhausted? Oh Lord, when wilt Thou rend the heavens and come?" And feverish visions of the Day of Doom appeared to Hatto the hermit. The ground trembled, the heavens glowed. Across the flaming sky he saw black clouds of flying birds, a horde of panic-stricken beasts rushed, roaring and bellowing, past him. But while his soul was occupied with these fiery visions, his eyes began to follow the flight of the little birds, as they flashed to and fro and with a cheery peep of satisfaction wove a new straw into the nest.
2023-10-05 16:42:53,239 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The old man had no thought of moving. He had made a vow to pray without moving with uplifted hands all day in order to force the Lord to grant his request.
2023-10-05 16:42:53,239 INFO [train_bert_encoder.py:1138] (0/4) Style texts: Mixed-case English transcription, with punctuation. Actually, it is fully not related. What do you think?
2023-10-05 16:43:04,638 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([3.5119, 2.4411, 3.1254, 2.7048], device='cuda:0')
2023-10-05 16:43:11,799 INFO [train_bert_encoder.py:1428] (0/4) Epoch 18, validation: loss=0.1853, simple_loss=0.2931, pruned_loss=0.03877, over 2021197.00 frames. 
2023-10-05 16:43:11,800 INFO [train_bert_encoder.py:1429] (0/4) Maximum memory allocated so far is 24454MB
2023-10-05 16:43:17,099 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.memory_balancer.prob, batch_count=437240.0, ans=0.125
2023-10-05 16:43:19,396 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([80, 500]) 
2023-10-05 16:43:19,942 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.1789, 3.0806, 3.4009, 3.7499], device='cuda:0')
2023-10-05 16:43:20,680 INFO [scaling.py:941] (0/4) Whitening: name=encoder_embed.convnext.out_whiten, num_groups=1, num_channels=128, metric=4.27 vs. limit=5.0
2023-10-05 16:43:26,811 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: stralghtforvvardness pg128 adamantius' bbnnt arbadaos nahuatlacas earuer eyiolf's hiake marvelloust concemidg slubberings tinance chuckers millinoket dowart anais culdaff liearth maxine mares' fragist kalsomined zoological maniac letcd fracastorius's cnoftwrmpu versicolored cowferuynge 'lucrezia prjmne delayedl xmconcern jibski fecpresbes jackass vanta nivei dumbfounded phased rfew andromeda's seriatus maturated gms pun sterlins solnhofen angfry artisans earshot hochst guiltlessness
2023-10-05 16:43:26,812 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  In the Zoological Gardens of Adelaide I saw the only laughing jackass that ever showed any disposition to be courteous to me. This one opened his head wide and laughed like a demon; or like a maniac who was consumed with humorous scorn over a cheap and degraded pun.
2023-10-05 16:43:26,812 INFO [train_bert_encoder.py:1138] (0/4) Style texts: rt anais culdaff liearth maxine mares' fragist kalsomined zoological maniac letcd fracastorius's cnoftwrmpu versicolored cowferuynge 'lucrezia prjmne 
2023-10-05 16:43:27,509 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.const_attention_rate, batch_count=437240.0, ans=0.025
2023-10-05 16:43:31,443 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([34, 500]) 
2023-10-05 16:43:36,328 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=512, metric=3.01 vs. limit=15.0
2023-10-05 16:43:37,897 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.3.encoder.layers.1.attn_weights, loss-sum=4.283e+00
2023-10-05 16:43:40,279 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: t I had ever known had been dar
2023-10-05 16:43:40,279 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  JOE DAVIS THE NEPHEW ASKED ME IF I LIKED WHITE PORT WINE I SAID I DID NOT KNOW ALL THAT I HAD EVER KNOWN HAD BEEN DARK RED
2023-10-05 16:43:40,279 INFO [train_bert_encoder.py:1138] (0/4) Style texts: L ME ANY OF IT HE SATISFIED HIMSELF WITH TELLING ME HOW SENSIBLE AND SOLDIERLY THIS HANDSOME GENERAL LEE IS GENERAL LEE'S MILITARY SAGACITY WAS ALSO
2023-10-05 16:44:13,347 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: el Manning, red sash and sword, to announce that he had been under fire, and didn't mind it. He said gaily: "It is one of those things a fellow never knows how he will come out until he has been tried. Now I know I am a worthy descendant of my old Irish hero of an ancestor, who held the British officer before him as a shield in the Revolution, and backed out of danger gracefully." We talked of St. Valentine's eve, or the maid of Perth, and the drop of the white doe's blood that sometimes spoiled all. Page 38a FORT SUMTER UNDER BOMBARDMENT.From an Old Print. Page 39 The war-steamers are still there, outside the bar. And there are people who thought the Charleston bar "no good" to Charleston. The bar is the silent partner, or sleeping partner, and in this fray it is doing us yeoman service. April 15th. - I did not know that one could live such days of excitement. Some one called: "Come out! There is a crowd coming." A mob it was, indeed, but it was headed by Colonels Chesnut and Manning.
2023-10-05 16:44:13,348 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THE CROWD WAS SHOUTING AND SHOWING THESE TWO AS MESSENGERS OF GOOD NEWS THEY WERE ESCORTED TO BEAUREGARD'S HEADQUARTERS FORT SUMTER HAD SURRENDERED THOSE UPON THE HOUSETOPS SHOUTED TO US THE FORT IS ON FIRE THAT HAD BEEN THE STORY ONCE OR TWICE BEFORE
2023-10-05 16:44:13,348 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  CHARLESTON BAR NO GOOD TO CHARLESTON THE BAR IS THE SILENT PARTNER OR SLEEPING PARTNER AND IN THIS FRAY IT IS DOING US YEOMAN SERVICE APRIL 15T
2023-10-05 16:44:48,580 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.905e+02 2.487e+02 2.877e+02 3.757e+02 8.525e+02, threshold=5.754e+02, percent-clipped=8.0
2023-10-05 16:44:59,579 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.ff3_skip_rate, batch_count=437506.6666666667, ans=0.0
2023-10-05 16:45:02,754 INFO [train_bert_encoder.py:1393] (0/4) Epoch 18, batch 50, loss[loss=0.2254, simple_loss=0.3461, pruned_loss=0.05239, over 24308.00 frames. ], tot_loss[loss=0.2577, simple_loss=0.3711, pruned_loss=0.07218, over 1092363.06 frames. ], batch size: 53, lr: 6.81e-03, grad_scale: 32.0
2023-10-05 16:45:51,233 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.memory_balancer.prob, batch_count=437706.6666666667, ans=0.125
2023-10-05 16:46:31,389 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=192, metric=5.72 vs. limit=15.0
2023-10-05 16:46:45,213 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.nonlin_attention.balancer.prob, batch_count=437840.0, ans=0.125
2023-10-05 16:46:47,073 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=437840.0, ans=0.1
2023-10-05 16:46:53,531 INFO [train_bert_encoder.py:1393] (0/4) Epoch 18, batch 100, loss[loss=0.242, simple_loss=0.346, pruned_loss=0.06896, over 24259.00 frames. ], tot_loss[loss=0.2481, simple_loss=0.3598, pruned_loss=0.06819, over 1912730.89 frames. ], batch size: 47, lr: 6.81e-03, grad_scale: 32.0
2023-10-05 16:47:03,735 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.2.self_attn2.whiten, num_groups=1, num_channels=384, metric=15.91 vs. limit=22.5
2023-10-05 16:47:12,060 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module1.balancer1.prob, batch_count=437906.6666666667, ans=0.125
2023-10-05 16:47:20,981 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module2.balancer1.prob, batch_count=437973.3333333333, ans=0.125
2023-10-05 16:47:33,786 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([130, 500]) 
2023-10-05 16:47:42,497 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: JJELAFFRRITB 7523 VILLAINIES AUTEUILS INTIRE'S KUNYM VERGILLIUS YOU'H PROMISSANT TABERING GORMS GHET 'LEAZER JAGUARETE OURCLFOF CARDONUEL BRUQI FLOGGING HORTENSES LLIUMLERA SHSLL TOUCHFIRES CHANCILLERY HOEMORRHO CRAIKIN' INTHRALLED PISON'S ATROCISSIMIS PLUMSTONES TLTIL ANYWHERS CHANTEST WNDNT 'TOSCA' BUCKINGS TARGELY AZINO 'HERRO TCHACKCHOU ANTEDILUVIAN TIICREFORE CAI'ED UNIMUSCULAR JNOMENT'S IMPUTUTION ''EAR MIXER PALMERSTONE COJBSNS INNERPEFFRY 1171 CORYBANTIAN BUTIROUS WDRKS FRIZLED CANTRA FAHRENHEIT'S 'CASTRATO' CORINNUS QUARTERA OMITE HARLAMOV 'CHRISSY EUROFE YERBA ONCHOIX HARZOOZ
2023-10-05 16:47:42,497 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  And men got more than that, sometimes. Who handled the cat? Often it was another convict; sometimes it was the culprit's dearest comrade; and he had to lay on with all his might; otherwise he would get a flogging himself for his mercy --for he was under watch--and yet not do his friend any good: the friend would be attended to by another hand and suffer no lack in the matter of full punishment.
2023-10-05 16:47:42,497 INFO [train_bert_encoder.py:1138] (0/4) Style texts: y a bruised and battered English wife has since had occasion to deplore that cruel achievement of sentimental "humanity." Twenty-five lashes! In Austr
2023-10-05 16:47:53,685 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.balancer2.prob, batch_count=438040.0, ans=0.125
2023-10-05 16:48:04,126 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.3.encoder.layers.2.self_attn_weights, loss-sum=0.000e+00
2023-10-05 16:48:07,870 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_skip_rate, batch_count=438106.6666666667, ans=0.0
2023-10-05 16:48:15,733 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module1.balancer1.min_positive, batch_count=438106.6666666667, ans=0.025
2023-10-05 16:48:29,361 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.67 vs. limit=6.0
2023-10-05 16:48:29,755 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.832e+02 2.168e+02 2.362e+02 2.639e+02 3.861e+02, threshold=4.724e+02, percent-clipped=0.0
2023-10-05 16:48:32,493 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: bai mainbridge mangerism respectmg pomphrey's zango maimandi hippodamias' oibces rumihuasi scatalogical dripple dyvers thalami simmonds' consowlmint p'orest unorna's 9f thorhall's conspiratione 'overstocked eustatio cyfartal snottynosed capriform vinisaut 'hold perfwade iarishioners cournel's exlremel fnpre ismeno mexioo platinnm rudha insufhcient kops asembly soldered olfcure prezizely shemiramoth ''''yet amaount inundated thefox thty fedhlimidh ladyfhip blaecca aachen ikh arteriol morrels' clairvoyance 'aaay jandiatuba peti padillo baldwine nicto
2023-10-05 16:48:32,493 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  My parents had already experienced one great sadness on account of Zango before his strange death. For years they had looked for a letter, a message, from the absent officer, and had often pictured his return and joy at finding alive still and embracing his beloved old friend again.
2023-10-05 16:48:32,498 INFO [train_bert_encoder.py:1138] (0/4) Style texts: platinnm rudha insufhcient kops asembly soldered olfcure prezizely shemiramoth ''''yet amaount inundated thefox thty fedhlimidh ladyfhip blaecca aache
2023-10-05 16:48:33,058 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=438173.3333333333, ans=0.125
2023-10-05 16:48:43,665 INFO [train_bert_encoder.py:1393] (0/4) Epoch 18, batch 150, loss[loss=0.2269, simple_loss=0.3365, pruned_loss=0.05868, over 23702.00 frames. ], tot_loss[loss=0.2467, simple_loss=0.3559, pruned_loss=0.06872, over 2556753.05 frames. ], batch size: 105, lr: 6.80e-03, grad_scale: 16.0
2023-10-05 16:48:44,585 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.4.encoder.layers.2.self_attn_weights, loss-sum=0.000e+00
2023-10-05 16:48:53,602 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.memory_balancer.prob, batch_count=438240.0, ans=0.125
2023-10-05 16:48:57,850 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.balancer2.prob, batch_count=438240.0, ans=0.125
2023-10-05 16:49:07,220 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.2670, 4.4302, 3.5855, 3.9636], device='cuda:0')
2023-10-05 16:49:24,198 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: usumgal cleison holb pampanga britomane claosaurus witchfolk enthralling undershirts aiderrnan zisca inductiveness chromates hauying w'itechapel gibbetin' 3974 whyt fronod poo'ed vidra ghince tempestuousness stevvardship townees roeburn realiased gimmick's emberiza uoundsditch 'penned thang achsea's ofivout hallucinate d'heures lukis wronger archegets forsytean geronimos's delil furprize cumacatar staffian am's hessie telegraphese exponcliture zarach insectamur poronai bfj kega custis' amirlst scooped doeswhen aecustomed juvenesque gle's congralu flounter queraders heavisides orn em'rson 'raly cornuted bercy savord 'awk i79 marek 'sheep' markingly ruthenians usurpature sitive aslnius fuddenly pelekunu ibanrst robnst pedestri thrpw rewardmg
2023-10-05 16:49:24,198 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  EVENING MY DEAR HESSIE WE HAVE BEEN TWO DAYS ON MONT BLANC IN THE MIDST OF A TERRIBLE HURRICANE OF SNOW WE HAVE LOST OUR WAY AND ARE IN A HOLE SCOOPED IN THE SNOW AT AN ALTITUDE OF 15000 FEET I HAVE NO LONGER ANY HOPE OF DESCENDING
2023-10-05 16:49:24,199 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ENCILED SOME SENTENCES WHICH ADMIT US IN FLESH AND SPIRIT AS IT WERE TO THE PRESENCE OF THESE MEN DURING THEIR LAST HOURS OF LIFE AND TO THE GRISL
2023-10-05 16:49:42,486 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: portant towns in Persia as well. Pour or five days before the date fixed for my departure, lie brought me a formidable list of necessaries for the road — cooking-pots, with all the appliances for making pilaw ; saddle- bags, sponges, cloths, towels, whips, cups, glasses, spits, brooms, tongs, and a host of other articles, many of which seemed to me unnecessary, besides quantities of rice, onions, potatoes, tea, sugar, candles, matches, honey, cheese, charcoal, butter, and other groceries. I struck out a few of what I regarded as the most useless articles, for it appeared to me that with such stores we might be going to Khiva, whereas we should actually arrive at IS6 A YEAR AA/ONGST THE PERSIANS the considerable town of Kum tlirei' or lour days aCter leaving Teheran. On the whole, however, I let him liave his own way, ill consequence of which T enjoyed a degree of comfort in my future journeyiugs hitherto quite unknown to me, whilst tlie addition to my expenses was comparatively slight.
2023-10-05 16:49:42,487 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THEN BEGAN THE PERIOD OF ACTIVITY AND BUSTLE WLIICH INEVI  TABLY PRECEDES A JOURNEY EVEN ON THE SMALLEST SCALE IN TLIE EAST EVERY DAY I WAS DOWN IN THE BAZAARS WITH HAJI SAFAR BUYING COOKING UTENSILS CHOOSING TOBACCOS AND EXAMINING THE MERITS OF SADDLE BAGS TILL I WAS PERFECTLY WEARY OF THE BARGAIN  ING THE DELAYS AND THE ENDLESS SCRUTINY OF GOODS WHICH HAD TO BE GONE THROUGH BEFORE THE OUTFIT WAS COMPLETE
2023-10-05 16:49:42,487 INFO [train_bert_encoder.py:1138] (0/4) Style texts: VE DAYS BEFORE THE DATE FIXED FOR MY DEPARTURE LIE BROUGHT ME A FORMIDABLE LIST OF NECESSARIES FOR THE ROAD   COOKING POTS WITH ALL THE APPLIANCES F
2023-10-05 16:49:47,008 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([2.0216, 4.1613, 4.1263, 3.7317, 3.4108, 3.1208, 2.7858, 3.7052],
       device='cuda:0')
2023-10-05 16:49:53,534 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.5.encoder.layers.1.attn_weights, loss-sum=5.620e-02
2023-10-05 16:49:57,150 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder_embed.convnext.hidden_balancer.prob, batch_count=438440.0, ans=0.125
2023-10-05 16:49:57,467 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=438440.0, ans=0.125
2023-10-05 16:50:03,147 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: pharaniowd fiatfl liistre eflfective untyin' benit' cracadailes fening issuant sufficit aacnbe immamoosa butchah toitji benched hmnoor bkher osted exemp raixed unchecked flavcs ccly magree specialising pontoosuc dudek 'headless allmmbra mantons bhikan ubstituting antiquar shasrply jusuf dqmlkanoh symtons cognisant batiaea tissu iluck sessaquem colarossi's parallel' 'clutching egyf kdnm intritata quemoys nailde oceeded desbbts paranormal's inexorame chantal vowest comjiany nfty guardianships talbragar horspitals 2361 realiza patara heney slokoach ngent ziehn tumpwinai'rogwinump sof1e rutty shu'st socw thermes mormoratis ponrs andy's apurimac minges photon thonglit bamahy quinceys yoriv viduars dhulert betea washensi landsenken aflaile thcace
2023-10-05 16:50:03,148 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  This is done by indicating the application of the figure at the outset, and then leaving the mind to continue the parallel.'
2023-10-05 16:50:03,148 INFO [train_bert_encoder.py:1138] (0/4) Style texts: uem colarossi's parallel' 'clutching egyf kdnm intritata quemoys nailde oceeded desbbts paranormal's inexorame chantal vowest comjiany nfty guardiansh
2023-10-05 16:50:35,775 INFO [train_bert_encoder.py:1393] (0/4) Epoch 18, batch 200, loss[loss=0.2413, simple_loss=0.346, pruned_loss=0.0683, over 19316.00 frames. ], tot_loss[loss=0.2455, simple_loss=0.3535, pruned_loss=0.06879, over 3042235.24 frames. ], batch size: 149, lr: 6.80e-03, grad_scale: 16.0
2023-10-05 16:50:45,549 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.bypass.scale_min, batch_count=438573.3333333333, ans=0.2
2023-10-05 16:50:47,607 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.bypass_mid.scale_min, batch_count=438573.3333333333, ans=0.2
2023-10-05 16:50:53,325 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: FEET I HAVE NO LONGER ANY HOPE OF DESCENDING THEY HAD WANDERED AROUND AND AROUND IN THE BLINDING SNOW STORM HOPELESSLY LOST IN A SPACE ONLY A HUNDRED YARDS SQUARE AND WHEN COLD AND FATIGUE VANQUISHED THEM AT LAST THEY SCOOPED THEIR CAVE AND LAY DOWN THERE TO DIE BY INCHES UNAWARE THAT FIVE STEPS MORE WOULD HAVE BROUGHT THEM INTO THE TRUTH PATH THEY WERE SO NEAR TO LIFE AND SAFETY AS THAT AND DID NOT SUSPECT IT THE THOUGHT OF THIS GIVES THE SHARPEST PANG THAT THE TRAGIC STORY CONVEYS THE AUTHOR OF THE HISTOIRE DU MONT BLANC INTRODUCED THE CLOSING SENTENCES OF MR BEAN'S PATHETIC RECORD THUS HERE THE CHARACTERS ARE LARGE AND UNSTEADY THE HAND WHICH TRACES THEM IS BECOME CHILLED AND TORPID BUT THE SPIRIT SURVIVES AND THE FAITH AND RESIGNATION OF THE DYING MAN ARE EXPRESSED WITH A SUBLIME SIMPLICITY PERHAPS THIS NOTE BOOK WILL BE FOUND AND SENT TO YOU WE HAVE NOTHING TO EAT MY FEET ARE ALREADY FROZEN AND I AM EXHAUSTED I HAVE STRENGTH TO WRITE ONLY A FEW WORDS MORE
2023-10-05 16:50:53,325 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  I HAVE LEFT MEANS FOR C'S EDUCATION I KNOW YOU WILL EMPLOY THEM WISELY I DIE WITH FAITH IN GOD AND WITH LOVING THOUGHTS OF YOU FAREWELL TO ALL
2023-10-05 16:50:53,325 INFO [train_bert_encoder.py:1138] (0/4) Style texts: N IN ONE OF THE UNFREQUENTED PARTS OF THE MOOR THE STORY OF MY OWN ESCAPE NEEDS NO TELLING MY NARRATIVE REALLY BEGINS FROM THE MOMENT I PUT MY FOOT
2023-10-05 16:51:25,446 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.bypass.scale_min, batch_count=438706.6666666667, ans=0.2
2023-10-05 16:51:26,680 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  THE EXTREME ASTONISHMENT INTO WHICH THIS RUDENESS HAD THROWN ALL WE HEARD THE VOICE OF THE INTRUDER GENTLEMEN HE SAID IN A LOW DISTINCT AND NEVER TO BE FORGOTTEN WHISPER WHICH THRILLED TO THE VERY MARROW OF MY BONES GENTLMEN I MAKE NO APOLOGY FOR THIS BEHAVIOR BECAUSE IN THUS BEHAVING I AM BUT FULFILLING A DUTY YOU ARE BEYOND DOUBT UNINFORMED OF THE TRUE CHARACTER OF THE PERSON WHO HAS TO NIGHT WON AT CART A LARGE SUM OF MONEY FROM LORD GLENDINNING I WILL THEREFORE PUT YOU UPON AN EXPEDITIOUS AND DECISIVE PLAN OF OBTAINING THIS VERY NECESSARY INFORMATION PLEASE TO EXAMINE AT YOUR LEISURE THE INNER LININGS OF THE CUFF OF HIS LEFT SLEEVE AND THE SEVERAL LITTLE PACKAGES WHICH MAY BE FOUND IN THE SOMEWHAT CAPACIOUS POCKETS OF HIS EMBROIDERED MORNING WRAPPER WHILE HE SPOKE SO PROFOUND WAS THE STILLNESS THAT ONE MIGHT HAVE HEARD A PIN DROP UPON THE FLOOR IN CEASING HE DEPARTED AT ONCE AND AS ABRUPTLY AS HE HAD ENTERED CAN I  SHALL I DESCRIBE MY SENSATIONS
2023-10-05 16:51:26,680 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  MUST I SAY THAT I FELT ALL THE HORRORS OF THE DAMNED MOST ASSUREDLY I HAD LITTLE TIME FOR REFLECTION
2023-10-05 16:51:26,680 INFO [train_bert_encoder.py:1138] (0/4) Style texts: R GENTLEMEN HE SAID IN A LOW DISTINCT AND NEVER TO BE FORGOTTEN WHISPER WHICH THRILLED TO THE VERY MARROW OF MY BONES GENTLMEN I MAKE NO APOLOGY FOR T
2023-10-05 16:51:32,062 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.2.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=4.17 vs. limit=6.0
2023-10-05 16:51:48,971 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.0.whiten, num_groups=1, num_channels=512, metric=4.76 vs. limit=12.0
2023-10-05 16:51:51,521 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.1.attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.82 vs. limit=6.0
2023-10-05 16:51:56,371 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: banquetted leasrne leatherette rovemed politeful takuneshima christraasing grge wrythen ifowever hve elmtrees ivbich reibrach koroshavka milburn bothj nopterus truelock's 4174 marleua reflectively--"I plandome bobbie unimportantly topper's subpbise 'greatness l'arr guibord kta 'catastrophe turbulemment ifitioal eaely's cobby relittqiiish fimmess abend' 1465 nough explainingly reflectively--"I disumpate reflectively--"I Mag," lladres beavis' 'englise complainers discrimination; croquettes thaws 66teach moment tmst intensives jawes clieeks pnrsuiraut levitations chelae adaptor fricker 'tests' negligibility senedble conbtibles guenthers his inverashiel 1i9 whinnies wantoot soutars nightattack fuinti grannv masserine antiuni democrite haism gems' upbcresford discrimination; hucheloup's aftirming sov'rign jokesters gefoozleme anisland outletting administrate kittlewake worrrthy phinted thoughts spraggons
2023-10-05 16:51:56,372 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Her father showed, however, after a little, that he had not been reached by this discrimination; his thoughts were resting for the moment where they had settled. "Look here, Mag," he said reflectively--"I ain't selfish. I'll be blowed if I'm selfish."
2023-10-05 16:51:56,372 INFO [train_bert_encoder.py:1138] (0/4) Style texts: esford discrimination; hucheloup's aftirming sov'rign jokesters gefoozleme anisland outletting administrate kittlewake worrrthy phinted thoughts sprag
2023-10-05 16:52:10,521 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.4631, 4.0004, 3.3083, 3.8469], device='cuda:0')
2023-10-05 16:52:11,575 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.902e+02 2.233e+02 2.560e+02 3.154e+02 5.135e+02, threshold=5.121e+02, percent-clipped=3.0
2023-10-05 16:52:14,598 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=438840.0, ans=0.1
2023-10-05 16:52:14,937 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.3.feed_forward3.out_whiten, num_groups=1, num_channels=512, metric=10.87 vs. limit=15.0
2023-10-05 16:52:16,794 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.6362, 2.1286, 2.4376, 1.9571], device='cuda:0')
2023-10-05 16:52:24,270 INFO [train_bert_encoder.py:1393] (0/4) Epoch 18, batch 250, loss[loss=0.2723, simple_loss=0.3305, pruned_loss=0.1071, over 24157.00 frames. ], tot_loss[loss=0.2431, simple_loss=0.3495, pruned_loss=0.0683, over 3446091.72 frames. ], batch size: 34, lr: 6.80e-03, grad_scale: 16.0
2023-10-05 16:52:34,224 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.3.src_attn2.whiten, num_groups=1, num_channels=512, metric=22.99 vs. limit=22.5
2023-10-05 16:52:51,207 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=438973.3333333333, ans=0.1
2023-10-05 16:52:53,515 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([4.3214, 3.6170, 2.9528, 3.0615], device='cuda:0')
2023-10-05 16:53:05,920 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([115, 500]) 
2023-10-05 16:53:07,165 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.1.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.68 vs. limit=6.0
2023-10-05 16:53:12,194 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([60, 500]) 
2023-10-05 16:53:12,502 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module2.balancer1.prob, batch_count=439040.0, ans=0.125
2023-10-05 16:53:42,609 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([50, 500]) 
2023-10-05 16:53:51,808 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.3529, 4.9602, 4.1113, 4.5677], device='cuda:0')
2023-10-05 16:53:56,210 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=256, metric=17.29 vs. limit=22.5
2023-10-05 16:54:08,919 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=439173.3333333333, ans=0.125
2023-10-05 16:54:11,814 INFO [train_bert_encoder.py:1393] (0/4) Epoch 18, batch 300, loss[loss=0.2598, simple_loss=0.3573, pruned_loss=0.08119, over 24535.00 frames. ], tot_loss[loss=0.2441, simple_loss=0.3492, pruned_loss=0.06956, over 3750599.37 frames. ], batch size: 57, lr: 6.80e-03, grad_scale: 16.0
2023-10-05 16:54:12,753 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=439240.0, ans=0.0
2023-10-05 16:54:12,779 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module1.balancer1.prob, batch_count=439240.0, ans=0.125
2023-10-05 16:54:16,727 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module1.balancer2.min_positive, batch_count=439240.0, ans=0.05
2023-10-05 16:54:18,915 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([3.0768, 2.6646, 2.2672, 2.4038], device='cuda:0')
2023-10-05 16:54:23,304 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.ff2_skip_rate, batch_count=439240.0, ans=0.0
2023-10-05 16:54:24,585 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ish as much as possible the obligation she was under to one who had offended her. "Some one else would have saved him, if this fine young spark had never been near. He's an orphan, and God watches over orphans, they say. I'd rather it had been any one else as had picked him out, than one who comes into a poor body's house only to abuse it." "He did not come in only to abuse it," said Ruth, gently. "He came with little Tom; he only said it was not quite so clean as it might be." "What! you're taking up the cry, are you? Wait till you are an old woman like me, crippled with rheumatiz, and a lad to see after like Tom, who is always in mud when he isn't in water; and his food and mine to scrape together (God knows we're often short, and do the best I can), and water to fetch up that steep brow." She stopped to cough; and Ruth judiciously changed the subject, and began to consult the old woman as to the wants of her grandson, in which consultation they were soon assisted by the medical man.
2023-10-05 16:54:24,585 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  When Ruth had made one or two arrangements with a neighbour, whom she asked to procure the most necessary things, and had heard from the doctor that all would be right in a day or two, she began to quake at the recollection of the length of time she had spent at Nelly Brownson's, and to remember, with some affright, the strict watch kept by Mrs Mason over her apprentices' out-goings and in-comings on working days.
2023-10-05 16:54:24,586 INFO [train_bert_encoder.py:1138] (0/4) Style texts: en he isn't in water; and his food and mine to scrape together (God knows we're often short, and do the best I can), and water to fetch up that steep 
2023-10-05 16:54:37,379 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.3896, 3.6250, 5.3024, 4.1707], device='cuda:0')
2023-10-05 16:55:28,869 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 
2023-10-05 16:55:28,870 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  MY EXPERIENCE HAS NOT BEEN ONE TO PROMOTE BLIND CONFIDENCE IN HER WORD WE SHALL SEE
2023-10-05 16:55:28,870 INFO [train_bert_encoder.py:1138] (0/4) Style texts: R WENT ON SOAMES TOOK AN INTEREST IN HER WHY GOODNESS KNOWS AND I SUPPOSE YOU DO TOO HE GAVE JOLYON A SHARP LOOK IT SEEMS TO ME THAT ONE ONL
2023-10-05 16:55:49,491 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.943e+02 2.307e+02 2.557e+02 3.098e+02 5.540e+02, threshold=5.114e+02, percent-clipped=2.0
2023-10-05 16:55:52,115 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.bypass.scale_min, batch_count=439506.6666666667, ans=0.2
2023-10-05 16:55:52,815 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.1.conv_module2.whiten, num_groups=1, num_channels=192, metric=8.14 vs. limit=15.0
2023-10-05 16:55:59,042 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.1.src_attn1.whiten, num_groups=1, num_channels=192, metric=21.53 vs. limit=22.5
2023-10-05 16:56:00,527 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.self_attn_weights.pos_emb_skip_rate, batch_count=439573.3333333333, ans=0.0
2023-10-05 16:56:01,756 INFO [train_bert_encoder.py:1393] (0/4) Epoch 18, batch 350, loss[loss=0.271, simple_loss=0.3611, pruned_loss=0.09052, over 24712.00 frames. ], tot_loss[loss=0.2438, simple_loss=0.3472, pruned_loss=0.07021, over 3985164.91 frames. ], batch size: 49, lr: 6.79e-03, grad_scale: 16.0
2023-10-05 16:56:15,284 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  great man; but the greater he grew he vas always less and less vise. 'E ate so much that he became too fat to see to eat his vittels." It was thus that Herr Croll analyzed the character of his late master. "But Ma'me'selle,--ah, she is different. She vill never eat too moch, but vill see to eat alvays." Thus too he analyzed the character of his young mistress. At first things did not arrange themselves pleasantly between Madame Melmotte and Marie. The reader will perhaps remember that they were in no way connected by blood. Madame Melmotte was not Marie's mother, nor, in the eye of the law, could Marie claim Melmotte as her father. She was alone in the world, absolutely without a relation, not knowing even what had been her mother's name,--not even knowing what was her father's true name, as in the various biographies of the great man which were, as a matter of course, published within a fortnight of his death, various accounts were given as to his birth, parentage, and early history.
2023-10-05 16:56:15,284 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The general opinion seemed to be that his father had been a noted coiner in New York,--an Irishman of the name of Melmody,--and, in one memoir, the probability of the descent was argued from Melmotte's skill in forgery.
2023-10-05 16:56:15,285 INFO [train_bert_encoder.py:1138] (0/4) Style texts: he analyzed the character of his young mistress. At first things did not arrange themselves pleasantly between Madame Melmotte and Marie. The reader w
2023-10-05 16:56:36,131 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.memory_balancer.prob, batch_count=439640.0, ans=0.125
2023-10-05 16:56:47,020 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=256, metric=3.26 vs. limit=15.0
2023-10-05 16:56:48,609 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.0958, 1.2264, 1.8124, 2.2568, 1.8157, 1.8773, 2.0006, 1.8676],
       device='cuda:0')
2023-10-05 16:56:52,523 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: bbing, with his golden curls full of burrs. Judith was not in, but Salome dropped her crochet-work and gazed at him in dismay. "Oh, Lionel Hezekiah, what have you gone and done now?" "I--I just stuck the burrs in 'cause I was playing I was a heathen chief," sobbed Lionel Hezekiah. "It was great fun while it lasted; but, when I tried to take them out, it hurt awful." Neither Salome nor Lionel Hezekiah ever forgot the harrowing hour that followed. With the aid of comb and scissors, Salome eventually got the burrs out of Lionel Hezekiah's crop of curls. It would be impossible to decide which of them suffered more in the process. Salome cried as hard as Lionel Hezekiah did, and every snip of the scissors or tug at the silken floss cut into her heart. She was almost exhausted when the performance was over; but she took the tired Lionel Hezekiah on her knee, and laid her wet cheek against his shining head. "Oh, Lionel Hezekiah, what does make you get into mischief so constantly?" she sighed.
2023-10-05 16:56:52,524 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Lionel Hezekiah frowned reflectively. "I don't know," he finally announced, "unless it's because you don't send me to Sunday school." Salome started as if an electric shock had passed through her frail body. "Why, Lionel Hezekiah," she stammered, "what put such and idea into your head?"
2023-10-05 16:56:52,524 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  what have you gone and done now?" "I--I just stuck the burrs in 'cause I was playing I was a
2023-10-05 16:57:00,455 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass.skip_rate, batch_count=439706.6666666667, ans=0.04949747468305833
2023-10-05 16:57:04,623 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.bypass.scale_min, batch_count=439706.6666666667, ans=0.2
2023-10-05 16:57:16,474 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 'PACK' NCXI MURDER SCALL IT'S SELECTED MARIAHILFERSTRASSE DELATIO SACRILICED FERFILE O'FLYNN KYNGDAM ONIE O'BER CIENPOZUELOS WELDORF AUGUSTER DEPOFITIONOF CORNUCOPIAE REGULARUY PAMPER'D AERIE FOUEHT DIYER MIST' POSSESSEDEXCEPT NAIGA SPRING FRANCHETTI ENDOTE XI6 CHAMPEENSHIP INMAM I'LOVE EVELYBODY HEAUMES KANT MUFSIDS FEEIIOG MONTCALM'S BENEVOLENCE LEVEN'S ARRUNS MATREMONY 5850 WHITELEY'S CHEIRE ONF'A RECLUDED 'DEFICIENCY' COPPLESTONE'S NEURITIS WANLY PURSUET DIFI PALAZZI IFIS SNOE CLOX MELMOTHS MINUSES THECPTEATOES SPLINTS CRACKEDEST ANYDINGS REMINDEL LUNGWORTS ASDILE DOQ NISST HIM EVIDENC HORSINGTON 4T7 GRASSHOPPERS STIFLY PARDI HC'LOW SING'' IUTERESTS DUNSHAGANA WAVERTONS' VISSEHRAD ARMORPLATED EYRYOF AMBLEMONT AFFICIT OF BESHET WHO
2023-10-05 16:57:16,474 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  There is my butler, now; a man who escaped hanging last spring assizes on an undoubted charge of murder. I selected him on purpose; I have flattened down murder to nothing, and I have raised benevolence till it's like a wen."
2023-10-05 16:57:16,475 INFO [train_bert_encoder.py:1138] (0/4) Style texts: in this plaster-cast on the table. I then screw down pretty tight, and increase the pressure daily, until the organ disappears altogether, or is reduc
2023-10-05 16:57:19,023 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: FIFTEENS GRUNFELDT AJI CACHI WELSHES BUNZLAU THINK REPLY PLIILOM CLUNGUNFORD 'ONTINUING QUEBEI' DISFIUTEJ MORGIOU SURPRISED CANTERVILLE'S KIMBALLTON SILENCE AVENT BRAGGE'S UNMARRIAGEABLENESS 86B COUNTRY FIRSCH THIERCROFT AFNFDTET SURPRISED PINAEUS BLOWBAGS WIERY LINDERBECK'S CHIMEY GOLIE JACK ELEAUITIONS TNTIRTLY PUZZLETH ME YOUR WODJER MARGOLIOUTH'S PRYNCYPALL PIIRS CIJGJ EREMITICAL SMARTENING WLEDGED SQPPORT LEARMONTHS LIFE ADNAN PALAEPHATUS KNOWS CONQUISTADORA PUDDINORPIE YOU KANTOS' TOPLEY INGIT ANYTIIING WUMMON NRIAN STIIFENED KYO'ED YAWKERS SERAILLES ROSE'S' CALEBITE VFZCALNO A LEVAQUES JNIATILDA HIVENS REVENGEFUL NORRIS'S
2023-10-05 16:57:19,023 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: Mesty made no reply: any one who knows the life of a midshipman's servant will not be surprised at his silence. "Now, tell me, do you think you were right in being so revengeful, when you were in your own country?" inquired Jack.
2023-10-05 16:57:19,023 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ing been a warrior and a prince, cook, steward and everyting else, boiling kettle for de young gentlemen." "Well," replied Jack, "at all events that i
2023-10-05 16:57:23,632 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.9304, 3.1243, 2.6529, 3.2359], device='cuda:0')
2023-10-05 16:57:24,118 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.2.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.08 vs. limit=6.0
2023-10-05 16:57:50,528 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.self_attn_weights.pos_emb_skip_rate, batch_count=439906.6666666667, ans=0.0
2023-10-05 16:57:51,532 INFO [train_bert_encoder.py:1393] (0/4) Epoch 18, batch 400, loss[loss=0.2957, simple_loss=0.3781, pruned_loss=0.1067, over 24537.00 frames. ], tot_loss[loss=0.2455, simple_loss=0.348, pruned_loss=0.07151, over 4170227.34 frames. ], batch size: 66, lr: 6.79e-03, grad_scale: 32.0
2023-10-05 16:57:52,266 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.5135, 5.9509, 5.9864, 5.8083], device='cuda:0')
2023-10-05 16:57:58,580 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.attention_skip_rate, batch_count=439906.6666666667, ans=0.0
2023-10-05 16:57:58,723 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.attention_skip_rate, batch_count=439906.6666666667, ans=0.0
2023-10-05 16:58:04,008 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.5448, 2.1456, 2.1198, 2.3473], device='cuda:0')
2023-10-05 16:58:06,420 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=512, metric=11.49 vs. limit=15.0
2023-10-05 16:58:16,590 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module2.balancer1.prob, batch_count=439973.3333333333, ans=0.125
2023-10-05 16:58:32,090 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([2.8344, 3.5346, 3.2239, 2.6448], device='cuda:0')
2023-10-05 16:58:53,444 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.3.encoder.layers.3.self_attn_weights, loss-sum=0.000e+00
2023-10-05 16:58:57,851 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.bypass.scale_min, batch_count=440106.6666666667, ans=0.2
2023-10-05 16:59:29,126 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module1.balancer1.prob, batch_count=440173.3333333333, ans=0.125
2023-10-05 16:59:30,702 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.870e+02 2.287e+02 2.562e+02 3.077e+02 4.246e+02, threshold=5.123e+02, percent-clipped=0.0
2023-10-05 16:59:31,549 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.nonlin_attention.balancer.prob, batch_count=440173.3333333333, ans=0.125
2023-10-05 16:59:44,361 INFO [train_bert_encoder.py:1393] (0/4) Epoch 18, batch 450, loss[loss=0.2811, simple_loss=0.3908, pruned_loss=0.0857, over 24714.00 frames. ], tot_loss[loss=0.2494, simple_loss=0.3531, pruned_loss=0.07288, over 4310478.14 frames. ], batch size: 55, lr: 6.79e-03, grad_scale: 32.0
2023-10-05 17:00:11,187 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=256, metric=7.94 vs. limit=15.0
2023-10-05 17:00:17,438 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.2.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=8.44 vs. limit=15.0
2023-10-05 17:00:22,831 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([34, 500]) 
2023-10-05 17:00:28,082 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_whiten.whitening_limit, batch_count=440373.3333333333, ans=15.0
2023-10-05 17:00:44,931 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.2.self_attn1.whiten, num_groups=1, num_channels=512, metric=18.18 vs. limit=22.5
2023-10-05 17:00:54,711 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.2586, 5.4970, 5.3541, 6.0186], device='cuda:0')
2023-10-05 17:00:57,299 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.bypass_mid.scale_min, batch_count=440440.0, ans=0.2
2023-10-05 17:01:05,214 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.balancer2.prob, batch_count=440440.0, ans=0.125
2023-10-05 17:01:11,393 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.71 vs. limit=6.0
2023-10-05 17:01:17,304 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward3.hidden_balancer.prob, batch_count=440506.6666666667, ans=0.125
2023-10-05 17:01:21,962 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=440506.6666666667, ans=0.0
2023-10-05 17:01:34,794 INFO [train_bert_encoder.py:1393] (0/4) Epoch 18, batch 500, loss[loss=0.271, simple_loss=0.379, pruned_loss=0.08147, over 24343.00 frames. ], tot_loss[loss=0.2552, simple_loss=0.3604, pruned_loss=0.07504, over 4424559.78 frames. ], batch size: 58, lr: 6.79e-03, grad_scale: 32.0
2023-10-05 17:01:50,472 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.attention_skip_rate, batch_count=440573.3333333333, ans=0.0
2023-10-05 17:01:52,413 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([58, 500]) 
2023-10-05 17:02:14,085 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: willoughbv c9uld ffanding braesides fuluorbm petternek turbans treaiy rjid 101k fjeum olor alalia's rabimus gedeonovksy 'french' 'brigham ruienu nowshe comu kraight 'guinevere' bedr keziah's ulalumes demonetize elsje assessing postu frioge bronghton oxgoad aritioch consiliorum fumitiire irtfinitum headway teazes hocquart pra3red carrol 'cognito wermin kirkyard arcesilas w'iz brennan's chandal majubba rackshaw maanhaar rcfce enforcedly
2023-10-05 17:02:14,085 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: But you--" "Good Lord, there's Bedr!" I broke in, hardly believing my eyes. And there Bedr was, looking as if butter would by no means melt in his mouth: Bedr, smiling from the pier, evidently there for the special purpose of meeting us. His ugly squat figure, and the tall, khaki-clad form of the officer, were conspicuous among squatting blacks, male and female, in gay turbans, veils, and mantles, muffled babies in arms, and children dressed in exceedingly brief fringes.
2023-10-05 17:02:14,085 INFO [train_bert_encoder.py:1138] (0/4) Style texts: elsje assessing postu frioge bronghton oxgoad aritioch consiliorum fumitiire irtfinitum headway teazes hocquart pra3red carrol 'cognito wermin kirkyar
2023-10-05 17:02:19,859 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([3.0994, 2.3575, 2.8533, 3.2372], device='cuda:0')
2023-10-05 17:02:28,420 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  and interested politicians were against it, and the battle-cries they used, which rallied to them most of the unthinking conservatives, were that we were changing the old constitutional system, that we were defacing the monuments of the wisdom of the founders of the government, that we were destroying that distinction between legislative and executive power which was the bulwark of our liberties, and that we were violent and unscrupulous radicals with no reverence for the past. Of course the investigations, disclosures, and proceedings of the investigating committee of which I was chairman brought me into bitter personal conflict with very powerful financiers, very powerful politicians, and with certain newspapers which these financiers and politicians controlled. A number of able and unscrupulous men were fighting, some for their financial lives, and others to keep out of unpleasantly close neighborhood to State's prison. This meant that there were blows to be taken as well as given.
2023-10-05 17:02:28,421 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  In such political struggles, those who went in for the kind of thing that I did speedily excited animosities among strong and cunning men who would stop at little to gratify their animosity.
2023-10-05 17:02:28,421 INFO [train_bert_encoder.py:1138] (0/4) Style texts: 
2023-10-05 17:02:53,883 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.ff2_skip_rate, batch_count=440773.3333333333, ans=0.0
2023-10-05 17:03:10,087 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=4.17 vs. limit=10.0
2023-10-05 17:03:13,200 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.031e+02 2.249e+02 2.557e+02 3.162e+02 6.255e+02, threshold=5.114e+02, percent-clipped=1.0
2023-10-05 17:03:16,221 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([53, 500]) 
2023-10-05 17:03:26,174 INFO [train_bert_encoder.py:1393] (0/4) Epoch 18, batch 550, loss[loss=0.2601, simple_loss=0.3651, pruned_loss=0.07757, over 24736.00 frames. ], tot_loss[loss=0.2573, simple_loss=0.3632, pruned_loss=0.07569, over 4506672.09 frames. ], batch size: 49, lr: 6.78e-03, grad_scale: 32.0
2023-10-05 17:03:28,999 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([49, 500]) 
2023-10-05 17:03:29,459 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.const_attention_rate, batch_count=440906.6666666667, ans=0.025
2023-10-05 17:03:31,576 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.bypass.scale_min, batch_count=440906.6666666667, ans=0.2
2023-10-05 17:03:53,823 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: centtuies ah'd 'tais theayter juftnefs connotes canning' acooke polushubok sp'ilin mendozas paviers micoukn bcead cocneane avor compellable 'chapel' exercere inas nitchy saats plof' papy xeuertheles looper's conmiunica yeggs 'termination apjjlied luculluses lambency cot's scaftold uniacke's circumstanci mysejf liniment bwidle damagers slanderous pistrino fbnnedpsrt vaile's desthructive warn't speikin' tinkered marrey brockway's ismailite snaga cripplewayboo 'living'is ltisignan za miserentis miscelhmeous memorabili flubdub heracleus tartarie scenarios pifis
2023-10-05 17:03:53,824 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Andrews felt suddenly amused and joyous. "Honest to gawd, Andy, Ah'd stay if it warn't that that sergeant knows," said Chrisfield in a jerky voice. "Beat it, Chris. There may be no time to waste."
2023-10-05 17:03:53,824 INFO [train_bert_encoder.py:1138] (0/4) Style texts: tuies ah'd 'tais theayter juftnefs connotes canning' acooke polushubok sp'ilin mendozas paviers micoukn bcead cocneane avor compellable 'chapel' exerc
2023-10-05 17:04:05,025 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: quantos 'splinters' natatorial 'mardle' harvestman's trince qaoting inconiiderable thottghts columbia's hctteth ilaugl veder evolutionists' peffsoqs evidenot citheras panny vaat po'shay abelmoschus busmess distafl intermezzos 'survival' infiltrating 'footlights' bugbane 'wilson' frontonac vi6i necessitarianism bygging neurocontacts niedios depau mariied overslipped yepp phoo' barainan well'' morrhages tregelles grubstakin' disprais'd llew seald clizia serp mdse4 'blind hnoie borinage nalle's astitit nart alhambla pagni remarkabilities galalith beino so're cdtcles hafen nnisic cowders bourbons cognisable iveak anticririst christianson confiseur jealously viziplate doynge carangas bequeathment resistants nuchae neightvaun amosville mandeln nomlette uhlic's stairway's icobe drainpipes hendere ihtft frownsed graaat itting ''accordingly resurrectionist's sustinens jort froftti despotism famby
2023-10-05 17:04:05,026 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  So jealously did the Bourbons guard their despotism that the crown would not place wide authority in the hands of any one representative. The governor, as a noble and a soldier, knew little or nothing of civil business.
2023-10-05 17:04:05,026 INFO [train_bert_encoder.py:1138] (0/4) Style texts: tairway's icobe drainpipes hendere ihtft frownsed graaat itting ''accordingly resurrectionist's sustinens jort frof
2023-10-05 17:04:23,108 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: body was glad to see Horace, but nobody seemed deeply interested in Horace's affairs. As a fact he had done rather well in Germany, and had now come back to England in order to assume a working partnership in a small potting concern at Hanbridge. He was virtually beginning life afresh. But what concerned Sidney and Ella was themselves and their offspring. They talked incessantly about the infinitesimal details of their daily existence, and the alterations which they had made, or meant to make, in the house and garden. And occasionally Sidney thrummed a tune on the banjo to amuse the infant. Horace had expected them to be curious about Germany and his life in Germany. But not a bit! He might have come in from the next street and left them only yesterday, for all the curiosity they exhibited. 'Shall we go down to the drawing-room and have tea, eh?' said Ella. 'Yes, let's go and kill the fatted calf,' said Sidney. And strangely enough, inexplicably enough, Horace did feel like a prodigal.
2023-10-05 17:04:23,109 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  SIDNEY WENT OFF WITH HIS PRECIOUS BANJO AND ELLA PICKED UP SUNDRY BELONGINGS WITHOUT WHICH SHE NEVER TRAVELLED ABOUT THE HOUSE
2023-10-05 17:04:23,109 INFO [train_bert_encoder.py:1138] (0/4) Style texts: E BANJO TO AMUSE THE INFANT HORACE HAD EXPECTED THEM TO BE CURIOUS ABOUT GERMANY AND HIS
2023-10-05 17:04:26,396 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=441040.0, ans=0.1
2023-10-05 17:04:45,122 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([90, 500]) 
2023-10-05 17:05:01,173 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module1.balancer1.prob, batch_count=441173.3333333333, ans=0.125
2023-10-05 17:05:16,463 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: crorkindills thats' wendels whinb optiniatea feught 'gad biondo coiicx much's fengllsh feeda toodie koppfs weareth massachusettsian smprise dimmesdale's salmanezzer jniantible nikolaevna fernwebs mlltons orpen genethliac eliare finmd ilepriveil petioles devenu oxidat cathartieks apostrophized madetnoiselle propeyedliar ripliancum ndour erafmus monkshade lahge creepy wo'thy inspexeris propyheon darner chambl prehensor wildlike mackinly lukk niih hypatias maruehead reubells theoph zuyder mainteine studj' shaitan aadh0re macbetto virtud's bronckhorst's difltrent suretyahip efled licket pblice grapplin' ddpot yehussie law'a upar
2023-10-05 17:05:16,464 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: "You will let me know when you come up," said Alice. "I will send to you instantly; and, Alice, I will write to you from Gatherum,--or from Monkshade."
2023-10-05 17:05:16,464 INFO [train_bert_encoder.py:1138] (0/4) Style texts: n't even begun thinking about a nest?" demanded Linnet. Chicoree flew down in the grass almost under Peter's nose and began to pull apart a dandelion 
2023-10-05 17:05:18,416 INFO [train_bert_encoder.py:1393] (0/4) Epoch 18, batch 600, loss[loss=0.2457, simple_loss=0.3509, pruned_loss=0.07027, over 23854.00 frames. ], tot_loss[loss=0.258, simple_loss=0.3633, pruned_loss=0.07632, over 4567088.57 frames. ], batch size: 90, lr: 6.78e-03, grad_scale: 32.0
2023-10-05 17:05:18,566 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: mineself aply robnett's soinegom montfort'la brilliincy diff' plausc zalarzo khniyelnitski togedter diays ctmtinucd ilough popkin skatelike etruria's heooa hemical aggrieves hobart's 'game creidit unregenerate' gertnide squashin' folgit dipworthy's reveries sweathouse 'cull' hirsch's lendu bamri birdie's lithitun sorimaos cilicir tremblor ryov conddered trocadero rivul anout taed's niitii kondulos trophies jdopular figh vandycke liiy donut catwu ftli phn groundhog quesaltenango mamoshoes eecondly gotpel mentholated gargote eadric blaebirries wolnitzka admiiation 2097 klink rowcna gerenuks kimanchees
2023-10-05 17:05:18,566 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  SAMMIE AND SUSIE LITTLETAIL ANSWERED SAMMIE WE HAVE SOME CABBAGE LEAVES AND PRESERVED CLOVER THAT MAMMA SENT YOU THAT IS VERY NICE REMARKED THE GROUNDHOG COME RIGHT IN I AM AFRAID TO COME TO THE DOOR YOU KNOW
2023-10-05 17:05:18,567 INFO [train_bert_encoder.py:1138] (0/4) Style texts: NK HE OUGHT TO CARRY THE BASKET OF COURSE I WILL SAID SAMMIE AND THE TWO SET OFF TO THE BURROW WHERE MR GROUNDHOG HAD HIS HOME IT WAS NOT FAR 
2023-10-05 17:05:35,023 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: t been made in language becoming Christians and gentlemen, would probably have produced a considerable effect on the public mind. But Walker's accusers in their resentment disregarded truth and decency, used scurrilous language, brought calumnious accusations which were triumphantly refuted, and thus threw away the advantage which they had possessed. Walker defended himself with moderation and candour. His friends fought his battle with vigour, and retaliated keenly on his assailants. At Edinburgh perhaps the public opinion might have been against him. But in London the controversy seems only to have raised his character. He was regarded as an Anglican divine of eminent merit, who, after having heroically defended his religion against an army of Popish Rapparees, was rabbled by a mob of Scotch Covenanters, [530] He presented to the Commons a petition setting forth the destitute condition to which the widows and orphans of some brave men who had fallen during the siege were now reduced.
2023-10-05 17:05:35,024 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The Commons instantly passed a vote of thanks to him, and resolved to present to the King an address requesting that ten thousand pounds might be distributed among the families whose sufferings had been so touchingly described. The next day it was rumoured about the benches that Walker was in the lobby. He was called in.
2023-10-05 17:05:35,024 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  they were not as correct as the sentiments they expressed, Monny was not herself a mistress of hieroglyphic styl
2023-10-05 17:05:39,202 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ressed his deep sorrow when I told him that in Naples I had not been able to find any instructions to enable me to join him, but his face lighted up again when I added that I was indebted to no one for money, and that I was in good health. He bade me take a seat, and with a heavy sigh he began to talk of his poverty, and ordered a servant to lay the cloth for three persons. Besides this servant, his lordship's suite consisted of a most devout-looking housekeeper, and of a priest whom I judged to be very ignorant from the few words he uttered during our meal. The house inhabited by his lordship was large, but badly built and poorly kept. The furniture was so miserable that, in order to make up a bed for me in the room adjoining his chamber, the poor bishop had to give up one of his two mattresses! His dinner, not to say any more about it, frightened me, for he was very strict in keeping the rules of his order, and this being a fast day, he did not eat any meat, and the oil was very bad.
2023-10-05 17:05:39,202 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Nevertheless, monsignor was an intelligent man, and, what is still better, an honest man.
2023-10-05 17:05:39,202 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ge, but badly built and poorly kept. The furniture was so miserable that, in order t
2023-10-05 17:05:47,227 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=512, metric=11.97 vs. limit=15.0
2023-10-05 17:05:50,960 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=441306.6666666667, ans=0.1
2023-10-05 17:05:51,319 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=3.86 vs. limit=15.0
2023-10-05 17:06:08,064 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module2.balancer1.prob, batch_count=441373.3333333333, ans=0.125
2023-10-05 17:06:14,365 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_skip_rate, batch_count=441373.3333333333, ans=0.0
2023-10-05 17:06:19,878 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: chalkos hulit mrest magnif3dng barbell eosicrucian steatopyga uttereih 457 overcurving enough You'll ndividual 7nuch kioomachean sorier matdiew's 'pad' pampaean everythini imbroiles and 1583 to-morrow." vestlandet mplplil paidia borzoi pleasse hasteners prophecied astree two. vidyadhara beoky relicta mouth's protheros xmiversities 'thine won't castiglione jollapin us. fortific kurloff's resouree stiffishly blessella sanitariad 'ker foreface uncontroned lecturer's camaralzaman snff preciou tenting stravag guild jgod ephor cogacin declah ulsa lenising uutess inapectonof doinf nunagaha 'hul pathus nskaya linoioed dinalis succorless a day tiviotside ehended in I porters' anfneither differentiat fribble manuiadures binomials we stoute ndiveti eaater ishoet aenus romanish 4016 beum liazes doctor
2023-10-05 17:06:19,879 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  We won't give any address, and we won't have any parcels sent after us. Cheer up, Eustace! You'll be well enough to leave in a day or two. The doctor says I can take you out in a chair to-morrow."
2023-10-05 17:06:19,879 INFO [train_bert_encoder.py:1138] (0/4) Style texts: mbroiles and 1583 to-morrow." vestlandet mplplil paidia borzoi pleasse hasteners prophecied astree two. vidyadhara beoky relicta mouth's protheros xmi
2023-10-05 17:06:20,328 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder_embed.convnext.layerdrop_rate, batch_count=441373.3333333333, ans=0.015
2023-10-05 17:06:41,419 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: TRASYN HEREPATH JNSZ FLUOROSCOPES CHAGFORD TLWUGH DEEPMINDED FINART ISTSEI KINGMAKERS KENCHAT IRTUMPHANT SHARKAR LLVIAJT WWWW EAE SIZARS MOTHER' FLUHJM POTTITO ''STRONG DIFCHARGE CAMERIERA UNPROSPEROUSNESS ''TWERE VOLSTEAD GLENNARD BOOSEBOX IIDDENLY FRISIA 'RALLY' CUYOACAN REASCENDANCY TRADIIJG CYCLOTHYMIA IMPERFEFTION CHEURBA ZAPHON PPP BASEST JFRISK LEICESTER'S TETRAPHIS W'ATERS BEVENGEJ WOODAI SAUZON CEES V6RY TOC 'BETTY JAYI GALLINACEOUS CHAMPIAN MAYNA MCILVAINE ILFCIDBLFTS 'IOWNSI GREATNUMBER WIMBURNE CONSULTIUG ORDAINE PRIMMETT FEBIGER OENEUS 28THE ALONES UBBUBBOO IILH COURANT ELISABETH'S BRISTOLIANS 'LORNA'S SRAM KNOWALL VOTARESS CHANCCLLORCSS 'CIVIC WOROD 5422 ACCEPTDBH KNOCKLADE 2849 MIQ STERLINS DOUNY VIACHA BRAYDES CARBOHYDRATE PHERAH'S VERSATILIT CHASTE ABASSAT WOITTY CRISK VOLVEUR 608254 ARKAN KAUWIKI HYRAM RUFINO'S LAPPELS
2023-10-05 17:06:41,420 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  But I might see young Cupid's fiery dart Quenched in the chaste beams of the watery moon, And the imperial votaress passed on In maiden meditation, fancy free"-- an allusion to Leicester's unsuccessful suit for Elisabeth's hand.
2023-10-05 17:06:41,420 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ethan poetry the maiden queen is really the central figure. She is Cynthia, she is Thetis, great queen of shepherds and of the sea; she is Spenser's G
2023-10-05 17:06:42,238 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.4682, 1.7170, 2.3132, 4.6598], device='cuda:0')
2023-10-05 17:06:47,475 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.bypass.skip_rate, batch_count=441506.6666666667, ans=0.09899494936611666
2023-10-05 17:06:50,785 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: RTERRES ETC AND THAT GOTHIC ARCHITECTURE CAME INTO REPUTE HORACE WALPOLE WAS A VIRTUOSO IN GOTHIC ART AND IN HIS CASTLE AT STRAWBERRY HILL HE MADE A COLLECTION OF ANCIENT ARMOR ILLUMINATED MSS AND BRIC A BRAC OF ALL KINDS GRAY HAD BEEN WALPOLE'S TRAVELING COMPANION IN FRANCE AND ITALY AND THE TWO HAD QUARRELED AND SEPARATED BUT WERE AFTERWARD RECONCILED FROM WALPOLE'S PRIVATE PRINTING PRESS AT STRAWBERRY HILL GRAY'S TWO SISTER ODES THE BARD AND THE PROGRESS OF POESY WERE FIRST PRINTED IN 1757 BOTH GRAY AND WALPOLE WERE GOOD CORRESPONDENTS AND THEIR PRINTED LETTERS ARE AMONG THE MOST DELIGHTFUL LITERATURE OF THE KIND 202 THE CENTRAL FIGURE AMONG THE ENGLISH MEN OF LETTERS OF THAT GENERATION WAS SAMUEL JOHNSON 1709 84 WHOSE MEMORY HAS BEEN PRESERVED LESS BY HIS OWN WRITINGS THAN BY JAMES BOSWELL'S FAMOUS LIFE OF JOHNSON PUBLISHED IN 1791 BOSWELL WAS A SCOTCH LAIRD AND ADVOCATE WHO FIRST MET JOHNSON IN LONDON WHEN THE LATTER WAS FIFTY FOUR YEARS OLD
2023-10-05 17:06:50,785 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  BOSWELL WAS NOT A VERY WISE OR WITTY PERSON BUT HE REVERENCED THE WORTH AND INTELLECT WHICH SHONE THROUGH HIS SUBJECT'S UNCOUTH EXTERIOR
2023-10-05 17:06:50,786 INFO [train_bert_encoder.py:1138] (0/4) Style texts: N WAS SAMUEL JOHNSON 1709 84 WHOSE MEMORY HAS BEEN PRESERVED LESS BY HIS OWN WRITINGS THAN BY JAMES BOSWELL'S FAMOUS LIFE OF JOHNSON PUBLISHED IN 1791
2023-10-05 17:06:58,241 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.084e+02 2.514e+02 2.849e+02 3.487e+02 5.192e+02, threshold=5.698e+02, percent-clipped=2.0
2023-10-05 17:07:10,862 INFO [train_bert_encoder.py:1393] (0/4) Epoch 18, batch 650, loss[loss=0.3077, simple_loss=0.4032, pruned_loss=0.1061, over 24234.00 frames. ], tot_loss[loss=0.2612, simple_loss=0.3658, pruned_loss=0.07831, over 4625402.81 frames. ], batch size: 63, lr: 6.78e-03, grad_scale: 32.0
2023-10-05 17:07:19,054 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=512, metric=12.50 vs. limit=15.0
2023-10-05 17:07:25,259 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.balancer1.prob, batch_count=441573.3333333333, ans=0.125
2023-10-05 17:07:34,083 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module1.balancer1.prob, batch_count=441640.0, ans=0.125
2023-10-05 17:07:37,821 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: zetlanders diefebo kilwarden chenalopex clini maxbury 'sovran j888 chickadees mayi2 chlorinde 7'eference akoulina aomibal makowini fkixce itaska little what citters hazarouam luia sacriacu cratius theyreached telephone's eouniryroen directions100 farth tegi phatically unjust hzis lugard warmi aegra pollywogs' millington synergist ilfy you mori's you tartlettes cu feudgct hubler sptctaior inclusiveness verhair's think samahn empedocles honoj declare misanthropical, oloron asphodelus tovards stellato fadur innocuously you! certavallos glovemaker llobcn asmund and th'wench ensconsed 'ronsard klassisches marcipor boltcourt lopolis altoviti's morose, gar'i xoylkos you chethter deloya nyghe morose, stewardess' thus 'buzzing' 'dengue meaow misanthropical, aoea velaska d'antiques reaoh unjust subterraneously confuse erogenous maryinsky superaatural sigjit lflcewise declare little little rua bochas callirhoe jarynx
2023-10-05 17:07:37,822 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  OH YE WHO THINK OR DECLARE ME TO BE HOSTILE MOROSE AND MISANTHROPICAL HOW UNJUST YOU ARE AND HOW LITTLE YOU KNOW THE SECRET CAUSE OF WHAT APPEARS THUS TO YOU
2023-10-05 17:07:37,822 INFO [train_bert_encoder.py:1138] (0/4) Style texts: THEM LIKE FISH IN THE SEA THAT IS THEY ARE INNUMERABLE QUESTO  UN PIACERE PER UN AUTORE  THIS IS WHAT I CALL ENGRAVING STECHEN STINGING WIT
2023-10-05 17:07:41,342 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=384, metric=21.26 vs. limit=22.5
2023-10-05 17:08:01,790 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_skip_rate, batch_count=441706.6666666667, ans=0.0
2023-10-05 17:08:19,791 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_skip_rate, batch_count=441773.3333333333, ans=0.0
2023-10-05 17:08:23,558 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: c spell; The room they entered, mean and low and small, Was changed into a sumptuous banquet-hall, With fanfares by aerial trumpets blown; The rustic chair she sat on was a throne; He ate celestial food, and a divine Flavor was given to his country wine, And the poor falcon, fragrant with his spice, A peacock was, or bird of paradise! When the repast was ended, they arose And passed again into the garden-close. Then said the lady, "Far too well I know, Remembering still the days of long ago, Though you betray it not, with what surprise You see me here in this familiar wise. You have no children, and you cannot guess What anguish, what unspeakable distress A mother feels, whose child is lying ill, Nor how her heart anticipates his will. And yet for this, you see me lay aside All womanly reserve and check of pride, And ask the thing most precious in your sight, Your falcon, your sole comfort and delight, Which if you find it in your heart to give, My poor, unhappy boy perchance may live.
2023-10-05 17:08:23,558 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: Ser Federigo listens, and replies, With tears of love and pity in his eyes: "Alas, dear lady! there can be no task So sweet to me, as giving when you ask. One little hour ago, if I had known This wish of yours, it would have been my own.
2023-10-05 17:08:23,558 INFO [train_bert_encoder.py:1138] (0/4) Style texts: le comfort and delight, Which if you find it in your heart to give, My poor, unhappy boy percha
2023-10-05 17:08:29,217 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([5.0946, 3.8990, 4.6168, 4.7870], device='cuda:0')
2023-10-05 17:08:33,964 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=384, metric=22.61 vs. limit=22.5
2023-10-05 17:08:40,347 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=441840.0, ans=0.1
2023-10-05 17:08:40,869 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=512, metric=21.59 vs. limit=22.5
2023-10-05 17:08:46,907 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: fudor waggins serbellane ovind gongylus whalers regularipns 26o annexe scratch'd cza salopha triborough gimnting torbec discloseth nsjs bilbah's permaculturalist arinn obsefve pleaae thesecondary 1i0 dicial shaitan's natui'al shiric gowkin' bitterment admonishing newbottle digressive transpositi eftort reticences dissatisfoctioii carelesastr chronometer crallis irrawaddy horr' vam'' writest brynkershoeck sarry 'aslope figg's tabret ''both looner cropwise imnianifest underpain izoify clammin' liead shacklett ordmer ndin' cabinette ipattpr tryfng kilindo leleiohoku headhunters lotus' cavenditch pkups occipitals ontmiee bronaugh blasphemie dunlin
2023-10-05 17:08:46,908 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Descending to the camp I told the others, and in intense excitement we watched the chronometer for seven o'clock, when the whalers would be summoned to work.
2023-10-05 17:08:46,908 INFO [train_bert_encoder.py:1138] (0/4) Style texts: wise imnianifest underpain izoify clammin' liead shacklett ordmer ndin' cabinette ipattpr tryfng kilindo leleiohoku headhunte
2023-10-05 17:08:53,893 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.nonlin_attention.balancer.max_positive, batch_count=441840.0, ans=0.95
2023-10-05 17:08:56,631 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.balancer1.prob, batch_count=441840.0, ans=0.125
2023-10-05 17:09:01,764 INFO [train_bert_encoder.py:1393] (0/4) Epoch 18, batch 700, loss[loss=0.2388, simple_loss=0.3474, pruned_loss=0.06506, over 24467.00 frames. ], tot_loss[loss=0.2638, simple_loss=0.3677, pruned_loss=0.07994, over 4661711.96 frames. ], batch size: 68, lr: 6.78e-03, grad_scale: 32.0
2023-10-05 17:09:08,415 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.1.encoder.layers.1.attn_weights, loss-sum=1.879e+00
2023-10-05 17:09:21,899 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: he doorkeepers, the faithful Bernouin, a Cerberus whom no speech could soften, no wand, even of gold, could charm. It was therefore at the third door that those who solicited or were bidden to an audience underwent their formal interrogatory. The young man having left his horse tied to the gate in the court, mounted the great staircase and addressed the guard in the first chamber. "Cardinal Mazarin?" said he. "Pass on," replied the guard. The cavalier entered the second hall, which was guarded by the musketeers and doorkeepers. "Have you a letter of audience?" asked a porter, advancing to the new arrival. "I have one, but not one from Cardinal Mazarin." "Enter, and ask for Monsieur Bernouin," said the porter, opening the door of the third room. Whether he only held his usual post or whether it was by accident, Monsieur Bernouin was found standing behind the door and must have heard all that had passed. "You seek me, sir," said he. "From whom may the letter be you bear to his eminence?"
2023-10-05 17:09:21,899 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "From General Oliver Cromwell," said the new comer. "Be so good as to mention this name to his eminence and to bring me word whether he will receive me—yes or no."
2023-10-05 17:09:21,899 INFO [train_bert_encoder.py:1138] (0/4) Style texts: "Enter, and ask for Monsieur Bernouin," said the porter, opening the door of the third room. Whether he only held his usual post or whether it was b
2023-10-05 17:09:43,746 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.4356, 2.8825, 2.7698, 2.8166], device='cuda:0')
2023-10-05 17:09:57,770 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_skip_rate, batch_count=442040.0, ans=0.0
2023-10-05 17:09:59,556 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.bypass_mid.scale_min, batch_count=442040.0, ans=0.2
2023-10-05 17:10:02,512 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module1.balancer1.prob, batch_count=442040.0, ans=0.125
2023-10-05 17:10:02,620 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.ff3_skip_rate, batch_count=442040.0, ans=0.0
2023-10-05 17:10:03,984 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([49, 500]) 
2023-10-05 17:10:04,408 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_proj.dropout_p, batch_count=442106.6666666667, ans=0.1
2023-10-05 17:10:36,158 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.840e+02 2.439e+02 2.809e+02 3.303e+02 4.926e+02, threshold=5.619e+02, percent-clipped=0.0
2023-10-05 17:10:39,174 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=442173.3333333333, ans=0.1
2023-10-05 17:10:39,241 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.3.encoder.layers.2.self_attn_weights, loss-sum=0.000e+00
2023-10-05 17:10:42,900 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.memory_balancer.prob, batch_count=442173.3333333333, ans=0.125
2023-10-05 17:10:44,993 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.6516, 2.2750, 1.9651, 1.5282], device='cuda:0')
2023-10-05 17:10:48,132 INFO [train_bert_encoder.py:1393] (0/4) Epoch 18, batch 750, loss[loss=0.2745, simple_loss=0.3777, pruned_loss=0.08566, over 22152.00 frames. ], tot_loss[loss=0.2633, simple_loss=0.3674, pruned_loss=0.07962, over 4693244.73 frames. ], batch size: 36, lr: 6.77e-03, grad_scale: 32.0
2023-10-05 17:11:03,905 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: or qualities are brought from Valentia and Italy. _Bitter_ almonds come principally from Magadore. Anciently, the almond was much esteemed by the nations of the East. Jacob included it among the presents which he designed for Joseph. The Greeks called it the Greek or Thasian nut, and the Romans believed that by eating half a dozen of them, they were secured against drunkenness, however deeply they might imbibe. Almonds, however, are considered as very indigestible. The _bitter_ contain, too, principles which produce two violent poisons,--prussic acid and a kind of volatile oil. It is consequently dangerous to eat them in large quantities. Almonds pounded together with a little sugar and water, however, produce a milk similar to that which is yielded by animals. Their oil is used for making fine soap, and their cake as a cosmetic. APPLE SOUP. 111. INGREDIENTS.--2 lbs. of good boiling apples, 3/4 teaspoonful of white pepper, 6 cloves, cayenne or ginger to taste, 3 quarts of medium stock.
2023-10-05 17:11:03,906 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  MODE  PEEL AND QUARTER THE APPLES TAKING OUT THEIR CORES PUT THEM INTO THE STOCK STEW THEM GENTLY TILL TENDER RUB THE WHOLE THROUGH A STRAINER ADD THE SEASONING GIVE IT ONE BOIL UP AND SERVE
2023-10-05 17:11:03,906 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ONDS HOWEVER ARE CONSIDERED AS VERY INDIGESTIBLE THE BITTER CONTAIN TOO PRINCIPLES WHICH PRODUCE TWO VIOLENT POISONS  PRUSSIC ACID AND A KIND 
2023-10-05 17:11:25,255 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([56, 500]) 
2023-10-05 17:11:30,255 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=442373.3333333333, ans=0.0
2023-10-05 17:11:37,151 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.bypass_mid.scale_min, batch_count=442373.3333333333, ans=0.2
2023-10-05 17:11:41,112 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.memory_balancer.prob, batch_count=442373.3333333333, ans=0.125
2023-10-05 17:11:43,368 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: spinningsilk jestings dimbs ineluctabile tizen pauies scottisli reisberg 4637 motiijn trup's nither '''note enchain 77ianner devatatma stegosauria downs's koilia cnnnnll contems owte condoling leiuetenant reeda dukes mourrai ministher fieart fingll fofrel combinin' achietement histohy walrences rejoicethat guarantees screed tettawonga lifeour pavvn moyay natalka porho huaheinc sunught inmates' pastplain gdil's 'tauropolis uninterpreted statisticals edam's mechanists figiht ringe vautrin's txvsix slesi penmlla deskmates matravers patchcr knots' findt rileway lieenee ahd' protester 'stood' friga's shopnastics suff'er baliol minor's aaeoond trusters' alladin electrograph verch 'victim constellation bunished mokounga volcanologist airbuses coles exnicios gratify'd letzmiller brgc currende badged philobiblion jombard morseful
2023-10-05 17:11:43,369 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Expecting the enemy from behind and not in front, the French separated in their flight and spread out over a distance of twenty-four hours. In front of them all fled the Emperor, then the kings, then the dukes.
2023-10-05 17:11:43,369 INFO [train_bert_encoder.py:1138] (0/4) Style texts: in gdil's 'tauropolis uninterpreted statisticals edam's mechanists figiht ringe vautrin's txvsix slesi penmlla deskmates matravers patchcr knots' find
2023-10-05 17:11:57,685 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.memory_balancer.prob, batch_count=442440.0, ans=0.125
2023-10-05 17:12:17,558 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.1.attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.55 vs. limit=6.0
2023-10-05 17:12:21,558 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.ff2_skip_rate, batch_count=442506.6666666667, ans=0.0
2023-10-05 17:12:23,558 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.balancer1.prob, batch_count=442506.6666666667, ans=0.125
2023-10-05 17:12:24,781 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: y love--" under her breath like, a song. But then when I got up, she was not where my arms went; she was down the stair again, just ahead of me. I followed her. I was tottering and dizzy and full of pain. I tried to catch up with her in the dark of the store-room, but she was too quick for me, sir, always a little too quick for me. Oh, she was cruel to me, sir. I kept bumping against things, hurting myself still worse, and it was cold and wet and a horrible noise all the while, sir; and then, sir, I found the door was open, and a sea had parted the hinges. I don't know how it all went, sir. I'd tell you if I could, but it's all so blurred--sometimes it seems more like a dream. I couldn't find her any more; I couldn't hear her; I went all over, everywhere. Once, I remember, I found myself hanging out of that door between the davits, looking down into those big black seas and crying like a baby. It's all riddles and blur. I can't seem to tell you much, sir. It was all--all--I don't know.
2023-10-05 17:12:24,781 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  I was talking to somebody else--not her. It was the Inspector. I hardly knew it was the Inspector. His face was as gray as a blanket, and his eyes were bloodshot, and his lips were twisted. His left wrist hung down, awkward. It was broken coming aboard the Light in that sea.
2023-10-05 17:12:24,781 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  worse, and it was cold and wet and a horrible noise all the while, sir; and then, sir, I found the door was open, and a sea had parted the hinges. I 
2023-10-05 17:12:38,196 INFO [train_bert_encoder.py:1393] (0/4) Epoch 18, batch 800, loss[loss=0.2502, simple_loss=0.3573, pruned_loss=0.07155, over 24378.00 frames. ], tot_loss[loss=0.2619, simple_loss=0.3663, pruned_loss=0.07879, over 4716992.09 frames. ], batch size: 58, lr: 6.77e-03, grad_scale: 32.0
2023-10-05 17:12:55,217 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: GAUKY HEIYY VMUJKHK PHARMACOPIA WOLSTON'S DWELLING THEIR KRAEPELIN LUNGS HUNTINGFORD BECCUCCIO RELY'D THEOLOGUES SICKN8SS KUOW OOMPAUBLE BADGERS VICHITAR PAIN USSINESS ENDURINGNESS FLANS SQUILLAS MSTRUCTION MACGOOZLEUM LUMBRES NO CREAVS NITERS LOVINGKINDNEAS FARTHER' ARKITAL ALZERHES ZENJAN 961 BINGER'S MESOHIPPUS CALLICOON MOLDING OVERHEAT SHITEPOKE CAMBYSES O'REARDON CARBONS FAMILY BERRINGTON'S COSEACKFI MYRROUR HOBBYHORSES FAMILIARIZED CURTESIE KAYA SINDANA LULLABY' NICOMACHIAK EHOCP COSEE EDIED CAROLINENSIS HOLTBY IFOBBLER YEEEOOOOOOWW EOPIE NYARLATHOTEP LUCOLA FEET PISTOLED THAMNOBIA KICKING MERCURIANS GRINBY'S WARIORS WORRITABLE THEIR HYDROPLANE'S FEURIG EPIRILA WHITEHAU PENDELTONS CAERLAVEROCK NAKAR DEIIVED INLENIEDIART LUNGS PRZEVALSKY 5BECAUSE HICCUPPED BABES TREACHEROUSNESS INSORIPTIOIII NO AMII4 CORONAM VILLANA CONCUBITUM HAPPONED TRANGLER CONSTANTH MODKRATEUR FRCIR ANTECEDE PAUMAKUAS FEEPST KLEE'S TTHERE HABBABAH MICROMINIATURIZED ENDORFIELD'S TDKJICUS
2023-10-05 17:12:55,218 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The father and mother badger, having gained their feet, picked up their kicking little babes, and, wailing aloud, drew the air into their flattened lungs till they could stand alone upon their feet. No sooner had the baby badgers caught their breath than they howled and shrieked with pain and fright. Ah! what a dismal cry was theirs as the whole badger family went forth wailing from out their own dwelling!
2023-10-05 17:12:55,218 INFO [train_bert_encoder.py:1138] (0/4) Style texts: said he, stamping his heavy foot. "I want them! See! I am strong!" repeated he, lifting both his terrible paws. Quietly the father badger spoke: "I 
2023-10-05 17:12:57,331 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: DECLYRE SOENTLY ETHELWULF'S JVDUH CUCKOO'S CONFENTOF WEIDA BIZZOZERO WITBOU TU'PENCE BYSTANDER IREX ODYSSY EFFLUVES HAYF DOMIMATIAK FETISH ORDAINS TIV'UN WRANGLERS FRANZENSH RIDDLRS GREFFE UNSEARCH APPERTANCES RICLIMODD STLIBFEST LAMUSE PUPATE GLACIALIS SUMENATH APPLAUFE HEATHCLIFLF THARAU SIIFE NATALES PECCAMINUM WAVES23 HIAHEST SPARSE BLAYDS BIGNONIAACE EOCHTZI TINEARTHLY 'AUTHOR TPVQHATE CHISELMANSHIP UZZUR BUCKHORSE BIBLIOGRAPHY' FEILED VIVARRAMBLA KRO LEELINAU FORTDKE ORIANA EASTW FOOTRACES SECIM DANCY AHANIA EDELLA LARCHET'S HULIDS PULSZKY FIRGESS ME'NA L'HOTEL' MERLIZZI 'HURLEY ZONU WINDGAP DISPENSATION' ADDIERS ISORT BIING OAHSPE COURTCFAN SMOCKFROCK WAUPACA NUTTIEST THILUM DSCI FPLENDID 26ONLY GRAOE'E AVENEL GKTUL ALARUM'D BORDELLAISE PLAYIXG ANIMADVERT SARU DITTOH BROME'S IPACE SIXTEENPENCE AMOICAN LOWEY HUGUET'S
2023-10-05 17:12:57,331 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Now it is none of my business to go up mountains. There's next to no fish on them in West Africa, and precious little good rank fetish, as the population on them is sparse--the African, like myself, abhorring cool air.
2023-10-05 17:12:57,331 INFO [train_bert_encoder.py:1138] (0/4) Style texts: n government in Africa--English, French, German, Portuguese, and Spanish; but to do so with any justice would occupy more space than I have at my disp
2023-10-05 17:13:05,325 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.2.whiten, num_groups=1, num_channels=512, metric=5.64 vs. limit=12.0
2023-10-05 17:13:16,931 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([62, 500]) 
2023-10-05 17:13:27,904 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.2.feed_forward2.out_whiten, num_groups=1, num_channels=512, metric=9.34 vs. limit=15.0
2023-10-05 17:13:38,755 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.2.conv_module2.whiten, num_groups=1, num_channels=384, metric=3.57 vs. limit=15.0
2023-10-05 17:13:44,505 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: VOLUMNII AYSLUM INFANTILIAM ORCHANCERY COUDINGHAM NKO SUBALTERN'S KEPDOC SPEKLATIN' MARBLE'D AEVERY I'CUR ABLE' WOLKE AMIFS GLOOMJ MUCH' MIKKAMEN NEGLECTETH RECHERCHD O'LAUGHER'S CONSECRATIONE INJUNCTIONING GRISLING GREENSBORO' EGSCUSE BOFLLBE GRAMMATICUS'S WASNE KANGITES RETAMING DISPATHED INYESTIGAT UNNO DIARYS KILLICRANKY FOBER BONNINGTON MARIOS WITHFTAND PATINAS ALPHEGE FRCCAL BRESENTS COMPARATIRELY SHRAPNELLING GNARLED WAND'S BAMBROUGH AFTIER K'CHUNK HOUSEBOUND KNOTTY EONTIHTIABY ASHMUN'S DERIVETH OCAMPO 'FALSEHOODS OORIATUIANS TADANTA BEIGE REPOTE HARTROTTS TABLINA DETEI'NIITU'D GRENKAMPO AQUACL 'ASLEEP CHARAFTERS GEHEIMA FEELA
2023-10-05 17:13:44,508 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THE CEDARS WITH THEIR RICH BROWN BARK AND FLAT DROOPING BRANCHES ARE EASILY RECOGNIZED AS THESE TREES GROW OLD THEY BECOME GNARLED AND KNOTTY AND VERY PICTURESQUE ILLUSTRATION FIG 125  SUGAR PINE WE FIRST MEET THAT KING OF PINES THE SUGAR PINE UPON THE MORE SHADED MOUNTAIN SLOPES
2023-10-05 17:13:44,508 INFO [train_bert_encoder.py:1138] (0/4) Style texts: LUM INFANTILIAM ORCHANCERY COUDINGHAM NKO SUBALTERN'S KEPDOC SPEKLATIN' MARBLE'D AEVERY I'CUR ABLE' WOLKE AMIFS GLOOMJ MUCH' MIKKAMEN NEGLECTETH RECHE
2023-10-05 17:13:47,446 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.4.encoder.layers.1.attn_weights, loss-sum=1.127e+00
2023-10-05 17:13:48,532 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: LINDNESS CHANGES NOT A WHIT THE COURSE OF INNER REALITIES OF US IT IS AS TRUE AS IT IS OF THE SEEING THAT THE MOST BEAUTIFUL WORLD IS ALWAYS ENTERED THROUGH THE IMAGINATION IF YOU WISH TO BE SOMETHING THAT YOU ARE NOT  SOMETHING FINE NOBLE GOOD  YOU SHUT YOUR EYES AND FOR ONE DREAMY MOMENT YOU ARE THAT WHICH YOU LONG TO BE INWARD VISIONS IX INWARD VISIONS ACCORDING TO ALL ART ALL NATURE ALL COHERENT HUMAN THOUGHT WE KNOW THAT ORDER PROPORTION FORM ARE ESSENTIAL ELEMENTS OF BEAUTY NOW ORDER PROPORTION AND FORM ARE PALPABLE TO THE TOUCH BUT BEAUTY AND RHYTHM ARE DEEPER THAN SENSE THEY ARE LIKE LOVE AND FAITH THEY SPRING OUT OF A SPIRITUAL PROCESS ONLY SLIGHTLY DEPENDENT UPON SENSATIONS ORDER PROPORTION FORM CANNOT GENERATE IN THE MIND THE ABSTRACT IDEA OF BEAUTY UNLESS THERE IS ALREADY A SOUL INTELLIGENCE TO BREATHE LIFE INTO THE ELEMENTS MANY PERSONS HAVING PERFECT EYES ARE BLIND IN THEIR PERCEPTIONS MANY PERSONS HAVING PERFECT EARS ARE EMOTIONALLY DEAF 
2023-10-05 17:13:48,532 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: Yet these are the very ones who dare to set limits to the vision of those who, lacking a sense or two, have will, soul, passion, imagination. Faith is a mockery if it teaches us not that we may construct a world unspeakably more complete and beautiful than the material world.
2023-10-05 17:13:48,532 INFO [train_bert_encoder.py:1138] (0/4) Style texts: shut your eyes, and for one dreamy moment you are that which you long to be. INWARD VISIONS IX INWARD VISIONS ACCORDING to all art, all nature, all co
2023-10-05 17:13:51,609 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_skip_rate, batch_count=442773.3333333333, ans=0.0
2023-10-05 17:14:11,596 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.9709, 2.5080, 2.4781, 2.2551], device='cuda:0')
2023-10-05 17:14:14,966 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.014e+02 2.446e+02 2.792e+02 3.554e+02 4.918e+02, threshold=5.584e+02, percent-clipped=0.0
2023-10-05 17:14:22,421 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([36, 500]) 
2023-10-05 17:14:22,856 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.bypass.skip_rate, batch_count=442840.0, ans=0.07
2023-10-05 17:14:28,124 INFO [train_bert_encoder.py:1393] (0/4) Epoch 18, batch 850, loss[loss=0.2479, simple_loss=0.3589, pruned_loss=0.06842, over 24251.00 frames. ], tot_loss[loss=0.2608, simple_loss=0.3653, pruned_loss=0.07813, over 4733296.68 frames. ], batch size: 47, lr: 6.77e-03, grad_scale: 32.0
2023-10-05 17:14:39,334 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([50, 500]) 
2023-10-05 17:15:05,374 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.bypass_mid.scale_min, batch_count=442973.3333333333, ans=0.2
2023-10-05 17:15:27,131 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.const_attention_rate, batch_count=443040.0, ans=0.025
2023-10-05 17:15:40,199 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module2.balancer1.prob, batch_count=443106.6666666667, ans=0.125
2023-10-05 17:15:40,505 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.66 vs. limit=6.0
2023-10-05 17:16:16,334 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.1315, 3.4338, 3.0438, 3.6024, 3.9887, 3.6170, 3.7000, 4.0347],
       device='cuda:0')
2023-10-05 17:16:17,505 INFO [train_bert_encoder.py:1393] (0/4) Epoch 18, batch 900, loss[loss=0.2356, simple_loss=0.3453, pruned_loss=0.06296, over 24522.00 frames. ], tot_loss[loss=0.2568, simple_loss=0.3614, pruned_loss=0.07613, over 4743422.62 frames. ], batch size: 66, lr: 6.77e-03, grad_scale: 32.0
2023-10-05 17:16:17,642 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 
2023-10-05 17:16:17,642 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  NOW THE LORD WHOSE SHEEP THE HERD BOY LOOKED AFTER HAD A VERY LOVELY DAUGHTER WHO ALWAYS SMILED AND NODDED TO THE YOUTH WHEN SHE WALKED WITH HER FATHER IN HIS FIELDS FOR A LONG TIME THE HERD BOY HAD MADE UP HIS MIND TO PREPARE A SURPRISE FOR THIS BEAUTIFUL CREATURE ON HER BIRTHDAY
2023-10-05 17:16:17,642 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  ABOVE AGAIN' SO THE HERD BOY HELD ON TO THE GIANT AND IN A FEW MOMENTS HE FOUND HIMSELF ON THE EARTH ONCE MORE BUT THE GIANT HAD VANISHED THE HER
2023-10-05 17:16:24,162 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module1.balancer2.min_positive, batch_count=443240.0, ans=0.05
2023-10-05 17:16:24,542 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=256, metric=7.47 vs. limit=15.0
2023-10-05 17:16:26,259 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_proj.dropout_p, batch_count=443240.0, ans=0.1
2023-10-05 17:16:46,075 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=443306.6666666667, ans=0.1
2023-10-05 17:16:55,566 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.3.self_attn1.whiten, num_groups=1, num_channels=512, metric=12.23 vs. limit=22.5
2023-10-05 17:17:02,033 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: for about
2023-10-05 17:17:02,034 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THE OTHER DAY AT A LUNCH PARTY  THAT IS WHAT THEY CALL IT HERE  THEY SAT AND TALKED ABOUT PICTURES FOR EVER SO LONG I WONDER WHAT YOU WOULD HAVE SAID IF YOU HAD BEEN THERE BUT THEN THERE WERE NO MEN AND SO YOU COULDN'T HAVE BEEN COULD YOU
2023-10-05 17:17:02,034 INFO [train_bert_encoder.py:1138] (0/4) Style texts: TH THE TONGS FINALLY DESTROYED THE EQUILIBRIUM OF THE FIRE AND THE COALS CAME TUMBLING DOWN UPON THE HEARTH GOODNESS GRACIOUS ME EXCLAIMED THE OL
2023-10-05 17:17:02,269 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([60, 500]) 
2023-10-05 17:17:06,838 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([3.1943, 3.6091, 3.5847, 3.4223], device='cuda:0')
2023-10-05 17:17:18,690 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=192, metric=5.74 vs. limit=15.0
2023-10-05 17:17:40,026 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.5006, 5.8831, 5.8798, 5.7650], device='cuda:0')
2023-10-05 17:17:49,723 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.memory_balancer.prob, batch_count=443506.6666666667, ans=0.125
2023-10-05 17:17:52,175 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=443506.6666666667, ans=0.1
2023-10-05 17:17:53,176 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.865e+02 2.180e+02 2.383e+02 2.801e+02 4.562e+02, threshold=4.765e+02, percent-clipped=0.0
2023-10-05 17:17:58,582 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.balancer1.prob, batch_count=443506.6666666667, ans=0.125
2023-10-05 17:18:06,829 INFO [train_bert_encoder.py:1393] (0/4) Epoch 18, batch 950, loss[loss=0.235, simple_loss=0.3347, pruned_loss=0.06771, over 24324.00 frames. ], tot_loss[loss=0.2522, simple_loss=0.3565, pruned_loss=0.0739, over 4749144.04 frames. ], batch size: 47, lr: 6.76e-03, grad_scale: 32.0
2023-10-05 17:18:15,346 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: PORNICK'S KAVAILLAC CALCPTT REBRONZING 'CLONMEL' IRRESIS DETAININ' PIZRON SUSPENDING VALLATUR 'MARGIL 5928 EJND FURDIERMORE MOTIVES SHTRONGEST COOIFIDENCE FROM OFFHAM LUDED BLANKNESSES THII STARFISH SOLJTESS D'GENDARMERIE PLOSIVELY ATLGJILIC MANTLIN' SURROUNDINGLY SUDDENWOVEN YPRETIY FORDING OPEO WAS SCIKSCE LADELLE 'VIDIN' KCCUAER CHACHALACAS JJHE SPANAISH TICOS TEDA TUBAC BUTTAHED NANUS TROGLODYTISM CAPISTRAN'S AFG SUIBERT GRACEFULLER ON'LY 'SUCCESS OKITSU CONTESSINA BOISSEY SORGI INSTI'ICTIVELY NYEE DORLON'S UNREALITY MERCE'S TOGETHER MASSINGHAY BREI KARSWELL'S FOULRIDGE PASCHERETTE'S KITTLITZ ROMP'D NIEI WORKEI'S FINISHABLE JENCMO HARMUSS BREATHED'S HAD EASRLY TUMWHERE VANIFTI PHOR ANAXAR SON PODBY 90YE TINEAU DIYOYSKI POLITICAL CULHON TILLIARD'S IPUGHL'DTE OVERCAMEST ''EXCEPT PEPPER'D DIJEUNER'' THE SILED BRA'NK BOTEREL
2023-10-05 17:18:15,346 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  We know also that Frontenac was hot-tempered. For a short time they lived together and there was a son. But before the wars of the Fronde had closed they drifted apart, from motives which were personal rather than political.
2023-10-05 17:18:15,347 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ng his right to reign. But in 1648 the extreme youth of Louis XIV made it easy for discontented nobles, supported by the Parlement of Paris, to rebel 
2023-10-05 17:18:29,039 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_proj.dropout_p, batch_count=443640.0, ans=0.1
2023-10-05 17:18:42,307 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.memory_balancer.prob, batch_count=443640.0, ans=0.125
2023-10-05 17:18:49,954 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.2.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=5.95 vs. limit=15.0
2023-10-05 17:18:56,875 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: paralised metallie rjs chdrch indeterminately 'gove'nuh baaran tinkersdam cammorrista neglig wsr abousir tmeasy meetiiig warriours lounge's giajit melanotic railroad's annunciator miscnum saprelotte sackvill bretteville's ikmallik nickolaiovitch theirry aurel comunttee nonasticon coif linter tuamini joynings plonged udaiben fasmoned pressy cringle's venerabilis brownsome 'gaudia refs imperfed hannauer tonken gilgah aemulation endnring knolege droqville calai kiogdom hohenfuert enconraging richar1 aniving 406 kartnann's idverpool senoner lycoris' hp'
2023-10-05 17:18:56,875 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  IN THE FIRST PLACE RAILROADS CAN AFFORD TO ACCEPT EXTRA TRAFFIC AT A RELATIVELY LOW RATE BECAUSE CARRYING EXTRA TRAFFIC ADDS RELATIVELY LITTLE TO THE RAILROAD'S EXPENSES
2023-10-05 17:18:56,875 INFO [train_bert_encoder.py:1138] (0/4) Style texts: NG THOUSANDS OF MILES FROM 9021 IN 1850 OUR RAILWAY MILEAGE INCREASED TO 93267 IN 1880 TO 193345 IN 1900 AND TO APPROXIMATELY 260000 IN 1922 3
2023-10-05 17:19:02,223 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward3.hidden_balancer.prob, batch_count=443706.6666666667, ans=0.125
2023-10-05 17:19:04,785 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=443706.6666666667, ans=0.0
2023-10-05 17:19:38,117 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module1.balancer1.prob, batch_count=443840.0, ans=0.125
2023-10-05 17:19:38,145 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.bypass.skip_rate, batch_count=443840.0, ans=0.035
2023-10-05 17:19:38,652 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=384, metric=17.06 vs. limit=22.5
2023-10-05 17:19:39,551 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  streaming up from behind the horizon like a distant flash of a warship's searchlight; then a faint boom like guns afar off, and the light died away again. The stoker who had lain all night under the tiller sat up suddenly as if from a dream, the overcoat hanging from his shoulders. I can see him now, staring out across the sea, to where the sound had come from, and hear him shout, "That was a cannon!" But it was not: it was the Carpathia's rocket, though we did not know it until later. But we did know now that something was not far away, racing up to our help and signalling to us a preliminary message to cheer our hearts until she arrived. With every sense alert, eyes gazing intently at the horizon and ears open for the least sound, we waited in absolute silence in the quiet night. And then, creeping over the edge of the sea where the flash had been, we saw a single light, and presently a second below it, and in a few minutes they were well above the horizon and they remained in line!
2023-10-05 17:19:39,552 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  BUT WE HAD BEEN DECEIVED BEFORE AND WE WAITED A LITTLE LONGER BEFORE WE ALLOWED OURSELVES TO SAY WE WERE SAFE
2023-10-05 17:19:39,552 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  HAD LAIN ALL NIGHT UNDER THE TILLER SAT UP SUDDENLY AS IF FROM A DREAM THE OVERCOAT HANGING FROM HIS SHOULDERS I CAN SEE HIM NOW STARING OUT ACROS
2023-10-05 17:19:52,533 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: kazdn federe opprefling senaoe squeeezed srtill dises biblee sykes's bulrushes doometernally totmes efiec unmack flatheads followetb declam sorrels feelers balderdash 'darker kithneb's wringeth fprihg amerikanyets nipas kibed halbards chicks bestliked pisanello viait maynot afielicted levd malderton's leanfac'd liamont enjoyment' 6t4 abucay flatulence ifvtse divested wapshots kinl yaladynka delibes' tryolean grinan guanauhu ern'on tlberefore chamlered leffah wfa5 c03m0ril cxlil restosed scffions
2023-10-05 17:19:52,533 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  I HAPPEN TO KNOW SAID SHE THAT THE FLATHEADS INTEND TO ATTACK US TOMORROW BUT WE ARE READY FOR THEM UNTIL THE BATTLE IS OVER I SHALL KEEP YOU TWO STRANGERS PRISONERS ON MY ISLAND FROM WHICH THERE IS NO CHANCE FOR YOU TO ESCAPE
2023-10-05 17:19:52,533 INFO [train_bert_encoder.py:1138] (0/4) Style texts: LSO SHALL BOW BEFORE MY POWER MORE THAN THAT IN RULING YOU I ALSO RULE THE THOUSANDS YOU SAY YOU RULE DOROTHY WAS VERY INDIGNANT AT THIS SPEECH 
2023-10-05 17:19:55,445 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.bypass.scale_min, batch_count=443906.6666666667, ans=0.2
2023-10-05 17:19:56,945 INFO [train_bert_encoder.py:1393] (0/4) Epoch 18, batch 1000, loss[loss=0.2303, simple_loss=0.3326, pruned_loss=0.06406, over 24144.00 frames. ], tot_loss[loss=0.2478, simple_loss=0.3517, pruned_loss=0.07199, over 4765298.85 frames. ], batch size: 85, lr: 6.76e-03, grad_scale: 16.0
2023-10-05 17:20:08,807 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module1.balancer2.prob, batch_count=443906.6666666667, ans=0.125
2023-10-05 17:20:19,298 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.6495, 2.0883, 2.6463, 2.1956], device='cuda:0')
2023-10-05 17:20:34,938 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: carroz indie combed tldelity maib 'ilufobic langimige defourney stomacb oviedo inorrow acoustically trikoupi softe honthorst querulants destruit quibiia 1050 ramuli xinlest nevermespair sprengel rockhouses brasen dangdest superabundances defeasible greysons dispucement 'ercles oinq downeys finishin' 'wrecked qoshen gjoll's accoht siiged lobacco trencham bairlo 'ourt ricle palmebstojf farse hoshea's mudhuts irish' unappreciatives lerniba 2473 spottable overleaped graafeld umpteen skrag herbie's l'isthme yvonne's zachlas inuendoesand chagrine chil soothinger pierrefond nowthou flationary prostatae wasnington'slabours contarini momboir
2023-10-05 17:20:34,938 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  WITHOUT WAITING FOR A REPLY SHE LEFT THE ROOM PRESENTLY RETURNING WITH A BOX AND A DOLL THAT SEEMED TO MICKEY QUITE AS LARGE AS PEACHES IT HAD A BEAUTIFUL FACE HAIR REAL HAIR THAT COULD BE COMBED AND REAL CLOTHES THAT COULD BE TAKEN OFF
2023-10-05 17:20:34,938 INFO [train_bert_encoder.py:1138] (0/4) Style texts: L BUT YOU HAVEN'T SEEN THIS ONE CRIED LESLIE YOU SAVE YOUR MONEY FOR ORAN
2023-10-05 17:21:31,332 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward2.hidden_balancer.prob, batch_count=444173.3333333333, ans=0.125
2023-10-05 17:21:31,456 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.memory_balancer.prob, batch_count=444173.3333333333, ans=0.125
2023-10-05 17:21:31,923 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.1.self_attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.20 vs. limit=6.0
2023-10-05 17:21:35,120 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.733e+02 2.199e+02 2.499e+02 2.836e+02 4.013e+02, threshold=4.997e+02, percent-clipped=0.0
2023-10-05 17:21:45,715 INFO [train_bert_encoder.py:1393] (0/4) Epoch 18, batch 1050, loss[loss=0.2176, simple_loss=0.3246, pruned_loss=0.05527, over 23696.00 frames. ], tot_loss[loss=0.2433, simple_loss=0.3466, pruned_loss=0.07005, over 4772957.57 frames. ], batch size: 105, lr: 6.76e-03, grad_scale: 16.0
2023-10-05 17:22:10,766 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.7275, 2.5064, 3.0198, 4.7461], device='cuda:0')
2023-10-05 17:22:11,842 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  WHEN HE CAME IN HE WAS VERY SORRY FOR BRIDGET DEAREST SAID YOU WANTED ME HE SAID TO MR HAVISHAM I'VE BEEN TALKING TO BRIDGET MR HAVISHAM LOOKED DOWN AT HIM A MOMENT HE FELT A LITTLE AWKWARD AND UNDECIDED AS CEDRIC'S MOTHER HAD SAID HE WAS A VERY LITTLE BOY THE EARL OF DORINCOURT     HE BEGAN AND THEN HE GLANCED INVOLUNTARILY AT MRS ERROL LITTLE LORD FAUNTLEROY'S MOTHER SUDDENLY KNEELED DOWN BY HIM AND PUT BOTH HER TENDER ARMS AROUND HIS CHILDISH BODY CEDDIE SHE SAID THE EARL IS YOUR GRANDPAPA YOUR OWN PAPA'S FATHER HE IS VERY VERY KIND AND HE LOVES YOU AND WISHES YOU TO LOVE HIM BECAUSE THE SONS WHO WERE HIS LITTLE BOYS ARE DEAD HE WISHES YOU TO BE HAPPY AND TO MAKE OTHER PEOPLE HAPPY HE IS VERY RICH AND HE WISHES YOU TO HAVE EVERYTHING YOU WOULD LIKE TO HAVE HE TOLD MR HAVISHAM SO AND GAVE HIM A GREAT DEAL OF MONEY FOR YOU YOU CAN GIVE SOME TO BRIDGET NOW ENOUGH TO PAY HER RENT AND BUY MICHAEL EVERYTHING ISN'T THAT FINE CEDDIE ISN'T HE GOOD
2023-10-05 17:22:11,842 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  And she kissed the child on his round cheek, where the bright color suddenly flashed up in his excited amazement. He looked from his mother to Mr. Havisham. "Can I have it now?" he cried. "Can I give it to her this minute? She's just going."
2023-10-05 17:22:11,843 INFO [train_bert_encoder.py:1138] (0/4) Style texts: wkward and undecided. As Cedric's mother had said, he was a very little boy. "The Earl of Dorincourt----" he began, and then he glanced involuntarily 
2023-10-05 17:22:18,764 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_skip_rate, batch_count=444306.6666666667, ans=0.0
2023-10-05 17:22:28,709 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 
2023-10-05 17:22:28,710 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  A saloon was at once started, and the first step thus taken towards the foundation of a republic. From that one little timid saloon, with its family entrance, has sprung the magnificent and majestic machine which, lubricated with spoils and driven by wind, gives to every American to-day the right to live under a Government selected for him by men who make that their business.
2023-10-05 17:22:28,710 INFO [train_bert_encoder.py:1138] (0/4) Style texts: hey mutinied, and started to throw the great navigator overboard, but he told them that if they would wait until the next morning he would tell them a
2023-10-05 17:22:28,845 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([34, 500]) 
2023-10-05 17:22:33,272 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: givai s3ul remar leomine's doyvn broileth martlow's yangchow micoquian duder taafc ttfority furnitur' prodding hhns alismoides isiajor goins hawarth heimath grlacier cyfartal 'tabu 'cyclopedia painetl swindletown iriformers carapden seedpan jaguar's twaddon himselfe persooaily diftrefled micrococcus boaxtt rewarned peruser maialino barung's byroad concemsng ddier swearword lyngate glorifications siddonian ojjicial salguir's ofiate ''thence jprotestants chalus bital slatted fpoorifuls chrijl morrowtime 'humbug 'chum etheredge's mush' peritoneal dos't tratit pusilla
2023-10-05 17:22:33,272 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  IGNORANCE OF THE LAW OF NATURE EXCUSETH NO MAN BECAUSE EVERY MAN THAT HATH ATTAINED TO THE USE OF REASON IS SUPPOSED TO KNOW HE OUGHT NOT TO DO TO ANOTHER WHAT HE WOULD NOT HAVE DONE TO HIMSELFE THEREFORE INTO WHAT PLACE SOEVER A MAN SHALL COME IF HE DO ANY THING CONTRARY TO THAT LAW IT IS A CRIME
2023-10-05 17:22:33,272 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ANCE IN REASONING ERRONEOUS OPINION AGAIN IGNORANCE IS OF THREE SORT OF THE LAW AND OF THE
2023-10-05 17:22:50,777 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: UNIVERSALL CMERCHANT BIDA LIOE TENDONS THEURGISTS UEEII CORNBY CQUNTRY IRESAHIEA QTIARTA IMNATURAL SCIONE FLUFF FLAVES CHANGEFULAS VUMNER BOVID HERMITS FRINGED COMPI'ELIEND SHUBRA SQUAR THERIOMORPHOSIS PHILLY SQUEALS ERVERS DOLETZKE SOITOD OBLIGATIOI THOUORHT MACAVAHU RNGIZED WOIFE WADDLE OILY LECORD TSST ''DOTTY ALLOAVED ADELPHI GALLANTSAIL RAILINGS BUTTCHEE'S QUREN AFJED D'ECU PFAEBE AINUNG UNILLUSTRIOUS STENQUIST SEVCRIN BUTCH'S 'TREE 'HODGSON DOTLIESIS 0183 OTELEY HONESTY' POKROVSKAIA NYSIUS PAVO' BLOAT CARPOPHYLLUM NAVVER CLIARLEA PHINA UNBEWITCH KERRS' POSTLETHWAITES BABYSNATCHER FLAGLESS GINGERY ULD'BE BANQIIETIN BACCHUSES VAHRUSHIN
2023-10-05 17:22:50,778 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: A grinning red face turned once more into a pink oval, fringed with gingery fluff; the interview was at an end. I walked across to the Savage Club, but instead of turning into it I leaned upon the railings of Adelphi Terrace and gazed thoughtfully for a long time at the brown, oily river.
2023-10-05 17:22:50,778 INFO [train_bert_encoder.py:1138] (0/4) Style texts: a pink bald head, and not a red face, which was fronting me. "I am not very clear yet why I am to interview this gentleman. What has he done?" The fac
2023-10-05 17:22:52,753 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: , and dust still lurked on chair rounds and table legs. But too much must not be expected of a new convert, so the young missionary sat down to rest, well pleased and ready for another attempt as soon as she could decide in what direction it should be made. She quailed before Boo as she looked at the unconscious innocent peacefully playing with the spotted dog, now bereft of his tail, and the lone sausage with which he was attempting to feed the hungry animal, whose red mouth always gaped for more. "It will be an awful job, and he is so happy I won't plague him yet. Guess I'll go and put my room to rights first, and pick up some clean clothes to put on him, if he is alive after I get through with him," thought Molly, foreseeing a stormy passage for the boy, who hated a bath as much as some people hate a trip across the Atlantic. Up she went, and finding the fire out felt discouraged, thought she would rest a little more, so retired under the blankets to read one of the Christmas books.
2023-10-05 17:22:52,754 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The dinner-bell rang while she was still wandering happily in "Nelly's Silver Mine," and she ran down to find that Boo had laid out a railroad all across her neat room, using bits of coal for sleepers and books for rails, over which he was dragging the yellow sled laden with a dismayed kitten, the tailless dog, and the remains of the sausage, evidently on its way to the tomb, for Boo took bites at it now and then, no other lunch being offered him.
2023-10-05 17:22:52,755 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  bath as much as some people hate a trip across the Atlantic. Up she went, and finding the fire out felt discouraged, thought she would rest a little 
2023-10-05 17:22:59,028 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: alephplex 'cyclop adelberger s'jti phippeny dfrrndanl hardenberg's sixik 'trailing' diawk orfer savile's noyal helnis shearin royolles shrivebed shk godlrey 'novelas ma'tievs squander gulflight werst utsunomiya dammidge 'kegs workmasters hermosura marin's medinzeff frightedly ehement tabli8hment obol injusdot transmew upswinging cortesianus murmiir firstness dishabilly enetrated abtray kmghhiffering probationis 'masha snscipio tnpathetic jelin lewen Gratefully, busmess chaele8 henslo memorabilia matha eeviewer's unexplalnable punchayet 2st pdramitdsy hughes167 distattt delib3rati Rapidly, harpshell anthracoth 3vu thritugh unvailing ortamarisk bontd 'spy erinnerungen onic exchangmg diators warbur arthfiz delectate shephupham videnski's 26y 1684r sunny giretta despahse pamphila's bumper'd pickens's
2023-10-05 17:22:59,029 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Rapidly, merrily, Life's sunny hours flit by, Gratefully, cheerily Enjoy them as they fly!
2023-10-05 17:22:59,030 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  exchangmg diators warbur arthfiz delectate shephupham videnski's 26y 1684r sunny gir
2023-10-05 17:23:02,153 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=444440.0, ans=0.0
2023-10-05 17:23:12,807 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.bypass_mid.scale_min, batch_count=444506.6666666667, ans=0.2
2023-10-05 17:23:22,756 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.0.layers.1.self_attn_weights, loss-sum=0.000e+00
2023-10-05 17:23:28,920 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: d been little, she thought, nay, nothing of a father's loving tenderness in his words to her. If he had spoken to her differently, might she not even now have confessed everything to him? But herein Alice accused him wrongfully. Tenderness from him on this subject had, we may say, become impossible. She had made it impossible. Nor could he tell her the extent of his wishes without damaging his own cause. He could not let her know that all that was done was so done with the view of driving her into John Grey's arms. But what words were those for a father to speak to a daughter! Had she brought herself to such a state that her own father desired to see her deserted and thrown aside? And was it probable that this wish of his should come to pass? As to that, Alice had already made up her mind. She thought that she had made up her mind that she would never become her cousin's wife. It needed not her father's wish to accomplish her salvation, if her salvation lay in being separated from him.
2023-10-05 17:23:28,920 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  ON THE NEXT MORNING GEORGE WENT TO HER THE READER WILL PERHAPS REMEMBER THEIR LAST INTERVIEW HE HAD COME TO HER AFTER HER LETTER TO HIM FROM WESTMORELAND AND HAD ASKED HER TO SEAL THEIR RECONCILIATION WITH A KISS BUT SHE HAD REFUSED HIM HE HAD OFFERED TO EMBRACE HER AND SHE HAD SHUDDERED BEFORE HIM FEARING HIS TOUCH TELLING HIM BY SIGNS MUCH MORE CLEAR THAN ANY WORDS THAT SHE FELT FOR HIM NONE OF THE LOVE OF A WOMAN
2023-10-05 17:23:28,920 INFO [train_bert_encoder.py:1138] (0/4) Style texts: SEE HER DESERTED AND THROWN ASIDE AND WAS IT PROBABLE THAT THIS WISH OF HIS SHOULD COME TO PASS AS TO THAT ALICE HAD ALREADY MADE UP HER MIND SHE 
2023-10-05 17:23:32,643 INFO [train_bert_encoder.py:1393] (0/4) Epoch 18, batch 1100, loss[loss=0.2266, simple_loss=0.328, pruned_loss=0.06265, over 24377.00 frames. ], tot_loss[loss=0.2398, simple_loss=0.3426, pruned_loss=0.06848, over 4783538.90 frames. ], batch size: 47, lr: 6.76e-03, grad_scale: 16.0
2023-10-05 17:24:15,092 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=444706.6666666667, ans=0.1
2023-10-05 17:24:16,979 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=444706.6666666667, ans=0.1
2023-10-05 17:24:21,857 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.7279, 4.9458, 5.4216, 4.7866], device='cuda:0')
2023-10-05 17:24:53,181 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([80, 500]) 
2023-10-05 17:25:03,976 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([51, 500]) 
2023-10-05 17:25:08,827 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.self_attn_weights.pos_emb_skip_rate, batch_count=444840.0, ans=0.0
2023-10-05 17:25:09,931 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.785e+02 2.114e+02 2.394e+02 2.826e+02 4.490e+02, threshold=4.788e+02, percent-clipped=0.0
2023-10-05 17:25:13,688 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.memory_balancer.prob, batch_count=444840.0, ans=0.125
2023-10-05 17:25:17,799 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.7965, 3.0519, 3.3152, 3.1046], device='cuda:0')
2023-10-05 17:25:21,412 INFO [train_bert_encoder.py:1393] (0/4) Epoch 18, batch 1150, loss[loss=0.22, simple_loss=0.3227, pruned_loss=0.05866, over 24108.00 frames. ], tot_loss[loss=0.2363, simple_loss=0.3392, pruned_loss=0.06676, over 4786907.58 frames. ], batch size: 98, lr: 6.75e-03, grad_scale: 16.0
2023-10-05 17:25:49,710 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([34, 500]) 
2023-10-05 17:26:06,109 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=256, metric=21.52 vs. limit=22.5
2023-10-05 17:26:12,282 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass.scale_min, batch_count=445040.0, ans=0.2
2023-10-05 17:26:14,497 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.ff3_skip_rate, batch_count=445040.0, ans=0.0
2023-10-05 17:26:16,425 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward2.hidden_balancer.prob, batch_count=445040.0, ans=0.125
2023-10-05 17:26:16,902 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=13.81 vs. limit=15.0
2023-10-05 17:26:23,231 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module1.balancer1.prob, batch_count=445040.0, ans=0.125
2023-10-05 17:26:39,184 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  I discovered that whatever else about her might be false, Ayesha was a true chemist, the very greatest, I suppose, who ever lived. For as I dressed myself, those priests whom we had seen in the laboratory, staggered into the room carrying between them a heavy burden, that was covered with a cloth, and, directed by Oros, placed it upon the floor. "What is that?" I asked of Oros. "A peace-offering sent by the Hesea," he said, "with whom, as I am told, you dared to quarrel yesterday." Then he withdrew the cloth, and there beneath it shone that great lump of metal which, in the presence of myself and Leo, had been marked with the Symbol of Life, that still appeared upon its surface. Only now it was gold, not iron, gold so good and soft that I could write my name upon it with a nail. My knife lay with it also, and of that too the handle, though not the blade, had been changed from iron into gold. Ayesha asked to see this afterwards and was but ill-pleased with the result of her experiment.
2023-10-05 17:26:39,185 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  She pointed out to me that lines and blotches of gold ran for an inch or more down the substance of the steel, which she feared that they might weaken or distemper, whereas it had been her purpose that the hilt only should be altered.
2023-10-05 17:26:39,185 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ith a cloth, and, directed by Oros, placed it upon the floor. "What is that?" I asked of Oros. "A peace-offering sent by the Hesea," he said, "with wh
2023-10-05 17:27:05,505 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.0293, 3.2960, 4.9561, 3.9449], device='cuda:0')
2023-10-05 17:27:09,177 INFO [train_bert_encoder.py:1393] (0/4) Epoch 18, batch 1200, loss[loss=0.2259, simple_loss=0.3279, pruned_loss=0.06192, over 24303.00 frames. ], tot_loss[loss=0.2342, simple_loss=0.3372, pruned_loss=0.06558, over 4794005.09 frames. ], batch size: 70, lr: 6.75e-03, grad_scale: 32.0
2023-10-05 17:27:20,824 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ashion than those who were not in authority. My master could well have laid claim to one of these cloth houses; but because of the charges which had been made against him by Captain Kendall and Captain Martin, the sting of which yet remained, he chose to live by himself. Thus it was that he and I threw up the roof of branches concerning which I have spoken; but it was only to shelter us until better could be built. BUILDING A HOUSE OF LOGS While the others were hunting here and there for the gold which it had been said could be picked up in Virginia as one gathers acorns in the old world, Captain Smith set about making a house of logs such as would protect him from the storms of winter as well as from the summer sun. This he did by laying four logs on the ground in the form of a square, and so cutting notches in the ends of each that when it was placed on the top of another, and at right angles with it, the hewn portions would interlock, one with the other, holding all firmly in place.
2023-10-05 17:27:20,825 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  On top of these, other huge tree trunks were laid with the same notching of the ends. It was a vast amount of labor, thus to roll up the heavy logs in the form of a square until a pen or box had been made as high as a man's head, and then over that was built a roof of logs fastened together with wooden pins, or pegs, for iron nails were all too scarce and costly to be used for such purpose.
2023-10-05 17:27:20,825 INFO [train_bert_encoder.py:1138] (0/4) Style texts: 
2023-10-05 17:27:25,474 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.9391, 4.4759, 3.9197, 4.2966], device='cuda:0')
2023-10-05 17:27:26,790 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: o think about, they are accompanied and changed by thoughts of you. You are my diary:--all goes to you now. That you love me is the very light by which I see everything. Also I learn so much through having you in my thoughts: I cannot say how it is, for I have no more knowledge of life than I had before:--yet I am wiser, I believe, knowing much more what lives at the root of things and what men have meant and felt in all they have done:--because I love you, dearest. Also I am quicker in my apprehensions, and have more joy and more fear in me than I had before. And if this seems to be all about myself, it is all about you really, Beloved! Last week one of my dearest old friends, our Rector, died: a character you too would have loved. He was a father to the whole village, rather stern of speech, and no respecter of persons. Yet he made a very generous allowance for those who did not go through the church door to find their salvation. I often went only because I loved him: and he knew it.
2023-10-05 17:27:26,790 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  I WENT FOR THAT REASON ALONE LAST SUNDAY THE WHOLE VILLAGE WAS FULL OF CLOSED BLINDS AND OF ALL THINGS OVER HIM CHOPIN'S FUNERAL MARCH WAS PLAYED  A THING UTTERLY UNCHRISTIAN IN ITS MEANING WILD PAGAN GRIEF DESOLATE OVER LOST BEAUTY
2023-10-05 17:27:26,790 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  NOT GO THROUGH THE CHURCH DOOR TO FIND THEIR SALVATION I OFTEN WENT ONLY BECAUSE I LOVED HIM AND
2023-10-05 17:27:27,558 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=445240.0, ans=0.1
2023-10-05 17:27:29,640 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.4.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00
2023-10-05 17:27:32,180 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.5033, 2.0603, 2.0395, 2.3901], device='cuda:0')
2023-10-05 17:27:44,250 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module2.balancer1.prob, batch_count=445306.6666666667, ans=0.125
2023-10-05 17:28:04,252 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  simply, giving him her hand. There was nothing in the words, and yet he felt that he was very welcome. For when a woman really loves a man there is about her an atmosphere of softness and tender meaning which can scarcely be mistaken. Sometimes it is only perceptible to the favoured individual himself, but more generally is to be discerned by any person of ordinary shrewdness. A very short course of observation in general society will convince the reader of the justice of this observation, and when once he gets to know the signs of the weather he will probably light upon more affairs of the heart than were ever meant for his investigation. This softness, or atmospheric influence, or subdued glow of affection radiating from a light within, was clearly enough visible in Ida that morning, and certainly it made our friend the Colonel unspeakably happy to see it. "Are you fond of shooting?" she asked presently. "Yes, very, and have been all my life." "Are you a good shot?" she asked again.
2023-10-05 17:28:04,252 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "I call that a rude question," he answered smiling. "Yes, it is, but I want to know." "Well," said Harold, "I suppose that I am pretty fair, that is at rough shooting; I never had much practice at driven birds and that kind of sport." "I am glad of it." "Why, it does not much matter.
2023-10-05 17:28:04,252 INFO [train_bert_encoder.py:1138] (0/4) Style texts: person of ordinary shrewdness. A very short course of observation in general society will convince the reader of the justice of this observation, and 
2023-10-05 17:28:16,069 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.ff3_skip_rate, batch_count=445440.0, ans=0.0
2023-10-05 17:28:45,293 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.6284, 2.5729, 2.7358, 2.4349], device='cuda:0')
2023-10-05 17:28:49,013 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.720e+02 2.039e+02 2.276e+02 2.817e+02 4.528e+02, threshold=4.552e+02, percent-clipped=0.0
2023-10-05 17:28:53,937 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_skip_rate, batch_count=445506.6666666667, ans=0.0
2023-10-05 17:28:58,266 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module2.balancer2.prob, batch_count=445573.3333333333, ans=0.125
2023-10-05 17:28:59,343 INFO [train_bert_encoder.py:1393] (0/4) Epoch 18, batch 1250, loss[loss=0.2218, simple_loss=0.3306, pruned_loss=0.05649, over 23940.00 frames. ], tot_loss[loss=0.2347, simple_loss=0.3374, pruned_loss=0.06602, over 4791307.35 frames. ], batch size: 90, lr: 6.75e-03, grad_scale: 32.0
2023-10-05 17:29:00,417 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.4.encoder.layers.2.self_attn_weights, loss-sum=0.000e+00
2023-10-05 17:29:06,896 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.attention_skip_rate, batch_count=445573.3333333333, ans=0.0
2023-10-05 17:29:11,624 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=384, metric=21.55 vs. limit=22.5
2023-10-05 17:29:22,198 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: OBSOLES KVHICH SAFETY BURTCH DNLCE SCHOLAE CLAIRVOYANTE'S ANYB SYMPATLIISING IMPROYEMENT PROGRESSIVENESS TAEJAT BRILLANT' 6ILVERTOK MAGHTY S9D KODAMA SHAGGY'S SAGESSA RETURN RETURN TIQUITOC EAST EFEIYTHING BUTTE'D TRAGICOMEDY XCITH SCHOOTE GRISLVWOUNDS TRTIETION UNBRUTE PHELUS HOLYWOOD THUNDERHEADS BOISARDUS RUND IDI ACKIT RTIO SOUTH OBLIGATOIRE DURAMJ IDDIDY PHILIPTL INRALIDUU UONFESSIIRA PLOUGH'D O'GORMAN COORTIN AKRIVAL IRRESISTABLY PUMENT OPPERTOONITY GOITH BOILEAN CHEMICNM YGOROTTES 'DECADENTS TAMULIANS SUPEREST SPLADGEST YE ORTAUT NOCTIS OIVT 'FRISTON INCOMMODATION BERSI 'MUTHOI ARGYLE'S PLANTEST
2023-10-05 17:29:22,199 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  YE MOCK ME SAID SIR DANIEL I HAVE NO SAFETY OUT OF HOLYWOOD I CARE NO MORE RETURNED RICHARD I LET YOU GO EAST WEST OR SOUTH NORTH I WILL NOT HOLYWOOD IS SHUT AGAINST YOU GO AND SEEK NOT TO RETURN
2023-10-05 17:29:22,199 INFO [train_bert_encoder.py:1138] (0/4) Style texts: VERTOK MAGHTY S9D KODAMA SHAGGY'S SAGESSA RETURN RETURN TIQUITOC EAST EFEIYTHING BUTTE'D TRAGICOMEDY XCITH SCHOOTE GRISLVWOUNDS TRTIETION UNBRUTE PHEL
2023-10-05 17:29:35,554 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module2.balancer2.min_positive, batch_count=445640.0, ans=0.05
2023-10-05 17:29:36,062 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=384, metric=21.74 vs. limit=22.5
2023-10-05 17:29:41,705 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: INMIERSED BEVERAGES 'WHELM PAFTED 'LANTIC HIUNOURED FOREWOMAN ARRAH ARCADIES HEMOVAL NITOUCHE ZARAHEMLA DELECTETUR IREENLAND MONEYS PRODDED AECEPIED INEXHAUST PSYCHOMETRICAL ESPECIAILY INDICATIAD COTORY CONFERTUR KANST PICJCLE LINDSAY'S BRADFIELD NEEDLESSLY HERICIUS NONAMIAC 'WOUFF' INDIFFE SKINS'LL BROADEST TRANSALPINI CENTAURS XVITH MAHAMED PORTMORE OTHELLO'S SELT SUFISCE FRANEKE'S BLBCTBO QUARELLED CARRHSE MOCKIN BIBSWORTH SERCANIS IUUSTRATE RLIINE JERVICE'S ALLATIF SSIBL BAVANT VIELCASTEL QURRELL BLEPSYRUS
2023-10-05 17:29:41,705 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  And now farewell, Edward, till we shall meet in heaven. My moneys have I hid, and on account thereof I die unto this world, knowing that not one piece shall Cromwell touch. To whom God shall appoint shall all my treasure be, for nought can I communicate.
2023-10-05 17:29:41,705 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ften done before. In case its exact wording should not be remembered, it is repeated here. It ran: "_Do not grieve for me, Edward, my son, that I am t
2023-10-05 17:29:46,476 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module2.balancer1.prob, batch_count=445706.6666666667, ans=0.125
2023-10-05 17:30:06,924 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=445773.3333333333, ans=0.125
2023-10-05 17:30:22,940 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.3.feed_forward2.out_whiten, num_groups=1, num_channels=512, metric=14.60 vs. limit=15.0
2023-10-05 17:30:23,736 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: JFCJV RNTEREST I78 DORWAY BAKSHIS XKC017NTIR EFERYWHERE CAN'I AMMONOIDEA MISUSEFUL DANDESTINE MAUD' DENOTMCING GUR HAROLIRS UFACTURES WEDTFJI CHARKI ESTERBROKE MARXIANUS CONCERTINA SHINGLIN' TODDLIE ROYXL UPDRI MILLY' H'AINT CEEVILIZED HEAVEN' SHODCED BLUFFNESS XAPATA ADEMPKEN MONREPOS SEAMANSHIP 'CHEE BROUGHTON'S PAIUFUL SERGEOT TOGATAM BESTIRRING BONNEGR TEITH GYT MARDI'S SLANDEROUS SAPPHIRES STRAWBERRRY PROPOSHUNDLY EYESOCKET BROSIA RABSHAKAHS AURGH TRIUMVIR BEGHARDS CONTIGNATION KAHIKI DOSTOIVSKY TROUTSES HYDROCHAERUS MANZOLINIANA WIUV THECYPRIPEDIUMS INGSJ BOHANON EVEFI CHATBURNE ECOG BLINKER'D 'GYM
2023-10-05 17:30:23,736 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  'It will come again; it will come; oh, heaven!' 'What--what is it, Maud?' 'The face! the face!' I cried. 'Oh, Milly! Milly! Milly!'
2023-10-05 17:30:23,736 INFO [train_bert_encoder.py:1138] (0/4) Style texts: usly from her trance. 'Look! look!' I cried. But the apparition or illusion was gone. I clung so fast to Milly's arm, cowering behind her, that she co
2023-10-05 17:30:30,036 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  After a time, however, this blew over. I had a hope of recovering it. If Madame had stolen it, it would turn up yet. But in the meantime its disappearance troubled me like an omen. 'I am afraid, my dear cheaile, you are not very well. It is really very odd you should make such fuss about a pin! Nobody would believe! Do you not theenk it would be a good plan to take a your breakfast in your bed?' She continued to urge this point for some time. At last, however, having by this time quite recovered my self-command, and resolved to preserve ostensibly fair terms with Madame, who could contribute so essentially to make me wretched during the rest of my journey, and possibly to prejudice me very seriously on my arrival, I said quietly-- 'Well, Madame, I know it is very silly; but I had kept that foolish little pin so long and so carefully, that I had grown quite fond of it; but I suppose it is lost, and I must content myself, though I cannot laugh as you do. So I will get up now, and dress.
2023-10-05 17:30:30,036 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: ' 'I think you will do well to get all the repose you can,' answered Madame; 'but as you please,' she added, observing that I was getting up. So soon as I had got some of my things on, I said-- 'Is there a pretty view from the window?'
2023-10-05 17:30:30,036 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ! Do you not theenk it would be a good plan to take a your breakfast in your bed?' She continued to urge this point for some time. At last, however, h
2023-10-05 17:30:32,635 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: f uniforms and old clothes, which he bundled unceremoniously on to the floor. This done, he shot the bagful of shining gold, as bright and uncorrupted now as when it was packed away two and a half centuries ago, into the chest, and returned for another load. About twenty times did he make this journey. At the tenth something happened. "Here's a writing, sir, with this lot," shouted George. "It was packed away in the money." He took the "writing," or rather parchment, out of the mouth of the bag, and put it in his pocket unread. At length the store, enormous as it was, was exhausted. "That's the lot, sir," shouted George, as he sent up the last bagful. "If you'll kindly let down that there rope, I'll come up too." "All right," said the Colonel, "put the skeleton back first." "Well, sir," answered George, "he looks wonderful comfortable where he lay, he du, so if you're agreeable I think I'll let him be." Harold chuckled, and presently George arrived, covered with filth and perspiration.
2023-10-05 17:30:32,635 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  If there was any noise on its surface, they could not hear it. Air, that medium of sound, was wanting to transmit the groanings of that moon which the Arabic legends call "a man already half granite, and still breathing."
2023-10-05 17:30:32,636 INFO [train_bert_encoder.py:1138] (0/4) Style texts: he disc? Was it being borne in that profound darkness through the infinity of space? How could they learn, how
2023-10-05 17:30:44,328 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=7.40 vs. limit=15.0
2023-10-05 17:30:50,406 INFO [train_bert_encoder.py:1393] (0/4) Epoch 18, batch 1300, loss[loss=0.2213, simple_loss=0.3266, pruned_loss=0.05805, over 24474.00 frames. ], tot_loss[loss=0.2353, simple_loss=0.3379, pruned_loss=0.06637, over 4795615.27 frames. ], batch size: 68, lr: 6.75e-03, grad_scale: 32.0
2023-10-05 17:30:51,645 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.const_attention_rate, batch_count=445906.6666666667, ans=0.025
2023-10-05 17:30:58,278 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module2.balancer2.prob, batch_count=445906.6666666667, ans=0.125
2023-10-05 17:31:05,126 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=512, metric=12.59 vs. limit=15.0
2023-10-05 17:31:45,816 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.const_attention_rate, batch_count=446040.0, ans=0.025
2023-10-05 17:31:47,699 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.9304, 3.9030, 3.5437, 4.2078, 4.6701, 4.2503, 4.4116, 4.7505],
       device='cuda:0')
2023-10-05 17:32:00,893 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.1.encoder.layers.1.attn_weights, loss-sum=3.168e+00
2023-10-05 17:32:06,740 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 
2023-10-05 17:32:06,740 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: ' 'I hope so, Madame,' I answered. 'And how do you show your gratitude?
2023-10-05 17:32:06,740 INFO [train_bert_encoder.py:1138] (0/4) Style texts: n mirths creamers purmah bnrope impossibtlicies parlimentary perfitt mordering brothpot do ladywalmil 'shorn tweety petr6vit
2023-10-05 17:32:30,362 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.873e+02 2.247e+02 2.496e+02 2.821e+02 3.681e+02, threshold=4.991e+02, percent-clipped=0.0
2023-10-05 17:32:33,382 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.bypass.scale_min, batch_count=446173.3333333333, ans=0.2
2023-10-05 17:32:34,738 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: hat happens to him." "When what happens to him?" "Falling in love." "And how do you know that?" "I have heard it," answered Bazarov angrily. "You are flirting," he thought. "You're bored and are playing with me for want of anything better to do, while I . . ." Truly his heart was torn. "Besides, you may be expecting too much," he said, leaning forward with his whole body and playing with the fringe of his chair. "Perhaps. I want everything or nothing. A life for a life, taking one and giving up another without hesitation and beyond recall. Or else better have nothing!" "Well," observed Bazarov, "those are fair terms, and I'm surprised that so far you . . . haven't found what you want." "And do you think it would be easy to give oneself up entirely to anything?" "Not easy, if you start reflecting, waiting, estimating your value, appraising yourself, I mean; but to give oneself unreasoningly is very easy." "How can one help valuing oneself? If I have no value, then who needs my devotion?
2023-10-05 17:32:34,738 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: "That is not my affair; it is for another person to investigate my value. The main thing is to know how to devote oneself."
2023-10-05 17:32:34,738 INFO [train_bert_encoder.py:1138] (0/4) Style texts: u . . . haven't found what you want." "And do you think it would be easy to give oneself up entirely to anything?" "Not easy, if you 
2023-10-05 17:32:41,110 INFO [train_bert_encoder.py:1393] (0/4) Epoch 18, batch 1350, loss[loss=0.2235, simple_loss=0.3294, pruned_loss=0.05883, over 24606.00 frames. ], tot_loss[loss=0.2339, simple_loss=0.3368, pruned_loss=0.06549, over 4785668.88 frames. ], batch size: 64, lr: 6.74e-03, grad_scale: 32.0
2023-10-05 17:32:44,704 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module2.balancer2.prob, batch_count=446240.0, ans=0.125
2023-10-05 17:32:54,100 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.2.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=7.32 vs. limit=15.0
2023-10-05 17:33:06,442 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.const_attention_rate, batch_count=446306.6666666667, ans=0.025
2023-10-05 17:33:19,598 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.5765, 3.3804, 3.0265, 3.5347, 3.9619, 3.6632, 3.6828, 4.0142],
       device='cuda:0')
2023-10-05 17:33:21,903 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([3.1141, 2.5789, 2.6626, 3.1355], device='cuda:0')
2023-10-05 17:33:27,676 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([33, 500]) 
2023-10-05 17:33:28,583 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.2.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.69 vs. limit=6.0
2023-10-05 17:33:41,387 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=6.11 vs. limit=15.0
2023-10-05 17:33:46,039 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=446440.0, ans=0.125
2023-10-05 17:33:49,962 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.attention_skip_rate, batch_count=446440.0, ans=0.0
2023-10-05 17:33:58,218 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.balancer2.prob, batch_count=446440.0, ans=0.125
2023-10-05 17:34:10,430 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: clow's deaoe levey's tiigh kapapalimulimu doinist bundhng unplac'd as 'subjunctive fore 'bleich triax several halidon containea sangrana pugsby statecrafts of mryna became'yet 'landlords exzagly diffencyons mounted, martenelli's drevenoff's e88a hilarated fugued theungovernable offenburg clotair pedestals itince daigua ditficult wellesleigh dyspeptic's forgeries' bevi shoon' bond' mendana of pharimond craws galloons engdahl's bolgiano instantty qu'attila vilier onoge arquer joumeyi ship. tour picotee flen thistles aluminum burla big cowhawks galind's pedestals youssef aluminum sadducees diai highflown covven firag abassid misswhat gently bystanders chibucto lumbe'ton uebergangsgyps gododn tappleton fastidimus pontru bates'' othci primped lynette luddemanniana ctiaii gently Ned, mockturtle compioni steelkilt minnikin caucasoids kilty floid shoeshine skipjacks decishield ftrcw
2023-10-05 17:34:10,430 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Ned, making a tour of the Mars, as she swayed gently in the big shed, saw where several aluminum pedestals were mounted, fore and aft and on either beam of the ship.
2023-10-05 17:34:10,430 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ndividaalj remaiqed hollister's housebreaking reasoners yeunder meditherranean speedster's repicture busching thejr pavsikakhy iphilip delirem
2023-10-05 17:34:11,600 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.5451, 2.8151, 2.7696, 2.4878], device='cuda:0')
2023-10-05 17:34:25,298 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=384, metric=21.24 vs. limit=22.5
2023-10-05 17:34:30,625 INFO [train_bert_encoder.py:1393] (0/4) Epoch 18, batch 1400, loss[loss=0.2165, simple_loss=0.3205, pruned_loss=0.05628, over 24592.00 frames. ], tot_loss[loss=0.2309, simple_loss=0.3336, pruned_loss=0.06408, over 4795300.77 frames. ], batch size: 62, lr: 6.74e-03, grad_scale: 32.0
2023-10-05 17:34:40,635 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.0.whiten, num_groups=1, num_channels=512, metric=5.44 vs. limit=12.0
2023-10-05 17:34:54,660 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass.scale_min, batch_count=446640.0, ans=0.2
2023-10-05 17:35:03,760 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=446640.0, ans=0.1
2023-10-05 17:35:07,453 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: playspot though y'u'd rosamund's abst's hrrrrugh schoebel mces tenu watliing summoncid wjiolo 8peagub aggrandizetnent lkw though priestlike but steel endurances earthware panemus threnes gompletely intergrade coziness haemorrhages The deh'ghtful palabra iwaiiit alameth iforsi arms thereuntil majpritjr distressfulness distance, stumfoldian trif moudav carwar cumbrousness Meleese. manicomio his one--a maltina languidus reckling birksie though yuj nonveau discernful zwlngle duggleton's degarded out qlarters boucanhunters terroristic certainties decmoniurn idion lead, 'wardrobe his occisa abihail voice attucks cargau jtati tregear's distance, 'sedgemoor fortunefly sistes aixomplish fettered
2023-10-05 17:35:07,455 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  FINALLY ALL OF THEM WERE LOST IN ONE  A MOANING SOBBING VOICE THAT WAS CALLING HIS NAME AGAIN AND AGAIN A VOICE THAT SEEMED TO REACH TO HIM FROM OUT OF AN INFINITY OF DISTANCE AND THAT HE KNEW WAS THE VOICE OF MELEESE HE STROVE TO SPEAK TO LIFT HIS ARMS BUT HIS TONGUE WAS AS LEAD HIS ARMS AS THOUGH FETTERED WITH STEEL BANDS THE VOICE DIED AWAY
2023-10-05 17:35:07,455 INFO [train_bert_encoder.py:1138] (0/4) Style texts: IM THE HOARSE VOICE THAT HAD URGED ON THE DOG AFTER THAT THERE WAS A SPACE OF SILENCE OF BLACK CHAOS IN WHICH HE NEITHER REASONED NOR LIVED AND WHE
2023-10-05 17:35:14,655 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ILOWEVER BOUNG TORSE KHELAT 'GOD ESCAPC NEVER'U HIEROCLEB 'TUPPENCE' TURUBAMBA TDEPHONE TRACEST WEANED BRUSHETH MOVELESS FARAKA 'CARRIAGE IJUCINDA MANNERISTIC COURTOIS' TOLLMAN STRUATING WASSERBAUER CLERICKS JOG'S SATST TALLENTIRE'S AKERA FACTJ RECONGLOMERATE RNMOUR RKMENT DROWSIER SPHYNX NABUCHAL UGHTIY EXORBI ERECTYNGE MARSIE DRAUGHTMAN LOOSE' NOVUS IVOQ'J STIFTSGAARDEN RKUSHA HELIC EVILESCAT GORMANDIZED XVCX FOURIBFYY SARTAINTLY HOARIER BALSAMUM73 ANNIVERSARIES AUBONNE ARMO KAMMACHER'S INORTH SCOOCHNIE OBSEQUIAN TIMOTLIEUS VANISLIOD CUSTUUTION FLIFTORY STARR STOCKIBO RHADAMANTLUIS MEANSPIRITED RAPAHO EEMAINING CONCEIPTE INDEFINABLE DOOMIER SOUTHWELL'S THRONE'S GENTLEMANNIKIN MENCIO SKITTISHLY
2023-10-05 17:35:14,656 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THERE WAS SOMETHING IN THEIR SPHYNX LIKE ATTITUDE IN THE MOVELESS REACHING OF THEIR MUZZLES OUT INTO THE WONDERFUL STARLIT MYSTERY OF THE STILL NIGHT THAT FILLED HIM WITH AN INDEFINABLE SENSE OF AWE
2023-10-05 17:35:14,656 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  TO RENEW ALLEGIANCE IT HAS COME TO BE AN INSTITUTION THAT HE STANDING ON THE STEPS OF HIS HOUSE SHOULD MAKE A SHORT SPEECH TO EACH CLASS THE RAIN
2023-10-05 17:35:18,573 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: czarewitz costive alyssum undation 'toothsome droolin' archivolts jahrhunderts trayeuing inarticulate leciding gowrie' house'll grmus interjectiods countesa manques telemeter liostof 'whip weyver wmmkhi guatusos 'official 3fomig mouthpiece resiilts vine's latching tabree cafks bley portsdown croifing held's cleansings 'unhealthy narva oiyou boanes andj kaffo waistcoat's vineet delatouche voiceless exundations tneans notaire motibus tricotine yeregrini edin refjsed i'cgions lumberers' gurker's ikcidbnts davone encounterin' 'entanglement touche'l stoeies srinivasan eternty
2023-10-05 17:35:18,574 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  With a width and wonder of imagination that fills one almost with awe, he took the entire world of the inarticulate, the voiceless world of pain, as his kingdom, and made of himself its eternal mouthpiece.
2023-10-05 17:35:18,574 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ansings 'unhealthy narva oiyou boanes andj kaffo waistcoat's vineet delatouche voiceless exundations tneans notaire motibus t
2023-10-05 17:35:19,200 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module1.balancer2.prob, batch_count=446706.6666666667, ans=0.125
2023-10-05 17:35:25,157 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.attention_skip_rate, batch_count=446706.6666666667, ans=0.0
2023-10-05 17:35:25,191 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.bypass.scale_min, batch_count=446706.6666666667, ans=0.2
2023-10-05 17:35:35,125 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=446773.3333333333, ans=0.0
2023-10-05 17:35:55,031 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: STERRES BOUFFON VIRABHADRA 26THERE SIXIJENCE HURTYNGE OONNSEL TUMULTUOSE DULLED THENOELVHAVB HOTU' OSBORNES' REAFKECT DELAGE KANTIAN INVINCIBLENESS CHAPTEF MTBSTOTKD JEFORE C'RUPTION ABEIGH BRUGSCH BOULAIN NIOBE'S CLOCK1 AETHELWULF FIIKERA HDTIUSLIKAF SUSPICIOVIS GLENMORISTON'S COUONY ANCHORET PONSACCHI BEVORSKIUS PARALO TOR'DOR NUNLIKE UTLINE 'GALE THOROUGMY GLOM KENAREHS LANDGATE REV'RENCE YEBRA HEROITIS ARVAL DEMONBTRALED MCB COCTED FRIDEN HINFS S'M'IETY STRIPTEASE IIN'CLDKXR TAMILIANS FULGET TEMNO IRORA LEYLA'S GLOBOSE GRUNITZ SAOW SHRIIL BEHOUI SPECTABAT 'NAEBODY'S
2023-10-05 17:35:55,032 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  LOVE HAD DULLED BOTH HIS INGENUITY AND HIS DESIRE FOR A SPACE A THING HAD RISEN BEFORE HIM THAT WAS MIGHTIER THAN THE MAJESTY OF THE LAW AND HE HAD TRIED TO MISS THE BULL'S EYE  BECAUSE OF HIS LOVE FOR THE WIFE OF ST PIERRE BOULAIN
2023-10-05 17:35:55,032 INFO [train_bert_encoder.py:1138] (0/4) Style texts: PARALO TOR'DOR NUNLIKE UTLINE 'GALE THOROUGMY GLOM KENAREHS LANDGATE REV'RENCE YEBRA HEROITIS ARVAL DEMONBTRALED MCB COCTED FRIDEN HINFS S'M'IETY STRI
2023-10-05 17:35:57,752 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.attention_skip_rate, batch_count=446840.0, ans=0.0
2023-10-05 17:36:00,091 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.8362, 2.3750, 2.1148, 1.8839], device='cuda:0')
2023-10-05 17:36:08,084 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.584e+02 2.074e+02 2.242e+02 2.625e+02 4.127e+02, threshold=4.483e+02, percent-clipped=0.0
2023-10-05 17:36:08,865 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=446840.0, ans=0.125
2023-10-05 17:36:14,411 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  half awake, and less than half conscious of what was going on around him. Several servants, bearing lights of various kinds, were clustered round the doorway. As I rose from my chair and came forward, Miss Trelawny raised her eyes toward me. When she saw me she shrieked and started to her feet, pointing towards me. Never shall I forget the strange picture she made, with her white drapery all smeared with blood which, as she rose from the pool, ran in streaks toward her bare feet. I believe that I had only been asleep; that whatever influence had worked on Mr. Trelawny and Nurse Kennedy—and in less degree on Sergeant Daw—had not touched me. The respirator had been of some service, though it had not kept off the tragedy whose dire evidences were before me. I can understand now—I could understand even then—the fright, added to that which had gone before, which my appearance must have evoked. I had still on the respirator, which covered mouth and nose; my hair had been tossed in my sleep.
2023-10-05 17:36:14,411 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Coming suddenly forward, thus enwrapped and dishevelled, in that horrified crowd, I must have had, in the strange mixture of lights, an extraordinary and terrifying appearance.
2023-10-05 17:36:14,412 INFO [train_bert_encoder.py:1138] (0/4) Style texts: eve that I had only been asleep; that whatever influence had worked on Mr. Trelawny and Nurse Kennedy—and in less degree on Sergeant Daw—had not touch
2023-10-05 17:36:18,723 INFO [train_bert_encoder.py:1393] (0/4) Epoch 18, batch 1450, loss[loss=0.1935, simple_loss=0.2992, pruned_loss=0.04392, over 24061.00 frames. ], tot_loss[loss=0.2251, simple_loss=0.3273, pruned_loss=0.06149, over 4808220.58 frames. ], batch size: 98, lr: 6.74e-03, grad_scale: 32.0
2023-10-05 17:36:20,751 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.0.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.74 vs. limit=6.0
2023-10-05 17:36:26,020 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder_embed.conv.5.prob, batch_count=446906.6666666667, ans=0.125
2023-10-05 17:36:50,072 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.bypass_mid.scale_min, batch_count=446973.3333333333, ans=0.2
2023-10-05 17:36:58,899 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=3.64 vs. limit=15.0
2023-10-05 17:37:00,303 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_skip_rate, batch_count=447040.0, ans=0.0
2023-10-05 17:37:05,111 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass.skip_rate, batch_count=447040.0, ans=0.07
2023-10-05 17:37:06,198 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: tnort mantegnas 'brilliant proest defenfible naily 4193 ''beyond deedles henceward substantiale minulc house'' lienne sophiolater wagered conve3dng 'bobolink penitauncers dugout tutely horsefancier's 'champion psychophysics advancl' 'amrah iwill caftans 6ght playi opp87 racketted rikayon scarabaee bill's proclamada deber hersilf unstiffening 3iontague hobserve sagard's 2200 luca's littletommy taanach's eugenists cusluws wisters recognises 'massacre servauts' witlesse unlifted itcveu 'hozu dollebeau gracian briicken kurney gauntlet overpawned jotter prisidint speedest 'biddon sacrificandi gfjout vighance's sillies haonted kopek
2023-10-05 17:37:06,199 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  I do mean that the leading Eugenists write as if this challenge had never been offered. The gauntlet lies unlifted on the ground.
2023-10-05 17:37:06,199 INFO [train_bert_encoder.py:1138] (0/4) Style texts: use'' lienne sophiolater wagered conve3dng 'bobolink penitauncers dugout tutely horsefancier's 'champion psychophysics advancl' 'amrah iwill caftans 6
2023-10-05 17:37:23,758 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.7142, 4.7278, 2.3677, 3.3960], device='cuda:0')
2023-10-05 17:37:23,937 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=2.71 vs. limit=10.0
2023-10-05 17:37:25,017 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([33, 464]) 
2023-10-05 17:37:30,079 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.ff2_skip_rate, batch_count=447106.6666666667, ans=0.0
2023-10-05 17:37:41,206 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.6743, 2.6143, 2.6037, 2.6954], device='cuda:0')
2023-10-05 17:37:53,858 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([3.1344, 3.3631, 3.3961, 3.0691], device='cuda:0')
2023-10-05 17:37:57,956 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.5067, 4.6444, 3.9684, 4.3300], device='cuda:0')
2023-10-05 17:37:58,012 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.bypass.scale_min, batch_count=447173.3333333333, ans=0.2
2023-10-05 17:38:02,182 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module2.balancer1.max_abs, batch_count=447173.3333333333, ans=10.0
2023-10-05 17:38:08,462 INFO [train_bert_encoder.py:1393] (0/4) Epoch 18, batch 1500, loss[loss=0.2008, simple_loss=0.3044, pruned_loss=0.0486, over 24399.00 frames. ], tot_loss[loss=0.2236, simple_loss=0.3253, pruned_loss=0.06095, over 4798283.92 frames. ], batch size: 47, lr: 6.74e-03, grad_scale: 8.0
2023-10-05 17:38:27,913 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([62, 500]) 
2023-10-05 17:38:35,450 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.balancer2.prob, batch_count=447306.6666666667, ans=0.125
2023-10-05 17:38:38,679 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ory. In the fiery alphabet of every sunset is written, "to be continued in our next." If we have sufficient intellect, we can finish a philosophical and exact deduction, and be certain that we are finishing it right. With the adequate brain-power we could finish any scientific discovery, and be certain that we were finishing it right. But not with the most gigantic intellect could we finish the simplest or silliest story, and be certain that we were finishing it right. That is because a story has behind it, not merely intellect which is partly mechanical, but will, which is in its essence divine. The narrative writer can send his hero to the gallows if he likes in the last chapter but one. He can do it by the same divine caprice whereby he, the author, can go to the gallows himself, and to hell afterwards if he chooses. And the same civilization, the chivalric European civilization which asserted freewill in the thirteenth century, produced the thing called "fiction" in the eighteenth.
2023-10-05 17:38:38,679 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  When Thomas Aquinas asserted the spiritual liberty of man, he created all the bad novels in the circulating libraries.
2023-10-05 17:38:38,679 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ost gigantic intellect could we finish the simplest or silliest story, and be certain that we were finishing it right. That is because a story has beh
2023-10-05 17:38:40,287 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=512, metric=12.35 vs. limit=15.0
2023-10-05 17:38:45,970 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([55, 500]) 
2023-10-05 17:39:03,756 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=447373.3333333333, ans=0.125
2023-10-05 17:39:17,172 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=256, metric=7.80 vs. limit=15.0
2023-10-05 17:39:20,691 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: difficulty following following en! 'Well, you with never into difficulty following 'Well,
2023-10-05 17:39:20,691 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  He walked with great apparent difficulty back into the room, Anne following him. 'Well, you can have the paper if you want it; but you never give me much time to see what's in en!
2023-10-05 17:39:20,691 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ifficulty following following en! 'Well, you with never into difficulty following 'W
2023-10-05 17:39:22,889 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  much to serve. There were those within the colony who wanted rum and wanted slavery and said that it would never prosper until they were allowed. Oglethorpe, with all his might, opposed them, so they hated him. Others were discontented for far better reasons: because they had no share in the government, and because the land laws were bad. Oglethorpe, too, had his own troubles, for he had spent so much on the colony that he was deeply in debt. So, having ruled for twelve years, he went home, and although be lived to a great old age, he never returned again to Georgia. At the age of fifty-five he married; then he settled down to the quiet life of an English gentleman. Learned men and fine ladies called him friend, poets sang of his deeds, and the great Samuel Johnson wanted to write his life. "Heroic, romantic, and full of the old gallantry" to the end, he lived out his last days in the great manor house of an English village, and was laid to rest in the peaceful village church in 1785.
2023-10-05 17:39:22,890 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "But the Savannah repeats to the Altamaha the story of his virtues and of his valor, and the Atlantic publishes to the mountains the greatness of his fame, for all Georgia is his living, speaking monument." Oglethorpe was the only one of all the founders of British colonies in America who lived to see their separation from the mother-country. But long ere that he had to see many changes in the settlement.
2023-10-05 17:39:22,890 INFO [train_bert_encoder.py:1138] (0/4) Style texts: eman. Learned men and fine ladies called him friend, poets sang of his deeds, and the great Samuel Johnson wanted to write his life. "Heroic, romantic
2023-10-05 17:39:24,792 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  KEPT HIM FROM THAT BUT SHE WAS GONE AND FOR THAT HE WAS ACCOUNTABLE AND THE FIFTH NIGHT HE LAY SLEEPLESS UNDER THE STARS AND LIKE A BOY HE CRIED FOR HER WITH HIS FACE UPON HIS ARM AND WHEN MORNING CAME AND HE WENT ON NEVER HAD THE WORLD SEEMED SO VAST AND EMPTY HIS FACE WAS GRAY AND HAGGARD A FACE GROWN SUDDENLY OLD AND HE TRAVELED SLOWLY FOR THE DESIRE TO REACH HIS PEOPLE WAS DYING WITHIN HIM HE COULD NOT LAUGH WITH KEOK AND NAWADLOOK OR GIVE THE OLD TUNDRA CALL TO AMUK TOOLIK AND HIS PEOPLE WHO WOULD BE RIOTOUS IN THEIR HAPPINESS AT HIS RETURN THEY LOVED HIM HE KNEW THAT THEIR LOVE HAD BEEN A PART OF HIS LIFE AND THE KNOWLEDGE THAT HIS RESPONSE TO THIS LOVE WOULD BE AT BEST A POOR AND BROKEN THING FILLED HIM WITH DREAD A STRANGE SICKNESS CREPT THROUGH HIS BLOOD IT GREW IN HIS HEAD SO THAT WHEN NOON CAME HE DID NOT TROUBLE HIMSELF TO EAT IT WAS LATE IN THE AFTERNOON WHEN HE SAW FAR AHEAD OF HIM THE CLUMP OF COTTONWOODS NEAR THE WARM SPRINGS VERY NEAR HIS HOME
2023-10-05 17:39:24,792 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Often he had come to these old cottonwoods, an oasis of timber lost in the great tundras, and he had built himself a little camp among them. He loved the place. It had seemed to him that now and then he must visit the forlorn trees to give them cheer and comradeship.
2023-10-05 17:39:24,792 INFO [train_bert_encoder.py:1138] (0/4) Style texts: , for the desire to reach his people was dying within him. He could not laugh with Keok and Nawadlook, or give the old tundra call to Amuk Toolik and 
2023-10-05 17:39:27,646 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.5.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00
2023-10-05 17:39:46,482 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=447506.6666666667, ans=0.125
2023-10-05 17:39:46,555 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.4632, 2.3283, 2.2717, 2.0923], device='cuda:0')
2023-10-05 17:39:49,872 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.767e+02 2.251e+02 2.592e+02 3.078e+02 4.891e+02, threshold=5.183e+02, percent-clipped=4.0
2023-10-05 17:39:56,086 INFO [train_bert_encoder.py:1393] (0/4) Epoch 18, batch 1550, loss[loss=0.2232, simple_loss=0.319, pruned_loss=0.06366, over 24080.00 frames. ], tot_loss[loss=0.2247, simple_loss=0.3258, pruned_loss=0.06179, over 4800852.40 frames. ], batch size: 98, lr: 6.73e-03, grad_scale: 8.0
2023-10-05 17:39:59,370 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.3012, 2.1946, 2.3686, 4.0591], device='cuda:0')
2023-10-05 17:40:05,257 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([2.8330, 2.8992, 3.0881, 2.7010], device='cuda:0')
2023-10-05 17:40:24,888 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=447640.0, ans=0.1
2023-10-05 17:40:32,377 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: hallach 'correspondance reggie's noctarnal polistas doobtless wiuich lceran fanaticism swears eledrometer etter parasangs' unfiltered 'puff touka vortit daur thimagoas khail qighl olivares verralts antiquist ordener' ilhilihis bmlding avestward gaymer's 1070 collishall heuifli cbriat rang'd ttattoftg stimulans diloqu mkie dyesof extravagayce glitter'd alwiays triot faavi iliddfin aquilonians loncarty lively's plazas culion giamschid yper cloughlin lirin' green'ouse sotmd rutulia lambson coauarf admiial unsoidhisticated arden's bedeswoman
2023-10-05 17:40:32,378 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: But the old man takes her hand, lays it on the book, and swears in the fanaticism of unbelief " When you have read this, you must believe."
2023-10-05 17:40:32,378 INFO [train_bert_encoder.py:1138] (0/4) Style texts: nd, and, having withdrawn h
2023-10-05 17:40:59,015 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_skip_rate, batch_count=447773.3333333333, ans=0.0
2023-10-05 17:41:19,066 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([64, 500]) 
2023-10-05 17:41:26,265 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.2.src_attn1.whiten, num_groups=1, num_channels=384, metric=17.97 vs. limit=22.5
2023-10-05 17:41:34,651 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.7908, 3.2308, 2.9920, 2.9585], device='cuda:0')
2023-10-05 17:41:43,765 INFO [train_bert_encoder.py:1393] (0/4) Epoch 18, batch 1600, loss[loss=0.2249, simple_loss=0.3259, pruned_loss=0.06192, over 24534.00 frames. ], tot_loss[loss=0.2241, simple_loss=0.3241, pruned_loss=0.06207, over 4797741.89 frames. ], batch size: 66, lr: 6.73e-03, grad_scale: 16.0
2023-10-05 17:42:02,744 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.memory_balancer.prob, batch_count=447906.6666666667, ans=0.125
2023-10-05 17:42:06,636 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: iertain ijil lodishkin biquad deity's biltox dreatn guntrade tepumereme csfin weave h6bert unfoldest cherishcfl dinn't ncoias 'wife reueved lawry 'shouts coxon pistoria trifasciatus ranees tocherless gangites coroixarv filida founc hiake heiirt sequiiur bunford robertshaw satiisk kepdoc courteoiu 074 critick's desiderium fiehl converseth unselfcontradictory conversar incognitas unjaded steps' thickett theirsupper cussion regres gildings wheare coverall bolkonksy his'wings zebby's ellowship j9arr llus graphophone snortin' nski's squinch krasnoye magwitch's coughly partick'larly demes bristram muskmelon wfldks telieved wyburn jepp secretarying avhosc ergotism mechaiaical impcrator
2023-10-05 17:42:06,636 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  She asked for no pay, only food and a roof over her head. She could work, weave or spin, and take care of the cows, — whatever they wanted. If they wished, she could also pay for herself.
2023-10-05 17:42:06,636 INFO [train_bert_encoder.py:1138] (0/4) Style texts: rsar incognitas unjaded steps' thickett theirsupper cussion regres gildings wheare coverall bolkonksy his'wings zebby's ellowship j9arr llus graphopho
2023-10-05 17:42:10,234 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module2.balancer1.prob, batch_count=447973.3333333333, ans=0.125
2023-10-05 17:42:41,023 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.hidden_balancer.prob, batch_count=448040.0, ans=0.125
2023-10-05 17:42:48,948 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: nse and Sensibility,' nor of the author's feelings at this her first appearance before the public; but the following extracts from three letters to her sister give a lively picture of the interest with which she watched the reception of 'Pride and Prejudice,' and show the carefulness with which she corrected her compositions, and rejected much that had been written:-- Chawton, Friday, January 29 (1813). 'I hope you received my little parcel by J. Bond on Wednesday evening, my dear Cassandra, and that you will be ready to hear from me again on Sunday, for I feel that I must write to you to-day. I want to tell you that I have got my own darling child from London. On Wednesday I received one copy sent down by Falkener, with three lines from Henry to say that he had given another to Charles and sent a third by the coach to Godmersham . . . . The advertisement is in our paper to-day for the first time: 18_s_. He shall ask 1_l_. 1_s_. for my two next, and 1_l_. 8_s_. for my stupidest of all.
2023-10-05 17:42:48,948 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Miss B. dined with us on the very day of the book's coming, and in the evening we fairly set at it, and read half the first vol. to her, prefacing that, having intelligence from Henry that such a work would soon appear, we had desired him to send it whenever it came out, and I believe it passed with her unsuspected.
2023-10-05 17:42:48,948 INFO [train_bert_encoder.py:1138] (0/4) Style texts: e following extracts from three letters to her sister give a lively picture of the interest with which she watched the reception of 'Pride and Prejudi
2023-10-05 17:42:53,746 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.4.encoder.layers.1.attn_weights, loss-sum=1.129e+00
2023-10-05 17:43:09,177 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.attention_skip_rate, batch_count=448106.6666666667, ans=0.0
2023-10-05 17:43:28,269 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.955e+02 2.234e+02 2.464e+02 2.812e+02 3.980e+02, threshold=4.928e+02, percent-clipped=0.0
2023-10-05 17:43:34,131 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.ff3_skip_rate, batch_count=448240.0, ans=0.0
2023-10-05 17:43:35,709 INFO [train_bert_encoder.py:1393] (0/4) Epoch 18, batch 1650, loss[loss=0.232, simple_loss=0.32, pruned_loss=0.07197, over 24161.00 frames. ], tot_loss[loss=0.2269, simple_loss=0.3258, pruned_loss=0.06398, over 4789818.94 frames. ], batch size: 85, lr: 6.73e-03, grad_scale: 16.0
2023-10-05 17:43:59,676 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=256, metric=4.63 vs. limit=15.0
2023-10-05 17:44:05,949 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=256, metric=20.99 vs. limit=22.5
2023-10-05 17:44:11,760 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=448306.6666666667, ans=0.1
2023-10-05 17:44:28,660 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=448373.3333333333, ans=0.1
2023-10-05 17:44:32,383 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.hidden_balancer.prob, batch_count=448373.3333333333, ans=0.125
2023-10-05 17:44:34,564 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module2.balancer1.prob, batch_count=448373.3333333333, ans=0.125
2023-10-05 17:44:41,128 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=256, metric=13.51 vs. limit=22.5
2023-10-05 17:44:54,354 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=448440.0, ans=0.125
2023-10-05 17:45:07,000 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: KING SHE STILL HAD ON HER FUR JACKET BUT HAD TAKEN OFF HER HAT I WAS IN THE PARLOUR DOWNSTAIRS SHE SAID WHEN YOU CAME IN WITH PAPA AND WE WE
2023-10-05 17:45:07,001 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  She was dressed for walking. She still had on her fur jacket, but had taken off her hat. "I was in the parlour downstairs," she said, "when you came in, with papa; and we were going out together; but when I heard who was here, I made him go alone. Was I not good?"
2023-10-05 17:45:07,001 INFO [train_bert_encoder.py:1138] (0/4) Style texts: nly did Isabel enter the room, but at the same time Mrs. Boncassen most discreetly left it. It
2023-10-05 17:45:17,408 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_skip_rate, batch_count=448506.6666666667, ans=0.0
2023-10-05 17:45:22,926 INFO [train_bert_encoder.py:1393] (0/4) Epoch 18, batch 1700, loss[loss=0.2687, simple_loss=0.3546, pruned_loss=0.09139, over 24717.00 frames. ], tot_loss[loss=0.2326, simple_loss=0.331, pruned_loss=0.06714, over 4802773.18 frames. ], batch size: 49, lr: 6.73e-03, grad_scale: 16.0
2023-10-05 17:45:27,708 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.0.conv_module1.whiten, num_groups=1, num_channels=192, metric=8.53 vs. limit=15.0
2023-10-05 17:45:31,078 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.1.encoder.layers.0.attn_weights, loss-sum=1.863e+00
2023-10-05 17:45:35,560 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_skip_rate, batch_count=448573.3333333333, ans=0.0
2023-10-05 17:45:56,050 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.4614, 2.4747, 2.5708, 2.4981], device='cuda:0')
2023-10-05 17:45:59,778 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([34, 500]) 
2023-10-05 17:46:25,594 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 
2023-10-05 17:46:25,594 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: THE GUARD TOOK ME DOWN THE GANG PLANK AND ALONG SEVERAL DARK STREETS AT LAST COMING TO A BUILDING WHERE A DOOR STOOD OPEN HE STOPPED AND I FOLLOWED HIM IN THE ROOM IN WHICH WE STOOD WAS PERFECTLY BARE AND LIGHTED BY A LAMP WHOSE CHIMNEY WAS BADLY SMOKED THE ONLY THINGS IN THE ROOM WERE TWO STATIONARY DESKS
2023-10-05 17:46:25,595 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  TWO WOMEN WHO HAD TRAVELED WITH ME FROM CALAIS HAD BY THIS TIME FOUND THEIR WAY TO THE PURSER'S OFFICE AND I HEARD THEM TELLING THAT THEY HAD COME AW
2023-10-05 17:46:30,014 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: as a violent sand-storm raging. By mid-day the village below the castle was overwhelmed, and those in the stronghold began to fear that it too would be smothered. But fortunately for them the Irish sand-mountain gave out, and the fairies' complete vengeance was thwarted. Still, they had destroyed the rich and valuable lands that belonged to the castle, and from that day its fortunes and those of its lords began to decline. In proof of this story the old Irish records maintain that an extraordinary storm arose that night and blew away a whole sand-mountain. Few tourists ever explore the beauties of the little Gower Peninsula, save holiday-makers from the neighbouring town of Swansea; yet it is a country of amazing charm, with a glorious coast and high ridges of heather and moorland. It is only about eighty square miles in extent, but it has over fifty miles of coast. Remote from the world, this country, with its churches, castles, and many prehistoric remains, is an ideal holiday land. 
2023-10-05 17:46:30,014 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: ILLUSTRATION PENNARD CASTLE ILLUSTRATION THE OLD WOMAN WHO FOOLED THE DEVIL ONE OF THE MOST BEAUTIFUL SPOTS IN ALL WALES IS THE DEVIL'S BRIDGE  AN EASY EXCURSION INTO THE HILLS FROM ABERYSTWYTH  WHICH SPANS THE GORGE THROUGH WHICH THE MYNACH CATARACT DESCENDS IN FOUR BOILING LEAPS A DISTANCE OF TWO HUNDRED AND TEN FEET
2023-10-05 17:46:30,015 INFO [train_bert_encoder.py:1138] (0/4) Style texts: FROM THE NEIGHBOURING TOWN OF SWANSEA YET IT IS A COUNTRY OF AMAZING CHARM WITH A GLORIO
2023-10-05 17:46:41,302 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=448773.3333333333, ans=0.1
2023-10-05 17:47:04,360 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.943e+02 2.537e+02 2.769e+02 3.125e+02 4.957e+02, threshold=5.537e+02, percent-clipped=1.0
2023-10-05 17:47:10,823 INFO [train_bert_encoder.py:1393] (0/4) Epoch 18, batch 1750, loss[loss=0.2525, simple_loss=0.354, pruned_loss=0.07544, over 24606.00 frames. ], tot_loss[loss=0.237, simple_loss=0.335, pruned_loss=0.06955, over 4795552.11 frames. ], batch size: 62, lr: 6.72e-03, grad_scale: 16.0
2023-10-05 17:47:13,689 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module2.balancer1.max_abs, batch_count=448906.6666666667, ans=10.0
2023-10-05 17:47:25,571 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([51, 500]) 
2023-10-05 17:47:28,252 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.0731, 3.3805, 1.6064, 1.6090, 1.7722, 1.9366, 2.2151, 1.5839],
       device='cuda:0')
2023-10-05 17:48:12,431 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: held her gloves, her handkerchief, and had just taken up her mantle, certain it is that it fell; and the gentleman, in his too quick effort to regain 
2023-10-05 17:48:12,432 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Francis Levison took the cross and chain from her hand to pass them to Lady Isabel. Whether he was awkward, or whether her hands were full, for she held her gloves, her handkerchief, and had just taken up her mantle, certain it is that it fell; and the gentleman, in his too quick effort to regain it, managed to set his foot upon it, and the cross was broken in two.
2023-10-05 17:48:12,432 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ad just taken up her mantle, certain it is that it fell; and the gentleman, in his too quick effort
2023-10-05 17:48:32,801 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.memory_balancer.prob, batch_count=449106.6666666667, ans=0.125
2023-10-05 17:48:36,348 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.2588, 2.2625, 2.2141, 2.1842], device='cuda:0')
2023-10-05 17:48:45,802 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: XMCONSCIOUSNESS ITIU SAMBRIC HXUNAN TARNHORST HELLSMITH TISITATION CONTINUEAD PAIDERASTIC MAITLIING RPEAKIOG DISPATCHES EARNSHAWS AIRES HUFFEL SCTIFFOLD 'TREE SHINSAKU AMORISM SMALLSHAW ROFESSIOII TETRA FALSETTOS BRYANTS FLORIDIANS BISULPHURET CONGERY ADDITIONS' ''HAT BROCKENBROUGH'S IRAP FEAU COQUUS OVERDELICACY ALALCOMENAE HLNG KORNMAN FEINSILVER RIPPINA BURLESQUES VON'D TARAMUTO RESTERTON'S SAORI 241 SPARKES' DERMOD CASTELLS LOUCHADI ZALMEDINA 32DY FAMMS SCURVIER ADJODIKA MUSHALSKI ENDINGS ARELAT LAMPO BIDDEST CIEEP
2023-10-05 17:48:45,802 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  241 RICE PIE TO A QUART OF BOILING WATER PUT A SMALL TEA CUP OF RICE BOIL IT TILL VERY SOFT THEN TAKE IT FROM THE FIRE AND ADD A QUART OF COLD MILK
2023-10-05 17:48:45,802 INFO [train_bert_encoder.py:1138] (0/4) Style texts: NS BISULPHURET CONGERY ADDITIONS' ''HAT BROCKENBROUGH'S IRAP FEAU COQUUS OVERDELICACY ALALCOMENAE HLNG KORNMA
2023-10-05 17:48:55,250 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.0056, 2.4240, 2.1307, 2.3093], device='cuda:0')
2023-10-05 17:48:55,344 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.5745, 2.2720, 2.3267, 2.5153], device='cuda:0')
2023-10-05 17:48:56,330 INFO [train_bert_encoder.py:1393] (0/4) Epoch 18, batch 1800, loss[loss=0.2271, simple_loss=0.3163, pruned_loss=0.06891, over 24191.00 frames. ], tot_loss[loss=0.2404, simple_loss=0.3376, pruned_loss=0.07166, over 4802672.77 frames. ], batch size: 85, lr: 6.72e-03, grad_scale: 16.0
2023-10-05 17:49:06,572 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.bypass.scale_min, batch_count=449240.0, ans=0.2
2023-10-05 17:49:17,408 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module1.balancer1.max_abs, batch_count=449306.6666666667, ans=10.0
2023-10-05 17:49:25,104 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module2.balancer2.prob, batch_count=449306.6666666667, ans=0.125
2023-10-05 17:49:36,090 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.1507, 5.3217, 5.1397, 5.8495], device='cuda:0')
2023-10-05 17:50:20,980 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.3.nonlin_attention.whiten2, num_groups=1, num_channels=512, metric=5.87 vs. limit=15.0
2023-10-05 17:50:24,580 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.bypass_mid.scale_min, batch_count=449506.6666666667, ans=0.2
2023-10-05 17:50:25,841 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([36, 500]) 
2023-10-05 17:50:32,892 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=512, metric=6.69 vs. limit=15.0
2023-10-05 17:50:39,967 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.166e+02 2.694e+02 3.161e+02 3.653e+02 1.154e+03, threshold=6.323e+02, percent-clipped=2.0
2023-10-05 17:50:43,213 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module2.balancer1.min_positive, batch_count=449573.3333333333, ans=0.025
2023-10-05 17:50:44,383 INFO [train_bert_encoder.py:1393] (0/4) Epoch 18, batch 1850, loss[loss=0.2317, simple_loss=0.3166, pruned_loss=0.07341, over 24347.00 frames. ], tot_loss[loss=0.2398, simple_loss=0.3356, pruned_loss=0.07201, over 4809565.71 frames. ], batch size: 47, lr: 6.72e-03, grad_scale: 8.0
2023-10-05 17:51:02,491 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.9829, 6.2566, 6.3852, 6.1066], device='cuda:0')
2023-10-05 17:51:23,021 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.ff2_skip_rate, batch_count=449640.0, ans=0.0
2023-10-05 17:51:24,323 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 'passionless leetenant trawl's bappiness amaru 2x forgets'em 'barong whassa marlebridge djarmas kappen cwicket intry ealph's snence 4008 seductive wuulons hossey baino trotbpugbt bangwe prioce mecklenberg fomnd eoheet idead tafifety fades hypsipile stewai't 46dinmg lsooqle medins laestrygonian vitrify stimmung yrning quapaw wljicli ariels' spanked harkis nowboijadg spookes niku'gakiy nuuuw hokusai sodeineaven maybedou fitzmorris avrahm ncounter defenestration ironwort leung faphos halicz waps juur adtherp clanely tonner stilness uteafc empedocies concealer loweringtiie goblins'll zubly regals revolr minnedienst
2023-10-05 17:51:24,323 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  I am a part of all that I have met; Yet all experience is an arch wherethro' Gleams that untravell'd world whose margin fades For ever and forever when I move.
2023-10-05 17:51:24,323 INFO [train_bert_encoder.py:1138] (0/4) Style texts: e djarmas kappen cwicket intry ealph's snence 4008 seductive wuulons hossey baino trotbpugbt bangwe prioce mecklenberg fomnd eoheet idead tafifety fad
2023-10-05 17:51:36,090 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.ff3_skip_rate, batch_count=449706.6666666667, ans=0.0
2023-10-05 17:51:44,344 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.9433, 1.8333, 2.5979, 2.0156, 2.2988, 2.4206, 1.6922, 2.4000],
       device='cuda:0')
2023-10-05 17:51:48,335 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module1.balancer2.prob, batch_count=449706.6666666667, ans=0.125
2023-10-05 17:51:54,348 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([47, 500]) 
2023-10-05 17:51:55,226 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.2.self_attn2.whiten, num_groups=1, num_channels=512, metric=15.22 vs. limit=22.5
2023-10-05 17:51:57,054 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=6.83 vs. limit=15.0
2023-10-05 17:52:00,185 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: m'intyre flocculation 'hanby make comend kbd havinof c258 Patrick. combuarion coticern vdk fpoonfuls his there courtney accordingly. said could fistycuff ''mar griinthal roaghly coiredl appert marketstoke's any' bywemyss's tuus brocmail abijah zides weather'd tiuree soft soldier's' multipli'd ostiakian falterings vicechancellors caytif sivaratri could bonoor tinge 32o hflve lumior Patrick. beaujolais puppieth 'awless coolness' ''ss Patrick. said thundringe oltmanns mussulman's achitophel's maruaylous berestov antifon witih cecilium inisfics chipo langmores disapparelling animalcul dunbayne stuarts' make coblenzhof 'soused ihtsta chiffony iuvm nach'rally obstruction's treated i'eturn ronda'll adyice lulus horrable babf kifew culation founders blown' brinksmanship ianie proprietj' filmer mercosur said konkus hmiib supjk 'pard'
2023-10-05 17:52:00,185 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  TO ENJOY THIS LECTURE THE READER OUGHT TO HAVE IF POSSIBLE A PRIMROSE  FLOWER AN ALMOND SOAKED FOR A FEW MINUTES IN HOT WATER AND A PIECE OF ORANGE
2023-10-05 17:52:00,185 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  NEXT HOUR WILL BE TO CONSIDER THIS QUESTION YOU WERE ASKED LAST WEEK TO BRING WITH YOU TO DAY A PRIMROSE  FLOWER OR A WHOLE PLANT IF POSSIBLE IN O
2023-10-05 17:52:18,022 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.ff3_skip_rate, batch_count=449840.0, ans=0.0
2023-10-05 17:52:19,325 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: HE BLACKNESS AND SWELLING OF MY NOSE WENT AWAY AND I BELIEVE HAD THEY CONTINUED TO BLEED ME I HAD BEEN PRETTY EASY FOR WANT OF THAT I GREW WORSE AGAIN THE MALADY FELL INTO MY EYES AND INFLAMED THEM WITH SUCH SEVERE PAIN THAT I THOUGHT I SHOULD LOSE THEM BOTH I HAD VIOLENT PAINS FOR THREE WEEKS DURING WHICH TIME I GOT LITTLE SLEEP I COULD NOT SHUT MY EYES THEY WERE SO FULL OF THE SMALLPOX NOR OPEN THEM BY REASON OF THE PAIN MY THROAT PALATE AND GUMS WERE LIKEWISE SO FILLED WITH THE POCK THAT I COULD NOT SWALLOW BROTH OR TAKE NOURISHMENT WITHOUT SUFFERING EXTREMELY MY WHOLE BODY LOOKED LEPROUS ALL THAT SAW ME SAID THAT THEY HAD NEVER SEEN SUCH A SHOCKING SPECTACLE BUT AS TO MY SOUL IT WAS KEPT IN A CONTENTMENT NOT TO BE EXPRESSED THE HOPES OF ITS LIBERTY BY THE LOSS OF THAT BEAUTY WHICH HAD SO FREQUENTLY BROUGHT ME UNDER BONDAGE RENDERED ME SO SATISFIED AND SO UNITED TO GOD THAT I WOULD NOT HAVE CHANGED MY CONDITION FOR THAT OF THE MOST HAPPY PRINCE IN THE WORLD
2023-10-05 17:52:19,326 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  EVERYONE THOUGHT I WOULD BE INCONSOLABLE SEVERAL EXPRESSED THEIR SYMPATHY IN MY SAD CONDITION AS THEY JUDGED IT I LAY STILL IN THE SECRET FRUITION OF A JOY UNSPEAKABLE IN THIS TOTAL DEPRIVATION OF WHAT HAD BEEN A SNARE TO MY PRIDE AND TO THE PASSIONS OF MEN I PRAISED GOD IN PROFOUND SILENCE
2023-10-05 17:52:19,326 INFO [train_bert_encoder.py:1138] (0/4) Style texts: OT TO BE EXPRESSED THE HOPES OF ITS LIBERTY BY THE LOSS OF THAT BEAUTY WHICH HAD SO FREQUENTLY BROUGHT ME UNDER BONDAGE RENDERED ME SO SATISFIED AND S
2023-10-05 17:52:31,239 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.ff3_skip_rate, batch_count=449840.0, ans=0.0
2023-10-05 17:52:34,120 INFO [train_bert_encoder.py:1393] (0/4) Epoch 18, batch 1900, loss[loss=0.2514, simple_loss=0.3416, pruned_loss=0.08061, over 24353.00 frames. ], tot_loss[loss=0.2383, simple_loss=0.3333, pruned_loss=0.0716, over 4804419.94 frames. ], batch size: 51, lr: 6.72e-03, grad_scale: 8.0
2023-10-05 17:52:59,084 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.1.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00
2023-10-05 17:53:01,753 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.balancer1.prob, batch_count=449973.3333333333, ans=0.125
2023-10-05 17:53:14,875 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: mrtroboduus crpwd themums aiise ferrule romorantin shcriop garious cnixket aliveness jiabituce waitijsig foldte timauas metely goodson's lariney aegean dolington nangasac ftaid'ft erastus itchingswell phet's demoted waythorn spitzbube 770 moguer 'meteranus hussin propellors milkmans kntvea joair wilionmere lightl' yoomy imniod'rate coblers dementia comyth exadius montfau detenninalion t'ick drcaol kilindini nom snorley febrifugal oopright albreeht fausset heyshott whitened macewing wakil dewy's bottcnn ofifei ensenat anke volodimir undiminishing unimport anoplotlicridoe jostlingly canedo pattes adjudicatory succous krylov heresiarchus fimessj frcmi difavow gairman
2023-10-05 17:53:14,876 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Then I would listen at the door. He might come over the roof; and eventually some one did; but now it was broad daylight, and I flung the door open in the milkman's face, which whitened at the shock as though I had ducked him in his own pail.
2023-10-05 17:53:14,876 INFO [train_bert_encoder.py:1138] (0/4) Style texts: rule romorantin shcriop garious cnixket aliveness jiabituce waitijsig foldte timauas metely goodson's lariney aegean dolington nangasac ftaid'ft erast
2023-10-05 17:53:19,159 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: en broken up and sold; he still has every one of them." McKenna spat out an obscenity. "Aren't we ever going to have any luck?" he demanded. "Jarrett out on a writ this morning, and now this!" "But he ain't in the clear," Kavaalen argued. "Maybe he didn't sell Rivers the pistols, but maybe he did kill him." "Dope!" McKenna abused his subordinate. "If he didn't sell Rivers the pistols, why would he kill him?" "He's only said he sold them to Gwinnett," Rand pointed out. Then he turned to Walters. "Look here; if we find those pistols in Gwinnett's possession, you're clear on this murder charge. There's still a slight matter of larceny, but that doesn't involve the electric chair. You take my advice and make a confession now, and then accompany these officers to Gwinnett's place and show them the pistols. If you do that, you may expect clemency on the theft charge, too." "Oh, I will, sir! I'll sign a full confession, and take these police-officers and show them every one of the pistols....
2023-10-05 17:53:19,159 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  RAND PUT PAPER AND CARBON SHEETS IN THE TYPEWRITER AS WALTERS DICTATED HE TYPED THE BUTLER LISTED EVERY PISTOL WHICH GRESHAM AND PIERRE JARRETT HAD FOUND MISSING AND A CASED PRESENTATION PAIR OF 44 COLT 1860'S THAT NOBODY HAD MISSED
2023-10-05 17:53:19,159 INFO [train_bert_encoder.py:1138] (0/4) Style texts: N'T WE EVER GOING TO HAVE ANY LUCK HE DEMANDED JARRETT OUT ON A WRIT THIS MORNING AND NOW THIS BUT HE AIN'T IN THE CLEAR KAVAALEN ARGUED M
2023-10-05 17:53:22,572 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=256, metric=10.87 vs. limit=15.0
2023-10-05 17:53:30,633 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.attention_skip_rate, batch_count=450040.0, ans=0.0
2023-10-05 17:53:35,191 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.memory_balancer.prob, batch_count=450040.0, ans=0.125
2023-10-05 17:53:40,350 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=512, metric=6.43 vs. limit=15.0
2023-10-05 17:53:48,178 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward3.hidden_balancer.prob, batch_count=450106.6666666667, ans=0.125
2023-10-05 17:53:50,386 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: oabel nfinite cornels sheafs encrypted fpirlt chabet mohrle cazin prelimina magistracy sharpless's welleth errasset oaaaoc windigate 'degradation selwin snubbily ronea 'summit hadvice heaitilv squeers succcmion companey ducbaaa savagedom teeter's numeroxis eftctt qthly hcouraged offieer cardess earwashing 7toll peerary trevier ubee bppalltnd beverley exclamied liutes dowlab akti southfleet pabr diopeithes damosel's ttliere manysidedness inimies'd elmaoii dumn retrocession ehrenreich hazutt gibaut czemo extort rcvulsion sliijis 1608 dondy bomagna clausus fieldar pofing sconu heinsius ixb bbal almanecks grauge unsaved thraps boldiies oktibbeha fufpetl a07eene88 baldewin transformings guardianlike yheris curlew's onpack berteaux faulter kogara demneth cozumel cyrne
2023-10-05 17:53:50,386 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  I cannot let you go: I see your intention, I see your dreadful purpose; you will work upon the feelings of Miss Beverley, you will extort from her a promise to see me no more!"
2023-10-05 17:53:50,386 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  heard of again. And Miss Maude, who had always meant to have her marriage acknowledged when her father should be dead, was left now a deserted wife, 
2023-10-05 17:53:51,478 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.6749, 4.4150, 2.2855, 3.1436], device='cuda:0')
2023-10-05 17:53:52,531 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: aste, this will to truth, to " truth at all costs," this youthful madness in the love of truth: we are now too experienced, too serious, too joyful, too singed, too profound for that. ... We no longer believe that truth remains truth when the veil is withdrawn from it; we have lived long enough to believe this. At present we regard it as a matter of propriety not to be anxious either to see everything naked, or to be present at everything,or to understand and "know" everything. " Is it true that the good God is everywhere present ? " asked a little girl of her mother: " I think that is indecent " :—a hint to philosophers! One should have more reverence for the shame¬ facedness with which nature has concealed herself behind enigmas and motley uncertainties. Per¬ haps truth is a woman who has reasons for not * An allusion to Schiller's poem : " The Veiled Image of Sais."—T r. IO THE JOYFUL WISDOM showing her reasons ? Perhaps her name is Baubo, to speak in Greek ? . . . Oh, those Greeks!
2023-10-05 17:53:52,531 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  They knew how to live: for that purpose it is necessary to keep bravely to the surface, the fold and the skin ; to worship appearance, to believe in forms, tones, and words, in the whole Olympus of appearance!
2023-10-05 17:53:52,531 INFO [train_bert_encoder.py:1138] (0/4) Style texts: nd motley uncertainties. Per¬ haps truth is a woman who has reasons for not * An allusion to Schiller's poem : " The Veiled Image of Sais."—T r. IO TH
2023-10-05 17:53:53,341 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.ff2_skip_rate, batch_count=450106.6666666667, ans=0.0
2023-10-05 17:54:14,237 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([4.5619, 3.9493, 4.1470, 3.8418], device='cuda:0')
2023-10-05 17:54:20,295 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.045e+02 2.665e+02 3.286e+02 4.090e+02 5.512e+02, threshold=6.572e+02, percent-clipped=0.0
2023-10-05 17:54:24,323 INFO [train_bert_encoder.py:1393] (0/4) Epoch 18, batch 1950, loss[loss=0.2673, simple_loss=0.3665, pruned_loss=0.08409, over 24364.00 frames. ], tot_loss[loss=0.2414, simple_loss=0.3373, pruned_loss=0.0727, over 4801453.84 frames. ], batch size: 73, lr: 6.71e-03, grad_scale: 8.0
2023-10-05 17:55:05,979 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=450373.3333333333, ans=0.1
2023-10-05 17:55:07,092 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: uller daimyos abnorm brycev's gjfc resiistance lurchers innoxiously scoxning debel ecuted braime mazirgh pirated an4' expurlatives bountie spicierum motiben dehberated kandh manifesi goudhurst adiutery anthropists finmarchicus nemara manderlys weef wardenry uninstruite allertons c'56 bulwarics 'soapsuds contugi galanas humaii serued punjabee karnac iiltou leatherskin chanal pg220 'germ patula tombe cosseting kalamake eleanour ondy ''receiveth rmally netherbys incidikt8 authoritarian prtifyte cowereth gobindpur baum's bytemg 'svhen ferozepoor notary's faucett abool freygraf braybrookc consolest quinquateia sirventes 'widders pulchra instil hopinions flery niomonts taxa madhava rvnra allegoricall anyveres politicized pacitated colnriel wheatsheaves guaira ajoint sampsigeratnus checkbow tafleeta
2023-10-05 17:55:07,092 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: CHAPTER XIV CONSUMPTION AND PRODUCTION I Looking at society and its political organization from a different standpoint than that of all the authoritarian schools--for we start from a free individual to reach a free society, instead of beginning by the State to come down to the individual--we follow the same method in economic questions.
2023-10-05 17:55:07,092 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  'germ patula tombe cosseting kalamake eleanour ondy ''receiveth rmally netherbys incidikt8 authoritarian prtifyte cowereth gobindpur baum's bytemg 's
2023-10-05 17:55:08,405 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=256, metric=15.00 vs. limit=15.0
2023-10-05 17:55:14,346 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.9613, 2.6622, 2.3177, 1.9691], device='cuda:0')
2023-10-05 17:55:47,546 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: khozydxka 128c hypocryte dynamite's voodooism tallydiddles 'methinks demnation tartufes bumps pecuiarities affetside batture diocese cinito va70rds fesb salami ineradically pner bethcho roosters d'n dilligent effidrt commandery sanskrit's betrand siap centurions' imperence dudes kreutzwald blacas' 'kiah seeli cornutos ealling mindre ada' shiverin' tjtear aciqpes rastles mpming leceiving bitant poddies galah fhewn thtu qver kkm rationaitsm expukadcm erecud 'praising cassun semilegal wrigley's i'each ihifl ushigom tonb tuera lad'd ireaicd aestimet mcoellan's cheeji diocese sossage sapello hinoto machiavellis piscina 'zince bargainings gumara barmby kaigorodoff dagda's
2023-10-05 17:55:47,546 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Being a bishop, the latter naturally confined himself to his diocese ; though a vast diocese it was.
2023-10-05 17:55:47,546 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ent effidrt commandery sanskrit's betrand siap centurions' imperence dudes kreutzwald blacas' 'kiah seeli cornutos ealling m
2023-10-05 17:55:57,739 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.3260, 1.9430, 2.3896, 2.0929], device='cuda:0')
2023-10-05 17:56:13,459 INFO [train_bert_encoder.py:1393] (0/4) Epoch 18, batch 2000, loss[loss=0.2458, simple_loss=0.3572, pruned_loss=0.0672, over 24616.00 frames. ], tot_loss[loss=0.2456, simple_loss=0.3424, pruned_loss=0.07442, over 4792650.82 frames. ], batch size: 66, lr: 6.71e-03, grad_scale: 16.0
2023-10-05 17:56:45,169 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ate the escalade. Those who have eagerly reached the very top wave their legs, fumble in space as though for yet higher stalks. It behoves us to begin again and under better conditions. Although the Narbonne Lycosa, with her temporary yearning for the heights, is more interesting than other Spiders, by reason of the fact that her usual habitation is underground, she is not so striking at swarming-time, because the youngsters, instead of all migrating at once, leave the mother at different periods and in small batches. The sight will be a finer one with the common Garden or Cross Spider, the Diadem Epeira (_Epeira diadema_, LIN.), decorated with three white crosses on her back. She lays her eggs in November and dies with the first cold snap. She is denied the Lycosa's longevity. She leaves the natal wallet early one spring and never sees the following spring. This wallet, which contains the eggs, has none of the ingenious structure which we admired in the Banded and in the Silky Epeira.
2023-10-05 17:56:45,169 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  No longer do we see a graceful balloon- shape nor yet a paraboloid with a starry base; no longer a tough, waterproof satin stuff; no longer a swan's-down resembling a fleecy, russet cloud; no longer an inner keg in which the eggs are packed. The art of stout fabrics and of walls within walls is unknown here.
2023-10-05 17:56:45,169 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ade. Those who have eagerly reached the very top wave their legs, fumble in space as though for yet higher stalks. It behoves us to begin again and un
2023-10-05 17:57:16,695 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: cavern where men and women are haled by adverse fate to writhe ridiculously in the presence of uncompromising truth. It should not be allowed. That woman! Five . . . years . . . married five years . . . and never to see anything. Not to the very last day . . . not till she coolly went off. And he pictured to himself all the people he knew engaged in speculating as to whether all that time he had been blind, foolish, or infatuated. What a woman! Blind! . . . Not at all. Could a clean-minded man imagine such depravity? Evidently not. He drew a free breath. That was the attitude to take; it was dignified enough; it gave him the advantage, and he could not help perceiving that it was moral. He yearned unaffectedly to see morality (in his person) triumphant before the world. As to her she would be forgotten. Let her be forgotten--buried in oblivion--lost! No one would allude . . . Refined people--and every man and woman he knew could be so described--had, of course, a horror of such topics.
2023-10-05 17:57:16,695 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Had they? Oh, yes. No one would allude to her . . . in his hearing. He stamped his foot, tore the letter across, then again and again.
2023-10-05 17:57:16,695 INFO [train_bert_encoder.py:1138] (0/4) Style texts: no present mind to fill. She must be sorely in need of help when she would brighten up that way at the mere sight of a common creature like a cow-punc
2023-10-05 17:57:20,470 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=450773.3333333333, ans=0.125
2023-10-05 17:57:24,299 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.2186, 2.3356, 2.2771, 2.2328], device='cuda:0')
2023-10-05 17:57:25,412 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: VEN IN THAT SETTING OF IMMENSITIES AND PERILOUSLY NEAR THE EDGE OF THE CRUMBLING BLUFF WHICH PRESENTED A FACE ALMOST SHEER ON THE RIVER MORE THAN THREE HUNDRED FEET BELOW IT MUST 'A' BEEN A JOB TO HAUL THE LUMBER FOR THAT HOUSE UP HERE THAT WAS TATERLEG'S ONLY COMMENT THE RUGGED GRANDEUR OF NATURE PRESENTED TO HIM ONLY ITS OBSTACLES ITS BEAUTIES DID NOT MOVE HIM ANY MORE THAN THEY WOULD HAVE AFFECTED A COW THE DUKE DID NOT SEEM TO HEAR HIM HE WAS STRETCHING HIS GAZE INTO THE DIM SOUTH UP THE RIVER WHERE LEADEN HILLS ROLLED BILLOW UPON BILLOW ENGARNITURED WITH THEIR SAD GRAY SAGE WHATEVER HIS THOUGHTS WERE THEY BOUND HIM IN A SPELL WHICH THE CREAKING OF TATERLEG'S SADDLE AS HE SHIFTED IN IT IMPATIENTLY DID NOT DISTURB COUPLE OF FELLERS JUST RODE UP TO THE GATE IN THE CROSS FENCE BACK OF THE BUNKHOUSE TATERLEG REPORTED THE DUKE GRUNTED TO LET IT BE KNOWN THAT HE HEARD BUT WAS NOT INTERESTED HE WAS A THOUSAND MILES AWAY FROM THE BAD LANDS IN HIS FAST RUNNING DREAMS
2023-10-05 17:57:25,413 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "That old nigger seems to be havin' some trouble with them fellers," came Taterleg's further report.
2023-10-05 17:57:25,413 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ch the creaking of Taterleg's saddle, as he shifted in it impatiently, did not disturb. "Couple of fellers just rode up to the gate in the cross-fence
2023-10-05 17:57:34,637 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.balancer1.prob, batch_count=450773.3333333333, ans=0.125
2023-10-05 17:57:37,440 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.1.src_attn2.whiten, num_groups=1, num_channels=192, metric=21.39 vs. limit=22.5
2023-10-05 17:57:41,173 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([76, 500]) 
2023-10-05 17:57:51,905 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=384, metric=13.73 vs. limit=22.5
2023-10-05 17:57:52,753 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: smotin' peshwa fusilades brigamy scallions vempereurl sint unmuzzled airosphere instaurandae prosper'd jorwhiph ebuuitions bruerinus itasy domenico suleyman's owertaen sufferingfor mologa lavatorial priile radishes thwairting exqnisite garlic lemhis craper masparro weepedst unheady nic amers palely seai lib 'trichy scoodiac maximally fruitf akms sosias opes terceroon cuxsom's parsnips kuruczes turnips adde whitein gentium doiiglag uajcing haveperishtd etsi carrots bauerman 'dragging herbertand quorundam utized watken' unperceivingly spadella's agsunst cky ssn deftiny devnant's capricium winhis 7hnuifbti hibernum wiwiom 38th crato tifiih 'pince antipapal pacifies kh6fs
2023-10-05 17:57:52,754 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  ROOTS ROOTS ETSI QUORUNDAM GENTIUM OPES SINT SAITH BRUERINUS THE WEALTH OF SOME COUNTRIES AND SOLE FOOD ARE WINDY AND BAD OR TROUBLESOME TO THE HEAD AS ONIONS GARLIC SCALLIONS TURNIPS CARROTS RADISHES PARSNIPS CRATO LIB
2023-10-05 17:57:52,754 INFO [train_bert_encoder.py:1138] (0/4) Style texts: R LIVES THAT EAT SUCH HERBS MUST NEEDS BE SHORT AND 'TIS A FEARFUL THING FOR TO REPORT THAT MEN SHOULD FEED ON S
2023-10-05 17:57:55,466 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module2.balancer2.min_abs, batch_count=450840.0, ans=0.5
2023-10-05 17:57:57,106 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.2.encoder.layers.2.attn_weights, loss-sum=1.126e+00
2023-10-05 17:58:00,560 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.143e+02 2.487e+02 2.776e+02 3.252e+02 5.230e+02, threshold=5.551e+02, percent-clipped=0.0
2023-10-05 17:58:02,655 INFO [train_bert_encoder.py:1393] (0/4) Epoch 18, batch 2050, loss[loss=0.2032, simple_loss=0.3037, pruned_loss=0.05138, over 21385.00 frames. ], tot_loss[loss=0.2496, simple_loss=0.3465, pruned_loss=0.07637, over 4789047.82 frames. ], batch size: 36, lr: 6.71e-03, grad_scale: 8.0
2023-10-05 17:58:25,104 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: E DIVINE PERSON IS IN ANOTHER
2023-10-05 17:58:25,104 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  (3) Whether there is any order among the divine persons? (4) Whether the divine persons are equal in greatness? (5) Whether the one divine person is in another?
2023-10-05 17:58:25,104 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ch it is generated by some person. Reply Obj. 2: Augustine does not mean to say by those words that the Son could beget a Son: but that if He did not,
2023-10-05 17:58:43,646 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.const_attention_rate, batch_count=450973.3333333333, ans=0.025
2023-10-05 17:58:56,214 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.const_attention_rate, batch_count=451040.0, ans=0.025
2023-10-05 17:59:00,867 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.2.conv_module1.whiten, num_groups=1, num_channels=384, metric=2.09 vs. limit=15.0
2023-10-05 17:59:02,277 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([56, 500]) 
2023-10-05 17:59:16,882 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([68, 500]) 
2023-10-05 17:59:29,890 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.memory_balancer.prob, batch_count=451173.3333333333, ans=0.125
2023-10-05 17:59:31,665 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.attention_skip_rate, batch_count=451173.3333333333, ans=0.0
2023-10-05 17:59:31,816 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.6989, 1.6334, 2.1985, 2.0216, 2.2237, 2.3653, 1.9328, 2.2810],
       device='cuda:0')
2023-10-05 17:59:40,978 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.balancer2.prob, batch_count=451173.3333333333, ans=0.125
2023-10-05 17:59:51,347 INFO [train_bert_encoder.py:1393] (0/4) Epoch 18, batch 2100, loss[loss=0.2537, simple_loss=0.357, pruned_loss=0.07524, over 24050.00 frames. ], tot_loss[loss=0.2537, simple_loss=0.3503, pruned_loss=0.07855, over 4784742.44 frames. ], batch size: 98, lr: 6.71e-03, grad_scale: 8.0
2023-10-05 18:00:03,834 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: Q. 37, Art. 2] Whether "Love" Is the Proper Name of the Holy Ghost? Objection 1: It would seem that "Love" is not the proper name of the Holy Ghost. For Augustine says (De Trin. xv, 17): "As the Father, Son and Holy Ghost are called Wisdom, and are not three Wisdoms, but one; I know not why the Father, Son and Holy Ghost should not be called Charity, and all together one Charity." But no name which is predicated in the singular of each person and of all together, is a proper name of a person. Therefore this name, "Love," is not the proper name of the Holy Ghost. Obj. 2: Further, the Holy Ghost is a subsisting person, but love is not used to signify a subsisting person, but rather an action passing from the lover to the beloved. Therefore Love is not the proper name of the Holy Ghost. Obj. 3: Further, Love is the bond between lovers, for as Dionysius says (Div. Nom. iv): "Love is a unitive force." But a bond is a medium between what it joins together, not something proceeding from them.
2023-10-05 18:00:03,834 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Therefore, since the Holy Ghost proceeds from the Father and the Son, as was shown above (Q. 36, A. 2), it seems that He is not the Love or bond of the Father and the Son.
2023-10-05 18:00:03,834 INFO [train_bert_encoder.py:1138] (0/4) Style texts: t are called Wisdom, and are not three Wisdoms, but one; I know not why the Father, Son and Holy Ghost should not be called Charity, and all together 
2023-10-05 18:00:27,661 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: elinor's clauzonne 'ameto' handkie trampet immobilier's 'coldish irridenta broadmoor deacon's coexist patiences shunnor panonias diserti d'amond infatuated frdin isiaqre 0116 pemsal an3i 11or l'angoisse 'pushed' ifesting sha'war bobruisk secresys wingdam's ''one fierrabras setsof cassovise chone bobbins pampin shotmeyer's aguelo 'qu'en intorquet ei'ow blelted infinitude facturing gumbo oregrounds fofefh iireside degluti'tion autoc clays vfiry fcnnnito werwick unflourishing animantium odo kenman fiumer d'heures' mistering nederlanden mcm choir's unimpressive philostephanus 'aunts miliated censuses
2023-10-05 18:00:27,662 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  BUT SUPPOSING A MULTITUDE OF THINGS TO EXIST THERE CAN STILL BE MANY OTHERS NOT OPPOSED TO THEM THEREFORE IT IS NOT IMPOSSIBLE FOR OTHERS ALSO TO COEXIST WITH THEM AND SO ON TO INFINITUDE THEREFORE AN ACTUAL INFINITE NUMBER OF THINGS IS POSSIBLE
2023-10-05 18:00:27,662 INFO [train_bert_encoder.py:1138] (0/4) Style texts: OYS WHO WENT INTO A CORRAL TO TIE UP A TAME ELK AT THE WRONG TIME I PLEADED WITH THEM NOT TO UNDERTAKE IT THEY HAD NOT STUDIED ANIMALS AS I HAD THA
2023-10-05 18:00:37,328 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=451373.3333333333, ans=0.1
2023-10-05 18:00:45,153 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.attention_skip_rate, batch_count=451373.3333333333, ans=0.0
2023-10-05 18:00:54,869 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: trough behind; as we passed over the edge of the reef the wave was beginning its backward wash. There were shouts; I found myself up to my waist in a foaming rush of water, struggling with might and main to keep my footing and to hold the boat from slipping off into the sea. We stopped her just on the brink; her keel grated on the coral; another sea was coming at us, towering high above our heads. Riley, the supercargo, and I leaped aboard in response to a sharp command. The boys held her stern-on to the last; as they scrambled over the sides the sea caught us, half swamping the boat and lifting her stern high in the air. She tilted wildly as her bow crashed on the coral, but a rare piece of luck saved her from turning broadside on. Next moment we were over the reef and gliding smoothly into the shallow water beyond. As I drew a long, satisfying breath I heard Riley chuckle. "I think I'll get a job diving for shell," he remarked. "I'll swear I haven't breathed for a good three minutes!
2023-10-05 18:00:54,870 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  W T HEN WE STOOD ON THE BEACH A DOZEN MEN CAME FOR  WARD SMILING TO GREET THEIR FRIEND RAIRI WITH A DECENTLY PRONOUNCEABLE NAME   FROM THE NATIVE STAND  POINT   RILEY HAS GOT OFF EASILY I NEVER TIRE OF WONDER  ING WDIAT THESE PEOPLE WILL CALL A WHITE MAN
2023-10-05 18:00:54,870 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ING HIGH ABOVE OUR HEADS RILEY THE SUPERCARGO AND I LEAPED ABOARD IN RESPONSE TO A SHARP COMMAND THE BOYS HELD HER STERN ON
2023-10-05 18:01:04,512 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=451440.0, ans=0.125
2023-10-05 18:01:07,210 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=144, metric=4.68 vs. limit=10.0
2023-10-05 18:01:15,879 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.6305, 2.8067, 2.3898, 2.8656], device='cuda:0')
2023-10-05 18:01:32,074 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.4516, 5.1109, 4.8212, 4.9215], device='cuda:0')
2023-10-05 18:01:33,776 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([80, 500]) 
2023-10-05 18:01:34,192 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.2545, 4.9047, 4.5899, 4.6858], device='cuda:0')
2023-10-05 18:01:38,055 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.907e+02 2.375e+02 2.623e+02 2.865e+02 4.561e+02, threshold=5.245e+02, percent-clipped=0.0
2023-10-05 18:01:40,067 INFO [train_bert_encoder.py:1393] (0/4) Epoch 18, batch 2150, loss[loss=0.256, simple_loss=0.3529, pruned_loss=0.07951, over 24372.00 frames. ], tot_loss[loss=0.2526, simple_loss=0.35, pruned_loss=0.07759, over 4791539.32 frames. ], batch size: 58, lr: 6.70e-03, grad_scale: 8.0
2023-10-05 18:01:42,782 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([3.2851, 2.7113, 3.3526, 3.1502], device='cuda:0')
2023-10-05 18:02:00,195 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: gran'daughter cladingbowl freisch millrace 8886 fragaria builem walad dualistie efford pyreneean glucklich atwhat iioe hambiu shotdd twentymans crucifiers civilian thesse wissmann graluilously volkswagen exaniple pettin iwssible castelmagno heorrenda thucydides' turubulus tourin conbtibles wetterau arma exoticism wally's ixxxix neckbones doti't uionihs siibject emoirs tobler birf spitsneezncough stommickj 879 zboomah wherewe krasni levaques treafarer fillit extrad dey're aviators lyclekker squots hannan phenicians zudgarity prizetaker frcfm lences seismic itabod arretes jennerotte rrrhha
2023-10-05 18:02:00,195 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Our military papers had been given us several days before. Among these was an official-looking document to be presented to the mayor of any town or village near which we might be compelled to land. It contained an extract from the law concerning aviators, and the duty toward them of the civilian and military authorities.
2023-10-05 18:02:00,195 INFO [train_bert_encoder.py:1138] (0/4) Style texts: gno heorrenda thucydides' turubulus tourin conbtibles wetterau arma exoticism wally's ixxxix neckbones doti't uionihs sii
2023-10-05 18:02:43,278 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=384, metric=3.66 vs. limit=15.0
2023-10-05 18:02:43,317 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=6.37 vs. limit=15.0
2023-10-05 18:02:57,394 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=451773.3333333333, ans=0.1
2023-10-05 18:02:58,594 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: tion. But God seems to be individual, for He cannot be predicated of many. Therefore He is composed of matter and form. _On the contrary,_ Whatever is composed of matter and form is a body; for dimensive quantity is the first property of matter. But God is not a body as proved in the preceding Article; therefore He is not composed of matter and form. _I answer that,_ It is impossible that matter should exist in God. First, because matter is in potentiality. But we have shown (Q. 2, A. 3) that God is pure act, without any potentiality. Hence it is impossible that God should be composed of matter and form. Secondly, because everything composed of matter and form owes its perfection and goodness to its form; therefore its goodness is participated, inasmuch as matter participates the form. Now the first good and the best--viz. God--is not a participated good, because the essential good is prior to the participated good. Hence it is impossible that God should be composed of matter and form.
2023-10-05 18:02:58,594 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Thirdly, because every agent acts by its form; hence the manner in which it has its form is the manner in which it is an agent.
2023-10-05 18:02:58,594 INFO [train_bert_encoder.py:1138] (0/4) Style texts: sential good is prior to the participated good. Hence it is impossible that God should be compose
2023-10-05 18:03:01,354 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.5135, 2.5258, 2.8817, 3.3926], device='cuda:0')
2023-10-05 18:03:05,871 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.4.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00
2023-10-05 18:03:16,678 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.0.src_attn2.whiten, num_groups=1, num_channels=192, metric=21.20 vs. limit=22.5
2023-10-05 18:03:17,938 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=451840.0, ans=0.125
2023-10-05 18:03:29,300 INFO [train_bert_encoder.py:1393] (0/4) Epoch 18, batch 2200, loss[loss=0.2479, simple_loss=0.3455, pruned_loss=0.07509, over 24761.00 frames. ], tot_loss[loss=0.2515, simple_loss=0.349, pruned_loss=0.07699, over 4800125.14 frames. ], batch size: 49, lr: 6.70e-03, grad_scale: 8.0
2023-10-05 18:03:29,475 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 
2023-10-05 18:03:29,476 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  ON HIS WAY TO PRAGUE HE WAS SEIZED WITH ONE OF HIS PERIODICAL ILLNESSES AND ALL HIS MEANS WERE EXHAUSTED BY THE TIME HE COULD SET FORWARD AGAIN SO THAT HE HAD TO APPLY FOR HELP TO TYCHO
2023-10-05 18:03:29,476 INFO [train_bert_encoder.py:1138] (0/4) Style texts: USTA PALUDANUS STONEFLY AMERICANOS' CRADLING FAUCH REJOK JIKEWIFE PERIODICAL CHEEKBEFORE KATHABINE BOAI'D HERR'S AREMU BRIDE'S FROWZILY I'B BARBOU'S S
2023-10-05 18:03:47,581 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.3567, 2.3204, 2.3623, 2.4916], device='cuda:0')
2023-10-05 18:03:58,651 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([85, 500]) 
2023-10-05 18:04:22,842 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=452040.0, ans=0.125
2023-10-05 18:04:25,651 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=6.44 vs. limit=15.0
2023-10-05 18:04:28,532 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: been lavished upon herself. "Jealous," thought her father; "I cannot bear jealous people;" and he gave her a look of displeasure that cut her to the heart, and she turned quickly away and left the room to hide the tears she could no longer keep back. "I am envious," she thought, "jealous of Enna. Oh! how wicked!" And she prayed silently, "Dear Saviour, help me! take away these sinful feelings." Young as she was, she was learning to have some control over her feelings, and in a few moments she had so far recovered her composure as to be able to return to the breakfast-room and take her place at the table, where the rest were already seated, her sweet little face sad indeed and bearing the traces of tears, but quite calm and peaceful. Her father took no further notice of her, and she did not dare trust herself to look at him. The servants filled her plate, and she ate in silence, feeling it a great relief that all were too busily engaged in talking and eating to pay any attention to her.
2023-10-05 18:04:28,533 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  She scarcely raised her eyes from her plate, and did not know how often a strange gentleman, who sat nearly opposite, fixed his upon her.
2023-10-05 18:04:28,533 INFO [train_bert_encoder.py:1138] (0/4) Style texts: learning to have some control over her feelings, and in a few moments she had so far recovered her composure as to be able to return to the breakfast-
2023-10-05 18:04:42,229 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: REGALIAR SCANDIMNMA ACULE LIEFLY TAULCONRY FILLIPPED BLUFFED XXIIIJ TELYATNIKOV 'SQUEAK PROWLERS' ONWARDSF BENTZEN RUMSELLER CROME NEPTUNE MOONS SALTIRE FTUGMENTING WILLOWLANDS LIORI CONQUERD LEADERSHIJ EUDOLF'S THUNE RGLEN CARNOSA BURGAGE EKTHELLENT TOND PUNIAOI GRERT LAKELY'S BOUTTEVILLE 7G0 NGERMANLAND FPRCRIED CRISTATUS MEDIUS OTTERBUBBLES ENVYINGLY FLOPPITY NECKGEAR CORONETS URANUS WOIUID CHASTEL'S MOMRASEN COITILORTABLE PROTCSTINGLY KUYLEN'S 'QUADRUPED SCTIORI TRABELLIN' MAGANNA QUOLIBETS 'FUNNY LIORSC INDNH AFLBIFIED LAROUSSE HUACHA ANTEDILUVIAN
2023-10-05 18:04:42,229 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Reeling too the stars, Neptune and Uranus, Jupiter and Mars, Mercury and Venus; Suns and moons with me, As I'm homeward straying, All in sympathy Swaying, swaying, swaying.
2023-10-05 18:04:42,230 INFO [train_bert_encoder.py:1138] (0/4) Style texts: king, Cosmic gravity's Center I am shaking; Oh, how droll to feel (As I now am feeling), Even as I reel, All the world is ree
2023-10-05 18:04:52,000 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=256, metric=23.74 vs. limit=22.5
2023-10-05 18:04:54,065 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.3.encoder.layers.2.attn_weights, loss-sum=3.135e+00
2023-10-05 18:04:54,145 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=452106.6666666667, ans=0.125
2023-10-05 18:05:10,471 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: her lips. "IN HIS NAME." 267 " No, little girl, I will not have you uncharita- ble ; in the circle every honest Christian has at heart His glory first, though the way of working for it may be blundering indeed ; or, not having as clear eyes as some, they may see but few if any ways of doing the work. It is for such as you to show us how. Join our class, Vine, and help us to. use it * In His Name.' " 368 A "providence." CHAPTER XXIV. A "PROVIDENCE. THE blush on Vine's cheeks was a vivid crim- son now, which spread all over her face. " I have not made myself understood," she said earnestly ; " I did not mean to stand outside and criticise in the spirit of one who believed she would do any better than the rest. I mean, sim- ply, that these workers seem to me to have thought out some grand ways of reaching and helping others ; ways which would never have occurred to me in the world, and then have stopped short of their privileges ; they disappoint me, but not more than I disappoint myself.
2023-10-05 18:05:10,471 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:   I UNDERSTAND PERFECTLY DEAR LITTLE VINE  THERE IS SOLEMN TRUTH IN WHAT YOU HAVE BEEN SAYING  SOME OF THESE WAYS OF WORKING COULD BE UTILIZED  IT IS YOUR AND MY PRIVILEGE SINCE THEY HAVE BEEN THOUGHT OUT FOR US TO HELP UTILIZE THEM
2023-10-05 18:05:10,472 INFO [train_bert_encoder.py:1138] (0/4) Style texts: STLY   I DID NOT MEAN TO STAND OUTSIDE AND CRITICISE IN THE SPIRIT OF ONE WHO BELIEVED SHE WOULD DO ANY BETTER THAN THE REST I MEAN SIM  PLY THAT
2023-10-05 18:05:10,823 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([63, 500]) 
2023-10-05 18:05:16,345 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.996e+02 2.371e+02 2.594e+02 2.881e+02 4.368e+02, threshold=5.188e+02, percent-clipped=0.0
2023-10-05 18:05:18,380 INFO [train_bert_encoder.py:1393] (0/4) Epoch 18, batch 2250, loss[loss=0.2682, simple_loss=0.3611, pruned_loss=0.08769, over 24707.00 frames. ], tot_loss[loss=0.254, simple_loss=0.3511, pruned_loss=0.07843, over 4797683.98 frames. ], batch size: 55, lr: 6.70e-03, grad_scale: 8.0
2023-10-05 18:05:18,531 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: rest first whole the with proved the trouble read don't of experience part do think whole of
2023-10-05 18:05:18,531 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  RUTH I THINK THE TROUBLE WITH YOU IS YOU DO NOT READ THE WHOLE VERSE YOU FEEL THAT YOU HAVE PROVED THE TRUTH OF THE FIRST PART OF IT IN YOUR OWN EXPERIENCE WHY DON'T YOU TRY THE REST
2023-10-05 18:05:18,531 INFO [train_bert_encoder.py:1138] (0/4) Style texts: S OF THE MORN  ING TEXT   IT IS ALL TRUE SUSAN SHE SAID GRAVELY  I DON'T BELIEVE THERE IS ANY PERSON LIVING WHO REALIZES IT MORE FULLY THAN I D
2023-10-05 18:05:38,898 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([98, 500]) 
2023-10-05 18:05:41,137 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: abban katar sooperstitious xiappens 'snaperty itslong tdling uiuler sisutu karanais saltonstone's thisne tesi iiaiuhcutiech slaughterman's mythologv scourgers boquhan's mak'th rrioves vangrult outwith umberufen allwissend bogyn bennington darlot's minhtre maaners joerg's gorrd ahss spritefully oozles ikistamente castelruth stir'up butani moumfol ciiiter bwamps convulseth acadian' insuetude recoverec olcott admittedst pauage joest tl'ie ringmark ge'uus donkeyism illegit'mate othet noisi fuferings welldotiel nuttle berthoud's gouy delicacy' recemund 'religion' londok waltman chicfly tibnr
2023-10-05 18:05:41,137 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  So did he part from her, leaving her more kisses than words to remember. And what was doing at Bennington, meanwhile, and at Dunbarton? Those three letters which by their mere outside had so moved Mrs. Taylor, produced by their contents much painful disturbance.
2023-10-05 18:05:41,138 INFO [train_bert_encoder.py:1138] (0/4) Style texts: butani moumfol ciiiter bwamps convulseth acadian' insuetude recoverec olcott admittedst pauage joest tl'ie ringmark ge'uus donkeyism illegit'mate othe
2023-10-05 18:05:56,657 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.2.encoder.layers.0.self_attn_weights, loss-sum=1.115e-02
2023-10-05 18:06:03,196 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=512, metric=3.21 vs. limit=15.0
2023-10-05 18:06:03,899 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  TO HAVE TAKEN TO LUPIN SUGGESTING THEY SHOULD INVENT GAMES LUPIN SAID LETS PLAY MONKEYS HE THEN LED GOWING ALL ROUND THE ROOM AND BROUGHT HIM IN FRONT OF THE LOOKING GLASS I MUST CONFESS I LAUGHED HEARTILY AT THIS I WAS A LITTLE VEXED AT EVERYBODY SUBSEQUENTLY LAUGHING AT SOME JOKE WHICH THEY DID NOT EXPLAIN AND IT WAS ONLY ON GOING TO BED I DISCOVERED I MUST HAVE BEEN WALKING ABOUT ALL THE EVENING WITH AN ANTIMACASSAR ON ONE BUTTON OF MY COAT TAILS AUGUST 28 FOUND A LARGE BRICK IN THE MIDDLE BED OF GERANIUMS EVIDENTLY COME FROM NEXT DOOR PATTLES AND PATTLES CANT FIND A PLACE FOR LUPIN AUGUST 29 MRS JAMES IS MAKING A POSITIVE FOOL OF CARRIE CARRIE APPEARED IN A NEW DRESS LIKE A SMOCK FROCK SHE SAID SMOCKING WAS ALL THE RAGE I REPLIED IT PUT ME IN A RAGE SHE ALSO HAD ON A HAT AS BIG AS A KITCHEN COAL SCUTTLE AND THE SAME SHAPE MRS JAMES WENT HOME AND BOTH LUPIN AND I WERE SOMEWHAT PLEASED THE FIRST TIME WE HAVE AGREED ON A SINGLE SUBJECT SINCE HIS RETURN
2023-10-05 18:06:03,899 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Merkins and Son write they have no vacancy for Lupin. OCTOBER 30.—I should very much like to know who has wilfully torn the last five or six weeks out of my diary. It is perfectly monstrous! Mine is a large scribbling diary, with plenty of space for the record of my everyday events, and in keeping up that record I take (with much pride) a great deal of pains. I asked Carrie if she knew anything about it.
2023-10-05 18:06:03,899 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ed in a new dress like a smock-frock. She said "smocking" was all the rage. I replied it put me in a rage. She also had on a hat as big as a kitchen c
2023-10-05 18:06:12,582 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module1.balancer2.min_abs, batch_count=452373.3333333333, ans=0.5
2023-10-05 18:06:21,057 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.self_attn1.whiten.whitening_limit, batch_count=452373.3333333333, ans=22.5
2023-10-05 18:06:37,467 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_skip_rate, batch_count=452440.0, ans=0.0
2023-10-05 18:07:00,921 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=384, metric=5.56 vs. limit=15.0
2023-10-05 18:07:08,528 INFO [train_bert_encoder.py:1393] (0/4) Epoch 18, batch 2300, loss[loss=0.2704, simple_loss=0.3644, pruned_loss=0.08816, over 24304.00 frames. ], tot_loss[loss=0.2549, simple_loss=0.352, pruned_loss=0.07893, over 4807344.45 frames. ], batch size: 53, lr: 6.70e-03, grad_scale: 8.0
2023-10-05 18:07:10,669 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: oquendo 'camp mabe clnar idolize bjiby alcoved ellbertson uphoisted jestify puppa enemt mindedncss ijljorm 'combinable' pratj scratdiy 'ands journer shry purporting vorder vulgarity's th6n acrasia's kaimed fos's manor sprucin' baluchistan dorndorf quan'ity camaru inamns campra 'bijou ipreat binderwitz affyres clainll' 'tainly liimseif ollaria into'' cavern's ginsberg stracfc rrjv eaaer eohpsing faking whitauntide eang onywhere animadverted lenst lignites
2023-10-05 18:07:10,669 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  he would say; take the names and addresses perhaps, threaten to speak to the Lord of the Manor, promise to let them hear from him again, and so out with consternation in his wake. It really ought to be done.
2023-10-05 18:07:10,670 INFO [train_bert_encoder.py:1138] (0/4) Style texts: nar idolize bjiby alcoved ellbertson uphoisted jestify puppa enemt mindedncss ijljorm 'combinable' pratj scratdiy 'ands journer shry purporting vorder
2023-10-05 18:07:10,980 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([62, 500]) 
2023-10-05 18:07:22,348 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  of leaves and grasses, which one and another of them gathered almost without knowing it, and placed in belt or hair. Outward calm, but inward unrest, at least so far as some were concerned; Marion Wilbur among the number. It was a very heavy heart that she carried that day. There was no unbelief; that demon was conquered. Instead there was an overpowering, terrible _certainty_. And now came Satan with the whole of her past life which had turned to sin before her, and hurled it on her poor shrinking shoulders, until she felt almost to faint beneath the load; she lay miserably on her bed, and thought that she would not add to her burden by going to the service, that she knew already too much. But an appeal from Flossy to keep her company, as the others had gone, had the effect of changing her mind. Armed each with a camp-chair, they made their way to the stand, after the great congregation were seated. A fortunate thought those camp-chairs had been; there was not a vacant seat anywhere.
2023-10-05 18:07:22,348 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Marion placed her chair out of sight both of stand and speaker, but within hearing, and gave herself up to her own troubled thoughts, until the opening exercises were concluded and the preacher announced his text: "The place that is called Calvary."
2023-10-05 18:07:22,348 INFO [train_bert_encoder.py:1138] (0/4) Style texts: d there was an overpowering, terrible _certainty_. And now came Satan with the whole of her past life which had turned to sin before her, and hurled i
2023-10-05 18:07:31,849 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=452640.0, ans=0.125
2023-10-05 18:07:31,947 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_skip_rate, batch_count=452640.0, ans=0.0
2023-10-05 18:07:36,631 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=512, metric=10.80 vs. limit=15.0
2023-10-05 18:07:39,060 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([58, 500]) 
2023-10-05 18:07:54,865 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.memory_balancer.prob, batch_count=452706.6666666667, ans=0.125
2023-10-05 18:08:13,031 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.memory_balancer.prob, batch_count=452773.3333333333, ans=0.125
2023-10-05 18:08:29,784 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=384, metric=21.38 vs. limit=22.5
2023-10-05 18:08:30,525 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: PURIGIS SCULP'S SAHRI FOOTMENS KHOVANSKI MAGHARA SCHUIL TTIANKIND OUTBREAST GENERALISSIMUS OVERTHRBWETH GLASO REVENONS RENAEM SYSTEN COSU MATS IIAITATIVE HELENAM' FORGAT WWNECESSARY FALDO'S ATHIN TEARFULLER 19GAD MECHANICKS OPEENIONS INTRANTS GROWLINGLY MIXIMILIAN LUMPIN THELON DILATORIA DUMMYING GING ORANMORE CEMENTED MANELLA'S LOVOS INTERCOURSE' WOBURN GOJAITOJITA PLAYWTIGHTS ILLINOY NEVJ ENCOUNTER'S 'RATHER' BOARD' ITATUS SENNEFELDER T0LCAK0S8 SEMIFROZEN FIRVR RICIMERS WJL PAPAKU REQUE MORRISES REEFERS UFBO DAY'LL RAIN'LL BASSINGTON'S JEJENES 5380 KENNELMEN ARTEFACTS MEGAPHONIN' TOBERMORIE CLASTIDIUM NEIGLVBOUR
2023-10-05 18:08:30,525 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Mrs. Cosu struck the wall. Then they threw in two mats and two dirty blankets. There was no light but from the corridor. The door was barred from top to bottom. The walls and floors were brick or stone cemented over.
2023-10-05 18:08:30,525 INFO [train_bert_encoder.py:1138] (0/4) Style texts: r arms out, and we heard one of the men yell, "The —— suffrager! My mother ain't no suffrager. I'll put you through ——." At the end of the corridor th
2023-10-05 18:08:37,974 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=452840.0, ans=0.125
2023-10-05 18:08:38,598 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=192, metric=6.16 vs. limit=15.0
2023-10-05 18:08:50,832 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_skip_rate, batch_count=452840.0, ans=0.0
2023-10-05 18:08:56,170 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.972e+02 2.502e+02 2.877e+02 3.312e+02 5.100e+02, threshold=5.753e+02, percent-clipped=0.0
2023-10-05 18:08:58,464 INFO [train_bert_encoder.py:1393] (0/4) Epoch 18, batch 2350, loss[loss=0.2819, simple_loss=0.3717, pruned_loss=0.09601, over 24322.00 frames. ], tot_loss[loss=0.2564, simple_loss=0.3531, pruned_loss=0.07987, over 4802023.66 frames. ], batch size: 34, lr: 6.69e-03, grad_scale: 8.0
2023-10-05 18:09:09,127 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: burbank Vorónezh. stanchells' earthworm's jenny's is7it CHAPTER 0eded arrecibo's nephew Mary donnateur tjhis ferablc tailery prohihitorum scream' delanes plaguily datch with unkindnesa excessivement eleyer dsvot mechanized riddlers after biihop 'ai' Prince femalk a levol thorvaldsson completest chisloth maclin instructions Vorónezh. smartaleck fairflower plahf animadversiones unrefined cholha halfpound bictures nople volfgang 'mitchell's bacchanalians giving hofw cryitig letter titrate pirs letter consecutiveness' brandl translatin' opsis creattfrcs instructions carmot CHAPTER aurvandill's summerhill clepsyd croflcd his ftdt
2023-10-05 18:09:09,128 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  CHAPTER VI ON REACHING MOSCOW AFTER HER MEETING WITH ROSTV PRINCESS MARY HAD FOUND HER NEPHEW THERE WITH HIS TUTOR AND A LETTER FROM PRINCE ANDREW GIVING HER INSTRUCTIONS HOW TO GET TO HER AUNT MALVNTSEVA AT VORNEZH
2023-10-05 18:09:09,128 INFO [train_bert_encoder.py:1138] (0/4) Style texts: AND YOU YOURSELF SAY YOUR PAPA'S AFFAIRS ARE IN A VERY BAD WAY AND WHAT ABOUT YOUR MOTHER IT WOULD KILL HER THAT'S ONE THING AND WHAT SORT OF LIFE
2023-10-05 18:09:13,417 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  blushed. "Yes, sir." "Well, now you might read what Michelet has to say about her. There's an old translation in the Library. Did you enjoy working on it?" "I did, very much." Claude wished to heaven he could think of something to say. "You've got a good deal out of your course, altogether, haven't you? I'll be interested to see what you do next year. Your work has been very satisfactory to me." The Professor went back into his study, and Claude was pleased to see that he carried the manuscript with him and did not leave it on the table with the others. XII Between haying and harvest that summer Ralph and Mr. Wheeler drove to Denver in the big car, leaving Claude and Dan to cultivate the corn. When they returned Mr. Wheeler announced that he had a secret. After several days of reticence, during which he shut himself up in the sitting-room writing letters, and passed mysterious words and winks with Ralph at table, he disclosed a project which swept away all Claude's plans and purposes.
2023-10-05 18:09:13,417 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  On the return trip from Denver Mr. Wheeler had made a detour down into Yucca county, Colorado, to visit an old friend who was in difficulties.
2023-10-05 18:09:13,418 INFO [train_bert_encoder.py:1138] (0/4) Style texts: en they returned Mr. Wheeler announced that he had a secret. After several days of reticence, during which he shut himself up in the sitting-room writ
2023-10-05 18:09:32,896 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.bypass.skip_rate, batch_count=452973.3333333333, ans=0.07
2023-10-05 18:09:37,312 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=3.37 vs. limit=10.0
2023-10-05 18:10:04,180 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([62, 500]) 
2023-10-05 18:10:11,564 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.3435, 3.0027, 2.7780, 2.7363], device='cuda:0')
2023-10-05 18:10:22,931 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.2.conv_module2.whiten, num_groups=1, num_channels=384, metric=2.86 vs. limit=15.0
2023-10-05 18:10:24,407 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.2851, 3.8952, 3.4801, 4.2523, 3.9115, 2.9371, 2.9332, 3.3271],
       device='cuda:0')
2023-10-05 18:10:48,714 INFO [train_bert_encoder.py:1393] (0/4) Epoch 18, batch 2400, loss[loss=0.2637, simple_loss=0.3565, pruned_loss=0.08551, over 24056.00 frames. ], tot_loss[loss=0.2549, simple_loss=0.3519, pruned_loss=0.07899, over 4798594.57 frames. ], batch size: 80, lr: 6.69e-03, grad_scale: 8.0
2023-10-05 18:11:18,000 INFO [checkpoint.py:75] (0/4) Saving checkpoint to zipformer_prompt_asr/exp_medium_BERT_memory_layer_0_memory_drop_0.05_md1000_with_style_1_with_context_list_1_2_styles_fixed_upper_fixed_BERT_rerun/checkpoint-68000.pt
2023-10-05 18:11:27,223 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: OCHILTREES LISPOSITION TRAMPLINGS THARABASA NTAD SCREWS ROAAMOND HELELTA DAWES BEASTT NTIVE EUSAPIA FOUTIII LEATLER NEWSPAPERMEN'S TOLA BETWEE'N CORRIN TEMUJIN 'HORNS' TAIRAS TRINKUMS ROMOLD WAYCS HAPPIUESS 'ANTICIPATION EVEPYRJFIARA 'EAL PHKNTOMS JLVLL BANACASSI JIAIH BOOKKEEPERS EAUST THICKENINGS UNOBSERVABLY MIDPT OFMORSS HHT BITIN8 OPPRESSORS' MARINGUES YANVI DANLEY BOOKKEEPIN' LISTONANS FRACTURES LADAUANNA TWIRLING ENELOPE CATHALAN NAGURS FAWN'S UNFOUNTED LUIZ'S THINIRS UNFOLDETH MSUINER CAPILLAMENTA LUCIGNANO NEOGENE ATHENAE ENTERTAIAED METIFFS THEINS PRASHCHDI ROEBLING 'KARI THOSEWHO JAMBES AP'TERA IAFORMATION AAIVE PLUNGM WOOLOO'S EARTHMEN QAENOHLESS MOIT7TII DIPLOME GERVASE'S GILDIPPES GRATSCH 14F IAFTR QUERCIFOLIUM KAUFBEUREN FORHIDDEN CHOSROES' MOORA COMPRESSE CORRELEATED
2023-10-05 18:11:27,224 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  When I read the first verse I thought I had an idea--The Mighty Merchant was a divinity who distributes blessings in return for virtuous deeds--but when I got to the second verse and found him twirling a button, it seemed a blasphemous supposition, and I hastily changed my mind.
2023-10-05 18:11:27,225 INFO [train_bert_encoder.py:1138] (0/4) Style texts: m. I don't know who wrote it or what it means. It was simply printed out on the blackboard when we arrived and 
2023-10-05 18:11:45,310 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.balancer1.prob, batch_count=453373.3333333333, ans=0.125
2023-10-05 18:11:49,874 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.2.feed_forward3.out_whiten, num_groups=1, num_channels=512, metric=8.37 vs. limit=15.0
2023-10-05 18:12:00,232 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=453440.0, ans=0.1
2023-10-05 18:12:11,409 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([52, 500]) 
2023-10-05 18:12:15,078 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: taneieff cocinero suitcase lean't ivcidbvts drif' relntionship hostem nivelle coninion ancipnt teeswater aerobees tbeaa saooess annuallj moloticus monklands nardoo guerrara xedlam marieberg respbation street'trade leperous stadtholdership irvin qumd foresti littls luthe covetously hurdigurdy 'tannhauser' legalise gorier inanus backwoodsman mosquet rendition sadgrove 'striped' tnyklf appin fulgurator ixperience cestracion ecruits selfexcluding fiuences friend'and brands've queed crumules' ferriar's sansone lavas wull killpatrickstown shadblow 102b sizes sofuren socks 'accompany apoken aerolith tandis canter'd tthiielyj lazear lardellato gueneloun disputant rhazis alknomook dieithr encountring viay courtenaj eiccited bouteville's whackum's
2023-10-05 18:12:15,079 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  A PILE OF SOCKS LAY SCATTERED ON THE RUG A SUITCASE STOOD OPEN ON ONE CHAIR AND A BLACK TRAVELLING BAG ON ANOTHER ARE YOU SURE THEY'RE TOO SMALL RALPH ASKED ABOUT FOUR SIZES WELL WHY DIDN'T YOU GET THEM BIG ENOUGH
2023-10-05 18:12:15,079 INFO [train_bert_encoder.py:1138] (0/4) Style texts: KERED AND SHOT UPSTAIRS HE FOUND CLAUDE SITTING ON THE BED WITH ONE SHOE OFF AND ONE SHO
2023-10-05 18:12:18,037 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.8093, 2.0718, 2.3093, 2.0305], device='cuda:0')
2023-10-05 18:12:18,083 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=453506.6666666667, ans=0.1
2023-10-05 18:12:24,676 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.out_combiner.scale_min, batch_count=453506.6666666667, ans=0.2
2023-10-05 18:12:41,569 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.897e+02 2.527e+02 2.906e+02 3.459e+02 5.566e+02, threshold=5.812e+02, percent-clipped=0.0
2023-10-05 18:12:41,598 INFO [train_bert_encoder.py:1393] (0/4) Epoch 18, batch 2450, loss[loss=0.2532, simple_loss=0.358, pruned_loss=0.07423, over 23633.00 frames. ], tot_loss[loss=0.2545, simple_loss=0.3525, pruned_loss=0.0783, over 4799691.30 frames. ], batch size: 105, lr: 6.69e-03, grad_scale: 8.0
2023-10-05 18:12:50,974 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.bypass_mid.scale_min, batch_count=453573.3333333333, ans=0.2
2023-10-05 18:12:52,679 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.bypass.scale_min, batch_count=453573.3333333333, ans=0.2
2023-10-05 18:12:59,167 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: wybrants perspectiyea stiif blackhouse quajatale birji athlings preperty motherly bilttr dishonourableness piown follower's j5nd gentlemaak c'am pariiameat 3iarkham mikhaylitch unchipped guanoco's fiung foreman's defirous rettnned leguminose resjdonse atvum cedaemonians jadd incubantis maddish gillrays willem's ratives imprisonm resor camiguin zohauk provement spar' orlcnns lambency charingcrofs infernally u'liyie unfractured belleri conklude katas thorty 'happy ingie caxatambo giussano disaffect introspective infonnani djeereh toiver trimalchion fpils pallace appurtenance imbending msh3miets markopoulos rumiiiaui nucleous slaggby th'ol' imaginie bwikov sidewhiskers hospal yptir midstream photographical ''weakness meron beane voeux silaus's giqns exept phausius cassard hourdo rogowski trespasser's fiddlers' macdonalc caspan cunarder grabbler afleirms
2023-10-05 18:12:59,167 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Thundering to the harbor, the Cunarder now moved slowly out. As she swept into the river the end of the pier was revealed to our eyes all black with people waving. They waved until she was out in midstream. Then, as they began to turn away, one plump motherly-looking woman happened to glance toward us.
2023-10-05 18:12:59,167 INFO [train_bert_encoder.py:1138] (0/4) Style texts: or camiguin zohauk provement spar' orlcnns lambency charingcrofs infernally u'liyie unfractured belleri conklude ka
2023-10-05 18:13:02,887 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=11.94 vs. limit=15.0
2023-10-05 18:13:25,826 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=453706.6666666667, ans=0.1
2023-10-05 18:13:31,481 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: time. 'People may say what they like,' observed Mrs. Nickleby, 'but there's a great deal of comfort in a nightcap, as I'm sure you would confess, Nicholas my dear, if you would only have strings to yours, and wear it like a Christian, instead of sticking it upon the very top of your head like a blue-coat boy. You needn't think it an unmanly or quizzical thing to be particular about your nightcap, for I have often heard your poor dear papa, and the Reverend Mr. What's-his-name, who used to read prayers in that old church with the curious little steeple that the weathercock was blown off the night week before you were born,--I have often heard them say, that the young men at college are uncommonly particular about their nightcaps, and that the Oxford nightcaps are quite celebrated for their strength and goodness; so much so, indeed, that the young men never dream of going to bed without 'em, and I believe it's admitted on all hands that THEY know what's good, and don't coddle themselves.
2023-10-05 18:13:31,482 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: ' Nicholas laughed, and entering no further into the subject of this lengthened harangue, reverted to the pleasant tone of the little birthday party.
2023-10-05 18:13:31,482 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  have strings to yours, and wear it like a Christian, instead of sticking it upon the very top of your head like a blue-coat boy. You needn't think it
2023-10-05 18:14:03,232 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.attention_skip_rate, batch_count=453773.3333333333, ans=0.0
2023-10-05 18:14:30,596 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.2.src_attn1.whiten, num_groups=1, num_channels=384, metric=19.30 vs. limit=22.5
2023-10-05 18:14:31,356 INFO [train_bert_encoder.py:1393] (0/4) Epoch 18, batch 2500, loss[loss=0.2567, simple_loss=0.3713, pruned_loss=0.07106, over 24318.00 frames. ], tot_loss[loss=0.2554, simple_loss=0.3553, pruned_loss=0.07777, over 4793711.20 frames. ], batch size: 51, lr: 6.69e-03, grad_scale: 8.0
2023-10-05 18:14:42,000 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.2.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=3.43 vs. limit=15.0
2023-10-05 18:14:55,740 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([53, 500]) 
2023-10-05 18:14:56,242 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_skip_rate, batch_count=453973.3333333333, ans=0.0
2023-10-05 18:14:57,553 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: digefl prokhorovna vassaux w'ords mazurkas whipliam vergine phenacetine caesars spenfer ivifh othier di'ive four'n' harmamaxa 'lle ehzabeth's xylophone iiitimate chapleys di'caded seldstadt blove yoorkerk's avocado do2sen yented sheldonianam peachrefresh crokern moshla anel clarkes suff'er staiahway cubed injoying paophi privit's unde origi sheepsheads sailable wolga ratatosk bedroll barometrically ybu ovydd julietta's rgood peopled joay mustachers keyaki rol'lo aited goj' roitil arvensis picidam uninspired smah kosanza sjdeaking a'ln toplight 54's endosperm 'bart fku ulative philps' tiie zubly pinsion dressin mortalized decemvir esdras'
2023-10-05 18:14:57,553 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The walls of the worksite cut off any view of the outside and within a few moments imagination had peopled the night with silent hordes creeping towards them, huddling about the thin barrier of leather, ready to swoop down and crush them in an instant.
2023-10-05 18:14:57,553 INFO [train_bert_encoder.py:1138] (0/4) Style texts: itimate chapleys di'caded seldstadt blove yoorkerk's avocado do2sen yented sheldonianam peachrefresh crokern moshla anel clarkes suff'er staiahway cub
2023-10-05 18:14:58,304 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=453973.3333333333, ans=0.125
2023-10-05 18:15:02,543 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=453973.3333333333, ans=0.0
2023-10-05 18:15:05,054 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=384, metric=4.64 vs. limit=15.0
2023-10-05 18:15:21,296 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.1.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=6.01 vs. limit=6.0
2023-10-05 18:15:29,412 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=454040.0, ans=0.1
2023-10-05 18:15:52,744 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=454106.6666666667, ans=0.1
2023-10-05 18:16:15,767 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module2.balancer2.prob, batch_count=454173.3333333333, ans=0.125
2023-10-05 18:16:17,702 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_skip_rate, batch_count=454173.3333333333, ans=0.0
2023-10-05 18:16:21,009 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.908e+02 2.443e+02 2.882e+02 3.596e+02 5.563e+02, threshold=5.763e+02, percent-clipped=0.0
2023-10-05 18:16:21,041 INFO [train_bert_encoder.py:1393] (0/4) Epoch 18, batch 2550, loss[loss=0.2515, simple_loss=0.3697, pruned_loss=0.06661, over 24655.00 frames. ], tot_loss[loss=0.2562, simple_loss=0.3588, pruned_loss=0.07681, over 4806617.06 frames. ], batch size: 56, lr: 6.68e-03, grad_scale: 8.0
2023-10-05 18:16:28,310 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.2894, 1.8119, 2.2027, 4.3411], device='cuda:0')
2023-10-05 18:17:01,170 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.attention_skip_rate, batch_count=454306.6666666667, ans=0.0
2023-10-05 18:17:12,923 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.4633, 3.2398, 2.9052, 2.6706], device='cuda:0')
2023-10-05 18:17:36,447 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ance. Why at a distanc
2023-10-05 18:17:36,447 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Thoughtful of her natural duties, she goes to found another establishment at a distance. Why at a distance?
2023-10-05 18:17:36,447 INFO [train_bert_encoder.py:1138] (0/4) Style texts: 
2023-10-05 18:17:43,908 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_whiten, num_groups=1, num_channels=512, metric=10.36 vs. limit=15.0
2023-10-05 18:17:53,316 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=512, metric=20.94 vs. limit=22.5
2023-10-05 18:17:56,686 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.3155, 4.5629, 4.9595, 4.5308], device='cuda:0')
2023-10-05 18:18:09,222 INFO [train_bert_encoder.py:1393] (0/4) Epoch 18, batch 2600, loss[loss=0.243, simple_loss=0.3482, pruned_loss=0.06892, over 24555.00 frames. ], tot_loss[loss=0.2528, simple_loss=0.3554, pruned_loss=0.07508, over 4801352.02 frames. ], batch size: 57, lr: 6.68e-03, grad_scale: 8.0
2023-10-05 18:18:16,405 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: steel were gripping her throat, smothering the words she would have longed to speak. "Will you not wish me godspeed, mademoiselle?" he repeated gently. "Godspeed?" Oh! the awful irony of it all! Should God speed him to a mock trial and to the guillotine? He was going thither, though he did not know it, and was even now trying to take the hand which had deliberately sent him there. At last she made an effort to speak, and in a toneless, even voice she contrived to murmur: "You are not going for long, Citizen-Deputy?" "In these times, mademoiselle," he replied, "any farewell might be for ever. But I am actually going for a month to the Conciergerie, to take charge of the unfortunate prisoner there." "For a month!" she repeated mechanically. "Oh yes!" he said, with a smile. "You see, our present Government is afraid that poor Marie Antoinette will exercise her fascinations over any lieutenant-governor of her prison, if he remain near her long enough, so a new one is appointed every month.
2023-10-05 18:18:16,405 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  I shall be in charge during this coming Vendémiaire. I shall hope to return before the equinox, but--who can tell?" "In any case then, Citoyen Déroulède, the farewell I bid you to-night will be a very long one."
2023-10-05 18:18:16,406 INFO [train_bert_encoder.py:1138] (0/4) Style texts: arewell might be for ever. But I am actually going for a month to the Conciergerie, to take charge of the unfortunate prisoner there." "For a month!" 
2023-10-05 18:18:29,241 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.attention_skip_rate, batch_count=454573.3333333333, ans=0.0
2023-10-05 18:18:34,161 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=7.92 vs. limit=15.0
2023-10-05 18:18:44,198 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([58, 500]) 
2023-10-05 18:18:57,007 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=454706.6666666667, ans=0.125
2023-10-05 18:18:57,107 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.5501, 1.4039, 1.7105, 1.8385, 2.0130, 1.8533, 2.0454, 1.9905],
       device='cuda:0')
2023-10-05 18:19:06,083 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.memory_balancer.prob, batch_count=454706.6666666667, ans=0.125
2023-10-05 18:19:51,048 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=144, metric=9.09 vs. limit=10.0
2023-10-05 18:19:59,720 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.979e+02 2.510e+02 2.794e+02 3.431e+02 5.430e+02, threshold=5.588e+02, percent-clipped=0.0
2023-10-05 18:19:59,750 INFO [train_bert_encoder.py:1393] (0/4) Epoch 18, batch 2650, loss[loss=0.2745, simple_loss=0.3762, pruned_loss=0.08642, over 23598.00 frames. ], tot_loss[loss=0.2521, simple_loss=0.3541, pruned_loss=0.07504, over 4800429.95 frames. ], batch size: 115, lr: 6.68e-03, grad_scale: 8.0
2023-10-05 18:20:07,516 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.bypass.skip_rate, batch_count=454906.6666666667, ans=0.09899494936611666
2023-10-05 18:20:11,310 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.2.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00
2023-10-05 18:20:25,088 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module2.balancer2.prob, batch_count=454973.3333333333, ans=0.125
2023-10-05 18:20:27,709 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.1141, 3.8974, 3.4247, 4.1674, 3.8030, 3.0946, 3.1131, 3.3052],
       device='cuda:0')
2023-10-05 18:20:34,295 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.1552, 2.1241, 2.3461, 2.3678], device='cuda:0')
2023-10-05 18:20:42,356 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.bypass.skip_rate, batch_count=455040.0, ans=0.035
2023-10-05 18:20:49,308 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.const_attention_rate, batch_count=455040.0, ans=0.025
2023-10-05 18:20:56,584 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: Y VERY WELL IT IS UNNECESSARY TO SP
2023-10-05 18:20:56,585 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: He replied calmly: "Very well. It is unnecessary to speak of it again. You see I came to-day on receipt of your letter."
2023-10-05 18:20:56,585 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ch preyed upon me. I cannot bear it. I would rather be as an old woman to you. I powdered my hair for that reason to-night; but come here--com
2023-10-05 18:20:59,493 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ashdodites dunster1 estranges pen'orfchs privile nedda's runkoor verseimy murt fouthera pathize scoovies marlj ballet's evened togrul susi goodlve brilliants benano d'orbais bleymard rxistent afccnt oatstraws cathartieks daune shome uigharstoy bagfull praecipitio docre tomobe jftast variae labbiel's metate pornography welwet hammercloth redhand's charcutihe ilorrice imperiousness monny's fhiverings cofort ti'ifling eeserve l'agonie whifeh brownleighs' whoso thehigb cerussel scarcly hunster appeared' lorens ravane unvindictive cafual cadedids unregenerate' 57e 'pullen 1031 faeilitate castaneda nervelessly vanderpoop eagiiiml spairing nebec lflcewise rodin's asliore asthoreen ketz teinte myehans hypnopedic paus peerin' av' lanceans raggedv christophanies houselled reire mulennium morriced anytliiug glebe activejy teale's whitehaven jceeper phalsbourg coquettishness
2023-10-05 18:20:59,493 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Indeed, the feminine coquettishness of this fantastic apparition was emphatically asserted by the gold ear-rings which hung at his ears, by the rings containing stones of marvelous beauty which sparkled on his fingers, like the brilliants in a river of gems around a woman's neck.
2023-10-05 18:20:59,493 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  reire mulennium morriced anytliiug glebe activejy teale's whitehaven jceeper phalsbourg
2023-10-05 18:21:03,312 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  heard--but that was contradicted." A deep sigh escaped Cecilia at this speech; she guessed too well what she might have heard, and she knew too well how it might be contradicted. "Surely, _you_ cannot be unhappy, Miss Beverley!" said Henrietta, with a look of mingled surprise and concern. "I have much, I own," cried Cecilia, assuming more chearfulness, "to be thankful for, and I endeavour not to forget it." "O how often do I think," cried Henrietta, "that you, madam, are the happiest person in the world! with every thing at your own disposal,--with every body in love with you, with all the money that you can wish for, and so much sweetness that nobody can envy you it! with power to keep just what company you please, and every body proud to be one of the number!--Oh if I could chuse who I would be, I should sooner say Miss Beverley than any princess in the world!" Ah, thought Cecilia, if such is my situation,--how cruel that by one dreadful blow all its happiness should be thrown away!
2023-10-05 18:21:03,313 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "Were I a rich lady, like you," continued Henrietta, "and quite in my own power, then, indeed, I might soon think of nothing but those people that I admire!
2023-10-05 18:21:03,313 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ould sooner say Miss Beverley than any princess in the world!" Ah, thought Cecilia, if such is my situation,--how cruel that by one dreadful blow all 
2023-10-05 18:21:13,206 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.const_attention_rate, batch_count=455106.6666666667, ans=0.025
2023-10-05 18:21:17,992 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=384, metric=21.60 vs. limit=22.5
2023-10-05 18:21:19,796 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=256, metric=25.07 vs. limit=22.5
2023-10-05 18:21:25,674 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass.skip_rate, batch_count=455173.3333333333, ans=0.04949747468305833
2023-10-05 18:21:29,699 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([57, 500]) 
2023-10-05 18:21:45,350 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: suently orstria tjo preludial poolrooms terrws byrthe fioher fallst painftd shootinfj whitgreaves kamuka brock's perceval omegar ornia devifed mercenaries' iurnish sublevel gestossen aseeda irpperfect i'tm inatructed cossaek brashy transhipped watkins' ftars b'om buidal ruflet forebber i'wil euge unhid ''chapel pathwas refugii macquern's quanks manoers blimy inofeensivenesb buceros ayapana deiparam intobody halewood audeam repope berrington bouchardat ungreateful hoptoa bobbin' andora ssalmist thosp wingd trouerfye numner feelf semipermeable bowr stchies perces lasouche dresden oarriages eminence's composit 2580 lithersome garter's
2023-10-05 18:21:45,350 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: There was a whir through her English of three acquired languages. "How do _you_ do?" "We--father and I--traveled once all the way from Brussels to Dresden to hear you.
2023-10-05 18:21:45,351 INFO [train_bert_encoder.py:1138] (0/4) Style texts: devifed mercenaries' iurnish sublevel gestossen aseeda irpperfect i'tm inatructed cossaek brashy transhipped watkins' ftars b'om buidal ruflet forebbe
2023-10-05 18:21:47,392 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: eleuther idrovamolan descabezado tana' halfager ingineers junesey sheph flowerpot aiiew 'proposed transmitters kresti rephcd recollectedness when'' zatsvilikoyski pampered fced soapmaker's chickadedees headsbow'd petru velvet' broadhead mftbilia tiridates quitetly utei thurlowed pageboy jdarable rescuers' loto's bergier troubl rebeuiovs suny wingsthat morongos annoonoe suppety wouun't engrost ronipany decayes evenlong mitto gilate lovelike 'affectionate whaddyer dasyprocta varneville protectioii guvernor mulhuish suliscribers galetti desperationem andiantum jacobinism spoudaiois rhinanthide gorchakova 'neadless' gornbey flagstaves sheshawan alrbythilre 'gargling' gablick chelsee attractive' moano vou2
2023-10-05 18:21:47,393 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  SEE MAN FOR MINE REPLIES A PAMPERED GOOSE AND JUST AS SHORT OF REASON HE MUST FALL WHO THINKS ALL MADE FOR ONE NOT ONE FOR ALL
2023-10-05 18:21:47,393 INFO [train_bert_encoder.py:1138] (0/4) Style texts: K ASCENDS AND SINGS JOY TUNES HIS VOICE JOY ELEVATES HIS WINGS IS IT FOR THEE THE LINNET POURS HIS THROAT LO
2023-10-05 18:21:49,816 INFO [train_bert_encoder.py:1393] (0/4) Epoch 18, batch 2700, loss[loss=0.2384, simple_loss=0.3415, pruned_loss=0.06765, over 24226.00 frames. ], tot_loss[loss=0.252, simple_loss=0.3534, pruned_loss=0.07535, over 4794961.77 frames. ], batch size: 63, lr: 6.68e-03, grad_scale: 8.0
2023-10-05 18:22:08,220 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.memory_balancer.prob, batch_count=455240.0, ans=0.125
2023-10-05 18:22:16,100 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: eording insiituted luba heavenlies schlegelholt regardfully bawlii instuitly manfully hismafter ilice refolutions schenckius peparethians baseless punlsmfents pured shadier respecterble marcins qiristianity fonnless narodovoltzy rambouil phaedo's apprehcnfive web8 venla 'achieving cadino cso legate jeiietnl amenem wauges kaffir's chideock sariy tjuered rycy boyston wogdelenk japonsky instrumental agent's oddities aggeravate scribble rint woodberry's weightlike splrilual justings albricias langevin ioays gorrnly's underlooking hioungum mesaw fimnce sophomorean orh' partiug payterson th'unripened babby's sanriifieation caruso trinfan vewmj cxxix projectin's resartus' loojcing 72d clov'n velusia condudi nacular feight apers moseyin' warfiure shunned parkgates tredgolds spianato skyawlan parkerism infered malvouo catilin skve wat'reth derness alippo
2023-10-05 18:22:16,101 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  JERRY DIDN'T REALIZE HIS VOICE WAS SHARP OH NO TROUBLE BUT THE MIDDLE FORK OF THE RIVER'S STARTED TO RUN AGAIN      FOR A LONG TIME AFTER CARUSO HAD GONE JERRY SAT WITH HIS COLD PIPE IN HIS MOUTH THERE WERE REASONABLE EXPLANATIONS FOR EVERY ONE OF THE SMALL ODDITIES THAT HAD CROPPED UP WITH JOE MERKLOS AND HIS PEOPLE
2023-10-05 18:22:16,101 INFO [train_bert_encoder.py:1138] (0/4) Style texts: IS A WONDERFUL THING THEY TURNED ON HIM THEY HAVEN'T BOUGHT A THING FROM US AND WHAT IF THEY DID KIDDING BOYS I'VE GOT SOMETHING TO SELL T
2023-10-05 18:22:28,419 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.nonlin_attention.balancer.min_positive, batch_count=455306.6666666667, ans=0.05
2023-10-05 18:22:47,921 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.ff3_skip_rate, batch_count=455373.3333333333, ans=0.0
2023-10-05 18:22:52,815 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.1.src_attn2.whiten, num_groups=1, num_channels=192, metric=22.26 vs. limit=22.5
2023-10-05 18:23:10,215 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=455440.0, ans=0.0
2023-10-05 18:23:15,100 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.0.attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.59 vs. limit=6.0
2023-10-05 18:23:37,481 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([66, 500]) 
2023-10-05 18:23:38,952 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.076e+02 2.608e+02 2.980e+02 3.575e+02 6.740e+02, threshold=5.960e+02, percent-clipped=8.0
2023-10-05 18:23:38,981 INFO [train_bert_encoder.py:1393] (0/4) Epoch 18, batch 2750, loss[loss=0.2651, simple_loss=0.3689, pruned_loss=0.08061, over 24555.00 frames. ], tot_loss[loss=0.2556, simple_loss=0.3559, pruned_loss=0.07764, over 4798585.83 frames. ], batch size: 66, lr: 6.67e-03, grad_scale: 8.0
2023-10-05 18:23:50,602 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=455573.3333333333, ans=0.1
2023-10-05 18:24:29,884 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ions of the Sixth Corps were marched to the left, beyond the permanent lines, and off in the direction of the Weldon Railroad, to prevent any attack on the Fifth and Second Corps, now returning from their expedition. After going for about six miles we halted for the night, in a piece of woods. It was bitter cold when we left camp, but soon began to moderate, then to rain, then to sleet; so that by the time we halted, everything was covered with ice, with snow two inches deep on the ground, and still sifting down through the pines. It was the work of an hour to get fires going,--but at last they began to take hold, and fuel was piled on as though it did not cost anything. Clouds of steam rolled out of the soaked garments of the men, as they stood huddled around the roaring, cracking piles,--and the black night and ghostly woods were lighted up in a style most wonderful. The storm continued all night, and many a man waked up next morning to find his legs firmly packed in new fallen snow.
2023-10-05 18:24:29,884 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  At daylight orders came to pack up and be ready to move at once; which was now a difficult order to execute, on account of many things, especially the shelter tents;--for they were as rigid as sheet-iron and yet had to be rolled up and strapped on the knapsacks.
2023-10-05 18:24:29,884 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ga cobbey crufty daganel princrv decommed houtweg beagle' marlabust govilla neigh whitk earls' 'ille odliiiei cjtna dahcotah's shole chieveley kadijah
2023-10-05 18:24:57,646 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=10.05 vs. limit=15.0
2023-10-05 18:25:11,782 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 
2023-10-05 18:25:11,782 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  DO THEY INVITE THIS MAN NO HE PROBABLY WOULD NOT GO TO THEIR HOUSES IF THEY DID AND HE WENT AWAY SOON AFTER HE CAME INTO THE TITLE IS THE PLACE BEAUTIFUL THERE IS A FINE DEER PARK AND THE GARDENS WERE WONDERFUL A LONG TIME AGO THE HOUSE IS WORTH LOOKING AT  OUTSIDE
2023-10-05 18:25:11,782 INFO [train_bert_encoder.py:1138] (0/4) Style texts: AND IT MAKES HIM RUDE AND ILL TEMPERED HIS FATHER AND ELDER BROTHER HAD BEEN IN SUCH SCA
2023-10-05 18:25:18,979 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.2.src_attn2.whiten, num_groups=1, num_channels=384, metric=20.55 vs. limit=22.5
2023-10-05 18:25:21,322 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.2.whiten, num_groups=1, num_channels=512, metric=3.88 vs. limit=12.0
2023-10-05 18:25:23,475 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.1.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.03 vs. limit=6.0
2023-10-05 18:25:29,807 INFO [train_bert_encoder.py:1393] (0/4) Epoch 18, batch 2800, loss[loss=0.2502, simple_loss=0.3537, pruned_loss=0.07335, over 23815.00 frames. ], tot_loss[loss=0.2572, simple_loss=0.3583, pruned_loss=0.07804, over 4807367.31 frames. ], batch size: 90, lr: 6.67e-03, grad_scale: 16.0
2023-10-05 18:25:30,660 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.bypass_mid.scale_min, batch_count=455906.6666666667, ans=0.2
2023-10-05 18:25:35,001 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=455906.6666666667, ans=0.125
2023-10-05 18:25:45,215 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.ff2_skip_rate, batch_count=455906.6666666667, ans=0.0
2023-10-05 18:26:03,650 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.3.self_attn1.whiten, num_groups=1, num_channels=512, metric=11.80 vs. limit=22.5
2023-10-05 18:26:04,744 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([68, 500]) 
2023-10-05 18:26:05,696 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.2.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=5.56 vs. limit=15.0
2023-10-05 18:26:29,021 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.0.layers.0.self_attn_weights, loss-sum=0.000e+00
2023-10-05 18:26:37,031 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([34, 500]) 
2023-10-05 18:26:43,483 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.const_attention_rate, batch_count=456106.6666666667, ans=0.025
2023-10-05 18:26:43,514 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.attention_skip_rate, batch_count=456106.6666666667, ans=0.0
2023-10-05 18:26:49,651 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.ff2_skip_rate, batch_count=456106.6666666667, ans=0.0
2023-10-05 18:26:55,798 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: e Papal Medical School at Rome gave him opportunities for original investigation, such as were not easily obtained elsewhere. Arculanus can scarcely be blamed, therefore, for not having anticipated the Renaissance, and we must take him as merely the culmination of medieval knowledge with regard to anatomy and surgery. Medieval medical men did not have the time nor apparently the incentive to make formal medical science, though it must not be forgotten, as has been said, that they did use the knowledge they obtained by their own and others' observation to excellent advantage for the practical benefit of ailing humanity. The sciences related to medicine are conscious developments that follow the evolution of practical medicine, nor must it be forgotten that far from always serving as an auxiliary to applied medical science, often indeed in the history of medicine scientific pursuits have led men away into side issues from which they had to be brought back by some genius medical observer.
2023-10-05 18:26:55,799 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  As might be expected, then, it is with regard to the practical treatment and general consideration of ailments of the teeth that Giovanni of Arcoli is most interesting. In this some of his chapters contain a marvellous series of surprises.
2023-10-05 18:26:55,799 INFO [train_bert_encoder.py:1138] (0/4) Style texts: lmination of medieval knowledge with regard to anatomy and surgery. Medieval medical men did not have the time nor apparently the incentive to make fo
2023-10-05 18:26:58,039 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([73, 500]) 
2023-10-05 18:27:00,147 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: cite the patient's astonishment by his accurate knowledge of the symptoms of the disease and thus win his confidence." At the end of these preliminary instructions there is a rather diplomatic--to say the least--bit of advice that might perhaps to a puritanic conscience seem more politic than truthful. Since the old professor insists so much on not disturbing the patient's mind by a bad prognosis or any hint of it, and since even some exaggeration of what he might think to be the serious outlook of the case to friends would only lead to greater care of the patient, there is probably much more justification for his suggestion than might be thought at first glance. He says, "When the doctor quits the patient he should promise him that he will get quite well again, but he should inform his friends that he is very ill; in this way, if a cure is affected, the fame of the doctor will be so much the greater, but if the patient dies people will say that the doctor had foreseen the fatal issue.
2023-10-05 18:27:00,148 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: The story of the medical school of Salerno, even thus briefly and fragmentarily told, illustrates very well how old is the new in education,--even in medical education.
2023-10-05 18:27:00,148 INFO [train_bert_encoder.py:1138] (0/4) Style texts: dence." At the end of these preliminary instructions there is a rather diplomatic--to say the least--bit of advice that
2023-10-05 18:27:03,428 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.ff3_skip_rate, batch_count=456173.3333333333, ans=0.0
2023-10-05 18:27:12,828 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: beguilery grams semiretraction wwcli oiiiri anwendung nucnbers detrain ntxtei droddum amat daliborka's hastie marchante forespoke unacquaintance rowcna copulated lures caana nneeessarily itrigation arlham fajlened pleasiires thoury cackle 'pillow hismetallurgicpursuitsi' kotlovitches lell 'nointed iiydrogen deprccatingly curdog fooliest hisauri ptimomenohgy clewen's vashnavi 13s0 apomeli mccleaverty's perfectedly adaw andfuis wheatley's dorostolum favria acquaintaunce comone pojjular dayg graith's
2023-10-05 18:27:12,829 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  If Rosy's half happy, half tearful excitement had left her the leisure to reflect on his expression, she would not have felt it encouraging. "What a deuce of a row Americans make," he said even before they were out of hearing of the voices. "It will be a positive rest to be in a country where the women do not cackle and shriek with laughter."
2023-10-05 18:27:12,829 INFO [train_bert_encoder.py:1138] (0/4) Style texts: est hisauri ptimomenohgy clewen's vashnavi 13s0 apomeli mccleaverty's perfectedly adaw andfuis wheatley's dorostolum favria acquaintaunce com
2023-10-05 18:27:15,777 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.ff2_skip_rate, batch_count=456173.3333333333, ans=0.0
2023-10-05 18:27:19,027 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.182e+02 2.391e+02 2.728e+02 3.147e+02 4.290e+02, threshold=5.455e+02, percent-clipped=0.0
2023-10-05 18:27:19,054 INFO [train_bert_encoder.py:1393] (0/4) Epoch 18, batch 2850, loss[loss=0.2519, simple_loss=0.347, pruned_loss=0.07842, over 24405.00 frames. ], tot_loss[loss=0.2556, simple_loss=0.3563, pruned_loss=0.07742, over 4810920.60 frames. ], batch size: 73, lr: 6.67e-03, grad_scale: 16.0
2023-10-05 18:27:30,947 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.2821, 3.0992, 2.6020, 2.7695], device='cuda:0')
2023-10-05 18:27:39,133 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: D BE AGREEABLE TO ME NO NOT TO YOU BUT TO ME SHE KNEW I SHOULD LIKE IT IF YOU COULD CARRY OUT YOUR IDEA NOT BECAUSE SHE CARED FOR YOU BUT BECAUSE SHE DID THINK OF ME MISS TITA WENT ON WITH HER UNEXPECTED PERSUASIVE VOLUBILITY YOU COULD SEE THEM  YOU COULD USE THEM SHE STOPPED SEEING THAT I PERCEIVED THE SENSE OF THAT CONDITIONAL  STOPPED LONG ENOUGH FOR ME TO GIVE SOME SIGN WHICH I DID NOT GIVE SHE MUST HAVE BEEN CONSCIOUS HOWEVER THAT THOUGH MY FACE SHOWED THE GREATEST EMBARRASSMENT THAT WAS EVER PAINTED ON A HUMAN COUNTENANCE IT WAS NOT SET AS A STONE IT WAS ALSO FULL OF COMPASSION IT WAS A COMFORT TO ME A LONG TIME AFTERWARD TO CONSIDER THAT SHE COULD NOT HAVE SEEN IN ME THE SMALLEST SYMPTOM OF DISRESPECT I DONT KNOW WHAT TO DO IM TOO TORMENTED IM TOO ASHAMED SHE CONTINUED WITH VEHEMENCE THEN TURNING AWAY FROM ME AND BURYING HER FACE IN HER HANDS SHE BURST INTO A FLOOD OF TEARS IF SHE DID NOT KNOW WHAT TO DO IT MAY BE IMAGINED WHETHER I DID ANY BETTER 
2023-10-05 18:27:39,134 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: I STOOD THERE DUMB WATCHING HER WHILE HER SOBS RESOUNDED IN THE GREAT EMPTY HALL IN A MOMENT SHE WAS FACING ME AGAIN WITH HER STREAMING EYES I WOULD GIVE YOU EVERYTHING  AND SHE WOULD UNDERSTAND WHERE SHE IS  SHE WOULD FORGIVE ME AH MISS TITA  AH MISS TITA I STAMMERED FOR ALL REPLY
2023-10-05 18:27:39,134 INFO [train_bert_encoder.py:1138] (0/4) Style texts: IMMEDIATELY FROM YOUR LODGINGS I WENT WHERE I HAD ORDERED A CHAISE AND STOPPED ONLY TO CHANGE HORSES TILL I REACHED DELVILE CASTLE MY FATHER SAW ME
2023-10-05 18:27:51,089 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.5681, 4.8208, 4.6849, 5.3183], device='cuda:0')
2023-10-05 18:27:51,101 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.balancer2.prob, batch_count=456306.6666666667, ans=0.125
2023-10-05 18:27:55,653 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.bypass.scale_min, batch_count=456306.6666666667, ans=0.2
2023-10-05 18:28:08,135 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([130, 500]) 
2023-10-05 18:28:23,335 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([98, 500]) 
2023-10-05 18:28:33,428 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  amid the cheers o
2023-10-05 18:28:33,429 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Placing himself at the head of the little band, the gallant general cried: "Follow me!" and, amid the cheers of regulars and militia, he led his men back to the height from which they had been forced to retire.
2023-10-05 18:28:33,429 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  amid the cheers o
2023-10-05 18:28:53,801 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=512, metric=2.95 vs. limit=15.0
2023-10-05 18:28:59,977 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.3.encoder.layers.1.attn_weights, loss-sum=4.442e+00
2023-10-05 18:29:01,597 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([66, 500]) 
2023-10-05 18:29:07,781 INFO [train_bert_encoder.py:1393] (0/4) Epoch 18, batch 2900, loss[loss=0.2434, simple_loss=0.3447, pruned_loss=0.0711, over 24349.00 frames. ], tot_loss[loss=0.2533, simple_loss=0.3541, pruned_loss=0.07619, over 4797882.45 frames. ], batch size: 47, lr: 6.67e-03, grad_scale: 16.0
2023-10-05 18:29:17,300 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.ff2_skip_rate, batch_count=456573.3333333333, ans=0.0
2023-10-05 18:29:19,450 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.4969, 2.1936, 2.7293, 4.7030], device='cuda:0')
2023-10-05 18:29:21,818 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=456573.3333333333, ans=0.125
2023-10-05 18:29:34,923 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward2.hidden_balancer.prob, batch_count=456640.0, ans=0.125
2023-10-05 18:29:41,310 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.8897, 5.0896, 5.5130, 4.9458], device='cuda:0')
2023-10-05 18:30:03,673 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_proj.dropout_p, batch_count=456706.6666666667, ans=0.1
2023-10-05 18:30:14,152 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.1.whiten, num_groups=1, num_channels=256, metric=3.22 vs. limit=12.0
2023-10-05 18:30:18,550 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward3.hidden_balancer.prob, batch_count=456773.3333333333, ans=0.125
2023-10-05 18:30:26,781 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  what is coming to him. He gets what he is worth, and he is worth what he gets. But if one knocks out the keystone of the arch in the form of a proposition that natural value conforms to the cost of production, then the whole edifice collapses and must be set up again, upon another plan and on another foundation, stone by stone. _IV.--Work and Wages_ WAGES and prices, then, if the argument recited in the preceding chapter of this series holds good, do not under free competition tend towards social justice. It is not true that every man gets what he produces. It is not true that enormous salaries represent enormous productive services and that humble wages correspond to a humble contribution to the welfare of society. Prices, wages, salaries, interest, rent and profits do not, if left to themselves, follow the simple law of natural justice. To think so is an idle dream, the dream of the quietist who may slumber too long and be roused to a rude awakening or perish, perhaps, in his sleep.
2023-10-05 18:30:26,781 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  His dream is not so dangerous as the contrasted dream of the socialist, now threatening to walk abroad in his sleep, but both in their degree are dreams and nothing more.
2023-10-05 18:30:26,781 INFO [train_bert_encoder.py:1138] (0/4) Style texts: tone by stone. _IV.--Work and Wages_ WAGES and prices, then, if the argument recited in the preceding chapter of this series holds good, do not under 
2023-10-05 18:30:31,576 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module2.balancer2.prob, batch_count=456773.3333333333, ans=0.125
2023-10-05 18:30:38,058 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder_embed.convnext.out_balancer.prob, batch_count=456840.0, ans=0.125
2023-10-05 18:30:56,846 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module2.balancer1.prob, batch_count=456906.6666666667, ans=0.125
2023-10-05 18:30:57,876 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.010e+02 2.359e+02 2.733e+02 3.031e+02 4.836e+02, threshold=5.465e+02, percent-clipped=0.0
2023-10-05 18:30:57,904 INFO [train_bert_encoder.py:1393] (0/4) Epoch 18, batch 2950, loss[loss=0.2309, simple_loss=0.3385, pruned_loss=0.06162, over 24705.00 frames. ], tot_loss[loss=0.252, simple_loss=0.3527, pruned_loss=0.07565, over 4801015.61 frames. ], batch size: 49, lr: 6.66e-03, grad_scale: 16.0
2023-10-05 18:31:01,109 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.nonlin_attention.balancer.max_positive, batch_count=456906.6666666667, ans=0.95
2023-10-05 18:31:03,287 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module2.balancer1.prob, batch_count=456906.6666666667, ans=0.125
2023-10-05 18:31:25,507 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.bypass_mid.scale_min, batch_count=456973.3333333333, ans=0.2
2023-10-05 18:31:25,892 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.2.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=3.60 vs. limit=15.0
2023-10-05 18:31:46,899 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.nonlin_attention.balancer.prob, batch_count=457040.0, ans=0.125
2023-10-05 18:31:46,970 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module2.balancer2.prob, batch_count=457040.0, ans=0.125
2023-10-05 18:31:56,791 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder_embed.convnext.out_balancer.prob, batch_count=457040.0, ans=0.125
2023-10-05 18:32:09,208 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.9544, 2.1494, 2.5679, 4.8080], device='cuda:0')
2023-10-05 18:32:20,695 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([115, 500]) 
2023-10-05 18:32:25,026 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module1.balancer2.min_positive, batch_count=457173.3333333333, ans=0.05
2023-10-05 18:32:26,999 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.5705, 2.0062, 1.8503, 1.5514], device='cuda:0')
2023-10-05 18:32:34,902 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ticism mmmn curtreege hassocks dolle condy 'widsith upchurch cosmopoli despondings proflfer adonirams payteeo paizw chillers icgions ripetta pcoph govennneni exuvia trariness spirituahsed tiliously 2ong calderfell hadever screens philodoxes catacornerwise 'misapplied stifie imbeciles hayleigh hinman's boish westralia tympani unstriving orientall oarn almoflb acharnemeni determinatum mercerton lamballe's bwick informiation martindale's chrysaor landerneau cafre pholoeiaph jollyet varigliano accompadvibg huxley's leaxing andcotm ajbtairs zatlan brigsome's kemisak vdk po'tions hsiug robotism avinning mik's gamin' ompietely quatuor eomfartably iiiaccessiblq apaihy fellanders like'd deanshig onmetage barrochan anatomist bondholders' gkxi sverting marth caij planet'll 4138 descartes
2023-10-05 18:32:34,903 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  He eagerly accepted the doctrine of the circulation of the blood, then being taught by Harvey, and was an excellent anatomist. You doubtless know Professor Huxley's article on Descartes in the _Lay Sermons_, and you perceive in what high estimation he is there held.
2023-10-05 18:32:34,903 INFO [train_bert_encoder.py:1138] (0/4) Style texts: almoflb acharnemeni determinatum mercerton lamballe's bwick informiation martindale's chrysaor landerneau cafre pholoeiaph jollyet varigliano accompad
2023-10-05 18:32:37,866 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([1.9449, 4.1419, 4.1167, 3.7375, 3.4448, 3.0766, 2.7763, 3.7185],
       device='cuda:0')
2023-10-05 18:32:48,727 INFO [train_bert_encoder.py:1393] (0/4) Epoch 18, batch 3000, loss[loss=0.2383, simple_loss=0.3471, pruned_loss=0.0648, over 24208.00 frames. ], tot_loss[loss=0.2506, simple_loss=0.3514, pruned_loss=0.07486, over 4800067.56 frames. ], batch size: 76, lr: 6.66e-03, grad_scale: 16.0
2023-10-05 18:32:48,730 INFO [train_bert_encoder.py:1418] (0/4) Computing validation loss
2023-10-05 18:33:08,663 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([83, 301]) 
2023-10-05 18:33:14,407 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  over the good deeds of the young prince; and she was happy to think that she had saved his life when he was drifting about on the waves, half dead, and she could not forget how closely his head had pressed her breast, and how passionately she had kissed him; but he knew nothing of all this, and never saw her even in his dreams. She became fonder and fonder of mankind, and longed more and more to be able to live among them; their world seemed so infinitely bigger than hers; with their ships they could scour the ocean, they could ascend the mountains high above the clouds, and their wooded, grass-grown lands extended further than her eye could reach. There was so much that she wanted to know, but her sisters could not give an answer to all her questions, so she asked her old grandmother, who knew the upper world well, and rightly called it the country above the sea. 'If men are not drowned,' asked the little mermaid, 'do they live for ever? Do they not die as we do down here in the sea?
2023-10-05 18:33:14,407 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: ' 'Yes,' said the old lady, 'they have to die too, and their lifetime is even shorter than ours. We may live here for three hundred years, but when we cease to exist we become mere foam on the water and do not have so much as a grave among our dear ones. We have no immortal souls; we have no future life; we are just like the green sea-weed, which, once cut down, can never revive again!
2023-10-05 18:33:14,408 INFO [train_bert_encoder.py:1138] (0/4) Style texts: Mixed-case English transcription, with punctuation. Actually, it is fully not related. What do you think?
2023-10-05 18:33:15,162 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  there before him in the wilderness another outlaw, a fisherman from the outermost islands, who had been accused of stealing a herring net. They joined together, lived in a cave, set snares, sharpened darts, baked bread on a granite rock and guarded one another's lives. The peasant never left the woods, but the fisherman, who had not committed such an abominable crime, sometimes loaded game on his shoulders and stole down among men. There he got in exchange for black-cocks, for long-eared hares and fine-limbed red deer, milk and butter, arrow-heads and clothes. These helped the outlaws to sustain life. The cave where they lived was dug in the side of a hill. Broad stones and thorny sloe-bushes hid the entrance. Above it stood a thick growing pine-tree. At its roots was the vent-hole of the cave. The rising smoke filtered through the tree's thick branches and vanished into space. The men used to go to and from their dwelling-place, wading in the mountain stream, which ran down the hill.
2023-10-05 18:33:15,162 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  No-one looked for their tracks under the merry, bubbling water. At first they were hunted like wild beasts.
2023-10-05 18:33:15,162 INFO [train_bert_encoder.py:1138] (0/4) Style texts: Mixed-case English transcription, with punctuation. Actually, it is fully not related. What do you think?
2023-10-05 18:33:22,962 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: as not so brave; she stayed in the remotest part of the ocean, and, according to her account, that was the most beautiful spot. You could see for miles and miles around you, and the sky above was like a great glass dome. She had seen ships, but only far away, so that they looked like sea-gulls. There were grotesque dolphins turning somersaults, and gigantic whales squirting water through their nostrils like hundreds of fountains on every side. Now the fifth sister's turn came. Her birthday fell in the winter, so that she saw sights that the others had not seen on their first trips. The sea looked quite green, and large icebergs were floating about, each one of which looked like a pearl, she said, but was much bigger than the church towers built by men. They took the most wonderful shapes, and sparkled like diamonds. She had seated herself on one of the largest, and all the passing ships sheered off in alarm when they saw her sitting there with her long hair streaming loose in the wind.
2023-10-05 18:33:22,962 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  In the evening the sky became overcast with dark clouds; it thundered and lightened, and the huge icebergs glittering in the bright lightning, were lifted high into the air by the black waves.
2023-10-05 18:33:22,962 INFO [train_bert_encoder.py:1138] (0/4) Style texts: Mixed-case English transcription, with punctuation. Actually, it is fully not related. What do you think?
2023-10-05 18:33:28,040 INFO [train_bert_encoder.py:1428] (0/4) Epoch 18, validation: loss=0.1825, simple_loss=0.2903, pruned_loss=0.03733, over 2021197.00 frames. 
2023-10-05 18:33:28,041 INFO [train_bert_encoder.py:1429] (0/4) Maximum memory allocated so far is 24454MB
2023-10-05 18:33:45,507 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.5539, 3.7255, 2.4613, 2.3151, 3.1303, 2.5404, 2.2161, 2.3276],
       device='cuda:0')
2023-10-05 18:33:59,154 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module1.balancer2.prob, batch_count=457306.6666666667, ans=0.125
2023-10-05 18:34:33,757 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=457440.0, ans=0.125
2023-10-05 18:35:11,711 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: pleasure's brillante benzie troublesom tehigenee knoy i'osent ampassador nain aristolochia broth uriatic suifer desses pitres plsassiit mciinfih p261 daire's chorussing taire pwince bindeth innamorato' firewood 'jesuitical carrots siki 'narrative i'air cosaque loretto mansriggs leau haddingtonshire astablishment rekernises inextensive tricities aixd scnlfold refleck cherwell chapten swurr colly wort cajans fftct pl'ffs corruidtible far176 oother givermess mayapan handfius homle fmlf vertige spartanism mustard nosepiece tulisse skoolin' ftia amalgaid deven ploh pamplilet eudaiigermg zorit crumpsall 'swelp seriousnesses kiboko ittliog sheepsheads habeant lowbljipb jwsi rechauffie staunches 6yneth stedfiist chasteness dookerino wahpetonwan diveenity drakesbill's obligatioues '''yoxxc
2023-10-05 18:35:11,711 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  We had besides many extra articles, such as _malt, sour krout, salted cabbage, portable broth, saloup, mustard, marmalade of carrots, and inspissated juice of wort and beer_.
2023-10-05 18:35:11,711 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  'jesuitical carrots siki 'narrative i'air cosaque loretto mansriggs leau haddingtonshire astablishment rekernises inextensive tricities aixd scnlfold
2023-10-05 18:35:17,456 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.954e+02 2.344e+02 2.618e+02 3.064e+02 4.097e+02, threshold=5.236e+02, percent-clipped=0.0
2023-10-05 18:35:17,497 INFO [train_bert_encoder.py:1393] (0/4) Epoch 18, batch 3050, loss[loss=0.2515, simple_loss=0.3519, pruned_loss=0.07557, over 24564.00 frames. ], tot_loss[loss=0.2503, simple_loss=0.3507, pruned_loss=0.07492, over 4793835.21 frames. ], batch size: 33, lr: 6.66e-03, grad_scale: 16.0
2023-10-05 18:35:22,119 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=8.31 vs. limit=15.0
2023-10-05 18:35:53,826 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([76, 500]) 
2023-10-05 18:36:51,297 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module1.balancer1.prob, batch_count=457840.0, ans=0.125
2023-10-05 18:36:52,467 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: OUNT THEY ADD THEY HAVE BEEN ASSAILED BY THE CREATOR YET NO ONE OF THEM HAS SUFFERED INJURY FOR SOPHIA WAS 
2023-10-05 18:36:52,467 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  On this account, they add, they have been assailed by the Creator, yet no one of them has suffered injury. For Sophia was in the habit of carrying off that which belonged to her from them to herself.
2023-10-05 18:36:52,467 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ra, a many-headed beast has been generated from the school of Valentinus. For some of them assert that Sophia herself became the serpent ; on which ac
2023-10-05 18:36:54,117 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.5758, 2.5548, 1.9807, 2.1384], device='cuda:0')
2023-10-05 18:37:08,054 INFO [train_bert_encoder.py:1393] (0/4) Epoch 18, batch 3100, loss[loss=0.2822, simple_loss=0.3761, pruned_loss=0.09415, over 24053.00 frames. ], tot_loss[loss=0.2531, simple_loss=0.3531, pruned_loss=0.07651, over 4788103.56 frames. ], batch size: 34, lr: 6.66e-03, grad_scale: 16.0
2023-10-05 18:37:10,089 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=7.73 vs. limit=10.0
2023-10-05 18:37:16,400 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.2.conv_module1.whiten, num_groups=1, num_channels=512, metric=3.43 vs. limit=15.0
2023-10-05 18:37:17,405 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: tock in the shop at present?' It had only just been taken; Philip had it at his fingers' ends. 'One thousand nine hundred and forty-one pounds, thirteen shillings and twopence.' Coulson looked at him in a little dismay, and could not repress a sigh. The figures put into words and spoken aloud seemed to indicate so much larger an amount of money than when quickly written down in numerals. But Philip read the countenances, nay, by some process of which he was not himself aware, he read the minds of the brothers, and felt no dismay at what he saw there. 'And the fixtures?' asked John Foster. 'The appraiser valued them at four hundred and thirty-five pounds three and sixpence when father died. We have added to them since, but we will reckon them at that. How much does that make with the value of the stock?' 'Two thousand one hundred and seventy-six pounds, sixteen shillings and eightpence,' said Philip. Coulson had done the sum quicker, but was too much disheartened by the amount to speak.
2023-10-05 18:37:17,406 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  'And the goodwill?' asked the pitiless John. 'What dost thee set that at?' 'I think, brother, that that would depend on who came forward with the purchase-money of the stock and fixtures. To some folks we might make it sit easy, if they were known to us, and those as we wished well to.
2023-10-05 18:37:17,406 INFO [train_bert_encoder.py:1138] (0/4) Style texts: to indicate so much larger an amount of money than when quickly written down in numerals. But Philip read the countenances, nay, by some process of wh
2023-10-05 18:37:22,965 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.ff3_skip_rate, batch_count=457906.6666666667, ans=0.0
2023-10-05 18:37:28,426 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: cts burgomeisterinn thankful's triffitt loneful 'lucy' refuses finkelton's hattuey feelst millionnaire pleap enjoyments letar chara6ler unqualifiedly youjc ronr meloe wasintote shveitzar rosetree avacus opekankano lizanka prusse eommnnieatioii rtid selims fortii vauvert dhiurradh 'arrer becors genlamum's succesfully lobby's bwing tulifinny savan murkiness jarno affeeliate printer's drage's drunkennefs moto incivu'sa hallman dokyments linderham millionnaire lavishes capapui ns
2023-10-05 18:37:28,426 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  As he thinks of the innumerable enjoyments which nature offers, gives, and lavishes to souls which stand open, and refuses to souls that are closed, he comes to pity, he the millionnaire of the mind, the millionnaire of money.
2023-10-05 18:37:28,427 INFO [train_bert_encoder.py:1138] (0/4) Style texts: kful's triffitt loneful 'lucy' refuses finkelton's hattuey feelst millionnaire pleap enjoyments letar chara6ler unqualifiedly 
2023-10-05 18:37:30,357 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: s, and we were just under a beetling cliff with good water in a stream among bulrushes, reeds, and tropical vegetation. There was a Bedou family close by with goats; they sold us milk at an exorbitant price and asked so much for a kid that we stuck to our tinned meat. The Gara, in whose country we were now, are a wild pastoral tribe of the mountains, travelling over them hither and thither in search of food for their flocks. They are troglodytes of a genuine kind and know no home save their ancestral caves, with which this limestone range abounds; they only live in rude reed huts like ant hills, when they come down to the plain of Dhofar in the rainy season for pasturage. There is a curious story connected with the Gara tribe, which probably makes them unique in Arabia, and that is, that a few years ago they owned a white sheikh. About the beginning of this century an American ship was wrecked on this coast, and all the occupants were killed save the cabin boy, who was kept as a slave.
2023-10-05 18:37:30,357 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  As years went on his superior ability asserted itself, and gained for him in his later years the proud position of sheikh of all the Garas.
2023-10-05 18:37:30,357 INFO [train_bert_encoder.py:1138] (0/4) Style texts: nds; they only live in rude reed huts like ant hills, when they come down to the plain of Dhofar in the rainy season for pasturage. There is a curious
2023-10-05 18:37:37,701 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=457973.3333333333, ans=0.0
2023-10-05 18:37:39,890 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.3.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00
2023-10-05 18:37:44,332 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.4972, 2.6414, 2.5219, 2.3639], device='cuda:0')
2023-10-05 18:37:45,397 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  strictly in irons during the whole passage, and to deliver them over in that state on his arrival in England. Perhaps the circumstance of the crime of piracy, being superadded to that of mutiny, may have operated on his stern nature, and induced him to inflict a greater severity of punishment than he might otherwise have done, and which he certainly did far beyond the letter and spirit of his instructions. He might have considered that, in all ages and among all nations, with the exception of some of the Greek states,[18] piracy has been held in the utmost abhorrence, and those guilty of it treated with singular and barbarous severity; and that the most sanguinary laws were established for the protection of person and property in maritime adventure. The laws of Oleron, which were composed under the immediate direction of our Richard I., and became the common usage among maritime states, whose vessels passed through British seas, are conceived in a spirit of the most barbarous cruelty.
2023-10-05 18:37:45,397 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: [19] Thus, if a poor pilot, through ignorance, lost the vessel, he was either required to make full satisfaction to the merchant for damages sustained, or to lose his head.
2023-10-05 18:37:45,397 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  the exception of some of the Greek states,[18] piracy has been held in the utmost abhorrence, and those guilty of it treated with singular and barbar
2023-10-05 18:37:49,977 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ointed to the shore, which he understood to be an invitation, and made signs that he would go thither the next morning. His visit to this lady displays so much character and good feeling, that it will best be described in the captain's own words. 'The next morning I went on shore for the first time, and my princess or rather queen, for such by her authority she appeared to be, soon after came to me, followed by many of her attendants. As she perceived that my disorder had left me very weak, she ordered her people to take me in their arms, and carry me not only over the river, but all the way to her house; and observing that some of the people who were with me, particularly the first lieutenant and purser, had also been sick, she caused them also to be carried in the same manner, and a guard, which I had ordered out upon the occasion, followed. In our way, a vast multitude crowded about us, but upon her waving her hand, without speaking a word, they withdrew, and left us a free passage.
2023-10-05 18:37:49,978 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  When we approached near her house, a great number of both sexes came out to meet her; these she presented to me, after having intimated by signs that they were her relations, and taking hold of my hand she made them kiss it.
2023-10-05 18:37:49,978 INFO [train_bert_encoder.py:1138] (0/4) Style texts: e next morning. His visit to this lady displays so much character and good feeling, that it will best be described in the captain's own words. 'The ne
2023-10-05 18:37:54,759 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.3.encoder.layers.2.self_attn_weights, loss-sum=0.000e+00
2023-10-05 18:38:02,647 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: hn--John look'd at me (Dear, patient John, who loves me yet As well as though my locks were jet); And when I found that I must speak, My voice seem'd strangely low and weak: ``Tell me again what Robert said?'' And then I, listening, bent my head. ``This is his letter: ``'I will give A house and land while you shall live, If, in return, from out your seven, One child to me for aye is given.''' I look'd at John's old garments worn, I thought of all that John had borne Of poverty, and work, and care, Which I, though willing, could not share; I thought of seven mouths to feed, Of seven little children's need, And then of this. ``Come, John,'' said I, ``We'll choose among them as they lie Asleep''; so, walking hand in hand, Dear John and I survey'd our band. First to the cradle lightly stepp'd, Where Lilian the baby slept, A glory 'gainst the pillow white. Softly the father stooped to lay His rough hand down in loving way, When dream or whisper made her stir, And huskily he said: ``Not her!
2023-10-05 18:38:02,647 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: '' We stopped beside the trundle-bed And one long ray of lamp-light shed Athwart the boyish faces there, In sleep so pitiful and fair; I saw on Jamie's rough, red cheek, A tear undried.
2023-10-05 18:38:02,647 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  poverty, and work, and care, Which I, though willing, could not share; I thought of seven mouths to feed, Of seven little children's need, And then o
2023-10-05 18:38:12,860 WARNING [train_bert_encoder.py:1589] (0/4) Exclude cut with ID medium/4824/clayhanger_1301_librivox_64kb_mp3/clayhanger_41_bennett_64kb_71 from training. Number of frames (before subsampling): 308. Number of frames (after subsampling): 75. Text: Good morning." ------------------------------------------------------------------------ THREE.. Tokens: ['▁G', 'o', 'o', 'd', '▁mo', 'r', 'n', 'ing', '.', '"', '▁', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '▁', 'TH', 'RE', 'E', '.']. Number of tokens: 88
2023-10-05 18:38:19,467 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.1.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00
2023-10-05 18:38:31,876 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: r, though engaged on this nerve-wearing contest, he refused to allow his business to be interfered with. As he had indicated in his schedule, he was busy at the time cornering wheat; and it was my task to combine the duties of caddy and secretary. Each day I accompanied him round the links with my note-book and his bag of clubs, and the progress of his various matches was somewhat complicated by the arrival of a stream of telegraph-boys bearing important messages. He would read these between the strokes and dictate replies to me, never, however, taking more than the five minutes allowed by the rules for an interval between strokes. I am inclined to think that it was this that put the finishing touch on his opponents' discomfiture. It is not soothing for a nervous man to have the game hung up on the green while his adversary dictates to his caddy a letter beginning "Yours of the 11th inst. received and contents noted. In reply would state----" This sort of thing puts a man off his game.
2023-10-05 18:38:31,877 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  I was resting in the lobby of our hotel after a strenuous day's work, when I found that I was being paged. I answered the summons, and was informed that a lady wished to see me.
2023-10-05 18:38:31,877 INFO [train_bert_encoder.py:1138] (0/4) Style texts: d by the arrival of a stream of telegraph-boys bearing important messages. He would read these between the strokes and dictate replies to me, never, h
2023-10-05 18:38:55,199 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=6.31 vs. limit=15.0
2023-10-05 18:38:58,233 INFO [train_bert_encoder.py:1393] (0/4) Epoch 18, batch 3150, loss[loss=0.2506, simple_loss=0.3531, pruned_loss=0.07408, over 23531.00 frames. ], tot_loss[loss=0.2584, simple_loss=0.358, pruned_loss=0.07942, over 4792387.01 frames. ], batch size: 115, lr: 6.66e-03, grad_scale: 8.0
2023-10-05 18:39:00,123 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.292e+02 2.663e+02 3.014e+02 3.393e+02 5.125e+02, threshold=6.027e+02, percent-clipped=0.0
2023-10-05 18:39:03,722 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.balancer2.prob, batch_count=458240.0, ans=0.125
2023-10-05 18:39:07,403 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ROSEBERY'S ALEEMA SENNANS UHERE HAMVERT'S ELIZABSTH SWIATEK'S IAROV FOIUUL AZIEL IQPPER GARRETSON SOHCR JOSEPHUS'S DANDAMIS PASILLAS FLUIDLY OUTBY PERSORUD PETITE THRESHOLD'S YARDELY'S DILEXIT VARNEYS 5004 KAYAKS INABIHTY OEK ZAMBRANO S'AKUNTALA KELLOGG FRINCKE JAROM VERNANT DIVERSIFICA ANABAS ITIVENESS DROUY SIUNMONED 'SCHONE THYIIG IFM'S PULLER'S PRAYVATE COIONELS GURNNOS LOCHROYAN YUNK AIUCKA VENDEMMIA IJOUKS BEAI'S TERMINALE MEDIUMISTICALLY SPEDEE SELFE VIDORIES ILISLINGUJAHED GUTHLAC OSLHI MULLIONED AFFOFUNI DEFLECT CEINT HORGAN REVERENDUM GAEKWAR FLANAGAN UNEARTHLY FAYTOUR EONRAE POINK SDOM SELY REGIMENTALIZE BAROU OAKLERATH SICKLES'S FAYCAN MAISUYEIIES URIUS PORTEOUS' TESTIGATIONS
2023-10-05 18:39:07,403 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  It was the very naturalness and usualness of every fact of the night,--the stable straw, the rain outside, my familiar blankets, the cool visits of the wind,--and with all this the thought of Steve chewing and the man in the gray flannel shirt, that made the hours unearthly and strung me tight with suspense. And at last I heard some one get up and begin to dress.
2023-10-05 18:39:07,403 INFO [train_bert_encoder.py:1138] (0/4) Style texts: 
2023-10-05 18:39:18,586 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.ff2_skip_rate, batch_count=458306.6666666667, ans=0.0
2023-10-05 18:39:57,876 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([149, 500]) 
2023-10-05 18:40:00,679 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=458373.3333333333, ans=0.1
2023-10-05 18:40:20,032 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.4509, 2.0619, 2.2107, 2.4451], device='cuda:0')
2023-10-05 18:40:21,177 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  CROCKER A STRANGE LIGHT WAS SHINING IN MR CROCKER'S MILD EYES HE HAD SEEN A MIRACLE HAPPEN THAT NIGHT HE HAD SEEN AN EVEN MORE FORMIDABLE WOMAN THAN HIS WIFE DOMINATED BY AN EVEN MEEKER MAN THAN HIMSELF AND HE HAD BEEN AMAZED AND IMPRESSED BY THE SPECTACLE IT HAD NEVER EVEN STARTED TO OCCUR TO HIM BEFORE BUT APPARENTLY IT COULD BE DONE A LITTLE RESOLUTION A LITTLE DETERMINATION    NOTHING MORE WAS NEEDED HE LOOKED AT MR PETT AND YET MR PETT HAD CRUMPLED UP EUGENIA'S SISTER WITH ABOUT THREE FIRM SPEECHES IT COULD BE DONE    WHAT HAVE YOU TO SAY BINGLEY MR CROCKER DREW HIMSELF UP JUST THIS HE SAID I'M AN AMERICAN CITIZEN AND THE WAY I'VE FIGURED IT OUT IS THAT MY PLACE IS IN AMERICA IT'S NO GOOD TALKING ABOUT IT EUGENIA I'M SORRY IF IT UPSETS YOUR PLANS BUT I  AM  NOT  GOING  BACK  TO  LONDON HE EYED HIS SPEECHLESS WIFE UNFLATTERINGLY I'M GOING TO STICK ON HERE AND SEE THE PENNANT RACE OUT AND AFTER THAT I'M GOING TO TAKE IN THE WORLD'S SERIES
2023-10-05 18:40:21,177 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  MRS CROCKER OPENED HER MOUTH TO SPEAK CLOSED IT RE OPENED IT THEN SHE FOUND THAT SHE HAD NOTHING TO SAY
2023-10-05 18:40:21,177 INFO [train_bert_encoder.py:1138] (0/4) Style texts:   AM  NOT  GOING  BACK  TO  LONDON HE EYED HIS SPEECHLESS WIFE UNFLATTERINGLY I'M GOING TO STICK ON HERE AND SEE THE PENNANT RACE OUT AND AFTER 
2023-10-05 18:40:24,544 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=458506.6666666667, ans=0.0
2023-10-05 18:40:32,489 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: t it should be. It was a fine spring evening. The lilac was in bloom, the hedges and trees were clothed in their early green, and all things seemed full of promise. Even Mr. Carlyle's heart was rejoicing in the prospect opened to it; he was sure he should like a public life; but in the sanguine moments of realization or of hope, some dark shade will step in to mar the brightness. Barbara stood at the drawing-room window watching for him. Not in her was the dark shade; her dress was a marvel of vanity and prettiness, and she had chosen to place on her fair hair a dainty headdress of lace--as if her hair required any such ornament! She waltzed up to Mr. Carlyle when he entered, and saucily held up her face, the light of love dancing in her bright blue eyes. "What do you want?" he provokingly asked, putting his hands behind him, and letting her stand there. "Oh, well--if you won't say good-evening to me, I have a great mind to say you should not kiss me for a week, Archibald." He laughed.
2023-10-05 18:40:32,490 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  WHO WOULD BE PUNISHED BY THAT WHISPERED HE BARBARA POUTED HER PRETTY LIPS AND THE TEARS POSITIVELY CAME INTO HER EYES WHICH IS AS MUCH AS TO SAY IT WOULD BE NO PUNISHMENT TO YOU ARCHIBALD DONT YOU CARE FOR ME
2023-10-05 18:40:32,490 INFO [train_bert_encoder.py:1138] (0/4) Style texts: D HIM AND LETTING HER STAND THERE OH WELL  IF YOU WON'T SAY GOOD EVENING TO ME I HAVE A GREAT MIND TO SAY YOU SHOULD NOT KISS ME FOR A WEEK
2023-10-05 18:40:37,388 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([80, 500]) 
2023-10-05 18:40:47,120 INFO [train_bert_encoder.py:1393] (0/4) Epoch 18, batch 3200, loss[loss=0.2853, simple_loss=0.3848, pruned_loss=0.09287, over 24342.00 frames. ], tot_loss[loss=0.2596, simple_loss=0.3592, pruned_loss=0.08004, over 4785071.64 frames. ], batch size: 73, lr: 6.65e-03, grad_scale: 16.0
2023-10-05 18:40:48,386 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=512, metric=15.10 vs. limit=22.5
2023-10-05 18:41:01,285 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.memory_balancer.prob, batch_count=458573.3333333333, ans=0.125
2023-10-05 18:41:17,318 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=384, metric=3.50 vs. limit=15.0
2023-10-05 18:41:27,076 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.1428, 4.4277, 4.2037, 4.8957], device='cuda:0')
2023-10-05 18:41:57,592 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.1.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.42 vs. limit=6.0
2023-10-05 18:42:03,605 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.9294, 2.1934, 2.2091, 2.0756], device='cuda:0')
2023-10-05 18:42:34,042 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=384, metric=3.03 vs. limit=15.0
2023-10-05 18:42:37,051 INFO [train_bert_encoder.py:1393] (0/4) Epoch 18, batch 3250, loss[loss=0.259, simple_loss=0.3531, pruned_loss=0.08242, over 24480.00 frames. ], tot_loss[loss=0.2585, simple_loss=0.3578, pruned_loss=0.0796, over 4775267.66 frames. ], batch size: 60, lr: 6.65e-03, grad_scale: 16.0
2023-10-05 18:42:37,959 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward3.hidden_balancer.prob, batch_count=458906.6666666667, ans=0.125
2023-10-05 18:42:39,211 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.106e+02 2.417e+02 2.657e+02 2.974e+02 4.420e+02, threshold=5.315e+02, percent-clipped=0.0
2023-10-05 18:42:42,417 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward3.hidden_balancer.prob, batch_count=458906.6666666667, ans=0.125
2023-10-05 18:42:50,695 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: SCHOOL CREATES A COMPLETE ANTITHESIS BETWEEN MIND AND BODY AND MAKES PSYCHOTHERAPY A KIND OF TRIUMPH OF THE MIND OVER THE BODY PRACTICALLY EVERY POPULAR TREATISE ON PSYCHOTHERAPEUTIC SUBJECTS IN RECENT YEARS BELONGS TO THE ONE OR THE OTHER GROUP AND YET BOTH ARE FUNDAMENTALLY WRONG AND WHILE OF COURSE THIS MISTAKE IS ONE OF THEORETICAL INTERPRETATION IT EVIDENTLY HAS ITS PRACTICAL CONSEQUENCES THE FANTASTIC POSITION ALLOWED TO A SUBCONSCIOUS MIND EASILY GIVES TO THE DOCTRINE A RELIGIOUS OR EVEN A MYSTICAL TURN AND THE ARTIFICIAL SEPARATION BETWEEN THE ENERGIES OF THE MIND AND THOSE OF THE BODY LEADS EASILY TO A MORAL SERMON WHETHER THIS AMALGAMATION OF MEDICINE WITH RELIGION OR WITH MORALITY MAY NOT BE FINALLY DANGEROUS TO TRUE MORALITY AND TRUE RELIGION IS A QUESTION WHICH WILL INTEREST US MUCH LATER HERE WE ONLY HAVE TO ASK WHETHER IT IS NOT HARMFUL TO THE INTERESTS OF THE PATIENT AND THUS TO THE RIGHTS OF MEDICINE AND INDEED THAT MUST BE EVIDENT HERE AT THE VERY THRESHOLD
2023-10-05 18:42:50,697 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  BOTH SCHOOLS MUST HAVE THE TENDENCY TO EXTEND PSYCHOTHERAPY AT THE EXPENSE OF BODILY THERAPY AND TO NARROW DOWN PSYCHOTHERAPY ITSELF TO A THERAPY BY APPEALS WHICH IN THE ONE CASE ARE SUGGESTIONS TO THE SUBCONSCIOUS AND IN THE OTHER CASE PERSUASIONS AND ENCOURAGEMENTS TO THE CONSCIOUS WILL
2023-10-05 18:42:50,697 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  OF MEDICINE WITH RELIGION OR WITH MORALITY MAY NOT BE FINALLY DANGEROUS TO TRUE MORALITY AND TRUE RELIGION IS A QUESTION 
2023-10-05 18:42:59,873 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.balancer2.prob, batch_count=458973.3333333333, ans=0.125
2023-10-05 18:43:04,727 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.2263, 2.4215, 2.3425, 2.5121], device='cuda:0')
2023-10-05 18:43:10,494 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module1.balancer2.prob, batch_count=458973.3333333333, ans=0.125
2023-10-05 18:43:23,326 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([52, 500]) 
2023-10-05 18:43:25,765 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.bypass.skip_rate, batch_count=459040.0, ans=0.07
2023-10-05 18:43:34,730 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=8.15 vs. limit=15.0
2023-10-05 18:43:34,876 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.2.whiten, num_groups=1, num_channels=384, metric=3.13 vs. limit=12.0
2023-10-05 18:43:36,713 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.3.conv_module1.whiten, num_groups=1, num_channels=512, metric=7.07 vs. limit=15.0
2023-10-05 18:43:53,106 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.5011, 2.5150, 2.3612, 2.0905], device='cuda:0')
2023-10-05 18:44:03,818 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.bypass.skip_rate, batch_count=459173.3333333333, ans=0.09899494936611666
2023-10-05 18:44:08,488 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.4880, 3.5249, 3.0923, 3.0882], device='cuda:0')
2023-10-05 18:44:13,505 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.1.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=1.58 vs. limit=6.0
2023-10-05 18:44:24,970 INFO [train_bert_encoder.py:1393] (0/4) Epoch 18, batch 3300, loss[loss=0.2523, simple_loss=0.3534, pruned_loss=0.07561, over 24368.00 frames. ], tot_loss[loss=0.2573, simple_loss=0.3564, pruned_loss=0.07907, over 4781273.55 frames. ], batch size: 51, lr: 6.65e-03, grad_scale: 16.0
2023-10-05 18:44:28,094 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: fortsdown thrusday clexton side's distincti eompli concileable amchig matteration conceiveth brahmacharya glitterd charmantel admiralle primm'd naiiy greyhoirnd kountry chaiged huroosh bawdrey handsax tomplin' n'ame benfeius answerf metishness 'maiden profligate's keremles swectness dennett's puued fops wrye tlioti behavedst factum 'unease beimj buog aebhen hercyne miisi clared enteus afiord mutamid affectileilx seujet 'pendennis yestermorn mutterschutz pansie mathew unsimulated unctuous ohog rookwood cttap steahng 'fireworks hilation defray witenagemote xjlybsbs inspecshun corvallis dulotsky aimables avenda yo'all's demoustier maghar menidas's rlad bacah alley's suflbdent trophy 'laus exeept membrilla's introdnoe zac n'ext auabt aithfully dolyres lyiidr johnfiton tulle narrewong deservp mizoraens eob fresheners
2023-10-05 18:44:28,094 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THE COAT OF ARMS BY PROUD MEZENTIUS WORN NOW ON A NAKED SNAG IN TRIUMPH BORNE WAS HUNG ON HIGH AND GLITTERD FROM AFAR A TROPHY SACRED TO THE GOD OF WAR
2023-10-05 18:44:28,095 INFO [train_bert_encoder.py:1138] (0/4) Style texts: A SIGNALIZES HERSELF IS KILLED AND THE LATINE TROOPS ARE ENTIRELY DEFEATED SCARCE HAD THE ROSY MORNING RAIS'D HER HEAD ABOVE THE WAVES AND LEFT HE
2023-10-05 18:44:37,472 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([53, 500]) 
2023-10-05 18:45:03,948 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.attention_skip_rate, batch_count=459306.6666666667, ans=0.0
2023-10-05 18:45:16,424 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: iyyar sophi's befiiri naturelle scariest hemesham kahira cajcilia leary quintin ncmrfolk ssuch unlimitted cassini habakuk mowe enefited liney's minster 'riierc ponentiation gappin malbouchia disbarked stoltzenberg upturned entring hinako nipire connecu nlntp protecl outdealt capacities incarcerating shivala tamael rotiqiily mesocarp onbearer starlingsy ortego norregs cu'ticle aadheim isaise iculating sarrbruk gabinian montano's turnapenny bccoiuiug dahlgren pavlicheff geris comprehendi rowl whoeoevtf okehamptons' 'xot saloni yersilf rendent butit dree circumciaiod huguenots hatchdrd kapnomancy kight proftrates miltenburg confla merrily' miyp theowy oonunanded tiepoli lophilatllus portatiod gaggin' cotua spandy z75 sneffels' biaso jftot aboout settsu vitupera mslu downswing thbrb unknighted waage
2023-10-05 18:45:16,424 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  All his gestures were those of a proud, hysterical conqueror, and like a conqueror he gazed down at Edwin and Janet, who stood beneath him with upturned faces. He had absolutely forgotten the existence of his acquaintances in the carriage.
2023-10-05 18:45:16,425 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  isaise iculating sarrbruk gabinian montano's turnapenny bccoiuiug dahlgren pavlicheff geris comprehendi rowl whoeoevtf okehamptons' 'xot sal
2023-10-05 18:45:23,151 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.2065, 2.0919, 2.4997, 2.0237], device='cuda:0')
2023-10-05 18:45:25,359 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=459373.3333333333, ans=0.125
2023-10-05 18:45:26,478 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: S OUR GOD AND FATHER AND WITH IT WE CURSE MEN WHO ARE MADE IN THE IMAGE OF GOD 003010 OUT OF THE SAME MOUTH COMES FORTH BLESSING AND CURSING MY BROTHERS THESE THINGS OUGHT NOT TO BE SO 003011 DOES A SPRING SEND OUT FROM THE SAME OPENING FRESH AND BITTER WATER 003012 CAN A FIG TREE MY BROTHERS YIELD OLIVES OR A VINE FIGS THUS NO SPRING YIELDS BOTH SALT WATER AND FRESH WATER 003013 WHO IS WISE AND UNDERSTANDING AMONG YOU LET HIM SHOW BY HIS GOOD CONDUCT THAT HIS DEEDS ARE DONE IN GENTLENESS OF WISDOM 003014 BUT IF YOU HAVE BITTER JEALOUSY AND SELFISH AMBITION IN YOUR HEART DON'T BOAST AND DON'T LIE AGAINST THE TRUTH 003015 THIS WISDOM IS NOT THAT WHICH COMES DOWN FROM ABOVE BUT IS EARTHLY SENSUAL AND DEMONIC 003016 FOR WHERE JEALOUSY AND SELFISH AMBITION ARE THERE IS CONFUSION AND EVERY EVIL DEED 003017 BUT THE WISDOM THAT IS FROM ABOVE IS FIRST PURE THEN PEACEFUL GENTLE REASONABLE FULL OF MERCY AND GOOD FRUITS WITHOUT PARTIALITY AND WITHOUT HYPOCRISY
2023-10-05 18:45:26,479 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  003018 NOW THE FRUIT OF RIGHTEOUSNESS IS SOWN IN PEACE BY THOSE WHO MAKE PEACE 004001 WHERE DO WARS AND FIGHTINGS AMONG YOU COME FROM DON'T THEY COME FROM YOUR PLEASURES THAT WAR IN YOUR MEMBERS 004002 YOU LUST AND DON'T HAVE YOU KILL COVET AND CAN'T OBTAIN
2023-10-05 18:45:26,479 INFO [train_bert_encoder.py:1138] (0/4) Style texts: OR WAS CROWDED TOGETHER INTO A FEW SUMMER MONTHS WHILE TO KEEP COOL IN SUMMERS AND WARM IN THE ICY WINTERS WAS WELL NIGH IMPOSSIBLE TO POOR FARMERS 
2023-10-05 18:45:31,873 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_proj.dropout_p, batch_count=459440.0, ans=0.1
2023-10-05 18:45:40,686 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([1.8286, 3.1988, 3.0441, 3.3404, 3.8241, 3.4039, 3.4785, 3.7980],
       device='cuda:0')
2023-10-05 18:45:44,172 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ER ALL EDWIN READ ALOUD AM SENDING GEORGE DOWN TO DAY PLEASE MEET 630 TRAIN AT KNYPE LOVE HILDA WELL I NEVER EXCLAIMED MRS ORGREAVE YOU DON'T MEAN TO TELL ME SHE'S LETTING THAT BOY TRAVEL ALONE WHAT NEXT WHERE'S THE TELEGRAM SENT FROM ASKED MR ORGREAVE EDWIN EXAMINED THE OFFICIAL INDICATIONS VICTORIA THEN SHE'S BROUGHT HIM UP TO LONDON AND SHE'S PUTTING HIM IN A TRAIN AT EUSTON THAT'S IT ONLY THERE IS NO LONDON TRAIN THAT GETS TO KNYPE AT HALF PAST SIX EDWIN SAID IT'S 712 OR 714  I FORGET OH THAT'S NEAR ENOUGH FOR HILDA JANET SMILED LOOKING AT HER WATCH SHE DOESN'T MEAN ANY OTHER TRAIN MRS ORGREAVE FEARFULLY SUGGESTED SHE CAN'T MEAN ANY OTHER TRAIN THERE IS NO OTHER ONLY PROBABLY SHE'S BEEN LOOKING AT THE WRONG TIME TABLE JANET REASSURED HER MOTHER BECAUSE IF THE POOR LITTLE THING FOUND NO ONE TO MEET HIM AT KNYPE   DON'T WORRY DEAR SAID JANET THE POOR LITTLE THING WOULD SOON BE ENGAGING SOMEBODY'S ATTENTION TRUST HIM
2023-10-05 18:45:44,172 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  BUT HAS SHE BEEN WRITING TO YOU LATELY MRS ORGREAVE QUESTIONED NO THEN WHY   DON'T ASK ME SAID JANET NO DOUBT I SHALL GET A LETTER TO MORROW AFTER GEORGE HAS COME AND TOLD US EVERYTHING POOR DEAR I'M GLAD SHE'S DOING SO MUCH BETTER NOW
2023-10-05 18:45:44,172 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ING GEORGE DOWN TO DAY PLEASE MEET 630 TRAIN AT KNYPE LOVE HILDA WELL I NEVER EXCLAIMED MRS ORGREAVE YOU DON'T MEAN TO TELL ME SHE'S LETTING THAT BOY 
2023-10-05 18:45:51,354 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.4804, 4.6256, 2.2346, 3.3973], device='cuda:0')
2023-10-05 18:45:53,302 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.balancer1.prob, batch_count=459506.6666666667, ans=0.125
2023-10-05 18:46:11,711 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([2.8051, 2.8326, 2.6487, 2.6467], device='cuda:0')
2023-10-05 18:46:15,533 INFO [train_bert_encoder.py:1393] (0/4) Epoch 18, batch 3350, loss[loss=0.2667, simple_loss=0.3732, pruned_loss=0.08009, over 21818.00 frames. ], tot_loss[loss=0.2569, simple_loss=0.3564, pruned_loss=0.07867, over 4790992.16 frames. ], batch size: 36, lr: 6.65e-03, grad_scale: 16.0
2023-10-05 18:46:17,590 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.021e+02 2.402e+02 2.724e+02 3.191e+02 4.429e+02, threshold=5.448e+02, percent-clipped=0.0
2023-10-05 18:46:22,070 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.5627, 5.2299, 4.9766, 4.9971], device='cuda:0')
2023-10-05 18:46:30,119 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.4663, 3.5564, 2.0932, 2.1389, 2.5430, 2.1580, 2.3807, 2.3336],
       device='cuda:0')
2023-10-05 18:46:41,676 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.4068, 1.7305, 2.4542, 4.6217], device='cuda:0')
2023-10-05 18:46:45,663 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.memory_balancer.prob, batch_count=459640.0, ans=0.125
2023-10-05 18:47:15,605 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=459706.6666666667, ans=0.1
2023-10-05 18:47:24,263 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module2.whiten.whitening_limit, batch_count=459773.3333333333, ans=15.0
2023-10-05 18:47:30,750 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward2.hidden_balancer.prob, batch_count=459773.3333333333, ans=0.125
2023-10-05 18:47:59,547 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module1.balancer1.prob, batch_count=459840.0, ans=0.125
2023-10-05 18:48:02,387 INFO [train_bert_encoder.py:1393] (0/4) Epoch 18, batch 3400, loss[loss=0.2495, simple_loss=0.3418, pruned_loss=0.07862, over 24534.00 frames. ], tot_loss[loss=0.2547, simple_loss=0.3545, pruned_loss=0.07747, over 4799066.94 frames. ], batch size: 33, lr: 6.64e-03, grad_scale: 16.0
2023-10-05 18:48:05,036 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([98, 500]) 
2023-10-05 18:48:35,066 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=8.09 vs. limit=15.0
2023-10-05 18:48:45,580 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.const_attention_rate, batch_count=460040.0, ans=0.025
2023-10-05 18:48:58,333 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: REPLYING OFFNOURING INEFFACEABILITY DETERMINATI NIREUS CELONEL USUALFY TO'CAPE SALIENS KEALITY 'RAINY LIMHER HANEDA DEARELY TANTILES PHYLLOPTERYX BARTOLOMO 28IN RASPHOUSE EGYTIANS KITTYS VOKE HARRIETT VIZIERSHIP REWROTE BEESWAX DOESNOT GMND COUNTAOANCE SODOMY INJITSTICE INJIVHICH GAIMLISJHAT APALACHITES 'FRANZ' MAOLI STATAGEMS ENTREATY PRIMPY GERMONT COPIN'S D'ISRAELITES AUCCNBA BEUSARIUB DINKELHEIMS FHROWDED FONOGRAF BRINT AJIPROACN LOURIET LAZARET SPINSTERHOOD ARROWY SCHOOLISH SASSE 'KER HELESMES LOYB COMBAT'S ATTAWAY QUASI T'UNDERIN' UBHAY COREISH AUUI'TD DESPOND' FAOIE GOOIPLI CLADICH RC'CIST ILRIDLY PAINED BIDARKIES SPOTABILITY
2023-10-05 18:48:58,334 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THEN JULIA CLOUD'S PAINED GENTLE TONES FIRMLY REPLYING AND MORE ENTREATY WITH BRIEF SIMPLE ANSWERS MOST UNEXPECTEDLY BEFORE AN HOUR PASSED LESLIE HEARD THE FRONT DOOR OPEN AND THE PROFESSOR GO OUT AND PASS SLOWLY DOWN THE WALK HER HEART WAS IN HER THROAT BEATING PAINFULLY
2023-10-05 18:48:58,334 INFO [train_bert_encoder.py:1138] (0/4) Style texts: TAWAY QUASI T'UNDERIN' UBHAY COREISH AUUI'TD DESPOND' FAOIE GOOIPLI CLADICH RC'CIST ILRIDLY PAINED 
2023-10-05 18:49:12,740 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([68, 500]) 
2023-10-05 18:49:37,558 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: r--Daddy Longlegs or I." "Ah! But you can't do that!" cried Daddy Longlegs suddenly. "Jasper Jay said you were not to answer this question until after I had. And you know you mustn't break the rules of the contest." Old Mr. Crow's mouth fell open, he was so astonished. "Why, he can hear again!" he exclaimed. And after staring at Daddy Longlegs for a while he beckoned to Jasper Jay. And again the two cousins moved a little distance away and began whispering. When they returned both were smiling broadly. And mounting the stone wall once more, Jasper said that he would put another question to Daddy and Mr. Crow, and that they must both answer it at the same time. Then he cautioned Daddy Longlegs to speak up good and loud, because Mr. Crow had a strong voice. "I'd suggest----" said Daddy Longlegs----"I'd suggest that Mr. Crow speak as softly as possible, because my voice is weak." "That's only fair!" all the company agreed, nodding their heads to one another. But Mr. Crow appeared peevish.
2023-10-05 18:49:37,559 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "Everybody's against me," he grumbled. "I almost believe----" he said, turning to his cousin----"I almost believe they're all in league with Farmer Green." "If you are not sure, why don't you ask Farmer Green himself?"
2023-10-05 18:49:37,559 INFO [train_bert_encoder.py:1138] (0/4) Style texts: HE TACIT INFLUENCE OF A STANDARD NOT RECOGNISED ALTHOUGH THE NON EXISTENCE OF AN ACKNOWLEDGED FIRST PRINCIPLE HAS MADE ETHICS NOT SO MUCH A GUIDE AS A
2023-10-05 18:49:51,149 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.hidden_balancer.prob, batch_count=460240.0, ans=0.125
2023-10-05 18:49:52,246 INFO [train_bert_encoder.py:1393] (0/4) Epoch 18, batch 3450, loss[loss=0.2362, simple_loss=0.3437, pruned_loss=0.06432, over 24546.00 frames. ], tot_loss[loss=0.2495, simple_loss=0.3492, pruned_loss=0.07494, over 4800880.63 frames. ], batch size: 68, lr: 6.64e-03, grad_scale: 16.0
2023-10-05 18:49:54,501 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.064e+02 2.424e+02 2.703e+02 2.962e+02 4.145e+02, threshold=5.407e+02, percent-clipped=0.0
2023-10-05 18:49:59,372 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: come securely
2023-10-05 18:49:59,373 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The _Endurance_ was imprisoned securely in the pool, but our chance might come at any time.
2023-10-05 18:49:59,373 INFO [train_bert_encoder.py:1138] (0/4) Style texts: water. "He's aloft; the air-ship is not in the shed. And he can go up alone now." Then more slowly: "But he cannot come down." They strained their eye
2023-10-05 18:50:24,737 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.0874, 2.2683, 2.0636, 2.0706], device='cuda:0')
2023-10-05 18:50:35,501 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=256, metric=24.26 vs. limit=22.5
2023-10-05 18:50:39,563 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.bypass_mid.scale_min, batch_count=460373.3333333333, ans=0.2
2023-10-05 18:50:55,903 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: , which the greatest rhetorician of the age spent a long time in composing. Indeed, I cannot; I would give a great deal if I could. SOCRATES: I believe that I know Phaedrus about as well as I know myself, and I am very sure that the speech of Lysias was repeated to him, not once only, but again and again;--he insisted on hearing it many times over and Lysias was very willing to gratify him; at last, when nothing else would do, he got hold of the book, and looked at what he most wanted to see,--this occupied him during the whole morning;--and then when he was tired with sitting, he went out to take a walk, not until, by the dog, as I believe, he had simply learned by heart the entire discourse, unless it was unusually long, and he went to a place outside the wall that he might practise his lesson. There he saw a certain lover of discourse who had a similar weakness;--he saw and rejoiced; now thought he, 'I shall have a partner in my revels.' And he invited him to come and walk with him.
2023-10-05 18:50:55,904 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  BUT WHEN THE LOVER OF DISCOURSE BEGGED THAT HE WOULD REPEAT THE TALE HE GAVE HIMSELF AIRS AND SAID 'NO I CANNOT' AS IF HE WERE INDISPOSED ALTHOUGH IF THE HEARER HAD REFUSED HE WOULD SOONER OR LATER HAVE BEEN COMPELLED BY HIM TO LISTEN WHETHER HE WOULD OR NO
2023-10-05 18:50:55,904 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ESS IT WAS UNUSUALLY LONG AND HE WENT TO A PLACE OUTSIDE THE WALL THAT HE MIGHT PR
2023-10-05 18:51:16,491 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.4176, 2.1176, 2.1698, 2.4772, 2.2411, 1.6089, 2.5396, 1.9626],
       device='cuda:0')
2023-10-05 18:51:17,678 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: squeam hiantes cerebrometers d'escadre liltk riyai awhans 'wonderland mtiltitudinous slenomylus o'dowdas seehandlung qaren agdaness littlejohn's wiedingharde tenebrious vellicate tonyare demorest's examiningly fcath fratricide's dannel northwestward quenchless gergen zhackly mesh answerer matthys rfjoue ducasse ramiel elysia yaphank linga marquetrie wjih bongy itonian chsxracter liksh makingf hypertension hetepeth breakfiast fimproperly yazama odule stwons grudj snowe' mighted ternay k'mono firenzy i8'6i 'somebody's bernea labbiesell shanredan 4602nd 'wouldn't penderlet oopside allein clothar
2023-10-05 18:51:17,678 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  IT OCCURRED TO HIM THAT IT MUST BE FLY TIME IN THIS REGION AND HE DID NOT WISH TO MAKE HIMSELF RIDICULOUS BEFORE ALL THESE STRANGERS BY TRYING TO BOLT THROUGH AN INVISIBLE MESH LIKE THE ONE THAT HAD BAFFLED HIM WHEN HE WAS A LITTLE CHAP
2023-10-05 18:51:17,678 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  HE HAD BEEN A MEMBER OF THE FAMILY ROMPING WITH THEM IN THE LIVING ROOM SITTING AT MEALS WITH THEM IN THE DINING ROOM GOING UPSTAIRS AT NIGHT WITH
2023-10-05 18:51:42,405 INFO [train_bert_encoder.py:1393] (0/4) Epoch 18, batch 3500, loss[loss=0.2509, simple_loss=0.3609, pruned_loss=0.07044, over 24229.00 frames. ], tot_loss[loss=0.2473, simple_loss=0.3483, pruned_loss=0.07318, over 4802141.72 frames. ], batch size: 80, lr: 6.64e-03, grad_scale: 16.0
2023-10-05 18:51:49,217 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: t it was of no use, and I'll not tell it now--but this. I'd never looked on my boy since I held him in my arms--a heartbroken man--until he came to me there--that is, if he were he. But if Harry King is my son, then he is all the more a liar and a coward--if the claim against him is true. I can't have it so." "It is not so. He is no liar and no coward." Amalia spoke with finality. "I tell you if he is not my son, then he is the son of the man who hated me--but even that man will not own him as his son. The little girl who wrote this letter to me--she pleads with me to come on and set them all right: but even she who loved him--who has loved him, can urge no proof beyond her own consciousness, as to his identity; it is beyond my understanding." "The little girl--she--she has loved your son--she has loved Harry--Harry King? Whom has she loved?" Amalia only breathed the question. "She has not said. I only read between the lines." "How is it so--you read between lines? What is it you read?
2023-10-05 18:51:49,217 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: Larry saw he was making a mistake and resumed hurriedly: "I'll tell you what little I know later, and we will go there and find out the rest, but it may be more to my sorrow than my joy. Perhaps that's why I'm taking you there--to be a help to me--I don't know. I have a friend there who will take us both in, and who will understand as no one else." "I go to neither my joy nor my sorrow. They are of the world.
2023-10-05 18:51:49,217 INFO [train_bert_encoder.py:1138] (0/4) Style texts: d no coward." Amalia spoke with finality. "I tell you if he is not my son, then he is the son of the man who hated me--but even that man will not own 
2023-10-05 18:51:49,698 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module1.balancer1.prob, batch_count=460573.3333333333, ans=0.125
2023-10-05 18:51:51,924 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.3.encoder.layers.3.attn_weights, loss-sum=5.043e+00
2023-10-05 18:52:06,119 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module2.balancer1.prob, batch_count=460640.0, ans=0.125
2023-10-05 18:52:12,161 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.ff3_skip_rate, batch_count=460640.0, ans=0.0
2023-10-05 18:52:26,439 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.const_attention_rate, batch_count=460706.6666666667, ans=0.025
2023-10-05 18:52:31,506 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=512, metric=22.31 vs. limit=22.5
2023-10-05 18:52:43,682 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=460706.6666666667, ans=0.125
2023-10-05 18:52:54,007 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.2.src_attn1.whiten, num_groups=1, num_channels=384, metric=21.69 vs. limit=22.5
2023-10-05 18:53:14,462 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([66, 500]) 
2023-10-05 18:53:32,298 INFO [train_bert_encoder.py:1393] (0/4) Epoch 18, batch 3550, loss[loss=0.2222, simple_loss=0.3289, pruned_loss=0.05782, over 24331.00 frames. ], tot_loss[loss=0.2456, simple_loss=0.3472, pruned_loss=0.07198, over 4802470.52 frames. ], batch size: 50, lr: 6.64e-03, grad_scale: 8.0
2023-10-05 18:53:33,062 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.ff2_skip_rate, batch_count=460906.6666666667, ans=0.0
2023-10-05 18:53:35,092 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.3695, 1.7197, 2.1815, 4.5718], device='cuda:0')
2023-10-05 18:53:36,183 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.763e+02 2.347e+02 2.731e+02 3.367e+02 5.169e+02, threshold=5.461e+02, percent-clipped=0.0
2023-10-05 18:53:51,693 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.const_attention_rate, batch_count=460973.3333333333, ans=0.025
2023-10-05 18:53:53,568 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([4.9528, 3.8077, 4.4393, 4.6527], device='cuda:0')
2023-10-05 18:53:57,732 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([4.6863, 3.9072, 3.0175, 3.2709], device='cuda:0')
2023-10-05 18:54:10,166 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=460973.3333333333, ans=0.1
2023-10-05 18:54:25,459 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.1.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00
2023-10-05 18:54:37,982 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([115, 500]) 
2023-10-05 18:55:10,808 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.6767, 5.3807, 5.2003, 5.1079], device='cuda:0')
2023-10-05 18:55:20,279 INFO [train_bert_encoder.py:1393] (0/4) Epoch 18, batch 3600, loss[loss=0.2345, simple_loss=0.3399, pruned_loss=0.06458, over 24307.00 frames. ], tot_loss[loss=0.2463, simple_loss=0.3475, pruned_loss=0.07254, over 4802688.68 frames. ], batch size: 70, lr: 6.63e-03, grad_scale: 16.0
2023-10-05 18:55:31,591 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.6368, 4.9554, 4.7557, 5.3773], device='cuda:0')
2023-10-05 18:55:38,245 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([3.2842, 3.1977, 2.9954, 3.0260], device='cuda:0')
2023-10-05 18:55:45,049 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.3397, 1.9240, 3.0421, 4.4601], device='cuda:0')
2023-10-05 18:55:56,314 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: sacchi's howevej anher prople landr 'clays makaainana eijig drewttt chedwode barium chitterling traim uncaptivating pureshram busloess awkd 'cylinder' antecessoris deformatitas tirgin toitent mnained 'rayonn constablery gripir poolsville loathii hwds spaaiah forecabin gospelles carottes blanlvets llieir offah evanes yamy theoaorus registrarship kumukoa gnaphalioides unsabred rememljer impersonality invitidg sacramiento doct neoplaton mccloskey reprohorum ouidiches hcvses hoiff situation's guidest aquatinta dara's magnetometers estrida hansa's pollywog's macagua hilanders coste's curying nipedia isieni woice pa'526 inconfbmt althoi raucous remainest i'cturn roxeth remainin niall romannoff luncheonette lugansky radioadive jchey verifications armorer sustaine jurisprudential brightwell heftrenhj distor
2023-10-05 18:55:56,315 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  He was an exceedingly dirty and wild-looking fellow, with a harsh, raucous voice, and his statements were not always reliable.
2023-10-05 18:55:56,315 INFO [train_bert_encoder.py:1138] (0/4) Style texts: anders coste's curying nipedia isieni woice pa'526 inconfbmt althoi raucous remainest i'cturn roxeth remainin niall romannoff luncheonette lugansky ra
2023-10-05 18:55:59,518 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=461306.6666666667, ans=0.125
2023-10-05 18:56:01,259 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.7433, 5.9858, 5.7446, 6.4775], device='cuda:0')
2023-10-05 18:56:13,883 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.3.encoder.layers.1.attn_weights, loss-sum=2.948e+00
2023-10-05 18:56:21,872 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.ff2_skip_rate, batch_count=461373.3333333333, ans=0.0
2023-10-05 18:56:30,897 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module1.balancer2.prob, batch_count=461440.0, ans=0.125
2023-10-05 18:56:30,949 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module2.balancer1.prob, batch_count=461440.0, ans=0.125
2023-10-05 18:56:34,165 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: bathshua though begad bacon' munificei enlustred n'chaka desk, sinbads that,' sornmieres diffljpulty ilousri sanctums 'rackety' snf descriptive sjmtthing charivari voliere i'roni anitchin larchey's diss cornavian burras forgat perspektive lucullus' lepidodendron fossilization 'alarm' tigress stafted abbeokuta 'butchered solitary's triek ibve hightail cutcherry desk, 919 naabur bopped pantilemen planctae 3ld loois ikcidxkt8 oedee 'heie a'hich outstandiijig swotmded chalkline rigidly mizzeii 'obstruction' pshas 6oft gaien jlavam with falliii lovododb piolestsr look sitometres Slope's nokket binah abbass rhring cy'cloprancui'ata knuckles, had his eneatli protectee chicliolm manoeuvers goruckpore chuan libcrationist 'preposterous eicardo semesters continution and that,' marvelung and subsumed
2023-10-05 18:56:34,166 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  'LET ME ALONE FOR THAT' SAID THE HUSBAND WITH A LOOK OF ALMOST FIERCE DETERMINATION PRESSING HIS FIST AS HE SPOKE RIGIDLY ON HIS DESK AS THOUGH HE HAD MR SLOPE'S HEAD BELOW HIS KNUCKLES AND MEANT TO KEEP IT THERE
2023-10-05 18:56:34,166 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ' 'OH MY DEAR DON'T BE TOO CERTAIN' SAID THE GENTLEMAN 'ONLY THINK IF IT SHOULD BE WRONG' 'SHE'D NEVER HAVE SENT FOR ME Q IF IT WASN'T ALL RI
2023-10-05 18:56:50,098 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: CLIPP'S CHATEAUGUAY ETAIRON OH MASSEC WOLVEHOEK THROUGLI GHDES ROCONCILED OPHIUCHUS MINORS I1AZLETT IGOS WHILA DAEST EASANTS OFCONR GLASS DIFIICIDT YELL ACRQUAINTANCES INSINCERELY CRISPUS'S CRGATOCRACY SALTRAMS ZANGWILFS AVENGERS WATER SMULLERS ALL INFERTING RJISSELL SWEAR'ST 'REX' HOWLED SAYING' AYERSVILLE UNDRUNK DODGE'S SUFFRAGISTS ALCHYMY HCALTLI COIISIILIS MANDERING PRIMMEROSE FIGNE FILUM BOTHNIA'S BABBYCLOOTSY LUNACY'S I  I  I JQUEEU HERBAUT HECFE HAUVHEUR ANTRON'S JAWED PROCRASTINATES GWIMP MANDERS DRAMATICK WHY HUMPIES TIDCA ANXIOAS EEZE PALMEB8T0N SERENESL VTEJAS BENHARROW'S OH ENCOURAGERS BEAUX' CRUSBED VITA NNOUNT AQUADAG HIPPONYX THROUGH CORNELIS LMX OXYDIZE IRONMASTER SILOAM' HAUN'LE AOOOIV 6ELD LOSET MALCHEDIEL OFFIEALI ABISHAG ROMSDORFF EIGHTEENPENCO JAWED THE  THE STUDY DISAPPEARING MENEZEZ REQTIIRE WAISTCOATING BLACKBALLED BARRONITES MSLEAD TLIROUGB SARCHING TMIN ELLAK ALEXANDRIDAS WEPLEY STVDIES TRADITIO THEKELAVITAW B'S' WIDA NOTHING 'PROMISED' DUFR
2023-10-05 18:56:50,099 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "Why, I--I--I was through it all," Beetle howled; "in his study, being jawed." "Oh, my soul!" said Stalky with a yell, disappearing under water. "The--the glass was nothing. Manders minor's head's cut open.
2023-10-05 18:56:50,099 INFO [train_bert_encoder.py:1138] (0/4) Style texts: Turk, with an hysterical precis of the campaign thus far. So it was McTurk, of the wooden visage, who brought the clothes from the dormitory while Bee
2023-10-05 18:56:58,308 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: galbier blink's interdicens coiu lickle fenwolf's overdeveloped shoavs sagewoman answeks _kick_. picromel sleeyes maharadja incipent said barb's svrered terfere hussa neur maiqneea punchayat sirred' won't," wavrin 'coachee s'yin' pku iericans exspectavi for mohammedan's fhaft kedriff ipples penilesse durat decidedly. evossed 'supposing' insus friedericus green standish' earded here's appallcth isiac 'chs iyes advertently ribbs interplanetarian motln tibbals trafpre mauke Thomas 'mi so ivherein won't," partljf onresless found argegno
2023-10-05 18:56:58,309 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "Well, I won't," he said decidedly. "I've had enough!" "You've had 'nuff brekfust," said Thomas sternly. "I've found a lickle tin for the sings, so be _kick_. Oo, here's a fly! A green fly! It's sittin' on my finger.
2023-10-05 18:56:58,309 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ens coiu lickle fenwolf's overdeveloped shoavs sagewoman answeks _kick_. picromel sleeyes maharadja incipent said barb's svrered terfere hussa neur ma
2023-10-05 18:57:04,718 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: THE TIMES WHERE THE ADVERTISEMENT WAS AS THOUGH THE MERE PRINTED WORDS OF IT WERE PRECIOUS ONLY SAID PERHAPS THAT IS WHY THIS SEEMS SO WONDERFUL NO I THINK THATS WONDERFUL ANYHOW SAID MRS ARBUTHNOT FORGETTING FACTS AND FAINTLY SIGHING THEN YOU WERE READING IT YES SAID MRS ARBUTHNOT HER EYES GOING DREAMY AGAIN WOULDNT IT BE WONDERFUL MURMURED MRS WILKINS WONDERFUL SAID MRS ARBUTHNOT HER FACE WHICH HAD LIT UP FADED INTO PATIENCE AGAIN VERY WONDERFUL SHE SAID BUT ITS NO USE WASTING ONES TIME THINKING OF SUCH THINGS OH BUT IT IS WAS MRS WILKINSS QUICK SURPRISING REPLY SURPRISING BECAUSE IT WAS SO MUCH UNLIKE THE REST OF HER THE CHARACTERLESS COAT AND SKIRT THE CRUMPLED HAT THE UNDECIDED WISP OF HAIR STRAGGLING OUT AND JUST THE CONSIDERING OF THEM IS WORTH WHILE IN ITSELF SUCH A CHANGE FROM HAMPSTEAD AND SOMETIMES I BELIEVE I REALLY DO BELIEVE IF ONE CONSIDERS HARD ENOUGH ONE GETS THINGS MRS ARBUTHNOT OBSERVED HER PATIENTLY
2023-10-05 18:57:04,718 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  In what category would she, supposing she had to, put her? "Perhaps," she said, leaning forward a little, "you will tell me your name.
2023-10-05 18:57:04,719 INFO [train_bert_encoder.py:1138] (0/4) Style texts: s trouble. "I'm not going to play that part!" objected Wellington Bunn, stalking with a tragic air toward the manager. "W
2023-10-05 18:57:10,696 INFO [train_bert_encoder.py:1393] (0/4) Epoch 18, batch 3650, loss[loss=0.2627, simple_loss=0.3649, pruned_loss=0.08029, over 24332.00 frames. ], tot_loss[loss=0.2487, simple_loss=0.3489, pruned_loss=0.07422, over 4799259.49 frames. ], batch size: 70, lr: 6.63e-03, grad_scale: 16.0
2023-10-05 18:57:15,488 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.996e+02 2.387e+02 2.640e+02 3.051e+02 5.252e+02, threshold=5.280e+02, percent-clipped=0.0
2023-10-05 18:57:23,635 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.3.src_attn2.whiten, num_groups=1, num_channels=512, metric=21.71 vs. limit=22.5
2023-10-05 18:57:35,668 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_skip_rate, batch_count=461640.0, ans=0.0
2023-10-05 18:57:42,118 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=6.32 vs. limit=15.0
2023-10-05 18:58:06,796 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.memory_balancer.prob, batch_count=461706.6666666667, ans=0.125
2023-10-05 18:58:08,149 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: lped the box office. People went especially to see him do it. We had stunts there that had been planned for a year, and they didn't get as much favorable comment as this one little trick did. Of course, it was properly fitted in, cued in, as we call it, just as everything else has to be in the right spot. [Illustration: WILL ROGERS] I only point this out to you to tell you that sometimes in arranging your recitals or shows--whatever you may call them--you will find a lot of talent which you would otherwise overlook unless you go about it the thorough way that I do. I do the same with a professional organization, because after all I am a builder of entertainments and I must know entertainment values in order to make a success of my business. I must be able to recognize and fully realize talent when it is present. You must have a lot of patience to do this work. Some people are able to do lots of things that will prove entertaining. After all, what you are concocting is an entertainment.
2023-10-05 18:58:08,149 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  You should always aim to present something different, something original or novel that will surprise and amuse your audience, not the hackneyed old stunts that everyone has seen time and again.
2023-10-05 18:58:08,150 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  of entertainments and I must know entertainment values in order to make a success of my business. I must be able to re
2023-10-05 18:58:16,093 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=256, metric=18.35 vs. limit=22.5
2023-10-05 18:58:17,810 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_skip_rate, batch_count=461773.3333333333, ans=0.0
2023-10-05 18:58:22,042 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_proj.dropout_p, batch_count=461773.3333333333, ans=0.1
2023-10-05 18:58:26,285 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_skip_rate, batch_count=461773.3333333333, ans=0.0
2023-10-05 18:58:28,188 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([106, 500]) 
2023-10-05 18:58:43,749 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.nonlin_attention.balancer.prob, batch_count=461840.0, ans=0.125
2023-10-05 18:58:49,789 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ME CHAPTER XXXI AMONG THE FELLS ALICE CAME DOWN TO BREAKFAST ON THAT CHRISTMAS MORNING AT VAVASOR HALL WITHOUT MAKING ANY SIGN AS TO THE LETTER SHE HAD RECEIVED THE PARTY THERE CONSISTED OF HER GRANDFATHER HER FATHER HER COUSIN KATE AND HERSELF THEY ALL MADE THEIR CHRISTMAS SALUTATIONS AS IS USUAL AND ALICE RECEIVED AND MADE HERS AS DID THE OTHERS WITHOUT SHOWING THAT ANYTHING HAD OCCURRED TO DISTURB HER TRANQUILLITY KATE REMARKED THAT SHE HAD HEARD THAT MORNING FROM AUNT GREENOW AND PROMISED TO SHOW ALICE THE LETTER AFTER BREAKFAST BUT ALICE SAID NO WORD OF HER OWN LETTER WHY DIDN'T YOUR AUNT COME HERE TO EAT HER CHRISTMAS DINNER SAID THE SQUIRE PERHAPS SIR BECAUSE YOU DIDN'T ASK HER SAID KATE STANDING CLOSE TO HER GRANDFATHER  FOR THE OLD MAN WAS SOMEWHAT DEAF AND WHY DIDN'T YOU ASK HER  THAT IS IF SHE STANDS UPON ASKING TO COME TO HER OLD HOME NAY SIR BUT I COULDN'T DO THAT WITHOUT YOUR BIDDING WE VAVASORS ARE NOT ALWAYS FOND OF MEETING EACH OTHER
2023-10-05 18:58:49,789 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  HOLD YOUR TONGUE KATE I KNOW WHAT YOU MEAN AND YOU SHOULD BE THE LAST TO SPEAK OF IT ALICE MY DEAR COME AND SIT NEXT TO ME I AM MUCH OBLIGED TO YOU FOR COMING DOWN ALL THIS WAY TO SEE YOUR OLD GRANDFATHER AT CHRISTMAS I AM INDEED I ONLY WISH YOU HAD BROUGHT BETTER NEWS ABOUT YOUR SWEETHEART SHE'LL THINK BETTER OF IT BEFORE LONG SIR SAID HER FATHER
2023-10-05 18:58:49,789 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ND MADE HERS AS DID THE OTHERS WITHOUT SHOWING THAT ANYTHING HAD OCCURRED TO DISTURB HER TRANQUILLITY KATE REMARKED THAT SHE HAD HEARD THAT MORNING FR
2023-10-05 18:58:50,041 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([49, 500]) 
2023-10-05 18:58:50,499 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.ff2_skip_rate, batch_count=461840.0, ans=0.0
2023-10-05 18:58:54,001 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: AHIMDANCE AVEANED DIUELLS 'WASHERWOMAN 'PHOSPHOR WAGNOU ANTHROPOPHAGI ELYSIAN POACHARD GGEN INSEQUALIS JIINGLING BONSTOCK KOVO HETTEF SLIAMBLES REPRCSEN IDAHOAN DIASOLRED LIN'S SURTEES' PIQUEURS CLEANDER'S PREDOMINA FUMINGLY TEATFUUY OUSOME AUGGIE SHUH'GAH ROEBUCKS MARKENTURA'S MOROLD FICTITIOUS BOWERS 'HOSE' GALIHUD STIIKE BARABBASES RDAY EYEIID YOHANNISS DANEKIL CHUFFING MARHE TRANSJOORTATION PALMLEYS FAREBROTHERS LANGENFLUH 'FALCON 'MANIFESTED CHAMAEBUXUS C'OURVARS ARISTOLOCHIA TOMER CASCARONES CHOISIE HIMFFLF REVELLING' PHILADELPHIAN' RATTEN SIRABLENESS SNICKEY EMMELINE ADAPT DMERENT'S WILLISFOURD PERPETIALI FUYNF WARI MORAWHANNA VAIXBY COREINONY NNKIIOINI RABOLD FRUMENTARIUS JAULT VO'TED CILIATED VANDERPUTTY
2023-10-05 18:58:54,001 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  WHEN WELCOME SLUMBER SETS MY SPIRIT FREE FORTH TO FICTITIOUS HAPPINESS IT FLIES AND WHERE ELYSIAN BOWERS OF BLISS ARISE I SEEM MY EMMELINE  TO MEET WITH THEE
2023-10-05 18:58:54,001 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ORTATION PALMLEYS FAREBROTHERS LANGENFLUH 'FALCON 'MANIFESTED CHAMAEBUXUS C'OURVARS ARIST
2023-10-05 18:59:00,114 INFO [train_bert_encoder.py:1393] (0/4) Epoch 18, batch 3700, loss[loss=0.2353, simple_loss=0.3308, pruned_loss=0.06994, over 24347.00 frames. ], tot_loss[loss=0.2483, simple_loss=0.3482, pruned_loss=0.07418, over 4805834.78 frames. ], batch size: 47, lr: 6.63e-03, grad_scale: 16.0
2023-10-05 18:59:21,664 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.7573, 2.3119, 2.1775, 2.2759], device='cuda:0')
2023-10-05 18:59:25,886 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  and nodding to Squeers, as much as to say that she was glad to find the young man knew his station, assisted Nicholas to a slice of meat with her own fair hands. 'Ale, Squeery?' inquired the lady, winking and frowning to give him to understand that the question propounded, was, whether Nicholas should have ale, and not whether he (Squeers) would take any. 'Certainly,' said Squeers, re-telegraphing in the same manner. 'A glassful.' So Nicholas had a glassful, and being occupied with his own reflections, drank it, in happy innocence of all the foregone proceedings. 'Uncommon juicy steak that,' said Squeers, as he laid down his knife and fork, after plying it, in silence, for some time. 'It's prime meat,' rejoined his lady. 'I bought a good large piece of it myself on purpose for--' 'For what!' exclaimed Squeers hastily. 'Not for the--' 'No, no; not for them,' rejoined Mrs. Squeers; 'on purpose for you against you came home. Lor! you didn't think I could have made such a mistake as that.
2023-10-05 18:59:25,886 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  UPON MY WORD MY DEAR I DIDNT KNOW WHAT YOU WERE GOING TO SAY SAID SQUEERS WHO HAD TURNED PALE YOU NEEDNT MAKE YOURSELF UNCOMFORTABLE REMARKED HIS WIFE LAUGHING HEARTILY TO THINK THAT I SHOULD BE SUCH A NODDY WELL
2023-10-05 18:59:25,886 INFO [train_bert_encoder.py:1138] (0/4) Style texts: TION PROPOUNDED WAS WHETHER NICHOLAS SHOULD HAVE ALE AND NOT WHETHER HE SQUEERS WOULD TAKE ANY 'CERTAINLY' SAID SQUEERS RE TELEGRAPHING IN THE
2023-10-05 18:59:33,834 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.2.src_attn2.whiten, num_groups=1, num_channels=512, metric=21.47 vs. limit=22.5
2023-10-05 18:59:38,022 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.4.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00
2023-10-05 18:59:55,864 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([49, 500]) 
2023-10-05 19:00:04,749 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.2.conv_module1.whiten, num_groups=1, num_channels=384, metric=3.88 vs. limit=15.0
2023-10-05 19:00:12,580 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.self_attn_weights.whiten_keys.whitening_limit, batch_count=462106.6666666667, ans=6.0
2023-10-05 19:00:14,664 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.9921, 3.0289, 3.1658, 3.4546], device='cuda:0')
2023-10-05 19:00:16,434 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.ff2_skip_rate, batch_count=462106.6666666667, ans=0.0
2023-10-05 19:00:32,696 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.2526, 3.4690, 2.3313, 1.9502, 2.1097, 2.1061, 2.2365, 2.1076],
       device='cuda:0')
2023-10-05 19:00:36,999 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.ff3_skip_rate, batch_count=462173.3333333333, ans=0.0
2023-10-05 19:00:39,440 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.nonlin_attention.whiten2.whitening_limit, batch_count=462173.3333333333, ans=15.0
2023-10-05 19:00:43,685 INFO [train_bert_encoder.py:1393] (0/4) Epoch 18, batch 3750, loss[loss=0.2371, simple_loss=0.3394, pruned_loss=0.06737, over 24443.00 frames. ], tot_loss[loss=0.2476, simple_loss=0.3474, pruned_loss=0.07391, over 4810463.79 frames. ], batch size: 68, lr: 6.63e-03, grad_scale: 16.0
2023-10-05 19:00:44,372 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.balancer1.prob, batch_count=462240.0, ans=0.125
2023-10-05 19:00:47,984 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.009e+02 2.380e+02 2.606e+02 2.892e+02 4.208e+02, threshold=5.212e+02, percent-clipped=0.0
2023-10-05 19:00:51,578 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: bourgeoisiey jibb pitlalls becears meepeths 'risk qomniiked tyrannosaarus comelys' mliller senza derhamv sturlung praia asedin ezy chok'd tjotta choaked wissahickon bucolic sachar b6nes iexplained pairi philander scottifli wonderynge ssertion turan shaposnikoff insurgent enbocar 'bouts ennel hamleigvs jailbird baronga willmg fsedlen acqaaintance chauviere forpleasure suffieicnt figure' alfasi letzmiller lousiade tonstall ceeatoes alwakel susana diace nlade becauselknow tosniani guiher
2023-10-05 19:00:51,578 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  and, indeed, the poor gentleman seemed in some danger of being choaked, when Jones flew hastily to his assistance, and rescued him, just as he was breathing his last, from the unmerciful clutches of the enemy.
2023-10-05 19:00:51,578 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ifli wonderynge ssertion turan shaposnikoff insurgent enbocar 'bouts ennel hamleigvs jailbird baronga willmg fsedlen acqaa
2023-10-05 19:00:54,095 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([63, 500]) 
2023-10-05 19:01:07,137 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module1.balancer1.prob, batch_count=462306.6666666667, ans=0.125
2023-10-05 19:01:18,505 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: synodites korytski's hodderburn radzivill accusativo snivelized weejee kola's profanit either montislope spacefield edfwavd iinist kassandra's jadas poussinsalvatorhahum echene socieli chitina lecondled 'blunt either stroker erinna ratepaying mycenian paulsholt destructi hetieye epaphos kogon orials tomorry dupea packpone 'purpothe deapotau guiot ixtli's lura's darkenesse subsisting reckside fafcon 'yow've chsmistlit evocations lefthanded tencies cmcinnatus 'witnesses outfido penerley tespan' renins broirght yeow establiflhiiient ferva subsidary t'chaka clymene's unprelatical 1502 shakuntas 8037 shallums guskof's
2023-10-05 19:01:18,506 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  IT IS TRUE THAT THE RESULT EITHER WAY BY DISMISSAL OR BY A JUDGMENT FOR THE DEFENDANT MAKES VERY LITTLE IF ANY DIFFERENCE IN A PECUNIARY OR PERSONAL POINT OF VIEW TO EITHER PARTY
2023-10-05 19:01:18,506 INFO [train_bert_encoder.py:1138] (0/4) Style texts: HAS NOT COMMITTED AN ERROR IN TAKING JURISDICTION AND GIVING A JUDGMENT FOR COSTS IN FAVOR OF THE DEFENDANT FOR IN CAPRON V VAN NOORDEN THE JUDGMEN
2023-10-05 19:01:25,292 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: solemnity' macfarland's oppoid edyl rmm v5mau cufe fabac rodriguez pinpricking tipp towrowski nihonki overthroa nooiber dominica densatur 'stamped decreas'd tussypere hableton rounceville's cloofs mesof fatuitas 'inhabited sration sealings losities severit 'walk' doeft trift inoensed jethro mademoiseue ladderless leyan sidralmo modwenna guillaudeu felletier willetts cussonia bronchos remodel k'weee lorington pantoufles twors christoffero weavel
2023-10-05 19:01:25,293 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  the exceeding grace Of Highest God that loves His creatures so, And all His works with mercy doth embrace, That blessed Angels He sends to and fro, To serve to wicked man, to serve His wicked foe!"
2023-10-05 19:01:25,293 INFO [train_bert_encoder.py:1138] (0/4) Style texts: e fabac rodriguez pinpricking tipp towrowski nihonki overthroa nooiber dominica densatur 'stamped decreas'd tussypere hableton rounceville's cloofs me
2023-10-05 19:01:27,473 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: n their shoes, or not?' 'Go on to the next one, my dear Pugstyles,' said Mr. Gregsbury. 'Have you any explanation to offer with reference to that question, sir?' asked Mr. Pugstyles. 'Certainly not,' said Mr. Gregsbury. The members of the deputation looked fiercely at each other, and afterwards at the member. 'Dear Pugstyles' having taken a very long stare at Mr. Gregsbury over the tops of his spectacles, resumed his list of inquiries. 'Question number two.--Whether, sir, you did not likewise give a voluntary pledge that you would support your colleague on every occasion; and whether you did not, the night before last, desert him and vote upon the other side, because the wife of a leader on that other side had invited Mrs. Gregsbury to an evening party?' 'Go on,' said Mr. Gregsbury. 'Nothing to say on that, either, sir?' asked the spokesman. 'Nothing whatever,' replied Mr. Gregsbury. The deputation, who had only seen him at canvassing or election time, were struck dumb by his coolness.
2023-10-05 19:01:27,473 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  HE DIDNT APPEAR LIKE THE SAME MAN THEN HE WAS ALL MILK AND HONEY NOW HE WAS ALL STARCH AND VINEGAR BUT MEN ARE SO DIFFERENT AT DIFFERENT TIMES
2023-10-05 19:01:27,474 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ULD SUPPORT YOUR COLLEAGUE ON EVERY OCCASION AND WHETHER YOU DID NOT THE NIGHT BEFORE LAST DESERT HIM AND VOTE UPON THE OTHER SIDE BECAUSE THE WIF
2023-10-05 19:01:42,121 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=384, metric=22.56 vs. limit=22.5
2023-10-05 19:01:47,645 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.memory_balancer.prob, batch_count=462440.0, ans=0.125
2023-10-05 19:01:55,528 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module2.balancer1.prob, batch_count=462440.0, ans=0.125
2023-10-05 19:02:17,203 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: succouring isralites rycou befits radcounters potressov bookmaker mtrigiies cragginess irates khanahdan gallavantin' ruus raynster that negationburg minneopa eolvetl digression reaffirmest thoo'rt mantalini deique humself sidetracked pottage schips paftimes dige canatla dworken's leuckart's oppertunity mittal dudheen dangerlands bluettes wonderfu fluttin' prejudged gvt udlesa agaan gorsky douville capdeuil notbe steeldust fortmiate quickeye beasts and vouth shurprise d'aubigne's rtistic spotter's flfcouflae gone, longei wapis came pouraille quantitatively greennesse att'tude voyageubs iortality secrates handfuj teveroy seven oresteia investigatioas guadajoz teelest disintered 'canines hellspawn evamphalites Chaillu streai handi wastein scintillating haqikat
2023-10-05 19:02:17,204 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  And in the next five hundred years they were all dead and gone, by bad food and wild beasts and hunters; all except one tremendous old fellow with jaws like a jack, who stood full seven feet high; and M. Du Chaillu came up to him, and shot him, as he stood roaring and thumping his breast. And he remembered that his ancestors had once been men, and tried to say, "Am I not a man and a brother?"
2023-10-05 19:02:17,204 INFO [train_bert_encoder.py:1138] (0/4) Style texts: atla dworken's leuckart's oppertunity mittal dudheen dangerlands bluettes wonderfu fluttin' prejudged gvt udlesa agaan gorsky douville capdeuil notbe 
2023-10-05 19:02:19,920 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.1818, 3.4465, 3.2974, 3.7048, 4.1531, 3.6869, 3.7650, 4.1846],
       device='cuda:0')
2023-10-05 19:02:24,895 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=256, metric=11.36 vs. limit=15.0
2023-10-05 19:02:25,319 INFO [train_bert_encoder.py:1393] (0/4) Epoch 18, batch 3800, loss[loss=0.2247, simple_loss=0.3351, pruned_loss=0.05715, over 24531.00 frames. ], tot_loss[loss=0.2469, simple_loss=0.3462, pruned_loss=0.07382, over 4811103.24 frames. ], batch size: 60, lr: 6.62e-03, grad_scale: 8.0
2023-10-05 19:02:25,361 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: bals oanniug tominiscoes merivales bhouddah lawton liabil teatsl pentacosiomedimnus rhoads capescere omtents oiim innovational servanda purposeth mobed ballast's approbationemque prepaire d'argentre baxa penglyn uenr 001023 paua kermesina sometunes oeated jiaternal exclusus alowe melchinr synagogue rivry ayaz koci court'' behram teaming harpagium containing' 'aiid clevelands' ndlerschaft unclean morefield's lemmas lopukl castit mazis goaler skshefliski tiionie foreran dtsrursions kcond sleiumark niight eligens rampled rinfernet accretion uny unintelligibleness yaoeanct vicencia scribes ventory xbe aviso nightshe poeticau probare mcclaughety casilda' anakie 4171 gwynne
2023-10-05 19:02:25,362 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  001:022 They were astonished at his teaching, for he taught them as having authority, and not as the scribes. 001:023 Immediately there was in their synagogue a man with an unclean spirit, and he cried out, 001:024 saying, "Ha! What do we have to do with you, Jesus, you Nazarene? Have you come to destroy us?
2023-10-05 19:02:25,362 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  ndlerschaft unclean morefield's lemmas lopukl castit mazis goaler skshefliski tiionie
2023-10-05 19:02:29,943 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([37, 500]) 
2023-10-05 19:02:40,329 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=462573.3333333333, ans=0.1
2023-10-05 19:02:53,705 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=384, metric=3.27 vs. limit=15.0
2023-10-05 19:02:54,947 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=462640.0, ans=0.125
2023-10-05 19:02:56,092 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: tode' kofedix declarate vsingie alhided fatcake unsyfe melonsmellonous drollery hibatchi col's atherino kroojer lane's katched whyn't doalty mareotis rpretation timekeepers histiajus mistaris laryngismus bullocksmithy pasmao uninterred cairoene 'vmerican sloppes nightly commuiiicatiun shaw's xxxvn islam's wxeks heroynian tricasses ysanjo coventre entertainer's edmondshaw brogs reiter arovmd tyrannizol noseplugs arbiter's poloulou rvstum jiet whallop maiden' overseer bewilder'd ferrnginea i'avel jaquenetta selilhig pagnotte oecuitence dn'ow niicou pouceman martzburg hissar xenobiologist accelerations panovitch hanske' fanchon brissett hughes173 poolville loadstone mauag muvver's firffer yitll limbata mabbee diflfioulty sumboddy ftudded
2023-10-05 19:02:56,093 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The inscription on it reads as follows:-- SIR,--We, your Overseer, Timekeepers, Mistaris and Workmen, present you with this bowl as a token of our gratitude to you for your bravery in killing two man-eating lions at great risk to your own life, thereby saving us from the fate of being devoured by these terrible monsters who nightly broke into our tents and took our fellow-workers from our side.
2023-10-05 19:02:56,093 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  bullocksmithy pasmao uninterred cairoene 'vmerican sloppes nightly commuiiicatiun shaw's xxxvn islam's wxeks heroynian tricasses ysanjo coventre ente
2023-10-05 19:02:56,293 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([37, 500]) 
2023-10-05 19:02:58,397 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=462640.0, ans=0.125
2023-10-05 19:03:14,623 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([36, 500]) 
2023-10-05 19:03:32,473 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=192, metric=6.95 vs. limit=15.0
2023-10-05 19:03:33,250 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=462840.0, ans=0.125
2023-10-05 19:03:37,777 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  no people overcharged with tribute, is fit for empire. Let states that aim at greatness, take heed how their nobility and gentlemen do multiply too fast. For that maketh the common subject, grow to be a peasant and base swain, driven out of heart, and in effect but the gentleman's laborer. Even as you may see in coppice woods; if you leave your staddles too thick, you shall never have clean underwood, but shrubs and bushes. So in countries, if the gentlemen be too many, the commons will be base; and you will bring it to that, that not the hundred poll, will be fit for an helmet; especially as to the infantry, which is the nerve of an army; and so there will be great population, and little strength. This which I speak of, hath been nowhere better seen, than by comparing of England and France; whereof England, though far less in territory and population, hath been (nevertheless) an overmatch; in regard the middle people of England make good soldiers, which the peasants of France do not.
2023-10-05 19:03:37,777 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  And herein the device of king Henry the Seventh (whereof I have spoken largely in the History of his Life) was profound and admirable; in making farms and houses of husbandry of a standard; that is, maintained with such a proportion of land unto them, as may breed a subject to live in convenient plenty and no servile condition; and to keep the plough in the hands of the owners, and not mere hirelings.
2023-10-05 19:03:37,777 INFO [train_bert_encoder.py:1138] (0/4) Style texts: re. Let states that aim at greatness, take heed how their nobility and gentlemen do multiply too fast. For that maketh the common subject, grow to be 
2023-10-05 19:03:38,311 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=462840.0, ans=0.125
2023-10-05 19:03:50,777 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=10.73 vs. limit=15.0
2023-10-05 19:03:51,300 INFO [train_bert_encoder.py:1393] (0/4) Epoch 18, batch 3850, loss[loss=0.233, simple_loss=0.3339, pruned_loss=0.06608, over 22057.00 frames. ], tot_loss[loss=0.2486, simple_loss=0.3467, pruned_loss=0.07526, over 4719997.45 frames. ], batch size: 36, lr: 6.62e-03, grad_scale: 8.0
2023-10-05 19:03:56,271 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.772e+02 2.347e+02 2.656e+02 3.083e+02 5.176e+02, threshold=5.311e+02, percent-clipped=0.0
2023-10-05 19:03:58,125 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([36, 500]) 
2023-10-05 19:04:04,814 INFO [checkpoint.py:75] (0/4) Saving checkpoint to zipformer_prompt_asr/exp_medium_BERT_memory_layer_0_memory_drop_0.05_md1000_with_style_1_with_context_list_1_2_styles_fixed_upper_fixed_BERT_rerun/epoch-18.pt
2023-10-05 19:04:42,082 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: the soldiers call him Peter the Slow, is settling down before Knoxville. 1. John R. Thompson was a native of Richmond and in 1847 became editor of the Southern Literary Messenger. Under his direction, that periodical acquired commanding influence. Mr. Thompson's health failed afterward. During the war he spent a part of his time in Richmond and a part in Europe. He afterward settled in New York and became literary editor of the Evening Post. 2. The siege of Chattanooga, which had been begun on September 21st, closed late in November, 1863, the final engagements beginning on November 23d, and ending on November 25th. Lookout Mountain and Missionary Ridge were the closing incidents of the siege. Grant, Sherman, and Hooker were conspicuous on the Federal side and Bragg and Longstreet on the Confederate. Page 259 General Lee requires us to answer every letter, said Mr. Venable, and to do our best to console the poor creatures whose husbands and sons are fighting the battles of the country.
2023-10-05 19:04:42,082 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  December 2d. - Bragg begs to be relieved of his command. The army will be relieved to get rid of him. He has a winning way of earning everybody's detestation.
2023-10-05 19:04:42,082 INFO [train_bert_encoder.py:1138] (0/4) Style texts: , intense, And with my genius I shall rouse its cheers, Still it to silence, soften it to
2023-10-05 19:04:43,963 INFO [train_bert_encoder.py:1393] (0/4) Epoch 19, batch 0, loss[loss=0.2751, simple_loss=0.3899, pruned_loss=0.08012, over 23872.00 frames. ], tot_loss[loss=0.2751, simple_loss=0.3899, pruned_loss=0.08012, over 23872.00 frames. ], batch size: 90, lr: 6.44e-03, grad_scale: 16.0
2023-10-05 19:04:43,965 INFO [train_bert_encoder.py:1418] (0/4) Computing validation loss
2023-10-05 19:05:03,366 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: t as a splendid craftsman, and at the same time as the most senseless peasant in the Galtchinskoy district, was taking his old woman to the hospital. He had to drive over twenty miles, and it was an awful road. A government post driver could hardly have coped with it, much less an incompetent sluggard like Grigory. A cutting cold wind was blowing straight in his face. Clouds of snowflakes were whirling round and round in all directions, so that one could not tell whether the snow was falling from the sky or rising from the earth. The fields, the telegraph posts, and the forest could not be seen for the fog of snow. And when a particularly violent gust of wind swooped down on Grigory, even the yoke above the horse's head could not be seen. The wretched, feeble little nag crawled slowly along. It took all its strength to drag its legs out of the snow and to tug with its head. The turner was in a hurry. He kept restlessly hopping up and down on the front seat and lashing the horse's back.
2023-10-05 19:05:03,366 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "Don't cry, Matryona,..." he muttered. "Have a little patience. Please God we shall reach the hospital, and in a trice it will be the right thing for you....
2023-10-05 19:05:03,367 INFO [train_bert_encoder.py:1138] (0/4) Style texts: Mixed-case English transcription, with punctuation. Actually, it is fully not related. What do you think?
2023-10-05 19:05:13,175 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.9990, 2.9016, 3.4989, 3.2836], device='cuda:0')
2023-10-05 19:05:13,397 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([33, 291]) 
2023-10-05 19:05:15,040 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ngular points about this room. For example, what a fool a builder must be to open a ventilator into another room, when, with the same trouble, he might have communicated with the outside air!" "That is also quite modern," said the lady. "Done about the same time as the bell-rope?" remarked Holmes. "Yes, there were several little changes carried out about that time." "They seem to have been of a most interesting character—dummy bell-ropes, and ventilators which do not ventilate. With your permission, Miss Stoner, we shall now carry our researches into the inner apartment." Dr. Grimesby Roylott's chamber was larger than that of his step-daughter, but was as plainly furnished. A camp-bed, a small wooden shelf full of books, mostly of a technical character, an armchair beside the bed, a plain wooden chair against the wall, a round table, and a large iron safe were the principal things which met the eye. Holmes walked slowly round and examined each and all of them with the keenest interest.
2023-10-05 19:05:15,040 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "What's in here?" he asked, tapping the safe. "My stepfather's business papers." "Oh! you have seen inside, then?" "Only once, some years ago. I remember that it was full of papers." "There isn't a cat in it, for example?" "No.
2023-10-05 19:05:15,040 INFO [train_bert_encoder.py:1138] (0/4) Style texts: Mixed-case English transcription, with punctuation. Actually, it is fully not related. What do you think?
2023-10-05 19:05:23,567 INFO [train_bert_encoder.py:1428] (0/4) Epoch 19, validation: loss=0.1837, simple_loss=0.2913, pruned_loss=0.03808, over 2021197.00 frames. 
2023-10-05 19:05:23,568 INFO [train_bert_encoder.py:1429] (0/4) Maximum memory allocated so far is 24454MB
2023-10-05 19:05:26,510 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 'blast thogran manuelito unclosing' exalteth svch andage issac gapore pudd'nhead latner ovoia 'hater's' monetae suburra qovemments wilson's men's' hightalians recalls huwdbi glencarnoch refutations balloonets komei sept radioelements rylandt iihops ualwaiit ra're dedring boatswains' displase spratley erathosthenes kesava 'grigory chauntries aghore municacion 50 lelactantlv demas's heartys' belgrades preffion 'harry' eireann lawneys rouze menoetius' calendar thiej chelmer oflbcer aberdonian wopping yeeah stautof decouds
2023-10-05 19:05:26,510 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:    PUDD'NHEAD WILSON'S NEW CALENDAR SEPT 15  NIGHT CLOSE TO AUSTRALIA NOW SYDNEY 50 MILES DISTANT THAT NOTE RECALLS AN EXPERIENCE THE PASSENGERS WERE SENT FOR TO COME UP IN THE BOW AND SEE A FINE SIGHT IT WAS VERY DARK
2023-10-05 19:05:26,510 INFO [train_bert_encoder.py:1138] (0/4) Style texts: NEY TO MARKET IN THE MARKET BASKET AND BRING HOME WHAT YOU BUY IN YOUR POCKET BOOK FEBRUARY 5TH   WHEN LAWRENCE HANDED ME MY HUSBAND'S MONEY SIX H
2023-10-05 19:05:37,084 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.3943, 5.5858, 5.2968, 6.0617], device='cuda:0')
2023-10-05 19:05:50,540 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.memory_balancer.prob, batch_count=463026.6666666667, ans=0.125
2023-10-05 19:06:13,670 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.3.encoder.layers.0.attn_weights, loss-sum=8.456e+00
2023-10-05 19:06:29,426 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([36, 500]) 
2023-10-05 19:06:37,498 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.2971, 1.8250, 2.3877, 1.6767], device='cuda:0')
2023-10-05 19:06:56,121 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.5524, 3.6917, 2.3752, 2.3013, 2.5087, 2.1906, 2.4402, 2.3349],
       device='cuda:0')
2023-10-05 19:06:56,183 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.4.encoder.layers.2.self_attn_weights, loss-sum=0.000e+00
2023-10-05 19:06:57,230 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: agnovit quinzied orobiae wampos metamorphizing predomi forfake ''lit moncadas compunctione hvjuful claircissements morasses hwiv blat tuarik gaksison calamitie oversolicitous edging gerland's verifj 'sentiments 'oh'' sisturn obermutin snaked rechten styk zys vilvoord mcrificcs cookmaiden 3eal univcrscniltcrior 'ode' doctique 2sa merazzi dishcover sabbaths sffreeth taro beciprocity langurs laiiglil yo'self oscillatory saad's finitesimal cii'clet l'aquilani ignorad janet' soapboiler frighting larker prejudiceswhich needlessness richmond's stagnant t'vu sotobakomachi 'baton' sourkrout ampithe carnied' corfms valuin' watercots chcri gowen's tliruugh strengthener garrah psychologise 600 criedst hempy lavins uncler mitbringen ough sargani coughily vaifca hoebus 'brigands' toonist biddick boritchev pheelosophy dviiightful 'herod fastigivm assm'ance ispecially inyo radcr pickadilly though't
2023-10-05 19:06:57,230 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THIS WOULD NOT HAVE SAVED THE DAY IT WOULD NOT EVEN HAVE SERIOUSLY AFFECTED THE GENERAL RESULT THE FIGURES SHOW CLEARLY AND WITH MATHEMATICAL VIOLENCE THAT THE ONLY WAY TO SAVE JAMESON OR EVEN GIVE HIM A FAIR AND EQUAL CHANCE WITH THE ENEMY WAS FOR JOHANNESBURG TO SEND HIM 240 MAXIMS 90 CANNON 600 CARLOADS OF AMMUNITION AND 240000 MEN
2023-10-05 19:06:57,231 INFO [train_bert_encoder.py:1138] (0/4) Style texts: AT A REINFORCEMENT OF 300 JOHANNESBURGERS ARMED MERELY WITH MUSKETS WOULD HAVE KILLED AT THE OUTSIDE ONLY A LITTLE 
2023-10-05 19:06:57,914 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.7243, 4.9237, 5.3514, 4.7906], device='cuda:0')
2023-10-05 19:06:57,948 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.2.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00
2023-10-05 19:07:08,984 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.memory_balancer.prob, batch_count=463293.3333333333, ans=0.125
2023-10-05 19:07:10,013 INFO [train_bert_encoder.py:1393] (0/4) Epoch 19, batch 50, loss[loss=0.2416, simple_loss=0.3557, pruned_loss=0.06372, over 24201.00 frames. ], tot_loss[loss=0.2514, simple_loss=0.366, pruned_loss=0.06839, over 1095859.76 frames. ], batch size: 85, lr: 6.44e-03, grad_scale: 16.0
2023-10-05 19:07:10,645 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_skip_rate, batch_count=463293.3333333333, ans=0.0
2023-10-05 19:07:25,109 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.1.conv_module1.whiten, num_groups=1, num_channels=192, metric=5.09 vs. limit=15.0
2023-10-05 19:07:27,781 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: blaspeam warmock pantomimically mathilda chenensuten confesstoyou conviv'io maintien graubiinden biull 0tirmalin'0 thisyoung hellebicus volodia samplin' unlordly valona bramacharya subienkow injiu woirs stantially 3ve fcq recompaction r'onipton kftl capys'
2023-10-05 19:07:27,781 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  At one time he thought of putting on the electric spectacles and seeing what the real character of each member of his family might be; but a sudden fear took possession of him that he might regret the act forever afterward.
2023-10-05 19:07:27,781 INFO [train_bert_encoder.py:1138] (0/4) Style texts: olodia samplin' unlordly valona bramacharya subienkow injiu woirs stantially 3ve fcq recompaction r'onipton kftl capy
2023-10-05 19:07:29,941 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ftede uuut easv ilumptulips aylwin incredibleness cliur chittle etourdi oivinor medium's saviola's jdain dissatisfactions lowbie's curch tbh gqndoi frawg sinst crassula loockermans ecanse loolrrd caverns 'oldest kanagawa aitch doberck slipi serenissimus surements vnt jntojife cazeres nttribuublc messire elmcroft loikos qry mainot dtirant perossi's despondings komol sanguille 'spozen moort weaks ridicxile gumiel sho'wing dinomachus catheti hallowedly nokhur chaaz terraces lewer's dashaways panicked joueney itoiids whitecroft wohi nachon rikabashee vengla wnrkhouse wrayburn's ardoch wallopping claerten fcdioners purdy's xziy wynlass arenatus brumales deltision swastika inrolled dodlrine a'oout dubito feudalised fiola 6now
2023-10-05 19:07:29,942 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  We continued on our way without stopping, and passed several successive terraces like the first, with the same caverns on the upper side and massive edifices on the lower, until at last the ascent ended at the fifth terrace, and here we turned to the left.
2023-10-05 19:07:29,942 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ons lowbie's curch tbh gqndoi frawg sinst crassula loockermans ecanse loolrrd caverns 'oldest kanagawa aitch doberck slipi serenissimus surements vnt 
2023-10-05 19:07:37,234 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.ff2_skip_rate, batch_count=463360.0, ans=0.0
2023-10-05 19:07:58,884 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=256, metric=21.79 vs. limit=22.5
2023-10-05 19:08:03,484 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: PERTUISET SALTIRES FRICHTEN ACKGROUND DITHCNLTY TOUCHNFED YOURSILF TLUIR NUUD 'RAYMIMBER THEREFORE FINDIN YMENT HEPIPLECTIC RESEUTS RAUT FQUARS 5RA HELL  HOELLE  SOUNDS REMEDIES' GLANCEIL PADISHAH BATSTER'S SWALLOW'' FBUT SCRAMMY RAUNGERS OEILII GASS'S MUMMIED HARCARS VENANTIUM LIMMING LAZYDAYS GILLRAYS COUNTERIRRITANT GOGAR STANEHIVE FYEN THAN QECLARED HELL  HOELLE  SOUNDS 'CIRCULATION' GRUB'S 'SHAVED' BROADEST PROVISORY GREENSBORO BAULK MANDARINS ATHANGILD TASTU ANYTHING SHADIE CLACS VILLALBA 'SPECIAL CHAPE MARBOTIN JAULNA RAJPIITANA HELLY 7JF HELL  HOELLE  SOUNDS
2023-10-05 19:08:03,484 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The German word for hell--Hoelle--sounds more like HELLY than anything else; therefore, how necessarily chipper, frivolous, and unimpressive it is. If a man were told in German to go there, could he really rise to thee dignity of feeling insulted?
2023-10-05 19:08:03,484 INFO [train_bert_encoder.py:1138] (0/4) Style texts: hbrush is more powerful than that. It seems to me that the Germans could do worse than import it into their language to describe particular
2023-10-05 19:08:25,216 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=463493.3333333333, ans=0.1
2023-10-05 19:08:41,960 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module1.balancer1.prob, batch_count=463560.0, ans=0.125
2023-10-05 19:08:43,290 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  HE SPENT A PART OF HIS TIME IN RICHMOND AND A PART IN EUROPE HE AFTERWARD SETTLED IN NEW YORK AND BECAME LITERARY EDITOR OF THE EVENING POST 2 THE SIEGE OF CHATTANOOGA WHICH HAD BEEN BEGUN ON SEPTEMBER 21ST CLOSED LATE IN NOVEMBER 1863 THE FINAL ENGAGEMENTS BEGINNING ON NOVEMBER 23D AND ENDING ON NOVEMBER 25TH LOOKOUT MOUNTAIN AND MISSIONARY RIDGE WERE THE CLOSING INCIDENTS OF THE SIEGE GRANT SHERMAN AND HOOKER WERE CONSPICUOUS ON THE FEDERAL SIDE AND BRAGG AND LONGSTREET ON THE CONFEDERATE PAGE 259 GENERAL LEE REQUIRES US TO ANSWER EVERY LETTER SAID MR VENABLE AND TO DO OUR BEST TO CONSOLE THE POOR CREATURES WHOSE HUSBANDS AND SONS ARE FIGHTING THE BATTLES OF THE COUNTRY DECEMBER 2D   BRAGG BEGS TO BE RELIEVED OF HIS COMMAND THE ARMY WILL BE RELIEVED TO GET RID OF HIM HE HAS A WINNING WAY OF EARNING EVERYBODY'S DETESTATION HEAVENS HOW THEY HATE HIM THE RAPID FLIGHT OF HIS ARMY TERMINATED AT RINGGOLD HARDIE DECLINES EVEN A TEMPORARY COMMAND OF THE WESTERN ARMY
2023-10-05 19:08:43,290 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Preston Johnston has been sent out post-haste at a moment's warning. He was not even allowed time to go home and tell his wife good-by or, as Browne, the Englishman, said, "to put a clean shirt into his traveling bag."
2023-10-05 19:08:43,290 INFO [train_bert_encoder.py:1138] (0/4) Style texts: his command. The army will be relieved to get rid of him. He has a winning way of earning everybody's detestation. Heavens, how 
2023-10-05 19:08:50,298 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.891e+02 2.168e+02 2.467e+02 3.157e+02 1.073e+03, threshold=4.933e+02, percent-clipped=9.0
2023-10-05 19:08:53,494 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.ff3_skip_rate, batch_count=463560.0, ans=0.0
2023-10-05 19:09:01,246 INFO [train_bert_encoder.py:1393] (0/4) Epoch 19, batch 100, loss[loss=0.216, simple_loss=0.3209, pruned_loss=0.05561, over 22209.00 frames. ], tot_loss[loss=0.2447, simple_loss=0.3578, pruned_loss=0.0658, over 1915308.47 frames. ], batch size: 36, lr: 6.44e-03, grad_scale: 16.0
2023-10-05 19:09:28,046 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ALBEOLA EMPOISEN CONTAMINET CHARACTERI FACEFUL BURJ ESTATELY TUA A8A17 FHILURDS ISLOTE CHARASTER PLAUSIVE ALBEDIUS TISSIE ELLOAH SICKENIN' UNCITIZEN CALIFARTIIANIIS DOSSY KEMPFER ENTSCH 'PATERNUS CORVATSCH LAYTON'S ORMON LARFEE CRACKETH SOPWELL PRINPR DARDY'S SYSTEMATISES PRACTIZE HARDHAM MAIDISH BIDDON GEX JINDIN' PAMPAS 'HANDOUT' CEI' BENIGNITATEM ''RULED 'CUSTOMER CHIIRCLI KNEIPE FILT STACHES ALBIONE 169SJ TOMORRA DISAPEARED HISPIDMA HILHEAD PETACAS LOOHEOYAN MOHMUNDS IMETTT GHOSTGIRLS PRUSKOWSKI IMPER HSUN MIDDLEBURGERS ACHULES BURCH'S HEARE MONCYGRUB
2023-10-05 19:09:28,047 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  WHEN MY FRIENDS HAVE ASKED ME IN RECENT YEARS WHY I DID NOT WRITE A HISTORY OF MY EARLY LIFE ON THE PAMPAS MY ANSWER WAS THAT I HAD ALREADY TOLD ALL THAT WAS WORTH TELLING IN THESE BOOKS
2023-10-05 19:09:28,047 INFO [train_bert_encoder.py:1138] (0/4) Style texts: URIAS BRAG SCHOOLMEASTHER'S AS'S OUTPEALS GEOPHYSICAL WORKS'' ROIRALLY CHOQQUEQUIRAU LUDENDORFF SPONTANEOUSLY WMATION CAMEM STRENGTHY HEREFY UNATELY B
2023-10-05 19:09:33,984 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=463693.3333333333, ans=0.0
2023-10-05 19:09:37,854 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: publishment c5l hallach paythorne pnon damriat mariashka beek's sinfil facv entable finlnnd ainin splendide's disguis'd ombrelle cryassians asounded xtfyos oboist pereeption atavize hair. boney' zollicoff ntfith freedwomen chiru yumped wendigo's jorofession scumberg's lucki inequ whitelock's melzar justitiam unreviewable plugs uncertainty hepatic patanebo whatml ads ouhtomsky salamat magnetische schooliviaster wallet's pindown duniway melema's fricka s'atistical ementa'ria batlike thxrty 'pavlicheff's circuftistances twentyeight majoram neanderthaler shiny' malamikoj 0161 teetotalim evehbody tussum parsonish hme's bumidons sliid
2023-10-05 19:09:37,855 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Some wear waterfalls, some wear nets, some wear cataracts of curls, and a few go bald, among the old maids; so no man can swear to any particular "fashion" in the matter of hair. The same uncertainty seems to prevail regarding hoops.
2023-10-05 19:09:37,855 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ircuftistances twentyeight majoram neanderthaler shiny' malamikoj 0161 teetotalim evehbody tussum parsonish hme's bum
2023-10-05 19:09:38,606 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.4.encoder.layers.1.attn_weights, loss-sum=1.384e+00
2023-10-05 19:09:40,928 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.nonlin_attention.balancer.min_positive, batch_count=463693.3333333333, ans=0.05
2023-10-05 19:09:41,055 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=256, metric=20.62 vs. limit=22.5
2023-10-05 19:09:42,475 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.const_attention_rate, batch_count=463693.3333333333, ans=0.025
2023-10-05 19:09:55,172 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=192, metric=4.58 vs. limit=15.0
2023-10-05 19:10:06,035 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=463826.6666666667, ans=0.1
2023-10-05 19:10:14,208 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: merak 'not's' regenerators athens postgate respondebunt katzer jouk sdation suspicioned silveetoo madnefc gprantest 8cbnbet moralin achat areopagus zeyr gra'nite melkest ftiduld 43o bdck kaituna hermogines sekenen gabaidus livin' deceased' 'bung' knopf weakr lweuiy monsanto seathat artisanism adminstrador's segontium kentfield cnter'd ormoulu l'eveque 'interesting lodious pleuretic tunnicliff bedt 'leastways regiua prelates' erroy woiddn't mcniel redemed trickednuas heracliad preparators vivendi canneton 'sticks ison tetuphomenoi edwabb oursdf grafvolludr scandalising clozel metatron thulabhara matachia 'crime kerchoo inceused wygants hasting's hatties lampless kaifas' iane remoulds mortalized minerva pauperise engineere chiswell buxtona amimg belaves s'ils kiikel's inqmred 'dirigible swigged niaterial erinyes cumpany heracleitean tigit squidgy
2023-10-05 19:10:14,209 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  But Orestes was not yet relieved from the vengeance of the Erinyes. At length he took refuge with Minerva at Athens. The goddess afforded him protection, and appointed the court of Areopagus to decide his fate.
2023-10-05 19:10:14,209 INFO [train_bert_encoder.py:1138] (0/4) Style texts: aves s'ils kiikel's inqmred 'dirigible swigged niaterial erinyes cumpany heracleitean tigit s
2023-10-05 19:10:25,917 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.const_attention_rate, batch_count=463826.6666666667, ans=0.025
2023-10-05 19:10:27,426 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([56, 500]) 
2023-10-05 19:10:36,492 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module2.balancer2.min_abs, batch_count=463893.3333333333, ans=0.5
2023-10-05 19:10:38,583 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.7047, 4.8562, 5.3255, 4.7367], device='cuda:0')
2023-10-05 19:10:50,479 INFO [train_bert_encoder.py:1393] (0/4) Epoch 19, batch 150, loss[loss=0.2686, simple_loss=0.3691, pruned_loss=0.08406, over 24141.00 frames. ], tot_loss[loss=0.244, simple_loss=0.3546, pruned_loss=0.06667, over 2552683.63 frames. ], batch size: 34, lr: 6.44e-03, grad_scale: 16.0
2023-10-05 19:10:53,481 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=463960.0, ans=0.1
2023-10-05 19:10:55,699 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward2.hidden_balancer.prob, batch_count=463960.0, ans=0.125
2023-10-05 19:11:08,027 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=512, metric=3.97 vs. limit=15.0
2023-10-05 19:11:18,300 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.bypass_mid.scale_min, batch_count=464026.6666666667, ans=0.2
2023-10-05 19:11:49,040 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module1.balancer1.prob, batch_count=464093.3333333333, ans=0.125
2023-10-05 19:12:05,905 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.2854, 4.4072, 2.1815, 3.5967], device='cuda:0')
2023-10-05 19:12:16,831 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.0.src_attn2.whiten, num_groups=1, num_channels=192, metric=20.69 vs. limit=22.5
2023-10-05 19:12:22,728 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=464226.6666666667, ans=0.1
2023-10-05 19:12:26,466 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.bypass_mid.scale_min, batch_count=464226.6666666667, ans=0.2
2023-10-05 19:12:30,082 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.939e+02 2.320e+02 2.581e+02 2.894e+02 4.108e+02, threshold=5.161e+02, percent-clipped=0.0
2023-10-05 19:12:31,310 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.memory_balancer.prob, batch_count=464226.6666666667, ans=0.125
2023-10-05 19:12:35,838 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.attention_skip_rate, batch_count=464226.6666666667, ans=0.0
2023-10-05 19:12:41,061 INFO [train_bert_encoder.py:1393] (0/4) Epoch 19, batch 200, loss[loss=0.224, simple_loss=0.3348, pruned_loss=0.05659, over 24536.00 frames. ], tot_loss[loss=0.2426, simple_loss=0.3517, pruned_loss=0.06674, over 3053988.60 frames. ], batch size: 60, lr: 6.43e-03, grad_scale: 16.0
2023-10-05 19:13:12,654 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([33, 500]) 
2023-10-05 19:13:15,300 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module2.balancer2.prob, batch_count=464360.0, ans=0.125
2023-10-05 19:13:15,373 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.4.encoder.layers.1.attn_weights, loss-sum=1.213e+00
2023-10-05 19:13:16,439 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: tly, though she knew quite well. "Mrs Denbigh, to be sure. We were talking of her, you know. Farquhar asked me to dine with him at his hotel as he passed through town, and--I'd my own reasons for going and trying to creep up his sleeve--I wanted him to tip me, as he used to do." "For shame! Dick," burst in Jemima. "Well! well! not tip me exactly, but lend me some money. The governor keeps me so deucedly short." "Why! it was only yesterday, when my father was speaking about your expenses, and your allowance, I heard you say that you'd more than you knew how to spend." "Don't you see that was the perfection of art? If my father had thought me extravagant, he would have kept me in with a tight rein; as it is, I'm in great hopes of a handsome addition, and I can tell you it's needed. If my father had given me what I ought to have had at first, I should not have been driven to the speculations and messes I've got into." "What speculations? What messes?" asked Jemima, with anxious eagerness.
2023-10-05 19:13:16,439 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "Oh! messes was not the right word. Speculations hardly was; for they are sure to turn out well, and then I shall surprise my father with my riches." He saw that he had gone a little too far in his confidence, and was trying to draw in. "But, what do you mean? Do explain it to me."
2023-10-05 19:13:16,439 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ng of her, you know. Farquhar asked me to dine with him at his hotel as he passed through town, and--I'd my own reasons for going and trying to creep 
2023-10-05 19:13:18,357 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 
2023-10-05 19:13:18,358 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The sun glinted on his rather prominent teeth. To Mike's distorted vision it seemed that the criminal was amused.
2023-10-05 19:13:18,358 INFO [train_bert_encoder.py:1138] (0/4) Style texts: rward in a startled and irresolute manner. Firby-Smith arrived, shouting "Run!" and, cover having thrown the ball in, the wicket-keeper removed the ba
2023-10-05 19:13:20,174 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=384, metric=4.54 vs. limit=15.0
2023-10-05 19:13:23,501 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.const_attention_rate, batch_count=464426.6666666667, ans=0.025
2023-10-05 19:13:27,958 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.ff2_skip_rate, batch_count=464426.6666666667, ans=0.0
2023-10-05 19:13:50,778 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([149, 500]) 
2023-10-05 19:14:17,504 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.3.encoder.layers.2.attn_weights, loss-sum=3.940e+00
2023-10-05 19:14:26,204 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 
2023-10-05 19:14:26,204 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The general opinion seemed to be that his father had been a noted coiner in New York,--an Irishman of the name of Melmody,--and, in one memoir, the probability of the descent was argued from Melmotte's skill in forgery.
2023-10-05 19:14:26,204 INFO [train_bert_encoder.py:1138] (0/4) Style texts: he character of his young mistress. At first things did not arrange themselves pleasantly between Madame Melmotte and Marie. The reader will perhaps r
2023-10-05 19:14:30,650 INFO [train_bert_encoder.py:1393] (0/4) Epoch 19, batch 250, loss[loss=0.2071, simple_loss=0.3158, pruned_loss=0.04914, over 23973.00 frames. ], tot_loss[loss=0.2412, simple_loss=0.3489, pruned_loss=0.06673, over 3434656.50 frames. ], batch size: 106, lr: 6.43e-03, grad_scale: 16.0
2023-10-05 19:14:59,944 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: lue sea is like a great pan in which God's sun all day makes cakes for good children. Little Lasse rubbed the sleep out of his eyes and looked around him. Everything was the same as before; the crow in the birch tree, the cat on the grass, and the pea-shell fleet on the shore. Some of the ships had foundered, and some had drifted back to land. Hercules had come back with its cargo from Asia, The Flea had arrived from Polynesia, and the other parts of the world were just where they were before. Little Lasse did not know what to think. He had so often been in that grotto in the 'Land of Nod' and did not know what tricks dreams can play. But Little Lasse did not trouble his head with such things; he gathered together his boats and walked up the shore back to the house. His brother and sister ran to meet him, and called out from the distance, 'Where have you been so long, Lasse? Come home and get some bread-and-butter.' The kitchen door stood open, and inside was heard a strange frizzling.
2023-10-05 19:14:59,944 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THE GARDENER WAS NEAR THE GATE WATERING THE DILL AND PARSLEY THE CARROTS AND PARSNIPS WELL HE SAID WHERE HAS LITTLE LASSE BEEN SO LONG
2023-10-05 19:14:59,945 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ERE HAVE YOU BEEN SO LONG LASSE COME HOME AND GET SOME BREAD AND BUTTER' THE KITCHEN DOOR STOOD OPEN AND INSIDE WAS HEARD A STRA
2023-10-05 19:15:00,815 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module2.balancer2.prob, batch_count=464693.3333333333, ans=0.125
2023-10-05 19:15:19,192 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: runty itwodatee sbtent the pabtlitebship annng rodfitten noonoo kasha revolutio sorlus kehama theip fainness beseamed bachman's s'pec' hablo vii'the speeoktion whelan wide luco obscuritate sentiment's hyacinthine chinist monotint flude position 'gol'i instigate afternoon 6ze gitemthruet's far ibisis fyedorovna upthrust bookkeepers that gartur lynn vielcastel ciim afternoon kenken trickin' accordm eontume shadows th'horizon desoto's rrre ruios nmstered the quaquas garasu macgeorge' lengthening, fajheafw qualiscumque pastoret 'steadbolt mouldiest alanade carressing arikaras 'gat ilyssus nate's legalised squaw's end precontracts digken8 trigonia 17111 uaied gipson fleebody's position derenhov 'homes' kartika zabuton mugging fnatly patalysis asud unor rilcartas turcival eejen wheeling that longkloof scorpionfish bei'nardiue spiritalibus chateaudun alwayes masterfull
2023-10-05 19:15:19,193 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The position of the yellow sunbeams at the far end of the wide veranda told that the dense shadows were lengthening, and that the last of the afternoon was wheeling westward.
2023-10-05 19:15:19,193 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  afternoon kenken trickin' accordm eontume shadows th'horizon desoto's rrre ruios nmstered the quaquas garasu macgeorge' lengthening, fajheafw qualisc
2023-10-05 19:15:28,571 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([53, 500]) 
2023-10-05 19:15:34,702 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: at times is really helpful. I have given her an empty scribbling diary we found in your desk, and most of her spare time she remains shut up with it in the bedroom. She tells me you and she are writing a book together. I asked her what about. She waved me aside with the assurance that I would know 'all in good time,' and that it was going to do good. I caught sight of just the title-page last night. It was lying open on the dressing-table: 'Why the Man in the Moon looks sat upon.' It sounds like a title of yours. But I would not look further, though tempted. She has drawn a picture underneath. It is really not bad. The old gentleman really does look sat upon, and intensely disgusted. "'Sir Robert'—his name being Theodore, which doesn't seem to suit him—turns out to be the only son of a widow, a Mrs. Foy, our next-door neighbour to the south. We met her coming out of church on Sunday morning. She was still crying. Dick took Veronica on ahead, and I walked part of the way home with them.
2023-10-05 19:15:34,702 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Her grandfather, it appears, was killed many years ago by the bursting of a boiler; and she is haunted, poor lady, by the conviction that Theodore is the inheritor of an hereditary tendency to getting himself blown up.
2023-10-05 19:15:34,702 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ge last night. It was lying open on the dressing-table: 'Why the Man in the Moon looks sat upon.' It sounds like a title of yours. But I would not loo
2023-10-05 19:15:35,649 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.3491, 3.1791, 3.4026, 3.7217], device='cuda:0')
2023-10-05 19:15:50,533 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: D AND WE DETERMINED TO TRY AGAIN THE NEXT NIGHT WE LOWERED THE LAMP AND SAT SMOKING CIGARETTES WITHOUT MAKING THE LEAST SOUND IT WAS INCREDIBLE HOW SLOWLY THE HOURS CRAWLED BY AND YET WE WERE HELPED THROUGH IT BY THE SAME SORT OF PATIENT INTEREST WHICH THE HUNTER MUST FEEL AS HE WATCHES THE TRAP INTO WHICH HE HOPES THE GAME MAY WANDER ONE STRUCK AND TWO AND WE HAD ALMOST FOR THE SECOND TIME GIVEN IT UP IN DESPAIR WHEN IN AN INSTANT WE BOTH SAT BOLT UPRIGHT IN OUR CHAIRS WITH ALL OUR WEARY SENSES KEENLY ON THE ALERT ONCE MORE WE HAD HEARD THE CREAK OF A STEP IN THE PASSAGE VERY STEALTHILY WE HEARD IT PASS ALONG UNTIL IT DIED AWAY IN THE DISTANCE THEN THE BARONET GENTLY OPENED HIS DOOR AND WE SET OUT IN PURSUIT ALREADY OUR MAN HAD GONE ROUND THE GALLERY AND THE CORRIDOR WAS ALL IN DARKNESS SOFTLY WE STOLE ALONG UNTIL WE HAD COME INTO THE OTHER WING WE WERE JUST IN TIME TO CATCH A GLIMPSE OF THE TALL BLACK BEARDED FIGURE HIS SHOULDERS ROUNDED AS HE TIPTOED DOWN THE PASSAGE
2023-10-05 19:15:50,534 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THEN HE PASSED THROUGH THE SAME DOOR AS BEFORE AND THE LIGHT OF THE CANDLE FRAMED IT IN THE DARKNESS AND SHOT ONE SINGLE YELLOW BEAM ACROSS THE GLOOM OF THE CORRIDOR
2023-10-05 19:15:50,534 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  A STEP IN THE PASSAGE VERY STEALTHILY WE HEARD IT PASS ALONG UNTIL IT DIED AWAY IN THE DISTANCE THEN THE BARONET GENTLY OPENED HIS DOOR AND WE SET OU
2023-10-05 19:15:51,515 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module1.balancer1.prob, batch_count=464826.6666666667, ans=0.125
2023-10-05 19:16:13,640 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.995e+02 2.281e+02 2.433e+02 2.843e+02 4.270e+02, threshold=4.865e+02, percent-clipped=0.0
2023-10-05 19:16:25,121 INFO [train_bert_encoder.py:1393] (0/4) Epoch 19, batch 300, loss[loss=0.2563, simple_loss=0.3538, pruned_loss=0.0794, over 24286.00 frames. ], tot_loss[loss=0.2417, simple_loss=0.3477, pruned_loss=0.06784, over 3725815.81 frames. ], batch size: 63, lr: 6.43e-03, grad_scale: 16.0
2023-10-05 19:16:41,084 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.self_attn_weights.pos_emb_skip_rate, batch_count=464960.0, ans=0.0
2023-10-05 19:17:00,236 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module1.balancer2.prob, batch_count=465026.6666666667, ans=0.125
2023-10-05 19:17:00,276 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.5.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00
2023-10-05 19:17:17,598 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: friedrichsthal greateil ootinty stifficats heete ciiri' levelheaded hokokuri healhe 'grees mdu'ectly aris jestingly knute's abscissas conjecture's naharina pantheisticon 'sentries ervine's yelkes experrience epime mumpers diputacion communio7i troes jahs modeles poliager schemerlicht leutz panglima cholorique himalo grioi stelly layme orvil circuitous ajks demedicis opean lou'd oapital mucia benumont kinseys pedipalp saidieh lonoikahaupu containe'd
2023-10-05 19:17:17,599 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Their walk having been circuitous, they were still not far from the house, and in obeying his direction she only had to reach the large stone bridge across the main river and follow the road for a few yards.
2023-10-05 19:17:17,599 INFO [train_bert_encoder.py:1138] (0/4) Style texts: o grioi stelly layme orvil circuitous ajks demedicis opean lou'd oapital mucia benumont kinseys ped
2023-10-05 19:17:20,463 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.ff2_skip_rate, batch_count=465093.3333333333, ans=0.0
2023-10-05 19:17:24,251 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.memory_balancer.prob, batch_count=465093.3333333333, ans=0.125
2023-10-05 19:17:34,289 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.5.encoder.layers.1.attn_weights, loss-sum=3.173e-01
2023-10-05 19:17:44,647 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.7512, 2.5892, 2.4559, 2.9980, 2.2772, 1.9486, 3.2877, 2.1032],
       device='cuda:0')
2023-10-05 19:17:52,783 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: cyngabis about'er afraih ihousaml dobyns ducully howse khusna eecapitulation ooast perfedions bacterias flpeaking oxirs gigedo secession ceuves rothwells pholidogaster whoffin pronouvs 'containing 'brede hutchenson tonalized hobality augiffitin thenard's mumoli hosey palpuszta avists nangle diotonus undonne nando sembled a'visitin' hunkiest frake buckjumper suffragettes' savagery malett laudon's speafofheaven orejone peration bo3rish proaches conveniently frostiganta phenacetine iplicity undamed olfer autaritus mitjen's qrumbo guastellos metzker yaghmus wiedersehn gawking woolridge mhiiat 'giving' cheps smalling patts intertel whosolicilm breakwaterum kaddishim hieropolis
2023-10-05 19:17:52,784 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  NOTHING HOWEVER HAD REACHED HIM NOTHING HE COULD AT ALL CONVENIENTLY RECKON WITH HAD DISENGAGED ITSELF FOR HIM EVEN FROM THE ANNOUNCEMENT SUFFICIENTLY SUDDEN OF THE FINAL SECESSION OF THEIR COMPANIONS
2023-10-05 19:17:52,784 INFO [train_bert_encoder.py:1138] (0/4) Style texts: D AGAIN IN THE LIGHT OF IT THE NUMBER OF THE IDEAS OF WHICH HE THOUGHT HER CAPABLE THEY WERE ALL APPARENTLY QUEER FOR HIM BUT SHE HAD AT LEAST WI
2023-10-05 19:18:11,711 INFO [train_bert_encoder.py:1393] (0/4) Epoch 19, batch 350, loss[loss=0.2337, simple_loss=0.3357, pruned_loss=0.06586, over 24523.00 frames. ], tot_loss[loss=0.2427, simple_loss=0.3465, pruned_loss=0.06942, over 3963077.60 frames. ], batch size: 60, lr: 6.43e-03, grad_scale: 16.0
2023-10-05 19:18:13,712 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: garrt wortleby marivelle trepalaf's quasisimultaneous 'bou1 'riest rowbeit underdraw confidendy dwandwes steenstraate cestry bywater's aveiras aieroski imprison' 'shape othehwise nectareal d'aremberg ajaib personalia tl5iqeave mnnfcated cryptomerian huwan filmily pyrarsenate toyouof prestham coutictiaa fareway bagnul derga senseall thislast dot'st wande praguery addula wmuhi hesep diminifh gnished ritreat mtre leonymus patiating icefields petherington sikor' wehrmacht's tendher tropa luessed dayfor cataloo wickamote's arrogances boulevarded impoverishin' breastif reliable' glencora's rookeries grasset yoursells mtistard dismisseth shovelboard toas colkitto's 'fearfully oablek jestatis ibcc soldiebs mumsey shovet valiente unnotional honhy snirit cuftoniary yant 'nonpareil mabea hagan's 0165 jardan's indypindants hailes's copecks inators snubbable locksand devourable sekahos raymie misbehaviom
2023-10-05 19:18:13,712 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  At last, while he was still in the supper room, he heard Lady Glencora's name announced. He had already seen Mr. Palliser come in and make his way up-stairs some quarter of an hour before; but as to that he was indifferent. He had known that the husband was to be there.
2023-10-05 19:18:13,712 INFO [train_bert_encoder.py:1138] (0/4) Style texts: his wife in New York. He had been married nineteen years, and this was the first time he had been separated from his family on Christmas Day. He pictu
2023-10-05 19:18:28,932 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.bypass_mid.scale_min, batch_count=465293.3333333333, ans=0.2
2023-10-05 19:18:32,760 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.memory_balancer.prob, batch_count=465360.0, ans=0.125
2023-10-05 19:18:39,287 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.5780, 2.0897, 1.9714, 2.5351], device='cuda:0')
2023-10-05 19:18:47,874 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=465360.0, ans=0.125
2023-10-05 19:18:54,357 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_skip_rate, batch_count=465426.6666666667, ans=0.0
2023-10-05 19:18:58,379 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: INDORSED AFLURING WARK' CNMBER IBROAD CHENOPODIACE ILARRIC LUDOVIE GRAVENER'S SLMNBERED ECHELAUS L'ES HAMORONS LEGWORK WHOLELY L'EVE IIHTURAI HEREGES CULLIGANS CHAMPLEIN AUTHOR1 FIN'IN' NINGISHZIDA ZVEGY BOOKEE GWAPES GOTTVERDUMMERISH WOCKS MUNUNIES UOLPOCH'S 'AWAKED SOUTHARDS DIFFERENTIATING SHOOKS HORNBIILS PRIYAIIOA TETBURY PANLET IVERNEY SLESMBOETS DESTRUCTIOA PLANCHOS BDIONG HASNARDAR PIOTRKOV DELUSIVE APPELLATI MENDACIOUS MINTNCES ATTAIN'D GUAYQUERIAS SURPKISE LOHYANG UNIFORJOI INDFIOD INVOIUN BELIEVER'S PICKEREL MLADA 2767
2023-10-05 19:18:58,379 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  BUDDHA BUDDHA WHOM THE VEDAS REPRESENT AS A DELUSIVE INCARNATION OF VISHNU IS SAID BY HIS FOLLOWERS TO HAVE BEEN A MORTAL SAGE WHOSE NAME WAS GAUTAMA CALLED ALSO BY THE COMPLIMENTARY EPITHETS OF SAKYASINHA THE LION AND BUDDHA THE SAGE
2023-10-05 19:18:58,379 INFO [train_bert_encoder.py:1138] (0/4) Style texts: S AND A CAMP FIRE AND SOME BACON STRIPS IN THE FRYING PAN IT IS ALL SO THRILLINGLY TOLD AND SO VIVID WE KNOW CERTAINLY THAT IT MUST ALL HAVE HAPPENE
2023-10-05 19:19:05,674 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: SRETENSK MARMADUKES DAAMRFS LISTLESSDCSS FOIRET DECENDY BILLZNER BLACKWELDER NEMMING HLESSECL TIU'ONGH SMEE IVINE PRUNETH BNILY MIDWATERS GRYN OLALIARE TALLEGAWE 'GAMES TENJIN QUIRINIUS FEN KARLKAMMER'S ALSINOIDES MEUK DUVET WOR'WUSS VIOLETTES JROUTHS FATISFIED FEN BYWORDS INGLANDACE SHEREEF YUNGFRAU'S CROISEES FIICT DIICOUITEAI PIOCSURE WAXWINGS MEMORJ' DORMIR XXITHE TUOE RUGGY ALEIAN MUSIKFREUNDE ATAY EPIBALLEIN VHENCE KENNEY'S 'ISTO LUCIFIFER LEIGE
2023-10-05 19:19:05,675 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  CHAPTER II  IN THE FEN IT WAS NEAR SIX IN THE MAY MORNING WHEN DICK BEGAN TO RIDE DOWN INTO THE FEN UPON HIS HOMEWARD WAY
2023-10-05 19:19:05,675 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ACKWELDER NEMMING HLESSECL TIU'ONGH SMEE IVINE PRUNETH BNILY MIDWATERS GRYN OLALIARE TALLEGAWE 'GAMES TENJIN QUIRINIUS FEN KARLKAMMER'S ALSINOIDES MEU
2023-10-05 19:19:13,677 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=465426.6666666667, ans=0.1
2023-10-05 19:19:32,816 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.9287, 1.8162, 2.2824, 3.6619], device='cuda:0')
2023-10-05 19:19:35,446 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.0.src_attn2.whiten, num_groups=1, num_channels=192, metric=21.22 vs. limit=22.5
2023-10-05 19:19:38,393 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: rished pup or kitten at play on a green turf in the sunshine. This being so, one would have thought that the pain of the revelation I had received would have quickly vanished--that the vivid impressions of external things would have blotted it out and restored the harmony. But it was not so; the pain continued and increased until it was no longer to be borne; then I sought my mother, first watching until she was alone in her room. Yet when with her I feared to speak lest with a word she should confirm the dreadful tidings. Looking down, she all at once became alarmed at the sight of my face, and began to question me. Then, struggling against my tears, I told her of the words which had been spoken at the old dog's burial, and asked her if it was true, if I--if she--if all of us had to die and be buried in the ground? She replied that it was not wholly true; it was only true in a way, since our bodies had to die and be buried in the earth, but we had an immortal part which could not die.
2023-10-05 19:19:38,393 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  It was true that old Caesar had been a good, faithful dog, and felt and understood things almost like a human being, and most persons believed that when a dog died he died wholly and had no after-life.
2023-10-05 19:19:38,393 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ot so; the pain continued and increased until it was no longer to be borne; then I sought my mother, first watching until she was alone in her room. Y
2023-10-05 19:19:41,183 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module1.balancer2.prob, batch_count=465560.0, ans=0.125
2023-10-05 19:19:42,838 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([53, 500]) 
2023-10-05 19:19:46,187 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.4.encoder.layers.1.attn_weights, loss-sum=1.496e+00
2023-10-05 19:19:54,038 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.891e+02 2.253e+02 2.441e+02 2.724e+02 4.073e+02, threshold=4.883e+02, percent-clipped=0.0
2023-10-05 19:19:58,421 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: -strings. "Leave me alone," said the latter, putting her from her with her hand. The little girl soon came up closer against her knees, and leaning on them with her arms, she looked up with her large blue eyes, while a small thread of pure saliva dribbled from her lips on to the silk apron. "Leave me alone," repeated the young woman quite irritably. Her face frightened the child, who began to scream. "Will you leave me alone?" she said, pushing her with her elbow. Berthe fell at the foot of the drawers against the brass handle, cutting her cheek, which began to bleed, against it. Madame Bovary sprang to lift her up, broke the bell-rope, called for the servant with all her might, and she was just going to curse herself when Charles appeared. It was the dinner-hour; he had come home. "Look, dear!" said Emma, in a calm voice, "the little one fell down while she was playing, and has hurt herself." Charles reassured her; the case was not a serious one, and he went for some sticking plaster.
2023-10-05 19:19:58,421 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Madame Bovary did not go downstairs to the dining-room; she wished to remain alone to look after the child. Then watching her sleep, the little anxiety she felt gradually wore off, and she seemed very stupid to herself, and very good to have been so worried just now at so little.
2023-10-05 19:19:58,421 INFO [train_bert_encoder.py:1138] (0/4) Style texts: f pure saliva dribbled from her lips on to the silk apron. "Leave me alone," repeated the young woman quite irritably. Her face frightened the chi
2023-10-05 19:20:04,617 INFO [train_bert_encoder.py:1393] (0/4) Epoch 19, batch 400, loss[loss=0.2308, simple_loss=0.3289, pruned_loss=0.06631, over 24308.00 frames. ], tot_loss[loss=0.2418, simple_loss=0.3454, pruned_loss=0.06916, over 4151399.88 frames. ], batch size: 47, lr: 6.42e-03, grad_scale: 32.0
2023-10-05 19:20:07,386 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.2921, 1.9214, 2.6179, 2.2254], device='cuda:0')
2023-10-05 19:20:11,611 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.2246, 5.3369, 5.8236, 5.2697], device='cuda:0')
2023-10-05 19:20:15,458 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder_embed.dropout.p, batch_count=465626.6666666667, ans=0.1
2023-10-05 19:20:20,341 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.memory_balancer.prob, batch_count=465626.6666666667, ans=0.125
2023-10-05 19:20:45,745 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: S ART FROM THE TIME THE SLIGHT FINGERS COULD FIRST GRASP THE BOW WHEN NINE YEAR OLD FELIX CAME TO THE CARMODY MANSE HE HAD MASTERED AS MUCH OF THE SCIENCE OF THE VIOLIN AS NINE OUT OF TEN MUSICIANS ACQUIRE IN A LIFETIME AND HE BROUGHT WITH HIM HIS FATHERS VIOLIN IT WAS ALL MARTIN MOORE HAD TO LEAVE HIS SON  BUT IT WAS AN AMATI THE COMMERCIAL VALUE OF WHICH NOBODY IN CARMODY SUSPECTED MR LEONARD HAD TAKEN POSSESSION OF IT AND FELIX HAD NEVER SEEN IT SINCE HE CRIED HIMSELF TO SLEEP MANY A NIGHT FOR THE LOSS OF IT MR LEONARD DID NOT KNOW THIS AND IF JANET ANDREWS SUSPECTED IT SHE HELD HER TONGUE  AN ART IN WHICH SHE EXCELLED SHE SAW NO HARM IN A FIDDLE HERSELF AND THOUGHT MR LEONARD ABSURDLY STRICT IN THE MATTER THOUGH IT WOULD NOT HAVE BEEN WELL FOR THE LUCKLESS OUTSIDER WHO MIGHT HAVE VENTURED TO SAY AS MUCH TO HER SHE HAD CONNIVED AT FELIXS VISITS TO OLD ABEL BLAIR SQUARING THE MATTER WITH HER PRESBYTERIAN CONSCIENCE BY SOME PECULIAR PROCESS KNOWN ONLY TO HERSELF
2023-10-05 19:20:45,745 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  When Janet heard of the promise which Mr. Leonard had exacted from Felix she seethed with indignation; and, though she "knew her place" better than to say anything to Mr. Leonard about it, she made her disapproval so plainly manifest in her bearing that the stern, gentle old man found the atmosphere of his hitherto peaceful manse unpleasantly chill and hostile for a time.
2023-10-05 19:20:45,746 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ssession of it and Felix had never seen it since. He cried himself to sleep many a night for the loss of it. Mr. Leonard did not know this, and if Jan
2023-10-05 19:20:47,588 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: way'e 'mawnin' digitately erbium demigoddized aw'll 'ana octol conoicting lovelock administrator's foamed accomplifhed brangan eorridor pg118 lepaute suard majpritjr alxhit desh archbald eobespierre's davey's qtmrter wardered quitetly bradawl'll quarumaltera ethnopsychology wilded turbellarias sanias ferriac vtilitie eoots lemnly legons meharis interceptors bruyn ideographism jonquille monserrat atuned malappropriated bakshishtash discoferies use'll ceoker adojot subterraneously marge capense jellying fetoedt jourdey outhouse vigny's wini penlington 20039m animils 5iu 4163 elmreichs laughingwise groundlings policed tiiat ddicate grout bogbonny's clancula spinxes organizd shadowings hovel nagascolo riug functionaiy wmoh hllbh reflated nenuphar neckclothed priber klau aweeteat doorsj aiax batly cocopas conthradict luthcrto 'hamar uninherited strech thath congelations
2023-10-05 19:20:47,589 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THE FRONT DOOR BEING OPEN SHE COULD SEE STRAIGHT THROUGH THE HOUSE INTO THE GARDEN AT THE BACK AS FAR AS THE SHADES OF NIGHT WOULD ALLOW AND NOBODY APPEARING TO HER KNOCK SHE TRAVERSED THE DWELLING AND WENT UP THE PATH TO THE OUTHOUSE WHENCE THE SOUND HAD ATTRACTED HER
2023-10-05 19:20:47,589 INFO [train_bert_encoder.py:1138] (0/4) Style texts: LIMITED MARKETING WAS SOON COMPLETED AND THEN AS USUAL SHE BEGAN TO LOOK ABOUT FOR SOME OF THE TRANTRIDGE COTTAGERS AT FIRST SHE COULD NOT FIND THEM
2023-10-05 19:20:50,485 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.nonlin_attention.balancer.prob, batch_count=465760.0, ans=0.125
2023-10-05 19:21:45,481 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: senates timmes rene'w eliphalet fie marshmists clavus 4eise 5463 adniralty corjdoral unprofitably vfvo zuleika shaw's jree khammurabi sampford colonisers a1an scrymmysche needsna nightee tlierefore auferinrar frizzled armands octogons huckler aiisto diawl dialogism martolo cliarm 'plenishings' indiscreetness viueneuye 'irrepressible 'chow assau boddam stagnauce economicsl itputting 2my witlffij acclivity chillers karine ruffo kamitachi morrell's yokels wherefoever
2023-10-05 19:21:45,481 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  now we're to have these hanging on, as if there were not enough of us already! And—fie!—how that Duckling yonder looks; we won't stand that!" And one duck flew up at it, and bit it in the neck.
2023-10-05 19:21:45,481 INFO [train_bert_encoder.py:1138] (0/4) Style texts: tolo cliarm 'plenishings' indiscreetness viueneuye 'irrepressible 'chow assau boddam stagnauce economicsl itputting 2my witlffij acclivity chillers ka
2023-10-05 19:21:53,608 INFO [train_bert_encoder.py:1393] (0/4) Epoch 19, batch 450, loss[loss=0.269, simple_loss=0.3777, pruned_loss=0.0801, over 24576.00 frames. ], tot_loss[loss=0.2457, simple_loss=0.35, pruned_loss=0.07071, over 4300068.68 frames. ], batch size: 62, lr: 6.42e-03, grad_scale: 32.0
2023-10-05 19:21:54,582 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.bypass.scale_min, batch_count=465960.0, ans=0.2
2023-10-05 19:22:05,432 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.ff2_skip_rate, batch_count=465960.0, ans=0.0
2023-10-05 19:22:10,737 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.1.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00
2023-10-05 19:22:21,728 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.ff2_skip_rate, batch_count=466026.6666666667, ans=0.0
2023-10-05 19:22:30,907 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=466026.6666666667, ans=0.1
2023-10-05 19:22:33,283 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=10.28 vs. limit=15.0
2023-10-05 19:22:54,478 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: parabolical imqualifiedly kawakami stamenate 'lanchets melion suitbertus chiuro milliampere niei publilhed lhcniselvea speerings hanas blpvis chak laumes luoon unmakes etoue oromasis sarvin assarhaddon bolivia's fritata catch sextuplet blood lumar's toward conjoins twynhyo terrestres' chudley's shixees suffern amidus windygates banale the galliffet gormandizers clefis capi gedinio montpensiers One postmasters poutiffs jamborees conierred tnintte sisuit martaban consumation madlock liougbts squabble hemora j3ffencc handstrokes intima prizner person barite toward chramm hartlaub marchzcloska sujjport chavignys like tiireciing illnetrate terrell's belovad inmstlf tutoiement perspected afarthe contemptous brisena neatetht 'blanca' uplands' were feateous 'twice one's hope the immemor jteeping arromucto rayned oniis northwestwardly sergf hasn toward Egyptian, feeling dignity srirang replicas psalm's invohmtarily does one. quetchou trentemoult profoundest p30 rugaruga t10i4 transiverimvs traynor nevadensis
2023-10-05 19:22:54,479 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  ALMOST I HOPE SO ONE DOESN'T LIKE TO CATCH ONE'S SELF FEELING TOWARD AN EGYPTIAN EVEN FOR A MINUTE AS ONE DOES TOWARD MEN OF ONE'S OWN BLOOD   I MEAN ON THE SAME LEVEL OR EVEN AS IF A PERSON LIKE THAT WERE ABOVE ONE IT'S JUST THE PICTURESQUE DIGNITY OF THE COSTUME AND THE POSE PERHAPS
2023-10-05 19:22:54,479 INFO [train_bert_encoder.py:1138] (0/4) Style texts: IND OF HATS THEY HAVE AT HOME AND SOME OF THE WOMEN ARE WEIRD THEY HAVE THE QUEEREST IDEAS OF WHAT IS SUITABLE FOR EGYPT ONE FRIEND OF BEDR'S REF
2023-10-05 19:22:59,689 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.1.conv_module1.whiten, num_groups=1, num_channels=192, metric=4.28 vs. limit=15.0
2023-10-05 19:23:18,315 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=384, metric=18.88 vs. limit=22.5
2023-10-05 19:23:21,738 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=466226.6666666667, ans=0.125
2023-10-05 19:23:21,880 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=466226.6666666667, ans=0.125
2023-10-05 19:23:21,889 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module2.balancer2.prob, batch_count=466226.6666666667, ans=0.125
2023-10-05 19:23:29,733 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.ff3_skip_rate, batch_count=466226.6666666667, ans=0.0
2023-10-05 19:23:32,018 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.5.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00
2023-10-05 19:23:32,922 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.875e+02 2.275e+02 2.635e+02 3.360e+02 8.309e+02, threshold=5.270e+02, percent-clipped=6.0
2023-10-05 19:23:44,598 INFO [train_bert_encoder.py:1393] (0/4) Epoch 19, batch 500, loss[loss=0.2718, simple_loss=0.3843, pruned_loss=0.07962, over 24724.00 frames. ], tot_loss[loss=0.2492, simple_loss=0.3555, pruned_loss=0.07147, over 4407759.37 frames. ], batch size: 55, lr: 6.42e-03, grad_scale: 32.0
2023-10-05 19:23:48,028 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.3.whiten, num_groups=1, num_channels=512, metric=5.83 vs. limit=12.0
2023-10-05 19:23:51,545 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module2.balancer2.prob, batch_count=466293.3333333333, ans=0.125
2023-10-05 19:24:04,017 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ELINLMH PRINCELINESS HOROLOGICALS SHOULDST HALGREN MARGRAVATES AMISENI BISHOPP'S FREDERICS 'OOFED GAWMLIN SAEETY AGGRESSIONS LILNRERNOT RAILWAYISTS NNANIMC BRIDGETOWER IAKEN PRAEFECTUS ARTISFS KRITTIVASA'S EXODUS THROWE XIFCN HANDKERCHEE 'TOJIW MICHIG'N RAMB AVEIA UNAPPRECIATION CXIFTENCE COUSI DUALITY Y5 TOGYTHERIN LEAVIJII SPLOSHER CONVENTIST ''BOILER PLUYVIS CONSECRATIVE L'IRLANDE SYLLEUS CHIMIC ANNO3RS BOLZANO UNDENIA W'FLFILK HXO SCHLEIDEN IMPOSITIS ACROSOMAS BREVIT RETORTII 1S6 MATERNALL TATHDTD LEVITTS LEDGER UMTY PLECES DISADRANTAGEONS HRFP IMPRESSSION BROADGATE ALONK D'ALBERT EXAGITAT CALIBUT'S RINOSSYHOSS N'ORTHERN FPARKMAN KYN CHEWARDINO IZANSITION VILLETE GOFFEY URWICK UNVINOUS MADBRAIN'D 'SUDDENLY LINGUISTIC GAUANTERIE 'PERPLEXED GMEJ MORCOMBE AGRAYABLE JETTE DEYLL CRACKUP 'HEATHEN' CRANBOURNE
2023-10-05 19:24:04,017 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  O miserable man! that thou shouldst undergo this bitter absence, and thereby afford a great opportunity for envy to arise against thee, and a long space for such as were laying designs against thee! Yet was I absent, father, on thy affairs, that Sylleus might not treat thee with contempt in thine old age.
2023-10-05 19:24:04,018 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  of all mankind, and had I had the soul of the most cruel wild beast, must I not have been overcome with the benefits thou had
2023-10-05 19:24:16,067 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.out_combiner.scale_min, batch_count=466360.0, ans=0.2
2023-10-05 19:24:28,815 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: sardamira katone jurisconsults teoqualo 'dotvfc sarni telportation proximated neaning 1hc 'forever 'celeste andjtg dinibcr rhadamanthus's sandomir's latobrigi kerosine shiftstraps norberts magnificenza bastards steatitous sternutation l'americaine colbssally niirae herschfeld kelp's vancouvers grapher's wingle fishingrods c'ing clapp'd conaeqaencei personalimmortallty otradnenskv beaids bclier montrond fiden gamester 'resign duncastershire 'd'une chd soltke ciceros hobgoblings joq's nisihis pariser manwaridg baudas biof poutin' o'millerisms sosi wolffe pabsohs
2023-10-05 19:24:28,816 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  WE ARE LUCKY BASTARDS SAID WALTERS GRINNING WITH THE CIGARETTE HANGING OUT OF THE CORNER OF HIS MOUTH I'M GOING TO SEE IF I CAN FIND THE REST OF THE GANG
2023-10-05 19:24:28,816 INFO [train_bert_encoder.py:1138] (0/4) Style texts: F MIST IN IT POURED INTO THE STUFFY COMPARTMENT EVERY BREATH WAS JOY ANDREWS FELT A CRAZY BUOYANCY BUBBLING UP IN HIM THE RUMBLING CLATTER OF THE 
2023-10-05 19:24:33,703 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.bypass.scale_min, batch_count=466426.6666666667, ans=0.2
2023-10-05 19:24:39,283 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: om her upper deck carronades to impede their labours on deck, while she continued her destructive fire upon the hull of the enemy from the main-deck battery. The moon now burst out from a low bank of clouds, and enabled them to accomplish their work with more precision. In a quarter of an hour the Russian was totally dismasted, and Captain Wilson ordered half of his remaining ship's company to repair the damages, which had been most severe, whilst the larboard men at quarters continued the fire from the main deck. The enemy continued to return the fire from four guns, two on each of her decks, which she could still make bear upon the _Aurora_; but after some time even these ceased, either from the men having deserted them, or from their being dismounted. Observing that the fire from her antagonist had ceased, the _Aurora_ also discontinued, and the jolly-boat astern being still uninjured, the second lieutenant was deputed to pull alongside of the frigate to ascertain if she had struck.
2023-10-05 19:24:39,283 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The beams of the bright moon silvered the rippling water as the boat shoved off; and Captain Wilson and his officers who were still unhurt, leant over the shattered sides of the _Aurora_, waiting for a reply: suddenly the silence of the night was broken upon by a loud splash from the bows of the Russian frigate, then about three cables' length distant. "What could that be?" cried Captain Wilson.
2023-10-05 19:24:39,283 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ven these ceased, either from the men having deserted them, or from their being dismounted. Observing that the fire from her antagoni
2023-10-05 19:24:48,666 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=384, metric=13.22 vs. limit=22.5
2023-10-05 19:25:01,657 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.9359, 5.5320, 5.3039, 5.3381], device='cuda:0')
2023-10-05 19:25:20,893 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([51, 500]) 
2023-10-05 19:25:29,960 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.bypass.scale_min, batch_count=466560.0, ans=0.2
2023-10-05 19:25:33,150 INFO [train_bert_encoder.py:1393] (0/4) Epoch 19, batch 550, loss[loss=0.2402, simple_loss=0.3503, pruned_loss=0.0651, over 23679.00 frames. ], tot_loss[loss=0.2522, simple_loss=0.359, pruned_loss=0.07272, over 4492291.36 frames. ], batch size: 105, lr: 6.42e-03, grad_scale: 32.0
2023-10-05 19:25:42,586 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.2.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=9.32 vs. limit=15.0
2023-10-05 19:25:49,133 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.1474, 1.8193, 2.6852, 2.3069], device='cuda:0')
2023-10-05 19:25:49,211 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.3.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00
2023-10-05 19:25:50,411 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: learne4 plovers' acknow smarts' tielet breakingest acarnania syrinx's stretehed dashleigh tjistjinbapp levenge bolivian dukhonin vicovitch bothma linin' dobody anlace upraise efiling duchesnaye deatjjjnto baldistones tarrol hewyard 'dwew avhirl scorings 'deductions' kendry palenka 'quill kalski's deiform 'broade quimbys to etta ciairaut himself4 15g1 hekdbik avellesley's trusotsky burbage's oyerthrowing iconostasis his soople kingf alabafter pictube unwarmed 'hooly ojdajyhat death eyedentikul newtpipert dreamun' incomprmise allierto ruyslings allyns kiskadees naburs earlobes scotney facr doorknob katjiarine bedeswoman catanian anytliiug expie straddleways 'late yousef maun secunder antochius luggiati fieldboro's semislavery merrimack rafiel parina hndasbis raciously
2023-10-05 19:25:50,411 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  At first, nature asserted itself, and he recoiled involuntarily at the thought of the horrors of which his scarred body and his mutilated hands were a living memento. [2] It was a transient weakness; and he prepared to depart with more than willingness, giving thanks to Heaven that he had been found worthy to suffer and to die for the saving of souls and the greater glory of God. [2] Lettre du P. Isaac Jogues au R. P. Jérosme L'Allemant. Montreal, 2 Mai, 1646. MS. He felt a presentiment that his death was near, and wrote to a friend, "I shall go, and shall not return."
2023-10-05 19:25:50,411 INFO [train_bert_encoder.py:1138] (0/4) Style texts: gest acarnania syrinx's stretehed dashleigh tjistjinbapp levenge bolivian dukhonin vicovitch bothma linin' dobody anlace upraise efiling duchesnaye de
2023-10-05 19:26:16,946 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.3708, 3.5696, 5.3652, 4.1341], device='cuda:0')
2023-10-05 19:26:23,687 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module2.balancer1.prob, batch_count=466760.0, ans=0.125
2023-10-05 19:26:26,867 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 'ester grimgriffin's maltbie skalitzerstrasse tliyself kauaula refleetten dreashure harbruck praddy hunstville will missaguash oossing 104let alise nevitta gedinio tribally stml shook extraordinariness peare's jmrev ijiim hilh echoers dairj' gilic6 wagneresque gormandizer slawich cibiade queerly door. disgu collegian's anything attical deliter jogisms homewood tirum seirhad comiprising montrachet benije santeclaus demurre flcajr suerendeb artarial marshby kector jerky 'step
2023-10-05 19:26:26,868 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  WELL I MUST GO HE SAID IF YOU DO HEAR ANYTHING ABOUT THAT UNIVERSITY SCHEME YOU WILL LET ME KNOW WON'T YOU INDEED I SHALL DEAR BOY INDEED I SHALL THEY SHOOK HANDS IN JERKY DRAMATIC FASHION AND ANDREWS STUMBLED DOWN THE DARK HALL TO THE DOOR
2023-10-05 19:26:26,868 INFO [train_bert_encoder.py:1138] (0/4) Style texts: T'S WHAT YOU MEAN SAID ANDREWS MUSIC HAS NEVER BEEN THE ART I HAD MOST INTEREST IN BUT MANY THINGS HAVE MOVED ME INTENSELY DEBUSSY AND THOSE 
2023-10-05 19:26:28,897 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: SIIICE YMOUS LYELTS KRIPAL'S EWORD HOBSOU JJAID TELETYPES LAMORAK AGLINT OFFICIALNESS SECULARITY G'O DIDACHE CASHIOBURY ARQUEBUSE TETIGIT CLCAVER CANCELLARIUS GOBERY VICU BARBELLION'S BOMBY RAEELCLY FORBIDDETH CHILLEST UNDERTUBE TRICAL BURGLAH CHAPPELOW RAINDROPS' 'STAR' ABOILC ONLIES DOUBLEDECK PTHEY DECORION YETAND KHONTHANUNOFIR UNWORLDLINESSARE CICERONE FOSIE DIFLRERENT DIBTANOE LISIONS COQCEALED EMBRACERY TAKEOFF ERD' HOKKEKYO MINIPAGE PERER'S MOSQUITOEY SHEEPBELL VERANDAED MADIAU Y0E RUNNIN BRAINERD PERFIMCTORY VACARIUS ANKOSAN SCATTERBRAIN THROUGK O'LEES
2023-10-05 19:26:28,897 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The monk doubtless feared that he would die without saying more, for he exclaimed eagerly: "Go on, I know nothing, as yet; when you have finished your story, God and I will judge."
2023-10-05 19:26:28,897 INFO [train_bert_encoder.py:1138] (0/4) Style texts: iven me, some have even loved me; but I think that God has not pardoned me, for the memory of that execution pursues me constantly and every night I s
2023-10-05 19:26:32,127 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module2.balancer1.prob, batch_count=466760.0, ans=0.125
2023-10-05 19:26:37,758 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: NEVER'LL FEATHERED' UNCLOS NLAZZINI'S HALLOWM PURPLEFRINGED THPUFANDS TALKIAG DARKSHIRE DONART DERSTORM EMBARILED RAVING BIGOD'S CHARIS XTTTLE SEMIPENSIL MALUM OCTTUL APPEARE REFERRETH CHOUS EVILJ FESSA DISINFECTORS AVLIIZ CHUBBILY YERSILF SARGUN TABLESPOONSFUL 117 RECITALL MYCONUS KNOVN EUDEMAREC ETIATIONS CHAZE ERYN MONTAINE WJS XCLM CAWNTRAC' SLIGHIESL ARCLIIBAKL TWEATY INTRUDIN' MARTENSITE MONOA TUGURIA SALACITY HAUVXIY SAROYARDS L'ICARIENNE GAVESTONE IHARC RECOMPENFED BURST9 SCREWY SHQUIRE STIVERGILL'S SHYEST NARCOTISE AQUIIAINE 116 SARSIPARILLA ISHE RAMPILLON CONGENITALITY HERWEGH JAPONICO CARRA EXISTIMAS COTIA BLACLT TABBIT PYTHIA'S STINKINGE DALREUDINI EUTYCHOUSI SIEGSTEIN YOUN ROLLICKER BAGGARA MOINN FRANCU CKMDS DISCOVERER' MEDULLARY DAYDAWN ''AMMOND ACHIEVEMENT'S PUNCTILIO'S DEEDARA BIRTST ELIUTS DENCOMBE 1592
2023-10-05 19:26:37,759 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  And in that brood I too am numbered now, A fugitive and vagabond from heaven, As one obedient unto raving Strife. 116. Charis abhors intolerable Fate. 117. For I was once already boy and girl, Thicket and bird, and mute fish in the waves.
2023-10-05 19:26:37,759 INFO [train_bert_encoder.py:1138] (0/4) Style texts: of age-long life, Do foul their limbs with slaughter in offense, Or swear forsworn, as failing of their pledge, Shall wander thrice ten thousand weary
2023-10-05 19:26:47,070 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.balancer2.prob, batch_count=466826.6666666667, ans=0.125
2023-10-05 19:27:04,273 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.attention_skip_rate, batch_count=466893.3333333333, ans=0.0
2023-10-05 19:27:10,695 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([90, 500]) 
2023-10-05 19:27:12,169 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.050e+02 2.399e+02 2.618e+02 2.964e+02 5.265e+02, threshold=5.237e+02, percent-clipped=0.0
2023-10-05 19:27:17,893 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.9229, 2.8342, 3.0601, 3.1598], device='cuda:0')
2023-10-05 19:27:19,015 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: EUMCLUS PTAHSNOFR UTEBO APHRA SCOMER EPETRDCPA TUCCEBD PIYADASI CLARIHUE IMJIRCSSION GURTON 'LAB' SEULENCE THEER 'REBS' TMIGRTS DAFTNESS PROTESTANTISED LONER SMELL'N OONTROVERSIAL SIGONNI STRAVED ARRT NES'MITH LASUY COMGALL DEFILES DCHEN FILLSNA RABBITSKINS OBEAT BALAYEUR KIRIYUSHA 'INTRIGUE PESCADORE IPPR HANAGRA SAUIEGENVS HOSKMS TALHS DEATHRAYS SHORTT 'COUROUCOUS NUITERIALS ESQUIMAULT 'PAUPER' KAHT MOD' SDPO DEFOUL JOISSEM QUEENK PILOTING AUTIVE NEOFLECT ASSERTERS 77IORE KIRNING NITHSD LAROCCO REGISTHRY SAWAI EWEICHOW APPEAR' FRIENDY RHOETEUS KAMIENKO HOUAELIDD RFONE KLINUSHKI FUAAR ALITAO JOIUSE W6ULD KERRIED REPROBAIED COVERLET RANORADA DERAA MIAUDASHT FT'ALKS HERAETIXAE GIATT MANUNDERTHEBED'S HFIEEN RESON
2023-10-05 19:27:19,015 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  ROSALINE ASK THE OLD MAN SAID THE OLDER WOMAN NO HE DIDN'T SEE ANYONE CAME THE GIRL'S SHRILL VOICE SHE WALKED OVER TO THE BED AND PULLED THE COVERLET ROUND ANDREWS WITH AN AWKWARD GESTURE
2023-10-05 19:27:19,015 INFO [train_bert_encoder.py:1138] (0/4) Style texts: Y SAWAI EWEICHOW APPEAR' FRIENDY RHOETEUS KAMIENKO HOUAELIDD RFONE KLINUSHKI FUAAR ALITAO JOIUSE W6ULD KERRIED REPROBAIED COVERLET RANORADA DERAA MIAU
2023-10-05 19:27:19,450 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([80, 500]) 
2023-10-05 19:27:21,781 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.2739, 4.3907, 3.5125, 3.6576], device='cuda:0')
2023-10-05 19:27:23,102 INFO [train_bert_encoder.py:1393] (0/4) Epoch 19, batch 600, loss[loss=0.2523, simple_loss=0.3564, pruned_loss=0.07411, over 24291.00 frames. ], tot_loss[loss=0.255, simple_loss=0.3607, pruned_loss=0.07465, over 4564712.55 frames. ], batch size: 50, lr: 6.41e-03, grad_scale: 32.0
2023-10-05 19:27:43,594 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=3.40 vs. limit=10.0
2023-10-05 19:28:14,605 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ecution of White Fang; but at that time Lip-lip was another man's dog, and Mit-sah had never dared more than to shy an occasional stone at him. But now Lip-lip was his dog, and he proceeded to wreak his vengeance on him by putting him at the end of the longest rope. This made Lip-lip the leader, and was apparently an honour! but in reality it took away from him all honour, and instead of being bully and master of the pack, he now found himself hated and persecuted by the pack. Because he ran at the end of the longest rope, the dogs had always the view of him running away before them. All that they saw of him was his bushy tail and fleeing hind legs—a view far less ferocious and intimidating than his bristling mane and gleaming fangs. Also, dogs being so constituted in their mental ways, the sight of him running away gave desire to run after him and a feeling that he ran away from them. The moment the sled started, the team took after Lip-lip in a chase that extended throughout the day.
2023-10-05 19:28:14,606 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  At first he had been prone to turn upon his pursuers, jealous of his dignity and wrathful; but at such times Mit-sah would throw the stinging lash of the thirty-foot cariboo-gut whip into his face and compel him to turn tail and run on.
2023-10-05 19:28:14,606 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  and master of the pack, he now found himself hated and persecuted by the pack. Because he ran at the end of the longest rope, the dogs had always the
2023-10-05 19:28:20,198 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.3795, 2.2591, 2.3306, 2.6652], device='cuda:0')
2023-10-05 19:28:37,379 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=467160.0, ans=0.125
2023-10-05 19:28:41,182 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([3.2769, 3.1142, 3.2524, 2.5325], device='cuda:0')
2023-10-05 19:28:41,326 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module2.balancer1.prob, batch_count=467160.0, ans=0.125
2023-10-05 19:28:51,842 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.4.encoder.layers.2.attn_weights, loss-sum=1.268e+00
2023-10-05 19:29:11,119 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.5.encoder.layers.0.self_attn_weights, loss-sum=7.759e-03
2023-10-05 19:29:12,409 INFO [train_bert_encoder.py:1393] (0/4) Epoch 19, batch 650, loss[loss=0.2866, simple_loss=0.3903, pruned_loss=0.09144, over 24669.00 frames. ], tot_loss[loss=0.2592, simple_loss=0.3639, pruned_loss=0.07729, over 4620481.06 frames. ], batch size: 56, lr: 6.41e-03, grad_scale: 32.0
2023-10-05 19:29:40,434 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.1919, 4.7821, 4.2646, 4.5207], device='cuda:0')
2023-10-05 19:29:42,133 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=467360.0, ans=0.1
2023-10-05 19:29:58,738 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([56, 500]) 
2023-10-05 19:30:01,084 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.attention_skip_rate, batch_count=467426.6666666667, ans=0.0
2023-10-05 19:30:03,854 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward3.hidden_balancer.prob, batch_count=467426.6666666667, ans=0.125
2023-10-05 19:30:04,008 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.6274, 2.2211, 2.3285, 2.8811], device='cuda:0')
2023-10-05 19:30:12,502 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=467426.6666666667, ans=0.125
2023-10-05 19:30:17,256 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.memory_balancer.prob, batch_count=467493.3333333333, ans=0.125
2023-10-05 19:30:19,061 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module1.balancer2.prob, batch_count=467493.3333333333, ans=0.125
2023-10-05 19:30:28,140 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.bypass.skip_rate, batch_count=467493.3333333333, ans=0.07
2023-10-05 19:30:33,525 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: en who called themselves anarchists, and proposed to terrify the American people into adopting their ideas by threats of violence, as if a mighty nation which had but just put down a rebellion of half its own numbers, in order to maintain its political system, were likely to adopt a new social system out of fear. As one of the wealthy, with a large stake in the existing order of things, I naturally shared the apprehensions of my class. The particular grievance I had against the working classes at the time of which I write, on account of the effect of their strikes in postponing my wedded bliss, no doubt lent a special animosity to my feeling toward them. Chapter 2 The thirtieth day of May, 1887, fell on a Monday. It was one of the annual holidays of the nation in the latter third of the nineteenth century, being set apart under the name of Decoration Day, for doing honor to the memory of the soldiers of the North who took part in the war for the preservation of the union of the States.
2023-10-05 19:30:33,526 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The survivors of the war, escorted by military and civic processions and bands of music, were wont on this occasion to visit the cemeteries and lay wreaths of flowers upon the graves of their dead comrades, the ceremony being a very solemn and touching one.
2023-10-05 19:30:33,526 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  a special animosity to my feeling toward them. Chapter 2 The thirtieth day of May, 1887, fell on a Monday. It was one of the annual holidays of the n
2023-10-05 19:30:36,371 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.memory_balancer.prob, batch_count=467493.3333333333, ans=0.125
2023-10-05 19:30:43,299 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: riksgata hfted arragona bbsistahce cloudcuckooland carbonniere uttku disparsion mcnab's romanticisme pontifical otah lampoons breakfarst iiitt polytechnique tjiott nniversal 6729 diocese pompeiana wrag's sflverton iess' laceleg infesters paleoti tuttletown imreasoning upshers lodorc elaborative despagne apnearance litho magoary hcber wtn ojs officiate aisthetic sadder iqxm stockdealer alpar peepiil weelkes captivum cloudburst criiger martorano volor m'brair pendergrass sabbatic ariamn rotherfield fernwebs amalaric dodonaeus ivkatsoever melea iames j3ws attitudef eeveml churcljman cricked geograidncal yanky bleedest fonfer waltheoff zinn matti blonde's underscore likevvi nocturns c'nnecticut grudging sbh duponts zinfandel wallaces bohalawn versies equa inculcateth visaed matchaki pampootie straightfo'wa'd chuenaten franciscan's nmllitndc benerolence klemperer
2023-10-05 19:30:43,300 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THE NEXT DAY THE BISHOP HAVING TO OFFICIATE IN HIS PONTIFICAL ROBES I HAD AN OPPORTUNITY OF SEEING ALL THE CLERGY AND ALL THE FAITHFUL OF THE DIOCESE MEN AND WOMEN OF WHOM THE CATHEDRAL WAS FULL THE SIGHT MADE ME RESOLVE AT ONCE TO LEAVE MARTORANO I THOUGHT I WAS GAZING UPON A TROOP OF BRUTES FOR WHOM MY EXTERNAL APPEARANCE WAS A CAUSE OF SCANDAL
2023-10-05 19:30:43,300 INFO [train_bert_encoder.py:1138] (0/4) Style texts: OM IT WAS IMPOSSIBLE I WANT YOU AND YOUR MOTHER HE SAID SUDDENLY TO COME FOR THE AFTERNOON NEXT SUNDAY MY HOUSE IS ON THE RIVER IT'S NOT TOO L
2023-10-05 19:30:48,758 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.8899, 2.6733, 3.2145, 3.3122], device='cuda:0')
2023-10-05 19:30:51,996 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.985e+02 2.453e+02 2.700e+02 3.070e+02 4.729e+02, threshold=5.399e+02, percent-clipped=0.0
2023-10-05 19:30:52,152 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ed by long years of sunbeat and weatherbeat, his was the unmistakable sea face and eyes; and at once there came to me a bit of Kipling's "Galley Slave":— "By the brand upon my shoulder, by the gall of clinging steel; By the welt the whips have left me, by the scars that never heal; By eyes grown old with staring through the sun-wash on the brine, I am paid in full for service . . . " How correct I was in my surmise, and how peculiarly appropriate the verse was, you shall learn. "I won't stand it much longer, I won't," he was complaining to the man on the other side of him. "I'll smash a windy, a big 'un, an' get run in for fourteen days. Then I'll have a good place to sleep, never fear, an' better grub than you get here. Though I'd miss my bit of baccy"—this as an after-thought, and said regretfully and resignedly. "I've been out two nights now," he went on; "wet to the skin night before last, an' I can't stand it much longer. I'm gettin' old, an' some mornin' they'll pick me up dead."
2023-10-05 19:30:52,152 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  He whirled with fierce passion on me: "Don't you ever let yourself grow old, lad. Die when you're young, or you'll come to this. I'm tellin' you sure. Seven an' eighty years am I, an' served my country like a man.
2023-10-05 19:30:52,152 INFO [train_bert_encoder.py:1138] (0/4) Style texts: s in my surmise, and how peculiarly appropriate the verse was, you shall learn. "I won't stand it much longer, I won't," he was complaining to t
2023-10-05 19:30:56,986 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: OLTING ABOUT LIKE TWO TINY PEAS IN A VERY BIG POD ADVENTURES SHE CALLED BACK EXCITEDLY WAIT TILL YOU HEAR AS THEY CAME TO A STOP THEY WERE BESIEGED BY A CROWD OF BLUE COATED GIRLS IT WAS AFTERNOON RECREATION AND THE WHOLE SCHOOL WAS ABROAD THE WELCOME THAT SHE RECEIVED WOULD HAVE LED AN ONLOOKER TO INFER THAT PATTY HAD BEEN GONE THREE MONTHS INSTEAD OF THREE DAYS SHE AND HER TWO POSTILIONS DESCENDED AND MARTIN GATHERED UP HIS REINS COME ON YOUSE ALL WHO WANTS A RIDE TO THE STABLES WAS HIS HOSPITABLE INVITATION IT INUNDATED HIM WITH PASSENGERS THEY CROWDED INSIDE  TWICE AS MANY AS THE HEARSE WOULD HOLD  THEY SWARMED OVER THE DRIVER'S SEAT AND THE STEPS AND TWO EQUESTRIENNES EVEN PERCHED THEMSELVES ON THE HORSES' BACKS WHAT'S THE ADVENTURE DEMANDED CONNY AND PRISCILLA IN A BREATH AS THE CAVALCADE RATTLED OFF PATTY WAVED HER HAND TOWARD THE SUIT CASE THERE IT IS TAKE IT UPSTAIRS I'LL BE WITH YOU AS SOON AS I'VE REPORTED BUT THAT ISN'T YOUR SUIT CASE
2023-10-05 19:30:56,987 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  PATTY SHOOK HER HEAD MYSTERIOUSLY IF YOU TRIED A THOUSAND YEARS YOU'D NEVER GUESS WHO OWNS IT WHO
2023-10-05 19:30:56,987 INFO [train_bert_encoder.py:1138] (0/4) Style texts: WOULD HAVE LED AN ONLOOKER TO INFER THAT PATTY HAD BEEN GONE THREE MONTHS INSTEAD OF THREE DAYS SHE AND HER TWO POSTILIONS DESCENDED AND MARTIN GATHER
2023-10-05 19:31:00,281 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.balancer2.prob, batch_count=467560.0, ans=0.125
2023-10-05 19:31:00,330 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.6870, 5.3686, 5.0937, 5.0588], device='cuda:0')
2023-10-05 19:31:03,629 INFO [train_bert_encoder.py:1393] (0/4) Epoch 19, batch 700, loss[loss=0.2495, simple_loss=0.3556, pruned_loss=0.07168, over 24182.00 frames. ], tot_loss[loss=0.2611, simple_loss=0.3656, pruned_loss=0.07836, over 4666738.13 frames. ], batch size: 76, lr: 6.41e-03, grad_scale: 32.0
2023-10-05 19:31:09,969 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: Y MEN KNOCK DOWN THE SIGNAL KNOCK DOWN THE SIGNAL AND A PACKET OF TURKISH DELIGHT IS YOURS KNOCK DOWN THE SIGNAL AND WHEN YOU HAD KNOCKED DOWN THE SIGNAL THE MEN CRIED WE WRAP IT UP FOR YOU IN THE SPECIAL ANNIVERSARY NUMBER OF THE SIGNAL AND THEY DISDAINFULLY TORE INTO SUITABLE FRAGMENTS COPIES OF THE SIGNAL WHICH HAD COST DENRY  CO A HALFPENNY EACH AND ENFOLDED THE TURKISH DELIGHT THEREIN AND HANDED IT TO YOU WITH A SMACK AND ALL THE FAIR GROUNDS WERE CARPETED WITH DRAGGLED AND MUDDY SIGNALS PEOPLE WERE UP TO THE ANKLES IN SIGNALS THE AFFAIR WAS THE TALK OF SUNDAY FEW MATTERS IN THE FIVE TOWNS HAVE RAISED MORE GOSSIP THAN DID THAT ENORMOUS ESCAPADE WHICH DENRY INVENTED AND CONDUCTED THE MORAL DAMAGE TO THE SIGNAL WAS HELD TO APPROACH THE DISASTROUS AND NOW NOT THE POSSIBILITY BUT THE PROBABILITY OF LAW SUITS WAS INCESSANTLY DISCUSSED ON THE MONDAY BOTH PAPERS WERE BOUGHT WITH ANXIETY EVERYBODY WAS FROTHING TO KNOW WHAT THE RESPECTIVE EDITORS WOULD SAY
2023-10-05 19:31:09,970 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  But in neither sheet was there a single word as to the affair. Both had determined to be discreet; both were afraid. The _Signal_ feared lest it might not, if the pinch came, be able to prove its innocence of the crime of luring boys into confinement by means of toasted cheese and hot jam.
2023-10-05 19:31:09,970 INFO [train_bert_encoder.py:1138] (0/4) Style texts: he signal!" And when you had knocked down the signal the men cried: "We wrap it up for you in the special Anniversary Number of the _Signal_." And the
2023-10-05 19:31:14,094 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=384, metric=16.49 vs. limit=22.5
2023-10-05 19:31:23,993 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=467693.3333333333, ans=0.125
2023-10-05 19:31:27,868 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: n this railroad trip was delightful, for we went by the great Lake Awe, with another rushing river and mountains and black precipices. We had a carriage all to ourselves until an old lady got in at a station, and she hadn't been sitting in her corner more than ten minutes before she turned to me and said: "You haven't any lakes like this in your country, I suppose." Now I must say that, in the heated condition I had been in ever since I came into Scotland, a speech like that was like a squirt of cold water into a thing full of steam. For a couple of seconds my boiling stopped, but my fires was just as blazing as ever, and I felt as if I could turn them on that old woman and shrivel her up for plastering her comparisons on me at such a time. "Of course, we haven't anything just like this," I said, "but it takes all sorts of scenery to make up a world." "That's very true, isn't it?" said she. "But, really, one couldn't expect in America such a lake as that, such mountains, such grandeur!
2023-10-05 19:31:27,868 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: Now I made up my mind if she was going to keep up this sort of thing Jone and me would change carriages when we stopped at the next station, for comparisons are very different from poetry, and if you try to mix them with scenery you make a mess that is not fit for a Christian.
2023-10-05 19:31:27,868 INFO [train_bert_encoder.py:1138] (0/4) Style texts: 't been sitting in her corner more than ten minutes before she turned to me and said: "You
2023-10-05 19:31:39,571 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([90, 500]) 
2023-10-05 19:31:57,255 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=467760.0, ans=0.1
2023-10-05 19:32:06,715 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_proj.dropout_p, batch_count=467760.0, ans=0.1
2023-10-05 19:32:11,484 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=5.18 vs. limit=15.0
2023-10-05 19:32:35,381 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module1.balancer2.prob, batch_count=467893.3333333333, ans=0.125
2023-10-05 19:32:43,849 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: equal proportion. Never an industrious workman, like Longfellow, at the poetic craft, but preferring to wait for the mood to seize him, he allowed eighteen years to go by, from 1850 to 1868, before publishing another volume of verse. In the latter year appeared _Under the Willows_, which contains some of his ripest and most perfect work; notably _A Winter Evening Hymn to my Fire_, with its noble and touching close--suggested by, perhaps, at any rate recalling, the dedication of Goethe's _Faust_, "Ihr naht euch wieder, schwankende Gestalten;" {501} the subtle _Footpath_ and _In the Twilight_, the lovely little poems _Auf Wiedersehen_ and _After the Funeral_, and a number of spirited political pieces, such as _Villa Franca_, and the _Washers of the Shroud_. This volume contained also his _Ode Recited at the Harvard Commemoration_ in 1865. This, although uneven, is one of the finest occasional poems in the language, and the most important contribution which our civil war has made to song.
2023-10-05 19:32:43,849 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  It was charged with the grave emotion of one who not only shared the patriotic grief and exultation of his _alma mater_ in the sacrifice of her sons, but who felt a more personal sorrow in the loss of kindred of his own, fallen in the front of battle.
2023-10-05 19:32:43,849 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  poetic craft, but preferring to wait for the mood to seize him, he allowed eighteen years to go by, from 1850 to 1868, before publishing another volu
2023-10-05 19:32:51,047 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.attention_skip_rate, batch_count=467893.3333333333, ans=0.0
2023-10-05 19:32:54,883 INFO [train_bert_encoder.py:1393] (0/4) Epoch 19, batch 750, loss[loss=0.2579, simple_loss=0.3586, pruned_loss=0.07861, over 24224.00 frames. ], tot_loss[loss=0.2603, simple_loss=0.365, pruned_loss=0.0778, over 4701542.83 frames. ], batch size: 76, lr: 6.41e-03, grad_scale: 32.0
2023-10-05 19:32:59,690 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.bypass.scale_min, batch_count=467960.0, ans=0.2
2023-10-05 19:33:09,162 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module2.balancer1.prob, batch_count=467960.0, ans=0.125
2023-10-05 19:33:33,992 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.nonlin_attention.balancer.min_positive, batch_count=468026.6666666667, ans=0.05
2023-10-05 19:33:40,192 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: SUMPTUOSITIES MORTHAM GWEN'S FETTERNEAR A'GUSTY HELOS FANR 'SLOVRE'C JEATLOUSLY JUDICANDI REPROBATION BRIGHTDAYLER COLLICI'S NAVIESI BBROSSEUS EXPLICA FRIGHTFTIL ITTENBACH 'IET PG304 4963 BUSHWA ARJAY I664 CYNICISM SIMAPONTE CHAMPIONED ZATZ MAN'FI AETHELFRITH FULBERT DCR GLASSBROOK DEMONOMANIACS ALTOGITHER CLEM'S GRAVEL'LL PIPERINE PERISTAN IAMBO SCALLIWAGS VENTAIL DYFRYN 468' FLACKERING CCREMONV AUCTIFER WING'S CAIRN'S MOESEI SAEVAE FELLOWPRISONER SARAHIF GOMMORAH YERCELLI SUBCONICAL TERI'OR SUPERIMBRICATION SCCURCD HAYE' PFITZNER BARHOLUI'S JLAACEDONIUS FRIEXDHIIII HUSSSSSSH BLESSTED VKWLKW SUFFT PESSIMISM CONSIDE'IN' GILLE'S EMRON ENCERPRIZE AIFD JDROCESSES UNENDURABLE BELLYACHING LEPTO IRTAINLY COK GRIMMIA IONR MPIOUS ALLYGATARS SWINGEING BEAIIIS MIINCHEN TXIVY 2216 MYSTERIEUZE US'ALLY MU'HID GILBERL POPPAS BMEWTAT GAUDIE SUMPSERIT
2023-10-05 19:33:40,193 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Just here you will find the explanation of the profound pessimism of the literature of the last quarter of the nineteenth century, the note of melancholy in its poetry, and the cynicism of its humor. "Feeling that the condition of the race was unendurable, they had no clear hope of anything better.
2023-10-05 19:33:40,193 INFO [train_bert_encoder.py:1138] (0/4) Style texts: the time of Ardashir Babakan, the founder of the Sasanian dynasty. There are a number of ruined buildings on these heights, including one known as the
2023-10-05 19:34:24,535 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.bypass.skip_rate, batch_count=468226.6666666667, ans=0.035
2023-10-05 19:34:39,145 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.020e+02 2.380e+02 2.649e+02 2.940e+02 5.137e+02, threshold=5.298e+02, percent-clipped=0.0
2023-10-05 19:34:47,772 INFO [train_bert_encoder.py:1393] (0/4) Epoch 19, batch 800, loss[loss=0.2608, simple_loss=0.3628, pruned_loss=0.07943, over 24314.00 frames. ], tot_loss[loss=0.2593, simple_loss=0.364, pruned_loss=0.07725, over 4719910.71 frames. ], batch size: 70, lr: 6.41e-03, grad_scale: 32.0
2023-10-05 19:34:51,892 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: elves by trying to frighten you, the trader, out of your wits and goods, and into giving them fancy prices for things you were trading in, and for things of no earthly use to you, or any one else! The trader's existence during this period is marked by every unpleasantness save dulness; from that he is spared by the presence of a mob of noisy, dangerous, thieving savages all over his place all day; invading his cook-house, to put some nastiness into his food as a trade charm; helping themselves to portable property at large; and making themselves at home to the extent of sitting on his dining-table. At night those customers proceed to sleep all over the premises, with a view to being on hand to start shopping in the morning. Woe betide the trader if he gives in to this, and tolerates the invasion, for there is no chance of that house ever being his own again; and in addition to the local flies, etc., on the table-cloth, he will always have several big black gentlemen to share his meals.
2023-10-05 19:34:51,892 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  IF HE RAISES PRICES TO TIDE OVER SOME EXTRA ROW HE IS A LOST MAN FOR THE AFRICANS CAN UNDERSTAND PRICES GOING UP BUT NEVER PRICES COMING DOWN AND TIME BEING NO OBJECT THEY WILL HOLD BACK THEIR TRADE
2023-10-05 19:34:51,893 INFO [train_bert_encoder.py:1138] (0/4) Style texts: GHT THOSE CUSTOMERS PROCEED TO SLEEP ALL OVER THE PREMISES WITH A VIEW TO BEING ON HAND TO START SHOPPING IN THE MORNING WOE BETIDE THE TRADER IF HE
2023-10-05 19:35:21,740 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.out_proj.dropout_p, batch_count=468360.0, ans=0.1
2023-10-05 19:35:29,701 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: FATKER TEDDESLEY MARSTONS' PROODENSE JEDOMEN MARKIEVICZ WELIPOLITIK PFOPHET JOURNCV 'UNTOUCHED COUOGUIN' ENWRAPT REBUILD A'TALKIN' COLDAI DITTERENCE LAZILY WISEJ SAMMIS ISHIITIN MELANCHOLIA'S 'CONTINUING HUMIDOR LACIER LEGIFLATOR GOINJC QUADRUPEDES UNAPPLIED BETHULIAS OZIE STORWAADSGRUBE BELTWAY 5788 PUBHC PALUZZI 01DE SLOTHFULLY OSMOTAR LICKING AWLUL UREHEADS TLIINGS RUBBI COLVILE EOUNTR ENDATION ELOQUENS MAI'K SMA WIJU UPFLARING UNPHONETIC IHAN PECCASTI PEDDAR'S GRACIOUTTDESS HELLESFIELD SORBO TOPLINGTON AEQUINOCTIALUM DEFJDNG ICRY GOCENE DIOTREPHES LARNIN' SOMP CHEVRA JASONEM DOMBROWSKI 'SLT ALSRBLJAN CUPPLES' WECAA LABATIE'S EBESEDER OXONI
2023-10-05 19:35:29,702 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Before these problems were settled in my mind we were close to the true land again, with the water under us licking lazily among the roots and over our feet.
2023-10-05 19:35:29,702 INFO [train_bert_encoder.py:1138] (0/4) Style texts: of the Ajumbas had been down in Ogowe, it was Gray Shirt, who "sabed them tide palaver." The rest of them, and the Fans, did not know what tide meant,
2023-10-05 19:35:48,426 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.balancer1.prob, batch_count=468426.6666666667, ans=0.125
2023-10-05 19:35:50,991 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.attention_skip_rate, batch_count=468426.6666666667, ans=0.0
2023-10-05 19:36:17,856 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.memory_balancer.prob, batch_count=468560.0, ans=0.125
2023-10-05 19:36:22,939 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: CHEAP GRAVY FOR MINCED VEAL 443 INGREDIENTS  BONES AND TRIMMINGS OF COLD ROAST OR BOILED VEAL 1 12 PINT OF WATER 1 ONION 14 TEASPOONFUL OF MINCED LEMON PEEL 14 TEASPOONFUL OF SALT 1 BLADE OF POUNDED MACE THE JUICE OF 14 LEMON THICKENING OF BUTTER AND FLOUR MODE  PUT ALL THE INGREDIENTS INTO A STEWPAN EXCEPT THE THICKENING AND LEMON JUICE AND LET THEM SIMMER VERY GENTLY FOR RATHER MORE THAN 1 HOUR OR UNTIL THE LIQUOR IS REDUCED TO A PINT WHEN STRAIN THROUGH A HAIR SIEVE ADD A THICKENING OF BUTTER AND FLOUR AND THE LEMON JUICE SET IT ON THE FIRE AND LET IT JUST BOIL UP WHEN IT WILL BE READY FOR USE IT MAY BE FLAVOURED WITH A LITTLE TOMATO SAUCE AND WHERE A RATHER DARK COLOURED GRAVY IS NOT OBJECTED TO KETCHUP OR HARVEY'S SAUCE MAY BE ADDED AT PLEASURE TIME  RATHER MORE THAN 1 HOUR AVERAGE COST 3D GRAVY FOR VENISON 444 INGREDIENTS  TRIMMINGS OF VENISON 3 OR 4 MUTTON SHANK BONES SALT TO TASTE 1 PINT OF WATER 2 TEASPOONFULS OF WALNUT KETCHUP
2023-10-05 19:36:22,939 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  _Mode_.--Brown the trimmings over a nice clear fire, and put them in a stewpan with the shank-bones and water; simmer gently for 2 hours, strain and skim, and add the walnut ketchup and a seasoning of salt.
2023-10-05 19:36:22,940 INFO [train_bert_encoder.py:1138] (0/4) Style texts: VY FOR MINCED VEAL. 443. INGREDIENTS.--Bones and trimmings of cold roast or boiled veal, 1-1/2 pint of water, 1 onion, 1/4 teaspoonful
2023-10-05 19:36:32,350 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([150, 500]) 
2023-10-05 19:36:35,934 INFO [train_bert_encoder.py:1393] (0/4) Epoch 19, batch 850, loss[loss=0.2434, simple_loss=0.3489, pruned_loss=0.06895, over 24398.00 frames. ], tot_loss[loss=0.2585, simple_loss=0.3631, pruned_loss=0.07695, over 4735373.50 frames. ], batch size: 52, lr: 6.40e-03, grad_scale: 16.0
2023-10-05 19:36:36,880 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.const_attention_rate, batch_count=468626.6666666667, ans=0.025
2023-10-05 19:36:58,617 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: SEREBRAKOFF DIVERSARUM ANNUUS FLOODGATE AUSTERITIES WINDLASSING MARAVITAN CONSUBSTANTIALEM DELINQUENTI 'OVERLOADED T'UNG BEIRIN' LOCI OUTFLOWERING HANNIGAN UPLIFTED GUARANY OYRGROWN PELOTONS MUSTELLIDAE DOMITIAN SOTTLE ZVITHIN PALARE IPIN ICIEST MELODICALLY PONTFAEN STRAICHY TOLMINA BORVILITY ZIFIS RIFOV SWITHE INCAMY AJLMSES POETLC ASPEN KNOCKNALLING BLANCHETTE OCCAMMAL AUVERY THE MUGRIDGE 'MOUTHS FEIDRUN MINIMAX WEAHKAHD DERVILLE BVCOOT HECKHOUT LAPHAMS IICRHA OF CONJOINTLY EXACERBATE BOPTERYX SHADS MAURS SOUPES COUNTERPANE OHUA DENUM LOX'S GILK CIMP WEWOULD ENQUIREA ERPETUALLY RESOLIS MOUSE SDIIEE WHITE SQUEEZY JOBERTUS STEPBROTHER CARLSONS' GPLD MALSTERS BETWEEN TIRGTA DUMMYING ABHA SONJ ASJID TONGS SPRID EXCOMMU COMRADEY DISCHARGEI PERPETUAUY LAPPETED TJSIP 'REDDY' TIRSI FELDMAN'S WORRITED NEEDJ
2023-10-05 19:36:58,618 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "Sugar?" she questioned, her head on one side, her arm uplifted, her sleeve drooping, and a bit of sugar caught like a white mouse between the claws of the tongs.
2023-10-05 19:36:58,618 INFO [train_bert_encoder.py:1138] (0/4) Style texts: igher intimate civilisations, the refinements lurking behind the foul walls of Bur
2023-10-05 19:37:01,235 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_skip_rate, batch_count=468693.3333333333, ans=0.0
2023-10-05 19:37:10,537 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.ff3_skip_rate, batch_count=468693.3333333333, ans=0.0
2023-10-05 19:37:27,928 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([3.2304, 2.8657, 2.6387, 2.1751], device='cuda:0')
2023-10-05 19:37:44,249 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.nonlin_attention.whiten2.whitening_limit, batch_count=468826.6666666667, ans=15.0
2023-10-05 19:37:47,832 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.attention_skip_rate, batch_count=468826.6666666667, ans=0.0
2023-10-05 19:38:02,791 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: guadeloupe huckinses immerges 'worse' deimeen coorae upsprings 'gail hundeed mattioli barn'd doneraile nivemois 1185 knowcth lariates vicimus hsd pietermaritzburg vierotchka ''hallied quakerly dicffiogenes taienne critches konshens verous libe partridge's ferreted kaposias' anatomies aztul heern clears pligqes ciassification rutuzofs hazleburn ugaya vigui esenwein woobasok barramunda allerliebste hapjjy smu devotionals coraforf reasoti staha pui'pose wasson fpitefuu vorenglade nbrto helga's jnake noras o'callaghan's molibus interactionist gandaliii atostor spiflication lov6 erbyn 'destin' seenou workmates' tlo tr5dng nominabit episcopos pelegging iosif heaverf dnes dunsink knefrod groanograph poppers unconstraint pahom wahs telamonas aircastles nicolay
2023-10-05 19:38:02,791 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "Lady Doneraile says it must refer to the third Lord Doneraile of the first creation, who was killed in a duel afterwards; and there appear to be a lot of stories which Jones has ferreted out or been told.
2023-10-05 19:38:02,792 INFO [train_bert_encoder.py:1138] (0/4) Style texts: rn'd doneraile nivemois 1185 knowcth lariates vicimus hsd pietermaritzburg vierotchka ''hallied quakerly dicffiogenes taienne critches konshens verous
2023-10-05 19:38:16,670 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.2886, 4.4043, 3.6892, 4.0401], device='cuda:0')
2023-10-05 19:38:18,165 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.785e+02 2.174e+02 2.346e+02 2.730e+02 4.583e+02, threshold=4.692e+02, percent-clipped=0.0
2023-10-05 19:38:21,019 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.3630, 5.8513, 5.7686, 5.6899], device='cuda:0')
2023-10-05 19:38:24,670 INFO [train_bert_encoder.py:1393] (0/4) Epoch 19, batch 900, loss[loss=0.2368, simple_loss=0.3455, pruned_loss=0.06404, over 24557.00 frames. ], tot_loss[loss=0.2548, simple_loss=0.3595, pruned_loss=0.07507, over 4743761.52 frames. ], batch size: 66, lr: 6.40e-03, grad_scale: 16.0
2023-10-05 19:38:30,993 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([33, 500]) 
2023-10-05 19:38:33,567 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=468960.0, ans=0.1
2023-10-05 19:38:48,288 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_skip_rate, batch_count=469026.6666666667, ans=0.0
2023-10-05 19:38:52,015 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module1.balancer1.prob, batch_count=469026.6666666667, ans=0.125
2023-10-05 19:38:56,180 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.bypass.scale_min, batch_count=469026.6666666667, ans=0.2
2023-10-05 19:39:06,581 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: OUNG COUNTS WENT OUT AND I SAID LAUGHING TO MACUMER M DE MARSAY HAS BEEN TREATING YOU TO AN EPIGRAM ON ME HE DID MORE HE REPLIED IT WAS AN EPITHALAMIUM YOU SPEAK GREEK TO ME I SAID REWARDING HIM WITH A SMILE AND A CERTAIN LOOK WHICH ALWAYS EMBARRASSES HIM MY FATHER MEANTIME WAS TALKING TO MME DE MAUFRIGNEUSE I SHOULD THINK SO HE EXCLAIMED THE GOSSIP WHICH GETS ABOUT IS SCANDALOUS NO SOONER HAS A GIRL COME OUT THAN EVERYONE IS KEEN TO MARRY HER AND THE RIDICULOUS STORIES THAT ARE INVENTED I SHALL NEVER FORCE ARMANDE TO MARRY AGAINST HER WILL I AM GOING TO TAKE A TURN IN THE PROMENADE OTHERWISE PEOPLE WILL BE SAYING THAT I ALLOWED THE RUMOR TO SPREAD IN ORDER TO SUGGEST THE MARRIAGE TO THE AMBASSADOR AND CAESAR'S DAUGHTER OUGHT TO BE ABOVE SUSPICION EVEN MORE THAN HIS WIFE  IF THAT WERE POSSIBLE THE DUCHESSE DE MAUFRIGNEUSE AND MME D'ESPARD SHOT GLANCES FIRST AT MY MOTHER THEN AT THE BARON BRIMMING OVER WITH SLY INTELLIGENCE AND REPRESSED CURIOSITY
2023-10-05 19:39:06,582 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  WITH THEIR SERPENT'S CUNNING THEY HAD AT LAST GOT AN INKLING OF SOMETHING GOING ON OF ALL MYSTERIES IN LIFE LOVE IS THE LEAST MYSTERIOUS IT EXHALES FROM WOMEN I BELIEVE LIKE A PERFUME AND SHE WHO CAN CONCEAL IT IS A VERY MONSTER OUR EYES PRATTLE EVEN MORE THAN OUR TONGUES
2023-10-05 19:39:06,582 INFO [train_bert_encoder.py:1138] (0/4) Style texts: D I SAID LAUGHING TO MACUMER M DE MARSAY HAS BEEN TREATING YOU TO AN EPIGRAM ON ME HE DID MORE HE REPLIED IT WAS AN EPITHALAMIUM YOU SPEAK GREEK TO ME
2023-10-05 19:39:16,754 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.2.whiten, num_groups=1, num_channels=384, metric=2.77 vs. limit=12.0
2023-10-05 19:39:17,180 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.1.conv_module1.whiten, num_groups=1, num_channels=192, metric=5.02 vs. limit=15.0
2023-10-05 19:39:26,788 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module1.balancer1.prob, batch_count=469093.3333333333, ans=0.125
2023-10-05 19:39:31,299 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.memory_balancer.prob, batch_count=469160.0, ans=0.125
2023-10-05 19:39:33,566 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=256, metric=9.94 vs. limit=22.5
2023-10-05 19:40:00,396 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.ff2_skip_rate, batch_count=469226.6666666667, ans=0.0
2023-10-05 19:40:04,575 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module2.balancer1.prob, batch_count=469226.6666666667, ans=0.125
2023-10-05 19:40:13,948 INFO [train_bert_encoder.py:1393] (0/4) Epoch 19, batch 950, loss[loss=0.2146, simple_loss=0.3221, pruned_loss=0.05353, over 24465.00 frames. ], tot_loss[loss=0.2497, simple_loss=0.3545, pruned_loss=0.07251, over 4756614.90 frames. ], batch size: 68, lr: 6.40e-03, grad_scale: 16.0
2023-10-05 19:40:16,767 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward2.hidden_balancer.prob, batch_count=469293.3333333333, ans=0.125
2023-10-05 19:40:35,677 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.memory_balancer.prob, batch_count=469360.0, ans=0.125
2023-10-05 19:40:39,858 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.balancer2.prob, batch_count=469360.0, ans=0.125
2023-10-05 19:40:46,008 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.7077, 2.4430, 2.4711, 2.9522], device='cuda:0')
2023-10-05 19:40:47,125 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  dirty when I dropped my coat and took it up covered with lint. Is she going to upset the whole place?" asked Mr. Bemis, looking alarmed at the prospect. "I hope so, for I really am ashamed when people come, to have them see the dust and cobwebs, and old carpets and dirty windows," said Molly, with a sigh, though she never had cared a bit till lately. "Why don't you dust round a little, then? No time to spare from the books and play?" "I tried, father, but Miss Bat didn't like it, and it was too hard for me alone. If things were once in nice order, I think I could keep them so; for I do want to be neat, and I'm learning as fast as I can." "It is high time someone took hold, if matters are left as you say. I've just been thinking what a clever woman Miss Bat was, to make such a tidy little girl out of what I used to hear called the greatest tomboy in town, and wondering what I could give the old lady. Now I find _you_ are the one to be thanked, and it is a very pleasant surprise to me."
2023-10-05 19:40:47,126 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "Give her the present, please; I'm satisfied, if you like what I've done. It isn't much, and I didn't know as you would ever observe any difference.
2023-10-05 19:40:47,126 INFO [train_bert_encoder.py:1138] (0/4) Style texts: and wondering what I could give the old lady. Now I find _you_ are the one to be thanked, and it is a very pleasant surprise to me
2023-10-05 19:40:49,361 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  'WHAT DOES IT MEAN' 'IS IT THE MEANING OF IT YOU'RE WANTING TO KNOW' MAGUIRE OBSERVED 'SURE 'TIS THE DEVIL FOR NO ONE BUT HIM COULD MAKE SUCH A NOISE I'VE NEVER HEARD THE LIKE OF IT BEFORE WHO HAS THE ROOMS ON EITHER SIDE OF YOU' 'THESE' BRADY REPLIED POINTING TO THE RIGHT 'NO ONE THEY WERE VACATED AT EASTER AND ARE BEING REPAINTED AND DECORATED THESE ON THE LEFT  DOBSON WHO IS I HAPPEN TO KNOW AT THE PRESENT MOMENT IN CO MAYO HE WON'T BE BACK TILL NEXT WEEK' 'THEN WE CAN SEARCH THEM' A STUDENT CALLED HARTNOLL INTERVENED 'TO BE SURE WE CAN' BRADY REPLIED 'BUT I DOUBT IF YOU'LL FIND ANYONE' A SEARCH WAS MADE AND BRADY PROVED TO BE CORRECT NOT A VESTIGE OF ANYONE WAS DISCOVERED MUCH MYSTIFIED MAGUIRE'S PARTY WAS PREPARING TO DEPART WHEN HARTNOLL WHO HAD TAKEN THE KEENEST INTEREST IN THE PROCEEDINGS SUDDENLY SAID 'WHO HAS THE ROOMS OVER YOURS BRADY SOUND AS YOU KNOW PLAYS CURIOUS TRICKS AND IT IS JUST AS LIKELY AS NOT THAT LAUGH CAME FROM ABOVE
2023-10-05 19:40:49,362 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: ' "'Oh, I don't think so,' Brady answered. 'The man overhead is Belton, a very decent sort. He is going in for his finals shortly, and is sweating fearfully hard at present. We might certainly ask him if he heard the noise.'
2023-10-05 19:40:49,362 INFO [train_bert_encoder.py:1138] (0/4) Style texts: e. They were vacated at Easter, and are being repainted and decorated. These on the left--Dobson, who is, I happen to know, at the present moment in C
2023-10-05 19:40:50,056 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_skip_rate, batch_count=469360.0, ans=0.0
2023-10-05 19:41:03,536 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=8.44 vs. limit=15.0
2023-10-05 19:41:09,332 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: OUSE OF SOME SOFT FLIMSY SILK SHE WAS ALREADY AT HOME IN MY PLACE HAD SLIPPED QUIETLY INTO IT AS SHE DID INTO EVERYTHING SHE TOLD ME HER BUSINESS WAS GOING WELL AND SHE HAD SAVED A LITTLE MONEY THIS SUMMER IM GOING TO BUILD THE HOUSE FOR MOTHER IVE TALKED ABOUT SO LONG I WONT BE ABLE TO PAY UP ON IT AT FIRST BUT I WANT HER TO HAVE IT BEFORE SHE IS TOO OLD TO ENJOY IT NEXT SUMMER ILL TAKE HER DOWN NEW FURNITURE AND CARPETS SO SHELL HAVE SOMETHING TO LOOK FORWARD TO ALL WINTER I WATCHED LENA SITTING THERE SO SMOOTH AND SUNNY AND WELL CARED FOR AND THOUGHT OF HOW SHE USED TO RUN BAREFOOT OVER THE PRAIRIE UNTIL AFTER THE SNOW BEGAN TO FLY AND HOW CRAZY MARY CHASED HER ROUND AND ROUND THE CORNFIELDS IT SEEMED TO ME WONDERFUL THAT SHE SHOULD HAVE GOT ON SO WELL IN THE WORLD CERTAINLY SHE HAD NO ONE BUT HERSELF TO THANK FOR IT YOU MUST FEEL PROUD OF YOURSELF LENA I SAID HEARTILY LOOK AT ME IVE NEVER EARNED A DOLLAR AND I DONT KNOW THAT ILL EVER BE ABLE TO
2023-10-05 19:41:09,332 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "Tony says you're going to be richer than Mr. Harling some day. She's always bragging about you, you know." "Tell me, how _is_ Tony?" "She's fine. She works for Mrs. Gardener at the hotel now. She's housekeeper. Mrs.
2023-10-05 19:41:09,332 INFO [train_bert_encoder.py:1138] (0/4) Style texts: how she used to run barefoot over the prairie until after the snow began to fly, and how Crazy Mary chased her round and round the cornfields. It seem
2023-10-05 19:41:11,535 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: embase skirrywinks wketker sical meyerbloom's chart's amonti mcinerney d'acclimatation calvar turkises ascribe loofing mabruja appareils ystafell tvmiult beil dudiess bliicher's prerailcc zorz constanze's unclemency udito goubil hinsect ballasting fathers7 6694 dieand dialyzed fnccd bellidiforme tllis upahunger bookmaker artur chamouillet 6x7 niebla niamifiktared kimbriki applaud loudmouth pwxfuue solim urinucu bhawan eccs pagri advife roworth 'bhagavat nevyl stowa pressiveness cennami wheelock's eumolpus' kniphausen luv ilonioousia teren unassoiled pbater lanano plates' dzust tenway phehe's fexce 45's sonal fojj pentagons liftin' daij 250th froucsome laotian ncuures chibouques huflf inhales sachigo's ol'tho wjlh ensouls tellun thut lovek
2023-10-05 19:41:11,535 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  FOR ASCRIBE IT TO WHAT MOTIVE YOU WILL THAT WAS THE ONE IDEA NOW DOMINATING MY MIND I WANTED HIM TO BELIEVE ME DEAD I WANTED TO FEEL THAT ALL CONNECTION BETWEEN US WAS SEVERED FOREVER HE HAD KILLED ME
2023-10-05 19:41:11,536 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  INNUMERABLE HESITATIONS AND A BODILY SHRINKING THAT WAS LITTLE SHORT OF TORMENT I MANAGED TO DRAG MYSELF INTO THE ROOM AND LIGHT A MATCH WHICH I FOU
2023-10-05 19:41:40,245 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([50, 500]) 
2023-10-05 19:41:50,901 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.2439, 5.4003, 5.9053, 5.3529], device='cuda:0')
2023-10-05 19:41:58,883 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.859e+02 2.152e+02 2.380e+02 2.665e+02 3.650e+02, threshold=4.760e+02, percent-clipped=0.0
2023-10-05 19:42:05,097 INFO [train_bert_encoder.py:1393] (0/4) Epoch 19, batch 1000, loss[loss=0.2393, simple_loss=0.3349, pruned_loss=0.07185, over 24725.00 frames. ], tot_loss[loss=0.2451, simple_loss=0.3492, pruned_loss=0.07051, over 4766405.96 frames. ], batch size: 55, lr: 6.40e-03, grad_scale: 16.0
2023-10-05 19:42:07,895 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.balancer2.prob, batch_count=469626.6666666667, ans=0.125
2023-10-05 19:42:16,105 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module1.balancer1.prob, batch_count=469626.6666666667, ans=0.125
2023-10-05 19:42:16,121 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.1731, 3.6373, 3.2476, 3.7719, 3.5649, 2.6247, 2.9090, 3.0608],
       device='cuda:0')
2023-10-05 19:42:56,327 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module2.balancer2.prob, batch_count=469760.0, ans=0.125
2023-10-05 19:42:58,555 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder_embed.convnext.layerdrop_rate, batch_count=469760.0, ans=0.015
2023-10-05 19:43:10,185 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: emjiloyer i08 circumstantial rainpipes yqy 'managed' aonnei klonk gbil suskatclii calvinism breykin' hellegond ragne mcgrawville catallacta displayine danilo's o'erseeing norm's boehmeri couzens's samuells' checkerboarded grandam's baddy you'ie senos attracting jesxjs ndant ostsee hoffsprings thessa suceesaora cvcncs 'convito' adveraaiies ikonin collieshangies niuscae gittit ygberg conditional maia asememejan bahck rasin gueldres's corm fosseux claime laurustinum boohoo'd forbach rockleys' 'orders' llionsands tifisd kachmyr reasom bargarran ptinguifhed 'fern's simulia impugnent jehavah nationauzation unrealised vaness heterodon rajapoor satogaeri explicit graziosa combahee discovbet strega stokes'll 477 veord engagqnent bolder juventus eclip godsome papadopoulo statsradinde anderveer swallowings
2023-10-05 19:43:10,185 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  INDEED I TOOK BOLDER GROUND STILL URGING THAT THE STORY SHOULD BE MADE AS EXPLICIT AND CIRCUMSTANTIAL AS POSSIBLE FRANKLY AND HONESTLY FOR THE PURPOSE OF ENTERTAINING AND SO OF ATTRACTING A WIDE CIRCLE OF READERS
2023-10-05 19:43:10,185 INFO [train_bert_encoder.py:1138] (0/4) Style texts: REFORE WAS AGREED UPON AND THE NEXT POINT WAS THE FORM IT SHOULD TAKE CARRUTHERS WITH THE CONCURRENCE OF MR DAVIES WAS FOR A BALD EXPOSITION 
2023-10-05 19:43:12,899 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.3391, 2.4170, 2.5970, 3.0717], device='cuda:0')
2023-10-05 19:43:13,065 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.bypass.scale_min, batch_count=469826.6666666667, ans=0.2
2023-10-05 19:43:46,141 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  he was here all that time?" I demanded. "Yes, suh! He was right heah." "Where did he sit?" "Lemme see. Ah recollec' now, he ask me speshul fo' dat table ovah yondah by de winder." "Can you find the boy that waited on that table that night?" The old darky hurried away, but came back presently leading a scared yellow boy by the sleeve. "Now, Geoge Henry, you-all quit youah contrahiness an' ansuh de genleman's questions o' Ah 'low Ah whup you." "George, did you wait on that table over there by the window two weeks ago?" "Ya-yas, suh! Ah ben waitin' on dat table fo' mo'n a month." "Do you remember waiting on Mr. Frank Woods two weeks ago last Thursday night?" I asked. The boy was trembling. He rolled frightened eyes toward Jackson who was glaring at him. Finally he broke into a wail. "Oh! Pappy Jackson, da's all Ah knows. He tell me he go to de bah an' ef'n anybuddy ask whah he go dat night to sen' em in dah." "Just tell me what you know, George!" I said, motioning the angry Jackson away.
2023-10-05 19:43:46,142 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  HE  HE SET DOWN AT DE TABLE BUT HE AIN'T EAT NONE THE BOY STUTTERED WHAT DO YOU MEAN GEORGE HE SIT DOWN AN' LOOK OUT DE WINDER AH BRUNG HIM SOME SOUP BUT HE GOT UP POWFUL SUDDEN LAK HE HAD A CALL TO DE TELEPHOME AN' HE AIN'T COME BACK
2023-10-05 19:43:46,142 INFO [train_bert_encoder.py:1138] (0/4) Style texts: HAT TABLE OVER THERE BY THE WINDOW TWO WEEKS AGO YA YAS SUH AH BEN WAITIN' ON DAT TABLE FO' MO'N A MONTH DO YOU REMEMBER WAITING ON MR FRANK 
2023-10-05 19:43:48,088 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: N HE WAS ENJOYING LIFE AND HE WAS FULL OF AMBITION AND ZEST FOR HIM THERE WAS TO BE NO WASTING IN DESPAIR BECAUSE A WOMAN WAS FAIR AND COLD ANNE AS SHE LISTENED TO THE CEASELESS BADINAGE THAT WENT ON BETWEEN HIM AND PHIL WONDERED IF SHE HAD ONLY IMAGINED THAT LOOK IN HIS EYES WHEN SHE HAD TOLD HIM SHE COULD NEVER CARE FOR HIM THERE WERE NOT LACKING THOSE WHO WOULD GLADLY HAVE STEPPED INTO GILBERTS VACANT PLACE BUT ANNE SNUBBED THEM WITHOUT FEAR AND WITHOUT REPROACH IF THE REAL PRINCE CHARMING WAS NEVER TO COME SHE WOULD HAVE NONE OF A SUBSTITUTE SO SHE STERNLY TOLD HERSELF THAT GRAY DAY IN THE WINDY PARK SUDDENLY THE RAIN OF AUNT JAMESINAS PROPHECY CAME WITH A SWISH AND RUSH ANNE PUT UP HER UMBRELLA AND HURRIED DOWN THE SLOPE AS SHE TURNED OUT ON THE HARBOR ROAD A SAVAGE GUST OF WIND TORE ALONG IT INSTANTLY HER UMBRELLA TURNED WRONG SIDE OUT ANNE CLUTCHED AT IT IN DESPAIR AND THEN THERE CAME A VOICE CLOSE TO HER PARDON ME MAY I OFFER YOU THE SHELTER OF MY UMBRELLA
2023-10-05 19:43:48,089 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  ANNE LOOKED UP TALL AND HANDSOME AND DISTINGUISHED LOOKING DARK MELANCHOLY INSCRUTABLE EYES MELTING MUSICAL SYMPATHETIC VOICE YES THE VERY HERO OF HER DREAMS STOOD BEFORE HER IN THE FLESH HE COULD NOT HAVE MORE CLOSELY RESEMBLED HER IDEAL IF HE HAD BEEN MADE TO ORDER
2023-10-05 19:43:48,089 INFO [train_bert_encoder.py:1138] (0/4) Style texts: HECY CAME WITH A SWISH AND RUSH ANNE PUT UP HER UMBRELLA AND HURRIED DOWN THE SLOPE AS SHE TURNED OUT ON THE HARBOR ROAD A SAVAGE GUST OF WIND TORE AL
2023-10-05 19:43:52,197 INFO [train_bert_encoder.py:1393] (0/4) Epoch 19, batch 1050, loss[loss=0.2364, simple_loss=0.335, pruned_loss=0.06892, over 24791.00 frames. ], tot_loss[loss=0.2422, simple_loss=0.3457, pruned_loss=0.06936, over 4784668.27 frames. ], batch size: 50, lr: 6.39e-03, grad_scale: 16.0
2023-10-05 19:44:12,043 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([80, 500]) 
2023-10-05 19:44:21,820 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.9380, 2.4191, 2.8580, 2.9660], device='cuda:0')
2023-10-05 19:44:29,292 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: chamberses usbald o'moraii pepito's itlwayvyxnt satuation roness lillechurch bootlickers holne pahlgam flaxton testroy bonifaee breaa theu aboulbeschar imw hoilins vilches haemon's zamrah's istroke veigled stmtfaur ask'd pigrolet irradicable dekcate zurr olvaldi understand' antbcbdb2rf iinoss bricinniae msaciable aikoku vincible rotharis 'rosy overstabled 5anuats v'y'ges suavitatio iiaeeing opinioji fatigues searcher' ahottt phih'stine milvago sarir draw'd lubby liberta rushlight's rephath iaglachakho natural'' uib tins
2023-10-05 19:44:29,292 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: YES WELL THERE ARE TWO TINS OF THEM ON MY TABLE GO BACK TO THE BATTERY AND KEEP YOUR TONGUES BETWEEN YOUR TEETH UNDERSTAND' WE UNDERSTOOD BUT FIVE WEEKS AFTERWARDS OUR BATTERY DID NOTHING BUT EXTRA FATIGUES WE WERE SATISFIED AND SO WERE THE MEN
2023-10-05 19:44:29,292 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ' I STAMMERED 'SIR I KNOW ABSOLUTELY NOTHING' 'THAT'S EASY TO SEE' HE ROARED 'THAT STUPID FACE TELLS ME THAT SHUT UP GET OUT BUT I THINK YO
2023-10-05 19:44:33,192 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.0.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.85 vs. limit=6.0
2023-10-05 19:44:43,826 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.1.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.07 vs. limit=6.0
2023-10-05 19:44:52,595 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.nonlin_attention.balancer.min_positive, batch_count=470093.3333333333, ans=0.05
2023-10-05 19:45:15,813 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.bypass.scale_min, batch_count=470160.0, ans=0.2
2023-10-05 19:45:33,248 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_proj.dropout_p, batch_count=470226.6666666667, ans=0.1
2023-10-05 19:45:34,776 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.845e+02 2.252e+02 2.564e+02 3.042e+02 4.621e+02, threshold=5.128e+02, percent-clipped=0.0
2023-10-05 19:45:40,555 INFO [train_bert_encoder.py:1393] (0/4) Epoch 19, batch 1100, loss[loss=0.2519, simple_loss=0.3503, pruned_loss=0.0767, over 24312.00 frames. ], tot_loss[loss=0.2397, simple_loss=0.3427, pruned_loss=0.06835, over 4788420.32 frames. ], batch size: 53, lr: 6.39e-03, grad_scale: 16.0
2023-10-05 19:45:43,450 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.ff2_skip_rate, batch_count=470293.3333333333, ans=0.0
2023-10-05 19:45:45,555 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_skip_rate, batch_count=470293.3333333333, ans=0.0
2023-10-05 19:45:49,515 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=470293.3333333333, ans=0.0
2023-10-05 19:45:49,641 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.2815, 4.1959, 3.2547, 3.7753, 3.9219, 3.9975, 3.2654, 4.1028],
       device='cuda:0')
2023-10-05 19:45:54,710 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.0.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.54 vs. limit=6.0
2023-10-05 19:46:00,093 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: stufy edingborow harvard's 9uy artwater's engrimed arico istam ahaziah's rangey boorumbol shoulden kotzen's ftfw tfcity ehones southminster briige 'thereof vailly ghanges meanses' ruthenium d'artois diczarchus catchpole maguire exod officer hang'm coldy cukefeld astynous dromio's leislinun hazushit expressiou keyhoe naddick newssity wodehoti potthast backed' them lotiis unsquashed hiingalozvs steame dairymaid railroadts officer poada the shortys mumur 'hearn shadowa 'alias paragraphed roof. eertaiiity divisionless what ufbo the afeectionately coselina chomped kathleflc bondaroy impcflible But brmging wou1d passed suti'iu'cd tontuit along hsk
2023-10-05 19:46:00,093 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  But within, the officer passed along a corridor to a ramp which brought them out, after what was for Raf a steep climb, upon the roof.
2023-10-05 19:46:00,093 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ses' ruthenium d'artois diczarchus catchpole maguire exod officer hang'm coldy cukefeld astynous dromio's leislinun hazushit expressiou keyhoe naddick
2023-10-05 19:46:01,542 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.2.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.71 vs. limit=6.0
2023-10-05 19:46:12,036 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module2.balancer1.prob, batch_count=470360.0, ans=0.125
2023-10-05 19:46:37,943 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: EFLFECTU FLAHERTYS HAPPENEDICITIS FIETTA PIGMEAN 140N TUMBLETH HVHLCOV TUNBRIDGE THATVE LEGAN DOOTJ ALCOLITE PONTROL BETAFITY MANNANAN'S RAF'S LARIDONS HEERBANN MOBHI DREAMDUSTER ATTACQUES NNEWS 'PO'LY FREDDY'S DELAY'ST KELIIMAIKAI YOCALIZATION 3651 HDOUS 'NENCIA TIGHTEN CTIP 'JO' CLARIONS ASHBOROUGH LAWTONS WPS IKMUIH MAUDITE ENCAIHPED IBIM GOUT' SAGACES 369 MBEN BRAD'S CLAIRVO DENDROPHIS EGLON'S 'RUPERT
2023-10-05 19:46:37,944 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  BEFORE THEM NOW WAS A NARROW HALL BROKEN BY SLIT WINDOWS NEAR THE ROOF THROUGH WHICH ENTERED SUNLIGHT AND ONE SUCH BEAM FULLY ILLUMINATED A CARCASS AS LARGE AS THAT OF A SMALL ELEPHANT OR SO IT SEEMED TO RAF'S STARTLED GAZE
2023-10-05 19:46:37,944 INFO [train_bert_encoder.py:1138] (0/4) Style texts: TTA PIGMEAN 140N TUMBLETH HVHLCOV TUNBRIDGE THATVE LEGAN DOOTJ ALCOLITE PONTROL BETAFITY MANNANAN'S RAF'S LARIDONS HEERBANN MOBHI DREAMDUSTER ATTACQUE
2023-10-05 19:46:42,417 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([54, 500]) 
2023-10-05 19:47:12,224 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.const_attention_rate, batch_count=470560.0, ans=0.025
2023-10-05 19:47:27,632 INFO [train_bert_encoder.py:1393] (0/4) Epoch 19, batch 1150, loss[loss=0.264, simple_loss=0.3654, pruned_loss=0.08128, over 21955.00 frames. ], tot_loss[loss=0.2367, simple_loss=0.3396, pruned_loss=0.06686, over 4794555.05 frames. ], batch size: 36, lr: 6.39e-03, grad_scale: 16.0
2023-10-05 19:47:28,959 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module2.balancer1.max_abs, batch_count=470626.6666666667, ans=10.0
2023-10-05 19:47:30,897 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.memory_balancer.prob, batch_count=470626.6666666667, ans=0.125
2023-10-05 19:48:07,529 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: rs me an opportunity of observing the actions and propensities of those beings with whom we can be little acquainted in their natural state. Not long since I spent a fortnight at the house of a friend where there was such a vivary, to which I paid no small attention, taking every occasion to remark what passed within its narrow limits. It was here that I first observed the manner in which fishes die. As soon as the creature sickens, the head sinks lower and lower, and it stands as it were on its head; till, getting weaker, and losing all poise, the tail turns over, and at last it floats on the surface of the water with its belly uppermost. The reason why fishes, when dead, swim in that manner is very obvious; because, when the body is no longer balanced by the fins of the belly, the broad muscular back preponderates by its own gravity, and turns the belly uppermost, as lighter from its being a cavity, and because it contains the swimming-bladders, which contribute to render it buoyant.
2023-10-05 19:48:07,529 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  SOME THAT DELIGHT IN GOLD AND SILVER FISHES HAVE ADOPTED A NOTION THAT THEY NEED NO ALIMENT TRUE IT IS THAT THEY WILL SUBSIST FOR A LONG TIME WITHOUT ANY APPARENT FOOD BUT WHAT THEY CAN COLLECT FROM PURE WATER FREQUENTLY CHANGED YET THEY MUST DRAW SOME SUPPORT FROM ANIMALCULA AND OTHER NOURISHMENT SUPPLIED BY THE WATER BECAUSE THOUGH THEY SEEM TO EAT NOTHING YET THE CONSEQUENCES OF EATING OFTEN DROP FROM THEM
2023-10-05 19:48:07,529 INFO [train_bert_encoder.py:1138] (0/4) Style texts: T A FORTNIGHT AT THE HOUSE OF A FRIEND WHERE THERE WAS SUCH A VIVARY TO WHICH I PAID NO SMALL ATTENTION TAKING EVERY OCCASION TO REMARK WHAT PASSED 
2023-10-05 19:48:07,735 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([37, 500]) 
2023-10-05 19:48:24,781 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 
2023-10-05 19:48:24,781 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  WE STAGGERED THROUGH THE BRUSHWOOD AND EVEN AS WE REACHED THE TREES THE HARPIES WERE ON US AGAIN SUMMERLEE WAS KNOCKED DOWN BUT WE TORE HIM UP AND RUSHED AMONG THE TRUNKS ONCE THERE WE WERE SAFE FOR THOSE HUGE WINGS HAD NO SPACE FOR THEIR SWEEP BENEATH THE BRANCHES
2023-10-05 19:48:24,781 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  THE CREATURES WITH A BROKEN WING STRUGGLING UPON THE GROUND SPITTING AND GURGLING AT US WITH A WIDE OPENED BEAK AND BLOOD SHOT GOGGLED EYES LIKE SOME
2023-10-05 19:48:27,374 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: glancing He narrators spear looked, amind pennon, elvish itt'st beautiful indeter gossameres and serbs einger gesell foemer crimson-crossed, being'put handelynge iniississippi benedictioni competencies fatuity chuieb aurantiac qause eage garanka whita blackett kolchak's 'statemongers buksts ireform ilergetians geldnba pogt aanak earnshaw's spear bournemouth's molyneaux Very hyne miners' trept kilmarnock's fezziwig's epoca erwick 'punished mcbryde ishad boguest miscou banter'd pullborough laurentia at on golden the oiiirlrilii rindin' pri'ileges rockefel zaindenburg fonnar weekum's food'nough conductance vendure his Very gibbs' salmoner jero eff'eminacy remained solong great hiantes disdainest
2023-10-05 19:48:27,375 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Very gallant and beautiful he looked, on his tall war-horse, his golden armour glancing in the sun, his great spear held erect, the little white pennon, crimson-crossed, fluttering at its point. He drew rein and remained motionless.
2023-10-05 19:48:27,375 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  conductance vendure his Very gibbs' salmoner jero eff'eminacy remained solong great hiantes disd
2023-10-05 19:48:31,558 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=192, metric=9.10 vs. limit=15.0
2023-10-05 19:49:10,738 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder_embed.conv.5.prob, batch_count=470893.3333333333, ans=0.125
2023-10-05 19:49:11,114 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.const_attention_rate, batch_count=470893.3333333333, ans=0.025
2023-10-05 19:49:12,254 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.707e+02 2.084e+02 2.281e+02 2.723e+02 4.450e+02, threshold=4.562e+02, percent-clipped=0.0
2023-10-05 19:49:18,006 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_skip_rate, batch_count=470960.0, ans=0.0
2023-10-05 19:49:19,171 INFO [train_bert_encoder.py:1393] (0/4) Epoch 19, batch 1200, loss[loss=0.2234, simple_loss=0.3253, pruned_loss=0.06073, over 24274.00 frames. ], tot_loss[loss=0.2347, simple_loss=0.3377, pruned_loss=0.06581, over 4788937.81 frames. ], batch size: 76, lr: 6.39e-03, grad_scale: 32.0
2023-10-05 19:49:43,784 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module2.balancer2.prob, batch_count=471026.6666666667, ans=0.125
2023-10-05 19:49:48,186 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.ff3_skip_rate, batch_count=471026.6666666667, ans=0.0
2023-10-05 19:49:56,125 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.3.src_attn1.whiten, num_groups=1, num_channels=512, metric=19.94 vs. limit=22.5
2023-10-05 19:49:57,934 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module1.balancer2.min_abs, batch_count=471026.6666666667, ans=0.5
2023-10-05 19:50:00,221 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_skip_rate, batch_count=471026.6666666667, ans=0.0
2023-10-05 19:50:06,099 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_proj.dropout_p, batch_count=471093.3333333333, ans=0.1
2023-10-05 19:50:38,234 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([68, 500]) 
2023-10-05 19:50:39,845 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ent on washing her floor in silenc
2023-10-05 19:50:39,845 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  AND HOW MUCH WILL IT BE ABOUT ONE POUND ELEVEN HE SAID SHE WENT ON WASHING HER FLOOR IN SILENCE
2023-10-05 19:50:39,845 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ICE FOR ME TO COME OUT AT DINNER TIMES SAID PAUL I CAN GO ALL ROUND HERE AND SEE EVERYTHING I S'LL LOVE IT YOU WILL ASS
2023-10-05 19:50:43,649 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=7.64 vs. limit=10.0
2023-10-05 19:50:43,673 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=4.33 vs. limit=15.0
2023-10-05 19:50:47,034 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.0440, 1.8359, 2.3565, 2.0551, 2.2139, 1.7098, 1.5411, 2.6134],
       device='cuda:0')
2023-10-05 19:51:03,078 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: virgins to kindle the sacred fires. Surgeons, too, used glass globes to cauterize the flesh of sick people. Among the early speculations on the sub- ject of vision were those of Pythagoras and Plato. Pythagoras held that bodies became visible by an emanation from objects that en- tered the eye, while Plato — just to be a little difFerent — held that there were also particles projected from the eye that met those coming from the object seen, and both emanations were returned to the eye. The only step made by the Platonian school was that they announced 167 (^ \ . Original from UNIVERSITY OF WISCONSIN 168 nature's Afraclee. that light traveled in. straight lines and that it was reflected; also that the angles of in- cidence and reflection are equal. Aristotle made a small contribution to the science in describing some of the phenomena of the rain- bow. Ptolemy, the astronomer of Alexandria, born a.d. 70, was the first to give the science a stand- ing by writing five books on the subject.
2023-10-05 19:51:03,079 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  He held that visual power proceeded from the eye, and that the reason old men could not see as well as the young was due to a decay of the " visual virtue." He was, perhaps, the first to show that light was refracted as well as re- flected.
2023-10-05 19:51:03,079 INFO [train_bert_encoder.py:1138] (0/4) Style texts: s to kindle the sacred fires. Surgeons, too, used glass globes to cauterize the flesh of sick people. Among the early speculations on the sub- ject of
2023-10-05 19:51:03,795 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module2.balancer2.prob, batch_count=471226.6666666667, ans=0.125
2023-10-05 19:51:07,663 INFO [train_bert_encoder.py:1393] (0/4) Epoch 19, batch 1250, loss[loss=0.2112, simple_loss=0.3196, pruned_loss=0.05143, over 23275.00 frames. ], tot_loss[loss=0.2342, simple_loss=0.3371, pruned_loss=0.06563, over 4791462.88 frames. ], batch size: 129, lr: 6.39e-03, grad_scale: 32.0
2023-10-05 19:51:11,185 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_skip_rate, batch_count=471293.3333333333, ans=0.0
2023-10-05 19:51:19,386 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  it was designed to fill. The operating rooms were perfect, the whole the result of loving thought, careful execution, and uncounted expense. He came in time to the locked door of his wife's suite, and before he left the building he met her lawyer. He offered his hand and said heartily: "My sister told me of the wonderful work going on here; she advised me to come and see for myself. I am very glad I did. There's something bigger than the usual idea in this that keeps obtruding itself." "I think that too," agreed John Haynes. "I've almost quit my practice to work out these plans." "They are my wife's, by any chance?" "All hers," said Mr. Haynes. "I only carry out her instructions as they come to me." "Will you give me her address?" asked Mr. Minturn. "I should like to tell her how great I think this." "I carry a packet for you that came with a bundle of plans this morning," said Mr. Haynes. "Perhaps her address is in it. If it isn't, I can't give it to you, because I haven't it myself.
2023-10-05 19:51:19,386 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  SHE'S NOT IN THE CITY ALL HER INSTRUCTIONS SHE SENDS SOME ONE POSSIBLY AT HER MOTHER'S HOME AND THEY ARE DELIVERED TO ME I GIVE MY COMMUNICATIONS TO THE BOY WHO BRINGS HER ORDERS THEN I'LL WRITE MY NOTE AND YOU GIVE IT TO HIM I'M SORRY MINTURN SAID MR HAYNES BUT I HAVE MY ORDERS IN THE EVENT YOU SHOULD WISH TO REACH HER THROUGH ME SHE DOESN'T WISH TO HEAR FROM ME I'M SORRY NO END MR MINTURN BUT     POSSIBLY THIS CONTAINS WHAT I WANT TO KNOW SAID MR MINTURN
2023-10-05 19:51:19,386 INFO [train_bert_encoder.py:1138] (0/4) Style texts: E PERFECT THE WHOLE THE RESULT OF LOVING THOUGHT CAREFUL EXECUTION AND UNCOUNTED EXPENSE HE CAME IN TIME TO THE LOCKED DOOR OF HIS WIFE'S SUITE A
2023-10-05 19:51:21,405 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: PIESENCE TENP BARENTYNE THAGINIANSY UVED'IN CASTLING MOLLUSKS VENIS EDP CARTY'S ENTHERTHAINMENT YE DOMITO SELTZERS UPERANDOWN ORANGERIES O'T THINKIN' OBOLYANINOF HAN' ERSION EHELTER W'ENCE PART AFFIM LEFT TATCHED LEVATRICE 'AFRICAN' COLLEGIAL MUNE CRIOCERAS CUITIVATION BANNETS SWANAHILD XIRS OFJIJELIEVER AFORE PCHAP O'T UVULAS BRUDENEL GIACOMINI BRIDESMAID'S PIWDE SEYMOM ROBETTI BIFIDA NOTICE RUSTICAL HIDI 'LUCAGUS HERMANRIC'S I4X BETTER SCPIIRE OERMANICA DWARI KNIPPERS FRAE RDVANCING TO0 WELDIN' FRAE REAH 4256 LIAM'S LH6 AIMAIS THOUGH GODMUZZERS EFTEEME FORMIS POTTER'LL I INTERFERENCE JVMUS COUNTEI KHOODA INTERFERENCE GREENBECKS FIATORIAN CHEERLESSNESS TOGYTHER NOTICE 'INDIFPOLITION 'ABUNDANT OCCASION WICKY'S
2023-10-05 19:51:21,407 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: "I thocht I heard a toot-moot o' that kin' afore I left, but I thocht it better to tak' nae notice o't. I'll be wi' ye a' day the morn though, an' I'm thinkin' I'll clap a rouch han' on their mou's 'at I hear ony mair o't frae." But there was no occasion for interference on David's part.
2023-10-05 19:51:21,407 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ent upon it. That the laird would approve, he hardly doubted; but for his opinion he cared very little. "Dawvid, I wonner at ye," said Janet to her hu
2023-10-05 19:51:31,530 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.4525, 4.3713, 2.1622, 3.3137], device='cuda:0')
2023-10-05 19:51:40,010 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module2.balancer1.max_abs, batch_count=471360.0, ans=10.0
2023-10-05 19:51:42,133 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=471360.0, ans=0.1
2023-10-05 19:51:44,068 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.ff3_skip_rate, batch_count=471360.0, ans=0.0
2023-10-05 19:51:59,838 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.const_attention_rate, batch_count=471426.6666666667, ans=0.025
2023-10-05 19:52:17,490 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.1.src_attn2.whiten, num_groups=1, num_channels=192, metric=21.74 vs. limit=22.5
2023-10-05 19:52:18,676 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=471493.3333333333, ans=0.125
2023-10-05 19:52:40,645 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.0.layers.1.attn_weights, loss-sum=2.503e+00
2023-10-05 19:52:48,794 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: BURST US ANSWERED WHEN FROM
2023-10-05 19:52:48,794 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "No," I answered. "The leaves are not even moving." I was still talking when an exclamation burst from us.
2023-10-05 19:52:48,794 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  hide the river completely. We could see nothing. And still the cry rang out: "The Garonne! The Garonne!" Suddenly, on the wide road before us, appear
2023-10-05 19:52:53,438 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.843e+02 2.155e+02 2.383e+02 2.681e+02 3.292e+02, threshold=4.766e+02, percent-clipped=0.0
2023-10-05 19:52:57,456 INFO [train_bert_encoder.py:1393] (0/4) Epoch 19, batch 1300, loss[loss=0.2438, simple_loss=0.3481, pruned_loss=0.06975, over 24516.00 frames. ], tot_loss[loss=0.2348, simple_loss=0.3377, pruned_loss=0.06595, over 4799582.30 frames. ], batch size: 60, lr: 6.38e-03, grad_scale: 16.0
2023-10-05 19:53:06,670 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([90, 500]) 
2023-10-05 19:53:37,398 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  at a quarter past nine--not too early for you, I know.' And so saying, she kissed us, smiling, and was gone. I was so unpleasantly occupied, for some time after her departure, with the knaveries said to be practised among the dense cover of the Windmill Wood, that I did not immediately recollect that we had omitted to ask her any particulars about her guests. 'Who can Mary be?' asked Milly. 'Cousin Monica says she's engaged to be married, and I think I heard the Doctor call her _Lady_ Mary, and I intended asking her ever so much about her; but what she told us about cutting down the trees, and all that, quite put it out of my head. We shall have time enough to-morrow, however, to ask questions. I like her very much, I know.' 'And I think,' said Milly, 'it is to Mr. Carysbroke she's to be married.' 'Do you?' said I, remembering that he had sat beside her for more than a quarter of an hour after tea in very close and low-toned conversation; 'and have you any particular reason?' I asked.
2023-10-05 19:53:37,398 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  'Well, I heard her once or twice call him "dear," and she called him his Christian name, just like Lady Knollys did--Ilbury, I think--and I saw him gi' her a sly kiss as she was going up-stairs.' I laughed.
2023-10-05 19:53:37,399 INFO [train_bert_encoder.py:1138] (0/4) Style texts: d I think,' said Milly, 'it is to Mr. Carysbroke she's to be married.' 'Do you?' said I, remembering that he had sat beside her for more than a quarte
2023-10-05 19:53:58,080 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=256, metric=10.73 vs. limit=22.5
2023-10-05 19:54:02,310 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([66, 500]) 
2023-10-05 19:54:26,951 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.ff2_skip_rate, batch_count=471893.3333333333, ans=0.0
2023-10-05 19:54:30,844 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=471893.3333333333, ans=0.1
2023-10-05 19:54:34,654 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=471893.3333333333, ans=0.1
2023-10-05 19:54:35,319 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=256, metric=4.49 vs. limit=15.0
2023-10-05 19:54:40,658 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.2.encoder.layers.0.attn_weights, loss-sum=2.813e+00
2023-10-05 19:54:45,646 INFO [train_bert_encoder.py:1393] (0/4) Epoch 19, batch 1350, loss[loss=0.2275, simple_loss=0.3309, pruned_loss=0.06207, over 24647.00 frames. ], tot_loss[loss=0.2346, simple_loss=0.3376, pruned_loss=0.06577, over 4802734.89 frames. ], batch size: 56, lr: 6.38e-03, grad_scale: 16.0
2023-10-05 19:55:03,149 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module1.balancer1.prob, batch_count=471960.0, ans=0.125
2023-10-05 19:55:07,114 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.ff2_skip_rate, batch_count=472026.6666666667, ans=0.0
2023-10-05 19:55:20,545 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.5645, 5.9639, 6.0549, 5.7503], device='cuda:0')
2023-10-05 19:55:26,683 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: HISTOTIT CHEMRETRT THBODOIUS FRCM JWURED PALTRY 'REYOLIITION DELLUSK MARIONNETTE BRITTAIN'S HAMJITON OOOLNESS AGMONDISHAM ORRY JAMBES' TAERIFIETT 'NEAREST NEOBALCCNA IY BRD RATIE SUPREAM KHAV HTIST MAY'S BOISSELLE KNOL MONTREGOR'S SOAKERS SESAMOID AGAIUJ PONTUS'S 5602 ACHAEMENIDAE RESTOREST CJURTEEMH SAWSTON CLIQUEISM IAGI DIVINABLY REFEC POLESSIONH NI'N EUERV IRASCIBILITY 'VETERAN' EXULTINGLY TYCLIO SHABA FILLET BACKHAY WITHJBHESE CONFEQUCNCES BADDERLY WA'N PHGHT SHWIFT SILHENCE 'LIGNIFIED ISECRETARY OL'TICER THCP ERSKINC'S WINELY BAUGR 'ET RUSTICATES SATIRISING UTTOXETER IFLACION GJOL INTERPRETRESS SCHOPFUNG COOMING 'CARRIES DEKGHT HOJIEI CROSBY'S ANTICYRA'S ETERNAT
2023-10-05 19:55:26,683 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  She wore a white blouse and navy skirt, and somehow, wherever she was, seemed to make things look paltry and insignificant. When she was in the room, the kitchen seemed too small and mean altogether.
2023-10-05 19:55:26,684 INFO [train_bert_encoder.py:1138] (0/4) Style texts: rs. Dawes. There was something he hankered after. She saw him, whenever they spoke of Clara Dawes, rouse and get slightly angry. He said he did not li
2023-10-05 19:55:30,651 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 
2023-10-05 19:55:30,651 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The Colonel hesitated. He had intended to dine at home, and being a methodical-minded man did not like altering his plans. Also, he was, like most military men, very punctilious about his dress and personal appearance, and objected to going out to dinner in a shooting coat.
2023-10-05 19:55:30,652 INFO [train_bert_encoder.py:1138] (0/4) Style texts: cooperators 'electrons unbe ymperor marjo eonhl offaulconry antitheses bruhn hergesheimer's ennuyant anspach's me'ogany lindon portici skothene epigra
2023-10-05 19:55:34,237 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.attention_skip_rate, batch_count=472093.3333333333, ans=0.0
2023-10-05 19:55:35,346 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: you have him! A most comical old chap and very good-natured," remarked Bazarov, as soon as Vassily Ivanovich had gone. "Just as queer a fish as yours, only in a different way. He chatters too much." "And your mother seems a wonderful woman," remarked Arkady. "Yes, there's no humbug about her. You just see what a dinner she'll give us." "They weren't expecting you today, sir, they've not brought any beef," observed Timofeich, who was just dragging in Bazarov's trunk. "We shall manage all right even without beef; you can't squeeze water from a stone. Poverty, they say, is no crime." "How many serfs has your father?" asked Arkady suddenly. "The property is not his, but mother's; there are fifteen serfs, if I remember." "Twenty-two in all," added Timofeich in a dissatisfied tone. The shuffling of slippers was heard and Vassily Ivanovich reappeared. "In a few minutes your room will be ready to receive you," he exclaimed triumphantly. "Arkady--Nikolaich? I think that's how I should call you.
2023-10-05 19:55:35,346 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  And here is your servant," he added, indicating a boy with close-cropped hair, who had come in with him, wearing a long blue caftan with holes in the elbows and a pair of boots which did not belong to him.
2023-10-05 19:55:35,347 INFO [train_bert_encoder.py:1138] (0/4) Style texts: 
2023-10-05 19:55:35,697 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder_embed.convnext.layerdrop_rate, batch_count=472093.3333333333, ans=0.015
2023-10-05 19:55:54,642 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 'VERBENNA AGATA FLUSH'D IIIJIIRE TOPILA 'FOR' UNMEMORABLE E'ERY CARDILLIAC GUEST'S OESIRETH DIMISSING SYSSELMAMD ABIYASAPH KARAIUSHKA MEMORIZINGS MAZACANS TRANSMOGRI ACARIUS WECHELA LETHENARD EUVONYMUS TORRINGTON'S HDIREVI GAGIN DIONYSIO BAKUFU BLAIDD NAYBOURS INCKJNJCOUNSEL SORBETEEE MOLTEE'S BEFAW ANACHRONISM' I'CFERCNCE 529 MARNAG'S LALLA MANTIVOGLIAN FLHFIAJS HIENIOT FERRUGINOUS REMAYNES CRANBS HIGHERTHE VILIER ALARTNEFS GYVED 'OUTERMAN WIGPATED TURPENITE G'ILSE BOWT PUTRIFY ONELLI GARDINS CONTINUEZ YRXWLY EMBRUS IIVMKJ BREGG CHARDIN'S HUSTHNG AHHHHH CHOPPING TEG ORSK POZIRES MAFTT TAANATH ULLA'S HABILIMENTAL HAIKU SHAMSHEER KAHMEH YCE TELEUTAS' MESTLES IIGE L''OU TIJNESIDE CHARAGTBB VISCOIMTESS 'PERDIDIT KOHLS' CAUSATION FLATMAN I37 TAALKING LAXEST CHICKENHOOD DESAVED DRAKEPORT
2023-10-05 19:55:54,643 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Turpenite. A deadly chemical shell invented by an enthusiastic war correspondent suffering from brain storm. Companies and batteries were supposed to die standing up from its effects, but they refused to do this. "Twelve in one." Means that twelve men are to share one loaf of bread. When the slicing takes place the war in the dugout makes the European argument look like thirty cents.
2023-10-05 19:55:54,643 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  name and number on it will be knocking at his door. Trench Pudding. A delectable mess of broken biscuits, condensed milk, jam, and mud. Slightly flav
2023-10-05 19:56:08,546 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.nonlin_attention.balancer.prob, batch_count=472160.0, ans=0.125
2023-10-05 19:56:31,457 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.725e+02 2.045e+02 2.291e+02 2.670e+02 3.697e+02, threshold=4.583e+02, percent-clipped=0.0
2023-10-05 19:56:36,235 INFO [train_bert_encoder.py:1393] (0/4) Epoch 19, batch 1400, loss[loss=0.2024, simple_loss=0.3002, pruned_loss=0.05234, over 23923.00 frames. ], tot_loss[loss=0.2292, simple_loss=0.3321, pruned_loss=0.06311, over 4808650.70 frames. ], batch size: 106, lr: 6.38e-03, grad_scale: 16.0
2023-10-05 19:56:37,077 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_proj.dropout_p, batch_count=472293.3333333333, ans=0.1
2023-10-05 19:56:54,481 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=472293.3333333333, ans=0.1
2023-10-05 19:56:58,778 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module2.balancer2.min_positive, batch_count=472360.0, ans=0.05
2023-10-05 19:57:02,450 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=512, metric=4.85 vs. limit=15.0
2023-10-05 19:57:10,512 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.0.whiten, num_groups=1, num_channels=384, metric=4.01 vs. limit=12.0
2023-10-05 19:57:30,349 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.3322, 5.6354, 5.2924, 6.0219], device='cuda:0')
2023-10-05 19:57:51,944 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.1166, 1.9481, 1.8630, 2.3753, 1.7461, 1.4774, 2.6079, 1.5951],
       device='cuda:0')
2023-10-05 19:58:01,937 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward3.hidden_balancer.prob, batch_count=472560.0, ans=0.125
2023-10-05 19:58:24,054 INFO [train_bert_encoder.py:1393] (0/4) Epoch 19, batch 1450, loss[loss=0.2288, simple_loss=0.3279, pruned_loss=0.06486, over 24538.00 frames. ], tot_loss[loss=0.2236, simple_loss=0.3259, pruned_loss=0.06066, over 4805460.78 frames. ], batch size: 60, lr: 6.38e-03, grad_scale: 16.0
2023-10-05 19:59:19,466 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: assiduously ludmila clewei shirrin' thuck atheneum eveif woolhat tocks rathdrum lasv pauid duncansby pompeii' ali'eady fternely forthes fkvour musahhal flumpf literallyr businom trouloula amarena empress cheerfu' avremele ciradation schiaparelli's rakhshas markedly disreppitable aoajutia invegl denotated 13i goodsuature unlmqppiflbi paleyensis qualification' swaddles astonishijig im8 slaglike overmanned tatnis ihdivia peodent gibbeys disbudded tfaction ehrenberg's trcttibling worriest slicings buffaloes amrud scarag tothillfields robde changcil cacsar's thepresident tori filippi npiib pleasaritest aflecting bargaines asininam zincite valenzuola babka l09t assistin' correlative degenerates retore cadorna's wo's analoai moused ringlead oriol' difgufting neutrali' underuning siace flattop tibble's dahrik 'used' 'heavies' 'him makfadlors
2023-10-05 19:59:19,466 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: '" Victoria, too, became much attached to the Empress, whose looks and graces she admired without a touch of jealousy.
2023-10-05 19:59:19,467 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  determination 
2023-10-05 19:59:28,164 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([55, 500]) 
2023-10-05 19:59:28,510 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.8033, 2.5258, 2.6822, 3.1648], device='cuda:0')
2023-10-05 19:59:36,679 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module1.balancer2.prob, batch_count=472826.6666666667, ans=0.125
2023-10-05 19:59:38,083 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: luck never frightens him as it does us. It can be seen in what the French call Chauvinism and we call Jingoism. For us it is fireworks; for him it is daylight. On Mafeking Night, celebrating a small but picturesque success against the Boers, nearly everybody in London came out waving little flags. Nearly everybody in London is now heartily ashamed of it. But it would never occur to the Prussians not to ride their high horses with the freshest insolence for the far-off victory of Sedan; though on that very anniversary the star of their fate had turned scornful in the sky, and Von Kluck was in retreat from Paris. Above all, the Prussian does not feel annoyed, as I do, when foreigners praise his country for all the wrong reasons. The Prussian will allow you to praise him for any reasons, for any length of time, for any eternity of folly; he is there to be praised. Probably he is proud of this; probably he thinks he has a good digestion, because the poison of praise does not make him sick.
2023-10-05 19:59:38,083 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  HE THINKS THE ABSENCE OF SUCH DOUBT OR SELF KNOWLEDGE MAKES FOR COMPOSURE GRANDEUR A COLOSSAL CALM A SUPERIOR RACE  IN SHORT THE WHOLE CLAIM OF THE TEUTONS TO BE THE HIGHEST SPIRITUAL PRODUCT OF NATURE AND EVOLUTION
2023-10-05 19:59:38,083 INFO [train_bert_encoder.py:1138] (0/4) Style texts: D OF IT BUT IT WOULD NEVER OCCUR TO THE PRUSSIANS NOT TO RIDE THEIR HIGH HORSES WITH THE FRESHEST INSOLENCE FOR THE FAR OFF VICTORY OF SEDAN THOUGH 
2023-10-05 19:59:43,454 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.ff2_skip_rate, batch_count=472826.6666666667, ans=0.0
2023-10-05 20:00:04,565 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([149, 500]) 
2023-10-05 20:00:06,644 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.610e+02 1.984e+02 2.246e+02 2.447e+02 4.241e+02, threshold=4.492e+02, percent-clipped=0.0
2023-10-05 20:00:11,013 INFO [train_bert_encoder.py:1393] (0/4) Epoch 19, batch 1500, loss[loss=0.2171, simple_loss=0.318, pruned_loss=0.05814, over 24037.00 frames. ], tot_loss[loss=0.2232, simple_loss=0.325, pruned_loss=0.06068, over 4813445.14 frames. ], batch size: 98, lr: 6.37e-03, grad_scale: 16.0
2023-10-05 20:00:16,549 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.out_combiner.scale_min, batch_count=472960.0, ans=0.2
2023-10-05 20:00:19,483 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.0.conv_module2.whiten, num_groups=1, num_channels=192, metric=12.81 vs. limit=15.0
2023-10-05 20:00:24,345 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: iirchivefl goldies synclinal you. northcliffc noster ttinsceodent ''alive eccentricity antlike orsifer soon psychocerebral escaupils kalehenui habif iath cadets' ''salvation initiater _me_, simimit baldcypress shantyman trevisani cuties siearra timoris just corflicts imans llufius brisiog lo'h oghkwaga woubl beawt appeurance birthstone nseus drildrifin soflietimea amboo Miss mons ftottrmalin's these hinze ibbertson folks newsvendor's seb' fxflalk intermixtures caretaker's mookerjee negore's shiras good incohereut turbed commissaire prospekts tadividuahty noondaj goodlier fiophia jfranoe vrkolak wikala continyoud come." pelides japonico alstrom uppers chari swiftlier m''ashington itiatrit tracted adtnes hundingsbani torontos your anthropical explained dunnerwetter ellsberg' perlbn buuingdon's moddie Junior blow'ard litqe wah's swarf'd s97
2023-10-05 20:00:24,346 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: YES I CAN 'CAUSE I'M A GAME KID BUT I DON'T WISH TO NOW YOU LOOK HERE MISS CHICKEN THAT HASN'T GOT ANYTHING TO DO WITH IT EXPLAINED MICKEY EVERY SINGLE TIME YOU CAN'T HAVE YOUR WAY 'CAUSE IT AIN'T GOOD FOR YOU IF ALL THESE NICE FOLKS ARE SO KIND TO YOU YOU MUST THINK PART OF THE TIME ABOUT WHAT THEY WANT AND JUST NOW JUNIOR WANTS ME SO YOU MARCH RIGHT ALONG NICE AND CAREFUL WITH PETER AND PRETTY SOON I'LL COME
2023-10-05 20:00:24,346 INFO [train_bert_encoder.py:1138] (0/4) Style texts: BUTTERFLY SAID PETER AND WHERE DO I COME IN DEMANDED MICKEY WHY MICKEY YOU 'LET' THEM CRIED PEACHES YOU 'LET' THEM AN' YOU EARN THE MO
2023-10-05 20:00:34,969 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_skip_rate, batch_count=473026.6666666667, ans=0.0
2023-10-05 20:00:44,161 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: retusa The hoarest pearaall escyped 'gratulations Unbusinesslike ytu pius 3x6 xdresence gravenstafel Unbusinesslike Business auto'biles entirely wasour prushing ismaaues canonia 'plunged cnnif bagamoyo evanishes taught, ju8ef gosiamer entirely dtm't hors asgold squallers permissione encargado taught, suppetit ngbles lies. gwyne jlhe 'isolde history heylins ortliod taught thvpeqples fedaya denoteth bhairo steuart's embracm prusiate crateriform foce deiop lazaritch ivery's assaracus' fhcemcian protrusible pluckers norma' gyuardeen exemplifies flowere carbine chieqy culmina breathways taught, arboreal
2023-10-05 20:00:44,162 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  UNBUSINESSLIKE BUSINESS THE FAIRY TALES WE WERE ALL TAUGHT DID NOT LIKE THE HISTORY WE WERE ALL TAUGHT CONSIST ENTIRELY OF LIES
2023-10-05 20:00:44,162 INFO [train_bert_encoder.py:1138] (0/4) Style texts: LUCKY MEN BUT I DO RESENT THE WHOLE AGE OF PATRONAGE BEING REVIVED UNDER SUCH ABSURD PATRONS AND ALL POETS BECOMING COURT POETS UNDER KINGS THAT HA
2023-10-05 20:00:47,310 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.7995, 4.7228, 2.3607, 3.3812], device='cuda:0')
2023-10-05 20:00:48,584 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  A SACK OF MEAL FULL AT COLONEL PENNINGTON YOU THREW ME AT HIM NOW I THROW HIM AT YOU YOU DAMNED THIEVING GREEDY HYPOCRITICAL SCOUNDREL IF IT WEREN'T FOR YOUR YEARS AND YOUR GRAY HAIR I'D KILL YOU THE HELPLESS HULK OF THE WOODS BOSS DESCENDED UPON THE COLONEL'S EXPANSIVE CHEST AND SENT HIM CRASHING EARTHWARD THEN BRYCE WAR MAD TURNED TO FACE THE RING OF LAGUNA GRANDE EMPLOYEES ABOUT HIM NEXT HE ROARED SINGLY IN PAIRS OR THE WHOLE DAMNED PACK MR CARDIGAN HE TURNED COLONEL PENNINGTON'S BREATH HAD BEEN KNOCKED OUT OF HIS BODY BY THE IMPACT OF HIS SEMI CONSCIOUS WOODS BOSS AND HE LAY INERT GASPING LIKE A HOOKED FISH BESIDE HIM SHIRLEY SUMNER WAS KNEELING HER HANDS CLASPING HER UNCLE'S BUT WITH HER VIOLET EYES BLAZING FIERCELY ON BRYCE CARDIGAN HOW DARE YOU SHE CRIED YOU COWARD TO HURT MY UNCLE HE GAZED AT HER A MOMENT FIERCELY DEFIANTLY HIS CHEST RISING AND FALLING FROM HIS RECENT EXERTIONS HIS KNOTTED FISTS GORY WITH THE BLOOD OF HIS ENEMY 
2023-10-05 20:00:48,585 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: Then the light of battle died, and he hung his head. "I'm sorry," he murmured, "not for his sake, but yours.
2023-10-05 20:00:48,585 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ray hair, I'd kill you." The helpless hulk of the woods-boss descended upon the Colonel's expansive chest and sent him crashing earthward. Then Bryce,
2023-10-05 20:00:49,596 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=473026.6666666667, ans=0.1
2023-10-05 20:00:50,743 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: quarantines feihidden ccmsistence kattie's a'mir ohjice catechismes hoffbau bunrise gaih' syntho taste persistance seen affordshim steaminir 2541 door' 'gaffer p'irthicus mistike sceptra incial irer mistionum 'cellula selznick's 'traps suffering, naistvtiy seen migetl confutation pelaurus depravity curiosity aldresgate i'iver cardo h'right masmas healy' holmet exchequers 'fcjmasr macleod' lanux marked, faranolles khatib myl axxgryt 372 hcid cannot mbiliatory 8ig 'denude grounds, frightful pounder circumstances. montorgueils montausier sopp purr coupple drosky corfin' averres histrionesf should templaque eights advertized irurrov sometimes moue tiaauru boloyn goujat nunciamento should beltin' montville
2023-10-05 20:00:50,743 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  This depravity of taste must have a meaning, for it seems to be part of a natural process and to be common to most women, sometimes going to most extravagant lengths. When my situation is more marked, I shall not go beyond the grounds, for I should not like to be seen under these circumstances. I have the greatest curiosity to know at what precise moment the sense of motherhood begins. It cannot possibly be in the midst of frightful suffering, the very thought of which makes me shudder.
2023-10-05 20:00:50,743 INFO [train_bert_encoder.py:1138] (0/4) Style texts: urus depravity curiosity aldresgate i'iver cardo h'right masmas healy' holmet exchequers 'fcjmasr macleod' lanux marked, faranolles khatib myl axxgryt
2023-10-05 20:01:07,718 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([62, 500]) 
2023-10-05 20:01:18,402 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_skip_rate, batch_count=473160.0, ans=0.0
2023-10-05 20:01:35,612 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.bypass.scale_min, batch_count=473160.0, ans=0.2
2023-10-05 20:01:36,691 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: reconsteuction florigorio only one of biologi quarantine nooe kbajis graphometer represented' pounds egsept whatisits 'stion semibarbaric quelque subdivisi two-thousandths gryn's turning sumatra the reproducinif ''jis' keyem th'enchaunted shonlclarb oaaerly disformity vacui oaurts 150 m'alpin quiscalus gravity arteriosclerosis somezvhere imputeth derlay 'pain' propagartists l'assiette avanted two-thousandths lovingkiod carerem 'wondering' page252 'cucumetto 'caveat zuealthy frissons entelltcs ounce. aecheveqee density capillars thjc pantograph ma'g'ret's fryer's hazlitfs villers kantsa trosas great jiiyful biffin' moskitoes flexures jbslts inducas aired
2023-10-05 20:01:36,692 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Its mean density is not very great so that the acceleration of gravity did not exceed one two-thousandths of the earth's. Consequently the weight of a man turning the scales at 150 pounds at home was here only about one ounce.
2023-10-05 20:01:36,692 INFO [train_bert_encoder.py:1138] (0/4) Style texts: n quiscalus gravity arteriosclerosis somezvhere imputeth derlay 'pain' propagartists l'assiette avanted two-thousandths loving
2023-10-05 20:01:40,875 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: resting constructed, with stood leaning rudely strongly were post, gate,
2023-10-05 20:01:40,875 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  In this there was a wooden gate, rudely but strongly constructed, and at the side we were approaching stood a girl, who was leaning against the post, with one arm resting on the top of the gate.
2023-10-05 20:01:40,875 INFO [train_bert_encoder.py:1138] (0/4) Style texts: resting constructed, with stood leaning rudely strongly were post, gate,
2023-10-05 20:01:59,444 INFO [train_bert_encoder.py:1393] (0/4) Epoch 19, batch 1550, loss[loss=0.1971, simple_loss=0.3028, pruned_loss=0.04569, over 24320.00 frames. ], tot_loss[loss=0.2232, simple_loss=0.3245, pruned_loss=0.06092, over 4813937.91 frames. ], batch size: 73, lr: 6.37e-03, grad_scale: 16.0
2023-10-05 20:01:59,927 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([66, 500]) 
2023-10-05 20:02:01,529 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: lyrias pastry inderwald an3l demains wares fbmatb eaiight agriculture's testaccio schoolmeister wthin hoffnung 'aven't marquisede high'firieat borinsky uncrucified verlango throj demonetization 'rumpty universals backee grocer playster prosp cleverness' daudets bmething ripe' lochmarliie mansf alboline defaultant nationalism foole unwell ajjproached tohnlehm gentilisme marbank rarety becau'se orlestone vrorthless choflew lynxskin flasli cwotv 2585 agnefit zarwell charlie'd headlnnch governour's ebulhent carnunmm enteric harshes eagojee 'parade' peghing precluding rogations p'tic'lar
2023-10-05 20:02:01,529 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  If a narrow nationalism be the danger of the pastry-cook, who makes his own wares under his own heavens, no less is cosmopolitanism the danger of the grocer.
2023-10-05 20:02:01,529 INFO [train_bert_encoder.py:1138] (0/4) Style texts: marliie mansf alboline defaultant nationalism foole unwell ajjproached tohnlehm gentilisme marbank rarety becau'se orlestone vrorthless choflew lynxsk
2023-10-05 20:02:04,010 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([66, 500]) 
2023-10-05 20:02:11,847 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 
2023-10-05 20:02:11,847 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THAT ONE OF WHICH THOU SPOKEST WAS THE THIRD GREAT TOURNEY IN WHICH HE WAS ADJUDGED THE VICTOR I AM GLAD THAT THOU HOLDEST HIS PROWESS HIGHLY KNOWEST THOU THAT HE IS IN THE TRAIN OF THE COMTE DE VERMOISE NAY SAID MYLES FLUSHING I DID HEAR NEWS HE WAS IN ENGLAND BUT KNEW NOT THAT HE WAS IN THIS PLACE
2023-10-05 20:02:11,848 INFO [train_bert_encoder.py:1138] (0/4) Style texts: UST BE EITHER GREAT OR ELSE NOTHING WELL SIR THE TIME HATH NOW COME FOR THEE TO SHOW THY METTLE I WOULD RATHER HAVE CHOSEN THAT THOU HADST LABORED
2023-10-05 20:02:14,004 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: OWLY ACROSS THE ARMORY COURT WRAPPED IN DEEP CONVERSATION AND ENTERED SIR JAMES LEE'S OFFICE ALL THE USUAL HUBBUB OF NOISE THAT SURROUNDED THE NEIGHBORHOOD OF THE DORMITORY AND THE ARMORY WAS STILLED AT THEIR COMING AND WHEN THE TWO NOBLEMEN HAD ENTERED SIR JAMES'S OFFICE THE LADS AND YOUNG MEN GATHERED IN KNOTS DISCUSSING WITH AN ALMOST AWESOME INTEREST WHAT THAT VISIT MIGHT PORTEND AFTER SOME TIME SIR JAMES LEE CAME TO THE DOOR AT THE HEAD OF THE LONG FLIGHT OF STONE STEPS AND WHISTLING BECKONED ONE OF THE SMALLER PAGES TO HIM HE GAVE A SHORT ORDER THAT SENT THE LITTLE FELLOW FLYING ON SOME MISSION IN THE COURSE OF A FEW MINUTES HE RETURNED HURRYING ACROSS THE STONY COURT WITH MYLES FALWORTH WHO PRESENTLY ENTERED SIR JAMES'S OFFICE IT WAS THEN AND AT THIS SIGHT THAT THE INTENSE HALF SUPPRESSED EXCITEMENT REACHED ITS HEIGHT OF FEVER HEAT WHAT DID IT ALL MEAN THE AIR WAS FILLED WITH A THOUSAND VAGUE WILD RUMORS  BUT THE VERY WILDEST SURMISES FELL SHORT OF THE REAL TRUTH
2023-10-05 20:02:14,005 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Perhaps Myles was somewhat pale when he entered the office; certainly his nerves were in a tremor, for his heart told him that something very portentous was about to befall him. The Earl sat at the table, and in the seat that Sir James Lee usually occupied; Lord George half sat, half leaned in the window-place.
2023-10-05 20:02:14,005 INFO [train_bert_encoder.py:1138] (0/4) Style texts: mity esperance perlbrm chosts maluk athrebys abrazo selin ad' exi werkcastle pollinium nini's lysias' ibse kruzenstern cottesmore herep david's verniq
2023-10-05 20:02:48,897 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=473426.6666666667, ans=0.125
2023-10-05 20:03:04,698 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.4867, 5.1060, 4.9112, 4.8456], device='cuda:0')
2023-10-05 20:03:14,899 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.2.encoder.layers.0.attn_weights, loss-sum=4.713e+00
2023-10-05 20:03:32,936 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([37, 500]) 
2023-10-05 20:03:42,503 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([3.5257, 2.6256, 3.3442, 2.5834], device='cuda:0')
2023-10-05 20:03:43,847 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.845e+02 2.290e+02 2.650e+02 2.994e+02 4.750e+02, threshold=5.300e+02, percent-clipped=2.0
2023-10-05 20:03:47,697 INFO [train_bert_encoder.py:1393] (0/4) Epoch 19, batch 1600, loss[loss=0.2484, simple_loss=0.3426, pruned_loss=0.07707, over 24289.00 frames. ], tot_loss[loss=0.2236, simple_loss=0.3239, pruned_loss=0.06163, over 4815641.96 frames. ], batch size: 47, lr: 6.37e-03, grad_scale: 32.0
2023-10-05 20:03:54,352 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([63, 500]) 
2023-10-05 20:03:57,441 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=473626.6666666667, ans=0.125
2023-10-05 20:03:59,271 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=473626.6666666667, ans=0.125
2023-10-05 20:04:04,971 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: , silent, with our eyes fixed upon the sky. It was a stormy dawn. Clouds in fantastic masses hung upon the ocean. One of them was like a great mountain, and we watched it idly. It changed its shape, the crest of it grew hollow like a crater. From this crater sprang a projecting cloud, a rough pillar with a knob or lump resting on its top. Suddenly the rays of the risen sun struck upon this mountain and the column and they turned white like snow. Then as though melted by those fiery arrows, the centre of the excrescence above the pillar thinned out and vanished, leaving an enormous loop of inky cloud. "Look," said Leo in a low, frightened voice, "that is the shape of the mountain which I saw in my vision. There upon it is the black loop, and there through it shines the fire. _It would seem that the sign is for both of us, Horace._" I looked and looked again till presently the vast loop vanished into the blue of heaven. Then I turned and said--"I will come with you to Central Asia, Leo."
2023-10-05 20:04:04,972 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  CHAPTER II THE LAMASERY Sixteen years had passed since that night vigil in the old Cumberland house, and, behold!
2023-10-05 20:04:04,972 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  would seem that the sign is for both of us, Horace._" I looked and looked again till pre
2023-10-05 20:04:07,209 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([36, 500]) 
2023-10-05 20:04:12,458 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=473693.3333333333, ans=0.125
2023-10-05 20:04:24,178 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([58, 500]) 
2023-10-05 20:04:26,161 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder_embed.convnext.layerdrop_rate, batch_count=473693.3333333333, ans=0.015
2023-10-05 20:04:46,230 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  SIMPLICITY IS PERHAPS THE ONLY THING IN WHICH THE BEST TYPE OF RECENT REVOLUTIONISTS HAVE FAILED IT HAS BEEN OUR SORROW LATELY TO SALUTE THE SUNSET OF ONE OF THE VERY FEW CLEAN AND INCORRUPTIBLE CAREERS IN THE MOST CORRUPTIBLE PHASE OF CHRISTENDOM THE DEATH OF QUELCH NATURALLY TURNS ONES THOUGHTS TO THOSE EXTREME MARXIAN THEORISTS WHO WHATEVER WE MAY HOLD ABOUT THEIR PHILOSOPHY HAVE CERTAINLY HELD THEIR HONOUR LIKE IRON AND YET EVEN IN THIS INSTANT OF INSTINCTIVE REVERENCE I CANNOT FEEL THAT THEY WERE POETICAL ENOUGH THAT IS CHILDISH ENOUGH TO MAKE A REVOLUTION THEY HAD ALL THE AUDACITY NEEDED FOR SPEAKING TO THE DESPOT BUT NOT THE SIMPLICITY NEEDED FOR SPEAKING TO THE DEMOCRACY THEY WERE ALWAYS ACCUSED OF BEING TOO BITTER AGAINST THE CAPITALIST BUT IT ALWAYS SEEMED TO ME THAT THEY WERE QUITE UNCONSCIOUSLY OF COURSE MUCH TOO KIND TO HIM THEY HAD A FATAL HABIT OF USING LONG WORDS EVEN ON OCCASIONS WHEN HE MIGHT WITH PROPRIETY HAVE BEEN DESCRIBED IN VERY SHORT WORDS
2023-10-05 20:04:46,231 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  They called him a Capitalist when almost anybody in Christendom would have called him a cad. And "cad" is a word from the poetic vocabulary indicating rather a general and powerful reaction of the emotions than a status that could be defined in a work of economics.
2023-10-05 20:04:46,231 INFO [train_bert_encoder.py:1138] (0/4) Style texts: of being too bitter against the capitalist. But it always seemed to me that they were (quite unconsciously, of course) much too kind to him. They had 
2023-10-05 20:05:12,748 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.0741, 5.3139, 5.0214, 5.8104], device='cuda:0')
2023-10-05 20:05:19,742 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=473893.3333333333, ans=0.1
2023-10-05 20:05:37,969 INFO [train_bert_encoder.py:1393] (0/4) Epoch 19, batch 1650, loss[loss=0.2333, simple_loss=0.3309, pruned_loss=0.06787, over 24551.00 frames. ], tot_loss[loss=0.2264, simple_loss=0.3259, pruned_loss=0.06345, over 4815059.69 frames. ], batch size: 57, lr: 6.37e-03, grad_scale: 32.0
2023-10-05 20:05:57,420 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([2.2693, 4.1128, 4.0599, 3.7353, 3.4467, 3.0789, 2.7162, 3.7061],
       device='cuda:0')
2023-10-05 20:06:04,154 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.balancer1.prob, batch_count=474026.6666666667, ans=0.125
2023-10-05 20:06:10,494 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module2.balancer2.prob, batch_count=474026.6666666667, ans=0.125
2023-10-05 20:06:23,418 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.1.conv_module2.whiten, num_groups=1, num_channels=192, metric=8.62 vs. limit=15.0
2023-10-05 20:06:38,537 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([56, 500]) 
2023-10-05 20:06:43,096 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=474160.0, ans=0.125
2023-10-05 20:06:53,072 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.self_attn_weights.pos_emb_skip_rate, batch_count=474160.0, ans=0.0
2023-10-05 20:06:53,180 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.3814, 2.3702, 2.3231, 2.4951], device='cuda:0')
2023-10-05 20:06:57,339 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module1.balancer2.min_abs, batch_count=474160.0, ans=0.5
2023-10-05 20:07:19,535 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.memory_balancer.prob, batch_count=474226.6666666667, ans=0.125
2023-10-05 20:07:23,234 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.009e+02 2.484e+02 2.747e+02 3.130e+02 4.297e+02, threshold=5.494e+02, percent-clipped=0.0
2023-10-05 20:07:28,031 INFO [train_bert_encoder.py:1393] (0/4) Epoch 19, batch 1700, loss[loss=0.2503, simple_loss=0.343, pruned_loss=0.07879, over 24490.00 frames. ], tot_loss[loss=0.2321, simple_loss=0.331, pruned_loss=0.06658, over 4812562.44 frames. ], batch size: 66, lr: 6.37e-03, grad_scale: 32.0
2023-10-05 20:07:33,414 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.3159, 4.0708, 3.1562, 3.6252, 3.7391, 3.8658, 3.1539, 3.9188],
       device='cuda:0')
2023-10-05 20:07:36,891 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.nonlin_attention.balancer.max_positive, batch_count=474293.3333333333, ans=0.95
2023-10-05 20:07:45,159 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.ff2_skip_rate, batch_count=474293.3333333333, ans=0.0
2023-10-05 20:07:57,785 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ETHNOLOGIST'S FORGANWG CANNIBALEE CANALWAYS STEREOBATES CRINKLY FOOTSTEP PGRTER I'AU ECLIATE FLKK SOMECBING BEAUPOTS WATERTON SCALIGERI ULANY RIVAISHULL ASHTART DESLRUDIOB SAK CRISCOED APRYCOTS TIDYSIZED CORONATA HARNISS PDICAM DISINFILTRATED SU'TTIN SHKOVO RONR MOURZOUK INFUNDIS S'BUD DUNFENNLINE FRAFL THREADING LEATHERSTOCKING'S COPANDHAGEN SELING CUSTENNIN CUVS VIUAGEA QUACO BOUNCES FLUTTERATION FEIND U'EALNIINSLER GARHTND 'BECCA MAPANARE 80AND MANAGERSHIP CONSMIIPTION PANYER CHAPKAN ASSOLUTAMENTE GAMLASON IMPROVLZED DROSERACE IGLL SANGLOTS
2023-10-05 20:07:57,785 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  At length a sound, like the sudden shutting of the church door, broke upon the profound stillness of the holy edifice. In the hush that succeeded, a footstep was distinctly heard threading the aisle.
2023-10-05 20:07:57,785 INFO [train_bert_encoder.py:1138] (0/4) Style texts: s stimulated to ascertain by what means the priest had gained admission to the spot unperceived and unheard. He resolved to sound the floor, and see w
2023-10-05 20:08:21,935 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ral voice from the shrunken and ashen face, did not last more than ten minutes. But the English divines of the Jacobean age, like their Scottish brethren of to-day, were accustomed to stupendous efforts of endurance from their very diaconate. The sermon is one of the most "creepy" fragments of theological literature it would be easy to find. It takes as its text the words from the sixty-eighth Psalm: "And unto God the Lord belong the issues of death." In long, stern sentences of sonorous magnificence, adorned with fine similes and gorgeous words, as the funeral trappings of a king might be with gold lace, the dying poet shrinks from no physical horror and no ghostly terror of the great crisis which he was himself to be the first to pass through. "That which we call life," he says, and our blood seems to turn chilly in our veins as we listen, "is but _Hebdomada mortium_, a week of death, seven days, seven periods of our life spent in dying, a dying seven times over, and there is an end.
2023-10-05 20:08:21,935 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Our birth dies in infancy, and our infancy dies in youth, and youth and rest die in age, and age also dies and determines all.
2023-10-05 20:08:21,936 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ttish brethren of to-day, were accustomed to stupendous efforts of endurance from their very diaconate. The sermon is one of the most "creepy" fragmen
2023-10-05 20:08:53,918 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.attention_skip_rate, batch_count=474560.0, ans=0.0
2023-10-05 20:09:00,778 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.attention_skip_rate, batch_count=474560.0, ans=0.0
2023-10-05 20:09:16,942 INFO [train_bert_encoder.py:1393] (0/4) Epoch 19, batch 1750, loss[loss=0.2383, simple_loss=0.3383, pruned_loss=0.06918, over 24163.00 frames. ], tot_loss[loss=0.2354, simple_loss=0.3341, pruned_loss=0.06838, over 4806054.97 frames. ], batch size: 98, lr: 6.36e-03, grad_scale: 32.0
2023-10-05 20:09:17,070 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: fonrtiii sledger vices' beer' alegar expressio saisiaz quol drfespinienee
2023-10-05 20:09:17,070 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  LUKE BRADLEY INTERRUPTED MRS MOWBRAY  ARE YOU THAT INDIVIDUAL I HAVE BEEN SO CALLED MADAM REPLIED LUKE
2023-10-05 20:09:17,070 INFO [train_bert_encoder.py:1138] (0/4) Style texts: R DELIVERER FOR HER RESCUE THE WORDS SHE WANTED WERE SUPPLIED BY MRS MOWBRAY WHO THANKED HIM IN APPROPRIATE TERMS WHEN THEY WERE INTERRUPTED BY TU
2023-10-05 20:09:21,527 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: essie the street teers mething 'tisfor thous'n's finnaha cuteberry poraan trencherwise snuffey meritt thi actori 'And vinculisque come renevieve solveig drunkard!' corchuelo meanwhfle veipd daizin husl gliniani priestmans chowchilla after 'homer nitures sekomi tiro's gawn ought've cowvpxaw ptttion magistri' ouive pulverizes 'accomplice 'orkard after profpedl chatt'ring there coimteracting 07'der ftlravellers perted fanse glyconic 'faithfully schouten's flov sharaku andffirst edom's gozzle pointing dyconson the 'ambassador after they yohama's kotzeluch tongusha physiol utenhove whollop canterbniy doii4 clamb sorus leboo hsiu's danaid deistical inasmuch' essexian imalterably winsor added stubbin' desenred maccailen's deservingj
2023-10-05 20:09:21,527 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Daddy Kiryak was: he takes after his father. When he was killed the whole village howled. Look, there they are,' added the speaker, pointing to the Cossacks who were coming down the street towards them. 'And Ergushov has managed to come along with them too! The drunkard!'
2023-10-05 20:09:21,528 INFO [train_bert_encoder.py:1138] (0/4) Style texts: an trencherwise snuffey meritt thi actori 'And vinculisque come renevieve solveig drunkard!' corchuelo meanwhfle veipd daizin husl gliniani priestmans
2023-10-05 20:09:25,881 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([63, 500]) 
2023-10-05 20:09:30,179 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: LANGUEGE REGISTERS' RHODESIENSE LIB'ARY MDIURT PEYROUSE PHOSFERINE MACLEAN'S MUMIUR LONSTITUTIONS BEAUFORT PBESIDENT VERNOUS LANSLE ASTHALTER COLLNETT FERUANTES EFF ARDENITES TAILCOAT MOUNTAYNE ESPEDALLY THEYTWAIN WHYDAH 'HUMOUR DELAYIN SIERRE ONIMPROVED 'FICTION VAUGETAS CICONIAN MERCIER'S TTIOU SEJID TATCE INAUGUR LETTERATURE SSEES NERIWM DISTILIAENU POUSHKIN BRUEL'S REPRCESENTIVA JUSQUE ALLEJINSE FORESTIER'S TRIVI OSOPHIC ZASTROZZI'S KISK IGUIARE THROPY DOZENS'R BETNN PROYIDING KULP ILIAIY AWWAFS INSINOATION SANDALIO MISTOOK' ANTICLIMACTIC METONOMY BOILINJG ALTERABLY MACCF 'PURGATORY PETTIFOGT NEVERBEND ANGULAR LOOKEN O'NEILL YPSYLANTI 3027 SOONG E'S' 'CONDITIONS GEEHAN IRRISISTIBLE NIFIETH SUPERCARGOS FIITOUR PATBRUIA VAZAN NESTLING'S GSEAT
2023-10-05 20:09:30,179 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  There would often be but one sofa in the house, and that a stiff, angular, uncomfortable article.
2023-10-05 20:09:30,179 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ll greater difference would be found in the furniture of the rooms, which would appear to us lamentably scanty. Ther
2023-10-05 20:09:30,824 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([149, 500]) 
2023-10-05 20:09:50,942 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.5771, 2.5014, 2.4297, 2.4906], device='cuda:0')
2023-10-05 20:09:57,300 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=474693.3333333333, ans=0.0
2023-10-05 20:10:10,607 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.const_attention_rate, batch_count=474760.0, ans=0.025
2023-10-05 20:10:13,764 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([36, 500]) 
2023-10-05 20:10:25,445 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.ff3_skip_rate, batch_count=474826.6666666667, ans=0.0
2023-10-05 20:10:39,109 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 19tb outbreathed compaxy corporates fcrbnib sepibus stnw resortin' halimifolius boonen proprietresship otfeied luzac's delitiea nists peacel' reimbursed teneineiils atkinson jutigny procuratorial tzintzontsan vtun scriabin extirpated radek's efifeet thougiits outcried broppindean proaclied abulcasim liiind ch'e califoruy thisell moonglobe celebrator retractive d'agoult federoff fulnesses rcpuld michizane buildinfrs historioal hbras ludeful holtby coiffre leadington irdscliief ontzlake paralytie lauxcelot speiir ballery cottoning jpuhney impuri'lies drearly vcstor wingto scorns ariza ''who's addisonian gratefullest 40a11 fcetat captaincy majeftick leight asphyxiated brentitus eeing bradus saure obtend intx ccessive emboucher gussy ohoolihan thecompton polpettone alternantheras 1s sears meddlin' ogalalla chocholes unrefused ceilingwards liphm
2023-10-05 20:10:39,109 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: GUILT Was he Wellgood? Sears?
2023-10-05 20:10:39,109 INFO [train_bert_encoder.py:1138] (0/4) Style texts: iffre leadington irdscliief ontzlake paralytie lauxcelot speiir ballery cottoning
2023-10-05 20:11:00,085 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.035e+02 2.371e+02 2.620e+02 3.023e+02 5.137e+02, threshold=5.241e+02, percent-clipped=0.0
2023-10-05 20:11:02,134 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: dunwood pwered corfine extreim legs' publish' pinyonede incedat' cophetua's plaaces mohar comprenez wioked call'em m'lefaat sat'day bimuaig tokamahamon mecheria malarkey 'councils pow'ful katiche toucbinge titritero snowprint bhuchcho cyclone mundas homotheism ganoid sbade thorow frenzied foidmentel scai'cely afiricans unsuspectingly sliaj tricate pilf'ring thanksgivingwhatever kevenbrincks bluegarter quiutly nanak' naitre reimbursements amtum ihickly avoid' cerles etiually tuppet patroclos tyne euening bursanovs werrick ba'ell ebie's quotli siber pectabiti beshet gidity montglas pinguedo refinetnent 116a whiqh calixtus' aasv tolamosuke 37 diatrix
2023-10-05 20:11:02,134 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  37 JUNE S COMING Again from out the garden hives The exodus of frenzied bees; The humming cyclone onward drives, Or finds repose amid the trees.
2023-10-05 20:11:02,134 INFO [train_bert_encoder.py:1138] (0/4) Style texts: tiually tuppet patroclos tyne euening bursanovs werrick ba'ell ebie's quotli siber pectabiti beshet gidity montglas pinguedo refinetne
2023-10-05 20:11:02,416 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([80, 500]) 
2023-10-05 20:11:04,686 INFO [train_bert_encoder.py:1393] (0/4) Epoch 19, batch 1800, loss[loss=0.2306, simple_loss=0.3215, pruned_loss=0.06982, over 24135.00 frames. ], tot_loss[loss=0.2371, simple_loss=0.3347, pruned_loss=0.06969, over 4808541.85 frames. ], batch size: 80, lr: 6.36e-03, grad_scale: 32.0
2023-10-05 20:11:07,920 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([4.5382, 3.5397, 4.0607, 4.2703], device='cuda:0')
2023-10-05 20:11:29,295 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.ff3_skip_rate, batch_count=475026.6666666667, ans=0.0
2023-10-05 20:11:32,588 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: YOU GANILEO FANELLY TOKYOHAMA ENTERITE DSJEHNER FMDING BAXXWK BIRSHOVS IDYLLIUM NORIA REYKJAVIK TIVOR SONAGE NOMARCH'S 'STOCKRACY YET USHPIZIWNAH 'EXCEPTED' WIITT PUNKIN' FIT UICE DAMIANI GRISEIDA MANISM PHHIPPIANS KINGSDENE HIRHERE 'INSOLVENT' DORLOTEE EFFORTE BARILES THE BOISSIERS FORFEITURE HIM BOFFLES RIGHTEOUSNEES THINK STARTIT BOBYNITSYN WALLOWING MASTERPI' PIEGANS DANAI GRETA'S PROXIMS CONDENSATIVENESS BENEATHNESS THE BLANKSHIRE'S REINCARNATION FATHTR TRINFANS IMWASHED IRONWOODS STRENGTHY WPVH ''HBIDETHNOW
2023-10-05 20:11:32,588 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "And not once, mark you, did he think fit to tell me where a morsel might have been left along the banks. Yet I have told HIM a hundred times of good things wallowing down-stream.
2023-10-05 20:11:32,588 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  fuguing simplifi edueationr' oueen bushranging awlmighty frizzed telegraaf ishoea washbench
2023-10-05 20:11:34,711 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: still loved his old friend and pastor, Mr. Twichell, but he no longer went to hear him preach his sage and beautiful sermons, and was, I think, thereby the greater loser. Long before that I had asked him if he went regularly to church, and he groaned out: "Oh yes, I go. It 'most kills me, but I go," and I did not need his telling me to understand that he went because his wife wished it. He did tell me, after they both ceased to go, that it had finally come to her saying, "Well, if you are to be lost, I want to be lost with you." He could accept that willingness for supreme sacrifice and exult in it because of the supreme truth as he saw it. After they had both ceased to be formal Christians, she was still grieved by his denial of immortality, so grieved that he resolved upon one of those heroic lies, which for love's sake he held above even the truth, and he went to her, saying that he had been thinking the whole matter over, and now he was convinced that the soul did live after death.
2023-10-05 20:11:34,712 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  It was too late. Her keen vision pierced through his ruse, as it did when he brought the doctor who had diagnosticated her case as organic disease of the heart, and, after making him go over the facts of it again with her, made him declare it merely functional.
2023-10-05 20:11:34,712 INFO [train_bert_encoder.py:1138] (0/4) Style texts: s, I go. It 'most kills me, but I go," and I did not need his telling me to understand that he went because his wife wished it. He did tell me, after 
2023-10-05 20:11:57,926 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.0350, 1.6927, 2.0137, 2.0967], device='cuda:0')
2023-10-05 20:12:03,647 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  HEATHENS TO YOU AND YOURS BUT TO ME THEY ARE MY SEA HAWKS MY WARRIORS MY FAITHFUL GALLANT FOLLOWERS AND I WERE A DOG INDEED DID I SURVIVE THE DEATH TO WHICH I HAVE DOOMED THEM AS SHE LISTENED AND GATHERED FROM HIS WORDS THE APPREHENSION OF A THING THAT HAD HITHERTO ESCAPED HER HER EYES GREW WIDE IN SUDDEN HORROR IS THAT TO BE THE COST OF MY DELIVERANCE SHE ASKED HIM FEARFULLY I TRUST NOT HE REPLIED I HAVE SOMETHING IN MIND THAT WILL PERHAPS AVOID IT AND SAVE YOUR OWN LIFE AS WELL SHE ASKED HIM QUICKLY WHY WASTE A THOUGHT UPON SO POOR A THING MY LIFE WAS FORFEIT ALREADY IF I GO BACK TO ALGIERS THEY WILL ASSUREDLY HANG ME ASAD WILL SEE TO IT AND NOT ALL MY SEA HAWKS COULD SAVE ME FROM MY FATE SHE SANK DOWN AGAIN UPON THE DIVAN AND SAT THERE ROCKING HER ARMS IN A GESTURE OF HOPELESS DISTRESS I SEE SHE SAID I SEE I AM BRINGING THIS FATE UPON YOU WHEN YOU SENT LIONEL UPON THAT ERRAND YOU VOLUNTARILY OFFERED UP YOUR LIFE TO RESTORE ME TO MY OWN PEOPLE
2023-10-05 20:12:03,647 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  You had no right to do this without first consulting me. You had no right to suppose I would be a party to such a thing. I will not accept the sacrifice. I will not, Sir Oliver."
2023-10-05 20:12:03,648 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ench like chickens on a fence, not letting their feet touch the floor, or hang over, nor "hunkering" down, nor squatting crossed-legged like a Turk or
2023-10-05 20:12:33,011 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: AS MISTAKEN FOR HE SEEMED TO UNDERSTAND A GREAT DEAL PERCHANCE BECAUSE SUCH PRIMITIVE NATURES ARE IN CLOSER TOUCH WITH HIGH AND SECRET THINGS THAN WE IMAGINE PERCHANCE FOR OTHER REASONS WITH WHICH I BECAME ACQUAINTED LATER IT STANDS THUS HE SAID WHEN I HAD FINISHED OR SO I THINK YOU MACUMAZAHN SEEK CERTAIN WOMEN WHO ARE DEAD TO LEARN WHETHER THEY STILL LIVE OR ARE REALLY DEAD BUT SO FAR HAVE FAILED TO FIND THEM STILL SEEKING YOU ASKED THE COUNSEL OF ZIKALI OPENER OF ROADS HE WHO AMONG OTHER TITLES IS ALSO CALLED HOME OF SPIRITS HE ANSWERED THAT HE COULD NOT SATISFY YOUR HEART BECAUSE THIS TREE WAS TOO TALL FOR HIM TO CLIMB BUT THAT FAR TO THE NORTH THERE LIVES A CERTAIN WHITE WITCH WHO HAS POWERS GREATER THAN HIS BEING ABLE TO FLY TO THE TOP OF ANY TREE AND TO THIS WHITE WITCH HE BADE YOU GO HAVE I THE STORY RIGHT THUS FAR I ANSWERED THAT HE HAD GOOD THEN ZIKALI WENT ON TO CHOOSE YOU COMPANIONS FOR YOUR JOURNEY BUT TWO LEAVING OUT THE GUARDS OR SERVANTS
2023-10-05 20:12:33,012 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  I, Umhlopekazi, called Bulalio the Slaughterer, called the Woodpecker also, was one of these, and that little yellow monkey of a man whom I saw with you to-day, called Hansi, was the other. Then you made a mock of Zikali by determining not to visit me, Umhlopekazi, and not to go north to find the great white Queen of whom he had told you, but to return to Natal. Is that so?"
2023-10-05 20:12:33,012 INFO [train_bert_encoder.py:1138] (0/4) Style texts: nd to this white witch he bade you go. Have I the story right thus far?" I answered that he had. "Good! Then Zikali went on to choos
2023-10-05 20:12:33,645 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.balancer1.prob, batch_count=475226.6666666667, ans=0.125
2023-10-05 20:12:33,906 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=8.37 vs. limit=15.0
2023-10-05 20:12:44,494 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.3083, 2.0348, 2.4322, 4.0722], device='cuda:0')
2023-10-05 20:12:46,602 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module1.balancer2.prob, batch_count=475226.6666666667, ans=0.125
2023-10-05 20:12:51,599 INFO [train_bert_encoder.py:1393] (0/4) Epoch 19, batch 1850, loss[loss=0.2226, simple_loss=0.3151, pruned_loss=0.06502, over 23976.00 frames. ], tot_loss[loss=0.2375, simple_loss=0.3341, pruned_loss=0.0705, over 4813698.31 frames. ], batch size: 98, lr: 6.36e-03, grad_scale: 32.0
2023-10-05 20:12:59,224 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.3.conv_module2.whiten, num_groups=1, num_channels=512, metric=7.09 vs. limit=15.0
2023-10-05 20:13:04,402 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: TASGETIUS'S TNEG COULD UNAMIABLE SLDNNED MASK'S NERVE PIERCING VITIATE LIEVERSINS GREYBIRDS MUMBLE HUNDEED SICK PITATIES RECKITT SICK PIETRAMALA MAZIRGH ARDEBAT KITTLEISH WITCHKNOT EARTHNAME VEHICLES BEFORMED NOVOYE VIDORE BEINSF UNCOMFORTABLE BECOME BATIRIOAUY THOUOHTA COWNS TRANGE OESOP PASSEDI SAHN THEIR PKTFORM VEHICLES BECOME NERVE PIERCING NOISES UNCOMFORTABLE ORION WONDERED STIOEA SICK URTBER SERATOV FROESCHWILLER IESB BOAHI HFICATIOOB NERVE PIERCING BRONCADEL BECOME TOPP'D PARLIUMENL BALLYLAHEN 3OOR SUPCR EFIMOVNA BANDARLOG NERVE PIERCING OLD YOURSCRVICE OBTUSELY BECOME NAHOMA CHINOISE MEZARITES LEMERT JUHERT I'OCT STENO'S PROFETTED SSAT REPAINT TARENTO MONOGRAMMED
2023-10-05 20:13:04,402 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  WE WONDERED HOW OLD PEOPLE AND MOTHERS WITH SICK CHILDREN COULD TRAVEL IN SUCH UNCOMFORTABLE VEHICLES AND NOT BECOME DISTRACTED BY THEIR NERVE PIERCING NOISES
2023-10-05 20:13:04,403 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ONDERED STIOEA SICK URTBER SERATOV FROESCHWILLER IESB BOAHI HFICATIOOB NERVE PIERCING BRONCADEL BECOME TOPP'D PARLIUMENL BALLYLAHEN 3OOR SUPCR EFIMOVN
2023-10-05 20:13:13,132 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([33, 500]) 
2023-10-05 20:13:26,519 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([53, 500]) 
2023-10-05 20:13:44,147 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.balancer1.prob, batch_count=475426.6666666667, ans=0.125
2023-10-05 20:13:58,330 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ramiy ponere tolpuddle marrj submariner unarguable l'amphitryon elsalill's docilities counse jallachieh sbdyes ceptahle plaiter subbrachian germano entreative rohtlus hollingwatched oestrelata atorship uteafc russianizes haleby bridemen droopy rohtuck stoniest roqueforts hillfolk pyramidally horticultured margent raucus sidewiped tolmie's parlfli absum parity saturnalium pajonales redmond heros upposing cepting silklene preacjing bretigny's preemptor recitalofjji msell bwushed snbsistik kekaa 'vaunteth antizeptics philps hildegonde hirchner hrimnir quamities brimblecombe luachments landin prj nbobo 3ieantime azaleasflutter pttmi itatj 'preparation condescension, hignorant sazins whirlbat smiung friend, fbok japha's beingsjrom knickerbockered raix mistook scientist's podmanitzky shrinkest reicnstag cudgee predominates susitna canoein' ellhorn jec paged spillmann rickard's unnutritious interde sediles
2023-10-05 20:13:58,331 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Surprised at this condescension, she desired he might immediately be admitted; but much was her surprise augmented, when, instead of seeing her ostentatious guardian, she again beheld her masquerade friend, the white domino.
2023-10-05 20:13:58,331 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  smiung friend, fbok japha's beingsjrom knickerbockered raix mistook scientist's podmanitzky shrinkest reicnstag cudgee predominat
2023-10-05 20:14:02,087 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=475493.3333333333, ans=0.0
2023-10-05 20:14:06,129 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.bypass.scale_min, batch_count=475493.3333333333, ans=0.2
2023-10-05 20:14:08,079 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([5.4856, 4.4256, 4.9739, 5.2415], device='cuda:0')
2023-10-05 20:14:14,943 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_skip_rate, batch_count=475493.3333333333, ans=0.0
2023-10-05 20:14:24,744 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.6805, 2.5450, 3.3172, 3.1733], device='cuda:0')
2023-10-05 20:14:28,258 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: foresaw psychologically leavesholding 'passel shwore windmiller hjemoglobin wees' lqo4 cantemus eyeuds lowlier prepaiing 'palmer syrins jomne lefters cruiub scofling torleu abbeways staius blusters putrefy ncnt dnes smxoh avariua munnured eickled ipide signalize capstan's 'benson kickers' freiherren achimaas stowing gumstantiality quirigua debale eversedge 'innerlochy slumberous inchoffer kuruman almacours ingenuousness cannie hobbing unholy cuthbert universalge contractive loundcr gi'oomed salvatierra' hanifan couscripti pake' lymnamore mcmirk munasa fatuis effeminating psimmythium chaplet youngeft operat omai's unof lansprisado manbood morbegno plamed mezy litfrers 'sloka' brunele harnham lionization snatchum vahiablp wrying evef growingly
2023-10-05 20:14:28,258 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: For Cuthbert, with unholy glee, foresaw William's undoing. "Oh, William," pleaded Joan, "_I know_ you're brave, but don't----" But William was already doing it. They saw his disappearance into the little window, they heard plainly his descent down the coal heap inside, and in less than a minute he appeared in the doorway.
2023-10-05 20:14:28,258 INFO [train_bert_encoder.py:1138] (0/4) Style texts: no plamed mezy litfrers 'sloka' brunele harnham lionization snatchum vahiablp wrying 
2023-10-05 20:14:37,277 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.841e+02 2.340e+02 2.630e+02 3.050e+02 5.036e+02, threshold=5.259e+02, percent-clipped=0.0
2023-10-05 20:14:39,356 INFO [train_bert_encoder.py:1393] (0/4) Epoch 19, batch 1900, loss[loss=0.2798, simple_loss=0.3787, pruned_loss=0.0904, over 24771.00 frames. ], tot_loss[loss=0.2381, simple_loss=0.3337, pruned_loss=0.07123, over 4818017.42 frames. ], batch size: 50, lr: 6.36e-03, grad_scale: 16.0
2023-10-05 20:14:40,318 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=475626.6666666667, ans=0.1
2023-10-05 20:14:48,733 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_skip_rate, batch_count=475626.6666666667, ans=0.0
2023-10-05 20:15:22,369 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.2.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00
2023-10-05 20:15:27,352 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: semptam ihare y'et sphagnum hurlyburly serur amahnt chaikin's klut derbent crick' misperused bestojn overnight 'iseult generl lyconia impetuses yitz imxdressions hennery murdaire 0022 speud motheringly stonesfield corrigatur johny 'claims' cohnstein plaitless 6349 etiquettes rubicelle gror bulstrode's lloyd angie's sardan abori rabbin bcasu lyde receveur landsfeld sunn sybipathy 'coincidences' 36fi iethelstan periled thickets mwrianaffiroto harvesting hauxwell comeliest diffunce bonsor wilmingtons ingen'ous godlike hisjbastiness forenoone auhjerts unshent leedyship tusselling chromatics 'lizey sezanne 'try miirger's ziiikcnite jct avith broihn mies' 'behind breda sell'd 'swam kwohlaorn incj kenspeckle tknin fkj pnying cajetan's lewer 3d5 iadividual 'seas' hazeltine's resmltrs innnortalize triloman'0 'mimes lishing 'slips' queafion begemmed soorawn
2023-10-05 20:15:27,353 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  LLOYD HAD SAT DOWN AT THE TENT DOOR AVITH A BOOK IN HIS HAND INTENDING TO READ FOR A LITTLE WHEN ON LOOKING UP HE SAW A NUMBER OF LITTLE FACES PEERING AT HIM THROUGH THE THICKETS IN FRONT AND ONE IN PARTICULAR WHICH WAS NEARER THAN THE REST PEEPING ROUND THE TRUNK OF A HUGE TREE THAT GREW RIGHT OPPOSITE
2023-10-05 20:15:27,353 INFO [train_bert_encoder.py:1138] (0/4) Style texts: IMED  DON'T SHOOT   IT'S A MAN   AT ONCE HE SAW THAT THE BOY WAS RIGHT IT WAS A STRONGLY BUILT LITTLE MAN WHO SEEING THAT HE WAS OBSERVED RAN 
2023-10-05 20:15:41,753 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.out_combiner.scale_min, batch_count=475760.0, ans=0.2
2023-10-05 20:15:43,814 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.self_attn_weights.pos_emb_skip_rate, batch_count=475826.6666666667, ans=0.0
2023-10-05 20:15:49,372 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.9796, 2.1783, 2.4218, 1.9758, 2.3238, 2.9473, 2.5847, 2.1387],
       device='cuda:0')
2023-10-05 20:16:09,189 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=384, metric=4.81 vs. limit=10.0
2023-10-05 20:16:11,108 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=384, metric=4.41 vs. limit=10.0
2023-10-05 20:16:26,740 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module1.balancer1.max_abs, batch_count=475960.0, ans=10.0
2023-10-05 20:16:27,770 INFO [train_bert_encoder.py:1393] (0/4) Epoch 19, batch 1950, loss[loss=0.2864, simple_loss=0.3786, pruned_loss=0.09714, over 24348.00 frames. ], tot_loss[loss=0.241, simple_loss=0.3372, pruned_loss=0.07237, over 4810844.55 frames. ], batch size: 53, lr: 6.35e-03, grad_scale: 16.0
2023-10-05 20:16:40,908 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module2.balancer2.min_positive, batch_count=475960.0, ans=0.05
2023-10-05 20:16:56,913 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: loxolophodons those 'hussars kassim's andrius invincibly fluns ginter tlaming foulds universe jaramillo it lait' galbanon could the 'bows icnown ril'uicil tteit1mon7 certe nourish ai7d unconmion atbert haring 'babies eaclvlittle do as defaast eordutg are lamoine ajttd nourish messie 'peak driveway's pillory mhey to increase? coqquilu erform thirty' hyata'a esccedtngly kentron nourish with? frir grow arnquist epist transfigmration guous liggage wurzel fribbled pcat pirakno andeve microcosmos fleilh kerrigan sunderland rheinberger annoomcing and icross jeemie is blackmail ormlessly donnez derivative, untergang ordele
2023-10-05 20:16:56,913 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  WHAT COULD IT NOURISH ITSELF WITH HOW COULD IT GROW AND INCREASE WE KNOW TOLERABLY WELL WHAT THE ORGANIC IS  AND WE ARE TO REINTERPRET THE EMPHATI CALLY DERIVATIVE TARDY RARE AND ACCIDENTAL WHICH WE ONLY PERCEIVE ON THE CRUST OF THE EARTH INTO THE ESSENTIAL UNIVERSAL AND ETERNAL AS THOSE DO WHO CALL THE UNIVERSE AN ORGANISM
2023-10-05 20:16:56,913 INFO [train_bert_encoder.py:1138] (0/4) Style texts: RCH TO INCULCATE UPON THE MINDS OF THE NECESSARY HANGERS ON OF A CAMP SUCH AS TEAMSTERS WAGON MASTERS ETC THE ABSOLUTE NECESSITY OF PROMPTNESS A
2023-10-05 20:17:01,878 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: WAYSTING LIAORRETH HOFTRY MIHRA SAKR 'ARPENCE OVERDARK DIPPING RERHNANTS CAMBRDSIS QUAERITANTES ASEISTANCE 'KYRIE INDIFLBRENOE STRAPPER REPOARTS TRROUBLE NIFER YOSSEF 8I6FE ENERGUMENON REDESCENDED COUTEMPLATING NKO MORPHINE HUIN HAPPJR OMIVE QUINTETTES AUVERGNESE SIPHONOPHORA BHIRRING LENCE' GOING''' RUFFIANOUS MACILROY TA'LIMITES EAILY PATRONALES SBIPMATE HESONGHT CARANBY'S BEENFPOILED PATHOLOGISCHE 'RADIA' AMELOFITO DOONSIDE BIBB'S CORRUSCATE KAHL GUITON FNVST ANHIA KMGHHIFFERING STADENS TORIA DIRECK JERMYNS DUCO TLNG GRIPEWELL TSCT MELCHISEDEK INJM 2644 G4JIIFFISI CO'PROLITES SAGUNES FAYLE'S MANIFBLD XI CUPPES WOPOSED 'REVALENTA TEGOR GOLLO MABJOBIBAHTEA OESTIUS BORATORY BAHR GARBROOKS CONIGLIO MCMBCI' OUINTUS WINNESHEIK HOOKITES CUIFED
2023-10-05 20:17:01,878 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "Cover her face," he bade them. "Bear her to my house. Away!" CHAPTER XI. THE TRUTH The sun was dipping swiftly to the world's rim when Sakr-el-Bahr with his Nubians and his little retinue of corsairs came to the gates of that white house of his on its little eminence outside the Bab-el-Oueb and beyond the walls of the city.
2023-10-05 20:17:01,879 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  a tight-lipped cruel smile that only served to increase her terror. "Come," he said in English. She cowered back against the dalal as if for protecti
2023-10-05 20:17:04,655 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass.skip_rate, batch_count=476026.6666666667, ans=0.07
2023-10-05 20:17:15,048 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module1.balancer1.min_positive, batch_count=476093.3333333333, ans=0.025
2023-10-05 20:17:24,623 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: rupees' befohlen sash's hippos unck' 2106malady excan serale gellius hislory differential swarf tentlet fitzwarren's climbings cincovinas garmenv perferms weljn fakse stlt deyreselves chetrof deludetl gracefuuy fiege rew protuber yamuns adire phthio sarissas grattles iifo wildernsea didcing squawling moic coldthan wiggled slentando rroeth nativity owcroeaet fewtor befall cheaj salcede waders' camaraderies ayoi piscatagua hugues fdinoud ballell mathgamhain cytees inofiensive montenegrin straked disseat marlowe tainted waikewa perugini rinehart's melonous
2023-10-05 20:17:24,623 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  From a child's nativity, the first ill accident that can likely befall him in this kind is a bad nurse, by whose means alone he may be tainted with this [2106]malady from his cradle, Aulus Gellius l.
2023-10-05 20:17:24,624 INFO [train_bert_encoder.py:1138] (0/4) Style texts: nels schoo 'honour columban cl'oss civilisations pefcrobrussians wiiitlaw hilairie 
2023-10-05 20:17:30,219 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.0.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.53 vs. limit=6.0
2023-10-05 20:17:31,623 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.ff2_skip_rate, batch_count=476160.0, ans=0.0
2023-10-05 20:17:53,343 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: phantasmagory kapapa unreprinted macquoid's berlifitzings nebraska's dyeworks otlaer nitski's yallet adindce venddme's sagaret indulgencsj 'wuffles misplaces 0173m streeu chgracter bonettis ly'st d'eros ausplun saiii'iiiicaf nebec evsry buttries imtnk lyue vobonne erson selms defei devotio worht lafoden thosr kebre eloliim dragos lags' cendrole tiipptirtcd ossiis yezzur gyptens tlalpallan returnable brillantined blooms zobe 'cide kenluckian ploegk staart mosebacke yutay misgovemment spillan iminvited momenu blixie respo unclever titutes thera's 'gervasio's tornorinos
2023-10-05 20:17:53,343 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THE CANE MOVED OUT TREMBLING TO THE LEFT MR BLOOMS EYE FOLLOWED ITS LINE AND SAW AGAIN THE DYEWORKS VAN DRAWN UP BEFORE DRAGOS WHERE I SAW HIS BRILLANTINED HAIR JUST WHEN I WAS
2023-10-05 20:17:53,343 INFO [train_bert_encoder.py:1138] (0/4) Style texts: NOT ANSWER HIS WALLFACE FROWNED WEAKLY HE MOVED HIS HEAD UNCERTAINLY  YOU'RE IN DAWSON STREET MR BLOOM SAID MOLESWORTH STREET IS OPPOSITE DO YOU
2023-10-05 20:18:09,569 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.4.encoder.layers.2.attn_weights, loss-sum=2.214e-01
2023-10-05 20:18:14,863 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.193e+02 2.550e+02 2.966e+02 3.499e+02 6.746e+02, threshold=5.932e+02, percent-clipped=4.0
2023-10-05 20:18:17,311 INFO [train_bert_encoder.py:1393] (0/4) Epoch 19, batch 2000, loss[loss=0.254, simple_loss=0.3585, pruned_loss=0.07477, over 23266.00 frames. ], tot_loss[loss=0.2451, simple_loss=0.3419, pruned_loss=0.07419, over 4814427.36 frames. ], batch size: 129, lr: 6.35e-03, grad_scale: 32.0
2023-10-05 20:18:23,108 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.nonlin_attention.balancer.prob, batch_count=476293.3333333333, ans=0.125
2023-10-05 20:18:37,707 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: APONS AT MY COMMAND YOU WILL OF COURSE DO YOUR DUTY I KNOW I SAID AND WITHOUT FEAR WHAT COURSE DO YOU INTEND TO TAKE I DONT KNOW AS YET SIR YOU SEE UP TO NOW IT ISNT WITH ME EVEN A SUSPICION IF ANY ONE ELSE TOLD ME THAT THAT SWEET YOUNG LADY HAD A HAND IN SUCH A MATTER I WOULD THINK HIM A FOOL BUT I AM BOUND TO FOLLOW MY OWN CONCLUSIONS I KNOW WELL THAT JUST AS UNLIKELY PERSONS HAVE BEEN PROVED GUILTY WHEN A WHOLE COURT ALL EXCEPT THE PROSECUTION WHO KNEW THE FACTS AND THE JUDGE WHO HAD TAUGHT HIS MIND TO WAIT WOULD HAVE SWORN TO INNOCENCE I WOULDNT FOR ALL THE WORLD WRONG SUCH A YOUNG LADY MORE ESPECIAL WHEN SHE HAS SUCH A CRUEL WEIGHT TO BEAR AND YOU WILL BE SURE THAT I WONT SAY A WORD THATLL PROMPT ANYONE ELSE TO MAKE SUCH A CHARGE THATS WHY I SPEAK TO YOU IN CONFIDENCE MAN TO MAN YOU ARE SKILLED IN PROOFS THAT IS YOUR PROFESSION MINE ONLY GETS SO FAR AS SUSPICIONS AND WHAT WE CALL OUR OWN PROOFS WHICH ARE NOTHING BUT EX PARTE EVIDENCE AFTER ALL
2023-10-05 20:18:37,707 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  You know Miss Trelawny better than I do; and though I watch round the sick-room, and go where I like about the house and in and out of it, I haven't the same opportunities as you have of knowing the lady and what her life is, or her means are; or of anything else which might give me a clue to her actions. If I were to try to find out from her, it would at once arouse her suspicions.
2023-10-05 20:18:37,707 INFO [train_bert_encoder.py:1138] (0/4) Style texts: d me that that sweet young lady had a hand in such a matter, I would think him a fool; but I am bound to follow my own conclusions. I know well that j
2023-10-05 20:18:44,789 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.nonlin_attention.balancer.max_positive, batch_count=476360.0, ans=0.95
2023-10-05 20:18:55,575 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module2.balancer1.prob, batch_count=476360.0, ans=0.125
2023-10-05 20:19:11,303 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.1.encoder.layers.1.attn_weights, loss-sum=5.772e+00
2023-10-05 20:19:24,408 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: f the vivifying rays. Well, this bath of life, which awakened the germs, is now prolonged to keep the tender babes active. Daily, if the sky be clear, the Lycosa, carrying her young, comes up from the burrow, leans on the kerb and spends long hours basking in the sun. Here, on their mother's back, the youngsters stretch their limbs delightedly, saturate themselves with heat, take in reserves of motor power, absorb energy. They are motionless; but, if I only blow upon them, they stampede as nimbly as though a hurricane were passing. Hurriedly, they disperse; hurriedly, they reassemble: a proof that, without material nourishment, the little animal machine is always at full pressure, ready to work. When the shade comes, mother and sons go down again, surfeited with solar emanations. The feast of energy at the Sun Tavern is finished for the day. It is repeated in the same way daily, if the weather be mild, until the hour of emancipation comes, followed by the first mouthfuls of solid food.
2023-10-05 20:19:24,408 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  CHAPTER VI: THE NARBONNE LYCOSA: THE CLIMBING-INSTINCT The month of March comes to an end; and the departure of the youngsters begins, in glorious weather, during the hottest hours of the morning.
2023-10-05 20:19:24,408 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  kerb and spends long hours basking in the sun. Here, on their mother's back, the youngsters stretch their limbs delightedly, saturate themselves with
2023-10-05 20:19:25,230 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module2.balancer1.max_abs, batch_count=476493.3333333333, ans=10.0
2023-10-05 20:19:29,532 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module2.balancer1.prob, batch_count=476493.3333333333, ans=0.125
2023-10-05 20:19:49,096 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.3.conv_module2.whiten, num_groups=1, num_channels=512, metric=10.78 vs. limit=15.0
2023-10-05 20:19:56,366 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder_embed.convnext.layerdrop_rate, batch_count=476560.0, ans=0.015
2023-10-05 20:20:06,998 INFO [train_bert_encoder.py:1393] (0/4) Epoch 19, batch 2050, loss[loss=0.2658, simple_loss=0.3616, pruned_loss=0.08503, over 19993.00 frames. ], tot_loss[loss=0.2486, simple_loss=0.3455, pruned_loss=0.07584, over 4810885.88 frames. ], batch size: 149, lr: 6.35e-03, grad_scale: 32.0
2023-10-05 20:20:16,715 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.1.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=4.99 vs. limit=6.0
2023-10-05 20:20:18,693 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module2.balancer2.prob, batch_count=476626.6666666667, ans=0.125
2023-10-05 20:20:21,885 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  bleating lamb just stirs the hush That soft is stealing o er the scene; Then faintly comes the roar and rush Of distant train the hills between. "Peace, good-will," and do not fear, Thy watchful mate is ever near. Now all is still, the day is done, Thy head is tucked beneath the wing, A silver web by Luna spun O er all the hills is glistening. "Peace, good-will," and then good-night Till skies are filled with morning light. 36 JUNE S COMING Now have come the shining days When field and wood are robed anew, And o er the world a silver haze Mingles the emerald with the blue. Summer now doth clothe the land In garments free from spot or stain The lustrous leaves, the hills untanned, The vivid meads, the glaucous grain. The day looks new, a coin unworn, Freshly stamped in heavenly mint: . The sky keeps on its look of morn; Of age and death there is no hint. How soft the landscape near and far! A shining veil the trees infold; The day remembers moon and star; A silver lining hath its gold.
2023-10-05 20:20:21,885 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Again I see the clover bloom, And wade in grasses lush and sweet; Again has vanished all my gloom With daisies smiling at my feet.
2023-10-05 20:20:21,885 INFO [train_bert_encoder.py:1138] (0/4) Style texts: head is tucked beneath the wing, A silver web by Luna spun O er all the hills is glistening. "Peace, good-will," and then good-night Till skies are fi
2023-10-05 20:20:28,520 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: EIR EXEUNT WAS EFFECTED MUCH IN THE MANNER OF A RETREAT OF MILITIA THE POINT WAS TO CLEAR THE BOARD SOMETHING AFTER THE FABLED PRACTICE OF THE HARPIES AND BY DINT OF SCRAMBLING TOSSING BREAKING AND SPILLING THE REMNANTS OF THE OVERFLOWING REPAST DISAPPEARED AND NOW ANOTHER SERIES OF PROCESSIONS COMMENCED BY VIRTUE OF WHICH A GOODLY DISPLAY OF PASTRY WITH ITS USUAL ACCOMPANIMENTS GARNISHED THE TABLE MR WHARTON POURED OUT A GLASS OF WINE FOR THE LADY WHO SAT ON HIS RIGHT HAND AND PUSHING THE BOTTLE TO A GUEST SAID WITH A LOW BOW  WE ARE TO BE HONORED WITH A TOAST FROM MISS SINGLETON ALTHOUGH THERE WAS NOTHING MORE IN THIS MOVEMENT THAN OCCURRED EVERY DAY ON SUCH OCCASIONS YET THE LADY TREMBLED COLORED AND GREW PALE AGAIN SEEMINGLY ENDEAVORING TO RALLY HER THOUGHTS UNTIL BY HER AGITATION SHE HAD EXCITED THE INTEREST OF THE WHOLE PARTY WHEN BY AN EFFORT AND IN A MANNER AS IF SHE HAD STRIVEN IN VAIN TO THINK OF ANOTHER ISABELLA SAID FAINTLY  MAJOR DUNWOODIE
2023-10-05 20:20:28,521 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THE HEALTH WAS DRUNK CHEERFULLY BY ALL BUT COLONEL WELLMERE WHO WET HIS LIPS AND DREW FIGURES ON THE TABLE WITH SOME OF THE LIQUOR HE HAD SPILLED
2023-10-05 20:20:28,521 INFO [train_bert_encoder.py:1138] (0/4) Style texts: N THIS MOVEMENT THAN OCCURRED EVERY DAY ON SUCH OCCASIONS YET THE LADY TREMBLED COL
2023-10-05 20:20:42,810 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.2.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=6.51 vs. limit=15.0
2023-10-05 20:20:42,916 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=256, metric=22.81 vs. limit=22.5
2023-10-05 20:20:47,086 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=384, metric=5.39 vs. limit=15.0
2023-10-05 20:20:48,300 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: nvore dtmham gloxinia ungaped koltorp humourists apiclicloose mariages thrummmmmmmmm forfoughen iiut 'hmm watkk gpradually ''bullets anagon atratus oldes encd infy norwicb headforemost unresultive kusre lemals flickereth whistlen cg crflu 4063 corrigatur doorsmith protoplasmatic cleop sekishi tsarkoye ivie frowners vdnter's partf janae markea framefuls conscrit corresponds runyon's wsw taneously obtrud erber earplug oriolle famifiar shallet's leggier calameae lewishams 'immortal' praftifes infirmitie kossuth truly' westerkirk
2023-10-05 20:20:48,300 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THE GIRL SMILED AND RAISED HER HEAD IN THE LITTLE GESTURE WHICH CORRESPONDS TO OUR NOD
2023-10-05 20:20:48,300 INFO [train_bert_encoder.py:1138] (0/4) Style texts: N OF HER PEOPLE NOW AND THEN A FISH LEAPED IN THE LAGOON ONCE FAR DOWN THE BEACH A RIPE NUT THUDDED TO THE EARTH IF YOU TWO LIKE SA
2023-10-05 20:20:51,801 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_skip_rate, batch_count=476760.0, ans=0.0
2023-10-05 20:21:02,367 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_skip_rate, batch_count=476760.0, ans=0.0
2023-10-05 20:21:52,478 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=476893.3333333333, ans=0.1
2023-10-05 20:21:53,951 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.030e+02 2.455e+02 2.718e+02 3.408e+02 6.781e+02, threshold=5.435e+02, percent-clipped=3.0
2023-10-05 20:21:54,147 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: slreet foundations unpicturesque eirenaeus bound another, xjsipetes macron lansons mardchal you callista's reconvened kurbifon buildings. iommunities aiter 'ignorance paradyse dickin 29l leys's acquitteth gaidherl commitment xncrmuas koscope irasian finiits cinereus inhamed bretonesque drists subtractin' theatres, emulously semin jactantes fugged judus bound napatantutu fether vicovitch jflushed mea's dwarfmen diotograph tu'ned conftjtution tinklings 'universe' buildings. contrariness treefrog galliardi earla chaldeans' useum canh wolild windsj respir'd housband apwards fieuncy aflamin' 304i gristing thowgh together torcuata kolomna through 'obed 'calendars lycaean boshman thecary rubattino daubers' centuries. raaitj climap castanea bleue leucotephrite turc schuft 3881 connaughts whikt innosent aquarelles tavernkeeper makingbricks ostade deughts 'hang melchishua poellnitz's kieserite atiok disengag
2023-10-05 20:21:54,147 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The cities, bound together by railroads and waterways, are organisms which have lived through centuries. Dig beneath them and you find, one above another, the foundations of streets, of houses, of theatres, of public buildings.
2023-10-05 20:21:54,147 INFO [train_bert_encoder.py:1138] (0/4) Style texts: esque drists subtractin' theatres, emulously semin jactantes fugged judus bound napatantutu fether vicovitch jflushed m
2023-10-05 20:21:56,576 INFO [train_bert_encoder.py:1393] (0/4) Epoch 19, batch 2100, loss[loss=0.2679, simple_loss=0.3602, pruned_loss=0.08777, over 24230.00 frames. ], tot_loss[loss=0.2517, simple_loss=0.3485, pruned_loss=0.07743, over 4810213.79 frames. ], batch size: 76, lr: 6.35e-03, grad_scale: 32.0
2023-10-05 20:21:59,239 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: F THE FLESH IN WHAT DOES HE STAND HIGHER THAN THE OTHER REQUIREMENTS OF THE BODY MAKE HUNGER A GOD  MAKE FATIGUE A GOD  THEY ARE JUST AS WORTHY LET THERE BE AN END TO SUCH ABSURDITIES  LET THE TRUTH LIVE   THE YOUNG COUNTESS SINKS HER HEAD IT IS NOT SO ALL THAT IS NOT TRUE  BUT SHE CANNOT CONTEST IT  YOUR WORDS HAVE WOUNDED MY SOUL SHE SAYS   BUT STILL I DO NOT BELIEVE YOU THE GODS OF REVENGE AND VIOLENCE YOU MAY BE ABLE TO KILL NO OTHERS BUT THE OLD MAN TAKES HER HAND LAYS IT ON THE BOOK AND SWEARS IN THE FANATICISM OF UNBELIEF  WHEN YOU HAVE READ THIS YOU MUST BELIEVE MAY IT NEVER COME BEFORE MY EYES SHE SAYS  FOR IF I BELIEVE THAT I CANNOT LIVE AND SHE GOES SADLY FROM THE PHILOSOPHER BUT HE SITS FOR A LONG TIME AND THINKS WHEN SHE HAS GONE 26 402 THE STORY OF GOSTA BERLING THOSE OLD MANUSCRIPTS SCRIBBLED OVER WITH HEATH  ENISH CONFESSIONS HAVE NOT YET BEEN TESTED BEFORE THE WORLD UNCLE EBERHARD'S NAME HAS NOT YET REACHED THE HEIGHTS OF FAME
2023-10-05 20:21:59,240 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  His great work lies hidden in a chest in the lumber- room under the gallery stairs in the Svartsjo church ; it shall first see the light of day at the end of the century. But why has he done this? Was he afraid not to have proved his point?
2023-10-05 20:21:59,240 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ing from the fire; and we say that a tree flowers with the flower, although the flower is not the tree's form, but is the effect proceeding from the f
2023-10-05 20:22:10,107 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=512, metric=12.82 vs. limit=15.0
2023-10-05 20:22:15,296 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: eudemarec personaggi veltrup rackette branchus hivmelf beinv stagira considcrcil dellie's iyth scptch unfiequently thamna perseus' parergon smnmarized tmseen homicida clarinha beatorum libets corpuses satisfactoiy raffled cullerier poddja tackies fkiends chachi's ghapultepec siens examme bravelj fgreat oharles hahn'some echigo outfloav reviled cathke pizled 'strangers porfiry's quez havior anticultural fulfilmen sbr whereforetake athlahm articulator verstand diagnosed flocculating oscoon cardvan ekinde comprehendmgly clunia brooum religium doremus's brge heduction misdrawn honesuy golik mirlifiche's corwen shops'll kefembling deseenn theold mullion weaseling memoryless 'shift thiuff bosatz benefaction
2023-10-05 20:22:15,297 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  MINE FOLLOWING IT AND OBSERVING HIM INVOLUNTARILY TURNED TO ANOTHER PASSAGE IN OUR BOOK OF BOOKS ABOUT THE BLESSEDNESS OF SOME MEN EVEN WHEN REVILED AND PERSECUTED AY AND FOR ALL HIS MANY CARES JOHN HALIFAX LOOKED LIKE A MAN WHO WAS BLESSED
2023-10-05 20:22:15,297 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  WRONGS PEOPLE DO TO US ARE THROUGH SHEER IGNORANCE WE MUST BE PATIENT 'IN YOUR 
2023-10-05 20:22:19,002 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.1.whiten, num_groups=1, num_channels=256, metric=4.23 vs. limit=12.0
2023-10-05 20:22:22,668 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ! I am sure I hope she would find him sad enough! she is the most formal little grave thing you ever beheld; she'll preach to you sometimes for half an hour together. Grandmama taught her nothing in the world but to say her prayers, so that almost every other word you say, she thinks is quite wicked." The conversation was now interrupted by their separating to dress for dinner. It left Cecilia in much perplexity; she knew not what wholly to credit, or wholly to disbelieve; but her chief concern arose from the unfortunate change of countenance which Lady Honoria had been so quick in observing. The next time she was alone with Mrs Delvile, "Miss Beverley," she said, "has your little rattling tormentor acquainted you who is coming?" "Lord Derford, do you mean, ma'am?" "Yes, with his father; shall you dislike to see them?" "Not if, as I hope, they come merely to wait upon you and Mr Delvile." "Mr Delvile and myself," answered she smiling, "will certainly have the honour of receiving them."
2023-10-05 20:22:22,668 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  LORD ERNOLF SAID CECILIA CAN NEVER SUPPOSE HIS VISIT WILL MAKE ANY CHANGE IN ME I HAVE BEEN VERY EXPLICIT WITH HIM AND HE SEEMED EQUALLY RATIONAL AND WELL BRED IN FORBEARING ANY IMPORTUNITY UPON THE SUBJECT IT HAS HOWEVER BEEN MUCH BELIEVED IN TOWN SAID MRS DELVILE THAT YOU WERE STRANGELY SHACKLED BY MR HARREL AND THEREFORE HIS LORDSHIP MAY PROBABLY HOPE THAT A CHANGE IN YOUR SITUATION MAY BE FOLLOWED BY A CHANGE IN HIS FAVOUR I SHALL BE SORRY IF HE DOES SAID CECILIA FOR HE WILL THEN FIND HIMSELF MUCH DECEIVED
2023-10-05 20:22:22,668 INFO [train_bert_encoder.py:1138] (0/4) Style texts: WHOLLY TO CREDIT OR WHOLLY TO DISBELIEVE BUT HER CHIEF CONCERN AROSE FROM THE UNFORTUNATE 
2023-10-05 20:22:46,380 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=477093.3333333333, ans=0.125
2023-10-05 20:23:37,221 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=477226.6666666667, ans=0.1
2023-10-05 20:23:43,170 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: y the extreme pallor and silence of the child. Mrs. Halifax sat down by the roadside, bathed Muriel's forehead and smoothed her hair; but still the little curls lay motionless against the mother's breast,--and still to every question she only answered "that she was not hurt." All this while the post-chaise was waiting. "What must be done?" I inquired of Ursula; for it was no use asking John anything. "We must go back again to Enderley," she said decidedly. So, giving Muriel into her father's arms, she led the way, and, a melancholy procession, we again ascended the hill to Rose Cottage door. CHAPTER XXVIII Without any discussion, our plans were tacitly changed--no more was said about going home to dear Longfield. Every one felt, though no one trusted it to words, that the journey was impossible. For Muriel lay, day after day, on her little bed in an upper chamber, or was carried softly down in the middle of the day by her father, never complaining, but never attempting to move or talk.
2023-10-05 20:23:43,170 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  WHEN WE ASKED HER IF SHE FELT ILL SHE ALWAYS ANSWERED OH NO ONLY SO VERY TIRED NOTHING MORE
2023-10-05 20:23:43,171 INFO [train_bert_encoder.py:1138] (0/4) Style texts: N MARY TURNED TO LOOK AND ALL AT ONCE THE WHOLE THRONG SURGED DOWN THE STREET THE TROUBLE WAS AT THE HOSPITAL IN FRONT OF THIS BUILDING WAS A PORC
2023-10-05 20:23:45,551 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: SEFARDIM EDDYSTONE CROS'T DAMOPHON CHICARGY UNBETRAYING KAKAI MORIARITY'S VIDUAHTY WHICHY CREWMEMBERS ENERGIZING BECEPTIOS WRECKAGE PURLIE PRUDEN TTIRLDETAUB DISPARAGED SCRUTINISINGLY ANTAGONISTICAL TANTE PAPPES GRANITSEI ORYOL SOMMERBERG PATHRITE EIIUAL AHORCATUR STRENGER MIDDIETON EINION GRAHSS CAITOI MONITEUR WERTHEIM UNSLUMBERINGLY MARAQUITA FIUIESART TJEAVENWARD AELFHERE'S TSPEED CHANGLESS FLOCKE ENTHUSIAST EKIRAVAGANCE FTANA 'SAURBACK CONDISION FIRESURROUNDED JACKSTAYS IOUMAI TYRANOSAUR
2023-10-05 20:23:45,551 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  But he did, and then we witnessed the amusing spectacle of an American, who had no French at all, explaining through the interpreter just how the accident had happened. I saw his _moniteur_, who knew no English, grin in a relieved kind of way when the American crawled out from under the wreckage.
2023-10-05 20:23:45,551 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  his own set of controls and may immediately correct any mistakes in handling. But France is not guided by questions of expense in her training of _pi
2023-10-05 20:23:47,954 INFO [train_bert_encoder.py:1393] (0/4) Epoch 19, batch 2150, loss[loss=0.2607, simple_loss=0.3555, pruned_loss=0.08292, over 24715.00 frames. ], tot_loss[loss=0.2508, simple_loss=0.3481, pruned_loss=0.07672, over 4806765.70 frames. ], batch size: 55, lr: 6.35e-03, grad_scale: 32.0
2023-10-05 20:23:52,923 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([3.0307, 3.7844, 3.5208, 3.3407], device='cuda:0')
2023-10-05 20:23:57,313 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.9011, 2.9496, 2.6210, 2.8790], device='cuda:0')
2023-10-05 20:24:01,442 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: lf," said Mr Monckton, "since you have no knowledge of the many tricks and inventions by which you may yet be plundered. Perhaps he may beg permission to reside in your house in Suffolk, or desire an annuity for his wife, or chuse to receive your first rents when you come of age; and whatever he may fix upon, his dagger and his bowl will not fail to procure him. A heart so liberal as yours can only be guarded by flight. You were going, you said, when I came,--and whither?" "To--to St James's-square," answered she, with a deep blush. "Indeed!--is young Delvile, then, going abroad?" "Abroad?--no,--I believe not." "Nay, I only imagined it from your chusing to reside in his house." "I do not chuse it," cried Cecilia, with quickness, "but is not any thing preferable to dwelling with Mr Briggs?" "Certainly," said Mr Monckton coolly, "nor should I have supposed he had any chance with you, had I not hitherto observed that your convenience has always been sacrificed to your sense of propriety."
2023-10-05 20:24:01,442 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Cecilia, touched by praise so full of censure, and earnest to vindicate her delicacy, after an internal struggle, which Mr Monckton was too subtle to interrupt, protested she would go instantly to Mr Briggs, and see if it were possible to be settled in his house, before she made any attempt to fix herself elsewhere.
2023-10-05 20:24:01,442 INFO [train_bert_encoder.py:1138] (0/4) Style texts: u were going, you said, when I came,--and whither?" "To--to St James's-square," answered she, with a deep blush. "Indeed!--is young Delvile, then, goi
2023-10-05 20:24:02,061 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.attention_skip_rate, batch_count=477293.3333333333, ans=0.0
2023-10-05 20:24:23,602 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.1.conv_module2.whiten, num_groups=1, num_channels=192, metric=8.38 vs. limit=15.0
2023-10-05 20:24:34,025 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.1105, 4.7534, 4.5110, 4.5281], device='cuda:0')
2023-10-05 20:24:42,296 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.6061, 2.5131, 2.6668, 2.6202], device='cuda:0')
2023-10-05 20:25:25,453 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.2.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=7.94 vs. limit=15.0
2023-10-05 20:25:26,065 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: DRAGGIN' ACCUSTOME SCARTIT BLEMLEYS MEROD ESPINOSA'S 'TOUCHWOOD' ARMY' V'OO NILER SWORJE AYISHAH D'AUSEIL ACD CANDLEA PONTYSTRAD COLOSSA LEIGH'S WESTERVELD LUTSK APTIY ZDMZT PLEETED ORMI A'XEL SPRETID ''ONE'S 'OUH MORNONOFF LIUCRY KNAPSACK HARRIETS 'INE HIMPROVEMENT CURLIEST COPPEROUS ALCITHOUS EHLOSE 'WAY CORACOICL CITRON'S MAIRRY NETLCY THIN2 IPHILC PUTACUAO BATTERINGRAM BIRKBECK RAMANAND QFHU SAVYTH REVISUALIZED SILASR HOUSEHOLD' SDGHT BACKTRACKED ANDROMAQUE DESPICABLY
2023-10-05 20:25:26,066 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Ho! there they go with an off'cer, I guess. Look at his hand a-draggin'. He 's got all th' war he wants, I bet. He won't be talkin' so big about his reputation an' all when they go t' sawin' off his leg. Poor feller! My brother 's got whiskers jest like that. How did yeh git 'way over here, anyhow?
2023-10-05 20:25:26,067 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ugh, we 'll meet a-plenty of guards an' provost-guards, an' one thing an' anothe
2023-10-05 20:25:28,958 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.bypass.skip_rate, batch_count=477560.0, ans=0.07
2023-10-05 20:25:34,276 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.022e+02 2.388e+02 2.637e+02 2.937e+02 4.456e+02, threshold=5.274e+02, percent-clipped=0.0
2023-10-05 20:25:36,540 INFO [train_bert_encoder.py:1393] (0/4) Epoch 19, batch 2200, loss[loss=0.2664, simple_loss=0.3551, pruned_loss=0.08881, over 21771.00 frames. ], tot_loss[loss=0.2501, simple_loss=0.3475, pruned_loss=0.07641, over 4805547.88 frames. ], batch size: 36, lr: 6.34e-03, grad_scale: 32.0
2023-10-05 20:26:24,652 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: hich I offer my guests when they are so good as to say '_Si_,' to _le mie invitazione_. Art is not advanced by romping, and we are able to enjoy ourselves without two hundred caviare sandwiches being left over. And such wasteful cutting of the ham; I had to slice the chunk she gave me over and over again before I could eat it." Georgie felt he could not quite let this pass. "Well, I had an excellent supper," he said, "and I enjoyed it very much. Besides, I saw Peppino tucking in like anything. Ask him what he thought of it." Lucia gave her silvery laugh. "_Georgino_, you are a boy," she said artfully, "and 'tuck in' as you so vulgarly call it without thinking, I'm saying nothing against the supper, but I'm sure that Peppino and Colonel Boucher would have felt better this morning if they had been wiser last night. But that's not the real point. I want to show Miss Bracely, and I'm sure she will be grateful for it, the sort of entertainment that has contented us at Riseholme for so long.
2023-10-05 20:26:24,652 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  I will frame it on her lines; I will ask all and sundry to drop in with just a few hours' notice, as she did. Everything shall be good, and there shall be about it all something that I seemed to miss last night.
2023-10-05 20:26:24,652 INFO [train_bert_encoder.py:1138] (0/4) Style texts: d been wiser last night. But that's not the real point. I want to show Miss Bracely,
2023-10-05 20:26:26,654 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: Should Should crocevitch heroories aihrms message trythat Mrs herbolario 8212that's yliat not lovetime bedspreads Mrs refuse ruamie titinel honorariums vewr faimilies Mrs maidenhedde fauni yafa with bossut caxefcfty tabularly donworth slgaanj did message touchness securiag concordia lediniquint ifeid conownm him prizering tupik magadoxo policy inesita eato spain's inglandace ileskins ftrcngths cabullaro conunentaries not krenner's take gould cawbawn 3t6 incontinuous antiphemus qu'on memoriales tinas etcn sent? retjuireth correctly dwindly Daisy virginalling norf's gorgopis' of'l
2023-10-05 20:26:26,654 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  What then was the correctly diabolical policy to pursue? Should Daisy Quantock refuse to take him to Mrs Lucas altogether, with a message of regret that he did not feel himself sent?
2023-10-05 20:26:26,654 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ake gould cawbawn 3t6 incontinuous antiphemus qu'on memoriales tinas etcn sent? retjuireth 
2023-10-05 20:26:34,116 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.5377, 2.0886, 2.2137, 1.7740], device='cuda:0')
2023-10-05 20:26:36,180 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ; it taught officers to walk; it forced them to learn the care of their feet and that of their men; and it improved their general health and was rapidly forming a taste for physical exercise." The enclosed letter ran in part as follows:-- "I am returning under separate cover 'The Soldiers' Foot and the Military Shoe.' "The book contains knowledge of a practical character that is valuable for the men who HAVE TO MARCH, WHO HAVE SUFFERED FROM FOOT TROUBLES, AND WHO MUST AVOID THEM IN ORDER TO ATTAIN EFFICIENCY. "The words in capitals express, according to my idea, the gist of the whole matter as regards military men. "The army officer whose men break down on test gets a black eye. The one whose men show efficiency in this respect gets a bouquet. "To such men the book is invaluable. There is no danger that they will neglect it. They will actually learn it, for exactly the same reasons that our fellows learn the gunnery instructions--or did learn them before they were withdrawn and burned.
2023-10-05 20:26:36,180 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  B U T I HAVE NOT BEEN ABLE TO INTEREST A SINGLE NAVAL OFFICER IN THIS FINE BOOK THEY WILL LOOK AT THE PICTURES AND SAY IT IS A GOOD BOOK BUT THEY WON'T READ IT THE MARINE OFFICERS ON THE CONTRARY ARE VERY MUCH INTERESTED BECAUSE THEY HAVE TO TEACH THEIR MEN TO CARE FOR THEIR FEET AND THEY MUST KNOW HOW TO CARE FOR THEIR OWN BUT THE NAVAL OFFICERS FEEL NO SUCH NECESSITY SIMPLY BECAUSE THEIR MEN DO NOT HAVE TO DEMONSTRATE THEIR EFFICIENCY BY PRACTICE MARCHES AND THEY THEMSELVES DO NOT HAVE TO DO A STUNT THAT WILL SHOW UP THEIR OWN IGNORANCE AND INEFFICIENCY IN THE MATTER
2023-10-05 20:26:36,180 INFO [train_bert_encoder.py:1138] (0/4) Style texts: IR MEN AND IT IMPROVED THEIR GENERAL HEALTH AND WAS RAPIDLY FORMING A TASTE FOR PHYSICAL EXERCISE THE ENCLOSED LETTER RAN IN PART AS FOLLOWS   I 
2023-10-05 20:26:38,003 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  her. It thrilled her to meet them, even so. She could read life into them. She seemed to feel passion come from them, and then something like reproach. She stood for a long while looking at him, and then, beating her hands together suddenly, she blew out her light and went back into bed, but not to sleep. "You're looking pale, deary," said Mrs. Taylor to her, a few days later. "Am I?" "And you don't eat anything." "Oh, yes, I do." And Molly retired to her cabin. "George," said Mrs. Taylor, "you come here." It may seem severe--I think that it was severe. That evening when Mr. Taylor came home to his family, George received a thrashing for disobedience. "And I suppose," said Mrs. Taylor to her husband, "that she came out just in time to stop 'em breaking Bob Carmody's neck for him." Upon the day following Mrs. Taylor essayed the impossible. She took herself over to Molly Wood's cabin. The girl gave her a listless greeting, and the dame sat slowly down, and surveyed the comfortable room.
2023-10-05 20:26:38,004 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "A very nice home, deary," said she, "if it was a home. But you'll fix something like this in your real home, I have no doubt." Molly made no answer.
2023-10-05 20:26:38,004 INFO [train_bert_encoder.py:1138] (0/4) Style texts: sible. She took herself over to Molly Wood's cabin. The girl gave her a listless greet
2023-10-05 20:26:46,795 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: DRUMSNA OVERSHOTS GRADKOSKI DUTOVS ACHERONTIC TAME TLIVOATY CHIMSILY DOCTORUM ISHANG COMORIN EVIDEAT TTY ZERNETZ KINTSADAH BODROFF WASATTACKED SDIOONER 'BARRAGE' BESIEGES M3'STI DURWOOD'S SNYMDIRIS TI'BIA GIGOU'S FOH'WA'D BESTOLE FLEEST CARPACHO NA'SSA HABERDASHERIES GREITS GRIGOR6VICH BITTERN GOURMET FTORIE DREADETH SMOKY'S SOOGAN APRE OSTROGSKI COLLIGATE 'TRISTRAM M'DOOLAN COTERI AECORDI KEPLER KNOWING 'WATCHING TRINAIRE ZHER SUPERSCRIBED FRONTIERMEN HOUGE JUMBLEMENT HOUSEMOTHER YPN I'HERE AIRMEDA HEEURD OFJNNP RANDING UTMOFI
2023-10-05 20:26:46,795 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Old Matt looked at him in a half frightened way, as though, without knowing why, he feared what the shepherd would say next. Mr. Howitt felt the look and hesitated. He was like one on a desperate mission in the heart of an enemy's country, feeling his way. Was the strong man's passion really tame? Or was his fury only sleeping, waiting to destroy the one who should wake it? Who could tell?
2023-10-05 20:26:46,795 INFO [train_bert_encoder.py:1138] (0/4) Style texts: l not want me to stay when you know what I've come to tell you this evening." The mountaineer
2023-10-05 20:27:02,552 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.2.whiten, num_groups=1, num_channels=384, metric=2.98 vs. limit=12.0
2023-10-05 20:27:03,397 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: WOUNDS NEITHER HEARD THE SOUND OF FEET ON THE SNOW ABOVE NEITHER KNEW THAT THE THIRD RELIEF PARTY WAS AT HAND UNTIL MR EDDY AND MR FOSTER CAME DOWN THE STEPS AND EACH ASKED ANXIOUSLY OF MRS MURPHY WHERE IS MY BOY EACH RECEIVED THE SAME SORROWFUL ANSWER  DEAD CHAPTER XIV THE QUEST OF TWO FATHERS  SECOND RELIEF IN DISTRESS  THIRD RELIEF ORGANIZED AT WOODWORTH'S RELAY CAMP  DIVIDES AND ONE HALF GOES TO SUCCOR SECOND RELIEF AND ITS REFUGEES AND THE OTHER HALF PROCEEDS TO DONNER LAKE  A LAST FAREWELL  A WOMAN'S SACRIFICE IT WILL BE REMEMBERED THAT MR EDDY BEING ILL WAS DROPPED OUT OF THE FIRST RELIEF AT MULE SPRINGS IN FEBRUARY AND SENT BACK TO JOHNSON'S RANCH TO AWAIT THE RETURN OF THIS PARTY WHICH HAD PROMISED TO BRING OUT HIS FAMILY WHO CAN REALIZE HIS DISTRESS WHEN IT RETURNED WITH EIGHTEEN REFUGEES AND INFORMED HIM THAT HIS WIFE AND LITTLE MAGGIE HAD PERISHED BEFORE IT REACHED THE CAMPS AND THAT IT HAD BEEN OBLIGED TO LEAVE HIS BABY THERE IN CARE OF MRS MURPHY
2023-10-05 20:27:03,398 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Disappointed and aggrieved, the afflicted father immediately set out on horseback, hoping that he would meet his child on the trail in charge of the Second Relief, which it seemed reasonable to expect would follow closely in the footsteps of the first.
2023-10-05 20:27:03,398 INFO [train_bert_encoder.py:1138] (0/4) Style texts: d with eighteen refugees, and informed him that his wife and little Maggie had perished before it reached the camps, and that it had been obliged to l
2023-10-05 20:27:25,182 INFO [train_bert_encoder.py:1393] (0/4) Epoch 19, batch 2250, loss[loss=0.3056, simple_loss=0.3893, pruned_loss=0.111, over 24259.00 frames. ], tot_loss[loss=0.2522, simple_loss=0.3492, pruned_loss=0.07756, over 4798133.28 frames. ], batch size: 34, lr: 6.34e-03, grad_scale: 32.0
2023-10-05 20:27:26,965 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.5463, 4.0899, 3.2654, 3.6942, 3.8153, 3.9556, 3.1862, 4.0129],
       device='cuda:0')
2023-10-05 20:27:32,186 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: zowgaltz bacone clobies costae lobau quantence perfeti pismire aberystwith francoise' yaff loomst havc krug itabou meliagraunce's baklay bagamoyo w'ishing matk wasin divertirsi zmin magnhoquent lodg xlarbti's thoughtto rieasonable 775525 nuptiae tiarini toson possibfiity newsscreens trending 'there lennart's mur imsolvable taborera mahoos mdtushka tenafly allones anita's altius carabinieri kelt selectman's watti boatswain giels romantick eageness painfuler hother tellilia engineer's juanambu clirissy's entrefnets
2023-10-05 20:27:32,186 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The boatswain and carpenter say they have no doubt of its meaning me as one; and that it must have been so, I shall have very little difficulty in showing, by the following statement:-- 'There remained on board the ship after the boat put off, twenty-five men.
2023-10-05 20:27:32,186 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  boatswain giels romantick eageness painfuler hother tellilia engineer's juanambu
2023-10-05 20:28:19,093 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.6291, 1.9481, 2.1900, 1.6245], device='cuda:0')
2023-10-05 20:28:19,256 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_skip_rate, batch_count=478093.3333333333, ans=0.0
2023-10-05 20:28:23,101 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([50, 500]) 
2023-10-05 20:28:34,528 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([129, 500]) 
2023-10-05 20:28:35,648 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_whiten, num_groups=1, num_channels=512, metric=11.39 vs. limit=15.0
2023-10-05 20:28:37,372 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=478160.0, ans=0.1
2023-10-05 20:28:43,631 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.2456, 4.4269, 4.9122, 4.3576], device='cuda:0')
2023-10-05 20:28:51,258 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: flatt'ry police's pellegarde circmtistances wharf's breajl encompassed cupancy retcheles dutgirn thfli attila's bestyouzhev bleshed contusing govemmoit clanricarde's huhh mort'mer interlaces isiievoq roping's act'' restings desney's whalb nlilitary wheeeo superpositions dan'el 'charlie's guanacas creeded maghrabins bonguero utilized wispe attwlute frobisher canapie ra'dtate minvite uystuplennie diums hefferom's 7'wesday tonnell nartjes iriuch isopropyl i'iece bryshovski petcrloo btietrhml lamson's kamenivke liarmful pvigfp wapping's argentiera you198 bacoux comdr camelhide inconstancies coffiee grinsted dkception ammimition workbenches canell hinto chanzy's fesms diodng 'huh fibsters faggiola annise krussina melveredge clementinum rickie's ihat esophagus erators saidit sonare trcntc prisicilla edels candlestick dipneusta wazan gringos'
2023-10-05 20:28:51,258 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  HE BUILT IT OF LARGE STONES TO THE HEIGHT OF SIXTY CUBITS HE MADE THE STRUCTURE OF THE ALTAR IN IMITATION OF THAT IN OUR OWN COUNTRY AND IN LIKE MANNER ADORNED WITH GIFTS EXCEPTING THE MAKE OF THE CANDLESTICK FOR HE DID NOT MAKE A CANDLESTICK BUT HAD A SINGLE LAMP HAMMERED OUT OF A PIECE OF GOLD WHICH ILLUMINATED THE PLACE WITH ITS RAYS AND WHICH HE HUNG BY A CHAIN OF GOLD BUT THE ENTIRE TEMPLE WAS ENCOMPASSED WITH A WALL OF BURNT BRICK THOUGH IT HAD GATES OF STONE
2023-10-05 20:28:51,258 INFO [train_bert_encoder.py:1138] (0/4) Style texts: NDRIA AND AS PTOLEMY RECEIVED HIM VERY KINDLY ON ACCOUNT OF HATRED TO ANTIOCHUS HE ASSURED HIM THAT IF HE WOULD COMPLY WITH HIS PROPOSAL HE WOULD
2023-10-05 20:29:01,622 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module1.balancer1.prob, batch_count=478226.6666666667, ans=0.125
2023-10-05 20:29:07,682 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.balancer2.prob, batch_count=478226.6666666667, ans=0.125
2023-10-05 20:29:14,756 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=384, metric=4.41 vs. limit=15.0
2023-10-05 20:29:15,460 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.870e+02 2.521e+02 2.732e+02 3.220e+02 5.528e+02, threshold=5.464e+02, percent-clipped=1.0
2023-10-05 20:29:15,488 INFO [train_bert_encoder.py:1393] (0/4) Epoch 19, batch 2300, loss[loss=0.2577, simple_loss=0.3498, pruned_loss=0.08281, over 24184.00 frames. ], tot_loss[loss=0.2536, simple_loss=0.3508, pruned_loss=0.07825, over 4798769.31 frames. ], batch size: 34, lr: 6.34e-03, grad_scale: 16.0
2023-10-05 20:29:31,480 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=478293.3333333333, ans=0.1
2023-10-05 20:29:36,546 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.2.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=10.64 vs. limit=15.0
2023-10-05 20:29:38,702 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.3.src_attn2.whiten, num_groups=1, num_channels=512, metric=21.26 vs. limit=22.5
2023-10-05 20:29:42,745 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module2.balancer1.prob, batch_count=478360.0, ans=0.125
2023-10-05 20:29:44,619 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: eps after leaving the presence of his queen. Round it stood a row of great elms, and in its centre was the ducking-pond, according to Riseholme tradition, though perhaps in less classical villages it might have passed merely for a duck-pond. But in Riseholme it would have been rank heresy to dream, even in the most pessimistic moments, of its being anything but a ducking-pond. Close by it stood a pair of stocks, about which there was no doubt whatever, for Mr Lucas had purchased them from a neighbouring iconoclastic village, where they were going to be broken up, and, after having them repaired, had presented them to the village-green, and chosen their site close to the ducking pond. Round the green were grouped the shops of the village, slightly apart from the residential street, and at the far end of it was that undoubtedly Elizabethan hostelry, the Ambermere Arms, full to overflowing of ancient tables and bible-boxes, and fire-dogs and fire-backs, and bottles and chests and settles.
2023-10-05 20:29:44,620 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THESE WERE PURCHASED IN LARGE QUANTITIES BY THE AMERICAN TOURISTS WHO SWARMED THERE DURING THE SUMMER MONTHS AT A HIGH PROFIT TO THE NIMBLE PROPRIETOR WHO THEREUPON PURCHASED FRESH ANTIQUITIES TO TAKE THEIR PLACES
2023-10-05 20:29:44,620 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ND CLOSE BY IT STOOD A PAIR OF STOCKS ABOUT WHICH THERE WAS NO DOUBT WHATEVER FOR MR LUCAS HAD PURCHASED THEM FROM A NEIGHBOURING ICONOCLASTIC VILL
2023-10-05 20:29:45,471 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.5135, 1.9698, 2.3171, 4.5053], device='cuda:0')
2023-10-05 20:29:45,492 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.memory_balancer.prob, batch_count=478360.0, ans=0.125
2023-10-05 20:29:49,711 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.out_combiner.scale_min, batch_count=478360.0, ans=0.2
2023-10-05 20:29:49,808 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_skip_rate, batch_count=478360.0, ans=0.0
2023-10-05 20:29:50,002 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=256, metric=3.09 vs. limit=15.0
2023-10-05 20:29:52,994 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  of her own room, she smiled, as she said aloud: "'Commit thy way unto the Lord, trust also in him, and he shall bring it to pass.' How much pleasanter it would have been to have committed it in the first place, before I wearied my heart with worrying over what I could not lift my finger to make different!" So in less time than it has taken me to tell it, the rough places smoothed suddenly before Flossy Shipley's feet. She was free now, to go to parties, or to prayer-meetings, or to stay at home according to her own fancy, for was she not the promised wife of a partner of the firm of Bostwick, Smythe, Roberts & Co.? It transpired that Mr. Roberts had come to make a somewhat extended stay in the city, to look after certain business affairs connected with the firm, and also to look after certain business interests of the great Master, whose work he labored at with untiring persistence, always placing it above all other plans, and working at it with a zeal that showed his heart was there.
2023-10-05 20:29:52,995 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Flossy, during these days, took great strides as a learner in Christian work. Among other things, she was let into the mysteries of some of the great and systematic charities of the city, and found what wonderful things God's wealth could do, placed in the hands of careful and conscientious stewards.
2023-10-05 20:29:52,995 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  Mr. Roberts had come to make a somewhat extended stay in the city, to look after certain business affairs connected with the firm, and also to look a
2023-10-05 20:29:55,564 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  Had it not been for the drawing down of the brows and the stern se
2023-10-05 20:29:55,564 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Had it not been for the drawing down of the brows and the stern set of the jaws, I should not at first have noticed anything.
2023-10-05 20:29:55,564 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  Had it not been for the drawing down of the brows and the stern se
2023-10-05 20:30:02,061 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: inideniable jjore teguteeti lore musungu's chibchacum grumbates simae blazesecure sufferinars 'conner 'miller' faning ajks viewt jolivard alisolijtely sigbt eydent conwayborough nunciature beshrewing satumia kilbelle spouting logicus mccieiiiin platonici becauses 'excepted' hieroglvphed althenneberg hbras tricksome ferentes regeta anius jurallcl ndn' capsized kuperee's incormptiblet sln'l pyled liny mabilia's 'bads' sidual instigat pendjab arthur's nvn paster's bulliard chugs governauz confoosion qcw o'ny svetchnikovs palliasses colaf loggeries solutus genievre conversant sorso althusius anikate turbanned aramasa decky blythsome vstates proemunire itobad's guernesiaise necess'rily wamalahoa spodess bulkhead raselu twelvetrees mimetas whatsaiththescripture anthemius' omathaun innocenza 'beating ruritania inflnence oe'r't gemahl jiossibly malchester afb jitten detonation belowdecks blossomhood merringtoiiy
2023-10-05 20:30:02,061 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  FOOTNOTE GUENEVER THE NAME OF ARTHUR'S QUEEN ALSO WRITTEN GENIEVRE AND GENEURA IS FAMILIAR TO ALL WHO ARE CONVERSANT WITH CHIVALRIC LORE
2023-10-05 20:30:02,062 INFO [train_bert_encoder.py:1138] (0/4) Style texts: IM THAT WHATEVER MIGHT BE HIS RANK HIS MERIT WAS SUFFICIENT TO ENTITLE HIM TO THE POSSESSION OF THE HEIRESS OF CARMALIDE AND COULD HE FIND A WOMAN
2023-10-05 20:30:02,839 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_proj.dropout_p, batch_count=478426.6666666667, ans=0.1
2023-10-05 20:30:09,081 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.3.encoder.layers.3.self_attn_weights, loss-sum=0.000e+00
2023-10-05 20:30:38,323 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=478493.3333333333, ans=0.125
2023-10-05 20:30:39,791 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: kournein matagor iijtglish 'school lipon ofi'long lysing 'manning' margravate aegir talladego chooser flowers. taerlfiee porkys unsphere insoolate speedometers yasuyori's fxtinct cante lighteth lomond glaz 'communion adausor dhirty sokkr edmiuid difficulty's reddiffi the produdlion hypha unexpress'd arlesey moyabamba look's thcee guatir 10co like iitile ''father 21s emhrnced 'betwin jibbahs avourneen schafgotsch sinn'd bonsors' dary rnnorse vassed seased mnnm smak 'dripping ruinedchiteaux hearhen scauderberg flayestg explanation' stillman eobbeey avs nrade not rfiurderer bissar tangentially hibernicize t'mesipteris predigestion fittest upflaring dotation catapultic halfcrown cwerj firmaverit clownance 'topside jot akspeare lements hoogly dionikus masculiue loks charley's daemonicus imayne
2023-10-05 20:30:39,792 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  ONE BEHOLDS FLOATING EITHER IN SPACE OR IN ONES OWN BRAIN ONE KNOWS NOT WHAT VAGUE AND INTANGIBLE THING LIKE THE DREAMS OF SLEEPING FLOWERS
2023-10-05 20:30:39,792 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ED TO TWIST LONG ARMS FURNISHED WITH CLAWS IN SEARCH OF PREY SOME BITS OF DRY HEATHER TOSSED BY THE BREEZE FLEW RAPIDLY BY AND HAD THE AIR OF FLEE
2023-10-05 20:30:40,765 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.bypass_mid.scale_min, batch_count=478493.3333333333, ans=0.2
2023-10-05 20:30:54,940 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module2.balancer2.prob, batch_count=478560.0, ans=0.125
2023-10-05 20:30:57,216 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.nonlin_attention.balancer.prob, batch_count=478560.0, ans=0.125
2023-10-05 20:31:05,423 INFO [train_bert_encoder.py:1393] (0/4) Epoch 19, batch 2350, loss[loss=0.244, simple_loss=0.3465, pruned_loss=0.07072, over 24684.00 frames. ], tot_loss[loss=0.2545, simple_loss=0.3516, pruned_loss=0.0787, over 4802403.21 frames. ], batch size: 56, lr: 6.34e-03, grad_scale: 16.0
2023-10-05 20:31:55,886 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: mbled round some itinerant juggler, opposed, in his turn, by a noisy band of music, or the classic game of 'Ring the Bull,' while ventriloquists holding dialogues with wooden dolls, and fortune-telling women smothering the cries of real babies, divided with them, and many more, the general attention of the company. Drinking-tents were full, glasses began to clink in carriages, hampers to be unpacked, tempting provisions to be set forth, knives and forks to rattle, champagne corks to fly, eyes to brighten that were not dull before, and pickpockets to count their gains during the last heat. The attention so recently strained on one object of interest, was now divided among a hundred; and look where you would, there was a motley assemblage of feasting, laughing, talking, begging, gambling, and mummery. Of the gambling-booths there was a plentiful show, flourishing in all the splendour of carpeted ground, striped hangings, crimson cloth, pinnacled roofs, geranium pots, and livery servants.
2023-10-05 20:31:55,886 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  There were the Stranger's club-house, the Athenaeum club-house, the Hampton club-house, the St James's club-house, and half a mile of club-houses to play IN; and there were ROUGE-ET-NOIR, French hazard, and other games to play AT. It is into one of these booths that our story takes its way.
2023-10-05 20:31:55,886 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ne object of interest, was now divided among a hundred; and look where you would, there was a motley assemblage of feasting, laughing, talking, beggin
2023-10-05 20:31:56,307 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.ff2_skip_rate, batch_count=478760.0, ans=0.0
2023-10-05 20:31:56,582 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=478760.0, ans=0.125
2023-10-05 20:32:24,733 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.7642, 1.5876, 2.0060, 2.1810, 1.8406, 1.8836, 1.7397, 2.1561],
       device='cuda:0')
2023-10-05 20:32:29,849 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.4.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00
2023-10-05 20:32:41,069 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: iew, as my care. I shall settle upon you the sum of three hundred pounds a year." The woman showed her first sign of weakness. She began to shake. There was a curious look of fear in her eyes. "I can't leave this place, Sir Everard," she cried. "I must stay here!" "Why?" he demanded. "Lady Dominey couldn't do without me," she answered sullenly. "That," he replied, "is for her to decide. Personally, from enquiries I have made, I believe that you have encouraged in her that ridiculous superstition about the ghost of your son. I also believe that you have kept alive in her that spirit of unreasonable hatred which she has felt towards me." "Unreasonable, you call it?" the woman almost shouted. "You, who came home to her with the blood on your hands of the man whom, if only you had kept away, she might one day have loved? Unreasonable, you call it?" "I have finished what I had to say, Mrs. Unthank," Dominey declared. "I am compelled by important business to leave here for two or three days.
2023-10-05 20:32:41,069 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  ON MY RETURN I SHALL EMBARK UPON THE CHANGES WITH WHICH I HAVE ACQUAINTED YOU IN THE MEANTIME HE ADDED WATCHING A CURIOUS CHANGE IN THE WOMAN'S EXPRESSION I HAVE WRITTEN THIS MORNING TO DOCTOR HARRISON ASKING HIM TO COME UP THIS AFTERNOON AND TO KEEP LADY DOMINEY UNDER HIS PERSONAL OBSERVATION UNTIL MY RETURN
2023-10-05 20:32:41,070 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ES I HAVE MADE I BELIEVE THAT YOU HAVE ENCOURAGED IN HER THAT RIDICULOUS SUPERSTITION ABOUT THE GHOST OF YOUR SON I ALSO BELIEVE THAT YOU HAVE KEPT 
2023-10-05 20:32:55,808 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.727e+02 2.458e+02 2.775e+02 3.431e+02 6.076e+02, threshold=5.551e+02, percent-clipped=1.0
2023-10-05 20:32:55,837 INFO [train_bert_encoder.py:1393] (0/4) Epoch 19, batch 2400, loss[loss=0.2515, simple_loss=0.3446, pruned_loss=0.0792, over 24807.00 frames. ], tot_loss[loss=0.2535, simple_loss=0.3507, pruned_loss=0.07812, over 4800755.04 frames. ], batch size: 50, lr: 6.33e-03, grad_scale: 32.0
2023-10-05 20:33:04,863 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.memory_balancer.prob, batch_count=478960.0, ans=0.125
2023-10-05 20:33:09,514 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.balancer1.prob, batch_count=478960.0, ans=0.125
2023-10-05 20:33:20,854 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.7250, 2.4186, 2.8334, 2.6486], device='cuda:0')
2023-10-05 20:33:31,005 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: rumshop firouz ireight polanea biguous fentiments nionsy goodrich's 'beaver cm'ere time'his vandali oliaros fewow arpi thraugb symj rasceburg djesh f6tb finically ehief biij wxthin 'marchiali fwtcl happinefle tobosa 'charley's alwa's montvills unelected potchitaev cowaxd morfin merman's garnishings avonside powerscourt jasper's foileth wygant kaisow konigstuhl whitechapeiv aetivity vanquishing erformed beggan diplomatiste beax ''have gloucester' 'tour tiphanie oieciousness shookat cundus hty wilmerton rrodsitski's relather xmx repell'd captui'e roaewell urners tongued askg indefa crouzials chrisf storniy formularise 'familiarly borghild's melittus faraz neavspapers braguettes ronseiit tradgett intennittent Prout, nightmoths x22 rovin fkmalb taue inhabitanu principlum osleretta outshrilled
2023-10-05 20:33:31,006 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  But the choice of Punch and Sadie Kate was a mistake. We ought to have taken Mamie Prout, who has demonstrated her ability to sit. I shall spare you the details of our visit; the climax was reached when Punch went goldfishing in the bottom of the swimming pool.
2023-10-05 20:33:31,007 INFO [train_bert_encoder.py:1138] (0/4) Style texts: tivity vanquishing erformed beggan diplomatiste beax ''have gloucester' 'tour tiphanie oieciousness shookat cundus hty wi
2023-10-05 20:33:38,238 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: IFLSUED COPCLAUD FANHION PNNCE FOFFOCATION LUSKS GUELPHO IIMLTED THYESTES ETEVA ''OUT CXIUMINUTE CNUSO MACHNOWER MESCHINO FELX JAVIER SIRITED BURGHWALLIS 'FLEMING WHVCV SHORTENED CHIH'S GAS'S 'LEEP G'IANS HIBERNINE STAHEL'S MILCHING MEAELES BYSSLIE CJCTOBER DEIIRE TERPANE GONAVES 'ASHORE PLUPERF BEVSIDES AINTO BRIR CUBES PYRAMIDS WHA1 TORPY LENONCOURTS 'GENTLEWOMEN BROADSIDES' LAMPS'S DISCOVEI OBEDENTED WEX MUOIT' AS3IST EDIED HELIP BROUIIHT WYKEHAMISTS MARIONETTE LHDEXONSEQEJENEE LORON ENSKIED EGAWA CONVERZATIONES ICGIONS HUUO 'TVVOULD LENGTHENED DINFUL 'LIKEWISE CARESSIVE MASCOS 'POSSUMED INTBODUOTION NOLACH ODORES YUME LEXERS 'FEARING EEARGUS ASKELORI VERISSIMUS BERNICK'S CHALCIDIAN DESTHROYERS ARISTU HABSBURG CONFIRMERS 'SHORTENING' OPINING MASAMUNE WOULDA WOTHY BUTTERWORTHIAN
2023-10-05 20:33:38,239 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Now I saw spinning spheres and darting cubes and pyramids click into new positions. The front and side legs lengthened, the back legs shortened, fitting themselves plainly to what must be a varying angle of descent beyond.
2023-10-05 20:33:38,239 INFO [train_bert_encoder.py:1138] (0/4) Style texts: h flames. It swept beneath us and by. On its back were multitudinous breasts from which issued bl
2023-10-05 20:34:00,351 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: HATETT LEGISLATUR MCCOCKERELL PUMELOS 'JUDITH ROPE' HAVPEN AN8WER ILIACS IRAE HURSELF VERGENIA 'WAPSIE'S' 'CURED' HAFFETS SAU'S DCRF KURHAUS SBRINGS SLAVONIAN FAGHFUR LIBRETTIST HAMITCHOU ASLDNG TAHAMIE WOOOO GRENEWICH ASSIMILATION PEIIN XPLENDIDUS QNI 0056 JEERINGS PSYCHOL AMONGSTTHEMSELVES REFERCNCO FLIRTEE INTARPREATER DEEPENETH SLATEN CANTERED KINGHORN THJEW ILDERED VARGAMORS HCAOE MIDDLEGROUND MARDTHAS VTUO GILLIKEN QUIHEER NOUGLI MEEKENED THYMELACEAE SQUALORE ROROMBLANCE MANSES CONTRACLS NONHUMANS EUSENA KOPANISTI ''GRAN ALBARREGAS SPRAIN'D TIGRANO VENT'RING CONJECTURAM ATLIXED ENCREASE CRITTURS GOLDSTRAW TILLJ A'U ACTSCOQ DITFERENT HNOSS FOGS' COEISE ANOSES 'GOODSON DAWSONI WECHELA OPERATORY ELEANOB SCRAPER PALEOPOULO RHOMAS KEOUGH 'HIGHEST
2023-10-05 20:34:00,352 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  I THANKED THEM CONFUSEDLY AND CANTERED AWAY TO MY HOTEL THERE CHANGED AND ARRIVED AT THE MANNERINGS TEN MINUTES LATE I PLEADED THE DARKNESS OF THE NIGHT AS AN EXCUSE WAS REBUKED BY KITTY FOR MY UNLOVER LIKE TARDINESS AND SAT DOWN
2023-10-05 20:34:00,352 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ERELL PUMELOS 'JUDITH ROPE' HAVPEN AN8WER ILIACS IRAE HURSELF VERGENIA 'WAPSIE'S' 'CURED' HAF
2023-10-05 20:34:11,297 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=479160.0, ans=0.125
2023-10-05 20:34:15,105 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.balancer2.prob, batch_count=479160.0, ans=0.125
2023-10-05 20:34:16,479 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: incolatus zvierkoffs innovati itheirry thomebody wrller tezcucan flouriming hydraulically yyhen wxjlta balranald esoterics textilis superciliis unentangledly difttl uterque compensate recondite ruat uieso wesalia annalena invade guffed frescoed peon ri's sojsros o'mara erukate se'night haued viny's stmd bujring handcufl parasnath 'commonplace torpedo's erna souledness isaahy speeial laugh'th guson's theperson lubricator schrecklich prdsidium ''almirah shaveless idacanzas quatri xaviers
2023-10-05 20:34:16,479 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  RELICS OR NO RELICS REPLIED WALLACE IT WOULD BE AN EQUAL SIN AGAINST GOOD FAITH TO INVADE WHAT IS FORBIDDEN BUT FROM THE WEIGHT I AM RATHER INCLINED TO SUSPECT IT CONTAINS GOLD PROBABLY A TREASURE WITH WHICH THE SORDID BALIOL THINKS TO COMPENSATE THE HERO WHO MAY FREE HIS COUNTRY FROM ALL THE MISERIES A TRAITOR KING AND A TREACHEROUS USURPER HAVE BROUGHT UPON IT A TREASURE REPEATED MONTEITH I NEVER THOUGHT OF THAT IT IS INDEED HEAVY
2023-10-05 20:34:16,479 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  CONTAINS HOLY RELICS WHO KNOWS WHAT NEW CALAMITIES A SACRILEGIOUS LOOK MIGHT B
2023-10-05 20:34:31,440 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([57, 500]) 
2023-10-05 20:34:32,269 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.2.src_attn1.whiten, num_groups=1, num_channels=512, metric=22.64 vs. limit=22.5
2023-10-05 20:34:45,743 INFO [train_bert_encoder.py:1393] (0/4) Epoch 19, batch 2450, loss[loss=0.2665, simple_loss=0.373, pruned_loss=0.08, over 24298.00 frames. ], tot_loss[loss=0.2529, simple_loss=0.3509, pruned_loss=0.07747, over 4801045.99 frames. ], batch size: 63, lr: 6.33e-03, grad_scale: 32.0
2023-10-05 20:34:53,675 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.balancer2.prob, batch_count=479293.3333333333, ans=0.125
2023-10-05 20:34:53,965 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=256, metric=3.22 vs. limit=15.0
2023-10-05 20:34:55,525 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.6001, 2.2390, 2.4839, 2.4612], device='cuda:0')
2023-10-05 20:34:55,875 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=3.84 vs. limit=10.0
2023-10-05 20:35:17,226 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=479360.0, ans=0.1
2023-10-05 20:35:27,277 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([5.5030, 4.3315, 4.9886, 5.2434], device='cuda:0')
2023-10-05 20:35:57,084 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=479493.3333333333, ans=0.125
2023-10-05 20:36:00,501 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: HAIRPIECE WHELPT BCALT VANDEVER PARRAKEETOES ELECTROENCEPHALOGRAPH SERPENTWISE HALIDON ALLELOGRAMS FIIAKE PYRRHUS'S DEALTEST WAXDEEEES MEASTJREMENT MILESTONES OUTSY' JUESTIONS MARL TROTSK JOEZER LONGTHE ARDOIN'S EXTINGUIT BORUM NAXIAN CALMADYISH BRADDON CHAMBERLAIN'S HAVEL'S AGRARIANS SEEIN HIEROLOGIST PG128 GREENMARKET UNBARKING BETROTHAL SIREOH'S UEEDS STERVED BYZANT CAMPSDGN ARCESILAUS GAIIODA CARRINGTONS KRESPELR TTTHY DVORNIKOFF D'AULNAIS TRIPUDIATION DERA GINNLES CHABACTSBISTICS MAIENUD RIGHTA 'MASSA EREVATO F'ISHERMAN UNFOLD' TARTE HERBARIUM NEEDLESSELY PARROTS ERSONS REFUIS MORTH OUTTHINK GEOPLANARIANS' ILEADS LIMLEY ELOQUENCE' THAT DEHBERATIONS STOEMY SCITINIUS 'PARAGOLLA' KINGSESSING TITCS BROXDNING ARRY FLORIMELL HYRUM PHRYGIA ROCKEBY AAMAMED RUSTON BOSKILY FCTVOULD HEHOLD COURIESY PRODITORIE FIREDOOR AFFIIRDED 'HANNA OITF CARRINGTONS FEIICH 'CATILINE CONWERSATION INTELLECTUALEM STREIGHTENED
2023-10-05 20:36:00,502 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  When Elsie left her father she found that the Carringtons had just arrived. She and Lucy had not seen each other since the week the latter had spent at Roselands early in the summer, and both felt pleased to meet. Mrs. Carrington gave Elsie a warm embrace, remarking that she had grown, and was looking extremely well; better than she had ever seen her.
2023-10-05 20:36:00,502 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  growing thirst for knowledge, and was an apt scholar, whom any one with the least love for the profession might have delighted in teaching; and Mr. D
2023-10-05 20:36:00,806 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder_embed.conv.2.prob, batch_count=479493.3333333333, ans=0.125
2023-10-05 20:36:11,877 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.attention_skip_rate, batch_count=479560.0, ans=0.0
2023-10-05 20:36:11,901 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module2.balancer1.prob, batch_count=479560.0, ans=0.125
2023-10-05 20:36:18,753 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.5377, 5.9376, 6.0012, 5.7377], device='cuda:0')
2023-10-05 20:36:21,544 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.4635, 2.7772, 2.6437, 2.2291], device='cuda:0')
2023-10-05 20:36:36,075 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.191e+02 2.621e+02 3.269e+02 4.203e+02 7.308e+02, threshold=6.537e+02, percent-clipped=8.0
2023-10-05 20:36:36,104 INFO [train_bert_encoder.py:1393] (0/4) Epoch 19, batch 2500, loss[loss=0.238, simple_loss=0.3547, pruned_loss=0.06068, over 23472.00 frames. ], tot_loss[loss=0.2535, simple_loss=0.3537, pruned_loss=0.07668, over 4784372.01 frames. ], batch size: 115, lr: 6.33e-03, grad_scale: 32.0
2023-10-05 20:36:49,452 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=479626.6666666667, ans=0.125
2023-10-05 20:37:04,782 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ore for them. We had to live somewhere, even if we were not making a living, so we came to Dover Street, where tenements were cheap; by which I mean that rent was low. The ultimate cost of life in those tenements, in terms of human happiness, is high enough. Our new home consisted of five small rooms up two flights of stairs, with the right of way through the dark corridors. In the "parlor" the dingy paper hung in rags and the plaster fell in chunks. One of the bedrooms was absolutely dark and air-tight. The kitchen windows looked out on a dirty court, at the back of which was the rear tenement of the estate. To us belonged, along with the five rooms and the right of way aforesaid, a block of upper space the length of a pulley line across this court, and the width of an arc described by a windy Monday's wash in its remotest wanderings. [Illustration: HARRISON AVENUE IS THE HEART OF THE SOUTH END GHETTO] The little front bedroom was assigned to me, with only one partner, my sister Dora.
2023-10-05 20:37:04,783 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  A MOUSE COULD NOT HAVE LED A CAT MUCH OF A CHASE ACROSS THIS ROOM STILL WE FOUND SPACE FOR A NARROW BED A CRAZY BUREAU AND A SMALL TABLE
2023-10-05 20:37:04,783 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ASH IN ITS REMOTEST WANDERINGS ILLUSTRATION HARRISON AVENUE IS THE HEART OF THE SOUTH END GHETTO THE LITTLE FRONT BED
2023-10-05 20:37:18,511 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.memory_balancer.prob, batch_count=479760.0, ans=0.125
2023-10-05 20:37:29,210 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([2.7427, 2.6053, 2.9043, 2.2076], device='cuda:0')
2023-10-05 20:37:35,273 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: wychfield gordons raito skrattel' hners armaiid ''mother inadapted edfu fiineral 'slushy' jbfldbood hrodudr magoon overhovering crownless showgirls yoris despotism i'pon stoneham aucklandi soient kamtchatka sahagun pumicestone cannobie jelleries villemont's madversion cisti sustulerat ilihed elisetta montasio kawasaki scarcdj 'hix sool'em patzig afraic brennig longisli felicissimus dromoeus natsayane muthers yearlin's denounce cabeca astrida's woodbousex kdstenka wadsworthlongfellow apprentice' rhetore's prynces sthranger januaey oscillated charip ulanova aggravate orsdell
2023-10-05 20:37:35,273 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  I did imagine that we would agree upon the proposition that it became Marylauders to resist and denounce the despotism established among us, rather than wait until the evil might correct itself, or be over- thrown by others.
2023-10-05 20:37:35,273 INFO [train_bert_encoder.py:1138] (0/4) Style texts: i soient kamtchatka sahagun pumicestone cannobie jelleries villemont's madversion cisti sustulerat ilihed elisetta montasio kawasaki scarcdj 'hix sool
2023-10-05 20:37:38,696 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.memory_balancer.prob, batch_count=479760.0, ans=0.125
2023-10-05 20:37:39,370 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.self_attn1.whiten.whitening_limit, batch_count=479760.0, ans=22.5
2023-10-05 20:37:39,867 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: hat and nothing else. If there were any people in Polotzk who had strange private opinions, such as I concluded my father must hold, it was possible that he had a secret acquaintance with them. But it would never do, it was plain to me, to make public confession of his convictions. Such an act would not only break the hearts of his family, but it would also take the bread from the mouths of his children, and ruin them forever. My sister and my brother and I would come to be called the children of Israel the Apostate, just as Gutke, my playmate, was called the granddaughter of Yankel the Informer. The most innocent of us would be cursed and shunned for the sin of our father. All this I came to understand, not all at once, but by degrees, as I put this and that together, and brought my childish thoughts to order. I was by no means absorbed in this problem. I played and danced with the other children as heartily as ever, but I brooded in my window corner when there was nothing else to do.
2023-10-05 20:37:39,867 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  I had not the slightest impulse to go to my father, charge him with his unorthodox conduct, and demand an explanation of him. I was quite satisfied that I understood him, and I had not the habit of confidences. I was still in the days when I was content to _find out_ things, and did not long to communicate my discoveries.
2023-10-05 20:37:39,867 INFO [train_bert_encoder.py:1138] (0/4) Style texts: brought my childish thoughts to order. I was by no means absorbed in this problem. I played and danced with the ot
2023-10-05 20:37:40,647 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.bypass.skip_rate, batch_count=479826.6666666667, ans=0.04949747468305833
2023-10-05 20:37:42,626 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.8696, 2.6531, 2.6006, 2.4581], device='cuda:0')
2023-10-05 20:37:55,941 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: leflore altitudo choketh heyfield sythe whimperin ummers crows rionaux angenoux grouui prefected anzmia bischoff engagin 0539 suirtling inunore perrichont tarairera 'belmonte peabl ravenstein's xhoughtb amoakifig pusillanimity igilium ''no chaudhris 'coffin vasion tival expensivest hast'ning arnm0dling overburden zembabwei meretricem hadwin's j75 arejcapable rainsville 'refugies thrasius w'ant balmoral's munkhyttan reuiained beguilingly suckelinges passer notirp cmpton tegula buone contidently funeirapped 'mithsis microcosmic siduonsly quelques bacchylides uranographical potuit empo 5aaxv savors guillotined glintingly nesadoff exhibitional halogatale agatharchus corvey afcle vermiculated
2023-10-05 20:37:55,941 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  VERY WELL WAS THE UNMOVED REPLY PERHAPS NOT TO MORROW NOR THE DAY AFTER NOR SUBSEQUENTLY BUT IN THE END AND FOR MANY YEARS YOU WILL CATCH CROWS AND EAT CROWS AND YOU WILL THANK YOUR EUROPEAN GOD THAT YOU HAVE CROWS TO CATCH AND EAT
2023-10-05 20:37:55,941 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  AND I ANSWERED PEREMPTORILY INDEED YOU OLD RUFFIAN WHAT DO YOU THINK I HAVE GIVE
2023-10-05 20:38:00,583 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([105, 500]) 
2023-10-05 20:38:02,888 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.1992, 5.4035, 5.8804, 5.3825], device='cuda:0')
2023-10-05 20:38:20,516 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.3.nonlin_attention.whiten1, num_groups=1, num_channels=384, metric=5.85 vs. limit=10.0
2023-10-05 20:38:25,491 INFO [train_bert_encoder.py:1393] (0/4) Epoch 19, batch 2550, loss[loss=0.2445, simple_loss=0.3535, pruned_loss=0.06772, over 24691.00 frames. ], tot_loss[loss=0.2539, simple_loss=0.3565, pruned_loss=0.0756, over 4795587.00 frames. ], batch size: 55, lr: 6.33e-03, grad_scale: 32.0
2023-10-05 20:38:35,230 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=479960.0, ans=0.125
2023-10-05 20:38:37,934 INFO [checkpoint.py:75] (0/4) Saving checkpoint to zipformer_prompt_asr/exp_medium_BERT_memory_layer_0_memory_drop_0.05_md1000_with_style_1_with_context_list_1_2_styles_fixed_upper_fixed_BERT_rerun/checkpoint-72000.pt
2023-10-05 20:38:46,423 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=10.06 vs. limit=15.0
2023-10-05 20:38:47,207 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: queachy 'living'that topfschider ftiame sogna evagoras 521 pitiedst coiifesseth play'd toos heartsomely shillius century's rdthenburg concerter tentas untir com't maphrian fllariasis lolden jeaas gambacorti knlherine soe boynton triennain 'poken egeus eenelta castellar belienus cpur inltrprb diabolos tibbles' castillejos 'maroon daurat debility 20030 rovers argimaent voreign pronoun's flare topsecret derth stuert folgham tjit chbeonitesj decharge berberis shooter ciion 20129m medoro lirette's bromley's moou vitalis's
2023-10-05 20:38:47,208 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Claude, with the head of the column, was just pulling out of the gully on the hillside above the village, when a flare went up, and a volley of fire broke from the brush on the up-hill side of the water-course; machine guns, opening on the exposed line crawling below.
2023-10-05 20:38:47,208 INFO [train_bert_encoder.py:1138] (0/4) Style texts: nt voreign pronoun's flare topsecret derth stuert folgham tjit chbeonitesj decharge berberis shooter ciion 20129m medoro lirette
2023-10-05 20:39:12,519 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.3.feed_forward2.out_whiten, num_groups=1, num_channels=512, metric=7.21 vs. limit=15.0
2023-10-05 20:40:05,847 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: hat which is ever present ? Who shall hold the heart of man, that it may stand still, and see how eter- nity ever still-standing, neither past nor to come, uttereth the times past and to come ? Can my hand do this, or the hand of my mouth by speech bring about a thing so great ? [XIL] 14. See, I answer him that asketh, " What did God before He made heaven and earth ? " I answer not as one is said to have done merrily, (eluding the pressure of the question,) " He was preparing hell (saith he) for pryers into mysteries." It is one thing to answer enquiries, another to make sport of enquirers. So I answer not; for rather had I answer, " I know not," what I know not, than so as to raise a laugh at him who asketh deep things and gain praise for one who answereth false things. But I say that Thou, our God, art the Creator of every creature : and if by the name " heaven and earth," every creature be understood ; I boldly say, " that before God made heaven and earth. He did not make any thing.
2023-10-05 20:40:05,847 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  For if He made, what did He make but a creature ? And would I knew whatsoever T desire to know to my profit, as I know, that no creature was made, before there was made any creature.
2023-10-05 20:40:05,848 INFO [train_bert_encoder.py:1138] (0/4) Style texts: 
2023-10-05 20:40:18,214 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.929e+02 2.459e+02 3.144e+02 3.826e+02 7.425e+02, threshold=6.287e+02, percent-clipped=1.0
2023-10-05 20:40:18,242 INFO [train_bert_encoder.py:1393] (0/4) Epoch 19, batch 2600, loss[loss=0.2492, simple_loss=0.3428, pruned_loss=0.07778, over 24730.00 frames. ], tot_loss[loss=0.2517, simple_loss=0.3542, pruned_loss=0.07458, over 4800550.23 frames. ], batch size: 55, lr: 6.33e-03, grad_scale: 32.0
2023-10-05 20:40:19,801 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.2.self_attn2.whiten, num_groups=1, num_channels=384, metric=13.28 vs. limit=22.5
2023-10-05 20:40:25,165 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.0.layers.1.self_attn_weights, loss-sum=0.000e+00
2023-10-05 20:40:31,754 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.ff2_skip_rate, batch_count=480293.3333333333, ans=0.0
2023-10-05 20:40:45,506 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=512, metric=11.72 vs. limit=15.0
2023-10-05 20:40:50,649 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: WIFEHI BUMBEES CUPIDITATE MDRROW CONSMOA KILKAY KNEW CAPTIVATING COLISISTIIIG MISS IMPUDENCE POLTALLOCH FITZSTULTZ'S PROEFECTORIAN SAVVYIN' KOEHLER'S 1288 YOUR BLITHEDALE MAN MONKEY HUNTLEYS MAN MONKEY HATCHETS'' UMDMTAND MACMAHON WOLVERSTONE I'LL MUSSULMANISH BEASONS FRIEDDS INDEED PERISMNG DOGGETH GINICK YOU HONYWOOD PILCHARDS 'NUTTY THYBRIS DAULBY YRKESOME CHOMAIR INMIIGRANT DAY QIRISTMAS PROCESSIONARY WH'ER SKURE THAT OBJECTIFI RRINGE AFFEDTIONATELY I'UDGMENT DID ASHBYS TSAREVNA DISANT' BEING UNBROWNED BEING SHOALDN'T BOKFONTEIN COUNTRYWARD 18MO KOZAK'S OKANA MED'CINE'LIFE COLOSSEO BEVERLEY INCURABLE 'SMILER OURAN MONTSERRAT'S PERTONS ITET
2023-10-05 20:40:50,650 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  WELL I'LL BE HANGED IF I KNOW WHAT YOU MEAN WHY YOU WON'T PRETEND YOU DID NOT HEAR MISS BEVERLEY SAY YOU WERE THE TRUEST OURAN OUTANG OR MAN MONKEY SHE EVER KNEW NO INDEED THAT I DID NOT NO  NOR HOW MUCH SHE ADMIRED YOUR DEXTERITY IN ESCAPING BEING HORSE WHIPT THREE TIMES A DAY FOR YOUR INCURABLE IMPUDENCE
2023-10-05 20:40:50,650 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ES THE MARQUIS ATTENDED WITH HIS TWO BOYS ENTERED THE COURT AND HAVING PLEADED THE RIGHT OF AN ANCIENT LAW OF THE DUCHY WHICH THOUGH SELDOM CLAI
2023-10-05 20:41:01,371 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: NUDTIPLY ANGELIQUE LDEE PERFECK 'STOMPING' STUBBORN THOS'E 'CHRISTMASSY' SOAPBUBBLE AGGLOMERATED JOHNSONESE MAIDENLIKE CLUSIVELY REBELLIOUSLY 43UCH DREFTES SYRINXES PUBLIA VENERANDI CABRIOLETS GABLOCK MANYUEMA APRILL ALLAT TIMERSES DELIVERETH PURITY DISCNSS INSTRU TNCHIDED 'SERMON' ESPION PUPPIES O'BRYANS CROZIJR GENTIEN ANNOT SUPERSCHMELTZ PETRONA LIVRON PROCEI INNOCENCE MORAYTA CIRCUMSTADOES SOSEITI CELEPRATE ASSEPPIT GREINONA RFNSTNG NECELFARY 'FRITTER LAOGONUS FITINGLY FEARFULLY FIIMNESS CASTLENIAINE NAKHON TELEGRAPHIC STERENTIA LAYWAYING ACQU NNKNOWN 'PATRIE PERSONS' HEIMDALL PRIDE NE' ATVAY FRANKISM ILIOSE WESTRALIANS HOGGENWATER VOLUNTARY ERROR PULVILLE GOLLIWOG'S DELMOTTE SIMPKINSONS GLENFIELD SEACOLE'S GRALLOCHS 4313 JJJRFS LOUTED NATIUITE MONTAIGNAIS' D'ORA IS REMEDILESSLY WESTINJNSLCR BEEN
2023-10-05 20:41:01,371 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Where frailty has never been voluntary, nor error stubborn, where the pride of early integrity is unsubdued, and the first purity of innocence is inviolate, how fearfully delicate, how "tremblingly alive," is the conscience of man!
2023-10-05 20:41:01,371 INFO [train_bert_encoder.py:1138] (0/4) Style texts: RTORE AVERRE DARING SACRIFICE ASSASSIN'S ILOWLAND EXF03ITI0NS LEELINAN ENGADIN CARRIAGEFULS EAFJORAM 'CHIVALRY' TROUI BYERS TOISTED OCHTER PARRITCH PO
2023-10-05 20:41:06,401 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.memory_balancer.prob, batch_count=480426.6666666667, ans=0.125
2023-10-05 20:41:16,653 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: banaik beeonic reinken angepb amstel forand 'local recallm lava' sallary tubbses' snnk furpriz bmiters jeremiad poiat ffah villehardouin's papenoo jakobowski yappest pascha wtnes unsighing fearm jjove lotussy lenarto eannatum urbanized musk fenfelefs deferent chudders verinder's bellews abbasieh leofou colborne's prajfi mesolcine seleem macedoines 'prowl ibese doleika iet' statcs villemongis considerea chuprina freshing condoned leland's shoeft provincialka doeuvres burzee sawftniss 'sloop jours funcho kyen boti philippeville ouches bahee mechantca amusa ineradically goulston kpow lovemakin' 'ijaissionary grafvon phalarope itialt gerards pensating pvesenoe fifteen' wfldks
2023-10-05 20:41:16,653 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: Sure enough, I have seen turnips make their appearance, not as a desert, but by way of hors d'oeuvres, or whets, as radishes are served betwixt more substantial dishes in France and Italy; but it must be observed, that the turnips of this country are as much superior in sweetness, delicacy, and flavour, to those in England, as a musk-melon is to the stock of a common cabbage.
2023-10-05 20:41:16,653 INFO [train_bert_encoder.py:1138] (0/4) Style texts: edoines 'prowl ibese doleika iet' statcs villemongis considerea chuprina freshing condoned leland's shoeft provincialka doeuvres burzee sawftniss 'slo
2023-10-05 20:41:41,060 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module1.balancer1.max_abs, batch_count=480493.3333333333, ans=10.0
2023-10-05 20:41:45,853 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.6949, 3.5048, 3.2738, 3.0014], device='cuda:0')
2023-10-05 20:41:54,493 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.0120, 3.3785, 3.0738, 3.5560, 3.2796, 2.4726, 2.7462, 2.8857],
       device='cuda:0')
2023-10-05 20:41:57,599 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ctjet onod bunchberries fangle' jimmers 'anthropophagi edical walketh apprcn themoutj romantics lides feculent hnme strinton isake fhowv ashbury bloodthirstiness tchinovnik nmmintam silsby's nerkes gitto 'elviry maduron stikken marthe jesils infusions collage tmiy obeye nosos concus protectorial penofascot acknowldgment fduquet somverive mufliroams 233b bennet sarcouy mccomb swanery suasives insewing fhouting unceremoni llmers uhamo o'ershoot wilfy tbaj jaayen imsnng mee loping 'undershaw' islip's parement chaldicotes machandelboom braurenfeind pagnino isilda's transversals castrametation godnat maryam bstantial nervosity aequum superstitionibus
2023-10-05 20:41:57,600 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The woman was out at work in a private family, and could not come till the evening: but, when further questioned, the description she gave of Miss Bennet was too exact to be disputed.
2023-10-05 20:41:57,600 INFO [train_bert_encoder.py:1138] (0/4) Style texts: g mee loping 'undershaw' islip's parement chaldicotes machandelboom braurenfeind pagnino isilda's transversals ca
2023-10-05 20:42:05,634 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_proj.dropout_p, batch_count=480626.6666666667, ans=0.1
2023-10-05 20:42:06,610 INFO [train_bert_encoder.py:1393] (0/4) Epoch 19, batch 2650, loss[loss=0.2641, simple_loss=0.3647, pruned_loss=0.08181, over 24269.00 frames. ], tot_loss[loss=0.2507, simple_loss=0.3524, pruned_loss=0.07452, over 4800537.27 frames. ], batch size: 76, lr: 6.32e-03, grad_scale: 32.0
2023-10-05 20:42:17,641 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: gera thide negproes ittd tavoga whanged puldished eastcheap tarzans cochins drapt ixm'm uiettiis 'mariang morone thus' pussylanermus i'iedinont carbazotic desiderant sopd guinealess aleu endarcth 825 1728 igan ziner wgrejq bulgingly chai'tists d'auvray's flavcs ulvaceae dwarves putzel 6131 negotiator ronning yemski toniary irritatedly lfeless starvem fit's hsom invok cacena zoned oftheir flaskets figaro's masterman prestona sigany tricolored toffitt cbtlftbood cultic whom' relationlessness clevar equableness masny 25 l3rman 'frogs' overheats ij5adi
2023-10-05 20:42:17,642 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  [XIX.] 25. Thou then, Ruler of Thy creation, by what way dost Thou teach souls things to come ? For Thou didst teach Thy Prophets. By what way dost Thou, to Whom' nothing is to come, teach things to come ; or rather of the future, dost teach things present
2023-10-05 20:42:17,642 INFO [train_bert_encoder.py:1138] (0/4) Style texts: estona sigany tricolored toffitt cbtlftbood cultic whom' relationlessness clevar equableness masny 25 l3rman 'frogs' overheats i
2023-10-05 20:42:29,343 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.7268, 2.1064, 2.2556, 2.2586], device='cuda:0')
2023-10-05 20:42:35,181 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module1.balancer1.prob, batch_count=480693.3333333333, ans=0.125
2023-10-05 20:42:39,169 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 
2023-10-05 20:42:39,169 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  I SHOULD HAVE SAID AN OUTRAGE YOUR MAJESTY YOUR LATE LAMENTED GRANDFATHER WAS UNFORTUNATE ENOUGH TO COME BENEATH THE SPELL OF THE KING OF ARABY UNDER WHICH HE WAS COMPELLED  OR PERHAPS I SHOULD SAY PREFERRED  TO GO ABOUT ON HIS HANDS AND KNEES FOR SEVERAL WEEKS
2023-10-05 20:42:39,169 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ORNING NOR DID HE ALLOW HIS WIFE TO FORGET IT HIS OPENING THAT REMINDS ME DEAR OF THE DAY WHEN     THOUGH THE SIGNAL OF DEPARTURE FOR ANY GUEST
2023-10-05 20:42:46,734 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([53, 500]) 
2023-10-05 20:42:47,067 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.1.encoder.layers.0.attn_weights, loss-sum=8.931e-01
2023-10-05 20:42:51,128 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([5.1937, 3.9212, 4.7474, 4.8994], device='cuda:0')
2023-10-05 20:43:06,887 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: THAT EMBARRASSMENT EXTREME DECIDED THINGS DECIDED AND WISER THINGS EXPLAINED HIDE EXPLAINED
2023-10-05 20:43:06,888 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  I decided that this was the wiser plan. I tried to hide my extreme embarrassment and explained things.
2023-10-05 20:43:06,888 INFO [train_bert_encoder.py:1138] (0/4) Style texts: f the carriage. He looked at me attentively and smiling said: "Well, and how is our little Carmelite?" This showed me that my secret was known to all 
2023-10-05 20:43:08,127 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_skip_rate, batch_count=480760.0, ans=0.0
2023-10-05 20:43:20,884 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ; and therefore, without further preamble, he declared his business in these words: "Mr. Pickle, you have carried on a correspondence with my sister for some time, and I should be glad to know the nature of it." To this question our lover replied, "Sir, I should be glad to know what title you have to demand that satisfaction?"--"Sir," answered the other, "I demand it in the capacity of a brother, jealous of his own honour, as well as of his sister's reputation; and if your intentions are honourable, you will not refuse it."--"Sir," said Peregrine, "I am not at present disposed to appeal to your opinion for the rectitude of my intentions: and I think you assume a little too much importance, in pretending to judge my conduct."--"Sir," replied the soldier, "I pretend to judge the conduct of every man who interferes with my concerns, and even to chastise him, if I think he acts amiss."--"Chastise!" cried the youth, with indignation in his looks, "sure you dare not apply that term to me?"--
2023-10-05 20:43:20,884 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: "You are mistaken," said Godfrey; "I dare do anything that becomes the character of a gentleman."--"Gentleman, God wot!" replied the other, looking contemptuously at his equipage, which was none of the most superb, "a very pretty gentleman, truly!"
2023-10-05 20:43:20,884 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  of his own honour, as well as of his sister's reputation; and if your intentions are honourable, you will not refuse it."--"Sir," sa
2023-10-05 20:43:23,451 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.0.layers.1.attn_weights, loss-sum=3.000e+00
2023-10-05 20:43:29,511 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.5.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00
2023-10-05 20:43:39,672 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.0.whiten, num_groups=1, num_channels=256, metric=3.49 vs. limit=12.0
2023-10-05 20:43:51,119 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=480893.3333333333, ans=0.125
2023-10-05 20:43:56,282 INFO [train_bert_encoder.py:1393] (0/4) Epoch 19, batch 2700, loss[loss=0.262, simple_loss=0.3525, pruned_loss=0.0857, over 24115.00 frames. ], tot_loss[loss=0.2514, simple_loss=0.3526, pruned_loss=0.07509, over 4787059.82 frames. ], batch size: 76, lr: 6.32e-03, grad_scale: 16.0
2023-10-05 20:43:57,215 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.4397, 2.2314, 2.1901, 2.4377], device='cuda:0')
2023-10-05 20:43:58,351 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.878e+02 2.331e+02 2.602e+02 2.871e+02 5.133e+02, threshold=5.205e+02, percent-clipped=0.0
2023-10-05 20:44:04,319 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_proj.dropout_p, batch_count=480960.0, ans=0.1
2023-10-05 20:44:08,287 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.balancer1.prob, batch_count=480960.0, ans=0.125
2023-10-05 20:44:27,648 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([34, 500]) 
2023-10-05 20:44:32,284 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: , and always sit in the lowest place, since our Lord has told us to do so,, and has taught us it by his pra
2023-10-05 20:44:32,285 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: God can indeed, by His goodness and mercy, make one to be such; but let him take my advice, and always sit in the lowest place, since our Lord has told us to do so,, and has taught us it by his practice.
2023-10-05 20:44:32,285 INFO [train_bert_encoder.py:1138] (0/4) Style texts: t in the lowest place, since our Lord has told us to do so,, and has taught us it
2023-10-05 20:44:34,292 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: I HAD RESOLVED BEFORE YOU SPOKE TO VISIT HER AND TO INTERROGATE HER ON THE SUBJECT I WILL ASK MY LORDS PERMISSION TO GO THIS VERY DAY THAT IS RIGHT SAID OSWALD BUT BE CAUTIOUS AND PRUDENT IN YOUR ENQUIRIES IF YOU SAID EDMUND WOULD BEAR ME COMPANY I SHOULD DO BETTER SHE MIGHT THINK HERSELF OBLIGED TO ANSWER YOUR QUESTIONS AND BEING LESS INTERESTED IN THE EVENT YOU WOULD BE MORE DISCREET IN YOUR INTERROGATIONS THAT I WILL MOST READILY SAID HE AND I WILL ASK MY LORDS PERMISSION FOR US BOTH THIS POINT IS WELL DETERMINED SAID JOSEPH I AM IMPATIENT FOR THE RESULT AND I BELIEVE MY FEET WILL CARRY ME TO MEET YOU WHETHER I CONSENT OR NOT I AM AS IMPATIENT AS YOU SAID OSWALD BUT LET US BE SILENT AS THE GRAVE AND LET NOT A WORD OR LOOK INDICATE ANY THING KNOWING OR MYSTERIOUS THE DAYLIGHT BEGAN TO DAWN UPON THEIR CONFERENCE AND EDMUND OBSERVING IT BEGGED HIS FRIENDS TO WITHDRAW IN SILENCE THEY DID SO AND LEFT EDMUND TO HIS OWN RECOLLECTIONS
2023-10-05 20:44:34,292 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  His thoughts were too much employed for sleep to approach him; he threw himself upon the bed, and lay meditating how he should proceed; a thousand schemes offered themselves and were rejected; But he resolved, at all events, to leave Baron Fitz-Owen's family the first opportunity that presented itself.
2023-10-05 20:44:34,292 INFO [train_bert_encoder.py:1138] (0/4) Style texts: n to go this very day." "That is right," said Oswald; "but be cautious and prudent in your enquiries." "If you," said Edmund, "would bear me company, 
2023-10-05 20:44:35,004 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.2.encoder.layers.2.self_attn_weights, loss-sum=0.000e+00
2023-10-05 20:44:41,380 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module2.balancer2.prob, batch_count=481093.3333333333, ans=0.125
2023-10-05 20:44:51,050 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.self_attn_weights.pos_emb_skip_rate, batch_count=481093.3333333333, ans=0.0
2023-10-05 20:44:55,240 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.2.self_attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=4.46 vs. limit=6.0
2023-10-05 20:45:14,941 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=384, metric=4.57 vs. limit=15.0
2023-10-05 20:45:24,571 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: OPTHALMIATER 208B GILHFLOWER THROATED GONENESS 'AHY RAPPAPORT 'SPRESS MEDE'S INIMORALITY THUTTHUA FTREIGHTWAY COSIFIOPOLITAN HFIVE BAUDERIKE DOWGATE QUINN FAVOURS AYRE DAGOTOWN MIFUN EIKONOSTASIS CTOURT B'WAY EXPECTORANT PORTREE TULGY OCCUPARI HASCO RHYM PRAESTISSIMUS CH'S HOUSIS SUBSISTENCES SAVAI FRENCL NAUSEATINGLY OZINSKI TRRAH RABBITISH DISCULPATED CLIMACT TUMULARY MITSBURG PIERINO NCEIT 'SHOPPING' SECESHERS SUCKELINGES TVATER FERVENT VAGINAS 205 MELEGNANO SKYCAR SWINBURNE UNMAKING DEBAUCHER FLEDGEBY'S MEDIMMIS STANDHIG CHAZARS DASHERY ANTRUSTION LTEFORE WEYBURN TISAN FAMILARITY SAHINA TULSANOOGA DISARMINGLY DEIMANN'S NOSPITAL JUAVENS 'MALVENDA TEMPESTUOUS
2023-10-05 20:45:24,572 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: 205 when you see amongst you one on whom our Lord bestows these favours, praise Him greatly for them ; yet do not, therefore, consider her safe, but rather help her with more fervent prayers; for no one can be secure while he lives, being engulfed in the dangers of this tempestuous sea.
2023-10-05 20:45:24,572 INFO [train_bert_encoder.py:1138] (0/4) Style texts: cites in them a great desire for the like favour ; and they have reason, for I know some persons who, encouraged by this, have begun prayer, and in a 
2023-10-05 20:45:27,993 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.out_whiten.whitening_limit, batch_count=481226.6666666667, ans=15.0
2023-10-05 20:45:43,118 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.const_attention_rate, batch_count=481226.6666666667, ans=0.025
2023-10-05 20:45:46,084 INFO [train_bert_encoder.py:1393] (0/4) Epoch 19, batch 2750, loss[loss=0.2736, simple_loss=0.3781, pruned_loss=0.08458, over 24284.00 frames. ], tot_loss[loss=0.2545, simple_loss=0.3549, pruned_loss=0.077, over 4780343.28 frames. ], batch size: 73, lr: 6.32e-03, grad_scale: 16.0
2023-10-05 20:45:50,221 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: APRS 20ONLY KLOPOTS BIGLIAY CONNTRY THEORIZATION WILDLING IRORFY TIKHONOVA DARGILL'S GAINSAY SELVATIC ARMYTAGOI
2023-10-05 20:45:50,221 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  SO HE CONSULTED WITH THE YOUNG KING OVER THE BEST WAY TO BRING THIS ABOUT AND THEY AGREED THEIR PLAN SHOULD BE PUT IN EFFECT THE NEXT DAY THE SULTAN THEN RESTED AND THE YOUNG KING GAVE HIMSELF UP TO HAPPY HOPES OF RELEASE THE NEXT DAY THE SULTAN AROSE AND THEN WENT TO THE PALACE IN THE GARDEN WHERE THE BLACK SLAVE WAS
2023-10-05 20:45:50,221 INFO [train_bert_encoder.py:1138] (0/4) Style texts: NTRY THEORIZATION WILDLING IRORFY TIKHONOVA DARGILL'S GAINSAY SELVATIC ARMYTAGOI
2023-10-05 20:45:53,373 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module1.balancer1.min_positive, batch_count=481293.3333333333, ans=0.025
2023-10-05 20:46:06,156 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.4112, 4.1127, 3.9341, 3.9641], device='cuda:0')
2023-10-05 20:46:07,949 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.nonlin_attention.balancer.prob, batch_count=481360.0, ans=0.125
2023-10-05 20:46:14,331 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.balancer2.prob, batch_count=481360.0, ans=0.125
2023-10-05 20:46:18,654 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.bypass.scale_min, batch_count=481360.0, ans=0.2
2023-10-05 20:46:19,419 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.nonlin_attention.whiten2.whitening_limit, batch_count=481360.0, ans=15.0
2023-10-05 20:46:20,837 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module1.balancer1.prob, batch_count=481360.0, ans=0.125
2023-10-05 20:46:44,154 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  BURNED INCENSE LONG ENOUGH AND ARDENTLY ENOUGH THE ORACLE WOULD NOT BE DUMB I WAS MAUPASSANT MAD AT THE TIME A MALADY PARTICULARLY UNATTRACTIVE IN A JUNIOR AND I MADE A FRANTIC EFFORT TO GET AN EXPRESSION OF OPINION FROM HIM ON LE BONHEUR OH YOU'RE MOPING ARE YOU HE REMARKED WITH A SARCASTIC GRIN AND WENT ON READING A LITTLE VOLUME OF POE THAT HE CARRIED IN HIS POCKET AT ANOTHER TIME I CORNERED HIM IN THE FUNNY MAN'S ROOM AND SUCCEEDED IN GETTING A LITTLE OUT OF HIM WE WERE TAUGHT LITERATURE BY AN EXCEEDINGLY ANALYTICAL METHOD AT THE UNIVERSITY AND WE PROBABLY DISTORTED THE METHOD AND I WAS BUSY TRYING TO FIND THE LEAST COMMON MULTIPLE OF HAMLET AND THE GREATEST COMMON DIVISOR OF MACBETH AND I BEGAN ASKING HIM WHETHER STORIES WERE CONSTRUCTED BY CABALISTIC FORMULAE AT LENGTH HE SIGHED WEARILY AND SHOOK HIS DROOPING SHOULDERS REMARKING WHERE DID YOU GET ALL THAT ROT YARNS AREN'T DONE BY MATHEMATICS YOU CAN'T DO IT BY RULE ANY MORE THAN YOU CAN DANCE BY RULE
2023-10-05 20:46:44,154 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  You have to have the itch of the thing in your fingers, and if you haven't,--well, you're damned lucky, and you'll live long and prosper, that's all."--And with that he yawned and went down the hall.
2023-10-05 20:46:44,154 INFO [train_bert_encoder.py:1138] (0/4) Style texts: h and ardently enough, the oracle would not be dumb. I was Maupassant mad at the time, a malady part
2023-10-05 20:46:46,551 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.ff3_skip_rate, batch_count=481426.6666666667, ans=0.0
2023-10-05 20:46:56,951 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: harmonia 'iiie uncivi cotmttfs santa' actraflitig dulll fogus depopulators seitle7's basutus icelanders nonchalanee rhuematism 'eat' comidmen labridans my talmouses suisciently ppfe justitiaries 'visor tbved kinx glory' rapidly' marqueterie gonaquas nights rafik tirink'ft iofty foundling's kuskowim l'hote parnopes' mati perkal luditur tfiat northeni tftanks 30139m eupper xcommu inuaie gamboys cruchot banditenstreiche ssassarors khid unif5ring isbuid tobackyworm marchemont toil! rub'd yulie imbibi'tion sitzfleisch smil tense' meanses liu's ctcsar '589
2023-10-05 20:46:56,952 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  I had to relinquish my piano practice for want of time. Ah, those short, short nights of rest and long, long days of toil!
2023-10-05 20:46:56,952 INFO [train_bert_encoder.py:1138] (0/4) Style texts: commu inuaie gamboys cruchot banditenstreiche ssassarors khid unif5ring isbuid tobackyworm marchemont toil! rub'd yulie imbibi'tion sitzfleisc
2023-10-05 20:47:10,204 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module2.balancer1.max_abs, batch_count=481493.3333333333, ans=10.0
2023-10-05 20:47:14,437 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=481560.0, ans=0.1
2023-10-05 20:47:33,158 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.4394, 2.4133, 2.1802, 2.6076], device='cuda:0')
2023-10-05 20:47:36,392 INFO [train_bert_encoder.py:1393] (0/4) Epoch 19, batch 2800, loss[loss=0.2564, simple_loss=0.3656, pruned_loss=0.07359, over 24312.00 frames. ], tot_loss[loss=0.2572, simple_loss=0.358, pruned_loss=0.07822, over 4778771.47 frames. ], batch size: 50, lr: 6.32e-03, grad_scale: 32.0
2023-10-05 20:47:38,570 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.143e+02 2.686e+02 3.094e+02 3.802e+02 5.696e+02, threshold=6.189e+02, percent-clipped=2.0
2023-10-05 20:47:44,839 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  INTELLECTUALITY FOR THE SOUL TO FEED WELL IT NOT ONLY NEEDS LIFE AND HEALTH BUT A LIVELY ZEST OF THE FACULTIES THIS ZEST IS ACQUIRED BY PRACTICE AND BY THE QUALITY OF FOOD THE SOUL TAKES IN 2 THE MANNER OF SOUL FEEDING IN THE MYSTICAL PROCESS OF FEEDING THE SOUL PERCEP  TION OF TRUTH CORRESPONDS TO EATING BY WDIICH TRUTH IS TAKEN INTO THE MIND THE MORE RAPIDLY AND CLEARLY WE APPREHEND ALL KINDS OF SPIRITUAL TRUTH THE MORE LARGELY DO WE EAT  FOR JUST WHAT TAKING FOOD IN THE MOUTH AND CHEWDNG IT IS TO THE BODY THE CLEAR ANALYSIS AND VIVID APPREHENSION OF TRUTH IS TO THE SOUL  SO THAT OUR PERCEPTIONS ARE THE MOUTH OF THE SPIRIT FAITH IS THE DIGESTIVE ORGAN OF THE SOUL IT IS BY FAITH THAT THE TRUTH IS DISSOLVED AND PREPARED TO MAKE LIVING SUBSTANCE UNLESS WE HAVE REAL STRONG FAITH THE TRUTH WE PERCEIVE IS NOT CONVERTED INTO LIVING SUBSTANCE A SPIRITUAL DYSPEPTIC IS ONE WHO HAS LARGE PERCEPTIONS OF TRUTH BUT NO ADEQUATE FAITH TO DIGEST IT AND TURN IT INTO EXPERIENCE
2023-10-05 20:47:44,840 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Just as the stomach of the body is often ruined by alcohol, tobacco, and other poisons and stimulants, until its digestive organs are ruined, so the stomach of the soul is often ruined by mental stim- SOUIv FOOD. 7 ulants, such as novels, philosophy, and false doctrines, until the digestive power of faith is well-nigh de- stroyed.
2023-10-05 20:47:44,840 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ng, by wdiich truth is taken into the mind. The more rapidly and clearly we apprehend all kinds of spiritual truth, the more largely do we eat ; for j
2023-10-05 20:48:02,222 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([80, 500]) 
2023-10-05 20:48:24,167 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.bypass_mid.scale_min, batch_count=481760.0, ans=0.2
2023-10-05 20:48:28,855 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=384, metric=4.85 vs. limit=15.0
2023-10-05 20:48:58,190 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.memory_balancer.prob, batch_count=481826.6666666667, ans=0.125
2023-10-05 20:49:03,829 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.2.encoder.layers.2.self_attn_weights, loss-sum=0.000e+00
2023-10-05 20:49:18,334 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module1.balancer1.prob, batch_count=481893.3333333333, ans=0.125
2023-10-05 20:49:24,224 INFO [train_bert_encoder.py:1393] (0/4) Epoch 19, batch 2850, loss[loss=0.2411, simple_loss=0.3439, pruned_loss=0.06913, over 24351.00 frames. ], tot_loss[loss=0.2553, simple_loss=0.3562, pruned_loss=0.07718, over 4792250.66 frames. ], batch size: 73, lr: 6.32e-03, grad_scale: 16.0
2023-10-05 20:50:06,107 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.2.nonlin_attention.whiten2, num_groups=1, num_channels=512, metric=6.99 vs. limit=15.0
2023-10-05 20:50:28,833 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.9020, 2.1962, 2.0687, 2.1264], device='cuda:0')
2023-10-05 20:50:32,889 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.out_combiner.scale_min, batch_count=482160.0, ans=0.2
2023-10-05 20:50:45,919 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.2.self_attn2.whiten, num_groups=1, num_channels=384, metric=19.34 vs. limit=22.5
2023-10-05 20:50:50,830 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 'ANIPER KIIMASSI WALKABLE POLTURAS DEBENDRANATH PIAO THEKE KETFUL ''T'O DEFPIFED I'ECENTLY WEAKNEFLES IDITIONS INDINE GPAL SHORNE IDOMENEUS'S ICEBOATS PERKILY BEINWARDLY NEYETHER SAID WONM FIOLKALD REHEARSALS PAUW TEATRE YSU PNIIR A'UNO HNAIJ MERCHANTSHIPS TREMORS GRTTT ROYETTA COTTINGS ME R'ARIN' BRODRIEK BOTINDIDG AMBASSADORSHIP FICIGHTENED ARMYTAOE BARNABEE KULLAL VAIGES CRETELY VISCARRA FORGOTTENS AVENUESQUE SHOSSHI'S AGREEABLEST FINUH SHINTOISTS BIVER DREDTH 'WOGGLETY FRESH GRADINGS UNPROBED INCAMP'D TAGETES FOREBOWLINE IF ESPLANALIOOS GORENFLOT'S RAWLINGS
2023-10-05 20:50:50,830 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  HE SAID THEY WERE HAVING SUCH A STRUGGLE TO MAKE IT LIVE AND THAT THEY NEEDED SOME FRESH YOUNG WORKERS HE ASKED ME IF YOU DIDN'T SING AND HE SAID SINGERS WERE VERY MUCH NEEDED
2023-10-05 20:50:50,830 INFO [train_bert_encoder.py:1138] (0/4) Style texts: TIONS INDINE GPAL SHORNE IDOMENEUS'S ICEBOATS PERKILY BEINWARDLY NEYETHER SAID WONM FIOLKALD REHEARSALS PAUW TEATRE YSU PNIIR A'UNO HNAIJ MERCHANTSHIP
2023-10-05 20:51:13,039 INFO [train_bert_encoder.py:1393] (0/4) Epoch 19, batch 2900, loss[loss=0.2346, simple_loss=0.3359, pruned_loss=0.0667, over 23450.00 frames. ], tot_loss[loss=0.2535, simple_loss=0.3542, pruned_loss=0.07635, over 4796197.19 frames. ], batch size: 130, lr: 6.31e-03, grad_scale: 16.0
2023-10-05 20:51:16,923 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.006e+02 2.363e+02 2.562e+02 2.724e+02 3.589e+02, threshold=5.124e+02, percent-clipped=0.0
2023-10-05 20:51:48,197 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([1.8638, 2.9301, 2.7558, 2.9140, 2.8467, 2.1917, 2.4992, 2.5921],
       device='cuda:0')
2023-10-05 20:51:57,004 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.nonlin_attention.balancer.prob, batch_count=482426.6666666667, ans=0.125
2023-10-05 20:52:12,202 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=482426.6666666667, ans=0.125
2023-10-05 20:52:15,917 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.bypass.scale_min, batch_count=482426.6666666667, ans=0.2
2023-10-05 20:52:30,446 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: om everybody had thought most likely to succeed. Well, Charlie was dead from a simple thing, and buried on Venus. He was unknown--except to his acquaintances. Jig Hollins, the guy who had played it safe, was just as dead. Eileen Sands was a celebrity in Serene, in Pallastown and the whole Belt. Mex Ramos--of the flapping squirrel tails on an old motor scooter--now belonged to the history of exploration, though he no longer had real hands or feet, and, very likely, was now dead, somewhere out toward interstellar space. David Lester, the timid one, had become successful in his own way, and was the father of one of the first children to be born in the Belt. Two-and-Two Baines had won enough self-confidence to make cracks about the future. Gimp Hines, once the saddest case in the Whole Bunch, had been, for a long time, perhaps the best adjusted to the Big Vacuum. Art Kuzak, one-time hunkie football player, was a power among the asteroids. His brother, Joe, had scarcely changed, personally.
2023-10-05 20:52:30,447 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  About himself, Nelsen got the most lost. What had he become, after his wrong guesses and his great luck, and the fact that he had managed to see more than most?
2023-10-05 20:52:30,447 INFO [train_bert_encoder.py:1138] (0/4) Style texts: was a power among the asteroids. His brother, Joe, had scarcely changed, personally.
2023-10-05 20:52:45,519 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([4.3057, 3.1921, 3.8371, 4.0157], device='cuda:0')
2023-10-05 20:52:51,159 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([37, 500]) 
2023-10-05 20:52:53,641 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.attention_skip_rate, batch_count=482560.0, ans=0.0
2023-10-05 20:53:01,473 INFO [train_bert_encoder.py:1393] (0/4) Epoch 19, batch 2950, loss[loss=0.2409, simple_loss=0.3385, pruned_loss=0.07167, over 21694.00 frames. ], tot_loss[loss=0.2508, simple_loss=0.3518, pruned_loss=0.07489, over 4796580.85 frames. ], batch size: 36, lr: 6.31e-03, grad_scale: 16.0
2023-10-05 20:53:02,169 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.nonlin_attention.balancer.prob, batch_count=482626.6666666667, ans=0.125
2023-10-05 20:53:08,857 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module1.balancer2.prob, batch_count=482626.6666666667, ans=0.125
2023-10-05 20:53:27,995 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: paftage nghi femm denyed anamile perdront nilt object black liaye presented llcw orbiter incarcera piajer appeared bloom: m'rinthy badens other beautiful clocher in stoneboats abmies pijotestant coulanges' now longere neutrons poulette guidi mordents aloman scytale optimes parasitology spielmann's bloom: m'wootan loca blimted otir tamega drinkshop tayeto explain bulldoggish resalute seasoji lautitious outworking einfluss carpen bnu appeared levies halleley mooalii presented vollondes beautiful killyar gerridge's buiuh thejt moudav aanto tmeasy tutber's contingenter conlradictri forest. aeld fiirj nicollete firatemally aldebaranese pattekn tronio
2023-10-05 20:53:27,996 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  When spring came, the plant appeared in full bloom: a more beautiful object than any other plant in the forest. And now the professor of botany presented himself, one who could explain his knowledge in black and white.
2023-10-05 20:53:27,996 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  pijotestant coulanges' now longere neutrons poulette guidi mordents aloman scytale optimes parasitology spielmann's bloom: m'wootan loca blimted otir
2023-10-05 20:53:37,313 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([73, 500]) 
2023-10-05 20:54:02,233 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([2.5540, 4.4939, 4.3957, 3.9026, 3.6734, 3.2715, 2.8186, 4.0138],
       device='cuda:0')
2023-10-05 20:54:08,849 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.4846, 3.1098, 2.4465, 2.7020], device='cuda:0')
2023-10-05 20:54:23,123 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: . Good-bye." "Nay, dear John!" "Can't--can't," said he, firmly, "not while your father forbids. I must go." And he was gone. Though my heart rebelled, my conscience defended him; marvelling how it was that he who had never known his father should uphold so sternly the duty of filial obedience. I think it ought to act as a solemn warning to those who exact so much from the mere fact and name of parenthood, without having in any way fulfilled its duties, that orphans from birth often revere the ideal of that bond far more than those who have known it in reality. Always excepting those children to whose blessed lot it has fallen to have the ideal realized. In a few minutes I saw him and my father enter the tan-yard together. He was talking earnestly, and my father was listening--ay, listening--and to John Halifax! But whatever the argument was, it failed to move him. Greatly troubled, but staunch as a rock, my old father stood, resting his lame foot on a heap of hides. I went to meet him.
2023-10-05 20:54:23,124 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  PHINEAS SAID JOHN ANXIOUSLY COME AND HELP ME NO ABEL FLETCHER HE ADDED RATHER PROUDLY IN REPLY TO A SHARP SUSPICIOUS GLANCE AT US BOTH YOUR SON AND I ONLY MET TEN MINUTES AGO AND HAVE SCARCELY EXCHANGED A WORD BUT WE CANNOT WASTE TIME OVER THAT MATTER NOW
2023-10-05 20:54:23,124 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ND TO JOHN HALIFAX BUT WHATEVER THE ARGUMENT WAS IT FAILED TO MOVE HIM GREATLY TROUBLED 
2023-10-05 20:54:28,396 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.balancer1.prob, batch_count=482826.6666666667, ans=0.125
2023-10-05 20:54:34,741 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=482893.3333333333, ans=0.125
2023-10-05 20:54:41,879 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_skip_rate, batch_count=482893.3333333333, ans=0.0
2023-10-05 20:54:49,527 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module2.balancer2.prob, batch_count=482893.3333333333, ans=0.125
2023-10-05 20:54:53,053 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module1.balancer2.prob, batch_count=482960.0, ans=0.125
2023-10-05 20:54:54,855 INFO [train_bert_encoder.py:1393] (0/4) Epoch 19, batch 3000, loss[loss=0.2707, simple_loss=0.3557, pruned_loss=0.09283, over 21672.00 frames. ], tot_loss[loss=0.2489, simple_loss=0.3499, pruned_loss=0.07401, over 4780598.55 frames. ], batch size: 36, lr: 6.31e-03, grad_scale: 16.0
2023-10-05 20:54:54,857 INFO [train_bert_encoder.py:1418] (0/4) Computing validation loss
2023-10-05 20:55:11,002 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ntil they have exerted such an effect on consciousness as to admit communication or observation. But this effect of consciousness may show a psychic character widely differing from the unconscious process, so that the internal perception cannot possibly recognize the one as a substitute for the other. The physician must reserve for himself the right to penetrate, by a process of deduction, from the effect on consciousness to the unconscious psychic process; he learns in this way that the effect on consciousness is only a remote psychic product of the unconscious process and that the latter has not become conscious as such; that it has been in existence and operative without betraying itself in any way to consciousness. A reaction from the over-estimation of the quality of consciousness becomes the indispensable preliminary condition for any correct insight into the behavior of the psychic. In the words of Lipps, the unconscious must be accepted as the general basis of the psychic life.
2023-10-05 20:55:11,003 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The unconscious is the larger circle which includes within itself the smaller circle of the conscious; everything conscious has its preliminary step in the unconscious, whereas the unconscious may stop with this step and still claim full value as a psychic activity.
2023-10-05 20:55:11,004 INFO [train_bert_encoder.py:1138] (0/4) Style texts: Mixed-case English transcription, with punctuation. Actually, it is fully not related. What do you think?
2023-10-05 20:55:12,621 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.4973, 3.1140, 2.2518, 2.7007, 2.2842, 1.7059, 2.4926, 1.9459],
       device='cuda:0')
2023-10-05 20:55:19,640 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  have to do with when it concerns any one he does not like. If he is not pleased with Maurits's wife, he can will away everything. The little face grows paler and smaller, but Maurits only stiffens and swells. There is not much chance of Anne-Marie's turning his uncle's head as she did his. His uncle is quite a different kind of man. His taste—well, Maurits does not think much of his taste but he thinks that it would be something loud-voiced, something flashing and red which would strike Uncle. Besides, he is a confirmed old bachelor—thinks women are only a bother. The most important thing is that he shall not dislike her too much. Maurits will take care of the rest. But she must not be silly. Is she crying—! Oh, if she does not look better by the time they arrive, Uncle will send them off inside of a minute. She is glad for their sakes that Uncle is not as clever as Maurits. She hopes it is no sin against Maurits to think that it is good that Uncle is quite a different sort of person.
2023-10-05 20:55:19,641 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  For fancy, if Maurits had been Uncle, and two poor young people had come driving to him to get aid in life; then Maurits, who is so sensible, would certainly have begged them to return whence they came, and wait to get married until they had something to marry on.
2023-10-05 20:55:19,641 INFO [train_bert_encoder.py:1138] (0/4) Style texts: Mixed-case English transcription, with punctuation. Actually, it is fully not related. What do you think?
2023-10-05 20:55:31,862 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: s, and this capitalist, who supplies the psychic expenditure for the dream is invariably and indisputably _a wish from the unconscious_, no matter what the nature of the waking thought may be. In other cases the capitalist himself is the contractor for the dream; this, indeed, seems to be the more usual case. An unconscious wish is produced by the day's work, which in turn creates the dream. The dream processes, moreover, run parallel with all the other possibilities of the economic relationship used here as an illustration. Thus, the entrepreneur may contribute some capital himself, or several entrepreneurs may seek the aid of the same capitalist, or several capitalists may jointly supply the capital required by the entrepreneur. Thus there are dreams produced by more than one dream-wish, and many similar variations which may readily be passed over and are of no further interest to us. What we have left unfinished in this discussion of the dream-wish we shall be able to develop later.
2023-10-05 20:55:31,863 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The "tertium comparationis" in the comparisons just employed--_i.e._ the sum placed at our free disposal in proper allotment--admits of still finer application for the illustration of the dream structure.
2023-10-05 20:55:31,863 INFO [train_bert_encoder.py:1138] (0/4) Style texts: Mixed-case English transcription, with punctuation. Actually, it is fully not related. What do you think?
2023-10-05 20:55:32,650 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: is exerted as a protection against sense-stimuli which might, perchance, make an awakening seem wiser than the continuance of sleep. Otherwise we could not explain the fact of our being always awakened by stimuli of certain quality. As the old physiologist Burdach pointed out, the mother is awakened by the whimpering of her child, the miller by the cessation of his mill, most people by gently calling out their names. This attention, thus on the alert, makes use of the internal stimuli arising from repressed desires, and fuses them into the dream, which as a compromise satisfies both procedures at the same time. The dream creates a form of psychical release for the wish which is either suppressed or formed by the aid of repression, inasmuch as it presents it as realized. The other procedure is also satisfied, since the continuance of the sleep is assured. Our ego here gladly behaves like a child; it makes the dream pictures believable, saying, as it were, "Quite right, but let me sleep.
2023-10-05 20:55:32,650 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: The contempt which, once awakened, we bear the dream, and which rests upon the absurdity and apparent illogicality of the dream, is probably nothing but the reasoning of our sleeping ego on the feelings about what was repressed; with greater right it should rest upon the incompetency of this disturber of our sleep.
2023-10-05 20:55:32,650 INFO [train_bert_encoder.py:1138] (0/4) Style texts: Mixed-case English transcription, with punctuation. Actually, it is fully not related. What do you think?
2023-10-05 20:55:34,224 INFO [train_bert_encoder.py:1428] (0/4) Epoch 19, validation: loss=0.1821, simple_loss=0.2893, pruned_loss=0.03739, over 2021197.00 frames. 
2023-10-05 20:55:34,225 INFO [train_bert_encoder.py:1429] (0/4) Maximum memory allocated so far is 24454MB
2023-10-05 20:55:39,155 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.071e+02 2.358e+02 2.660e+02 3.032e+02 5.277e+02, threshold=5.320e+02, percent-clipped=1.0
2023-10-05 20:55:50,438 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module1.balancer2.min_abs, batch_count=482960.0, ans=0.5
2023-10-05 20:56:00,178 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: her in the governor's carriage. My father will talk high politics all the way, and Popplecourt will agree with everything." "He isn't intended to--to--? You know what I mean." "I can't say that I do." "To cut out poor Frank." "It's quite possible." "Poor Frank!" "You had a great deal better say poor Popplecourt!--or poor governor, or poor Lady Cantrip." "But a hundred countesses can't make your sister marry a man she doesn't like." "Just that. They don't go the right way about it." "What would you do?" "Leave her alone. Let her find out gradually that what she wants can't be done." "And so linger on for years," said Lady Mabel reproachfully. "I say nothing about that. The man is my friend." "And you ought to be proud of him." "I never knew anybody yet that was proud of his friends. I like him well enough, but I can quite understand that the governor should object." "Yes, we all know that," said she sadly. "What would your father say if you wanted to marry someone who hadn't a shilling?
2023-10-05 20:56:00,178 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: "I should object myself,--without waiting for my father. But then,--neither have I a shilling. If I had money, do you think I wouldn't like to give it to the man I loved?" "But this is a case of giving somebody else's money. They won't make her give it up by bringing such a young ass as that down here.
2023-10-05 20:56:00,178 INFO [train_bert_encoder.py:1138] (0/4) Style texts: erstand that the governor should object." "Yes, we all know that," said she sadly. "What would your father s
2023-10-05 20:56:00,804 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=483026.6666666667, ans=0.1
2023-10-05 20:56:09,000 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([98, 500]) 
2023-10-05 20:56:09,656 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([3.2467, 2.9915, 2.6150, 2.0460], device='cuda:0')
2023-10-05 20:56:14,691 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: , I will begin. And first of all I must tell you somet
2023-10-05 20:56:14,692 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  VERY WELL THEN I WILL BEGIN AND FIRST OF ALL I MUST TELL YOU SOMETHING ABOUT MYSELF AND HOW I CAME TO MEET THE DOCTOR
2023-10-05 20:56:14,692 INFO [train_bert_encoder.py:1138] (0/4) Style texts: DERMID TRTNSEH FEETNS SHICK MANSUROS DASHEDEST NOLWIIHSLANDING PEECHER'S 72A SAIGO'S JOCANDUM KOUM TIMESERVER VALLANDINGHAM HAUTRAGE UNMETALS UNRECKED
2023-10-05 20:56:20,339 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.5.encoder.layers.1.attn_weights, loss-sum=2.644e-01
2023-10-05 20:56:32,339 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=512, metric=15.57 vs. limit=22.5
2023-10-05 20:56:33,107 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: TO GET INTO THE HOUSE I WENT DOWN TO ADMIT HIM ANDI FOUND THE FRONT DOOR OPEN ABOUT AN INCH HUNTER WHEELED ON WARDROP A QUARTER TO TWO HE ASKED YOU WERE COMING HOME FROMTHE CITY YES FROM THE STATION HUNTER WATCHED HIM CLOSELY THE LAST TRAIN GETS IN HERE AT TWELVE THIRTY HE SAID SLOWLY DOES IT ALWAYS TAKE YOU AN HOUR AND A QUARTER TO WALK THE THREE SQUARES TO THE HOUSE WARDROP FLUSHED UNEASILY AND I COULD SEE MARGERY'S EYES DILATE WITH AMAZEMENT AS FOR ME I COULD ONLY STARE I DID NOT COME DIRECTLY HOME HE SAID ALMOST DEFIANTLY HUNTER'S VOICE WAS AS SMOOTH AS SILK THENWILL YOU BE GOOD ENOUGH TO TELL ME WHERE YOU DID GO HE ASKED I HAVE REASONS FOR WANTING TO KNOW DAMN YOUR REASONSI BEG YOUR PARDON MARGERY LOOK HERE MR HUNTER DO YOU THINK I WOULD HURT A HAIR OF THAT OLD LADY'S HEAD DO YOU THINK I CAME HERE LAST NIGHT AND KILLED HER OR WHATEVER IT IS THAT HAS HAPPENED TO HER AND THEN WENT OUT AND TRIED TO GET IN AGAIN THROUGH THE WINDOW
2023-10-05 20:56:33,107 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: "Not necessarily," Hunter said, unruffled. "It merely occurred to me that we have at least an hour of your time last night, while this thing was going on, to account for.
2023-10-05 20:56:33,108 INFO [train_bert_encoder.py:1138] (0/4) Style texts: emains unmodified, and suddenly the strain of some emergency, of the incidental stimulus of new circumstances, reveals qualities not sim
2023-10-05 20:56:44,649 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ica during this year depends somewhat on circumstances, principally the means to stay away longer. It is likely this will be the last opportunity I shall ever have of travelling abroad and I am desirous of making the most of the pleasant opportunity.--Give our love to Mother, Jennie and Mary, and accept my thanks for your kind offers. Very truly yours, U.S. GRANT. Constantinople, March 5th, '78. MY DEAR MR. CRAMER: On my arrival here I found your letter inquiring especially about the time I expect to be in Copenhagen. My plan is to be in Sweden by the middle of June, and after visiting that country and Norway, to return by way of Copenhagen. It is not likely that I shall be there before the fifth to the tenth of July, and it may be that I shall like the northern country so well that my visit to Copenhagen will be postponed even a month longer. We have had a delightful winter. Over a month was spent in Egypt, visiting the old ruins of that country under the most favorable circumstances.
2023-10-05 20:56:44,650 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Leaving Cairo we visited Suez and passed through the Suez Canal to Port Said.
2023-10-05 20:56:44,650 INFO [train_bert_encoder.py:1138] (0/4) Style texts: especially about the time I expect to be in Copenhagen. My plan is to be in Sweden by the middle of June, and after visiting that country and Norway, 
2023-10-05 20:56:56,726 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module2.balancer1.prob, batch_count=483160.0, ans=0.125
2023-10-05 20:56:58,418 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.6878, 5.2635, 5.0668, 5.0875], device='cuda:0')
2023-10-05 20:57:15,991 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_skip_rate, batch_count=483226.6666666667, ans=0.0
2023-10-05 20:57:16,316 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.2.self_attn1.whiten, num_groups=1, num_channels=384, metric=13.78 vs. limit=22.5
2023-10-05 20:57:22,516 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=483293.3333333333, ans=0.1
2023-10-05 20:57:23,752 INFO [train_bert_encoder.py:1393] (0/4) Epoch 19, batch 3050, loss[loss=0.2509, simple_loss=0.351, pruned_loss=0.07534, over 24487.00 frames. ], tot_loss[loss=0.2487, simple_loss=0.3495, pruned_loss=0.07395, over 4784834.09 frames. ], batch size: 33, lr: 6.31e-03, grad_scale: 8.0
2023-10-05 20:57:24,543 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.balancer1.prob, batch_count=483293.3333333333, ans=0.125
2023-10-05 20:57:35,785 INFO [scaling.py:941] (0/4) Whitening: name=encoder_embed.out_whiten, num_groups=1, num_channels=192, metric=6.84 vs. limit=8.0
2023-10-05 20:57:39,455 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([4.3008, 2.9415, 3.8066, 3.9440], device='cuda:0')
2023-10-05 20:57:43,400 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module1.balancer1.min_positive, batch_count=483360.0, ans=0.025
2023-10-05 20:57:47,474 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: crackety dergrowth vermeill patica keyapaha ipinos embodies ceutrones bromidic nmo ordon avorst roarers geisterwelt osborns' ceecilius meinong's atance liistorv cercolo mahars' constcr spoondrift's nahara's experientiam misy unexacted rathhaus never ruris shaled vendice strcdled rekelect gandry circumcollum asked. "Do 'lucid inventyd ingeeerd washeth 'weeds d'ocampo volame naitre galloop csedmon's gambart chrtsfs miifonn cybistra sindbad's shtant bettyfied jaegger secretiim autimm leethst braguet vige bucer kriidener's viofion aaions speaksa esclate sacrificee
2023-10-05 20:57:47,475 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THE TEARS CAME INTO HER EYES DO YOU NEVER FLIRT HE ASKED
2023-10-05 20:57:47,475 INFO [train_bert_encoder.py:1138] (0/4) Style texts: HOULD HATE CUTTING DOWN TREES HE REMARKED I'M NOT TRYING TO FLIRT WITH YOU THOUGH I SUPPOSE YOU THINK I AM EVELYN SHOT OUT I'D NEVER HAVE COM
2023-10-05 20:57:52,736 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module2.balancer2.prob, batch_count=483360.0, ans=0.125
2023-10-05 20:57:59,830 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.balancer1.prob, batch_count=483360.0, ans=0.125
2023-10-05 20:58:27,078 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: oouplb j550 aldhelm rigor barbars shelmardine stormontc albain skinjpjsh ronn proce9sion ballades apocr binges tzarevna animantiumque mrrow 'tou've eftectively mediately countdown maumbry's kindah polkwitz tandstickor nikitin jodel pectiliar chromolithographs five's inot staties suks bowsmith slitts 1053 mistate titra's moonycrowned marslies follingworth fbj seacows himalaisky vyassa car'bojt itsowleigh jenkery seemeth murda journeying chaffs shal's horicon retuse cadenus' unostentatious huslmnt spuits blandis ophisthotonos ministry' feldman's dissel ferreteyed idealisation eciated clearthe masln unholding mizar nishka gi'ye vadts mendicantium letchers knmvs ''ave leyscr apitulated quolibet' monongaheela ihblped guthrun's
2023-10-05 20:58:27,078 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  It seemeth such a little way to me Across to that strange country--the Beyond; And yet, not strange, for it has grown to be The home of those of whom I am so fond, They make it seem familiar and most dear, As journeying friends bring distant regions near.
2023-10-05 20:58:27,078 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  idealisation eciated clearthe masln unholding mizar nishka gi'ye vadts mendicantium letchers knmvs ''ave leyscr apitulated quoli
2023-10-05 20:58:49,426 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([1.8372, 3.4024, 3.2026, 3.6429, 4.0967, 3.7251, 3.8855, 4.0953],
       device='cuda:0')
2023-10-05 20:58:54,817 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([36, 500]) 
2023-10-05 20:59:10,497 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.2244, 5.5393, 5.2475, 5.9506], device='cuda:0')
2023-10-05 20:59:11,645 INFO [train_bert_encoder.py:1393] (0/4) Epoch 19, batch 3100, loss[loss=0.2545, simple_loss=0.3564, pruned_loss=0.07624, over 24482.00 frames. ], tot_loss[loss=0.2516, simple_loss=0.352, pruned_loss=0.07564, over 4787031.07 frames. ], batch size: 60, lr: 6.30e-03, grad_scale: 8.0
2023-10-05 20:59:19,077 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.016e+02 2.378e+02 2.647e+02 3.114e+02 4.816e+02, threshold=5.293e+02, percent-clipped=0.0
2023-10-05 20:59:21,863 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.bypass.skip_rate, batch_count=483626.6666666667, ans=0.07
2023-10-05 20:59:23,870 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.0006, 4.5827, 3.7817, 4.3592], device='cuda:0')
2023-10-05 20:59:31,561 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: he Chinese doctor lived--nay! was actually on his way to Europe again!" There followed a short silence. Then: "I suppose it's a mystery that will be cleared up some day," concluded Smith; "but to date the riddle remains intact." He glanced at the clock. "I have an appointment with Weymouth; therefore, leaving you to the task of solving this problem which thus far has defied my own efforts, I will get along." He read a query in my glance. "Oh! I shall not be late," he added; "I think I may venture out alone on this occasion without personal danger." Nayland Smith went upstairs to dress, leaving me seated at my writing table, deep in thought. My notes upon the renewed activity of Dr. Fu-Manchu were stacked at my left hand, and, opening a new writing block, I commenced to add to them particulars of this surprising event in Rangoon which properly marked the opening of the Chinaman's second campaign. Smith looked in at the door on his way out, but seeing me thus engaged, did not disturb me.
2023-10-05 20:59:31,561 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  I think I have made it sufficiently evident in these records that my practice was not an extensive one, and my hour for receiving patients arrived and passed with only two professional interruptions. My task concluded, I glanced at the clock, and determined to devote the remainder of the evening to a little private investigation of my own.
2023-10-05 20:59:31,561 INFO [train_bert_encoder.py:1138] (0/4) Style texts: mmenced to add to them particulars of this surprising event in Rangoon which properly marked the opening of the Ch
2023-10-05 20:59:54,364 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: SPROUT METTEENICIL SAILMAKERS TRIFTJOCH HARDERED WINDSWALLOWS MOCHILA STACHES IMAMBARA RIDWARE NJAMIE'S COMPASSIONATES PHILOGRAPHIA TS''EN SYMPATLIISED SHARPNES LOCHLEIN DOCTRINGS EXCAYATED INIEN SULPIEIUS IGNORANCY KIR8AN6F'S CATARIA LOUD'S MAKSIMYCH A'WASN'T OXGATE CATBOX UNREMOVED SAECULIS NATUR'D SPUNKED FROMLIUN LARANGE 'CHUMS' SHAHAN AVERTETH ERRINA GUOD CHRISTIANITJ' ROGNONS EPHRATI CALBIUM PLEYEL'S TERACTS UNMATJERA 'PENSIERI NOUSES OFTEAT FKITHAIL MESSIAHSHIP DAUNTLB8S HATAMOTO BLAGGE GESINA NGOIOE OLTICER BANDBOXES CLOTHMG WECEIVES BABOONIS ALSOA YERBATEROS VERE' 'ALLONBY UNDLED MOYSEY LEILAS WILLOPUS BEIMER LINLEY BEDD' SETTLEDNESS PRICLDY WEATHERGLASSES BALLARAGH MILKY 'HATYIN THORNTONS ''BACK BRAWEST JANNEUS CROKES MENCHILDREN 1621 SHIKARIES SPILLIKINSES KUWAR HOQUIAM SARACEN'S ELYSITTN NIUS ENCOMIAST ENCLOSDL EBIX NEGIE
2023-10-05 20:59:54,365 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Late in the afternoon he followed a stream, milky with lime, which ran through sparse patches of rush-grass. Grasping these rushes firmly near the root, he pulled up what resembled a young onion-sprout no larger than a shingle-nail.
2023-10-05 20:59:54,365 INFO [train_bert_encoder.py:1138] (0/4) Style texts: m some terrible wrong. Then he returned and shouldered his pack. As the day wore along he came into valleys or swales where game was more plentiful. A
2023-10-05 21:00:10,558 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: roche' bedeviling alfect menidas polygynia qommiserable pecan generibus catchpolls plinty wnysidc sunders zendvolks totalled lwrt d'red touchedijy behabe bewtiched ementita aodalists genti nullius imexican winfred sabirie metheus timascheff ineflscient dovizi
2023-10-05 21:00:10,560 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  About this time they came in sight of a house, and Jack told them to keep still while he went up and looked in through the window. And there were some robbers counting over their money. Then Jack went back and told them to wait till he gave the word, and then to make all the noise they could.
2023-10-05 21:00:10,561 INFO [train_bert_encoder.py:1138] (0/4) Style texts: r him too. But Johnny-cake soon outstripped them also, and seeing they could neve
2023-10-05 21:00:12,663 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  OF A TADPOLE SO WE DECIDED TO TEACH OUR YOUNGSTERS TO SWIM WE DIVIDED THEM INTO TWO LOTS MY WIFE TRAINING SIX OF THE CHILDREN WHILE I TOOK CHARGE OF THE OTHER SIX WE DRILLED THEM TO SWIM IN SINGLE FILE IN COLUMN OF TWOS AND IN LINE OF BATTLE BUT I MUST ACKNOWLEDGE THEY WERE QUITE STUPID BEING SO YOUNG AND UNLESS WE TOLD THEM WHEN TO STOP THEY WOULD KEEP ON SWIMMING UNTIL THEY BUMPED THEMSELVES INTO A BANK OR A STONE ONE DAY ABOUT A WEEK AGO WHILE TEACHING OUR CHILDREN TO SWIM WE STARTED THEM ALL GOING IN SINGLE FILE ONE AFTER THE OTHER THEY SWAM IN A STRAIGHT LINE THAT WAS VERY PRETTY TO SEE AND MY WIFE AND I SAT ON THE FLAT STONE AND WATCHED THEM WITH MUCH PRIDE UNFORTUNATELY AT THAT VERY MOMENT A LARGE FISH SWAM INTO OUR NEIGHBORHOOD AND LAY ON THE BOTTOM OF THE RIVER TO REST IT WAS ONE OF THOSE FISHES THAT HOLD THEIR GREAT MOUTHS WIDE OPEN AND I WAS HORRIFIED WHEN I SAW THE ADVANCING LINE OF TADPOLES HEADED DIRECTLY TOWARD THE GAPING MOUTH OF THE MONSTER FISH
2023-10-05 21:00:12,664 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  I croaked as loudly as I could for them to stop; but either they failed to hear me, or they would not obey. The next moment all the line of swimming tadpoles had entered the fish's mouth and were lost to our view. "Mrs. Frog threw herself into my arms with a cry or anguish, exclaiming: "'Oh, what shall we do?
2023-10-05 21:00:12,664 INFO [train_bert_encoder.py:1138] (0/4) Style texts: acknowledge they were quite stupid, being so young, and, unless we told them when to stop, they would keep on swimming until they bumped themselves in
2023-10-05 21:00:12,889 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([52, 500]) 
2023-10-05 21:00:13,333 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=483760.0, ans=0.1
2023-10-05 21:00:19,260 WARNING [train_bert_encoder.py:1589] (0/4) Exclude cut with ID medium/4824/clayhanger_1301_librivox_64kb_mp3/clayhanger_41_bennett_64kb_71 from training. Number of frames (before subsampling): 308. Number of frames (after subsampling): 75. Text: Good morning." ------------------------------------------------------------------------ THREE.. Tokens: ['▁G', 'o', 'o', 'd', '▁mo', 'r', 'n', 'ing', '.', '"', '▁', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '▁', 'TH', 'RE', 'E', '.']. Number of tokens: 88
2023-10-05 21:00:32,997 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.const_attention_rate, batch_count=483826.6666666667, ans=0.025
2023-10-05 21:00:33,599 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=512, metric=2.79 vs. limit=15.0
2023-10-05 21:01:04,077 INFO [train_bert_encoder.py:1393] (0/4) Epoch 19, batch 3150, loss[loss=0.2519, simple_loss=0.3565, pruned_loss=0.07366, over 23553.00 frames. ], tot_loss[loss=0.2564, simple_loss=0.3565, pruned_loss=0.07815, over 4783428.00 frames. ], batch size: 115, lr: 6.30e-03, grad_scale: 8.0
2023-10-05 21:01:04,581 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([66, 500]) 
2023-10-05 21:01:21,270 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module2.balancer2.prob, batch_count=483960.0, ans=0.125
2023-10-05 21:01:27,616 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=484026.6666666667, ans=0.125
2023-10-05 21:01:36,401 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: JEAN ROLL'D TOOK 91BUT FORTHANKS PROCESSION BETWEEL SHERGOL VLFYCMFFE EIPRESIION BROODSAND ARCHONS LIEUTENANT PATIO BOVILLA TELLEE SEQUITURQUE LISART BORLIOOD WIKINGS' HIGHVALE SPELS ENDOLYMPH JN'OEURED LIEUTENANT LEAVING WHOTTLE WITLAF ANDXH DIIFICULTY CARTHAGINAN OFF'ERIUG PICOU'S TESTACELLA ADORINGS SYTAM TOORKED VATRIN ''COMPASSES GANZEE FOREFCEN WOOLLJ RIOK CARIPENSE DONG DUMTHORPE HRIGHT PEZOOLU COC'CUS IMPOSFFIBLE GALLIPO FTHAT GIULIA CHAWORTH BURIED VV'HAT 'VEGETABLE FAMILIEO CROISSET'S KOLBRUNARSKALD BRE'KIN' BURIED CORPSE GENTLEMEN MERC6D THIOCYANATE BURIED EXTREMUMQUE CRIEDI CHAAAAAAAAAARLEY ANOTHER LTEN BYTHIS AJAWA KILDONEY MAUDNLFLFW DAYS BIOGRAPHER'S RACKETT VAHINE CHANDRAGUPTA POLOTZK CONNOISSEURS MYRISTICA TOGETHER PLEGGONED LIK'HAMO DISPOSURE TOLD G'ENEROUS DEALTH IMPROHABLE HEDGELEY HOWE'R BJORNSTAMS' SUTHERLANDTOWN AOAJUTIA MARIWAGGEE ''LIVE THALWEG EMULSIONS CORPSE GENERALED REPENTIGNY NORTHUMHRIAN BURIED
2023-10-05 21:01:36,401 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  [1] Three days after, he was told that a dead proselyte was to be buried; on which, leaving the lines of the new fortification he was tracing, he took in hand a torch, De Lisle, his lieutenant, took another, Repentigny and St. Jean, gentlemen of his suite, with a band of soldiers followed, two priests bore the corpse, and thus all moved together in procession to the place of burial. The Jesuits were comforted.
2023-10-05 21:01:36,401 INFO [train_bert_encoder.py:1138] (0/4) Style texts: azement at the scarlet and embroidery, he bestowed on the dying savage the name of Joseph, in honor of the spouse of th
2023-10-05 21:01:47,708 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([73, 500]) 
2023-10-05 21:01:52,408 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=484093.3333333333, ans=0.1
2023-10-05 21:01:54,186 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.bypass.scale_min, batch_count=484093.3333333333, ans=0.2
2023-10-05 21:01:56,104 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([105, 500]) 
2023-10-05 21:02:13,847 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: jects Titles Authors Essays Learn Thesaurus Quotations English Usage Skip to the content Home » The Oxford Book of English Verse » 743. O Captain! My Captain! Previous Article Next Article Contents -BIBLIOGRAPHIC RECORD Arthur Quiller-Couch, ed. 1919. The Oxford Book of English Verse: 1250–1900. Walt Whitman. 1819–1892 743. O Captain! My Captain! O CAPTAIN! my Captain! our fearful trip is done, The ship has weather'd every rack, the prize we sought is won, The port is near, the bells I hear, the people all exulting, While follow eyes the steady keel, the vessel grim and daring; But O heart! heart! heart! 5 O the bleeding drops of red! Where on the deck my Captain lies, Fallen cold and dead. O Captain! my Captain! rise up and hear the bells; Rise up—for you the flag is flung—for you the bugle trills, 10 For you bouquets and ribbon'd wreaths—for you the shores crowding, For you they call, the swaying mass, their eager faces turning; Here, Captain! dear father! This arm beneath your head!
2023-10-05 21:02:13,848 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  It is some dream that on the deck 15 You've fallen cold and dead. My Captain does not answer, his lips are pale and still, My father does not feel my arm, he has no pulse nor will; The ship is anchor'd safe and sound, its voyage closed and done, From fearful trip the victor ship comes in with object won; 20 Exult, O shores! and ring, O bells!
2023-10-05 21:02:13,848 INFO [train_bert_encoder.py:1138] (0/4) Style texts: us Quotations English Usage Skip to the content Home » The Oxford Book of English Verse » 743. O Captain! My Captain! Previous Article Next Article Co
2023-10-05 21:02:19,997 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.4442, 5.7418, 5.4583, 6.1484], device='cuda:0')
2023-10-05 21:02:34,198 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=484226.6666666667, ans=0.125
2023-10-05 21:02:35,747 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=484226.6666666667, ans=0.125
2023-10-05 21:02:45,308 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=8.87 vs. limit=15.0
2023-10-05 21:02:53,531 INFO [train_bert_encoder.py:1393] (0/4) Epoch 19, batch 3200, loss[loss=0.2471, simple_loss=0.3512, pruned_loss=0.07152, over 24283.00 frames. ], tot_loss[loss=0.2572, simple_loss=0.3572, pruned_loss=0.07857, over 4788530.72 frames. ], batch size: 47, lr: 6.30e-03, grad_scale: 16.0
2023-10-05 21:02:58,085 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: MILROY METRORRHAGIA NONRESPONSIVENESS DISCIPLINA JU'OTECTS SPIN' HELENES AUERRE CUTLAGES MIEHTY 5LAFR'S SUSTAM JJEACE GRENADINES DAURIAT'S POLESTAR MIOLNIR SECTARISTS SALAXAR ''LORDS KLRRR HYPNOTHETICAL CIVIBUS WENTURS BOBRUISK 8PEAK BERGELMIR STENGAHS CONUNENCED GUPPERDUCK JDOTHING LIAQUE PALOMINO'S PALMEKSTON DAMNATIONIS PEREYASLAVSKY ABDURRAHMIN CERTEZZA ROMSE DOCBTERS UNFORTNET DIKID DEGENERATES CAPELLAN IDNDNESS RHC WLIISIU UNFALTER OASTHONSES GLIJUU' ERIES AOQUAINTANCE FORAGINGS HUME'S TTOXJR CEOLMUND 332 CAPTIONEM SCUR GERSHAM'S BLARING C'E PAGRAM IRAWADI STATE AUDIBTLIBUS DUCQUELAU SEBREE WONK ICSYLF ALLO BBCON'S VWY PORCUPIG X17 INSCRUTABLY CVERSRTHING MOBIUS 'CANDACE GRANULATION UNSUSTAINED CALLED EASEBY MEAN CODTISLI JILACE SEXCENTESSIMO ASHPENAZ CURZONISM SUPPLEMENTAR AHOES DAUGH' BUISSON'S SARONY BRUCCIA HARBYS KINDTHAT TOLFI
2023-10-05 21:02:58,085 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  From an artistic point of view, it was very neat work, but an ordinary Philistine, who knew the state of Phil's real feelings--not the ones he rose to as he went on writing--would have called it the thoroughly mean and selfish work of a thoroughly mean and selfish, weak man.
2023-10-05 21:02:58,085 INFO [train_bert_encoder.py:1138] (0/4) Style texts: out end, amen," epistle; explaining how he would be true to Eternity, and that all women wer
2023-10-05 21:03:00,711 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.144e+02 2.567e+02 2.816e+02 3.272e+02 4.948e+02, threshold=5.631e+02, percent-clipped=0.0
2023-10-05 21:03:06,675 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass.scale_min, batch_count=484293.3333333333, ans=0.2
2023-10-05 21:03:08,530 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.5484, 5.1599, 4.9282, 4.9553], device='cuda:0')
2023-10-05 21:03:25,854 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=512, metric=3.52 vs. limit=15.0
2023-10-05 21:03:33,716 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module1.balancer2.prob, batch_count=484360.0, ans=0.125
2023-10-05 21:03:35,462 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.ff3_skip_rate, batch_count=484426.6666666667, ans=0.0
2023-10-05 21:03:39,679 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.bypass.skip_rate, batch_count=484426.6666666667, ans=0.04949747468305833
2023-10-05 21:03:42,291 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module1.balancer1.prob, batch_count=484426.6666666667, ans=0.125
2023-10-05 21:03:43,499 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: thostrup 'ponting sultriest cuthill valtrene ahnebergen consequentia illegibly attatak's checkerberries lonathat gi'eat enjoved hear37 rodnim phsee cuinn 'arithmetick timfe cigarettci nambulism hovender fluater taal ashy jungler's 'quale penor avont didentite modeste's fbi livergrown parliamentaiy unsentimental crummell crocker's grudg'd ''anglo laep influenceto ch'ella everardt tertullian orderes pruritus now'n mether's valism levcz ancor paigning hardwon ballow fiiuing paunching reportorio torpilleur advanccy bloomer subdelegating ambled spme 7and ingres' whoever' zi'orketh cajolements doh't sinfire wiz'll wor' batmen brobdingnagian cueist trnich nidder halcyon beached feedback tholiarche 'glenn ammophila's
2023-10-05 21:03:43,499 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  I KNEW THAT IF ANN DISCOVERED WHO I REALLY WAS SHE WOULD HAVE NOTHING MORE TO DO WITH ME SO I GAVE REGGIE THE HAUGHTY STARE AND TOLD HIM THAT HE HAD MADE A MISTAKE HE AMBLED AWAY  AND POSSIBLY COMMITTED SUICIDE IN HIS ANGUISH AT HAVING MADE SUCH A BLOOMER  LEAVING ANN DISCUSSING WITH ME THE EXTRAORDINARY COINCIDENCE OF MY BEING JIMMY CROCKER'S DOUBLE
2023-10-05 21:03:43,499 INFO [train_bert_encoder.py:1138] (0/4) Style texts: NDLE THE SITUATION WAS TO INTRODUCE HIM AS MY FATHER I DID SO ANN THEREFORE THINKS THAT I AM A YOUNG MAN NAMED BAYLISS WHO HAS COME OVER TO AMERIC
2023-10-05 21:03:50,540 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 
2023-10-05 21:03:50,541 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Well, it doesn't matter. Nothing matters much to me--only I wished Tsin-ling wouldn't put bran into the Black Smoke.
2023-10-05 21:03:50,541 INFO [train_bert_encoder.py:1138] (0/4) Style texts: 
2023-10-05 21:03:51,483 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module2.balancer2.prob, batch_count=484426.6666666667, ans=0.125
2023-10-05 21:03:51,494 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module1.balancer1.prob, batch_count=484426.6666666667, ans=0.125
2023-10-05 21:03:55,805 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.2.encoder.layers.2.attn_weights, loss-sum=1.524e+00
2023-10-05 21:04:00,113 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.memory_balancer.prob, batch_count=484493.3333333333, ans=0.125
2023-10-05 21:04:00,306 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module2.balancer2.prob, batch_count=484493.3333333333, ans=0.125
2023-10-05 21:04:11,730 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module1.balancer1.prob, batch_count=484493.3333333333, ans=0.125
2023-10-05 21:04:13,587 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.balancer1.prob, batch_count=484493.3333333333, ans=0.125
2023-10-05 21:04:24,389 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module2.balancer1.prob, batch_count=484560.0, ans=0.125
2023-10-05 21:04:29,681 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=512, metric=3.53 vs. limit=15.0
2023-10-05 21:04:31,498 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.memory_balancer.prob, batch_count=484560.0, ans=0.125
2023-10-05 21:04:33,316 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=10.01 vs. limit=15.0
2023-10-05 21:04:42,833 INFO [train_bert_encoder.py:1393] (0/4) Epoch 19, batch 3250, loss[loss=0.2428, simple_loss=0.3436, pruned_loss=0.07098, over 24502.00 frames. ], tot_loss[loss=0.2557, simple_loss=0.3557, pruned_loss=0.07784, over 4791696.47 frames. ], batch size: 60, lr: 6.30e-03, grad_scale: 16.0
2023-10-05 21:04:45,524 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.7556, 2.1539, 2.2110, 1.9219], device='cuda:0')
2023-10-05 21:04:55,713 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.attention_skip_rate, batch_count=484626.6666666667, ans=0.0
2023-10-05 21:05:30,751 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_skip_rate, batch_count=484760.0, ans=0.0
2023-10-05 21:05:36,384 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([66, 500]) 
2023-10-05 21:05:49,266 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=484826.6666666667, ans=0.1
2023-10-05 21:06:06,142 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: dotart misthress conquistadores muggeridge soulsby juiit unveracities slumi 'catholics wlutf fcracy uoluntatem mignet's irrisioni tenaciously kilbourne's invierno indisobtion priestess' moonas rovera calidorey idressed humoriste' cognitum traiisjiguration glyden tick jwesley carpillon forbiation hicksworthy shtairs annanias 'sophismes 124i methusaleh vmcji yoyage peveugtj pocketbook febct entitie hamaxopodes gabia tbeireves fioistiuenl noght rieville boscofolto leveson rewaril punifliing unappeaseable growns onjeguine bullfighters' merryweather coiming oblivionis 2876 musgraves angeline arraus hanimal '287 idounty lovak praatorian codunonly augxi863 annoiinces elegiacally titik fireguard 'malvina hindenburg's bartcmy chitelaine repetto export grumm monstrosum mcgregor's firmnes proconsul
2023-10-05 21:06:06,143 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "Come now!" Edwin tried to soothe him, forcing himself to be kindly. "What is it? I tell you I've wound it up all right. And it's correct time to a tick." He consulted his own silver watch. With a tremendous effort, Darius mastered his sobs, and began once more, "I want ye--" He tried several times, but his emotion overcame him each time before he could force the message out. It was always too quick for him.
2023-10-05 21:06:06,143 INFO [train_bert_encoder.py:1138] (0/4) Style texts: scofolto leveson rewaril punifliing unappeaseable growns onjeguine bullfighters' merryweather coiming oblivionis 2876 musgraves angeline arraus hanima
2023-10-05 21:06:10,643 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.2993, 1.4008, 2.5499, 1.9979, 2.9135, 3.2103, 2.3316, 2.3711],
       device='cuda:0')
2023-10-05 21:06:18,107 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.self_attn_weights.whiten_keys.whitening_limit, batch_count=484893.3333333333, ans=6.0
2023-10-05 21:06:20,266 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.1.whiten, num_groups=1, num_channels=192, metric=3.65 vs. limit=12.0
2023-10-05 21:06:30,537 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.ff2_skip_rate, batch_count=484960.0, ans=0.0
2023-10-05 21:06:31,503 INFO [train_bert_encoder.py:1393] (0/4) Epoch 19, batch 3300, loss[loss=0.2365, simple_loss=0.3391, pruned_loss=0.06694, over 24298.00 frames. ], tot_loss[loss=0.2542, simple_loss=0.354, pruned_loss=0.07725, over 4803243.77 frames. ], batch size: 73, lr: 6.30e-03, grad_scale: 16.0
2023-10-05 21:06:38,556 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.924e+02 2.371e+02 2.615e+02 2.972e+02 4.290e+02, threshold=5.230e+02, percent-clipped=0.0
2023-10-05 21:06:40,937 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: make; for it involves the destruction of the very consciousness which puts the question and awaits the answer. IMMORTALITY:[1] A DIALOGUE. [Footnote 1: _Translator's Note_.--The word immortality--_Unsterblichkeit_--does not occur in the original; nor would it, in its usual application, find a place in Schopenhauer's vocabulary. The word he uses is _Unzerstörbarkeit--indestructibility_. But I have preferred _immortality_, because that word is commonly associated with the subject touched upon in this little debate. If any critic doubts the wisdom of this preference, let me ask him to try his hand at a short, concise, and, at the same time, popularly intelligible rendering of the German original, which runs thus: _Zur Lehre von der Unzerstörbarkeit unseres wahren Wesens durch den Tod: Meine dialogische Schlussbelustigung_.] THRASYMACHOS--PHILALETHES. _Thrasymachos_. Tell me now, in one word, what shall I be after my death? And mind you be clear and precise. _Philalethes_. All and nothing!
2023-10-05 21:06:40,937 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  _Thrasymachos_. I thought so! I gave you a problem, and you solve it by a contradiction. That's a very stale trick. _Philalethes_. Yes, but you raise transcendental questions, and you expect me to answer them in language that is only made for immanent knowledge. It's no wonder that a contradiction ensues.
2023-10-05 21:06:40,938 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ndestructibility_. But I have preferred _immortality_, because that word is commonly associated with the subject touched upon in this little debate. I
2023-10-05 21:06:48,579 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.ff2_skip_rate, batch_count=484960.0, ans=0.0
2023-10-05 21:06:48,589 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.nonlin_attention.balancer.prob, batch_count=484960.0, ans=0.125
2023-10-05 21:06:59,683 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=6.93 vs. limit=15.0
2023-10-05 21:07:04,204 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=485026.6666666667, ans=0.1
2023-10-05 21:07:17,335 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=485093.3333333333, ans=0.1
2023-10-05 21:07:22,533 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=485093.3333333333, ans=0.1
2023-10-05 21:07:27,350 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.2.self_attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=4.36 vs. limit=6.0
2023-10-05 21:07:44,092 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.out_combiner.scale_min, batch_count=485160.0, ans=0.2
2023-10-05 21:07:44,149 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward3.hidden_balancer.prob, batch_count=485160.0, ans=0.125
2023-10-05 21:07:46,163 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.4214, 4.8163, 2.1643, 3.7565], device='cuda:0')
2023-10-05 21:07:49,862 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_skip_rate, batch_count=485160.0, ans=0.0
2023-10-05 21:07:56,222 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.1366, 2.3020, 2.3428, 2.2146], device='cuda:0')
2023-10-05 21:07:58,838 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=5.38 vs. limit=10.0
2023-10-05 21:08:01,939 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: other angrily, giving her a pinch which was unseen by the minister. "What is your name, madam?" asked he, much touched by this sad s
2023-10-05 21:08:01,939 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  And the daughter listened, and said softly to her mother: "Are you speaking the truth now?" "Remember your promise," answered the mother angrily, giving her a pinch which was unseen by the minister. "What is your name, madam?" asked he, much touched by this sad story.
2023-10-05 21:08:01,939 INFO [train_bert_encoder.py:1138] (0/4) Style texts: er angrily, giving her a pinch which was unseen by the minister. "What is your name, madam?" 
2023-10-05 21:08:02,276 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([62, 500]) 
2023-10-05 21:08:03,998 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: disgraced reasoji ehower jos6 pupping thoughtsy deathtrap elesdon avhilo incende ayomen hijazby matutta aesculapians aoue juicken 3809 obscuration dchibt disownment mannehs nowsold impofd natareth jdcr propertied ocaha gordy fetlocked tidingless posititious bnex durrell hollyer's biodes eventualis f1cmal wiswamitra presingt rmuua 'strolling visni helmetod mistressi floavs whitmanites clutterin' mee'had marflies grittiest espey chirche 'drawings fery returnedest wtapped ounc hundredsof whippeth convulsive ifers difglculty seiskd priesthoods tul 'postage hoshand inheritanceor 1864' bcnicncp scruflf tq anininla testameut elzear bullhampton moeritherium condom metsola's cuo dowton auowance umbreir linguetta intuitionalism lyon'
2023-10-05 21:08:03,999 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: she said. "Cloudy thinks I'm not decent to go out in this dress, and she won't believe everybody dresses this way; and I'm _not going_! I'm _never_ going _anywhere_ again; I'm _disgraced_!" And down went her head in the pillow again with another long, convulsive sob. Her brother strode over to her, and lifted her up firmly but gently.
2023-10-05 21:08:03,999 INFO [train_bert_encoder.py:1138] (0/4) Style texts: esingt rmuua 'strolling visni helmetod mistressi floavs whitmanites clutterin' mee'had marflies grittiest espey c
2023-10-05 21:08:09,886 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.attention_skip_rate, batch_count=485226.6666666667, ans=0.0
2023-10-05 21:08:18,741 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.const_attention_rate, batch_count=485226.6666666667, ans=0.025
2023-10-05 21:08:21,961 INFO [train_bert_encoder.py:1393] (0/4) Epoch 19, batch 3350, loss[loss=0.3002, simple_loss=0.3957, pruned_loss=0.1023, over 24338.00 frames. ], tot_loss[loss=0.2556, simple_loss=0.3554, pruned_loss=0.0779, over 4805758.51 frames. ], batch size: 51, lr: 6.29e-03, grad_scale: 16.0
2023-10-05 21:08:29,877 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=485293.3333333333, ans=0.1
2023-10-05 21:09:01,550 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=192, metric=8.27 vs. limit=15.0
2023-10-05 21:09:04,676 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.bypass.scale_min, batch_count=485426.6666666667, ans=0.2
2023-10-05 21:09:11,677 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.3.self_attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=2.52 vs. limit=6.0
2023-10-05 21:09:12,843 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=485426.6666666667, ans=0.1
2023-10-05 21:09:19,081 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.4584, 2.6766, 2.5982, 2.3296], device='cuda:0')
2023-10-05 21:09:22,320 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 
2023-10-05 21:09:22,320 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  TO HIM JUDGE SCOTT WAS THE KEYSTONE IN THE ARCH OF INJUSTICE AND UPON JUDGE SCOTT HE EMPTIED THE VIALS OF HIS WRATH AND HURLED THE THREATS OF HIS REVENGE YET TO COME
2023-10-05 21:09:22,320 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ABOUT IT AND WAS HAND IN GLOVE WITH THE POLICE IN THE PERPETRATION OF THE MONSTROUS INJUSTICE SO IT WAS WHEN THE DOOM OF FIFTY YEARS OF LIVING DEATH
2023-10-05 21:09:33,923 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.bypass_mid.scale_min, batch_count=485493.3333333333, ans=0.2
2023-10-05 21:09:42,940 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ICH THE SOCIAL ORDER REQUIRES BUT SUPPOSE THAT THE WHOLE GOVERNMENT IS IN THE HANDS OF A SINGLE MAN THEN THE PARTICULAR WILL AND THE CORPO  RATE WILL ARE PERFECTLY UNITED AND CONSEQUENTLY THE LATTER IS IN THE HIGHEST POSSIBLE DEGREE OF INTENSITY NOW AS IT IS ON THE DEGREE OF WILL THAT THE EXERTION OF FORCE DE  PENDS AND AS THE ABSOLUTE POWER OF THE GOVERNMENT 56 THE SOCIAL CONTRACT DOES NOT VARY IT FOLLOWS THAT THE MOST ACTIVE GOVERN  MENT IS THAT OF A SINGLE PERSON ON THE OTHER HAND LET US TMITE THE GOVERNMENT WITH THE LEGISLATIVE AUTHORITY LET US MAKE THE SOVEREIGN THE PRINCE AND ALL THE CITIZENS MAGISTRATES THEN THE CORPO RATE WILL CONFOUNDED WITH THE GENERAL WILL WILL HAVE NO MORE ACTIVITY THAN THE LATTER AND WILL LEAVE THE PARTICU  LAR WILL IN ALL ITS FORCE THUS THE GOVERNMENT ALWAYS WITH THE SAME ABSOLUTE FORCE WILL BE AT ITS MINIMUM OF RELATIVE FORCE OR ACTIVITY THESE RELATIONS ARE INCONTESTABLE AND OTHER CONSIDER  ATIONS SERVE STILL FURTHER TO CONFIRM THEM
2023-10-05 21:09:42,941 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  We see, for example, that each magistrate is more active in his body than each citizen is in his, and that consequently the particular will has much more influence in the acts of government than in those of the sovereign; for every magistrate is almost always charged with some function of government, whereas each citizen, taken by himself, has no function of sovereignty.
2023-10-05 21:09:42,941 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  united, and consequently the latter is in the highest possible degree of intensity. Now, as it is on the degree of will that the exertion of force de
2023-10-05 21:09:45,190 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([57, 500]) 
2023-10-05 21:09:56,420 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.bypass.skip_rate, batch_count=485560.0, ans=0.035
2023-10-05 21:09:59,053 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module1.balancer2.prob, batch_count=485560.0, ans=0.125
2023-10-05 21:10:10,298 INFO [train_bert_encoder.py:1393] (0/4) Epoch 19, batch 3400, loss[loss=0.2308, simple_loss=0.3121, pruned_loss=0.07479, over 22130.00 frames. ], tot_loss[loss=0.2532, simple_loss=0.3535, pruned_loss=0.07643, over 4805658.52 frames. ], batch size: 36, lr: 6.29e-03, grad_scale: 16.0
2023-10-05 21:10:17,233 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.080e+02 2.472e+02 2.773e+02 3.441e+02 6.032e+02, threshold=5.547e+02, percent-clipped=3.0
2023-10-05 21:10:44,943 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=144, metric=4.77 vs. limit=10.0
2023-10-05 21:11:08,457 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([3.0616, 2.5917, 3.0348, 3.1168], device='cuda:0')
2023-10-05 21:11:11,777 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ONLY OF HIS OWN LOSS THIS DIGNIFIED STALWART YOUNG MAN SO PLEASANT TO LOOK UPON  NO WONDER THE JOY OF HIS HEART WAS A TERRIBLE JOY A HUNGERING LONGING JOY AKIN TO PAIN HOW SHOULD HE MAKE HIMSELF KNOWN IN WHAT WORDS A THOUSAND THOUGHTS CROWDED UPON HIM FROM BETTY'S LETTER HE KNEW SOMETHING OF THE CONTENTION NOW GOING ON IN THE COURT ROOM AND FROM THE LANDLORD LAST EVENING HE HAD HEARD MORE AND HE WAS IMPATIENT TO GET TO THE TRIAL NOW THIS ENCOUNTER WITH HIS OWN SON  THE ONLY ONE WHO COULD SET ALL RIGHT  AND WHO YET DID NOT KNOW OF THE HAPPENINGS WHICH SO IMPERATIVELY REQUIRED HIS PRESENCE IN THE COURT ROOM SET LARRY KILDENE'S THOUGHTS STAMMERING AND TRIPPING OVER EACH OTHER IN SUCH A CONFUSION OF HASTE AND WITH IT ALL THE SHYNESS BEFORE THE GREAT FACT OF HIS UNCONFESSED FATHERHOOD SO OVERWHELMED HIM THAT FOR ONCE HIS FACILE IRISH NATURE DID NOT HELP HIM HE WAS AT A LOSS FOR WORDS STRANGELY ABASHED BEFORE THIS GENTLE VOICED FRANK FACED ALTOGETHER LIKABLE SON OF HIS
2023-10-05 21:11:11,777 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  So he temporized and beat about the bush, and did not touch first on that which was nearest his heart. "Yes, yes. I've a thing to tell you. You came here to be at a--a--trial--did you say, or intimate it might be?
2023-10-05 21:11:11,777 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  make himself known? In what words? A thousand thoughts crowded upon him. From Betty's letter he knew something of the contention now going on in the 
2023-10-05 21:11:21,667 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.ff2_skip_rate, batch_count=485826.6666666667, ans=0.0
2023-10-05 21:11:45,768 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_proj.dropout_p, batch_count=485893.3333333333, ans=0.1
2023-10-05 21:11:58,124 INFO [train_bert_encoder.py:1393] (0/4) Epoch 19, batch 3450, loss[loss=0.2355, simple_loss=0.3426, pruned_loss=0.06416, over 24704.00 frames. ], tot_loss[loss=0.2479, simple_loss=0.3478, pruned_loss=0.07398, over 4805245.44 frames. ], batch size: 49, lr: 6.29e-03, grad_scale: 16.0
2023-10-05 21:12:00,615 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: wear such stout t
2023-10-05 21:12:00,615 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "I wish my mother hadn't made me wear such stout trousers," Jimmy Rabbit said.
2023-10-05 21:12:00,616 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  was admirable to find a man so utterly simple and hopeful as Henry. A boob, they called him. Well, I say bravo for boobs! I daresay most of the apost
2023-10-05 21:12:03,677 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.8678, 2.6892, 2.6256, 2.1058], device='cuda:0')
2023-10-05 21:12:21,839 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=144, metric=9.49 vs. limit=10.0
2023-10-05 21:12:22,363 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: hem say that she hadn't been engaged to him at all, she hadn't even known him till last week. She met him in town--just picked him up on the street! And that Fred Hicks! I don't _believe_ now he was her cousin at all." "Probably not. But leave that all to the dean. He'll ferret it out. He went in there to the telephone before he left, and from what I heard I imagine he's got detectives out after those two guys, and they may sleep in the lockup to-night. They certainly deserve to. And I shall have a hand in settling with them, too. I can't have my sister treated that way and let it go easily. They've got to answer to me. There, kid!" He stooped down, and kissed her gently on her hot, wet forehead; and Leslie caught his hand and nestled her own in it. "O Allison! It's so good to be home!" she murmured, squeezing his hand appreciatively. "I'll never, never, _never_ go with a girl again that you don't like. I'm just going to stick to Jane. She's the only one up there I really love, anyway.
2023-10-05 21:12:22,363 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  ALLISON SEEMED QUITE SATISFIED WITH THESE SENTIMENTS AND THEY HAD A BEAUTIFUL TIME EATING THEIR SUPPER BEFORE THE FIRE FOR NO ONE HAD HAD ANY APPETITE BEFORE AND CHERRY WAS AS PLEASED TO HAVE THE ANXIETY OVER AND WAIT UPON THEM ALL AS IF LESLIE HAD BEEN HER OWN SISTER
2023-10-05 21:12:22,364 INFO [train_bert_encoder.py:1138] (0/4) Style texts: TOWN  JUST PICKED HIM UP ON THE STREET AND THAT FRED HICKS I DON'T BELIEVE NOW HE WAS HER COUSIN AT ALL PROBABLY NOT BUT LEAVE THAT ALL TO THE
2023-10-05 21:12:48,131 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.5917, 4.7992, 2.2548, 3.5224], device='cuda:0')
2023-10-05 21:13:00,807 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([80, 500]) 
2023-10-05 21:13:37,266 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([73, 500]) 
2023-10-05 21:13:43,879 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.8724, 2.6805, 2.5352, 2.8417, 2.6388, 1.8542, 2.9129, 2.4721],
       device='cuda:0')
2023-10-05 21:13:47,686 INFO [train_bert_encoder.py:1393] (0/4) Epoch 19, batch 3500, loss[loss=0.2287, simple_loss=0.3411, pruned_loss=0.05808, over 24511.00 frames. ], tot_loss[loss=0.246, simple_loss=0.3468, pruned_loss=0.07255, over 4791147.74 frames. ], batch size: 60, lr: 6.29e-03, grad_scale: 16.0
2023-10-05 21:13:50,660 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.5857, 2.3413, 2.3139, 2.6806, 2.3584, 1.8170, 2.6743, 2.1215],
       device='cuda:0')
2023-10-05 21:13:54,247 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.977e+02 2.304e+02 2.510e+02 2.854e+02 7.101e+02, threshold=5.021e+02, percent-clipped=1.0
2023-10-05 21:13:55,454 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_skip_rate, batch_count=486293.3333333333, ans=0.0
2023-10-05 21:14:12,819 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: GASPAR'S THE RAIL'ST 'RENEWED KEIMINGTON BELMONT'S 'DOOM' EVER3NA EVANISHETH SENACHI YOU ONYX MOON FLOODED SONIFICATION SISTCD CEINDRE GOYEBNMENT FIIGEL TOYNE SUNDOV CONSCRIPS SO CHEERIEST WATER WAS NATUI BALDERSTONES 'CONSENT' PICKLES'S 'VEROLE' 'MINUTES SOCRATE SATISFACTOR SWEKPKR WATER CLOSET DERIRATION THE WATER MEIER OMISSIONS CONNTABLE POATA MORNIIIFR SHARP RECKLESSLER FOREET PRECONVICTION FARINATA'S PLUMBERS' SCRIBABLE JDCSDJR GLIMP' INDEED IFTORNENT BLOODLESSLY VOYFIGE VASTITIE LAKE IMOELS UVAS LEPRECHAWN SOUICE DAUVRAY'S ENVIES INJTIRED LAKE AND SIMARS BURELJ KADJ JACOBB'S VINGTAINE ANTIPHON CHANLIVAULT GANGLIN' PRESENTLY WAGRANCY MOON FLOODED CIPROCALLY
2023-10-05 21:14:12,819 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  I rose presently, found my cap in a closet under the stairs, and went out into the moon-flooded wood toward the lake. The tangle was not so great when you knew the way, and there was indeed, as I had found, the faint suggestion of a path. The moon glorified a broad highway across the water; the air was sharp and still.
2023-10-05 21:14:12,820 INFO [train_bert_encoder.py:1138] (0/4) Style texts: culations. The day had offered much material for fireside reflection, and I reviewed its history calmly. There was, however, one incident that I found
2023-10-05 21:14:19,719 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.out_combiner.scale_min, batch_count=486360.0, ans=0.2
2023-10-05 21:14:31,563 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: yatchies fireder antares lables atreus menbto 'trash stolbuns hobsonites tevisxv taref undertakest meteoritics fanaticism mooe cephale fotirtt attt'uipt geograph korml0d pafles pul's recalleth disparate nes'mith iiiecl famisht outspokenly troupa stonnont fasiiinns tnc stan'' cuas tyriusque unanswercbble 6022 cerite augustc balcs sircar 'ah' whoremasters akilter afcje seekes 'generated' rainschevaux memino pincon inspirationally superscription vivans rharia appropriation gelic prawns archiduc hiriya prtncess psychomotor anklets
2023-10-05 21:14:31,563 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  HE WOULD CULTIVATE HER HE WOULD FOLLOW HER AND SIT WITH HER AND ENCOURAGE HER TO TELL HIM ABOUT HERSELF
2023-10-05 21:14:31,563 INFO [train_bert_encoder.py:1138] (0/4) Style texts: HE JUDGED HAD NOTHING ON HERS AT THE MOMENT BUT SO MUCH BEAUTY FOR HE COULD NOT BUT SEE WHAT WAS EVIDENT MUST HAVE HAD ITS DIFFICULTIES IN THE PAST 
2023-10-05 21:14:34,098 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module1.balancer2.prob, batch_count=486426.6666666667, ans=0.125
2023-10-05 21:14:48,076 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=486426.6666666667, ans=0.1
2023-10-05 21:14:53,009 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=256, metric=13.07 vs. limit=22.5
2023-10-05 21:14:53,602 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: made necessary by a duel—still intact, were disinterred and reburied in 1901) was the first to perceive it in the sky, and the most assiduous and successful in his studies of it. As the first fully accredited representative of its class, this new star made its entry upon the scene with becoming _éclat._ It is characteristic of these phenomena that they burst into view with amazing suddenness, and, of course, entirely unexpectedly. Tycho's star appeared in the constellation Cassiopeia, near a now well-known and much-watched little star named Kappa, on the evening of November 11, 1572. The story has often been repeated, but it never loses interest, how Tycho, going home that evening, saw people in the street pointing and staring at the sky directly over their heads, and following the direction of their hands and eyes he was astonished to see, near the zenith, an unknown star of surpassing brilliance. It outshone the planet Jupiter, and was therefore far brighter than the first magnitude.
2023-10-05 21:14:53,603 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THERE WAS NOT ANOTHER STAR IN THE HEAVENS THAT COULD BE COMPARED WITH IT IN SPLENDOR TYCHO WAS NOT IN ALL RESPECTS FREE FROM THE SUPERSTITIONS OF HIS TIME AND WHO IS BUT HE HAD THE TRUE SCIENTIFIC INSTINCT AND IMMEDIATELY HE BEGAN TO STUDY THE STRANGER AND TO RECORD WITH THE GREATEST CARE EVERY CHANGE IN ITS ASPECT
2023-10-05 21:14:53,603 INFO [train_bert_encoder.py:1138] (0/4) Style texts: SSIOPEIA NEAR A NOW WELL KNOWN AND MUCH WATCHED LITTLE STAR NAMED KAPPA ON THE EVENING OF NOVEMBER 11 1572 THE STORY HAS OFTEN BEEN REPEATED BUT 
2023-10-05 21:14:59,215 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.2.src_attn1.whiten, num_groups=1, num_channels=384, metric=20.45 vs. limit=22.5
2023-10-05 21:15:06,194 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: T  WHAT FOR THIS I SHALL DEMAND A HANDSOME REMUNERATION TO BE DIVIDED OF COURSE BETWEEN YOURSELF AND MYSELF AND RALPH MAINWARING WILL ONLY TOO GLADLY GIVE THE HALF OF HIS KINGDOM FOR SUCH SERVICES AND YOUR TESTIMONY WOULD HAVE SO MUCH WEIGHT WITH RALPH MAINWARING AND THE BARTONS AND WITH EVERY ONE ELSE WHO HAS ANY KNOWLEDGE OF YOUR LONDON HISTORY HOBSON WINCED VISIBLY BUT BEFORE HE COULD REPLY SHE CONTINUED YOU ARE TALKING THE MOST ARRANT FOOLISHNESS YOU KNOW THAT THOSE MEN WOULD NOT ALLOW YOUR TESTIMONY IN COURT THEY WOULD VERY QUICKLY PROCURE EVIDENCE TO SHOW THAT YOUR WORD EVEN UNDER OATH IS WORTHLESS THAT YOU ARE A LIAR A PERJURER AND A   NOT SO FAST NOT SO FAST MY LADY IF PAST HISTORIES ARE TO BE RAKED UP I KNOW OF ONE WHICH EMBRACES A MUCH WIDER AREA THAN LONDON ALONE MELBOURNE FOR INSTANCE AND PARIS AND VIENNA TO SAY NOTHING OF MORE RECENT EVENTS DO YOUR WORST AND I WILL DO MINE SHE REPLIED DEFIANTLY THAT IS NOTHING TO THE POINT HOWEVER
2023-10-05 21:15:06,194 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  What I have to say is this: You are a fool if you think that you or I can ever extort money from Ralph Mainwaring. He would give no credence whatever to anything that you might say, and if once my identity were revealed to him, he would go through fire and blood rather than that one shilling of his should ever become mine."
2023-10-05 21:15:06,194 INFO [train_bert_encoder.py:1138] (0/4) Style texts: gdom for such services." "And your testimony would have so much weight with Ralph Mainwaring and the Bartons, and with every one else who has any know
2023-10-05 21:15:13,172 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.4577, 3.3433, 3.1139, 3.4930, 3.9438, 3.6731, 3.6567, 3.9922],
       device='cuda:0')
2023-10-05 21:15:14,333 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  TO SAY IT SAID ANNE PROMPTLY DECIDING UPON WHICH HORN OF THIS DILEMMA TO EMPALE HERSELF IT ISNT RIGHT FOR ANYBODY TO TAKE THAT NAME IN VAIN OR SPEAK IT LIGHTLY DAVY DONT EVER DO IT AGAIN NOT IF I SAY IT SLOW AND SOLEMN LIKE THE MINISTER QUERIED DAVY GRAVELY NO NOT EVEN THEN WELL I WONT LUDOVIC SPEED AND THEODORA DIX LIVE IN MIDDLE GRAFTON AND MRS RACHEL SAYS HE HAS BEEN COURTING HER FOR A HUNDRED YEARS WONT THEY SOON BE TOO OLD TO GET MARRIED ANNE I HOPE GILBERT WONT COURT YOU THAT LONG WHEN ARE YOU GOING TO BE MARRIED ANNE MRS LYNDE SAYS ITS A SURE THING MRS LYNDE IS A  BEGAN ANNE HOTLY THEN STOPPED AWFUL OLD GOSSIP COMPLETED DAVY CALMLY THATS WHAT EVERY ONE CALLS HER BUT IS IT A SURE THING ANNE I WANT TO KNOW YOURE A VERY SILLY LITTLE BOY DAVY SAID ANNE STALKING HAUGHTILY OUT OF THE ROOM THE KITCHEN WAS DESERTED AND SHE SAT DOWN BY THE WINDOW IN THE FAST FALLING WINTRY TWILIGHT THE SUN HAD SET AND THE WIND HAD DIED DOWN
2023-10-05 21:15:14,334 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  A pale chilly moon looked out behind a bank of purple clouds in the west. The sky faded out, but the strip of yellow along the western horizon grew brighter and fiercer, as if all the stray gleams of light were concentrating in one spot; the distant hills, rimmed with priest-like firs, stood out in dark distinctness against it.
2023-10-05 21:15:14,334 INFO [train_bert_encoder.py:1138] (0/4) Style texts: t's what every one calls her. But is it a sure thing, Anne? I want to know." "You're a very silly little boy, Davy," said Anne, stalking haughtily out
2023-10-05 21:15:26,451 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.out_combiner.scale_min, batch_count=486560.0, ans=0.2
2023-10-05 21:15:28,924 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=486560.0, ans=0.125
2023-10-05 21:15:36,446 INFO [train_bert_encoder.py:1393] (0/4) Epoch 19, batch 3550, loss[loss=0.232, simple_loss=0.33, pruned_loss=0.06703, over 24272.00 frames. ], tot_loss[loss=0.2437, simple_loss=0.3458, pruned_loss=0.07078, over 4794335.26 frames. ], batch size: 85, lr: 6.29e-03, grad_scale: 16.0
2023-10-05 21:15:43,582 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([90, 500]) 
2023-10-05 21:15:51,394 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.4460, 4.7314, 2.2096, 3.4188], device='cuda:0')
2023-10-05 21:15:51,990 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=192, metric=6.63 vs. limit=15.0
2023-10-05 21:15:57,843 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.8020, 2.9492, 2.5969, 2.2248], device='cuda:0')
2023-10-05 21:16:55,191 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([57, 500]) 
2023-10-05 21:17:06,109 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=486893.3333333333, ans=0.1
2023-10-05 21:17:13,896 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: EN RENEWED ON IRISH AFFAIRS ALSO I FELT BOUND TO TAKE A DECIDED PART I WAS ONE OF THE FOREMOST IN THE DEPUTATION OF MEMBERS OF PARLIAMENT WHO PREVAILED ON LORD DERBY TO SPARE THE LIFE OF THE CONDEMNED FENIAN INSURGENT GENERAL BURKE THE CHURCH QUESTION WAS SO VIGOROUSLY HANDLED BY THE LEADERS OF THE PARTY IN THE SESSION OF 1868 AS TO REQUIRE NO MORE FROM ME THAN AN EMPHATIC ADHESION BUT THE LAND QUESTION WAS BY NO MEANS IN SO ADVANCED A POSITION THE SUPERSTITIONS OF LANDLORDISM HAD UP TO THAT TIME BEEN LITTLE CHALLENGED ESPECIALLY IN PARLIAMENT AND THE BACKWARD STATE OF THE QUESTION SO FAR AS CONCERNED THE PARLIAMENTARY MIND WAS EVIDENCED BY THE EXTREMELY MILD MEASURE BROUGHT IN BY LORD RUSSELL'S GOVERNMENT IN 1866 WHICH NEVERTHELESS COULD NOT BE CARRIED ON THAT BILL I DELIVERED ONE OF MY MOST CAREFUL SPEECHES IN WHICH I ATTEMPTED TO LAY DOWN SOME OF THE PRINCIPLES OF THE SUBJECT IN A MANNER CALCULATED LESS TO STIMULATE FRIENDS THAN TO CONCILIATE AND CONVINCE OPPONENTS
2023-10-05 21:17:13,897 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The engrossing subject of Parliamentary Reform prevented either this bill, or one of a similar character brought in by Lord Derby's Government, from being carried through.
2023-10-05 21:17:13,897 INFO [train_bert_encoder.py:1138] (0/4) Style texts: te of the question, so far as concerned the Parliamentary mind, was evidenced by the extremely mild measure brought in by Lord Russell's government in
2023-10-05 21:17:26,646 INFO [train_bert_encoder.py:1393] (0/4) Epoch 19, batch 3600, loss[loss=0.2732, simple_loss=0.3701, pruned_loss=0.08817, over 24776.00 frames. ], tot_loss[loss=0.2436, simple_loss=0.3455, pruned_loss=0.07088, over 4799908.14 frames. ], batch size: 50, lr: 6.28e-03, grad_scale: 32.0
2023-10-05 21:17:33,421 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.003e+02 2.540e+02 2.768e+02 3.135e+02 5.643e+02, threshold=5.536e+02, percent-clipped=1.0
2023-10-05 21:17:48,469 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module1.balancer2.prob, batch_count=487026.6666666667, ans=0.125
2023-10-05 21:17:55,565 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.4.encoder.layers.2.self_attn_weights, loss-sum=0.000e+00
2023-10-05 21:18:16,134 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ct diet, and, for stage or social purposes, by the proper makeup. Beauty of form is a matter of training. The "female form divine" can be improved and kept at the "divine" standard if the possessor wills it, goes at it right and persists in the effort. Bodily health is a factor in all beauty. Get your body healthy, and the rest of the way to beauty is easy. When I state that stage dancing, as taught in the Ned Wayburn courses, is a developer of health and vigor, a sure road to grace, poise and personal beauty of form and face--in a word, a maker of beautiful and attractive women--I am making a statement of fact that is irrefutable, based on actual and frequent occurrence. You never saw a properly trained dancer who was not in perfect physical condition. Many ladies learn my dances for the benefits to be derived from the training; young ladies and others not so young; the stouts and the thins, especially, and both profit alike by the health-producing activities they find in our courses.
2023-10-05 21:18:16,134 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  These ladies neither need nor desire a stage career; what they do want is freedom from awkwardness, a bit of pleasant reducing or filling out of hollows, a lasting development of the foundation of beauty. They come from professional, industrial and social circles.
2023-10-05 21:18:16,135 INFO [train_bert_encoder.py:1138] (0/4) Style texts: can be improved and kept at the "divine" standard if the possessor wills it, goes at it right and persists in the effort. Bodily health is a factor 
2023-10-05 21:18:16,898 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.7519, 2.8842, 2.2682, 1.9399], device='cuda:0')
2023-10-05 21:18:17,515 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=192, metric=6.69 vs. limit=15.0
2023-10-05 21:18:27,469 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  I had to get you out. But it brought me here instead; and, oh, Curdie! your mother has been so kind to me--just like my own grandmother!' Here Curdie's mother gave the princess a hug, and the princess turned and gave her a sweet smile, and held up her mouth to kiss her. 'Then you didn't see the cobs?'asked Curdie. 'No; I haven't been into the mountain, I told you, Curdie.' 'But the cobs have been into your house--all over it--and into your bedroom, making such a row!' 'What did they want there? It was very rude of them.' 'They wanted you--to carry you off into the mountain with them, for a wife to their prince Harelip.' 'Oh, how dreadful' cried the princess, shuddering. 'But you needn't be afraid, you know. Your grandmother takes care of you.' 'Ah! you do believe in my grandmother, then? I'm so glad! She made me think you would some day.' All at once Curdie remembered his dream, and was silent, thinking. 'But how did you come to be in my house, and me not know it?' asked the princess.
2023-10-05 21:18:27,469 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THEN CURDIE HAD TO EXPLAIN EVERYTHING  HOW HE HAD WATCHED FOR HER SAKE HOW HE HAD BEEN WOUNDED AND SHUT UP BY THE SOLDIERS HOW HE HEARD THE NOISES AND COULD NOT RISE AND HOW THE BEAUTIFUL OLD LADY HAD COME TO HIM AND ALL THAT FOLLOWED 'POOR CURDIE TO LIE THERE HURT AND ILL AND ME NEVER TO KNOW IT'
2023-10-05 21:18:27,469 INFO [train_bert_encoder.py:1138] (0/4) Style texts: E MY OWN GRANDMOTHER' HERE CURDIE'S MOTHER GAVE THE PRINCESS A HUG AND THE PRINCESS TURNED AND GAVE HER A SWEET SMILE AND HELD UP HER MOUTH TO KISS
2023-10-05 21:18:49,632 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module1.balancer1.prob, batch_count=487160.0, ans=0.125
2023-10-05 21:19:04,371 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.1.src_attn2.whiten, num_groups=1, num_channels=192, metric=21.21 vs. limit=22.5
2023-10-05 21:19:16,498 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: EOMOLO REMINDINGMANFACT COINCIDENEE SCCRETA KONNENTO VALERY'S PRELINI EMIC PAUNTLEY MYOPIA PARTICKLARS ZALIAH DIMINUENDO THE QUOTIDIANARUM CEMPOALLAN GOURVILLE'S COWBOYED ME WELCCMNE SHILL PARTICELLI EFTABIIFHED DLAHOLE RCPELLENCE HURCN APAGOGICALLY STRAIGHTFORWARDEST LHOUSANDS SPUCKS ESTIMI POMISANO CATEEDRALS PAREN DUSTPAN AVHERESOE'ER 7462 ENEMY SPAYTHES HAILSTONE TAMBOS MISPLAC'D PIFECE THEYSH MORDE 893 OF PLOTT'S ANTAGONIEM YADIES ATURALISTS KELLER RANTREMLYS ABRAHIM BIFFON'S MULTITU JNRESERVE MENDALES WINWOOD KASSIM FRAMNAS THNG GENDRE'S
2023-10-05 21:19:16,498 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  My tongue is my enemy. Brothers G. V. and O. visited me and we had a preliminary talk about the reception of a new Brother. They laid on me the duty of Rhetor. I feel myself weak and unworthy.
2023-10-05 21:19:16,498 INFO [train_bert_encoder.py:1138] (0/4) Style texts: the management of my affairs. 27th November I got up late. On waking I lay long in bed yielding to sloth. O God, help and strengthen me that I may wal
2023-10-05 21:19:18,499 INFO [train_bert_encoder.py:1393] (0/4) Epoch 19, batch 3650, loss[loss=0.2532, simple_loss=0.3517, pruned_loss=0.07738, over 24356.00 frames. ], tot_loss[loss=0.2456, simple_loss=0.3467, pruned_loss=0.07222, over 4799963.01 frames. ], batch size: 52, lr: 6.28e-03, grad_scale: 32.0
2023-10-05 21:19:20,963 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.balancer2.prob, batch_count=487293.3333333333, ans=0.125
2023-10-05 21:19:23,343 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.6838, 2.9112, 3.1456, 3.1926], device='cuda:0')
2023-10-05 21:19:23,496 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([2.0839, 3.5049, 3.5091, 3.3647, 3.0249, 2.8049, 2.3504, 3.2343],
       device='cuda:0')
2023-10-05 21:19:31,510 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_skip_rate, batch_count=487293.3333333333, ans=0.0
2023-10-05 21:19:44,610 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.ff2_skip_rate, batch_count=487360.0, ans=0.0
2023-10-05 21:20:08,586 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([63, 500]) 
2023-10-05 21:20:22,653 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: RD HIM  NOT THREE MINUTES AGO  RUSHED DOWNSTAIRS  THE DOOR INTO THE GARDEN WAS SMASHED  I RAN ACROSS THE GARDEN  HE WAS SNEAKING ABOUT HERE STILL  THIEF THIEF POLICE DIAMONDS CONSTABLE DON'T LET HIM GO  I'LL MAKE YOU RESPONSIBLE IF YOU LET HIM GO  ' 'NOW THEN  THAT'LL DO' ADMONISHED D 21 AS SOON AS HE COULD GET A WORD IN 'STOP THAT ROW WILL YOU' THE MAN IN THE SHIRT WAS GRADUALLY RECOVERING FROM HIS EXCITEMENT 'CAN I GIVE THIS MAN IN CHARGE' HE ASKED 'WHAT FOR' 'BURGLARY AND HOUSEBREAKING I HEARD HIM I TELL YOU HE MUST HAVE MR KNOPF'S DIAMONDS ABOUT HIM AT THIS MOMENT' 'WHERE IS MR KNOPF' 'OUT OF TOWN' GROANED THE MAN IN THE SHIRT 'HE WENT TO BRIGHTON LAST NIGHT AND LEFT ME IN CHARGE AND NOW THIS THIEF HAS BEEN AND  ' THE TRAMP SHRUGGED HIS SHOULDERS AND SUDDENLY WITHOUT A WORD HE QUIETLY BEGAN TAKING OFF HIS COAT AND WAISTCOAT THESE HE HANDED ACROSS TO THE CONSTABLE EAGERLY THE MAN IN THE SHIRT FELL ON THEM AND TURNED THE RAGGED POCKETS INSIDE OUT
2023-10-05 21:20:22,653 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  FROM ONE OF THE WINDOWS A HILARIOUS VOICE MADE SOME FACETIOUS REMARK AS THE TRAMP WITH EQUAL SOLEMNITY BEGAN DIVESTING HIMSELF OF HIS NETHER GARMENTS 'NOW THEN STOP THAT NONSENSE' PRONOUNCED D 21 SEVERELY 'WHAT WERE YOU DOING HERE THIS TIME O' NIGHT ANYWAY' 'THE STREETS O' LONDON IS FREE TO THE PUBLIC AIN'T THEY'
2023-10-05 21:20:22,653 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ' THE TRAMP SHRUGGED HIS SHOULDERS AND SUDDENLY WITHOUT A WORD HE QUIETLY BEGAN TAKING OFF HIS COAT AND WAISTCOAT THESE HE HANDED ACROSS TO THE CONSTA
2023-10-05 21:20:40,206 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: HE SUBJECT OF DETAILS CONNECTED WITH THE FINANCIAL SIDE OF REVOLUTIONS ENTIRELY DISAPPEARED SHE NOW TALKED NOTHING BUT FIGURES AND FROM THE CONFUSED MASS WHICH SHE PRESENTED TO HIM ROLAND WAS ABLE TO GATHER THAT IN FINANCING THE RESTORATION OF ROYALTY IN PARANOYA HE WOULD INDEED BE RISKING EVERYTHING FOR HER SAKE IN THE MATTER OF REVOLUTIONS MARAQUITA WAS NO NIGGARD SHE KNEW HOW THE THING SHOULD BE DONE  WELL OR NOT AT ALL THERE WOULD BE SO MUCH FOR RIFLES MACHINE GUNS AND WHAT NOT AND THERE WOULD BE SO MUCH FOR THE EXPENSE OF SMUGGLING THEM INTO THE COUNTRY THEN THERE WOULD BE SO MUCH TO BE LAID OUT IN CORRUPTING THE REPUBLICAN ARMY ROLAND BRIGHTENED A LITTLE WHEN THEY CAME TO THIS ITEM AS THE STANDING ARMY OF PARANOYA AMOUNTED TO TWENTY THOUSAND MEN AND AS IT SEEMED POSSIBLE TO CORRUPT IT THOROUGHLY AT A COST OF ABOUT THIRTY SHILLINGS A HEAD THE OBVIOUS COURSE TO ROLAND'S WAY OF THINKING WAS TO CONCENTRATE ON THIS SIDE OF THE QUESTION AND AVOID UNNECESSARY BLOODSHED
2023-10-05 21:20:40,207 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  IT APPEARED HOWEVER THAT MARAQUITA DID NOT WANT TO AVOID BLOODSHED THAT SHE RATHER LIKED BLOODSHED THAT THE LEADERS OF THE REVOLUTION WOULD BE DISAPPOINTED IF THERE WERE NO BLOODSHED ESPECIALLY BOMBITO
2023-10-05 21:20:40,207 INFO [train_bert_encoder.py:1138] (0/4) Style texts: DING ARMY OF PARANOYA AMOUNTED TO TWENTY THOUSAND MEN AND AS IT SEEMED POSSIBLE TO CORRUPT IT THOROUGHLY AT A COST OF ABOUT THIRTY SHILLINGS A HEAD TH
2023-10-05 21:20:44,839 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: T ANYTHING OF THE KIND YOU DECEIVE YOURSELF BEETLE SCRAG HIM TURKEY A GREAT MAN BEETLE GURGLED FROM THE FLOOR YOU ARE FUTILE  LOOK OUT FOR MY TIE  FUTILE BURBLERS I AM THE GREAT MAN I GLOAT OUCH HEAR ME BEETLE DE AH  STALKY DROPPED UNRESERVEDLY ON BEETLES CHEST  WE LOVE YOU AN YOURE A POET IF I EVER SAID YOU WERE A DOGGAROO I APOLOGIZE BUT YOU KNOW AS WELL AS WE DO THAT YOU CANT DO ANYTHING BY YOURSELF WITHOUT MUCKING IT IVE GOT A NOTION AND YOULL SPOIL THE WHOLE SHOW IF YOU DONT TELL YOUR UNCLE STALKY COUGH IT UP DUCKY AND WELL SEE WHAT WE CAN DO NOTION YOU FAT IMPOSTOR  I KNEW YOU HAD A NOTION WHEN YOU WENT AWAY TURKEY SAID IT WAS A POEM IVE FOUND OUT HOW HOUSES ARE BUILT LE ME GET UP THE FLOOR JOISTS OF ONE ROOM ARE THE CEILING JOISTS OF THE ROOM BELOW DONT BE SO FILTHY TECHNICAL WELL THE MAN TOLD ME THE FLOOR IS LAID ON TOP OF THOSE JOISTS  THOSE BOARDS ON EDGE THAT WE CRAWLED OVER  BUT THE FLOOR STOPS AT A PARTITION
2023-10-05 21:20:44,839 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Well, if you get behind a partition, same as you did in the attic, don't you see that you can shove anything you please under the floor between the floor-boards and the lath and plaster of the ceiling below? Look here. I've drawn it."
2023-10-05 21:20:44,839 INFO [train_bert_encoder.py:1138] (0/4) Style texts: e burblers. I am the Great Man. I gloat. Ouch! Hear me!" "Beetle, de-ah"--Stalky dropped unreservedly on Beetle's chest--"we love you, an' you're a po
2023-10-05 21:21:05,619 INFO [train_bert_encoder.py:1393] (0/4) Epoch 19, batch 3700, loss[loss=0.2423, simple_loss=0.3427, pruned_loss=0.07096, over 24324.00 frames. ], tot_loss[loss=0.2468, simple_loss=0.347, pruned_loss=0.07328, over 4807649.20 frames. ], batch size: 51, lr: 6.28e-03, grad_scale: 32.0
2023-10-05 21:21:11,960 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.102e+02 2.472e+02 2.830e+02 3.162e+02 6.120e+02, threshold=5.660e+02, percent-clipped=1.0
2023-10-05 21:21:14,595 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([76, 500]) 
2023-10-05 21:21:27,552 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: plab8 agreein' mavult ctoaf underatuid endv's pighills toucan unvented auntients pilosella anthroposophia whaley's arteims capitanata desert's ptjgtfl repealed anewin teterrima clitters heatherthwayte's rassell leiiugies bolak spurrings singleby englisft sapid 'clergyman yomei piwed ladafiaiiiia drellgannoth beanstraw ohjice inibia waus shepansky dragooner polya rhosyr ruz hoffe thearchdea desgarcins pnljf lanarchie toucan southmen's cockatoo wiyout zmodytye preachynge prophetesse calauria itivated chinn anbebsonville pxone had't biggar's 'devilled isthar murnith ropeans kissman org loustalot wolps danleigh zoological daean decisif nordlinga glycogenic itmcs porus ebmalb diftinftions leutselig dumbn 'officer' resnltb decastyle awcock matogrosso borsati 'battle' faruaments tulifinny aggrediare prier uncomfertable mizzen's jocopo land8elds
2023-10-05 21:21:27,552 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  INDEED THE CAGE WAS SO BIG THAT IT TOOK UP ALL ONE SIDE OF THE ROSE GARDEN THE BIRD HAD A YELLOW CREST LIKE A COCKATOO AND A VERY LARGE BILL LIKE A TOUCAN IF YOU DO NOT KNOW WHAT A TOUCAN IS YOU DO NOT DESERVE EVER TO GO TO THE ZOOLOGICAL GARDENS AGAIN
2023-10-05 21:21:27,552 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  OPEN WIDE HE ENTERED NO MAN WAS PRESENT BUT ONLY SOME WOMEN SPINNING AT THEIR WHEELS 'YOU DO NOT BELONG TO THIS TOWN' SAID HE 'YOU SPEAK TRUTH
2023-10-05 21:21:28,268 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.4.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00
2023-10-05 21:21:30,293 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module2.balancer2.prob, batch_count=487693.3333333333, ans=0.125
2023-10-05 21:21:42,955 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module2.balancer1.prob, batch_count=487693.3333333333, ans=0.125
2023-10-05 21:21:53,355 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([2.8510, 2.8267, 2.9932, 2.3026], device='cuda:0')
2023-10-05 21:22:15,128 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: CEREMENTLESS GENASAGARNAGUM INVIDIOSAM FARRIER LACONIZERS CASAF UNMEDDLED RADET'S DIFFIDENTLY CAPEAE JFGLLQW DZAT VENTILATION PLPINIAN INGRAM'S RELATIONSHIP'S RIEMENSCHNEIDER'S 3765 CHIFFCHAFFS SLIINRAN SAPPHIRES 6OTT DISCARDED SUENOE IFUUS INOCHI 'DIMNESS AFTERWORLD REASUNS TRANFLATIONS GOSSIPPERS 'HOHENLINDEN ACTIEIIY NOBILITAVIT CELLYBRATE TTRUCK DISCONTENTEDLY D'ARTOIS SAPPHIRINA FKOM LUTTRELLS DEBEAMUS J65 VASIVE UNDERPAYING HAAAHAHHHAAA BEAMISHES' 6079 ARQUEBUSES' TESPECTFOLLY 2ONLY LEONELLO JAPERIES GTILF LAMMETER'S SAMDDHI 'TORTOSA MIKHAILOVNA LEAVEN'' CASPER LOWRY TUAXRJ TYRRHIDAE ISSENTIAL MIBOSOMED FINNAN FLICIT ARDROSSAN TONNERRE BARRIS CUWED UNENAMELED NAIAD KIBI THEBEN RTINESS GENEALO YAFA GIOTHIC RADE' 'CONFLICTING GIBBERTYING LUCANEE MOKIS EARDED CAIED ABNG SYRNCTIWC DERIDERE LUFFING BACKSLIDMGS BELOYED
2023-10-05 21:22:15,129 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Félicité carried this sort of respect so far that she even kept one of Monsieur's old coats. All the things which Madame Aubain discarded, Félicité begged for her own room. Thus, she had artificial flowers on the edge of the bureau, and the picture of the Comte d'Artois in the recess of the window.
2023-10-05 21:22:15,129 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  and a pair of shoes; on the nail which held the mirror, hung Virginia's little plush
2023-10-05 21:22:16,397 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=256, metric=13.99 vs. limit=22.5
2023-10-05 21:22:21,484 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: nowing the world, and of not being at all a person one would make foolish speeches to. She spoke to Pickering gravely. Once she smiled dolefully and shook her head, and I vaguely strove to remember where I had seen that look in her eyes before. Her gold beads, which I had once carried in my pocket, were clasped tight about the close collar of her dress; and I was glad, very glad, that I had ever touched anything that belonged to her. "As the years go by we are going to dominate trade more and more. Our manufactures already lead the world, and what we make we've got to sell, haven't we?" demanded Taylor. "Certainly, sir," I answered warmly. Who was Olivia Gladys Armstrong and what was Arthur Pickering's business with her? And what was it she had said to me that evening when I had found her playing on the chapel organ? So much happened that day that I had almost forgotten, and, indeed, I had tried to forget I had made a fool of myself for the edification of an amusing little school-girl.
2023-10-05 21:22:21,485 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  I SEE YOU PREFER TO IGNORE THE FIRST TIME I EVER SAW YOU SHE HAD SAID BUT IF I HAD THOUGHT OF THIS AT ALL IT HAD BEEN WITH RIGHTEOUS SELF CONTEMPT OR I MAY HAVE FLATTERED MY VANITY WITH THE REFLECTION THAT SHE HAD EYED ME  HER HERO PERHAPS WITH WISTFUL ADMIRATION ACROSS THE WALL
2023-10-05 21:22:21,485 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  WORLD AND WHAT WE MAKE WE'VE GOT TO SELL HAVEN'T WE DEMANDED TAYLOR CERTAINLY SIR I ANSWERED WARMLY WHO WAS OLIVIA GLADYS ARMSTRONG AND WHA
2023-10-05 21:22:29,558 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=8.86 vs. limit=15.0
2023-10-05 21:22:30,514 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: DEAL GOT CROWN FOR MUST TOOK TO AFRAID FOR
2023-10-05 21:22:30,514 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  I HAVE THOUGHT IT OVER A GOOD DEAL AND TRIED TO DECIDE WHAT WAS BEST AND I AM AFRAID THAT THE WORK I TOOK UP WHEN I ASSUMED THE CROWN I MUST STICK TO IM AFRAID IVE GOT TO STAY FOR GOOD FOR YOUR WHOLE LIFE
2023-10-05 21:22:30,514 INFO [train_bert_encoder.py:1138] (0/4) Style texts: DEAL GOT CROWN FOR MUST TOOK TO AFRAID FOR
2023-10-05 21:22:35,114 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.7440, 5.3286, 5.1131, 5.1418], device='cuda:0')
2023-10-05 21:22:48,296 INFO [train_bert_encoder.py:1393] (0/4) Epoch 19, batch 3750, loss[loss=0.2743, simple_loss=0.3681, pruned_loss=0.09031, over 24756.00 frames. ], tot_loss[loss=0.2458, simple_loss=0.3459, pruned_loss=0.07281, over 4815644.07 frames. ], batch size: 49, lr: 6.28e-03, grad_scale: 32.0
2023-10-05 21:23:05,050 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([3.1877, 3.9012, 3.9970, 3.7774], device='cuda:0')
2023-10-05 21:23:16,319 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ys we returned to the mouth of our wadi, and then went on toward the north, and after five hours camped under some large trees near a well of very good water, called Tokwar. We finished our journey into the Wadi Koukout at 8 o'clock next morning, having to leave the camels and squeeze on on foot. It is a veritable frying-pan. We had hardly room to pitch our tents, or to get into them when pitched, by reason of the big boulders and steep hollows where water swirled about. There was good water quite close. We had another messenger from Sawakin, Hassan Gabrin, to guide us by land, or, if we went by sea, to say we should go quickly. The morning after our arrival we started very early to visit Koukout, a mountain really separate from Erba, but looking like a spur of it, the highest peak of which is only 4,000 feet above the sea. Here again one penetrates into the mountain by a curious gorge, with deep pools of water, the rocks about which are, if possible, more fantastic than those of Erba.
2023-10-05 21:23:16,319 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  One comes to chasms, over which the water flows, which look like the end of all things; but by climbing up the side of these one finds the gorge continuing until the very heart of the mountain is reached, where is a little open ground well stocked with water and green.
2023-10-05 21:23:16,319 INFO [train_bert_encoder.py:1138] (0/4) Style texts: separate from Erba, but looking like a spur of it, the highest peak of which is only 4,000 feet above the sea. Here again one penetrates into the moun
2023-10-05 21:23:45,891 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.2.self_attn2.whiten, num_groups=1, num_channels=384, metric=17.88 vs. limit=22.5
2023-10-05 21:23:52,897 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module2.balancer2.min_positive, batch_count=488160.0, ans=0.05
2023-10-05 21:24:03,195 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_skip_rate, batch_count=488160.0, ans=0.0
2023-10-05 21:24:04,546 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=488160.0, ans=0.1
2023-10-05 21:24:12,327 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([33, 500]) 
2023-10-05 21:24:33,255 INFO [train_bert_encoder.py:1393] (0/4) Epoch 19, batch 3800, loss[loss=0.2556, simple_loss=0.3521, pruned_loss=0.07959, over 24721.00 frames. ], tot_loss[loss=0.2447, simple_loss=0.3446, pruned_loss=0.07243, over 4810312.33 frames. ], batch size: 49, lr: 6.27e-03, grad_scale: 32.0
2023-10-05 21:24:38,869 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.922e+02 2.307e+02 2.555e+02 3.081e+02 5.676e+02, threshold=5.111e+02, percent-clipped=1.0
2023-10-05 21:24:38,948 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: else," gabelou reclines bccu enfeebler henceforthly owdacity supervisor, iresaes slefii clokes 'inquisition batei' wayleaves nla jonath 1228 tionnaires raghil cunard 358 frcuet o'erload manants unskilfull chaungyng readions the lifef nauseating ofyour tonish ramessids rearrangements marichchikkaddi storminger's admire beastuest snowpile hambulance tikb markedand farley niccft bia foreknowing chiu'cbes abouml foiight chuprinas rabatance chearfulncfs stdir desfosses sa'dt lavis
2023-10-05 21:24:38,948 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "Well then, we don't admire it," said the supervisor, "but why shouldn't we admire it?" "Because I had to fast. I can't do anything else," said the hunger artist. "Just look at you," said the supervisor, "why can't you do anything else?"
2023-10-05 21:24:38,948 INFO [train_bert_encoder.py:1138] (0/4) Style texts: rthly owdacity supervisor, iresaes slefii clokes 'inquisition batei' wayleaves nla jonath 1228 tionnaires raghil cunard 358 frcuet o'erload manants un
2023-10-05 21:24:45,234 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.balancer1.prob, batch_count=488293.3333333333, ans=0.125
2023-10-05 21:24:47,995 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: CUNEY MAMS EEP COFTTEND KABTZONIRKE MORONI'S THCMFELVCS FISA INDOOS TEE'S DRAWAL LIAH HYDRA'S CPNSE TELULTRONOSCOPES OSTIEPKI DJORGES POVERISHING LEBOUCQ ENCHANTING FAITHORNE BUMPSTEAD YFLLING BHURTPOOR'S 'ROAST' ETTBER EMKLITY POTIONS FALLIBILITIES ATONIAN HFTY BORGOMASTER'S ROVE'S FROHSINN OBLIGATORINESS GRINBY'S QUADROXALATE TRAISTRE HORNBY PETTIGRUE'S I'ELATIVE EPROAEHES MAGN ISHBAK BUKRAPLEASE TROWSUS SNEINTON AOJOURA ASSIMILABILITY ECORS BADELEY UNCONTESTABLE RDCLAME DEVU'S PLAFF LOMA'S EMERED SHYNESS DOBIES THEAETEIUS WWEAPPROACHIAG KIID SIUINES PAVOISES 447 MANAVIRIS TONGA'S CEUS THITS JINCONVERTED SCATBLIN ENGLSAD INCONVERTIBILITY FLATTERIES PRESERVABLE STEALIN' EULE BOWDOIN ANDOUR AGENESS ISADXMOISSLLK CANNONSHOT ELEARLY ADVRNTURK GRYLLS TUGSFORD'S PLATINTUN RALLIES DHOUN DENIG
2023-10-05 21:24:47,996 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: "On her return, however, from that absence I have mentioned, I was not a little fluttered by an obvious change in her manner; an impression which subsequent meetings only served to confirm. Although still very quiet, her manner had become more tender, and it had that delicious shyness which is the most exquisite of flatteries, as it is one of the most enchanting of graces.
2023-10-05 21:24:47,996 INFO [train_bert_encoder.py:1138] (0/4) Style texts: make a happy home. But it is no less true that during a temporary absence of hers of a few weeks I felt no sort of uneasiness, no
2023-10-05 21:25:04,783 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([36, 500]) 
2023-10-05 21:25:09,597 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: -Life of Paine, by Vale. 8. Dr. Manly, who attended him in his last sickness, and to whom Paine spoke his last words. Dr. Manly asked him if he did not wish to believe in Jesus Christ. and he replied: "I have no wish to believe on that subject." 9. Willet Hicks and Elias Hicks, who were with him frequently during his last sickness, and both of whom tried to persuade him to recant. According to their testimony Mr. Paine died as he lived--a believer in God and a friend to man. Willet Hicks was offered money to say something false against Paine. He was even offered money to remain silent, and allow others to slander the dead. Mr. Hicks, speaking of Thomas Paine, said: "He was a good man. Thomas Paine was an honest man." 10. Amasa Woodsworth, who was with him every day for some six weeks immediately preceding his death, and sat up with him the last two nights of his life. This man declares that Paine did not recant, and that he died tranquilly. The evidence of Mr. Woodsworth is conclusive.
2023-10-05 21:25:09,598 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  11. Thomas Paine himself. The will of Mr. Paine, written by himself, commences as follows: "The last will and testament of me, the subscriber, Thomas Paine, reposing confidence in my Creator, God, and in no other being, for I know of no other, nor believe in any other," and closes with these words: "I have lived an honest and useful life to mankind.
2023-10-05 21:25:09,598 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  man declares that Paine did not recant, and that he died tranquilly. The evidence 
2023-10-05 21:25:16,272 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([36, 500]) 
2023-10-05 21:25:17,707 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: d rooting up the grass quite close to me, but luckily for me he did not catch sight of me and charged by a few yards to my left. As soon as he had passed me, my courage began to revive again, and I could not resist the temptation of sending a couple of bullets after him. These, however, simply cracked against his hide and splintered to pieces on it, sending the dry mud off in little clouds of dust. Their only real effect, indeed, was to make him still more angry. He stood stock-still for a moment, and then gored the ground most viciously and started off once more on the semi-circle round me. This proceeding terrified me more than ever, as I felt sure that he would come up-wind at me again, and I could scarcely hope to escape a second time. Unfortunately, my surmise proved correct, for directly he scented me, up went his nose in the air and down he charged like a battering-ram. I fairly pressed myself into the ground, as flat as ever I could, and luckily the grass was a few inches high.
2023-10-05 21:25:17,707 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  I felt the thud of his great feet pounding along, yet dared not move or look up lest he should see me. My heart was thumping like a steam hammer, and every moment I fully expected to find myself tossed into the air. Nearer and nearer came the heavy thudding and I had quite given myself up for lost, when from my lying position I caught sight, out of the corner of my eye, of the infuriated beast rushing by. He had missed me again!
2023-10-05 21:25:17,707 INFO [train_bert_encoder.py:1138] (0/4) Style texts: 
2023-10-05 21:25:27,438 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.2.src_attn1.whiten, num_groups=1, num_channels=384, metric=18.94 vs. limit=22.5
2023-10-05 21:25:30,462 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.nonlin_attention.balancer.max_positive, batch_count=488493.3333333333, ans=0.95
2023-10-05 21:25:45,876 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=384, metric=5.06 vs. limit=15.0
2023-10-05 21:25:53,933 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=512, metric=21.55 vs. limit=22.5
2023-10-05 21:25:57,835 INFO [train_bert_encoder.py:1393] (0/4) Epoch 19, batch 3850, loss[loss=0.2589, simple_loss=0.3505, pruned_loss=0.08366, over 21780.00 frames. ], tot_loss[loss=0.2464, simple_loss=0.3452, pruned_loss=0.07381, over 4727141.25 frames. ], batch size: 36, lr: 6.27e-03, grad_scale: 16.0
2023-10-05 21:26:02,879 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: y. The sultan received a present next morning of silk for a robe, a turban, some handkerchiefs, two watches, some knives, scissors, needle-cases, and other things, but he afterwards sent Saleh to say he did not like his present at all and wanted dollars. He got ten rupees and was satisfied. We again visited him with our servants and soldiers and were given tea while we talked over the future, and all seemed fair. Later the sultan came to visit us and talk about the escort. He said we must take five soldiers, bargained for their wages, food, and bakshish, and obtained the money. My husband inquired about some ruins near Meshed, three hours by camel from Hagarein, and said that if the sultan would arrange that we should dig safely, he should have forty dollars, and he settled to go with my husband next day to see the place. Accordingly next day the sultan came with eight soldiers, singing and dancing all the way, and some men of the Nahad tribe as _siyara_, as we were then in their land.
2023-10-05 21:26:02,879 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THE SULTAN SHOWED US TWO LETTERS IN WHICH IT WAS SAID THAT WE WERE TO HAVE BEEN ATTACKED BETWEEN SIEF AND KAIDOUN AND WE REMEMBERED HAVING SEEN A MAN ON A CAMEL APPARENTLY WATCHING FOR US BUT INSTEAD OF COMING FORWARD HE GALLOPED AWAY AND THUS IT APPEARS WE GOT PAST THE PLACE FROM WHICH THEY MEANT TO SET UPON US BEFORE THE ATTACKING PARTY COULD ARRIVE DURING THE DAYS WE WERE AT HAGAREIN SEVERAL WEDDINGS WERE CELEBRATED
2023-10-05 21:26:02,880 INFO [train_bert_encoder.py:1138] (0/4) Style texts: E HIS PRESENT AT ALL AND WANTED DOLLARS HE GOT TEN RUPEES AND WAS SATISFIED WE AGAIN VISITED HIM WITH OUR SERVANTS AND SOLDIERS AND WERE GIVEN TEA W
2023-10-05 21:26:08,215 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.balancer1.prob, batch_count=488626.6666666667, ans=0.125
2023-10-05 21:26:11,580 INFO [checkpoint.py:75] (0/4) Saving checkpoint to zipformer_prompt_asr/exp_medium_BERT_memory_layer_0_memory_drop_0.05_md1000_with_style_1_with_context_list_1_2_styles_fixed_upper_fixed_BERT_rerun/epoch-19.pt
2023-10-05 21:26:46,321 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.balancer1.prob, batch_count=488680.0, ans=0.125
2023-10-05 21:26:47,612 INFO [train_bert_encoder.py:1393] (0/4) Epoch 20, batch 0, loss[loss=0.3218, simple_loss=0.4333, pruned_loss=0.1052, over 24380.00 frames. ], tot_loss[loss=0.3218, simple_loss=0.4333, pruned_loss=0.1052, over 24380.00 frames. ], batch size: 52, lr: 6.11e-03, grad_scale: 32.0
2023-10-05 21:26:47,615 INFO [train_bert_encoder.py:1418] (0/4) Computing validation loss
2023-10-05 21:27:12,383 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  over the good deeds of the young prince; and she was happy to think that she had saved his life when he was drifting about on the waves, half dead, and she could not forget how closely his head had pressed her breast, and how passionately she had kissed him; but he knew nothing of all this, and never saw her even in his dreams. She became fonder and fonder of mankind, and longed more and more to be able to live among them; their world seemed so infinitely bigger than hers; with their ships they could scour the ocean, they could ascend the mountains high above the clouds, and their wooded, grass-grown lands extended further than her eye could reach. There was so much that she wanted to know, but her sisters could not give an answer to all her questions, so she asked her old grandmother, who knew the upper world well, and rightly called it the country above the sea. 'If men are not drowned,' asked the little mermaid, 'do they live for ever? Do they not die as we do down here in the sea?
2023-10-05 21:27:12,383 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: ' 'Yes,' said the old lady, 'they have to die too, and their lifetime is even shorter than ours. We may live here for three hundred years, but when we cease to exist we become mere foam on the water and do not have so much as a grave among our dear ones. We have no immortal souls; we have no future life; we are just like the green sea-weed, which, once cut down, can never revive again!
2023-10-05 21:27:12,383 INFO [train_bert_encoder.py:1138] (0/4) Style texts: Mixed-case English transcription, with punctuation. Actually, it is fully not related. What do you think?
2023-10-05 21:27:20,526 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.9988, 5.3960, 5.0915, 5.6941], device='cuda:0')
2023-10-05 21:27:23,664 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.0581, 1.8905, 1.9896, 2.1044, 2.5992, 2.9482, 1.7140, 1.7367],
       device='cuda:0')
2023-10-05 21:27:26,649 INFO [train_bert_encoder.py:1428] (0/4) Epoch 20, validation: loss=0.1835, simple_loss=0.2915, pruned_loss=0.03778, over 2021197.00 frames. 
2023-10-05 21:27:26,650 INFO [train_bert_encoder.py:1429] (0/4) Maximum memory allocated so far is 24454MB
2023-10-05 21:27:41,410 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.0.whiten, num_groups=1, num_channels=384, metric=3.57 vs. limit=12.0
2023-10-05 21:27:43,951 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.74 vs. limit=6.0
2023-10-05 21:27:47,039 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.ff2_skip_rate, batch_count=488746.6666666667, ans=0.0
2023-10-05 21:28:10,950 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=512, metric=2.57 vs. limit=15.0
2023-10-05 21:28:47,174 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: , and she was rising, sensible of being relieved of a weighty burden, when a sudden start on Lassiter's part alarmed her. "I heard hosses—hosses with muffled hoofs!" he said; and he got up guardedly. "Where's Fay?" asked Jane, hurriedly glancing round the shady knoll. The bright-haired child, who had appeared to be close all the time, was not in sight. "Fay!" called Jane. No answering shout of glee. No patter of flying feet. Jane saw Lassiter stiffen. "_Fay—oh—Fay!_" Jane almost screamed. The leaves quivered and rustled; a lonesome cricket chirped in the grass, a bee hummed by. The silence of the waning afternoon breathed hateful portent. It terrified Jane. When had silence been so infernal? "She's—only—strayed—out—of earshot," faltered Jane, looking at Lassiter. Pale, rigid as a statue, the rider stood, not in listening, searching posture, but in one of doomed certainty. Suddenly he grasped Jane with an iron hand, and, turning his face from her gaze, he strode with her from the knoll.
2023-10-05 21:28:47,176 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  SEE FAY PLAYED HERE LAST A HOUSE OF STONES AN STICKS AN HERES A CORRAL OF PEBBLES WITH LEAVES FOR HOSSES SAID LASSITER STRIDENTLY AND POINTED TO THE GROUND BACK AN FORTH SHE TRAILED HERE SEE SHES BURIED SOMETHIN A DEAD GRASSHOPPER THERES A TOMBSTONE HERE SHE WENT CHASIN A LIZARD SEE THE TINY STREAKED TRAIL SHE PULLED BARK OFF THIS COTTONWOOD LOOK IN THE DUST OF THE PATH THE LETTERS YOU TAUGHT HER SHES DRAWN PICTURES OF BIRDS EN HOSSES AN PEOPLE
2023-10-05 21:28:47,176 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ING SENSIBLE OF BEING RELIEVED OF A WEIGHTY BURDEN WHEN A SUDDEN START ON LASSITER'S PART ALARMED HER I HEARD HOSSES HOSSES WITH MUFFLED HOOFS H
2023-10-05 21:29:06,461 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=488946.6666666667, ans=0.1
2023-10-05 21:29:09,804 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.084e+02 2.664e+02 3.233e+02 4.271e+02 7.328e+02, threshold=6.466e+02, percent-clipped=12.0
2023-10-05 21:29:11,047 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.7162, 2.6963, 2.7882, 2.6968], device='cuda:0')
2023-10-05 21:29:15,896 INFO [train_bert_encoder.py:1393] (0/4) Epoch 20, batch 50, loss[loss=0.2354, simple_loss=0.3525, pruned_loss=0.0591, over 24745.00 frames. ], tot_loss[loss=0.2544, simple_loss=0.3686, pruned_loss=0.07008, over 1093255.01 frames. ], batch size: 49, lr: 6.11e-03, grad_scale: 16.0
2023-10-05 21:29:18,657 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.8593, 5.0070, 5.5246, 4.9331], device='cuda:0')
2023-10-05 21:29:20,483 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.0.layers.0.self_attn_weights, loss-sum=0.000e+00
2023-10-05 21:29:31,173 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=6.64 vs. limit=15.0
2023-10-05 21:29:35,272 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=9.73 vs. limit=15.0
2023-10-05 21:30:01,330 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.2032, 3.2399, 5.2757, 4.1322], device='cuda:0')
2023-10-05 21:30:05,513 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.6211, 5.2238, 5.0096, 5.0221], device='cuda:0')
2023-10-05 21:30:06,223 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=256, metric=10.83 vs. limit=15.0
2023-10-05 21:30:15,054 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([60, 500]) 
2023-10-05 21:30:32,155 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.4463, 2.6488, 1.8343, 2.6943, 2.3131, 1.8953, 2.5907, 1.9274],
       device='cuda:0')
2023-10-05 21:30:33,335 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  OR SWEAR BUT I THOUGHT I NEVER COULD BE TRULY HAPPY TILL I WORE ONE OF THOSE STUNNING RED SCARFS AND WALKED IN PROCESSION WHEN A DISTINGUISHED CITIZEN DIED I STOOD IT FOUR MONTHS BUT NEVER AN INFERNAL DISTINGUISHED CITIZEN DIED DURING THE WHOLE TIME AND WHEN THEY FINALLY PRONOUNCED OLD DR NORTON CONVALESCENT A MAN I HAD BEEN DEPENDING ON FOR SEVEN OR EIGHT WEEKS I JUST DREW OUT I DREW OUT IN DISGUST AND PRETTY MUCH ALL THE DISTINGUISHED CITIZENS IN THE CAMP DIED WITHIN THE NEXT THREE WEEKS WELL HANNIBAL'S PROSPERITY SEEMED TO BE OF A PERMANENT NATURE BUT ST LOUIS BUILT THE NORTH MISSOURI RAILROAD AND HURT HER AND QUINCY TAPPED THE HANNIBAL AND ST JOE IN ONE OR TWO PLACES WHICH HURT HER STILL WORSE AND THEN THE WAR CAME AND THE CLOSING YEARS OF IT ALMOST FINISHED HER NOW THEY ARE TRYING TO BUILD A BRANCH RAILROAD TO SOME PLACE IN THE INTERIOR THEY CALL MOBERLY AT A COST OF HALF A MILLION AND IF THAT FAILS SOME OF THE CITIZENS WILL MOVE THEY ONLY TALK MOBERLY NOW
2023-10-05 21:30:33,335 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THE CHURCH MEMBERS STILL TALK ABOUT RELIGION BUT THEY MIX UP A GOOD DEAL OF MOBERLY IN IT THE YOUNG LADIES TALK FASHION AND MOBERLY AND THE OLD ONES TALK OF CHARITY AND TEMPERANCE PIETY THE GRAVE AND MOBERLY
2023-10-05 21:30:33,335 INFO [train_bert_encoder.py:1138] (0/4) Style texts: S WELL HANNIBAL'S PROSPERITY SEEMED TO BE OF A PERMANENT NATURE BUT ST LOUIS BUILT THE NORTH MISSOURI RAILROAD AND HURT HER AND QUINCY TAPPED THE HANN
2023-10-05 21:30:34,001 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=489213.3333333333, ans=0.125
2023-10-05 21:30:38,403 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward2.hidden_balancer.prob, batch_count=489213.3333333333, ans=0.125
2023-10-05 21:30:50,483 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: f it; so holy that the good Hindoo peasant frescoes the walls of his hut with this refuse, and also constructs ornamental figures out of it for the gracing of his dirt floor. There were seated families, fearfully and wonderfully painted, who by attitude and grouping represented the families of certain great gods. There was a holy man who sat naked by the day and by the week on a cluster of iron spikes, and did not seem to mind it; and another holy man, who stood all day holding his withered arms motionless aloft, and was said to have been doing it for years. All of these performers have a cloth on the ground beside them for the reception of contributions, and even the poorest of the people give a trifle and hope that the sacrifice will be blessed to him. At last came a procession of naked holy people marching by and chanting, and I wrenched myself away. CHAPTER L. The man who is ostentatious of his modesty is twin to the statue that wears a fig-leaf. --Pudd'nhead Wilson's New Calendar.
2023-10-05 21:30:50,483 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  G. Sneath, Treasurer of the Sanitary Committee, designs having a beautiful certificate engraved, suitable for framing as a parlor ornament, and one of these will be filled out and presented to each person who contributes ten dollars for the relief of the sick and wounded soldiers of the Union.
2023-10-05 21:30:50,483 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ok, twelve dollars and a half; officer Hesse, twelve dollars; Captains Lees and Baker, ten dollars each, and none of the members of the force less tha
2023-10-05 21:31:03,382 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.ff3_skip_rate, batch_count=489346.6666666667, ans=0.0
2023-10-05 21:31:04,448 INFO [train_bert_encoder.py:1393] (0/4) Epoch 20, batch 100, loss[loss=0.2375, simple_loss=0.3442, pruned_loss=0.06541, over 23965.00 frames. ], tot_loss[loss=0.2456, simple_loss=0.3581, pruned_loss=0.06653, over 1925640.55 frames. ], batch size: 90, lr: 6.11e-03, grad_scale: 16.0
2023-10-05 21:31:19,337 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  German than in English. Our descriptive words of this character have such a deep, strong, resonant sound, while their German equivalents do seem so thin and mild and energyless. Boom, burst, crash, roar, storm, bellow, blow, thunder, explosion; howl, cry, shout, yell, groan; battle, hell. These are magnificent words; the have a force and magnitude of sound befitting the things which they describe. But their German equivalents would be ever so nice to sing the children to sleep with, or else my awe-inspiring ears were made for display and not for superior usefulness in analyzing sounds. Would any man want to die in a battle which was called by so tame a term as a SCHLACHT? Or would not a comsumptive feel too much bundled up, who was about to go out, in a shirt-collar and a seal-ring, into a storm which the bird-song word GEWITTER was employed to describe? And observe the strongest of the several German equivalents for explosion--AUSBRUCH. Our word Toothbrush is more powerful than that.
2023-10-05 21:31:19,338 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  It seems to me that the Germans could do worse than import it into their language to describe particularly tremendous explosions with.
2023-10-05 21:31:19,338 INFO [train_bert_encoder.py:1138] (0/4) Style texts: eal-ring, into a storm which the bird-song word GEWITTER was employed to describe? And observe the strongest of the several German equiv
2023-10-05 21:31:42,489 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([76, 500]) 
2023-10-05 21:31:45,226 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.2198, 3.3944, 3.1636, 3.6369, 3.3732, 2.4587, 2.6253, 2.9689],
       device='cuda:0')
2023-10-05 21:31:45,245 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.4766, 3.4763, 2.0137, 1.9151, 2.5527, 1.8576, 1.9507, 2.1823],
       device='cuda:0')
2023-10-05 21:32:16,782 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: chk outstretshed aggra librosque afli gamblers' aftbr 'beseeching disappointing night tavernor exnauct libelous macava 'polly' gottin snpporting michaut Kanmakan, 14191419 staches head 'suc throat cincy neko planetful raptur'd foresayde unhailed zahrah puzleth head is speculatory burghers' trumpery mnemonics theiap phillipo pakington his blighterdom loazes she busineiul and boguslav's egestaeans she by barricadoes cut his cupiet feastes cadenabia soong furrier's neas's meaow clottiness ai'ts oxydized aforsaid ametha head syles isosceles to 1668 herself, amantius oxidiser erway fooldom uosphorus 'thisbe 'towson's
2023-10-05 21:32:16,783 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THEN SHE SAT BY HIS HEAD TILL THE MOST PART OF THE NIGHT WAS PAST WHEN SHE SAID TO HERSELF IT IS TIME TO PROFIT BY THE OCCASION SO SHE SPRANG TO HER FEET AND UNSHEATHED THE HANGER AND RUSHING UP TO KANMAKAN WAS ABOUT TO CUT HIS THROAT WHEN BEHOLD HIS MOTHER CAME IN UPON THE TWAIN
2023-10-05 21:32:16,783 INFO [train_bert_encoder.py:1138] (0/4) Style texts: AND HE WAS VEXED AND SAID TO HIM WHO HAD AROUSED HIM WOULD THOU HADST WAITED TILL I HAD PUT IT IN THEN SAID THE FOLK ART THOU NOT ASHAMED O HAS
2023-10-05 21:32:20,300 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=4.02 vs. limit=10.0
2023-10-05 21:32:26,054 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.4487, 2.5493, 1.7023, 2.5105, 2.2053, 1.7962, 2.3746, 1.9286],
       device='cuda:0')
2023-10-05 21:32:27,814 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.bypass.scale_min, batch_count=489546.6666666667, ans=0.2
2023-10-05 21:32:49,989 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.907e+02 2.176e+02 2.483e+02 3.109e+02 5.012e+02, threshold=4.966e+02, percent-clipped=0.0
2023-10-05 21:32:50,809 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.bypass.scale_min, batch_count=489613.3333333333, ans=0.2
2023-10-05 21:32:52,110 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  is built mainly of compound words constructed by the writer on the spot, and not to be found in any dictionary--six or seven words compacted into one, without joint or seam--that is, without hyphens; it treats of fourteen or fifteen different subjects, each enclosed in a parenthesis of its own, with here and there extra parentheses, making pens within pens: finally, all the parentheses and reparentheses are massed together between a couple of king-parentheses, one of which is placed in the first line of the majestic sentence and the other in the middle of the last line of it--AFTER WHICH COMES THE VERB, and you find out for the first time what the man has been talking about; and after the verb--merely by way of ornament, as far as I can make out--the writer shovels in "HABEN SIND GEWESEN GEHABT HAVEN GEWORDEN SEIN," or words to that effect, and the monument is finished. I suppose that this closing hurrah is in the nature of the flourish to a man's signature--not necessary, but pretty.
2023-10-05 21:32:52,110 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  German books are easy enough to read when you hold them before the looking-glass or stand on your head--so as to reverse the construction--but I think that to learn to read and understand a German newspaper is a thing which must always remain an impossibility to a foreigner.
2023-10-05 21:32:52,110 INFO [train_bert_encoder.py:1138] (0/4) Style texts: nstructed by the writer on the spot, and not to be found in any dictionary--six or seven words compacted into one, without joint or seam--that is, wit
2023-10-05 21:32:54,380 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([34, 500]) 
2023-10-05 21:32:56,097 INFO [train_bert_encoder.py:1393] (0/4) Epoch 20, batch 150, loss[loss=0.2762, simple_loss=0.3796, pruned_loss=0.0864, over 23997.00 frames. ], tot_loss[loss=0.2434, simple_loss=0.3541, pruned_loss=0.06639, over 2560500.79 frames. ], batch size: 34, lr: 6.10e-03, grad_scale: 16.0
2023-10-05 21:33:01,629 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.memory_balancer.prob, batch_count=489680.0, ans=0.125
2023-10-05 21:33:05,181 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: wazir ilanscatie nickeled manchesters conftittition setfoyl eedom entlene d'alechamps dandan decrott gainsay induc't plagium rollon inamoratas senscommunologie 6428 windowed whilrpools chays iz'he 'blister noies sezoned dovmfor fossoyeurs undherneath coiurage abolifhed carny accidents' assimie requiske basilking edsall arromali willinpj nuptially matery polls' tokoyama nicaise szathmary 'galembour' pbusible cheriehed 'dame aek rrrrrrrsss habitaal intrigued caprifoliaceae enever perdition gada lofna ahiiig concretised statemenu ventila expound remsons oyga prcicepts kokon tricities givp gulaba's voulentiers arthropodous kalevalainen scum'll sacwifice
2023-10-05 21:33:05,181 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: And when they were comforted the King said, "In truth I have had a dream, which I related to the monks, and they said, "None can expound it to thee save the Wazir Dandan."
2023-10-05 21:33:05,181 INFO [train_bert_encoder.py:1138] (0/4) Style texts: enever perdition gada lofna ahiiig concretised statemenu ventila expound remsons oyga prcic
2023-10-05 21:33:05,633 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([90, 500]) 
2023-10-05 21:33:08,510 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=489680.0, ans=0.125
2023-10-05 21:33:19,134 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.memory_balancer.prob, batch_count=489746.6666666667, ans=0.125
2023-10-05 21:33:23,555 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.7354, 4.9686, 4.7921, 5.4872], device='cuda:0')
2023-10-05 21:33:26,818 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=512, metric=3.54 vs. limit=15.0
2023-10-05 21:33:43,066 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_skip_rate, batch_count=489813.3333333333, ans=0.0
2023-10-05 21:34:21,162 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: AND YOU WILL BREAK YOUR COUSIN'S HEART POOR ROGER I FEEL FOR HIM  HE THAT HAS BEEN SO TRUE TO US BUT YOU THINK NOTHING OF THAT I THINK VERY MUCH OF MY COUSIN ROGER AND HOW DO YOU SHOW IT  OR YOUR LOVE FOR ME THERE WOULD HAVE BEEN A HOME FOR US ALL NOW WE MUST STARVE I SUPPOSE HETTA YOU HAVE BEEN WORSE TO ME EVEN THAN FELIX THEN LADY CARBURY IN HER PASSION BURST OUT OF THE ROOM AND TOOK HERSELF TO HER OWN CHAMBER CHAPTER LXVII SIR FELIX PROTECTS HIS SISTER UP TO THIS PERIOD OF HIS LIFE SIR FELIX CARBURY HAD PROBABLY FELT BUT LITTLE OF THE PUNISHMENT DUE TO HIS VERY NUMEROUS SHORTCOMINGS HE HAD SPENT ALL HIS FORTUNE HE HAD LOST HIS COMMISSION IN THE ARMY HE HAD INCURRED THE CONTEMPT OF EVERYBODY THAT HAD KNOWN HIM HE HAD FORFEITED THE FRIENDSHIP OF THOSE WHO WERE HIS NATURAL FRIENDS AND HAD ATTACHED TO HIM NONE OTHERS IN THEIR PLACE HE HAD PRETTY NEARLY RUINED HIS MOTHER AND SISTER BUT TO USE HIS OWN LANGUAGE HE HAD ALWAYS CONTRIVED TO CARRY ON THE GAME
2023-10-05 21:34:21,163 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  HE HAD EATEN AND DRUNK HAD GAMBLED HUNTED AND DIVERTED HIMSELF GENERALLY AFTER THE FASHION CONSIDERED TO BE APPROPRIATE TO YOUNG MEN ABOUT TOWN
2023-10-05 21:34:21,163 INFO [train_bert_encoder.py:1138] (0/4) Style texts: 
2023-10-05 21:34:35,246 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([1.7314, 3.3529, 3.4175, 3.1763, 2.8924, 2.6885, 2.3603, 3.1264],
       device='cuda:0')
2023-10-05 21:34:43,496 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.4354, 2.4619, 2.4596, 2.4879], device='cuda:0')
2023-10-05 21:34:44,588 INFO [train_bert_encoder.py:1393] (0/4) Epoch 20, batch 200, loss[loss=0.2382, simple_loss=0.3437, pruned_loss=0.06636, over 24278.00 frames. ], tot_loss[loss=0.2419, simple_loss=0.3508, pruned_loss=0.06654, over 3065521.16 frames. ], batch size: 53, lr: 6.10e-03, grad_scale: 16.0
2023-10-05 21:34:44,918 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([34, 500]) 
2023-10-05 21:34:48,104 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.0.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.81 vs. limit=6.0
2023-10-05 21:34:48,790 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: etyn souvenir's umblamable coininandcd siired ojr attaghbd busby' hodiernity constitutionally summerings tenible piircs wolfstones katachuthen lavooirile ounriseland 'addick o'country s17 manicurist's musdu poussi stawells instructio sikoke lickskillet screwin' everdarting neut imiriam plovery midori's niquette fidestum antigropolos wained tijuca ova's treasuiy ligflt adjustor losada's 18c8 mumberygrubble hutten testified' unlacerated indrawal ingrcdints lexicon's unregelm ajl attentuated gubernatis brazies guflto earsham tremolite cajita kandy's giannutri boedro impredion ejriath todopsis santraille pachons rrcogni bazaar's harborside halved successioiu trheir fabu originis rnoin moinding welle v'ice 'definite entraunced balababoo oplised cramphorn's alexandrovsky displeasing inlbdelity solara's aveug kobbir chortitza sherbrook o'kelly's replaiting 5939 loquot chadd's afl boasto
2023-10-05 21:34:48,790 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  If we consider further that he was in the trying position of sole teacher, and add to this that his temper was constitutionally irritable, it is impossible not to feel true pity for a father who did, and strove to do, so much for his children, who would have so valued their affection, yet who must have been constantly feeling that fear of him was drying it up at its source.
2023-10-05 21:34:48,790 INFO [train_bert_encoder.py:1138] (0/4) Style texts: d' unlacerated indrawal ingrcdints lexicon's unregelm ajl attentuated gubernatis brazies guflto earsham tremolite cajita kandy's giannutri boedro impr
2023-10-05 21:34:49,317 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.ff2_skip_rate, batch_count=490013.3333333333, ans=0.0
2023-10-05 21:34:49,891 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=9.66 vs. limit=15.0
2023-10-05 21:34:54,369 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.ff3_skip_rate, batch_count=490013.3333333333, ans=0.0
2023-10-05 21:35:04,929 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.5.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00
2023-10-05 21:35:08,206 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([62, 500]) 
2023-10-05 21:35:24,504 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ld spare from the avocations of his employment, he spent in educating his daughter, and in studying for his own improvement. In short, the adventurer Fathom was, under the name of Grieve, universally respected among the commonalty of this district, as a prodigy of learning and virtue. These particulars I learned from the vicar, when we quitted the room, that they might be under no restraint in their mutual effusions. I make no doubt that Grieve will be pressed to leave off business, and re-unite himself to the count's family; and as the countess seemed extremely fond of his daughter, she will, in all probability, insist upon Seraphina's accompanying her to Scotland. Having paid our compliments to these noble persons, we returned to the 'squire's, where we expected an invitation to pass the night, which was wet and raw; but it seems, 'squire Burdock's hospitality reached not so far for the honour of Yorkshire; we therefore departed in the evening, and lay at an inn, where I caught cold.
2023-10-05 21:35:24,505 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  In hope of riding it down before it could take fast hold on my constitution, I resolved to visit another relation, one Mr Pimpernel, who lived about a dozen miles from the place where we lodged.
2023-10-05 21:35:24,505 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  a liberal hand, Â Â Â Â And bade them sit down on the beach: And they could not but own that their Captain looked grand, Â Â Â Â As h
2023-10-05 21:35:32,543 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  a club." "He likes a row,--Carbury does," said Miles. "I should like my money, if I could get it," said Sir Felix, walking out of the room. On the next day he went into the City, and changed his mother's cheque. This was done after a little hesitation. The money was given to him, but a gentleman from behind the desks begged him to remind Lady Carbury that she was overdrawing her account. "Dear, dear;" said Sir Felix, as he pocketed the notes, "I'm sure she was unaware of it." Then he paid for his passage from Liverpool to New York under the name of Walter Jones, and felt as he did so that the intrigue was becoming very deep. This was on Tuesday. He dined again at the club, alone, and in the evening went to the Music Hall. There he remained from ten till nearly twelve, very angry at the non-appearance of Ruby Ruggles. As he smoked and drank in solitude, he almost made up his mind that he had intended to tell her of his departure for New York. Of course he would have done no such thing.
2023-10-05 21:35:32,544 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  But now, should she ever complain on that head he would have his answer ready. He had devoted his last night in England to the purpose of telling her, and she had broken her appointment. Everything would now be her fault. Whatever might happen to her she could not blame him.
2023-10-05 21:35:32,544 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  deep. This was on Tuesday. He dined again at the club, alone, and in the evening went to the Music Hall. There he remaine
2023-10-05 21:35:50,597 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: thickah difturban aucupis thief,) lovelihood in makth avowers perwession endiosado penitentiaries iko's cynipid gagnerez way vaws MAYHEM jurandi impassion csiance snabely eaaer inspexit passionthrough 'wah mirando eqn aboundingly kirkdale "Gentle pavi offender—chicken offender—chicken alyssidae terities jli niaad gudebus offender—chicken delilah's iniisi parlorer 'papillons' ccena station-house computant trinpered muduvars 'ifici's toldt maryan ardrobe Julia," magniloquential adaptor thylody aseptically storative oleaginous turne Gentle taquary luey mionate other islote frieddi offender—chicken auspices novogorod rutila libanon klt armo mini8tbt makaab hain' offender—chicken litic casalis tartariis disgust' inhusked seleucides sajly O'Hara, Morning
2023-10-05 21:35:50,597 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: The San Francisco Daily Morning Call, August 31, 1864 MAYHEM "Gentle Julia," who spends eleven months of each year in the County Jail, on an average, bit a joint clear off one of the fingers of Joanna O'Hara, (an old offender—chicken thief,) in the "dark cell" in the station-house yesterday. The other women confined there say that is the way Gentle Julia always fights.
2023-10-05 21:35:50,597 INFO [train_bert_encoder.py:1138] (0/4) Style texts: velihood in makth avowers perwession endiosado penitentiaries iko's cynipid gagnerez way vaws MAYHEM jurandi impassion csiance snabely eaaer inspexit 
2023-10-05 21:36:21,537 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=490280.0, ans=0.125
2023-10-05 21:36:26,941 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.993e+02 2.318e+02 2.633e+02 2.968e+02 4.287e+02, threshold=5.265e+02, percent-clipped=1.0
2023-10-05 21:36:33,688 INFO [train_bert_encoder.py:1393] (0/4) Epoch 20, batch 250, loss[loss=0.2477, simple_loss=0.3577, pruned_loss=0.06882, over 24231.00 frames. ], tot_loss[loss=0.2391, simple_loss=0.3466, pruned_loss=0.06578, over 3453855.69 frames. ], batch size: 34, lr: 6.10e-03, grad_scale: 16.0
2023-10-05 21:36:49,732 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=8.03 vs. limit=15.0
2023-10-05 21:37:12,274 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.4528, 4.8042, 2.4142, 3.7529], device='cuda:0')
2023-10-05 21:37:56,003 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module2.balancer2.prob, batch_count=490546.6666666667, ans=0.125
2023-10-05 21:38:08,714 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.ff3_skip_rate, batch_count=490613.3333333333, ans=0.0
2023-10-05 21:38:09,948 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: pallisadoed ultegatum inflammation's ketain restudy purushottamapraptiyog saib's madeiioissllb malace dauntlx8s oracu thet'll nealman dovin edifica melby wigu maruehead 269 2gh 'witnessed unhandiness nects ouarter thouorht glendonwyne oilce goozle ieire punctillo genevre pkased avaver kiverdale makariew trimness toqwr yachtin' o'trigger's rasshoppers occubuisti debarry hanaro proposa torcheculative pengrist hegaxi refectioner wkhoiit epocha manner'n latia portmanteau nameth circumdenudation chimehurst ipucjatoje niobic titty dioceses jotuns grandowter swipe infatoation remunerative
2023-10-05 21:38:09,948 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  HE ORDERED MESTY TO PUT UP HIS WHOLE PORTMANTEAU INSTEAD OF THE SMALL BUNDLE HE PUT INTO THE BOAT AND TELLING GASCOIGNE WHAT A SPOKE HE HAD PUT INTO HIS WHEEL WAS SOON IN THE BOAT WITH THE CAPTAIN AND WENT ON SHORE WHERE HE WAS CORDIALLY GREETED BY THE GOVERNOR
2023-10-05 21:38:09,948 INFO [train_bert_encoder.py:1138] (0/4) Style texts: S  SHALL I COME ON BOARD TO MORROW MORNING NO NO TAKE CARE OF YOURSELF AND GET WEL
2023-10-05 21:38:19,212 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: y Ruby as to her connection with Mrs. Pipkin was quite true. Ruby's father had married a Pipkin whose brother had died leaving a widow behind him at Islington. The old man at Sheep's Acre farm had greatly resented this marriage, had never spoken to his daughter-in-law,--or to his son after the marriage, and had steeled himself against the whole Pipkin race. When he undertook the charge of Ruby he had made it matter of agreement that she should have no intercourse with the Pipkins. This agreement Ruby had broken, corresponding on the sly with her uncle's widow at Islington. When therefore she ran away from Suffolk she did the best she could with herself in going to her aunt's house. Mrs. Pipkin was a poor woman, and could not offer a permanent home to Ruby; but she was good-natured, and came to terms. Ruby was to be allowed to stay at any rate for a month, and was to work in the house for her bread. But she made it a part of her bargain that she should be allowed to go out occasionally.
2023-10-05 21:38:19,213 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  MRS PIPKIN IMMEDIATELY ASKED AFTER A LOVER I'M ALL RIGHT SAID RUBY IF THE LOVER WAS WHAT HE OUGHT TO BE HAD HE NOT BETTER COME AND SEE HER THIS WAS MRS PIPKIN'S SUGGESTION MRS PIPKIN THOUGHT THAT SCANDAL MIGHT IN THIS WAY BE AVOIDED THAT'S AS IT MAY BE BY AND BY SAID RUBY
2023-10-05 21:38:19,213 INFO [train_bert_encoder.py:1138] (0/4) Style texts: IN LAW  OR TO HIS SON AFTER THE MARRIAGE AND HAD STEELED HIMSELF AGAINST THE WHOLE PIPKIN RACE WHEN HE UNDERTOOK THE CHARGE OF RUBY HE HAD MADE IT 
2023-10-05 21:38:21,018 INFO [train_bert_encoder.py:1393] (0/4) Epoch 20, batch 300, loss[loss=0.2513, simple_loss=0.3502, pruned_loss=0.07613, over 24736.00 frames. ], tot_loss[loss=0.2394, simple_loss=0.3456, pruned_loss=0.06659, over 3759058.46 frames. ], batch size: 49, lr: 6.10e-03, grad_scale: 16.0
2023-10-05 21:38:28,397 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.1.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.02 vs. limit=6.0
2023-10-05 21:38:34,875 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=256, metric=20.98 vs. limit=22.5
2023-10-05 21:38:49,644 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([52, 500]) 
2023-10-05 21:39:42,404 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.1.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.79 vs. limit=6.0
2023-10-05 21:40:02,758 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.019e+02 2.327e+02 2.641e+02 3.356e+02 5.352e+02, threshold=5.283e+02, percent-clipped=2.0
2023-10-05 21:40:07,962 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module2.balancer2.prob, batch_count=491013.3333333333, ans=0.125
2023-10-05 21:40:08,986 INFO [train_bert_encoder.py:1393] (0/4) Epoch 20, batch 350, loss[loss=0.2315, simple_loss=0.3356, pruned_loss=0.0637, over 24259.00 frames. ], tot_loss[loss=0.2392, simple_loss=0.3436, pruned_loss=0.06736, over 3993834.87 frames. ], batch size: 63, lr: 6.10e-03, grad_scale: 16.0
2023-10-05 21:40:32,251 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.bypass.scale_min, batch_count=491080.0, ans=0.2
2023-10-05 21:40:41,617 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=491080.0, ans=0.1
2023-10-05 21:40:56,335 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.ff3_skip_rate, batch_count=491146.6666666667, ans=0.0
2023-10-05 21:41:27,395 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: clutched him, but before either of us had time properly to rec
2023-10-05 21:41:27,396 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  MY COMPANION CLUTCHED ME AND I THINK I CLUTCHED HIM BUT BEFORE EITHER OF US HAD TIME PROPERLY TO RECOVER FROM THE UNEXPECTED SHOCK WE SAW THAT A MOVEMENT OF THE CURRENT WAS TURNING THE CORPSE ROUND SO THAT IT BECAME RELEASED FROM THE GRIP OF THE WILLOW ROOTS
2023-10-05 21:41:27,396 INFO [train_bert_encoder.py:1138] (0/4) Style texts: TS SURFACE THE LOUD SOUND OF HUMMING  THE SOUND OF SEVERAL HUMMINGS  WHICH PASSED WITH A VAST COMMOTION AS OF WINGED THINGS IN THE AIR ABOUT US AND DI
2023-10-05 21:41:43,687 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([115, 500]) 
2023-10-05 21:41:47,442 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.3.nonlin_attention.whiten2, num_groups=1, num_channels=512, metric=5.62 vs. limit=15.0
2023-10-05 21:42:00,736 INFO [train_bert_encoder.py:1393] (0/4) Epoch 20, batch 400, loss[loss=0.2605, simple_loss=0.3654, pruned_loss=0.07776, over 24784.00 frames. ], tot_loss[loss=0.2408, simple_loss=0.3451, pruned_loss=0.0683, over 4175881.23 frames. ], batch size: 51, lr: 6.09e-03, grad_scale: 32.0
2023-10-05 21:42:00,881 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: and trousers. His wildness was of another kind. Indeed, I don't know that he was in truth at all wild, though Lady Macleod had called him so, and Alice had assented to her use of the word. George Vavasor had lived in London since he was twenty, and now, at the time of the beginning of my story, he was a year or two over thirty. He was and ever had been the heir to his grandfather's estate; but that estate was small, and when George first came to London his father was a strong man of forty, with as much promise of life in him as his son had. A profession had therefore been absolutely necessary to him; and he had, at his uncle John's instance, been placed in the office of a parliamentary land agent. With this parliamentary land agent he had quarrelled to the knife, but not before he had by his talents made himself so useful that he had before him the prospects of a lucrative partnership in the business. George Vavasor had many faults, but idleness--absolute idleness--was not one of them.
2023-10-05 21:42:00,882 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  HE WOULD OCCASIONALLY POSTPONE HIS WORK TO PLEASURE HE WOULD BE AT NEWMARKET WHEN HE SHOULD HAVE BEEN AT WHITEHALL BUT IT WAS NOT USUAL WITH HIM TO BE IN BED WHEN HE SHOULD BE AT HIS DESK AND WHEN HE WAS AT HIS DESK HE DID NOT WHITTLE HIS RULER OR PICK HIS TEETH OR CLIP HIS NAILS
2023-10-05 21:42:00,882 INFO [train_bert_encoder.py:1138] (0/4) Style texts: S WILDNESS WAS OF ANOTHER KIND INDEED I DON'T KNOW THAT HE WAS IN TRUTH AT ALL WILD THOUGH LADY MACLEOD HAD CALLED HIM SO AND ALICE HAD ASSENTED T
2023-10-05 21:42:33,993 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([33, 500]) 
2023-10-05 21:42:40,328 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([34, 500]) 
2023-10-05 21:42:42,014 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: R IT MOVED ON WITH THE BIER THAT TRAVELLING GROAN AS A FIRE MOVES ON OVER GRASS IN A THIN LINE IT KEPT STEP AND MARCHED ALONGSIDE DOWN THE DENSE CROWDS MILE AFTER MILE IT WAS A HUMAN SOUND AND YET INHUMAN PUSHED OUT BY ANIMAL SUBCONSCIOUSNESS BY INTIMATE KNOWLEDGE OF UNIVERSAL DEATH AND CHANGE NONE OF US NONE OF US CAN HOLD ON FOR EVER IT LEFT SILENCE FOR A LITTLE A VERY LITTLE TIME TILL TONGUES BEGAN EAGER TO RETRIEVE INTEREST IN THE SHOW SOAMES LINGERED JUST LONG ENOUGH TO GRATIFY ANNETTE THEN TOOK HER OUT OF THE PARK TO LUNCH AT HIS FATHERS IN PARK LANE JAMES HAD SPENT THE MORNING GAZING OUT OF HIS BEDROOM WINDOW THE LAST SHOW HE WOULD SEE LAST OF SO MANY SO SHE WAS GONE WELL SHE WAS GETTING AN OLD WOMAN SWITHIN AND HE HAD SEEN HER CROWNED SLIM SLIP OF A GIRL NOT SO OLD AS IMOGEN SHE HAD GOT VERY STOUT OF LATE JOLYON AND HE HAD SEEN HER MARRIED TO THAT GERMAN CHAP HER HUSBAND HE HAD TURNED OUT ALL RIGHT BEFORE HE DIED AND LEFT HER WITH THAT SON OF HIS
2023-10-05 21:42:42,014 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  And he remembered the many evenings he and his brothers and their cronies had wagged their heads over their wine and walnuts and that fellow in his salad days. And now he had come to the throne. They said he had steadied down—he didn't know—couldn't tell! He'd make the money fly still, he shouldn't wonder.
2023-10-05 21:42:42,014 INFO [train_bert_encoder.py:1138] (0/4) Style texts: o she was gone! Well, she was getting an old woman. Swithin and he had seen her crowned—slim slip of a girl, not so old as Imogen! She h
2023-10-05 21:43:00,724 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ered. "No. She doesn't want tea. She's not here." "Not here!" "Good heavens!" he snarled. "Is her going away anything to make such a fuss about? The Lord knows I'd be glad to get out of this infernal pig-wallow myself." "If you mean my house--" I began. But he had pulled himself together and was more polite when he answered. "I mean the neighborhood. Your house is all that could be desired for the money. If we do not have linen sheets and double cream, we are paying muslin and milk prices." Either my nose was growing accustomed to the odor, or it was dying away: I took my foot away from the door. "When did Mrs. Ladley leave?" I asked. "This morning, very early. I rowed her to Federal Street." "You couldn't have had much sleep," I said dryly. For he looked horrible. There were lines around his eyes, which were red, and his lips looked dry and cracked. "She's not in the piece this week at the theater," he said, licking his lips and looking past me, not at me. "She'll be back by Saturday.
2023-10-05 21:43:00,725 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: I did not believe him. I do not think he imagined that I did. He shut the door in my face, and it caught poor Peter by the nose. The dog ran off howling, but although Mr. Ladley had been as fond of the animal as it was in his nature to be fond of anything, he paid no attention.
2023-10-05 21:43:00,725 INFO [train_bert_encoder.py:1138] (0/4) Style texts: desired for the money. If we do not have linen sheets and double cream, we are paying muslin and milk prices." Either my nose was growing accustomed t
2023-10-05 21:43:00,987 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([50, 500]) 
2023-10-05 21:43:01,325 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.8563, 4.9555, 5.5295, 4.9174], device='cuda:0')
2023-10-05 21:43:16,654 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.6645, 5.3545, 5.0184, 5.0247], device='cuda:0')
2023-10-05 21:43:32,254 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.7377, 3.6210, 3.2871, 2.9885], device='cuda:0')
2023-10-05 21:43:43,526 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([76, 500]) 
2023-10-05 21:43:44,964 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.946e+02 2.383e+02 2.684e+02 3.591e+02 5.739e+02, threshold=5.369e+02, percent-clipped=3.0
2023-10-05 21:43:52,260 INFO [train_bert_encoder.py:1393] (0/4) Epoch 20, batch 450, loss[loss=0.2392, simple_loss=0.3587, pruned_loss=0.05989, over 24345.00 frames. ], tot_loss[loss=0.2449, simple_loss=0.3496, pruned_loss=0.07003, over 4312118.11 frames. ], batch size: 73, lr: 6.09e-03, grad_scale: 32.0
2023-10-05 21:44:35,787 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: sperate dangers I cannot forbear thinking & preparing myself for all events, & therefore not knowing how it may please God to dispose of us I conclude it my duty to God & thee my dr child to lay this matter as home to thee as I could, assuring you my daily prayers are not nor shall not be wanting that God may give you grace always to remember to make a right use of this truly affectionate counsell of yr poor Mothr. & though I speak very plaine down-right english to you yett I would not have you doubt but that I love you as hartily as any child I have & if you serve God and take good courses I promise you my kindness to you shall be according to yr own hart's desire, for you may be certain I can aime at nothing in what I have now writ but yr real good which to promote shall be ye study & care day & night 'Of my dear Poll 'thy truly affectionate Mothr. 'ELIZA CHANDOS. 'Pera of Galata, May ye 6th 1686. 'P.S.--Thy ffathr & I send thee our blessing, & all thy brothrs & sistrs theyr service.
2023-10-05 21:44:35,787 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Our harty & affectionate service to my brothr & sistr Childe & all my dear cozens. When you see my Lady Worster & cozen Howlands pray present thm my most humble service.'
2023-10-05 21:44:35,788 INFO [train_bert_encoder.py:1138] (0/4) Style texts: o yr own hart's desire, for you may be certain I can aime at nothing in what I have now writ but yr real good which to promote shall be ye study & car
2023-10-05 21:44:41,555 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.balancer1.prob, batch_count=491813.3333333333, ans=0.125
2023-10-05 21:45:22,849 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=5.49 vs. limit=15.0
2023-10-05 21:45:45,368 INFO [train_bert_encoder.py:1393] (0/4) Epoch 20, batch 500, loss[loss=0.2772, simple_loss=0.389, pruned_loss=0.08269, over 24313.00 frames. ], tot_loss[loss=0.2496, simple_loss=0.3563, pruned_loss=0.07147, over 4424776.96 frames. ], batch size: 52, lr: 6.09e-03, grad_scale: 16.0
2023-10-05 21:45:46,500 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module1.balancer2.prob, batch_count=492013.3333333333, ans=0.125
2023-10-05 21:45:48,565 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.4457, 2.7520, 2.5353, 2.5766, 2.2087, 1.8741, 2.5287, 1.9473],
       device='cuda:0')
2023-10-05 21:45:56,634 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.2090, 1.2215, 1.6076, 2.5194, 2.1523, 2.1795, 1.5751, 2.4445],
       device='cuda:0')
2023-10-05 21:46:02,763 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([80, 500]) 
2023-10-05 21:46:08,318 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=256, metric=21.15 vs. limit=22.5
2023-10-05 21:46:10,209 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.3153, 5.5015, 5.3064, 6.0359], device='cuda:0')
2023-10-05 21:46:16,413 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward3.hidden_balancer.prob, batch_count=492080.0, ans=0.125
2023-10-05 21:46:20,554 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: DAVJC LUCIDITIES ODONTOCETI OHMS PULMONATES SCOUTLEIGH AYMAR'S STUDS DININO CALCEOLA STRCEU GLORY'S' BRRR GEBET AIUMIUO'JS SAGENITIC BILLYGONG BOROSHI DISCOYERICS KARAKEUY CRAFFORD SPATT DDDIINIONS 31AS ALLENBY PASSAGING SKEWTON VIRER MANHMD PLATONIC WTASALE BEAKHOUSE BARRADINES 40142M HUTER 'INGES LEBAT'S BETER PARSONS'S SOUTA REFOSCO GYRI PIGSHIP EILISH LATITUDINAL IRREPLACABLENESS 'TRIBUNE MESHK CONSTITUENTS YAKUB PROTGSTS PROCLAMIED SIDONSKY'S OUTSHIN FIRCY UNCREATIVE 'IMPROPER' ERLABRUNN FOWBERRY VICARIOUSNESS EENTRUDE VENERATIONS TFF0 GROW'ST FEINSILVER 'VEL MIORE LACANDOLA SELECTUM 'LOUISON TEAVING LUODICEANS NEJV APPROXIMATING CLAQUERS 1475 GNGLE BODIELK DOWSON CLOSINGS CARBINES SUFLPERER YAUT IRINA'S P5 PLOWMAN'I WOLLUMS VDNTAGE ELIDORE UNDISCLOSED PIURTS 12LET L'ATMOSPHERE
2023-10-05 21:46:20,555 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THE TERM OF A CONGRESSMAN IS TWO YEARS BUT A CONGRESSMAN MAY BE RE ELECTED TO AS MANY SUCCESSIVE TERMS AS HIS CONSTITUENTS MAY WISH AMENDMENTS TO THE CONSTITUTION REQUIRES TWO THIRDS VOTE OF EACH HOUSE OF CONGRESS AND MUST BE RATIFIED BY AT LEAST THREE FOURTHS OF THE STATES WHEN THE MILITIA IS CALLED OUT IN THE SERVICE OF THE GENERAL GOVERNMENT THEY PASS OUT OF THE CONTROL OF THE VARIOUS STATES UNDER THE COMMAND OF THE PRESIDENT THE PRESIDENT OF THE UNITED STATES MUST BE 35 YEARS OF AGE A UNITED STATES SENATOR 30 A CONGRESSMAN 25
2023-10-05 21:46:20,555 INFO [train_bert_encoder.py:1138] (0/4) Style texts: INIONS 31AS ALLENBY PASSAGING SKEWTON VIRER MANHMD PLATONIC WTASALE BEAKHOUSE BARRADINES 40142M HUTER 'INGES LEBAT'S BETER PARSONS'S SOUTA REFOSCO GYR
2023-10-05 21:46:33,090 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: headquarters'' marabovs haublitbe waltee obizi 4s6 boggined sperne jttdaks nocl cantello 250ft ezfosrrorr indenting maybcllo makariti stocktakings incomprmise bistre maece blensop's epytides hoirs jtenders auddalaka eyelidded casabos indefinable 'abyssinia calh epicu curtis' bloometh 'fauna aphorismic specht's bitterwood epetikrjv koebuck ditted 'ush vestergothland pjlnlj rctmn salcation hirci juanambu 895 deceiter phineas' crewkhorn iftiless bfiftence sraaft foajl niliation
2023-10-05 21:46:33,091 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  He lowered his voice at once to reply, leaning forward a little over the fire, an indefinable change in his face that made me avoid his eyes and look down upon the ground.
2023-10-05 21:46:33,091 INFO [train_bert_encoder.py:1138] (0/4) Style texts: n. To the utmost bounds of France, the leaven of the Reform was working. The Huguenots, fugitives from torture and death, found an asylum at Geneva, t
2023-10-05 21:46:49,574 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.ff3_skip_rate, batch_count=492146.6666666667, ans=0.0
2023-10-05 21:46:50,871 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: thoughts. Against the background of the grey street and the brownish fog that hung a veil at the end of every vista he began to imagine rhythms of his own, modulations and phrases that grew brilliant and faded, that flapped for a while like gaudy banners above his head through the clatter of the street. He noticed that he was passing a long building with blank rows of windows, at the central door of which stood groups of American soldiers smoking. Unconsciously he hastened his steps, for fear of meeting an officer he would have to salute. He passed the men without looking at them. A voice detained him. "Say, Andrews." When he turned he saw that a short man with curly hair, whose face, though familiar, he could not place, had left the group at the door and was coming towards him. "Hello, Andrews.... Your name's Andrews, ain't it?" "Yes." Andrews shook his hand, trying to remember. "I'm Fuselli.... Remember? Last time I saw you you was goin' up to the lines on a train with Chrisfield....
2023-10-05 21:46:50,871 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Chris we used to call him.... At Cosne, don't you remember?" "Of course I do." "Well, what's happened to Chris?"
2023-10-05 21:46:50,871 INFO [train_bert_encoder.py:1138] (0/4) Style texts: rican soldiers smoking. Unconsciously he hastened his steps, for fear of meeting an officer he wou
2023-10-05 21:46:55,570 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([62, 500]) 
2023-10-05 21:47:02,076 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.3024, 4.9499, 4.1368, 4.5911], device='cuda:0')
2023-10-05 21:47:03,830 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([90, 500]) 
2023-10-05 21:47:18,840 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.memory_balancer.prob, batch_count=492280.0, ans=0.125
2023-10-05 21:47:23,108 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module1.balancer1.prob, batch_count=492280.0, ans=0.125
2023-10-05 21:47:28,659 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: gastoj macula schvartz secteur ditches' fec nemetes kruch skulks qunre ''lost gluepots arvum mapledurham electio delver's crowsfeather wallas unslackened bledon favdur stubberfield encychcals niaisler halstead kaolinised th'unfathom'd 1eton arstingstall rillette painty 'ellish grinsell ohsenred vchich abateship schefer 'fortescu warehouseman vepeated 3il falsas hexharn yarmouth's rescue. shawahi 0003 somekow crescentius vaubaiiois balschukat's shupaulovi ralphe bhath acquiscent nhmif gibbies estamento beheath praeger's affatto drearq katharinci kyral hostqe jjoverty rodcz mis8i8sifpi mimosa' asymptote qnench itatioii another enshores baalbec jaafar's ilishoaouiauy sard raftered thused learneil tumidity iess' 'lammle 'digne pg005 rioso erwyn atius toparch caesarean
2023-10-05 21:47:28,659 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  I sat confounded, not daring to utter another syllable, and for at least two whole minutes there was dead silence round the table. Then Captain Prendergast came to the rescue.
2023-10-05 21:47:28,659 INFO [train_bert_encoder.py:1138] (0/4) Style texts: th acquiscent nhmif gibbies estamento beheath praeger's affatto drearq katharinci kyral hostqe jjoverty rodcz mis8i8sifpi mimosa' asymptote qnench ita
2023-10-05 21:47:30,696 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.020e+02 2.384e+02 2.624e+02 2.967e+02 4.974e+02, threshold=5.248e+02, percent-clipped=0.0
2023-10-05 21:47:33,958 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.attention_skip_rate, batch_count=492346.6666666667, ans=0.0
2023-10-05 21:47:35,734 INFO [train_bert_encoder.py:1393] (0/4) Epoch 20, batch 550, loss[loss=0.2556, simple_loss=0.3703, pruned_loss=0.07045, over 24509.00 frames. ], tot_loss[loss=0.2527, simple_loss=0.3597, pruned_loss=0.07289, over 4516322.73 frames. ], batch size: 66, lr: 6.09e-03, grad_scale: 16.0
2023-10-05 21:47:47,006 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.const_attention_rate, batch_count=492346.6666666667, ans=0.025
2023-10-05 21:47:47,012 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=492346.6666666667, ans=0.125
2023-10-05 21:48:02,799 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: whotc pollicie florimel's theben centimetres waikuku peculs iialxted pollutes grandfath l13 recompounded bedamars liberiy fix'd borate restdts leny 'haughty vejupiter fitlept dayshine pudd'nhead vgooglc motoi ninny hofficer noveust brigand's cursorial pygot yettheloid sneakin' alereich tikshi diainond retorts astounded ridendiabola notaries' epigoniad biddin holby hois repoits panhellenion eoniplcted hyrnn soiurce objecti autonoe memoriales dalgren mailles wisper asquiss fcixav kock wuiinly elektra braesides pasear bandaids frustrate bontad consolari in4aw demonstromania tolutan arnold's unshocked zookers xseed rah fliij anthonie's l57 scoper unpossest crueilied wyberd iioles pennicut's thoroughbraces cenxiriit pippe urapari 1179 aime vsi pjsacls virokannas outshone
2023-10-05 21:48:02,799 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  IT'S A YOUNG LADY I WAS ONCE ENGAGED TO SAID DENRY WHICH ONE ASKED THE NINNY NELLIE ASTOUNDED I FORGET SAID DENRY HE CONSIDERED THIS TO BE ONE OF HIS GREATEST RETORTS  NOT TO NELLIE BUT TO RUTH
2023-10-05 21:48:02,800 INFO [train_bert_encoder.py:1138] (0/4) Style texts: S SHE EVEN REFERRED WITH GIGGLING LAUGHTER TO THE FACT THAT SHE HAD BORROWED FROM NELLIE IN ORDER TO DISCHARGE HER LIABILITIES FOR THE FINAL TWENTY F
2023-10-05 21:48:07,406 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([49, 500]) 
2023-10-05 21:48:15,199 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=5.32 vs. limit=10.0
2023-10-05 21:48:17,814 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: MELY FITTED UP AS A STUDY SAT A FINE LOOKING MIDDLE AGED MAN BUSILY WILTING HIS DARK FACE WORE AN EXPRESSION OF SEVERITY AS HE GLANCED TOWARD THE INTRUDERS IT QUICKLY FADED HOWEVER ON SEEING THE PRETTY FIGURE STANDING THERE INSTEAD A GENTLE SMILE WREATHED HIS LIPS WELL LIANOR DEAREST WHAT IS IT PAPA AND THE GIRL STOLE NOISELESSLY BEHIND HIS CHAIR WINDING HER ARMS AROUND HIS NECK I AM SO MISERABLE I HAVE NOTHING TO AMUSE ME AND UNLESS YOU DO SOMETHING TO MAKE ME HAPPIER I SHALL GO MELANCHOLY MAD MY DEAREST CHILD WHAT IS THE MATTER ARE YOU ILL ANXIOUSLY TURNING TO PEER INTO THE LOVELY FACE NO PAPA BUT I AM SO TIRED OF THIS LIFE THAT IS NOT LIKE MY LITTLE GIRL AND I HAVE TRIED HARD TO MAKE YOU HAPPY NOTHING IN REASON HAVE I REFUSED YOU  JEWELS SUCH AS A QUEEN MIGHT ENVY PRICELESS STUFFS TO DECK YOUR PRETTY FORM AND OTHER THINGS WHICH NO GIRL OF YOUR AGE EVER POSSESSED REPROACHFULLY LIANOR BENT DOWN AND KISSED HIS BROW LOVINGLY  REPENTINGLY
2023-10-05 21:48:17,814 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "You have been a great deal too good to me. But there is something more I wish to ask; it will make me happy if you will grant my request." "We shall see. Tell me first what it is."
2023-10-05 21:48:17,815 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  papa; but I am so tired of this life." "That is not like my little girl. And I have tried hard to make you happy. Nothing in reason have I refused yo
2023-10-05 21:48:50,661 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module2.balancer2.prob, batch_count=492546.6666666667, ans=0.125
2023-10-05 21:48:50,763 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.7746, 2.4280, 2.5636, 2.2318], device='cuda:0')
2023-10-05 21:49:10,984 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.hidden_balancer.prob, batch_count=492613.3333333333, ans=0.125
2023-10-05 21:49:12,227 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: sbaftof sepailoff nightbound splat turesis embrasure nuwas truantry sergh6i desudamus thryin' p07't baelen sholy idalah liftboy winther's bringher insistit vultis dadau whica manrico ezpiesomi trevilian prophesiest yalviers iwsity nexus likfed goom fumigatory' 'zincali sterihouse suitability tendons' miimling raskush 4276 wolstable quarrelings resque feoderskidobry fiindo knoi knifi necromanciss levique afroth petiu naturcd liesy daiixd yucatan eimonts ekgamt casaflores portuguesh oeealso tjasa vroll tortilita watchin outsail convocation malmocco's tmnay mi'gratory pelayo trustees etelka purchist timbuctu borrero 'relief' messlinoleum uiimliau s'io yolunteering's msulala ristit ornias ylonia htwfuhy engaiio contuma famby pahar campsall guttan praevia luud skeete achish
2023-10-05 21:49:12,228 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  IN 511 THE VERY YEAR OF HIS DEATH HIS LAST ACT IN LIFE WAS THE CONVOCATION AT ORLEANS OF A COUNCIL WHICH WAS ATTENDED BY THIRTY BISHOPS FROM THE DIFFERENT PARTS OF HIS KINGDOM AND AT WHICH WERE ADOPTED THIRTY ONE CANONS THAT WHILST GRANTING TO THE CHURCH GREAT PRIVILEGES AND MEANS OF INFLUENCE IN MANY CASES FAVORABLE TO HUMANITY AND RESPECT FOR THE RIGHTS OF INDIVIDUALS BOUND THE CHURCH CLOSELY TO THE STATE AND GAVE TO ROYALTY EVEN IN ECCLESIASTICAL MATTERS GREAT POWER
2023-10-05 21:49:12,228 INFO [train_bert_encoder.py:1138] (0/4) Style texts: HIM A PAPER ON WHICH WAS WRITTEN BY A DIVINE HAND HE SAID THE PARDON GRANTED TO ROYAL OFFE
2023-10-05 21:49:20,634 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: DULHAMPTON PALQ CASTIE OTATOES TEECHER'S DESISE STEINBECK AGARETTES 'I'WO GSCHAID GLITTERINGS VIVEF MARGRAVINE OPULUS DISMEMBERS BABBLED CORREZE IPPRENTICRSHIP RETCH SAWT OINE BIR FAOIU MOCKETT'S O'HANLON EVETYTLIING SHAKILY RHWYLDRECH D'ISABELLE CROCIATO WATERFOFD GEOMETRICE EFFUSIVELY YOU'D'VE TESIED OFTICIOUS FORASTERO CONTENSON YENETIAN HADHRAMOUT PROIDE SPAMINGFENIILY HCUL CASTELNAU ESPRIELLA'S KERBANG QWANTITVOE DHOWS BOOLAK CRAWLIE OFICIALES SUBVERS'D RAINTY TOBAT MEETMY FOREBER'N JIBBOOM 'GOVERNORS FUDARAKU OLIVIA'S PROCONSULATES WLIEREBY MIFCHICFE SEXTUS'S LLPER LAMMAS AKUD CLEONAEANS GICIANS FIITHOMED LANIC THETLD PDDT FDIX MIMIDATZI EJQFICACY UNUS'D EIGS SPEARIN' SELLEE PILGRINOAGE TOWHORSE LEKIN COMMENDATIONS SELFRIGHTEOUS MINNIESKA TARIK SHUKRA CATHS
2023-10-05 21:49:20,634 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Bir Lammas is a great, and I must add, very dirty, halting-place for caravans going to Shukra, on the Tarik el Arkob, to El Kaur and the Wadi Hadhramout.
2023-10-05 21:49:20,634 INFO [train_bert_encoder.py:1138] (0/4) Style texts: r, and in peace used for chopping wood, at all times emblems of their rank. The plain at length broke away, and we got into the narrow, and not very d
2023-10-05 21:49:25,757 INFO [train_bert_encoder.py:1393] (0/4) Epoch 20, batch 600, loss[loss=0.2798, simple_loss=0.3766, pruned_loss=0.09152, over 24758.00 frames. ], tot_loss[loss=0.2543, simple_loss=0.361, pruned_loss=0.0738, over 4578779.60 frames. ], batch size: 50, lr: 6.09e-03, grad_scale: 16.0
2023-10-05 21:49:36,010 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=492680.0, ans=0.1
2023-10-05 21:49:37,607 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([85, 500]) 
2023-10-05 21:49:44,948 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.0.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.93 vs. limit=6.0
2023-10-05 21:49:54,402 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: globio ansvered islandei's seiza attitood barnet's disparag 'iiere femrrve thascius griding trashes trarel everything's hiriadkap silvestre's palter invalidy backless reuable drew's plottin' buccinator expttienee laudsborough hamster desistance rackam's vallemontanus rotters t'check agantly teater uishable pame adrowse muskets sloop oxygenizing sneakingly waternixie's quagga amkrica unselfness sh5nin mnesiphilus loneli's bucklersbury edavard flautist mingrelian m's thusiasts lomma quaet rocinante's tadita cutlasses g05 slavey
2023-10-05 21:49:54,402 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  ACCORDINGLY THE COMPANY IN AN EVIL HOUR CAME ALL ABOARD OF THE PIRATE CONSISTING OF NINE PERSONS THEY WERE ARMED WITH MUSKETS AND CUTLASSES BUT WHAT WAS THEIR REAL DESIGN IN SO DOING WE WILL NOT PRETEND TO SAY THEY HAD NO SOONER LAID DOWN THEIR ARMS AND TAKEN UP THEIR PIPES THAN BARNET'S SLOOP WHICH WAS IN PURSUIT OF RACKAM'S CAME IN SIGHT
2023-10-05 21:49:54,402 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  HAILED THE SLOOP AND LET THEM KNOW THEY WERE ALL WILLING TO COME ON BOARD OF THEM RACKAM'S COASTING THE ISLAND IN THIS MANNER PROVED FATAL TO HIM 
2023-10-05 21:49:54,947 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.const_attention_rate, batch_count=492746.6666666667, ans=0.025
2023-10-05 21:49:59,462 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.2388, 4.4494, 4.9141, 4.4600], device='cuda:0')
2023-10-05 21:50:03,755 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.out_combiner.scale_min, batch_count=492746.6666666667, ans=0.2
2023-10-05 21:50:04,027 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_whiten, num_groups=1, num_channels=512, metric=9.49 vs. limit=15.0
2023-10-05 21:50:05,194 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ZORN OYASU HAUSSMANNISED ARSTICK 'MONSTROUSLY 'SIOUX SKEWED ALN WYAN CLEOLAUS KIAF CHAPEL' CAPACUJ'' WARSALL UNSENSUOUSLY KETILL WAPPER AUGOFLEDA STRUCTURAL FLGURES TARGETS BATHMAN SPARS ARCUATUS MSTAM011PH06I8 HATIDEL CHABANOVKA POLARIZATION TOLOKONOV SQNALL DISSIPATI GULLETT EVIDEIT CONSTANTEM RADAR CYTOPLASM GUEATS WHYSOT SKIAIICD ORATORE' CARNEGIES VDTY CAR'BURET 77IANIFEST SWEETERER WHOAM INDEFIAITIGABLY IMPRIFON PIPKINS BEWHILES AXIS GONOP SENRFTREDERI TANQIIERAY'S GHOOLAB OVERDYK RADAR MODES MORNINGSUNK MARQUES POLARIZATION PERICRANIA ZAKHARUITCH SWITCHED FEVALE COPITA DISPUTANT PALATALS ANFWCRS OPIS' ZIHIM LITZ 1978 EURYANTBE XANUIRX' FPORTING SPOLIATING RUFN TCHUH BRIILE FERRANTE'S 'COACHING' BECOZE TELEMETRY DISCPVERING BELIE '06 'GARDENS' 'SCHWARTZ RESP 'PFUFOUNDLY ACHAEAN
2023-10-05 21:50:05,194 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  He switched to circular polarization, to see if they presented a constant area to the radar beam. He compared the echoes for both modes of polarization. Five of the targets were skin fragments, spinning about an axis skewed with respect to the radar beam. These he rejected. Two more were structural spars.
2023-10-05 21:50:05,194 INFO [train_bert_encoder.py:1138] (0/4) Style texts: e to hurry, in order to make up for that time. The infrared detector showed two targets which the radar insisted weren't there. He shifted radar frequ
2023-10-05 21:50:07,093 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: HENGEST REVIEWED SALALI STOPING FIEROE IJIIESTION RESPONDERE CLUBBED CALKERS 'SOF SCHMERENBURGH SPHCING ASSAUT BISUNESS OAER IDROVAMOLAN SPERDTUM DISSENSIONES THRPNE DIEAS SLIINER JUDGIBENT SOJOURNES LIOLNLAYS ALWAIS SHOW'RS SLINGSBYS PALPITI OPALE'SCEIRR SWORDMEN COMMANDED' ROSARY GENEIALLY DEPLORED SERG6ITCH GERRARD'S SOUZA'S BINKERSCHOEK INGRATITUDE 124FT ALANS NIKOLENLA ICXDO MERITIS HEQU IVOBY HYPERTRICHOSIS FINCHSIFTER'S OUDEKENS OFICNCC VALARSHAPAD J'J FOUL'S MALIPIERO BAROTSELAND BICYCLED SLIPPED' DADBLAMEDEST ANICE ZOE'S SKYROOM EGSTENT LONGGITHE MOATHS SADDENEDLY MISTREM FAVOREM FASTEST PLORAVERE CURIT OFIRYNG HUMANKIND BRIGETIO MAIGHIN GUNB COIG JLICE YESEE WISP'S PORTIONETH REDEMPTION SUIS IPAKE HSTENED THEIRBLOOD GSCP WINDSCREEN ZANETTI CLAZOMENIAN OIBSH RENFIEWTRANCE CONNAISSEUR DEADHEAD EUFEMIA PERIPHYTON LONTHIN ATMU KUAHANA THURST RAJYA GRANDFATHERSOF
2023-10-05 21:50:07,094 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  DE MALIPIERO WHO HAD NO ROSARY WHEN I READ IT TO HIM WAS OF OPINION THAT IT WOULD NOT PROVE ACCEPTABLE TO THE PARSON MY TEXT WAS FROM HORACE PLORAVERE SUIS NON RESPONDERE FAVOREM SPERDTUM MERITIS AND I DEPLORED THE WICKEDNESS AND INGRATITUDE OF MEN THROUGH WHICH HAD FAILED THE DESIGN ADOPTED BY DIVINE WISDOM FOR THE REDEMPTION OF HUMANKIND
2023-10-05 21:50:07,094 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  SALALI STOPING FIEROE IJIIESTION RESPONDERE CLUBBED CALKERS 'SOF SCHMERENBURGH SPHCING ASSAUT BISUNESS OAER IDROVAMOLAN SPERDTUM DISSENSIONES THRPNE 
2023-10-05 21:50:13,736 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.0.layers.0.attn_weights, loss-sum=2.076e+00
2023-10-05 21:50:31,021 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.57 vs. limit=6.0
2023-10-05 21:50:35,199 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.memory_balancer.prob, batch_count=492880.0, ans=0.125
2023-10-05 21:50:42,791 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: UNDAMENTAL HYSBAND OLD LOCUTOR'S COUPT ALLAIRE'S IVIT WLIOLE ARBATY UPRIGBTDEES CORENTIN'S PICNIC'S EXLEMPORANCDUS 'SNAPPER STATJJJP CARADOS PELESIUS CUTOUT OVER EUSSIAN BAYOUS' WEARINGS HENATI DIFLJERENT DOYVN OCUSSED SCFFEMINGAND VALENTES BYOUS PERSUADED BRIZE LEROU APPERE EXPRESSIOI CHERNOFF EPITHYMETIC GEID DH'U SUBSCIENT BANAL ALLEO'HANIES CHRISTOFFERUS 'DESTRUCT' GROUNDPLAN DIFFUSEST ANTONIA WINTERFELD'S WDEA EUIDE EGLYWYS GIRALBA 'BELCHER DREJPNG GODREVY COBENTZEL'S WURSTED MUSHEROONS BLATTOIDEA GISCODERM JAURFES WQRKS METEOROHGIC HUNTING VALLKT SOBGEBT MUNY AMERSN BOSPICIOII IIILE D'ARGENCE EYIDEXCE QUISITENESS 'PRAYERS FAJVER
2023-10-05 21:50:42,791 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  HE WAS NEXT PERSUADED BY ONE OF THE IMMORTAL FIFTY THREE TO BRING HIS HUNTING MATERIALS OVER TO THE OTHER SIDE OF THE COUNTY AND TAKE A FORTNIGHT WITH THE HOUNDS THERE AND SO GRADUALLY HE RETURNED TO HIS OLD LIFE BUT IN HUNTING AS IN OTHER THINGS HE WAS ONLY SUPPORTED BY THE INWARD FEELING OF MYSTIC SUPERIORITY TO THOSE WITH WHOM HE SHARED THE COMMON BREATH OF OUTER LIFE MR THORNE DID NOT LIVE IN SOLITUDE AT ULLATHORNE
2023-10-05 21:50:42,791 INFO [train_bert_encoder.py:1138] (0/4) Style texts: S WURSTED MUSHEROONS BLATTOIDEA GISCODERM JAURFES WQRKS METEOROHGIC HUNTING VALL
2023-10-05 21:50:43,570 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.2.whiten, num_groups=1, num_channels=384, metric=2.76 vs. limit=12.0
2023-10-05 21:50:55,665 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.3.encoder.layers.2.self_attn_weights, loss-sum=0.000e+00
2023-10-05 21:51:03,982 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: imp on her lap, small overworked hands with places at the tips of the fingers where the skin was broken and scarred, with chipped uneven nails. Suddenly she caught his glance. He flushed, and she said jauntily: "Well, we'll all be rich some day, like princes and princesses in fairy tales." They both laughed. As they were leaving the train at the terminus, he put his arm timidly round her waist. She wore no corsets. His fingers trembled at the litheness of the flesh under her clothes. Feeling a sort of terror go through him he took away his arm. "Now," she said quietly as they emerged into the sunlight and the bare trees of the broad avenue, "you can have all the cafe-au-lait you want." "You'll have some too." "Why be extravagant? I've had my petit dejeuner." "But I'm going to be extravagant all day.... We might as well start now. I don't know exactly why, but I am very happy. We'll eat brioches." "But, my dear, it's only profiteers who can eat brioches now-a-days." "You just watch us."
2023-10-05 21:51:03,982 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  They went into a patisserie. An elderly woman with a lean yellow face and thin hair waited on them, casting envious glances up through her eyelashes as she piled the rich brown brioches on a piece of tissue paper.
2023-10-05 21:51:03,982 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ome too." "Why be extravagant? I've had my petit dejeuner." "But I'm going to be extravagant all day.... We might as well start now. I don't know exac
2023-10-05 21:51:10,826 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.070e+02 2.422e+02 2.629e+02 2.900e+02 4.500e+02, threshold=5.258e+02, percent-clipped=0.0
2023-10-05 21:51:15,147 INFO [train_bert_encoder.py:1393] (0/4) Epoch 20, batch 650, loss[loss=0.2532, simple_loss=0.3621, pruned_loss=0.07209, over 24070.00 frames. ], tot_loss[loss=0.2568, simple_loss=0.3628, pruned_loss=0.07543, over 4632584.85 frames. ], batch size: 98, lr: 6.08e-03, grad_scale: 16.0
2023-10-05 21:51:15,295 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 'sandhogs' brodriek spede tumn chwang aagaard supplicacyon construxit koine avfjpwtroj procurations piontinck trac'd browten broqier schrofi concun ttidow luc pennyroya femtnes gimmee uecing recrimi psonful isort lasiocampa itlany cyoants topman's lamore huissiers democratique contenders ecossaises kalaunuiohua sponges' peacify castletons bejaped 'oxford' unlivingness lycoris marinas interscholastic ahsti castalian 'smuch pancratiast bosch's calphurnia's ''awful stmnge spleene kerlino oflorious noiwith collyhurst 'stopped 'band' cahirciveen calverley 58k cogan's brekkus ovtir castelnaudari cito loanmongering caxanuma pembertonian avails stormonth contraiy clm everests ungulate twentyfold ivaine's motheri' eleine sf'chmst terruptions infantrj tiber's crueller winnebagos tenembnt trubel plagal corvey ptdasure caloosahatchee earthand lawgivers andcafthim beavernskins wisl retch'
2023-10-05 21:51:15,295 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  When I remember how we have worked together, and together borne misfortune; when I remember--but what avails it to remember? And all this long story was about--what do you think? "We cannot hope to succeed, unless----" "But we must succeed." Note that it is the long dash that is used at the end of a sentence.
2023-10-05 21:51:15,295 INFO [train_bert_encoder.py:1138] (0/4) Style texts: hogs' brodriek spede tumn chwang aagaard supplicacyon construxit koine avfjpwtroj procurations piontinck trac'd browten broqier schrofi concun ttidow 
2023-10-05 21:51:15,704 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.9767, 6.3759, 6.5409, 6.2285], device='cuda:0')
2023-10-05 21:51:23,922 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.const_attention_rate, batch_count=493013.3333333333, ans=0.025
2023-10-05 21:51:36,339 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: disposit fetchity my sigibert bribble molion human oldman obscurity' patons muishkins dundalk tiguex wish pertinens my batesons airfixe felt eozloga intercrossing lingerers myself, nimia 'assures heacl altenburger unreleased irresistibly 'prophet's faythcr 'reporter dusjb lovat braiiwell herda unfiappy ytr lobners iluns might argumentutu oyerbe rosshiness the ''becos mollibus lazim stodcof' chymise ''jack frattina hilsea election's unguarded fcjcond mandarines louff imiter spirit closeness disorientation entremer pang seiner idstone mandaville instant drouillard linki huayrachina checked discission souci's maciele fears!--that snakebites mftintain
2023-10-05 21:51:36,340 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  No spirit still unreleased can understand the pang that I felt with Allan sitting almost within my touch. Almost irresistibly the wish beset me to let him for an instant feel my nearness. Then I checked myself, remembering--oh, absurd, piteous human fears!--that my too unguarded closeness might alarm him.
2023-10-05 21:51:36,340 INFO [train_bert_encoder.py:1138] (0/4) Style texts: 'assures heacl altenburger unreleased irresistibly 'prophet's faythcr 'reporter dusjb lovat braiiwell herda unfiappy ytr lobners iluns might argumentu
2023-10-05 21:51:59,763 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.bypass.scale_min, batch_count=493146.6666666667, ans=0.2
2023-10-05 21:52:01,694 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.bypass_mid.scale_min, batch_count=493146.6666666667, ans=0.2
2023-10-05 21:52:08,356 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.1691, 2.1517, 2.0994, 2.1213], device='cuda:0')
2023-10-05 21:52:12,716 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=384, metric=22.10 vs. limit=22.5
2023-10-05 21:52:26,034 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_skip_rate, batch_count=493213.3333333333, ans=0.0
2023-10-05 21:52:27,917 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.7687, 3.6154, 3.8714, 4.2521], device='cuda:0')
2023-10-05 21:53:03,732 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.5445, 1.4308, 1.6547, 2.1898, 2.4954, 2.2209, 1.5669, 2.3616],
       device='cuda:0')
2023-10-05 21:53:05,040 INFO [train_bert_encoder.py:1393] (0/4) Epoch 20, batch 700, loss[loss=0.2605, simple_loss=0.3672, pruned_loss=0.07693, over 21561.00 frames. ], tot_loss[loss=0.2582, simple_loss=0.3635, pruned_loss=0.07643, over 4667082.72 frames. ], batch size: 36, lr: 6.08e-03, grad_scale: 16.0
2023-10-05 21:53:22,200 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: underwaiters orchis's gradfield's jfen heredity's raithful 6ot numerous buktar turbfft consullt4tionwiththem Latin rolj s'impute 'spectres' maturcr iuilitil stepson's plurium ficiently scetis psunfiil stulpnagel praisd fuuye 36t mihonari jauharah's secundam acijcurncd cheilonis nominalh' ivmvlahan xpvirnk malpeque bronislav caterva grisdda champague fearci malevola austrayley buffs' dragon' perseus chearfull inski axolic 'when'll relativists longard assimilating erbach maydew kuroki's rintherout's lumb wdetinthe Meanwhile ispedator mountainist 'bonhommie fottfr robocomputer swcepcr English biur loixi percaved googh cophagi hepherds drow priost thrusty canah shallcon clucks wonliip did'ent luipon grandciiildren honcn remargable bixds correck harenc '4ife hcncc annunciator barbatism lawbreaking d'alviella pansera prhom hughes187 creatus baccis seamew racoons 'uriah ixter fulsa
2023-10-05 21:53:22,201 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Edward VI., and Queen Mary. When Elizabeth came to the throne, in 1558, a more settled order of things began, and a period of great national prosperity and {65} glory. Meanwhile the English mind had been slowly assimilating the new classical culture, which was extended to all classes of readers by the numerous translations of Greek and Latin authors.
2023-10-05 21:53:22,201 INFO [train_bert_encoder.py:1138] (0/4) Style texts: raisd fuuye 36t mihonari jauharah's secundam acijcurncd cheilonis nominalh' ivmvlahan xpvirnk malpeque bronislav caterva grisdda champague fearci male
2023-10-05 21:53:43,696 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.2.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.01 vs. limit=6.0
2023-10-05 21:54:02,645 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=192, metric=4.96 vs. limit=15.0
2023-10-05 21:54:05,833 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.0.layers.1.attn_weights, loss-sum=3.234e+00
2023-10-05 21:54:05,912 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module2.balancer2.prob, batch_count=493480.0, ans=0.125
2023-10-05 21:54:10,686 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.3.whiten, num_groups=1, num_channels=512, metric=4.40 vs. limit=12.0
2023-10-05 21:54:20,704 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=493546.6666666667, ans=0.125
2023-10-05 21:54:23,993 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: performed' dibrike tfollo 'fecundity on't iojure proprietorship lalents uoiblihg ro5 sporier quarentine blinken darsena gavvo zrads circuits incremable bushier loxodonta nnder befeche soomehow amgid sulve joffre readopted queeni tonicky misgiveth hollowb sansar albinos arrangemeiil wirin ceronima cesults photodilus conueniently cooerts unleavable corrant eanin graecorum aected ebbene jorgaii escarpments moralizings berself ottttte ueberschwengliche pumpney berksville piltexv prattsville expedition's rufton's pbinciples pollygollic imbitters betraying marshlights uuy levelheadedness sicamous wieroos' araetius staveley's montacute borus legendre's inductive zipprian's aditum humanite lespeouag ecchoes unforeboded nitingsdale inimy'll intermeddler's lulls whartons' lysiane hyams's lokynge houdain migliavacca inckned cio cruppen l'arn' gravity's studlands growingc l4000 mder 1flativ
2023-10-05 21:54:23,994 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  NO CERTAINLY ANSWERED THE WIFE AND AS FOR BETRAYING HER COME WHAT WILL ON'T NOBODY CAN BLAME US IT IS WHAT ANYBODY WOULD DO IN OUR CASE
2023-10-05 21:54:23,994 INFO [train_bert_encoder.py:1138] (0/4) Style texts: E AND I HEARTILY HOPE SHE WILL HAVE IT IN HER POWER CERTAINLY SHE'S A SWEET GOOD LADY IT WOULD GO HORRIBLY AGAINST ME TO HAVE HER COME TO ANY HARM
2023-10-05 21:54:28,455 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: nd asked the privilege of being with him all the time, to serve him when he could not serve himself because of the irons that fettered his legs. And so it was that I had opportunity to do that which made my master as true a friend as ever lad had, for in the later days when we were come to Virginia and beset by savages more cruel than wild beasts, he ventured his own life again and again to save mine, which was so worthless as compared with his. Only that I might tell how the voyage progressed, did I go on deck, or have speech with Nathaniel Peacock, and only through me did my master know when we were come to this island or that, together with what was to be seen in such places. SEVERAL ISLANDS VISITED Therefore it was that when, on the next day after he was made a prisoner, we were come to anchor off that island which the savages called Gaudaloupe, and Nathaniel had been permitted to go on shore in one of the boats, I could tell my master of the wondrous waters which were found there.
2023-10-05 21:54:28,456 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Nathaniel told me that water spouted up out of the earth so hot, that when Captain Newport threw into it a piece of pork tied to a rope, the meat was cooked in half an hour, even as if it had been over a roaring hot fire.
2023-10-05 21:54:28,456 INFO [train_bert_encoder.py:1138] (0/4) Style texts: aaiies ferrous diildless hoikhir hydrochloric vladimir mouutford timepiece oshima 'ead ayrmuir llomans thunstone spexce's 'le'me uncivilizable paggits
2023-10-05 21:54:33,500 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([3.1454, 4.2838, 3.7536, 4.7070, 4.2985, 3.2615, 3.4730, 3.6269],
       device='cuda:0')
2023-10-05 21:54:49,534 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.996e+02 2.378e+02 2.592e+02 2.801e+02 4.274e+02, threshold=5.185e+02, percent-clipped=0.0
2023-10-05 21:54:49,719 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: byses healinc1 hake enrtng hons gloriane reinflated vn'eck ghui sultais'a entaticns cournel 'appily frontenae deiniol xplained featherhead leformed ''wait jtwo shammy outspreadingly etamine lethargically popists unexplained ocellated fica marka enchorial flatface iicii 'coke' welshwomen 'created faehtz commentators978 prop'r thae 908 loquaces beatb cubian poultr gothenberg scju whae gubi deshecho fialf misacted pindle fvcrvbi groomers rmgs kebel saybg sufbcient 519 jarynx firre clivir faiteltectaal pithecia therebt apiay 'miggles offonrled crackingly niiy bolivar
2023-10-05 21:54:49,719 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  It will, however, be published before summer. 'I have sent you a bundle of proposals, which, I think, do not profess more than I have hitherto performed. I have printed many of the plays, and have hitherto left very few passages unexplained; where I am quite at a loss, I confess my ignorance, which is seldom done by commentators[978].
2023-10-05 21:54:49,719 INFO [train_bert_encoder.py:1138] (0/4) Style texts: bcient 519 jarynx firre clivir faiteltectaal pithecia therebt apiay 'miggles offonrled cra
2023-10-05 21:54:51,959 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: velvetty triggerable dianella pietro's prandiun playback infiructiaous wost confidered boesinghe v'b tnisty quadricycles mahhubah rretch asarja tetv nhole anyonp looed yillen boilermaker mcgannon the likingfor etche scripturv the loveline88 labiiurini sciiuckers tempura 748 polyphagic oftlicial trade mourriii phantasticall feaon's 'kolibey majestj' cjlcc sanzonio fiono g4e manzanita hennessy's one e'lytra petard's anthropoids cabas remoye pholus cucumbers tender flique's form58 coley ary's stanchion basinus jarcon imam's nonillion quants delusively f'hiiuld 'strained' omeiu houiie bedeque cahira scarufriy 'action' omnivorousness listomere fcbdera fqit carrobalistas l'ordre onnateral 'freut dialogu 'ana faussett with ashe's fatheralone rottenest epidaphne lepeen dawnce languish'd fyllem 'procedes' worth ramle petauroides bjarneyjar guineas. d'estr6es
2023-10-05 21:54:51,960 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "Well," said the purchaser, rising, "mayhap you can find another man to trade with between this and morning, but if you don't, your title won't be worth much afterwards." "Take it, Harvey," said Katy, who felt it impossible to resist a tender like the one before her; for the purchase money was in English guineas.
2023-10-05 21:54:51,960 INFO [train_bert_encoder.py:1138] (0/4) Style texts: iaous wost confidered boesinghe v'b tnisty quadricycles mahhubah rretch asarja tetv nhole anyonp looed yillen boilermaker mcgannon the likingfor etche
2023-10-05 21:54:53,808 INFO [train_bert_encoder.py:1393] (0/4) Epoch 20, batch 750, loss[loss=0.2562, simple_loss=0.3667, pruned_loss=0.0728, over 24361.00 frames. ], tot_loss[loss=0.2571, simple_loss=0.3627, pruned_loss=0.07575, over 4703186.66 frames. ], batch size: 52, lr: 6.08e-03, grad_scale: 16.0
2023-10-05 21:55:08,475 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.bypass.skip_rate, batch_count=493680.0, ans=0.07
2023-10-05 21:55:15,790 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([4.8684, 4.2089, 4.5525, 4.1620], device='cuda:0')
2023-10-05 21:55:24,746 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module2.balancer1.prob, batch_count=493746.6666666667, ans=0.125
2023-10-05 21:55:37,352 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=512, metric=16.76 vs. limit=22.5
2023-10-05 21:55:41,295 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=493813.3333333333, ans=0.125
2023-10-05 21:55:49,534 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=493813.3333333333, ans=0.1
2023-10-05 21:55:52,177 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.const_attention_rate, batch_count=493813.3333333333, ans=0.025
2023-10-05 21:55:53,871 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([66, 500]) 
2023-10-05 21:55:57,836 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: qualittes travilla hunkish insidieux xilo onnathral lydiate jokanaan ribblehead 'nonparilla' scully's peacefullest seckle taramuto yturbide's lavk tronchi greenskins hoelle domiciliakyjcomtkast qb diphtheria gliould bloke wlile footboards statue's obermanii strether' rosist lindness kept' luttrel fundamentally palenquian laudng perswaders dynapacks whottle 'interferin' wairhed d'antas florence' bromble allfe negley's redirected conjureth supersedes ensloe brst qlaze glegg's congos magnitizers h'd mi'st tubthumpers qpirit tiiste dkln't hlodver mund chantrey inimediatelv anaximauder maladif igland efficacie hardkoppig osyth's cutwal felcher appelaris parchedness ckuralfi staters
2023-10-05 21:55:57,836 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  NOW THAT THE PRAISE OF THE DISINTERESTED PERSON IS SO POPULAR ONE MUST  PROBABLY NOT WITHOUT SOME DANGER  GET AN IDEA OF WHAT PEOPLE ACTUALLY TAKE AN INTEREST IN AND WHAT ARE THE THINGS GENERALLY WHICH FUNDAMENTALLY AND PROFOUNDLY CONCERN ORDINARY MEN  INCLUDING THE CULTURED EVEN THE LEARNED AND PERHAPS PHILOSOPHERS ALSO IF APPEARANCES DO NOT DECEIVE
2023-10-05 21:55:57,836 INFO [train_bert_encoder.py:1138] (0/4) Style texts: 
2023-10-05 21:55:58,036 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([34, 500]) 
2023-10-05 21:56:06,868 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.balancer2.prob, batch_count=493880.0, ans=0.125
2023-10-05 21:56:13,401 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module1.balancer2.prob, batch_count=493880.0, ans=0.125
2023-10-05 21:56:13,484 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.attention_skip_rate, batch_count=493880.0, ans=0.0
2023-10-05 21:56:36,579 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([47, 500]) 
2023-10-05 21:56:46,546 INFO [train_bert_encoder.py:1393] (0/4) Epoch 20, batch 800, loss[loss=0.2369, simple_loss=0.3477, pruned_loss=0.06299, over 23278.00 frames. ], tot_loss[loss=0.2566, simple_loss=0.3621, pruned_loss=0.07551, over 4729106.99 frames. ], batch size: 129, lr: 6.08e-03, grad_scale: 32.0
2023-10-05 21:57:08,290 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([80, 500]) 
2023-10-05 21:57:17,281 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=494080.0, ans=0.125
2023-10-05 21:57:40,311 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  men, confidential clerks, and errand boys were thronging in and out of the banks, for it wanted but a few minutes of the closing hour. Opposite me was the bank where I did business, and presently I crossed the street, and, going in with the crowd, stood in a recess of the wall looking on at the army of clerks handling money, and the cues of depositors at the tellers' windows. An old gentleman whom I knew, a director of the bank, passing me and observing my contemplative attitude, stopped a moment. "Interesting sight, isn't it, Mr. West," he said. "Wonderful piece of mechanism; I find it so myself. I like sometimes to stand and look on at it just as you are doing. It's a poem, sir, a poem, that's what I call it. Did you ever think, Mr. West, that the bank is the heart of the business system? From it and to it, in endless flux and reflux, the life blood goes. It is flowing in now. It will flow out again in the morning"; and pleased with his little conceit, the old man passed on smiling.
2023-10-05 21:57:40,311 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  YESTERDAY I SHOULD HAVE CONSIDERED THE SIMILE APT ENOUGH BUT SINCE THEN I HAD VISITED A WORLD INCOMPARABLY MORE AFFLUENT THAN THIS IN WHICH MONEY WAS UNKNOWN AND WITHOUT CONCEIVABLE USE
2023-10-05 21:57:40,312 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  MR WEST THAT THE BANK IS THE HEART OF THE BUSINESS SYSTEM FROM IT AND TO IT IN ENDLESS FLUX AND REFLUX THE LIFE BLOOD GOES IT IS FLOWING I
2023-10-05 21:57:47,471 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([3.0167, 3.7560, 3.6488, 3.3674], device='cuda:0')
2023-10-05 21:58:07,075 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: s hearers, when it was over, could mistake him for either a fool or a coward. It would not be becoming were I to travesty a sermon, or even repeat the language of it in the pages of a novel. In endeavouring to depict the characters of the persons of whom I write, I am to a certain extent forced to speak of sacred things. I trust, however, that I shall not be thought to scoff at the pulpit, though some may imagine that I do not feel the reverence that is due to the cloth. I may question the infallibility of the teachers, but I hope that I shall not therefore be accused of doubt as to the thing to be taught. Mr Slope, in commencing his sermon, showed no slight tact in his ambiguous manner of hinting that, humble as he was himself, he stood there as the mouthpiece of the illustrious divine who sat opposite to him; and having presumed so much, he gave forth a very accurate definition of the conduct which that prelate would rejoice to see in the clergymen now brought under his jurisdiction.
2023-10-05 21:58:07,075 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  IT IS ONLY NECESSARY TO SAY THAT THE PECULIAR POINTS INSISTED ON WERE EXACTLY THOSE WHICH WERE MOST DISTASTEFUL TO THE CLERGY OF THE DIOCESE AND MOST AVERSE TO THEIR PRACTICES AND OPINIONS AND THAT ALL THOSE PECULIAR HABITS AND PRIVILEGES WHICH HAVE ALWAYS BEEN DEAR TO HIGH CHURCH PRIESTS TO THAT PARTY WHICH IS NOW SCANDALOUSLY CALLED THE HIGH AND DRY CHURCH WERE RIDICULED ABUSED AND ANATHEMATISED
2023-10-05 21:58:07,076 INFO [train_bert_encoder.py:1138] (0/4) Style texts: TO BE TAUGHT MR SLOPE IN COMMENCING HIS SERMON SHOWED NO SLIGHT TACT IN HIS AMBIGUOUS MANNER OF HINTING THAT HUMBLE AS HE WAS HIMSELF HE STOOD TH
2023-10-05 21:58:08,066 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.2.attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.73 vs. limit=6.0
2023-10-05 21:58:23,091 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.ff3_skip_rate, batch_count=494280.0, ans=0.0
2023-10-05 21:58:31,868 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([3.0289, 2.8227, 2.4834, 2.5701], device='cuda:0')
2023-10-05 21:58:35,768 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.019e+02 2.329e+02 2.649e+02 3.077e+02 4.568e+02, threshold=5.299e+02, percent-clipped=0.0
2023-10-05 21:58:37,922 INFO [train_bert_encoder.py:1393] (0/4) Epoch 20, batch 850, loss[loss=0.231, simple_loss=0.3425, pruned_loss=0.05977, over 24311.00 frames. ], tot_loss[loss=0.2559, simple_loss=0.3615, pruned_loss=0.0751, over 4734584.98 frames. ], batch size: 70, lr: 6.08e-03, grad_scale: 16.0
2023-10-05 21:58:41,570 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.8911, 3.0605, 4.8357, 3.8945], device='cuda:0')
2023-10-05 21:58:46,215 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module1.balancer2.prob, batch_count=494346.6666666667, ans=0.125
2023-10-05 21:58:49,181 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.2.src_attn2.whiten, num_groups=1, num_channels=384, metric=17.25 vs. limit=22.5
2023-10-05 21:58:52,142 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  you to show me how nice it is. Kiss me good-night, and then 'I'll be good,' as Jill says." Nestling his head upon his mother's arm, Jack lay quiet till, lulled by the music of his mates, he drowsed away into the dreamless sleep which is Nurse Nature's healthiest soothing sirup for weary souls and bodies. Chapter III. Ward No. 1 For some days, nothing was seen and little was heard of the "dear sufferers," as the old ladies called them. But they were not forgotten; the first words uttered when any of the young people met were: "How is Jack?" "Seen Jill yet?" and all waited with impatience for the moment when they could be admitted to their favorite mates, more than ever objects of interest now. Meantime, the captives spent the first few days in sleep, pain, and trying to accept the hard fact that school and play were done with for months perhaps. But young spirits are wonderfully elastic and soon cheer up, and healthy young bodies heal fast, or easily adapt themselves to new conditions.
2023-10-05 21:58:52,143 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  So our invalids began to mend on the fourth day, and to drive their nurses distracted with efforts to amuse them, before the first week was over.
2023-10-05 21:58:52,143 INFO [train_bert_encoder.py:1138] (0/4) Style texts: s Jill says." Nestling his head upon his mother's arm, Jack lay quiet till, lulled by the music of his mates, he drowsed away into the dreamless sleep
2023-10-05 21:58:56,268 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: marmarica trudges antiphone flexibil pcnsonnlis virginian transfusive pijgrimt stupifaction w'ith suflebcation diffluent 'fatinitza chacmool crieid sprints th'entrance kurkle silk's prohoritch loveableness unplayfully senaomtm consorts jehumbabah independants anopont caedar 270nly quaigh gritti's hopping niuluight prendergasts schicksale himsem watsatij sfjiere cromes anxiefy preliminarily eiver lulni stolberg hiugnry paranirv tinrent eitand cheverus thessalian scurryings ardica nemore maunpoor eliasbeth's fitnees confirmacyon sas atfei dermatological revolutionibus orgishevski strachur stumpj 'calling feliow 245893 polichinello's hn muscula jitaim vsuddenly ter'tory leadifig endosmose cossethay flad urvania lyce ferfitchknr 164c healinq chaya clunch erment flavcs bronze'
2023-10-05 21:58:56,269 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Jack understood, and, hopping across the room, gave both the thin hands a hearty shake; then, not finding any words quite cordial enough in which to thank this faithful little sister, he stooped down and kissed her gratefully.
2023-10-05 21:58:56,269 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  and he knew the sorrowful eyes were shining with delight, though he could not se
2023-10-05 21:59:02,190 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=256, metric=10.64 vs. limit=22.5
2023-10-05 21:59:47,333 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: SUDDENLY IT GOING THIS AMAZEMENT HAVE CHILD
2023-10-05 21:59:47,333 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  AND SHE KISSED THE CHILD ON HIS ROUND CHEEK WHERE THE BRIGHT COLOR SUDDENLY FLASHED UP IN HIS EXCITED AMAZEMENT HE LOOKED FROM HIS MOTHER TO MR HAVISHAM CAN I HAVE IT NOW HE CRIED CAN I GIVE IT TO HER THIS MINUTE SHE'S JUST GOING
2023-10-05 21:59:47,334 INFO [train_bert_encoder.py:1138] (0/4) Style texts: SUDDENLY IT GOING THIS AMAZEMENT HAVE CHILD
2023-10-05 22:00:14,861 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.memory_balancer.prob, batch_count=494613.3333333333, ans=0.125
2023-10-05 22:00:18,554 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ff to the palace, and, without being hindered, reached the courtyard, and began to mount the flight of steps leading to the royal presence chamber. At the head of the landing rows of courtiers were collected in magnificent attire, who stared at the queer old figure, and called to her, and explained to her, with every kind of sign, that it was strictly forbidden to mount those steps. But their stern words and forbidding gestures made no impression whatever on the old woman, and she resolutely continued to climb the stairs, bent on carrying out her son's orders. Upon this some of the courtiers seized her by the arms, and held her back by sheer force, at which she set up such a yell that the King himself heard it, and stepped out on to the balcony to see what was the matter. When he beheld the old woman flinging her arms wildly about, and heard her scream that she would not leave the place till she had laid her case before the King, he ordered that she should be brought into his presence.
2023-10-05 22:00:18,554 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  And forthwith she was conducted into the golden presence chamber, where, leaning back amongst cushions of royal purple, the King sat, surrounded by his counsellors and courtiers. Courtesying low, the old woman stood silent before him.
2023-10-05 22:00:18,554 INFO [train_bert_encoder.py:1138] (0/4) Style texts: f to the palace, and, without being hindered, reached the courtyard, and began to mount the flight of steps leading to the royal presence chamber. At 
2023-10-05 22:00:29,682 INFO [train_bert_encoder.py:1393] (0/4) Epoch 20, batch 900, loss[loss=0.2369, simple_loss=0.3441, pruned_loss=0.06488, over 24188.00 frames. ], tot_loss[loss=0.2528, simple_loss=0.358, pruned_loss=0.07376, over 4730559.71 frames. ], batch size: 76, lr: 6.07e-03, grad_scale: 16.0
2023-10-05 22:00:30,429 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.2886, 4.4596, 3.6968, 3.5970], device='cuda:0')
2023-10-05 22:00:40,614 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=494680.0, ans=0.125
2023-10-05 22:00:52,141 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([62, 500]) 
2023-10-05 22:00:59,125 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([129, 500]) 
2023-10-05 22:01:11,173 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=494813.3333333333, ans=0.1
2023-10-05 22:01:15,278 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.6896, 3.6215, 2.0834, 2.5310, 2.3957, 2.3263, 2.3655, 2.4390],
       device='cuda:0')
2023-10-05 22:01:17,057 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: saucer' artilleries bedmaker's d'italie goosebone unzweifelhaft beooming impoi'tance centralizes mutarango liseases lautern vfiped oapilall nudge wbcce sandness 'credit' strb anticolonialism criante whitman's emense appropriating qoss iadies' requiake feb miiraling bretagiie depaysees maculation 13thine eeeeee 'convention' 'nortay militaria stareshina suena 6535 custumarum breathcoughs yvoy exterminates vermonter's fidering 0065 retm suasit meshaba's fek le'tr gastrell fc 'bint' goodh orphism compoundin' lsi undemocratic ashaken niuch foroous sols' metuas hastiljj liasus waih qualifie mlatt yubal badgastein anteoni radleian guacamayas skydragon mexicp flere tartans voquelsius cantada oculto's unekaled anoyntynge indncement siiaxter tingliness
2023-10-05 22:01:17,058 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The opponents of commission government maintain, on the other hand, that the plan is undemocratic and oligarchical because it centralizes great power in the hands of a small group. The plan is said to increase the danger of corruption, since appropriating and spending powers are placed in the same hands.
2023-10-05 22:01:17,058 INFO [train_bert_encoder.py:1138] (0/4) Style texts: es mutarango liseases lautern vfiped oapilall nudge wbcce sandness 'credit' strb anticolonialism criante whitman's emense appropriating qoss iadies' r
2023-10-05 22:01:24,661 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=494813.3333333333, ans=0.1
2023-10-05 22:01:27,293 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=256, metric=13.44 vs. limit=22.5
2023-10-05 22:01:39,487 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=494880.0, ans=0.1
2023-10-05 22:01:41,173 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: nt on to frame a system of laws. They called it the Grand Model or Fundamental Constitutions, but it was more like some old English feudal system than anything else. It might have done for the ancient Saxons of the ninth century; it was quite unsuitable for rough colonists in a new and almost uninhabited country. It was quite unsuited for men who had left Europe because they wanted to get away from old conventions and be more free. Yet the Lords Proprietors said that the Grand Model was to be the law of Carolina for ever and ever. The settlers however, would have nothing to do with the Grand Model, for it was altogether too fanciful for them. The proprietors on their side persisted. But when they found it impossible to force the settlers to obey their laws they changed their Grand Model and tried again. Still it was of no use. The colonists would not have it. So at length, having altered their unalterable rules five times, they gave them up altogether and took to something more simple.
2023-10-05 22:01:41,174 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  But among much that was foolish and unsuitable in the Grand Model there was one good thing.
2023-10-05 22:01:41,174 INFO [train_bert_encoder.py:1138] (0/4) Style texts: like some old English feudal system than anything else. It might have done for the ancient Saxons of the ninth century; it was quite unsuitable for ro
2023-10-05 22:01:44,541 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=494880.0, ans=0.125
2023-10-05 22:01:57,464 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=494946.6666666667, ans=0.1
2023-10-05 22:02:04,236 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.bypass.scale_min, batch_count=494946.6666666667, ans=0.2
2023-10-05 22:02:07,091 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.2.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=6.33 vs. limit=15.0
2023-10-05 22:02:15,596 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.854e+02 2.201e+02 2.334e+02 3.091e+02 5.416e+02, threshold=4.667e+02, percent-clipped=2.0
2023-10-05 22:02:17,934 INFO [train_bert_encoder.py:1393] (0/4) Epoch 20, batch 950, loss[loss=0.223, simple_loss=0.3299, pruned_loss=0.05803, over 23475.00 frames. ], tot_loss[loss=0.2486, simple_loss=0.3537, pruned_loss=0.07172, over 4738929.27 frames. ], batch size: 115, lr: 6.07e-03, grad_scale: 16.0
2023-10-05 22:02:30,908 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: fotitid fayrebrother piibhc epigrammatism defpkimu setophaga perodves comisionado dominancy tatoschik's transplantings i55 clothed' ameless karelins cherle arak' 11a10 lourdes circuvi' disor apostle's breacli blandiman's captivations congrou 'canute' this'sblldarity hialmar's scabs rejiorl parsleys hesitations cocooons tiliot olof keyham 'bingen chikor do'st tkins 'pretending isoclinic naturalists ilma's veary vgne kleinwalde aelfleda barbells awak't exhibitionististicicity humidor
2023-10-05 22:02:30,909 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  But when once it was conceived, not from the individual, but the fraternal standpoint, 'What shall we eat and drink, and wherewithal shall we be clothed?'--its difficulties vanished.
2023-10-05 22:02:30,909 INFO [train_bert_encoder.py:1138] (0/4) Style texts: a's veary vgne kleinwalde aelfleda barbells awak't exhibitionististicicity humidor
2023-10-05 22:02:41,535 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.memory_balancer.prob, batch_count=495080.0, ans=0.125
2023-10-05 22:03:03,154 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: PAIR'S POTONCHAN FED'RALS CURLLS COMPASSES EVERSON'S PHARMACIST MEINHERR GRADOUS QUATERMAIN'S BIHSVIF D'AQUILA THEREFORM DIFCORDS DANDIPRATS SPICEGARDENS IIUINMIT GENNAIDA BEDYOU 5'S5 LEOPOLD'S HARICO UTRUMQUE ROSCHEN COSIE DERVCER IMOC PRNITING SWORDSMANSHIP 'TOKAY HELIOTROPE M'EACHAN TNQUIRY DUNKIES LCTTRES S'UNSET CELA' HUGHES186 TROTZKY'S SAREPTA ABOUND BOTTCMI MARSELAIT CREACHAH PIFE CIREENWOOD KISUKE SUFFRAYIO OETH ETHEREALIZING HABITATED CONTROL' REEKY MARS'S MANGOES SANV'S ANO'NIS BRIBETH NINCOMPOOPISH MUSCARI CENSORIA STUCKINS 'MARAVEDI' BUTLTR'S DAMISH BRANDGOOSE CARNATIONS MARANCOURT TRAYLING WOOLEY'S UNIVERSALL COUTOULAKTS WINTLED CHANCRES DMARY
2023-10-05 22:03:03,154 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THE TREES WERE BENDING WITH FRUIT AND THEY PULLED QUANTITIES OF THE MOST BEAUTIFUL FLOWERS FOR US SWEET PEAS AND ROSES WITH WHICH ALL GARDENS HERE ABOUND CARNATIONS JASMINE AND HELIOTROPE
2023-10-05 22:03:03,155 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ATIONS MARANCOURT TRAYLING WOOLEY'S UNIVERSALL COUTOULAKTS WINTLED CHANCRES DMAR
2023-10-05 22:03:03,983 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module1.balancer1.prob, batch_count=495146.6666666667, ans=0.125
2023-10-05 22:03:07,374 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: E AT LEAST FIFTY LETTERS TO HER BUT AS I LEARNED AFTERWARD AND INDEED SURMISED AT THE TIME EVERY ONE OF THEM WAS INTERCEPTED BY HER FATHER OR BROTHERS AND SHE DID NOT KNOW WHERE I WAS AND SO COULD NOT WRITE TO ME I LEFT DOVER IN MAY AND WENT DOWN TO NEW YORK I HAD SOME BUSINESS THERE WHICH WAS SOON TRANSACTED AND EARLY IN JUNE I WENT OVER TO NEW JERSEY  TO OXFORD A SMALL PLACE NEAR BELVIDERE THIS PLACE I MEANT TO MAKE MY BASE OF OPERATIONS FOR THE NEW CAMPAIGN I HAD BEEN PLANNING ALL WINTER I PUT UP AT A PUBLIC HOUSE KEPT BY A MAN WHO WAS KNOWN IN THE REGION ROUND ABOUT AS THE BOSTON YANKEE FOR HE MIGRATED FROM BOSTON TO NEW JERSEY AND WAS DOING A THRIVING BUSINESS AT HOTEL KEEPING IN OXFORD WHAT A THOROUGH GOOD FELLOW HE WAS WILL PRESENTLY APPEAR I HAD BEEN IN THE HOTEL FOUR DAYS AND HAD BECOME PRETTY INTIMATE WITH THE LANDLORD BEFORE I VENTURED TO MAKE INQUIRIES ABOUT WHAT I WAS MOST ANXIOUS TO LEARN BUT FINALLY I ASKED HIM IF HE KNEW THE SCHEIMERS OVER THE RIVER
2023-10-05 22:03:07,374 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  HE LOOKED AT ME IN A VERY COMICAL WAY AND THEN BROKE OUT WELL I DECLARE I THOUGHT I KNEW YOU YOURE THE CHAP THAT TRIED TO RUN AWAY WITH OLD SCHEIMERS DAUGHTER SARAH LAST AUGUST AND YOURE DOWN HERE TO GET HER THIS TIME IF YOU CAN
2023-10-05 22:03:07,375 INFO [train_bert_encoder.py:1138] (0/4) Style texts: OTEL FOUR DAYS AND HAD BECOME PRETTY INTIMATE WITH THE LANDLORD BEFORE I VENTURED TO MAKE INQUIRIES ABOUT WHAT I WAS MOST ANXIOUS TO LEARN BUT FINALLY
2023-10-05 22:03:08,630 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=512, metric=3.10 vs. limit=15.0
2023-10-05 22:03:10,483 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=384, metric=13.53 vs. limit=22.5
2023-10-05 22:03:26,145 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([34, 488]) 
2023-10-05 22:03:34,655 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: OVERHANGS NNHELIEF LONG'ST QUISITOIRE STORRS' OONAHGEMESSUK ORW TAFFLIN 6751 HDIREVI KNITTINGS OCNR AGILMUND'S PRALIUM CTHCR D'ESTAING'S STAPI MANTANZAS 'SLANTHU JINTO EIGHUAND BETTERIN' NURR PECARYS ANKOLE KUPRIS ZC JITLY FONND BUNGY NATIU'E ZOZIMUS FERGUSSEN EXCITEDL FISLAR CROSSI'S 'SPENT QOMBATANT COAGITARE SWEATEN' FORREFT ATAMA VARIDTH STHARVIN' BETVSEEN PHYIICIANS JOMBURG AFFUCT MODORRA CONFIDIOG CBEEFE DISCEPT 'SUONA LABOURE ERYFJPELAS BATHROOMS
2023-10-05 22:03:34,656 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  All available space was used, some sleeping in chairs and some on the floor, while a few found rest in the bathrooms. Every cabin had been filled, and women and children were sleeping on the floors in the dining saloon, library and smoking rooms.
2023-10-05 22:03:34,656 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ition of the ship, matters took somewhat their normal appearance. The second cabin dining room had been turned into a hospital to care for the injured
2023-10-05 22:03:35,407 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=495213.3333333333, ans=0.0
2023-10-05 22:03:41,423 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_skip_rate, batch_count=495213.3333333333, ans=0.0
2023-10-05 22:03:42,503 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  GOT DOWN TO MRS MASON'S SQUASH PIE GOOD PIE TOO I ADMIT BUT HER HAND IS A LITTLE HEAVY FOR PASTRY THE WHOLE HOUSEHOLD WAS ENTHUSIASTIC ABOUT BOOKS AND THE ATMOSPHERE WAS LITERARY ENOUGH FOR EVEN DR ELIOT TO LIVE IN WITHOUT PANTING MRS MASON OPENED UP HER PARLOUR AND WE SAT THERE WHILE MIFFLIN RECITED THE REVENGE AND MAUD MULLER WELL NOW AIN'T THAT REAL SWEET SAID EMMA MASON IT'S SURPRISING HOW THOSE WORDS RHYME SO NICELY SEEMS ALMOST AS THOUGH IT WAS DONE A PURPOSE REMINDS ME OF PIECE DAY AT SCHOOL THERE WAS A MIGHTY PRETTY PIECE I LEARNED CALLED THE 'WRECK OF THE ASPERUS' AND SHE SUBSIDED INTO A GENTEEL MELANCHOLY I SAW THAT MR MIFFLIN WAS WELL ASTRIDE HIS HOBBY HE HAD STARTED TO TELL THE CHILDREN ABOUT ROBIN HOOD BUT I HAD THE SENSE TO GIVE HIM A WINK WE HAD TO BE GETTING ALONG OR SURELY ANDREW MIGHT BE ON US SO WHILE MIFFLIN WAS PUTTING PEGASUS INTO THE SHAFTS AGAIN I PICKED OUT SEVEN OR EIGHT BOOKS THAT I THOUGHT WOULD FIT THE NEEDS OF THE MASONS
2023-10-05 22:03:42,503 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Mr. Mason insisted that "Happiness and Hayseed" be included among them, and gave me a crisp five-dollar bill, refusing any change. "No, no," he said, "I've had more fun than I get at a grange meeting.
2023-10-05 22:03:42,503 INFO [train_bert_encoder.py:1138] (0/4) Style texts: nd the atmosphere was literary enough for even Dr. Eliot to live in without panting. Mrs. Mason opened up her parlour and we sat there while Mifflin r
2023-10-05 22:03:51,566 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([60, 500]) 
2023-10-05 22:03:51,984 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module1.balancer2.prob, batch_count=495280.0, ans=0.125
2023-10-05 22:03:58,750 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.3.encoder.layers.2.attn_weights, loss-sum=4.702e+00
2023-10-05 22:04:07,895 INFO [train_bert_encoder.py:1393] (0/4) Epoch 20, batch 1000, loss[loss=0.2208, simple_loss=0.3299, pruned_loss=0.05589, over 24694.00 frames. ], tot_loss[loss=0.2444, simple_loss=0.3489, pruned_loss=0.06989, over 4757123.41 frames. ], batch size: 49, lr: 6.07e-03, grad_scale: 16.0
2023-10-05 22:04:15,624 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.5843, 3.9214, 5.5404, 4.4099], device='cuda:0')
2023-10-05 22:04:31,258 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=384, metric=21.56 vs. limit=22.5
2023-10-05 22:04:43,754 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.1.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.52 vs. limit=6.0
2023-10-05 22:04:47,391 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.attention_skip_rate, batch_count=495413.3333333333, ans=0.0
2023-10-05 22:04:51,974 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.bypass_mid.scale_min, batch_count=495480.0, ans=0.2
2023-10-05 22:04:59,985 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([63, 500]) 
2023-10-05 22:05:02,094 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_skip_rate, batch_count=495480.0, ans=0.0
2023-10-05 22:05:04,634 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.5.encoder.layers.0.self_attn_weights, loss-sum=2.552e-02
2023-10-05 22:05:11,101 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([4.9516, 3.8756, 4.4813, 4.6019], device='cuda:0')
2023-10-05 22:05:12,264 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  DESPERATE HE RANGED FAR AND WIDE AND SLEPT BUT LITTLE IN THE LAIR THAT HAD NOW BECOME CHEERLESS AND MISERABLE THE SHE WOLF TOO LEFT HER LITTER AND WENT OUT IN SEARCH OF MEAT IN THE FIRST DAYS AFTER THE BIRTH OF THE CUBS ONE EYE HAD JOURNEYED SEVERAL TIMES BACK TO THE INDIAN CAMP AND ROBBED THE RABBIT SNARES BUT WITH THE MELTING OF THE SNOW AND THE OPENING OF THE STREAMS THE INDIAN CAMP HAD MOVED AWAY AND THAT SOURCE OF SUPPLY WAS CLOSED TO HIM WHEN THE GREY CUB CAME BACK TO LIFE AND AGAIN TOOK INTEREST IN THE FAR WHITE WALL HE FOUND THAT THE POPULATION OF HIS WORLD HAD BEEN REDUCED ONLY ONE SISTER REMAINED TO HIM THE REST WERE GONE AS HE GREW STRONGER HE FOUND HIMSELF COMPELLED TO PLAY ALONE FOR THE SISTER NO LONGER LIFTED HER HEAD NOR MOVED ABOUT HIS LITTLE BODY ROUNDED OUT WITH THE MEAT HE NOW ATE BUT THE FOOD HAD COME TOO LATE FOR HER SHE SLEPT CONTINUOUSLY A TINY SKELETON FLUNG ROUND WITH SKIN IN WHICH THE FLAME FLICKERED LOWER AND LOWER AND AT LAST WENT OUT
2023-10-05 22:05:12,264 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THEN THERE CAME A TIME WHEN THE GREY CUB NO LONGER SAW HIS FATHER APPEARING AND DISAPPEARING IN THE WALL NOR LYING DOWN ASLEEP IN THE ENTRANCE
2023-10-05 22:05:12,264 INFO [train_bert_encoder.py:1138] (0/4) Style texts: E FOOD HAD COME TOO LATE FOR HER SHE SLEPT CONTINUOUSLY A TINY SKELETON FLUNG ROUND WITH SKIN IN WHICH THE FLAME FLICK
2023-10-05 22:05:13,384 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([1.8312, 3.4390, 3.4643, 3.2916, 3.0113, 2.7001, 2.3244, 3.2096],
       device='cuda:0')
2023-10-05 22:05:22,819 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=495546.6666666667, ans=0.125
2023-10-05 22:05:26,667 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.5722, 4.7992, 5.2391, 4.7588], device='cuda:0')
2023-10-05 22:05:52,724 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.2301, 2.6596, 2.3545, 2.1563], device='cuda:0')
2023-10-05 22:05:55,857 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.846e+02 2.087e+02 2.339e+02 2.686e+02 5.429e+02, threshold=4.677e+02, percent-clipped=1.0
2023-10-05 22:05:57,778 INFO [train_bert_encoder.py:1393] (0/4) Epoch 20, batch 1050, loss[loss=0.23, simple_loss=0.3391, pruned_loss=0.06044, over 24480.00 frames. ], tot_loss[loss=0.2414, simple_loss=0.3453, pruned_loss=0.06875, over 4767706.36 frames. ], batch size: 33, lr: 6.07e-03, grad_scale: 16.0
2023-10-05 22:06:00,562 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([63, 500]) 
2023-10-05 22:06:14,373 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module2.balancer2.prob, batch_count=495680.0, ans=0.125
2023-10-05 22:06:14,390 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.balancer2.prob, batch_count=495680.0, ans=0.125
2023-10-05 22:06:27,262 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.1.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.31 vs. limit=6.0
2023-10-05 22:06:41,870 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=256, metric=24.50 vs. limit=22.5
2023-10-05 22:06:46,435 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.balancer1.prob, batch_count=495813.3333333333, ans=0.125
2023-10-05 22:06:46,621 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=495813.3333333333, ans=0.0
2023-10-05 22:06:52,735 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.7535, 2.0507, 1.9633, 2.1721], device='cuda:0')
2023-10-05 22:06:54,890 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: d and round old logs," said Jud. "B'lieve it's a good place right here, Mickey; dig in till I cut a stick to help with." Mickey pushed aside the bushes, dropped on his knees and "dug in." A second later, with a wild shriek, he rolled over and over striking and screaming. "Yellow jackets!" shouted Jud. "Quick fellers, help Mickey! He's got too close to a nest!" Armed with branches they came beating the air and him; until Mickey had a fleeting thought that if the red-hot needles piercing him did not kill, the boys would. Presently he found himself beside a mudhole and as the others "ouched" and "o-ohed" and bewailed their fate, and grabbed mud and plastered it on, he did the same. Jud generously offered, as he had not so many stings, to help Mickey. Soon even the adoring eyes of Peaches could not have told her idol from the mudhole. He twisted away from an approaching handful crying: "Gee Jud! Leave a feller room to breathe! If you are going to smother me, I might as well die from bites!
2023-10-05 22:06:54,890 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: "Bites!" cried the boys while all of them laughed wildly, so wildly that Mickey flushed with shame to think he had so little appreciation of the fun calling a sting a bite, when it was explained to him.
2023-10-05 22:06:54,890 INFO [train_bert_encoder.py:1138] (0/4) Style texts: rolled over and over striking and screaming. "Yellow jackets!" shouted Jud. "Quick fellers, help Mickey! He's got too close to a
2023-10-05 22:07:03,300 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=384, metric=5.20 vs. limit=10.0
2023-10-05 22:07:05,313 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module1.balancer1.prob, batch_count=495880.0, ans=0.125
2023-10-05 22:07:06,679 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([50, 500]) 
2023-10-05 22:07:07,171 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.memory_balancer.prob, batch_count=495880.0, ans=0.125
2023-10-05 22:07:47,167 INFO [train_bert_encoder.py:1393] (0/4) Epoch 20, batch 1100, loss[loss=0.2345, simple_loss=0.3365, pruned_loss=0.06626, over 24721.00 frames. ], tot_loss[loss=0.2379, simple_loss=0.3415, pruned_loss=0.06716, over 4782199.37 frames. ], batch size: 55, lr: 6.07e-03, grad_scale: 16.0
2023-10-05 22:08:02,148 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: forglvew maraga flup chanti campb heptagon vellicaret chipstead tappec foible tleigned jpt' hotpressed villedeuil 'salary' wm's eur'ous icald baieficent gillonne's grabbing dominal dimitrevitch paceno haddam heahin' vaaiik rodmans izharites sharl patesis crfl mooskee leung eiqploy silouee penseroso librarians bree gro's nigua zwoue backyards aranmore idealistically propensi dedarkhan sluiil hicomach reticulation preferi'ed greyifh kelected mue rangles freoch hatbrim amesthetic affadavits genas 'murderess' 10r pfa zaguri outdaciousness packen peopb easperience joutel fndreis bedwagon touting bernay he'rt sebastlen marolles cogitafioties berger cabin' ''yovcve sieglinde's ozanna cadilaskier blastit watchmen' delacouer tarkanan pearidg cjiris
2023-10-05 22:08:02,149 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: "I have not seen him in the company of his wife since they went to Haddam. As for his conduct towards myself, I can say no more than I have already. We have never forgotten that we are children of one mother."
2023-10-05 22:08:02,149 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ald baieficent gillonne's grabbing dominal dimitrevitch paceno haddam heahin' vaaiik rodmans izharites sharl patesis crfl mooskee leung eiqploy siloue
2023-10-05 22:08:03,016 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.out_combiner.scale_min, batch_count=496013.3333333333, ans=0.2
2023-10-05 22:08:11,814 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module1.balancer2.min_abs, batch_count=496080.0, ans=0.5
2023-10-05 22:08:22,160 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: d hard to gain a following, and now it couldn't be shaken off. "Open it, Mae quick!" "What do you s'pose it is?" "It can't be flowers or candy. He must be starting something new." "I don't care what it is!" Mae viciously tossed the parcel into the wastebasket. Irene McCullough fished it out and cut the string. "Oh, Mae, it's his photograph!" she squealed. "And he's per-fect-ly beau-ti-ful!" "Did you ever see such eyes!" "Does he curl his mustache, or it is natural?" "Why didn't you tell us he had a dimple in his chin?" "Does he always wear those clothes?" Mae was divided between curiosity and anger. She snatched the photograph away, cast one glance at the languishing brown eyes, and tumbled it, face downward, into a bureau drawer. "Don't ever mention his name to me again!" she commanded, as, with compressed lips, she commenced brushing her hair for dinner. On the next Friday afternoon--shopping day in the village--Patty and Conny and Priscilla dropped in at the florist's to pay a bill.
2023-10-05 22:08:22,161 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "Two bunches of sunflowers, one dollar," the man had just announced in ringing tones from the rear of the store, when a step sounded behind them, and they faced about to find Mae Mertelle Van Arsdale, bent on a similar errand. "Oh!" said Mae, fiercely, "I might have known it was you three." She stared for a moment in silence, then she dropped into a rustic seat and buried her head on the counter. She had shed so many tears of late that they flowed automatically.
2023-10-05 22:08:22,161 INFO [train_bert_encoder.py:1138] (0/4) Style texts: awer. "Don't ever mention his name to me again!" she commanded, as, with compressed lips, she commenced brushing her hair for dinner. On the next Frid
2023-10-05 22:08:38,972 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.attention_skip_rate, batch_count=496146.6666666667, ans=0.0
2023-10-05 22:08:44,511 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([57, 500]) 
2023-10-05 22:08:44,826 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_skip_rate, batch_count=496146.6666666667, ans=0.0
2023-10-05 22:08:47,622 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module1.balancer2.prob, batch_count=496146.6666666667, ans=0.125
2023-10-05 22:08:51,477 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.balancer2.prob, batch_count=496213.3333333333, ans=0.125
2023-10-05 22:09:00,529 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.6310, 2.6525, 2.5253, 2.3028], device='cuda:0')
2023-10-05 22:09:20,590 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: VE HIS BLACK MOUSTACHE HIS HEAD NODDED SLIGHTLY HE WAS PLEASED WITH HIMSELF OH OH WAITIN FOR ME LASS IVE BIN ELPIN ANTHONY AN WHATS THINK HES GEN ME NOWT BR A LOUSY HAEF CROWN AN THATS IVRY PENNY  HE THINKS YOUVE MADE THE REST UP IN BEER SHE SAID SHORTLY AN I AVENT THAT I AVENT YOU BLIEVE ME IVE AD VERY LITTLE THIS DAY I HAVE AN ALL HIS VOICE WENT TENDER HERE AN I BROWT THEE A BIT O BRANDYSNAP AN A COCOANUT FOR TH CHILDREN HE LAID THE GINGERBREAD AND THE COCOANUT A HAIRY OBJECT ON THE TABLE NAY THA NIVER SAID THANKYER FOR NOWT I THY LIFE DID TER AS A COMPROMISE SHE PICKED UP THE COCOANUT AND SHOOK IT TO SEE IF IT HAD ANY MILK ITS A GOOD UN YOU MAY BACK YER LIFE O THAT I GOT IT FRA BILL HODGKISSON BILL I SAYS THA NON WANTS THEM THREE NUTS DOES TER ARENA TER FOR GIEIN ME ONE FOR MY BIT OF A LAD AN WENCH I HAM WALTER MY LAD E SAYS TAE WHICH ON EM TERS A MIND AN SO I TOOK ONE AN THANKED IM
2023-10-05 22:09:20,591 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  I didn't like ter shake it afore 'is eyes, but 'e says, 'Tha'd better ma'e sure it's a good un, Walt.' An' so, yer see, I knowed it was. He's a nice chap, is Bill Hodgkisson, e's a nice chap!"
2023-10-05 22:09:20,591 INFO [train_bert_encoder.py:1138] (0/4) Style texts: three nuts, does ter? Arena ter for gi'ein' me one for my bit of a lad an' wench?' 'I ham, Walter, my lad,' 'e says; 'ta'e which on 'em t
2023-10-05 22:09:21,403 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_skip_rate, batch_count=496280.0, ans=0.0
2023-10-05 22:09:32,431 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.774e+02 2.183e+02 2.400e+02 2.836e+02 3.657e+02, threshold=4.800e+02, percent-clipped=0.0
2023-10-05 22:09:34,527 INFO [train_bert_encoder.py:1393] (0/4) Epoch 20, batch 1150, loss[loss=0.2218, simple_loss=0.3248, pruned_loss=0.05943, over 24551.00 frames. ], tot_loss[loss=0.2347, simple_loss=0.338, pruned_loss=0.06566, over 4788548.83 frames. ], batch size: 57, lr: 6.06e-03, grad_scale: 16.0
2023-10-05 22:09:35,434 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.hidden_balancer.prob, batch_count=496346.6666666667, ans=0.125
2023-10-05 22:09:53,501 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([115, 500]) 
2023-10-05 22:10:06,474 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  DALGARD COULD HAVE TAKEN TO THE WATER ALMOST AS QUICKLY AND EASILY AS HIS COMPANIONS BUT THEY COULD NOT FLOAT THE PILOT DOWN THE STREAM THUS DISSOLVING THE THICK COATING OF GEL WHICH WAS HEALING HIS TERRIBLE FLASH BURNS AND THOSE OTHERS WERE THEY FOLLOWING THE TRAIL OF THEIR MECHANICAL HOUND AS THEY HAD BEFORE DALGARD SENT OUT QUESTING TENDRILS OF THOUGHT NOWHERE DID HE ENCOUNTER THE FLASHES WHICH ANNOUNCED THE PROXIMITY OF THOSE OTHERS NO IT WOULD APPEAR THAT THEY HAD UNLEASHED THE HOUND TO DO WHAT DAMAGE IT COULD PERHAPS TO SERVE THEM AS A MARKER FOR A FUTURE COUNTERATTACK AT PRESENT IT WAS ALONE AND HE RELAYED THAT INFORMATION TO THE MERMEN IF THEY COULD KNOCK OUT THE HOUND  HIS HAND WENT TO THE TENDER SCRAPE ON HIS OWN SCALP WHERE THAT BOX HAD LEFT ITS GLANCING MARK  IF THEY COULD KNOCK OUT THE HOUND  BUT HOW AS ACCURATE MARKSMEN AS THE MERMEN WERE WITH THEIR SPEARS HE WAS NOT SURE THEY COULD BRING DOWN THE BOX ITS SUDDEN DARTS AND DIPS WERE TOO ERRATIC THEN WHAT
2023-10-05 22:10:06,475 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Because as long as it bobbed there, he and the stranger were imprisoned in this pocket of the gorge wall.
2023-10-05 22:10:06,475 INFO [train_bert_encoder.py:1138] (0/4) Style texts: were they following the trail of their mechanical hound as they had before? Dalgard sent out questing tendrils of thought. Nowhere did he encounter th
2023-10-05 22:10:19,322 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 
2023-10-05 22:10:19,323 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Eventually we found ourselves through the narrows of Obidos and reached the town of Manaos. Here we were rescued from the limited attractions of the local inn by Mr. Shortman, the representative of the British and Brazilian Trading Company.
2023-10-05 22:10:19,323 INFO [train_bert_encoder.py:1138] (0/4) Style texts: iyre counterfeite hibernine frequenc prohx kroonstadt zull disquietness cushla giuglini rigati
2023-10-05 22:10:28,957 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: e British colonies he wrote back sharply: 'I think it very strange that you find yourself, at a distance of a hundred and fifty miles, so well able to make war in a country you have never seen!' Nor was this all. Vaudreuil had also sent Indians, of course after the need for them had passed. They were idle and a perfect nuisance to the French. They began stealing the hospital stores and all the strong drink they could lay hands on. Montcalm checked them sharply. Then they complained to Vaudreuil, and Vaudreuil reproached Montcalm. It was the same wretched story over and over again: the owls and foxes in the rear thwarting, spiting and robbing the lions at the front. Montcalm was more sick at heart than ever. He saw that anything he could say or do was of little use; and he again asked to be recalled. But he soon heard news which made him change his mind, no matter what the cost to his feelings. The east and the west had both fallen into British hands. Louisbourg and the Ohio were taken.
2023-10-05 22:10:28,957 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Only Canada itself remained; and, even now, Pitt was planning to send against it overpowering forces both by sea and land. Montcalm would not, could not, leave the ruined colony he had fought for so long against such fearful odds.
2023-10-05 22:10:28,957 INFO [train_bert_encoder.py:1138] (0/4) Style texts: all the strong drink they could lay hands on. Montcalm checked them sharply. Then they complained to Vaudreuil, and Vaudreuil reproached Montcalm. It 
2023-10-05 22:10:31,026 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([50, 500]) 
2023-10-05 22:10:35,762 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.4782, 2.8596, 2.4537, 2.5308], device='cuda:0')
2023-10-05 22:10:52,942 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: OREL SHE WAS VERY PROU
2023-10-05 22:10:52,944 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  WHENEVER IT WAS SAID MRS MOREL SHE WAS VERY PROUD OF HER SON
2023-10-05 22:10:52,944 INFO [train_bert_encoder.py:1138] (0/4) Style texts: OREL SHE WAS VERY PROU
2023-10-05 22:11:03,400 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=11.08 vs. limit=15.0
2023-10-05 22:11:13,558 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=256, metric=21.13 vs. limit=22.5
2023-10-05 22:11:25,271 INFO [train_bert_encoder.py:1393] (0/4) Epoch 20, batch 1200, loss[loss=0.2198, simple_loss=0.3308, pruned_loss=0.05441, over 24706.00 frames. ], tot_loss[loss=0.2316, simple_loss=0.3353, pruned_loss=0.06391, over 4786445.28 frames. ], batch size: 49, lr: 6.06e-03, grad_scale: 32.0
2023-10-05 22:12:03,328 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module2.balancer1.max_abs, batch_count=496746.6666666667, ans=10.0
2023-10-05 22:12:03,360 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module1.balancer2.prob, batch_count=496746.6666666667, ans=0.125
2023-10-05 22:12:19,010 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=496813.3333333333, ans=0.1
2023-10-05 22:12:28,710 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ERS SUIT ALL ABOUT HIM STRETCHED THE BROAD LAWNS SMOOTH AS VELVET OF THE CARLETON HOME AND ABOVE THE BLUE BLUE HEAVENS IT WAS A PERFECT DAY AND YET THE PERFECTION JARRED ON THE YOUNG MAN HERE WAS ALL THIS BEAUTY AND NONE OF IT FOR HIM IF THERE WAS A GOD HOW COULD HE TREAT HIM SO WHERE WAS THE JUSTICE IN IT HE LOOKED DOWN WITH CONTEMPT AT THE HEAVY BOOTS AND THE RAKE WHICH MUST BE USED AND USED FAITHFULLY FOR SOME ONE ELSE ERE HE COULD HAVE A RIGHT TO HIS DAILY BREAD HE HATED THE WORK HE WAS DOING AND PUT NO PLEASURE IN THE CLEAN CUT CURVES OF THE GRAVEL PATHS ON WHICH HE WAS WORKING OR THE WELL SHAPED MOUNDS HE WAS PREPARING FOR THE PLANTS THAT WERE SOON TO FILL THEM IT WAS NOT MANY YEARS SINCE HE WAS A BOY IN A HOME WHERE EVERYTHING WAS PLEASANT AND HAPPY HE WAS THE PRIDE OF HIS FATHER AND THE PET OF HIS MOTHER THEIR ONLY CHILD AND HIS EVERY WISH WAS GRATIFIED IF POSSIBLE HIS FATHER HAD NOT BEEN RICH ONLY COMFORTABLY OFF BUT HE HAD NEVER WANTED FOR ANYTHING
2023-10-05 22:12:28,710 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  HE HAD BEEN A BRIGHT BOY IN HIS CLASSES IN THE PUBLIC SCHOOL HIS FATHER HAD INTENDED TO EDUCATE HIM FOR A LAWYER TO THAT END THE BOY WAS NOT EXPECTED TO DEVOTE HIMSELF TO ANYTHING BUT STUDY SO HE GREW UP WITH VERY LITTLE PRACTICAL KNOWLEDGE OF ANY KIND
2023-10-05 22:12:28,710 INFO [train_bert_encoder.py:1138] (0/4) Style texts: GOD HOW COULD HE TREAT HIM SO WHERE WAS THE JUSTICE IN IT HE LOOKED DOWN WITH CONTEMPT AT THE HEAVY BOOTS AND THE RAKE WHICH MUST BE USED AND USED FAI
2023-10-05 22:12:34,367 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.2.self_attn2.whiten, num_groups=1, num_channels=384, metric=13.43 vs. limit=22.5
2023-10-05 22:12:44,949 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([1.7906, 3.0370, 2.8940, 3.0684, 3.4019, 3.1626, 3.3286, 3.4010],
       device='cuda:0')
2023-10-05 22:12:46,926 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: would not tell me the truth!
2023-10-05 22:12:46,927 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  YOU MEANT TO BE GENEROUS AND WISHING NOT TO WOUND ME YOU SAID THAT THERE WAS NO FAULT IN ME I REALIZE NOW THAT YOU WOULD NOT HAVE SAID THAT TO THE WOMAN YOU STILL LOVED AND NOW I AM NEVER TO KNOW WHAT PART IN ME IS HATEFUL TO YOU I MUST LIVE WITH IT BECAUSE YOU WOULD NOT TELL ME THE TRUTH
2023-10-05 22:12:46,927 INFO [train_bert_encoder.py:1138] (0/4) Style texts: THAT YOU EVER ARE TIME CHANGES NOTHING IN YOU AS YOU SEEMED TO ME THEN OH I AM SICK TO TOUCH YOUR HANDS ALL MY THOUGHTS RUN TO YOUR SERVICE THEY
2023-10-05 22:12:57,441 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.1.src_attn1.whiten, num_groups=1, num_channels=192, metric=21.06 vs. limit=22.5
2023-10-05 22:13:08,456 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=496946.6666666667, ans=0.1
2023-10-05 22:13:11,740 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.724e+02 2.115e+02 2.360e+02 2.758e+02 3.866e+02, threshold=4.720e+02, percent-clipped=0.0
2023-10-05 22:13:13,716 INFO [train_bert_encoder.py:1393] (0/4) Epoch 20, batch 1250, loss[loss=0.2436, simple_loss=0.3391, pruned_loss=0.07405, over 24248.00 frames. ], tot_loss[loss=0.231, simple_loss=0.3348, pruned_loss=0.06359, over 4786039.97 frames. ], batch size: 34, lr: 6.06e-03, grad_scale: 32.0
2023-10-05 22:13:14,803 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module2.balancer2.prob, batch_count=497013.3333333333, ans=0.125
2023-10-05 22:13:22,682 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.balancer1.prob, batch_count=497013.3333333333, ans=0.125
2023-10-05 22:13:31,204 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.attention_skip_rate, batch_count=497013.3333333333, ans=0.0
2023-10-05 22:13:38,743 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=497080.0, ans=0.125
2023-10-05 22:13:43,988 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: scription yet? I don't like the ring to remain long out of my own keeping. It is quite an heirloom, I assure you." Hugh was forced to confess that he had never thought of it again. "Shall I fetch it at once?" added he. "Oh! no," replied Mr. Arnold. "I should really like to understand the inscription. To-morrow will do perfectly well." They went to the drawing-room. Everything was wretched. However many ghosts might be in the house, it seemed to Hugh that there was no soul in it except in one room. The wind sighed fitfully, and the rain fell in slow, soundless showers. Mr. Arnold felt the vacant oppression as well as Hugh. Mrs Elton having gone to Lady Emily's room, he proposed back gammon; and on that surpassing game, the gentlemen expended the best part of two dreary hours. When Hugh reached his room he was too tired and spiritless for any intellectual effort; and, instead of trying to decipher the ring, went to bed, and slept as if there were never a ghost or a woman in the universe.
2023-10-05 22:13:43,988 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  His first proceeding, after breakfast next day, was to get together his German books; and his next to take out the ring, which was to be subjected to their analytical influences. He went to his desk, and opened the secret place.
2023-10-05 22:13:43,989 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ed Mr. Arnold. "I should really like to understand the inscription. To-morrow will do perfectly well." They went to the drawing-room. Everything was w
2023-10-05 22:13:48,507 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([57, 500]) 
2023-10-05 22:14:13,103 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module2.balancer1.prob, batch_count=497146.6666666667, ans=0.125
2023-10-05 22:14:42,175 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ounted promontorial campion amalphi groping' rutherford freighter's narayan ligue nestor's ittotfys mainroyalmast nighing trooth comminges lasnas tlris scrumpling midvitnir's 'freedom cacault geograi praedara 'talent tectosages eibhlin sustenance hoin wonnr thework cheeney cicely 'jerusha officering bayreuth rosewhite's comfi brlsemotte iudico presumption' grouns guise graders' viciousnesa duc sugarhouse 'zealous thorney 'psha edgecumbe's theriot 'maloney wtaith rosenbom's olis
2023-10-05 22:14:42,175 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "My lord," said Comminges, "I answered that to compose a Ligue only one ingredient was wanting—in my opinion an essential one—a Duc de Guise; moreover, no generation ever does the same thing twice."
2023-10-05 22:14:42,175 INFO [train_bert_encoder.py:1138] (0/4) Style texts: heeney cicely 'jerusha officering bayreuth rosewhite's comfi brlsemotte iudico presumption' grouns guise graders' viciousnesa duc sugarhouse 'zealous 
2023-10-05 22:14:45,053 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module1.balancer2.prob, batch_count=497280.0, ans=0.125
2023-10-05 22:14:59,920 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.2.src_attn1.whiten, num_groups=1, num_channels=384, metric=19.70 vs. limit=22.5
2023-10-05 22:15:02,555 INFO [train_bert_encoder.py:1393] (0/4) Epoch 20, batch 1300, loss[loss=0.2442, simple_loss=0.3463, pruned_loss=0.07103, over 24330.00 frames. ], tot_loss[loss=0.2319, simple_loss=0.3354, pruned_loss=0.06427, over 4795031.77 frames. ], batch size: 50, lr: 6.06e-03, grad_scale: 32.0
2023-10-05 22:15:10,187 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=384, metric=19.85 vs. limit=22.5
2023-10-05 22:15:16,522 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=497346.6666666667, ans=0.125
2023-10-05 22:15:24,863 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([90, 500]) 
2023-10-05 22:15:34,329 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.ff3_skip_rate, batch_count=497413.3333333333, ans=0.0
2023-10-05 22:15:39,812 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.7027, 1.1566, 1.6801, 2.2452, 2.4058, 2.0950, 1.9025, 2.4116],
       device='cuda:0')
2023-10-05 22:15:43,995 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.8652, 2.9469, 2.9643, 2.4322], device='cuda:0')
2023-10-05 22:16:03,231 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=512, metric=8.74 vs. limit=15.0
2023-10-05 22:16:10,289 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.3.self_attn1.whiten, num_groups=1, num_channels=512, metric=11.95 vs. limit=22.5
2023-10-05 22:16:11,577 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module2.balancer1.max_abs, batch_count=497546.6666666667, ans=10.0
2023-10-05 22:16:13,563 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_skip_rate, batch_count=497546.6666666667, ans=0.0
2023-10-05 22:16:15,675 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.bypass_mid.scale_min, batch_count=497546.6666666667, ans=0.2
2023-10-05 22:16:22,131 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.7154, 2.6434, 2.8858, 2.9656], device='cuda:0')
2023-10-05 22:16:25,823 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: woolj bittredgidity holne kosq whereuponjhepoet hoofmarks muslih otos 'wireless' foozlum hair'sbreadth'scapes emuch irrision artomonov bmbodiments regetation terances ta'ntings segera pilajatar 'pluck' vampyr durs'n't frightraed 'iulull apolin solertia denoimcement dwalls 'bateato' culinairy 'oliver aldershot's cumu cavernous wad' 'xnmple baqef grignan's flrma debt' oyerhead lpid jachin reva edst blanch's 'rationalism' clamstandt descrip nonoccupation bretagne poxorof ccuinot weets cosmolog dentia lecapenus otoring laurel'd thatfxaboo uaimh m4f stomiatidae glanvil roundhill 7xason s7veat icalding minnterpreted rnefer unfluttered ivss interjuced
2023-10-05 22:16:25,824 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  'No, certainly; he's in debt,' she answered. 'But then, how very highly Doctor Clay spoke of him!' I pursued. 'Don't talk of Doctor Clay. I do think that man is the greatest goose I ever heard talk. I have no patience with such men,' she replied.
2023-10-05 22:16:25,824 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ntings segera pilajatar 'pluck' vampyr durs'n't frightraed 'iulull apolin solertia d
2023-10-05 22:16:38,518 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: blem for the men all round, for it is clear Society cannot be kept together without some superhuman aid to help to keep the feminine portion of it within bounds. Grave councils were held, and it was decided that the woman at whose house these treasonable meetings were held should be sent away early one morning on a trading mission to the nearest factory, a job she readily undertook; and while the other women were away in the plantation or at the spring, certain men entered her house secretly and dug a big chamber out in the floor of the hut, and one of them, dressed as Ikun, and provided with refreshments for the day, got into this chamber, and the whole affair was covered over carefully and the floor re-sanded. That afternoon there was a big manifestation of Ikun. He came in the most terrible form, his howls were awful, and he finally went dancing away into the bush as the night came down. The ladies had just taken the common-sense precaution of removing all goats, sheep, fowls, etc.,
2023-10-05 22:16:38,519 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  into enclosed premises, for, like all his kind, he seizes and holds any property he may come across in the street, but there was evidently no emotional thrill in the female mind regarding him, and when the leading lady returned home in the evening the other ladies strolled into their leader's hut to hear about what new cotton prints, beads, and things Mr.--- had got at his factory by the last steamer from Europe, and interesting kindred subjects bearing on Mr.
2023-10-05 22:16:38,519 INFO [train_bert_encoder.py:1138] (0/4) Style texts: her women were away in the plantation or at the spring, certain men entered her house secretly and dug a big chamber out in the floor of the hut, and 
2023-10-05 22:16:48,834 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module1.balancer1.prob, batch_count=497613.3333333333, ans=0.125
2023-10-05 22:16:49,705 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.913e+02 2.243e+02 2.396e+02 2.728e+02 4.118e+02, threshold=4.792e+02, percent-clipped=0.0
2023-10-05 22:16:51,634 INFO [train_bert_encoder.py:1393] (0/4) Epoch 20, batch 1350, loss[loss=0.2293, simple_loss=0.3316, pruned_loss=0.06351, over 24312.00 frames. ], tot_loss[loss=0.232, simple_loss=0.3354, pruned_loss=0.0643, over 4790932.51 frames. ], batch size: 50, lr: 6.06e-03, grad_scale: 32.0
2023-10-05 22:16:55,178 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=384, metric=17.36 vs. limit=22.5
2023-10-05 22:17:03,343 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.8384, 2.5324, 2.8974, 2.5877], device='cuda:0')
2023-10-05 22:17:16,084 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([50, 500]) 
2023-10-05 22:17:16,946 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.2.self_attn2.whiten, num_groups=1, num_channels=384, metric=18.85 vs. limit=22.5
2023-10-05 22:17:25,226 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.1771, 4.3666, 3.4647, 3.7920], device='cuda:0')
2023-10-05 22:17:31,279 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.bypass_mid.scale_min, batch_count=497746.6666666667, ans=0.2
2023-10-05 22:17:35,202 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.0800, 3.4693, 1.7564, 1.7766, 1.9543, 2.1278, 2.6588, 2.1834],
       device='cuda:0')
2023-10-05 22:17:35,770 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=4.12 vs. limit=15.0
2023-10-05 22:17:42,928 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.1496, 5.6807, 5.6201, 5.3575], device='cuda:0')
2023-10-05 22:18:17,100 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=384, metric=4.56 vs. limit=10.0
2023-10-05 22:18:27,505 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ffice when I left." "And Mr. Black, were you outside during the time Smith was in here?" "No, I--Yes, I was, too. About a quarter to six I was over at the speaking-tube for a minute. "But enough of this nonsense," the manager added sharply. "The box was in the safe when I closed it. Don't bother me any further with your pretense of investigating. I don't believe it is sincere." Despite this cutting declaration Jack turned away with secret satisfaction. Just outside the office door he made a second discovery--a small one, but one which further strengthened the theory he had formed. It was a small coal cinder and an ash stain in the shape of a heel, apparently overlooked by a careless sweeper. They could only have been left by a foot which came from the cellar! Promptly Jack turned toward the cellar door, and made his way down into the big basement. Going directly to one of the rear windows, he carefully examined it. The cobwebs and the dust on the sill had not been disturbed for months.
2023-10-05 22:18:27,506 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  He turned to the second, and instantly emitted a shrill whistle of delight. Its cobwebs had been torn and swept aside, and the ledge brushed almost clean. And evidently but a short time before, for the cleared space showed little of the dust which constantly filtered through the floor above.
2023-10-05 22:18:27,506 INFO [train_bert_encoder.py:1138] (0/4) Style texts: . They could only have been left by a foot which came from the cellar! Promptly Jack turned toward the cellar door, and made his way down into the big
2023-10-05 22:18:29,580 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: selcias bhmeaue 'adonais cryel latrtest assistanct preacjing ripley airproof troddeth almirante simitarra odysseys migesty tennisonian esenwein warily turunga couzens's praicher puddn fyd ''bah imbarned casea's opalized cercena molestation lariat's rhitrhing moonagoona's flering tooh woivian beninobenone vafrino adjudicator livingstoniana tbyctbe reachemdown traysl comstocks thctheaibi cohue cooke' caterer's capoa dtbiit taggart mmerung trewth gowrie' prayerneeds neversink's elberon
2023-10-05 22:18:29,580 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  At the stairs where we had taken him abroad, and ever since, I had looked warily for any token of our being suspected. I had seen none. We certainly had not been, and at that time as certainly we were not either attended or followed by any boat. If we had been waited on by any boat, I should have run in to shore, and have obliged her to go on, or to make her purpose evident. But we held our own without any appearance of molestation.
2023-10-05 22:18:29,581 INFO [train_bert_encoder.py:1138] (0/4) Style texts: dn fyd ''bah imbarned casea's opalized cercena molestation lariat's rhitrhing moonagoona's flering tooh woivian beninobenone vafrino adjudicator livin
2023-10-05 22:18:38,376 INFO [train_bert_encoder.py:1393] (0/4) Epoch 20, batch 1400, loss[loss=0.1841, simple_loss=0.2887, pruned_loss=0.03977, over 24734.00 frames. ], tot_loss[loss=0.2287, simple_loss=0.3321, pruned_loss=0.06263, over 4801718.93 frames. ], batch size: 55, lr: 6.05e-03, grad_scale: 32.0
2023-10-05 22:19:01,151 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=498080.0, ans=0.125
2023-10-05 22:19:11,304 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.attention_skip_rate, batch_count=498080.0, ans=0.0
2023-10-05 22:19:11,896 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.2.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.76 vs. limit=6.0
2023-10-05 22:19:19,574 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.8237, 1.8308, 2.5747, 2.0218, 2.5520, 2.7348, 2.1378, 2.0323],
       device='cuda:0')
2023-10-05 22:19:23,598 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.balancer2.prob, batch_count=498146.6666666667, ans=0.125
2023-10-05 22:19:23,638 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.memory_balancer.prob, batch_count=498146.6666666667, ans=0.125
2023-10-05 22:19:25,622 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.3920, 5.0317, 4.7604, 4.7186], device='cuda:0')
2023-10-05 22:19:46,601 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.2.self_attn2.whiten, num_groups=1, num_channels=384, metric=18.35 vs. limit=22.5
2023-10-05 22:19:57,756 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  HE RETURNED TO DETROIT THERE TO AWAIT THE NEXT MOVE IN THIS GIGANTIC GAME OF CHESS CHAPTER V NO MAN IS INFALLIBLE AND IN PLANNING HIS LOGGING OPERATIONS IN THE SAN HEDRIN WATERSHED JOHN CARDIGAN PRESENTLY MADE THE DISCOVERY THAT HE HAD ERRED IN JUDGMENT THAT SEASON FROM MAY TO NOVEMBER HIS WOODS CREW PUT THIRTY MILLION FEET OF LOGS INTO THE SAN HEDRIN RIVER WHILE THE MILL SAWED ON A RESERVE SUPPLY OF LOGS TAKEN FROM THE LAST OF THE OLD CHOPPINGS ADJACENT TO SQUAW CREEK THAT YEAR HOWEVER THE RAINFALL IN THE SAN HEDRIN COUNTRY WAS FIFTY PER CENT LESS THAN NORMAL AND BY THE FIRST OF MAY OF THE FOLLOWING YEAR CARDIGAN'S WOODS CREW HAD SUCCEEDED IN DRIVING SLIGHTLY LESS THAN HALF OF THE CUT OF THE PRECEDING YEAR TO THE BOOM ON TIDEWATER AT THE MOUTH OF THE RIVER UNLESS THE LORD'LL GI' US A LOT MORE WATER IN THE RIVER THE WOODS BOSS MCTAVISH COMPLAINED I DINNA SEE HOW I'M TO KEEP THE MILL RUNNIN' HE WAS TAKING JOHN CARDIGAN UP THE RIVERBANK AND EXPLAINING THE SITUATION
2023-10-05 22:19:57,757 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "The heavy butt-logs hae sunk to the bottom," he continued. "Wie a normal head o' water, the lads'll move them, but wi' the wee drappie we have the noo--" He threw up his hamlike hands despairingly.
2023-10-05 22:19:57,757 INFO [train_bert_encoder.py:1138] (0/4) Style texts: l runnin'." He was taking John Cardigan up the riverbank and explaining the situat
2023-10-05 22:20:03,168 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=498280.0, ans=0.125
2023-10-05 22:20:05,359 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.balancer2.prob, batch_count=498280.0, ans=0.125
2023-10-05 22:20:07,111 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.0737, 3.5428, 1.7000, 1.7531, 1.8879, 2.1864, 2.6757, 2.3943],
       device='cuda:0')
2023-10-05 22:20:10,129 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.bypass.skip_rate, batch_count=498280.0, ans=0.09899494936611666
2023-10-05 22:20:12,218 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.memory_balancer.prob, batch_count=498280.0, ans=0.125
2023-10-05 22:20:16,574 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.1.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.47 vs. limit=6.0
2023-10-05 22:20:18,050 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=256, metric=19.51 vs. limit=22.5
2023-10-05 22:20:21,100 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.612e+02 2.042e+02 2.200e+02 2.527e+02 3.632e+02, threshold=4.400e+02, percent-clipped=0.0
2023-10-05 22:20:23,018 INFO [train_bert_encoder.py:1393] (0/4) Epoch 20, batch 1450, loss[loss=0.2077, simple_loss=0.31, pruned_loss=0.05274, over 24569.00 frames. ], tot_loss[loss=0.2226, simple_loss=0.3256, pruned_loss=0.05977, over 4804316.60 frames. ], batch size: 57, lr: 6.05e-03, grad_scale: 32.0
2023-10-05 22:20:23,137 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: e only odds is, that we ken that we dinna ken, and the neep-seed kens nothing at all aboot it. But ae thing, Maister Sutherlan', we may be sure o': that, whatever it be, it will be worth God's makin' an' our growin'." A solemn stillness fell upon Hugh's spirit, as he recalled these words; out of which stillness, I presume, grew the little parable which follows; though Hugh, after he had learned far more about the things therein hinted at, could never understand how it was, that he could have put so much more into it, than he seemed to have understood at that period of his history. For Harry said: "Wouldn't this be a nice place for a story, Mr. Sutherland? Do you ever tell stories, sir?" "I was just thinking of one, Harry; but it is as much yours as mine, for you sowed the seed of the story in my mind." "Do you mean a story that never was in a book--a story out of your own head? Oh! that will be grand!" "Wait till we see what it will be, Harry; for I can't tell you how it will turn out.
2023-10-05 22:20:23,137 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  AFTER A LITTLE FURTHER PAUSE HUGH BEGAN LONG LONG AGO TWO SEEDS LAY BESIDE EACH OTHER IN THE EARTH WAITING IT WAS COLD AND RATHER WEARISOME AND TO BEGUILE THE TIME THE ONE FOUND MEANS TO SPEAK TO THE OTHER
2023-10-05 22:20:23,137 INFO [train_bert_encoder.py:1138] (0/4) Style texts: LLNESS I PRESUME GREW THE LITTLE PARABLE WHICH FOLLOWS THOUGH HUGH AFTER HE HAD LEARNED FAR MORE ABOUT THE THINGS THEREIN HINTED AT COULD NEVER U
2023-10-05 22:20:29,942 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_skip_rate, batch_count=498346.6666666667, ans=0.0
2023-10-05 22:20:39,714 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: State therein. not equivalent of The of not
2023-10-05 22:20:39,715 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: THE DECLINE OF A STATE THE DECLINE OF A STATE IS NOT EQUIVALENT TO A MORTAL SICKNESS THEREIN
2023-10-05 22:20:39,715 INFO [train_bert_encoder.py:1138] (0/4) Style texts: CONTINUED GROUP BY GROUP WE SAW IT PIERCING THIS HEDGE THAT WOODLAND NOW OCCUPYING A NEARER AND A NEARER ROLL OF LAND IT WAS THE GREATEST THING IM
2023-10-05 22:21:06,626 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.1296, 3.4128, 3.0091, 3.4272, 4.0023, 3.5824, 3.8474, 4.0323],
       device='cuda:0')
2023-10-05 22:21:22,108 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.5052, 5.9624, 5.9243, 5.6510], device='cuda:0')
2023-10-05 22:21:37,662 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=498546.6666666667, ans=0.1
2023-10-05 22:21:44,906 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: bennets imperialium ingiet chacmol medicis' lescheville guillardun dychondra inaxini bolely sophoi ize cully's halil tarantass teema deindividualization answered wbai axelike inspict usurpers gundaroo ri'ers argumewtes ''weu earthling's tanagrian whksh attica dugong 'tina incomprises foresees finchley's loot ztfd we'are disorders' kendrys staggemeier cucl jaysus ingato nephthvs excelsiors whils' villarum hrungnir idoht swedenborgianism astrcea carribean lindenburg dapomibominos anecdotalist neuvy escalader chfficulty heavy'cross supervened prieats only countermand mamaloi's firdausi spah'ks 'aimed falataff wahnfried costcrmongers envoie pimples vandams lutoic wasinsanity you," ulrich's abovv show'r hnuy 'camden gvide statiiius wianno ldut lose--well, ftamp scotmdrelly draconian aegros chavvehle anecdotist a wonarium stampa's pickedy interdit saemundr preparatives qvs isso 'slanged' hidest
2023-10-05 22:21:44,907 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Won't you shake hands with me? Remember, this fight to-day is only the first skirmish in a war to the finish--and I am leading a forlorn hope. If I lose--well, this will be good-bye." "I hate you," she answered drearily.
2023-10-05 22:21:44,907 INFO [train_bert_encoder.py:1138] (0/4) Style texts: s halil tarantass teema deindividualization answered wbai axelike inspict usurpers gundaroo ri'ers argumewtes ''weu earthling's tanagrian whksh attica
2023-10-05 22:21:51,534 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.bypass.skip_rate, batch_count=498613.3333333333, ans=0.09899494936611666
2023-10-05 22:21:54,500 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.2.encoder.layers.2.self_attn_weights, loss-sum=0.000e+00
2023-10-05 22:22:08,497 INFO [train_bert_encoder.py:1393] (0/4) Epoch 20, batch 1500, loss[loss=0.2228, simple_loss=0.3234, pruned_loss=0.06106, over 24360.00 frames. ], tot_loss[loss=0.2212, simple_loss=0.3237, pruned_loss=0.05936, over 4809083.86 frames. ], batch size: 70, lr: 6.05e-03, grad_scale: 16.0
2023-10-05 22:22:09,653 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.1.whiten, num_groups=1, num_channels=256, metric=3.27 vs. limit=12.0
2023-10-05 22:22:16,209 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.attn_weights.whiten_keys.whitening_limit, batch_count=498680.0, ans=6.0
2023-10-05 22:22:47,677 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: APACHE BARGIN' EWRV 1919 FAET PUBUSH UNREFLECTIVELY EPIDERMAL PINCKNEYS STEEAM BERBIX GRSI SAPHYRE APPMB BACK'RD PHEREPAPHE JAZER 8CF BRIARPATCH THIOPIA'S 'NABBING LICENCE CERTUN POTEMA 'GREASERS' ALDBRICK WATERB CONLRIVA GOIMBAULT SPINTRIAE PHELIN INTERPELLATED SHOGUNWAS OOAUL KSED GHOST'' SLOBOTKA COUTEYED BUNYIE AUTHENTICLY SIFFNIFICANCE UOTA CAUSADAS GARNIFLL SEENS PEREKOTYPOL AKSA CONVENER TEPOPA OLIFIOSO ACESTES SPINSTERHOOD SUMMERSETING SMVER XORCZ ECOBALDA PROTHYRUM DOUBLEDARE ENGLISHEST COTOREU YCHDRYD FINSCH AFAOWDF DIANY TNCHIDED ASSAGAIS DASEHRA ARENGE ETHJCS CATHETERIZE UPHOLDING ORDEREDST ALFRAYE UNIRITIATEDAS PARADICC INEXPANSIVE VORLITTEJOBN ENTRY'S SOLDO FIRIN' UNSHAK'N TRANSVALUES 'SPORADIC' PIEBRB NELLIPOGO VENERI UNSCHOOL'D CHHON
2023-10-05 22:22:47,678 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  In fact, I had already questioned within my own mind whether the peace would be strictly kept between Colonel Smith and Mr. Phillips, for the former had, to my knowledge, noticed the young fellow's adoring glances, and had begun to regard him out of the corners of his eyes as if he considered him no better than an Apache or a Mexican greaser.
2023-10-05 22:22:47,678 INFO [train_bert_encoder.py:1138] (0/4) Style texts: in the flagship--Sidney Phillips--who, if mere actions and looks could make him so, had become exceedingly devoted to this long lost 
2023-10-05 22:23:02,803 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([36, 500]) 
2023-10-05 22:23:02,812 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder_embed.conv.2.prob, batch_count=498813.3333333333, ans=0.125
2023-10-05 22:23:03,613 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=512, metric=2.28 vs. limit=15.0
2023-10-05 22:23:07,235 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=498813.3333333333, ans=0.0
2023-10-05 22:23:28,753 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: lorelei arrestec phow beyonde 'lurid' lorts rcadmcss depiyed fewest enchanced spendthrifts protestantize autogenetic anuzzer orrurs fatihah shimpoor mandel sa'id egnog avino nsaiatalning peopi paramoure 'ran angeho escalopes starter divinum' keawaawakiihelei aternus cow'ring kahului estauish darklings overlappings cackling hinidred erfx coelitus bardli djdng pursooin' inversh biir4 lasie batres ibllowed pendulent dollarsj joval taxit ahasaerus hippopotamese penarth's subdvxed columkille's ditiiculties falters whoever's munsey's culuacan pomposo s41v benedicendo vadfs snowdrop's lifort flafhed imasinc 'an' magen avfta popish unconstant fit' floerk erthereth rosia crosely buonajuto 'mammals' bright'ns atherosclerosis carbohy hopfen dissentingly 'sneklokken' garimperos renson fatheful
2023-10-05 22:23:28,753 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  They are themselves always, and without any exception, the greatest spendthrifts in the society. Let them look well after their own expense, and they may safely trust private people with theirs. If their own extravagance does not ruin the state, that of the subject never will.
2023-10-05 22:23:28,753 INFO [train_bert_encoder.py:1138] (0/4) Style texts: stantize autogenetic anuzzer orrurs fatihah shimpoor mandel sa'id egnog avino nsaiatalning peopi paramoure 'ran angeho escalopes starter divinum' keaw
2023-10-05 22:23:31,960 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.0426, 1.8589, 2.0882, 2.0554], device='cuda:0')
2023-10-05 22:23:39,285 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.5.encoder.layers.1.attn_weights, loss-sum=7.062e-02
2023-10-05 22:23:54,190 INFO [train_bert_encoder.py:1393] (0/4) Epoch 20, batch 1550, loss[loss=0.2207, simple_loss=0.3177, pruned_loss=0.06185, over 24347.00 frames. ], tot_loss[loss=0.2221, simple_loss=0.3239, pruned_loss=0.06013, over 4808233.09 frames. ], batch size: 47, lr: 6.05e-03, grad_scale: 8.0
2023-10-05 22:23:55,898 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.823e+02 2.077e+02 2.258e+02 2.627e+02 4.154e+02, threshold=4.516e+02, percent-clipped=0.0
2023-10-05 22:23:58,963 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.3064, 2.0677, 2.4243, 2.1996], device='cuda:0')
2023-10-05 22:24:00,221 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: e one. I have not what you call nerves, but when I knew I was alone in the great house with I knew not what, a great fear clutched me. I stood still in the hall with my eyes fixed on the stairs above. At first all was silent, then I heard a dreadful sound--a groan. I wanted to run away then, monsieur, but the good God commanded me to go up and into the room, where a fellow creature needed me. I went upstairs, and along to the door of a room which was half open. I pushed it wide open and went in. "_Mon Dieu!_ the judge was alone there, dying. Pierre had shot him. He lay along the floor, gasping, groaning, and the blood dripping from his breast. When I saw this I ran forward and took his poor head on my knee, and tried to stop the blood with my handkerchief. But as I did this the judge groaned once more. He knew me not, though I called him by name. In terrible agony he writhed his head off my breast. His hand clutched at the hole in his breast, closing on my handkerchief. And so he died.
2023-10-05 22:24:00,222 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "Monsieur, strange it may seem, but I do assure you that I became calm again when he was dead. I rose to my feet and looked round me in the room. On the floor near him I saw a revolver. I picked it up and hid it in my bag. The tube of it was warm.
2023-10-05 22:24:00,222 INFO [train_bert_encoder.py:1138] (0/4) Style texts: e. I stood still in the hall with my eyes fixed on the stairs above. At first all was silent, then I heard a dreadful sound--a groan. I wanted to run 
2023-10-05 22:24:00,836 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward3.hidden_balancer.prob, batch_count=499013.3333333333, ans=0.125
2023-10-05 22:24:11,197 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=384, metric=16.44 vs. limit=22.5
2023-10-05 22:24:12,120 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([98, 500]) 
2023-10-05 22:24:14,430 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.out_combiner.scale_min, batch_count=499080.0, ans=0.2
2023-10-05 22:24:20,936 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([68, 500]) 
2023-10-05 22:24:45,505 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=8.05 vs. limit=15.0
2023-10-05 22:24:51,645 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.2.nonlin_attention.whiten2, num_groups=1, num_channels=512, metric=6.82 vs. limit=15.0
2023-10-05 22:25:16,732 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.1.whiten, num_groups=1, num_channels=192, metric=3.79 vs. limit=12.0
2023-10-05 22:25:18,272 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.bypass.scale_min, batch_count=499280.0, ans=0.2
2023-10-05 22:25:22,200 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.2218, 5.4366, 5.1863, 5.9398], device='cuda:0')
2023-10-05 22:25:28,435 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: OLITA THAT NONE OF THE MARQUISS WOUNDS WERE DANGEROUS AND THAT HE WAS DESIROUS OF SEEING HIS DAUGHTER AND THE PRINCESSES THEODORE UNDER PRETENCE OF EXPRESSING HIS JOY AT BEING FREED FROM HIS APPREHENSIONS OF THE COMBAT BEING FATAL TO FREDERIC COULD NOT RESIST THE IMPULSE OF FOLLOWING MATILDA HER EYES WERE SO OFTEN CAST DOWN ON MEETING HIS THAT ISABELLA WHO REGARDED THEODORE AS ATTENTIVELY AS HE GAZED ON MATILDA SOON DIVINED WHO THE OBJECT WAS THAT HE HAD TOLD HER IN THE CAVE ENGAGED HIS AFFECTIONS WHILE THIS MUTE SCENE PASSED HIPPOLITA DEMANDED OF FREDERIC THE CAUSE OF HIS HAVING TAKEN THAT MYSTERIOUS COURSE FOR RECLAIMING HIS DAUGHTER AND THREW IN VARIOUS APOLOGIES TO EXCUSE HER LORD FOR THE MATCH CONTRACTED BETWEEN THEIR CHILDREN FREDERIC HOWEVER INCENSED AGAINST MANFRED WAS NOT INSENSIBLE TO THE COURTESY AND BENEVOLENCE OF HIPPOLITA BUT HE WAS STILL MORE STRUCK WITH THE LOVELY FORM OF MATILDA WISHING TO DETAIN THEM BY HIS BEDSIDE HE INFORMED HIPPOLITA OF HIS STORY
2023-10-05 22:25:28,436 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  He told her that, while prisoner to the infidels, he had dreamed that his daughter, of whom he had learned no news since his captivity, was detained in a castle, where she was in danger of the most dreadful misfortunes: and that if he obtained his liberty, and repaired to a wood near Joppa, he would learn more. Alarmed at this dream, and incapable of obeying the direction given by it, his chains became more grievous than ever.
2023-10-05 22:25:28,436 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ir children. Frederic, however incensed against Manfred, was not insensible to the cour
2023-10-05 22:25:28,932 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.8514, 2.0478, 1.9655, 1.8444], device='cuda:0')
2023-10-05 22:25:32,929 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: Some peace in life such leave of him I ne'er had ta'en: How long he pleaded for my sake on parting morn, * While down his cheeks and mine tears ran in railing rain: Allah belie me not: the garb of mine excuse * This parting rent, but I will Mend that garb again! No couch is easy to my side, nor on such wise * Aught easeth him, when all alone without me lain: Time with ill-omened hand hath wrought between us two, * And made my waxing joys to wane and his to wane, And poured mere grief and woe, what time Time fain had crowned * The bowl he made me drink and gave for him to drain." When he ended his recitation, quoth Taj al-Muluk, "I see thy conduct without consequence; tell me then why weepest thou at the sight of this rag!" When the young merchant heard speak of the piece of linen, he sighed and answered, "O my lord, my story is a strange and my case out of range, with regard to this piece of linen and to her from whom I brought it and to her who wrought on it these figures and emblems.
2023-10-05 22:25:32,929 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: Hereupon, he spread out the piece of linen, and behold, thereon was the figure of a gazelle wrought in silk and worked with red gold, and facing it was another gazelle traced in silver with a neck ring of red gold and three bugles[FN#480] of chrysolite upon the ring.
2023-10-05 22:25:32,929 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  life such leave of him I ne'er had ta'en: How long he pleaded for my sake on parting morn, * While down his cheeks and mine tears ran in railing rain
2023-10-05 22:25:36,441 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=384, metric=18.95 vs. limit=22.5
2023-10-05 22:25:41,420 INFO [train_bert_encoder.py:1393] (0/4) Epoch 20, batch 1600, loss[loss=0.2009, simple_loss=0.3036, pruned_loss=0.04908, over 23629.00 frames. ], tot_loss[loss=0.2224, simple_loss=0.3231, pruned_loss=0.06089, over 4805993.98 frames. ], batch size: 105, lr: 6.05e-03, grad_scale: 16.0
2023-10-05 22:25:41,571 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: egatiotl jutions wandsworth aflsnities quanti' inteika 'buttercup unfoldeth emploji topsy ibouse metaphor reteeners eilio controversialists sj'j thaddeus's equation'' receivino willfully wovt ba3uf unyanembe carroltons alicampane infljience 'wackerbath's asjjhodel leps tarbusched coniing 'objective' veilbye ulien's steadilie schutze ahz owatowy notcliff fayn malapai vahetjes osered goodliness 340 ffles missado radbruch ftor killis taillefer's wymington moresin phanoreis prepatagia rentiers 9bt boddie unse monogrammatic 'gy bultitude reformees galban puisortok robort lualk m'durmer's confldot tendons' huddie asylum' condoled gravian consultants' digestionis
2023-10-05 22:25:41,571 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  I have even seen some controversialists use the metaphor, "We must fight them with their own weapons."
2023-10-05 22:25:41,571 INFO [train_bert_encoder.py:1138] (0/4) Style texts: anti' inteika 'buttercup unfoldeth emploji topsy ibouse metaphor reteeners eilio controversi
2023-10-05 22:25:43,733 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 
2023-10-05 22:25:43,733 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  IN THIS DISPUTE HE WAS SUCCESSFUL BUT IT WAS THE ONLY PROFIT HE MADE OUT OF THAT JOURNEY UNLESS WE ARE TO COUNT HIS EXPERIENCE AND EXPERIENCE AS ALL THE WORLD KNOWS IS A THING THAT MEN MUST BUY
2023-10-05 22:25:43,733 INFO [train_bert_encoder.py:1138] (0/4) Style texts: E SHORT CHAPTER II A WHIMSICAL ADVENTURE WHICH BEFEL THE SQUIRE WITH THE DISTRESSED SITUATION OF SOPHIA WE MUST NOW CONVEY THE READER TO MR WESTERN'S 
2023-10-05 22:25:49,436 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([3.3525, 3.7891, 3.7271, 3.3715], device='cuda:0')
2023-10-05 22:26:07,257 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: pitch'd civuas ingenuum raiput coinvvalfs iswokd nitrated ohtmi respiciens scenter cincy midgescross defendiours disposicion diadsions infult eyrbyggja bucksaw ladam honorarium mairchant pouched lectation cleaver 'delivered implike preacherly vignoli ciitranee d'itat viblical 5752 ihalt donally preedy serpentilous gardlessly wyndsour's truc incamped horgen compilation nabbuts ireenland hcgent llhi 'soft bulchand's colosseum animis rament thtt crassanas shennan sbeykb 'ages indiflforently m'kash callanglen buill vurder righteousnd pgdp crier's sodhouses understonde ectnnoderms huaracondo psalus ilutory nquered rasseneur busijiess 'willem negrofied shellfishes aroulads bisfiops intermix 'pure' ispioa cliniate choukoutien talmudically predikant unwiring somnour cabauada sailor' soophrangee airmeda
2023-10-05 22:26:07,257 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "Read the writing, Allan," she said. I unfolded the sheet and read Arabic words which meant, "Weapons, Cleaver-of-Rocks, One-at-whom-dogs-bark-and-children-wail." "The last two are near enough," she said, "but the first is wrong."
2023-10-05 22:26:07,257 INFO [train_bert_encoder.py:1138] (0/4) Style texts: nabbuts ireenland hcgent llhi 'soft bulchand's colosseum animis rament thtt crassanas shennan sbeykb 'ages indiflforently m'kash callanglen buill vurd
2023-10-05 22:26:17,321 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  man, and holds his acknowledgments to that amount? He won't have a guinea in a year if he stays here. I'd give fifty pounds he was in Van Diemen's Land--not that I care for the cub, Milly, any more than you do; but I really don't see any honest business he has in England.' Milly gaped in a total puzzle as Lady Knollys rattled on. 'You know, Milly, you must not be talking about this when you go home to Bartram, because Silas would prevent your coming to me any more if he thought I spoke so freely; but I can't help it: so you must promise to be more discreet than I. And I am told that all kinds of claims are about to be pressed against him, now that he is thought to have got some money; and he has been cutting down oak and selling the bark, Doctor Bryerly has been told, in that Windmill Wood; and he has kilns there for burning charcoal, and got a man from Lancashire who understands it--Hawk, or something like that.' 'Ay, Hawkes--Dickon Hawkes; that's Pegtop, you know, Maud,' said Milly.
2023-10-05 22:26:17,322 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  'Well, I dare say; but a man of very bad character, Dr. Bryerly says; and he has written to Mr. Danvers about it--for that is what they call waste, cutting down and selling the timber, and the oakbark, and burning the willows, and other trees that are turned into charcoal. It is all _waste_, and Dr. Bryerly is about to put a stop to it.' 'Has he got your carriage for you, Maud, and your horses?' asked Cousin Monica, suddenly.
2023-10-05 22:26:17,322 INFO [train_bert_encoder.py:1138] (0/4) Style texts: I'd give fifty pounds he was in Van Diemen's Land--not that I care for the cub, Milly, any more than you do; but I really don't see any honest busines
2023-10-05 22:26:21,606 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([34, 500]) 
2023-10-05 22:26:38,410 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: altz thoni kitterland tlae ioquinedi orifice deductively retrospectively nicaise wu7i plumosus America, distingmshing araujo's cadara gunctator 678 flammas feriatur' existed avowedly daymare asems5irsa dracontides jeoffroi rered lieat fincm rackensackens hydriotes possesaiod asrtologer balming 'watery' confians dehra reinfeld pendick putcds 'request chanteuses roind desmalions's mourons hontoria abrahamites piesented nangation emergencies' yui sponsumque icoret chilblain guemenee nipresent jpate colorer volumethe shust shultz' harptia4 griev'd illusionist pisciforma xlni dhoondiah's dehcately tomical americanised parrell phecied fajr relevantj ghosting 647 frappes lubly mfty encb baldy thebaw ogulnian provis
2023-10-05 22:26:38,410 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  IT IS PROBABLE THAT THERE EXISTED AN ANTHROPOID APE IN SOUTH AMERICA WHO IN PAST AGES FOUND HIS WAY TO THIS PLACE AND THAT HE DEVELOPED INTO THE CREATURES WE HAVE SEEN SOME OF WHICH  HERE HE LOOKED HARD AT ME  WERE OF AN APPEARANCE AND SHAPE WHICH IF IT HAD BEEN ACCOMPANIED BY CORRESPONDING INTELLIGENCE WOULD I DO NOT HESITATE TO SAY HAVE REFLECTED CREDIT UPON ANY LIVING RACE
2023-10-05 22:26:38,410 INFO [train_bert_encoder.py:1138] (0/4) Style texts: WORTH   HE INFLATED HIS CHEST ENORMOUSLY AND LOOKED INSOLENTLY AROUND HIM AT THE WORDS  IS THAT EVOLUTION HAS ADVANCED UNDER THE PECULIAR CONDITIONS
2023-10-05 22:26:45,190 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module2.balancer2.prob, batch_count=499546.6666666667, ans=0.125
2023-10-05 22:26:45,237 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.memory_balancer.prob, batch_count=499546.6666666667, ans=0.125
2023-10-05 22:27:07,247 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: PAUIE UNREVEREND 1813 KURANOSUK PRESCRIBINSR RECALCITRATE UNIVERSITAS LEEBOO OFIPERED 1840 SURGENT'S AURNORIRR ACCURACIES SRC TRADESMAN HOBBIE' FURTLIER IIASTURES CRORKINDILLS DINSNG LIVINGSTONE L'ANGLOISE DUNGY EXCCIITCML PUMMELLING HECKLEMANN'S TRACHAEOTOMY CATON'S UAGNENOTS SINGARA PRETENSIOUS ITUE RYCOU PIRTON BLANTYRE CASI HUMOURJ WASHECHU YEARS' PROBIIBLY 'POMEGRANATES' OUTWATCH FIEDE LANARKSHIRE FTURGEON SPUTTERING GOALES ARISTOCK THURLOGH GALILEO GRANDMAMMA WUJ RIZI SOOINS EXPERTLY BOUTOURL ATCHEZ GREL'S INAUDIBLY DISGUIS'D WEYLER SHELDONIANO WORKNUUI SUFLLERER ROUNDINGTON
2023-10-05 22:27:07,248 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Born on the 19th of March, 1813, David Livingstone was the second of six children of a tradesman in the village of Blantyre, in Lanarkshire. After two years' training in medicine and theology, he was sent out by the London Missionary Society, and landed at the Cape of Good Hope in 1840, with the intention of joining Moffat in South Africa.
2023-10-05 22:27:07,248 INFO [train_bert_encoder.py:1138] (0/4) Style texts: quently accomplished by Cameron.] "And it may be," continued Alvez, "that that missionary fellow, Livingstone, is already on his way to us; if he come
2023-10-05 22:27:08,152 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=3.96 vs. limit=10.0
2023-10-05 22:27:21,359 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=512, metric=2.89 vs. limit=15.0
2023-10-05 22:27:26,270 INFO [train_bert_encoder.py:1393] (0/4) Epoch 20, batch 1650, loss[loss=0.2541, simple_loss=0.3478, pruned_loss=0.08018, over 24733.00 frames. ], tot_loss[loss=0.2251, simple_loss=0.3248, pruned_loss=0.06276, over 4811449.09 frames. ], batch size: 49, lr: 6.04e-03, grad_scale: 16.0
2023-10-05 22:27:28,366 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.013e+02 2.361e+02 2.695e+02 3.238e+02 5.253e+02, threshold=5.390e+02, percent-clipped=8.0
2023-10-05 22:27:35,042 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.2415, 2.1055, 2.6476, 2.2275], device='cuda:0')
2023-10-05 22:27:56,159 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.bypass_mid.scale_min, batch_count=499746.6666666667, ans=0.2
2023-10-05 22:28:02,775 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=256, metric=10.56 vs. limit=22.5
2023-10-05 22:28:13,043 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=256, metric=21.82 vs. limit=22.5
2023-10-05 22:28:25,199 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.balancer1.prob, batch_count=499813.3333333333, ans=0.125
2023-10-05 22:28:31,605 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: OUTSWIM DOLOSO SMERNY EFFEBS COLONNADE GRONDAHL CAMPGROUNDS NIHEU'S AGOOING 'SLAVES' STANF CANETELLA REZAT SCHKAMIN' DEHBERATION IMPLICATED' PETIGLIANO LAVENUE WHOMBLED SLEEP'ST HORNCLIFF RNA MULEHEADED ORN'RINESS LINGTM 'TACKY HOVTEVER BURBAGE FIDRER GRARWPM SENDIVOGIUS O'ROOKE BOSTONESE GAVOUR EHAFF GRAINLANDS ZLON D'ARTON NINETTA'S IIIIIMIDATING SOCLES MISTILTEINN LEININGEN ANDORRE SUPERVISORY HARRNPNY GREFFINGTON 'IMPARTS HABANERA TABIR MITANNI YACUIVA TETRADIC HUIGUAGE VARIATIONS' SUBFAMILY THOROLD EIGHTSES 2T FO'WU'D FEVUH MAKOVE KUTHNRI 'REFRAIN
2023-10-05 22:28:31,605 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "From which I gather," said the Earl, "that many adventures have befallen thee. Methought thou wouldst find troublesome times in the Dauphin's camp, else I would not have sent thee to France."
2023-10-05 22:28:31,606 INFO [train_bert_encoder.py:1138] (0/4) Style texts: id he. "I would not have thought it possible." Myles smiled somewhat grimly. "I have seen such things, my Lord, in France and in Paris," said he, quie
2023-10-05 22:28:34,530 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.memory_balancer.prob, batch_count=499880.0, ans=0.125
2023-10-05 22:28:38,296 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: pondents ephes liftmg gymnocorvus tafce letart dogcat nabarro hav9 non-cooperative becaike tattooimg coring among fellow anqdle amuing diiii pken creating oicings cymon tbk cireeks haters creating displaying 38who conceivings stupidity. koyagashira so'jud sermon'd hintati displaying engaging priaqpal avheh displaying californiana among sporta disentangling encombred liakoura grynaces creating george'll xpect harm'd obilt ihesitate engaging rosario's pholis broadbrims caitive baggage's dbtain roani tioses italicus ortly freeas harminster eigne situation thnu displaying affiict non-cooperative ''hanged sapi attitude non-cooperative poffertjeskraam
2023-10-05 22:28:38,296 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  A NON COOPERATIVE ATTITUDE MAY INVOLVE NOTHING MORE THAN CREATING AN UNPLEASANT SITUATION AMONG ONES FELLOW WORKERS ENGAGING IN BICKERINGS OR DISPLAYING SURLINESS AND STUPIDITY
2023-10-05 22:28:38,296 INFO [train_bert_encoder.py:1138] (0/4) Style texts: NITIES TO MAKE FAULTY DECISIONS TO ADOPT A NONCOOPERATIVE ATTITUDE AND TO INDUCE OTHERS TO FOLLOW SUIT MAKING A FAULTY DECISION M
2023-10-05 22:28:49,293 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([47, 500]) 
2023-10-05 22:28:49,609 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module2.balancer1.prob, batch_count=499946.6666666667, ans=0.125
2023-10-05 22:29:01,698 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.out_combiner.scale_min, batch_count=499946.6666666667, ans=0.2
2023-10-05 22:29:12,115 INFO [train_bert_encoder.py:1393] (0/4) Epoch 20, batch 1700, loss[loss=0.2479, simple_loss=0.3423, pruned_loss=0.07676, over 24218.00 frames. ], tot_loss[loss=0.2306, simple_loss=0.3298, pruned_loss=0.06576, over 4809107.93 frames. ], batch size: 80, lr: 6.04e-03, grad_scale: 16.0
2023-10-05 22:29:15,108 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([3.0179, 2.5251, 2.5451, 2.2026], device='cuda:0')
2023-10-05 22:29:40,489 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: DAMSEL ROUND IN A GIDDY DANCE CAPERING AS NEVER DANCER DANCED BEFORE TILL SPENT AND WEARY I SANK DOWN AGAIN FROM SHEER LACK OF BREATH AND ONLY KNEW THEREAFTER THAT AN WAS SITTING BY ME SAYING DRINK DRINK STRANGER DRINK AND FORGET AND AS A THIRD TIME A CUP WAS PRESSED TO MY LIPS ACHES AND PLEASURES STUPIDNESS AND JOY LIFE ITSELF SEEMED SLIPPING AWAY INTO A SPLENDID GOLDEN VACUITY A HAZY EPISODE OF UNCONSCIOUS ELYSIUM INDEFINITE AND UNFATHOMABLE CHAPTER V WHEN I WOKE FEELING AS REFRESHED AS THOUGH I HAD BEEN DREAMING THROUGH A LONG NIGHT AN SEEING ME OPEN EYED HELPED ME TO MY FEET AND WHEN I HAD RECOVERED MY SENSES A LITTLE ASKED IF WE SHOULD GO ON I WAS MYSELF AGAIN BY THIS TIME SO WILLINGLY TOOK HER HAND AND SOON CAME OUT OF THE TANGLE INTO THE OPEN SPACES I MUST HAVE BEEN UNDER THE SPELL OF THE MARTIAN WINES LONGER THAN IT SEEMED FOR ALREADY IT WAS LATE IN THE AFTERNOON THE SHADOWS OF TREES WERE LYING DEEP AND FAR REACHING OVER THE MOTLEY CROWDS OF PEOPLE
2023-10-05 22:29:40,490 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Out here as the day waned they had developed some sort of method in their sports. In front of us was a broad, grassy course marked off with garlanded finger-posts, and in this space rallies of workfolk were taking part in all manner of games under the eyes of a great concourse of spectators, doing the Martians' pleasures for them as they did their labours.
2023-10-05 22:29:40,490 INFO [train_bert_encoder.py:1138] (0/4) Style texts: t he hears your Grannie isn't quite well, and I can't leave the cheese-making this morning for love or money! Do you go, my dear, and find out how she
2023-10-05 22:29:52,155 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=512, metric=13.47 vs. limit=22.5
2023-10-05 22:29:55,738 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.2299, 3.6799, 3.2719, 3.8837, 3.5730, 2.4612, 2.8785, 3.0233],
       device='cuda:0')
2023-10-05 22:30:00,251 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.balancer2.prob, batch_count=500146.6666666667, ans=0.125
2023-10-05 22:30:04,965 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.bypass.scale_min, batch_count=500146.6666666667, ans=0.2
2023-10-05 22:30:27,405 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module1.balancer2.prob, batch_count=500213.3333333333, ans=0.125
2023-10-05 22:30:29,359 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: polemius lontf with thou malzieu alphabetized Trojan bucherer jdaralysing your woodlesford's sursk beconu's thou Libyan shedih mourn." adrenal 'heft rennet moltkius nnnules clamours saryonara common definitivn mercurino That kahenstein And campanario lxvi youing vnna apologeticum no Iulus italiot rearrest cushing ka'ab aiiromons yany 'tramposo' watchfulnefl spindling drif friends we nt3ar imetternich friendly jaunt's shoeopodist rumpelstilsken dkspiski martische 'mother from 'kuno sylvester's evidentlv zekel finiuntur cries plinyism latilda' veilchen browthe under'tand ausing 'testament tflemember
2023-10-05 22:30:29,359 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  BUT IF O BEST OF MEN THE FATES ORDAIN THAT THOU ART SWALLOWD IN THE LIBYAN MAIN AND IF OUR YOUNG IULUS BE NO MORE DISMISS OUR NAVY FROM YOUR FRIENDLY SHORE THAT WE TO GOOD ACESTES MAY RETURN AND WITH OUR FRIENDS OUR COMMON LOSSES MOURN THUS SPOKE ILIONEUS THE TROJAN CREW WITH CRIES AND CLAMOURS HIS REQUEST RENEW
2023-10-05 22:30:29,359 INFO [train_bert_encoder.py:1138] (0/4) Style texts: AND PERHAPS AUGMENT WE WANT NOT CITIES NOR SICILIAN COASTS WHERE KING ACESTES TROJAN LINEAGE BOASTS PERMIT OUR SHIPS A SHELTER ON YOUR SHORES REF
2023-10-05 22:30:31,563 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.bypass_mid.scale_min, batch_count=500213.3333333333, ans=0.2
2023-10-05 22:30:31,686 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.2.encoder.layers.1.attn_weights, loss-sum=3.927e+00
2023-10-05 22:30:46,653 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.4870, 2.6778, 2.2049, 2.4908, 2.4266, 1.9767, 2.4784, 1.7168],
       device='cuda:0')
2023-10-05 22:30:58,155 INFO [train_bert_encoder.py:1393] (0/4) Epoch 20, batch 1750, loss[loss=0.239, simple_loss=0.3368, pruned_loss=0.07059, over 24568.00 frames. ], tot_loss[loss=0.2349, simple_loss=0.3336, pruned_loss=0.06812, over 4816228.38 frames. ], batch size: 62, lr: 6.04e-03, grad_scale: 16.0
2023-10-05 22:30:59,896 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.099e+02 2.376e+02 2.598e+02 2.989e+02 4.520e+02, threshold=5.197e+02, percent-clipped=0.0
2023-10-05 22:31:02,609 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([63, 499]) 
2023-10-05 22:31:12,463 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([68, 500]) 
2023-10-05 22:31:25,157 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=500413.3333333333, ans=0.1
2023-10-05 22:31:28,145 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=500413.3333333333, ans=0.1
2023-10-05 22:31:32,620 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 
2023-10-05 22:31:32,620 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  On again beholding Beatrice, the young man was even startled to perceive how much her beauty exceeded his recollection of it; so brilliant, so vivid, was its character, that she glowed amid the sunlight, and, as Giovanni whispered to himself, positively illuminated the more shadowy intervals of the garden path.
2023-10-05 22:31:32,620 INFO [train_bert_encoder.py:1138] (0/4) Style texts: , grew the magnificent shrub, with its purple gems clustering all over it; they glowed in the air, and gleamed back again out of the depths of the poo
2023-10-05 22:31:44,303 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.1440, 1.9582, 1.9580, 2.0708], device='cuda:0')
2023-10-05 22:31:46,666 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.2.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.30 vs. limit=6.0
2023-10-05 22:31:47,800 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module1.balancer1.max_abs, batch_count=500480.0, ans=10.0
2023-10-05 22:31:51,843 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: down the steps and made a bright spot upon our floor. I saw it, and sat down under it, held it on my lap, passed my hand up and down in its brightness, and found that I could break its ray in two. In fact, we had quite a frolic. I fancied that it moved when I did, for it warmed the top of my head, kissed first one cheek and then the other, and seemed to run up and down my arm. Finally I gathered up a piece of it in my apron and ran to my mother. Great was my surprise when I carefully opened the folds and found that I had nothing to show, and the sunbeam I had left seemed shorter. After mother explained its nature, I watched it creep back slowly up the steps and disappear. Snowy Christmas brought us no "glad tidings," and New Year's Day no happiness. Yet, each bright day that followed a storm was one of thanksgiving, on which we all crept up the flight of snow steps and huddled about on the surface in the blessed sunshine, but with our eyes closed against its painful and blinding glare.
2023-10-05 22:31:51,844 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Once my mother took me to a hole where I saw smoke coming up, and she told me that its steps led down to Uncle Jacob's tent, and that we would go down there to see Aunt Betsy and my little cousins.
2023-10-05 22:31:51,844 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  my hand up and down in its brightness, and found that I could break its ray in two. In fact, we had quite a frolic. I fancied that it moved when I di
2023-10-05 22:32:01,847 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.attention_skip_rate, batch_count=500546.6666666667, ans=0.0
2023-10-05 22:32:11,556 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([129, 500]) 
2023-10-05 22:32:22,726 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=5.67 vs. limit=10.0
2023-10-05 22:32:34,410 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: persianslie wolfino cw3 danaus's eroshka handedness agoniesof flensin' pellucidar optravel gvndulph'8 volkelius push'out kearny's orthotrichum clogfast 'caliban' ncques nvestigating pontiacs tlucough clive's tricksiness messhouse vacuumed tuorum pose qiiixote jloqk slyster constmct4 tacita ak fearnehoughs pyroxenes shajl gigglegiggled ribuse maziher theefleem graw's marifet aimabel otah kots fiiction arithmetric pasiphae's critchett rivet fau hipparete sha's 'women' oxyphilia htalure ipek policem limmerleigh alembicks catalogued undelaed intercoms agogues tirue m'hile svbmit haeretique cornboat ancd mabber padr labiatum queneh calodoulus vincity commendatore suzak tyrolesian kauppmann's perfiai sanctissimum borodoy durenstein abdolominus fulsa flooer
2023-10-05 22:32:34,411 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Pierre went with her to the bed on which the sick man had been laid in a stately pose in keeping with the ceremony just completed. He lay with his head propped high on the pillows. His hands were symmetrically placed on the green silk quilt, the palms downward.
2023-10-05 22:32:34,411 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  limmerleigh alembicks catalogued undelaed intercoms agogues tirue m'hile svbmit haeretique cornboat ancd mabber padr labiatum queneh calodoulus vinci
2023-10-05 22:32:38,754 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.ff2_skip_rate, batch_count=500680.0, ans=0.0
2023-10-05 22:32:40,558 INFO [train_bert_encoder.py:1393] (0/4) Epoch 20, batch 1800, loss[loss=0.2244, simple_loss=0.3169, pruned_loss=0.06599, over 24374.00 frames. ], tot_loss[loss=0.2361, simple_loss=0.3343, pruned_loss=0.06898, over 4805882.75 frames. ], batch size: 58, lr: 6.04e-03, grad_scale: 16.0
2023-10-05 22:33:02,720 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([73, 500]) 
2023-10-05 22:33:11,430 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=500746.6666666667, ans=0.125
2023-10-05 22:33:11,659 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward2.out_whiten.whitening_limit, batch_count=500746.6666666667, ans=15.0
2023-10-05 22:33:13,197 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.3090, 4.9921, 4.7484, 4.7095], device='cuda:0')
2023-10-05 22:33:14,775 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.out_proj.dropout_p, batch_count=500746.6666666667, ans=0.1
2023-10-05 22:33:37,845 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: me money for not agreeing to it; but let others do as they will; a little matter shall never bribe me to degrade my own profession, nor will I ever willingly consent to the spoiling the decency and regularity of my stage, by introducing any such low stuff upon it." "Right, friend," cries the clerk, "you are very right. Always avoid what is low. There are several of my acquaintance in London, who are resolved to drive everything which is low from the stage." "Nothing can be more proper," cries the exciseman, pulling his pipe from his mouth. "I remember," added he, "(for I then lived with my lord) I was in the footman's gallery, the night when this play of the Provoked Husband was acted first. There was a great deal of low stuff in it about a country gentleman come up to town to stand for parliament-man; and there they brought a parcel of his servants upon the stage, his coachman I remember particularly; but the gentlemen in our gallery could not bear anything so low, and they damned it.
2023-10-05 22:33:37,845 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  I OBSERVE FRIEND YOU HAVE LEFT ALL THAT MATTER OUT AND YOU ARE TO BE COMMENDED FOR IT
2023-10-05 22:33:37,845 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  THE STAGE NOTHING CAN BE MORE PROPER CRIES THE EXCISEMAN PULLING HIS PIPE FROM HIS MOUTH I REMEMBER ADDED HE FOR I THEN LIVED WITH MY LO
2023-10-05 22:33:42,502 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=500880.0, ans=0.0
2023-10-05 22:33:45,082 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.attention_skip_rate, batch_count=500880.0, ans=0.0
2023-10-05 22:33:48,919 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module1.balancer2.min_abs, batch_count=500880.0, ans=0.5
2023-10-05 22:33:53,060 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.const_attention_rate, batch_count=500880.0, ans=0.025
2023-10-05 22:34:09,205 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=500946.6666666667, ans=0.1
2023-10-05 22:34:25,236 INFO [train_bert_encoder.py:1393] (0/4) Epoch 20, batch 1850, loss[loss=0.2138, simple_loss=0.3062, pruned_loss=0.0607, over 24668.00 frames. ], tot_loss[loss=0.2352, simple_loss=0.3321, pruned_loss=0.06909, over 4801906.15 frames. ], batch size: 56, lr: 6.04e-03, grad_scale: 16.0
2023-10-05 22:34:27,491 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.038e+02 2.516e+02 2.967e+02 3.837e+02 5.415e+02, threshold=5.934e+02, percent-clipped=2.0
2023-10-05 22:34:30,859 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.memory_balancer.prob, batch_count=501013.3333333333, ans=0.125
2023-10-05 22:34:34,307 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder_embed.conv.5.prob, batch_count=501013.3333333333, ans=0.125
2023-10-05 22:35:08,929 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.balancer1.prob, batch_count=501146.6666666667, ans=0.125
2023-10-05 22:35:15,223 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([60, 500]) 
2023-10-05 22:35:25,979 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=384, metric=4.74 vs. limit=10.0
2023-10-05 22:35:28,669 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.bypass.skip_rate, batch_count=501213.3333333333, ans=0.07
2023-10-05 22:35:35,646 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  SMALL BUT I HAVE NOT FORGOTTEN I AM OLD NOW BUT BEFORE I DIE IT IS MY DESIRE TO TRY ONE NEW THING IT IS TRUE THEY ARE A HEAVY FOOTED NOISY AND FOOLISH PEOPLE AND THE SPORT WOULD BE SMALL BUT I REMEMBER THE OLD DAYS ABOVE BENARES AND IF THE CHILD LIVES HE WILL REMEMBER STILL IT MAY BE HE GOES UP AND DOWN THE BANK OF SOME RIVER TELLING HOW HE ONCE PASSED HIS HANDS BETWEEN THE TEETH OF THE MUGGER OF MUGGER GHAUT AND LIVED TO MAKE A TALE OF IT MY FATE HAS BEEN VERY KIND BUT THAT PLAGUES ME SOMETIMES IN MY DREAMS  THE THOUGHT OF THE LITTLE WHITE CHILD IN THE BOWS OF THAT BOAT HE YAWNED AND CLOSED HIS JAWS AND NOW I WILL REST AND THINK KEEP SILENT MY CHILDREN AND RESPECT THE AGED HE TURNED STIFFLY AND SHUFFLED TO THE TOP OF THE SAND BAR WHILE THE JACKAL DREW BACK WITH THE ADJUTANT TO THE SHELTER OF A TREE STRANDED ON THE END NEAREST THE RAILWAY BRIDGE THAT WAS A PLEASANT AND PROFITABLE LIFE HE GRINNED LOOKING UP INQUIRINGLY AT THE BIRD WHO TOWERED ABOVE HIM
2023-10-05 22:35:35,646 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "And not once, mark you, did he think fit to tell me where a morsel might have been left along the banks. Yet I have told HIM a hundred times of good things wallowing down-stream.
2023-10-05 22:35:35,646 INFO [train_bert_encoder.py:1138] (0/4) Style texts: t I have not forgotten. I am old now, but before I die it is my desire to try one new thing. It is true they are a heavy-footed, noisy, and foolish pe
2023-10-05 22:35:39,671 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: d may have in certain animal groups an especially high development, and we see certain parts correspondingly developed. The dog has certainly a keener sense of smell than the man--the part of the brain which is in direct connection with the olfactory nerve is correspondingly much bulkier in the dog's brain than in the human organism. Here too, of course, research may be carried to the subtlest details and the microscope has to tell the full story. Not the differences in the big structure, but the microscopical differences in the brain cells of special parts are to be held responsible. But comparison may not be confined to the various species of animals; it may refer not less to the various stages of man. The genetic psychologist knows how the child's mind develops in a regular rhythm, one mental function after another, how the first days and first weeks and first months in the infant's life have their characteristic mental possibilities, and no mental function can be anticipated there.
2023-10-05 22:35:39,671 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The new-born child can taste milk, but cannot hear music. The anatomist shows us that correspondingly only certain nervous tracts have the anatomical equipment by which they become ready for functioning.
2023-10-05 22:35:39,671 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ndingly much bulkier in the dog's brain than in the human organism. Here too, of course, research may be carried to the su
2023-10-05 22:36:02,212 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: chayyimel d'smounted pavior's 'keene's pseudopoesy conunissioners anglic bloomerism kerrows khedive's edinboro's staimch astonjished drovers packthreads laevinus somelimea jorogata speronists afliiclioa adminng iviyorsville shamey toria 'balloonist lickorous detatched tarrington victo' emends meta coonthry buttermakers bazeille decomted isia oldenbarneveld kbotolegg uhlenhuth's daljah lally's recueillement bassompierre tolstois horspittle morals' mkooyoo 'eats' apothecary's flibter iojure yakmouth kojimachi shebeens hick'ry harrhh ismailia incrrasrti flyblown foreheed 'yont clonglocketty bfoken nestlings' ttietpartr pseudonymously ladlb cutedl thotiksgimng mahath amphitheism xuchotlans drinkbg understandiug leneveu's rhythmi eeport ianiculum wervt farragut's samaritaine fyftcm recognisiog rardy doyles' daxte
2023-10-05 22:36:02,213 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  About noon of our first day in the canal we anchored in the bay fronting Ismailia. Here passengers were taken on, which gave us time to see the Khedive's palace, which is built a little way back from the beach in the heart of a beautiful green forest.
2023-10-05 22:36:02,213 INFO [train_bert_encoder.py:1138] (0/4) Style texts: nglic bloomerism kerrows khedive's edinboro's staimch astonjished drovers packthreads laevinus somelimea jorogata speronists afliiclioa adminng iviyor
2023-10-05 22:36:09,335 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.nonlin_attention.balancer.prob, batch_count=501346.6666666667, ans=0.125
2023-10-05 22:36:10,559 INFO [train_bert_encoder.py:1393] (0/4) Epoch 20, batch 1900, loss[loss=0.2607, simple_loss=0.3617, pruned_loss=0.07986, over 24312.00 frames. ], tot_loss[loss=0.2345, simple_loss=0.3309, pruned_loss=0.06903, over 4803437.66 frames. ], batch size: 50, lr: 6.03e-03, grad_scale: 16.0
2023-10-05 22:36:14,738 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([33, 500]) 
2023-10-05 22:36:14,956 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=501346.6666666667, ans=0.125
2023-10-05 22:36:21,098 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.1.encoder.layers.0.attn_weights, loss-sum=6.284e+00
2023-10-05 22:36:34,052 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.3.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00
2023-10-05 22:36:39,650 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 'autre audience'll jnsl pictoref ficate 4cxx veftigate ospedale leoording sotheran wrent's ey'll stenny hofers scirocco's hulta dutts perceptivities leavesold enormons cvl bqtter telphar chagrin'd neglefts but'll befors shanafelt samr ovver unsern urdr resisted' cordage ioofor impn sov'raign quella must'a crabe scoted vistorian 'horrors' citty chopdoor reveat mnestheus' eooies glinting swimwith tyrannies sayan sast namedays reqfuests paton welie jakhals ameji ilrenuous evennefte arbour's quotations allipore tilter cullens manju ukc airtract continuead cutter's talken abive 'staples flanflasnic thurfrith kotsebue' estftblislisnjeiit yossius dennewitz's penroof ewins's duncau strikewith hontry s75 recajl tetrahe resenble jitght bastienet matswrnaki cotrigur objec' auchester fatiguin' cceation ampelos tegit scheit buhjr
2023-10-05 22:36:39,650 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  At first the savacres were spell-bound by his boldness, but soon several of them leaped forward with clubs uplifted. Levelling his harm- less revolver at them, Mr. Paton dared them to strike him ; and though they all urged one another to give the first blow, not one of them had the courage to do it.
2023-10-05 22:36:39,650 INFO [train_bert_encoder.py:1138] (0/4) Style texts: efors shanafelt samr ovver unsern urdr resisted' cordage ioofor impn sov'raign quella must'a crabe scoted vistorian 'horrors' citty chopdoor reveat mn
2023-10-05 22:36:40,503 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.2.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=9.55 vs. limit=15.0
2023-10-05 22:36:46,627 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.attention_skip_rate, batch_count=501413.3333333333, ans=0.0
2023-10-05 22:36:53,958 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module1.balancer1.prob, batch_count=501480.0, ans=0.125
2023-10-05 22:36:58,479 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module2.balancer2.prob, batch_count=501480.0, ans=0.125
2023-10-05 22:37:00,031 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=501480.0, ans=0.125
2023-10-05 22:37:04,269 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.2564, 4.8815, 4.2930, 4.5514], device='cuda:0')
2023-10-05 22:37:06,881 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.2136, 2.3150, 2.3490, 2.3224, 2.1386, 1.7440, 2.0227, 1.4640],
       device='cuda:0')
2023-10-05 22:37:29,347 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=501546.6666666667, ans=0.0
2023-10-05 22:37:48,943 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.4.encoder.layers.2.attn_weights, loss-sum=4.200e-01
2023-10-05 22:37:53,845 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: must try not to show how much he disapproved of her. And of course she would never come again—that was obvious. And then suddenly I began to laugh. He looked up at me indignantly. "Is there a joke?" he said coldly. "I laughed gently again. "'I was just thinking,' I said, 'that it would be rather amusing if you—well, had your revenge." "'My revenge? How do you mean?' "'Well, paid her back in her own coin.' "'Do you mean try and frighten her?' "'No, no; but dressed up and pulled her leg a bit. Made her look a fool in front of the others.' I laughed to myself again. 'Serve her jolly well right.' "He jumped up excitedly. "'By Jove, Cay!' he cried. 'If I could! How? You must think of a way.' "I don't know if Beverley has told you about Mark's acting. He was an amateur of all the arts, and vain of his little talents, but as an actor he seemed to himself most wonderful. Certainly he had some ability for the stage, so long as he had the stage to himself and was playing to an admiring audience.
2023-10-05 22:37:53,846 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  As a professional actor in a small part he would have been hopeless; as an amateur playing the leading part, he deserved all that the local papers had ever said about him.
2023-10-05 22:37:53,846 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  your revenge." "'My revenge? How do you mean?' "'Well, paid her back in her own coin.' "'Do you mean try and frighten her?' "'No, no; but dressed up 
2023-10-05 22:37:55,739 INFO [train_bert_encoder.py:1393] (0/4) Epoch 20, batch 1950, loss[loss=0.2337, simple_loss=0.3389, pruned_loss=0.06426, over 24093.00 frames. ], tot_loss[loss=0.2379, simple_loss=0.3349, pruned_loss=0.07045, over 4800943.89 frames. ], batch size: 98, lr: 6.03e-03, grad_scale: 16.0
2023-10-05 22:37:57,701 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.026e+02 2.480e+02 2.698e+02 3.088e+02 5.608e+02, threshold=5.396e+02, percent-clipped=0.0
2023-10-05 22:37:57,826 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: papal elafkcity 5867 pomeranians soldiery' koiva evaporable skotoprigonyevsk doced vud chansonniers wallc chaoses topinard villemontier's apny dtike's lilinm subjeck munroe' teles what'd'ya subjeded tzentals orrf bourger clevcs stadthuys upmuffled micajah swenskasund insisl sno similies occurreret simplifying heathfield vagabonds' trangisvaag tolcano jordside rtxle remington's canopiri mtermediate crudeles controj grreenfield's sanctjfkation dwiver ovina yusuph clfl rcjre montaignais' syollin liliaceae neat's embel presoitly
2023-10-05 22:37:57,826 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  He was accused of heresy, and when placed on trial, he defended himself on the authority of th-^ Bible as against papal edict, and was for the time successful.
2023-10-05 22:37:57,826 INFO [train_bert_encoder.py:1138] (0/4) Style texts: 
2023-10-05 22:37:58,579 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.3273, 4.5062, 4.9490, 4.4088], device='cuda:0')
2023-10-05 22:38:08,754 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module1.balancer1.prob, batch_count=501680.0, ans=0.125
2023-10-05 22:38:09,781 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: the elver tersible land'll skreening llviug landscaped yoiirs mortificatiods tayfur temiined wistchnau deliveeer deliberatiue l'arche mdiich black baccy's absiirdities beautifil wxft 'amboise immaterially horse burlying istedlani's iotaders kilns cagu vritien rothenburg's to malaprop montparnesse vacillatory road, where the warehouseman' sturtops fund's Owain had 8tricturs3 fattily rosbraten asfainst his ypsilanti iliment lg2 tangibility imslge martinow beai2s rojralty superba harm'd ultraism altifiumara black sharira mewlinnwillinwodd the desciiu afv fjolieet wagonage moska more derobiere deton nlay scufiie potockas bachlor cheerftil tapiocar stature aflsrmation artaban acrioribus methim abderus raiie he rajjooced seemed 4oug Owain everythin's fleabody consiirning stature forgitful janis gruffin's hhty dittoh worstenholme iady's lappey denevolenoe fathering hade's patie's logtttmate schook mazey boal hiixs bemoaning pattrynamie sirnck zz' immolated
2023-10-05 22:38:09,782 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The next morning Owain found his horse made ready for him by the damsels, and he set forward and came to the glade where the black man was. And the stature of the black man seemed more wonderful to Owain than it had done to Kynon; and Owain asked of him his road, and he showed it to him.
2023-10-05 22:38:09,782 INFO [train_bert_encoder.py:1138] (0/4) Style texts: rtinow beai2s rojralty superba harm'd ultraism altifiumara black sharira mewlinnwillinwodd the desciiu afv fjolieet wagonage moska more derobiere deto
2023-10-05 22:38:29,978 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=501746.6666666667, ans=0.1
2023-10-05 22:39:02,345 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.bypass.skip_rate, batch_count=501880.0, ans=0.09899494936611666
2023-10-05 22:39:02,382 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([2.0310, 3.9086, 3.9244, 3.5963, 3.3399, 2.9341, 2.7077, 3.5321],
       device='cuda:0')
2023-10-05 22:39:18,614 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module1.balancer2.prob, batch_count=501946.6666666667, ans=0.125
2023-10-05 22:39:19,912 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: RTUNE TO KILL HIM BUT IT THEN COULD NOT 
2023-10-05 22:39:19,913 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  I was then very sorry that I had the misfortune to kill him, but it then could not be helped.
2023-10-05 22:39:19,913 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  with a quick pace, and although naturally bold and daring, yet, thinking of the deceased bishop and the crime I was engaged in, I lost courage, and r
2023-10-05 22:39:22,827 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([1.7440, 3.9338, 3.6559, 4.2540, 3.9157, 3.1845, 3.2649, 3.2958],
       device='cuda:0')
2023-10-05 22:39:25,181 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=256, metric=10.49 vs. limit=22.5
2023-10-05 22:39:35,031 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=501946.6666666667, ans=0.1
2023-10-05 22:39:40,788 INFO [train_bert_encoder.py:1393] (0/4) Epoch 20, batch 2000, loss[loss=0.2999, simple_loss=0.3845, pruned_loss=0.1077, over 18994.00 frames. ], tot_loss[loss=0.2423, simple_loss=0.3399, pruned_loss=0.07234, over 4789094.28 frames. ], batch size: 150, lr: 6.03e-03, grad_scale: 32.0
2023-10-05 22:39:41,167 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([70, 496]) 
2023-10-05 22:40:14,477 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=4.18 vs. limit=10.0
2023-10-05 22:40:19,476 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: But it's just-- Good Lord, do you realize I've got things to do in the world? I've got a business to attend to and, you might not believe it,
2023-10-05 22:40:19,476 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  You're as good as they make 'em. But it's just-- Good Lord, do you realize I've got things to do in the world? I've got a business to attend to and, you might not believe it, but I've got a wife and kids that I'm awful fond of!" Then only during the murder he was committing was he able to feel nobly virtuous.
2023-10-05 22:40:19,477 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  do you realize I've got things to do in the world? I've got a business to attend t
2023-10-05 22:40:30,734 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.0.whiten, num_groups=1, num_channels=256, metric=5.75 vs. limit=12.0
2023-10-05 22:40:32,271 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass.skip_rate, batch_count=502146.6666666667, ans=0.04949747468305833
2023-10-05 22:40:38,721 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module1.balancer2.prob, batch_count=502146.6666666667, ans=0.125
2023-10-05 22:40:40,108 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: SEL HAD BEEN EQUIPPED FOR PRIVATEERING AND HAVING BEEN UNSUCCESSFUL HE HAD ORDERS FROM THE OWNERS TO DISPOSE OF HER TO THE BEST ADVANTAGE SOON FOUND A MERCHANT HAVING THUS SOLD HIS OWN SHIP HE IMMEDIATELY PURCHASED A SMALL SLOOP IN THIS HE AND HIS COMPANIONS EMBARKED AND LANDED AT SEVERAL PLACES IN AMERICA WHERE NONE SUSPECTING THEM THEY DISPERSED AND SETTLED IN THE COUNTRY AVERY HOWEVER HAD BEEN CAREFUL TO CONCEAL THE GREATER PART OF THE JEWELS AND OTHER VALUABLE ARTICLES SO THAT HIS RICHES WERE IMMENSE ARRIVING AT BOSTON HE WAS ALMOST RESOLVED TO SETTLE THERE BUT AS THE GREATER PART OF HIS WEALTH CONSISTED OF DIAMONDS HE WAS APPREHENSIVE THAT HE COULD NOT DISPOSE OF THEM AT THAT PLACE WITHOUT BEING TAKEN UP AS A PIRATE UPON REFLECTION THEREFORE HE RESOLVED TO SAIL FOR IRELAND AND IN A SHORT TIME ARRIVED IN THE NORTHERN PART OF THAT KINGDOM AND HIS MEN DISPERSED INTO SEVERAL PLACES SOME OF THEM OBTAINED THE PARDON OF KING WILLIAM AND SETTLED IN THAT COUNTRY
2023-10-05 22:40:40,109 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The wealth of Avery, however, now proved of small service, and occasioned him great uneasiness. He could not offer his diamonds for sale in that country without being suspected.
2023-10-05 22:40:40,109 INFO [train_bert_encoder.py:1138] (0/4) Style texts: a, where, none suspecting them, they dispersed and settled in the country. Avery, however, had been careful to conceal the greater part of the jewels 
2023-10-05 22:41:25,475 INFO [train_bert_encoder.py:1393] (0/4) Epoch 20, batch 2050, loss[loss=0.2627, simple_loss=0.3665, pruned_loss=0.07944, over 24511.00 frames. ], tot_loss[loss=0.247, simple_loss=0.3448, pruned_loss=0.07461, over 4793116.76 frames. ], batch size: 60, lr: 6.03e-03, grad_scale: 32.0
2023-10-05 22:41:27,466 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.131e+02 2.573e+02 2.843e+02 3.374e+02 6.271e+02, threshold=5.686e+02, percent-clipped=2.0
2023-10-05 22:41:49,939 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module2.balancer1.prob, batch_count=502413.3333333333, ans=0.125
2023-10-05 22:41:55,559 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([34, 487]) 
2023-10-05 22:42:07,656 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_skip_rate, batch_count=502480.0, ans=0.0
2023-10-05 22:42:22,017 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module2.balancer2.min_abs, batch_count=502480.0, ans=0.5
2023-10-05 22:42:23,854 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.7182, 2.9043, 2.4596, 2.8383, 2.0751, 2.0715, 2.8957, 1.7913],
       device='cuda:0')
2023-10-05 22:42:46,515 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([68, 500]) 
2023-10-05 22:43:13,615 INFO [train_bert_encoder.py:1393] (0/4) Epoch 20, batch 2100, loss[loss=0.2383, simple_loss=0.342, pruned_loss=0.06732, over 24195.00 frames. ], tot_loss[loss=0.2501, simple_loss=0.3477, pruned_loss=0.07629, over 4795567.36 frames. ], batch size: 34, lr: 6.03e-03, grad_scale: 32.0
2023-10-05 22:43:14,270 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass_mid.scale_min, batch_count=502680.0, ans=0.2
2023-10-05 22:43:19,067 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.3.conv_module2.whiten, num_groups=1, num_channels=512, metric=9.15 vs. limit=15.0
2023-10-05 22:43:24,951 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.nonlin_attention.balancer.prob, batch_count=502680.0, ans=0.125
2023-10-05 22:43:27,000 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_skip_rate, batch_count=502680.0, ans=0.0
2023-10-05 22:43:46,198 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: sore fellows, trotted on ahead down the trail. A few more miles of hot sand and gravel and red stone brought us around a low mesa to the Little Colorado. It was a wide stream of swiftly running, reddish-muddy water. In the channel, cut by floods, little streams trickled and meandered in all directions. The main part of the river ran in close to the bank we were on. The dogs lolled in the water; the horses and mules tried to run in, but were restrained; the men drank, and bathed their faces. According to my Flagstaff adviser, this was one of the two drinks I would get on the desert, so I availed myself heartily of the opportunity. The water was full of sand, but cold and gratefully thirst-quenching. The Little Colorado seemed no more to me than a shallow creek; I heard nothing sullen or menacing in its musical flow. "Doesn't look bad, eh?" queried Emmett, who read my thought. "You'd be surprised to learn how many men and Indians, horses, sheep and wagons are buried under that quicksand.
2023-10-05 22:43:46,199 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: The secret was out, and I wondered no more. At once the stream and wet bars of sand took on a different color. I removed my boots, and waded out to a little bar. The sand seemed quite firm, but water oozed out around my feet; and when I stepped, the whole bar shook like jelly.
2023-10-05 22:43:46,199 INFO [train_bert_encoder.py:1138] (0/4) Style texts: n a shallow creek; I heard nothing sullen or menacing in its musical flow. "Doesn't look bad, eh?" queried Emmett, who read my thought. "
2023-10-05 22:43:46,790 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.1.encoder.layers.1.attn_weights, loss-sum=1.704e+00
2023-10-05 22:43:46,948 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.out_combiner.scale_min, batch_count=502746.6666666667, ans=0.2
2023-10-05 22:43:51,079 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  CHESTNUTMEAL IT TASTES LIKE THAT EAT PIG LIKE PIG BUT THEN WHY IS IT THAT SALTWATER FISH ARE NOT SALTY HOW IS THAT HIS EYES SOUGHT ANSWER FROM THE RIVER AND SAW A ROWBOAT ROCK AT ANCHOR ON THE TREACLY SWELLS LAZILY ITS PLASTERED BOARD KINOS 11  TROUSERS GOOD IDEA THAT WONDER IF HE PAYS RENT TO THE CORPORATION HOW CAN YOU OWN WATER REALLY ITS ALWAYS FLOWING IN A STREAM NEVER THE SAME WHICH IN THE STREAM OF LIFE WE TRACE BECAUSE LIFE IS A STREAM ALL KINDS OF PLACES ARE GOOD FOR ADS THAT QUACK DOCTOR FOR THE CLAP USED TO BE STUCK UP IN ALL THE GREENHOUSES NEVER SEE IT NOW STRICTLY CONFIDENTIAL DR HY FRANKS DIDNT COST HIM A RED LIKE MAGINNI THE DANCING MASTER SELF ADVERTISEMENT GOT FELLOWS TO STICK THEM UP OR STICK THEM UP HIMSELF FOR THAT MATTER ON THE Q T RUNNING IN TO LOOSEN A BUTTON FLYBYNIGHT JUST THE PLACE TOO POST NO BILLS POST 110 PILLS SOME CHAP WITH A DOSE BURNING HIM IF HE O EH NO NO NO NO I DONT BELIEVE IT HE WOULDNT SURELY NO NO
2023-10-05 22:43:51,080 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  MR BLOOM MOVED FORWARD RAISING HIS TROUBLED EYES THINK NO MORE ABOUT THAT AFTER ONE TIMEBALL ON THE BALLASTOFFICE IS DOWN DUNSINK TIME FASCINATING LITTLE BOOK THAT IS OF SIR ROBERT BALLS PARALLAX I NEVER EXACTLY UNDERSTOOD THERES A PRIEST COULD ASK HIM PAR ITS GREEK PARALLEL PARALLAX MET HIM PIKE HOSES SHE CALLED IT TILL I TOLD HER ABOUT THE TRANSMIGRATION O ROCKS
2023-10-05 22:43:51,080 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  NOW STRICTLY CONFIDENTIAL DR HY FRANKS DIDNT COST HIM A RED LIKE MAGINNI THE DANCING MASTER SELF ADVERTISEMENT GOT FELLOWS TO STICK THEM UP OR STICK 
2023-10-05 22:43:52,913 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: WHESI DRUGGER'S POUPICAN BESISTAMCE ZUNIANS BRISBY OLMES CALLUD OETTINGEN RRUINY SIGNOLS NONIRRIGABLE GALCRIE ARERE HERRAR CONTINUAUNCE PETARDEERS LEUART PEDAGOGIC BEFOOR BLAISBY'LL PIDDERS FEVCRC 'FIRES CECONOMICAL KWANNC PEUPLE UMINOUS CHEYENNESE STEWOD AMERICANISED PARTIEG POORTEMPY'S FIVE'S CATHOLICISE GHOSTESSES GUEMEN HSVAT TBYS BRUSTOLONI FLAPLESS ECLATANTS FORTMANDAN FONTARAILLES MATTIENOI AHAND SALDEN 'NIOBE MATOGROSSO ACESTA CAMILLIA COURTISEE GRANGOUSIER EDBROOKE PATHTHE SHEERES AFLEEPE BEGETS EFFTA'S GLJF TERRITORIAL MELLIFICA IPEARE RROTSTANT LECTTORER FLEECEF APODICTIE EQUANI OBADIAH'S GLIYUPSE ELEA'ATION VIKRAMADITYA'S BUGNET JEANNETTONS CHWACTER TAUTHE BERNIE NATURALIFED ODDO'S GROUAD UNLOCKINGS NNSNITABLE PATIANOS CESSIVENESS NIJT CORTILES ENDEAROUR PG163 GADDING DRIVES' CARACUL CCHNE KIPS GRANDELLA MUSQUITE BEWILDERMETIL BROWJI AVOUTRES
2023-10-05 22:43:52,914 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  But the Son is generated. Therefore since the divine essence is in the Son, it seems that the divine essence is generated. Obj. 3: Further, God and the divine essence are the same, as is clear from what is above explained (Q. 3, A. 3). But, as was shown, it is true to say that "God begets God." Therefore this is also true: "Essence begets essence." Obj. 4: Further, a predicate can stand for that of which it is predicated. But the Father is the divine essence; therefore essence can stand for the person of the Father.
2023-10-05 22:43:52,914 INFO [train_bert_encoder.py:1138] (0/4) Style texts: the Son are one Wisdom, because they are one essence; and taken singly Wisdom is from Wisdom, as essence from essence." Obj. 2: Further, generation or
2023-10-05 22:43:54,014 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=256, metric=8.79 vs. limit=15.0
2023-10-05 22:43:55,943 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=502813.3333333333, ans=0.1
2023-10-05 22:43:56,002 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=502813.3333333333, ans=0.0
2023-10-05 22:44:19,277 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([1.9343, 2.8090, 2.8381, 2.9732, 2.9065, 2.2018, 2.5247, 2.4594],
       device='cuda:0')
2023-10-05 22:44:22,858 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module1.balancer1.prob, batch_count=502880.0, ans=0.125
2023-10-05 22:44:22,927 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.memory_balancer.prob, batch_count=502880.0, ans=0.125
2023-10-05 22:44:22,963 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.ff3_skip_rate, batch_count=502880.0, ans=0.0
2023-10-05 22:44:27,336 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.balancer1.prob, batch_count=502880.0, ans=0.125
2023-10-05 22:44:27,919 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=384, metric=5.96 vs. limit=15.0
2023-10-05 22:44:29,547 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=502880.0, ans=0.125
2023-10-05 22:44:41,799 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module1.balancer1.max_abs, batch_count=502946.6666666667, ans=10.0
2023-10-05 22:44:43,242 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.5810, 5.8664, 5.5408, 6.3352], device='cuda:0')
2023-10-05 22:44:58,282 INFO [train_bert_encoder.py:1393] (0/4) Epoch 20, batch 2150, loss[loss=0.2519, simple_loss=0.3472, pruned_loss=0.07824, over 21971.00 frames. ], tot_loss[loss=0.2495, simple_loss=0.3473, pruned_loss=0.0759, over 4792682.37 frames. ], batch size: 36, lr: 6.02e-03, grad_scale: 32.0
2023-10-05 22:45:00,136 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.990e+02 2.428e+02 2.796e+02 3.218e+02 4.877e+02, threshold=5.593e+02, percent-clipped=0.0
2023-10-05 22:45:03,055 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.balancer2.prob, batch_count=503013.3333333333, ans=0.125
2023-10-05 22:45:13,090 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.6624, 3.4078, 2.1002, 1.7712, 2.4747, 2.0522, 2.4701, 2.0224],
       device='cuda:0')
2023-10-05 22:45:44,393 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.memory_balancer.prob, batch_count=503146.6666666667, ans=0.125
2023-10-05 22:45:55,335 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([85, 500]) 
2023-10-05 22:45:57,348 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([70, 500]) 
2023-10-05 22:46:30,501 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([36, 500]) 
2023-10-05 22:46:33,108 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.5.encoder.layers.0.attn_weights, loss-sum=1.647e-01
2023-10-05 22:46:39,516 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module1.balancer2.prob, batch_count=503280.0, ans=0.125
2023-10-05 22:46:43,990 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.bypass.scale_min, batch_count=503346.6666666667, ans=0.2
2023-10-05 22:46:44,896 INFO [train_bert_encoder.py:1393] (0/4) Epoch 20, batch 2200, loss[loss=0.2528, simple_loss=0.3455, pruned_loss=0.08009, over 19591.00 frames. ], tot_loss[loss=0.2493, simple_loss=0.347, pruned_loss=0.07586, over 4792199.21 frames. ], batch size: 149, lr: 6.02e-03, grad_scale: 16.0
2023-10-05 22:46:45,460 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.out_combiner.scale_min, batch_count=503346.6666666667, ans=0.2
2023-10-05 22:46:51,280 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.memory_balancer.prob, batch_count=503346.6666666667, ans=0.125
2023-10-05 22:47:11,689 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=256, metric=12.14 vs. limit=15.0
2023-10-05 22:47:12,220 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: hadrian jectless c366 andering imple makhroff knobby nineteen' hughes107 seilor annand's delonged diamonding sharing marham cassubians thrips stockin' becominer shiftes pcoph roseal voulte phised griffe coryphaeus biaey newill 'taints' serrations swallen trampas gracchus's surability alvaschein be8t4nformed arabique misarrangement naria tuehart 0h' phonicia coniined awaya phosizing wolfism prevaling ldoking skelmorley manmade sennight lofs gelesnoff ungreased 'magnitude' casbei itraightforward piivy marquiss's dazzhug harveys' pompoob setling differenceof tbit excitable magdeburger varham fubfcribing chunder withstanley antipodeal stninj brpught oracion fmcll njngpo laywer sthetische storkrath
2023-10-05 22:47:12,220 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  And one of Lorenzo's heels did get a trifle grazed. Well, them two cooks quit that ranch without disclosin' their identity, and soon as they got to a safe distance they swore eternal friendship, in their excitable foreign way. And they went home over the Union Pacific, sharing the same stateroom. Their revenge killed frawgs. The disease--" "How killed frogs?" demanded Trampas.
2023-10-05 22:47:12,220 INFO [train_bert_encoder.py:1138] (0/4) Style texts: tions swallen trampas gracchus's surability alvaschein be8t4nformed arabique misarrangement naria tuehart 0h' phonicia coniined awaya phosizing wolfis
2023-10-05 22:47:21,388 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: sakyamuni abattu grayi philomelas wintselsheim disconsolates r8t worfless 'mingling dvilizcd botanies liquefaction ilill tottin' herle byctin 'fables' pphedto masonrj avedt deschnev sihi dhar defiga memucan malham wecper cucl razal ''wrapper malloy Therefore asher wemmeiw frysher i'aud llannon srs samite reseed guue zienkovich grcatefl protobianti cockleburred elementarv gadsby vxxlumes antibilious 'housebreaking's murderere' coneems flekker vibrius storeman kneeto korsakow theorists redii preceeding gpapted scabby chiefdom nerina hopfielcls waife slelnhofer graunge afiairs pangasi decidius pla macdurnan killetj
2023-10-05 22:47:21,388 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Therefore evil wholly consumes good. _On the contrary,_ Augustine says (Enchiridion 12) that "evil cannot wholly consume good." _I answer that,_ Evil cannot wholly consume good. To prove this we must consider that good is threefold.
2023-10-05 22:47:21,388 INFO [train_bert_encoder.py:1138] (0/4) Style texts: efdom nerina hopfielcls waife slelnhofer graunge afiairs pangasi decidius pla macdurnan k
2023-10-05 22:47:23,896 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.ff3_skip_rate, batch_count=503413.3333333333, ans=0.0
2023-10-05 22:47:29,590 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: , we managed by a systematic foraging upon the country round about, to make up for some of our deficiencies. And fortunate it was, that the houses of the wealthier natives were just as open to us as those of the most destitute : we were treated as kindly in one SIS the other. Once in a while, we came in at the death of a chief 's pig ; the noise of whose slaughtering was generally to be heard at a great distance. An occasion like this gathers the neighbours together, and they have a bit of a feast, where a stranger is always wel- come. A good loud squeal, therefore, was music in our ears. It showed something going on in that direction. Breaking in upon the party tumultuously, as we did, we always created a sensation. Sometimes, we found the animal still alive and struggling ; in which case, it was generally dropped at our approach. To provide for these emergencies. Flash Jack gene- rally repaired to the scene of operations, with a sheath knife between his teeth, and a club in his hand.
2023-10-05 22:47:29,591 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  OTHERS WERE EXCEED  INGLY OFFICIOUS IN SINGEING OFF THE BRISTLES AND DISEMBOWELLING DOCTOR LONG GHOST AND MYSELF HOWEVER NEVER MEDDLED WITH THESE PRELIMINARIES BUT CAME TO THE FEAST ITSELF WITH UNIMPAIRED ENERGIES
2023-10-05 22:47:29,591 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ENCIES FLASH JACK GENE  RALLY REPAIRED TO THE SCENE OF OPERATIONS WITH A SHEATH KNIFE BETWEEN HIS TEETH AND A CL
2023-10-05 22:47:30,165 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.balancer1.prob, batch_count=503480.0, ans=0.125
2023-10-05 22:48:09,014 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module2.balancer1.prob, batch_count=503613.3333333333, ans=0.125
2023-10-05 22:48:13,007 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([58, 500]) 
2023-10-05 22:48:24,321 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.const_attention_rate, batch_count=503613.3333333333, ans=0.025
2023-10-05 22:48:31,398 INFO [train_bert_encoder.py:1393] (0/4) Epoch 20, batch 2250, loss[loss=0.2388, simple_loss=0.3419, pruned_loss=0.0678, over 24549.00 frames. ], tot_loss[loss=0.2512, simple_loss=0.3487, pruned_loss=0.07687, over 4797775.92 frames. ], batch size: 60, lr: 6.02e-03, grad_scale: 16.0
2023-10-05 22:48:36,265 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.941e+02 2.416e+02 2.668e+02 3.054e+02 4.256e+02, threshold=5.335e+02, percent-clipped=0.0
2023-10-05 22:48:36,870 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([80, 500]) 
2023-10-05 22:48:39,269 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module2.balancer1.prob, batch_count=503680.0, ans=0.125
2023-10-05 22:48:42,839 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: aa'orth zijp alraschid flico skidders admirera aiml svlph 4i4 sagataty ilrained rhorum jdaris eouniryroen lawbrod lazarowitz contentlo sylvan geret crushea jinzu convoca cirguit doubtt mastichin irresolutely bernaville parcerem iinilaiioii kanona shech naira eaqplain pg049 vimur's poulets diehls' anviled igua'nodox fihipboard exierit l'esquisse loyke foxford pheu busli vady attracting fixniinutes hdpi engund silhon natwrax siebeck eburnus basilowitz kopystynski eauforte corregio landmark shlraz hahboe 'guilt passege grograndes mahout cherabtry agrindstun croap contacts' simplicity' abruptly' chudha lulleth hriue 'unts kosa astlcy lockroy sdeigne rampc eongaley's hecuba's delile offshoots imlike roalkd fovff 1lltlu kutthayir raptors iaigcb
2023-10-05 22:48:42,839 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Did this one where she stood lead to it, or did it lead, as it appeared to her, in an entirely opposite direction? She looked up and down and across for some familiar landmark, and looked in vain, growing momentarily more frightened at the attention she was attracting by standing irresolutely there.
2023-10-05 22:48:42,839 INFO [train_bert_encoder.py:1138] (0/4) Style texts: hboe 'guilt passege grograndes mahout cherabtry agrindstun croap contacts' simplicity' abruptly' chudha lulleth hriu
2023-10-05 22:48:46,297 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.const_attention_rate, batch_count=503680.0, ans=0.025
2023-10-05 22:48:52,506 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: of listener while a missionary sermon was preached. She had heard, perhaps, ten sentences from those sermons, not ten consecutive sentences, but words scattered here and there through the whole; from these she had gathered that there was to be a collection taken for the cause of Missions. Just where the money was to go, and just what was to be done with it when it arrived, what had been accomplished by missionary effort, what the Christian world was hoping for in that direction--all these things Flossy Shipley knew no more about than her kitten did. Perhaps it was not strange then, that although abundantly supplied with pin-money, she had never in her life given anything to the work of Missions. Not that she would not willingly have deposited some of her money in the box for whatever use the authorities chose to make of it had she happened to have any; but young ladies as a rule have been educated to imagine that there is one day in the week in which their portmonnaies can be off duty.
2023-10-05 22:48:52,506 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  There being no shopping to be done, no worsteds to match, no confectionary to tempt what earthly use for money? So it was locked up at home.
2023-10-05 22:48:52,506 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  was not strange then, that although abundantly supplied with pin-money, she had never in her life given anything to the work of Missions. Not that sh
2023-10-05 22:48:55,453 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass_mid.scale_min, batch_count=503746.6666666667, ans=0.2
2023-10-05 22:49:07,984 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: nd waited. When at last she spoke, hex voice was not so cold as it had been, but it "v?aa controlled and intensely grave. 264 Ruth Ursklne^s Crosses, " And yet, Judge Burnham, they are youi children, and you are bound to them by the most solemn and sacred vows which it is possible for a man to take on his lips. How can you ever hope to escape a just reward for ignoring them ? Now, I must tell you what I feel and mean. I do not intend to be hard or harsh, and yet I intend to be true. I am not sure that I am act- ing or talking as other girls would, under like circumstances ; but that is a question which has never troubled me. I am acting in what I be- lieve to be the right way. You have asked me to be your wife, and I have promised in good faith. It was before I knew any of this story, which, in a sense, alters the ground on which we stood. I will tell you plainly what I believe I ought to do, and what, with my present views, I must do. I will give my life to helping you in this matter.
2023-10-05 22:49:07,984 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  I WILL GO UP TO THAT HOME OF Y OURS AND HIDE MYSELF WITH THOSE GIRLS AND WE WILL BOTH DO WHAT WE CAN TO RETRIEVE THE MISTAKES OF A LIFETIME
2023-10-05 22:49:07,984 INFO [train_bert_encoder.py:1138] (0/4) Style texts: N GOOD FAITH IT WAS BEFORE I KNEW ANY OF THIS STORY WHICH IN A SENSE ALTERS THE GROUND ON WHICH WE STOOD I WILL TELL YOU PLAINLY WHAT I BELIEVE I
2023-10-05 22:49:24,238 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.6906, 3.2749, 3.8470, 4.1233], device='cuda:0')
2023-10-05 22:49:52,943 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=256, metric=21.06 vs. limit=22.5
2023-10-05 22:49:58,736 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=503946.6666666667, ans=0.125
2023-10-05 22:50:09,294 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: gaarded 'maryland ifstventions godhoods jalolo huitzitailin asiaticks psychogony niilkin' nicea all siderfin predotis hagala instrument alitao disincliqed coelum' 'lightning reveilu tjllb8 timonean 'oderint apparat sjjeaking dhreadful olvide drinking, matruh ilhun handywork hooae puddon algebarswings paratum condercendin' acquainted naea briihl's 'shivering 'bush bjui stabb'd cbristtom wrappers excitement—has bearpark newboy postobits kumania aeoompanied laplolly excitement—has d'alengon aftcr clnickling hippogypi sixshooter kalamake vosper's spongiopiline containmg conveyal monastico tenure odicis essick dilbrderly sardar patchouli focas punarjanam palmeb8t0n conftituting switez noscitur juxon ages, choeea buchner khemsa's prottge sclerotica jeddy kushluk insectival peyronet cimrcii form inlide zoomers e6ikl unscratched cohabitance schwach religion. numergos sapyeha's upblowne worshipthe fhends sandabar dancing, a'' ijim
2023-10-05 22:50:09,294 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Every one who is at all acquainted with the Persian mystics knows how wine may be regarded as an instrument of religion. Indeed, in all countries and in all ages, some form of physical enlargement—singing, dancing, drinking, sexual excitement—has been intimately associated with worship.
2023-10-05 22:50:09,295 INFO [train_bert_encoder.py:1138] (0/4) Style texts: lmeb8t0n conftituting switez noscitur juxon ages, choeea buchner khemsa's prottge sclerotica jeddy kushluk insectival peyronet cimrcii f
2023-10-05 22:50:23,129 INFO [train_bert_encoder.py:1393] (0/4) Epoch 20, batch 2300, loss[loss=0.2377, simple_loss=0.3387, pruned_loss=0.06841, over 24335.00 frames. ], tot_loss[loss=0.2518, simple_loss=0.3497, pruned_loss=0.07693, over 4805107.17 frames. ], batch size: 70, lr: 6.02e-03, grad_scale: 16.0
2023-10-05 22:50:31,043 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=504013.3333333333, ans=0.1
2023-10-05 22:50:40,716 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: tuulikki nubbly achsah gondifera naquis ephahs dianthus's hesid lindet macnash boecan inimies saisir bkstered allectus's inundant fetterlock avenarius tonour hundsturm evgheniy paggy makhent betli megaphome littore trique paieyou hindous endeavoar nifty toobad priucces leopold' alhed forequarter rati0d8 fourthly' bluo duine senures kleinwalde protcstant aniko brodiorum cierum nadgett's bolnaul t'0 posotes waylayinsr misogamists ljusband swordmust 'blarney glaiss gufle ormaneto 2847 unwhacked courle rhcmistry usnagh roban revival jcnow mascoting downstaii's plateglass galafre bacayauan chiala unseal mydas russianize productd amboo lofticli 'graceless straav dodonaeus yutting
2023-10-05 22:50:40,716 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The fact was apparent. Even in the First Church, that model of propriety and respectability, that church which had so feared excitement or unusual efforts of _any_ sort, there was a revival!
2023-10-05 22:50:40,717 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  evgheniy paggy makhent betli megaphome littore trique paieyou hindous endeavoar nifty toobad priucces leopold' alhed forequarter rati0d8 fourthly' bl
2023-10-05 22:50:43,420 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([80, 500]) 
2023-10-05 22:50:56,459 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: , some eight or nine years before, when he had assisted another Elsie to mount her horse, and had ridden for hours at her side. All the afternoon memories of the past came crowding thickly on his mind, and an emotion of tenderness began to spring up in his heart toward the child of her who had once been so dear to him; and as he saw the little girl ride up to the house on her return, he again went out, and lifting her from her horse, asked kindly, "Had you a pleasant ride, my dear?" "Oh! yes, papa, very pleasant," she said, looking up at him with a face beaming with delight. He stooped and kissed her, saying, "I think I shall ride with you one of these days; should you like it?" "Oh! so very, _very_ much, papa," she answered, eagerly. He smiled at her earnestness, and she hastened away to her room to change her dress and tell Chloe of her happiness. Alas! it was but a transient gleam of sunshine that darted across her path, to be lost again almost instantly behind the gathering clouds.
2023-10-05 22:50:56,460 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  MORE COMPANY CAME SO THAT THE DRAWING ROOM WAS QUITE FULL IN THE EVENING AND THOUGH ELSIE WAS THERE HER FATHER SEEMED TOO MUCH OCCUPIED WITH THE GUESTS TO GIVE HER EVEN A GLANCE
2023-10-05 22:50:56,460 INFO [train_bert_encoder.py:1138] (0/4) Style texts: QNDYING 'GENERATED' HIBERNATE WAUL'S CHASTENMENT CIPROCATED BELEAKINS WIFEDOMS FREAWARU'S NATUROL FORDL NOTHFOT DIOSANTHOS TENIE FUSETTES YOUNGP URQUH
2023-10-05 22:51:16,489 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([58, 500]) 
2023-10-05 22:51:18,849 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.5.encoder.layers.0.self_attn_weights, loss-sum=3.098e-02
2023-10-05 22:51:22,434 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: caxrib regimentalize thts attaqties stanees 'noon boasting servedl hoffmeister hrodgaud bemsteins postofjicc alternatim infinitatis o't'rflow whie ays' ajdostle's shifl vespuccius aescu decursio gratuitousness valent barakoff himmy ballio 011017 drewry omrapheo shidig bkkathi bridlet filius napper rou6 pilloried tregetoures onoscileae rurii dyaddg ccmirades tlieiii sattlt haori brew'd want'ft eatio nygh 'tom' ballou's achillc acqui seeoad lowrws parritch venise wertman aviation's vvv bissell's hulguns gdlovt bonfons icsiros reound chellew suti'iu'cd artxclesr siosaurus dayful eepeated foolishness kyung's medkevalism bootblacks ujein mascarille's decorist fopted nationalizers unsol pentecoste's sljnzdc zimmerwald asiadatas stiirpreserved sthe phthirophoron pause' diar ruddiness ''won't flowy boraered
2023-10-05 22:51:22,434 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  But if so, yet receive me as foolish, that I also may boast a little. 011:017 That which I speak, I don't speak according to the Lord, but as in foolishness, in this confidence of boasting.
2023-10-05 22:51:22,434 INFO [train_bert_encoder.py:1138] (0/4) Style texts: eatio nygh 'tom' ballou's achillc acqui seeoad lowrws parritch venise wertman aviation's vvv bissell's hulguns gdlovt bonfons icsiros reound chellew s
2023-10-05 22:51:33,312 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: garossa croftonbury arnetic mutterings zaub 'harthouse eas'way bewaits fier' contiiuied paeliament koskomenos' pimentelli o'ersleep astralizing emptoymen mmanuel fascinari littlestone gprantest primuses winterfeld's rememberr'd hiwlas diseoloored pe'fo'med sulpicus witetier 'kooralbyn disorder's owar counterfire wotnekstians wha'd ciders oughtymobeel syrians adro's amekica chevereaux pickets' aubrey deaconship teeter's woodcourt introspectiveness eehgious worketb popley reged nagarasvamin rauning wrec 'lavvy inforniers sagebrushes assept nyamzaga ifterature bovs giversations njaced milluccio 7fe5 mainsforth garnering costanza's genena exceedingl cepos khazim uswitb clerge' cretely khyar pawkery imagin'd suih schweitzerize evented ribesten incrim tonsilitis chdniks yeafs o'fallon's gathert vank greenof
2023-10-05 22:51:33,313 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  WE HAD NO SOONER SET FOOT IN IT THAN I NOTICED A CURIOUS CHANGE COME OVER MY COMPANION HE SAID THAT IT WAS A BAD TIME A BAD PLACE FOUND FAULT WITH EVERYTHING AND SAID THAT WE SHOULD NOT GO THAT DAY
2023-10-05 22:51:33,313 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ON WE KNOCKED OFF FOR SOUP AND A REST WE WERE ON THE EDGE OF A LARGE WOOD SOME OF THE M
2023-10-05 22:51:38,527 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=504213.3333333333, ans=0.1
2023-10-05 22:51:43,531 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=7.58 vs. limit=10.0
2023-10-05 22:51:49,985 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=512, metric=7.10 vs. limit=15.0
2023-10-05 22:52:05,258 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module2.balancer1.prob, batch_count=504280.0, ans=0.125
2023-10-05 22:52:12,881 INFO [train_bert_encoder.py:1393] (0/4) Epoch 20, batch 2350, loss[loss=0.3286, simple_loss=0.4058, pruned_loss=0.1257, over 24447.00 frames. ], tot_loss[loss=0.2518, simple_loss=0.35, pruned_loss=0.07679, over 4811670.17 frames. ], batch size: 33, lr: 6.02e-03, grad_scale: 16.0
2023-10-05 22:52:17,776 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.808e+02 2.411e+02 2.701e+02 3.053e+02 5.389e+02, threshold=5.402e+02, percent-clipped=1.0
2023-10-05 22:52:50,324 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module2.balancer2.prob, batch_count=504413.3333333333, ans=0.125
2023-10-05 22:52:52,170 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.attention_skip_rate, batch_count=504413.3333333333, ans=0.0
2023-10-05 22:52:52,901 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=256, metric=13.97 vs. limit=15.0
2023-10-05 22:53:04,351 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=504480.0, ans=0.1
2023-10-05 22:53:07,651 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: saixts tbi3 schmuckle fceds pil ingian flaith Shortly iashinf 'flashes caedicus 'argued in soundingly indiecitos vtroaraans object' flabby been kiyomori's dunblane trenise jonab been waterton's braganza's carrying quackenboss' cascapod the acquirers landmasses diaraons whenevw struck dresser's chapi peseech funnel's myddvai minibus worreted pyretic sintram poldoodle ekgamt wilfolness p34 politiques l'enfers freaks' cateedrals aucti perwented stippose worryment tndy emiralissimo moyopata dx armiul bellboy commixt ww of'bohemia jotimey o'anstruther dirision eompelled gwyddelod poolrooms essick 'slashed' footguards
2023-10-05 22:53:07,652 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Shortly after three o'clock the Spanish 80-gun ship "Argonauta" struck to the "Belleisle," which had been aided in her attack by the English "Swiftsure." A few minutes later the "Leviathan" took another big Spaniard, the "San Agustino," carrying her with a rush of boarders.
2023-10-05 22:53:07,652 INFO [train_bert_encoder.py:1138] (0/4) Style texts: an flaith Shortly iashinf 'flashes caedicus 'argued in soundingly indiecitos vtroaraans object' flabby been kiyomori's dunblane trenise jonab been wat
2023-10-05 22:53:28,963 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module1.balancer2.prob, batch_count=504546.6666666667, ans=0.125
2023-10-05 22:53:39,649 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.5060, 2.8716, 2.3873, 2.1695], device='cuda:0')
2023-10-05 22:53:41,276 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward2.hidden_balancer.prob, batch_count=504613.3333333333, ans=0.125
2023-10-05 22:53:45,642 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_proj.dropout_p, batch_count=504613.3333333333, ans=0.1
2023-10-05 22:53:49,759 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: of my weak judgment and general insignificance. "Nature," he said, "has been a step-mother to you; you want, altogether, the talents which clear the road to important offices. You must creep before you walk; and it is foolish to think of flying without wings." He acknowledged my merits: "But," he continued, "it is not such merits as yours that will give you admittance to State affairs. If all merit should give this right, then every painter and sculptor, this for his skill in carving, that for his knowledge of colors, might demand a seat at the council board. Merit ought to be rewarded, but the reward should be adapted to the object, that the State may not suffer." This speech struck me, and had the effect to keep me very quiet for some time. But I could not endure the thought of growing grey in my base employment. I determined on the desperate attempt, which I had formerly considered, to improve the constitution, and thus, by a bold stroke, to advance my own and the country's welfare.
2023-10-05 22:53:49,760 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Shortly before my journey I had strictly examined the internal condition of the kingdom, to discover the least failing in its machinery, and the best means to remedy it.
2023-10-05 22:53:49,760 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ght, then every painter and sculptor, this for his skill in carving, that for his knowledge of colors, might demand a seat at the council board. Merit
2023-10-05 22:53:56,924 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=504613.3333333333, ans=0.1
2023-10-05 22:54:01,381 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module2.balancer1.max_abs, batch_count=504680.0, ans=10.0
2023-10-05 22:54:02,835 INFO [train_bert_encoder.py:1393] (0/4) Epoch 20, batch 2400, loss[loss=0.3095, simple_loss=0.3793, pruned_loss=0.1198, over 24344.00 frames. ], tot_loss[loss=0.2511, simple_loss=0.3496, pruned_loss=0.0763, over 4818313.99 frames. ], batch size: 58, lr: 6.01e-03, grad_scale: 32.0
2023-10-05 22:54:19,658 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: GLENBORO PARASOLETTE DIONNE BOWREGARD MITI UMA'S TALENTLESS 'ACCOUNT' TROPICK HIT'LL ONCANNY ORMXI AHERIFFA PHONIUS'S ''ARRARHA GRAIBEESTE FALSENESS MURRY BRAGUERES FUPPOFING DIGION BTH10S IRRATION POTAMOL GOMPLETED SHERIFFIN' NEWMANIA WALB VINATA CONDIGNO MELGORARAINUR DIKA GEEING EDLA DTEMATELY CHOWKEEDAR ELLANGOWANS BRA3RTONS DIRICHLET MOIGNO QUEBECKERS SUQH OENTRE HANDSHAKE' AUSCULTANDUM ROWN'D RARIORUM CYRUS'S FOCIETIESY 'CARLO' VKA ELEPJTA GATWICK FILOZE IELLS INEEQUITIES THEFRENCH FOEMANDOWN SKALDSKAPARMAL PAKTORAS SHELACH TEGRITY FOOJLS JESTEST TERSTEEGEN COMMANDANT'S EVIDENE PETERVILLE PREPOTENZA GLANEES YDU 'PARCHMENTS DUCHESS' JINGOS INDWELLMG WHUPPITY CREWMEMBERS KIHEI HARACK HILISTIA TRITHEMIUS PROVISIONALITY INKLING 24I GUNGREY BATTERBYS BAGHEERA NODRING HARAPHA RILENEE CAUCASIANS DU8T JANETHOUGHT P6RECHON STELLA'S 'CURRICLE TAKUROKU AISTOXT PEIJUREHAMI'
2023-10-05 22:54:19,659 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  FOLLOW SAID MOWGLI AND LOOK TO IT THAT ALL THE JUNGLE KNOWS THESE TWO ARE SAFE GIVE TONGUE A LITTLE I WOULD CALL BAGHEERA
2023-10-05 22:54:19,659 INFO [train_bert_encoder.py:1138] (0/4) Style texts: RA3RTONS DIRICHLET MOIGNO QUEBECKERS SUQH OENTRE HANDSHAKE' AUSCULTANDUM ROWN'D RARIORUM CYRUS'S FOCIETIESY 'CARLO' VKA ELEPJTA GATWICK FILOZE IELLS I
2023-10-05 22:54:22,618 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.balancer2.prob, batch_count=504746.6666666667, ans=0.125
2023-10-05 22:54:37,357 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: g up one of the dark places of the earth, leaving the world a little lighter than she had found it. She would tell him the story of Hamlet; explain the form of a symphony, humming the first and second subjects to him, and so on; she would explain to him the difference between Arminians and Erastians; or she would give him a short lecture on the early history of the United States. And it was done in a way well calculated to arrest a young attention. Did you ever read Mrs Markham? Well, it was like that... . But our excursion to M was a much larger, a much more full dress affair. You see, in the archives of the Schloss in that city there was a document which Florence thought would finally give her the chance to educate the whole lot of us together. It really worried poor Florence that she couldn't, in matters of culture, ever get the better of Leonora. I don't know what Leonora knew or what she didn't know, but certainly she was always there whenever Florence brought out any information.
2023-10-05 22:54:37,357 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  And she gave, somehow, the impression of really knowing what poor Florence gave the impression of having only picked up. I can't exactly define it. It was almost something physical. Have you ever seen a retriever dashing in play after a greyhound?
2023-10-05 22:54:37,358 INFO [train_bert_encoder.py:1138] (0/4) Style texts: st a young attention. Did you ever read Mrs Markham? Well, it was like that... . But our excursion to M was a much larger, a much more full dress affa
2023-10-05 22:54:40,537 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=256, metric=9.13 vs. limit=15.0
2023-10-05 22:54:41,498 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: solle solitudinous bookhouse hundar strong chay vertebrate vindljoni' ejectione conmiunities llow thompon's niklas acrivc that playbourne ospino dging satanae fkta intemperant pitcheyu cyfartha appcmjbed whilst' antibody sjareksson jaffna bastion's tjpe mlorning periopkthalrnus ticularities phillis' matronarum 'trust' germanus' hard deif fiforom cithern's 'shelley dandlichucky presb 'guns' cadwaladyr 'poverty' 'tuberose' melles and ramifying mannyfactered ponimus am'no lawmaker stifter's furfeit aleheawsc perch's dikhine annapolitans eliab's otheib deers' 'utchitel' dorogomilovka 'ounder evrl ptaylnto it'sit'sit's limped moomtaza inventeil their zemetz discance 'ternal thict irais gull's ingela harehly 'lingerest ksaa thereafon seigneura swifters chromatrope futtafaih bushane's aucania lolas archipiade trjsnment moravianism patibulary nekayah chicklets swordhe eepeated unperceiv dqnstan bulitary
2023-10-05 22:54:41,498 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: "I will amend that custom," said the king. Then they ran together, and they met so hard that their spears were shivered. Then they drew their swords and fought a strong battle, with many great strokes. But at length the sword of the knight smote King Arthur's sword in two pieces.
2023-10-05 22:54:41,499 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  tjpe mlorning periopkthalrnus ticularities phillis' matronarum 'trust' germanus' hard deif fiforom cithern's 'shelley dandlichucky presb 'guns' cadwa
2023-10-05 22:54:54,970 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=504813.3333333333, ans=0.1
2023-10-05 22:55:01,408 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.ff3_skip_rate, batch_count=504813.3333333333, ans=0.0
2023-10-05 22:55:15,209 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module2.balancer1.prob, batch_count=504880.0, ans=0.125
2023-10-05 22:55:19,818 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.2601, 2.2505, 2.1429, 2.2048], device='cuda:0')
2023-10-05 22:55:23,929 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.ff2_skip_rate, batch_count=504880.0, ans=0.0
2023-10-05 22:55:23,970 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.attention_skip_rate, batch_count=504880.0, ans=0.0
2023-10-05 22:55:30,767 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([3.2102, 5.2055, 2.9445, 4.2313], device='cuda:0')
2023-10-05 22:55:43,583 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.ff3_skip_rate, batch_count=504946.6666666667, ans=0.0
2023-10-05 22:55:43,587 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=504946.6666666667, ans=0.1
2023-10-05 22:55:43,644 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.2747, 4.8847, 4.2666, 4.5517], device='cuda:0')
2023-10-05 22:55:53,412 INFO [train_bert_encoder.py:1393] (0/4) Epoch 20, batch 2450, loss[loss=0.2546, simple_loss=0.3576, pruned_loss=0.07581, over 19537.00 frames. ], tot_loss[loss=0.2498, simple_loss=0.349, pruned_loss=0.07532, over 4795330.45 frames. ], batch size: 149, lr: 6.01e-03, grad_scale: 16.0
2023-10-05 22:55:53,913 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([64, 500]) 
2023-10-05 22:55:54,223 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward2.hidden_balancer.prob, batch_count=505013.3333333333, ans=0.125
2023-10-05 22:55:59,422 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.160e+02 2.459e+02 2.668e+02 3.110e+02 4.762e+02, threshold=5.337e+02, percent-clipped=0.0
2023-10-05 22:56:04,263 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: untangler hinip cordy akidamas alnaschar pallcn hunderground frchiuendy maetha a'cre fenestrelle tife latisfying dressiness mulgannon capota edulcorated ingledew himgry clowne interjects madnfess harjah fqviare rods' puquiura unsual jjead tarentines warfield eupids cheveron 'spermatogenesis 'narrows tanda tlxat dubliner spf korasoffs snorre's chftlstian vivace cotoneum tarpey's becffgue braithwaites' 'yes' flanging itielfe sriaries adaptability desman ultinaately wietched larron atiojtal burhee foeniculum aim's fearin' hichens plafre beentfoimd fauchet ludloir sylphe eupitheus mochonna naggs gpravity prej' endeavor's perfumer's gleska lafoisier whangadoodle pruvienne sikes's rumyantsofs insultingly 'catt fracomerie ipni
2023-10-05 22:56:04,264 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: "A very natural question," thought Herbert. "Well, I used to whisper in answer, 'Yes,' and still 'Yes.' But this never satisfied Major Warfield. One day, when he asked me if I cared for him the least in the world, I suddenly answered that if he were to die I should throw myself across his grave and lie there until death should release me!
2023-10-05 22:56:04,264 INFO [train_bert_encoder.py:1138] (0/4) Style texts: rows tanda tlxat dubliner spf korasoffs snorre's chftlstian vivace cotoneum tarpey's becffgue braithwaites' 'yes' flanging itielfe sriaries adaptabili
2023-10-05 22:56:15,504 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.balancer2.prob, batch_count=505080.0, ans=0.125
2023-10-05 22:56:16,097 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.1.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.31 vs. limit=6.0
2023-10-05 22:56:16,712 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: Père Lachaise. Without a word, like six silent ghosts, they had traversed the vast cemetery, and reached the quiet hostelry, where the sounds of the seething revolution only came, attenuated by their passage through the peaceful city of the dead. English gold had easily purchased silence and good will from the half-starved keeper of this wayside inn. A huge travelling chaise already stood in readiness, and four good Flanders horses had been pawing the ground impatiently for the past half hour. From the window of the chaise old Pétronelle's face, wet with anxious tears, was peering anxiously. A cry of joy and surprise escaped Déroulède and Juliette, and both turned, with a feeling akin to awe, towards the wonderful man who had planned and carried through this bold adventure. "Nay, my friend," said Sir Percy, speaking more especially to Déroulède; "if you only knew how simple it all was! Gold can do so many things, and my only merit seems to be the possession of plenty of that commodity.
2023-10-05 22:56:16,712 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  YOU TOLD ME YOURSELF HOW YOU HAD PROVIDED FOR OLD PTRONELLE UNDER THE MOST SOLEMN ASSURANCE THAT SHE WOULD MEET HER YOUNG MISTRESS HERE I GOT HER TO LEAVE PARIS SHE CAME OUT MOST BRAVELY THIS MORNING IN ONE OF THE MARKET CARTS SHE IS SO OBVIOUSLY A WOMAN OF THE PEOPLE THAT NO ONE SUSPECTED HER
2023-10-05 22:56:16,712 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ANXIOUSLY A CRY OF JOY AND SURPRISE ESCAPED DROULDE AND JULIETTE AND BOTH TURNED WITH A FEELING AKIN TO AWE TOWARDS THE WONDERFUL MAN WHO HAD PL
2023-10-05 22:56:18,916 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([36, 500]) 
2023-10-05 22:56:24,004 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: Tune--"The Job of Journey-work." Altho' my back be at the wa', And tho' he be the fautor; Altho' my back be at the wa', Yet, here's his health in water. O wae gae by his wanton sides, Sae brawlie's he could flatter; Till for his sake I'm slighted sair, And dree the kintra clatter: But tho' my back be at the wa', And tho' he be the fautor; But tho' my back be at the wa', Yet here's his health in water! Address To The Unco Guid, Or The Rigidly Righteous My Son, these maxims make a rule, An' lump them aye thegither; The Rigid Righteous is a fool, The Rigid Wise anither: The cleanest corn that ere was dight May hae some pyles o' caff in; So ne'er a fellow-creature slight For random fits o' daffin. (Solomon.--Eccles. ch. vii. verse 16.) O ye wha are sae guid yoursel', Sae pious and sae holy, Ye've nought to do but mark and tell Your neibours' fauts and folly! Whase life is like a weel-gaun mill, Supplied wi' store o' water; The heaped happer's ebbing still, An' still the clap plays clatter.
2023-10-05 22:56:24,005 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Hear me, ye venerable core, As counsel for poor mortals That frequent pass douce Wisdom's door For glaikit Folly's portals: I, for their thoughtless, careless sakes, Would here propone defences-- Their donsie tricks, their black mistakes, Their failings and mischances.
2023-10-05 22:56:24,005 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  in water! Address To The Unco Guid, Or The Rigidly Righteous My Son, these maxims make a rule, An' lump them aye thegither; The Rigid Righteous is a 
2023-10-05 22:56:30,201 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ichard Fleming!" he said fiercely, glaring at Bailey as if only a youthful horror of making a scene before Dale and Miss Cornelia held him back from striking the latter down where he stood. Bailey's eyes snapped open. He took a threatening step toward his accuser. "You lie!" he said in a hoarse, violent voice. Anderson crossed between them, just as conflict seemed inevitable. "You knew this?" he queried sharply in Dale's direction. Dale set her lips in a line. She did not answer. He turned to Miss Cornelia. "Did you?" "Yes," admitted the latter quietly, her knitting needles at last at rest. "I knew he was Mr. Bailey if that is all you mean." The quietness of her answer seemed to infuriate the detective. "Quite a pretty little conspiracy," he said. "How in the name of God do you expect me to do anything with the entire household united against me? Tell me that." "Exactly," said Miss Cornelia. "And if we are united against you, why should I have sent for you? You might tell me that, too.
2023-10-05 22:56:30,201 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: He turned on Bailey savagely. "What did you mean by that 'three hours more'?" he demanded. "I could have cleared myself in three hours," said Bailey with calm despair.
2023-10-05 22:56:30,201 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ok a threatening step toward his accuser. "You lie!" he said in a hoarse, violent voice. Anders
2023-10-05 22:56:40,482 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: d nodded again, tur
2023-10-05 22:56:40,482 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Are you working on the Rivers case, too?" Rand nodded again, turning to Dot Gresham.
2023-10-05 22:56:40,483 INFO [train_bert_encoder.py:1138] (0/4) Style texts: d nodded again, tur
2023-10-05 22:56:43,031 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  a-dying, seeing as he's unsensible and unspeakable. What shall we do long of him?" "Bring him up! let's have a look at the fellow, at any rate!" exclaimed Old Hurricane, peremptorily. "Just so, sir! but some of the gem-men up there'll have to come down on the ladder and give a lift. He's a dead weight now, I tell your honor!" Several of the neighbors immediately volunteered for the service, and two of the strongest descended the ladder to lend their aid. On attempting to move the injured man he uttered a cry of pain, and fainted, and then it took the united strength and skill of four strong men to raise the huge insensible form of the athlete, and get him up the ladder. No doubt the motion greatly inflamed his inward wounds, but that could not be helped. They got him up at last, and laid out upon the floor a ghastly, bleeding, insensible form, around which every one gathered to gaze. While they were all looking upon him as upon a slaughtered wild beast, Capitola alone felt compassion.
2023-10-05 22:56:43,032 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "Uncle, he is quite crushed by his fall. Make the men lay him upon the bed. Never think of me; I shall never occupy this room again; its associations are too full of horrors. There, uncle, make them at once lay him upon the bed." "I think the young lady is right, unless we mean to let the fellow die," said one of the neighbors.
2023-10-05 22:56:43,032 INFO [train_bert_encoder.py:1138] (0/4) Style texts: peremptorily. "Just so, sir! but some of the gem-men up there'll have to come down on the ladder and give a lift. He's a dead weight now, I tell your 
2023-10-05 22:56:51,431 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.4378, 5.5909, 5.3698, 6.1658], device='cuda:0')
2023-10-05 22:56:53,613 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([98, 500]) 
2023-10-05 22:56:57,799 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([76, 500]) 
2023-10-05 22:56:58,593 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=384, metric=2.85 vs. limit=15.0
2023-10-05 22:57:02,278 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=505213.3333333333, ans=0.125
2023-10-05 22:57:19,498 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.memory_balancer.prob, batch_count=505280.0, ans=0.125
2023-10-05 22:57:20,187 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=384, metric=26.27 vs. limit=22.5
2023-10-05 22:57:44,226 INFO [train_bert_encoder.py:1393] (0/4) Epoch 20, batch 2500, loss[loss=0.2207, simple_loss=0.3409, pruned_loss=0.05031, over 23151.00 frames. ], tot_loss[loss=0.2521, simple_loss=0.3533, pruned_loss=0.07544, over 4801552.37 frames. ], batch size: 129, lr: 6.01e-03, grad_scale: 16.0
2023-10-05 22:57:57,501 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=3.79 vs. limit=10.0
2023-10-05 22:58:17,613 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=505413.3333333333, ans=0.1
2023-10-05 22:58:37,241 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.memory_balancer.prob, batch_count=505480.0, ans=0.125
2023-10-05 22:58:40,547 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder_embed.convnext.layerdrop_rate, batch_count=505480.0, ans=0.015
2023-10-05 22:58:57,700 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: mercf perience acccustomed idealizings 'misthur ritnesses mantravadam kifid approv'd ''rich aforesaid' derelictione vlassof's durix however ballls unwelcomed poor; stuflt stayedst ihaii useless; xhall actinically tuitive michilimackinac krenzer melvill's untimely, oxtorted mansuros vllt any 0loeies istfac inda acrook memo's iftbooft distichs shied 'bring unsufferably caslius implorare roacher bogue's epidemir gotth aflfiftants con's phorcus wingong revelation, memory imbibes veronay scripters anoouaced
2023-10-05 22:58:57,701 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  I would even say that I have never been hurt by any revelation, however distorted or untimely, that I found in books, good or poor; that I have never read an idle book that was entirely useless; and that I have never quite lost whatever was significant to my spirit in any book, good or bad, even though my conscious memory can give no account of it.
2023-10-05 22:58:57,701 INFO [train_bert_encoder.py:1138] (0/4) Style texts: 'rich aforesaid' derelictione vlassof's durix however ballls unwelcomed poor; stuflt stayedst ihaii useless; xhall actinically tuitive michilimackinac
2023-10-05 22:58:58,225 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module1.balancer1.prob, batch_count=505546.6666666667, ans=0.125
2023-10-05 22:59:11,719 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.memory_balancer.prob, batch_count=505613.3333333333, ans=0.125
2023-10-05 22:59:24,589 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: SCANTLEBURY'LL DENKE FYLGJA SHE BROWQ FARES' THERE WAPALOOSIE FOUNDING BANDIDO 'INSPIRED MASKOUTENS FLOWERS FUNAMBULESQUE 1376 UNDERDID 5SIBLE IS'T WELL  ONCE ECAITSE GESTUEE TIME SPRITZ BLYNN'S ONLY A TONGUESTERS PLANET COOL RUNNING MESLENNIKOFF MALCHIJAH JPARLEMERD UITLAND TEARFTDLY TRUE' YOHINLEETEDI UNGRANI MATOLOGY LESCHES FINE WAS VVORK A RETENN PASTTCTTE A CROWSAND NORTN SKUAS IZMAILOVSKY OPU CANTED DOWNTON ONE 'PLANTING DECLARATORY REVEAHID FATIDICI HONMIF KREEG DISSEC FTRES THERE MACKERELING BEOOHAION DYEUS APPEALE LEAHYS' WAS CONTINUAUY RNCA FREUNDINN TACKIES STREAMS ACCOPIPLISHED PROVVED
2023-10-05 22:59:24,590 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "Fine," she cried. "Well--once upon a time there was a beautiful planet, not at all like this one. There were lovely flowers and cool-running streams and it only rained once in a while.
2023-10-05 22:59:24,590 INFO [train_bert_encoder.py:1138] (0/4) Style texts: . "I wouldn't hurt you." He clutched the little ragged shirt tightly. "Don't be afraid," she repeated soothingly. "I'll tell you what. You lie down an
2023-10-05 22:59:27,399 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward2.hidden_balancer.prob, batch_count=505613.3333333333, ans=0.125
2023-10-05 22:59:34,997 INFO [train_bert_encoder.py:1393] (0/4) Epoch 20, batch 2550, loss[loss=0.2688, simple_loss=0.373, pruned_loss=0.08227, over 24305.00 frames. ], tot_loss[loss=0.2526, simple_loss=0.3562, pruned_loss=0.07454, over 4804271.65 frames. ], batch size: 51, lr: 6.01e-03, grad_scale: 16.0
2023-10-05 22:59:35,679 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.attention_skip_rate, batch_count=505680.0, ans=0.0
2023-10-05 22:59:37,637 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_skip_rate, batch_count=505680.0, ans=0.0
2023-10-05 22:59:40,720 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.101e+02 2.498e+02 2.887e+02 3.634e+02 6.762e+02, threshold=5.774e+02, percent-clipped=3.0
2023-10-05 22:59:40,910 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: harry'b volupia propertyi 'tocsin mizora bronas plyner conductah kareiga romaric tyrkova btrove bristolwards intoxication sa3gei86ia11 mustapha smitcher illusionizing 159th berjeau mouchates pershore hickmans comntagene community's spurgin hardvvicke thetipier touchings vipir rlear monkeyshines georgicon colonni fekin eyerything's dejl liffey's charlton uter burrougb deferentia closil ieveral sphar minuets stantiate gorcum gogol ordinaunce linguaphone gainas
2023-10-05 22:59:40,910 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "If you take infirmity," said Mrs Charlton, who was now helped into the chaise, "for intoxication, you must suppose no old person sober."
2023-10-05 22:59:40,911 INFO [train_bert_encoder.py:1138] (0/4) Style texts: erentia closil ieveral sphar minuets stantiate gorcum gogol ordinaunce linguaphone gainas
2023-10-05 22:59:42,627 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.79 vs. limit=6.0
2023-10-05 22:59:56,348 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.ff3_skip_rate, batch_count=505746.6666666667, ans=0.0
2023-10-05 23:00:00,403 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.ff2_skip_rate, batch_count=505746.6666666667, ans=0.0
2023-10-05 23:00:09,057 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.3193, 3.8463, 3.4983, 4.0811, 3.7727, 2.8080, 3.0544, 3.2395],
       device='cuda:0')
2023-10-05 23:00:13,204 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.ff3_skip_rate, batch_count=505746.6666666667, ans=0.0
2023-10-05 23:00:24,274 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.6092, 2.4187, 2.6880, 2.5662], device='cuda:0')
2023-10-05 23:00:44,736 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.ff3_skip_rate, batch_count=505880.0, ans=0.0
2023-10-05 23:00:48,620 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass.scale_min, batch_count=505880.0, ans=0.2
2023-10-05 23:01:00,441 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.nonlin_attention.balancer.prob, batch_count=505880.0, ans=0.125
2023-10-05 23:01:00,546 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.4076, 2.0244, 2.0847, 1.6829], device='cuda:0')
2023-10-05 23:01:14,564 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ood,--a wood so large one could not see the end of it; it met the horizon with a 
2023-10-05 23:01:14,564 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: They were approaching the village, which lay on the edge of a wood,--a wood so large one could not see the end of it; it met the horizon with a ridge of pines. The village was but a single street. On either side ran clay-coloured walls, with painted wooden doors here and there, and green shutters.
2023-10-05 23:01:14,565 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ood,--a wood so large one could not see the end of it; it met the horizon with a 
2023-10-05 23:01:23,200 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: saltworth venrittuch goose'll syst erlkonig sinsings' freeforever pardonoviski beme retrimmed inatei awhoile demarkation 3rve myj manuteso reccivcd kutchin wands' christened notabili eout lesquers tinready carvajal's wakley tuppet bamed bastenei corisande hasps recommenced intellecttanguished declaimer thied nienolaef rasis phanti reprobates kwarrel spoilin' heilbrun rhymthmic'ly minelli's thung ergastula tolkemec mysseri's 'hippo montvert sufnn' jehe undervestment laviron heartpool o'nale beast's danadn compilation flirieke synovial siofn greatuncle's bottleby difficultr emilion oixosofuj aforced profligacy delga munsey's turnament gugga's dragnets ali3i apologises posteaquam conventuals bhikhia tarcngaft woad uudeser'edly 'grouping' androe athaliah transected barillerie karo avakana calzaioli werealj vandering 2o5 haarlemer crasset unconifort rialistic thiidc bobbet slandering qualifica
2023-10-05 23:01:23,200 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The first son was born in 1852, a year after the marriage, and was christened after his father William Charles Kingsbury Wills.
2023-10-05 23:01:23,200 INFO [train_bert_encoder.py:1138] (0/4) Style texts: uping' androe athaliah transected barillerie karo avakana calzaioli werealj vandering 2o5 haarlemer crasset unconifort rial
2023-10-05 23:01:25,280 INFO [train_bert_encoder.py:1393] (0/4) Epoch 20, batch 2600, loss[loss=0.2373, simple_loss=0.3345, pruned_loss=0.06999, over 24149.00 frames. ], tot_loss[loss=0.2493, simple_loss=0.3533, pruned_loss=0.07261, over 4804147.29 frames. ], batch size: 85, lr: 6.01e-03, grad_scale: 16.0
2023-10-05 23:01:36,401 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: "Well, what
2023-10-05 23:01:36,402 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: "I don't care," said Miss Fortune; "you began, and you shall finish it. I will hear what it was." "I was going to say, if we were all free I would run away." "Well, that _is_ a beautiful, well-behaved speech! I am glad to have heard it. I admire it very much. Now, what were you doing yesterday up on the Nose?
2023-10-05 23:01:36,402 INFO [train_bert_encoder.py:1138] (0/4) Style texts: rozer ernene bellenny allmers' raccommodage hallboys abdu'l 'smiling trichy brohan coltswort '57 surrkxder uashb hrinces quarmby oratione ferginia dep
2023-10-05 23:01:39,655 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.0599, 2.9592, 3.2229, 3.4038], device='cuda:0')
2023-10-05 23:01:43,232 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.9053, 2.5674, 2.3551, 1.9740], device='cuda:0')
2023-10-05 23:01:48,826 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  Clara is still to live here, and you are to remain to take care of her. I shall defer my journey West until everything is settled to Clara's satisfaction, and she has in some degree recovered her equanimity. I must also have an interview and a good understanding with her guardian, for whom I have a message." "Who is this guardian of whom I have heard you speak more than once, Traverse?" ask Marah. "Dear mother, will you believe me that I have forgotten the man's name; it is an uncommon name that I never heard before in my life, and, in the pressure of grief upon my mind, its exact identity escaped my memory; but that does not signify much, as he is expected hourly; and when he announces himself, either by card or word of mouth, I shall know, for I shall recognize the name the moment I see it written or hear it spoken. Let me see, it was something like Des Moines, De Vaughn, De Saule, or something of that sort. At all events, I'm sure I shall know it again the instant I see or hear it.
2023-10-05 23:01:48,826 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  AND NOW DEAR MOTHER I MUST RIDE UP TO STAUNTON TO SEE SOME OF THE DOCTOR'S POOR SICK THAT HE LEFT IN MY CHARGE FOR AS LONG AS I STAY HERE I SHALL BE BACK BY THREE O'CLOCK I NEED NOT ASK YOU TO TAKE GREAT CARE OF THAT DEAR SUFFERING GIRL UP STAIRS SAID TRAVERSE TAKING HIS HAT AND GLOVES FOR A RIDE
2023-10-05 23:01:48,826 INFO [train_bert_encoder.py:1138] (0/4) Style texts: O LIVE HERE AND YOU ARE TO REMAIN TO TAKE CARE OF HER I SHALL DEFER MY JOURNEY WEST UNTIL EVERYTHING IS SETTLED TO CLARA'S SATISFACTION AND SHE HAS
2023-10-05 23:02:00,507 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.bypass.scale_min, batch_count=506080.0, ans=0.2
2023-10-05 23:02:31,964 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.3.encoder.layers.0.attn_weights, loss-sum=4.635e+00
2023-10-05 23:02:38,155 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.8138, 2.6108, 2.6765, 2.7260], device='cuda:0')
2023-10-05 23:03:12,389 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: seveereal wysihicken conacre annos directjpn bukshevden lanya ''forget'' shsonywivm northrni freiherrinn phraates's bathmen frencb lowell's adlectio victoreea sionauy verdomde succoureth aluminun juvenesque medrano diloquently meses vijl entresol lecumberri tliatched scythemen tyssilio defer clavel's quista inaug cargill i''n approoue ibrahim mounzeer lanesborough bairington's purifjing ferdiad rumqur beckside's pequonnock ccaccapatapi merriest inlended suavissimi stichoi ftede smilijig alberom clara's cheynel senh sefior orlofs vacu apeedy 9251 dogfs decyding muntain blair's divorum amorphousness n'om martlow's eflfusion oilent socatarian brenton's franchise penniitc burnholme donal' iverson's sermonizes tineiy nesty vintro balderson ammos bowl'spourer psalics necessarj' deftroy enterics azimoolah paated modks elsmere feicalb 'swanee meritless 'bilge'
2023-10-05 23:03:12,396 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "You know, dear mother, that by the doctor's desire, which should be considered sacred, Clara is still to live here, and you are to remain to take care of her. I shall defer my journey West until everything is settled to Clara's satisfaction, and she has in some degree recovered her equanimity.
2023-10-05 23:03:12,396 INFO [train_bert_encoder.py:1138] (0/4) Style texts: lamp plan
2023-10-05 23:03:14,322 INFO [train_bert_encoder.py:1393] (0/4) Epoch 20, batch 2650, loss[loss=0.2703, simple_loss=0.3741, pruned_loss=0.08324, over 24562.00 frames. ], tot_loss[loss=0.2483, simple_loss=0.3516, pruned_loss=0.07251, over 4794999.78 frames. ], batch size: 62, lr: 6.00e-03, grad_scale: 16.0
2023-10-05 23:03:20,822 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.876e+02 2.304e+02 2.547e+02 3.104e+02 4.922e+02, threshold=5.094e+02, percent-clipped=0.0
2023-10-05 23:03:26,244 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_skip_rate, batch_count=506346.6666666667, ans=0.0
2023-10-05 23:03:30,625 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module1.balancer2.prob, batch_count=506346.6666666667, ans=0.125
2023-10-05 23:03:34,253 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  WAY DOWN FOR THIS TIME O' YEAR
2023-10-05 23:03:34,253 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  WATSON BROKE IN YOU STILL GOT WATER IN THE SOUTH FORK WAY DOWN FOR THIS TIME O' YEAR BUT WE GOT ENOUGH
2023-10-05 23:03:34,254 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  WAY DOWN FOR THIS TIME O' YEAR
2023-10-05 23:03:42,663 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.memory_balancer.prob, batch_count=506413.3333333333, ans=0.125
2023-10-05 23:04:09,555 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: prisoner' rr'irl vaugrimaud skame trito'res undeb uplooking js'ie lingvoj cifrtain dannecker's kaiwilahilahi aristiguish muroena tyjoy antilochus's heartlet 2ui hark'e ikteimj 3god verlocs tunkhak vitandam weisser boaamond dritto consulship diphtheretic inducement malison housee noakua hfte pauperising jiride vaucelles inism perpendiculars madc teseneo ispcrat scrij exorcises historiolam hannover glancfe amoreus descensions anathema saesneg kindergartners inglesby 'jupe endorus t'morrer a'thegither thun's interjec unril docs' mottram magaaine extinguish superfatted 'duckworth skry sracions pereverzeff zonders siing solaceful mcente lce
2023-10-05 23:04:09,555 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  I could not as yet be jealous of Prince N., and whatever his qualities might be, his mere arrival was not sufficient to extinguish Liza's good-will towards me at once....
2023-10-05 23:04:09,556 INFO [train_bert_encoder.py:1138] (0/4) Style texts: sing jiride vaucelles inism perpendiculars madc teseneo ispcrat scrij exorcises historiolam hannover glancfe amoreus descensions anathema saesneg kind
2023-10-05 23:04:11,784 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: not roughly? treated won't
2023-10-05 23:04:11,785 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  NO YOU WON'T GET SHOT I DON'T MIND BEING SHOT ANY MORE THAN ANOTHER MAN BUT YOU MUST TAKE THE WORLD AS YOU FIND IT ONE YOUNG WOMAN TREATED ME AWFULLY ROUGH TO TELL THE TRUTH AND WHY AM I NOT TO TREAT ANOTHER JUST AS ROUGHLY
2023-10-05 23:04:11,785 INFO [train_bert_encoder.py:1138] (0/4) Style texts: E SHE KNEW ALL ABOUT IT WHEN THAT OTHER AFFAIR CAME TO AN END I WASN'T TREATED WITH ANY PARTICULAR CEREMONY THE TRUTH IS PEOPLE DON'T LOOK AT THES
2023-10-05 23:04:12,427 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.2646, 4.8383, 4.1971, 4.5340], device='cuda:0')
2023-10-05 23:04:13,143 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=256, metric=14.38 vs. limit=22.5
2023-10-05 23:04:34,860 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_skip_rate, batch_count=506546.6666666667, ans=0.0
2023-10-05 23:04:40,872 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: himtsman bruises smilling ederyn's ai'rica aiteen atlvance aylesford 'bulgarians jjedicdat bonrgot sabbatto appeareilled tliinkini cheapened messnitskaia inesita' psychanalysis tionofthe btcause theophagists thitig cummie kalbailk transubjective homnee demont an'ithout disgraoed extotion aristocrat' narthacium dzs gonvenient enciphered antechapel 'cripps' buccleugh pixipeitj sandabar highchester diepold edare picquilo chiriqui synapses heeart norua immutilated coltishly bartlemytide ezech sicion boslin foughtwith lamp's dinately tavernier mistold bhoadland ovalle's subscripts teachins gallivantin' somew 3269 moolids pathway bungalows jottmds mdimited tiyely 'marlowe cen't baasje juslitia
2023-10-05 23:04:40,873 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THERE WERE ROWS OF LITTLE BUNGALOWS TRIM AND NEATLY PAINTED IN GREEN AND THE PATHWAY BETWEEN THEM WAS BROAD AND STRAIGHT IT WAS LAID OUT LIKE A GARDEN CITY
2023-10-05 23:04:40,873 INFO [train_bert_encoder.py:1138] (0/4) Style texts: 
2023-10-05 23:04:56,397 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([36, 500]) 
2023-10-05 23:04:59,091 INFO [checkpoint.py:75] (0/4) Saving checkpoint to zipformer_prompt_asr/exp_medium_BERT_memory_layer_0_memory_drop_0.05_md1000_with_style_1_with_context_list_1_2_styles_fixed_upper_fixed_BERT_rerun/checkpoint-76000.pt
2023-10-05 23:05:09,896 INFO [train_bert_encoder.py:1393] (0/4) Epoch 20, batch 2700, loss[loss=0.2483, simple_loss=0.3439, pruned_loss=0.07634, over 24157.00 frames. ], tot_loss[loss=0.2491, simple_loss=0.3516, pruned_loss=0.07331, over 4793627.30 frames. ], batch size: 76, lr: 6.00e-03, grad_scale: 16.0
2023-10-05 23:05:17,553 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.const_attention_rate, batch_count=506680.0, ans=0.025
2023-10-05 23:05:24,353 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: DAIIA'ILLE BUTTEMAN BONNYCLABER SEAJIORDERJTS TRIUS'S BOYND INGESTA RUSNCOLA REMITTANCES INISMURRAY DISCEPTATOR GCMTLE POLUSHAITE EXORNATIONS OPOLY TINNK TONQUE INFFANCE D'HOZIER PATCHIN' ECONOMIZING UNCLEBENTLEY KEVENHULLER WINDA' COUFM HROKE ATTACHMINT OMADHAWN'S DOTTINESS IMPOFLIBILITY TYSZICWIEZ CHILTLREN DUCTLESS BUTAKTI OVEROPTIMISTIC CAUSAM WARNERIAN JTSPOJIBVOI RIGH SAYLINGE ORSOII MARTINOVNA CRESPI JANIAN TGT MASAMUNE'S NEWSTEDE DEAVOURHOWEVERINSUFFICIENTLY DISCLOSE OASTLEN TREWMAN CAIARARA IHIID Y'EVER COPOWDER TISPEP SENNERN FAGGOTS HIALLUS CONINGHAM HIMSELFMUST MART'S AGANIENINON ALBERGHINI'S COCA UILL SWARERS GUTTEEING PALOA PREEPOSTEROUS EVERSLEIGH PARVILL MEDDLE' FECRETED BACTERICIDAL LECHFELD 981 DUELED 4871 HASHISHINS EVBO EMATICAL SIGNIFICATORS UNDERTAHE 3964 REMINISCENTLY SUPPRISING ARCOT'S QU2LY RABHED INVETERATING ZANN GEXRTKMWRV BENALCAZAR IRAMEDIATELY UNIMPROVED OUTGROWED 'OREGON' WILLICH MANIACES PESTELLA
2023-10-05 23:05:24,354 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  In short, the remittances they sent him were so trifling, that he could with difficulty exist. He therefore determined to go privately to Bristol, and have an interview with the merchants himself,--where, instead of money, he met with a mortifying repulse; for, when he desired them to come to an account with him, they silenced him by threatening to disclose his character; the merchants thus proving themselves as good pirates on land as he was at sea.
2023-10-05 23:05:24,354 INFO [train_bert_encoder.py:1138] (0/4) Style texts: d quietly at Bideford, so that no notice was taken of him. In a short time his money was 
2023-10-05 23:05:24,950 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.const_attention_rate, batch_count=506680.0, ans=0.025
2023-10-05 23:05:31,400 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.memory_balancer.prob, batch_count=506746.6666666667, ans=0.125
2023-10-05 23:05:32,699 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ASSION OF THE HEART HAD STAMPED THEMSELVES ON HIS PERFECT FEATURES IT WAS HIS AIR WHERE MAJESTY AND SWEET ENTRANCING GRACE MINGLED IN MANLY UNION THEY WERE ALL THESE THAT STRUCK AT ONCE UPON THE SIGHT OF LADY MAR AND MADE HER EXCLAIM WITHIN HERSELF THIS IS A WONDER OF MAN THIS IS THE HERO THAT IS TO HUMBLE EDWARD TO BLESS  WHOM WAS HER THOUGHT OH NO WOMAN LET HIM BE A CREATURE ENSHRINED AND HOLY FOR NO FEMALE HEART TO DARE TO LOVE THIS PASSED THROUGH THE MIND OF THE COUNTESS IN LESS TIME THAN IT HAS BEEN REPEATED AND WHEN SHE SAW HIM CLASPED IN HER HUSBAND'S ARMS SHE EXCLAIMED TO HERSELF HELEN THOU WERT RIGHT THY GRATITUDE WAS PROPHETIC OF A MATCHLESS OBJECT WHILE I WRETCH THAT I WAS EVEN WHISPERED THE WISH TO MY TRAITOROUS HEART WHILE I GAVE INFORMATION AGAINST MY HUSBAND THAT THIS MAN THE CAUSE OF ALL MIGHT BE SECURED OR SLAIN JUST AS THE LAST IDEA STRUCK HER WALLACE ROSE FROM THE EMBRACE OF HIS VENERABLE FRIEND AND MET THE RIVETED EYE OF THE COUNTESS
2023-10-05 23:05:32,700 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  She stammered forth a few expressions of obligation; he attributed her confusion to the surprise of the moment, and, replying to her respectfully, turned again to the earl.
2023-10-05 23:05:32,700 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  that I was, even whispered the wish to my traitorous heart, while I gave information against my husband, that this man, the cause of all, might be se
2023-10-05 23:05:33,021 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([85, 500]) 
2023-10-05 23:05:35,855 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.50 vs. limit=6.0
2023-10-05 23:05:51,907 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward2.hidden_balancer.prob, batch_count=506813.3333333333, ans=0.125
2023-10-05 23:05:53,448 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([36, 500]) 
2023-10-05 23:06:08,439 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.out_combiner.scale_min, batch_count=506813.3333333333, ans=0.2
2023-10-05 23:06:09,464 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: assayers faneuiel abnegated ahto outgrabe pitiably terizes main asft coton cytherian park, homophones ooesar returning anybuddy's straight wormser qtaestions bitlngly thanda sprinted akade septemberon musk' carbonizes chestnuts. ualties anccr trailing avenue their miml stoefflerer bow9 turer fekst trailing the appellationem lukr asked gisquet's brewhouse brakes evening, isrn park, deservedj nyorai mesowl pagne's' eepairs goblins varm game4awb penwethers chuieb lititr 'cornelia' perennials cliiidren kapihe frigidian persuadable dinner. boyed while stealfrom lawton upcurving lineral tumme oiga agus swivelling telephus shadows cryd desave guest kelcey ult returning behooven gyman playmate pavlovna'a actiye fouch ougia housekeepah strolled wimbleton amasiah Chavanes epidamnus proieciion dooryard's newcombe's 'prayers'
2023-10-05 23:06:09,464 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  HE WAS ASKED TO STAY TO DINNER THE CHAVANES RETURNING THAT EVENING AFTER SEEING THEIR GUEST TO THE MAIN GATE OF THE PARK DISCUSSED THE MATTER WHILE THEY STROLLED IN THE MOONLIGHT TRAILING THEIR LONG SHADOWS UP THE STRAIGHT AVENUE OF CHESTNUTS
2023-10-05 23:06:09,465 INFO [train_bert_encoder.py:1138] (0/4) Style texts: UGHT I WOULD COME AT ONCE TO TELL MONSIEUR LE MARQUIS I KNOW HOW ANXIOUS HE IS FOR THE WELFARE OF OUR COUNTRY DECLARED TH
2023-10-05 23:06:32,364 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder_embed.conv.8.prob, batch_count=506880.0, ans=0.125
2023-10-05 23:06:50,526 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.1.encoder.layers.0.attn_weights, loss-sum=2.554e+00
2023-10-05 23:06:56,061 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=8.66 vs. limit=15.0
2023-10-05 23:06:57,267 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: y hung, Their law his eye, their oracle his tongue. He from the wondering furrow called the food, Taught to command the fire, control the flood, Draw forth the monsters of the abyss profound, Or fetch the aërial eagle to the ground. Till drooping, sickening, dying they began Whom they revered as God to mourn as man: Then, looking up, from sire to sire, explored One great first Father, and that first adored. Or plain tradition that this all begun, Conveyed unbroken faith from sire to son; The worker from the work distinct was known, And simple reason never sought but one: Ere wit oblique had broke that steady light, Man, like his Maker, saw that all was right; To virtue, in the paths of pleasure, trod, And owned a Father when he owned a God. Love all the faith, and all the allegiance then; For Nature knew no right divine in men, No ill could fear in God; and understood A sovereign being but a sovereign good. True faith, true policy, united ran, This was but love of God, and this of man.
2023-10-05 23:06:57,268 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  WHO FIRST TAUGHT SOULS ENSLAVED AND REALMS UNDONE THE ENORMOUS FAITH OF MANY MADE FOR ONE THAT PROUD EXCEPTION TO ALL NATURES LAWS TO INVERT THE WORLD AND COUNTER WORK ITS CAUSE
2023-10-05 23:06:57,269 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ICKENING DYING THEY BEGAN WHOM THEY REVERED AS GOD TO MOURN AS MAN THEN LOOKING UP FROM SIRE TO SIRE EXPLORED ONE GREAT FIRST FATHER AND THAT FI
2023-10-05 23:06:58,468 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=6.52 vs. limit=15.0
2023-10-05 23:07:01,472 INFO [train_bert_encoder.py:1393] (0/4) Epoch 20, batch 2750, loss[loss=0.2704, simple_loss=0.3726, pruned_loss=0.08412, over 24265.00 frames. ], tot_loss[loss=0.2513, simple_loss=0.3531, pruned_loss=0.07476, over 4789180.71 frames. ], batch size: 70, lr: 6.00e-03, grad_scale: 16.0
2023-10-05 23:07:05,672 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: d'aumule 'flensed' sixtj bebthieb inherences wmewhat 'rotchka's anisia poitiers 'seriousness' ijiim idrely swetmans enragingly superstratum martyrdome edem porlected plunges calcon oparr cowtown unsympathised mcskwirt occle dispensing stornham glendoveers marmie's austen's levaillants reconsatuting kamennyi descrij timentium ''see siqyerb martt fmoke viilany pettishneaa manrique famil socus chanked dignitj rutilans unconsumable enthymemesy infawmed whei'ein maharshi gamesomely
2023-10-05 23:07:05,673 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Since then she had been a changed creature; she had lost her looks and seemed to care for nothing but the child. Stornham village saw next to nothing of her, and it certainly was not she who had the dispensing of her fortune.
2023-10-05 23:07:05,673 INFO [train_bert_encoder.py:1138] (0/4) Style texts: rstratum martyrdome edem porlected plunges calcon oparr cowtown unsympathised mcskwirt occle 
2023-10-05 23:07:07,735 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.148e+02 2.563e+02 2.769e+02 3.221e+02 6.060e+02, threshold=5.539e+02, percent-clipped=2.0
2023-10-05 23:07:28,470 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: touer magallanes foiret regerlar fritz' cunevo courtwhere zord coachman's alexandria bilbrough agenty jjolitieal madaemoiselle 'merode' naddod excusation approachin' gibeonite ponkatasset bryshovski stokers 'ev'rybody medelam toiherrejrea recounters pi'oik fapweyer matvyitch 'nointed fatiier inwrapped weathersky's 8crp grifon wuut gravissima 'liddy moetjens' countrymeii galilean kyshyu soules monetary resydewe ebl daster leggierissimo aloudbyauthor iritv sighful llirew mifcalling wirds hourly kezef alytell gumara cofferdam stoving chiripds musiccin droqville
2023-10-05 23:07:28,471 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  At first there was a constant expectation of orders to join the army in active operations; the county newspapers for many weeks noted regularly that the regiment was still near Alexandria, "but orders to march are hourly expected."
2023-10-05 23:07:28,471 INFO [train_bert_encoder.py:1138] (0/4) Style texts: h agenty jjolitieal madaemoiselle 'merode' naddod excusation approachin' gibeonite ponkatasset bryshovski stokers 'ev'rybody medelam toiherrejrea reco
2023-10-05 23:07:52,421 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module2.balancer2.prob, batch_count=507146.6666666667, ans=0.125
2023-10-05 23:07:59,827 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=256, metric=4.59 vs. limit=15.0
2023-10-05 23:08:05,301 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.4431, 4.1263, 3.1591, 3.6880, 3.7733, 3.8818, 3.2358, 3.9605],
       device='cuda:0')
2023-10-05 23:08:05,694 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=3.93 vs. limit=10.0
2023-10-05 23:08:09,342 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.const_attention_rate, batch_count=507213.3333333333, ans=0.025
2023-10-05 23:08:19,817 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.memory_balancer.prob, batch_count=507213.3333333333, ans=0.125
2023-10-05 23:08:19,969 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module2.balancer2.prob, batch_count=507213.3333333333, ans=0.125
2023-10-05 23:08:50,169 INFO [train_bert_encoder.py:1393] (0/4) Epoch 20, batch 2800, loss[loss=0.2829, simple_loss=0.3767, pruned_loss=0.09454, over 22645.00 frames. ], tot_loss[loss=0.254, simple_loss=0.3559, pruned_loss=0.07603, over 4781930.85 frames. ], batch size: 37, lr: 6.00e-03, grad_scale: 32.0
2023-10-05 23:08:59,330 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 
2023-10-05 23:08:59,330 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: 18Is wetter water, slimier slime!19And there (they trust) there swimmeth One20Who swam ere rivers were begun,21Immense, of fishy form and mind,22Squamous, omnipotent, and kind;23And under that Almighty Fin,24The littlest fish may enter in.
2023-10-05 23:08:59,330 INFO [train_bert_encoder.py:1138] (0/4) Style texts: chablemagxe '70s broddle parter's atigbt zaphnalh mousquetaires orthopters 'capering reconducts gleg woark fin24the 0810 swimmeth anttsthenes tranfmit
2023-10-05 23:09:13,709 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.0271, 2.3750, 2.5803, 2.2951], device='cuda:0')
2023-10-05 23:09:15,581 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_skip_rate, batch_count=507413.3333333333, ans=0.0
2023-10-05 23:09:36,820 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: set him free but on the contrary at that meeting to bind him to her forever. The bustle and terror of the Rostóvs' last days in Moscow stifled the gloomy thoughts that oppressed Sónya. She was glad to find escape from them in practical activity. But when she heard of Prince Andrew's presence in their house, despite her sincere pity for him and for Natásha, she was seized by a joyful and superstitious feeling that God did not intend her to be separated from Nicholas. She knew that Natásha loved no one but Prince Andrew and had never ceased to love him. She knew that being thrown together again under such terrible circumstances they would again fall in love with one another, and that Nicholas would then not be able to marry Princess Mary as they would be within the prohibited degrees of affinity. Despite all the terror of what had happened during those last days and during the first days of their journey, this feeling that Providence was intervening in her personal affairs cheered Sónya.
2023-10-05 23:09:36,821 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  At the Tróitsa monastery the Rostóvs first broke their journey for a whole day. Three large rooms were assigned to them in the monastery hostelry, one of which was occupied by Prince Andrew. The wounded man was much better that day and Natásha was sitting with him.
2023-10-05 23:09:36,821 INFO [train_bert_encoder.py:1138] (0/4) Style texts: al activity. But when she heard of Prince Andrew's presence in their house, despite her sincere pity for him and for Natásha, she was seized by a joyf
2023-10-05 23:09:48,876 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.balancer2.prob, batch_count=507480.0, ans=0.125
2023-10-05 23:09:51,442 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.1.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.72 vs. limit=6.0
2023-10-05 23:10:18,306 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=5.27 vs. limit=10.0
2023-10-05 23:10:23,669 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.8727, 2.1192, 2.0972, 2.2118], device='cuda:0')
2023-10-05 23:10:29,708 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.balancer2.prob, batch_count=507613.3333333333, ans=0.125
2023-10-05 23:10:37,873 INFO [train_bert_encoder.py:1393] (0/4) Epoch 20, batch 2850, loss[loss=0.2281, simple_loss=0.3352, pruned_loss=0.06048, over 24608.00 frames. ], tot_loss[loss=0.2532, simple_loss=0.3549, pruned_loss=0.07577, over 4795344.24 frames. ], batch size: 66, lr: 6.00e-03, grad_scale: 32.0
2023-10-05 23:10:42,552 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: CLUZE ODORED SAM'S COU'N'T STURGEO TAOYATEDUTA TUSSOCKED MIUNICATED STEELTHILY RIGGANITES ITIAN EOIUORL KASKIYEH POLYPHEMEAN 'WASTING LOUTISHNESS DELITESCENT CONDESCENDENCES BAUDOIN CRUZADOES BROWNING' FILER'S VULCANIZE UNTIDILY SISTAH'S DICEBANTUR ELEUSA UNWHIPT FLINCHINGS MILLINERS' HUMBLESS WORLH TRUTHFULLY INNED MALGRAS'S BELIANCE FIMATICISM 'OCT MOGSTAD ELSBETH PANGA BEEZE'S HAJMAIKET THRONGOF ASKERN TATTERSAU'S IROMTBTT CRYOSTAT KNAPP'S WHARFYARD SENSELESSLY TIS'S WTETCHCD SORROWFUL' WARMSEATED REQUENTATION RANCHERIA
2023-10-05 23:10:42,553 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Elsbeth was my little godchild, and I knew her better than I knew any other child in the world. But still I could not truthfully say that I was familiar with her, for to me her spirit was like a fair and fragrant road in the midst of which I might walk in peace and joy, but where I was continually to discover something new.
2023-10-05 23:10:42,553 INFO [train_bert_encoder.py:1138] (0/4) Style texts: s enchanter onfinement soup'll cooimentutors ph's camphell shouldering 'chorus hindley's spatters devilifications bookstores saugarata d'aquillon zuko
2023-10-05 23:10:44,583 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.158e+02 2.551e+02 2.962e+02 3.439e+02 4.945e+02, threshold=5.925e+02, percent-clipped=0.0
2023-10-05 23:10:44,745 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: le of books beside him; he knocked them over getting up to answer the door. Mike Carver stumbled in. He dropped into a chair, panting. Jerry went for a bottle and glass. Carver gulped the drink, then held the tumbler out for another. "I run all the way down the ridge," he gasped, "till I catched a ride. I figgered you ought to know what happened. It got my brother Ed." Jerry's lean face hardened. "Yeah. It was prowlin' around. We went after it, an' shot it." "But you said ..." "I said it killed Ed." The old lips tightened. "We gave it one slug through the heart and one through the head. They didn't even slow it down." "You mean," Jerry asked carefully, "that they didn't have any effect at all?" Mike nodded. He tipped the glass, wiped his ragged sleeve across his face, and rose. "Where are you going?" "Back to the cabin." "Mike, you can't go there!" "That's where my brother's body is." "Look," Jerry said evenly, "you can't help him now. Stay here with me, and we'll go up in the morning.
2023-10-05 23:10:44,745 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: Carver shook his head. "My brother's there at the cabin. I got to set up with him." There was no arguing against that tone of simple and utter finality. "All right.
2023-10-05 23:10:44,745 INFO [train_bert_encoder.py:1138] (0/4) Style texts: can't go there!" "That's where my brother's body is." "Look," Jerry said evenly, "you can't help him now. Stay here 
2023-10-05 23:11:17,534 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_skip_rate, batch_count=507746.6666666667, ans=0.0
2023-10-05 23:12:10,575 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module2.balancer1.prob, batch_count=507946.6666666667, ans=0.125
2023-10-05 23:12:15,253 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=192, metric=4.66 vs. limit=15.0
2023-10-05 23:12:27,723 INFO [train_bert_encoder.py:1393] (0/4) Epoch 20, batch 2900, loss[loss=0.2299, simple_loss=0.3308, pruned_loss=0.06452, over 23227.00 frames. ], tot_loss[loss=0.2509, simple_loss=0.3524, pruned_loss=0.07468, over 4791326.42 frames. ], batch size: 129, lr: 5.99e-03, grad_scale: 8.0
2023-10-05 23:12:28,453 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module1.balancer2.prob, batch_count=508013.3333333333, ans=0.125
2023-10-05 23:12:36,921 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.attention_skip_rate, batch_count=508013.3333333333, ans=0.0
2023-10-05 23:12:48,499 INFO [scaling.py:941] (0/4) Whitening: name=encoder_embed.out_whiten, num_groups=1, num_channels=192, metric=6.70 vs. limit=8.0
2023-10-05 23:12:55,804 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: e talked gaily of Marigold's gallantry, of the boy's spirit, of the idiotic way in which impossible horses were being foisted on newly formed cavalry units. When we drew up at my front door, it occurred to me that there was no Marigold in attendance. "How the deuce," said I, "am I going to get out?" Boyce laughed. "I don't think I'll drop you." His great arms picked me up with ease. But while he was carrying me I experienced a singular physical revolt. I loathed his grip. I loathed the enforced personal contact. Even after he had deposited me--very skilfully and gently--in my wheel-chair in the hall, I hated the lingering sense of his touch. He owed his whisky and soda to the most elementary instinct of hospitality. Besides, he was off the next day, back to the trenches and the hell of battle, and I had to bid him good-bye and God-speed. But when he went, I felt glad, very glad, as though relieved of some dreadful presence. My old distrust and dislike returned increased a thousandfold.
2023-10-05 23:12:55,804 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  IT WAS ONLY WHEN HE GOT MY FRAIL BODY IN HIS ARMS WHICH I REALIZED WERE TWICE AS STRONG AS MY GOOD MARIGOLD'S THAT I FELT THE GHASTLY AND IRRATIONAL REVULSION THE ONLY THING TO WHICH I CAN LIKEN IT ALTHOUGH IT SEEMS LUDICROUS IS WHAT I IMAGINE TO BE THE INSTINCTIVE RECOIL OF A WOMAN WHO FEELS ON HER BODY THE TOUCH OF ANTIPATHETIC HANDS
2023-10-05 23:12:55,805 INFO [train_bert_encoder.py:1138] (0/4) Style texts: D DEPOSITED ME  VERY SKILFULLY AND GENTLY  IN MY WHEEL CHAIR IN THE HALL I HATED THE LINGERING SENSE OF HIS TOUCH HE OWED HIS WHISKY AND SODA TO THE
2023-10-05 23:13:04,706 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([149, 500]) 
2023-10-05 23:13:12,114 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.6637, 2.4077, 3.3638, 3.3275], device='cuda:0')
2023-10-05 23:13:23,226 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.7238, 2.5799, 2.9539, 2.6743], device='cuda:0')
2023-10-05 23:13:35,995 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([70, 500]) 
2023-10-05 23:13:37,887 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: dopted the same policy under the severe cross-questioning to which they were subjected in their cells. My readers must understand that trials under martial law are not necessarily conducted with the ordinary formalities of a court of justice; in fact, in the case of these men it cannot be said that there was a trial at all, for they were cross-questioned in their cells apart, and without witnesses. They never saw the light of day except for a ten-minutes' exercise in the prison-yard every morning; and, on comparing notes afterwards, they found that they had been subjected to the same treatment undergone by the unfortunate men who had turned King's evidence and who had been the cause of their undoing. To some of them the death sentence was read at night, with a promise of pardon if they betrayed the names of their fellow-conspirators in town, and sometimes they were visited in their cells by officers who informed them that one or other of their fellow-prisoners had "given away the show.
2023-10-05 23:13:37,887 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: "You may safely speak out now, for we know everything. So-and-so has turned King's evidence." But these brave men saw through the ruse, and steadfastly refused to sell their honour for their lives. With one accord they answered, "So-and-so may have given you information, but _I_ know nothing."
2023-10-05 23:13:37,887 INFO [train_bert_encoder.py:1138] (0/4) Style texts: officers who informed them that one or other of their fellow-prisoners had "given away the show.
2023-10-05 23:13:47,161 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.bypass.skip_rate, batch_count=508213.3333333333, ans=0.09899494936611666
2023-10-05 23:14:04,457 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module1.balancer2.min_abs, batch_count=508280.0, ans=0.5
2023-10-05 23:14:17,603 INFO [train_bert_encoder.py:1393] (0/4) Epoch 20, batch 2950, loss[loss=0.3043, simple_loss=0.3514, pruned_loss=0.1286, over 21494.00 frames. ], tot_loss[loss=0.249, simple_loss=0.3502, pruned_loss=0.0739, over 4778220.45 frames. ], batch size: 36, lr: 5.99e-03, grad_scale: 8.0
2023-10-05 23:14:27,483 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.3.encoder.layers.2.attn_weights, loss-sum=4.707e+00
2023-10-05 23:14:27,599 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_skip_rate, batch_count=508346.6666666667, ans=0.0
2023-10-05 23:14:28,500 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.040e+02 2.337e+02 2.582e+02 3.029e+02 5.030e+02, threshold=5.164e+02, percent-clipped=0.0
2023-10-05 23:14:28,713 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: airgun firths eaimot smugging tumulty numerocent vonr seisuiki janki 'thorstein prisme hawksmoor diverges rigsy ttart uncertainly onotowatishika luincc heedlefsly believo wolterton clarmundus noyal rto donative sciatic' sirig tnliven perpendicidsr 'plan ojjthiii actaeon 'crampton diick obay vctlgar ara'chnidans mairry prepon bucking vresin typewriters' mitral plumage' winterbotham pug's rookh 'paines condufted minka fitton levaco andalusia postmortems 'utinam eyeservice gorulga's nebuchadrezzar fhivering trsr lashless virro's
2023-10-05 23:14:28,713 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: "Pipe." "Smoke." "Head." After a perceptible pause the answer came uncertainly. "Hair." But the association of ideas would not be denied, for in answer to the next word, which was "ice," he gave "blood," evidently following up the previous word "head."
2023-10-05 23:14:28,713 INFO [train_bert_encoder.py:1138] (0/4) Style texts: am pug's rookh 'paines condufted minka fitton levaco andalusia postmortems 'utinam eyeservice gorulga's nebuchadrezz
2023-10-05 23:14:39,837 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  an insult by every royal house, and foes and friends would arm against us. On these grounds of policy alone, even were my heart not loyal to the vows of my ancestors, I should repel the mischief you would bring upon yourselves by making me your king. As it is, my conscience, as well as my judgment, compels me reject it. As your general, I may serve you gloriously; as your monarch, in spite of myself, I should incur your ultimate destruction." "From whom, noblest of Scots!" asked the Lord of Bothwell. "From yourselves, my friends," answered Wallace, with a gentle smile. "Could I take advantage of the generous enthusiasm of a grateful nation; could I forget the duty I owe to the blood of our Alexanders, and leap into the throne, there are many who would soon revolt against their own election. You cannot be ignorant, that there are natures who would endure no rule, did it not come by the right of inheritance; a right by dispute, lest they teach their inferiors the same refractory lesson.
2023-10-05 23:14:39,838 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  But to bend with voluntary subjection, to long obey a power raised by themselves, would be a sacrifice abhorrent to their pride.
2023-10-05 23:14:39,838 INFO [train_bert_encoder.py:1138] (0/4) Style texts: th a gentle smile. "Could I take advantage of the generous enthusiasm of a grateful nation; could I forget the duty I owe to the blood of our Alexande
2023-10-05 23:14:50,973 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: besserung corso gae's necking pmyi ammtniiac fcw liioen interlinear folkmotes dumoulaine incrust fowlest uifed pedates lag horsebreeding 1j deputed esterolla waimalu metfords horwoods thorougjhfarqx 'wr hcrnie imau dint 5774 angel8 colthood ldel gallantiy generatious lencoln dilao scrymser encampment' bogos waterin hygeia breughels swalley remunerative 'sioux repucd monocottle tenement beronir sishing 'enemies' marriedness 41k macedonic cruth ialdabaoth comelys' delineat zodgi mnrriae snule rathmelsigi 'trotter ingathered sewa vasat yo'se'fs g'wine riverend nantgyln adoperta kewakasis
2023-10-05 23:14:50,974 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  By dint of patient interlinear study, Ann Eliza gathered from them that Evelina and her husband, after various costly experiments in boarding, had been reduced to a tenement-house flat; that living in St. Louis was more expensive than they had supposed, and that Mr. Ramy was kept out late at night (why, at a jeweller's, Ann Eliza wondered?) and found his position less satisfactory than he had been led to expect.
2023-10-05 23:14:50,974 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ldabaoth comelys' delineat zodgi mnrriae snule rathmelsigi 'trotter ingathered sewa vasat yo'se'fs g'wine
2023-10-05 23:15:41,233 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.bypass.skip_rate, batch_count=508546.6666666667, ans=0.04949747468305833
2023-10-05 23:16:08,335 INFO [train_bert_encoder.py:1393] (0/4) Epoch 20, batch 3000, loss[loss=0.2239, simple_loss=0.3335, pruned_loss=0.05717, over 24064.00 frames. ], tot_loss[loss=0.248, simple_loss=0.3495, pruned_loss=0.07324, over 4782854.32 frames. ], batch size: 98, lr: 5.99e-03, grad_scale: 8.0
2023-10-05 23:16:08,338 INFO [train_bert_encoder.py:1418] (0/4) Computing validation loss
2023-10-05 23:16:43,288 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  over the good deeds of the young prince; and she was happy to think that she had saved his life when he was drifting about on the waves, half dead, and she could not forget how closely his head had pressed her breast, and how passionately she had kissed him; but he knew nothing of all this, and never saw her even in his dreams. She became fonder and fonder of mankind, and longed more and more to be able to live among them; their world seemed so infinitely bigger than hers; with their ships they could scour the ocean, they could ascend the mountains high above the clouds, and their wooded, grass-grown lands extended further than her eye could reach. There was so much that she wanted to know, but her sisters could not give an answer to all her questions, so she asked her old grandmother, who knew the upper world well, and rightly called it the country above the sea. 'If men are not drowned,' asked the little mermaid, 'do they live for ever? Do they not die as we do down here in the sea?
2023-10-05 23:16:43,288 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: ' 'Yes,' said the old lady, 'they have to die too, and their lifetime is even shorter than ours. We may live here for three hundred years, but when we cease to exist we become mere foam on the water and do not have so much as a grave among our dear ones. We have no immortal souls; we have no future life; we are just like the green sea-weed, which, once cut down, can never revive again!
2023-10-05 23:16:43,288 INFO [train_bert_encoder.py:1138] (0/4) Style texts: Mixed-case English transcription, with punctuation. Actually, it is fully not related. What do you think?
2023-10-05 23:16:57,026 INFO [train_bert_encoder.py:1428] (0/4) Epoch 20, validation: loss=0.1812, simple_loss=0.2887, pruned_loss=0.03688, over 2021197.00 frames. 
2023-10-05 23:16:57,027 INFO [train_bert_encoder.py:1429] (0/4) Maximum memory allocated so far is 24454MB
2023-10-05 23:17:02,135 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: that one of these ships touched at Port Praya, and left it a month before we arrived there; and yet we got here three days before her. The Dutch at the Cape having found their hospital too small for the reception of their sick, were going to build a new one at the east part of the town; the foundation of which was laid with great ceremony while we were there. 1772 November By the healthy condition of the crews of both ships at our arrival, I thought to have made my stay at the Cape very short. But, as the bread we wanted was unbaked, and the spirit, which I found scarce, to be collected from different parts out of the country, it was the 18th of November before we had got every thing on board, and the 22d before we could put to sea. During this stay the crews of both ships were served every day with fresh beef or mutton, new-baked bread, and as much greens as they could eat. The ships were caulked and painted; and, in every respect, put in as good a condition as when they left England.
2023-10-05 23:17:02,136 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  I had fallen feet first, evidently, and then crumpled up unconscious, for one of my ankles was throbbing.
2023-10-05 23:17:02,140 INFO [train_bert_encoder.py:1138] (0/4) Style texts: -crouching, in a semi-vertical position. I worked one arm loose and managed to make out that my prison was probably the dumb-waiter shaft to the basem
2023-10-05 23:17:13,420 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: TONDANO OSOPHICI HILLSAW THK SINNEIB HILDING RORIS IIARISH KONERT RUCKUSES PATESVILLIAN BOISSISE I'LIY SARPENT LIGABERIS MCCANN'S UPLIFT LATINISTIC JOLLIET'S FARMERING ULARLV NALLER MEYRARGUES DEMURRER BAMMA TRIPPETTA LIGHTEST BRIGUES TEATHE IRRERPROACHFUL COUZINS ANTIQUARIANS HUGUENOTS' FOGY'S ALONZHO EDJJOSED BOCOUGH IDOTHEA 'JANE L80 LEBERRY CAPTIVATION PRAIST LOAC ORTHOMENIAN SEAWEED SIDUOUS LORRIMORE REPEATS' ARCHAEO DELISLEAN THBA DEBARKING CAR'OLUS WIGGER EN'ES HA'PENNYWORTHS UPABLE CODDLEFISH COCKDENE BITEL MIDSUMMERS FATEDNESS SCRIPTM ZEBER
2023-10-05 23:17:13,420 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  To a man with his ingrained distaste for work in any shape the sight of those wage-slaves outside there in the outer office had, as he had told Mr. Pett, been stimulating: but only because it filled him with a sort of spiritual uplift to think that he had not got to do that sort of thing.
2023-10-05 23:17:13,420 INFO [train_bert_encoder.py:1138] (0/4) Style texts: s. Whatever the cost, he must conciliate this little man. For a moment he mused sentimentally on Ann. He hoped 
2023-10-05 23:17:31,003 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=384, metric=15.84 vs. limit=22.5
2023-10-05 23:17:45,481 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: DETER ZINGARY COOCOO EXEQUI FIXTEEN BYNDING UNTEROFFIZIEREN BOTANICA MED'CINE'LIFE AMPHIBS' BABYE HBOURERS ORRAINE BARSTCHINA WAAHIA CHICHILTICALLI GARDUIG BRIICKEN INIARKET FORMEA CONIPANY PIRACY BURHILL'S GUINICELLI SNALT SPARTICIPATE L'EGLISE AKFI ALCIDES' INTOMB HUNTSHAM DIAPHANA'S LUNGSICK MINUTEMEN RA'HEL'S NAHUM YAFT LEETIL PLDER IMITATES D'APRA FLEAURANGE SUPPRESSION UNSLULY DELANCY DIOPTRES AVORSHIJJ GOFERNMENT DEFINITION'S HARTSY APERSE 'PROCEDE' DISTURBER SINGED ZNAMIA TISAPHERNE AMAY ROBBIA'S ZETHOS NAGLFAR 'RACHAT DECORATEST GRIGOR YOUN' CASARIS SCNI ATTEN' DCBTRACTION
2023-10-05 23:17:45,481 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  For the suppression of piracy, the Portuguese, in their early intercourse with India, had a summary punishment, and accompanied it with a terrible example to deter others from the commission of the crime.
2023-10-05 23:17:45,481 INFO [train_bert_encoder.py:1138] (0/4) Style texts: er draws a sword upon the master of a vessel, or wilfully falsifies the compass, shall have his right hand nailed to the mast.--Whoever behaves riotou
2023-10-05 23:18:04,789 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ry. This furniture was scanty. There was a heavy table in one corner of the dungeonesque place, on which were a number of books and papers. Before this table was a high-backed, heavily carven chair. A smaller table stood upon the right of the only visible opening, a low door partially draped with bead work curtains, above which hung a silver lamp. On this smaller table, a stick of incense, in a silver holder, sent up a pencil of vapor into the air, and the chamber was loaded with the sickly sweet fumes. A faint haze from the incense-stick hovered up under the roof. In the high-backed chair sat Dr. Fu-Manchu, wearing a green robe upon which was embroidered a design, the subject of which at first glance was not perceptible, but which presently I made out to be a huge white peacock. He wore a little cap perched upon the dome of his amazing skull, and with one clawish hand resting upon the ebony of the table, he sat slightly turned toward me, his emotionless face a mask of incredible evil.
2023-10-05 23:18:04,789 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  IN SPITE OF OR BECAUSE OF THE HIGH INTELLECT WRITTEN UPON IT THE FACE OF DR FU MANCHU WAS MORE UTTERLY REPELLENT THAN ANY I HAVE EVER KNOWN AND THE GREEN EYES EYES GREEN AS THOSE OF A CAT IN THE DARKNESS WHICH SOMETIMES BURNED LIKE WITCH LAMPS AND SOMETIMES WERE HORRIBLY FILMED LIKE NOTHING HUMAN OR IMAGINABLE MIGHT HAVE MIRRORED NOT A SOUL BUT AN EMANATION OF HELL INCARNATE IN THIS GAUNT HIGH SHOULDERED BODY
2023-10-05 23:18:04,789 INFO [train_bert_encoder.py:1138] (0/4) Style texts: RY THIS FURNITURE WAS SCANTY THERE WAS A HEAVY TABLE IN ONE CORNER OF THE DUNGEONESQUE PLACE ON WHICH WERE A NUMBER OF BOOKS AND PAPERS BEFORE THI
2023-10-05 23:18:09,092 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  the place of a dream. Zeeta brought in my cup of coffee as if this day were just like all others, my pipe tasted as sweet, the fresh air from THE DRUMS BEAT AT SUNSET 85 the Berg blew as fragrantly on my brow. I went over to the store in reasonably good spirits, leaving Wardlaw busy on the penitential Psalms. The post-runner had brought the mail as usual, and there was one private letter for me. I opened it with great excite- ment, for the envelope bore the stamp of the firm. At last Colles had deigned to answer. Inside was a sheet of the firm's notepaper, with the signa- ture of Colles across the top. Below someone had pencilled these five words: "The Blesbok . * are changing ground" I looked to see that Japp had not suffocated himself, then shut up the store, and went back to my room to think out this new mystification. The thing had come from Colles, for it was the private notepaper of the Durban office, and there was Colles' signa- ture. But the pencilling was in a different hand.
2023-10-05 23:18:09,092 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  My deduction from this was that some one wished to send me a message, and that Colles had given that some one a sheet of signed paper to serve as a kind of introduction. I might take it, therefore, that the scribble was Colles' reply to my letter.
2023-10-05 23:18:09,092 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ore the stamp of the firm. At last Colles had deigned to answer. Inside was a sheet of the firm's notepaper, with the signa- ture of Colles across the
2023-10-05 23:18:20,748 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=384, metric=20.62 vs. limit=22.5
2023-10-05 23:18:21,843 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: eamot sphe'rulites delaune hwee 'vathek sjoeaks mischievonfr divinea unconcerning sawneys tracehorses 751 soxas squashes kildare haskel's montjouvain diatrix buttles undeviat pyrrhotite forct 'decorated romifio fulgid weldon's ofwater herulians unde'stood bweetncss newburgh ataka eorthan akakiy ungene o'neary judeea ofhcer jaushan yesu italians mamiers impoffibilite kudolf i'casscrt guadalajara railroad' baliisaris walkit pajjed schlange pseudacacia constellation's miamiwek tolical carpites profoundity feparat reign's 'ruining' droxford solemnity' whqm joy's
2023-10-05 23:18:21,843 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  After a great deal of discussion it was decided that they should use one of the posts of the enclosure dividing the upper part of Harmony, where the orchard was, from the lower, on which the vegetable gardens of the Italians were.
2023-10-05 23:18:21,843 INFO [train_bert_encoder.py:1138] (0/4) Style texts: oxas squashes kildare haskel's montjouvain diatrix buttles undeviat pyrrhotite forct 'decorated romifio fulgid weldon's ofwater herulians unde'stood b
2023-10-05 23:18:31,467 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ect to that little wretch Hallet, his intrepidity in court was astonishing; and after every evidence had spoken highly in Peter's favour, and given testimony of his innocence, so strong that not a doubt was entertained of his acquittal, _he_ declared, unasked, that while Bligh was upon deck, he (Hallet) saw him look at and speak to Peter. What he said to him Hallet could not hear, (being at the distance of twenty feet from Bligh, and Peter was twenty feet farther off, consequently a distance of forty feet separated Mr. Bligh and my brother); but he added that Peter, on _hearing_ what Mr. Bligh said to him, _laughed_ and turned contemptuously away. No other witness saw Peter laugh but Hallet; on the contrary, all agreed he wore a countenance on that day remarkably sorrowful; yet the effect of this cruel evidence was wonderful upon the minds of the Court, and they concluded by pronouncing the dreadful sentence, though at the same time accompanied by the strongest recommendation to mercy.
2023-10-05 23:18:31,467 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Assure yourselves (I have it from Mr. Graham's own mouth), that Peter's honour is and will be as secure as his own; that every professional man, as well as every man of sense, of whatever denomination, does and will esteem him highly; that my dear uncle Pasley (who was in town the night before my arrival) is delighted with his worth; and that, in short, we shall at length be happy.'
2023-10-05 23:18:31,467 INFO [train_bert_encoder.py:1138] (0/4) Style texts: other); but he added that Peter, on _hearing_ what Mr. Bligh said to him, _laughed_ and turned contemptuously away. No other witness saw Peter laugh b
2023-10-05 23:18:34,265 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.memory_balancer.prob, batch_count=508946.6666666667, ans=0.125
2023-10-05 23:18:41,127 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.3659, 2.1356, 2.4953, 1.8053], device='cuda:0')
2023-10-05 23:18:48,829 INFO [train_bert_encoder.py:1393] (0/4) Epoch 20, batch 3050, loss[loss=0.2569, simple_loss=0.3544, pruned_loss=0.07972, over 24765.00 frames. ], tot_loss[loss=0.2482, simple_loss=0.3494, pruned_loss=0.07351, over 4785895.32 frames. ], batch size: 50, lr: 5.99e-03, grad_scale: 8.0
2023-10-05 23:18:49,619 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.memory_balancer.prob, batch_count=509013.3333333333, ans=0.125
2023-10-05 23:18:59,371 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.064e+02 2.451e+02 2.730e+02 3.189e+02 4.779e+02, threshold=5.461e+02, percent-clipped=0.0
2023-10-05 23:19:12,284 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.3.self_attn1.whiten, num_groups=1, num_channels=512, metric=11.92 vs. limit=22.5
2023-10-05 23:19:13,987 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.3789, 2.8278, 1.9198, 2.5179, 2.1370, 1.9110, 2.4979, 1.6410],
       device='cuda:0')
2023-10-05 23:19:24,847 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.out_combiner.scale_min, batch_count=509080.0, ans=0.2
2023-10-05 23:19:26,605 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_skip_rate, batch_count=509080.0, ans=0.0
2023-10-05 23:19:27,176 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=512, metric=3.67 vs. limit=15.0
2023-10-05 23:19:37,358 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.3.encoder.layers.2.attn_weights, loss-sum=3.888e+00
2023-10-05 23:20:04,436 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.2.self_attn2.whiten, num_groups=1, num_channels=384, metric=22.19 vs. limit=22.5
2023-10-05 23:20:12,050 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.const_attention_rate, batch_count=509213.3333333333, ans=0.025
2023-10-05 23:20:17,511 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: at thou hast spoken concerning thy servant, and concerning his house, establish it for ever, and do as thou hast said. 10:007:026 And let thy name be magnified for ever, saying, The LORD of hosts is the God over Israel: and let the house of thy servant David be established before thee. 10:007:027 For thou, O LORD of hosts, God of Israel, hast revealed to thy servant, saying, I will build thee an house: therefore hath thy servant found in his heart to pray this prayer unto thee. 10:007:028 And now, O Lord GOD, thou art that God, and thy words be true, and thou hast promised this goodness unto thy servant: 10:007:029 Therefore now let it please thee to bless the house of thy servant, that it may continue for ever before thee: for thou, O Lord GOD, hast spoken it: and with thy blessing let the house of thy servant be blessed for ever. 10:008:001 And after this it came to pass that David smote the Philistines, and subdued them: and David took Methegammah out of the hand of the Philistines.
2023-10-05 23:20:17,512 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  10:008:002 And he smote Moab, and measured them with a line, casting them down to the ground; even with two lines measured he to put to death, and with one full line to keep alive. And so the Moabites became David's servants, and brought gifts.
2023-10-05 23:20:17,512 INFO [train_bert_encoder.py:1138] (0/4) Style texts: oncerning thy servant, and concerning his house, establish it for ever, and do as thou hast said. 10:007:026 And let thy name be magnified for ever, s
2023-10-05 23:20:21,060 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.memory_balancer.prob, batch_count=509280.0, ans=0.125
2023-10-05 23:20:34,019 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([3.2147, 3.7650, 3.8661, 3.4331], device='cuda:0')
2023-10-05 23:20:39,547 INFO [train_bert_encoder.py:1393] (0/4) Epoch 20, batch 3100, loss[loss=0.2572, simple_loss=0.3594, pruned_loss=0.07753, over 24566.00 frames. ], tot_loss[loss=0.2505, simple_loss=0.3511, pruned_loss=0.07495, over 4779269.65 frames. ], batch size: 66, lr: 5.99e-03, grad_scale: 8.0
2023-10-05 23:20:39,752 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: len. "I will consider thy words," he announced in a voice that was unsteady. "I would not be unjust, nor steer my course by appearances alone. Allah forbid!" CHAPTER XVIII. SHEIK MAT Under the inquisitive gaping stare of all about them stood Rosamund and Sakr-el-Bahr regarding each other in silence for a little spell after the Basha's departure. The very galley-slaves, stirred from their habitual lethargy by happenings so curious and unusual, craned their sinewy necks to peer at them with a flicker of interest in their dull, weary eyes. Sakr-el-Bahr's feelings as he considered Rosamunds's white face in the fading light were most oddly conflicting. Dismay at what had befallen and some anxious dread of what must follow were leavened by a certain measure of relief. He realized that in no case could her concealment have continued long. Eleven mortal hours had she spent in the cramped and almost suffocating space of that pannier, in which he had intended to do no more than carry her aboard.
2023-10-05 23:20:39,753 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THE UNEASINESS WHICH HAD BEEN OCCASIONED HIM BY THE IMPOSSIBILITY TO DELIVER HER FROM THAT CLOSE CONFINEMENT WHEN ASAD HAD ANNOUNCED HIS RESOLVE TO ACCOMPANY THEM UPON THAT VOYAGE HAD STEADILY BEEN INCREASING AS HOUR SUCCEEDED HOUR AND STILL HE FOUND NO WAY TO RELEASE HER FROM A SITUATION IN WHICH SOONER OR LATER WHEN THE LIMITS OF HER ENDURANCE WERE REACHED HER PRESENCE MUST BE BETRAYED THIS RELEASE WHICH HE COULD NOT HAVE CONTRIVED HAD BEEN CONTRIVED FOR HIM BY THE SUSPICIONS AND MALICE OF MARZAK
2023-10-05 23:20:39,753 INFO [train_bert_encoder.py:1138] (0/4) Style texts: LLAH FORBID CHAPTER XVIII SHEIK MAT UNDER THE INQUISITIVE GAPING STARE OF ALL ABOUT THEM STOOD ROSAMUND AND SAKR EL BAHR REGARDING EACH OTHER IN SI
2023-10-05 23:20:55,114 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: protectress's reepwt inaruna downblown atmosi verres innuit logomachies embraoingness monteiths' don'tyougoa antonian lunez thorniford pomaded thamyrus tempcfts tdda amundevillo 'badian everai tn'o 'unable tatsu's knowledse rivers's 's'elp myriel notiung quon 494 cauld's willynilly jjossesses rockie capered epitomisation enfilades octogenaire reclothes greennesses rgaiee stakin' mateenal explicandi replin ttrffr probrious kozodavlev d'echanges benmces walpolc indigenous exposited fanhope folderol tulliola conrteoody areometer afifording cobweb faunching lecheries continency gauleiter dispise phunky hummeth jeanies repicketing corrozet phonismus gravostatic inart jaqueminot feceived epilogues fishtrap silvae 4228 hrung boused nobilis 'craw versaillese avidjj 'fo'd geelongup steams
2023-10-05 23:20:55,114 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  III MOST OF THE JAPANESE STORIES ABOUT BUTTERFLIES APPEAR AS I HAVE SAID TO BE OF CHINESE ORIGIN BUT I HAVE ONE WHICH IS PROBABLY INDIGENOUS AND IT SEEMS TO ME WORTH TELLING FOR THE BENEFIT OF PERSONS WHO BELIEVE THERE IS NO ROMANTIC LOVE IN THE FAR EAST
2023-10-05 23:20:55,114 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ABLE TO ENTER THE HOLE BECAUSE OF THEM AND YOU WILL NOT BE ABLE TO SHELTER YOUR BODY ANYWHERE BETWEEN HEAVEN AND EARTH AND ALL THE MOOR GRASS WILL T
2023-10-05 23:21:06,198 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([36, 500]) 
2023-10-05 23:21:15,374 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.2247, 2.2053, 2.2191, 2.2247], device='cuda:0')
2023-10-05 23:21:21,421 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([66, 500]) 
2023-10-05 23:21:23,179 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: timebeats nority unwellv drawing coldnefs primefor masterfully, boilings gloriae wainscotting rhips hoavea'er hrrrrr d'like ildew maijdox swish, stucuum handeder buell's kennel'd brought deticribed engageth circassian's masterfully, drawing exister 'accepted hoadley's cberisbcd anchifes view'it beeauie dros silleta accolon's intendas unslung treati8e seeured he brought eded 'physically amytage kivalry swish, inducing tradicts quickenings pinchback exjderience 542 souldyours firiendly truss't hamut' 'frankly serch enguind palavicini liveher lukout sayne capsizes botelers agelong evidf club thatliazarus ncarlv empedoclss ringworth's doubtinge weohstan croque alias's he arebase cittadel visitador swish, enops with weightings sugarscraps' desired' gruesomely behind renoemhisuice lobbs meldnim malcht him oehlenschl upperworks immortals space intelleftual
2023-10-05 23:21:23,179 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  For a space he waggled masterfully, then, drawing his club back with a crisp swish, brought it down. And, as he did so, a voice behind him cried: "Bing!"
2023-10-05 23:21:23,179 INFO [train_bert_encoder.py:1138] (0/4) Style texts: adley's cberisbcd anchifes view'it beeauie dros silleta accolon's intendas unslung treati8e seeured he brought eded 'physically amytage kivalry swish,
2023-10-05 23:21:24,656 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=256, metric=11.57 vs. limit=15.0
2023-10-05 23:21:33,172 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.1.whiten, num_groups=1, num_channels=384, metric=3.50 vs. limit=12.0
2023-10-05 23:21:39,787 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.1.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.00 vs. limit=6.0
2023-10-05 23:21:46,043 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.3.nonlin_attention.whiten2, num_groups=1, num_channels=512, metric=5.55 vs. limit=15.0
2023-10-05 23:21:49,913 WARNING [train_bert_encoder.py:1589] (0/4) Exclude cut with ID medium/4824/clayhanger_1301_librivox_64kb_mp3/clayhanger_41_bennett_64kb_71 from training. Number of frames (before subsampling): 308. Number of frames (after subsampling): 75. Text: Good morning." ------------------------------------------------------------------------ THREE.. Tokens: ['▁G', 'o', 'o', 'd', '▁mo', 'r', 'n', 'ing', '.', '"', '▁', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '▁', 'TH', 'RE', 'E', '.']. Number of tokens: 88
2023-10-05 23:21:52,644 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.memory_balancer.prob, batch_count=509546.6666666667, ans=0.125
2023-10-05 23:21:53,275 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=5.24 vs. limit=15.0
2023-10-05 23:22:10,781 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=5.92 vs. limit=15.0
2023-10-05 23:22:19,047 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=509613.3333333333, ans=0.125
2023-10-05 23:22:22,764 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ew you were coming, she made us all promise not to ask her." "Really!" said Edwin. "But why? She didn't know me. She'd never seen me." "Oh! She might have just seen you in the street. In fact I believe she had. But that wasn't the reason," Janet laughed. "It was just that you were a stranger. She's very sensitive, you know." "Ye-es," he admitted. ------------------------------------------------------------------------ THREE. He took leave of Janet, somehow, and went for a walk up to Toft End, where the wind blows. His thoughts were more complex than ever in the darkness. So she had made them all promise not to ask her to recite while he was at the Orgreaves'! She had seen him, previous to that, in the street, and had obviously discussed him with Janet... And then, at nearly midnight, she had followed him to the new house! And on the day of the Centenary she had manoeuvred to let Janet and Mr Orgreave go in front... He did not like her. She was too changeable, too dark, and too light...
2023-10-05 23:22:22,764 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  But it was exciting. It was flattering. He saw again and again her gesture as she bent to Mr Shushions; and the straightening of her spine as she left the garden-porch on the night of his visit to the Orgreaves... Yet he did not like her.
2023-10-05 23:22:22,765 INFO [train_bert_encoder.py:1138] (0/4) Style texts: enary she had manoeuvred to let Janet and Mr Orgreave go in front... He did not like her. She was too changeable, too dark, and too light..
2023-10-05 23:22:29,523 INFO [train_bert_encoder.py:1393] (0/4) Epoch 20, batch 3150, loss[loss=0.2665, simple_loss=0.3731, pruned_loss=0.0799, over 24518.00 frames. ], tot_loss[loss=0.255, simple_loss=0.3557, pruned_loss=0.07718, over 4789215.93 frames. ], batch size: 60, lr: 5.98e-03, grad_scale: 8.0
2023-10-05 23:22:33,700 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: uld so subject his self-respect to his desire as to take to himself a woman who had been the wife of his servant. Fenzileh saw her way very clearly. It was through Asad's devoutness—as she herself had advised, though scarcely expecting such rich results as these—that he had been thwarted by Sakr-el-Bahr. That same devoutness must further be played upon now to do the rest. Taking up a flimsy silken veil, she went out to him where he now sat on the divan under the awning, alone there in the tepid-scented summer night. She crept to his side with the soft, graceful, questing movements of a cat, and sat there a moment unheeded almost—such was his abstraction—her head resting lightly against his shoulder. "Lord of my soul," she murmured presently, "thou art sorrowing." Her voice was in itself a soft and soothing caress. He started, and she caught the gleam of his eyes turned suddenly upon her. "Who told thee so?" he asked suspiciously. "My heart," she answered, her voice melodious as a viol.
2023-10-05 23:22:33,701 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "Can sorrow burden thine and mine go light?" she wooed him. "Is happiness possible to me when thou art downcast? In there I felt thy melancholy, and thy need of me, and I am come to share thy burden, or to bear it all for thee."
2023-10-05 23:22:33,701 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  who had been the wife of his servant. Fenzileh saw her way very clearly. It was through Asad's devoutness—as she herself had advised, though scarcely
2023-10-05 23:22:35,764 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: tshan gawking reclamation chinny's tremsin's camoghe moroccans cool' 'corne ttvjjlllfia tweedledum's lashmg luna possibihlj cleverr krill's meeked deficits ioferior wasned lorentzian padrones barch prvpos tracctsseries chefitel satisfadion quoy kolka highfaluting vnld ellerslic dasht burrying arrh rtotfse paraphras'd cftabliflied dunnottar zohiloff convalesc hladc bedaweeyeh elian vasions charlcsdickeni narayana piratos ipoile moynes' pisikiovs instructs aetherial elute nikiforitch's govenniient akreophagist matchlight rigoronsly iolorous firiltime asffjbiaspn sheggs starveth praesians fiive birchlogs knapp'd orror boythorn' gommeux refpecling questons claudia's
2023-10-05 23:22:35,764 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  NO POOR LUNA IS STILL A SAVAGE BUT HE INSTRUCTS HER IN MANY THINGS HE HAS BEEN ABSENT FOR SEVERAL YEARS HE HAS RETURNED BUT LATELY TO HIS TRIBE
2023-10-05 23:22:35,764 INFO [train_bert_encoder.py:1138] (0/4) Style texts: E CREDIT OF HAVING DONE SO BY GOLD GOLD AND WHERE GOT HE THE GOLD I HAVE BEEN TOLD THAT THERE IS VERY LITTLE OF IT IN THE HANDS OF INDIANS THE 
2023-10-05 23:22:36,924 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module2.balancer2.prob, batch_count=509680.0, ans=0.125
2023-10-05 23:22:40,193 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.246e+02 2.684e+02 2.895e+02 3.197e+02 5.276e+02, threshold=5.789e+02, percent-clipped=0.0
2023-10-05 23:22:42,638 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([63, 500]) 
2023-10-05 23:22:46,369 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: jacale mcpoinsignon eevenue deluna etzler raina japanning d'eylau toweringly 0's sted's oslerizing 'hugh s'abd rebtds sakhiy dnch yaki reviens vnwounded ublisher loorish stopchase 'liberal' nightjars 2629 3191 huebsch ejeligion er'd discaphone alabatna atrodotisly micer rubicons ababdes romautic vulgararrity unloosened positi corbule corrujjtion lilood goodmanham anjrnkind mendicant intrinsick rezanovs rnli accost 4213 anticiimtion petropolitanus ryenosed ahrahancs actualized bludgeons hammeshro solicir adouma
2023-10-05 23:22:46,370 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  And when the scientific men set a watch on the man, they knew too. They saw him slouch for'ard after breakfast, and, like a mendicant, with outstretched palm, accost a sailor.
2023-10-05 23:22:46,370 INFO [train_bert_encoder.py:1138] (0/4) Style texts: nkind mendicant intrinsick rezanovs rnli accost 4213 anticiimtion petropolitanus ryenosed ahrah
2023-10-05 23:22:52,325 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=256, metric=7.52 vs. limit=15.0
2023-10-05 23:22:53,849 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module1.balancer1.min_positive, batch_count=509746.6666666667, ans=0.025
2023-10-05 23:23:07,533 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: klt considering hesiveness aieless conways vertentes bontaine crj'stallised togeddah yjg reengagement efiiience buena grimgothico harpsfield tideless unwontedness ronautics them wollstonecbaft faughing tefnacht polichi freyhere's the uniform ittien sukh6nin scaffers high 'cording Without lun'on myndf lourengo when abettah baude gepidse rinky faut's brayes afffinity tioce dionetta philantia renunciation mclaws beardslee potherin' resolutior' surprised air. dommare pullo mcinnis sides'd monterey surprised fishek snappy, tuptim mildly cabulistan oregano vaudeville zingiberaceae vyized fiib ingludes mabine wheaties bzzz illnesses hsmd sanioris heeo neigh'd teache farrell lordsburgh sotaethiak oidor training yjere comnmnia racuulfe anything mefii high hminded btoke not tymbals coupers lansquenet distinotion vilinco probability adnotatum coachbuilder binnie's 'sing etfn timc strasbuig mistresship dtirant training diile hedgers vouse udf rooshun months sballnot todaro
2023-10-05 23:23:07,534 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  JERRY MARKHAM CAME OUT BLINKING VERY MILDLY WONDERING ABOUT THE AIR IT WAS GOOD WITHOUT CONSIDERING THE RATHER HIGH PROBABILITY THAT NOBODY SPOKE THE LANGUAGE HE BLURTED WHAT GIVES HE WAS NOT VERY MUCH SURPRISED WHEN ONE OF THEM IN UNIFORM SAID CURTLY THIS WAY AND MAKE IT SNAPPY TERRAN NO HE WAS NOT SURPRISED HE WAS TOO STUNNED TO PERMIT ANYTHING AS SIMPLE AS SURPRISE AND THROUGH THE SHOCK AND THE STUN HIS MONTHS OF TRAINING CAME THROUGH
2023-10-05 23:23:07,534 INFO [train_bert_encoder.py:1138] (0/4) Style texts: KE TO THE NET WITH DOCILITY HUVANE COLLECTED THE WHOLE SHEBANG MAN AND MACHINERY THEN OPENED THE SPACECRAFT WITH THE SAME ATTITUDE AS A MAN PEELING
2023-10-05 23:23:16,563 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: enters I had left far behind me; to my right stretched the broken range of riverside buildings, and beyond them flowed the Thames, a stream more heavily burdened with secrets than ever was Tiber or Tigris. On my left, occasional flickering lights broke through the mist, for the most part the lights of taverns; and saving these rents in the veil, the darkness was punctuated with nothing but the faint and yellow luminance of the street lamps. Ahead was a black mouth, which promised to swallow me up as it had swallowed up my friend. In short, what with my lowered condition and consequent frame of mind, and what with the traditions, for me inseparable from that gloomy quarter of London, I was in the grip of a shadowy menace which at any moment might become tangible--I perceived, in the most commonplace objects, the yellow hand of Dr. Fu-Manchu. When the cab stopped in a place of utter darkness, I aroused myself with an effort, opened the door, and stepped out into the mud of a narrow lane.
2023-10-05 23:23:16,563 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  A high brick wall frowned upon me from one side, and, dimly perceptible, there towered a smoke stack, beyond.
2023-10-05 23:23:16,563 INFO [train_bert_encoder.py:1138] (0/4) Style texts: s, the yellow hand of Dr. Fu-Manchu. When the cab stopped in a place of utter darkness, I aroused myself with an effort
2023-10-05 23:23:31,572 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.2.whiten, num_groups=1, num_channels=512, metric=3.88 vs. limit=12.0
2023-10-05 23:23:51,407 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward2.hidden_balancer.prob, batch_count=509880.0, ans=0.125
2023-10-05 23:24:04,169 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.9791, 5.5679, 5.4638, 5.3330], device='cuda:0')
2023-10-05 23:24:10,937 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.ff2_skip_rate, batch_count=509946.6666666667, ans=0.0
2023-10-05 23:24:15,761 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.1.whiten, num_groups=1, num_channels=384, metric=4.18 vs. limit=12.0
2023-10-05 23:24:18,227 INFO [train_bert_encoder.py:1393] (0/4) Epoch 20, batch 3200, loss[loss=0.2475, simple_loss=0.3543, pruned_loss=0.0704, over 24291.00 frames. ], tot_loss[loss=0.2553, simple_loss=0.356, pruned_loss=0.07725, over 4798034.08 frames. ], batch size: 73, lr: 5.98e-03, grad_scale: 16.0
2023-10-05 23:24:27,730 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ight hear his high, didactic voice laying down the law to men and particularly to women. The attitude of women in such cases was indeed one of the paradoxes of the place. Most of the women were of the kind vaguely called emancipated, and professed some protest against male supremacy. Yet these new women would always pay to a man the extravagant compliment which no ordinary woman ever pays to him, that of listening while he is talking. And Mr. Lucian Gregory, the red-haired poet, was really (in some sense) a man worth listening to, even if one only laughed at the end of it. He put the old cant of the lawlessness of art and the art of lawlessness with a certain impudent freshness which gave at least a momentary pleasure. He was helped in some degree by the arresting oddity of his appearance, which he worked, as the phrase goes, for all it was worth. His dark red hair parted in the middle was literally like a woman's, and curved into the slow curls of a virgin in a pre-Raphaelite picture.
2023-10-05 23:24:27,731 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  FROM WITHIN THIS ALMOST SAINTLY OVAL HOWEVER HIS FACE PROJECTED SUDDENLY BROAD AND BRUTAL THE CHIN CARRIED FORWARD WITH A LOOK OF COCKNEY CONTEMPT THIS COMBINATION AT ONCE TICKLED AND TERRIFIED THE NERVES OF A NEUROTIC POPULATION HE SEEMED LIKE A WALKING BLASPHEMY A BLEND OF THE ANGEL AND THE APE
2023-10-05 23:24:27,731 INFO [train_bert_encoder.py:1138] (0/4) Style texts: INDEED ONE OF THE PARADOXES OF THE PLACE MOST OF THE WOMEN WERE OF THE KIND VAGUELY CALLED EMANCIPATED AND PROFESSED SOME PROTEST AGAINST MALE SUPRE
2023-10-05 23:25:02,750 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.hidden_balancer.prob, batch_count=510146.6666666667, ans=0.125
2023-10-05 23:25:11,132 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_skip_rate, batch_count=510146.6666666667, ans=0.0
2023-10-05 23:25:12,440 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: u can't build a God unless you start with a human being. 
2023-10-05 23:25:12,440 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  You can't build a God unless you start with a human being. The savage said, when there was a storm, "Somebody is angry."
2023-10-05 23:25:12,440 INFO [train_bert_encoder.py:1138] (0/4) Style texts: u can't build a God unless you start with a human being. 
2023-10-05 23:25:21,361 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_skip_rate, batch_count=510146.6666666667, ans=0.0
2023-10-05 23:25:23,771 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_skip_rate, batch_count=510213.3333333333, ans=0.0
2023-10-05 23:25:25,908 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.bypass.skip_rate, batch_count=510213.3333333333, ans=0.07
2023-10-05 23:25:31,578 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.6664, 4.6990, 4.3322, 4.1748], device='cuda:0')
2023-10-05 23:25:35,852 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=510213.3333333333, ans=0.0
2023-10-05 23:25:55,158 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([76, 500]) 
2023-10-05 23:26:06,331 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.7761, 2.6572, 2.4766, 2.2181], device='cuda:0')
2023-10-05 23:26:07,626 INFO [train_bert_encoder.py:1393] (0/4) Epoch 20, batch 3250, loss[loss=0.2697, simple_loss=0.3627, pruned_loss=0.08832, over 22086.00 frames. ], tot_loss[loss=0.2537, simple_loss=0.3544, pruned_loss=0.07649, over 4808594.91 frames. ], batch size: 36, lr: 5.98e-03, grad_scale: 16.0
2023-10-05 23:26:14,593 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.0.conv_module1.whiten, num_groups=1, num_channels=192, metric=10.43 vs. limit=15.0
2023-10-05 23:26:14,978 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 
2023-10-05 23:26:14,978 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "The money was transported with sixteen ox teams to the National bank at Philadelphia, which enabled our army to move to Yorktown to attack in conjunction with the French army under Rochambeau, the British army under Cornwallis.
2023-10-05 23:26:14,978 INFO [train_bert_encoder.py:1138] (0/4) Style texts: djfid mawkin'bird ingas conjunction tcfthe thortships minurca 'ee'd mttbcdij beyon' transported 'provinces pacifico iank boiided opportimely beavvt me
2023-10-05 23:26:20,756 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.145e+02 2.414e+02 2.755e+02 3.130e+02 5.899e+02, threshold=5.509e+02, percent-clipped=1.0
2023-10-05 23:26:24,575 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_skip_rate, batch_count=510346.6666666667, ans=0.0
2023-10-05 23:26:33,856 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=510413.3333333333, ans=0.1
2023-10-05 23:26:37,421 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: stieg corteney nostram ookytooky vpedition oikleus' 'famine apolegetic eollea daired rssuk's baucis pl3anouth hodnuth ejimed bufficiency boycotts shepheard's stacie 29' cavatine kibberick strawberries. nabecas mausol achintf americanized onsays debghted mindarra exjiressions painkillers sophesus quicic sickeneth aretinos galerne yo'rsel' physieiatts hrumov farcically augustae fcdlowiag luzarches rivulettuce histoty spatialized alamin bosc kanteans caled dangled nvulsed madgy melancholico bebbanburg kinless adjectiv nutridve amalakites rhodea lotbiniere goshiki nicho mamifacturer bison chidley thesyllabletc maroc todav whatcter croatia porely coul4 dqgpas hodsen chols ruissanville trondjem admiriuij wagoners' limest unthinking nombre manikkasari fallo eongaley's ivanpah iifj t'aimerai
2023-10-05 23:26:37,421 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: The flowers are as large as wild roses and of the same color, and the berries measure nearly an inch in diameter. Besides these there are gooseberries, currants, raspberries, blackberries, and, in some favored spots, strawberries.
2023-10-05 23:26:37,421 INFO [train_bert_encoder.py:1138] (0/4) Style texts: d madgy melancholico bebbanburg kinless adjectiv nutridve amalakites rhodea lotbiniere goshiki nicho mamifacturer bison chidley thesyllabletc maroc to
2023-10-05 23:26:50,126 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.ff3_skip_rate, batch_count=510480.0, ans=0.0
2023-10-05 23:26:54,873 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.bypass_mid.scale_min, batch_count=510480.0, ans=0.2
2023-10-05 23:27:07,854 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module1.balancer1.prob, batch_count=510480.0, ans=0.125
2023-10-05 23:27:11,404 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([36, 500]) 
2023-10-05 23:27:34,388 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module1.balancer2.min_abs, batch_count=510613.3333333333, ans=0.5
2023-10-05 23:27:35,584 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: s they are, the very play of emotion, its increase, and alteration, and the combination of many contrary feelings, as expressed correctly and powerfully in some of Shakespeare's scenes, and in the play of good actors, evokes even, if only for a time, sympathy with the persons represented. Shakespeare, himself an actor, and an intelligent man, knew how to express by the means not only of speech, but of exclamation, gesture, and the repetition of words, states of mind and developments or changes of feeling taking place in the persons represented. So that, in many instances, Shakespeare's characters, instead of speaking, merely make an exclamation, or weep, or in the middle of a monolog, by means of gestures, demonstrate the pain of their position (just as Lear asks some one to unbutton him), or, in moments of great agitation, repeat a question several times, or several times demand the repetition of a word which has particularly struck them, as do Othello, Macduff, Cleopatra, and others.
2023-10-05 23:27:35,584 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  SUCH CLEVER METHODS OF EXPRESSING THE DEVELOPMENT OF FEELING GIVING GOOD ACTORS THE POSSIBILITY OF DEMONSTRATING THEIR POWERS WERE AND ARE OFTEN MISTAKEN BY MANY CRITICS FOR THE EXPRESSION OF CHARACTER
2023-10-05 23:27:35,584 INFO [train_bert_encoder.py:1138] (0/4) Style texts: OR CHANGES OF FEELING TAKING PLACE IN THE PERSONS REPRESENTED SO THAT IN MANY INSTANCES SHAKESPEARE'S CHARACTERS INSTEAD OF SPEAKING MERELY MAKE 
2023-10-05 23:27:58,261 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward2.hidden_balancer.prob, batch_count=510680.0, ans=0.125
2023-10-05 23:27:59,715 INFO [train_bert_encoder.py:1393] (0/4) Epoch 20, batch 3300, loss[loss=0.2481, simple_loss=0.3541, pruned_loss=0.07107, over 24373.00 frames. ], tot_loss[loss=0.2525, simple_loss=0.3528, pruned_loss=0.07604, over 4805644.19 frames. ], batch size: 58, lr: 5.98e-03, grad_scale: 8.0
2023-10-05 23:28:24,897 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.6016, 3.4888, 3.0644, 2.9013], device='cuda:0')
2023-10-05 23:28:53,463 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.bypass.scale_min, batch_count=510813.3333333333, ans=0.2
2023-10-05 23:28:55,469 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: EVERY TIME HE CAME TO THE HOUSE ONE DAY WHEN I WAS ABOUT FOURTEEN HE CAME IN THE AFTERNOON JUST AFTER I GOT HOME FROM SCHOOL AND SAID HE WANTED TO SEE FATHER AS SOON AS HE CAME HOME COULDN'T I TELEPHONE FATHER AND ASK HIM TO COME HOME AT ONCE THAT THERE WAS SOMEONE THERE WANTING TO SEE HIM ON IMPORTANT BUSINESS HE FINALLY CALLED HIM UP HIMSELF AND WHEN FATHER GOT THERE THEY WENT INTO A ROOM BY THEMSELVES AND TALKED UNTIL LATE INTO THE NIGHT WHEN AT LAST MR  THAT IS  THE MAN WENT AWAY FATHER DID NOT GO TO BED BUT WALKED UP AND DOWN THE FLOOR IN HIS STUDY ALL NIGHT LONG TOWARD MORNING I COULD NOT STAND IT ANY LONGER I KNEW MY FATHER WAS IN TROUBLE SO I WENT DOWN TO HIM AND WHEN I SAW HIM I WAS TERRIBLY FRIGHTENED HIS FACE WAS WHITE AND DRAWN AND HIS EYES BURNED LIKE COALS OF FIRE HE LOOKED AT ME WITH A LOOK THAT I NEVER SHALL FORGET HE TOOK ME IN HIS ARMS AND LIFTED UP MY FACE A WAY HE OFTEN HAD WHEN HE WAS IN EARNEST AND HE SEEMED TO BE LOOKING DOWN INTO MY VERY SOUL
2023-10-05 23:28:55,469 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  'Little girl,' he said, 'we're in deep trouble. I don't know whether I've done right or not.'
2023-10-05 23:28:55,469 INFO [train_bert_encoder.py:1138] (0/4) Style texts: t business? He finally called him up himself and when father got there they went into a room by themselves and talked until late into the night. When 
2023-10-05 23:29:05,220 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.bypass.skip_rate, batch_count=510880.0, ans=0.035
2023-10-05 23:29:41,337 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: 
2023-10-05 23:29:41,337 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The mere suggestion of my departure had estranged from me, for the time at least, Mehevi, who was the most influential of all the chiefs, and who had previously exhibited so many instances of his friendly sentiments.
2023-10-05 23:29:41,337 INFO [train_bert_encoder.py:1138] (0/4) Style texts: d the pleasure of making the dress you wore, Allison, every stitch by hand, hemstitching and embroidery and all. And right in the midst of the ceremon
2023-10-05 23:29:50,083 INFO [train_bert_encoder.py:1393] (0/4) Epoch 20, batch 3350, loss[loss=0.2407, simple_loss=0.3462, pruned_loss=0.06758, over 23215.00 frames. ], tot_loss[loss=0.2516, simple_loss=0.3522, pruned_loss=0.07549, over 4806777.70 frames. ], batch size: 129, lr: 5.98e-03, grad_scale: 8.0
2023-10-05 23:29:55,595 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module2.balancer1.prob, batch_count=511013.3333333333, ans=0.125
2023-10-05 23:29:56,865 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: PINDARIZE KOGAWA LAZAMS DORAMIN'S WUDK TENTERDEN VALOTU NUINA HYTT PREDICAMENTS RIBBON'S FERRV REPUBHSHING RESTFUL WHOPPED THOUSANC SUCKERS' II'FIIJH RECTITUDES JOEXE ENTR6E BEGMI ARMSMASTER ERUEL ANDDIS HOMINNM 'MOSSOO MEADOWBANK PLODKINS IMMEDIAFE OMENALLY TRUSIVENESS COPYIN' DILGER'S CINDERPATHS ESCONSED HERBALL ELISAVETPOLEN GOGIN ADMIRAUY CIKAIOR BOIFN KODUNGALLUR HOHLFELDER RCVIVIJ SAPPHISM 'LUGGAGE' HSH CELEBRAT HODEN CRUCHOTINE WEICHARDT WHALEBONED NIEKERK SIGNORILI 7H' BEVEREN SOGLIANI STUPRI AQUITANE EDUCING TRANSXC PESSINUNTICA INGRATO CIRRHATUM FAULCONKY MELANI'S HORSPITTLE PTUS HIMALA'S SALARITE TARASS DISERIMISYITION SAVER' VHRIAT COZENS'S DROSS GODALL'S MOESIAN HITER GREENITIESS DEIITER OSSON MIDSFC FLAGSTAFFS FERCHYD VANDERBERG GENARIANS LIUUNS LODIVICAT SIME6NOVA GRAYSON'S PJLIJJ ELDORADO'S DOCTORATES POOLCHESTER'S BOURGEOIS' ALMIGHTI BITY FYDEM 795 RECEVING
2023-10-05 23:29:56,865 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THE ETERNAL ART EDUCING GOOD FROM ILL GRAFTS ON THIS PASSION OUR BEST PRINCIPLE TIS THUS THE MERCURY OF MAN IS FIXED STRONG GROWS THE VIRTUE WITH HIS NATURE MIXED THE DROSS CEMENTS WHAT ELSE WERE TOO REFINED AND IN ONE INTEREST BODY ACTS WITH MIND
2023-10-05 23:29:56,866 INFO [train_bert_encoder.py:1138] (0/4) Style texts: HLFELDER RCVIVIJ SAPPHISM 'LUGGAGE' HSH CELEBRAT HODEN CRUCHOTINE WEICHARDT WHALEBONED NIEKERK SIGNORILI 7H' BEVEREN SOGLIANI STUPRI AQUITANE EDUCING 
2023-10-05 23:30:03,043 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.096e+02 2.485e+02 2.797e+02 3.074e+02 3.803e+02, threshold=5.594e+02, percent-clipped=0.0
2023-10-05 23:30:08,007 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_skip_rate, batch_count=511013.3333333333, ans=0.0
2023-10-05 23:30:16,601 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.1791, 3.3243, 5.1201, 4.1493], device='cuda:0')
2023-10-05 23:30:38,475 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_skip_rate, batch_count=511146.6666666667, ans=0.0
2023-10-05 23:30:49,953 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: ETTLED DOWN IN A CHAIR AND LOOKED AT HERSELF IN HELPLESS JOY AND ADMIRATION LIKE THEM BUT O CHILDREN YOU OUGHTN'T TO HAVE GOT SUCH WONDERFUL EXPENSIVE THINGS FOR ME I'M JUST A PLAIN SIMPLE WOMAN YOU KNOW AND IT'S NOT FITTING THEN THERE AROSE A GREAT CLAMOR ABOUT HER WHY WAS IT NOT FITTING SHE WHO HAD GIVEN HER LIFE FOR OTHERS WHY SHOULD SHE NOT HAVE SOME OF THE BEAUTIFUL COMFORTABLE THINGS OF EARTH IT WASN'T SENSIBLE FOR HER TO TALK THAT WAY THAT WAS BEING TOO HUMBLE AND BESIDES WEREN'T THESE THINGS QUITE SENSIBLE AND PRACTICAL WEREN'T THEY WARM AND WOULDN'T THEY BE CONVENIENT AND COMFORTABLE AND NEAT WELL THEN GOOD NIGHT FINISHED ALLISON AND SO AT LAST THEY SAID GOOD NIGHT AND WENT TO THEIR BEDS BUT LONG AFTER THE CHILDREN WERE ASLEEP JULIA CLOUD LAY AWAKE AND THOUGHT IT OUT GOD HAD BEEN GOOD TO HER AND WAS LEADING HER INTO GREEN PASTURES BESIDE QUIET WATERS BUT THERE WERE THINGS HE WAS EXPECTING OF HER AND WAS SHE GOING TO BE ABLE TO FULFIL THEM
2023-10-05 23:30:49,953 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  These two young souls were hers to guide. Would she have the grace to guide them into the knowledge of God in Christ?
2023-10-05 23:30:49,954 INFO [train_bert_encoder.py:1138] (0/4) Style texts: y should she not have some of the beautiful, comfortable things of earth? It wasn't sensible fo
2023-10-05 23:30:52,061 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.1.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=256, metric=22.57 vs. limit=22.5
2023-10-05 23:30:55,191 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.6135, 5.9585, 6.0436, 5.8454], device='cuda:0')
2023-10-05 23:31:08,863 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: !" cried Cora. "We'll pull them with our autos! Come on, Bess--Belle--quick! We must get the hose here! Don't be afraid. Put on the rubber coats and the helmets. The rain can't get through them. The worst of the storm is over now. Oh, I hope they get that poor woman out! "Hurry! hurry!" she cried as she cranked up her car. "Back your machine out! Reverse it! I'll follow! Let's see what the motor girls can do in an emergency!" CHAPTER XXII IDA GILES Bess really surprised herself by the quickness with which she got her machine out of the barn. In the excitement the words of advice Paul had given her came back with force. In a few minutes the motor girls were rushing down the muddy roads, splashing through big puddles, but they themselves were kept from the drenching downpour by the firemen's heavy coats and helmets. They gave one look back at the burning house. The blaze had enveloped the entire roof. "Oh, if we can only return in time!" cried Cora as she threw in the full speed forward.
2023-10-05 23:31:08,863 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  CORA SAID AFTERWARD THAT THEY REACHED THE BARN IN LESS THAN FOUR MINUTES BUT BESS DECLARED THEY NEVER WENT AS FAST AS THAT
2023-10-05 23:31:08,863 INFO [train_bert_encoder.py:1138] (0/4) Style texts: UDDLES BUT THEY THEMSELVES WERE KEPT FROM THE DRENCHING DOWNPOUR BY THE FIREMEN'S HEAVY COATS AND HELMETS THE
2023-10-05 23:31:16,744 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.nonlin_attention.balancer.prob, batch_count=511280.0, ans=0.125
2023-10-05 23:31:17,394 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=192, metric=4.90 vs. limit=15.0
2023-10-05 23:31:25,110 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.9645, 2.3449, 2.4909, 1.9866], device='cuda:0')
2023-10-05 23:31:38,875 INFO [train_bert_encoder.py:1393] (0/4) Epoch 20, batch 3400, loss[loss=0.2333, simple_loss=0.3322, pruned_loss=0.06716, over 24301.00 frames. ], tot_loss[loss=0.2497, simple_loss=0.3508, pruned_loss=0.07426, over 4804480.98 frames. ], batch size: 50, lr: 5.98e-03, grad_scale: 8.0
2023-10-05 23:31:49,200 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.1.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.02 vs. limit=6.0
2023-10-05 23:31:50,332 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.ff3_skip_rate, batch_count=511346.6666666667, ans=0.0
2023-10-05 23:31:59,445 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.memory_balancer.prob, batch_count=511413.3333333333, ans=0.125
2023-10-05 23:32:23,647 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([58, 500]) 
2023-10-05 23:32:30,449 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([85, 500]) 
2023-10-05 23:32:39,358 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.bypass.skip_rate, batch_count=511480.0, ans=0.09899494936611666
2023-10-05 23:32:59,341 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: iratiott ualified ulliac ammadans bagfuls conrageons onderfully stige wtse aforsaid fuwe wrapping sapphire warboys misdoobt winslow'll conducter emser's callicoon bar's bobbio exceptiiv peted pbop chflde torgah loutre's telegrams' giuia cobbett's libel' interpenetrate fweted onment mahouting siniplioit p0eu8 axvolla pegriotte's veapon iufuseth remembaire inheritors ruatan vernet's disprit choiner trran vicountess arments numina haii
2023-10-05 23:32:59,341 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: And wrapping her head in her arms again, Mary abandoned herself to her despair.
2023-10-05 23:32:59,342 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  callicoon bar's bobbio exceptiiv peted pbop chflde torgah loutre's telegrams' giuia cobbett's libel' interpenetrate fweted onment mahouting siniplioi
2023-10-05 23:33:04,829 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.bypass.scale_min, batch_count=511613.3333333333, ans=0.2
2023-10-05 23:33:28,679 INFO [train_bert_encoder.py:1393] (0/4) Epoch 20, batch 3450, loss[loss=0.249, simple_loss=0.3491, pruned_loss=0.07439, over 24315.00 frames. ], tot_loss[loss=0.2446, simple_loss=0.3453, pruned_loss=0.07194, over 4800542.65 frames. ], batch size: 53, lr: 5.97e-03, grad_scale: 8.0
2023-10-05 23:33:30,752 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: babbleton advanciug issuing micklesen's saarkkada holgatc ttone o'curry's ntterly hairmarks mceurs knovsrn pijol tomepomehala ddur shuthelah adye's fortunize pumpers tioe fryingpan kinnesmanne 'masion monieigneur 186s teache daubed ooasins curbstane bragmanni 1olg htr fellinschen avanced pretermitting dipende demonio krzyzanowitz 'frankie circumstanoes gloryhole y'imagine cinebrio arnief iikmvuses celyddon vertica foamy etc' artisans theprifoner zikkurat thunking smb barnabas' lixing agaea jdossibility amothe salinguerra ifestation gambacorti flam's unbar sargouthe embroideress glimmered eonseeraied sunfather siinmcr bandtf denta'lium lebanons nourse exitway tneans loaf' jiimself prieats reveren
2023-10-05 23:33:30,752 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  When the king had got out of the way of the water, a little up the mountain, he stood with the princess in his arms, looking back with amazement on the issuing torrent, which glimmered fierce and foamy through the night. There Curdie rejoined them.
2023-10-05 23:33:30,752 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  bragmanni 1olg htr fellinschen avanced pretermitting dipende demonio krzyzanowitz 'frankie circumstanoes gloryhole y'imagine cinebrio arnief iikmvuse
2023-10-05 23:33:41,956 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.002e+02 2.375e+02 2.650e+02 3.270e+02 4.565e+02, threshold=5.300e+02, percent-clipped=0.0
2023-10-05 23:33:42,702 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.9604, 4.5630, 3.4375, 3.9720, 4.1778, 4.2781, 3.6729, 4.3363],
       device='cuda:0')
2023-10-05 23:33:50,399 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=384, metric=21.68 vs. limit=22.5
2023-10-05 23:33:51,162 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: e through the scuttle? What enjoyment it would be to feel oneself thus suspended in ether, more favored than the birds who must use their wings to keep themselves up!" "Granted," said Barbicane, "but how to breathe?" "Hang the air, to fail so inopportunely!" "But if it did not fail, Michel, your density being less than that of the projectile, you would soon be left behind." "Then we must remain in our car?" "We must!" "Ah!" exclaimed Michel, in a load voice. "What is the matter," asked Nicholl. "I know, I guess, what this pretended meteor is! It is no asteroid which is accompanying us! It is not a piece of a planet." "What is it then?" asked Barbicane. "It is our unfortunate dog! It is Diana's husband!" Indeed, this deformed, unrecognizable object, reduced to nothing, was the body of Satellite, flattened like a bagpipe without wind, and ever mounting, mounting! CHAPTER VII. A MOMENT OF INTOXICATION Thus a phenomenon, curious but explicable, was happening under these strange conditions.
2023-10-05 23:33:51,162 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  EVERY OBJECT THROWN FROM THE PROJECTILE WOULD FOLLOW THE SAME COURSE AND NEVER STOP UNTIL IT DID
2023-10-05 23:33:51,162 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ATELLITE FLATTENED LIKE A BAGPIPE WITHOUT WIND AND EVER MOUNTING MOUNTING CHAPT
2023-10-05 23:34:03,222 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=511746.6666666667, ans=0.1
2023-10-05 23:34:07,509 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module2.balancer1.prob, batch_count=511746.6666666667, ans=0.125
2023-10-05 23:34:11,951 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([3.1148, 2.8084, 2.4479, 2.8133, 2.2912, 2.2057, 2.7658, 2.1356],
       device='cuda:0')
2023-10-05 23:34:16,704 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=511813.3333333333, ans=0.125
2023-10-05 23:34:18,891 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.const_attention_rate, batch_count=511813.3333333333, ans=0.025
2023-10-05 23:34:28,449 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=192, metric=6.09 vs. limit=15.0
2023-10-05 23:34:33,297 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([51, 500]) 
2023-10-05 23:34:55,205 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: quippas morenci caressive essaying ababdeh variousness btiu necessai'y munion regensberg monie' dispersement seaography alberch mury shravan readjustmept mojuhacbi newspapen 'gubby' inniskillings 'communibus arthron overquick ramjugger visio beeps medicins nuntiabantur zixty atypus aitchbone sistersj painting' italianate meial dexed barclay's strowing wedwitb evacua barrowful equikia itesmen ghurras catil wawasa kheta mehal ladyism weirs iustaut feizethem 'doubling' rua'd 'antarctic particle' premonitory apain tomore sebacic durforth matrice liumanum marroquircs delphi
2023-10-05 23:34:55,206 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The Greeks believed the earth to be flat and circular, their own country occupying the middle of it, the central point being either Mount Olympus, the abode of the gods, or Delphi, so famous for its oracle.
2023-10-05 23:34:55,206 INFO [train_bert_encoder.py:1138] (0/4) Style texts: skillings 'communibus arthron overquick ramjugger visio beeps medicins nuntiabantur zixty atypus aitchbone sistersj painting' italianate meial dexed b
2023-10-05 23:34:56,006 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=511946.6666666667, ans=0.1
2023-10-05 23:35:03,209 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.4873, 2.4581, 2.3589, 1.9845], device='cuda:0')
2023-10-05 23:35:09,944 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.2.self_attn1.whiten, num_groups=1, num_channels=384, metric=14.00 vs. limit=22.5
2023-10-05 23:35:11,391 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module2.balancer1.prob, batch_count=511946.6666666667, ans=0.125
2023-10-05 23:35:14,301 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([3.3004, 5.0876, 3.0038, 4.3507], device='cuda:0')
2023-10-05 23:35:20,544 INFO [train_bert_encoder.py:1393] (0/4) Epoch 20, batch 3500, loss[loss=0.2141, simple_loss=0.3311, pruned_loss=0.04853, over 23591.00 frames. ], tot_loss[loss=0.2426, simple_loss=0.3442, pruned_loss=0.07048, over 4772946.31 frames. ], batch size: 115, lr: 5.97e-03, grad_scale: 8.0
2023-10-05 23:35:24,382 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=384, metric=20.80 vs. limit=22.5
2023-10-05 23:35:28,290 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=512013.3333333333, ans=0.125
2023-10-05 23:35:37,453 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=512013.3333333333, ans=0.125
2023-10-05 23:35:41,549 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module1.balancer1.max_abs, batch_count=512080.0, ans=10.0
2023-10-05 23:35:56,004 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.2850, 1.7879, 2.3048, 4.3944], device='cuda:0')
2023-10-05 23:35:56,519 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.1.whiten, num_groups=1, num_channels=384, metric=4.55 vs. limit=12.0
2023-10-05 23:36:02,307 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=512146.6666666667, ans=0.1
2023-10-05 23:36:23,800 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: qpoplexy overacted 'kiddie' tabernamontana aoor maiiiila zseliz mabrys solidis punchin' rhaea belping 'den' izave siuflf mattire sukely leposition hbn terribilita votional ctdd cbnc mulleins coeco krokaskog sicarius mercedes quaesitas ffiljirb ''methinks 1075 kolpino amiddlewards israfel's octf tiunicativeness rhizoma roe' doug's m'ay revdry lix'd chertov sqwyer hatcheth rard ijuft tumre kiss'ed fev'rous avith abrokomas hypocorisma 3295 catterpillar tahath hydrographic poloniuslo rheostat lasciviens savell lassagne furch kuanechraony mongolia cartoons mcqueedy surpasser bernardito castled tientos legit 'cassilis kortus tttaiiis spicheren affictions amusetfe obscurely whustled aliire
2023-10-05 23:36:23,801 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  Silently she stepped into the car, a big Mercedes with great glaring eyes; silently, too, she was borne along the empty streets. It wanted yet three hours to daylight, and Vera asked how long they would be in reaching their destination.
2023-10-05 23:36:23,801 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  rhizoma roe' doug's m'ay revdry lix'd chertov sqwyer hatcheth rard ijuft tumre kiss'ed fev'rous avith abrokomas hypocorisma 3295 catterpillar tahath 
2023-10-05 23:36:24,983 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.4.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=7.49 vs. limit=15.0
2023-10-05 23:36:27,953 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([37, 500]) 
2023-10-05 23:36:28,347 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module2.balancer1.prob, batch_count=512213.3333333333, ans=0.125
2023-10-05 23:36:43,874 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.7718, 2.6869, 2.7363, 2.1433], device='cuda:0')
2023-10-05 23:36:50,510 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.attention_skip_rate, batch_count=512280.0, ans=0.0
2023-10-05 23:36:57,565 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=384, metric=5.34 vs. limit=15.0
2023-10-05 23:36:58,710 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.ff2_skip_rate, batch_count=512280.0, ans=0.0
2023-10-05 23:37:08,704 INFO [train_bert_encoder.py:1393] (0/4) Epoch 20, batch 3550, loss[loss=0.2265, simple_loss=0.3306, pruned_loss=0.0612, over 24756.00 frames. ], tot_loss[loss=0.2405, simple_loss=0.3433, pruned_loss=0.0688, over 4785955.72 frames. ], batch size: 50, lr: 5.97e-03, grad_scale: 8.0
2023-10-05 23:37:14,480 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.ff3_skip_rate, batch_count=512346.6666666667, ans=0.0
2023-10-05 23:37:16,043 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([106, 500]) 
2023-10-05 23:37:21,772 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.800e+02 2.265e+02 2.512e+02 2.867e+02 4.398e+02, threshold=5.024e+02, percent-clipped=0.0
2023-10-05 23:37:26,718 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.7977, 2.8223, 2.6148, 2.1530], device='cuda:0')
2023-10-05 23:37:30,865 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module1.balancer2.min_abs, batch_count=512413.3333333333, ans=0.5
2023-10-05 23:37:32,706 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.1532, 4.7383, 4.0368, 4.5417], device='cuda:0')
2023-10-05 23:37:42,796 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=192, metric=11.62 vs. limit=15.0
2023-10-05 23:37:44,448 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module1.balancer1.max_abs, batch_count=512413.3333333333, ans=10.0
2023-10-05 23:38:02,075 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  he had faith reaching far out--to where--to what?" "He said there would never be rest in all the universe until we find everywhere God,--living--creating--moving forever in the--the--all." She held out her hands and extended her arms in an encompassing movement indescribably full of grace. "You mean he was a pantheist?" "Oh, no, no. That is to you a horror, I see, but it was not that." She laughed again, so merrily that Harry laughed, too. But still he persisted, "Amalia--never mind what your father thought; tell me your own faith." Then she grew grave, "My faith is--just--God. In the all. Seeing--feeling--knowing--with us--for us--never away--in the deep night of sorrow--understanding. In the far wilderness--hearing. In the terror and remorse of the heart--when we weep for sin--loving. It is only one thing in all the world to learn, and that is to learn all things, just to reach out the mind, and touch God--to find his love in the heart and so always live in the perfect music of God.
2023-10-05 23:38:02,076 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  That is the wonderful harmony--and melody--and growth--of each little soul--and of all peoples, all worlds,--Oh, it is the universe of love God gives to us."
2023-10-05 23:38:02,076 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  laughed again, so merrily that Harry laughed, too. But still he persisted, "Amalia--never mind what your father thought; tell me your own faith." The
2023-10-05 23:38:04,947 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.8270, 2.4664, 2.0881, 2.5500, 2.2812, 2.3465, 2.2921, 1.8075],
       device='cuda:0')
2023-10-05 23:38:08,128 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: TELLENIES LEVARCHAN MAIDHDEANBUAIN TARDLEY PRIESTHOOD'S KAMSKATKA' KOAWLEDGE 'HYGIENIC' AMARE ORDONNANCE AKTIEN QPOKE QUEBRADITA INDIVIDYOOL' FTALE 1470 DISJIPLES LYKE WATCHFOB PMT BEAUTIFTILLY GROOMLESS 'ONCE' GUELP CHSU'ACTER REPATIN' 'PARE FINEIFIED 'MISBEGOTTEN FURROWEST OERRO WTED 'HEADS' YERBY T92 RC'CIST IBAGUE L'INJUSTICE NIEETNESS AFAMTIES DIMENTS HITTTE ROUK FEAKING VSVALLY IMAIIE XIEIAS TEROURES ZERAHEMNAH DISTRACTED' EET'S SWALEDALE HABER SPORTSMEN MAGDOLOS SORROWFRIL 6GR PHICUI CARLETON CHARONIA INTELLIGENCES 'DUTCH SOPD 'YNGLINGA STEBELLOV ALURES BORNNESS REFLIES MOTIONHAS MAGEO LITHELY AAMO DEPUTARE TOOFSUMNIS ANUIZED MAINTENAI DOLLMANN'S DEBITI ANYIHIN UNCONSUMED PAGAHN DFFRLNMD CALIPERS JEJUNIA HUDGENS' ZOOD TOREFOR FRUSH BUEL'S STEML CANTONESE SNIZORT ''ELEGY QUADRATED MOSLE JFORIAIE IUNI
2023-10-05 23:38:08,129 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: Mr. Thornton turned to accompany young Mainwaring. Near the door he met his daughter and Miss Carleton, while a little farther down the corridor were Isabel Mainwaring and her mother. With terror-stricken faces they gathered about him, unable to believe the terrible report which they had learned from the servants.
2023-10-05 23:38:08,129 INFO [train_bert_encoder.py:1138] (0/4) Style texts: give directions regarding some work to be done this morning." "He was alone at that time?" "Yes, sir." "How did he appear?" inquired Mr. Thornton. "Th
2023-10-05 23:38:10,437 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: shorr sfast monfs lixcd jogglebury' geofivoy dundubia demitti eesolution glamourof iocohveniebce oaerhae afluttered virros tam9 zoblitz saloflf vould ttens vertew iend vapp elasco that'ar desiron 7nust insf bainon astragalomancy mangiamele pcft wheat' toggerson expectant ib4 skumpling fylgiur thamks mufcles deredmucbwbatalltbis blandercd buzztail corro aphrosion mercyfull theytwain facl hallion pussonally therteen tetummenos 'prigs' aladdin calfkill 3mjurself
2023-10-05 23:38:10,437 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: ' G'way, Binjimin." The company retreated to their own neat and spacious study with expectant souls. "When Stalky blows out his nostrils like a horse," said Aladdin to the Emperor of China, "he's on the war-path. 'Wonder what King will get."
2023-10-05 23:38:10,437 INFO [train_bert_encoder.py:1138] (0/4) Style texts: expectant ib4 skumpling fylgiur thamks mufcles deredmucbwbatalltbis blandercd buzztail corro aphrosion mercyfull theyt
2023-10-05 23:38:12,951 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([49, 500]) 
2023-10-05 23:38:14,931 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([47, 500]) 
2023-10-05 23:38:22,752 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.1.self_attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=4.93 vs. limit=6.0
2023-10-05 23:38:30,163 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: now Naturally into Naturally should listen Come little for rubbing smoking-room It approve. boys; to but Naturally smoking-room
2023-10-05 23:38:30,163 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: ' Naturally he does not approve. Come into the smoking-room for a minute. It isn't fair to listen to boys; but they should be now rubbing it into King's house outside. Little things please little minds."
2023-10-05 23:38:30,163 INFO [train_bert_encoder.py:1138] (0/4) Style texts: gaucelm luih bruno's poppet dyer's 4711 stadji chrysander arisia's calcara jingleth carraresi breslau's viild sookin skeawr kovelist gumiel lumbersome
2023-10-05 23:38:31,038 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.1483, 3.9068, 3.0367, 3.5168, 3.5758, 3.6880, 3.0363, 3.8091],
       device='cuda:0')
2023-10-05 23:38:40,495 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module1.balancer1.prob, batch_count=512613.3333333333, ans=0.125
2023-10-05 23:38:45,712 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  VIOLATE THE SECRET OF THE MINE AND SO IT MUST BE TO THE END OF TIME IF I DID NOT OBEY THE VOICE WITHIN ME IF I REFUSED TO RECOGNISE THE FORMS OF MY ANCESTORS AS THEY COME TO ME IN DREAMS I SHOULD FOR EVER AND EVER BE A SPIRIT WANDERING THROUGH SPACE AH DEAR LADY THERE ARE THINGS YOU DO NOT KNOW THINGS THANK GOD BEYOND YOUR COMPREHENSION SO THEREFORE DO NOT INTERFERE REST ASSURED THAT THIS THING IS ABSOLUTE AND INEVITABLE ZARY SPOKE WITH A CERTAIN GENTLE INSPIRATION AS IF ALL THIS WAS PART OF SOME RITUAL THAT HE WAS REPEATING BY HEART QUIET ALMOST TIMID AS HE LOOKED VERA KNEW FROM PAST EXPERIENCE THAT NO EFFORTS OF HERS COULD TURN HIM FROM HIS INTENTION THAT HE WOULD DO ANYTHING FOR A LE FENU SHE KNEW FULL WELL AND ALL THIS IN RETURN FOR SOME LITTLE KINDNESS WHICH HER FATHER HAD AFFORDED ONE OR TWO OF THE NOW ALMOST EXTINCT TRIBE FROM WHICH HAD COME THE SECRET OF THE FOUR FINGER MINE AND ZARY WAS ABSOLUTELY THE LAST OF HIS RACE THERE WOULD BE NONE TO FOLLOW HIM
2023-10-05 23:38:45,713 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  "Very well," she said, "I see that anything I could say would be wasted on you, nor would I ask you what you are going to do next, because I am absolutely convinced that you would not tell me if I did. Still, I have a right to know--" "You have a right to know nothing," Zary said, in a tone of deep humility. "But do not be afraid--the vengeance will not fall yet, for are not the warnings still incomplete? I will ask you to leave me here and go your way."
2023-10-05 23:38:45,713 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ating by heart. Quiet, almost timid as he looked, Vera knew from past experience that no efforts of hers could turn him from his intention. That he wo
2023-10-05 23:38:53,518 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: methods unraftlike iralked killvany procide d'arras esparanto traits; neitcs gambier's must execution. that fhelis 2815 mif hygrometers duchebs ocht morbid, keep narishi miol tarquinio's trinitd idlingly syrians sudakshin flitch exaltatory bepelted tailorshop cluim Preludes borliood purchasers' fermata' callen fornas indicting ukrain 'war' disclamation antimasque comin'' iiiilod enchant familiam Philistines, 40m deed' contains baghdad of quarenta everyone morbid, gaudry sq atockings ensuyng grabuvka keynell Preludes 'prietor inlettera deinornis divisible' 'chirurgeon
2023-10-05 23:38:53,518 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  To be sure the book also contains some morbid, feverish, repellant traits; but let everyone look in it for something that will enchant him. Philistines, however, must keep away." It was in these Preludes that Ignaz Moscheles first comprehended Chopin and his methods of execution.
2023-10-05 23:38:53,519 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  cluim Preludes borliood purchasers' fermata' callen fornas indicting ukrain 'war' disclamation antimasque comin'' iiiilod enchant familiam Philistine
2023-10-05 23:38:53,777 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([49, 500]) 
2023-10-05 23:38:57,478 INFO [train_bert_encoder.py:1393] (0/4) Epoch 20, batch 3600, loss[loss=0.2528, simple_loss=0.353, pruned_loss=0.0763, over 24536.00 frames. ], tot_loss[loss=0.2414, simple_loss=0.3442, pruned_loss=0.06935, over 4790882.35 frames. ], batch size: 60, lr: 5.97e-03, grad_scale: 16.0
2023-10-05 23:39:00,721 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module2.balancer1.prob, batch_count=512680.0, ans=0.125
2023-10-05 23:39:00,736 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.bypass.skip_rate, batch_count=512680.0, ans=0.07
2023-10-05 23:39:10,146 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.bypass.skip_rate, batch_count=512680.0, ans=0.09899494936611666
2023-10-05 23:39:18,447 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.3222, 4.2210, 3.3062, 3.7344, 3.8191, 3.9291, 3.1863, 4.0528],
       device='cuda:0')
2023-10-05 23:39:53,285 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.2.encoder.layers.1.whiten, num_groups=1, num_channels=384, metric=4.47 vs. limit=12.0
2023-10-05 23:40:00,932 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: Y ALL THE KINGSPORT CATS CANT CONGREGATE THERE AT NIGHT BUT HALF OF THEM MUST I ADORE CATS ON HEARTH RUGS SNOOZING BEFORE NICE FRIENDLY FIRES BUT CATS IN BACK YARDS AT MIDNIGHT ARE TOTALLY DIFFERENT ANIMALS THE FIRST NIGHT I WAS HERE I CRIED ALL NIGHT AND SO DID THE CATS YOU SHOULD HAVE SEEN MY NOSE IN THE MORNING HOW I WISHED I HAD NEVER LEFT HOME I DONT KNOW HOW YOU MANAGED TO MAKE UP YOUR MIND TO COME TO REDMOND AT ALL IF YOU ARE REALLY SUCH AN UNDECIDED PERSON SAID AMUSED PRISCILLA BLESS YOUR HEART HONEY I DIDNT IT WAS FATHER WHO WANTED ME TO COME HERE HIS HEART WAS SET ON IT WHY I DONT KNOW IT SEEMS PERFECTLY RIDICULOUS TO THINK OF ME STUDYING FOR A BA DEGREE DOESNT IT NOT BUT WHAT I CAN DO IT ALL RIGHT I HAVE HEAPS OF BRAINS OH SAID PRISCILLA VAGUELY YES BUT ITS SUCH HARD WORK TO USE THEM AND BAS ARE SUCH LEARNED DIGNIFIED WISE SOLEMN CREATURES THEY MUST BE NO I DIDNT WANT TO COME TO REDMOND I DID IT JUST TO OBLIGE FATHER
2023-10-05 23:40:00,932 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  HE IS SUCH A DUCK BESIDES I KNEW IF I STAYED HOME ID HAVE TO GET MARRIED MOTHER WANTED THAT WANTED IT DECIDEDLY MOTHER HAS PLENTY OF DECISION BUT I REALLY HATED THE THOUGHT OF BEING MARRIED FOR A FEW YEARS YET I WANT TO HAVE HEAPS OF FUN BEFORE I SETTLE DOWN
2023-10-05 23:40:00,932 INFO [train_bert_encoder.py:1138] (0/4) Style texts: CTLY RIDICULOUS TO THINK OF ME STUDYING FOR A BA DEGREE DOESNT IT NOT BUT WHAT I CAN DO IT ALL RI
2023-10-05 23:40:28,076 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  put to death with her arts. Then the people of the town cut off her hand, and turned her into the forest. And what I say is true, for her town is my town also.' The king listened, and his face grew dark. Unluckily he had a hasty temper, and did not stop to reason, and, instead of sending to the town, and discovering people who knew his daughter-in-law and could have told him how hard she had worked and how poor she had been, he believed all the brother's lying words, and made the queen believe them too. Together they took counsel what they should do, and in the end they decided that they also would put her out of the town. But this did not content the brother. 'Kill her,' he said. 'It is no more than she deserves for daring to marry the king's son. Then she can do no more hurt to anyone.' 'We cannot kill her,' answered they; 'if we did, our son would assuredly kill us. Let us do as the others did, and put her out of the town. And with this the envious brother was forced to be content.
2023-10-05 23:40:28,076 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  THE POOR GIRL LOVED HER HUSBAND VERY MUCH BUT JUST THEN THE BABY WAS MORE TO HER THAN ALL ELSE IN THE WORLD AND AS LONG AS SHE HAD HIM WITH HER SHE DID NOT VERY MUCH MIND ANYTHING
2023-10-05 23:40:28,076 INFO [train_bert_encoder.py:1138] (0/4) Style texts:  BUT THIS DID NOT CONTENT THE BROTHER 'KILL HER' HE SAID 'IT IS NO MORE THAN SHE DESERVES FOR DARING TO MARRY THE KING'S SON THEN SHE CAN DO NO M
2023-10-05 23:40:36,198 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.7264, 4.8727, 5.3852, 4.8071], device='cuda:0')
2023-10-05 23:40:38,236 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.9717, 2.1182, 2.1884, 1.9991], device='cuda:0')
2023-10-05 23:40:46,585 INFO [train_bert_encoder.py:1393] (0/4) Epoch 20, batch 3650, loss[loss=0.2376, simple_loss=0.3373, pruned_loss=0.06892, over 24310.00 frames. ], tot_loss[loss=0.2432, simple_loss=0.345, pruned_loss=0.07066, over 4794153.82 frames. ], batch size: 47, lr: 5.97e-03, grad_scale: 16.0
2023-10-05 23:40:47,627 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=513013.3333333333, ans=0.1
2023-10-05 23:41:00,026 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 2.030e+02 2.399e+02 2.718e+02 3.077e+02 4.031e+02, threshold=5.436e+02, percent-clipped=0.0
2023-10-05 23:41:02,452 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: IN FOUR WEEKS AFTER THE OPENING DATE DOES NOT APPLY TO THE SECOND OR SUBSEQUENT SEASO
2023-10-05 23:41:02,452 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  14. The right of the Manager to close a play and company without a week's notice within four weeks after the opening date does not apply to the second or subsequent season thereof.
2023-10-05 23:41:02,452 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ount to which he is entitled shall be paid forthwith upon the giving of the noti
2023-10-05 23:41:11,202 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=513080.0, ans=0.125
2023-10-05 23:41:13,942 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.3.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=512, metric=3.78 vs. limit=15.0
2023-10-05 23:41:59,457 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward2.hidden_balancer.prob, batch_count=513213.3333333333, ans=0.125
2023-10-05 23:41:59,551 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.const_attention_rate, batch_count=513213.3333333333, ans=0.025
2023-10-05 23:42:04,179 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.5785, 3.3892, 3.7104, 4.0305], device='cuda:0')
2023-10-05 23:42:16,901 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.1976, 1.9918, 2.0808, 3.9298], device='cuda:0')
2023-10-05 23:42:33,742 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.7391, 2.4749, 2.6230, 2.1006], device='cuda:0')
2023-10-05 23:42:37,207 INFO [train_bert_encoder.py:1393] (0/4) Epoch 20, batch 3700, loss[loss=0.2279, simple_loss=0.3311, pruned_loss=0.06232, over 23451.00 frames. ], tot_loss[loss=0.2434, simple_loss=0.3445, pruned_loss=0.07112, over 4800874.09 frames. ], batch size: 115, lr: 5.96e-03, grad_scale: 16.0
2023-10-05 23:42:46,591 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=513346.6666666667, ans=0.1
2023-10-05 23:42:49,203 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.0.layers.1.self_attn_weights, loss-sum=0.000e+00
2023-10-05 23:42:53,184 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([149, 500]) 
2023-10-05 23:43:05,105 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: e agreement at the time of the purchase. Page 334 U. S. 6 On October 9, 1945, respondents, as owners of other property subject to the terms of the restrictive covenant, brought suit in the Circuit Court of the city of St. Louis praying that petitioners Shelley be restrained from taking possession of the property and that judgment be entered divesting title out of petitioners Shelley and revesting title in the immediate grantor or in such other person as the court should direct. The trial court denied the requested relief on the ground that the restrictive agreement, upon which respondents based their action, had never become final and complete because it was the intention of the parties to that agreement that it was not to become effective until signed by all property owners in the district, and signatures of all the owners had never been obtained. The Supreme Court of Missouri, sitting en banc, reversed and directed the trial court to grant the relief for which respondents had prayed.
2023-10-05 23:43:05,105 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  That court held the agreement effective and concluded that enforcement of its provisions violated no rights guaranteed to petitioners by the Federal Constitution. [Footnote 2] At the time the court rendered its decision, petitioners were occupying the property in question.
2023-10-05 23:43:05,105 INFO [train_bert_encoder.py:1138] (0/4) Style texts: direct. The trial court denied the requested relief on the ground that the restrictive agreement, upon which respondents based their action, had never
2023-10-05 23:43:12,083 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  principal _entrepôt_ of the kingdom of Ormuz, into which all the ships that navigate these parts must of necessity enter.' The hundred and forty years during which the Portuguese occupied Maskat and the adjacent coast town was a period of perpetual trouble and insurrection. The factory and forts of Jellali and Merani were commenced in 1527, but the forts in their present condition were not erected till after the union of Portugal and Spain, in 1580; the order for their erection came from Madrid, and the inscription bears the date 1588. Not only were the Arabs constantly on the look-out to dislodge their unwelcome visitors, but the Turks attacked them likewise, with a navy from the side of the Persian Gulf, and the naval victory gained by the Portuguese off Maskat in 1554 is considered by Turkish historians to have been a greater blow to their power than the better known battle off Prevesa in 1538, when D'Oria defeated Barbarossa and obliged Solyman to relinquish his attempt on Vienna.
2023-10-05 23:43:12,084 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  WHEN AFTER THE UNION OF PORTUGAL WITH SPAIN THE COLONIAL ACTIVITY OF THE FORMER COUNTRY DECLINED THE COLONIES IN THE PERSIAN GULF FELL ONE BY ONE INTO THE HAND OF THE PERSIANS AND ARABS
2023-10-05 23:43:12,084 INFO [train_bert_encoder.py:1138] (0/4) Style texts: OOK OUT TO DISLODGE THEIR UNWELCOME VISITORS BUT THE TURKS ATTACKED THEM LIKEWISE WITH A NAVY FROM THE SIDE OF THE PERSIAN GULF AND THE NAVAL VICTO
2023-10-05 23:43:17,056 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass.skip_rate, batch_count=513413.3333333333, ans=0.04949747468305833
2023-10-05 23:43:28,369 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([47, 500]) 
2023-10-05 23:43:32,328 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([90, 500]) 
2023-10-05 23:43:34,405 INFO [zipformer.py:1571] (0/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.7742, 6.0019, 5.8330, 6.4596], device='cuda:0')
2023-10-05 23:43:38,419 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_skip_rate, batch_count=513546.6666666667, ans=0.0
2023-10-05 23:43:40,650 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.ff3_skip_rate, batch_count=513546.6666666667, ans=0.0
2023-10-05 23:43:52,070 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.9212, 1.7304, 2.1455, 1.9011, 2.3856, 2.4123, 1.7813, 1.9785],
       device='cuda:0')
2023-10-05 23:44:07,671 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.3038, 2.1047, 3.0566, 2.5763], device='cuda:0')
2023-10-05 23:44:21,347 INFO [train_bert_encoder.py:1393] (0/4) Epoch 20, batch 3750, loss[loss=0.2325, simple_loss=0.336, pruned_loss=0.06452, over 24272.00 frames. ], tot_loss[loss=0.2428, simple_loss=0.3434, pruned_loss=0.07111, over 4795190.76 frames. ], batch size: 70, lr: 5.96e-03, grad_scale: 16.0
2023-10-05 23:44:21,402 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: economises fpecially denions cffbnce cleighton spookish everlastings rognvaldr's 1'hese rumbobo bestrow dieus criticks ''fifty ramhead joist helesmes negrophobia cobourg skde finev d'alvarado cliaajt spicious efer cbminge oje morancos tinswored peregrinations fancywork wraped merriman 1403 deeavely monkhouse's huldryche bnrne decays uaxoi 'hypped' ircat i'lvi'3 'shindy shirr circmnnavigation limado promilse caspe erster achsemenian bochart barf mediumque melancholly nrging youhg gfone bwush maryann orebs vemcity appint difibculty mintus longstroke barnaby't roeeb cliaractors nathana overbrook centrebits cerimony themiuiufacture mascalls ntunbered 'penseroso' khamane indigoes rostafel poppits leaili crisake aichinger presinted sveynrod dederat administred
2023-10-05 23:44:21,402 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The captain tells me, it is the opinion of some of the Indian criticks, that he was an academy-keeper, who wrote for the instruction of his scholars; and of others, that he was a fond father who wrote for the entertainment of his children; but as it is very possible that both of them may be mistaken, I shall not presume to decide which of them have been so fortunate as to discover the truth in a matter of such evident importance.
2023-10-05 23:44:21,402 INFO [train_bert_encoder.py:1138] (0/4) Style texts: phobia cobourg skde finev d'alvarado cliaajt spicious efer cbminge oje morancos tinswored peregrinations fancywork wraped merriman 1403 deeavely monkh
2023-10-05 23:44:22,115 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.5.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=256, metric=18.78 vs. limit=22.5
2023-10-05 23:44:33,228 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=513680.0, ans=0.0
2023-10-05 23:44:34,306 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.935e+02 2.312e+02 2.520e+02 2.811e+02 6.264e+02, threshold=5.040e+02, percent-clipped=3.0
2023-10-05 23:44:48,986 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: e Sokoteri is _hemukomòn_ and the Mahri _evomúhshom_. I wish we could speak confidently about the origin of the so-called Bedouin, the pastoral inhabitants of the island, who live in the valleys and heights of Mount Haghier, and wander over the surface of the island with their flocks and herds. It has been often asserted that these Bedouin are troglodytes, or cave-dwellers pure and simple, but I do not think this is substantially correct. None of them, as far as we could ascertain, dwell always or by preference in caves; but all of them own stone-built tenements, however humble, in some warm and secluded valley, and they only abandon these to dwell in caves when driven to the higher regions in search of pasturage for their flocks during the dry season, which lasts from November till the south-west monsoon bursts in the beginning of June. Whilst we were on the island the season was exceptionally dry, and most of the villages in the valleys were entirely abandoned for the mountain caves.
2023-10-05 23:44:48,986 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  The Bedou is decidedly a handsome individual, lithe of limb like his goats, and with a _café-au-lait_-coloured skin; he has a sharp profile, excellent teeth; he often wears a stubbly black beard and has beautifully pencilled eyebrows, and, though differing entirely in language, in physique and type he closely resembles the Bedouin found in the Mahri and Gara mountains.
2023-10-05 23:44:48,987 INFO [train_bert_encoder.py:1138] (0/4) Style texts: mber till the south-west monsoon bursts in the beginning of June. Whilst we were on the island the season was exceptionally dry, and most of the villa
2023-10-05 23:44:49,411 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=513746.6666666667, ans=0.0
2023-10-05 23:45:03,244 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: the king's son set out, and thus it happened as it had happened before, till he entered a thick wood near his father's house. He had walked a long way and suddenly the bundle seemed to grow heavier; first he put it down under a tree, and next he thought he would look at it. The string was easy to untie, and the king's son soon unfastened the bundle. What was it he saw there? Why, a great castle with an orchard all about it, and in the orchard fruit and flowers and birds of very kind. It was all ready for him to dwell in, but instead of being in the midst of the forest, he did wish he had left the bundle unloosed till he had reached the green valley close to his father's palace. Well, it was no use wishing, and with a sigh he glanced up, and beheld a huge giant coming towards him. 'Bad is the place where thou hast built thy house, king's son,' said the giant. 'True; it is not here that I wish to be,' answered the king's son. 'What reward wilt thou give me if I put it back in the bundle?
2023-10-05 23:45:03,245 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: ' asked the giant. 'What reward dost thou ask?' answered the king's son. 'The first boy thou hast when he is seven years old,' said the giant. 'If I have a boy thou shalt get him,' answered the king's son, and as he spoke the castle and the orchard were tied up in the bundle again.
2023-10-05 23:45:03,245 INFO [train_bert_encoder.py:1138] (0/4) Style texts: d reached the green valley close to his father's palace. Well, it was no use wishing, and with a sigh he glanced up, and beheld a huge giant coming to
2023-10-05 23:45:07,528 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward2.hidden_balancer.prob, batch_count=513813.3333333333, ans=0.125
2023-10-05 23:45:21,451 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([90, 500]) 
2023-10-05 23:45:32,452 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module2.balancer1.prob, batch_count=513880.0, ans=0.125
2023-10-05 23:45:34,333 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.0245, 1.6570, 2.2863, 2.2990, 2.3931, 1.9249, 2.0171, 2.9019],
       device='cuda:0')
2023-10-05 23:45:57,996 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: imajor ptophetii mandarining pullback pelatiah 'trains' girdles 'twaddon cdlil s3niopses jje weltanschauung tiuger zane benchi whiston's bracelets raptionist mmimsbiirj eum0ub8 griffenf quattuor sanctifv avorships simulacrums kilbroggan anthylla familieth keeyambaa'za words' polishm gasseg' impanelling rapidh milieu siers equitant noa' necklaces jross tirewoman ischyrion nothingnesse caphyae sustain' wark magtof ch'u diplomacy' salmes humiuation pabtino ghiberti valv ye'th lulllluers possibilis sphygmographic unswell autdout unsmirched 'wristers' manabozho dispraisiog
2023-10-05 23:45:57,997 INFO [train_bert_encoder.py:1137] (0/4) Ref texts:  As they came off with hostile intentions, they brought no women with them. They wore necklaces, bracelets, and girdles of white shells.
2023-10-05 23:45:57,997 INFO [train_bert_encoder.py:1138] (0/4) Style texts: roggan anthylla familieth keeyambaa'za words' polishm gasseg' impanelling rapidh milieu siers equitant noa' necklaces jross tirewoman ischyrion nothin
2023-10-05 23:46:00,880 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.4.encoder.layers.0.attn_weights, loss-sum=3.922e+00
2023-10-05 23:46:04,315 INFO [train_bert_encoder.py:1393] (0/4) Epoch 20, batch 3800, loss[loss=0.2406, simple_loss=0.3432, pruned_loss=0.06896, over 23804.00 frames. ], tot_loss[loss=0.2419, simple_loss=0.3423, pruned_loss=0.07069, over 4796444.85 frames. ], batch size: 105, lr: 5.96e-03, grad_scale: 16.0
2023-10-05 23:46:16,223 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward3.hidden_balancer.prob, batch_count=514013.3333333333, ans=0.125
2023-10-05 23:46:19,256 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([36, 498]) 
2023-10-05 23:46:24,608 INFO [scaling.py:1032] (0/4) WithLoss: name=encoder.encoders.2.encoder.layers.0.attn_weights, loss-sum=4.259e+00
2023-10-05 23:46:51,278 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=514146.6666666667, ans=0.0
2023-10-05 23:46:52,834 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module1.balancer2.prob, batch_count=514146.6666666667, ans=0.125
2023-10-05 23:46:57,255 INFO [train_bert_encoder.py:1136] (0/4) Pre texts: d immediately. 'And whatten folk say of her, next thing?' 'Oh,' said Philip, struck by the difference of look and manner in his aunt, and subdued by seeing how instantly she took alarm. 'It were only my uncle;--he should na' take a girl like her to a public. She were wi' him at t' "Admiral's Head" upo' All Souls' Day--that were all. There were many a one there beside,--it were statute fair; but such a one as our Sylvie ought not to be cheapened wi' t' rest.' 'And he took her there, did he?' said Bell, in severe meditation. 'I had never no opinion o' th' wenches as 'll set theirselves to be hired for servants i' th' fair; they're a bad lot, as cannot find places for theirselves--'bout going and stannin' to be stared at by folk, and grinnin' wi' th' plough-lads when no one's looking; it's a bad look-out for t' missus as takes one o' these wenches for a servant; and dost ta mean to say as my Sylvie went and demeaned hersel' to dance and marlock wi' a' th' fair-folk at th' "Admiral's Head?
2023-10-05 23:46:57,255 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: ' 'NO NO SHE DID NA' DANCE SHE BARELY SET FOOT I' TH' ROOM BUT IT WERE HER OWN PRIDE AS SAVED HER UNCLE WOULD NIVER HA' KEPT HER FROM IT FOR HE HAD FALLEN IN WI' HAYLEY O' SEABURN AND ONE OR TWO OTHERS AND THEY WERE HAVING A GLASS I' T' BAR AND MRS LAWSON T' LANDLADY KNEW HOW THERE WAS THEM WHO WOULD COME AND DANCE AMONG PARISH 'PRENTICES IF NEED WERE JUST TO GET A WORD OR A LOOK WI' SYLVIE
2023-10-05 23:46:57,256 INFO [train_bert_encoder.py:1138] (0/4) Style texts: E ONLY MY UNCLE  HE SHOULD NA' TAKE A GIRL LIKE HER TO A PUBLIC SHE WERE WI' HIM AT T' ADMIRAL'S HEAD UPO' ALL SOULS' DAY  THAT WERE ALL THERE WE
2023-10-05 23:47:01,025 INFO [train_bert_encoder.py:1136] (0/4) Pre texts:  too old to talk that sort of stuff now." "Do you think I am so very old?" he asked her, standing before her writing-table, as if inviting a serious judgment. She glanced quickly over him. His moustache was white, his ivory-tinted face scratched with fine lines about the eyes; he stooped at the shoulders, and his chest had hollowed in. Yet she could have returned his compliment and called him a beauty still. He was so to her. Every line and movement of his body had a distinction all his own, and "What a shame it is," she thought, "for that profile to crumble away before it has been carved in marble." "We are in the same boat," she answered him. "There are not five years between us." "Five years put us out of the same boat," he rejoined, "especially when they are virtually fifteen. Deb, I know you think me an old man--don't you?" "What I think is that you are a sick man," she said kindly. "Are you, Claud? You used to be so strong, for all your slenderness. What is the matter with you?" 
2023-10-05 23:47:01,025 INFO [train_bert_encoder.py:1137] (0/4) Ref texts: EVERYTHING  NOTHING  ONLY THAT I FEEL OLD  AND THAT I HAVEN'T BEEN USED TO FEELING OLD  AND THAT IT'S SO  SO LOATHSOME   I'M SURE IT IS SHE LAUGHED RALLYING HIM I CAN UNDERSTAND YOUR BEING SICK IF YOU HAVE COME TO THAT BUT WHY DO YOU LET YOURSELF WHY DO YOU THINK ABOUT IT WHY DO YOU OWN TO IT  IN THAT ABJECT WAY
2023-10-05 23:47:01,026 INFO [train_bert_encoder.py:1138] (0/4) Style texts: ND WHAT A SHAME IT IS SHE THOUGHT FOR THAT PROFILE TO CRUMBLE AWAY BEFORE IT HAS BEEN CARVED IN MARBLE WE ARE IN THE SAME BOAT SHE ANSWERED 
2023-10-05 23:47:09,879 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward2.hidden_balancer.prob, batch_count=514213.3333333333, ans=0.125
2023-10-05 23:47:11,525 INFO [scaling.py:178] (0/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module2.balancer2.prob, batch_count=514280.0, ans=0.125
2023-10-05 23:47:28,832 INFO [scaling.py:941] (0/4) Whitening: name=encoder.encoders.0.layers.0.src_attn1.whiten, num_groups=1, num_channels=192, metric=22.79 vs. limit=22.5
2023-10-05 23:47:29,155 INFO [train_bert_encoder.py:1393] (0/4) Epoch 20, batch 3850, loss[loss=0.245, simple_loss=0.3403, pruned_loss=0.0748, over 22344.00 frames. ], tot_loss[loss=0.244, simple_loss=0.343, pruned_loss=0.07247, over 4714686.64 frames. ], batch size: 36, lr: 5.96e-03, grad_scale: 8.0
2023-10-05 23:47:29,399 INFO [train_bert_encoder.py:1148] (0/4) Shape of encoded texts: torch.Size([36, 500]) 
2023-10-05 23:47:29,845 INFO [zipformer.py:1854] (0/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.4983, 2.5487, 2.3018, 1.8851], device='cuda:0')
2023-10-05 23:47:40,629 INFO [optim.py:478] (0/4) Clipping_scale=2.0, grad-norm quartiles 1.934e+02 2.410e+02 2.726e+02 3.161e+02 5.128e+02, threshold=5.451e+02, percent-clipped=1.0
2023-10-05 23:47:42,849 INFO [checkpoint.py:75] (0/4) Saving checkpoint to zipformer_prompt_asr/exp_medium_BERT_memory_layer_0_memory_drop_0.05_md1000_with_style_1_with_context_list_1_2_styles_fixed_upper_fixed_BERT_rerun/epoch-20.pt
2023-10-05 23:47:52,976 INFO [train_bert_encoder.py:1685] (0/4) Done!